Difference between revisions of "Changing to UTF-8"

From TNG_Wiki
Jump to navigation Jump to search
Line 101: Line 101:
 
If you have a cust_text.php file that you need to convert to UTF-8, then the steps below will help you do that on Macintosh and Windows computers
 
If you have a cust_text.php file that you need to convert to UTF-8, then the steps below will help you do that on Macintosh and Windows computers
  
== on Macintosh system ==
+
= on Macintosh system =
  
 
5.3 - Here's what works on Macintosh. You need [[TextWrangler]] - a free text editor. If you don't already have TextWrangler, or its big brother BBEdit, you can download the free TextWrangler from
 
5.3 - Here's what works on Macintosh. You need [[TextWrangler]] - a free text editor. If you don't already have TextWrangler, or its big brother BBEdit, you can download the free TextWrangler from

Revision as of 10:55, 1 February 2009

Changing your TNG site to UTF-8

Notes on changing a TNG Site and database to UTF-8 provided by User:TheKiwi


Create a copy of your database

1 - Use phpMyAdmin to make a copy of your database (always make a copy in case something goes wrong!!!!!!!)

1.1 - In the databases list on the left click on the Database to select it.

1.2 - On the right side click on the Operations Tab. In the section called "Copy Database to" enter a name for your new database, and make sure that "Structure and Database" and "CREATE DATABASE before copying" are both checked, then click "Go".

When done this will have created a copy of your database.

Change database to UTF-8

2 - You need to run a script to change the structure of your database to UTF-8

2.1 - download the script from

http://www.phoca.cz/documents/38-tools/154-how-to-change-collation-in-database

and put it into your site's folder, then load the page to your site.

http://URLToYourSite/tool_phoca_changing_collation/

(NOTE - you need to replace "URLToYourSite" above with the actual URL to your site.)

2.2 - Fill out the 5 boxes with the values needed for your site including choosing a collation of the form

utf8_xxxxxx_ci

xxxxxx is whatever you choose to use as your utf8 collation - eg swedish, general, unicode etc. The collation affects how characters are sorted, eg does ø sort with o or come at the end of the alphabet? Does å sort with a or come at the end of the alphabet? Does ß sort with s or with ss? etc. This page

http://dev.mysql.com/doc/refman/5.0/en/charset-unicode-sets.html

has some information about how some characters are handled in different collations. Choose the collation based on the principal language that your site is about.

Once you've decided on what collation you want to use and entered it into the 5th box, click the Submit button. This will convert the database, tables and columns to the collation you've chosen. The progress will be shown as it goes along. The last tables altered are tng_xnotes, so if you don't see these as the last items listed in the output the script didn't complete.

2.3 - when this has completed (on a large database it can take some time) a message appears at the bottom of the page with a link back to the Home Page. Until this link appears the script hasn't completed.

2.4 - When the script has completed, you should delete the folder "tool_phoca_changing_collation" from your website.

TNG 6 Mod to globallib.php

3 - edit the tng_db_connect function in globallib.php to get the queries performed in UTF-8. Otherwise even though the data is now stored in the database as UTF-8 it's not being retrieved properly by the queries

NOTE THIS STEP ISN'T REQUIRED FOR TNG 7

3.1 - Change line 52 of globallib.php to be

	global $textpart, $session_charset;

3.2 - Add these two lines

	if ($session_charset == 'UTF-8')<br>
	 	@mysql_query("SET NAMES 'utf8'");


after line 53 so that the whole function becomes


function tng_db_connect($dbhost,$dbname,$dbusername,$dbpassword) {<br>
	global $textpart, $session_charset;<br>
	$link = @mysql_connect($dbhost, $dbusername, $dbpassword);<br>
	if ($session_charset == 'UTF-8')<br>
	 	@mysql_query("SET NAMES 'utf8'");<br>
	if( $link && mysql_select_db($dbname))<br>
		return $link;<br>
	else if( $textpart != "setup" && $textpart != "index" ) {<br>
		echo "Error: TNG is not communicating with your database. Please check your database settings and try again.";<br>
		exit;<br>
	}<br>
	return( FALSE );<br>
}


Change TNG settings

4 Change the settings that TNG is using for Character Set.

4.1 - In the TNG Admin ------> Setup ------> General Settings ------> Language change the Character Set to "UTF-8" (without the "" marks) and save the changes.

4.2 - Do the same in TNG Admin ------> Languages ------> each language you support to change them to UTF-8


Make sure that you are using UTF-8 Encoded language files

5.1 - You can download UTF-8 encoded files in all of the languages that TNG supports from the TNG Downloads page - the same page you download your TNG software and TNG updates from. There is a download for each language supported and that download includes the files text.php, admintext.php, alltext.php and an empty cust_text.php file.

5.2 - If you have made changes to any of the language files for your own purposes, you should move all of those changes to the file cust_text.php for each language, so that you can use "standard" language files as supplied by TNG, with the changes being made only in cust_text.php so that you don't lose your changes in any future TNG updates. You should use the supplied empty cust_text.php that came with your UTF-8 encoded language files from 5.1 above.

If you have a cust_text.php file that you need to convert to UTF-8, then the steps below will help you do that on Macintosh and Windows computers

on Macintosh system

5.3 - Here's what works on Macintosh. You need TextWrangler - a free text editor. If you don't already have TextWrangler, or its big brother BBEdit, you can download the free TextWrangler from

http://www.barebones.com/products/textwrangler/

5.3.1 - For each language folder open the file cust_text.php in TextWrangler

5.3.2 - After the file is open, go to the File menu and choose "Reopen using Encoding ------> Western (Windows Latin 1).

5.3.3 - At the bottom of the TextWrangler window is a pop up menu should now say "Western (Windows Latin 1)". Click on this and choose "Unicode™ (UTF-8, no BOM)"

5.3.4 - Save the file.

on Windows system

5.4 - need instructions here on how to do this on Windows. It needs to use a text editor that can handle saving files in UTF-8 format WITHOUT writing the BOM (Byte Order Mark) to the file, which causes odd characters  to appear on TNG pages, and your Admin Menu may display a blank page because there are characters before the <?php which should be first in the file.

5.4.1 - Notepad++ version 5.1.2 will allow converting the $text variable files to UTF-8 without a BOM.

That's it. Your site should now be running in UTF-8.

Related links