Difference between revisions of "Changing to UTF-8"

From TNG_Wiki
Jump to navigation Jump to search
(→‎Change database to UTF-8: document change coding mod)
(move TNGmod template to top so it is obvious where the download link is and added to Mods for TNG v12)
(14 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
{{Languages|Changing to UTF-8}}
 
{{Languages|Changing to UTF-8}}
 
{|align=right
 
|__TOC__
 
|}
 
 
Notes on changing a TNG Site and database to UTF-8 provided by [[User:TheKiwi]]
 
 
 
== Create a copy of your database ==
 
1 - Use phpMyAdmin to make a copy of your database (always make a copy in case something goes wrong!!!!!!!)
 
 
1.1 - In the databases list on the left click on the Database to select it.
 
 
1.2 - On the right side click on the Operations Tab. In the section called "Copy Database to" enter a name for your new database, and make sure that "Structure and Database" and "CREATE DATABASE before copying" are both checked, then click "Go".
 
 
When done this will have created a copy of your database.
 
 
== Change database to UTF-8 ==
 
 
{{TNGmod
 
{{TNGmod
 
| mod_name        = Change coding
 
| mod_name        = Change coding
 
| mod_summary    = Allows a simple change of character set and collation sequence for the TNG database
 
| mod_summary    = Allows a simple change of character set and collation sequence for the TNG database
 
| mod_validation  =  
 
| mod_validation  =  
| download_link  = [http://mossfamilytree.info/download.php?mod=changecoding&version=11.0.0.1 11.0.0.1]
+
| mod_last_update = 23 Mar 2018
 +
| download_link  = [http://mossfamilytree.info/download.php?mod=changecoding&version=11.1.0.1 11.1.0.1]
 
| download_stats  = [http://mossfamilytree.info/downloadstats.php?mod=changecoding show]
 
| download_stats  = [http://mossfamilytree.info/downloadstats.php?mod=changecoding show]
 
| mod_author      = [[user:Chris Moss|Chris Moss]]
 
| mod_author      = [[user:Chris Moss|Chris Moss]]
Line 28: Line 11:
 
| mod_support    = [http://mossfamilytree.info/suggest.php contact author]
 
| mod_support    = [http://mossfamilytree.info/suggest.php contact author]
 
| mod_contact    = [http://mossfamilytree.info/suggest.php contact author]
 
| mod_contact    = [http://mossfamilytree.info/suggest.php contact author]
| mod_version    = 11.0.0.1
+
| mod_version    = 11.1.0.1
 
| min_TNG_ver    =  
 
| min_TNG_ver    =  
 
| max_TNG_ver    = 11.1.2
 
| max_TNG_ver    = 11.1.2
Line 35: Line 18:
 
| notes          = only available with English instructions
 
| notes          = only available with English instructions
 
}}
 
}}
You need to run a script to change the structure of your database to UTF-8. The previously recommended script has some problems so a new mod has been created which is much simpler to use and works in a flexible way.
+
{| style="margin-right:0.5 em;" align="right"
 +
| __TOC__
 +
|}
 +
Notes on changing a TNG Site and database to UTF-8 provided by [[User:TheKiwi]]
  
Download the mod in the normal way, unzip, place it in your mods directory and click the install button.
+
== Create a copy or backup of your database ==
 +
You can use phpMyAdmin to make a copy of your database (always make a copy in case something goes wrong!!!!!!!)
 +
# In the databases list on the left click on the Database to select it.
 +
#  On the right side click on the Operations Tab. In the section called "Copy Database to" enter a name for your new database, and make sure that "Structure and Database" and "CREATE DATABASE before copying" are both checked, then click "Go".
  
[[File:change_to_utf8.png|thumb|right|After running the change coding mod]]
+
When done this will have created a copy of your database.  
When it is installed, click on the green "Installed" line and then hit the "Change database" button below. The conversion will take a number of seconds and when it is complete a summary will appear similat to that shown at the right. Note in this case that the default language is English but the language actually in use when the conversion was done is German. The "utf8_general_ci" collation is suitable for most languages, but if you want a different one, then you can edit the options of the mod and put another collation, as long as it's consistent with the character set chosen.
 
  
If for any reason you want to change back to one of the other two supported character sets, ISO-8859-1 and ISO-8859-2, then choose the Edit options button on the mod and put one of these in the first box. In this case you will need to change the collation to latin1_swedish_ci or similar (there is no 'general' for the latin languages, the MySQL variant of ISO/8859).
+
Alternatively you can back up your database using any of the methods described in [[Database - Backup]].
  
The mod alters all the appropriate settings for the different language folders, including cookies and session variables, so you shouldn't need to do anything else. Once you have changed the database it's ok to uninstall the mod as it shouldn't be needed again.
+
== Change database to UTF-8 ==
  
== Change TNG settings ==
+
You need to run a script to change the structure of your database to UTF-8. The previously recommended script has some problems so a new mod has been created which is much simpler to use and works in a flexible way.
3 Change the settings that TNG is using for Character Set.
 
  
3.1 - In the TNG Admin ------> Setup ------> General Settings ------> Language change the Character Set to "UTF-8" (without the "" marks) and save the changes.
+
Download the mod in the normal way, unzip, place it in your mods directory and click the install button.
  
3.2 - Do the same in TNG Admin ------> Languages ------> each language you support to change them to UTF-8
+
[[File:change_to_utf8.png|thumb|left|380px|After running the change coding mod]]
 +
When it is installed, click on the green "Installed" line and then hit the "Change database" button below. The conversion will take a number of seconds and when it is complete a summary will appear similat to that shown at the left. Note in this case that the default language is English but the language actually in use when the conversion was done is German. The "utf8_general_ci" collation is suitable for most languages, but if you want a different one, then you can edit the options of the mod and put another collation, as long as it's consistent with the character set chosen.
  
== Make sure that you are using UTF-8 Encoded language files ==
+
If for any reason you want to change back to one of the other two supported character sets, ISO-8859-1 and ISO-8859-2, then choose the Edit options button on the mod and put one of these in the first box. In this case you will need to change the collation to latin1_swedish_ci or similar (there is no 'general' for the latin languages, the MySQL variant of ISO/8859).
4.1 - You can download UTF-8 encoded files in all of the languages that TNG supports from the TNG Downloads page - the same page you download your TNG software and TNG updates from. There is a download for each language supported and that download includes the files text.php, admintext.php, alltext.php and an empty cust_text.php file.
 
 
 
4.2 - If you have made changes to any of the language files for your own purposes, you should move all of those changes to the file cust_text.php for each language, so that you can use "standard" language files as supplied by TNG, with the changes being made only in cust_text.php so that you don't lose your changes in any future TNG updates. You should use the supplied empty cust_text.php that came with your UTF-8 encoded language files from 4.1 above.
 
 
 
:If you have a cust_text.php file that you need to convert to UTF-8, then the steps below will help you do that on Macintosh and Windows computers
 
  
== on Macintosh system ==
+
The mod alters all the appropriate settings for the different language folders, including cookies and session variables, so you shouldn't need to do anything else. Once you have changed the database it's ok to uninstall the mod as it shouldn't be needed again. But's ok to run the mod again if, for instance, an incompatible table has been installed into your database or the settings have been disturbed by an update.
  
4.3 - Here's what works on Macintosh. You need [[TextWrangler]] - a free text editor. If you don't already have TextWrangler, or its big brother BBEdit, you can download the free TextWrangler from
+
But '''don't''' try and import an ISO Gedcom file into a UTF-8 database (or vice versa). There will be no checking or conversion and you will end up with a mess if it contains accented characters.
  
:http://www.barebones.com/products/textwrangler/
+
== Make sure that you have all the necessary UTF-8 Encoded language files ==
 +
In the languages directory, there are pairs of folders for each language, e.g. French and French-UTF8, which have all the strings used in the two different character sets. If you have deleted the -UTF8 directories to save space, then these should be restored before running the change script above. Most mods will also include changes to both of these sets, normally changing the cust_text.php file.
  
4.3.1 - For each language folder open the file cust_text.php in TextWrangler
+
However if you made changes to cust_text.php file yourself then you need to make sure these changes are also in the corresponding -UTF8 version of cust_text.php. The steps below will help you do that on Macintosh and Windows computers
  
4.3.2 - After the file is open, go to the File menu and choose "Reopen using Encoding ------> Western (Windows Latin 1).
+
=== On Macintosh system ===
  
4.3.3 - At the bottom of the TextWrangler window is a pop up menu should now say "Western (Windows Latin 1)". Click on this and choose "Unicode™ (UTF-8, no BOM)"  Note: Starting with TextWrangler Ver 3.5, the UI terminology has been changed, and "UTF-8, no BOM" is no longer listed as an option. Per the Release Notes, "Unicode (UTF-8, no BOM)" has been renamed to "Unicode (UTF-8)".
+
Here's what works on Macintosh. You need BBEdit - a free text editor. If you don't already have BBEdit, you can download it from [https://www.barebones.com/products/bbedit/download.html https://www.barebones.com/products/bbedit/download.html]
  
4.3.4 - Save the file.
+
# For each language folder open the file cust_text.php in BBEdit
 +
# After the file is open, go to the File menu and choose "Reopen using Encoding ------> Western (Windows Latin 1).
 +
# At the bottom of the BBEdit window is a pop up menu should now say "Western (Windows Latin 1)". Click on this and choose "Unicode (UTF-8)".
 +
# Save the file and place in the appropriate directory.
  
== on Windows system ==
+
=== On Windows system ===
  
4.4 - need instructions here on how to do this on Windows. It needs to use a text editor that can handle saving files in UTF-8 format WITHOUT writing the BOM (Byte Order Mark) to the file, which causes odd characters <font style="font-size:200%"></font> to appear on TNG pages, and your Admin Menu may display a blank page because there are characters before the <?php which should be first in the file.
+
You need to use a text editor that can handle saving files in UTF-8 format WITHOUT writing the BOM (Byte Order Mark) to the file such as Notepad++ ([http://notepad-plus-plus.org/download/ Current Version]).
  
4.4.1 - Notepad++ ([http://notepad-plus-plus.org/download/ Current Version]) will allow converting the $text variable files to UTF-8 without a BOM.
+
Load the cust_text.php file and save it specifying UTF-8 encoding without BOM, and place it in the corresponding folder (e.g. French-UTF8).
 
 
:That's it. Your site should now be running in UTF-8.
 
  
 
== Related links ==
 
== Related links ==
Line 88: Line 73:
 
* [[TNG charset]]
 
* [[TNG charset]]
 
*[[Database_Collation_-_Explain_Choosing|Selecting your TNG Database Collation]]
 
*[[Database_Collation_-_Explain_Choosing|Selecting your TNG Database Collation]]
 
+
*[[UTF-8 translator]] - a utility for replicating the ISO or UTF parts of a mod using html entities
  
 
[[Category:Charset]]
 
[[Category:Charset]]
 
[[Category:Programmer]]
 
[[Category:Programmer]]
 +
[[Category: Mods for TNG v12]]
 +
[[Category: Mods for TNG v11]]
 +
[[Category: Mods for TNG v10]]

Revision as of 06:59, 10 June 2018

Change coding
Summary Allows a simple change of character set and collation sequence for the TNG database
Validation
Mod Updated 23 Mar 2018
Download link 11.1.0.1
Download stats show
Author(s) Chris Moss
Homepage
Mod Support contact author
Contact Developer contact author
Latest Mod 11.1.0.1
Min TNG V
Max TNG V 11.1.2
Files modified
Related Mods
Notes
only available with English instructions


Notes on changing a TNG Site and database to UTF-8 provided by User:TheKiwi

Create a copy or backup of your database

You can use phpMyAdmin to make a copy of your database (always make a copy in case something goes wrong!!!!!!!)

  1. In the databases list on the left click on the Database to select it.
  2. On the right side click on the Operations Tab. In the section called "Copy Database to" enter a name for your new database, and make sure that "Structure and Database" and "CREATE DATABASE before copying" are both checked, then click "Go".

When done this will have created a copy of your database.

Alternatively you can back up your database using any of the methods described in Database - Backup.

Change database to UTF-8

You need to run a script to change the structure of your database to UTF-8. The previously recommended script has some problems so a new mod has been created which is much simpler to use and works in a flexible way.

Download the mod in the normal way, unzip, place it in your mods directory and click the install button.

After running the change coding mod

When it is installed, click on the green "Installed" line and then hit the "Change database" button below. The conversion will take a number of seconds and when it is complete a summary will appear similat to that shown at the left. Note in this case that the default language is English but the language actually in use when the conversion was done is German. The "utf8_general_ci" collation is suitable for most languages, but if you want a different one, then you can edit the options of the mod and put another collation, as long as it's consistent with the character set chosen.

If for any reason you want to change back to one of the other two supported character sets, ISO-8859-1 and ISO-8859-2, then choose the Edit options button on the mod and put one of these in the first box. In this case you will need to change the collation to latin1_swedish_ci or similar (there is no 'general' for the latin languages, the MySQL variant of ISO/8859).

The mod alters all the appropriate settings for the different language folders, including cookies and session variables, so you shouldn't need to do anything else. Once you have changed the database it's ok to uninstall the mod as it shouldn't be needed again. But's ok to run the mod again if, for instance, an incompatible table has been installed into your database or the settings have been disturbed by an update.

But don't try and import an ISO Gedcom file into a UTF-8 database (or vice versa). There will be no checking or conversion and you will end up with a mess if it contains accented characters.

Make sure that you have all the necessary UTF-8 Encoded language files

In the languages directory, there are pairs of folders for each language, e.g. French and French-UTF8, which have all the strings used in the two different character sets. If you have deleted the -UTF8 directories to save space, then these should be restored before running the change script above. Most mods will also include changes to both of these sets, normally changing the cust_text.php file.

However if you made changes to cust_text.php file yourself then you need to make sure these changes are also in the corresponding -UTF8 version of cust_text.php. The steps below will help you do that on Macintosh and Windows computers

On Macintosh system

Here's what works on Macintosh. You need BBEdit - a free text editor. If you don't already have BBEdit, you can download it from https://www.barebones.com/products/bbedit/download.html

  1. For each language folder open the file cust_text.php in BBEdit
  2. After the file is open, go to the File menu and choose "Reopen using Encoding ------> Western (Windows Latin 1).
  3. At the bottom of the BBEdit window is a pop up menu should now say "Western (Windows Latin 1)". Click on this and choose "Unicode (UTF-8)".
  4. Save the file and place in the appropriate directory.

On Windows system

You need to use a text editor that can handle saving files in UTF-8 format WITHOUT writing the BOM (Byte Order Mark) to the file such as Notepad++ (Current Version).

Load the cust_text.php file and save it specifying UTF-8 encoding without BOM, and place it in the corresponding folder (e.g. French-UTF8).

Related links