Character set problems (charset, iso-8859-1, utf-8, ...)

[rlegros]

I had some problems with character sets and Mambo Open Source. By default, MOS installs itself in English and uses the iso-8859-1 character set. I have installed the french translation packages and do run a MOS site in French. And there came the problems.

This information is valid for all non English Mambo Open Source installations.

The first problem I had was that the non ASCII characters ( e acute and so on ) were not displayed correctly. With IE 6.0 and Mozilla 1.6.

The second problem was that the menu had completely disappeared in the MOS back-end !

The real problem was that the browser tried to display the pages produced by Mambo using the UTF-8 character set. Doing so the Javascript code with non ASCII strings was broken. But why UTF-8 when MOS was configured to output ISO-8859-1 ?

To understand this, it must be known that the character set in a (X)HTML page is defined by using a HTTP header such as :

[code:1:4c466a0e36]Content-Type: text/html; charset=ISO-8859-1[/code:1:4c466a0e36]

This header is a HTTP one. If no charset is defined at the HTTP level, the browser will look in the HTML file at a META tag such as

[code:1:4c466a0e36]<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />[/code:1:4c466a0e36]

HTTP is low level, (X)HTML is higher level. A definition at HTTP level will take precedence over a definition in the HTML source.

So, even if Mambo is correctly configured, you have to check if the other pieces of the puzzle are too. An easy way to make such checks is to use wget. Available on all unix platforms and also on Windows ones ( see http://www.gnu.org/software/wget/ ). Do as follows :

[code:1:4c466a0e36]wget --server-response http://demo.mamboserver.com/[/code:1:4c466a0e36]

Doing so will download the requested URL in a file and will display all the HTTP headers generated by the server for that URL.

Now where are the places where this is configured ?

In the PHP configuration file php.ini (location depends on your distribution / installation), you have a line such as :

[code:1:4c466a0e36]default_charset = "iso-8859-1"[/code:1:4c466a0e36]

If this line does exist and is not commented out (preceded by a semicolon), it will define the HTTP header.

Otherwise, and only for those running Apache, have a look at the httpd.conf file. There, you will find a line such as :

[code:1:4c466a0e36]AddDefaultCharset ISO-8859-1[/code:1:4c466a0e36]

Here again, if the line does exist and is not commented out (preceded by a #), it will define the HTTP header.

php.ini definition takes precedence over httpd.conf definition. Only if nothing is defined in these two files will the Mambo definition be active.

And now for the long term solution.

The output of HTTP headers is possible in a PHP source. This will override the definitions that are (or could be) in the different configuration files. In each the Mambo source files each occurence of

[code:1:4c466a0e36]<meta http-equiv="Content-Type" content="text/html; <?php echo _ISO; ?>" />[/code:1:4c466a0e36]

must be deleted and replaced by a line such as

[code:1:4c466a0e36]<? header("Content-Type: text/html; charset=" . _ISO ); ?>[/code:1:4c466a0e36]

put at the beginning of the file (before the output of the (X)HTML tag).

That's it !

[chukks]

Bonjour Robert,

vu la pertinance de ton thread, je suis persuadé que tu va pouvoir m'aider...

Je sais que MOS4.5 n'est pas concu pour fonctionner avec de l'unicode, en particulier UTF-8, mais je pense que quelques manipulations pourraient rendre cela possible.

j'ai commencé par installer la nouvelle version de mysql 4.1.2alpha
J'ai modifié le my.ini pour le charset
default-character-set=utf8

ensuite j'ai créé une base de données en precisant egalement le charset!!
create database mambo character set utf8

en cherchant à créer les differents Table, du fichier sql fourni avec les dossiers d'installation de mambo, j' obtiens l'erreur suivante:

CREATE TABLE `mos_core_acl_aro` (

`aro_id` int( 11 ) NOT NULL AUTO_INCREMENT ,
`section_value` varchar( 240 ) NOT NULL default '0',
`value` varchar( 240 ) NOT NULL default '',
`order_value` int( 11 ) NOT NULL default '0',
`name` varchar( 255 ) NOT NULL default '',
`hidden` int( 11 ) NOT NULL default '0',
PRIMARY KEY ( `aro_id` ) ,
UNIQUE KEY `section_value_value_aro` ( `section_value` , `value` ) ,
UNIQUE KEY `mos_gacl_section_value_value_aro` ( `section_value` , `value` ) ,
KEY `hidden_aro` ( `hidden` ) ,
KEY `mos_gacl_hidden_aro` ( `hidden` )
) TYPE = MYISAM

MySQL a répondu:


#1071 - Specified key was too long; max key length is 1000 bytes

es ce la bonne demarche??

En fait je voudrais realiser un portail multilingue avec des langues comme le chinois et l'arabe, mais de maniere a ce que les données soient stockés ds la base sous une autre forme que #;xxx...

Si j'ai bien compris pour la config PHP et apache, il faudrait enlever du php.ini et du http.conf les def de charset ISO-8859-1 et laisser mambo lui meme fixer le charset à utiliser...



Merci

[lulali]

Thanks a lot :hugs: