Textpattern CMS support forum
You are not logged in. Register | Login | Help
- Topics: Active | Unanswered
Character Code Problems
I’ve setup textpattern both on my server, and on my local machine as a backup/mirror/demo.
Things run fine, but I noticed an annoying character code glitch. In the admin>preferences>advanced, there are several character code glitches. Here’s an image showing some of them
This is the latest 4.0.4 release, and I run the same on my server, which does NOT show the same glitch! Weird.
Does anyone know how this sort of character set problem?
More facts: both server and localhost show utf-8 in the html header as the character code. The local machine is a Mac, using localhost/~myself as the url.
Owen
Last edited by backspaces (2006-12-08 01:09:44)
Offline
#2 2006-12-08 01:13:32
- Mary
- Sock Enthusiast
- Registered: 2004-06-27
- Posts: 6,236
Re: Character Code Problems
…show utf-2 in the html header as the character code.
UTF-2? That would mean you’ve modified at least one core file.
Please post your diagnostics.
Offline
Re: Character Code Problems
Mary wrote:
…show utf-2 in the html header as the character code.
UTF-2? That would mean you’ve modified at least one core file. Please post your diagnostics.
Err..I meant “8” .. sorry!
Owen
Diagnostics:
Textpattern version: 4.0.4 (r1956)
Last Update: 2006-10-18 14:44:46/2006-09-01 00:47:58
Document root: /Library/WebServer/Documents
$path_to_site: /Users/owen/Sites
Textpattern path: /Users/owen/Sites/textpattern
Permanent link mode: section_id_title
Temporary directory path: /tmp
Site URL: localhost/~owen
PHP version: 4.4.1
Server Local Time: 2006-12-07 21:12:13
MySQL: 5.0.27-standard
Locale: en_US.UTF-8
Server: Apache/1.3.33 (Darwin) PHP/4.4.1
Apache version: Apache/1.3.33 (Darwin) PHP/4.4.1
PHP Server API: apache
RFC 2616 headers:
Server OS: Darwin 8.8.2
Active plugins: mdn_sectionlink-1.0, mdn_if_section-1.0, zem_link-0.2, mem_glinklist-0.3m, mrw_spamkeywords_urlcount-0.1, asy_stopdude-0.1
Pre-flight check:
————————————
/Users/owen/Sites/textpattern/setup/ still exists
Clean URL test failed.
————————————
.htaccess file contents:
————————————
#DirectoryIndex index.php index.html
#Options +FollowSymLinks
#RewriteBase /relative/web/path/
<IfModule mod_rewrite.c> RewriteEngine On RewriteCond %{REQUEST_FILENAME} -f [OR] RewriteCond %{REQUEST_FILENAME} -d RewriteRule ^(.+) – [PT,L]
RewriteRule ^(.*) index.php </IfModule>#php_value register_globals 0
————————————
Offline
#4 2006-12-08 05:34:57
- Mary
- Sock Enthusiast
- Registered: 2004-06-27
- Posts: 6,236
Re: Character Code Problems
Pardon me, we need the “High” output results.
Offline
Re: Character Code Problems
Have you been exporting/importing your database yourself? Or did you make two seperate installations without any table-dumping?
If it’s the former, please give us the mysql-versions of each installation; tell us from where you made the dump (“original” installation), and how you made the dump, and how you made the import.
Offline
Re: Character Code Problems
Hi mary, sencer and backspaces
I ran into the same problem (for the first time after many database import/exports I did before) while preparing a site move from a txp install from (barclay at) textdrive (mysql 4.1) to a nl hosting company pcextreme (mysql 5) which I am trying out for my nl sites. I used this instruction from textbook to the letter.
Here are part of the diagnostics at textdrive and all at pcextreme. Result of the import is scrambled display of languages in txp admin and incorrect display of accents etc. in postst which I have allready corrected by hand. This database was created in an older version of mysql. This is why table charsets are latin1. How can I do the import so that all utf8 caracters are displayed correctly?
kees
diagnostics @textdrive
Charset (default/config): latin1/
character_set_client: latin1
character_set_connection: latin1
character_set_database: latin1
character_set_results: latin1
character_set_server: latin1
character_set_system: utf8
character_sets_dir: /usr/local/share/mysql/charsets/
19 Tables: OK
diagnostics @pcextreme
/home/polychrome/domains/polychrome.nl/public_html/dev/textpattern
Permanente link modus: section_id_title
Tijdelijke bestanden map: /home/polychrome/domains/polychrome.nl/public_html/dev/textpattern/tmp
Site URL: dev.polychrome.nl
PHP versie: 5.2.0
Lokale server tijd: 2006-12-08 10:24:54
MySQL: 5.0.27-standard
Localisatie: nl_NL.UTF-8
Server: Apache/2.0.59 (Unix) mod_ssl/2.0.59 OpenSSL/0.9.7e
Apache versie: Apache/2.0.59 (Unix) mod_ssl/2.0.59 OpenSSL/0.9.7e
PHP server API: apache2handler
RFC 2616 headers:
Besturingssysteem van de server: Linux 2.6.16.29-xen
Actieve plugins: zem_popup_cat-0.1
inhoud .htaccess bestand:
————————————
DirectoryIndex index.php index.html
Options +FollowSymLinks
#RewriteBase
<IfModule mod_rewrite.c> RewriteEngine On RewriteCond %{REQUEST_FILENAME} -f [OR] RewriteCond %{REQUEST_FILENAME} -d RewriteRule ^(.+) – [PT,L]
RewriteRule ^(.*) index.php </IfModule>#php_value register_globals 0
————————————
Charset (default/config): latin1/utf8
character_set_client: utf8
character_set_connection: utf8
character_set_database: latin1
character_set_filesystem: binary
character_set_results: utf8
character_set_server: latin1
character_set_system: utf8
character_sets_dir: /usr/local/mysql-standard-5.0.27-linux-i686-glibc23/share/mysql/charsets/
19 Tables: textpattern is latin1, txp_category is latin1, txp_css is latin1, txp_discuss is latin1, txp_discuss_ipban is latin1, txp_discuss_nonce is latin1, txp_file is latin1, txp_form is latin1, txp_image is latin1, txp_lang is latin1, txp_link is latin1, txp_log is latin1, txp_log_mention is latin1, txp_page is latin1, txp_plugin is latin1, txp_prefs is latin1, txp_priv is latin1, txp_section is latin1, txp_users is latin1
Last edited by kees-b (2006-12-08 10:38:43)
Offline
Re: Character Code Problems
kees, your problem is due to the fact that you are using a different config.php, or more specifically a different character-set configuration. You probably made a new install on the new host and replaced the db. In your specific case, you need to edit your new config.php file, and replace the “utf8” with “latin1” (or delete the line entirely, as it is in your old config.php).
Offline
Re: Character Code Problems
Sencer wrote:
kees, your problem is due to the fact that you are using a different config.php, or more specifically a different character-set configuration. You probably made a new install on the new host and replaced the db. In your specific case, you need to edit your new config.php file, and replace the “utf8” with “latin1” (or delete the line entirely, as it is in your old config.php).
Hi Sencer, you are right. I did a clean install and after that replaced the database content. I deleted the line as you suggested and got the ’19 tables ok’ in diagnostics but I still see ???? where I should see non latin characters in admin/language. Also all my previous edits for the accents are lost again.
k
Last edited by kees-b (2006-12-08 10:59:18)
Offline
Re: Character Code Problems
After a refresh of the language install and manual (re-)editing of some posts everything is displayed ok!
thanks
-k
Last edited by kees-b (2006-12-08 11:17:55)
Offline
Re: Character Code Problems
Hi Mary. Here’s the long diagnostic. I think my problem may be similar to Kees. There certainly are differences in versions w/ MySql for example. There is also a difference between the two config.php files
Local:
$txpcfg[‘dbcharset’] = ‘utf8’;
Server:
$txpcfg[‘dbcharset’] = ‘latin1’;
Is there a pointer in textbook or the forum that discusses these sorts of problems? Actually, my goal would be to have EVERYTHING be utf8 and be done with it! I’m not sure how to convert the existing DB to utf8, however.
Textpattern version: 4.0.4 (r1956)
Last Update: 2006-10-18 14:44:46/2006-09-01 00:47:58
Document root: /Library/WebServer/Documents
$path_to_site: /Users/owen/Sites
Textpattern path: /Users/owen/Sites/textpattern
Permanent link mode: section_id_title
Temporary directory path: /tmp
Site URL: localhost/~owen
PHP version: 4.4.1
Server Local Time: 2006-12-08 09:19:23
MySQL: 5.0.27-standard
Locale: en_US.UTF-8
Server: Apache/1.3.33 (Darwin) PHP/4.4.1
Apache version: Apache/1.3.33 (Darwin) PHP/4.4.1
PHP Server API: apache
RFC 2616 headers:
Server OS: Darwin 8.8.2
Active plugins: mdn_sectionlink-1.0, mdn_if_section-1.0, zem_link-0.2, mem_glinklist-0.3m, mrw_spamkeywords_urlcount-0.1, asy_stopdude-0.1
Pre-flight check:
————————————
Clean URL test failed.
————————————
.htaccess file contents:
————————————
#DirectoryIndex index.php index.html
#Options +FollowSymLinks
#RewriteBase /relative/web/path/
<IfModule mod_rewrite.c> RewriteEngine On RewriteCond %{REQUEST_FILENAME} -f [OR] RewriteCond %{REQUEST_FILENAME} -d RewriteRule ^(.+) – [PT,L]
RewriteRule ^(.*) index.php </IfModule>#php_value register_globals 0
————————————
Charset (default/config): latin1/utf8
character_set_client: utf8
character_set_connection: utf8
character_set_database: latin1
character_set_filesystem: binary
character_set_results: utf8
character_set_server: latin1
character_set_system: utf8
character_sets_dir: /usr/local/mysql-standard-5.0.27-osx10.4-i686/share/mysql/charsets/
19 Tables: textpattern is latin1, txp_category is latin1, txp_css is latin1, txp_discuss is latin1, txp_discuss_ipban is latin1, txp_discuss_nonce is latin1, txp_file is latin1, txp_form is latin1, txp_image is latin1, txp_lang is latin1, txp_link is latin1, txp_log is latin1, txp_log_mention is latin1, txp_page is latin1, txp_plugin is latin1, txp_prefs is latin1, txp_priv is latin1, txp_section is latin1, txp_users is latin1
PHP extensions: xml, tokenizer/0.1, standard/4.4.1, sockets, session, posix, pcre, overload, odbc, mysql, mbstring, ldap, ftp, exif/1.4 $Id: exif.c,v 1.118.2.37.2.3 2005/10/10 06:07:16 helly Exp $, dbx, curl, ctype, zlib/1.1, apache
/include/txp_category.php: r1879 (aee777474b2f67ca07fc25756ba25c15)
/include/txp_plugin.php: r1917 (74184c0d8ed8608f840707a255178617)
/include/txp_auth.php: r1879 (b1dd4072b7daf4e997c6ff65ce3d1b2d)
/include/txp_form.php: r1913 (16ec600b41438b4cca10d2c8a19b2db8)
/include/txp_section.php: r1891 (2959593586ba3e97bc602f369c32e738)
/include/txp_tag.php: r1915 (3b4a7f73d92f9bbbe09985c5aa830d29)
/include/txp_list.php: r1892 (41f4d32fd070234b78f94adefebd5234)
/include/txp_page.php: r1913 (34331a5468bbb18dd9f6a282f3aa11e8)
/include/txp_discuss.php: r1909 (9b9ee934a30f52cd7a4d8cb45c8380ab)
/include/txp_prefs.php: r1946 (05615b6275d8927a2a0d51918d70a896)
/include/txp_log.php: r1919 (ed54d02e865319f2506c642a6bde768b)
/include/txp_preview.php: r1238 (5a4ae3ff0d68f4cb573d6d62a00ce9e8)
/include/txp_image.php: r1955 (20be975e67fa7c4aa9a1a3e51bfaf379)
/include/txp_article.php: r1889 (7749f699c03d0c57e04fafe17dbfa94c)
/include/txp_css.php: r1897 (f5cf1c20badb96a063c7c180e9020359)
/include/txp_admin.php: r1879 (d36dac010d21df7bcf9cf5e242b34d58)
/include/txp_link.php: r1879 (0652287df8bb32c66cfa1b939402404a)
/include/txp_diag.php: r1902 (96697ade63048e517177f4129d47de76)
/include/txp_file.php: r1895 (6ed67b094522e51b028dc88baa07444c)
/include/txp_import.php: r1238 (634e75d1b61958875ff275e3130f23ad)
/lib/admin_config.php: r1747 (1563fcbaffe25b3272b0d85ff9d5571d)
/lib/txplib_misc.php: r1956 (182c50b86195f1abe9dbe15728df3cae)
/lib/taglib.php: r1535 (04806ef864d5b0d2974e0e5f6397a2d7)
/lib/txplib_head.php: r1887 (b110efd071e9a5bb395beea66ced128a)
/lib/classTextile.php: r1943 (2c559991e34738eef1990dc079bd91c4)
/lib/txplib_html.php: r1937 (c206ca9cb9a54a7a95f3355b77fd0fa2)
/lib/txplib_db.php: r1879 (d68b6ea69950e405c4fec23b8641d9c2)
/lib/IXRClass.php: r765 (cbe59b59246dce060a4b4a52b4d448d8)
/lib/txplib_forms.php: r1887 (0049a228dc8eb346f8603478a7c1b2e2)
/lib/class.thumb.php: r1955 (12961180eee3add5096e69e0a154284e)
/lib/constants.php: unknown (0e40251c717c52b2b7fe992b62a3e97a)
/lib/txplib_update.php: r1239 (757f8189fcc53a795d7c807f17b2e788)
/lib/txplib_wrapper.php: unknown (584448787b4a3488200722672c0eee0d)
/publish/taghandlers.php: r1949 (3fa1b9ded18e6074b2495a3f4e3c33b5)
/publish/atom.php: r1864 (50602e2f1c443819a0a60f14f39d3093)
/publish/log.php: r1637 (a4a772567079f18101a1752446f3f6d4)
/publish/comment.php: r1951 (a3f803d744fea80808eb27a3f6b28674)
/publish/search.php: r1748 (b0182abc287055fe0932c263b2a5266d)
/publish/rss.php: r1864 (ae43eaa9ebe6b00e63810ae60ca7c6b6)
/publish.php: r1945 (abff727405efc6c4ec8b1cb403290063)
/index.php: r1948 (adf86f44861797f4969373c708ef48fb)
/css.php: r944 (763fa7658fc19ad23a5b2126fcdf366c)
Offline
Re: Character Code Problems
I thought a dump of the server diagnostics might also help.
Textpattern version: 4.0.4 (r1956)
Last Update: 2006-10-18 14:44:46/2006-10-18 14:40:34
Document root: /home/backspac/public_html
$path_to_site: /home/backspac/public_html
Textpattern path: /home/backspac/public_html/textpattern
Permanent link mode: section_id_title
open_basedir: /home/backspac/:/usr/lib/php:/usr/local/lib/php:/tmp
Temporary directory path: /tmp
Site URL: backspaces.net
PHP version: 4.4.2
Register globals: 1
Server Local Time: 2006-12-08 11:32:26
MySQL: 4.1.21-standard
Locale: en_US.UTF-8
Server: Apache
Apache version: Apache
PHP Server API: apache
RFC 2616 headers:
Server OS: Linux 2.4.21-47.0.1.ELsmp
Active plugins: mdn_sectionlink-1.0, mdn_if_section-1.0, zem_link-0.2, mem_glinklist-0.3m, mrw_spamkeywords_urlcount-0.1, asy_stopdude-0.1
.htaccess file contents:
————————————
#DirectoryIndex index.php index.html
#Options +FollowSymLinks
#RewriteBase /relative/web/path/
<IfModule mod_rewrite.c> RewriteEngine On RewriteCond %{REQUEST_FILENAME} -f [OR] RewriteCond %{REQUEST_FILENAME} -d RewriteRule ^(.+) – [PT,L]
RewriteRule ^(.*) index.php </IfModule>#php_value register_globals 0
————————————
Charset (default/config): latin1/latin1
character_set_client: latin1
character_set_connection: latin1
character_set_database: latin1
character_set_results: latin1
character_set_server: latin1
character_set_system: utf8
character_sets_dir: /usr/share/mysql/charsets/
19 Tables: OK
PHP extensions: zip, xslt, xmlrpc/0.51, xml, wddx, tokenizer/0.1, swf, standard/4.4.2, sockets, session, pspell, posix, pdf, overload, mysql, ming, mhash, mcrypt, mbstring, imap, iconv, gettext, gd, ftp, exif/1.4 $Id: exif.c,v 1.118.2.37.2.4 2006/01/01 13:46:52 sniper Exp $, domxml/20020815, curl, ctype, calendar, bcmath, zlib/1.1, pcre, openssl, apache, Zend Optimizer
pretext_data: array (
‘id’ => ‘’,
‘s’ => ‘’,
‘c’ => ‘’,
‘q’ => ‘’,
‘pg’ => ‘’,
‘p’ => ‘’,
‘month’ => ‘’,
‘author’ => ‘’,
‘request_uri’ => ‘/25daa3a1f1c1dcb8a72c310a53abc293/?txpcleantest=1’,
‘qs’ => ‘txpcleantest=1’,
‘subpath’ => ‘\\/’,
‘req’ => ‘/25daa3a1f1c1dcb8a72c310a53abc293/?txpcleantest=1’,
)
/include/txp_category.php: r1879 (aee777474b2f67ca07fc25756ba25c15)
/include/txp_plugin.php: r1917 (74184c0d8ed8608f840707a255178617)
/include/txp_auth.php: r1879 (b1dd4072b7daf4e997c6ff65ce3d1b2d)
/include/txp_form.php: r1913 (16ec600b41438b4cca10d2c8a19b2db8)
/include/txp_section.php: r1891 (2959593586ba3e97bc602f369c32e738)
/include/txp_tag.php: r1915 (3b4a7f73d92f9bbbe09985c5aa830d29)
/include/txp_list.php: r1892 (41f4d32fd070234b78f94adefebd5234)
/include/txp_page.php: r1913 (34331a5468bbb18dd9f6a282f3aa11e8)
/include/txp_discuss.php: r1909 (9b9ee934a30f52cd7a4d8cb45c8380ab)
/include/txp_prefs.php: r1946 (05615b6275d8927a2a0d51918d70a896)
/include/txp_log.php: r1919 (ed54d02e865319f2506c642a6bde768b)
/include/txp_preview.php: r1238 (5a4ae3ff0d68f4cb573d6d62a00ce9e8)
/include/txp_image.php: r1955 (20be975e67fa7c4aa9a1a3e51bfaf379)
/include/txp_article.php: r1889 (7749f699c03d0c57e04fafe17dbfa94c)
/include/txp_css.php: r1897 (f5cf1c20badb96a063c7c180e9020359)
/include/txp_admin.php: r1879 (d36dac010d21df7bcf9cf5e242b34d58)
/include/txp_link.php: r1879 (0652287df8bb32c66cfa1b939402404a)
/include/txp_diag.php: r1902 (96697ade63048e517177f4129d47de76)
/include/txp_file.php: r1895 (6ed67b094522e51b028dc88baa07444c)
/include/txp_import.php: r1238 (634e75d1b61958875ff275e3130f23ad)
/lib/admin_config.php: r1747 (1563fcbaffe25b3272b0d85ff9d5571d)
/lib/txplib_misc.php: r1956 (182c50b86195f1abe9dbe15728df3cae)
/lib/taglib.php: r1535 (04806ef864d5b0d2974e0e5f6397a2d7)
/lib/txplib_head.php: r1887 (b110efd071e9a5bb395beea66ced128a)
/lib/classTextile.php: r1943 (2c559991e34738eef1990dc079bd91c4)
/lib/txplib_html.php: r1937 (c206ca9cb9a54a7a95f3355b77fd0fa2)
/lib/txplib_db.php: r1879 (d68b6ea69950e405c4fec23b8641d9c2)
/lib/IXRClass.php: r765 (cbe59b59246dce060a4b4a52b4d448d8)
/lib/txplib_forms.php: r1887 (0049a228dc8eb346f8603478a7c1b2e2)
/lib/class.thumb.php: r1955 (12961180eee3add5096e69e0a154284e)
/lib/constants.php: unknown (0e40251c717c52b2b7fe992b62a3e97a)
/lib/txplib_update.php: r1239 (757f8189fcc53a795d7c807f17b2e788)
/lib/txplib_wrapper.php: unknown (584448787b4a3488200722672c0eee0d)
/publish/taghandlers.php: r1949 (3fa1b9ded18e6074b2495a3f4e3c33b5)
/publish/atom.php: r1864 (50602e2f1c443819a0a60f14f39d3093)
/publish/log.php: r1637 (a4a772567079f18101a1752446f3f6d4)
/publish/comment.php: r1951 (a3f803d744fea80808eb27a3f6b28674)
/publish/search.php: r1748 (b0182abc287055fe0932c263b2a5266d)
/publish/rss.php: r1864 (ae43eaa9ebe6b00e63810ae60ca7c6b6)
/publish.php: r1945 (abff727405efc6c4ec8b1cb403290063)
/index.php: r1948 (adf86f44861797f4969373c708ef48fb)
/css.php: r944 (763fa7658fc19ad23a5b2126fcdf366c)
Offline
#12 2006-12-09 02:30:12
- Mary
- Sock Enthusiast
- Registered: 2004-06-27
- Posts: 6,236
Re: Character Code Problems
Your local Textpattern’s config.php will need to be latin1 too.
Converting from latin1 to utf-8 can be tricky, it’s not really something for beginners. But, you can try if you want to, just make sure you don’t modify the original database or the backup for it (always work on a copy of the db dump). Some relevant links:
Offline