My encodings are messed up

colak · 2010-07-11 09:19:30

After the last languages update all instances of letters with accents have messed up. The config file is the same for some years no with $txpcfg['dbcharset'] = 'latin1'; in place.

to explain: I write “São Paulo” in the admin side and I get SÃ£o Paulo in the public one.

Diagnostics

high:

Charset (default/config): latin1/latin1
character_set_client: latin1
character_set_connection: latin1
character_set_database: latin1
character_set_results: latin1
character_set_server: latin1
character_set_system: utf8
character_sets_dir: /usr/local/share/mysql/charsets/
24 Tables: OK

low

Textpattern version: 4.2.0 (r3275)
Last Update: 2009-08-30 07:20:18/2009-08-30 07:13:38
Document root: /users/home/yyy/web/public
$path_to_site: /users/home/yyy/web/public/main
Textpattern path: /users/home/yyy/web/public/main/textpattern
Permanent link mode: id_title
upload_tmp_dir: /usr/local/www/php/tmp
Temporary directory path: /users/home/yyy/web/public/main/textpattern/tmp
Site URL: www.neme.org/main
PHP version: 5.1.4
GD Image Library: bundled (2.0.28 compatible); supported formats: GIF, JPG, PNG.
Server TZ: -
Server Local Time: 2010-07-11 09:10:19
DST enabled?: 1
Automatically adjust DST setting?: 
Time Zone: GMT+02.0 (7200)
MySQL: 4.1.18-log
Locale: en_GB.ISO_8859-1
Server: Apache
PHP Server API: cgi-fcgi
RFC 2616 headers: 
Server OS: FreeBSD 5.4-STABLE
Active plugins: zem_redirect-1.2.1m, fha_time_diff-0.23m, mdn_count-1.4, mem_glinklist-0.7.4, ob1_googlenav-2.0m, glx_countdown-0.1, glx_admin_ping-0.1m, mrh_email_article_link-0.3m, glx_if-0.7, jnm_recent_comments_unique-0.4, etz_striptags-0.2, wet_if_page-0.1, sed_afu-0.6m, chh_if_data-0.10, wet_slimpattern-0.4.1m, csb_if_newer_than-1.0, wet_opensearch-0.1, ike_slideshow-0.02m, wlk_qt-0.9.7, the_video-0.7, rvm_privileged-0.2, SimplePie Plugin-1.2.2m, yab_image-0.3, ako_random_comment-0.2, ajw_notepad_public-0.1m, glx_admin_image-2006.1, rss_admin_db_manager-4.3m, upm_savenew-0.4.2, zem_article_info-0.1, bas_locate_me-0.1, rss_article_edit-0.1, stm_javascript-0.1m, rvm_css-0.5, rah_sitemap-0.5, rah_external_output-0.3, rah_unlog_me-0.1, ied_plugin_composer-0.83, smd_where_used-0.2, smd_prefalizer-0.12
Admin-side theme: remora 4.2.0

.htaccess file contents: 
------------------------
RewriteEngine On
RewriteRule ^sitemap.xml.gz?$ ?rah_sitemap=sitemap

RewriteEngine On
#Options +FollowSymlinks

#DirectoryIndex index.php index.html

#Options +FollowSymLinks
#Options -Indexes


<IfModule mod_rewrite.c>
	RewriteEngine On
	#RewriteBase /relative/web/path/

	RewriteCond %{REQUEST_FILENAME} -f [OR]
	RewriteCond %{REQUEST_FILENAME} -d
	RewriteRule ^(.+) - [PT,L]

	RewriteCond %{REQUEST_URI} !=/favicon.ico
	RewriteRule ^(.*) index.php

	RewriteCond %{HTTP:Authorization}  !^$
	RewriteRule .* - [E=REMOTE_USER:%{HTTP:Authorization}]
</IfModule>

#php_value register_globals 0

------------------------

maniqui · 2010-07-11 14:57:56

Hi Yiannis,

I would suggest:

1) Backup the DB
2) Install rvm_latin1_to_utf8
3) Follow its instructions (and remember to check if config.php is set to utf8).
4) Optional: you may want reinforce things by adding AddDefaultCharset UTF-8 at the top of .htaccess.

colak · 2010-07-11 16:40:53

Hi Julián

This seems to have done it. Thanks! With over 1000 articles it’ll take me some time to check if all worked fine.

In any case these are the charsets in high diagnostics

Charset (default/config): latin1/utf8
character_set_client: utf8
character_set_connection: utf8
character_set_database: latin1
character_set_results: utf8
character_set_server: latin1
character_set_system: utf8
character_sets_dir: /usr/local/share/mysql/charsets/
24 Tables: txp_bot_privs is latin1, txp_js is latin1, txp_log_mention is latin1, txp_nlprefs is latin1, txp_note is latin1, txp_subscribers is latin1

Are they what they are supposed to be?

maniqui · 2010-07-11 16:44:44

I’m not sure, as this exceeds my understandings. It seems some tables are still latin1, although those tables look like non-standard, plugin related TXP tables, correct?

colak · 2010-07-11 17:41:55

Yep those are plugin tables. Ruud’s plugin found them but did not check them and I was not sure if I had to… so I didn’t:)

Gocom · 2010-07-11 19:20:05

colak wrote:

Yep those are plugin tables. Ruud’s plugin found them but did not check them and I was not sure if I had to… so I didn’t:)

Many plugin authors tend to use DBMS’s default settings, and don’t define the charset, or use the charset defined from the config.php, in the plugin installation scripts. Usually also the plugin created tables should be set to UTF-8, as the content strored most likely is UTF-8 encoded.

colak · 2010-07-11 19:59:52

Gocom wrote:

Many plugin authors tend to use DBMS’s default settings, and don’t define the charset, or use the charset defined from the config.php, in the plugin installation scripts. Usually also the plugin created tables should be set to UTF-8, as the content strored most likely is UTF-8 encoded.

Thanks Jukka. So what’s the advise from here?

Gocom · 2010-07-11 20:58:21

colak wrote:

Thanks Jukka. So what’s the advise from here?

If everything works perfectly, and you are not doing searches in those tables, then I wouldn’t worry about it.

Altho, you could change the tables to UTF-8 which they probably should be.

Textpattern CMS

Textpattern CMS support forum

#1 2010-07-11 09:19:30

My encodings are messed up

#2 2010-07-11 14:57:56

Re: My encodings are messed up

#3 2010-07-11 16:40:53

Re: My encodings are messed up

#4 2010-07-11 16:44:44

Re: My encodings are messed up

#5 2010-07-11 17:41:55

Re: My encodings are messed up

#6 2010-07-11 19:20:05

Re: My encodings are messed up

#7 2010-07-11 19:59:52

Re: My encodings are messed up

#8 2010-07-11 20:58:21

Re: My encodings are messed up

Board footer