rvm_latin1_to_utf8 (v0.5)

jstubbs · 2007-12-02 11:32:39

Interesting. I just did the following:

1. Created a new DB on Nicola
2. Tried to copy the site DB to the new DB from phpMyAdmin, could not due to error with default charset being set to Latin1 (MySQL 4.1.22 so all new DB’s are UTF-8 by default)
3. Instead exported the site DB, then imported the DB to the test DB. Selected Latin1 as the character set of the file
4. In TXP config.php, changed the site DB to the new test DB. Immediately, I see the foreign characters are mangled!
5. Ran rvm_latin1_to_utf8 plugin
6. Checked, and still see all foreign characters are messed up

After this, I thought I would test the php.ini and config.php settings. I changed config.php dbcharset to UTF8, and php.ini also to UTF-8.

No change. Still messed up.

??!!

ruud · 2007-12-02 11:55:25

Can you post the part of the TXP full diagnostics on Nicola that shows the charset information?

jstubbs · 2007-12-02 12:11:12

I’d love to, but I am getting a 500 error on the diagnostics page :-(

Will keep trying.

jstubbs · 2007-12-03 19:35:05

Ruud, I can’t get past a 500 internal server error on the diagnostics page (on Nicola/TextDrive/FreeBSD), so I can’t check the charset for you.

Mamash over at Joyent says its a TXP error that hits the FastCGI limits on TXD – any idea how I can get the diagnostics?

I have a ticket open on TXD, but in the meantime, any help appreciated :-)

ruud · 2007-12-03 20:02:42

In PHPmyAdmin, look at the collation/charset of the various text columns in textpattern tables. What does the charset look like there?

What have you set in textpattern/config.php as dbcharset?

jstubbs · 2007-12-03 20:50:46

config.php: $txpcfg[‘dbcharset’] = ‘latin1’;

Nicola MySQL: 4.1.22
MySQL charset: UTF-8 Unicode (utf8)
MySQL connection collation: utf_unicode_ci

All the tables are listed as MyISAM and latin1_swedish_ci

ruud · 2007-12-03 21:02:36

Perhaps you should open a new topic about this, because it’s not really related to the plugin this topic is about. My plugin should be able to convert your tables to use utf8 on Nicola. After that just a matter of migrating the database to a different server and it becomes a PHPmyAdmin export/import question.

Last edited by ruud (2007-12-03 21:03:22)

moslib · 2008-03-26 23:34:28

I get Fatal error: Call to undefined function safe_pfx() in /public_html/textpattern/lib/txplib_misc.php(459) : eval()'d code on line 115 on older Txp sites

ruud · 2008-03-27 09:23:43

First upgrade to a recent TXP version.

colak · 2014-08-18 05:15:27

I am trying to update the character set of an old site using this plugin but it truncates all article bodies.

The site was using txp4.0.3 in joyent and it has now been moved to bluehost where I upgraded it to the latest stable release, 4.5.5.

>Edited to add some diagnostics

Textpattern version: 4.5.5 (r5575)
PHP version: 5.2.17
gd_library: gd_info: bundled (2.0.34 compatible), GIF, JPG, PNG
MySQL: 5.5.38-35.2-log
Locale: en_GB.UTF-8
Server: Apache
php_sapi_mode: cgi-fcgi
rfc2616_headers: 0
os_version: Linux 3.12.26.1406932904
active_plugins: glx_if-0.7m, hak_textile_tags-0.1m, czg_if_comment_cookies-0.1m, ob1_title-2.2, rss_live_search-0.2.2m, zem_article_image-0.1, ob1_googlenav-2.0m, dca_pop-0.1m, walk_qt-0.4.2, zem_contact_lang-4.0.3.6, zem_contact_reborn-4.0.3.20, upm_img_popper-0.9.9m, asy_sitemap-0.7m, rss_admin_db_manager-4.3, smd_where_used-0.30
theme_name: remora 4.5.5

Last edited by colak (2014-08-18 07:03:33)

candyman · 2014-08-18 12:58:14

I’ve missed that, very interesting.
I would need a thing like this to migrate a phpBB forum (collation problem): is there something that could help me?
Any docs?

Thanks in advance.

Last edited by candyman (2014-08-18 12:58:22)

ruud · 2014-08-19 19:58:30

colak wrote #282988:

I am trying to update the character set of an old site using this plugin but it truncates all article bodies.

Only the article bodies… and all the bodies or just a few? Did it completely empty the article body? Could it be that the original articles contained non-utf8 characters?

candyman wrote #283019:

I would need a thing like this to migrate a phpBB forum (collation problem): is there something that could help me? Any docs?

You could have a look at the plugin code. That might get you started. Otherwise ask around on the phpBB forum. Perhaps someone else already wrote something similar.

colak · 2014-08-20 05:21:17

ruud wrote #283061:

Only the article bodies… and all the bodies or just a few? Did it completely empty the article body? Could it be that the original articles contained non-utf8 characters?

It truncated all article bodies without emptying them completely. The original articles contained texts in english copy pasted from word.

ruud · 2014-08-20 17:30:25

How long are the truncated article bodies? The part that remains, I mean, in bytes?

colak · 2014-08-20 18:07:40

I restored the site and it is currently using Latin1 but I can repeat the installation of your plugin tomorrow and answer your question. Is there anything other info you may find helpful and I should look out for?

Textpattern CMS

Textpattern CMS support forum

#31 2007-12-02 11:32:39

Re: rvm_latin1_to_utf8 (v0.5)

#32 2007-12-02 11:55:25

Re: rvm_latin1_to_utf8 (v0.5)

#33 2007-12-02 12:11:12

Re: rvm_latin1_to_utf8 (v0.5)

#34 2007-12-03 19:35:05

Re: rvm_latin1_to_utf8 (v0.5)

#35 2007-12-03 20:02:42

Re: rvm_latin1_to_utf8 (v0.5)

#36 2007-12-03 20:50:46

Re: rvm_latin1_to_utf8 (v0.5)

#37 2007-12-03 21:02:36

Re: rvm_latin1_to_utf8 (v0.5)

#38 2008-03-26 23:34:28

Re: rvm_latin1_to_utf8 (v0.5)

#39 2008-03-27 09:23:43

Re: rvm_latin1_to_utf8 (v0.5)

#40 2014-08-18 05:15:27

Re: rvm_latin1_to_utf8 (v0.5)

#41 2014-08-18 12:58:14

Re: rvm_latin1_to_utf8 (v0.5)

#42 2014-08-19 19:58:30

Re: rvm_latin1_to_utf8 (v0.5)

colak wrote #282988:

candyman wrote #283019:

#43 2014-08-20 05:21:17

Re: rvm_latin1_to_utf8 (v0.5)

ruud wrote #283061:

#44 2014-08-20 17:30:25

Re: rvm_latin1_to_utf8 (v0.5)

#45 2014-08-20 18:07:40

Re: rvm_latin1_to_utf8 (v0.5)

Board footer