Quotes in pasted blocks of text

Bloke · 2016-10-03 12:34:57

StFual wrote #301919:

Is “utf8mb4” a default

Yep. You can change it if you like, right there as you create the config file. Since the rest of your installation seems (mostly) to be using latin1, you could try that value and see what happens. Not a very long-term solution, though, unless you know you’re only ever going to be publishing in standard Western / European characters.

StFual · 2016-10-04 10:26:18

These were the settings from a known working system from phiw13 (thankyou)

Charset (default/config): latin1/utf8
character_set_client: utf8
character_set_connection: utf8
character_set_database: utf8
character_set_filesystem: binary
character_set_results: utf8
character_set_server: latin1
character_set_system: utf8

On my buggy system they are currently

Charset (default/config): latin1/utf8mb4
character_set_client: latin1
character_set_connection: latin1
character_set_database: utf8mb4
character_set_filesystem: binary
character_set_results: latin1
character_set_server: latin1
character_set_system: utf8

I note that the default on both systems is latin1. It seems that most hosting systems are set with a default of swedish latin1

I have been able to change “character_set_database” from ‘latin1’ to ‘utf8mb4’ by changing the collation setting in Phpmyadmin after database creation, prior to running install script. It hasn’t changed anything re. the problem

Presumably the settings on phiw13 host were either configured by the install script, or manually, or get set in a session script or .ini file.
Can i set the the others as session variables in .htaccess ? Anyone have any idea how they get configured on textpattern “supported” hosts like Arvixe and Dreamhost ?

StFual · 2016-10-05 13:36:40

After much testing what appears to be happening in my environment is the installation language pack causes the problem.

Do an install with any of the english language packs
Problems with double and single quotes
Empty the “txp_lang” table,
Reinstall the same language pack from within Textpattern
It works OK.

I can actually see a difference in the stored data in “txp_lang” between the install and post install language packs for the same double byte fields. Its the same data but the coding is different.

The only things I can imagine are:-

1: The installation population of the “txp_lang” table is using a different encoding method than population by installing a language pack post installation. Maybe the application is initialising something the installation script isnt ?
2: Its a different language pack used during Installation than is used post installation. Perhaps one is embedded and the other comes from the web ?

So my installation work around for quotes problem is empty the “txp_lang” table after install and reinstall the language.

Last edited by StFual (2016-10-05 13:38:10)

Bloke · 2016-10-05 14:26:48

StFual wrote #301980:

I can actually see a difference in the stored data in “txp_lang” between the install and post install language packs for the same double byte fields. Its the same data but the coding is different.

Thanks for your continued testing (and patience!)

This sounds intriguing. I’d love to get my hands on some (not all!) of this data so I can compare it side-by-side. Maybe, as you suggest, the installation is doing it differently to the lang injection. Installation uses raw SQL inserts, while any time thereafter it uses the API to install lang packs.

There could be something (line endings? encoding?) in the installed lang pack that your system chokes on for some reason, or the SQL statements might be mangling things, or there could be a bug under certain conditions.

Any chance you could send me some of the data from both installed and post-installed environments? Even via some freebie cloud storage provider would be ace. Thanks.

StFual · 2016-10-06 08:28:07

Thanks for the continued effort to support this. I do understand that this is almost certainly an environment problem however there are only so many times I can ask the hosting company support to change things and i’ve used up my silver bullets there i think. It would be good to know what it is so i can be specific in any request. I do think its worth investigation as Textpattern is a good application, shared hosting users dont have full control over the environment and typically like me dont always have detailed technical knowledge of how to configure the environment.
Id be happy to write a shared hosting install pitfall guide for this if I can make it work.

Just one note before i dump data. If I divide the “txp_lang” table contents as AW “application working” and ANW “application not working” data.

For the “txp_lang” “zh-cn” field (Chinese characters)
ANW data displays correctly using host Pgmyadmin tools suggesting that the intial data load is using environment default coding. This same data appears as garbage in the ANW Textpattern Language drop down.
AW data displays Chinese characters as garbage in Pgmyadmin but correctly in Textpattern.

I can set a temporary area on my site and post the data there or email them to you. Do you just want AW and ANW database dumps or anything else ? As i can’t be confident of what coding export tools will use i’ll include screen prints of what i see.

Textpattern CMS

Textpattern CMS support forum

#37 2016-10-03 12:34:57

Re: Quotes in pasted blocks of text

StFual wrote #301919:

#38 2016-10-04 10:26:18

Re: Quotes in pasted blocks of text

#39 2016-10-05 13:36:40

Re: Quotes in pasted blocks of text

#40 2016-10-05 14:26:48

Re: Quotes in pasted blocks of text

StFual wrote #301980:

#41 2016-10-06 08:28:07

Re: Quotes in pasted blocks of text

Board footer