Languages bundled in core

makss · 2017-11-29 14:57:48

It seems it turned out to solve the problem with multilingual plugin textpack.
I invite everyone to test this

phiw13 · 2017-11-29 23:08:57

Yesterday, I fetched a DB-dump from one of my live sites (running 4.6.2) which has 3 languages installed: English (British) (default), French and Farsi. French was used for a while for a second author through Wet’s plugin.

I installed that locally (4.6. code base), after a quick check that things were working (fixing the various paths in Preferences), I updated to 4.7-dev. That went very well, site continued to function as before.

But one thing puzzled me. When I checked the Languages panel, English (British) was flagged as in need of an update (expected, I think), but the two other languages were flagged as good-to-go.

I should note that the two secondary languages were not up-to-date on the live server.

Bloke · 2017-11-30 00:08:17

phiw13 wrote #308049:

When I checked the Languages panel, English (British) was flagged as in need of an update (expected, I think), but the two other languages were flagged as good-to-go.

Two possible reasons for this:

1. You have plugin strings in your database that were installed before the file timestamp of the `en-gb` file. So when it checked the file timestamp and found it was newer than the oldest string, it flags it as updateable.

2. Conversely, you don’t have any plugin strings for other two languages, and/or these two had database timestamps that were newer than the timestamp of their respective language files.

This brings up an interesting point/opportunity. For the first time in Textpattern’s history, we have the ability to perform an actual update of all installed languages during upgrade. Rather than relying on the nebulous RPC server which had all strings from time immemorial and we coudn’t hit automatically during upgrade, we now know that all strings bundled with the release are the ones that the system needs and that any untranslated strings will be covered by fallbacks, as far as practicable.

Thus we could take the step of calling installFile() on all currently-installed languages in the _update.php file after all the _to_4.x.y.php files have run.

It means one less panel to visit after upgrade – especially since we now land on Diagnostics – and I don’t think there’s a downside or any chance of killing anyone’s language experience. Is there?

phiw13 · 2017-11-30 00:39:52

Bloke wrote #308050:

Two possible reasons for this:

1. You have plugin strings in your database that were installed before the file timestamp of the `en-gb` file. So when it checked the file timestamp and found it was newer than the oldest string, it flags it as updateable.

2. Conversely, you don’t have any plugin strings for other two languages, and/or these two had database timestamps that were newer than the timestamp of their respective language files.

(1) is certainly possible, as I haven’t installed / upgraded any plugins on that site for a while (nothing newer available), and the en-gb has seen (recentish) some updates. (2) I’m not sure.

It is still a confusing experience. I switched the UI-language to French, and sure enough, a raw string for User language. I’ve since deleted and reinstalled the two languages.

This brings up an interesting point/opportunity. For the first time in Textpattern’s history, we have the ability to perform an actual update of all installed languages during upgrade. Rather than relying on the nebulous RPC server which had all strings from time immemorial and we coudn’t hit automatically during upgrade, we now know that all strings bundled with the release are the ones that the system needs and that any untranslated strings will be covered by fallbacks, as far as practicable.

Thus we could take the step of calling installFile() on all currently-installed languages in the _update.php file after all the _to_4.x.y.php files have run.

Oh yes, yes. If that is possible, it would avoid the confusion (and the raw string I mentioned above). I’ll keep the DB-dump for that site, and another personal site that I plan to upgrade to 4.7-dev soon so I can test if needed.

It means one less panel to visit after upgrade – especially since we now land on Diagnostics – and I don’t think there’s a downside or any chance of killing anyone’s language experience. Is there?

As updating the language doesn’t blow away any plugin strings that might be available, I don’t see any downside.

phiw13 · 2017-12-13 07:43:10

@ bloke,

If I understand correctly, you latest two commits today are supposed to automatically update installed languages, right? That worked fine for an update dev -> dev, as far as I can tell.

Does it also update the installed languages when moving from TXP-4.6.2 -> dev? If yes, then this failed on my side. My test install has 3 languages (default is en-gb, additional: fa and fr). The default language was updated correctly (I think, I haven’t scrutinised everything), but the two other languages were not updated – quick test: switch to another UI language, it shows active_language_ui, raw string. But the color of the tiles on the languages panel tell me they are up to date (green). That one puzzles me.

When I look at the lang table in the DB, all strings have todays date for lastmod. Searching for active_language_ui in the DB only returns an entry for en-gb. Yes French should show up for that one. It is not yet translated for fa, and doesn’t fetch the en-gb one.

Bloke · 2017-12-13 09:52:08

phiw13 wrote #308277:

supposed to automatically update installed languages, right?

Right. But I’ve been trialling something out on dev (naughty) to fix a whole bunch of issues after Oleg improved the upgrade process.

Basically, when we landed on a known endpoint after upgrade, it was easy. We could update the languages, redirect to the known point (which flushes all the cached stuff on the page), hide the Diagnostics textarea for prettiness but leave any upgrade messages there at the top. Life was good.

Now we can land on any panel, the problem is that there’s loads of stale content left on the page after upgrade. The process is:

Bootstrap index.php.
Load languages, populate globals and stuff because, as far as Txp is concerned, it’s a regular page view.
Detect it’s an upgrade, divert.
Perform update scripts and DB wrangling, including language updating.
Carry on loading the same page we were originally on.

Thus the information that was loaded before the update is still there on the page. Same installed languages (with outdated lastmod times) so if you’re on the Languages panel you see ‘needs update’ yellow boxes, but if you refresh the page, they immediately turn green because they’ve actually been updated in the DB but the page didn’t get the memo.

Crucially, the new language strings – including the nice ‘new version of Textpattern has been installed’ message – are installed but NOT available on the page on that first page load after upgrade, so the message you see isn’t friendly at all: raw string. This is the same for everything. Old strings everywhere until you click away.

What I was trying to do – and it’s only been partially successful so far – is to update the languages in the DB table and then reload them onto the page so we get them all straight away. Sounds easy, right? It isn’t. At least, I didn’t want to go through the pain – and it is a lot of slow queries – to upgrade the languages and then have to go back and reload the one that was active (or, in the case of 4.6.2 -> 4.7.0-dev, the one that was previously in use on the public site). The upgrade process is slow enough as it is with many languages installed and I didn’t want to add another half-second or so to flush everything and reload from the DB.

We need to be clever here. Reduce the number of queries required to update a language, and introduce some mechanism (preferably without resorting to a refresh, but it might come to that) to update the stale, cached info on the page immediately after upgrade.

There are still issues with regularly installing a language too. If you add a new language, all plugin strings are reloaded from the plugin’s Textpack except the new language you just installed. I changed the code last night to try and fix that, but it was 2am and I wasn’t seeing straight. Maybe with a fresh head on me today I can figure something out. Or Oleg/makss can bail me out of this hole we’ve dug!

Thanks for your patience while we tinker. I should have done this in the lang-on-demand branch because that needs merging in before we go much further, but it requires new Crowdin strings so I was waiting for Nicolas to finish up with Themes so we can do all the strings at once instead of catching up later.

jakob · 2017-12-13 09:59:35

hide the Diagnostics textarea for prettiness

Ah, so what I have been observing recently is deliberate. It makes sense, I just wondered if I had a glitch because it is different to how it used to be.

Bloke · 2017-12-13 10:05:48

jakob wrote #308285:

Ah, so what I have been observing recently is deliberate. It makes sense, I just wondered if I had a glitch because it is different to how it used to be.

That was a holdover from when we forced you to land on Diagnostics. It won’t do that any more as I’ve reinstated the full view no matter what happens before it.

phiw13 · 2017-12-13 23:47:09

Bloke wrote #308282:

Thanks for explaining the state of things. And don’t worry, I am patient (and I’m under no illusion the that stuff is easy).

phiw13 · 2017-12-22 00:01:55

What is the current status regarding updating languages on Version update?

As far as I can tell, an update dev -> dev kinda works, but only when update.php runs? IIRC, a couple of days ago, there was an update to fa.ini and the language was updated transparently and automatically. Earlier today I updated a dev install that was 2 days behind, there was an update to fr.ini but this was not updated automatically (the Languages panel flagged it correctly).

On the other hand, I updated a TXP4.62 site to the latest dev. Default language is en-gb, a couple of other languages installed. Prior to updating, I checked the status of the languages, and updated as needed. Then I swapped all TXP files, let the Updater do its magic. After that I checked the status of the installed languages. The en-gb file is up-to-date; the other languages, although flagged as up-to-date on the Languages panel are not in fact up-to-date.

Textpattern CMS

Textpattern CMS support forum

#31 2017-11-29 14:57:48

Re: Languages bundled in core

#32 2017-11-29 23:08:57

Re: Languages bundled in core

#33 2017-11-30 00:08:17

Re: Languages bundled in core

phiw13 wrote #308049:

#34 2017-11-30 00:39:52

Re: Languages bundled in core

Bloke wrote #308050:

#35 2017-12-13 07:43:10

Re: Languages bundled in core

#36 2017-12-13 09:52:08

Re: Languages bundled in core

phiw13 wrote #308277:

#37 2017-12-13 09:59:35

Re: Languages bundled in core

#38 2017-12-13 10:05:48

Re: Languages bundled in core

jakob wrote #308285:

#39 2017-12-13 23:47:09

Re: Languages bundled in core

Bloke wrote #308282:

#40 2017-12-22 00:01:55

Re: Languages bundled in core

Board footer