Languages bundled in core

Bloke · 2017-11-25 02:28:03

phiw13 wrote #307984:

Once she has chosen her preferred UI language, she’ll have no reason to change it again, right ?

I would expect so. This is a minor inconvenience that we could get round if we wanted to, but unless it’s an easy fix I’m not super bothered.

I think the message should read in the “new” language.

I concur. I’ve changed it in my local install but it’ll be bundled up with a whole raft of other tweaks (probably tomorrow now). I’ve just refactored a load of language-related code and think I’ve managed to eradicate one more global in the process: $textarray. Yay! Just gotta test if setup still works…

makss · 2017-11-25 10:59:16

Bloke wrote #307982:

Short of refreshing the page twice when you change admin language, or committing the language change via AJAX and then refreshing the page

Attempt without using AJAX

I will try to solve the issue of language packs of plugins(no existing translations) next week.

Bloke · 2017-11-26 15:17:17

Thanks, makss.

Everyone, please check out the refactored lang handling process in the lang-tweaks branch. This is an attempt to encapsulate the lang strings a little further by removing the reliance on the global $textarray and making strings a class variable of L10n\Lang.php.

I had to introduce a way for things like setup to push its own strings to the class so that it could load its strings from file and inject them into the class. We could probably do this by adding a loadFromFile() method (or similar) and use the refactored methods in the class a little more ingeniously. Maybe someone can improve on what’s there now?

But as it stands it seems to work fine and I’m happy to see the end of another global variable. Please everyone, test it – including setup. If there are no objections to this workflow in a few days I’ll merge it to dev.

Side note: this doesn’t have makss’ recent patch for the double-refresh on language switch. Maybe we can build on this lang-tweaks branch to see about fixing that and the plugin language handling?

makss · 2017-11-27 10:59:28

Bloke wrote #308007:

Everyone, please check out the refactored lang handling process in the lang-tweaks branch. This is an attempt to encapsulate the lang strings a little further by removing the reliance on the global $textarray and making strings a class variable of L10n\Lang.php.

Great idea! I tested everything works fine, can merge with dev.

Side note: this doesn’t have makss’ recent patch for the double-refresh on language switch. Maybe we can build on this lang-tweaks branch to see about fixing that and the plugin language handling?

As you will be more comfortable, my patches can make the code not stable for several days.

Bloke · 2017-11-27 11:54:15

Thanks for testing, makss. I’ll merge this to dev today, then you can merge it and play with the plugin/language thing in your branch.

EDIT: I’ve made one other enhancement to the Languages panel in this branch: a count of the number of other users that are using a language as their UI. This is so that admins know that deleting a language has fallout for the given list of users. Clicking the number takes you to the Users panel and lists the affected users, but I’m not sure this is such a hot idea as it might break the GET params bombshelter if there are hundreds of users. Experimental. Thoughts?

Last edited by Bloke (2017-11-27 12:43:28)

Bloke · 2017-11-27 11:57:44

makss wrote #308018:

my patches can make the code not stable for several days.

Mine too :-) That’s why we test features in branches!

It’s especially important to try and keep the dev branch reasonably clean/usable now, given that Pete’s offering it live every few hours thanks to your awesome auto-installer.

makss · 2017-11-29 14:57:48

It seems it turned out to solve the problem with multilingual plugin textpack.
I invite everyone to test this

phiw13 · 2017-11-29 23:08:57

Yesterday, I fetched a DB-dump from one of my live sites (running 4.6.2) which has 3 languages installed: English (British) (default), French and Farsi. French was used for a while for a second author through Wet’s plugin.

I installed that locally (4.6. code base), after a quick check that things were working (fixing the various paths in Preferences), I updated to 4.7-dev. That went very well, site continued to function as before.

But one thing puzzled me. When I checked the Languages panel, English (British) was flagged as in need of an update (expected, I think), but the two other languages were flagged as good-to-go.

I should note that the two secondary languages were not up-to-date on the live server.

Bloke · 2017-11-30 00:08:17

phiw13 wrote #308049:

When I checked the Languages panel, English (British) was flagged as in need of an update (expected, I think), but the two other languages were flagged as good-to-go.

Two possible reasons for this:

1. You have plugin strings in your database that were installed before the file timestamp of the `en-gb` file. So when it checked the file timestamp and found it was newer than the oldest string, it flags it as updateable.

2. Conversely, you don’t have any plugin strings for other two languages, and/or these two had database timestamps that were newer than the timestamp of their respective language files.

This brings up an interesting point/opportunity. For the first time in Textpattern’s history, we have the ability to perform an actual update of all installed languages during upgrade. Rather than relying on the nebulous RPC server which had all strings from time immemorial and we coudn’t hit automatically during upgrade, we now know that all strings bundled with the release are the ones that the system needs and that any untranslated strings will be covered by fallbacks, as far as practicable.

Thus we could take the step of calling installFile() on all currently-installed languages in the _update.php file after all the _to_4.x.y.php files have run.

It means one less panel to visit after upgrade – especially since we now land on Diagnostics – and I don’t think there’s a downside or any chance of killing anyone’s language experience. Is there?

phiw13 · 2017-11-30 00:39:52

Bloke wrote #308050:

Two possible reasons for this:

1. You have plugin strings in your database that were installed before the file timestamp of the `en-gb` file. So when it checked the file timestamp and found it was newer than the oldest string, it flags it as updateable.

2. Conversely, you don’t have any plugin strings for other two languages, and/or these two had database timestamps that were newer than the timestamp of their respective language files.

(1) is certainly possible, as I haven’t installed / upgraded any plugins on that site for a while (nothing newer available), and the en-gb has seen (recentish) some updates. (2) I’m not sure.

It is still a confusing experience. I switched the UI-language to French, and sure enough, a raw string for User language. I’ve since deleted and reinstalled the two languages.

This brings up an interesting point/opportunity. For the first time in Textpattern’s history, we have the ability to perform an actual update of all installed languages during upgrade. Rather than relying on the nebulous RPC server which had all strings from time immemorial and we coudn’t hit automatically during upgrade, we now know that all strings bundled with the release are the ones that the system needs and that any untranslated strings will be covered by fallbacks, as far as practicable.

Thus we could take the step of calling installFile() on all currently-installed languages in the _update.php file after all the _to_4.x.y.php files have run.

Oh yes, yes. If that is possible, it would avoid the confusion (and the raw string I mentioned above). I’ll keep the DB-dump for that site, and another personal site that I plan to upgrade to 4.7-dev soon so I can test if needed.

It means one less panel to visit after upgrade – especially since we now land on Diagnostics – and I don’t think there’s a downside or any chance of killing anyone’s language experience. Is there?

As updating the language doesn’t blow away any plugin strings that might be available, I don’t see any downside.

phiw13 · 2017-12-13 07:43:10

@ bloke,

If I understand correctly, you latest two commits today are supposed to automatically update installed languages, right? That worked fine for an update dev -> dev, as far as I can tell.

Does it also update the installed languages when moving from TXP-4.6.2 -> dev? If yes, then this failed on my side. My test install has 3 languages (default is en-gb, additional: fa and fr). The default language was updated correctly (I think, I haven’t scrutinised everything), but the two other languages were not updated – quick test: switch to another UI language, it shows active_language_ui, raw string. But the color of the tiles on the languages panel tell me they are up to date (green). That one puzzles me.

When I look at the lang table in the DB, all strings have todays date for lastmod. Searching for active_language_ui in the DB only returns an entry for en-gb. Yes French should show up for that one. It is not yet translated for fa, and doesn’t fetch the en-gb one.

Bloke · 2017-12-13 09:52:08

phiw13 wrote #308277:

supposed to automatically update installed languages, right?

Right. But I’ve been trialling something out on dev (naughty) to fix a whole bunch of issues after Oleg improved the upgrade process.

Basically, when we landed on a known endpoint after upgrade, it was easy. We could update the languages, redirect to the known point (which flushes all the cached stuff on the page), hide the Diagnostics textarea for prettiness but leave any upgrade messages there at the top. Life was good.

Now we can land on any panel, the problem is that there’s loads of stale content left on the page after upgrade. The process is:

Bootstrap index.php.
Load languages, populate globals and stuff because, as far as Txp is concerned, it’s a regular page view.
Detect it’s an upgrade, divert.
Perform update scripts and DB wrangling, including language updating.
Carry on loading the same page we were originally on.

Thus the information that was loaded before the update is still there on the page. Same installed languages (with outdated lastmod times) so if you’re on the Languages panel you see ‘needs update’ yellow boxes, but if you refresh the page, they immediately turn green because they’ve actually been updated in the DB but the page didn’t get the memo.

Crucially, the new language strings – including the nice ‘new version of Textpattern has been installed’ message – are installed but NOT available on the page on that first page load after upgrade, so the message you see isn’t friendly at all: raw string. This is the same for everything. Old strings everywhere until you click away.

What I was trying to do – and it’s only been partially successful so far – is to update the languages in the DB table and then reload them onto the page so we get them all straight away. Sounds easy, right? It isn’t. At least, I didn’t want to go through the pain – and it is a lot of slow queries – to upgrade the languages and then have to go back and reload the one that was active (or, in the case of 4.6.2 -> 4.7.0-dev, the one that was previously in use on the public site). The upgrade process is slow enough as it is with many languages installed and I didn’t want to add another half-second or so to flush everything and reload from the DB.

We need to be clever here. Reduce the number of queries required to update a language, and introduce some mechanism (preferably without resorting to a refresh, but it might come to that) to update the stale, cached info on the page immediately after upgrade.

There are still issues with regularly installing a language too. If you add a new language, all plugin strings are reloaded from the plugin’s Textpack except the new language you just installed. I changed the code last night to try and fix that, but it was 2am and I wasn’t seeing straight. Maybe with a fresh head on me today I can figure something out. Or Oleg/makss can bail me out of this hole we’ve dug!

Thanks for your patience while we tinker. I should have done this in the lang-on-demand branch because that needs merging in before we go much further, but it requires new Crowdin strings so I was waiting for Nicolas to finish up with Themes so we can do all the strings at once instead of catching up later.

Textpattern CMS

Textpattern CMS support forum

#25 2017-11-25 02:28:03

Re: Languages bundled in core

phiw13 wrote #307984:

#26 2017-11-25 10:59:16

Re: Languages bundled in core

Bloke wrote #307982:

#27 2017-11-26 15:17:17

Re: Languages bundled in core

#28 2017-11-27 10:59:28

Re: Languages bundled in core

Bloke wrote #308007:

#29 2017-11-27 11:54:15

Re: Languages bundled in core

#30 2017-11-27 11:57:44

Re: Languages bundled in core

makss wrote #308018:

#31 2017-11-29 14:57:48

Re: Languages bundled in core

#32 2017-11-29 23:08:57

Re: Languages bundled in core

#33 2017-11-30 00:08:17

Re: Languages bundled in core

phiw13 wrote #308049:

#34 2017-11-30 00:39:52

Re: Languages bundled in core

Bloke wrote #308050:

#35 2017-12-13 07:43:10

Re: Languages bundled in core

#36 2017-12-13 09:52:08

Re: Languages bundled in core

phiw13 wrote #308277:

Board footer