multilingual sites

colak · 2025-03-29 15:14:39

Following the idea on forum.textpattern.com/viewtopic.php?pid=339403, the valuable responses, and the need for detailed discussion regarding multilingual sites, I thought that it would be prudent to start a thread where possible future options could be discussed and analysed.

The discussion could include the role of, and changes in

Sections
Tags
Articles
Caption fields of images
Categories
URLs
Title and description fields
Search functionality
or anything else that may be needed.

Being fully aware that this is not an idea that can happen in the immediate future, here are some thoughts.

Multi-lingual articles could reside under the same id, using different title, article, and excerpt fields. This could be achieved using horizontal tabs each one named after the initials of the languages to use. Languages could be defined under the preferences, with a simple comma separated list. A default language could be also defined in the prefs, and the article url could reflect that. This idea could run through in the individual image pane and the individual file one.

Search functionality should limit the results to a particular language.

Category titles could have their translations but the user defines the default language (as is now).

Urls. This is where it gets interesting. When a user defines an extra language, the language initials, ie hr could be included in the urls after the site_url. ie www.site.tld/hr/.... This could be for the home page, section landing pages, category(ies) landing pages, articles, etc, depending on what URL pattern is in use. Messy urls? not sure, but I’m sure that there is a standard method:)

Advantages. Each article and its alternate language(s) resides in one “write” pane. The same goes with section and category titles, etc. No linking of ids which can be very complex and confusing. Multilingual sites could work seamlessly out of the box without and acrobatics by their designers.

Tags could have their lang specific lang and hreflang attributes. The issue remains with constructions such as <html lang="<txp:lang />" dir="<txp:text item="lang_dir" />"> that may need an evaluation of the url to change the lang. Any idea of how the lang_dir could change?

The above of course are just ideas for the future of txp, based both on my experience with txp, and the web. Please feel to attack, enhance, deconstruct, or just agree with them.

jakob · 2025-03-29 16:22:09

It’s complex because multilingual sites can be made in different ways, but certainly a valid topic…

colak wrote #339411:

Multi-lingual articles could reside under the same id, using different title, article, and excerpt fields. This could be achieved using horizontal tabs each one named after the initials of the languages to use. Languages could be defined under the preferences, with a simple comma separated list.

On the UI side of things, Twill CMS take a similar(-ish) approach with a toggle switch (rather than tabs) to switch the fields between different languages. The fields are then swapped out and are given a language label for clarity. In the sidebar dropdown, you can mark if a language version exists. You can try it out in their online demo to see it in action. Quite neat, I think.

colak · 2025-03-30 03:43:51

jakob wrote #339412:

It’s complex because multilingual sites can be made in different ways, but certainly a valid topic…

On the UI side of things, Twill CMS take a similar(-ish) approach with a toggle switch (rather than tabs) to switch the fields between different languages. The fields are then swapped out and are given a language label for clarity. In the sidebar dropdown, you can mark if a language version exists. You can try it out in their online demo to see it in action. Quite neat, I think.

Although their demo is very limited, this appears to be a possible way for us to start thinking about it.

The reason I suggested tabs is because they do not need reloading, so a post can be saved in two or more languages at once.

TheEric · 2025-03-30 07:07:18

I would love a version of this that doesn’t have this built in. It’s bloat for something that isn’t used often.

phiw13 · 2025-03-30 07:40:17

colak wrote #339413:

The reason I suggested tabs is because they do not need reloading, so a post can be saved in two or more languages at once.

Hmm, on that. I’m always a little wary and worried about saving “hidden” content. To easy to make mistakes.

I like your “tab” idea though.

Bloke · 2025-03-30 08:56:46

It’s fairly easy to add this without making it bloated or interfering with monoglot sites. Splitting our tables into pairs works. One holds the immutable stuff like ID, section, probably blogid. One record per article set.

The second table has all the content in it. Body text, title, blah blah and one extra column for the language designator. Joining these two tables gives you 1 row in single-language sites and multiple rows with the same ID in multilingual sites. The interface can be drawn accordingly adding tabs or a dropdown of languages or whatever.

Wouldn’t look any different for single langauge sites. Feed updates still work. Custom fields link to one of the tables. It all just works. Similar approach with other content types. And even the section table. Move the title to its one table. Link (maybe as a View). Job done.

The only bit that requires thought is what to do with URLs. Installing a second language potentially changes irks to add the /lang/section/title. Maybe the primary language never has a language designator URL? And if you change the primary language, well, you take the hit on the URL changes. Not sure.

phiw13 · 2025-03-30 09:21:20

The only bit that requires thought is what to do with URLs. Installing a second language potentially changes irks to add the /lang/section/title. Maybe the primary language never has a language designator URL? And if you change the primary language, well, you take the hit on the URL changes. Not sure.

I don’t think installing/enabling a second language should automatically trigger/imply there is a translation (a lang/section/title or whatever URL scheme). I think a setting might be nicer or safer.
the primary language should (? I think yes) use its own lang. designation (quite a few sites of global Japanese corporations land you first on a region selection page, e.g muji.com). That avoids problems later, unless you’re a top-class artist with redirects when changing.

Thought: possibility of having main site at example.tld, French site at fr.example.tld and so on, a pattern I regularly see.

Bloke · 2025-03-30 09:34:57

There is a slight other wrinkle. Does posted/modified/lastmodID hang off the translation table or is the same date stamp and last modded by author the same for all translations?

My gut feel is that, yes, each rendition has its own publication dates and owner/updater id. But that has a knock on effect as phiw13 touched on above: saving.

In a tabbed situation, a Save would trigger saving all content even if it was hidden behind a tab. Not sure I like that.

A dropdown, however, could trigger a page reload which loads the rendition in (bonus points for showing the original language content somehow, to aid translation). That means a save would save only that rendition, which I think is a safer option, and allows each rendition to have its own posted/modified/etc data.

Bloke · 2025-03-30 09:39:18

phiw13 wrote #339417:

I don’t think installing/enabling a second language should automatically trigger/imply there is a translation

I agree. It’s fraught with difficulty and may only be for users of the back-end anyway.

the primary language should (? I think yes) use its own lang. designation

Noted. That makes sense.

possibility of having main site at example.tld, French site at fr.example.tld and so on

Would be nice but it’s server reliant. Not everyone has access to subdomains, and it’s an additional overhead for admins to set up site configuration.

If there’s a way we could play nicely with subdomains per language, that’d be a bonus. But I don’t think we’d be able to offer it out of the box by default due to the config overhead and the fact we (the Textpattern installation) doesn’t have access to a user’s control panel to set up domains and redirects etc.

colak · 2025-03-31 03:01:30

Bloke wrote #339418:

There is a slight other wrinkle. Does posted/modified/lastmodID hang off the translation table or is the same date stamp and last modded by author the same for all translations?

The initial idea of the “tabs” included all the “write tab” fields for each language.

As such this would be the case also with some custom_fields. If a CF is used for images, no problem (except for caption, and alt text which will also require attention), but if the field is used for textual input, such as a street address, then the particular field could also be ‘duplicated’ in the table in the same way the title/article/excerpt fields will.

For starters posted/modified/lastmodID could be the same as the default lang article but we know that in a few years users will be complaining, as those stamps can also be language specific, but also an article may be posted first and its translation in hours, days, weeks later.

TheEric wrote #339414:

I would love a version of this that doesn’t have this built in. It’s bloat for something that isn’t used often.

I imagine it to be similar to custom_fields. Txp comes with two already in place but more can be added in the admin tab.

Default installs will not have it until the site admin decides to go multi-lingual.

Multi-lingual sites may not be many in the txp community because of the lack of proper ways of implementing them. Note that this forum is populated with people who do not have english as their first language.

phiw13 wrote #339415:

colak wrote #339413:

Hmm, on that. I’m always a little wary and worried about saving “hidden” content. To easy to make mistakes.

I hear you.

Bloke wrote #339418:

A dropdown, however, could trigger a page reload which loads the rendition in (bonus points for showing the original language content somehow, to aid translation).

yes!!!

abutcher · 2025-04-10 13:52:37

Is multilingual support in the Textpattern core a goal for the near future?

I have been asked recently for a multilingual site (10 languages) and for this kind of project I would prefer to use Textpattern, but managing multilingual articles is a real pain for authors. I saw that the MLP plugin is broken/abandoned, and honestly I agree that the reason why you don’t see many multilingual sites on Textpattern is because there’s no clear path.

I believe it’s better to have an implementation of this in the core, especially because the impact is large, especially considering things like search and article URL patterns. As long as it’s tactfully implemented, sites that only use one language won’t see any impact, and there would be no penalty for sites that would use it.

There are additional concerns here when dealing with internationalisation, including support for locale switching when generating dates, times, numbers, etc. It would be important to know how far this would go, i.e. support for translations vs. support for true localisation.

I think Bloke is on the right track – split the textpattern table so the content is in a separate table from the identifiers and common fields. If compatibility is such a big issue, put all non-default language content into a separate table, but the solution gets uglier at that point and is more akin to a plugin’s solution.

I think it would be easy to support subdomains, but the another very common case is example.com/pagename for the default language, example.com/langcode/pagename for other languages. It would be necessary to have an easy way to output alternative links to the page in other languages (<link rel="alternate" hreflang="langcode" href="/langcode/pagename"), which is why it would be important to link the articles together with the same ID.

I think the last modified/published time/etc need to be on the content level, with the option to take a “latest modified” time if you wanted it to be the same for all languages. For caching/feeds/search engines/etc, only the content level timestamps are of interest anyway.

I think publication status should be on the content level, with a fall-back if the content is not published. This allows for phased translation of sites and working with a team to provide translations, or even sites where only core pages are translated.

I don’t think adding a language in the back-end automatically implies it should be enabled in the front-end.

I think tabs to switch between language versions on the article edit page is a good way to go if there are a small number of tabs. The important thing would be to clearly see which languages are available, which have content, and which have been published, so it’s easy to understand the status of an article across all languages.

Likewise, I think it would be important to allow easy switching between languages on the same article without having to saving before switching. It’s often necessary to make a minor change on all language versions, and while translating it’s common to need to reference existing translations to check for consistent phrasing/tone.

Sorry for the long post :)

Bloke · 2025-04-10 16:10:42

That’s all good stuff thanks for the feedback. I agree with all your points. I’m not sure on locale switching. That’s tricky when cooked content like Textile is already pre-rendered, but since we know the target language at save time, it might be possible to ensure it’s encoded in the correct locale if it’s marked up correctly in the source (and Textile supports it!).

The only point I’m now reversing is the tabs for each language. That does not fit with the idea of separate posted/modified, author, publication status, etc. Each translation needs to be loaded independently and the UX needs to be clear that if you make a change to one, you are only saving that rendition. With tabs, that is unclear.

So a selector is, imo, the way to go. Hadn’t thought about showing translation status and stuff, but that’s a good idea. Not sure how to display that for best in a select list. Ideas welcome.

As far as implementation goes, we’re a bit stuck at the moment. We’ve applied for funding to bring this to Textpattern under the banner of being the first fully multilingual native (plugins not necessary) European CMS that helps more people publish more to a wider audience. We’re awaiting a decision from the funding panel.

If we get it, that will be a HUGE boost to bringing this to reality within a relatively short timescale (6 months to a year). But we can’t start work on it until the funding decision has been made one way or the other, or it nullifies our application. And that may take until June before we even know if we’ve made it to the second round.

We’ll still press on with all the other good bits in Textpattern 5, and whether the funding comes through or not, I’m confident we can pull it off. We have prior form for being the first CMS to do something natively innovative (dev lanes for eradicating the staging server is one example), and we’ve drafted a project plan to manage the migration and develop the functionality.

If we don’t get through to round 2, I’m happy to start as soon as we know this and we’ll plod towards a solution (help is always appreciated!). But if we do, I doubt dev will start until later in the year, if that helps your decision.

Textpattern CMS

Textpattern CMS support forum

#1 2025-03-29 15:14:39