Trimming the lang directory for byte savings

gaekwad · 2024-09-09 21:47:51

I have automated backups for my Textpattern sites, and I leave the core files intact to avoid any problems when restoring. I’m setting up a new Textpattern site for development over winter, and I’m building out the backup scaffold + schedule before dev work commences. The site will be a single tenant initially, and future collaborators will very likely use ‘English (British)’. This means – in theory, at least – that the other language packs are surplus.

Until tonight , I’d never deleted a language pack before. Since the site is new, completely vanilla, and essentially rebuildable from scratch quickly & easily, I figured I could test what happens when unused language packs are removed from the file system. So, I did. Nothing exploded, I’m pleased to report

The reduction in overall bytes used is proportionally significant, especially when the files are compressed as part of the backup process

Note: I’m not saying you should remove unused language packs, I’m saying I did and it made a big difference. This is the tree output:

[2.0M]  /mnt/diamond_backups_01/backups/phpbu/***.**/@/20-daily/files
├── [1.3M]  @.***.**_files_20-daily_20240909215345.tar.xz
├── [780K]  @.***.**_files_20-daily_20240909220356.tar.xz

A vanilla Textpattern 4.9.0 beta 1 with an empty theme scaffold and the standard language packs compresses to a 1.3MB .xz’d tarball archive. That same site with the language packs trimmed to just ‘English (British)’ compresses to 780KB, which is a reduction in file size of around 40%.

Bloke · 2024-09-09 23:31:23

When we bundled the language packs with the download instead of relying on the annoying RPC server, the bundle byte count did jump significantly.

There is absolutely nothing preventing you from removing language packs from the lang directory, as long as you don’t remove the in-use one(s). All you’ll see is a reduced set of ‘available’ languages from the Admin>Languages panel.

It’s a feature :)

Pat64 · 2024-09-10 05:49:47

Maybe what Pete suggest: the ability to remove some languages we don’t need directly from the languages page (the admin interface)?

phiw13 · 2024-09-10 07:34:18

Pat64 wrote #337812:

Maybe what Pete suggest: the ability to remove some languages we don’t need directly from the languages page (the admin interface)?

But then you might want the ability to re-fetch the deleted language pack(s) in case a new user logs in who’d like that deleted language?

@ gaekwad (Pete)

What do you actually gain, apart from space saving on the server (/lang/ is aprox. 3.5Mb, English language pack about 250kb, I think) and smaller backups? Performance should not be affected, I would think.

Bloke · 2024-09-10 07:48:55

phiw13 wrote #337813:

But then you might want the ability to re-fetch the deleted language pack(s) in case a new user logs in who’d like that deleted language?

This.

etc · 2024-09-10 08:29:02

phiw13 wrote #337813:

What do you actually gain, apart from space saving on the server (/lang/ is aprox. 3.5Mb, English language pack about 250kb, I think) and smaller backups? Performance should not be affected, I would think.

You’d save few milliseconds when listing available languages on their tab, but the gain is marginal.

I’m all for reducing the waste of space/time, and the language packs would be an easy win. But, when distributing txp, we do not know what languages are needed and have to include them all. Afterwards, the unused ones can be moved to/from a separate folder, but this process is manual atm.

I guess the idea of a remote server has been discussed somewhere..

Bloke · 2024-09-10 09:08:19

etc wrote #337818:

I guess the idea of a remote server has been discussed somewhere..

Hahaha 😎

Been there done that, shredded the T-shirt.

gaekwad · 2024-09-10 11:19:40

Bloke wrote #337811:

There is absolutely nothing preventing you from removing language packs from the lang directory, as long as you don’t remove the in-use one(s)

[…]

It’s a feature :)

And it’s a welcome feature! I realise I’m an edge case (in so many ways…), and knowing that I can remove the languages that I will never use is ‘safe’ and won’t clobber anything is really useful.

None of this discussion was intended to be a jab at current practices. Literally – I have loads of backups and they’re distributed with SyncThing to other computers, so saving a few MB here and there soon adds up.

The Textpattern forum database is backed up hourly. It runs to about 80MB per hour, with a retention of 48 hours of backups. My self-hosted time tracking app is backed up daily and that chomps 67MB a day. It might sound trivial with MB quantities on servers dealing in GB, but it’s less I have to pay for, fewer bytes to throw down the internet tubes with SyncThing, and faster to restore when something goes wrong. I have nightmares about restoring Magento sites when something breaks, the complexity is off the charts!

gaekwad · 2024-09-10 11:20:37

Pat64 wrote #337812:

Maybe what Pete suggest: the ability to remove some languages we don’t need directly from the languages page (the admin interface)?

Voila!

gaekwad · 2024-09-10 11:22:43

phiw13 wrote #337813:

But then you might want the ability to re-fetch the deleted language pack(s) in case a new user logs in who’d like that deleted language?

True. I was careful to mention my case is initially single-tenant Textpattern, and is likely to stay that way for some time. I know the other potential collaborator, and we both use English as a first language, so the requirements are met already.

What do you actually gain, apart from space saving on the server (/lang/ is aprox. 3.5Mb, English language pack about 250kb, I think) and smaller backups? Performance should not be affected, I would think.

Smaller backups. That’s exactly it.

gaekwad · 2024-09-10 11:24:18

etc wrote #337818:

You’d save few milliseconds when listing available languages on their tab, but the gain is marginal.

Exactly. I wasn’t doing this for any UI / UX gain, at least not consciously so. Literally smaller backups.

I guess the idea of a remote server has been discussed somewhere..

I’ll add it to the Textpattern 5 considerations thread.

Textpattern CMS

Textpattern CMS support forum

#1 2024-09-09 21:47:51