Handling multiple image formats

Bloke · 2023-04-16 17:26:07

As we move towards native multi-size image handling in core, we need to figure out the best way to handle image fallback. Currently, if you upload a .webp image, core (and smd_thumbnail) will dutifully make you… a smaller .webp image. Which is fine, but if you want to offer, for example, .jpg as a fallback in your srcset attribute, you need to resort to all sorts of shenanigans uploading multiple formats and linking them via a (currently plugin-based) custom field.

Has anybody any thoughts on how we might best handle multiple formats? I guess we could introduce image prefs that allow you to pick what format(s) your smaller sizes will be created in. So instead of just sizes or pixel densities, you also specify you want jpg, avif and webp. That triples the number of files it creates, which is quite a server impact on upload, so we’d need to tread carefully. And also (as it stands) bloats the /images directory. Still need to find the best way to distribute the images across the filesystem.

In the short term, I could update smd_thumbnail to handle this (how??) but it’d be nice to get some feedback or ideas floating around on how best to handle all this.

Thoughts appreciated.

giz · 2023-04-16 19:37:48

I feel generating the image size variants upon upload of each image is the root of the problem.

Our image resizing script should trigger on-demand only, when it is called via an images tag. Only the requested sizes and formats would be created (no redundant sizes for each image).

SLIR uses this approach – image sizing requirements are specified in a Textpattern form, and generated at runtime. The average admin user can avoid the intricacies of responsive image creation; its handled in the code.

My preferred approach (assuming I had the skills) would be to refactor SLIR for php8. Its other issues mentioned elsewhere on the forum are moot;

hammering the server with multiple SLIR-based url calls eg. /slir/w320/images/1.jpg, /slir/w321/images/1.jpg, /slir/w322/images/1.jpg, etc can be prevented by specifying allowable sizes in slir_config
the 10,000 item folder limit is yet to materialise on a site with 12,000 images, each with 3 size variants. When the limit is eventually reached, it’ll be simple to switch to a 2nd SLIR installation for images with an id greater than 10,000 (or with a little more effort, using the same SLIR install but with new cache folders).

Last edited by giz (2023-04-16 19:38:37)

Bloke · 2023-04-16 20:19:06

All good points, though I’m not so worried about the sizing at the moment as I am the formats.

As you point out (and I’d not thought about, so thank you) on-demand will help here too. It’s more server friendly. If someone visits an <img srcset> tag and their system can show a .webp at 480px wide, generate it and serve it. If the next person visits the same page and their <img srcset> determines they can only see a jpg at 480px, well, generate that for them and serve it. The next visitor benefits from the cached copies of either, so slowdowns only affect the first visitor at each viewport size.

As you say, as long as the system only permits certain sizes to be returned, it’s manageable and isn’t a security risk. Plus, if you change your mind, well, you just get a bunch of images left behind. But that’d be the same in the pre-defined setup too. Blatting the image repo and letting it rebuild itself with your new size requirements is probably faster anyway than worrying about what to keep and what to chuck.

Regarding the file limit per folder, that’s arguably immaterial for the host OS these days. However, there are subtleties. Leaving unused files around is a problem. Ask Pete, who had to diagnose a problem with a server that had terabytes of free space left, but was throwing “can’t create file” errors and was crawling to a halt. The problem? The inode limit, which is shared across all directories and, oftentimes, limited by hosting providers per customer. Delete a bunch of cached files and, boom, sanity is restored. So we do need to be mindful of limiting file creation if we can.

The major problem, as far as I can tell, is the fact that querying 12K, 20K, 50K, etc images in one directory to provide a list of files in, say, an FTP client is ludicrously slow – to the point it can close the connection before the list has been populated. Sure, there’s rsync if your server/local OS is equipped enough to handle it and you’re geeky enough to fashion a call to do it. And if you’re backing up, there are server-to-server or peer-to-peer backup systems for $$ or additional configuration effort.

But all the while files are stored in a remote location and people have a need to down/upload that content to migrate stuff or perform backups, I feel we should do what we can to ease the pain of attaining those files, while keeping the folder structure reasonably logical if possible. That’s what I’m seeking here. A way to minimise the pain of configuring/creating a tonne of files of varying sizes and varying formats per image, with a filesystem layout that helps evenly distribute content while making it reasonably easy to find particular content if any troubleshooting needs to be done.

Bloke · 2023-04-16 20:41:42

If we can find a rapid, lightweight, freely available, modern image generation script, I’d certainly investigate it. Failing that, I’m pretty sure we can leverage Intervention or Imagick to do it, which is what we’re planning to use for everything else. Not tested it for performance yet. Both support WebP but at the time of writing, AVIF support is limited/non-existent unless you’re on PHP 8.1+.

The foundations of image upload and handling and manipulation (crop/rotate/etc, extensible by plugin) are in the txp image branch. It could be further built upon so that admins could fill in the gaps to specify the sizes they will permit (prefs?) and then their <txp:image> tags could take these defined sizes into account to build srcset or picture/source elements. Image generation would be done on the fly.

We can store the original uploaded image (optionally post-processed via manipulation in the Images panel) in a directory structure under our control. Presumably if an image is altered, we need to either delete the cached copy(ies) to avoid them being served stale, or somehow notify the generating script that the image is ‘dirty’ (changed) so it can auto flush and regenerate the cache next time someone wants an image from that file. It might even handle all that on our behalf.

The missing piece of the puzzle is how the script would cache it all, but maybe we could just put our fingers in our ears and go la la la, and hope for the best!

Last edited by Bloke (2023-04-16 21:22:09)

giz · 2023-04-16 21:20:04

I’m used to all the garbage collection (and caching) being handled by my image script, ever since I first installed it back in 2014. I don’t know how it does it, but it handles caching and garbage collection for me; I’ve never had to worry about it.

The script creates files in 2 folders:

cache/
   rendered/
      69d777f01ce76fac77f8ce391f207283
      545c44e5ec5f72edd4584756e96d78c3
      0279790813f31aef0e154cab7bed0687
      a11095f466f57e006a7573849c52cedc
      etc
   request/
      20bf2ec0606bcaaccc7ddc8edbef2a58
      96cc4dca608c785e0e975f564d6a9437
      749a7a44fa24ada11c9b5c8fa5d9ae0e
      300926855770a5f6edd2b3b0bde6aa13
      a35997bd99ddc5082b73b474bba7a786
      etc

When I now visit the rendered folder it has 14793 entries (opened in about 2secs in Nova), the oldest being created on 29/11/22.

Bloke · 2023-04-16 21:30:38

Good to know, thanks. Shame the code is at least 8 years old!

Intervention offers good resizing options and most formats bar AVIF, but we’d probably need to write a wrapper API around it to translate URL params via a sanitizer into something that we can create on the fly and cache (somehow) and then serve.

giz · 2023-04-16 21:34:20

With formats I may be missing something, but I don’t see the problem…

If we upload a ‘master’ image into Textpattern, it will be the highest quality available, whatever format it is in. We want our script tag to spit out this image at multiple sizes and formats. Surely all that is required is for our image script to be capable of transforming .webp images in addition to the usual formats?

Talking of usual formats, I sorely need svg files to be stored as images and not files!

Bloke · 2023-04-16 21:49:09

giz wrote #335314:

Surely all that is required is for our image script to be capable of transforming .webp images in addition to the usual formats?

Yes, exactly. That’s what the image branch does at the moment. You upload a main image and it’s stored but never (ever) used except to generate files used throughout the system. That’s both on the admin side (for displaying image grids/lists, thumbnails etc, and images for your website, which would be generated through your <txp:image> tags. The upload bit is done. The admin-side manipulation bit is getting there. The generation at various sizes isn’t built yet.

The main bottleneck when creating images from the master image is the size of the image itself. If you upload a 14MP image that’s 40MB, Intervention has to load that off disk/SSD into memory, resize it and spit it out. Each time you want a different size. That’s a lot of overhead.

So it’s in your best interests to make your “golden master image” as decent quality as possible to perform the resize successfully to a decent standard, without being silly high quality that it grinds your server to a halt when it encounters a bunch of simultaneous requests for images on one page or from multiple connections.

giz · 2023-04-16 21:50:28

When using SLIR under php 8 it runs without any errors, but if you specify a width in your string it generates a png image Trying to access array offset on value of type null.

Simple to fix, no? ;-)

I experimented with Intervention, but didn’t last long. It smells of alpha-geek to me… Why does it need to be so complex and rely on so many libraries for a relatively simple task?

Bloke · 2023-04-16 21:50:52

giz wrote #335314:

Talking of usual formats, I sorely need svg files to be stored as images and not files!

Yes

Bloke · 2023-04-16 21:57:57

giz wrote #335316:

I experimented with Intervention, but didn’t last long. It smells of alpha-geek to me… Why does it need to be so complex and rely on so many libraries for a relatively simple task?

Haha. I’m not 100% sold on it yet. I pared it down to the bare minimum requirements and it hasn’t bloated the code much. I’m not sure how relatively simple things like rotate, flip, crop, blur, graycale, gamma/colorize/filter etc are, but that’s what we’re using it for. The resize comes along for the ride so we might as well use it rather than add something else.

colak · 2023-04-17 05:05:55

I’m happy the image handling is back on the discussion.

I tend to agree with giz that image resizing and cacheing may be the way to go as it will save the average user from a lot of grief when uploading images.

That said, i also agree with Bloke about the issue of the formats

I would also add the issue of the browser orientation, especially on mobile devices, ie srcset.

I think that the ideal would be if we had a script that it could handle all of the above.

Textpattern CMS

Textpattern CMS support forum

#1 2023-04-16 17:26:07