Multiple image handling: profiles

Bloke · 2020-08-03 09:08:50

As we gear up for 4.9 and beyond, the topic of image handling is once again on my mind.

You may (or may not) know I’m using smd_thumbnail as a sort of proving ground for figuring out the best way to handle multiple “thumbnails” (or actually “images of any size” if we’re being pedantic). The idea is that when we hit the best workflow there, we distill the good bits into core and jettison the rest of the plugin.

Sidestepping the actual storage of different image sizes in the filesystem (which we’ll cover in a separate discussion) I want to find out your thoughts on the general process of handling image creation and, specifically, rules governing it thereof.

smd_thumbnail uses the concept of profiles. Each image size you potentially want to use in your site has a different profile so it knows what sizes to make when you upload an original. Whether this is the best approach is open to interpretation (and I genuinely would love to hear any alternative ideas you may have on this).

The general guide is that you make images for certain use cases and then employ media queries and/or HTML srcset/picture to let you and/or the browser pick the best image size based on environment or size of viewport.

Example: You allow people to upload the full size image from their cameras (because it’s less burden on them) and you have a bunch of profiles like this:

Max: 1600w
Max2x: 3200w
Large: 1280w
Large2x: 2560w
Grid: 420w
Grid2x: 840w
Thumb: 100wx150h
Thumb2x: 200wx300h

Great. Now every time you upload a huge 8000w px image, it’s stored but you also, automatically, get a whole bunch of smaller images made that the site can use in templates. If it makes a mess of resizing, people can always resize them in an image editing program and replace individual thumbs later. Or if particular sizes need to highlight particular areas for art direction purposes.

Problems:

If you upload portrait images, they’re squished in the Thumb and Thumb2x profiles because an explicit size was defined. Creating extra profiles with a different aspect means that there is potentially a lot of wasted space creating portrait images for landscape pics (and vice versa) that will never be used. On image heavy sites, this is annoying.
The system has no way of knowing that Thumb2x is a ‘double density’ image for Retina-type displays. Sure the browser might pick it by virtue of it being close to the correct size the device can handle, but we need more than that if we’re going to size it correctly in CSS so the image isn’t blown up to double its size. Should we instead define a single “CSS pixel” dimension profile and then list which densities we want to create/support in the profile itself? Thus, one ‘profile’ could have multiple images associated with it, each suffixed with “@{density}x” if it’s not 1 (or perhaps for consistency, even if it is 1, so ‘@1x’ is defined).
If I upload a small image like an icon or logo (yeah, yeah, I know the database is not the correct location for presentational image elements but hear me out) then it’s blown up to huge proportions and goes pixelated. Again, useless as we probably don’t need the extra images at all in this case, though we might want to upload, say, a 2x or 4x density icon and have it automatically create scaled down versions for lower resolution displays.

Tricky. A few ideas:

Profile Groups
Define groups of profiles based on usage. So you can pick from, say, ‘landscape’, ‘portrait’ and ‘square’ types (maybe others?) when you create a profile. Or, they (internally) go into the correct group if you specify both w and h dimensions (because we then know, by design, in which group they belong).
Profile rules
A profile is only used on an image if defined conditions are met. Such as w>200. So you write this freeform rule (or pick from a list of available comparisons?) when you make each profile.

Both have benefits and drawbacks in some ways. There may be other ways to approach it (ideas anyone?!). Primarily, this means there are potentially ‘holes’ in the image set because you could drag two images – a landscape and a portrait – and get two different sets of images created for you. Good for disk space saving, but when you come to display those on the site, it makes template code more tricky as you need to detect which images are available.

And what about the icon example? The Groups won’t solve that (unless we can define arbitrary groups AND associated rules at the group level?). The Rules system could potentially work here because you could say that images less than 100px wide only get the ‘Icon2x’ and ‘Icon’ profiles applied. All the others remain unused, which avoids the jaggies.

When it comes to output the pics, we have the new size attribute in <txp:images> which can cater for only displaying certain sizes/aspect ratios of image. That’s nice but doesn’t solve the density issue. Ideas welcome here, please, with a view to the minimum practical considerations we need to address for sensible output of most <img> attributes.

In smd_thumbnail we can iterate over the profiles to create srcset rules like this:

<img itemprop="url contentUrl" src="<txp:image_url />"
   srcset="<txp:smd_thumbnail type="SMD_ACTIVE" break="," trim>
   <txp:smd_thumbnail_info item="url, w" break=" " />w
</txp:smd_thumbnail>"
   sizes="(min-width: 84rem) 1600px,
          (min-width: 69rem) 1280px,
          (min-width: 55rem) 1024px,
          (min-width: 46rem) 864px,
          (min-width: 38rem) 576px,
          100vw"
   alt="<txp:image_info type="alt" />"
   class="image-banner">

(I think) that works to display the correct sizes, but in my tests when you zoom out (CMD and minus) the low-res version is loaded for some reason: I was expecting that to simulate what happens on large displays, wider than my physical screen allows, but maybe I’m being unrealistic here.

Anyway, the SMD_ACTIVE in the code above only iterates over active profiles. In the groups/rules world we would need a way to iterate over active profiles and only pluck out ones that have been used for that image, which adds another layer of complexity as it involves testing image files to see which are defined and stored on disk – and that’s potentially time expensive.

Maybe we store which profiles have been used alongside the metadata in each image, so we can quickly look it up? We’d still need to check if it actually exists on disk when we come to output it, but that’s down to the image rendering portion to gracefully fail or throw a warning, as it does now, if an image is missing. It only burdens the output stage, not the template coder, to build the check in – unless you want to display placeholders or something, in which case, it’s down to you.

Other things to think about:

Art direction (picking a point/region on an image thumb to allow it to scroll/zoom to that area when the image is loaded).
What about the thumbnails shown on the admin side? smd_thumbnail allows you to pick a ‘default’ profile to display here but this is probably too much configuration. If we need to always make a thumbnail, that’s fine. Just what dimensions make sense? And what aspect? And do we make this available as a “fixed” (undeletable) profile name such as “thumbnail” (so there’s less likelihood of someone using the same name for their own profile).
Grid view. I think we’ve decided that we’re going to always create a grid view image, regardless, so we can use it on the admin side if needed. If someone uploads a portrait image and we opt for a landscape grid, do we pad with space? Use CSS to pad it in the grid? Crop? And, again, do we add it to the set of pre-defined profiles and call it “grid” (or something). Maybe someone could use this on their site and, between “thumb” and “grid”, have enough sizes to manage a responsive site without the need for creating more profiles? If so, great!

In short, I’m trying to figure out how to handle this:

in the most sensible manner
without introducing ridiculous levels of configuration
using sensible conventions
that allow us to build flexible image galleries for responsive layouts
without wasting disk space for unneeded sizes

Any thoughts on the above, improvements, features, edge cases I’ve not thought of, limitations, conventions we could adopt, anything that I could try out in smd_thumbnail as a proving ground, please let your voice be heard.

Thank you.

zero · 2020-08-03 10:53:42

My immediate thought is that being able to select Portrait, Landscape or Square when uploading an image would be most intuitive to me. So then, having created desired thumb sizes for each profile, the images are all output in a similar fashion to how you have it now. The designer could more easily specify image sizes than at present, making galleries easier to create. I wouldn’t consider that too much work and it would be easier than at present. Too much automation might also add to confusion rather than reduce it — too many choices, too much to think about? But I only occasionally work on image-heavy sites.

Bloke · 2020-08-03 11:15:11

Yes, part of the issue here is that smd_thumbnail requires endless configuration for no actual gain. It simply adds complexity to have to create new profiles for different ratios or different pixel densities.

So I’m looking for a better method to support and define that without all the faff.

Vienuolis · 2020-08-03 13:28:26

The first desire from my experience using txp:image (and txp:smd_thumbnail, txp:images, etc.) is not to name them full-size pictures by default, because they are never used on a webpage in its full size. I would like to set images on a page with no attributes for code simplicity and lesser manual work.

Bloke · 2020-08-03 14:55:10

Vienuolis wrote #325087:

not to name them full-size pictures by default, because they are never used on a webpage in its full size. I would like to set images on a page with no attributes for code simplicity and lesser manual work.

Hmm, see, this is something of a double-edged sword. As designers yes, I fully expect the “largest” size uploaded to be the one used on the widest desktops, and then we upload (or generate) smaller version of that for use on narrower viewpoints.

But if we’re handing over the site to users (e.g. a bunch of mums running a charity) then it becomes difficult to say to them:

“Okay, if you want to add a new blog post, it’s easy. Head to the Write page, type your blog post content and drag your gallery images into the dropzone. Oh, but before you do that you’ll need to resize each image down to 1280×834 otherwise it’ll be huge and slow your site down. If it’s a portrait image, you’ll need to pad it with whitespace on the vertical edges to make sure it shows up without being squished…”

They’re just going to look at me blankly. And then go and use another CMS where you can crop and resize and edit the image in the CMS itself, because “Textpattern is too difficult”.

It’s far better for users to just drag an image – whatever the size (max file upload limits notwithstanding) – and have Txp automatically generate every image size that we – the designers – have decreed we require. So it makes us a max-res 1600px image, plus a 1280px, plus an 800px and a 300px image because we know that our design supports those sizes.

The fact we never use the “collosal” original is a small price to pay for the end-user convenience of just being able to throw pretty much any image at the CMS and have the designer (the person who has set up the image profiles and breakpoints and templates) handle everything on their behalf.

So I totally see what you mean about it being a waste of space and a little extra code in terms of attributes. But it’s kind of handy having the original there. Because it also means it’s easy to regenerate the smaller versions from it at the click of a button if anything gets trashed by mistake.

Perhaps one day when we have an uploader that allows you to crop, resize, reposition, set the focal point for art direction and so on for each uploaded image, then we can get rid of the original ‘colossal’ image and only use it to generate what we want at first and make the ‘biggest’ version the one we actually need.

Until then, I think I’m more comfortable with the overhead. Anyone else have any views on this? Or things we can do?

Last edited by Bloke (2020-08-03 14:56:47)

gaekwad · 2020-08-03 15:16:52

How do Automattic do images with Wordpress? Anything we can take inspiration from?

Bloke · 2020-08-03 15:34:01

I think it’s theme dependent in WP. Not sure entirely. Someone will know for sure.

Once we get ImageMagick or suitable equivalent in place we should be able to do much more in terms of image manipulation, either after upload (per image when you click each to edit it) or, if we can figure out the UX, in batch. In the latter cases you still have the hassle if you upload 10 images, how do you make a useful interface upon upload to manipulate them all?

Do we show them one after the other so you can scroll down to each and manipulate them, one by one?
Do we set the parameters for one and allow it to apply to all?
Do we show a box so the alt (and caption?) for each can be entered as well?
Should we try and read any image (a.k.a.) camera metadata – as vast and vendor-specific as it is) to pre-populate anything?
Do we set parameters first – like we do with category – that will then apply to all images upon upload. And how would that interface look on the Images list panel?
Do we show a preview of each image prior to upload and allow manipulation at that stage, before you click the upload button?
…

Soooo many things to think about and wade through.

It’s like most things here: because we’re able to see how the leaders all do it (and fail) we can take inspiration from their approaches, choose the best bits, get rid of the crap, and do it right, like we did with the themes dev/live workflow.

gaekwad · 2020-08-03 15:49:18

Bloke wrote #325092:

It’s like most things here: because we’re able to see how the leaders all do it (and fail) we can take inspiration from their approaches, choose the best bits, get rid of the crap, and do it right, like we did with the themes dev/live workflow.

It’s funny you mentioned that, my earlier reply originally had a follow-on sentence of “Is there anything we can – wait for it – improve on?”, but it sounded crass so I removed it before posting.

jakob · 2020-08-03 16:03:09

Bloke wrote #325092:

I think it’s theme dependent in WP. Not sure entirely. Someone will know for sure.

WP has a few standard sizes:

Thumbnail size (150px square)
Medium size (maximum 300px width and height)
Large size (maximum 1024px width and height)
Full size (full/original image size you uploaded)

and then themes can register their own sizes:

add_image_size( 'post-thumbnail size', 800, 350 );

as described here for example.

It has downsides when you try out various different themes and the image sizes aren’t cleaned up properly, you can end up with a huge number of images, but otherwise it’s very flexible.

Once we get ImageMagick or suitable equivalent in place we should be able to do much more in terms of image manipulation, either after upload (per image when you click each to edit it) or, if we can figure out the UX, in batch.

One big plus that WP has is that there’s a plugin to run the images through Smushit to optimise their sizes. ImageOptim also has an API (hattip to Pete) including a resize option but it’s not free.

That’s something we could definitely do with.

michaelkpate · 2020-08-03 16:13:22

gaekwad wrote #325091:

How do Automattic do images with Wordpress? Anything we can take inspiration from?

Wordpress has a Media Library which handles “images, video, recordings, and files that you upload and use in your blog.”

gaekwad · 2020-08-03 16:43:28

jakob wrote #325094:

ImageOptim also has an API (hattip to Pete) including a resize option but it’s not free.

There’s this – any good?: github.com/ImageOptim/php-imageoptim-api

giz · 2020-08-03 19:05:21

Bloke wrote #325085:

Yes, part of the issue here is that smd_thumbnail requires endless configuration for no actual gain. It simply adds complexity to have to create new profiles for different ratios or different pixel densities.

So I’m looking for a better method to support and define that without all the faff.

I’ve found most clients (admin users) are bowled over by the complexity around managing images / resolution / pixels; adding profiles and sets of images to choose between would burden them further.

I like to keep the images tab as a master repository of images in their original resolution and proportion – ready for further manipulation / display via whatever template you’re using in TXP.

I use SLIR to handle all image sizing – its rather good , generates images on demand, caches them, and allows the designer to control everything from the code.

Try it!

Last edited by giz (2020-08-03 19:05:54)

Bloke · 2020-08-03 20:44:11

giz wrote #325106:

I’ve found most clients (admin users) are bowled over by the complexity around managing images / resolution / pixels; adding profiles and sets of images to choose between would burden them further.

Yes. And that’s an excellent point. It’s not just end users, but admin users too.

I use SLIR

I did look at that briefly, then forgot all about it, sorry. Considering what it does, and that it only requires GD, the fact it’d add a few hundred KB to our core seems fairly modest. Yes, it’s a good few years old now. But if it works, well, who’s to say we can’t just try it out.

I’d like to see how it stores all those files (in that empty /pel directory?). If it keeps them in subdirs somehow, then even better. Guess it’s not that important: running your site with 7K images is a good testament to it working well!

So if we didn’t bother with profiles at all and stuck SLIR in Txp’s /vendors directory, I wonder how we’d use it in core? We’d have to maybe symlink (bah, Windows) to it or use mod_rewrite to pass control to the correct dir. Or simply install it in our /images directory (is that dangerous?). I’m a bit wary of polluting the root file system with another dir, but that’s the worst case scenario.

After that, do we just let it do its job? Leave it entirely up to the site designer to create whatever images they want from originals as suits their design? If they want to make srcsets from the images, all they need to know is the syntax for creating their own images (if they wanted to do it by hand).

Or we could hardwire <txp:image> tag attributes such as width and height to pass them through /slir before fetching them. If we retrofitted them with some other attributes like crop and quality and fill etc then it might just be all we need. No admin overhead. I kinda like the sound of that. Especially when core could do exactly the same calls to auto-generate thumbs and grid images as needed.

The thing I really like is that it forbids upscaling. So if you do try and upload a teensy image, so what? You get the same size image when you pass it through /slir and ask for a bigger one, right? That has serious merit. Does make it interesting if someone uploads a crap quality/size image as it might break a layout, but caveat utilitor.

Y’know, the more I think about this, the more I like it. There has to be a downside. Is there?

EDIT: I guess we don’t get the option to do art direction or custom crops / thumbs at all, because there is always only one main image from which all others are taken. Unless we could extend it to permit cropping at a given point. Or use CSS on the output stage to handle reframing?

Last edited by Bloke (2020-08-03 21:01:31)

giz · 2020-08-03 21:02:29

Bloke wrote #325110:

I’d like to see how it stores all those files (in the pel directory?). If it keeps them in subdirs somehow, then even better.

Wherever you put slir and run its install script, it creates a folder structure like so:

{my-slir-install}
- cache
  - rendered
  - request

Calling slir from anywhere is as simple as <img src="path-to-slir/my-slir-install/w640/<txp:image_url />" />. If you omit the sizing string, slir returns the original image url.

So if we didn’t bother with profiles at all and stuck SLIR in Txp’s /vendors directory, I wonder how we’d use it in core? I mean, do we just let it do its job? Leave it entirely up to the site designer to create whatever images they want from originals as suits their design? If they want to make srcsets from the images, all they need to know is the syntax for creating their own images (if they wanted to do it by hand).

Or we could hardwire <txp:image> tag attributes such as width and height to pass them through /slir before fetching them. If we retrofitted them with some other attributes like crop and quality and fill etc then it might just be all we need. No admin overhead. I kinda like the sound of that. Especially when core could do exactly the same calls to auto-generate thumbs and grid images as needed.

The second one!

Y’know, the more I think about this, the more I like it. There has to be a downside. Is there?

None that I know of. The biggest problem I’ve had was once when a client’s host changed data centres and time zones, and some of the images came up as 404 on the public site. The fix was deleting slir’s cache so it could start afresh.

Last edited by giz (2020-08-03 21:05:14)

Bloke · 2020-08-03 21:47:05

giz wrote #325111:

Wherever you put slir and run its install script

So that installation might have to be something we could automate somehow as part of the Txp upgrade process. But if we could conquer that then I don’t see why we can’t install it inside an /images/slir/fetch/ directory. So the cache sits there, and all the image handling code and full size pics can sit alongside it. Nice and compartmentalised.

One day we should consider rejigging our storage methodology (as I alluded to in the OP) so the main images are not all in one dir, but that does make generating images via slir potentially more tricky since the /path/to/image might not always be the same. Needs to be pondered.

Calling slir from anywhere is as simple as <img src="path-to-slir/my-slir-install/w640/<txp:image_url />" />.

So, let’s see. Today:

<txp:images category="products" wraptag="div" class="gallery">
   <txp:image width="0" height="0" />
</txp:images>

Creates:

<div class="gallery">
   <img src="/images/widgetA.jpg" alt="Widget A" />
   <img src="/images/widgetB.jpg" alt="Widget B" />
   <img src="/images/widgetC.jpg" alt="Widget C" />
   ...
</div>

We could do this:

<txp:images category="products" width="450" background="ccc" wraptag="div" class="gallery">
   <txp:image width="0" height="0" />
</txp:images>

Which would create a gallery thusly instead:

<div class="gallery">
   <img src="/images/slir/fetch/w450-bccc/images/widgetA.jpg" alt="Widget A" />
   <img src="/images/slir/fetch/w450-bccc/images/widgetB.jpg" alt="Widget B" />
   <img src="/images/slir/fetch/w450-bccc/images/widgetC.jpg" alt="Widget C" />
   ...
</div>

That would make all images in that set 450px wide with optional light grey background if they needed scaling in any dimension, dump them on the page and cache them for the next person.

If so, I’m sold. We can close this thread, add the feature and go home :)

Realistically, I wonder if there are any security implications of leaving /install lying around after the fact? I mean, in theory anyone could re-run it and alter the installation dir, which would break the feature or potentially redirect elsewhere.

Maybe we could run it once on a local install, see what it does, and what it generates. There’s mention of a config file so I wonder if that might give some clues. i.e. does it just create the dir structure you highlighted or does it do more?

Then there’s also what to do with multi-site installs. And what about people who use ihu to serve images from a cookieless domain other than the one in Txp? That might not have the ability to run PHP, so housing slir in there might not work.

It’d be nice if slir’s core PHP itself could be uploaded to, say, /vendors and we could just “install” the cache to /images/fetch. i.e. split it. If it does permit that, things get a lot easier – especially for upgrades. That might give us the opportunity to add a pref that could define the endpoint of your cache. If that’s set, slir wakes up when you Save the prefs and all the image tags start to use it. Without that pref, you get image links as you do today.

Not sure what would happen if you changed your mind and moved the cache by changing the pref. That sounds awkward, so maybe that won’t work and we just automatically assume ihu is the base path of where the cache will live. Then again, if you change your mind, the worst thing that happens is that you lose the thumbs that have been cached and start again. The only way round that would be to manually move the cache over to the new location. And if you don’t move/delete the cache, it just stagnates and takes up space.

Hmmm, I can feel some experimentation coming on…

Textpattern CMS support forum

#1 2020-08-03 09:08:50