Multiple image handling: profiles

Bloke · 2020-08-03 09:08:50

As we gear up for 4.9 and beyond, the topic of image handling is once again on my mind.

You may (or may not) know I’m using smd_thumbnail as a sort of proving ground for figuring out the best way to handle multiple “thumbnails” (or actually “images of any size” if we’re being pedantic). The idea is that when we hit the best workflow there, we distill the good bits into core and jettison the rest of the plugin.

Sidestepping the actual storage of different image sizes in the filesystem (which we’ll cover in a separate discussion) I want to find out your thoughts on the general process of handling image creation and, specifically, rules governing it thereof.

smd_thumbnail uses the concept of profiles. Each image size you potentially want to use in your site has a different profile so it knows what sizes to make when you upload an original. Whether this is the best approach is open to interpretation (and I genuinely would love to hear any alternative ideas you may have on this).

The general guide is that you make images for certain use cases and then employ media queries and/or HTML srcset/picture to let you and/or the browser pick the best image size based on environment or size of viewport.

Example: You allow people to upload the full size image from their cameras (because it’s less burden on them) and you have a bunch of profiles like this:

Max: 1600w
Max2x: 3200w
Large: 1280w
Large2x: 2560w
Grid: 420w
Grid2x: 840w
Thumb: 100wx150h
Thumb2x: 200wx300h

Great. Now every time you upload a huge 8000w px image, it’s stored but you also, automatically, get a whole bunch of smaller images made that the site can use in templates. If it makes a mess of resizing, people can always resize them in an image editing program and replace individual thumbs later. Or if particular sizes need to highlight particular areas for art direction purposes.

Problems:

If you upload portrait images, they’re squished in the Thumb and Thumb2x profiles because an explicit size was defined. Creating extra profiles with a different aspect means that there is potentially a lot of wasted space creating portrait images for landscape pics (and vice versa) that will never be used. On image heavy sites, this is annoying.
The system has no way of knowing that Thumb2x is a ‘double density’ image for Retina-type displays. Sure the browser might pick it by virtue of it being close to the correct size the device can handle, but we need more than that if we’re going to size it correctly in CSS so the image isn’t blown up to double its size. Should we instead define a single “CSS pixel” dimension profile and then list which densities we want to create/support in the profile itself? Thus, one ‘profile’ could have multiple images associated with it, each suffixed with “@{density}x” if it’s not 1 (or perhaps for consistency, even if it is 1, so ‘@1x’ is defined).
If I upload a small image like an icon or logo (yeah, yeah, I know the database is not the correct location for presentational image elements but hear me out) then it’s blown up to huge proportions and goes pixelated. Again, useless as we probably don’t need the extra images at all in this case, though we might want to upload, say, a 2x or 4x density icon and have it automatically create scaled down versions for lower resolution displays.

Tricky. A few ideas:

Profile Groups
Define groups of profiles based on usage. So you can pick from, say, ‘landscape’, ‘portrait’ and ‘square’ types (maybe others?) when you create a profile. Or, they (internally) go into the correct group if you specify both w and h dimensions (because we then know, by design, in which group they belong).
Profile rules
A profile is only used on an image if defined conditions are met. Such as w>200. So you write this freeform rule (or pick from a list of available comparisons?) when you make each profile.

Both have benefits and drawbacks in some ways. There may be other ways to approach it (ideas anyone?!). Primarily, this means there are potentially ‘holes’ in the image set because you could drag two images – a landscape and a portrait – and get two different sets of images created for you. Good for disk space saving, but when you come to display those on the site, it makes template code more tricky as you need to detect which images are available.

And what about the icon example? The Groups won’t solve that (unless we can define arbitrary groups AND associated rules at the group level?). The Rules system could potentially work here because you could say that images less than 100px wide only get the ‘Icon2x’ and ‘Icon’ profiles applied. All the others remain unused, which avoids the jaggies.

When it comes to output the pics, we have the new size attribute in <txp:images> which can cater for only displaying certain sizes/aspect ratios of image. That’s nice but doesn’t solve the density issue. Ideas welcome here, please, with a view to the minimum practical considerations we need to address for sensible output of most <img> attributes.

In smd_thumbnail we can iterate over the profiles to create srcset rules like this:

<img itemprop="url contentUrl" src="<txp:image_url />"
   srcset="<txp:smd_thumbnail type="SMD_ACTIVE" break="," trim>
   <txp:smd_thumbnail_info item="url, w" break=" " />w
</txp:smd_thumbnail>"
   sizes="(min-width: 84rem) 1600px,
          (min-width: 69rem) 1280px,
          (min-width: 55rem) 1024px,
          (min-width: 46rem) 864px,
          (min-width: 38rem) 576px,
          100vw"
   alt="<txp:image_info type="alt" />"
   class="image-banner">

(I think) that works to display the correct sizes, but in my tests when you zoom out (CMD and minus) the low-res version is loaded for some reason: I was expecting that to simulate what happens on large displays, wider than my physical screen allows, but maybe I’m being unrealistic here.

Anyway, the SMD_ACTIVE in the code above only iterates over active profiles. In the groups/rules world we would need a way to iterate over active profiles and only pluck out ones that have been used for that image, which adds another layer of complexity as it involves testing image files to see which are defined and stored on disk – and that’s potentially time expensive.

Maybe we store which profiles have been used alongside the metadata in each image, so we can quickly look it up? We’d still need to check if it actually exists on disk when we come to output it, but that’s down to the image rendering portion to gracefully fail or throw a warning, as it does now, if an image is missing. It only burdens the output stage, not the template coder, to build the check in – unless you want to display placeholders or something, in which case, it’s down to you.

Other things to think about:

Art direction (picking a point/region on an image thumb to allow it to scroll/zoom to that area when the image is loaded).
What about the thumbnails shown on the admin side? smd_thumbnail allows you to pick a ‘default’ profile to display here but this is probably too much configuration. If we need to always make a thumbnail, that’s fine. Just what dimensions make sense? And what aspect? And do we make this available as a “fixed” (undeletable) profile name such as “thumbnail” (so there’s less likelihood of someone using the same name for their own profile).
Grid view. I think we’ve decided that we’re going to always create a grid view image, regardless, so we can use it on the admin side if needed. If someone uploads a portrait image and we opt for a landscape grid, do we pad with space? Use CSS to pad it in the grid? Crop? And, again, do we add it to the set of pre-defined profiles and call it “grid” (or something). Maybe someone could use this on their site and, between “thumb” and “grid”, have enough sizes to manage a responsive site without the need for creating more profiles? If so, great!

In short, I’m trying to figure out how to handle this:

in the most sensible manner
without introducing ridiculous levels of configuration
using sensible conventions
that allow us to build flexible image galleries for responsive layouts
without wasting disk space for unneeded sizes

Any thoughts on the above, improvements, features, edge cases I’ve not thought of, limitations, conventions we could adopt, anything that I could try out in smd_thumbnail as a proving ground, please let your voice be heard.

Thank you.

zero · 2020-08-03 10:53:42

My immediate thought is that being able to select Portrait, Landscape or Square when uploading an image would be most intuitive to me. So then, having created desired thumb sizes for each profile, the images are all output in a similar fashion to how you have it now. The designer could more easily specify image sizes than at present, making galleries easier to create. I wouldn’t consider that too much work and it would be easier than at present. Too much automation might also add to confusion rather than reduce it — too many choices, too much to think about? But I only occasionally work on image-heavy sites.

Bloke · 2020-08-03 11:15:11

Yes, part of the issue here is that smd_thumbnail requires endless configuration for no actual gain. It simply adds complexity to have to create new profiles for different ratios or different pixel densities.

So I’m looking for a better method to support and define that without all the faff.

Vienuolis · 2020-08-03 13:28:26

The first desire from my experience using txp:image (and txp:smd_thumbnail, txp:images, etc.) is not to name them full-size pictures by default, because they are never used on a webpage in its full size. I would like to set images on a page with no attributes for code simplicity and lesser manual work.

Bloke · 2020-08-03 14:55:10

Vienuolis wrote #325087:

not to name them full-size pictures by default, because they are never used on a webpage in its full size. I would like to set images on a page with no attributes for code simplicity and lesser manual work.

Hmm, see, this is something of a double-edged sword. As designers yes, I fully expect the “largest” size uploaded to be the one used on the widest desktops, and then we upload (or generate) smaller version of that for use on narrower viewpoints.

But if we’re handing over the site to users (e.g. a bunch of mums running a charity) then it becomes difficult to say to them:

“Okay, if you want to add a new blog post, it’s easy. Head to the Write page, type your blog post content and drag your gallery images into the dropzone. Oh, but before you do that you’ll need to resize each image down to 1280×834 otherwise it’ll be huge and slow your site down. If it’s a portrait image, you’ll need to pad it with whitespace on the vertical edges to make sure it shows up without being squished…”

They’re just going to look at me blankly. And then go and use another CMS where you can crop and resize and edit the image in the CMS itself, because “Textpattern is too difficult”.

It’s far better for users to just drag an image – whatever the size (max file upload limits notwithstanding) – and have Txp automatically generate every image size that we – the designers – have decreed we require. So it makes us a max-res 1600px image, plus a 1280px, plus an 800px and a 300px image because we know that our design supports those sizes.

The fact we never use the “collosal” original is a small price to pay for the end-user convenience of just being able to throw pretty much any image at the CMS and have the designer (the person who has set up the image profiles and breakpoints and templates) handle everything on their behalf.

So I totally see what you mean about it being a waste of space and a little extra code in terms of attributes. But it’s kind of handy having the original there. Because it also means it’s easy to regenerate the smaller versions from it at the click of a button if anything gets trashed by mistake.

Perhaps one day when we have an uploader that allows you to crop, resize, reposition, set the focal point for art direction and so on for each uploaded image, then we can get rid of the original ‘colossal’ image and only use it to generate what we want at first and make the ‘biggest’ version the one we actually need.

Until then, I think I’m more comfortable with the overhead. Anyone else have any views on this? Or things we can do?

Last edited by Bloke (2020-08-03 14:56:47)

gaekwad · 2020-08-03 15:16:52

How do Automattic do images with Wordpress? Anything we can take inspiration from?

Bloke · 2020-08-03 15:34:01

I think it’s theme dependent in WP. Not sure entirely. Someone will know for sure.

Once we get ImageMagick or suitable equivalent in place we should be able to do much more in terms of image manipulation, either after upload (per image when you click each to edit it) or, if we can figure out the UX, in batch. In the latter cases you still have the hassle if you upload 10 images, how do you make a useful interface upon upload to manipulate them all?

Do we show them one after the other so you can scroll down to each and manipulate them, one by one?
Do we set the parameters for one and allow it to apply to all?
Do we show a box so the alt (and caption?) for each can be entered as well?
Should we try and read any image (a.k.a.) camera metadata – as vast and vendor-specific as it is) to pre-populate anything?
Do we set parameters first – like we do with category – that will then apply to all images upon upload. And how would that interface look on the Images list panel?
Do we show a preview of each image prior to upload and allow manipulation at that stage, before you click the upload button?
…

Soooo many things to think about and wade through.

It’s like most things here: because we’re able to see how the leaders all do it (and fail) we can take inspiration from their approaches, choose the best bits, get rid of the crap, and do it right, like we did with the themes dev/live workflow.

gaekwad · 2020-08-03 15:49:18

Bloke wrote #325092:

It’s like most things here: because we’re able to see how the leaders all do it (and fail) we can take inspiration from their approaches, choose the best bits, get rid of the crap, and do it right, like we did with the themes dev/live workflow.

It’s funny you mentioned that, my earlier reply originally had a follow-on sentence of “Is there anything we can – wait for it – improve on?”, but it sounded crass so I removed it before posting.

jakob · 2020-08-03 16:03:09

Bloke wrote #325092:

I think it’s theme dependent in WP. Not sure entirely. Someone will know for sure.

WP has a few standard sizes:

Thumbnail size (150px square)
Medium size (maximum 300px width and height)
Large size (maximum 1024px width and height)
Full size (full/original image size you uploaded)

and then themes can register their own sizes:

add_image_size( 'post-thumbnail size', 800, 350 );

as described here for example.

It has downsides when you try out various different themes and the image sizes aren’t cleaned up properly, you can end up with a huge number of images, but otherwise it’s very flexible.

Once we get ImageMagick or suitable equivalent in place we should be able to do much more in terms of image manipulation, either after upload (per image when you click each to edit it) or, if we can figure out the UX, in batch.

One big plus that WP has is that there’s a plugin to run the images through Smushit to optimise their sizes. ImageOptim also has an API (hattip to Pete) including a resize option but it’s not free.

That’s something we could definitely do with.

michaelkpate · 2020-08-03 16:13:22

gaekwad wrote #325091:

How do Automattic do images with Wordpress? Anything we can take inspiration from?

Wordpress has a Media Library which handles “images, video, recordings, and files that you upload and use in your blog.”

gaekwad · 2020-08-03 16:43:28

jakob wrote #325094:

ImageOptim also has an API (hattip to Pete) including a resize option but it’s not free.

There’s this – any good?: github.com/ImageOptim/php-imageoptim-api

giz · 2020-08-03 19:05:21

Bloke wrote #325085:

Yes, part of the issue here is that smd_thumbnail requires endless configuration for no actual gain. It simply adds complexity to have to create new profiles for different ratios or different pixel densities.

So I’m looking for a better method to support and define that without all the faff.

I’ve found most clients (admin users) are bowled over by the complexity around managing images / resolution / pixels; adding profiles and sets of images to choose between would burden them further.

I like to keep the images tab as a master repository of images in their original resolution and proportion – ready for further manipulation / display via whatever template you’re using in TXP.

I use SLIR to handle all image sizing – its rather good , generates images on demand, caches them, and allows the designer to control everything from the code.

Try it!

Last edited by giz (2020-08-03 19:05:54)

Textpattern CMS

Textpattern CMS support forum

#1 2020-08-03 09:08:50