Multiple image handling: profiles

Bloke · 2020-08-03 20:44:11

giz wrote #325106:

I’ve found most clients (admin users) are bowled over by the complexity around managing images / resolution / pixels; adding profiles and sets of images to choose between would burden them further.

Yes. And that’s an excellent point. It’s not just end users, but admin users too.

I use SLIR

I did look at that briefly, then forgot all about it, sorry. Considering what it does, and that it only requires GD, the fact it’d add a few hundred KB to our core seems fairly modest. Yes, it’s a good few years old now. But if it works, well, who’s to say we can’t just try it out.

I’d like to see how it stores all those files (in that empty /pel directory?). If it keeps them in subdirs somehow, then even better. Guess it’s not that important: running your site with 7K images is a good testament to it working well!

So if we didn’t bother with profiles at all and stuck SLIR in Txp’s /vendors directory, I wonder how we’d use it in core? We’d have to maybe symlink (bah, Windows) to it or use mod_rewrite to pass control to the correct dir. Or simply install it in our /images directory (is that dangerous?). I’m a bit wary of polluting the root file system with another dir, but that’s the worst case scenario.

After that, do we just let it do its job? Leave it entirely up to the site designer to create whatever images they want from originals as suits their design? If they want to make srcsets from the images, all they need to know is the syntax for creating their own images (if they wanted to do it by hand).

Or we could hardwire <txp:image> tag attributes such as width and height to pass them through /slir before fetching them. If we retrofitted them with some other attributes like crop and quality and fill etc then it might just be all we need. No admin overhead. I kinda like the sound of that. Especially when core could do exactly the same calls to auto-generate thumbs and grid images as needed.

The thing I really like is that it forbids upscaling. So if you do try and upload a teensy image, so what? You get the same size image when you pass it through /slir and ask for a bigger one, right? That has serious merit. Does make it interesting if someone uploads a crap quality/size image as it might break a layout, but caveat utilitor.

Y’know, the more I think about this, the more I like it. There has to be a downside. Is there?

EDIT: I guess we don’t get the option to do art direction or custom crops / thumbs at all, because there is always only one main image from which all others are taken. Unless we could extend it to permit cropping at a given point. Or use CSS on the output stage to handle reframing?

Last edited by Bloke (2020-08-03 21:01:31)

giz · 2020-08-03 21:02:29

Bloke wrote #325110:

I’d like to see how it stores all those files (in the pel directory?). If it keeps them in subdirs somehow, then even better.

Wherever you put slir and run its install script, it creates a folder structure like so:

{my-slir-install}
- cache
  - rendered
  - request

Calling slir from anywhere is as simple as <img src="path-to-slir/my-slir-install/w640/<txp:image_url />" />. If you omit the sizing string, slir returns the original image url.

So if we didn’t bother with profiles at all and stuck SLIR in Txp’s /vendors directory, I wonder how we’d use it in core? I mean, do we just let it do its job? Leave it entirely up to the site designer to create whatever images they want from originals as suits their design? If they want to make srcsets from the images, all they need to know is the syntax for creating their own images (if they wanted to do it by hand).

Or we could hardwire <txp:image> tag attributes such as width and height to pass them through /slir before fetching them. If we retrofitted them with some other attributes like crop and quality and fill etc then it might just be all we need. No admin overhead. I kinda like the sound of that. Especially when core could do exactly the same calls to auto-generate thumbs and grid images as needed.

The second one!

Y’know, the more I think about this, the more I like it. There has to be a downside. Is there?

None that I know of. The biggest problem I’ve had was once when a client’s host changed data centres and time zones, and some of the images came up as 404 on the public site. The fix was deleting slir’s cache so it could start afresh.

Last edited by giz (2020-08-03 21:05:14)

Bloke · 2020-08-03 21:47:05

giz wrote #325111:

Wherever you put slir and run its install script

So that installation might have to be something we could automate somehow as part of the Txp upgrade process. But if we could conquer that then I don’t see why we can’t install it inside an /images/slir/fetch/ directory. So the cache sits there, and all the image handling code and full size pics can sit alongside it. Nice and compartmentalised.

One day we should consider rejigging our storage methodology (as I alluded to in the OP) so the main images are not all in one dir, but that does make generating images via slir potentially more tricky since the /path/to/image might not always be the same. Needs to be pondered.

Calling slir from anywhere is as simple as <img src="path-to-slir/my-slir-install/w640/<txp:image_url />" />.

So, let’s see. Today:

<txp:images category="products" wraptag="div" class="gallery">
   <txp:image width="0" height="0" />
</txp:images>

Creates:

<div class="gallery">
   <img src="/images/widgetA.jpg" alt="Widget A" />
   <img src="/images/widgetB.jpg" alt="Widget B" />
   <img src="/images/widgetC.jpg" alt="Widget C" />
   ...
</div>

We could do this:

<txp:images category="products" width="450" background="ccc" wraptag="div" class="gallery">
   <txp:image width="0" height="0" />
</txp:images>

Which would create a gallery thusly instead:

<div class="gallery">
   <img src="/images/slir/fetch/w450-bccc/images/widgetA.jpg" alt="Widget A" />
   <img src="/images/slir/fetch/w450-bccc/images/widgetB.jpg" alt="Widget B" />
   <img src="/images/slir/fetch/w450-bccc/images/widgetC.jpg" alt="Widget C" />
   ...
</div>

That would make all images in that set 450px wide with optional light grey background if they needed scaling in any dimension, dump them on the page and cache them for the next person.

If so, I’m sold. We can close this thread, add the feature and go home :)

Realistically, I wonder if there are any security implications of leaving /install lying around after the fact? I mean, in theory anyone could re-run it and alter the installation dir, which would break the feature or potentially redirect elsewhere.

Maybe we could run it once on a local install, see what it does, and what it generates. There’s mention of a config file so I wonder if that might give some clues. i.e. does it just create the dir structure you highlighted or does it do more?

Then there’s also what to do with multi-site installs. And what about people who use ihu to serve images from a cookieless domain other than the one in Txp? That might not have the ability to run PHP, so housing slir in there might not work.

It’d be nice if slir’s core PHP itself could be uploaded to, say, /vendors and we could just “install” the cache to /images/fetch. i.e. split it. If it does permit that, things get a lot easier – especially for upgrades. That might give us the opportunity to add a pref that could define the endpoint of your cache. If that’s set, slir wakes up when you Save the prefs and all the image tags start to use it. Without that pref, you get image links as you do today.

Not sure what would happen if you changed your mind and moved the cache by changing the pref. That sounds awkward, so maybe that won’t work and we just automatically assume ihu is the base path of where the cache will live. Then again, if you change your mind, the worst thing that happens is that you lose the thumbs that have been cached and start again. The only way round that would be to manually move the cache over to the new location. And if you don’t move/delete the cache, it just stagnates and takes up space.

Hmmm, I can feel some experimentation coming on…

colak · 2020-08-04 04:21:31

Hi guys,

I’m worried about incorporating an unsupported project as part of the core. Also, I understand that SLIR has issues when working on MAMP/WAMP offline environments and it will more tedious for new users to set up or ignore it.

I nevertheless like the idea of SLIR and I am wondering if there is another supported script which can do more or less what it does.

phiw13 · 2020-08-04 05:04:38

so… what I understand so far from this thread, you are mostly focussed on generating images for gallery type of pages / sites. what I work with mostly is very far away from that. What I do is articles – text narrative – which may contain images. the narrative lines between text and images is sometimes complementary, sometimes diverging, sometimes even clashing. And then those embedded images need various sizes (accommodating large and small screens). to make it more fun, for one set of articles I may need large and small images, e.g somewhere in the article you see a 800px wide image, other places use a couple of 300px images side by side. other sets of articles have different requirements / needs when it comes to images.

It is even possible that an article contains, in the middle or towards/at the end some kind of mini-gallery of rather thumbnail sized things.

and more fun: images were sometimes reused in other articles in a different context with different t size requirements…

smd_thumbnails is useful in managing all this although it requires quite a mess of profiles…

Bloke · 2020-08-04 08:51:20

phiw13 wrote #325115:

so… what I understand so far from this thread, you are mostly focussed on generating images for gallery type of pages / sites.

Not at all. Sorry if that came across. It was late and my brain couldn’t face coming up with more than one (simple) example, so I chose gallery.

I haven’t thought it through fully yet, but for the multi-size case you might be able to do this:

<txp:images name="JapanIn1805.jpg" width="800, 1280, 300" />

And that would generate:

<img src="/images/fetch/800/images/JapanIn1805.jpg"
    srcset="/images/fetch/800/images/JapanIn1805.jpg 800w,
        /images/fetch/1280/images/JapanIn1805.jpg 1280w,
        /images/fetch/300/images/JapanIn1805.jpg 300w" />

That’s one image, multiple sizes that the browser can choose from, by using <txp:images> (plural) to deliver the set. And note the first one is the ‘fallback’ image used for browsers that don’t understand srcset (if there are any left out there that don’t).

Now, that’s only a third of the story. You still need to somehow specify the HTML sizes attribute to give the browser an indication of how to handle the images at your breakpoints. I guess we could just add a sizes attribute and that could be passed verbatim to each image in the “set” (which in this case isn’t a gallery: just the same image at different widths).

If you combined the above with not just one image but an entire category of images, you could create a gallery where each image also has a bunch of sizes for viewports, so it makes each image have the same set of image widths you define in the <txp:images> tag). I think that would work.

The beauty of the slir approach is that once the heavy lifting has been done on first render at any given device width, the images at those sizes are cached so future paints are near-instant. So you get an infinite number of profiles without having to define them up-front – which is where the smd_thumbnail approach falls down.

The final third of the puzzle is perhaps using <txp:images> to output <picture> wrappers and somehow getting it to render <source> tags, as well as being able to output srcset. In that situation it’s sometimes not the same image each time because you want to do art direction such as offering a zoomed-in view that uses a completely different image.

It might be too much to ask for a single tag to do it all, so I’m open to suggestions how to handle this for the least amount of confusion and maximum flexibility that allows us to build sensible and powerful HTML tags with a little Txp magic.

somewhere in the article you see a 800px wide image, other places use a couple of 300px images side by side. other sets of articles have different requirements / needs when it comes to images.

And this is a perfect example of why you don’t need pre-defined profiles. They’re counter-productive because you get all these images created on your behalf that you could use, but probably won’t ever need, which wastes space.

An on-demand image generator such as slir would give you the flexibility to render a single image at 800, then two later in the article at 300. For maximum enjoyment you could make yourself a shortcode to do it and pass the width(s) and filename(s) you need to a form.

and more fun: images were sometimes reused in other articles in a different context with different size requirements…

Again, on-demand image generation covers this. When your visitors encounter the same image in a different width, a cache miss occurs, slir goes off and generates the image for you at the given size, serves it and caches it for next time.

Does that give you more insight into the idea?

Last edited by Bloke (2020-08-04 08:56:43)

philwareham · 2020-08-04 08:56:52

I’ve only skimmed this thread as I’m really busy this week – but my opinion on image handling is in agreement with others here, that:

You upload an original image at whatever max size it was created at. No processing is done on that image (i.e. it stays at original dimensions/file size). That is your master image, for want of a better word.
Textpattern then creates one or a number of different sizes of that image depending on what profiles you have set up (we can define some defaults as a starting point). Cropping can occur based on original versus profile.
Textpattern also creates a thumbnail image square (cropping done) which it uses on the images list panel(s). This can be used on live site too, if required for some reason.
Possibly, user can upload replacement images for any that have been auto-created by the profile.
Cropping/rotating/etc of original file can be done directly within Textpattern panel, but the edited file never overwrites the original.

This above process means that if the user decides at some future date to change their theme, column width, whatever, the original images are still available to generate new sizes from.

It’s going to be a careful balance between simplicity, perceived user ability and features to get this working in the best way. I need to play with my latest local WordPress install too and see if there is any learning we can get from their process.

Bloke · 2020-08-04 09:07:48

colak wrote #325114:

I’m worried about incorporating an unsupported project as part of the core.

It’s a valid concern. The project is over 5 years old, despite being written in a forward-thinking object/class manner (from skimming the code). I don’t know of any alternatives that do the same thing, because I haven’t looked. If there’s a better project, great.

We can’t shield ourselves indefinitely from unsupported code. phpass, which we include, has ceased development it seems. jQueryUI is gradually dying it seems. So what seemed like a good idea 5 years ago is now not so. There’s nothing we can do but adapt.

The good news is that if we do find that slir (or whatever) breaks under some future version of PHP or something there’s nothing to stop us forking it, because it’s open source code. PHP might decide one day to drop GD, but if they do that, Txp is stuffed as it is today anyway in the thumbnail department as we’ve assumed it’s available.

Yes, it’s possibly a bind to fork third party code and maintain it, as we’d need to grok it and maybe in the process come up with ways to extend it to do more. But the option is there should we need it. It’s only code.

The fact Gary (giz) has been using it for years and it’s still working is testament to the fact it must be reasonably well written in the first place. It would need thorough testing. And if anybody does stumble upon a more recent project that does something similar for the roughly the same (or smaller) code/memory footprint then let’s consider it. I’m rapidly going off the idea of “image profiles” because they’re way too restrictive.

I understand that SLIR has issues when working on MAMP/WAMP offline environments

This I don’t know. If that’s the case then maybe we’ll need to fork and patch it from the get-go. If the “installation” procedure isn’t automatable or if the slir code must reside in the image directory where your cached images are stored then it’s also a show-stopper for me. I don’t like the idea of code lying around in my /images dir. If that was the case, we’d either need to find an alternative project that does allow it or fork and make the changes ourselves.

phiw13 · 2020-08-04 09:32:38

Bloke wrote #325120:

Not at all. Sorry if that came across. It was late and my brain couldn’t face coming up with more than one (simple) example, so I chose gallery.

[…]

Does that give you more insight into the idea?

That explanation definitively gives more insight into the concept. It does indeed offer some possibilities (ok… more than just some).

Code wise, for most use cases, <img src="" srcset="" sizes=""> ought to be sufficient. And, as now, <txp:images /> could still be used as a container tag to manually build the needed construct (thinking the use of <picture /> to have dedicated light and dark mode images e.g. screenshots of software UI).

Bloke · 2020-08-04 09:34:51

philwareham wrote #325122:

You upload an original image at whatever max size it was created at.

Yes, absolutely. One main, full size image. No thumbnails, no nothing else. Whether this image is physically stored in a single /images dir (as today) or we put some algorithm in place to spread the load is a decision for later.

Textpattern then creates one or a number of different sizes of that image depending on what profiles you have set up

This is where my thinking has skewed away from profiles. After looking more at slir (and maybe there are other similar projects out there) I’m of the opinion that on-demand thumbnail generation really is a much better solution… sort of.

When you upload the original and (today’s) Images panel refreshes its list view, it could encounter an <img> tag that says “go grab this image at 150px square”. It would do so. It would generate one right then at that size. Cache it. Next time you refresh the page, boom, it’s there immediately. And also available at that size if anybody wants to use it in front-end land.

If, however, you were in grid view, core could say “go grab the image at 450px square, crop it and pad it with light grey to fit” (just an example, we might not bother with padding backgrounds and use CSS for that). It would do so, generate it cache it, done. Same for every image in the panel. And, likewise, if anybody in front-end land crafts a tag that requires an image at that size with that colour background, it’s already available in the cache.

Yes, potentially, it will create a lot of images if you have multiple background colours and stuff like that on your site, for the images on each of your pages. But the point is, it’s only creating ones you’ve used. Not ones that don’t make sense or have no business being created (e.g. upscaled large images from tiny icons).

We could craft it so the Images panel requests 450px square AND 900px square images as its own srcset so the <img> tag could use the second for double-density displays. Then we get retina support on the admin side out of the box. Heck, we could even set the default sizes we use on the admin side as a (hidden/advanced/config.php) variable or pref. Then if someone wanted to make different size images for their quad-density super crunjo display, well, go for it. Edit the pref or override the values and next time you visit the admin panel, it automatically rebuilds your thumbs in the size specified for the current device being used.

Dunno. Just freewheeling.

Possibly, user can upload replacement images for any that have been auto-created by the profile.

Now this is where slir falls down. Unless we can understand how it caches images and retrieve them, we have no idea how many images at varying sizes and crops and whatever have been created. So we can’t offer to individually replace them.

That is a bind. If, as you say, you want a different thumbnail for some specific reason, you’re out of luck. I wonder if we could have a parallel system that acts like today so if you have specific reasons for uploading an image of a particular dimension or whatever, you can do so and it goes into our regular Txp /images cache as it does today with a suffix (or in a subdir). Then there’s nothing to stop us offering a way in the tags to say “I know what I’m doing, use my image and not the auto-generated one, please.”

Dunno.

Another unknown, given the age of the code, is if slir supports WebP or any other new format. I suspect not. So as the formats grow in popularity and the VHS-Betamax wars eventually resolve themselves, we may need to patch the code to support them. We need to patch Txp anyway as it can’t handle anything beyond the main three formats.

Cropping/rotating/etc of original file can be done directly within Textpattern panel, but the edited file never overwrites the original.

Again, this is something slir doesn’t give us. So if you upload an image that is on its side from the camera, I don’t know how to handle this. If we’re saying we’re not changing the original in any way once it’s uploaded, this is going to be an issue.

Now, if we were to alter the image and allow some rudimentary editing of the original so you could zoom in, crop, rotate inside Txp and then commit your modifications, how does that affect any cached images that have already been created? Presumably they have no knowledge of the fact you’ve changed it, so when you ask for your 450px version, you get the old one from the cache. Gary might be able to supply some info on this if he’s ever replaced any original pics from the Image Edit panel.

If slir is clever, it will check the timestamp of the original and if it differs from that of the cache, it’ll invalidate the cache version and overwrite it with a fresh copy generated from the (changed) original. If it does that, I’d be happy with just allowing people to edit the original and have it cascade to the image cache on an as-needed basis.

I need to play with my latest local WordPress install too and see if there is any learning we can get from their process.

That’d be ace, thanks. Anything we can learn from and do better works for me.

If we can’t solve the ‘individual replace a thumb’ issue mentioned above, then we may need to use hard-coded profiles, or come up with some way to query the current cache for “all the images that have been created from this original”. At least we could then display them on the Image edit panel so they can be manipulated individually, perhaps.

Last edited by Bloke (2020-08-04 09:58:18)

philwareham · 2020-08-04 09:57:26

Erm, you know Pete has already suggested a modern, supported alternative to SLIR which does much more besides (and can use both/either ImageMagick and GD)? Please see github.com/textpattern/textpattern/issues/1472.

I agree that if the profiles thing can be automated via something like the above, then that is way simpler for the user. Also, I’m not totally wedded to not being able to crop/edit the original – as long as users are aware that the process is destructive.

Bloke · 2020-08-04 09:59:20

philwareham wrote #325129:

Erm, you know Pete has already suggested a modern, supported alternative to SLIR which does much more besides

Jeez, my Swiss-cheese brain. Okay, right. I’m going to get in my box now and shut up.

EDIT: I seem to recall when I looked at it that it was Laravel-heavy so maybe it has a large byte count. Not sure.

EDIT 2: Seems to be 250KBish unpacked out of the box, which for all that functionality is remarkable if that truly is the extent of it. I suspect when running composer require it’ll pull in a tonne of library code from elsewhere… maybe? But if 250KB is all there is to it, then maybe we can use this. Perhaps wrap an interface around it to create images on the fly in a manner similar to the way slir does when we craft certain image-ish URLs in core code or via our <txp:image*> tag suite.

Last edited by Bloke (2020-08-04 10:14:21)

Textpattern CMS

Textpattern CMS support forum

#13 2020-08-03 20:44:11

Re: Multiple image handling: profiles

giz wrote #325106:

#14 2020-08-03 21:02:29

Re: Multiple image handling: profiles

Bloke wrote #325110:

#15 2020-08-03 21:47:05

Re: Multiple image handling: profiles

giz wrote #325111:

#16 2020-08-04 04:21:31

Re: Multiple image handling: profiles

#17 2020-08-04 05:04:38

Re: Multiple image handling: profiles

#18 2020-08-04 08:51:20

Re: Multiple image handling: profiles

phiw13 wrote #325115:

#19 2020-08-04 08:56:52

Re: Multiple image handling: profiles

#20 2020-08-04 09:07:48

Re: Multiple image handling: profiles

colak wrote #325114:

#21 2020-08-04 09:32:38

Re: Multiple image handling: profiles

Bloke wrote #325120:

#22 2020-08-04 09:34:51

Re: Multiple image handling: profiles

philwareham wrote #325122:

#23 2020-08-04 09:57:26

Re: Multiple image handling: profiles

#24 2020-08-04 09:59:20

Re: Multiple image handling: profiles

philwareham wrote #325129:

Board footer