Automatic thumbnails for Textpattern

Bloke · Yesterday 16:37:33

Just as a discussion point, here’s what’s going on.

Currently

Tag on page encountered with width/height attributes.
imageBuildURL() invoked to construct a URL that can be thrown at the resizer. A token is created in the database.
The resizer looks at the request and checks if it has an image in the cache.
If so, it is returned, fully formed (as an image). [ this bit might not be working because of the tokens – I’m investigating ].
If there’s no matching image, it looks at the passed token, recomputes the hash and compares it to the stored token to validate the request.
If the tokens match, it creates a thumbnail at the desired size and adds it to the cache for next time.

Soon (hopefully)

Tag on page encountered with width/height attributes.
imageBuildURL() invoked and asks the filesystem if that image already exists at the given size/crop.
If yes, it receives a direct URL to the cached image, which is put directly in the src attribute, bypassing the tokenization process (and meaning multiple identical images are possible).
If no, a token is created and then the process above takes over from step 5 to fetch the fully formed image.

Future (maybe) blazing performance

Tag on page encountered with width/height attributes.
imageBuildURL() invoked and computes a hash of these attributes+image ID.
A direct link to the cache is retrieved, using that hash as the filename, whether the thumb exists or not and put in the src attribute.
The browser sees the URL request and, if the file exists, happy days. Image is returned.
If the file doesn’t exist, a 404 is issued (cache miss), the resizer is invoked to create the image at the given sizes/crop, and cached using the same hash from step 2 as a reference on disk.

Somewhere in that last process, we need to factor in security to avoid DOS attacks. Whether it’s simply unlikely that anyone can fabricate a hash from the parameters (if we’re salting the values), maybe that’s good enough. Not figured that out yet.

gaekwad · Yesterday 16:58:54

Bloke wrote #341659:

That may, however, require some time that we don’t have, so it could be something we reserve for a future release. Depends how fast he can code after I’ve done this first level of optimisation :)

I won’t lie, I’ve missed this pre-release window code cramming…it’s been too long!

phiw13 · Today 07:37:51

Bloke wrote #341659:

For anyone following along, the location of the cache has moved. Now it’s hard-coded beneath the /images dir. So you’ll need a little housekeeping:

The transition went smoothly on all 3 local installs. At first visit of the images panel on one of the installs (48 images / thumbs per panel page, ~ 900 images total) there was a brief delay before images appeared. When I say ‘brief’ we’re talking about ~1 or 2 seconds max. As far as I can tell my test articles and the 3 shortcode blocks I had updated away from the smd_thumbnail based constructs are working well.

Your future development looks interesting.

jakob · Today 08:50:44

phiw13 wrote #341662:

The transition went smoothly on all 3 local installs …

Are your srcsets working correctly? On my local test installation with that funny 3-aspect-ratio srcset test setup, I now get the same image served for all srcset and src images. The browser will load the most appropriate image for the respective viewport width on load but doesn’t augment the others when you increase viewport width. Rolling over the img src and srcset urls in the web inspector shows the same “natural dimensions” (not sure what it’s called in English) for all the images and the network panel shows only the originally loaded image.

phiw13 · Today 09:06:11

jakob wrote #341663:

Are your srcsets working correctly? On my local test installation with that funny 3-aspect-ratio srcset test setup, I now get the same image served for all srcset and src images. The browser will load the most appropriate image for the respective viewport width on load but doesn’t augment the others when you increase viewport width. Rolling over the img src and srcset urls in the web inspector shows the same “natural dimensions” (not sure what it’s called in English) for all the images and the network panel shows only the originally loaded image.

Oh, indeed. Some odd things happening. In your example only the largest srcset image (1200w) appears in the code. My own srcset shows all 3 images in the code, but only the largest thumbnail image is listed in the network panel.

Edit, but to be honest I am not completely sure which image should be used in my case, given the pixel density e.a (viewed in the responsive design mode in Firefox and Safari with phone as selected device 2x or 3x DPR).

Last edited by phiw13 (Today 09:10:20)

jakob · Today 09:34:41

And when you do a fresh reload in a narrow viewport, then widen the window? As I understand it, srcset doesn’t post-load smaller images if a suitable resolution image already exists, as that would waste bandwidth (the ‘wide-to-narrow’ behaviour you see at the end of the video linked below): that means the “test” only works when transitioning from a narrow to a wide viewport. FWIW: I also have a retina display, and my image dimensions work(ed) in the test.

Previously it was as shown in this screen video, i.e. Textpattern provides the other thumbnail sizes to the browser as requested.
As of yesterday evening, the image that is first loaded is whatever the browser determines is best for the start viewport, but then the other requested sizes (including the src) are then “falsely” recognized as being that image, so when the browser asks for a new image to match a widened viewpoint, Textpattern points it to the same already cached image. From wide-to-narrow that’s not a problem, but from narrow-to-wide it will mean people getting blurry images. I can also imagine it impacting on lightbox setups where the larger image is required for the popup.

Bloke · Today 10:41:01

Not entirely sure why the behaviour has changed. All I did was move the location of the thumb cache. Hmmm….

etc · Today 11:06:55

It looks like there is some path problem. I have changed publish.php ($u5 -> $u4), which seems to correctly retrieve thumbnail parameters. But I get

Image does not exist: C:/xampp/htdocs/674.jpg

when accessing http://localhost/txp/images/thumb/w200-h100/674.jpg?token=..., as if the image path were resolved from the root. The right path (in my setup) should be C:/xampp/htdocs/txp/images/674.jpg.

And when trying to delete a thumbnail (missing txp chunk):

getimagesize(C:/xampp/htdocs/images/683.png): Failed to open stream: No such file or directory

etc · Today 11:12:40

gaekwad wrote #341661:

I won’t lie, I’ve missed this pre-release window code cramming…it’s been too long!

Served, Sir!

phiw13 · Today 11:28:18

I am still not sure if the behaviour actually changed that much.

This below works correctly. On small viewports (original loading) an image thumb w=362px is shown, scaling up horizontally I see the thumb as having 755 width (viewport is 680). There is a very brief flash when Firefox passes through one to other sizes constrain.

Browser source (I stripped the token out, added linebreaks):

<img srcset="http://e-t13.test/images/thumb/w400/images/161.webp 400w,
  http://e-t13.test/images/thumb/w600/images/161.webp 600w,  
  http://e-t13.test/images/thumb/w1200/images/161.webp 1200w"
  alt="A wild violet"
  sizes="(min-width: 1220px) 894px, (min-width: 960px) calc(55vw + 234px), 90vw" 
  src="http://e-t13.test/images/161.webp" itemprop="url contentUrl" width="1600" height="900">

Input, inside a if::article_image txp:images block

<img srcset="<txp:image_url thumbnail width='400' /> 400w,
  <txp:image_url thumbnail width='600' /> 600w, 
  <txp:image_url thumbnail width='1200' /> 1200w" 
  alt="<txp:image_info type='alt' />" 
  sizes="(min-width: 1220px) 894px, (min-width: 960px) calc(55vw + 234px), 90vw"
   src="<txp:image_url />" itemprop="url contentUrl" width="<txp:image_info type='w' />" height="<txp:image_info type='h' />">

–^–

Btw, Stef, my favourite testing site had originally (smd_thumnail) a /images/thumb/ folder, containing the thumbs used for admin purposes (default_thumb). The caches folder was then added, the original images left untouched.

jakob · Today 16:28:09

I spent the afternoon wrangling with a layout in Word. Came out bruised and battered but victorious … 🤕💪 … but I missed your posts as a result.

etc wrote #341667:

It looks like there is some path problem. I have changed publish.php ($u5 -> $u4), which seems to correctly retrieve thumbnail parameters. …@

After applying Oleg’s last changes, I now get the wrong path (too?):

http://newimage.test/images/thumb/w1200-c2x3/images/14.jpg?token=…

and no auto-generated thumbs appear. I would expect it not to have the second /images/ in the url:

http://newimage.test/images/thumb/w1200-c2x3/14.jpg?token=…

If I change that manually, I then get (but now the /images/ is missing of course):

Image does not exist: /Users/julian/Sites/newimage/14.jpg

phiw13 wrote #341669:

This below works correctly. On small viewports (original loading) an image thumb w=362px is shown, scaling up horizontally I see the thumb as having 755 width (viewport is 680) … Browser source (I stripped the token out, added linebreaks):

<img srcset="http://e-t13.test/images/thumb/w400/images/161.webp 400w,...

I see you get that url too. FWIW: I was (previously) getting the correct code output but not the correct images back.

I’ll do a fresh install when I get home to check it’s not legacy stuff from only applying successive code changes.

Bloke · Today 17:06:50

etc wrote #341667:

It looks like there is some path problem. I have changed publish.php ($u5 -> $u4)

SLIR currently requires the full URL path (relative to docroot) to the image, presented as its last set of params after the transformation block. In most cases that’s $u4 and $u5 for us.

But if you’ve moved your /images dir to, say, /images/test then it’ll currently break. I’m not exactly sure how to get round that, because instead of $url being:

array (
  'u0' => '/images/thumb/w200-h200-c1x1/images/10.jpg',
  'u1' => 'images',
  'u2' => 'thumb',
  'u3' => 'w200-h200-c1x1',
  'u4' => 'images',
  'u5' => '10.jpg',
)

it becomes:

array (
  'u0' => '/images/test/thumb/w200-h200-c1x1/images/test/10.jpg',
  'u1' => 'images',
  'u2' => 'test',
  'u3' => 'thumb',
  'u4' => 'w200-h200-c1x1',
  'u5' => 'images',
  'u6' => 'test',
  'u7' => '10.jpg',
)

A few things wrong with this:

We first of all match $u1 with $img_dir which is set to /images/test to we know we’re dealing with an image. That breaks if the $img_dir uses subdirs.
If we can fix the above, next it checks $u2 to see if it matches the thumbnail dir thumb. If you haven’t moved your images, it does. If you have moved them, it breaks and doesn’t think we’re dealing with a thumb resize, so all your thumbs break.
Assuming we can fix that, $u3 should be the transformation, but in the second situation with the moved images dir, it’s $u4. If we were in a subdir install, it should take care of itself because it removes ihu from the start, which includes the subdir portion of the URL. But I haven’t tested exotic multi-directory subdirs yet.
Then we’re left with the path to the image itself, which is the remainder, and may be $u4 + $u5. Or perhaps $u5 / $u6 and $u7. The bit we really care about is the ID and extension. The very last part. So we could simply ignore the $u array and read the last positional param in the array by checking the array length and then manually prepend $img_dir to it. OR, as a curveball, we could alter things so we only pass the ID+extension after the transform, and then inject the $img_dir by hand by tinkering with the way SLIR builds its paths. That might be preferable.

Inventive solutions welcome.

Textpattern CMS

Textpattern CMS support forum

#109 Yesterday 16:37:33

Re: Automatic thumbnails for Textpattern

Currently

Soon (hopefully)

Future (maybe) blazing performance

#110 Yesterday 16:58:54

Re: Automatic thumbnails for Textpattern

Bloke wrote #341659:

#111 Today 07:37:51

Re: Automatic thumbnails for Textpattern

Bloke wrote #341659:

#112 Today 08:50:44

Re: Automatic thumbnails for Textpattern

phiw13 wrote #341662:

#113 Today 09:06:11

Re: Automatic thumbnails for Textpattern

jakob wrote #341663:

#114 Today 09:34:41

Re: Automatic thumbnails for Textpattern

#115 Today 10:41:01

Re: Automatic thumbnails for Textpattern

#116 Today 11:06:55

Re: Automatic thumbnails for Textpattern

#117 Today 11:12:40

Re: Automatic thumbnails for Textpattern

gaekwad wrote #341661:

#118 Today 11:28:18

Re: Automatic thumbnails for Textpattern

#119 Today 16:28:09

Re: Automatic thumbnails for Textpattern

etc wrote #341667:

phiw13 wrote #341669:

#120 Today 17:06:50

Re: Automatic thumbnails for Textpattern

etc wrote #341667:

Board footer