Automatic thumbnails for Textpattern

etc · 2025-12-11 17:34:13

Ah. Ok. Reverted it to $un, which is the last chunk, please test. Thumbnails seem to be generated now, but the deletion still fails.

Bloke · 2025-12-11 18:02:19

etc wrote #341672:

Ah. Ok. Reverted it to $un, which is the last chunk, please test.

Thank you. I’m trying to make the matches more robust now. Tricky…

Thumbnails seem to be generated now, but the deletion still fails. http://localhost/txp/images/thumb/w200-h100/674.jpg

Yes, for it to work your path should (I think) be:

http://localhost/txp/images/thumb/w200-h100/txp/images/674.jpg

I suspect the fact it’s ‘shifted’ across to the right by 1 array index becuase of your subdir isn’t helping here. I thought subdirs were handled but maybe not. Hmmmm.

It’d be nice if we didn’t have to specify the subdir in the final URL path.

Last edited by Bloke (2025-12-11 18:11:16)

etc · 2025-12-11 18:16:16

Bloke wrote #341673:

It’d be nice if we didn’t have to specify the subdir in the final URL path.

Absolutely. And it would be perfect if SLIR physically stored thumbnails in

//path/to/site/image_dir/thumb/w200-h100/674.jpg

jakob · 2025-12-11 18:22:19

I was wondering if we have an /images/ too many in the url? Isn’t the intention to have:

https://domain.com/images/thumb/w200-h100/674.jpg

in a standard (non-subdirectory) installation.

Bloke · 2025-12-11 18:33:10

jakob wrote #341675:

I was wondering if we have an /images/ too many in the url?

Not as it stands. It’s required. But yes, I think I’ll hack SLIR to remove the reliance on the full path at the end, if I can. It’s seriously annoying.

Bloke · 2025-12-12 09:21:59

etc wrote #341674:

And it would be perfect if SLIR physically stored thumbnails in //path/to/site/image_dir/thumb/w200-h100/674.jpg...

I’m getting there. After refactoring the way it worked last night I think I can hack out the md4 code and swap it for the physical path. If that works, we can just jump straight to your proposal of using the absence of thumb to trigger generation.

It makes garbage collection more tricky but I’ll disable that for now.

I’m also considering removing the secondary request cache. Doesn’t seem worth the hassle, especially with the symlink rubbish that doesn’t work cross-platform.

phiw13 · 2025-12-12 09:39:11

Bloke wrote #341678:

I’m getting there. After refactoring the way it worked last night I think I can hack out the md4 code and swap it for the physical path. If that works, we can just jump straight to your proposal of using the absence of thumb to trigger generation.

Fwiw, you latest code changes (path) work pretty well here with the various test articles and my various shortcodes.

etc · 2025-12-12 10:13:25

Bloke wrote #341678:

I’m getting there. After refactoring the way it worked last night I think I can hack out the md4 code and swap it for the physical path.

That would be fab and much easier on server resources for big image galleries. Thank you!

Bloke · 2025-12-12 10:20:02

phiw13 wrote #341679:

Fwiw, you latest code changes (path) work pretty well here with the various test articles and my various shortcodes.

Fabulous. That’s great to know.

If anybody does want to jump in, please be my guest. I won’t be able to look at this until late this afternoon/evening.

I think to change the paths to store as Oleg suggests means altering requestURL() and the function above it that makes the md4. Just remove the md4, and alter the URL function to grab the trailing path of he URL verbatim, after sanitisation.

Then create any w200-h300-c2x3 directory etc that doesn’t exist as part of the SLIR squishing, and disable garbage collection in defaultconfig.php.

After that, we should be good to get txplib_misc.php to create a thumb URL based on the passed params and add it to the src attribute directly.

Bloke · Yesterday 18:36:20

A few steps forward. We’re now solely delivering cached automatic thumbnails and, any time a thumbnail doesn’t exist, Textpattern’s 404 handler creates one. This seems to be working well. A few things to note:

If you choose to delete an auto thumbnail, it isn’t actually deleted. It stays in the cache but the thumbnail type is set to 0 (no thumb) so you don’t see it. If you flick back to Automatic, the thumb will immediately reappear.
Thumbnail deletion will be handled by the garbage collector (when I’ve fixed it to operate with the new way of working). So at some unspecified point while the site is in operation, images over a certain age (which I’ll probably expose as a define value to be overridden in config.php) may be purged.
The garbage collector doesn’t run on every request, only a percentage probability of it running. By default that’s about 1 in 200 requests (0.5% chance), which we may well keep. So if your images don’t immediately disappear after expiry, that’s why.
Using multiple tags to render identical images on the same page still doesn’t work properly. See below.

Question: for garbage collection, what sort of timeout is a reasonable limit to impose on the life of a thumbnail? By default it’s a week, which seems a bit… short to me for a website. I was thinking more along the lines of 6 months. What do you think?

For the multiple tags scenario, here’s what I think is happening:

<!--Txp asks for the cached thumbnail. The system says "there isn't one"
generates a token, and returns the URL to the (non-existent) cached image -->
<txp:thumbnail id="4" width="500" />

<!--Txp asks for the cached thumbnail. The system says "there isn't one"
because the first is still in the output buffer waiting to run on secondpass,
generates a differrent token, and returns the URL to the (non-existent)
cached image -->
<txp:thumbnail id="4" width="500" />

<!-- ditto -->
<txp:thumbnail id="4" width="500" />
...

So all the thumbnail <img> tags are getting the same URL in their src attribute. Txp then runs secondpass to interpret the buffer contents, sees the first <img> tag and tries to fetch the thumbnail. It doesn’t exist. The 404 triggers publish.php to attempt to render it. It checks the token but it’s already been replaced by the token generated by the last thumbnail attempt on the page, so when it computes the checksum, it comes up with a different answer and thinks you’re hacking the system. It returns no thumbnail.

The second image tag suffers the same fate: no thumbnail is returned.

The third one matches the token, generates the thumb and all is good… except the first two tags have already been and gone further up the page so it’s too late for them to ‘see’ the newly created thumbnail.

My initial thought is that this is bypassable by using the [n] syntax to force the last identical thumbnail tag to run first, then the cached image will be built and the first two will (should) return the cached content. Not tried it.

But thinking further, maybe it’s better if we can hack the security system so it will always generate the same token for any given image size and thumbnail id. That way, the system will overwrite the token in the database with the same token on subsequent identical tags, meaning they’ll all be valid. Or (perhaps better) it will see that there already is a token for that particular image combo and just return the same token as-is.

Unless you can fabricate the token in the first place for that request, it’s not going to be much less secure. Because you still won’t be able to use the same token to generate w=100, w=101, w=102, w=103 and perform denial of service, since each width still has a unique token.

I’ll look into doing the latter, but I’d still be interested to find out if the tag reordering / rendering hack works.

etc · Yesterday 19:19:30

Ha, that’s why they seem not browser cacheable: different tokens! Sure, fixed per params tokens would be more performant.

etc · Yesterday 22:15:27

Bloke wrote #341684:

I’d still be interested to find out if the tag reordering / rendering hack works.

I don’t think so. If I get it right, thumbnails are not generated on tags parsing, but when a browser tries to fetch them. Whatever order of parsing, txp will output something like

        <!--Txp asks for the cached thumbnail. The system says "there isn't one"
generates a token, and returns the URL to the (non-existent) cached image -->
<img src="http://path/to/site/images/thumb/cache/rendered/w200/672.png?token=a2ce5ac52c472e7734833118a9150539423c1b935680015009cbffbb8bfe8499afdded611e74" alt="" width="200">

<!--Txp asks for the cached thumbnail. The system says "there isn't one"
because the first is still in the output buffer waiting to run on secondpass,
generates a differrent token, and returns the URL to the (non-existent)
cached image -->
<img src="http://path/to/site/images/thumb/cache/rendered/w200/672.png?token=0f47a482071ea2dbf78810a46e6344d1e6991aa3133da92ccc0eda58d4d52753d1858a5a6734" alt="" width="200">

<!-- ditto -->
<img src="http://path/to/site/images/thumb/cache/rendered/w200/672.png?token=a839fe02b9186f34da2a6daa16d512b03dafb1ce337dffb31eba135de464177e281fb99208b7" alt="" width="200">

and send it to the browser, who will try to fetch three thumbnails, triggering their generation. But only one of them has a valid token and will be generated. Dunno what will happen for identical tokens, probably some kind of concurrent thumbnails generation, which we’d better prevent.

Textpattern CMS

Textpattern CMS support forum

#121 2025-12-11 17:34:13

Re: Automatic thumbnails for Textpattern

#122 2025-12-11 18:02:19

Re: Automatic thumbnails for Textpattern

etc wrote #341672:

#123 2025-12-11 18:16:16

Re: Automatic thumbnails for Textpattern

Bloke wrote #341673:

#124 2025-12-11 18:22:19

Re: Automatic thumbnails for Textpattern

#125 2025-12-11 18:33:10

Re: Automatic thumbnails for Textpattern

jakob wrote #341675:

#126 2025-12-12 09:21:59

Re: Automatic thumbnails for Textpattern

etc wrote #341674:

#127 2025-12-12 09:39:11

Re: Automatic thumbnails for Textpattern

Bloke wrote #341678:

#128 2025-12-12 10:13:25

Re: Automatic thumbnails for Textpattern

Bloke wrote #341678:

#129 2025-12-12 10:20:02

Re: Automatic thumbnails for Textpattern

phiw13 wrote #341679:

#130 Yesterday 18:36:20

Re: Automatic thumbnails for Textpattern

#131 Yesterday 19:19:30

Re: Automatic thumbnails for Textpattern

#132 Yesterday 22:15:27

Re: Automatic thumbnails for Textpattern

Bloke wrote #341684:

Board footer