Textpattern CMS support forum
You are not logged in. Register | Login | Help
- Topics: Active | Unanswered
Re: Images and the file system layout from 4.9.0
Thanks, Phil. Kicking and screaming, I’m looking into how to get Glide up and running ;-)
Offline
Re: Images and the file system layout from 4.9.0
Total drive by here, but if I’m looking for images by ID, I’ll personally okay with one big directory or one directory until it has a lot in it and then the system starts another directory for image IDs after that. If they’re sorted by ID, it’s super easy to scroll through even really large directories.
Second, totally agree on requesting image sizes via URL. When an image is uploaded:
1. Save original image
2. Create low res version of image
When txp tag is used and image size is included in URL, txp can output img element with a lowsrc attribute that points to the low res image. That means you can gate or limit image requests to manage potential DOS attacks and end-user still sees an image.
Most requests would return the right image. DOS or oddly high traffic could pump the breaks before someone has 5G of files added to their hosting account. (Or maybe txp has a max total file size where it cuts off and requires human intervention?
Offline
Re: Images and the file system layout from 4.9.0
agovella wrote #329796:
Total drive by here, but if I’m looking for images by ID, I’ll personally okay with one big directory or one directory until it has a lot in it and then the system starts another directory for image IDs after that.
That is a consideration. And a good one. All we need to do is decide on what constitutes ‘a lot’ – bearing in mind that there are an unknown number of thumbs going to be created – and do some simple modulus maths to work out which dir to shove an image in.
I’ve been doing some tests on this concept over the last few days, but still using subdirs to limit the number of dirs off the main /image folder by splitting them into ‘volumes’ for want of a better word. The concept is using the ID to determine which subdir it goes in by taking the last two digits. Super simple:
/images/vol001/01/1.jpg
/images/vol001/01/1_1280x960.jpg
/images/vol001/01/101.jpg
/images/vol001/01/101_1280x960.jpg
/images/vol001/01/201.jpg
/images/vol001/01/201_1280x960.jpg
...
/images/vol001/02/2.jpg
/images/vol001/02/2_1280x960.jpg
/images/vol001/02/102.jpg
/images/vol001/02/102_1280x960.jpg
/images/vol001/02/202.jpg
/images/vol001/01/201_1280x960.jpg
...
/images/vol002/01/25001.jpg
/images/vol002/01/25001_1280x960.jpg
/images/vol002/01/25101.jpg
/images/vol002/01/25101_1280x960.jpg
/images/vol002/01/25201.jpg
/images/vol002/01/25201_1280x960.jpg
...So as you see, I’ve set the hard limit to 25K images per volume, which is:
- 100 subdirs in each volume – easily navigable and not too taxing on an FTP client.
- 250 image IDs per subdir – ditto, even taking into account…
- … that someone might have 4 thumbs per image, that’s still 1250 image per subdir which is not ideal, but a reasonable compromise.
The number of volumes is never-ending but you’d need:
- 10 volumes for quarter of a million images
- 100 volumes for 2.5 million image IDs
- 1000 volumes for 25 million image IDs
So, before your FTP client starts grinding to a halt at anything above that number of volumes in your root /images dir, you’ve already got 25 million image IDs, which is 125 million actual images if you have 4 thumbs (plus the original) per ID.
To stress it to near-ax-level limits of 600 million images where the inode count might start to wobble in some filesystems, you’d need about 5000 volumes. That is going to start to hurt performance, but it’s the price we pay for not capping the volume number using a better algorithm.
The advantage to using the last 2 digits of the ID, modulo 25000, is it’s piss easy for humans to find an image given its ID. No need to run it through a hashing algorithm to find out where it lives. Just lop off the last two digits to figure out the subdir, then look at the first bit of the ID (essentially divide by 25k and round up) to locate which volume to find that subdir.
We could make it slightly more future proof by taking the last three digits of the ID. That gives us a thousand subdirs per volume. It’s pushing it, but that can then (in theory) support more images per volume, say 100K, which means if you do start to get into astronomical numbers of image files, your top-end volume number is still manageable.
Three-digit subdirs might be preferable, but I haven’t crunched the numbers to find out where the sweet spot of max images per volume lies yet. And 99% of users are going to live in vol001 anyway.
I’ll think about your ideas for resizing later. Just gotta run off now.
The smd plugin menagerie — for when you need one more gribble of power from Textpattern. Bleeding-edge code available on GitHub.
Txp Builders – finely-crafted code, design and Txp
Offline
#52 2021-04-13 15:00:23
- singaz
- Member
- Registered: 2017-03-12
- Posts: 150
Re: Images and the file system layout from 4.9.0
John Mueller goes over handy tips on how to optimize your images for Google Search.
1.
2.
3.
4.
5. descriptive filenames
6.
“Pick good filenames for your images” – it would be nice to teach the engine textpattern to do this
video 1.37 minutes – https://www.youtube.com/watch?v=XR3KjgLpT7A
Sorry my horror English. I’m learning textpattern, I’m learning English
Offline
Re: Images and the file system layout from 4.9.0
Bloke wrote #329582:
thumbnails: maybe a square cropped image can be used.
No idea how it would work. Cropped might cut too much. Rescaled may be too tiny to be of use. I’m open to ideas here. In any case, if we expose all the sizes you can choose your own thumb. And if you don’t like the grid, there’s always list view :)
Maybe we could copy Apple in their implementation of the Photos app: a simple toggle between square and original proportions?
Offline
Re: Images and the file system layout from 4.9.0
giz wrote #329809:
Maybe we could copy Apple in their implementation of the Photos app: a simple toggle between square and original proportions?
I’ll have to play with that and see how it works, thanks for the tip. Not spent much time in Photos.
I’ve had a few more ideas about ways to help set up image sizes and metadata during the upload process. How much of it is actually implementable I’m not sure. I’ve sent a few ideas to Phil for consideration. If anyone can tame my random blue sky thinking down to a tangible and usable interface (or tell me which bits are plain daft and should be jettisoned into space) it’s him.
Last edited by Bloke (2021-04-13 22:26:38)
The smd plugin menagerie — for when you need one more gribble of power from Textpattern. Bleeding-edge code available on GitHub.
Txp Builders – finely-crafted code, design and Txp
Offline