Can TXP handle many (thousands) of images?

lwray · 2006-01-10 23:20:46

How many images can Textpattern handle? I’ve just started uploading my photo galleries using a few plugins, but over time I might have as many as several thousand images. I guess I’m just looking for some reassurance that my work won’t be in vain at some point in the future!

(I know this doesn’t really fit here, but I didn’t know where else to put it…)

zem · 2006-01-10 23:29:59

See this FAQ. Though it’s about articles, the same is true of images.

marye · 2006-01-11 04:22:36

I had the same question (with a thought on a coding solution) in the following post:

http://forum.textpattern.com/viewtopic.php?id=13825

While articles post from the DB, the images are in a single directory. On some OS’s this can have scaling (reading, writing, managing, backup) problems when you get up to 4000-5000 files (remember you probably have one thumb per image, so that’s 2000-2500 image uploads).

If anyone else develped has a solution, pls share. I’ve done a prototype and there changes involved in a few places, and I was moving on to figuring out a plugin. My solution takes advantage that the uploaded file is renamed to the db key for its entry and uses these names to sort into subdirectories. It’s not ideal, but the path names are not stored in the DB table. A column extension would be a lot more work.

M.

zem · 2006-01-11 05:37:44

You’re missing a step: first, measure the extent of the problem.

There might be measurable performance issues, or there might not. I doubt any modern OS demonstrates significant degradation for common operations at 5000 files. I’d wager you’ll run into UI issues (e.g. how do you manage all this stuff) before anything else.

Jeremie · 2006-01-11 06:43:36

Beside UI issues, I do remember there are issues with large numbers of files in one directory. However, even with ext2 or similarly old filesystem, I’m not quite sure the threshold is that low.

marye · 2006-01-12 04:32:37

Zem,

You are correct, my concern was more about me than the OS (the managing, searching and the listing piece). My upper number, admittedly subjective, was based upon some forum entries in a gallery that ran in safe mode (hence one directory). The gallery seemed to fall apart around 2000-2500 photos according to some notes I read. Could be the text file length or the file mgmt code (I had no interest in finding out …). I had no intention of ever reaching even 200 in a folder for my own sanity.

For me, I’m happy with my solution with the file hacks I did – easy enough to replicate on future versions.

Mary

Last edited by marye (2006-01-12 04:36:57)

hcgtv · 2006-01-12 05:54:25

I can confirm that there are issues with certain galleries and a large number of images.

Most galleries keep their thumbnails in one directory, this cache directory can easily top 5000 files in no time if each image generates a few thumbnails.

In Linux, rm . freaks out with a directory with that many files, You’re forced to do pattern matching to clean it out. Also, most FTP programs will only display 2000 files per directory.

I shut down my gallery for this very reason, I had over 5000 images in seperate directories but the thumbnail directory was giving me headaches.

zem · 2006-01-12 06:38:58

Sure. Does any of this affect Textpattern?

lwray · 2006-01-12 08:41:24

zem, the FAQ sounds good, if a little overly eager to reassure!

I don’t anticipate any management issues — my current workflow to create the galleries doesn’t involve messing with the built-in image management, since I use aba_image_import. I’m more concerned with where Textpattern will hit its upper limit. Counting thumbnails, it’s more than likely I’ll get above 2,000 – 3,000 pretty fast.

hcgtv, were these problems with Textpattern specifically, or just general filesystems? I am running the site on a Linux server, but I’m not doing anything fancy with it since it’s not even mine. Any experience with large numbers of images in TXP?

hcgtv · 2006-01-12 16:07:11

lwray, my problems were filesystem based.

zem, nothing to do with Textpattern, just an observation.

Textpattern CMS

Textpattern CMS support forum

#1 2006-01-10 23:20:46

Can TXP handle many (thousands) of images?

#2 2006-01-10 23:29:59

Re: Can TXP handle many (thousands) of images?

#3 2006-01-11 04:22:36

Re: Can TXP handle many (thousands) of images?

#4 2006-01-11 05:37:44

Re: Can TXP handle many (thousands) of images?

#5 2006-01-11 06:43:36

Re: Can TXP handle many (thousands) of images?

#6 2006-01-12 04:32:37

Re: Can TXP handle many (thousands) of images?

#7 2006-01-12 05:54:25

Re: Can TXP handle many (thousands) of images?

#8 2006-01-12 06:38:58

Re: Can TXP handle many (thousands) of images?

#9 2006-01-12 08:41:24

Re: Can TXP handle many (thousands) of images?

#10 2006-01-12 16:07:11

Re: Can TXP handle many (thousands) of images?

Board footer