Textpattern CMS support forum
You are not logged in. Register | Login | Help
- Topics: Active | Unanswered
Re: Massive Lag Before Page Load
Bloke wrote #289019:
It seems to happen sporadically on some sites and I’ve never go the bottom of this, but would love to do so.
Here it seems to happen on every individual page, so I would start by checking the corresponding forms. And I get 25s lag even on a soft refresh (browser-dependent?)
Offline
Re: Massive Lag Before Page Load
Bloke wrote #289015:
And how many images do you have in the images folder? When the number of files starts to exceed a certain amount (usually over about 700 – 1000 is the sweet spot) things start to slow down when listing directories or performing file operations.
Well, being that this image exists: http://www.rossharvey.com/images/13848.jpg, which such a high ID, I bet the image folder has lots and lots of images…
@Ross, do you have a local copy of your website? does it shows the same symptoms when loading a page? If not, then the issue may be somewhere on your hosting/server (related or not with the image folder having lots of images)…
Re: lots of images in the image folder. Other CMSs and frameworks solve this by creating subfolders (sometimes, using a hashed value for the image file, and grouping the images in subfolders —and even subsubfolders— that start with the same two digits of a hash.
If the problem is indeed related to the amount of images on your image folder, you may want to try grouping them in folders starting with the same digits (ie. images/12/, /images/13/, etc) and then apply some rewrite rule (via .htaccess) to let the server find & serve the correct image. Yes, this may sound like a PITA, and you may have to do it manually to begin with and check if that improves your situation. Then, if it does improve (ie. lag disappears), then you could think on some automation..
Offline
Re: Massive Lag Before Page Load
maniqui wrote #289021:
Other CMSs and frameworks solve this by creating subfolders (sometimes, using a hashed value for the image file, and grouping the images in subfolders that start with the same two digits of a hash.
I’ve been mulling this over as a potential (future) approach to smd_thumbnail’s image handling actually. Or even a separate plugin that can be run with a higher priority to ensure it can intercept any further plugin requests for images and direct them to the right folder.
It may be overkill to put in core since it only affects a small percentage of people with tonnes of images, so a plugin isn’t an unreasonable expectation. But I’ll take discussion on this in another thread if anyone wants to chip in.
The smd plugin menagerie — for when you need one more gribble of power from Textpattern. Bleeding-edge code available on GitHub.
Hire Txp Builders – finely-crafted code, design and Txp
Offline
Re: Massive Lag Before Page Load
Thanks so much for looking guys. Amazed at the fast help and ideas you’ve all put forward. Your skills are way beyond mine – with this single Holly image, does that rule out the need for me to try removing the async stuff?
Bloke – you’re more than welcome to an admin login if you’d like a snoot around :¬)
Offline
Re: Massive Lag Before Page Load
I don’t have a local copy, and yes, there are thousands of images in the image folder. Can’t really avoid that with a photography blog :¬)
Mani – I don’t have the time to go through and manually set rewrites for certain image ranges and folders unfortunately.
Last edited by rossharvey (2015-03-13 14:03:39)
Offline
Re: Massive Lag Before Page Load
maniqui wrote #289021:
Well, being that this image exists: http://www.rossharvey.com/images/13848.jpg, which such a high ID, I bet the image folder has lots and lots of images…
I totally missed that. Good spot, Julián.
Edit: I have a vague memory that ext2 file systems under Linux have notable performance problems when you hit about 10,000 files in a directory, or something like that. It’s a possibility.
Another edit: Ignore that part I just added. I loaded 200 images in plain HTML and there’s issue with load time at all.
Last edited by gaekwad (2015-03-13 14:56:34)
Offline
Re: Massive Lag Before Page Load
I don’t use the <images> tag, the images are added as lines of HTML code (using smd_image_selector).
If anyone can help me fix it, I’ll either donate £150 to TXP or to the person directly. Not as an incentive to get help, but as a thank you for doing so. I’m out of my depth here and I very much appreciate the minds that have popped in to try and pinpoint it.
Offline
Re: Massive Lag Before Page Load
rossharvey, have you asked from the website’s hosting provider if they happened to make recently some changes to server settings?
I would ask them about that. Also could be worth testing what effect comes from creating a new Page template, saving it something like default2, then putting inside it something simple, like:
<txp:output_form form="struct_header" />
<txp:output_form form="struct_footer" />After that from the Sections some sections could be assigned to use default2. For example section called contact could be set to use default2. And then finally you could load http://www.rossharvey.com/contact and see if it’s still lagging. By changing code inside default2 you can sort of detect what parts of the code are causing problems.
Last edited by kuopassa (2015-03-13 16:03:05)
Offline
Re: Massive Lag Before Page Load
kuopassa wrote #289028:
rossharvey, have you asked from the website’s hosting provider if they happened to make recently some changes to server settings?
I would ask them about that. Also could be worth testing what effect comes from creating a new Page template, saving it something like
default2, then putting inside it something simple, like:
<txp:output_form form="struct_header" />...After that from the Sections some sections could be assigned to use
default2. For example section calledcontactcould be set to usedefault2. And then finally you could load http://www.rossharvey.com/contact and see if it’s still lagging. By changing code insidedefault2you can sort of detect what parts of the code are causing problems.
The contact page is working fine, I will however try creating a duplicate form for a blog item (the pages that have issues) and add bit by bit. Will take time, but hopefully it will help.
Unless it is related to the sheer number of images in the CMS. Then I’m screwed :¬)
Offline
Re: Massive Lag Before Page Load
rossharvey wrote #289029:
Unless it is related to the sheer number of images in the CMS. Then I’m screwed :¬)
Not necessarily. With a tiny bit of core hacking (adding a callback) and a small plugin which I’ve just written (untested yet), we could migrate all your images to sub-folders. Date-based schemes work best for keeping file numbers down:
/images/2014/0925/1.jpg
/images/2014/0925/2.jpg
/images/2014/0928/3.jpg
/images/2015/0604/1397896.jpg
...Since the image creation date never changes (ha! well, need to check that things stay the same at DST changes), it’s a simple process for the plugin to iterate over your images and move them to the new folders en-masse, and from then on, it’ll keep order on your behalf. Would need thoroughly testing first of course :-) And I’m not sure how well it would play with smd_thumbnail or other image manipulation plugins until they’re tweaked. In theory this little plugin falls back on the /images folder, but if you’ve already moved your images to subfolders, you don’t want another plugin trampling over that and using the base folder for its own stuff.
Anyway, that’s all detail. Bottom line: there’s a solution if it turns out to be the sheer number of files. The first step is ascertaining if that is actually the cause.
The smd plugin menagerie — for when you need one more gribble of power from Textpattern. Bleeding-edge code available on GitHub.
Hire Txp Builders – finely-crafted code, design and Txp
Offline
Re: Massive Lag Before Page Load
Blimey, that was quick Stef, thank you! Sounds like a handy plugin regardless. I’ll be doing some (simple) testing by elimination soon, although it’ll be related to form content only. That’s as far as my TXP debugging ability extends :¬)
Offline
Re: Massive Lag Before Page Load
Bloke wrote #289030:
Not necessarily. With a tiny bit of core hacking (adding a callback) and a small plugin which I’ve just written (untested yet), we could migrate all your images to sub-folders. Date-based schemes work best for keeping file numbers down:
/images/2014/0925/1.jpg...Since the image creation date never changes (ha! well, need to check that things stay the same at DST changes), it’s a simple process for the plugin to iterate over your images and move them to the new folders en-masse, and from then on, it’ll keep order on your behalf. Would need thoroughly testing first of course :-) And I’m not sure how well it would play with smd_thumbnail or other image manipulation plugins until they’re tweaked. In theory this little plugin falls back on the
/imagesfolder, but if you’ve already moved your images to subfolders, you don’t want another plugin trampling over that and using the base folder for its own stuff.Anyway, that’s all detail. Bottom line: there’s a solution if it turns out to be the sheer number of files. The first step is ascertaining if that is actually the cause.
Would it be easy to test the sub folder theory by moving only one image? The image used in the Holly post that was mentioned in this thread, perhaps?
Offline
Re: Massive Lag Before Page Load
Once the lag thing is resolved, I’d recommend following some of the advice laid out here too.
In particular:
- Use progressive JPEGs in future if possible
- Use a CDN (such as CloudFlare, which is free)
Offline
Re: Massive Lag Before Page Load
rossharvey wrote #289032:
Would it be easy to test the sub folder theory by moving only one image?
Not really. I could do with a few images to test at first, then I’d like to let it rip on a few thousand files at once and see how it goes. I need to write that part of the plugin yet though…
Pete and I are setting up an installation with a few thousand files in it to see if we can replicate what you’re experiencing, then I’ll use that to test the plugin.
This might be one of those things whereby it’s not one thing in isolation. We just need to chase down the variables and eliminate them one by one until Bob’s your mother’s parrot or something.
The smd plugin menagerie — for when you need one more gribble of power from Textpattern. Bleeding-edge code available on GitHub.
Hire Txp Builders – finely-crafted code, design and Txp
Offline
Re: Massive Lag Before Page Load
Bloke wrote #289036:
Not really. I could do with a few images to test at first, then I’d like to let it rip on a few thousand files at once and see how it goes. I need to write that part of the plugin yet though…
Pete and I are setting up an installation with a few thousand files in it to see if we can replicate what you’re experiencing, then I’ll use that to test the plugin.
This might be one of those things whereby it’s not one thing in isolation. We just need to chase down the variables and eliminate them one by one until Bob’s your mother’s parrot or something.
You guys are fast! Thank you, lost for words :¬)
Offline