Textpattern CMS support forum

You are not logged in. Register | Login | Help

#11 2013-01-21 17:40:55

wet
Developer
From: Lenzing, Austria
Registered: 2005-06-06
Posts: 3,267
Website

Re: Better (?) <txp:link_to_prev/next />

etc wrote:

Probably, but isn’t that true for SELECT ... WHERE ... ORDER BY ... query?

Apparently this is not universally true. According to this post MySQL uses a filesort algorithm to perform ORDER BY statements.

Filesort is based on quicksort. A clever quicksort implementation may be of O(log n) order while array_search is of O(n) order. MySQL’s runtime would only grow logarithmically with the haystack’s size.

about 250 articles

Try 2,500 or 25,000. How does this affect runtime?

Offline

#12 2013-01-21 17:42:19

etc
Developer
Registered: 2010-11-11
Posts: 3,198
Website

Re: Better (?) <txp:link_to_prev/next />

colak wrote:

Hi Oleg, one of my sites has over 1500 entries. If you want and promise you will not bring it to its knees I can grand you access. The problem is that it is not using the latest txp due to the postmaster plugin.

Hi Yiannis, I only can promise not to hurt it intentionally, and it would need an upgrade to 4.5.4 anyway. Thank you for the proposal, maybe once I’m through with it on my server…

I was wrong about 250 articles, they are not all in a same section, so I was actually testing on smaller datasets. With ~150 articles, the plugin is a little slower than the core, but the reason is not array_search, I keep investigating.


etc_[ query | search | pagination | date | tree | cache ]

Offline

#13 2013-01-21 18:07:14

colak
Admin
From: Cyprus
Registered: 2004-11-20
Posts: 7,288
Website

Re: Better (?) <txp:link_to_prev/next />

etc wrote:

and it would need an upgrade to 4.5.4 anyway.

I wish! We currently have over 2100 newsletter subscribers. Unless a solution is found for postmaster to work on 4.5+, my organisation’s committee will attack me for choosing the wrong software… That’s why I’m hanging with my teeth on the older txp version.


Yiannis
——————————
neme.org | hblack.net | LABS | State Machines | Respbublika! | NeMe @ github

Offline

#14 2013-01-21 21:35:27

etc
Developer
Registered: 2010-11-11
Posts: 3,198
Website

Re: Better (?) <txp:link_to_prev/next />

wet wrote:

According to this post MySQL uses a filesort algorithm to perform ORDER BY statements.

Filesort is based on quicksort. A clever quicksort implementation may be of O(log n) order while array_search is of O(n) order.

Robert, thanks for the link, interesting. Is it cleverly implemented – who knows? From what I have benchmarked, array_search takes less than 1% of the runtime. I don’t know how it scales, but strongly suspect that this memory-stored O(n) will not be a bottleneck until we have 100,000 or so records. And at this level, from a visitor’s pov, there is no real difference between 1 minute and 1 hour.

My empiric conclusion for this evening: if there is an index to use, like for Posted desc, the core function is about 2 times faster (of course). Otherwise, the plugin wins by 30% or so. Tested on ~250 articles.

Try 2,500 or 25,000. How does this affect runtime?

It would be nice to have a test txp install at hand, are there any?


etc_[ query | search | pagination | date | tree | cache ]

Offline

#15 2013-01-22 08:02:11

Gocom
Plugin Author
From: Helsinki, Finland
Registered: 2006-07-14
Posts: 4,532
Website

Re: Better (?) <txp:link_to_prev/next />

etc wrote:

O(n) will not be a bottleneck until we have 100,000 or so records

While the limits may seem miles away, it must be able to handle tens of thousands of articles with ease. A round number like 50,000 is very real and can be reached easily in the real world. I’m looking at site that has 30,000 items in the table right now.

It would be nice to have a test txp install at hand, are there any?

If you need bigger dataset, you could just insert dummy rows to the database with varying values. From which you can then run different test and post timings from. The machine you do those tests doesn’t matter.

if there is an index to use, like for Posted desc, the core function is about 2 times faster

That’s not terribly good.

Last edited by Gocom (2013-01-22 08:11:36)

Offline

#16 2013-01-22 08:16:58

Gocom
Plugin Author
From: Helsinki, Finland
Registered: 2006-07-14
Posts: 4,532
Website

Re: Better (?) <txp:link_to_prev/next />

jstubbs wrote:

That’s funny, changing the status of an issue to invalid renders it invisible, so I never saw Jukka’s reply.

The issue should stay intact no matter what the status is changed to. By default the issues list shows only open issues. The search form at the top can be used to filter it. May not be terribly intuitive, but that’s how Google code’s issue list works.

Last edited by Gocom (2013-01-22 08:19:09)

Offline

#17 2013-01-22 09:22:00

etc
Developer
Registered: 2010-11-11
Posts: 3,198
Website

Re: Better (?) <txp:link_to_prev/next />

Actually, quicksort is log(n) in memory, but n log(n) in time, so scaling should not be a problem if there is enough memory.

Gocom wrote:

That’s not terribly good.

For sort="LastMod desc" I get: core=0.007s, plugin=0.0044s, with additional benefice of link_to_first/last. :P

Edit: maybe, it’s possible to combine both approaches, checking existing indexes (SHOW INDEX FROM textpattern).

Last edited by etc (2013-01-22 09:34:38)


etc_[ query | search | pagination | date | tree | cache ]

Offline

#18 2013-02-01 11:58:50

wet
Developer
From: Lenzing, Austria
Registered: 2005-06-06
Posts: 3,267
Website

Re: Better (?) <txp:link_to_prev/next />

etc wrote:

It would be nice to have a test txp install at hand, are there any?

Try wet_lorem_ipsum to create any number of dummy articles.

Offline

#19 2013-02-01 12:17:34

etc
Developer
Registered: 2010-11-11
Posts: 3,198
Website

Re: Better (?) <txp:link_to_prev/next />

wet wrote:

Try wet_lorem_ipsum to create any number of dummy articles.

Nice toy for this week-end, thanks! I will try to fork to the default link_to_next if EXPLAIN uses indexes. Are there any changes in link_to_next in 4.6 to keep in mind?


etc_[ query | search | pagination | date | tree | cache ]

Offline

#20 2013-02-01 22:44:50

etc
Developer
Registered: 2010-11-11
Posts: 3,198
Website

Re: Better (?) <txp:link_to_prev/next />

Done some tests on 10000 articles sorted by LastMod using filesort. Retrieving the whole list is about 15-20% faster (0.065) than generating two prev/next links (2*0.04). Something (array_search or dirty code?) in the plugin adds extra 0.01s, but it is still a little faster than core. They don’t benefit from each others cache, of course. I have tested array_search on range(1,10^6) without any problem.

Last edited by etc (2013-02-05 09:03:33)


etc_[ query | search | pagination | date | tree | cache ]

Offline

Board footer

Powered by FluxBB