Textpattern CMS support forum
You are not logged in. Register | Login | Help
- Topics: Active | Unanswered
Re: Making plugins first-class citizens
etc wrote #290601:
No, not values, we’d cache only arrays like
('section'=>'<txp:section />', ... )and still process<txp:section />on each iteration. It spares only multiplepreg_match_allcalls.
Parsing depends on the quoting type used, so you’d have to store somewhere which attributes in that array have to be parsed. Alternatively, you could choose not to cache when one of the attributes requires parsing.
Some benchmarks (doesn’t include calls to parse()). Times shown are average times in microseconds.- Zero column: how many times the same attribute set re-occurs.
- First column: original splat()
- Second column: splat() caching attributes except when one or more need parsing
- Third column: splat caching attributes, including when parsing is needed.
$attribs = ' ';
1 1.5732 2.8905 2.9502
2 1.4850 2.1611 2.2236
5 1.4505 1.6318 1.7352
10 1.4230 1.5409 1.6043
50 1.4353 1.3568 1.4323
100 1.4302 1.3264 1.4120
1000 1.4197 1.3085 1.3962Since empty attribute sets are very common, I think it’s safe to say that the difference here is close to zero.
$attribs = 'key="value"';
1 5.1557 6.6670 6.8195
2 5.1697 4.1464 4.2235
5 5.0831 2.4120 2.5232
10 5.0107 1.8785 1.9464
50 5.0365 1.4596 1.5477
100 4.9929 1.3663 1.4497
1000 4.9951 1.3785 1.4525$attribs = 'some="thing" and="other" thing="here" bool="1"';
1 13.6018 15.0676 15.1140
2 13.3633 8.3727 8.4256
5 13.4126 4.1424 4.1995
10 13.3064 2.7207 2.7986
50 13.1960 1.5853 1.6812
100 13.2398 1.4421 1.5210
1000 13.1873 1.3191 1.4341$attribs = 'some="thing" and=\'<txp:body>\'';
1 9.3086 10.3602 11.5734
2 9.1794 10.2371 6.4980
5 9.3709 10.4906 3.4871
10 9.0563 10.2729 2.4346
50 9.0690 10.0472 1.6399
100 9.0445 10.1068 1.5272
1000 9.0088 10.1408 1.4700As you can see, you lose time if attributes are unique (mainly the sha1 call which costs around 1 microsecond). But you definitely gain time if attributes occur multiple times. The question now is, does the reduction in time for non-unique attribute sets outweigh the loss of time for unique sets and the added code complexity?
Ideally the first column for each attribute set would have the exact same value in each row. What you see is fluctuations because of other programs claiming CPU time and a slow decrease in lower rows due to resetting of variables that happens less often there.
Offline
Re: Making plugins first-class citizens
ruud wrote #290603:
Parsing depends on the quoting type used, so you’d have to store somewhere which attributes in that array have to be parsed. Alternatively, you could choose not to cache when one of the attributes requires parsing.
Ah yes, we should set some do_parse flag in the array instead of strpos check.
Some benchmarks (doesn’t include calls to parse()). Times shown are average times in microseconds.
…
As you can see, you lose time if attributes are unique (mainly the sha1 call which costs around 1 microsecond). But you definitely gain time if attributes occur multiple times. The question now is, does the reduction in time for non-unique attribute sets outweigh the loss of time for unique sets and the added code complexity?
Speaking of no-attribute tags, we could even do the checking in processTags() to spare splat() call:
if (maybe_tag($tag))
{
$out = $tag(ltrim($atts) === '' ? array() : splat($atts), $thing);
}Now, parsing few dozens of one-timer tags takes no time anyway, I wouldn’t care about them even if we loose 10-15%. The main target of these optimizations imo are constructions like
<txp:article_custom limit="999">
<txp:if_variable name="flag" value="">...</txp:if_variable>
</txp:article_custom>And here you get 400% acceleration without real effort.
Offline
Re: Making plugins first-class citizens
etc wrote #290607:
Now, parsing few dozens of one-timer tags takes no time anyway, I wouldn’t care about them even if we loose 10-15%. The main target of these optimizations imo are constructions like
<txp:article_custom limit="999">...
I use that one:)
Yiannis
——————————
NeMe | hblack.art | EMAP | A Sea change | Toolkit of Care
I do my best editing after I click on the submit button.
Offline
Re: Making plugins first-class citizens
To get some real world data, what I would love to see is some examples of which tags (and attributes) are used within such monstrous limit=999 constructs and what the runtime and query times are (as shown in the HTML source while in testing mode). I’ll give you one of mine:
<txp:if_different><txp:posted format="%b %d, %Y" /></txp:if_different>
<txp:permlink><txp:title /></txp:permlink>Runtime: 0.3308
Query time: 0.056466Using the splat() optimisations discussed here would save 5ms, which is only 18% faster (not 400%). However, this affects only a single page on the website involved, which has 2500+ other pages which do not contain any looping tags and would therefore load slightly slower.
Offline
Re: Making plugins first-class citizens
Nobody says your site will be 4x faster, only splat(). But, -15% of 0.33s (and another -15% from parse() optimization) on heavy pages is more perceptible than +2% of 0.1s. And it costs almost nothing, just a little extra memory.
Edit: here was my real life example. And who cares about the real life, anyway :)
Last edited by etc (2015-05-09 22:13:41)
Offline
Re: Making plugins first-class citizens
Um… something is not right. “-dev” is the current development branch, “-parser” is my parser branch, “-splat” adds the splat optimisations to my parser branch.
http://undented.com/
TXP 4.5.7: Runtime: 0.1231 / Query time: 0.101193 / Queries: 46 / Memory: 881Kb
TXP 4.6-dev: Runtime: 0.1137 / Query time: 0.089166 / Queries: 46 / Memory: 919Kb
TXP 4.6-parser: Runtime:0.1136 / Query time: 0.090757 / Queries: 46 / Memory: 954Kb
TXP 4.6-splat: Runtime: 0.1147 / Query time: 0.093998 / Queries: 46 / Memory: 966Kb
Not much difference here. In all cases 0.023 seconds required excluding query time. Note the increase in memory usage.
http://undented.com/us?author=woj (hitting the 1000 articles limit)
TXP 4.5.7: Runtime: 0.3582 / Query time: 0.061736 / Queries: 29 / Memory: 1409Kb
TXP 4.6-dev: Runtime: 0.4825 / Query time: 0.070246 / Queries: 29 / Memory: 1451Kb
TXP 4.6-parser: Runtime: 0.4411 / Query time: 0.068155 / Queries: 29 / Memory: 1487Kb
TXP 4.6-splat: Runtime: 0.3846 / Query time: 0.061738 / Queries: 29 / Memory: 1495Kb
Looks like 4.6-dev somehow got a lot slower compared to 4.5.7. The optimisations I added help a little bit, but it’s still a lot worse than 4.5.7. WTF happened in -dev?!
Offline
Re: Making plugins first-class citizens
Strange figures, indeed, if run on the same server. Still a clear 25% gain of runtime – querytime in the second group, at the price of only 45Kb. It’s even surprising that regex parsing takes so much time, we should check if it’s used in some other loops.
Edit: if you use the default 4.6 article form, it might be heavier than your 4.5.7 form and explain the difference.
Last edited by etc (2015-05-10 09:29:02)
Offline
Re: Making plugins first-class citizens
I ran this on the exact same server with an identical database. Only difference is the TXP version.
I found part of it.
322ms with the registry code in processTags
284ms without that registry code.
So that’s 40ms. But the difference between 4.5.7 and 4.6-dev was 116ms. So we’re losing more elsewhere.
Offline
Re: Making plugins first-class citizens
ruud wrote #290622:
registry code in processTags
To cache, for me.
Another 25% <txp:article /> acceleration: reduce the number of UDF calls. If I replace in doArticles() (v. 4.5.7)
while($a = nextRows($rs))with
while($a = mysql_fetch_assoc($rs)) // and mysql_free_result($rs); after the loopand cache column maps in populateArticleData() (not sure it will work for 4.6 custom fields):
function populateArticleData($rs)
{
global $thisarticle, $production_status;
static $column_map;
if(empty($column_map)) $column_map = article_column_map();
if ($production_status === 'debug') trace_add("[".gTxt('Article')." {$rs['ID']}]");
foreach ($column_map as $key => $column) {
$thisarticle[$key] = $rs[$column];
}
}I gain 25% on ~300 articles. Edit: probably the result of MySQL caching, though.
Last edited by etc (2015-05-10 12:13:21)
Offline
Re: Making plugins first-class citizens
If we ever want TXP to be compatible with other databases, I don’t think it’s a good idea to use specific mysql functions.
I’m not sure where else 4.6-dev loses time compared to 4.5.7. Finding out would take me quite a bit of time and I wonder if – in the cases where this actually matters – it would be faster to optimize page loading by caching the result of the entire block of TXP code instead of micro-optimizing as we do now. There’s no point in gaining a few microseconds here and there if we lose more than that elsewhere in TXP.
For the example I gave, the runtime reduction is IMHO not enough to justify merging the suggested patches. Perhaps you can supply a comparison between txp-dev and txp-parser with a more complex bit of example code where it actually does pay off.
Offline
Re: Making plugins first-class citizens
Look, you care about loosing 0.1s in 4.6 (which I do too), and hesitate about gaining 0.1s by minor code changes. For me, it’s of the same order that storing textiled html rather than processing it each time — no real difference for a small article, but a better programming style. This said, caching blocks would be much more efficient, but probably yields a user decision whether to cache or not to cache. And txp users are not always tech savvies…
I’ve posted a simple proposal here, no much feedback atm.
Offline
Re: Making plugins first-class citizens
ruud wrote #290625:
If we ever want TXP to be compatible with other databases
We are considering Doctrine DBAL as a db layer for 4.6.
Offline
Re: Making plugins first-class citizens
ruud wrote #290625:
If we ever want TXP to be compatible with other databases, I don’t think it’s a good idea to use specific mysql functions.
You are right. Anyway, I think I’ve been mistaken by db cache, the real difference is rather 10 than 25%, sorry. However, static $column_map looks more natural in 4.5, but I don’t know how custom fields will be implemented in 4.6 and above.
Offline
Re: Making plugins first-class citizens
Let’s do some insane benchmarks. I’ve used wet_lorem_ipsum to generate 10.000 articles. I’m using your real world example to generate a page.
master:
23.169 0.483281 5638Kb
18.741 0.346480 5633Kb
20.098 0.420495 5634Kb
21.685 0.454559 5633Kb
22.746 0.494149 5633Kb
commit a406449 (new parser)
18.512 0.497731 5795Kb
17.086 0.470386 5794Kb
18.341 0.492000 5794Kb
18.387 0.467958 5794Kb
16.653 0.482271 5794Kb
commit c6994fd (parse_else)
16.940 0.424650 5769Kb
16.660 0.496910 5764Kb
17.226 0.473320 5769Kb
15.010 0.810858 5764Kb
15.207 0.459667 5764Kb
commit d970063 (various small optimisations)
14.164 0.411276 5757Kb
13.505 0.624712 5754Kb
13.964 0.492843 5754Kb
14.460 0.430434 5753Kb
14.862 0.501641 5753Kb
commit 842c22a (short tags)
14.767 0.434522 5756Kb
12.060 0.419026 5753Kb
14.329 0.456034 5753Kb
13.762 0.963029 5753Kb
15.032 0.456183 5753Kb
splat modifications:
13.508 0.480264 5775Kb
13.043 0.413073 5768Kb
13.324 0.458568 5768Kb
13.388 0.451295 5768Kb
12.419 0.481908 5768Kb
- adding short tag support doesn’t have to cost anything in performance.
- splat optimisations don’t seem to help.
- the various small optimisations may be more interesting to apply than rewriting the parser.
- this is definitely not a realistic real-world test.
- i need a more reliable (less fluctuating) way to benchmark.
Offline
Re: Making plugins first-class citizens
ruud wrote #290611:
Looks like 4.6-dev somehow got a lot slower compared to 4.5.7. WTF happened in -dev?!
Classes, probably. My guess is it’s a combination of the class loader, tag registry, singletons, adapters, factories, layers of abstraction and all that other OO “goodness”, which adds up to a more maintainable, albeit slower system.
Ah the good old days of functional programming where you knew exactly where you were: one stack, push, pop, easy. In the light of your findings (especially about the tag registry being called twice), your optimisations are going to be even more important to counteract the overheads of OO, if that is what is causing the slowdown.
The smd plugin menagerie — for when you need one more gribble of power from Textpattern. Bleeding-edge code available on GitHub.
Hire Txp Builders – finely-crafted code, design and Txp
Offline