Textpattern 4.8.3 Apache Bench

hcgtv · 2020-11-17 01:13:47

Textpattern 4.8.4, Live site setting, short tags are off, default install.

bert@jessie:~$ ab -n 1000 -c 10 http://localhost/projects/textpattern-dev/

Document Path:          /projects/textpattern-dev/ (4.8.4)
Document Length:        11955 bytes

Concurrency Level:      10
Time taken for tests:   6.991 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      12188000 bytes
HTML transferred:       11955000 bytes
Requests per second:    143.03 [#/sec] (mean)
Time per request:       69.913 [ms] (mean)
Time per request:       6.991 [ms] (mean, across all concurrent requests)
Transfer rate:          1702.45 [Kbytes/sec] received

TXP 4.8.4-dev runs nice on my local system, have not encountered any issues.

etc · 2020-11-17 14:03:30

I have tested 4.6.2, 4.7.3 and 4.8.4 with empty page (just html header and body, nada mas):

Document Path:          /textpattern462/
Document Length:        51 bytes

Concurrency Level:      10
Time taken for tests:   5.448 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      267000 bytes
HTML transferred:       51000 bytes
Requests per second:    183.54 [#/sec] (mean)
Time per request:       54.484 [ms] (mean)
Time per request:       5.448 [ms] (mean, across all concurrent requests)
Transfer rate:          47.86 [Kbytes/sec] received

-----------------------------------------------------

Document Path:          /textpattern473/
Document Length:        51 bytes

Concurrency Level:      10
Time taken for tests:   6.122 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      267000 bytes
HTML transferred:       51000 bytes
Requests per second:    163.34 [#/sec] (mean)
Time per request:       61.224 [ms] (mean)
Time per request:       6.122 [ms] (mean, across all concurrent requests)
Transfer rate:          42.59 [Kbytes/sec] received

-----------------------------------------------------

Document Path:          /textpattern484/
Document Length:        51 bytes

Concurrency Level:      10
Time taken for tests:   6.216 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      267000 bytes
HTML transferred:       51000 bytes
Requests per second:    160.87 [#/sec] (mean)
Time per request:       62.163 [ms] (mean)
Time per request:       6.216 [ms] (mean, across all concurrent requests)
Transfer rate:          41.94 [Kbytes/sec] received

It looks like we have lost ~7ms per request between 4.6 and 4.7. This is not surprising, given how much we have squeezed out of 4.6 and all we have added to 4.7 (themes, global attributes). We loose this 7ms mainly in preamble (preloading classes etc) and I don’t see how to compress it. There is more stuff to load in 4.7, period.

There is however a case where 4.8 is much faster than 4.6: 304 Not Modified responses. A regular visitor revisiting a not modified site will get a response up to 3 times faster in 4.8 than in 4.6.

hcgtv · 2020-11-17 14:49:39

etc wrote #326864:

I have tested 4.6.2, 4.7.3 and 4.8.4 with empty page (just html header and body, nada mas):

Glad to have someone confirm jessie’s findings.

TXP is still pretty fast compared to let’s say Joomla! or Drupal, which are the ones right behind WP in popularity.

Over the years I’ve seen the downloadable .zip file of TXP go from:

TXP 4.0.0.zip (250 KB (256,088 bytes)
TXP 4.5.0.zip (0.99 MB (1,043,767 bytes)
TXP 4.8.3.zip (1.87 MB (1,970,951 bytes)

Progress, I reckon, but I’m glad the core is still plenty fast.

Oh, just for reference, the poetry people:

WP 5.5.3.zip (13.3 MB (14,012,416 bytes)

etc · 2020-11-17 15:43:45

hcgtv wrote #326866:

Glad to have someone confirm jessie’s findings.

Without concurrent requests the game is more equal:

Document Path:          /textpattern462/
Document Length:        47 bytes

Concurrency Level:      1
Time taken for tests:   3.340 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      26300 bytes
HTML transferred:       4700 bytes
Requests per second:    29.94 [#/sec] (mean)
Time per request:       33.398 [ms] (mean)
Time per request:       33.398 [ms] (mean, across all concurrent requests)
Transfer rate:          7.69 [Kbytes/sec] received

----------------------------------------------------

Document Path:          /textpattern484/
Document Length:        47 bytes

Concurrency Level:      1
Time taken for tests:   3.371 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      26300 bytes
HTML transferred:       4700 bytes
Requests per second:    29.67 [#/sec] (mean)
Time per request:       33.707 [ms] (mean)
Time per request:       33.707 [ms] (mean, across all concurrent requests)
Transfer rate:          7.62 [Kbytes/sec] received

Over the years I’ve seen the downloadable .zip file of TXP go from:

TXP 4.0.0.zip (250 KB (256,088 bytes)
TXP 4.5.0.zip (0.99 MB (1,043,767 bytes)
TXP 4.8.3.zip (1.87 MB (1,970,951 bytes)

These are swallowed plugins :-)

hcgtv · 2020-11-17 15:55:57

etc wrote #326869:

Without concurrent requests the game is more equal:

Interesting, wonder why?

These are swallowed plugins :-)

I know, my very popular, #1 on the download list, is the hcg_templates plugin, and now it’s been swallowed by the core, I’m very happily upset ;)

Logically, I always thought the most popular plugins would make it into core one day.

etc · 2020-11-17 16:42:26

hcgtv wrote #326870:

Interesting, wonder why?

Physical limitations? When I switch from -c 1 to -c 10, 4.6.2 climbs from 33ms to 55ms. 4.8.4 being fatter, it might arrive to saturation more quickly. This would also explain why WP is so bad in your tests.

But your site must be very popular to get concurrent requests.

Bloke · 2020-11-17 17:21:40

hcgtv wrote #326866:

Over the years I’ve seen the downloadable .zip file of TXP go from: TXP 4.0.0.zip (250 KB (256,088 bytes) TXP 4.5.0.zip (0.99 MB (1,043,767 bytes) TXP 4.8.3.zip (1.87 MB (1,970,951 bytes)

The file size spike from 4.5.0 to 4.6.0 is mainly docblock comments and beginning to migrate to a more object-oriented code base, which has far more files that contain our text preamble.

From 4.7.0+ it’s also because every language file is bundled with core instead of being fetched from rpc.textpattern.com. Local vs network access = more deterministic response times (generally).

Last edited by Bloke (2020-11-17 17:24:00)

hcgtv · 2020-11-17 17:29:07

Bloke wrote #326881:

The file size spike from 4.7.0+ is also because every language file is bundled with core instead of being fetched from rpc.textpattern.com. Local vs network access = more deterministic response times (generally).

Yeah, I saw all those .ini files in setup.

Also, in 4.6.0 the textpattern/vendors folder showed up. Sneaky vendors :)

Bloke · 2020-11-17 17:50:32

hcgtv wrote #326882:

Also, in 4.6.0 the textpattern/vendors folder showed up. Sneaky vendors :)

Yeah, and one of those ‘vendors’ is /Textpattern. Tsk.

etc · 2020-11-18 10:43:24

What bugs me more than a slight mean runtime increase is the standard deviation, which is notably larger in 4.8.4. The minima In 4.6 and 4.8 are roughly the same, and the medians are close, but from time to time 4.8.4 yields much longer runs. The difference seems to come from include_once txpath.'/publish/taghandlers.php', but damn if I see why. I have even tried using 4.6 taghandlers.php in 4.8, that did not help much.

Bloke · 2020-11-18 11:35:02

Taghandlers is huge.

I noticed recently that someone (might even have been you?) did some tweaks there with the few tags we have in classes. Is there something more we could do here as we move forward to use inheritance?

I’m thinking predominantly of things like ‘file_download_list is a list tag that uses the conditional interface’ kind of setup.

In theory, once a base class is in memory, any other tags that are more specific variants of it should only require to load that class and already benefit from what’s already loaded. Classes will increase the memory footprint, sure, but that might give us more performance advantage as we’ll only load tags we need on a page – on-demand rather than injecting all of them into the global namespace every page. Not sure.

Last edited by Bloke (2020-11-18 11:36:21)

etc · 2020-11-18 17:26:27

If it were just a regular slowdown, I wouldn’t be so surprised (the size matters). But how to explain this:

4.6.2
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    31   34   3.3     31      47
Waiting:       31   34   3.3     31      47
Total:         31   34   3.3     31      47

-----------------------------------------

4.8.4
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    22   39  13.6     31      78
Waiting:       22   39  13.6     31      78
Total:         22   39  13.6     31      78

As you see, median connection times are the same (and min is even better for 4.8.4). But 4.6.2 looks much more stable, while 4.8.4 is skewed to the left, with rare but high max spikes.

etc · 2020-11-19 11:49:57

Bloke wrote #326915:

I noticed recently that someone (might even have been you?) did some tweaks there with the few tags we have in classes. Is there something more we could do here as we move forward to use inheritance?

I have actually done it the other way, considering that loading classes for this little is an overkill and replacing some of them (if_first/last) with plain functions. But for more specific tag groups (like comments) putting them in an autoloaded class is certainly a good idea, needs testing performance-wise.

gaekwad · 2020-11-19 11:53:52

I realise the A / B testing between versions is relative within the bounds of the same db and PHP setup, but are these tests using vendor-supported PHP releases, out of interest?

Bloke · 2020-11-19 12:41:45

etc wrote #326944:

for more specific tag groups (like comments) putting them in an autoloaded class is certainly a good idea, needs testing performance-wise.

Yes, I’m not sure of the performance impact. Classes tend to be slower, BUT taghandlers is so big and has so much repetition and pollutes the global function space, that it’s a hog.

If we’re clever with inheritance and create a hierarchy of tags with some interface traits we might be able to mitigate the slowdown by only autoloading the tags needed to process each page. That has to help, surely?

I’m thinking of a similar approach to the UI library whereby the ‘base’ class is loaded on-demand for any front-end tag encountered, and that houses a lot of the functionality like handling attributes and setting up internal class variables for things like $thing. Then maybe interfaces can take over to provide ‘type of’ functionality, and specific database queries or global/URL access do the dirty work. Finally ->render() (and/or magic echo) squirts out the result. Not sure.

That might be the wrong approach. Perhaps we should be looking at factory classes or adapters instead. But either way, it’d be nice to factor out a lot of the repetition. This could be a branch swinging off 4.9, where we can experiment, perhaps?

Textpattern CMS support forum

#16 2020-11-17 01:13:47

Re: Textpattern 4.8.3 Apache Bench

#17 2020-11-17 14:03:30

Re: Textpattern 4.8.3 Apache Bench

#18 2020-11-17 14:49:39

Re: Textpattern 4.8.3 Apache Bench

etc wrote #326864:

#19 2020-11-17 15:43:45

Re: Textpattern 4.8.3 Apache Bench

hcgtv wrote #326866:

#20 2020-11-17 15:55:57

Re: Textpattern 4.8.3 Apache Bench

etc wrote #326869:

#21 2020-11-17 16:42:26

Re: Textpattern 4.8.3 Apache Bench

hcgtv wrote #326870:

#22 2020-11-17 17:21:40

Re: Textpattern 4.8.3 Apache Bench

hcgtv wrote #326866:

#23 2020-11-17 17:29:07

Re: Textpattern 4.8.3 Apache Bench

Bloke wrote #326881:

#24 2020-11-17 17:50:32

Re: Textpattern 4.8.3 Apache Bench

hcgtv wrote #326882:

#25 2020-11-18 10:43:24

Re: Textpattern 4.8.3 Apache Bench

#26 2020-11-18 11:35:02

Re: Textpattern 4.8.3 Apache Bench

#27 2020-11-18 17:26:27

Re: Textpattern 4.8.3 Apache Bench

#28 2020-11-19 11:49:57

Re: Textpattern 4.8.3 Apache Bench

Bloke wrote #326915:

#29 2020-11-19 11:53:52

Re: Textpattern 4.8.3 Apache Bench

#30 2020-11-19 12:41:45

Re: Textpattern 4.8.3 Apache Bench

etc wrote #326944:

Board footer