Making plugins first-class citizens

Bloke · 2015-06-26 09:10:31

ruud wrote #292118:

During the install, test if the uploaded TXP files are owned by a different user than the process running PHP scripts. If so, abort the install and recommend the user to switch to a more secure host.

I like that! Nice and simple. Good idea.

[SSDs] are, actually. Pick a decent host :)

OK, my fault, sorry. I should have checked.

IMHO, servers should use UTC or at least a timezone setting which isn’t affected by DST.

Amen. All this safe_strftime(), timezone and DST conversion junk is noise I could do without.

I like the fact that we have forms/templates in the DB. Storing them as flat files feels old-fashioned.

I like them in the DB too. If it weren’t for the VCS thing, I’d agree completely. It’d just be nice to have a way to fairly seamlessly manage them externally under version control as well if that suits your workflow, and make it less nebulous what’s going to happen to your site when you make changes. Might be onto something here. Not sure yet, needs a little thought and refining.

hcgtv · 2015-06-26 14:13:19

ruud wrote #292093:

Same style sheet, ab -n 100 -c 10, admittedly by performing the test from the server itself, to avoid the 100ms latency between me and the server from influencing the results.:
<txp:rvm_css>: 4000 requests/sec
<txp:css>: 50 requests/sec

Ah yes I remember, <txp:rvm_css> – tried to test v1.0 on 4.5.7, getting no output from the tag on the default page.

You won’t get similar results by putting forms/pages on the filesystem, because those would still be called from within a PHP script, while <txp:rvm_css> results in a request for a static file by the browser.

On my local VM, using Textpattern 4.5.7, I get the same times whether I use <txp:css format="link" media="" /> or <link rel="stylesheet" href="<txp:site_url />themes/default.css">. Apache bench reports between 4.9 and 5.1 seconds to complete ab -n 100 -c 10.

Did we say more tests!

My dynamic sites are on a shiny new VPS at DreamHost, 2000MB of RAM, 60GB of SSD disk space.
PHP 5.4.36 with XCache enabled on all domains.

Textpattern 4.4.1e – default template, only plugin installed is hcg_templates:

[ps######]$ ab -n 100 -c 10 http://nupusi.com/
Document Length:        3150 bytes
Concurrency Level:      10
Time taken for tests:   0.694 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      330400 bytes
HTML transferred:       315000 bytes
Requests per second:    144.13 [#/sec] (mean)
Time per request:       69.380 [ms] (mean)
Time per request:       6.938 [ms] (mean, across all concurrent requests)
Transfer rate:          465.05 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.8      0       3
Processing:    26   66  11.3     66      82
Waiting:       25   66  11.4     66      82
Total:         29   66  11.0     66      84
Percentage of the requests served within a certain time (ms)
  50%     66
  66%     71
  75%     74
  80%     77
  90%     80
  95%     82
  98%     82
  99%     84
 100%     84 (longest request)

“Even though you are on the right track – you will get run over if you just sit there.”
Will Rogers

hcgtv · 2015-06-26 16:50:38

This has been an interesting exercise, if nothing else to help me tweak Debian “Jessie” running in VirtualBox.

My previous development VM was a Debian “Lenny” 32bit install, later upgraded to “Wheezy”. This 80GB VDI, was assigned 1 CPU and 2GB of ram. When I installed “Jessie” in a new 100GB VDI, I opted for the 64bit version. So while I’ve been benchmarking Textpattern, I’ve also been tweaking the virtual machine.

Tweaking memory has it’s benefits to a certain degree, for my setup, “Jessie” with the Gnome 3 desktop, 2GB of ram feels about right. I have a total of 8GB of ram on my desktop, this leaves me plenty of ram to run the programs I need on the Windows 7 host side. Assigning 3GB of ram to the VM saw no real improvement in times, whether from Apache running a PHP app or my Perl script that generates the cross references over at PHPCrossref.com.

The one option I’ve never played with was the number of CPUs, so I decided to give the VM access to both the CPUs on the machine’s Intel E5700 Dual Core, running at 3.0GHz. Not a scorcher of a machine, but I’m not rendering Jurassic World, just doing 2D text wrangling. What a surprise I was met with, the 64bit linux operating system took to the dual CPUs like it was made for them. Brought my BogoMIPS up to 11,971, highest number I’ve ever achieved.

So here are the new tests – 2 CPUs, 2GB of RAM, PHP OPcache is off (not conducive to dev work):

Textpattern 4.5.7

Document Length:        9983 bytes
Concurrency Level:      10
Time taken for tests:   2.846 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      1027900 bytes
HTML transferred:       998300 bytes
Requests per second:    35.13 [#/sec] (mean)
Time per request:       284.624 [ms] (mean)
Time per request:       28.462 [ms] (mean, across all concurrent requests)
Transfer rate:          352.68 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    2   7.3      0      33
Processing:   120  277  91.2    270     814
Waiting:      118  260  88.1    256     769
Total:        121  279  92.0    270     814
Percentage of the requests served within a certain time (ms)
  50%    270
  66%    286
  75%    300
  80%    306
  90%    329
  95%    389
  98%    677
  99%    814
 100%    814 (longest request)

Textpattern 4.6-dev

Document Length:        12157 bytes
Concurrency Level:      10
Time taken for tests:   3.138 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      1238200 bytes
HTML transferred:       1215700 bytes
Requests per second:    31.86 [#/sec] (mean)
Time per request:       313.834 [ms] (mean)
Time per request:       31.383 [ms] (mean, across all concurrent requests)
Transfer rate:          385.29 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    2   6.2      0      27
Processing:   126  304  79.9    293     706
Waiting:      124  286  78.1    271     691
Total:        126  306  81.2    293     706
Percentage of the requests served within a certain time (ms)
  50%    293
  66%    314
  75%    321
  80%    331
  90%    355
  95%    401
  98%    676
  99%    706
 100%    706 (longest request)

After years of playing around with VirtualBox, my current setup has all the VDI files on a separate partition. This partition has been added to the Avast! exclusions, no need to waste precious disk reads on checking for viruses in Linux.

Next month, I plan on getting a 240GB SSD drive to run Windows 10. 240GB is plenty of room for Windows 10 and the 100GB Debian “Jessie” VDI disk file. This should increase my VM performance quite substantially, and I get to leave my Windows 7 install intact while I transition over to Windows 10.

My Lenovo desktop specs, if anyone is interested.

ruud · 2015-06-26 22:05:55

100GB? You can probably lose a zero.

hcgtv · 2015-06-26 22:19:42

ruud wrote #292153:

100GB? You can probably lose a zero.

A Debian 8.1 install with Gnome 3 as the default desktop is 3.7GB, so yes, 10GB is all you really need to setup a quick desktop/lamp server.

My need for 100GB is that I use my VM to work on PHPCrossRef. The cross reference dev environment occupies upwards of 50GB at the moment, and I plan on adding more projects. It’s pretty cool to use sed or perl on the huge source tree and look for stuff.

Shopping for an SSD now, can’t wait to compare the speed.

etc · 2017-12-06 16:00:18

Just a poll: short tags are there since 4.6. Is anybody using them?

Bloke · 2017-12-06 16:01:33

/me raises hand.

etc · 2017-12-06 16:03:36

Stef, I know for you :-)

colak · 2017-12-06 16:09:27

etc wrote #308129:

Just a poll: short tags are there since 4.6. Is anybody using them?

not yet:(

etc · 2017-12-06 16:24:43

Okay, since we’ve got a user, we can not revert it back, but could make it optional (on/off pref). There are at least two reasons you might want to switch it off:

your content/code contains <some::patterns /> that must not be processed by Textpattern;
you gain ~5% speed in long loops (e.g. large article lists).

Is it worth introducing a pref? The downside is that switching short tags off will break your (or others) code if it contains them.

Bloke · 2017-12-06 16:51:42

I’m only using it on my dev site. So don’t worry about backwards compat if it’s only me.

I agree it’s nice to be able to switch it off if there are any XML paths that might clash or (potentially) slow down your site as the entire XML tree is passed through the Txp parser.

Unless there could be some other way to shield portions of your code from the parser? <txp:hide> would have been a candidate, but now we can use that to do internal processing(?), I guess that’s not an option any more.

etc · 2017-12-06 20:19:31

Bloke wrote #308134:

Unless there could be some other way to shield portions of your code from the parser? <txp:hide> would have been a candidate, but now we can use that to do internal processing(?), I guess that’s not an option any more.

Interesting idea. We can modify <txp:hide /> so that <txp:hide process="">thing</txp:hide> will finally (after secondpass) output the unprocessed thing. Though thing will still be parsed (without processing), this happens only once, so there should be no noticeable slowdown. Will commit after some testing.

But this does not protect against copy-pasting others code with <short::tags /> and forgetting to enable them. Caveat utilitor?

Textpattern CMS

Textpattern CMS support forum

#109 2015-06-26 09:10:31

Re: Making plugins first-class citizens

ruud wrote #292118:

#110 2015-06-26 14:13:19

Re: Making plugins first-class citizens

ruud wrote #292093:

#111 2015-06-26 16:50:38

Re: Making plugins first-class citizens

#112 2015-06-26 22:05:55

Re: Making plugins first-class citizens

#113 2015-06-26 22:19:42

Re: Making plugins first-class citizens

ruud wrote #292153:

#114 2017-12-06 16:00:18

Re: Making plugins first-class citizens

#115 2017-12-06 16:01:33

Re: Making plugins first-class citizens

#116 2017-12-06 16:03:36

Re: Making plugins first-class citizens

#117 2017-12-06 16:09:27

Re: Making plugins first-class citizens

etc wrote #308129:

#118 2017-12-06 16:24:43

Re: Making plugins first-class citizens

#119 2017-12-06 16:51:42

Re: Making plugins first-class citizens

#120 2017-12-06 20:19:31

Re: Making plugins first-class citizens

Bloke wrote #308134:

Board footer