smd_prognostics: monitor your Txp installation for suspicious activity

maverick · 2010-11-12 01:47:05

I’m away from the computer for the rest of the night but will try some of path suggestions when I get back.

Some things are duplicated. Some are not – like admin themes – but with the sym links it looks like the files of each them show up repeatedly. With the various sym links the files showing in the panel add up fast. As in several thousand.

That’s a lot to wade through to find the ones to monitor.

My other thought was more along privacy/security. It’s not an issue as long all the domains are for sites I run. But if I were to use it on an install powering multiple sites for various other people and they have publisher privileges it wouldn’t seem very private.

The solution may be individual installs.

maverick · 2010-11-12 02:07:30

If it would help your development I can give you a login to this multisite install. I use it as a sandbox and for a couple of personal sites – so no issues there.

Bloke · 2010-11-12 18:53:17

Thanks Mike, yes it might.

In the meantime, v0.12 is available. It’s highly recommended to upgrade to it and visit the Setup tab then Save you settings as there’s a new pref available. Features in this release:

Added file quantity check so you can now process your list of files in bite size chunks, meaning you can run it more often with fewer files each time
Fixed white screen of death on Files Save (binary files are now left unprocessed)
Improved performance

I’ve got it live on my site and it seems to be working now with public side clicks enabled. Hopefully that’s the end of the white screen of death on my site! Let me know how you get on.

Last edited by Bloke (2010-11-12 18:54:21)

thebombsite · 2010-11-12 20:59:07

Bloke wrote:

2) checking a lot of files (around 700 of them). So perhaps the script is hitting some PHP resource limit or something on the public side due to the amount of other stuff going on (guessing the load is lower on the admin side)

Is that all Stef? When I first installed the plugin I was looking at around 3500 files! It didn’t seem to want to know when I tried to select all of them so maybe there are limits that it has to work within.

Anyway I figure that I shall have to be very specific about the folders and individual files I specify for the top level directory (thebombsite) and have separate plugins in each Txp install in sub-directories. I mention that in case other users have several sub-sites.

But other than that I haven’t encountered any problems. ;)

Oh and it looks great in Vitraux, including the help docs. :)

Last edited by thebombsite (2010-11-13 01:33:50)

thebombsite · 2010-11-13 01:32:27

From the Help file:-

The message delivered to you contains a link to the Acknowledgement page (which can also be reached by clicking the Alarms button from the Prognostics tab).

I did an SVN update and it sent me an email about the files that I had changed. OK so far. When I clicked on the contained link it simply took me to my site and not to admin. There was no “/textpattern/index.php” included in the link. I’m thinking that’s wrong.

Bloke · 2010-11-13 09:08:28

thebombsite wrote:

When I first installed the plugin I was looking at around 3500 files!

Oh my giddy aunt. Didn’t your admin side slow down at all? In versions prior to v0.12 it checked every file whenever the timeout was reached so you could be waiting a long time. Not to mention the fact that the Files tab will:

take ages to load
take forever to do a select all, in Firefox at least: click the top file, scroll to the bottom (if you have enough wafer-thin scrollbar to grab :-) and shift-click. In mine it goes back to the top and scrolls through the entire list, selecting each file as it goes. Very boring to watch. Dunno if there’s anything the plugin can do to help here (any ideas anyone?)

Anyway I figure that I shall have to be very specific about the folders and individual files I specify for the top level directory (thebombsite) and have separate plugins in each Txp install in sub-directories.

Yes. Depending how you set it up, you might be better off selecting a smaller quantity in the main site and then install prognostics to check the specific files in each sub-dir. It’ll be way more efficient and keep your sites nippy, especially if you also set a fairly small amount of files to check each click. You can still collect all checksum files in a single dir (use the Unique prefix option) so your sites don’t get cluttered with yet more files.

When I clicked on the contained link it simply took me to my site and not to admin. There was no “/textpattern/index.php” included in the link. I’m thinking that’s wrong.

Ah, right. Well caught. If your intrusion is detected on the public side there’s no ‘texpattern’ directory in the URL so the destination URL is wrong. I’ll need to address that, thanks.

btw, there’s also a slight bug in v0.12 on the Alarms panel. Even though your files are only checked in small batches everywhere else, on the Alarms panel it’s supposed to check them all so it always gives you a complete picture of what’s been changed. It’s not doing that at the moment. Not a show stopper, but slightly annoying. Simple one-line fix; I’ll issue a new version later.

Oh and it looks great in Vitraux, including the help docs. :)

I’m checking all my plugins on both Vitraux and classic now as standard ;-)

Last edited by Bloke (2010-11-13 09:10:11)

hcgtv · 2010-11-13 15:19:54

Upgraded to v0.12 on PHPXref, noticing a lag on a page where I have feeds supplied by SimplePie. The feeds page does display, but it takes about 10 seconds, where it should be displayed instantly, since the feeds are refreshed every hour via a cron job.

This site is running TxP 4.2.0, should I upgrade to 4.3.0?

Edit: A couple of sites are feeding slowly this morning, so it’s not the plugin.

Last edited by hcgtv (2010-11-13 15:42:58)

Bloke · 2010-11-13 15:47:43

hcgtv wrote:

noticing a lag on a page where I have feeds supplied by SimplePie.

Hmm, the plugin shouldn’t care about the content. In theory it just runs and quits. TXP 4.3.0 might help, but the only major difference is the fact that the prefs work nicer so I doubt that’ll help.

Questions:

1) How many files are you monitoring, out of how many overall?
2) What’s the plugin timeout value?
3) How many files per run are you processing?
4) Have you saved the prefs since you upgraded? The new setting won’t take effect until you Save
5) What priority is the plugin? Does it make any difference if you back it off a bit?
6) Is there anything else on that Page that you think might interfere? If you can post the code or any relevant form snippets it might help me figure out what’s causing this
7) If you disable the plugin does the page consistently load quickly?

Very odd behaviour in all. Will have to put me thinking cap on based on your findings from the above questions. Thanks in advance.

Edit after reading your edit: oh, ok. Must admit that the Internet is horribly slow here today. Think there may be some global DNS/router issues somewhere. It’s horrendous.

Last edited by Bloke (2010-11-13 15:48:57)

maverick · 2010-11-13 16:35:23

maverick wrote:

With the various sym links the files showing in the panel add up fast. As in several thousand.

thebombsite wrote:

I was looking at around 3500 files!

Bloke wrote:

Oh my giddy aunt.

Ditto on Stuarts number — when I said several thousand, mine was 3546.

# take ages to load

Surprisingly, not as bad as you might think

take forever to do a select all,

Keyboard shortcut to select all was speedy. However, selecting all led to the white page of death. Selecting a smaller amount of files worked okay.

thebombsite wrote:

When I clicked on the contained link it simply took me to my site and not to admin. There was no “/textpattern/index.php” included in the link. I’m thinking that’s wrong.

maverick wrote:

a href=“http://www.domain.com//index.php?event=smd_prognostics&step=smd_prognostics_ack&smd_prognostics_suppress=1”>Acknowledge alarms

Ditto – I noticed that even if I had used a traditional admin install (domain.com/textpattern), that “textpattern” was missing from the url (see above). Though my url did give the index.php

Bloke wrote:

EDIT: yah nuts. Yeah it uses hu to return the path to the sitefor acknowledging alarms which I believe is wrong in multi-site. Hmmm. Needs some thought.

Other plugins are running into the same issue

The “ihu” for hosting images on a subdomain is what made me wonder if creating another preference for the admin subdomain url would work.

Bloke wrote:

Thanks Mike, yes it might.

I have to take off again for a while, but I’ll set up a login and send it as soon as I get a chance.

Mike

Bloke · 2010-11-13 17:37:06

maverick wrote:

Keyboard shortcut to select all was speedy. However, selecting all led to the white page of death.

That should have been fixed in v0.12. If it’s still doing it, let me know. Wish FireFox had a keyboard shortcut for ‘select all items in select list’. Or at least if it does, I don’t know about it. Perhaps the slowness is not a FireFox thing but a Windows thing…

v0.13 is in the works still. Refactoring some stuff and tweaking a few things on the journey.

“textpattern” was missing from the url

Yeah that was a stupid oversight on my part. Fix on its way.

The “ihu” for hosting images on a subdomain is what made me wonder if creating another preference for the admin subdomain url would work.

I saw your post and it’s not the first time I’ve wished for such a constant. I’m sure someone cleverer than me can figure out how to patch TXP to do this.

I have to take off again for a while, but I’ll set up a login and send it as soon as I get a chance.

Brill, thanks.

Incidentally I’ve just had notification of a suspected injection hit at phpxref. Prognostics caught it and prevented it, which I’m pretty chuffed about. I’m dissecting the frognostics and adding the info to the knowledge base.

thebombsite · 2010-11-14 00:19:54

frognostics – love it. I think you should apply to the O.E.D. for inclusion in the next edition. ;)

thebombsite · 2010-11-14 16:11:39

Here’s a good one considering my “special needs”. I was doing an update to the “Vitraux” php file and when I saved it threw me a message:-

Your request has been denied by smd_prognostics. Nice try.

and it’s nice to know that it works but…

Now I’ve removed all the “/themes/” files from the file list but I still get the message. I should point out that my modifications were actually saved. I’m thinking there is probably something else I should be doing??

Last edited by thebombsite (2010-11-14 16:12:19)

Bloke · 2010-11-14 16:54:05

thebombsite wrote:

Your request has been denied by smd_prognostics. Nice try.

If you haven’t already, turn off the Admin-side check for SQL Injections. That (experimental) feature has some holes in it right now which means that if you have certain content in the thing you’re trying to save on the admin side it’ll trigger the injection warning. Currently if your content contains # or -- or any SQLish commands like drop, insert, update and so on it’ll trigger, which is very annoying when trying to save the smd_prognostics plugin itself as it contains all those words and symbols :-)

I’m working on ways round this. Currently I have a few avenues to explore:

Allow you to specify admin-side events and steps that you wish to bypass the SQL injection. Primarily this might be Pages/save, Forms/save, Stylesheets/save, possibly Articles/save and things like ied_plugin_composer/save and smd_admin_themes/save, among others
Allow you to only notify that the injection has taken place, or silently capture it and then continue instead of dying
Something else that may come to me randomly as I think this through, or that someone else suggests as a viable alternative
Get rid of the stupid admin-side feature altogether

Best advice: switch it off for now until we’ve figured out the best way to do it.

Last edited by Bloke (2010-11-14 16:56:04)

ruud · 2010-11-14 19:41:28

Incidentally I’ve just had notification of a suspected injection hit at phpxref. Prognostics caught it and prevented it, which I’m pretty chuffed about. I’m dissecting the frognostics and adding the info to the knowledge base.

Knowledge base? If TXP is vulnerable, let’s fix the bug. If not, just ignore it.

Allow you to specify admin-side events and steps that you wish to bypass the SQL injection.

Another way to deal with it: deactivate the account of the user which triggers this. Just because a user can’t do SQL injection, doesn’t necessarily prevent him/her from doing other damage like uploading massive amounts of files, changing articles and so on (depending on privileges).
For single user installs, checking admin side actions probably isn’t interesting.

A feature that would be nice to have (if it isn’t there already): new TXP version notification. The best way to stay safe is to keep software up-to-date.

Looking at amount of code between checking and updating the smd_prognostics_lastcheck value. What’s the chance of a race-condition occurring (file/db-update) or multiple checks happening at once?

Another feature that would be nice to have: being able to specify a preferred time slot (outside peak hours) in which to check files. Basically imitating cron for those poor souls on Windows hosting ;)

Bloke · 2010-11-14 20:29:01

ruud wrote:

Knowledge base? If TXP is vulnerable, let’s fix the bug. If not, just ignore it.

Sorry, I meant my own knowledge base. I’m using the output from the various attacks to find ways to improve the plugin and either predict or at least add options to help people fight the prospective attacks. If we happen to find a demonstrable TXP vulnerability along the way, then that’ll be fixed pronto.

Another way to deal with it: deactivate the account of the user which triggers this.

Not sure I follow. An admin-side “attack” is one that begins http://site.com/textpattern/some_file?attack=content (or a POST equivalent). Since the callbacks are different for the two sides, the only way I could see to detect if someone on the outside was targetting something on the inside was to add a callback on head_end — the earliest point a plugin can run, istr. On the public side I have the pretext callback to attach to. So it’s not an attack from the ‘inside’ as such, but a side-effect is that it affects logged-in users too. Which stunning realisation has just led me to the fix: if $txp_user is set during an “attack” (primarily a save operation) don’t run the prognostics check. Simple. Thank you!

What I will probably do when I implement this is remove the distinction between admin and public sides and just have SQL Injection on/offm, since at the moment it is a tad confusing.

A feature that would be nice to have (if it isn’t there already): new TXP version notification.

A good idea, thanks. I’ll see if I can find a way to grab that on the advice page.

What’s the chance of a race-condition occurring (file/db-update) or multiple checks happening at once?

I’ll have to check. I have noticed that if you set the timeout too short and add quite a few files on the Files page, before the time the checksums file has been updated, the prognostics warning fires that the checksums file has changed!

being able to specify a preferred time slot (outside peak hours) in which to check files.

That would be neat yeah. Will see if I can find a way to do it.

Last edited by Bloke (2010-11-14 20:31:34)

Textpattern CMS

Textpattern CMS support forum

#16 2010-11-12 01:47:05

Re: smd_prognostics: monitor your Txp installation for suspicious activity

#17 2010-11-12 02:07:30

Re: smd_prognostics: monitor your Txp installation for suspicious activity

#18 2010-11-12 18:53:17

Re: smd_prognostics: monitor your Txp installation for suspicious activity

#19 2010-11-12 20:59:07

Re: smd_prognostics: monitor your Txp installation for suspicious activity

#20 2010-11-13 01:32:27

Re: smd_prognostics: monitor your Txp installation for suspicious activity

#21 2010-11-13 09:08:28

Re: smd_prognostics: monitor your Txp installation for suspicious activity

#22 2010-11-13 15:19:54

Re: smd_prognostics: monitor your Txp installation for suspicious activity

#23 2010-11-13 15:47:43

Re: smd_prognostics: monitor your Txp installation for suspicious activity

#24 2010-11-13 16:35:23

Re: smd_prognostics: monitor your Txp installation for suspicious activity

#25 2010-11-13 17:37:06

Re: smd_prognostics: monitor your Txp installation for suspicious activity

#26 2010-11-14 00:19:54

Re: smd_prognostics: monitor your Txp installation for suspicious activity

#27 2010-11-14 16:11:39

Re: smd_prognostics: monitor your Txp installation for suspicious activity

#28 2010-11-14 16:54:05

Re: smd_prognostics: monitor your Txp installation for suspicious activity

#29 2010-11-14 19:41:28

Re: smd_prognostics: monitor your Txp installation for suspicious activity

#30 2010-11-14 20:29:01

Re: smd_prognostics: monitor your Txp installation for suspicious activity

Board footer