visitor logs v slimstat v access log

colak · 2009-02-04 17:57:54

There seems to be a bad bot which is repeatedly accessing a particular page from one of my sites.

I have the visitor preferences to log all hits but it only lists 4 hits in the last 24 hours
Slimstat has registered 49 54 hits since 13.56 GMT
The server access logs registered 517 since 17:14:29 server time most of which were 403s.

Slimstat’s interface is confusing particularly when you are trying to pair an IP with a page/referrer/etc but the server access logs reveal that the visitors are coming from various, apparently non related IPs such as

77.245.xxx.xx
208.109.xxx.xxx
89.185.xxx.xxx
etc

The reason I am posting this in Troubleshooting instead of the general discussions forum is because I’m not sure why there is a discrepancy regarding the visitor numbers.

Also if anyone has any suggestions on what actions I should take to would be appreciated.

> Update: the numbers above have increased

slimstat: 78 hits
Access logs: 601 hits
Visitor Logs: 4

The access logs show most of those hits to be 403s. Does this mean that the visitor logs do not record 403 errors whereas slimstat only records some of them?

Last edited by colak (2009-02-04 20:13:00)

colak · 2009-02-05 07:31:52

Digging into slimstat the particular url has the following stats

DATE	HITS	VISITS	IPS
February 2009	1094	1020	149
567	512	194
148	110	85

The problem has started last month whereas January seems to be normal. Considering it is only the beginning of February, you can see the unhealthy rise of hits/visits/IPs all targeting an old page.

Again, the access logs show a lot of 403s which I take it that the server is aware and they are blocking those IPs, would this be a slimstat problem?
Can anyone advise me here?

Last edited by colak (2009-02-05 16:05:20)

colak · 2009-02-05 16:03:45

OK, The hits to the site are relentless. Just today, the Access logs registered 360 hits…

Any input would be appreciated even if it is confirming that 403 is a good error:)

els · 2009-02-05 19:41:41

I once had that, and when I googled for my page it was really ridiculous what showed up, all kinds of dubious, seemingly non-existent sites… What I did was change the url-title of that article, which of course meant that everyone got to see a 404, but the page in question wasn’t that important, and could easily be found using the search on the site. Don’t know if that’s an option for you? Or maybe you can block the user agent?

colak · 2009-02-05 19:52:20

Hi Els, thanks for coming to the rescue. The problem is that there is no use agent. According to the access logs the visitors are all coming apparently using “compatible; MSIE 6.0; Windows NT 5.1”. All from very different IPs. Deleting the page or saving it as New (thanks Mary:) would probaly not necessarily solve the problem as they don’t seem to want to stop. Another detail I did not mention above is that they all use the POST. method.

Gocom · 2009-02-06 03:58:36

colak wrote:

Another detail I did not mention above is that they all use the POST. method.

And those requests are without referral and on that page you have a form, like comment form (or there was a form in past)? If that is the case, around 101% of those referraless POST request (that don’t leave comment or anything), are infact spam bots.

It’s just rule that everytime you put a comment form up on site (or open up it for article), the form will get a massive number of spam hits, untill the form is removed. Thanks to Textpattern’s spam protection, almost none of those mass “idiot” script kiddie bots spam post get on your db. Best protection is to remove all forms from site altogether.

But, I wouldn’t recommend to ban all of them. Because some of those IPs might be next dynamic IP of your real visitor – that is the problem with IP banning. Those spam bots are just something you have to live with. They will always be there, always be eating bandwidth, always trying to find emails, submit forms and use exploits – that’s just how life is meant to be.

You can of course ban all those bots, but bad bots aren’t going to end. Well, you can minify the bandwidth loss by doing that… but there is always a new IP to ban (a new IP that might be your next visitor’s IP).

And the fact why those numbers are different (in different log platforms), is because slimstats tries to validate requests and only logs hits it can (aka where it is called), when server access logs log everything, and TXP visitor logs… just log the one thing.

Last edited by Gocom (2009-02-06 04:12:13)

Textpattern CMS

Textpattern CMS support forum

#1 2009-02-04 17:57:54

visitor logs v slimstat v access log

#2 2009-02-05 07:31:52

Re: visitor logs v slimstat v access log

#3 2009-02-05 16:03:45

Re: visitor logs v slimstat v access log

#4 2009-02-05 19:41:41

Re: visitor logs v slimstat v access log

#5 2009-02-05 19:52:20

Re: visitor logs v slimstat v access log

#6 2009-02-06 03:58:36

Re: visitor logs v slimstat v access log

Board footer