spam, .htaccess, and txp

jayrope · 2006-11-06 19:30:07

Colak, looking great.
How would you redirect to your own site’s start page for instance?
edit: oh by
bq. RewriteRule (.*) /mysite.com/index.php

maybe?
no that doesn’t work here…

Last edited by jayrope (2006-11-06 20:49:31)

NyteOwl · 2006-11-06 21:30:40

Addendum: oh yeah, the other thing is I want to block IP addresses, not URLs. Will those ideas also work with IP numbers?

IP’s are easy:

order deny,allow
deny from xxx.xxx.xxx.xxx
deny from xxx.xxx.xxx.xxx
deny from xxx.xxx.xxx.xxx
deny from xxx.xxx.xxx.xxx
.
.
.

Mary · 2006-11-06 21:33:03

Assuming the spammer is stupid enough to not hide/change his IP, of course. IP banning should be considered a temporary measure, helpful for when you’re getting hammered by the same IP.

jayrope · 2006-11-06 21:36:52

here’s some typical code for denying hotlinking on media files

RewriteCond %{HTTP_REFERER} ^http(s)?://(www\.)?singingfish.com/.*$ [NC,OR]
RewriteRule .*\.(mp3|jpg|gif|png|avi|mpg|wmv)$ / ®

uhm, answering my own quest i guess. thanx colak. i drew inspiration (means knowledge) from apaches 1.3 mod_rewrite docs

colak · 2006-11-07 07:53:41

NyteOwl wrote:

bq. Addendum: oh yeah, the other thing is I want to block IP addresses, not URLs. Will those ideas also work with IP numbers?

IP’s are easy:

order deny,allow
deny from xxx.xxx.xxx.xxx
deny from xxx.xxx.xxx.xxx
deny from xxx.xxx.xxx.xxx
deny from xxx.xxx.xxx.xxx
.
p. Kind of what you need is

order deny,allow
deny from xxx.xxx.xxx.xxx
deny from xxx.xxx.xxx.xxx
deny from xxx.xxx.xxx.xxx
deny from xxx.xxx.xxx.xxx
allow from all

you need the last line to define the allow conditional

@ jayrope

Just visited this thread glad you found the solution…

Last edited by colak (2006-11-07 07:54:57)

jameslomax · 2006-11-07 09:18:57

Hmm interesting.

Can I just insert that anywhere in the .htaccess code colak? Or does it need to go inside another variable/whatever? It certainly looks the simplest way of doing this……albeit, as Mary says, its probably not advisable to permanently block a series of IPs.

Ha! – losers.
We understand this stuff, we can block this stuff, you lose.

Last edited by jameslomax (2006-11-07 10:26:31)

jayrope · 2006-11-07 15:51:42

The problem with banning IPs is mostly, that you won’t grab a spammer on a dynamic IP with this.
I’m pretty sure, that spammers are aware of this and prefer dynamic IP ISPs to spread the evil.
What does the user agent information on the spammers look like? you might be able to define a rule, that denies by user agent.

Here’s an article about doing this without the use of mod_rewrite, although that doesn’t affect spammers, rather harvesters and alike

and here’s one about blocking by IP.

Last edited by jayrope (2006-11-07 16:04:09)

colak · 2006-11-07 18:16:53

There is the secondary problem with filling the htaccess with all these “denials”. For every visitor your site gets, the htaccess is loaded first so if you have a 10k htaccess and 100 legit visitors/day 1Mb of your bandwidth goes to waste. Also the speed of the site can be affected as the htaccess file gets larger. In reality spammers keep on changing their ips so keeping up with them can become a full time job. If they are comment spammers there are enough plugins here to help us strengthen the already quite strong txp comment system, if they are referrer spammers, unless they are persistent (say over 5-10 hits a day for days on end), I tend to ignore them.

jameslomax · 2006-11-18 18:39:27

Hey guys, my file’s not working.
Can anyone see whats wrong with this:

DirectoryIndex index.php index.html
Options +FollowSymLinks RewriteBase /relative/web/path/
order deny,allow
deny from 81.177.15.237
deny from 81.177.15.236
deny from 193.93.237.33
deny from 206.83.210.191
allow from all
<IfModule mod_rewrite.c> RewriteEngine On RewriteCond %{REQUEST_FILENAME} -f [OR] RewriteCond %{REQUEST_FILENAME} -d RewriteRule ^(.+) – [PT,L] RewriteRule ^(.*) index.php
</IfModule>

jayrope · 2006-11-22 12:29:12

sorry james, i can’t help you on above question intuitively, maybe you can try the apache help docs on mod_rewrite.

on another note i found this incredibly helpful tutorial on how to built an automated trap for bots that don’t respect the robots.txt file.

it also contains a package on automating insertion of such bots into your .htaccess file, but nevertheless one should keep in mind, that this file probably into something huge pretty fast, which will increase bandwidth used on a server per user significantly then. nevertheless i thougt you migth be interested.

Last edited by jayrope (2006-11-22 12:31:28)

Alex McKee · 2006-11-22 15:18:11

Nice link Jayrope. Cheers.

James, I’ve been suffering the same problem. It’s a pain isn’t it. :(

jameslomax · 2006-11-28 20:10:46

colak wrote:

NyteOwl wrote:

bq. Addendum: oh yeah, the other thing is I want to block IP addresses, not URLs. Will those ideas also work with IP numbers?

IP’s are easy:

order deny,allow
deny from xxx.xxx.xxx.xxx
deny from xxx.xxx.xxx.xxx
deny from xxx.xxx.xxx.xxx
deny from xxx.xxx.xxx.xxx
.
p. Kind of what you need is

order deny,allow
deny from xxx.xxx.xxx.xxx
deny from xxx.xxx.xxx.xxx
deny from xxx.xxx.xxx.xxx
deny from xxx.xxx.xxx.xxx
allow from all

you need the last line to define the allow conditional

……………As far As I can tell (see my posted code), this is what I did but its not working…..

colak · 2006-11-29 07:41:10

jameslomax wrote:

……………As far As I can tell (see my posted code), this is what I did but its not working…..

This is very strange… May I ask who is your host?

marios · 2006-12-02 01:13:17

@Colak,
One question, If you allow me:

I found the following refferrrs in my logs. recently:

http://www.google.com/custom?q=lego+play&hl=en&lr=&safe=off&client=pub-2456819124576563&cof=FORID:1%3BL:www2.incredimail.com/images/google_h_p/envelope_38_30.gif%3BLH:30%3BLW:38%3BLP:1%3BVLC:%23551a8b%3BALC:%23ff0000%3BGFNT:%237777cc%3BGIMP:%23a90a08%3BDIV:%23f4f4f4%3BS:mystart.incredimail.com/english%3B&start=50&sa=N.

From this domain:

cable.ubr03.glen.blueyonder.co.uk

They seem to steel Keyword traffic from my Site.

How can I get rid of them without changing the permlink locations ?

Last edited by marios (2006-12-02 01:19:12)

colak · 2006-12-02 07:37:36

hi marios
I am deducing these from my observations of friends living in the uk visiting my sites
blueyonder is one of the main isps in the uk and cable.ubr03.glen.blueyonder.co.uk seems that it is just one user whose account name in blueyonder is glen whom you can block deny from cable.ubr03.glen.blueyonder.co.uk.

the refferer is is really from http://mystart.incredimail.com/english/ which might be the guilty site – see http://www.incredimail.com. which you can block too…

looking back ‘glen’ might be ok and incredimail seems that is the spammer.

What I would do is block incredimail…

RewriteEngine On
RewriteCond %{HTTP_REFERER} ^http(s)?://(www\.)?incredimail.com/.*$ [NC]
RewriteRule - [R=403,L]

Once the spammer is blocked, I would just submit a sitemap to google and wait…

Textpattern CMS

Textpattern CMS support forum

#16 2006-11-06 19:30:07

Re: spam, .htaccess, and txp

#17 2006-11-06 21:30:40

Re: spam, .htaccess, and txp

#18 2006-11-06 21:33:03

Re: spam, .htaccess, and txp

#19 2006-11-06 21:36:52

Re: spam, .htaccess, and txp

#20 2006-11-07 07:53:41

Re: spam, .htaccess, and txp

#21 2006-11-07 09:18:57

Re: spam, .htaccess, and txp

#22 2006-11-07 15:51:42

Re: spam, .htaccess, and txp

#23 2006-11-07 18:16:53

Re: spam, .htaccess, and txp

#24 2006-11-18 18:39:27

Re: spam, .htaccess, and txp

#25 2006-11-22 12:29:12

Re: spam, .htaccess, and txp

#26 2006-11-22 15:18:11

Re: spam, .htaccess, and txp

#27 2006-11-28 20:10:46

Re: spam, .htaccess, and txp

#28 2006-11-29 07:41:10

Re: spam, .htaccess, and txp

#29 2006-12-02 01:13:17

Re: spam, .htaccess, and txp

#30 2006-12-02 07:37:36

Re: spam, .htaccess, and txp

Board footer