Textpattern CMS support forum
You are not logged in. Register | Login | Help
- Topics: Active | Unanswered
Re: Automatic Referral spam Blocking.
Reid, I’m going to open my comments up on old entries. I’ve modified them via Jon Hicks’ Comment Hack, so I’ll see if that part’s correct. They won’t have anything blocking them from commenting since the preview option is gone ATM.
Hopefully, it can lead to some helpful information, though, I doubt it.
Last edited by PeterS (2005-01-21 22:38:57)
The following is true
The above statement is false.
Offline
#14 2005-01-21 23:15:56
- zem
- Developer Emeritus
- From: Melbourne, Australia
- Registered: 2004-04-08
- Posts: 2,579
Re: Automatic Referral spam Blocking.
> You should talk to zem (Alex). he is deep in the process of developing an anti-spam tool for Txp.
Referrer spam is the first on my list.
Ironically, in many respects it’s harder to block than comment spam.
Alex
Offline
Re: Automatic Referral spam Blocking.
I saw this method for stopping referer spam a couple days ago; it’s not a bad idea but it’d be nice to have something which scales.
You cooin’ with my bird?
Offline
#16 2005-01-22 00:59:20
- zem
- Developer Emeritus
- From: Melbourne, Australia
- Registered: 2004-04-08
- Posts: 2,579
Re: Automatic Referral spam Blocking.
> I saw this method for stopping referer spam a couple days ago; it’s not a bad idea but it’d be nice to have something which scales.
It’s fine if you like spending hours each day playing whack-a-mole.
Patience, fellas.
Alex
Offline
Re: Automatic Referral spam Blocking.
And if you want to try and counter-program referral spammers, know your enemy and their techniques.
“My guess is that spammers will start using HTTPS next. That way madness lies, since the SSL handshake alone will bring your average server to its knees if it starts happening in batches of 80 or so.”
Oh, geez…
TextPattern user since 04/04/04
Offline
Re: Automatic Referral spam Blocking.
I’ll have the first version available by Sunday night. This initial release will only have a keyword based filter editable through a web form. (txp?) and will not be for ‘ransom’
Following versions will support ‘bayesian’ filtering and a basic pattern detection (this itself, is proving to be the most difficult)
Offline
Re: Automatic Referral spam Blocking.
Interesting discovery. I’ve been researching the tools that these idiots use, and not a single one of them “follows” any redirect as they operate on a basic http level and just issue GET requests to as many URLS as possible.
Following this, I’ve realized that it would be possible to write a simple script that upon loading, redirects the legit visitor to the page they intended to visit.
Benefits?
- Bandwidth.
- Ability to log who does /not/ redirect and ban the ip at the htaccess level.
I can see that using this would pretty much destroy the whole point of referrers, but they are pretty much useless now anyway.
Offline
#20 2005-01-26 23:16:19
- zem
- Developer Emeritus
- From: Melbourne, Australia
- Registered: 2004-04-08
- Posts: 2,579
Re: Automatic Referral spam Blocking.
> TheEric wrote:
> Interesting discovery. I’ve been researching the tools that these idiots use, and not a single one of them “follows” any redirect as they operate on a basic http level and just issue GET requests to as many URLS as possible.
My research shows a very different result: most referrer spammers follow redirects.
Alex
Offline
Re: Automatic Referral spam Blocking.
>My research shows a very different result: most referrer spammers
>follow redirects.
Nope. I obtained a copy of one of the primary window applications used, “Reffy” and it simply issues the HTTP GET with the appropriate information, and then closes that connection. End result? Referrer spam and no redirect.
Now, the great thing about any client that MAY follow the redirect? redirect it back to itself.
Win Win.
Offline
Re: Automatic Referral spam Blocking.
TheEric, two things to think about: If Zem is seeing referrers that follow redirects, you can’t dismiss that with a nope. Obviously there are people spamming that don’t use “Reffy”. Second, if you redirect clients that follow redirects, how do you propose to have legitimate clients get to your site?
Offline
#23 2005-01-27 00:25:30
- zem
- Developer Emeritus
- From: Melbourne, Australia
- Registered: 2004-04-08
- Posts: 2,579
Re: Automatic Referral spam Blocking.
> > most referrer spammers
>follow redirects.
> Nope.
213.172.36.62 – - [14/Dec/2004:23:39:25 -0500] “GET /article/1925 HTTP/1.0” 301 0 “http://www.texasproptax.com/” “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT5.2; .NET CLR 1.1.4322)”
213.172.36.62 – - [14/Dec/2004:23:39:31 -0500] “GET /article/1925/wired-fluffy-bunny-no-longer-energized HTTP/1.0” 200 12944 “http://www.texasproptax.com/” “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.1.4322)”
..etc. Almost all spam that’s hitting my honeypot displays similar capabilites (and then some).
They moved on from Reffy a while back. Whatever tool they’re using now is far more sophisticated. More details here, here, and here.
Make no mistake: these guys are clever, motivated and well funded. They won’t fall for a circular redirect, and they know exactly how to turn your keyword blocker into a quagmire.
Alex
Offline
#24 2005-01-30 22:39:20
- Kibitzer
- Member
- From: Melbourne, Australia
- Registered: 2004-05-24
- Posts: 44
Re: Automatic Referral spam Blocking.
I saw this article about blocking referral spam and thought it might be interesting. I think it’s what you guys are talking about; the article was helpful in explaining the problem to me. I know you’re looking for an automatic solution but perhaps this’ll be useful to others in the meantime.
VC200 Member #69 — VCTWO Member — Mixed Gorilla
“YES!” “That would be an ecumenical matter!”
Offline