Txp human detection

skewray · 2025-09-17 17:35:45

I’ve been on the OCD warpath to reduce bots from crawling my website. With the help of this forum, I added cookies and then used them to prevent deep links . Progress!

I still get bots scarfing up articles. I do have a solution, which is to check for the cookie; if it isn’t there, ask the human to refresh. Awkward. I guess that would look like:

<txp:php>
if( cookie ) {
    <Somehow tell Txp that I found a cookie! Who's a good boy!>
    }
</txp:php>
(Txp checks variable and does smart things)

How does php set a Txp variable? (I bet there is already an overly complicated plugin for checking cookies…)

If I don’t want human interaction, I can do, at the very top of a Txp page:

<txp::php>
if( !cookie ) {
    setcookie(...) ;
    header( refresh:0 ) ;
    <tell Txp to not display anything>
    }
</txp::php>

Looks like an infinite loop if the browser is rejecting cookies. Do I need to do php session stuff here? Because yuk. Does Txp have session tracking?

Bloke · 2025-09-17 17:54:30

There is a cookie plugin, oui_cookie which works very well. And if you want to track session info, you can do so via PHP’s $_SESSION.

As far as using cookies in Txp, there are a few examples dotted around the core.

Inserting variables inside PHP blocks can be done in a few ways. From least to most future-proof:

<txp:variable name="my_var" value="Hello world" />
<txp:php>
    global $variable;
    echo $variable['my_var'];
</txp:php>

<txp:variable name="my_var" value="Hello world" />
// 4.9.0+ only
<txp:php my_var='<txp:variable name="my_var" />'>
    echo $my_var;
</txp:php>

<txp:variable name="my_var" value="Hello world" />
<txp:php>
    echo parse('<txp:variable name="my_var" />');
</txp:php>

Each one has a cost associated with performance vs maintenance. Oleg will be able to supply more concrete information on this but my rough guess would be that the above are in order of performance, fastest first, but is most likely to break first in future.

skewray · 2025-09-17 18:48:57

Choices! I like all of them. I can do the non-refresh version with oui_cookie and no php at all.

TIL: A php session won’t help if cookies are disabled; php sessions set a cookie to track the session.

(ps…how did you get context coloring on your forum post?)

Last edited by skewray (2025-09-17 18:50:49)

Bloke · 2025-09-17 19:14:43

skewray wrote #340549:

(ps…how did you get context coloring on your forum post?)

Add the language after the bc. then your content beginning the next line.

bc. php
bc. txp
bc. html
Probably others (js? css? javascript? stylesheet?). I don’t have the definitive list we support to hand.

phiw13 · 2025-09-18 02:13:09

skewray wrote #340549:

(ps…how did you get context coloring on your forum post?)

It is documented in the forum help page (linked next to the“logout“ link at the top of each page): forum.textpattern.com/help.php#:~:text=The%20forum%20also%20supports%20syntax%20highlighting.

Some of the suggested languages never worked for me, iirc, but the most important ones (e.g. css, html. php, txp) do and that colour coding is helpful.

gaekwad · 2025-09-18 16:32:09

skewray wrote #340547:

I’ve been on the OCD warpath to reduce bots from crawling my website.

What’s your hosting setup like? Shared? Managed? VPS? Other?

Might be worth a look to see if some of the heavy lifting has already been done, e.g.:

github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker

skewray · Yesterday 17:59:04

gaekwad wrote #340553:

What’s your hosting setup like? Shared? Managed? VPS? Other?

Shared, so I have almost no control. It is cpanel; all I can do is tinker with .htaccess.

skewray · Yesterday 18:23:11

My strategy of having the user refresh the browser if there is no cookie doesn’t work. The server returns a 304 Not Modified. Is it Txp that determines if the content hasn’t changed? If so, how do I modify that response?

etc · Today 10:40:14

Have you tried to disable Send "Last-Modified" header Publish pref?

Anyway, evil bots can automate headless browsers nowadays, so I doubt a cookie would be bullet-proof.

skewray · Today 14:52:04

I know this is definitely not bullet-proof. But most bots are dumb, so I can make a big dent.

I don’t think I want to disable Last-Modified across the site. I just want to poke Txp to not 304 this particular time. I suppose the issue is that I don’t know how Txp decides if the URL content has changed; I had php add an HTML comment, but that was insufficient. Will setting a Txp variable be? Or if that variable causes a change in content?

Textpattern CMS

Textpattern CMS support forum

#1 2025-09-17 17:35:45