Second pass: do we still need it?

etc · 2021-01-07 10:37:33

For the background: currently txp parses and processes the pages twice. For example, a page like

<txp:variable name="test" value='<txp:site_name />' output />

will output My Site Name on the first pass, which will not change on the second pass. But a page like

<txp:variable name="test" value="<txp:site_name />" output />

will firstly produce the string <txp:site_name />, transformed into My Site Name on the second pass. Recall that double-quoted tags attributes are not parsed.

While this is fun (and I like it), the result can be surprising and even unsecure. Consider the following block of code:

<txp:article break="article">
    <txp:variable name="count" add />
    <txp:excerpt />
</txp:article>

<p>Found <txp:variable name="count" /> articles.</p>

We are expecting something like

<article>First excerpt.</article>
...
<article>Sixth excerpt.</article>

<p>Found 6 articles.</p>

But suppose that some malicious article’s excerpt contains

<txp:variable name="count" value="<txp:php>die();</txp:php>" />

Then we’ll get

<article>First excerpt.</article>
...
<article>Sixth excerpt.</article>

<p>Found ...<txp:php>die();</txp:php>... articles.</p>

on the first pass, which will kill the page on the second pass. You may say ‘admins do what they want’, but this hack is available even for restricted accounts since <txp:php /> block will be run outside its article context.

So the question is: do and how you use the second pass? My gut feeling is that with new txp abilities we could/should get rid of it.

Bloke · 2021-01-07 11:05:02

My only thoughts for, historically, including secondpass were to catch attributes as you say. And also to permit things like pgonly to be run, preload the results and then have them output elsewhere. That need’s been largely – if not completely – eradicated in recent Txp’s thanks to your amazing parser improvements.

So, in theory I have no problem losing it. Open questions:

What happens to <txp:hide process="n"> constructs if we have no second, third, fourth, … pass to process tags that are not picked up on first pass?
Does this affect the ability to have forms inside forms?
Does this restrict what we can do with deferring calls in, say, smd_query to allow for replaced content to be loaded first and then executed?
Any other tags affected, or have they all been designed to work with the ‘can appear anywhere on the page’ paradigm?

etc · 2021-01-07 12:02:37

Bloke wrote #328070:

My only thoughts for, historically, including secondpass were to catch attributes as you say. And also to permit things like pgonly to be run, preload the results and then have them output elsewhere.

That’s what I’d like to see — when do you need to postpone processing? In 4.8.4 we can alter the order via <txp:tag[n] /> syntax, would this suffice?

What happens to <txp:hide process="n"> constructs if we have no second, third, fourth, … pass to process tags that are not picked up on first pass?

Does this affect the ability to have forms inside forms?

Does this restrict what we can do with deferring calls in, say, smd_query to allow for replaced content to be loaded first and then executed?

Any other tags affected, or have they all been designed to work with the ‘can appear anywhere on the page’ paradigm?

they won’t work, but currently n is limited to 2 anyway. But they could be replaced with <txp:hide[n] process="1" />.
nope, the first pass goes into full nesting depth.
dunno :-)
no, only few were designed to use the second pass

We actually have a hidden secondpass pref that I would expose in 4.8.5. Setting secondpass to 0 will disable it, so anyone can test and see if something breaks. And if you need it and trust your authors, there is no problem to enable multiple passes.

Bloke · 2021-01-07 12:06:19

etc wrote #328071:

In 4.8.4 we can alter the order via <txp:tag[n] /> syntax, would this suffice?

Yes!

We actually have a hidden secondpass pref that I would expose in 4.8.5.

I like this idea.

etc · 2021-01-07 17:22:42

How this preference should be named: Number of parser passes (so 1 = no second pass) or Number of extra passes (0 = no second pass)?

Bloke · 2021-01-07 17:33:16

etc wrote #328077:

How this preference should be named: Number of parser passes (so 1 = no second pass) or Number of extra passes (0 = no second pass)?

Is it a numeric input or yes/no radio?

If it’s a numeric input (spinner): “Parser passes” is fine IF min="1" and the max is, well, some reasonable value.
If it’s a numeric input (spinner): “Extra parser passes” is fine IF min="0" and the max is as above.
If it’s a freeform entry: “Extra parser passes” is better with default 0, sanitized to number. Otherwise someone might enter 0 and find they have no passes at all!
If it’s boolean: “Extra parser passes” gest my vote with no being 0 and yes being 1.

Last edited by Bloke (2021-01-07 17:34:21)

etc · 2021-01-07 17:56:25

Internally it’s an integer, currently 0 = no second pass. Do we have a numeric input in 4.8 prefs?

Bloke · 2021-01-07 18:48:57

etc wrote #328079:

Internally it’s an integer, currently 0 = no second pass. Do we have a numeric input in 4.8 prefs?

We don’t officially, but you can render one using a callback in txp_prefs.php, returning fInput('number'). It’s in the UI library so we can add an html input type for it in 4.9.0. I think I’m going to hit the txp_prefs panel first as it’ll be a good test of the library.

For compatibility with the existing hidden pref, I’d go with ‘Extra parser passes’ as a text input for now, and ensure it’s cast to (int) before use. That should do the trick.

P.S. Are we saying that if you set it to ‘2’ then you get two additional passes? Or does anything > 0 trigger a single secondpass?

Last edited by Bloke (2021-01-07 18:50:32)

etc · 2021-01-07 22:01:03

Bloke wrote #328080:

We don’t officially, but you can render one using a callback in txp_prefs.php, returning fInput('number').

Hm, can’t see how to convert fInput('number') to callback, but this can wait until 4.9.

Are we saying that if you set it to ‘2’ then you get two additional passes?

Yes, as much as you need, the parser will stop anyway if there is no more tags to parse.

Textpattern CMS

Textpattern CMS support forum

#1 2021-01-07 10:37:33