CF branch: call for syntax

etc · 2022-11-28 10:52:32

Thanks to Stef, we already have a nice db storage mechanism and an admin interface for the upcoming cf txp version. It is time to decide now how cf could be queried and output via txp tags.

In txp 4.8, all core cf are ‘monolithic’, e.g. containing a single value. To mimic multi-valued cf, one has to cheat. For example, if we want to select articles by colour, we can define a colour cf like blue, red, white. Now, to output ‘blue’ articles, we can call

<txp:article colour="%blue%" />

Outputting ‘blue or red’ articles is more tricky. We have to either use two tags

<txp:article colour="%blue%" />
<txp:article colour="%red%" />

taking care to somehow eliminate duplicates, or to alter super-global php variables:

<txp:php>
$_GET['colour'] = array('%blue%', '%red%')
</txp:php>

<txp:article match="colour" />

In cf branch, multiple values (‘red’, ‘blue’, etc) can be stored separately. Moreover, each custom field can have a delimiter, defined on the admin side. For example, we can set colour’s delimiter to ,. It then becomes easy to run multi-choice queries, i.e.

<txp:article colour="blue, red" />

to retrieve blue or red articles. It will be mapped to colour="blue" OR colour="red" db query.

Some questions I’m thinking about are

how do we output <txp:custom_field name="colour" />, given that it is internally split into multiple values, stored in separate db records? Should we just join these values by delimiter or be able to access them individually? Which syntax then?
do we need a syntax for AND queries? Seemingly, colour="blue" AND colour="red" makes no sense, but what if we need to output all articles that contain both colours?
anything else?

Bloke · 2022-11-29 00:39:29

etc wrote #334214:

how do we output <txp:custom_field name="colour" />

My initial (perhaps rudimentary) thought was to permit them to be iterated. If you output them as a single tag, it does something sensible like concatenate them with the internal delimiter set by the custom field definition and spits the set out. But if you use it as a container then <txp:item> (did we ever implement that?) or equivalent would permit you to format any multi-value fields as you need. Combined with global wraptag, break, breakby and so forth, this would allow some varied output options.

There are, however, two strains of things to consider here:

Values that are set (usually used for for outputting / searching) so current state can be displayed.
Values that are available (usually used for building UIs and dropdowns/checkbox sets) so people can choose/search from them.

do we need a syntax for AND queries?

Yes, but I’m at a loss how to indicate that at the tag level.

Part of the power of custom fields is allowing people to slice and dice them to offer nuanced search. My go-to example is the estate agent website where you want to offer people the ability to find houses (articles) that have custom fields:

bedroom >= 4
garage = yes
75000 > price < 200000

Clearly an AND relationship exists between fields, so if someone builds a UI search facility with sliders and dropdowns and freeform text fields to enable customers to find matching properties, these values need to be transmitted via the URL and read in to make a sensible filter query.

But things like price have an implied AND relationship within the field because someone might be looking for properties within a budget range. Quite how we interpret incoming values and allow people to make sensible filters is something I’m not entirely sure how to implement.

etc · 2022-11-29 10:19:13

Bloke wrote #334216:

My initial (perhaps rudimentary) thought was to permit them to be iterated. If you output them as a single tag, it does something sensible like concatenate them with the internal delimiter set by the custom field definition and spits the set out. But if you use it as a container then <txp:item> (did we ever implement that?) or equivalent would permit you to format any multi-value fields as you need. Combined with global wraptag, break, breakby and so forth, this would allow some varied output options.

Thank you, that was my thought too, though I’d rather introduce a general-purpose iterator tag (i.e. <txp:list />):

<txp:list data='<txp:custom_field />' breakby='???'>
    <txp:item />
</txp:list>

But internally ‘delimited’ cfs (like checkboxsets) with no delimiter are problematic. We currently retrieve their values from db as concatenated strings, because we need to agglomerate all article data in a single row to populate $thisarticle array. So $thisarticle['colour'] could contain bluered, for example. But how would we split it to iterate over individual items if the cf delimiter is empty? There are two natural solutions:

either store $thisarticle['colour'] as array, but this is risky bwc-wise
or require a delimiter for each ‘delimited’ field, but this is a bit restrictive

There are, however, two strains of things to consider here:

Values that are set (usually used for for outputting / searching) so current state can be displayed.

Values that are available (usually used for building UIs and dropdowns/checkbox sets) so people can choose/search from them.

Yep, makes sense. We probably should store all cf metadata in memory, to avoid db calls.

Part of the power of custom fields is allowing people to slice and dice them to offer nuanced search. My go-to example is the estate agent website where you want to offer people the ability to find houses (articles) that have custom fields:

bedroom >= 4

garage = yes

75000 > price < 200000

Ranges are already possible, via bedroom="4%%" and price="75000%%200000", but not things like

amenity LIKE "%hairdryer%" AND amenity LIKE "%dishwasher%"

The latter can be retrieved via nested <txp:article /> tags (or by some kind of inverse logic), but it would be nice to be able to do it in one go.

etc · 2022-11-29 11:46:47

BTW, the separate storage of ‘delimited’ cfs values yields an interesting bug/feature(?): if colour of some article contains ‘blue’ and ‘red’, the following code

<txp:article colour="red">
    <txp:custom_field name="colour" />
</txp:article>

outputs red, and not blue,red as one would expect.

Dragondz · 2022-11-30 09:23:04

etc wrote #334218:

BTW, the separate storage of ‘delimited’ cfs values yields an interesting bug/feature(?): if colour of some article contains ‘blue’ and ‘red’, the following code

<txp:article colour="red">...

outputs red, and not blue,red as one would expect.

Hi if i use txp:custom’field i should expect to see all it s content, then i will call it a bug!

Bloke · 2022-11-30 23:00:15

etc wrote #334218:

<txp:article colour=“red”> <txp:custom_field name=“colour” />
</txp:article>

outputs red, and not blue,red as one would expect.

Yeah, so this is where we need to think about syntax and what is the most useful/best default behaviour.

Examples of things we might do:

<custom::field name="colour" /> – display only matching custom field info
<custom::field name="colour" show /> – display the set/selected custom field info
<custom::field name="colour" show="all" /> – (or showall?) display all custom field values, set or not

Whether that’s the best, expected scenario, or whether we swap it round so show only displays the matched stuff, and the absence of the attribute displays all set/selected field values, is something we’ll have to decide. I’ll take whichever syntax requires least typing to do something reasonably useful and logical.

Bloke · 2022-11-30 23:11:06

etc wrote #334217:

Ranges are already possible, via bedroom="4%%" and price="75000%%200000"

How very… SQLish ;)

Good luck to anyone who is coding a UI to allow people to search min and/or max values…

<if::custom_field name="minprice" value>
    <if::custom_field name="maxprice" value>
        <txp:article price='<custom::field name="minprice" />@@<custom::field name="maxprice" />' />
    <txp:else />
        <txp:article price='<custom::field name="minprice" />@@' />
    </if::custom_field>
<txp:else />
    <if::custom_field name="maxprice" value>
        <txp:article price='@@<custom::field name="minprice" />' />
    </if::custom_field>
</if::custom_field>
...

Contrived maybe? There might be a simpler way. But that’s just one parameter.

What if the user wants to search more than one? How would we even begin to make this useful so people could build code that would match a bunch of fields, some verbatim, some wild, some with ranges, etc? Makes my head spin.

etc · 2022-12-01 09:07:05

Bloke wrote #334222:

Yeah, so this is where we need to think about syntax and what is the most useful/best default behaviour.

The problem is that our current db query retrieves only matching values of multi-valued cf. So

<txp:article colour="red">
    <txp:custom_field name="colour" /> <!-- outputs 'red' -->
    <txp:custom_field name="colour" show /> <!-- should output 'blue,red' -->
</txp:article>

would require a separate db call for the second <txp:custom_field name="colour" show /> tag. Or we need to modify our db query to retrieve all values, but then the first <txp:custom_field name="colour" /> tag would need to filter them somehow.

I would stick with the current dev behaviour (output all values, not only the matched ones) as Rabah implies. But this

either slows down the query (though this yet needs testing with many cf)
or breaks multi-range search ability (see the next post)

And we yet have to think about possible all|any|exact match implementation…

etc · 2022-12-01 09:35:21

Bloke wrote #334223:

How very… SQLish ;)

Inherited from the ancient cf="%value%" txp syntax, I have not invented a wheel :-)

Contrived maybe? There might be a simpler way.

Like this?

<txp:article price='<custom::field name="minprice" />%%<custom::field name="maxprice" />' />

If one (or both) of cf is empty (say minprice), this will result in price='%%maxprice', as required.

What if the user wants to search more than one? How would we even begin to make this useful so people could build code that would match a bunch of fields, some verbatim, some wild, some with ranges, etc? Makes my head spin.

This is what the delimiter was designed for. If price has comma as delimiter, multiple values/ranges searches are easy:

<txp:article price="100%%300, 400, 500%%" />

Nothing stops you from adding another field, with its own internal delimiter (e.g. pipe):

<txp:article price="100%%300, 400, 500%%" city="Paris, France | Paris, Texas" amenity="%hairdryer%, %dishwasher%" />

This will be transformed into

(price BETWEEN 100 AND 300 OR price LIKE 400 OR price>=500)
AND
(city LIKE "Paris, France" OR city LIKE "Paris, Texas")
AND
(amenity LIKE "%hairdryer%" OR amenity LIKE "%dishwasher%")

That’s fine for price and city, but for amenity you’d rather expect both:

(amenity LIKE "%hairdryer%" AND amenity LIKE "%dishwasher%")

We can imagine something like this:

<txp:article match="amenity: all, city: any"
    price="100%%300, 400, 500%%" city="Paris, France | Paris, Texas" amenity="%hairdryer%, %dishwasher%" />

Ideas welcome.

colak · 2022-12-01 11:49:12

A question, be it a naive one. What happens if there are separate entries like Artemis and Artemisia?

etc · 2022-12-02 08:39:32

Dragondz wrote #334220:

Hi if i use txp:custom’field i should expect to see all it s content, then i will call it a bug!

It is no more, whatever it was. The whole cf content is output now, as before.

colak wrote #334226:

A question, be it a naive one. What happens if there are separate entries like Artemis and Artemisia?

The usual thing:

<txp:article entry="Artemis" /> <!-- matches 'Artemis' but not 'Artemisia' -->
<txp:article entry="Artemis%" /> <!-- matches 'Artemis' OR 'Artemisia' -->
<txp:article entry="Artemis, Artemisia" /> <!-- idem, but it will be possible to replace OR with AND -->

Textpattern CMS

Textpattern CMS support forum

#1 2022-11-28 10:52:32