[RFC] TXP5 Custom Fields Management

Bloke · 2025-10-09 09:39:47

etc wrote #340894:

Not quite mistakenly, sorry, currently Write link is attached to the entity 1 (article), just to mimic txp4 behaviour.

Okay, yeah. As you say, if entities are renamed we’re in the same boat, so IDs are fine. I’d rather adopt the convention that we have a bunch of core base types/entities/whatever that you can extend or add to.

But we are going to refactor it somehow, if I read you right.

Not sure. Depends if it’s worth it, given…

more I think of it, more the rah_wrach way looks natural: chose the structure first, then just write

… I agree with this. Very few people sit down and think “I’m going to write an article now… but I don’t know what it’s about. I’ll decide later.” Choosing the “type” up-front adds one step to everyone’s Just Write experience but simplifies the process. Sidesteps complicated Ajax logic. And, perhaps advantageously, gives a nice splash screen we can build on if necessary. Want to add a delta field just for this article? Do it at content selection time.

Maybe we should clarify what is meant by ‘types’ here.

Yes! We need to be clear about naming conventions and stuff. A content type is not an entity and not a fieldset, but they’re related. That’s what we need to cement.

if you see how to manage labels, be my guest.

Hehe. You make a good point to phiw13. Defining strings for use on the admin side is not the same thing as writing content for public consumption.

We could be militant. Forget the title. No point using gTxt() on it if it’s going to be fixed in the current admin-side language. We could make the type keys English with first letter capitalised for niceness, or just put up with the types being all lower case in English throughout the back-end interface.

etc · 2025-10-09 19:58:25

Bloke wrote #340896:

Hehe. You make a good point to phiw13. Defining strings for use on the admin side is not the same thing as writing content for public consumption.

We could be militant. Forget the title. No point using gTxt() on it if it’s going to be fixed in the current admin-side language. We could make the type keys English with first letter capitalised for niceness, or just put up with the types being all lower case in English throughout the back-end interface.

I think we should somehow separate field/entity definition pane and language strings anyway. Name strings after ids (txpcf_1 or whatever), then you don’t need to be bothered with them when a field is updated. And any eligible user will be able to edit strings without touching fields definitions.

Bloke · 2025-10-09 20:03:30

etc wrote #340897:

I think we should somehow separate field/entity definition pane and language strings anyway. Name strings after ids (txpcf_1 or whatever), then you don’t need to be bothered with them when a field is updated. And any eligible user will be able to edit strings without touching fields definitions.

I like this.

etc · 2025-10-09 20:14:47

I’m done with the next round of modifications. Please test various cf manipulations on the admin side (currently mainly for articles) and report any weirdness.

The interface is basic and will certainly be improved once the structure is stable.

What slightly annoys me is that every item (even a stand-alone one) requiring custom fields, must be attached to some entity, even for ad hoc definition. If you delete all entities, custom fields become impossible to use until you create a new one.

phiw13 · 2025-10-10 08:52:51

etc wrote

Who would translate these strings? Site admins are not always polyglots, and letting authors edit fields/entities/sections/etc is (currently) error-prone. We’d need to find a way to separate editing language strings from other definitions.

There is a job for translators there :-), a site admin’s job would be to push it though the system. Language packs is an option (that works fine for the jcr_image_custom and friends managerie). I certainly don’t argue for any author to edit those fields. Further in the thread a possible idea seem to emerge – where the CF’s are only defined by ID and name (I suppose), and label or title is added per admin language. I guess the primary language first, and the second/third/… language would “inherit” the language if the particular language doesn’t have strings for some CF’s yet.

Bloke:

Defining strings for use on the admin side is not the same thing as writing content for public consumption.

I never argued that that would be in anyway related.

–^–

Idea – the Edit field for CF’s could have additional secondary “label” fields for translations when more that one admin language is active. That might be complicated though.

etc · 2025-10-10 20:00:36

phiw13 wrote #340904:

Further in the thread a possible idea seem to emerge – where the CF’s are only defined by ID and name (I suppose), and label or title is added per admin language.

No, only id, names are mutable. This makes labels export/import cumbersome, but names are not universal, neither.

Idea – the Edit field for CF’s could have additional secondary “label” fields for translations when more that one admin language is active. That might be complicated though.

I would totally separate help/labels from other fields, they require different privs.

phiw13 · 2025-10-11 09:51:18

etc wrote #340914:

I would totally separate help/labels from other fields, they require different privs.

That works too.

etc · 2025-10-22 15:01:38

We are progressively moving to a more workable (?) cf interface. The custom fields section of the Write tab now contains entity/fields select boxes to load any fieldset you need. The custom fields inputs themselves have moved to the ‘main’ area. UX is still rather ugly, but we might smoothen it later (drag’n‘drop etc) if the overall workflow is ok. And our theme gurus will somehow prettify the UI :-)

Please test whether editing/saving articles cfs works as expected.

phiw13 · 2025-10-23 06:06:37

Yay, one more small step forward. It is a little messy for now, but prettifying the whole lot will have to wait till the workflow / UI is a little more settled.

For now two notes

you’ll need a <label> for both select widgets under the “custom fields” block on the side column
the construct for each custom field is wrapped in a class-less div. .txp-form-field is what you want (see all other similar blocks).

Editing my dummy article with 2 CF’s for now, Saving, and changing from Draft to Live works fine it seems. (after leaving the write panel, visit some other panel and returning, all content is still there in the state I left it after saving…).

etc · 2025-10-23 08:48:06

Thanks, will wrap. Wrapped.

etc · 2025-11-01 16:53:50

I’m testing so-called virtual custom fields, which are actually db functions/queries definable by site admins. Few dummy examples of article custom fields might be

2*ID
REGEXP_SUBSTR(Title, '\d+')

and so on. But, actually, one can use more complex queries, referring to the current article/image/etc, like

SELECT GROUP_CONCAT(id) FROM some_table WHERE content_id = textpattern.ID

One of the advantages over php post-processing is that one can pre-filter the items on db fetch, preserving the pagination.

Does anyone see a reasonable objection/usage of it, before too much work is done? More generally, what types of custom fields would you like to see in txp5?

Bloke · 2025-11-01 17:09:02

Intriguing. How does that affect storage efficiency? Or is this solely for output wrangling? i.e. is the custom field “definition” written when you create the field and stored as, say, a varchar query string, which is then interpreted every time the field is accessed instead of merely fetched and the corresponding global populated?

Also, at the back of my mind, is that one of us (probably me) is going to have to bite the bullet and write a parameterized query / PDO layer to replace all the txplib_db.php safe_*() functions. There are precisely zero PDO/ORM wrappers in existence that aren’t tied to laravel or some other huge framework that’s 10x the entire footprint of Txp. The closest is probably DBAL, if it’s decoupled from Doctrine, so that might act as a starting point. If not, we’ll roll our own.

Anyway, yeah, if we’re adding virtual fields, keep in mind how we’d interface the capabilities with a parameterized query layer. If it’s possible to preserve pagination in fields (and so forth) and we can expose this and other things like cursors in our abstraction layer, then we’ll have world class efficient querying at our fingertips, across a variety of database vendors.

This is a seriously cool idea, btw. I’ll hopefully have some time later today or tomorrow to play with the branch.

etc · 2025-11-01 17:39:06

Bloke wrote #341058:

Intriguing. How does that affect storage efficiency? Or is this solely for output wrangling? i.e. is the custom field “definition” written when you create the field and stored as, say, a varchar query string, which is then interpreted every time the field is accessed instead of merely fetched and the corresponding global populated?

Yes, we just store the query definition and then retrieve it and plug into cfs population queries instead of SELECT value ... The overhead for functional queries like SUBSTR(Title, 2) is minimal, more complex queries can be more pricey.

if we’re adding virtual fields, keep in mind how we’d interface the capabilities with a parameterized query layer.

We do not PDO for PDO, right? Some queries would benefit from it, but others might stay old-fashioned, especially since only admins can define them.

Bloke · 2025-11-01 18:10:58

etc wrote #341059:

Yes, we just store the query definition and then retrieve it and plug into cfs population queries instead of SELECT value ...

Perfect. That’s insanely cool.

We do not PDO for PDO, right?

Well herein lies my beef with PDO. It’s fine, but the stated aims of it being database agnostic are not quite true. Because some databases like SQL Server and Oracle do things differently to MySQL, which has support for stuff that we use where they don’t (like FIELDS() for extracting stuff in sort order, for example).

So moving lock stock to PDO means we gain some support for other DB engines and get rid of all the doSlash() nonsense with parameterized queries, but lose niceties we have in MySQL derivatives because our queries all need to use (e.g. SQL 92) lowest common denominator features. Plus, last time I checked, things like WHERE field IN (?,?,?,?,?,...) isn’t supported, so we still need some PHP wrangling to transcribe such queries into parameterized versions before sending them off to the DB.

It’s a bit of a mess, so I suspect we’ll need either a halfway house approach with a mix of PHP and SQL to maintain our feature set, OR we stick with mysqli and just retool the queries to use parameters.

Factoring an ORM on top doesn’t seem worth the overhead, especially since our table structures aren’t completely 4NF.

PDO seems like it should be the right way to go, but the devil is in the details, and those details sacrifice some of our performance and flexibility. So perhaps sticking with MySQL derivatives and parametrized mysqli is a better option.

Either way, building support in our DB access layer for cool features like cursors and virtual CFs, pagination, filtering, maybe even Views, should be taken into consideration.

Bloke · 2025-11-02 23:57:06

Didn’t have much luck with the CF values being saved when I tried earlier.

The fieldsets could be toggled on and off based on the entity type, and I could individually then show/hide them. That’s neat. But entering values and saving didn’t work. I got a green article / image saved announce message but when I reloaded the page, the fields reset to empty.

These fields/entities were defined last week before the latest changes, though. So maybe it didn’t run some necessary upgrade script, or the fields need recreating first so their definitions are synced. Hmmm.

Textpattern CMS

Textpattern CMS support forum

#76 2025-10-09 09:39:47

Re: [RFC] TXP5 Custom Fields Management

etc wrote #340894:

#77 2025-10-09 19:58:25

Re: [RFC] TXP5 Custom Fields Management

Bloke wrote #340896:

#78 2025-10-09 20:03:30

Re: [RFC] TXP5 Custom Fields Management

etc wrote #340897:

#79 2025-10-09 20:14:47

Re: [RFC] TXP5 Custom Fields Management

#80 2025-10-10 08:52:51

Re: [RFC] TXP5 Custom Fields Management

#81 2025-10-10 20:00:36

Re: [RFC] TXP5 Custom Fields Management

phiw13 wrote #340904:

#82 2025-10-11 09:51:18

Re: [RFC] TXP5 Custom Fields Management

etc wrote #340914:

#83 2025-10-22 15:01:38

Re: [RFC] TXP5 Custom Fields Management

#84 2025-10-23 06:06:37

Re: [RFC] TXP5 Custom Fields Management

#85 2025-10-23 08:48:06

Re: [RFC] TXP5 Custom Fields Management

#86 2025-11-01 16:53:50

Re: [RFC] TXP5 Custom Fields Management

#87 2025-11-01 17:09:02

Re: [RFC] TXP5 Custom Fields Management

#88 2025-11-01 17:39:06

Re: [RFC] TXP5 Custom Fields Management

Bloke wrote #341058:

#89 2025-11-01 18:10:58

Re: [RFC] TXP5 Custom Fields Management

etc wrote #341059:

#90 2025-11-02 23:57:06

Re: [RFC] TXP5 Custom Fields Management

Board footer