[RFC] Custom Fields: expirable?

Bloke · 2025-01-15 18:16:31

Duh. Fixed. Thanks.

Bloke · 2025-01-15 19:09:07

Now I remember why I did not implement the change content type and change render type actions. The major issue is what to do about content loss.

Imagine you have created an article custom field called location and you populate a few articles with location values, such as Japan, USA, UK. An article with an ID of 7 is assigned to the location Japan. You change the content type to image. Submit. Now, all existing location content mappings are assigned to images so if you click to edit image number 7, you will see Japan stored against it.

That to me seems wrong, because the image and article IDs are arbitrary and there’s no intrinsic link between them.

The only sane thing I can think of doing is if you alter the content type either via multi-edit or by directly editing the field, is to empty any content assigned to it. Does that seem reasonable? And if so, that needs documenting or warning or something when you are on the meta edit panel. The multi-edit already has an are you sure prompt so that is covered.

A similar issue exists if you change data type. It’s not so bad if you perhaps change from text input to a text area because they are broadly compatible. Going from text area to text input may truncate the data. But going from text input to number or date will trash the content.

So, again, the question is how far do we try and keep the data if you perform a destructive operation like this, or do we just assume that if you change the data type, all current data assigned to it will be lost?

Bloke · 2025-01-15 19:13:44

An alternative scenario: once the content type and render type are chosen and the field has been created, forbid them to be changed from the interface. That way, the only way to preserve data is if you do it by hand at the database level after creating a new field with different type assignments. That is far simpler, and has definite appeal.

etc · 2025-01-15 21:51:33

Altering content/render type directly seems to work. What happens behind the scenes in this case?

Overall, this can wait until the dust settles me thinks. I’m not even sure custom fields need a type, especially if we allow new content types definitions. Just pick whatever field/family you need via some ‘section’.

Bloke · 2025-01-15 22:18:18

Currently, the (rather complicated) Field->save() routine tries its level best to migrate the content to the new type without data loss. It creates a new table datatype if there isn’t one already, clones the mapping and values into it and deletes any old refs in the original data table. It also migrates Lang strings if you rename the field.

So if you had a varchar field before, and you changed it to int, the values would be converted to int so only numeric data would come across: the rest would be lost.

That’s kinda handy if you have a varchar table that stores only numbers, because you can save storage space by converting it to int or float data. But if the old and new data types aren’t compatible you’ll just get garbage in the value column.

What it doesn’t currently do is touch the actual data if you alter the content type. So that is still mapped to the old content id rows. I’m of the opinion if you change content type, you lose the data because it makes no sense of it’s all assigned to images or categories that bear nonrelation to where it came from. But if you change data type, we can try to store what we can.

If we do expose these abilities to multi-edit, it would make sense to read the current field in, clone the current datatype field to data_type_orig, overwrite the column with the new data type and then just call Field->save() which will handle the entire, transactioned, process of storing/migrating data.

All we need to do is decide whether we keep or ditch data when changing content type, update save() to handle that, and then wire the multi-edit options up to call it.

Bloke · 2025-01-15 22:26:58

etc wrote #338817:

I’m not even sure custom fields need a type, especially if we allow new content types definitions.

So how does the data get stored against a particular type of content then? So that it can be retrieved (on the front end) or updated (from the back end).

If you created a single “location” custom field and told it that this field can be attached to articles and images and a new type called “houses”, how do you store three different values against the same field if you don’t differentiate them somehow?

The only way to do it would be to either:

A) create 3 custom fields called location_article, location_image and location_house (either the user would be forced to do this or we could duplicate the field each time it’s attached to a new content type).
B) store multiple values in the single ‘value’ field; either delimited or in some structured format (like JSON), which would mean all fields would be, what, varchar I guess?

etc · 2025-01-15 22:45:15

What if we had no cf content type at all? Just the id of the ‘content’, be it ‘article’, ‘image’ or whatever (still the idea of ‘merging’ content types)? Cf collections would be tied to ‘content’ via its ‘type’ (‘article’, ‘image’, ‘reduced_article’ etc). Altering a ‘type’ cf collection removes unused cf from ‘content’. Changing a ‘content’ ‘type’ acts in the same way. So, cfs attached to ‘content’ with id=7 always stay attached to it, unless deleted.

Re changing the render type your routine looks great for me.

Bloke · 2025-01-15 22:56:34

Okay, so we effectively push the content_type into the content type definition instead of in the field definition, by virtue of creating a content type that contains a “location” field. That works.

It’s probably neater to put some effort into designing content types than working out what to do if a field has its type changed. In that case, the save routine is essentially done then :) Aside from the corner cases noted as To-do in the code.

Lemme think about this a bit.

etc · 2025-01-15 22:59:41

Bloke wrote #338819:

If you created a single “location” custom field and told it that this field can be attached to articles and images and a new type called “houses”, how do you store three different values against the same field if you don’t differentiate them somehow?

I wouldn’t tell a cf where to attach itself. Instead, I would tell ‘article’, ‘image’ and ‘house’ types that they contain ‘location’ cf. This means that ‘content’ of any type must have a unique (among all types) id. Basically, it means all ‘content’ identifiers are stored in the same table (textpattern?) extended with an extra type column, and we need a new txp_type table to relate types to cf collections.

Well, it’s getting late here :~~) edit: and I don’t need to explain it to you :~~) reedit: textile has some quirks with smileys

Last edited by etc (2025-01-15 23:01:47)

phiw13 · 2025-01-15 23:31:25

etc wrote #338808:

This is the demo server response on delete attempt:

<pre dir="auto">Warning "Array to string conversion"...

Weird,I did not get any debug warnings/errors here (PHP 8.4.2), with the pref set to debugging on. (and it is definitely on as I ~~get~~ had the wall-of-errors (strtotime() on the prefs panel – issue1969)

–^–

Ok, after updating to the latest code, the wall-of-errors (strtotime() seems gone and the deleting of CFs now works as it should.

Bloke · 2025-01-16 00:50:22

etc wrote #338822:

‘content’ of any type must have a unique (among all types) id.

Well that’s going to make data migration next to impossible. Imagine reindexing and renaming 8000 images so they don’t clash with article IDs, and then seeking out any code that uses <txp:image(s) id="nn" /> :-O

I know I know, clean break, not bwc, but if we can help people at least upgrade, it’s not a brick wall for adoption.

Basically, it means all ‘content’ identifiers are stored in the same table (textpattern?) extended with an extra type column, and we need a new txp_type table to relate types to cf collections.

That’s one way of approaching it. Here’s a counter approach:

A txp_content_types table that is administered from a back-end panel where you can name and design content. They all come hard-wired with (unique) id, plus name, created, modified, expires and (unique) table columns. Plus maybe others as we see fit (not sure about author because it clashes with the txp_users table and would make custom fields to build bios more tricky, but we do need some way of associating each record with a user so I’d probably include it and worry about circular refs on the txp_users table later).
- When creating a content type, you can Add multiple (custom) fields and select each from one of the ones already defined.
- A bunch of fields are defined out of the box. Body, Excerpt, Image, etc. And the 4 content types are pre-defined to mimic how they are today, minus custom_1…10.
For each content type, a table is created to hold its core row-data (it’s named, as defined in the content type definition). So, textpattern, txp_file, txp_image, etc tables are defined by default.
The individual content type tables are linked to CFs via txp_content_fields, whereby the existing implementation takes over (minus the content_type column).
- txp_content_fields has columns: content_id (foreign key to a row in the individual content type table), meta_id (the field identifier), collection (to group it for display purposes), and ordinal (to show the fields in this order within their collection).

That should allow us to define mappings and data structures to house content, and the CFs manage their own data types, as they do now. And if you migrate a data type it tries to keep its field data as intact as possible.

Due to the extra (link) table for each content type:

the unique (across type) ID restriction you mention goes away
there’s a migration path for existing sites
we have a mechanism for grouping and displaying content fields on panels for (row) data collection purposes

How we then offer people the ability to set up the panels is immaterial. It could be drag n drop, which would reorder the fields and alter the collection groupings as you drag. Or it could be done “by hand” in the content type definition panel’s Edit step.

Quite how custom fields are mapped to tags, I don’t know.

Probably missed something. It’s late.

etc · 2025-01-16 10:07:20

Ignoring bwc would make coding easier, but your neat table idea seems to work. Well, it means one can not transform, say, ‘file’ into ‘image’, but it’s an edge case.

It is not quite clear atm which (custom) tables would be available on new content type creation, but we can start with the core ones. Probably, textpattern (reduced to a bare fields minimum) will suffice in most cases.

Cute, let’s try it?

Textpattern CMS

Textpattern CMS support forum

#49 2025-01-15 18:16:31

Re: [RFC] Custom Fields: expirable?

#50 2025-01-15 19:09:07

Re: [RFC] Custom Fields: expirable?

#51 2025-01-15 19:13:44

Re: [RFC] Custom Fields: expirable?

#52 2025-01-15 21:51:33

Re: [RFC] Custom Fields: expirable?

#53 2025-01-15 22:18:18

Re: [RFC] Custom Fields: expirable?

#54 2025-01-15 22:26:58

Re: [RFC] Custom Fields: expirable?

etc wrote #338817:

#55 2025-01-15 22:45:15

Re: [RFC] Custom Fields: expirable?

#56 2025-01-15 22:56:34

Re: [RFC] Custom Fields: expirable?

#57 2025-01-15 22:59:41

Re: [RFC] Custom Fields: expirable?

Bloke wrote #338819:

#58 2025-01-15 23:31:25

Re: [RFC] Custom Fields: expirable?

etc wrote #338808:

#59 2025-01-16 00:50:22

Re: [RFC] Custom Fields: expirable?

etc wrote #338822:

#60 2025-01-16 10:07:20

Re: [RFC] Custom Fields: expirable?

Board footer