Textpattern CMS support forum
You are not logged in. Register | Login | Help
- Topics: Active | Unanswered
Offline
Offline
Re: Situation report for Textpattern 4.6 and beyond
after taking a quick look at bolt and locomotive… I will patiently await Textpattern’s next version
…. texted postive
Online
Re: Situation report for Textpattern 4.6 and beyond
etc wrote #281071:
I don’t know how popular Locomotive CMS is, but they feature custom (and interrelated) content types. We shouldn’t miss the train…
Fantastic idea! circa 2007 ;)
Offline
Re: Situation report for Textpattern 4.6 and beyond
mrdale wrote #281084:
Fantastic idea! circa 2007 ;)
Patent number, por favor :) Now in 2014 the devs are working hard on custom fields. That will be a really big change, and custom content types might be not too difficult to add.
Offline
#90 2014-07-01 03:47:28
- DragonBard
- Member
- Registered: 2014-07-01
- Posts: 16
Re: Situation report for Textpattern 4.6 and beyond
Bloke wrote #279529:
Sorry to disappoint everyone, but this is the current database structure I have:
# Dump of table txp_meta...No doubt that’s all wrong because I’m just a beginner at SQL and relational databases. The notion of storing things in flat tables is utterly retarded, imo. I get so frustrated having to “downgrade” my thinking from true direct ERD-mapped databases to this rigid rows and columns abomination that’s dominated the market for over thirty years. I started out working with E-R databases for ten years (had never heard of relational at the time). Then someone showed me tables, rows, columns, 4NF and SQL, and I died a little inside. sigh
Anyway, with my rudimentary table wrangling, that’s what I have so far. Shoot it down if you want, but I’d rather someone improved it instead.
1. Relational is not “flat tables”. That’s just a convenient handle for comprehension. Relations are multi-dimension, not 2D.
2. MyISAM is NOT relational. It is pseudo-relational. Please switch to InnoDB so you can use constraints and impart some sanity on your development efforts.
3. You are falling into the SQL trap of believing the data type is the domain. It is not. The best way to demonstrate this is the idiocy of AddressLine1, AddressLine2, AddressLine3 I see everywhere. Unless you are doing GIS you don’t need to separate the elements of an address (and if you are doing GIS, you will separate it entirely different). The domain is address, so store it as an address. Yes, that’s right store the entire address as a large nvarchar, allow the enter key, and be done with it. Maybe if you are doing large mailings you might want to separate the city, state, and postal code for presorting purposes, but otherwise don’t bother.
4. You can get what you are after with far fewer tables. Don’t fall for storing data as it’s “true” data-type. Just store it as a reasonable length string, with appropriate linkages to where used. Use a hierarchical (aka self-referential) table to store the custom field types in their registry. The registry should consist of an ID, name, possibly a label, and help tip. Here is the fun part. Add two to four large fields, possibly text. The purpose of the fields is to provide the code to handle the admin side and published side view of the custom fields (two fields if no JavaScript, four fields if you will allow JavaScript). Let the implementer worry about how to interpret, verify, format, spindle, mutilate, and fold their own data. In other words, use the exact same idea as textpattern forms, in a smaller scale. The domain you should be implementing is “custom field”. You are currently implementing a custom SQL table editor.
5. You should be able to implement core custom fields with about 4 tables. Registry, Hierarchy, FieldValues, and FieldUse. Of course, modifying it to fit textpattern and adding security will grow the solution set. I’m only just beginning in using textpattern, but I built a similar custom field implementation back in 2001 in classic ASP, and it was plenty fast, even with loading the display handler code from the database for every page view. Cache the handlers and it should be plenty fast. That was a project for work, which died due to lack of interest.
By now, my suggestions may be too late. Take them in the spirit of cooperation they are intended and do with them as you will.
DragonBard.
Offline
#91 2014-07-01 03:54:00
- DragonBard
- Member
- Registered: 2014-07-01
- Posts: 16
Re: Situation report for Textpattern 4.6 and beyond
You are probably wondering how to handle select boxes and radio buttons with the above setup. Simple, this is where the hierarchy comes in. The implementer defines a “ComboBox” type that provides the base wrapping code, it calls on a sub-type to provide the items. The sub-type handles wrapping the individual items. IF you care about controlling the order, then you will have to provide some way to control that, most likely an additional table for maximum flexibility.
Offline
#92 2014-07-01 04:10:09
- DragonBard
- Member
- Registered: 2014-07-01
- Posts: 16
Re: Situation report for Textpattern 4.6 and beyond
DragonBard wrote #281776:
4. You can get what you are after with far fewer tables. Don’t fall for storing data as it’s “true” data-type. Just store it as a reasonable length string, with appropriate linkages to where used. Use a hierarchical (aka self-referential) table to store the custom field types in their registry. The registry should consist of an ID, name, possibly a label, and help tip. Here is the fun part. Add two to four large fields, possibly text. The purpose of the fields is to provide the code to handle the admin side and published side view of the custom fields (two fields if no JavaScript, four fields if you will allow JavaScript). Let the implementer worry about how to interpret, verify, format, spindle, mutilate, and fold their own data. In other words, use the exact same idea as textpattern forms, in a smaller scale. The domain you should be implementing is “custom field”. You are currently implementing a custom SQL table editor.
5. You should be able to implement core custom fields with about 4 tables. Registry, Hierarchy, FieldValues, and FieldUse. Of course, modifying it to fit textpattern and adding security will grow the solution set. I’m only just beginning in using textpattern, but I built a similar custom field implementation back in 2001 in classic ASP, and it was plenty fast, even with loading the display handler code from the database for every page view. Cache the handlers and it should be plenty fast. That was a project for work, which died due to lack of interest.
By now, my suggestions may be too late. Take them in the spirit of cooperation they are intended and do with them as you will.
DragonBard.
Thinking on this some more, the code should be in a separate table so that the implementer can provide different code for different contexts, say “default”, “article”, “image”, “article_listing”, etc. This should take care of the desire to have one custom field definition apply to multiple textpattern contexts.
By letting the implementer specify the code, this allows the implementer to define formatted fields, such as telephone number, with whatever formatting, verification, sanitization, and storage semantics they desire, without textpattern having to guess it all ahead. All you are really after here is specifying what textpattern will replace the <txp:custom_Field /> tag with. Keep the data storage simple, and let the custom field implementer worry about storage and formatting. The custom field implementer shouldn’t expect the moon, so don’t promise it to them. That’s what plugins are for.
DragonBard.
Offline
Re: Situation report for Textpattern 4.6 and beyond
DragonBard wrote #281776:
Relations are multi-dimension, not 2D.
Point taken, but it’s ends up being semantics. The relational model was invented to take advantage of disk-based storage over tape, but retained the notion of storing data in contiguous blocks to minimise head movement. SQL grew around this and thus pushes us to think in terms of rows and columns because you have to:
- Define the data structure (indexed, hierarchical, network, relational, columnar, …)
- Put data into the structure
- Use the relationships defined by the structure to retrieve data
My point was simply that this structured data storage approach does my head in!
You are falling into the SQL trap of believing the data type is the domain.
Maybe. I’m a novice at SQL. Where were you six months ago when I was thinking this through? ;-)
if you are doing large mailings you might want to separate the city, state, and postal code for presorting purposes, but otherwise don’t bother.
Sure, it’s OK to modge it into one field for rudimentary applications, but most of the time I happen to need to perform tax calculations, or offer delivery options based on location. City, Postcode, Country are all important separate entities in those cases as you say, so I tend to apply that notion across the board to other systems. My reasoning is that, invariably, the question comes up further down the line “how can I bill my customers for…” and I don’t like having to re-engineer things.
You can get what you are after with far fewer tables.
No doubt.
Don’t fall for storing data as it’s “true” data-type.
Noted. I’m kind of abstracting it away from the user, but essentially it’s not using a true representation, just a ‘Best Fit’. People don’t care if their Yes/No radio button data is stored as a tinyint, int, varchar, or text, but they’ll damn well care when they have a million records and it takes fifteen seconds to extract the information. That (and space concerns) are my motivation behind using an appropriate data representation for the ‘type’ of data stored, nothing more.
The registry should consist of an ID, name, possibly a label, and help tip
Yes, with the exception that we’d forego the literal label and tool tip in lieu of names that can be referenced for l10n purposes. We’re also looking at date stamping fields with at least a ‘created’ date. Preferably an expiry too if I can figure it out: just one small implementation detail to iron out there at present.
Here is the fun part. Add two to four large fields, possibly text.
If using Text, the table would not be able to be cached by the database engine, so every request would be to disk. Not my definition of ‘fun’.
You are probably wondering how to handle select boxes and radio buttons with the above setup. Simple, this is where the hierarchy comes in… IF you care about controlling the order, then you will have to provide some way to control that, most likely an additional table for maximum flexibility.
Order is important. As are default values for all fields.
By letting the implementer specify the code, this allows the implementer to define formatted fields, such as telephone number, with whatever formatting, verification, sanitization, and storage semantics they desire
I like this idea, and it did cross my mind as a potential nice-to-have. To that end, at the moment all fields are wide open (because as you say we can’t second guess anyone), but my aim is that plugins will be able to step in and add data constraints to each field if desired. Textpattern offers very little validation (in terms of data, at least. It checks rudimentary things like valid categories, section names, dates, and such like, but doesn’t validate custom field content, image names, description, etc) so I’d probably keep it that way and leave specifics up to plugins.
At the moment, plugins can override the built-in storage mechanism mapping and augment them by adding new widget types. Txp handles the back-end of managing the physical tables on your behalf.
Thank you for taking the time to post. I can see where you’re coming from and they were my initial thoughts too. It’s just that the more I thought about it, the uglier it became to stuff everything into a varchar and hope it scales well.
MySQL is not the fastest database in the world and needs all the help it can get from well-designed implementation “hacks” to help it do its job of delivering data as fast as possible. Sure, it adds application code complexity but my goal is to bridge the gap between having to burden users with the nitty gritty of understanding data types at the database level while still delivering performance that scales into the millions of rows, wherever possible.
I hope I’m on the right track…
The smd plugin menagerie — for when you need one more gribble of power from Textpattern. Bleeding-edge code available on GitHub.
Txp Builders – finely-crafted code, design and Txp
Offline
#94 2014-07-01 13:49:05
- DragonBard
- Member
- Registered: 2014-07-01
- Posts: 16
Re: Situation report for Textpattern 4.6 and beyond
Bloke wrote #281789:
Maybe. I’m a novice at SQL. Where were you six months ago when I was thinking this through? ;-)
I was working on an Exchange 2010 migration. I hadn’t even heard of Textpattern at that point. Now that the migration is done, I decided to maybe do some recreational computing. I used to do database and web app development work, but my true talents are in Systems and Network administration, so it has been years since I last played with anything database or web.
I’ve never liked MySQL, and never understood why so many think it is so great. As you say, it does not scale well and seems to introduce problems to every project that uses it. I’ve also never like PHP, which is why I like Textpattern so much. I can do nearly anything I want without ever having to touch PHP (unlike WordPress). I very much like the <txt… /> tags and how they work. If I got too involved in development, you would end up with Textpattern rewritten as a .NET app that maintains textile and the txp tags, but not much else.
My main point for this discussion is that it is far too easy to get mired in attempting to map your problem onto SQL datatypes when what you need to do is map SQL datatypes onto your problem. Also, the domain of your data often will not match SQL datatypes at all. Normalization does NOT imply deconstruction of your data into base datatypes. Normalization is all about the domain of your data. One must look beyond datatypes to how the data will be used and manipulated.
I really like Textpattern. I want to see it advance. I’m also highly opinionated and don’t mind too much being ignored.
Thank you for considering my input.
DragonBard.
Offline
Re: Situation report for Textpattern 4.6 and beyond
DragonBard wrote #281808:
I really like Textpattern. I want to see it advance. I’m also highly opinionated and don’t mind too much being ignored.
Welcome aboard, sir/miss.
Offline
Re: Situation report for Textpattern 4.6 and beyond
DragonBard wrote #281808:
it is far too easy to get mired in attempting to map your problem onto SQL datatypes when what you need to do is map SQL datatypes onto your problem.
Amen to that! I’d love to be able to use varchar for all data as it’d make everything so much simpler, but MySQL has kinda forced me to think differently, for better or worse. btw, I’ve no idea how this’ll work when we move away from sole MySQL support to using something like PDO.
Also, the domain of your data often will not match SQL datatypes at all.
Yeah, that’s a pain too. Storing complex objects seems natural when stuff is related (your address example, for instance), but the SQL paradigm is one-field-to-one-piece-of-data which forces you to either explode your thinking, use a large/blob field type, or implode your object to a serialized text form. One creates additional work, the next is resource-expensive, the other is frowned upon by database stalwarts as being “wrong”. No-win.
One must look beyond datatypes to how the data will be used and manipulated.
Yes. But since MySQL treats datatypes differently under the hood, we must pander to its storage anomalies or risk creating a lumbering behemoth. Striking the balance between the user experience and doing things for the sake of making it ultra flexible is where I’m trying to position this implementation. That is, adopt some sensible convention over configuration to ease the burden on both sides.
I really like Textpattern. I want to see it advance. I’m also highly opinionated and don’t mind too much being ignored.
Glad you like it. We do :-)
And please keep throwing ideas around and challenge our thinking. Even though you’re not a PHP fan, feel free to chuck patches and stuff this way. When we move to GitHub we’re hoping that process will be even simpler. I sincerely hope you don’t feel you’re being ignored, either now or in future. I don’t profess to hold the answers (heaven forbid that day!), I’m just trying to bring something to fruition that I hope addresses a long-standing feature request and add something which (currently, afaik) no other CMS can do to any wholly satisfactory degree. They’re all either far too limiting or way too complex to configure.
Believe me, I need all the help I can get…
The smd plugin menagerie — for when you need one more gribble of power from Textpattern. Bleeding-edge code available on GitHub.
Txp Builders – finely-crafted code, design and Txp
Offline