To 4.6 and beyond ...

Bloke · 2015-07-06 14:06:50

candyman wrote #292643:

Glad to hear that glz_custom_fieds and tom_image_grid could be included: I use both.

glz_cf is not being included in core. It handles fields in a completely different way to the manner in which I’m proposing, not least of which is that it’s wasteful of space (because of Txp’s underlying table structure I hasten to add, no fault of Gerhard’s code).

Under the hood the core’s new feature is called The Meta Store, because it’s all about efficiently and flexibly storing meta data against content. Not necessarily just articles, but all content types, including Users, Sections and Categories.

It’s all about customisation. If you wanted one article custom field in Txp today, it has to be a text string under 255 characters. In the next version, if you only want one custom field, it can be any widget you like. Text input, checkbox, textarea, select list, multi-select checkbox set, radio set, … and the underlying core technology routes it to a table that is most suited to that type of data. If we do our jobs right, you’ll never know that there are five different tables serving the data for a single article, it’ll just work.

From an application viewpoint (and this is what’s taken the time in development, and four rewrites, virtually from scratch), the notion of a ‘Field’ and a ‘Field Set’ manage all the database interaction, creating and managing data based on its nominated data type. It maps those data types to widgets, migrating data from type to type if it can, and lots more besides. All this is presented from a single panel to manage your entire fleet of custom fields, as simply as possible.

Plugins can add their own data types. Heck, a plugin can add its own content type (video, anyone?) and hook into The Meta Store to stash meta data against it. No need to use its own tables unless really necessary. As far as I’m aware, nobody out there in CMS land offers this kind of potential, packaged in this flexible way, and stored as efficiently as this.

People who use glz_cf today will have a much richer palette of options to choose from when designing their sites. The next main issue to tackle is how to convert existing glz_cf data to the new Meta Store. It doesn’t do that. Someone (me, probably) needs to write a mini helper plugin that can map glz_cf data safely. It’s not reasonable that the core panders to plugins, regardless of how popular they are because it opens up a whole slew of compatibility issues. Txp only officially supported text input custom fields, so the Meta Store only migrates text input data over. Hence a helper plugin is required to either move glz_cf data of other types over first, or to instruct Txp which types of data each field represents in the Meta Store.

It’s a similar story with tom_image_grid. A grid has been on my core wish list for ages. Sacripant wrote a plugin and it has some elements that make a lot of sense for core, so it would be silly of us to write it from scratch when we could borrow the best ideas from the plugin. Some of the problems I’d been scratching my head about were solved in quite an elegant manner in that plugin.

This is not about adopting plugins, it’s about growing the core in places that benefit the most users while keeping it nimble and extensible enough that more functionality can be easily added by plugins. That’s always been the Txp philosophy, and it’s not one I’m about to change.

hcgtv · 2015-07-06 14:14:05

Bloke wrote #292641:

Admin layout update, with suitable UI for the tag builders. Prefs panel improvements included. Integration with the help system would be a bonus, along with more complete Textpack translations.

This is the best part of the next version, makes it worthwhile.

Reducing reliance on the RPC server, packaging up help locally for distribution if possible. Pophelp improvements to go along with this. Requires community involvement to get stray help files translated.

How about a textpattern-4.6.zip with an additional textpattern-lang-4.6.zip ?
Include en-gb.txt in the default install, need more languages, download the lang pack.

Or, do like Dokuwiki does and prepare the download with the language packs you select.

Basic themes support, now I have a clear idea of how to do it. Templates and Packages (the extensions to Themes you proposed) can appear in subsequent 4.6 point releases. Basic themes slated for inclusion this week, providing no major interruptions.

Na, this can wait, we don’t need no stinking Themes :)

The improved Ruud/etc parser and associated speed optimisations.

This I’d like to see added so I can re-run tests and make bumper stickers with the benchmarks.

BS #1: My CMS kicks your CMS’s ass.
BS #2: Textpattern CMS, you have to use it to believe it. (sounds familiar can’t quite pun my finger on it)
BS #3: Textpattern, sucks less than WordPress.
BS #4: PHP in templates is so last decade.
BS #5: Textpattern CMS, when you’ve outgrown the hand holding.
BS #6: Textpattern CMS, when you’re tried of constant security updates.

Squash mysql_*() deprecated errors, even with a temporary fix like using mysqli_*() instead.

These would be PHP error messages to view I presume, time to change my PHP error logging parameters.

Given that MLP is not going to work under 4.6 (and it’d be a real headache to make it compatible), some mechanism to allow a proper multi-lingual plugin to appear, or better internationalisation support baked into core. Without this, people with MLP sites won’t be able to upgrade.

This is something I’ve never toyed with, there’s always Google Translate.

Given how long it took me to get the first commit done, unless there’s significant community involvement in the custom-fields branch, that might wait. Or it may be that we get Article fields done for 4.6.0 with a suitable glz_cf migration plugin available, and then roll out CFs across other content types in subsequent point releases.

This is a major change to how we do things, it would be better to test this either on it’s own stable branch, or after all the rest of the changes have been deemed stable.

Finishing the object code as much as makes sense (/vendors directory), although that’s a little above my pay grade right now so it’d be better if someone else could do that.

Could you elaborate on this, cause this changes quite a bit of stuff.

Bloke · 2015-07-06 16:03:35

hcgtv wrote #292648:

Could you elaborate on this [/vendors], cause this changes quite a bit of stuff.

I wish I understood half of it to elaborate! It helps with a few things. The autoloader means no long lists of includes are necessary. Using the static Txp::get('Textpattern_Some_Object') is great because you can call methods on it straight away without having to instantiate the object first. Saves a step when you’re just fetching stuff.

But as to how it all works under the hood with factories and stuff, well, I find it rather impenetrable. Although written from the context of frameworks, this piece on Why I hate Frameworks sums up the whole way that object orientation is going. You no longer write software to do a job well, with maximal reuse and tonnes of encapsulation, you write software to make other chunks of software that make other bits of software that ultimately do stuff. Meh.

At a simple level, the /vendors folder takes a lot of stuff that was bunged in txplib_misc.php or other “central” locations, and separates it into a tree of component parts that can be more easily used and reused. Easier to find stuff too as it’s all packaged into logical groups based on what that set of things do. If you poke around in each folder you’ll see how stuff is organised.

Effectively, we need to do more of that with other objects. Ultimately, we’d like to separate the display parts of the /include folder too so plugins can do more without being burdened by our enforced UI rendering steps. But that’s waaaay down the line.

hcgtv · 2015-07-06 16:43:10

Bloke wrote #292652:

At a simple level, the /vendors folder takes a lot of stuff that was bunged in txplib_misc.php or other “central” locations, and separates it into a tree of component parts that can be more easily used and reused. Easier to find stuff too as it’s all packaged into logical groups based on what that set of things do. If you poke around in each folder you’ll see how stuff is organised.

I’ll give the Vendors folder a more intense look over. Thanks for the explanation.

gaekwad · 2015-07-06 16:48:21

Bloke wrote #292647:

Under the hood the core’s new feature is called The Meta Store, because it’s all about efficiently and flexibly storing meta data against content. Not necessarily just articles, but all content types, including Users, Sections and Categories.

Cool. When can we see what you’re working on?

Destry · 2015-07-06 16:48:42

hcgtv wrote #292593:

why not release a [4.6/4.7 split of what’s currently planned for 4.6]

:) +1

I’ve voiced for this idea before. But for me it was even the little things that would make a big difference, like markup filters, new adjustments to image captions (so title="" attributes didn’t throw them), etc.

I’m sure there are a lot of little goodies like that in there that site managers can start implementing to improve workflow and architecture now. Get it done and dusted and be ready for bigger changes later.

Last edited by Destry (2015-07-07 08:29:51)

Bloke · 2015-07-06 17:24:06

gaekwad wrote #292657:

Cool. When can we see what you’re working on?

Here.

gomedia · 2015-07-06 23:55:07

Bloke wrote #292647:

The next main issue to tackle is how to convert existing glz_cf data to the new Meta Store.

Will the existing glz_cfs run in parallel with the new Meta Store? If so, users will at least be able to upgrade to 4.6 and then convert.

This is not about adopting plugins, it’s about growing the core in places that benefit the most users while keeping it nimble and extensible enough that more functionality can be easily added by plugins.

Yes, it’s about TXP adopting ideas – wherever they come from. Textpattern as a CMS can’t stand still.

And “keeping it nimble”, or more importantly “making it more nimble” applies to the admin workflow as well – anything that smooths and greases the path of making websites using TXP is to be applauded.

Bloke · 2015-07-07 00:14:48

gomedia wrote #292694:

Will the existing glz_cfs run in parallel with the new Meta Store? If so, users will at least be able to upgrade to 4.6 and then convert.

The plan was to make them exclusive. On upgrade, every custom_N column is created in the Meta Store table and then all CF data is copied from the textpattern table to value tables of the appropriate type. I was going to add a pre-upgrade callback hook in which I was planning to tell the core that what glz_cf calls a select_list is a core selectList, and a multi-checkbox is a checkboxSet, and so forth. Then, during upgrade Txp can migrate the column data safely.

But if we simply leave it so that only known types are converted, in theory (untested as yet) anything out of the ordinary will be left behind. The entries in prefs that define the custom fields remain. The columns in the textpattern table remain. In that case, you’d have a hybrid system, but I don’t know how it will manage at present. Will probably explode very ungracefully!

Take a look at the bottom of update/_to_4.6.0.php in the custom-fields branch and see what you think. As of right now it only removes columns and custom field pref definitions if every bit of data from every article is successfully migrated. When I was testing, sometimes that didn’t happen but it may have been because I was not going from a “clean” Txp 4.5.7 system each time, as there may have been some remnants of prior upgrade attempts floating around. Or my maths could have been broken.

I’d much rather NOT run them side by side and figure out a way to migrate everything on upgrade. But I’m not sure if that’s possible, especially when factoring in the glz custom scripts. If you can bring fresh eyes to this, I’d be most grateful for any assistance in figuring out the best path. Ta.

gomedia · 2015-07-07 00:41:19

Bloke wrote #292695:

I’d much rather NOT run them side by side …

Nothing wrong with the big bang approach … forcing the issue is often the best way.

As you can guess, adi_matrix is my elephant in the room. At least now I know I don’t have to install glz_cfs on my SVN site. But I will need to download the 4.6 unlimited CFs branch/twig/tree-limb-analogy to have a play. adi_menu is on the operating table at the moment, so bear with me. Next patient please!

makss · 2015-07-07 17:48:08

265 needs testing.

I always use this patch since Sep 3, 2012. With txp 4.6 it works too.
I would suggest the next version of this patch. How is it best done? via Pull Request?

Example trace output after the change: (next version of this patch)

<!-- txp tag trace: 
Mem(Kb)_|_+(Kb)_|_Trace___
   5092 |  5092 |[SQL (0,0003349781036377): select name, data from test1com_txp_lang as txp_lang where lang='ru-ru' and event in('public','common')]
   5165 |    73 |[SQL (0,00012803077697754): select name, code, version from test1com_txp_plugin as txp_plugin where status = 1 AND type IN (0,1,5) order by load_order asc, name asc]
   5202 |    37 |[SQL (0,00012397766113281): select page, css from test1com_txp_section as txp_section where name = 'default' limit 1]
   5118 |       |[SQL (0,00012302398681641): select user_html from test1com_txp_page as txp_page where name = 'default']
   5123 |     5 |[Страница: default]
   5166 |    43 |<txp:lang />
   5177 |    11 |<txp:text item="lang_dir" />
   5177 |       |<txp:page_title />
   5178 |     1 |<txp:css format="link" media="" />
   5178 |       |<txp:site_url />
   5179 |     1 |<txp:if_search>
   5180 |     1 |	[<txp:if_search>: false]
   5188 |     8 |	<txp:if_category>
   5190 |     2 |		[<txp:if_category>: false]
   5194 |     4 |		<txp:if_author>
   5195 |     1 |			[<txp:if_author>: false]
   5194 |       |		</txp:if_author>
   5190 |       |	</txp:if_category>
   5190 |       |	<txp:if_section name="">
   5196 |     6 |		[<txp:if_section name="">: true]
   5198 |     2 |		<txp:site_url />
   5191 |       |	</txp:if_section>
   5182 |       |</txp:if_search>
...skip...

The first column – used memory. The second column – an additional allocation memory. Reducing the memory not shown so as not to clutter up the output.

For me, this is very useful trace output.

Bloke · 2015-07-08 23:06:06

makss wrote #292775:

For me, this is very useful trace output.

Yes, I like this. Very handy.

Merged, thank you very much for writing, testing and submitting the Pull Request.

Textpattern CMS

Textpattern CMS support forum

#13 2015-07-06 14:06:50

Re: To 4.6 and beyond ...

candyman wrote #292643:

#14 2015-07-06 14:14:05

Re: To 4.6 and beyond ...

Bloke wrote #292641:

#15 2015-07-06 16:03:35

Re: To 4.6 and beyond ...

hcgtv wrote #292648:

#16 2015-07-06 16:43:10

Re: To 4.6 and beyond ...

Bloke wrote #292652:

#17 2015-07-06 16:48:21

Re: To 4.6 and beyond ...

Bloke wrote #292647:

#18 2015-07-06 16:48:42

Re: To 4.6 and beyond ...

hcgtv wrote #292593:

#19 2015-07-06 17:24:06

Re: To 4.6 and beyond ...

gaekwad wrote #292657:

#20 2015-07-06 23:55:07

Re: To 4.6 and beyond ...

Bloke wrote #292647:

#21 2015-07-07 00:14:48

Re: To 4.6 and beyond ...

gomedia wrote #292694:

#22 2015-07-07 00:41:19

Re: To 4.6 and beyond ...

Bloke wrote #292695:

#23 2015-07-07 17:48:08

Re: To 4.6 and beyond ...

#24 2015-07-08 23:06:06

Re: To 4.6 and beyond ...

makss wrote #292775:

Board footer