Testers needed: flat development (4.7+ only)

philwareham · 2018-04-12 10:05:58

Just wondering what the status is on this – are we likely to see something akin to this in core of 4.7.0.

And also web hooks to auto-inject flat files into database (as happens with rah_flat and oui_flat)?

Sorry to ask, but these are great additions to make themes much more useable in dev process.

etc · 2018-04-12 10:49:21

philwareham wrote #310908:

Just wondering what the status is on this – are we likely to see something akin to this in core of 4.7.0.

I think it would be more prudent to postpone it to 4.7.1+. We need some time/feedback to avoid introducing future backwards incompatibility problems.

Right now, I would add few features:

a possibility to sync without passing by Themes tab (webhook, link, etc);
a more granular sync mechanism: sometimes you need to sync just, say, CSS files or some form.

But the themes code is rather complex (for me), I wouldn’t touch it just few days before 4.7 release.

Bloke · 2018-04-12 12:05:07

Not being a user of *_flat, how does the webhook mechanism work? Is it some unique (md5ish) URL that is assigned so if you touch it, the new content is brought into the DB? Is it one URL per theme? Is it destructive (a complete resync)?

Another option is to introduce buttons in each row of the Themes table (in an ‘Actions’ column or something) that will import/export that theme immediately, no prompts, no nothing. If you’re concerned that you might hit it by mistake, the column can be hidden via the list options. Is that any faster than the multi-edit dance? It’d be a cinch to add, and no compatibility issues since the ability to do so is already there.

The only sticking point is the ‘delete/sync’ thing. But maybe we can get away with reserving the sync capability to multi-edit only, and the simple import/export buttons per row only do a merge. Less destructive.

Desirable or not?

jakob · 2018-04-12 13:07:18

Bloke wrote #310912:

Not being a user of *_flat, how does the webhook mechanism work? Is it some unique (md5ish) URL that is assigned so if you touch it, the new content is brought into the DB? Is it one URL per theme? Is it destructive (a complete resync)?

In the ‘olden days’ ;-) we had just one set of pages, forms etc. rah_flat / oui_flat syncs them into the db on every page load (as far as I am aware) when you are in testing or debug mode. It syncs everything in the designated folder with the db removing everything that’s not there.
The only files it skips are those that don’t match the naming scheme, so you could exclude (or temporarily remove) files by, for example, prefixing them with an underscore. That can be useful when trying out variants.

Once you switched to live, it would not perform the sync unless you called the URL to manually trigger a sync. On installation, the plugin generates an md5ish URL but you could change that in a pref to an URL of your choosing. If you use a build script or deploy script, you could rsync your files to the server and then call the webhook to bring them into the DB without having to go into the admin area.

jakob · 2018-04-12 13:16:43

Bloke wrote #310912:

Another option is to introduce buttons in each row of the Themes table (in an ‘Actions’ column or something) that will import/export that theme immediately, no prompts, no nothing …

I’m not sure I understand you – or I understand the advantage of that. Do you mean setting something that sets “auto-sync” from now on?

The big advantage of rah_flat/oui_flat is that you don’t “need” the admin-side at all for a lot of dev work. The two key dev situations (from my point of view) are:

When developing a theme, you edit your template in your code editor, save it and refresh your browser to see what happens (or you have some build tool that automatically does that for you). Much the same as when you edit your css and reload. It’s just way faster and perfect for trial-and-error development ;-)
When transferring changes from a site developed locally or on a staged site to the live site, you sync your changes to the live server and get your deploy script to call the webhook when it’s done. Scripts like beam (which you put me on to) and others make that easy.

I thought the suggestion you made once before of having a hook you could define in the theme’s manifest file for your particular theme was a good idea. Maybe the question of whether “standard sync“ or “wipe and sync” takes place could also be set in the manifest file. Thinking aloud, it might actually be better to put the sync credentials in a separate file alongside the manifest.json to allow you to ignore that file from a git repo.

etc · 2018-04-12 13:17:03

Bloke wrote #310912:

Another option is to introduce buttons in each row of the Themes table (in an ‘Actions’ column or something) that will import/export that theme immediately, no prompts, no nothing.

Desirable or not?

If these can be just plain URL links (not a form) then yes. We could then use them in any place on the admin side. I have tried, but some parameters of import/export requests (dunno which) seem to be fetched only from $_POST, not $_GET.

etc · 2018-04-12 13:30:55

jakob wrote #310915:

rah_flat / oui_flat syncs them into the db on every page load (as far as I am aware) when you are in testing or debug mode.

That looks a bit risky (easy to loose db edits) and suboptimal (what if nothing is changed?) I would prefer something less automatic, like fetching directly from files in dev mode and syncing once everything is fine. That’s what this plugin does, but it lacks an interface to quickly switch between two modes or sync them.

colak · 2018-04-12 14:20:29

I am testing this and loving it. I can see the styles, forms and pages being saved but we are yet to have a native way regarding js files. Will it be that hard to include if not on 4.7 on 4.7.1?

Basically, what will be needed is an extra pane under the presentation tab.

jakob · 2018-04-12 14:21:22

etc wrote #310919:

That looks a bit risky (easy to loose db edits) and suboptimal …

Sure, it was downright dangerous if you hadn’t exported stuff in the DB. I was just explaining what it did. I think your proposal of swapping to loading from files instead of the DB until such time as one is ready to bring it into the DB is good.

The hook to trigger a sync would still be useful, though.

etc · 2018-04-12 15:59:45

jakob wrote #310921:

The hook to trigger a sync would still be useful, though.

Agree, if it can be made secure enough.

colak wrote #310920:

I am testing this and loving it. I can see the styles, forms and pages being saved but we are yet to have a native way regarding js files. Will it be that hard to include if not on 4.7 on 4.7.1?

Basically, what will be needed is an extra pane under the presentation tab.

It’s not very orthodox, but in 4.7 you can create a form called, say, myScript.js, export it to the filesystem and serve from there. The server should then set the appropriate headers based on .js extension.

Anyway, I’m not sure we need a pane by mimetype. CSS and JS are the most used ones, but not unique. You might as well want to serve .txt or .html files one day.

philwareham · 2018-04-12 16:08:42

I’m of the feeling that a JS panel is best left to a plugin, if you want that editable in core. I personally wouldn’t have included a CSS panel if I was designing Textpattern from scratch – but I can see why people use it.

etc · 2018-04-12 16:15:12

philwareham wrote #310927:

I personally wouldn’t have included a CSS panel if I was designing Textpattern from scratch – but I can see why people use it.

Now that it’s there and synced with the filesystem, I would hack it for serving all type of content (CSS, JS, whatever) based, say, on name extension (CSS by default, JS for .js, etc). That’s quite easy to do.

Bloke · 2018-04-12 16:16:06

etc wrote #310918:

I have tried, but some parameters of import/export requests (dunno which) seem to be fetched only from $_POST, not $_GET.

Shame. Will have to grep the Skin directory and see if we can find ps() and change to gps().

etc wrote #310919:

I would prefer something less automatic, like fetching directly from files in dev mode and syncing once everything is fine.

If that’s possible to factor in cleanly, then great. If not, I’m happy for it to be a plugin as a stop-gap for now until we can think it through for a future release. Probably won’t be 4.7.1, since that’s a patch release (unless we can convince ourselves that this feature “patches” the behaviour of 4.7.0!) In that regard, if we can get something working now prior to 4.7.0 final, we can always tweak it in 4.7.1 as it will then be classed as a patch. Remember we have the option of 4.7.0-rc after beta.3 if we need a week of pre-launch soak testing.

colak wrote #310920:

I am testing this and loving it.

Yay!

but we are yet to have a native way regarding js files. Will it be that hard to include

I’ve always resisted a dedicated JS pane in the admin side, even though it’s not much more tricky conceptually than CSS, for exactly the reason that etc states: it’s very niche. CSS is (arguably because there’s not much else) a fundamental part of how to style a website. JS isn’t, it’s optional.

I’d far rather offer an ability to handle files that contain human-readable text better and let you decide how to serve and interpret them. Then you could use the UI to write and store JSON, text files, javascript, less, SASS, SCSS, whatever, and have their counterparts represented in the theme. EDIT: and I see etc is on the same wavelength, just a minute faster than me at replying!

We’re not there yet. Maybe we never will. But I think restricting ourselves to the one mime type is going to make it harder to offer flexibility in future as we’ll end up marginalising other equally important or valid file formats.

Last edited by Bloke (2018-04-12 16:17:00)

etc · 2018-04-12 16:22:24

Bloke wrote #310929:

Will have to grep the Skin directory and see if we can find ps() and change to gps().

Good luck, I’ve failed to find the right one.

Probably won’t be 4.7.1, since that’s a patch release (unless we can convince oursleves that this feature “patches” the behaviour of 4.7.0!)

Oh, why not. Themes are the main new feature of 4.7, so improving it based on users feedback suits my patch definition.

Bloke · 2018-04-12 16:22:48

etc wrote #310928:

Now that it’s there and synced with the filesystem, I would hack it for serving all type of content (CSS, JS, whatever) based, say, on name extension (CSS by default, JS for .js, etc). That’s quite easy to do.

I’m totally with you here.

My original thought, waaaaay back, was a ‘type’ field but using extension is probably cleaner. It means that <txp:css /> needs to only consider the relevant file types. We could keep that tag around for compatibility and convenience, but introduce a new tag that could output “text file stuff” from this table with appropriate mime types; CSS would be one of the supported things we could output, even though it conceptually duplicates the <txp:css> tag, you might want to to output it raw or something, which is something we can’t do now.

What we’d call the panel in the Admin interface is questionable though…

Last edited by Bloke (2018-04-12 16:26:35)

Textpattern CMS support forum

#16 2018-04-12 10:05:58

Re: Testers needed: flat development (4.7+ only)

#17 2018-04-12 10:49:21

Re: Testers needed: flat development (4.7+ only)

philwareham wrote #310908:

#18 2018-04-12 12:05:07

Re: Testers needed: flat development (4.7+ only)

#19 2018-04-12 13:07:18

Re: Testers needed: flat development (4.7+ only)

Bloke wrote #310912:

#20 2018-04-12 13:16:43

Re: Testers needed: flat development (4.7+ only)

Bloke wrote #310912:

#21 2018-04-12 13:17:03

Re: Testers needed: flat development (4.7+ only)

Bloke wrote #310912:

#22 2018-04-12 13:30:55

Re: Testers needed: flat development (4.7+ only)

jakob wrote #310915:

#23 2018-04-12 14:20:29

Re: Testers needed: flat development (4.7+ only)

#24 2018-04-12 14:21:22

Re: Testers needed: flat development (4.7+ only)

etc wrote #310919:

#25 2018-04-12 15:59:45

Re: Testers needed: flat development (4.7+ only)

jakob wrote #310921:

colak wrote #310920:

#26 2018-04-12 16:08:42

Re: Testers needed: flat development (4.7+ only)

#27 2018-04-12 16:15:12

Re: Testers needed: flat development (4.7+ only)

philwareham wrote #310927:

#28 2018-04-12 16:16:06

Re: Testers needed: flat development (4.7+ only)

etc wrote #310918:

etc wrote #310919:

colak wrote #310920:

#29 2018-04-12 16:22:24

Re: Testers needed: flat development (4.7+ only)

Bloke wrote #310929:

#30 2018-04-12 16:22:48

Re: Testers needed: flat development (4.7+ only)

etc wrote #310928:

Board footer