ELI5: front side theme asset dev workflow, low(est) friction route

gaekwad · 2021-12-21 18:16:49

It’s that time of year where I fancy myself as a Textpattern theme builder, and before something random happens in a month or so that completely floors any good intentions, I’m trying to get my head around the most efficient way of developing theme assets on a development-grade Textpattern.

I have not done theme stuff properly before, and my understanding of how they work is a bit…basic…so please bear with me as I kludge my through a scenario. There are a few questions snuck away in the words below, I would really appreciate your advice on what to do.

For this use case, let’s say I am making a theme called Example and it has the following basic structure:

./forms
  ./forms/file
    ./forms/file/files.txp
  ./forms/comment
    ./forms/comment/comment_form.txp
    ./forms/comment/comments.txp
    ./forms/comment/comments_display.txp
  ./forms/link
    ./forms/link/plainlinks.txp
  ./forms/article
    ./forms/article/default.txp
  ./styles
    ./styles/default.css
./pages
  ./pages/default.txp
  ./pages/error_default.txp
./manifest.json

There are various files inside numerous directories, all of which need injecting into our database so Textpattern can use them. My preferred way of development is an incremental change to one of more files, then see how it looks in a few browsers, adapt my code, reload the browser, compare to before, commit to a git repo, then start the process again.

I can copy + paste file contents from an IDE to the Textpattern editing space, then save it, then reload the browsers. Or, I could automate more, because I prefer to front load work where possible and be ~~lazy~~ more efficient down the line. The less time jumping through hoops to get the code into place, the better. Human error (especially mine) is a real thing, and anything I can do to minimise this would be time spent wisely. It also means I could develop one or more mediocre (or worse) themes much more rapidly…you lucky people.

How do you get code from your IDE into Textpattern for theme dev?

Me being me, I am considering a more automatic workflow:

edit the theme files in-place (i.e. in /themes/example/…
set up a background process to watch for changes in files above
have said process convert the files to SQL inserts, then insert them into the database without interaction
have another process reload my browsers locally on a regular schedule (every minute?)

(I do not know how to do all this just yet, but I’m sure it’s a solved problem with existing tools and my healthy disregard for Bash best practice.)

This would mean I could live in my IDE more, have fewer distractions, way fewer mouse clicks, glance over at one or more browsers, and avoid the risky hoopla of copy + paste into Textpattern (along with sidestepping the ever-present risk of me mullering a production site by pasting into the wrong page template since all the admin-sides use the same theme).

Is there something already available or doable that I’ve missed?

Bloke · 2021-12-21 18:58:05

A few options.

1. Edit in IDE, have the Themes panel open with your theme checked, use multi edit to Import the changes from disk. Test in browser.
2. Use the etc_flat plugin which I believe reads assets directly from disk all the while your site is in a non-production state. I could be wrong but Oleg will confirm.

Longer term, we will introduce some kind of webhook somehow but that will likely rely on better callback names across the board, and I’m only partway through refactoring all those in a local branch.

jakob · 2021-12-21 20:21:48

Yes, etc_flat can help you get some of the way there and while it won’t solve everything you mention, it makes development a lot easier. Thank you Oleg!

As Stef mentioned – and Oleg says here – it’s not quite the same as rah_flat once was. rah_flat used to import the entire theme files into the database on every page load (rah_flat still exists in a slightly simplified form but I haven’t tested it in a long while).

etc_flat instead by-passes the database and uses the templates you have in the file system when you are logged in to textpattern and the site is set to Testing or Debugging. That’s perfectly fine when developing but there are a few things worth remembering:

There’s no concept of live or development themes.
It only knows about theme files that are in the database. If you create a new form in the file system, it won’t see it until you manually “update from disk” in the Presentation › Themes panel.
Before going live, you need to actually update the theme from disk to bring in your changes to Textpattern’s database version. (this is the update SQL part).
If you want other form types, such as shortcodes, you first need to activate the advanced options in the preferences, and then add “shortcode” as a form type.

I haven’t myself set up a workflow where the browser auto-refreshes – mostly because I experiment a lot in the web inspector and auto-refresh variants tend to wipe out my experimenting before I’m done. But if you’re good at writing code without seeing what happens, an auto-refresh on change could work brilliantly.

jakob · 2021-12-21 20:51:50

Some more thoughts not strictly pertaining to your question:

If you want to make a self-contained portable theme, you need your css and js to be within the theme folder. If you’re just developing a standalone site, that’s not essential. In such cases I usually have a separate assets folder for css, js, icons, webfonts, etc. Usually I have two IDE windows open, one with the tree of txp template files and the other with the assets folder and scss/css/js etc.

To build your front-end assets, you can use a task runner like gulp/grunt/webpack etc. (like Phil does) or an app like CodeKit or a windows-equivalent (there are few like Prepos or Koala). They can also do auto-refresh on css changes too. How important auto-refresh is to you may depend on your approach to writing HTML + CSS. If you use Tailwind, the classes you add to the HTML in your forms do the work on how the site looks and your CSS changes comparatively little, so you will be more interested in auto-refresh on form changes. If you tend towards BEM-style component class naming, your CSS does more work once you have your forms in place, so auto-refresh on css changes will be more relevant. I do the latter and try and establish some parity between form names and SCSS-file naming to make it easier to quickly find the styles that pertain to a component in my forms.

Some further ideas:

Use favicons to help distinguish front-end and back-end tabs on tab-laden browser windows.
If you use SCSS and do your styling more in CSS than as HTML classes, I find it useful to work with non-compressed CSS and JS with source maps activated. The CSS and JS is still readable with my commented code and the browser web inspector tells you exactly where to find a specific CSS rule in your SCSS files. For production, however, you will want minified css and js files. By checking for the production_status in your theme, you can load the file.min.css/js on production_status=“live”. In Codekit or your task runner, build both variants. In a task runner, I believe you can simply chain the build process from non-compressed to compressed. In Codekit, you create a second main.min.scss file that imports just main.scss and has different output settings: main.scss builds to non-compressed css with source map while main.min.scss builds to minified, compressed output plus any postCSS post-processing / autoprefixing and no source map.

etc · 2021-12-22 10:38:28

IIRC, etc_flat is prior to dev themes introduction and needs to be updated (if someone still uses it). But, in some sense, it is a dev theme, because flat themes are served only in debug mode (for everyone) or in testing mode for logged-in users.

There is no automatic db sync, but I think it can be achieved with a help from etc_post if you editor is able to send http requests.

jakob · 2021-12-22 12:46:20

etc wrote #332279:

IIRC, etc_flat … if someone still uses it.

I use it all the time. I had to slightly adapt it to use it with Laravel Valet because it gets the paths wrong with the standard Valet PHP settings but otherwise it’s a key component for me.

OTOH: I practically never use the live/dev facility.

gaekwad · 2021-12-22 15:29:02

Sirs, I am humbled by your time and assistance here. Thank you so very much.

Bloke · 2021-12-22 18:22:35

jakob wrote #332284:

I practically never use the live/dev facility.

Hehe, yet I use it loads. In fact, right at this moment I’m completely retooling a live site and the current web visitors are in blissful ignorance that I have installed and am customising Hilary’s Proximo theme behind the scenes ready for flicking the switch with a deft mouse click or two. Conversely, the client can see what I’m doing, track progress and we can have dialogue over how it’ll look as the new site is developed.

This is the first site that I’ve done that hasn’t been a simple incremental update. It’s a complete departure from the original. The live/dev split being operational on live data is a game-changer for me.

jakob · 2021-12-22 18:25:14

Bloke wrote #332290:

Hehe, yet I use it loads.

Yes, it’s good there are many different ways. With you method, the owner doesn’t have to give you so much access, but I guess you’re also tied to pasting into the Presentation › Pages and Forms panels, or do you have another workflow there?

Bloke · 2021-12-22 18:41:58

Currently yes, I develop in the Txp backend which has benefits and weaknesses. Now, if there was a streamlined way to get new/changed assets onto the screen for the client, I’d adopt it. Because I would love to put the source under version control.

The main issues are:

With the site being operational on a 3rd party host, I don’t have shell access.
Changing local files means there’s an additional FTP step before they can be used by the site. Yes there are probably neater alternatives, e.g.:
1. right-click and edit file directly from Filezilla, which would copy it down, let me change it and upload on save.
2. set my IDE (Sublime Text) up to use the remote filesystem as its source.
3. write a script to watch for changes and rsync.
Creating new assets on the version-controlled content doesn’t push it to the site via etc_flat yet. Ditto with deleting stuff.
There’s no way to make new Form types from the source. We must address this so it’s possible during theme import and, by extension, development.
Syncing only works between known asset types. So new JS libraries and CSS need manual intervention anyway to make them live.

There have been a few challenges. Hilary’s new theme has a ‘gallery’ Misc Form, for example. I already used that name as a shortcode so when I flicked to the dev lane, all the gallery code broke. Hence I had to choose to either rename my shortcode, and update all places it was used, or rename the theme’s Form name and hunt for everywhere that was used in the new theme. smd_where_used to the rescue!

Also, because the assets already in use by the existing theme were read from disk in a theme-specific directory, I had to copy those across to the new theme before stuff worked. Not a biggie, just something to be aware of. IIRC, even if you clone an existing theme, it only clones the pages, forms and styles directories so any extra custom content needs to be manually copied. Just minor things to be aware of that would probably go away if the operations could be totally controlled from a disk environment.

But on the whole, the coexistence of live/dev is a relevation.

gaekwad · 2021-12-22 21:18:49

Bloke wrote #332293:

But on the whole, the coexistence of live/dev is a relevation.

You should pass your appreciation to the dev, he’s quite amenable.

Textpattern CMS

Textpattern CMS support forum

#1 2021-12-21 18:16:49