Plugin development and publication using ais_txpplugin_packager

abutcher · 2024-02-12 16:03:45

Since making one of the plugins I’ve developed public (ais_feed), I thought I might also discuss the way I package plugins in-house.

The standard way of building a plugin – as documented – is to build it as one monolithic PHP file that includes the code, manifest, textpack, and help files, but this never felt right with me. Not only does it not fit my workflow, it doesn’t fit nicely with version control, and as plugin complexity increases it clutters the file, especially when the help text is large. Rolling everything into one file as per the zem_plugin/zem_tpl methodology felt wrong on too many levels, because it doesn’t even separate different file formats.

Instead of this, I’ve always developed plugins by hand – making the folder, building the PHP file, writing the manifest.json file, the help.textile (or help.html) file, and the textpack.txp file, and then package them using my own tool, ais_txpplugin_packager.

This doesn’t do anything special, it just runs as a command line tool allowing you to package a plugin by specifying a plugin folder. It will pick up the manifest and files, and pack them together into a zipped text file for distribution to a client. The output format is equivalent to zem_tpl.php, and therefore compatible with the current version of Textpattern.

If this interests you, you can download the tool here.

Typically I write plugins to fit a very niche purpose, so it never makes sense to publish them publicly, therefore I didn’t think this would have much interest for people, but now I thought I should mention it. Note that until now this tool was an internal tool, so it’s a bit rough inside, but you might find some use.

Fortunately, with the upcoming Textpattern version (v4.9) having the possibility to export plugins out of the box, it is expected this code should become obsolete!

jakob · 2024-02-12 17:46:29

Hi abutcher and welcome, and thanks for the cool new plugin. I asked about this just the other day and it looks like exactly what I mentioned, so it’s very timely. I’ll give it a whirl soon.

One thing that I think is missing: there is also the possibility to include the file data.txp in the plugin, which can contain, for example, a dependency. By way of example, see my jcr_email_enkode plugin. Try installing that and then look in the /textpattern/plugins/ folder it creates.

abutcher · 2024-02-12 20:29:48

Hi jakob! Funny, what a coincidence!

You’re absolutely right, it didn’t support data.txp, until about a few minutes ago… :) I’ve updated it to pack data.txp too, if available. I’ve tested it on repacking the latest version of jcr_email_enkode and it seemed to pack things up properly. You should fetch the latest version from GitHub if you’re interested in trying it.

If it becomes “popular” then I will rewrite it properly. It was really just cobbled together over time as needed in a very lazy way.

etc · 2024-02-12 21:29:25

abutcher wrote #336610:

The standard way of building a plugin – as documented – is to build it as one monolithic PHP file that includes the code, manifest, textpack, and help files, but this never felt right with me. …

Instead of this, I’ve always developed plugins by hand – making the folder, building the PHP file, writing the manifest.json file, the help.textile (or help.html) file, and the textpack.txp file, and then package them using my own tool, ais_txpplugin_packager.

To make things clearer, since txp 4.8 plugins can be distributed and installed as zip folders, including standard files (manifest etc) but also eventual additional assets (images, css, js etc)

Fortunately, with the upcoming Textpattern version (v4.9) having the possibility to export plugins out of the box, it is expected this code should become obsolete!

And because of these additional assets, txp 4.9 (currently) exports plugins as zip folders, not as txt files, since we have no evident convention how to package them. So, your plugin might remain quite useful for ‘assetless’ plugins distribution, event after txp 4.9 release.

jakob · 2024-02-12 22:06:51

abutcher wrote #336612:

Hi jakob! Funny, what a coincidence! You’re absolutely right, it didn’t support data.txp, until about a few minutes ago… :) I’ve updated it to pack data.txp too, if available. I’ve tested it on repacking the latest version of jcr_email_enkode and it seemed to pack things up properly.

Thank you for so rapidly including that. There are admittedly very few plugins that use it, but it does offer potential.

You should fetch the latest version from GitHub if you’re interested in trying it.

I did and have tried it on several plugins. I initially ran into two small problems:

I get an “array to string conversion” error with PHP 8.3 in the routine that outputs the informative descriptive text strings for the plugin flags. It’s easily resolved.
It was using the integer value for plugin type rather the plugin flags – just the result of copying the line above and omitting to change both instances of the array key.

I submitted a PR for your appraisal.

…

After that everything worked fine and it makes quickly converting a directory to a compiled installer very straightforward. Really, it’s just what I was asking for in the other thread. Big 👍!

Currently the plugin outputs the txt-installer in the current folder and I was about to ask whether a custom output directory could be specified for the compiled txt-installer, but reading the plugin code, I saw you can optionally tag the name and destination on to the end as a final argument:

php ais_txpplugin_packager.php <plugin_path> [<output_file>]

In short: I like it!

EDIT: next thing to try out: make an alias of it similar to with go com’s installer described here so it can be called from wherever you need it.

etc wrote #336613:

So, your plugin might remain quite useful for ‘assetless’ plugins distribution, event after txp 4.9 release.

I think so too. Much as I like the convenience and familiarity of the .zip package installer, one can still run into difficulties with it:

The standard zip on macOS creates an additional resource fork folder tree that confuses the plugin installer. Either the resource fork needs to cleaned from such zip files, or a different command-line zip must be used.
Some systems automatically unpack .zip files on downloading and place the zip in the trash leaving you with the unpacked folder. That can confuse new users as to where the zip file has gone.

Bloke · 2024-02-12 22:12:32

If this approach is generally useful as a companion to the zip export feature then I’m happy to see what we can do to build it in or offer it as an official supported project alongside the plugin complier.

This is kind of a missing piece of the plugin puzzle and would fit nicely in with the 4.9.0 plugin changes.

jakob · 2024-02-12 22:16:15

Bloke wrote #336615:

If this approach is generally useful as a companion to the zip export feature then I’m happy to see what we can do to build it in or offer it as an official supported project alongside the plugin complier.

Both are good options :-)

etc · 2024-02-13 13:53:46

jakob wrote #336614:

The standard zip on macOS creates an additional resource fork folder tree that confuses the plugin installer. Either the resource fork needs to cleaned from such zip files, or a different command-line zip must be used.

Yep, I saw it in one of your earlier posts. Does it happen also when you create a new plugin via txp interface?

We could probably ignore these on plugin import, if there is a mean to distinguish them from ‘regular’ assets.

Bloke · 2024-02-13 15:02:21

etc wrote #336624:

We could probably ignore these on plugin import, if there is a mean to distinguish them from ‘regular’ assets.

If we could ignore dot files then that’d be awesome. It’s a royal pain not being able to zip up stuff on macOS without shell intervention. I can’t remember if the resource fork folder starts with a dot or not though.

jakob · 2024-02-13 20:18:49

etc wrote #336624:

Yep, I saw it in one of your earlier posts. Does it happen also when you create a new plugin via txp interface?

No, the zip exported from Textpattern doesn’t include the parallel folder tree. That only occurs when zipping via the finder (the mac’s file explorer).

But there are other things that can cause confusion. For example, the Safari browser will automatically unpack what it deems as safe zip files, so what you see is a folder. The .zip itself is placed in the trash, where you can retrieve it if you want. But you need to know that and many users may not be aware. You can actively deactivate that behaviour by unchecking the Safari setting ‘Open safe files after downloading’ but again users need to know that.

FWIW, Mac users don’t see this folder at all in day to day running, and it most often becomes apparent as feedback after sending a zip to users of other OSes.

We could probably ignore these on plugin import, if there is a mean to distinguish them from ‘regular’ assets.

The parallel folder tree within the zip always start with a folder called __MACOSX so is recognisable. The other file macOS sometimes adds, usually after working for a while within the folder, is a .DS_Store file.

Bloke wrote #336625:

If we could ignore dot files then that’d be awesome. It’s a royal pain not being able to zip up stuff on macOS without shell intervention.

Hence my little automator app mentioned in the other thread so you can drag and drop onto the app to clean a zip, or run it as an action from the dropdown. You could probably use Hazel or similar to watch and clean automatically when zips are created. There are other programs like Keka (donateware) which will create zips without the resource fork.

abutcher · 2024-02-14 02:26:47

Wow – had I known this would even get noticed, I might have rewritten the tool. I’m now a little embarrassed because it’s really an internal junk script :)

jakob wrote #336614:

I submitted a PR for your appraisal.

Thank you for the patches! They’ve been merged. I forgot the flag output wasn’t working when I “cleaned” it up to make it public – woops. As for PHP 8.3, my dev environments are usually “out of date” to match the target hosts, but yes it was a silly brain-dead bug.

EDIT: next thing to try out: make an alias of it similar to with go com’s installer described here so it can be called from wherever you need it.

It actually started life with a #! /usr/bin/php shebang at the top so it could be an executable, but changed later on because for some plugins it’s used in a set of scripts to build things. It’s a good idea. I’m thinking about trying to carve out some spare time to clean it up first…

etc wrote #336613:

To make things clearer, since txp 4.8 plugins can be distributed and installed as zip folders, including standard files (manifest etc) but also eventual additional assets (images, css, js etc)

And because of these additional assets, txp 4.9 (currently) exports plugins as zip folders, not as txt files, since we have no evident convention how to package them. So, your plugin might remain quite useful for ‘assetless’ plugins distribution, event after txp 4.9 release.

I hope not :) it should be rewritten if that’s the case! I think zip packages are fine so long as it’s robust and relies on the metadata. As much as I don’t like working with the plugin template, I do like a structured package format because it encourages a consistent installation path. I have plugins that use the plugin lifecycle functionality to set up additional database schema, so they get packaged as .txt to ensure it gets installed via the UI without confusion.

There is nothing easier than having a blob of text you can copy and paste into to install or update a plugin. It’s neat and saves maintenance time for sure. I probably would have structured it a bit differently to be a bit more robust as a format in itself, but I don’t think I’m involved in the project enough to comment.

jakob · 2024-02-14 09:30:26

abutcher wrote #336633:

Wow – had I known this would even get noticed, I might have rewritten the tool. I’m now a little embarrassed because it’s really an internal junk script :)

No worries. I’m no different, and many of these things only come to light through other pairs of eyes and systems. The plugin helps me, so I’m thankful. Thanks for taking on the PR 👍

Textpattern CMS

Textpattern CMS support forum

#1 2024-02-12 16:03:45