Compose a plugin on demand

etc · 2013-04-01 18:20:10

Dear txp gods,

what would be the best way to automatically compose and compress a plugin on the fly according to user request? I think of including/excluding some calendars in etc_date via, say, html form.

Thank you.

MattD · 2013-04-01 19:45:01

The plugin composer plugin does this doesn’t it? Take a look at how?

etc · 2013-04-01 20:36:26

MattD wrote:

The plugin composer plugin does this doesn’t it? Take a look at how?

That’s what I had to think of, indeed (c’mon, boy, it’s only 3000 lines of code :). It looks like ied_plugin_save_as_file function contains all plugin ingredients.

Thanks, Matt!

etc · 2013-04-02 08:33:01

I’m mostly through technicalities now, but need your advice on conceptual questions: how such customizable plugins should be designed? How to store the main “template” and the pluggable “components” (calendars): as deactivated plugins, as files or something else? How to integrate it with plugin composer for easy testing? Is such a thing interesting at all?

Bloke · 2013-04-03 08:53:30

If you really wanted to do this I’d be tempted to store each plugin code (.php) as a Txp file and each help text in Textile format as a .textile file. Outside document root of course (fwiw I have a plugin for that called smd_remote_file. The new — not yet publicly available — version I’ve written and had working successfully on sites allows you to store “secure” files inside your Txp filesystem out of docroot, and manage them all from Txp’s Files panel. If you’d like a copy, just let me know).

Anyway, if you have say:

/some/non-docroot/path/to/etc_date_v0.20.php
/some/non-docroot/path/to/etc_date_v0.20.textile

Then there’s nothing to stop you setting up a Page template that has the plugin template in it, like the one in ied_plugin_save_as_file.

If your URL was site.com/plugin/create?plugin=etc_date&version=0.20 then you could extract that URL var, sanitize it and at suitable places in your page template fetch the content of the file to put between the # BEGIN PLUGIN CODE and # BEGIN PLUGIN HELP markers. Once you have that bit done, a few lines of PHP at the end of the Page template could set the correct mime header and serve the file either as a PHP file or it could chunk_split / base64_encode / gzip as required and serve the compressed / text version. Again, those parameters could be supplied in the URL site.com/plugin/create?plugin=etc_date&version=0.20&as_text=1&zip=1.

Sky’s the limit :-) I could probably build this feature into Composer if it was generally useful.

etc · 2013-04-03 14:06:10

Many thanks for your advice, Stef, much appreciated. Yes, that’s the way I’d go, but if you have time/interest for it, this is the solution I prefer, and by far:

I could probably build this feature into Composer if it was generally useful.

That would be great, but challenging (hehe)! Pluggable calendars is a natural usecase. Another one I can think of is rah_unlog_me: it could be served in admin or admin+public flavors following the selected options. Are there other cases – that’s a part of the question.

Well, if we assume it’s worth doing – how would you do it? I guess, there would be no Page template anymore, you’d rather make it part of plugin development process. Does it mean you have to introduce some $plugin['template'] field for development, and use plugin['code'] for distribution? Should you also add some $plugin['components'] field to store the list of admissible URL vars and create corresponding checkboxes for easier testing?

Anyway, you’ve got already a first supporter/tester for this, hopefully not last. :)

Bloke · 2013-04-03 16:45:13

etc wrote:

if we assume it’s worth doing – how would you do it? I guess, there would be no Page template anymore, you’d rather make it part of plugin development process.

Not really thought it through, but wouldn’t a tag suffice?

<txp:ied_plugin name="etc_date" version="0.20" zip="1" text="1" />

That would go and fetch the values for the given plugin name+version, fit them into the template, set the template attributes and then return the plugin as a .php template file or (compressed) .txt file. The tag could be used on public pages and also from the admin side if you used something like smd_tabber.

Therefore, how you plug the values into the tag is up to you: from the results of a form, adi_gps, from the URL / direct from a hyperlink, from AJAX, etc. You could choose to embed it in a Page template and set the mime types accordingly to serve the content as a download or display it on the page or in a textarea or whatever. There might be scope for having the ied_plugin tag optionally set the relevant mime types, content-disposition and content-length for you (maybe?)

Would that work? Any further/better ideas?

Last edited by Bloke (2013-04-03 16:46:38)

etc · 2013-04-03 19:37:36

It looks great, save for testing facility. Currently, when you save a plugin, it is immediately available. Here, you have to go on the public side (or to smd_tabber tab), fill out a form, grab the plugin and install it. Would it be possible to make it work the good old way?

Bloke · 2013-04-03 20:11:43

etc wrote:

It looks great, save for testing facility. Currently, when you save a plugin, it is immediately available. Here, you have to go on the public side (or to smd_tabber tab), fill out a form, grab the plugin and install it. Would it be possible to make it work the good old way?

I’m afraid you’ve lost me :-s

Your opening post asked about composing and compressing a plugin on the fly from a user request , a.k.a. the public side, right? If the composer had a public tag, you could do that, yes?

If you want to compose and export a plugin you have written, you use the admin side tools to create / upload it, write the help docs and export it in a variety of formats using the direct links on the Plugin List panel or under the ‘Distribution’ group at the bottom of the Edit panel (the things you see here in the interface are configurable from the plugin’s Setup panel).

Once you have composed the plugin and have set the “enabled” checkbox from Composer’s interface then yes it is immediately ‘available’ (active, running). The tag could have an attribute to only output ‘enabled’ plugins if that would help? I guess also a <txp:ied_plugin_list> tag might be handy so you can list certain types of plugin and output various bits of info about them.

But I’m not quite sure I follow your logic here so I’ve probably missed something obvious. What do you mean by a testing facility? And what is it exactly you are trying to achieve that the Composer doesn’t currently do?

Last edited by Bloke (2013-04-03 20:13:03)

etc · 2013-04-03 20:45:13

I was not clear enough, sorry. Suppose that we write some date-related plugin, that includes various calendars, like this:

... main code ...

function calendar1 (...) {...} // Hijri calendar

function calendar2 (...) {...} //US holidays

function calendar3 (...) {...} // Jalali calendar

Now, a typical user wont need all these calendars at once (and they could be many). So, the idea was to let him choose the ones he needs, by sending a request like ?plugin=etc_date&calendar[]=hijri&calendar[]=jalali, and compose a plugin on the fly, including only the required functions.

How would <txp:ied_plugin /> do it? We could create a plugin template in the spirit of

... main code ...

//# IF REQUEST hijri function calendar1 (...) {...} // Hijri calendar

//# IF REQUEST us_holidays function calendar2 (...) {...} //US holidays

//# IF REQUEST jalali function calendar3 (...) {...} // Jalali calendar

and then process it with <txp:ied_plugin /> to generate the code

... main code ...

function calendar1 (...) {...} // Hijri calendar

function calendar3 (...) {...} // Jalali calendar

that will be delivered to the user as (zipped, php, etc) plugin. Is that what you meant?

Bloke · 2013-04-03 22:19:35

Yikes, conditional compilation. That’s a bit heavier than I anticipated from the OP!

I now see where you’re going with this, thanks for the clarification. But I can’t immediately see a neat way of offering that functionality from the Composer itself. In your case it might be fairly simple as long as each calendar is a standalone block and there are no references to them from other parts of the code. I assume these are public tags, yes? But in other use cases there may be internal references that would break code if parts were omitted so this really is up to the developer to make sure all possible modules work both independently and in combination.

Something like this starts moving into the realm of bundled content, like jQuery UI where you can pick and choose which bits of the API you need. That in turn brings about the notion of ‘packages’ and ‘dependencies’ which is something the old skool plugin template format does not currently support.

One way might be to introduce such a dependency paradigm and have the plugins as separate ‘modules’ — essentially different files — that can be clumped together into a single download. I know Jukka was thinking about multi-install plugins but in this case it’s not very clean because they’re all actually part of one plugin so you don’t want the user to see 2, 3, 4, 5, 6… different plugins in their database based on their chosen options.

The other way is with special markers in the code that wrap or somehow delineate conditional blocks; a bit like you suggest. The various parts could only be included in the final output if some incoming data stream matched the marker. Then of course there’s what to do with documentation for omitted functions(!). And how to handle this in an OOP environment.

I don’t really know at the moment if this sort of thing is broadly useful given our current template format. Perhaps if the structure of plugins was improved with proper meta data associated with each and maybe bundled folders of plugin code and docs, then it might be useful. But then part of Textpattern’s plugin appeal is that each one is a simple block of text that can be treated and served as text and installed. Any improvements to the format should not do so at the expense of making things massively more complicated for the end user.

Last edited by Bloke (2013-04-03 22:21:27)

etc · 2013-04-04 08:59:39

Conditional compilation – oh yeah! – sounds great. I had no clear idea about the way to choose for calendars distribution (include them in the main plugin? make them library plugins? other?) in the OP. This conditional thing came from your replies, though we thought of not exactly the same stuff. :) I realize that’s a big and, probably, unnecessary move, but anyway:

it should extend, but not break the current format;
plugins are under developers responsibility: if you don’t need/understand this compiling directives – don’t use them;
dependency/metadata looks like much too big move, but I am too ignorant to judge;
special markers seem to be the most straightforward way, but could be too limited.

Maybe, we should wait for feedback from plugins authors, to see if conditional compilation could be useful. Hopefully, the topic is clearer now from your explanations.

Thank you again!

Gocom · 2013-04-04 12:37:33

The default plugin package should be nothing more than package standard, a standardized format. Please, let’s keep it that way too.

As compiling goes, if you want to compile plugins with dependencies and what ever, you can. Building compilers ‘very’ exactly hard, especially if you just need to bundle data. For data, only thing you would need to is to bundle that said data in some storage format to the source code. Done.

Altho, if you just need to have configuration or some extra options, I would do them with individual optional plugins, be libraries or whatever, that extended the main plugin. On-demand compiled custom packages are cool, but what you end up with is horrible mess of where no one knows what the fuck they are actually using. It’s serious debugging hell.

As far as my work goes, I myself do some minimal extra ‘compiling’ processes. Basically it’s nothing more than joining and composing package from multiple separate source files with the help of MassPlugCompiler and rah_blobin.

I previously had rah_plugin_download that did on-demand packaging directly from the database, but dropped that since it was the stupidest idea of the year. Horrible mess. Made actual development complicated and the package and database format aren’t even 1on1.

etc · 2013-04-04 16:05:09

Jukka, thanks for your reply.

Gocom wrote:

The default plugin package should be nothing more than package standard, a standardized format. Please, let’s keep it that way too.

Compiling (or whatever they are) directives would not invalidate the standard format, just add some php-neutral markup.

For data, only thing you would need to is to bundle that said data in some storage format to the source code.

Sure, but since most of us use Composer for plugin development, having some #import directive at hand would simplify things.

Altho, if you just need to have configuration or some extra options, I would do them with individual optional plugins, be libraries or whatever, that extended the main plugin.

I agree, and will probably end like that, but one has to care about naming conventions. In my case (pluggable calendars), I don’t know in advance which calendars will be installed, but would like to retrieve their names.

On-demand compiled custom packages are cool, but what you end up with is horrible mess of where no one knows what the fuck they are actually using. It’s serious debugging hell.

Agree too.

As far as my work goes, I myself do some minimal extra ‘compiling’ processes. Basically it’s nothing more than joining and composing package from multiple separate source files with the help of MassPlugCompiler and rah_blobin.

I will give them a try, thanks.

I previously had rah_plugin_download that did on-demand packaging directly from the database, but dropped that since it was the stupidest idea of the year.

It’s a new year. :)

Last edited by etc (2013-04-04 16:06:32)

colak · 2013-04-04 16:55:35

This discussion is out of my depth but what if there is a standard plugin which calls for “libraries” which can be downloaded on demand. Would this make the process simpler?

Textpattern CMS

Textpattern CMS support forum

#1 2013-04-01 18:20:10

Compose a plugin on demand

#2 2013-04-01 19:45:01

Re: Compose a plugin on demand

#3 2013-04-01 20:36:26

Re: Compose a plugin on demand

#4 2013-04-02 08:33:01

Re: Compose a plugin on demand

#5 2013-04-03 08:53:30

Re: Compose a plugin on demand

#6 2013-04-03 14:06:10

Re: Compose a plugin on demand

#7 2013-04-03 16:45:13

Re: Compose a plugin on demand

#8 2013-04-03 19:37:36

Re: Compose a plugin on demand

#9 2013-04-03 20:11:43

Re: Compose a plugin on demand

#10 2013-04-03 20:45:13

Re: Compose a plugin on demand

#11 2013-04-03 22:19:35

Re: Compose a plugin on demand

#12 2013-04-04 08:59:39

Re: Compose a plugin on demand

#13 2013-04-04 12:37:33

Re: Compose a plugin on demand

#14 2013-04-04 16:05:09

Re: Compose a plugin on demand

#15 2013-04-04 16:55:35

Re: Compose a plugin on demand

Board footer