soo_plug_up: Automatic version check and update for compatible plugins

candyman · 2011-01-09 12:34:36

I’ve missed this thread: wow! It ‘s a long time that I’m looking something like this! I hope that it could become a standard. Txp community deserves its Txp App Store! ;)

jsoo · 2011-01-09 12:42:29

Something similar (though without the one-click upgrade option) is planned for Txp 5 as a core feature. That is, check the .org feed and let you know when plugin upgrades are available. Even after Txp 5 is released I expect to keep my plugin available for those who prefer to live dangerously and like the one-click ;)

Themroc · 2011-01-24 21:51:22

Some plugin update mechanism would be a good thing. But why try to force the download sites to install extra software? I’d suggest asking plugin authors to add one little text-file, perhaps called http:/example.com/tpx-plugins/txp_plugins_list.txt:

xyz_awesome_thingy 0.1 4.2.0-4.3.0 http://example.com/tpx-plugins/xyz_awesome_thingy_v0.1.txt
xyz_stupid_crap 0.0.1 4.0.1- file://dev/zero
xyz_too_old 1.0 -4.0.1 http://web.archive.org/web/19991203101453/http://example.com/….

and do a http “head” request for ‘em like every hour.

Last edited by Themroc (2011-01-24 21:55:13)

jsoo · 2011-01-24 22:07:49

Thanks Themroc. I’ve dropped my original idea for using feeds direct from plugin download sites. I’m going to use the .org feed instead.

Themroc · 2011-01-24 23:03:57

jsoo wrote:

Thanks Themroc. I’ve dropped my original idea for using feeds direct from plugin download sites. I’m going to use the .org feed instead.

Well, Bloke’s #6 sounded like he’s also planning something (imo) overly complicated which plugin-download-sites would have to implement.

Bloke · 2011-01-24 23:28:08

Themroc wrote:

Well, Bloke’s #6 sounded like he’s also planning something (imo) overly complicated which plugin-download-sites would have to implement.

[OTish]: plugin download sites don’t have to do anything other than be repositories for the code. If you want to continue to manage your plugins on your own site and manually update textpattern.org, that’s entirely up to you. Phase 2 is a (sketchy) plan for a way of automating the procedure so the information held on .org could be updated directly from the plugin author’s site to save you the hassle of doing it twice.

Unfortunately, I don’t believe it’s as simple as writing a text file on your site for a few reasons:

1) security: you need to only be able to update your own entries on .org (or those which you officially maintain) and the backfeed must be impervious to attack / injection. TXP has a strong security record and I don’t want someone’s plugin entries being hijacked to redirect to someone else’s site when you hit ‘Download’ where a malicious version of the plugin resides
2) changelogs: I wanted the option to be able to write optional changelog description info to the .org meta data. If the data is supplied in the feed, fine. If not I wanted to give you the option of writing it there and then or deferring it till later (or not bothering at all, if that’s your thing)
3) you don’t always want all updates to be pushed to .org — think development systems, libraries and such like
4) by the time you’ve gone to the hassle of editing a txt file, entering the new stuff, commenting out the bits you don’t want, and uploading it to your site you could have just hit the button in your .org profile page to automatically suck everything in and uncheck the one(s) you don’t want. I’m trying to remove the barriers to people keeping .org up-to-date, not put more in the way!
5) cron: I’ll never be trusted enough to have cron access to write a script that polls 700 web sites every hour for a HEAD feed. Never.

That said, if you can come up with a way to address all the above for plugin authors like me with 30 plugins that are in a constant state of flux and allow me to sync my site with .org wth one (or no!) clicks so it can a) figure out exactly what I want it to do, b) never clobber anyone else’s entries, and c) never be compromised by someone injecting malicious code into the backfeed, then I’ll seriously consider it. I do have a tendency to over engineer stuff: I’m all for making it simpler to manage and sometimes I just need to be shown the light.

jsoo · 2011-01-24 23:30:07

Bloke’s genius doesn’t necessarily include the ability to explain simple things simply :)

Seriously, yes it isn’t the ideal zero-step solution for plugin authors. Bloke and I (and others) all want to see a setup where a plugin author doesn’t necessarily have to visit .org to update info. But I think we will have to approach that by degrees.

Bloke · 2011-01-24 23:31:43

jsoo wrote:

Bloke’s genius doesn’t necessarily include the ability to explain simple things simply :)

Touché :-)

Themroc · 2011-01-25 18:44:46

Bloke wrote:

[OTish]: plugin download sites don’t have to do anything other than be repositories for the code. If you want to continue to manage your plugins on your own site and manually update textpattern.org, that’s entirely up to you.

By “would have to implement” i meant “would have to implement if they wanna participate in that new auto-update-scheme”. Sorry, i often seem to assume ppl can read my thoughts…

1) security: you need to only be able to update your own entries on .org (or those which you officially maintain) and the backfeed must be impervious to attack / injection. TXP has a strong security record and I don’t want someone’s plugin entries being hijacked to redirect to someone else’s site when you hit ‘Download’ where a malicious version of the plugin resides

Nobody wants that. But it does not matter if you send a “GET” to

http://example.org/update?cmd=get_version&plugin=name and then http://example.org/update?cmd=get_dl_url&plugin=name or
http://example.org/update?cmd=get_all_data_at_once&plugin=name or
http://example.org/plugin_name_list.txt

the level of security would be the same in all cases.

2) changelogs: I wanted the option to be able to write optional changelog description info to the .org meta data. If the data is supplied in the feed, fine. If not I wanted to give you the option of writing it there and then or deferring it till later (or not bothering at all, if that’s your thing)

That is a valid point imo. Automatically adding changelog info to the .org-site would make the textfile method a bit more complicated, but still simpler overall. See below.

3) you don’t always want all updates to be pushed to .org — think development systems, libraries and such like

NP, they just don’t include those in the text-file (or you could introduce a “don’t update!” flag in there).

4) by the time you’ve gone to the hassle of editing a txt file, entering the new stuff, commenting out the bits you don’t want, and uploading it to your site you could have just hit the button in your .org profile page to automatically suck everything in and uncheck the one(s) you don’t want. I’m trying to
remove the barriers to people keeping .org up-to-date, not put more in the way!

That text file doesn’t have to be a real static file. If a plugin author runs his own textpattern (or maybe some other cms), he can have this file generated on-the-fly from his db. But for ppl who have their HP on some hoster without php or db-access, it is impossible to install software for auto-update there.

5) cron: I’ll never be trusted enough to have cron access to write a script that polls 700 web sites every hour for a HEAD feed. Never.

One poll needs at most 400 bytes, including reply. Multiplied by 700 gives 280K bytes per hour (or 78 bytes per second). Ok, that could be a problem… But we don’t have to poll every hour, once a day should be enaugh. In urgent cases (like security updates) plugin authors could still hit that new “update now!” button on .org. Furthermore, it aint 700 sites. http://textpattern.net/wiki/index.php?title=Reserved_Plugin_Prefixes lists only 230 Authors, half of whom surely are no longer active. So, assuming 150 active authors, that cron-job would have to shuffle only 60 Kbytes per day or 0.7 Bytes per second or 2 Mbytes a month. Now, that shouldn’t hurt anybody nowadays. You don’t even need access to the local crontab, use something like http://www.onlinecronjobs.com/. Or, if the polling really has to be do more often, a vserver without traffic restriction is available for 5 euro a month. Currently, i have one of those, it could easily poll every 5 minutes.

The updated “textfile”-scenario:

Plugin author has just implemented the wonderful new feature X in his “awesome_thingy”-plugin. Now, to present it to the world, he has to upload it. This has to be done either way, and it doesn’t really matter if he shuffles it with some FTP-client to his static-files-only-site or if he clicks “upload” in his txp-installation. Then, he has to add “Update: Wonderful new feature X added! Yadda, yadda…” to the changelog and again it makes not much difference if he has to select “awesome_thingy.changelog” and click “edit” in his FTP-client or if he has to enter it in some textarea of his txp-admin-side. The last step would be to modify that txp_plugins_list.txt-file. The poor static-files-only-site-owner would have to change the line

xyz_awesome_thingy 0.1 4.2.0-4.3.0 http://example.com/xyz_awesome_thingy_v0.1.txt http://example.com/xyz_awesome_thingy.changelog

to

xyz_awesome_thingy 0.2 4.2.0-4.3.0 http://example.com/xyz_awesome_thingy_v0.2.txt http://example.com/xyz_awesome_thingy.changelog

and wait til it gets polled or log in at .org and click “update now”. If he has included the changelog-file in an iframe on his site, he doesn’t have to update anything else.

For the cool guy with his own txp otoh the last step would consist of beeing happy that his great software will spit out the txp_plugins_list.txt-file whenever it is needed.

Yet another option

would be to let plugin-authors maintain an rss (or atom) feed and have it pushed by pubsubhubbub. That would avoid all that ugly polling stuff above and still allow static-only-sites to participate, although a bit harder than with the simple textfile method…

Gocom · 2011-01-25 18:57:14

Themroc wrote:

Nobody wants that. But it does not matter if you send a “GET” to

http://example.org/update?cmd=get_version&plugin=name and then http://example.org/update?cmd=get_dl_url&plugin=name or

http://example.org/update?cmd=get_all_data_at_once&plugin=name or

http://example.org/plugin_name_list.txt

the level of security would be the same in all cases.

Stef was probably referring to pinging that plugin authors do to announce .org that there is new version/release available. The announcing would obviously protected, possibly using oAuth – at least the query would require some sort of login. As far as the end-user checking updates goes, that’s not really an issue.

Textpattern CMS

Textpattern CMS support forum

#13 2011-01-09 12:34:36

Re: soo_plug_up: Automatic version check and update for compatible plugins

#14 2011-01-09 12:42:29

Re: soo_plug_up: Automatic version check and update for compatible plugins

#15 2011-01-24 21:51:22

Re: soo_plug_up: Automatic version check and update for compatible plugins

#16 2011-01-24 22:07:49

Re: soo_plug_up: Automatic version check and update for compatible plugins

#17 2011-01-24 23:03:57

Re: soo_plug_up: Automatic version check and update for compatible plugins

#18 2011-01-24 23:28:08

Re: soo_plug_up: Automatic version check and update for compatible plugins

#19 2011-01-24 23:30:07

Re: soo_plug_up: Automatic version check and update for compatible plugins

#20 2011-01-24 23:31:43

Re: soo_plug_up: Automatic version check and update for compatible plugins

#21 2011-01-25 18:44:46

Re: soo_plug_up: Automatic version check and update for compatible plugins

The updated “textfile”-scenario:

Yet another option

#22 2011-01-25 18:57:14

Re: soo_plug_up: Automatic version check and update for compatible plugins

Board footer