Go to main content

Textpattern CMS support forum

You are not logged in. Register | Login | Help

#11 2015-05-18 13:47:34

jakob
Admin
From: Germany
Registered: 2005-01-20
Posts: 3,897
Website

Re: TXP git repo best practices?

Phil wrote:

Jukka gave a good workflow for this

Can we see this anywhere? I thought Jukka would have found answers to a lot of these questions.

Phil wrote:

And then this is what I use as a .gitignore in the public (built) directory:

I didn’t get that bit. Is the public directory then also your remote repo? Is there a particular advantage in excluding files and images on the remote / public side as opposed to in the local repo? Do you manually add the .gitignore file to the remote repo via s/ftp?

The above deploy situation deploys from a local/remote repo to a public directory, so the remote repo and deploy directories are separate. The exclude pattern prevents files/images/cache files/google webmaster files etc. from being deleted in a sync.

Sacripant wrote:

Actually I versioning .sql dump, .sql it’s text file and textpattern database is small.

A diff comparison of two sql dumps does work if your changes aren’t numerous and your public site is not constantly being updated. You can make this a bit easier by doing an sql export from the live and staging/local database with modified settings. In phpmyadmin (or via the mysqldump command), you can:

  1. Make it export one INSERT statement per db record (see here for an example), and omit some of the verbose table infos and CREATE infos. Make the settings the same on both database exports.
  2. Omit certain unneeded tables from the export, e.g. aks_cache, txp_css, txp_form, txp_lang, txp_log, txp_page, txp_discuss_nonce … (adjust as needed per your database – I’ve excluded txp_css, _page, _form here because those are all updated by rah_flat, but if you use the txp css panel, you’ll need to include it.

You can then do a diff with your favourite diff tool and the entries that have changed will be highlighted. You’ll also have INSERT statements that you can copy (more or less) wholesale back into phpmyadmin or Sequel Pro, or whatever you use to administer your databases.

Sacripant wrote:

Versioning database configurations is not easy; I see rah_flat (not tested yet) can manage site preferencies, templates, sections with flat files, it’s very interesting. But for a complete versionning site configuration, what solution about plugins (when install new plugin) …?

You can get part way there with a combination of rah_flat and a plugin directory with ied_plugin_composer.

Forms / pages / css: rah_flat is easily set up and is more robust and reliable than cnk_versioning. See the instructions on Jukka’s github repo (download from releases). You set the directory with your templates in the prefs and create subdirectories for /pages, /forms, and /css. If you leave one out, it won’t be synced.

Beware that rah_flat syncs only in one direction: from files to db. It doesn’t export them first, so make a db backup or copy out the forms you need into files. It will replace your forms wholesale! It follows the same naming convention as cnk_versioning, e.g. /forms/form-name.(misc/article/link/image).txp and /pages/page-name.txp.

In debugging / testing mode it imports the templates on every page load, so don’t be worried, when you’re db-query count is suddenly very high. Once you switch to live, that only happens if you specifically call the public hook url (which you can define to be non-guessable). Despite the regular importing, I haven’t noticed a performance hit when in debug/live mode.

There’s only one real conflict I’ve noticed and that’s with adi_variables. The pane is no longer accessible and you can only continue using adi_variables by manually editing the adi_variables.misc.txp file. Jukka didn’t entertain the idea of having an exclude list, arguing that adi_variables should really employ a different method… If you switch off rah_flat when in live mode and allow your users to edit adi_variables, you must remember to make a note of any changes before switching to debug mode…

Plugins: Set a plugin directory in the txp prefs and use ied_plugin_composer to make php files of your plugins that you can then drop into the plugin directory. You can then sync or version these files. There are occasionally disadvantages with using this method (originally for developing plugins) because copying the files to the plugin directory, doesn’t necessarily trigger the install routines. ied_plugin_composer does allow you to trigger install / lifecycle events but there are times when a plugin doesn’t play well. In those cases I install using the regular method, or I install and activate regularly, then switch to the php file.

Jukka has a separate plugin for this (rah_blobin) which is supposed to overcome some of these limitations, but also has a different plugin structure. I have still to investigate this.


TXP Builders – finely-crafted code, design and txp

Online

#12 2015-05-18 14:09:00

philwareham
Core designer
From: Haslemere, Surrey, UK
Registered: 2009-06-11
Posts: 3,343
Website

Re: TXP git repo best practices?

jakob wrote #290872:

Can we see this anywhere? I thought Jukka would have found answers to a lot of these questions.

I didn’t get that bit. Is the public directory then also your remote repo? Is there a particular advantage in excluding files and images on the remote / public side as opposed to in the local repo? Do you manually add the .gitignore file to the remote repo via s/ftp?

public is the build directory, regardless of whether it is on local or a remote server. It’s where all your files are built to by the Grunt script. Effectively that would be the root folder of your website. You will need to ignore certain files in that as you won’t want them to appear in your repository. Things like Textpattern itself (which you can install via Composer), there is no need to include that in your repo, and content images/files.

Don’t confuse this with a backup snapshot of your site – they are not the same thing. The repo should contain only the development files themselves.

Jukka made the workflow in the Textpattern.com repo (well, the TXP Magazine and forum repos actually which I copied over to .com repo). Pick it apart and you should be able to understand how it works.

Beware that rah_flat syncs only in one direction: from files to db. It doesn’t export them first, so make a db backup or copy out the forms you need into files. It will replace your forms wholesale! It follows the same naming convention as cnk_versioning, e.g. /forms/form-name.(misc/article/link/image).txp and /pages/page-name.txp.

Since you are version controlling anyway, you’ll need flat files of your templates. But yes, copy all your forms/pages out of Textpattern into actual files manually if necessary when first setting up, in the directory structure expected by rah_flat.

Offline

#13 2015-05-18 15:18:55

Destry
Member
From: Haut-Rhin
Registered: 2004-08-04
Posts: 4,415
Website

Re: TXP git repo best practices?

Realizing now that this discussion is about production workflow, for people who build new sites on an ongoing basis. That’s not really what I’m after with CSF, of course, so that probably didn’t make any sense. Sorry for the distraction.

/ slips out /

Offline

#14 2015-05-18 16:13:46

jakob
Admin
From: Germany
Registered: 2005-01-20
Posts: 3,897
Website

Re: TXP git repo best practices?

public is the build directory, regardless of whether it is on local or a remote server.

Ah, thanks to your pointer I now understand. Your templates, assets etc. are all in the /src directory and the actual site are in the parallel /build directory. That now makes perfect sense to me.

Since you are version controlling anyway, you’ll need flat files of your templates. But yes, copy all your forms/pages out of Textpattern into actual files manually if necessary when first setting up, in the directory structure expected by rah_flat.

Agreed. If you are making your own forms from scratch or have a ready-built startup configuration of templates, it won’t matter, but it may surprise people coming from cnk_versioning, hence the warning. BTW, are there not some default forms that are advisable to keep, because they are used as default when no user-defined form is specified?


TXP Builders – finely-crafted code, design and txp

Online

#15 2015-05-18 16:18:54

jakob
Admin
From: Germany
Registered: 2005-01-20
Posts: 3,897
Website

Re: TXP git repo best practices?

One more thing on the DB sync front. I read that MySQL Workbench (available for multiple platforms) is supposed to have a db compare module.


TXP Builders – finely-crafted code, design and txp

Online

#16 2015-05-18 22:16:51

sacripant
Plugin Author
From: Rhône — France
Registered: 2008-06-01
Posts: 478
Website

Re: TXP git repo best practices?

Thank you all for these exchanges, and Jacob for his answers.
If I can work on a new project textpattern, I’ll improve a lot of things.

Another question about rah_flat : with this plugin, Is it possible to organize flat files into subfolders ? is he searched recursively files into “form” folder per exemple to make these imports ? This would allow me to organize more clearly the many forms :

Forms/
  - site-stucture/
    - site-header.misc.txp
    - site-footer.misc.txp
    - main-nav.misc.txp
  - Components/
    - btn.misc.txp
    - accordeon.article.txp
  - external-output/
    - rah_eo_custom-atom.misc.txp
    - rah_eo_some-json.misc.txp

Offline

#17 2015-05-18 22:58:28

jakob
Admin
From: Germany
Registered: 2005-01-20
Posts: 3,897
Website

Re: TXP git repo best practices?

Is it possible to organize flat files into subfolders?

To be honest, I don’t know.

But you could achieve something similar by adopting a naming convention for your forms, e.g.:

forms/
- layout_site-header.misc.txp
- layout_site-footer.misc.txp
- components_main-nav.misc.txp
- components_hero.misc.txp
- components_accordeon.article.txp
- objects_btn.misc.txp
- rah_eo_custom-atom.misc.txp
- rah_eo_some-json.misc.txp

… or however you like to organise them. There are only a few cases (like rah_eo_ and smd_tabber) that have to follow a naming convention, but it doesn’t break the idea.


TXP Builders – finely-crafted code, design and txp

Online

#18 2015-11-09 14:32:47

maniqui
Moderator
From: Buenos Aires, Argentina
Registered: 2004-10-10
Posts: 3,070
Website

Re: TXP git repo best practices?

If you are considering migrating to flat (text) files for managing your pages, forms & styles, you might find these little python scripts helpful. Basically, they will help you with the initial exporting of pages, forms & styles to flat files. Really helpful if you have many forms & pages and want to avoid manual work.


La música ideas portará y siempre continuará

TXP Builders – finely-crafted code, design and txp

Offline

#19 2015-11-10 09:08:03

jakob
Admin
From: Germany
Registered: 2005-01-20
Posts: 3,897
Website

Re: TXP git repo best practices?

Thanks Julián. That’s really useful and fills a gap in rah_flat. I saw your post and tried it a week or two ago but kept getting python errors (nothing to do with your script – probably my python installation). I must try it again with more diligence.

I did think that this is something that would have great potential as a separate txp plugin. It’s really a one-time operation, so it would be fine as a standalone plugin. Potentially that could be extended for other uses, e.g. as a data output to flat-file exporter for article data for use in other contexts – e.g. for porting to another system in haml, xml, json or csv structure, for structured output of data for inclusion in print merge cases, or for giving to site owners as a content inventory for editing. I’ve done this once or twice before by creating a custom page template or using rah_external_output and then saved the page source as text. That’s fine when you need one long stream but more laborious when you need lots of small items – either you call up and save each page one by one or you save the long stream and then have to break it down manually into chunks. For small sites, that’s okay if a little tedious, for larger sites, it’s a major bore … or even impractical.

I’ve veered off topic here, but I’d love to hear if others have found good solutions for this – in a new thread perhaps…


TXP Builders – finely-crafted code, design and txp

Online

Board footer

Powered by FluxBB