Textpattern CMS support forum

You are not logged in. Register | Login | Help

#91 2018-03-17 16:13:11

Bloke
Developer
From: Leeds, UK
Registered: 2006-01-29
Posts: 8,959
Website

Re: Feedback to: Textpattern CMS 4.7.0 beta released

etc wrote #310068:

I meant if needed while developing, fetch themes directly from fs, without updating db.

Ah then we were talking at cross-purposes, sorry. I’m talking about the webhook that Phil and others want us to explore now. Fetching from disk directly, fine if it’s an easyish win.

CodeWalker wrote #310069:

I would strongly consider ditching storing the template code in the database all together, and going 100% flat files as far as a template is concerned.

We’ve been back and forth on this for nearly a decade. While all our competitors pretty much rely wholesale on the file system for serving templates, we’ve stuck with the database. And we outperform most of them – sometimes by several orders of magnitude. Architecture, or coincidence? Maybe a bit of both. Difficult to say. Apples. Oranges.

Ultimately, I’m with Oleg on the access argument. Storing, searching and indexing a database is more flexible (even on crappy relational DB technology, virtually unchanged since it was conceived to work with the disk environments of the 1970s) than trying to do the same thing across a file system with current tools. Yes, file systems are slowly catching up at storing useful metadata but we have little control over what they store. In a DB we choose the metadata and how to index it.

My original stance has always been that the cost of moving the disk head when you’re already serving a) core files, and b) database caches for stuff in columns > VARCHAR is immense. Adding templates to that is just more of a bandwidth bottleneck. The latter is slightly moot in some respects because all our template columns are TEXT or higher so are likely to incur a disk hit anyway, unless the database content is capable of being cached after first load by your chosen DB engine (and we do caching internally anyway so it only really affects first load of each asset per page).

SSDs tip the playing field a bit. Random access NAND flash is fast enough, albeit still at the expense of having to tear down entire blocks on rewrites or for wear levelling algorithms, to make it worth considering. We’re still limited by the disk buffer, plus how many concurrent operations the controller permits, and the raw performance depends on which memory architecture the manufacturer chooses. But on the whole, for reading – which is where it matters when serving a website – it’s far better than spinning platters.

I’m all for a hybrid approach if we can manage it sensibly. Index and content in the DB for searchability; content optionally served from disk if someone wishes.


The smd plugin menagerie — for when you need one more gribble of power from Textpattern. Bleeding-edge code available on GitHub.

Txp Builders – finely-crafted code, design and Txp

Offline

#92 2018-03-17 16:37:31

philwareham
Core designer
From: Farnham, Surrey, UK
Registered: 2009-06-11
Posts: 3,240
Website

Re: Feedback to: Textpattern CMS 4.7.0 beta released

I’m personally fine with templates stored in the database. It’s easily cacheable and allows for some pretty dynamic code.

Flat files for me is solely so we can edit, build and version control using modern workflow and tools. The web hook URLs if available can be built into any build tool script (Webpack, Grunt, Gulp, NPM, whatever) so effectively that can mimic browser reload anyway.

Offline

#93 2018-03-17 17:30:21

CodeWalker
Member
From: Hampshire, UK
Registered: 2010-01-08
Posts: 110
Website

Re: Feedback to: Textpattern CMS 4.7.0 beta released

Bloke wrote #310082:

Ah then we were talking at cross-purposes, sorry. I’m talking about the webhook that Phil and others want us to explore now. Fetching from disk directly, fine if it’s an easyish win.

We’ve been back and forth on this for nearly a decade. While all our competitors pretty much rely wholesale on the file system for serving templates, we’ve stuck with the database. And we outperform most of them – sometimes by several orders of magnitude. Architecture, or coincidence? Maybe a bit of both. Difficult to say. Apples. Oranges.

Ultimately, I’m with Oleg on the access argument. Storing, searching and indexing a database is more flexible (even on crappy relational DB technology, virtually unchanged since it was conceived to work with the disk environments of the 1970s) than trying to do the same thing across a file system with current tools. Yes, file systems are slowly catching up at storing useful metadata but we have little control over what they store. In a DB we choose the metadata and how to index it.

My original stance has always been that the cost of moving the disk head when you’re already serving a) core files, and b) database caches for stuff in columns > VARCHAR is immense. Adding templates to that is just more of a bandwidth bottleneck. The latter is slightly moot in some respects because all our template columns are TEXT or higher so are likely to incur a disk hit anyway, unless the database content is capable of being cached after first load by your chosen DB engine (and we do caching internally anyway so it only really affects first load of each asset per page).

SSDs tip the playing field a bit. Random access NAND flash is fast enough, albeit still at the expense of having to tear down entire blocks on rewrites or for wear levelling algorithms, to make it worth considering. We’re still limited by the disk buffer, plus how many concurrent operations the controller permits, and the raw performance depends on which memory architecture the manufacturer chooses. But on the whole, for reading – which is where it matters when serving a website – it’s far better than spinning platters.

I’m all for a hybrid approach if we can manage it sensibly. Index and content in the DB for searchability; content optionally served from disk if someone wishes.

Gracious. A very thorough answer :) Trust me to stoke up a decade old argument. Again. I totally get both arguments, but as a front-ender, I would just love a speedy way to develop with TXP without having flick back and forth into the backend to press a button when I change a template.

philwareham wrote #310083:

I’m personally fine with templates stored in the database. It’s easily cacheable and allows for some pretty dynamic code.

Flat files for me is solely so we can edit, build and version control using modern workflow and tools. The web hook URLs if available can be built into any build tool script (Webpack, Grunt, Gulp, NPM, whatever) so effectively that can mimic browser reload anyway.

Thats exactly what I want, and if that means a button to suck it up into the database for production, then so be it.

I’m simply aiming to speed up local development, using tools such as you mentioned (Npm + Browsersync). If i press save on a template file thats open in Atom, it should reload in the browser without intervention.

Offline

#94 2018-03-17 18:00:57

michaelkpate
Moderator
From: Avon Park, FL
Registered: 2004-02-24
Posts: 1,220
Website

Re: Feedback to: Textpattern CMS 4.7.0 beta released

CodeWalker wrote #310085:

Trust me to stoke up a decade old argument.

You aren’t the first. You won’t be the last. :)

It would be nice to be able to edit a template live in Atom or Sublime or Notepad++ without copying and pasting, though. Maybe someday.

Offline

#95 2018-03-17 18:15:08

colak
Admin
From: Cyprus
Registered: 2004-11-20
Posts: 7,531
Website

Re: Feedback to: Textpattern CMS 4.7.0 beta released

CodeWalker wrote #310069:

I would strongly consider ditching storing the template code in the database all together, and going 100% flat files as far as a template is concerned.

Please don’t:). Some of us love to work in computers without an ftp client. I actually prefer to work on the browser. What I believe we are in need for, is for the functionalities of rvm_css and spf_js to become part of the core. That is, working on the browser and on save, the files are also stored as files.


Yiannis
——————————
neme.org | hblack.net | LABS | State Machines | Respbublika! | NeMe @ github

Online

#96 2018-03-17 18:23:43

CodeWalker
Member
From: Hampshire, UK
Registered: 2010-01-08
Posts: 110
Website

Re: Feedback to: Textpattern CMS 4.7.0 beta released

colak wrote #310087:

Please don’t:). Some of us love to work in computers without an ftp client. I actually prefer to work on the browser. What we are though in need for, I believe, is for the functionalities of rvm_css and spf_js to become part of the core. That is, working on the browser and on save, the files are also stored as files.

Oh man, I’m really causing trouble today :) Sure, working out of the browser suits some people, it’s just not for me (unless im writting content). I think it should be possible to choose either way without being forced in either direction, I was probably a bit brutal when i suggested ditching it all together, I know the feature has some love out there.

Last edited by CodeWalker (2018-03-17 18:24:17)

Offline

#97 2018-03-17 20:27:55

etc
Developer
Registered: 2010-11-11
Posts: 3,527
Website

Re: Feedback to: Textpattern CMS 4.7.0 beta released

CodeWalker wrote #310088:

I think it should be possible to choose either way without being forced in either direction

Just create and enable a public side plugin like

register_callback('etc_flat', 'page.fetch');
register_callback('etc_flat', 'form.fetch');

function etc_flat($event, $step, $rs) {
    global $pretext;
    extract($rs);
    $page = false;

    if ($event == 'page.fetch') {
        $page = @file_get_contents($pretext['path_to_site'].DS.'themes'.DS.$theme.DS.'pages'.DS.$name.'.txp');
    } else {
        foreach (glob($pretext['path_to_site'].DS.'themes'.DS.$skin.DS.'forms'.DS.'*') as $dir) {
            if ($page = @file_get_contents($dir.DS.$name.'.txp')) break;
        }
    }

    return $page;
}

and your pages and forms will (should) be fetched from flat files, without any db sync. It can be easily amended to run only for logged users, only in debug mode etc.

Offline

#98 2018-03-17 22:17:30

michaelkpate
Moderator
From: Avon Park, FL
Registered: 2004-02-24
Posts: 1,220
Website

Re: Feedback to: Textpattern CMS 4.7.0 beta released

This whole discussion is taking me back to 2003 because Movable Type (which some of us who have been around a very long time used before Dean released Textpattern to the public) has this very feature:

Optionally link this template to a file on your filesystem. This enables you to edit the templates in a text editor or other tool of your choice instead of through the Movable Type template editing screen. If you select this option and link the template to a file, Movable Type will keep its internal copy synchronized with the external file. – Managing your Blog’s Templates

Offline

#99 2018-03-18 05:51:27

bici
Member
From: vancouver
Registered: 2004-02-24
Posts: 1,541
Website

Re: Feedback to: Textpattern CMS 4.7.0 beta released

colak wrote #310087:

Please don’t:). Some of us love to work in computers without an ftp client. I actually prefer to work on the browser. What I believe we are in need for, is for the functionalities of rvm_css and spf_js to become part of the core. That is, working on the browser and on save, the files are also stored as files.

No FTP required:
I use Mountee to mount my ExperssionEngine Templates on my desktop and edit them as files in my favourite editing app. no FTP required. And better than working in the Browser. Multiple undos as an added bonus. And easy duplicate them in case you wish to roll back to zero.


…. texted postive

Offline

#100 2018-03-18 10:46:37

Bloke
Developer
From: Leeds, UK
Registered: 2006-01-29
Posts: 8,959
Website

Re: Feedback to: Textpattern CMS 4.7.0 beta released

bici wrote #310099:

Multiple undos as an added bonus. And easy duplicate them in case you wish to roll back to zero.

Not wishing to dismiss your arguments for using an external editor – they’re all valid, plus you get versioning for free. Just to mention that in 4.7.0 we have the CTRL-S shortcut across the board. Saving templates and stylesheets also performs an AJAX save (permitting undo as long as you haven’t closed the tab) and the same intelligent template duplication feature from 4.6.x that allows you to rename/change a template and then click Duplicate to have it copied and saved in one move.


The smd plugin menagerie — for when you need one more gribble of power from Textpattern. Bleeding-edge code available on GitHub.

Txp Builders – finely-crafted code, design and Txp

Offline

Board footer

Powered by FluxBB