Textpattern demo site overhaul/refresh for 4.7.0 and beyond

gaekwad · 2018-03-05 22:19:32

Back in the hazy days of summer 2014, I had an idea to build a Textpattern demo site to increase the visibility of my preferred CMS to clients, you folks, and anyone else curious enough to want to poke around and see how things work. Version 1 launched in July 2014 and continued to work OK until earlier this year when I was upgrading the server and realised it was time for an overhaul.

As we prepare for the arrival of Textpattern 4.7.0-beta, I am pleased to push this refresh for you to use. Some new things have appeared, some stuff has stayed the same. The landing page URL is:

https://textpattern.co/demo

Other changes since 2014:

landing page site and demo sites are served up by Nginx over HTTP/2.0, with automatic (forced) SSL from Let’s Encrypt!
demo sites use the Textpattern 4.7.0 automated installer (instead of a known-good database being restored every time)
the VPS relocated from SFO1 (San Francisco, USA) to AMS3 (Amsterdam, Netherlands) datacenter at Digital Ocean
switched CSS framework from ZURB Foundation 5 to Tachyons (tricky to get my head around at first, but now I love it)
Flash-free username/password copy to clipboard
the Bunch O’ Bash© holding it together is somewhat leaner, and a bit more elegant

And here’s the current status, at the time of writing:

development site pulls the latest dev branch straight from GitHub
the beta site is going live shortly after Textpattern 4.7.0-beta is released (VERY SOON)
the release site will go live when Textpattern 4.7.0 is released

…and these three sites will automatically repave every three hours commencing midnight (0000) UTC. More info if you’re curious:

teardown and rebuild of a demo site takes 9 seconds under normal load, the automated Textpattern installer takes 4 seconds to do its thing
Nginx and Textpattern make a really, really good pairing – sites are wicked-fast under normal load

Future plans (because Everything Is Going To Be Alright, as we know) and stuff I need to fix:

regional servers if there’s demand (provisionally North America [Toronto, Canada] and Asia [Singapore or Bangalore, India] at the moment)
containerise the servers for the next iteration, likely with LXD, Vagrant or something Docker/Kubernetes-flavoured so plugins can be used in relative safety
figure out a way to add (safe) images and files to play with

Over to you – what else do you need from a demo site?

Last edited by gaekwad (2018-03-05 22:52:23)

Bloke · 2018-03-06 10:52:20

Looking good there, Sir. Thank you for keeping this running, it’s going to be a fantastic resource for beta testers to try out without the hassle of spinning up their own.

Apologies for the delay to 4.7.0-beta caused by a couple of edge cases in language handling. Should be fixed now: testing in progress. Standby for launch today, with luck.

philwareham · 2018-03-06 14:29:24

Lovely work! Thanks Pete.

gaekwad · 2018-03-07 12:10:37

Images are now added as part of the three-hourly rebuild.

The adding of images adds a few more seconds to the build time as they’re grabbed from GitHub every time. That site itself is scrapped and rebuilt from scratch, then the images are dropped in with some SQL and copious cp-ing. So, timeline-wise (under normal load, timings approximate):

0s – teardown demo site
1s – teardown database
2s – get Textpattern from GitHub
6s – install Textpattern
10s – drop in users SQL
12s – site open for business, cp the images (individually – yuck) into place and import the relevant SQL
15s – images ready

5x PNGs & 5x JPGs right now, with thumbnails, and there’re some SVG and WebP on hand for when support arrives.

Last edited by gaekwad (2018-03-07 12:32:24)

phiw13 · 2018-03-07 13:45:58

@ gaekwad, You should publish that script you use

gaekwad · 2018-03-07 13:56:41

phiw13 wrote #309722:

@ gaekwad, You should publish that script you use

That’s what I’m aiming for, definitely – likely after 4.7.0 lands proper. I’ll write it all up neatly.

I want to get the dev-, beta- and release-demo sites using a similar scaffold, and I’m fixing/tweaking as we approach beta. As dev- is readily available, I’m using that as the base and cribbing for beta- when it’s out, and then I can replicate that to release- when that drops.

Right now it’s (deep breath):

#!/bin/bash
### Textpattern CMS dev-demo build script

### vhost teardown
sudo rm -rf /var/www/vhosts/textpattern.co/dev-demo/live

### vhost rebuild & permissions
sudo mkdir -p /var/www/vhosts/textpattern.co/dev-demo/live
sudo chown -R www-data:www-data /var/www/vhosts/textpattern.co/dev-demo/live
sudo chmod -R 775 /var/www/vhosts/textpattern.co/dev-demo/live

### database teardown
sudo mysql --user="[SNIPPED]" --password=[SNIPPED] --host="127.0.0.1" < /var/www/vhosts/textpattern.co/dev-demo/staging/dev-demo_nuke.sql

### obtain Textpattern
sudo rm -rf /var/www/vhosts/textpattern.co/dev-demo/staging/github-dev
sudo git clone https://github.com/textpattern/textpattern.git --branch dev --depth 1 /var/www/vhosts/textpattern.co/dev-demo/staging/github-dev
sudo cp -R /var/www/vhosts/textpattern.co/dev-demo/staging/github-dev/* /var/www/vhosts/textpattern.co/dev-demo/live

### install Textpattern
sudo php /var/www/vhosts/textpattern.co/dev-demo/live/textpattern/setup/setup.php --config=/var/www/vhosts/textpattern.co/dev-demo/staging/dev-demo.json
sudo chown -R www-data:www-data /var/www/vhosts/textpattern.co/dev-demo/live
sudo chmod -R 775 /var/www/vhosts/textpattern.co/dev-demo/live
sudo rm -rf /var/www/vhosts/textpattern.co/dev-demo/live/textpattern/setup/

### add some users
sudo mysql --user="[SNIPPED]" --password=[SNIPPED] --host="127.0.0.1" --database="dev-demo" < /var/www/vhosts/textpattern.co/dev-demo/staging/add_users.sql

### add some images
sudo rm -rf /var/www/vhosts/textpattern.co/dev-demo/staging/resources
sudo svn export https://github.com/petecooper/textpattern-demo-resources/trunk/dist/images /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/blue-linear_base_2000x1055.png /var/www/vhosts/textpattern.co/dev-demo/live/images/1.png
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/green-linear_base_2000x1055.png /var/www/vhosts/textpattern.co/dev-demo/live/images/2.png
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/red-linear_base_2000x1055.png /var/www/vhosts/textpattern.co/dev-demo/live/images/3.png
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/spectrum-angle_base_2000x1125.png /var/www/vhosts/textpattern.co/dev-demo/live/images/4.png
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/spectrum-gradient-transparent_base_2000x1600.png /var/www/vhosts/textpattern.co/dev-demo/live/images/5.png
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/_thumbnails/blue-linear_thumbnail_200x105.png /var/www/vhosts/textpattern.co/dev-demo/live/images/1t.png
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/_thumbnails/green-linear_thumbnail_200x105.png /var/www/vhosts/textpattern.co/dev-demo/live/images/2t.png
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/_thumbnails/red-linear_thumbnail_200x105.png /var/www/vhosts/textpattern.co/dev-demo/live/images/3t.png
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/_thumbnails/spectrum-angle_thumbnail_200x113.png /var/www/vhosts/textpattern.co/dev-demo/live/images/4t.png
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/_thumbnails/spectrum-gradient-transparent_thumbnail_200x160.png /var/www/vhosts/textpattern.co/dev-demo/live/images/5t.png
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/field_base_2000x1333_80pc.jpg /var/www/vhosts/textpattern.co/dev-demo/live/images/6.jpg
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/fruit_base_2000x1125_80pc.jpg /var/www/vhosts/textpattern.co/dev-demo/live/images/7.jpg
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/insect_base_2000x1600_80pc.jpg /var/www/vhosts/textpattern.co/dev-demo/live/images/8.jpg
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/street_base_2000x2000_80pc.jpg /var/www/vhosts/textpattern.co/dev-demo/live/images/9.jpg
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/zebra_base_2000x750_80pc.jpg /var/www/vhosts/textpattern.co/dev-demo/live/images/10.jpg
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/_thumbnails/field_thumbnail_200x133_90pc.jpg /var/www/vhosts/textpattern.co/dev-demo/live/images/6t.jpg
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/_thumbnails/fruit_thumbnail_200x113_90pc.jpg /var/www/vhosts/textpattern.co/dev-demo/live/images/7t.jpg
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/_thumbnails/insect_thumbnail_200x160_90pc.jpg /var/www/vhosts/textpattern.co/dev-demo/live/images/8t.jpg
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/_thumbnails/street_thumbnail_200x200_90pc.jpg /var/www/vhosts/textpattern.co/dev-demo/live/images/9t.jpg
sudo cp -f /var/www/vhosts/textpattern.co/dev-demo/staging/resources/images/_thumbnails/zebra_thumbnail_200x75_90pc.jpg /var/www/vhosts/textpattern.co/dev-demo/live/images/10t.jpg
sudo chown -R www-data:www-data /var/www/vhosts/textpattern.co/dev-demo/live
sudo chmod -R 775 /var/www/vhosts/textpattern.co/dev-demo/live
sudo mysql --user="[SNIPPED]" --password=[SNIPPED] --host="127.0.0.1" --database="dev-demo" < /var/www/vhosts/textpattern.co/dev-demo/staging/add_images.sql

### aftercare

### done
exit

add_users is a loooong list of SQL inserts:

INSERT INTO `txp_users` (`user_id`, `name`, `pass`, `RealName`, `email`, `privs`, `last_access`, `nonce`)
VALUES
  (1,'managing-editor1',PASSWORD('managing-editor1'),'Managing Editor #1','managing-editor1@example.com',2,NULL,NULL),
  (2,'managing-editor2',PASSWORD('managing-editor2'),'Managing Editor #2','managing-editor2@example.com',2,NULL,NULL),
  (3,'managing-editor3',PASSWORD('managing-editor3'),'Managing Editor #3','managing-editor3@example.com',2,NULL,NULL), […]

I splat the database with this:

SET FOREIGN_KEY_CHECKS = 0;
DROP DATABASE IF EXISTS `[SNIPPED]`;
CREATE DATABASE IF NOT EXISTS `[SNIPPED]`;
SET FOREIGN_KEY_CHECKS = 1;

…and my JSON is quite barebones:

{
    "site": {
        "language_code": "en",
        "public_url": "dev-demo.textpattern.co",
        "admin_url": "",
        "admin_theme": "",
        "public_theme": "four-point-seven",
        "content_directory": "",
        "cookie_domain": ""
    },
    "database": {
        "user": "[SNIPPED]",
        "password": "[SNIPPED]",
        "host": "127.0.0.1",
        "db_name": "[SNIPPED]",
        "charset": "",
        "table_prefix": "",
        "client_flags": ""
    },
    "user": {
        "full_name": "Textpattern Demo Robot",
        "email": "domo-arigato-mr-roboto@example.com",
        "login_name": "textpattern-robot",
        "password": "[SNIPPED]"
    }
}

It’s not optimised yet, I’m still picking through it and fiddling until I’m happy with it.

phiw13 · 2018-03-07 22:28:04

gaekwad wrote #309725:

That’s what I’m aiming for, definitely – likely after 4.7.0 lands proper. I’ll write it all up neatly. (…)

Wow! Impressive so far. Looking forward to see the final version (is there ever a “final” version?) and some write up.

I and kinda interested eventually, to be able to quickly set up something for development locally. Thanks for sharing.

bici · 2018-03-07 23:12:55

i too would like to use this script for quick development of a test site

gaekwad · 2018-03-07 23:56:00

…and Textpattern 4.7.0-beta is go. Links to public- and admin-side are here:

https://textpattern.co/demo

(It’s late, I need sleep, don’t break it while I’m gone.)

Last edited by gaekwad (2018-03-08 00:20:34)

gaekwad · 2018-05-04 13:18:34

~~Bumping this thread with some important legal info.~~

When GDPR kicks in later this month, things may change on the demo site – if the current Textpattern beta and dev version contain any kind of visitor/commenter logging with identification (e.g. IP address etc), the sites will be disabled with an explanatory note.

As the person doing the hosting, I cannot in good faith expect everyone using the site to abide by GDPR by leaving logging off, leaving disclaimers etc in place, removing any GDPR-enforcing plugins and so on.

~~The dev team are working diligently on GDPR compliance here and if that goes smoothly before the commencement of GDPR, we’ll be all systems go on the demo site.~~

~~Any questions, please ask and I’ll do my best to answer them (I am not a lawyer).~~

All resolved, no longer relevant.

Last edited by gaekwad (2018-05-10 13:21:07)

Bloke · 2018-05-04 14:01:43

gaekwad wrote #311617:

As the person doing the hosting, I cannot in good faith expect everyone using the site to abide by GDPR by leaving logging off, leaving disclaimers etc in place, removing any GDPR-enforcing plugins and so on.

We’ll have it sorted by then.

In the meantime, I get the sentiment, but I think you’re covered anyway since everything’s trashed every few hours so there’s no long-term storage. Barely any short-term storage.

Anyone inviting users to ‘click here and win a polar bear’ pointing to a demo site with the express purpose of capturing that info for direct marketing with opt-in is going to have a hard time getting anyone to part with their details in time, let alone there’ll be no record of the transaction three hours later.

gaekwad · 2018-05-04 14:03:49

Bloke wrote #311621:

We’ll have it sorted by then.

Thank you. I have no doubt. Don’t take this (potential) action as sabre rattling or any kind of mutiny, it’s just me covering my behind from a biz/org point of view and a worst-case scenario.

Textpattern CMS

Textpattern CMS support forum

#1 2018-03-05 22:19:32