Go to main content

Textpattern CMS support forum

You are not logged in. Register | Login | Help

#1 2020-06-29 12:42:01

gaekwad
Server grease monkey
From: People's Republic of Cornwall
Registered: 2005-11-19
Posts: 4,134
GitHub

Textpattern web servers - the next 18 months

On July 1st 2020 I will be contacting our hosting providers DigitalOcean with a renewal request. We’ve managed to make full use of the donated hosting credits, and DO as a platform has been superb to work with. I’m confident our application for a hosting renewal will be worthy.

Some of the web ops / dev ops plans for the next 12-18 months are outlined below. There is a call to action at the end. In no particular order of priority or importance:

We will upgrade our infrastructure to the latest Ubuntu long-term support (LTS) release. Ubuntu has its advantages and disadvantages, but with some time and attention it’s a well-supported foundation for our servers with very good support coverage from third party vendors. The most recent LTS release from April 2020 is actively supported for many years and in A/B testing has shown itself more performant for us than the previous release.
Expected timeframe: Q3 or Q4 2020.

The forum will have its own virtual server, with a new IP address and hostname. The forum web address itself will not change, just the computer it’s hosted on. I mention the IP address because any email alerts you receive will be sent from a different origin. If I can iron out some snags, these emails will be tagged with various email security features to confirm validity (i.e. DKIM, DANE, etc), though email is a tricky beast and this needs time and patience. I will give as much notice as I can before the forum IP switch, with relevant details for interested parties. Our forum is the most active of our sites, for both human-led visits and automated hits from RSS feeds etc. The new server will be optimised for the forum.
Expected timeframe: Q3 or Q4 2020.

We will be self-hosting application performance management (APM) and reporting tools to have data on how Textpattern performs and how the underlying infrastructure is affected. This data will be useful for the development team to better understand the content management system that we all play a part in. I am confident we can run our own Sentry instance and a self-hosted status page for the various sites we offer, so we can better communicate what’s going on to the community.
Expected timeframe: Q4 2020.

We will also be self-hosting a community chat server and self-hosted video conferencing for community meets. The first half of 2020 has been…eventful, to say the least, and now more than ever it’s vital that we have a way to connect the humans behind the words you see here. The broad plan is to make the chat server (likely Matrix Synapse) available for self-registration and 24/7 use, while the video conferencing server (likely Jitsi) will be made available for scheduled events. The resource requirements for an active, self-hosted Jitsi are orders of magnitude more than Synapse, and it’s vital we don’t overrun our allocated resources to the detriment of other services (.com site, forum, etc).
Expected timeframe: Q4 2020 or Q1 2021.

If all goes to plan, we should also be self-hosting Textpattern documentation. Right now, the files are on GitHub Sites and we’re limited with what we can do with that. Hosting our own docs site has been on my to-do list for a while, and I hope to investigate it more thoroughly as the daylight hours get shorter in the latter part of 2020.
Expected timeframe: Q4 2020 or Q1 2021.

And here’s the call to action:

What else do we need? What have I missed from this list?

Let me know in the comments, along with any questions you have.

Thanks for reading.

Edit: to clarify, I should say that I will be explaining some of the things we’re intending to accomplish to DigitalOcean, but not everything – there’s no inherent time limit on your feedback here, so don’t rush before July 1st.

Last edited by gaekwad (2020-06-30 15:54:31)

Offline

#2 2020-06-29 15:46:45

colak
Admin
From: Cyprus
Registered: 2004-11-20
Posts: 9,007
Website GitHub Mastodon Twitter

Re: Textpattern web servers - the next 18 months

Hi Pete, there is also the issue of the .org, txp.tips and txpmag.com sites which I understand that they are currently all under refurbishment. Are those going to be hosted with DO? Are they part of the timeline? Also, we have to start thinking seriously about the hosting of themes again.

If DO does not allow us more domains, we may start thinking giving up some of them and creating subdomains using txp’s multi-site feature, or even think of deeper url structures and a truly centralised site!

I also think that there is a semantic issue with our main site having a .com extension and i believe that a .org one would be more appropriate. As such the .com content and design could be moved to the .org domain and differentiate the content using deeper url structures.

For years I have been running 3 txp sites for neme and I have managed to reduce it to one1, about 3 years ago. This has actually less than halved my work.

I understand that your post was mostly about the infrastructure, which is indeed very important, but in the end, it is about how we can serve the existing community and as importantly, how to expand it. Centralising may be a methodology.

Re Jitsi, I agree with all you said there, although it might be a good idea to have scheduled monthly support meetings with people who may need it. Say for 90minutes every first Saturday of the month at a time we think that will be convenient for most time zones. These meetings could give a personal touch with the support we are trying to give here… (just an idea:).

1 newsletter pending migration.


Yiannis
——————————
NeMe | hblack.art | EMAP | A Sea change | Toolkit of Care
I do my best editing after I click on the submit button.

Offline

#3 2020-06-29 17:09:07

jakob
Admin
From: Germany
Registered: 2005-01-20
Posts: 4,578
Website

Re: Textpattern web servers - the next 18 months

Looks very thorough! Thanks Pete for managing that all so brilliantly. No new ideas. Maybe the rpc server can be decommissioned at some point, with simply a json file or similar for the new version notification?

I love the idea of a ‘safe/own space‘ for meetings / chat / video calls. But … how much will we use it? And do we risk watering down our presence/availability? I did like speaking with you all a few weeks back, and I do like being able to message one or the other of you from time to time with a question, so the idea of it I like very much, but the forum isn’t hugely busy (compared with other fora) and if the frequent posters here on the forum shift to another communications channel, it will become ever less frequented. I already miss stuff that’s been posted as GitHub issues. We once had Facebook and icg chat a while ago and I was on neither, so they were closed to me. So my only concern on this front is that things may get spread across too many channels.

colak wrote #324092:

There is also the issue of the .org, txp.tips and txpmag.com sites which I understand that they are currently all under refurbishment. Are those going to be hosted with DO? Are they part of the timeline?

txptips.com (not tax-tips as autocorrect tried to make it) is on my server. I just have to finish that off (sorry^1000).

The textpattern.org site won’t be needed any more once the plugin directory is up but it is on Textpattern’s servers.

No idea about txpmag but I think that’s also a textpattern-owned domain.

I also think that there is a semantic issue with our main site having a .com extension and i believe that a .org one would be more appropriate.

Maybe the .org can then serve “team/organisational” aspects thereafter but personally I’d be for not rocking the main domain boat any more now that Pete so diligently transferred everything back from the .io.

Also, we have to start thinking seriously about the hosting of themes again.

Maybe once Phil has a good working system for the plugin directory (looks like it’s getting closer thanks to Phil and Bloke), the same principle can be used for the themes, and they are kept up to date by the respective theme author in their own space.


TXP Builders – finely-crafted code, design and txp

Offline

#4 2020-06-29 17:21:29

gaekwad
Server grease monkey
From: People's Republic of Cornwall
Registered: 2005-11-19
Posts: 4,134
GitHub

Re: Textpattern web servers - the next 18 months

colak wrote #324092:

there is also the issue of the .org, txp.tips and txpmag.com sites which I understand that they are currently all under refurbishment. Are those going to be hosted with DO? Are they part of the timeline? Also, we have to start thinking seriously about the hosting of themes again.

The details of each site is really outside the scope of this post, but I know there is work going on with some things. As jakob mentions above, the tips site is on his server currently. There is going to be capacity for it to be hosted on our servers, but I don’t know specifics of what’s happening with that domain specifically – nor would I want to talk on behalf of jakob.

If DO does not allow us more domains, we may start thinking giving up some of them and creating subdomains using txp’s multi-site feature, or even think of deeper url structures and a truly centralised site!

We are unlimited on the number of domains and subdomains. It’s essentially DIY hosting so we don’t use a control panel to manage the servers, it’s all scripts and an empty canvas.

I understand that your post was mostly about the infrastructure, which is indeed very important, but in the end, it is about how we can serve the existing community and as importantly, how to expand it. Centralising may be a methodology.

That’s beyond my pay grade. The .com site is well established and has a good search engine presence, so I don’t see that going away.

Re Jitsi, I agree with all you said there, although it might be a good idea to have scheduled monthly support meetings with people who may need it. Say for 90minutes every first Saturday of the month at a time we think that will be convenient for most time zones. These meetings could give a personal touch with the support we are trying to give here… (just an idea:).

We’ll know more when we know how Jitsi works (if it’s even for us, we’ll have to find out), but I’ll be guided by the community on it. If there’s a need, we’ll figure something out!

Offline

#5 2020-06-29 17:25:42

gaekwad
Server grease monkey
From: People's Republic of Cornwall
Registered: 2005-11-19
Posts: 4,134
GitHub

Re: Textpattern web servers - the next 18 months

jakob wrote #324096:

Maybe the rpc server can be decommissioned at some point, with simply a json file or similar for the new version notification?

Site-specific stuff is outside this post scope, but yes – I can see that being a flat file setup or even just a big Nginx rewrite block. I haven’t looked into the logs for RPC for a while, but it’s on another list.

I love the idea of a ‘safe/own space‘ for meetings / chat / video calls. But … how much will we use it? And do we risk watering down our presence/availability? I did like speaking with you all a few weeks back, and I do like being able to message one or the other of you from time to time with a question, so the idea of it I like very much, but the forum isn’t hugely busy (compared with other fora) and if the frequent posters here on the forum shift to another communications channel, it will become ever less frequented. I already miss stuff that’s been posted as GitHub issues. We once had Facebook and icg chat a while ago and I was on neither, so they were closed to me. So my only concern on this front is that things may get spread across too many channels.

I share your concerns, all valid. I remember the Slack channel we had, and that fizzled out…so perhaps that’s one for the back burner for now.

No idea about txpmag but I think that’s also a textpattern-owned domain.

Owned and hosted.

Offline

#6 2020-06-30 11:05:35

philwareham
Core designer
From: Haslemere, Surrey, UK
Registered: 2009-06-11
Posts: 3,564
Website GitHub Mastodon

Re: Textpattern web servers - the next 18 months

colak wrote #324092:

Hi Pete, there is also the issue of the .org, txp.tips and txpmag.com sites which I understand that they are currently all under refurbishment. Are those going to be hosted with DO? Are they part of the timeline?

The plugins website we are building right now, I’m hoping to get an final alpha build live in 2 to 3 weeks time, so we can start allowing user testing of the functionality within. This will be at plugins.textpattern.com going forward – the Textpattern.org plugins site will eventually be shuttered and archived, and the domain redirected (we’ll keep hold of the domain name though).

Also, we have to start thinking seriously about the hosting of themes again.

I think as Jakob mentioned, a themes site will follow on after the plugins site is live and we are happy it is working as expected. A lot of the site code, methods and processes used in the plugins site are also to be reused on the themes site build (this is actually much simpler to build than the plugins site, which itself has evolved into something quite powerful and exciting over the last months).

I also think that there is a semantic issue with our main site having a .com extension and i believe that a .org one would be more appropriate. As such the .com content and design could be moved to the .org domain and differentiate the content using deeper url structures.

The Textpattern .com suffix is here to stay and all main parts of the Textpattern site will be housed under it. Semantically you could rightly argue .org (organisation/noncommercial) is correct terminology for us instead of .com (commercial), but we have around 18 years of domain history and link juice1 which would be folly to throw away now.

1 Apart from our brief dalliance with the .io suffix, which was an emergency measure in a period when we didn’t directly control the domain names.

Offline

#7 2020-06-30 13:26:49

gaekwad
Server grease monkey
From: People's Republic of Cornwall
Registered: 2005-11-19
Posts: 4,134
GitHub

Re: Textpattern web servers - the next 18 months

1 Apart from our brief dalliance with the .io suffix, which was an emergency measure in a period when we didn’t directly control the domain names.

…and we still get considerable traffic to .io, which bounces to .com, so we can’t get rid of that just yet.

Offline

#8 2020-06-30 13:41:39

michaelkpate
Moderator
From: Avon Park, FL
Registered: 2004-02-24
Posts: 1,379
Website GitHub Mastodon

Re: Textpattern web servers - the next 18 months

gaekwad wrote #324089:

If all goes to plan, we should also be self-hosting Textpattern documentation. Right now, the files are on GitHub Sites and we’re limited with what we can do with that. Hosting our own docs site has been on my to-do list for a while, and I hope to investigate it more thoroughly as the daylight hours get shorter in the latter part of 2020.

I googled “moving from github pages to self-hosting” – I found Moving from GitHub pages to self-hosted and Self-hosted GitHub pages

Offline

#9 2020-06-30 13:53:34

gaekwad
Server grease monkey
From: People's Republic of Cornwall
Registered: 2005-11-19
Posts: 4,134
GitHub

Re: Textpattern web servers - the next 18 months

Thanks, Michael – the key seems to be picking the right static site generator, of which there are…well, probably a bajillion.

Offline

#10 2020-06-30 14:13:13

philwareham
Core designer
From: Haslemere, Surrey, UK
Registered: 2009-06-11
Posts: 3,564
Website GitHub Mastodon

Re: Textpattern web servers - the next 18 months

gaekwad wrote #324124:

Thanks, Michael – the key seems to be picking the right static site generator, of which there are…well, probably a bajillion.

Well, surely we would be sticking with Jekyll regardless of where we host it? The site has been created for that platform and I have no desire to remake it.

The main issues with GitHub Pages are:

  1. No control over the caching, HTTP headers, compression, security headers, etc. Server-side stuff.
  2. The reduced Jekyll feature set – mainly the heavy restriction on Jekyll plugins when used on GitHub.
  3. Reliance on GitHub’s platform being up and running normally.

Offline

#11 2020-06-30 14:14:08

gaekwad
Server grease monkey
From: People's Republic of Cornwall
Registered: 2005-11-19
Posts: 4,134
GitHub

Re: Textpattern web servers - the next 18 months

philwareham wrote #324125:

Well, surely we would be sticking with Jekyll regardless of where we host it? The site has been created for that platform and I have no desire to remake it.

No reason why we can’t, I just haven’t found/made time to figure it out yet.

Edit: to clarify I’m totally OK with Jekyll, I have no wish to make work for anyone.

Last edited by gaekwad (2020-06-30 14:20:29)

Offline

#12 2020-06-30 14:24:52

Bloke
Developer
From: Leeds, UK
Registered: 2006-01-29
Posts: 11,250
Website GitHub

Re: Textpattern web servers - the next 18 months

All great stuff, Pete. Solid plan.

philwareham wrote #324125:

The reduced Jekyll feature set

Yeah, I’d love to have some better Liquid support so we could do far better automation than we have now. The limited looping and variable scoping available on GitHub Pages mean we have quite a bit of repeated code hanging around. I’d love to refactor some of the repetitive content from tags, for example, into more manageable chunks with dynamic data squirted into it.


The smd plugin menagerie — for when you need one more gribble of power from Textpattern. Bleeding-edge code available on GitHub.

Txp Builders – finely-crafted code, design and Txp

Offline

Board footer

Powered by FluxBB