Adventures in Linux Land

Algaris · 2024-08-21 13:03:52

I recently realised that I’ve been posting quite a bit about Linux and having discussions with Pete regarding its various aspects. I also noticed that the discussions were spread across multiple topics and threads.

In order to consolidate things (and not derail other topics) I thought I’d try to bring things under one roof. This way other people can contribute here if they want to, and if they find any of the information useful all the better.

So, from now on I’ll try to post all my Linux related questions, discoveries, and musings in this topic.

Ross talks about TrueNas and Ubuntu Server’s for Raspberry Pi’s

Ross and Pete talk about the differences between Ubuntu and Debian

viewtopic.php?pid=334849#p334849

Ross struggles to understand Linux permissions

* viewtopic.php?id=52076

Ross tries (and fails) to understand Docker

viewtopic.php?id=52287

Ross and Pete discuss freenginx, Nginx, Caddy, and migrating from Ubuntu to Debian

Ross discovers FrankenPHP

viewtopic.php?pid=337658#p337658

Last edited by Algaris (2024-08-21 13:46:08)

gaekwad · 2024-08-21 13:58:13

Yay!

Edit: get this in yer ears: selfhosted.show

Last edited by gaekwad (2024-08-21 13:58:59)

Algaris · 2024-08-23 13:30:34

gaekwad wrote #337698:

Edit: get this in yer ears: selfhosted.show

Oooh. Something else to explore.

Last edited by Algaris (2024-08-23 13:43:47)

Algaris · 2024-08-23 13:43:36

So it turns out the main way to use FrankenPHP is via Docker containers. I’m still trying to wrap my head around how Docker works.

Working with Docker containers is the opposite of my current workflow. For each project I spin up a new virtual machine, install Debian, configure Caddy, PHP, MySQL, etc. and then install the application I want to work with i.e. Textpattern. From my limited understanding Docker uses one server, one OS, and each component PHP, MySQL, Textpattern, etc. resides in its own container which can then communicate with each other. It allows you to run multiple projects on one server instead of having to spin a new one up for each project.

Until I can understand Docker I probably won’t move any current or near future projects over to FrankenPHP. I just don’t have the time to fully learn Docker and make the necessary changes to my workflow given my deadlines (the school I work for starts back in a matter of weeks).

Last edited by Algaris (2024-08-23 13:48:29)

Algaris · 2024-08-28 13:30:51

gaekwad wrote “#337740”: ../viewtopic.php?pid=337740#p337740

Upcoming:

PHP via Nginx Unit or other app server

So would Nginx Unit replace PHP-FPM? I’m guessing this is similar to how with FrankenPHP you don’t need PHP-FPM (except that FrankenPHP replaces the whole web server as well)?

gaekwad · 2024-08-28 13:33:46

Algaris wrote #337742:

So would Nginx Unit replace PHP-FPM?

Correct. I don’t have any data to know it’s more performant in real terms, but time will tell.

I’m guessing this is similar to how with FrankenPHP you don’t need PHP-FPM (except that FrankenPHP replaces the whole web server as well)?

That I don’t know – I haven’t looked at Caddy / FrankenPHP properly yet, so I’ll defer to your know-how there!

Algaris · 2024-08-28 13:49:36

gaekwad wrote #337744:

That I don’t know – I haven’t looked at Caddy / FrankenPHP properly yet, so I’ll defer to your know-how there!

With Caddy you still need PHP-FPM. During my research I came across the following:

FrankenPHP simplifies your deployment process by eliminating the need for separate PHP-FPM and Nginx processes.

I just needed to dig a little deeper I guess.

Algaris · 2024-10-21 10:34:03

I’ve been on another rabbit hole research session this time regarding how to upgrade to a new server version of Debian. From my research I found that you need to edit the /etc/apt/sources.list file before doing an upgrade.

The process seems quite different to Ubuntu. In Ubuntu you’re usually informed when SSHing into the server and can upgrade using the do-release-upgrade.

To get another perspective on this I was wondering what people’s best practices are for upgrading to a new Debian server version are? I also have a few questions if anyone is able to offer advice on this matter.

1. Does Debian inform you when there’s a new server version when SSHing into the server or are you expected to keep abreast of the latest releases yourself?

2. What’s the best way to edit the sources.list and specify the new Debian version? Can you just edit it with your text editor of choice (for example Nano)?

3. Once you’ve updated the sources.list do you just upgrade with:

sudo apt-get update
sudo apt-get dist-upgrade

If anyone notices that I have any knowledge gaps in this process please let me know. Currently I don’t know what I don’t know. I’m having fun trying to plug the gaps.

gaekwad · 2024-10-21 11:36:44

Algaris wrote #338035:

To get another perspective on this I was wondering what people’s best practices are for upgrading to a new Debian server version are? I also have a few questions if anyone is able to offer advice on this matter.

For me, I don’t do in-place upgrades. I build new and ‘rotate’ servers every year or two to benefit from newer (better) hardware and general spring cleaning to de-cruft.

I did a few in-place upgrades when I was testing Debian 11 -> Debian 12, and they did work OK. Here’s what I did:

if \
[[ $(awk -F= '$1=="VERSION_CODENAME" { print $2 ;}' /etc/os-release) = "bullseye" ]] \
; then \
export DEBIAN_FRONTEND=noninteractive \
&& apt update \
&& apt -y dist-upgrade \
&& apt -y full-upgrade \
&& apt -y autoclean  \
&& apt -y autoremove --purge \
&& sed -i 's/bullseye/bookworm/g' /etc/apt/sources.list \
&& sed -i -- 's/bullseye/bookworm/g' /etc/apt/sources.list.d/*.list \
&& apt clean \
&& apt update \
&& apt -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" dist-upgrade \
&& apt -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" full-upgrade \
&& apt -y autoclean  \
&& apt -y autoremove --purge \
&& reboot \
; fi

This is old code that I haven’t touched in a while, so test in development first. Also, it was the first-run task on a new server, so no sudo preface (i.e. I was raw dogging as root). You’ll need to sudo if you’re not root.

1. Does Debian inform you when there’s a new server version when SSHing into the server or are you expected to keep abreast of the latest releases yourself?

That will be down to your server vendor’s motd stuff. You can change all the login stuff you see by modifying the contents of /etc/update-motd.d/. The numbers relate to the order in which they’re executed (shown to you).

2. What’s the best way to edit the sources.list and specify the new Debian version? Can you just edit it with your text editor of choice (for example Nano)?

I’d use sed from the command line, but you can edit in your editor of choice, yes. The benefit of sed is it’s not liable to you missing an instance.

3. Once you’ve updated the sources.list do you just upgrade with:

sudo apt-get update
sudo apt-get dist-upgrade

Yup, that’s about it. I embellish it a little bit (see code above) but you’re basically on the right track.

If anyone notices that I have any knowledge gaps in this process please let me know. Currently I don’t know what I don’t know. I’m having fun trying to plug the gaps.

Welcome to my world, amigo! See also “The Rumsfeld Matrix” – here’s the gist:

Reports that say that something hasn’t happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don’t know we don’t know. And if one looks throughout the history of our country and other free countries, it is the latter category that tends to be the difficult ones.

I’m up to 7,645 commits in my server build guide…ya kinda get used to iterating and learning, it’s good for the soul.

Last edited by gaekwad (2024-10-21 12:45:45)

Algaris · 2024-10-23 08:22:58

Thank you, it’s helpful to know I’m on the right track.

It’s interesting that you don’t do in place upgrades. That’s mostly how I work, apart from the odd time I’ve needed to build a replacement server. I understand most (but not all) of your upgrade script it looks pretty much belts and braces.

I didn’t know about motd and how you could customise the welcome message. I’ll need to take a look at that. I’d like to be informed of any new Debian versions when logging in.

Again, I didn’t know about sed. I’ll need to look into this and see how it works.

Ah, The Rumsfeld Matrix. Yes that seems highly applicable to my situation when it comes to Linux. Every time I think I’m making progress something comes along to pull the rug out from under me, I learn something new which completely changes my workflow, or I find out I’ve been doing things wrong from the get go (Linux permissions comes to mind here).

gaekwad · 2024-10-23 08:56:59

Algaris wrote #338051:

It’s interesting that you don’t do in place upgrades.

This is mostly down to these things:

The confidence I have in building a new server from scratch to a deadline (the build guide is the codex)
Being burned in the past when I blindly upgraded from Ubuntu 18.04 to Ubuntu 20.04 on a production server (once is enough for that, lesson learned)
I like newer hardware for its performance & lifespan (a three-year old VPS is likely running on at least three-year old hardware, and new datacenters are often better equipped than legacy ones
A well-used VPS gathers cruft, which bugs me
It’s a good opportunity to try new things (see below)

I pushed myself to try Debian a few years ago and now I’m migrating all my servers to Debian on the next iteration of rebuild. I pushed myself to try Hetzner for the aarch64/ARM64 servers and they’re fantastic for the price point. This involves me learning how to compile some stuff from source where previously I’d‘ve used the package manager. Almost all apt packages on Debian & derivatives are available for amd64/x64, but coverage for ARM64 is lower. Now I can compile Percona, MariaDB and Oracle MySQL from source on ARM64. It takes hours, but it works.

This also lets me have (for example) multiple database servers running on the same VPS without clashing (like I do with multiple PHP instances): installing each product to /opt/vendor/product/version/ (e.g., /opt/percona/mysql/8.4/ rather than the standard package install path means I can tool up a whole raft of providers on the same box, each using different ports, distinct service names and systemd files, and I’m only limited by server resources (and my own sanity). It should, in theory, be possible for me to run three MySQL vendors each with two or three LTS versions on the same server, and none of them will clash.

I understand most (but not all) of your upgrade script it looks pretty much belts and braces.

Let me know what you’re not sure on, and I’ll happily explain what & why.

I didn’t know about motd and how you could customise the welcome message.

My plan over the coming months is to scrub what the OS vendor provides and drop my own scripts in. I’ve started a public ‘server conf’ repo so I can get fresh motd stuff added, and other stuff.

My .profile file does some fancy stuff with checking for a swapfile, running duf to show me disk space used, and a few other things.

Again, I didn’t know about sed. I’ll need to look into this and see how it works.

sed is a bit of a dark art, and you will need to understand some regular expressions stuff, but you can kludge your way along with online examples that can be remixed to suit.

Every time I think I’m making progress something comes along to pull the rug out from under me, I learn something new which completely changes my workflow, or I find out I’ve been doing things wrong from the get go (Linux permissions comes to mind here).

Amen, brother. Couldn’t have put that better myself! Iterate, iterate, iterate.

Algaris · 2024-10-24 10:58:06

Thank you for your detailed reply Pete.

These are the parts of the script I’m unsure about

[[ $(awk -F= '$1=="VERSION_CODENAME" { print $2 ;}' /etc/os-release) = "bullseye" ]] \

export DEBIAN_FRONTEND=noninteractive \

&& sed -i 's/bullseye/bookworm/g' /etc/apt/sources.list \
&& sed -i -- 's/bullseye/bookworm/g' /etc/apt/sources.list.d/*.list \

&& apt -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" dist-upgrade \

&& apt -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" full-upgrade \

running duf to show me disk space used, and a few other things.

duf looks amazing. Thank you for sharing. It’s so much better than running df -h.

Almost all apt packages on Debian & derivatives are available for amd64/x64, but coverage for ARM64 is lower.

Interesting. I didn’t think about that. Trying a Hetzner server has been on my list of things to look at. I’m not brave enough to compile things from source. I just want to install Percona, Caddy, PHP, Composer, etc. using apt.

sed is a bit of a dark art, and you will need to understand some regular expressions stuff…

I’m guessing RegEx is the shortened name for Regular Expressions? I’ve heard people whisper about RegEx in hushed tones of horror.

Textpattern CMS

Textpattern CMS support forum

#1 2024-08-21 13:03:52