Textpattern CMS support forum
You are not logged in. Register | Login | Help
- Topics: Active | Unanswered
Re: Adventures in Linux Land
sed and awk are amazing tools but I confess to using them so infrequently that I have to search for potted examples to bastardize to suit my needs.
Regular expressions can be quite scary if you’re doing advanced things like negative lookahead/behind, but thankfully most of the times only a handful of simpler expressions are necessary.
The smd plugin menagerie — for when you need one more gribble of power from Textpattern. Bleeding-edge code available on GitHub.
Txp Builders – finely-crafted code, design and Txp
Offline
Re: Adventures in Linux Land
I’ll break down the script into parts, so it makes more sense…usual house rules: let me know if anything isn’t clear.
Here’s the script in full:
if \
[[ $(awk -F= '$1=="VERSION_CODENAME" { print $2 ;}' /etc/os-release) = "bullseye" ]] \
; then \
export DEBIAN_FRONTEND=noninteractive \
&& apt update \
&& apt -y dist-upgrade \
&& apt -y full-upgrade \
&& apt -y autoclean \
&& apt -y autoremove --purge \
&& sed -i 's/bullseye/bookworm/g' /etc/apt/sources.list \
&& sed -i -- 's/bullseye/bookworm/g' /etc/apt/sources.list.d/*.list \
&& apt clean \
&& apt update \
&& apt -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" dist-upgrade \
&& apt -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" full-upgrade \
&& apt -y autoclean \
&& apt -y autoremove --purge \
&& reboot \
; fiThe first and last lines are an if statement, which breaks down like this:
if \
[[ CONDITION ]] \
; then \
DO STUFF \
; fiThe condition is:
$(awk -F= '$1=="VERSION_CODENAME" { print $2 ;}' /etc/os-release) = "bullseye"The $(…) wrapper can be interpreted as “the result of running the contents” and we can strip that away, which leaves:
awk -F= '$1=="VERSION_CODENAME" { print $2 ;}' /etc/os-releaseThe last part is a file with operating system release info. This is a standard file on Debian and derivates, and it contains a bunch of lines with one chunk of info per line, similar to this:
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"High level: the awk part is checking for the line beginning VERSION_CODENAME, and if the value is bookworm then the condition is true. This means the ‘do stuff’ part can proceed. If the condition is false (i.e., the line doesn’t exist, or the value is set to something that’s not bookworm, the ‘do stuff’ part won’t happen.
export DEBIAN_FRONTEND=noninteractive is setting a name-value pair as part of the environment. If you export a variable, it is passed to child processes. If you set a variable without using export (e.g. DEBIAN_FRONTEND=noninteractive), it won’t be passed to child processes. The upgrade procedure for Debian kicks off a lot of processes, so it’s more efficient to tell it to “crack on m8 lol” and just get the job done.
sed -i 's/bullseye/bookworm/g' /etc/apt/sources.list is using sed to find all instances of bullseye and replace with bookworm in the file /etc/apt/sources.list. The -i part does it in-place (or inline), so it makes changes to the file specified, rather than making changes to a newly-created file and leaving the original intact.
sed -i -- 's/bullseye/bookworm/g' /etc/apt/sources.list.d/*.list is similar to the above, but it’s a wildcard for *.list files, and I honestly can’t recall why -- is in there. It might have something to do with the wildcard.
apt -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" dist-upgrade is essentially telling apt to run with options with regards to config files. --force-confdef means “[I]f a conffile has been modified and the version in the package did change, always choose the default action without prompting.”, and --force-confold means “[I]f a conffile has been modified and the version in the package did change, always keep the old version without prompting, unless the --force-confdef is also specified, in which case the default action is preferred.”
Or, another way, don’t break my configs if I’ve changed it from the default.
In apt land, dist-upgrade and full-upgrade are the same thing, and I can probably remove one of them. It’s been a while since I used this script, so mea culpa there. I think I only made this script for the period where DigitalOcean had Debian 11 but not Debian 12, and I used it on a vanilla Debian 11 to force it to be Debian 12 after a reboot or two.
Bloke wrote #338056:
sed and awk are amazing tools but I confess to using them so infrequently that I have to search for potted examples to bastardize to suit my needs.
This, so much. It’s where a build guide comes into its own, once you’ve got a working version you can crib it for another task.
Compiling software is very similar in that respect. I started with compiling Nginx from source some years ago, and that gave me a framework for other software compiling. Take this example for ImageMagick:
imagemagick_source_version="7.1.1-39" \
&& sudo mkdir -p \
/opt/imagemagick/ \
&& rm -rf \
"$HOME"/imagemagick-source \
"$HOME"/imagemagick-source.tar.gz \
&& mkdir -p \
"$HOME"/imagemagick-source \
&& curl -Lo \
"$HOME"/imagemagick-source.tar.gz \
https://github.com/ImageMagick/ImageMagick/archive/"$imagemagick_source_version".tar.gz \
&& tar xzvf \
"$HOME"/imagemagick-source.tar.gz \
-C "$HOME"/imagemagick-source \
&& cd "$HOME"/imagemagick-source/ImageMagick-"$imagemagick_source_version" \
&& ./configure \
--disable-docs \
--prefix=/opt/imagemagick \
&& make -j"$(nproc)" \
&& sudo make -j"$(nproc)" install \
&& sudo ldconfig /usr/local/lib \
&& sudo make -j"$(nproc)" clean \
&& cd "$HOME" \
&& rm -rf \
"$HOME"/imagemagick-source \
"$HOME"/imagemagick-source.tar.gz \
&& echo -e '\n=> Checking ImageMagick version...' \
&& /opt/imagemagick/bin/identify -versionThe workflow there is: set the version number as a variable, build the destination scaffold, prep the build area scaffold, get the source, expand the source, configure the source, compile the source, install the compiled source, update shared libraries, clear the build area, check it’s working. When there’s a new version of ImageMagick, I update the version number and rebuild it.
That’s about 30 lines of code. A PHP compile is about 90 lines. An Nginx compile is about 240 lines because it uses a bunch of external modules and other config settings, but the workflow is largely the same.
A Percona MySQL 8.4 compile is about 25 lines, and takes hours to finish (at least on ARM64) because it uses a different build system. From memory, the Percona install I did over the weekend on a 4-core Hetzner ARM64 took 5 hours.
Last edited by gaekwad (2024-10-24 15:29:54)
Offline