Adventures in Linux Land

gaekwad · Today 12:09:57

This was the proposed command:

curl -fsSL https://repo.mysql.com/RPM-GPG-KEY-mysql-2022 | gpg --dearmor | sudo tee /usr/share/keyrings/mysql.gpg > /dev/null

…let’s chop it out a bit. The first task is to look at the | characters as essentially ‘glue’ between three commands that are concatenated into one. The three commands are:

curl -fsSL https://repo.mysql.com/RPM-GPG-KEY-mysql-2022
gpg --dearmor
sudo tee /usr/share/keyrings/mysql.gpg > /dev/null

The pipe (|) character essentially takes the output of the previous command and chucks it into the following to process. In the case of the proposed command at the top, there are three commands with two bits of pipe ‘glue’, so the result of command 1 of 3 gets chucked into command 2 of 3, and the results of command 2 of 3 get chucked int command 3 of 3.

Command 1 of 3

curl -fsSL https://repo.mysql.com/RPM-GPG-KEY-mysql-2022 fetches the file at that URL, and the -fsSL parameter chunk are the options for curl to run in a certain way. Briefly (and man curl is your friend here for bedtime reading):

-f is short for --fail which means if the web server at the other end throws a result that’s not the file you’re expecting (e.g. file not found HTML page), it won’t download that file as the thing you’re looking for…it just won’t download anything.
-s is short for --silent: don’t output anything to the screen.
-S (note: capital) is short for --show-error: shows the error if there is one. This overrides the -s part if there’s an error, so you will see there’s an error.
-L is short for --location: bounces the curl request to the new location if it’s moved, for example.

Command 1 of 3 should – in theory, at least – show nothing on the screen at all if there are no errors: it’s downloading a URL, but not saving it, and not displaying it (i.e. -s is preventing it from being shown. Here’s where the fun stuff happens: the | command essentially sends (pipes) the curl output into the second command.

Command 2 of 3

Command 2 of 3 literally runs the de-armoring part of gpg of the stuff it’s been given. In this case, the original URL contents are:

That’s an encoded (not encrypted) text block that gpg needs to de-armor (decode) before it can be used. The contents are a (presumably valid) key for MySQL updates in a format that the system understands. I won’t post the de-armor’d results here as it’s gibberish on the human-readable front.

Story so far: curl (command 1 of 3) has acquired (but not saved) content from a URL, and fed it into gpg (command 2 of 3) which has de-armor’d it, and now command 3 of 3 has the net result of those two commands.

Command 3 of 3

tee is a read + write command. It’s handy when you have a need for it, and it works really well at what it sets out to do. tee in this case takes the output of command 2 of 3, which is piped (|) into tee, and it saves that output to the file /usr/share/keyrings/mysql.gpg. Since it’s doing this as root (sudo) it should go straight through and save to that file without any permissions notifications. The clever part here is the > /dev/null bit on the end…that sends the screen output of the tee command (which does show its output to the screen by default) to /dev/null, which is a UNIX / Linux device that is a black hole…that is to say, the output goes to somewhere that it can’t be seen or retrieved from…in other words, there’s nothing to see here.

Command 1 of 3 has no output to screen (assuming no HTTP errors);
Command 2 of 3 has no output to screen since it’s piping the result to command 3 of 3; and…
Command 3 of 3 has no output to screen since it’s outputting (>) to /dev/null.

…so if you run that original command, if all goes to plan you’ll see nothing on the screen and be returned to a prompt. Kinda scary, but expected.

That hopefully explains the | stuff to an extent that you can wrap your head around the basics. Now take a look at my command a few posts later:

gaekwad wrote #341053:

curl -Lo \
"$HOME"/mysql-apt-config.deb \
https://dev.mysql.com/get/mysql-apt-config_0.8.35-1_all.deb \
&& sudo dpkg -i \
"$HOME"/mysql-apt-config.deb

There are two things I’d like to explain to you that might help you out: && and \.

The && part is another ‘glue’ connector between commands, but in this case it translates to “if that command completes, run the next command”. In this instance, if my curl command completes, then run the dpkg command.

The \ part means the command runs onto the following line, and is mainly for legibility and clarity in my own docs. The example above is two commands split across 5 lines. Removing the \ and && boils it down to two lines:

curl -Lo "$HOME"/mysql-apt-config.deb https://dev.mysql.com/get/mysql-apt-config_0.8.35-1_all.deb
sudo dpkg -i "$HOME"/mysql-apt-config.deb

Essentially: download the URL to a local file, then run dpkg against that downloaded file. I then clean up the downloaded file so there’s nothing left over.

The \ is really helpful for me with docs since I can do a mass find & replace should I decide to change a directory structure or filename format, and if I’m doing emergency stuff early or late in the day, I don’t have to have peak brainmeat to decipher some of the stuff I’ve written. It also makes for improved copy & pasting for text blocks if I need to repeat stuff in a more complex install, like compiling something from source.

Example (don’t be scared) – let’s compile libgd from source:

libgd_version="2.3.3" \
&& sudo mkdir -p \
/opt/libgd/ \
&& rm -fr \
"$HOME"/libgd-source \
"$HOME"/libgd-source.tar.gz \
&& mkdir -p \
"$HOME"/libgd-source \
&& curl -Lo \
"$HOME"/libgd-source.tar.gz \
https://github.com/libgd/libgd/releases/download/gd-"$libgd_version"/libgd-"$libgd_version".tar.gz \
&& tar xzvf \
"$HOME"/libgd-source.tar.gz \
-C "$HOME"/libgd-source \
&& cd \
"$HOME"/libgd-source/libgd-"$libgd_version" \
&& ./configure \
--prefix=/opt/libgd \
&& make \
-j"$(nproc)" \
&& sudo make \
-j"$(nproc)" \
install \
&& sudo ldconfig \
/usr/local/lib \
&& sudo make \
-j"$(nproc)" clean \
&& cd \
"$HOME" \
&& rm -fr \
"$HOME"/libgd-source \
"$HOME"/libgd-source.tar.gz \
&& echo -e '*********************' \
&& echo -e '* `libgd` compiled. *' \
&& echo -e '*********************' \
&& echo -e "$(date --iso-8601=seconds)"' libgd '$libgd_version >> /var/log/build/core-helpers-libraries.log

Essentially:

Make a directory in /opt for stuff to live in.
Remove any existing directory and tarball relating to libgd.
Download the libgd source.
Unpack the libgd source.
Configure the libgd source.
Build the libgd source.
Install the built libgd.
Remove any existing directory and tarball relating to libgd.

That’s about 30 or so lines of run-on commands to compile a library, but in reality it’s only about 15-20 commands. It’s easier for me to read ~~and wonder what the hell I was thinking~~ when I come back to it after some time. When a new libgd is released, I update the version number on line 1 and recompile. Easy peasy, and takes a few minutes tops.

My compile script for Nginx is about 200 lines, PHP is about 140, and Percona Server for MySQL is about 30. The more finicky finagling needed, the more lines in the script…and having && ‘glue’ is handy.

Edits: Textile hates me.

Last edited by gaekwad (Today 12:12:55)

Algaris · Today 15:44:21

😮 Wow, thank you so much, Pete. This is amazing and incredibly well explained. It makes a lot more sense now. I understood the | piping glue and got that && was another type of glue, but I didn’t realise it was conditional. I’ll need to reread your post a few times for it to completely sink in.

Whenever we meet up next, I definitely owe you a beer or two for all the assistance you’ve given me.

Textpattern CMS

Textpattern CMS support forum

#61 Today 12:09:57

Re: Adventures in Linux Land

Command 1 of 3

Command 2 of 3

Command 3 of 3

gaekwad wrote #341053:

#62 Today 15:44:21

Re: Adventures in Linux Land

Board footer