{"id":1596,"date":"2024-02-14T12:40:08","date_gmt":"2024-02-14T11:40:08","guid":{"rendered":"https:\/\/mpr-projects.com\/?p=1596"},"modified":"2024-02-27T07:37:45","modified_gmt":"2024-02-27T06:37:45","slug":"rpi-nas-borg-backup","status":"publish","type":"post","link":"https:\/\/mpr-projects.com\/index.php\/2024\/02\/14\/rpi-nas-borg-backup\/","title":{"rendered":"RPi NAS: Borg Backup"},"content":{"rendered":"\n

In this post we’re going to use the software packages Borg Backup<\/a> and Borgmatic<\/a> to do system backups of both our Raspberry Pi and our main computer. We’ll also schedule automatic backups using both cron and systemd.<\/p>\n\n\n

This post is part of a series about building a Network-Attached Storage (NAS) with redundancy using a Raspberry Pi (RPi). See here<\/a> for a list of all posts in this series.<\/p>\n\n\n\n

\n\n\n\n

During the main part of this series we set up a network-attached storage solution (NAS) using a Raspberry Pi. The software we used to provide data redundancy, Greyhole, is not suitable for storing lots of small files like those in a system backup (because of Greyhole’s overheads, see here<\/a> for more information). So in this post we’ll set up a proper system backup.<\/p>\n\n\n\n

We’ll be using Borg Backup, which is available on Linux and Mac. Running it on Windows is only experimental. The advantage of such a backup software (vs just manually copying data) is that it works incrementally. So only files that have changed are saved during the backup, which makes the backup faster and require less space1<\/a><\/sup>. It also allows us to easily encrypt our backups and we can browse through them if we just need to restore a few files. Our actual interaction with Borg Backup will happen through a helper software called borgmatic<\/a>.<\/p>\n\n\n\n

Let’s get on with it. First we’ll do a simple setup on the Raspberry Pi that runs our NAS. Then we’ll set up a slightly more complex version on my main computer.<\/p>\n\n\n\n

Borg on the Raspberry Pi<\/h3>\n\n\n\n

This is going to be a very simple backup. All we’re going to back up is our configuration files and scripts so we don’t have to redo the Samba and Greyhole setup if the SD card of the Raspberry Pi stops working. There’s a great description of the installation process of borgmatic here<\/a>. Below I’ll go through the steps for the specific setup on the Raspberry Pi.<\/p>\n\n\n\n

Basic Setup<\/h4>\n\n\n\n

First we need to install the software with<\/p>\n\n\n\n

sudo apt install borgmatic<\/code><\/pre>\n\n\n\n
This command will automatically install all required software, such as Borg Backup. Next we create a sample configuration file. On the version currently (in Jan-24) available on Raspberry Pi OS we do that with the following command.<\/p>\n\n\n\n
sudo generate-borgmatic-config<\/code><\/pre>\n\n\n\n
This command generates the file \/etc\/borgmatic\/config.yaml<\/em>. Newer versions of borgmatic require a slightly different command, which we’ll use in the next section. Let’s open the configuration file with<\/p>\n\n\n\n
sudo vim \/etc\/borgmatic\/config.yaml<\/code><\/pre>\n\n\n\n
It’s quite a long file with lots of options for customization. For this simple backup we’re only using the following lines:<\/p>\n\n\n\n
\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n\n\n\n
The source_directories<\/em> section specifies the folders we want to back up2<\/a><\/sup>. Under repositories<\/em> we can choose one or more locations where the backup should be saved (we can easily store them in a redundant way by adding additional locations). Section exclude_patterns<\/em> allows us to define rules to skip certain files or folders3<\/a><\/sup>.<\/p>\n\n\n\n
Finally, it’s usually not necessary to keep all backups we’ve ever made. Typically, we care most about the latest backups (occasionally older backups can be useful, e.g. if we want to recover a specific, older file). To save some space we can specify how many daily, monthly, weekly and yearly backups should be kept. That’s done in section retention<\/em>. Borg\/borgmatic will automatically remove backups that are in excess of what we specify there. I don’t expect the configuration files to change very often so I’m only keeping very few backups.<\/p>\n\n\n\n
After closing the file we can create the repository for our backups with<\/p>\n\n\n\n
sudo borgmatic init --encryption none<\/code><\/pre>\n\n\n\n
You’ll notice that I’ve not set up encryption for this repository. That’s because I don’t think it’s necessary for this specific backup (there’s nothing private on there). In the next section, which backs up our main system, we will<\/em> encrypt our backup.<\/p>\n\n\n\n
Now let’s create our initial backup:<\/p>\n\n\n\n
sudo borgmatic create --verbosity 1 --list --stats<\/code><\/pre>\n\n\n\n
On a larger system this command can take quite a while. On our Raspberry Pi, however, we’ve only specified a small number of folders to back up and there’s not much data in them so the initialization won’t take long.<\/p>\n\n\n\n
Automatic Backups with Cron<\/h4>\n\n\n\n
Since we don’t want to run the command above manually (we’ll probably forget to do it + we’d have to log into the Raspberry Pi all the time) we automate them with cron. Borgmatic provides a cron script<\/a> which you can download but I prefer to add the line below to the crontab file4<\/a><\/sup>. Let’s open it with<\/p>\n\n\n\n
sudo crontab -e<\/code><\/pre>\n\n\n\n
and add the following line<\/p>\n\n\n\n
\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n\n\n\n
With this line there will be a backup every day at 3am and a summary will be written to the log-file saved in nas_user<\/code>‘s home directory (you could also write the log file to one of the NAS’s Samba shares so you don’t have to log into the RPi to view it).<\/p>\n\n\n\n
Backing up to a Remote System<\/h3>\n\n\n\n
Now that we’ve done a simple setup for our Raspberry Pi let’s look at something a bit more complex. I want my main computer to do a backup every 3 hours. Again the data will be saved on the storage drives of the NAS (but you could use any local or remote repository).<\/p>\n\n\n\n
Basic Setup<\/h4>\n\n\n\n
First we need to install Borg. If you’re on a system that uses apt<\/em> then you can run the same command as in the previous section. I use Arch Linux on my main system so the following command installs Borg and borgmatic (and all other requirements):<\/p>\n\n\n\n
sudo pacman -S borgmatic<\/code><\/pre>\n\n\n\n
The backup command will be executed by the root user5<\/a><\/sup> and it will run over ssh, so we have to make sure that the root user can access the Raspberry Pi via ssh. First we create a separate ssh key for our root user and save it under \/root\/.ssh\/key_xps17_root_user<\/em> (using sudo ssh-keygen<\/code>). Then we create or update the file \/root\/.ssh\/config<\/em> with the following contents:<\/p>\n\n\n\n
\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n\n\n\n
Now we copy the public key we’ve just created into the authorized_keys<\/em> file on the Raspberry Pi. With this setup our root user will be able to ssh into the Raspberry Pi without being asked for a password (as long as you didn’t set a password during the generation of the ssh key).<\/p>\n\n\n\n
You could also reuse the ssh key of your regular user6<\/a><\/sup>. Whichever key you use, make sure that it’s listed in the authorized_keys<\/em> file on the Raspberry Pi (\/home\/nas_user\/.ssh\/authorized_keys<\/em>) so root can connect7<\/a><\/sup>.<\/p>\n\n\n\n
Next we generate a configuration file. On my Arch System the latest version of borgmatic is available. With this version the command to generate a config file is:<\/p>\n\n\n\n
sudo borgmatic config generate<\/code><\/pre>\n\n\n\n
The configuration file will be saved in \/etc\/borgmatic\/config.yaml<\/em> 8<\/a><\/sup>. We’ll start with a simple file and then we’ll add a few useful features to it.<\/p>\n\n\n\n
\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n\n\n\n
You’ll notice a few changes to the previous section. First, the repository is now remote and we connect to it via ssh. Second, we’ve added an encryption_passphrase<\/em>. As a result our backups will be encrypted and can only be accessed with the password (and the key but that’s stored with the repository). <\/p>\n\n\n\n
Make sure to also save the encryption password in another, secure location! Otherwise, if your computer breaks and you don’t have access to the configuration file, you won’t be able to decrypt your backups! Borg\/borgmatic also recommend saving a copy of the key9<\/a><\/sup>.<\/p>\n\n\n\n
After you’ve adjusted the configuration file to your needs we can create the repository with:<\/p>\n\n\n\n
sudo borgmatic rcreate --encryption repokey<\/code><\/pre>\n\n\n\n
Then we export the key of the repository to the current directory so we can securely save it somewhere.<\/p>\n\n\n\n
sudo borg key export ssh:\/\/nas_user@192.168.50.210\/nas_mounts\/hdd0\/borg_backups\/demo .\/borg_demo_key_backup<\/code><\/pre>\n\n\n\n
Before we do our first backup let’s add exclude_patterns<\/em>, which allow us to ignore certain files or folders:<\/p>\n\n\n\n
\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n\n\n\n
We also enable the following options<\/p>\n\n\n\n
\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n\n\n\n
which allow us to manually exclude certain directories from being backed up (that can be quite useful in situations where some of your custom code creates a lot of temporary data).<\/p>\n\n\n\n
When I set up my repository I started with a small number of directories and gradually expanded them (so I could make sure that no large, unnecessary files were saved). So let’s now create an initial backup with the command below (depending on your source directories, this may take a while):<\/p>\n\n\n\n
sudo borgmatic create --verbosity 1 --list --stats<\/code><\/pre>\n\n\n\n
Additional Features<\/h4>\n\n\n\n
There are a few more features I’d like to have. First, I only want the backup to run when the remote server can be reached (e.g. there’s no point in attempting a backup if you’re on a laptop and away from home). We can achieve that by ping’ing the Raspberry Pi’s IP address and checking for a response.<\/p>\n\n\n\n
\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n\n\n\n
Next I want to be informed when a backup is taking place. For that we’ll use Desktop Notifications<\/a>. If you’re on a system like Ubuntu then you should already have a notification daemon installed. Since Arch Linux is quite bare bones10<\/a><\/sup> I had to set it up myself (I use mako<\/a> which works on Wayland\/Sway). You can test if you have a working daemon by sending a test message:<\/p>\n\n\n\n
notify-send -t 0 \"hi there\"<\/code><\/pre>\n\n\n\n
With the option -t 0<\/em> your message will not automatically disappear after a few seconds but stay on screen until the user clicks on it.<\/p>\n\n\n\n
This works well if the message is sent and received by the same user. In our case the message will be sent by the root user (who executes the backup) and received by our regular user. That makes it a bit more difficult because the correct session bus needs to be used when sending the message. I wrote the helper script below, which works well on Wayland11<\/a><\/sup>. The script is saved under \/root\/notify_send_to_user<\/em> (the file is owned by root which has execute permissions on it).<\/p>\n\n\n\n
\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n\n\n\n
If the root user wants to send a non-disappearing message to user mpr<\/em> then it would run<\/p>\n\n\n\n
\/root\/notify_send_to_user mpr 0 \"message\"<\/code><\/pre>\n\n\n\n
You can test if it works for you by temporarily becoming root with sudo su<\/code> (or using sudo<\/em>) and running the command above. A notification should pop up on your desktop.<\/p>\n\n\n\n
Now we adjust borgmatic’s configuration file so it informs us about the update process (replace the previous before_everything<\/em> section with the one below):<\/p>\n\n\n\n
\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n\n\n\n
This will send a message when the backup starts and when it has finished. If the RPi couldn’t be reached then we get a notification and the backup stops.<\/p>\n\n\n\n
Finally, I have multiple WiFi networks at home and sometimes I’m not connected to the one with the Raspberry Pi. In that case the backup fails and I get a notification saying Couldn’t connect to rpi-nas.<\/em> The options below tell borgmatic to try again three minutes later, which gives me some time to connect to the correct WiFi.<\/p>\n\n\n\n
\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n\n\n\n
Automatic Backups with Systemd<\/h4>\n\n\n\n
In the previous section we used cron to automate backups. In this section we’ll use systemd. Borgmatic already provides us<\/a> with the two files we need:<\/p>\n\n\n\n