Install and Configure BorgBackup

Install Borg Dependencies Debian / Ubuntu

---

For Debian/Ubuntu: The FUSE library is available from the main package repository.
Install the dependencies with development headers:

sudo apt-get install python3 python3-dev python3-pip python3-virtualenv \
libacl1-dev libacl1 \
libssl-dev \
liblz4-dev libzstd-dev libxxhash-dev \
build-essential \
pkg-config python3-pkgconfig
sudo apt-get install libfuse-dev fuse    # needed for llfuse
sudo apt-get install libfuse3-dev fuse3  # needed for pyfuse3

To be able send an email from the backup bash command, we have to install mailutils:

apt install mailutils

Look at Borg Source is still not outdated?

Create a Virtual Environment

---

Virtualenv can be used to build and install Borg without affecting the system Python or requiring root access.

virtualenv --python=python3 borg-env
source borg-env/bin/activate

OR with virtualenvwrapper
which python3 # Output: /usr/bin/python3 
mkvirtualenv --python=/usr/bin/python3 borg-env

Install BorgBackup using pip

---

FUSE is required if we want to use Borg mount

 

# might be required if your tools are outdated
pip install -U pip setuptools wheel

# pkgconfig MUST be available before borg is installed!
pip install pkgconfig

# install Borg + Python dependencies into virtualenv
pip install borgbackup
# or alternatively (if you want FUSE support):
pip install borgbackup[pyfuse3]  # to use pyfuse3
pip install borgbackup[llfuse]  # to use llfuse

Configuration SSH connection

---

Generating SSH keys or Copy from another server.

server> ssh-keygen -f $HOME/.ssh/id_rsa_borg
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa_borg):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa_borg.
Your public key has been saved in /root/.ssh/id_rsa_borg.pub.
The key fingerprint is:
cb:3c:xx:39:69:xx:ec:xx:d5:xx:f3:c5:92:99:2f:e1 root@server
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|                 |
|                 |
|         .   =   |
|      . S = * o  |
|   . = = + + =   |
|    X o =   E .  |
|   o + . .   .   |
|    .            |
+-----------------+

Optional: Converting your key to RFC4716 format

This is only necessary if you would like to use SCP or SFTP via SSH port 22.
To convert the public SSH key into the correct format, enter the following command:

server> ssh-keygen -e -f $HOME/.ssh/id_rsa_borg.pub | grep -v "Comment:" > $HOME/.ssh/id_rsa_borg_rfc.pub

The public SSH key should now look like the one below:

server> cat .ssh/id_rsa_borg_rfc.pub
---- BEGIN SSH2 PUBLIC KEY ----
AAAAB3NzaC1yc2EAAAABIwAAAQEAz+fh731CVfH3FPM0vK5hX7NT5HogdBEQ4ryGJIeVMv
mCQJWwrFtdWh1pXMyXsYzXq1xbjILgCZGn+H0qUBKopJaa/Pzsw5U0UyRgiFhU2k0eiHUq
pkiixTbHcLsCj3kjAv5i07wZJ/ot246hLQD1PtSQtcX7nHvhdhenOTGO+ccpM2KEdX1E64
eaTtO9Bf7X4OTXnRxS7tjYH9sls5DOunpvoIZLvbmcVw1+wMdJBXOAU6/tnkN5N3mYE4Hu
JjnRtBAI9MS9Tt3DNAp1K/udUHA6hfYf08fxYs9uwsCM793b7FczmVvHEIwIKszG7Jwiwo
Dqit4EExR8bNNCeD6D3Q==
---- END SSH2 PUBLIC KEY ----

Creating authorized_keys file

Insert the required public SSH keys into a new local authorized_keys file.
For SSH over port 23 (SCP, SFTP, Rsync and Borg Backup), add the public SSH key in OpenSSH format:

cd /root/
server> cat .ssh/id_rsa_borg.pub >> storagebox_authorized_keys

If you converted your public SSH key to RFC4716 format in the previous step, add it as well:

server> cat .ssh/id_rsa_borg_rfc.pub >> storagebox_authorized_keys

You can also add the public SSH key in both formats.

Uploading authorized_keys to StorageBox

Now you need to upload the generated authorized_keys file to the Storage Box/Backup account. To do this, create the directory .ssh with the file rights 0700 (rwx------) and create the file authorized_keys with the public SSH keys and the file rights 0600 (rw-------).

Upload with ssh-copy-id (OpenSSH 8.5+). You can use ssh-copy-id to upload the new key:

ssh-copy-id -p 23 -s uXXXXX@uXXXXX.your-storagebox.de 

Upload Manually. You can do this with the following command, for example:

server> echo -e "mkdir .ssh \n chmod 700 .ssh \n put storagebox_authorized_keys .ssh/authorized_keys \n chmod 600 .ssh/authorized_keys" | sftp <username>@<username>.your-storagebox.de
u12345@u12345.your-storagebox.de's password:
Connected to u12345.your-storagebox.de'.
sftp> mkdir .ssh
sftp>  chmod 700 .ssh
Changing mode on /.ssh
sftp>  put storagebox_authorized_keys .ssh/authorized_keys
Uploading storagebox_authorized_keys to /.ssh/authorized_keys
storagebox_authorized_keys                               100% 2916     2.0MB/s   00:00
# To see file permission use: stat -c "%a %n"
sftp>  chmod 600 .ssh/authorized_keys
Changing mode on /.ssh/authorized_keys

Depending on which SSH key format you have uploaded, you should be able to log in without a password on port 22 and/or 23:

sftp -i /root/.ssh/id_rsa_borg -P <22 or 23> <username>@<username>.your-storagebox.de

Connected to <username>.your-storagebox.de.
sftp> quit

Initialize Borg repository

---

If you are using an SSH key, and this is not the default key, you have the option to specify the desired key using the environment variable BORG_RSH. You can specify the SSH command that Borg should use. The standard would be just ‘ssh’.
$ export BORG_RSH='ssh -i /root/.ssh/id_rsa_borg'

When initializing Borg, you will be prompted for a password for your repository. Only with this password can you access the repository in the future. It is therefore required for every read or write operation on the repository. You must be able to remember the password well because it cannot be restored! To avoid having to enter the password every time Borg calls, you can optionally set the environment variable BORG_PASSPHRASE.
$ export BORG_PASSPHRASE="top_secret_passphrase"

Now you have to create the directory for the backup repository in the storage box. For example, create a folder ‘backups’, and below that, a folder ‘jobsitehr’. The folder ‘jobsitehr’ will then be initialised as a Borg repository in the next step. Under ‘backups’ you could then create further directories for other servers you want to back up.
/backups/jobsitehr

sftp -i /root/.ssh/id_rsa_borg -P 23 u123456@u123456.your-storagebox.de

mkdir backups

Now, we need to initialize the Borg repository. The repository is nothing more than a folder on your storage box that Borg provides with some basic structures. All backups are stored in this folder.
The following command for the primary server initialises the /backups/jobsitehr folder on your storage box:

borg init --encryption=none ssh://u12345@u12345.your-storagebox.de:23/./backups/jobsitehr

# OR with encryption

borg init --encryption=repokey ssh://u12345@u12345.your-storagebox.de:23/./backups/jobsitehr

Backup Key if used --encryption=repokey (borg key export)

---

If repository encryption is used, the repository is inaccessible without the key. This command allows one to backup this essential key. Note that the backup produced does not include the passphrase itself (i.e. the exported key stays encrypted). In order to regain access to a repository, one needs both the exported key and the original passphrase

borg key export ssh://u12345@u12345.your-storagebox.de:23/./backups/jobsitehr /root/.config/borg/keys/backup_repo_key

Create first backup

---

For example, use the following command to back up the src and build folders from your home directory to the repository on your storage box. You must give each backup a unique name. A timestamp is useful for creating unique names.

$ borg create ssh://u123456@u123456.your-storagebox.de:23/./backups/jobsitehr::2022_06_28_initial ~/src ~/built

You can call Borg create using many other options. You can do this, for example, to view the progress of a backup while it is processing or to see statistics about the backup once it is finished. In addition, you can specify exclude patterns and other things.
For more information, please visit the Borg create documentation.

Mount BACKUP STORAGE (borg mount)

---

This command mounts an archive as a FUSE filesystem. This can be useful for browsing an archive or restoring individual files. Unless the --foreground option is given the command will run in the background until the filesystem is umounted.

export BORG_REPO="ssh://u123456@u123456.your-storagebox.de:23/./backups/jobsitehr"
borg mount :: ./mount_backup

# OR 

borg mount ssh://u12345@u12345.your-storagebox.de:23/./backups/jobsitehr ./to_mount
# To unmount
borg umount mount_backup/

# OR CURLFTPS
curlftpfs uXXXXXX:[password]@uXXXXXX.your-storagebox.de /mnt/
curlftpfs u123456:[paswoord]@u123456.your-storagebox.de ./mount_backup/
curlftpfs u123456:[paswoord]@u123456.your-backup.de ./to_mount/

If you get the error like undefined symbol: _PyGen_Send:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /usr/local/venvs/borg-env/lib/python3.10/site-packages/pyfuse3.cpython-310-x86_64-linux-gnu.so: undefined symbol: _PyGen_Send
>>> \q
  File "<stdin>", line 1
    \q
     ^
SyntaxError: unexpected character after line continuation character

Look at pyfuse3 Issue

The solution is using llfuse, istead of pyfuse3, so install it:

pip install llfuse

Get information about a repository or an archive

---

borg info

This command displays detailed information about the specified archive or repository.

Please note that the deduplicated sizes of the individual archives do not add up to the deduplicated size of the repository (“all archives”), because the two are meaning different things:

This archive / deduplicated size = amount of data stored ONLY for this archive = unique chunks of this archive. All archives / deduplicated size = amount of data stored in the repo = all chunks in the repository.

Borg archives can only contain a limited amount of file metadata. The size of an archive relative to this limit depends on a number of factors, mainly the number of files, the lengths of paths and other metadata stored for files. This is shown as utilization of maximum supported archive size.

borg info ssh://u123456@u123456.your-storagebox.de:23/./backups/jobsitehr
Archive name: 2022-06-29T11:00-srv
Archive fingerprint: b2f1beac2bd553b34e06358afa45a3c1689320d39163890c5bbbd49125f00fe5
Comment:
Hostname: myhostname
Username: root
Time (start): Thu, 2022-06-29 11:03:07
Time (end): Thu, 2022-06-29 11:03:13
Duration: 5.66 seconds
Number of files: 17037
Command line: /usr/sbin/borg create /path/to/repo::2022-06-29T11:00-srv /srv
Utilization of max. archive size: 0%
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
This archive:               12.53 GB             12.49 GB              1.62 kB
All archives:              121.82 TB            112.41 TB            215.42 GB

                       Unique chunks         Total chunks
Chunk index:                 1015213            626934122

borg list

This command lists the contents of a repository or an archive.

# borg list ssh://u123456@u123456.your-storagebox.de:23/./backups/jobsitehr
borg list --format '{archive}{NL}' /path/to/repo
ArchiveFoo
ArchiveBar
...

# {VAR:NUMBER} - pad to NUMBER columns.
# Strings are left-aligned, numbers are right-aligned.
# Note: time columns except ``isomtime``, ``isoctime`` and ``isoatime`` cannot be padded.
$ borg list --format '{archive:36} {time} [{id}]{NL}' /path/to/repo
ArchiveFoo                           Thu, 2021-12-09 10:22:28 [0b8e9a312bef3f2f6e2d0fc110c196827786c15eba0188738e81697a7fa3b274]
$ borg list --format '{mode} {user:6} {group:6} {size:8} {mtime} {path}{extra}{NL}' /path/to/repo::ArchiveFoo
-rw-rw-r-- user   user       1024 Thu, 2021-12-09 10:22:17 file-foo
...

> The next step is to automate Borg backups with Cron