rsync with Email Notification

backup

This post describes how an email can be sent automatically from a rsync backup script to inform the user about the backup status. This is helpful to ensure that the backup is executed successfully and that the necessary countermeasures can be taken in case of an error. An error may occur if the storage space for the backup target is too small, not accessible, etc. It seems obvious that for personal use nobody would check the backup log every day. Therefore, it must be possible to quickly interpret the backup status via email. This can be accomplished by writing the status directly in the subject of the email e.g.: SUCCESS or ERROR. For better interpretability the rsync log is written into the body of the email. This should only be done if the email server is under your own control. Otherwise the email provider that might also be the cloud provider can still see the structure of your backup. e.g.: Microsoft or Google.

Requirements

The following points must be ensured before configuring the email notification and creating the backup script.

  • The sources to be backed up needs to be mounted
  • The target to store the backup needs to be mounted
  • Access to a SMTP Server

The target to store the backup might be a local HDD connected with USB or in my case a cloud service. It is highly recommended to encrypt the data locally if the backup is stored on a cloud service like Dropbox, OneDrive, Google Drive or myCloud. The general setup with encryption was threated in detail in a previous post: Encrypted Cloud Backup .

Setting up the Email Notification

This section describes briefly the setup and how emails can be sent in Linux over the terminal. This is practical to automatically send the status after the backup is completed. First, the necessary packages need to be installed. Mailutils is used for handling the emails via terminal and Postfix MTA is used to send the emails over an external SMTP server such as Gmail, Amazon SES or any other SMTP service. In my case I use my own SMTP server.

sudo apt-get install mailutils postfix libsasl2-2 libsasl2-modules

Once the installation is finished, the Postfix configuration is open with the text editor.

sudo nano /etc/postfix/main.cf

First the hostname is set to the FQDN as configured during the setup of Postfix. Then the configuration of the external SMTP server is adapted.

# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_security_level=may

smtp_tls_CApath=/etc/ssl/certs
#smtp_tls_security_level=may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_un>
myhostname = hostname.domain.ch
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
mydestination = $myhostname, backup-server, localhost.localdomain, , localhost
relayhost = [relayhost-url.ch]:465
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all

smtp_sasl_password_maps = hash:/etc/postfix/smtp_auth
sender_canonical_maps = hash:/etc/postfix/sender_canonical

smtp_sasl_auth_enable = yes
smtp_sasl_security_options = noanonymous
smtp_tls_security_level = encrypt
smtp_use_tls = yes
smtp_tls_wrappermode = yes

Next the credentials for the external SMTP server authentification are stored in the file smtp_auth. The file can be opened with the following command.

sudo nano /etc/postfix/smtp_auth

The SMTP server URL and port is added as well as the user and password.

[relayhost-url.ch]:465 address@email.ch:password

The hash db file for Postfix is created with the postmap command.

sudo postmap /etc/postfix/smtp_auth

The Canonical senders are specified in the file sender_canonical.

sudo nano /etc/postfix/sender_canonical

The linux users are assigned to the sender email address by the following lines.

ubuntu address@email.ch
root address@email.ch

and convertet into the hash db format by postmap.

sudo postmap /etc/postfix/sender_canonical

And finally Postfix is restarted.

sudo service postfix reload

A test email can be send with the following command to receiver-address@email.ch.

mail -s 'Test message header' receiver-address@email.ch <<< 'Test message body'

With the following command it can be verified if the email was sent successfully or not.

tail -f /var/log/mail.log

mail + rsync Script

Finally, the mail command is integrated in the rsync backup script. The script is saved under /usr/local/bin/rsync_script. Different SOURCES can be specified at the beginning of the script. The TARGET is in my case the mounted and decrypted backup folder under my home directory. In case the backup succeeds the email subject contains SUCCESS and otherwise ERROR. Moreover, the body of the email contains a description of the rsync log and the detailed rsync report. The sender name and organization may be adapted with the -a option of the mail command.

#!/bin/bash

# Specifications
SOURCES=("/media/source1 /media/source2" )
TARGET="/home/ubuntu/secured_backup"

EMAIL_RECIPIENT="receiver-address@email.ch"
LOGFILE="rsync.log"

rm $LOGFILE

# Perform the backup using rsync
rsync -av --exclude '@eaDir' \
        --exclude ".*" \
        --no-perms \
        --no-group \
        --no-owner \
        --inplace \
        --exclude=[#]recycle \
        --log-file=$LOGFILE \
        --log-file-format="File changed! %f %i" \
        $SOURCES $TARGET
        RESULT=$?

if [[ $RESULT -gt 0 ]]
then
  MAIL_SUBJECT="ERROR - BKPSRV - Rsync Report"
else
  MAIL_SUBJECT="SUCCESS - BKPSRV - Rsync Report"
fi

# Send the email
echo "Mail Subject: " $MAIL_SUBJECT
(echo -e "$MAIL_SUBJECT\n\nrsync legend:\n"; \
        cat rsync_man.txt; \
        echo -e "\nThe backup on the server has terminated with the following message:\n"; \
        cat $LOGFILE) | \
        mail \
        -a "From: Sender Name <address@email.ch>"  \
        -a "Organization: Organization" \
        -s "$MAIL_SUBJECT" \
        $EMAIL_RECIPIENT

Conclusions

The presented automatic email notification about the rsync backup helps to keep track of the backups performed and informs the user if something went wrong. This reduces the response time and prevents possible data loss due to a non-functioning backup system.