docs: add backup-to-external-disk quick-start guide and convert existing tutorial to quick-start guide

refs #219
fixes #329
This commit is contained in:
Christian Schwarz
2020-06-11 18:50:39 +02:00
parent 9a8d813d14
commit a827894274
15 changed files with 358 additions and 185 deletions
@@ -0,0 +1,36 @@
.. include:: ../global.rst.inc
.. _quickstart-backup-to-external-disk:
Local Snapshots + Offline Backup to an External Disk
====================================================
This config example shows how we can use zrepl to make periodic snapshots of our local workstation and back it up to a zpool on an external disk which we occassionally connect.
The local snapshots should be taken every 15 minutes for pain-free recovery from CLI disasters (``rm -rf /`` and the like).
However, we do not want to keep the snapshots around for very long because our workstation is a little tight on disk space.
Thus, we only keep one hour worth of high-resolution snapshots, then fade them out to one per hour for a day (24 hours), then one per day for 14 days.
At the end of each work day, we connect our external disk that serves as our workstation's local offline backup.
We want zrepl to inspect the filesystems and snapshots on the external pool, figure out which snapshots were created since the last time we connected the external disk, and use incremental replication to efficiently mirror our workstation to our backup disk.
Afterwards, we want to clean up old snapshots on the backup pool: we want to keep all snapshots younger than one hour, 24 for each hor of the first day, then 360 daily backups.
A few additional requirements:
* Snapshot creation and pruning on our workstation should happen in the background, without interaction from our side.
* However, we want to explicitly trigger replication via the command line.
* We want to use OpenZFS native encryption to protect our data on the external disk.
It is absolutely critical that only encrypted data leaves our workstation.
**zrepl should provide an easy config knob for this and prevent replication of unencrypted datasets to the external disk.**
* We want to be able to put off the backups for more than three weeks, i.e., longer than the lifetime of the automatically created snapshots on our workstation.
**zrepl should use bookmarks and holds to achieve this goal**.
* When we yank out the drive during replication and go on a long vacation, we do *not* want the partially replicated snapshot to stick around as it would hold on to too much disk space over time.
Therefore, we want zrepl to deviate from its :ref:`default step-hold behavior <step-holds-and-bookmarks>` and sacrifice resumability, but nonetheless retain the ability to do incremental replication once we return from our vacation.
**zrepl should provide an easy config knob to disable step holds for incremental replication**.
The following config snippet implements the setup described above.
You will likely want to customize some aspects mentioned in the top comment in the file.
.. literalinclude:: ../../config/samples/quickstart_backup_to_external_disk.yml
:ref:`Click here <quickstart-apply-config>` to go back to the quickstart guide.
@@ -0,0 +1,97 @@
.. include:: ../global.rst.inc
.. _quickstart-continuous-replication:
Continuous Backup of a Server
=============================
This config example shows how we can backup our ZFS-based server to another machine using a zrepl push job.
* Production server ``prod`` with filesystems to back up:
* ``zroot/var/db``
* ``zroot/usr/home`` and all its child filesystems
* **except** ``zroot/usr/home/paranoid`` belonging to a user doing backups themselves
* Backup server ``backups`` with
* Filesystem ``storage/zrepl/sink/prod`` + children dedicated to backups of ``prod``
Our backup solution should fulfill the following requirements:
* Periodically snapshot the filesystems on ``prod`` *every 10 minutes*
* Incrementally replicate these snapshots to ``storage/zrepl/sink/prod/*`` on ``backups``
* Keep only very few snapshots on ``prod`` to save disk space
* Keep a fading history (24 hourly, 30 daily, 6 monthly) of snapshots on ``backups``
* The network is untrusted - zrepl should use TLS to protect its communication and our data.
Analysis
--------
We can model this situation as two jobs:
* A **push job** on ``prod``
* Creates the snapshots
* Keeps a short history of local snapshots to enable incremental replication to ``backups``
* Connects to the ``zrepl daemon`` process on ``backups``
* Pushes snapshots ``backups``
* Prunes snapshots on ``backups`` after replication is complete
* A **sink job** on ``backups``
* Accepts connections & responds to requests from ``prod``
* Limits client ``prod`` access to filesystem sub-tree ``storage/zrepl/sink/prod``
Generate TLS Certificates
-------------------------
We use the :ref:`TLS client authentication transport <transport-tcp+tlsclientauth>` to protect our data on the wire.
To get things going quickly, we skip setting up a CA and generate two self-signed certificates as described :ref:`here <transport-tcp+tlsclientauth-2machineopenssl>`.
For convenience, we generate the key pairs on our local machine and distribute them using ssh:
.. code-block:: bash
:emphasize-lines: 6,13
openssl req -x509 -sha256 -nodes \
-newkey rsa:4096 \
-days 365 \
-keyout backups.key \
-out backups.crt
# ... and use "backups" as Common Name (CN)
openssl req -x509 -sha256 -nodes \
-newkey rsa:4096 \
-days 365 \
-keyout prod.key \
-out prod.crt
# ... and use "prod" as Common Name (CN)
ssh root@backups "mkdir /etc/zrepl"
scp backups.key backups.crt prod.crt root@backups:/etc/zrepl
ssh root@prod "mkdir /etc/zrepl"
scp prod.key prod.crt backups.crt root@prod:/etc/zrepl
Note that alternative transports exist, e.g. via :ref:`TCP without TLS <transport-tcp>` or :ref:`ssh <transport-ssh+stdinserver>`.
Configure server ``prod``
-------------------------
We define a **push job** named ``prod_to_backups`` in ``/etc/zrepl/zrepl.yml`` on host ``prod`` :
.. literalinclude:: ../../config/samples/quickstart_continuous_server_backup_sender.yml
.. _tutorial-configure-prod:
Configure server ``backups``
----------------------------
We define a corresponding **sink job** named ``sink`` in ``/etc/zrepl/zrepl.yml`` on host ``backups`` :
.. literalinclude:: ../../config/samples/quickstart_continuous_server_backup_receiver.yml
Go Back To Quickstart Guide
---------------------------
:ref:`Click here <quickstart-apply-config>` to go back to the quickstart guide.