adminer/zrepl_patched
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

replication + endpoint: replication guarantees: guarantee_{resumability,incremental,nothing}

Browse Source 
This commit

- adds a configuration in which no step holds, replication cursors, etc. are created
- removes the send.step_holds.disable_incremental setting
- creates a new config option `replication` for active-side jobs
- adds the replication.protection.{initial,incremental} settings, each
  of which can have values
    - `guarantee_resumability`
    - `guarantee_incremental`
    - `guarantee_nothing`
  (refer to docs/configuration/replication.rst for semantics)

The `replication` config from an active side is sent to both endpoint.Sender and endpoint.Receiver
for each replication step. Sender and Receiver then act accordingly.

For `guarantee_incremental`, we add the new `tentative-replication-cursor` abstraction.
The necessity for that abstraction is outlined in https://github.com/zrepl/zrepl/issues/340.

fixes https://github.com/zrepl/zrepl/issues/340
This commit is contained in:
Christian Schwarz
2020-06-27 23:53:33 +02:00
parent 27673a23e9
commit 30cdc1430e
36 changed files with 1503 additions and 757 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/configuration/overview.rst
+29 -27
View File
@@ -132,23 +132,10 @@ The following high-level steps take place during replication and can be monitore
The idea behind the execution order of replication steps is that if the sender snapshots all filesystems simultaneously at fixed intervals, the receiver will have all filesystems snapshotted at time ``T1`` before the first snapshot at ``T2 = T1 + $interval`` is replicated.
Placeholder Filesystems
^^^^^^^^^^^^^^^^^^^^^^^
.. _replication-placeholder-property:
**Placeholder filesystems** on the receiving side are regular ZFS filesystems with the placeholder property ``zrepl:placeholder=on``.
Placeholders allow the receiving side to mirror the sender's ZFS dataset hierarchy without replicating every filesystem at every intermediary dataset path component.
Consider the following example: ``S/H/J`` shall be replicated to ``R/sink/job/S/H/J``, but neither ``S/H`` nor ``S`` shall be replicated.
ZFS requires the existence of ``R/sink/job/S`` and ``R/sink/job/S/H`` in order to receive into ``R/sink/job/S/H/J``.
Thus, zrepl creates the parent filesystems as placeholders on the receiving side.
If at some point ``S/H`` and ``S`` shall be replicated, the receiving side invalidates the placeholder flag automatically.
The ``zrepl test placeholder`` command can be used to check whether a filesystem is a placeholder.
ZFS Background Knowledge
^^^^^^^^^^^^^^^^^^^^^^^^
This section gives some background knowledge about ZFS features that zrepl uses to guarantee that
**incremental replication is always possible and that started replication steps can always be resumed if they are interrupted.**
This section gives some background knowledge about ZFS features that zrepl uses to provide guarantees for a replication filesystem.
Specifically, zrepl guarantees by default that **incremental replication is always possible and that started replication steps can always be resumed if they are interrupted.**
**ZFS Send Modes & Bookmarks**
ZFS supports full sends (``zfs send fs@to``) and incremental sends (``zfs send -i @from fs@to``).
@@ -166,41 +153,56 @@ An incremental send can only be resumed if ``@to`` still exists *and* either ``@
**ZFS Holds**
ZFS holds prevent a snapshot from being deleted through ``zfs destroy``, letting the destroy fail with a ``datset is busy`` error.
Holds are created and referred to by a user-defined *tag*. They can be thought of as a named, persistent lock on the snapshot.
Holds are created and referred to by a *tag*. They can be thought of as a named, persistent lock on the snapshot.
.. _zrepl-zfs-abstractions:
ZFS Abstractions Managed By zrepl
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
With the background knowledge from the previous paragraph, we now summarize the different on-disk ZFS objects that zrepl manages to provide its functionality.
.. _replication-placeholder-property:
**Placeholder filesystems** on the receiving side are regular ZFS filesystems with the placeholder property ``zrepl:placeholder=on``.
Placeholders allow the receiving side to mirror the sender's ZFS dataset hierarchy without replicating every filesystem at every intermediary dataset path component.
Consider the following example: ``S/H/J`` shall be replicated to ``R/sink/job/S/H/J``, but neither ``S/H`` nor ``S`` shall be replicated.
ZFS requires the existence of ``R/sink/job/S`` and ``R/sink/job/S/H`` in order to receive into ``R/sink/job/S/H/J``.
Thus, zrepl creates the parent filesystems as placeholders on the receiving side.
If at some point ``S/H`` and ``S`` shall be replicated, the receiving side invalidates the placeholder flag automatically.
The ``zrepl test placeholder`` command can be used to check whether a filesystem is a placeholder.
.. _replication-cursor-and-last-received-hold:
Guaranteeing That Incremental Sends Are Always Possible
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
**Replication cursor** bookmark and **last-received-hold** are managed by zrepl to ensure that future replications can always be done incrementally.
The **replication cursor** bookmark and **last-received-hold** are managed by zrepl to ensure that future replications can always be done incrementally.
The replication cursor is a send-side bookmark of the most recent successfully replicated snapshot,
and the last-received-hold is a hold of that snapshot on the receiving side.
Both are moved aomically after the receiving side has confirmed that a replication step is complete.
Both are moved atomically after the receiving side has confirmed that a replication step is complete.
The replication cursor has the format ``#zrepl_CUSOR_G_<GUID>_J_<JOBNAME>``.
The last-received-hold tag has the format ``zrepl_last_received_J_<JOBNAME>``.
Encoding the job name in the names ensures that multiple sending jobs can replicate the same filesystem to different receivers without interference.
.. _tentative-replication-cursor-bookmarks:
**Tentative replication cursor bookmarks** are short-lived boomkarks that protect the atomic moving-forward of the replication cursor and last-received-hold (see :issue:`this issue <340>`).
They are only necessary if step holds are not used as per the :ref:`replication.protection <replication-option-protection>` setting.
The tentative replication cursor has the format ``#zrepl_CUSORTENTATIVE_G_<GUID>_J_<JOBNAME>``.
The ``zrepl zfs-abstraction list`` command provides a listing of all bookmarks and holds managed by zrepl.
.. _step-holds-and-bookmarks:
Guaranteeing That Sends Are Always Resumable
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. _step-holds:
**Step holds** are zfs holds managed by zrepl to ensure that a replication step can always be resumed if it is interrupted, e.g., due to network outage.
zrepl creates step holds before it attempts a replication step and releases them after the receiver confirms that the replication step is complete.
For an initial replication ``full @initial_snap``, zrepl puts a zfs hold on ``@initial_snap``.
For an incremental send ``@from -> @to``, zrepl puts a zfs hold on both ``@from`` and ``@to``.
Note that ``@from`` is not strictly necessary for resumability -- a bookmark on the sending side would be sufficient --, but size-estimation in currently used OpenZFS versions only works if ``@from`` is a snapshot.
The hold tag has the format ``zrepl_STEP_J_<JOBNAME>``.
A job only ever has one active send per filesystem.
Thus, there are never more than two step holds for a given pair of ``(job,filesystem)``.
**Step bookmarks** are zrepl's equivalent for holds on bookmarks (ZFS does not support putting holds on bookmarks).
They are intended for a situation where a replication step uses a bookmark ``#bm`` as incremental ``from`` that is not managed by zrepl.
They are intended for a situation where a replication step uses a bookmark ``#bm`` as incremental ``from`` where ``#bm`` is not managed by zrepl.
To ensure resumability, zrepl copies ``#bm`` to step bookmark ``#zrepl_STEP_G_<GUID>_J_<JOBNAME>``.
If the replication is interrupted and ``#bm`` is deleted by the user, the step bookmark remains as an incremental source for the resumable send.
Note that zrepl does not yet support creating step bookmarks because the `corresponding ZFS feature for copying bookmarks <https://github.com/openzfs/zfs/pull/9571>`_ is not yet widely available .
docs/configuration/replication.rst
+47
View File
@@ -0,0 +1,47 @@
.. include:: ../global.rst.inc
Replication Options
===================
::
jobs:
- type: push
filesystems: ...
replication:
protection:
initial: guarantee_resumability # guarantee_{resumability,incremental,nothing}
incremental: guarantee_resumability # guarantee_{resumability,incremental,nothing}
...
.. _replication-option-protection:
``protection`` option
--------------------------
The ``protection`` variable controls the degree to which a replicated filesystem is protected from getting out of sync through a zrepl pruner or external tools that destroy snapshots.
zrepl can guarantee :ref:`resumability <step-holds>` or just :ref:`incremental replication <replication-cursor-and-last-received-hold>`.
``guarantee_resumability`` is the **default** value and guarantees that a replication step is always resumable and that incremental replication will always be possible.
The implementation uses replication cursors, last-received-hold and step holds.
``guarantee_incremental`` only guarantees that incremental replication will always be possible.
If a step ``from -> to`` is interrupted and its `to` snapshot is destroyed, zrepl will remove the half-received ``to``'s resume state and start a new step ``from -> to2``.
The implementation uses replication cursors, tentative replication cursors and last-received-hold.
``guarantee_nothing`` does not make any guarantees with regards to keeping sending and receiving side in sync.
No bookmarks or holds are created to protect sender and receiver from diverging.
**Tradeoffs**
Using ``guarantee_incremental`` instead of ``guarantee_resumability`` obviously removes the resumability guarantee.
This means that replication progress is no longer monotonic which might lead to a replication setup that never makes progress if mid-step interruptions are too frequent (e.g. frequent network outages).
However, the advantage and :issue:`reason for existence <288>` of the ``incremental`` mode is that it allows the pruner to delete snapshots of interrupted replication steps
which is useful if replication happens so rarely (or fails so frequently) that the amount of disk space exclusively referenced by the step's snapshots becomes intolerable.
.. NOTE::
When changing this flag, obsoleted zrepl-managed bookmarks and holds will be destroyed on the next replication step that is attempted for each filesystem.
docs/configuration/sendrecvoptions.rst
-20
View File
@@ -16,8 +16,6 @@ Send Options
filesystems: ...
send:
encrypted: true
step_holds:
disable_incremental: false
...
:ref:`Source<job-source>` and :ref:`push<job-push>` jobs have an optional ``send`` configuration section.
@@ -36,24 +34,6 @@ Filesystems matched by ``filesystems`` that are not encrypted are not sent and w
If ``encryption=false``, zrepl expects that filesystems matching ``filesystems`` are not encrypted or have loaded encryption keys.
.. _job-send-option-step-holds-disable-incremental:
``step_holds.disable_incremental`` option
-----------------------------------------
The ``step_holds.disable_incremental`` variable controls whether the creation of :ref:`step holds <step-holds-and-bookmarks>` should be disabled for incremental replication.
The default value is ``false``.
Disabling step holds has the disadvantage that steps :ref:`might not be resumable <step-holds-and-bookmarks>` if interrupted.
Non-resumability means that replication progress is no longer monotonic which might result in a replication setup that never makes progress if mid-step interruptions are too frequent (e.g. frequent network outages).
However, the advantage and :issue:`reason for existence <288>` of this flag is that it allows the pruner to delete snapshots of interrupted replication steps
which is useful if replication happens so rarely (or fails so frequently) that the amount of disk space exclusively referenced by the step's snapshots becomes intolerable.
.. NOTE::
When setting this flag to ``true``, existing step holds for the job will be destroyed on the next replication attempt.
.. _job-recv-options:
Recv Options