|
|
@@ -7,7 +7,7 @@ Implementation Overview
|
|
|
|
|
|
|
|
|
|
|
|
Incomplete / under construction
|
|
|
|
Incomplete / under construction
|
|
|
|
|
|
|
|
|
|
|
|
The following design aspects may convince you that `zrepl` is superior to a hacked-together shell script solution.
|
|
|
|
The following design aspects may convince you that ``zrepl`` is superior to a hacked-together shell script solution.
|
|
|
|
|
|
|
|
|
|
|
|
Testability & Performance
|
|
|
|
Testability & Performance
|
|
|
|
-------------------------
|
|
|
|
-------------------------
|
|
|
@@ -23,12 +23,14 @@ software engineering.
|
|
|
|
RPC protocol
|
|
|
|
RPC protocol
|
|
|
|
------------
|
|
|
|
------------
|
|
|
|
|
|
|
|
|
|
|
|
While it is tempting to just issue a few `ssh remote 'zfs send ...' | zfs recv`, this has a number of drawbacks:
|
|
|
|
While it is tempting to just issue a few ``ssh remote 'zfs send ...' | zfs recv``, this has a number of drawbacks:
|
|
|
|
|
|
|
|
|
|
|
|
* The snapshot streams need to be compatible.
|
|
|
|
* The snapshot streams need to be compatible.
|
|
|
|
* Communication is still unidirectional. Thus, you will most likely
|
|
|
|
* Communication is still unidirectional. Thus, you will most likely
|
|
|
|
|
|
|
|
|
|
|
|
* either not take advantage of features such as *compressed send & recv*
|
|
|
|
* either not take advantage of features such as *compressed send & recv*
|
|
|
|
* or issue additional `ssh` commands in advance to figure out what features are supported on the other side.
|
|
|
|
* or issue additional ``ssh`` commands in advance to figure out what features are supported on the other side.
|
|
|
|
|
|
|
|
|
|
|
|
* Advanced logic in shell scripts is ugly to read, poorly testable and a pain to maintain.
|
|
|
|
* Advanced logic in shell scripts is ugly to read, poorly testable and a pain to maintain.
|
|
|
|
|
|
|
|
|
|
|
|
zrepl takes a different approach:
|
|
|
|
zrepl takes a different approach:
|
|
|
@@ -42,7 +44,7 @@ zrepl takes a different approach:
|
|
|
|
* No blank root shell access is given to the other side.
|
|
|
|
* No blank root shell access is given to the other side.
|
|
|
|
* Instead, an *authenticated* peer can *request* filesystem lists, snapshot streams, etc.
|
|
|
|
* Instead, an *authenticated* peer can *request* filesystem lists, snapshot streams, etc.
|
|
|
|
* Requests are then checked against job-specific ACLs, limiting a client to the filesystems it is actually allowed to replicate.
|
|
|
|
* Requests are then checked against job-specific ACLs, limiting a client to the filesystems it is actually allowed to replicate.
|
|
|
|
* The {{< zrepl-transport "transport mechanism" >}} is decoupled from the remaining logic, keeping it extensible.
|
|
|
|
* The :ref:`transport mechanism <transport>` is decoupled from the remaining logic, keeping it extensible.
|
|
|
|
|
|
|
|
|
|
|
|
Protocol Implementation
|
|
|
|
Protocol Implementation
|
|
|
|
~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
@@ -50,11 +52,11 @@ Protocol Implementation
|
|
|
|
zrepl implements its own RPC protocol.
|
|
|
|
zrepl implements its own RPC protocol.
|
|
|
|
This is mostly due to the fact that existing solutions do not provide efficient means to transport large amounts of data.
|
|
|
|
This is mostly due to the fact that existing solutions do not provide efficient means to transport large amounts of data.
|
|
|
|
|
|
|
|
|
|
|
|
Package [`github.com/zrepl/zrepl/rpc`](https://github.com/zrepl/zrepl/tree/master/rpc) builds a special-case handling around returning an `io.Reader` as part of a unary RPC call.
|
|
|
|
Package `github.com/zrepl/zrepl/rpc <https://github.com/zrepl/zrepl/tree/master/rpc>`_ builds a special-case handling around returning an ``io.Reader`` as part of a unary RPC call.
|
|
|
|
|
|
|
|
|
|
|
|
Measurements show only a single memory-to-memory copy of a snapshot stream is made using `github.com/zrepl/zrepl/rpc`, and there is still potential for further optimizations.
|
|
|
|
Measurements show only a single memory-to-memory copy of a snapshot stream is made using ``github.com/zrepl/zrepl/rpc``, and there is still potential for further optimizations.
|
|
|
|
|
|
|
|
|
|
|
|
Logging & Transparency
|
|
|
|
Logging & Transparency
|
|
|
|
----------------------
|
|
|
|
----------------------
|
|
|
|
|
|
|
|
|
|
|
|
zrepl comes with [rich, structured and configurable logging]({{< relref "configuration/logging.md" >}}), allowing administators to understand what the software is actually doing.
|
|
|
|
zrepl comes with :ref:`rich, structured and configurable logging <logging>`, allowing administators to understand what the software is actually doing.
|
|
|
|
Christian Schwarz