diff --git a/README.md b/README.md index b7fd784..dd329aa 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,8 @@ Check out the *Coding Workflow* section below for details. * Make sure your distro is compatible with the paths in `docs/installation.rst`. * Ship a default config that adheres to your distro's `hier` and logging system. * Ship a service manager file and _please_ try to upstream it to this repository. - * `dist/systemd` contains a Systemd unit template + * `dist/systemd` contains a Systemd unit template. +* Ship other material provided in `./dist`, e.g. in `/usr/share/zrepl/`. * Use `make release ZREPL_VERSION='mydistro-1.2.3_1'` * Your distro's name and any versioning supplemental to zrepl's (e.g. package revision) should be in this string * Make sure you are informed about new zrepl versions, e.g. by subscribing to GitHub's release RSS feed. @@ -62,6 +63,7 @@ Make sure to develop an understanding how zrepl is typically used by studying th │   ├── streamrpcconfig # abstraction for configuration of go-streamrpc │   └── transport # transports implementation ├── docs # sphinx-based documentation +├── dist # supplemental material for users & package maintainers │   ├── **/*.rst # documentation in reStructuredText │   ├── sphinxconf │   │   └── conf.py # sphinx config (see commit 445a280 why its not in docs/) diff --git a/docs/changelog.rst b/docs/changelog.rst index 672cd61..5230fcb 100644 --- a/docs/changelog.rst +++ b/docs/changelog.rst @@ -94,10 +94,15 @@ Changes * |feature| ``zrepl status`` for live-updating replication progress (it's really cool!) * |feature| :issue:`67`: Expose `Prometheus `_ metrics via HTTP (:ref:`config docs `) + * Compatible Grafana dashboard shipping in ``dist/grafana`` + * |break_config| Logging outlet types must be specified using the ``type`` instead of ``outlet`` key * |break| :issue:`53`: CLI: ``zrepl control *`` subcommands have been made direct subcommands of ``zrepl *`` * |bugfix| Goroutine leak on ssh transport connection timeouts * |bugfix| :issue:`81` :issue:`77` : handle failed accepts correctly (``source`` job) +* |bugfix| :issue:`100`: fix incompatibility with ZoL 0.8 +* |feature| :issue:`115`: logging: configurable syslog facility +* |feature| Systemd unit file in ``dist/systemd`` .. |lastrelease| replace:: 0.0.3 @@ -107,8 +112,3 @@ Previous Releases .. NOTE:: Due to limitations in our documentation system, we only show the changelog since the last release and the time this documentation is built. For the changelog of previous releases, use the version selection in the hosted version of these docs at `zrepl.github.io `_. - - - - -W diff --git a/docs/configuration/jobs.rst b/docs/configuration/jobs.rst index 077784f..9a2ea89 100644 --- a/docs/configuration/jobs.rst +++ b/docs/configuration/jobs.rst @@ -14,7 +14,7 @@ Job Types & Replication Overview & Terminology ---------------------- -A *job* is the unit of activity tracked by the zrepl daemon and configured in the |mainconfig|. +A *job* is the unit of activity tracked by the zrepl daemon. Every job has a unique ``name``, a ``type`` and type-dependent fields which are documented on this page. Replication always happens between a pair of jobs: one is the **active side**, and one the **passive side**. @@ -91,7 +91,8 @@ The following steps take place during replication and can be monitored using the * Perform replication steps in the following order: Among all filesystems with pending replication steps, pick the filesystem whose next replication step's snapshot is the oldest. - * After a successful replication step, update the replication cursor bookmark (see below) + * Create placeholder filesystems on the receiving side to mirror the dataset paths on the sender to ``root_fs/${client_identity}``. + * After a successful replication step, update the replication cursor bookmark (see below). The idea behind the execution order of replication steps is that if the sender snapshots all filesystems simultaneously at fixed intervals, the receiver will have all filesystems snapshotted at time ``T1`` before the first snapshot at ``T2 = T1 + $interval`` is replicated. @@ -102,6 +103,14 @@ It is a bookmark of the most recent successfully replicated snapshot to the rece It is is used by the :ref:`not_replicated ` keep rule to identify all snapshots that have not yet been replicated to the receiving side. Regardless of whether that keep rule is used, the bookmark ensures that replication can always continue incrementally. +**Placeholder filesystems** on the receiving side are regular ZFS filesystems with the placeholder property ``zrepl:placeholder``. +Placeholders allow the receiving side to mirror the sender's ZFS dataset hierachy without replicating every filesystem at every intermediary dataset path component. +Consider the following example: ``S/H/J`` shall be replicated to ``R/sink/job/S/H/J``, but neither ``S/H`` nor ``S`` shall be replicated. +ZFS requires the existence of ``R/sink/job/S`` and ``R/sink/job/S/H`` in order to receive into ``R/sink/job/S/H/J``. +Thus, zrepl creates the parent filesystems as placeholders on the receiving side. +If at some point ``S/H`` and ``S`` shall be replicated, the receiving side invalidates the placeholder flag automatically. +The ``zrepl test placeholder`` command can be used to check whether a filesystem is a placeholder. + .. ATTENTION:: Currently, zrepl does not replicate filesystem properties. diff --git a/docs/configuration/logging.rst b/docs/configuration/logging.rst index a9077bb..7a5bb5a 100644 --- a/docs/configuration/logging.rst +++ b/docs/configuration/logging.rst @@ -7,7 +7,7 @@ Logging zrepl uses structured logging to provide users with easily processable log messages. -Logging outlets are configured in the ``global`` section of the |mainconfig|. +Logging outlets are configured in the ``global`` section of the config file. :: diff --git a/docs/configuration/monitoring.rst b/docs/configuration/monitoring.rst index f0e97dc..138b968 100644 --- a/docs/configuration/monitoring.rst +++ b/docs/configuration/monitoring.rst @@ -5,7 +5,7 @@ Monitoring ========== -Monitoring endpoints are configured in the ``global.monitoring`` section of the |mainconfig|. +Monitoring endpoints are configured in the ``global.monitoring`` section of the config file. .. _monitoring-prometheus: diff --git a/docs/global.rst.inc b/docs/global.rst.inc index bbe9204..0074484 100644 --- a/docs/global.rst.inc +++ b/docs/global.rst.inc @@ -1,3 +1,4 @@ .. global list of substitutions -.. |mainconfig| replace:: :ref:`main configuration file ` +.. + .. |macroname| replace:: macro expansion diff --git a/docs/index.rst b/docs/index.rst index 6fe0e7e..8fc5b50 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -77,7 +77,8 @@ We are happy about any help we can get! * These docs live in the ``docs/`` subdirectory * Document any non-obvious / confusing / plain broken behavior you encounter when setting up zrepl for the first time -* Check the *Issues* and *Projects* sections for things to do +* Check the *Issues* and *Projects* sections for things to do. + The `good first issues `_ and `docs `_ are suitable starting points. .. admonition:: Development Workflow :class: note diff --git a/docs/installation.rst b/docs/installation.rst index a9086ca..a3c8cc5 100644 --- a/docs/installation.rst +++ b/docs/installation.rst @@ -80,8 +80,6 @@ Either way, all build results are located in the ``artifacts/`` directory. It is your job to install the apropriate binary in the zrepl users's ``$PATH``, e.g. ``/usr/local/bin/zrepl``. Otherwise, the examples in the :ref:`tutorial` may need to be adjusted. -.. _mainconfigfile: - What next? ---------- diff --git a/docs/tutorial.rst b/docs/tutorial.rst index 0d639b6..c40400d 100644 --- a/docs/tutorial.rst +++ b/docs/tutorial.rst @@ -171,7 +171,7 @@ The Linux equivalent might look like this: :: Summary ------- -Congratulations, you have a working pull backup. Where to go next? +Congratulations, you have a working push backup. Where to go next? * Read more about :ref:`configuration format, options & job types ` * Configure :ref:`logging ` \& :ref:`monitoring `.