- Creates new skill to automatically update docs/supporters.rst
- Extracts contributors from git history and changelog.rst
- Supports time ranges (--since) and commit ranges (v0.6.0..v0.7.0)
- Handles new contributors and recurring contributors (moves them up)
- Records processed range in RST comment
- Integrated into release workflow in README.md
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This PR simplifies how we build and publish docs:
- **Publish from `master` branch, retire `stable` branch.** The `stable`
branch was a manual step in the release process and often out of date.
Docs are now built and published directly from `master`.
Release-specific docs are available in the `zrepl-noarch.tar` asset on
each GitHub release.
- **build dependencies**: use `uv` for dependency management
- **zrepl.github.io: retire multi-version docs**: before this PR we used
`sphinx-multiversion` to publish multiple docs versions to
`zrepl.github.io`. This was never worth the pain, so, this PR removes it
in order to simplify stuff. Old docs are available in the GitHub
releases, and the docs now have a version dropdown that links there for
a hand-curated set of versions.
- **GitHub pages repo checkout**: use HTTPS because that's what I use
these days for all things GitHub. Switch CircleCI to a fine-grained PAT.
Refs
- docs bug https://github.com/zrepl/zrepl/issues/895
- links to config examples should work again after this PR
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This PR allows for the distribution of zrepl job definitions across
multiple YAML files that are included from the main config.
```
global: ...
include:
- ./zrepl.yml.d
- /opt/zrepl.yml
- ...
```
Refer to the docs changes for details.
Co-authored-by: Christian Schwarz <me@cschwarz.com>
Added installation instructions for openSUSE as discussed in #776. Sorry
for the delay.
I decided against a solution involving lsb_release, since openSUSE
MicroOS, being a minimal OS, ships without it. Hope the edit is OK.
---------
Co-authored-by: Christian Schwarz <me@cschwarz.com>
**Problem 1**: Users following the EasyRSA script in the docs with their
more recent distro version of EasyRSA find that the generated certs no
longer include a subject-alternative-name (SAN), which is required as of
Go 1.15 / zrepl 0.3.0.
**Problem 2**: Newer openssl versions with the old EasyRSA prompt the
user for a CA password in `init-pki` and depending on EasyRSA version
there are other interactive prompts at later commands in the script.
**Changes**:
- upgrade the pinned EasyRSA version to the latest release
- use `--auto-san` option to make EasyRSA set the SAN. This has the
benefit of also working for IP addresses.
I checked the easy-rsa git history and `--subject-alt-name` appears to
have been present in all 3.X releases, so, adding the flag shouldn't
break users of older EasyRSA 3.X.
(And these are exemplary instructions anyway.)
refs
- fixes https://github.com/zrepl/zrepl/issues/900
- obsoletes https://github.com/zrepl/zrepl/pull/906
- obsoletes https://github.com/zrepl/zrepl/pull/877
Co-authored-by: Andrew Lorimer <andrew@lorimer.id.au>
This PR adds a new field optional field `timestamp_location` that allows
the user to specify a timezone different than the default UTC for use in
the snapshot suffix.
I took @mjasnik 's PR https://github.com/zrepl/zrepl/pull/785 and
refactored+extended it as follows:
* move all formatting logic into its own package
* disallow `dense` and `human` with formats != UTC to protect users from
stupidity
* document behavior more clearly
* regression test for existing users
Because some jobs add client identity to root_fs and other jobs don't do
that,
we can't reliable detect overlapping of filesystems. And and the same
time we
need an ability to use equal or overlapped root_fs for different jobs.
For
instance see this config:
```
- name: "zdisk"
type: "sink"
root_fs: "zdisk/zrepl"
serve:
type: "local"
listener_name: "zdisk"
```
and
```
- name: "remote-to-zdisk"
type: "pull"
connect:
type: "tls"
root_fs: "zdisk/zrepl/remote"
```
As you can see, two jobs have overlapped root_fs, but actually datasets
are not
overlapped, because job `zdisk` save everything under
`zdisk/zrepl/localhost`,
because it adds client identity. So they actually use two different
filesystems:
`zdisk/zrepl/localhost` and `zdisk/zrepl/remote`. And we can't detect
this
situation during config check. So let's just remove this check, because
it's
admin's duty to configure correct root_fs's.
---------
Co-authored-by: Christian Schwarz <me@cschwarz.com>
The sphinxcontrib-versioning seems unmaintainted and I can't
get the fork that we used before this PR working on Python 3.10.
The situation wrt maintenance doesn't seem much better for
sphinx-multiversion, but, at least I could get it to work
with current sphinx versions.
The main problem with sphinx-multiversion is that it doesn't render
anything at `/`. I.e., `https://zrepl.github.io/configuration.html` will
be 404.
That's different from `sphinxcontrib-versioning`, and thus switching
to sphinx-multiversion would break URLs.
We host on GitHub pages and don't control the webserver,
so, we can't use webserver-level redirects to keep the URLs working.
We could create JS-level redirects, or `http-equiv`, but that's ugly as
well.
The simplest solution was to fork sphinx-multiversion and hard-code
zrepl's specific needs into that fork.
The fork is based off v0.2.4 and pinned via requirements.txt.
Here are its unique commits:
https://github.com/Holzhaus/sphinx-multiversion/compare/master...zrepl:sphinx-multiversion:zrepl
We should revisit `sphinx-polyversion` in the future once its docs
improve.
See
https://github.com/Holzhaus/sphinx-multiversion/issues/88#issuecomment-1606221194
This PR updates the various Python packages, as I couldn't get
sphinx-multiversion to work with the (very old) versions that were
pinned in `requirements.txt` prior to this PR.
This PR's `requirements.txt` is from a clean Python 3.10 venv on Ubuntu
22.10 after running
```
pip install sphinx sphinx-rtd-theme
pip install 'git+https://github.com/zrepl/sphinx-multiversion/@52c915d7ad898d9641ec48c8bbccb7d4f079db93#egg=sphinx_multiversion'
```
For this kind of debugging, we switched to env vars a while ago.
For example, ZREPL_RPC_DEBUG.
I don't think we have a substitute for the RPCLog stuff.
However, NetConnLogger is still in the codebase.
obsoletes https://github.com/zrepl/zrepl/pull/661
Before this change, resuming from an unencrypted dataset with
send.raw=true specified wouldn't work with zrepl due to overly
restrictive resume token checking.
An initial PR to fix this was made in https://github.com/zrepl/zrepl/pull/503
but it didn't address the core of the problem.
The core of the problem was that zrepl assumed that if a resume token
contained `rawok=true, compressok=true`, the resulting send would be
encrypted. But if the sender dataset was unencrypted, such a resume would
actually result in an unencrypted send.
Which could be totally legitimate but zrepl failed to recognize that.
BACKGROUND
==========
The following snippets of OpenZFS code are insightful regarding how the
various ${X}ok values in the resume token are handled:
- https://github.com/openzfs/zfs/blob/6c3c5fcfbe27d9193cd131753cc7e47ee2784621/module/zfs/dmu_send.c#L1947-L2012
- https://github.com/openzfs/zfs/blob/6c3c5fcfbe27d9193cd131753cc7e47ee2784621/module/zfs/dmu_recv.c#L877-L891
- https://github.com/openzfs/zfs/blob/6c3c5fc/lib/libzfs/libzfs_sendrecv.c#L1663-L1672
Basically, some zfs send flags make the DMU send code set some DMU send
stream featureflags, although it's not a pure mapping, i.e, which DMU
send stream flags are used depends somewhat on the dataset (e.g., is it
encrypted or not, or, does it use zstd or not).
Then, the receiver looks at some (but not all) feature flags and maps
them to ${X}ok dataset zap attributes.
These are funnelled back to the sender 1:1 through the resume_token.
And the sender turns them into lzc flags.
As an example, let's look at zfs send --raw.
if the sender requests a raw send on an unencrypted dataset, the send
stream (and hence the resume token) will not have the raw stream
featureflag set, and hence the resume token will not have the rawok
field set. Instead, it will have compressok, embedok, and depending
on whether large blocks are present in the dataset, largeblockok set.
WHAT'S ZREPL'S ROLE IN THIS?
============================
zrepl provides a virtual encrypted sendflag that is like `raw`,
but further ensures that we only send encrypted datasets.
For any other resume token stuff, it shoudn't do any checking,
because it's a futile effort to keep up with ZFS send/recv features
that are orthogonal to encryption.
CHANGES MADE IN THIS COMMIT
===========================
- Rip out a bunch of needless checking that zrepl would do during
planning. These checks were there to give better error messages,
but actually, the error messages created by the endpoint.Sender.Send
RPC upon send args validation failure are good enough.
- Add platformtests to validate all combinations of
(Unencrypted/Encrypted FS) x (send.encrypted = true | false) x (send.raw = true | false)
for cases both non-resuming and resuming send.
Additional manual testing done:
1. With zrepl 0.5, setup with unencrypted dataset, send.raw=true specified, no send.encrypted specified.
2. Observe that regular non-resuming send works, but resuming doesn't work.
3. Upgrade zrepl to this change.
4. Observe that both regular and resuming send works.
closes https://github.com/zrepl/zrepl/pull/613
There were and still is too many words. It's a very white paper vibe.
Docs needs to be more brief, exact, and on-point.
closes https://github.com/zrepl/zrepl/pull/618
Previously with unordered list, and 'are considered'
left if unsure whether one or all files are 'considered'.
In reality, the first valid is used, so an ordered list and
perhaps better wording communicates this fact.
refs https://github.com/zrepl/zrepl/pull/618
Config:
```
- type: push
...
conflict_resolution:
initial_replication: most_recent | all | fali
```
The ``initial_replication`` option determines which snapshots zrepl
replicates if the filesystem has not been replicated before.
If ``most_recent`` (the default), the initial replication will only
transfer the most recent snapshot, while ignoring previous snapshots.
If all snapshots should be replicated, specify ``all``.
Use ``fail`` to make replication of the filesystem fail in case
there is no corresponding fileystem on the receiver.
Code-Level Changes, apart from the obvious:
- Rework IncrementalPath()'s return signature.
Now returns an error for initial replications as well.
- Rename & rework it's consumer, resolveConflict().
Co-authored-by: Graham Christensen <graham@grahamc.com>
Fixes https://github.com/zrepl/zrepl/issues/550
Fixes https://github.com/zrepl/zrepl/issues/187
Closes https://github.com/zrepl/zrepl/pull/592
dependabot[bot]