Compare commits

...

5 Commits

Author SHA1 Message Date
Christian Schwarz 2efbf19c62 REWORD ToString in log message 2020-02-25 15:44:23 +01:00
Christian Schwarz 3ff1966cab docs/installation: use && for early exit if build-in-docker step fails 2020-02-17 18:02:04 +01:00
Christian Schwarz a3842155c5 zrepl test filesystems: support snap job type 2020-02-17 18:02:04 +01:00
Christian Schwarz 02b3b4f80c fix some typos 2020-02-17 18:02:04 +01:00
Christian Schwarz 0882290595 README update
- donation links
- package overview
2020-02-17 18:02:04 +01:00
7 changed files with 28 additions and 16 deletions
+21 -9
View File
@@ -1,8 +1,9 @@
[![GitHub license](https://img.shields.io/github/license/zrepl/zrepl.svg)](https://github.com/zrepl/zrepl/blob/master/LICENSE)
[![Language: Go](https://img.shields.io/badge/language-Go-6ad7e5.svg)](https://golang.org/)
[![User Docs](https://img.shields.io/badge/docs-web-blue.svg)](https://zrepl.github.io)
[![Donate via PayPal](https://img.shields.io/badge/donate-paypal-yellow.svg)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=R5QSXJVYHGX96)
[![Donate via Patreon](https://img.shields.io/endpoint.svg?url=https%3A%2F%2Fshieldsio-patreon.herokuapp.com%2Fzrepl%2Fpledges&style=flat&color=yellow)](https://www.patreon.com/zrepl)
[![Donate via Liberapay](https://img.shields.io/liberapay/receives/zrepl.svg?logo=liberapay)](https://liberapay.com/zrepl/donate)
[![Donate via PayPal](https://img.shields.io/badge/donate-paypal-yellow.svg)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=R5QSXJVYHGX96)
[![Twitter](https://img.shields.io/twitter/url/https/github.com/zrepl/zrepl.svg?style=social)](https://twitter.com/intent/tweet?text=Wow:&url=https%3A%2F%2Fgithub.com%2Fzrepl%2Fzrepl)
# zrepl
@@ -23,6 +24,7 @@ zrepl is a one-stop ZFS backup & replication solution.
If so, think of an expressive configuration example.
2. Think of at least one use case that generalizes from your concrete application.
3. Open an issue on GitHub with example conf & use case attached.
4. **Optional**: [Post a bounty](https://www.bountysource.com/teams/zrepl) on the issue, or [contact Christian Schwarz](https://cschwarz.com) for contract work.
The above does not apply if you already implemented everything.
Check out the *Coding Workflow* section below for details.
@@ -46,11 +48,8 @@ zrepl is written in [Go](https://golang.org) and uses [Go modules](https://githu
The documentation is written in [ReStructured Text](http://docutils.sourceforge.net/rst.html) using the [Sphinx](https://www.sphinx-doc.org) framework.
To get started, run `./lazy.sh devsetup` to easily install build dependencies and read `docs/installation.rst -> Compiling from Source`.
### Overall Architecture
The application architecture is documented as part of the user docs in the *Implementation* section (`docs/content/impl`).
Make sure to develop an understanding how zrepl is typically used by studying the user docs first.
`lazy.sh` uses `python3-pip` to fetch the build dependencies for the docs - you might want to use a [venv](https://docs.python.org/3/library/venv.html).
If you just want to install the Go dependencies, run `./lazy.sh godep`.
### Project Structure
@@ -62,14 +61,15 @@ Make sure to develop an understanding how zrepl is typically used by studying th
│   └── samples
├── daemon # the implementation of `zrepl daemon` subcommand
│   ├── filters
│   ├── hooks # snapshot hooks
│   ├── job # job implementations
│   ├── logging # logging outlets + formatters
│   ├── nethelpers
│   ├── prometheus
│   ├── pruner # pruner implementation
│   ├── snapper # snapshotter implementation
├── docs # sphinx-based documentation
├── dist # supplemental material for users & package maintainers
├── docs # sphinx-based documentation
│   ├── **/*.rst # documentation in reStructuredText
│   ├── sphinxconf
│   │   └── conf.py # sphinx config (see commit 445a280 why its not in docs/)
@@ -78,6 +78,7 @@ Make sure to develop an understanding how zrepl is typically used by studying th
│   └── public_git # checkout of zrepl.github.io managed by above shell script
├── endpoint # implementation of replication endpoints (=> package replication)
├── logger # our own logger package
├── platformtest # test suite for our zfs abstractions (error classification, etc)
├── pruning # pruning rules (the logic, not the actual execution)
│   └── retentiongrid
├── replication
@@ -92,14 +93,13 @@ Make sure to develop an understanding how zrepl is typically used by studying th
│ ├── transportmux # TCP connecter and listener used to split control & data traffic
│ └── versionhandshake # replication protocol version handshake perfomed on newly established connections
├── tlsconf # abstraction for Go TLS server + client config
├── transport # transports implementation
├── transport # transport implementations
│ ├── fromconfig
│ ├── local
│ ├── ssh
│ ├── tcp
│ └── tls
├── util
├── vendor # managed by dep
├── version # abstraction for versions (filled during build by Makefile)
└── zfs # zfs(8) wrappers
```
@@ -134,3 +134,15 @@ There will not be a big refactoring (an attempt was made, but it's destroying to
However, new contributions & patches should fix naming without further notice in the commit message.
### RPC debugging
Optionally, there are various RPC-related environment varibles, that if set to something != `""` will produce additional debug output on stderr:
https://github.com/zrepl/zrepl/blob/master/rpc/rpc_debug.go#L11
https://github.com/zrepl/zrepl/blob/master/rpc/dataconn/dataconn_debug.go#L11
https://github.com/zrepl/zrepl/blob/master/rpc/dataconn/stream/stream_debug.go#L11
https://github.com/zrepl/zrepl/blob/master/rpc/dataconn/heartbeatconn/heartbeatconn_debug.go#L11
+2
View File
@@ -60,6 +60,8 @@ func runTestFilterCmd(subcommand *cli.Subcommand, args []string) error {
confFilter = j.Filesystems
case *config.PushJob:
confFilter = j.Filesystems
case *config.SnapJob:
confFilter = j.Filesystems
default:
return fmt.Errorf("job type %T does not have filesystems filter", j)
}
+3 -3
View File
@@ -101,9 +101,9 @@ and serves as a reference for build dependencies and procedure:
::
git clone https://github.com/zrepl/zrepl.git
cd zrepl
sudo docker build -t zrepl_build -f build.Dockerfile .
git clone https://github.com/zrepl/zrepl.git && \
cd zrepl && \
sudo docker build -t zrepl_build -f build.Dockerfile . && \
sudo docker run -it --rm \
-v "${PWD}:/src" \
--user "$(id -u):$(id -g)" \
-1
View File
@@ -262,7 +262,6 @@ func (p *Planner) doPlanning(ctx context.Context) ([]*Filesystem, error) {
return nil, err
}
sfss := slfssres.GetFilesystems()
// no progress here since we could run in a live-lock on connectivity issues
rlfssres, err := p.receiver.ListFilesystems(ctx, &pdu.ListFilesystemReq{})
if err != nil {
+1 -1
View File
@@ -49,7 +49,7 @@ type Wire interface {
// No data that could otherwise be Read is lost as a consequence of this call.
// The use case for this API is abortive connection shutdown.
// To provide any value over draining Wire using io.Read, an implementation
// will likely use out-of-bounds messaging mechanisms.
// will likely use out-of-band messaging mechanisms.
// TODO WaitForPeerClose() (supported bool, err error)
}
+1 -1
View File
@@ -32,7 +32,7 @@ func TLSListenerFactoryFromConfig(c *config.Global, in *config.TLSServe) (transp
serverCert, err := tls.LoadX509KeyPair(in.Cert, in.Key)
if err != nil {
return nil, errors.Wrap(err, "cannot parse cer/key pair")
return nil, errors.Wrap(err, "cannot parse cert/key pair")
}
clientCNs := make(map[string]struct{}, len(in.ClientCNs))
-1
View File
@@ -63,7 +63,6 @@ type FilesystemPlaceholderState struct {
// For nonexistent FS, err == nil and state.FSExists == false
func ZFSGetFilesystemPlaceholderState(p *DatasetPath) (state *FilesystemPlaceholderState, err error) {
state = &FilesystemPlaceholderState{FS: p.ToString()}
state.FS = p.ToString()
props, err := zfsGet(p.ToString(), []string{PlaceholderPropertyName}, sourceLocal)
var _ error = (*DatasetDoesNotExist)(nil) // weak assertion on zfsGet's interface
if _, ok := err.(*DatasetDoesNotExist); ok {