Commit Graph

945 Commits

Author SHA1 Message Date
Christian Schwarz 1aae7b222f docs: fix confusing description of the role of client identity for sink jobs 2018-12-01 15:19:59 +01:00
Christian Schwarz 3535b251ab freeze Go build dependencies in Gopkg.lock
* use pseudo-depdencies in build/build.go to convince dep
* update Travis, Dockerfile and Docs
* build.Dockerfile image now contains the Go build dependencies
* => faster builds
* bump pdu file after protoc update

fixes #106
2018-12-01 14:36:40 +01:00
Christian Schwarz 707f070a3c build: fix dirty detection at the end of release build
was using Bashisms
2018-12-01 14:36:40 +01:00
InsanePrawn 160a3b6d32 more gofmt, drop snapjob.go_prefmt after it was accidentally added 2018-11-21 22:14:43 +01:00
InsanePrawn d977796f18 Add SnapJob docs 2018-11-21 16:59:46 +01:00
InsanePrawn 3cef76d463 Refactor snapJob() to snapJobFromConfig() 2018-11-21 14:37:03 +01:00
InsanePrawn e9564a7e5c Inlined a couple legacy leftover functions from the mode copypasta 2018-11-21 14:35:40 +01:00
InsanePrawn b79ad3ddc3 Honour PruneKeepNotReplicated.KeepSnashotAtCursor in SinglePrunerFactory 2018-11-21 14:17:38 +01:00
InsanePrawn d0f898751f Gofmt snapjob.go 2018-11-21 14:02:21 +01:00
InsanePrawn 22d9830baa Fix prometheus with multiple jobs 2018-11-21 04:26:03 +01:00
InsanePrawn c4e23862cd Added status view for SnapJob. 2018-11-21 04:06:13 +01:00
InsanePrawn e10dc129de Make getPruner() private 2018-11-21 03:39:03 +01:00
InsanePrawn dd11fc96db Touchups in job.go 2018-11-21 03:27:39 +01:00
InsanePrawn 7de3c0a09a Removed the references to a pruning 'side' in the singlepruner logging code and the snapjob prometheus thing. 2018-11-21 02:52:33 +01:00
InsanePrawn 141e49727c Missed a last reference to tasks 2018-11-21 02:51:23 +01:00
InsanePrawn 442d61918b remove most of the watchdog machinery 2018-11-21 02:42:13 +01:00
InsanePrawn 58dcc07430 Added SnapJobStatus 2018-11-21 02:08:39 +01:00
InsanePrawn 19d0916e34 remove snapMode, rename snap_ActiveSide to SnapJob 2018-11-21 01:54:56 +01:00
InsanePrawn 1265cc7934 pruned unused lines and comments ;) 2018-11-21 01:34:50 +01:00
InsanePrawn 3d2688e959 Ugly but working inital snapjob implementation 2018-11-20 19:30:15 +01:00
Christian Schwarz 7ab51fad0d zfs: add 'received' property source, handle 'any' source correctly and use 'any' for placeholder FS detection
we want was first noticed in zfs 0.8rc1

Upstream doc PR: https://github.com/zfsonlinux/zfs/pull/8134
2018-11-16 13:07:13 +01:00
Christian Schwarz 3472145df6 pruner + proto change: better handling of missing replication cursor
- don't treat missing replication cursor as an error in protocol
- treat it as a per-fs planning error instead
2018-11-16 12:21:54 +01:00
Christian Schwarz 5e1ea21f85 pruning: add 'Negate' option to KeepRegex and expose it in config 2018-11-16 12:21:54 +01:00
Christian Schwarz 2db3977408 cli: add 'test placeholder' subcommand for placeholder debugging 2018-11-16 12:21:54 +01:00
Christian Schwarz ca6d5d3bb5 build: Travis CI configuration 2018-11-16 12:10:58 +01:00
Christian Schwarz 163c2bc533 docs: update requirements.txt 2018-11-16 12:10:58 +01:00
Christian Schwarz dd286aa12e client: fix status bytes per second measurement
still far from perfect, but better than incorrect values
2018-11-05 01:37:51 +01:00
Christian Schwarz 80babe3ab4 docs/README: update package hierarchy overview 2018-10-26 22:05:57 +02:00
Christian Schwarz ca0cab0a15 docs/tutorial: fix headlines 2018-10-26 21:52:49 +02:00
JMoVS ad8be226fd fix small typo 0.1.0-rc3 2018-10-22 11:32:37 +02:00
Christian Schwarz 9b3e5c38e2 docs: fix changelog + invocations of wakeup subcommand 2018-10-22 11:27:00 +02:00
Christian Schwarz 7e1c5f5d1f docs: discourage use of ssh+stdinserver transport due to inferior error handling 2018-10-22 11:25:16 +02:00
Christian Schwarz 98bc8d1717 daemon/job: explicit notice of ZREPL_JOB_WATCHDOG_TIMEOUT environment variable on cancellation 2018-10-22 11:03:31 +02:00
Christian Schwarz 2889a5d5ff client/status: current bytes/second + spinning progress bar 2018-10-21 23:15:21 +02:00
Christian Schwarz 0b8c19c620 docs/tutorial: switch to push setup & use mutual TLS (2 machines) 2018-10-21 22:20:35 +02:00
Christian Schwarz a62b475f46 docs/transport/tls: document self-signed certs procedure for 2-machine setup 2018-10-21 22:20:07 +02:00
Christian Schwarz 1691839c6b replication: handle context cancellation errors as GlobalError 2018-10-21 19:06:35 +02:00
Christian Schwarz 36265ff349 fixup 438f950be3: forgotten ErrorCount in printf 2018-10-21 18:37:57 +02:00
Christian Schwarz 94427d334b replication + pruner + watchdog: adjust timeouts based on practical experience 2018-10-21 18:37:57 +02:00
Christian Schwarz b2844569c8 replication: rewrite error handling + simplify state machines
* Remove explicity state machine code for all but replication.Replication
* Introduce explicit error types that satisfy interfaces which provide
  sufficient information for replication.Replication to make intelligent
  retry + queuing decisions

  * Temporary()
  * LocalToFS()

* Remove the queue and replace it with a simple array that we sort each
  time (yay no generics :( )
2018-10-21 18:37:57 +02:00
Christian Schwarz ae5e60b1ae client/status: display problems as wrapped + indented if they do not fit the current line 2018-10-21 17:50:08 +02:00
Christian Schwarz fffda09f67 replication + pruner: progress markers during planning 2018-10-21 17:50:08 +02:00
Christian Schwarz 5ec7a5c078 pruner: report: fix broken checks for state (wrong precedence rules) 2018-10-21 13:37:08 +02:00
Christian Schwarz 190c7270d9 daemon/active + watchdog: simplify control flow using explicit ActiveSideState 2018-10-21 12:53:34 +02:00
Christian Schwarz f704b28cad daemon/job: track active side state explicitly 2018-10-21 12:52:48 +02:00
Christian Schwarz 5efeec1819 daemon/control: stop logging status endpoint requests 2018-10-20 12:50:31 +02:00
Christian Schwarz 438f950be3 pruner: improve cancellation + error handling strategy
Pruner now backs off as soon as there is an error, making that error the
Error field in the pruner report.
The error is also stored in the specific *fs that failed, and we
maintain an error counter per *fs to de-prioritize those fs that failed.
Like with replication, the de-prioritization on errors is to avoid '
getting stuck' with an individual filesystem until the watchdog hits.
2018-10-20 12:46:43 +02:00
Christian Schwarz 50c1549865 pruner: fixup 69bfcb7bed: add missing progress updates for watchdog 2018-10-20 10:58:22 +02:00
Christian Schwarz 6e21a67473 build: detect if generate made things dirty and break release build in that case 0.1.0-rc2 2018-10-19 17:52:49 +02:00
Christian Schwarz 17ab39d646 build: add missing subpackages 2018-10-19 17:23:00 +02:00