Commit Graph

167 Commits

Author SHA1 Message Date
Christian Schwarz a6497b2c6e add platformtest: infrastructure for ZFS compatiblity testing 2019-09-14 13:43:46 +02:00
Christian Schwarz 77d3a1ad4d build: drop go Dep, switch to modules, support Go 1.13
bump enumer to v1.1.1
bump golangci-lint to v1.17.1

no `go mod tidy` because 1.13 and 1.12 seem to alter each other's output

fixes #112
2019-09-14 13:36:44 +02:00
Christian Schwarz 921b34235e daemon: env var for autostarting pprof endpoint 2019-09-07 19:50:57 +02:00
Christian Schwarz b167de3096 filters: add some basic test cases for DatasetMapFilter 2019-07-14 18:28:06 +02:00
Christian Schwarz f31b54582f daemon/job: fix receiving-job root-fs overlap detection
fixup for 7756c9a5
fixes #160
2019-03-27 13:12:26 +01:00
Christian Schwarz 5b97953bfb run golangci-lint and apply suggested fixes 2019-03-27 13:12:26 +01:00
Christian Schwarz afed762774 format source tree using goimports 2019-03-22 19:41:12 +01:00
Christian Schwarz 0ab62f197a Merge branch 'problame/overlapping-dataset-hierarchy-improvements' into 'master'
closes #153
2019-03-22 11:02:57 +01:00
Christian Schwarz 9a2d1b3074 Merge branch 'problame/138-snapshotter-fails-silently' into 'master'
closes #152
2019-03-22 10:59:58 +01:00
Christian Schwarz 26ec29d8b2 snapper: retry on errors during syncUp and log them
fixes #138
2019-03-21 17:17:10 +01:00
Christian Schwarz c0028c1c44 daemon/logging: add replication logic logger 2019-03-21 17:03:34 +01:00
Christian Schwarz 7756c9a55c config + job: forbid non-verlapping receiver root_fs
refs #136
refs #140
2019-03-21 12:07:55 +01:00
Christian Schwarz 6f7467e8d8 Merge branch 'InsanePrawn-master' into 'master' 2019-03-20 19:45:00 +01:00
Christian Schwarz 5aefc47f71 daemon: remove last traces of watchdog mechanism 2019-03-19 18:15:34 +01:00
Christian Schwarz 158d1175e3 rename SinglePruner to LocalPruner 2019-03-17 21:18:25 +01:00
Christian Schwarz b25da7b9b0 job: snap: comment fix 2019-03-17 21:07:42 +01:00
Christian Schwarz 5cd2593f52 job: snap: workaround for replication cursor requirement 2019-03-17 21:07:01 +01:00
Christian Schwarz d8d9e34914 pruner: single: remove unused member considerSnapAtCursorReplicated 2019-03-17 20:57:34 +01:00
Christian Schwarz 17818439a0 Merge branch 'problame/replication_refactor' into InsanePrawn-master 2019-03-17 17:33:51 +01:00
Christian Schwarz da3ba50a2c Merge remote-tracking branch 'origin/master' into problame/replication_refactor 2019-03-16 14:48:01 +01:00
Christian Schwarz 4ee00091d6 pull job: support manual-only invocation 2019-03-16 14:24:05 +01:00
Christian Schwarz aff639e87a Merge remote-tracking branch 'origin/master' into InsanePrawn-master 2019-03-15 21:05:20 +01:00
Christian Schwarz a0f301d700 syslog logging: fix priority parsing + add test for default facility 2019-03-15 18:18:16 +01:00
Ximalas fc311a9fd6 syslog logging: support setting facility in config 2019-03-15 17:55:11 +01:00
Christian Schwarz 7584c66bdb pruner: remove retry handling + fix early give-up
Retry handling is broken since the gRPC changes (wrong error classification).
Will come back at some point, hopefully by merging the replication
driver retry infrastructure.

However, the simpler architecture allows an easy fix for the problem
that the pruner practically gave up on the first error it encountered.

fixes #123
2019-03-13 21:04:39 +01:00
Christian Schwarz d78d20e2d0 pruner: skip placeholders + FSes without correspondents on source
fixes #126
2019-03-13 20:42:37 +01:00
Christian Schwarz c87759affe replication/driver: automatic retries on connectivity-related errors 2019-03-13 15:00:40 +01:00
Christian Schwarz 07b43bffa4 replication: refactor driving logic (no more explicit state machine) 2019-03-13 15:00:40 +01:00
Christian Schwarz 796c5ad42d rpc rewrite: control RPCs using gRPC + separate RPC for data transfer
transport/ssh: update go-netssh to new version
    => supports CloseWrite and Deadlines
    => build: require Go 1.11 (netssh requires it)
2019-03-13 13:53:48 +01:00
InsanePrawn 160a3b6d32 more gofmt, drop snapjob.go_prefmt after it was accidentally added 2018-11-21 22:14:43 +01:00
InsanePrawn 3cef76d463 Refactor snapJob() to snapJobFromConfig() 2018-11-21 14:37:03 +01:00
InsanePrawn e9564a7e5c Inlined a couple legacy leftover functions from the mode copypasta 2018-11-21 14:35:40 +01:00
InsanePrawn b79ad3ddc3 Honour PruneKeepNotReplicated.KeepSnashotAtCursor in SinglePrunerFactory 2018-11-21 14:17:38 +01:00
InsanePrawn d0f898751f Gofmt snapjob.go 2018-11-21 14:02:21 +01:00
InsanePrawn 22d9830baa Fix prometheus with multiple jobs 2018-11-21 04:26:03 +01:00
InsanePrawn e10dc129de Make getPruner() private 2018-11-21 03:39:03 +01:00
InsanePrawn dd11fc96db Touchups in job.go 2018-11-21 03:27:39 +01:00
InsanePrawn 7de3c0a09a Removed the references to a pruning 'side' in the singlepruner logging code and the snapjob prometheus thing. 2018-11-21 02:52:33 +01:00
InsanePrawn 141e49727c Missed a last reference to tasks 2018-11-21 02:51:23 +01:00
InsanePrawn 442d61918b remove most of the watchdog machinery 2018-11-21 02:42:13 +01:00
InsanePrawn 58dcc07430 Added SnapJobStatus 2018-11-21 02:08:39 +01:00
InsanePrawn 19d0916e34 remove snapMode, rename snap_ActiveSide to SnapJob 2018-11-21 01:54:56 +01:00
InsanePrawn 1265cc7934 pruned unused lines and comments ;) 2018-11-21 01:34:50 +01:00
InsanePrawn 3d2688e959 Ugly but working inital snapjob implementation 2018-11-20 19:30:15 +01:00
Christian Schwarz 3472145df6 pruner + proto change: better handling of missing replication cursor
- don't treat missing replication cursor as an error in protocol
- treat it as a per-fs planning error instead
2018-11-16 12:21:54 +01:00
Christian Schwarz 5e1ea21f85 pruning: add 'Negate' option to KeepRegex and expose it in config 2018-11-16 12:21:54 +01:00
Christian Schwarz 98bc8d1717 daemon/job: explicit notice of ZREPL_JOB_WATCHDOG_TIMEOUT environment variable on cancellation 2018-10-22 11:03:31 +02:00
Christian Schwarz 94427d334b replication + pruner + watchdog: adjust timeouts based on practical experience 2018-10-21 18:37:57 +02:00
Christian Schwarz b2844569c8 replication: rewrite error handling + simplify state machines
* Remove explicity state machine code for all but replication.Replication
* Introduce explicit error types that satisfy interfaces which provide
  sufficient information for replication.Replication to make intelligent
  retry + queuing decisions

  * Temporary()
  * LocalToFS()

* Remove the queue and replace it with a simple array that we sort each
  time (yay no generics :( )
2018-10-21 18:37:57 +02:00
Christian Schwarz fffda09f67 replication + pruner: progress markers during planning 2018-10-21 17:50:08 +02:00