WIP runtime-controllable concurrency for replication

Changes done so far:
- signal to route concurrency request up to the stepQueue
    - pretty hacky, no status reporting yet
- stepQueue upsizing (confirmed that it works, no intermediary commit)

Stuck at: stepQueue downsizing
- idea was to have the stepQueue signal to the activated step that it
  should suspend
- ideally, we'd just kill everything associated with the step and track
  it as restartable
    - the driver model doesn't allow for that though within an attempt
    - would need to start a separate run for the step
- less perfect: tell the downsized steps to stop copying, but leave
  all zfs sends + rpc conns open
    - - doesn't give back the resoures that a step aquired before
      being selected as downsize victim (open zfs processe + TCP conn,
      some memory in the pipe)
    - - looks weird to user if the ps aux
    - + re-waking a step is easy: just tell it to proceed with copying
    - (the impl would likely pass a check function down into Step.do
      and have it check that functino periodically. the suspend should
      be acknowledged, and stepQueue should only remove the step from
      the active queue _after_ that step has acknowledged that is
      suspended)
This commit is contained in:
Christian Schwarz
2020-02-01 15:28:52 +01:00
parent b8d9f4ba92
commit 17add553d3
14 changed files with 244 additions and 75 deletions
@@ -17,7 +17,7 @@ import (
// (relies on scheduler responsivity of < 500ms)
func TestPqNotconcurrent(t *testing.T) {
var ctr uint32
q := newStepQueue()
q := newStepQueue(1)
var wg sync.WaitGroup
wg.Add(4)
go func() {
@@ -29,7 +29,7 @@ func TestPqNotconcurrent(t *testing.T) {
}()
// give goroutine "1" 500ms to enter queue, get the active slot and enter time.Sleep
defer q.Start(1)()
defer q.Start()()
time.Sleep(500 * time.Millisecond)
// while "1" is still running, queue in "2", "3" and "4"
@@ -77,8 +77,9 @@ func (r record) String() string {
// Hence, perform some statistics on the wakeup times and assert that the mean wakeup
// times for each step are close together.
func TestPqConcurrent(t *testing.T) {
q := newStepQueue()
concurrency := 5
q := newStepQueue(concurrency)
var wg sync.WaitGroup
filesystems := 100
stepsPerFS := 20
@@ -104,8 +105,7 @@ func TestPqConcurrent(t *testing.T) {
records <- recs
}(fs)
}
concurrency := 5
defer q.Start(concurrency)()
defer q.Start()()
wg.Wait()
close(records)
t.Logf("loop done")