WIP runtime-controllable concurrency for replication
Changes done so far:
- signal to route concurrency request up to the stepQueue
- pretty hacky, no status reporting yet
- stepQueue upsizing (confirmed that it works, no intermediary commit)
Stuck at: stepQueue downsizing
- idea was to have the stepQueue signal to the activated step that it
should suspend
- ideally, we'd just kill everything associated with the step and track
it as restartable
- the driver model doesn't allow for that though within an attempt
- would need to start a separate run for the step
- less perfect: tell the downsized steps to stop copying, but leave
all zfs sends + rpc conns open
- - doesn't give back the resoures that a step aquired before
being selected as downsize victim (open zfs processe + TCP conn,
some memory in the pipe)
- - looks weird to user if the ps aux
- + re-waking a step is easy: just tell it to proceed with copying
- (the impl would likely pass a check function down into Step.do
and have it check that functino periodically. the suspend should
be acknowledged, and stepQueue should only remove the step from
the active queue _after_ that step has acknowledged that is
suspended)
This commit is contained in:
@@ -176,6 +176,18 @@ func (s *jobs) reset(job string) error {
|
||||
return wu()
|
||||
}
|
||||
|
||||
func (s *jobs) setConcurrency(jobName string, concurrency int) error {
|
||||
s.m.RLock()
|
||||
defer s.m.RUnlock()
|
||||
|
||||
job, ok := s.jobs[jobName]
|
||||
if !ok {
|
||||
return errors.Errorf("Job %q does not exist", job)
|
||||
}
|
||||
|
||||
return job.SetConcurrency(concurrency)
|
||||
}
|
||||
|
||||
const (
|
||||
jobNamePrometheus = "_prometheus"
|
||||
jobNameControl = "_control"
|
||||
|
||||
Reference in New Issue
Block a user