Scheduling

The scheduler is responsible for matching queued jobs to available bots. It runs background threads called assigners which periodically query for an unassigned job, select a suitable bot, and write the assignment back to the database. Multiple assigner instances can run concurrently for throughput, and each sleeps for a configurable interval between iterations (shortened when work is being actively assigned).

Assigner Types

Both assigner types support preemption when instance quotas are configured. If a job’s instance is below its minimum quota and no bot is free, the scheduler can evict a lower-priority job from another instance in the same cohort to reclaim capacity. Preemption only triggers after the job has been queued longer than the configured preemption_delay.

PriorityAgeJobAssigner

A single-threaded assigner that picks the next unassigned job ordered by priority (lower value = higher priority, following REAPI convention) and then by queue age. A priority_percentage parameter controls how often priority ordering is used versus pure age ordering, allowing occasional starvation relief for older, lower-priority jobs.

When an assignment attempt fails (no healthy bot matches), the job is given a schedule_after backoff so it is not reconsidered immediately.

CohortAssigner

A cohort-aware assigner that spawns one thread per cohort. Each thread independently assigns jobs whose property label belongs to its cohort, which partitions the assignment workload by bot group.

The key difference from PriorityAgeJobAssigner is instance-level prioritization: the CohortAssigner first looks for jobs in instances that are below their minimum quota, and only then considers instances that are below their maximum quota. This ensures that under-served instances receive capacity before opportunistic work is scheduled.

Proactive Fetching

When a bot synchronizes its leases after completing work, the scheduler can fetch new jobs for it. The proactive_fetch_to_capacity flag controls how aggressively this happens.

When disabled (the default), the scheduler only fetches as many new jobs as the bot just completed. It restricts candidates to jobs whose schedule_after is at or after the current time (schedule_after >= now), which includes newly created jobs and jobs still within their backoff window, but excludes jobs whose backoff has already elapsed. Those elapsed-backoff jobs are left for the assigner threads to pick up on their next iteration.

When enabled, the scheduler fills all of the bot’s spare capacity in a single synchronization, and the schedule_after filter is removed entirely so that all unassigned queued jobs are eligible regardless of their backoff state.

Bot Assignment Strategy

The assigner delegates bot selection to a pluggable strategy. The only strategy currently implemented is AssignByCapacity, which selects a healthy bot (status OK, capacity > 0) whose platform capabilities match the job’s requirements.

By default, the first matching bot is locked and assigned. When sampling is enabled, the assigner uses a two-phase approach that can make better placement decisions under load.

Bot Sampling

When a SamplingConfig is attached to the strategy, the assigner reads a random sample of candidate bots without locking them, scores each candidate, and then attempts to lock the highest-scoring bot. If the chosen bot is no longer available (taken by another assigner), it falls back to locking any healthy bot. The max_attempts parameter controls how many sampling rounds are tried before falling back.

Each candidate is scored with a weighted linear combination:

score = priority * w_priority + capacity * w_capacity + locality_hit * w_locality

where locality_hit is 1 if the bot has a locality hint matching the job and 0 otherwise. The weights and their defaults are:

Weight	Default	Effect
priority	-10.0	Lower bot priority value produces a higher score (per REAPI convention).
locality	5.0	Bonus applied when the bot matches the job’s locality hint.
capacity	1.0	Bots with more spare capacity score higher, spreading load.

With the default weights, priority dominates: a bot with a priority advantage of 1 outweighs a locality match. Adjusting the weights allows operators to tune placement toward locality-affinity or load-balancing as needed.