Migration

Some releases of BuildGrid introduce breaking changes which require a migration step, or changes which require consideration when upgrading to the new version.

The bgd migrate CLI can be used to apply any database migrations which have been added in a new release.

This section documents such changes in each release. If a release is missing from this page, it can be assumed to be too old to need documenting here, or to not have any such changes.

0.5.0

Removal of !assign-by-locality

The !assign-by-locality bot assignment strategy has been removed. Locality-based bot selection is now handled by !assign-by-capacity with a !sampling-config that includes locality in its !sampling-weights.

To migrate, replace any !assign-by-locality configuration with !assign-by-capacity using a !sampling-config. For example:

bot-assignment-strategy: !assign-by-capacity
  sampling: !sampling-config
    sample-size: 5
    weights: !sampling-weights
      locality: 5.0
      capacity: 1.0

The fallback parameter is no longer needed, as !assign-by-capacity with sampling weights naturally handles the case where no bot with a matching locality hint is available.

0.4.11

Scheduler schema changes

BuildGrid 0.4.11 includes a database migration which needs to be applied; which can be found at data/revisions/b1d6fb2ff810->2b5f9653537e.sql.

This migration adds a new index to the jobs table, to improve proactive scheduling performance (where bots pick up new jobs directly to help preserve queue order in high load scenarios).

0.4.10

Scheduler schema changes

BuildGrid 0.4.10 includes a database migration which needs to be applied; which can be found at data/revisions/146448692d39->b1d6fb2ff810.sql.

This migration adds a new priority column to the bots table, to support prioritising specific bots for assignment when using sampling assignment.

0.4.7

Scheduler schema changes

BuildGrid 0.4.7 includes a database migration which needs to be applied; which can be found at data/revisions/d1de6a7df71e->146448692d39.sql.

This migration adds new columns to the jobs table for the preferred property label and preferred requirements hash for the job. This is part of the preferred properties functionality added to the scheduler in this release, allowing some platform properties to be specified as preferred, but not strictly required, when considering which bot to assign a job to.

0.4.5

Minimum Python version

BuildGrid 0.4.5 bumps the minimum supported Python version to 3.12. The range of supported versions is now 3.12 - 3.14 inclusive.

0.4.1

Scheduler schema changes

BuildGrid 0.4.1 includes a database migration which needs to be applied; which can be found at data/revisions/3737630fc9cf->d1de6a7df71e.sql.

This migration adds a new table to keep a log of jobs cancelled by the cohort assigner when maintaining minimum quotas. See Eviction cooldown for details on how this log is used.

0.4.0

BuildGrid 0.4.0 introduces several changes which require care when migrating.

BuildBox version

Due to reintroduction of long-polling when bots have spare capacity, the minimum supported BuildBox version is now 1.3.51. Older versions of BuildBox will continue to work with BuildGrid 0.4.0, but bots with capacity of more than 1 will experience delays in communication of execution completion when they have spare capacity.

SQLIndex schema changes

The deleted column of the index table is removed. This column was used by the cleanup daemon to perform a three-step deletion process where blobs were marked as deleted in the index, removed from the backend storage, and then deleted from the index.

When migrating to BuildGrid 0.4.0 with an SQL index in use, any cleaners should be temporarily paused to avoid the upgrade effectively “undeleting” blobs which are part-way through this multi-step deletion.

Tip

Ensure that your cleaners are all updated to 0.4.0 before running the database migration to actually drop the deleted column.

A generated SQL database migration to apply this change can be found in data/revisions/ff09fbc30c3e->3737630fc9cf.sql.

Removal of SQLIndex fallback-on-get

The fallback-on-get configuration option for an SQL index is removed in this release. Usage of this option for data migration to a newly-added index should be replaced with a !replicated-storage with a replica for the newly indexed stack.