API reference

BuildGrid’s Application Programming Interface (API) reference documentation.

Configuration Parser API

The tagged YAML nodes in the reference configuration are handled by the YAML parser using the following set of objects:

class buildgrid._app.settings.parser.YamlFactory

Bases: yaml.YAMLObject

Base class for contructing maps or scalars from tags.

classmethod from_yaml(loader, node)

Convert a representation node to a Python object.

class buildgrid._app.settings.parser.Channel(port, insecure_mode, credentials=None)

Bases: buildgrid._app.settings.parser.YamlFactory

Creates a GRPC channel.

The Channel class returns a grpc.Channel and is generated from the tag !channel. Creates either a secure or insecure channel.

Parameters
  • port (int) – A port for the channel.

  • insecure_mode (bool) – If True, generates an insecure channel, even if there are

credentials. Defaults to True.

credentials (dict, optional): A dictionary in the form:

tls-server-key: /path/to/server-key
tls-server-cert: /path/to/server-cert
tls-client-certs: /path/to/client-certs
yaml_tag = '!channel'
class buildgrid._app.settings.parser.ExpandPath

Bases: buildgrid._app.settings.parser.YamlFactory

Returns a string of the user’s path after expansion.

The ExpandPath class returns a string and is generated from the tag !expand-path.

Parameters

path (str) – Can be used with strings such as: ~/dir/to/something or $HOME/certs

yaml_tag = '!expand-path'
class buildgrid._app.settings.parser.ReadFile

Bases: buildgrid._app.settings.parser.YamlFactory

Returns a string of the contents of the specified file.

The ReadFile class returns a string and is generated from the tag !read-file.

Parameters

path (str) – Can be used with strings such as: ~/path/to/some/file or $HOME/myfile or /path/to/file

yaml_tag = '!read-file'
class buildgrid._app.settings.parser.Disk

Bases: buildgrid._app.settings.parser.YamlFactory

Generates buildgrid.server.cas.storage.disk.DiskStorage using the tag !disk-storage.

Parameters

path (str) – Path to directory to storage.

yaml_tag = '!disk-storage'
class buildgrid._app.settings.parser.LRU

Bases: buildgrid._app.settings.parser.YamlFactory

Generates buildgrid.server.cas.storage.lru_memory_cache.LRUMemoryCache using the tag !lru-storage.

Parameters

size (int) – Size e.g 10kb. Size parsed with buildgrid._app.settings.parser._parse_size().

yaml_tag = '!lru-storage'
class buildgrid._app.settings.parser.S3

Bases: buildgrid._app.settings.parser.YamlFactory

Generates buildgrid.server.cas.storage.s3.S3Storage using the tag !s3-storage.

Parameters
  • bucket (str) – Name of bucket

  • endpoint (str) – URL of endpoint.

  • access-key (str) – S3-ACCESS-KEY

  • secret-key (str) – S3-SECRET-KEY

yaml_tag = '!s3-storage'
class buildgrid._app.settings.parser.Redis

Bases: buildgrid._app.settings.parser.YamlFactory

Generates buildgrid.server.cas.storage.redis.RedisStorage using the tag !redis-storage.

Parameters
  • host (str) – hostname of endpoint.

  • port (int) – port on host.

  • password (str) – redis database password

  • db (int) – db number

yaml_tag = '!redis-storage'
class buildgrid._app.settings.parser.Remote

Bases: buildgrid._app.settings.parser.YamlFactory

Generates buildgrid.server.cas.storage.remote.RemoteStorage using the tag !remote-storage.

Parameters
  • url (str) – URL to remote storage. If used with https, needs credentials.

  • instance_name (str) – Instance of the remote to connect to.

  • credentials (dict, optional) –

    A dictionary in the form:

    tls-client-key: /path/to/client-key
    tls-client-cert: /path/to/client-cert
    tls-server-cert: /path/to/server-cert
    

yaml_tag = '!remote-storage'
class buildgrid._app.settings.parser.WithCache

Bases: buildgrid._app.settings.parser.YamlFactory

Generates buildgrid.server.cas.storage.with_cache.WithCacheStorage using the tag !with-cache-storage.

Parameters
  • url (str) – URL to remote storage. If used with https, needs credentials.

  • instance_name (str) – Instance of the remote to connect to.

  • credentials (dict, optional) –

    A dictionary in the form:

    tls-client-key: /path/to/certs
    tls-client-cert: /path/to/certs
    tls-server-cert: /path/to/certs
    

yaml_tag = '!with-cache-storage'
class buildgrid._app.settings.parser.SQLDataStoreConfig

Bases: buildgrid._app.settings.parser.YamlFactory

Generates buildgrid.server.persistence.sql.impl.SQLDataStore using the tag !sql-data-store.

Parameters
  • storage (buildgrid.server.cas.storage.storage_abc.StorageABC) – Instance of storage to use for getting actions and storing job results.

  • connection_string (str) – SQLAlchemy connection string to use for connecting to the database.

  • automigrate (bool) – Whether to attempt to automatically upgrade an existing DB schema to the newest version (this will also create everything from scratch if given an empty database).

yaml_tag = '!sql-data-store'
class buildgrid._app.settings.parser.MemoryDataStoreConfig

Bases: buildgrid._app.settings.parser.YamlFactory

Generates buildgrid.server.persistence.mem.impl.MemoryDataStore using the tag !memory-data-store.

Parameters

storage (buildgrid.server.cas.storage.storage_abc.StorageABC) – Instance of storage to use for getting actions and storing job results.

yaml_tag = '!memory-data-store'
class buildgrid._app.settings.parser.SQL_Index

Bases: buildgrid._app.settings.parser.YamlFactory

Generates buildgrid.server.cas.storage.index.sql.SQLIndex using the tag !sql-index.

Parameters
  • storage (buildgrid.server.cas.storage.storage_abc.StorageABC) – Instance of storage to use.

  • connection_string (str) – SQLAlchemy connection string

  • automigrate (bool) – Attempt to automatically upgrade an existing DB schema to the newest version.

  • window_size (uint) – Maximum number of blobs to fetch in one SQL operation (larger resultsets will be automatically split into multiple queries)

  • inclause_limit (int) – If nonnegative, overrides the default number of variables permitted per “in” clause. See the buildgrid.server.cas.storage.index.sql.SQLIndex comments for more details.

  • fallback_on_get (bool) – By default, the SQL Index only fetches blobs from the underlying storage if they’re present in the index on get_blob/bulk_read_blobs requests to minimize interactions with the storage. If this is set, the index instead checks the underlying storage directly on get_blob/bulk_read_blobs requests, then loads all blobs found into the index.

yaml_tag = '!sql-index'
class buildgrid._app.settings.parser.Execution

Bases: buildgrid._app.settings.parser.YamlFactory

Generates buildgrid.server.execution.service.ExecutionService using the tag !execution.

Parameters
  • storage (buildgrid.server.cas.storage.storage_abc.StorageABC) – Instance of storage to use.

  • action_cache (Action) – Instance of action cache to use.

  • The base URL to use to generate Action Browser links to users (action_browser_url) –

  • data_store (DataStoreInterface) – Instance of data store to use for state

  • property_keys – The allowed property keys for jobs

  • bot_session_keepalive_timeout – The longest time (in seconds) we’ll wait for a bot to send an update before it assumes it’s dead. Defaults to 600s (10 minutes).

  • endpoints – List of service/endpoint types to enable. Possible services are execution, operations``, and bots. By default all three are enabled.

  • discard_unwatched_jobs (bool) – Allow operation id to persist without a client connection.

yaml_tag = '!execution'
class buildgrid._app.settings.parser.Bots

Bases: buildgrid._app.settings.parser.YamlFactory

Generates buildgrid.server.bots.instance.BotsInterface using the tag !bots.

Parameters
  • storage (buildgrid.server.cas.storage.storage_abc.StorageABC) – Instance of storage to use.

  • action_cache (Action) – Instance of action cache to use.

  • data_store (DataStoreInterface) – Instance of data store to use for state

  • bot_session_keepalive_timeout – The longest time (in seconds) we’ll wait for a bot to send an update before it assumes it’s dead. Defaults to 600s (10 minutes).

yaml_tag = '!bots'
class buildgrid._app.settings.parser.Action

Bases: buildgrid._app.settings.parser.YamlFactory

Generates buildgrid.server.actioncache.service.ActionCacheService using the tag !action-cache.

Parameters
  • storage (buildgrid.server.cas.storage.storage_abc.StorageABC) – Instance of storage to use.

  • max_cached_refs (int) – Max number of cached actions.

  • allow_updates (bool) – Allow updates pushed to CAS. Defaults to True.

  • cache_failed_actions (bool) – Whether to store failed (non-zero exit code) actions. Default to True.

yaml_tag = '!action-cache'
class buildgrid._app.settings.parser.S3Action

Bases: buildgrid._app.settings.parser.YamlFactory

Generates buildgrid.server.actioncache.service.S3ActionCache using the tag !s3-action-cache.

Parameters
  • storage (buildgrid.server.cas.storage.storage_abc.StorageABC) – Instance of storage to use.

  • allow_updates (bool) – Allow updates pushed to CAS. Defaults to True.

  • cache_failed_actions (bool) – Whether to store failed (non-zero exit code) actions. Default to True.

  • bucket (str) – Name of bucket

  • endpoint (str) – URL of endpoint.

  • access-key (str) – S3-ACCESS-KEY

  • secret-key (str) – S3-SECRET-KEY

yaml_tag = '!s3action-cache'
class buildgrid._app.settings.parser.RemoteAction

Bases: buildgrid._app.settings.parser.YamlFactory

Generates buildgrid.server.actioncache.remote.RemoteActionCache using the tag !remote-action-cache.

Parameters
  • url (str) – URL to remote action cache. If used with https, needs credentials.

  • instance_name (str) – Instance of the remote to connect to.

  • credentials (dict, optional) –

    A dictionary in the form:

    tls-client-key: /path/to/client-key
    tls-client-cert: /path/to/client-cert
    tls-server-cert: /path/to/server-cert
    

yaml_tag = '!remote-action-cache'
class buildgrid._app.settings.parser.WriteOnceAction

Bases: buildgrid._app.settings.parser.YamlFactory

Generates buildgrid.server.actioncache.remote.WriteOnceActionCache using the tag !write-once-action-cache.

Parameters
  • url (str) – URL to remote action cache. If used with https, needs credentials.

  • instance_name (str) – Instance of the remote to connect to.

  • credentials (dict, optional) –

    A dictionary in the form:

    tls-client-key: /path/to/client-key
    tls-client-cert: /path/to/client-cert
    tls-server-cert: /path/to/server-cert
    

yaml_tag = '!write-once-action-cache'
class buildgrid._app.settings.parser.Reference

Bases: buildgrid._app.settings.parser.YamlFactory

Generates buildgrid.server.referencestorage.service.ReferenceStorageService using the tag !reference-cache.

Parameters
yaml_tag = '!reference-cache'
class buildgrid._app.settings.parser.CAS

Bases: buildgrid._app.settings.parser.YamlFactory

Generates buildgrid.server.cas.service.ContentAddressableStorageService using the tag !cas.

Parameters

storage (buildgrid.server.cas.storage.storage_abc.StorageABC) – Instance of storage to use.

yaml_tag = '!cas'
class buildgrid._app.settings.parser.ByteStream

Bases: buildgrid._app.settings.parser.YamlFactory

Generates buildgrid.server.cas.service.ByteStreamService using the tag !bytestream.

Parameters

storage (buildgrid.server.cas.storage.storage_abc.StorageABC) – Instance of storage to use.

yaml_tag = '!bytestream'
buildgrid._app.settings.parser.get_parser()