API reference
BuildGrid’s Application Programming Interface (API) reference documentation.
Configuration Parser API
The tagged YAML nodes in the reference configuration are handled by the YAML parser using the following set of objects:
- class buildgrid._app.settings.parser.YamlFactory
Bases:
YAMLObject
Base class for contructing maps or scalars from tags.
- yaml_tag: str | None = None
- schema: str | None = None
- classmethod from_yaml(loader, node)
Convert a representation node to a Python object.
- class buildgrid._app.settings.parser.Channel(_yaml_filename: str, insecure_mode: bool, address: str | None = None, port: int | None = None, credentials=None)
Bases:
YamlFactory
Creates a GRPC channel.
The
Channel
class returns a grpc.Channel and is generated from the tag!channel
. Creates either a secure or insecure channel.- Usage
- !channel address (str): Address for the channel. (For example, 'localhost:50055' or 'unix:///tmp/sock') port (int): A port for the channel (only if no address was specified). insecure-mode: false credentials: tls-server-key: !expand-path ~/.config/buildgrid/server.key tls-server-cert: !expand-path ~/.config/buildgrid/server.cert tls-client-certs: !expand-path ~/.config/buildgrid/client.cert
- Parameters:
port (int) – A port for the channel.
insecure_mode (bool) – If
True
, generates an insecure channel, even if there are credentials. Defaults toTrue
.credentials (dict, optional) –
A dictionary in the form:
tls-server-key: /path/to/server-key tls-server-cert: /path/to/server-cert tls-client-certs: /path/to/client-certs
- yaml_tag: str | None = '!channel'
- schema: str | None = 'misc/channel.yaml'
- class buildgrid._app.settings.parser.ExpandPath(_yaml_filename: str, path: str)
Bases:
YamlFactory
Returns a string of the user’s path after expansion.
The
ExpandPath
class returns a string and is generated from the tag!expand-path
.- Usage
path: !expand-path ~/bgd-data/cas
- Parameters:
path (str) – Can be used with strings such as:
~/dir/to/something
or$HOME/certs
- yaml_tag: str | None = '!expand-path'
- class buildgrid._app.settings.parser.ExpandVars(_yaml_filename: str, path: str)
Bases:
YamlFactory
Expand environment variables in a string.
The
ExpandVars
class returns a string and is generated from the tag!expand-vars
.- Usage
endpoint: !expand-vars $ENDPOINT
- Parameters:
path (str) – Can be used with strings such as:
http://$ENDPOINT
- yaml_tag: str | None = '!expand-vars'
- class buildgrid._app.settings.parser.ReadFile(_yaml_filename: str, path)
Bases:
YamlFactory
Returns a string of the contents of the specified file.
The
ReadFile
class returns a string and is generated from the tag!read-file
.- Usage
secret_key: !read-file /var/bgd/s3-secret-key
- Parameters:
path (str) – Can be used with strings such as:
~/path/to/some/file
or$HOME/myfile
or/path/to/file
- yaml_tag: str | None = '!read-file'
- class buildgrid._app.settings.parser.Disk(_yaml_filename: str, path: str)
Bases:
YamlFactory
Generates
buildgrid.server.cas.storage.disk.DiskStorage
using the tag!disk-storage
.- Usage
- !disk-storage path: /opt/bgd/cas-storage
- Parameters:
path (str) – Path to directory to storage.
- yaml_tag: str | None = '!disk-storage'
- schema: str | None = 'storage/disk.yaml'
- class buildgrid._app.settings.parser.LRU(_yaml_filename: str, size: str)
Bases:
YamlFactory
Generates
buildgrid.server.cas.storage.lru_memory_cache.LRUMemoryCache
using the tag!lru-storage
.- Usage
- !lru-storage size: 2048M
- Parameters:
size (int) – Size e.g
10kb
. Size parsed withbuildgrid._app.settings.parser._parse_size()
.
- yaml_tag: str | None = '!lru-storage'
- schema: str | None = 'storage/lru.yaml'
- class buildgrid._app.settings.parser.S3(_yaml_filename: str, bucket: str, endpoint: str, access_key: str, secret_key: str, read_timeout_seconds_per_kilobyte: float | None = None, write_timeout_seconds_per_kilobyte: float | None = None, read_timeout_min_seconds: float = 120, write_timeout_min_seconds: float = 120, versioned_deletes: bool = False)
Bases:
YamlFactory
Generates
buildgrid.server.cas.storage.s3.S3Storage
using the tag!s3-storage
.- Usage
- !s3-storage bucket: bgd-bucket-{digest[0]}{digest[1]} endpoint: http://127.0.0.1:9000 access_key: !read-file /var/bgd/s3-access-key secret_key: !read-file /var/bgd/s3-secret-key read_timeout_seconds_per_kilobyte: 0.01 write_timeout_seconds_per_kilobyte: 0.01 read_timeout_min_seconds: 120 write_timeout_min_seconds: 120
- Parameters:
bucket (str) – Name of bucket
endpoint (str) – URL of endpoint.
access-key (str) – S3-ACCESS-KEY
secret-key (str) – S3-SECRET-KEY
read_timeout_seconds_per_kilobyte (float) – S3 Read timeout in seconds/kilobyte
write_timeout_seconds_per_kilobyte (float) – S3 Write timeout in seconds/kilobyte
read_timeout_min_seconds (float) – The minimal timeout for S3 read
write_timeout_min_seconds (float) – The minimal timeout for S3 writes
versioned_deletes (bool) – Query and use the VersionId when performing deletes.
- yaml_tag: str | None = '!s3-storage'
- schema: str | None = 'storage/s3.yaml'
- class buildgrid._app.settings.parser.Redis(_yaml_filename: str, host: str, port: int, password: str | None = None, db: int | None = None)
Bases:
YamlFactory
Generates
buildgrid.server.cas.storage.redis.RedisStorage
using the tag!redis-storage
.- Usage
- !redis-storage host: 127.0.0.1 port: 6379 password: !read-file /var/bgd/redis-pass db: 0
- Parameters:
host (str) – hostname of endpoint.
port (int) – port on host.
password (str) – redis database password
db (int) – db number
- yaml_tag: str | None = '!redis-storage'
- schema: str | None = 'storage/redis.yaml'
- class buildgrid._app.settings.parser.Remote(_yaml_filename: str, url: str, instance_name: str, credentials: Dict[str, str] | None = None, channel_options: Dict[str, Any] | None = None, retries: int = 3, max_backoff: int = 64, request_timeout: float | None = None)
Bases:
YamlFactory
Generates
buildgrid.server.cas.storage.remote.RemoteStorage
using the tag!remote-storage
.- Usage
- !remote-storage url: https://storage:50052/ instance-name: main credentials: tls-server-key: !expand-path ~/.config/buildgrid/server.key tls-server-cert: !expand-path ~/.config/buildgrid/server.cert tls-client-certs: !expand-path ~/.config/buildgrid/client.cert channel-options: lb-policy-name: round_robin request-timeout: 15
- Parameters:
url (str) – URL to remote storage. If used with
https
, needs credentials.instance_name (str) – Instance of the remote to connect to.
credentials (dict, optional) –
A dictionary in the form:
tls-client-key: /path/to/client-key tls-client-cert: /path/to/client-cert tls-server-cert: /path/to/server-cert
channel-options (dict, optional) –
A dictionary of grpc channel options in the form:
some-channel-option: channel_value other-channel-option: another-channel-value
https (See) – //github.com/grpc/grpc/blob/master/include/grpc/impl/codegen/grpc_types.h
options (for the valid channel) –
retries (int) – Max number of times to retry (default 3). Backoff between retries is about 2^(N-1), where N is the number of attempts
max_backoff (int) – Maximum backoff in seconds (default 64)
request_timeout (float) – gRPC request timeout in seconds (default None)
- yaml_tag: str | None = '!remote-storage'
- schema: str | None = 'storage/remote.yaml'
- class buildgrid._app.settings.parser.WithCache(_yaml_filename: str, cache: StorageABC, fallback: StorageABC, defer_fallback_writes: bool = False, fallback_writer_threads: int = 20)
Bases:
YamlFactory
Generates
buildgrid.server.cas.storage.with_cache.WithCacheStorage
using the tag!with-cache-storage
.- Usage
- !with-cache-storage cache: !lru-storage size: 2048M fallback: !disk-storage path: /opt/bgd/cas-storage defer-fallback-writes: no
- Parameters:
cache (StorageABC) – Storage instance to use as a cache
fallback (StorageABC) – Storage instance to use as a fallback on cache misses
defer-fallback-writes (bool) – If true, commit_write returns once writing to the cache is done, and the write into the fallback storage is done in a background thread
fallback-writer-threads (int) – The maximum number of threads to use for writing blobs into the fallback storage. Defaults to 20.
- yaml_tag: str | None = '!with-cache-storage'
- schema: str | None = 'storage/with-cache.yaml'
- class buildgrid._app.settings.parser.SizeDifferentiated(_yaml_filename: str, size_limited_storages: List[_SizeLimitedStorageConfig], fallback: StorageABC)
Bases:
YamlFactory
Generates
buildgrid.server.cas.storage.size_differentiated.SizeDifferentiatedStorage
using the tag!size-differentiated-storage
.- Usage
- !size-differentiated-storage size-limited-storages: - max-size: 1M storage: !lru-storage size: 2048M fallback: !disk-storage path: /opt/bgd/cas-storage
- Parameters:
size_limited_storages (list) – List of dictionaries. The dictionaries are expected to have
max-size
andstorage
keys, defining a storage provider to use to store blobs with size up tomax-size
.fallback (StorageABC) – Storage instance to use as a fallback for blobs which are too big for the options defined in
size_limited_storages
.
- yaml_tag: str | None = '!size-differentiated-storage'
- schema: str | None = 'storage/size-differentiated.yaml'
- class buildgrid._app.settings.parser.SQLConnection(_yaml_filename: str, automigrate: bool = False, connection_string: str | None = None, connection_timeout: int = 5, lock_timeout: int = 5, connect_args: Dict[str, Any] | None = None, max_overflow: int | None = None, pool_pre_ping: bool | None = None, pool_recycle: int | None = None, pool_size: int | None = None, pool_timeout: int | None = None)
Bases:
YamlFactory
Generates
buildgrid.server.sql.provider.SqlProvider
using the tag!sql-connection
.Example
- !sql-connection &sql connection_string: postgresql://bgd:insecure@database/bgd automigrate: yes connection_timeout: 5 lock_timeout: 5 pool-size: 5 pool-timeout: 30 max-overflow: 10
- yaml_tag: str | None = '!sql-connection'
- schema: str | None = 'connections/sql.yaml'
- class buildgrid._app.settings.parser.SQLSchedulerConfig(_yaml_filename: str, storage: StorageABC, sql: SqlProvider | None = None, sql_ro: SqlProvider | None = None, connection_string: str | None = None, automigrate: bool = False, connection_timeout: int = 5, lock_timeout: int = 5, pruner_job_max_age: Dict[str, float] | None = None, pruner_period: Dict[str, float] | None = None, pruner_max_delete_window: int | None = None, **kwargs)
Bases:
YamlFactory
Generates
buildgrid.server.persistence.sql.impl.SQLDataStore
using the tag!sql-scheduler
.Example
- !sql-scheduler storage: *cas-storage sql: *sql pruner-job-max-age: days: 90
This usage example assumes that the
cas-storage
reference refers to a storage backend, eg.!disk-storage
, and thesql
reference refers to an SQL connection manager using!sql-connection
.- Parameters:
storage (
buildgrid.server.cas.storage.storage_abc.StorageABC
) – Instance of storage to use for getting actions and storing job results. This must be an object constructed using a YAML tag ending in-storage
, for example!disk-storage
.sql (
buildgrid.server.sql.provider.SqlProvider
) – A configured SQL connection manager. This must be an object with an!sql-connection
YAML tag.sql_ro (
buildgrid.server.sql.provider.SqlProvider
) – Similar to sql, but used for readonly backend transactions. If set, it should be configured with a replica of main DB using an optional but encouraged readonly role. Permission check is not executed by BuildGrid. If not set, readonly transactions are executed by sql object.pruner_job_max_age (dict) – Allow the storage to remove old entries by specifying the maximum amount of time that a row should be kept after its job finished. If this value is None, pruning is disabled and the background pruning thread is never created.
pruner_period (dict) – How often to attempt to remove old entries. If pruning is enabled (see above) and this value is None, it is set to 5 minutes by default.
pruner_max_delete_window (int) – Maximum number of records removed in a single cleanup pass. If pruning is enabled and this value is None, it is set to 10000 by default. This allows to put a limit on the time that the database will be blocked on a single invocation of the cleanup routine. (A smaller value reduces the performance impact of removing entries, but makes the recovery of storage space slower.)
connection_string (str) –
SQLAlchemy connection string to use for connecting to the database.
This parameter is deprecated in favour of
sql
.automigrate (bool) –
Whether to attempt to automatically upgrade an existing DB schema to the newest version (this will also create everything from scratch if given an empty database).
This parameter is deprecated in favour of
sql
.connection_timeout (int) –
Time to wait for an SQLAlchemy connection to be available in the pool before timing out.
This parameter is deprecated in favour of
sql
.lock_timeout (int) – The timeout to use when the connection holds a lock in the database. This is supported only if the database backend is PostgresQL.
- yaml_tag: str | None = '!sql-scheduler'
- schema: str | None = 'scheduler/sql.yaml'
- class buildgrid._app.settings.parser.SQLDataStoreConfig(_yaml_filename: str, storage: StorageABC, connection_string: str | None = None, automigrate: bool = False, connection_timeout: int = 5, lock_timeout: int = 5, **kwargs)
Bases:
YamlFactory
Generates
buildgrid.server.persistence.sql.impl.SQLDataStore
using the tag!sql-data-store
.Warning
This is deprecated and only used for compatibility with old configs.
- Usage
- !sql-data-store # This assumes that a storage instance is defined elsewhere # with a `&cas-storage` anchor storage: *cas-storage connection_string: postgresql://bgd:insecure@database/bgd automigrate: yes connection_timeout: 5
- Parameters:
storage (
buildgrid.server.cas.storage.storage_abc.StorageABC
) – Instance of storage to use for getting actions and storing job results. This must be an object constructed using a YAML tag ending in-storage
, for example!disk-storage
.connection_string (str) – SQLAlchemy connection string to use for connecting to the database.
automigrate (bool) – Whether to attempt to automatically upgrade an existing DB schema to the newest version (this will also create everything from scratch if given an empty database).
connection_timeout (int) – Time to wait for an SQLAlchemy connection to be available in the pool before timing out.
lock_timeout (int) – The timeout to use when the connection holds a lock in the database. This is supported only if the database backend is PostgresQL.
- yaml_tag: str | None = '!sql-data-store'
- schema: str | None = 'scheduler/sql.yaml'
- class buildgrid._app.settings.parser.MemorySchedulerConfig(_yaml_filename: str, storage: StorageABC)
Bases:
YamlFactory
Generates
buildgrid.server.persistence.mem.impl.MemoryDataStore
using the tag!memory-scheduler
.- Usage
- !memory-scheduler # This assumes that a storage instance is defined elsewhere # with a `&cas-storage` anchor storage: *cas-storage
- Parameters:
storage (
buildgrid.server.cas.storage.storage_abc.StorageABC
) – Instance of storage to use for getting actions and storing job results. This must be an object constructed using a YAML tag ending in-storage
, for example!disk-storage
.
- yaml_tag: str | None = '!memory-scheduler'
- schema: str | None = 'scheduler/memory.yaml'
- class buildgrid._app.settings.parser.MemoryDataStoreConfig(_yaml_filename: str, storage: StorageABC)
Bases:
YamlFactory
Generates
buildgrid.server.persistence.mem.impl.MemoryDataStore
using the tag!memory-data-store
.Warning
This is deprecated and only used for compatibility with old configs. Use
MemorySchedulerConfig
instead.- Usage
- !memory-data-store # This assumes that a storage instance is defined elsewhere # with a `&cas-storage` anchor storage: *cas-storage
- Parameters:
storage (
buildgrid.server.cas.storage.storage_abc.StorageABC
) – Instance of storage to use for getting actions and storing job results. This must be an object constructed using a YAML tag ending in-storage
, for example!disk-storage
.
- yaml_tag: str | None = '!memory-data-store'
- schema: str | None = 'scheduler/memory.yaml'
- class buildgrid._app.settings.parser.SQL_Index(_yaml_filename: str, storage: StorageABC, connection_string: str | None = None, sql: SqlProvider | None = None, automigrate: bool = False, window_size: int = 1000, inclause_limit: int = -1, fallback_on_get: bool = False, max_inline_blob_size: int = 0, refresh_accesstime_older_than: int = 0, **kwargs)
Bases:
YamlFactory
Generates
buildgrid.server.cas.storage.index.sql.SQLIndex
using the tag!sql-index
.- Usage
- !sql-index # This assumes that a storage instance is defined elsewhere # with a `&cas-storage` anchor storage: *cas-storage sql: *sql window-size: 1000 inclause-limit: -1 fallback-on-get: no max-inline-blob-size: 256 refresh-accesstime-older-than: 0
- Parameters:
storage (
buildgrid.server.cas.storage.storage_abc.StorageABC
) – Instance of storage to use. This must be a storage object constructed using a YAML tag ending in-storage
, for example!disk-storage
.connection_string (str) – SQLAlchemy connection string
automigrate (bool) – Attempt to automatically upgrade an existing DB schema to the newest version.
window_size (uint) – Maximum number of blobs to fetch in one SQL operation (larger resultsets will be automatically split into multiple queries)
inclause_limit (int) – If nonnegative, overrides the default number of variables permitted per “in” clause. See the buildgrid.server.cas.storage.index.sql.SQLIndex comments for more details.
fallback_on_get (bool) – By default, the SQL Index only fetches blobs from the underlying storage if they’re present in the index on
get_blob
/bulk_read_blobs
requests to minimize interactions with the storage. If this is set, the index instead checks the underlying storage directly onget_blob
/bulk_read_blobs
requests, then loads all blobs found into the index.max_inline_blob_size (int) – Blobs of this size or smaller are stored directly in the index and not in the backing storage (must be nonnegative).
refresh-accesstime-older-than (int) – When reading a blob, its access timestamp will not be updated if the current time is not at least refresh-accesstime-older-than seconds newer than the access timestamp. Set this to reduce load associated with frequent timestamp updates.
- yaml_tag: str | None = '!sql-index'
- schema: str | None = 'storage/sql-index.yaml'
- class buildgrid._app.settings.parser.Execution(_yaml_filename: str, storage, sql: SqlProvider | None = None, action_cache=None, action_browser_url=None, data_store=None, scheduler=None, property_keys=None, wildcard_property_keys=None, operation_stream_keepalive_timeout=600, bot_session_keepalive_timeout=600, permissive_bot_session=False, endpoints=('execution', 'operations', 'bots'), discard_unwatched_jobs=False, max_execution_timeout=7200, max_list_operations_page_size=1000, logstream=None, metering_service_client: MeteringServiceClient | None = None)
Bases:
YamlFactory
Generates
buildgrid.server.execution.service.ExecutionService
using the tag!execution
.- Usage
# This assumes that the YAML anchors are defined elsewhere - !execution storage: *cas-storage sql: *sql action-cache: *remote-cache action-browser-url: http://localhost:8080 scheduler: *state-database property-keys: - runnerType wildcard-property-keys: - chrootDigest operation-stream-keepalive-timeout: 600 bot-session-keepalive-timeout: 600 endpoints: - execution - operations - bots discard-unwatched-jobs: no max-execution-timeout: 7200 max-list-operations-page-size: 1000
- Parameters:
storage (
buildgrid.server.cas.storage.storage_abc.StorageABC
) – Instance of storage to use. This must be an object constructed using a YAML tag ending in-storage
, for example!disk-storage
.sql (
buildgrid.server.sql.provider.SqlProvider
) – A configured SQL connection manager. This must be an object with an!sql-connection
YAML tag.action_cache (
ActionCache
) – Instance of action cache to use.action_browser_url (str) – The base URL to use to generate Action Browser links to users
scheduler (
DataStoreInterface
) – Instance of data store to use for the scheduler’s state.property_keys (list) – The platform property keys available to use in routing Actions to workers
wildcard_property_keys (list) – The platform property keys which can be set in Actions but are not used to select workers
operation_stream_keepalive_timeout (int) – The longest time (in seconds) we’ll wait before sending the current status in an Operation response stream of an Execute or WaitExecution request. Defaults to 600s (10 minutes).
bot_session_keepalive_timeout (int) – The longest time (in seconds) we’ll wait for a bot to send an update before it assumes it’s dead. Defaults to 600s (10 minutes).
permissive_bot_session – Whether to accept UpdateBotSession requests from bots that haven’t explicitly called CreateBotSession with this particular server instance.
endpoints (list) – List of service/endpoint types to enable. Possible services are
execution
,operations
, andbots
. By default all three are enabled.discard_unwatched_jobs (bool) – Allow operation id to persist without a client connection.
max_execution_timeout (int) – The maximum time jobs are allowed to be in ‘OperationStage.EXECUTING’. This is a lazy check tha happens before job deduplication and when new peers register for an ongoing Operation. When this time is exceeded in executing stage, the job will be cancelled.
max_list_operations_page_size (int) – The maximum number of operations that can be returned in a ListOperations response. A page token will be returned with the response to allow the client to get the next page of results.
metering_service_client – Optional client to check whether resource usage of a client is above a predefined threshold
- yaml_tag: str | None = '!execution'
- schema: str | None = 'services/execution.yaml'
- class buildgrid._app.settings.parser.Bots(_yaml_filename: str, storage, sql: SqlProvider | None = None, action_cache=None, bot_session_keepalive_timeout=600, data_store=None, scheduler=None, permissive_bot_session=False, logstream=None, metering_service_client: MeteringServiceClient | None = None)
Bases:
YamlFactory
Generates
buildgrid.server.bots.instance.BotsInterface
using the tag!bots
.- Usage
# This assumes that the YAML anchors are defined elsewhere - !bots storage: *cas-storage sql: *sql action-cache: *remote-cache scheduler: *state-database bot-session-keepalive-timeout: 600 permissive-bot-session: yes
- Parameters:
storage (
buildgrid.server.cas.storage.storage_abc.StorageABC
) – Instance of storage to use. This must be an object constructed using a YAML tag ending in-storage
, for example!disk-storage
.sql (
buildgrid.server.sql.provider.SqlProvider
) – A configured SQL connection manager. This must be an object with an!sql-connection
YAML tag.action_cache (
Action
) – Instance of action cache to use.scheduler (
DataStoreInterface
) – Instance of data store to use for the scheduler’s state.bot_session_keepalive_timeout (int) – The longest time (in seconds) we’ll wait for a bot to send an update before it assumes it’s dead. Defaults to 600s (10 minutes).
permissive_bot_session – Whether to accept UpdateBotSession requests from bots that haven’t explicitly called CreateBotSession with this particular server instance.
metering_service_client – Optional client to publish resource usage of operations
- yaml_tag: str | None = '!bots'
- schema: str | None = 'services/bots.yaml'
- class buildgrid._app.settings.parser.Action(_yaml_filename: str, storage: StorageABC | None = None, max_cached_refs: int | None = None, allow_updates: bool = True, cache_failed_actions: bool = True, cache: ActionCacheABC | None = None)
Bases:
YamlFactory
Generates
buildgrid.server.actioncache.service.ActionCacheService
using the tag!action-cache
.- Usage
# This assumes that the YAML anchors are defined elsewhere - !action-cache cache: *lru-cache
- Parameters:
cache (ActionCacheABC) – The ActionCache backend to use for this cache.
- yaml_tag: str | None = '!action-cache'
- schema: str | None = 'services/action-cache.yaml'
- class buildgrid._app.settings.parser.WithCacheAction(_yaml_filename: str, cache: ActionCacheABC, fallback: ActionCacheABC, allow_updates: bool = True, cache_failed_actions: bool = True)
Bases:
YamlFactory
Generates:class:buildgrid.server.actioncache.caches.with_cache.WithCacheActionCache using the tag
!with-cache-action-cache
.- Usage
# This assumes that the YAML anchors are defined elsewhere - !with-cache-action-cache storage: *cas-storage cache: *cache-ac fallback: *fallback-ac
- Parameters:
cache (ActionCacheABC) – ActionCache instance to use as a local cache
fallback (ActionCacheABC) – ActionCache instance to use as a fallback on local cache misses
allow_updates (bool) – Allow updates pushed to the Action Cache. Defaults to
True
.cache_failed_actions (bool) – Whether to store failed (non-zero exit code) actions. Default to
True
.
- yaml_tag: str | None = '!with-cache-action-cache'
- schema: str | None = 'caches/with-cache.yaml'
- class buildgrid._app.settings.parser.LruAction(_yaml_filename: str, storage: StorageABC, max_cached_refs: int, allow_updates: bool = True, cache_failed_actions: bool = True)
Bases:
YamlFactory
Generates
buildgrid.server.actioncache.caches.lru_cache.LruActionCache
using the tag!lru-action-cache
.- Usage
# This assumes that the YAML anchors are defined elsewhere - !lru-action-cache storage: *cas-storage max-cached-refs: 1024 cache-failed-actions: yes allow-updates: yes
- Parameters:
storage (
buildgrid.server.cas.storage.storage_abc.StorageABC
) – Instance of storage to use.max_cached_refs (int) – Max number of cached actions.
allow_updates (bool) – Allow updates pushed to the Action Cache. Defaults to
True
.cache_failed_actions (bool) – Whether to store failed (non-zero exit code) actions. Default to
True
.
- yaml_tag: str | None = '!lru-action-cache'
- schema: str | None = 'caches/lru-action-cache.yaml'
- class buildgrid._app.settings.parser.S3Action(_yaml_filename: str, storage: StorageABC, allow_updates: bool = True, cache_failed_actions: bool = True, entry_type: str | None = None, migrate_entries: bool | None = False, bucket: str | None = None, endpoint: str | None = None, access_key: str | None = None, secret_key: str | None = None)
Bases:
YamlFactory
Generates
buildgrid.server.actioncache.caches.s3_cache.S3ActionCache
using the tag!s3action-cache
.- Usage
# This assumes that the YAML anchors are defined elsewhere - !s3action-cache storage: *cas-storage allow-updates: yes cache-failed-actions: yes entry-type: action-result-digest migrate-entries: no bucket: bgd-action-cache endpoint: http://localhost:9000/ access-key: !read-file /var/bgd/s3-access-key secret-key: !read-file /var/bgd/s3-secret-key
- Parameters:
storage (
buildgrid.server.cas.storage.storage_abc.StorageABC
) – Instance of storage to use. This must be an object constructed using a YAML tag ending in-storage
, for example!disk-storage
.allow_updates (bool) – Allow updates pushed to the Action Cache. Defaults to
True
.cache_failed_actions (bool) – Whether to store failed (non-zero exit code) actions. Default to
True
.entry_type (str) – whether entries in S3 will store an
'action-result'
or an'action-result-digest'
(default).migrate_entries (bool) – Whether to automatically update the values of entries that contain a different type of value to entry_type as they are queried. Default to
False
.bucket (str) – Name of bucket
endpoint (str) – URL of endpoint.
access-key (str) – S3-ACCESS-KEY
secret-key (str) – S3-SECRET-KEY
- yaml_tag: str | None = '!s3action-cache'
- schema: str | None = 'services/s3-action-cache.yaml'
- class buildgrid._app.settings.parser.RemoteAction(_yaml_filename: str, url: str, instance_name: str, retries: int = 3, max_backoff: int = 64, request_timeout: float | None = None, credentials: Dict[str, str] | None = None, channel_options: Dict[str, Any] | None = None)
Bases:
YamlFactory
Generates
buildgrid.server.actioncache.caches.remote.RemoteActionCache
using the tag!remote-action-cache
.- Usage
- !remote-action-cache url: https://action-cache:50053 instance-name: main credentials: tls-server-key: !expand-path ~/.config/buildgrid/server.key tls-server-cert: !expand-path ~/.config/buildgrid/server.cert tls-client-certs: !expand-path ~/.config/buildgrid/client.cert channel-options: lb-policy-name: round_robin
- Parameters:
url (str) – URL to remote action cache. If used with
https
, needs credentials.instance_name (str) – Instance of the remote to connect to.
credentials (dict, optional) –
A dictionary in the form:
tls-client-key: /path/to/client-key tls-client-cert: /path/to/client-cert tls-server-cert: /path/to/server-cert
channel-options (dict, optional) –
A dictionary of grpc channel options in the form:
some-channel-option: channel_value other-channel-option: another-channel-value
https (See) – //github.com/grpc/grpc/blob/master/include/grpc/impl/codegen/grpc_types.h
options (for the valid channel) –
- yaml_tag: str | None = '!remote-action-cache'
- schema: str | None = 'services/remote-action-cache.yaml'
- class buildgrid._app.settings.parser.WriteOnceAction(_yaml_filename: str, action_cache)
Bases:
YamlFactory
Generates
buildgrid.server.actioncache.caches.write_once_cache.WriteOnceActionCache
using the tag!write-once-action-cache
.This allows a single update for a given key, essentially making it possible to create immutable ActionCache entries, rather than making the cache read-only as the
allow-updates
property of other ActionCache implementations does.- Usage
# This assumes that the YAML anchors are defined elsewhere - !write-once-action-cache action-cache: *remote-cache
- Parameters:
action_cache (ActionCache) – The action cache instance to make immutable.
- yaml_tag: str | None = '!write-once-action-cache'
- schema: str | None = 'services/write-once-action-cache.yaml'
- class buildgrid._app.settings.parser.RedisAction(_yaml_filename: str, storage: StorageABC, host: str | None = None, port: int | None = None, allow_updates: bool = True, cache_failed_actions: bool = True, entry_type: str | None = None, migrate_entries: bool | None = False, password: str | None = None, db: int = 0, dns_srv_record: str | None = None, sentinel_master_name: str | None = None, retries: int = 3)
Bases:
YamlFactory
Generates
buildgrid.server.actioncache.caches.redis_cache.RedisActionCache
using the tag!redis-action-cache
.This creates an Action Cache which stores the mapping from Action digests to ActionResults in Redis.
- Usage
# This assumes that the YAML anchors are defined elsewhere - !redis-action-cache storage: *cas-storage allow-updates: yes cache-failed-actions: yes entry-type: action-result-digest migrate-entries: no host: redis port: 6379 db: 0 dns-srv-record: <Domain name of SRV record> sentinel-master-name: <service_name of Redis sentinel's master instance> retries: 3
- Parameters:
storage (
buildgrid.server.cas.storage.storage_abc.StorageABC
) – Instance of storage to use. This must be an object constructed using a YAML tag ending in-storage
, for example!disk-storage
.allow_updates (bool) – Allow updates pushed to the Action Cache. Defaults to
True
.cache_failed_actions (bool) – Whether to store failed (non-zero exit code) actions. Default to
True
.entry_type (str) – whether entries in Redis will store an
'action-result'
or an'action-result-digest'
(default).migrate_entries (bool) – Whether to automatically update the values of entries that contain a different type of value to entry_type as they are queried. Default to
False
.host (str) – The hostname of the Redis server to use.
port (int) – The port that Redis is served on.
db (int) – The Redis database number to use.
dns-srv-record (str) – Domain name of SRV record used to discover host/port
sentinel-master-name (str) – Service name of Redis master instance, used in a Redis sentinel configuration
retries (int) – Max number of times to retry (default 3). Backoff between retries is about 2^(N-1), where N is the number of attempts
- yaml_tag: str | None = '!redis-action-cache'
- schema: str | None = 'caches/redis-action-cache.yaml'
- class buildgrid._app.settings.parser.Reference(_yaml_filename: str, storage: StorageABC, max_cached_refs: int, allow_updates: bool = True)
Bases:
YamlFactory
Generates
buildgrid.server.referencestorage.service.ReferenceStorageService
using the tag!reference-cache
.Note
This is intended for use by old versions of BuildStream as an artifact cache, and isn’t part of the REAPI/RWAPI specifications. There’s no need for this component if you aren’t using BuildStream.
- Usage
# This assumes that the YAML anchors are defined elsewhere - !reference-storage storage: *cas-storage max-cached-refs: 1024 allow-updates: yes
- Parameters:
storage (
buildgrid.server.cas.storage.storage_abc.StorageABC
) – Instance of storage to use. This must be an object constructed using a YAML tag ending in-storage
, for example!disk-storage
.max_cached_refs (int) – Max number of cached actions.
allow_updates (bool) – Allow updates pushed to CAS. Defaults to
True
.
- yaml_tag: str | None = '!reference-cache'
- schema: str | None = 'services/reference-cache.yaml'
- class buildgrid._app.settings.parser.CAS(_yaml_filename: str, storage: StorageABC, read_only: bool = False)
Bases:
YamlFactory
Generates
buildgrid.server.cas.service.ContentAddressableStorageService
using the tag!cas
.- Usage
# This assumes that the YAML anchors are defined elsewhere - !cas storage: *cas-storage
- Parameters:
storage (
buildgrid.server.cas.storage.storage_abc.StorageABC
) – Instance of storage to use. This must be an object constructed using a YAML tag ending in-storage
, for example!disk-storage
.
- yaml_tag: str | None = '!cas'
- schema: str | None = 'services/cas.yaml'
- class buildgrid._app.settings.parser.ByteStream(_yaml_filename: str, storage: StorageABC | None = None, stream_storage: StreamStorageABC | None = None, read_only: bool = False, disable_overwrite_early_return: bool = False)
Bases:
YamlFactory
Generates
buildgrid.server.cas.service.ByteStreamService
using the tag!bytestream
.- Usage
# This assumes that the YAML anchors are defined elsewhere - !bytestream storage: *cas-storage
- Parameters:
storage (
buildgrid.server.cas.storage.storage_abc.StorageABC
) – Instance of storage to use. This must be an object constructed using a YAML tag ending in-storage
, for example!disk-storage
.
- yaml_tag: str | None = '!bytestream'
- schema: str | None = 'services/bytestream.yaml'
- class buildgrid._app.settings.parser.LogStream(_yaml_filename: str, prefix='', stream_storage=None)
Bases:
YamlFactory
Generates
buildgrid.server.cas.logstream.instance.LogStreamInstance
using the tag!logstream
.- Usage
- !logstream prefix: test
- yaml_tag: str | None = '!logstream'
- schema: str | None = 'services/logstream.yaml'
- class buildgrid._app.settings.parser.MemoryStream(_yaml_filename: str, *args, **kwargs)
Bases:
YamlFactory
Generates
buildgrid.server.cas.logstream.stream_storage.MemoryStreamStorage
using the tag!memory-logstream-storage
.- Usage
- !memory-logstream-storage
- yaml_tag: str | None = '!memory-logstream-storage'
- class buildgrid._app.settings.parser.MemoryBuildEvents(_yaml_filename: str, *args, **kwargs)
Bases:
YamlFactory
Generates
buildgrid.server.build_events.storage.BuildEventStreamStorage
using the tag!memory-build-events-storage
.- Usage
- !memory-build-events
- yaml_tag: str | None = '!memory-build-events'
- class buildgrid._app.settings.parser.MeteringServiceClientFactory(_yaml_filename: str, base_url: str, token_path: str | None = None, retry_max_attempts: int = 0, retry_exp_base: float = 1.5, retry_multiplier: float = 1.0, retry_max_wait: float = 10.0, retry_http_statuses: List[int] | None = None, retry_exceptions: List[str] | None = None, retry_cause_exceptions: List[str] | None = None)
Bases:
YamlFactory
Generates
buildgrid_metering.client.MeteringServiceClient
using the tag!metering-service-client
.- Usage
- !metering-service-client token-path: /tmp/path/to/token retry-max-attempts: 3 retry-exp-base: 2 retry-multiplier: 1 retry-http-statuses: [503] retry-exceptions: ["metering-service-client-error"]
- yaml_tag: str | None = '!metering-service-client'
- schema: str | None = 'clients/metering-service-client.yaml'
- buildgrid._app.settings.parser.get_logstream_connection_info(logstream) Tuple[str | None, Dict[str, str] | None, str | None]
- buildgrid._app.settings.parser.get_parser()
- buildgrid._app.settings.parser.get_schema()
- buildgrid._app.settings.parser.check_type(*expected_types: Any) Callable[[Any, Any], bool]
- buildgrid._app.settings.parser.get_validator(schema=None)