buildgrid.server.client.cas module

buildgrid.server.client.cas.create_digest_from_file(file_obj: BinaryIO) → Digest

Computed the Digest of a file-like object.

The Digest contains a hash of the file’s contents and the size of those contents. This function only reads the content in chunks for hashing, so is safe to use on large files.

Parameters:

file_obj (BinaryIO) – A file-like object of some kind.

Returns:

The Digest for the given file object.

Return type:

Digest

buildgrid.server.client.cas.merkle_tree_maker(directory_path: str) → Iterator[tuple[buildgrid._protos.build.bazel.remote.execution.v2.remote_execution_pb2.FileNode | buildgrid._protos.build.bazel.remote.execution.v2.remote_execution_pb2.DirectoryNode, BinaryIO, str]]

Walks a local folder tree, generating FileNode and DirectoryNode.

Parameters:

directory_path (str) – absolute or relative path to a local directory.

Yields:

Message, bytes, str – a tutple of either a FileNode or DirectoryNode message, the corresponding blob and the corresponding node path.

class buildgrid.server.client.cas.Downloader(channel: Channel, instance: str | None = None, retries: int = 0, max_backoff: int = 64, should_backoff: bool = True)

Bases: object

Remote CAS files, directories and messages download helper.

The Downloader class comes with a generator factory function that can be used together with the with statement for context management:

from buildgrid.server.client.cas import download

with download(channel, instance='build') as downloader:
    downloader.get_message(message_digest)
get_blob(digest: Digest) → bytearray | None

Retrieves a blob from the remote CAS server.

Parameters:

digest (Digest) – the blob’s digest to fetch.

Returns:

the fetched blob data or None if not found.

Return type:

bytearray

get_blobs(digests: list[buildgrid._protos.build.bazel.remote.execution.v2.remote_execution_pb2.Digest]) → list[bytes]

Retrieves a list of blobs from the remote CAS server.

Parameters:

digests (list) – list of :obj:`Digest`s for the blobs to fetch.

Returns:

the fetched blob data list.

Return type:

list

Raises:

NotFoundError – if a blob is not present in the remote CAS server.

get_available_blobs(digests: list[buildgrid._protos.build.bazel.remote.execution.v2.remote_execution_pb2.Digest]) → list[tuple[bytes, buildgrid._protos.build.bazel.remote.execution.v2.remote_execution_pb2.Digest]]

Retrieves a list of blobs from the remote CAS server.

Skips blobs not found on the server without raising an error.

Parameters:

digests (list) – list of :obj:`Digest`s for the blobs to fetch.

Returns:

the fetched blob data list as (data, digest) pairs

Return type:

list

get_message(digest: Digest, message: T) → T

Retrieves a Message from the remote CAS server.

Parameters:

  • digest (Digest) – the message’s digest to fetch.

  • message (Message) – an empty message to fill.

Returns:

message filled or emptied if not found.

Return type:

Message

get_messages(digests: list[buildgrid._protos.build.bazel.remote.execution.v2.remote_execution_pb2.Digest], messages: list[google.protobuf.message.Message]) → list[google.protobuf.message.Message]

Retrieves a list of :obj:`Message`s from the remote CAS server.

Note

The digests and messages list must contain the same number of elements.

Parameters:

  • digests (list) – list of :obj:`Digest`s for the messages to fetch.

  • messages (list) – list of empty :obj:`Message`s to fill.

Returns:

the fetched and filled message list.

Return type:

list

download_file(digest: Digest, file_path: str, is_executable: bool = False, queue: bool = True) → None

Retrieves a file from the remote CAS server.

If queuing is allowed (queue=True), the download request may be defer. An explicit call to flush() can force the request to be send immediately (along with the rest of the queued batch).

Parameters:

  • digest (Digest) – the file’s digest to fetch.

  • file_path (str) – absolute or relative path to the local file to write.

  • is_executable (bool) – whether the file is executable or not.

  • queue (bool, optional) – whether or not the download request may be queued and submitted as part of a batch upload request. Defaults to True.

Raises:

  • NotFoundError – if digest is not present in the remote CAS server.

  • OSError – if file_path does not exist or is not readable.

download_directory(digest: Digest, directory_path: str) → None

Retrieves a Directory from the remote CAS server.

Parameters:

  • digest (Digest) – the directory’s digest to fetch.

  • directory_path (str) – the path to download to

Raises:

  • NotFoundError – if digest is not present in the remote CAS server.

  • FileExistsError – if directory_path already contains parts of their fetched directory’s content.

flush() → None

Ensures any queued request gets sent.

close() → None

Closes the underlying connection stubs.

Note

This will always send pending requests before closing connections, if any.

buildgrid.server.client.cas.download(channel: Channel, instance: str | None = None, u_uid: str | None = None, retries: int = 0, max_backoff: int = 64, should_backoff: bool = True) → Iterator[Downloader]

Context manager generator for the Downloader class.

class buildgrid.server.client.cas.Uploader(channel: Channel, instance: str | None = None, u_uid: str | None = None, retries: int = 0, max_backoff: int = 64, should_backoff: bool = True)

Bases: object

Remote CAS files, directories and messages upload helper.

The Uploader class comes with a generator factory function that can be used together with the with statement for context management:

from buildgrid.server.client.cas import upload

with upload(channel, instance='build') as uploader:
    uploader.upload_file('/path/to/local/file')
put_blob(blob: IO[bytes], digest: Digest | None = None, queue: bool = False, length: int | None = None) → Digest

Stores a blob into the remote CAS server.

If queuing is allowed (queue=True), the upload request may be defer. An explicit call to flush() can force the request to be send immediately (along with the rest of the queued batch).

The caller should set at least one of digest or length to allow the uploader to skip determining the size of the blob using multiple seeks.

Parameters:

  • blob (IO[bytes]) – a file-like object containing the blob.

  • digest (Digest, optional) – the blob’s digest.

  • queue (bool, optional) – whether or not the upload request may be queued and submitted as part of a batch upload request. Defaults to False.

  • length (int, optional) – The size of the blob, in bytes. If digest is also set, this is ignored in favour of digest.size_bytes.

Returns:

the sent blob’s digest.

Return type:

Digest

put_message(message: Message, digest: Digest | None = None, queue: bool = False) → Digest

Stores a message into the remote CAS server.

If queuing is allowed (queue=True), the upload request may be defer. An explicit call to flush() can force the request to be send immediately (along with the rest of the queued batch).

Parameters:

  • message (Message) – the message object.

  • digest (Digest, optional) – the message’s digest.

  • queue (bool, optional) – whether or not the upload request may be queued and submitted as part of a batch upload request. Defaults to False.

Returns:

the sent message’s digest.

Return type:

Digest

upload_file(file_path: str, queue: bool = True) → Digest

Stores a local file into the remote CAS storage.

If queuing is allowed (queue=True), the upload request may be defer. An explicit call to flush() can force the request to be send immediately (allong with the rest of the queued batch).

Parameters:

  • file_path (str) – absolute or relative path to a local file.

  • queue (bool, optional) – whether or not the upload request may be queued and submitted as part of a batch upload request. Defaults to True.

Returns:

The digest of the file’s content.

Return type:

Digest

Raises:

  • FileNotFoundError – If file_path does not exist.

  • PermissionError – If file_path is not readable.

upload_directory(directory_path: str, queue: bool = True) → Digest

Stores a local folder into the remote CAS storage.

If queuing is allowed (queue=True), the upload request may be defer. An explicit call to flush() can force the request to be send immediately (allong with the rest of the queued batch).

Parameters:

  • directory_path (str) – absolute or relative path to a local folder.

  • queue (bool, optional) – wheter or not the upload requests may be queued and submitted as part of a batch upload request. Defaults to True.

Returns:

The digest of the top Directory.

Return type:

Digest

Raises:

  • FileNotFoundError – If directory_path does not exist.

  • PermissionError – If directory_path is not readable.

upload_tree(directory_path: str, queue: bool = True) → Digest

Stores a local folder into the remote CAS storage as a Tree.

If queuing is allowed (queue=True), the upload request may be defer. An explicit call to flush() can force the request to be send immediately (allong with the rest of the queued batch).

Parameters:

  • directory_path (str) – absolute or relative path to a local folder.

  • queue (bool, optional) – wheter or not the upload requests may be queued and submitted as part of a batch upload request. Defaults to True.

Returns:

The digest of the Tree.

Return type:

Digest

Raises:

  • FileNotFoundError – If directory_path does not exist.

  • PermissionError – If directory_path is not readable.

flush() → None

Ensures any queued request gets sent.

close() → None

Closes the underlying connection stubs.

Note

This will always send pending requests before closing connections, if any.

buildgrid.server.client.cas.upload(channel: Channel, instance: str | None = None, u_uid: str | None = None, retries: int = 0, max_backoff: int = 64, should_backoff: bool = True) → Iterator[Uploader]

Context manager generator for the Uploader class.