buildgrid.server.cas.service module

CAS services

Implements the Content Addressable Storage API and ByteStream API.

class buildgrid.server.cas.service.ContentAddressableStorageService

Bases: ContentAddressableStorageServicer, InstancedServicer[ContentAddressableStorageInstance]

REGISTER_METHOD(server)

FULL_NAME: ClassVar[str] = 'build.bazel.remote.execution.v2.ContentAddressableStorage': The full name of the servicer, used to match instances to the servicer and configure reflection. This value should be declared on the class of any Servicer implementations.

FindMissingBlobs(request: FindMissingBlobsRequest, context: ServicerContext) → FindMissingBlobsResponse

Determine if blobs are present in the CAS.

Clients can use this API before uploading blobs to determine which ones are already present in the CAS and do not need to be uploaded again.

Servers SHOULD increase the lifetimes of the referenced blobs if necessary and applicable.

There are no method-specific errors.

BatchUpdateBlobs(request: BatchUpdateBlobsRequest, context: ServicerContext) → BatchUpdateBlobsResponse

Upload many blobs at once.

The server may enforce a limit of the combined total size of blobs to be uploaded using this API. This limit may be obtained using the [Capabilities][build.bazel.remote.execution.v2.Capabilities] API. Requests exceeding the limit should either be split into smaller chunks or uploaded using the [ByteStream API][google.bytestream.ByteStream], as appropriate.

This request is equivalent to calling a Bytestream Write request on each individual blob, in parallel. The requests may succeed or fail independently.

Errors:

INVALID_ARGUMENT: The client attempted to upload more than the

server supported limit.

Individual requests may return the following errors, additionally:

RESOURCE_EXHAUSTED: There is insufficient disk quota to store the blob.
INVALID_ARGUMENT: The

[Digest][build.bazel.remote.execution.v2.Digest] does not match the provided data.

BatchReadBlobs(request: BatchReadBlobsRequest, context: ServicerContext) → BatchReadBlobsResponse

Download many blobs at once.

The server may enforce a limit of the combined total size of blobs to be downloaded using this API. This limit may be obtained using the [Capabilities][build.bazel.remote.execution.v2.Capabilities] API. Requests exceeding the limit should either be split into smaller chunks or downloaded using the [ByteStream API][google.bytestream.ByteStream], as appropriate.

This request is equivalent to calling a Bytestream Read request on each individual blob, in parallel. The requests may succeed or fail independently.

Errors:

INVALID_ARGUMENT: The client attempted to read more than the

server supported limit.

Every error on individual read will be returned in the corresponding digest status.

GetTree(request: GetTreeRequest, context: ServicerContext) → Iterator[GetTreeResponse]

Fetch the entire directory tree rooted at a node.

This request must be targeted at a [Directory][build.bazel.remote.execution.v2.Directory] stored in the [ContentAddressableStorage][build.bazel.remote.execution.v2.ContentAddressableStorage] (CAS). The server will enumerate the Directory tree recursively and return every node descended from the root.

The GetTreeRequest.page_token parameter can be used to skip ahead in the stream (e.g. when retrying a partially completed and aborted request), by setting it to a value taken from GetTreeResponse.next_page_token of the last successfully processed GetTreeResponse).

The exact traversal order is unspecified and, unless retrieving subsequent pages from an earlier request, is not guaranteed to be stable across multiple invocations of GetTree.

If part of the tree is missing from the CAS, the server will return the portion present and omit the rest.

Errors:

NOT_FOUND: The requested tree root is not present in the CAS.

class buildgrid.server.cas.service.ResourceNameRegex

Bases: object

READ = '^(.*?)/?(blobs/.*/[0-9]*)$'

WRITE = '^(.*?)/?(uploads/.*/blobs/.*/[0-9]*)'

class buildgrid.server.cas.service.ByteStreamService

Bases: ByteStreamServicer, InstancedServicer[ByteStreamInstance]

REGISTER_METHOD(server)

FULL_NAME: ClassVar[str] = 'google.bytestream.ByteStream': The full name of the servicer, used to match instances to the servicer and configure reflection. This value should be declared on the class of any Servicer implementations.

Read(request: ReadRequest, context: ServicerContext) → Iterator[ReadResponse]: Read() is used to retrieve the contents of a resource as a sequence of bytes. The bytes are returned in a sequence of responses, and the responses are delivered as the results of a server-side streaming RPC.

Write(request_iterator: Iterator[WriteRequest], context: ServicerContext) → WriteResponse

Write() is used to send the contents of a resource as a sequence of bytes. The bytes are sent in a sequence of request protos of a client-side streaming RPC.

A Write() action is resumable. If there is an error or the connection is broken during the Write(), the client should check the status of the Write() by calling QueryWriteStatus() and continue writing from the returned committed_size. This may be less than the amount of data the client previously sent.

Calling Write() on a resource name that was previously written and finalized could cause an error, depending on whether the underlying service allows over-writing of previously written resources.

When the client closes the request channel, the service will respond with a WriteResponse. The service will not view the resource as complete until the client has sent a WriteRequest with finish_write set to true. Sending any requests on a stream after sending a request with finish_write set to true will cause an error. The client should check the WriteResponse it receives to determine how much data the service was able to commit and whether the service views the resource as complete or not.

QueryWriteStatus(request: QueryWriteStatusRequest, context: ServicerContext) → QueryWriteStatusResponse

QueryWriteStatus() is used to find the committed_size for a resource that is being written, which can then be used as the write_offset for the next Write() call.

If the resource does not exist (i.e., the resource has been deleted, or the first Write() has not yet reached the service), this method returns the error NOT_FOUND.

The client may call QueryWriteStatus() at any time to determine how much data has been processed for this resource. This is useful if the client is buffering data and needs to know which data can be safely evicted. For any sequence of QueryWriteStatus() calls for a given resource name, the sequence of returned committed_size values will be non-decreasing.