buildgrid.server.cas.root_storage module

RootStorage

The consumer-facing view of a top-level storage, which resolves spliced blobs on every read.

A spliced blob stores no bytes of its own; it is recorded as an ordered list of chunk digests. Plain storage reads therefore miss it, and missing_blobs can report it present after one of its chunks is evicted. RootStorage looks up the chunk mapping before reading a digest large enough to have been spliced, and serves a spliced blob from its chunks.

This must run at the top-level storage: on a ShardedStorage a blob’s chunks may span shards, so reads route across the whole storage while the chunk mapping lives only on the shard that owns the blob digest. RootStorage is deliberately not a StorageABC, so no storage can contain it. Per-storage mapping state is delegated to the storage’s get_chunk_digests / bulk_get_chunk_digests / record_spliced_blob hooks.

It also backs the SpliceBlob and SplitBlob RPCs (splice_blob / split_blob). Request policy, such as whether a client is trusted to skip verification, stays with the CAS instance.

class buildgrid.server.cas.root_storage.RootStorage(storage: StorageABC)

Bases: object

The consumer-facing view of a top-level storage. Resolves spliced blobs on every read.

Built only by consumers, from the storage they were configured with. Never pass it into a storage, and never keep the raw storage alongside it.

property supports_chunking: bool

Whether the storage can record and resolve spliced-blob chunk mappings.

start() → None

stop() → None

get_blob(digest: Digest) → IO[bytes] | None

Return a readable, seekable handle to the blob, or None if it is missing.

A spliced blob is rebuilt into create_write_session, so a large one goes to a temporary file rather than memory.

bulk_read_blobs(digests: list[Digest]) → dict[str, bytes]

Return the bytes of the present blobs among digests, keyed by hash.

Spliced blobs are rebuilt in memory; everything else is read with a single storage call.

stream_read_blob(digest: Digest, chunk_size: int, offset: int = 0, limit: int = 0) → Iterator[bytes]

Return a generator that yields the blob in pieces no larger than chunk_size, honouring offset / limit.

Raises:

NotFoundError – the blob is missing. For a spliced blob this can be raised mid-stream when a chunk is missing, and its now-unreadable mapping is then deleted.

get_message(digest: Digest, message_type: type[_M]) → _M | None

Retrieve the Protobuf message with the given digest and type, or None if missing.

get_tree(root_digest: Digest, raise_on_missing_subdir: bool = False) → Iterator[Directory]

Walk the directory tree under root_digest, yielding the root directory first.

missing_blobs(digests: list[Digest]) → list[Digest]

Return the missing blobs among digests.

A spliced blob is present only when its mapping and every one of its chunks are present. A mapping that survives with missing chunks is deleted.

commit_write(digest: Digest, write_session: IO[bytes]) → None

stream_write_blob(digest: Digest, chunks: Iterator[bytes]) → None

bulk_update_blobs(blobs: list[tuple[Digest, bytes]]) → list[Status]

put_message(message: MessageType) → Digest

Store the given Protobuf message in CAS, returning its digest.

splice_blob(blob_digest: Digest, chunk_digests: Sequence[Digest], *, trusted: bool = False) → Digest

Verify that concatenating the chunks produces blob_digest, record the mapping, and return the computed digest.

Parameters:

trusted – skip reading the chunks, and only check their presence and declared total size. The idempotency and already-exists checks below are unaffected.

split_blob(blob_digest: Digest) → list[Digest]

Return the ordered chunk digests of a previously spliced blob.

A spliced blob is present only when its mapping row and every one of its chunks are present, checked with a single missing_blobs stat over the mapping and the already-fetched chunk digests that also refreshes their lifetimes.

Raises:

NotFoundError – the blob has no chunk mapping, or the mapping/one of its chunks is missing from storage (a surviving-but-unreconstructable mapping is dropped).