buildgrid.server.cas.root_storage module

RootStorage

The consumer-facing view of a top-level storage, which resolves spliced blobs on every read.

A spliced blob stores no bytes of its own; it is recorded as an ordered list of chunk digests. Plain storage reads therefore miss it, and missing_blobs can report it present after one of its chunks is evicted. RootStorage looks up the chunk mapping before reading a digest large enough to have been spliced, and serves a spliced blob from its chunks.

This must run at the top-level storage: on a ShardedStorage a blob’s chunks may span shards, so reads route across the whole storage while the chunk mapping lives only on the shard that owns the blob digest. RootStorage is deliberately not a StorageABC, so no storage can contain it. Per-storage mapping state is delegated to the storage’s get_chunk_digests / bulk_get_chunk_digests / record_spliced_blob hooks.

It also backs the SpliceBlob and SplitBlob RPCs (splice_blob / split_blob). Request policy, such as whether a client is trusted to skip verification, stays with the CAS instance.

An optional TreeCache, which may be shared by multiple RootStorage objects, maps root directory digests to Tree messages stored in CAS, so get_tree can serve a previously walked tree with a single read.

class buildgrid.server.cas.root_storage.RootStorage(storage: StorageABC, *, tree_cache: TreeCache | None = None)

Bases: object

The consumer-facing view of a top-level storage. Resolves spliced blobs on every read.

Built only by consumers, from the storage they were configured with. Never pass it into a storage, and never keep the raw storage alongside it.

property supports_chunking: bool

Whether the storage can record and resolve spliced-blob chunk mappings.

start() → None

stop() → None

get_blob(digest: Digest) → IO[bytes] | None

Return a readable, seekable handle to the blob, or None if it is missing.

A spliced blob is rebuilt into create_write_session, so a large one goes to a temporary file rather than memory.

bulk_read_blobs(digests: list[Digest]) → dict[str, bytes]

Return the bytes of the present blobs among digests, keyed by hash.

Spliced blobs are rebuilt in memory; everything else is read with a single storage call.

stream_read_blob(digest: Digest, chunk_size: int, offset: int = 0, limit: int = 0) → Iterator[bytes]

Return a generator that yields the blob in pieces no larger than chunk_size, honouring offset / limit.

Raises:

NotFoundError – the blob is missing. For a spliced blob this can be raised mid-stream when a chunk is missing, and its now-unreadable mapping is then deleted.

get_message(digest: Digest, message_type: type[_M]) → _M | None

Retrieve the Protobuf message with the given digest and type, or None if missing.

get_tree(root_digest: Digest, raise_on_missing_subdir: bool = False) → Iterator[Directory]

Walk the directory tree under root_digest, yielding the root directory first.

missing_blobs(digests: list[Digest]) → list[Digest]

Return the missing blobs among digests.

A spliced blob is present only when its mapping and every one of its chunks are present. A mapping that survives with missing chunks is deleted.

commit_write(digest: Digest, write_session: IO[bytes]) → None

stream_write_blob(digest: Digest, chunks: Iterator[bytes]) → None

bulk_update_blobs(blobs: list[tuple[Digest, bytes]]) → list[Status]

Store the given blobs, returning a status for each in the order of blobs.

put_message(message: MessageType) → Digest

Store the given Protobuf message in CAS, returning its digest.

splice_blob(blob_digest: Digest, chunk_digests: Sequence[Digest], *, trusted: bool = False) → Digest

Verify that concatenating the chunks produces blob_digest, record the mapping, and return the computed digest.

Parameters:

trusted – skip reading the chunks, and only check their presence and declared total size. The idempotency and already-exists checks below are unaffected.

split_blob(blob_digest: Digest) → list[Digest]

Return the ordered chunk digests of a previously spliced blob.

A spliced blob is present only when its mapping row and every one of its chunks are present, checked with a single missing_blobs stat over the mapping and the already-fetched chunk digests that also refreshes their lifetimes.

Raises:

NotFoundError – the blob has no chunk mapping, or the mapping/one of its chunks is missing from storage (a surviving-but-unreconstructable mapping is dropped).