buildgrid.server.cas.split_splicer module

SplitSplicer

Top-level orchestration for the SpliceBlob (and, in future, SplitBlob) RPCs, and for reconstructing spliced blobs on read.

A spliced blob stores no bytes of its own; it is recorded as an ordered list of chunk digests. On write, splice_blob verifies the chunks assemble to the expected digest and records the mapping; on a read miss, the CAS and ByteStream instances call back here to reassemble the bytes from those chunks (bulk_read_spliced_blobs / stream_spliced_blob).

Both paths must run at the top-level storage: on a ShardedStorage a blob’s chunks may span shards, so reads route across the whole storage while the chunk mapping lives only on the shard that owns the blob digest. Per-storage mapping state is delegated to the storage’s get_chunk_digests / bulk_get_chunk_digests / record_spliced_blob hooks.

class buildgrid.server.cas.split_splicer.SplitSplicer(storage: StorageABC)

Bases: object

splice_blob(blob_digest: Digest, chunk_digests: Sequence[Digest]) → Digest

Verify that concatenating the chunks produces blob_digest, record the mapping, and return the computed digest.

split_blob(blob_digest: Digest) → list[Digest]

Return the ordered chunk digests of a previously spliced blob.

A spliced blob is present only when its mapping row and every one of its chunks are present, checked with a single missing_blobs stat over the mapping and the already-fetched chunk digests that also refreshes their lifetimes.

Raises:

NotFoundError – the blob has no chunk mapping, or the mapping/one of its chunks is missing from storage (a surviving-but-unreconstructable mapping is dropped).

bulk_read_spliced_blobs(blob_digests: Sequence[Digest]) → dict[str, bytes]

Reconstruct the spliced blobs among blob_digests, keyed by blob hash.

Non-spliced (or unknown) digests are omitted. A blob whose chunks can no longer be gathered is dropped from the result and its now-unreadable mapping is deleted.

bulk_find_missing_blobs(blob_digests: Sequence[Digest]) → list[Digest]

Return the missing blobs among blob_digests.

Non-spliced blobs are checked directly against storage; spliced blobs are resolved to their chunks and have chunk existence validated. Any spliced blob mappings which exist but reference missing chunks are cleaned up.

Parameters:

blob_digests (Sequence[Digest]) – Sequence of digests to validate existence of, must already be deduplicated by hash.

bulk_delete_spliced_blobs(blob_digests: Sequence[Digest]) → None

Delete the chunk mappings for the given spliced blobs whose chunks can no longer be gathered: the mapping is unreadable, so it is removed. The single delete primitive shared by every splice path (FindMissingBlobs and the read fallbacks).

Deletion routes to the splice-aware owning storage, which drops the metadata and chunk mapping without attempting to delete non-existent backing bytes.

stream_spliced_blob(blob_digest: Digest, chunk_size: int, offset: int = 0, limit: int = 0) → Iterator[bytes]

Stream a spliced blob’s bytes in pieces no larger than chunk_size, honoring offset / limit.

Raises:

NotFoundError – if blob_digest is not a spliced blob, or if a chunk overlapping the requested range is missing from storage (its now-unreadable mapping is then deleted).