buildgrid.server.cas.split_splicer module

SplitSplicer

Top-level orchestration for the SpliceBlob (and, in future, SplitBlob) RPCs, and for reconstructing spliced blobs on read.

A spliced blob stores no bytes of its own; it is recorded as an ordered list of chunk digests. On write, splice_blob verifies the chunks assemble to the expected digest and records the mapping; on a read miss, the CAS and ByteStream instances call back here to reassemble the bytes from those chunks (bulk_read_spliced_blobs / stream_spliced_blob). An authorized client that derived the chunk list from the blob’s own bytes may have that verification skipped, in which case only the chunks’ presence and declared total size are checked.

Both paths must run at the top-level storage: on a ShardedStorage a blob’s chunks may span shards, so reads route across the whole storage while the chunk mapping lives only on the shard that owns the blob digest. Per-storage mapping state is delegated to the storage’s get_chunk_digests / bulk_get_chunk_digests / record_spliced_blob hooks.

class buildgrid.server.cas.split_splicer.SplitSplicer(storage: StorageABC, *, chunked_by_buildbox_allowed_subjects: list[str] | None = None)

Bases: object

splice_blob(blob_digest: Digest, chunk_digests: Sequence[Digest], *, client_chunked: bool = False) → Digest

Verify that concatenating the chunks produces blob_digest, record the mapping, and return the computed digest.

Parameters:

client_chunked – whether the request carried the chunked-by-buildbox header. Verification is skipped only when the authenticated client’s subject is in this splicer’s allowlist. The idempotency and already-exists checks below are unaffected.

split_blob(blob_digest: Digest) → list[Digest]

Return the ordered chunk digests of a previously spliced blob.

A spliced blob is present only when its mapping row and every one of its chunks are present, checked with a single missing_blobs stat over the mapping and the already-fetched chunk digests that also refreshes their lifetimes.

Raises:

NotFoundError – the blob has no chunk mapping, or the mapping/one of its chunks is missing from storage (a surviving-but-unreconstructable mapping is dropped).

bulk_read_spliced_blobs(blob_digests: Sequence[Digest]) → dict[str, bytes]

Reconstruct the spliced blobs among blob_digests, keyed by blob hash.

Non-spliced (or unknown) digests are omitted. A blob whose chunks can no longer be gathered is dropped from the result and its now-unreadable mapping is deleted.

bulk_find_missing_blobs(blob_digests: Sequence[Digest]) → list[Digest]

Return the missing blobs among blob_digests.

Non-spliced blobs are checked directly against storage; spliced blobs are resolved to their chunks and have chunk existence validated. Any spliced blob mappings which exist but reference missing chunks are cleaned up.

Parameters:

blob_digests (Sequence[Digest]) – Sequence of digests to validate existence of, must already be deduplicated by hash.

bulk_delete_spliced_blobs(blob_digests: Sequence[Digest]) → None

Delete the chunk mappings for the given spliced blobs whose chunks can no longer be gathered: the mapping is unreadable, so it is removed. The single delete primitive shared by every splice path (FindMissingBlobs and the read fallbacks).

Deletion routes to the splice-aware owning storage, which drops the metadata and chunk mapping without attempting to delete non-existent backing bytes.

stream_spliced_blob(blob_digest: Digest, chunk_size: int, offset: int = 0, limit: int = 0) → Iterator[bytes]

Stream a spliced blob’s bytes in pieces no larger than chunk_size, honoring offset / limit.

Raises:

NotFoundError – if blob_digest is not a spliced blob, or if a chunk overlapping the requested range is missing from storage (its now-unreadable mapping is then deleted).