buildgrid.utils module

class buildgrid.utils.BrowserURL(base_url, instance_name=None)

Bases: object

for_message(message_type, message_digest)

Completes browser URL initialization for a protobuf message.

generate()

Generates a browser URL string.

class buildgrid.utils.Condition(lock=None)

Bases: threading.Condition

Subclass of threading.Condition with wait overridden.

In this implementation, wait only releases the lock if other threads are actually waiting for the lock, otherwise it does nothing.

wait(timeout=None)

Wait if other threads are trying to acquire the lock.

If other threads have attempted to acquire the lock for this Condition using with, this method behaves the same as wait on a regular threading.Condition.

If only one thread has attempted to acquire the lock, then that thread must be the current thread. In that case, this method doesn’t release the lock or wait at all, it simply returns True as if it had been woken by threading.Condition.notify.

class buildgrid.utils.TypedEvent

Bases: object

Wrapper around a threading.Event to support event ‘types’

set(type=None)
clear()
notify_change()
notify_stop()
wait(last_received=None)
class buildgrid.utils.JobState(job)

Bases: object

class buildgrid.utils.JobWatchSpec(job)

Bases: object

Structure to track what operations are being watched for a given job.

This also contains the event used for notifying watchers of changes, and the state that the job was in after a change was last detected.

property peers
peers_for_operation(operation_name)

Returns a copy of the list of peers for the given operation.

If the operation is not being watched, or for some reason has no “peers” key, the empty list is returned.

Parameters

operation_name (string) – The name of the operation to get the list of peers for.

add_peer(operation_name, peer)

Add a peer to the set of peers watching the job this spec is for.

Takes an operation name and a peer and tracks that the peer is watching the given operation.

Parameters
  • operation_name (string) – The name of the operation that the peer is watching for updates on.

  • peer (string) – The peer that is starting to watch for updates.

remove_peer(operation_name, peer)

Remove a peer from the list watching an operation for this job.

The inverse of add_peer. Takes an operation name and a peer and removes that peer from the list of peers watching that operation. If this leaves the operation with no peers watching it, the operation is removed from the JobWatchSpec.

Parameters
  • operation_name (string) – The name of the operation that is no longer being watched by the peer.

  • peer (string) – The name of the peer that is stopping watching.

buildgrid.utils.get_hostname()

Returns the hostname of the machine executing that function.

Returns

Hostname for the current machine.

Return type

str

buildgrid.utils.get_hash_type()

Returns the hash type.

buildgrid.utils.create_digest(bytes_to_digest)

Computes the Digest of a piece of data.

The Digest of a data is a function of its hash and size.

Parameters

bytes_to_digest (bytes) – byte data to digest.

Returns

The Digest for the given byte data.

Return type

Digest

buildgrid.utils.parse_digest(digest_string)

Creates a Digest from a digest string.

A digest string should alway be: {hash}/{size_bytes}.

Parameters

digest_string (str) – the digest string.

Returns

The Digest read from the string or None if

digest_string is not a valid digest string.

Return type

Digest

buildgrid.utils.read_file(file_path)

Loads raw file content in memory.

Parameters

file_path (str) – path to the target file.

Returns

Raw file’s content until EOF.

Return type

bytes

Raises

OSError – If file_path does not exist or is not readable.

buildgrid.utils.write_file(file_path, content)

Dumps raw memory content to a file.

Parameters
  • file_path (str) – path to the target file.

  • content (bytes) – raw file’s content.

Raises

OSError – If file_path does not exist or is not writable.

buildgrid.utils.merkle_tree_maker(directory_path)

Walks a local folder tree, generating FileNode and DirectoryNode.

Parameters

directory_path (str) – absolute or relative path to a local directory.

Yields

Message, bytes, str – a tutple of either a FileNode or DirectoryNode message, the corresponding blob and the corresponding node path.

buildgrid.utils.output_file_maker(file_path, input_path, file_digest)

Creates an OutputFile from a local file and possibly upload it.

Note

file_path must point inside or be relative to input_path.

Parameters
  • file_path (str) – absolute or relative path to a local file.

  • input_path (str) – absolute or relative path to the input root directory.

  • file_digest (Digest) – the underlying file’s digest.

Returns

a new OutputFile object for the file pointed by file_path.

Return type

OutputFile

buildgrid.utils.output_directory_maker(directory_path, working_path, tree_digest)

Creates an OutputDirectory from a local directory.

Note

directory_path must point inside or be relative to input_path.

Parameters
  • directory_path (str) – absolute or relative path to a local directory.

  • working_path (str) – absolute or relative path to the working directory.

  • tree_digest (Digest) – the underlying folder tree’s digest.

Returns

a new OutputDirectory for the directory pointed by directory_path.

Return type

OutputDirectory

buildgrid.utils.convert_values_to_sorted_lists(dictionary: Dict[str, Union[str, Sequence[str]]]) → None

Given a dictionary, do the following:

  1. Turn strings into singleton lists

  2. Turn all other sequence types into sorted lists with list()

This is a mutating operation. No value is returned.

buildgrid.utils.hash_from_dict(dictionary: Dict[str, List[str]]) → str

Get the hash represntation of a dictionary