buildgrid.server.sql.utils module

Holds constants and utility functions for the SQL scheduler.

class buildgrid.server.sql.utils.SortKeySpec(column_name, table_name)

Bases: tuple

column_name

Alias for field number 0

table_name

Alias for field number 1

buildgrid.server.sql.utils.is_sqlite_connection_string(connection_string: str) → bool

buildgrid.server.sql.utils.is_psycopg2_connection_string(connection_string: str) → bool

buildgrid.server.sql.utils.is_sqlite_inmemory_connection_string(full_connection_string: str) → bool

class buildgrid.server.sql.utils.SQLPoolDisposeHelper(cooldown_time_in_secs: int, cooldown_jitter_base_in_secs: int, min_time_between_dispose_in_minutes: int, sql_engine: Engine)

Bases: object

Helper class for disposing of SQL session connections

check_dispose_pool(session: Session, e: Exception) → bool

For selected exceptions invalidate the SQL session - returns True when a transient sql connection error is detected - returns False otherwise

time_until_active_pool() → timedelta

The time at which the pool is expected to become active after a pool disposal. This adds small amounts of jitter to help spread out load due to retrying clients

buildgrid.server.sql.utils.strtobool(val: str) → bool

Convert a string representation of truth to true (1) or false (0). True values are ‘y’, ‘yes’, ‘t’, ‘true’, ‘on’, and ‘1’; false values are ‘n’, ‘no’, ‘f’, ‘false’, ‘off’, and ‘0’. Raises ValueError if ‘val’ is anything else.

buildgrid.server.sql.utils.parse_list_operations_sort_value(value: str, column: InstrumentedAttribute[Any]) → Any

Convert the string representation of a value to the proper Python type.

buildgrid.server.sql.utils.dump_list_operations_token_value(token_value: Any) → str

Convert a value to a string for use in the page_token.

buildgrid.server.sql.utils.build_pagination_clause_for_sort_key(sort_value: Any, previous_sort_values: list[Any], sort_keys: list[buildgrid.server.operations.filtering.filter.SortKey]) → ColumnElement[bool]

Build part of a filter clause to figure out the starting point of the page given by the page_token. See the docstring of build_page_filter for more details.

buildgrid.server.sql.utils.build_page_filter(page_token: str, sort_keys: list[buildgrid.server.operations.filtering.filter.SortKey]) → ColumnElement[bool]

Build a filter to determine the starting point of the rows to fetch, based on the page_token.

The page_token is directly related to the sort order, and in this way it acts as a “cursor.” It is given in the format Xval|Yval|Zval|…, where each element is a value corresponding to an orderable column in the database. If the corresponding rows are X, Y, and Z, then X is the primary sort key, with Y breaking ties between X, and Z breaking ties between X and Y. The corresponding filter clause is then:

(X > Xval) OR (X == XVal AND Y > Yval) OR (X == Xval AND Y == Yval AND Z > Zval) …

buildgrid.server.sql.utils.build_page_token(operation: OperationEntry, sort_keys: list[buildgrid.server.operations.filtering.filter.SortKey]) → str

Use the sort keys to build a page token from the given operation.

buildgrid.server.sql.utils.extract_sort_keys(operation_filters: list[buildgrid.server.operations.filtering.filter.OperationFilter]) → tuple[list[buildgrid.server.operations.filtering.filter.SortKey], list[buildgrid.server.operations.filtering.filter.OperationFilter]]

Splits the operation filters into sort keys and non-sort filters, returning both as separate lists.

Sort keys are specified with the “sort_order” parameter in the filter string. Multiple “sort_order”s can appear in the filter string, and all are extracted and returned.

buildgrid.server.sql.utils.build_sort_column_list(sort_keys: list[buildgrid.server.operations.filtering.filter.SortKey]) → list[sqlalchemy.sql.elements.UnaryExpression[Any]]

Convert the list of sort keys into a list of columns that can be passed to an order_by.

This function checks the sort keys to ensure that they are in the parameter-model map and raises an InvalidArgumentError if they are not.

buildgrid.server.sql.utils.convert_filter_to_sql_filter(operation_filter: OperationFilter) → ColumnElement[bool]

Convert the internal representation of a filter to a representation that SQLAlchemy can understand. The return type is a “ColumnElement,” per the end of this section in the SQLAlchemy docs: https://docs.sqlalchemy.org/en/13/core/tutorial.html#selecting-specific-columns

This function assumes that the parser has appropriately converted the filter value to a Python type that can be compared to the parameter.

buildgrid.server.sql.utils.build_custom_filters(operation_filters: list[buildgrid.server.operations.filtering.filter.OperationFilter]) → list[sqlalchemy.sql.elements.ColumnElement[bool]]