.. _indexed-cas:

Indexed CAS
===========

To improve the performance of CAS-related requests like ``FindMissingBlobs``,
it is possible to configure an "indexed" CAS. This also facilitates intelligent
cleanup of blobs, currently only for S3.

This index keeps track of all the blobs that are currently stored in CAS, as
well as the last time they were accessed at.


Configuration
-------------

In a configuration file, the CAS index acts just like another storage backend
definition. It fully implements the same API interface as the other storage
backends, so you can just pass it to the service definition as with any
other storage implementation.

For example, here is a basic disk-based CAS configuration

.. literalinclude:: ../data/basic-disk-cas.yml
   :language: yaml

To add an index to this, we simply need to add a CAS index to the list of
storages, point it at the disk storage, and then use the index in the service
definitions rather than the disk storage.

.. literalinclude:: ../data/postgresql-index-cas-only.yml
   :language: yaml

.. warning::

   If adding an index to an existing storage which is used by another service
   which stores digests, such as an ActionCache or Asset API, either those
   services need to be cleared or the ``fallback_on_get`` config option needs
   to be temporarily set to True on the index.

   By default, the index will report that a blob is missing if it isn't in the
   index, whether it exists in storage or not. ``fallback_on_get`` is a pretty
   big burden on performance so is off by default, and should ideally not be
   left on permanently.


Index Types
-----------

The CAS index implementation is pluggable, similarly to the storage
implementation. Currently both Redis and SQL are supported as storage indices.

Redis
'''''

The Redis Index uses Redis to store TTLs for CAS objects. The keys themselves
are used to determine existence of a blob in CAS, and the TTL of the key is
set to the expiry time of the blob. The value is set to a dummy value of 1,
since we only care about existence checking and TTLs.

.. autoclass:: buildgrid.server.app.settings.parser.RedisIndex
   :noindex:


SQL
'''

As the name suggests, the SQL index uses a PostgreSQL database to store the index.

The storage instance and connection string must be provided, but the other
parameters have defaults that should be functional.

.. autoclass:: buildgrid.server.app.settings.parser.SQL_Index
   :noindex: