.. _using-logstream:

Using the LogStream service
===========================

BuildGrid's LogStream service can be used both for streaming stdout/stderr
logs from Actions in a BuildGrid execution service, and as a more generic
standalone LogStream service implementation.


Streaming build logs
--------------------

If deployed as part of a larger BuildGrid remote execution service, the
LogStream service will be used by BuildBox workers to stream a combined
stdout/stderr log. The resource names are contained in the ``metadata``
field of the ``Operation`` messages sent in response to ``Execute`` or
``WaitExecution`` requests.

The resource names can then be used by a ByteStream client to make a
``Read`` request for the resource to the LogStream service. Once an
Action has finished executing, there is no guarantee that the stream
will exist, and using the stdout/stderr inlined in the ActionResult
(or the relevant digests to get logs from CAS) should be preferred.

One such ByteStream client is provided as part of BuildGrid,

.. code-block:: sh

    bgd logstream read resource-name

An alternative is the ``logstream-tail`` tool, which is `part of
buildbox-tools`_.

.. _part of buildbox-tools: https://gitlab.com/BuildGrid/buildbox/buildbox-tools/-/tree/master/cpp/logstreamtail


Standalone LogStream service
----------------------------

When deployed as a standalone service, the same broad concepts hold.
However, there will need to be a way to create streams (using a
``CreateLogStream`` request), and communicate the read and write
resource names to clients who need them.

There is a BuildGrid command to create a stream,

.. code-block:: sh

    bgd logstream create some-identifier-to-include-in-the-name

Once a stream has been created, it can be written to by making a
ByteStream ``Write`` request to the LogStream server using the
writeable resource name returned by ``CreateLogStream``.

There is also a BuildGrid command to get an interactive prompt
for writing to a stream,

.. code-block:: sh

    bgd logstream write write-resource-name

When using this tool sending an empty message will finalize and
commit the stream, which will cause the stream to be deleted when
all readers are finished reading.

A more generally useful tool for use with a standalone LogStream
is the ``outputstreamer`` tool from `buildbox-tools`_. This streams
the contents of stdin to a LogStream server's ByteStream endpoint.

.. _buildbox-tools: https://gitlab.com/BuildGrid/buildbox/buildbox-tools/-/tree/master/cpp/outputstreamer

The buildgrid-logstream repository contains a simple docker-compose file containing
a standalone LogStream server for demo/experimentation purposes. This
can be spun up using (in the root of the repository)

.. code-block:: sh
    git clone https://gitlab.com/BuildGrid/buildgrid-logstream
    docker-compose up

This serves the LogStream and ByteStream APIs at http://localhost:50051.