What is buildbox?¶
buildbox is a set of tools for remote worker build execution, conforming to the remote workers api, and the remote execution api. It is highly configurable and extensible while also having support for Linux, AIX, and Solaris. To learn more about buildbox, its architecture and how to build the various tools, please visit the buildbox landing page.
This section will explain how to connect a simple
buildbox-worker to BuildGrid.
buildbox-worker is a worker written in C++ that implements the Remote Workers API.
buildbox-worker runs its underlying job using a pluggable “runner.” For this example,
we will be using the
buildbox-run-hosttools runner, which simply runs the command
you send to it directly with no sandboxing.
buildbox-run-hosttools. Instructions for each can be
found in the respective READMEs. As part of the build process, you will also need to build
buildbox-common, which has several dependencies that need to be installed.
Then, start a BuildGrid instance. If you have followed the installation guide and also have the BuildGrid source on your machine, you can use the default configuration:
bgd server start data/config/default.conf
Running the worker is pretty simple –you just need to pass it the locations of the
BuildGrid server, BuildGrid CAS server, and
buildbox-run-hosttools. For example,
if your server has an execution service and a CAS service both located at localhost:50051,
you can invoke the worker like this:
/path/to/buildbox-worker --bots-remote=http://localhost:50051 --cas-remote=http://localhost:50051 --buildbox-run=/path/to/buildbox-run-hosttools my_bot
If all goes well, you should have a functioning worker named
my_bot that’s ready to accept jobs.
Using a local cache (buildbox-casd)¶
Currently, whenever a worker needs to fetch blobs, it must make a request to the
CAS is most likely on a separate machine, this can be an expensive operation.
The worker’s performance can be greatly improved if there is a local cache which minimized
requests to the
buildbox-casd acts as a local cache; serving as a proxy to
buildbox-worker. If it doesn’t have a blob which the worker requests, it will
fetch it from
CAS, cache it, and then provide it to the worker. Consequently, future
requests for the blob (by any workers on the machine) will hit
buildbox-casd's local cache
instead of the remote
You will first need to build
buildbox-casd. Instructions for how to do this can be found
on it’s README. Note that
buildbox-casd also depends on
You can then launch buildbox casd with the following command:
/path/to/buildbox-casd --cas-remote=http://localhost:50051 --bind=127.0.0.1:50011 /location/to/store/local/cache
You can then launch the worker, using the following command:
/path/to/buildbox-worker --bots-remote=http://localhost:50051 --cas-remote=http://127.0.0.1:50011 --buildbox-run=/path/to/buildbox-run-hosttools --runner-arg=--prefix-staged-dir my_bot
You can additionally tell the worker where to run the action with the workspace path argument:
/path/to/buildbox-worker --bots-remote=http://localhost:50051 --cas-remote=http://127.0.0.1:50011 --buildbox-run=/path/to/buildbox-run-hosttools --runner-arg=--prefix-staged-dir --runner-arg=--workspace-path=/stage/location my_bot
You should now have local caching enabled on your worker setup.
Enabling the local cas protocol¶
You can further optimize the local caching by enabling the local cas protocol. In order to understand what this does, we will need some definitions.
Stage means placing the files needed to execute your action.
Capture refers to saving the action’s outputs.
Enabling the local cas protocol means that
buildbox-casd will stage and capture
the files instead of
buildbox-worker. The benefit of this is that
buildbox-casd can do
this much more quickly and efficiently, resulting in a huge performance gain.
In order to do this safely you will need to run
separate users. This prevents the actions from corrupting the local cache.
To launch buildbox-casd, use the following command:
sudo -u casduser /path/to/buildbox-casd --cas-remote=http://localhost:50051 --bind=127.0.0.1:50011 /location/to/store/local/cache
To launch buildbox-worker, with the local cas protocol enabled, run:
sudo -u workeruser /path/to/buildbox-worker --bots-remote=http://localhost:50051 --cas-remote=http://127.0.0.1:50011 --buildbox-run=/path/to/buildbox-run-hosttools --runner-arg=--use-localcas --runner-arg=--prefix-staged-dir my_bot
The local cas protocol should now be enabled. Enjoy your super fast builds!
Sandboxed workers (buildbox-run-userchroot)¶
So far, we have been using
buildbox-run-hosttools as our runner.
While this is fine in simple cases, it presents a serious security risk. When using
buildbox-run-hosttools, actions have full access to the entire filesystem of the machine.
This means that they can corrupt the files on the machine, access potentially sensitive
information, or just leave behind unwanted files.
buildbox-run-userchroot, you automatically use userchroot to sandbox the actions, making the
entire system much safer. To build
buildbox-run-userchroot please refer to the repo’s README. Please
note that you will need userchroot installed and configured on your machine in order to use it.
It is highly recommeded that you are using
buildbox-casd with the local cas protocol
enabled before using
buildbox-run-userchroot. Otherwise it can be very slow.
buildbox-run-userchroot you will need to upload a chroot image to
get its digest. You can use casupload to upload chroot images and get their respective root digests.
Refer to the README on recc to build casupload; building recc will automatically build casupload.
You will need to specify the root digest as a platform property on the client side,
buildgrid’s config file, and in the workers’ arguments. To launch a worker with
enabled, run the following command:
sudo -u workeruser /path/to/buildbox-worker --bots-remote=http://localhost:50051 --cas-remote=http://127.0.0.1:50011 --buildbox-run=/path/to/buildbox-run-userchroot --platform chrootRootDigest=root-digest-of-chroot --runner-arg=--use-localcas --runner-arg=--workspace-path=/path/to/userchroot/base my_bot
Congratulations! You now have sandboxed workers!