Blog>>Software development>>Bazel build system: build containerized applications faster

Bazel build system: build containerized applications faster

For the last nine months, I’ve been working on a large migration of multiple interconnected software projects from the CMake to the Bazel build system. While doing so, I’ve learned about Bazel’s capability for building and publishing container images.

What is Bazel?

Bazel is a multi-language build system with a strong focus on hermeticity, which means that when given the same source code, Bazel will produce the same build artifacts, even on different machines (if properly configured). This property allows you to set up a remote cache for the artifacts that you can share between your developers and continuous integration (CI) system to significantly speed up your workflow.

If you want to learn what CI/CD is, read our previous article.

Services test automation

Why should you use Bazel in software projects?

To illustrate, let’s say you have a project with 10 binary targets that get built automatically on each commit in your CI pipeline. When you introduce a change to one of the binaries, Bazel will rebuild only the binary that has changed and reuse the cached build for the other 9 binaries.

Now, let’s add an additional step to our imaginary CI pipeline that puts the binaries into containers and publishes them to the container registry (e.g. AWS ECR). It would be nice if we could build and publish only the image of the binary that changed, right? Bazel enables you to do just that!

You can find other examples of CI/CD pipeline tools in our previous article.

The goal of the article

Here is a simple binary target of a simple “Hello World” Go program defined in Bazel:

in BUILD file:

load("@io_bazel_rules_go//go:def.bzl", "go_binary")

go_binary(
    name = "app",
    srcs = ["app.go"],
)

In the next sections I’ll explain how to containerize such a binary using Bazel. I’ll show how to build, run and push the image.

NOTE: This article isn’t meant as a comprehensive guide to Bazel, please refer to the official documentation      link-icon for details.

Choosing a library

Typically, containers are built from a Dockerfile that contains all the commands to assemble an image. If we want to take advantage of Bazel’s caching, we will need to define the image using Bazel’s rules instead.

Bazel has a modular ecosystem and it doesn’t come with Docker support out of the box. There are two libraries we can choose from: 

For the purpose of this article, rules_oci will be used. OCI stands for Open Container Initiative and it’s a project formed under the Linux Foundation that creates open industry standards around container formats and runtimes. rules_oci doesn’t assume a Docker runtime, it produces OCI images that can be consumed by other runtimes such as podman      link-icon. Another advantage of rules_oci is that it supports multi-arch images.

Setup

Before we start, we have to download the dependencies. Copy the initialization code from the rules_go      link-icon, rules_pkg      link-icon and rules_oci      link-icon release pages into your WORKSPACE file.

Pulling a base image

We need to pull a base image for our application. Let’s use the oci*pull rule to pull a [golang](https://hub.docker.com/*/golang      link-icon) image from docker.io      link-icon.

in WORKSPACE file

load("@rules_oci//oci:pull.bzl", "oci_pull")

oci_pull(
    name = "go_base",
    image = "index.docker.io/library/golang",
    digest = "sha256:690e4135bf2a4571a572bfd5ddfa806b1cb9c3dea0446ebadaf32bc2ea09d4f9",
    platforms = [
        "linux/amd64",
    ],
)

Creating an image in build files

Now, create an app_image target for the app binary we defined before:

in BUILD file

load("@rules_pkg//:pkg.bzl", "pkg_tar")
load("@rules_oci//oci:defs.bzl", "oci_image")

pkg_tar(
    name = "app_layer",
    srcs = [":app"],
    package_dir = "/",
)

oci_image(
    name = "app_image",
    base = "@go_base",
    tars = [":app_layer"],
    entrypoint = ["/app"]
)

Images are composed with layers. Each layer is a .tar file that contains a diff of the filesystem changes. We used a pkg_tar      link-icon rule to define a layer that will hold our app binary.

oci_image rule can be used to define an image composed with many layers. This is useful when you want to embed multiple binary, data or configuration files in the image. Refer to the documentation      link-icon to see all the fields supported by oci_image rule.

Running the image locally

An OCI image consists of an image manifest      link-icon, image configuration      link-icon, and one or more filesystem serializations      link-icon. You can see what it looks like under the bazel-bin/app_image directory. An image in this form is not directly loadable by runtimes. We will need to pack it into a tarball.

in BUILD file

load("@rules_oci//oci:defs.bzl", "oci_tarball")

oci_tarball(
    name = "app_tarball",
    image = ":app_image",
    repo_tags = ["app:latest"]
)

Next, we need to build the tarball, load it into Docker runtime and run the image.

$ bazel build app_tarball
$ docker load < bazel-bin/app_tarball/tarball.tar
$ docker run --rm app:latest
hello world

Publishing the image

in BUILD file

load("@rules_oci//oci:defs.bzl", "oci_push")

oci_push(
    name = "app_push",
    image = ":app_image",
    repository = "index.docker.io/wwieclaw/hello",
    remote_tags = ["latest"]
)

Simplify this process using macros

rules_oci is a barebones, low-level library by design. If you have many images that you want to build in a similar manner, you could write a macro that abstracts away all the nitty-gritty details.

Here is an example go_image macro:

in bazel/go_image.bzl file

load("@io_bazel_rules_go//go:def.bzl", "go_binary")
load("@rules_pkg//:pkg.bzl", "pkg_tar")
load("@rules_oci//oci:defs.bzl", "oci_image")

def go_image(name, base = "@go_base", tars = [], **kwargs):
    '''
    Creates a containerized binary from Go sources.
    Parameters:
        name:  name of the image
        base: base image
        tars: additional image layers
        kwargs: arguments passed to the go_binary target
    '''
    binary_name = "{}_binary".format(name)
    layer_name = "{}_layer".format(name)

    go_binary(
        name = binary_name,
        **kwargs
    )

    pkg_tar(
        name = layer_name,
        srcs = [binary_name],
        package_dir = "/",
    )

    oci_image(
        name = name,
        tars = [layer_name] + tars,
        entrypoint = ["/{}".format(binary_name)],
        base = base,
    )

in BUILD file

load("//bazel:go_image.bzl", "go_image")

go_image(
    name = "app",
    srcs = ["app.go"],
)

Summary

There are important topics that I didn’t touch upon in this article such as stamping and building multi-arch images. I strongly recommend joining the Bazel Slack Channel      link-icon, there are a lot of examples and knowledgeable people that are willing to help.

References

Więcław Wiktor

Wiktor Więcław

Software Engineer

Wiktor Więcław is a software engineer and author on CodiLime's blog. Check out the author's articles on the blog. Read about author >

Read also

Get your project estimate

For businesses that need support in their software or network engineering projects, please fill in the form and we'll get back to you within one business day.