Adam McKay: TIL

Using Docker BuildKit to Handle Secrets and SSH Keys During Build

#docker

I needed to install a private Python package from a Git repository during a Docker build, which required passing an SSH key. Rather than copying the key into the image (where it would remain in the layer history) or using a multi-stage build with it copied around, I found that Docker BuildKit provides secure ways to handle this.

BuildKit offers two main approaches for this: a specialized SSH mount and a general-purpose secret mount.

Using an SSH Mount (type=ssh)

For handling SSH keys, the easiest method is to use an SSH mount (--mount=type=ssh). This forwards the host’s SSH agent connection to the builder, so your private key never leaves your machine, and avoids exposing the key file directly in the Docker build.

Dockerfile with SSH Mount:

# syntax=docker/dockerfile:1
FROM python:3.12

# Add github.com to known_hosts to avoid interactive prompt
RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts

# Mount the SSH agent socket during the RUN command to install the package
RUN --mount=type=ssh \
    pip install git+ssh://[email protected]/private/repo.git

Build with SSH agent forwarding:

To build, you need to have BuildKit enabled (default on recent Docker versions) and tell Docker to forward the SSH agent.

# If your SSH key is loaded in the default agent
docker build --ssh default .

This command makes the SSH agent available only to RUN commands that explicitly request it with --mount=type=ssh.

Using a Secret Mount (type=secret)

For general-purpose secrets like API tokens or passwords, the type=secret mount method is necessary. It can expose a secret as an environment variable or a file, again, available only for the RUN command that requests it.

Example: Using a PyPI Token from an Environment Variable

This is a common scenario, for instance, when downloading packages from a private Python package index that requires an API token.

Dockerfile:

# syntax=docker/dockerfile:1
FROM python:3.12

# The PYPI_TOKEN var is only available during this RUN command
RUN --mount=type=secret,id=pypi_token,env=PYPI_TOKEN \
    pip install \
      --index-url https://__token__:[email protected]/simple \
      my-private-package

Build with secret:

To build, you pass the secret from a host environment variable. The id in the build command (pypi_token) must match the id in the Dockerfile.

# Export the token on your host machine
export MY_PYPI_TOKEN="pypi_xxxxxxxxxxxx"

# Pass the host variable as a secret to the build
docker build --secret id=pypi_token,env=MY_PYPI_TOKEN .

The PYPI_TOKEN variable is available only to RUN command’s environment and is not persisted in any image layer.

Alternative: Mounting a secret as a file

You can also mount a secret as a file.

# Pass secret from a file on the host
docker build --secret id=terraform_config,src=~/config.tfvars .

The Dockerfile would then mount this secret to a target path:

RUN --mount=type=secret,id=terraform_config,target=/var/config.tfvars,mode=0644 \
    terraform init
←Top