Understanding Docker Build Args, Environment Variables and Docker Compose Variables

UPDATE: There’s a new, improved article on this topic - Docker ARG, ENV and .env, a complete guide. Check it out, or continue with this one if you like :)

When working with Docker, you’ll stumble over lots of confusing details. I struggled with understanding how to pass variables to Docker images, and how to configure dockerized applications properly.

Do these sound familiar?

• How can you set variables during build time?
• Who gets what environment vars, and what can you do to change them?
• How to override defaults or set your own in your Docker files?

In this article, I’d like to share a focused view of what I learned in the process.

Let’s do a deep dive into Docker environment and build-time variables, and how they can be used to build images and running Docker containers. Both with the Docker CLI, as well as with docker-compose.

Before We Do

This is a long, in-depth article. I understand that you might be busy and not looking to learn all there is to know, but fix your current problem. Here are shorter articles, which will help you make progress as quickly as possible:

• Understand the difference between ARG and ENV
• Set default ENV values dynamically during the image build
• Use the output of a command during your Docker image build in subsequent steps
• Debug a broken docker-compose service which crashes right away

If you’re kinda new to the topic or not firm in the basics, you might want to invest a bit of time into fixing that. It’s a quick read and you will have an easier time using Docker.

Docker Image Build-time Variables

You can define variables inside of a Dockerfile, to help you not to repeat yourself.

(Dockerfile:)

ARG some_variable_name
# or with a default:
#ARG some_variable_name=default_value

RUN echo "Oh dang look at that $some_variable_name"
# or with ${some_variable_name}

relevant docs

As you see, we tell the Dockerfile to let Docker know, that it expects a variable named some_variable_name to be passed to it during the build. Subsequent lines can reference that variable with a dollar notation.

This might look like we’re passing something to the bash environment where the echo command is being executed, but you can use ARG-defined values in other directives (like USER) as well. Docker is taking care of the substitution.

When building a Docker image from the commandline, you can set those values using –build-arg:

$ docker build --build-arg some_variable_name=a_value

Running that command, with the above Dockerfile, will result in the following line being printed in the process:

Oh dang look at that a_value

When you try to set a variable which is not ARG mentioned in the Dockerfile, you’ll get a warning.

Notive how we use ARG? ARG are build-time variables, which are not available to future running containers anymore. ENV can be used to define default environment variables. More on this later. Here is an overview of the ARG and ENV availability :

An overview of ARG and ENV availability.

Building the Image with Docker Compose

So, how does this translate to using docker-compose?

In docker-compose, you can specify values to pass on for ARG, in an args block:

(docker-compose.yml file)

version: '3'

services:
  somename:
    build:
      context: ./app
      dockerfile: Dockerfile
      args:
        some_variable_name: a_value

relevant docs

Here’s what happens above: You set variables to be passed to docker when building a new image from “Dockerfile” in directory “./app” If the Dockerfile contains an ARG entry as above, a_value will be passed into it and available as $some_variable_name.

When building an image, no other variables apart from those listed in “args” are used. Environment stuff only applies to containers, not images. Just a heads-up to prevent confusion. Read on for more info on that. Those ARG variables will not be available in containers started based on the built image without further work.

If you want ARG entries to change and take effect, you need to build a new image. Probably you’ll need to manually delete any old ones.

Setting Environment Variables in Your Dockerfiles

So, how do you set stuff so it’s available to future running containers? ENV. Take a look at this Dockerfile snippet:

(Dockerfile snippet)

ENV foo /bar
# or ENV foo=/bar
ADD . $foo
# or ADD . ${foo}
# translates to: ADD . /bar

relevant docs

You see, we can reference an environment variable with the same notation, as build-arg before. If ENV is specified after an ARG entry with the same variable name, the ENV value is used. Unlike ARG, ENV variables will still be available in containers.

You can inspect images, and see which ENV entries are set by default:

# first, get the images on your system and their ids
$ docker images
# use one of those ids to take a closer look
$ docker inspect image-id

# look out for the "Env" entries

Setting Dynamic Default ENV Values During The Image Build

Let’s combine ARG and ENV. Assuming we want to set default ENV values for future containers on build, here’s how you would do that:

(Dockerfile snippet)

ARG some_variable_name
ENV env_var_name=$some_variable_name

relevant docs

Once you build a container, specifying –build-arg, the ENV line makes sure, that the value can be accessed in future containers (unless manually overridden).

Setting Environment Variables When Running a Container

So, are we bound to setting ENV values only when building images? Nope.

You can override ENV entries in images when starting containers, by specifying your own values. When using the Docker commandline, this looks this way:

$ docker run -e "env_var_name=another_value" alpine env

relevant docs

the -e flag sets a variable with name “env_var_name”. Maybe your value is secret and you don’t want to type it out? Well, read on :)

Do variables from the host automatically get passed to the container? Nope again. Only those environment variables will be set when starting a container. Your host environment does not get passed through. Unless…

Passing Environment Variables From the Host Into a Container

Unless, you don’t specify the value of the environmant variable in the command line, but just the name:

$ docker run -e env_var_name alpine env

relevant docs

In this case, the local value of the host environment variable env_var_name will be passed through to the container. Just don’t specify the value. To be honest, this feels kinda dangerous as it’s not explicit enough for my taste.

Luckily, there’s another way.

Using an ’env_file’ to Set Environment Variables When Running a Container

Instead of writing the variables out or hard-coding them (not in good taste according to the 12-factor folks), we can specify a file to read values from. The contents of such a file look something like this:

env_var_name=another_value

The file above is called env_file_name and it’s located in the current directory.

You can reference the filename, which is parsed to extract the environment variables to set:

$ docker run --env-file=env_file_name alpine env

relevant docs

So much for plain-docker-CLI. On to docker-compose.

Environment Variables using Docker Compose

As above, you can either spell the values out, or reference an env_file to read from. In both cases, the values will be passed into the container which is being started.

(docker-compose.yml file:)

version: '3'

services:
  example:
    image: ubuntu
    environment:
      - env_var_name=another_value

Relevant docs

As before, we have another option to writing the variables out. Once again, this will keep the values out of the docker-compose file, as is in good taste.

We just reference a *env_file", and parse it for the variables to set.

(docker-compose.yml file:)

version: '3'

services:
  example:
    image: ubuntu
    env_file: env_file_name

Relvant docs

But wait, there’s more

When working with docker-compose, you have one more way to use variables. You can do string substitution inside of the docker-compose file. Neat-o, but also a bit confusing, when you don’t have an overview.

This one has nothing to do with ENV, ARG, or anything Docker-specifig explained above. It’s exclusively a docker-compose thing.

It’s also known as the .env file.

An .env file resides in the same folder as your docker-compose.yml file. It’s the same folder from which you run the docker-compose commands.

It contains values like:

(.env file:)

VARIABLE_FOR_DOCKER_COMPOSE_1=yet_another_value

which is exactly the same format as an env_file.

Those values don’t get injected into container environments. They are however available inside the docker-compose.yml file. They are used to substitute strings in the format of ${some_name} or $some_name. That’s it.

Of course, we can use this to pass values into args, or environments, to name your images, and many more. Basically, you can think of your docker-compose.yml file as a template, from which a temporary docker-compose.yml file is created and used with the values from the .env file.

Let’s look at an example! In the case below, we simply set an environment variable of the future container, based on the value we got from the .env file. Just putting it directly into the docker-compose.yml file. Think string replacement.

(docker-compose.yml file:)

version: '3'

services:
  example:
    image: ubuntu
    environment:
      - env_var_name=$VARIABLE_FOR_DOCKER_COMPOSE_1

Hint: When working with an .env file, you can debug your docker-compose.yml files quite easily. Just type docker-compose config. This way you’ll see if everything looks after the substitution step.

An Overview

Let’s go through a complete example once again, and see where different ways to set variables are being used.

We start only with a docker-compose.yml file and a running Docker service without images. You don’t have any images built.

This file can contain strings like ${some_variable} or $some_variable, which are used to perform string replacement. The replacement happens before anything else, using values from a .env file, located in the same directory from which you are running your docker-compose commands.

The first step, is to build a new Docker image. When using docker-compose, you can define build blocks for single services. Those are used to build fresh images if there are none available. The build block can point to a particular Dockerfile location and say which directory to use as context. It can also provide args entries.

If the Dockerfile has ARG entries, values from docker-compose.yml in the “args” block can be passed through, and will be available during the image build. They can be used with the same dollar notation inside a Dockerfile.

Once an image is built, you can start containers from it. Those containers have access to ENV variables defined in the Dockerfile which produced the original image. However, those values can be overridden by providing single environment variables, or env_files, from which environment variables are parsed and passed into the container. The precedence is: values from environment entries, values from the env_file(s) and finally Dockerfile defaults.

Of course, once a process runs inside the container, or when a command is evaluated they can once again change the env values for themselves. Stuff like:

$ docker run myimage SOME_VAR=hi python app.py

will completely override any SOME_VAR you might have set otherwise for the app.py script.

Where To Go From Here?

I hope, that this article has helped you to get an overview of Docker build args, environment variables and Docker Compose variables. Working on the basics will help you to build your Docker images and configure your dockerized apps with confidence.

If you’re interested in another take on providing an overview, check out this section of the docs.

Best of luck!