vsupalov

Understanding Docker Build Args, Environment Variables and Docker Compose Variables

An overview of ways to set and use variables when building images, starting containers and using docker-compose.

[ docker ]

UPDATE:

Check out the new, updated article to understand Docker ARG, ENV and .env quicker.

When working with Docker, you’ll stumble over lots of confusing details. Sometimes, the documentation is there, but very fragmented across different parts and each is focused on detailed views - this can come off as unclear at worst. For practical usecases, you usually have to connect the dots. It can be hard to find your way through towards a good understanding without reading the whole thing.

That’s the case when building and running containers. How can you set variables during build time? Who gets what environment vars, and what can you do to change them? How to override defaults or set your own in your Docker files?

Let’s do a deep dive and get an overview of all the possible ways to use variables while creating and running Docker containers. Both with the Docker CLI, as well as with docker-compose.

Before We Do

This is a long, in-depth article. I understand that you might be busy and not looking to learn all there is to know, but fix your current problem. Here are shorter articles, which will help you make progress as quickly as possible:

If you’re kinda new to the topic or not firm in the basics, you might want to invest a bit of time into fixing that. It’s a quick read and you will have an easier time using Docker :)

Get a handy cheat sheet about Docker and Docker Compose variables!

Wanna get handy overview of all the ways variables can be passed when using docker-compose and Docker? Enter your email below, and I will send you a neat cheat sheet which you can use during your work.

When Building the Image

Let’s assume, you have a Dockerfile, and nothing else. You want to use the Dockerfile to build the image. Depending on the content of the Dockerfile, it allows you to specify arguments at buildtime.

(Dockerfile:)

ARG some_variable_name
# or with a default:
#ARG some_variable_name=default_value

RUN echo "Oh dang look at that $some_variable_name"
# or with ${some_variable_name}

relevant docs

As you see, we tell the Dockerfile to let Docker know, that it expects a variable named some_variable_name to be passed to it during the build. Subsequent lines can reference that variable with a dollar notation.

This might look like we’re passing something to the bash environment where the echo command is being executed, but you can use ARG-defined values in other directives (like USER) as well. Docker is taking care of the substitution.

Anyway, when building a Docker image from the commandline, you can set those values using –build-arg:

$ docker build --build-arg some_variable_name=a_value

Running that command, with the above Dockerfile, will result in the following line being printed (among others):

Oh dang look at that a_value

When you try to set a variable which is not ARG mentioned in the Dockerfile, you’ll get a warning.

Here is an overview of the ARG and ENV availability:

An overview of ARG and ENV availability.

Building the Image with Docker Compose

So, how does this translate to using docker-compose?

In docker-compose, you can specify values to pass on for ARG, in an args block:

(docker-compose.yml file)

version: '3'

services:
  somename:
    build:
      context: ./app
      dockerfile: Dockerfile
      args:
        some_variable_name: a_value

relevant docs

Here’s what happens above: You set variables to be passed to docker when building a new image from “Dockerfile” in directory “./app” If the Dockerfile contains an ARG entry as above, a_value will be passed into it and available as $some_variable_name.

When building an image, no other variables apart from those listed in “args” are used. Environment stuff only applies to containers, not images. Just a heads-up to prevent confusion. Read on for more info on that. Those ARG variables will not be available in containers started based on the built image without further work.

If you want ARG entries to change and take effect, you need to build a new image. Probably you’ll need to manually delete any old ones.

Setting Variables in Dockerfiles

So, how do you set stuff so it’s available to future running containers? ENV. Take a look at this Dockerfile snippet:

(Dockerfile snippet)

ENV foo /bar
# or ENV foo=/bar
ADD . $foo
# or ADD . ${foo}
# translates to: ADD . /bar

relevant docs

You see, we can reference an environment variable with the same notation, as build-arg before. If ENV is specified after an ARG entry with the same variable name, the ENV value is used. Unlike ARG, ENV variables will still be available in containers.

You can inspect images, and see which ENV entries are set by default:

# first, get the images on your system and their ids
$ docker images
# use one of those ids to take a closer look
$ docker inspect image-id

# look out for the "Env" entries

Setting Default ENV Values in Images

Let’s combine ARG and ENV. Assuming we want to set default ENV values for future containers on build, here’s how you would do that:

(Dockerfile snippet)

ARG some_variable_name
ENV env_var_name=$some_variable_name

relevant docs

Once you build a container, specifying –build-arg, the ENV line makes sure, that the value can be accessed in future containers (unless manually overridden).

Setting Environment Variables When Running a Container

So, are we bound to setting ENV values only when building images? Nope.

You can override ENV entries in images when starting containers, by specifying your own values. When using the Docker commandline, this looks this way:

$ docker run -e "env_var_name=another_value" alpine env

relevant docs

the -e flag sets a variable with name “env_var_name”. Maybe your value is secret and you don’t want to type it out? Well, read on :)

Do variables from the host automatically get passed to the container? Nope again. Only those environment variables will be set when starting a container. Your host environment does not get passed through. Unless…

Passing Environment Variables From the Host Into a Container

Unless, you don’t specify the value of the environmant variable in the command line, but just the name:

$ docker run -e env_var_name alpine env

relevant docs

In this case, the local value of the host environment variable env_var_name will be passed through to the container. Just don’t specify the value. To be honest, this feels kinda dangerous as it’s not explicit enough for my taste.

Luckily, there’s another way.

Using an ‘env_file’ to Set Environment Variables When Running a Container

Instead of writing the variables out or hard-coding them (not in good taste according to the 12-factor folks), we can specify a file to read values from. The contents of such a file look something like this:

env_var_name=another_value

The file above is called env_file_name and it’s located in the current directory.

You can reference the filename, which is parsed to extract the environment variables to set:

$ docker run --env-file=env_file_name alpine env

relevant docs

So much for plain-docker-CLI. On to docker-compose.

Environment Variables using Docker Compose

As above, you can either spell the values out, or reference an env_file to read from. In both cases, the values will be passed into the container which is being started.

(docker-compose.yml file:)

version: '3'

services:
  plex:
    image: linuxserver/plex
      environment:
        - env_var_name=another_value

Relevant docs

As before, we have another option to writing the variables out. Once again, this will keep the values out of the docker-compose file, as is in good taste.

We just reference a *env_file”, and parse it for the variables to set.

(docker-compose.yml file:)

version: '3'

services:
  plex:
    image: linuxserver/plex
      env_file: env_file_name

Relvant docs

But wait, there’s more

When working with docker-compose, you have one more way to use variables. You can do string substitution inside of the docker-compose file. Neat-o, but also a bit confusing, when you don’t have an overview.

This one has nothing to do with ENV, ARG, or anything Docker-specifig explained above. It’s exclusively a docker-compose thing.

It’s also known as the .env file.

An .env file resides in the same folder as your docker-compose.yml file. It’s the same folder from which you run the docker-compose commands.

It contains values like:

(.env file:)

VARIABLE_FOR_DOCKER_COMPOSE_1=yet_another_value

which is exactly the same format as an env_file.

Those values don’t get injected into container environments. They are however available inside the docker-compose.yml file. They are used to substitute strings in the format of ${some_name} or $some_name. That’s it.

Of course, we can use this to pass values into args, or environments, to name your images, and many more. Basically, you can think of your docker-compose.yml file as a template, from which a temporary docker-compose.yml file is created and used with the values from the .env file.

Llet’s look at an example! In the case below, we simply set an environment variable of the future container, based on the value we got from the .env file. Just putting it directly into the docker-compose.yml file. Think string replacement.

(docker-compose.yml file:)

version: '3'

services:
  plex:
    image: linuxserver/plex
      environment:
        - env_var_name=$VARIABLE_FOR_DOCKER_COMPOSE_1

Hint: When working with an .env file, you can debug your docker-compose.yml files quite easily. Just type docker-compose config. This way you’ll see if everything looks after the substitution step.

An Overview

Let’s go through a complete example once again, and see where different ways to set variables are being used.

Get a handy cheat sheet about Docker and Docker Compose variables!

Wanna get handy overview of all the ways variables can be passed when using docker-compose and Docker? Enter your email below, and I will send you a neat cheat sheet which you can use during your work.

We start only with a docker-compose.yml file and a running Docker service without images. You don’t have any images built.

This file can contain strings like ${some_variable} or $some_variable, which are used to perform string replacement. The replacement happens before anything else, using values from a .env file, located in the same directory from which you are running your docker-compose commands.

The first step, is to build a new Docker image. When using docker-compose, you can define build blocks for single services. Those are used to build fresh images if there are none available. The build block can point to a particular Dockerfile location and say which directory to use as context. It can also provide args entries.

If the Dockerfile has ARG entries, values from docker-compose.yml in the “args” block can be passed through, and will be available during the image build. They can be used with the same dollar notation inside a Dockerfile.

Once an image is built, you can start containers from it. Those containers have access to ENV variables defined in the Dockerfile which produced the original image. However, those values can be overridden by providing single environment variables, or env_files, from which environment variables are parsed and passed into the container. The precedence is: values from environment entries, values from the env_file(s) and finally Dockerfile defaults.

Of course, once a process runs inside the container, or when a command is evaluated they can once again change the env values for themselves. Stuff like:

$ docker run myimage SOME_VAR=hi python app.py

will completely override any SOME_VAR you might have set otherwise for the app.py script.

If you’re interested in another take on providing an overview, check out this section of the docs. It’s neat as well.

Phew!

That’s it. With all of those details and similar names, it’s no wonder people are confused about using variables with their Docker workflow. There are just so many, and it’s easy to confuse them. But each one has its merrits and use.

Wanna read more about topics which will help you to master Docker in the real world? Enter your email address below, and I’ll send you a mail when there’s a new article. Thanks a lot for your interest and all the best!

Master Docker ARG and ENV in 5 Days

Sign up to the free 5-day email course, and learn all you need to know about using environment and build-time variables with Docker. Get actionable advice, best practices and nifty tricks to use for your next project.