Docker ARG, ENV and .env - a Complete Guide

Error: Docker image build failed.

Building Docker images and configuring your dockerized apps doesn’t have to be a try-fail-repeat Google extravaganza. This article will help you work with Docker ARG, ENV, env_file and .env files with confidence.

The only prerequisite: make sure that you’re comfortable with the basics of Docker.

Read on and you will understand how to configure your Docker images and dockerized apps with ease - with the power of Docker build-time variables, environment variables and docker-compose templating.

Frequent Misconceptions

This is a long, in-depth read. Let’s start with something you can use right now, without having to read the whole thing!

Here’s a list of easy takeaways:

• The .env file, is only used during a pre-processing step when working with docker-compose.yml files. Dollar-notation variables like $HI are substituted for values contained in an “.env” named file in the same directory.
• ARG is only available during the build of a Docker image (RUN etc), not after the image is created and containers are started from it (ENTRYPOINT, CMD). You can use ARG values to set ENV values to work around that.
• ENV values are available to containers, but also RUN-style commands during the Docker build starting with the line where they are introduced.
• If you set an environment variable in an intermediate container using bash (RUN export VARI=5 && …) it will not persist in the next command. There’s a way to work around that.
• An env_file, is a convenient way to pass many environment variables to a single command in one batch. This should not be confused with a .env file.
• Setting ARG and ENV values leaves traces in the Docker image. Don’t use them for secrets which are not meant to stick around (well, you kinda can with multi-stage builds).

An Overview

Alright, let’s get started with the details. The guide is split up into the following topics:

• The Dot-Env File (.env)
• ARG and ENV Availability
• Setting ARG Values
• Setting ENV Values
• Overriding ENV Values

Feel free to jump right to the one you need right now. However, you’ll get the best result if you read through them all thoroughly.

The Dot-Env File (.env)

This one is quite simple, and only confusing because of bad examples and similar concepts using the same format, sounding a lot like it. What’s important is the dot in front of env - .env, not an “env_file”.

If you have a file named .env in your project, it’s only used to put values into the docker-compose.yml file which is in the same folder. Those are used with Docker Compose and Docker Stack. It has nothing to do with ENV, ARG, or anything Docker-specific explained above. It’s exclusively a docker-compose.yml thing.

The values in the .env file are written in the following notation:

VARIABLE_NAME=some value
OTHER_VARIABLE_NAME=some other value, like 5

Those key-value pairs, are used to substitute dollar-notation variables in the docker-compose.yml file. It’s kind of a pre-processing step, and the resulting temporary file is used. This is a nice way to avoid hard-coding values. You can also use this to set the values for environment variables, by substituting the string, but that does not happen automatically.

Here is an example docker-compose.yml file, relying on values provided from a .env file:

version: '3'

services:
  plex:
    image: linuxserver/plex
      environment:
        - env_var_name=${VARIABLE_NAME} # here it is

Hint: When working with an .env file, you can debug your docker-compose.yml files quite easily. Just type docker-compose config. This way you’ll see how the docker-compose.yml file content looks after the substitution step has been performed without running anything else.

Here’s a gotcha you should know: environment variables on your host can override the values in your .env file. Read more here.

ARG and ENV Availability

When using Docker, we distinguish between two different types of variables - ARG and ENV.

ARG are also known as build-time variables. They are only available from the moment they are ‘announced’ in the Dockerfile with an ARG instruction up to the moment when the image is built. Running containers can’t access values of ARG variables. This also applies to CMD and ENTRYPOINT instructions which just tell what the container should run by default. If you tell a Dockerfile to expect various ARG variables (without a default value) but none are provided when running the build command, there will be an error message.

However, ARG values can be easily inspected after an image is built, by viewing the docker history of an image. Thus they are a poor choice for sensitive data.

ENV variables are also available during the build, as soon as you introduce them with an ENV instruction. However, unlike ARG, they are also accessible by containers started from the final image. ENV values can be overridden when starting a container, more on that below.

Here is a simplified overview of ARG and ENV availabilities around the process around building a Docker image from a Dockerfile, and running a container. They overlap, but ARG is not usable from inside the containers.

An overview of ARG and ENV availability.

Setting ARG Values

So, you have your Dockerfile, which defines ARG and ENV values. How to set them, and where? You can leave them blank in the Dockerfile, or set default values. If you don’t provide a value to expected ARG variables which don’t have a default, you’ll get an error message.

Here is a Dockerfile example, both for default values and without them:

ARG some_variable_name
# or with a hard-coded default:
#ARG some_variable_name=default_value

RUN echo "Oh dang look at that $some_variable_name"
# you could also use braces - ${some_variable_name}

relevant docs

When building a Docker image from the commandline, you can set ARG values using –build-arg:

$ docker build --build-arg some_variable_name=a_value

Running that command, with the above Dockerfile, will result in the following line being printed (among others):

Oh dang look at that a_value

So, how does this translate to using docker-compose.yml files? When using docker-compose, you can specify values to pass on for ARG, in an args block:

(docker-compose.yml file)

version: '3'

services:
  somename:
    build:
      context: ./app
      dockerfile: Dockerfile
      args:
        some_variable_name: a_value

relevant docs

When you try to set a variable which is not ARG mentioned in the Dockerfile, Docker will complain.

Setting ENV Values

So, how to set ENV values? You can do it when starting your containers (and we’ll look at this a bit below), but you can also provide default ENV values directly in your Dockerfile by hard-coding them. Also, you can set dynamic default values for environment variables!

When building an image, the only thing you can provide are ARG values, as described above. You can’t provide values for ENV variables directly. However, both ARG and ENV can work together. You can use ARG to set the default values of ENV vars. Here is a basic Dockerfile, using hard-coded default values:

# no default value
ENV hey
# a default value
ENV foo /bar
# or ENV foo=/bar

# ENV values can be used during the build
ADD . $foo
# or ADD . ${foo}
# translates to: ADD . /bar

relevant docs

And here is a snippet for a Dockerfile, using dynamic on-build env values:

# expect a build-time variable
ARG A_VARIABLE
# use the value to set the ENV var default
ENV an_env_var=$A_VARIABLE
# if not overridden, that value of an_env_var will be available to your containers!

Once the image is built, you can launch containers and provide values for ENV variables in three different ways, either from the command line or using a docker-compose.yml file. All of those will override any default ENV values in the Dockerfile. Unlike ARG, you can pass all kinds of environment variables to the container. Even ones not explicitly defined in the Dockerfile. It depends on your application whether that’ll do anything however.

1. Provide values one by one

From the commandline, use the -e flag:

$ docker run -e "env_var_name=another_value" alpine env

relevant docs

From a docker-compose.yml file:

version: '3'

services:
  plex:
    image: linuxserver/plex
      environment:
        - env_var_name=another_value

Relevant docs

2. Pass environment variable values from your host

It’s the same as the above method. The only difference is, you don’t provide a value, but just name the variable. This will make Docker access the current value in the host environment and pass it on to the container.

$ docker run -e env_var_name alpine env

For the docker-compose.yml file, leave out the equation sign and everything after it for the same effect.

version: '3'

services:
  plex:
    image: linuxserver/plex
      environment:
        - env_var_name

3. Take values from a file (env_file)

Instead of writing the variables out or hard-coding them (not in good taste according to the 12-factor folks), we can specify a file to read values from. The contents of such a file look something like this:

env_var_name=another_value

The file above is called env_file_name (name arbitrary) and it’s located in the current directory. You can reference the filename, which is parsed to extract the environment variables to set:

$ docker run --env-file=env_file_name alpine env

relevant docs

With docker-compose.yml files, we just reference a env_file, and Docker parses it for the variables to set.

version: '3'

services:
  plex:
    image: linuxserver/plex
    env_file: env_file_name

Relvant docs

Here is a small cheat sheet, combining an overview of ARG and ENV availability with common ways to set them from the command line.

An overview of ARG and ENV availability.

Before we move on: a frequent gotcha, if you’re new to Docker and not used to think about images and containers: if you try to set the value of an environment variable from inside a RUN statement like RUN export VARI=5 && ..., you won’t have access to it in any of the next RUN statements. The reason for this, is that for each RUN statement, a new container is launched from an intermediate image. An image is saved by the end of the command, but environment variables do not persist that way.

If you’re curious about an image, and would like to know if it provides default ENV variable values before the container is started, you can inspect images, and see which ENV entries are set by default:

# first, get the images on your system and their ids
$ docker images
# use one of those ids to take a closer look
$ docker inspect image-id

# look out for the "Env" entries

Phew, that was quite a bit. The only thing left is - if you have so many different ways of setting the values of ENV variables, which ones override others?

Overriding ENV Values

Assuming, you have an image built from a Dockerfile, which provides default ENV values. Containers started from it, have access to ENV variables defined in the Dockerfile. However, those values can be overridden by providing single environment variables, or env_files, from which environment variables are parsed and passed into the container.

Once a process runs inside the container, or when a command is evaluated, they can change the environment values for themselves. Stuff like:

$docker run myimage SOME_VAR=hi python app.py

will completely override any SOME_VAR you might have set otherwise for the app.py script, even if there were some value with a -e flag before the final command.

The precedence is, from stronger to less-strong: stuff the containerized application sets, values from single environment entries, values from the env_file(s) and finally Dockerfile defaults.

In Conclusion

This was a really thorough look at all the ways you can set ARG and ENV variables when building Docker images and starting containers. By now, you should have a really good overview build-time arguments, environment variables, env_files and docker-compose templating with .env files. I hope you got a lot of value out of it, and can use the knowledge to save yourself lots of bugs in the future.

To really master these concepts, just reading about them is not enough. You have to see them in action and apply them to your own work to truly make them part of your tool belt. The best way to make sure you will be able to make use of this information, is to learn by doing – go ahead and try some of those techniques in your current projects!