Speed Up The Build of Your Python Docker Image

Are you waiting for ages, every time you’re changing a tiny detail in your dockerized Python project and need to build an image? Literally any comment change, makes it necessary to install every. single. pip. dependency. Over and over! Aaaargh.

That’s not an issue if you’re working in your dev environment, as you’re simply mounting your code and have a volume for your virtualenv directory - but building a new image for testing, staging or production takes a long time. It almost feels like you’d rather say ‘I’m not slacking off, my Docker image is building’ and go for a coffee every time that happens. Why did you even start using Docker? The eternal waiting times are not worth it. Or is there a way around it?

The Problem

Your Dockerfile probably contains something like this:

ADD code /app # executed on every small change
RUN pip install -r /app/requirements.txt
# and here we go again...

You’re adding your project code (Flask or Django project?) after installing the necessary libraries and setting up a virtual environment. Then, you’re running pip to install the exact versions of every Python dependency needed for the project in a “requirements.txt” file.

You’re not using the Docker cache as well as you could. The good news is: there’s a simple way to fix that.

Use The Docker Cache

You can prevent the perpetual re-execution of the dependency-installation step if there were not actual changes to the stuff you’re using. There’s no tricky volume mounting or multi-stage build kung-fu needed.

The ADD directive only needs to run if the referenced file changed since the last time it was executed. If it did, every single build step needs to run again, but if it’s the same you can just use a version from the Docker cache and skip to the next one.

If you add the requirements.txt file before your other code, and run the pip install step right after it, both will only be executed if the file changes. Not on every build.

ADD code/requirements.txt /app/requirements.txt
RUN pip install -r /app/requirements.txt
# the steps above only depend on the requirements.txt file!
ADD code /app

This way, you can skip the expensive operation if nothing changed and reuse a cached state. You Docker image build will be faster in most cases. Once that’s not enough anymore, there are more elaborate ways to improve on the process.