Speed Up The Build of Your Python Docker Image
Are you waiting for ages, every time you’re changing a tiny detail in your dockerized Python project and need to build an image? Literally any comment change, makes it necessary to install every. single. pip. dependency. Over and over! Aaaargh.
That’s not an issue if you’re working in your dev environment, as you’re simply mounting your code and have a volume for your virtualenv directory - but building a new image for testing, staging or production takes a long time. It almost feels like you’d rather say ‘I’m not slacking off, my Docker image is building’ and go for a coffee every time that happens. Why did you even start using Docker? The eternal waiting times are not worth it. Or is there a way around it?
Your Dockerfile probably contains something like this:
ADD code /app # executed on every small change RUN pip install -r /app/requirements.txt # and here we go again...
You’re adding your project code (Flask or Django project?) after installing the necessary libraries and setting up a virtual environment. Then, you’re running pip to install the exact versions of every Python dependency needed for the project in a “requirements.txt” file.
You’re not using the Docker cache as well as you could. The good news is: there’s a simple way to fix that.
Use The Docker Cache
You can prevent the perpetual re-execution of the dependency-installation step if there were not actual changes to the stuff you’re using. There’s no tricky volume mounting or multi-stage build kung-fu needed.
The ADD directive only needs to run if the referenced file changed since the last time it was executed. If it did, every single build step needs to run again, but if it’s the same you can just use a version from the Docker cache and skip to the next one.
If you add the requirements.txt file before your other code, and run the pip install step right after it, both will only be executed if the file changes. Not on every build.
ADD code/requirements.txt /app/requirements.txt RUN pip install -r /app/requirements.txt # the steps above only depend on the requirements.txt file! ADD code /app
This way, you can skip the expensive operation if nothing changed and reuse a cached state. You Docker image build will be faster in most cases. Once that’s not enough anymore, there are more elaborate ways to improve on the process.
Join the mailing list!
Subscribe to get notified about future articles and stay in touch via email.
I write about Kubernetes, Docker, automation- and deployment topics, but would also like to keep you up to date about news around the business-side of things.