Catching Up on 5 Years of Docker Updates

Okay, how much has Docker has changed since 2018?

Sometimes you are so comfortable with a tool, it fades into the background. You don’t even notice that it is moving and changing, while you are happily using it - after all, it’s doing everything you want from it, and it does not get in the way.

This is pretty much what has happened to me and Docker.

Occasionally, single topics like nifty features of BuildKit have caught my attention, but I have not taken the time to follow all developments. So, to make sure I have a sufficiently accurate mental model, and to see if there’s something cool I might be missing I decided to take the time and go through the docs and versions. Sharpening your tools and all.

I would like to share my approach and the most interesting findings with you!

The Many Versions of Docker

Let’s start with an overview of the major versions that were released, where to find detailed information about them, and the most interesting changes (according to me and my needs) they introduced. I created from browsing the Docker Engine release notes.

Here is an overview of versions I found interesting, and what caught my eye about them. I will skip releases which don’t spike my curiosity. Each release is linked, and has a (year-month) next to it. You will see why.

• 17.03 (2017-03): this was the first release when Docker switched to a YY.MM versioning scheme. Before that, the version was 1.13. The release frequency they targeted was monthly, so it’s not easy to spot major changes just by the version number here. Before that, Docker announced a “community edition” and an “enterprise edition” (see the HN discussion here for some context. There was drama.)
• 17.05 (2017-05): in this release, multi-stage builds landed!
• 17.09 (2017-09): added the –chown flag to COPY. Useful!
• 17.11 (2017-11): containerd was introduced, for now in beta. To my understanding this was an important step to separating out the container runtime into its own logical layer. Docker uses it, but it’s on the best way to be a stand-alone component.
• 18.02 (2018-02): manifest handling appears here, as an experimental feature.
• 18.06 (2018-07): Docker is switching up the release and maintenance cadence. BuildKit is introduce as experimental feature.
• 18.09 (2018-11 to 2019-09): (notice the date?) containerd is not a daemon managed by the Docker engine daemon anymore. It needs to be installed as an own package. You can run BuildKit builds without experimental mode now.
• 19.03 (2019-07 to 2021-02): Dockerfile version docker/dockerfile:1.1 is assumed by default in BuildKit. There’s a lot of entries, but I don’t see much more interesting points to mention here…
• 20.10 (2020-12 to 2023-04): A tiny thing: ENV name value is deprecated, and ENV name=value is preferred in Dockerfiles. ARM support for the CLI in 2021-04. Docker compose v2 can be installed as plugin to get a docker compose command. It is optional for now.
• 23.0 (2023-02): BuildKit is now the default builder on Linux. Docker Engine changed the versioning from CalVer (starting with the year) to SemVer! From now on, it will be easier to see major functionality jumps by the version. Woohoo! (well, in theory at least) Also, you need to install plugins like docker-buildx-plugin or docker-compose-plugin it seems? I gotta look into those.
• 24.0 (2023-05): Where we are at now. I see that there is a version jump, but for the love of me - I can’t tell what the major user-facing changes would be from the release notes. At least not from the release notes I have looked at. I know that docker-compose v1 is deprecated since recently in this version.

So much for an overview. I was expecting much more major changes, to be honest! Let’s dive into the topics which will have an impact on most workfows.

BuildKit

This is the biggest change to me. I will definitely want to revisit this topic, and make sure I understand every aspect of building images with the new build architecture. I’ll surely be writing about this in the future, but let me share a detail I found useful to learn about:

Previously, you had to make sure to enable BuildKit, and to specify what version of the builder to use in your Dockerfile. Here is what the line looks like:

# syntax=docker/dockerfile:1

Here is what is hiding behind this line: the actual builder of BuildKit runs (best) in a container.

There is a “built-in Dockerfile frontend” in BuildKit, which does not use an external container for building, but it’s recommended not to rely on it for the sake of flexibility.

If you look closely, you will see that the line looks very similar to a Docker image tag. Because it basically is.

You can read about specifying the syntax here. Why would you want to specify a particular version you ask? To use features which only become available starting at a certain version of the builder. Look for docker/dockerfile: in the Dockerfile reference or even the description of the BuildKit Dockerfile frontend image. Some features need 1.2 or higher, some need something like 1.3-labs to use in your Dockerfile.

Docker(-)Compose

RIP docker-compose. At least when it comes to the command syntax. You can still type docker-compose and there is an alias being invoked, but it’s recommended to switch to writing docker compose instead.

But take care! The container names that v2 starts are not the same as v1 started. There might be other gotchas. I gotta read up on this, but don’t have a long-running compose stack that still runs on v1.

Done, Apart From The Details

I guess those have been the biggest changes which I spotted. I really want to dive deep into BuildKit and see if there are features I could use in the future. Especially fancy and upcoming Dockerfile instructions! But also instructions which were added to Dockerfiles which I might have missed… Well, gotta read the reference again and browse the docs some more. Just in case something that got lost to me in the release notes hides there.

Are there some major exciting features I missed, apart from what can be found close to BuildKit? I’m looking forward to find out eventually. Until then, I hope that this short writeup on my journey through the last few years of Docker Engine release notes has been helpful to you.