vsupalov

Using CircleCI 2.0 to Build and Push a Docker Image to the Google Container Registry

When building the deployment pipeline for your dockerized app, you’ll have to take care of building a Docker image based on your Git repository and pushing it to a remote Docker registry for further use.

Although those are almost basic building blocks, I found them to be a bit less documented than you’d expect when using CircleCI. The instructions are there, but you have to fish them out from several places, and glue everything together.

I’d like to save anybody who wants to make use of CircleCI and get on with working on their project.

Here’s a walkthrough of everything you have to know to get started, Detailed instructions on configuring CircleCI 2.0 to build your Docker image and push it to the Google Container Registry - all in one place and with links to further resources.

Preparing the Project

If you haven’t done so before, log into CircleCI with your GitHub account, as described here.

Go to the app and create a project corresponding to your GitHub repository.

The instructions will tell you to create a .circleci folder in your repository and add a config.yaml file. If you select your project’s language, you’ll be provided with a functional default config file. If your app is a simple one, you can simply commit the default config file, push the changes, and watch CircleCI build your project already. Neat!

As we’re going to be using the Google Container Registry, there’s one more thing to do before we can start to customize the CircleCI config file further.

First, you’ll need to enable the Google Container Registry as described here.

You’ll have to create a service account in the Google Cloud Platform console for your project. Be sure, to use the right “project” on the top left for your future app work. Follow the instructions here to create a service account. Be sure to make it an “Owner” of the project, and generate a private key in JSON format, you’ll need that data.

Creating the service account.

Copy the content of the downloaded key, and create a new environment variable named “GOOGLE_AUTH” in cour CircleCI project. In the CircleCI interface, click on “builds”, and then on the cog symbol next to the listed project.

Getting to the project settings in CircleCI.

When selecting “Environment Variables”, you can create a new one. Paste the content of the JSON file as value, and name it “GOOGLE_AUTH”.

With the CircleCI project created, a Google Cloud Platform service account in place and the environment variable available to CircleCI we’re all set to build and push a Docker image.

Customizing the CircleCI Config File

The default file provided by CircleCI for your project’s language should be good enough to build the project. Below, you see the configuration for a NodeJS project which has been slightly adjusted. It’s also the first part of the final config file.

version: 2
jobs:
  test-locally:
    docker:
      - image: circleci/node:8.9

      # Specify service dependencies here if necessary
      # CircleCI maintains a library of pre-built images
      # documented at https://circleci.com/docs/2.0/circleci-images/
      - image: circleci/mongo:3.6.3 # those version numbers depend on your project
      - image: redis:4.0.8

    working_directory: ~/repo

    steps:
      - checkout

      # Download and cache dependencies
      - restore_cache:
          keys:
          - v1-dependencies-{{ checksum "package.json" }}
          # fallback to using the latest cache if no exact match is found
          - v1-dependencies-

      - run: npm install .

      - save_cache:
          paths:
            - node_modules
          key: v1-dependencies-{{ checksum "package.json" }}

      # run tests!
      - run: npm test

Running Tests Without Docker

The first step of a build pipeline, serves the purpose to make sure the project might be good enough to go into more elaborate steps. Usually, you run a linter to make sure the code looks good, run unit tests to cover basic functionality and only move on if both pass.

Although we’re going to build a Docker image, you don’t have to work with the Docker image right away for this step. In general, you don’t have to use Docker for EVERYTHING. It’s completely alright to have a non-Dockerized dev workflow, as long as you keep the tradeoffs in mind.

In this case, we won’t have to build an image before running tests, we can save on work by using CircleCI deafault configs which are well tested and make use of caching. If there are no special OS dependencies which are needed to run tests for your code that’s fine.

Using almost-the-same environment at each stage of your deployment pipeline does have its merits! However, if your project needs custom libraries and more elaborate dependencies at this stage, you’ll be better off to use your own Docker image, or install those dependencies, which might be tricky.

Building the image and pushing it to Google Container Registry

After your project has been checked in a first approximation, you’ll want to build your Docker image and push it.

This happens by appending another step to the CircleCI config file, right below the previous snippet.

  # https://circleci.com/docs/2.0/google-auth/
  # https://circleci.com/docs/2.0/google-container-engine/
  # https://cloud.google.com/container-registry/docs/pushing
  push-image:
    machine: true
    steps:
      - checkout
      - run: echo ${GOOGLE_AUTH} > ${HOME}/gcp-key.json
      - run: docker build --rm=false -t eu.gcr.io/${GCP_PROJECT}/${IMAGE_NAME}:$CIRCLE_SHA1 .
      - run: gcloud auth activate-service-account --key-file ${HOME}/gcp-key.json
      - run: gcloud --quiet config set project ${GCP_PROJECT}
      - run: gcloud docker -- push eu.gcr.io/${GCP_PROJECT}/${IMAGE_NAME}:$CIRCLE_SHA1

Here’s an explanation of what happens above. We’re using the machine operator as it has everything we need for this step installed out of the box - the gcloud tool and Docker running. We take the value of the GOOGLE_AUTH variable defined previously, and save it to a file. The file is later used, to authenticate with the Google service.

We build the docker image in the second run command, with a very specific tag. Two more variables have beeen specified in the same matter as GOOGLE_AUTH in the CircleCI interface to keep it flexible and save on typing. The GCP_PROJECT and the IMAGE_NAME of the final container. The “eu” in front of the tag, means that the image will be stored in the European Union region. The last command pushed the image with our tag to the Google Container Registry.

Be sure to define those two additional variables for your CircleCI project!

Connecting everything

Now that we have a step to run tests, and one to build+push the Docker image, we’d like to make them depend on each other, so they run in the right order, and so we can add more steps in the future.

We can use CircleCI’s workflows for this. Here’s what you should add to the config file for this:

# see https://circleci.com/docs/2.0/workflows/
workflows:
  version: 2
  test-and-build-image:
    jobs:
      - test-locally
      - push-image:
          requires:
            - test-locally

This creates a new workflow named “test-and-build-image”, where “test-locally” runs first, and “push-image” afterwards.

A nice side-effect of using workflows, is a cool overview graph of tasks in the CircleCI interface, which is updated in almost-real-time while the project is building.

The workflow view of CircleCI.

In Conclusion

Combining those three CircleCI config file snippets into one, will result in a valid configuration to run tests for a NodeJS application, build a Docker image and push it to the Google Container Registry, and having a CircleCI workflow which can be expanded in the future.

I hope you’ll be able to save some time when building the first version of your deployment pipeline, and get to the point where you can build a Docker image and push it to the Google Container Registry quickly and without figuring out frustrating errors :)

Have you thought about what good next steps are, once the image-building-and-pushing works? Here’s a good set of next steps you can take from here. Once this basic part of your deployment pipeline is in place, you should focus on establishing a clearly defined workflow of deploying your app into a testing/staging environment, running integration tests there and then automating the whole process.

Then, you can include those steps into your project’s CircleCI config. This way you’ll end up with a “skeleton pipeline” which can be extended in the future, depending on the state of the project and the needs of the team.

Join the mailing list!


Subscribe to get notified about future articles and stay in touch via email.

I write about Kubernetes, Docker, automation- and deployment topics, but would also like to keep you up to date about news around the business-side of things.

Privacy and your data: You can get more information about the usage of your data, the storage of your registration, sending out mails with the US-provider ConvertKit, statistical analysis of emails sent and your possibility to unsubscribe in my Privacy Policy.

I use the US-provider ConvertKit for email automation. By clicking to submit this form, you acknowledge that the information you provide will be transferred to ConvertKit for processing in accordance with their Privacy Policy and Terms.

We won't send you spam. Unsubscribe at any time. Powered by ConvertKit