CircleCI vs Google Cloud Build

Choosing the perfect CI/CD tool for your project can be tricky. In this post we compare two managed CI/CD services - CircleCI and Google Clo
ud Build. Sami Alajrami is on hand to score our exciting CI/CD matchup.

In the old days, achieving continuous integration in your team meant that you had to install and manage a CI/CD server like Jenkins or TeamCity. Fast forward a few years and the pains of managing self-hosted CI/CD servers has led to the rise of managed CI/CD solutions.

Managed CI/CD service providers take away the burden of operational maintenance to let your team focus on the software you are creating. However, such solutions often vary in their features and cost models and don’t always fit your projects needs, forcing you to continue to use a self-hosted solution for all or part of your projects. Moreover, different managed CI/CD service providers have strengths and weaknesses. There is no one-size-fits-all solution.

You can read more on the differences between self-hosted and managed CI/CD here.

Comparison

Recently, we were asked by a customer to evaluate two managed CI/CD services: CircleCI and Google Cloud Build (GCB) to help them choose the one that would work best for their needs. This post summarizes our findings.

We did the comparison by building the same customer project with both CI/CD services and made a “compare and contrast” between the two in the context of this project. The project in question is an open source monorepo Kotlin application built with Gradle. Although the comparison was done in the context of a customer-specific project, the outcome is quite generic. We compared the two services based on the following factors:

UI Maturity

Winner: CircleCI

CircleCI has been in the market for quite a while so it’s had time to polish the user interfaces to provide you with easy access to builds and workflows and view logs of specific steps in your build jobs. GCB needs to invest more effort in its user interface.

Debugging Support

Winner: CircleCI

At the beginning of a project it is very likely that your workflows and scripts will be buggy and will fail several times until you perfect them. In such cases having the means to debug your workflows and jobs is crucial. CircleCI allows you to rerun a job with SSH. This means you can connect to the running build container/VM over SSH. Additionally, CircleCI provides a local CLI tool which enables you to reproduce the CircleCI environment locally which is very useful for debugging. GCB does not offer any debugging support.

Built-in Notifications

Winner: CircleCI (but only just!)

CircleCI wins this category because it supports Email, Slack, HipChat, Flowdock, Campfire and IRC notifications out of the box while GCB supports none! However, CircleCI notifications are not always useful; e.g. see here. CircleCI has recently updated its notifications. However, you might find yourself having to implement your own custom notifications as part of your workflows (which is also possible on GCB).

Cloud Provider Integrations

Winner: Google Cloud Build

CircleCI 2.0 does not have any special integration mechanisms with specific cloud providers. This means you have to connect to your provider from your jobs as you would do from any other machine. You also need to provide credentials for that.

GCB on the other hand is part of Google Cloud Platform (GCP) and can gain access to different GCP services without providing extra credentials.

GCB only wins in some cases though. If you use other providers, or do not want to be vendor locked, this GCB feature is useless.

Lock-in Risk

Winner: Neither!

CircleCI is neutral in that it works in exactly the same way with every cloud provider. Google Cloud Build clearly -and understandably- stands by Google Cloud. It offers you easy integration with other GCP services which can drag you into build your projects around GCP specific services. However, it might be okay to get locked-in to GCP if the choices of services are justified.

Declarative Config

Winner: CircleCI

CircleCI provides a more mature and feature-rich declarative workflow config compared to GCB. For example, in CircleCI you can configure filters for custom builds on specific tags/branches. In GCB the same can be achieved, but you have to use Triggers which is configured from the console interface and not as part of the declarative config. GCB does have a nice feature that lets you split your workflows into different files that you can associate with different triggers. It’s not enough to win this one though.

Available Build Infrastructure

Winner: Neither

CircleCI supports three types of executors: Docker Containers, Virtual Machines and macOS. You can buy additional computational resources for Docker Containers up to 8 vCPU and 16 GB RAM.

GCB offers only Docker Containers as an executor. However, it provisions a VM to run all your build containers on it. The VM can use one of two machine types (max 32 vCPUs/28.8 GB RAM). The advantage of running all containers on a single VM is that in Google Cloud Build all your jobs run on the same Docker network and can have access to artifacts from previous jobs out of the box. In CircleCI you have to persist artifacts to the workspace and attach them where needed. This adds more lines in your CircleCi config and might increase the execution time depending on how big your artifacts are.

Feedback to GitHub

Winner: CircleCI

It is quite important to see the result of your builds in GitHub checks. CircleCI supports this out of the box. GCB has recently introduced an Alpha release of their Github App. This app can trigger builds and provide feedback to GitHub. However, it does not currently work with triggers. This means that it is useless if you have custom branch or tag filters configured in triggers. CircleCI supports building only from pull requests. GCB does not support building pull requests from forks.

Secret Management

Winner: Neither

Most projects will need to pass some secrets during build, test and deploy jobs. How these secrets are passed and managed is important as you don’t want your secrets to be leaked or end up in malicious hands. In either service you will have to trust a third party with your secrets.

CircleCI uses HashiCorp Vault to encrypt secrets and stores them as environment variables which are made available to jobs. The risk is that, for public repositories, CircleCI makes all your job logs public. Therefore, you have to be careful about logging your secrets in your jobs. Also, any GitHub user who has to write permissions on the repository can trigger a re-run with SSH and print all environment variables, including secrets.

GCB uses Google KMS to decrypt secrets at runtime and make them available as environment variables. This means that you have to manually create a KMS key and use it to encrypt your secret. You then add the encrypted secret in your version-controlled GCB config file and GCB decrypts it at run time. It is worth noting though that there is a limit of 2MB on the size of your encrypted secret. A Google Cloud service account json file can often exceed this when encrypted. This means you would have to store it in a Google bucket and pull the file at runtime since access to buckets can be achieved without credentials.

Pricing

Winner: Google Cloud Build

GCB offers 120 free build minutes a day and up to 10 concurrent builds. After that, you will be charged $0.003 per minute for default VM machine type (see here for details). CircleCI offers 1,500 build minutes per month and 1 concurrent job for free for Linux builds. You will be charged $50 per month for each additional parallel container. Pricing is different for macOS builds (see here for details).

Summary

It is quite clear that CircleCI provides a more mature and feature-rich CI/CD service compared to younger Google Cloud Build. However, choosing a tool should always depend on the project specific needs. For example, if you are already tied to using several services from Google Cloud, and you don’t mind the GCB limitations mentioned above, choosing GCB would make sense. If GCB does not work for you right now it is still worth keeping an eye on its development (even if progress seems to have been slow in the past).

But, what really matters is that you make an informed decision! The tool is not as important as the way you use it. You shouldn’t pick a tool and adapt your work around its capabilities. Instead, try to make the tool work for your needs as long as you follow best practices. Know your needs and know the tradeoffs and caveats of the tools. If you do that you can choose the tool that will work best for you.