GitOps — which tools should you choose?

In today's fast-paced tech landscape, software development teams are constantly seeking new ways to improve efficiency and speed up the release process. GitOps has emerged as a powerful way to streamline the software delivery process. As GitOps tools have quickly become a go-to solution for many organizations, it is essential to research the right tools to meet your project needs and requirements.

In this article, you will find an overview of some of the GitOps tools on the market and their key features highlighted. Overall, this guide will help you make an informed decision about which GitOps tool is right for your organization. 

What is GitOps? 

GitOps is an alternative technique of continuous deployment (CD) of infrastructure and modern cloud-native applications which brings important advantages to the table. It uses the Git repository as a single source of truth. It also utilizes GitOps agents responsible for the continuous synchronization of deployed systems to the desired state expressed declaratively in Git.

Fig.1: General principles of GitOpsSource: GitOps      

When should you start using a GitOps tool?

Implementing a GitOps deployment tool can bring important benefits. If you experience one or more situations presented in the following table you should seriously think about GitOps. 

Tab. 1 Problems addressed by GitOps

Check out our other related articles:

• GitOps vs. DevOps — what is the difference?
• Unlocking the power of GitOps for network automation

Git in GitOps

Declarative manifests need to be stored and versioned somewhere. Currently, there is only one king in this area - Git      . Git offers very fast branching and merging and extensive functionality. As a result, many hosting solutions have been developed around GitHub      , GitLab       and BitBucket      , for example. These Git providers offer an extensive UI for managing the repository and related functionalities like CI/CD pipelines.

There is no important difference in which Git hosting is used for GitOps. All most popular Git providers are equally good for integration with GitOps tools. The cooperation between the GitOps agent and the Git provider may work in many ways:

• The GitOps agent reads the repository via a generated access token (or Git credentials, which is less secure),
• The Git provider may provide the user identity for the GitOps tool - this is a way to establish single sign-on for GitOps users,
• The Git provider’s CI pipelines may call GitOps webhooks in order to notify the presence of a new commit and this way greatly speed up GitOps pull-based deployment workflows,
• The GitOps tool can also use the Git provider notification webhooks in order to update the commit status visible for developers or platform operators with information about resource reconciliation performed by the GitOps tool.

If you are looking for new ways to use Git repositories, we invite you to check out our precious article about Git commands. 

Git is not always the only source of truth

GitOps tools have been improved over time and more extensive functionality added. The interesting one is the ability to monitor the following non-Git repositories:

• Helm repositories
• S3-compatible buckets

Docker registries such as DockerHub      , GitLab      , Harbor      , JFrog      , Quay      , and AWS ECR       can perform the role of the Helm repository. These Helm repositories contain K8s manifests and Go templates bundled in special archives called Helm Charts. Helm Charts       are versioned in the Helm repository and the GitOps tool can use the Helm repository as the only source of truth. When a continuous integration process creates a new version of the Helm Chart (an application release) then the GitOps agent detects the change and updates the environment. However, a more flexible way is to use the Helm repository together with the Git repository. In that case the Git repository contains the configuration of the Helm repository for the GitOps tool and narrows the scope of possible changes, e.g. sets a semantic versioning limit for Helm Chart reconciliation. 

Object storage, like Amazon S3      , is also used to store the results of CI application builds. For this reason, the usage of S3 buckets, as the source of truth, can be found in some GitOps tools.

Why is Kubernetes used for GitOps?

The guiding principles of GitOps are not tied to a specific technology, however, a fully functional GitOps implementation (which fulfills all four principles       of GitOps) is only available using the Kubernetes platform. Kubernetes (K8s)       is an advanced resource orchestrator whose internal architecture contributed greatly to the birth of the GitOps concept.

The most crucial elements under the hood of Kubernetes are Resource Controllers      . Each of these controllers implements a control loop mechanism that is responsible for monitoring a specific type of K8s resource (e.g. Pod, Deployment, ReplicaSet) and ensuring that the state of the resource does not differ from the target state. In K8s, the target state of each resource is described declaratively by the user using YAML manifests. Resource target specification and actual resource status are bundled together in the form of abstraction called the resource object, which is the first class citizen of the K8s user interface.

The significant effectiveness of control loop and resource reconciliation in K8s is achieved using informers, work queues       and comparing a hash of a K8s resource object with a hash of the real resource state. Any change applied by the user to the resource object causes that K8s Controller is notified about the change and the configuration of the real resource is adjusted. The K8s Controller is also notified about change of real resource (i.e.: resource lifecycle state), the controller updates status of the resource object and that change is visible to the user.

Another important characteristic of Kubernetes is its extensibility. This is possible thanks to the concept of operators      . An operator is a controller that manages a custom resource representing an entity residing inside or outside the cluster. For this purpose, an operator adds Custom Resource Definition       (CRD) as an extension to the Kubernetes API. The above-mentioned capabilities of K8s became the basis for the implementation of the first GitOps solution, called Flux. The Flux       operator is tied to a new type of custom resource called GitRepository      . When a GitRepository resource is added to K8s then Flux starts to monitor a given external Git repository, reads new or changed K8s manifest files from the repository, and applies the fetched manifests in order to create or update applications working in the cluster.

apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: GitRepository
metadata:
  name: podinfo
  namespace: default
spec:
  interval: 5m0s
  url: https://github.com/stefanprodan/podinfo
  ref:
    branch: master

Lis. 1 GitRepository CDR in Flux GitOps tool

GitOps without Kubernetes

When there is no requirement for constant drift detection and configuration reconciliation for the deployed system then it is possible to implement limited GitOps. Limited GitOps doesn’t require Kubernetes as a runtime platform. It can be established using quite simple tooling, like git-repo-watcher       running in the deployment environment. Be aware that some custom scripting could be required. A Git watcher observes a Git repository. A new commit to the repository causes pulling upstream changes to the deployment environment and invocation of a custom deployment script. In the case of an infrastructure as code (IaC) use case, the custom script can invoke an IaC tool like Terraform       for the deployment.

What GitOps agents are available?

Looking at the full-featured GitOps tools, only three solutions are worth comparing. The first is Flux      , which has existed since 2017. It is a very well-developed project where you will find features not available in other GitOps tools (e.g. support for Terraform or S3 bucket). The second tool is Argo CD      , which was created in 2018 and is currently one of the fastest adopted solutions       among all CNCF projects. These two solutions were developed as part of the LF CNCF open-source initiative and are categorized as graduated solutions      . The third tool is Fleet       and is being developed as of 2020 by Rancher      . It is not as extensive as its two older rivals, but it has all the most necessary GitOps functionalities.

Criteria	Flux	Argo CD	Fleet
Target users	Platform operators.	Development teams.	Platform operators.
Main focus	Composability       and extensibility      .	Focused on user experience and UI.	Managing very large groups of clusters.
Runtime platform	Kubernetes	Kubernetes	Kubernetes
Supported K8s resources customizations	Helm and Kustomize      . Flux can also natively manage and reconcile Terraform resources      .	Helm      , Kustomize       and Jsonnet      .	Helm and Kustomize      .
Multi-repository	Yes      . Supports Git and Helm repositories. S3-capable buckets       can also be added as a repository.	Yes      . Supports Git and Helm repositories. Also has the ability to specify multiple sources       for a single application.	Yes. Supports Git and Helm repositories.
Multi-tenancy and tenant access control	Yes      . Relies strictly on the RBAC capabilities       of Kubernetes.	Yes. Isolation done by its own very granular permission system       which may restrict source repositories, destination clusters and namespaces.	Yes. Namespace      -based. Together with Rancher provides great resource isolation. Relies on Kubernetes RBAC.
Multi-clusters	Supported with usage of other tools like CAPI       or Karmada      .	The concept of clusters       is directly supported by Argo CD. Very straightforward with ApplicationSets      . Scalable up to ~100 clusters.	This is the main Fleet functionality. Fleet provides the concept of clusters and cluster groups      . Fleet was designed to scale to many thousands of clusters. Such a scalability is achieved with an internal additional level of Helm repositories distributing bundles      .
Deployment pipelines	Flux Alerts       may trigger external pipelines such as Github Actions       or Tekton      .	PreSync, Sync, PostSync and FailSync       hooks that nicely integrate with pipelines (Argo Events       and Argo Workflows       or Tekton      ) covering more advanced use cases.	No.
Progressive deployment strategies	Yes, using Flagger      .	Yes, using Argo Rollouts      .	With usage of additional tools       like Flagger.
Image update automation	Yes (Image Update Automation      ).	Yes (uses Argo CD Image Updater      ).	Yes      .
User management	No. K8s service account       is used for Flux.	Implements internal user/group management      . Also single sign-on       authentication using OICD providers.	No. Based on K8s service accounts.
User Interface	CLI first approach. WebUI possible after installation of Weave GitOps UI      .	Yes. It gives a great overview of installed applications. It really shines in the case of a large amount of deployed K8s resources.	CLI embedded only. Rancher provides a nice UI       for Fleet.

Tab. 2 Full-featured GitOps tools comparison

Jenkins X - a tool with limited GitOps capabilities

Jenkins X       is a CI/CD solution running natively in Kubernetes. Jenkins X’s hot-feature is environment promotions       in which a separate repository is required for each supported environment. It offers very minimalistic GitOps capabilities      , e.g. no drift detection of K8s resources. After a new commit to a Git repository Jenkins X creates a new K8s Job to execute a user-created makefile which is supposed to update K8s resources (i.e. invoke kubectl/Helm commands).

Which GitOps agent to choose?

For those new to GitOps, the friendliest tool will be Argo CD. Its user interface will be very helpful. Infrastructure operators can start with Flux, its nicely-designed CDRs encourage usage of Kustomize. Rancher users may start with Fleet. It's also good to know that Fleet is the perfect tool for having an Argo CD deployed on all clusters! 

Which infrastructure as code tools can be used in GitOps?

GitOps is used not only to deploy applications and required services, but also for managing infrastructure based on the infrastructure as code (IaC) approach. IaC is the use of imperative code or declarative schemas to automatically create and configure infrastructure. Since infrastructure description code or schemas are placed in a Git repository, the use of GitOps for infrastructure management has become one of the most important use cases of GitOps. This greatly simplifies the work of development teams, which, using the Git repository, easily declare what infrastructure they need to run an application.

The best IaC tools to be used together with GitOps:

Crossplane

Crossplane       - provides a set of custom resource definitions       representing various external cloud resources (AWS, GCP, Azure, etc.) Adding a manifest with Crossplane CDRs to the Kubernetes cluster (adding may be done with a GitOps tool) causes the Crossplane K8s Operator to create declared resources in the external cloud and later monitor the health of the created resources. With Crossplane installed, Kubernetes becomes an orchestrator of external, non-Kubernetes resources.

Pulumi

Pulumi       - a collection of libraries for many general-purpose languages (Python, Go, C#, etc.) allowing development of code that manages cloud resources using language conditionals, loops, and variables. Pulumi is especially useful when management of cloud resources, services or even applications must be accomplished with more complex logic. Pulumi can be enabled for GitOps by the installation of pulumi-kubernetes-operator       in the cluster. 

Terraform

Terraform       - the most popular IaC solution created by HashiCorp       which uses an unique configuration language (HCL      ). HCL is based on JSON and defines infrastructure resources to be created or modified. It currently contains 3000+       various infrastructure/resource providers. Terraform resources defined in HCL cannot be natively installed in Kubernetes thus its usage in K8s-based GitOps tools is problematic. However, Weaveworks has implemented the Flux Terraform controller       for Kubernetes. It supports both the desired state enforcement of Terraform resources as well as the drift detection required for automatic resource state reconciliation. 

Summary

Deploying a GitOps solution requires choosing a set of appropriate tools. Different tools cover different aspects of GitOps functionality. The selection of a range of appropriate tools must meet specific functional needs. Most of the mandatory elements of GitOps are briefly covered in this blog post, however some others, like deployment pipelines, progressive delivery, policy enforcement and secret management, would require dedicated articles to explain.

The table below summarizes the different categories of useful tools available within the Kubernetes platform for creating a fully-functional GitOps continuous deployment system.

GitOps continuous deployment function	Tools performing the function
Repository of declarative manifests	Git, Helm repositories (DockerHub      , GitLab      , Harbor      , JFrog      , Quay      ), S3-compatible buckets (AWS S3      , Minio      , BackBaze      , Wasabi      )
Repository management and continuous integration	GitHub      , GitLab      , BitBucket
User management	OICD providers (GitHub, GitLab, Octa      , OneLogin      , KeyClock      , Google      , Microsoft      , Auth0      )
Docker registry	DockerHub, GitLab, Harbor, JFrog, Quay, AWS ECR      ,
Orchestration platform	Kubernetes
K8s resource customization	Kustomize      , Helm      , Jsonnet
GitOps agent	Flux      , ArgoCD      , Fleet      , Jenkins X
Deployment pipeline	Argo Workflow & Events      , Tekton      , Rafay      , K8s Jobs      , GitHub Actions      , Spinnaker
Infrastructure provisioning	IaC (Crossplane      , Pulumi      , Terraform      )
Progressive delivery	Flagger      , Argo Rollouts      , Spinnaker
GitOps monitoring	Prometheus      , InfluxDB      , Grafana
Policy enforcement	Policy as code (OPA      , Kyverno      , Datree      , jsPolicy      )
Secrets management	Vault Secrets Operator      , Sealed Secrets      , Kamus      , Mozilla SOPS

Tab. 3 Ecosystem of GitOps tools