Containerization has been a game changer for software development and operations. It offers a consistent, efficient, portable and scalable way to build, package, test, deliver and deploy applications. It was also a significant milestone in the deployment of microservices.
Kubernetes has emerged as the leading container orchestration platform and “natural” environment for microservices due to its versatility in addressing a wide range of applications and their deployment needs. While it has resolved numerous challenges, it has also revealed several functionality gaps that necessitate supplementary tools.
From an applications’ deployment point of view there was (is) a need for an easy and structured process which allows smooth customization, packaging, and distribution of Kubernetes manifests. This is addressed by tools like Helm or Kustomize for instance.
From a product life cycle management perspective there is a need for tools to increase the efficiency of debugging and verification without increased cost (including the time needed for creating and configuring remote environments).
From an application development point of view there is a need for an easy and consistent way to set up an environment and shorten feedback time. The main challenges are streamlining workflows, accelerating iteration cycles, and creating authentic (as production-like as possible) development environments.
In this article we will look closely at some of the tools that can address these development challenges as well as enhancing debugging and testing possibilities.
Why do we need Kubernetes development tools?
While Kubernetes is powerful for container orchestration, it can be complex to manage and quite expensive (Kubernetes itself is open source and free to use, the infrastructure required to run it and application workloads typically is not).
That is why some development tools are essential. They can help to reduce costs, increase development speed, improve debugging and troubleshooting.
Let's look a bit closer:
-
Cost reduction
We need environments to deploy our services, test them, debug issues, etc. especially when we are dealing with integration cases. Remote environments are expensive, while our workstations are getting more and more powerful. It is then natural to start using them as hosts for our Kubernetes clusters (with the help of projects like kind
, k3d , Microk8s , minikube , Docker for Desktop or Rancher Desktop ) and performing testing locally. -
Development speed
The software development loop looks like this: we code, we build, we deploy, we test. Relying on CI/CD pipelines for all the steps except “code” might not be effective as most often such pipelines create an environment, build all needed artifacts, push them into registries and then deploy on target cluster(s) from scratch. This process is time-consuming and utilizes pipeline resources, which usually are limited (and potentially causes delays for other CI/CD jobs).
Developing directly against a local Kubernetes cluster significantly reduces the feedback loop, enables rapid iteration, and allows us to quickly make changes, rebuild, and redeploy only the needed elements without the overhead of pushing artifacts to remote environments or waiting for a CI/CD pipeline.
-
Debugging and troubleshooting
Developing locally with Kubernetes simplifies debugging and troubleshooting processes. Developers can easily access logs, inspect container states using known and favorite tools (IDEs, debuggers, etc.) which is not always the case with remote environments.
-
Isolation and reproducibility
Each developer can have their own local Kubernetes cluster, ensuring an isolated and reproducible environment. This prevents conflicts and interference between different developers or teams working on the same codebase.
-
Automated workflow(s)
The whole cycle of code->build->deploy/run->test can be easily automated.
-
Collaboration
Unified environments: these tools provide a consistent development environment, making it easier for teams to collaborate on projects.
-
Learning and experimentation
Local Kubernetes environments provide a safe and controlled environment for developers to learn and experiment with Kubernetes concepts, configurations, and tools without risking disruption to production environments.
-
Version control
Integration with version control systems allows for better tracking of changes and collaborative development.
What tools do we have?
There are many tools that can help developers when it comes to working with microservices and Kubernetes environments. Choosing one might be hard as different projects or scopes have different needs. Sometimes we only need to debug one service to quickly fix/test a bug in stage or production. Sometimes we need to have a fully deployed application when we are preparing a framework for e2e tests, running integrations or profiling and improving the whole system.
In this article, we will focus on tools that cover at least the latter. We will have a look at four (there are more) that can manage your whole microservices application environment: Skaffold, Tilt, Garden and DevSpace.
Skaffold
The oldest in the list and well known, Skaffold was developed by Google for automating the workflow for building, pushing, and deploying applications to Kubernetes. It simplifies the development process by providing a unified CLI tool to manage the entire life cycle of a Kubernetes application, from local development to CI/CD pipelines. Thanks to its maturity, quite a big community, and fine-tuned project initialization and configuration defaults, it is a really smooth process to start using Skaffold.
Moreover, Skaffold is an open-source tool and is available for free.
Benefits:
- Automates the build, push, and deploy phases of the development workflow.
- Supports policy-based image tagging, resource port-forwarding, and logging.
- Highly optimized for local development, offering fast and repeatable builds.
- Modular architecture allows customization to fit specific CI/CD needs.
- Works everywhere, with easy sharing and setup across different environments.
- Skaffold is able to automatically detect language runtimes and reconfigures the pods for debugging; also forwards debugging ports to create debugging sessions.
- Existing plugins for VS Code and Intellij Cloud Code, which can be used with Skaffold debugging sessions thanks to the above point.
Use cases:
- Local development and testing of Kubernetes applications.
- CI/CD pipelines for Kubernetes deployments.
- Sharing projects easily with others through simple ’git clone’ and ’skaffold run’ commands.
Useful resources:
Tilt
Tilt is a developer tool designed to simplify the process of iterating over applications development (coding, deploying, testing) particularly those applications running on Kubernetes. It provides a unified interface for managing the entire life cycle of an application, from development to deployment, aiming to bridge the gap between development and operations teams.
Tilt is best suited for teams that want to accelerate their development cycles and improve collaboration between devs and ops. Its ability to handle complex Kubernetes setups with ease makes it a reliable choice for projects that require development with frequent deployments and quick iterations.
The main feature/mechanism is called Control Loop: the Tilt engine actively monitors your source code and reacts to any changes according to your needs (configuration) whenever it is rebuilding and redeploying, or live updating by syncing your files with the ones on the containers or executing any local command.
Tilt’s main goal is to support local development on local clusters and by default it will not let you use a remote cluster. Tilt detects your locally running cluster and supports two generic protocols for auto-discovering the cluster’s local registry. In most cases, creating a cluster and a local registry is all you need to do. The team behind Tilt also develop and maintain CLI ctlptl which simplifies local cluster (and local image registries) creation.
Tilt is open source and free to use, making it accessible to developers and teams of all sizes.
Benefits:
- Streamlined development workflow with automatic builds and deployments.
- Real-time feedback through a web-based GUI or classic terminal UI, reducing the need for manual checks. These interfaces provide visibility into logs, build status, and deployment health across all your microservices.
- Simplifies Kubernetes management, making it accessible to developers who may not have deep Kubernetes expertise.
- Configuration by “scripting” (in Starlark ) enables true flexibility.
- Live updates: Tilt can update your running containers without requiring a full rebuild, dramatically speeding up iteration time. You can define fine-grained update behaviors for even faster iterations.
- Extensibility: Tilt allows you to define custom workflows and integrate with other tools in your development pipeline.
- Resource dependencies: enables specification of relationships between your services
- Local resource definitions: run non-containerized processes as part of your dev environment.
- Developed with local environments in mind.
- Clear and comprehensive documentation and a lot of examples.
Limitations:
- Steep learning curve for users unfamiliar with Docker, Kubernetes, or Python.
- May require additional configuration for complex applications.
- Debugging of your deployed services has to be configured separately.
Use Cases:
- Rapid development and deployment of microservices.
- Aligning continuous integration and continuous deployment (CI/CD) pipelines with local development.
Useful resources:
Below, you can watch my comprehensive demo of Tilt:
Garden
Garden.io is an automation tool designed to streamline the development and testing of Kubernetes applications. It allows teams to create production-like environments on demand for development, testing, and CI, ensuring consistency across stages and significantly speeding up build and test times through smart caching.
In contrast to Skaffold or Tilt, Garden encourages use of its platform to host environments but the main features (build, deploy, test) work locally.
Garden.io offers a free tier with basic features. For advanced features and enterprise-level support, they have paid plans starting with a powerful solution for platform teams, including features like 30x faster container builds with Cloud Builder, one-click preview environments, automatic environment cleanup, streaming log viewer, insights for workflow analysis, and more. Pricing details can be found on their plans page .
Benefits:
- Facilitates rapid iteration of code through live syncing into running containers and environments.
- Offers a seamless blend of local and remote development capabilities.
- Codifies in-house practices with templates, custom commands, and workflows.
- Integrates with a wide range of platforms and tools, including Docker, Kubernetes, Terraform, Pulumi, and custom scripts.
Limitations:
- The documentation and community support might not cover all specific use cases or configurations.
- Initial setup and configuration can be complex for users unfamiliar with Kubernetes or Garden.io or when adopting Garden for an already complex application/project.
- Debugging of your deployed services has to be configured separately.
- It is still in v0 and some backward incompatible changes might occur, leading to misleading docs (see 0.13.0 update) but Garden team is trying to address such issues.
Use Cases:
- Rapid development and testing of Kubernetes applications.
- Establishing consistent environments across the software delivery pipeline.
- Automating CI/CD pipelines for Kubernetes applications.
- Transitioning to a more efficient and reliable development workflow.
Useful resources:
DevSpace
DevSpace is specifically designed for Kubernetes, aimed at accelerating the development and deployment of cloud-native software. It integrates seamlessly with Kubernetes, allowing developers to deploy, develop, and debug applications directly within the Kubernetes environment. DevSpace streamlines the development workflow by automating common tasks such as building, deploying, and managing Kubernetes resources, making it easier for developers to focus on coding rather than infrastructure management.
DevSpace is an open-source tool and is available for free.
DevSpace supports a wide range of development activities, including:
- Initializing and deploying applications to Kubernetes.
- Monitoring artifacts in a web UI running on localhost.
- Developing and debugging applications directly within the Kubernetes cluster.
Benefits:
- Automates the deployment workflow, saving time and reducing errors.
- Allows direct development and debugging within the Kubernetes environment.
- Offers a localized UI optimized for Kubernetes development.
- Supports a wide range of Kubernetes features and integrations.
Limitations:
- The tool is specifically designed for Kubernetes, limiting its applicability to non-Kubernetes environments.
- Requires familiarity with Kubernetes concepts and terminology.
Use cases:
- Rapid development and deployment of cloud-native applications on Kubernetes.
- Debugging and testing applications directly within the Kubernetes cluster.
- Automating the deployment workflow for Kubernetes applications.
Useful resources:
Comparison of K8s local development tools
Let’s grab some crucial elements from a usage point of view and compare these four tools.
| Skaffold | Tilt | Garden | DevSpace | |
|---|---|---|---|---|
| How to use it / User Interfaces | CLI | CLI, GUI on localhost | CLI, GUI (remote on Garden.io after auth with GitHub account) | CLI, GUI on localhost |
| Configuration | yaml | Starlark | yaml | yaml |
| Built-in build systems support | Dockerfile, Bazel, Jib, Buildpacks, Kaniko | Dockerfile, docker compose | Dockerfile, Jib, Kaniko | Dockerfile, Kaniko |
| Built-in deployment systems support | kubectl, kustomize, helm | kubectl, kustomize, helm, Docker compose | kubectl, helm, terraform, pulumi | kubectl, kustomize, helm |
| Built-in notion of testing | YES | NO | YES | NO |
| Built-in support for executing local commands | YES | YES | YES | YES |
| Customizable functions and commands | NO | YES | NO | YES |
| Workflows/Pipeline functionalities | YES | NO | YES (and for enterprise users can be natively triggered by VCS events) | YES |
| Built-in Profiles notion | YES | NO | YES | NO |
| Sync/Hot Reload/Live Updates | YES | YES | YES | YES |
| Env snapshots | NO | YES | NO | NO |
| Built-in support for debugging | YES (Go, NodeJS, Python, Java, .NET Core) | NO | NO | NO |
| IDEs support | Cloud Code extension for VS Code and Intelij | VS Code extension for Tiltfiles (for other IDEs it is possible to configure Starlark LSP) | NO | NO |
Choosing the right tool
Over the last few years, the number of available tools has increased and we will probably see new solutions in the years to come as every project and every team has some specific use cases that are not covered out of the box by existing tools. But still, what we can currently use and achieve with Skaffold, DevSpace, Tilt or Garden is quite satisfactory.
Let me try to draw some conclusions to highlight the use cases for them. If you are looking for a tool that can be used to set up your whole development environment and cover all of your app’s services then you can consider any one of Tilt, Skaffold, DevSpace or Garden.
When you do not need the remote resources, dashboards, etc. and what you need at the end of the day is one or two commands to execute and have an up and running local env with all the goodies like hot reload, automatic tests, static analysis verification, etc. then Skaffold or Tilt might be best fit. When your teams (devs and ops) are fine with using Starlark and are more then happy with the possibilities of easy extensions and conditional rules that might be incorporated into this env, then I would recommend Tilt.
If you need a more declarative approach and a more readable configuration that can also cover your CI/CD pipelines then you should be fine with Skaffold. But if you would like to have a GUI available for your developers then try the newest kid on the block, DevSpace. You can also give Garden a try if you would like to also use its cloud environments for your CI/CD. But both DevSpace and Garden will require more implementation time, especially for big and complex projects/applications.
