Blog>>Operations>>DevOps>>Exploring Spotify’s Backstage for streamlined software development

Exploring Spotify’s Backstage for streamlined software development

Backstage is a Spotify-backed framework for creating developer portals, bridging the gap between developers and infrastructure teams responsible for maintaining systems that applications will be running on. Similar to a physical backstage in a theater, it is what's hidden from the view of users, but helpful to the staff to facilitate their day-to-day work. 

Backstage is a way to support developers in all phases of software development. Starting with an initial architecture, Backstage can help steer developers to choose vetted technologies and designs. It can support developers through the development and deployment phase, where it can help with setting up standardized CI/CD setup, and provide tools used to create architecture and system documentation meeting internal standards, helping with the visualization of dependencies between components. In addition to its role in facilitating collaboration between developers and infrastructure teams, Backstage integrates seamlessly with identity providers (IDPs), offering a centralized authentication and authorization mechanism. This feature serves as a single pane of glass for access control, ensuring unified access across development stages. Finally, it can facilitate "Day 2 operations" giving developers and infrastructure teams a single place to monitor the status of their software. 

Backstage is a two-way portal between developers and teams responsible for infrastructure and platforms. It can help create best practices and make sure that it's being used across the entire company. If utilized properly, it can ensure consistency in the technologies used by often-separate teams, ensuring that agreed upon standards for deployment and observability are followed.

The essence of platform engineering in developer work

Platform engineering and its relation to developers, operations, and infrastructure teams is a vast topic but it's worth understanding how it fits into the whole picture to better understand the need for tools like Backstage.

DevOps, SRE, infrastructure and platform engineering 

The underlying principles that form the basis of platform engineering have been present for a long time, only to get tangibly organized in the late 2010s. This came from the experience of transitioning development and operations teams into various types of DevOps-related structures, as a way to think about the entire product stack in a more holistic manner. The goal is to provide developers with tools that they can use to quickly iterate on their own projects, while making sure that their architecture fits into a larger picture. To learn more about the basics of platform engineering, check out our full introduction to the topic in the article Platform engineering — everything you need to know.

A book that touches on the subject of platform engineering is Team Topologies      link-icon. In it, the authors define four fundamental team topologies, one of them being the Platform Engineering Team, responsible for creating and maintaining an internal product used by other teams to deliver their own products (internal & external).

How much “platform” is in platform engineering?

Finally, it's worth noting that the "product" managed by the Platform Engineering Team does not need to be the same for every company. It can be a sprawling system that gives developers all sorts of automated tools to manage their environments and get insight into software development and production status. But it can also be a wiki where consensus on supported architectures, components and libraries is described.

What is Backstage?

Backstage is described as an open platform for building developer portals. What that means is that the scope of the platform created for a given company is up to whoever is going to be using that platform. Its primary function is to visualize components of the system and relationships between components, APIs, and resources. For example, the component backend-service can make use of resources like the SQL database and expose gRPC API to other components. 

The framework can provide a central location for all the architecture documentation, API description, and deployment notes. It can be configured to let developers monitor the state of their applications from development all the way through to production, like showing GitHub actions’ results for the component and allowing users to re-run failing steps, integrating with Sentry for error reporting, visualizing Prometheus metrics and PagerDuty alerts. 

Finally, it provides both SDK and a catalog of plugins that can be used to integrate with all kinds of third-party services and tools, giving developers a better overview of their work.

Components, APIs & resources in Backstage

To visualize the interdependencies between various sub-projects, Backstage introduces the concepts of components, APIs, and resources. These can be used to better show the relations between various parts of a larger project.

Components 

Components form the core of Backstage project management, mapping a single service like backend service, data processing pipeline, and mobile application. They can be tracked in a version control system or can be external software: either open source or commercial. 

Components can depend on other components implicitly by consuming APIs, or explicitly by providing a reference in the component definition. Finally, they can also depend on any resources that are defined within the Systems and domains section below.

APIs

Backstage treats any API provided by components as its own first-class entities, browse-able through the portal. They are provided and consumed by components and can be public, restricted, and private. This allows for documented access control to the component: public APIs can be consumed by any other component, restricted APIs are only available to white-listed components, while private APIs are only available within the system they belong to. 

Backstage can render API specifications from protobuf files, and provide documentation for REST/MQTT protocols. It can give developers access to GraphQL endpoints to see schema and send requests.

Exploring Spotify’s Backstage for streamlined software development
Exploring Spotify’s Backstage for streamlined software development

Resources 

Backstage also provides a way to define and reference any infrastructure-level elements of the project that components can consume. Anything from Object Buckets, through Pub/Sub topics and databases can be added to Backstage as a resource, to document who owns it and how it relates to other components.

Systems and domains

Components, APIs, and resources can be combined into systems. Those are collections of resources and components that expose public or restricted APIs that other systems and external users can consume. This allows us to reason about interactions between separate systems without having to track the internal relations between components, resources, and APIs. One example of such a system could be invoicing, which interacts with other systems like metering. Underneath, both systems can be composed of multiple components and resources but both expose some sort of API that other systems can consume. 

Furthermore, systems can be grouped into domains, a collection of systems that provide end products to external customers. They can share KPIs, metrics, and documentation. They probably belong to a single business unit and are developed closely together.

Groups and users 

Finally, all components, resources, and APIs can have their owners: either specific users, be it developers or operations team members, or entire groups, like teams responsible for given parts of the system, business units owning a given system or domain, or even just an informal group of people interested in a given part of the system.

Component overview 

Each component is defined as a separate entity, with its own overview page and a number of different features that can be enabled. Some of the choices available by default are Code Repository Integration, CI/CD overview, Pull Requests status, Sentry for error reporting, and Kubernetes integration for live status updates.

Code repository integration 

Backstage can integrate with the most popular code repositories like GitHub, GitLab, or BitBucket. This integration can then be used to provide additional details to the overview of the component: status of pull requests and status of recent CI/CD jobs (when integrated with supported CI/CD systems). Backstage supports both CI/CD systems integrated with code repositories (e.g. GitHub Actions) as well as external systems (e.g. CircleCI, Jenkins).

  1. CI/CD
    Backstage can show the CI/CD status of the components, allowing developers to track the status of their builds without going to an external service. CI/CD Overview shows a list of recent jobs, with their status and a way of restarting failed jobs.

    Exploring Spotify’s Backstage for streamlined software development

    CI/CD workflows can be expanded to show details of the job, with the author, all steps of the job (passing and failing) as well as logs from the entire workflow.

    Exploring Spotify’s Backstage for streamlined software development
  2. Pull requests
    Backstage’s integration with Code Repositories allows it to display pull requests for a given component, letting developers track the status of their work from a single place.

    Exploring Spotify’s Backstage for streamlined software development
  3. Kubernetes
    Backstage provides Kubernetes integration, showing some basic information about deployments and pods, allowing developers to track the status of their components across different environments.

    Exploring Spotify’s Backstage for streamlined software development

Tech Docs 

Technical documentation is a must when multiple teams are working to bring a single product into existence. To avoid constant back and forth between teams to discuss details of their corresponding components, it's better to have up to date documentation. Backstage provides a place to share this documentation between teams. The source can be stored in the repository, with the code, and then rendered in Backstage for the benefit of all teams that need to interact with a given component. This documentation, along with API reference, builds a foundation for making teams independent of each other.

Tech Radar

Larger organizations often face the challenge of communicating official guidelines on the use of languages, frameworks, platforms, or practices to all development teams. Using the concept of a technology radar, initially created by ThoughtWorks      link-icon, Backstage can give developers a clear status on the implementation of various technologies across the company. For example, some core technologies can be marked as Adopt, clearly indicating that they should be the first choice when evaluating options. Other technologies can be marked as Trial or Assess, indicating that their use should be critically evaluated, possibly with the help of the people responsible for their implementation. Some technologies can even be marked as Hold, meaning that their future use is discouraged or outright banned.

Lighthouse integration 

Google Lighthouse      link-icon is an automated test utility provided by Google that can be used to measure the quality of web pages and can give insights into how to better optimize web pages for the Google ranking algorithm. Backstage provides an overview of all previously ranked audits, so that long-term trends can be visualized and tracked to ensure that the website ranking is not affected by the quality of the code.

Cost insights

Tracking the costs of an entire product is vital, and Backstage can help visualize those costs as well as communicate them to developers so that they are aware of the impact of their code, and resources used, on the bottom line. This allows them to optimize their usage to ensure better utilization of the underlying infrastructure that the product is running on.

Score Board

Often, there is a need to track product compliance with various internal and external requirements. Internal compliance may be related to the proper Pull Request review process, keeping documentation up to date, and tracking the production-readiness of the given component. External compliance may be related to any ongoing or planned audits or security requirements. For such cases, Backstage provides Score Board - a single place where all requirements and their implementation across all entities can be tracked. The following screenshots show that what constitutes a passing grade can differ between Backstage instances, or even systems.

Exploring Spotify’s Backstage for streamlined software development
Exploring Spotify’s Backstage for streamlined software development

Extensibility 

Additionally to the mentioned features, Backstage comes with additional included plugins, as well as a plugin directory that can be used for installation and integration with additional services. This allows teams to customize their Backstage experience even more, often without writing any additional code. Some of the more interesting plugins are Argo CD integration, giving teams an overview of their GitOps-managed deployments, or GitHub Release Manager, which can serve as a starting point to automate Release Management for projects hosted on GitHub - it automates the creation of release branches, cherry-picking patches, and creating releases themselves.

Who is Backstage for? 

In the end, Backstage provides a platform that developers can use, but at the same time, the platform itself has to be maintained. Some of the Backstage features, like Tech Docs, Tech Radar, Score Board can be maintained as a side-project by developers. Other features, like Kubernetes integration, or installing and maintaining third-party plugins, as well as fully mapping relations between components, APIs, and resources, may require extra work, and turn into a separate project - at some point enabling additional features in Backstage, choosing third-party plugins, and creating internal plugins will require gathering requirements from internal users, and tracking their satisfaction. 

Backstage can be a boon for larger organizations when the need arises to standardize the way teams communicate with each other - switching from ad-hoc discussions to more defined contracts via documentation and API definitions. This is doubly important when an organization has geographically distributed teams, where each team owns its set of components. In such a scenario, making sure that all knowledge is documented helps to ensure that teams can work independently. 

Another use for an extended Backstage deployment is auditing - the Score Board can ensure that all tasks related to audits are fully done, and created diagrams of connections between components can become part of the architecture documentation, augmenting documentation created in Tech Docs.

Klimonda Krzysztof

Krzysztof Klimonda

Senior DevOps Engineer

Krzysztof Klimonda is a Senior Network Engineer and author on CodiLime's blog. Check out the author's articles on the blog.Read about author >

Read also

Get your project estimate

For businesses that need support in their software or network engineering projects, please fill in the form and we'll get back to you within one business day.