Spotify Backstage is an open source internal platform developed by Spotify to manage their extensive engineering and development operations. It serves as a unified interface for various tools and services utilized by Spotify's engineering teams, providing a centralized hub for managing tasks related to development, deployment, infrastructure, and more. Backstage aims to enhance developer productivity and collaboration by offering a single, integrated platform for accessing and interacting with different tools and resources. In essence, Spotify Backstage acts as a comprehensive developer portal, facilitating the efficient management of Spotify's complex engineering ecosystem. Backstage is a tool in the platform engineering domain, you can read more about it in our previous publication about exploring Spotify’s Backstage for streamlined software development. 

In this article, we present the core Backstage features and customizations. In order to do this, we will use the OpenTelemetry demo       as an example system.

OpenTelemetry demo

OpenTelemetry is a framework and toolkit for observability crafted to generate and oversee telemetry data, including traces, metrics, and logs. To provide a reference implementation of OpenTelemetry, the community developed a demo system.

The OpenTelemetry demo showcases a microservices-driven distributed system designed to showcase the application of OpenTelemetry within a near-realistic environment. The demonstration application is an e-commerce platform of numerous microservices. Its incorporation of examples in multiple programming languages supported by OpenTelemetry's SDKs renders it an excellent resource for developers seeking guidance on OpenTelemetry utilization.

The OpenTelemetry demo accomplishes the following:

• Offers an authentic portrayal of a distributed system, ideal for showcasing the utilization of OpenTelemetry instrumentation and observability.
• Establishes a foundation for vendors, developers of tools, and others to expand upon and exhibit their integrations with OpenTelemetry.
• Presents a dynamic model for OpenTelemetry contributors to utilize in testing new iterations of the API, SDK, and other components or improvements.

You can read more on OpenTelemetry in our other posts:

• A guide to using OpenTelemetry Operator for Kubernetes
• Explainer on instrumentation with OpenTelemetry       
• OpenTelemetry for reliability and control: Aperture

Backstage

Backstage has a range of valuable features useful in a developer platform. By utilizing templates, engineers can initiate a new microservice project while incorporating your organization's established best practices from the outset. Engineers can compose documentation in Markdown files alongside their code effortlessly. 

Furthermore, Backstage Search empowers users to locate precise information within the Backstage ecosystem. With a versatile backend, search capabilities can be extended beyond the Software Catalog, indexing diverse sources such as Confluence, and Stack Overflow. Finally, Backstage can be heavily customized with plugins, and it can be aligned to fit company policies and processes.

All of this contributes to maintenance of organizational standards and best practices, fast and simple builds of software components, and more.

Modifying application for Backstage

The OpenTelemetry demo is maintained in a monorepo. In Backstage, this monorepo features multiple Catalog entries, each related to a distinct demo component (this includes APIs, services, and databases). This setup results in having multiple Backstage metadata files defining the ’Component’ Kind incorporated in our demo fork. Each component may be owned by a different user group which reflects structural organization. In this configuration, a singular metadata file with the Location kind is employed at the root level of the monorepo.

In this approach, a backstage.yaml file is created for each component, colocated with its respective code within the monorepo. The structure looks like this:

.
├── backstage.yaml
└── src
    ├── accountingservice
    │   ├── backstage.yaml
    │   └── Dockerfile, src, etc
    ├── adservice
    │   ├── backstage.yaml
    │   └── Dockerfile, src, etc
    └── (...)

In this setup, each component can be handled independently by their owners. New components may be added or removed flawlessly, and need to be reflected in the root backstage.yaml.

Now, let’s take a look at some of the relations between the services. For instance, we have Checkout Service which calls Cart Service via a gRPC proxy, to manage the cart state, and Cart Service communicates with Redis to store it. 

Let’s start with creating an API object. It will be referenced by API consumers and providers.

apiVersion: backstage.io/v1alpha1
kind: API
metadata:
  name: pb
spec:
  type: grpc
  lifecycle: production
  system: opentelemetry-demo
  definition:
	$text: ./demo.proto

The association between Cart Service and Redis is established through a `dependsOn` property, and ‘providesApis’ links the component as API provider.

apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: cartservice
spec:
  type: service
  system: opentelemetry-demo
  dependsOn:
	- component:default/featureflagservice
	- resource:default/redis
  providesApis: [pb]

Let’s link Checkout Service to Cart Service as API consumer – simply set consumesApis field.

apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: checkoutservice
spec:
  type: service
  system: opentelemetry-demo
  dependsOn:
	- component:default/currencyservice
	- component:default/emailservice
	- component:default/paymentservice
	- component:default/productcatalogservice
	- component:default/shippingservice
	- resource:default/kafka
  consumesApis: [pb]

In the redis directory we define Redis itself:

apiVersion: backstage.io/v1alpha1
kind: Resource
metadata:
  name: redis
spec:
  type: database
  system: opentelemetry-demo

When we take a look at Cart Service in Backstage, we can see its dependencies and explore them.

By leveraging relationships like dependsOn, providesApi, and consumesApi, Backstage empowers users to model software architectures with ease. For now, the dependencies need to be integrated manually, but potentially a service mesh or API gateway could detect dependencies automatically in the future.

Integrating with Backstage

Backstage prerequisites are a Unix-based operating system, Node, and yarn.

We forked the OpenTelemetry demo and integrated it with Backstage. You can find our forked repository in GitHub. Inside the backstage directory you can type make run to run Backstage with the demo and Backstage plugins already installed. Now, let’s see what this looks like inside the developer portal.

We automatically install the OpenTelemetry demo. This is done by specifying the ‘Location’ root url in the configuration file app-config.yaml. The Location itself (‘backstage.yaml’ in the repository root) is described below. 

apiVersion: backstage.io/v1alpha1
kind: Location
metadata:
 name: opentelemetry-demo
 annotations:
   github.com/project-slug: codilime/opentelemetry-demo
spec:
 targets:
   - ./backstage-org.yaml
   - ./pb/backstage.yaml
   - ./src/accountingservice/backstage.yaml
   - ...

To explore all features in this demo, we encourage you to integrate with GitHub. The credentials are set up in app-config.local.yaml.  Values are read from environment variables, so you should set GITHUB_TOKEN, GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET, according to GitHub docs      .

TechDocs

TechDocs is Spotify's internally developed documentation-as-code solution seamlessly integrated into Backstage. Engineers compose their documentation using Markdown files, co-located with their code, and with minimal setup, generate a visually appealing documentation site within Backstage.

TechDocs use mkdocs       to render the documentation. This means that various mkdocs plugins can be used to enrich the docs. Backstage supports multiple providers like GitHub, GitLab, or BitBucket. TechDocs can be searched by Backstage Search which can be customized to fit an organization's needs.

First, we need to create an mkdocs.yaml file in the root of our component documentation. Here, we can also specify custom plugins, nav pages and other mkdocs features.

site_name: 'Ad Service'

plugins:
  - techdocs-core

To use TechDocs, we need to annotate the documentation locations. OpenTelemetry hosts their documentation in a separate repository, so we set backstage.io/techdocs-ref       to point to the location of mkdocs.yaml. Here is a definition of Ad Service:

metadata:
 name: adservice
 description: This service determines appropriate ads to serve to users based on context keys. The ads will be for products available in the store.
 annotations:
backstage.io/techdocs-ref: url:https://github.com/codilime/opentelemetry.io/tree/main/content/en/docs/demo/services/ad
github.com/project-slug: codilime/opentelemetry-demo

This works for a project with documentation like this:

content/en/docs/demo/services/adservice
├── docs
│   └── index.md
└── mkdocs.yml

After committing the changes, we can browse the docs. First, access the documentation within Backstage, start by locating the corresponding service in your service catalog. Once found, click on it to open the Overview page.

Within the Overview page, navigate to the tab bar and select "Docs." Your documentation should now be displayed for viewing.

Plugins

Backstage plugins offer additional functionalities to the Backstage application.

Each plugin functions as an independent web application and has the capability to incorporate nearly any type of content. Utilizing a shared set of platform APIs and reusable UI components, plugins ensure consistency across the Backstage ecosystem. They have the ability to retrieve data from external sources using standard browser APIs or by relying on external modules to facilitate this process. Plugins are usually written in TypeScript and can be collected in the Plugin Directory      .

For the sake of this demo, we provided several plugins:

• Security Insights – an overview to monitor the security insights of the repository
• GitHub Actions – an overview of GitHub workflow runs
• Lighthouse – accessibility, performance, SEO, and adherence to best practices of your website. Lighthouse requires running lighthouse-audit-service      , we set it up in our demo, so you can run it with make run-lighthouse-srv

Installing new plugins requires downloading with yarn and then patching React components to include entities in Backstage pages. This is why it is important to create a team or a company solution and distribute it among developers. This way, they can be onboarded much faster and with fewer errors.

Conclusion

Platform engineering is transforming software development, enhancing efficiency, collaboration, and enjoyment within organizations. With its extensive features and plugins, Backstage serves as a powerful tool tailored for platform engineers. Backstage's emphasis on optimizing workflows positions it as a leading solution, providing an adaptable and integrated platform to meet diverse development requirements.

Developer portals are valuable resources for scale-ups and larger enterprises managing tools and services. Backstage can be utilized in constructing effective developer portals. By treating it as a framework and engaging with end users, its full potential can be realized, facilitating the creation of a robust development platform.