Allow me to tell you about Backstage in the form of a story. Assume you're at a medium-to-large-sized company, in a team tasked with boosting developer productivity. You've developed a kick-ass pipeline to calculate test flakiness based on historical build logs and some data from your internal userinfo service. Problem is, userinfo has suddenly started exhibiting p99 response times in the 10s of seconds. It's not feeling well.
You go to backstage.yourcompany.net and search for userinfo. You reach the overview page for the service. It shows a ton of information, some of it pertaining to who owns and operates the service. Apparently it's the devcore team, they're on #devcore-support on slack according to the page but they're in the US west coast timezone so they're asleep. Well you are using the PagerDuty plugin in Backstage so there's a big "page us" button right there.
So now you've got the issues solved and you need to expose the pipeline output somehow. Cool, you go to backstage.yourcompany.net and click "Create", and select one of the templates. It scaffolds a Go service in your GitHub org, according to your company's best practices. All the builds and deployments are already set up and hot. You add some code to your service and shortly it's up and running and serving the pipeline output with caching out of S3. You check backstage.yourcompany.net to see how all aspects of your service's runtime status is looking. All good. Now, how do you actually get that useful data into your company's other devs' hands?
Well you COULD make an entire microsite, or some CLI tool orrrr... Come to think of it, they are already all using backstage.yourcompany.net, and there's already an integration in there showing all builds for all services, and it has all of the utilities and auth and bells and whistles already in place for you. So you make an internal backstage plugin. You find that it's pretty much a matter of cloning your internal backstage repo, running a create-plugin script, and boom you are up and running. You add a data source feeding off of your new Go service, you add a column to the builds table ... and you have your own team review and merge it because it's covered by CODEOWNERS. And everybody rejoiced and told you how useful this new column was to them. You start getting a bit famous on your internal Slack, actually.
Now you are getting greedy. You feel that actually, this plugin could be useful to the general public - as long as the build logs are in a known format, you can eat them. You start open sourcing the pipeline, and the service - and, critically, also the backstage plugin.
This is how things are internally at Spotify today. What we released here today is not this entire story yet, but we are lifting more and more out into the open at a rapid pace. Check the repo and microsite for more info, and hold tight.
More is coming.
- "to me it seems like it's a way of abstracting all the different infrastructure tooling that a large-scale company has."
- "It looks like an onboarding tool, monitoring, and knowledge database."
- "So it's a frontend to put in front of your existing frontends so developers can infrastructure the infrastructure?"
- "Seems like a developer-oriented version of Confluence, if I'm reading it correctly, but I'm also a bit unsure."
- "Seems like a competitor for Datadog and other monitoring tools to me."
That message really couldn't be more unclear.
Good example here is the “open, plugin etc.” feature being almost right the top. I don’t care if it’s open or that I can make a plug-in if I have no clue what it does.
Consider a team that wants to deploy something to the cloud in a company. They are probably engineers who aren't necessarily well-versed in all cloud tooling, yet they are required to know and understand Terraform, GCloud/AWS/Azure CLIs, Gitlab CI, Prometheus, various monitoring and alerting tools, Kubernetes, Docker, etc. And this is just to deploy their application. Once other resources come into play (databases, queueing, etc), they need even MORE knowledge.
A tool like Backstage allows developers to get a uniform overview of all their resources, regardless of how and where they are running. I'm not sure if it also allows the creation of new resources, but I guess that would be a fairly obvious direction to go in.
It won't solve the need to at some point know a bit more about the technical details about all the underlying tools, but it can allow teams to gradually learn about all this new tooling instead of having to know about everything upfront.
Here's an old talk about it (2016):
and here's some slides from a newer talk (2017):
In a nutshell, the aim is to make it easy for developers to find out info about other services as well as make changes to the services they own.
It looks like Backstage is about open sourcing big parts of System-Z so that others can build on it.
Disclosure: I'm CTO @ OpsLevel, where we make a SaaS-based service catalog product.
- Phase 1: Extensible frontend platform (now) - You will be able to easily create a single consistent UI layer for your internal infrastructure and tools. A set of reusable UX patterns and components help ensure a consistent experience between tools.
- Phase 2: Manage your stuff (next 2-3 months) - Manage anything from microservices to software components to infrastructure and your service catalog. Regardless of whether you want to create a new library, view service deployment status in Kubernetes, or check the test coverage for a website -- Backstage will provide all of those tools - and many more - in a single developer portal.
- Phase 3: Ecosystem (later) - Everyone's infrastructure stack is different. By fostering a vibrant community of contributors we hope to provide an ecosystem of Open Source plugins/integrations that allows you to pick the tools that match your stack.
It looks like an onboarding tool, monitoring, and knowledge database.
In our case, I might want to use that to create a frontend for QA to see relevant information summaries from GitLab, build information from Jenkins, and current test plan status from our integration testing system.
For managers, I might want a frontend that shows them activity on certain branches of certain repos, the number of open merge requests/change orders, and the % of progress towards a milestone.
I hope I'm right, because that could be super useful.
The juiciest part is the animated gif, but all I want is a lot of screenshots and bit of text explaining each feature.
(Now I need to take my own advice and revisit my projects. hahaha)
Any other competitors / SRE dashboard/metric aggregators that folks like? I think this is a level of abstraction above raw datadog/grafana dashboards.
Gitlab seems to be moving into this space a little with their deploy boards and k8s monitoring tools.
Integration with GCP/AWS is also great, I know plenty of people on my team that struggle with their UIs - this would really help them out