
Backstage: an open platform for building developer portals - JamesQuigley
https://github.com/spotify/backstage
======
freben
Member of the Backstage team here. Sorry if the messaging is unclear; we'll
work on it.

Allow me to tell you about Backstage in the form of a story. Assume you're at
a medium-to-large-sized company, in a team tasked with boosting developer
productivity. You've developed a kick-ass pipeline to calculate test flakiness
based on historical build logs and some data from your internal userinfo
service. Problem is, userinfo has suddenly started exhibiting p99 response
times in the 10s of seconds. It's not feeling well.

You go to backstage.yourcompany.net and search for userinfo. You reach the
overview page for the service. It shows a ton of information, some of it
pertaining to who owns and operates the service. Apparently it's the devcore
team, they're on #devcore-support on slack according to the page but they're
in the US west coast timezone so they're asleep. Well you are using the
PagerDuty plugin in Backstage so there's a big "page us" button right there.

So now you've got the issues solved and you need to expose the pipeline output
somehow. Cool, you go to backstage.yourcompany.net and click "Create", and
select one of the templates. It scaffolds a Go service in your GitHub org,
according to your company's best practices. All the builds and deployments are
already set up and hot. You add some code to your service and shortly it's up
and running and serving the pipeline output with caching out of S3. You check
backstage.yourcompany.net to see how all aspects of your service's runtime
status is looking. All good. Now, how do you actually get that useful data
into your company's other devs' hands?

Well you COULD make an entire microsite, or some CLI tool orrrr... Come to
think of it, they are already all using backstage.yourcompany.net, and there's
already an integration in there showing all builds for all services, and it
has all of the utilities and auth and bells and whistles already in place for
you. So you make an internal backstage plugin. You find that it's pretty much
a matter of cloning your internal backstage repo, running a create-plugin
script, and boom you are up and running. You add a data source feeding off of
your new Go service, you add a column to the builds table ... and you have
your own team review and merge it because it's covered by CODEOWNERS. And
everybody rejoiced and told you how useful this new column was to them. You
start getting a bit famous on your internal Slack, actually.

Now you are getting greedy. You feel that actually, this plugin could be
useful to the general public - as long as the build logs are in a known
format, you can eat them. You start open sourcing the pipeline, and the
service - and, critically, also the backstage plugin.

This is how things are internally at Spotify today. What we released here
today is not this entire story yet, but we are lifting more and more out into
the open at a rapid pace. Check the repo and microsite for more info, and hold
tight.

More is coming.

------
kostarelo
Predictions so far:

\- "to me it seems like it's a way of abstracting all the different
infrastructure tooling that a large-scale company has."

\- "It looks like an onboarding tool, monitoring, and knowledge database."

\- "So it's a frontend to put in front of your existing frontends so
developers can infrastructure the infrastructure?"

\- "Seems like a developer-oriented version of Confluence, if I'm reading it
correctly, but I'm also a bit unsure."

\- "Seems like a competitor for Datadog and other monitoring tools to me."

That message really couldn't be more unclear.

~~~
tnolet
A sort of meta lesson here for bootstrappers and technical founders is that
explaining in simple terms what your product does is actually extremely
difficult. YC hammers that point a lot too.

Good example here is the “open, plugin etc.” feature being almost right the
top. I don’t care if it’s open or that I can make a plug-in if I have no clue
what it does.

~~~
kostarelo
Also, start with the problem. Get people engaged first, then offer a solution.

------
royjacobs
For the people here that are confused as to what it does, to me it seems like
it's a way of abstracting all the different infrastructure tooling that a
large-scale company has.

Consider a team that wants to deploy something to the cloud in a company. They
are probably engineers who aren't necessarily well-versed in all cloud
tooling, yet they are required to know and understand Terraform,
GCloud/AWS/Azure CLIs, Gitlab CI, Prometheus, various monitoring and alerting
tools, Kubernetes, Docker, etc. And this is just to deploy their application.
Once other resources come into play (databases, queueing, etc), they need even
MORE knowledge.

A tool like Backstage allows developers to get a uniform overview of all their
resources, regardless of how and where they are running. I'm not sure if it
also allows the creation of new resources, but I guess that would be a fairly
obvious direction to go in.

It won't solve the need to _at some point_ know a bit more about the technical
details about all the underlying tools, but it can allow teams to gradually
learn about all this new tooling instead of having to know about everything
upfront.

~~~
tra3
Borg, Kubernetes, et al? Abstracting complexity away is great, but at a
certain level you just can't hide it, I think. Would love to be proven wrong,
though.

------
kenrose
This is by Stefan Ålund from Spotify, who was (is still?) product manager for
System-Z. System-Z is Spotify's internal service catalog and developer
productivity suite. (I'm not sure if they still use the name System-Z
internally).

Here's an old talk about it (2016):
[https://www.youtube.com/watch?v=7XDA044tl8k](https://www.youtube.com/watch?v=7XDA044tl8k)

and here's some slides from a newer talk (2017):
[https://microcph.dk/media/1024/conference-
microcph-2017.pdf](https://microcph.dk/media/1024/conference-
microcph-2017.pdf)

In a nutshell, the aim is to make it easy for developers to find out info
about other services as well as make changes to the services they own.

It looks like Backstage is about open sourcing big parts of System-Z so that
others can build on it.

Disclosure: I'm CTO @ OpsLevel, where we make a SaaS-based service catalog
product.

------
kyeb
I just skimmed much of their website, and felt like I was left still wondering
what Backstage actually _does_.

~~~
moralsupply
It's a god system. It tries to do everything, but in the end does nothing
well.

~~~
sjoeboo
as a user of it, I beg to differ.

------
adim86
Is it me or is this really unclear? I get it is supposed to simplify the back-
end for developers but for what exactly? Scheduling tasks? looking at metrics?
I get I can install plugins but what plugins? Maybe this isn't for me, but it
is unclear to me precisely what scenario this is solving for?

------
cmckn
So it's a frontend to put in front of your existing frontends so developers
can infrastructure the infrastructure?

~~~
danudey
The first read I had on this (which sounds like it's not correct based on
other posts) is that it's a simply-extensible frontend for creating portals
from a series of disparate frontends of existing systems.

In our case, I might want to use that to create a frontend for QA to see
relevant information summaries from GitLab, build information from Jenkins,
and current test plan status from our integration testing system.

For managers, I might want a frontend that shows them activity on certain
branches of certain repos, the number of open merge requests/change orders,
and the % of progress towards a milestone.

I hope I'm right, because that could be super useful.

------
simplecto
I agree with other comments here about unclear.

The juiciest part is the animated gif, but all I want is a lot of screenshots
and bit of text explaining each feature.

(Now I need to take my own advice and revisit my projects. hahaha)

------
jakearmitage
That's really useful for big companies having problems with documenting their
ecosystem without relying on tons of Word documents.

------
hbcondo714
Looks like a developer portal similar to the AWS[1] and Azure[2] open source
offerings where a non-developer can execute API calls in the web browser.

[1] [https://github.com/awslabs/aws-api-gateway-developer-
portal](https://github.com/awslabs/aws-api-gateway-developer-portal)

[2] [https://github.com/Azure/api-management-developer-
portal](https://github.com/Azure/api-management-developer-portal)

------
wgerard
Seems like a developer-oriented version of Confluence, if I'm reading it
correctly, but I'm also a bit unsure.

------
theptip
Sounds like similar to the pitch for
[https://buoyant.io/2019/11/18/announcing-
dive/](https://buoyant.io/2019/11/18/announcing-dive/), which is in closed
beta and I haven’t tested yet.

Any other competitors / SRE dashboard/metric aggregators that folks like? I
think this is a level of abstraction above raw datadog/grafana dashboards.

Gitlab seems to be moving into this space a little with their deploy boards
and k8s monitoring tools.

------
seemslegit
New thing aims to help developers deal with the overhead of having too many
things to worry about, uses plugins.

------
jezzadebate
It needs clearer messaging, but I think this looks great! Like a turbo CMS
built for developers.

Integration with GCP/AWS is also great, I know plenty of people on my team
that struggle with their UIs - this would really help them out

------
beardedman
Wow, so a dashboard + backend? Or a framework? Or both?

------
bellwether
Seems like a competitor for Datadog and other monitoring tools to me.

~~~
sjoeboo
its not a monitoring platform (though we have our own in-house opensource tool
for that), its a portal for developers and engineers to mange...everything.
create new components, manage capacity and deployments, see your CI builds,
find documentation, etc etc etc. ties all the separate systems and tools
together into a single site.

~~~
kingosticks
So it's an extensible dashboard / intranet site? A lot of people here are
having a hard time understanding what this project is for. I think the README
needs work.

