
Embrace, extend, and finally extinguish – Microsoft plays their hand - MartijnBraam
https://drewdevault.com/2020/08/27/Microsoft-plays-their-hand.html
======
chomp
This feels like an advertisement wrapped in a bad take. Speaking as someone
who has a couple commits in the Linux kernel, yes the email thing is a pain to
get started. Configuring your client to do the text thing, keeping up with
conversations, learning who's who, you definitely can't jump straight in. The
only way I was able to stay sane is setting up a bunch of filters to tag
conversations and people. And that's not even mentioning the different patch
submission guidelines, perl scripts I have to run, and other random things to
make the patch even just readable by the community.

Configuring all this stuff is definitely a barrier to entry. Now what tooling
looks like to make it easier, that's an entire different conversation. Email
has worked for the Linux dev community for decades and that should be
protected and not disrupted. However, anyone who says there's no room to
improve the onboarding experience, well, I just can't take their opinion
seriously.

~~~
ddevault
>However, anyone who says there's no room to improve the onboarding
experience, well, I just can't take their opinion seriously.

From TFA:

> I agree that the UX of email-driven development could be better!

This is followed by six examples of such improvements, the first of which is
explicitly targeted at improving onboarding:

[https://git-send-email.io](https://git-send-email.io)

~~~
chomp
My man, those tools are great and I really respect the contribution of those
tools to the community. But here's the kernel guidelines, which I am sure you
are already familiar with:
[https://www.kernel.org/doc/html/latest/process/submitting-
pa...](https://www.kernel.org/doc/html/latest/process/submitting-patches.html)

Even with the tooling you provided, it's still a crappy experience (maybe a
little less with the Sourcehut stuff). I know it's easy to read into the
comments and be like "well this person just wants to shove the kernel into
Microsoft's product," but it doesn't have to be this way. The sad thing is,
for better or for worse, Github can handle almost all of those steps in the
patch submission process, save for the ones where you have to actually
describe what's changing. that's like, 15 steps that a newcomer does not have
to even deal with.

I think we all agree on "No Linux Kernel controlled by Microsoft platforms,
please" but Sarah's comments that "this workflow sucks", and then your actions
on "this workflow sucks" \- y'all are saying the same thing.

~~~
ddevault
These "15 steps" are 90% unrelated to the email workflow, and would not be
solved by GitHub. Did you read this document? It's telling you how to be a
good contributor _and_ the mechanics of how to contribute. That being said,
this documentation is sorely in need of a git-oriented overhaul, assuming the
programmer isn't using git is not doing any favors for the 99% who are.

------
toyg
Don't particularly agree with the tone of this "advertorial" but I agree that
the issue of the ecosystem becoming excessively github-centric is real.

I wish Github, Gitlab, sourcehut and (why not) Atlassian, would sit down and
hash out a bunch of formats and protocols that make it easy to collaborate
across _and off_ them. A galaxy of tools, with everyone using their favorite
one, would be a fabulous step forward for developers everywhere.

I know it's not going to happen anytime soon, but one can dream.

~~~
kanox
It's already trivial to move code between git service providers, or even run
your own.

Code review and issue trackers are harder because they're effectively forums
are only useful when centralized.

~~~
ddevault
>Code review and issue trackers are harder because they're effectively forums
are only useful when centralized.

What? This claim requires evidence.

------
belltaco
>They embraced git, and then rather than building an interface on top of email
— the collaboration mechanism that git was designed to use, and which is still
used for Linux kernel development — they built their “pull requests”
mechanism.

How is git designed to use email?

>Sarah Novotny, Microsoft employee, transitive owner of GitHub, and patroness
saint of conflicts of interests

Followed by:

>I represent sourcehut, a GitHub competitor which does what GitHub wouldn’t —
interoperate with open, distributed protocols, and in the form of 100% free
and open-source software

Yeah no conflict of interest there /s

~~~
ddevault
>How is git designed to use email?

git was designed for use with the Linux kernel, which has collaborated over
email since its inception. It includes many features for email-based
collaboration out-of-the-box with no need for a centralized hosting service of
any kind.

[https://www.git-scm.com/docs/git-send-email](https://www.git-
scm.com/docs/git-send-email)

[https://www.git-scm.com/docs/git-request-pull](https://www.git-
scm.com/docs/git-request-pull)

[https://git-scm.com/docs/git-am](https://git-scm.com/docs/git-am)

[https://git-scm.com/docs/git-imap-send](https://git-scm.com/docs/git-imap-
send)

Both Linux and git itself are developed via email, along with dozens of other
projects, *BSD, gcc & clang, coreutils, busybox, ffmpeg, qemu, PostgreSQL,
vim, emacs, most Linux distros...

~~~
belltaco
All those don't sound like git was designed to be used with email, those
projects using email to collaborate sounds like a coincidence and orthogonal
to me.

>It includes many features for email-based collaboration out-of-the-box with
no need for a centralized hosting service of any kind.

What many features?

~~~
cnst
The fact that you refer to those internal commands that are part of every Git
installation as "projects" shows that you're likely unfamiliar with Git CLI.

Because git is so large, every individual command is documented in a separate
manual page, with `git diff` being documented in the `git-diff` manual page,
and so forth.

All the mentioned commands are probably present on your Mac in Apple's
distribution of Git, for example. Even though Apple probably doesn't use any
of those commands for internal development (because big corp, and not invented
here).

------
PaulDavisThe1st
This is a bit tangential, but I wanted to mention how the (GPL'ed) ardour
project decided to view github.

We got burned by the SourceForge apocalypse back in the mid-2000's, which led
to me (the BDFL) declaring that the project would never use a 3rd party host
for our source code repository ever again. Granting a 3rd party the level of
control that SF had (and then abused) is a non-starter for us. This means that
github was never an option for us.

.... as the canonical repository.

However, we recognized the value of the tools that github offered,
particularly allowing random 3rd party developers to create patches in quite
effective ways. Some of the their other tools are useful, too.

So, we self-host a gitlab instance at ardour.org, and that acts as our
canonical repository, but we use hooks to push all commits to the github
version automatically. Random 3rd parties can use any and all of the github
toolset with our code (as can we), but we have no reliance on github in any
important way. We don't even have much reliance on gitlab either - it exists
only to provide access roles and account management on top of the bare git
server. We don't use any of gitlab's "features" either.

After about 5 years, this continues to work well for us.

------
martius
Pull-requests and forks are only a piece here.

It's not the only one implemented by tools that extend Git (Phabricator and
Gerrit, for instance, support pre-commit workflow).

Github also brings a bug tracker (albeit a simple one), discoverability,
hosting (obviously) and integration with many useful tools (CI, documentation
generators, etc). All of this is easy to use.

These pieces greatly contribute to the success of many open-source projects
(of various sizes). Github "extinguishes" alternative because of this, not
because it designed an alternative to email-based workflows.

I suppose that many projects (open-source or not) want to use tools that
extends Git. Maybe what we need is a standard making these tools interoperable
and open.

~~~
bccdee
Yeah, I agree. There are open-source Github clones like Gitlab and Gitea that
still follow Github's web UI model, so I think it's weird to pose this as a
conflict between Github and plain-text email when it's two separate conflicts:
one between proprietary Github software and other open-source hosted git
systems, and the other between the plain-text email model and the more popular
web UI model.

~~~
Seirdy
I responded to this argument in another comment:
[https://news.ycombinator.com/item?id=24295610](https://news.ycombinator.com/item?id=24295610)

TLDR: the alternatives are also centralized and give you vendor lock-in since
issues/PRs are tied to the instance.

------
eterps
What has always suprised me about open source software is that the source is
always 'out of band'. I can imagine a future where all open source software
you use has an 'edit source' function (as opposed to a mere view-source
function we have on the web currently).

Even more useful would be the ability to make a change within a certain
context, similar to how 'inspect element' works in the browser.

A change in the source would result in a patch which can then stay 'personal'
or be automatically shared with others looking for similar functionality, or
getting noticed by the maintainers who might incorporate it in the main
releases after a refactoring.

Of course I realise that this workflow is not technically possible in the way
software is currently built, but it does seem worthwhile in the future.

~~~
cxr
> What has always suprised me about open source software is that the source is
> always 'out of band'

From "Open Source is not enough" by Adam Spitz[1]:

 _The Open Source movement is great, but it doesn’t go far enough. […] When I
feel the Urge To Tinker, only rarely does it feel like a loud voice shouting
in my brain with enough energy to propel me to find the website and download
the source code and figure out how to find the part of the code that
corresponds to the thing I’m looking at on the screen and make the change and
restart the program and retrace my steps. Much more often it’s just a quiet
voice mumbling, “Hey, it’d be kinda neat if…” and then I think, “Well, it’s
Open Source, I guess I could go download the source code… but… meh, it’s so
far out of my way, not worth it,” and the urge fizzles out. I think that a lot
of potential human creativity is being wasted this way._

> Of course I realise that this workflow is not technically possible in the
> way software is currently built, but it does seem worthwhile in the future

I've been thinking a lot about this and working on it. On a technical level
it's more possible than it seems—even with "conventional" toolchains—but the
primary obstacle is going to be social rather than technical. Funnily enough,
GitHub's popularity and its measurably less-than-optimal workflows actually
serve as a preview of how difficult it will be overcoming stalwartism from
people who are technophiles first and only think of productivity as an _n_ th-
level concern—somewhere at a lower priority than social media presence and
ginning up demand for the skills and practices they've alrealy invested in.

1\.
[https://web.archive.org/web/20151016184630/http://adamspitz....](https://web.archive.org/web/20151016184630/http://adamspitz.com/avocado/2011/05/05/open-
source-is-not-enough/)

~~~
eterps
Can you share some of your thoughts how this might be technically possible
even with conventional toolchains?

~~~
cxr
I've written about it before, but nothing I've worked on is in a good state
that I can point to as an example. A big part of turning the tide is going to
be about getting the messaging right, because even with the ability to
grandfather in conventional toolchains, it requires a perspective
shift/attitude adjustment on the part of developers, and _oh boy_ are
developers going to revel in shitting on the idea and downplaying the costs of
continuing to do things the same way we always have. It involves non-
destructive compilation and the ability to (re)publish tooling for a common
platform.

I've seen multiple comments here on HN in threads about compiler development
where chrisseaton points out that compilers are pure functions. All you really
need to do is combine that observation with the observation that we already
have a near-universal substrate for doing computation. (And it might help
further to recognize that releasing an open source project is fundamentally an
act of publication, almost-but-not-quite in the vein of Knuth's literate
programming.)

------
johnduhart
The author took an article about somebody suggesting that having email be the
only mechanism for contributing to the Linux kernel might be a barrier to
entry for new developers, and construed it into this piece on how the evil
Micro$oft is looking to dismantle distributed git development.

From the original article:

> Picking her words carefully, she said work is being done towards “moving
> from a more text-based, email-based, or not even moving from, but having a
> text-based, email-based patch system that can then also be represented in a
> way that developers who have grown up in the last five or ten years are more
> familiar with."

That seems like an entirely fair statement to make for somebody who is on the
board of The Linux Foundation.

~~~
cxr
The interesting thing to note is that the "way that developers who have grown
up in the last five or ten years" refers to the way of working on GitHub, and
that GitHub made it deliberately difficult to interoperate with other
development models (like the kernel's, or old-style Bugzilla-and-patches) for
business reasons. When you start out with trying to be a social network for
developers, you'll happily take inspiration from the way Facebook divides the
world into "Facebook" and "not Facebook", which naturally results in erecting
barriers between your userbase and outsiders so long as those barriers can
feed into your growth strategy of converting the outsiders into even more
users.

Having said that, this blog post is foremost not actually a criticism of
Microsoft/GitHub, but instead trying to draw attention to Sourcehut. Sourcehut
is not a good example of how to solve GitHub's problems; if we were in a
hypothetical world where the choices were GitHub or Sourcehut, the skew would
remain forever what it is today, and the lead developer doesn't seem capable
of understanding why and fixing Sourcehut's flaws.

~~~
sreevisakh
I feel that the author is drawing attention to Sourcehut because there are no
alternatives that use the workflow he is pitching. It's not as great as Github
at the moment, but he is trying to show what it can be. He always comes off as
incapable of fixing sourcehut's flaws because a lot of suggestions he gets are
contradictory to the fundamental design he has in mind. Despite this,
sourcehut is making steady improvements. Drew also puts up invaluable
resources which helps people like me get into the email workflow. Sourcehut
currently has a steep learning curve, after which you would immediately
recognize the design as a safer bet in the long term.

------
mouldysammich
There are an awful lot of people in the comments here calling this an
advertisement and being upset about it. How are those that are upset about it
not upset that this is an obvious conflict of interest and advertisement for
microsofts github platform.

~~~
cnst
This is the most upsetting part about it, IMHO. The reason Microsoft pays a
salary to this person is precisely to get GitHub mentioned into these sorts of
conversations -- even though it's widely known that GitHub doesn't support
such usecases.

But, of course, there's never any time to mention such facts; but here's an
example if anyone's curious: [https://blog.ffwll.ch/2017/08/github-why-cant-
host-the-kerne...](https://blog.ffwll.ch/2017/08/github-why-cant-host-the-
kernel.html).

Yet when a one-person team tries to provide relevant examples on the issues of
dealing with Microsoft, pitching a potential solution, and offering their take
on it -- fully open-sourced -- they get discounted as advertisement -- even
though their service is free and open source.

~~~
kasabali
> Yet when a one-person team tries to provide relevant examples on the issues
> of dealing with Microsoft, pitching a potential solution, and offering their
> take on it -- fully open-sourced -- they get discounted as advertisement --
> even though their service is free and open source.

it's upsetting indeed.

I'm 100% sure had he didn't mentioned SorceHut the same people would've
complained instead that how there were no projects for improving mail based
workflow, why didn't OP offer a suggestion, or that it's open source so why he
didn't shut up and wrote a patch instead.

------
tboyd47
Calls to mind the way Google, Facebook, etc. took over OAuth by flooding the
committee with their own representatives and turned it into something
completely different with OAuth2.

~~~
sreevisakh
Reminded me of a quote from another one of Drew's blog posts: "The web and web
browsers have become Lovecraftian horrors of an unprecedented scale. They’ve
long since left “scope creep” territory and entered “oh my god please just
stop” territory, and are trucking on through to hitherto unexplored degrees of
obscene scope." [1]. I sometimes feel like many standards and software (incl
open source) are designed to make it impossible for an individual or small
team to implement or manage.

[1] [https://drewdevault.com/2020/08/13/Web-browsers-need-to-
stop...](https://drewdevault.com/2020/08/13/Web-browsers-need-to-stop.html)

------
rsweeney21
This is an ad for sourcehut, a GitHub/Microsoft competitor.

~~~
eterps
Sourcehut is competing with Github/Microsoft? That's new to me :-D

Focusing on a niche hardly counts as competition to me.

'an alternative to Github' sounds more realistic.

------
koluna
This person seems very upset about something that customers actually love.
Speaking for myself —- GitHub pull requests are amazing. I have an easy way to
collaborate and propose changes to my projects through a human-friendly
interface that doesn’t require me to use tech from early 90s.

Surprise - technology evolves. We’re not all going to stick with BBS just
because “it’s a standard everyone uses, ZOMG don’t extinguish it!”

~~~
croes
Wait until you need a Microsoft account for accessing github, wait until your
account gets blocked. All of sudden all of your code is out of reach.

~~~
koluna
Wait until your mail server goes down or experiences a malfunction that erases
all mail threads. Wait until your hard drive fails and you lose all the code.

There can be a lot of hypotheticals. GitHub does not preclude you from using
git for what it’s intended to be - a distributed source control management
system. If you don’t have backups, that’s on you.

~~~
ddevault
Ah yes, because apparently no one besides GitHub is qualified to run reliable
services.

[https://status.sr.ht/](https://status.sr.ht/)

[https://www.githubstatus.com/](https://www.githubstatus.com/)

[https://sourcehut.org/blog/2020-07-03-how-we-monitor-our-
ser...](https://sourcehut.org/blog/2020-07-03-how-we-monitor-our-services/)

[https://drewdevault.com/2019/01/13/Backups-and-redundancy-
at...](https://drewdevault.com/2019/01/13/Backups-and-redundancy-at-
sr.ht.html)

When your mail server goes down, nothing happens, the internet continues to
function normally and then when it comes back up you will receive all of the
emails that were sent to you when it was offline. If the mailing list goes
down, you can still send emails and you're collagues probably won't even
notice the outage because they were CC'd. When the mailing list comes back up,
it'll receive all of those emails in the same way and get caught back up.

When GitHub goes down, everything stops. And GitHub goes down often!

~~~
sreevisakh
I started trying out sourcehut out of curiosity and only recently started
understanding the resilience and convenience of email workflow compared to
Fork-PR workflow. It's also is clear to me that the pains with the email
workflow is the user experience and not the workflow itself. None of this was
apparent in the beginning (and hence the complaints). It would be great if
there was a single page listing comparison of both workflows against specific
technical points like these.

------
algorithmsRcool
I have a hard time taking this seriously. Email is a pain to use compared to a
dedicate feature UI.

I think the real problem is the need for a standardized pull request protocol
that people can wrap. Then let people use email, a website, an API or whatever
they want.

~~~
koluna
100% this. I will take GitHub’s PR process over emailing diffs any day. It’s
just a matter of improving user productivity.

~~~
ddevault
Absolute nonsense. Emails are far more productive. I've made a video
demonstrating the difference here:

[https://spacepub.space/videos/watch/1619c000-7c44-4330-9177-...](https://spacepub.space/videos/watch/1619c000-7c44-4330-9177-29a0854bd759)

~~~
koluna
Literally nothing from what you showed is a pleasant customer experience.
Overly complex workflow that can be simplified by what GitHub is doing.

Think of a beginner developer who wants to learn how to contribute to an open
source project. If they would be put through the process you showed, it will
be a meat grinder that is going to be error prone. Zero need for that when
GitHub has a great UX that focuses on getting shit done instead of fiddling
with conventions from the 90s.

~~~
cnst
That's why there's shell completion, and history search; then with practice,
keyboard shortcuts as well; all vastly improve productivity.

I think a lot of people wrongly assume that GitHub is so "easy" that you don't
actually have to spend any time to learn how to use it.

But that's not the case at all.

Moreover, GitHub has so many quirks, that most Enterprise shops don't use it
internally for their own internal projects (unless they release said projects
into OSS); so, when you work for any company that's not just a startup, you
likely have to learn yet another set of tools. And again if you switch the
company.

Compare this with email. As the original article reveals, Linux and OpenBSD
happen to have almost entirely identical workflows for patch submission, which
is actually REALLY weird, considering that the two projects have entirely
different histories, methodology, and use entirely different revision control
systems, where OpenBSD still uses CVS, yet Linus Torvalds hated CVS so much, I
recall that he refused to use it even before Git was ready and the licences
for the predecessor system have already been withdrawn (I recall he hated CVS
so much he simply used plain patches or tarballs).

~~~
algorithmsRcool
> I think a lot of people wrongly assume that GitHub is so "easy" that you
> don't actually have to spend any time to learn how to use it.

For a lot of people, probably the large majority of people, GitHub's (or
GitLab's) UI is simpler to work with than a terminal. The overwhelming success
and popularity of GitHub has demonstrated the value of it's UI. I just don't
see how you could convince the Atom/VSCode generation of developers to go back
to mailing lists.

> so, when you work for any company that's not just a startup, you likely have
> to learn yet another set of tools. And again if you switch the company.

As far as I am concerned, the email workflow is just "another set of tools"
too. Except no company I've worked with or worked for has used it.

~~~
kasabali
> For a lot of people, probably the large majority of people, GitHub's (or
> GitLab's) UI is simpler to work with than a terminal.

The topic isn't about majority of users. It is specifically about Linux kernel
developers.

~~~
algorithmsRcool
Fair point, I've strayed from the topic at hand.

------
vikramkr
What is Microsoft extinguishing here? Email?

~~~
fnord123
Distributed git development. Moving us back to central repos.

~~~
dgrin91
But you can still do distributed repos with github... nothing it preventing
you from changing your remote origin...

~~~
nikivi
Only issue is if you decide to use GitHub features like issues/prs or comments
on code, those won't be transferrable I think. At least easily.

~~~
kanox
Email archives are even harder to transfer, and to make sense of in general.

For example if you want to find the review comment thread for linux commits
there is no reliable way to do it, your best bet is to google the title.

~~~
Seirdy
> Email archives are even harder to transfer

Mailing lists are trivial to distribute. Thanks to maildir and mbox, everybody
has a full copy of the mailing list just as they have a full copy of the
repository.

~~~
kanox
> Thanks to maildir and mbox, everybody has a full copy of the mailing list.

No, that only covers emails that you personally received. You need another
mechanism for older emails.

~~~
ddevault
A mechanism which is provided by mailing list archives in the form of mbox
downloads. If you question the reliability of this (but apparently not the
proven unreliability of GitHub), as a fallback you can just ask an older
subscriber to give you their mbox.

------
api
It’s not a conspiracy. If customers demanded decentralized, companies would be
pushed down that path. They don’t. Customers demand ease of use, features, and
speed, and all that is technically easier to deliver with a centralized model.
Why impose extra engineering cost for decentralization almost nobody cares
about?

GitHub’s largest value add IMHO is search, and that is probably the hardest
thing to decentralize.

~~~
bccdee
Github absolutely would not make itself interoperable just because users
wanted it to. That would be a terrible business move -- it would lead to
Github hemorrhaging users and projects as they left to other, smaller
competitors while still begin able to interact with the users and projects
that remain on Github.

It's not Github's job to do what customers want. It's Github's job to do
whatever will make them money, whether it's good for customers & the broader
tech community or not. You're right that it's not a conspiracy though --
there's nothing secret about this at all. It's just business.

~~~
api
You can already mirror a repo easily to/from GitHub, and moving is pretty easy
unless you are deeply invested in GitHub proprietary features.

People use GitHub because people use GitHub. It’s a classic network effect.

~~~
bccdee
I mean like me forking your Github repo to Gitlab and then submitting a PR
from my Gitlab fork to your Github repo. Can't do that. If I want to
contribute to a Github project, I need a Github account. That doesn't apply to
interoperable protocols like email -- you can use gmail and I can use yahoo
mail, or even a self-hosted mail server, and we can still send messages to
each other.

This would certainly be doable, but they wouldn't do it, because it would
damage their market share.

------
nova22033
_I represent sourcehut, a GitHub competitor_

Maybe lead with that?

------
kanox
Good: Fuck plain-text email.

We should stop relying on duct-taped solutions from the 70s. Web-based code
review solutions are vastly better.

~~~
santoshalper
Agree. The problem with github is not that it is web-based instead of email-
based. The problem is that it is a proprietary walled garden.

An open source direct competitor to github is what's needed, not going back to
email as the UX for development.

~~~
kanox
I don't see a shortage of github competitors.

For example GitLab works well and also allows for self-hosting.

~~~
bccdee
But Gitlab is its own walled garden (albeit an open-source one). What we'd
ideally have is a platform-agnostic PR protocol. Sadly, we're not likely to
get one.

That being said, I agree that "let's all use plain-text email" isn't a very
good alternative. It's perfectly possible to use Gitlab or Gitea and have the
web UI experience without marrying yourself to a proprietary ecosystem.

~~~
sreevisakh
Is the plain-text the problem? Because Gitlab/hub use them too - as markdown.
Their UI layer is built on top of it. Why not do the same thing for emails?
Plain-text in SMTP as a communication layer, but a rich UI layer on top of
that at client level? Make a combined mail + list + git client?

~~~
bccdee
I don't have any inherent problem with plain text, I just think that the
pull/push/PR workflow is nicer than emailing patches to a mailing list.

------
simonkafan
tl;dr: A Github competitor is looking for reasons to rage against
Github/Microsoft.

