
The advantages of an email-driven Git workflow - Cmerlyn
https://drewdevault.com/2018/07/02/Email-driven-git.html
======
matharmin
The article describes a lot about the workflow, but only two sentences about
the advantages:

> A very large body of email-related software exists and is equally reliable
> and well-understood. You can interact with email using only open source
> software and customize your workflow at every level of the stack -
> filtering, organizing, forwarding, replying, and so on; in any manner you
> choose.

Using email software in general is well-understood, but using it for Git is
much harder than using GitHub. Customizing the workflow is also something you
probably shouldn't want to do unless you're really working with a massive
project such as the Linux kernel.

~~~
Sir_Cmpwn
Author here. I didn't load all of the advantages into a single conveniently-
quotable part of the article, but I think I covered them all the same. In
addition to what you quoted:

>reviewing patches is quite easy with the email approach as well [...] It’s
just email - you can reply, forward the message, Cc interested parties, start
several chains of discussion, and so on

The quote you gave also misses an important point directly preceeding it,
which is that email is decentralized and federated. I also wouldn't understate
the importance of all of it being open source.

~~~
cryptonector
That's all true, but (and note: I didn't downvote you) the problem I have with
email for code review is that finding all the comments made over time on one
patch is... a chore. Mind you, for many years I did code review over email, so
I'm used to it, and since I'm a TUI sort of person, it works very very well
for me _except_ for the chore that is finding and collating all comments.

This chore is very real, and very time consuming. Reviewers need to make sure
that new versions of a patch are responsive to past [accepted] commentary.
Authors need to find all commentary when updating patches in order to be
responsive to it.

Not that code review tools from GitHub, GitLab, and others, are all that good.
I've yet to find a code review tool that really works for me and the people I
work with.

Whether you use email as a transport and/or interface, or something more weby,
or whatever, at the end of the day, for me, the real win lies in code review
and commentary management and UIs. Because I'm a TUI person, my approach would
be to build a code/patch+commentary format, something like (designing on the
fly here):

    
    
        - foo.c
        |     /* This is code (or patch) */
        ? jane   ^^^^^^^^^^^^
        ? jon    The above is code, yes, but this is commentary!
        |     while (!review_done)
        |         ...;
    

or

    
    
        - 0001-This-is-a.patch
        | diff ...
        ...
    

Something more markdown-y would work too.

An XML version wouldn't be a bad idea, provided a) that the schema is trivial,
b) that there are tools for converting between a simple text format (e.g.,
like the above), c) preferably things round-trip. Obviously, with XSLT/XPath,
(b) should be feasible, especially if we heed (a). And (c) should be possible
if the XML schema is kept simple enough (but see below).

What's nice about this is that as long as there is version information we can
merge commentary from many emails into one document (per file/patch), which
then could make automation of commentary collation possible.

This needs lots more thought, I grant that. For example: how to cut-down
quoted text without losing the ability to trivially merge commentary? Also, it
would be very tempting to add lots of metadata, which would drive one to XML.
I'm not a huge fan of XML, mostly due to lack of TUI editors for it... but XML
does have the advantage of XSLT/XPath.

It seems to me that this is a problem that can be solved reasonably well, and
that this is a problem worth solving. If we can solve it in a way that works
well with TUIs, it will also work well with email, so this should be of
interest to you (I hope so as you seem to have the time and energy to work on
this!).

Thoughts?

~~~
Sir_Cmpwn
Thanks for your detailed thoughts, I'm taking all of this down and using it to
influence my work on lists.sr.ht (the mailing list and code review tool I'm
building).

The chores of this approach - finding and collating feedback, sheparding a
patch through the flow, and so on - are all things I'm acutely aware of. I
don't think we need to throw the baby out with the bathwater like GitHub does,
though, and I see these as places to improve our tools rather than replace
them.

Take a look at this here:

[https://git.sr.ht/~emersion/python-
emailthreads/tree/test/da...](https://git.sr.ht/~emersion/python-
emailthreads/tree/test/data/multiple-replies/output3.txt)

The specific output format you see here is just a testing tool, this isn't the
final user-facing product. The important part is ingesting a bunch of emails
and creating a thread where you can associate all of the responses over time
with the particular parts of the patch. Combining this with a gerrit-like tool
which tracks new patches over time and avoids duplicating review work is
planned.

~~~
cryptonector
Thinking about it more... There is not much need for XML.

The inputs are plaintext replies (or HTML formatted as plaintext).

The original document is immutable.

There is no state (because it can be collated from scratch every time). EDIT:
If updates are allowed in any non-email based way (e.g., via a BUI, REST APIs)
then email must be generated and sent so that all the state is in the email
archives and all participants can collate the same state.

But the result of the collation is more like a database and less like a
document. In particular there's not much need for marking up commentary.

Multiple collation output formats will be needed: plain text, XML, XHTML,
JSON. These formats/schemas are only needed for display purposes, not
interchange, as the collation results can be computed independently by all who
have access to the emails. This means there is nothing to standardize!

Thus collation state can be kept any way you want. XML, SQLite3, some NoSQL,
any in-memory representation you like -- whatever suits you. Still, I'd use
anything that has a nice, declarative query language. XML (XSLT/XPath), JSON
(JSONPath, jq), or an RDBMS (SQL) -- all will do. For you, and given your
emphasis on open source, SQLite3 or PostgreSQL will work best, I think. You
can generate all of the output formats you need from such a database.

------
kazinator
> _Popular email clients have caused terrible ideas like HTML email to
> proliferate_

That's a very backwards attitude that will turn away people.

HTML in e-mails is a useful tool. I encourage it in the dev mailing lists that
I operate.

There is no conflict between HTML use and all the various dev-related use
cases like posting patches or pull requests or whatever.

~~~
andreareina
Could you go into why you encourage HTML in the dev mailing lists you operate?

~~~
kazinator
Yes. I like the HTML formatting; it makes for more effective communication.
You can use bulleted paragraphs; bold, italic, typewriter font.

In a HTML message, you can have an indented code block which is not physically
indented. When it is copied and pasted, it has no indent.

Hyperlinks and images can be used in HTML messages, which is convenient.

Color is possible: syntax colored code can be incorporated into an HTML
e-mail.

Also, I use an mailing list archiver called Lurker, which I customized such
that it allows HTML messages to be incorporated into the archive. So poor
archiving of HTML mails isn't any objection.

------
jefurii
The author mentions this several times, but the Linux kernel project workflow
is built around email, as described on LWN[0] and discussed here[1] and on
Reddit[2]. [edited for format]

[0] [https://lwn.net/Articles/702177/](https://lwn.net/Articles/702177/)

[1]
[https://news.ycombinator.com/item?id=15370620](https://news.ycombinator.com/item?id=15370620)

[2]
[https://www.reddit.com/r/programming/comments/73gpys/why_lin...](https://www.reddit.com/r/programming/comments/73gpys/why_linux_kernel_development_still_uses_email/)

~~~
ChrisSD
And git was specifically designed for that workflow.

~~~
ChrisSD
Since this is oddly getting down votes for some reason, I'll quote
wikipedia[0]:

> Git was created by Linus Torvalds in 2005 for development of the Linux
> kernel, with other kernel developers contributing to its initial
> development.

[0] [https://en.wikipedia.org/wiki/Git](https://en.wikipedia.org/wiki/Git)

~~~
paulddraper
Further, git provides utilities specifically meant to work with email
workflows, e.g. git am.

------
nunez
I'm a huge fan of Mutt and hate HTML email, but OP comes off as a kind-of
curmudgeon. Reviewing pull requests semi-interactively on a browser or even
though vim-diff in a single place has long beat email patch diffs for me.

Also, Drew doesn't really delve into what makes email clients "bad." Email is
so much more accessible than it used to be thanks to Outlook, Hotmail, Gmail
and the like. HTML allows people and companies to be creative/abusive with
their email. Plain text is quite restrictive.

~~~
cryptonector
Suppose there was a text-based format for mixing code/patch and commentary,
and that this format lent itself well to processing in such ways as:

    
    
         - merge/collate commentary (from many emails, say)
         - display source with collapsed/collapsable commentary
         - track commentary accepted/rejected/extant/addressed status
         - track commentary metadata (who, when)
    

Would that address your comment / needs?

I mean, I use GUI/web-y code review tools because I find them much easier to
use than email, but only because the problem they solve is: keeping track of
all commentary. Collating comments (and status) from many emails is a time-
consuming chore -- I hate it, and it's the only reason I don't want to do code
review over email, but it's the _only_ reason I have for not wanting to do
code review over email. If we can solve this problem, then I'd be ecstatic to
do code review over email.

~~~
sahil-kang
I believe at least some of the Linux maintainers use _patchwork_ :
[http://jk.ozlabs.org/projects/patchwork/](http://jk.ozlabs.org/projects/patchwork/)

~~~
StillBored
But patchwork is a pale comparison to many of the web based collaborative code
review tools. Its basically a patch tracker plus some simple state tracking,
rather than a full blown interactive communications tool.

------
ahnick
How would someone new to a project who might benefit from looking at
historical patches AND the commentary surrounding them be able to access that
information in an email-driven workflow? In a pull request model on GitHub,
you can go in and view all the old comments from the pull request and get a
good understanding of how/why a feature was implemented. Missing the
contextual explanation of a feature and only being able to look at the commit
history if you weren't on the mailing list to begin with seems like a big
limitation to me. Or am I missing something?

~~~
Sir_Cmpwn
Mailing lists usually have public archives. Here are the archives for Linux
and its various subsystems:

[http://vger.kernel.org/vger-lists.html](http://vger.kernel.org/vger-
lists.html)

~~~
StillBored
Right but, then you have to sift through the dozens of unrelated comments, and
if there are multiple versions of the patch frequently finding them and cross
referencing the changes can be painful.

You need look no further than the versionX->versionX+1 logs some of the kernel
developers do to understand the problem. From one patch set to another the
conversation around a piece of code gets lost, so much so that for long
running patch series people will frequently show up and make the same request
someone earlier in the series made because they missed the 20 emails talking
about the pros/cons of doing something a certain way 6 months earlier.

------
na85
I seem to be in the minority of HN users that actually like Gmail and strongly
dislike things like mutt or pine.

Nothing else syncs to my phone in a manner that "just works" and doesn't
require excessive fiddling, or using k9mail which appears to quite simply not
work at all.

So with that said, how does a person use git via email with the Gmail ui?

~~~
pm215
I use gmail for code review on a "send email patches" project (QEMU), and what
I do is:

* for sending patches, use git-send-email from the command line

* for reading, use a greasemonkey script to make gmail use a fixed width font

* when I need to apply a patchset locally I use the "patches" tool [https://github.com/stefanha/patches](https://github.com/stefanha/patches)

I also have a couple of gmail "canned responses" set up for things like "write
my reviewed-by tag into my email reply".

It's not fantastically ergonomic, but it's good enough that I haven't really
felt the need to use something other than gmail web interface.

~~~
stefan_
I have the fixed width font, but how do you deal with the 78 columns limit? I
guess you can have GMail auto-break it for you as it does when sending as
text, but I think that ends up looking horrible over manually selected break
points. But I'm really lacking some vertical bar to hint me where the limit
is.

~~~
pm215
I have my gmail signature set to an ascii art 'ruler', so I know where column
75/80 are while I'm writing, and then I delete it when I'm done (and I do
manual line breaks and if necessary manual rewrap of paragraphs)... Clunky but
it works.

------
jwilk
If you send patches directly to the maintainer (rather than to a dedicated
mailing list), you should put the project name in the subject, so that the
maintainer doesn't have to guess which project this is about:

    
    
      $ git config format.subjectprefix 'PATCH foobar'

~~~
rwmj
It's be nice if this could be defaulted to something like the base name of the
containing directory, so I could put this setting in ~/.gitconfig

~~~
Sir_Cmpwn
A simple wrapper script or alias around git-send-email could help you out
here. The config option has the same effect as passing `--subject-
prefix="..."`

------
golergka
> It works well for > 15 million lines of code, thousands of developers: paid
> and unpaid, professional and amateurs alike. So it can benefit your projects
> as well.

While I like git in general, this particular argument's logic is completely
backwards. If something works well for 15 million lines of code and thousands
of developers, then most likely it will be a gigantic overkill for your small
project with a team of four, and overhead that goes unnoticed at Linux's scale
will be crushing for you.

At least, that's what I would think if I didn't know git first-hand. The fact
that git turned out to be useful for small project is very atypical for a
solution born out of needs of a huge one.

------
sarah180
> Popular email clients have caused terrible ideas like HTML email to
> proliferate

Regardless of whether it was a good idea or not, this ship sailed many years
ago. In the 21st century, email is rendered in HTML, and nobody but a few
engineers would even contemplate taking away the ability to render links,
formatting and images in email.

Personally, I would argue that email is on its way out. Personal
communications have largely transitioned to things like SMS, Snapchat,
Facebook Messenger, iMessage, Twitter, etc. Transactional email notices are
increasingly being supplanted by mobile push notifications. Businesses are
making heavy use of hybrid push-pull channel systems like Slack.

IMO, the decentralized, federated nature of email was its downfall. It became
a cesspool of spams, scams and unwanted notices. The research I've seen
strongly suggests that it only takes a small amount of unwanted noise messages
for people to go longer between reading email and reduce the amount of email
they send.

Of course these are problems that can be solved, but my prediction is that
they will be solved by email's successors. Of course, technology predictions
are hardly worth the bandwidth that delivers them, but that doesn't stop
people from making them. ;-)

~~~
Sir_Cmpwn
>Regardless of whether it was a good idea or not, this ship sailed many years
ago. In the 21st century, email is rendered in HTML, and nobody but a few
engineers would even contemplate taking away the ability to render links,
formatting and images in email.

git is a tool for engineers. I could reiterate this comment for most of your
points. We're not here to debate plaintext email for end-users, we're
engineers talking about engineering tools.

~~~
Operyl
I have collaborators who are not necessarily engineers .. we’ve made it easy
for them to use git, we really don’t need to be outcasting then because they
aren’t an “engineer”.

~~~
Sir_Cmpwn
Then this workflow doesn't work for those groups. So what? It works for many
others.

------
cryptonector
My comment on this is
[https://news.ycombinator.com/item?id=17443263](https://news.ycombinator.com/item?id=17443263)

Tl;DR? My problem with code review over email is that keeping track of all
commentary and metadata is _difficult_ when reviewing over email. This seems
like a solvable problem with a text format for interleaving source and
commentary that is suitable for post-processing, particularly one that makes
merging commentary easy, and preferably too that makes changing commentary
metadata (e.g., comment accepted/deferred/rejected/addressed status) possible
while retaining the ability to merge commentary.

------
mankash666
Email workflows existed prior to the GitHub's of world. The latter won because
of genuinely superior workflow

------
hagreet
Welcome to 2018. This is so sad I am exploding out both ends.

------
Areading314
This might work for slow moving teams with a few contributors, but I can't
imagine using this in the way I've seen github used on projects with many
contributors... Email just doesn't scale as well

~~~
hyperpape
Since he mentions Linux as the primary example of a project that uses email, I
think this particular criticism needs some refinement.

~~~
Areading314
I'm sure plenty of Linux maintainers would much prefer using github if they
could...

~~~
Areading314
Here is an interesting post about the Linux project and why they don't use
GitHub

[https://blog.ffwll.ch/2017/08/github-why-cant-host-the-
kerne...](https://blog.ffwll.ch/2017/08/github-why-cant-host-the-kernel.html)

