
CodeFlow: Improving the Code Review Process at Microsoft (2018) - rammy1234
https://queue.acm.org/detail.cfm?id=3292420
======
pianoben
(former Microsoft employee here)

CodeFlow's workflow was nice, but everything else about it was _awful_.
Capturing iterations, comments, threads, etc was the good stuff. The UI - eye-
bleeding and amateurish. Network operations - terrible.

Despite all that, the workflow was so good that I'd take it over github CRs in
a heartbeat.

~~~
ahartmetz
As a contributor to a F/OSS project with some serious infrastructure (KDE),
Github's uncontested success feels weird. It's a really anemic "solution" to
the collaboration problem, especially with projects using it as their only
collaboration platform. Most larger ones at least run mailing lists and a wiki
or so. Some have a more sophisticated bugtracker, but most seem to stick with
Github's... placeholder for one. Probably due to its integration with code
review.

~~~
DaiPlusPlus
Github’s success stems from its simplicity imo.

Pre-Github, projects would need to fire-up their own Bugzilla or Fogbugz
instances, set-up all the project tracking criteria (areas, iterations,
severity, etc).

But unstructured label tags seem to work surprisingly well in practice:
projects that would have otherwise demanded 20+ fields for work-item tracking
now seem fine with issue labels.

~~~
vbezhenar
If you need to put 20+ tags it's more work, because you have to figure out
what tag for what field, etc. Filling 20+ select boxes is easier.

~~~
CodeCube
Needing to put 20+ tags IMO is a hint that something about the process is
wrong ... I'd love to hear some ideas about what could _possibly_ be enabled
by having 20+ tags on a given work item. What workflow could possibly need
that level of detail?

~~~
vbezhenar
I agree, but that relates to the original complains about Bugzilla users who
required 20+ fields. I encountered 3-5 fields maximum and usually they were
optional and I would prefer form with 3-5 proper fields to arbitrary tag
string.

------
dqpb
Differential is a pretty great open source code review tool:
[https://www.phacility.com/phabricator/differential/](https://www.phacility.com/phabricator/differential/)

~~~
Game_Ender
That is sort of underselling it, Differential has the following features in
common with CodeFlow:

\- Easy to see the changes an author makes over time to a review

\- Integrates with the Package and Ownership system to automatically assign
reviewers

\- Easy work on changes that touch many files

\- Inline lint and code coverage results

\- Built in display of unit test and CI results and overall coverage %

\- Draft mode to delay review until tests & lint are clean

\- Comments move properly as you make changes

\- Easy to mark comments as done and find outstanding ones

\- Comment on multiple lines instead of a single one

Things that are missing:

\- No ability to comment on single characters

\- Cannot mark certain files as important

\- No way to explicitly mark a change as "simple" (like a rename)

\- No web UI to make quick fixes (compared to GitHub is what is most missed)

------
yodon
I'm sure I'm not the only person who wishes VS Code had capability like this
(which isn't to say everyone needs it or wants it, just that when it's needed
and wanted it sounds amazing)

~~~
humanrebar
You could have VS Code plug into a system like this, but there is
infrastructure that is needed to run CI, track statuses, manage comment data,
etc.

Theoretically if there were a standard for managing all that data within the
git source graph, you could implement this solely as a VS Code plugin.

------
mark_l_watson
Good article. I liked the analysis of how code reviews are performed at
Microsoft (yes, computer science is a science!).

My team is fairly happy with github enterprise and the PR workflow but some
things about CodeFlow sound better: being able to comment on a single
character in a line or share comments between disparate code; and, having a
local (non-web) client that runs faster.

------
m0zg
What y'all really want is Google Critique. You just don't know it yet, because
you haven't experienced it, as it's internal to Google. The closest public
product that I could find is [http://reviewable.io](http://reviewable.io) (no
affiliation) which, while it looks weird/amateurish, nails most of the points
which make Critique so amazing. There's also Gerritt, but it's too invasive in
terms of workflow, whereas Reviewable integrates with GitHub in a more
lightweight fashion.

~~~
piotrkaminski
Reviewable author here -- thanks for the compliment! Redoing the UI is on the
to-do list, once the migration from Angular to Vue is done.

~~~
hopler
"edoing the UI is on the to-do list, once the migration from Angular to Vue is
done."

Is what makes me uncomfortable about modern webdev. Valuable product work
constantly deprioritized by migrations to new frameworks. Magpieing has always
been a risk in hobby projects, but in the past there was a smaller torrent of
nee things to always be migrating to.

~~~
piotrkaminski
Eh, I don't think it's as bad as you make it sound. It's not "constantly",
it's the first major migration effort for Reviewable. And honestly, I probably
could've stayed with Angular 1.x for a few more years (it's still being
developed, after all), I just got tired of working around its performance
issues. There's also 4 years of learning how best to architect an application
like this that led to the creation of a Firebase-centric model layer where Vue
happened to be a good platform to build on. And even if we didn't switch
platforms, a thorough refactoring would've been necessary after many years of
accumulated technical debt.

------
alexeiz
Since this thing is coming from Microsoft, I really doubt the tool or the
whole process can work outside of Microsoft. It's likely very much tailored to
Microsoft needs and full of idiosyncrasies of the Microsoft development
workflow.

Here's an example of how things usually are at MS. I worked at MS awhile ago.
At that time Git was just being developed, and MS used it's own VCS based on
Perforce (which is pretty good, I should say for a centralized VCS). Code
reviews were done over email. Kind of like Linux kernel development. If you
wanted somebody to review your code, you used a tool to pack your changes into
a *.cmd file which contained a diff itself and a Perl (!) script to unpack and
view the diff. Then you sent this file to a reviewer over email, and he just
ran it on his machine to see the changes you had done. This tool was simple,
limited in features, and brilliant. It was hacked by somebody on the Windows
NT team back in the 90s. Even if you couldn't unpack a diff into your
repository checkout, you could still open the pack as a text file and review
the changes, or manually apply them, since it was just a freaking text file.

Come the time MS decided to migrate everything to .NET. Some team at MS
similar to TSE responsible for development tools decided that the old Perl
tool was inadequate. You don't need no stinking Perl scripts! They developed a
replacement in C#. Full of features you can't live without and maybe even an
integration with the dev environment (I don't remember). The new tool was
distributed in the binary form. The code changes for review were packed into a
binary format with plenty of metadata in addition to the actual diff. The
problem? Only this tool knew how to unpack this binary format. It worked most
of the time and when it worked it was great. But when it didn't work, you were
screwed. If it couldn't unpack your code change for any reason, it would throw
some C# exception. You couldn't fix the exception and rebuild the tool (you
didn't have access to the tool source code or knew how to build it). You
couldn't extract the change yourself, since it was packed in a binary format
only the tool could extract. You pretty much lost your change if you didn't
store it elsewhere.

And this is generally how things were (and still are?) done at MS. They have
teams of project managers analyzing and spec'ing out every bit of the
development process. Then they create extensive tools to support it with
elaborate UIs and automation of everything they can think of. But if you need
to do something just a little bit differently, you're left out in the cold
because those tools are too close and too brittle to adapt to your specific
needs.

~~~
WorldMaker
> (and still are?) done at MS

It sounds like the DevOps winds have shifted in the last few years at
Microsoft. Even Windows (!) is now in git and properly dogfooding Azure DevOps
(Boards, Repos, et al), according to blog posts and BUILD talks.

~~~
sterlind
Yup. I needed to search some Windows code so I simply went to DevOps and typed
my query. Results were lightning fast and it handled a complex query; I like
DevOps for the search if nothing else.

