
Prolific Engineers Take Small Bites - tbassetto
https://blog.gitprime.com/check-in-frequency-and-codebase-impact-the-surprising-correlation/
======
minimaxir
This post is a good example of my current issues with modern data journalism
as content marketing: you can say _anything you want_ without evidence to back
it up, but hey, there's a pretty chart, so it doesn't matter. (In this blog
post, the data points _don 't correspond to actual data!_)

This is also the primary reason I have switched to Jupyter/R Notebooks for my
blog posts: if I make ridiculous claims, people can check my work as evidence.
This post doesn't even provide any quantifiable metrics, just "we analyzed
millions of commits."

~~~
lamontcg
I just got done staring at some code written by a prolific upper quadrant
coder, that I'm going to have to noodle on and figure out how to disentangle
some bad design decisions. I might be able to do a fairly short commit to
refactor and extend, in which case my metrics will be crap. I might wind up
looking at all the outstanding issues with this codebase and rewrite every
class in this directory (in which case my LoC metrics will improve
dramatically). Right now I'm probably one of those "stuck" coders that need
"help". Kinda? But most software developers hit problems like this and run
away in terror, and looking around me IDK who can really help me that much,
I'll run my ideas by some colleagues, but the other ones who work on problems
this gnarly are also neck deep in their own issues...

~~~
hinkley
Mikado Method might offer you some clues, or at least confirm some of your
existing habits.

Basically, start a top-down refactor. When you find the bottom you stop,
revert your code, and refactor from the bottom up.

You probably won't find the actual bottom, but in my experience the moment
when I was about to give up and call it quits was usually just before I
reached the bottom. Taking a break, and starting over from the spot where I
stopped, usually the solution became obvious, and far less invasive than I
expected.

~~~
daveguy
Do I understand this correctly:

You start refactoring at a high level -- essentially set out to rewrite the
whole chunk of code in which your refactoring work lies. And focus on the most
important until you get to the inner-most structure.

Then when you are to the "sum(a,b)" level you start over with the original
code base and refactor bottom up. With the knowledge of how it works all the
way up to the top? Re-creating your original refactorings?

It seems interesting. Like tackling a math problem or puzzle from two
directions. Are there certain situations in which it is most effective?
Architectures? Paradigms?

~~~
hinkley
Most refactoring is from the bottom up. Make the change easy, then make the
easy change. I said "start a top down refactor" but redesigning from the top
or all at once isn't really refactoring. Some of the actions look similar but
you get few of the benefits of refactoring.

When the architecture problem has gotten away from you, or only becomes
apparent late in the project, it's hard to see the trees for the forest. The
only way to decompose the problem is to start somewhere and see what happens.
With a top down rewrite the number of bits in flux becomes overwhelming, which
is the anxiety I picked up from the previous commenter.

Mikado is just an exploratory development trick to help you find a way to make
the change with refactoring. You try rewriting pieces until the number of
concerns starts to multiply, you keep going essentially until your brain
starts telling you, "this is nuts, you should stop", and you take that detail
that broke the camel's back and do just that part, and build up and out from
there.

------
trevyn
Non-technical co-founder of Git analysis tool writes blog post that makes
assertions about all sorts of things and then justifies this with mockup
charts whose axes are unlabeled. Then offers tools to help micro-manage
engineers.

~~~
thebent
@trevyn Co-founder here.

You’re right: this post is a narrative about product development, and a strong
correlation we found between two variables across 20 Million+ commits that we
thought was fascinating and supports general 'kitchen logic' around best
practices. The axes are not labeled, but if you like we can set you up with a
demo account and walk through your data with you.

One other note: the typical common use case for the product has been
stakeholder management; something we have doubled down on in product
development. Any specific critique about how we can improve most welcome!

~~~
DenisM
> The axes are not labeled, but if you like we can set you up with a demo
> account and walk through your data with you.

Really? Rather than going back and labeling you axes you are converting this
into a sales call???

~~~
thebent
My actual suggestion was to hop on with the product team/co-founder. This
article is a narrative about something we found pretty interesting during
product development.

Similarly, the offer here is to take a deeper look at what we're building and
a (quite genuine) offer to incorporate any suggestions you might have into our
roadmap.

~~~
bojo
Congratulations, you have completely missed the point.

------
NumberSix
Neither in the article or in the referenced article/blog post linked to
"Impact" is impact defined.

[https://blog.gitprime.com/impact-a-better-way-to-measure-
cod...](https://blog.gitprime.com/impact-a-better-way-to-measure-codebase-
change)

    
    
      What I found is:
    

Impact takes the following into account:

    
    
        The amount of code in the change
        What percentage of the work is edits to old code
        The surface area of the change (think ‘number of edit locations’)
        The number of files affected
        The severity of changes when old code is modified
        How this change compares to others from the project history
    
    

No exact or detailed formula for "impact" is given. "takes the following into
account" is extremely vague as it could indicate any relationship. Is impact
higher with more files changed in the commit? Why would it be better if more
files are changed? Is it lower? Why would it be better if fewer files are
changed? Is it some totally unobvious non-linear function such as a
trigonometric sine?

Based on the vague description above, nothing in "impact" is directly related
to the actual end user/paying customer experience or a reasonable proxy such
as systematic end user testing by a QA team.

This lack of a direct relationship to the desired end result is the same
problem that lines of code (loc or LoC) and many other metrics of software
engineer output have.

The "impact" metric, whatever it precisely is, looks suspiciously like it
would naturally be positively correlated with a large number of commits/high
frequency of commits.

Also the plot is labeled with "volume" on the horizontal axis and not the
mysterious "impact" metric. The text implies this horizontal axis is the
"impact" metric. Why is the horizontal axis not labeled impact?

Even more peculiar, "impact" is claimed to measure cognitive load:

Impact attempts to answer the question: “Roughly how much cognitive load did
the engineer carry when implementing these changes?”

A good engineer will attempt to find a low or no cognitive load solution to a
problem! In general this will be faster and less error prone and cheaper!
Reinventing the wheel has a very high cognitive load.

~~~
debaserab2
Not to mention that's a pretty easy to game metric. By that rational, renaming
a variable name across a few files is going to make me quite impactful.

If my job were being judged by this metric, I'd sure be thinking a lot about
that.

This measurement also seems to favor overengineering, which is one of the
biggest things I've learned to stay away from. When I was early on in my
career, I thought I could implement anything myself, and so I often did. It
wasn't until years later that I understand the implications of maintaining
these implementations.

A true measure of impact to me would be a ratio of the most concise (but still
understandable) amount of code to the amount that it solves the requirement
it's built for. It's impossible to measure that without the context of the
problem you are trying to solve.

------
rsp1984
The title, well actually the entire article, is one of those things where A ->
B is stated, well knowing (and almost expecting) that readers will understand
it as B -> A or B <-> A. It's the developer version of the typical _" 10
Things Successful People Do_" clickbait.

Yes, a lot of (although far from all!) good coders have the habit of taking
small bites. That doesn't mean though that by taking smaller bites you are (or
are becoming) a good coder. I've had the past displeasure of dealing with code
that hit all the superficial checkmarks (small commits, unit-tested, peer-
reviewed, you name it...) yet completely stunk.

------
traverseda
Jesus that site was annoying. I go to read the article, and a popup beeps at
me distracting me. I think "what the hell is this crap" and move my mouse
towards the "about" page. A pop-up pops up (as they do) and distracts me from
even that.

Both times I tried to get information out of their site they found a way to
interrupt me.

Then I leave without reading the article.

~~~
sean_patel
Heard of Adblocker much? I didn't have your experience on that page :)

~~~
traverseda
I've got ublock installed under ff, and there weren't popups in the sense of a
new window, but in the sense of an in page element.

------
dasil003
I large agree with this, although with some provisos.

First of all, sometimes it's most useful to take a step back and think about
problems in a wider context. Often the biggest impact is from the code you
don't write. There's really no way those kinds of contributions come out in
any kind of metrics, but the team will sure remember them.

More commonly, I prefer to commit often to aid my dev process but then rebase
those commits into more cohesive units before merging. How chunky to make
those final commits is somewhat a matter of taste, and probably shaped by the
domain one is working in. Personally I like the finest grain possible without
any build breakage, that way git-bisect still works, but hopefully there's
still enough granularity to help with trickier merge/rebase scenarios around
dependency upgrades and/or db migrations. But pardoning the digression, my
point is the output seen by the rest of the team may not represent the
original frequency of commits.

~~~
jimaccount
I tend towards larger commits, large enough that the commit message can be
more "why" than "what". That way git blame works better.

(My issue tracker isn't coupled to my repo, and bisect is mostly useless for
me because my builds take so long. So like you said, depends on the codebase.)

------
xt00
Title of the article uses the word "Engineers" rather than software engineers,
and if you lump together virtually anybody who makes changes to a code base
(testers, sw architects, sw leads, etc), it seems like its easy to have the
data obviously skewed by people who essentially spam the commit process. I
think most people can look at a commit log and sort of eye-ball who are the
people who really make shit happen on a project--and who is probably causing
more grief than anybody else. I agree its hard to quantify, but to be honest I
didn't take away much from this article. I like the goal of the article, just
think it didn't really hit the mark.

------
tedmiston
Extrapolating a "prolific engineer" from one's commit log is no less absurd
than from lines of code.

Their methodology fails to account for so many things, commit squashing
feature branches, for example. A fundamentally flawed "study".

~~~
mateo411
The methodology also suggests a particular workflow is better. A workflow
where you commit and merge, instead of commit, rebase, then merge.

------
coldcode
What are they measuring? Public git repositories? The article doesn't say
anything about whatever data is being analyzed. If its public git that's
hardly comparable to a large corporate entity.

------
Dowwie
Analyzing churn is tricky and can be easily misinterpreted by management. This
reminds me of what was revealed from studies in behavioral sciences and
presented by Dan Ariely:
[https://www.youtube.com/watch?v=Q92BqouxyX4](https://www.youtube.com/watch?v=Q92BqouxyX4)

------
ninjakeyboard
Can you make any claims about the number of commits? That's another metric
that changes depending on tools and processes.

If you're using a code review tool well (if you are more experienced with it)
then you'll automatically be making lots of small commits vs people that may
not be well versed in working that way.

If you're working with large or prolific teams then you'll be committing all
the time with eg feature toggles to ensure everyone is as close to the current
code with their changes as possible.

So making general claims about more commits without taking into consideration
the ways that metric is bent and changes with tools and processes and
different experiences seems to be dangerous.

------
kristianc
Unless I'm missing something, how is 'Impact' derived here?

Impact as a metric is fine, but you have to show what it is and how you
arrived at that conclusion, and have verified that against a test set of data
otherwise it's turtles all the way down.

[https://en.m.wikipedia.org/wiki/Turtles_all_the_way_down](https://en.m.wikipedia.org/wiki/Turtles_all_the_way_down)

------
keeptrying
This article and company can be safely ignored.

------
ns8sl
We have a hard rule that you do not commit code that doesn't pass unit tests.

This leads to bad coders not being able to commit as frequently as good
coders.

However, this seems like a very, very loose heuristic.

------
ianstallings
Huh, strange. My made up charts say something completely different.

------
awinter-py
concepts missing from the article: 'staging' and 'commit size'.

People who have worked in code review shops tend to stage their commits, i.e.
do a bunch of work but commit it with git commit -p.

Article also doesn't look at deployment frequency, and 'merge' appears only
once. We don't know if these 'high impact' devs are in their own branch for 6
weeks cooking up the PR from hell. That's one way to have a high impact.

------
agentultra
I'm curious what the general vehemence is to quantifying and understanding
this information is? It's right there in your commit history. I've run my own
analyses out of curiosity in the past... statistics just isn't my favourite
subject. I think it's an interesting metric and I'd certainly like to know
more about how it's calculated.

(Although I'm sure that's part of the secret sauce).

~~~
overgard
I think people get uncomfortable because people will run with the data and
enact policies that are potentially detrimental, and then people will game it.
(You see this happen all the time with "velocity" at agile shops). Data itself
is great, it's just scary what happens with it when it falls into unwise
hands.

------
nano1237
We tried this. It gave us an unmaintainable code base that our platform could
no longer add features unless we threw so many people at it.

------
alphanumeric0
Well I guess I'd be labelled a poor programmer since I wait to commit until my
code has passed a few core specification tests.

------
nano1237
We tried this. Fast commits for small features gave us an unmaintainable code
base that eventually didn't work anymore.

------
bertr4nd
This reminds me that one of the things I loved about working on server-side
efficiency is that for any given diff, you can measure the performance
improvement, multiply by hardware costs, and say, "That diff saved $X." Makes
it really easy to measure impact and feel that my salary is justified.

~~~
scalesolved
That is one side to the coin, however if you made a change that was a
performance improvement yet the code was highly cryptic and hard for anyone
else to understand then this wouldn't be a good thing.

Essentially what I'm getting at is that peer review from team members is the
best metric for what is good and isn't.

~~~
bertr4nd
Absolutely true. Still, it helps when one side of the coin is precisely
quantifiable, since it helps frame the more subjective questions like code
complexity.

------
ticktockten
This is misleading, however I would like to point out that if Small Bites ==
Kaizen. Then, Prolific Engineers like any other prolific professionals may be
Kizening the crap out of their codebases. No data here either to back the
smart ones up though.

------
arnon
Unfortunately know some bottom-right quadrant people. We got rid of some of
those not too long ago - wouldn't want them on my team again. Super hard to
manage and work with.

------
z3t4
How do you finish a project ? One step at a time !

------
clifanatic
There's a great, fun, programmer-centric website called thedailywtf.com
(sorry, I just scheduled the rest of your afternoon for you). Developers can
submit WTF's that they've found in other's code - as the site admin says,
"curious perversions in information technology".

One thing that strikes me about the majority of the submissions, as funny as
they are, is that they mostly boil down to "so-and-so didn't know that such-
and-such feature existed, so wrote reams of code to implement that feature in
a complex way". It also strikes me that just this article's sort of analysis
of "prolific" (aka "good") engineers/programmers drives this same sort of
behavior. If every developer is supposed to be committing code all day, every
day, there's no time left over to read the product documentation, try out a
new feature, review a reference implementation, read a blog post: to be
"good", you must be spending as much time as possible _typing_, because that's
what you're paid to do. This (ubiquitous) management mentality is how we end
up with roll-your-own crypto, or five competing Javascript frameworks, parsing
using regular expressions... it's not so much that what they did was wrong -
and trust me, if it works, it won't be removed - it's that it's pointless.

~~~
darkerside
It can also work the opposite way. You can think you understand the
requirements and go hole yourself away for a couple of days pumping out code.
When you finally merge your work, another developer tells you, "Why didn't you
just use function X or package Y?".

If you're doing your work in small, sharp bursts, you actually end up with
more time in between work for talking with other devs, reading up on
technology, etc.

~~~
Bartweiss
I think this is the better insight.

A lot of DailyWTF posts are about people not knowing a function existed _and
then reimplementing it horribly_. Many of them include a note like "the other
10,000 lines were similar".

Ignorance is not embarrassing, it's the default state when all of our tools
grow and change daily. But a lot of the worst wtf moments are clearly cases
where something wasn't sanity checked by anyone until it was presented as a
finished product - sometimes too late to keep out of production. That's the
benefit of frequent commits, reviews, and research breaks; they might not
prevent our ignorance, but they keep it under control.

