What is going on here? A lot of the comments on this thread seem to be confusing bad management with metric visibility. And it seems they would prefer to bury the metrics so they are not used to "snitch out" on people. Looks like a mix of dishonesty and toxic culture at work.
We basically built these metrics with Prometheus and grafana because our review queue kept growing. Now we keep everyone accountable, and make these metrics part of our stand-up meetings.
Our PR queue is now smaller, and review time is shorter. We use these metrics not to point fingers, but to ensure people are not overcommitting, overworked, and that people know how many things they have on their table.
I'm quite glad something like this is now available off the shelf
We use these metrics not to point fingers, but to ensure people are not overcommitting, overworked, and that people know how many things they have on their table.
Perhaps people have little faith in most management, and expect that tools like this will be used to point fingers.
You're not wrong about misuse of these tools being a symptom of bad management, but if 90% of the management in an industry is inclined to misuse such tools, it's reasonable to be dismayed when those tools are made more widely available.
You're right, it's reasonable to be dismayed, but imho not so reasonable to want to keep the metrics hidden.
Feels like we, as an industry, already had this conversation in infosec - abot security by obfuscation - which has some parallels here. The conclusion was that hiding things is generally a bad solution to the underlying problems.
This has nothing to do with security by obscurity. There aren't hundreds/thousands of anonymous managers probing into your teams metrics trying to find the single metric that will defeat your team.
Maybe fear for even more corporate bullshit interfering with the already difficult job a developer has to do?
> confusing bad management with metric visibility
Hmm not so confusing IMO. And what is your definition of 'bad management'? Most management I've worked with as a developer have no clue about coding and the complexity we're working in. These metrics could be valuable for someone who knows more about the complexity of the underlying technology and what is needed to actually make the PR happen.
I think peoples concerns are quite valid, but a lot of the criticism comes from poorly designed tools. In full disclosure, I've been researching developer insights for quite a while and I will be releasing something in the near future that focuses on "developer first metrics". The goal is to showcase how code metrics can be used by developers to improve their day to day routine. And by doing so, make software metrics, less of a taboo.
When I took a hiatus from my startup a year and a half ago, I learned a lot about code metrics and how it is misused and why there is such a desperation for it. There is a reason why, GitPrime was acquired for about 180 million USD. I don't think it (GitPrime) serves developers well, but it does highlight how desperately management wants to know what is going on.
Since the details on "GitHub Insights" is pretty much hidden (unless you are willing to pay for it), I can't speak much for how "developer centric" it is, but if the focus for "GitHub Insights" is to appeal to developers as well, then I think it will go a long way to make software metrics a good thing for the industry as a whole.
Right now I'm keeping things under wraps, but I've attached some screenshots of what kind of metrics you can surface that can help developers. Note, the screenshots are not the final product, as the product is still being heavily refined. And what makes my solution unique is, everything can be validated. That is, if you want to have a conversation around why a piece of code (or codes) had such a high churn, you can drill down to the source code level to see how the changes were derived.
As I mentioned early, I've been studying code metrics for a while and I strongly believe you will need developer buy in, if you want any chance of success with code metrics. So in my opinion, how positive/useful "GitHub Insights" is, will depend heavily on how developer friendly it is.
> and make these metrics part of our stand-up meetings
To each their own, if your team likes it, fine. If they just tolerate it because the decision came from above, yikes, what a nightmare, more management surveillance.
Data isn't insight. More metrics isn't more better. Most of these metrics are incommensurable across teams, people, projects...counterproductive to expose them. They aren't even robust to gaming.
For those worried about the data being used maliciously, it looks like most of the metrics they'll be introducing are focused around things that can actually help unblock teams and aren't focused on individual contributors. Eg. distribution of code reviews across team members for better load balancing, average code review turnaround times across the team.
It also looks like you'll be able to proactively set goals for some of these insights as well to improve over time. I can see where this can go awry, but optimistically this should help teams better measure and improve their process!
I'm really happy for your sake that you've never dealt with the worst case scenario of an organization utilizing this type of product.
I also feel great pity for you knowing that you have no idea how to deal with an organization utilizing this type of product.
I hope that the industry continues to fight against these bean counter products and produce higher quality developers in the future so that we can preserve the innocence of people like you.
The moment a product mentions "From developer to CEO" (ie. includes the word "CEO"), this is bad news. As soon as you introduce any metrics involving "participation awards" (whether based on lines of code, commits, reviews/pull-requests, etc.), the company is doomed to fail at understanding how productivity/worth is measured in the development world.
The fact they even mention "CEO" in their tagline is all one needs to know. There is not a single CEO in the world who should give a goddamn shit about actions taken at the source control or release level. It's all marketing drivel intended to promise management that the product being sold can somehow deliver insight into a world they will never understand. The fact is developers will just waste time inflating their metrics to satisfy productivity algorithms, while costing the business time–and therefore money–developers could have spent actually improving the product.
I'm inclining on agreeing. Gaming commit counts and commit graphs on GitHub is already a thing. I've been asked by many recruiters why my commit graph on GitHub has gone stale, when I've worked at places that don't use GitHub, or if my commits are 99% of the time towards private repositories.
I can already see the resumes of developers touting their statistical brilliance from metrics gained from this tool. It's not a world I want to be a part of, to be honest.
For the past 10 years I worked for companies with NDA and exclusivity clauses in the contracts.
I can only describe what I worked on superficially, I cannot contribute to other projects because all code I produce (even off-hours) that has my name attached to it is company property.
My GitHub and BitBucket profiles are bare, save for bug reports.
Every time I get interviewed I have to explain why my résumé is not more detailed and why I don't contribute to the open source community.
However, we have to try to come up with some. The idea that if you simply take away the metrics we will automatically improve the product is flawed.
Consider rewrites that burn a year of time for dubious value. Or fancy "refactors" that are really big heavy redesigns that lock you into abstractions that don't make sense in six months. Or devs that open 2000-line after 2000-line pull request...
At that point, code metrics aren't just about performance management, it's about me wanting to make sure I'm actually moving the code in the right direction.
Something I'm very interested in is if dev commit patterns line up with different types of work, and if some of those are "behavior smells" - e.g., not refactoring things one at a time, and getting your tests happy in between each move.
I'm not sure. If the majority of the metrics give a false or misleading sense of reality, then maybe taking away the metrics would be a net win. It's entirely plausible that the whole system of measurement we've come up with in this domain is fundamentally flawed and has more costs than benefits (see financial system risk ratings circa 2008). The skepticism of new metrics could very well be warranted until there's a real demonstration of success and good faith in their use.
If I were a CEO of a technical company and my company did a major change, say switching release strategies or adopted microservices vs monolith, these type of global metrics could indicate the benefits, or a problem. I agree that drilling down on individual people or trying to optimise blindly is the wrong move, but I think you're taking too much of a negative view on this. Shitty people will use more powerful tools for their shitty purposes, but that doesn't mean we should keep our tooling crappy.
As a profession, developers are likely more responsible for imposing mechanical, inhuman metrics and instrumentation on society than anyone since double-entry accounting was invented.
Everything from upvotes/likes/internet points to in-house fraud detection to click tracking to Captchas to facial recognition and tracking... Of course programmers are going to emit signals that those signing checks are going to take interest in.
I recommend you learn to live with it, everyone else has to.
CEO of Code Climate here. We have a product Velocity (https://codeclimate.com/) which offers what we call Engineering Intelligence. There's some great discussion about the value and appropriate use of data in software engineering, so I thought I'd chime in.
What we've seen is that engineers inherently want to be productive, and are happiest when they can work friction-free. Unfortunately, it can be quite difficult to get visibility into roadblocks that slow down developers (e.g. overly nitpicky code review, late changing product requirements, slow/flaky CI), especially for managers who are one or two levels removed from programming. These are situations where data-backed insights can be helpful for diagnosis.
After diagnosing issues, with data or simply qualitative insights from a retrospective or 1:1, we also see teams sometimes struggle to set goals and achieve desired improvements. A common theme is the recurring retrospective item that people agree is important but doesn't seem to be resolved. When it comes to implementing improvements, data can be useful to make objectives concrete and make progress visible to the entire team.
It’s important that metrics do not become the objectives themselves, but rather serve as a way to demonstrate the true outcome was achieved. Metrics also are not a strategy, and quantitative data cannot be used alone to understand performance of teams.
When quantitative data is used properly in combination with qualitative information, strong communication, and trust, we’ve found the results can go beyond what can be achieved without metrics.
This HBR article "Many Strategies Fail Because They’re Not Actually Strategies", while not entirely about metrics, has some great recommendations for how leaders can avoid these pitfalls:
Their top recommendations are: A) Communicate the logic behind what you are trying to achieve; B) Make strategy execution a two-way process, not top-down; C) Let selection happen organically, through systems that cause strong initiatives to rise up to to the top; D) Find ways to make change the default, to help move beyond the status quo and existing habits
Devs complain about exec picking irrelevant metrics to evaluate them (and they're right), but what I see even more often is teams full of inefficiencies that like being a black box and don't want to make an effort to introspect and self-improve. Often they're barely teams, they are just a collection of devs working on the same thing
I'm happy this tool exist and I'm sure my team will be better off because of it.
Is this what they did with the PullPanda acquisition? We had started using that a bit, but not being able to add new users as of late and the general pause in feature development seemed like something else was up.
They moved a bunch of it to Github out of PullReminders yesterday. I would go to your personal settings in GH and look at "Scheduled Reminders", which is where the old PullReminders functionality got moved. It's possible it's only there for pre-acquisition users, but that's where it should end up if it's generally available now.
I feel like I'm the guy in the comics who takes advantage because they tried to roll out something like this in my work and I figured out you could get more points by sinking other teams than actually writing code.
After voicing it in a meeting, it magically went away.
Ah I was hoping based on the headline that this would be some sort of analysis of the actual code -- aka a pie graph of the different language features each team member uses. Kinda like an aggregated `blame` :D
If they really had guts, they'd ask programmers to declare their background (years of experience, with that language, with static/dynamic typed languages, ...) and environment (private office / open floorplan / remote, music / silence / white noise, ...), and use this to settle once-and-for-all some basic questions the industry has had.
I am researching these kind of tools at the moment, and, although I'm not convinced myself yet, I am curious to learn. What kind of vanity metrics are gitprime and code climate plagued with?
We basically built these metrics with Prometheus and grafana because our review queue kept growing. Now we keep everyone accountable, and make these metrics part of our stand-up meetings.
Our PR queue is now smaller, and review time is shorter. We use these metrics not to point fingers, but to ensure people are not overcommitting, overworked, and that people know how many things they have on their table.
I'm quite glad something like this is now available off the shelf