
Agile Development: Where’s the Proof? - fogus
http://www.softwareresults.us/2010/06/agile-development-wheres-proof.html
======
caustic
The problem with software development is it is just a very broad field. When
someone talks about programming, he/she usually talk about some specific kind
of programming. You see, there are many programming worlds, like system
programming, middleware/framework programming, multimedia/game programming,
embedded devices programming, business applications development and so on.
These are very different and require different skill sets. One will have hard
time transitioning from writing dating sites in PHP to writing high-performant
web servers, from writing SQL queries to designing database engines.

I am convinced that Agile is implicitly related to business application
development, and when someone is saying he/she is using Agile methodology,
they most likely develop web applications for enterprises. And probably this
is the most widespread niche today. But frankly speaking, I don't think this
is not the most challenging work for programmers. Been there, done that.

But I am yet to see Agile evangelist who clearly states what kind of projects
Agile is appropriate for. They all speak about software development in general
without making distinction, as if Linux kernel or Java virtual machine or Ruby
on Rails framework should be developed with their methodology. Certainly you
can come up with many more examples of successfull hi-tech projects that
didn't use Agile. Ironically, these projects is what makes Agile possible at
all. Millions of man-haours were invested in development of operating systems,
databases, virtual machines, libraries, middleware frameworks, programming
languages, protocols and so on and on. Agile developers take all this as
granted and try to teach everyone else how to do software development.

Or thise evangelists simply have too narrow scope, then this ignorance
annoying. Does Agile work for software projects that require heavy
engineering? I don't think so.

~~~
philwelch
Your comment reminds me of this:
[http://ravimohan.blogspot.com/2007/04/learning-from-
sudoku-s...](http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-
solvers.html) [http://pindancing.blogspot.com/2009/09/sudoku-in-coders-
at-w...](http://pindancing.blogspot.com/2009/09/sudoku-in-coders-at-work.html)

It's a discussion of two people, Peter Norvig and a TDD enthusiast, both
building a Sudoku solver. Norvig takes a no-nonsense AI approach and the TDD
enthusiast flails for five blog posts without getting anywhere. To me, the
moral of the story isn't to mock TDD, but to point out that different problems
require different approaches. TDD can't be naively applied to an algorithmic
problem the way it can be applied to a nuts-on-bolts web app, for instance.

------
stcredzero
_I do agree (in part) with his answer to this question that I asked: “How
should the industry be approaching software development, in your opinion?”

He answered, “With humility and measurement.

"The Principle of Dispassionate Methodology: Don’t get your method advice from
a method enthusiast. The best advice comes from people who care more about
your problem than about their solution."_

This makes me want to read that other guy's blog! Sounds like an approach to
software methodology that would come out of LessWrong.

~~~
plinkplonk
"This makes me want to read that other guy's blog! Sounds like an approach to
software methodology that would come out of LessWrong."

IIRC Isaac (the guy who is replying to the Agile enthusiast on the thread) did
a superb job of demolishing the agile "evidence" presented by Craig Larman and
the article was first put up on Ron Jeffries's site for "fairness" but was
later pulled (I guess because it was causing serious credibility issues for
the methodology vendor folks).

As he says,

"Ron Jeffries challenged me to write specifically about Craig Larman's
"evidence", and for a while what I wrote and Craig Larman's response was shown
on www.xprogramming.com - but now it seems to be the only article going back
through 1998 that isn't available from the website archives ;-)

Doing that research taught me that I don't enjoy debunking - proximity to
snake oil leaves a bad taste."

"The Principle of Dispassionate Methodology: Don’t get your method advice from
a method enthusiast. The best advice comes from people who care more of
Wisdom.

~~~
alextp
Does he have a blog or any such thing? Is this text on archive.org?

~~~
nickelplate
[http://web.archive.org/web/20060720193601/www.xprogramming.c...](http://web.archive.org/web/20060720193601/www.xprogramming.com/xpmag/misstatingtheevidence.htm)

Edit: Damn, he indeed superbly demolishes Larman's "evidence".

------
mhd
I think the first comment says it all: _"Agile doesn’t exist, therefore, I am
“not a fan” and neither are you."_

Agile/XP is a hodgepodge of methodologies and approaches to software
development, plus the usual empty phrases any software engineering practice
seems to carry with them (at least it’s not on a level with the ol’ Unified
Process in this regard).

I could see some good scientific studies done on individual parts, e.g. TDD or
pair programming, but anything else would be a rather blurry definition. (I
remember some TDD studies been done, but if I remember correctly both the
approach and the conclusions were a bit weird).

~~~
jacabado
I guess the problem comes from the fact that you can't convince any
programmer(engineer?) with rational arguments so the evangelists and team
leaders need to resort to fairy tales.

It's an art in itself which can be mastered without any technical knowledge.
When this art comes to discussion it's easily taken down.

I'm still reading all the related posts here but the humility and measurement
could be my new passion. :D

~~~
mhd
You can't convince engineers with facts? That's a very, very sad assumption
and/or experience. Care to elaborate?

------
bguthrie
I am a developer who works on teams that use agile processes. I happen to
believe they work better, based on my subjective experiences with other
approaches, but I don't really have the kind of numbers one really needs to
back up each individual practice, which is something I struggle with.

I don't struggle much, though. The most important aspect of my definition of
agile is that it speeds up the feedback loop, which allows the person signing
the checks to gather whatever kinds of metrics they need to on a week to week
basis. Essentially, weekly iterations make quantitative analysis of the
software development process possible. Every other practice teams do or don't
adopt falls out from this newfound ability to gather and disseminate metrics
which are, in principle, of much higher quality. Do write tests, or don't [ed.
note: do]; but either way, track weekly, and let that guide your process.

------
j_baker
I think that the agilist response to the issue is that there _isn't_ any way
to measure agile's effectiveness. See Martin Fowler's article on the subject:
[http://www.martinfowler.com/bliki/CannotMeasureProductivity....](http://www.martinfowler.com/bliki/CannotMeasureProductivity.html)

Like it or not, programmer productivity is difficult if not impossible to
measure. I'm all for reasoned debate on the subject, but methodologies are
very subjective. Expecting proof out of an agilist is a bit like asking a
liberal to prove that liberalism is correct: you're never going to get it
because it can't be provided.

~~~
Tamerlin
All of the (few) successful projects I've been on have been ones where the
team whole-heartedly rejected agile practices.

On the other hand, every single "agile" project I've been on devolved rapidly
into a sweatshop environment with a hack-ridden nightmare of tightly coupled
bad code cobbled together with more tightly coupled bad code.

I've come to the conclusion that "agile" is what the talentless hackers hide
behind in order to avoid the scrutiny of being found out as incompetent. As
long as "heroes" garner rewards, those folks will be the norm rather than the
rule, and any process you use with a team like that will lead to failure.

Making things worse (for agile) is that the most vocal proponents for it are
typically the sort of people who would rather avoid the had part of actually
doing any sort of engineering, and would rather just jump in and start writing
code, even if that means writing code with an actual goal in mind. Managers
appear to love this, because they still think that lines of code =
productivity.

The result? Agile projects fail.

No process can save a project from its own team, though.

~~~
calibraxis
Which aspects of the Agile Manifesto led to problems? And were the teams using
sufficiently agile languages? (Say, Python or Lisp?)

Or was it more a matter of the teams saying "agile" and meaning "cowboy
coding"?

~~~
Tamerlin
Ruby.

The strongest agile proponents that I have met prefer Java. I think that's
probably enough to estimate their "caliber" right off the bat.

The strongest proponents of agile that I've met use agile as an excuse for
cowboy coding, though my friends and I generally refer to them as hackers,
because all they do is hack things together in a big hurry.

You work in "agile" environments like that for a few years, and you start to
equate "agile" with "sweatshop."

Since without exception every "agile" project I've worked on has been a
sweatshop with an agile label, you'd be hard pressed to convince me to use
agile methods for anything even vaguely related to developing software.

~~~
stcredzero
Whatever happened to the "40 Hour Week" pillar of XP?

~~~
Tamerlin
I don't think the agilistas get it -- there's too much hero worship in the
industry, and the newbie developers who haven't bothered to learn how to
design software that actually works play right into the hands of the legions
of incompetent managers out there who still believe that lines of code and
hours in seat are measures of productivity.

~~~
calibraxis
That makes sense. When I checked the Agile Manifesto, I expected to find
something about a 40 hour workweek. What I found instead was something
watered-down about "sustainability", which effectively means next to nothing.

"Self-organizing" also is near-meaningless; it maybe means that teams won't be
micromanaged (not that many managers want to deal with the most tedious issues
of a team anyway), but working hours are treated as a macro issue. ("I don't
know what you're doing, but you'd better do a lot of it!")

~~~
Tamerlin
I've had a number of projects in which the management complained that the
developers weren't writing any code, but they refused to take the time to
answer our questions about requirements. One actually said that if he was to
write down the project's requirements, he might as well just write the
software. (That project failed pretty impressively after I left it... I
refused to bail it out a 2nd time.)

------
agentultra
Evidence would be hard to obtain.

I find it highly improbable that any software development methodology can
provide deterministic, repeatable results.

The metric to measure isn't _productivity_ , rather one should be looking at
_happiness._ If it works for your team and they feel like they get a lot done
by using an agile approach then you're using the right methodology. Agile
isn't for everyone though and that's not a failing of the methodology.

------
spenrose
See also: "Evidence-based Software Engineering":
[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.113...](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.113.571&rep=rep1&type=pdf)

and the work of Greg Wilson: <http://www.slideshare.net/gvwilson/bits-of-
evidence-2338367>

------
spivey
Scott Ambler has conducted surveys that may interest you.

<http://www.ambysoft.com/surveys/>

