TDD/BDD doesn't fit the mold of startups. Here's why:
TDD/BDD assumes you know the problem and are coding to create a solution. In startups, however, you do not know the problem. Sure, you can imagine some kind of customer that might want some kind of code to do something or another. But that's all pipe dreams. You have no idea if method X is useful or not. Sure, you might know if method X meets the requirements of your imagination, or your architecture dreamed up in your imagination to deal with your imaginary customers with their imaginary problems, but you really don't know.
So it's a waste of time. Startups are all about providing value -- not flexibility, not bug-free code, not code you can hang on a wall and take pictures of. Value. If you want to be the guy who can make business solutions happen, the guy that customers can come to with any problem and you can make it all happen, you need to bone up on this stuff. But in the business world, you've already got the money, your job is to make the solution happen. In the startup world, you don't have the money yet. Big difference. Big, big difference.
Look at it this way: 19 out of 20 startups fail. That means that odds are that you will never see this code again. You'd be a fool to spend any more time on it than absolutely necessary. But the math works out completely differently in the commercial world, where most things you write stay around forever.
What I found over and over again with Agile is teams and individuals buying into the marketing/herd mentality of agile and forgetting about the adaptive/iterative nature. Everybody wants to either use a recipe book or just repeat their last project that they thought was really cool. "True" agile means ditching whatever isn't working. Pronto. There are no sacred cows. Everything is on the table.
TDD/BDD assumes you know the problem and are coding to create a solution. In startups, however, you do not know the problem.
This seems like two ideas conflating into one. The business 'problems' (1) and the problem you're solving in a particular piece of code (2). These aren't usually directly related. For example, if I wanted to build a YouTube killer, the TDD process isn't relevant to the high level "problems" I want to solve (which, as you say, one might not know of yet).
TDD comes to play (from a developer POV) when I actually start to write some code that, say, transcodes a video or provides an authentication system. In those areas, the problem is obvious and contained and TDD can work well.
That means that odds are that you will never see this code again. You'd be a fool to spend any more time on it than absolutely necessary.
Which does not necessarily mean TDD is "a waste of time". If practicing TDD in a particular situation will result in fewer hours spent developing a feature, the upfront "cost" of those hours is not a waste of time if you spend more hours debugging your way through building a non-tested equivalent.
A developer should have a feel for which way works for them. In my case, I know that the time I spend is ultimately lowered through using some TDD principles (though not all) vs tiresome debugging of untested code. The reward cycle of TDD is commonly overlooked. "Code->Yay!->Code->Yay!" beats "Code->Code->Code->Code->2 hours of debugging->FFFFUU!!" any day.
"True" agile means ditching whatever isn't working. Pronto. There are no sacred cows. Everything is on the table.
Agreed, but that runs counter to making absolute statements like "TDD doesn't fit the mold of startups" and that "it's a waste of time." It can fit and it can save time (just not always, sure, or if you're 'doing it wrong' for your situation).
Ditto. Couldn't agree more, in fact, I was going to write something very similar.
At high level also, Start-ups do need a TDD philosophy. For Start-up business ideas, the market is the test suite (to take the software analogy further) that is already there and changing the execution plan to make sure you pass those tests is vital.
I'm not sure I agree that the market makes a great test suite. It kinda hurts when you fail.
But yeah, there's other things you can use, like - A/B testing (for web apps). You can automate "code smell" detectors, and get a continuous graph on how good your code seems to be.
Of course, things like converters and math functions (anything non-GUI) can benefit from unit tests.
Any contained problem will benefit will from TDD given that the code will be revisited by others and yourself later in time. For a startup testing hypotheses, this is rarely the case and most code will end up in the junkyard.
Though usually the same thing in practice, a test first environment is not conceptually the same thing as Test driven development.
The reward cycle of TDD and the focus that TDD brings to writing code are the reasons I use TDD. The added confidence in correct code is also extremely helpful.
As I see it, if one were "in the zone" from 8-5 for a week, and had a clearly defined and understood spec, most software wouldn't take more than a few weeks to build.
Since being "in the zone" and understanding a well-defined spec aren't really that easy. Most time spent (in my experience) is understanding the spec and wrapping your head around the problem. This is a reason why pair programming can be so helpful.
Regression testing aside, writing tests is valuable to me mostly because it helps to define the problem for me enabling me to more quickly write code.
I've worked on an awful lot of teams and projects over the years. This discussion and the avoidance or embracing of the process flavor of the month, is not new. Agile has been around since people stopped drinking the RUP cool aid and came up with XP (a process that keeps meetings short by standing on your feet!). RUP lost its own gravity when Rational then IBM acquired the IP and turned it into another legendary white whale, from the origins of humble Objectory.
The one great thing I learned from RUP, a cocktail napkin makes a great realization document. Especially when originally served under a cool drink.
Communication and being clear on what the tree-swing looks like and needs to do is all that matters. Your companies will develop culture as they grow based on their pain points and their experiences.
I take the approach of "automate, automate, automate". I also enforce regular activities in intervals of by day of the week. I insist on profiling. I insist on peer code review; which can be done in the IDE or with a cocktail napkin, but I like my developers to be able to communicate what they've just written in case they get hit by a bus.
As a leader, not just the coder, you DO have to be concerned that you're writing something that can hold a the weight of a business on top of it. Would I worry about TDD or BDD before my Series A? No. But I would go back and reiterate and reinforce everything written prior, shortly after.
the TDD process isn't relevant to the high level "problems" I want to solve
In most businesses the opposite is true. Test Driven Development isn't really about testing low-level functions, it's about making sure the code and the business logic behave in the same way. TDD really comes into its own when you start using the tests as documentation on how the system is supposed to perform. A company using TDD can very quickly adapt to change because they can be confident in their code.
The reasons TDD isn't suited to startups is more that it's a heavy process which doesn't fit well with a small, tech-heavy group of people who aren't selling code.
One of the problems I face as coach is that people ask me simple, yes/no questions and I always answer "it depends". It's not that I'm trying to be evasive, it's just that life is full of little complexities. So I took a bit of artistic license and made a generalization. As you point out, we could speculate all day on various scenarios where TDD/BDD might be useful. That line of reasoning was too much for a short comment on HN, though.
One of the things I really don't think you get -- something I had an extremely difficult time getting -- is just how worthless your code is. Startups are not rewarded for writing good, maintainable code. Hell, they're not even rewarded for writing code at all. There's a lot of folks that say to advertise, create a vapor-ware product, and only once you know for sure people are willing to pay money for it should you start programming. Whether you like that idea or not, it shows just how far down the line programming is in the scheme of things for a startup. "Maintainable" or "Great" code is even further down.
Startups are about making things that people want that can scale out to huge numbers. Coding gives you that ability to scale out -- in certain instances and under certain situations. But you could just as easily write 10 lines of javascript and make a million dollars as you could write 100K lines of C. In fact, there's another huge school of thought that says the more you code the farther you've probably drifted in your mind from where the actual market might be. Each little test you create is further reinforcing in your mind -- that little code->yay cycle -- that what you are doing is cool. And it's probably not. "Yay" is people paying you money. Not having a test run.
And that gets to agile principles, which I think are very applicable. Do a little, test the market. Do a little more, test the market. I have been doing that for a few months, and I've found that most people who want stuff could care less whether it involves programming or not, or how well it's programmed. They just want stuff. If a guy is drowning, he's perfectly happy with you throwing him a life saver that falls apart and is crappy -- as long as it saves him from drowning. You might not even need a lifesaver -- you could do something else. In fact, the more you speculate on what the drowning guy might need, the more off-course you are. That means that any structure you put in your code early on is smoke and based on pipe dreams. The iterate-test cycle for me is what has verified beyond a doubt how low on the totem pole programming is. Find something people want. Period. More customers, less anything else.
Coming from many years of loving to learn how to be a better craftsman with my code, kinda sucks, huh? All those guys telling you what a great programmer is and how to be one, and then Average Joe Sixpack doesn't recognize how cool your TDD and programming language are. Dang customers. :)
Agile is meaningless at the point. It started to die the minute the first methodology guy with no actual coding experience thought it would be a good marketable product for his portfolio. It's been diluted and redefined so many times, and so many people are so quick to swear by their one true definition, that it's an impossible term to even discuss rationally anymore.
My feeling on automated testing is that it's far more valuable and saves more time than people who are inexperienced with it can understand, and yet it has shortcomings and limitations that go far beyond what any zealot is willing to admit.
I do encourage every developer to spend 3 months doing pure TDD, just to learn how to test. It's not that testing first inherently makes code better (it doesn't; test code is meta), but it's the fact that to realize the value of testing you have to be good at it, and to be good at it you have to practice. Once you are good at testing you realize that in many circumstances writing an automated test takes exactly the same amount of time as manual testing, plus then you have a regression test.
As far as test coverage is concerned, if you are coding a medium to large application in Rails, you should have base coverage on every controller, period. This is a good basic sanity check that is necessary for a language as dynamic as Ruby (if you're using Java it's an order of magnitude less important, if you're using Haskell, two orders of magnitude). The only excuse is if you're really sure that this code is going to either live on in exactly the same state or be erased. The fact that you're a startup doesn't mean that tests have less value. To the contrary, if you are iterating quickly, you need a safety net when you are refactoring. Once the application reaches a certain size, you will never have time to add test coverage, and there will be certain classes of changes that become impossible because they are just too risky.
"if you are coding a medium to large application in Rails, you should have base coverage on every controller, period"
I actually find this to be a huge time sink for most projects. Usually in a rails app for a client's application there's only a few fancy controller methods that do anything outside of typical REST scaffolding (there's no way I'm testing all of the typical admin CRUD). Add in something like inherited_resources using thin controllers and you're wasting even more time. IMO only write tests for functionality that's non-standard, which is probably going to be a very small percentage of your controller code.
For client work I find that most of the complicated or non-standard functionality happens on the client side, and rarely on the rails side. Which means a huge portion of my tests would just be fluff (like commenting method names and params manually) if I were to test all of my controllers.
Functional tests in Rails can be cheap integration tests. What happens when one of the models that is touched by one of this thin REST scaffolding controllers has an incompatible change introduced? The person making the change may no even know about the controller in question. Who's to blame when it blows up in production? Writing tests to cover every conceivable failure condition is indeed a time sink, but the first test is a very valuable sanity check that pays dividends over times.
Also, I have to question how much time you save by not writing tests for boilerplate. How much boilerplate is there really? Most business code isn't exactly novel or exciting, but neither is it scaffolding. Even if it is, you should just crank it out and be done with it right? It shouldn't be taking a large percentage of your time, and neither should the tests.
What I find is that I teach agile as being results and value driven first, everything else doesn't matter. All the stuff like TDD and BDD are just canned versions of patterns other teams have found useful -- each team can take them or leave them (assuming that they understand them first -- many teams are all too willing to ditch things that they have no idea what they are talking about. Helping teams make these decisions is a big part of coaching)
The fascinating thing is that once I teach that it's all flexible, guess what? People come in and make it all rigid and orthodox anyway. It's not Agile that's the problem, it's the people that, over and over again, just beat the living hell out of agile trying to make it into some rigid 1-2-3 formula for success. No matter how I beg and plead with them not to do this, still they persist. Strange.
I'm left with the conclusion that the basic problem in software development is people's fear of having a truly unpredictable, flexible, and open way of working -- even the agile adherents (especially the agile adherents, it seems). That's simply too much for some folks. They'd rather have a fixed list of things to do each day, even if they know the list is full of crap, they are taking ten times longer than they should, and they are doing busywork things that provide no value. As an example, I've had teams be very successful keeping their list of things to do on post-its on a wall. Then I leave for a year and come back and they've got some kind of spreadsheet monstrosity that has hundreds of items nobody can remember. Nobody likes the new system (except for perhaps the person who runs it, who will also admit that they don't like it, but only in private), it doesn't fit their needs, it's a huge time sink, yet they don't know how to make it stop. They are prisoners of their own need for fixed systems and over-engineering everything. This is because somehow they feel by over-engineering things that they make them "safer". Even though at some level they realize that this is not the case at all. Very strange.
Accidentally upvoted, meant to downvote (reading on an iPhone, go figure). Who the hell posts such a retarded comment? Truly evocative of the vicious anti-intellectualism of our age.
@pragdave recently gave a keynote speech at the Magic Ruby conference and made a big point of saying "Agile is not a noun." Agile isn't about subscribing to practices and then writing everything in stone. It's about doing that works for you situation.
So don't blame 'Agile' for being inflexible, blame the people who are using it for not actually being it.
The rub with agile is people conflate process with communication. Things agile (hopefully) expedite commuincation, however, are only as good as those who guide it. So perhaps, more precarious.
I've never quite understood the anti-TDD attitude around here, but an experienced developer and coach such as Mr. Markham being so confused about the scope to which TDD/BDD is applied is eye-opening.
The kinder gentlemen above have explained why the parent is fundamentally incorrect.
TDD/BDD assumes you know the problem and are coding to create a solution. In startups, however, you do not know the problem.
I think this is where these posts on HN always end up talking at cross-purposes. "Startup" can mean many different stages. I definitely agree with this. It's one of those things where when I think "startup" I think "they've got a problem, they're solving it" whereas some people read it at that very early stage instead. TDD doesn't suit that stage, because, like you said, you don't even have an idea of what success looks like.
I'm not sure what your suggestion here is instead of TDD. Just wing it?
Your whole premise is based on the the idea that your startup is going to fail. In that case why write any code at all?
If your startup doesn't fail then you've really made things extra hard on yourself. One of my favorite things about working on my startup is getting to learn and apply good practices.
It's just as easy to write good code as it is to write bad code. I find that TDD speeds me up not slows me down. It's not as much overhead as people seem to suggest.
You can write new code and then keep running the program to see if it works. Or you could write the code and keep running a test until it works. The test is typically faster for this because you don't have to deal with the rest of the program which is unrelated to what you're working on. Once you have working unit tests the implementation is typically trivial.
In practice, I don't write tests for everything but I code in a way that means I could test everything if I wanted to.
Note: This response is about software TDD and not some high level TDD
If you don't know the problem then you are just winging it.
Writing software is always with a goal (can it be without?) and thus you should be able to write a test (first) that will tell you if it does what you expect. If you expect it to record click stream, then it records click stream or whatever.
When I was new to TDD, I too found it to be very cumbersome. It took me some time to realize that I don't have to figure out the entire design before I write any test code. I can start with a test, then write the functionality that it is testing and when I realize that this is not what I want but something else then I change my test to test that new thing. Iterate. That is what agile is about.
Discovery that you want something else, happens whether or not you are doing TDD.
This whole thread should be required reading for startups, especially bootstrapped ones, but you made the key point in a sentence:
"19 out of 20 startups fail. That means that odds are that you will never see this code again. You'd be a fool to spend any more time on it than absolutely necessary."
TDD/BDD assumes you know the problem and are coding to create a solution.
Actually, no.. TDD is almost never used when you fully know the problem!
Look at this way.. would you trust that free source clone of facebook with your data knowing the site security holes i the first release simply because they did not test using something like TDD/BDD? I would not..
That being said TDD/BDD is not agile! Neither is waterfall!
WHAT?!
Agile is critical analysis of processes and tools used in the dev process to reach a set of dev and management objectives and goals by modifying those processes and tools.
TDD/BDD doesn't fit the mold of startups. Here's why:
TDD/BDD assumes you know the problem and are coding to create a solution. In startups, however, you do not know the problem. Sure, you can imagine some kind of customer that might want some kind of code to do something or another. But that's all pipe dreams. You have no idea if method X is useful or not. Sure, you might know if method X meets the requirements of your imagination, or your architecture dreamed up in your imagination to deal with your imaginary customers with their imaginary problems, but you really don't know.
So it's a waste of time. Startups are all about providing value -- not flexibility, not bug-free code, not code you can hang on a wall and take pictures of. Value. If you want to be the guy who can make business solutions happen, the guy that customers can come to with any problem and you can make it all happen, you need to bone up on this stuff. But in the business world, you've already got the money, your job is to make the solution happen. In the startup world, you don't have the money yet. Big difference. Big, big difference.
Look at it this way: 19 out of 20 startups fail. That means that odds are that you will never see this code again. You'd be a fool to spend any more time on it than absolutely necessary. But the math works out completely differently in the commercial world, where most things you write stay around forever.
What I found over and over again with Agile is teams and individuals buying into the marketing/herd mentality of agile and forgetting about the adaptive/iterative nature. Everybody wants to either use a recipe book or just repeat their last project that they thought was really cool. "True" agile means ditching whatever isn't working. Pronto. There are no sacred cows. Everything is on the table.