Any chance you could expand on this, especially the part about "I've been guilty of this earlier in my career".
I am interested in TDD because so many people are such huge advocates of it (although I've seen many people being huge advocates of all sorts of things over the years), and I have an open mind, but this article does nothing to convince me, in fact is reinforces my skepticism.
"The feeling of productivity because you are writing lots of code."
Never mind if that code is useful, as long as it makes you "feel good".
I'm open to being convinced, but I honestly don't have to spend a lot of time fixing and chasing down bugs in my code, and when I do, I can't imagine the wildly obscure and diverse tests I'd have to have written to actually catch them.
I've asked this before...if anyone has a genuinely convincing article advocating TDD for the uninformed, I'd love to read it. The typical response to this is read x, y z book and it will make sense. My response is, I've been writing software for 15+ years, if you have something to sell me, do I have to buy your book to get the idea? I'm willing to meet you halfway, but TDD advocates don't seem to have the same mindset.
At this point I would certainly not consider myself a great expert on testing, but I think I've acquired a bit more intuition on what and how to test things. I don't usually work in a strictly "test first" manner because often times it's easier for me to "sketch" out a few classes and go through a couple attempts at organizing things before I find an approach that works for me. In these cases, testing first is a pain in the butt.
Other times, for example if I'm implementing some kind of tricky data filter, writing the tests first helps me think about the possible problems I might have to deal with in my implementation.
In general though, my projects usually end up with close to 100% code coverage, even if some of the tests are not very granular. Throughout the lifecycle of the code I maintain, I add tests when fixing bugs to avoid regressions. The tests that I write obviously don't catch every bug, but they do usually stop me from making embarrassing, silly mistakes which I think covers a pretty significant percentage of bugs in general. I don't usually have to worry about breaking things when refactoring or adding new features, which is a great feeling that I pretty much never felt in either my "no testing" or "way too much testing" phases.
So basically I have a very pragmatic attitude towards testing which I think would be non-threatening to a skeptic like yourself. :-) Part of the problem is that you might be letting yourself get turned off to valuable and useful techniques, not because of the techniques themselves, but the way in which they are being presented to you.
But at the end of the day, you have to find whatever works best for you to make the best code you can, and don't worry about what other people think.
I'm sure there are issues w/ the various studies it cites, but it is the only one to actually address TDD from a productivity standpoint with some research data that I've seen. (More / better links welcome too!)
I say this as a person who likes Norvig more than I like TDD (I'm interested in TDD to see if it is useful for my day to day work, Norvig is more of my high level guilty pleasure, such as his spell checker article or his speech on using lots of data and then algorithm becomes less important)
What I see this is as you plug straight into the positive reinforcement part of your brain where you reward yourself over time then have a big reward at the end. Thats the biggest advantage that TDD gives you because it makes coding addictive like games can become.
I'd argue that a test suite should be as small as possible to test the critical areas of the app, at least until the app ships, and then new tests should be added as the app takes a more definite shape, etc.
How much fun is it to refactor a test suite b/c the product changed substantially and you are now mired in the cognitive dissonance of a very un-agile (slow) change or in a bunch of broken tests.
If however your tests are breaking when you change unrelated things then you have brittle tests which are bad.
This early in the game a good testing strategy may be to write no tests. It all depends.
Tests are great when you have a clear spec and you just have to keep the app reliable as you make minor iterations. Or when you're doing something so complex that it's hard to keep in your head at once (this is, I think, the most fun TDD).
I think for most web apps a simple integration test that verifies that pages don't error and that basic forms work properly is going to be far more valuable than a bunch of code that tests very simple usages of underlying library code (which is sadly what many test suites I've read for web apps actually do).
My reward? When an experiment is a success and the new feature works. My second reward is when the feature ends up in production, which can only happen after tests are written.
To me, that's the coolest thing about programming. You create something and send it out into the world and it creates things too.
Ok, I think I like writing code, not tests.
(Oh, and btw, how do you test, (while true (write code)) ? )