The ideologies that developers publicly claim are often described as religions. One way they resemble religions is that people who espouse them don't live the way you would expect them to if they really believed in them.
XP (and by inference TDD) is often described this way.
But after one has practiced it for a while, it's hard to go back.
The way I describe it is this: When most people start as programmers, they write a big 1000 line file that should completely solve the problem. Then they run it, realize it was very wrong, and spend hours debugging it.
As programmers learn, they tend to practice "error driven development": they get an idea what they want to happen, they run the program (usually "hello world" to start), and slowly run, edit, run, edit until it evolves into what they want to see.
If you practice 2nd process above, you are one step away from TDD. You are still testing, you are just using manual tests. Once you learn to automate those tests easily, why wouldn't you TDD?
Is it at all possible that different people like different things, or maybe think differently, or perhaps find different workflows to be productive? If you could buy any of those as anything less than totally preposterous, then you may be able to make the next jump to considering why people like a development practice that differs from the one your prefer.
I like tests as more of a 'checkpoint' thing, myself. To each their own. Also, I spend most of my time in a REPL, which I suppose is pretty close to what people tend to get from a TDD kind of loop.
> Is it at all possible that different people like different things, or maybe think differently, or perhaps find different workflows to be productive?
Yes. But I don't think that's entirely down to innate or inborn characteristics, I think it mostly comes down to experiences. I tried to learn TDD from Beck's book and walked away thinking it was nonsense.
Then I went to work at Pivotal Labs and I got a chance to learn it with people whose job consists largely of teaching it.
Neither experience is about my largely inherited characteristics. They were experiences. Both were valid in themselves.
I have experienced TDD in the good and the bad. I prefer the good, but I know why most folks consider it to be bullshit: that's how they've experienced it.
My experience has been that TDD tends to do a poor job of testing complex systems with complex interactions. If any of the assumptions the tests are based on change (or are incorrect/brittle) then you will silently experience failures.
Integration testing is the more valuable side of testing imo but much harder and slower. Also not amiable to TDD in my experience.
None of those tests will silently fail. That’s their point for existing. They complain when the code stops operating as the developer designed it to work.
Integration testing is valuable. It just isn’t scalable to the whole code base (because, as you said, it’s slow), that’s why macro tests should be targeted to the parts that can’t be micro tested (10-15%), leaving the bulk of the code to be tested with blazing fast micro tests. Agile Thoughts podcast goes over this during a series on the Testing Pramid, episodes 1 through 8. https://agilenoir.biz/series/agile-thoughts/
It’s not that the tests fail, so much as they become misleading. They still pass, but they pass because their assumptions are the programs assumptions, but their assumptions are wrong.
I don't think of TDD as being confined to unit tests and I was taught to test from outside in, meaning I'll have acceptance and integration testing as a matter of course.
It’s best to have both micro tests (TDD) and macro tests (Acceptance Tests, which if you write acceptance tests first, is referred to as ATDD). TDD refers to micro tests. ATDD is what you should say to communicate clearly what your doing. Saying TDD when you mean ATDD will cause confusion with those who have mature test automation practices. Macro versus micro are important distinctions as they need to be managed differently. Mix those two test inventories together actually causes damage as all your test infrastructure will be slow rather than having a micro test suite that’s faster and run more frequently. Check out Agile Thoughts episodes 1 through 8 for background on this. https://agilenoir.biz/series/agile-thoughts/
> When most people start as programmers, they write a big 1000 line file that should completely solve the problem. Then they run it, realize it was very wrong, and spend hours debugging it.
No, I start line by line, and tend to be thorough. When I start out there is zero complexity, everything I add to it I don't add blindly, with a "rough idea" of what it does. I know exactly what it does. I know where user input and other code ends, and my code begins, at the least, and how I normalize things that cross the threshold.
Yes, I'm talking about a solo dev making little things for their own needs, sure. I have the luxury of being as thorough and slow as I want, too.
It's not lazyness, the idea of tests actually excited me when I first heard about it. But for example, when I write a particle system, how would I test that? Do I test all combinations of parameters and then compare them with images, which takes a million years? How do I generate the images, with another particle system? How do I test that, in turn? It really does strike me as a "now you have two problems" kind of deal in that situation. In others, the code is just too trivial and unchanging to write a test.
What's a "real" application that has good and complete tests? Something like the GIMP, or Blender, or an audio editor, or a complex game? I wouldn't know where to look, and the tests I saw when browsing random github repos often seemed too simple to be useful, often just covering some "token input".
But what if your "escape string" function has a bug that kicks in at exactly 15 characters? What if your "get user ID from username" has a bug that ONLY happens when the string "whoops" is in the username? Fuzz everything? Who really does that for applications without serious security implications?
> No, I start line by line, and tend to be thorough. When I start out there is zero complexity, everything I add to it I don't add blindly, with a "rough idea" of what it does. I know exactly what it does. I know where user input and other code ends, and my code begins, at the least, and how I normalize things that cross the threshold.
TDD isn't explosively incompatible with being careful.
> What's a "real" application that has good and complete tests?
Cloud Foundry has 4 or 5 million lines of Golang, about a million lines or Ruby and I think half a million lines of Java. Pretty much all test-driven. It's been a while since I checked the SLOC figures.
Kubernetes is of similar (now greater and definitely faster-growing) magnitude and the community has tooling to track coverage (I think the target is > 90%), though TDD is not the norm. I've personally worked on maybe two dozen systems where TDD was the ordinary practice, both in Cloud Foundry and for clients.
But it's a rare practice. Just on the numbers, almost nobody practices TDD. I'm aware that this argument is a kissing cousin to No True Scotsman, but there it is.
> But for example, when I write a particle system, how would I test that? Do I test all combinations of parameters and then compare them with images, which takes a million years?
> But what if your "escape string" function has a bug that kicks in at exactly 15 characters?
Equivalence classes: Analyze which groups of inputs all exercise the same codepath, and test each group once. Boundary value testing: Anywhere there's a boundary between two different types of behavior, test the boundary, one less than the boundary, and one more than the boundary.
I'd love to know how to test graphics and audio programs, frankly.
TDD works very well for things with a defined input and output, including:
1. compilers
2. databases
3. all web apps (output is the DOM)
In regards to the escape string, if there was a bug at exactly 15 characters, there must be a branch somewhere that isn't being hit, and it should be pretty easy using code coverage reports to find it, and generate the correct input sequence to exercise it.
When you get to fuzzing, you're hitting the edge of state-of-the-art. There is good research going on at automating some of that: http://lcamtuf.coredump.cx/afl/
What I do for graphics (never had to test audio programs) is to make the actual graphics code as simple as possible - no logic, very few lines: just take this thing and put it on screen. This way, I can inspect it visually, test it manually once and be reasonably certain that it will work. (Which can be incorrect, of course, but it's relatively rare. Lost a contract because of it once, so it can also be painful :D)
Put all the logic in a class that can be tested and that returns something that can be used in the "get this, put it on screen" part mentioned above.
> But for example, when I write a particle system, how would I test that? Do I test all combinations of parameters and then compare them with images, which takes a million years?
A couple of assumptions:
- your particle system is not one monolithic function, right ?
- you also didn't just write it in a single pass and it had all features.
- you start with a very simple (system) test that describes the minimal particle system. something like: given a particle system with particle count 1 and an emitter with probability 1 and velocity (1,0,0) and random seed 0; then after 1 step, there is a exactly one particle with velocity 1,0,0 at position 0.
- and you make this test compile and run and eventually pass.
- eventually you split out different functions; you develop also a test first. and then write the new function. and then change the original large function of the particle system to use the smaller functions. :)
- when it comes to rendering, you probably never want to write a system test that renders a full blown particle system with random paratemers ( because the test will likely be unstable unless you have a good idea about to keep it stable ).
- what is easier is to take the rendering functions and let them render aspects separately, i.e it renders a single particle at a known position. and you check the rendered image if it is correct; this because pretty simple with the image resolution is for example is just a pixel or 9 and that you can compare with a single static value. color animation can be tested that way as well. etc.
- thus in general even a particle system can be tested; break the system into small functions and test them separate and fully.
- then integration test the function in concert. use stable test scenarios.
- test manually; and when you find a bug (and you will). write first a test for the simple buggy function to expose the issue on unittest level. and then you decide if you want to add an extra scenario on the high levels.
Because the focus should be on correct software and there's a sunk cost fallacy in arguing that you're almost there anyways so why not just buy in to TDD for some purported value.
It might help some developers, but it is not a panacea and should not be promoted as such.
Sometimes I like to rely on gut-feelings while coding.
Ever filled in that user-signup form for the thousandth time?
Ever start with nice input: John Doe, 555 fake St
Then after the 500th time you are just inputing:
aaa
abc
wsdfg
And clicking "sign up"?
That is the point you should know in your heart: "I missed writing an automated test somewhere".
If you can't relate to the above experience (and have never had your code broken by a coworker), then maybe for some reason you are in a position where you're fine without TDD.
It’s a false equivalence to equate TDD to religions as the effects are observable and measurable with TDD. In contrast, none of my loved ones have contacted me from the afterlife nor has any diety ever responded to my exhortations.
