Hacker News new | past | comments | ask | show | jobs | submit login
An unexpected benefit of unit tests (matthewc.dev)
189 points by matthewfcarlson on Jan 21, 2023 | hide | past | favorite | 228 comments



I like it.

I appreciate someone writing into the void how this helps them focus and be productive in their workflow. Who cares whether it 'fits the technical semantic definition of INSERT_ACRONYM_HERE'.

Unit tests are a literal investment - somewhat like buying insurance. Sometimes they're really great - almost necessary - and other times they're not worth the price. The article is offering a point of view of how tests are paying for themselves in an indirect way, which is useful to think about in the cost benefit equation.

The position of "insurance is useless" means maybe you just haven't been in a situation where it could really, really save your bacon. Likewise, if your rule of thumb is "buy all the insurance, all the time" - then BestBuy will gladly sell you a $5 'product protection plan' on your $10 purchase.


Yes, you're getting it exactly. This was for a personal project. The expected audience for it was my spouse and I and writing tests seemed like a nuisance. I don't even know why I started as getting the environment building in a testing harness was actually quite a bit of work. I noticed a benefit I didn't expect and wanted to share.

Did I not have a concrete idea of what TDD was when I wrote it? Yes. I had muddled test first development with test driven development as the two people I knew who were big on TDD were focused on TFD as they were refactoring large legacy codebases. I'm still murky on what TDD is (and it sounds like opinions differ) but I don't think I've discovered anything new. Just new to me.


Even if you don't want to read Beck's book on it, TDD is ludicrously simple and still somehow greatly misunderstood:

1. Write a test

2. See that it fails (if it passes, you're testing a feature you already implemented or you're testing something you don't mean to be testing)

3. Make it pass (by changing the code, not the test, seen that too many times)

4. Refactor

5. Repeat

The catchphrase is "red-green-refactor". I don't know why people try and make it more complicated than that.

Regarding your post, just wanted to say that if you took my other comments as criticism of you they were not. Your misunderstanding of TDD is actually very common and I was intending to address that, specifically. I actual consider the fact that you both rejected the flawed TFD-style (and mass creation of tests that comes with it) and rediscovered TDD (the above process, more or less) both amusing and a positive thing. It shows you're doing what many others aren't, actually thinking and experimenting to discover techniques that work (at least for you and on this project).


>I don't know why people try and make it more complicated than that.

Given that the author of the term wrote a whole book about it rather than just tweeting those steps, I think over-complicating may be part of its genetics. Others are just continuing the trend.

---

For the record: yeah, short-cycle red/green/refactor fits how I like to do TDD. I don't know that I'd claim that it is TDD though - by far the majority I've run across take it to extremes, though of course my coding-exposure circle is skewed and may not even remotely be representative.

In that way, TDD sits in a very similar brain-space for me as Scrum does. Useful foundations, utter nonsense if taken to extremes, unsure if a net-positive influence (globally, not personally) in the end. I have seen a looooot of negative-value tests, in ever-increasing quantity as time goes on, in the name of TDD.


As a partial defense for the length of the book, it was "Test-Driven Development by Example". Hard to fit examples into the page or two needed to explain the core concept itself.

Letting ideas like this move into the "similar brain-space ... as Scrum" is dangerous. You're actively choosing not to think at that point, you're choosing anti-thought over thought. Move too many things into that brain-space and you're going to become thoughtless. Don't let that happen, you have more to contribute to the world than becoming another mindless automaton.

Engage with the idea instead, understand why it does or does not make sense to you. Why it is or is not applicable to some area. Why other people have different ideas about it or disagree on its applicability. You'll be much better for it. And as much as I also dislike Scrum, it's worth engaging with its core ideas as well.


I think you may be reading too much into my Scrum brain-space. Or maybe projecting?

I've looked at those things. Found what I think are the core valuable bits, and how they apply to me and my situation. And I do not want to interact with them in a professional setting, because they are almost always distorted beyond any reasonable interpretation of their core, losing all meaning of the term and being used as an excuse to continue bad behaviors. They're quite similar in that respect.


Step 1 is wrong, it should read:

1. Write a feature

2. Test it

When your feature is your whole app "I want to build a TODO list app", your test is "I can write a TODO list app".

When you inevitably can't do that (because you cannot write a unit test for this), you break it down "I want to open a window/app", test that "can I launch the app window", and iterate.

"and still somehow greatly misunderstood"

It misunderstood because its presented as a methodology involving some testing suite or library, then you're constrained to testing things that may have little to no value, like testing that classes have data on them, or that you call functions and they return data.

And considering (most) engineer's first contact is something like a "write unit tests for the class you wrote", its understandable that its seen as a mixture of useless (often is) and arcane (often is) instead of an extension of a methodology for creating and validating features.


The book A philosophy of software design https://www.amazon.co.uk/Philosophy-Software-Design-John-Ous... Points out that TDD can inadvertently lead to tactical design rather than strategic design. In my experience, this can be true. I'd never noticed until it was pointed out.

Im on a team that is currently attempting shift left. One part of this has been an argument that testers should review unit tests. I think this is not a good idea. Unit tests tend not to focus on the behaviour of system as a whole.


It is said that Hemingway leveraged the same effect by leaving a sentence unfinished when stopping writing for the day. I use the same technique to remind myself I am not finished with something. This is also called the Zeigarnik effect: https://en.wikipedia.org/wiki/Zeigarnik_effect


I knew about the Hemingway method but didn’t know there was a psychological name for that effect.

It also goes well with the pomodoro technique, because that almost forces you to interrupt your progress while in the middle of something, leaving things unfinished, so they end up easier to pick up later.


Absolutely it maps one to one with Hemingway, idea. It also works very well for personal projects or projects that you rarely have time to dedicate but want to use any opportunity to make process you can


Came here to post this.

> The best way is always to stop when you are going good and when you know what will happen next. If you do that every day … you will never be stuck.


Says TDD is silly. Then literally describes text book TDD as his preferred approach?

Writing all tests up front has never been TDD.


I’ll admit, it was a fundamental flaw in my understanding. The comment section here has given me a few things to think about and I’ll likely add an editors note onto the original article.


I am not a TDD evangelist by any means, but if people would read a bit more detail about it, and use it as a set of guidelines we would be better off.

For instance, one of the things they say beyond the "red/green/refactor" is "as the tests get more specific, the code should get more generic." This is an interesting concept in itself.

Edit:

To clarify, if you are writing the least code to make it pass, and do not have the concept that the code needs to get more generic, you /will/ think the whole thing is silly.


Next up: Fighting Distraction with Pair Programming.

I'm a distracted developer, but you'll never guess the unexpected advantage I discovered of pair programming...


> While not traditional TDD or likely not a new concept, I’ve done something I’ve dubbed TLD (Test Led Development). Rather than writing a whole smattering of tests and then coding until they all pass, TLD focuses on ping-ponging between test and code.

TDD commonly gets mischaracterized as a two-step process of writing a suite of tests upfront, then writing some implementation to make them all pass. It's actually much closer to what is described as TLD in the post. You write a failing test, do the bare minimum to make it pass, refactor if appropriate and start the cycle again. It's a development process that will (in theory) produce a high quality implementation and suite of tests at the end.


I was going to say the same thing. His characterization of TDD as "writing a whole smattering of tests and then coding until they all pass" suggests he's gotten some bad information about TDD. Sadly common, though.

His TLD is pretty much TDD except he doesn't mention refactoring after passing the tests. Even leaving a failing test at the end of the day as a kind of "todo" for the next day, I'm pretty sure Beck mentioned using that idea in his TDD book.

EDIT: Found it finally. From Beck's Test-Driven Development by Example (page unknown, ebook copy, chapter 27):

> How do you leave a programming session when you're programming alone? Leave the last test broken.

> Richard Gabriel taught me the trick of finishing a writing session in midsentence. When you sit back down, you look at the half-sentence and you have to figure out what you were thinking when you wrote it. Once you have the thought thread back, you finish the sentence and continue. Without the urge to finish the sentence, you can spend many minutes first sniffing around for what to work on next, then trying to remember your mental state, then finally getting back to typing.

> I tried the analogous technique for my solo projects, and I really like the effect. Finish a solo session by writing a test case and running it to be sure it doesn't pass. When you come back to the code, you then have an obvious place to start. You have an obvious, concrete bookmark to help you remember what you were thinking; and making that test work should be quick work, so you'll quickly get your feet back on that victory road.

> I thought it would bother me to have a test broken overnight. It doesn't, I think because I know that the program isn't finished. A broken test doesn't make the program any less finished, it just makes the status of the program manifest. The ability to pick up a thread of development quickly after weeks of hiatus is worth that little twinge of walking away from a red bar.


Beck had this process (test, code, refactor) in place in his original eXtreme Programming book, which helped spawn the whole “agile” movement. It’s kind of cool to see how the process was refined over time.

It’s also sad to see how twisted people’s idea of both agile and tdd have become, usually because they never read the source material.


Having been involved in that world from pretty early on, I must admit I'm pretty appalled at the odd interpretations I get both on HN an in real life.

I'm not sure where to put the blame, but "Agile" does get a particularly bad rap these days. Mostly I'd say a corporate watering down of Agile concepts, money hungry consultants who didn't know much, and I h ave a personal dislike of how most of SCRUM tends to be implemented.

TDD in particular has fallen off the map in a way that I find very surprising. Generational amnesia I suppose.


I mean we should get used to this kind of stuff and just accept that Agile means "corporate agile" and just call classical agile that. Just like how OOP today means "like c++" and not whatever smalltalk is.


> corporate watering down of Agile concepts

I think inverting agile concepts is more accurate.


It's very common. When I was working at a USAF base they decided to adopt Lean. What they actually implemented was almost the exact opposite of Lean in every way, almost comically so.

A key element of Lean is empowering the workers to improve the processes. Let them come up with ideas that improve things and run experiments (guided by management perhaps, but not directed by). But the way USAF did it, the managers would watch a process being done, identify "wasted" movement, and then rewrite the process/procedures to eliminate that wasted movement. It was clearly just Scientific Management but being called Lean because they "leaned out" the processes. Naturally, the actual workers did not like coming in every other week and having to learn their job all over again. After a while they still held "Lean Events" but by then it was for show rather than to actually effect change in how things were done.


I once interviewed at a place that told me they did "design sprints", "coding sprints", "integration sprints", and "testing sprints"


Corporate agile looks from a distance like waterfall in a dress if you get my drift.


Once the scrum "masters" start becoming unspoken management, then you know the process has got off the rails and is no longer agile.

Sometimes it makes sense, corps don't actually have any need to churn out new features, at that point you just have kanban, or even waterfall. Be nice to stop pretending, though.


Agile is a lot like Communism: there's a lot of people who swear it works great but everywhere it seems to have been tried it hasn't gone well... And the apologists just insist "those places didn't understand it and do it right".


I worked at a big company which did a full agile transformation, including full-time, agile coaches. I imagine everyone hired afterwords hated agile for the very real swirl the process caused, but those of us who’d been there before know it was FAR better than that chaos.

I also worked at a funded start up that implemented agile from the beginning, worked pretty great.


I don't understand this recent pushback against agile development.

Do you really want to go back to the waterfall and V model style of development? Those are basically guaranteed failures in a fast moving industry like software development. If anything your comparison should be applied to the waterfall/V model because it is essentially central planning.

If you are doing things like building MVPs, iterated/incremental development with frequent changes and deployments to production then you are doing agile development.

Maybe it is not hyper formalized like Scrum but it is agile nevertheless.


I've seen agile working well at a fairly big company.


There's a bias there, I think there are a lot of small shops where agile works great and you mostly hear about big shops that use it where it will never work great because you probably don't care that much about SmallTechCo.

Versus communism where people (e.g. Benjamin Tucker) were predicting its primary failure mode decades before it ever was instantiated at any scale.


Small shops have a benefit of freedom of choice that large corps often don't give to workers. Sometimes it's accidental, sometimes deliberate, but large corporations have a tendency to want a single defined process. Variation (or as they'd call it "deviation") from that process is bad (it's not, but from the corporate management perspective it is). It has to be justified, you have to prove the thing you want to do will work before you can even try it to demonstrate that it does or does not work.

This hinders freedom to experiment with new techniques and methods within development teams, but it doesn't stop it. A "trick" is to provide all the artifacts that they want as if you followed their process to the letter, but still do things the way you want so long as it gets the job done. The problem with that is that you have no evidence you did things differently than the defined process and so they'll continue to believe the defined process is perfectly fine, if not excellent. Then some exec will decide to write a book about it and become a consultant selling the (broken) defined process (I assume this is how SAFe came to be, an ironically very rigid "agile" process).


50 dev teams, every team can choose how they work and their own tooling.

The result: if you're not using safe you're lead is going to be removed. Zero consistency between project tooling.

It's like the worst of both worlds.


Process is not the problem.

People are the problem.

What I mostly see it is people doing it to themselves. Someone thinks he has to be guardian of the process and refuses to let the experiment run.


Excellent points.


I don't know why you'd care about a brand new test being broken overnight. It's not like you're submitting the change yet anyway. It's just leaving a breadcrumb.


I think it is a good practice to push only working code, I think it stems from this. And it has a good point - you have finished something. Sometimes if you leave something “hanging” you keep thinking about it for a while, especially when falling asleep.


its a good idea to push code before signing off, what if you get sick and someone needs to pick it up, or your computer dies. A small broken test can be fixed likely faster than reinventing the whole thing again.


You commit and push your working code, then write a failing test and go home.


Yes, you are right about the technical aspect, but my comment was about psychology. Writing test and going to sleep is abit like asking a question to your friend and waiting for answer to the morning - it might get annoying.


Have you tried it? It's actually quite refreshing since you don't need to remember what you were doing in the back of your mind. The test is there for whenever you get back to it.


I like this a lot actually l


> (page unknown, ebook copy, chapter 27)

page 148 in the physical book, at least my copy.

However, you leave out the following section titled "Clean Check-in" in which Beck argues the exact opposite when working on a team. Clearly it's the days before git and trusting code to sit on your PC overnight without checking it in (I, too, would not trust Windows98 with code that long).

But this is the general problem with the book and TDD. It's outdated and overrated. The book is not a well-written book, even for year 2000 standards, and I'm quite surprised people are still referencing a 20 year old book that is almost entirely composed of trivial examples using Java classes and objects. It is my least favorite tech book on my shelf.

> leaving a failing test at the end of the day

As far as this point goes, you pretty much have to. The TDD methodology, per the book, is to get to green as fast as possible. If you are testing for a function to have inputs of 5 and 2 and expect an output of 10 then the book literally tells you to do:

  function() {
    return 5 * 2;
  }
What's going to happen is you write that code, get distracted or need to quit for the day, and you come back and your test is green. You forget that you wrote some total shit like the above and move on to the next Jira ticket.

As a methodology or system it's just bad. Imagine instead of writing the implementation to pass the tests that you instead are writing tests for some AI to "implement". You set the inputs and the expected outputs and the AI goes to work and does the implementation. In AI this would be called overfitting. Yes, the tests pass. But only for the cases you wrote. There is no guarantee your code works for the general case. Now replace AI with you and the same thing will happen. If all you care about is green tests you're liable to stop thinking of what the implementation should be doing.

Sure, you won't do that. But I promise you people, in general, do. One code base I worked on had minimum coverage required. More than half of the tests were total garbage. They either tested nothing useful, or were false positives that could never fail (because no one knows to check for failure, ever!). This is the opposite scenario, but the same outcome. It's the difference between the letter of the law and the spirit of the law. Give the people a rule to follow and they will mindlessly follow it.

Agile and TDD are both too nuanced and leave too much up for interpretation that it's no surprise we continually see debate on "true TDD" or "true Agile".

There are useful ideas in TDD. But this would be better packaged and sold as "here are some ways to build software under X scenario." These are techniques applicable to a time and place and not a paradigm.


Yes, I was actually considering writing another comment about the next section and how that's (somewhat) changed thanks to git and better distributed version control systems these days.

Regarding that code snippet:

If you forgot that you hadn't finished it, and then check it in, then that's on you. Good news, hopefully you and your office aren't morons and you aren't relying just on the tests from the TDD bits, because TDD itself doesn't directly address creating integration and end to end tests. So your integration tests will catch that. And if not, it'll make it to production and your customers will, rightly, call you a moron. And you'll be embarrassed, write a test to catch the error, fix it, and hopefully not fuck up like that again.

TDD doesn't aim or claim to cover all the testing needed to verify and validate a system. It's one part of the whole (if you use it at all). In fact, it doesn't address validation at all so that's something you have to cover another way entirely.

> If all you care about is green tests you're liable to stop thinking of what the implementation should be doing.

If all you care about are green tests, I'd say you're aren't just liable to stop thinking but that you have stopped thinking. You have become, in my more polite way of saying it these days, a fool. My advice: Don't be a fool, you have a brain, use it.


There is an awful lot of discussion out there about TDD that does not mention the Red-Green-Refactor concept at all. I think that really changes the impression of TDD.


It certainly did for me. A former teammate did a lunchtime session on TDD, which introduced me to the concept. I’m self-taught, and my foray into testing was very much a matter of trying to bolt tests on after the fact. So this concept of red-green-refactor was wild to me. A little intimidating at first, and I didn’t adopt it right away.

But when I did, it not only made testing better, it made my code better too. Not only because it’s more testable, but because it makes me think about the interface first, and the implementation truly as a black box as much as possible.


One of the most disappointing parts of moving away from .NET for work was the loss of NCrunch. That one tool made Red-Green-Refactor a breeze.

In Ruby/Java it is certainly a bit more of a chore to remember to do.


First time I've ever heard of that tool, it looks incredible. I wish Xcode had some of that functionality. It's close in some ways, all the building blocks are there.


Holy shit.

Link for the lazy: https://www.ncrunch.net/


I don't see benefits of R->G step


What do you mean? The Red-Green step is "test fails, test passes". You don't see the benefit of the test passing? Or you don't see the benefit of having known that the code previously didn't pass?

If you skip straight to green you don't know, for certain, that the test actually tested what you expected. This isn't even a TDD thing. When you're working on an existing, deployed, system and a user finds a problem, you generate a new test (well, sensible orgs and people will). That test will fail, because you haven't addressed the issue yet. That is, it's "red". Then you make it pass by fixing the system, it becomes "green". That's it. If you fix the system and then write the test, do you know that the test actually recreated the original failure? Or is it merely exercising the new or altered code?


When fixing bug it makes sense.

But when writing new code?


> [R->G] But when writing new code?

Absolutely, and most prominently when writing new code. The red-green transition is absolutely essential for new code.

You should not write any production code except to make a red test green.

Think of the tests as the specification of your system.

If all tests are green, your system meets the specification. Thus there is no need to write production code.

So in order to make the system do something new, you need to first change the specification. So you add a test. When you add this test, it will almost certainly fail. After all, you haven't written the code to implement the feature.

Then you make the test green, and now the system once again matches the (now updated) specification. Commit/Refactor/Commit.

Having the test that is red also validates your tests. If your tests are always green, how do you know that you're actually testing something?

In fact, it sometimes happens that you write a test that you think should be red, because you haven't implemented the feature yet, but then it starts of as green. Meaning you inadvertently already built the feature. This can be very confusing... :-)


Yes, new code too. If the test doesn't fail at first, you have a different problem than you thought.


There is a saying that coding is debugging a blank piece of paper. ("Piece of paper" tells you how old this saying is.)

"New code" just means "I want a program to do a thing, and it doesn't do it yet. That's a bug." The difference between "bug" and "new feature" is more a matter of perspective than actual development effort.


Bugs are features which were not expected.


If you write the code first and then the test, how do you know your test works? You've only ever run it against working code.

Like if there was a virulent disease for which there was a 100% cure, but you can't get the cure unless you test positive for the disease. I give you a test and say "Yep, test says you are healthy". Ok. What if the test always says people are healthy? "Have you ever tested an unhealthy person and the test detected that they were unhealthy?" "Oh, no, we've done this test a thousand times and it always says people are healthy!"

You write the test first, because your code does not yet have the feature that you are testing. Your current code is a perfect test for the test. Anyone who has done TDD for even a short amount of time has written a test that should have failed but instead it passed. Sometimes the was just a simple error in the test. You fix the test so it can detect what you are looking for (i.e. the test now fails). Other times a fundamental misconception was discovered that blows everyone's mind.


I'm replying to this one, but really, it applies to each of the replies to tester756. I want to say you all are insane, but I'm wondering if it's just that I do a different type of programming, or that the language I'm using doesn't lend itself well to TDD. Example, I have a function, in C [1], that runs another program and returns true or false. The prototype is:

    extern bool run_hook(char const *tag,char const **argv); // [2]
tag is informational; argv is an array where argv[0] is the program name, the rest are arguments. Okay, how would you write a test first for this? You literally can't, because the function has to exist to even link it to a test program. Please tell me, because otherwise, this has to be the most insane way to write software I've come across.

[1] LEGACY CODE!

[2] I go more into testing this function here: <https://boston.conman.org/2022/12/21.1>. The comments I've received about that have been interesting, including "that's not a unit test." WTF?


What's the problem here? You would write the test (if you're doing TDD, which you wouldn't always do anyways because you recognize it's a tool and not all tools are appropriate for all situations). The test would fail because it doesn't compile, so you make it compile. Not compiling counts as a failed test, even fools get that.

Unless of course you're only working no trivial programs (based on your write up, not the case) or an absolute genius you must have at some point or another encountered a failed compilation and used that as feedback to change the code. This is no different.


Then I'm not even a fool, because I didn't get that "not compiling counts as a failed test." It feels like the goal posts get shifted all the time.

Yes, I've gotten failed compilations, and every time it's because of a typo (wrong or missing character) that is trivial to fix, no test needed (unless you count the compiler as a "testing facility"). That is different from compiles that had warnings, which are a different thing in my mind (I still fix the majority of them [1]).

But I'm still interested in how you would write test cases for that function.

[1] Except for warnings like "ISO C forbids assignment between function pointer and `void *'", which is true for C, but POSIX allows that, and I'm on a POSIX system.


I just want to point about that sentence of "goal posts get shifted". In his book Test Driven Development by Example (2003), Kent Beck, on the preface, page X says: "Red-Write a little test that doesn't work, and perhaps doesn't even compile at first"

There is no goal post moving.

More likely, as we transmit information, we don't do it correctly, and knowledge/data gets lost. I found quite enlightening to always go back to the source.


I think you've mistaken me for a unit testing (your restricted definition) and TDD zealot. I wouldn't aim for 100% code coverage using the restricted definition of unit testing you've provided, or any other testing mechanism. The lines you're missing coverage for, in particular, are ones where syscalls fail. Those are hard to force in general, and I'm not going to be bothered to mock the whole OS and standard library. I do see a way to cause `open` to fail, though, you can change the permissions to /dev/null, but that doesn't get you your desired version of a unit test that doesn't touch the file system.

At some point, I, and probably most people, operate under the assumption that we don't need to test (ourselves) that syscalls will do what they say they will do. Until they actually fail to act correctly and then I'd investigate it, and write tests targeting it to try and reliably replicate the failure for others to address since I'm not a Linux kernel developer.


My post is a reaction to my previous job, where management cared more for tests than the actual product, and I was tasked with proving a negative. I was asking for a definition of "unit testing" and I always get different responses, so I've taken to asking using my own projects.

It may seem cynical, but I assumed that anyone into "testing" (TDD, unit testing, what have you) wouldn't bother with testing that function, or with limited testing of that function (as I wrote). You aren't the first to answer with "no way am I testing that function to that level," but no where have I gotten an answer to "well then, what level?"

This may seem like a tangent to TDD, but in every case, I try to see how I could apply, in this case, TDD, to code I write, and it never seems like it's a good match. What I'm doing isn't a unit test (so what's a unit? Isn't a single function in a single file enough of a unit?). I'm not doing TDD because I have to write code first (but then, the testing code fails to compile, so there's not artifact to test).

People are dogmatic about this stuff, but there's no discussion about the unspoken assumptions surrounding this stuff. Basically, the whole agile, extreme, test driving design seems to have fallen out of the enterprise area, where new development is rare and updating code bases that are 10, 20, 40 years old are the norm and management are treating engineers like assembly line workers, each one easily replaceable because of ISO 9000 documentation. And "agile consultants" are making bank telling management what they want to hear, engineers be damned because they don't pay the bills (they're a cost center anyway).


I had written more, but my cat decided pounding on the keyboard was a good idea and got lucky with F5.

Anyways, you never asked me "well then, what level"? and I thought I did answer it but here's an answer anyways (to your unasked question): I'd test it to the point that made sense. I wouldn't follow some poorly considered hard and fast rule (morons do that, we're not morons, we are humans with brains and a capacity to exercise judgement in complex situations). A hard and fast 70% code coverage rule is stupid, as is 100%, even a strict 1% rule is stupid (though for other reasons, like that it's trivially achieved with useless tests for almost every program). If I'm writing code and 90% of it is handling error codes from syscalls, then you'll likely end up getting 10% code coverage from tests (of various sorts, not just unit) out of me. I'm not going to mock all those syscalls to force my code to execute those paths, and I'm not going to work out some random incantation that somehow causes fork to fail for this one program or process but also doesn't hose my entire test rig. Especially not when the error handling is "exit with this error code or that error code". If it were more complex (cleanly disconnects from the database, closes out some files) then I'd find a way to exercise it, but not by mocking the whole OS. That's just a waste of time.

To reiterate my take: We have brains, we have the opportunity to use them. Use the appropriate techniques to the situation, and don't waste time doing things like mocking every Linux syscall just because your manager is a moron. Educate them, explain why it would be a waste of time and money and demonstrate other ways to get the desired results they want (in a situation like your example, inspecting the code since it's so short should be fine).


> The test would fail because it doesn't compile, so you make it compile.

But in this situation the test will fail regardless of what you wrote in the test code. So the supposed usefulness of the test failure showing that you are actually testing what you mean to be testing is inexistent and the exercise of making it fail before making it pass is pointless.


Then don't do it that way? I don't get why people are hung up on this. As I've said in other comments, you have a brain. Use it. Exercise judgement. Think.

If this is actually the one thing that trips you up on TDD, then don't do this one thing and try the rest. This is the easiest part of TDD to skip past without losing anything.


I like that write up on [2]. I have not really been exposed to C in a very long time, and that has been quite informative.

I also like that Set of Unit Testing Rules. That is basically correct, external systems are a no-no on unit testing.

Usually, you deal with mocks through indirections, default arguments, and other stuff like that so you can exclusively test the logic of the function, which is more difficult in C, from what I've seen on your write up, than in other languages. But if you care about not having that on your code for performance reasons, then more likely than not, you will not be able to unit test. And that is fine. You have an integration test (because you are using outside systems). You can still do integration test first, as long as they help you on capturing the logic and flow. The issue is that they tend to be far more involved, and far more brittle (as they depend on those outside systems).


I’d say be pragmatic rather than pedantic? And thirdly, “The pirates’ code is more what you’d call ‘guidelines’ than actual rules.”


It does make sense with new code, yes. Whether for all code or not, or whether unit tests or integration or end to end tests though is up to your judgement. No one but you, who knows your system and your capabilities and knowledge, can decide for you. Only fools expect a technique to replace thinking. I assume you're not a fool.

For new code, the reason it makes sense is that your system is bugged. It does not do what it's intended to do yet because you haven't written the code to do it (or altered the existing code to add the new capability). So the test detects the difference between the current state of the system and the desired state, and then you implement the code and now the test detects that you have achieved your desired state (at least as far as the test can detect, you could still have other issues).


The red step is how you test your test: if you don't get red at first, your test is wrong.

If I only see green for a given test, I have no way of knowing if it is asserting anything at all, much less if it's testing what I thought it was.


I’m not sure about you, but in my experience, trying to ship code while it still has failing tests tends to not get approved during code review.


It's a development process that will (in theory) produce a high quality implementation and suite of tests at the end.

In fact the theory is that this approach will produce a high quality design as an emergent property. This is an extraordinary theory that requires an extraordinary proof - one I haven't seen so far.


It's fairly straightforward: TDD is red/green/refactor, which means you're working in very small steps (about a minute or two each) and thinking about design during two out of three of those steps.

During "red", you're thinking about the design of your public interface.

During "green," you're focusing on implementation.

During "refactor," you're thinking about how to improve the quality of your implementation and how to improve the overall design, and making those changes.

If you believe that spending a lot of time thinking about and improving your design will produce a high-quality design, then TDD will produce a high-quality design. QED.

If you don't accept the axiom, then it's a longer discussion, but that's the proof, and my experience is that it does in fact work.

(If you're looking for a rigorous study and proof, you won't find it, because there are no rigorous studies that formally prove what creates high-quality design. Partially because there is no formal definition of "high-quality design" in the first place.)


TDD is obviously a hill climbing strategy as far as the end result is concerned and the outcome has the associated baggage.

"Refactor" is named this way to emphasize that changes you are making are closely related to the tests you already have and the tests you are about to introduce.

That does not leave enough space to justify a QED. If you choose to design beyond that, the process stops being TDD - at least as described by Kent Beck


TDD has several big issues that lead to bad design:

1. It assumes your spec is good and rigid. In reality, most specs are shitty and fluid. And your first understanding of spec is wrong.

2. It assumes your first implementation of the spec is good enough to justify automated testing.

3. It leads to high test coverage which inhibits refactoring (despite zealots telling you otherwise).

4. Almost always it leads to obsession with testing, which leads to a ton of unnecessary complexity (e.g. dependency injection for the sake of testing, weird practices like "don't mock what you don't own", etc)

I always thought it's great that it forces you to think about public interface, but I came to believe that thinking cannot be forced with a ritual.


> 3. It leads to high test coverage which inhibits refactoring (despite zealots telling you otherwise).

The major flaw with TDD is if you get your test wrong, you get the wrong code. The intent is to inhibit refactoring, because the assumption is the tests are correct, so any refactoring must be done within the constrains imposed by those tests.

OFC the tests and supporting design are usually just as flawed as the code. This is why I say the first step of TDD is wrong, write your feature first, not your test. If you don't have a feature, and can't logically reconcile it with your other features, then its not worth even writing the test in the first place.

Tests are just a supporting tool once you (believe) you have the feature written, which functions on one hand to protect the other features you have written (at least as well as they are tested), and to validate that you aren't wildly breaking the system expectations. A large number of tests is a measurement that something is wrong, but it doesn't tell you if the feature itself is wrong or your design is wrong, just that one of the two is true.

That's how it helps you refactor, a "good" design will add new features and few tests will break, as more tests are added and total test failures approach zero over time, you gain some confidence that the system is good. You never gain certainty, just the knowledge that your constrained refactor probably didn't break anything.


Two-step TDD can work if you implement a well-known, stable spec, say, an IP stack.

Such cases are relatively few.


There's no such thing as "two-step TDD." That's a cargo cult.

Real TDD is red/green/refactor/repeat in very small steps, writing a handful of lines of code at each step.


I see; I should have quoted it. I was referring to the expression used in the grandparent post.


Right, and in the absence of a well-known stable spec, then it seems to me that TDD (and related) is just a variant on the very old and very discredited "Waterfall Model" [0]

[0] https://en.wikipedia.org/wiki/Waterfall_model


Hardly, if it was a variant of Waterfall you wouldn't have testing mixed in with the development of new code. Waterfall has explicit barriers between the development and test phases which is why those systems usually turn out to be clusterfucks (unless they're small or you have, usually by luck, a correct specification). In real-world Waterfall projects (which I have suffered through, worst was a multi-billion dollar disaster for tax payers, and a multi-billion dollar success for the contractors that we took it over from), testing happens after development which means you have no useful feedback while developing that you are building the wrong thing or building it incorrectly.


I had in mind that Waterfall separates architecture design from detailed design from implementation, and in the above case, the spec being precise and finished certainly means that the top level design (or architecture or whatever) is 100% finished before the TDD development begins.

Hardly a reason to downvote me.


But TDD does not separate architecture design from detailed design from development from testing like Waterfall. It integrates them, or at least enables integrating them. It is literally not Waterfall, which is an idealized (and thus its primary flaw) process predicated on that strong separation that TDD deliberately breaks. TDD is predicated, instead, on the idea that we don't know everything up front. Otherwise we wouldn't need or want to write tests in the middle of development.

As to the downvote, I guess you thought it was me, it was not. But I don't plan to upvote you either.


Fair enough.

For the moment I will assume that I have a misconception that led me astray, and that I should correct in the future once I double check the thinking and the definitions etc., and I'll upvote you now for leading me in a better direction.


> For the moment I will assume that I have a misconception that led me astray

If I understood your posts right, it's the same misconception a few others have had in here:

In TDD, it's not write all tests, then red->green->refactor. It's write one test, red->green->refactor, then write one more test, red->green-refactor, repeat until done.


If it was a variant of the waterfall model then it would just put the testing phase up front and once you developed all your unit tests you start making them pass.


Waterfall TDD would be writing all the tests for the entire project, seeing them all fail, and only then start writing any code for the project.


Also, as with almost everything else in life, it's important to not be so dogmatic about the "rules". Everything is negotiable - I'd never recommend literally only ever follow the strict red-green-refactor workflow.


I recommend people learn the “proper” technique, even if they don’t apply it in every context. As they say, _you need to be in the mould to break the mould_.


TDD properly done is a 4 step iterative process. 1) Write test. 2) Run test and see that it fails. If it doesn't fail figure out why and fix it so it does. Either the test is wrong, or what you're testing is trivial. 3) Write code so test passes. 4) Refactor code so it's good. Next go back to step 1.

Step 4 is the trickiest and most important part. Refactoring transformations must be such that they do not invalidate the results of step 3. But if you don't execute step 4 properly you'll end up with crap code.


1. 90% of the time what happens during step 4 is realization that your tests are crap. So you have to throw them away and start from scratch. Tests are not helping with refactoring, they are inhibiting it.

2. Step 4 is indeed the most important part, and yet TDD priests and scriptures don't cover it at all. TDD is actually distracting you from what's important, because it focuses on steps 1,2,3. Eventually you'll become disciplined enough to not get distracted, and you'll think that TDD works. But the reality is that you never needed TDD in the first place.


I’ve used TDD as a kind of proxy for predicate transformer semantics. Test first is a way of sneaking specification first into a team’s workflow. But let’s be clear, overspecification is bad specification! And that’s why as you say a lot of times the tests get in the way: they are overly specific and thus inhibit desirable refactoring. In my experience it’s better to err on the side of underspecifying. Most everyone agrees with me on that, since the total lack of specification you generally see is a species of extreme underspecification. Don’t think I’m being snarky either, even that level of underspecification can be appropriate in the exploratory phase, although I think it’s generally worth the effort to start from a specification that’s at least slightly more restrictive than the always true predicate.


Yeah, that's how I think about testing in general. Tests should be a reflection of your specs. But if your specs are bad (most of them are), you should wait for them to improve. Not carve them in stone.


Even after all these years, people still have this misconception about TTD. No idea where it came from.


Microsoft. It came from Microsoft.

https://www.jamesshore.com/v2/blog/2005/microsoft-gets-tdd-c...

(Well, they popularized it.)


Dogmatic evangalizers


Sadly, this was my impression of TDD for years. I always thought that writing every single test case up front would never make any sense. It was only when I read the original definition of TDD from Kent Beck that it clicked. I use it quite often now and it makes development a lot easier.


It always amazes me when someone has the audacity and sheer lack of curiosity to decide that something they've read about means something else, and then "improves it" and "I've done something I've dubbed" to what the real thing is in the first place.

TDD has a wikipedia page [1] FFS, which is the #1 hit entering TDD into google, and which very clearly lays out that TDD is a test/code cycle (red, green, refactor anyone?). What the author claims is TDD is called TFD (Test First Development).

How do people develop this kind of hubris?

[1] https://en.wikipedia.org/wiki/Test-driven_development


I'm not sure if I go even further or not, but I'm perfectly happy to write code first and then write tests. And I will sometimes not write tests until I start debugging code and start thinking to myself "is it a buggy bit of this function I haven't tested yet?" and so I'll write the test to prove it isn't that bit.

With code I'm getting paid for I'll write more tests up front and won't skip that step, but for code I'm playing with then as long as I'm just enjoying myself writing code I'll write zero tests for awhile and just code. Then as I hit the debugging/refactoring step I'll do the backfilling as a form of debugging. Often I'll write the code that fixes the bug, then write the test, then quickly and temporarily revert just the code to ensure that the tests fail and then proceed. That gets you the same safety check as doing it the test-driven way to validate your test actually tested the right thing.

I really tend to hate "thou shalt start by writing tests" as some kind of immutable golden rule. It does always feel good to me when I just naturally wind up doing it, but forcing myself to do it every single time just isn't any fun at all.

At the same time tests are absolutely essential when it comes to refactoring and debugging. When you get too far out ahead of yourself with code then you start needing to shore up the foundations and use tests to eliminate bugs in the code that you've already written. Some code though is obviously correct enough that in personal hobby projects I won't ever code tests for them (unless I do hit the point where I start to doubt their correctness due to some funny bug at which point the situation has changed so I add some tests to prove it one way or the other).

The whole point of this though is that the tests are always serving me and they aren't in the drivers seat quite the way that all the TDD 101 blog posts like to ram down your throat and which I suspect turns people off from that approach so much.

The end result is also that you'll tend to wind up with the tests that are actually useful, covering the code that is particularly hairy or essential and the edge conditions that you really need to make sure to get correct, and you wind up having a test suite which is composed mostly of useful tests instead of all the largely useless ones that infect codebases.

I'll also happily omit tests on lower level functions that are well tested at the level above them, because I don't need to test the same thing at 18 different levels (again, for professional use I'm more likely to include tests at every level if they're fairly mechanical to produce). I also have a flexible definition of what the system under test is, which often encompasses more than just the immediate object that I'm testing and I don't bother wasting mental effort thinking about how to mock the whole world.

I don't know what kind of TXX that is. I still wind up with tests, they're legitimately essential to have, I just don't get there via some prescriptive route. I wind up with good code coverage, but I don't necessarily wind up with it looking as comprehensive as rotely banging out lots of unit test. I typically wind up with tests that I know are useful because they were produced by hitting actual bugs or where I had real questions about the behavior of the code and needed to assert some invariants and prove the code worked.


An important element of TDD is that it wasn't supposed to be prescriptive. It's a tool/technique to use. Sensible people know when to use it and when not to. Even Beck (who many here and elsewhere assume is dogmatic about TDD) has said on many occasions he doesn't use it 100% of the time and wouldn't use it 100% of the time.


Does test driven mean you need to write the test first? Seems pretty clear to me. I dont think testing needs to be so dogmatic but TDD kinda is.


That's not dogma, that's the technique. If you aren't writing the test first then you aren't doing TDD, and that's fine. There's nothing wrong with not doing TDD at all if that's what you choose.

The dogma most people see or claim to see is that TDD is meant to be used everywhere (or nearly). Which some fools, yes, believe. But they're just that, fools. People who use their brains (aka, non-fools) know them to be fools and do what works for them and the circumstances because they spend some time thinking about things instead of parroting a dogma (or an anti-dogma).


> The dogma most people see or claim to see is that TDD is meant to be used everywhere

And yet TDD preachers are never drawing the boundaries for TDD applicability. They are always extremely vague: "sometimes I see that TDD doesn't work for the problem that I'm solving and I don't use it". Well, how do you see it? What types of problems is it bad for?

If TDD works, they take credit. If it doesn't work, "it's just a tool" or "you used it wrong".

It is literally impossible to prove that TDD doesn't work. Which makes it a religion.


The "fools" are writing most of the blog posts on TDD.

At some point the No True Scotsman fallacy kicks in pretty hard and that is just what TDD actually is.


Dogma tends to be a sign of an intermediate experienced developer.


If you're doing TDD then yes, you're writing the tests first. But even Kent Beck says you don't need to do TDD all the time. So where's the dogmatism?


Unfortunately Uncle Bob used to preach 100% usage and IIRC Beck never really specified when it should be used.


I mean, how can he? You're the one that knows your system so you have to be the judge on whether TDD is appropriate or not for it, or for parts of it. He says as much in his talks on TDD and in his book. You have a brain, use it, don't expect him to think for you on systems he has no knowledge of.


Pretty much the same for me: I'm only able to write any sort of decent tests if there's some initial version of the code already there. Typically that means a prototype-ish only-works-on-the-happy-path version I wrote in an exploratory way, then that first test can be for the happy path. After that it's pretty easy to copy the test, tweak parameters, and figure out where the first version breaks TDD-style.


> I really tend to hate "thou shalt start by writing tests" as some kind of immutable golden rule

The only study I've seen on this shows only tests were correlated with more correct programs, but doing tests first or last showed no significant difference.


What I find so interesting is that we all effectively do TDD, except without commitment to it - Imagine writing some code and not at least testing it out with several test cases.

The absolute bare minimum should be for you to write those informal tests down and commit them to the repo. That's some golden knowledge I've gotten over time

It just requires the overhead of setting up the test runner


But the orthodoxy seems to be write the tests first, see them fail, then write code, not write a little bit of code, write some test cases, repeat. How "test first, then code" works in a compiled language like C++ or Rust be beyond me (unless I'm taking this to a literal conclusion).


> How "test first, then code" works in a compiled language like C++ or Rust be beyond me

It's not really any different than in dynamic/interpreted/weakly-typed languages. "Writing the test" for a function sometimes just includes writing a method/function with the appropriate type signature that does nothing (maybe returns a dummy value).

Forcing you to view the code you're implementing from the viewpoint of someone calling that code from the very beginning is one of the advantages/goals of TDD. If you find that it's difficult to set up the objects/data you need to write a test, eg, your code has a bunch of implicit dependencies on other components being in a particular state or it takes a ton of arguments that all have to be constructed, that's usually a strong indicator that you should rethink the design. You're getting early feedback on your API design before you waste time implementing it.


Combine this with usage testing. At least for libraries, which I seem to end up writing a lot of, writing usage tests that don’t necessarily test units but functionality. Doing this early on helps flesh out the friction in usage and will help with testing as it often can have broad coverage. It also, now sits as a potential example for usage in documentation.


These are way, way more useful than unit tests IME.


Yep, agreed. I tend to just code a single giant integration test where I map out the happy path to start with, and then when it works code tests for known edge cases at specific parts.

Keeps me focused.


TDD may not be a two-step process, but its first step is always "write tests" hence the term test-driven, which author clearly diverged from.


TDD's first step is write *a test*, not "write tests". Batch size of one. If it was "write tests," it'd be more waterfall.


Still, you write a test first which, again, the author diverged from.


I don’t either. But somehow TDD is considered the sanctioned way to work with unit tests, and that it is almost pointless to write tests unless you’re doing TDD. I am relieved to read that I am not the only one.


it is just the classic arrogant blogger isn't it?

"I rediscovered something obvious, mischaracterised what exists already, and I make up a term and pretend like it is novel, and credit myself with it"

Hoping someday to be the new Fowlers, surely.


You're getting downvotes for snark but you're not wrong.


the truth is ill heard lol


It's hardly a secret, but it's still not widely understood that one huge advantage of starting with tests is that it forces you to think about how an API will be used.

Instead of just writing a class with a bunch of methods that seem useful, you write the tests to figure out what a user actually needs, and iterate on that. And as a wonderful byproduct, you get a suite of regression tests so you can comfortably refactor and add features in the future.


I learned TDD as a development method. Too many people imagine TDD is about testing.

It's explained right there on the cover Test *Driven* Development. Not a testing method. A development process guided by unit "tests". In quotes as unit tests are at least 51% about forcing you to write small, isolated blocks of code with well defined and simple interfaces. Units. A large percentage of remainder is enabling ruthless refactoring (refactoring being a huge part of TDD's development practice/philosophy) by ensuring you have not violated those interfaces. Only a few percent is actually about having "correct" code.


This is one motivation for BDD - it doesn't contain 'test' in the title, so it's easier to explain that it's a development method.


I often implement top-down, calling non-existent methods that do what I wished they existed to do. Then implement the called functions or even classes. The source should be comprehensible on its own at each level with good naming and separation of concerns. This is a generalization of how to write a recursive function, pretend it exists before you write it.


I'm actually in the middle of a hobby project, where I am keeping a devlog.

I've repeatedly, throughout my devlog, emphasised that I'm calling functions that don't exist, then I create stubs of those functions (a one liner returning an error), then I populate the stub with actual logic.

To me, this is the most natural way of writing code.


I write code bottom up. Break problem to first principles, write/reuse primitive, combine them. Fixing bugs is easy as one of underlying things doesn’t work and you can scope it down quickly and patch with test.


When it comes to interface design, I draw top & down to middle. Find what interface or mechanism that is the common thing that makes the lower-level agnostic and the upper-level about applying or reapplying that mechanism in varying contexts.

Some might call that thing a 'narrow waist' but I don't quite equate it to that although it's a good example. Is the thing you want a stream of bytes, sequence of messages, unsequenced messages, or priority-ordered messages?

The problem with bottom-up is that you don't know the context. The problem with top-down is that you do know the context--and can leak it into what should be without. I find the latter works out better if you're aware of avoiding it. How do you avoid not having context without going top-down?

An analogy in UX would be Windows vs Mac. Windows builds things then puts the UI on what they've built--in many different kinds of settings places. Mac figures out what information & controls the user should have and how to name and group them.

Edit: I actually have a recent real-world example. Making a subsystem for querying and mutating things, what they are doesn't matter. We set out making an 'adjust' operation and a 'move' operation because we wanted to preserve paired decrement/increment amount for moves. We were also certain that we only needed to move between two things, or adjust many things but only of one kind. We got all the way up to the top where the public API was getting close to complete. We discovered use-cases that made sense to to more than those operations. We were able to shuffle things around and ended up with a fully-capable 'adjust' operation that can work on any number of kinds/things, and a fully-capable 'move' operation that can work on any number of kinds/things. On top of that we added checks to only do what we need now. The more general things were substantially harder to make work efficiently which is why we chose not to do it when we were sure we didn't need it. We were wrong.


I don't get what you're trying to say.

You know the context. You have the whole problem in front of you. The question is about which direction you're going to solve it.

For "you want a stream of bytes, sequence of messages, unsequenced messages, or priority-ordered messages?" the answer is you probably want type parametrized iterator at this level if you can ask this kind of question.

To put it in other words you put more attention into trying to find underlying composition of algebras in problems than doing adhoc, inlined imperative constructions littered with if statements that you add every time some issue is discovered.


I didn't understand what this meant when I first read it. Now I see it's basically saying the same thing as I was trying to get across. The "underlying composition of algebras" are the mechanisms and policies with the components being the mechanisms and the differing compositions being the policies chosen when used. If this same pattern can be used for all problems, then you're effectively skipping the step where in my case I would find a partially pre-composed set and make that the interface.


An example of what I'm describing is separation of mechanism and policy, but not specifically in the sense of OS authorization. The policy is where the different contexts are applied and the machanism is free of context but capable of executing it.


The problem with that is that no one, not even you, has 100% accurate foresight.

IOW, you will never have your bottom layer done correctly or completely.

At some top layer you're going to think "well, looks like I won't be need that function", OR, "Well, looks like I am missing a function for $FOO"/


That's what refactoring/rewriting is for. You have your piece-parts, you start combining them at the next layer up. Realize some of them aren't what you need. Maybe you need more, maybe they should be combined in some fashion, or maybe their behavior should change. So you do. It's not like people (well, fools might, but who wants to be a fool?) say, "You know, I already finished that bottom layer, I should never, ever, ever change it. That would be admitting I was wrong and not psychic. I can't have that." Since most people aren't fools they will, instead, say something like, "Oops, that doesn't compose well. Time to fix it."


> That's what refactoring/rewriting is for. You have your piece-parts, you start combining them at the next layer up. Realize some of them aren't what you need. Maybe you need more, maybe they should be combined in some fashion, or maybe their behavior should change. So you do. It's not like people (well, fools might, but who wants to be a fool?) say, "You know, I already finished that bottom layer, I should never, ever, ever change it. That would be admitting I was wrong and not psychic. I can't have that." Since most people aren't fools they will, instead, say something like, "Oops, that doesn't compose well. Time to fix it."

I'm not saying it never gets fixed, I'm saying it's extra work compared to top-down.

Top-down produces only and exactly what is needed to get the current layer to compile, run, and pass/fail.

Bottom-up requires that the bottom-most unimplemented layer be written to provide at least what the next higher level needs.

And since you cannot predict that perfectly all the time, you will have to come back and refine (not refactor) that layer: remove some functionality that turned out not to be needed, or add some other functionality that turns out was needed.

This step is never going to happen in a top-down effort.

[PS. At work I've almost always done bottom-up, because that suits the workflow constraints better: 1) Never leave commit a non-working build, 2) never do a PR for an incomplete module, 3) Always make PRs as small as possible, etc.

For my own projects/hobbies/experiementation, I used to do the same thing. Now I'm experimenting with the top-down approach and my velocity is a little faster because I am never spending time removing stuff that was written.]


This is exactly my knee-jerk opposition to 'refactoring'. In many cases the cause for it isn't that the factor (seams along which the problem was separated) has changed, but rather the factor wasn't known at the time and was more like primitives composed without context or deduplicating patterns/machinery. Going top-down you'd be conceptualizing/picking/naming the factors as you go. Sometimes they change too but not at arbitrary points in layers of implementation, and this all happens sooner than later.


It doesn't have to be perfect to be a decent approach sometimes. A spare or missing function isn't necessarily that bad.

It's the premise of On Lisp, for example, http://www.paulgraham.com/onlisp.html


Bottom layers have functionality that is needed. There is also reuse, dovetailing happens, you have opportunities to see new problems from lenses of already solved ones.

We're using this approach in trading production system for about 5 years now and I recommend it, works very well.

There is also side effect where your dependency tree is more shallow.


> Bottom layers have functionality that is needed. There is also reuse, dovetailing happens, you have opportunities to see new problems from lenses of already solved ones.

> We're using this approach in trading production system for about 5 years now and I recommend it, works very well.

I haven't worked anywhere since 2004 that used any other approach; it's the dominant approach to development - commit code, starting from the lowest layer to the highest.

My experimentation thus far using "commit code, starting from the highest level to the lowest" has been more satisfying to me in terms of velocity.


This is sometimes called the London school of test driven development. At least, if you then mock the non-existent methods to make your top level test pass.


I'm willing to accept that there exists somewhere a large mass of developers that don't think about how their code will be used before writing it.

But the one thing I can't agree on is that teaching them TDD is either necessary or sufficient for them to start thinking about that. I'm pretty sure that if you can manage to write code without knowing how it will be used, you will write tests that way too. Writing a test doesn't force you to think about your API any more than writing the API.


It "prompts" you to think about the API is probably a better way of putting it. That prompt is helpful.


Depends on what you're doing. If you're adding functionality because of a need, you've got a minimal "API" (or rather, interface) already defined. If you ignore that, you might end up with an interface that's not complete and with features that won't be used, but need to be maintained anyway.

There's no fast and hard rule, I'm afraid.


How often do you write methods that you dont use? I only write a method if I'm going to use it. In which case testing doesnt offer that advantage.


It also doubles as documentation that’s forced to stay up to date. Code has a way of becoming hard to follow especially when edge cases and performance enhancements are added.


> starting with tests is that it forces you to think about how an API will be used

And tests that describe how an API can be called are BDD. Hold the Cucumber.


I like what you are saying, but I've seen enough test driven development to know that it doesn't force any thinking about how the API will be used. It certainly can help illuminate that, but I've been constantly surprised by the convoluted "use case" tests that I've seen people create and think nothing of it.


I just started working at a 15-person startup where I am on a small team of 5 devs. People typically "don't have time to write tests" and are encouraged to pump out new shiny features every 2 weeks.

I am a new to development but I've noticed that lots of errors pump through code where "No method X defined for nil object" or something like that. I wonder if I could somehow make the case that we'd spend less time reacting to problems and bugs if we spent more time up front writing tests that could give us more than 19% code coverage.

Does anyone have advice for starting this conversation with my boss or during one of our standups? I know I could somehow pitch the value as "less problems later for more time spent now" but is there a more effective way to say it?


Given that you're new to development, you'll probably lack the sway to get people onboard with just words. It's likely that the team already knows about testing and the benefits of it (its not a secret, the text is everywhere,) and yet they choose to not do it still. They believe tests will slow them down, probably based on past experience, and using the words of people who are not there on the front lines is unlikely to sway them.

To get out of that situation, I'd recommend leading by example and showing how you have personally managed to use tests, in the space they are operating in, to make your life better and develop faster. Once you have something that has provably worked for you, you can start evangelizing it and onboarding other people.


> I am a new to development but I've noticed that lots of errors pump through code where "No method X defined for nil object" or something like that.

Thats the kind of thing a good programming language is supposed to help with.


You could introduce (or write more) tests from your side for new feature that you write or for bugs that appear. But in general it takes a bit more experience to get unit testing into existing code, but not undoable and you could practise this for a smaller, independent or even hobby project. Still, the critical part is to get support, not just acceptance, from your boss as sometimes "less problems later" argument does not convince them as "it always worked well without tests".

And just adding tests isn't the only thing it takes to improve the software quality.


I will be a little contrarian maybe but I don't really start with tests.

The rationale being that until the API is fixed, I don't want to have to adapt the tests to the changing API all the time.


I don't get the obsession with TDD. Maybe it's because I have been in the BI/DE space for awhile, but nothing is defined well enough to justify TDD in the first place. Everything is hobbled together and functional integration tests are much more important. Unit tests aren't going to catch when my CF stacks are broken. Good dev tools and architectural design are much more important in my experience.

I totally understand where TDD shines. When you have a well defined problem to solve, unit tests are the definition itself, and the business logic therein.


> The rationale being that until the API is fixed, I don't want to have to adapt the tests to the changing API all the time.

The rationale behind test-first is that you won't know what the API should look like until you try writing some sample client code that uses your provisional API to access some data. So why not make those tests, iron out the kinks in your API and have a few functional tests at the end of it; two birds, one stone.


Still makes more sense to go API first then unit tests rather than the reverse to me.

Since an API is a composition of multiple parts that can be tested (the unit in unit testing), it only makes sense to create these tests once the relations between these parts/units are well-established.

That doesn't have to always be after the API is completely done of course. Some internal functionalities are likely to be immutable. They can be tested quite early in the iteration timeline.


We're not really talking about unit tests though, we're talking test-first. They don't have to be unit tests, which is why I described writing sample client code because that's what you need to do to anyway to design your API.


Hum... Usually TDD is associated with unit tests and the article even mentions "unit tests" in its title...

What you describe however seems closer to integration/system testing which is one approach I and other people in this thread also tend to favor. But I've been writing UI code predominantly so I am biased here as unit testing has lower value.


I tend to describe programming as building a house of cards in your mind. It's a way of thinking about it that's used sometimes to explain why interrupting a developer can be so frustrating (because they have to rebuild the house).

Writing code in pieces (functions) where each does one thing, even if that thing isn't reused elsewhere outside of the one place you need it, if one way to reduce the impact of that house of cards; because you only need to focus on the logic that's important "where you are".

Unit tests are another way. You can describe the behavior of your code in tests and, if you break something because you couldn't keep the entire model (and individual pieces) in your head at the same time, your tests help you notice that. If you're building your code initially, your tests can help you identify and focus on individual behaviors you need to work.

I think the above is why I think both pre-code and post-code tests are useful. It's helpful to write tests before you write the code, to guide your development. But it can also be helpful to write tests after; especially as you identify things that don't work correctly because you didn't realize they were requirements when the code was originally written. They are, effectively, regression tests... but they're more than that, too.


Not quite binary ping-ponging, but I've have often found rotating through writing functional code, writing documentation of what's desired, and writing test cases very helpful in defining "What should each function do? What parameters and context should each take? How should the overall solution be structured?" in cases where that isn't entirely clear (e.g. first pass at solving new or complex problems).

Sketching functional code (and/or pseudocode/stub-only functions) lays down a concrete hypothesis about what the code (or at least its key parts) should be. Writing docs fleshes out the aspirations and expectations (including helping you define non-goals and not-yets). Writing tests demands thinking through edge, corner, and other hard cases. Each informs one's understanding and intuition about the other two, and almost automagically drives gap analysis. "Whoops! Haven't thought about/implemented/documented/tested that yet! Let's go do that now! (or at least put it on the TODO list)" It's a strong tight loop.

Would also suggest you don't limit your "write tests" phase/work to pure "does just one thing" unit tests. You don't initially want full end-to-end tests that assume and require everything's working in order to start testing, but some of the tests can and probably should venture into the "requires multiple components interacting" space usually called "integration tests." I think of that happy middle ground / hybrid between pure unit and full integration tests as "functional tests" or "functional block tests" where the size/complexity of the functional blocks under test have more leeway than unit testing/TDD dogma usually allows.


I always think, the next time with tests.

Then I have to integrate a monster of a library that needs a whole battery of polyfills, and does it's own thing rendering modals somewhere, and I'm not in the mood anymore.

Mocking that whole thing? Hmnothanks.


Despite the niche obsession with 100% test coverage, tests aren't an all-or-nothing thing.

Test what can easily be tested. Architect your code such that business logic, etc is modular and testable. Don't worry too much about testing the hard stuff, especially if it's not likely to break.


But isn't hard stuff the most ugly one when it breaks without a test?

Stuff that's easy to test is usually also easy to debug.


IMO there are 3 main purposes for tests, and knowing which one you're doing helps decide if a test is worthwhile, as well as how in-depth the group of tests needs to be:

1) Helping you work through a complicated piece of logic.

2) Encoding some sort of requirement so future refactoring/bugfixes/features - perhaps written by a new developer - don't break that requirement.

3) When fixing a bug, ensuring the bug doesn't reoccur.

Tests that fall under (2) often feel the most useless, but I've found to be the most useful. They're typically the simple ones that don't feel like they need a test, but years down the line not every developer knows these requirements. Documentation is easily missed or ignored, but a test that's started failing? Sure there's still a chance they'll just remove/change/skip the test, but they can't just ignore or forget about it like with documentation.

Tests that fall under (3) are very similar to (2), except it's not an external requirement known from the start. These are ones that I've seen people occasionally write while they're fixing the bug, then remove afterwards so as not to clutter up the tests. Or do manually in the shell and never write a test in the first place (I'm definitely guilty of that). But whatever happened here was just complicated enough that the previous develop(ers) missed the conditions that caused the bug - so future changes to this part of the code have a good chance of reintroducing it or something similar. These are worth keeping.

Tests that fall under (1) are definitely useful while the code is being written, and typically people want to keep these because the logic is complicated (or even only write them because of the complicated logic, even if they didn't need it to write the code), but I'd say there's a further question here: How likely is this code to change (ever)? If you didn't write tests with the initial code, it all works, and it's something relatively generic that is unlikely to change... it might not be worth it. If it's likely to change it could end up falling under (2) or (3) in the future, so it might be worth a detour in writing the tests. If the tests already exist because you needed them for case (1), then it shouldn't hurt to just not delete them.

(I'm sure there's other purposes that don't fall under these three, but these are the main reasons for tests in my mind)


> Stuff that's easy to test is usually also easy to debug.

Stuff that has a test often doesn't need to be debugged.


I wonder how I'm supposed to test everything I'm developing.

Testing is already hard enough on its own even when following practices to make everything replaceable and mockable.

At some point I have taken the code so far apart that I am testing nothing useful.


Yes.

Testing feels a bit like the sea shore problem.

The more detailed you measure a shoreline, the longer it gets, but what's the real length?


I'm seeing the terrible practice of people writing code bases where a lot of logic depends on the ui framework.

I guess your case is something like this as well, that's why it's so hard to test.


True story.

Problem is, often you get an easy integration route, where lib and UI come in one package or a hard route, where you have to build a UI around the lib.

To save time, you use the full package, but then testing becomes a nightmare.

And the sad thing is, the stuff that's hard to test is crucial to test.


Haha totally agree :)

I guess in this case what matters is the estimated lenght of the product lifecycle.

I'm right now writing open source library in ts hoping that a ui will pick it up. I may need to write the UI myself at the end, but that will be much easier after I put every feature in my library I can think about :)


Being honest most companies I worked, unit tests have no use at all. They are just writen because someone determined some arbitraty percentage of code that needs to be covered, and because it takes as much time to write the tests as to write the code, people just write useless tests that test absolutaly nothing usefull, and are worth nothing.

And this is not small companies we are talking about. They are banks, big educational institutes, etc. In the end the only thing most managers care is that the litle label in the pipeline dashboard that indicates the test percentage stais green.


Unit tests need to: A. Be taken seriously by the dev team. B. Taken at least moderately seriously by management. Getting the unit tests passing isn't always the most important thing this very second, but they do need to be something we get back to. C. Run automatically, and preferably as a gateway to the commit process. Some sort of automatic continuous integration thing is critical. I have a lot of smaller projects rather than one big project so my CI is often just a git pre-commit hook. This works fine at a small scale; if "CI" is less than ten seconds I think it's advantageous to keep it as local to the dev as possible rather than shoving it off to some external server to complain later.

Without these things, they're mostly theatre. With them, they're an incredibly valuable aspect of development that I don't intend to ever develop with ever again, by which I mean, if I do end up taking a job where this wasn't already in place, and I'm not allowed to put it into place, I will shortly be at a new job. Life is too short for the kinds of stupid debugging you have to put in to an untested, uncovered code base. (I don't mind debugging in general. It's a fact of the job. But that pull your hair out every single time, that's an uncovered code base.)


Thank you! I feel like all of this Unit test stuff is just a case of Emperor's New Cloths: everyone writes tests, no one knows why but won't admit it. I've never never ever experienced in my life a time where a major problem was automatically found by some unit tests before it got pushed to production.


"I've never never ever experienced in my life a time where a major problem was automatically found by some unit tests before it got pushed to production."

Then you, or somebody near you, is doing it very wrong. I do not mean this as a moral judgment, I mean it as an engineering process diagnostic feedback. I literally can't count the number of times it has popped up a bug that I wouldn't have expected because of some change.

It is that very characteristic that makes me love them so much. No matter how carefully you code you can never get away from the problem of a small change over there causing a breaking over here because of something you couldn't even have anticipated, but you don't have to wait until some large-scale QA process or even production deployment to find out; you can find out 15 seconds later, and then fix it, or realize your new change is fundamentally untenable, or any number of things. I'd say "I don't know how people develop without these things", except I do; the code bases are treated like quick-set concrete and nothing can be changed once laid down. What a stultifying way to code. I would hate to work at a job like that.


> Then you, or somebody near you, is doing it very wrong. I do not mean this as a moral judgment, I mean it as an engineering process diagnostic feedback.

IMHO thats the biggest problem with TDD. People often think of TDD as a set of requirements that all push for some set of benefits.

But it is really a development methodology where you as a developer heavily utilize tests to drive the development process itself. It is a mindset, not a checklist.

Perhaps easiest to explain (with an inaccurate comparison) as - it is like REPL-driven development but with persistent and sharable artifacts.

Behavioral driven development is IMHO stronger in this regards, because people tend to understand how the additional artifacts (beyond what you might see with a requirements list or use case document) are part of a methodology. People can better separate the methodology from the benefits/outcome.


Here is the thing: Most managers want you to implement the test and the code in the same time It would take to write only the code.

The thing is, in the end most devs do shit tests becase theres no time allocated to that, and the test end up being just a number that needs to be met so the code could run to the dev ops engine and generte a new version of the software.


Tests typically let you write code faster because you can rely on the test suite to check most standard assumptions and inputs/outputs. Thus, you have significantly less to test as time goes on.


Not writing test code is faster the first day.

It's somewhat faster the second day.

It's a bit faster the third day.

It's probably net faster the fourth day, though by now you're developing noticeably more slowly.

It may still be net faster the fifth day.

It's net slower the sixth day, and the delta gets worse from that point on.

And I don't mean "per bug" on that, either. I mean, per project. By the sixth day of the project, you are net slower not writing any test code. And again let me emphasize net; by day six you are already losing overall.

Expecting test code to be written in the same amount of time as normal code is actually eminently reasonable, indeed it's the only sensible way to do it... if you do it right.


That's because by definition it is not going to be "major" problem since the unit test acted as gateway before it got pushed to production, instead you'll probably be 'meh' and fix it once the unit test fails.

This reminds me the saying of a manager arguing why do we need so many SREs since the system is working fine.


This is sort of why I wrote the article. Professionally, I’ve seen tests just cause tons of noise and offer little benefit. So I tried applying it to a personal project on a whim and found some unexpected benefits


Test driven development on one project would often save me weeks per bug fix.

This was because it maintained an environment where I could replicate portions of the business logic (as assembled modules) outside a production environment. This made it much easier to do analysis/fuzzing of one component of the system, vs trying to replicate and do post-mortem crash analysis on the entire deployed system.

But I clarify this is TDD, not unit tests. A development methodology where unit tests are wedged in after the fact do not promote the sort of modular programming needed to be able to do this.


I think it can be taken too far but why not have some unit tests? They are the easiest kind of test to write and the fastest to run.

Also, you dont see it because those bugs were fixed before production.


I have. A lot.


Our team too, time saved is fairly quantifiable.


I mean, you probably need automated CI before this becomes likely as otherwise people just forget to run the tests...


You absolutely need that.


Yes, test frequently and automatically.


Also most examples of unit tests that I see in various examples I've seen amounted to testing "Does the code do what the code does?" Unless you have a clear spec to implement, or you have some weird edge cases that you need to make sure get handled correctly, I see very little use in unit tests (integration tests are very helpful though). The only time I've really used unit tests were when we had some specific financial formulas that we were implementing and needed to make sure the code matched them.


PS: dont take me wrong. Unit tests are useful if used properly. But people just don't use it correctly.

To most people what is important is the percentage of code tested, instead of test critical logic conditions for your software piece.

I work in a big bank in my country, that would insist us to do unit test in UI. Things like, if I set the button visibility to true, the button needs to be visible. Like... no shit Sherlock? What do you expect to happen?


> people just write useless tests that test absolutaly nothing usefull, and are worth nothing.

Oh man, those are my favorite. They're super difficult to critique in code review. "This test doesn't really test anything" isn't very helpful but writing a proper "do this" is often more work than writing a proper test yourself.


Is that not helpful? It's definitely helpful.

"This test doesn't do anything", then reject & request changes, "add substantive tests that cover the following invariants: ...".


Thats the thing. Its hard to describe, but you know when you see it.


Early on in my career I was overly enthusiastic about writing unit tests for maximum coverage. I wouldn't say they were totally useless, but anecdotally I don't think they caught many bugs/regressions and over time it was a lot of code to maintain. While it made refactoring feel 'safe' to do, it also slowed me down quite a bit.

Now, my strategy is to instead focus on writing a few solid end-to-end/integration tests. These tests often find just as many bugs/regressions, are actually testing the entire system, and much easier to maintain. Most bugs happen due to bad interactions between systems. I still write unit tests for some tricky code, it just isn't my first choice.


> While it made refactoring feel 'safe' to do, it also slowed me down quite a bit.

I find that unit tests actually make refactoring harder.

The problem is, unit tests need to be rewritten, or at least substantially shuffled around, every time you refactor. If you're making a "big bang" change for a major new feature, and that requires refactoring, then it's possible to justify that cost. But that's not how it normally happens.

Instead, a bunch of small changes over time usually make it apparent that the code would be a lot clearer and easier to maintain of the arrangement of responsibilities were different. It's already hard to ever justify the cost of that refactoring under any one small change. But add in the cost of updating all the unit tests and it becomes even more unlikely. Those updates also feels like demoralising busy work, whereas the actual change feels productive, so it also adds a human factor.

Overall, the net effect is that code with a lot unit tests ends up with a stagnant and confusing design.

As you say, a lot of the practical benefits can still be gained by end to end tests, while avoiding both the cost of writing so many tiny tests and the impact on long term design.


After many years of experience, I agree.

Tests are a business investment, of tech resources, to create business value.

It's good to focus on the Testing Pyramid [1]. High level tests are slow and brittle, but connect the low-level code to business features. Unit tests are fast and detail oriented.

In practice I write 1-2 high level tests (generally end to end, sometimes UI, or API/integration tests) to help focus development and have something that the business understands. Unit tests are helpful to "smooth the path forward" to ensure the code works as expected. Integration tests are great to iterate on, so that new code and tests actually work with real APIs correctly.

Tests are not free. However they create a lot of value -- they create (business and tech) confidence that the system is working as expected. Like you mentioned they assist refactoring, which makes the code much cheaper and easier to work with.

[1] https://martinfowler.com/articles/practical-test-pyramid.htm...

(disclaimer: writing a book about tech feedback loops, e.g. tests)


Uncovered code tells you something useful. That there are no assertions made about some code.

Covered code tells you nothing. It tells you this code may or may not have assertions made about it.

Tracking coverage is good because it shows you what code isnt tested. But once its covered, you have no idea. So instead of increasing coverage, you should be evaluating the new assertions being made to uncovered code. But virtually everywhere I've been has just used coverage going up to mean your tests are sufficient.


Unit tests are more about writing good code and less about catching regressions.

Code that you can easily write an automated test for generally is easier to maintain.


What do you use primarily for writing integration/end-to-end tests? Do you have any specific strategies for it? I’ve been using cypress but not having a ton of success with it, it just feels like everything is permanently broken, and generating/maintaining mock data is exhausting.


I wish HN had a central thread feature for discussing oft-debated topics like TDD, microservices, and agile. This way, we could contribute to the conversation instead of constantly rehashing the same points through new posts.


Terms like “TDD” come loaded with all kinds of varying opinions on what that means and how it should direct your other behaviours. It’s okay to write unit tests first without ever thinking or saying “TDD.”

“But that’s TDD!”

“I dunno. I just wrote my tests first.”

“But you’re doing TDD wrong because you didn’t go and make it minimally pass before improving it!”

“I dunno. I just wrote my tests first.”

Doesn’t mean it’s not right to adopt a pattern when you identify that you’re implementing something similar to it. But you aren’t obligated to get trapped in its gravity well.


> all kinds of varying opinions on what [TDD] means

I guess I go by what Kent Beck, who coined the term said TDD meant.

https://www.goodreads.com/book/show/387190.Test_Driven_Devel...


> TDD shines when refactoring a codebase. Treat it as a black box, document and validate the behavior, and start rebuilding the inside.

This looks to me like writing tests around code that didn't have any before. It is crucial for refactoring to have tests; moreover, I would say that refactoring is impossible without tests. But it is not TDD, it is just adding tests where it should be in the first place.

> when you’re cobbling together a project from scratch, iterating and hacking away until something decent works, writing test code to throw it all away seems like a waste

This is exactly what TDD is for: to thoroughly think over the architecture and implement it right from the first take, without throwing away a lot of code. This is exactly the problem of people like me or author of the post: we just love coding as the process. Attention deficit greatly adds to that, making good architecture an impossible feat. It is easier for us to rewrite everything from scratch several times.


> TLD focuses on ping-ponging between test and code.

I like this same concept for top-down and bottom-up design.

Top down advocates might say if you focus on implementation concerns, you risk building what's easy to build instead of what's needed.

Bottom up advocates might say if you focus on what you'd like to have, you risk building something that just can't be implemented well when something else (that can) might have been fine instead.

I think, instead, you should ping pong between both. Start at the top, think about the high-level design you want, then work downward and see if it makes sense. Maybe even write some code to try out some ideas. (Like, if I use this design, I'm going to end up needing this database query. Can I make that run efficiently?) Then percolate those lower-level concerns and the results of that research upward and see what adjustments you can make at a top level. Then repeat until you find something that works well at all levels.


My rule is simple: if I want to test something, I have to code it. I don't know if it's full TDD, but if it's worth testing it's worth automating. If there's some edge case where I trust the logic to Just Work, I don't test it at all.

The important thing I find that helps with the mental overhead of testing is a long script of integration tests is better than nothing. Too often "TDD" is interpreted as creating perfect isolation of unit tests on each method -- which is fantastic! But is also a lot of work.

Worse is better. If just checking in some precondition data and loading it in and then sequentially running a script of automated tests that all have side-effects that could impact the next test is your MVP for a test framework, do that! Having a suite that you have to run fully end-to-end isn't great, but it's better than nothing.

When that frustrates you, refactor it into true isolated unit test.


I don't think this is an "An unexpected benefit", I do it like for a long time conceit, that idea is based on "The Hemingway bridge" idea from the Nobel prize winner of literature in 1954.

I wrote about it a few months ago called it "1. Always go home with a broken build"


For anyone who hates to write tests, there's another testing strategy that i also found practical and useful, it's called Snapshot Testing. Basically, you can write a script to dump all outputs. And everytime you change the code, just try to compare the old vs new snapshot output.


Wonder if you could do this with some kind of deterministic fuzzing


Hm, it's new to me. Do you have any guide for this technique ?


The title of this post doesn't do the author any favours. It can be understood to be directed at the general reader, and I can assure you, that what is described is very much an expected benefit of unit tests.

But, it's also just a tool. That has to be used for the right reasons and the right circumstances.

One unexpected use for them is to implement the "right way" to do something as a unit test. Instead of having to remind them of what we agreed on, making a test is a great investment, and not the typical use case for tests.


This is pretty much how I work.

I describe my approach here[0].

I write tests and functionality together, and my tests are frequently harnesses; not "unit tests," as they are understood, today (In "the elder days," we used to call test harnesses "unit tests," but what did we know?).

[0] https://littlegreenviper.com/various/testing-harness-vs-unit...


> I’ve always poo-pood the idea of test-driven development (TDD), as it seems wild to me to write a test for something that you don’t know what it needs to do yet

Right. If you don't know what it needs to do, why in the HECK would you be writing any code at this point??

If you're just hacking / prototyping, there's no conceivable reason to involve tests.


While unit test are important for various reasons "The broken unit test stuck out like a sore thumb." is not one of them.

In the precise place in code you want to resume work, add in free text whatever mental bookmark you need, the failing compilation will stick out well enough.


I don't like the dogmatic test driven development appraoch - so here's mine:

---------------------

1. Use a programming language with an expressive statical type-system

2. Write only type-signatures for everything that I want to do (using type-holes) and continously compile my code

3. Once I'm finished with #2 and it all compiles, I start implementing all the methods/functions

4. If I can't implement a method because I got the types wrong (e.g. my types say that I will return a number, but in some cases I cannot and have to return nothing or an error) then I change the type-signature and go back to #2

5. If I can implement a method but I'm not sure it will do the right thing (despite compiling) without running it manually to see if it works, then I write one or more unit tests for the method. If I believe that changing the method will easily break it I also write a test. Otherwise I write no tests.

6. Once finished I write some integration tests for what I deem necessary - both positive as well as negative tests.

7. Done


Sometimes, I find I need to revisit some type signatures after I've begun to write code. And this can lead to my code not type checking for longer than I'd like. But I can still run my tests if I defer type errors to runtime (they still appear at compile time but as warnings). See (Deferring Type Errors to Runtime)[https://downloads.haskell.org/~ghc/9.4.4/docs/users_guide/ex...]


A "good stopping place" is the worst place to stop. This article is a good example of making it easier to get started next time by leaving some loose ends for your future self.


One reason for unit tests is to know when you've won. For example ChatGPT is already a limited form of general AI, but makes some silly mistakes for certain kinds of thinking.

An example would be - oh wait, it just passed it, nevermind. (I wanted to quote a kind of numerical thinking test it fails at it but it just passed.)

If this had been a test previous GPT's failed at, but it was coded up in a unit test, then maybe more people would get it. At some point AI will just pass all the tests we throw at it. It would be nice to have a spreadsheet of tests to 1) know whether we're there 2) show people that we're there.


As others have noted, this is basically TDD, and his idea of what TDD is was faulty.


I think you just re-discovered what people actually do when they say they do TDD.


> TLD (Test Led Development)

I've also heard it referred to as "weak TDD"


I like TDD myself but I'd rather write E2Es or integration tests.


Why not both? That's what I do, it's very effective and nothing about one prevents the other.


shameless plug of my side project: https://github.com/adamwong246/testeranto


You can also leave the code you've worked on last broken.


This just in: man tried TDD and likes it.


typescriptifying some legacy code is a similar idle brain task that i've sometimes done


I think the sincerity of this makes this the funniest thing I've read on HN.


I like the way that the author literally reinvents TDD while badmouthing "TDD." Which garners lots of commenters not talking about the author's content, but just here to also badmouth TDD.

It's possibly the best testing-related Poe/trolling I've seen.


Yes, this comment thread has correctly pointed out that my understanding of TDD was and still is incomplete. I think there's some aspects of TDD (such as refactoring after implementation) that I'm not interested in. The two folks I know who are big TDD proponents tend to gravitate towards test first development (TFD). In my mind, that was TDD. You write the tests and that drives how you develop the tests.

In a recent personal project I started using tests on a bit of a whim and found an unexpected benefit that I wanted to share.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: