What’s with the recent HN push against unit tests? Yes you need other tests too, but they serve a purpose. You can’t build on bad foundations! And the search space of integration tests is larger so it’s much harder to have good coverage of non-happy paths
I think it's a critical mass of people experiencing the "unit tests pass but the code is broken" problem. When unit tests are used to test glue code you end up testing your mocks and nothing else. Mocks are often done using a framework which encourages one off implementations per unit test. This introduces a maintenance burden where all the mocks have to be kept in sync.
Unit tests are very useful but we somehow landed ourselves in a place where we have lots of line coverage in our unit tests but little confidence that the code actually works.
There are at least 3 different applications at work where they unit tests are green while the code is broken or red when it's not. The cause is almost always the mocks. They either presume too much, making then fragile or they are flat out incorrect. Despite having a lot of test coverage there is little confidence for the developer that their change is correct.
In a sense the writers of the tests were "doing it wrong". The burst of articles on unit tests and their failure modes are a reaction to the prevalence of this in our industry.
Similarly, the unit tests never catch bugs and you're always changing them because of refactors or changing requirements. At some point it's like 'hey this set of tests have been rewritten 3 times and they never caught anything, so ... like does that mean that we just wasted time for the first two times?'
I've seen unit tests as documentation cause problems a few times. Like, maybe if you've got some sort of DSL where it's actually obvious what behavior is expected. However, more often it's 10-50 lines of setup / mock code and then some number of asserts and you're left trying to decide what the point is (ops, it turns out something got misinterpreted and the test is actually nonsense).
Finally, it seems like using the type system to design code where illegal states are not representable is starting to make some headway. Additionally, we're seeing increasingly powerful type systems make it into industry acceptable languages.
Even with typed driven development you still want unit tests in the form of property based tests. Also a nice way to resolve the 10-50 lines issue is to follow the Arrange, Act, Assert pattern where you could look at the second block to see the actions. Move as much of the Arrange section into a setup and it should be a bit clearer :)
> people experiencing the "unit tests pass but the code is broken"
Arguing against unit testing is like arguing against type-safety (and, usually, anti-unit-testing people are anti-type-safety people, too). The presumption always seems to be that if it doesn't solve every problem, it's unnecessarily slowing things down.
In my experience it's the pro-unit-testing people who are more likely to be anti-type safety people. The argument against type safety typically goes something like this:
I already have to write unit tests so why would I bother with types they don't add any real value.
Both camps are wrong. Unit tests are an unambiguous good. Types are also an unambiguous good. Both have some rather common failure modes though and guarding against those failure modes is useful.
Unit tests of purely functional code where you only need to provide an input and validate an output provide tremendous value. The unit test can treat the code as a black box and as a result the unit test is robust and resilient to changes in the black box while verifying that the box still produces the correct answer. Unit tests of code with hidden dependencies that need to be mocked require a lot more care to construct properly. Mock scripting frameworks encourage a number of bad habits. Things like "How many times did this method get called". Or "Always provide the same answer when this method get's called." The result is a hundred reimplementations of the same interface that are at best correct for the current version of the code they are testing and at worst completely incorrect reimplementations of whatever they are mocking. They all need to be kept in sync and maintained over time.
A shared in memory Fake will in general provide more value and be less fragile over time while also ensuring that your tests are actually testing the code and not the particular script you defined for the mock.
> (and, usually, anti-unit-testing people are anti-type-safety people, too)
This is the opposite of my experience. Most of the anti-unit testing people I've talked to are very much pro-type people. I wonder if anyone has done any studies that shows what the actual numbers look like.
> Arguing against unit testing is like arguing against type-safety
I disagree with this. Types (at least when they've been built on top of an actual logic) have the benefit of real costs and benefits. You can show what programs you are unable to write and you can show (mathematically) that certain failures won't happen.
Unit tests on the other hand are much more hand wavy. You can show that some refactors seem easier, but you can't prove it without a lot of data that has to be collected on a project by project basis. I'm not saying that I don't want unit tests if I'm doing a non-trivial refactor, but I am saying that that desire is more of an intuition thing. It's not like I can make any proofs around it like I can do with a type system.
you can have good experiences with unit tests or bad ones depending on the health of your codebase. Hard to test code is a smell that many people interpret as being a problem with the concept of unit testing.
Most of my experience is with React, and the majority of react devs I've seen don't unit test their code, because they don't write enough pure testable functions. As a result the community has leaned heavily on React Testing Library which is AMAZING for integration testing. Instead of checking that your function returns the right value, people will mount the component and then check to see that the right value is displayed in the rendered DOM. This obviously works, but writing the logic as a pure function and unit testing that function with a lot of different inputs gives you much more confidence.
It takes skill to know which parts to test with unit tests.
If you start striving for over 100% unit test coverage, then you'll be testing if the IDE pre-generated setters and getters actually set and get the value. This adds zero value to the codebase and you'll be testing things that, if they fail, will break half the world anyway.
Unit Tests are for algorithms, stuff that does something complex and not immediately obvious. Preferably deterministic, every time X goes in, X+Y comes out type of stuff.
Most tests should be either integration tests, testing the interfaces between different parts of the software or automated tests pretending to be the user, made with Robot Framework or something similar.
More like all languages are not equally easy to test. Just like all languages are not equally easy to write. It's incredibly easy to write mocks/spies in javascript without a single third party library. It's tedious to do that in Java.
There is also a corporate culture of tests that mandates bloated test frameworks, which leads to developers spending more time on writing tests than writing functional code.
That depends on if you are shipping tests to customers. Most companies are shipping code, not tests, so time spent on test is time not spent on code. You only want tests that make code easier to write.
You’re not just shipping code, you’re aiming to ship a correct program right? And tests are a part of that and will be on the list of risks the customer may be concerned about. It’s not just about making code easier, it’s about showing the code works as expected.
