> So if your code is ascertained to work at the high level, you also know that it must be working at the lower level too.
No, that is not guaranteed.
Integration and E2E tests can only cover certain code paths, because they depend on the input and output from other systems (frontend, databases, etc.). This I/O might be crafted in ways that never trigger a failure scenario or expose a bug within the lower-level components. This doesn't mean that the issue doesn't exist—it just means that you're not seeing it.
Furthermore, the fact that, by their nature, integration and E2E tests are often more expensive to setup and run, there will be fewer of them, which means they will not have full coverage of the underlying components. Another issue is that often these tests, particularly E2E and acceptance tests, are written only with a happy path in mind, and ignore the myriad of input that might trigger a failure in the real world.
Another problem with your argument is that it ignores that tests have different audiences. E2E and acceptance tests are written for the end user; integration tests are written for system integrators and operators; and unit tests are written for users of the API, which includes the author and other programmers. If you disregard one set of tests, you are disregarding that audience.
To a programmer and maintainer of the software, E2E and acceptance tests have little value. They might not use the software at all. What they do care about is that the function, method, object, module, or package, does what says on the tin; that it returns the correct output when given a specific input; that it's performant, efficient, well documented, and so on. These users matter because they are the ones who will maintain the software in the long run.
So thinking that unit tests are useless because they're a chore to maintain is a very shortsighted mentality. Instead, it's more beneficial to see them as guardrails that make your future work easier, by giving you the confidence that you're not inadvertently breaking an API contract whenever you make a change, even when all higher-level tests remain green across the board.
> This I/O might be crafted in ways that never trigger a failure scenario or expose a bug within the lower-level components.
You mean just like unit tests where every useful interaction between units is mocked out of existence?
> Furthermore, the fact that, by their nature, integration and E2E tests are often more expensive to setup and run, there will be fewer of them
And that's the main issue: people pretend that only unit tests matter, and as a result all other forms of testing are an afterthought. Every test harness and library is geared towards unit testing, and unit testing only.
> You mean just like unit tests where every useful interaction between units is mocked out of existence?
Sure, that is a risk. But not all unit tests require mocking or stubbing. There may be plenty of pure functions that are worth testing.
Writing good tests requires care and effort, like any other code, regardless of the test type.
> And that's the main issue: people pretend that only unit tests matter, and as a result all other forms of testing are an afterthought.
Huh? Who is saying this?
The argument is coming from the other side with the claim that unit tests don't matter. Everyone arguing against this is saying that, no, all tests matter. (Let's not devolve into politics... :))
The idea of the test pyramid has nothing to do with one type of test being more important than another. It's simply a matter of practicality and utility. Higher-level tests can cover much more code than lower-level ones. In projects that keep track of code coverage, it's not unheard of for a few E2E and integration tests to cover a large percentage of the code base, e.g. >50% of lines or statements. This doesn't mean that these tests are more valuable. It simply means that they have a larger reach by their nature.
These tests also require more boilerplate to setup, external system dependencies, they take more time to run, and so on. It is often impractical to rely on them during development, since they slow down the write-test loop. Instead, running the full unit test suite and a select couple of integration and E2E tests can serve as a quick sanity check, while the entire test suite runs in CI.
Conversely, achieving >50% of line or statement coverage with unit tests alone also doesn't mean that the software works as it should when it interacts with other systems, or the end user.
So, again, all test types are important and useful in their own way, and help ensure that the software doesn't regress.
> Sure, that is a risk. But not all unit tests require mocking or stubbing.
Not all integrations require mocking or stubbing either. Yet somehow your argument against integration tests is that they somehow won't trigger failure scenarios.
> The argument is coming from the other side with the claim that unit tests don't matter.
My argument is that the absolute vast majority of unit tests are redundant and not required.
> The idea of the test pyramid has nothing to do with one type of test being more important than another. It's simply a matter of practicality and utility.
You're sort of implying that all tests are of equal importance, but that is not the case. Unit tests are the worst of all tests, and provide very little value in comparison to most other tests, and especially in comparison to how many unit tests you have to write.
> it's not unheard of for a few E2E and integration tests to cover a large percentage of the code base, e.g. >50% of lines or statements. This doesn't mean that these tests are more valuable.
So, a single E2E tests a scenario that covers >50% of code. This is somehow "not valuable" despite the fact that you'd often need up to a magnitude more unit tests covering the same code paths for that same scenario (and without any guarantees that the units tested actually work correctly with each other).
What you've shown, instead, is that E2E tests are significantly more valuable than unit tests.
However, true, E2E tests are often difficult to set up and run. That's why there's a middle ground: integration tests. You mock/stub out any external calls (file systems, API calls, databases), but you test your entire system using only exposed APIs/interfaces/capabilities.
> These tests also require more boilerplate to setup, external system dependencies, they take more time to run, and so on.
And the only reason for that is this: "people pretend that only unit tests matter, and as a result all other forms of testing are an afterthought." It shouldn't be difficult to test your system/app using it the way your users will use, but it always is. It shouldn't be able to mock/stub external access, but it always is.
That's why instead of writing a single integration test that tests a scenario across multiple units at once (at the same time testing that all units actually work with each other), you end up writing dozens of useless unit tests that test every single unit in isolation, and you often don't even know if they are glued together correctly until you get a weird error at 3 AM.
> To a programmer and maintainer of the software, E2E and acceptance tests have little value. They might not use the software at all. What they do care about is that the function, method, object, module, or package, does what says on the tin; that it returns the correct output when given a specific input; that it's performant, efficient, well documented, and so on. These users matter because they are the ones who will maintain the software in the long run.
I completely disagree with this take, as a professional programmer and maintainer of software. What I care about are two things: (1) if I make a change, I don't break existing user-visibile functionality, and (2) any feature I build actually addresses the use case I built it for.
E2E or integration tests are good for both of these purposes. Unit tests can help with purpose 1 as well, but not so much with purpose 2.
The disadvtange of relying solely on high level tests is that they don't pinpoint the error - if a test that integrates 5 components fails, I need to debug to even see which of these 5 components has a problem. In contrast, with a unit test, if a test fails, I typically know at least which specific file has a bug, if not the specific function. This is really nice, but not as critical - which is why unit tests are generally a nice to have, while integration/E2E tests are a necessity.
> So thinking that unit tests are useless because they're a chore to maintain is a very shortsighted mentality. Instead, it's more beneficial to see them as guardrails that make your future work easier, by giving you the confidence that you're not inadvertently breaking an API contract whenever you make a change, even when all higher-level tests remain green across the board.
This is the kind of dogmatism I want people to understand. I’m not saying unit tests are useless but they have very narrow use, in units that encompass slightly complicated logic. Most of us write classes that just have a few for loops, if conditions, metrics and a few transformations. The overhead of writing a unit tests where, mocking all external services and continuously maintaining them when every small code change causes unit tests to break (false positives) is pretty high.
> Most of us write classes that just have a few for loops, if conditions, metrics and a few transformations.
You're describing code. At what point does code become "worthy" of a unit test? How do you communicate this to your team members? This type of ambiguity introduces friction and endless discussions in code reviews, to the point that abiding to the convention that all code should be unit tested whenever possible is a saner long-term strategy. This doesn't have to be a strict rule, but it makes sense as a general convention. Besides, these days with LLMs, writing and maintaining unit tests doesn't have to be a chore anymore. It's one thing the tech is actually reasonably good at.
What I think we fundamentally disagree about is the value of unit tests. That small function with a few for loops and if conditions still has users, which at the end of the day might be only yourself. You can't be sure that it's working as intended without calling it. You can do this either manually; automatically by the adjacent code that calls it, whether that's within an integration/E2E test or in production; or with automated unit tests. Out of those options, automated unit tests are the ones that provide the highest degree of confidence, since you have direct control over its inputs and visibility of its outputs. Everything else has varying degrees of uncertainty, which carries a chance of exposing an issue to end users.
Now, you might be fine with that uncertainty, especially if you're working on a solo project. But this doesn't mean that there's no value in having extensive coverage from unit tests. It just means that you're willing to accept a certain level of uncertainty, willing to tradeoff confidence for convenience of not having to write and maintain code that you personally don't find valuable, and willing to accept the risk of exposing issues to end users.
No, that is not guaranteed.
Integration and E2E tests can only cover certain code paths, because they depend on the input and output from other systems (frontend, databases, etc.). This I/O might be crafted in ways that never trigger a failure scenario or expose a bug within the lower-level components. This doesn't mean that the issue doesn't exist—it just means that you're not seeing it.
Furthermore, the fact that, by their nature, integration and E2E tests are often more expensive to setup and run, there will be fewer of them, which means they will not have full coverage of the underlying components. Another issue is that often these tests, particularly E2E and acceptance tests, are written only with a happy path in mind, and ignore the myriad of input that might trigger a failure in the real world.
Another problem with your argument is that it ignores that tests have different audiences. E2E and acceptance tests are written for the end user; integration tests are written for system integrators and operators; and unit tests are written for users of the API, which includes the author and other programmers. If you disregard one set of tests, you are disregarding that audience.
To a programmer and maintainer of the software, E2E and acceptance tests have little value. They might not use the software at all. What they do care about is that the function, method, object, module, or package, does what says on the tin; that it returns the correct output when given a specific input; that it's performant, efficient, well documented, and so on. These users matter because they are the ones who will maintain the software in the long run.
So thinking that unit tests are useless because they're a chore to maintain is a very shortsighted mentality. Instead, it's more beneficial to see them as guardrails that make your future work easier, by giving you the confidence that you're not inadvertently breaking an API contract whenever you make a change, even when all higher-level tests remain green across the board.