Hacker News new | past | comments | ask | show | jobs | submit login

> end-to-end tests should have the best maintainability score.

End to end tests encompass far more total details than implementation or unit tests. If you're testing a website, moving a button breaks a test. Making the button have a confirmation breaks the test. The database being slower breaks the tests. The number of items in a paginated list changing breaks the tests. You're testing not just the behavior and output of interfaces, you're testing how they're composed. The marketing team putting a banner in the wrong place breaks the tests. The product team putting a new user tour popover on the wrong button breaks the tests. The support team enabling the knowledge base integration in your on-page support widget breaks the tests.

Moreover, the cost of fixing the tests is also often higher, because end to end tests are necessarily slower and more complicated. Tests often become flaky because of a larger number of dependencies on external systems. It's often not clear why a test is failing, because the test can't possibly explain why its assertion is no longer true ("The button isn't there" vs "The button now has a slightly different label").




Not all systems are web sites. Those issues are not inherent to the concept of end-to-end testing.


It still illustrates how E2E testing encompasses far more details than other forms of testing, which necessarily makes them more brittle


Sure, they often are less reliable, which is the R of the smurf and not the M.


E2E are less maintainable because by definition. They are the type of tests that has the most dependencies on sub-systems and other systems - this means if a test fails you ll need to work more to figure out what went wrong, than in an integration test, which depends on less sub-systems.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: