I sincerely hope HTML snapshot testing is not being sold as a useful testing technique in the front-end world anymore. Edited snapshots are often glanced over in PR reviews especially when they are large and/or there are many of them. Visual Regression Testing (VRT) is more powerful in my opinion because it will flag any changes that will affect the UI.
I don’t know what the analog of VRT is in the infra world but I certainly see folks glancing over giant yaml snapshot diffs much like React component snapshots.
I just added snapshot tests to a project (yesterday!), in preparation for validating a major refactor. They’re invaluable now, and helped me catch two very important but very subtle regressions I couldn’t have expected to test. The output isn’t directly visible, it’s just plain markup which gets styled by downstream users/libraries. And there’s a strong expectation that it’s stable, which makes refactoring especially risky.
Now, I can’t predict whether the snapshots will remain valuable over the long term. I expect they won’t, but mainly because I’d like to see this library eventually fade away and its responsibilities move into its primary downstream target. But that may take years, it took nearly two years to get to this refactor which will help enable that. I’m betting the snapshots will be equally valuable whenever that goal becomes possible. In the meantime, they’re my best assurance that I can honestly tell people I believe the current work I’m doing is safe and correct.
I don’t think they’re right for every project or every testing scenario in a given project, but knowing when they are a great fit is important too.
I think that depends on signal to noise ratio. If your snapshots change a lot normally and it's irrelevant to the code you're trying to alter, you'll get used to skipping by them.
But if you use the snapshots as a means of understanding the templating and checking what it alters, people are much happier to examine the diff.
Agreed that VRT is more powerful: Jest was the first place I found this technique, though I've seen it for other things like SQL approvals before.
A few off the top of my head. Many e2e testing tools can capture and compare screenshots but I think to be maximally effective (and minimally frustrating) you want a tool to manage approving, reviewing, etc.
This is a short post about how to apply tests to problems outside the space of normal code. It covers two strategies – snapshot testing and verification of real production systems – using examples taken from my career, where investing in a way to test was really worthwhile.
The broader point is that processes or strategies you find in one part of software engineering can often be used to great effect in others, if you're willing to try.
I've got weirder examples, such as hacking TypeScript/BigQuery/dbt/Metabase to inject user-defined-functions into a production system, but these felt the most generally useful.
