On: Launch HN: Momentic (YC W24) – Agents that catch bugs by using your app

The screenshot script is the scarier half of that story, because the seven failing tests did their job while the previews kept coming out at 40% and looked like work getting done. I had a similar situation, the engine landed on a screen that looked almost like the expected end state and reported a pass, and it took a person going through the step screenshots by hand to notice. Did measuring inside the frame fix it for good, or you need to check the previews as well?

Measuring inside the iframe removed the cause. The editor now opens oversized pages already fitted, so the zoom loop rarely has anything to do.

But nothing checks the screenshots themselves. A person still looks at them after every run.

What specific value or UI state could a test have checked in your case to catch the false pass?