This has been my mantra as well. Test big and you'll likely fail or win a lot faster.
I'd also suggest making sure you approach testing by starting with a hypothesis and then testing that hypothesis. By voicing or writing down your hypothesis, you will be more likely to avoid the "stupid test trap", a phrase I use to describe running tests like changing button colors -- what hypothesis is changing button colors trying to prove? That people like red more, "for reasons"?
As mentioned above, each A/B test you run has a serious opportunity cost. I'm fortunate enough to have thousands of people going through my sales funnel daily but, even then, a test will still take several weeks or months to validate and anything approaching a change of 5% or less will really eat away at your ability to test more meaningful things.
TL;DR - Test big, drastic changes instead of fiddling with button colors and headlines. Examples of drastic changes:
- Entirely different homepage with different messaging.
- Change "Features" page to "Benefits" page and change its content accordingly.
- If you're offering a free download or free product, test asking for an email first.
- If your SaaS has a long signup form, test allowing people to jump right in with just an email address.
- If you're sending a robotic "welcome" email, test sending a very short personalized email instead.