As an agency, think of building pods of 2-3 people, maybe a designer + 2 developers. You'll be part of the first pod, so step one is to build your first team. The idea (simplified) is that if you figure your way to 2 pods, then you can scale to 10+.
As a SaaS you want to think to a recurring problem your customers have and automate them. It won't be the full 1:1 consulting, just a little piece. But you may already convert these existing customers.
The two are not mutually exclusive at the beginning in the sense that you can start finding a partner, build a SaaS together part time and see how it works/what you like better.