What about going into quant finance? :P I've heard it said that that's a better way to become a multimillionaire than trying to do startups.
But regarding startups... doing a technical startup that leverages PhD research (or just background knowledge), or even just a "highly specialized" software consultancy, is probably a better strategy than trying to build the next stupid app that anybody who can program can build.
One key step to me going into grad school was realizing that I could always do startups, but I could only go to grad school while I was young. (Technically you can always go, but for me, there was a very strong preference to do it "now" or do it never.) So... you can have both, but probably only if you do grad school first, not startups first.