Laws. Culture. Press. These things stand in the way of trying to be public while saying you won't act like a public company (by being a b-corp for example).
Public companies have functionally modular governance, which means if I don't like you (manager) and your ideas, my friends and I can buy enough governance units until you have to listen to us. It's not quite that straightforward but it's pretty much how it works.
In a way, it's similar to venture—you can try to escape the gravity of venture norms but to do that you need viral + network effect growth.
In the Market, you can try to escape the gravity of public company norms but to do that you need Amazon level revenue growth and the temerity of Jeff Bezos. Even then its an outside bet.
Rabois says startups should go public and I suspect his approach to the problem is he believes (im armchairing here) that the problem of founders hiding mistakes / making intellectually lazy decisions / not serving the financial goals / creating weird preference structures trying to raise money at scale from funds >> public market shareholders killing a geese for next quarters egg.