If you don't pay, it destroys your credit rating, which in many areas means you won't be able to rent a place to live from anything but a private party. I believe they could also potentially get an injunction against you and garnish your wages, but I'm not certain about that. Your college also may withhold your transcript, which can impact some employers who require that.
Lots of ways they can make your life just as miserable for not paying.
True. There are always consequences. But if you refuse to pay then you learn how to survive all of the threats you mention which is worth the cost. The point is that the vast majority of debts are legally invalid and unenforceable. They are just a con. Whether you chose to be a mark or not is up to you.
Lots of ways they can make your life just as miserable for not paying.