The social pressure can be overwhelming, unless you're not super-super strong, mentally. We cannot assume the average person to be so strong.
Why is the social pressure overwhelming? Socially sometimes it seems that it's better to crawl into a cave and never be seen in your poverty, than be seen as a 40-45+ year old doing janitorial work (or other kinds of jobs most people find degrading, despite the fact that to your face they will deny it), by your friends and family. Also a lot of people can't accept the downturn in their lives. That's how you get alcoholism, depression, drug abuse. It's much harder to wake up and say: I've failed all these years, I need to start over.
If we go by anecdotes, I knew only two persons who switched careers at mid life successfully: one was a woman and the other a man and both had a spouse who brought decent money to the household and was supportive.
