I can manage my time just fine! The problem is that I find it very demotivating to be working completely alone, and enjoy working with others around. It's a personal preference, no more complex than that.
Forcing others into the office just because you want their company is definitely a bad idea.
The way forward is finding ways to work together that the entire team is comfortable with. And employers thinking of employees more as humans and less as resources to be manipulated and squeezed. It will take compromises and growth, but it's far from impossible to achieve.
