I suspect loneliness/isolation is the biggest issue for most.
At the moment it's great because I'm at home with three housemates/friends. But I think I'd get lonely pretty quickly if it was just me. Co-working spaces might solve this a bit (never worked in one so don't know how much people socialise).
That is what I thought a the beginning, I am very social and I will talk/listen to anyone on the street, but yet it works very well for me and I don't really feel lonely. Sometimes I even have to close slack because it is too distracting :) Also we are all a better version of ourselves when we do email, that is because we can put more brain time in what we are going to say. Working remotely gets two big thumbs up for me!
At the moment it's great because I'm at home with three housemates/friends. But I think I'd get lonely pretty quickly if it was just me. Co-working spaces might solve this a bit (never worked in one so don't know how much people socialise).