Wearing headphones every day will permanently damage your hearing. If you really want noise, put on headphones. Respectful quiet should be the default.
It's not quite so absolute - it has to do with what kind of headphones, how loud, and for how long. The problem is that people tend to use crappy earbuds, crank the volume, and listen for hours on end.
If you instead use a good pair of over-the-ear headphones so that you can get the sound you want at a low volume and without the assistance of any active noise cancellation, it's not quite the same story. Probably still shouldn't be doing it all day just to be on the safe side, but from what I've been able to find in the literature it look like it's probably fine.
The biggest factor appears to be the level of sound isolation before you play anything, the better the isolation the lower the volume you can comfortably listen at.
I don't use expensive headphones (mostly in-ears as they are more comfortable) but I went through a few different sets til I found some that worked like ear plugs.
