There is a big difference between making 100k in salary and 100k from investments. If you have to work to put food on the table then you are definitely not the upper class. I have read about a good middle class marker recently - if you have or will have a mortgage, then you are middle class. Upper class don't need one and lower class won't get approved for one.
But come on, the fact that your career path doesn't tolerate 2 week vacations doesn't move you down the percentile ladder.
As far as your last paragraph, that's my point. If you're making six figures, you're "rich" in America, still, whether you want to accept it or not.
To think you're not is to be very, very out of touch.