That's great for you, but it gives you no excuse to make other people uncomfortable just because you don't think a joke is too far. And no, saying "it's just a joke" isn't an apology.
And that's where context matters! Know when to and when to not joke is key. But the subject doesn't matter IMO. Also bullying 'jokingly' is not joking. It's bullying. But true jokes can be applied on every subject.