I think you may be assuming that I am arguing against people buying handbags. I am not.
If it’s your money, spend it as you wish.
Morally speaking society should only make sure that you didn’t earn money through harmful activities (e.g mafia dons), but it shouldn’t regulate regular spending on luxury goods.
I agree that society shouldn't regulate what people spend their money on, but I'd argue it ought to limit the amount of money that any one individual can accrue.
Of course some will try to do this (not everyone). The aim is to limit their ability to do so. And one doesn't have to succeed entirely for it to be worth the effort.