If your phone number being stolen causes your savings to get drained for long enough that you run into problems making important payments like rent, taxes, car payments; That can pretty quickly spiral into even worse situations. In a world/country where many people have too few savings to go even a month without being paid, losing even that can get extremely dangerous. Not to mention the stress of such a situation alone will probably take quite a bit of your life expectancy off.
While I absolutely understand the point you're making....
At least in the United States, we also live in a society where the financial ramifications of getting shot could lead to equally bad financial outcomes (whether directly or indirectly).