A lot of countries require require registering the phone number to your name.
Relying on phone numbers as uniqie identifiers especially in "crypto" hipster apps (Signal) is stupid bordering on malicious
If said malicious actors pwn the phone of the person you were talking to, suddenly they have a pretty good way of mapping a contact called "My Best Friend" to a human through billing records.
Or even easier, they type the phone number into Google and find that the Syrian dissident they've just arrested has been corresponding with the NYTimes or BBC.
If they know only that they are talking to firstname.lastname@example.org they could, uh, get Google to release their IP address. Google are fairly unlikely to honour a legal demand for disclosure from Libya or North Korea or some other tyrannical/fucked-up hellhole.
I like Signal, but I'm not totally sure about the threat model.