The only way to get the ip of someone from their Tox id is by knowing their public DHT key which they will only send you if they are your friend.
How this works is described in detail: https://github.com/irungentoo/toxcore/blob/master/docs/Preve...
also, if the message is encrypted, why not use a "torrent" of all the messages being shared and discarded after a while (no need for offline messages anyway) and each node decrypts its own as they please but relay all of them in the same stream? no way to know who could decode it. and it was delivered to everyone. much like a bitcoin transaction, but without the metadata.
it would probably be vulnerable to a dos by flooding it with bogus, expensive to try to read, requests... but will be the most reliable and private you can get
They have to be friends with you.
Note that being friends in this context means the users have added each other.