Because Signal has one (set of) servers which contain all messages sent through the Signal network. With Matrix if you are communicating with only a handful of people that either self-host or otherwise don't use Matrix.org then the largest homeservers don't even have a copy of your (encrypted) message history. In order for Matrix to be equivalent to Signal in this aspect, every single Matrix user would have to use Matrix.org (which isn't the case).
