I think being able to tell the difference would have to be answered by solving the information loss problem. Because otherwise, every property is either dependent on or constrained only by the mass. And yes, even if they were antimatter black holes eating matter, it wouldn't matter because all the energy from the resulting annihilation would just stay in the black hole and due to mass-energy equivalence nothing would be different to the outside observer.
Couldn't the accretion disk of a black hole give us a clue though? Likely not all of the material would've been beyond the event horizon. Some of it would've stayed outside of it in the accretion disk. It's likely that the antimatter that stayed out would at some point interact with the regular matter that falls towards the black hole, but even that might give us a clue based on how energetic the accretion disk is (how much it radiates for example).
