I can't remember who I heard talk about this, but scientists have considered this. I think there was a good reason for why it doesn't seem to match observations.
If a significant portion of dark matter was made of these we would see a lot more gravitational lens distorions of distant objectes. There are further hard limits on how much baryonic dark matter there can be from big bang nuceleo synthesis. I think that would also put limits on contributions from lone black hole contribution.
