Slavery systematically destroyed the family unit of slaves because it was a risk to the power structure. If there was any indication of healthy family structures forming they would split for children and men from women.
But one wouldn't reasonably expect that to continue, no? The critics who say "slavery was a long time ago" have a point on this one. Humans getting married is very normal - if you disrupt that for a generation or two, you'd still expect it to resume. We are many generations past slavery.
The cultural forces behind monogamous family units has weakened quite a bit. It's not super surprising that they can't pick themselves up by their boot straps in short order.
It's my understanding that this one is less about the legacy of slavery per se, and more just a feature of poverty.