The degree of inbreeding depression, should itself be a function of the mating patterns. After a long period of inbreeding the genetics themselves may start to develop robustness against it. For instance genes that are only beneficial in heterozygotes would be pruned out. E.g. https://en.wikipedia.org/wiki/Sickle_cell_disease#Genetics
Inbreeding in humans is pretty political, as incest has been used traditionally to keep wealth in the extended family. Inter-marriage on the other hand has been used to form alliances.