In addition to the comment you mentioned that Yale could have filled 3 identical classes from their applicant pool, I've heard quotes from Harvard admission officers saying 95% of their applicants would be capable Harvard students.
Another thing that is important to remember is that not only are some of the admission decisions arbitrary but they are differentiations among young people whose lives have been vastly shaped by their family and schooling. Whatever merit is, at the undergraduate level students are already being evaluated on resume and preparedness. This is not at all a competition on equal grounds for students across the country, let alone across the globe as elite US undergraduate admissions are increasingly a global competition.
Echoing your last point, instead of arguing that undergraduate admissions decisions are significant judgments we should be thankful that we live long lives that allow us to do great work outside of a system that at its highest levels does not have the volume or capacity to discover meaningful distinctions between applicants.
so why aren't more position made available for these applicants? Isn't the root cause of the problem one of not enough spaces for the number of applicants? If each applicant paid their own way (either through their own, or borrowed money), how come there isn't enough spaces to satisfy everyone? A more educated society is a better society.
You don't need the prestige of the Ivies to get a great education.