We just poll all the people, including the dumb ones, and then average out the answers. The dumbs all cancel out in the end. (Yes, of course, I know it is a lot more complex than this)
The whole point of democracy is that we don't know what is the dumb or smart answer so we have these complex functions and dances to squeeze out the best one we can think of at the time. (Yes, I know that the reasons for states to turn democratic are complex)
We don't poll all of the people. We exclude everybody below 18, for example. We exclude a lot of people with criminal records. We use a multi-stage representative democracy, such that large numbers of voters basically don't count at all. (And that's before we go out of our way to draw districts specifically to exclude people).
To the degree that democracy works, it works as you say. But we systematically fail to include all of the people, so the dumbs often don't cancel out.
I remember a professor once told us “democracy might not be the best kind of government. It’s just the best we have found to organize a peaceful power transition”.
And .. to this date .. I agree .. it’s not about making the best decision.. it just allows us to come to a decision on a seemingly fair process.
But it doesn't do that. It allows a majority to do whatever it wants, including redefine the rules to guarantee its permanent majority.
Or the rules are written to avoid that, in which case the power rests with the minority who is happiest with the status quo.
It works only if most people don't want to push their advantages to the limits. But if men were angels, there would be no need of laws. The system will always fall prey to whoever is most willing to push their advantage as far as it will go.
Democracy dies when stupid people become convinced that they know best and have all the answers.