On: We can't all use AI. Someone has to generate the training data

[still_grokking]

Sole humans can still trace every gate on a modern chip, or even the gate's components, and how exactly they work, to some explicit specification. It's completely deterministic and a human can fully understand all details. Otherwise it wouldn't be possible to debug it, or manufacture it in the first place.

OTOH we can't track or understand the results of AI. It spits out results but we can't understand how exactly it deduced those results, usually. That's the unfortunate current state actually.

So the analogy doesn't hold.

But this seems not even relevant. Because no level of technological progress will invalidated the old "garbage in, garbage out" principle. If you make a loop out of it, there's more or less only one thing to expect…

An AI would need to know how to teach itself "reasonable things". But to be able to do so it would need human level intelligence.

Growing AI is likely like growing children: Children can't teach themself in the beginning. You need to teach them some basics before they (maybe) learn how to teach themself. AI is not even near to "know" any basics. So it will need a lot of well thought out human input before it can proceed to teach itself.

[meh8881 → still_grokking]

Writing garbage, then receiving feedback on your writing, and progressing to write less garbage is a common pattern of improvement however.

The origin of the corpus matters much less than the quality of its curation.