Hacker News new | past | comments | ask | show | jobs | submit login

In communication, meaning is a collaboration between writer and reader. The writer does their best to convey something; the reader does their best to understand.

There's also the kind of meaning that scientists and researchers talk about when they extract knowledge from data. That's pretty different from communication; it's more a process of internal generation of notions and explanations that could later be conveyed in communication.

And then there's the meaning that is even more internal. E.g., reading tarot cards or tea leaves, people generate meaning out of nothing. And then there's Pareidolia: https://en.wikipedia.org/wiki/Pareidolia

If you're saying machine-generated text is meaning in the sense of that last category then sure, it's something we loosely call meaning. But it's fundamentally the same as other sorts of cleromancy [1], just more elaborate.

[1] https://en.wikipedia.org/wiki/Cleromancy




a process of internal generation of notions

How is this "knowledge extraction from data" process different?


Immanuel Kant would call the former "rendering a synthetic judgement" and David Hume termed the latter revealing the a priori knowledge. Plato would call both simply giving substance to the forms.

It seems none of this is something new. It's just that you no longer need a good education to learn about old ideas; you can come up with them on your own.


Different from what? Regular communication? Because the first involves trying to sync up two minds to have the same idea. The latter involves one mind, trying to generate an idea that ends up being useful. Useful in the George Box sense: "All models are wrong, some models are useful."


I meant how is knowledge extraction done by a language model different from knowledge extraction done by a human scientist?


Sorry, I don't understand the question.


GPT-3 does some form knowledge extraction, right?

You said above: "There's also the kind of meaning that scientists and researchers talk about when they extract knowledge from data."

So if we agree that in both cases some form of knowledge extraction is happening, I wonder how these two forms compare.


A search for "GPT-3 knowledge extraction" doesn't yield much, so I don't know what you're talking about. But as far as I'm concerned, knowledge exists in the heads of people, so I don't think we agree on that.


knowledge exists in the heads of people

How is this knowledge encoded in the heads of people? Perhaps through the choice of synapse strength and connectivity between neurons?

What is being encoded in GPT-3 weights as it reads through millions of pages of text? How is it different from tuning biological synapses?

Still don't know what I'm talking about? It's OK, neither do I :)




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: