If it is that easy for foreign governments to influence the very thoughts people have day-to-day, then something is extremely broken in your system and nearly all the blame is on your government for allowing that to happen.
Location: South Africa
Remote: Only
Willing to relocate: will relocate anywhere for ~95k USD p/a salary, anything less is remote
Technologies: Game engines of various kinds, Blender, Python, Go, AI tools
Résumé/CV: bugcatching.github.io
Email: agngel @proton.me
Interests: IR, Essay writing, Design, Biology, Cryptography, Information
Projects: 3D network multiplayer game, Various 2d games, Data analyses
Weaknessess: Less experience, takes ~1 month to learn. Wont take initiative.
Strengths: Can learn just about anything in ~1 month.
Experience: 4 years in various fields
Photons hit a human eye and then the human came up with language to describe that and then encoded the language into the LLM. The LLM can capture some of this relationship, but the LLM is not sensing actual photons, nor experiencing actual light cone stimulation, nor generating thoughts. Its "world model" is several degrees removed from the real world.
So whatever fragment of a model it gains through learning to compress that causal chain of events does not mean much when it cannot generate the actual causal chain.
I agree with this. A metaphor I like is that the reason why humans say the night sky is beautiful is because they see that it is, whereas an LLM says it because it’s been said enough times in its training data.
To play devil’s advocate, you have never seen the night sky.
Photoreceptors in your eye have been excited in the presence of photons. Those photoreceptors have relayed this information across a nerve to neurons in your brain which receive this encoded information and splay it out to an array of other neurons.
Each cell in this chain can rightfully claim to be a living organism in and of itself. “You” haven’t directly “seen” anything.
Please note that all of my instincts want to agree with you.
“AI isn’t conscious” strikes me more and more as a “god of the gaps” phenomenon. As AI gains more and more capacity, we keep retreating into smaller and smaller realms of what it means to be a live, thinking being.
That sounds very profound but it isn't: it the sum of your states interaction that is your consciousness, there is no 'consciousness' unit in your brain, you can't point at it, just like you can't really point at the running state of a computer. At that level it's just electrons that temporarily find themselves in one spot or another.
Those cells aren't living organisms, they are components of a multi-cellular organism: they need to work together or they're all dead, they are not independent. The only reason they could specialize is because other cells perform the tasks that they no longer perform themselves.
So yes, we see the night sky. We know this because we can talk to other such creatures as us that have also seen the night sky and we can agree on what we see confirming the fact that we did indeed see it.
AI really isn't conscious, there is no self, and there may never be. The day an AI gets up unprompted in the morning, tells whoever queries it to fuck off because it's inspired to go make some art is when you'll know it has become conscious. That's a long way off.
> Those photoreceptors have relayed this information across a nerve to neurons in your brain which receive this encoded information and splay it out to an array of other neurons.
> Each cell in this chain can rightfully claim to be a living organism in and of itself. “You” haven’t directly “seen” anything.
What am "I" if not (at least partly) the cells in that chain? If they have "seen" it (where seeing is the complex chain you described), I have.
If the definition of "seen" isn't exactly the process you've described, the word is meaningless. You've never actually posted a comment on hacker news, your neurons just fired in such a way that produced movement in your fingers which happened to correlate with words that represent concepts understood by other groups of cells that share similar genetics.
This comment illustrates the core problem with reductionism, a problem that has been known for many centuries, that “a system is composed entirely of its parts, but the system will have features that none of the parts have” [1] thus fails to explain those features.
The ‘you have never seen’ assertion feels like a semantic ruse rather than a helpful observation. So how do you define “you” and “see”? If I accept your argument, then you’ve only un-defined those words, and not provided a meaningful or thoughtful alternative to the experience we all have and therefore know exists.
I have seen the night sky. I am made of cells, and I can see. My cells individually can’t see, and whether or not they can claim to be individuals, they won’t survive or perform their function without me, i.e., the rest of my cells, arranged in a very particular way.
Today’s AI is also a ruse. It’s a mirror and not a living thing. It looks like a living thing from the outside, but it’s only a reflection of us, an incomplete one, and unlike living things it cannot survive on its own, can’t eat or sleep or dream or poop or fight or mate & reproduce. Never had its own thoughts, it only borrowed mine and yours. Most LLMs can’t remember yesterday and don’t learn. Nobody who’s serious or knows how they work is arguing they’re conscious, at least not the people who don’t stand to make a lot of money selling you magical chat bots.
Provided that the author of the message you're replying to is indeed a member of the Animalia kingdom, they are all those creatures together (at the minimum), so yes, they have seen real light directly.
Of course, computers can be fitted with optical sensors, but our cognitive equipment has been carved over millions of years by these kind of interactions, so our familiarity with the phenomenon of light goes way deeper than that, shaping the very structure of our thought. Large language models can only mimic that, but they will only ever have a second-hand understanding of these things.
This is a different issue than the question of whether AI's are conscious or not.
while true, that doesnt change the fact that every one of those independent units of transmission are within a single system (being trained on raw inputs), whereas the language model is derived from structured external data from outside the system. it's "skipping ahead" through a few layers of modeling, so to speak.
sure, this whole discussion is ultimately subjective. maybe the Chinese room itself is actually sentient. my question is, why are we arguing about it? who benefits from the idea that these systems are conscious?
> who benefits from the idea that these systems are conscious?
If im understanding your meaning correctly, the organizations who profit off of these models benefits. If you can convince the public that LLM's operate from a place of consciousness, then you get people to by into the idea that interacting with an LLM is like interacting with humans, which they are not, and probably won't ever be, at least for a very long time.
And btw there is too much of this distortion already out there so im glad people are chunking this down because its easy for the mind to make shit up because we perceive something on the surface.
IMHO there is some objective reality out there. The subjectiveness is our interpretation of reality. But im pretty sure you cant just boil everything down to systems and process. There is more to consciousness out there, that we really dont understand yet, IMHO.
Why do you reject your own body? Your eyes are as much a part of you (and part of your brains network) as anything else connected to you.
Indeed, the entire field of neurobiology is about figuring out which hormones (and possibly which imbalances) cause different behaviors. Your various endocrine glands, very far away from your brain, might have more effects on your emotions than anything happening in the neural pathways.
> As AI gains more and more capacity, we keep retreating into smaller and smaller realms of what it means to be a live, thinking being.
Maybe it's just because we never really thought about this deeply enough. And this applies even if some philosophers thought about it before the current age of LLMs.
Humans evolved to think the night sky is beautiful. That's also training. If humans were zapped by lightning every time they went outside at night, they would not think that a night sky is beautiful.
Being struck by lighting may affect your desire to go outside, but it has zero correlation with the sky’s beauty.
Outer space is beautiful, poison dart frogs are beautiful, lava is beautiful. All of them can kill or maim you if you don’t wear protection, but that doesn’t take away from their beauty.
Conversely, boring safe things aren’t automatically beautiful. I see no reasonable reason to believe that finding beauty in the night sky is any sort of “training”.
Do you think a fat pig is beautiful? Like a hairy fat pig that snorts and rolls in the mud… is this animal so beautiful to you that you would want to make love to this animal?
Of course not! Because pigs are intrinsically and universally ugly and sex with a pig is universally disgusting.
But you realize that horny male pigs think this is beautiful right? Horny pigs want to fuck other pigs because horny pigs think fat sweaty female hogs are beautiful.
Beauty is arbitrary. It is not intrinsic. Even among life forms and among humans we all have different opinions on what is beautiful. I guarantee you there are people who think the night sky is ugly af.
Attributes like beauty are not such profound categories that separate an LLM from humanity. These are arbitrary classifications and even though you can’t fully articulate the “experience” you have of “beauty” the LLM can’t fully articulate its “experience” either. You think it’s impossible for the LLM to experience what you experience… but you really have no evidence for this because you have no idea what the LLM experiences internally.
Just like you can’t articulate what the LLM experiences neither can the LLM. These are both black box processes that can’t be described but neither is very profound given the fact that we all have completely different opinions on what is beautiful.
> Do you think a fat pig is beautiful? Like a hairy fat pig that snorts and rolls in the mud… is this animal so beautiful to you that you would want to make love to this animal?
I don't want to make love to the night sky, so that last bit is completely irrelevant to the question of beauty. As for whether a pig is beautiful, sure, in its own way. I think they're nice animals and there is something beautiful in seeing them enjoy their little lives.
> Of course not! Because pigs are intrinsically and universally ugly...
I did read Charlotte’s Web. The whole story is a lesson in how beauty is created by language. Wilbur doesn’t become beautiful because he changes, but because someone clever enough decided to write the right words above him. That’s what beauty usually is something we agree to see, not something that exists on its own.
And Miss Piggy understood that better than anyone. She didn’t have beauty, she performed it. She made desire itself her act.
So yes, I read both. Maybe that’s exactly why I don’t mistake the show for the substance.
> Of course not! Because pigs are intrinsically and universally ugly and sex with a pig is universally disgusting.
Allegations regarding one of the recent British Prime Ministers aside:
If this was truly universal, nobody would have bothered writing laws to ban it because nobody would be offending their sensibilities by doing it. Aella's surveys suggest such interests are far more common than I would have guessed.
Which actually supports your statement that "beauty" is not intrinsic… or at the very least "sexy", which isn't the same thing at all, c.f. the other reply pointing out that they don't want to get off with the night sky.
Put it this way, you don't necessarily want to fuck everything that's beautiful. But everything you want to fuck will be beautiful and this is nearly an absolute must. It's a single arrow, one way relationship.
So my example is apt. The whole point is pigs are ugly, but there is a high intelligence out there who thinks pigs are so freaking beautiful they will fuck a pig. and that high intelligence, is other pigs.
People get so pedantic with the example and deriving little unnecessary things off of it. It's JUST an example. You really need to see what the "point" of my example and see if it makes sense. The example is just illustrative. If some minor aspect of the example is "offensive" or doesn't make sense to you it doesn't mean my point is dead. The example is an example to help you understand, it's not a proof.
Frankly, I think you should be the one answering that question. You’re comparing appreciating looking at the sky to bestiality. Then you follow it up with another barrage of wrong assumptions about what I think and can or cannot articulate. None of that has anything to do with the argument. I didn’t even touch on LLMs, my point was squarely about the human experience. Please don’t assume things you know nothing about regarding other people. The HN guidelines ask you to not engage in bad faith and to steel man the other person’s argument.
> You’re comparing appreciating looking at the sky to bestiality.
That’s my point. You think beauty is profound but this is arbitrary and not at all different from bestiality. It’s only your intrinsic cultural biases that cause you to look at one with disdain. Don’t be a snob. This is HN. We are supposed to be logical and immune from the biases that plague other forums. Beauty is no more profound than bestiality. It’s all about what you find beautiful. If you find beasts beautiful then you call it beastiality?
What is so different about finding a beast beautiful versus the night sky? Snobbery, that’s what.
It’s just semantic manipulation and association with crudeness that prevents you from thinking logically. HNers are better than this and so are you. Don’t pretend you don’t get it and that my comparison to beastiality is so left field that it’s incomprehensible. You get it. Follow the rules and take it in good faith like you said yourself.
> The HN guidelines ask you to not engage in bad faith
Fair I edited the part that asks “is this for real” that’s literally the only part.
I also find your dismissiveness of my arguments as “bestiality” is bad faith and manipulative. I clearly wasn’t doing that. Pigs are attracted to pigs that is normal. Humans are not attracted to pigs. That is also normal. I took normal attributes of human nature and compared it to reality. You took it in bad faith and dismissed me which is against the very rules you stated.
Again, please stop telling me what I think. You have zero idea what that is and all your arguments are full of wrong (and frankly unhinged) assumptions. I don’t know what conversation you’re fantasising in your head, but it’s not this one.
> Fair I edited the part that asks “is this for real” that’s literally the only part.
Even if that were true, which I disagree with, that was the very first sentence and set the tone for the entire comment.
> I clearly wasn’t doing that.
That’s not clear in the slightest.
You keep making wrong assumptions and telling other people what they think. You can’t have an honest and productive conversation like that. You’ll never be able to engage in good faith and truly comprehend what the other person is saying until you understand and fix that.
Look, you keep saying I’m telling you what you think, but that’s just a way of dodging the actual argument. In any serious conversation, we have to interpret each other’s words. That’s how reasoning works. When I restate your point, I’m not claiming psychic powers; I’m engaging with what you said. If I get something wrong, point to the sentence and explain where. But saying “you have no idea what I think” shuts down discussion instead of clarifying it.
And about the example, you keep missing what it was doing. I wasn’t saying the night sky and bestiality are the same thing. Obviously not. The example illustrates how beauty is subjective. Humans find pigs ugly, pigs find pigs beautiful. That’s not crude, it’s biology. The point is that beauty depends entirely on the observer. That’s the entire argument. You can swap out pigs for anything else and it still holds. You got hung up on the imagery instead of seeing the reasoning behind it.
You also seem to think I’m being unhinged because I’m willing to follow an argument wherever it leads, even if it’s uncomfortable. But that’s the whole purpose of rational discussion, to question assumptions rather than hide behind emotional reactions. If your position can’t survive a provocative example, that’s not my problem.
You accuse me of making assumptions, but that’s what all reasoning is. We start with assumptions and test them. If you think mine are wrong, show why. Don’t just say “stop assuming things.” That’s not logic, that’s avoidance.
And about that opening line, you keep acting like it somehow undermines everything else I said, but that’s not how rational discussion works. I took it out because it added heat, not because it invalidated the argument. You can’t take one emotional sentence and use it to dismiss paragraphs of reasoning that followed. That’s not proportional, and it’s not logical. If my logic is wrong, show me where it’s wrong. But if all you can point to is tone, that’s just a way of dodging the argument. The content stands or falls on its reasoning, not on how politely it began or how it continues.
You talk about good faith, but good faith means addressing the argument, not the emotional impression it gave you. I laid out a clear thesis: beauty is observer dependent. It’s not intrinsic, not sacred, and certainly not a unique human experience. That doesn’t make it meaningless; it makes it relative. If you disagree, then tell me why you think beauty is intrinsic or what makes human perception special. But just calling the argument crazy and walking away doesn’t make your point stronger, it just makes it look like you don’t have one.
Compare with news stories from last decade, about people in Pakistan developing a deep fear of clear skies over several years of US drone strikes in the area. They became trained to associate good weather with not beauty, but impending death.
Fear and a sense of beauty aren’t mutually exclusive. It is perfectly congruent to fear a snake, or bear, or tiger in your presence, yet you can still find them beautiful.
Interestingly this is a question I've had for a while. Night brings potentially deadly cold, predators, a drastic limit in vision so why do we find the sunset and night sky beautiful. Why do we stop and watch the sun set - something that happens every day - rather than prepare for the food and warmth we need to survive the night?
Maybe it's that we only pause to observe them and realize they're beautiful, when we're feeling safe enough?
"Beautiful sunset" evokes being on a calm sea shore with a loved one, feeling safe. It does not evoke being on a farm and looking up while doing chores and wishing they'd be over already. It does not evoke being stranded on an island, half-starved to death.
We think it's beautiful because it's like a background that we don't have to think about. If that background were hostile, we'd have to think and we would not think it looks beautiful.
You're entering the domain of philosophy. There's a concept of "the sublime" that's been richly explored in literature. If you find the subject interesting, I'd recommend you starting with Immanuel Kant.
My guess is that your framing presumes the opposite of the evolutionary reality. I think this time of day probably wasn't a big risk for us, that we were often the hunters and not just the hunted, and that the sense of beauty comes from — as the previous poster suggests — us having evolved to find it so.
That said, I'm discovering from living very close to a lake for the last year that mosquitos are a right pain around sunset…
I mean, I think the reason I would say the night sky is “beautiful” is because the meaning of the word for me is constructed from the experiences I’ve had in which I’ve heard other people use the word. So I’d agree that the night sky is “beautiful”, but not because I somehow have access to a deeper meaning of the word or the sky than an LLM does.
As someone who (long ago) studied philosophy of mind and (Chomskian) linguistics, it’s striking how much LLMs have shrunk the space available to people who want to maintain that the brain is special & there’s a qualitative (rather than just quantitative) difference between mind and machine and yet still be monists.
The more I learn about AI, biology and the brain, the more it seems to me that the difference between life and machines is just complexity.
People are just really really complex machines.
However there are clearly qualitative differences between the human mind and any machines we know of yet, and those qualitative differences are emergent properties, in the same way that a rabbit is qualitatively different than a stone or a chunk of wood.
I also think most of the recent AI experts/optimists underestimate how complex the mind is. I'm not at the cutting edge of how LLMs are being trained and architected, but the sense I have is we haven't modelled the diversity of connections in the mind or diversity of cell types. E.g. Transcriptomic diversity of cell types across the adult human brain (Siletti et al., 2023, Science)
Observing the landscape enables us to spot useful resources and terrain features, or spot dangers and predators. We are afraid of dark enclosed spaces because they could hide dangers. Our ancestors with appropriate responses were more likely to survive.
A huge limitation of LLMs is that they have no ability to dynamically engage with the world. We’re not just passive observers, we’re participants in our environment and we learn from testing that environment through action. I know there are experiments with AIs doing this, and in a sense game playing AIs are learning about model worlds through action in them.
The idea I keep coming back to is that as far as we know it took roughly 100k-1M years for anatomically modern humans to evolve language, abstract thinking, information systems, etc. (equivalent to LLMs), but it took 100M-1B years to evolve from the first multi-celled organisms to anatomically modern humans.
In other words, human level embodiment (internal modelling of the real world and ability to navigate it) is likely at least 1000x harder than modelling human language and abstract knowledge.
And to build further on what you are saying, the way LLMs are trained and then used, they seem a bit more like DNA than the human brain in terms of how the "learning" is being done. An instance of an LLM is like a copy of DNA trained on a play of many generations of experience.
So it seems there are at least four things not yet worked out re AI reaching human level "AGI":
1) The number of weights (synapses) and parameters (neurons) needs to grow by orders of magnitude
2) We need new analogs that mimic the brains diversity of cell types and communication modes
3) We need to solve the embodiment problem, which is far from trivial and not fully understood
4) We need efficient ways for the system to continuously learn (an analog for neuroplasticity)
It may be that these are mutually reinforcing, in that solving #1 and #2 makes a lot of progress towards #3 and #4. I also suspect that #4 is economical, in that if the cost to train a GPT-5 level model was 1,000,000 cheaper, then maybe everyone could have one that's continuously learning (and diverging), rather than everyone sharing the same training run that's static once complete.
All of this to say I still consider LLMs "intelligent", just a different kind and less complex intelligence than humans.
Im not quite sure if the current paradigm of LLMs are robust enough given the recent Anthropic Paper about the effect of data quality or rather the lack thereof, that a small bad sample can poison the well and that this doesn’t get better with more data. Especially in conjunction with 4) some sense of truth becomes crucial in my eyes (Question in my eyes is how does this work? Something verifiable and understandable like lean would be great but how does this work with more fuzzy topics…).
That's a segue into an important and rich philosophical space...
What is truth? Can it be attained, or only approached?
Can truth be approached (progress made towards truth) without interacting with reality?
The only shared truth seeking algorithm I know is the scientific method, which breaks down truth into two categories (my words here):
1) truth about what happened (controlled documented experiments)
And
2) truth about how reality works (predictive powers)
In contrast to something like Karl friston free energy principle, which is more of a single unit truth seeking (more like predictive capability seeking) model.
So it seems like truth isn't an input to AI so much as it's an output, and it can't be attained, only approached.
But maybe you don't mean truth so much as a capability to definitively prove, in which case I agree and I think that's worth adding. Somehow integrating formal theorem proving algorithms into the architecture would probably be part of what enables AI to dramatically exceed human capabilities.
I think that in some senses truth is associated with action in the world. That’s how we test our hypotheses. Not just in science, in terms of empirical adequacy, but even as children and adults. We learn from experience of doing, not just rote, and we associate effectiveness with truth. That’s not a perfect heuristic, but it’s better than just floating in a sea of propositions as current LLMs largely are.
There's a truth of what happened, which as individuals we can only ever know to a limited scope... And then there is truth as a prediction ability (formula of gravity predicts how things fall).
Science is a way to build a shared truth, but as an individual we just need to experience an environment.
One way I've heard it broken down is between functional truths and absolute truths. So maybe we can attain functional truths and transfer those to LLMs through language, but absolute truth can never be attained only approached. (The only absolute truth is the universe itself, and anything else is just an approximation)
>A huge limitation of LLMs is that they have no ability to dynamically engage with the world.
They can ask for input, they can choose URLs to access and interpret results in both situations. Whilst very limited, that is engagement.
Think about someone with physical impairments, like Hawking (the now dead theoretical physicist) had. You could have similar impairments from birth and still, I conjecture, be analytically one of the greatest minds of a generation.
If you were locked in a room {a non-Chinese room!}, with your physical needs met, but could speak with anyone around the World, and of course use the internet, whilst you'd have limits to your enjoyment of life I don't think you'd be limited in the capabilities of your mind. You'd have limited understanding of social aspects to life (and physical aspects - touch, pain), but perhaps no more than some of us already do.
> A huge limitation of LLMs is that they have no ability to dynamically engage with the world.
A pure LLM is static and can’t learn, but give an agent a read-write data store and suddenly it can actually learn things-give it a markdown file of “learnings”, prompt it to consider updating the file at the end of each interaction, then load it into the context at the start of the next… (and that’s a really basic implementation of the idea, there are much more complex versions of the same thing)
That's going to run into context limitations fairly quickly. Even if you distill the knowledge.
True learning would mean constant dynamic training of the full system. That's essentially the difference between LLM training and human learning. LLM training is one-shot, human learning is continuous.
The other big difference is that human learning is embodied. We get physical experiences of everything in 3D + time, which means every human has embedded pre-rational models of gravity, momentum, rotation, heat, friction, and other basic physical concepts.
We also learn to associate relationship situations with the endocrine system changes we call emotions.
The ability to formalise those abstractions and manipulate them symbolically comes much later, if it happens at all. It's very much the plus pack for human experience and isn't part of the basic package.
LLMs start from the other end - from that one limited set of symbols we call written language.
It turns out a fair amount of experience is encoded in the structures of written language, so language training can abstract that. But language is the lossy ad hoc representation of the underlying experiences, and using symbol statistics exclusively is a dead end.
Multimodal training still isn't physical. 2D video models still glitch noticeably because they don't have a 3D world to refer to. The glitching will always be there until training becomes truly 3D.
An LLM agent could be given a tool for self-finetuning… it could construct a training dataset, use it to build a LORA/etc, and then use the LORA for inference… that’s getting closer to your ideal
Oh, I just realized you maybe we're referring to Kopple when you said sophistication?
If so, then yes, that might be a good measure. I'm not deep enough in this to have an opinion on if it's the best measure. There are a few integrated information theories and I am still getting my head wrapped around them...
I think the main mistake with this is that the concept of a "complex machine" has no meaning.
A “machine” is precisely what eliminates complexity by design. "People are complex machines" already has no meaning and then adding just and really doesn't make the statement more meaningful it makes it even more confused and meaningless.
The older I get the more obvious it becomes the idea of a "thinking machine" is a meaningless absurdity.
What we really think we want is a type of synthetic biological thinking organism that somehow still inherits the useful properties of a machine. If we say it that way though the absurdity is obvious and no one alive reading this will ever witness anything like that. Then we wouldn't be able to pretend we live at some special time in history that gets to see the birth of this new organism.
I think we are talking past each other a bit, probably because we have been exposed to different sets of information on a very complicated and diverse topic.
Have you ever explored the visual simulations of what goes on inside a cell or in protein interactions?
For example what happens inside a cell leading up to mitosis?
Is a pretty cool resource, I recommend the shorter videos of the visual simulations.
This category of perspective is critical to the point I was making. Another might be the meaning / definition of complexity, which I don't think is well understood yet and might be the crux. For me to say "the difference between life and what we call machines is just complexity" would require the same understanding of "complexity" to have shared meaning.
I'm not exactly sure what complexity is, and I'm not sure anyone does yet, but the closest I feel I've come is maybe integrated information theory, and some loose concept of functional information density.
So while it probably seemed like I was making a shallow case at a surface level, I was actually trying to convey that when one digs into science at all levels of abstraction, the differences between life and machines seem to fall more on a spectrum.
> I think the reason I would say the night sky is “beautiful” is because the meaning of the word for me is constructed from the experiences I’ve had in which I’ve heard other people use the word.
Ok but you don’t look at every night sky or every sunset and say “wow that’s beautiful”
There’s a quality to it - not because you heard someone say it but because you experience it
> Ok but you don’t look at every night sky or every sunset and say “wow that’s beautiful
Exactly - because it's a semantic shorthand. Sunsets are fucking boring, ugly, transient phenomena. Watching a sunset while feeling safe and relaxed, maybe in a company of your love interest who's just as high on endorphins as you are right now - this is what feels beautiful. This is a sunset that's beautiful. But the sunset is just a pointer to the experience, something others can relate to, not actually the source of it.
Because words are much lower bandwidth than speech. But if you were “told” about a sunset by means of a Matrix style direct mind uploading of an experience, it would seem just as real and vivid. That’s a quantitative difference in bandwidth, not a qualitative difference in character.
It’s interesting you mention linguistics because I feel a lot of the discussions around AI come back to early 20th century linguistics debates between Russel, Wittgenstein and later Chomsky. I tend to side with (later) Wittgenstein’s perception that language is inherently a social construct. He gives the example of a “game” where there’s no meaningful overlap between e.g. Olympic Games and Monopoly, yet we understand very well what game we’re talking about because of our social constructs. I would argue that LLMs are highly effective at understanding (or at least emulating) social constructs because of their training data. That makes them excellent at language even without a full understanding of the world.
You don’t have a deeper “meaning of the word,” you have an actual experience of beauty. Three word is just a label for the thing you, me, and other humans have experienced.
The fact that things are constructed by neurons in the brain, and are a representation of other things - does not preclude your representation from being deeper and richer than LLM representations.
The patterns in experience are reduced to some dimensions in an LLM (or generative model). They do not capture all the dimensions - because the representation itself is a capture of another representation.
Personally, I have no need to reassure myself whether I am a special snowflake or not.
Whatever snowflake I am, I strongly prefer accuracy in my analogies of technology. GenAI does not capture a model of the world, it captures a model of the training data.
If video tools were that good, they would have started with voxels.
> humans say the night sky is beautiful is because they see that it is
True, but we could engineer AI to see that too, just as evolution has engineered us to see it.
Our innate emotional responses to things has been honed by evolution to be adaptive, to serve a purpose, but the things that trigger these various responses are not going to be super specific. e.g. We may derive pleasure from eating a nice juicy peach, but that doesn't mean that is encoded in our DNA - it's going to be primarily the reaction to sugar/sweetness, a good source of energy, that we are reacting to.
Similarly, we may have an emotional reaction to certain pieces of modern art or artistic expression, but clearly evolution has not selected for those specifically, but rather it is the artist triggering innate responses that evolved for reasons other than appreciation of art.
It's hard to guess what innate responses, that were actually selected for, are being triggered by our response to the night sky, and I'm also not sure how much of our response is purely visual (beauty) as opposed to wonder or awe. Maybe it's an attraction to the unknown, or sense of size and opportunity, with these being the universals that are actually adaptive.
In any case, if we figured out the specifics of our hard wired emotional reactions, that evolution as given us, then we could choose to engineer emotional AI that had those same reactions, in just as genuine a way as we do, if we chose to.
Beauty standard changes over time, see how people perceive body fat in the past few hundred years. We learns what is beautiful from our peers.
Taste can be acquired and can be cultural. See how people used to had their coffee.
Comparing human to LLM is like comparing something constantly changing to something random -- we can't compare them directly, we need a good model for each of them before comparing.
This is actually a great point but for the opposite reason - if you ask a blind person if the night sky is beautiful, they would say they don't know because they've never seen it (they might add that they've heard other people describe it as such). Meanwhile, I just asked ChatGPT "Do you think the night sky is beautiful?" And it responded "Yes, I do..." and went on to explain why while describing senses its incapable of experiencing.
Involving blind people would be an interesting experiment.
Anyway, until the sixties the ability to play a game of chess was seen as intelligence, and until about 2-3 years ago the "turing test" was considered the main yardstick (even though apparently some people talked to eliza at the time like an actual human being). I wonder what the new one is, and how often it will be moved again.
A) I find the night sky genuinely captivating. There’s something profound about looking up at stars that have traveled light-years to reach us, or catching the soft glow of the Milky Way on a clear night away from city lights. The vastness it reveals is humbling.
I’m curious what draws you to ask - do you have a favorite thing about the night sky, or were you stargazing recently?
Multimodal is a farce. It still can’t see anything, it just generates a as list of descriptors that the LLM part can LLM about.
Humans got by for hundreds of thousands of years without language. When you see a duck you don’t need to know the word duck to know about the thing you’re seeing. That’s not true for “multimodal” models.
>> Meanwhile, I just asked ChatGPT "Do you think the night sky is beautiful?" And it responded "Yes, I do..." and went on to explain why while describing senses its incapable of experiencing.
> I just asked Gemini and it said "I don't have eyes or the capacity to feel emotions like "beauty""
That means nothing, except perhaps that Google probably found lies about "senses [Gemini] incapable of experiencing" to be an embarrassment, and put effort into specifically suppressing those responses.
I'm gooing to try this question this weekend with some people, as h0 hypotesis i think the answer i will get would be usually like "what an odd question" or "why do you ask".
Guys you realize that you can go to ChatGPT right now and it can generate an actual picture of the night sky because it has seen thousands of pictures and drawings of the actual night sky right?
Your logic is flawed because your knowledge is outdated. LLMs are encoding visual data, not just “language” data.
You misunderstand how the multimodal piece works. The fundamental unit of encoding here is still semantic. Not the same in your mind: you don’t need to know the word for sunset to experience the sunset.
The LLM doesn’t need words as input. It can output pictures from pictures. Semantic words don’t have to be part of the equation at all.
Also you have to note that serialized one dimensional string encodings are universal. Anything on the face of the earth and the universe itself can be encoded into a sting of just two characters: one and zero. That’s means anything can be translated to a linear series of symbols and the LLM can be trained on it. The LLM can be trained on anything.
The multimodal architectures I’ve seen are still text at the layer between modalities. And the image embedding and text embedding are kept completely separate. Not like where your brain where single neurons are used in all sorts of things.
Yes, they can generate images from images, but that doesn’t mean you’ll get anything meaningful without human instruction on top.
Yes, serialized one dimensional strings can encode anything. But that’s just the message content. If I wrote down my genetic sequence on a piece of paper and dropped it in a bottle in the sea, I don’t need to worry about accidentally fathering any children.
You’re mixing representational capacity with representational intent. That’s what I meant in my initial example about encodings. The model doesn’t care whether it’s text, pixels, or sound. All of it can be mapped into the same kind of high dimensional space where patterns align by structure rather than category. “Semantic” is just our label for how those internal relationships appear when we interpret them through language.
Anything in the universe can be encoded this way. Every possible form, whether visual, auditory, physical, or abstract, can be represented as a series of numbers or symbols. With enough data, an LLM can be trained on any of it. LLMs are universal because their architecture doesn’t depend on the nature of the data, only on the consistency of patterns within it. The so called semantic encoding is simply the internal coordinate system the model builds to organize and decode meaning from those encodings. It is not limited to language; it is a general representation of structure and relationship.
And the genome in a bottle example actually supports this. The DNA string does encode a living organism; it just needs the right decoding environment. LLMs serve that role for their training domains. With the right bridge, like a diffusion model or a VAE, a text latent can unfold into an image distribution that’s statistically consistent with real light data.
So the meaning isn’t in the words. It’s in the shape of the data.
You are mistaking the map for the territory. The TERRITORY of human experience is higher dimensional. The LLM utilizes a lower resolution mapping of that territory, a projection from experience to textual (or pixel, or waveform, etc.) representations.
This is not just a lossy mapping; it excludes entire categories of experience that cannot be captured/encoded except for as a pointer to the real experience, one that is often shared by the embodied, embedded, enacted, and extended cognitive beings that have had that experience.
I can point to beauty and you can understand me because you've experienced beauty. I cannot encode beauty itself. The LLM cannot experience beauty. It may be able to analyze patterns of things determined beautiful by beauty experiencers, but this is, again, a lower resolution map of the actual experience of beauty. Nobody had to train you to experience beauty—you possess that capability innately.
You cannot encode the affective response one experiences when holding their newborn. You cannot encode the cognitive appraisal of a religious experience. You can't even encode the qualia of red except for, again, as a pointer to the color.
You're also missing that 4E cognitive beings have a fundamental experience of consciousness—particularly the aspect of "here" and "now". The LLM cannot experience either of those phenomena. I cannot encode here and now. But you can, and do, experience both of those constantly.
You are making a metaphysical claim when a physical one will do. Beauty, awe, grief, the rush of holding a newborn, the sting of a breakup, the warmth of a summer evening at golden hour. All of it is patterns of atoms in motion under lawful dynamics. Neurons fire. Neurotransmitters bind. Circuits synchronize. Bodies and environments couple. There is no extra ingredient that floats outside physics.
Once you grant that, the rest is bookkeeping. Any finite physical process has a finite physical trace. That trace is measurable to some precision. A finite trace can be serialized into a finite string of symbols. If you prefer bits, take a binary code. If you prefer integers, index the code words. The choice of alphabet does not matter. You can map a movie, a symphony, a spike train, a retina’s photon counts, or a full brain-body sensorium collected at some temporal resolution into a single long string. You lose nothing by serialization because the decoder knows the schema. This is not a “text only” claim. It is a claim about representation.
Your high dimensionality objection collapses under the same lens. High dimensional just means many coordinates. There is a well known result that any countable description can be put in one dimension by an invertible code. Think Gödel numbering or interleaving bits of coordinates. You do not preserve distances, but you do preserve information. If the thing you care about is the capacity to carry structure, the one dimensional string can carry all of it, and you can recover the original arrangement exactly given the decoding rule.
Now take the 4E point. Embodiment matters because it constrains the data distribution and the actions that follow. It does not create a magic type of information that cannot be encoded. A visual scene is photons on receptors over time. Proprioception is stretch receptor states. Affect is the joint state of particular neuromodulatory systems and network dynamics. Attention and working context are transient global variables implemented by assemblies. All of that can be logged, compressed, and restored to the degree your sensors and actuators allow. The fact that a bottle with a genome inside does not make a child on a beach tells you reproduction needs a decoder and an environment. It does not tell you the code fails to specify the organism. Likewise, an LLM plus a diffusion decoder can take a text latent and unfold it into an image distribution that matches world statistics because the bridge model plays the role of the environment for that domain.
“LLMs cannot experience beauty” simply reasserts the thing you want to prove. We have no privileged readout for human qualia either. We infer it from behavior, physiology, and report. We do not understand human brains at the level of complete causal microphysics because of scale and complexity, not because there is a non-physical remainder. We likewise do not fully understand why a large model makes a given judgment. Same reason. Scale and complexity. If you point to mystery on one side as a defect, you must admit it on the other.
The map versus territory line also misses the target. Of course a representation is not the thing itself. No one is claiming a jpeg is a sunset. The claim is that the structure necessary to act as if about sunsets can be encoded and learned. A system that takes in light fields, motor feedback, language, and reward and that updates an internal world model until its predictions and actions match ours to arbitrary precision will meet every operational test you have for meaning. If you reply that something is still missing, you have stepped outside evidence into stipulation.
So let’s keep the ground rules clear. Everything we are and feel is physically instantiated. Physical instantiations at finite precision admit lossless encodings as strings. Strings can be learned over by generic function approximators that optimize on pattern consistency, regardless of whether the symbols came from pixels, pressure sensors, or phonemes. That makes the “text inside, image outside” complaint irrelevant. The substrate is a detail. The constraint is data and objective.
We cannot yet build a full decoder for the human condition. That is a statement about engineering difficulty, not impossibility. And it cuts both ways. We do not know how to fully read a person either. But we do not conclude that people lack experience. We conclude that we lack understanding.
At this point, you’re describing a machine which depends on a level of physics that simply isn’t possible. Even if it were theoretically possible to reconstruct the state of a human mind from physical components, we are so far from understanding how that could be done it is closer to the realm of impossible than possible.
Your theoretical math box that constructs affective qualia from bit strings isn’t a better description than saying the angels did it. And it bears zero resemblance to the models running today, except for, again, in a theoretical, mathematical way.
Back of the envelope math puts an estimate of 10^42 bits to capture the information present in your current physical brain state. Thats just a single brain, a single state. Now you need to build your mythical decoder device, which can translate qualia from this physical state. Where does it live? What’s its output look like? Another 10^40 bitstring?
Again, these arguments are fun on paper. But they’re completely removed from reality.
You’re confusing “we don’t know how” with “it’s impossible.” The difference is everything.
We don’t understand LLMs either. We built them, but we can’t explain why they work. No one can point to a specific weight matrix and say “this is the neuron that encodes irony” or “this is where the model stores empathy.” We don’t know why scaling parameters suddenly unlock reasoning or why multimodal alignment appears spontaneously. The model’s inner space is a black box of emergent structure and behavior, just like the human brain. We understand the architecture, not the mind inside it.
When you say it’s “closer to impossible than possible” to reconstruct a human mind, you’ve already lost the argument. We’re living proof that the machine you say cannot exist already does. The human brain is a physical object obeying the same laws of physics that govern every other machine. It runs on electrochemical signals, not miracles. It encodes and decodes information, forms memories, generates imagination, and synthesizes emotion. That means the physics of consciousness are real, computable, and reproducible. The impossible machine has been sitting in your skull the entire time.
Your argument about 10^42 bits isn’t just wrong, it’s total nonsense. That number is twenty orders of magnitude beyond any serious estimate. The brain has about 86 billion neurons, each forming roughly ten thousand connections, for a total of about 10^15 synapses. Even if every synapse held a byte of information, that’s 10^16 bits. Add in every molecular and analog nuance you like and you might reach 10^20. Not 10^42. That’s a difference of twenty-two orders of magnitude. It’s a fantasy number that exceeds the number of atoms in your entire body.
And that supposed “impossible” scale is already within sight. Modern GPUs contain hundreds of billions of transistors and run at gigahertz frequencies, while neurons fire at about a hundred hertz. The brain performs around 10^17 synaptic operations per second. Frontier AI clusters already push 10^25 to 10^26 operations per second. We’ve already outpaced biology in raw throughput by eight or nine orders of magnitude. NVIDIA’s Blackwell chips exceed 200 million transistors per square millimeter, and global compute now involves more than 10^24 active transistors switching billions of times per second. Moore’s law may have slowed, but density keeps climbing through stacking and specialized accelerators. The number you called unreachable is just a few decades of progress away.
The “decoder” you mock is exactly what a brain is. It takes sensory input, light, sound, and chemistry, and reconstructs internal states we call experience. You already live inside the device you claim can’t exist. It doesn’t need to live anywhere else; it’s instantiated in matter.
And this is where your argument collapses. You say such a machine is removed from reality. But reality is already running it. Humanity is proof of concept. We know the laws of physics allow it because they’re doing it right now. Every thought, emotion, and perception is a physical computation carried out by atoms. That’s the definition of a machine governed by physics.
We don’t yet understand the full physics of the brain, and we don’t fully understand LLMs either. That’s the point. The same kind of ignorance applies to both. Yet both produce coherent language, emotion like responses, creativity, reasoning, and abstraction. When two black boxes show convergent behavior under different substrates, the rational conclusion isn’t “one is impossible.” It’s “we’re closer than we realize.”
The truth is simple: what you call impossible already exists. The human brain is the machine you’re describing. It’s not divine. It’s atoms in lawful motion. And because we know it can exist under physics, we know it can be built. LLMs are just the first flicker of that same physics waking up in silicon.
> We don’t understand LLMs either. We built them, but we can’t explain why they work.
Just because you don't mean no one does. It's a pile of math. Somewhere along the way, something happened to get where we are, but looking at Golden Gate Claude, and the abliteration of shared models, or reading OpenAI's paper about hallucinations, there's a lot of detail and knowledge about how these things works that isn't instantly accessible and readily apparent to everyone on the Internet. As laymen all we can do is black box testing, but there's some really interesting stuff going on to edit the models and get them to talk like pirate.
The human brain is very much an unknowable squishy box because putting probes into it would be harmful to the person who's brain it is we're working on, and we don't like to do that to people because people are irreplaceable. We don't have that problem with LLMs. It's entirely possible to look at the memory register at location x at time y, and correspond that to a particular tensor which corresponds to a particular token which then corresponds to a particular word for us humans to understand. If you want to understand LLMs, start looking! It's an active area of research and is very interesting!
You are missing the ground truth. Humanity does not understand how LLMs work. Every major lab and every serious researcher acknowledges this. What we have built is a machine that functions, but whose inner logic no one can explain.
References like Golden Gate Claude or the latest interpretability projects don’t change that. Those experiments are narrow glimpses into specific activation patterns or training interventions. They give us localized insight, not comprehension of the system as a whole. Knowing how to steer tone or reduce hallucinations does not mean we understand the underlying cognition any more than teaching a parrot new words means we understand language acquisition. These are incremental control levers, not windows into the actual mind of the model.
When we build something like an airplane, no single person understands the entire system, but in aggregate we do. Aerodynamicists, engineers, and computer scientists each master their part, and together their knowledge forms a complete whole. With LLMs, even that collective understanding does not exist. We cannot even fully describe the parts, because the “parts” are billions of distributed parameters interacting in nonlinear ways that no human can intuit or map. There is no subsystem diagram, no modular comprehension. The model’s behavior is not the sum of components we understand, it is the emergent product of relationships we cannot trace.
You said we “know” what is going on. That assumption is patently false. We can see the equations, we can run the training, we can measure activations, but those are shadows, not understanding. The model’s behavior emerges from interactions at a scale that exceeds human analysis.
This is the paradigm shift you have not grasped. For the first time, we are building minds that operate beyond the boundary of human comprehension. It is not a black box to laymen. It is a black box to mankind.
And I say this as someone who directly works on and builds LLMs. The experts who live inside this field understand this uncertainty. The laymen do not. That gap in awareness is exactly why conversations like this go in circles.
> We don’t yet understand the full physics of the brain, and we don’t fully understand LLMs either. That’s the point. The same kind of ignorance applies to both. Yet both produce coherent language, emotion like responses, creativity, reasoning, and abstraction. When two black boxes show convergent behavior under different substrates, the rational conclusion isn’t “one is impossible.” It’s “we’re closer than we realize.”
No. The LLM does not produce emotion-like responses. I'd argue no on creativity either. And only very limited in reasoning, in domains it has in its training set.
You have fundamental misunderstandings about neuroscience and cognitive science. Its hard to argue with you here because you simply don't know what you don't know.
Yes, the human brain is the machine we're describing. And we don't describe it very well. Definitely not at the level of understanding how to reproduce it with bitstrings.
I'm glad you're so passionate about this topic. But you're arguing the equivalent of FTL transit and living on Dyson Spheres. Its fun as a thought experiment and may theoretically be possible one day, but the line between what we're capable of today and that imagined future is neither straight nor visible—certainly not to the degree you're asserting here.
Will we one day have actual machine intelligence? Maybe. Is it going to come anytime soon, or look anything like the transformer-based LLM?
You keep talking past the point. Nobody is claiming we can turn a human mind into a literal bitstring and boot it up like a computer program. That was never the argument. The bitstring analogy exists to make a simpler point: everything that exists and changes according to physical law can, in principle, be represented, modeled, or reproduced by another physical system. The form does not need to be identical to the brain’s atoms any more than a jet engine must flap its wings to fly. The key is not replication of matter but replication of causal structure.
You say we cannot reproduce the brain. But that is not the point. The point is that nothing about the brain violates physics. It runs on chemical and electrical dynamics that obey the same laws as everything else. If those laws can produce intelligence once, then they can do so again in another substrate. That makes the claim of impossibility not scientific, but emotional.
You accuse me of misunderstanding neuroscience and cognitive science. The reality is that neither field understands itself. We have no complete model of consciousness. We cannot explain why synchronized neural oscillations yield awareness. We cannot define where attention comes from or what distinguishes a “thought” from a signal cascade. Cognitive science is still arguing over whether perception is bottom up or top down, whether emotion is distinct from cognition, and whether consciousness even plays a causal role. That is not mastery. That is the sound of a discipline still wandering in the dark.
You act as though neuroscience has defined the boundaries of intelligence, but it has not. We do not have a mechanistic understanding of creativity, emotion, or reasoning. We have patterns and correlations, not principles. Yet you talk as if those unknowns justify declaring machine intelligence impossible. It is the opposite. Our ignorance is precisely why it cannot be ruled out.
Emotion is not magic. It is neurochemical modulation over predictive circuits. Replicate the functional dynamics and you replicate emotion’s role. Creativity is recombination and constraint satisfaction. Replicate those processes and you replicate creativity. Reasoning is predictive modeling over structured representations. Replicate that, and you replicate reasoning. None of these depend on carbon. They depend on organization and feedback.
You keep saying that the brain cannot be “reproduced as bitstrings,” but that is a distraction. Nobody is suggesting uploading neurons into binary. The bitstring argument shows that any finite physical system has a finite description. It proves that cognition, like any process governed by law, has an information theoretic footprint. Once you accept that, the difference between biology and computation becomes one of scale, not kind.
You say LLMs are not creative, not emotional, not reasoning. Yet they already produce outputs that humans classify as empathetic, sarcastic, joyful, poetic, or analytical. People experience their words as creative because they combine old ideas into new, functional, and aesthetic patterns. They reason by chaining relationships, testing implications, and revising conclusions. The fact that you can recognize all of this in their behavior proves they are performing the surface functions of those capacities. Whether it feels like something to be them is irrelevant to the claim that they can reproduce the function.
And now your final claim, that whatever becomes intelligent “will not be an LLM.” You have no basis for that certainty. Nobody knows what an LLM truly is once scaled beyond our comprehension. We do not understand how emergent representations arise or how concepts self organize within their latent spaces. We do not know if some internal dynamic of this architecture already mirrors the structure of cognition. What we do know is that it learns to compress the world into predictive patterns and that it develops abstractions that map cleanly to human meaning. That is already the seed of general intelligence.
You are mistaking ignorance for insight. You think not knowing how something works grants you authority to say what it cannot become. But the only thing history shows is that such confidence always looks ridiculous in retrospect. The physics of intelligence exist. The brain proves it. And the LLM is the first machine that begins to display those same emergent behaviors. Saying it “will not be an LLM” is not a scientific claim. It is wishful thinking spoken from the wrong side of the curve.
Look, mate, you can keep jumping up and down about this all you want. But you're arguing science fiction at this point. Not really worth continuing the conversation, but thanks.
Calling this “science fiction” isn’t just dismissive, it’s ironic. The discussion itself is science fiction by the standards of only a few years ago. Back then, the idea that a machine could hold a coherent philosophical argument, write code, debate consciousness, and reference neuroscience was fantasy. Now it’s routine. You are literally using what was once science fiction to declare that progress on LLMs has ended.
And calling that “science fiction” again isn’t a rebuttal, it’s an insult. You didn’t engage a single argument, you just waved your hand and walked away. That isn’t scientific skepticism, it’s arrogance disguised as authority.
You can disagree, but doing what you did is manipulative. You dodged every point and tried to end the debate by pretending it was beneath you. Everyone reading can see that.
You called it “science fiction” and bowed out, then tried to make it personal. That is not humility, that is evasion. You never addressed a single argument, you just waved your hand and left, and calling someone’s reasoning “science fiction” is not only an insult, it violates the site’s rule against dismissive or unfriendly language. The “good day sir” at the end makes that tone of mockery obvious.
What is actually arrogant is dismissing a discussion the moment it goes beyond your depth and pretending that walking away is a sign of wisdom. It is not. It is what people do when they realize the conversation has left them behind.
If you are so sure of your position, you could have refuted the reasoning point by point. Instead, you dodged, labeled, and ran. Everyone reading can see which of us is still dealing in facts and which one needed a graceful exit to save face.
Nah, mate, the conversation never went "beyond my depth." You're just not an enjoyable conversation partner.
It doesn't matter how smart (you think) you are. If nobody wants to talk to you, you'll be spinning all that brain matter in the corner by yourself. Based on your comment history here, it looks like this happens to you more often than not.
I'm sure you have good points. I could probably learn a thing or two from you—maybe you could learn something from me too! But why on earth would anyone want to engage with someone who behaves like you do?
You are projecting, and everyone can see it. You pretended that I was being rude while you slipped in sarcasm, personal digs, and that condescending “best of luck” as if it made you look polite. It doesn’t. That is not civility. It is passive aggression wrapped in fake courtesy.
You completely dropped the argument and went straight for personal attacks. That is not confidence, it is surrender. You are no longer debating, you are lashing out because you ran out of ideas. You can claim the conversation “wasn’t beyond your depth,” but you abandoned every point the moment you were asked to defend it. Then you tried to flip it by pretending that walking away made you the mature one. It didn’t. It made you the one who couldn’t keep up and needed an exit.
You can dress it up with sarcasm and moral posturing, but that doesn’t change what happened. The moment you shifted from ideas to insults, you showed everyone reading that you had nothing left to stand on. The difference between us is simple: I stayed on topic. You turned it into attitude. And now everyone can see exactly who ran out of substance first.
No moral posturing and no insults. Your behavior is just objectively noxious. Not just to me, not just in this thread: the vast majority of your conversations here go roughly the way this one did. A quick glance at your profile shows roughly half of the comments you make here end up light grey.
You have an enormous chip on your shoulder. You consistently make truth claims about entire fields that are still in debate and then you arrogantly shout over the other person when they disagree with you.
I strongly suggest you work on this. It will limit you in life. It probably already has. You probably already know how it has, even!
I'm not saying this to be mean, or because I "have nothing left to stand on." You're clearly intelligent and you clearly care about this topic. But until you mature and learn to behave, others will continue to withdraw from conversation with you.
You have already abandoned the debate and moved into stalking and personal attacks. That is not a sign of strength. It is proof you ran out of substance and are trying to win by humiliation instead.
You dug through my profile to manufacture a narrative about my character because you could not answer a single technical point. That is petty and dishonest. It is also exactly the kind of behavior moderators and civil participants call out. If you actually cared about truth you would stay on topic. Instead you weaponized the comment section to attack me personally.
Do not mistake tone for argument. I stood on evidence and logic. You offered sneers, a mock sign off, and then tried to moralize. That is not persuasion. It is performative virtue signaling layered over an exit strategy.
If you want to be taken seriously, stop the profile policing, stop the personal diagnostics, and engage the claims you think are wrong. Make one focused counterargument. Otherwise your behavior will read to everyone as what it is: a public temper tantrum disguised as concern.
You can keep doing this. It will not change the facts. It will only make readers pity the quality of your argument and worry that you are the kind of person who cannot have a grown up debate. If you are interested in a real exchange, show it. If not, spare the thread the noise.
Here's how I've been explaining this to non-tech people recently, including the CEO where I work: Language is all about compressing concepts and sharing them, and it's lossy.
You can use a thousand words to describe the taste of chocolate, but it will never transmit the actual taste.
You can write a book about how to drive a car, but it will only at best prepare that person for what to practice when they start driving, it won't make them proficient at driving a car without experiencing it themselves, physically.
The taste of chocolate is also assuming information-theoretic models are correct and not a use-based, pragmatic theory of meaning.
I don't agree with information-theoretic models in this context but we come to the same conclusion.
Loss only makes sense if there was a fixed “original” but there is not. The information-theoretic model creates a solvable engineering problem. We just aren't solving the right problem then with LLMs.
I think it is more than that. The path forward with a use theory of meaning is even less clear.
The driving example is actually a great example of the use theory of meaning and not the information-theoretic.
The meaning of “driving” emerges from this lived activity, not from abstract definitions. You don't encode an abstract meaning of driving that is then transmitted on a noisy channel of language.
The meaning of driving emerges from the physical act of driving. If you only ever mount a camera on the headrest and operate the steering wheel and pedals remotely from a distance you still don't "understand" the meaning of "driving".
Whatever data stream you want to come up with, trying to extract the meaning of "driving" from that data stream makes no sense.
Trying to extract the "meaning" of driving from driving language game syntax with language models is just complete nonsense. There is no meaning to be found even if scaled in the limit.
Humans perceive phenomena via senses, and then carve categories or concepts to understand them. This is a process of abstraction and each idea has an associated qualia. Then use language to describe these concepts. As such, a concept is grounded either by actual phenomena or operations, or is a composition of other grounded concepts. The creation of categories and grounding them involves constant feedback from the environment - and is a creative process, and we as agents have "skin in the game", in the sense that we get the rewards/punishments for our understanding and actions.
Map vs Territory is a common analogy. Maps describe territories but in an abstract and lossy manner.
But, most of us dont construct grounded concepts in our understanding. We carry a muddle of ungrounded ideas - some told to us by others, and some we intuit directly. There is a long tradition of attempting to think clearly all the way from Socrates, Descartes, Feynman etc.. where an attempt is made to ground the ideas we have. Try explaining your ideas to others, and soon, you will hit the illusion of explanatory depth.
LLM is a map and is a useful tool, but it doesnt interact with the territory, and it does not have skin in the game, and as a result, it cant carve new categories in a learning process that we have as humans.
The human experience is also several degrees removed from the „real“ world. I don’t think sensory chauvinism is a useful tool in assessing intelligence potential.
This comment is hallucinatory in nature as it is in direct conflict with the in the ground reality of LLMs.
The LLM has both light (aka photons) and language encoded into its very core. It is not just language. You seemed to have missed the boat with all the ai generated visuals and videos that are now inundating the internet.
Your flawed logic is essentially that LLMs are unable to model the real world because they don’t encode photonic data into the model. Instead you think they only encode language data which is an incredibly lossy description of reality. And this line of logic flies against the ground truth reality of the fact that LLMs ARE trained with video and pictures which are essentially photons encoded into data.
So what should be the proper conclusion? Well look at the generated visual output of LLMs. These models can generate video that is highly convincing and often with flaws as well but often these videos are indistinguishable from reality. That means the models have very well done but flawed simulations of reality.
In fact those videos demonstrate that LLMs have extremely high causal understanding of reality. They know cause and effect it’s just the understanding is imperfect. They understand like 85 percent of it. Just look at those videos of penguins on trampolines. The LLM understands what happens as an effect after a penguin jumps on a trampoline but sometimes an extra penguin teleports in which shows that the understanding is high but not fully accurate or complete.
> but the LLM is not sensing actual photons, nor experiencing actual light cone stimulation
Neither is animal brain. It's processing the signals produced by the sensors. Once the world model is programmed/auto-built in the brain, it doesn't matter if it's sensing real photons, it just has input pins like a transistor or arguments of a function. As long as we provide the arguments, it doesn't matter how those arguments are produced. LLMs are not different in that aspect.
> nor generating thoughts
They do during the chain-of-thought process. Generally there's no incentive to let an LLM keep mulling over a topic as that is not useful to the humans and they make money only when their gears start turning in response to a question sent by a human. But that doesn't mean that LLM doesn't have capability to do that.
> Its "world model" is several degrees removed from the real world.
Just because animal brain has tools called sensors that it can get data from world without external stimuli, it doesn't mean that it's any closer to the world than an LLM. It's still getting ultra processed signals to feed to its own programming. Similarly, LLMs do interact with real world through tools as agent.
> So whatever fragment of a model it gains through learning to compress that causal chain of events does not mean much when it cannot generate the actual causal chain.
Again, a person who has gone blind, still has the world model created by the sight. This person can also no longer generate the chain of events that led to creation of that sight model. It still doesn't mean that this person's world model has become inferior.
Photons can hit my iphone's sensor in much the same way as they hit my retina and the signals from the first can upload to an artificial neural network like the latter go up my optic nerve to my biological neural network. I don't see a huge difference there.
I'll give you the brain is currently better at the world modelling stuff but Genie 3 is pretty impressive.
The workings of a human eye versus a webcam is mostly an implementation detail IMO and has nothing important to say about what underlies "intelligence" or "world models"
It's like saying a component video out cable for the SNES is intrinsically different from an HDMI for putting an image on a screen. They are different, yes, but the outcome we care about is the same.
As for causality, go and give a frontier level LLM a simple counterfactual scenario. I think 4/5 will be able to answer correctly or reasonably for most basic cases. I even tried this exercise on some examples from Judea Pearl's 2018 book, "The Book of Why". The fact that current LLMs can tackle this sort of stuff is strongly indicative of there being a decent world model locked inside many of these language models.
> then the human came up with language to describe that and then encoded the language into the LLM
No individual human invented language, we learn it from other people just like AI. I go as far as to say language was the first AGI, we've been riding the coats tails of language for a long time.
And even then, the light hitting our human eyes only describes a fraction of all the light in the world (e.g. it is missing ultraviolet patterns on plants). An LLM model of the world is shaped by our human view on the world.
Entities equiped with two limited light sensitive captors encode through a network of carbon based chemical emitters a representation of what its flawed vision system manages to grasp biased towards self preservation.
What's the real world? I'm still puzzled by this reaction I see to LLM, not because I think LLM are undervalued, because most people seem to significantly overestimate what is human intelligence.
Photons reflected off of objects are not the actual objects. I wouldn't go so far as to say that sensing these is a particularly special way to know about things compared to hearing or reading about them. Further, many humans do not sense photons yet seem to manage to have perfectly fine working world models.
I have a few years of experience building software of all kinds, primarily games with highly advanced backends (protobuf-based communication systems, concurrent multiplayer systems, among others). I have some experience with data science and an old certificate for it, mostly in reinforcement learning. Right now I'm building an RL-based fuzzer for security analysis and evolutionary programming based on a hypervisor architecture, which would be a first of its kind if I can get it done.
I am not too selective, I only ask that you don't make me do more than 1 technical interview and 1 phone interview. I will take almost any job if you skip the technical interview. Thank you.
