Right? It's such a foreign form of intelligence to me. I think the paper "What is it like to be a bat" by Thomas Nagel made me realize that I can't even imagine what it's like to be my next door neighbor, let alone a being that has senses that differ from mine. Helen Keller's mind must work in a greatly different way than yours or mine. When I think, it's in English. I visualize things. Smell, touch and taste are never really involved. It's like they are the lesser of senses and yet that's all she had. It's incredible.
Andy Weir in Project Hail Mary and Adrian Tchaikovsky in Children of [Time|Ruin|Memory] do a great job of describing what other forms of consciousness might be like, but still falls flat, I only really think in sight and sound.
Blindsight by Peter Watts also discusses what can be intelligent but not conscious. In the current hypefest of LLMs it’s interesting to consider that they may be similar.
I was thinking the same. if there's anything that is what it is like to be an LLM (and I'm not saying that there is - in fact, I doubt it, while supposing that it is a possibility for future machines) I suspect it would be like this, but more so, and inverted: while Keller had some experience of an external world but no experience of language, the entire universe for an LLM is language, without any obvious way to suppose that this language is about an external world.
I think that LLMs might go through the reverse journey, being fluent in tokens (words-ish) and working backwards towards the physical reality we all inhabit.
i think the "problem" here is that for all of human history we have always been able to use mastery of language as a signal for intelligence and competence, of which LLMs are neither. it's possible this is even instinctual it's so ingrained in our concepts of "other minds". so we're going to have to get used to the fact that just using language well isn't enough to prove intelligence, certainly not consciousness.
which then begs the the question, what is the magic ingredient, on top of use of language, that we have that bestows these qualities?
and also the observation that whatever this ingredient is, it must be very difficult to measure or prove which is maybe why we stuck with the crude, but easy to wield, "use-of-language" test for so long.
"We do not like annoying cousins." Yes, exactly. The, uh, confident fluency of LLM responses, which can at the same time contradict what was said earlier, reminded me exactly of that. I don't know if you've ever met one of those glib psychopaths, but they have this characteristic of non-content communication, where it feels like words are being arranged for you, like someone composing a song using words from a language they do not know. See also: "you're talking a lot, but you're not saying anything."
Hm. The contradictions specifically are a thing I notice in humans that I think are entirely normal[0]. But the early LLMs with the shorter context windows, those reminded me of my mum's Alzheimer's.
That said, your analogy may well be perfect, as they are learning to people-please and to simulate things they (hopefully) don't actually experience.
(Not that it changes your point, but isn't that Machiavellian rather than psychopathic?)
[0] one of many reasons why I disagree with Wittgenstein about:
> If there were a verb meaning 'to believe falsely', it would not have any significant first person, present indicative.
Just because it's logically correct, doesn't mean humans think like that.
The part that really gets ME about that thought, is that those glib psychopaths/sociopaths fill an important role in human society, generally as leaders. I'm sure we can all think of some prominent political figures who are very good at arranging words to get their audience excited, but have a tenuous connection to fact (at best). Actually factual content seems almost irrelevant to their ability to lead, or to their followers' desire to follow.
If that's the function which we can now automate at scale, it's not the jobs the machines will ultimately take; it's the leadership.
I don't think it's that strange. My thoughts and my physical sensations are separate, imaging a different body different senses isn't that much of a stretch. I speak English but I don't think in it, thoughts don't have a language.
I think that this is false, as in intersubjectively not true for the human experience. First, because our physical state has a huge influence on our thoughts, not just their content, but direction, "color."
Secondly, and more importantly, while some thoughts may not have a language (image memories, mental maps), others certainly do, they're narrative. I only speak two languages but well enough (English is my second language) that I can think in both, and often come to a point where I have to decide which it will be for this train of thought.
Shape rotators vs wordcels distinction strikes again, I guess.
Having no inner voice, imagery, or whatever seems to be poorer rather than richer experience to me. I don't think the existence of deaf people invalidates the importance of music to human experience.
I don't think a deaf person's inability to listen to music with their ears makes them incapable of depth and clarity of thought, no.
I don't think people who aren't hard of hearing necessarily have particularly deep or clear thoughts simply because they listen to music with their ears either. It's very easy to confuse correlation with causation.
(I've specified "with their ears" because deaf people can perceive music through other means than the cochlea + cochlear nerve.)
Nearly every post that uses exclamation marks like this is off-putting. Fake enthusiasm is creepy. There is no way you are enthusiastic about people having no inner voice.
Of course there is. Maybe they are one of those people. I know multiple people who say they have no inner voice the way I experience it and I don’t get it, but yes they are enthusiastic about saying that they can still think perfectly well!
Based on the fact that people speaking different languages can lack basic abstract concepts or reason about them very differently, I think thoughts do have a language or at least often follow a language.
A quote as a sample:
So let me tell you about some of my favorite examples. I'll start with an example from an Aboriginal community in Australia that I had the chance to work with. These are the Kuuk Thaayorre people. They live in Pormpuraaw at the very west edge of Cape York. What's cool about Kuuk Thaayorre is, in Kuuk Thaayorre, they don't use words like "left" and "right," and instead, everything is in cardinal directions: north, south, east and west. And when I say everything, I really mean everything. You would say something like, "Oh, there's an ant on your southwest leg." Or, "Move your cup to the north-northeast a little bit." In fact, the way that you say "hello" in Kuuk Thaayorre is you say, "Which way are you going?" And the answer should be, "North-northeast in the far distance. How about you?"
I feel like my grasp of language allows some very complex thoughts, but I often wonder if it is limiting. I seem nearly unable to think without forming phrases in my head, and even if I anticipate the conclusion I feel the need to go through the whole sentence. I know there are people with all their senses intact without any internal monologue, but mine is very much in charge. Rigorous exercise or flow state seems able to quiet it for a bit.
> I seem nearly unable to think without forming phrases in my head, and even if I anticipate the conclusion I feel the need to go through the whole sentence.
I try this ever so often and can’t get a hold of it. It feels like I know what the final sentence will be, like it’s shape, in a way, before my narrator has read it, but he needs to read it for the meaning to materialise, to commit to my reasoning state. Every time I think just how much faster I would be thinking if I could get rid of the monologue somehow.
And then I notice that thinking happens very fast, and that the perceived speaking speed of the narrator probably doesn’t correlate with the time it would take me to actually spell things out loud, my brain only pretends it’s way slower than the actual thought process.
It's not growing, it's cutting and amplifying that happens on this step. Without it there is less certainty the attention will go the right way. When it is fully pronounced- as if you heard it with your ears - it sort of ripples through your brain, like a physical sensation, echoing for some time. Pulling it out word by word feels natural so You think it was already there, but The thought is not "activated" full strength until you walk it fully with attention, theres only a vague shape, a direction, broad and uncertain. Without it you get faster less lasting thoughts that won't chain well to form a cohesive picture. less sharpness, less resolution, a kind of intuition, maybe it is intuition. You probably get those sometimes, a complex idea that is hard to verbalise, but you can sense it, and you are probably aware how ephemeral they are if you dont stop to give them all your attention. You already know how to think like that, it just doesn't register that you are doing it because it doesn't "echo", you still walk them to remember them
I also find it astonishing that I can feel like I had an entire sentence in my head without any of the words, and fluidly produce all of the words as I say them, without having to search for them or consciously line them up. They're just there, one after the other, like tokens waiting to be picked up. (LLM anyone?) I don't even think that my conscious brain knows exactly which words are going to pop out, say 5 words on. It seems to magically find each word as I speak, without having to pause or rebuffer.
I don't think that language is slowing me down. I actually think that my brain is full of shit and needs to run thoughts through checkers (lint, syntax, logic, fact). I think it makes the language center of our brains all the more magical. As you say, it all happens so fast, and yet it assembles and sanity-checks those raw thoughts as you crystalise them into words.
How many times have I started explaining something, only to realise midway through that I'm taking crap, or that I'm extremely fuzzy on some important detail. Or maybe I infer some important new fact or make some new connection for the first time, while talking about it?
Dogs have thoughts... but we can speak. And every time there's been an innovation in the storage, retrieval or communication of language (not raw thoughts), we've had a gigantic evolutionary leap forward. Isaac Newton was a genius. But when he took up the challenge of explaining the motion of the planets, I bet that not even he knew what he was going to end up with at the end, and I bet that he realised, discovered or rained out a whole bunch of things in the writing of it.
Something else I've wondered. How come my brain holds a million different facts, records of! historical interactions with others, and a pretty decent track of time (like, I know the time, day, month and year and what I did-or-didn't do yesterday), but my dreams are total gibberish? Like I was in a hotel lobby last night with a bunch of people I don't know, realised I'm wasn't wearing any pants, then paniced because my phone was in my pants, how would I call my wife? So I turn to my (deceased) sister and asked which room I'm staying in... If my brain is so good, how come it does crap like that when the conscious bit is switched off?
I would never assume that the data inside my brain, or the subconscious babble that counts for thought, adds up to a genius that is hindered by some clunky language. Very much the opposite.
Side note: all of this is the basis for my extremely strong view that freedom of speech is an absolute necessity for continued prosperity, science, democracy etc. If people are unable to turn their ideas into concrete language, and to do this together as a group, without fear, then they are unable to reason things out properly and make good decisions. I only feel like adding that because within my lifetime I have seen an erosion of the importance of that freedom, to the point where it's no longer possible to discuss mundane, everyday things, or to point out some obvious truth.
> Side note: all of this is the basis for my extremely strong view that freedom of speech is an absolute necessity for continued prosperity, science, democracy etc. If people are unable to turn their ideas into concrete language, and to do this together as a group, without fear, then they are unable to reason things out properly and make good decisions. I only feel like adding that because within my lifetime I have seen an erosion of the importance of that freedom, to the point where it's no longer possible to discuss mundane, everyday things, or to point out some obvious truth.
A fun tangent :)
I think "freedom of speech" is perhaps the wrong place to describe the line: if everyone used words to try to learn about the world, to test their models against reality, this would be flawless.
But that does not fully describe us: we are social creatures, we use language not only to scout, but to fight; and freedom of speech also means freedom for rhetoric. It's cliché to criticise ethos these days, to say that arguments don't depend on the qualifications of one making them. Logos is the one I think you're interested in, based on what you wrote here. Pathos is the one I fear, because I know it works and it makes people believe falsely.
Still, I don't know how to actually get to just "freedom of logos". Some pathos may be necessary to avoid accidentally prohibiting some logos. Some pathos may be simply unavoidable, as the reason to care in the first place (see explanations of why "straw Vulcans" are made of straw: https://tvtropes.org/pmwiki/pmwiki.php/Main/StrawVulcan).
It isn't straw Vulcans, but a completely real thing.
Highly dimensional problems can only be solved through dimensionality reduction, you extract some key features that encompass the problem, get something that at least partially works, and eventually get to the actual solution, even for problems that would be too complex and multifaceted to approach analytically.
It's been a few days but I was thinking about this, and really - are we sure?
Deja vu exists, and as I understand is a bit of a processing issue where something we only just did seems to have happened a little while ago. As I understand it, our brain is in the habit of rewriting timelines a bit to have things make sense, and this could be another case of that.
Social media silencing thoughts during COVID-19 that later to turn out to be right. And mostly nobody is very mad about views being silenced all over the internet.
Right or wrong, people talking their way around around keywords during the pandemic, when not even discussing the pandemic, was weird. They were scared of the YouTube algorithm falsely identifying them and harming their livelihood so would not even say things like "virus" in a video game or movie review.
Then you have the whole trend on various platforms with terms like "unalive" as a replacement for kill/suicide, or censoring words like "r*pe" or (even worse to me) "grape". Some of this is to dodge the algorithms too, but I've seen it said it's to also avoid triggers - which seems to defeat the purpose, it makes it hard to filter and eventually people learn and they become triggers anyway.
So I do see a definite chilling effect on discourse based on community standards and algorithms, even if it's not Government lead. There are two subs for my country on Reddit I check occasionally, one is heavily moderated and very predictable in how a comment will be received, while I feel the other has a more honest reflection of the widespread population. I don't particularly like the racists that share their thoughts, but on the balance of things I'm not sure it's best to lock them out completely. Twenty years ago the internet was less sanitised, but it also felt less hostile on a personal level, and more honest overall. Talking to discuss, not to score points.
Yes - to me the speech feels more like a forcing layer that drives thought. The actual thought is a recurrent neural network underneath. Nearest I have of conscious access to itis non verbal awareness of complex concepts.
This reminds me of some experiment (that I will never be able to find again) that was basically having people count in their head while doing something else, say reading.
Some people were very good at it, others horrible.
One revealed the method they were using - they didn't count audibly, they visualized a ticker tape moving across their vision with numbers increasing. Or say a rotating scale with the numbers rotating. This let them read or internal monologue as the senses are now separate.
I tried to practice for a bit, still impossible to do without thinking about it. Kind of like how people default to counting money in their native way.
That's interesting. I read your comment in my head while counting and seemed able to keep up, but something more complex might be hard, such as reading out loud.
On another tangent, I've been trying Ritalin after speaking to a doctor. The first thing I noticed when I took it was that it became very difficult to hold multiple trains of thought at the same time. My typical routine (coping mechanism) was to work and have YouTube playing and a little attention on each, because this stopped me getting bored. But it wasn't long before I realised I simply could not hear the videos. It was a strange feeling but nice. A little similar to what you describe in how abilities vary.
These questions keep me up at night. That we only get to experience though our own senses.
I pretty much can only pay attention to one thing at a time. I've tried to watch movies or YouTube while coding or other things on my PC but I end up realizing I wasn't paying attention at all so now I don't even try.
When most people focus on one thing, the active part of the brain will actually 'recruit' neighboring parts (which normally do other things) to help with the main activity. That recruitment may not happen with certain people when they're not on Ritalin. (I am not a neurobiologist)
I'm the other way. I can ignore people talking, but when a coworker has the radio on and the spoken words are faint enough or muddled enough that they're hard to make out, I lose focus on what I'm trying to do because the 'listen to words' part of my brain is trying really hard to figure out what's being said, and pulls in extra brainpower to do it.
It really was eye opening how much difference a small 10mg dose made, and the structures of the modern world made a whole lot more sense with the knowledge that maybe a lot of people are just like this, all the time. I had experienced focus before but it was almost always what people call "hyperfocus" which is a more extreme locking out of the world. I kept notes for my appointment in 3 months since I just noticed so many little things I'd be sure to forget and it'll be fun to review those.
I have been learning English for close to ~18 years by now, if you count primary school.
To this day I can't really count in English unless I force myself to.
Is this the norm? I can have an internal dialog but I mainly visualize things, I'd say that 90% of my thinking is visual. I'm not even sure how you'd solve, for instance, an algorithmic problem without visualizing the process. Maybe this is why I feel like a slower thinker than most peers, answers just seem to come them while I have to visualize things first. In college I'd generally take longer than the fast smart people but end up doing slightly better in the end, which always puzzled me. I have terrible memory for facts though.
The answer is that it is hugely variable between people!
Hurlburt has great research on this using Descriptive Experience Sampling.
Some people mainly use images, others mainly speech, others mainly emotion etc. And many more use a varied mix.
Also the way each modality of thought is used is hugely variable - exactly what people see and with what quality or how precisely they feel emotional in their body etc.
To me it explains a huge amount of how different people are good at different skills.
I've a podcast on this topic ("Imagine an apple") if you're interested in more.
You can visualize an algorithm?? Makes no sense to me. To me, when thinking about an algorithm, it's more navigating the data flow. Following connections between concepts. No words nor visuals.
Yeah, most things lend themselves to visualizing, but even things that don't I have to visualize something, the answer doesn't just come to me unless I have something to look at if that makes sense.
Let me ask, when you decide what to eat for dinner, how do you arrive at the answer? I visualize what's in the fridge, what I can cook with the ingredients. Or if I want to eat out I visualize our neighborhood like Google Maps and look around at the different options.
When I'm solving an algorithmic problem I visualize the data, spatially if possible otherwise just as text, lists or vectors. It's like I'm using a notebook but it's faster and a bit more fuzzy.
It's not 100% for me but just the vast majority. I do visualise things that are almost purely spacial like geometry or recalling how to do an exercise. Though from what I've read, even this is news to some people who express surprise that "mind's eye" is a little more literal than they assumed. I'm pretty good at remembering facts and trivia but not so much actual life experiences, not sure if that's related.
I can visualize algorithms but I have to do so deliberately. Unlike the parent poster I don't always think in internal monologues either.
Sometimes it's a keyword/concept thing where I'll think of the main items and I get a feeling that I know how to fill in the blanks. I haven't actually visualized or verbalized what would fill those blanks though (and sometimes the feeling is wrong).
I think pretty much all of the senses can be used to do some form of thinking. I can imagine songs in my head, touch, feelings etc. Rarely are they useful for problem solving though, but some of these are nice for falling asleep in unknown environments.
Oh and then there's the thinking where nothing seems to happen. I stare at a piece of paper and after a while I know what to do next. How did I arrive at that conclusion? I don't know, but it definitely wasn't verbal, visual, aural or anything else. This tends to not solve complex problems like math, but it basically tells me what I should do to try to solve it (usually verbal or visual).
I have an "inner voice" which "wants" to turn my word-shaped-thoughts into an inner audio stream, and "gets annoyed" if, upon "my" realisation that I've already got the entire sentence, I can save time by not "reading" it "aloud".
(All those scare quotes because this is not at all literal, just how it feels from the inside).
Interestingly, when I'm in this state (the thought has to already exist) I can let my fingers type it out for me while I'm paying attention to something else entirely — but I can't simultaneously read while listening to someone talk.
"The idea of linguistic relativity, known also as the Whorf hypothesis, the Sapir–Whorf hypothesis (/səˌpɪər ˈhwɔːrf/ sə-PEER WHORF), or Whorfianism, is a principle suggesting that the structure of a language influences its speakers' worldview or cognition, and thus individuals' languages determine or influence their perceptions of the world."
