> Dumping tokens into a pile of linear algebra doesn't magically create sentience.
More precisely: we don't know which linear algebra in particular magically creates sentience.
Whole universe appears to follow laws that can be written as linear algebra. Our brains are sometimes conscious and aware of their own thoughts, other times they're asleep, and we don't know why we sleep.
And that's fine, but I was doing the same to you :)
Consciousness (of the qualia kind) is still magic to us. The underpants gnomes of philosophy, if you'll forgive me for one of the few South Park references that I actually know: Step 1: some foundation; step 2: ???; step 3: consciousness.
Right, I don't disagree with that. I just really objected to the "must", and I was using "pile of linear algebra" to describe LLMs as they currently exist, rather than as a general catch-all for things which an be done with/expressed in linear algebra.
Garbage collection, for one thing. Transfer from short-term to long-term memory is another. There's undoubtedly more processes optimized for or through sleep.
Those are things we do while asleep, but do not explain why we sleep. Why did evolution settle on that path, with all the dangers of being unconscious for 4-20 hours a day depending on species? That variation is already pretty weird just by itself.
Worse, evolution clearly can get around this, dolphins have a trick that lets them (air-breathing mammals living in water) be alert 24/7, so why didn't every other creature get that? What's the thing that dolphins fail to get, where the cost of its absence is only worthwhile when the alternative is as immediately severe as drowning?
Because dolphins are also substantially less affected by the day/night cycle. It is more energy intensive to hunt in the dark (less heat, less light), unless you are specifically optimized for it.
That's a just-so story, not a reason. Evolution can make something nocturnal, just as it can give alternating-hemisphere sleep. And not just nocturnal, cats are crepuscular. Why does animal sleep vary from 4-20 hours even outside dolphins?
Sure, there's flaws with what evolution can and can't do (it's limited to gradient descent), but why didn't any of these become dominant strategies once they evolved? Why didn't something that was already nocturnal develop the means to stay awake and increase hunting/breeding opportunities?
Why do insects sleep, when they don't have anything like our brains? Do they have "Garbage collection" or "Transfer from short-term to long-term memory"? Again, some insects are nocturnal, why didn't the night-adapted ones also develop 24/7 modes?
Everything about sleep is, at first glance, weird and wrong. There's deep (and surely important) stuff happening there at every level, not just what can be hypothesised about with a few one-line answers.
Yes, actually. Insects have both garbage collection & memory transfer processes during sleep. They rely on the same circadian rhythm for probably the same reasons.
And the answer to "Why not always awake?" is very likely "Irreversible decision due to side effects". Core system decisions like bihemispheric vs unihemispheric sleep can likely only be changed in relatively simple lifeforms because the cost of negative side effects increases in more complex lifeforms due to all the additional systems depending on the core system "API".
"This statistical model is governed by physics": true
"This statistical model is like our brain": what? no
You don't gotta believe in magic or souls or whatever to know that brains are much much much much much much much much more complex than a pile of statistics. This is like saying "oh we'll just put AI data centers on the moon". You people have zero sense of scale lol
We, all of us collectively, are deeply, deeply ignorant of what is a necessary and sufficient condition to be a being that has an experience. Our ignorance is broad enough and deep enough to encompass everything from panpsychism to solipsism.
The only thing I'm confident of, and even then only because the possibility space is so large, is that if (if!) a Transformer model were to have subjective experience, it would not be like that of any human.
Note: That doesn't say they do or that they don't have any subjective experience. The gap between Transformer models and (working awake rested adult human) brains is much smaller than the gap between panpsychism and solipsism.
Ok, how about "a pile of linear algebra [that is vastly simpler and more limited than systems we know about in nature which do experience or appear to experience subjective reality]"?
Agreed; "disorienting" is perhaps a poor choice of word, loaded as it is. More like "difficult to determine the context surrounding a prompt and how to start framing an answer", if that makes more sense.
You're replying to me, but I don't agree with your take - if you simulate the universe precisely enough, presumably it must be indistinguishable from our experienced reality (otherwise what... magic?).
My objection was:
1. I don't personally think anything similar is happening right now with LLMs.
2. I object to the OP's implication that it is obvious such a phenomenon is occurring.
Your response is at the level of a thought terminating cliche. You gain no insight on the operation of the machine with your line of thought. You can't make future predictions on behavior. You can't make sense of past responses.
It's even funnier in the sense of humans and feeling wetness... you don't. You only feel temperature change.
Until the price of gas starts to remotely reflect the medium to long term costs of climate change I basically always celebrate anything that increases gas or carbon-based energy prices. Like, it sucks... but there's lots of data that consumers respond to these prices in their choices.
The way I think about it, the entirety of global civilization is massively, massively subsidizing carbon emission.
I agree. I’m just addressing the notion raised in the post above that oil companies will bear cost increases in an industry where everyone sells an identical product and consumers can just cross the street to save $0.10 a gallon.
If you wanted to pay for direct air capture of CO2 to directly "undo" your climate effect of driving, the cost would currently be about $6 per gallon. Price comes from [1], found [2] looking for a second opinion on current direct air capture cost.
Direct air capture is just not feasible at a world scale.
And the whole circus around it, manufacturing (and extracting the natural resources for that) of all the machinery for it, clearing land to place it (and all the NIMBY circus), all the energy generation for it, the transmission lines, the maintenance, the burying of the captured carbon. It's all going to lead to lots of pollution and CO2 emissions even if the things are powered by 100% green energy.
It's just a pipe dream of the people looking for a quick fix so we can continue doing what we've been doing.
But we'll just need so hellish many of them to make a dent in global CO2 levels in time to prevent the worst effects of climate change. It's just impossible.
The only way to really fix things is not emitting the stuff in the first place but most people prefer putting their fingers in their ears.
Hoisting 100 tons of stuff high into the air, and then efficiently converting that into the high RPM needed to drive a generator seems like it would take a truly staggering construction effort. Suspending that amount of weight high above your house also has some... interesting potential failure modes.
I don't even think the gravity battery thing is viable for individual residential power storage at all. I was just wondering why you'd assume that the 100 ton weight would be placed directly above your house given the obvious problems with that approach, and the obvious way to avoid those problems.
The comment I was replying to literally said "For home use", and a heavy object 10m in the air does not have to be directly above something to be meaningfully (and dangerously) above something.
It's a silly scenario anyway, but I was doing a bit of guesswork about typical "home" lot sizes.
Yeah I understand it's for home use. I am imagining a tower in the back yard or something. It would be closed so that nobody can walk under the weight. Or it could be internal to the house like an elevator shaft.
Anyway I agree it's silly, definitely not a realistic idea
Right - if a tower in the back yard falls down it can still hit your house, since it isn't guaranteed to neatly collapse straight down. Worst case, it may tip over from the base and directly smash stuff up to its height away (and 10m is pretty far).
I have trees in my back yard I'm kind of worried about, which is why this immediately came to mind.
... yeah? You'd expect the false positive rate to be HIGHER when you're not looking at an enriched patient subset. That's why we're careful about recommending certain kinds of screening. See also: PSA screening.
Well, you missed my point. I'm not talking about "you look at the MRI and see something and say it's a positive", I'm referring to the process of reading MRIs as like a statistical model (even if in practice it exists in the minds of radiologists) which is trained on the corpus of MRI data. That model will depend in some way on the distribution of positive/negative examples in the corpus; if the corpus changes the model has to then be updated to match.
Point is, the false positive concern is only a concern if you use the old model with the new corpus. Don't do that! That's dumb!
The net effect of MRIing everyone on public health would likely be enormously positive as long as you don't do that.
Take PSA, since it's a simpler example. You're right that, if we screen everyone, taking action based on the outcome causes more harm than good. The response is to calibrate... which means we don't learn anything usefully actionable from the test and shouldn't apply it.
With the MRI, you don't get back simple dichotomous things, but you get back potential indications. That can be scary - talk about calibration all you want, but if patients see things and start thinking about the big C word there are likely to be a lot of unnecessary biopsies.
The bottom line is that it's possible to imagine a benefit, but it is not reasonable to pretend it's as simple as "just re-calibrate your interpretation of the results!". There's a reason that a lot of thought goes into when to do screening.
> which means we don't learn anything usefully actionable from the test and shouldn't apply it.
This just isn't true. In practice any such screening model can ALWAYS improve with more data—basically because the statistical power goes up and up—up to an asymptote set by noise in the physical process itself.
> That can be scary
Handling that is the job of professionals, is now and will continue to be.
It is extremely reasonable to imagine a benefit! What is doubtful is imagining there wouldn't be one!
I find the line of reasoning in this whole anti-MRI-everyone argument to be bewildering. I think it is basically an emotional argument, which has set in as "established truth" by repetition; people will trot it out by instinct whenever they encounter any situation that suggests it. It reflects lessons collectively learned from the history of medicine, its over-estimation of its own abilities and its overfitting to data, and its ever-increasing sensitivity to liability.
But it is not inherently true—it is really a statement about poor statistical and policy practices in the field, which could be rectified with concerted effort, with a potential for great public upside.
Not that any of this matters at the current price point. But, on a brief investigation, the amortized cost of a single MRI scan is ~$500-800—perhaps 1/5 what I would have guessed!
> This just isn't true. In practice any such screening model can ALWAYS improve with more data—basically because the statistical power goes up and up—up to an asymptote set by noise in the physical process itself.
That isn't how this works at all.
1. If you assume the test results are iid, sure you can increase your precision (presuming you're talking about repeatedly testing people?), but biology is messy and the tests are correlated. You can get all kinds of individual-specific cross-reactivity on a lab assay, for example. As another example, you can't just keep getting more MRIs to arbitrarily improve your confidence that something is cancer/not cancer/a particular type of cancer etc.
2. Statistical power is not relevant here, but rather different kinds of prediction error. It turns out that in the general population, it is NOT medically relevant that PSA is correlated with the presence of prostate cancer, because it is NOT predictive of mortality, and it IS a cause for unnecessary intervention and thus harm to patients.
I really don't mean to cause offense, but you're talking about this like someone who has no idea how these concepts interact with reality in the biomedical world. Like, you seem to be applying your intuition about how tabular data analysis tends to work in systems you're familiar with, and assuming it generalizes to a context where you don't have experience.
> this whole anti-MRI-everyone argument to be bewildering
It's not about being against MRIs, it's about the idea that (even ignoring costs/cost effectiveness) there are known real-world effects of over-screening people for things.
> But it is not inherently true—it is really a statement about poor statistical and policy practices in the field, which could be rectified with concerted effort, with a potential for great public upside.
This is still not at all a certainty. Let's say you lock this behind a screening system run by data scientists so that there's no patient or provider pressure to act in what you're calling a statistically poor manner. Ok, then what? They have to come up with a decision rule about when to dig deeper and get more data (which again, isn't an MRI, but rather is often an invasive procedure). It is not obvious that there exist any decision rule that could reasonably be arrived at that would be a good trade-off in terms of false positives and the corresponding additional burden.
I am 1000% willing to entertain the idea that new screening can be a net benefit, but we'd need to know what kind of sensitivity/specificity tradeoff would be involved to even start approximating the numbers, and then you'd need to do a trial to demonstrate that it's worthwhile, and even then you'd need to do post-trial monitoring to make sure there aren't unexpected second order effects. People DO, in fact, do this work.
The idea that "more data == better" is just way too simplistic when the data is messy and necessarily inconclusive, the outcomes of interest are rare, and the cost of additional screening can be severe - again also ignoring that all of this is expensive in the first place.
Must it? I fail to see why it "must" be... anything. Dumping tokens into a pile of linear algebra doesn't magically create sentience.
reply