Chaotic nonlinear dynamics have been an object of mathematical research for a very long time and we have built up good mathematical tools to work with them, but in spite of that turbulent flow and similar phenomena (brains/LLM's) remain poorly understood.
The problem is that the macro and micro dynamics of complex systems are intimately linked, making for non-stationary non-ergodic behavior that cannot be reduced to a few principles upon which we can build a model or extrapolate a body of knowledge. We simply cannot understand complex systems because they cannot be "reduced". They are what they are, unique and unprincipled in every moment (hey, like people!).
Once we start to see diminishing returns to model size and model performance begins to plateau (we are already nearing the plateau) we will have a set of models all pretty close to one another in performance which will be made available for downstream workloads on demand.
The consequences of this will be twofold. First, there will be less incentive for new entrants to train proprietary models as performance across SOTA models will be uniform and thus less possibility of differentiation and gaining market share. Second, there will be less incentive for SOTA model developers to invest heavily in GPUs because of diminishing returns.
Now the beauty of LLMs is that inference requires orders of magnitude less compute than training. So once we have a stable of performance plateaued models, compute demand will drop off a cliff.
Once we get over this mad rush to train train train, GPU orders are going to crater.
The two critical assumptions I'm making are 1) we are rapidly approaching a performance plateau and 2) models other than LLM's are much less widely useful than LLM's. Image generation I see as a fad, I played around with it a ton last year but haven't touched it since. Other niche models will have very little general applicability compared to LLM's.
Sitting in traffic is much more inefficient than one would think, though if you're cruising on the interstate at 70 mph you would obviously be correct.
Why not take a bike ride to the train and then ride the train in? People wanting autopilot driving cars....that's literally what a train does, complete with wifi.
I think it's fair at this point to say there is a clear relationship. The earliest article I have in my (relatively cursory) research file on the metabolic relationship for AD is from 1994:
Fukuyama, Hidenao, Masafumi Ogawa, Hiroshi Yamauchi, Shinya Yamaguchi, Jun Kimura, Yoshiaru Yonekura, and Junji Konishi. “Altered Cerebral Energy Metabolism in Alzheimer’s Disease: A PET Study.” Journal of Nuclear Medicine 35, no. 1 (January 1, 1994): 1–6.
Here are some more recent reviews discussing the connection (recent reviews are good not just because they take into account the latest developments, but they also give you better references to go through for further research). Note, these reviews are from completely different/unrelated research teams from different countries:
Yoon, Ji Hye, JooHyun Hwang, Sung Un Son, Junhyuk Choi, Seung-Won You, Hyunwoo Park, Seung-Yun Cha, and Sungho Maeng. “How Can Insulin Resistance Cause Alzheimer’s Disease?” International Journal of Molecular Sciences 24, no. 4 (February 9, 2023): 3506. https://doi.org/10.3390/ijms24043506.
Rhea, Elizabeth M., Manon Leclerc, Hussein N. Yassine, Ana W. Capuano, Han Tong, Vladislav A. Petyuk, Shannon L. Macauley, et al. “State of the Science on Brain Insulin Resistance and Cognitive Decline Due to Alzheimer’s Disease.” Aging and Disease, August 17, 2023, 0. https://doi.org/10.14336/AD.2023.0814.
Kshirsagar, Viplav, Chetan Thingore, and Archana Juvekar. “Insulin Resistance: A Connecting Link between Alzheimer’s Disease and Metabolic Disorder.” Metabolic Brain Disease 36, no. 1 (January 2021): 67–83. https://doi.org/10.1007/s11011-020-00622-2.
Men who were prone to alternative/extreme states of consciousness would have been better able to summon up the wild emotions necessary for savagery and violence, making them better fighters and hence giving them better mating prospects.
They would be constantly vigilant and unpredictable to someone unfamiliar with that particular trait. When wandering you might only have yourself as a reference so they would be a counter to that
If they've even lived long enough, no absolutely not. Unless you think someone who is routinely depressed is 'vigilant'.
[Hypo]mania is not the primary state of a bipolar patient. Can't speak to schizophrenia, but for those I've seen with that illness, functioning isn't their primary state, either.
Mania, maybe, but not depression. It clouds the mind and saps motivation in a way that would undermine efficacy in both hunting and surprise encounters.
Obviously I don't think it can be turned on-off, why would you even think that I was suggesting that? It's just easy for me to imagine these scenarios triggering mania in me whether I wanted it to or not, and I don't think I'm that unusual, that's all.
No, that's not how [hypo]mania works. You can 'imagine' all you want, but thinking that a specific scenario or time is when [hypo]mania 'lights up' is flat out incorrect.
And someone who experiences mania indicates an uncontrolled illness. Depression and mania prevent normal functioning. You seem to believe an unmedicated bipolar individual is a functional individual with some inkling of rational thought that could 'go into battle with no fear'.
If you don't know, don't post. And you don't know.
Your statements about what you know about me and my lived experiences, and my level of knowledge of those experiences, is bizarrely cruel. Please do not do this. Please don't try to silence me just because those things don't match up with your own experiences.
I don't know where you are getting the notion that I am saying that mania is rational or controllable. I was speculating based on personal experience around high stress situations. I do not understand why you are choosing to attack the notion that there could be evolutionarily advantageous benefits conferred upon the group.
Edit: we agree that I am speculating. I disagree with you that I have no right to speculate
I know both pretty well, indirectly or not. Given how much they can influence your energy and "will to live" I don't think they'd pair well with having to fight for your life.
That ship sailed with Quantum physics. Nearly perfect at prediction, very poor at giving us a concrete understanding of what it all means.
This has happened before. Newtonian mechanics was incomprehensible spooky action at a distance, but Einstein clarified gravity as the bending of spacetime.
I think this relies on either the word “concrete” or a particular choice of sense for “concrete understanding”.
Like, quantum mechanics doesn’t seem, to me, to just be a way of describing how to predict things. I view it as saying substantial things about how things are.
Sure, there are different interpretations of it, which make the same predictions, but, these different interpretations have a lot in common in terms of what they say about “how the world really is” - specifically, they have in common the parts that are just part of quantum mechanics.
The qau that can be spoken in plain language without getting into the mathematics, is not the eternal qau, or whatever.
One way to understand the legitimacy and value of non-analytic philosophy is first to realize that the unmeasurable complexity of the world defies complete exposition by rigorous analytical methods. We must then ask ourselves, are we to bury our heads in the sand and consider only what is provable and what yields to such methods? There is certainly a great deal of value in doing this, modern civilization is built upon brilliant scientists and technicians who have done just that. But to deny the very existence of things outside the scope of analytical methods is to deny fundamental truths about the world.
Ok, if we want to pick our heads up and look around, what are we to make of what we see? How are we to navigate in this strange environment where grotesque and beautiful things undulate, merge, unmerge and dance almost tauntingly all around us at a frenetic pace? The first thing is to give up any hope of tracing this process by unambiguous cause and effect, which in turn means giving up on proof or replicability. Crucially however, one need not give up on truth, for there is a world out there and things are really happening, and it is possible to describe the action you observe, in spite of it not being reducible to neat and tidy chains of reason.
The difficulty with this is that we cannot necessarily share our findings and observations with others and expect them to agree, or hope to build a stable consensus, because the sorts of truths we are concerned with are not trivially derived in the way that mathematical truth is, in its exposition, trivial. Thus in occupying ourselves with these things, we condemn ourselves in a way to perpetual conflict, both internal and external. But the important things to remember is, reality is still out there, it's still happening in a particular way at a particular time, whether we can agree on it or not.
I don't believe that analytic philosophy covers everything that the world has to offer.
But I have yet to find anything of value to me in "continental philosophy". I realize that the distinction is arbitrary and tendentious, and the "continental philosophers" would not use that term for themselves.
Still... if we're not going to place the values of analytic philosophers at the fore, then we have to use our own. The only tool I have at my disposal is "I don't like it". Other people find meaning in it, and that's fine for them, but we don't seem to have any meeting ground and no good way to find one.
Which is fine. Neither of us is wrong. I just have to avoid resenting the ways that (in my estimation) continental philosophers act as if they were possessed of arcane secrets and it's my fault that I fail to understand them.
I think it’s entirely reasonable to resent this behaviour. By portraying their musings as weighty and learned discourse rather than unfalsifiable speculation, they’re wasting our time.
Falsifiability isn't the be-all and end-all. It's an important way of looking at the world, but it's also limiting.
The limits are great at helping us identify absolute bullshit. And all of that bullshit is aggravating because it makes it seem impossible that there could be anything else. But it's still important to avoid the intellectual laziness of dismissing it automatically out of hand and potentially missing something else that matters to us.
>the unmeasurable complexity of the world defies complete exposition by rigorous analytical methods.
Do you mean that it would be impossible to explain the world in such a way even if we had an unlimited amount of time and ability, or just that we would never have access to what would even be theoretically necessary to rigorously explain the world?
The latter seems undeniable, the former I don't think we can claim to know for sure but I want to say the opposite is true.
>Crucially however, one need not give up on truth, for there is a world out there and things are really happening, and it is possible to describe the action you observe, in spite of it not being reducible to neat and tidy chains of reason.
It seems like you're saying that we have mathematical truths, and "tricky truths" that can't be derived but should still be sought out. This totally makes sense to me. However, it seems like you're trying to imply a kind of equivalence between these kinds of truths and de-emphasizing the importance of mathematical reasoning when it seems like instead there should be a preference for mathematical truths that don't as easily lead to the kinds of conflict or disagreements that you're describing.
I.e. it seems like beliefs about these "tricky truths" should be held less strongly than beliefs about mathematical truths, should be grounded in something like Bayesian reasoning that we can derive mathematical truths about, and we should be constantly searching for ways to tie our beliefs about these "tricky truths" together in ways that follow something like rigorous analytical methods.
I think you’re exactly right. In philosophy this Bayesian thinking is called “verisimilitude”, or to quote Stephen Colbert, “truthiness”.
Hume and Wittgenstein did a good job in their own ways of elucidating why judgements cannot stem only from facts. The “is ought problem” and “picture theory of language” are approachable places to google on the matter.
But we make judgements all day and it does seem like there are better judgements than others. We don’t have easy answers for what is right, wrong, beautiful, ugly, redeemable, or important. But these unscientific concerns are the most important aspects of anyone’s life, and philosophers can provide a lot of insight.
>One way to understand the legitimacy and value of non-analytic philosophy
I don’t think HN is even sold on the value and legitimacy of analytic philosophy. It’s best to get people on board with that first, before moving on to the weird stuff.
If the world is immeasurably complex, then we will only see the part of that we can measure, meaning our world is measurably complex. We don't have any ability to measure or otherwise interact with the immeasurable part, such as the Universe beyond the observable Universe.
I used to think similarly, but learning about the proven incompleteness of sufficiently rigorous analytic systems has moved me towards appreciating less rigorous ways of interfacing with reality.
Veritasium has a nice video on Godel’s incompleteness theorem. It’s quite ironic that it was mathematically provable that analytic systems can never be complete, and can’t be provably coherent.
Oh I studied this, I'm well aware of those theorems, and I don't think they are the death bell of logic some make them to be.
If you somehow find an unprovable theorem (quite rare), you can always try with a different set of axioms. Mathematics are not about proving absolute truths of the universe but rather of pushing reasoning over a set of axioms the furthest possible.
Also as a nitpick, analytic systems that don't contain arithmetic can be complete and proven coherent.
As for the real world, facts that can neither be proven true nor false (existence of an immaterial soul for example), I think should be left at that. It is useless speculating about things we can't ever know of. I leave that to religion/spirituality.
"It is useless speculating about things we can't ever know of."
And how do you know which things we can know of and which we cannot? Trusting your gut instinct on that isn't scientific. And why is speculating about immaterial things useless? I'm sure many great mathematicians heard some form of "what you're doing is useless and has no use or relation in the real world" especially in the realm of pure mathematics.
You bring up the soul which is a convenient example, but let's instead use a concept which YOU know exists for yourself which is consciousness. Can we ever know anything more about the mystery of consciousness or life or why any of this world and universe exists? Are those unknowable? Should we not talk about them? Should we only try to apply the lens of science here and for some reason not try to advance our understanding using philosophy even though it might not be as formal and unambiguous as math?
Yea, the previous posters take is just outright crazy to me....
Imagine reality as the problem space of all things that could exist within the constraints of physics. The problem with observational evidence that it is only providing a tiny window into what is possible, really only the most probable are going to be what you see for the most part. Philosophy gives a means of meta views of systems and simplified system views that allow us to find otherwise unreachable islands of what can exist in our reality.
Here is my reasoning: immaterial objects (such as the soul) are by definition outside of the material world, and thus unobservable. We can't ever (dis)prove the existence of an unobservable object.
As for the other questions you mentioned, I still haven't found any reason to believe their answers are unkowable.
If anyone disproves my first reasoning, I will have to consider the question of the soul as worth pursuing again.
Just like in mathematics, if you have sufficient proof that a theorem is unprovable, it is useless trying to prove it!
So no, I don't trust my gut feeling about wether or not to seek the answer to something.
Consider turbulent systems, or systems which are affected by random processes. Such things exist in the world. The sorts of things I'm talking about as being beyond cause and effect or our ability to consider analytically are not merely the "big questions" but are real physical processes!
Not death of logic, since logic is still consistent. Gödel didn't say math was wrong, just incomplete.
So philosophy might step into the space beyond where a current logical set of axioms can reach. To explore something un-provable and maybe find a new set of axioms that were not reachable through logic built on the previous axioms.
> I think that if someone is completely unable to justify an observed truth to others, then it might not be a truth at all.
What if the observed truth that someone is trying to communicate is paradoxical and hard to communicate in and of itself? What if the truth is ambiguous? What constitutes ambiguous or unambiguous?
At the end of the day ambiguity is a real concept, so is a paradox, therefore there will exist things that are ambiguous and paradoxical and pointing that out does have value.
There is no reason that ambiguities or paradoxes can't be expressed analytically and formally. Math and computer science are full of such things and they are celebrated.
Being hard to communicate is precisely why it's important to communicate rigorously and formally.
We have a ton of examples of great mathematicians who also happened to be great philosophers and vice-versa. Some philosophers also tried to incorporate mathematical symbols and such into their work. We value both their philosophical works and mathematical works. They were smart people and chose different mediums to express different concepts.
How can you try to explore the ego, consciousness, unconsciousness, dreams, suffering, life's purpose, subjective beauty, symbolism, truth, religion, god, ethics and whatever else that is not easily formalized? We might very well arrive at a formal and unambiguous description of these sometime in the far future, so are we not supposed to at least try to talk about these concepts now? You use different tools for different concepts, and science and philosophy is just 2 of those tools. At the end of the day philosophy undeniably changed the world, so there is at least some value to it. Philosophy is not anti-logic, it is very much for logic.
It is a practical question. Sometimes we need to have or choose a hard answer to make a decision. It inevitably isn't going to be solved formally.
Any more than what is the dividing line between a chair and not-chair. Many patterns we encounter have fuzzy non-formal edges.
Perfect consensus is impossible, but any consensus is valuable. So we invent and argue about the "best" way to "understand" these things.
These arguments are partly objective, partly subjective, partly emergent, and partly just farmed out to favorite "authorities" or social pressure. But important and unavoidable.
--
At the highest level, even how we percieve reality is important. It impacts our values, our motivations, our ethics, how we cope with events, etc. Trickling down to every day choices.
"What is real?" ends up being an important question, no matter how lacking in formal rigor the answers we each have are.
The astronauts add that raw aspect of bravery and potential sacrifice to the story. Not that long hours over years and years dedicated to a single project is not sacrifice. But that doesn't hold a candle to a risk your life type of sacrifice.
In 2024 people still get nervous going on domestic flights. I can't imagine how someone could muster the sheer courage to volunteer as an astronaut, either then or now. It's a testament to humanity's willingness to risk even death in pursuit of something greater.
"humanity's willingness to risk even death in pursuit of something greater."
I would rephraze this as 'humanity's capability to sustain a large range of psychological profiles and the capability to filter out from the general population the particular individuals fitting for a given task'. My understanding is the astronauts were a very specific niche group and mostly wanted to fly faster and higher than anyone before them, and were willing to pay any price to achieve this.
The problem is that the macro and micro dynamics of complex systems are intimately linked, making for non-stationary non-ergodic behavior that cannot be reduced to a few principles upon which we can build a model or extrapolate a body of knowledge. We simply cannot understand complex systems because they cannot be "reduced". They are what they are, unique and unprincipled in every moment (hey, like people!).
reply