When people imagined AI/AGI, they imagined something that can reason like we can, except at the speed of a computer, which we always envisioned would lead to the singularity. In a short period of time, AI would be so far ahead of us and our existing ideas, that the world would become unrecognizable.
That's not what's happening here, and it's worth remembering: A caveman from 200K years ago would have been just as intelligent as any of us here today, despite not having language or technology, or any knowledge.
In Carolyn Porco's words: "These beings, with soaring imagination, eventually flung themselves and their machines into interplanetary space."
When you think of it that way, it should be obvious that LLMs are not AGI. And that's OK! They're a remarkable piece of technology anyway! It turns out that LLMs are actually good enough for a lot of use cases that would otherwise have required human intelligence.
And I echo ArekDymalski's sentiment that it's good to have benchmarks to structure the discussions around the "intelligence level" of LLMs. That _is_ useful, and the more progress we make, the better. But we're not on the way to AGI.
It's interesting to me how much effort the AI companies (and bloggers) put into claiming they can do things they can't, when there's almost an unlimited list of things they actually can do.
This generally just keeps being the "the Emperor has no clothes" moment for all these AI bull companies.
Microsoft just replaced their native Windows Copilot application with an Electron one. Highly ironic.
Obviously the native version should run much faster and will use less memory. If Copilot (via either GPT or Claude) is so godlike at either agentic or guided coding, why didn't they just improve or rewrite the native Copilot application to be blazing fast, with all known bugs fixed?
And many of them so unexpected, given the unusual nature of their intellegence emerging from language prediction. They excel wherever you need to digest or produce massive amounts of text. They can synthesize some pretty impressive solutions from pre-existing stuff. Hell, I use it like a thesaurus to sus out words or phrases that are new or on the tip of my tounge. They have a great hold on the general corpus of information, much better than any search engine (even before the internet was cluttered with their output). It's much easier to find concrete words for what you're looking for through an indirect search via an LLM. The fact that, say, a 32GB model seemingly holds approximate knowlege of everything implies some unexplored relationship between inteligence and compression.
What they can't they do? Pretty much anything reliably or unsupervised. But then again, who can?
They also tend to fail creatively, given their synthesize existing ideas. And with things involving physical intuition. And tasks involving meta-knowlege of their tokens (like asking them how long a given word is). And they tend to yap too much for my liking (perhaps this could be fixed with an additional thinking stage to increase terseness before reporting to the user)
My current way of thinking about LLMs is "an echo of human intelligence embedded in language".
It's kind of like in those sci fi or fantasy stories where someone dies and what's left behind as a ghost in the ether or the machine isn't actually them; it's just an echo, an shallow, incomplete copy.
Only because they have compressed and encoded the entire sum of human knowledge at their disposal. There are models for everything in there, but they can only do what has been done before.
What's more amazing to me is the average human, only able to hold a relatively small body on knowledge in their mind, can generate things that are completely novel.
People assume training on past data means no novelty, but novelty comes from recombination. No one has written your exact function, with your exact inputs, constraints, and edge cases, yet an LLM can generate a working version from a prompt. That’s new output. The real limitation isn’t novelty, it’s grounding.
Just because there are lots of tasks which can be accomplished without the need for anything novel doesn't mean LLMs can match a human. It just means humans spend a lot of time doing some really boring stuff.
I hear this constantly. Can you produce something novel, right here, demonstrably, that an LLM couldnt have produced? Nobody ever can, but it’s sure easy to claim.
I'm going to assume you mean this seriously, so I will answer with that in mind.
Yes, I can.
- I can build an unusual, but functional piece of furniture, not describe it, not design it. I can create a chair I can sit on it. An LLM is just an algorithm. I am a physically embodied intelligence in a physical world.
- I can write a good piece of fiction. LLMs have not demonstrated the ability to do that yet. They can write something similar, but it fails on multiple levels if you've been reading any of the most recent examples.
- I can produce a viable natural intelligence capable of doing anything human beings do (with a couple of decades of care, and training, and love). One of the perks of being a living organism, but that is an intrinsic part of what I am.
- I can have a novel thought, a feeling, based on qualia that arise from a system of hormones, physics, complex actions and inhibitors, outrageously diverse senses, memories, quirks. Few of which we've even begun to understand let alone simulate.
- And yes I can both count the 'r's in strawberry, and make you feel a reflection of the joy I feel when my granddaughter's eyes shine when she eats a fresh strawberry, and I think how close we came to losing her one night when someone put 90 rounds through the house next door, just a few feet from where her and her mother were sleeping.
So yeah, I'm sure I can create things an LLM can't.
So the only thing I am seeing here is physical or personal (I have no idea how you feel or what your emotions are. You are a black box just as an LLM is a black box.)
The only thing you mentioned is the fan fic and I would happily take the bet that an LLM could win out against a skilled person based on a blind vote.
Was an individual mind responsible for us as humanity landing on the moon? No. Could an individual mind have achieved this feat? Also no.
Put differently, we should be comparing the compressed blob of human knowledge against humanity as a collective rather than as individuals.
Of course, if my individual mind could be scaled such that it could learn and retain all of human knowledge in a few years, then sure, that would be a fair comparison.
I want to see an LLM create an entirely novel genre of music that synthesizes influences from many different other genres and then spreads that genre to other musicians. None of this insulated crap. Actual cultural spread of novel ideas.
The hype has gotta keep going or the money will dry up. And hype can be quantified by velocity and acceleration, rather than distance. They need to keep the innovation accelerating, or the money stops. This is of course completely unreasonable, but also why the odd claims keep happening.
Why would the money dry up when we have companies willing to spend $1000/developer/month on AI tooling when they would have balked at $5/u/mo for some basic tooling 2-3 years ago?
First in some cases it is more than $1000/dev/month.
Those companies spending 1000+/developer are doing it with the same hope that at some point those $1000/month will replace the developer salary per month. Or because by doing so more investors will put more money into them.
Take away the promise of AI replacing developers and see how much a company is willing to pay for LLMs. It is not zero as there are very good cases for coding assisted by LLM or agentic engineering.
Because most of these things are not multi-trillion-dollar ideas. "We found a way to make illustrators, copyeditors, and paralegals, and several dozen other professions, somewhat obsolete" in no way justifies the valuations of OpenAI or Nvidia.
>Because most of these things are not multi-trillion-dollar ideas.
That's right, but there's more. When you think about the cost of compute and power for these LLM companies, they have no choice. It MUST be a multi-trillion-dollar idea or it's completely uninvestable. That's the only way they can sucker more and more money into this scheme.
This is literally the first time I've heard this. What is your source? I can type the exact same query three times and though the general meaning may be the same, the actual output is unique every single time. How do you explain this if it's cached?
I don't know about OpenAI, but Nvidia's valuation seems more justifiable based on their actually known revenue and profit, and because it's publicly traded.
Though if the bubble(?) bursts and Nvidia starts selling fewer units year-over-year, that could be problematic.
Well, for starters, they definitively passed the Turing test a few years ago. The fact that many regard them as equivalent in skill to a junior dev is also, IMO, the stuff of science fiction.
how do you market that as a product that is needed by other people?
there are already companies that advertise Ai date partners, Ai therapists and Ai friends - and that gets a lot of flame about being manipulative and harmful
I've been pushing Opus pretty hard on my personal projects. While repeatability is very hard to do, I'm seeing glimpses of Opus being well beyond human capabilities.
I'm increasingly convinced that the core mechanism of AGI is already here. We just need to figure out how to tie it together.
Generating 3000 lines of esoteric rendering code within minutes, to raster generative graphics of anything you can imagine and it just works? From natural language instructions. Seriously think about that my dude.
This is a bit of an anti-evolutionary perspective. At some point in our past, we were something much less intelligent than we are now. Our intelligence didn't spring out of thin air. Whether or not AI can evolve is yet to be seen I think.
Sure, but then basically whatever it was, it was not "us". "Us" and our intelligence had to appear at some point. It's 100% not "anti-evolutionary" to say some years ago humans became as mentally capable as a baby born today. We just have to figure out how many years ago that was. It wasn't last decade. As far as I know most anthropologists agree it was around ~70k years ago (not 200k).
I could gather that you disagreed with GP, but I don't see a salient point in your response? You are ostensibly challenging GP on the idea that a homo sapien baby from 200,000 years ago would have been capable of modern mental feats if raised in the present day.
> This is a bit of an anti-evolutionary perspective.
Nice, seems like you have something meaningful to add.
> At some point in our past, we were something much less intelligent than we are now.
I agree with this, but "at some point in our past"? Is that the essence of this rebuttal?
> Our intelligence didn't spring out of thin air.
Again, I could not tell what this means, nor do I see the relevance.
> Whether or not AI can evolve is yet to be seen I think.
The OP is very pointedly talking about LLMs. Is that what you mean to reference here with "AI"?
I implore you to contribute more meaningfully. Especially when leading with statements like "This is a bit of an anti-evolutionary perspective", you ought to elaborate on them. However, your username suggests maybe you are just trolling?
If you think you are equipped to discuss the topic of evolution of general intelligence in homo, and you haven't read about GWAS and EDU PGS, then at this point you are either a naive layman, or a convinced discourse commando.
Because it is really hard and hopeless endeavor to make an objective case that the current human populations have similar PGS scores on key mental traits and diseases compared to 200k years ago.
The myth that humans remain unchanged for 200k years is forever parroted as truth.
What is the origin of this silly myth? Its come from either anatomical similarity of fossils to modern day human or a comparison to modern (5k ago) humans being conflated with 200k humans
> caveman from 200K years ago would have been just as intelligent as any of us here today, despite not having language
There is evidence to the contrary. Not having language puts your mental faculties in a significant disadvantage. Specifically, left brain athropy. See the critical period hypothesis. Perhaps you mean lacking spoken language rather than having none at all?
How do you arrive at the statement that a cavemen would have the same intelligence as a human today? Intelligence is surely not usually defined as the cognitive potential at birth but as the current capability. And the knowledge an average human has today through education surely factors into that.
Your attempt to commingle intelligence and knowledge is not needed to support your initial question. The original statement that a caveman 200K years ago would have the same intelligence as a modern human was blankly asserted without any supporting evidence, and so it is valid to simply question the claim. You do not need to give a counterclaim, as that is unnecessarily shifting the burden of proof.
The knowledge that everything is made out of atoms/molecules however makes it much easier to reason about your environment. And based on this knowledge you also learn algorithms, how to solve problems etc. I dont think its possible to completely separate knowledge from intelligence.
But an intelligent being could learn that, do you think they become more intelligent if you tell them things are made out of atoms? To me the answer is very simple, no they don't become more intelligent.
There’s a lot of research out there about the general flexibility of the brain to adapt to whatever stimulus you pump into it. For example taxi cab drivers have larger areas in their hippocampus dedicated to place cells relative to the general population [1]. There’s also all kinds of work studying general flexibility of the brain in response to novel stimulus like the visual cortex of blind people being dedicated to auditory processing [2 is a broad review]. I guess you could argue that the ability to be flexible is intelligence but the timescales over which a brain functionally changes is longer than a general day to day flexibility. Maybe some brains come into an initial state that’s more predisposed to the set of properties that we deem as “intelligence” but development is so stimulus dependent that I think this definition of a fixed intelligence is functionally meaningless. There are definitely differences in what you can learn as you age but anyone stating we have any causal measure of innate intelligence is claiming far more than we actually have evidence for. We have far more evidence to suggest that we can train at least the appearance and usage of “intelligence”. After all no one is born capable of formal logical reasoning and it must be taught [3,4 kind of weak citations foe this claim but there’s a lot to suggest this that I don’t feel like digging up]
Would you also say that you cannot "train" intelligence?
I would agree that generally, purely acquiring knowledge does not increase intelligence. But I would also argue that intelligence (ie your raw "processing power") can be trained, a bit like a muscle. And acquiring and processing new knowledge is one of the main ways we train that "muscle".
There's lots of examples where your definition of intelligence (intelligence == raw processing power) either doesn't make sense, or is so narrow that it becomes a meaningless concept. Let's consider feral children (ie humans growing up among animals with no human contact). Apparently they are unable or have trouble learning a human language. There's a theory that there's a critical period after which we are unable to learn certain things. Wouldn't the "ability to learn a language" be considered intelligence? Would you therefore consider a young child more intelligent than any adult?
And to answer your question, whether learning about atoms makes you more intelligent: Yes, probably. It will create some kind of connections in your brain that didn't exist before. It's a piece of knowledge that can be drawn upon for all of your thinking and it's a piece of knowledge that most humans would not figure out on their own. By basically any sensible definition of intelligence, yes it does improve your intelligence.
The claim that books know things seems suspicious to me. I consider the act of knowing to be embodied, it is something a person has learned to do and has control over.
Is that how you approach PDF files? Do you feel it in your bones that these flows of bytes are knowing?
> The claim that books know things seems suspicious to me
I didn't say the book knows things, but everyone can agree that books has knowledge in them. Hence something possessing knowledge doesn't make it intelligent.
For example, when ancient libraries were burnt those civilizations lost a lot of knowledge. Those books possessed knowledge, it isn't a hard concept to understand. Those civilizations didn't lose intelligence, the smart humans were still there, they just lost knowledge.
Would you consider taking a dump and then butchering an animal and then eating without washing your hands first, to be an issue of intelligence or knowledge?
The whole thing about washing hands comes from (some approximation of) germ theory of illness, and in practice, it actually just boils down to stories of other people practicing hygiene. So if one's answer here isn't "knowledge", it needs some serious justification.
Expanding that: can you think of things that are "intelligence" that cannot be reduced like this to knowledge (or combination of knowledge + social expectations)?
I think in some sense, separating knowledge and intelligence is as dumb a confusion of ideas as separating "code" and "data" (doesn't stop half the industry from believing them to be distinct thing). But I'm willing to agree that hardware-wise, humans today and those from 10 000 years ago, are roughly the same, so if you teleported an infant from 8000 BC to this day, they'd learn to function in our times without a problem. Adults are another thing, brains aren't CPUs, the distinction between software and hardware isn't as clear in vivo as it is in silico, due to properties of the computational medium.
Because comparing the human brain and the way it is thinking and seeing and interacting to/with the world to physical/mechanical things like CPU/SSD brings with it huge abstraction gaps, to the point of making the comparison null.
The definitions of the words are contingent on human experience, even more so than "code" are "data" where we try to be more mechanistic, and still most people make the mistake of thinking they're distinct categories (spoiler: they're not; whether something is "code" or "data" depends entirely on your perspective).
If we want to draw computing device analogies, then the brain is an FPGA that is continuously reconfiguring itself throughout its runtime.
I think the core idea is that if a baby with "caveman genetics" so to speak were to be born today, they could achieve similar intellectual results to the (average?) rest of the population.
At least that's how I interpret it.
It's even sillier than that. You can look at populations in the modern world and see there are huge differences in intelligence due to various factors such as cousin marriage and nutrition.
> A caveman from 200K years ago would have been just as intelligent as any of us here today, despite not having language or technology, or any knowledge.
Doubt. If we would teleport cavemen babies right out of the womb to our times, I don't think they'd turn into high IQ individuals. People knowledgeable on human history / human evolution might now the correct answer.
From what I understand, in terms of genetic changes to intellectual abilities, there's not much evidence to suggest we're so much smarter that your proposed teleported baby would be noticeably stupider - at best they'd be on the tail of the bell curve, well within a normal distribution. Maybe if we teleported ten thousand babies, their bell curve would be slightly behind ours. Take a look at "wild children" for the very few examples we can find of modern humans developed without culture. Seems like above everything, our culture, society, and thus education is what makes us smart. And our incredibly high calorie food, of course.
That is exactly what civilization is about - for new generations to start not from scratch, but from some baseline their parents achieved (accumulated knowledge and culture). This allows new generations to push forward instead of retreading the same path.
it's impossible to prove the counterfactual (I guess, as I imagine we don't have enough gene information that far back). But I'd imagine that the high calorie food you can get starting with the advent of agriculture is exactly what could drive evolution in a certain direction that helps brains grow. We've had ~1000 generations since then, that should be enough for some change to happen. Our brains use up 20% of the body's energy. Do we know that this was already the case during the stone age?
The advent of agriculture did not provide better food, it was just the only solution to avoid extinction due to the lack of food.
The archaeological evidence shows that for many generations the first neolithic farmers had serious health problems in comparison with their ancestors. Therefore it is quite certain that they did not transition to agriculture willingly, but to avoid starvation.
Later, when the agriculturalists have displaced everywhere the hunter-gatherers, they did not succeed to do this because they were individually better fed or stronger or smarter, but only because there were much more of them.
The hunter-gatherers required very large territories from which to obtain enough food. For a given territory size, practicing agriculture could sustain a many times greater population, and this was its advantage.
The maximum human brain size had been reached hundreds of thousands years before the development of agriculture, and it regressed a little after that.
There is a theory, which I consider plausible, that the great increase in size of the human brain has been enabled by the fact that humans were able to extract bone marrow from bones, which provided both the high amount of calories and the long-chain fatty acids that are required for a big brain.
I've seen the bone marrow hypothesis also, which is very interesting. Afaik. evidence shows at least that there was enough specialization during neolithic era to have bone marrow cooks where the hunters delivered their bones. Something you wouldn't expect based on just school knowledge (at least back in 90s/2000s).
I see your point about agriculture at first degrading quality of food. Are you aware of evidence of brain size degrading even? Is it visible in the temple bones?
It is known that 200k years ago human brain sizes were actually greater than today, even if this does not necessarily correlate with a lower IQ in the present, because it is more likely that the parts of the brain that have reduced may have been related with things like fine motor skills and spatial orientation, which are no longer important today for most people.
A human being has the potential for intelligence. For that to get realized, you need circumstances, you need culture aka "societal" software and the resources to suspend the grind of work in formative years and allow for the speed-running of the process of knowledge preloading before the brain gets stable.
The parents then must support this endeavor under sacrifices.
There is also a ton of chicken-egg catch22s buried in this whole thing.
If the society is not rich then no school, instead childlabour. If child-labour society is pre-industrial ineffective and thus, no riches to support and redistribute.
Also is your societies culture root-hardened. Means - on a collapse of complexity in bad times, can it recover even powering through the usual "redistribute the nuts and bolts from the bakery" sentiments rampant in bad times. Can it stay organize and organize centralizing of funds for new endeavors. Organizing a sailing ship in a medieval society, means in every village 1 person starves to death. Can your society accomplish that without riots?
Of course they were. A human from 200,000 years ago would be almost genetically identical to one from today. That's what makes us homo sapiens. 200,000 years is absolutely nothing on an evolutionary timescale with generations as long as ours and reproduction rates as low as ours.
A human is more then the hardware.
A human is hardware with cultural software.
A human is decorated with parental education.
A human is decorated with local cultural influences.
A human inherits his economic circumstances and behavior.
A modern human is a complex artifact and can not be produced everywhere.
The ability to cooperate, form institutions and build complex tools may be severely restricted even today. Of course its also restricted in the past.
A human is decorated with local cultural influences.
A human inherits his economic circumstances and behavior.
A modern human is a complex artifact and can not be produced everywhere.
The ability to cooperate, form institutions and build complex tools may be severely restricted even today. Of course its also restricted in the past.
We all come from monke, monkey from 10 million years ago would definitely be unable to even learn spoken language at a basic level. Would he even have the anatomy to produce the required sounds? I don't think so.
What about monke from 1 million years ago? 200 thousand years ago?
ChatGpt says spoken language only emerged 50k - 200k years ago and that a cavemen baby from 200k years ago could learn spoken language if brought up by modern parents.
The evolution of the human brain appears to have reached its peak long before 200k years ago.
Nowadays humans have smaller brains on average, though that is almost certainly not correlated with a lower skill in computer programming, but with lower skills in the techniques that one needed to survive as a hunter of big animals.
How could we know this? AFAIK all we can say is the volume of the brain has been relatively stable for that long, how can we say the structures of the brain have not evolved since then? It seems plausible to me anyway that humans could have co-evolved with ideas in a way.
Ah yes, the 0.50$/h support infrastructure from the places that cannot refuse the deal. "frontier" LLMs currently cosplay a dunk with google and late alzheimer's. Surely, they speed up brute-forcing correct answer a lot by trying more likely texts. And? This overfed markov chain doesn't need supporing infrastructure — it IS supporting infastructure, for the cognitive something that is not being worked on prominently, because all resources are needed to feed the markov chain.
The silence surrounding new LLM architectures is so loud that an abomination like "claw" gets prime airtime. Meanwhile models keep being released. Maybe the next one will be the lucky draw. It was pure luck, finding out how well LLMs scale, in the first place. Why shouldn't the rest of progress be luck driven too?
Pretty much, it's just that these overfed Markov chains when given a proper harness and agentic framework are able to produce entire software projects in a fraction of the time it used to take.
Sorry, I tought you meant "support infrastructure" in a much wider sense — yeah, LLMs are frighteningly good at lockpicking tests using source code shaped inputs. It's just that they are also frighteningly good at finding insane ways to game the tests, too. I wouldn't say that LLMs are very "G" in the AI they do — present them with confusing semantics, and they fall off the self-contradiction cliff. No capability of developing theory systematically from observations.
> A caveman from 200K years ago would have been just as intelligent as any of us here today, despite not having language or technology, or any knowledge.
Source? This does not sound possibly true to me (by any common way we might measure intelligence).
I'm not disagreeing that humans 200,000 years ago were approximately anatomically equivalent to humans today; I'm disagreeing that they would be just as intelligent without today's language, technology, or knowledge. I don't think you can define or measure intelligence in a way that ignores those things.
Eh I don’t think it’s something we can ever discount. Some cavewoman could’ve daydreamed the entire theory of general relativity in her own private language while weaving a basket and we would never know because she never felt the need to talk about it. On the other hand there are people today who will emit novels of profound nothingness.
Technology and language is sort of like speaking in this sense, it’s evidence of mind but it’s not mind. And the absence of evidence is not the evidence of absence and all that
I posted my own comment but I agree with you. Our modern society likes to claim we are somehow "more intelligent" than our predecessors/ancestors. I couldn't disagree more. We have not changed in terms of intelligence for thousands of years. This is a matter that's beyond just engineering, it's also a matter of philosophy and perspective.
200k years just isn't much time for significant evolutionary changes considering the human population "reset" a couple times to very very small numbers.
Reich's lab actually found evidence of meaningful genetic changes that improved intelligence over the past 10,000 years, but not so much prior to that:
Our big brains are a recent mutation and haven't been fully field tested. They seem like more of a liability than anything, they've created more existential risks for us than they've put to rest.
Yes, but also antibiotics, vaccinations, child mortality down down down, life expectancy up up up. I wouldn't trade for living even 100 years prior compared to today, or 500-200k years ago for that matter.
With everything wrong and sick with today's world, let's not take the achievements of our species for granted.
You wouldn't make that trade because you are part of the last generation (loosely speaking, a collection of generations) before it all comes crumbling down. We are living unbelievably privileged lives because we are burning all of the world's resources to the ground. In the process, we're destroying the ecosystem and driving a mass extinction event. Nothing about the way we live is sustainable long-term. We're literally consuming hundreds of millions of years worth of planet-wide resource buildup over a span of a couple of centuries. Even if we avoid the worst case scenario, humans 200 years from now will almost certainly not be able to live anywhere near as luxuriously as we do now, unless there's a culling of billions. In the actual worst case scenario, we may render the planet uninhabitable for anything we regard as intelligent life.
In that sense, we have just enough collective intelligence to be dangerous and not enough intelligence to moderate ourselves, which may very well result in an evolutionary deadend that will have caused untold damage to life on Earth.
We also live in an era we can create hydrocarbon fuel DIRECTLY from the atmosphere and desalinate fresh water in unlimited supply, from power derived directly from the sun or atomics.
We also live in a time where the human population, where it is most concentrated, is declining rather than growing, so far without too disastrous consequences.
Greening of the earth has been happening since the 1980s- i.e. about a .3% coverage increase per year in recent decades.
Places that were miserable and poor, like China, have been lifted to prosperity and leading out in renewable tech.
There is much to celebrate and after the recent passing of Paul Ehrlich, we should pause and consider just how wrong pretty much every prediction he made was.
You lost me when you started narrating the fossil doom visage.
With the current progress in solar, as well as the remaining coal, gas and uranium reserves, energy is not going to be what finishes our civilization.
While I don't think we are going to get true collapse, I think we are going to get a lot of technical progress compensating for biosocial deterioration.
The demographics, mental health and dysgenics are all real, quantified trends, and we are going to face the reality of less capable, less taxable population for the rest of this century. It's baked in at this point.
Doomerism is a kind of religion that goes back as far as they eye can see. What's interesting about it is that in spite of being perpetually incorrect in its myriad predictions, it continues to adapt and attract new adherents.
See also (recent only):
- Paul Ehrlich's Population Bomb (Malthusian collapse)
- The Club of Rome's The Limits of Growth (resource exhaustion)
- Thomas Malthus' Population growth / famine cycle
- James Lovelock's Global warming catastrophe predictions
I am not a doomer, nor a Malthusian, merely a realist. There are a few points I could make briefly:
- Everything lasts forever, until it doesn't. Ancient Egyptian civilization lasted for thousands of years, until it didn't. Any Egyptian could point to thousands of years of their heritage and say it hasn't ended yet, therefore any prediction that it will end is clearly bad and dumb. Then it was conquered by Romans, and then by Islam, with its language, culture, and religion extinguished, extant only in monuments, artifacts and history books.
- We have nuclear weapons now. Any prediction of an imminent end of human civilization before then would be purely religious, but there is a real reason to believe things have changed. We are currently in a time of relative peace secured by burning resources for prosperity, but what happens when those resources run out and world conflict for increasingly scarce resources is renewed with greater vigor?
- Note that I did not outright predict the end of human civilization, merely noted it as a plausible worst-case scenario. If civilization continues on more-or-less as it is, in the next couple of hundred years, we will drive countless more species to extinction. We will destroy so much more of our environment with climate change, deforestation, strip mining, overfishing, pollution, etc. We will deplete water reservoirs and we will deplete oil, helium, phosphorus, copper, zinc, and various rare earth elements. Not a complete depletion, but they will become so scarce as to not be widely available or wasted for the general population's benefit. If billions of people are still alive then, which I explicitly suggested was a possibility, they will as a simple matter-of-fact live much less comfortably prosperous lives than us. It will not take a great catastrophe to result in a massive reduction in living standards, because our current living standards are inherently unsustainable.
That seems both fatalistic and doomerist to me, but time will tell. I would assume germ theory would survive regardless, as would immunology, so I'd hold on to those two at least.
Humans, like all animals, have not stopped evolving. A random caveman from 200K years ago would have very different genetics to that of a typical HN reader and even more so of the best of the HN readers.
Around 3,200 years ago there was a notable uptick in alleles associated with intelligence.
"open source" has a specific definition[0], which this project does not meet. When people say "open source", that is the definition that they are referencing. It's the reason why there's been endless discussion about "open weights" models not being "open source".
"source available"[1] is a different thing, and you're right that this project is "source available".
Yeah like I remember prior to reasoning models, their guidance was to use <think> tags to give models space for reasoning prior to an answer (incidentally, also the reason I didn't quite understand the fuss with reasoning models at first). It's always been XML with Anthropic.
Exactly the same story here. I still use a tool that just asks them to use <think> instead of enabling native reasoning support, which has worked well back to Sonnet 3.0 (their first model with 'native' reasoning support was Sonnet 3.7)
I'm a dev, not a salesperson, but let's be realistic. A company tells you "yeah we're interested in signing at $1M/yr, but we really need this feature, when will you have it by?", to which saying "eh we don't know - it'll be done when it's done" will lead to the company saying "ok well reach out when you have it, we can talk again then" (or just "eh ok then not a good fit sorry bye"), and in the meantime they'll go shopping around and may end up signing with someone else.
Having a promised date lets you keep the opportunity going and in some cases can even let you sign them there and then - you sign them under the condition that feature X will be in the app by date Y. That's waaaay better for business, even if it's tougher for engineers.
“Sign up and pay at least part of it now and we’ll prioritize the feature”.
I’ve seen enough instances of work being done for a specific customer that doesn’t then result in the customer signing up (or - once they see they can postpone signing the big contract by continuing to ask for “just one more crucial feature”, they continue to do so) to ever fall for this again.
Why do that if your competitor already has it? I'd just go talk to the competitor instead. If you aren't able to ballpark when the feature will be done, why should I trust you will once I pay part of the price?
Because you have other benefits, so we'd really like to switch over to you, but we can't unless you support this dealbreaker feature that your competitor we're currently using has.
Just to consider the opposite viewpoint, I sometimes wonder if it's not better that they do churn in that case.
Assuming the sales team is doing their job properly, there are other prospects who may not need that feature, and not ramming the feature in under time constraints will lead to a much better product.
Eventually, their feature will be built, and it will have taken the time that it needed, so they'll probably churn back anyway, because the product from the vendor they did get to ram their feature in is probably not very good.
I understand the intuition, but it's a misunderstanding of how software sales operates. There's no tradeoff between prospects who need new features and prospects who don't, because salespeople love that second category and you'll have no problem hiring as many as you need to handle all of them.
Unless its the first time they are hearing about it, when a customer asks about a feature, sales should've done their homework and checked with the team doing the work to get a rough estimate instead of pulling a number out of their behinds.
I use this approach for a ticket based customer support agent. There are a bunch of boolean checks that the LLM must pass before its response is allowed through. Some are hard fails, others, like you brought up, are just a weighted ding to the response's final score.
Failures are fed back to the LLM so it can regenerate taking that feedback into account. People are much happier with it than I could have imagined, though it's definitely not cheap (but the cost difference is very OK for the tradeoff).
I agree - and it's not just what gets you promoted, but also what gets you hired, and what people look for in general.
You're looking for your first DevOps person, so you want someone who has experience doing DevOps. They'll tell you about all the fancy frameworks and tooling they've used to do Serious Business™, and you'll be impressed and hire them. They'll then proceed to do exactly that for your company, and you'll feel good because you feel it sets you up for the future.
Nobody's against it. So you end up in that situation, which even a basic home desktop would be more than capable of handling.
I have been the first (and only) DevOps person at a couple startups. I'm usually pretty guilty of NIH and wanting to develop in-house tooling to improve productivity. But more and more in my career I try to make boring choices.
Cost is usually not a huge problem beyond seed stage. Series A-B the biggest problem is growing the customer base so the fixed infra costs become a rounding error. We've built the product and we're usually focused on customer enablement and technical wins - proving that the product works 100% of the time to large enterprises so we can close deals. We can't afford weird flakiness in the middle of a POC.
Another factor I rarely see discussed is bus factor. I've been in the industry for over a decade, and I like to be able to go on vacation. It's nice to hand off the pager sometimes. Using established technologies makes it possible to delegate responsibility to the rest of the team, instead of me owning a little rats nest fiefdom of my own design.
The fact is that if 5k/month infra cost for a core part of the service sinks your VC backed startup, you've got bigger problems. Investors gave you a big pile of money to go and get customers _now_. An extra month of runway isn't going to save you.
The issue is when all the spending gets you is more complexity, maintenance, and you don't even get a performance benefit.
I once interviewed with a company that did some machine learning stuff, this was a while back when that typically meant "1 layer of weights from a regression we run overnight every night". The company asked how I had solved the complex problem of getting the weights to inference servers. I said we had a 30 line shell script that ssh'd them over and then mv'd them into place. Meanwhile the application reopened the file every so often. Zero problems with it ever. They thought I was a caveman.
I work for a small company with a handful of devs. We don't have a dedicated devops person, so I do it all. Everything is self-hosted. Been that way for years. But, yeah, if I go on vacation and something foes screwy, the business is hosed. However, even if it were hosted on AWS or elsewhere, it would not be any better. If anything, it may be worse. Instead of a person being well versed in standards based tech, they'd have to be an AWS expert. Why would we want that?
I have recently started using terraform/tofu and ansible to automate nearly all of the devops operations. We are at a point where Claude Code can use these tools and our existing configs to make configuration changes, debug issues by reviewing logs etc. It is much faster at debugging an issue than I am and I know our stuff inside and out.
I am beginning to think that AI will soon force people to rethink their cloud hosting strategy.
I identify as a caveman and I fucking love it. I build a 250k sloc C++ project hundreds of times a day with a 50 line bash script. Works every time, on any machine, everywhere.
Those scripts have logs, right? Log a hostname and path when they run. If no one thinks to look at logs, then there's a bigger problem going on than a one-off script.
That becomes a problem if you let the shell script mutate into an "everything" script that's solving tons of business problems. Or if you're reinventing kubernetes with shell scripts. There's still a place for simple solutions to simple problems.
You can literally have a 20 line Python script on cron that verifies if everything ran properly and fires off a PagerDuty if it didn't. And it looks like PagerDuty even supports heartbeat so that means even if your Python script failed, you could get alerted.
Which is why you take the time to put usage docs in the repo README, make sure the script is packaged and deployed via the same methods that the rest of the company uses, and ensure that it logs success/failure conditions. That's been pretty standard at every organization I've been at my entire professional career. Anyone who can't manage that is going to create worse problems when designing/building/maintaining a more complex system.
Yah. A lot of the complexity in data movement or processing is unneeded. But decent standardized orchestration, documentation, and change management isn't optional even for the 20 line shell script. Thankfully, that stuff is a lot easier for the 20 line standard shell script.
Or python. The python3 standard library is pretty capable, and it's ubiquitous. You can do a lot in 50-100 lines (counting documentation) with no dependencies. In turn it's easy to plug into the other stuff.
I've seen the ramifications of this "CV first" kind of engineering. Let's just say that it's a bad time when you're saddled with tech debt solely from a handful of influential people that really just wanted to work elsewhere.
This. It is resume-driven development. Especially at startups where the engineers aren't compensated well enough or don't believe the produce can succeed.
I've seen that in a few places, yeah! I think I personally would just put something in the footer and have a specific page for it that I can link people to.
I really hope that I never end up in a situation where someone tells me "well the conversion rate would be much higher if you just stopped fighting it and put up the damn banner".
Most of the changes are completely reasonable - a lot are internal cleanup that would require no code changes on the user side, dropping older browsers, etc.
But the fact that there are breaking API changes is the most surprising thing to me. Projects that still use jQuery are going to be mostly legacy projects (I myself have several lying around). Breaking changes means more of an upgrade hassle on something that's already not worth much of an upgrade hassle to begin with. Removing things like `jQuery.isArray` serve only to make the upgrade path harder - the internal jQuery function code could literally just be `Array.isArray`, but at least then you wouldn't be breaking jQuery users' existing code.
At some point in the life of projects like these, I feel like they should accept their place in history and stop themselves breaking compatibility with any of the countless thousands (millions!) of their users' projects. Just be a good clean library that one can keep using without having to think about it forever and ever.
I don’t understand your use case. If you’ve got legacy projects that you don’t want to touch, why upgrade a dependency to a new major version?
You can keep using jquery without having to think about it. Just keep using version 3.7 and don’t even think about version 4.
I recently had to upgrade from jQuery 2 to the latest version, because an client demanded it (security issues), and just ran into compatibility issues with third party libs/plugins.
I love the interview at the end of the video. The kubectl-inspired CLI, and the feedback for improvements from Claude, as well as the alerts/segmentation feedback.
You could take those, make the tools better, and repeat the experience, and I'd love to see how much better the run would go.
I keep thinking about that when it comes to things like this - the Pokemon thing as well. The quality of the tooling around the AI is only going to become more and more impactful as time goes on. The more you can deterministically figure out on behalf of the AI to provide it with accurate ways of seeing and doing things, the better.
Ditto for humans, of course, that's the great thing about optimizing for AI. It's really just "if a human was using this, what would they need"? Think about it: The whole thing with the paths not being properly connected, a human would have to sit down and really think about it, draw/sketch the layout to visualize and understand what coordinates to do things in. And if you couldn't do that, you too would probably struggle for a while. But if the tool provided you with enough context to understand that a path wasn't connected properly and why, you'd be fine.
I see this sentiment of using AI to improve itself a lot but it never seems to work well in practice. At best you end up with a very verbose context that covers all the random edge cases encountered during tasks.
For this to work the way people expect you’d need to somehow feed this info back into fine tuning rather than just appending to context. Otherwise the model never actually “learns”, you’re just applying heavy handed fudge factors to existing weights through context.
I've been playing around with an AI generated knowledge base to grok our code base, I think you need good metrics on how the knowledge base is used. A few things is:
1. Being systematic. Having a system for adding, improving and maintaining the knoweldge base
2. Having feedback for that system
3. Implementing the feedback into a better system
I'm pretty happy I have an audit framework and documentation standards. I've refactored the whole knowledge base a few times. In the places where it's overly specific or too narrow in it's scope of use for the retained knowledge, you just have to prune it.
Any garden has weeds when you lay down fertile soil.
Sometimes they aren't weeds though, and that's where having a person in the driver's seat is a boon.
The features it asked for in this case were better tools, I thought they were really sensible. It said it wanted a —dry-run (like the CLIs the rct one was modelled on), it wanted to be able to segment guest feedback, and it wanted better feedback from its path tools. Those might not be actually possible in rct, but in a different context they’re pretty smart requests and not just verbose edge cases.
That's not what's happening here, and it's worth remembering: A caveman from 200K years ago would have been just as intelligent as any of us here today, despite not having language or technology, or any knowledge.
In Carolyn Porco's words: "These beings, with soaring imagination, eventually flung themselves and their machines into interplanetary space."
When you think of it that way, it should be obvious that LLMs are not AGI. And that's OK! They're a remarkable piece of technology anyway! It turns out that LLMs are actually good enough for a lot of use cases that would otherwise have required human intelligence.
And I echo ArekDymalski's sentiment that it's good to have benchmarks to structure the discussions around the "intelligence level" of LLMs. That _is_ useful, and the more progress we make, the better. But we're not on the way to AGI.
reply