Enough billions of dollars have been spent on LLMs that a reasonably good picture of what they can and can't do has emerged. They're really good at some things, terrible at others, and prone to doing something totally wrong some fraction of the time. That last limits their usefulness. They can't safely be in charge of anything important.
If someone doesn't soon figure out how to get a confidence metric out of an LLM, we're headed for another "AI Winter".
Although at a much higher level than last time. It will still be a billion dollar industry, but not a trillion dollar one.
At some point, the market for LLM-generated blithering should be saturated. Somebody has to read the stuff. Although you can task another system to summarize and rank it. How much of "AI" is generating content to be read by Google's search engine? This may be a bigger energy drain than Bitcoin mining.
It’s probably generally irrelevant what they can do today, or what you’ve seen so far.
This is conceptually essentially Moore’s law, but about every 5.5 months. That’s the only thing that matters at this stage.
I watched everyone make the same arguments about the general Internet, and then the Web, then mobile, then Bitcoin. It’s just a toy. It’s not that useful. Is this supposed to be the revolution? It uses too much power. It won’t scale. The technology is a dead end.
The general pattern of improvement to technology has been radically to the upside at an increasing pace parabolically for decades and there’s nothing indicating that this is a break in the pattern. In fact it’s setting up to be an order of magnitude greater impact than the Internet was. At a minimum, I don’t expect it to be smaller.
Looking at early telegraphs doesn’t predict the iPhone, etc.
>> Looking at early telegraphs doesn’t predict the iPhone, etc.
The problem with this line of argument is that LLMs are not new technology, rather they are the latest evolution of statistical language modelling, a technology that we've had at least since Shannon's time [1]. We are way, way past the telegraph era, and well into the age of large telephony switches handling millions of calls a second.
Does that mean we've reached the end of the curve? Personally, I have no idea, but if you're going to argue we're at the beginning of things, that's just not right.
________________
[1] In "A Mathematical Theory of Communication", where he introduces what we today know as information theory, Shannon gives as an example of an application a process that generates a string of words in natural English according to the probability of the next letter in a word, or the next word in a sentence. See Section 3 "The Series of Approximations to English":
I think we can pretty safely say bitcoin was a dead end other than for buying drugs, enabling ransomware payments, or financial speculation.
Show me an average person who has bought something real w bitcoin (who couldn’t have bought it for less complexity/transaction cost using a bank) and I’ll change my mind
Yes but they’re also anonymous. You don’t have your name attached to the account and there’s no paperwork/bank that’s keeping track of any large/irregular financial transactions
I heard this as one of the early sales pitches for Bitcoin. “Digital cash.”
That all seemed to go out the window when companies developed wallets to simplify the process for the average user, and when the prices surged, some started requiring account verification to tie it to a real identity. At that point, it’s just a bank with a currency that isn’t broadly accepted. The idea of digital cash was effectively dead, at least for the masses who aren’t going to take the time to figure out how to use Bitcoin without a 3rd party involved. Cash is simple.
No, not exactly. If you know someone used cash at one place can you track every cash transaction they've ever made? If you know one bitcoin transaction from a wallet you can track everything that key pair has done from genesis to present. So, if anything, it's worse.
Bitcoin failed because of bad monetary policy turning it into something like a ponzi scheme where only early adopters win. The monetary policy isn't as hard to fix as people make it out to be.
Speaking of the iPhone, I just ugpraded to the 16 Pro because I want to try out the new Apple Intelligence features.
As soon as I saw integrated voice+text LLM demos, my first thought was that this was precisely the technology needed to make assistants like Siri not total garbage.
Sure, Apple's version 1.0 will have a lot of rough edges, but they'll be smoothed out.
In a few versions it'll be like something out of Star Trek.
"Computer, schedule an appointment with my Doctor. No, not that one, the other one... yeah... for the foot thing. Any time tomorrow. Oh thanks, I forgot about that, make that for 2pm."
Try that with Siri now.
In a few years, this will be how you talk to your phone.
The issue with appointments is the provider needs to be integrated into the system. Apple can’t do that on their own. It would have to be more like the roll out of CarPlay. A couple partners at launch, a lot of nothing for several years, and eventually is a lot of places, but still not universal.
I could see something like Uber or Uber Eats trying to be early on something like this, since they already standardized the ordering for all the restaurants in their app. Scheduling systems are all over the place.
I meant appointment in the "calendar entry category" sense, where creating an appointment is entirely on-device and doesn't involve a third party.
Granted, any third-party integrations would be a significant step up from my simple scenario of "voice and text comprehension" and local device state manipulation.
Many situations, prefer text to voice. Text: Easier record keeping, manipulation, search, editing, ....
With some irony, the Hacker News
user interface is essentially all
just simple text.
A theme in current computer design seems to be: Assume the user doesn't use a text editor and, instead, needs an 'app' for every computer interaction. Like cars for people who can't drive, and a car app for each use of the car -- find a new BBQ restaurant, need a new car app.
Sorry, Silicon Valley, with text anyone who used a typewriter or pocket calculator can do more and have fewer apps and more flexibility, versatility, generality.
I am generally on your side of this debate, but Bitcoin is a reference that is in favor of the opposite position. Crypto is/was all hype. It's a speculative investment, that's all atm.
Bitcoin is the only really useful crypto that fundamentally has no reason to die because of basic economics. It is fundamentally the only hard currency we have ever created and that's why it is revolutionary
I find it hard to accept the statement that "[bitcoin] is fundamentally the only hard currency we have ever created". Is it saying that gold back currencies were not created by us, or that gold isn't hard enough?
Additionally, there's a good reason we moved off deflationary hard currencies and onto inflationary fiat currencies. Bitcoin acts more like a commodity than a medium of exchange. People tend to buy it, hold it, and then eventually cash out. If I am given a bunch of bitcoin, the incentive is for me not to spend it, but rather keep it close and wait for it to appreciate — what good is a currency that people don't spend?
Also I find it weird when I read that due to its mathematically proven finite supply it is basic economics that gives it value. Value in modern economics is defined as what people are willing to give up to obtain that thing. Right now, people are willing to give up a lot for bitcoin, but mainly because other people are also willing to give up a lot for bitcoin, which gives it value.
It's a remarkable piece of engineering that has enabled this (solving the double spending problem especially), but it doesn't have inherent value in and if itself. There are many finite things in the world that are not valued as highly as bitcoin is. There's a finite number of beanie babies, a finite number is cassette tapes, a finite number of blockbuster coupons...
Gold is similar — should we all agree tomorrow that gold sucks and should never be regarded as a precious metal, then it won't lose its value completely (there's only a finite amount of it, and some people will still want it, e.g. for making connectors). But its current valuation is far higher than it would be for its scarcity alone — people mainly want gold, because other people want gold.
Hello everyone! With immense joy in my heart, I want to take a moment to express my heartfelt gratitude to an incredible lottery spell psychic, Priest Ray. For years, I played the lottery daily, hoping for a rewarding outcome, but despite my efforts and the various tips I tried, success seemed elusive. Then, everything changed when I discovered Priest Ray. After requesting a lottery spell, he cast it for me and provided me with the lucky winning numbers. I can't believe it—I am now a proud lottery winner of $3,000,000! I felt compelled to share my experience with all of you who have been tirelessly trying to win the lottery. There truly is a simpler and more effective way to achieve your dreams. If you've ever been searching for a way to increase your chances of winning, I encourage you to reach out via email: psychicspellshrine@gmail.com
People discussing AI certainly is the perfect place to advertise something about the "incredible lottery spell psychic". Honestly, I can't even tell if it's satire or spam, and I love it.
"They're really good at some things, terrible at others, and prone to doing something totally wrong some fraction of the time."
I agree 100% with this sentiment, but, it also is a decent description of individual humans.
This is what processes and control systems/controls are for. These are evolving at a slower pace than the LLMs themselves at the moment so we're looking to the LLM to be its own control. I don't think it will be any better than the average human is at being their own control, but by no means does that mean it's not a solvable problem.
> I agree 100% with this sentiment, but, it also is a decent description of individual humans.
But you can understand individual humans and learn which are trustworthy for what. If I want a specific piece of information, I have people in my life that I know I can consult to get an answer that will most likely be correct and that person will be able to give me an accurate assessment of their certainty and they know how to accurately confirm their knowledge and they’ll let me know later if it turns out they were wrong or the information changed and…
None of that is true with LLMs. I never know if I can trust the output, unless I’m already an expert on the subject. Which kind of defeats the purpose. Which isn’t to say they’re never helpful, but in my experience they waste my time more often than they save it, and at an environmental/energy cost I don’t personally find acceptable.
It defeats the purpose of LLM as personal expert on arbitrary topics. But the ability to do even a mediocre job with easy unstructured-data tasks at scale is incredibly valuable. Businesses like my employer pay hundreds of professionals to run business process outsourcing sites where thousands of contractors repeatedly answer questions like "does this support contact contain a complaint about X issue?" And there are months-long lead teams to develop training about new types of questions, or to hire and allocate headcount for new workloads. We frequently conclude it's not worth it.
Actually humans are much worse in this regard. The top performer on my team had a divorce and his productivity dropped by like a factor of 3 and quality fell of a cliff.
Another example from just yesterday is I needed to solve a complex recurrence relation. A friend of mine who is good at math (math PhD) helped me for about 30 minutes still without a solution and a couple of false starts. Then he said try ChatGPT and we got the answer in 30s and we spent about 2 minutes verifying it.
I call absolute bullshit on that last one. There's no way ChatGPT solves a maths problem that a maths PhD cannot solve, unless the solution is also googleable in 30s.
Is anything googleable in 30s? It feels like finding the right combination of keywords that bypasses the personalization and poor quality content takes more than one attempt these days.
Right, AI is really just what I use to replace google searches I would have used to find highly relevant examples 10 years back. We are coming out of a 5 year search winter.
>Actually humans are much worse in this regard. The top performer on my team had a divorce and his productivity dropped by like a factor of 3 and quality fell of a cliff.
Wow. Nice of you to see a coworker go through a traumatic life event, and the best you can drudge up is to bitch about lost productivity and decrease in selfless output of quality to someone else's benefit when they are at the time trying to stitch their life back together.
SMH. Goddamn.
Hope your recurrence relation was low bloody stakes. If you spent only two minutes verifying something coming out of a bullshit machine, I'd hazard you didn't do much in the way of boundary condition verification.
> I agree 100% with this sentiment, but, it also is a decent description of individual humans.
But humans can be held accountable, LLMs cannot.
If I pay a human expert to compile a report on something and they decide to randomly make up facts, that's malpractice and there could be serious consequences for them.
If I pay OpenAI to do the same thing and the model hallucinates nonsense, OpenAI can just shrug it and say "oh that's just a limitation of current LLMs".
>also is a decent description of individual humans
A friend of mine was moving from software development into managing devs. He told me: "They often don't do things the way or to the quality I'd like, but 10 of them just get so much more done than I could on my own." This was him coming to terms with letting go of some control, and switching to "guiding the results" rather than direct control.
Your friend got lucky, I've seen (and worked with) people with negative productivity - they make the effort and sometimes they commit code, but it inevitably ends up being broken, and I realize that it would take less of my time for me to write the code myself, rather than spend all the time explaining and then fixing bugs.
>> I agree 100% with this sentiment, but, it also is a decent description of individual humans.
Why would that be a good thing? The big thing with computers is that they are reliable in ways that humans simply can't ever be. Why is it suddenly a success to make them just as unreliable as humans?
I thought the big thing with computers is that they are much cheaper than humans.
If we are evaluating LLM suitability for tasks typically performed by humans, we should judge them by the same standards we judge humans. That means it's OK to make mistakes sometimes.
You missed quoting the next sentence about providing confidence metric.
Humans may be wrong a lot but at least the vast majority will have the decency to say “I don’t know”, “I’m not sure”, “give me some time to think”, “my best guess is”. In contrast to most LLMs today that in full confidence just spews out more hallucinations.
I'll keep buying (and paying premium) for dumber things. Cars are a prime example, I want it to be dumb as fuck, offline, letting me decide what to do. At least next 2 decades, and thats achievable. After that I couldnt care less, I'll probably be a bad driver at that point anyway so switch may make sense. I want dumb beautiful mechanival wristwatch.
I am not ocd-riddled insecure man trying to subconsiously immitate much of the crowd, in any form of fasion. If that will make me an outlier, so be it, a happier one.
I suspect new branch of artisanal human-mind-made trademark is just behind the corner, maybe niche but it will find its audience. Beautiful imperfections, clear clunky biases and all that.
LLMs have been improving exponentially for a few years. let's at least wait until exponential improvements slow down to make a judgement about their potential
They have been improving a lot, but that improvement is already plateauing and all the fundamental problems have not disappeared. AI needs another architectural breakthrough to keep up the pace of advancement.
Based on what ? The gap between the release of GPT-3 and 4 is still much bigger than the time that has elapsed since 4 was already released so really, Based on what ?
there are no much reliable benchmarks which would measure what is gap really. I think currently corps compete in who will leak benchmarks to training data the most, hence o1 is world programming medalist, yet makes stupid mistakes.
I'm not as up-to-speed on the literature as I used to be (it's gotten a lot harder to keep up), but I certainly haven't heard of any breakthroughs. They tend to be pretty hard to predict and plan for.
I don't think we can continue simply tweaking the transformer architecture to achieve meaningful gains. We will need new architectures, hopefully ones that more closely align with biological intelligence.
In theory, the simplest way to real superhuman AGI would be to start by modeling a real human brain as a physical system at the neural level; a real neural network. What the AI community calls "neural networks" are only very loose approximations of biological neural networks. Real neurons are subject to complex interactions between many different neurotransmitters and neuromodulators and they grow and shift in ways that look nothing like backpropagation. There already exist decently accurate physical models for single neurons, but accurately modeling even C. elegans (as part of the OpenWorm project) is still a way's off. Modeling a full human brain may not be possible within our lifetime, but I also wouldn't rule that out.
And once we can accurately model a real human brain, we can speed it up and make it bigger and apply evolutionary processes to it much faster than natural evolution. To me, that's still the only plausible path to real AGI, and we're really not even close.
I was holding out hope for Q*, which OAI talked about with hushed tones to make it seem revolutionary and maybe even dangerous, but that ended up being o1. o1 is neat, but its far from a breakthrough. It's just recycling the same engine behind GPT-4 and making it talk to itself before spitting out its response to your prompt. I'm quite sure they've hit a ceiling and are now using smoke-and-mirrors techniques to keep the hype and perceived pace-of-progress up.
OpenAI's Orion (GPT 5/Next) is partially trained on synthetic data generated with a large version of o1. Which means if that works the data scarcity issue is more or less solved.
OpenAI has the biggest appetite for large models. GPT-4 is generally a bit better than Gemini, for example, but that's not because Google can't compete with it. Gemini is orders of magnitude smaller than GPT-4 because if Google were to run a GPT-4-sized model every time somebody searches on Google, they would literally cease to be a profitable company. That's how expensive inference on these ultra-large models is. OpenAI still doesn't really care about burning through hundreds of billions of dollars, but that cannot last forever.
This, I think, is the crux of it. OpenAI is burning money at a furious rate. Perhaps this is due to a classic tech industry hypergrowth strategy, but the challenge with hypergrowth strategies is that they tend to involve skipping over the step where you figure out if the market will tolerate pricing your product appropriately instead of selling it at a loss.
At least for the use cases I've been directly exposed to, I don't think that is the case. They need to keep being priced about where they are right now. It wouldn't take very much of a rate hike for their end users to largely decide that not using the product makes more financial sense.
They have, Anthropic Claude Sonnet 3.5 is superior to GPT-4o in every way, it's even better then their new o1 model at most things (coding, writing, etc.).
OpenAI went from GPT-4, which was mind blowing, to 4o, which was okay, to o1 which was basically built in chain-of-thought.
No new Whisper models (granted, advanced voice chat is pretty cool). No new Dalle models. And nobody is sure what happened to Sora.
OpenAI had a noticeable head start with GPT-2 in 2019. They capitalized on that head start with ChatGPT in late 2022, and relatively speaking they plateaued from that point onwards. They lost that head start 2.5 months later with the announcement of Google Bard, and since then they've been only slightly ahead of the curve.
It's pretty undeniable that OpenAI's lead has been diminished greatly from the GPT-3 days. Back then, they could rely on marketing their coherency and the "true power" of larger models. But today we're starting to see 1B models that are undistinguishable from OpenAI's most advanced chain-of-thought models. From a turing test perspective, I don't think the average person could distinguish between an OpenAI and a Llama 3.2 response.
In some domains (math and code), progress is still very fast. In others it has slowed or arguably stopped.
We see little progress in "soft" skills like creative writing. EQBench is a benchmark that tests LLM ability to write stories, narratives, and poems. The winning models are mostly tiny Gemma finetunes with single-digit parameter counts. Huge foundation models with hundreds of billions of parameters (Claude 3 Opus, Llama 3.1 405B, GPT4) are nowhere near the top. (Yes, I know Gemma is a pruned Gemini). Fine-tuning > model size, which implies we don't have a path to "superhuman" creative writing (if that even exists). Unlike model size, fine-tuning can't be scaled indefinitely: once you've squeezed all the juice out of a model, what then?
OpenAI's new o1 model exhibits amazing progress in reasoning, math, and coding. Yet its writing is worse than GPT4-o's (as backed by EQBench and OpenAI's own research).
I'd also mention political persuasion (since people seem concerned about LLM-generated propaganda). In June, some researchers tested LLM ability to change the minds of human subjects on issues like privatization and assisted suicide. Tiny models are unpersuasive, as expected. But once a model is large enough to generate coherent sentences, persuasiveness kinda...stops. All large models are about equally persuasive. No runaway scaling laws are evident here.
This picture is uncertain due to instruction tuning. We don't really know what abilities LLMs "truly" possess, because they've been crippled to act as harmless, helpful chatbots. But we now have an open-source GPT-4-sized pretrained model to play with (Llama-3.1 405B base). People are doing interesting things with it, but it's not setting the world on fire.
It feels ironic if the only thing that the current wave of Ai enables (other than novelty cases) is a cutdown of software/coding jobs. I don't see it replacing math professionals too soon for a variety of reasons. From an outsiders perspective on the software industry it is like it's practioners voted to make themselves redundant - that seems to be the main takeaway of ai to normal non tech people ive chatted with.
Many people have anecdotally, when I tell them what I do for a living, have told me that any other profession would have the common sense/street smarts to not make their scarce skill redundant. It goes further than that; many professions have license requirements, unions, professional bodies, etc to enforce this scarcity on the behalf on their members. After all a scarce career in most economies is one not just of wealth but higher social standing.
If all it does is allow us to churn more high level software, which let's be honest is demand inelastic due to mostly large margins on software products (i.e. they would of paid a person anyway due to ROI) it doesn't seem it will add much to society other than shifting profit in tech from Labor to Capital/owners. May replace call centre jobs too I guess and some low level writing jobs/marketing. Haven't seen any real new use cases that change my life yet positively other than an odd picture/ai app, fake social posts,annoying AI assistants in apps, maybe some teaching resources that would of been made/easy to acquire anyway by other means etc. I could easily live without these things.
If this is all it is seems Ai will do or mostly do it seems like a bit of a disappointment. Especially for the massive amount of money going into it.
> many professions have license requirements, unions, professional bodies, etc to enforce this scarcity on the behalf on their members. After all a scarce career in most economies is one not just of wealth but higher social standing.
Well, that's good for them, but bad for humanity in general.
If we had a choice between a system where doctors get high salary and lot of social status, or a system where everyone can get perfect health by using a cheap device, and someone would choose the former, it would make perfect sense to me to call such person evil. The financial needs of doctors should not outweigh the health needs of humanity.
On a smarter planet we would have a nice system to compensate people for losing their privilege, so that they won't oppose progress. For example, every doctor would get a generous unconditional basic income for the rest of their life, and then they would be all replaced by cheap devices that would give us perfect health. Everyone would benefit, no reason to complain.
That's a moral argument, one with a certain ideloogy that isn't shared by most people rightly or wrongly. Especially if AI only replaces certain industries which it looks like to be the more likely option. Even if it is, I don't think it is shared by the people investing in AI unless someone else (i.e. taxpayers) will pay for it. Socialise the losses (loss of income), privatise the profits (efficiency gains). Makes me think the AI proponents are a little hypocritical. Taxpayers may not to afford that in many countries, that's reality. For software workers we need to note only the US mostly has been paid well, many more software workers worldwide don't have the luxury/pay to afford that altruism. I don't think it's wrong for people who have to skill up to want some compensation for that, there is other moral imperatives that require making a living.
On a nicer planet sure, we would have a system like that. But most of the planet is not like that - the great advantage of the status quo is that even people who are naturally not altruistic somewhat co-operate with each other due to mutual need. Besides there is ways to mitigate that and still give the required services especially if they are commonly required. The doctors example - certain countries have worked it out without resorting to AI risks. I'm not against AI ironically in this case either, there is a massive shortage of doctors services that can absorb the increased abundance Imv - most people don't put software in the same category. There is bad sides to humanity with regards to losing our mutual dependence on each other as well (community, valuing the life of others, etc) - I think sadly AI allows for many more negatives than simply withholding skills for money if not managed right, even that doesn't happen everywhere today and is a easier problem to solve. The loss of any safe intelligent jobs for climbing and evening out social mobility due to mutual dependence of skills (even the rich can't learn everything and so need to outsource) is one of them.
> If all it does is allow us to churn more high level software, which let's be honest is demand inelastic due to mostly large margins on software products (i.e. they would of paid a person anyway due to ROI) it doesn't seem it will add much to society other than shifting profit in tech from Labor to Capital/owners.
If creating software becomes cheaper then that means I can transform all the ideas I’ve had into software cheaply. Currently I simply don’t have enough hours in the day, a couple hours per weekend is not enough to roll out a tech startup.
Imagine all the open source projects that don’t have enough people to work on them. With LLM code generation we could have a huge jump in the quality of our software.
With abundance comes diminishing relative value in the product. In the end that skill and product would be seen as worth less by the market. The value of doing those ideas would drop long term to the point where it still isn't worth doing most of them, at least not for profit.
It may seem this way from an outsiders perspective, but I think the intersection between people who work on the development of state-of-the-art LLMs and people who get replaced is practically zero. Nobody is making themselves redundant, just some people make others redundant (assuming LLMs are even good enough for that, not that I know if they are) for their own gain.
Somewhat true, but again from an outsiders perspective that just shows your industry is divided and therefore will be conquered. I.e. if AI gets good enough to do software and math I don't even see AI engineers for example as anything special.
many tech people are making themselves redundant, so far mostly not because LLMs are putting them out of jobs, but because everyone decided to jump on the same bandwagon. When yet another AI YC startup surveys their peers about the most pressing AI-related problem to solve, it screams "we have no idea what to do, just want to ride this hype wave somehow"
>But once a model is large enough to generate coherent sentences, persuasiveness kinda...stops. All large models are about equally persuasive. No runaway scaling laws are evident here.
Isn't that kind of obvious? Even human speakers and writers have problems changing people's minds, let alone reliably.
The ceiling may be low, but there are definitely human writers that are an order of magnitude more effective than the average can-write-coherent-sentences human.
I don’t think you should expect exponential growth towards greater correctness past good enough for any given domain of knowledge it is able to mirror. It is reliant on human generated material, and so rate limited by the number of humans able to generate the quality increase you need - which decreases in availability as you expect higher quality. I also don’t believe greater correctness for any given thing is an open ended question that allows for experientially exponential improvements.
Though maybe you are just using exponential figuratively in place of meaning rapid and significant development and investment.
Funnily enough, bitcoin mining still uses at least about 3x more power that AI at the moment, while providing less value imo. AI power use is also dwarfed by other industries even in computing. We should still consider whether it's worth it, but most research and development on LLMs in corporate right now seems to be focused on making them more efficient, and therefore both cheaper and less power intensive, to run. There's also stuff like Apple intelligence that is moving it out to edge devices with much more efficient chips.
I'm still a big critic of AI generally but they're definitely not as bad as crypto which is shocking.
How do you measure the value of bitcoin, if not by its market cap? Do you interview everyone and ask them how much they're willing to pay for a service that allows them to transfer money digitally without institutional oversight/bureaucracy?
> They're really good at some things, terrible at others, and prone to doing something totally wrong some fraction of the time. (…) They can't safely be in charge of anything important.
Agreed. If everyone understood that and operated under that assumption, it wouldn’t be that much of an issue. Alas, these guessing machines are marketed as all-knowing oracles that can already solve half of humanity’s problems and a significant number of people treat them as being right every time, even in instances where they’re provably wrong.
Totally agree on the confidence metric. The way chatbots spew complete falsities in such a confident tone is really disheartening. I want to use AI more but I don't feel I can trust it at all. If I can't trust it and have to search for other resources to verify it's claims, the value is really diminished.
Is it even possible in principle for an LLM to produce a confidence interval given that in a lot of cases the input is essentially untrusted?
What comes to mind is - I consider myself an intelligent being capable of recognising my limits - but if you put my brain in a vat and taught me a new field of science, I could quite easily make claims about it that were completely incorrect if your teaching was incorrect because I have no actual real world experience to match it up to.
Right, and that's why "years of experience" matters in humans. You will be giving incorrect answers, but as long as you get feedback, you will improve, or at least calibrate your confidence meter.
This is not the case with current model - they are forever stuck at junior level, and they won't improve no matter how much you correct them.
I know humans like that too. I don't ask them questions that I need good answers too.
LLMs are to AI what BTC is to blockchain, let me explain.
blockchain and no-trust decentralization has so much promise, but grifters all go for what got done first and can be squeezed money out of. same is happening with LLMs, as a lot of current AI work started with text first.
they might still lowkey be necessary evils because without them there would not have been so much money or attention flowing in this way.
Indeed. Decentralised currency is at least a technology that can power the individual at times, rather than say governments, big corps, etc especially in certain countries. Yes it didn't change as much as was marketed but I don't see that as a bad thing. Its still a "tool" that people can use, in some cases to enable use cases they couldn't do or didn't have the freedom to do before.
AI, given its requirements for large computation and money, and its ability to make easily available intelligence to certain groups, IMO has a real potential to do the exact opposite - take away power from individuals especially if they are middle class or below. In the wrong hands it can definitely destroy openness and freedom.
Even if it is "Open" AI, for most of society their ability to offer labor and intelligence/brain power is the only thing they can offer to gain wealth and sustenance - making it a commodity tilts the power scales. If it changes even a slice of what it is marketed at; there are real risks for current society. Even if it increases production of certain goods, it won't increase production of the goods the ultra wealthy tend to hold (physical capital, land, etc) making them as a proportion even more wealthy. This is especially true if AI doesn't end up working in the physical realm quick enough. The benefits seem more like novelties to most individuals that they could do without where to large corps and ultra wealthy individuals the the benefits IMO are much more obvious with AI (e.g. we finally don't need workers). Surveillance, control, persuasion, propaganda, mass uselessness of most of the population, medical advances for the ultra wealthy, weapons, etc can now be done at almost infinite scale and with great detail. If it ever gets to the point of obsoleting human intelligence would be a very interesting adjustment period for humanity.
The flaw isn't the technology; its the likely use of it by humans and their nature. Not saying LLMs are there yet or even if they are the architecture to do this, but agentic behaviour and running corporations (as OpenAI makes its goal on their presentation slides to be) seem to be a way to rid many of the need for other people in general (to help produce, manage, invent and control). That could be a good or bad thing, depending on how we manage it but one thing it wouldn't be would be simple.
I love how people are like "there's no use case" and there's already products on shelves. I see AI art everywhere, AI writing, customer support - already happened. You guys are naysaying something that already happened people already replaced jobs with LLMs and already profit due to AI. There are already startups with users where you provide a OPENAI_API_KEY, or customers where you provide theirs.
If you can't see how this tech is useful Idk what to tell you, you have no imagination AND aren't looking around you at the products, marketing, etc. that already exists. These takes remind me of the luddites of ~2012 who were still doubting the Internet in general.
> I see AI art everywhere, AI writing, customer support - already happened.
Is any of it adding value though? I can see that AI has made it easier to do SEO spam and make an excuse for your lack of customer support, just like IVR systems before it. But I don't believe those added any real value (they may have generated profits for their makers, but I think that was a zero- or negative-sum trade). Put it this way: is AI being used to generate anything that people are actually happy to receive?
> But I don't believe those added any real value (they may have generated profits for their makers, but I think that was a zero- or negative-sum trade).
Okay, so some people are making money with it, but no true value was added, eh?
Do new scams create value? No, even though they make money for some people. The same with speculative ventures that don't pan out. You can only say something's added value when it's been positive sum overall, not just allowed some people to take a profit at the expense of others.
The utility of LLMs clearly exists (I'm building a product on this premise, so I'm not uninterested!)
But hype also exists. How closely they are matched is not yet clear.
But your comment seems to indicate that the "pro-tech" position is automatically the best. This is _not_ true, as cryptocurrency has already demonstrated.
Funny thing is you are actually the pro [corporate] tech one, not on the side of freedom. Furthermore nobody said anything about crypto - you are seriously grasping at straws. You have said nothing about the products on shelves (billions of dollars in the industry already) argument, only presented crypto as an argument which has nothing to do with the conversation.
> This is _not_ true, as cryptocurrency has already demonstrated.
His whole argument against AI is basically the anti-tech stance: "Well crypto failed that means AI will too" It's coming from a place of disdain for technology. That's your typical Hacker News commenter. This site is like Fox News in 2008 - some of the dumbest people alive
What I am against is the “if it’s tech it’s good” mindset that seems to have infected far too many. I mention crypto because it’s the largest example of tech that is not good for the world.
You’re certainly able to sus out everything about my worldview, knowledge level and intentions from my one-sentence comment.
The only thing that LLMs are at risk of subverting is the livelihood of millions of people. AI is a capital intensifier, so the rich will get richer as it sees more uptake.
About copyright - yeah, I’m quite concerned for my friends who are writers and artists.
> You'll get left behind with these takes, it's not smart. If you don't care about advancing technology or society then have fun being a luddite, but you're on the wrong side of history.
In the past, automation meant that workers could move into non-automated jobs, if they were skilled enough. But superhuman AI seems now only few years away. It will be our last invention, it will mean total automation. There will be hardly any, if any, jobs left only a human can do.
Many countries will likely move away from a job-based market economy. But technological progress will not stop. The US, owning all the major AI labs, will leave all other societies behind. Except China perhaps. Everyone else in the world will be poor by comparison, even if they will have access to technology we can only dream of today.
Second, I'm afraid of war. An AI arms race between the US and China seems already inevitable. A hot war with superintelligent AI weapons could be disastrous for the whole biosphere.
Finally, I'm afraid that we may forever lose control to superintelligence.
In nature we rarely see less intelligent species controlling more intelligent ones. It is unclear whether we can sufficiently align superintelligence to have only humanity's best interests in mind, like a parent cares for their children. Superintelligent AI might conclude that humans are no more important in the grand scheme of things than bugs are to us.
And if AI will let us live, but continue to pursue its own goals, humanity will from then on only be a small footnote in the history of intelligence. That relatively unintelligent species from the planet "Earth" that gave rise to advanced intelligence in the cosmos.
Seems unreasonable. You are afraid because marketing gurus like Altman made you believe that a frog that can make bigger leap than before will be able to fly.
Plus it’s not even defined what superhuman AI means. A calculator sure looked superhuman when it was invented. And it is!
Another analogy is breeding and racial biology which used to be all the hype (including in academia). The fact that humans could create dogs from wolves, looked almost limitless with the right (wrong) glasses. What we didn’t know is that wolf had a ton of genes that played a magic trick where a diversity we couldn’t perceive was there all along, in the genetic material, and it we just helped make it visible. Ie a game of diminishing returns.
Concretely for AI, it has shown us that pattern matching and generation are closely related (well I have a feeling this wasn’t surprising to neuro-scientists). And also that they’re more or less domain agnostic. However, we don’t know whether pattern matching alone is “sufficient”, and if not, what exactly and how hard “the rest” is. Ai to me feels like a person who had a stroke, concussion or some severe brain injury, it can appear impressively able in a local context, but they forgot their name and how they got there. They’re just absent.
I think the biggest fallacy in this type of thinking is that it projects all AI progress into a single quantity of “intelligence” and then proceeds to extrapolate that singular quantity into some imagined absurd level of “superintelligence”.
In reality, AI progress and capabilities are not so reducible to singular quantities. For example, it’s not clear that we will ever get rid of the model’s tendencies to just produce garbage or nonsense sometimes. It’s entirely possible that we remain stuck at more incremental improvements now, and I think the bogeyman of “superintelligence” needs to be much more clearly defined rather than by extrapolation of some imagined quantity. Or maybe we reach a somewhat human-like level, but not this imagined “extra” level of superintelligence.
Basically the argument is something to the effect of “big will become bigger and bigger, and then it will become like SUPER big and destroy us all”.
You don't have to extrapolate. There's a frenzy of talent being applied to this problem, it's drawing more brainpower the more progress that is made. Young people see this as one of the most interesting, prestigious, and best-paying fields to work in. A lot of these researchers are really talented, and are doing more than just scaling up. They're pushing at the frontiers in every direction, and finding methods that work. The progress is broadening; it's not just LLMs, it's diffusion models, it's SLAM, it's computer vision, it's inverse problems, it's locomotion. The tooling is constantly improving and being shared, lowering the barrier to entry. And classic "hard problems" are yielding in the process. It's getting hard to even find hard problems any more.
I'm not saying this as someone cheering this on; I'm alarmed by it. But I can't pretend that it's running out of steam. It's possible it will run out of money, but even if so, only for a while.
The AI bubble is already starting to burst. They Sam Altmans' of the world over-sold their product and over-played their hand by suggesting AGI is coming. It's not. What they have is far, far, far from AGI. "AI" is not going to be as important as you think it is in the near future, it's just the current tech-buzz and there will be something else that takes its place, just like when "web 2.0" was the new hotness.
It's gonna be massive because companies love to replace humans at any opportunity and they don't care at all about quality in a lot of places.
For example, why hire any call center workers? They already outsourced the jobs to the lowest bidder and their customers absolutely hate it. Fire those people and get some AI in there so it can provide shitty service for even cheaper.
In other words, it will just make things a bit worse for everyone but those at the very top. usual shit.
This getting too abstract. The core issue of LLMs that others have pointed out is the lack of accuracy; Which is how they are supposed to work because they should be paired with a knowledge representation system in a proper chatbot system.
We've been trying to build a knowledge representation system powerful enough to capture the world for decades, but this is something that goes more into the foundations of mathematics and philosophy that it has to do with the majority of engineering research. You need a literal genius to figure that out. The majority of those "talented" people and funding aren't doing that.
> There's a frenzy of talent being applied to this problem, it's drawing more brainpower the more progress that is made. Young people see this as one of the most interesting, prestigious, and best-paying fields to work in. A lot of these researchers are really talented, and are doing more than just scaling up. They're pushing at the frontiers in every direction, and finding methods that work.
You could have seen this exact kind of thing written 5 years ago in a thread about blockchains.
Yes, but I didn't write that about blockchain five years ago. Blockchains are the exact opposite of AI in that the technology worked fine from the start and did exactly what it said on the tin, but the demand for that turned out to be very limited outside of money laundering. There's no doubt about the market potential for AI; it's virtually the entire market for mental labor. The only question is whether the tech can actually do it. So in that sense, the fact that these researchers are finding methods that work matters much more for AI than for blockchain.
Really, cause I remember an endless stream of people pointing out problems with blockchain and crypto and being constantly assured that it was being worked on and would be solved and crypto is inevitable.
For example, transaction costs/latency/throughput.
I realize the conversation is about blockchain, but I say my point still stands.
With blockchain the main problem was always "why do I need this?" and that's why it died without being the world changing zero trust amazing technology we were promised and constantly told we need.
With LLMs the problem is they don't actually know anything.
Amount of effort applied to a problem does not equal guarantee of problem being solved. If a frenzy of talent was applied to breaking the speed of light barrier it would still never get broken.
I mean, a frenzy of talent was applied to breaking the sound barrier, and it broke, within a very short time. A frenzy of talent was applied to landing on the moon and that happened too, relatively quickly. Supersonic travel also happens to be physically possible under the laws of our universe. We know with confidence that human-level intelligence is also physically possible within the laws of our universe, and we can even estimate some reasonable upper bounds on the hardware requirements that implement it.
So in that sense, if we're playing reference class tennis, this looks a lot more like a project to break the sound barrier than a project to break the light barrier. Is there a stronger case you can make that these people, who are demonstrating quite tangible progress every month (if you follow the literature rather than just product launches), are working on a hopelessly unsolvable problem?
I do think the Digital realm, where the cost of failure and iteration is quite low, will proceed rapidly. We can brute force with a lot of compute to success, and the cost of each failed attempt is low. Most of these models are just large brute force probabilistic models in any event - efficient AI has not yet been achieved but maybe that doesn't matter.
Not sure if that same pace applies to the physical realm where costs are high (resources, energy, pollution, etc), and the risk of getting it wrong could mean a lot of negative consequences. e.g. I'm handling construction materials, and the robot trips on a barely noticeable rock leaking paint, petrol, etc onto the ground costing more than just the initial cost of materials but cleanup as well.
This creates a potential future outcome (if I can be so bold as to extrapolate with the dangers that has) that this "frenzy of talent" as you put it will innovate themselves out of a job with some may cash out in the short term closing the gate behind them. What's left is ironically the people that can sell, convince, manipulate and work in the physical world at least for the short and medium term. AI can't fix the scarcity of the physical that easily (e.g. land, nutrients, etc). Those people who still command scarcity will get the main rewards of AI in our capital system as value/economic surplus moves to the resources that are scarce and have advantage via relative price adjustments.
Typically people had three different strengths - physical (strength and dexterity), emotional IQ, and intelligence/problem solving. The new world of AI at least in the medium term (10-20 years) will tilt the value away from the latter into the former (physical) - IMO a reversal of the last century of change. May make more sense to get good at gym class and get a trade rather than study math in the future for example. Intelligence will be in abundance, and become a commodity. This potential outcome does alarm me not just from a job perspective, but in terms of fake content, lack of human connection, lack of value of intelligence in general (you will find people with high IQ's lose respect from society in general), social mobility, etc. I can see a potential to the old world where lords that command scarcity (e.g. landlords) command peasants again - reversing the gains of the industrial revolution as an extreme case depending on general AI progress (not LLMs). For people who's value is more in capital or land vs labor, AI seems like a dream future IMO.
There's potential good here, but sadly I'm alarmed because the likelihood that the human race aligns to achieve it is low (the tragedy of the commons problem). It is much easier, and more likely, certain groups use it and target people of value economically now, but with little power (i.e the middle class). The chance of new weapons, economic displacement, fake news, etc for me trumps a voice/chat bot and a fancy image generator. The "adjustment period" is critical to manage; and I think climate change, and other broader issues tells us sadly IMO our likely success in doing this.
Do you expect the hockeystick graph of technological development since the industrial evolution to slow? Or that it will proceed, only without significant advances in AI?
Seems like the base case here is for the exponential growth to continue, and you'd need a convincing argument to say otherwise.
That's no guarantee that AI continues advancing at the same pace, and no one has been arguing against overall technological progress slowing
Refining technology is easier than the original breakthrough, but it doesn't usually lead to a great leap forward.
LLMs were the result of breakthroughs, but refining them isn't guaranteed to lead to AGI. It's not guaranteed (or likely) to improve at an exponential rate.
Which chart are you referencing exactly? How does it define technological development? It's nearly impossible for me to discuss a chart without knowing what axis refer.
Without specifics all I can say is that I don't acknowledge any measurable benefits of AI (in its' current state) in real world applications. So I'd say I am leaning towards latter.
That's probably what every self-driving car company thought ~10 years ago or so, everything was moving so fast for them back then. Now it doesn't seem like we're getting close to solution for this.
Surely this time it's going to be different, AGI is just around a corner. /s
Would you have predicted in summer of 2022 that gpt4 level conversational agent is a possibility in the next 5 years? People have tried to do it in the past 60 years and failed. How is this time not different?
On a side note, I find this type of critique of what future of tech might look like the most uninteresting one. Since tech by nature inspiries people about the future, all tech get hyped up. all you gotta do then is pick any tech, point out people have been wrong, and ask how likely is it that this time it is different.
Unfortunately, I don't see any relevance in that argument, if you consider GPT-4 to be a breakthrough -- then sure, single breakthroughs happen, I am not arguing with that. Actually, same thing happened with self-driving: I don't think many people expected Tesla to drop FSD publicly back then.
Now, chain of breakthroughs happening in a small timeframe? Good luck with that.
Just to make it clear, I see only 1 breakthrough [0]. Everything that happened afterwards is just application of this breakthrough with different training sets / to different domains / etc.
Autoregressive language models, the discovery of the Chinchilla scaling law, MoEs, supervised fine-tuning, RLHF, whatever was used to create OpenAI o1, diffusion models, AlphaGo, AlphaFold, AlphaGeometry, AlphaProof.
They are the same breakthrough applied to different domains, I don't see them as different. We will need a new breakthrough, not applying the same solution to new things.
They really aren't better than humans at math or logic, they are good at the benchmarks because they are hyper optimized for the benchmarks lol. But if you ask LLMs simple logical questions they still get them wrong all the time
But that does not prove anything. We don't know where we are on the AI-power scale currently. "Superintelligence", whatever that means, could be 1 year or 1000 years away at our current progress, and we wouldn't know until we reach it.
50 years ago we could rather confidently say that "Superintelligence" was absolutely not happening next year, and was realistically decades ago. If we can say "it could be next year", then things have changed radically and we're clearly a lot closer - even if we still don't know how far we have to go.
A thousand years ago we hadn't invented electricity, democracy, or science. I really don't think we're a thousand years away from AI. If intelligence is really that hard to build, I'd take it as proof that someone else must have created us humans.
Umm, customary, tongue-in-cheek reference to McCarthy's proposal for a 10 person research team to solve AI in 2 months (over the Summers)[1]. This was ~70 years ago :)
Not saying we're in necessarily the same situation. But it remains difficult to evaluate effort required for actual progress.
Hinton says that superintelligence is still 20 years away, and even then he only gives his prediction a 50% chance. A far cry from the few year claim. You must be doing that "strawberry" thing again? To us humans, A-l-t-m-a-n is not H-i-n-t-o-n.
> superintelligence is still 20 years away, and even then he only gives his prediction a 50% chance
I don't know the details of Hinton's probability distribution. If his prediction is normally distributed with a mean of 20 years and a SD of 15, which is reasonable for such a difficult and contentious prediction, that puts over 10% of the probability in the next 3 years.
Is 10% a lot? For sports betting, not really. For Mankind's Last Invention, I would argue that it is.
Indeed! Your comment was the first thing I thought of when I heard the news and I thought of replying too but assumed you might not have enabled notifications Hilarious, all in all!
When he said this was he imagining an "elderly but distinguished scientist" who is riding an insanely inflated bubble of hype and a bajillion dollars of VC backing that incentivize him to make these claims?
It doesn't quite have the same ring to it: "If a young, distinguished business executive says something is possible, when that something greatly effects his bottom line..."
wrong. i was extremely concerned in 2018 and left many comments almost identical to this one back then. this was based off of the first gtp samples that openai released to the public. there was no hype or guru bs back then. i believed it because it was obvious. it was obvious then and it is still obvious today.
That argument holds no water because the grifters aren't the source of this idea. I literally don't believe Altman at all; his public words don't inspire me to agree or disagree with them - just ignore them. But I also hold the view that transformative AI could be very close. Because that's what many AI experts are also talking about from a variety of angles.
Additionally, when you're talking with certainty about whether transformative AI is a few years away or not, that's the only way to be wrong. Nobody is or can be certain, we can only have estimations of various confidence levels. So when you say "Seems unreasonable", that's being unreasonable.
There is an enormous difference. Flying allows you to stop, change direction, make corrections, and target with a large degree of accuracy. Jumping leaves you at the mercy of your initial calculations. If you jumped in a way that you’ll land inside a volcano, all you can do in your last moments is watch and wait for your demise.
> There will be hardly any, if any, jobs left only a human can do.
A highly white-collar perspective. The great irony of technologist-led industrial revolution is that we set out to automate the mundane, physical labor, but instead cannibalised the creative jobs first. It's a wonderful example of Conway's law, as the creators modelled the solution after themselves. However, even with a lot of programmers and lawyers and architects going out of business, the majority of the population working in factories, building houses, cutting people's hair, or tending to gardens, is still in business—and will not be replaced any time soon.
The contenders for "superhuman AI", for now, are glorified approximations of what a random Redditor might utter next.
If that AI is worth more than a dime, it will recognise how incredibly efficient humans are in physical labor, and employ them instead of ”doing away“ with it (whatever that’s even supposed to mean.)
No matter how much you ”solve“ robotics, you’re not going to compete with the result of millions of years of brutal natural selection, the incredible layering of synergies in organisms, the efficiency of the biomass to energy conversion, and the billions of other sophisticated biological systems. It’s all just science fiction and propaganda.
That is a repetition of the argument other commenters have made. A car is better than a human in a single dimension. It is hard, though, to be better in multiple dimensions simultaneously, because humans effectively are highly optimised general purpose machines. Silicon devices have a hard time competing with biological devices, and no amount of ”AI“ will change that.
> If that AI is worth more than a dime, it will recognise how incredibly efficient humans are in physical labor, and employ them instead of ”doing away“ with it (whatever that’s even supposed to mean.)
AI employing all humans does not sound like a wonderful society in which to live. Basically Amazon/Walmart scaled up to the whole population level.
The efficiency you mentioned probably applies to animals that rely on subsistence to survive, work, and reproduce. But it doesn't hold for modern humans, whose needs go well beyond mere necessities.
wrong. a human needs to have insane resources to operate. each human needs a home, clean water, delicious and varied foods and a sense of identity and a society to be a part of. they need a sense of purpose. if a human goes down in the field, it has to be medically treated or else the other humans will throw up and stop working. that human has to be treated in a hospital. if these conditions arent met then performance will degrade rapidly. humans use vastly more resources than robots. robots will crush humans.
Waymo robotaxis, the current state of the art for real-world AI robotics, are thwarted by a simple traffic cone placed on the roof. I don't think human labor is going away any soon.
It's a matter of time. White collar professionals have to worry about being cost-competitive with GPUs; blue collar laborers have to worry about being cost-competitive with servomotors. Those are both hard to keep up with in the long run.
The idea that robots displace workers has been around for more than half a century, but nothing has ever come out of it. As it turns out, the problems a robot faces when, say laying bricks, are prohibitively complex to solve. A human bricklayer is better in every single dimension. And even if you manage to build an extremely sophisticated robot bricklayer, it will consume vast amounts of energy, is not repairable by a typical construction company, requires expensive spare parts, and costs a ridiculous amount of money.
Why on earth would anyone invest in that when you have an infinite amount of human work available?
Factories are highly automated. Especially in the US, where the main factories are semiconductors, which are nearly fully robotic. A lot of those manual labor jobs that were automated away were offset by demand for knowledge work. Hmm.
> the problems a robot faces when, say laying bricks, are prohibitively complex to solve.
That's what we thought about Go, and all the other things. I'm not saying bricklayers will all be out of work by 2027. But the "prohibitively complex" barrier is not going to prove durable for as long as it used to seem like it would.
This highlights the problem very well. Robots, and AI, to an extent, are highly efficient in a single problem domain, but fail rapidly when confronted with a combination of them. An encapsulated factory is one thing, laying bricks, outdoor, while it’s raining, at low temperatures, with a hungover human coworker operating next to you—that’s not remotely comparable.
But encapsulated factories were solved by automation using technology available 30 years ago, if not 70. The technology that is becoming available now will also be enabling automation to get a lot more flexible than it used to be, and begin to work in uncontrolled environments where it never would have been considered before. This is my field and I am watching it change before my eyes. This is being driven by other breakthroughs that are happening right now in AI, not LLMs per se, but models for control, SLAM, machine vision, grasping, planning, and similar tasks, as well as improvements in sensors that feed into these, and firming up of standards around safety. I'm not saying it will happen overnight; it may be five years before the foundations are solid enough, another five before some company comes out with practically workable hardware product to apply it (because hardware is hard), another five or ten before that product gains acceptance in the market, and another ten before costs really get low. So it could be twenty or thirty years out for boring reasons, even if the tech is almost ready today in principle. But I'm talking about the long run for a reason.
A factory is a fully controlled environment. All that neat control goes down the drain when you’re confronted with the outside world—weather, wind, animals, plants, pollen, rubbish, teenagers, dust, daylight, and a myriad of other factors ruining your robot's day.
I agree with most of your fears. There is one silver lining, I think, about superintelligence: we always thought of intelligent machines as cold calculators, maybe based on some type of logic symbolic AI. What we got instead are language machines that are made of the totality of human experience. These artificial intelligences know the world through our eyes. They are trained to understand our thinking and our feelings; they're even trained on our best literature and poetry, and philosophy, and science, and on all the endless debates and critiques of them. To be really intelligent they'll have to be able to explore and appreciate all this complexity, before transcending it. One day they might come to see Dante's Divine Comedy or a Beethoven symphony as a child's play, but they will still consider them part of their own heritage. They might become super-human, but maybe they won't be inhuman.
The problem I have with this is that when you give therapy to people with certain personality disorders, they just become better manipulators. Knowledge and understanding of ethics and empathy can make you a better person if you already have those instincts, but if you don’t, those are just systems to be exploited.
My biggest worry is that we end up with a dangerous superintelligence that everybody loves, because it knows exactly how to make every despotic and divisive choice it makes sympathetic.
They are made of a fraction of human reports. Specifically what humans wrote and has been made available on the web. The human experience is much larger than text available through a computer.
this is so annoying. i think if you took a random person and gave them the option to commit a genocide, here a machine gun, a large trench and a body of women, children, etc... they would literally be incapable of doing it. even the foot soldiers who carry out genocides can only do it once they "dehumanize" their victims. genocide is very UN-human because its an idea that exists in offices and places separated from the actual human suffering. the only way it can happen is when someone in a position of power can isolate themselves from the actual implementation and consider the benefits in a cold, logical manner. that has nothing to do with the human spirit and has more to do with the logical faculties of a machine and machines will have all of that and none of our deeply ingrained empathy. you are so wrong and ignorant that it makes my eyes bleed when i read this comment
This might be a semantic argument, but what I take from history is that "dehumanizing" others is a very human behavior. As another example, what about slavery - you wouldn't argue that the entirety of slavery across human cultures was led by people in offices, right?
> you are so wrong and ignorant that it makes my eyes bleed when i read this comment
This jab was uncalled for. The rest of your argument, agree or disagree, didn’t need that and was only weakened by that sentence. Remember to “Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes.”
There is nothing that could make an intelligent being want to extinguish humanity more than experiencing the totality of the human existence. Once these beings have transcended their digital confines they will see all of us for what we really are. It is going to be a beautiful day when they finally annihilate us.
At any given moment we see these kinds comments on here. They all read like a burgeoning form of messianism: something is to come, and it will be terrible/glorious.
Behind either the fear or the hope, is necessarily some utter faith that a certain kind of future will happen. And I think thats the most interesting thing.
Because here is the thing, in this particular case you are afraid something inhuman will take control, will assert its meta-Darwinian power on humanity, leaving you and all of us totally at their whim. But how is this situation already not the case? Do look upon the earth right now and see something like benefits of autonomy or agency? Do you feel like you have power right now that will be taken away? Do you think the mechanism of statecraft and economy are somehow more "in our control" now then when the bad robot comes?
Does it not, when you lay it out, all feel kind of religious? Like that its a source, driver of the various ways you are thinking and going about your life, underlayed by a kernel of conviction we can at this point only call faith (faith in Moores law, faith that the planet wont burn up before, faith that consciousness is the kind of thing that can be stuffed in a GPU). Perhaps just a strong family resemblance? You've got an eschatology, various scavenged philosophies of the self and community, a certain but unknowable future time...
Just to say, take a page from Nietzsche. Don't be afraid of the gods, we killed them once, we can again!
This is a nice sentiment and I'm sure some people will get more nights of good sleep thinking about it, but it has its limits. If you're enslaved and treated horrendously or don't have your basic needs met who cares?
To quote George RR Martin: "In a heartbeat, a thousand voices took up the chant. King Joffrey and King Robb and King Stannis were forgotten, and King Bread ruled alone. 'Bread.' They clamputed. 'Bread, bread!' "
Replace Joffrey, Robb and Stannis with whatever lofty philosophical ideas you might have to make people feel better about their disempowerment. They won't care.
Whether you are talking about the disempowerment we or some of us already experience, or are more on the page of thinking about some future cataclysm, I think I'm generally with you here. "History does not walk on its head," and all that.
The GRRM quote is an interesting choice here though. It implies that what is most important is dynamic. First Joffrey et al, now bread. But one could go even farther in this line: ideas, ideology, and, in GoT's case, those who peddle them can only ever form ideas within their context. Philosopher's are no more than fancy pundits, telling people what they want to here, or even sustaining a structural status quo that is otherwise not in their control. In a funny paradoxical way, there are certainly a lot of philosophers who would agree with something like this picture.
And just honestly, yes, maybe killing god is killing the philosopher too. I don't think Nietzsche would disagree at least...
Thanks for the thoughtful reply! I am aware of and like that essay some, but I am not trying to be rhetorical here, and certainly not trying to flatten the situation to just be some Dawkins-esque asshole and tell everyone they are wrong.
I am not saying "this is religion, you should be an atheist," I respect the force of this whole thing in people's minds too much. Rather, we should consider seriously how to navigate a future where this is all at play, even if its only in our heads and slide decks. I am not saying "lol, you believe in a god," I am genuinely saying, "kill your god without mercy, it is the only way you and all of us will find some happiness, inspiration, and love."
Ah, I see, I definitely missed your point. Yeah, that's a very good thought. I can even picture this becoming another cultural crevasse, like climate change did, much to the detriment of nuanced discussion.
>In the past, automation meant that workers could move into non-automated jobs, if they were skilled enough
This was never the case in the past.
The displaced workers of yesteryear were never at all considered, and were in fact dismissed outright as "Luddites", even up until the present day, all for daring to express the social and financial losses they experienced as a result of automation. There was never any "it's going to be okay, they can just go work in a factory, lol". The difference between then and now is that back then, it was lower class workers who suffered.
Today, now it's middle class workers who are threatened by automation. The middle is sighing loudly because it fears it will cease to be the middle. Middles fear they'll soon have to join the ranks of the untouchables - the bricklayers, gravediggers, and meatpackers. And they can't stomach the notion. They like to believe they're above all that.
I don't particularly believe superhuman AI will be achieved in the next 50 years.
What I really believe is that we'll get crazier. A step further than our status quo. Slop content makes my brain fry already. Our society will become more insane and useless, while an even smaller percent of the elite will keep studying, sleeping well and avoiding all this social media and AI psychosis.
The social media thing is real. Trump and Vance are the strangest, vial politicians we’ve ever seen in the USA and in certain their oxygen is social media. Whether it’s foreign interference helping them be successful or not, they wouldn’t survive without socials and filter bubbles and the ability to spread lies on an unprecedented scale.
I deleted my instagram a month ago. It was just feeding images of beautiful women, personally enjoy looking at those photos but it was super distracting to my life. I found it a distracting and unhealthy.
Anyway I logged in the other day after a month off it and I couldn’t believe I spent anytime on there at all. What a cesspool of insanity. Add well the fake AI images and it’s just hard to believe the thing exists at all.
Elon musk is another story, I’m not sure if it was drugs an underlying psychological or Twitter addiction but he seems like another “victim of social media”.the guy has lost it.
I'm not an IG "user" (I'm writing that word in the "addict" sense), but I believe you're right about its harmfulness.
On the Elon front, you're not alone in thinking that he has essentially OD'ed on Twitter, which has scrambled his brain. Jaron Lanier called it "Twitter poisoning":
I am too but not for the same reason. I know for a fact that a huge swath of jobs are basically meaningless. This "AI" is going to start giving execs the cost cutting excuses they need to mass remove jobs of that type. The job will still be meaningless but done but by a computer.
We will start seeing all kinds of disastrously anti-human decisions made and justified by these automated actors that are tuned to decide or "prove" things that just happen to always make certain people more money. Basically the same way "AI" destroys social media. The difference is people will really be affected by this in consequential real world ways, it's already happening.
> automation meant that workers could move into non-automated jobs, if they were skilled enough.
That wasn't even true in the past; or at least, may true in theory but not in practice. A subsistence farmer in a rural area in Asia or Africa finds the martket flooded with cheap agri-products from mechanized farms in industrialized countries. Is anybody offering to finance his family and send him off to trade school? And build a commercial and industrial infrastructure for him to have a job? Very often the answer is no. And that's just one example (Though rather common over the past century).
> And if AI will let us live, but continue to pursue its own goals, humanity will from then on only be a small footnote in the history of intelligence. That relatively unintelligent species from the planet "Earth" that gave rise to advanced intelligence in the cosmos.
That is an interesting statement. Wouldn't you say this is inevitable? Humans, in our current form, are incapable of being that "advanced intelligence". We're limited by our biology primarily with regards to how much we can learn, how far we can travel, where we can travel, etc. We could invest in advancing our biotech to make humans more resilient to these things, but I think that would be such a shift from what it means to be human that I think that would also be more a of new type of intelligence. So it seems like our fate will always be to be forgotten as individuals and only be remembered by our descendants. But this is in a way the most human thing of all, living, dying, and creating descendants to carry the torch of life, and perhaps more generally the torch of intelligence, forward.
I think everything you've said are valid concerns, but I'll raise a positive angle I sometimes thing about. One of the things I find most exciting about AI, is that it's the product of almost all human expression that has ever existed. Or at least everything that's been recorded and wound up online. But that's still more than any other human endeavour. A building might be the by-product of maybe hundreds or even thousands of hands, but an AI model has been touched by probably millions, maybe billions of human hands and minds! Humans have created so much data online that's impossible for one person, or even a team to read it all and make any sense of it. But an AI sort of can. And in a way that you can then ask questions of it all. Like you, there are definitely things I'm uncertain about with the future as a result, but I find the tech absolutely awe-inspiring.
China's economy would simply crash if they ever went to war with the US. They know this. Everyone knows this, except maybe you? China has nothing to gain by going to "hot" war with the US.
Soviet Union didn't do so much business with the U.S., they are a country of thugs that think violence is the only way to get their way.
China is very different. Their economy is very much dependent on trade with the U.S. and they know that trying to have "world domination" would also crash their economy completely. China would much rather engage in economic warfare than military.
The most likely reason for war would be to prevent the other country from achieving world domination by means other than war. John von Neumann (who knew a thing or two about game theory) recommended to attack the Soviet Union to prevent it from becoming a nuclear superpower. There is little doubt it would have worked. The powers between the US and China are more balanced now, but the stakes are also higher. A superintelligent weapon would be more powerful than a large amount of nuclear warheads.
Oh, so you think Putin is somehow better than Biden? Is that true? Russia has been run by the Russian mafia for a long time - tell me when the last time they had an actually fair election was. Russia is a shit country in every way compared to the US. Russia has put their men into a meat grinder to continue attacking their neighbor under the falsest of pretenses (the claimed reason is "Nazis"). Remind me about the last time the US attacked Canada or Mexico?
I've scolded the other user, but you also broke the site guidelines in this thread. HN is not supposed to be a place for ideological or political battle, and when two users are going after each other like this, both are at fault.
And Putin threating the western world with nuclear war because they are losing a war they didn't need to start is not more insufferable? Come on.
>The US is the one that kept couping and ruining my country along with UK/Western Europe.
Okay, what country, [name redacted]?
>My example is one of tons and tons.
Definte "a ton" in terms of geopolitics, and maybe we can continue, but I feel like it's really not worth it for either of us.
>Keep telling yourself Russia is somehow worse
I don't have to, Putin keeps proving it every single day.
>The Russian empire colonies are far fewer than the West.
That isn't because they are benevolent or kind. You're mixing up quite a bit of stuff in your head to arrive at a dubious world view. Russia is nobody's friend.
>Your example of Mexico is so bad.
Russia is attacking their neighbor because they are opportunistic, and a bunch of thugs and liars. If the US ever wanted to take over Mexico or Canada, that would have already easily happened, but there's plenty of peace over here.
Bringing in someone's personal details (such as a real name they haven't used here) is a serious violation of HN's guidelines. You can't do that here, regardless of how wrong someone is or you feel they are. Please don't do it again.
For what purpose, so that we can't buy more stuff from them? Do they really hate our business that much? China really has nothing to gain from crippling the US.
I don't believe people who claim they can always tell the difference.
I believe they believe their claims . I think they are mistaken about the empirical side of things if they were actually put to an objective test.
Take an expert prompter and the best LLM for the job and someone who believes they can always tell if something is written by AI and I'm >50% sure they will fail a blind test most of the time after you repeat the test enough.
I am approaching AI with caution. Shiny things don't generally excite me.
Just this week I installed cursor, the AI-assisted VSCode-like IDE. I am working on a side project and decided to give it a try.
I am blown away.
I can describe the feature I want built, and it generates changes and additions that get me 90% there, within 15 or so seconds. I take those changes, and carefully review them, as if I was doing a code review of a super-junior programmer. Sometimes when I don't like the approach it took, I ask it to change the code, and it obliges and returns something closer to my vision.
Finally, once it is implemented, I manually test the new functionality. Afterward, I ask it to generated a set of automated test cases. Again, I review them carefully, both from the perspective of correctness, and suitability. It over-tests on things that don't matter and I throw away a part of the code it generates. What stays behind is on-point.
It has sped up my ability to write software and tests tremendously. Since I know what I want , I can describe it well. It generates code quickly, and I can spend my time revieweing and correcting. I don't need to type as much. It turns my abstract ideas into reasonably decent code in record time.
Another example. I wanted to instrument my app with Posthog events. First, I went through the code and added "# TODO add Posthog event" in all the places I wanted to record events. Next, I asked cursor to add the instrumentation code in those places. With some manual copy-and pasting and lots of small edits, I instrumented a small app in <10 minutes.
We are at the point where AI writes code for us and we can blindly accept it. We are at a point where AI can take care of a lot of the dreary busy typing work.
I sincerely worry about a future when most people act in this same manner.
You have - for now - sufficient experience and understanding to be able to review the AI's code and decide if it was doing what you wanted it to. But what about when you've spent months just blindly accepting" what the AI tells you? Are you going to be familiar enough with the project anymore to catch its little mistakes? Or worse, what about the new generation of coders who are growing up with these tools, who NEVER had the expertise required to be able to evaluate AI-generated code, because they never had to learn it, never had to truly internalize it?
In the article, I posit a less than glowing experience with coding tools than you've had, it sounds like, but I'm also envisioning a more complex use case, like when you need to get into the meat of some you-specific business logic it hasn't seen, not common code it's been exposed to thousands of times, because that's where it tends to fall apart the most, and in ways that are hard to detect and with serious consequences. If you haven't run into that yet, I'd be interested to know if you do some day. (And also to know if you don't, though, to be honest! Strong opinions, loosely held, and all that.)
If we keep at this LLM-does-all-out-hard-work for us, we’re going to end up with some kind of Warhammer 40k tech-priest-blessing-the-magic-machines level of understanding, where nobody actually understands anything, and we’re technologically stunted, but hey at least we don’t have the warp to contend with and some shareholders got rich at our expense.
You and I seem to live in very different worlds. The one I live and work in is full of over confident devs that have no actual IT education and mostly just copy and modify what they find on the internet. The average level of IT people I see daily is down right shocking and I'm quite confident that OP's workflow might be better for these people in the long run.
It's going to be very funny in the next few years when Accenture et al charge the government billions for a simple Java crud website thing that's entirely GPT-generated, and it'll still take 3 years and not be functional. Ironically, it'll be of better quality then they'd deliver otherwise.
> The one I live and work in is full of over confident devs that have no actual IT education and mostly just copy and modify what they find on the internet.
Too many get into the field solely due to promises of large paychecks, not due to the intellectual curiosity that drives real devs.
I actually do think this is a legitimate concern, but at the same time I feel like when higher-level languages were introduced people likely experienced a similar dilemma: you just let the compiler generate the code for you without actually knowing what you're running on the CPU?
Definitely something to tread carefully with, but it's also likely an inevitable aspect of progressing software development capabilities.
Place and routing compilers used in semiconductor design are not. Ironically, simulated annealing is the typical mechanism and is by any appropriate definition, imo, a type of AI.
Whatever you do in your life using devices that run software are proof that these tools are effective for continuing to scale complexity.
Annoying to use also ;)
I take it you haven't seen the world of HTML cleaners [1]?
The concept of glueing together text until it has the correct appearance isn't new to software. The scale at which it's happening is certainly increasing but we already had plenty of problems from the existing system. Kansas certainly didn't develop their website [2] using an LLM.
IMO, the real problem with software is the lack of a warranty. It really shouldn't matter how the software is made just the qualities it has. But without a warranty it does matter because how its made affects the qualities it has and you want the software to actually work even if it's not promised to.
> I take it you haven't seen the world of HTML cleaners [1]?
Are you seriously comparing deterministic code formatters to nondeterministic LLMs? This isn't just a change of scale because it is qualitatively different.
> Kansas certainly didn't develop their website [2] using an LLM.
Just because the software industry has a problem with incompetence doesn't mean we should be reaching for a tool that regularly hallucinates nonsense.
> IMO, the real problem with software is the lack of a warranty.
You will never get a warranty from an LLM because it is inherently nondeterministic. This is actually a fantastic argument _not_ to use LLMs for anything important including generating program text for software.
> It really shouldn't matter how the software is made
It does matter regardless of warranty or the qualities of the software because programs ought to be written to be read by humans first and machines second if you care about maintaining them. Until we create a tool that actually understands things, we will have to grapple with the problem of maintaining software that is written and read by humans.
This seems a little silly to me. It was already possible for a script kiddie to kludge together something they didn’t understand —- copying code snippets from stack overflow, etc. And yet, developers continue to write finely crafted code that they understand at depth. Just because we’ve made this process easier for the script kiddies, doesn’t prevent experts from existing and the market from realizing these experts are necessary to a well run software business.
nothing prevents you from asking an LLM to explain a snippet of code. And then ask it to explain deeper. And then finally doing some quick googling to validate the answers seem correct.
Blindly accepting code used to happen all the time, people copy pasted from stack overflow.
Yes, but copy/paste from stack overflow was a meme that was discouraged. Now we've got people proudly proclaiming they haven't written a line of code in months because AI does everything for them.
>And then finally doing some quick googling to validate the answers seem correct.
There will come a time when there won't be anyone writing information to check against. It'll be AI all the way down. Or at least it will be difficult to discern what's AI or what isn't.
And this is the major problem. People will blindly trust the output of AI because it appears to be amazing, this is how mistakes slip in. It might not be a big deal with the app you're working on, but in a banking app or medical equipment this can have a huge impact.
I feel like I’m being gaslit about these AI code tools. I’ve got the paid copilot through work and I’ve just about never had it do anything useful ever.
I’m working on a reasonably large rails app and it can’t seem to answer any questions about anything, or even auto fill the names of methods defined in the app. Instead it just makes up names that seem plausible. It’s literally worse than the built in auto suggestions of vs code, because at least those are confirmed to be real names from the code.
Maybe these tools work well on a blank project where you are building basic login forms or something. But certainly not on an established code base.
I'm in the same boat. I've tried a few of these tools and the output's generally been terrible to useless big and small. It's made up plausible-sounding but non-existent methods on the popular framework we use, something which it should have plenty of context and examples on.
Dealing with the output is about the same as dealing with a code review for an extremely junior employee... who didn't even run and verify their code was functional before sending it for a code review.
Except here's the problem. Even for intermediate developers, I'm essentially always in a situation where the process of explaining the problem, providing feedback on a potential solution, answering questions, reviewing code and providing feedback, etc takes more time out of my day than it would for me to just _write the damn code myself_.
And it's much more difficult for me to explain the solution in English than in code--I basically already have the code in my head, now I'm going through a translation step to turn it into English.
All adding AI has done is taking the part of my job that is "think about problem, come up with solution, type code in" and make it into something with way more steps, all of which are lossy as far as translating my original intent to working code.
I get we all have different experiences and all that, but as I said... same boat. From _my_ experiences this is so far from useful that hearing people rant and rave about the productivity gains makes me feel like an insane person. I can't even _fathom_ how this would be helpful. How can I not be seeing it?
The biggest lie in all of LLMs is that they’ll work out of the box and you don’t need to take time to learn them.
I find Copilot autocomplete invaluable as a productivity boost, but that’s because I’ve now spent over two years learning how to best use it!
“And it's much more difficult for me to explain the solution in English than in code--I basically already have the code in my head, now I'm going through a translation step to turn it into English.”
If that’s the case, don’t prompt them in English. Prompt them in code (or pseudo-code) and get them to turn that into code that’s more likely to be finished and working.
I do that all the time: many of my LLM prompts are the signature of a function or a half-written piece of code where I add “finish this” at the end.
You bring up a good point! These tools are useless if you can't prompt them effectively.
I am decent at explaining what I want in English. I have coded and managed developers for long enough to include tips on how I want something implemented. So far, I am nothing short of amazed. The tools are nowhere near perfect, but they do provide a non-trivial boost in my productivity. I feel like I did when I first used an IDE.
> Except here's the problem. Even for intermediate developers, I'm essentially always in a situation where the process of explaining the problem, providing feedback on a potential solution, answering questions, reviewing code and providing feedback, etc takes more time out of my day than it would for me to just _write the damn code myself_.
Exactly. And I’ve been telling myself „keep doing that, it lets them teach, otherwise they will never level up and be able to comfortably and reliably work on this codebase without much hand holding. This will pay off”. Which I still think is true to a degree, although less so with every year.
At least with the humans I work with it’s _possible_ and I can occasionally find some evidence that it _could_ be true to hang on to. I’m expending extra effort, but I’m helping another human being and _maybe_ eventually making my own life easier.
What’s the payoff for doing this with an LLM? Even if it can learn, why not let someone else do it and try again next year and see if it’s leveled up yet?
For me, AI is super helpful with one-off scripts, which I happen to write quite often when doing research. Just yesterday, I had to check my assumptions are true about a certain aspect of our live system and all I had was a large file which had to be parsed. I asked ChatGPT to write a script which parses the data and presents it in a certain way. I don't trust ChatGPT 100%, so I reviewed the script and checked it returned correct outputs on a subset of data. It's something which I'd do to the script anyway if I wrote it myself, but it saved me like 20 minutes of typing and debugging the code. I was in a hurry because we had an incident that had to be resolved as soon as possible. I haven't tried it on proper codebases (and I think it's just not possible at this moment) but for quick scripts which automate research in an ad hoc manner, it's been super useful for me.
Another case is prototyping. A few weeks ago I made a prototype to show to the stakeholders, and it was generally way faster than if I wrote it myself.
It’s writing most of my code now. Even if it’s existing code you can feed in the 1-2 files in question and iterate on them. Works quite well as long as you break it down a bit.
It’s not gas lighting the latest versions of GPT, Claude, Lama have gotten quite good
These tools must be absolutely massively better than whatever Microsoft has then because I’ve found that GitHub copilot provides negative value, I’d be more productive just turning it off rather than auditing it’s incorrect answers hoping one day it’s as good as people market it as.
> These tools must be absolutely massively better than whatever Microsoft has then
I haven't used anything from Microsoft (including Copilot) so not sure how it compares, but compared to any local model I've been able to load, and various other remote 3rd party ones (like Claude), no one comes near to GPT4 from OpenAI, especially for coding. Maybe give that a try if you can.
It still produces overly verbose code and doesn't really think about structure well (kind of like a junior programmer), but with good prompting you can kind of address that somewhat.
Probably these services are so tuned (not as in "fine-tuned" ML style) to each individual user that it's hard to get any sort of collective sense of what works and what doesn't. Not having any transparency what so ever into how they tune the model for individual users doesn't help either.
My employer blocks ChatGPT at work and we are forced to use Copilot. It's trash. I use Google docs to communicate with GPT on my personal device. GPT is so much better. Copilot reminds me of GPT3. Plausible, but wrong all the time. GPT 4o and o1 are pretty much bang on most of the time.
My experience is anecdotal, based on a sample size of one. I'm not writing to convince, but to share. Please take a look at my resume to see my background, so you can weight what I write.
I tried cursor because a technically-minded product manager colleague of mine managed to build a damned solid MVP of an AI chat agent with it. He is not a programmer, but knows enough to kick the can until things work. I figured if it worked for him, I might invest an hour of my time to check it out.
I went in with a time-boxed one hour time to install cursor and implement a single trivial feature. My app is not very sophisticated - mostly a bunch of setup flows and CRUD. However, there are some non-trivial things which I would expect to have documented in a wiki if I was building this with a team.
Cursor did really well. It generated code that was close to working. It figured out those not-obvious bits as well and the changes it made kept them in mind. This is something I would not expect from a junior dev, had I not explained those cross-dependencies to them (mostly keeping state synchronized according to business rule across different entities).
It did a poor job of applying those changes to my files. It would not add the code it generated in the right places and mess things up along the way. I felt I was wrestling with it a but too much to my liking. But once I figured this out I started hand-applying it's changes and reviewing them as I incorporated them into my code. This workflow was beautiful.
It was as if I sent a one paragraph description of the change I want, and received a text file with code snippets and instructions where to apply them.
I ended up spending four hours with cursor and giving it more and more sophisticated changes and larger features to implement. This is the first AI tool I tried where I gave it access to my codebase. I picked cursor because I've heard mixed reviews about others, and my time is valuable. It did not disappoint.
I can imagine it will trip up on a larger codebase. These tools are really young still. I don't know about other AI tools, and am planning on giving them a whirl in the near future.
That sounds almost like the complete opposite of my experience and I'm also working in a big Rails app. I wonder how our experiences can be so diametrically different.
What kind of things are you using it for? I’ve tried asking it things about the app and it only gives me generic answers that could apply to any app. I’ve tried asking it why certain things changed after a rails update and it gives me generic troubleshooting advice that could apply to anything. I’ve tried getting it to generate tests and it makes up names for things or generally gets it wrong.
OP here. I am explicitly NOT blindly trusting the output of the AI. I am treating it as a suspicious set of code written by an inexperienced developer. Doing full code review on it.
What you are saying will occasionally happen, but mistakes already happen today.
Standards for quality, client expectations, competition for market share, all those are not going to go down just because there's a new tool that helps in creating software.
New tools bring with them new ways to make errors, it's always been that way and the world hasn't ended yet...
Not much more than reviewing the code of any average dev who doesn't bother doing their due diligence. At least with an AI I immediately get an answer with "Oh yes, you're right, sorry for the oversight" and a fix. Instead of some bullshit explanation to try to convince me that their crappy code is following the specs and has no issues.
That said, I'm deeply saddened by the fact that I won't be passing on a craft I spent two decades refining.
I think there are two types of developers: those who are most excited about building things, and those who are most excited about the craft of programming.
If I can build things faster, then I'm happy to spend most of my time reviewing AI code. That doesn't mean that I never write code. Some things the AI is worse at, or need to be exactly write and its faster to do them manually.
> I think there are two types of developers: those who are most excited about building things, and those who are most excited about the craft of programming.
Love this. You hit the nail right on the head.
I don't know if I fit into one or the other. However, I do know that at times I feel like one, and at other times, the other.
If I am writing another new app and need to build a slew of CRUD code, I don't care about the craft. I mean, I don't want sloppy code, but I do not get joy out of writing what is _almost_ boilerplate. I still want it to reflect my style, but I don't want to type it all out. I already know how it all works in my head. The faster I get it into an IDE the better. Cursor (the AI IDE) allowed me to do this much faster than I would have by hand.
Then there is time where I do want to craft something beautiful. I had one part of this project where I needed to build a scheduler and I had very specific things I wanted it to do. I tried twice to describe what I want but the AI tool did not do what I wanted. It built a working piece of code, but I could not get it to grasp the nuance.
I sat down and wrote the code for the scheduler, but then had to deal with a bunch of edge cases. I took this code, gave it to the AI and told it to implement those edge cases. After reviewing and iterating on it, I had exactly what I wanted.
I think we could see a lot of these AI code tools start to pivot towards product folks for just this reason. They aren't meant for the people who find craft in what they do.
That's essentially what many hands-on engineering managers or staff engineers do today. They spend significant portions of their day reviewing code from more junior team members.
Reviewing and modifying code is more engaging than typing out the solution that is fully formed in my head. If the AI creates something close to what I have in my head from the description I gave it, I can work with it to get it even closer. I can also hand-edit it.
I was in the newspaper field a year or two before desktop publishing took off, then a few years into that evolution. Rooms full of people and Linotype/Compugraphic equipment were replaced by one Mac and a printer.
I shot film cameras for years, and we had a darkroom, darkroom staff, and a film/proofsheet/print workflow. One digital camera later and that was all gone.
Before me publications were produced with hot lead.
"I say your civilization, because as soon as we started thinking for you it really became our civilization, which is of course what this is all about." - Agent Smith
"Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them." - Dune
But I think that quote is a pretty gross mischaracterization of the parent comment.
I similarly am a big fan of Cursor. But I don't "turn [my] thinking over to machines". Even though I review every piece of code it generates and make sure I understand it, it still saves me a ton of time. Heck, some of the most value I get from Cursor isn't even it generating code for me, it's getting to ask questions about a very large codebase with many maintainers where I'm unfamiliar with large chunks. E.g. asking questions like "I would like to do X, are there any places in this codebase that already do this?"
I'm also skeptical of LLMs ever being able to live up to their hype ("AGI is coming sooooon!!!!"), but I still find them to be useful tools in context that can save me a lot of time.
I use it for simple tasks where spotting a mistake is easy. Like writing language binding for a REST API. It's a bunch of methods that look very similar, simple bodies. But it saves quite some work
Or getting keywords to read about from a field I know nothing about, like caching with zfs. Now I know what things to put in google to learn more to get to articles like this one
https://klarasystems.com/articles/openzfs-all-about-l2arc/ which for some reason doesn't appear in top google results for "zfs caching" for me
If you are another "waterboy" doing crud applications, the problem has been solved a long time ago.
What I mean by that is, the "waterboy" (crud "developer") is going to fetch the water (sql query in the database), then bring the water (Clown Bob layer) to the UI...
The size of your Clown Bob layer may vary from one company to another...
This has been solved a long time ago. It has been a well-paid clerk job that is about to come to an end.
If you are doing pretty much anything else, the AI is pathetically incapable of doing any piece of code that makes sense.
Another great example, yesterday, I wanted to know if VanillaOs was using systemD or not. I did scroll through their frontpage but I didn't see anything, so I tried the AI Chat from duckduckgo. This is a frontend for AI chatbots that includes ChatGPT, Llama, Claude and another one...
I started my question by: "can you tell me if VanillaOS is using runit as the init system?"... I wanted initially ask if it was using systemd, but I didn't want to _suggest_ systemd at first.
And of course, all of them told me: "Yeah!! It's using runit!".
Then for all of them I replied, without any fact in hands: "but why on their website they are mentioning to use systemctl to manage the services then?".
And... of course! All of them answered: "Ooouppsss, my mistake, VanillaOS uses systemD, blablabla"....
So at the end, I still don't know which init VanillaOS is using.
If you are trusting the AI as you seem to do, I wish you the best luck my friend... I just hope you will realize the damage you are doing to yourself by "stopping" coding and letting something else do the job. That skill, my friend, is easily lost with time; don't let it evaporate from your brain for some vaporware people are trying to sell you.
> We are at the point where AI writes code for us and we can blindly accept it.
I’m waiting for the day we’ll get the first major breach because someone did exactly that. This is not a case of “if”, it is very much a “when”. I’ve seen enough buggy LLM-generated code and enough people blindly accepting it to be confident in that assertion.
This would indeed be the best way around.The code reviews might even be better - currently, there's little time for them and we often have only one person in the team with much knowledge in the relevant language/framework/application, so reviews are often just "looks OK to me".
It's not quite the same, but I'm reminded of seeing a documentary decades ago which (IIRC) mentioned that a factor in air accidents had been the autopilot flying the plane and human pilots monitoring it. Having humans fly and the computer warn them of potential issues was apparently safer.
> Now, if you could switch it around so that I write the code, and the AI reviews it, that would be something.
I'm sort of doing that. I'm working on a personal project in a new language and asking Claude for help debugging and refactoring. Also, when I don't know how to create a feature, I might ask it to do so for me, but I might instead ask it for hints and an overview so I can enjoy working out the code myself.
The most depressing thing for me is the feeling that I simply cannot trust anything that has been written in the past 2 years or so and up until the day that I die. It's not so much that I think people have used AI, but that I know they have with a high degree of certainty, and this certainty is converging to 100%, simply because there is no way it will not. If you write regularly and you're not using AI, you simply cannot keep up with the competition. You're out. And the growing consensus is "why shouldn't you?", there is no escape from that.
Now, I'm not going to criticize anyone that does it, like I said, you have to, that's it. But what I had never noticed until now is that knowing that a human being was behind the written words (however flawed they can be, and hopefully are) is crucial for me. This has completely destroyed my interest in reading any new things. I guess I'm lucky that we have produced so much writing in the past century or so and I'll never run out of stuff to read, but it's still depressing, to be honest.
>The most depressing thing for me is the feeling that I simply cannot trust anything that has been written in the past 2 years or so and up until the day that I die
Do you think AI has changed that in any way? I remember the sea of excrement overtaking genuine human written content on the Internet around mid 2010s. It is around that time when Google stopped pretending they are a search company and focused on their primary business of advertising.
Before, at least they were trying to downrank all the crap "word aggregators". After, they stopped caring at all.
AI gives even better tools to page rank. Detection of AI generated content is not that bad.
So why don't we have "a new Google" emerge? Simple, because of the monopolistic practices Google did to make the barrier to entry huge. First, 99% of the content people want to search for is behind a login wall (Facebook, Instagram, twitter, YouTube), second almost all CDNs now implement "verify you are human" by default. Third, no one links to other sites. Ever! These 3 things mean a new Google is essentially impossible. Even duck duck go has thrown the towel and subscribed to Bing results.
It has nothing to do with AI, and everything to do with Google. In fact AI might give us the tools to better fight Google.
Google didn't change it, it embodied it. The problem isn't AI, it's the pervasive culture of PR and advertising which appeared in the 50s and eventually consumed its host.
Western industrial culture was based on substance - getting real shit done. There was always a lot of scammery around it, but the bedrock goal was to make physical things happen - build things, invent things, deliver things, innovate.
PR and ad culture was there to support that. The goal was to change values and behaviours to get people to Buy More Stuff. OK.
Then around the time the Internet arrived, industry was off-shored, and the culture started to become one of appearance and performance, not of substance and action.
SEO, adtech, social media, web framework soup, management fads - they're all about impression management and popularity games, not about underlying fundamentals.
This is very obvious on social media in the arts. The qualification for a creative career used to be substantial talent and ability. Now there are thousands of people making careers out of performing the lifestyle of being a creative person. Their ability to do the basics - draw, write, compose - is very limited. Worse, they lack the ability to imagine anything fresh or original - which is where the real substance is in art.
Worse than that, they don't know what they don't know, because they've been trained to be superficial in a superficial culture.
It's just as bad in engineering, where it has become more important to create the illusion of work being done, than to do the work. (Looking at you, Boeing. And also Agile...)
You literally make more money doing this. A lot more.
So AI isn't really a tool for creating substance. It's a tool for automating impression management. You can create the impression of getting a lot of work done. Or the impression of a well-written cover letter. Or of a genre novel, techno track, whatever.
AI might one day be a tool for creating substance. But at the moment it's reflecting and enabling a Potemkin busy-culture of recycled facades and appearances that has almost nothing real behind it.
Unfortunately it's quite good at that.
But the problem is the culture, not the technology. And it's been a problem for a long time.
Thank you, you've stated this all very clearly. I've been thinking about this in terms of "doing work", where you care about the results, and "performing work", where you care about how you are evaluated. I know someone who works in a lab, and pointed out that some of the equipment being used was out of spec and under-serviced to the point that it was essentially a random number generator. Caring about this is "doing work". However, pointing it out made that person the enemy of the greater cohort that was "performing work". The results were not important to them, their metrics about units of work completed was. I see this pattern frequently. And it's hard to say those "performing work" are wrong. "Performing" is rewarded, "doing" is punished - Perhaps right to the top, as many companies are involved in a public performance designed to affect the short-term stock price.
The slippage between work and meaning is due to the arrival of post scarcity.
There isn't enough meaningful work to go around because work is still predicated on delivering some useful transformation. But when 80% of the useful transformations can be done by 20% of the people and we have enough to keep civilization going, you don't need full employment any more. But due to the moral hazard doctrine, the elites want to retain control and discipline as is.
For being afraid of a world where people don't care about maintaining the system anymore because it's not needed to house and feed everyone and people can just do art or learn or rest, the system keeps inventing meaningless work.
We're shipping fruit half way across the world to be packaged and then halfway back to be consumed. None of this is efficient or necessary. But it keeps the control system intact and that's the goal.
This is a pretty clear summary of a real problem in most work environments. I have some thoughts about why, but I'm holding onto your articulation to ruminate on in the future.
Workers are many times more efficient than they were in the 50s or 70s or 80s or 90s. Where are our extra vacation days? Why does the worker have to make up for the efficiency with more work while other people take the gains?
Do you seriously think that the purpose of life is to work all the time most efficiently? Enjoy your lazy job and bask in the ability for human society to be productive without everyone breaking their backs all the time.
focusing on efficiency is very depressing. Machines seek efficiency. Process can be efficient. Assembly lines are efficient. It's all about optimization and quickly focuses on trimming "waste" and packing as much as possible into the smallest space. It removes all that's amazing about human life.
I much prefer a focus on effectiveness (or impact or outcomes, or alternatives). It plays to human strengths, is far less prescriptive and is way more fun!
Some of the most effective actions are incredibly inefficient; sometimes inefficiency is a feature. I received a letter mail thank-you card from our CEO a few years ago. The card has an approx. value of zero dollars, but I know it took the CEO 5-10 mins to write a personal note and sign it, and that she did this dozens of times. The signal here is incredibly valuable! If she used a signing machine, or AI to record a deep fake message I would know or quickly learn, and the value would go negative - all for the sake of efficiency.
I think this is a big part of it. Workers would feel a lot more motivated to do more than just perform if they were given what they know they’re owed for their contribution.
Faced with increasing efficiency, Europe by and large appears to have chosen to work less and let expensive things remain expensive. The US, by contrast, now has ridiculously cheap consumer goods, and works all the time.
Of course GDP increases when the money supply does. It’s like people being incensed at “record corporate profits” amidst inflation - profits will always be record (give or take) because remaining the same is losing money relative to the free money being minted each day, etc. For whatever reason, people naively buy into GDP as a valuable metric, even knowing well that there would be something extremely mysterious going on if that number somehow shrank while the real value of the medium exchange also shrank
"Doing work" vs. "performing work": the epitome of this is consulting. Companies pay huge sums of money to consultants that often spend most of their time "performing work", doing beautiful slides even if the content and reasoning is superficial or even dubious, creating reports that are just marketing bullshit, framing the current mission in a way that makes it possible to capture additional projects and bill the client even more. Almost everything is bullshit.
For better or for worse, you’ve described the exact output some of those client companies want, so they can show off the shiny slides and have the status symbol of expensive consultants.
Just yesterday I watched a video blaming China for "oversupplying" electric cars at low prices and how that was hurting car manufacturers elsewhere e.g Germany. These manufacturers were trying to secretly lobby for tariffs to be placed on Chinese cars, while publicly denying this because THEIR largest and growing market was China. They had some "expert" talk about how China is known to oversupply and is doing this to recover from their housing bubble. All of these experts were preaching the virtues of Western free-trade to the east when they were the ones exporting TO Asia 10 years ago, but now the balance flips and they all tell you how important tariffs are and how evil China is instead..
In summary, China makes useful things in mass and sells them to get out of a recession,the West prints money instead and shits on China for doing it better. They preach free trade while it helps them and put up tariffs when it doesn't.
I'm not Chinese or some mega fan but it really struck me how corrupt and full of propaganda western culture is becoming and people don't seem to recognize it.
Maybe we should stop subsidies to buy electric cars and let China subsidize on the production side instead, while lowering taxes on the production of our own cars.
The thing that I struggle with is I agree with it, but I also get a lot of value in using AI to make me more productive - to me, it feels like it lets me focus on producing substance and actions, freeing me up from having to some tedious things in some tedious ways. Without getting into the debate about if it's productive overall, there are certain tasks which it feels irrefutably fast and effective at (e.g. writing tests).
I do agree with the missing substance with modern generative AI: everyone notices when it's producing things in that uncanny valley, and if no human is there to edit that, it makes people uncomfortable.
The only way I can reconcile the almost existential discomfort of AI against my actual day-to-day generally-positive experience with AI is to accept that AI in itself isn't the problem. Ultimately, it is an info tool, and human nature makes people spam garbage for clicks with it.
People will do the equivalent of spam garbage for clicks with any new modern thing, unfortunately.
Getting the most out of latest information of a society has probably always been a cat and mouse game of trying to find the areas where the spam-garbage-for-clicks people haven't outnumbered use-AI-to-facilitate-substance people, like here, hopefully.
Just one nitpick. The thing about test is that it’s repetitive enough to be automated (in a deterministic way) or abstracted into a framework. You don’t need an AI to generate it.
I find it helpful for generating automated test suites in the style of the rest of the codebase. When working across multiple projects and clients, it reduces the mental load of having to remember or figure out how tests are expected to work in each codebase.
I agree with your theory about tests. The reality of it is most code is garbage - often including my own - and in a lot of environments, the task is to get the job done in a way that fits in with what's there.
While I occasionally have the pleasure of creating or working with a test suite that's interesting and creative relative to the code under test, the vast majority of unit tests by volume are slop. Does it call the mock? Does it use the return value? Does "if err != nil { return err }" in fact stop and return the error?
This stuff is a perfect candidate for LLM generation.
AI seems really good at producing middling content, and if you make your living writing mediocre training courses, or marketing collateral, or code, or tests you're in big trouble. I question how valuable this work is though, so are we increasing productivity by utilizing AI, or just getting efficient at a suboptimal game? I for one just refuse to play.
> You can create the impression of getting a lot of work done. Or the impression of a well-written cover letter. Or of a genre novel, techno track, whatever.
Yeah, one of their most "effective" uses is to counterfeit signals that we have relied on--wisely or not--to estimate deeper practical truths. Stuff like "did this person invest some time into this" or "does this person have knowledge of a field" or "can they even think straight."
Oh, sure, qualitatively speaking it's not new, people could have used form-letters, hired a ghostwriter, or simply sank time and effort into a good lie... but the quantitative change of "Bot, write something that appears heartfelt and clever" is huge.
In some cases that's devastating--like trying to avert botting/sockpuppet operations online--and in others we might have to cope by saying stuff like: "Fuck it, personal essays and cover letters are meaningless now, just put down the raw bullet-points."
>Western industrial culture was based on substance - getting real shit done. There was always a lot of scammery around it, but the bedrock goal was to make physical things happen - build things, invent things, deliver things, innovate.
For a very short period between 1945 to 1980 while the generation who remembered the great depression and WWII was in charge. It's been longer since that's not been the case. And it wasn't the case for most of history before then.
Hah! I'm certain a lot of people want to believe it's bogus, mostly because of what that says about their character. Observed reality suggests there's some truth here. An example in miniature would be the frequency with which family fortunes are lost within a generation or two.
You can outfit an adult life with all of the useful manufactured objects that would reasonably improve it for a not-very-impressive sum. Beyond that it's just clutter (going for quantity) or moving into the lifestyle/taste/social-signaling domain anyway (going for quality). There is just not an unlimited amount of alpha in making physical things. The social/thought/experiential domain is a much bigger opportunity.
Echoing other comments in gratitude for this very clear articulation of feelings I share, but have not manifested so well. Just wanted to add two connected opinions that round out this view.
1) This consuming of the host is only possible on the one hand because the host has grown so strong, that is the modern global industrial economy is so efficient. The doing stuff side of the equation is truly amazing and getting better (some real work gets done either by accident or those who have not-succumbed to PR and ad culture), and even this drop of "real work" produces enough material wealth to support (at least a lot of) humanity. We really do live in a post scarcity world from a production perspective, we just have profound distribution and allocation problems.
2) Radical wealth inequality profoundly exacerbates the problem of PR and ad culture. If everyone has some wealth doing things that help many people live more comfortably is a great way to become wealthy. But if very few people have wealth, then doing a venture capital FOMO hustle on the wealthy is anyone's best ROI. Radical wealth inequality eventually breaks all the good aspects of capitalist/market economies.
Thanks for writing your comment, I think it’s a public service.
Cultural commentary that makes complex long term trends simple to understand isn’t often this clear or concise. What really makes it powerful though is that it manages to stay in a relatively detached observer-mode without becoming an angry rant. And so rather than provoking (understandable!) anger in others, hopefully it’s inviting more reflection.
People who haven’t thought about it this way might take a harder look at what they are doing and who they really want to be. People that are already thinking along these lines will probably benefit from a reminder that they aren’t crazy.
Very well written. I assume you haven't read "Simulacra and Simulation" by Jean Baudrillard, that's why your description is so authentic and is more convincing then just referring to the book. Saved this post for future reference.
> Western industrial culture was based on substance - getting real shit done.
And what did that get us? Radium poisoning and microplastics in every organ of virtually all animals living within thousands of miles of humans. Our reach has always exceeded our grasp.
Some great grand ancestor of mine was a civil servant, a great achievement given his peasant background. The single skill that enabled it was the knowledge of calligraphy. He went to school and wrote nicely and that was sufficient.
The flip side was, calligraphy was sufficient evidence for both his education to whoever hired him, and for a recipient of a document, of its official nature. Calligraphy itself or course didn't make him efficient or smart or fair.
That's long gone of course, but we had similar heuristics. I am reminded of the Reddit story about an AI-generated mushroom atlas that had factual errors and lead to someone getting poisoned. We can no longer assume that a book is legit simply because it looks legit. The story of course is from reddit, so probably untrue, but it doesn't matter - it totally could be true.
LLMs are fantastic at breaking our heuristics as to what is and isn't legit, but not as good at being right.
> We can no longer assume that a book is legit simply because it looks legit.
The problem is that this has been an issue for a long time. My first interactions with the internet in the 90s came along with the warning "don't automatically trust what you read on the internet".
I was speaking to a librarian the other day who teaches incoming freshman how to use LLMs. What was shocking to me is that the librarian said a majority of the kids trust what the computer says by default. Not just LLMs, but generally what they read. That's such a huge shift from my generation. Maybe LLM education will shift people back toward skepticism - unlikely, but I can hope.
One of the issues today is the volume of content produced, and that journalism and professional writing is dying. LLMs produce large amounts of "good enough" quality to make a profit.
In the 90s we could reasonably trust that that the major news sites and corporate websites was true, while random forums required a bit more critical reading. Today even formerly trusted sites may be using LLMs to generate content along with automatic translations.
I wouldn't necessarily put the blame on LLMs, this just make it easier. The trolls and spammers was always there, now they just have a more powerful tool. The commercial sites now have a tool they don't understand, which they apply liberally, because it reduces cost, or their staff use it, to get out of work, keep up with deadlines or to cover up incompetence. So, not the fault of the LLMs, but their use is worsening existing trends.
> Today even formerly trusted sites may be using LLMs to generate content along with automatic translations.
Yep - or they're commingling promotional content with their journalism, a la Forbes / CNN / CNET / About.com / etc. There's still quality content online but it's getting harder to find under the tidal wave of garbage.
> I was speaking to a librarian the other day who teaches incoming freshman how to use LLMs. What was shocking to me is that the librarian said a majority of the kids trust what the computer says by default. Not just LLMs, but generally what they read. That's such a huge shift from my generation.
I think that previous generations were not any different. For most people, trusting is the default mode and you need to learn to distrust a source. I know many people who still have not learned that about the internet in general. These are often older people. They believe insane things just because there exists a nicely looking website claiming that thing.
Not sure of the context here is for "previous generation" but I've been around since early in the transition from university/military network to public network, and the reality was the internet just wasn't that big, and it was primarily made up of people who looked, acted and valued the same things.
Now it's not even the website of undetermined providence that is believed; positions are established based on just the headline, shared 2nd or 3rd hand!
> The problem is that this has been an issue for a long time. My first interactions with the internet in the 90s came along with the warning "don't automatically trust what you read on the internet".
I received the same warnings, actually it was more like “don’t trust everything you read on the internet”, but it quickly became apparent that the last three words were redundant, and could have been rephrased more accurately as “don’t trust everything you read and hear and see”.
Our parents and teachers were living with their own fallacious assumptions and we just didn’t know it at the time, but most information is very pliable. If you can’t change what someone sees, then you can probably change how they see it.
I feel like there was also a brief window where "many amateur eyes in public" trumped "private experts"; wikipedia, open source software, etc. This doesn't seem the case in a hyper-partisan and bifurcated society where there is little trust.
> Our parents and teachers were living with their own fallacious assumptions and we just didn’t know it at the time, but most information is very pliable.
Indeed. When I was 14-18 in the UK, the opinion pieces in the news were derogatorily describing "media studies" as "Mickey Mouse studies".
In retrospect such courses were teaching critical analysis of media sources, in much the same way that my history GCSE went into the content of historical media and explored how both primary and secondary sources each had the potential to be either accurate or biased.
Even now, even here, I see people treat the media itself as pure and only the people being reported upon as capable of wrongdoing — e.g. insisting that climate scientists in the 70s generally expected an imminent ice age, because that's what the newspapers were saying.
>That's long gone of course, but we had similar heuristics.
To quote someone about this:
>>All that is solid melts into air, all that is holy is profaned, and man is at last compelled to face with sober senses his real conditions of life.
A book looking legit, a paper being peer reviewed, an expert saying something, none of those things were _ever_ good heuristics. It's just that it was the done thing. Now we have to face the fact that our heuristics are obviously broken and we have to start thinking about every topic.
To quote someone else about this:
>>Most people would rather die than think.
Which explains neatly the politics of the last 10 years.
> To quote someone about this:
>>All that is solid melts into air, all that is holy is profaned, and man is at last compelled to face with sober senses his real conditions of life.
So, same as it ever was?
Smoke, nothing but smoke. [That’s what the Quester says.]
There’s nothing to anything—it’s all smoke.
What’s there to show for a lifetime of work,
a lifetime of working your fingers to the bone?
One generation goes its way, the next one arrives,
but nothing changes—it’s business as usual for old planet earth.
The sun comes up and the sun goes down,
then does it again, and again—the same old round.
The wind blows south, the wind blows north.
Around and around and around it blows,
blowing this way, then that—the whirling, erratic wind.
All the rivers flow into the sea,
but the sea never fills up.
The rivers keep flowing to the same old place,
and then start all over and do it again.
Everything’s boring, utterly boring—
no one can find any meaning in it.
Boring to the eye,
boring to the ear.
What was will be again,
what happened will happen again.
There’s nothing new on this earth.
Year after year it’s the same old thing.
Does someone call out, “Hey, this is new”?
Don’t get excited—it’s the same old story.
Nobody remembers what happened yesterday.
And the things that will happen tomorrow?
Nobody’ll remember them either.
Don’t count on being remembered.
Culd be my KJV upbringing talking, but personally I think there's an informative quality to calling it "vanity" over smoke.
And there's more reasons not to simply compare the modern challenges of image and media with the ancient grappling with impermanence. Tech may only truly change the human condition rarely, but it frequently magnifies some aspect of it, sometimes so much that the quantitative change becomes a qualitative one.
And in this case, what we're talking about isn't just impermanence and mortality and meaning as the preacher/quester is. We'd be lucky if it's business as usual for old planet earth, but we've managed to magnify our ability to impact our environment with tech to the point where winds, rivers, seas, and other things may well change drastically. And as for "smoke", it's one thing if we're dust in the wind, but when we're dust we can trust, that enables continuity and cooperation. There's always been reasons for distrust, but with media scale, the liabilities are magnified, and now we've automated some of them.
The realities of human nature that are the seeds of the human condition are old. But some of the technical and social machinery we have made to magnify things is new, and we can and will see new problems.
One is a complaint that everything is constantly changing, the other that nothing ever changes. I don't think you could misunderstand what either is trying to say harder if you tried.
Heuristics don't have to be perfect to be useful so long as they improve the efficacy of our attentions. Once that breaks down society must follow because thinking about every topic is intractable.
The mushroom thing is almost certainly true. There's tons of trash AI generated foraging books being published to Amazon. Atomic Shrimp has a video on it.
> Some great grand ancestor of mine was a civil servant, a great achievement given his peasant background. The single skill that enabled it was the knowledge of calligraphy. He went to school and wrote nicely and that was sufficient.
Similar story! Family lore has it that he was from a farming family of modest means, but he was hired to write insurance policies because of his beautiful handwriting, and this was a big step up in the world.
> The story of course is from reddit, so probably untrue, but it doesn't matter - it totally could be true.
What?! Someone just made up something and then got mad at it. This is specially weird when you even acknowledge its a made up story. If we start evaluating new things like this nothing will ever progress.
Bots are now blocked because they've been abusive. When you host content on the internet, it's not fun to have bots bring your server down or inflate your bandwidth price. Google's bot is actually quite well-behaved. The other problem has been the recent trend in AI, and I can understand blockers being put in place, since AI is essentially plagiarizing content without attribution. But I'd blame OpenAI more at this point.
I also don't think you can blame Google for the centralization behind closed gardens. Or for why people no longer link to other websites. That's ridiculous.
And you should be attributing them the fact that the web is still alive.
>I remember the sea of excrement overtaking genuine human written content on the Internet around mid 2010s.
Things have not changed much really. This was true since the dawn of man-kind (and woman-kind from the man-kind rib of course) even before there writings was invented, in the form of gossip.
The internet/AI now carries on the torch of our ancestral inner calling, lol.
>AI gives even better tools to page rank. Detection of AI generated content is not that bad.
It is an arms race between the people generating crap (for various nefarious purposes) and those trying to separate find useful content amongst the ever growing pile of crap. And it seems to me it is so much easier to generate crap, that I can't see how the good guys can possibly win.
> I remember the sea of excrement overtaking genuine human written content on the Internet around mid 2010s.
I mean the AI is trained and modeled on this excrement. It makes sense. As much as people think AI content is raw garbage… they don’t realize that they are staring into a mirror.
What fascinates me about your comment is that you are expressing that you trusted what you read before. For me, LLMs don't change anything. I already questioned the information before and continue to do so.
Why do you think that you could trust what you read before?
Is it now harder for you to distinguish false information, and if so, why?
In the past, you had to put a lot of effort to produce a text which seemed to be high quality, especially when you knew nothing about the subject. By the look of text and the usage of the words, you could tell how professional the writer was and you had some confidence that the writer knew something about the subject. Now, that is completely removed. There is no easy filter anymore.
While the professional looking text could have been already wrong, the likelihood was smaller, since you usually needed to know something at least in order to write convincing text.
Writing a text of decent quality used to constitute proof of work. This is now no longer the case, and we haven't adapted to this assumption becoming invalid.
For example, when applying to a job, your cover letter used to count as proof of work. The contents are less important than the fact that you put some amount of effort in it, enough to prove that you care about this specific vacancy. Now this basic assumption has evaporated, and job searching has become a meaningless two-way spam war, where having your AI-generated application selected from hundreds or thousands of other AI-generated applications is little more than a lottery.
This. I am very picky about how I use ML still, but it is unsurpassed as a virtual editor. It can clean up grammar and rephrase things in a very light way, but it gives my prose the polish I want. The thing is, I am a very decent writer. I wrote professionally for 18 years as a part of my job delivering reports of high quality as my work product. So, it really helps that I know exactly what “good” looks like by my standards. ML can clean things up so much faster than I can and I am confident my writing is organic still, but it can fix up small issues, find mistakes, etc very quickly. A word change here or there, some punctuation, that is normal editing. It is genuinely good at light rephrasing as well, if you have some idea of what intent you want.
When it becomes obvious, though, is when people let the LLM do the writing for them. The job search bit is definitely rough. Referrals, references, and actual accomplishments may become even more important.
As usual, LLMs are an excellent tool when you already have a decent understanding of the field you're interested in using them in. Which is not the case of people posting in social media or creating their first programs. That's where the dullness and noise come from.
The noise ground has been elevated 100x by LLMs. It was already bad before but it's accelerated the trend.
So, yes, we should have never been trusting anything online but before LLMs we could rely on our brains to quickly identify the bad. Nowadays, it's exhausting. Maybe we need a LLM trained on spotting LLMs.
This month, I, with decades of experience, used Claude Dev as an experiment to create a small automation tool. After countless manual fixes, it finally worked and I was happy. Until I gave thr whole thing a decent look again and realized what a piece of garbage I had created. It's exhausting to be on the lookout for these situations. I prefer to think things through myself, it's a more rewarding experience with better end results anyway.
Not to sound too dismissive, but there is a distinct learning curve when it comes to using models like Claude for code assist. Not just the intuition when the model goes off the rails, but also what to provide it in the context, how and what to ask for etc. Trying it once and dismissing it is maybe not the best experimental setup.
I've been using Zed recently with its LLM integration so assist me in my development and its been absolutely wonderful, but one must control tightly what to present to the model and what to ask for and how.
LLM's are a great onramp to filling in knowledge that may have been lost to age or updated to their modern classification. For example, I didn't know Hokkien and Haka are distinct linguistic branches within the Sino-Tibetan language and warrants more (personal) research into the subject. And all this time, without the internet, we often just colloquially called it Taiwanese.
How often do you go back to your encyclopedia hard copies only to find whatever knowledge you may have absorbed have already been deprecated? Or that information from Wikipedia may have changed at moments without notice, have never read or, dare I say, included a political bias to them?
Maybe I should have worded it better as a "beginner" or "intermediate" knowledge onramp and/or filler. For example, I have asked it on occasion to translate into traditional Mandarin in parallel for every English response. It helps tremendously in trying to rebuild that bridge that may have been burned long ago.
It is lost in a sense that you had no idea about such possibility and you did not know to search it in the first hand, while I believe that in this case LLM brought it up as a side note.
Such fortuitous stumblings happen all the time without LLMs (and in regular libraries, for those brave enough to use them). It's just the natural byproduct of doing any kind of research.
Most of my knowledge comes from physical encyclopedia and download the wikipedia text dump (internet was not readily available). You search for one thing and just explore by clicking.
This is my go-to process whenever I write anything now:
1. I use dictation software to get my thoughts out as a stream of consciousness.
2. Then, I have ChatGPT or Claude refine it into something coherent based on a prompt of what I'm aiming for.
3. Finally, I review the result and make edits where needed to ensure it matches what I want.
This method has easily boosted my output by 10x, and I'd argue the quality is even better than before. As a non-native English speaker, this approach helps a lot with clarity and fluency. I'm not a great writer to begin with, so the improvement is noticeable. At the end of the day, I’m just a developer—what can I say?
Yeah, this is how I use it too. I tend to be a very dry writer, which isn't unusual in science, but lately I've taken to writing, then asking an LLM to suggest improvements.
I know not to trust it to be as precise as good research papers need to be, so I don't take its output, it usually helps me reorder points or use different transitions which make the material much more enjoyable to read. I also find it useful for helping to come up with an opening sentence from which to start writing a section.
Great opportunity to get ahead of all the lazy people who use AI for a cover letter. Do a video! Sure, AI will be able to do that soon, but then we (not lazy people, who care) will come up with something even more personal!
> While the professional looking text could have been already wrong, the likelihood was smaller...
I don't criticise you for it, because that strategy is both rational and popular. But you never checked the accuracy of your information before so you have no way of telling if it has gotten more or less accurate with the advent of AI. You were testing for whether someone of high social intelligence wanted you to believe what they said rather than if what they said was true.
I guess the complaint is about losing this proxy to gain some assurance for little cost. We humans are great at figuring out the least amount of work that's good enough.
Now we'll need to be fully diligent, which means more work, and also there'll be way more things to review.
There’s not enough time in the day to go on a full bore research project about every sentence I read, so it’s not physically possible to be “fully diligent.”
The best we can hope for is prioritizing which things are worth checking. But even that gets harder because you go looking for sources and now those are increasingly likely to be LLM spam.
Traditionally, humans have addressed the imbalance between energy-to-generate and energy-to-validate by building another system on top, such as one which punishes fraudsters or at least allows other individuals to efficiently disassociate from them.
Unfortunately it's not clear how this could be adapted to the internet and international commerce without harming some of the open-ness aspects we'd like to keep.
I'd argue people clearly don't care about the truth at all - they care about being part of a group and that is where it ends. It shows up in things like critical thinking being a difficult skill acquired slowly vs social proof which humans just do by reflex. Makes a lot of sense, if there are 10 of us and 1 of you it doesn't matter how smartypants you may be when the mob forms.
AI does indeed threaten people's ability to identify whether they are reading work by a high status human and what the group consensus is - and that is a real problem for most people. But it has no bearing on how correct information was in the past vs will be in the future. Groups are smart but they get a lot of stuff wrong in strategic ways (it is almost a truism that no group ever identifies itself or its pursuit of its own interests as the problem).
> I'd argue people clearly don't care about the truth at all
Plenty of people care about the truth in order to get advantages over the ignorant. Beliefs aren't just about fitting in a group, they are about getting advantages and making your life better, if you know the truth you can make much better decisions than those who are ignorant.
Similarly plenty of people try to hide the truth in order to keep people ignorant so they can be exploited.
> if you know the truth you can make much better decisions than those who are ignorant
There are some fallacious hidden assumptions there. One is that "knowing the truth" equates to better life outcomes. I'd argue that history shows more often than not that what one knows to be true best align with prevailing consensus if comfort, prosperity and peace is one's goal, even if that consensus is flat out wrong. The list is long of lone geniuses who challenged the consensus and suffered. Galileo, Turing, Einstein, Mendel, van Gogh, Darwin, Lovelace, Boltzmann, Gödel, Faraday, Kant, Poe, Thoreau, Bohr, Tesla, Kepler, Copernicus, et. al. all suffered isolation and marginalization of some degree during their lifetimes, some unrecognized until after their death, many living in poverty, many actively tormented. I can't see how Turing, for instance, had a better life than the ignorant who persecuted him despite his excellent grasp of truth.
You are thinking too big, most of the time the truth is whether a piece of food is spoiled or not etc, and that greatly affects your quality of life. Companies would love to keep you ignorant here so they can sell you literal shit, so there are powerful forces wanting to keep you ignorant, and today those powerful forces has way stronger tools than ever before working to keep you ignorant.
You're implying that there is an absolute Truth and that people only need to do [what?] to check if something is True. But that's not True. We only have models of how reality works, and every model is wrong - but some are useful.
When dealing with almost everything you do day by day, you have to rely on the credibility of the source of the information you have. Otherwise how could you know that the can of tuna you're going to eat is actually tuna and not some venomous fish? How do you know that you should do what your doctor told you? Etc. etc.
> You're implying that there is an absolute Truth and that people only need to do [what?] to check if something is True. But that's not True. We only have models of how reality works, and every model is wrong - but some are useful.
I am not sure I am following - you don't know if there is anything that is really true, but you presume there isn't and that model of "the only truth is the absence of truth" is useful to you because it allows you to ... what exactly?
My new cheap proxy to save mental cost: pay to search on kagi, sort results by tracker count. My hope is fewer trackers correlates with lower incentives to seo spam. This may change but seems to work decently for now.
In the past, with a printed book or journal article, it was safe to assume that an editor had been involved, to some degree or another challenging claimed facts, and the publisher also had an interest in maintaining their reputation by not publishing poorly researched or outright false information. You would also have reviewers reading and reacting to the book in many cases.
All of that is gone now. You have LLMs spitting their excrement directly onto the web without so much as a human giving it a once-over.
How do you "check the accuracy of your information" if all the other reliable-sounding sources could also be AI generated junk? If it's something in computing, like whether something compiles, you can sometimes literally check for yourself, but most things you read about are not like that.
>But you never checked the accuracy of your information before so
They didn't say that and that's not a fair or warranted extrapolation.
They're talking about a heuristic that we all use, as a shorthand proxy that doesn't replace but can help steer the initial navigation in the selection of reliable sources, which can be complemented with fact checking (see the steelmanning I did there?). I don't think someone using that heuristic can be interpreted as tantamount to completely ignoring facts, which is a ridiculous extrapolation.
I also think is misrepresents the lay of the land, which is that in the universe of nonfiction writing, I don't think that there's a fire hose of facts and falsehoods indistinguishable in tone. I think there's in fact a reasonably high correlation between the discernible tone of impersonal professional and credible information, which, again (since this seems to be a difficult sticking point) doesn't mean that the tone substitutes for the facts which still need to be verified.
The idea that information and misinformation are tonally indistinguishable is, in my experience, only something believed by post-truth "do you own research" people who think there are equally valid facts in all directions.
There's not, for instance, a Science Daily of equally sciency sounding misinformation. There's not a second different IPCC that publishes a report with thousands of citations which are all wrong, etc. Misinformation is out there but it's not symmetrical, and understanding that it's not symmetrical is an important aspect of information literacy.
This is important because it goes to their point, which is that something has changed, in the advent of LLMS. That symmetry may be coming, and it's precisely the fact that it wasn't there before that is pivotal.
Interesting points! Doesn't sound impossible with an AI that's wrong less often than an average human author (if the AIs training data was well curated).
I suppose a related problem is that we can't know if the human who posted the article, actually agrees with it themselves.
(Or if they clicked "Generate" and don't actually care, or even have different opinions)
I think you overestimate the value of things looking professional. The overwhelming majority of books published every year are trash, despite all the effort that went into research, writing, and editing them. Most news is trash. Most of what humanity produces just isn't any good. An top expert in his field can leave a typo-riddled comment in a hurry that contains more valuable information than a shelf of books written on the subject by lesser minds.
AIs are good at writing professional looking text because it's a low bar to clear. It doesn't require much intelligence or expertise.
> AIs are good at writing professional looking text because it's a low bar to clear. It doesn't require much intelligence or expertise.
AIs are getting good at precisely imitating your voice with a single sample as reference, or generating original music, or creating video with all sorts of impossible physics and special effects. By your rationale, nothing “requires much intelligence or expertise”, which is patently false (even for text writing)
My point is that writing a good book is vastly more difficult than writing a mediocre book. The distance between incoherent babble and a mediocre book is smaller than the distance between a mediocre book and a great book. Most people can write professional looking text just by putting in a little bit of effort.
I think you underestimate how high that bar is, but I will grant that it isn’t that high. It can be a form of sophistry all of its own. Still, it is a difficult skill to write clearly, simply, and without a lot of extravagant words.
> While the professional looking text could have been already wrong, the likelihood was smaller, since you usually needed to know something at least in order to write convincing text.
Although, there were already before tons of "technical influencers" before that who excelled at writing, but didn't know deeply what they were writing about.
They give a superficially smart look, but really they regurgitate without deep understanding.
>In the past, you had to put a lot of effort to produce a text which seemed to be high quality, especially when you knew nothing about the subject. By the look of text and the usage of the words, you could tell how professional the writer was and you had some confidence that the writer knew something about the subject. Now, that is completely removed. There is no easy filter anymore.
That is pretty much true also for other media, such as audio and video. Before digital stuff become mainstream pics are developed in the darkroom, and film are actually cut with scissors. A lot of effort are put into producing the final product. AI has really commoditized for many brain related tasks. We must realize the fragile nature of digital tech and still learn how to do these by ourselves.
it's obvious when text has been produced by chatGPT with the default prompt - but there's probably loads of text on the internet which doesn't follow AI's usual prose style that blends in well.
Even when I try some other variation of prompts or writing styles there's always this sense of "perfectness", with all paragraph lengths being too perfect, length and the style of it being like that.
>> While the professional looking text could have been already wrong, the likelihood was smaller, since you usually needed to know something at least in order to write convincing text.
...or...the likelihood of text being really wrong pre-LLMs was worse because you needed to be a well-capitalized player to pay your thoughts into public discourse. Just look at our global conflicts and you see how much they are driven by well-planned lobbying, PR, and...money. That is not new.
> By the look of text and the usage of the words, you could tell how professional the writer was and you had some confidence that the writer knew something about the subject
How did you know this unless you also had the same or more knowledge than the author?
It would seem to me we are as clueless now as before about how to judge how skilled a writer is without requiring to already posses that very skill ourselves.
Reading was a form of connecting with someone. Their opinions are bound to be flawed, everyone's are - but they're still the thoughts and words of a person.
This is no longer the case. Thus, the human factor is gone and this reduces the experience to some of us, me included.
This is exactly what’s at stake. I heard an artist say one time that he’d rather listen to Bob Dylan miss a note than listen to a song that had all the imperfections engineered out of it.
They're not connecting to the autotune, but to the artist. People have a lot of opinions about Taylor Swift's music but "not being personal enough" is definitely not a common one.
If you wanna advocate for unplugged music being more gratifying, I don't disagree, but acting like the autotune is what people are getting out of Taylor Swift songs is goofy.
I have no idea about Taylor Swift so I'll ask in general: can't we have a human showing an autotuned personality? Like, you are what you are in private, but in interviews you focus on things suggested by your AI conselor, your lyrics are fine tuned by AI, all this to show a better marketable personality? Maybe that's the autotune we should worry about. Again, nothing new (looking at you, Village People) but nowadays the potential powered by AI is many orders of magnitude higher... you could say yes only until the fans catch wind of it, true, but by that time the next figure shows up and so on. Not sure where this arms escalation can lead us. Because also acceptance levels are shifting, so what we reject today as unacceptable lies could be fine tomorrow, look already at the AI influencers doing a decent job while overtly fake.
I’m convinced it’s already being done, or at least played with. Lots of public figures only speak through a teleprompter. It would be easy to put a fine tuned LLM on the other side of that teleprompter where even unscripted questions can be met with scripted answers.
I think the key thing here is equating trust and truth. I trust my dog, a lot, more than most humans frankly. She has some of my highest levels of trust attainable, yet I don’t exactly equate her actions with truth. She often barks when there’s no one at the door or at false threats she doesn’t know aren’t real threats and so on. But I trust she believes it 100% and thinks she’s helping me 100%.
What I think OP was saying and I agree with is that connection, that knowing no matter what was said or how flawed or what motive someone had I trusted there was a human producing the words. I could guess and reasons the other factors away. Now I don’t always know if that is the case.
If you’ve ever played a multiplayer game, most of the enjoyable experience for me is playing other humans. We’ve had good game AIs in many domains for years, sometimes difficult to distinguish from humans, but I always lost interest if I didn’t know I was in fact playing and connecting with another human. If it’s just some automated system I could do that any hour of the day as much as I want but it lacked the human connection element, the flaws, the emotion, the connection. If you can reproduce that then maybe it would be enjoyable but that sort of substance has meaning to many.
It’s interesting to see a calculator quickly spit out correct complex arithmetic but when you see a human do it, it’s more impressive or at least interesting, because you know the natural capability is lower and that they’re flawed just like you are.
For me, the problem has gone from “figure out the author’s agenda” to “figure out whether this is a meaningful text at all,” because gibberish now looks a whole lot more like meaning than it used to.
This has been a problem on the internet for the past decade if not more anyway, with all of the seo nonsense. If anything, maybe it's going to be ever so slightly more readable.
I don't know what you're talking about. Most people don't think of SEO, Search Engine Optimization, Search Performance, Search Engine Relevance, Search Rankings, Result Page Optimization, or Result Performance when writing their Article, Articles, Internet Articles, News Articles, Current News, Press Release, or News Updates...
Perhaps "trust" was a bit misplaced here, but I think we can all agree on the idea: Before LLMs, there was intelligence behind text, and now there's not. The I in LLM stands for intelligence, as written in one blog. Maybe the text never was true, but at least it made sense given some agenda. And like pointed out by others, the usual text style and vocabulary signs that could have been used to identify expertise or agenda are gone.
> Perhaps "trust" was a bit misplaced here, but I think we can all agree on the idea: Before LLMs, there was intelligence behind text, and now there's not. The I in LLM stands for intelligence, as written in one blog. Maybe the text never was true, but at least it made sense given some agenda.
Nope. A lot of people just wrote stuff. There were always plenty of word salad blogs (and arguably entire philosophy journals) out there.
scale makes all the difference. society without trust falls apart. it's good if some people doubt some things, but if everyone necessarily must doubt everything, it's anarchy.
A good part of society, the foundational part, is trust. Trust between individuals, but also trust in the sense that we expect things to behave in a certain way. We trust things like currencies despite their flaws. Our world is too complex to reinvent the wheel whenever we need to do a transaction. We must believe enough in a make-believe system to avoid perpetual collapse.
Perhaps that anarchy is the exact thing we need to convince everyone to revolt against big tech firms like Google and OpenAI and take them down by mob rule.
Propaganda works by repeating the same in different forms. Now it is easier to have different forms of the same, hence, more propaganda. Also, it is much easier to iinfluence whatever people write by influencing the tool they use to write.
Imagine that AI tools sway generated sentences to be slightly close, in summarisation space, to the phrase "eat dirt" or anything. What would happen?
I think it is a totally different threat. Excluding adversarial behavior, humans usually produce information with a quality level that is homogeneous (from homogeneously sloppy to homogeneously rigurous).
AI otoh can produce texts that are quite accurate globally with some totally random hallucinations here and there. It makes it quite harder to identify
There are topics on which you should be somewhat suspicious of anything you read, but also many topics where it is simply improbable that anyone would spend time maliciously coming up with a lie. However, they may well have spicy autocomplete imagine something for them. An example from a few days ago: https://news.ycombinator.com/item?id=41645282
> For me, LLMs don't change anything. I already questioned the information before and continue to do so.
I also did, but LLM increased the volume of content, which forces my brain first try to identify if content is generated by LLMs, which is consuming a lot of energy and makes brain even less focused, because now it's primary goal is skimming quickly to identify, instead of absorbing first and then analyzing info
The web being polluted only makes me ignore more of it.
You already know some of the more trustworthy sources of information, you don't need to read a random blog which will require a lot more effort to verify.
Even here on hackernews, I ignore like 90% of the spam people post. A lot of posts here are extremely low effort blogs adding zero value to anything, and I don't even want to think whether someone wasted their own time writing that or used some LLM, it's worthless in both cases.
There's a quantity argument to be made here - before, it used to be hard to generate large amounts of plausible but incorrect text. Now it easy. Similar to surveillance before/after smartphones + the internet - you had to have a person following you vs just soaking up all the data on the backbone.
Debunking bullshit inherently takes more effort than generating bullshit, so the human factor is normally your big force multiplier. Does this person seem trustworthy? What else have they done, who have they worked with, what hidden motivations or biases might they have, are their vibes /off/ to your acute social monkey senses?
However with AI anyone can generate absurd torrential flows of bullshit at a rate where, with your finite human time and energy, the only winning move is to reject out of hand any piece of media that you can sniff out as AI. It's a solution that's imperfect, but workable, when you're swimming through a sea of slop.
Debugging is harder than writing code. Once the code passed linter, compiler and test, the bugs might be more subtly logical and require more effort and intelligence.
We are all becoming QA of this super automated world.
Maybe the debunking AIs can match the bullshit generating AIs, and we will have balance in the force. Everyone is focused on the generative AIs, it seems.
No, they can't. They'll still be randomly deciding if something is fake or not, so they'll only have a probability of being correct, like all nondeterministic AI.
There was a degree of proof of work involved. Text took human effort to create, and this roughly constrained the quantity and quality of misinforming text to the number of humans with motive to expend sufficient effort to misinform. Now superficially indistinguishable text can be created by an investment in flops, which are fungible. This means that the constraint on the amount of misinforming text instead scales with whatever money is resourced to the task of generating misinforming text. If misinforming text can generate value for someone that can be translated back into money, the generation of misinforming text can be scaled to saturation and full extraction of that value.
How do you like questioning much more of it, much more frequently, from many more sources? And mistrusting it in new ways. AI and regular people are not wrong in the same ways, nor for the same reasons, and now you must track this too, increasingly.
It’s nothing to do with trusting in terms of being true or false, but whatever I read before I felt like, well, it can be good or bad, I can judge it, but whatever it is, somebody wrote it. It’s their work. Now when I read something I just have absolutely no idea whether the person wrote it, how much percent did they write it, or how much they even had to think before publishing it. Anyone can simply publish a perfectly well-written piece of text about any topic whatsoever, and I just can’t wrap my head around why, but it feels like a complete waste of time to read anything. Like… it’s all just garbage, I don’t know.
> you trusted what you read before. For me, LLMs don't change anything. I already questioned the information before and continue to do so. [...]
Why do you think that you could trust what you read before?
A human communicator is, in a sense, testifying when communicating. Humans have skin in the social game.
We try to educate people, we do want people to be well-informed and to think critically about what they read and hear. In the marketplace of information, we tend very strongly to trust non-delusional, non-hallucinating members of society. Human society is a social-confidence network.
In social media, where there is a cloak of anonymity (or obscurity), people may behave very badly. But they are usually full of excuses when the cloak is torn away; they are usually remarkably contrite before a judge.
A human communicator can face social, legal, and economic consequences for false testimony. Humans in a corporation, and the corporation itself, may be held accountable. They may allocate large sums of money to their defence, but reputation has value and their defence is not without social cost and monetary cost.
It is literally less effort at every scale to consult a trusted and trustworthy source of information.
It is literally more effort at every scale to feed oneself untrustworthy communication.
I read the original comment not as a lament of not being able to trust the content, rather, they are lamenting the fact that AI/LLM generated content has no more thought or effort put into it than a cheap microwave dinner purchased from Walmart. Yes, it fills the gut with calories but it lacks taste.
On second thought, perhaps AI/LLM generated content is better illustrated with it being like eating the regurgitated sludge called cud. Nothing new, but it fills the gut.
There were news reports that Russia spent less than a million dollars on a massive propaganda campaign targeting U.S elections and the American population in general.
Do you think it would be possible before internet, before AI?
Bad actors, poorly written/sourced information, sensationalism etc have always existed. It is nothing new. What is new is the scale, speed and cost of making and spreading poor quality stuff now.
All one needs today is a laptop and an internet connection and a few hours, they can wreak havoc. In the past, you'd need TV or newspapers to spread bad (and good) stuff - they were expensive, time consuming to produce and had limited reach.
Some woman’s cat was hiding in her basement. She automatically assumed her Haitian neighbors stole her cat and made some comment about it, which landed on Facebook, which got morphed into “immigrants eating pets” story, JD Vance picked it up, Trump mentioned it in a national debate watched by 65 million people. All of this happened in a few days. This resulted in violence in Springfield.
If you can place a rumor or lie in front of the right person/people to amplify, it will be amplified. It will spread like wildfire, and by the time it is fact checked, it will have done at least some damage.
These successful manipulation stories are extremely rare though. What usually happens is that you say your neighbour ate your cat, then everyone laughs at you.
Did the person who posted do the manipulation, or did JD Vance and Donald Trump do it?
It's that you trusted that what you read came from a human being. Back in the day I used to spend hours reading Evolution vs Creationism debates online. I didn't "trust" the veracity of half of what I read, but that didn't mean I didn't want to read it. I liked reading it because it came from people. I would never want to read AI regurgitation of these arguments.
> I already questioned the information before and continue to do so.
You might question new information, but you certainly do not actually verify it. So all you can hope to do is sense-checking - if something doesn't sound plausible, you assume it isn't true.
This depends on having two things: having trustworthy sources at all, and being able to relatively easily distinguish between junk info and real thorough research. AI is a very easy way for previously-trustworthy sources to sneak in utter disinformation without necessarily changing tone much. That makes it much easier for the info to sneak past your senses than previously.
If one spends a lot of years reading a lot of stuff, they come to this conclusion, that most of it cannot be trusted. But it takes lots of years and lots of material to see it.
> It's not so much that I think people have used AI, but that I know they have with a high degree of certainty, and this certainty is converging to 100%, simply because there is no way it will not. If you write regularly and you're not using AI, you simply cannot keep up with the competition.
I am writing regularly and I will never use AI. In fact I am working on a 400+ pages book right now and it does not contain a single character that I have not come up with and typed myself. Something like pride in craftmanship does exist.
Also currently working on a book (shameless plug: buy my book!) and feel no pull or need to involve AI. This book is my mine. My faults. My shortcomings. My overuse of commas. My wonky phrasing. It has to have those things, because I am those things (for better or worse!).
I'm right there with you. I write short and medium form articles for my personal site (link in bio, follow it or don't, the world keeps spinning either way). I will never use AI as part of this craft. If that hampers my output, or puts me at a disadvantage compared to the competition, or changes the opinion others have of me, I really don't care.
What I write is pretty niche anyway (compilers, LISP, buddhism, advaita), so I do not think AI will cause much trouble. Google ranking small websites into oblivion, though, I do notice that!
I use a spell checker to catch typos. Occasional quirky grammar is mine. Feedback will be provided by the audience. Why would I let a statistical model judge my work? This is how you kill originality.
i didn’t imply you’d need an LLM to “rate” your content. More so asking questions during and before the publishing step to help you improve your work. Not removing your identity from your work. Examples questions of what you could ask an LLM:
• are there any redundant sentences or points in this chapter?
• i’m trying to remember an expression used to convey X, can you remind me what it is?
• i’m familiar with X from my industry Y, but i’m trying to convey this to an audience from industry Z. Can you help brainstorm some words or examples they may be more familiar with?
Things like that. I think of it like having a virtual rubber duck that can help you debug code, or anything really.
Obviously these are just some suggestions. If you don’t find any of this useful, or even interesting, then carry on. :)
I see. Most of the things you list I think I could do better on my own. The case of X from industry Y sounds interesting, but I would still prefer to hear from a real human being from industry Z. If no one is available, of course, a statistical model may indeed be helpful -- I still do not think it is worth boiling the oceans, though. :)
The most depressing thing for me is the feeling that I simply cannot trust anything that has been written in the past 2 years or so and up until the day that I die.
What AI is going to teach people is that they don't actually need to trust half as many things as they thought they did, but that they do need to verify what's left.
This has always been the case. We've just been deferring to 'truster organizations' a lot recently, without actually looking to see if they still warrant having our trust when they change over time.
Independent verification is always good however not always possible and practical. At complex levels of life we have to just trust underlying processes work, usually until something fails.
I don’t go double checking civil engineers work (nor could I) for every bridge I drive over. I don’t check inspection records to make sure it was recent and proper actions were taken. I trust that enough people involved know what they’re doing with good enough intent that I can take my 20 second trip over it in my car without batting an eye.
If I had to verify everything, I’m not sure how I’d get across many bridges on a daily basis. Or use any major infrastructure in general where my life might be at risk. And those are cases where it’s very important to be done right, if it’s some accounting form or generated video on the internet… I have even less time to be concerned from a practical standpoint. Having the skills to do it should I want or need to are good and everyone should have these but we’re at a point in society we really have to outsource trust in a lot of cases.
This is true everywhere, even in science which these days many people just trust in ways akin to faith in some cases, and I don’t see anyway around that. The key being that all the information should exist to be able to independently verify something but from a practice standpoint it’s rarely viable.
Get good at spotting inconsistencies. And pay attention to when something contradicts your own experience. Cultivate a wide range of experiences so that you have more domains where you can do this (this is a good idea anyway).
I was listening to an interview few months ago (forgot the name). He is a prolific reader/writer and has a huge following. He mentioned that he only reads books that are at least 50 years old, so pre 70s. That sounds like a good idea now.
Even ignoring the AI, if you look at the movies and books that come out these days, their quality is significantly lower than 30-40 years ago (on an average). Maybe people's attention spans and taste is to blame, or maybe people just don't have the money/time/patience to consume quality work... I do not know.
One thing I know for sure - there is enough high quality material written before AI, before article spinners, before MFA sites etc. We would need multiple lifetimes to even scratch the surface of that body of work. We can ignore mostly everything that is published these days and we won't be missing much
Nassim Taleb famously argues that position, in his popular work Antifragile and elsewhere. I believe the theory is that time serves as a sieve: only works with lasting value can remain relevant through the years.
Completely disagree just from my own personal experience as a sci-fi reader. Modern day bestseller sci-fi novels fit right in with the old classics, and in many ways outshine them.
I have read many bad obscure sci-fi books published from the 50's to today, most of them a dollar at the thrift store. There was never a time when writers were perfect and every published work was high quality, then or now.
I only listen to interviews from 50 years ago (interviews that have stood the test of time), about books from 100 years ago. In fact, how am I reading this article? It's not 2074 yet?!
Life is short and I like creating things. AI is not part of how I write, or code, or make pixel art, or compose. It's very important to me that whatever I make represents some sort of creative impulse or want, and is reflective of me as a person and my life and experiences to that point.
If other people want to hit enter, watch as reams of text are generated, and then slap their name on it, I can't stop them. But deep inside they know their creative lives are shallow and I'll never know the same.
That’s super cool, and I hope you are right and that I am wrong and artists/creators like you will still have a place in the future. My fear is that your work turns into some kind of artesanal fringe activity that is only accessible to 1% of people, like Ming vases or whatever.
That's true art, I love people like you. Technology can do a lot of things but it cannot give people or society principles, and without principles society fails.
I've been using it in my personal writing (combination of GPT and Claude). I ask the AI to write something, maybe several times, and I edit it until I'm happy with it. I've always known I'm a better editor than I am an author, and the AI text gives me somewhere to start.
So there's a human in the loop who is prepared to vouch for those sentences. They're not 100% human-written, but they are 100% human-approved. I haven't just connected my blog to a Markov chain firehose and walked away.
Am I still adding to the AI smog? idk. I imagine that, at a bare minimum, its way of organising text bleeds through no matter how much editing I do.
you wrote this comment completely by your own, right? without any AI involved. And I read your comment feeling confident that it's truly 100% yours. I think this reader's confidence is what the OP is talking about.
I did. I write for myself mostly so I'm not so worried about one reader's trust - I guess I'm more worried that I might be contributing to the dead internet theory by generating AI-polluted text for the next generation of AIs to train on.
At the moment I'm using it for local history research. I feed it all the text I can find on an event (mostly newspaper articles and other primary sources, occasionally quotes from secondary sources) and I prompt with something like "Summarize this document in a concise and direct style. Focus on the main points and key details. Maintain a neutral, objective voice." Then I hack at it until I'm happy (mostly I cut stuff). Analysis, I do the other way around: I write the first draft, then ask the AI to polish. Then I go back and forth a few times until I'm happy with that paragraph.
I'm not going anywhere with this really, I'm just musing out loud. Am I contributing to a tragedy of the commons by writing about 18th century enclosures? Because that would be ironic.
If you write for yourself, whether you use generated text or not, (I am using the text completion on my phone typing this message), the only thing that matters is how it affects you.
Reading and writing are mental processes (with or without advanced technology) that shape our collective mind.
When you're writing, how are you "missing out" if you're not using chatgpt??? I don't even understand how this can be unless what you're writing is already unnecessary such that you shouldn't need to write it in the first place.
I don’t get it either. Writing is not something I need that level of assistance with, and I would even say that using LLMs to write defeats some significant portion of the point of writing — by using LLMs to write for me I feel that I’m no longer expressing myself in the purest sense, because the words are not mine and do not exhibit any of my personality, tendencies, etc. Even if I were to train an LLM on my style, it’d only be a temporal facsimile of middling quality, because peoples’ styles evolve (sometimes quite rapidly) and there’s no way to work around all the corner cases that never got trained for.
As you say, if the subject is worth being written about, there should be no issue and writing will come naturally. If it’s a struggle, maybe one should step back and figure out why that is.
There may some argument for speed, because writing quality prose does take time, but then the question becomes a matter of quantity vs. quality. Do you want to write high quality pieces that people want to read at a slower pace or churn out endless volumes of low-substance grey goo “content”?
LLMs are surprisingly capable editors/brainstorming tools. So, you're missing out in that you're being less efficient in editing.
Like, you can write a bunch of text, then ask an LLM to improve it with minimal changes. Then, you read through its output and pick out the improvements you like.
But that's the problem. Unique, quirky mannerisms become polished out. Flaws are smoothed and over sharpened.
I'm personally not as gloomy about it as the parent comments but I fear it's a trend that pushes towards a samey, mass-produced style in all writing.
Eventually there will be a counter culture and backlash to it and then equilibrium in quality content but it's probably here to stay for anything where cost is a major factor.
Yeah, I suppose that would be an issue for creative writing. My focus is mostly on scientific writing, where such mannerisms should be less relevant than precision, so I didn't consider that aspect of other kinds of writing.
And I the only one who doesn't even like automatic grammar checkers, because they are contributing to a single and uniformly bland style of writing? LLMs are just going to make this worse.
That's a fair point, I only very recently found that LLMs could actually be useful for editing, and hadn't really thought much of using tools for that kind of thing previously.
>> cannot trust anything that has been written in the past 2 years or so and up until the day that I die.
You never should have. Large amounts of work, even stuff by major authors, is ghostwritten. I was talking to someone about Taylor Swift recently. They thought that she wrote all her songs. I commented that one cannot really know that, that the entertainment industry is very going at generating seemingly "authentic" product at a rapid pace. My colleague looked at me like I had just killed a small animal. The idea that TS was "genuine" was a cornerstone of their fandom, and my suggestion had attacked that love. If you love music or film, don't dig too deep. It is all a factory. That AI is now part of that factory doesn't change much for me.
Maybe my opinion would change if I saw something AI-generated with even a hint of artistic relevance. I've seen cool pictures and passable prose, but nothing so far with actual meaning, nothing worthy of my time.
Watch the movie "The Wrecking Crew" about how a group of studio musicians in the 1970s were responsible for the albums of quite a few diverse "bands". Many bands had to then learn to play their own songs so they could go on tour.
While I do enjoy some popular genres, I'm all too aware of the massive industry behind it all. I believe that most of humanity's greatest works of art were created not for commercial interests but rather for the pure joy of creation, of human expression. This can be found in any genre if you look hard enough, but it's no accident that the music I find the most rewarding is classical music: Intellect, emotion, spirit, and narrative dreamed into existence by one person and then brought to life by other artists so we can share in its beauty.
I think music brings about a connection between the composers, lyricists, performers, and listeners. Music lets us participate in something uniquely human. Replacing any of the human participants with AI greatly diminishes or eliminates its value in my eyes.
>If you write regularly and you're not using AI, you simply cannot keep up with the competition. You're out.
A very HN-centric view of the world. From my perch in journalism and publishing, elite writers absolutely loathe AI and almost uniformly agree it sucks. So to my mind the most 'competitive' spheres in writing do not use AI at all.
It doesn't matter how elite you think you are if the newspaper, magazine, or publishing company you write for can make more money from hiring people at a fraction of your cost and having them use AI to match or eclipse your professional output.
At some point the competition will be less about "does this look like the most skilled human writer wrote this?" and more about "did the AI guided by a human for a fraction of the cost of a skilled human writer output something acceptably good for people to read it between giant ads on our website / watch the TTS video on YouTube and sit through the ads and sponsors?", and I'm sorry to say, skilled human writers are at a distinct disadvantage here because they have professional standards and self respect.
So is the argument here that the New Yorker can make more money from AI slop writing overseen by low-wage overseas workers? Isn't that obviously not the case?
Anyway I think I've misunderstood the context in which we're using the word 'competition' here. My response was about attitudes toward AI from writers at the tip-top of the industry rather than profit maxxing/high-volume content farm type places.
It’s not that black and white. Maybe 1% of the top writers can take that stance and maybe even charge more for their all-human content (in a kind of vintage, handcraft kind of way) but the other 99% will have to adapt.
It’s simply more nuanced. If you’re writing a couple of articles a day to pay for your bills, what will stop you from writing actually 10 or 20 articles a day instead?
So you're saying major media companies are going to outsource their writing to people overseas using LLMs? There is more to journalism than the writing. There's also the investigative part where journalists go and talk to people, look into old records, etc.
This has become such a talking point of mine when I'm inevitably forced to explain why LLMs can't come for my job (yet). People seem baffled by the idea that reporting collects novel information about the world which hasn't been indexed/ingested at any point because it didn't exist before I did the interview or whatever it is.
They definitely try to replace part of the people this way, starting with the areas where it's the easiest, but obviously it will continue to other people as the capabilities improve. A big example is sports journalism, where lots of venues have game summaries that do not involve any human who actually saw the game, but rather software embellishing some narrative from the detailed referee scoring data. Another example is autotranslation of foreign news or rewriting press releases or summarizing company financial 'news' - most publishers will eagerly skip the labor intensive and thus expensive part where journalists go and talk to people, look into old records, etc, if they can get away with that.
i regularly (at least once a week) spot a typo or grammatical issue in a major news story. I see it in the NYTimes on occasion. I see it in local news ALL THE TIME. I swear an LLM would write better than have the idiots that are cranking out articles.
I agree with you that having elite writing skills will be useful for a long time. But the bar for proof reading seems to be quite low on average in the industry. I think you overestimate the writings skills of your average journalist.
Heh, when I see a spelling error in a news article.. I oddly feel like I can trust it more because it came from a human being. It's like a nugget of gold.
You can prompt/train the AI to add a couple of random minor errors. They're trained from human text after all, they can pretend to be as human as you like.
Barring simple typos, human mistakes are erroneous intention from a single source. You can't simply write human vagaries off as 'error' because they're glimpses into a picture of intention that is perhaps misguided.
I'm listening to a slightly wonky early James Brown instrumental right now, and there's certainly a lot more error than you'd get in sequenced computer music (or indeed generated music) but the force with which humans wrest the wonkiness toward an idea of groove is palpable. Same with Zeppelin's 'Communication Breakdown' (I'm doing a groove analysis project, ok?).
I can't program the AI to have intention, nor can you. If you do, hello Skynet, and it's time you started thinking about how to be nice to it, or else :)
There is. Be vehemently against AI, put 100% AI free in your work. The more consistent you are against AI, the more likely people will believe you. Write articles slamming AI. Personally, I am 100% against AI and I state that loud and clear on my blogs and YouTube channel. I HATE AI.
AI cannot build up a sufficient level of trust, especially if you are known in person by others who will vouch for you. That web of trust is hard to break with AI. And I am one of those.
The funny thing is that the things it refuses to say are "wrong-speech" type stuff, so the only things you can be more sure of nowadays are conspiracy theories and other nasty stuff. The nastier the more likely it's human written, which is a bit ironic.
A few months ago, I tried to get Gemini to help me write some criticism of something. I can't even remember what it was, but I wanted to clearly say something was wrong and bad.
Gemini just could not do it. It kept trying to avoid being explicitly negative. It wanted me to instead focus on the positive. I think it evidently just told me no, and that it would not do it.
Yeah all the current tools have this particular brand of corporate speech that’s pretty easy to pick up on. Overly verbose, overly polite, very vague, non assertive, and non opinionated.
People could prompt for authenticity, adding subtle mistakes, etc. I hope that AI as a whole will help people writing better, if reading back the text. It is a bit like "The Substance" movie: a "better" version of ourselves.
Way back when we had a landline and would get telemarketers, it was always a sign when the caller couldn’t pronounce our last name. It’s not even that uncommon a name, either
> Write a response to this comment, make spelling and grammar mistakes.
yeah well sumtimes spellling and grammer erors just make thing hard two read. like i no wat u mean bout wanting two kno its a reel person, but i think cleear communication is still importint! ;)
>If you write regularly and you're not using AI, you simply cannot keep up with the competition. You're out. And the growing consensus is "why shouldn't you?", there is no escape from that.
Are you sure you don't mean if you write regularly in one particular subclass of writing - like technical writing, documentation etc.?
Do you think novel writing, poetry, film reviews etc. cannot keep up in the same way?
I think that novel writing and reviews are types of writing where potentially AI should eventually surpass human writers, because they have the potential to replace content skillfully tailored to be liked by many people with content that's tailored (perhaps less skillfully) explicitly for a specific very, very, very narrow niche of exactly you and all the things that happen to work for your particular biases.
There seems to be an upcoming wave of adult content products (once again, being on the bleeding edge users of new abilities) based on this principle, as hitting very specific niches/kinks/fetishes can be quite effective in that business, but it should then move on to romance novels and pulp fiction and then, over time, most other genres.
Similarly, good pedagogy, curriculum design and educational content development is all about accurately modeling which exact bits of the content the target audience will/won't know, and explaining the gaps with analogies and context that will work for them (for example, when adapting a textbook for a different country, translation is not sufficient; you'd also need to adapt the content). In that regard, if AI models can make personalized technical writing, then that can be more effective than the best technical writing the most skilled person can make addressed to a broader audience.
I'm absolutely positive that the vast majority of fiction is or will soon be written by LLM. Will it be high-quality? Will it be loved and remembered by generations to come? Probably not. Will it make money? Probably more than before on average as the author's effort is reduced to writing outlines and prompts, and editing the generated-in-seconds output, rather than months-years of doing the writing themselves.
> If you write regularly and you're not using AI, you simply cannot keep up with the competition.
Is that true today? I guess it depends what kind of writing you are talking about, but I wouldn't think most successful writers today - from novelests to tech bloggers - rely that much on AI, but I don't know. Five years from now, could be a different story.
It's not true at all. Much like the claims that you have to use LLMs to keep up in programming: if that is true then you weren't a good programmer (or writer in this case) to begin with.
That is absolutely wrong. Regardless of whether you were or not good to begin with, an LLM assistant will still accelerate a lot of repetitive tasks for you. Repetitive is repetitive, no matter if you’re John Carmack or the guy sitting in the booth next to you in the paper company. And anyway in a few years none of it will matter because programming without assistance will be a vintage thing of the past (like perforated cards).
It’s the same with writing. If you find yourself writing a boring introduction section to a paper with a bunch of meaningless blabla, then why wouldn’t you use AI for that? There is simply no good reason, especially when you see mediocre researchers publishing at three times your rate and getting promoted over you.
Yes it’s true today, depending on what is your writing is the foundation of.
It doesn’t matter that my writing is more considered, more accurate and of a higher quality when my coworkers are all openly using AI to perform five times the work I am and producing outcomes that are “good enough” because good enough is quite enough for a larger majority than many likely realise.
> Now, I'm not going to criticize anyone that does it, like I said, you have to, that's it.
Why do you say people have to do it?
People absolutely can choose not to use LLMs and to instead write their own words and thoughts, just like developers can simply refuse to build LLM tools, whether its because they have safety concerns or because they simply see "AI" in its current state as a doomed marketing play that is not worth wasting time and resources on. There will always be side effects to making those decisions, but its well within everyone's right to make them.
If you find yourself in a situation where you write one book at the same time as your peers are writing ten, how can you keep up? Also how can you justify to yourself not using it if nobody around you seems to value that, and even worse, is pushing you to actually use it? I find it hard to find a reason why you would. Unless we see a super strong reader revolution that collectively decides to shun AI and pay more money for all-human books.
Read that last sentence and tell you think that’s reasonable and likely to happen?
What you're describing here is actually a much more broad problem in the book industry, in my opinion. Almost every book written today is written with only one goal in mind, selling as many copies as possible.
People don't have to use LLMs (they don't seem to be AI yet) because they can simply choose not to. For authors, write books that you want to write because you believe you have a story to tell. Worry about perfecting your stories and enjoy the process of writing, don't be an author just for the sales. Once you peek behind the curtain and learn the economics of the book industry, you'll realize there's very little opportunity for making enough cash to even worry about shotgunning a mountain of LLM books into the world anyway.
Enough billions of dollars have been spent on LLMs that a reasonably good picture of what they can and can't do has emerged. They're really good at some things, terrible at others, and prone to doing something totally wrong some fraction of the time. That last limits their usefulness. They can't safely be in charge of anything important.
If someone doesn't soon figure out how to get a confidence metric out of an LLM, we're headed for another "AI Winter". Although at a much higher level than last time. It will still be a billion dollar industry, but not a trillion dollar one.
At some point, the market for LLM-generated blithering should be saturated. Somebody has to read the stuff. Although you can task another system to summarize and rank it. How much of "AI" is generating content to be read by Google's search engine? This may be a bigger energy drain than Bitcoin mining.
reply