Yeah, I think it's at the level of self-driving Excel.
Now, Excel also makes mistakes while trying to get clever - remember the biologists who had to rename several genes to prevent Excel treating them as dates - but it's also a ubiquitous business tool. And you can combine them. But equally Excel hasn't replaced every white collar job.
In 90s the show Friends, Chandler Bing’s job is “statistical analysis and data reconfiguration” which is likely unfamiliar to you because it is gone!
This was a job to manually collate financial reports into aggregated sums to sent up the hierarchy to the next level up.
This is now a trivial BI query and doesn’t require manual processing by humans.
A bigger shift is that now I only see CEOs or at most CxOs with secretaries. Mobile phones and Outlook calendars have largely wiped out that profession.
Is it? I don’t really hire for analysts anymore because you can mostly just expose chat with your data mcp tools that end users can use and that gets you 80% of the way there. The person who puts together dashboards and writes sql against the star schema is pretty much a thing of the past.
Data engineers, on the other hand… way more relevant than ever before
The issue is that data complexity is also exploding in the age of LLMs, so it still takes a lot of archaeology to get reliable (as opposed to plausible) numbers.
I think this dynamic also requires more data engineers, but I don't personally see the split in demand between those roles that you describe. Data analysts often have business or domain context that data engineers lack.
Perhaps we'll start rolling data analysis work into "operations".
It's the difference between Architects and Draftsmen. We still need the former, but the latter is gone, replaced by Solidworks, AutoCAD, and digital plotters or large-format printers.
> A bigger shift is that now I only see CEOs or at most CxOs with secretaries. Mobile phones and Outlook calendars have largely wiped out that profession.
Depends heavily on the field. Most fields don't use the term "secretary", but e.g. my mid-sized company of several hundred employees has a single executive assistant, vs. my wife works at a law firm where it's pretty close to 1:1 professionals to assistants.
> This is now a trivial BI query and doesn’t require manual processing by humans.
It also doesn't require AI. Management wants to see how the same measurements change over time. They don't necessarily want the latest AI model's hot take on this month's data. Or maybe they do, but I doubt it's very helpful.
Replicating the brain functionality is magic? Or the brain itself is magic, so we need magic to replicate its functionality? Nah, learn more.
An LLM with CoT is Turing-complete, so there's no wall to its capabilities. There's no theorems that establish limits of reinforcement learning either. There's no evidence of quantum computations in the brain.
C++ is Turing complete. Hell, HTML+CSS is Turing complete.
> so there's no wall to its capabilities.
There are multiple well understood and well explained walls and limits to LLM capabilities. And none of those walls, or perceived lack of them, stem from something being Turing complete.
> There's no theorems that establish limits of reinforcement learning either.
There's a very obvious and immediate one: availability and size of training data. And compute. And time.
> There's no evidence of quantum computations in the brain.
What does quantum computation have to do with brains or LLMs?
> All you have is a strong belief.
Says the person who just listed a lot of beliefs in marketing bullshit.
Oh! Do you know how does the brain function? Heck, we don't know in detail how LLMs do their thing. The trivial observation that the brain in not a Turing machine is irrelevant if we are talking about functionality and not computational substrate.
But a Turing machine that implements functionality of a target system is the information-theoretical optimal compression of the data produced by the target system. So it stands to reason that LLMs that need to compress vast amounts of data approximate such a machine.
> C++ is Turing complete. Hell, HTML+CSS is Turing complete.
This is absolutely irrelevant to the question of the limits of LLM training. It's a common surface-level patter matching response, though.
> There's a very obvious and immediate one: availability and size of training data. And compute. And time.
Yeah, practical limitations. And practical results like seven hundred solved math problems show that you need, at least, to stop and think.
> What does quantum computation have to do with brains or LLMs?
If you have to ask that, it means that you haven't dug into all this mess deep enough. Basically, BQP computational complexity class is more powerful than P-class. If the brain employs such computations (although we don't know the appropriate quantum algorithms, so it's just wishful thinking mostly), the corresponding Turing machine is impractically large and slow.
> Says the person who just listed a lot of beliefs in marketing bullshit.
It's like Software, its value is to be a complement and/or an abstraction of existing businesses. All of that sits on the very basics of the essence of civilization, which is how much things we can get off the soil and what how quickly and effective we can transform it.
Not to be anti-tech, but every new advancement, no matter how next-level, has to sell more of the same goods and services people have been using since the dawn of civilization, and those goods and services need to solve the mundane problems people have been getting since millenia.
Software industry, because the construction industry still requires a human to sign off and approve the computer-aided design.
There are companies which are involved with building and marketing their own software for the construction industry, but they are also actively involved in the construction process in a consultative capacity. Only then could they even be classified as construction industry companies.
BI is a good example of something where the benefits take a long time to show up. It has the most use at a senior management level where they can't use their every day experience to tell what is going on. But they don't make a lot of decisions over the course of a year and it often takes a few years before the impact is felt.
I worry about this. Dumping data and getting a report on which actions are based prolly isn’t giving real data.
I say that because I have Claude review code and then review the review and it will point out all the things it got wrong the first time. And I end up having to review it myself. Don’t get me wrong, I’m burning tokens all day and love the legit productivity gains but I don’t see how this replaces everybody in tech for a while.
>He loves being able to just dump in a bucket of seemingly unrelated data, and getting actionable BI from it.
Sorry, just don't trust accurate LLM data analysis even from the latest claude model. Have had some embarrassing bloopers over the last year. It is just better to code a deterministic tool to do the data analysis.
Even for the code. It will sometimes shit a very complex code for something with a very easy solution, and if you ask why it is so complex, it will invent some reason why. Always on 3 points, the first is true but have nothing to do with the subject, the second is plainly false, and the third is weird bullshit.
Don't get me wrong, I still use it, but the more you do, the more I see the obvious limitations.
I hope he validates the output, the number of times colleagues have done exactly that and gotten the wrong output is.. well, not an insignificant amount.
Not saying it can't be done, but my recommendation is still "ask your LLM to make a tool that process the data, do not process it via the LLM itself".
I don't doubt it. Earlier today I got an estimate for the revenue, profit margin, and headcount for a potential supplier of a hardware component to a product I'm working on. It would've taken a couple hours of really tedious work for me to do that with Google translate, a search engine, and various market research sites. The result is I know more what I can expect from that vendor in terms of customization and support.
But those are the kinds of estimates that I can eyeball and be more than 80% certain they are plausible. That won't cut it in a lot of domains.
Too true. Even on the software side i see people blindly “claude make it faster with no mistakes” and gushing about it when the output is genuinely not good not because claude isnt good but because the user of it isnt.
We got a new a CNC machine and everyone is carving away marveling at how easily it chips away at metal when the result is a jagged mess.
Yeah but theres also no null test here. He might have done anything and still been successful. He might have done something else more optimal and done even better. We can't really tell from a single anecdote.
It's really quite interesting how many HNers hate AI; which is odd, considering the zeitgeist of this community. It may even be more intense than the Apple-hatred.
Hate marketing BS and wealth extraction? Absolutely.
I see OpenAI, Anthropic, etc (the whole LLM-hype/boom) leaning far more towards being the latter than the former, in which context the dislike makes perfect sense.
I don’t hate it. I just hate the cult-like enthusiasm for it and the associated false claims. Again, your isolated anecdote does not trump the carefully compiled data presented in this post.
We get way too hung up on data; especially when it reinforces our personal bias.
Some time ago, I had a discussion here, where I mentioned that the average commute time from the suburban community in which I live, to downtown Manhattan, was generally 90 minutes to two and a half hours.
This was based on purely anecdotal data. That data includes the daily news on the radio and TV, which considers commutes from suburbs, important enough to dedicate significant airtime.
It was attacked, because some organization published some statistics, that said the average suburban commute time was 27 minutes. I was told that I was exaggerating, and being overly negative.
Here's a math word problem:
Harry lives 25 miles, as the crow flies, from his destination. However, the roads make it 40 miles.
Of those roads, 10 miles are suburban side roads that Harry needs to take, to get to the highway. The average speed is 25 MpH.
Once Harry hits the highway, though, the real fun begins. It's stop-and-go traffic (20MpH) for 28 miles. Once he gets to the city, he needs to navigate another 2 miles, through the most God-awful traffic on Earth, to where he needs to drive around for up to a half hour, looking for a parking space.
How does Harry make the drive in 27 minutes?
A fairly significant number of folks just took that ridiculous statistic, and were quite comfortable, calling me a liar, based on it. It was kind of jaw-dropping; considering the types of folks that we have, here.
That sounds a bit more "cultish" than we might be comfortable acknowledging.
> We get way too hung up on data; especially when it reinforces our personal bias.
While I don't disagree that we generally get too hung up on data, "It's a problem when the data matches the hypothesis" is an odd blanket interpretation of the scientific method.
I'm confused, your rebuttal to statistics is another anecdote? Was the quoted statistics limited to car commute or includes public transport? Your commute may be long, but it may not represent the average as well. Maybe it's a sign to switch away from cars instead?
Anecdotes are poor because it is highly context dependent. You had to make a long description of Harry to make your point, and yet it has not, to my mind, disproved the statistics, though I would like the source to the stats myself if you could provide it, thanks.
The commute only takes 27 minutes if we ignore that everybody else has an LLM- erh, car I mean.
Think you’re clever because your LLM contacts 100 businesses? Imagine 100000 businesses doing the same thing. You’ve just eradicated human support, because nobody’s going to sit through this BS bot future. Good luck when you get auto-flagged for some reason and there’s not a single human to revert or care.
No, my point is LLMs are machines designed to create responses to prompts that pass as plausibly human and intelligent. They are working as designed here. The fact that they get stuff right on occasion doesn't change that what it is at it's core is a randomness generator.
Kidding of course. It could be that he's an expert in his domain and can quickly ascertain whether the results are quality. It may also save a lot of time and be the best modern heuristic process.
I certainly will never need to write code anymore, and I can judge the output of these agentic processes. Same thing.
If companies can better understand their customers so they can give me products that better fit my needs, or inform me of products I couldn't have known about otherwise, that's a win for everyone.
I’ve always had doubts about the validity of this “actionable BI,” even before it was slop, serving, mostly to serve leadership’s biases and justify their preferred ideas.
If accuracy was never that valuable to begin with then this should not be a problem.
This morning, I was talking to a friend of mine, who is a marketing (not tech) guy, and he was gushing about LLMs.
He loves being able to just dump in a bucket of seemingly unrelated data, and getting actionable BI from it.
I think we're in for a wild ride. Fasten your seat belt, and keep your hands inside the vehicle...