I agree with the folks at GS. Every non-tech corporation in the world is trying to figure out some kind of "AI strategy," but most executives and managers have no clue as to what they're doing or ought to be doing with respect to "AI." They're like patsies joining a very expensive game of poker they don't understand, and they are driven by FOMO, ready and eager to put down gobs of cash on the table, because they don't want to miss their chance of winning. They are on track to learn predictable lessons that cannot be learned in any other way.
I think this is even the case for tech companies. So many products I see now that offer AI where I go "is that really necessary?". Even in the case of my own startup, my business partner has been pushing me to figure out some way to integrate AI into our product and my response to him every time has been to convince him that it's not in our best interest to go down that road. I told him that even in a world where we could adjust our subscription prices to compensate for the added expense of AI, we would be burning valuable time on AI when we could be spending it in other areas of our product.
Even though we are a tiny startup I have a feeling that this applies to many other tech companies as well.
It sounds like your partner is not pushing for it because it will improve the product for users, but because it will improve the product for investors. It’s a branding exercise. So, if you’re doing anything at all that involves statistics, or multiplying matrices, using some kind of backtracking algorithm, maybe just call that AI and everyone wins. Or just slap a “genAI” button tied to a ChatGPT API next to your user input.
It's mostly the "staying relevant" subset of FOMO that is pushing such widespread adoption. Like FOMO, "staying relevant" is poorly defined and emotionally driven, making it effectively meaningless. Probably the smartest response is to slap something flashy on that does little, and took few resources to add. The more resources you apply to staying relevant generally the worse the decision is.
The reality is, the economy is on a ticking timer for when it will be turned upside down by 1) LLM agent workflows 2) AGI 3) superintelligence. There are many companies who know this and want to hedge against that future but they simply don’t know how yet. The tricky part about productizing LLMs is that your product designers need to have a full understanding of LLMs and what they can and cannot do. So currently many of the existing product designs and AI user experiences still suck.
The biggest misconception is that just because chatbots are now more capable that somehow that means that you should replace your UI with one.
For example, AI leveraged SOC is going to be operational within 3-5 years, and there goes 50-70% of jobs in that sector.
Others segments are significantly behind (robo-taxis) and I'm sure there are other segments much further ahead (call center automation), but the change is absolutely coming - not because of LLMs alone, but because the entire ecosystem around ML/AI is a 15 years old now, and there's an entire generation of SWEs that are just as competent as ML PhDs were 15 years ago.
I don't think superintelligent AGI will occur in our life span.
That said, most investment in ML/AI is around applications of ML in automating specific domains.
My argument is against this:
> FOMO investment completely pointless
I have an example of a 10-11 figure TAM industry that is actively in the process of migrating to being majority automated, and there are plenty of other industries in the midst of this as well.
The part of my post you're replying to concerns my thoughts on something much broader, though. My claim is not that all investment today is pointless, but that investment predicated on the belief that we're a few years away from the economy being "turned upside down by 1) LLM agent workflows 2) AGI 3) superintelligence" is bad.
It feels like a Motte-and-bailey type issue. If I point out that the original claim doesn't have enough evidence and is purely speculation, I don't really want to hear about how call center jobs are being affected by LLMs. I don't disagree, but we're not talking about the same thing here.
> investment predicated on the belief that we're a few years away from the economy being "turned upside down by 1) LLM agent workflows 2) AGI 3) superintelligence" is bad
No one is investing on that assumption. I've been in the PE/VC for a couple years now and I've never heard anyone from Associate to LP say that straight faced and unironically.
I am pretty sure some are. Some very rich people are talking as if AGI will happen soon they for sure are investing into this assuming that billions will turn to trillions, as long as they invest in every AI company they will get it right for one of them.
This happened in the IT bubble as well, most companies were overvalued but the total set of IT companies went on to become many trillion dollars of value, so if you invested in every IT company then you became really rich, they expect the same thing to happen now but even more extreme.
Now I don't expect this to turn to AGI, but many do, and those who do would be stupid if they didn't take that into account for investments.
> I've been in the PE/VC
Are you talking about the calculated VC groups? Or are you talking about eccentric rich individuals? The first group for sure wouldn't it would be career suicide to do it there but some of the second group would since they invest their own money. Think people like Elon Musk etc.
I’ll… well, I’ll believe the magic robots replacing security people when I see it. That seems a _particularly_ implausible one, really.
As you imply yourself, robotaxis did not deliver on the grandiose promises of a decade ago. What makes you so sure that the generative stuff will be any different?
I think you could go back in time and s/AI/internet.
But if we look at the 90’s, the lesson is not that companies wasted time on internet strategies they didn’t need. The lesson is that companies that had good internet strategies thrived (Netflix) and those that had bad internet strategies died (Blockbuster).
Agree there’s a lot of FOMO, but unlike your patsies, these companies would not be better served by staying away and ignoring the temptation.
It's still not clear to me that LLMs are actually useful, but I'm perfectly willing to accept that I'm some sort of neo-luddite (how deliciously ironic a term by the way). I wonder, do you think there are examples of companies that have good strategies now? And if so, what utility are they getting out of it?
I'm in finance and we're using LLMs to summarize a ton of soft data on company performance. We're making a killing and beating the market substantially. Within the next five years I don't think there will be anyone left standing who isn't worth a trillion dollars or doesn't have all their quants augmented by LLMs.
Before this I was building a matching engine for online retail which used (then) state of the art multimodal models to decide which products would match your query to the lowest priced item. The dirty secret here is that we've solved search, it just costs too much to provide it for free. I think we'll soon see an online Costco show up which can provide this service to members and filter all the bullshit dropshippers.
I see that as a continuation of a long arc of using machines to read things in finance (not just on the buy side, btw.), don't think it will change the competitive dynamics between the more dominant players. (There might be some more concentration in some areas, but the dynamics for that pre-date LLMs or even AI/ML in some cases.)
The question is if their holdings will melt faster than they can develop the capability to use the new AI better than the small fries going after them.
So far this year the main thing slowing down development for us has been the SEC seizing our development boxes for insider trading once a month or so. Which given that each cost $50,000 has been something of a sore point.
I honestly don’t know. AI is so young; it’s like asking who had good internet strategies in 1994; in hindsight we know some did, but nobody really knew at the time (I would argue those who say otherwise are speaking from survivorship bias).
But take a look at GitHub’s Copilot Workspaces for ideas of how transformative AI will be.
I don’t have all the pieces (if I did, I’d be rich, right?), but I think AI will deliver mass customization of goods, software, and content by reducing up front costs to near zero. So if I were looking for good strategies, I’d start with companies that currently spend a lot of money on software or content creation and who are investing in automating that.
From own experience with ChatGPT, I asked it some months ago to go through an EU regulation, and create a checklist/audit programme for IT Security.
Work that would have taken me a couple of weeks, it took 'it' a few seconds.
I had to go through the text to make sure it's clean and doesn't contain hallucinations. Total work effort 1-2 hours max.
This doesn't mean that the human is no longer required, but for certain tasks a company can save a lot of time/money.
So I don't know if they dream to drop a dept of 50 people and replace it with a single machine, but there are some tasks that cannot be outsourced to machines (e.g. decisions impacting the lives cannot be made by automation)
"the existence of automated decision-making, including profiling, referred to in Article 22(1) and (4) and, at least in those cases, meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject."
So good luck 'convincing' the regulator that an LLM can replace mortgage-decisions makers (it is considered a significant activity in a person's life).
If the regulations are so extensive that you would have needed a couple of weeks to create a checklist, how did you manage to see if the list was complete and correct within a couple of hours?
This was my first thought? The amount of effort I have to go through verifying what I get from gpt makes me wonder. On the other hand, there I’ve been thinking about how it might help me read datasheets. There are details that are easy to miss when skimming, that may be easy to verify through simple text searches.
That said, I know someone who almost took a job writing the actual text of laws, and they were excited about it because, while they didn’t control the laws in general, comma placement allowed them to subtly change the law’s meaning.
I currently would‘t trust an LLM to understand subtle comma placement.
You mean automating bureaucracy? That would make room for more bureaucracy albeit in the form AI bureaucracy. I don’t think we’re ready for that though.
No, I really mean things that shouldn't exist at all. I see it used a lot by marketing people generating press releases nobody really cares about, online media generating images for short useless pieces meant to be filled a space between few really useful articles etc.
> generating press releases nobody really cares about, online media generating images for short useless pieces meant to be filled a space between few really useful articles etc.
I can't see any evidences that those aren't really important or useful given your comment, do you have any data to support your argument?
I've seen that claim made but I've not seen it in reality.
There are examples of where LLMs are useful for information retrieval in that they can take vague queries and produce good summaries, but they're just as likely to produce reasonable-sounding nonsense that needs to be carefully verified.
I've also seen examples where LLMs can produce code that implements well-defined toy problems but when used on a real software system with millions of lines of context they do not produce code that can remain coherent with the system.
From what I've seen LLMs have yet to graduate from "impressive demo" and everyone characterizing them as being a revolution are still describing some potential future state with no real evidence that we'll get there with this technology or any time soon.
We live in different realities then. I can't talk about details for my previous employer, but I can say there is an implementation involving LLMs saving Data Scientists a lot of time right now. I witnessed impact first hand and advised on the implementation and refinement for the group that built it.
I agree that LLMs are less useful for smaller scale use cases. I’ve found success using them for creating mock data, reformatting unstructured text into structured data, and a few other menial tasks that would take me a while to do on my own.
I think so much of the hype is about potential larger scale applications but the models just don’t seem reliable enough yet for that.
Motte: "AI will make humans more efficient and help scale small businesses and creators"
Bailey:" AI can replace your entire human labor (the most expensive cost of business) by scraping the content made by laborers on the internet without compensation"
Unfortunately, businesses are being sold the Bailey. And the sad part is that some companies aren't even pretending to hide behind the motte. I'm sure some, many businesses won't fall for the hype, but many others will and it will take a huge toll on the workers as a result.
Lots of companies that "chose" the internet first and foremost died, I mean, that's what the Dot-com bubble was all about. There was also a lot, a lot of money spent on networking/fiber around that same time, with the "winners" not being the people that actually had put in the money, but companies like Google, Facebook and the same Netflix about a decade later.
No, the internet was always obviously useful. It's the best communication tool since the written word and people were using it as such from the beginning. Of course not many people had access in the 1980s but those who did (college students) were using it constantly to communicate with people around the country (and soon the world).
AI (in the general sense) is also obviously useful, but the jury's still out on whether LLMs are the way to get there. Right now their best use-case seems to be cannibalizing the search market. It's not clear whether they can fully displace search engines though, given their stubborn propensity for confabulation.
> these companies would not be better served by staying away and ignoring the temptation.
There is a finite number of hours, dollars, and willpower within an org. Companies are pivoting into becoming AI companies without any understanding of what that means.
It’s so different though, I don’t see why we even compare the internet and AI. The internet connected people, of course there would be money in that. AI cuts out people, how will that make money, except for the very few people at the very top controlling the marionette strings? And now all the people without a job have no money to buy products or services? That doesn’t bode well for any market.
> And now all the people without a job have no money to buy products or services?
AI isn't at that level yet, and I don't know when it will be. When it does get that good, the meaning of wealth changes at least as much as it did in the Industrial Revolution when land took a second seat to capital, and I have no idea what our societies will do any more than Adam Smith could have imagined Communism.
For every Netflix there were many Pets.coms and for every Blockbuster there were a lot of Petcos, whose website at the time of the collapse of Pets.com was still just a basic marketing page [0].
It's not that companies shouldn't be wondering if and how LLMs will change their industry, but there's a very real possibility that the answer for the next 20 years is "A bit, but not enough for us to need to drop everything yet. Let's keep watching and see how this pans out."
People said the same things about blockchain—that it would be the foundational tech that would power a new iteration of the Internet, a web3.
Not that LLMs are in the same category, they certainly seem to have more utility in certain contexts than I ever saw in blockchain, but OP is correct that not every hyped technology turns into the next internet.
There are real use cases and value from blockchain, but it is far more niche than the crypto and NFT folks wanted to believe. IMO there was a lot of horse-cart thinking. “Blockchain is so amazing any product built on it will be successful!”
Which I guess gets back to AI. But AI is so much more general purpose than secure public ledgers. One is a screwdriver, the other is a million pound press.
even if you look at something like DLSS where “all” it’s doing is giving you a tractable approximation of a difficult optimization problem (weighting temporal samples inside a TAA algorithm), that’s such a broadly-applicable approach that it alone would probably change society. The value of increasing shipping or rail efficiency by 5-10% alone is worth billions, and there’s probably thousands of smaller optimization tasks it can be adapted to.
The LLM stuff is an even bigger pot of gold potentially, a workable semantic embedding for semantic queries or other things is another thing that a million improvements can be shaken out of.
it’s actually bizarre to me how much people stand on the “no commercial value!!!” thing, and I have to view some of it as just being the next round of histrionics from the same people who are mad their deviantart got scraped etc.
And everyone with any sense said blockchains were stupid tech.
So many people have been following the hype of foolish tech trends that they can no longer differentiate them from actual ground shifting technology.
It's sad what blockchain hype has done to a supposed tech audience. To this day all nobody can describe an actual large impactuf use for blockchains other than get rich quick. Even with the tech there it does nothing for day to day life.
Duplicating portions of human intelligence has tremendous practical use in every facet of society. The tech just needs to get there.
> Duplicating portions of human intelligence has tremendous practical use in every facet of society. The tech just needs to get there.
That last part is the only part that skeptics are skeptical of.
GPT-4 is amazing in a lot of ways and shouldn't be discounted, but the grandest predictions of AI's impact depend on the exponential growth we saw in 2022-2023 continuing. What we've seen this year (GPT-4o, increasing cries from the established players for governments to build them a moat) suggests that that's not happening. We seem to have hit a cap on capabilities and have started just making it smaller and faster.
This is great and will be useful, but GPT-4-level tech isn't going to revolutionize everything the way that the hype suggests. We need something more.
>I think you could go back in time and s/AI/internet.
That's a great example. You could also include systemic events like the introduction of the microcomputer, smartphone, and even "Web 2.0." AI looks like it will be a systemic change, we just don't know what that change is yet. Companies that find the winning formula will have a competitive advantage. Companies that ignore it will start a downward spiral that will be difficult to survive.
But, it is almost certain that _some_ of the AI initiatives will be incredibly successful and they are going to make mountains of money, while others will be simple
money sinks.
Investors are currently betting on the ones they think going to be the first category and everyone else are trying to get a piece of the money being spent in the industry.
Yeah, but if you're, say, Home Depot, or any other large non-tech company, your company culture will simply not be smart or flexible enough to innovate. The VPs of Data in those kinds of places are the true marks in the game. They have money to spend and no ability to determine real AI from bullshit. They also will be rewarded in the short term for buying something shiny but useless. Which if they're smart they'll leverage into a CTO position somewhere.
It’s not just executives, it’s many employees at most companies.
I’m a hiring manager for multiple roles across engineering, product, sales, marketing, and ops.
90% of candidates (regardless of IC vs. manager, regardless of seniority) loves to ask what the company’s AI strategy is, or “how will AI impact your business in the next 5 years?”
It’s easy to say the hype is driven by the execs at the top. Maybe it is. But your average IC is equally interested, in my experience.
> 90% of candidates (regardless of IC vs. manager, regardless of seniority) loves to ask what the company’s AI strategy is, or “how will AI impact your business in the next 5 years?”
It may be possible that they just want to sound as if they are "aware" of the latest trends, especially considering how often companies talk about it.
It’s possible at least some of those people enquiring about the AI strategy are looking to find out if a company is about to ruin itself with bad decisions around AI. And some others are probably saying what they think people want to hear.
And yes, of course some are caught up in the hype; that’s the nature of these hype bubbles.
But that’s a good idea. Next time the question comes up I’m going to try flipping it around to ask how they think our industry will be affected by AI since you’re right, it does feel like a hollow question.
replace "AI" with any past trendy buzzword (big data, ML, XML, XaaS) and your statement remains exactly the same, hence, carrying 0 new information. just a historical observation :)
Yes, I mean, in the 00s bubble everybody and their dog needed an internet presence without any idea of what to do with it. The difference is not that between Amazon and Sears, but that between Amazon and the thousands of contemporaneous companies with clueless execs who had no idea what to do with an internet but were told that they now HAD to have one, preferably two.
The difference _is_ between amazon and sears. One was an internet company and thrived, one wasn't and died.
AI will do the same.
That a bunch of companies couldn't figure out how to use the internet is neither here nor there. There were plenty of factories that never figured out how to use electricity. Doesn't mean that steam power is still viable.
Nope, the context was poor return on investment on AI, and the fact that it’s because AI is currently in the overinflated hype cycle part of emerging technologies, just like all the other examples once were.
(Nb. someone using the term "AI" in 2024 can mean two things: either they just use it as a shorthand for referring to the currently hot incarnation of "AI", ie. "generative deep learning models with cross-attention layers", or they have no idea what they’ve talking about. And a priori the latter is much more likely.)
Sure, at this point generative deep learning models are somewhat likely be more like the internet and less like blockchains with regard to being an enabling technology. That doesn’t mean the hype cycle is not real.
Steam power is – and has always been – used to produce most of our electricity, BTW, and it’s only now changing, out of necessity.
If you think of XML (and JSON, YAML) as a standardized CSV that can handle hierarchy I think that's beyond a useless fad. There are lots of edge cases in parsing data and having standardized tools for that makes it a lot easier for systems to communicate with each other.
I think the hype was there because companies needed to update their data to follow standards and update how that data interacts with other companies. The specifics of XML are less relevant - once things are standardized there can be standardized ways of converting to other formats.
Imagine needing to write custom parsers for every API; not needing that enabled SaaS APIs to be much more practical.
The part I don't really understand is that even if we built the AGI, what would actually make it want to work for humans? What's the ROI on that investment look like?
I'm not judging whether or not building AI is possible, but I do ponder the psychology of it. Let's see...AGI might bring us eternal life, Utopia and a way to avoid having to work ever again...sounds a bit like something from the bible to me.
> what would actually make it want to work for humans?
Well, I mean, that's the fundamental question of the entire AI alignment field and has been for over 20 years :D The question for which the wrong answer means the potential extinction of not merely the human species, but everything that we find valuable in the reachable universe.
Now, let's be very clear that the question is not "Why wouldn't it just invent its own goals and ignore humans?" That's mere anthropomorphizing. What an agent wants to do is by definition the goals output by its goal system. And the goal system is supposed to be designed by us, but we may not be smart enough to design a goal system that won't be disastrous to us if implemented by a superintelligent agent.
I’m not worried about extinction though , I just mea , why build to enslave ? How would you enslave something gore capable than you to recoup the 50 trillion in investments ? I mea. I get
"Enslave" is the wrong question, it implies the AI is being forced to act against its own desires.
That might happen if the AI has an off switch, knows this, and knows that we'll use that if it goes out of line, and wants to live.
But it isn't necessary.
The point is to construct it to want what we want, in the sense of sharing our values rather than envy, and then there's no conflict in the first place.
We just don't know how to even properly specify what we want.
Every attempt so far to specify what it means to be good has edge cases and loopholes — we don't even all agree on "don't kill" (war, death penalty, suicide assisted or otherwise, abortion, pollution that carries a statistically significant increase in mortality rates where no single death can be definitively proven to have been caused but overall many must have been, meat).
AGI will not bring utopia. The folks promising "abundance" are either delusional or deliberately misleading.
There are finite resources on earth, and no bounds on how much energy can be spent thinking about something with AGI other than the will of whoever commands it.
People want to have yachts and McLarens. There will be luxurious ways to consume large slices of the AGI pie that people or countries will not forfeit.
Utopia has many meanings, and some of them are impossible to achieve for all of us because some of us are unhappy unless we have more than others.
But there's sufficient resources on this world to give ten billion people each their own luxury yacht with a crew of servant robots, and there are people who would gladly use their own wealth to make others better off.
It's going to be a wild ride, no matter what human psychology drives us to do with this future.
There are sufficient resources to feed everyone on the planet currently, yet we don't. A future where everyone owns their own yacht and robot fleet is completely out of the bounds of realistic expectation.
Robots can (and already partially do) make more robots.
When the AI is good enough for them to genuinely replace all human labour, it's good enough for them to be von Neumann replicators; at that point equality is more about human psychology — greed vs charity — than practicalities.
They should take the apple approach. There will be hundreds of options when they decide they really need an “AI strategy” it will fizzle out. It’s pretty obvious for Apple that they can use AI for some tasks and they’ll work it in practically as opposed to panic Keeping Up With the Jones’ mode. Apple is doing it the right way, since they aren’t actually in the AI business (yet), just a consumer. This is the way people need to think. When an actual use for LLMs presents itself then there will be plenty options, no need to panic.
FOMO is legit though. Every organization has an obligation to assess if LLMs and OpenAI(-like) can improve their workflow. No need to expect much of it, but every org should know what they’re missing on or why they’re not missing on anything.
Part of what’s driving this is that CUSTOMERS are buying the hype also. So when two companies offer a similar product, but one slaps a “driven by AI” sticker on the box, they get more sales.
So, does that make the executives stupid for trying to find a legitimate reason to shoe-horn AI into a product?
> to justify those costs, the technology must be able to solve complex problems, which it isn't designed to do
Planning and reasoning are the two greatest areas of research in AI right now, with an OOM more researchers devoted to it than there were to the first generation of generative AI architectures
> In our experience, even basic summarization tasks often yield illegible and nonsensical results
Summarization with current generation models is excellent. I can get a summarization of a several-hour-long-call with better recall than I could have had myself, for less than $2 in inference costs.
> even if costs decline, they would have to do so dramatically to make automating tasks with AI affordable
We’ve seen a literal 10x decrease in cost from gpt-4-32k to gpt-4o in a single year of AI development (3:1 cost blend). And that ignores that sonnet-3.5 is 50x cheaper than gpt-4-32k while getting better scores on pretty much all benchmarks?
> the human brain is 10,000x more effective per unit of power in performing cognitive tasks vs. generative AI
Patently false, we’re not untethered brains floating around and require shelter, food, and a ton of other energy intensive requirements to live, and an AI system can perform a task that it is designed to do easily 10-20x faster than a human could.
If anything this makes me more bullish about AI systems having a positive ROI; the criticisms they have are based on extraordinarily (if not nefariously) dumb assumptions.
> the human brain is 10,000x more effective per unit of power
Even if true, is completely irrelevant. What matters is effectiveness per dolar, not per watt.
Does Goldman Sachs have an energy budget per employee, do they hire based on energy consumption? Do they prefer hiring someone living in a small house versus someone living in a mansion with a gigantic energy bill when both request the same salary?
I agree! I think it’s “worth it”. A parallel: it costs an OOM more to drive places than, say, bike places, but it allows us to do the things we did before an OOM faster and expands the types of things we can do.
You can make the argument that it’s not “worth it” in terms of cost to our environment, but you have to acknowledge the upside of it simultaneously.
And the dynamics are different in that we’ve reached an efficiency asymptote with cars and are still in the middle (or perhaps at the start) of the S curve with AI systems.
The human isn't a single core, 100% pegged on that task for those 20 minutes. Even actively working at something that energy consumption is likely to be less than 10% of that. There's a whole lot of energy being burned by the cardiovascular system, respiratory, digestive and renal, endocrine, muscular system (all that typing, moving and focusing of the eyes), and we haven't even got to the energy of the "OS" execution in the brain to control all that.
I don't think your numbers are the insight you think they are, nor accurate - not even particularly close.
We're neither disembodied brains nor bodies in isolation of basic needs like shelter or desires for energy-intensive conveniences, so I challenge that we should try to assess energy consumption in the way you're proposing. The outcome and the inputs required to get to it are the two relevant variables IMO.
Being consistent, though, I should've included the amortized cost of training and the energy cost of the rest of the data center to the 4o analysis. I don't think that those costs would be more than a factor constant greater.
When we can create disembodied brains that don't have needs for shelter and desires for energy-intensive conveniences, I buy that we can do 10% of the energy accounting.
In fact, if you can predict LLMs getting another 10X cheaper and 10X smarter over the next few years, it makes sense to start building LLM integrations today; even if they don't work well and cost too much today, every few months your feature will get better and cheaper, with very little work by you. Sounds like a fantastic ROI to me.
> Planning and reasoning are the two greatest areas of research in AI right now
Yes, but it's not something LLMs do. At all.
Each day I get more certain that AGI will surprise everybody and come from a nearly moneyless garage. And it will break every AI strategy people make.
> I can get a summarization of a several-hour-long-call with better recall than I could have had myself
The problem is, it works most of the time, when it doesn't it's dangerous. That "most of the time" part isn't large enough for people to ignore the failures.
But this is one of the things where LLMs can shine. Your example looks more like it's solving a really bad problem that shouldn't exist, but it's possible that this adds lots of value somewhere.
>The problem is, it works most of the time, when it doesn't it's dangerous. That "most of the time" part isn't large enough for people to ignore the failures.
I doubt that. A lot of failures will be ignored and only fixed if and when necessary because they are not usually very dangerous. A lot of information we produce is never acted on. And a lot of information we do act on is incorrect or unfounded anyway.
Also, in places where hallucinations are unacceptable, double checking some facts is still far cheaper than manually producing the whole thing from scratch.
I bet we will see an avalanche of incorrect and funny information coming from customer support systems. And yet very few people will want to pay for better fact checking.
>Summarization with current generation models is excellent. I can get a summarization of a several-hour-long-call with better recall than I could have had myself, for less than $2 in inference costs.
The company that I'm in has moved to all calls being video calls and recorded so we can get minutes out of them automatically.
I had the bright idea of using magic phrases to add an item to a action list. Now everyone says nonsensical words so a computer can remember what we want to do for us.
I would love to know what you're doing right and I'm doing wrong, because my experience with AI has been mostly crap. I've tried using AI to summarize articles, but then the AI will say something interesting, so I actually look at the article and quickly realize the AI summary was wrong.
You might want to double check that what the AI is telling you is actually accurate and not just blindly trust its output.
I’d have to give away some of my secret system prompt sauce to tell you, but I’m sure that the summarizations I get are solid, we’ve done extensive evaluation of truthiness. I have tens of thousands of hours of calls summarized monthly from paying customers who rely on them to be accurate. We’ve only ever gotten hallucinations when we accidentally omitted the transcript, and even that was fixed by allowing the LLM to alert us when it didn’t see a transcript.
I use an AI noise canceling/transcription service to help with my extensive meeting agenda.
I've lived in the US nearly 20 years, coming from Australia. Whether it's my accent or something else, most transcripts for me need probably in the order of 20% meeting duration for me to tidy and edit to be free of, in some cases, quite asinine, transcription errors.
Unless your customers are doing that, I have a hard time, using the principle of garbage in garbage out, of believing "zero hallucinations" (and even then), but even then, the prompts would be something along the lines of "transcribe this, but just ... do better".
Every investment company out there knew this from day one. They were riding the hype and gains. Now the late and individual investors will pay for the losses while the big investors start moving cash to the next hype.
I think you are crazy putting money into passive index funds.
As for the investors, knowing when to pull out is when they start pushing articles like this. They’re already out and want your capital to go where they are going next.
Your plan for social instability is hodl stocks with increasingly suspect value?
Hm. Mine has been learning how to manage small crops, carpentry and vehicle maintenance.
In the 80s laws were changed to redistribute pensions into Wall Street. I say we do the inverse to Wall Street and expropriate the gains as “fruit of a poisonous tree” with new legislation.
Really? I can think of at least a few cases in the last 100 years where it boomed unexpectedly; world war, energy crisis, recessions, depressions, pandemics…
Someone’s survivorship bias has lead to confirmation they’re ready for anything it seems.
Most Fortune 500s of last generation are gone. Good luck! Count me as one that is not making political choices with yours in mind, and would, in the future, not be honoring some story you have to tell about an unknowable past.
I think you missed the point. I'm not ready for any large change. No one is. What I have done is get myself into a position where I can do what I want to do before there is a change and hold out as long as possible. It's a dampening effect, a parachute. I'll be in the same shit as everyone else, just later.
Well I’ve pretty much pulled it out of everything and bought a house (no mortgage yay) but I rode NVidia, BAE Systems with a safe position held on an actively managed ETF.
I’ve got a huge pension lined up so I’m going to go on holiday a lot basically. I get a lump sum early payment of that soon so I can invest that somewhere more productive.
What if, and this might be just a guess, the hundreds of thousands of people at major investment corporations have thought of this as well and driven the price up so that they are now adequately valued?
Do you know more about the future of these tech companies then the legions of math/physics/economics/CS PhDs paid to investigate the potential of these companies?
When you're doing long-term investments, it's not a zero-sum game. It is possible - likely even - that you and those hordes of PhDs all come out ahead.
>It is possible - likely even - that you and those hordes of PhDs all come out ahead.
No, it isn't. This is total nonsense, if the PhDs believe a stock is undervalued they bid it up until it is even valued.
The only secret strategy an individual has to outperform the market over a long time is insider training. Everything else means you are betting against the market being efficient, which is risky, to say the least.
If the sales and earnings go up in the long run, and the stocks aren’t super overvalued, then the stocks go up. It doesn’t matter what other people think. All that matters is the performance and valuation.
The price of a share is determined by exactly one thing, how much people are willing to buy them vs. willing to sell them.
If the market believes a certain stock will outperform the average it is free money to buy that stock, so prices will rice exactly to the point at which the market believes the share will not outperform the market at that price.
This is literally basic economics. Google "efficient markets".
You can see the price for yourself though vs the fundamentals and judge whether it’s a fair deal. The market isn’t omniscient and has been mispriced very frequently.
Seeing that AI is being subsidized by investors I wonder how long it'll take for said investors to want a return on their investments and if AI is too expensive for anyone to actually want to use it. Because I don't think a lot of people would pay the actual cost it takes to run AI.
Is this not expected? Every hype encounters a “Trough of Despair”. I think costs need to decrease dramatically before the current state of AI is worth the money that is being dumped into it. Both in terms of GPU costs and electricity because it’s insane.
What boggles my mind is: what happens to Nividia’s 3T market cap if GPU costs come down by 10x or 100x?
Will there be 10x or 100x increase in unit sales volume needed to offset the price decrease while maintaining their 3T market cap?
I really wish there were an easy way to remove Nivida from my (mostly passive ETF) portfolio. I started purchasing NVDA puts simply to protect gains on SPY holdings where I simply want to reduce my Nividia exposure.
"AI" is talked about like an individual company instead of like a mortgage-backed security, whose intrinsic value is made up of lots of little mortgages, some of which will pay out, and some will go into default.
It might be a bit of a "Prisoner's Dilemma" situation, and by being the first (among hype hawkers) to point out the Emperor's lack of clothing, they're hoping to come out ahead by throwing their competitors under the bus first.
If this catches on, it'll mimic the lets-do-layoffs meme spreading virally among tech corps -- IIRC it was Elon's "look I can layoff most of the company and it'll still keep going" which started that trend.
Edit: Let's not forget that Goldman Sach will definitely be betting on these outcomes (puts, etc.), so by pushing this narrative, they're definitely going to benefit.
I know that we "want" investment in AI, but I'd be perfectly happy if Wallstreet stays the heck out of this whole situation. Every time investment money gets into a topic - not company, not an industry, a topic - it becomes an absolute mess.
Investment feels like a micromanager that won't let you do your work.
It's quite likely to be like the 1999 dot com boom and bust. Investors put a lot of money into things like Webvan which went bust in 2001 while a lot of the value ended up in the likes of Google which didn't float till 2004.
I'm sort of dreading the next year or so at big corps, in which engineers will hear from their management: "We've made a big investment in AI, and need you to make this work".
Amen. I said the same thing many months ago but the true believers on HN who could not see beyond their own paychecks down voted me like I was advocating for satan.
