Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Gen AI: too much spend, too little benefit? (goldmansachs.com)
64 points by FinnKuhn on July 2, 2024 | hide | past | favorite | 51 comments


You know you're in trouble when even GOLDMAN SACHS is like "maybe this thing is a bubble"


I read somewhere that their accuracy is on par with a coin flip, but that's also true for all analysis that is posted/voiced publicly.


50% accuracy out of the gate is incredibly good for investment analysis.


How so?

Isn't that equivalent to saying on heads I buy, on tails I sell? What makes that incredibly good?


If I could predict with 50% accuracy which startups would unicorn and which would die I'd be extraordinarily wealthy.

For example, invest $1000 in two companies, and expect a 100x gain and a $1000 loss over the next 5 years.

(I say "out of the gate" because you can change your bets as situations and positions evolve, and I'm sure MS do.)


If you had <50% accuracy in expectation you could improve your analysis by inverting it, I suppose...


And, to improve your chances further, you'd just need to know when you'd have more to gain by inverting your choice versus not inverting.


It could end up like Chatbot (pre-Gen AI) and 3D Printers. I started a company a few years back, literally with the name "bot" in it, and bought myself a 3D printer. Both are left in the dust.

I don't think Gen AI will be totally bust, but it won't be as promised (anytime soon). Just like in a software project, the last 10% is another 90%.



I decided to not use this link directly to the .pdf as on my iPhone this appears to take you to the top of the .pdf and not the specific article.


Do the people who produce this kind of document not have mobile phones?

Reading a two column PDF on even a high end iPhone is not a pleasant experience.

(And no, I'm not going to break out a laptop to read something like this.)


1) You are not the intended audience.

2) Much like a research paper, reports like this benefit from the standard page size. You will fit relevant information on the same page for the benefit of the narrative/reader. This is one of those areas where I believe its hard to make it work on a mobile device.

3) This is just the standard for reports.


I think it's a bad standard.


The rest of the industry disagrees with you. PDF is the standard for reports, it is not meant to be a web format. You could definitely do something like arxiv where you could make an HTML version but again the standard is a PDF so it can be a challenge to get graphics/charts have the same meaning and scope of view when the format of a page has changed.


I wonder if industry standards will change as more people spend more time working on their phones.

I reckon a new generation moving into influential decision-making roles will probably sort this out over the next decade.


Doubtful as the industries that consume these types of reports are not doing work on their phones. The "new generation" are the ones using tablets and laptops to consume reports instead of printing it out.

Similar to a powerpoint presentation, pdf reports are not going anywhere.


You can't do productive work (or game) a phone. It's an inherently inferior medium.

There's both too little real estate to present information and too few control inputs to manipulate it.


There is something poetic about using GPT to summarize that pdf so you can digest it using a small screen.


You're not the intended audience. You can read the cliff notes on Engadget later.


Maybe they think it’ll get printed out for CEOs to read?

Love your blog!


A whole lot of large tech companies are proudly shipping their “it demoed well” hackathon projects. Investors are far more eager for these things to get released than customers are to use them.


I think you’re underestimating how much ChatGPT is already creeping into people’s lives. We now have studies are showing it’s in 10% of new scientific papers, for example. It came up in Matteo Lane’s latest comedy special where someone used it to write a rejection text to someone they were dating. It’s already enshitifting the internet and being used by Russian political bots on Twitter.

Keep in mind it came out about 1.5 years ago. That is insane market capture in a wide diversity of ways already.


Stuff can creep into people’s lives while still being unprofitable for companies that are building on it. Crypto is clearly a big deal to a bunch of folks, but when large tech companies jumped on the blockchain bandwagon almost none of those efforts bore fruit.

LLMs are clearly more useful to the average user than blockchain. But there’s still an amount of over-eagerness.


How about for companies products? Do you feel that this AI everything is justifiable? I don't think anybody questions that ChatGPT doesn't have any uses but is it profitable to the investors? I guess only if everybody buys into the hype and they make their money.


> I think you’re underestimating how much ChatGPT is already creeping into people’s lives. We now have studies are showing it’s in 10% of new scientific papers

It also excels in another kind of spam which is to generate articles to drive traffic from SEO.


AI does have some legitimate uses (i.e., Grammarly and GitHub Copilot). It’s just when people say they are going to replace a whole operating system with AI that they are full of shit lol


Like the article asks, what is the 1 trillion dollar problem gen-AI solves? Sure there are use cases - but the point is that the potential value doesn't remotely line up with the investments being made.


World military spending is estimated at over $2 tn per year. Ongoing Russia's invasion in Ukraine shows that a hypothetical UAV with fully autonomous capabilities (because every known frequency is soon jammed) will meet enormous demand.


GenAI won't help with autonomous systems.


My assumption is that only part of those $1 tn will go specifically into generative applications. The Goldman Sachs' paper in most instances mentions AI in general.

But one can also argue that actively seeking and choosing targets is a generative task, given a proper encoding of a battlefield situation and current mission.


Except that in spite of enormous investments and engineering efforts we still have problems with fully autonomous vehicles and it's unclear it would really happen anytime soon, and here the classification problem is whether you're going to bomb an enemy or your own people. I'm not saying it's not gonna happen but it's terribly hard and 80% accuracy can wreak havoc on innocent people.


Russian air force has already destroyed numerous civilian buildings, in many cases killing dozens of people far behind the frontlines[0]. In at least one case, when 59 people attending a soldier's funeral were killed, there are "reasonable grounds to believe that the reception was the intended target of an attack, using a precision weapon"[1]. I think it shows that the military will be happy with much lower quality of classification than e.g. acceptable for driving in downtown San Francisco.

[0] https://en.wikipedia.org/wiki/Attacks_on_civilians_in_the_Ru... [1] https://www.ohchr.org/en/documents/country-reports/attack-fu...


— Hey ChatUAV, please bomb enemy’s military base

— Certainly! bombs own civilian hospital

— No, the hell are you doing…

— You are right. I apologise for the oversight.


I had no idea it had reached US$ 1tn Capex, that's an insane amount of money. That's 1/62th of what a Stanford study said would take to move the whole global population completely away from fossil fuels for our energy demands...


But can clean energy rewrite War and Peace in the style of a Green Day song?


this comment made my day.thnx


I wish!

Green energy already sees nearly $2 trillion in yearly capex: https://www.iea.org/reports/world-energy-investment-2023/ove...

Total previous green investments are $8-10 trillion, while fossil fuel consumption's not gone down (fossil fuel investment is falling, though.) That Stanford study undercounts and presumes a massive increase in storage.

If really dig in, our current renewable layout focuses on the low hanging fruit, but storing solar energy till the night time or a different part of the year is infeasible (look into "duck curves") and currently done by burning natural gas. Current storage production is up a full magnitude, but we need multiple further magnitudes more to do a full transition. The increase in transmission lines alone is more than all known copper (mined and unmined).


if you want to achieve some greater good, capitalism is not the ideal model to begin with. it creates a lot of wealth, but needs strong external forces to not only focus on profits or destroy everything in the process.


I believe Europe in general is the closest to that ideal. You have enough money to live comfortably and relatively strong guardrails in place to prevent greedy corporations from poisoning you in various ways etc.


Feel free to move the the countries with few if any successful capitalists(e.g. Chad, South Sudan)


Maybe we should consider something post-capitalism then. This rhetoric of "There's No Alternative" repeated ad nauseam on HN is very unimaginative.

This system has no imbued morals or ethics, and the more we try to imbue some to it the more it fights back (again, the ad nauseam "less regulation" repeated over here). We need a new system that evolves this bullshit into something more humane.

It's not like the USA with its hypercapitalism, and absurdly immense wealth has achieved the greatest utopia on Earth so we kinda know capitalism is not the final answer.


There's no way that that amount of projected spend over the next 10 years doesn't result in a lot of new efficiencies in chip production costs and energy usage, particularly a move away from graphics cards to more purpose built chips and algorithms designed for purpose-built chips.


It seems like the killer apps for generative AI right now are:

1) Automating boring reading and writing tasks. Think marketing copy, recommendation letters, summarizing material, writing proposals, etc. LLMs are pretty good at this stuff but these are not many people's core job responsibilities (though they may take up a lot of their time). Consider it a productivity booster for the most part. Some entry level jobs will be eliminated, and this may create problems down the road as the pipeline of employees to oversee LLMs erodes.

2) Code writing tools a la Copilot for certain "boilerplate" code in commonly used languages. I think the impact is similar to (1) where entry level jobs erode and this may impact employee pipelines.

The core problem (as I see it) is that LLMs don't produce outputs good enough to be used without human oversight except on a small subset of tasks. So you end up needing humans (maybe fewer of them) to check the LLM output is headed in the right direction before you let it out into the world.

Consider voice interface LLMs for customer service. When will they get good enough to do the job with real money on the line? If your airline help desk keeps giving away free flights or on the flip side infuriating passengers by refusing allowed changes, can you really use it in production? My sense is they aren't good enough to replace the usual phone tree just yet.

When accuracy doesn't matter that much, LLMs will really shine because then they can be used without a human in the loop. Think some marketing/advertising and especially, especially propaganda.

I think the existing killer apps don't yet have enough money/savings in them to justify the spend. If generative AI technologies can get good enough on the accuracy front to remove humans from the loop in more contexts, we will be talking about much more dramatic value.


> much more dramatic value

We will in fact be talking about the most valuable thing ever


What is lacking in AI is a concept of pain and suffering. Those are key to the human experience. If we want a truly capable AI, or even an AGI, we need to add in a suffering feature to these models.


Is there any way to short Open AI and this nonsense that AGI is round the corner? They might prove to be useful to some degree (niche areas) but as soon as I give the LLMs information or problems they haven't seen before they completely struggle to do anything sensible. Try getting them to compare two different JSON documents, they are very confident and produce absolute garbage if these have not been seen at the training stage.

If they can't handle new ideas humans will always be much more useful and these systems are good for references and human learning are not good for creating something new and of value. I've noticed for text LLMs are quite weirdly repetitive and have an empty style that requires a lot of editing to get it into a shape humans would craft.

People will say the improvements are coming but I think most of them have come from more data which is running out. I think one of the most profound things about real intelligence is being able to define and update concepts within your own mind… how to add new information to LLMs in realtime and have that reflected across the board seems intractable given the training and refinement these things would sit upon. There is no clear unit of information about a concept that links to all the other ideas. LLMs seem quite limited by this.

The brain is so much more complex than these algorithms too and so much more flexible, I don't see how a very good encyclopaedia with some fuzzy AI concept extraction capability is in any way the same as the human brain being able to apply and adapt concepts from all around art, science, literature and the human experience.


"Try getting them to compare two different JSON documents"

If you're using ChatGPT, try adding "use Python" and see what happens.

Tool usage like Code Interpreter dramatically increases the capabilities of existing models.


Remember, the market can remain irrational longer than you can stay solvent. Shorting is expensive.


> Is there any way to short Open AI and this nonsense that AGI is round the corner?

NVDA is a decent proxy for OpenAI, their market cap would evaporate if big tech stopped buying.


People said the same thing a few years back during the cryptocurrency bubble...


NVDA peaked around 30 in 2022 and went down to 10 after ETH went PoS. BTC went from 60k to 20k in the same period. Seems like a perfectly good proxy to me?

You could make the argument that rate hikes caused both, but NVDA was looking iffy in between crypto mining and ChatGPT.

I did not say I would short NVDA personally, mind you.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: