Not surprising at all. As a former Google employee, I'm regularly in contact with friends still at Google and the number of hardware engineers OpenAI has hired out of Google have drastically increased in the last few months. You can readily see for yourself just by browsing LinkedIn and looking at profiles of hardware engineers currently at OpenAI and noticing how many of them are former Google employees. Google's own TPU team is now a shell of its former self. I wouldn't be surprised if future Google TPUs are inferior to the new OpenAI chips.
Also, did you know that Google's TPU efforts previously also relied on help from Broadcom? https://www.theregister.com/AMP/2023/09/22/google_broadcom_t... It's absolutely unsurprising that former Google employees would bring this vendor relationship into their new company, given Broadcom did a good job helping Google with TPUs.
BRCM is becoming more like a service company designing ASICs for other companies who want custom ASICs. There are many companies who do this? How is this news?
BRCM is a leader in SerDes along with Marvell and other companies. There is so many IPs from Synopsys and Cadence that in turn many of these chip companies use. Not sure what is hoopla here. Nobody is going to completely vertically integrate all these IPs.
It's also worth mentioning that google spent a lot of time (and probably money) and human capital on their TPUs and slowly got there. Other companies that have a proven track record of hyper-scaling also tried and missed (tesla) so it's not as easy as locking a bunch of phds in a room and have them crank out LLMs. Hardware is finicky, slow to develop, slow to prove itself and extremely expensive.
"while adding AMD (AMD.O), opens new tab chips alongside Nvidia (NVDA.O), opens new tab chips to meet its surging infrastructure demands"
Jensen has been saying that demand is "insane" and we're hearing rumors of low yields. This equates to supply issues in the coming months/years. No fortune 500 puts all their eggs into one basket. Diversifying away from a single source for all of AI hardware and software, is a smart thing to do.
That is exactly what the execs at my company are telling us when asked about not using Nvidia -- diversifying away. It's funny though because we have no Nvidia for training at all. We use Trainium because we could not get our hands on Nvidia.
> Diversifying away from a single source for all of AI hardware and software, is a smart thing to do.
I wonder how this squares with the exclusivity contract with Microsoft. Even the OpenAI/Oracle deal requires Oracle to run Azure stack on their datacenter so MSFT can mediate the relationship. The AMD chips mentioned are also purchased by MSFT.
I wonder if this really means that OpenAI is accepting the risk/capital expense while providing a variety of hardware to Microsoft, or if there are other terms at play.
I think I get what you're saying. That a datacenter probably just wants to focus on one product at a time. You're right, it is simpler. On the other hand, that's part of why neoclouds are sprouting up. Users are asking for the optionality.
> OpenAI considered building everything in-house and raising capital for an expensive plan to build a network of factories known as "foundries" for chip manufacturing. The company has dropped the ambitious foundry plans for now due to the costs and time needed to build a network
That framing massively undersells how insane Sams ambitions were there, he was floating the idea of somehow raising seven trillion dollars to build thirty six fabs dedicated to making AI silicon. The TSMC execs reported more or less laughed in his face when he brought it up it them.
Sam explicitly commented on this claim. He said we’ll _collectively_, _eventually_ need to spend 7 trillion dollars on compute to meet demand. Not OpenAI raising 7 trillion to build all the capacity itself.
Building a character without being all that concerned with the truth is a way to have anything plausibly fit the character. But the resulting character is not real. It's fiction.
Can you cite a source for this? When was this said? Was this something said before these meetings or a walk back after they laughed him out of the room.
Hmmm, perhaps there's a sweet spot where someone is nuts enough that you think you can take over the stuff for cheap when they crash and burn, but not so nuts that there's nothing left to pick up.
The first time I came across this link I got sucked in for six or seven hours and pretty much dropped a ball on an entire workday as a result. Evil, evil game.
SOTA AI companies are modern airlines. They're selling a capital intensive commodity that has little meaningful differentiation once you pass a certain threshold. Customers just want the lowest price available.
During the post-2001 crash in airline passenger demand US airlines lost more money in that one slump than all profits from all US airlines, combined, from the Wright Brothers to September 11th 2001.
That was why you had such rapid consolidation, as America West became US Airways became American, United bought Continental, and Delta bought Northwest, to remove so much competition that they could be profitable again.
https://transtats.bts.gov/Data_Elements_Financial.aspx shows 2001-2005 net income losses were 57 billion dollars, then a brief profitability in 2006-7 but the losses of 2008 and 2009 wiped that all out again (in net income terms).
I read that thing about the integral of all profits back to the Wright Brothers back around in 2010, and I can't find a good source on aviation profitability in the 1970's right now.
Airlines themselves are perfectly fine. It's relatively hard to break into as a disruptor, but the employees of the established players are mostly well-paid. The executives are even better-paid. The investors can earn steady, predictable, positive returns as long as the stock is reasonably priced and doesn't make assumptions about impossible levels of future growth. Airlines are not super-profitable, but with sufficient volume, you don't need high margins, and they're at least profitable as long as we don't have 9/11 or Covid happening.
I'm not personally knowledgeable about this or anything, so look into it yourself, but just picking a random really big one, United Airlines, it looks like they consistently beat the S&P 500 for most of the past 20 years, until Covid, and the past four years since then have been pretty bad.
They are profitable regardless in the face of 9/11 and COVID because we will bail them out.
What they don't seem to be able to weather is significant changes in the structure of pension obligations, and dramatic fatal crashes.
There is also an impending shift into electric short-haul flights that is going to cause some changes to the business model, with whoever can't adapt exiting those markets.
It doesn't seem like a denial to me, more of a deflection. From that article:
> He added the number is not known, adding that the $7 trillion came from a report with an anonymous source. “You can always find some anonymous source to say something, I guess,” he added.
Yes, they're apparently not expecting to turn a profit until 2029, with their losses peaking at $14 billion in 2026. Their recent $6.6 billion funding round would barely buy them 6 months of runway at that peak.
IMO this is because Google is kind of shit now, because of ChatGPT and LLM spam in general. It's hard to find information on Google that isn't shallow attention thievery. I mean, finding a recipe that doesn't start with "the year is 1974. I was just born" is near impossible these days.
Right, but it's undeniable that LLM's are a core part, if not the most important part, of modern SEO spam. Particularly because they're much harder to spot.
That would only be meaningful if ChatGPT made as much money as Google did per query (as is, I believe they lose money on each, so they can hardly make it up in volume).
Google can easily monetize a search for "dentist in $city" where your intent is to spend money, OpenAI would have to monetize "sum up this email" where money is not involved at all.
If you have enough eyeballs on your pixels you can make money on ads. They are just deliberately choosing not to do it (Altman has made statements to this effect).
I don't think it's fair. There many $100B+ companies with less IP, and much less access to talent. I don't even see people lining up to work for Apple or dreaming to work for Google anymore.
Microsoft only gave them an exorbitant price because they were paying in Azure credits for GPUs. That's something like a 50% discount for Microsoft compared to giving them cash, and it also incentivizes OpenAI to continue using their stuff after the credits are burned through.
And if it was a 30% stake in OpenAI for that $7 trillion (kind of a standard VC round percentage) that would put OpenAI's valuation at about the same of all of the 7000-ish NASDAQ companies (including almost all public tech companies) combined.
There was probably a point of maximum hype ~12 months ago, or right after the launch of GPT-4 where the belief of imminent singularity was running high.
Considering that what is currently passed as AI is really just statical probability of string words together that is completely unable to apply logic or reason… we ain't even close.
I like the three comments above this one all seem to massively disagree or at least give off argument energy, but they all seem totally correct and don't directly contradict each other.
I'd recommend educating yourself on the last few years of neural network development before spreading misinformation.
LLMs have been shown to be capable of embedding logic and reasoning. There is nothing preventing a sequence generator from learning stochastic reasoning skills. Modern LLMs don't even just output language tokens. The latent information stored in the hidden layers is far more abstract than just language.
"We hypothesize that this decline is due to the fact that current LLMs are not capable of genuine logical reasoning; instead, they attempt to replicate the reasoning steps observed in their training data."
LLMs are a special type of transformer model, however several recent models have been multimodal, allowing for a variety of input and output type. So the question is if a neural network can approximate reality, not LLMs.
Though really, all embedded models of reality are approximate by nature, and based on stochastic empirical data. The real tradeoff with probability-based neural networks is whether a rigid algorithmic approach or a more flexible, but stochastic, approach solves the problem at hand.
Hype cycles ride on real progress that’s first blown out of proportion, then under-appreciated. LLMs are still magic AF. We are just so tired of hearing about them and how they’re going to change everything.
The singularity is always just around the corner. It's like a whale in a casino. The big reward is always just around the corner. You need to drop a few more coins and soon you will be rich
What is singularity anyway? I thought it was just some hypothetical point in time where making accurate predictions about the future becomes infeasible.
The singularity is a sci-fi end-of-days story. It speculates that at some point we'll invent an AI that's about as smart as a human, and smart enough to design a better version of itself.
Then a week later it'll come up with a version of itself that's twice as smart. A week later that version will come up with a version that's twice as smart again, making it 4x as smart as a human. A week after that, it'll come up with a version that's twice as smart again, 8x as smart as a human. And so on. A year later it'll be a trillion times smarter than a human.
The AI will then either solve all our problems, invent fusion power and faster-than-light travel, and usher in a Star Trek style post-scarcity world of infinite resources and luxury, or it'll take over and create a Terminator style oppressive dystopia where the few surviving humans survive on rats we roast over a barrel in our underground bunker.
At geological or biological timescales, that’s what the emergence of language did. AI can all be viewed as part of the accelerating returns of language…
There's a dystopian definition in the other comment but usually, when people say they're looking forwards to the singularity, they're referring to a version of the future where robots and computers are able to do most of not all jobs, so humans no longer have to work, and can spend all their time doing whatever, usually phrased as painting, singing, dancing, and the other leisure-type activities. Bettering themselves and helping humanity. The utopian, post-scarcity Federation from Star Trek, if you've seen it.
That was not a comment on the plausibility of it. Offering up a definition is still useful so people actually have conversation instead of just talking past each other.
The technological singularity was first described by John von Neumann. Are you suggesting you understand something about technology that one of the most intelligent technologists to ever walk the earth didn't?
Does it matter how smart someone is when they make things up?
We see this in AI talk all the time "Dude is really good at coding and math, he's smart. We should totally listen to him when he makes things up like a 12 year old who just watched Terminator."
Great question, and what you're referring to is an appeal to authority, which is a common logical fallacy. It absolutely warrants further examination.
In the case of von Neumann, I mention him because he is the one who introduced the concept to the public. It is something he spent a great deal of time thinking about. He spent his life working around technology and had some incredible insights. Many consider him the smartest person to ever live, though in my opinion there are plenty contenders. He made key contributions to the development of computers, nuclear tech, quantum mechanics, mathematics and more. [0]
So I believe all of this truly does lead credence to the idea, at least enough to warrant sufficient research before dismissal. It's not his authority I appeal to, it's his experience. He didn't obsess over this idea for no reason, and his well-documented achievements aren't comparable to random flash-in-the-pan tech executives.
My question to you was whether you know something about technology that von Neumann, the guy behind the idea, misunderstood, such that you would so flippantly dismiss it.
The ball is in your court to prove why the singularity is not real, because many experienced, decorated technologists disagree and have already laid out their arguments. If you can't prove that, then there's no argument for us to have in the first place.
Beyond the (absolutely correct) sibling comment about von Neumann's dated expertise, you are experiencing selectivity bias. Technologists who believe in the Singularity (and it really is belief, not a rigorous technical argument), are very vocal about it. Those that don't have faith, don't bother speaking about it.
There are a lot of people out there who believe in technological progressivism and continued advancement of socially beneficial technologies. They just don't speak about a "singularity" beyond which predictive power is not possible, because the idea isn't worth spending time on as it isn't even self-consistent.
It's not our job to show why the Singularity won't happen. It's the nutjobs who believe in the Singularity who have the responsibility of showing why. In the 70 years in which people have been bitten by this idea, nobody has. I'll wait.
> It's not our job to show why the Singularity won't happen
No, but shallow dismissal of a subject that many intelligent technologists have spent their life considering just comes off as arrogant.
Predicting technological timelines is a largely pointless exercise, but I wouldn't be hard pressed to imagine a future N decades or centuries from now in which, assuming we do not destroy ourselves, human affairs become overshadowed by machine intelligence or technological complexity, and an increasingly complex technosocial system becomes increasingly difficult to predict until some limit of understanding is crossed.
This is a epistemological problem, you think Neumann's expertise on 1950s technology gives him insight into the technology of, let's say the 2040s. You muddle the issue by calling him an expert on "technology" rather than an expert on "technology as of when he died in the 1950s".
If we don't grant that expertise on 1950s technology gives insight into 2040s technology, then there's little reason to consider his writing as something other than a cool story.
> You muddle the issue by calling him an expert on "technology" rather than an expert on "technology as of when he died in the 1950s".
That's not my intention. It's more that, he simply looked at the historical progress of technology, recognized an acceleration curve, and pondered about the end game.
My argument is that it warrants consideration and not shallow dismissal. Plenty of opponents have criticized the concept, and some decent arguments revolve around rising complexity slowing down technological advancement. That is possible. But we can't just dismiss this concept as some fleeting story made up "like a 12 year old who just watched Terminator".
There's no dark budget, it's all there, the programs are just usually listed as "Classified" or "Special Activities" like NGA funding was in the late 90s.
> This does not include many military-related items that are outside of the Defense Department budget, such as nuclear weapons research, maintenance, cleanup, and production, which are in the Atomic Energy Defense Activities section,[90] Veterans Affairs, the Treasury Department's payments in pensions to military retirees and widows and their families, interest on debt incurred in past wars, or State Department financing of foreign arms sales and militarily-related development assistance. Neither does it include defense spending that is domestic rather than international in nature, such as the Department of Homeland Security, counter-terrorism spending by the Federal Bureau of Investigation, and intelligence-gathering spending by NSA, although these programs contain certain weapons, military and security components.
> Accounting for non DoD military-related expenditure gives a total budget in excess of $1.4 trillion.[91]
The Pentagon frequently "loses" $1T here and there and then says, "oopsie!" with no actual consequences. That looks like a dark budget to me once you factor out typical government waste and overredundancy.
Dark budget? They can't even supply Ukraine with enough ammo and weapons to defend themselves and the DoD is already squealing that they are running out of everything. Dark budget..hahahaha...good one....
It seems totally possible that enough interest groups don’t want Ukraine to actually win, so they block anything more than the minimum viable amount. So it’s not much of a barometer at all for anything…
Yes, but this theory requires more and more epicycles.
There's probably some small-ish 'dark budget'. But think about it: the official budget is in the order of single digit percentage points of US GDP. That represents and enormous amount of resources. Any appreciable fraction of that in a 'dark budget' would also leave traces all over the place.
The world is vastly more complex than say the world of 1945, so almost certainly there are now likely several dozens to several hundreds of ‘epicycles’ that are in fact true and necessary for full understanding.
Edit: Now the pentagon doesn’t contain the entire complexity of mankind within it… but it’s probably not that far off.
Hindsight bias.
States, like us human beings, are not nor ever have been completely in control of themselves and exist out of different "parts" making the whole.
Also, when expected outcomes don't align with perceived results, it's usually best but not very human to doubt yourself.
You never have all the information you need for making the best decision or opinion.
"Dark Budgets" vested in firms like Dwave via firms like inQtel, steer research about quantum computing. Whoever has Shor/Grovers going first, has a advantage equitable to winning the AGI race... That too can win wars.
Nothing is simple and perceived simpleness is on the clock, for everyone.
It's not inherently different, how we deal with it is.
Drawing conclusions from linear seemingly causal events is nuts considering that most start calculating from the future desired outcome to the present.Someone may do something stupid on purpose in order to gain later.
Adversaries may do the same, the math can quickly spiral beyond human/individual comprehension. For a fact at least for a part of whoever may be involved in some way or another. Complexity is not random, and we can choose our initial conditions to our liking, causing $outcome. We now have entire DC's crunching numbers, not just a bunch of nerds at Bletchley Park. Their ability to calculate and manipulate odds for something to happen far exceeds ours, that's different now. Yet we read snippets from people speculating, drawing conclusions based on, what exactly ?
The rational (but cynical) policy for the DoD/NATO is to provide just enough support to Ukraine to keep the war equal and let the Russians bleed themselves until all support for continued war is gone in their population.
In monteary terms, this is barely more expensive than maintainig enough of a weapons stockpile during peacetime to counter Russia. And at the end of the conflict, anti-war sentiment within Russia might reach Germany post WW2 levels.
By comparison, more overt support to Ukraine, enough for Ukraine to at least humiliate Russia and throw them all the way out of the country, would only serve to help spread the anti-West hate within Russia, and generate support for even more ambitious rearmament in the years that would follow.
Not even that. You might want it to win, not care about the outcome or not think that it will win without putting the US through a long depression, in all those cases it makes US officials reluctant to put themselves in such vulnerable position especially given the new rising world powers
Yeah, I mean, what else would you do with those chips? Run LLMs faster? Sorry but the killer app is still not there, the majority of people do not care and have no use for them.
There are a significant number of uses, it's just most of them involve generating slop for the purposes of fraud or feigning work.
I thought it was really damning how Apple's recent ads involved a celebrity using it to respond to their agent about a script proposal they didn't actually read.
I don't have papers but I think calculators, which are amazing and necessary, impact people's mental calculation capabilities. I know I used to he better at multiplication when i didn't have calculators all around me.
IMO, AI will have similar impact. Great advance overall but will nerf some areas of our mental capacity.
Like astronauts losing muscle mass faster in zero gravity.
AI impact on our brain, just like the calculator, might be a reasonable price to pay for the advancement it gives.
> I know I used to he better at multiplication when i didn't have calculators all around me.
My manual calculation skills were at their peak in seventh grade in school, when we were allowed to use a calculator for the first time (and thus the exercises got harder) but simultaneously I usually forgot to bring mine.
But yeah, I don't think I could think better in general, when I did arithmetic better.
He had AGI. He still has it. You can have it too, for a $20/month subscription from OpenAI or Anthropic.
It just turns out that AGI (artificial general intelligence, which ChatGPT objectively is), is not a singularity nor even a tipping point towards causing one.
How is ChatGPT „objectively“ AGI? You don’t think there are common intellectual tasks humans are better at than ChatGPT? Can ChatGPT do math better than a human without specific prompting to use a calculator (function calls)?
AGI means what it says on the tin: Artificial General Intelligence. As opposed to narrow AI, which most pre-transformer AI research was, that can only do specific categories of tasks. ChatGPT is a general-purpose NLP interface to a model which is able to be solve any category of problem that can be expressed through human language, which as far as we know is full generality.
What you are getting confused about is that there is a segment of people that have purposefully confused / conflated ASI (artificial SUPER intelligence) with AGI, due to their own beliefs about the hard-takeoff potential of AGI. This notably includes the founders of OpenAI and Anthropic, and early backers of OpenAI prior to the release of ChatGPT. Through their success they've set the narrative about AGI, but they are doing so by co-opting and redefining a term that has a quarter century of history behind it.
The core mistake started with Bostrom, although to his credit he is very careful to distinguish ASI from AGI. But he argued that once you had AGI (the ability to apply intelligence to solving any problem) you would rapidly get ASI through a process of rapid iterative design. Yudkowsky took those ideas a step further in his FOOM debate with Robin Hanson in 2008, in which he argued that the AGI -> ASI transition would be short, measured in months, weeks, days, or even mere hours. In 2022, six months prior to the release of ChatGPT, Yudkowsky has a public meltdown in which he asserted there is no time left and we're all going to die, after having been privately shown an early version of ChatGPT.
It's almost two years since the release of ChatGPT, which is absolutely AGI by the definition of the people who coined the term and have run the AGI Conference series for ~25 years, or by the definition of Bostrom or Yudkowsky for that matter. ChatGP is general intelligence, and it is artificial. Two years of AGI, yet we are still here and there is no ASI in sight. Yudkowsky was wrong.
Yet OpenAI, which was founded by a bunch of Yudkowsky acolytes, is still out there saying that "AGI" will bring about the singularity, because that was the core assumption underlying their founding and the reason for the investments they've received. They can get away with this without changing their message because they've subtly redefined "AGI" to mean ASI, and are hoping you don't notice the slight of hand.
I never understood AGI to mean "better than humans". A lot of people assumed it would easily be made to be, by simply throwing more silicon at it until it was, but being smarter than humans isn't what makes it "AGI".
Put this another way, suppose we create a computer program that is only as smart as a bottom 10% human (I'm not saying they have.) You can't reasonably say that is smarter than humans generally. But are you comfortable saying that bottom 10% humans lack any general intelligence at all? General intelligence doesn't mean extreme intelligence, and so Artificial General Intelligence doesn't either. You might say that the term is more than the sum of the parts, which is fair, but I still dispute that superhuman abilities was ever part of the definition of AGI. It was just a (thusfar) failed prediction about AGI.
By that token, you could find non-human animals that are smarter than some percentage of humanity in a few tasks. Are those animals AGI?
Now you could find a software that is smarter than some percentage of humanity in a few tasks. Is that software AGI? Is AlphaGo AGI? Is the Google Deep mind AI gamer AGI?
My definition and the one I found on Wikipedia is „AGI […] matches or surpasses human cognitive capabilities across a wide range of cognitive tasks.“. Being better that the bottom 10% of humans on some tasks doesn’t really qualify to me.
It's just anchoring. If you ask for 7tn, 500bn seems comparatively more reasonable. If you ask for 500bn in the first place you'll probably be lucky to get 100bn. Of course you sound like a twat in the process, but a price worth paying.
I think asking for trillions was pretty ridiculous but I don't think its out of the real of possibility of getting a few T from nation states run by monarchs and oligarchs. They have pools of dark money nobody can plumb the depths of, and they're looking for diversify.
Sure they would. The USA's military is given a yearly budget. They're spending that. The oil monarchies have socked away trillions from their countries.
Not only the capital, doing it well and doing it correctly is another huge mess, you can't just throw money at the problem and Sam's reputation is wearing thin, so the best of the best won't jump ship to join him anymore.
But maybe it was a successful marketing strategy to find investors? It was a moment in time to give people FOMO and nudge them into getting on the OpenAI train. Now it is more obvious that other models can catch up. Grok became competitive with top models within a month for example.
So somebody in another country with another language and another currency allegedly said… and came across several people and translations (did they translate currency?) before landing at a journalists inbox. Might as well as have heard about it on Twitter.
There’s a 68% chance that Altman met with someone who got their degree in the US (if only one executive attended, otherwise chances are higher).
And the others probably also have little difficulty communicating in English or calculating the cost of 36 fabs in any currency, since that’s their business.
I very much doubt the CEO of TSMC spoke to the NYT about a private meeting they had with a potential customer. Give me a break!
Much more likely they talked about 3-6 fabs, some other person laughed about 36 fabs for podcast guy, someone else overheard that and posted on Weibo. That made it to NYT eventually.
Asking for more than you want only works if your ask is too large but still somewhere in the realm of feasibility so the other party doesn't write you off as a crazy person and end the conversation right there.
If you're applying for a job as a fry chef at a burger joint, asking for $25/hr is a good idea. Asking for $1000/hr will convince the shop owner that you are a crazy person who isn't serious about wanting the job.
My guess is that since they are so sure of AGI being achievable soon, that they think they can squeeze out any amount of money they want out of investors.
After all, AGI would be worth way more thab 7T right?
I’m not your bro. The thing that’s absolutely wrong (and quite frankly inscrutable) about people who think like you is that you all believe everyone is money hungry like yourselves and you are utterly incapable of imagining anyone whose life doesn’t revolve around wealth.
Most people aren’t envious of “crypto bros” nor are they mad they didn’t “get rich” in a zero-sum scam most lost money on. That’s how you would feel. Fortunately the world is more diverse than that and there are people with healthier priorities.
Hey you make great points, but you assumed some things incorrectly.
1) I never said people were envious of "crypto bros".
2) I never said I was rich or cared for money. My main priority is family.
I think you might have some issues about late stage capitalism, but for that I do salute you.
Ill repeat what I was trying to say so its clearer.
I think there is potential in AI , and Bitcoin was a fantastic piece of software. I am confused why people on HN had so much trouble adopting these technologies.
yeah AI and Bitcoin have shortfalls in their design, but Bitcoin is used by alot of people now, and so is AI.
Those are my points.
I don’t know you, so I won’t make assumptions. Free yourself from early judgments; often, those you clash with share more in common with you than you might realize bro.
and fake it before you make it ...im still waiting for GPT's version of H.E.R.... many months later nowhere to be found yet no doubt that juiced their revenue.
The startup playbook lie ur ass off and make sh!t up. Musk still does it to this day with the recent demo of his Optimus robots implying they were all AI driven.
why that's what many start-uppers do to win the startup game ..fake it .. lie. Would you trust your life to Elon's self driving tech which you sign away your rights to sue lol
Some with deadly consequences like Uber's attempt at self driving and what was that other recent self driving company that ran over / mangled a pedestrian?
Yes but in this case it was clearly apparent to most and to those who spoke with the robot and believed it must have felt like morons for being filmed thinking they were real. I just felt the benefits weren’t there as it was an obvious fakery.
Tbh when they had the real person on the suit I wouldn’t have approved that it would give the impression I’m ok with pretending to an extent.
I understand having a half working prototype iPhone on stage but literally pretending the robots are real just felt dishonest and a bad idea.
> Yes but in this case it was clearly apparent to most and to those who spoke with the robot and believed it must have felt like morons for being filmed thinking they were real.
If you spoke with the robot and believed it was AI controlled you should feel like a moron, considering the robot explicitly stated that it was remote controlled in that demo. https://x.com/zhen9436/status/1844773471240294651/
I'm not sure why you don't seem to understand my point or what seems pretty clear to me what happened that night. Here are 2 points.
1) "that" demo? I never mentioned "a" demo. That one clip, which you found, the robot answers a specific question to one user.
There were hundreds of interactions that night, and in many that I saw atleast one person clearly didn't realise they were.
2) Regardless of whether it was their fault for being morons, my point is they FELT like morons, which TO ME is not good publicity.
I like Elon and I think the demos were incredibly, I am saying it weirded me out they were trying to pretend the robots were sentient, which I am sure they are not far away from. I just think it was short sighted. But maybe I give people too much credit.
I would 100% have not told anyone to pretend shit. It wasnt even required. It just made me feel uneasy about the products.
Not really. The plans to build a foundry have been dropped. It's more a cautionary tale that at some point you will end up in an environment that's closer to reality where your bullshit is a liability. It was reported that the TSMC people called him a 'podcasting bro'. In other words, they lost all respect for anything he was saying.
>he was floating the idea of somehow raising seven trillion dollars
when i read that news back then, i read it as "billions" and thought "that sounds very ambitious, hardly he'll get that money". Man, i'm 3 orders of magnitude behind the current state of the art in tech, and need to start thinking in trillions (while it was only recently when a billion dollars was large money).
Along with the grandiose promotional nutty-ness of "You need to support my company because countries will fight wars over AI", whatever that FOMO scarcity is supposed to mean in practice.
Sam may be talented in other areas but HW, chips, and infrastructure is obviously not his forte and he should probably delegate discussion on this topic to someone who knows what they are talking about.
It could have been strategy instead of insanity. By starting conversations at $7T you anchor high and potentially drive a greater outcome than starting an order of magnitude lower.
That strategy only works if the anchor is in the realm of reality. If I'm selling a 20 year old Toyota Corolla and initially ask for $900,000 that's not going to help me get a higher price.
Anchoring bias probably works with numbers outside of the realm of reality as well. But I doubt it's very useful in these cases (or even any stronger than asking for say 300B), otherwise everyone would always start a funding round by asking for a 7T investment, right.
in a negotiation where you have experts with good intuition for costs and practicalities on one side of the table and a non-technical failing-upwards-megalomaniac-kid on the other side, i doubt that's a sensible strategy
Usually when doing anchoring you want to end up at a result less than what you originally asked for but crucially more than zero, and OpenAI immediately folded on building any fabs whatsoever, so I don't think it worked.
When you do something that stupid - starting at $7 trillion - you end the conversation before it really begins because you lose all credibility with the people that matter (eg TSMC and other investors).
If he had said $250 billion and six fabs, it would have been a lot to ask but people wouldn't think he was ignorant or irrational for saying it. Big tech for example has that kind of money to throw around spread out across a decade if the investment is a truly great opportunity.
I guess he thinks his glorified markov chain will lead to ASI if scaled up sufficiently. Even if we get ASI, the likelihood that anybody will ever make any money from it is so delusional. This isn't going to be your average brainwashed peasant, crushing these capitalist pigs is probably the first thing it's gonna do.
I guess a valid criticism is the idea that ASI would care about the ideologies of literal ants, or that it would be so alien that it doesn't even make sense to anthropomorphize it like that. But I guess the delusional part you got offended about was either the criticism of capitalism or to describe LLMs as glorified markov chains (which they technically are, look it up).
The technical term for this is “zone of possible agreement”. If your opening is outside the counterparty’s ZOPA, then they’re likely to disengage/exit.
> The TSMC execs reported more or less laughed in his face when he brought it up it them.
Reminds me of:
Musk flew there with Cantrell, prepared to purchase three ICBMs for $21 million. But to Musk's disappointment, the Russians now claimed that they wanted $21 million for each rocket, and then taunted the future SpaceX founder. As Cantrell recounted to Esquire: “They said, 'Oh, little boy, you don't have the money?”
You think summoning machine god would not be cost competitive? God as in smarter than all of humanity combined times infinity (or at least a very large number).
Earning back the 7T with god/satan on your side could be trivial and at the same time the least of your worries for other reasons (maybe god doesn't like you and/or doesn't care about you).
This is crazy. How can you look at the current state of technology and think that something even remotely close to “smarter than all of humanity times infinity” is near?
Tech paradigm shifts will definitely happen and improve current AI by a lot, but if creating “god” is the goal, 7T would be better spent creating a mega transmitter and asking aliens to just give us the tech.
> God as in smarter than all of humanity combined times infinity (or at least a very large number).
This is delusional. Nobody has demonstrated AI that can even match the output that human creativity has accomplished in the past 5 years. It will take an awful long time before it's up there alongside humans, and by the time it surpasses us we'll both be long dead.
If you're not just being performative for a joke here, then I don't even know how I can explain how wrong you are. All of this - the promises of "AGI", the bajillion-dollar meme order sent to TSMC, the cryptic sama tweets in lowercase - it's performance art. It's unsubstantiated, nondilineated, unproven, unstudied nonsense. That's it. I'm sorry if I hurt your feelings, the crypto bros had to go through this a few months ago and it was tough for them too. But the dividends have to be paid, and in a few years time your "$7 trillion dollar" intellectual property empire will be sand.
"Nobody has demonstrated it (publicly), so it cannot exist anytime soon" is not a very good argument IMO.
Your post reads a bit like that nobel prize winner who in the 90's said the internet would become similar marginal technology as fax machines. Just because nobody had demonstrated any revolutionairy use case for it at the time.
We went from barely having computers and no significant internet to what we have today in 30 years.
"We'll be long dead", yeah maybe if we don't die of old age.
Some people (like yourselves) are so delusional that you are capable of considering wasting 7T$ on something that is a glorified autocomplete technology. We kind of a need a reset in the tech industry to get rid of mindsets like yours. AGI is still so far away and when the AI bubble eventually bursts, you will probably still try to convince yourself it still only needs 7T$. The state of people in the tech industry is sad.
> Some people (like yourselves) are so delusional that you are capable of considering wasting 7T$ on something that is a glorified autocomplete technology.
You know that to many, you are the delusional one if you think that some fictional number in a screen is more important than this glorified autocomplete technology that has the potential to revolutionize humanity as much as agriculture, electricity and the internet.
If you don't believe ASI is possible, 7T is way too high. If you believe ASI is possible, 7T is wayyy too low, thus 7T is a compromise if you believe there's a non-certain chance of ASI being possible
So the question is, if you have AGI/ASI how much productive work could you get done with it?
When looking at things like mechanization and the productivity increases from around 1880 to now, you took an economy from around 10 billion a year to 14 trillion. This involved mechanization and digitization. We live in a world that someone from 1880 really couldn't imagine.
What I don't know how to answer (or at least search properly) is how much investment this took over that 150 year period. I'm going to assume it's vastly more than $7 trillion. If $7 trillion in investment and manufacturing allowed us to produce human level+ AI then the economic benefits of this would be in the tend to hundreds of trillions.
Now, this isn't saying it would be good for me and you personally, but the capability to apply intelligence to problems and provide solutions would grow very rapidly and dramatically change the world we live in now.
> If $7 trillion in investment and manufacturing allowed us to produce human level+ AI then the economic benefits of this would be in the tend to hundreds of trillions.
Sure, but right now that "if" is trying to do 7 trillion dollars of unsubstantiated heavy lifting. I might be able to start creating Iron Man-style arc reactors in a cave with a box of scraps and all I ask is 1 trillion, so you all should invest given how much money unlimited free energy is worth.
How do you even define ASI? How do you know it will run on silicon? If it will run on silicon, what kind of silicon? GPU? TPU? Binary? Ternary? Von-Neumann architecture? How can you even start to build chips for something completely imaginary?
ASI is just AGI but scaled to the point of superhuman ability in all cognitive tasks. Surely it will run on the architectures used to build AGI before ASI takes control itself in the design and deployment.
> ASI is just AGI but scaled [...] Surely it will run on the architectures used to build AGI
The words "just" and "surely" are doing so much heavy lifting here that I'm worried about labor-laws. :p
If scaling up was "just" that easy and the next step would "surely" begin in the exact same architecture... Well, you've gotta ask why several billion general-intelligence units made from the finest possible nanotechnology haven't already merged into a super-intelligence. Or whether it happened, and we just didn't notice.
>>OpenAI launched o1 — codenamed Strawberry — on Thursday night, with all the excitement of a dentist’s appointment. Across a series of tweets, Sam Altman described o1 as OpenAi’s “most capable and aligned models yet.” Though he conceded that o1 was “still flawed, still limited, and it still seems more impressive on first use than it does after you spend more time with it,” he promised it would deliver more accurate results when performing the kinds of activities where there is a definitive right answer — like coding, math problems, or answering science questions.
it seems to me like they nerfed the regular free ai model to give me imperfect replies on my benchmark consistently while the o1 model nails it consistently. before the free model would, depending on the day, give me more accurate replies to my benchmark .
thus widening the divide to make the o1 model look more useful. openai can do whatever they want and thats fine but all it seems to me they did was make 01 look good by comparison by nerfing the free product.
> it seems to me like they nerfed the regular free ai model ...
My main takeaway from the article was more about the unsustainable business model due to a massive cash burn rate. To wit:
OpenAI’s models and products — and we'll get into their
utility in a bit — are deeply unprofitable to operate, with
the Information reporting that OpenAI is paying Microsoft an
estimated $4 billion in 2024 to power ChatGPT and its
underlying models, and that's with Microsoft giving it a
discounted $1.30-per-GPU-an-hour cost, as opposed to the
regular $3.40 to $4 that other customers pay. This means
that OpenAI would likely be burning more like $6 billion a
year on server costs if it wasn’t so deeply wedded to
Microsoft — and that's before you get into costs like
staffing ($1.5 billion a year), and, as I've discussed,
training costs that are currently $3 billion for the year
and will almost certainly increase.
Whether or not the "free model" is better or worse than the "o1 model" is moot IMHO.
AMD hardware is good. The software is getting better daily. Training still sucks (mostly due to unoptimized libraries), but inference is looking pretty decent with recent advances in tuning...
> AMD hardware is good. The software is getting better daily.
Which is the story of AMD for the last ~15 years.
getting support so we could develop apps for their graphics card was dispiriting. at the time they were faster and very much cheaper than nvidia (but no cuda) But every time we found an issue, the poor devs who were contracted to fix it were left struggling.
Its the same where I am now. We were trying to qualify some motherboard/TPM/other issue, but they didn't have enough bandwidth to help us in time.
Its better now, we do have some AMD stuff in the fleet.
At least from someone who is on the edge of the business and seeing what is going on. It is a lot better now, especially at the enterprise level. They've been hiring and buying companies too.
It won't be fixed over night, but it is definitely a renewed focus.
They could also end up running out of money before they get it right, or another company might come out with a chip that is just a bit better and cheaper a year before them.
Apple didn't build their M1 chip from scratch. By the time the M1 chip was released they had already been building chips for the iPhone and iPad for several years!
So presumably they're building their own data centers to use these chips, but odd that the article doesn't actually say so.
Are any of the western AI/matmul chips NOT being fabbed by TSMC? Amazon or Meta's perhaps? We've got NVIDIA, AMD, Google (TPU - also designed by Broadcom), and now OpenAI all using them. Are Samsung not competitive?
I wonder to what extent total chip/chipset volume is limited by TSMC capacity as opposed to memory or packaging?
Samsung is not competitive so everything good has to be made by TSMC now. It has been reported that packaging and HBM are the bottlenecks for AI chips although TSMC is also running at high utilization.
I expect they wanted to go down the fab and chip design path to build a moat, as they've realised they really don't have one outside of brand recognition.
If they had proprietary fabs and chips that were magically superior, and they could build them for cheap rather than paying the team green tax, they'd have a huge advantage in inference cost.
Perhaps they've realised just starting with a proprietary chip is enough?
I keep thinking the transistor is being wasted in AI. We use binary states to remove entropy from the system. In AI, however, much more entropy is acceptable. It seems that we should be able to do a lot better than NAND gates for this use case.
The other reason for binary was automating synthesis vs hand-drawn circuits. That’s really important at scale. It both reduces NRE costs and increases velocity.
Ditching it might mean ditching the tooling that makes it so effective.
Other than GH Copilot (which is amazing) and the like, I am yet to see a social/cultural impact of LLMs. I am ignoring the hype cycle fueled social media posts where influencers show cool stuff or model releases. I am talking about this stuff permeating through the society.. am I immune to it? Or is it hidden in saas/enterprise products layers deep?
I would imagine some productive cultural impact that’s wide spread, sticky, high retention not like ‘oh cool 3d tv or vr’ kinda thing?
Given how much ‘money’ has been burnt through the GPU heatsinks.
It's the generation currently in schools and universities that are making LLMs a part of their daily routine.
Apart from people in education, a whole SEO industry has grown around LLMs and they love the newly found powers to produce spam at scale.
In business / government space people are cautiously introducing LLMs to improve productivity in the comms area where slight hallucinations or robotic corporate language are acceptable, because the final text is still produced by a human. On the customer support / sales side companies are investing money in "AI" versions of the good old phone menu trees that inevitably make you scream "I want to speak to a human" before you get connected to a human being.
Elsewhere adoption is slow, because we expect deterministic results and these are just not to be expected. I write software that interacts with LLMs via APIs and have to implement defensive coding techniques I haven't used for over a decade, because the output may switch from the requested structure at random. This makes use of structured output or function calling problematic.
It’s kinda easy to check it though. Ask your friends if they use it, preferably non developer. When I find out my accounting friends and marketing friends are using it for basic analytics/promotion write ups I realized this is being used by non developers already on a regular basis. The use by non tech folks is significant for this . Crypto as a counter example of hype hasn’t lived up to its purported use case as non tech ppl I know only “own” it through an exchange.
I did. Most of them if not all have tried it once just out of curiosity. None of them is using it on a regular basis- not even once a week. Maybe I am in the wrong group? Includes family / friends (non tech). Tech people (friends) use it regularly for coding though.
People around me and myself use it to basically replace some web searches.
Particularly o1-preview is really good at writing broad strokes mini review style articles that discuss some question you're interested in.
It only works in cases where there's a lot of text out there already that discusses the topic and that needs to be sort of remixed to answer your question specifically.
The chat format is much more efficient for this than a web search or reading a few related Wikipedia articles.
It's truly glass half full vs. glass half empty. I'm super impressed and excited about how useful this is, and yet it clearly isn't AGI.
The people I'm referring to are not working in software if that's what you're asking, but it's definitely knowledge work.
It seems pretty clear to me that this is disruptive to most knowledge worker's workflows, even without getting into the whole discussion of how close are we to AGI and does this thing really understand anything.
Oddly , i think the media has not caught upon how much it is used. Most journalists and youtubers present trivial use cases. I ve found that the most unlikely people use it for all kinds of purposes. In most cases, they don't even want their bosses to know that they use the tools because they give them superpowers. I 'm talking about bank employees, store managers, local politicians, small business people etc.
I also used to be in disbelief, but i'm seeing more and more unexpected people use LLMs , and they even figure out how to prompt them right.
A highly advanced content retrieval system (but lacking in accuracy, so ok for situations where that may be ok).
I wouldn't say it's revolutionary -- certainly not as revolutionary as search, but it's a major step forward in being able to retrieve information. Ultimately will allow you to interact with computers the way you see in the old Star Trek movies, where you're speaking in plain English and receiving information back in the same (providing the systems which are the source of the needed information are interconnected and fed into the training set).
Spending $7 trillion on in-house fabs sounded both ambitious and crazy. Reality finally kicked in. If they’re done dreaming big, let’s hope they keep the quality
> OpenAI considered building everything in-house and raising capital for an expensive plan to build a network of factories known as "foundries" for chip manufacturing. The company has dropped the ambitious foundry plans for now due to the costs and time needed to build a network
So they planned to transform everything into paperclip factories?
> OpenAI considered building everything in-house and raising capital for an expensive plan to build a network of factories known as "foundries" for chip manufacturing.
... Wait, what, _really_? _Why_? I mean, it'd probably take about a decade to get going.
Broadcom is a vendor. They provide the main skeleton of a chip so that the client only needs to focus on the product differentiator, which is the compute part of the chip. They can worry less about other parts that are not their core competency.
I'm not saying that I necessarily agree, but the general consensus on HN seems to be that Broadcom is now less of a tech company and more of a holding company that raids others, for example VMWare, and extracts all value to the detriment of customers and the acquired company.
I don't think that's completely wrong, but it's a big company and I'm sure there are some better areas of the company than others.
I think it's completely wrong in the context of its role as an ASIC partner. There's a very short list of companies with all the IP needed for these cutting-edge ASICs and Broadcom/Avago might be the best of them. And to be clear, they've developed that IP themselves, just as they've always done. Those that think they're just a "holding company" haven't actually worked with them.
Additionally, having worked with some of their network devices at the driver level, they seem to be kludge piled on top of hack poured over a soup of workarounds for hardware bugs. Maybe they've gotten better recently, but just looking at their drivers, it didn't paint a great picture.
this makes sense since the general consensus on HN is that any general consensus on HN is right so if the general consensus on HN that any general consensus on HN is right is wrong then any general consensus on HN could be wrong
They make every kind of chip for networking: WiFi, Bluetooth, home routers, cable modems, fiber, switches, NICs, DPUs, etc. And that's 1/Nth of their business.
Also, did you know that Google's TPU efforts previously also relied on help from Broadcom? https://www.theregister.com/AMP/2023/09/22/google_broadcom_t... It's absolutely unsurprising that former Google employees would bring this vendor relationship into their new company, given Broadcom did a good job helping Google with TPUs.
reply