> Some investors have questioned such deals because companies like Google and Amazon are investing money that ends up bolstering their own revenues.
This is brilliant. It's a win-win for cloud providers like Microsoft and Amazon to invest billions into AI companies because the "investment" is guaranteed to flow back in as cloud revenue (on which they have huge valuation multiples) regardless of the AI company's fate or eventual profitability.
The AI companies then get to quote massive valuations based on these investments, even though the investments were made under very unique circumstances (ie "most of this investment will flow back to us as revenue").
Finally, other investors[1] who do not benefit from the cloud spending and who may be less discerning on price will FOMO in at whatever the current valuation is. These investments not only have no stipulations about how the funds are used, but often aren't used for the business at all - instead directly paying out founders/employees.
It's a brilliant triple whammy - the technology itself is super impressive, cloud providers are naturally motivated to pump up baseline valuations due to the symbiotic relationship of exclusive cloud spend, and these valuations are taken at face value by non-cloud-provider investors eager to get in on AI hype at whatever the prevailing price is.
"from a company's perspective it's like taking out of your left pocket and putting it in your right pocket ... I mean a big skeptic would say you're using your balance sheet to to drive your your income statement which you know should be a no-no"
It's very similar to the wash-trading pump & dumps common in the crypto space. Feed money back to yourself at inflated prices to make the line go up, convince dumber money to enter the market based on the line going up, cash out at their expense. Somehow it's illegal when a crypto company does it but not when a major tech executive does it.
Wash trading generates a fake image. Everyone knows this is happening. They are saying: we invested at this price, these companies are buying our stuff.
Your assumption is: the people making these investments are so clever...I mean, it looks really stupid...but they are so clever, they must be doing this to take advantage of other people.
Reality: the people making these investments are really that stupid.
Reality: they aren't creating money out of thin air, they are investing money into their customers...fine, but the bet is that their customers eventually make money (spoiler: they won't, every technological innovation has been an incinerator for cash and AI is no different).
(This should be obvious too, Microsoft and Google are doing this for defensive reasons...it is like a middle-aged man buying a low spec Porsche...no, his life isn't going okay now that he has a Porsche, he is just going to be miserable but inside a different car).
> they won't, every technological innovation has been an incinerator for cash
this sounds like you're saying that nobody has ever made huge profits off of technological innovations which seems to be heavily contradicted by the fact that I did not have to roll my stone wheel using a stick from my cave to work this morning.
Well said. Investing in customers is the opposite of wash trading: the goal is to create value.
And it’s probably better to look at it like a VC fund: the expectation is that most customers will fail, but if the next Adobe or Facebook or whatever comes out of the exercise and is heavily technically dependent on your cloud / AI infra, that covers a lot of lost investments.
If I take $10 and give it to you for 10% of your company and you give it back to me to rent a room in my building I now own 10% of your company for $0 outlay and you get to use the room in my building for your business, and in these arrangement the room is rented at cost. Everyone wins in this arrangement. This isn’t stupid, this isn’t offloading capital into income. This is an efficient way to maximize my investment by effectively using existing infrastructure spend and boosting my investments cash flow position by reducing their infrastructure spend.
Normally when you rent a room for $10, it implies that the room is worth at least $10 to the tenant, and the opportunity cost to the landlord was < $10, otherwise the transaction wouldn't have happened. And similarly, if someone invests $10 for 10% of a company at $100 pre, they're assuming that the discounted cash flows of the company are worth at least $100. People who analyze financial statements and trade in markets assume this to be true - it's fundamental to the price discovery mechanism.
But if the actual cost to the tenant was $0, the value of the service might have been any number greater than $0. And similarly, if all the money in the investment comes back to the investor as revenue, the actual cost of their $10 investment wasn't $10, it was $0. You can't make inferences about what they actually value these services and investments at, because the prices were not set by a free competitive market, they were set by companies colluding (or cooperating, depending on framing) for mutual benefit.
We saw the same issue with all the food-delivery startups that were offering 6 free meals - they booked the revenue and used it to justify outlandish valuations, and they booked the promotion as a marketing expense, but the root problem is that consumers wouldn't buy the product at list price, they were only buying it because of the investor-funded promotion. Or with Groupon - it was economically unviable for the businesses that participated, but they wouldn't find that out until after their revenue could be used to justify the next funding round and cash out in an IPO.
Wash-trading is just the limiting case of this, where it's apparent to everyone that the economic value of the transaction is zero because there is no actual product being traded, but a self-dealing transaction is used to set a price that other participants in the market anchor off of.
The tricky part is that sometimes this tactic can be economically viable, and offering discounted or financed products is a way to get consumers to get over the activation energy of trusting a new product, and once they've converted they become regular customers of the normal product offering and go on to spend a lot more. This is the point of a sale, or a rebate, or a promotion, or a financing deal. Which I guess answers my original question about why it's legal here but not for crypto wash-trades: it can sometimes result in true economic value being provided, and it's difficult to prove malice instead of incompetence.
But the caution being offered in this thread is to not take revenue figures or valuations of AI startups or cloud providers at face value, lest you be the one who is incompetent.
I’m not sure I follow your logic. There is a market for, say, cloud computing services. If it helps you could see it as an exchange of cloud computing services for equity, but the reality is the cash invested likely was not 1:1 exchanged to cloud computing and they had optionality. That’s somewhat irrelevant, but it’s still a real truth. Any money used to purchase cloud services has real expenses and capital allocations associated, real economic value for the company who exchanged equity, and the investment still sits as an asset and liability. It’s hard to argue an AI model company derives no value from GPU hours, or Amazon derives no value from owning equity in and locking an AI model company into their platform and business relationships. This isn’t a wash trade, which literally has no economic value, and it isn’t fake revenues as there is a real exchange of economic value for both parties. The fact the exchange involves at cost services isn’t nothing - it magnifies the investment as otherwise they would have to purchase the services at a higher cost. The idea that reserved capital equipment and supporting services have no material cost or value to Amazon is on the surface false - they have a very real cost, both opportunity and capital, especially in a time of high end GPU shortage. The idea that reserved high end GPU at cost has no value for an AI company is absurd. The fact it was done as cash gives flexibility in how the investment is used, and the fact it counts as a tiny amount of aws revenues is not materially relevant to anyone.
These investments don't seem either dishonest or stupid.
Only a fool would buy compute capacity from Google (at least without a special deal), and Google needs apparent wins to sell GCE. They're paying with compute capacity for advertising. They're paying a lot less than they would otherwise be, since costs don't equal sticker price. Right now, AWS has 1/3 market share and Microsoft 1/4. Google claims 1/10th, but that's only by gaming numbers e.g. by including Google Workspace into figures. Now, they've got a massive amount of $$$ to keep those figures pumped up to maintain that 10% number.
Amazon wants a bigger moat, or at least doesn't want Azure eating its lunch in the near future due to a lack of a reasonable AI service (MS is leading here, due to the deal with OpenAI). It needs a reasonable AI service on AWS. It is nowhere close to being able to make one themselves. It's giving money so it can have one, and have one from a company it's part-owner of.
Thing is, both of these investments make absolute sense even without financial games, antitrust, incompetence, or much of anything else.
As a footnote, if I were Amazon, I'd be looking at player like Groq. That could leapfrog Azure + OpenAI for many uses.
A realist would say the money is spent somewhere and better on your platform. Once infrastructure is built it’s hard to offload, so the margin comes from future spend. But revenues are still revenues, and the capital spend is invested in capital growth to support the load from the investment.
It’s a good idea for large companies to use their profit to stay profitable, and if it supports their cloud business…
Where it’s a No-No is in accounting: those should probably not be counted as “sales” but as gifts in nature or some particular category. Bill Gurley and Brad Gerstner are investors, so naturally, they balk when they hear that the documents they read, financial statements, have a creative take on reality. Still, I don’t think they think that lowers Google’s overall value. They just want that in a separate column.
That's what caused the 2002 crash. 99% of the revenues for internet companies came from other internet companies. When the easy money started drying up, revenues collapsed.
99%? Let's not forget the hardware suppliers (e.g. Sun, Cisco, and so many others) who sold hardware to startups and loaned them the money to buy said hardware. They then had hardware revenue to tell shareholders about as well as interest revenue and the NPV of the loan on their balance sheets.
What could possibly go wrong? We all know...and it did.
Indeed it is and that approach nuked both GM and GE. As bombcar noted in a parallel comment though, it was even worse in the Internet crash.
I suppose in today's Bayes-obsessed world another lesson is: don't finance your customers' purchases if your company name is a two-letter acronym starting with G.
GE’s decline wasn’t because they offered financial services. It's a textbook case of mismanagement of an overly complex business.
Tons of companies offer financing for products they manufacture just like car companies have done so since the 1920s — more than *100 years! — and in many cases longer than that. It’s not new. It’s not unusual. What’s different about this? The subject matter? The cloud-as-a-service component? How is it really different from support contracts? It’s really not.
Sitting around trying to make it sound like a manufacturer offering financing is some deep conspiracy is crazy talk, no matter how many times I get downvoted for saying it.
(Downvoting on hacker news is supposed to be used for deemphasizing poor comments not downvoting people you don’t agree with btw. This isn’t Reddit. The idea is to encourage high quality discussions not win popularity contests and we should endeavor to keep this community differentiated.)
I would love to see a coherent argument as to what’s so massively unique about providers financing their own equipment to their channels. I just don’t see it as being anything different from standard. Neither is investing in the most promising companies in your ecosystem unusual for Norge companies. These aren’t unusual things: they are well trodden paths used by many kinds of companies.
It's the dose that makes the poison. Drinking too much water will kill you by messing up your electrolyte balance, yet nobody says drinking water is unusual.
There were indeed plenty of missteps, but the imbalance due to GE Capital was the wrecking ball for the company.
> The effective end of GE Capital is the end of an important era for the former conglomerate.
> The company has been trying to become leaner and more profitable in recent years, selling off its appliance business in 2016, its NBC Universal entertainment unit in 2011 and its light bulb business in 2020, among other divisions.
> And while those businesses had a higher public profile than GE Capital, no business was more important to the conglomerate during its heyday than GE’s finance arm. It provided financing on many of its industrial products, such as jet engines and electric power plants, as well as a key source of financing for small business and consumers. It even became a major player in the subprime mortgage business.
> But when the housing bubble burst and the subsequent meltdown in financial markets in 2008, no part of the GE business was wounded more than GE Capital, and what had once been a key driver of the company’s success became an albatross...
Or "How decades of bad decisions broke GE" which sounds like your basic thesis (and I agree there were many problems) but describes GE Capital as "the finance company that dealt GE a near-fatal blow during the 2008 crisis." https://money.cnn.com/2017/11/20/investing/general-electric-...
GMAC / Ally Bank likewise distorted GM, but the example of one company is enough for now.
But to generalize GE as a case study is still incorrect. They are an atypical example.
We can find any metaphor we want, cite one or two example companies like GE (and it was bad company management not the fact that they were in finance that sank them IMO — I was a customer of theirs and they were so poorly run), but it won’t make up for more than a century of the practice working across most major industries without problems. I’ve read several books about the GE failure and am widely aware of the problems, but disagree with the mainstream analysis. Nobody wants to pin it on the management but they deserve the criticism. Read * Lights Out: Pride, Delusion, and the Fall of General Electric* if you haven’t — it’s worth the time.
Don’t you agree that narrowing down the argument that financing your own product is bad because GE took themselves out with a finance division becoming too critical to their business and then blowing up their business with poor management choices to be overly specific as an example?
Jeez, I feel like you’re arguing a point to make a point that nobody is contesting.
During the internet boom people accepted and even celebrated the vendor financing because it is a common mechanism without bothering to pay attention that the recipients were themselves not solvent. Sun, Cisco, Nortel and so many others were using this mechanism to juke the stats.
And I gave the GE and GM examples to show that it the same sort of thing can happen with major corporations too.
But nobody is claiming that it’s 100% inherently bad. It’s just you trying to refute this non-position.
Certainly not my intention to waste time or focus on issues that aren’t relevant like your insinuating. Let’s leave it here and keep it polite as I respect your point and appreciate the real effort to talk about it with me. I don’t want you to feel that way. :(
It may be brilliant, but in my book that sure sounds like it should be illegal and likely regulated as anticompetitive behavior.
I expect it isn't actually illegal today, they pay plenty of expensive lawyers to cover their ass, but I don't think a majority of the public would want this to be legal if they understood what was being done.
But, who is it anti-competitive from or to? I wonder if there’s evidence out there that the cloud providers are selectively investing in AI companies that use their clouds. That would be a little sketchy I guess. But if Amazon, Microsoft, and Google all pump up AI companies generally with the expectation that it’ll generally benefit their sector, that seems… fine, I guess? Or at least not anti-competitive.
It'd be anticompetitive to any newcomers in the hosting space, wouldn't it?
Fly.io recently launched a GPU product. If they can't afford to front $10B to OpenAI they're locked out of the market.
That also doesn't get to a potential money laundering issue. Again I assume it isn't technically illegal accounting, but Microsoft sure seems to have functionally moved money from one side of the books to the other, delaying and reducing potential taxes they would end up paying.
> It'd be anticompetitive to any newcomers in the hosting space, wouldn't it?
> Fly.io recently launched a GPU product. If they can't afford to front $10B to OpenAI they're locked out of the market.
That seems like the first scenario I described. If MS was investing in OpenAI contingent on their using MS’s cloud, then yeah it seems obviously anticompetitive, right?
In this scenario, if the investment is contingent on OpenAI using MS’s cloud, that seems just blatantly anticompetitive.
If it is not, I don’t see any problem.
If MS selectively invests in companies that are already using their cloud services and hopes they stay by inertia… I dunno, seems grey.
> That seems like the first scenario I described. If MS was investing in OpenAI contingent on their using MS’s cloud, then yeah it seems obviously anticompetitive, right?
My understanding is that this is precisely how it was structured, that most of the investment was effectively in MS Cloud credits. If that is inaccurate then I'm not as strong on the anticompetitive behavior being an issue.
If Microsoft's investment came with strings attached pushing OpenAI to use Azure, how is that not functionally anticompetitive behavior (even if they found some legal gymnastics to keep it technically legal)?
But nothing stopping smaller guy to sweeten the deal no? It is the same thing why new entrants tend to offer lower prices and stuff like that. Nobody argues that new entrant to some market should offer something in order to compete with the established giant. Established giant cannot hire goons or suppress the competitor directly, like - "if you use the competitor, then you will lose access to our big store".
So, in this example, you don't see any problem with a market leader offering billions in investment earmarked for use in buying that market leader's products? Assuming that there is some level of economies of scale with regards to cloud computing, how would a new entrant to the market compete?
And is this not anticompetitive only because they used a massive pile of cash to corner the market? If so, why are mergers and acquisitions reviewed at all?
With regards to the accounting of it, are you okay with Microsoft being able to use this as a way to offset billions in profits today and effectively amortize those gains over a few years as OpenAI spends the cash on Azure hosting?
Forget the legal technicalities for a moment, would you as a citizen be okay with Microsoft being able to act this way? Do we want massive corporations able to block markets off as long as they can afford it, and hide away profits to avoid paying taxes?
> If so, why are mergers and acquisitions reviewed at all?
they are reviewed to check if it is lessening of the competition. Like if some new grassroots company decides to offer new OS, Microsoft probably won't be able to go and buy it. It also cannot pay other companies not to use this OS.
> Forget the legal technicalities for a moment, would you as a citizen be okay with Microsoft being able to act this way? Do we want massive corporations able to block markets off as long as they can afford it, and hide away profits to avoid paying taxes?
Acting what way? It is like asking if you are okay if somebody can afford something, you can't.
> It also cannot pay other companies not to use this OS.
Oh I'd assume they could pull that never if they chose to, at least in certain situations.
Just take the OpenAI / Microsoft deal as an example. If a $10B investment into a company comes with strings attached, say with board seats and an exclusivity deal to use Azure products, they just blocked out any other hosting service that could compete. If that is legally questionable but they can invest most of the capital as credits for Azure products, they've effectively done the same thing without it clearly being coercion.
For a new OS, Microsoft this spent decades building an ecosystem and marketshare in the enterprise space. MS could pretty easily make sure none of their services work on the new OS. They wouldn't be fully blocking the new OS out but it would make it an extremely difficult challenge.
> Acting what way? It is like asking if you are okay if somebody can afford something, you can't.
My understanding is that part of the OpenAI investment was in Azure credits. That may technically be legal, but it blocks any competition in the space and defers paying taxes on the value of those credits.
If MS simply made the investment and OpenAI chose to use Azure of their own accord, there's absolutely nothing wrong with that. If MS strong arms OpenAI into using Azure through a combination of strings to the funding and/or board seats, I strongly disagree with that.
> Just take the OpenAI / Microsoft deal as an example. If a $10B investment into a company comes with strings attached, say with board seats and an exclusivity deal to use Azure products, they just blocked out any other hosting service that could compete. If that is legally questionable but they can invest most of the capital as credits for Azure products, they've effectively done the same thing without it clearly being coercion.
Microsoft invested in OpenAI and OpenAI has become a huge success. That's all to it. They offered them a deal with azure credits, profits etc. and OpenAI succeeded. They could have failed and doubt anybody would be concerned about MSFT's deal with OpenAI...It is not like Microsoft forced OpenAI's hand there - they could have gone to Amazon.
> For a new OS, Microsoft this spent decades building an ecosystem and marketshare in the enterprise space. MS could pretty easily make sure none of their services work on the new OS. They wouldn't be fully blocking the new OS out but it would make it an extremely difficult challenge.
They could but they won't due to their high market share and public outrage. That's the whole point. If Microsoft tried to ask for 30% from any Windows app installation then would be in a huge legal trouble immediately. Hell, even bundling Windows Media Player led to some fines...I can have any store on Windows and I don't need to pay Microsoft a thing but if MSFT will try to use Windows API to make Steam work worse than Windows Store, that would cause legal trouble for them.
> My understanding is that part of the OpenAI investment was in Azure credits. That may technically be legal, but it blocks any competition in the space and defers paying taxes on the value of those credits.
But here is the thing - OpenAI went to Microsoft and agreed to that deal. They could have gone to AWS or GCP.
Additionally, such an investment effectively makes it cheaper to acquire the company down the line (since they already own some percentage of it). In such a scenario their investment would both flow back as revenue and reduce to the eventual cost of acquiring such a company.
Yes we’ve traditionally called it a Ponzi scheme or pyramid. We are in the part of the cycle where no one actually has to make money from AI. We’ve been here before with Uber and WeWork. All these free queries aren’t free and run on incredibly expensive hardware. The final endgame is a Dyson sphere around the sun so that we can ask ChatGPT-XX if it’s raining outside and the owners of the AI and chip companies blast off to their colony on Mars.
Is the plan or expectation of the stock to go to zero (or otherwise dropping precipitously)? If not, I do not see the value in labeling it a Ponzi scheme.
If there exists a business plan and the bet is to invest in something that provides utility, then it seems like any other investment in any other business.
>because the "investment" is guaranteed to flow back in as cloud revenue
But it can't flow back in as profit. Cloud providers are giving away GPUs for free in a time when GPUs are scarce and could be rented out for a premium. This isn't as free of a scheme as you make it sound.
> These investments not only have no stipulations about how the funds are used, but often aren't used for the business at all - instead directly paying out founders/employees.
One additional factor is you need to beg for GPU capacity from cloud providers, so they are investing in them, and also giving them preferential capacity over companies they don’t invest in.
I wonder what's happening to all that money. Back when they originally released Claude, they were second only to OpenAI as far as chatbot models were concerned. Although Claude wasn't as smart as GPT-4, it had a more pleasant writing style, and Anthropic later released 100k context. At the time, I expected Anthropic to be the next company to release a GPT-4-level model.
But since then Claude has been passed by Mistral's mistral-medium and Google's Gemini Ultra. More concerningly for Anthropic, each subsequent release of Claude has actually performed _worse_ on the Chatbot Arena Leaderboard. (Claude-1 outranks Claude-2.0, which outranks Claude-2.1.) The reason for the decline in ranking is seemingly that the most noticeable update is to make the model refuse more requests.
In an additional blow, the needle-in-a-haystack independent benchmark revealed that Claude's long context is not actually used effectively by the model.
All-in-all, Anthropic is not looking in a good spot, despite the massive investment. They need to start releasing legitimately better models, or risk irrelevance.
A bit off-topic, but it's very cool how competitive this market is. It's only been barely a year and we're all expecting every company to do better within spans of months. Even compared to haydays of cloud hosting investments, people weren't expecting updates from each company on a weekly basis.
Part of me really wants to get into the game somehow, as it looks very invigorating and motivating from outside. Although not entirely sure where I would need to start as I'm not at some cutting edge AI/ML company.
For getting a flavor of training LLMs without needing to be at one of the pre-training companies, it's very accessible to fine-tune a relatively small open source LLM such as Mistral 7B. (There are many tutorials.)
I think there's a lot of funny business going on with accounting, misinformation, and hype generation. I'm kinda tangentially involved (I work at one of the players on projects that use LLMs, but don't train them), but wouldn't want to get directly involved in the core areas, because I suspect that by the time I could develop the skillset the hype cycle will end and the bubble will burst.
It could be an attractive place to be after the bubble bursts - there are some real technological developments that have been made, just not nearly as revolutionary or widely applicable as have been claimed. But AI has a history of 10-20 year hype cycles, so it could be a while before it gets hot again afterwards.
At the MIT event, Altman was asked if training GPT-4 cost $100 million; he replied, “It's more than that.”
Training costs are decreasing, but whenever there's an update to GPT-4 (e.g. training data cutoff updated to December 2023), it means the model has been retrained. The compute costs of training a model like Claude 2 are significant.
Also, keep in mind that not every trained model becomes ready for production. Some are discarded, similar to how you might burn a few cookies while baking.
Seems like the Occam's razor hypothesis would be "company that split from OpenAI over safety concerns is overly focused on safety at the expense of performance."
Claude doesn't refuse requests if you have API access and use a prefill, for what it's worth. If anything it actually complies more than GPT 4.
When you say Claude is worse than Mistral Medium are you going by the Chatbot Arena Leaderboard or some other benchmark? I wouldn't use Claude for coding but I find it to be pretty good at other tasks.
> When you say Claude is worse than Mistral Medium are you going by the Chatbot Arena Leaderboard or some other benchmark?
I'm mainly going by the arena leaderboard, but it's also true in my limited experience with the two. (I mainly use either GPT-4 or open models.) And it's the only model I can remember getting an ethical refusal from. (I don't push hard in that aspect, so it was surprising.) I know it can be jailbroken, but, precisely because I don't push the models hard, I'm not skilled at jailbreaking.
By the way, the mention of API access reminded me how weird it is that the Claude API is still application-only, unlike OpenAI, Google, and Mistral.
I wonder if it's a situation where the knowledge required to improve and enhance these models at a fundamental level vs just more data for testing or screwing around with system prompts is so deep and rare that a single person leaving an organization can have a gigantic impact on the overall trajectory of a startup. No amount of money will fix that problem if the skills required are that rare. Established large companies like Google, Facebook, etc have a much deeper bench of specialists than a startup I would assume and so can survive some people jumping ship.
I doubt it. A lot of this is like voodoo and throwing darts at a board.
Some people developed some intuition for it, but it's, well, random. We're evolving big things we don't really understand, and for reasons we don't really understand, some things evolve better than others. At some point, people starting getting feelings in their gut that some "architectures" worked better for vision than NLP, and things diverged. There continues to be queasy-gut intuition, but when you read all the papers, if you cut through all the hairy math language, it's a lot of voodoo and throwing darts at a board.
I think it's much more the same problem you'd have at any shop, where the smartest people want to work for FAANG / sexy startups / elite universities, etc., and the bottom end of the market just wants a job, and lands with lower-tier employers. Talent naturally consolidates. At the same time, org culture, strategy, and leadership can have a huge impact too.
They want to tank the company to the point another big tech buys it out. And since they focus so much on "safety", that big tech could be Apple or some other conservative company.
I've been using Claude recently and as a product, regardless of the benchmarks it feels much nicer to use, I haven't spent a lot of time playing with open models though
I just tried Claude and it give good concise responses to software questions. I don't like AI services from Microsoft or Google because I have been burned out by them more than once when they change the terms of their services or kill a whole services just because. Not that Open AI or Anthropic can do the same but at least I give them the benefit of the doubt.
OpenAI does the same, indeed definitely much more often than Microsoft and probably more often than even Google. In fairness, it's all tagged beta. However:
1) Models change. Things I did a year or two ago behave differently, and usually stupider.
2) OpenAI just announced they're killing GPT-3 and other older models. It's annoying, since I like GPT-3. It doesn't have "safety" built in, which makes it better at a lot of creative work. If you have other models baked into your codebase, you're looking at a massive migration.
Extra credit: Find any public information from OpenAI about this. See how long it takes you. (in fairness, customers did receive an email)
3) OpenAI decided to kill the completion API. This is annoying, since a lot of my uses don't look like a chatbot. I understand chatbot is THE killer app that's come up, but a lot of stuff works better as, well, completion. Indeed, something broke in the past month since I am getting
Extra credit: Same as above.
A normal company would continue to include this stuff on e.g. their pricing page marked "deprecated," have announcement, etc. Here, it's like GPT-3 never existed. And I really miss the old soulful models.
There's a major move towards open-source models in my industry since a lot of things built a year ago on OpenAI no longer work or at least no longer work the same.
Critically, you can't validate apps built on cloud-based services since you have no idea when models change and your app might suddenly do something dumb.
Boy I just can't stop thinking about that weekend when Sam got booted from OpenAI, and Anthropic turned down the offer to acquire OpenAI. If I were CEO of Anthropic, I would have had one job that weekend.
I am very curious about Anthropic's strategy right now; they launched with "More progressive and careful about AI risks," which in my opinion is a good AI worker recruiting strategy, but not a good commercial strategy, or even a good quality strategy, based on Claude's current state.
They have done some outstanding research this year -- in particular, I loved their work on tracking conceptual bundles through different layers, and I'd say their paper on embedding adversarial behavior in LLMs was my favorite paper of the year -- But, I'd like to see that research turn into more capable models.
As I write this, I wonder what I'd do if I were in charge there. I think I'd probably try and one-up Mistral, and release larger more capable models more quickly. I'd try and buy Groq and build a custom vertically integrated stack that's 5x faster in tokens/s; worst case Amazon could help with the acquisition and deploy Grok into AWS. Basically the moat would be deployed tokens/s at or above certain levels of quality. I'd stop running anything but my own models on Groq hardware.
There are two next-gen directions for inference in the cloud, high quality multi-modal foundation models, which are definitely coming, but also clearly need some UI work to get into a compelling product format, and very long context length models, so I guess I'd possibly be trying to leapfrog there. It's highly competitive in the model space right now, so a model-only strategy isn't good.
Alternatively, I could sit back and relax and let AWS pimp Claude while we just scaled it up to 250k tokens and used MoE to get non-exponential scaling / possibly explore an SSM / Mamba architecture, and not worry too much since we have 4bn in the bank. I wouldn't be surprised if that latter path is the one they're on, sadly.
The board approached the Anthropic CEO (Dario Amodei) and proposed a merger with him as CEO. Possibly because he presents as safety conscious. He declined.
Wow that seems incredibly stupid. But I guess maybe he knew that he would get kneecapped by some backroom corporate jiujitsu by Sam who would then take over the new merged company, leaving Dario with nothing except for an incredibly fat severance payout and all his stock.
We've seen this before. Lots of investments in the '99 runup had ad buys associated with them which did the same kind of double counting. The example would be Yahoo investing in some startup at an inflated valuation, paying for some or all of the investment with ad inventory priced arbitrarily and which was, for Yahoo, free to produce. Lots of things led to the 2001 crash, but valuation distorting shenanigans like that didn't help. To the extent it feeds through to Microsoft, Google, and Amazon valuations the AI dynamic could follow a similar pattern.
The quality of discussion in the NYT comments is exceedingly bad.
In our industry it’s easy to forget that the average person doesn’t understand what’s happening currently in AI and fears the worst, based on the FUD being shouted by mouthpieces who have something to gain by being the gatekeepers of who uses AI and how.
Since then they’ve been growing and now they are the biggest ~3T jaggernauts.
My biggest fear is by their sheer momentum they keep on getting bigger, capturing regulators, acquiring companies, doing cloud credit deals. They will be the technology dictators like Putin.
US anti-trust is quite weak. Lina khan hasn’t had much of a win.
If OpenAI cracks AGI and MSFT has the sole license to extract the most profits and set the regulations, we are in a tough spot as humanity.
If Apple and Google keep their duopoly on the OS that most people use and every transaction is rent extracted at 30%, then that is a sad future.
Is there room for a new player to climb the ranks and be top 2?
Google and Meta are almost set for life as tougher they have a massive duopoly on online ads.
https://archive.ph/SGpyY