I am not surprised. This worked out fairly well for them in the past.
While I dislike this personally - I'd rather have competing chips on the best node - I don't think this violates any moral principles. I do wonder if there is a bidding war with other companies for this capacity, though.
This risk should be evaluated iteratively. It won't kill Apple if some small fraction of the time, their releases are delayed. Overall, if their record is that they have stuff their competitors can't even get their hands on, they win.
Even though it's not technically illegal, there's something about this that feels unfair and bad for the market.
They're certainly the leading player in terms of cutting edge capabilities, but there are plenty of companies working on fabs. It wasn't long ago that Intel used to have the leading fab and simply got complacent/slow opening the door for TSMC to surpass them.
In the macro sense there seems to be more widespread awareness and money going towards fab competition than ever before. I suspect we'll trend closer to performance parity between competitors in the coming years.
Nobody will be able to compete against Apple CPUs while they're 1 process node ahead of competitors though.
I recently moved to Linux from MacOS, but probably will have to go back to Apple on the m4 for local LLM capabilities (high RAM specs much cheaper than commercial GPUs). Too bad Asahi is still missing some pretty critical capabilities (microphone, USB-C monitor connection etc)
Nobody will be able to compete against Apple CPUs while they're 1 process node ahead of competitors though.
If this artificial advantage allows Apple to slack off in their other areas of competitive advantage, then this is bad for the consumer, overall. This creates an environment of cynicism, where there's even degraded incentive to try and beat Apple with a better product.
If Apple slacks off in any way, some other company can step in with innovation and consumers benefit by increased choice and innovation. We shouldn't worry when a company slacks off except when its moats are illegal. If Apple's business power were properly regulated, they'd be forced to compete more of the time and when they didn't they'd face more consumer friendly Apple unfriendly results. Fix the antitrust enforcement and we'll fix the problem.
TSMC being the only game in town is what's bad for the market.
Beyond some threshold, any meta-gaming of the market is bad for the market. It's like patents. When they were first instituted, they did some good by incentivizing innovation. Then, people started meta-gaming them, and used them as legal weapons to suppress competition well beyond the original intention.
I don't think there's a good argument that monopolizing a feature is good for the market, even if the means that allows it is technically legal.
What could be argued, is if legal remedies to this kind of meta-gaming might also be meta-gamed in return and become worse than the thing they were trying to remedy. This seems to always happen in a variety of forms.
>Nvidia and AMD at one point (still?) exclusively use TSMC.
While Nvidia and AMD both uses TSMC, they are usually not the client for leading edge node. Yield will always be low on leading edge and you want small die size at preferably below 100mm2. You also want extremely high margin products to justify the highest bid and large investment in all the adjacent tools.
Apple does this all the time so far as I understand. Back when I worked in embedded firmware, ten or fifteen years ago, I remember hearing about a new MEMS chip we could not get - which nobody could get - because Apple had bought the entire production output.
I wonder what Apple's priorities are for the M5 chip. I'm all in on inference and training in my personal life right now, so if I were deploying $20bn with TSMC, (or maybe more this year!), I guess I'd be trying to spec out a scalable architecture with as much high bandwidth memory as possible.
I wonder if Apple's willing to spec out 350W datacenter chips or not, and if they're going to make a play at datacenter AI with this generation. It feels like it might be a good time to do it. Anything competitive at all will suck a little oxygen out of NVIDIA and probably pay for itself in market cap.
Either way, I'm pleased my Apple tax is going to buy me 2nm chips next year or so; silicon feels very interesting in the next few years. Juicy.
> Anything competitive at all will suck a little oxygen out of NVIDIA and probably pay for itself in market cap.
To whom? If you're Apple I guess it makes sense to invest in your own architecture. But Apple doesn't ship a CUDA equivalent, they don't support server OSes, and they make no attempt to support the few cross-platform solutions shipping today. In a lot of ways, they're even further behind than AMD's server-side AI offerings.
Apple ships MLX, which has found support in creator communities, and is starting to see support at places like hugging face. In fact, it seems to me to be growing faster than ROCM support, but of course, it is starting from scratch, and this is just anecdotal.
And, MPS are a drop in replacement for many programs in pytorch -- just replacing device="cuda" with device="mps" works relatively well, much of the time. For those doing inference at the edge / playing / testing, mps works well -- it is entirely the reason for llama.cpp's popularity, for instance.
Rumors are that Apple is looking at using high memory M2 chips for datacenter inference -- 192GB of RAM is pretty great, yo.
So I guess I'm saying I think you're wrong on the facts -- Apple does support pytorch, it does have an alternate full stack offering with MLX, and I would not be surprised if more tokens are generated through llama.cpp's downstream offerings on Apple Silicon than are generated on AMD.
MLX is not a cross-platform solution though, and it's certainly not a one-size-fits-all GPGPU SDK a-la CUDA. It's a nice acceleration option on top of Accelerate framework and MPS but it's not going to kill Nvidia (or probably even steal market-share; it's local versus server we're talking about here). At best, it's another executor to throw onto the ONNX pile of Competing Standards: https://onnxruntime.ai/docs/execution-providers/
> Rumors are that Apple is looking at using high memory M2 chips for datacenter inference -- 192GB of RAM is pretty great, yo.
It also helps that Nvidia has deep industry buy-in and supports the server market much more than Apple does. But strictly comparing Apple's potential server offerings to Nvidia's Grace systems is a blowout; Apple genuinely needs custom hardware to stand out in the server market.
It's my opinion that Apple just can't do this. They're an entire TSMC node ahead of Nvidia and still are getting smoked in performance-per-watt by their consumer-grade chips: https://browser.geekbench.com/opencl-benchmarks
Obviously gb200 and m2 studios are very different price points and power envelopes, per other comments.
I agree that Apple does not have a competitive server offering right now. But I think I'd still be spending toward that goal if I were reserving all the 2nm chips in the world next year.
Your claim that NVIDIA has better performance per watt really surprised me, so I did some digging. My prior is that opencl is medium support at NVIDIA and very low support at Apple, so an OpenCL benchmark might not be fair.
Reporting tokens/s, using llamafile's llama.cpp benchmarks for Mistral 7b f16 from Justine tunney's website (so this is non-batched): 192GB Mac Studio gives 79 prompt tok/sec and 15 eval tok/sec.
A gb200-nvl72 seems to cost [1] about $3,000,000. I expect Apple could throw together a few, several even, 192GB M2's for that sort of money - and they may not have a 16TB bus, but I suspect (for the same price) their quantity might make up for it in the problem-sphere that a gb200-nvl72 would expect to be in...
It's expensive, but that's because 72 GB200s are faster than 144 networked M2 Ultras. You're tackling bigger workloads with better-standardized software, genuine first-party support and proprietary integration that goes even further than Apple's memory coherency. Datacenter customers aren't drooling over the advantages of Apple Silicon because most of them are table-stakes at their scale; Apple has to go the extra mile to stand out, specifically for these customers.
I won't deny, Nvidia charges the sort of margins that would make Apple blush. But they do that (and are successful) because they're scaling the workloads Apple won't. Apple turned their back on Nvidia, gave up funding Kronos' alternative to CUDA, and now wants a piece of the pie they've spent the past 15 years trying to avoid. It just doesn't seem like they're capable of winning here; their strategy is to dominate or die, and domination isn't an option.
I worked in Apple HW for several years, and it's not nefarious like that...but I understand how it could look that way.
It's more like placing an order for N units of whatever, but the total global capacity (emphasis: total _global_ capacity) is < N.
That leaves open the speculation that shady business was going on, but it's just that they ordered early because they're at the moment first to need whatever, and businesses expand footprints to try filling as much capacity as Apple would like to order, when Apple needs more than is available.
Unless you were involved with negotiating the deals, what happens in face to face meetings (i.e. no email and paper trail) during contract negotiations is a not-so-subtle battle of wills based on perceived leverage -- of which a trillion dollar company has a lot of.
I have been on the software side of things for most of my career, but over the last handful of years I have been deeply involved in the bizdev stuff. To say the wool has been pulled back from my naive eyes is an understatement.
Except both Tim Cook and Jeff Williams knows even with Apple's $200B capital it is still not be enough fund a future TSMC competitor ( Intel ) that is price competitive with TSMC's current offering.
They may have a better chance helping Samsung Foundry but they dont want their SoC IP to be known to their direct competitor, that goes the same to Intel.
On the other hand TSMC dont start building out a $20B 2nm GigaFab and then look for 2nm customers. That is just not how it works.
It takes more than just money though. Apple couldn't for example make self-driving cars with despite all the money in the world. It also takes special raw materials, talent, and culture, with the combination being too difficult to materialize.
> Secret meeting between Apple and TSMC reported, /possibly/ to reserve all 2nm capacity
> Secret meeting /may/ have been about 2nm exclusivity
> The low-profile visit was made to secure TSMC’s advanced manufacturing capacity, /potentially/ 2nm process, booked for Apple’s in-house AI-chips, according to the report.
It's nothing new at all for Apple to buy up all the capacity and this isn't even an article on that actually happening. It's speculation on top of speculation. Also "secret meeting", ooooooo secret, give me a break... Show me the public meetings that other companies have with TSMC or hell the public meetings they have with any of their suppliers.
Plenty of companies have meetings with other companies where their executives arrive in front of cameras and parade the meeting's existence usually to alert investors, customers, or competitors.
There are other meetings they don't advertise that they attempt to keep from others by not inviting press, releasing statements, and otherwise publicizing. Those are called "secret" meetings not because their content is secret, as that's true for most corporate to corporate meetings, but because their existence is secret.
Not understanding something as simple as this suggests a feeble or naive mind.
While I dislike this personally - I'd rather have competing chips on the best node - I don't think this violates any moral principles. I do wonder if there is a bidding war with other companies for this capacity, though.