768GB RAM pipe dreams make no sense to Apple. By discontinuing 256GB / 512GB M3 Ultra and raising prices $5000 -> $7000 on Macbook pro with 128GB they basically confirmed how badly RAM shortage affecting them.
768GB is 64-times of 12GB which is rumored to be amount of RAM in new iPhones. Imagine what profit margin 768GB Mac Studio gonna need in order to justify making one instead of 64 iPhones.
Apple is the company that is okay about selling microfiber cloth for $100 and wheels for $700. Imagine how bad price hike for M3 Ultra 256GB / 512GB had to be in order for them to just discontinue them instead of getting free money out of desperate local AI folks.
The message I’m getting is that Apple will never compromise on its healthy margins. If something becomes basically unaffordable for their target market, they’d cut the production and even discontinue the product, than take a hit on margins. Their business model is refreshingly simple.
Apple is not gonna risk their iphones, as they are their flagship (aside even from giving them higher margins). My opinion is that, as we are talking about ram SHORTAGE (not just for ram price hike) they have to cut the more ram hungry models to be able to keep up with their projected production/demand (at reasonable ram prices). Getting iphones "sold out" is not a great thing for apple.
Once/if the ram shortage ends, they will continue increasing the ram caps as they were already doing, because then selling ram-heavy macs will not interfere with the rest of their products.
It's just a planned economy failing the way planned economies often do: the central planner failed to predict the demand correctly. Instead of trying to secure additional stock from the market at spot prices, they are simply waiting for the next batches they had planned for.
They could treat the extreme spec machines separately from the prosumer ones, like they did with the Xserve. Let business customers spec up to 768GB (say) who are prepared for a $20-25k price tag, while keeping them away from the stores and usual consumer supply chains (Amazon et al). It may not be a big enough market segment for them to care about anymore, though.
They can do it, but that gonna need to be different SKU not Mac Studio. Otherwise news will be full of discussions about Apple price hike from $8000 to $24,000 or who the hell knows $48,000.
So yeah the only way I see them selling it is usual "call us" enterprise price tag.
But since its not what Apple usually do its easier to sell 4x Mac Studio 256GB RAM boxes with interconnect for lets say $12,000 - $15,000 each.
I don't think it's needs 'call us' just a separate SKU, I mean if they called it Xserve Ultra and it was just a studio in a 1u format with dual PSUs and extra RAM, it would fly off the shelves.
Isn't that the same thing you can get from ordinary 2S Epyc/Xeon servers at a similar price that have 24 memory channels (when the M3 Ultra has the equivalent of 16)?
And the reason people rarely use that for AI is that the enterprise GPUs from AMD and Nvidia are only moderately more expensive but are significantly faster because they use HBM instead of DDR5.
Yeah kind of, I think a 24 channels DDR5 works out approx 1TB/s, but the cost is astronomical, a M5 studio would probably beat that performance for around half the cost. You also get to use the GPU/NPU cores of the mac vs CPU only on the servers.
M5 ultra studio with 128GB RAM could probably beat out a sever with a RTX 6000 pro at half the price.
24 channels of DDR5-6400 is 1.2TB/s, M3 ultra is 0.8TB/s. They both use DDR5-6400.
> a M5 studio would probably beat that performance for around half the cost.
A barebones 2S system with no CPUs or memory is ~$2000, a pair of 16 core CPUs another ~$1000 each, and then however much memory you want. The price seems pretty comparable. The "problem" with doing this is actually that 128GB is too little memory, because you want to populate all the channels, but even using 16GB sticks, 24x16GB is already 384GB.
> You also get to use the GPU/NPU cores of the mac vs CPU only on the servers.
You only need enough cores to make sure the bottleneck is memory bandwidth.
M3 ultra is obviously 1-2 generations behind and new the studio is expected 'any day now. Even if this was M4 Ultra it would still be ~comparable to any EPYC system in bandwidth, but get to use the GPU for compute so potentially faster than the EPYC. Total Cost of Ownership in the Epyc is going to be WAY higher because of electricity costs, the EYPC is going to be consuming probably 5X the electricity and is probably not going to sit quietly on your desk. More RAM though, but again it's more about the ratio of RAM (size) to RAM (Memory Bandwith) to Compute and you may find a model bigger than e.g 70b suddenly is bottlenecked by the CPUs or memory bandwidth and therefore the extra RAM (size) is wasted. But maybe not, different use cases will yeild different results I guess.
> A barebones 2S system with no CPUs or memory is ~$2000, a pair of 16 core CPUs another ~$1000 each, and then however much memory you want.
As you say, the thing is it's not 'however much memory you want' it's 24 sticks which at $300 a stick for 16GB is $7200, then you also need at least one NVME disk so you're looking at what $13,000?
> That would be the kind of thing you could sell as many as you could make.
People said that about the M1 Ultra Mac Pro, a few years before it was discontinued. I don't think there are many HPC customers looking at Apple hardware.
> Let business customers spec up to 768GB who are prepared for a $20-25k price tag
There is a clear difference between $25k and $100k.
64 iPhones at retail price is already around $64k. For something at 768GB to be profitable at Apple's terms, this has to retail at $100k for it to be profitable. That was the OP's point.
Honestly you're still looking at (from my understanding) ~3 minutes prefill (TTFT) even with architectural improvements and so on with a 32k context window (against a large model). How is this going to be competitive with Nvidia and all of the tricks massive scale get's you to parallelise context across many machines?
Is it supposed to be? I think the point with some of these Macs is you get the capability in something the size of a heatsink from Intel's Netburst architecture era, or a Macbook light enough to stick in a backpack and take with you to lunch.
If you're talking about chaining together multiple GPUs you're talking about a different game -- I suspect, anyway. Seems like a high-spec Mac would be good for development and testing. Arrays of GPUs, better aimed at production use.
It's even worse than that. Demand is so high so every next GB will be sligly more expensive. There is a lot of smaller hardware manufacturers that unable to secure DRAM chips at any price, there just no free capacity on market.
Of course Apple is massive, but if they announce inference boxes that everyone wants they need to make 10,000s or even 100,000s of them.
And it's very much likely they'd rather sell 600,000 - 10,000,000 more iPhones or Macbooks Neo. End users bring Apple money with every OpenAI / Claude subscriprion sold through their platform.
And inference boxes is just one-off sale of hardware that will bring no further income.
If they cared about local AI market they would price hike M3 Ultra instead of discontinuing them. After all they conviniently introduced RDMA just few months before that.
Initially when it happened everyone expected they did it because they planned to announce M5 Ultra shortly, but its not looks like this is happening.
Now IMHO its indicates they simply run out of RAM supply.
If you’re truly serious about local AI you’re not using a Mac. A Mac is for people who want the best of both worlds, GPUs are much faster if memory is equal.
They are assuming that they are able to get ram in the future, once the AI bubble either dissipates or pops. Its far easier to build something you planned for 3 years ago, than crash build it in 3 months.
Of course if RAM prices crash Apple might of bring high RAM options back. We just cant bet on it as consumers.
Right now RAM shortages are bad to the point where likely even Apple have to decide what products they make and what they discontinue.
There been short time where M3 Ultra with 256GB / 512GB been best offet on market because Apple lagged with price increase. Now HN crowd expect Apple of all companies jump into price war with Nvidia and to subsidize their inference hardware.
Apple is actually interesting. They are one of the few companies with a chip / PC play with real power AND basically no play I'm the hyperscalar market.
That means they're actually incentivized at least short term, to benefit PCs becoming strong enough to do local LLMs. Which makes this play make even more sense. Though, I've been saying for a while that the local AI inflectiom point is the death knell for these frontier labs.
> Though, I've been saying for a while that the local AI inflectiom point is the death knell for these frontier labs.
"Death knell" is a touch hyperbolic. Hardware that can only run quantized models that take up GBs in VRAM falls short of even an A100 (by almost an order of magnitude[0]), which in turn falls short of what an 8xH100 cluster can do (also by another order of magnitude[0]).
I'm an avid believer in local LLMs, but I cannot deceive myself - data center accelerators will win on power dissipation numbers alone[1], even when giving generous allowances for higher efficiency on Apple chips - and assuming the Apple-efficiency advantage persists on the same TSMC process node.
0. Based on my unscientific fine-tuning training experiments across local and rented GPUs. YMMV for inference.
1. Unless Apple surprises everyone and brings back the XServe with M7, if not, then laptop and desktop for factors simply can't dump heat fast enough to compete head-to-head, and will be designed for lower input wattage.
Doesn’t need to be a winner head to head. If it can do 90% of the tasks the big boys do, at 50% speed, for virtually no extra overhead cost save for the power consumed by a prompt - that’s gonna work for a lot of people. And that’s also basically where we’re at today. Qwen3.6 35b running quantized on 10 year old hardware solves basically all of my uses cases for agents except for coding.
The frontier models are faster, and better at coding, but not so much that i’ll pay $200/month for them.
Consider this. One of the smallest Qwen models (4B parameters) powers my home automation voice assistant, and runs on CPU alone at >20 tok/s. It is enough for that use case, and could be made even better/faster with a modest GPU. It isn't as smart as some cloud-connected thingamajig, but I would never allow a literal Google or Amazon bug in my home. Huge SOTA models aren't relevant everywhere. Most people use LLMs for rather trivial tasks such as finding typos or drafting text.
But with Apple's AFM 3 architecture, we might end up with huge SOTA adjacent on devices with limited RAM.
They use a technique where you only load between 1B and 4B of a 20B dense model for an entire prompt run, not token by token like a MoE, and use mostly the low power ANE instead of GPU cores.
Now, imagine if/when they scale up to 100B or more? On a chip using 2W?
I think we're also ignoring a potential innovative move in how models work.
If someone could splinter or fragment the models into more specific tasks i.e "spellchecker AI" and get these working as well as Sonnet 4.6-4.8 on those tasks on a personal laptop. You then question the $100 a month fee.
Bear in mind these laptops are likely to be $5000 or so because of the memory, HDD and M7 chip they likely need.
It feels to me like the beginning of the inflection point but software updates not hardware updates will be the accelerant.
"That’s where EMO comes in.
We show that EMO – a 1B-active, 14B-total-parameter (8-expert active, 128-expert total) MoE trained on 1 trillion tokens – supports selective expert use: for a given task or domain, we can use only a small subset of experts (just 12.5% of total experts) while retaining near full-model performance."
> If it can do 90% of the tasks the big boys do, at 50% speed
I want to live in this world too, but these numbers, as of today, are very aspirational and far removed from reality.
I'm no tokenmaxxer; I find my modest local setup useful, I also know the limitations, it's slow and it sucks (relatively) at high-level and/or long-context planning, compared to frontier models. Only a minority of my prompts are max-effort - its not all I do, but, it also means frontier labs aren't dying any time soon
Consider also that right now LLMs run slowly enough you can watch them think. I've seen a demo of an LLM running at an absurdly high speed and it reminds me of when I moved from a 2400 baud modem to a 14.4 - BBS screens that I could watch draw were all of a sudden nigh-interactive. Faster-than-realtime video generation is also coming, and will also continue to require huge hardware for a long while yet.
I love local models - I have a machine at home that runs a few for me and it's a lot of fun - but for the time being they are not super trustworthy on tool calls and staying on script. Another year or so might change all that!
Yea, is almost "scary fast" in a sense... the amount of compute you can do in parallel one one of those chips is amazing. hopefully they get their next chip completed as that will be a lot more useful for general workloads. I think the current one is based on llama3.
The weights they “etched” into the FPGA card that’s used for the ChatJimmy demo are that of a Llama 3-something 8b model.
The actually impressive and novel thing is that Taalas’ve managed to automate that process (clearly – nobody transforms 8 billion numbers into a physical representation by hand).
So now, they can work on scaling this process up, and with low enough lead times (I’ll be convinced they have inside connections to TSMC if they can actually deliver on the promised mere 3-4 months delay), will be able to offer 30-100b+ parameter models under half a year after they’re released, at thousands of tokens per second while probably drawing less wattage (per token, not sure about overall).
I’m sure you’re right, for the things you are asking of an llm, just as I am right about the things I am asking of an llm.
The real question is, what are 90% of people going to ask llms to do. I’d argue mostly it’s going to be stuff that works-now or almost-works on local models, but that’s just an opinion. It also depends on the frontier models hitting a wall of steeply diminishing returns, since they set the expectations for all of this stuff - my gut says that’s happened already they just won’t admit it for a while - but we’ll see.
This is what makes sense for me as well. All I need a local model is for playing with simple graphics: no gradients, at most ten colours which I can push through VTracer to get an SVG. Draw Things does the job, usually in 120 seconds or less.
Sometimes, I need a quick throwaway bit of python. That can take 30 minutes of my time.
The established AI players have no financial interest to make LLM available locally. They aren't hardware companies and if running LLM requires paying them to host the models as well then they can naturally capture more of the value chain = more revenue.
Apple is the only player here where it would play into their natural hardware incentive to get you to pay more for better hardware. It would make sense for them to find a way to run LLM locally (eg, newer architectures that others here have pointed out).
Is it hyperbolic though? One of the best things about the compute and memory shortage is that people are going to insane lengths to optimize things to run on lower memory / lower compute devices. If we keep this up for a while and then ramp up memory and local compute production, that AI inflection point may actually come.
That’s about 4 years in hardware cadence alone. There is a lot of room to improve memory bandwidth, and performance is a given with every process node. IBM has shown yesterday they can do limited runs on 0.7nm (density equivalent).
The big question for local LLMs is whether there is a 100 tok/s model which requires less than 16 GB of memory and is competitive on most tasks with the cloud models.
There is some signal that this is possible through both hardware innovation and training/data improvements.
Cloud models have their own constraints - I can’t have opus4.8 spend 4 hours on a deep research question I had in the shower without spending money. I can’t do real time video game upscaling and graphics work in the cloud period.
A laptop is about an order of magnitude cheaper than a cloud server thanks to economies of scale, uptime requirements, and other factors.
if you do the electricity math you'll see that you pay more on local models while getting less (local is more heavily quantized) compared with OpenRouter.
I'm not talking local Gemma/Qwen vs cloud Opus, but against OpenRouter same Gemma/Qwen
there are reasons to run local - privacy, availability, but cost is not one of them
I am allowed to plug in 800w of solar panels into a wall socket here in spain. That would more then cover my current computer with 16gb vram. Now if i went and built a LLM server, at full load i would probably be closer to 3600w (Dual Epyc CPUs that gives you 8 x16 PCI channels and up to 8 cards - Way overkill, i know). If i half that with 1 EPYC and 4 x16 PCI channels, and add the same amd 7800xt i currently have then i should in theory be able to run at around 1800w under full load. Now that could still be covered with a 2000w solar install (get a professional setup OR get a battery unit like a EcoFlow that can output 2000w and can input about the same amount of solar).
Now, this all brings the upfront costs way up, the solar panels are cheap, its all the rest around them that tends to cost money.
There has been a lot of market-subsidy in AI which is starting to fade away: e.g. the copilot quotas/pricing. When VC switches from investing to wanting a return, the price equation is likely to change.
for me it would be about $2 per day in electricity to generate 8 mil tok of Gemma4-26B at 4 bit quantization. this is excluding how much the GPU cost (no amortization)
ignoring the fact that I could get more free tokens per day for this model from Google/OpenRouter, it would cost $4 per day on OpenRouter if paid, but they would run it at full 16 bit precission
this would be the most "profitable" model for me
for Gemma4-31B I can generate only 1 mil tok per day, and so I pay more to get less quality than OpenRouter (ignoring that this model is also free on Google)
> The big question for local LLMs is whether there is a 100 tok/s model which requires less than 16 GB of memory and is competitive on most tasks with the cloud models.
Benchmarks maybe? Real world, no.
You just need the context otherwise. There's no way around it.
Context is more available locally. You can have the LLM operate for arbitrarily long periods, use your credentials to access services (if desired), store memory locally etc.
Whether such a model exists or not is a different question.
> Hardware that can only run quantized models that take up GBs in VRAM
That's the today hardware.
Now suppose Apple goes to any of Samsung/Micron/Hynix and says "we'll pay you the entire cost of building another DRAM fab and in exchange we want its entire output" and then releases M7 devices with enough memory and compute to run bigger models.
> Unless Apple surprises everyone and brings back the XServe with M7, if not, then laptop and desktop for factors simply can't dump heat fast enough to compete head-to-head, and will be designed for lower input wattage.
Laptops maybe. Desktops can dissipate more heat than the amount of electricity you can draw from a typical household wall outlet.
> Now suppose Apple goes to any of Samsung/Micron/Hynix and says "we'll pay you the entire cost of building another DRAM fab and in exchange we want its entire output"
It's revealing that they aren't doing this: no one wants to fund that gamble on the state of AI demand 12-18 months out, but ate happy to capitalize on their current product lines/capacity.
> Desktops can dissipate more heat than the amount of electricity you can draw from a typical household wall outlet
100% agree, but the data center power and cooling infra are not limited by home wiring, and go way beyond what a wall outlet can safely provide (1,440W max on a typical 15A circuit at 120V). A single H100 maxes out at 700W
The thing is, with the level of hard investment AI vendors have, even a small reduction of their addressable market is significant. They aren’t profitable, and inference is getting commoditized fast, so even if they eventually become profitable (not via financial engineering) they won’t be able to have good margin. The pressure of both open models AND local models is pretty bad imho
We'll likely see a transformation in how frontier models are trained as a result of a push towards local inference. While it seems unlikely now, given current pricing for RAM, in 10-15 years it's not unthinkable to assume we could see individual machines with 10-12TB (and well beyond that) of RAM which are accessible to the GPU. Min/max system RAM increased a LOT from 2010-2025 and largely because it was cheap. Once the hyperscalers aren't generating revenue for the RAM manufacturers, I wouldn't be surprised to see a massive push towards consumers in order to maintain gross profit. Not to mention new players who enter the market because the margins are measurably absurd right now.
At some point there will be diminishing returns towards the "just throw more RAM at it" approach the current frontier models are taking. Commoditization is just as inevitable as it ever was... and in doing so will enable actual leaps of what AI/ML is capable of. That's not to say there won't be a place for 99.999999% accurate vs 99.99999% but those cases will be limited and likely prime to disruption based on real innovation vs access to capital.
10TB is about 80 times that, 200K in today’s money. A lot of capacity is coming online in the next 5 years and it’s reasonable to think we can get there with better process and stacking (the latter does little for pricing, but enables shorter latencies).
Indeed. Local models becoming available and halfway decent don't obviate the laws of scale. And because there's no ceiling to what scaling more will buy you in terms of capability, there's no reason not to scale more, there's no incentive for billionaires not to grab all the fab capacity they can.
Enjoy paying $1000 or more for a little 4 GiB cloud terminal that connects you to all your online accounts where all your actual work gets done. This is the future.
There's a limit that won't be breached without a fundamental breakthrough in physics of computation, but we're not there yet by a long shot. You can train bigger models, faster, and infer with them faster and more precisely, by throwing more compute at the problem for the foreseeable.
At some point, and I can already see it, they’ll be better than us at writing code. We are still in the loop to coerce them into architecting well, but that’s nothing magical.
What’s frontier now is prosumer in a couple years and commonplace in a couple more.
It's plausible but is the Apple Tax for a 1TB memory machine on top of current memory prices really worth it? I paid around $4000 for 4090m laptop with 16GB VRAM back in 2023, it's great but DoA for even quantized LLMs. I can run SLMs and fine tune it but that's it.
We need one of those specialized inference chip startups to succeed and a PC manufacturer willing to bet on them against Nvidia for the local AI to find mass market appeal.
I recently bought a Mac mini M4 16 GB - mostly to run Immich. I assumed I needed a Linux box. After a lot of researched I was quite surprised that the mac was the cheapest option. So not always an Apple tax.
I'm not sure why you would need large AI models for Immich, the face detection is pretty cheap and will run on 10 year old hardware without a blip.
I think the decision comes primarily on how much data you would like to store for Immich, if you want to go cheaply, a 100 bucks used laptop will do the job, if you have too much data, a NAS will be more suitable (and you are certainly not going to get a mac where you can plug multiple internal hard drives for the price of a NAS)
>" After a lot of researched I was quite surprised that the mac was the cheapest option. So not always an Apple tax."
Apple has always been the most cost effective choice for the value you get going all the way back to the Apple II, it's just that the floor of that cost has always been high. Anyone who thinks otherwise is a just a fanboy one way or the other.
If you're happy with the base model, then Apple has a great value/price ratio, but there's nothing below it and any upgrade is really bad value for the money.
That's true only for the entry level macs. My M4 Mac Mini has the best Performance/value. But my workstation laptop with 32 cores, 96GB DDR5, Nvidia GPU costs lesser than Macs with lesser performance; not to mention I upgraded the RAM post purchase.
It really depends what you factor in as value, because wintel laptops like you described tend to require noise canceling headphones when working on them.
That's how much many developers currently spend on tokens - every day. Whatever "Apple Tax" applies to a device that can run a capable model offline will amortise itself in a blink.
>Whatever "Apple Tax" applies to a device that can run a capable model offline will amortise itself in a blink.
Current high-end Mac Studio with 32-core M3 Ultra chip and 96 GB of memory is $6800, 96GB is not enough to run GLM 5.2 without extreme quantization or stacking HW; but for the sake of discussion let's run quantized version on a single high end Mac Studio.
GLM 5.2 Max plan costs $ 112/m, so it would take ~60 months to recover the costs assuming the machine was bought just for AI. By then the current AI landscape would have changed drastically.
I use local AI on both Linux and Mac every single day, there's freedom, privacy and peace of mind in running the model locally. But I feel cost/value of local AI is overblown.
"Apple tax" is such a lazy and inaccurate accusation to level. Sure we've had expensive wheels on the cheese grater (ie Mac Pro) but we've also had:
1. When Apple came out with the real Macbook Air in 2010/2011 (not the silly 2008 one), nobody could compete with it with those specs at that price and they couldn't for years. And every competitor usually sucked in some major way, most often the trackpad;
2. The Mac Mini is an outstanding piece of hardware for $600. Or was;
3. I've generally found that "Apple tax" complaints levelled against the iPhone to be nothing more than Android cope;
4. The M-series silicon has been an absolute game-changer. I honestly thought the first-generation M1s would be not great but they came out swinging. And the price points for these Macbooks have all been great, much better than the last-gasp-of-Johnny-Ive touch bar butterfly keyboard series, which were objectively awful.
I didn't have a single Apple device in my house until a month ago when I bought a Neo. The last Apple devices I had before that were an iPod Nano and a PowerMac G5 many many years ago.
Apple has pretty good competition in every segment with the exception of maybe the iPad, but I'm not a tablet user.
Some folks like to have a computing environment free of proprietary influences and extremely strong vendor lock-in. I cannot claim to posses any apple devices.
I wasn't thinking of Asahi. Just pointing out that you can run all the standard unix/open source tools and apps on Mac OS (vi, git, qgis, blender, vsc, python, node, etc). With the advantage of higher quality hardware and generally less fiddling.
But if you don't like it, switch. I don't see vendor lock-in.
there a many people who don't own Apple. Why are you so surprised? I certainly don't and never will. What's it got that I can't get on a standard PC + Linux?
Tangential: About 8 years ago ex-Apple chip engineers left to design server-grade chips, this was Nuvia, and they got sued by Apple to the point that they had to get acquired by Qualcomm.
I worked at a hyperscaler when the M1 came out. A MacBook Air M1, running a Linux VM was faster and more energy efficient than anything we had in the data center.
Nope, across all benchmarks, Linux running in a VM on the M1 had higher perf (4 cores) than any instance type I launched across Intel, AMD and Arm. Triad, coremark, openfoam, verilator, etc. This may have changed, but my hunch is this is still probably true with M5 vs all the cloud providers.
They do stand in front of a great opportunity that would also benefit consumers, which seems rare in the llm era.
If people can get opus4.6/gpt5.5-like models locally, labs could raise their prices and sell token speed, better reasoning, mobile-focused improvements, you name it.
Not all consumers are power users and many will be happy to pay for flexibility.
The comparison is against your own computer, which is powerful enough, no need to add a box and set it up, and a very expensive cloud service. It’s nothing like setting up a NAS versus using iCloud (because we are talking Apple here).
The article says base M7 memory bandwidth is targeted at 240GB/s.
M1 had 70 GB/s, M1 Pro: 200, M1 Max 400, M1 Ultra 800.
Modern RTX 6000: ~1,600 or so.
If we get a 1,200-1,500 GB/s bandwidth M7 variant in late 2027 with 512GB of RAM, that will be a very interesting chip. Tracking LLM size and performance improvements, I can imagine that being a sort of inflection point for local inference. I wonder what the power budget would be in desktop format.
A hypothetical M7 Ultra with LPDDR6 14.4Gbps memory would be 1.85 Tb/s.
You're look at about 100 tokens/s for a 1T MoE 37B active 4bit model.
It'd probably cost $30k or more I'm guessing if memory prices do not come down. Even at $30k, it could still be a relative bargain since an RTX Pro 6000 Blackwell 96GB card costs $12k today. The M3 Ultra with 512GB was around $8k before Apple discontinued it. I expect an M7 Ultra to have 768GB or 1024GB.
Apple Silicon Macs were on their way to becoming cheap local LLM machines relative to professional GPUs before this memory crisis. It may still emerge as such in a few years.
Here's some interesting math: At 512GB, an Ultra chip could make 42 pro iPhones. Assume a 55% profit margins, and $1200 ASP, you're looking at $28,160 in profit from making iPhones instead. No wonder Apple discontinued the M3 Ultra 512GB. If they only have a limited supply of RAM for all their products, it makes no sense to produce an $8000 M3 Ultra 512GB when you can produce 42 pro iPhones. You can only configure an M3 Ultra up to 96GB today as of June 2026.
Apple would have to raise the price of a 512GB Ultra Mac to around $50k to match iPhone profits.
> Assume a 55% profit margins, and $1200 ASP, you're looking at $28,160 in profit from making iPhones instead. No wonder Apple discontinued the M3 Ultra 512GB.
How would that work? They purchase 512GB from Samsung and then it doesn't matter if that's like 128x 4GB or 4x 128GB?
Note that this reserved capacity now has competition from OpenAI, Anthropic, xAI, Meta, Microsoft, Chinese data centers and so on, all willing to pay premium.
If comapnies keep spending half a macbook neo worth of subscription on AI plans monthly per person, Apple is going to have a hard time competing.
In British English the "an" is correct, even though most English dialects don't actually render the H as silent. It's a French-derived word that had a silent H originally, ergo we use "an".
I’d assume by next year the open weights models will be outlawed the way things are going nowadays :/
Edit: for those of you downvoting I don’t celebrate this prospect. I’m merely realistic about where things are going given the rapid vibe shift from the administration on AI since the start of June.
192gb or 256gb of RAM would be enough ! We could run real time large MoE models, REAPed for our usage (e.g. english agentic coding), dynamic quant 2-4bits
Apple is finally going to realize Jobs vision where sand comes into the factory, is turned into RAM and CPU chips, then installed in a Mac or iPhone then shipped to a customer.
Well yes. But similar to the Apple TSMC relationship, could Apple step in with large orders to established RAM makers such that the RAM makers can invest with stability?
No it isn't, DRAM is made with a different process and those are chiplets, perfectly possible to outsource, and the only possibility really as TSMC does not make DRAM.
Well yeah but NVidia just released a contender to their silicon and the M6 is probably already set in stone. Best to reshift resources to a great M7 than having a mediocre M6 and M7.
(This is assuming Apple will deliver, but this area is one of the biggest ones they have in AI, and they need the developer ecosystem to exist and survive)
Come to think of it, modern cars have a lot of electronics such as touchscreens, cameras, and sensors. It wouldn’t surprise me if new car prices are not immune to what’s happening with RAM and storage prices.
I'm pretty sure the vision/hwa reqs for cars is much less than an LLM/genai in general so that doesn't quite work out. But it would be nice to have an AI server with wheels :p
What's their backup plan if the AI world doesn't pan out? What if it turns out people want base compute capability and lots of RAM for filestore cache and programs?
Maybe this strategy works, even in that world.
Remember when we all thought (were told we thought) the world was heading to 3D views of our 2D lived experience like a solid Cube of GUI we could rotate around and live inside? Well Apple took the simple 2D square pane of virtual desktops and .. made it a SONY strip. One variable: sideways.
So here we are being told AI is the future. Apple seems to be saying "yes but it will run local" which might be a safe bet if AI comes true but I wonder how many of us want the AI outcome, which is morally speaking the 3D immersive GUI cube here: what if we don't want that?
I can't imagine any world where we put this AI stuff back in the box. It is simply too useful and too powerful. And as we start seeing all his upheaval where models are getting banned, etc, I can even see the appeal of on-device AI increasing for a lot of use cases.
So I think Apple has the right instinct. In fact, I've had the thought multiple times that I really want a lot of workflows just running on my device. Workflows like fast vector search (already fast on the m4, but I want it more common place), or realtime transcription and summarization to be even faster, on device, etc.
To me AI is on par with the internet and what made it so powerful was piracy and porn and just the wide spectrum of things that are possible when you connect machines together. We are going to need the same thing again. Freedom to use any model that does any thing we want.
They're still the only ones releasing any open model at all?
Plus, not like chatgpt is going to reply honestly about sensitive current hot topics in USA politics. Normally it has filters and refuses to respond as well.
The worst case scenario is that we're at a plateau and LLMs max out around here. And it'd stand to reason that if that happens we'd see local models catch up at least to some extent. Compared to 5 years ago, that's a pretty good world.
I doubt inference costs will scale up significantly, but even if they do, it simply strengthens the strategic case for Apple's focus on local inference.
Without AI everyone’s computing needs were pretty well satisfied with current phones and laptops. LLMs are the one thing that could drive new demand if they can run locally.
AI was the only reason I bought a new computer (a refurb M3 max with 64GB). Without AI, no idea what we should bother with, it depends on what application comes out to drive local computing power (maybe better games? Yawn).
this is the backup strategy. the "AI doesn't pan out" scenario is basically if claude and openai go bankrupt, we continue running local models on our hardware.
there isn't a future where we all just decide that nah, we don't want AI anymore. usefuly things don't disappear.
In between flat material-ish (hehe) design windows and 3d compiz cube with burn effects we have settled on transparency and blur effects with a bit of visual planes thrown in.
It's highly unlikely we will end up tokenmaxxing everything, it's highly unlikely the genie can lose enough weight to fit back into the bottle. We will end up somewhere in between that strikes a balance between nice, productive and cost effective.
Anything AI focused in silicon is also valuable for a ton of other use cases. If LLMs and GenAI don’t pan out, that silicon just gets used for other processing. Then they scale back on the dedicated die space in subsequent generations.
So I think it's fair to say that AI isn't going away. That doens't mean that SpaceX, OpenAI and Anthropic won't crash. But I've long believed that within 5 years we'll have access to relatively cheap hardware that can run sufficient but not cutting-edge models locally. You can buy a 5090 PC for <$5000 already so I guess it's already true but I think we'll do even better.
So what happens? Nothing. If Apple make M7 Max/Ultra computes with 128-768GB of RAM and nobody buys them then... nobody buys them. Apple isn't betting the entire company on AI just like Google isn't. The rest of the internals are the same Macbook, Mac Mini or Mac Studio. You're just selling something with less RAM.
I find it very unlikely that nobody buys them, some people will definitely buy them to run giant open LLMS. But there's also not much risk to apple because those configs would probably be made to order
Google sold (will sell?) about $70bn worth of Google shares to fund AI infrastructure build outs. It's also issued bonds (=debt; I forget the number, $30bn?) to pay for more infrastructure. Fairly sure it also has established a shadow company, a Special Purpose Vehicle (SPV) to stash away unpleasant financial things it doesn't want to show, also for the AI build out.
Amazon, Google, Meta, Oracle are overstretching at the moment. They are predicted to become cash flow negative (more money going out than coming in) if they keep going at this rate, some time in 2027 or 2028.
Now, they won't go bankrupt but it's possible they will be hit by huge restructuring waves once the dust settles.
Google has a market cap over $4 trillion and it's 2025 financials were $130 billion profit on $400 billion revenue, which was something like 15% Y/Y growth.
$100 billion in equity and bond offerings is not betting the farm on AI.
Additionally, Google's data centers are notoriously efficient and they build their own networking hardware as as their TPUs. Now their TPUs lag behind NVidia offerings obviously but they'll keep getting better.
Google is the company I'm least concerned about in the AI space.
As for Amazon, I'm not sure what their split is on in-house AI build out and usage vs AWS. I suspect the majority is for AWS and I also suspect their ROI is insanely short (ie less than 5 years) on any AI capex.
A lot of us on HN dismiss Oracle with good reason but it's also fair to say they've survived and thrived through the dot-com crash, the GFC and the pandemic until now. They're clearly doing something right.
Meta I think is the Sick Man of Tech. Their social media assets are of declining value (IMHO) and they seem completely unable to adapt to changing conditions. The whole Metaverse was a massive $70B+ boondoggle on something long before they had anything resembling a product-market fit. They seem unable to leverage their social media assets for building any competitive AI products or tech. Plus they seem unable to hire and retain key staff in this space.
AI isn't going anywhere, this is akin to the .com bubble. It burst, but the internet didn't go anywhere. While companies can fail, this technology is with us for the long run now, short of societal collapse.
> What's their backup plan if the AI world doesn't pan out? What if it turns out people want base compute capability and lots of RAM for filestore cache and programs?
I think reducing the die area dedicated to ai stuff is not going to be a problem.
And in fairness apple already has essentially ai-less hardware in the form of the MacBook neo and it’s been an astonishing success.
I have one and it’s a very good laptop, particularly for the price i paid it.
Do we have a choice? It's being forced upon us by folks who have the power to distort any market they want. Energy prices are rising, and the PC industry is about to be destroyed by component prices. It will be dumb clients that run the software our feudal overlords of the data centers will have the grace to grant us. And the government lets it happen because it furthers their interests.
As we all know, Intel used to be famous for their engineering and their ability to scale up a newer, smaller process with way earlier commercial viability. This all ended with the Sisyphean 10nm move that was years late and honestly Intel just don't seem to have recovered from it.
So Intel seemingly has underutilized fab capacity whereas the likes of TSMC and Samsung can probably produce every chip they make with demand to spare. Given the CHIPS Act that was passed under Biden, the Trump admin taking a stake in Intel and the environment of tariffs and a push for American manufacturing, everything seems to be lining up for someone to take advantage of Inte's physical fabs and American production and that could be Apple.
It's not so much ripping off the designs - nothing of what Apple Silicon is doing is particularly surprising and both x86 and Intel's microarchitectures are sufficiently different to Apple Silicon/ARM that knowledge of specific implementation approaches wouldn't be directly useful in most cases.
The real advantage is knowing exactly what Apple is launching months or years in advance, because that can inform strategic planning.
M1 released 6 years ago, but AMD/Intel still can't get close to ARM cores in IPC. Anandtech was observing that Apple had better IPC in their phone chips YEARS before M1. Lots of people discredited it as "apples and oranges" because the ISAs were different, but investigative teams from Intel and AMD absolutely HAD to know the truth.
This has both a technical and human component.
On the human side, top x86 execs refused to see any threat coming. They must have thought Apple couldn't overcome the x86 software moat, thought the chips were for servers, consoles, or some other non-PC device, or perhaps they simply couldn't believe what their investigative teams told them.
At the same time, we're 6 years post-launch. The proof of ARM's capability is clear. x86 server marketshare is about to hit just 50% and Microsoft is pushing ARM hard as a replacement for x86. Either all the x86 engineers are completely incompetent and incapable of learning from years of ARM designs or there are aspects of x86 that makes copying those designs infeasible.
> M1 released 6 years ago, but AMD/Intel still can't get close to ARM cores in IPC.
> teams from Intel and AMD absolutely HAD to know the truth.
These people are professionals that acknowledge IPC is a stupid metric. If you switch your statement to SIMD throughput, now ARM NEON has the lower IPC and x86 looks like space age technology. They're optimized for different workloads.
x86 vendors recognized that they could recoup the majority of efficiency that Apple Silicon has without buying an architectural license for ARM. Intel invested early on big.LITTLE, and AMD drilled down on denser nodes for their preexisting designs. As both businesses converge on each other's ideas, their SOCs have adapted most of ARMs' greatest mobile innovations. Even before that, x86 hardware was always usable - AMD was shipping faster integrated GPUs than the M1 Pro before the M1 ever hit shelves.
All of this makes sense, nothing objectively prevents the x86 architecture from being power-efficient. Arm LTD. would have gouged any of those vendors for their IP, and even with an architectural license it's not like AMD or Intel would get usable core designs from Arm. There was no reason to pivot to ARM for either company, they both saw Qualcomm and could read the writing on the wall.
> x86 server marketshare is about to hit just 50% and Microsoft is pushing ARM hard as a replacement for x86
That's Nvidia's work, no credit is due to Microsoft or Apple for reshaping the server market. Apple's early ARM hardware was outright ignored for server/HPC applications, leading to the discontinuation of the Mac Pro. Apple was entirely incapable of pivoting their mobile chipsets to the server scale, surprising nobody that had paid attention to Apple's godawful raster/GPGPU acceleration stack. The Ultra hardware looked like a dog's dinner compared to x86 arches like CDNA.
The Graviton and Grace chips that displaced x86 servers did it because they are slower, cheaper and less feature-dense. Graviton for the bare minimum of Raspberry Pi-tier web serving, and Grace for the high-end of "we need CUDA and enough bandwidth for Infiniband" that made trillions in the HPC market.
> The real advantage is knowing exactly what Apple is launching months or years in advance, because that can inform strategic planning.
While I'm sure some level of internal leakage does take place, at least on paper the fab's planning needs to be firewalled off from their own chip roadmap.
I'm also not sure how much Apple actually cares, tbh. Yes, they currently have an edge in silicon, but it's heavily due to being willing to outspend everyone else, and their real superpower is vertical integration - which Intel isn't in a position to compete with.
I think Apple doesn't really have a choice. They've been very strongly encouraged by the current US government to move as much chip manufacturing to the US as possible, and particularly to make Intel Foundry work, or face... problems.
Also the AI boom means NVIDIA et al. can afford to buy TSMC's best processes at scale, which means less available capacity for Apple.
I'm sure given no other forces at work, Apple would prefer to stick with what they were doing previously, buying the lion's share of TSMC's best process.
I mean in the opposite sense. If Intel can glean enough from Apple's roadmap to close the performance/watt gap, great, but they still can't match the vertical integration Apple has
Seems like a made-up distinction that shouldn't be necessary since M6 has not even released. I suspect this is a marketing ploy to meant to drive up both interest while also increasing prices for the next generation of Mac hardware.
What it's saying is that the M6 will be released, but not the M6 Pro or M6 Max. Instead, Apple will wait to release new Max/Pro chips for a future generation.
It's not simply marketing since the Pro/Max chips of a generation use the same cores as the regular version, just more of them or different combinations of performance and efficiency cores.
> Seems like a made-up distinction that shouldn't be necessary since M6 has not even released.
The claim is that M6 will be released, but the only variants will be lower end.
When they get to the M7 generation, they will make high end variants.
It's a real distinction because each generation of parts shares an architecture.
The article has an entire section speculating what the M6 parts will be, but says they'll top out around 200GB/s memory bandwidth and 12 graphics cores.
> Seems like a made-up distinction that shouldn't be necessary since M6 has not even released.
Why would it? Each generation of the M series has an architectural improvement on their chipsets. The difference between an M1 and an M1 Pro is the allocation and arrangement not the architecture. M6 to M7 presumably will have architectural changes.
This is no different than them skipping the “Ultra” chips on some generations. The only real difference is it going all the way down to skipping the “Pro” line. So, only the MacBook Air, low end MBP, and maybe the iPad Pro and Mac Mini get the M6.
Made up how? They'll do a refresh of lower end devices, but not the high core count versions.
It's the same thing as how the Mac Studio got an M4 Max refresh, but they didn't make an M4 Ultra so if you want the 28+ core CPU or 60+ core GPU, that's still using an M3 Ultra.
This time it'll be across all the Pro, Max, and Ultra versions, if you want those they'll stay at the previous generation for the M6 cycle.
Not that weird - Apple has a huge set of chips and hardware and software products. Putting every single thing on a fixed identical update cycle together won't always make sense.
Except that is not what's happening. The article clarifies something that is misleading if you interpret the headline in isolation: "high-end M6" means "the high-end variants of the M6 line", not "the entire M6 line".
Whether it matters for the consumer (who only sees released and announced end results) or not is irrelevant.
It can still be a very real, not made-up distinction, if the actual facts on the ground are that Apple designed an M6 line, but then scrapped that design and asked the team to create a new design with emphasis on AI-focused specs.
It's not the name that's important (the M7 could still come out as M6), is them skipping a design, or cpu "Tick-Tock model" step.
Perhaps to support demand for the products with recent price hikes, and/or the upcoming Mac Studio with M5 Ultra, rather than have customers sit on the sidelines thinking they'll wait this generation out.
I am still skeptical that Apple intentionally leaked this because they normally are so tight-lipped, but there are reasons in favor of leaking this.
Well, I guess this is the silver lining to the price increases. I'd been thinking about an M5 128GB for local inference (eg DS4), probably off the table now given that it jumped $2k overnight. But I was on the fence about it for a long time given that even the M5 is not that good compared to even a 4090. It would have been good, but not "omg" good.
If they are pulling out all the stops to make the M7 more competitive.. guess I can wait for that?
In the long run I truly believe local AI will win and Apple will be the world's most important AI company because of these chips. Imagine something like today's Opus running for free and in complete privacy on your local machine with a beautiful Apple UX on top. For most tasks for most people, that's a much better proposition than a frontier model in the cloud you have to pay for and send all your data to and that only works when you're online.
>In the long run I truly believe local AI will win
What do you mean by 'win'?
For a normal coder/person's use cases, yes. But AI companies are becoming more specialised in different fields and these tailored models will be leagues ahead in those niches.
The way I see it - Opus 4.8 xhigh can do any programming task with a programmer instructing it. If Apple releases local model together with a device that can run said model it would render OpenAI/Anthropic useless for vast majority of usecases.
And if a local mcahine can run something like Opus 4.8, who is to say that those "specialized" models would just not come at a later date, or even loading open models wouldn't be an option with something like M7-verified flag from huggingface that would make it extremely easy for any consumer to just play around.
There is built-in demand for local LLMs. An obvious example is law firms where using remote AI tools may be breaking privilege [1]. Any medical applications may likewise run into legal issues.
The problem is basically that we can't have nice things. AI chat logs themselves become another commodity to sell and to train on. We recently had a story about how Chinese firms are reselling Claude tokens [2]. The chat logs are a commodity here.
The only way to avoid this is to run LLMs locally. Even if you trust someone like Anthropic or Google, case law simply hasn't been established that the chat logs aren't discoverable.
Add to that that a sub-$5000 PC with a 5090 can already run a 31B model at reasonable inference speeds. Not amazing but good enough for many applications. Obviously that can't compete with Mythos but it doesn't have to. It also shows where the trend line is going for hardware. A $10k Nvidia GPU from 10 years ago now sells for scrap. What a consumer-level computer in 5 years can run locally will probably shock a lot of people.
I would say local AI is very real. I use it but so many here am on other forums do so nowadays as well. This is the reason I just cannot fathom the valuations of the AI firms out there.
The M7 Pro and M7 Max are scheduled for as early as the end of 2027, while the M7 Ultra is on track for 2028.
This means there won't be a redesigned MBP this year since there won't be M6 Pro/Max chips. People were expecting a redesigned slimmer MBP with OLED display later this year, myself included.
I was holding out for one until I decided to switch from an M1 Pro 16" MBP to an M5 Air 15" due to the expected price increase. I think many M1 Pro/Max generation people were waiting to upgrade this year.
Current MBPs are such a delight, I really don't want to think about a thinner MBP again, I just get shivers remembering the Ive butterfly keyboard models
I can see why people would want a more powerful machine but as someone who moves around a lot, the 16" MBP weight is a pain. The 14" MBP screen is not big enough.
Isn't that switch basically a downgrade? You get some more single core performance and some weight savings, but also a worse (and smaller) screen, less multicore performance, less GPU performance, less video encoding performance and a smaller battery? I'm on an M2 Max myself, and glad they introduced a larger form factor Air, but it seems like a long way from an upgrade.
M5 is faster than the M1 Pro in ST, MT, GPU. Not sure about video encoding as it's something I rarely use. It's a smaller battery but overall a battery life improvement since my 5 year old M1 Pro only had 79% battery capacity left.
The optics and marketing is already fucked, the MBP goes to M5 Max, the Mini has the M4, the Studio has M2 or M3, the iMac apparently has two different kinds of M4s, it's all fucked.
Mac mini Pro line is doomed, they never made enough of it; skipped M5 Pro, now skipping M6 Pro, it is like 2014-2018 again. Now ordering a custom M4 Pro build take 3 months+ to ship with an increased price.
I was waiting for a MacBook Pro M6 Max and now I don’t know what to do, especially with the price increase I feel like I really screwed up not just getting an MBP M5 Max a month ago
Apple to skip high-end versions of M6 Mac chips...
I read it as the M6 being "high-end" in general, and Apple skipping the whole generation, which made no sense to me. But they are going to use the M6 at all, just not bother to create Max and Ultra versions of it.
Given that M6 will be on TSMC smaller 2nm node and the first smaller node size in 3-years, it seems like the oddest of all years for the high-end Macs to skip.
Well this kind of sucks. I've been waiting for the M6 MBPs because they're rumored (strong rumors, though) to finally remove the notch that has been a historic self-own. But it sounds like I might as well wait longer for the M7 lineup. Or maybe get a Framework Pro instead.
I'm not even convinced this is a bug. Menu icons proliferated over the last couple of years, this isn't a problem unless you enable more than you have room for.
There’s so many annoying bugs in Mac OS (like the screwed up window management and alt-tab not working properly), that the notch seems like an odd complaint at this point. The OS is fighting the user constantly, and there’s not much we can do…
I'm a huge fan of Star Trek, David Warner, and Patrick Stewart, and I'm really not. The notch isn't taking anything away, it's a way to give users more screen space than they could otherwise get. Normally the camera has to be above the screen, this was a way to get the menu bar off of the rest of the screen real estate and get you more usable pixels.
It’s a complete embarrassment. They added it for aesthetic alignment with the iPhone 13. And then the 14 removed the notch soon after. They’ve kept it for years since then. It has no functional purpose. It’s not there for face ID or because they couldn’t figure out how to do a hole punch camera.
I agree. It was very annoying to me to spend the money (and on the nano matte one too) and still have that stupid notch. But it never makes any difference at all which is good news.
Same, have a very old MBP. Not sure what to do because I don’t want to wait a year and a half. That coupled with today’s price increases make it a tougher decision.
So the big question: Is this an excuse to save on memory costs and delaying stuff like the M5 Ultra, M6 Max, etc until 2027 when memory prices come back down?
because America can't compete. Build a fab in the US, labor unions, labor costs, regulations, land, energy, taxes, government, water, etc all make this not economical. Everything would cost twice as much and you'd rather buy the cheaper product and it'll be bankrupt. There were reasons why all the manufacturing went overseas to Asia. You're right, the demand right now is HUGE but it won't always be huge. At this point, we don't have the talent or the knowledge to do it well anyway which is why we needed TSMC and Samsung to bring employees over to train people. https://www.cppionline.org/wp-content/uploads/2017/07/The-De...
I am waiting till apple copies the "allocation" concept from high end car manufacturers. "Sure, buy the 25 iphones ans we will gladly put you on the waitlist."
And it's not like Apple hasn't dabbled in the luxury space, with utterly predictable results. Anyone remember the 24 carat gold plated Apple Watch Series 0, sold at ultra-high-end luxury boutiques?
Do you really think the average Apple user will use it when there’s already better AI provided by OpenAI and Anthropic which don’t require advanced local hardware?
I was just countering your argument with an equally compelling counter-argument.
As for how it helps: we're not talking about this year's AI ecosystem, or even next year's. This rumor, assuming it's true, is talking about two chip generations into the future — and probably at least three or four chip generations before it's a mature AI platform. What will AI be doing for us in five years from now? How does Apple plan for that future? Will concerns of privacy increase or decrease in that time?
1. NVidia aggressively segments the market on VRAM and will continue to do so. A 5090 with 32GB of RAM, ~21k CUDA cores and 1800GB/s of memory bandwidth is $3-4k. An RTX 6000 Pro with 96GB of RAM, ~24k CUDA cores and 1800GB/s memory bandwidth is ~$11k;
2. The 5090 won't be replaced until late 2028 or even 2029. There has been no mid-cycle refresh (eg 4080 Super vs 4080) and likely won't be either at all or for at least a year. If there is in a year, it basically confirms that the 6000 series won't be until 2028/2029. Also, the x090 never got a mid-cycle refresh so the current consumer high-end is staying that way for years;
3. The 6090 whenever it comes will still have 32GB of VRAM unless the memory market drastically changes;
4. Many have anticipated an M5 Max/Ultra refresh of the Mac Studio line in Q3. Given that Apple chose to hike the prices on Studios rather than discontinue them, I now think this isn't going to happen. We may not see a Studio refresh for up to 2 years. Apple has done this before with the Mac Pro;
5. M7 Max/Ultra will probably go to a memory bandwidth of 1.2-1.8TB/s vs the current tops of M3 Ultra, M4 Max and M5 Max of 600-900GB/s. This simply needs to go up to boost inference speed;
6. You'll also see the number of GPU cores go up. All of this will add up to an M7 Max being 50-80%+ of the performance of a 5090. That's huge given the shared memory architecture;
7. We may see the return of Apple using its massive cash pile for vendor-financing of an exclusive memory supply. This was one of Tim Apple's [sic] big innovations.
Apple is very late to the AI party. By the time M7 is shipped, Nvidia will announce 6090 and people will be buying used (3|4|5)090 GPUs to run local models at much better performance than heat throttled M7.
I would prefer a Studio if it does a decent enough job even if throttles a bit under load, way less power usage and noise than those GPUs plus the PC you need to put those in.
RAM is a commodity and nvidia will be paying the same prices. The used market will reflect the cost of RAM. nvidia owns the top of the market but many of us don't need that.
What people? Are you seriously thinking the hundreds of millions of customers Apple have is going to be buying run-to-the-ground GPUs second hand and build local workstations for AI? Might as well ask them to self host email while you’re at it.
The difference between these two is that one of them is an unsolved research problem that we’ve all spent far too much time on, and the other is just running an LLM.
apple needs to skip Israeli chips... they cannot ignore the collapse of Brand Israel without stock damage and reputational catastrophe... NVidia is at catastrophic Israel exposure already
Apple isn't just transitioning to TSMC's 2nm node, they are also transitioning to a chiplet based design using TSMC's advanced packaging.
> What sets the A20 apart isn’t just the node shrink—it’s the revolution in packaging. Apple is transitioning to Wafer-Level Multi-Chip Module (WLCM) integration, meaning that RAM will no longer be situated beside the chip, but rather on the chip wafer itself, integrated alongside the CPU, GPU, and Neural Engine.
This shift eliminates the need for silicon interposers and substrates, thereby enhancing signal integrity, improving thermal dissipation, and facilitating faster memory access with lower latency. The benefits? Better multitasking, smoother AI processing (hello, Apple Intelligence), improved battery life, and potentially a smaller chip footprint—freeing up space for other components.
Do we have any explanations of what WLCM means that are more industry focused? I couldn't find anything that didn't look like blogspam. And that explanation of the DRAM being on the same wafer doesn't really make sense. For one, at that point there's no "multi chip" part if you're integrating more onto the same die rather than less.
And their explanation isn't really passing the smell test for me for other reasons, for instance the fact that DRAM processes are pretty radically different than bulk logic processes, which wouldn't really let you put it all on the same wafer, much less the same die. Even back in the day when you had eDRAM blocks (like the Xbox 360's eDRAM die), that was really a DRAM process with a bit of logic cells that wouldn't be competitive if they weren't sitting right next to the DRAM blocks.
I could be wrong here though, my examples are more than a bit long in the tooth.
The terms to search for are fan-out wafer level packaging (FOWLP) and TSMC InFO. The chiplets come from different wafers and are reconstituted into a molded plastic wafer, allowing multiple die side-by-side. Then multiple layers of wires are built on top, terminating in a BGA.
Ok, part of my confusion was that it was being presented in contrast to InFO-oS and InFO-PoP, but it appears to mostly be a modified version of InFO-PoP called InFO-M? Because Apple has been using InFO-PoP for almost a decade at this point, starting with the A10.
You can start by reading up on TSMC's name for the tech (although there are many versions at TSMC and TSMC isn't the only company packaging chiplets and memory on top of a silicon interposer).
InFO combines chiplets, but does not do so by bonding those chiplets directly to a silicon interposer as CoWoS does.
The article compares CoWoS to InFO.
> CoWoS uses a passive silicon interposer—etched with thousands of fine interconnects—that sits between the active dies and the package substrate. This interposer provides high-density connections between chiplets and supports memory integration, most notably High Bandwidth Memory (HBM) stacks.
It also mentions that CoWoS is the tech used to build Nvidia's high end AI accelerators.
> AI Accelerators: Nvidia H100, B100 (HBM3 via CoWoS)
Apple is taking a tech that has previously only been used in very high end enterprise applications and is using it to make consumer SOCs starting with this years iPhone.
So far the only thing I've seen useful out of apple intelligence is running parakeet natively and effectively... which should have been their very first feature... given it's been on phones for 10+ years.
As someone who wants to run effective llms locally for many things their other big benefit has been the unified memory studios for a small bit.
A kind request - please try to write HN replies without AI, but if you're going to, please at least edit out any "it's not X its Y" or "isn't just X, but also Y" AI tics. A lot of us come here to get away from talking to AIs all day.
‘isn’t just / it’s also’ AI-ism should at least turn your AI radar on, then you get this weirdly formal structure that sounds like a trying-to-be-relatable press release:
“This shift eliminates A, thereby enhancing B, improving C, and facilitating D. The benefits? Better U, smoother V (hello, W), improved X, and potentially Y—freeing up Z.”
Question to self interjection, chipper ‘(hello, W)’ aside, topped off with a zero spaced emdash. 10000% AI, stylistically this isn’t text tuned to an HN audience, comments never sound like this. What’s funny is the paragraph it’s quoting has nearly the same style, the LLM probably picked that up.
Not trying to call anyone out, just pointing out the stylistic tells we should all be aware of.
768GB is 64-times of 12GB which is rumored to be amount of RAM in new iPhones. Imagine what profit margin 768GB Mac Studio gonna need in order to justify making one instead of 64 iPhones.
Apple is the company that is okay about selling microfiber cloth for $100 and wheels for $700. Imagine how bad price hike for M3 Ultra 256GB / 512GB had to be in order for them to just discontinue them instead of getting free money out of desperate local AI folks.
reply