Weird how the title says, "macOS Sequoia is available today, bringing iPhone Mirroring, Apple Intelligence, and more to Mac"
Then you look at the paragraph for this and it says, "Coming Soon: Apple Intelligence"
They announced this in June, and it's just loosely coming "this fall." I have a feeling they are finding it very difficult to do anything with any kind of consistency or value on their 8gb RAM phones / Macbooks. Their models they have demonstrated so far are a relatively low parameter count with LoRA adapters for various use cases.
I have a feeling that the majority of Apple Intelligence will eventually just be farmed out to their "secure cloud" or whatever they were calling it.
I wouldn't be surprised if they're struggling full-stop. Doing anything with "apple-level" polish and reliability on current-gen LLMs is a challenge. It looks great in demos and maybe even 95% of real-world use cases, but the remaining 5% tends to be embarrassing.
And it's not just a case of "the last 5% takes 95% of the work", it's more like "the last 5% is an open research problem".
That is the biggest hurdle, in my opinion. If we could even reply with, "sorry, I don't know about that", it would be such an improvement over what we have today. Sadly, from what I understand, the only way to say "sorry, I don't know about that" is to just say that to every single question?
There's no specific reason why LLMs couldn't be trained to say "Don't know" when they don't know. Indeed, some close examination shows separate calculation patterns when it's telling the truth, when it's making a mistake and when it's deliberately bullshitting, with the latter being painfully common.
The problem is we don't train them that way. They're trained on what data is on the internet, and people... people really aren't good at saying "I don't know".
Applying RLHF on top of that at least helps reduce the deliberate lies, but it isn't normal to give a thumbs-up to an "I don't know" response either.
> There's no specific reason why LLMs couldn't be trained to say "Don't know" when they don't know.
Yes there is, it's that we don't know how. We don't have anywhere close to the level of understanding to know when an LLM knows something and when it doesn't.
Training on material that includes "I don't know" will not work. That's not the solution.
If we knew how, we'd be doing it, since that's the #1 user complaint, and the company that fixed it would win.
Do you think it's really a training set problem? I don't think you learn to say that you don't understand by observing people say it, you learn to say it by being introspective about how much you have actually comprehended, understanding when your thinking is going in multiple conflicting directions and you don't know which is correct, etc.
Kids learn to express confusion and uncertainty in an environment where their parents are always very confident of everything.
Overall though, I agree that this is the biggest issue right now in the AI space; instead of being able to cut itself off, the system just rambles and hallucinates and makes stuff up out of whole cloth.
> Do you think it's really a training set problem? I don't think you learn to say that you don't understand by observing people say it, you learn to say it by being introspective about how much you have actually comprehended, understanding when your thinking is going in multiple conflicting directions and you don't know which is correct, etc.
I really do think it's a training set problem. It's been amply proven that the models often do know when they lie.
Sure, that's not how children learn to do this... is it? I think in some cases, and to some degree, it is. They also learn by valuing consistency and separately learning morals. LLMs also seem to learn morals to some degree, but to the degree they're even able to reason about consistency, it certainly doesn't feed back into their training.
---
So yeah, I think it's a training set issue, and the reason children don't need this is because they have capabilities the LLMs lack. This would be a workaround.
iOS 18.1 and macOS 15.1 -- the versions with the full Apple Intelligence -- are both in betas, and there are enormous numbers of people blogging and vlogging about their experiences. And yes we all realize that on-device models aren't going to be competing with trillion parameter models, but people are finding it pretty useful.
They just don't feel like it's release ready so they're refining. Apple has done this with major releases over several iterations now where a couple of features are held back for the .1 release while it's perfected.
At least in Photos, the model is not what one would consider working. A quick example: "Halloween with [x]" works, because Halloween is recognized as a specific date. It works well. Christmas throws it for a loop. Christmas isn't thought of as a time but a state. So it'll show photos that look appropriately Christmassy, but the picks appear semi-random: it'll show one of about five taken in quick succession, but the one it picks is someone wearing a Santa hat in profile, but the other 4 are straight on and one would think more likely to be returned.
I don't think anyone can say with a straight face that being unable to properly grok the biggest holiday of the year for many Western countries is sufficient.
That isn't "Apple Intelligence" (which is the generative stuff held over to the .1 releases). What you're describing is inference metadata + basic search logic and has been in iOS and macOS for several major versions now, constantly improving.
And given that Christmas resolves to a date (it actually offers up the autocomplete of "Christmas Day" to make it easier, then simply making the search criteria a calendar date), for me it literally shows all photos on Dec 25. I guess mileage varies.
Well, except Apple specifically calls out this exact photo and video search function on their page titled "Apple Intelligence." Sure they've had some basic search already, but in their demos and advertising, they promise that Apple Intelligence is used to find photos by descriptions.
>Search for photos and videos in the Photos app simply by describing what you’re looking for. Apple Intelligence can even find a particular moment in a video clip that fits your search description and take you right to it.
If we accept Apple's claim that Apple AI is coming in beta in fall 2024 (or per the Canada page December 2024), and I have the release of 18 which by that restriction is not Apple AI enabled, I can already do complex semantic search which means that functionality can't be "Apple AI", right? And person on date has been in iOS for at least two prior generations as well. iOS has been allowing you to search by people in images, places, events, and text appearing in the image, along with broad categorizations like "sunset", "beach", etc, for at least two generations.
However when you're typing in a search, for each term it tries to contextualize it via selections. For instance if you already had "{Person} on " and then typed Christmas, it will let you pick if you mean Christmas the event, Christmas an "object", or Christmas the literal text (an icon of a couple of lines of text in a photo frame). I suspect the poster selected, probably unintentionally, Christmas the text and it gave them images where that text appears somewhere in the image. Just out of curiosity I did that and it gave me a set of images that I thought must be mistaken, but on each image somewhere the text Christmas could be found. In one it was a crazily distorted cursive writing on a table cloth hanging over the edge of a shelf, which is just crazy.
"iterated" implies that there is no improvement. Why do you thing 15.1 wouldn't be an improvement over 15.0? I do agree with you that "perfected" is also not the correct word choice. I think I would have gone with "refined" or "improved"
Unless it can be completely turned off I will never upgrade and I guess I will be selling my year-old M3 Max in favor of some shitty PC (or I’ll eventually just run Asahi once it supports my hardware well).
Apples pretty decent about putting toggle switches on stuff, for instance you don't have to enable iCloud or even associate it with an Apple account if you don't care for FindMy and remote erase etc
But I'm with you, since Apple signaled going all-in on "an assistant that has access to everything" I switched to an android with the intention of never enabling Google services and certainly not the voice assistant. Unfortunately I've found it too annoying to go completely without Google, I've read that RCS messaging won't work with an unlocked bootloader, nor will precise location, so I'm stuck with some evil in the name of features parity.
I find that a little hard to believe. I would suppose the vast majority of customers don't think a lot about the timing of software features in a purchase decision.
absolutely anecdotal, but from my experience, people who are normally excited for iPhones really don’t care about AI at all.
i think Apple hyping AI so much was a mistake, doesn’t have nearly the same impact to them as hyping up a new camera feature or a new screen or a new color or something.
The same AI features are coming to the entire iPhone 16 line. The major benefit of the 16 Pro, just as with prior generations, is a faster SoC and better cameras.
The only reason the iPhone 15 Pro was the only of the 15 generation to get the AI stuff is that they happened to put 8GB in that device, giving it enough headroom to not majorly impact user enjoyment trying to cram a multi-GB model into memory. But all of the iPhone 16s have 8GB (+?)
Ming-Chi Kuo is the golden standard for reports on Apple's production apparatus. He has the best sources, the best leaks, the best insight into very specific discussions regarding parts acquisition for Apple.
When it comes to what Apple does with those parts, or market segmentation, or sale projection, he's got no special knowledge. In this specific case, pardon the technical language, but he's full of ...
It’s not as if the hardware features on the 16 Pro are groundbreaking — a slightly nicer camera, a slightly nicer display, a slightly nicer battery life, and a $600 upgrade fee. No, thank you, the software features are what make-or-break the upgrade.
Phones nowadays are made to last years, even 6+ years easily. So the update this year is not for the people who bought it last year but for people with an iPhone X or 11. They will see major improvements. But it’s silly to update (any) flagship phone every year.
Okay, but then, why upgrade to a 16 Pro instead of the much cheaper 15 Pro? The price difference applies there too — there needs to be some reason to get the newest, most expensive one. This generation it is not hardware.
Well this applies to everything then. Why buy the latest AMD or Intel CPU or Nvidia GPU ?
Some people upgrade from iPhone X to 15 this year because it’s cheaper. Some have an old iPhone and just want the latest model to go for 6 or 7 more years.
Even incremental upgrades like this year are welcome. Before there was an S model every other year. This year it’s the « iPhone 15S pro ». Slightly better and more powerful. The other thing is, I think it’s pretty hard to say « well last year model was good enough, we’ll go on a 2 year cycle this time ». It could send the wrong message to investors so they’re trapped in the race maybe ?
Usually because there’s some advantage — you actually need/want the performance offered by Zen4 over Zen3, you need four-cycle AXV512, you have a target framerate in a game, you need RTX or CUDA14… because if you don’t, obviously buying older and used saves you money and you don’t need the new stuff anyway.
> I would suppose the vast majority of customers don't think a lot about the timing of software features in a purchase decision.
The ones who buy on day one do. I don’t find it surprising that the people who even know about and anticipate the release date of a product are the ones who are eager for specific features. If those aren’t there yet, there’s no rush to buy.
I’m no analyst but I expect demand in the EU won’t be that (comparatively) great either. When the flagship feature isn’t available, there’s little reason to spend the extra money.
I would guess that the vast majority of customers don’t even know when a new iPhone is released and therefore don’t buy in the first few weeks after it’s released. Of the customers that are aware of the release, I would assume that most are aware because they’re anticipating new features
My understanding is that 8GB is plenty enough to play with small models (7B or so) in a lightly quantized version. (Especially since model params are kept on disk and only really "mapped" to RAM, with it acting as a cache.) 4GB is where it gets dicey, as you're constrained to tiny models that just don't do anything very interesting.
Thing is, you do need all the params usually, so if the model is only partially mapped to RAM, it's the equivalent of an app swapping in and out as it runs. Which is to say, it means that inference is much slower.
Local Apple models are likely in the 2-3B range, but fine-tuned to specific tasks.
I think it's actually easier on macOS because you can just swap everything if necessary. For some reason I don't understand, iOS does not employ swap, so while it can kill background apps, it can't kill foreground app.
That is true. I'm not sure why iOS doesn't do that while macOS does. Maybe mobile storage isn't performant enough? At least when iOS was originally developed.
This is such an Apple brainwash, like how many apps in background you think has everyone else, my linuxbox has almost none, the rest is just non-linux from third party, unless you only install apps from Apple and like having a subpar experience, everything you install is just from third party on which they have no control on. And killing apps doesn't mean anything for AI, it doesn't run anything useful on 8GB, doesn't run anything useful on 8GB, no matter if you kill all the processes. I wish I could just have a conversation where people doesn't cyborgically repeats Apple's marketing bullshit and turn on the human brain
Because other OSes don't kill OOM apps? Or any app they want? It is only Apple and only possible on mobile? The only thing that apple has, is the inability to have background apps, so that if you want to use third party sync services, like syncthing, you can't and have to stick to apple products, the rest is just like anyone else
iOS isn’t killing OOM apps. It’s killing inactive apps. It’s something that wouldn’t fly on a desktop or server OS under general use, but works reasonably well in the mobile space.
Then you look at the paragraph for this and it says, "Coming Soon: Apple Intelligence"
They announced this in June, and it's just loosely coming "this fall." I have a feeling they are finding it very difficult to do anything with any kind of consistency or value on their 8gb RAM phones / Macbooks. Their models they have demonstrated so far are a relatively low parameter count with LoRA adapters for various use cases.
I have a feeling that the majority of Apple Intelligence will eventually just be farmed out to their "secure cloud" or whatever they were calling it.