Cool project, also curious about the economics. Have been thinking recently with the increasing quality and affordability of 3d printers and assistive benefits of LLM's it would be cool to basically have the equivalent of lego builds kits for all kinds of consumer goods.
If it really costs them 30x more surely they must plan on putting pretty significant usage limits on any rollout to the Plus tier and if that is the case i'm not sure what the point is considering it seems primarily a replacement/upgrade for 4o.
The cognitive overhead of choosing between what will be 6 different models now on chatGPT and trying to map whether a query is "worth" using a certain model and worrying about hitting usage limits is getting kind of out of control.
Wow you aren't kidding, 30x input price and 15x output price vs 4o is insane. The pricing on all AI API stuff changes so rapidly and is often so extreme between models it is all hard to keep track of and try to make value decisions. I would consider a 2x or 3x price increase quite significant, 30x is wild. I wonder how that even translates... there is no way the model size is 30 times larger right?
Per Altman on X: "we will add tens of thousands of GPUs next week and roll it out to the plus tier then". Meanwhile a month after launch rtx 5000 series is completely unavailable and hardly any restocks and the "launch" consisted of microcenters getting literally tens of cards. Nvidia really has basically abandoned consumers.
AI GPUs are bottlenecked mostly by high-bandwidth memory (HBM) chips and CoWoS (packaging tech used to integrate HBM with the GPU die), which are in short supply and aren't found in consumer cards at all
Although you are correct, Nvidia is limited on total output. They can't produce 50XXs fast enough, and it's naive to think that isn't at least partially due to the wild amount of AI GPUs they are producing.
No, but the supply constraints are part of what is driving the insane prices. Every chip they use for consumer grade instead of commercial grade is a potential loss of potential income.
It's a real shame the memory isn't a 512 bit bus or higher for comparable bandwidth to apple's M max series. If they had accomplished that at this price they would sell like crazy I think for AI. Having the downsides of soldered ram without the upsides of pushing it to a much higher bandwidth is a bummer. I assume that is on AMD though and not something Framework could control. Regardless happy to have more options. I really hope there are more options in the near future for integrated boards with a large variety of bandwidth choices and ram capacities.
I think this is much more on a botched rollout and a lack of vision. Apple should have been working on this for years and have some pretty big advantages. A deeply integrated version of the best o1 chat and vision models with context from all various apps and ability to do complex app interactions could be amazing. They could even have the ability to have AI basically create on demand personalized new applications for specific use cases. They could integrate it to be a proactive AI agent regularly interacting with you regarding various goals (instead of just answering questions). Instead so far we get the most barebones largely unimpressive features like basic text summaries and toy things like image gen and they fail to have even the modest features announced in June ready for latest iPhone launch earlier this year.
This wouldn't align with what Apple is going for though. Their selling point in this category is first and foremost that your data stays private. They're looking to do as much as possible on-device, not in the cloud. This rules out models akin to o1 because they're just too large in every respect. I agree that Apple has the chance to do amazing things here, but I think it will take a while until small models are as capable as they need them to be.
The problem is they compromised on both fronts (privacy, compute) and produced something that is ultimately a useless joke.
They could have produced something that was good and required accepting a privacy disclaimer to use. Or they could have waited until they managed to catch up with the rest of the industry wrt good 3B parameter models (the one they're using is incredibly terrible.)
Instead we got this half-assed compromise.
The execs in charge of Siri are demonstrably terrible at their job and should feel bad. It's incredible they get paid the millions they do while doing such a horrible job.
I agree. This feels like the norm now a days though. An initial soft launch, iterative improvements from there. If something catches on, great! If it doesnt, less money/time spent upfront investing in it.
I agree, but based on their software track record over the last couple of years, I have little hope that we’ll get much beyond a disparate collection of mostly half-baked features.
> I have little hope that we’ll get much beyond a disparate collection of mostly half-baked features.
Really? AI has driven most of the interesting features they've released in the last decade. Notably, organization of the photos app and indexing of people, pets, places, etc. (Nb I have not yet updated to ios 18 and I understand the photos app is somewhat controversial.)
The GP comment was calling for complex context-aware cross-app functionality. The new Photos app features do have their use, but they are limited to the particular compartmentalized use cases envisaged by Apple. For example, they support cats and dogs specifically, but not pets in general.
There’s also the tendency that once a feature has shipped, it’s apparently considered “done”, and the lead developers likely move on to other things. It doesn’t feel like there is a continuous ownership and coherent vision behind the features and apps, or behind the overall OS trajectory.
It's strange to me that relative performance would even be a primary factor for defining performance as "excellent". You could be number one on the list and still deeply unsettled if performance is dropping or stagnant. I would want to focus on generational improvement targeting say 10%-20% improvement across the board every 10 years or so.
Depends on the system. This would be my second to last choice as an engineer. First choice should always be design the system so you can't get hurt in the first place - but nobody has any ideas on how to do that to a table say (or we have ideas but it no longer can do the job of a table saw and so must reject them). Second is to put guards in places - we have been doing that since at least the 1980s (probably before, but I'm not old enough to remember), but guards are not perfect and so people still can lose a finger even with guards used correctly (cheap guards often limit the functionality of the table saw by enough that everyone just removes them, but even good guards are not perfect). Only after the above would I look at stopping the system when a problem is detected. Last, but only if all of the above fails - is you put warning stickers on.
Let me emphasize: you should run the above list in order. If you can design a problem out then you are not allowed to put guards, brakes, or warning stickers on./
Most industrial machinery is designed with the above process. there is a lot of machinery from early days still around with out safety, but most industry has been adding guards and brakes to those were possible and replacing (machines from the 1950s are probably worn out anyway) the old stuff. Industry also has extensive safety training for the dangers they they cannot prevent other ways. The safety results for industry is much better than it was 100 years ago. Not perfect by any means, but much better and getting better [I was going to write every year, but random chance means some years there are more accidents than others despite the safety situation overall improving yearly]
An example might be railroads putting a derailer at the bottom of a hill to protect industry or businesses. The derailer is removed when servicing the line, but put back to protect the end of the line industry from run-away cars. Conclusion: they'd rather have run-away cars de-rail and have to recover them, then letting them damage a factory or business.
Price doesn't make any sense in the context of nothing between $20 and $200 (unless you just use the API directly which for a large subset of people would be very inconvenient). Assuming they didn't change the limit from o1-preview to o1 of 50 a week it's obnoxious to not easily have an option to just get 100 a week for $40 a month or after you hit 50 just pay per request. When I last looked at API pricing for o1-preview I estimated most of my request/responses were around 8 cents. 50 a week is actually more than it sounds as long as you just don't default to o1 for all interactions and use it more strategically. If you pay for $20 a month plan and spent the other $180 on api o1 responses that is likely more than 2000 additional queries. Not sure what subset of people this $200 plan is good value for (60+ o1 queries, or really just all chatGPT queries) every day is an awful lot outside of a scenario where you are using it as an API for some sort of automated task.
reply