Hacker News new | past | comments | ask | show | jobs | submit | smcleod's comments login

LiteLLM requires you to call them to discuss pricing to enable SSO... sure part of it is open source but not the part you need to scale it.

16GB? In 2025 that seems very small for anything other than gaming.

These are gaming cards. If you think AMD is lacking here with just 16GB I can only assume you never bought NVIDIA gaming cards in the last 10 years.

You can buy used nvidia tesla p40 with 24GB from 2016 and unlike AMD, they still have CUDA support. The only thing they are missing is support for the latest data types like fp8.

I don't know if I'm in a parallel Universe or hallucinating, but these are graphics cards, designed and created for people to install in they deskop PCs and run games. Just like they have been doing it for decades now.

What's with the P40, CUDA and fp8? Seriously people, chill. AI has been using graphics cards because that was the best available route at the time, not the other way around.

Otherwise I must question, why don't you talk about DisplayPort2.1, DirectX12, FidelityFX.. and other dozen features graphics related.


Because, unless you are a Big Corp, you can't afford to buy H100s. Being able to run AI on consumer hardware is essential for many AI hackers out there. And no, renting hardware on "AI clouds" is not reliable — still outrageously expensive, and the rug can be pulled out of you at any moment.

My rig is 4x4090, and it did cost a fortune to me already (still less expensive than a single H100). I would have happily used cheaper AMD cards, but they are not available for reasons like this.

Last time I checked, this site was called "Hacker News", not "Bigtech Corporate Employee News".


> I don't know if I'm in a parallel Universe or hallucinating, but these are graphics cards, designed and created for people to install in they deskop PCs and run games.

Graphics cards have been for more then running games even before they were broad-purpose computer engines that they have been since NVidia radically reshaped the market—decades ago.

Heck, Nvidia has had multipe driver series for the same cards because of this, even before AI was a major driver.


> You can buy used nvidia tesla p40 with 24GB from 2016

Then do that instead if it fits your workload?

There's more to a video card than the amount of RAM, though.


Which games support fp8?

I'm actually happy about that. It specifically targets gamers who want to play video games.

NVIDIA offers one new consumer card with over 16gb.

And that one card is $2500 in practice and sold out (only launched with like 500 inventory across USA). 5090 is pure unobtanium right now unless you feel like paying scalper prices in $3k++

> And that one card is $2500 in practice and sold out (only launched with like 500 inventory across USA).

And the power connector melts if you look at it funny.


These cards are $550-$600

16GB is great for what these are: Gaming cards.

Even on the nVidia side you have to spend $2000 (likely more) to get more RAM than that.

You could buy 3 of these for the price of a single nVidia 5090.


I suppose it's dissapointing that the 6800xt launched over 4 years ago at $649 with 16gb as welll. Inflation can partly explain this, buts still - we had been used to progress across the board each generation previously, at least with AMD.

On the Nvidia side every GPU released in the last ten years competes against what AMD is doing today. People who need more VRAM go with dual 3090.

For the low low price of $2.5k you can outcompete a $600 card. Wow.

They don't want to make cards good for AI, at the price gamers will pay, thus annoying the gamers and cutting into their AI chip sales.

nVidia created the AI chip market on the backs off gamers, but that doesn't mean that trend will continue forever. The demand is there, so they can extract greater profits and margins on AI chips.


To be fair, you can buy ~3 of these for the price Nvidia charges for 24GB/32GB models.

If people want more VRAM the 24GB 7900xtx is right there and has been there for years.

Yes but you can't easily put 3 GPUs in 1 PC

High ram cards are just nice for their longer lifespans though.

Since Nvidia dominates the AI market anyway, AMD has the opportunity here to not try and help them protect it, and sell cards with more RAM. IMO that’d be a good move. They’d be keeping gamers happy by not hampering their product artificially. And, some poor grad students might use their cards to build the “next thing.”


It's a gaming card.

GPT 4.5 is insanely over price, it makes Anthropic look affordable!

Especially not one owned by Bezos

IMO the only good feature in it is the notification summaries, they're pretty accurate and useful.

Siri on the other hand is embarrassingly bad.


> the notification summaries, they're pretty accurate and useful.

Far from a popular opinion.

https://arstechnica.com/apple/2024/11/apple-intelligence-not...

https://www.bbc.com/news/articles/cge93de21n0o


Siri doesn’t use Apple Intelligence. It is 10+ year-old technology that does not do learning. We won’t seen a truly smarter Siri until June when Apple previews the LLM Siri.


> "We won’t seen a truly smarter Siri until June when Apple previews the LLM Siri."

Really, they should have started with Siri, the moment LLMs became a thing. Not try to graft battery-draining AI assistants into existing apps in often-not-very-useful ways!


Perhaps replacing Siri with an LLM within the resources of a phone is complicated and takes time to get right. I’m willing to wait to see where they go with this a while longer.

   Siri on the other hand is embarrassingly bad.
and any attempt to replace this crapware with something better is sabotaged by the App Store guardians


> Siri on the other hand is embarrassingly bad

Always has been (c.2012)


But hey, I bet the metrics are up this year!

You can no longer disable it without breaking CarPlay, and it’s started randomly activating when you press the power button on my phone when carplay is on.

My theory is that typing a GPS destination into my car dashboard at a stoplight is too dangerous, and typing it on a phone keyboard is also too dangerous, but unlocking the phone, frantically yelling “cancel” until you find the “STFU Siri” button on the steering wheel, dismissing random useless carplay displays and THEN typing the address into the phone is completely safe.

On a related note, they keep adding more distractions to the apple maps carplay screen. The other day, I was at an intersection, and the map rendered the sidewalks similarly to the road. It knew I was driving. Does it think using sidewalks as shortcuts is legal or something?


It should be noted here they're talking about cold water (above freezing) - not ice baths which is what a lot of people seem to think of and is quite a fad in the "sports medicine" industry.


It's a meta study that potentially includes these as well as other cold water exposure:

> undergoing acute or long-term CWI exposure via cold shower, ice bath, or plunge with water temperature ≤15°C for at least 30 seconds.


Yes, but also above 7°C


Yes, they could work on their grammatical ambiguity when using 'or'. From the studies list, it appears they found no ice baths to study perhaps due to safety concerns.


Actual is cool but its lack of solid integration with the open banking standard (as is used in Australia and other countries) made it a bit annoying as its partnered integrations are pretty rough.


The open banking standard isn't actually all that open.

As a customer you cannot gain access, at least easily. You need to apply for accreditation as a third party financial services provider.


I get around 4-5t/s with the unsloth 1.58bit quant on my home server that has 2x3090 and 192GB of DDR5 Ryzen 9, usable but slow.


how much context size?


Just 4K. Because deepseek doesn't allow for the use of flash attention it means you can't run quantised qkv


That qkv PR was mine! Small world.


Stats is great! The only issue I've had with it is each time it updates itself as it's unsigned the binary is flagged and can't be opened until you run xattr -rc /Applications/Stats.app on it.


Huh? That doesn't happen to me. Mine is signed.


Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: