The phrase “unsustainable gigantic moral hazard” applies to the current pharmaceutical and health industry. You know, the one that killed millions with opioids.
As far as I am concerned, burn it all down and put the most unhinged cokehead VCs in charge, who cares at this point. Hell, add some crypto AI NFTs in the mix while we’re at it.
The number of societal-wide problems that occur if your Lightroom database gets corrupted is zero.
The amount of hell that would be unleashed if the financial systems layers upon layers of database transactions got broken is impossible to comprehend.
So if you mean “important” as “necessary for society to function”, then no, your browser bookmark files, contact list, or the other two dozen things your laptop and phone use SQLite for are not important.
Literally all smartphones dying at once is preferable because they are all backed up into heavy duty databases. You can restore every phone from the cloud in this thought experiment.
You can not restore a data center from a pile of phones.
The goalposts keep moving here. You can just take any arbitrary definition of important and use it to exclude SQLite deployments.
It's all semantics, and also irrelevant to the original article anyway, since nowhere does it argue that SQLite holds the most data (important or otherwise).
I didn’t comment earlier so I can’t move a goalpost I never set down. I agree with the root comment that there may be more instances of SQLite, the most important data is not in SQLite.
For what it’s worth, I have used any number of databases over the years and SQLite is very good for a number of things.
None of those things are the core infrastructure that stores your emails, money, and other must have, shared, high availability data.
There are different tools for different jobs, that’s fine.
The false provenance was discovered by an Italian company, and then Spirit did their own investigation and found they had titanium from the same supplier with the same issue of false provenance. Spirit notified both Boeing and Airbus. Spirit produces parts for both Boeing and Airbus. This isn't about the FAA helping Boeing cover their asses, this is a real issue that impacts both Boeing and Airbus since the titanium ended up in planes from both companies.
They don't have 6 months. They have 1 day. People might not even be checking their inboxes on this Friday, leading to unannounced charges in an unreasonable timeframe or the extortion of paying or having downtime.
1B of tokens for Gemini Flash (which is on par with llama3-70b in my experience or even better sometimes) with 2:1 input-output would cost ~600 bucks (ignoring the fact they offer 1M tokens a day for free now). Ignoring electricity you'd break even in >8 years. You can find llama3-70b for ~same prices if you're interested in the specific model.
I answered the financial thinking in another reply, but another factor is I need to know if the model today is exactly the same as tomorrow for reliable scientific benchmarking.
I need to tell if I change I made was impactful, but if the model just magically gets smarter or dumber at my tasks with no warning then I can’t tell if I made an improvement or a regression.
Whereas the model on my GPU doesn’t change unless I change it. So it’s one less variable and LLM are black box to start with.
I may be wrong for Gemini, but my impression is all the companies are constantly tweaking the big models. I know GPT on Monday is not always the same GPT on Thursday for example.
I've worked professionally over the last 12 months hosting quite a few foundation models and fine tuned LLMs on our own hardware, aws + azure vms and also a variety of newer "inference serving" type services that are popping up everywhere.
I don't do any work with the output, I'm just the MLOps guy (ahem, DevOps).
You mention expense but on a purely financial basis I find any of these hosted solutions really hard to justify against GPT 3.5 turbo prices, including building your own rig. $5k + electricity is loads of 3.5 Turbo tokens.
Of course none of the data scientists or researchers I work with want to use that though - it's not their job to host these things or worry about the costs.
So my main motivation is not so much to have the lowest cost, but to have the most predictable cost.
Knowing up front this is my fixed ML budget gives me peace of mind and gives me room to try stupid ideas without worrying about it.
Whereas doing it in the cloud you can a) get slammed with some crazy bill by accident, b) have to think more about what resources testing an idea will take, or conversely c) getting GPU FOMO and thinking “if just upgrade a level all my problems will be solved”.
It works for me, everybody mileage varies but personally I like to budget; spend; and then totally focus on my goals and not my cloud spend.
I’m also from the pre-cloud era, so doing stuff on my own bare metal is second nature.
I’m doing non-interactive tasks, but in terms of the A6000 running llama3 70b in chat mode it’s as usable as any of the commercial offerings in terms of speed. I read quickly and it’s faster than I read.
This is why you need proactive privacy evaluations before you ship.
The standard of the past 25 years of “let’s violate every privacy law and know it won’t catch up with us” is over.
You either ship privacy complaint product, which means painful and slow review and adjustment which is an obvious financial cost…OR you go to market out of compliance, get slammed by the press and regulators, and the entire project eats shit.
What seems like short-term cost saving is really just torching the entire investment.
The underlying reason Boeing planes fall out of the sky and these privacy hostile products fail is the same: speed and greed.
As far as I am concerned, burn it all down and put the most unhinged cokehead VCs in charge, who cares at this point. Hell, add some crypto AI NFTs in the mix while we’re at it.
Literally cannot get worse than it is now.