The difference is that Meta and the FAANG companies make hundreds of billions of dollars in annual revenue, and are capable of hiring top talent to solve this problem of their AI running well on any GPU they choose for their data center.
Consumers, open-source solutions and smaller companies unfortunately can't afford this, so they would be dependent fully on AMD and other providers to solve this implementation gap; so ironically smaller companies may prefer to use Nvidia just so they don't have to worry about odd GPU driver issues for their workloads.
But Meta is the main company behind Pytorch development. If they make it work and upstream it, this will cascade to all Pytorch users.
We don't have to imagine far, it's slowly happening. Pytorch for ROCm is getting better and better!
Then they will have to fix the split between data-center and consumer GPU for sure. From what I understand, this is on the roadmap with the convergence of both GPU lines on the UDNA architecture.
I'm guessing most recent dissertations have been digitized, but this is probably the norm only in the last 10-15 years? Most universities likely have never given thought to digitize anything from before then due to the extra costs that would be involved in digitizing those physical copies. I am curious how much such an effort would cost though.
I should have qualified this with "the engineering departments at UC Berkeley". Everything we put out (papers, technical reports, open source software) was on the Internet. Formats were varied; LaTeX and Postscript were commonly used. PDF a bit later.
As someone learning 3D animation for indie games, it seems to take at least a year to get good enough. And even after spending that much time drinking from the firehose of information that Youtube is, I suspect that someone with a 4-year degree in animation would have a significant leg up over someone learning by themselves.
I think there are a lot of people who are going to school for animation, but maybe supply and demand are mismatched? But at the same time, a lot of VFX, animation and game studios are doing layoffs, so there probably shouldn't be that much a shortage of talent? I suspect GP comment may be right that the studios that are not able to find animation talent may just not be willing to compete on salaries.
For anyone curious before watching the video, it is about a company called "EK Water Blocks" which is (was?) the world's largest manufacturer of water cooling components for the "build your own computer" crowd.
You said water blocks and immediately I thought Gamers Nexus and there was his face. (Haven't watched LTT since LinusGate.) There was another up and comer company that's been reportedly doing great things in that area. I'll watch this drama, better than TV/Netflix.
Looks like this is a completely free project, 100% supported by donations, and no option for a paid plan https://exercism.org/donate
This is one case where I wonder if the founders could keep the 'core' free and open source, and have a paid model on top of it to generate enough revenue to work full-time on it.
I'm not the GP, and I don't have a dog in this fight, but I think what they are complaining about is this sequence of events:
- ElasticSearch was open-source
- Amazon offered ElasticSearch open-source as a paid service
- ElasticSearch was not happy about this and changed their license
- Amazon forked ElasticSearch (the open-source version) and created OpenSearch based on that, continuing to serve OpenSearch
- (Few years pass)
- Amazon and ElasticSearch are now buddies
I think GP is talking about the events that transpired a while back before Amazon and ElasticSearch made up.
What, when Amazon forked an open source project, as was allowed by the license, and continued to support that open source license even when the original company abandoned theirs? I’m not an Amazon fan, but they did the same thing (forking) that many others do, and they did it perfectly legally according to the license.
The issue that Elastic had is that their entire business model was offering a managed elasticsearch solution. Amazon then created their own offering of the same thing, but of course since it is Amazon it was more tightly coupled with AWS and benefited from being a native AWS solution. There was simply no way for Elastic to compete with that.
Now, there can be a lot of opinions on whether that is a good thing or a bad thing for the open source community, but it should be pretty obvious why elastic didn’t like it. They were a company who had a product they were selling, and then the biggest competitor in the world starts selling THE EXACT same thing with the EXACT same name. They needed to do something to compete.
So they did, and forced Amazon to change the name of their offering to opensearch instead of Elasticsearch. Once they achieved that, they reverted the change.
> since it is Amazon it was more tightly coupled with AWS and benefited from being a native AWS solution. There was simply no way for Elastic to compete with that.
That's one interpretation. I've another, which I've seen play out multiple times now across multiple OSS projects: company invents a thing, thinks that because they're the inventor of said thing they'll be able to sell a managed version of it, belatedly realise that inventing a piece of software doesn't magically make you the best in the world at running it at scale.
What Elastic, and most like them, can't compete with is the ability to run highly available/reliable software at the scale of Amazon.
>The issue that Elastic had is that their entire business model was offering a managed elasticsearch solution. Amazon then created their own offering of the same thing
Amazon first offered Elasticsearch as a managed service in 2015.
Elastic began offering managed services in 2018.
That’s not true. Elastic acquired Found in 2015 and immediately offered their managed Elasticsearch service. It was publicly announced at their user conference early that year.
Several years ago, I led a project at a startup to move from Postgres to Elasticsearch for geographic search. I chose Elasticsearch because it had the capability to do geohashing and so it provided a credible alternative to PostGIS for our particular use case. Geohashing-based search is particularly computationally intensive, which is somewhat different from traditional full-text search, which can be memory intensive.
I wanted to pay Elasticsearch to host our search cluster. And I did for a while. But it became clear we were paying for gobs of RAM that we weren't using and we didn't have enough CPU to really cover our search needs. I talked to the head of sales at the time and he said they were working on a plan that would allow us to choose machines that were more CPU heavy but that that was in the pipeline and there was no ETA.
So we switched to AWS and everything worked just fine.
All this is to say, Amazon was not offering the exact same service. They were offering a better service.
It's a bit more complicated than that because Elastic during all of this had some of the plugins in Amazon's OpenDistro (now OpenSearch) project recycling proprietary code from Elastic's commercial, source available codebase under OpenDistro's permissive Apache-2.0 license.
Your link specifically says Amazon didn’t steal the code, some German company did.
I get it, Amazon is bad, I agree they are too, but not because they’re malicious, Amazon is bad because they’re too large to compete on level ground with anyone other than Google or Microsoft in the cloud.
My peeve is with the companies like elastic that claim they are for open source but they try to prevent the open source from being used as such. It’s a scam to attract developers who care about open source. If I made code I wanted to be open source, I’d understand that means everyone can use it, like a public road or a public park. That includes big corps.
At least Amazon actually supports the Apache licensed OpenSearch product! They don’t even go around acting all superior like those “open source” corps do about it.
> Your link specifically says Amazon didn’t steal the code, some German company did.
Yeah it's not that Amazon stole the code, it's that they were distributing stolen code. It's not as bad but it's still problematic unless Amazon immediately pulled said code when they were notified.
> My peeve is with the companies like elastic that claim they are for open source but they try to prevent the open source from being used as such. It’s a scam to attract developers who care about open source.
I think this was less about preventing open source from being used and more about picking the wrong license for their project. The way I see it they took the long way around because they were afraid of the AGPL.
They dual licensed under the SSPL (which is the AGPL with one change that makes it problematic) and the Elastic License (which they originally also provided code under a previous version of).
Then they are now finally getting around to moving from the SSPL to the AGPL (while technically still offering the SSPL).
Had they gone straight to the AGPL none of this would have been even worth discussing but a lot of people are afraid of that license in the same way people used to be scared of the GPL.
> If I made code I wanted to be open source, I’d understand that means everyone can use it, like a public road or a public park. That includes big corps.
Sure however if you chose an open source license, you probably don't want companies selling access to your software with a few extra closed source bits bolted on without contributing anything back. It's not legally wrong but it's a dick move and against the spirit of FOSS. So even then Amazon hadn't broke any laws but it'd make sense for a FOSS oriented company to pivot to a license they think would force upstream contribution. Elastic just fucked up and chose a bad license (SSPL) because they feared the AGPL. This is just them getting over that fear and picking the license they should have picked from day 1.
> At least Amazon actually supports the Apache licensed OpenSearch product!
They do now. When Elastic was Apache licensed they did not. That was the problem. It was only when they re-licensed Elastic that Amazon open sourced their fork. Had they not, OpenSearch would still be the closed source AWS ElasticSearch.
I disagree with this. Most people use FOSS and do not give anything back, individuals included. The spirit of FOSS is creating things that others will use without compensation. If I release anything open source, it's because I'm donating it as a whole to humanity, including big corps and individuals. I understand that, because I've thought long and hard about what it means to release something with, say, an MIT license. It means you lose having full control of your creation. If I wanted to limit who can use my software, I'll sell it or license it accordingly. Complaining later that your FOSS software was "stolen" or "exploited" or whatever is just sour grapes.
If rando small business or rando dude is just using FOSS software without giving back, whatever people have priorities.
If megacorp is re-boxing it in a closed source manner and making mass profits off of it without dedicating at least some level of engineering hours or money to the project, that's a dick move.
Being unhappy with that situation is totally fine and it's understandable to change the licensing to a more copyleft license to reflect your intents.
That's what elastic did. They essentially went Apache-2.0 -> ... -> AGPLv3 but it took a while for them to figure it out.
And my complaint isn't that people make money off FOSS software or don't give back enough. It's that they make money off FOSS software and don't give back essentially at all while depriving their users of the same rights/licensing terms that the upstream gave them.
In any case, Amazon did give back to Elastic via code commits before the fork, so they didn't just steal the code, they also made it better. I can't believe I am defending Amazon, but this is a hill that I will die on: FOSS means free, as in beer.
The true irony being that elasticsearch would be no where near what it is now if it wasn't opensource and receiving contributions in the first place. AWS had employees dedicated to contributing to Elasticsearch (same as Redis).
Key takeway from the article: OpenAI's ChatGPT APIs might actually be, surprise surprise, profitable.
"As of June 2024, OpenAI's API was very likely profitable, with surprisingly high margins. Our median estimate for gross margin (not including model training costs or employee salaries) was 75%." "Once all traffic switches over to the new August GPT-4o model and pricing, OpenAI plausibly still will have a healthy profit margin. Our median estimate for the profit margin is 55%."
Ye the article is pretty interesting - I was trying to calculate inference provider's profit margins for say Llama-3.1 8b including electricity costs and the power consumption of GPUs, and I seem to always get to around $0.20/1M tokens as the minimum cost if you amortize a GPU's cost over 1 GPU. If you amortize it over 2 years, then maybe $0.15/1M tokens.
I'm very intrigued about OpenAI's profit margins, if they are indeed true.
Fair point! Ye if it's 3-4 year amortization that's reasonable - quantization could be another reason why some benchmarks for 4o aren't that good vs pure float8 / float16, and also how they made it much faster
Consumers, open-source solutions and smaller companies unfortunately can't afford this, so they would be dependent fully on AMD and other providers to solve this implementation gap; so ironically smaller companies may prefer to use Nvidia just so they don't have to worry about odd GPU driver issues for their workloads.
reply