A feature wasn't 10000 loc written by a AI before that no one except the AI with the context understands. If you review all that to understand it fully your productivity gain diminishes, the gain might not go away fully but it is much less when you can be woken up at 3am because of an incident and start reviewing everything.
Well, for me it makes a difference. I often get in a sense a "theory of mind" of the other developers when I read other code. I don't get the same thing with AI code.
Well you aren't writing the code, the AI is and you are letting the AI debug that created it in the first place and it doesn't learn from the experience in the same way. Hopefully you understand the problem in such a degree that you can spec away the problem in the next iteration. I'm seeing that issue now, people just forget to learn what the issue is and keep repeating mistakes that are regurgitated from the training material.
Well from my point of view. When they talk about gigawatt datacenters, then yes it is economically nonviable. You just need to know the scale of a gigawatt to realize that we need to start building power plants and fortifying the power grid to ship a gigawatt of power to a single location. Until the build out which takes years mind you, it is competing with other consumers of power. Lets take another huge consumer of power like a large steel mills use 100 megawatt. So if that power becomes more expensive because of datacenters, then the price of steel will go up. And if the price of steel goes up it affects a lot of things in the economy.
We are facing a situation that the short term effects are on memory and storage prices going up and lack of jet engines. Long term we wont be able to build actual buildings and ships without financing it with even more debt than today and everyone in the economy is going to service that debt through the price.
but the costs of inference have been going down 20x to 30x over the years. so how can you tell it is nonviable? unless you are saying they are not paying market rate for the inference
So, they still booked up all the ram and ssd in the world and still going to use gigawatts of power. The price of energy production is not going to go down 20x and 30x it just means that they can cram in more inference on the same energy consumption if the cost goes down. But they aren't paying the market rate for inference because everything is subsidized with debt and investors money to scale as fast as possibly. They are flushed with money and that is why they can book up all silicon production.
This claim sounds extremely fancy when AI companies bleed money, and will keep bleeding money in the foreseeable future.
I don't pretend to know the future. Maybe LLMs become economically viable and are the future, maybe not. I don't really care either way, to be frank.
And I use LLMs, btw. I pay for a ChatGPT account, but I find it only moderately useful. I always sort of question myself upon renewal date if it is worth the 20 bucks I spend monthly on it.
In no small part I keep using it to keep myself up to date on the best practices of using them in case it becomes standard.
The graph you linked seems to compare different OpenAI models in terms of "price per million tokens".
I am very skeptical of any financial information that comes from OpenAI. I have no idea how truthful those numbers are, or how creatively they can be collected to paint a rosier future for them.
Even if the numbers are truthful, I have no idea how the calculate price there. Is it in terms of cost of compute they rent? Is this cost subsidized or not?
Also, I don't know this "epoch.ai" website, I don't know their stance. The website name itself does not inspire my confidence on their reporting of anything related to AI. "Eat meat, says the butcher" vibes and all.
You can claim that the AI bleeds money because training is expensive, but inference is cheap. So it will only be financially viable when they stop training models? So they would need to stop improving their capabilities entirely for it to make any sense, is that your claim?
Even if I take this claim at face value (and that would take a lot of faith I don't have to give), it doesn't sound as good as you think it does.
>To analyze the decline in LLM prices over time, we focused on the most cost-effective LLMs above a certain performance threshold at each point in time. To identify these models, we iterated through models sorted by release date. In each iteration, we added a model to the set of cheapest models if it had a lower price than all previous models that scored at or above the threshold.
Can you look at the analysis? It will make it clear. I mean its so obvious because GPT 4 costs way more than GPT 5.2-mini but much worse performance.
>Even if the numbers are truthful, I have no idea how the calculate price there. Is it in terms of cost of compute they rent? Is this cost subsidized or not?
Do you think they are subsidising 900x or simply that the costs have gone down?
Overall you have shown what I feel is extreme skepticism in something that is obvious. You can literally run a model in your laptop that matches an older closed model. Costs are obviously going down, I have shown data. Use your own anecdotes and report.
Extreme skepticism in such a way doesn't do any help.
> Overall you have shown what I feel is extreme skepticism in something that is obvious.
I think you show extreme faith in something that is very obscure.
For me to believe in the analysis I would need to trust the numbers that the analysis is based upon. I see no reason why I should trust this. What sort of regulatory body or neutral third party inspects those numbers to ensure they are not a fabrication?
But you can claim I am a hater if it justifies your worldview. Skepticism is sinful for the believer.
> For our language model benchmarking, we note that we consider endpoints to be serverless when customers only pay for their usage, not a fixed rate for access to a system. Typically this means that endpoints are priced on a per token basis, often with different prices for input and output tokens.
Okay, correct me if I am wrong, so this is measuring the inference costs for clients of AI services, not the the inference costs that the AI service itself has when they offer the service?
I mean, the other guy's claim is that inference costs had come down 20x-30x. But the analysis, if I understood correctly, is based on how much clients are paying for it, not how much it actually costs.
I can charge you 20x less for a service and have massive losses for it.
It could be that OpenAI is subsidising their models by _fifty times_. Do you really think they are doing that? In some cases the costs went down by 200x. Do you really think OpenAI is subsidising their models by 200??
Its easier to just admit that technological advances helped decrease the cost instead of coming up with more complicated reasons like VC funding, subsidies and so on.
For instance take Deepseek and other opensource models - even they have reduced their costs by a huge margin. What explanation is there for opensource models?
> It could be that OpenAI is subsidising their models by _fifty times_. Do you really think they are doing that?
Possibly. I don't know.
It could be unfeasible to increase prices so much whenever a new model was released.
Any assumption made here is based on vibes. I see no reason to drop my skepticism.
> Its easier to just admit that technological advances helped decrease the cost instead of coming up with more complicated reasons like VC funding, subsidies and so on.
They raised an absurd amount of cash, and still bleed money to an absurd degree.
VCs make money when they exit. OpenAI only needs to "make sense" until an IPO happens. Once private investors have their exit, the markets can be left to handle the resulting dumpster fire.
> For instance take Deepseek and other opensource models - even they have reduced their costs by a huge margin.
Chinese companies are very opaque. I don't pretend to have insight into it.
Is the company behind Deepseek profitable?
> What explanation is there for opensource models?
What opensource models have to do with inference?
Your argument is that training is expensive but inference is cheap (something I see no evidence of). Why would a company give away the expensive part of the work?
>It could be unfeasible to increase prices so much whenever a new model was released.
This means you have no idea what I have been saying. A new model is costlier, but they release mini versions of old models that are way cheaper and compete with older models.
GPT 5 mini is way cheaper than GPT 4 but around the same performance
GPT-5 mini:
Input tokens: ~$0.25 per 1 M
Cached input: ~$0.025 per 1 M
Output tokens: ~$2 per 1 M
-----
GPT-4 (legacy flagship):
Input roughly $2.00 per 1 M
Output roughly $8.00 per 1 M
>Chinese companies are very opaque. I don't pretend to have insight into it.
False. The models are not opaque, you can literally download it and host it yourself. They have also released papers on how they reduced cost in certain areas.
This is literally them documenting the cost-profit ratio theoretical at 500%
>The above statistics include all user requests from web, APP, and API. If all tokens were billed at DeepSeek-R1’s pricing (*), the total daily revenue would be $562,027, with a cost profit margin of 545%.
Not only that, there are other providers hosting these opensource models, there are so many companies - just go to openrouter.com
So this is your skepticism
- openai is subsidising their models so much that each year the keep doing it 20x and eventually reached 100x reduction
- all the investors are stupid and they still invest in openai despite unprofitability
- employees of openai and anthropic who have claimed that the unit costs are not high are also lying
- all other providers are in on the lie
- the chinese models like Deepseek is also in on the lie by posting research that is not plausible
- the fact that you can run models in your laptop today that beat previous years models is also not enough
> openai is subsidising their models so much that each year the keep doing it 20x and eventually reached 100x reduction
If that's the truth, then originally they were subsidizing their models by the same factors.
This is not a great argument no matter how you cut it. And even then I would need to see evidence that this is true.
> all the investors are stupid and they still invest in openai despite unprofitability
Much to the opposite, those people are very smart. OpenAI can be extremely unprofitable and they can still profit massively through an exit event.
> employees of openai and anthropic who have claimed that the unit costs are not high are also lying
Possibly? Especially if they are in the position to profit in the case of an exit event, they would have every incentive to paint a rosier picture about the company.
> all other providers are in on the lie
I have no idea who you are talking about.
> the chinese models like Deepseek is also in on the lie by posting research that is not plausible
As I previously stated, I have no idea if Deepseek is profitable. By the looks of things, neither do you. Mentioning Deepseek's research is a non-sequitur.
> the fact that you can run models in your laptop today that beat previous years models is also not enough
My little anecdote of breaking the spell. Really I might not been truly under the spell, but I had to go far in to my project to loose the "magic" of the code. The trick was simply going back to a slower way of using it with a regular chat window. Then really reading the code and interrogation everything that looks odd. In my case I saw a .partial_cmp(a).unwrap() in my rust code and went ahead an asked is there an alternative. The LLM returned .total_cmp(a) as an alternative. I continued on asking why it generated the "ugly" unwrap, LLM returned that it didn't become available later version of rust with only a tiny hint of that it .partial_cmp is more common in the original trainingsets. The final shattering was simply asking it why it used .partial_cmp and got back "A developer like me... ". No it is an LLM, there is somewhere in the system prompt to anthropomorphize the responses and that is the subtle trick beyond "skinner box" of pulling the lever hoping to get useful output. There are a bunch of subtle cues that hijacks the brain of treating the LLM like a human developer. So when going back to the agentic flow in my other projects I try to disabling these tricks in my prompts and the AGENTS file and the results are more useful and I'm more prone to realizing when the output has sometimes has outdated constructs and be more specific on what version of tooling I'm using. Occasionally scraping whole branches when I realize that it is just outdated practices or simply a bad way of doing things that are simply more common in the original training data, restarting with the more correct approaches. Is it a game changer... no but it makes it more like a tool that I use instead of a developer of shifting experience level.
They have made huge investments into hardware so everyone is getting more expensive hardware, and now begging everyone else to make their investments worthwhile. Don't mind that they are driving up prices for hardware and requiring new hardware for Windows 11 upgrades. I'm suspecting that we don't have enough memory manufacturing capacity in the world to do both AI datacenters and replace all hardware that they made obsolete with their forced upgrade. AI didn't turn everyone into paperclips but it turned everyone to memory and AI processors in datacenters that can't be powered or has no useful economic utility.
Fun fact about BSTR, it uses memory before the string pointer to store the length.
From the CComBSTR documentation from microsoft: "The CComBSTR class is a wrapper for BSTRs, which are length-prefixed strings. The length is stored as an integer at the memory location preceding the data in the string.
A BSTR is null-terminated after the last counted character but may also contain null characters embedded within the string. The string length is determined by the character count, not the first null character." https://learn.microsoft.com/en-us/cpp/atl/reference/ccombstr...
From the book ATL internals that I read about 24 years ago.
"Minor Rant on BSTRs, Embedded NUL Characters in Strings, and Life in General
From the book ATL internals that i read about 24 years ago.
The compiler considers the types BSTR and OLECHAR* to be synonymous. In fact, the BSTR symbol is simply a typedef for OLECHAR. For example, from wtypes.h:
typedef / [wire_marshal] / OLECHAR __RPC_FAR BSTR;
This is more than somewhat brain damaged. An arbitrary BSTR is not an OLECHAR, and an arbitrary OLECHAR is not a BSTR. One is often misled on this regard because frequently a BSTR works just fine as an OLECHAR *.
STDMETHODIMP SomeClass::put_Name (LPCOLESTR pName) ; BSTR bstrInput = ... pObj->put_Name (bstrInput) ; // This works just fine... usually SysFreeString (bstrInput) ;
In the previous example, because the bstrInput argument is defined to be a BSTR, it can contain embedded NUL characters within the string. The put_Name method, which expects a LPCOLESTR (a NUL-character-terminated string), will probably save only the characters preceding the first embedded NUL character. In other words, it will cut the string short."
I wont link to the pirated edition which is never than the one I read.
So if there is code in outlook that relies on the preceding bytes being the string length it can be the cause of the memory corruption. It would require a sesssion in the debugger to figure it out.
Because he showed the action fps was possible on limited hardware. Also he has had some good ideas in the software- design and architecture of Wolfenstein, Doom and Quake, that where apparent when he open sourced their engines.
I'm dumbstruck that Crowstrike exists with George Kurtz still at the helm. There is no accountability at all. Kurtz was CTO of McAfee when their update caused back in 2010. Why does these things keep following him?
I've turned off my history on all logged in profiles, to get YouTube not giving me recommendations! I had a constant humm of YouTube content in my background during work. So when I was offered due to the EU to turn off my recommendations by turning off history I grabbed the offer. It is just enough for me to make my YouTube consumption intentional and avoid it. I've experimented with having a non-logged in browser with recommendations but occasionally it manages to find my poison, so after I nuke the cookies it is back to intentional usage of YouTube there too. Because the worst part of the YouTube, it gives you useful stuff that improves your life, but it also just sucks you into a black hole of content.
Also it is true that these things with procrastination and addictive behaviors are very much emotional issues and so on... and in my case whenever I notice that I'm avoiding dealing with my emotions and the hard stuff with YouTube content black holes I do the resets.
I'm old as dirt but I recall one of the arguments for TPM was shoved down our throats was the ability to tie documents to machines and organizations. Something something... industrial espionage. Now we know that is a lie. They just wanted to fill landfills with old working computers.
reply