On: Prompt Caching

[cma]

I'm sure internally they use it for the system prompt at least, probably since launch. And maybe for common initial user queries that exactly match.

[Onavo → cma]

They are certainly not passing the savings on to the users.

[minimaxir → Onavo]

Yet. I suspect OpenAI will release a similar offering soon. (hooray, free market competition!)

[HeatrayEnjoyer → Onavo]

That $100 billion data center has to get paid for somehow.