
How much does it cost to run GPT-3? - bendee983
Hi everyone. I need some help calculating the costs of running GPT-3 based on what they have released (transformer with 175 billion params).
I&#x27;m looking for two metrics:
1- How much VRAM&#x2F;RAM and CPU power would you need to run the trained model with decent response time (1-5 seconds) for a single input, and what would be the financial costs? (basically, I want to calculate that, even if OpenAI made the model available to the public, how much would they have to spend on hardware&#x2F;cloud infrastructure costs)
2- How much does a single inference cost in terms of FLOPS and eventually in monetary value? (Basically, if say, I was running a customer service chatbot on GPT-3 with an average response length of 150 characters, how much would each response cost me)
Does anyone have any idea how to calculate this?
======
Crossfold
I don't think GPT 3 is free for use. It's limited to some corporates only.

~~~
bendee983
I know it's not open to the public. I'm trying to estimate the costs of
running/inference based on what we know about the computation costs of
transformers and scaling them to 175 billion params

