On: Show HN: RAGstack – private ChatGPT for enterprise VPCs, built with Llama 2

[WillPostForFood]

Approximately, what would the hourly cost of running this be on Google Cloud?

>In the default-pool > Nodes tab, set:

>Machine Configuration from General Purpose to GPU

>GPU type: Nvidia TF

>Number of GPUs: 1

>Enable GPU time sharing

>Max shared clients per GPU: 8

>Machine type: n1-standard-4

>Boot disk size: 50 GB

>Enable nodes on spot VMs

Not familiar with GCP, but I see n1-standard-4's are in an instance type that is $.19/hr. Are there any other significant costs to take into account?

[jasonwcfan → WillPostForFood]

Just ran our deployed cluster through GCP's pricing calculator and it's about $300 USD per month with Llama 2