It's likely that bare metal is lower cost than spot - it is for me.
It's likely that your issue is allocation of capital - you'd rather put your money into burning dollar bills into AWS versus pre-paying for 18 months of servers with capital.
I'm happy your company is doing great.
I agree GCP is a great competitor to AWS.
I think you're being close minded to building out a 20-60 cabinet data center to offload some of your workload.
It's hard to do a DC with over 20 racks and pay more than 1/2 of AWS's prices.
I understand why would think so, because you don't know our setup. Just to give you an idea, we process 16PB of data every single month. This is an ingress traffic. If we would pay for this traffic going out (egress), we would end up paying over $1M just for the traffic itself. By keeping everything in a single spot it costs us literally 0.
That said, I've tried. I reached out to many hosters, like OVH and others. They just don't have the capacity we need. 20 servers will not make a change for us. We wanted to start with 500, but it would take them 9 months.
Your $1M estimate is 3X too high at 2c/GB.
In two hours I'll save you $25k per month. In 6 months $300k/month.
Either you are growing and should be investing in cost efficiency or you run a staid lifestyle business. Which is fine, but if you aren't growing, get off expensive cloud.
I'd say it's time to forget about the infrastructure. Should optimize the software and rewrite in C++ :D