The fundamental difference seems to be that Google's prices are fixed, whereas AWS uses a "market" model, which frequently sees crazy prices (well over the on-demand price) especially in us-east-1.
I say "market" because no-one really knows how the spot market place actually works. We've had machines run for weeks, and other times the prices fluctuate in bizarre ways and we can't get out preferred instance types for hours or (in the worst case) days. There's an interesting analysis here: http://santtu.iki.fi/2014/03/20/ec2-spot-market/
We had the same problem with getting priced out of c3.8xlarge in Virginia. We fixed it by changing our allocation algorithm to find alternate instance types and zones. For example, instead of 1 c3.8xlarge, it might pick 2 c3.4xlarge instances or a cc2.8xlarge. Seems to work so far.
I looked through the pricing table and played with the calculator, it seems something equivalent to our needs would cost around a third more on google but each cpu would have twice as much ram. Not worth it for us.
For people coming from AWS, we currently don't have an instance shape that lines up with the c3/c4 ratio (pushing you either to our n1-standards or n1-highcpu-. Can you get by with less memory?
Note: we're very aware of this pain point, and maybe you'll see something soon ;).
We need maybe 1GB for each job, but we need some leeway to avoid the OOM killer when a group of work units is larger. Swapping takes so long that it's not cost effective.
But to be honest about the situation, the cost would have to be much lower to make it worthwhile for me to rewrite our scheduler on Google's API.
I don't really see a fundamental difference except that Google's offering is more opaque and less flexible. Instead of "your instance got killed, but you can restart it if you're willing to pay $x/hour", you get "your instance got killed, tough luck."
EC2's spot market fluctuates based on supply and demand, and there's no reason to think the same forces won't apply to GCE.
It's not really that different: If your preemptible GCE instance gets killed, you can try restarting it as a 'normal' (non-preemptible) instance. But in Google's case, both prices will be predictable.
Well, spot varies. Sometimes it's low, but the price varies based on demand. Using prices as of right now for example, Currently in California, c3.8xl instances are quite popular, running $1.68 an hour on an instance that's normally 1.912 an hour, roughly a 13% discount. 70% strikes me as better than 13%, especially when it's predictable. (and 70% below an already substantially lower hourly price...)
The long-run price for a c3.8xlarge in California is somewhere between $0.40 and $0.50, though you're correct that right now they're $1.68.
California tends to be more expensive than other AWS regions, though (I can't remember the reason - perhaps just availability?). If you're in us-east-1 the price sticks around $0.20 per hour, and eu-west-1 is rock-solid at $0.32 per hour.
If you have work that can be done on spots it can be a good idea to make it region-agnostic so that you can take advantage of better prices for different instances in different regions.
The idea with Google is that you don't need to worry about which instances you run, at which price, and in which region.
For example, with Google you don't need to get a specialized instance type to use it's monster-fast Local SSD - just use the same instance types. This alone should simplify use of Preemptible VMs.
You can find a list of our prices here: https://cloud.google.com/compute/#pricing . Preemptible VMs have flat per-machine type pricing compared to EC2's bidding-based pricing. For example, a Preemptible n1-standard-1 is 1.5 cents per hour in the US and a preemptible n1-standard-4 is 6.0 cents per hour.
