Heaven forbid you make a configuration change that triggers a blue-green deployment and during the deploy one of the AZs runs out of that instance SKU. Your deployment has halted until you can get ahold of AWS support to get them to unstick it (this takes a couple days even with enterprise support). There's no way to know if the AZ has capacity or not, it's something you find out in the deploy.
The workaround AWS support proposed was to reserve 2x capacity so we wouldn't run into this issue on subsequent deploys.
> Heaven forbid you make a configuration change that triggers a blue-green deployment
The main usability problem is that they don't tell you when that will be the case. It used to be the case when scaling up instances which was completely surprising. It's not the case anymore for scale up, so it's improving.
We bitch and moan about Google labeling everything "beta"(or even "alpha"), but the beta moniker would be appropriate here.
I am probably missing something here: If I understand what you're calling a blue/green deploy correct, you essentially want the ability to run 2x capacity for at least a little time (deploy time). So why wouldn't you reserve 2x?
Or switch to a AB like deploy? (deploy 5% or so, test against 5% from original deploy and decide on future).
> you essentially want the ability to run 2x capacity for at least a little time
This isn't the customer's choice. The customer does not want this.
As the article talks about, AWS Elasticsearch isn't actually elastic. On standard Elasticsearch, you can add and remove nodes at will and it will automatically handle rebalancing. AWS Elasticsearch can't do that. It has to spin up a new cluster of the desired size, copy everything over, and then turn off the old cluster. That is a form of blue/green deploy.
> So why wouldn't you reserve 2x?
Why would you want to pay double all the time because AWS can't use Elasticsearch correctly? AWS should foot the bill to ensure that everything works properly within their broken implementation when something requires the cluster to be duplicated and redeployed, not the customers.
> Or switch to a AB like deploy?
To reiterate: this isn't their choice. AWS forces this inefficient methodology on users of AWS Elasticsearch, which is why the article strongly recommends against using AWS Elasticsearch.
Right. It should be trivial when instituting a change that would trigger an event like this to take inventory that the required instances are available.
Doing a Blue/Green deployment setup costs 1.003x assuming a 5 minute switchover once a day. Being able to provision an extra server instance for a few minutes is kinda the whole point of moving to "the cloud"
Ok -I mixed up some GCP terms with AWS ones then: At least in GCP you can 'reserve' stuff that you want to use and pay for the ones you actually use. I am assuming they cap the reserve - I am yet to run into a situation where the reserved cores were unavailable.
I remember doing capacity planning like that for dynamo - but elastic search might be different.
The workaround AWS support proposed was to reserve 2x capacity so we wouldn't run into this issue on subsequent deploys.