

CloudAMQP experiences ops nightmare today - bigsassy
http://status.cloudamqp.com/incidents/vj62pnp62tj9

======
DrJ
So I just read this, the CEO added the below

    
    
        insts = [
          @ec2.instances.with_tag('Name', "CloudAMQP-#@name0*").to_a,
          @ec2.instances.with_tag('Name', "CloudAMQP-#@name-*").to_a,
          @ec2.instances.with_tag('Service', 'CloudAMQP').with_tag('Name', "#@name-*").to_a,
        ].flatten
    

unless they have a variable named "@name0" that first line would've evaluated
to "CloudAMQP-*", which would then have the fun happen.

------
bigsassy
CloudAMQP's CEO/CTO from the support chat room:

    
    
      Yes, it seems to have been a bug in our instance deletion code,
      that deleted all instances in that region, instead of one.
      This of course is a disaster

