

How I Got Stung By Amazon EC2 - ysekand

So a few months ago I launched https://socialcrawlytics.com/ and to save a few bucks I thought I should use an Amazon Micro instance to run some workers off, it is free for the first year so it was a good option. Must admit, I really wanted to get on Heroku but Amazon's free tier was much more attractive.<p>For those of you who are interested, Social Crawlytics is based on the following technology:<p>1. Redis
2. FuelPHP
3. Node
4. Pubnub
5. MySQL<p>To cut the chase, within a couple of weeks I discovered that my pet project was attracting a lot of attention and out of nowhere the app was processing hundreds of large reports a day - which meant we needed more workers. So we developed the app pretty much based on Amazon's AMIs, every time there was a huge spike the app would auto launch new instances and instantiate workers until the spike was over and then it would terminate the extra instances.<p>This was obviously an expensive decision because we had instantly moved from a "free tier" to basing pretty much most of the app on Amazon's infrastructure. I knew the costs would add up but I sought solace in setting up Alerts so every time a certain threshold was hit I would get an alert. Turns out that Amazon's alert had a bug and and so did Social Crawlytics, basically in a matter of hours we clocked up over £640 bill and I wasn't alerted by Amazon at all.<p>Luckily, I just logged in to AWS and when I saw the cost I almost had a minor attack. So why did I share this here? Simple! If you are on Amazon watch your account like a hawk, you just never know what the costs would add up to and if you have a bug, it could cost you thousands in matter of hours.<p>I hope this will help someone somewhere.
======
rabidonrails
If there was a bug at Amazon I recommend giving them a call on the phone. I've
found that the customer service reps for AWS are quite nice, thoughtful and
genuinely seem to care about developers on AWS.

A while back I found a bug on AWS which caused our costs to skyrocket. SO, I
called and pointed out the issue and they gladly refunded the money and
apologized.

~~~
ysekand
This issue was caused around 3 months ago and you are completely accurate
about how Amazon deals with such events.

Basically, if I didn't have an Alert set up I would have been completely
liable for the costs and that is the point that I am trying to put across.

------
manidoraisamy
I had some great experience with Google in this regard. When cloud SQL didn't
have alert, we overshot budget due to a bug in our app. They analyzed the
whole problem for us & reverted almost 80% of the bill!

~~~
ysekand
I have never used their cloud infrastructure but what you are saying sounds
promising.

I was lucky that I had set up an alert otherwise I would have not recovered
the costs.

------
AznHisoka
Does Social Crawlytics work on a delayed report basis? Ie. you request a
report, and all your AWS instances start building it and you get it X hours
later?

What are your monthly costs so far?

~~~
ysekand
It is not delayed, you get real time crawling and data hence we had to use
Pubnub.

Monthly costs vary really, its normally upwards of $600.

