It's about damned time. They've been trying to pretend EC2 is for utility computing...like running large distributed apps for a short period of time. But nobody is using it for that...people want Amazon's network and a cheap way to scale from zero to huge for websites. Now that they've finally embraced that, if they'll just resolve the reliability issues, EC2 will be a fantastic hosting option.
Lots of people actually do use EC2 for utility computing, often powered by Hadoop. Most of these folks aren't especially vocal about what they do, but there's data mining on a large scale, drug discovery, and similar stuff happening inside.
Heh, thinking of exactly that, I was going to write about how turkeys are fed and well taken care of by humans for 99% of their lives, until that last day, when things take a turn for the worse.
That said, things will run just fine, even with downtime, if you're not depending on the machine in any way, don't have any local storage, and are capable of switching over to other systems. However the lack of downtime might lull people into not worrying about that stuff.
You raise an important concern, but I wonder if the analogy is the right one. For a lot of startups/businesses, a period of downtime (in hours, not days) is not death. It might suck, but you won't die. In the late '90s, Ebay was down for three days straight (!) and it's still doing OK.
I will be surprised if Amazon doesn't work out the reliability issues anyway, given the massive resources and smart people it has available to it.
A bit of downtime is certainly not a catastrophe, but it might turn into one if you lose a bunch of data because you weren't properly shipping it elsewhere.
A much more sane response to the overwrought fears of EC2 downtime. If you don't know your hourly/daily income rate (often $0), then you can't even correctly value uptime.
Thats just it. Downtime is expensive if you rely on your web site to make you money. If you are a small eCommerce shop with all your eggs in the EC2 basket, downtime can be expensive.
I still think they need to resolve the disappearing disk issue, or continue to work on database type things like SimpleDB for persistent storage. Something using an actual relational DB like Postgres would be even better.
For me, the biggest step in resolving the reliability issues would be for Amazon to provide load balancing in the API. With a static IP, I could run my own LVS-based load balancer, but I might screw it up. If Amazon provided load balancing, their other customers would help shake out the major bugs, and I'd expect it to work much more reliably.
Does this mean emails sent from EC2 will not be automatically marked as spam? (right now I think Hotmail marks all EC2 emails as spam). I read that reverse DNS and PTR records are needed to accomplish this. So, this solves just half the problem?
I am asking these as exploratory questions... I don't really have a good grasp on what I am talking about. Please point me to resources.
BTW, do you know of any sample Python code that is used to act on incoming emails in the server?
DomainKeys are being checked by some webmail providers, but it's only a non-spammy characteristic at this point, not mandatory for delivery. As long as nothing else about your message is stupid (e.g. RFC compliant SMTP server, reasonable DNS records, and you aren't sending 100 per minute to addresses in the same domain), it'll go through.
I'd suggest using whatever Python library will allow you to send using a real SMTP server (or just pipe your email into the mail or sendmail command, which use the local smtpd to send, by default), like Postfix or Sendmail. There may be a standalone library that is actually RFC compliant and handles error conditions properly, but it seems unlikely from a historical standpoint (most such libraries, particularly in the PHP world, are renowned for epic failure). As long as Postfix or Sendmail are configured properly, your mail will be RFC-compliant and will look far less spammy than those coming from a half-assed implementation of the spec.
I have no idea. I haven't worked with Python in several years. But, I know that both Mailman and Trac are written in Python and they both do stuff with incoming mail...so, there's certainly some python code out their for dealing with incoming mail. That's where I'd start my search if I were in your shoes.
one way to do it is to use procmail to filter incoming mail and piping the ones you are interested into your python program which will take the rfc822 message on stdin. the email module (http://docs.python.org/lib/module-email.html) will let you parse these messages.
this is provided that your MX dns entry points to your machine and that you have the appropriate ports unlocked, of course.
I was speaking with an Amazon WS evangelist last year and he was recommending a 5 server system: load balancer, two web/app servers and two DB servers.
The issue I brought up was that you couldn't guarantee colocation, so you might be making DB calls cross-country. If you're using a poorly optimized ORM (untweaked ActiveRecord, Hibernate, etc.), that is issuing tons of queries, the net lag between the app server and the DB could slow you to a crawl.
The availability zones seem like the could be an answer to that, but they're being marketed as an answer to disaster recovery and failover. It would definitely be nice to request your app server and DB server are close to each other.
Perhaps this is just a coincidence, but interesting nevertheless.
This story was submitted to reddit at least two times: the original submission was downmodded to zero (or less), a resubmit eventually to 1. Here, it's at 51 points.
Yeah, I was going to use it for my online gambling (legal) site, and found the dynamic IP problem to be the real decision maker for me. I use smartfox server as a socket server, and renewing a license with a new ip address takes 24 to 48 hours, which means death to an online gaming site using social networking.
The static IPs make it possible to use AWS now. I can finally add voice chat functionality because of the power behind the large and extra large instances.
so is it much easier to use EC2 as a server now that IPs don't change? IE, you don't need dyndns and/or a load balancer? I wouldn't mind hosting a site of mine on there..
