Hacker News new | comments | show | ask | jobs | submit login

Full disclosure: I work for Linode and I am super biased. This is my own opinion etc etc

Perhaps the first AWS mistake you might make is... using AWS? Even before I started at Linode, I thought it was terrible. It's extremely, unreasonably pricey. The UI is terrible. Their offerings are availble elsewhere. I started MediaCrush, a now-defunct media hosting website, on AWS. After a while, we switched to dedicated hosting (really sexy servers). We were looking at $250 a month and scaled up to millions of visitors per day! I ran our bandwidth and CPU usage and such numbers through the AWS price calculator a while ago - over $20,000 per month. AWS is a racket. It seems to me like the easiest way to burn through your new startup's seed money real fast.

Edit: not trying to sell you on Linode, just disclosing that I work there. There are lots of options, just do the research before you reach for AWS.

Since you're dealing in absolutes, let me return the favor; you're being a little silly.

Yes, Linode beats AWS when it comes to price. On the other hand, Linode's offer is incredibly basic and simplistic. AWS offers service after service that Linode simply doesn't and realistically cannot.

Sure, you can emulate a subset of these services (and a subset of their features) using open-source software but at what price ? That's the major flaw in your reasoning. Getting to the point where your installations are as stable and reliable as AWS', given a large stress on the system, will cost you a lot of money and time. Directly comparing the cost of hardware access is ignoring other costs and headaches that are not very easy to estimate.

There's a market for a service like Linode and there's a market for a service like AWS. You've simply never worked on a project/system that works better on AWS than on Linode. I know I couldn't run the systems that I currently operate on Linode without multiplying the workload that is needed to maintain them.

Linode and AWS are competitors but there's space in the market for both; they simply fill different niches. Establishing one as absolutely superior to the other is silly and closed-minded. A lot of people chose AWS; go and ask them why (feel free to reach out to me at nick at nasx dot io - I'll be more than happy to talk to you).

Do you even understand his point? He is just nice enough to disclose his current employer. He's not recommending Linode. Really, all he's doing is just asking you to do some research and it needn't be Linode, there are plenty of alternatives out there.

If we keep attacking the people for their honest disclosures, we're only discouraging them from expressing their conflict of interest in the future. Please. Don't.

I'm really not trying to compare Linode to AWS here. I just felt that it was necessary to mention that I work for Linode because there _is_ some overlap. I'm really also not trying to sell anyone on Linode in particular. I'm pointing out that from my own personal experience, AWS is grossly overpriced and worse than the alternatives for a large number of use-cases.

AWS is insanely overpriced. It's weird to me that everyone jumping on AWS and some of their customers never come close to attracting the internet traffic required to set up CloudFormation. I think if you're starting a webapp out of your own pocket and not millions of dollars of investors, it makes sense to save where you can.

Using locust.io, I've seen that my current site on two $10/month Linodes can scale up to approximately 300k people/day and increasing that substantially just means I press a button and upgrade my app/database servers.

If it came to a point where I was growing at a pace I didn't want to manage and money was flowing in, and I was out of ideas on software optimization, only then I would consider spending tens of thousands/month on AWS.

I'm not denying the great benefits AWS gives, I honestly would love to use it now and just be done with most of my devop headaches, but the costs are prohibitive.

What are some alternatives to AWS? Are there realistic alternatives?

The "alternatives" depend on your situation.

Picture a continuum between brain-dead simple websites and business-critical complex websites:

  simple:  static website, WordPress blog
  moderate:  small business CMS, etc
  complex:  Netflix, AirBNB
If you're running a simple WordPress blog, the AWS prices are absurdly overkill. For this use-case, there are a zillion alternatives. Linode, Digital Ocean, Rackspace bare metal, etc, etc.

On the other end of the spectrum, you want to run a high-availability website with failover across multiple regions like Netflix. You need the value-added "services" of a comprehensive cloud provider (the "I" and "S" in "IaaS" as in "Infrastructure Services"). For that scenario, there are currently 4 big competitors: AWS, MS Azure, Google Compute Cloud, and IBM SoftLayer. However, many observers see that Google and IBM are not keeping pace with AWS and Azure on features so at the moment, it's more of a 2 horse race than a 4.

Keep in mind that the vast majority of cost comparisons showing AWS to be overpriced are based on comparing Amazon's EC2 vs bare metal. The EC2 component is a small part of the complete AWS portfolio.[1] If you're doing more complicated websites, you have to include the costs of Linux admins + devops programmers to reinvent what AWS has out of the box. (The non-EC2 services.) Even if you use OpenStack as a baseline for a "homegrown AWS", you'll still need extensive staffing to configure and customize it for your needs. It may very well turn out that homegrown on Linode is cheaper but most articles on the web do not have quality cost analysis on the more complicated business scenarios. Anecdotes yes! But comprehensive unbiased spreadsheets with realistic cost comparisons?!? No.


Digitalocean is a great alternative for simple things.

I use it for dev / early projects and as things get complex or need more redundancy I make the production spend on AWS.


Much like I wouldn't trust a person's whose only comments on hn are AWS puff, why should I trust a person whose only posts are Google Cloud puff?

I've found Google Compute Engine decent, but with App Engine I ran into difficulties with vendor lock-in. I had to first modify my code to get the app run on my local computer. After that I never got it deployed because GAE requires me to bind their own TCP context for outbound connections. This was a shame because I would have wanted to have my app on GAE and database on Heroku.

Just saying that calling 90% of Google Cloud's products to be far better than AWS (or any equivalent provider for that matter) is bold.

App Engine has a lot of faults, but your use case sounds incredibly niche. You're using the multiregion availability of App Engine's HTTP servers but want to use Heroku's single AZ database across relatively flaky, slow internet instead of App Engine's multiregion available database on a managed internal SDN. I'm having a hard time imagining a scenario where that setup would make sense.

Google Cloud is not just App Engine now. Its pretty much the equivalent to aws in features. Let me also say I have PhD in Cloud and big data (graduated in 2013), worked for Amazon aws, worked for an rackspace for year and half, I was a core contributor to OpenStqck for the products OpenStack Poppy and OpenStack Zaqar. I hope you his puts se more weight behind my statements. If you are interested to know the details and why Google Cloud is better that aws for 90% of products, reach out to me.

They aren't even close to each other in featuers. All the various additional Services AWS supports go way beyond what Google has at the moment. Numerous Database offerings, Queueing, transcoder, ... Go way beyond what Google has in place.

And no the ability to run it yourself on Google is not the same.

Your statement is simply false.

Here's a material answer - https://cloud.google.com/docs/google-cloud-platform-for-aws-...

In short, Google has parity with AWS on many fronts, and exceeds AWS on many others. Only material AWS advantage at this point is full IAM. Biggest thing you gotta remember is AWS is stuck in "VM" world and only slightly deviates from that. Google's advantage is in its fully managed services, which AWS does poorly (don't tell me Redshift is "fully managed").

Google has Bigtable, which AWS has no competitor for. Google's Pubsub is vastly superior to SQS. Google doesn't need Firehose because Google's services scale to Firehose levels without needing a new product and a new price. Google has Zync transcoder service.

Google has BigQuery, which is vastly superior to Redshift in price, performance, scale, and manageability. Google has Dataflow, which AWS has no competitor for.

Even for VMs, Google offers better networking, faster disks, more reliability (live migration), better load balancer, etc.

And to put the cherry on top, Google's no-committment, no-contract price is only beat if you lock yourself into AWS's 3-year contract.

(disclaimer: work on BigQuery)

Cool story, bro.


Please take a look into Google Cloud. It's now feature equivalent with AWS (https://cloud.google.com/docs/google-cloud-platform-for-aws-...). Google Cloud is better than AWS on all fronts other than Relational Databases. Google Cloud has MySQL only. AWS has MySQL, PostreSQL, Oracle, MSSQl. Rest of the services Google beats AWS by a fair and wide margin. I have done a fair amount of benchmarking Google vs AWS in past 3 months and conclusions are shocking. With Google Cloud, you can save about 50% of your bill (I have taken into account of reserved instances), huge increase in companies agility (Speed in taking decisions like capacity planning, instance type, sizing, zones, cluster sizes ... due to simplified infrastructure and pricing & performance wise too, blazing fast disks, live vm migration), far better security (data encryption at rest and on wire by default for all products, ssh key management). I am presenting my findings at our meetup group (http://www.meetup.com/Cloud-Big-Data-and-Data-Science-Group-...) in a couple of months. If any of you want to talk, reach out to me at obulpathi at gmail.

You'll need a better source than a Google marketing page to support claims of "feature parity". An objective comparison also shouldn't start with "Please look into Google cloud".

That is why I am sharing my findings in detail at out local meetup. If interested to know more, just shoot an email to me at Obulpathi at gmail. Also see my another comment above.

Thanks; I hadn't seen that that page from Google and that's a pretty good idea to make that available.

You know what would be a cool idea: if Google developed a method of copying an Amazon Machine Image (AMI) to Google Cloud. That would give us an easy way to try out our same servers at Google without having to rebuild everything, as we do not use containers yet.

You're right, but I know of a few startup teams that chose Linode, Rackspace etc. at start to account for future 'scalability' concerns. Well the number of users never came and they spent a lot of time just configuring stuff. Of course I have also seen teams that couldn't build a decent business on AWS.

I think the net cost of an extra personnel (a server admin) in a startup team, is more than the cost of the server.

Huh? In general AWS is chosen for 'scalability' concerns given it's auto-scaling functionality, not Linode/Rackspace/etc.

If anything, I see a lot of people choose AWS for 'scalability' concerns when they never end up needing to scale.

Your 300k people/day metric conveys little information if we don't get an idea of the workload each user entails?

Aww and digital ocean customer here(as well internally hosted).

Main advantages I see is Security - aws has security zones, FW ETC FOR FREE Momentum - constant new features(apache spark, lambda, sqs)

Your post basically picks the one thing he voluntarily disclosed and ignores the rest of his perfectly reasonable statements in some manic attempt to defend the one true stack.

The only points you even remotely bring up are vague and wishy-washy and could easily be applied to any statement, even outside of programming.

Most of the reasons to go with AWS as a major strategic decision are to avoid paying for 3-4 (or perhaps... 40+) ops engineers to run and manage so many things that really don't add much business value in themselves and that you could hold to some form of an SLA (especially with enterprise support accounts) and recoup some costs if things came down to it. Furthermore, engineering a lot of AWS services would cost you developer time as well.

I consult / contract for a Fortune 50 customer that pays an insane amount of money to AWS per month and it seems ridiculous what they're paying for and what they get. But do you know what's even more ridiculous? Paying three times as much but getting even worse reliability and a massive amount of time wasted while waiting weeks to provision servers. One group internally used to have massive outages daily resulting in needing to hire a dedicated team of about 6 just to handle that load of production incidents (terrible reliability also resulting in loss of revenue too) and since migrating their software with even the most bone-headed SPF-everywhere AWS based software architecture haven't had more than one or two per week for outages now. The cost benefits get better the more incompetent / incapable your internal IT organization is. The amount of wasted resources due to internal bureaucracy, legacy, and working with companies that don't really practice any form of technology at scale has been an incredible amount of savings for my customer.

And knowing how much my customer relies upon AWS support for the most menial of tasks (primarily a cultural thing with how they treat their vendors as well as internal resources) I am sure that nobody but massive companies that deal with external bureaucracies effectively could deliver the size and kind of support team to handle the volume of support requests generated by my customer for the most ridiculous of tasks (I've seen executives file AWS support requests to reboot VMs at 3 am because the bureaucracy is extremely brittle here and reflects into the software systems reliability). We may throw millions / month at AWS, but Amazon has to hire at least a few dozen engineers and several account managers just to handle support alone which I might argue could be a net loss to handle the customer.

Most vendors that work with my customer have a very hard time working with them because they approach sourcing quite similar to how Walmart does and abuses them to the point where the vendors abuse their employees.

"I've seen executives file AWS support requests to reboot VMs at 3 am"

Isn't this falls outside amazon's responsibly area?

Atleast the documentation I read and a few aws promotional talks I attended seems to suggest the contrary.

Or I have understood completely wrong and amazon is now also providing managed IT services?

There are some legitimate cases where AWS support is the right place to issue a request. Just the other day I had a VM go down and I issued a request to stop the instance and it just hung there with no valid way to keep shutting it down. Turns out AWS support is the only way for most users to get an instance on a host that suddenly goes down while stopping your VM to be correctly stopped because the underlying API call does not go through and it seems like a small critical section of sorts.

But when you pay AWS so much they are pretty beholden to your requests and can wind up becoming a lot like an MSP. I've spent countless hours with those poor guys waiting on a traceroute failure or for mtr or sat to give some anomalous event that everyone suspects is something wrong in AWS when I've insisted AWS is almost never at fault because our incompetence is almost certainly the problem.

The AWS support team's worst engineer is probably at least in the top 25% of our support engineers I'd argue so I'm sure if my customer had the option they'd want to pay to have them troubleshoot our own internal networks but that's what my team is for I guess.

"The cost benefits get better the more incompetent / incapable your internal IT organization is. "

Yes. There will always be a less stupid solution out there that is not as bad as the bad solution.

Or, you could fix IT. But that takes thought.

Easier to give money to a vendor you can blame...?

Fixing completely dysfunctional F500 IT organizations is something that many, many people have tried across completely different company cultures with maybe a handful of successes I've heard of. Leading huge organizations (especially ones damaged by previously failed leadership efforts - think how attempts to refactor a legacy codebase go and multiply by an order magnitude in number of variables) is very hard and because large organizations tend to have so much capital working for them, IT management failures create a large opportunity (in theory) for optimization by others. In practice, consulting for organizations that don't want to fundamentally change how they do X (or think $ must solve everything and leadership becomes so fragmented into non-communicating middle managers) while wanting large changes to X are the equivalent of an obese person wanting to lose weight without changing their diet or exercising. Yet we know there's a huge "fitness" industry with large profits being made off of those that do not use their products and subscriptions effectively.

Change is tricky. What you said reminded me of an interesting tweet I came across: https://twitter.com/JICare/status/662339652825780224

This is a fairly naive way of looking at it. Often it's easier to deal with an external organization than to fix your own organization's behaviour.

In a lot of pathologically-managed companies or divisions, there is effectively zero chance of an employee being able changing the pathology.

I have heard of a few poorly managed companies pulling out of ruin by using consultants and third party labor to supplement their collapsing workforce. But I've seen poorly managed companies on occasion fire their executives and stabilize as well. People love to spend 7 figures on consultants and refuse to take their advice in favor of suck-ups that recommend that they just slightly change what they're doing or just sit in silence and continue to bill out. This is why I stopped consulting / contracting in the federal government space - nothing I do even if I was a company founder can fundamentally fix even a tiny fraction of the deep flaws. People got rich off of manipulating contracts to do as little as legally possible while the best at delivering were cut because they actually finished the job right. Reward for failure is a system that exists plenty in the private world, but only in public sector is it institutionalized.

This sounds about right. This is why I don't believe in the "industry is more efficient than government" dogma. Large organizations of people are generally inefficient unless extraordinary care has gone into the engineering and maintenance of the organization.

Are there any particularly memorable situations in your consulting career that illustrate the political realities of such organizations? Would be really awesome if you can share any :)

Hi, first of all, thanks for your comment, and your full-disclosure. I am a happy Linode customer, and I have been using your service since 2010-09-02 (that is the date of the first invoice), but I have a few issues with your product that you might be able to look at.

- It is too hard to figure out that you offer hourly billing, I found it by accident, and a year too late.

- I tried out your Nodebalancers, as I thought they would be a nice help for my stack, but I ended up building my own Haproxy solution since the performance wasn't good. When I pay for a dedicated loadbalancers, I would expect it to be top-tuned from your side.

- Your API is a mess, and so is your documentation. Everything is GET-requests. And creating a new instance, shouldn't take 11 requests, and a dance to set every little detail correctly. And to be fair, your documentation doesn't even document that, this is a subset of what I use: https://gist.github.com/kaspergrubbe/756f0a227db8aeb92818 I even managed to find some errors in the documentation when I went through it, so it didn't feel very polished or tested, but maybe it isn't used a lot. It also seems strange that to emulate one or two clicks in the interface, needs 11 requests through the API.

- Your advanced settings are extremely hard to find, especially "Edit Configuration Profile" and the "Auto-configure Networking"

- Elastic IPs. I feel like it should be easier to swap IPs around Linodes, without me using heartbeatd, and some strange configs. This is much easier with Amazon, and one of the reason why I prefer them.

- Anycast routing. Linode has several datacenters, it would be awesome if my setup in the US could have the same IP as my setup in the UK, because right now I am doing ugly DNS hacks to send my customers to the closest datacenter. Is this planned?

- Lack of an object store, even though I use Linode, I am still using AWS S3, which means added latency, and I have to pay Amazon for outside traffic, you can't even compare with AWS on these things without starting to have these things.

Can I setup a Linode through the API with private networking enabled?

I am a happy customer, and I loved your KVM upgrade, I love your prices, and I love your service. But it would be nice to see more innovations happening.

Hi there. I wasn't looking to represent Linode tonight (just my personal opinions here), but I'll go ahead and fill you in. I can't say more than "I've made a note" of most of these things, but I wanted to mention that we're looking at overhauling some things that will definitely fix a lot of your gripes. Linode has been around for 13 years now, and a huge part of my job and the reason I was hired is to modernize a lot of the stale parts, including lots of the things that are frustrating you.

I'm also a happy customer and want to address a few of these points.

> It is too hard to figure out that you offer hourly billing, I found it by accident, and a year too late.

It's still a pretty new feature, only about 18 months old, and their price sheet has clearly listed e.g. ".06/hr to $40/mo" for quite a while now. I vaguely remember getting an email and seeing a blog post about it when it was announced...

> When I pay for a dedicated loadbalancers, I would expect it to be top-tuned from your side.

I think it is top-tuned, but only on $20/month worth of hardware and networking. ;-) They explicitly say in some blog posts to add more NB and scale out if you need more concurrent connections. I agree that this is a little alarming, and I'd be willing to pay more for beefier NBs—just like I do with regular nodes. Building out your own HAproxy nodes is a fine solution and what I plan to do if this becomes an issue for us.

> Your API is a mess... It also seems strange that to emulate one or two clicks in the interface, needs 11 requests through the API.

It's not the greatest API in the history of computers but it gets the job done and I found the documentation to be more than adequate. It's bare-bones and you're going to want an abstraction over it, whether you write your own (like I did), use their CLI, or use Salt or Knife or what have you. Whether it's one API call or 100 doesn't matter if all you have to do on a regular basis is `linode-create foo` or `ansible-playbook create-linode.yml bar`.

Anyway, I just ran through a quick build using the web interface and it takes like 10+ clicks to build a node with the default disks, no private interface, no label, no forward or reverse DNS, not booted, etc. For this to take four API calls to allocate the node, create root and swap volumes, and build the config doesn't seem like a big deal to me. Tack on another four API calls to set the label and display group, set the DNS, add a private IP, and boot the node. I'm not seeing the problem here.

> Lack of an object store

I mean, sure, Linode lacks a lot of the things that AWS provides. They also don't provide a database aaS or a queue aaS or a CDN aaS or... you get the idea. I don't think they're trying to compete with Amazon at this level. Where do you draw the line between IaaS and PaaS? Can you implement your own object store using some open source software just like you run your own HAproxy and database instances?

> Can I setup a Linode through the API with private networking enabled?

Yes, but it's a separate API call. :-) https://www.linode.com/api/linode/linode.ip.addprivate

> I am a happy customer, and I loved your KVM upgrade, I love your prices, and I love your service. But it would be nice to see more innovations happening.

Agreed on all points. My innovation wishlist includes VPC (I really just want my own private VLAN) and more flexible node configurations (another commentor mentioned that you can't add disk without adding CPU and RAM—and vice-versa—and this is a huge pain point). I haven't had to deal (yet) with IP swapping and anycast but I can see those becoming issues for us in the future. And I actually do agree that Linode should implement some kind of a object store to compete with S3... I would categorize that as IaaS, not PaaS, and it's the most glaring hole in Linode's offering right now.

Linode offers the equivalent of a tiny, tiny subset of AWS. I haven't looked recently, but do they have something equivalent to SQS? Autoscaling groups? DynamoDB? SES? Cloudwatch?

If you are going to end up re-building AWS's services from scratch on top of what are essentially naked linux machines (or doing your integration work between services from many vendors), you are doing a huge amount of work that isn't necessary and will be a lot more expensive than that $20k/ month you could have been paying to Amazon.

$20k/month is $240k/year, which is at least two developer salaries. Do you really need two full time devops/whatever to manage the open source equivalents to the AWS services?

Aren't there tools that can provide a similarly high level/seamless/polished experience running on your own computer, using your choice of cloud provider as the backend? If not, why not?

(I wouldn't know - I don't even run a website, and if I did, I would avoid the locked-in services like the plague, out of principle. But I'm curious.)

Do you really need two full time devops/whatever to manage the open source equivalents to the AWS services?

No, you need many more than that. AWS's offerings are complex systems, each backed by multiple large teams of developers and operations people. And that's not just because they have tons of customers.

You want to run a partitioned, replicated data store with five nines of reliability? Good luck doing even just that with two people. Throw in metrics (CloudWatch), a queuing system (SQS), monitoring/failover with on-demand provisioning (EC2 + auto scaling groups), and you're talking a lot more than just two devops people... well, unless you want them both on-call 24/7/365, and you want them to burn out in a matter of weeks.

AWS isn't perfect by any means, and you can certainly argue that it can be overpriced... but maybe you pick and choose, and use AWS for the things you don't have expertise in or time for, and roll your own for the rest. The nice thing is that AWS lets you offload all that stuff when you're small and don't want to or can't retain that kind of expertise in-house, and then you can focus on actually building your awesome product that runs on AWS's plumbing. And when you have more time and people later, you can look into doing things differently to save money, piece by piece, when it makes sense to do so.

From a management standpoint, it takes at least 2 developers to run something equivalent. Perhaps not full time, but it doesn't scale out that much - not more than 10 highly available services per team. A good example of this is roughly a 3 instance RabbitMQ cluster used by ~100 people.

1. The "Bus Factor" - https://en.wikipedia.org/wiki/Bus_factor

2. Context switching - if you have a developer that also works on coding or other things.

So maybe it's not a full $240k/year for only 1 service, but the price in the door IS that. 2 to 3 extra services will cost no more - that's where AWS will make the money.

I'd think you'd need those staff (or approximate) regardless of whether you used AWS or not, no? It ain't the marketing team that orchestrates all that amazon stuff.

At scale, the arguments about needing 10 admins vs needing 15, for example, start making sense.

It depends on which services, but for simpler services with well understood open source equivalents (e.g. route53 and BIND) you can probably manage two per sysadmin at a similar quality to AWS with a basic API. For something like RDS or Redshift, if you're very lucky you'll get away with one sysadmin per service.

This covers not just running the services, but keeping up to date with security updates, scaling the systems, feature additions, bug fixes and keeping up with the state of the art.

Services like RDS are an amazing net cost saving for small to medium users. Something like Cloudfront would cost you a million just to get going. Raw EC2 is nearer break even.

"Something like Cloudfront would cost you a million just to get going."

We've used CDNs in the past, way below a million/year for a medium, growing startup. care to elaborate?

Ti believe the context is running a CDN as opposed to using.

Look into Kubernetes. Since everyone is migrating to Docker and Microservices, Kubernetes provides a perfect fit to run your workloads on local premise and then migrate to Cloud when extra capacity is needed. It's opensource and supported by several vendors.

Or even VPC. I'd like to be able to run a small Kubernetes cluster on Linode at some point, but I'm not sure I'd want to run it without a VPC.

I'm involved in two different startups, one on Linode (because that's all it can afford) and the other on AWS. For it's current offering, Linode is great for MVPs, and you like your servers as pets, but it's doubtful we'd stay with Linode once we start having to scale.

False. Most AWS services are implementations of open source solutions. Once you implement it yourself on Linux, the ongoing cost drops. With AWS you pay more the larger you get.

> Most AWS services are implementations of open source solutions.

This is fundamentally not understanding what AWS is.

- You could say "EC2 is just Xen". But next time there is a 0-day exploit, I'll have AWS working all weekend to patch my servers. And Xen still doesn't have an API for scaling physical hardware..

- You could say "S3 is just Apache". But I will never see a "disk full" message, I will never get paged if something is borked (but it will still get fixed), I will never worry about DDOS attacks, etc.

> Once you implement it yourself on Linux, the ongoing cost drops.

That's like saying "it's cheaper if you change your own oil". Might be true, but doesn't matter. I'm still taking my car in. Lots of other people do. You might try asking them why.

Some services are unique. EMR and RDS are not. Others are not.

Outsource if you want. When your bill hits $500k/mo and you realize you're paying for things you can do yourself, your position may change.

Have you been through an AWS outage and been paged? It happens. When you realize your business depends on an opaque organization you may want to diversify.

If you're small it makes sense to outsource sometimes. Not always and not forever.

I am especially curious to see that 'cheaper if you do it' done for their databases offers.

I'm using them in prod already, and trying to create a similar setup on real hardware for enterprise customers using pg, pgpool2 and haproxy is already a pain, and yet you couldn't autorecover them as conveniently when they go belly up.

What? Implement RDS/SQS/Redshift myself? That is not easy, believe me.

It's not easy, but also not every difficult. The company I work for has two Dev Ops (only one until a year ago) and runs on AWS serving millions of users.

We are using MySQL RDS, but are migrating to self-managed Cassandra. We use RabbitMQ instead of SQS. We use Hive/Presto instead of Redshift.

You sure can adapt the open-sourced solution, though I don't think Cassandra/Presto can match up to what RDS/Redshift can offer.

The key point is, running those things yourself, you are likely to lose all the nice failover mechanisms/monitoring/auto-snapshotting stuff that AWS offers. To live up to that, it will require you to not only have extensive understanding of the software you are running, but also a considerable amount of your time will be dedicated to Ops side, which can lead to some really big frustration from time to time. In that sense, I don't think you can re-IMPLEMENT something by assuming too much of comprises.

Cassandra in particular doesn't compete with RDS as much as DynamoDB and, respectfully, setting up failover and monitoring and autosnapshotting isn't actually that difficult. I'm a big fan of outsourcing that stuff--the new system I'm building uses AWS Lambda, for example--but when using a managed solution requires compromising on features or tying yourself into knots to make things work, hosting your own is doable, especially in the age of devops. Automation is your friend!

It is very difficult to run it at AWS's scalability, reliability, security and automation level. These all cost money and/or time. Not all companies require these features to the same extent, of course, so AWS is not always the right choice, but it isn't always the wrong choice either.

I've been a customer of linode for over a year and soon will be migrating to another provider, most probably Amazon. The main reason is a lot of connectivity issues and down time. I always hosted pet projects and simple web app on Linode, but just this week, I used it a for a major project. Now, the whole region of Fremont,CA has connectivity issues and is down for a couple of hours per day. The same happens for other regions as well. Maybe always it's been like that, but I've never paid attention. You can see their incident history in here: http://status.linode.com (I browsed it a little bit and it seems they had a lot issues in the past as well) I probably will lost about 6 times more money that Linode has saved me over AWS only for the down time in this week.

So my suggestion is for a small-non critical projects you can use Linode, but for anything serious you better use something that is reliable.

They've been getting DDoSed for the past week. I've had good luck getting credit from them after past outages.

There's a lot more that goes into the decision than service costs. There are lots of other costs that must be considered. E.g. if using AWS can save you one employee, that alone might make it worth the extra service costs.

I'm not saying you're wrong, just that your analysis is incomplete.

I'm sure there's a situation where AWS saves money on TCO. If you ever find it, please let us know.

Umm, does linode offer higher level services like SQS, Lambda, SWF and friends?

Their open source alternatives running on Linux?

Because ops is free.

+ the cost of paying people to setup, deploy & manage these these things.

In the past during security incidents (and there have been more three than major ones in as many years), there has been absolutely no transparency from Linode management.

That's more than enough reason for serious customers to avoid Linode...

I don't think it's appropriate for me to comment on security incidents (before my time, anyway), but I'm not trying to sell people on Linode here. MediaCrush was actually using a combination of Voxility and Digital Ocean (and Route 53, actually). There are lots of options, guys.

I was going to say the same thing. The amount of transparency in general from aws is DRAMATICALLY better than linode.

I simply don't believe the $250 - $20,000 month comparison. Could you please provide some real-world numbers/hardware/stack information?

I just did this migration (non-AWS -> AWS), and we saw a ~20% reduction in monthly prices with 1-year contract at AWS vs. physical hosting (admittedly we were getting terrible pricing with our physical hosts, but still). This will be reduced with the addition of t2.nano servers as we run enough t2.micro instances for little odds and ends that it will put a dent in the bill.

Your parent is [rightfully] including bandwidth. If he was able to get 1gbps unmetered, then the math is in the right ballpark (that's a huge if though).

Still, I agree with the parent's general sentiment re price. The gap used to be much worse, but it's still anywhere from 2x-20x depending on what you're doing.

- EC2 prices are simply high and exclude things like bandwidth.

- The CPUs are usually 1-3 generations behind (they've gotten better with this).

- Virtualization adds overhead (again, this can be very significant if you're doing a lot of system calls)

- IO options are significantly worse

- Their network connectivity is, at best, average

You said it yourself, you're comparing a "terrible" deal to AWS and only saved 20%. Sounds to me like AWS is only 20% better than terrible ;)

I think in this case I could have interchanged "terrible" with "enterprise" and it would've had a more ""correct"" meaning (seriously I wish I could keep adding quotes to all of that...). I had to fight and argue with the previous host (Rackspace) to get them to lower our prices. It also took weeks to deploy a server. Now it takes 5-25 seconds with a prebuilt AMI that I've cooked together, deployed on exactly the hardware that I want, with the network configuration that I setup, and the security rules (bi-directional! Rackspace firewall only offers inbound at the level we had) that I want/need. On top of this, to get lower prices, I open a ticket and say "Hello, I'll be needing this server for 1 year, lower prices please." No phone calls involved. I'm not trying to diss Rackspace, they seem to be doing a lot of cool things... I simply had a permanantly sour taste left in my mouth. AWS is a big juicy BBQ sandwich in comparison.

On your points about AWS itself I'm not going to argue with as they certainly are valid, however in my specific setup (everything production is c4/r3 types, 1tb gp2 volumes as all volumes, and running network benchmarks to ensure everything is properly sized) I've actually had a pretty big performance gain over dedicated servers in a DMZ behind a hardware firewall.

With regards to the bandwidth, it seems a common issue is when you have very chatty services (Apache Kafka) deployed across multiple AZ's the bill gets real big real fast. Thus far, we're not even cracking 3-digits in bandwidth monthly. Maybe we're simply not at a scale to notice these problems yet.

This is correct. The biggest cost I saw was bandwidth, followed by cycles on their machines. MediaCrush was a "media hosting website" (see original comment) and as such, used a lot of bandwidth and did a lot of work transcoding things, and had a large storage requirement.

Were you comparing bandwidth/data transfer in your numbers for EC2, S3 or CloudFront?

I don't know, it was an anecdotal story from a while ago. Before we left AWS we were utilizing all three (iirc these numbers didn't factor in CloudFront though). Point I'm making is - do the research and don't just reach immediately for AWS.

Oh okay I was just curious. I completely agree. The AWS bill at my startup went up quite a bit after the free trial ended since there are many variables that are included with each service such as how many requests/mo, tiered bandwidth, etc. The AWS calculator is nice and so is the billing estimator, but it's still quite confusing.

We started with Linode. We moved to AWS. Here's the main reason: you can't decouple disk space from RAM. So we paid thru the nose just to get more disk space. With AWS it's pay for what you need. That's the killer feature for us.

Sure but you could have also just moved to SoftLayer or any number of dedicated hardware providers for the same, cheap probably cheaper, result.

Which ones decouple disks from computer instances when it comes to paying?

As neither a AWS or Linode customer (but might become one) I have a couple questions:

- Can Linode offer short-term compute or memory intensive service? e.g., If I want to consume 100 teraflops 24x7 for about a couple weeks, or maybe 10 teraflops but a terabyte of memory. (Pay-as-you-go, not monthly/yearly subscription)

- On top of that, Can I do above with my IPython notebook code, which is already setup (it uses numpy at the backend so python slowdown is not a problem), with a step-by-step process of how to make it run on Linode (without having to install python scientific stack)?

These are some of the use cases that I think are getting popular these days, and for which AWS is known for, e.g., in the areas of machine learning, 3D rendering, etc.

Good luck trying to get huge capacity from AWS with no history.

Their service limits start low and ratchet slowly, if they have the capacity to give.

All these features and elasticity sound nice until you get big and they don't deliver.

This is categorically false. If you have large capacity needs talk to your TAMs/Account Manager and they can make it happen.

It's been true five times in the last three years for me. The elasticity isn't always there and service limits are not always granted when needed.

I don't know what an AWS TAM is but the SAs have tried and failed.

Make a new account and try to run 1000 c4.4xl in parallel for 1 hour. It won't happen.

That's not what happened the five times, but it's illustrative.

And who defines what is a need and what is a want?

That's because of a history of bitcoin mining abuse. People post their API keys on github every day and Amazon refunds almost all of the lost compute time.

I've asked and got basically a "no". I can believe that if you have a history of spending big $$$ on AWS they get increasingly flexible with raising the limits. But if you don't, they aren't going to let you spin up 500 instances, even if you ask nicely and have a reasonable justification.

I'm working at a medium, growing startup and wonder when/why to use AWS?

We currently use a mixture of root servers (DB) and VPS (App, HAProxy, NGinx) at a hoster.

* We use a Docker/Consul/Nomad/SaltStack setup to install new servers for replacing broken ones and growth

* App setup is Redis/Disque, Postgres(DB, BI) and app servers

* New dedicated servers we get in around 1h, VPS in around 1min (Only VPS over API)

* We have around 0.2 Ops FTE (devs rollout apps, ops for security updates, new infrastructure etc. and incidents)

* Price seems to be way below Amazon

0.2 Full-time employees to build all the automation, scale up (and down!) the stack as needed, monitor, operate and maintain it at an HA level? That is a huge feat, and I'd love to read about how you pulled it off. Please consider sharing your story, and perhaps your Salt recipes ;)

With all those things that you have in place, AWS would not be a big win for you, though you still might find some use of AWS services like S3 for reliable object storage, Route53 for DNS, or Lambda for "server-free" event handling. The unique thing about AWS is the huge portfolio of services they have and the absolutely amazing rate at which they manage to pump out new (and useful!) ones.

It takes some weeks to get it working, I'd say two to four, depending how often you've done it. Then for weeks there is nearly no Ops at all (security updates only mostly while the servers are humming) with releases done with Docker push by developers. Adding and provisioning a new VPS takes some seconds (and some seconds later checking), with Consul the stuff running on the VPS usually integrates itself). Sometimes hardware breaks down, then it's either setup a new VPS (setup_server <IP> <Role>) which takes around 5sec or mail the hoster if it's a DB server with a (mirrored RAID) SSD crash, which takes around 2h to fix all in all. So on average per year I'd guess 0,2 Ops FTE (which equals 10 weeks a year, which is a lot if you're not Facebook and don't need to do releases of new app versions). Today ops is relativly easy with a well trodden path, it's not 1999.

The Salt is of the shelf, with some bash scripts on top to auto generate some config and make installation of a new server one script call.

We surely could put endless hours into Ops, playing around with stuff etc. but the benefit would be marginal.

Our DNS setup is simple, not much to say, we use DNS Made Easy for failover, we use S3 though I do not consider it "AWS" (S3 is comparativly cheap).

I've wondered about Mesosphere with autoscaling etc. but with adding a new VPS in seconds setting up Mesosphere/Kubernetes taking two weeks would take quite some time to amortize.

If you're just using AWS for EC2 then you're probably better off with another provider, but I think the key advantage AWS has is in the 'WS' part of their name.

Cloudwatch, SQS, SNS, Dynamo, S3 etc offer good managed services that you can plug into your applications/systems with not that much effort - and crucially very little OPS needed.

FWIW, I'm a current Linode customer that will be migrating to AWS soon. I like Linode as a basic provider, but I really need the better features such as ASGs, AMIs, and RDS. That's worth the higher cost per hour for me.

AWS hands out credits like candy and there's tons of ways to get AWS credit or earn it. I wouldn't be surprised if cash-strapped startups built on AWS credit to go out and get funding and then just never move off of it.

Correct. Lock-in is a key part of their strategy.

Hmm. Well, somehow these heavy-hitters have presumably done their research and then chose AWS:


I see your characterization of AWS above as "terrible", but I'm curious how that squares with the (probably very well-informed) decisions these people made to use AWS.

I would not "do my research" on the sales site of one of the vendors, if that's what you're suggesting.

The poster is suggesting that the firms listed on Amazon's page did their research before buying, then settled on AWS and felt strongly enough that it was a good choice to make a public recommendation.

Or got enough of a discount/payment to make a public recommendation in return?

People pay for convenience. AWS is just convenient. Managing your own servers requires more work. It is far far far cheaper in terms of server costs. You do have to:

- Know how to maintain a server by yourself

- Hire someone who can

AWS removes part of that and thus people pay for it. An average (good enough) sys admin costs the same as a programmer.

A disclosure does not allow you to make a blanket statement like "Perhaps the first AWS mistake you might make is... using AWS?".

Reply by kawsper to your comment highlights several drawbacks of Linode. Specifically this feedback "Your API is a mess, and so is your documentation. Everything is GET-requests. And creating a new instance, shouldn't take 11 requests" alone, if true, is sufficient for me to not consider Linode for hosting. As being in an ultra small team of 2, we can't afford the luxury of API requests failing so many times.

Having used AWS since 2010, and since 2007 for backups(s3), I can say that your comparison $20,000 in AWS vs $250 is wrong. Have you assumed a very foolish selection by a user? Where in a user will just pick up all their services (e.g. RDS for a DB, etc.)

Just as a fact, we don't use RDS but install MySql by ourselves as it allows us to have other things on that instance, and also comes out cheap in comparison. On this note my contribution to the OP (article) would have been bundle different services on an instance, rather than buying stereotyped instances. Of course YMMV.

The biggest advantage is for ultra small teams, where you have comfort of a stable environment, with respect to instances launching; APIs working; volumes getting attached/unattached; snapshots getting taken; backups done on s3. All taking place automatically, while you rest peacefully.

I am not arguing that apples to apples Linode (or some other service) will not be cheaper than AWS. I am sure it will be. But still many people would like to stick with AWS because of stability and maturity of its cloud.

Also you got to give credit to AWS for pioneering hourly billing, and disrupting the cloud environment. Despite agreeing with you mainly on the cheaper point, when I moved to AWS from dedicated hosting in 2010, my monthly bills reduced.

Lastly. The experimentation. The wide variety of instances it has from micro (to the recently launched nano) to ultra large instances, allow you to experiment a lot. For example: in the past few months: I moved from couple m3.large to couple c4.xlarge, while experimenting in between on c3.xlarge type of instances.

Finally, when you make such a blanket statement, you not only demonstrate your naivete but also undermine the decisions of thousands of satisfied AWS users.

All said, I would like you (Linode) and others to be a good alternative. I am glad you are there. This is to keep AWS on its toes. Which is in my interest as their customer.

PS: Lest my handle makes someone think, I represent AWS. Please be assured I do not. I just created this handle, when I had to ask an AWS specific question (you can check my first post in this handle). And then continued with it for other things.

Edit: minor correction

As an AMZN shareholder, I encourage you to please use AWS. That is all.

I can't believe this is the top comment. This is supposed to be about AWS advice, not your personal feelings about AWS. This comment is bad and you should feel bad and everyone who upvoted it should feel bad.

This definitely reads like an ad for Linode to me and it really is turning me off from your company. Does the company know you are posting these comments on HN?

This definitely reads like an ad for Linode to me and it really is turning me off from your company. Does the company know you are posting these comments on HN?

Really? I didn't get that at all.

What I saw was a post giving negative opinions on various aspects of AWS, and stating that AWS' offerings are available elsewhere. There was an example given, of a previous project where he'd switched away from AWS to dedicated physical servers (at an unnamed provider). There was no suspicious boosterism/puffery in the direction of his own employer, like you generally see when a shill is trying to get away with something.

In fact, it seems like the only real mention he made of linode was in the disclosure that he kindly added, pointing out that he worked there and was therefore biased. Most ads don't bother to do that.

If this counts as an ad, then by those standards nobody could ever post in any thread covering the industry in which they work.

Things are looking really bad for Linode:


I feel like you wouldn't be too pleased if a super-biased Amazon employee posted a similarly anecdata-filled post about Linode. Is it really a smart idea to bad-mouth AWS in a post that really doesn't intersect with what Linode provides in any substantive way?

Considering he put the disclaimer that he is super-biased up front, I personally appreciate the insider review

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact