
Cost of hosting the Meta Discourse forum on AWS - berns
https://meta.discourse.org/t/meta-is-moving-to-the-cloud-cloud-with-lightning/72819/30
======
ljoshua
I also really liked Sam's comments at the end:

 _> The bottom line is that $10-$80 is perfectly fine for an unmonitored
monolithic setup. But once you need to start talking SLAs and need to know
this thing will be rock solid and survive random failures… well costs start
mounting._

I've run sites that do tens or hundreds of thousands of uniques a week with
base-level VPSes like Linode or DO, spending just $10-20/mo on it. But if you
require high availability or must meet strict SLAs, you start hitting the
Pareto principle pretty hard. You end up needing to pay a lot for that
increased certainty, even if you are only squeezing a few more minutes of
uptime or flexibility out of it.

~~~
insensible
One of my favorite career moments was the time I tossed up a Varnish instance
in a different data center than the site it cached, then used grace mode plus
DNS Made Easy's failover to get a significant degree of georedundancy.

There have been multiple outages that nobody noticed, some of them hours long.
Usually the backend is the one to fail, so more than 90% of end user page
loads are straight from RAM nearly all the time. And it costs $20/month or so.

------
patrickg_zill
You can, right now on eBay, get a machine with many high powered cores and
256GB of RAM, for under $3k USD.

Then, you can colocate it for $200 per month or less, with a huge amount of
bandwidth included. And $200 is not the much-cheaper colos that are out there,
so it is on the high side.

HA? Buy a second one and mirror in a different DC.

HA config via this method: $6k servers + $2k disk + $400/month colo. First
year: $12,800 (still cheaper than AWS). Second and third year: $4800 per year.

AWS is not the solution for every task.

~~~
Tehchops
>AWS is not the solution for every task.

For self-hosting/personal stuff? Sure, EC2 is not cost effective. But you've
just described a pretty powerful server for self-hosting, and you've left out
a decent chunk of context around the considerations that go in to deploying
HA, web-facing architecture.

~~~
parliament32
>considerations that go in to deploying HA, web-facing architecture

Easier than AWS, that's for sure. Getting HA to actually work in AWS is a
nightmare of proprietary configuration and obscure documentation. Hardware-in-
a-DC HA has been around approximately forever, and there is far more
documentation available on the hundreds of ways to get it working rock-
solid... all without vendor lock-in.

~~~
Tehchops
>Easier than AWS, that's for sure

You finding it hard != _is_ hard

>Getting HA to actually work in AWS is a nightmare of proprietary
configuration and obscure documentation

Strange. I have no problem with it. Most of my peers don't either. There is
_waaay_ worse documentation than AWS.

>Hardware-in-a-DC HA has been around approximately forever, and there is far
more documentation available on the hundreds of ways to get it working rock-
solid... all without vendor lock-in.

If you could point me to all this documentation you speak of that guarantees a
"rock-solid" co-lo experience...

~~~
apple4ever
Actually it is hard. Most of your peers and yourself may be familiar with the
quirks, but that doesn't mean it isn't hard. AWS's docs are terrible. Others
being worse does't change that fact.

As for rock solid colo docs- there is this thing called Google. And in fact
you barely need that. It's real easy to set up.

~~~
Tehchops
>Actually it is hard. Most of your peers and yourself may be familiar with the
quirks, but that doesn't mean it isn't hard. AWS's docs are terrible. Others
being worse does't change that fact.

I think you've confused what "facts" are...

Doesn't matter. I guess some people are just destined to remain rack 'n stack
sysadmins. Fine, more money and less competition for jobs then.

>As for rock solid colo docs- there is this thing called Google. And in fact
you barely need that. It's real easy to set up.

I think you misunderstood my point. I know there are a thousand-and-one guides
for setting up Nagios and Postfix and sticking HAProxy in front of it, but my
_point_ was that no amount of documentation can abstract away the actual pain
points of dealing with co-los, including their owners, staff, and all the fun
little problems that crop up with dealing with them.

For when it _actually_ matters beyond a dollar value, I trust AWS engineers a
lot more than I do HPE/Equinix etc...

~~~
indigodaddy
Seems you've gone a bit personal here.

------
weiming
What percent of the cost is spent on EC2 instances? Curious how much that can
be reduced by using something more "lightweight" like Elixir or even Node.
Basically, not needing to spin up thousands of OS processes. (Discourse uses
Ruby.)

~~~
jabl
Thread hijack, asking for a friend.. What's the state of forum software these
days? Some friends want to move their private/closed forum off vbulletin,
which apparently hasn't kept up with the times (in particular, mobile use is
apparently painful). Is discourse the best way these days, or is there
something more lightweight e.g with elixir as mentioned but still reasonably
mature and full featured?

~~~
msumpter
Another vote for XenForo, my previous employer ran several forums all using
vBulletin. IMO the direction of vBulletin 5 was a big miss, and like other
commenters have said, the core product manager/developer of vBulletin 3 & 4
left to start XenForo. There was some significant legal drama with the current
owner of vBulletin suing XenForo for infringement & theft intellectual
property. Those all have been dismissed in favor of XenForo (all from memory,
probably worthwhile to search this out just in case).

I've seen some enthusiasm around Discourse but for users who are used to the
look & feel of a traditional discussion forum or the workflow of vBulletin, it
can be very jarring. I believe there was a decent community split when Ubuntu
tried to retire their legacy vBulletin forum in favor of Discourse; I think
they are still running both side by side with almost different groups using
them. XenForo is a more natural progression from vBulletin and would be my
platform of choice.

~~~
h1d
Discourse looks different from the traditional style but to my surprise I got
used to using it quite fast and liking it much and not just about the
usability but with its API, customizability, administration capability and
mobile support. It shows when someone has experience creating stackoverflow.

------
l5870uoo9y
> The main piece of advice I have is … don’t do it. Don’t take on a complex,
> “enterprisey” cloud install unless you have to. It’s extremely expensive for
> what you get. Compare to a simple monolithic Digital Ocean droplet running
> our standard Docker image, which can get you a very long way even at the $40
> and $80 per month price points.

I really wonder where the line goes between a decent VPS (properly provisioned
and configured to handle traffic) and AWS?

~~~
ezrast
There isn't really a line because "a decent VPS" is exactly what EC2 is. You
use it if you

* are averse to the kind of risk that comes with being tied to a smaller provider, and/or

* are not averse to the kind of risk that comes with being tied to a provider who cares not one whit for your well-being because your business is insignificant to them, and/or

* actually have a use for, and staff with the expertise to take advantage of, any of the million proprietary bells and whistles that AWS provides, and

* are not averse to absolutely abysmal UX.

Discourse falls mostly into category 3 above; as Sam enumerates they are
taking advantage of a bunch of various AWS offerings and have split off
several of their supporting services onto their own instances. Those things
can be built up gradually; you don't just wake up one day and decide to become
"Enterprise" and watch all your costs increase by two orders of magnitude.

~~~
apple4ever
There is a line because AWS, including EC2, is more complex and unique than a
VPS.

------
CM30
Honestly, forum hosting requirements are pretty minimal in general. Indeed,
for many scripts, you can get away with a cheap shared hosting account until
you have about 50-100,000 points and 10,000 members or so. In many smaller
communities cases, they'll never outgrow it.

And for something like Discourse, well a cheap Digital Ocean VPS or something
would work quite well for most people and sites. Most aren't active enough to
put any real pressure on a hosting account.

------
pjc50
Conclusion in the comments from JefF: "The main piece of advice I have is …
don’t do it. Don’t take on a complex, “enterprisey” cloud install unless you
have to. It’s extremely expensive for what you get. Compare to a simple
monolithic Digital Ocean droplet running our standard Docker image, which can
get you a very long way even at the $40 and $80 per month price points."

~~~
sametmax
ohv has 3€ a month vps that will be largely enough for small communities, and
probably average ones.

~~~
indigodaddy
Also check out netcup, great VMs on rock solid hardware at incredible prices
(better value than OVH)

------
zitterbewegung
I’ve been running some on my side projects just using a docker containers on
digital ocean and t1.micros and apparently that was the most cost-effective
way to run it ?

------
blattimwind
(Site requires various third-party JS to display anything)

------
Tehchops
I think Sam's comment is important:

>Note it is important to have full perspective on costs here...

All too often I see the "AWS is too expensive" circlejerk devolve into a flat
cost argument.

It is _absolutely_ more expensive in that regard.

That being said, I see very few comparisons that take in to account all of the
engineering effort saved by some of the AWS feature set.

There are non-trivial things that simply _aren 't a problem anymore_ on AWS,
and time can be spent on actually interesting/difficult problems.

~~~
apple4ever
Right but then there are other non-trivial things that you have to spend on
that you wouldn't on non-AWS.

And really there are very few hints that are non-trivial and not a problem on
non-AWS either.

In the end, AWS is still more expensive time and money wise, it's just a
question of whether it's worth it and what you want to spend your time on.
(For me, spending it on the AWSy things is not interesting)

