
Use AWS Lambda to self-host the comments for your blog - jarmitage
https://jimpick.com/2016/05/05/introducing-lambda-comments/
======
dtnewman
This is a great idea. I setup something similar for my brother's website
contact form (host the site on S3 and then use lambda to shoot him an email
when someone submits their data on the contact form). The costs for running
his website are just a few cents a month (above the $0.50/month charge for
hosting on Route 53).

For those who work in Python, there is a project called Zappa
([https://github.com/Miserlou/Zappa](https://github.com/Miserlou/Zappa)) that
allows you to take an existing project in a framework such as Flask and "host"
the whole thing with lambda. The project is still young, but I imagine this
sort of thing could be very useful to people doing a hobby project that don't
want to pay for full-time hosting, but want their site to be available full-
time. Hopefully, we get to a point soon where this sort of framework is ready
for production use, though I don't know that it's there yet.

~~~
Aaronontheweb
Not to be an empty-calorie comment, but genuinely thanks for sharing that.
Found it to be very helpful!

~~~
dtnewman
Glad you found it useful. The great thing about Lambda, I think, is the cost,
which is so cheap as to be effectively free for a lot of uses (according to
[https://aws.amazon.com/lambda/pricing/](https://aws.amazon.com/lambda/pricing/),
the first million requests each month are free). So if you could design a web
service to be deployable on lambda (via zappa or whatever framework), you can
effectively host a website for close to no cost assuming you don't have a lot
of traffic. On the other hand, you lose some of the control that comes with
having your service run on a EC2 instance. For example, I like being able to
SSH in to the server to troubleshoot things when they go wrong. But for a
hobby project that I put together in a few hours, I might not care enough to
pay for the EC2 instance.

~~~
MustardTiger
Don't you need to use the API gateway on top of lambda? The pricing there
seems like it doesn't go low enough for a lot of uses (1 million requests for
$3.50 is fine, but if I only have 10,000 requests a month I am paying 100x
what I need to). Or am I interpreting the pricing wrong and that 1 million
requests could take 2 years to happen? Amazon pricing is way too confusing.

~~~
copperx
Correct me if I'm wrong, but do you really think $3.50 is too high for hosting
a website?

~~~
vertex-four
If you can write your site in PHP, nearlyfreespeech.net will host it for a
total of 3 cents a day including mySQL, or roughly 90 cents a month.

Lambda is more flexible, but not as cheap.

------
cowholio4
We use Lambda for our contact form; it's great. Our website is completely
hosted on s3/cloudfront so it's much cheaper and stable than running a
dedicated machine.

If I get some time this weekend, I'll clean it and push it up to github.

[EDIT] Here it is. This has been working for us for almost a year. If there is
interest, I can clean it up.

[https://github.com/cowholio4/lambda-contact-
form](https://github.com/cowholio4/lambda-contact-form)

~~~
appleflaxen
How does it work? You POST to the lambda service, and it emails the url
parameters with the specified formatting? This requires a separate HTML form
to be served from your website first, right?

Very cool idea! Thanks for sharing it :)

~~~
cowholio4
Yeah, I didn't really make that part clear. You have to create an API Gateway.

Luckily, AWS lets you share API definitions so I added that to the repo. It's
the swagger.json file. I'll try to get this all automated to make it super
simple, but for now you just have to replace all those placeholders.

------
jimpick
Cool. It's great to see my project on Hacker News!

Feel free to ask me any questions here, or using the comment system on the
blog post. :-)

~~~
ShakataGaNai
Love the idea. For those who aren't as Lambda savvy you might want to provide
some "real world" information on pricing. Ex: This code takes roughly 500ms to
run, which means you should expect to pay $0.10 per 96,000 comments on your
blog.

I know it's stupidly small costs (Lambda is wonderful for that), but some
people still don't understand how to translate the pricing structure to "the
real world".

~~~
jimpick
Looking at the CloudWatch Logs, each comment seems to take approximately
(Billed Duration): 400ms for the initial request, and 800-1000ms for the
worker to concatenate the request into the published JSON.

It's running in the minimum size of RAM: 128MB

~~~
prodigal_erik
This is making me wonder whether we should be paying for GC or just having
Lambda abandon workers whose heaps are full and spin up more.

~~~
pmontra
Lambda doesn't destroy the instance at the end of the response. It keeps it
running just in case other requests arrive. If that happens you might want GC
or a worker eventually runs out of memory.

What could happen with Lambda and similar services is that costs will take
back under the spotlight "old" languages that run with little memory and
little CPU cicles. I'm even thinking about C. I'm not into Go, but that looks
like another candidate.

Scenarios: those ubiquitous web services running on a single VM because it's
more than enough. The customer has to pay for it all the time and could save
money if the app can run on Lambda. Enough to pay more in developer time?

~~~
jimpick
What surprised me with Lambda is that it 'freezes' the VM your code is running
in, and 'unfreezes' it when more requests come in. Very clever - they can
store a lot of frozen VMs on an SSD. It surprised me when I discovered that a
cache I had written in my code was already populated!

I think most code running on lambda is not going to be CPU bound. In lambda-
comments case it's mostly going to be idling as it waits for TCP/IP traffic
to/from Akismet, S3, DynamoDB, Slack notification webhooks, etc.

------
nickjj
Have you been able to quantify the exact pricing per month?

From the looks of the git repo it's at least 65 cents per month but then
there's a number of other services with no price reference.

What would you estimate it would cost per month to host a blog that gets 50
comments per month and what would the cost look like at the end of the year
given costs would rise over time due to needing to store more info.

For argument's sake, let's assume no AWS free tier.

~~~
jimpick
I don't have "real life" numbers yet, since I just set it up yesterday. I've
used up my free tier, so I should be able to get real numbers from Cost
Explorer. I'm not sure if everything is tagged nicely for that.

On paper, it should cost under a dollar per month. The main cost is just the
DynamoDB instance. I don't think it's going to consume much S3 storage.

I guess if your blog has a lot of traffic, the network transfer might add up
-- but it's just little JSON files.

~~~
cdnsteve
With the HN burst traffic, it would be good to report back costs after things
settle down :)

~~~
jimpick
Cost report:

[https://jimpick.com/2016/05/05/introducing-lambda-
comments/#...](https://jimpick.com/2016/05/05/introducing-lambda-
comments/#comment-BGy5U1MkSzCEAss7bcaVOg)

For 8000 visitors, it cost me $1.37.

The numbers might be skewed upwards a bit because somebody posted a gigantic
blob for one of the comments to test if I had implemented a limit on the
length (I hadn't).

~~~
nickjj
Thanks for the solid numbers based on real data. At the time of this posting
there's 103 comments too which IMO seems waaaaay high given 8k visitors due to
the nature of the post.

Hard to say for sure but I wouldn't be surprised if a normal blog received 5-6
comments per 10,000 views.

------
nzoschke
This is extremely cool. I'm starting to see a serious new trend, where
automating around IaaS can completely eliminate the need for SaaS.

Why pay for a subscription comment service when it can be implemented as a
script in the cloud that only runs on-demand?

------
jimpick
Here's a little timelapse video of the Google Analytics real-time page showing
4 hours of being on the home page of Hacker News. :-)

[https://twitter.com/jimpick/status/728712464687239168](https://twitter.com/jimpick/status/728712464687239168)

------
eistrati
Great work! The fact that data is stored in my account (to be read: under my
control) instead of outsourcing to 3rd parties is a huge plus.

If you will be able to make it less "technical" and more "plug-and-play", I
think this will be cool viable solution to integrate (to be read: for
customers like ours, that uses serverless computing in enterprise apps ->
[https://github.com/MitocGroup/deep-
framework](https://github.com/MitocGroup/deep-framework)).

------
jrockway
I eliminated all spam and abuse from my blog by requiring comments to be PGP-
signed :)

I did still get some comments too!

~~~
copperx
The statement in your website "On some level, the era of personal websites is
over" made me sad and nostalgic. I'll keep it old-school by maintaining my own
personal website.

~~~
imglorp
I don't think that's true. It might at least be accurate to say there's an end
to open comment sections and other abuse magnets.

Anybody can still make a site in a few minutes on a number of places, they
just aren't going to solve the abuse problem themselves without using a
hardened tool.

------
alasano
I just finished setting up my personal site with Hugo+S3+Lambda yesterday.
Lambda runs Hugo when the S3 bucket detects a file change, takes about 5000 ms
per blog post which is fractions of a penny.

Although this adds a bit of complexity compared to Disqus it sounds like a fun
solution to implement, potentially with the addition of a way to moderate
comments.

~~~
jimpick
A future project for me is to see if I can render the comments out to a string
at the moment when Hugo is rebuilding my blog.

Then the comments would be in the DOM before the JavaScript is even loaded.
I've done that with React apps before - it works great.

If that was working, I could rebuild the blog after every comment even, using
a system like yours.

~~~
alasano
I think the serverless(ish) space is headed in an exciting direction overall,
especially if we can drastically reduce wasted compute time. I'm watching your
repo and hopefully will get inspired! Thanks for the reply Jim

------
stephenr
A tool that is utterly dependent on 5 aws-specific services is not what I'd
call "self hosted".

It's also not "serverless" as the repo readme claims.

~~~
cdnsteve
5 AWS services that don't need patching or maintenance from a developer, seem
serverless to me.

~~~
spriggan3
> seem serverless to me

Why do you feel the need for yet another marketing buzzword ? IMHO saying that
makes one sound like an someone that don't know what a server is, but maybe
it's just me. Just because you're not managing a server doesn't mean it
doesn't exist, it obviously does, the code isn't running on each client but
remotely.

~~~
jdminhbg
> Just because you're not managing a server doesn't mean it doesn't exist

Wireless internet is still connected to the rest of the internet via a wire
somewhere; the obvious point being made is that your phone or laptop doesn't
need a wire. Serverless here doesn't mean that servers no longer exist, but
that you are not working at the server level to accomplish what you're trying
to do.

~~~
dragonwriter
> Serverless here doesn't mean that servers no longer exist, but that you are
> not working at the server level to accomplish what you're trying to do.

So, just like every PaaS everywhere, since long before the new buzzword.

~~~
pmontra
Understood, but that means that Rails has been serverless almost forever
because of Heroku. The console is simpler than the one of AWS and you deploy
with git push. You didn't have to worry about the db because you only had
PostgreSQL for a long time. And you could buy add on services to do almost
everything. It just happened before the buzzword was born.

------
mattbeckman
Lambda is great. Few friends and I just put up what we call LDAN (Lambda,
DynamoDB, Aurelia, and Node) aka Lieutenant Dan. Pretty cool refreshing change
that is in stark contrast to our larger EC2+MongoDB stack.

[http://ldan.io/](http://ldan.io/)

~~~
pmontra
I went for the demo and I stopped right away when it asked me to login with
Google or Facebook. I went to the repo and it's very light on documentation.
At least I learnt about aurelia.io

------
pjc50
I appreciate your realisation that spam and abuse are #1 implementation
problems in commenting systems.

------
seandavidfisher
One step closer to my dream of "Wordpress on Lambda"!

What other projects exist to run a blog on Lambda? Apart from going full in on
a locally-compiled static site that I just upload to S3, that is? (Hugo, Hexo,
Jekyll, etc) I'm imagining a full admin web interface that uses a Lambda-
driven API, themes, a front page with dynamically generated content from the
Lambda-driven API, etc.

~~~
xemoka
Zappa is pretty cool, I haven't had a chance to use it myself but want to find
a project to apply django-zappa ( [https://github.com/Miserlou/django-
zappa](https://github.com/Miserlou/django-zappa) ) on django-cms or wagtail...
not too sure how that'd work out though.

~~~
Mizza
Zappa author here - Wagtail works great on Django-Zappa. I have also been
slowly writing my own CMS, [https://github.com/miserlou/zappa-
cms](https://github.com/miserlou/zappa-cms), although that's not ready for
public consumption yet!

------
bikamonki
As you describe the goals and features of your system, I think the AWS
solution is too complicated, even over-kill, for what you are trying to
achive. I use AWS extensively for the 'big projects', I also use it as a CDN
for all projects, but for the small projects (in terms of complexity and
capacity), I find it much simpler to implement a two-tier service. For
example, for comments (form submission in general), it is much simpler to
write a node app, host it on a free-tier and persist the data also on a free-
tier (for example Heroku+Firebase). Parse.com was great b/c you could do it
all in one tier since the offered server side logic.

Granted, if you ever pass the free tier limits, normally the price breaks of
these PAAS providers are high so cost does not scale 'by the cent' like AWS;
however, if you pass the free tier you must be making good traffic which
should translate to good income so infra cost shouldn't be a problem.

~~~
bikamonki
Downvote all you want but IMO this is a Rube Goldberg machine that uses 5!!!
AWS services to persist what? 5k comments for the entire lifecycle of the
website?

The modern programmer is quite good at playing Legos with the myriad of micro
webservices out there (which is good play/training) but has forgotten how to
think like an 'engineer' must think.

~~~
jimpick
Any working solution actually has thousands of components.

I'd defend this particular Rube Goldberg machine by pointing out that 99% of
the time, none of my code is running on the server side - it's all Amazon S3.
The only time my code runs in the cloud is when a comment gets submitted.

I'd argue that it's pretty easy to reason about. I've got some more blog posts
coming. :-)

~~~
bikamonki
Don't get me wrong, what you've done is how we will build web stack solutions
from now on: orchestrated microservices. My hopefully constructive critique is
that as engineers we ought to find the best solution for a given problem, even
at the expense of such solution not being universal. Take the myriad of tiny
to medium websites that use WP, it is like renting an 18-wheeler to move one
tiny box. The engineered correct solution would be a static site. Now, for a
site with just a handful of posts and comments we should think of a much
simpler solution for comments persistence.

In fact, after reading about your software I am inspired to think and try to
come up with such solution, let's see :)

------
wnevets
I used google sheets as my database for a similar setup with s3 and lambda.
Sheets is completely is free AFAIK

~~~
imaginenore
Me too. And it has an API.

------
akkartik
Very interesting! I'd like to migrate off Disqus, but I'd like to not leave my
old comments orphaned. Do people know of any export services? (This may be an
ill-posed question since comments arguably belong to their authors..)

------
thenomad
This is really cool. Nice one.

It will _desperately_ need a moderation queue.

Even smaller established websites get a ton of comment spam these days. I'd
not even consider setting up any commenting system that doesn't have robust
spam protection.

~~~
jimpick
That's definitely on the To Do list.

I need to add a little authentication interface for the admin - the complexity
starts to pile up. But I think it can be done nicely.

~~~
tracker1
The admin interface could be something that runs local though, would save the
time in terms of adding authentication.

~~~
jimpick
I've already had to edit the JSON file on S3 to remove/edit a couple of
comments. That was pretty easy to do from the command-line, but I think most
people will want a web interface.

------
ekiara
Pricing for AWS Lambda is complicated, but reading the examples clarifies
things... a bit.

Is The Lambda free tier something separate from the free tier you get on first
signing up for AWS? I used up my AWS EC2 free tier years ago and I don't think
you are eligible again once you use your free tier. I wish they'd put regular
pricing at the top instead, i.e. for someone who doesn't qualify for the free
tier you start paying from request #1.

------
e12e
Nice project.

I wish there was away to get stuff like this to work in a way that didn't take
an all-or-nothing approach to noscript-filtering. As it is, now I have the
option of no comments, or allowing scripts from: "s3-us-west-2.amazonaws.com",
which almost might as well be "anywhere and everywhere". FWIW I see a similar
problem with cloudfront and CDNs in general.

------
euphemize
I am not sure when this was written, but Lambda supports ES6 directly as of
about a month ago. [https://aws.amazon.com/blogs/compute/node-
js-4-3-2-runtime-n...](https://aws.amazon.com/blogs/compute/node-
js-4-3-2-runtime-now-available-on-lambda/)

I think you can avoid the transpiling/babel dependency.

Great work!

~~~
jimpick
Yep, I started it before that. I'd love to lessen the dependency on
Babel/Webpack as much as possible.

I'm still using async/await, ES6 modules and some other more experiment ES-
next features, so I think it will be awhile before Babel/Webpack can be
completely removed. Of course, I don't really have to use those features. :-)

~~~
tracker1
If the babel/webpack builds happen before deployment to Lambda, I'm not sure
it really matters at this point... once we see Node > 7 or so drop, with ES6
module support, it might then be time to consider dropping babel, but it's
going to be around a while, and still nicer in a lot of ways than
alternatives.

------
hn_user2
This is really cool to see! I have had this type of thing on a back burner for
a long time. Glad to see someone wrote it.

I wonder if the author considered AWS SimpleDB instead of dynamo. Costs are a
bit cheaper.

[https://aws.amazon.com/simpledb/](https://aws.amazon.com/simpledb/)

~~~
jimpick
I'm not really using DynamoDB to store data. I'm using it to sequence parallel
requests going into the 'worker' that concatenates things together -- so two
simultaneous incoming comments don't clobber each other.

At the moment, I think only DynamoDB Streams and Kinesis event sources will
batch events together and deliver them to a single function (instead of many
parallel functions). It's hard to understand just by reading the docs.

[http://docs.aws.amazon.com/lambda/latest/dg/intro-core-
compo...](http://docs.aws.amazon.com/lambda/latest/dg/intro-core-
components.html)

I think the platform is evolving quickly, so I wouldn't be surprised if they
come up with some more options in the future. I'm using DynamoDB streams since
it's cheaper than Kinesis.

~~~
Arelius
The problem with DynamoDB is it has minimum provisioned throughput
requirements that would make this rather expensive for a blog that was not
free-tier eligible even if it got no traffic at all.

In contrast, SimpleDB seems to only charge for actual throughput used.

~~~
jimpick
Correct. I'm provisioning the minimum amount. I'd love to find a cheaper
solution.

------
amelius
I wish more people added a comments section to the documentation of their
software projects.

~~~
jimpick
I think it might be possible to embed lambda-comments into an iframe that
could then appear on GitHub README.md files, etc. It's just an untested theory
for now. ;-)

~~~
kasbah
I don't think you can. [http://stackoverflow.com/questions/11804820/embed-a-
you-tube...](http://stackoverflow.com/questions/11804820/embed-a-you-tube-
video)

You might be able to on Wikis though.

------
yarper
You could also use elastic-beanstalk, which I think is also free for very low
traffic sites.

~~~
jimpick
AWS has a free tier that gives you some free resources for the first 12
months. After that, you have to pay.

------
jasonszhao
Funny how it says at the bottom: "Error loading comments."

~~~
jimpick
I haven't done lots of testing on it yet. I just found out that happens when
browsing in 'Incognito' mode. Probably a bug.

~~~
jimpick
I was using localStorage, which apparently you can't do if you are on Safari
running in private mode. I've changed the code, and it should work now.

------
rando289
Use a hosted service to self-host? Come on. Does it also have more cloud in
it?

~~~
jimpick
In the blog post, I do link to some projects you can run on your own server.
Actually, it wouldn't take much work to modify the code in this project to run
in a Node.js daemon. I might just do it anyways to make testing/dev easier.

~~~
tracker1
Been working, slowly-but surely in terms of writing my own blog engine (static
generator) and then publishing to gh-pages, but was considering using discus
for comments... thinking of doing something similar as a docker contained
service that I can throw up on my dokku server.

TBH, the thought of managing the attack surface of comments is kind of
scary... Will definitely be referring back to this.

~~~
jimpick
My approach was to just pump the entire comment through the "markdown-it"
markdown processor, which promises to emit safe HTML. It's a popular project,
so I'm banking on the fact that they go a good job of sanitizing in their
pipeline.

------
pitchka
Why not just adapt Isso for this? It already has a bunch of nice
functionality.

I personally just delegate the commenting to github and render the comments
with javascript through github api.

[http://ivanzuzak.info/2011/02/18/github-hosted-comments-
for-...](http://ivanzuzak.info/2011/02/18/github-hosted-comments-for-github-
hosted-blogs.html)

Here's a nice overview of how this works.

~~~
jimpick
That's pretty cool.

