

Digital Ocean: Ubuntu, Nginx, Unicorn, Rails - jonmccartie
http://blog.mccartie.com/2014/08/28/digital-ocean.html

======
meritt
Great writeup. One suggestion on the nginx front I might suggest is you add an
entry to drop requests for unknown hosts. e.g.
[http://104.131.41.220](http://104.131.41.220)

[https://github.com/h5bp/server-configs-
nginx/blob/master/sit...](https://github.com/h5bp/server-configs-
nginx/blob/master/sites-available/no-default)

That github repo is a goldmine for understanding nginx configs too.

~~~
latch
other things to consider:

    
    
         # if you compiled --with-http_spdy_module
         listen 443 ssl spdy;
    
         # amusing how many of these you'll get
         location ~*\.php {
           add_header "Not Found" 404;
         }
    
         # pretty sure you need to do this to have keepalive working
         # between nginx and the upstream
         proxy_set_header Connection "";
         proxy_http_version 1.1;
    
         # buffer writes to disk (for a busy site, you can use much larger values than 1K)
         access_log /var/log/nginx/access.log buffer=1K;
    
         # cache the ssl connction parameters
         ssl_session_cache shared:SSL:20m;
         ssl_session_timeout 10m;

~~~
jonmccartie
The .php block gave me a good laugh. Great idea. Just rejecting /wp-admin.php
should reduce load significantly. :)

------
akbar501
Also worth noting that you should run a firewall as part of the basic
configuration. AWS includes this via the Security Groups, but with DO you'll
need to use iptables or ifw.

~~~
jonmccartie
Good point. I didn't mention it in this post, but DO has a great article on
getting started with firewalls:
[https://www.digitalocean.com/community/tutorials/how-to-
set-...](https://www.digitalocean.com/community/tutorials/how-to-set-up-a-
firewall-using-iptables-on-ubuntu-14-04)

------
xtrumanx
Quick question; I don't know much about setting up a Linux server but I found
it interesting the post had nothing related to security besides setting up a
ssh key and separate user for deployment.

What security-related tasks do you do when setting up a new server? Besides
the above, the only things that come to mind for me are:

1\. Change ssh port from default. 2\. Block unwanted traffic via iptables. 3\.
Protect ssh with fail2ban.

~~~
akbar501
Some very basic security:

1\. Change SSH port

2\. Block all password logins

3\. Disable root login (namely, a non-root user must login via a key, then su
or sudo to root)

4\. Lock down everything with IPTables

5\. Allow only encrypted connections to the few ports open via IPTables
(normal exception is port 80 for http).

6\. Segment the network (with DO private networking your machines are open to
anyone within the same facility, so you need to isolate your private network
from other DO customers)

7\. Lock down each running service/daemon per it's standard configuration. For
example, if you're running Postgres, then lock it down per pg best practices.

8\. Create individual, non-privileged users for each service that you're
starting. ex. If you're running an app server, then it should have its own
user.

9\. Sandbox everything that can be.

10\. Turn off everything that is not being used, then uninstall it.

11\. If you don't need them, then uninstall build tools such as compilers.
Have a separate build machine.

12\. Only open edge of network machines to the outside world. On DO, turn off
eth0 on all machines that customers do not directly interact with.

13\. All services that are internal user only should have public networking
totally disabled. Ex. Redis should only run on a private IP, should have the
public interface disabled, and should only be accessible via a secure
connection (ideally with some type of true authentication, replay protection,
etc. Or, just run these services on a trusted network and pray that nobody
penetrates your network deep enough to find out that you're totally
unprotected once inside your network.

14\. Don't keep private keys/certs in places that they are not needed.

15\. Have a map of each service, the user that it runs under, what resources
that user has access to, and a general map of what sequence someone has to
follow to control your network. If you catch someone mid-attack, you can start
to lock off parts of your network.

16\. Logging! And monitor them for attacks. And ship logs off the machine in
question so that you can review logs even if a machine is compromised.

There's lots more, but this is a basic list that should be configured via
Ansible/Chef/Puppet/Salt for each box.

One other item I've heard is that some people like to run heterogeneous
networks.

~~~
xtrumanx
Thank you so much for all that. I took a great introductory Linux course by
the Linux Foundation over at edx[0] which really filled me in on a lot of
stuff about Linux one misses out on when they learn on their own. Sadly, the
subsequent Linux Foundation training courses are way out of my price range.
Even the intro course offered I took for free on edx is offered by the Linux
Foundation for thousands of dollars.

The good thing is I can mostly learn stuff on my own but have a hard time
figuring out what I should be learning. Your comment helps a lot on that
department. Even if I don't quite understand how to do some of the stuff your
saying e.g. segment the network? Sandbox? Isn't eth0 my primary port to
connect to my server? Which logs should I be monitoring (I've only heard of
auth.log)?

Thanks.

[0]
[https://www.edx.org/course/linuxfoundationx/linuxfoundationx...](https://www.edx.org/course/linuxfoundationx/linuxfoundationx-
lfs101x-introduction-1621)

------
frik
I heard Ansible over SSH is good to automate the installation. Has some
experience with it? I am interested in a good tutorial similar to OP's
article.

~~~
elithrar
Have you looked at these?
[https://www.digitalocean.com/community/tutorials/how-to-
crea...](https://www.digitalocean.com/community/tutorials/how-to-create-
ansible-playbooks-to-automate-system-configuration-on-ubuntu) \+
[https://www.digitalocean.com/community/tutorials/how-to-
inst...](https://www.digitalocean.com/community/tutorials/how-to-install-and-
configure-ansible-on-an-ubuntu-12-04-vps)

~~~
castell
Thanks for sharing! Great that DigitalOcean has many useful tutorials.

The tutorial doesn't touch the "replace" command
([http://docs.ansible.com/replace_module.html](http://docs.ansible.com/replace_module.html)),
that seem to be useful to modifying existing config files.

------
davidlumley
I used to prefer the flexibility and efficiency of having my VPS setup, and
it's certainly cheaper. Over the past 2 years though, I've saved so much time
and sanity using Heroku.

I certainly can understand saving thousands of dollars and improving
performance by moving from Heroku to a VPS or bare metal solution, but $90 is
not uncomfortable enough for me to warrant the change.

Interesting article!

~~~
donw
Question -- do you have any background as a sysadmin? If not, then I can
totally understand the attractiveness of something like Heroku. It really does
take care of a lot of nastiness.

From my side, I've got a _fantastic_ Ansible setup for databases, security,
ssl certificate distribution, and so on, and so Heroku doesn't really buy me
anything for the extra money.

~~~
davidlumley
None as a fulltime sysadmin, most of my experience has been in operations, and
in configuring servers for my own projects.

I've used Puppet and Chef before, haven't checked out Ansible. My main
dissatisfaction with those tools was that there wasn't a super easy way to get
started on a new VPS. I had to configure roles and settings to install Puppet
or Chef, and _then_ I could get started automating things. It's certainly not
hard, but I've never been in a position where it was something I did full time
so have really valued platforms that do a lot of this for me.

~~~
donw
That's why I picked Ansible. The only dependency is SSH, and you can write
modules in whatever language you choose.

Plus -- and this is a big win -- Ansible actually has usable documentation and
practical examples, which coupled with a really clean design, make it way
easier to get developers up to speed on the operations half of things.

~~~
frik
Can you recommend tutorials, examples for Ansible? Is Ruby knowledge required
to efficiently use Ansible? I would like to automate the installation of
nginx, php, nodejs, mysql, etc. incl. modify a few config files.

~~~
willthames
Ansible is mostly configuration based - Ansible itself is written in python
but all you'll typically need to write for most usages is configuration in
YAML. So no ruby at all.

There are likely existing roles for most things at
[http://galaxy.ansible.com/](http://galaxy.ansible.com/)

The Ansible docs at [http://docs.ansible.com/](http://docs.ansible.com/) are
pretty good, and there are lots of getting started guides most weeks on the
Ansible weekly mailing list ([https://devopsu.com/newsletters/ansible-weekly-
newsletter.ht...](https://devopsu.com/newsletters/ansible-weekly-
newsletter.html))

And there are a few pages on my github blog on a few fundamental principles of
modelling configuration e.g. [http://willthames.github.io/2014/03/17/ansible-
layered-confi...](http://willthames.github.io/2014/03/17/ansible-layered-
configuration-for-aws.html) [http://willthames.github.io/2014/04/02/modelling-
credentials...](http://willthames.github.io/2014/04/02/modelling-credentials-
configuration-in-ansible.html) to give a bit of flavour of how to manage more
complicated setups. Installing things is easy, getting the configuration right
for your needs is the tricky bit!

------
locusm
What about backup, check out Duplicity for backing up to S3. Rackspace has
quite a nice built in tool for backups too if you ever leave your current
setup. Duplicity [http://duplicity.nongnu.org/](http://duplicity.nongnu.org/)

------
buttermint
Would love to see a part two of this, for when you app gets popular...
covering three simple (but perplexing for beginners) things:

1) dedicated database server 2) two application servers 3) git push to
multiple application servers

~~~
jonmccartie
Great idea, thanks!

------
tovmeod
> sudo apt-get install nginx

no, the ubuntu repository is outdated, you should add the nginx team ppa:

sudo add-apt-repository ppa:nginx/stable

now you can 'sudo apt-get install nginx'

~~~
xxdesmus
Another potential repo:

[https://launchpad.net/~chris-lea/+archive/ubuntu/nginx-
devel](https://launchpad.net/~chris-lea/+archive/ubuntu/nginx-devel)

------
MaxGabriel
I think this is a typo; "deafult" instead of "default"

    
    
        nano /etc/nginx/sites-enabled/deafult

~~~
jonmccartie
Ack! Good catch! Fixed!

------
thejerz
I wouldn't recommend running Nginx, Unicorn, Rails, Redis, and PostgreSQL all
on one instance. Better to offload the databases onto their own VPS's.

~~~
threeseed
Even better idea is not to host the databases at all.

Only those who have never experienced a corrupted backup or failed slaves
think a database is something that is relatively trivial to manage.

You're much better off looking at platforms like RDS, MongoHQ, Cloudant etc.

~~~
sespindola
In my experience, hosted database providers are almost always either cost or
latency prohibitive.

Nowadays, in PostgreSQL is quite simple to replicate a db via log shipping[1].
You can even stream the WAL to an S3 bucket.

[1]: [http://www.postgresql.org/docs/9.3/static/warm-
standby.html](http://www.postgresql.org/docs/9.3/static/warm-standby.html)

~~~
threeseed
I currently use MongoHQ and get response times in the tens of milliseconds for
$18/month. Hardly cost or latency prohibitive. I've seen similar times for
less money with Cloudant.

Making sure that master/slave scenario works when you need it to requires
extensive testing and care. You also failed to address backups. I've yet to
meet anyone who runs their own database who actually tests their backup.

------
dmourati
I find the references to the nano text editor endearing.

~~~
jonmccartie
LOL! It was intentional. :)

------
michaelbuckbee
While I appreciate the time and effort put into both this hosting move (and
the subsequent writeup), I really can't help but feel the time and effort
would have been better spent on gaining more traffic and users than on the
move.

OP is saving $90/mo, less than the cost of adding just one new monthly
subscriber at his 'Premium' plan.

~~~
jonmccartie
It didn't take long ... and I wanted to take a break from staring at my non-
performant Facebook ads and making cold calls.

