
How to set up continuous deployment using free hosted tools - simonw
https://simonwillison.net/2017/Oct/17/free-continuous-deployment/
======
jfaucett
I can't believe the author doesn't mention gitlab. Setting up full CI on
gitlab is amazing. They have the easily integratable gitlab runners i.e.
travis-ci functionality + different build stages for automatic deployment via
ansible/chef/whatever and unlimited free private projects to boot. If you're
working on anything that can't be public I highly recommend gitlab. Its an
amazing service.

FYI: I am in no way affiliated with Gitlab.

~~~
tnitsche
> can't believe the author doesn't mention gitlab.

I absolutely agree. We use Gitlab in our company for continuous deployment and
it works like a charm.

Apart from the other tools I miss some performance monitoring. Sentry is
great, but if your application suffers from bad performance after the latest
deployment, you won't notice that soon enough.

~~~
sytse
We're working on adding more performance monitoring to GitLab
[https://docs.gitlab.com/ee/administration/monitoring/perform...](https://docs.gitlab.com/ee/administration/monitoring/performance/introduction.html)

------
jonatanheyman
A great self-hosted CI alternative is Drone.io
([https://github.com/drone/drone](https://github.com/drone/drone)).

To set it up is simply a matter of starting a Docker image and creating a
.drone.yml file (very similar to .travis.yml).

Note: I'm not affiliated with the project in any way. I'm just a happy user
who think the project deserves way more buzz.

------
NightMKoder
Once you need something a bit more heavyweight than Travis (e.g. more CPU) I
would highly suggest Buildkite
([https://www.buildkite.com](https://www.buildkite.com)). It’s a “bring your
own machines” solution, which means you can have beefy machines (e.g. via
buildkite on EC2 spot autoscaling), or even machines that run inside
production without handing buildkite itself any secrets. We’ve been using it
at Ladder and it’s been phenomenal.

Also it has a lot of emojis.

~~~
nicodjimenez
+1 for mentioning Buildkite, also works great for machine learning

~~~
mopierotti
Is there a particular feature that's suited to machine learning? I'm having
trouble understanding the connection.

~~~
spinlock
the way one process can feed into the next. you can setup data pipelines that
are nice for ML projects.

------
callumlocke
Heroku does have a free tier, but it's intended for experiments. It's very
limited and wouldn't be acceptable for any serious website, certainly not a
'world class' setup. Unless you're very careful to have one free dyno per
account, and avoid using features like staging apps, they will power down your
server for several hours a day.

~~~
simonw
Yeah, the “world-class” hook was more about the quality of these tools than
their applicability for use at scale for free - I encourage people to pay real
money at various points in the article.

I’m currently running my blog on a single $7/month Heroku Dyno (plus $9/month
for Heroku PostgreSQL) behind a free Cloudflare account - with the right
caching headers I can easily handle huge amounts of read traffic. I was going
to mention Cloudflare in this piece but I decided it didn’t fit the theme of
continuous deployment as well as the other tools.

------
milancurcic
> Once you’ve set up Travis, it will automatically test every commit to every
> branch

Small correction: It will automatically test the latest commit on every push
to every branch

~~~
simonw
Thanks - I updated that sentence to say push instead of branch.

------
kellyjandrews
I'm the developer advocate for Codeship.

Codeship ([https://codeship.com/features](https://codeship.com/features))
allows forever free builds for public repos. It has support for both
Continuous Integration and Delivery/Deployment. Maybe you can give it a spin
as well?

~~~
wahnfrieden
We did not pursue Codeship due to lack of a "codeship.yml"-style configuration
so that we could track config in source control.

~~~
moritzplassnig
Codeship founder here. We support .yml file config for our container/Docker
based product (called Codeship Pro). We're looking into offering a UI + yml
file approach for both products because we see a 50-50 split when it comes to
what people prefer. Thanks for the feedback :)

------
kkapelon
>The integration of Travis with GitHub runs deep

At the moment there are 10+ companies that do the exact same thing.

[http://blog.codepipes.com/hosted-ci-comparison/hosted-ci-
com...](http://blog.codepipes.com/hosted-ci-comparison/hosted-ci-
comparison.html)

If you don't have time to click the link:

    
    
      - Buddy Works
      - CircleCI
      - Codefresh
      - Codeship
      - DeployBot
      - Distelli
      - SemaphoreCI
      - Shippable
      - SolanoLabs
      - Vexor
      - Wercker

~~~
manigandham
[https://buddy.works/](https://buddy.works/) is great, fast builds, slick UI,
and a free self-hosted enterprise version.

------
drinchev
Only comment I would make is that you can use CircleCI instead of Travis to
get for free running tests on private repos. The configuration is similar and
the benefits as well.

~~~
simonw
Thanks for the tip - I just added a note about Circle CI to the article.

------
grogenaut
Doesn't show any automated verification of code in deployed fashion, no load
tests or verifications which would be part of a world class pipeline. Also no
canary or red-blue tree type rollout hooked to sentry. At least it has a
staging but don't know what they're using it for other than to test migrations
before promoting to prod. Nice little write up but far from world class.

~~~
sublimino
Exactly - continuous deployment is about moving some/most testing to
production, not eliminating it altogether.

Acceptance testing the system once deployed is the cornerstone of
organisational acceptance of the CD doctrine - errors will always happen, it's
just that they can now be fixed much faster in production.

Using an acceptance test suite as a deployment safety harness should
dramatically increase the quality of tests, and hopefully the application too.

------
oneweekwonder
> Need more? Pay for it!

or be willing to move vendor and re-implement the stuff.

For comparison my magic combination(and not all free or realistic for
everyone):

    
    
      - gitlab
      - jenkins/looking into gl ci
      - linode/cloudafrica(ah-hah hosting is not free)
      - sentry is a tough one. We rolling a custom mqttwarn/zabbix solution
      - irc(fine I will take your slack and use the irc-bridge pfftt)

~~~
jherdman
I really wanted to use Gitlab for my next side project, but Circle CI doesn't
support them, and I really don't want to pay Travis for my shitty proj.
Hopefully this changes some day soon.

~~~
enzanki_ars
Gitlab has great built in support for CI [0], and you can easily add and run
your own CI runners [1].

[0]:
[https://docs.gitlab.com/ce/ci/README.html](https://docs.gitlab.com/ce/ci/README.html)

[1]:
[https://docs.gitlab.com/ce/ci/runners/README.html](https://docs.gitlab.com/ce/ci/runners/README.html)

------
jrudolph
Heroku is awesome for this use case, but there are also great alternatives
based on Cloud Foundry. Cloud Foundry is an open source PaaS with support for
Heroku buildpacks + docker containers. At a median of ~5€/GB RAM per month
they will also be a bit cheaper than Heroku.

Some providers that offer Cloud Foundry including a free tier for small Apps:

\- Meshcloud (hosted in Germany, 2 Locations)
[https://panel.meshcloud.io/#/register](https://panel.meshcloud.io/#/register)
(Disclaimer: I'm a founder there)

\- Pivotal Web Services (hosted on AWS US):
[https://run.pivotal.io/](https://run.pivotal.io/)

\- IBM Bluemix [https://www.ibm.com/cloud-
computing/bluemix/](https://www.ibm.com/cloud-computing/bluemix/) (various
locations, a bit more expensive than the other two though)

------
adnanh
Shameless plug: I wrote an open source incoming webhook server that can run
your build & redeployment scripts:
[https://github.com/adnanh/webhook](https://github.com/adnanh/webhook)

~~~
nullify88
Thank you very much for this!

We use this to listen for Github webhooks and execute jobs with Saltstack.

------
bfirsh
I have a similar setup, but using Docker images instead of Heroku buildpacks.
Buildpacks are horrible if you want to do anything custom.

To make this work, I use Codeship instead of Travis. Codeship builds a Docker
image, runs the tests against the image, then pushes the image to Heroku if
the tests pass: [https://documentation.codeship.com/pro/continuous-
deployment...](https://documentation.codeship.com/pro/continuous-
deployment/heroku/)

------
AdeptusAquinas
You can also do every step listed, for free using a single platform: VSTS
(Visual Studio Team Services).

Online, free for small teams, Git source control, CI/build/release, supports
every major build stack, not just MS. It even has a top quality agile work
tracker, that I've found better (or at least more balanced) than Jira/Trello
equivs.

EDIT: link: [https://www.visualstudio.com/team-
services/](https://www.visualstudio.com/team-services/)

------
planetjones
Not sure bitbucket pipelines are mentioned here. But for free I manage to
commit Hugo source files to a private git repository, generate the static
files by spinning up a docker container and generating them using Hugo and
finally rsyncing the static output to my Webserver. All for free. You only get
50 minutes of time now (for free) in the docker instance but for simple
continuous deployment cases like mine it’s very effective.

~~~
Domenic_S
Why is that better than generating static locally? Seems like a lot of
fiddling for no benefit.

~~~
planetjones
It’s fully automated. I do a commit and push from any machine that I have
cloned the repo to and it gets published automatically.

Now I don’t need to download Hugo, remember the Hugo commands, remember my web
server login details, etc. I think it’s a lot less fiddling as all I need to
remember now is how to add a file to a git repo to get a post published.

~~~
Domenic_S
Don't you miss out on Hugo's archetypes doing it that way, or at least need
the templates locally? Also you shouldn't need your web server login details
for rsync... Hey whatever works, it just seems like substantially more work to
me.

------
benhoyt
I just set up CI tests for some of my Python libraries on GitHub, and another
tool to note is Appveyor -- very useful for running tests on Windows (which
Travis doesn't do yet). Example:
[https://github.com/benhoyt/scandir/blob/master/appveyor.yml](https://github.com/benhoyt/scandir/blob/master/appveyor.yml)
and
[https://ci.appveyor.com/project/benhoyt/scandir](https://ci.appveyor.com/project/benhoyt/scandir)

------
markogrady
I have started using visual studio team services, it is really good and it's
free. [https://www.visualstudio.com/team-
services/](https://www.visualstudio.com/team-services/)

I used to use team city, octopus and jenkins in previous jobs. Personally, it
is really easy to use and you don't have to use the microsoft stack. It
supports Python, Ruby, node etc... It feels to me like they waited to see all
the good bits people have done and then just implemented them.

I didn't want to manage a server so it is perfect for me.

------
HugoDaniel
GitHub and Travis are not self-hosted tools.

It would be nice to see a setup with something like Gitea[0] and/or Jenkins[1]
_edit_ and some other slack alternative like mattermost[2]

[0] [https://try.gitea.io/](https://try.gitea.io/)

[1] [https://jenkins.io/](https://jenkins.io/)

[2]
[https://about.mattermost.com/download/](https://about.mattermost.com/download/)

~~~
growtofill
The title says 'hosted', which is the opposite of 'self-hosted'.

------
masklinn
A bit that's missing: bors-ng [https://bors.tech](https://bors.tech) to only
merge PRs which passed testing.

------
IgorPartola
Does anyone have a solution for a situation where I have a Django backend and
a VueJS front end and want to run the Vue build step before deploying?
Currently doing this with Git hooks and committing the resultant build, but
that is bad for a number of reasons. This is running on Heroku. Would their
release phase allow for this without significantly slowing things down? Would
it deal with npm installs?

~~~
simonw
You might be able to solve this using a bin/post_compile script - most
standard Heroku buildpacks will run these as part of the build step, and they
can be used to run commands that generate files which will then be included in
the compiled app that gets deployed to Heroku.

Here's an example that might be useful: [https://github.com/nigma/heroku-
django-cookbook/blob/master/...](https://github.com/nigma/heroku-django-
cookbook/blob/master/bin/post_compile)

------
stickfigure
For Java/Postgres/React/Google App Engine, here's a skeleton project using
CircleCI:

[https://github.com/stickfigure/gstrap-
skeleton/](https://github.com/stickfigure/gstrap-skeleton/)

Deployments are _git checkout production; git merge master; git push_. I love
being able to use _git log_ to see what changes are live.

------
onekiloparsec
An alternative to travis-ci which also works well: circleci.com. It has a free
tier (whatever the open/close status).

------
luord
I use gitlab instead of GitHub and Travis and tend to use the free tiers of
GAE and Heroku indistinctly. But man, I wasn't even aware of something like
Sentry existing, I'll have to get me some of that.

I also need to think of the messaging system. I'll probably start using
mattermost instead of slack, though; because GitLab.

------
daneah
I still have my stuff on AWS instead of Heroku and use Rollbar instead of
Sentry (not for any specific reason), but overall I'm really happy to hear
that I'm tracking well with this setup. I find it smooth as butter most days.
Thanks so much for sharing!

------
stephengillie
Did Travis replace Jenkins?

Thanks for posting this. I'm looking to get any application and start
deploying through a CI/CD process, to learn how this process works. Any
application suggestions for a CI "Hello World"?

~~~
aaronbrager
I think you mean CD “Hello World”, not CI. You can make a “hello world” web
app (or even just an API endpoint) in your favorite language and set it up to
deploy on each commit.

CD means each commit that passes tests is deployed

CI means your code is regularly integrated (merged) vs having long QA / code
review cycles.

CI and CD work great together but you can have one without the other.

~~~
stephengillie
Yes, I want to better understand the process of committing code to the
repository of an application, have it run automated tests, report via a
communications tool the result, and automatically deploy if successful.
Understanding this efficient process is high in demand and thus highly
profitable.

I've created and worked on several projects that have involved APIs, reading
from Zendesk, and writing to Slack or HTML, among other places. I've supported
Jenkins and related tools multiple times in production. Now I want to fill
this gap in my knowledge.

------
analytically
You're missing Concourse

~~~
twic
Two hours after posting, and this is the only comment about Concourse?
jacques_chester must be on holiday.

Is it possible get get Concourse running entirely on free tier stuff? I don't
think there's a hosted Concourse service yet - airplane.ci is coming, but
still in beta - and i imagine you'd struggle to get a whole setup on a
t2.micro.

------
desireco42
I like Bitbucket better for my personal private repo's. Also would like to
mention few more tools I like, Netlify for static site hosting, SemaphoreCI as
alternative to Travis, I think both are good.

------
ainiriand
When they say that AI is not going to take over dev jobs, show them this
thread and tell them that the time cut down because of automation is going to
be spent in something more meaningful, like coding!

------
tarantor
I would prefer Jenkins instead of TravisCI. And bitbucket provides private
repos and I always prefer BitBucket or even Gitlab but no GitHub.

------
machbio
This Talks about Freehosted tools on Code that is Opensource - for Closed
source - this does not work.

------
renw0rp
any suggestions for self hosted stack?

~~~
brodock
Instead of Slack -> Mattermost or Rocket.Chat

Sentry also has self-hosted option available (and it's also opensource).

Instead of GitHub + Travis CI, you can use GitLab (it also has bundled CI
capability, based on Docker).

with GitLab, you can also have Continuos Deployment, wired up with the CI
solution. The best experience will be deploying to Kubernetes, as it will give
you even monitoring, automatically, but you can code custom deployment logic
as a step in the CI pipeline.

------
mholmes680
In my world, its not world-class if i don't get some SOX compliance around it.
Its a cool example though.

~~~
dasil003
So you equate "world-class" with "enterprise"? I disagree. Tools that scale
down can also be world-class, and in fact that ability to run cheaply is a big
part of what allows startups to get off the ground and bootstrap their way to
eventually challenge the big boys. Heck, even if they stay small, they can
still have world-class tech. Once you get big and purchasing decisions are
made far from the front lines I'd argue it gets _harder_ to really have the
best tech.

~~~
mholmes680
>>So you equate "world-class" with "enterprise"

Well as part of my current world, sure I do. I have to satisfy the needs of
not just dev, but operations, compliance, security, business, the people
overseas that i've never met, etc. What I'd really like to see are startups
that bootstrap this stuff to think that way prior to growing.

My original point was that this was cool for the dev crowd and perfect for the
blog use case, but "world"-class seems hyperbolic. But if you are working for
other people: people walking around who have strung together various
automation stacks are a dime a dozen -- I've literally seen 4 different takes
this month. Execution matters, and execution for your audience matters more.

>> Tools that scale down can also be world-class

We are of the same opinion. I love me some elegant free solutions, and I need
to because then I can judge what I'm buying when I need to buy something.

>>Once you get big and purchasing decisions are made far from the front lines
I'd argue it gets harder to really have the best tech.

I guess I see where you could be coming from, but it doesn't HAVE TO be that
way does it? Isn't the whole point of work to empower yourself to affect the
company decision-making. Failing at doing so is sort of, without offending
you, you not doing your job. At least that's how I look at it.

[edit: formatting]

