
GIT is the new FTP - bitsweet
http://coderwall.com/p/xczkaq
======
rickmb
Every time someone comes up it with a "cool" way to deploy via Git, it turns
out their definition of "deployment" is nothing more than dumping something on
a server. It completely ignores every aspect of deployment other than putting
the files somewhere else, which is 5% of the work and 1% of the complexity of
deployment.

That's not even a sufficient professional scenario for anything but the
simplest of non-essential website. Hell, this won't even do for the simplest
of WordPress sites.

BTW, who on earth still uses FTP?

~~~
StacyC
_BTW, who on earth still uses FTP?_

I use Transmit to FTP updated files to my web server. Then I send smoke
signals to my colleagues to inform them of the changes, and document the
updates on a papyrus using quill and ink. Should I be doing it differently?

~~~
glesica
I actually assumed "smoke signals" was some sort of Mac app that I didn't know
about (it actually sounds like it could be a 37 Signals product). I even
wondered if "papyrus" was a team-management wiki or the like. Then I got to
"quill and ink" and realized you were joking.

~~~
numbsafari
He wasn't joking. "Quill and ink" was a Microsoft prototype tablet.

------
loup-vaillant
Err, what about rsync? My blog is versioned with git, but compiled with USSM¹.
I don't want to version the generated html code, so I deploy it with a one
line shell script that just calls rsync through ssh. Simple, and fast.

So in my case, rsync looks better than git at deployment. I can't imagine a
scenario where the reverse is true. Does someone have a counterexample?

[1] My home-made Jekyll clone: <http://loup-vaillant.fr/projects/ussm/>

~~~
ZoFreX
An alternative, if you do want to use VCS for deployment, is rather than
versioning the compiled HTML, create a hook that compiles it.

My website is stored in mercurial currently, and when I push to the server it
runs bundle and compiles the site.

------
languagehacker
This tutorial is missing a lot of steps. It therefore expects the user to know
how to use Git in order to consistently commit and push their changes under
such a scheme. If the reader is a self-sufficient Git user (i.e. doesn't need
a full step-by-step walkthrough), they wouldn't need this tutorial.

Anyhow, this scheme sort of old hat for Git users. Most of us require a build
to deploy code, which generally results in SCM-polling continuous integration
solutions like Jenkins to fully automate the preparations required to run new
code.

This blog post sort of suggests a WordPress crowd's version of continuous
integration. Go ahead and use it for a blog theme, or a set of assets. If
you're doing this for a web application, you're probably doing something
wrong.

~~~
webreac
Hello, I am learning git. Even if this tutorial does not comply to the rules
of the art or omit basic git knowledge, it fits exactly my actual learning
level. I have read the most basic tutorials and this kind of "almost second"
level tutorial is perfect to have a better understanding of how pieces fit
together.

Thank you Kerrick Long

------
drewmclellan
The very first sentence is why Git is _not_ the new FTP:

> First, create a directory on your server and initialize an empty git
> repository.

The number of people who deploy sites via FTP who also have SSH access and the
ability to install Git on their hosting is small. Most people deploying to the
web do not control their own servers.

~~~
Sunlis
I almost got kicked off of my old shared host for installing Java on my
machine after I faxed them a copy of my driver's license to get SSH access.
Got myself a shiny new VPS a few months ago and haven't looked back. I can
finally do things I could only dream about with the shared host.

------
Xion
Unfortunately, this doesn't take into account the fact that stuff you keep in
your development repository is almost never identical to stuff you want on
your deployment server. There are files you keep in the former but not
necessarily want/need in the latter (unit tests, documentation, etc.).
Similarly, there are stuff to be put on deployment server which don't want to
clutter your dev repository, like compiled CSS/JS assets.

As other have said, deployment should really be done by continuous integration
server, and preferably using a protocol that's designed for transmitting
files, not version control.

------
Kerrick
Wow, my coderwall post got posted to HN? That's pretty cool! Interesting title
choice, though... FTP will still be useful for a long time.

I didn't really write this as a tutorial as much as an easy way for me to look
back and find the commands when I have a forgetful moment. That's why it
doesn't go into explaining how or why things work, explaining how to set up
git, explaining how to set up your web server to serve files from a directory,
or other preliminary information.

~~~
loeschg
That's interesting, haha. Would you or somebody be willing to give
explanations for the instructions?

------
nikic
Just a bit feedback on the blog: The background image was very distracting for
me. My eyes were always wandering off from the text. Not sure if that's just
me or if others had that problem too.

~~~
alan_cx
Me too.

Had similar problem with a site called TVCatchup. They stream free TV in the
UK. Their player sits in the middle of the page and that is surrounded by an
advertising border. Nothing else on the page of consequence. What I found was
that I found it very hard to focus on the player in the middle. Often I would
get a headache watching. Told the admins, who didn't want to know. Basically I
got an angry "Foxtrot Oscar". Ended up using an ad blocker until they found a
way to defeat that. Now I don't use the site any more.

~~~
JonWood
Completely off topic, but TVCatchup provide an XBMC plugin which completely
avoids the advertising they add. No idea why, since it presumably destroys
their business model, but it's great.

------
bnegreve
I would rather say ssh (scp) is the new ftp.

~~~
polyfractal
I just found out recently that SCP is horribly slow when transferring lots of
small files. I found a blog post showing how to send compressed tarballs over
SSH and it worked really well:

ssh username@from_server "tar czf - directory_to_get" | tar xzvf - -C
path_to_save

~~~
zemo
...or use rsync?

    
    
       rsync -avz somehost:/path/to/src /path/to/dest

~~~
jackalope
For a more accurate sync, you can also delete obsolete files on the
destination:

    
    
        rsync -avz --delete src dest
    

This assumes there are no files you want to keep on the destination, of
course. You can exclude them or reorganize to keep them safe.

------
jackalope
This is clever, but the idea of serving a website directly from a repository
makes me very uncomfortable. If you already have git and ssh access on a
server, then you probably have rsync access, as well. I simply include a
Makefile in my repo with a publish target that runs a command like this:

    
    
        rsync -avz --exclude '*.swp' --delete site/ $(WEBUSER)@$(WEBHOST):site/
    

This way I'm only putting the necessary files on the server and transferring
changes is incredibly fast and secure (ssh is used by default).

In rare cases where I need to deploy from a repo on the server, I'll clone it
to the remote host and use rsync locally to copy the files. But I still avoid
serving directly out of the repo. There's too much sensitive information in
version control to even risk exposing it during a misconfiguration.

~~~
cgdangelo
I'm curious what kind of sensitive information you're concerned about
exposing. I don't mind deploying with rsync--in fact, I'll do it quite a lot
when the remote host lacks git--but I'd usually prefer to use git where
possible.

I typically deploy with a similar setup that you described, but instead of
rsync'ing the files locally I just make a clone of the repo and serve that.
Though it is another step to fetch/merge, it stops me from losing any changes
that someone did to production without telling me.

I only recently used Capistrano to deploy a project and it was very
satisfying, so I'm gravitating towards that as my default deployment method.

~~~
donavanm
> I'm curious what kind of sensitive information you're concerned about
> exposing

+1. Secrets should not be in your VCS repo. I'd guess the parent is talking
about user/pass or pub/priv key creds.

I really wish "we" had better tools for passing around config, and secrets in
particular. Chefs data bags are close, but I still don't want the master
knowing my secrets.

------
jameswyse
In my node.js application I use the excellent Up[1] library along with the Up-
Hook[2] plugin.

This allows me to update my code locally, push to github and have my server
automatically download the changes (using girror[3]) and update the code, all
with zero downtime.

What I like most about it is that it's all nicely contained inside the same
code as my application!

[1] <https://github.com/LearnBoost/up> [2] <https://github.com/LearnBoost/up-
hook> [3] <https://github.com/node-migrator-bot/node-girror>

~~~
obtu
I like this girror thing, it looks like it will fit my non-node projects much
better than Capistrano's awkward git integration. (For some reason you linked
to a fork, but it's no longer current)

~~~
jameswyse
woops! the correct link is: <https://github.com/eladb/node-girror>

------
Tichy
Wouldn't this approach also put the .git directory below ~/www, making it
accessible to the world? That might not be such a good idea.

~~~
Xymak1y
Hidden directories are usully not accessible from outside I think.

~~~
gurkendoktor
I had to change my Apache config on OS X 10.6 _exactly_ because I was leaking
.svn directories. Unbelievable.

------
lsiebert
When git software is easy for ordinary people to use and install on windows,
easy enough that people are using it to share porn and pirated material, then
maybe it will be like ftp.

As it is to run git on windows you, best case, have to install a lightweight
nix OS. Cygwin mingw,whatever. I think we techies and power users don't
realize how messed up that is if you want git to be more than a program for
programmers. Ftp was a protocol, you could make a native client for
everything. Git is a great tool, but not yet comparable to ftp.

------
nviennot
Once, I had to maintain a website on godaddy, where I only had ftp access. I
hated it. We would have interns working on it, and giving them the ftp
password sucked. Having git as a proxy, hooked to sync master on ~/www and
staging on ~/www/staging was awesome. If you want it:
<https://github.com/ezyang/git-ftp>

------
olliesaunders
If you do this make sure you set the permissions such that it is not possible
to make changes to the files on the server independently of the remote
otherwise next time you push you’ll end up with conflicts and, if you’re using
an interpreted language, the version control markup that indicates where the
conflicts are in files will break your code and bring down your website.

~~~
VBprogrammer
Best way to do this is to init the remote repository using:

    
    
      git init --bare

~~~
uxp
Bare repositories cannot have a working tree.

    
    
        fatal: core.bare and core.worktree do not make sense

------
firemanx
We use TeamCity and deploy right from our build server. We have a build
configuration called "Deploy" that requires a couple parameters - the artifact
to deploy, the environment to deploy to, and the build number (so we can
deploy past builds, ie roll back). Our application software is built of a
series of distributed .war and .jar files, but each represent a complete
"service" within our system, and can be upgraded independently. This means we
can deploy single artifacts and not cause conflicts within the overall system.

When we click the build button, a small Python script is called that rsyncs
over ssh the already built artifact (pulled in from the existing TC build)
over to the environment, which runs JBoss, and gets auto-deployed.

For our developers, our build process amounts to committing their code,
getting an email from TeamCity when the build completes, then clicking the
Deploy button. The tools take care of the rest, and we end up with a nice
trail of logs available in case anything goes wrong, and a clear rollback
strategy.

We also don't get an ever-growing git repository, or have to maintain a bunch
of FTP configuration. Everything is done over ssh via passwordless pki. Best
of all, it didn't take long at all to set up (the longest part was probably
writing the python deployment script, which has some basic logic to know which
environments live at which addresses, and also how to run some post-deploy
commands for the 1 non-JBoss deployed .jar based service we have).

It's not perfect - we don't have an automated database migration mechanism
right now, though in a distributed system like ours I'm not entirely sure one
would work at this point (and we've got other plans to deal with that weakness
anyway). I'd also like to move to Hudson or Jenkins at some point, as there is
a real attraction to the rich ecosystem there (TeamCity is a commercial
product).

------
ioquatix
One problem with this technique is that over time you will end up with more
and more data on the production server, even stuff that has since been
deleted. Even for a small website over the course of a few years can end up
being GBs of wasted space which on production VPSs can be quite expensive.

~~~
Someone
Also, if your production server is breached, attackers can mine that data, for
example for that accidentally checked-in password, or to search for security
fixes that other sites might not have applied.

------
EnderMB
Using post-commit hooks to handle your deployment for you can have undesired
effects if your scripts aren't perfect.

I would recommend looking into continuous integration over this method. I've
been using TeamCity for just over a year and it is fantastic for a staging
environment.

------
snowwrestler
Reverting a push should be at least as fast and easy as the push itself. This
minimizes downtime if you introduce a new bug. (Which I know you, the reader,
would never do, but other people might, so this is for them.)

Using this workflow, if you break your site, reverting would require either a)
fixing the bug and then doing another push or b) looking up and typing in the
hash of the last known good commit. Neither of these are as fast and easy as
"git push".

Our solution is to deploy with tags. Every push to production gets its own
tag, and that is what we checkout into production. If the site breaks, you
just checkout the previous tag name and you're reverted. This way reverting is
exactly as easy and fast as pushing new code.

~~~
mnutt
You could even streamline it a bit, by having a 'previous' branch that you
automatically sync with the previous tag on deploy. So if something goes wrong
you can always just `git checkout previous`.

------
dinkumthinkum
No it's not. Let's revisit this in 5 years. We'll still be using FTP. Thanks
though. :)

------
mkopinsky
Best move I did for a past client was getting them to deploy using Assembla's
FTP tool. There is essentially a post-commit hook in their SVN repo that
automatically pushes to the test environment on every commit. Deployments to
prod use the same tool, but it is manually triggered rather than on every
commit. (Competing for best move, was getting them to have separate test and
prod environments.)

If you (or an unfortunate client) are using a cheap shared host that only
provides you with FTP, this approach at least guarantees that the only code on
the site is what exists in your VCS.

------
rocky1138
I almost never FTP stuff except when I have to get a single, unimportant file
somewhere or if we're sharing stuff between friends' houses.

Springloops has a useful deploy-by-ftp feature. When you commit a change using
their Subversion host it automatically pushes it out to the development or
production server.

It's easy for designers to use since they can update a jpg or two for a
client, commit a change, then hit refresh on the live site and it shows up.

I know it's not perfect, but for small web shops and non-technical people, it
works great.

------
danmofo
As someone who uses FTP (it's all I've ever known, at work I just check my
work into CVS and I have 0 idea how the build process works), what would be a
better way to deploy a website?

~~~
olefoo
It depends on the site, and the stack you are using to manage it, but ideally
you would have the following features.

1\. All assets are under version control.

2\. Committing code of any sort (HTML, CSS, Javascript, Ruby, Python, PHP,
...etc) triggers the unit tests and a failure of the unit tests prevents the
commit from completing. #unit tests should take n < 3 seconds

3\. Pushing the code to the deploy repo (or branch) triggers the acceptance
tests (behavior driven tests that look at the functionality of the site
overall, account creation, login, profile_view, checkout) failure of the
acceptance tests sends an email to the whole team. Code changes affecting
security sensitive modules should be considered failing until they have been
manually qualified (e.g. anything touching passwords or money).

4\. If the acceptance tests pass and no critical modules were affected by the
commit, that commit gets deployed into the production environment. That fact
is recorded on the dashboard and emailed to the team.

This is difficult to achieve but once reached, it's like being over the step
on a boat that hydroplanes; friction is reduced and you get greater velocity
for the power expended.

------
Osiris
I built a system to deploy to our test servers that uses a git repository on
the test server, clones the repo into a named folder, and checks out a
specific branch. QA is able to use a webpage to select a branch, which is
automatically checked out in its own folder and a subdomain or cookie is used
to do isolated testing of that branch.

So for us, git is an important part of our deployment process, though there's
more that goes on especially for production deployments.

------
sry_not4sale
The first thing that came to mind after reading this post's title was
insecurity (e.g. .git directories accessible in the wwwroot)

Isn't that what FTP is most known for - being insecure?

~~~
samuellb
I agree the blog post should warn about it - but it can be perfectly secure if
you do it correctly. I usually put all public files inside a directory inside
the repo, e.g. public_html, and/or block access to dot-files altogether in the
server configuration. E.g. in Apache 2:

    
    
      <Files ~ "^\.">
          Order allow,deny
          Deny from all
      </Files>

------
telent
I'd question the assumption that ftp was ever the old ftp, if by "ftp" we mean
"deployment channel to production services". As soon as your web site is
complicated enough to need a database, you have database migrations or their
moral equivalent to worry about as part of the upgrade process, an simply
dumping files in a directory somewhere doesn't cut it

------
emmelaich
I've been looking into git (or dvcs generally) for deployment and have
realised that deployment is like the inverse (dare I say dual :) of normal
version control.

vc: metadata (permissions) are irrelevant. dep: metadata is critical

vc: version everything by default (explicitly exclude) dep: version by
explicit inclusion

I've have a doc with numerous links and notes that I should share sometime.

------
j_col
I thought SCP was the new FTP. Or rsync.

~~~
veyron
git is closer to rsync than to ftp. scp and ftp are file transfer protocols,
while git and rsync are higher-level systems that can use other underlying
file transfer protocols

------
iambibhas
I feel, Your exact method might be useful for Test servers. I won't want all
my git pushes reflect on the live server. For deployment, I think it's better
if one maintains the post-commit-hook to a certain branch, named, say,
"Deploy". Any push to Deploy branch would make it deploy maybe?

~~~
tikhonj
That sounds exactly like a system I set up with Jenkins a while back. It would
deploy to a test server and run the tests from any branch. If you pushed to
the deploy branch, it would deploy to the real server assuming all the tests
passed.

The actual deploying and testing was all done with Fabric, but the idea was
the same.

I think that's a pretty good case for using a CI system like Jenkins,
especially because it's _really_ easy to set up. It didn't support Git by
default, but there was a plugin that was both easy to install and fairly
comprehensive.

------
reidrac
I think posts like this tells you only part of the story. Ops it is not just
pushing code live.

I'd be great they explain how to know which code is live (ie. tag "releases"),
how to revert to a previous version (ie. something went wrong), potentially
something needs to be restarted, etc.

------
earnubs
FWIW if you use Bash shell you can also use history expansion to shorten the
mkdir && cd line further (the line already includes tilde expansion):

    
    
      mkdir ~/www/example.com && cd !$
    

!$ is the last argument of the preceding command

~~~
obtu
You mean "$_". !$ refers to the previous command in the history.

~~~
seclorum
!! refers to the previous command in the history. !$ refers to the last cmd-
line option in the history.

~~~
obtu
I know that, the bit I want to emphasize is that it won't be the mkdir
argument that is on the same line.

------
trebor
I could do automated deployments via FTP (I wrote scripts to upload certain
parts of a project during the build process) with a cron job on the server.
Just check for changes and do the real "deployment" stuff on the server
itself.

------
pitzips
Here is the same principle, but better documented:
[http://www.saintsjd.com/2011/03/automated-deployment-of-
word...](http://www.saintsjd.com/2011/03/automated-deployment-of-wordpress-
using-git/)

------
debacle
There was a post yesterday about deploying small sites and I mentioned that
svn export was the way to go, not thinking that there really is no simple git
equivalent.

You can do a one-liner with git archive but I don't think it's as easy.

------
ww520
While people are not too hot on the idea of pushing the git codeline to the
production server, the same idea is actually pretty good for kicking off a
build or running test suits after a git push.

------
rubendevries
afaik serving websites from your home dir is a bit odd, since the user running
nginx/apache shouldn't have access to it, something like /var/www is way more
common.

and asuming you don't just use git for deployment but also for it's core
purpose version control you should setup the origin to the actual origin and
name the remote where you push to accordingly (production, dev, test, staging)
and you shouldn't set it as the default push, to avoid excedently deploying
when you just wanted to checkin the code to the origin repo!

------
ivanhoe
text looks very much like a copycat of this tutorial, minus all the
explanations: <http://toroid.org/ams/git-website-howto>

~~~
obtu
No, the toroid one is much better because it changes the worktree to export
the data. That way the git directory won't be exposed to the web server.

------
fcurella
I've been following this <http://toroid.org/ams/git-website-howto> for several
years now.

------
lixef
saying that git is the new ftp is to my mind not correct. I read that git
doesnt handle big files very well the coder from the backup tool "bup" has
written. That was why he invented bup.

Correct me if I am wrong, I would really love to replace ftp with git.

~~~
reidrac
You're right. git may require lots of RAM for big changesets (involving big
files), and definitely more RAM than SFTP (oh, please never use FTP).

But the post is about deployments and not about transferring files.

------
veyron
As used, I think the more appropriate comparison is

    
    
        GIT is the new RSYNC

------
cduser
For non-technical users, dropbox (self hosted) is a good replacement for FTP,
IMO

~~~
danbee
How do you self host dropbox?

~~~
cduser
[http://www.readwriteweb.com/cloud/2011/05/4-ways-to-build-
yo...](http://www.readwriteweb.com/cloud/2011/05/4-ways-to-build-your-own-
dropbox.php)

[http://lifehacker.com/5821145/how-to-set-up-a-
file+syncing-d...](http://lifehacker.com/5821145/how-to-set-up-a-file+syncing-
dropbox-clone-you-control)

------
Tyr42
This is kinda what Heroku does, right?

------
pfanner
can anyone recommend a tutorial for diving into git?

~~~
dt7
The book on the Git website (Pro Git) is an easy read (at least to begin
with): <http://www.git-scm.com/documentation>

