

Faster Web Development With Virtual Machines at deviantART - kemayo
http://dt.deviantart.com/blog/35461946/

======
mitchellh
Disclaimer: I am the creator of what I'm about to plug here.

If anyone is looking for an easy, low-risk, documented method of beginning
development in VMs, check out Vagrant[1], which is what I wrote this for.
Creating dev VMs from scratch is pretty tedious and there are a lot of steps
involved. Vagrant abstracts this away and provides a tool to automate all of
this in a reproducible manner.

[1]: <http://vagrantup.com>

~~~
denimboy
I started using vagrant last week to debug my Chef scripts and it is indeed
awesome. Vagrant uses Chef (chef-solo) configure your VM from bare image to
working server.

I see that is has a plug in system for other provision systems like Puppet,
but the chef support works great out of the box.

I was hoping to contribute to the project by integrating fog the ruby library
which talks to many cloud systems including EC2 so someone could build an
image via Chef or Puppet, dev and test in their image, then provision and
upload to the cloud. Could probably do this without integrating with vagrant
by building a clever rake or capistrano script.

------
sammcd
After talking with a few small PHP teams this seems like the logical
conclusion whenever growing a team of PHP developers. Its just very hard to
perfectly recreate your server environment locally on multiple OS's.

Its a great solution, but I much prefer working with RoR or Python/Django
where recreating the server environment locally is much easier.

~~~
leftnode
How is creating the local server environment on RoR or Python/Django easier
than PHP/anything?

I've been developing locally for years and while I can't mimic the exact
hardware of the live server, I wouldn't be able to do that regardless of stack
I chose.

~~~
Pewpewarrows
For the Python world, virtualenvs and pip requirement files make deploying a
local copy of a project take about 30 seconds, and it's completely independent
of your overall system and all other projects you happen to be working on.

~~~
leftnode
Gotcha. With git and a Phing build script, I can do the same in PHP :)

------
notmyname
I've found that developing against a VM is a great tool for a dev. For
OpenStack swift, we develop all of our code against a VM. Since we deploy on
debian and I develop on a Mac, I can use OS X tools to develop (TextMate, etc)
and code against an environment similar to our production. After code works on
a VM, we can move to a lab or a staging environment to do more scale testing
before deploying to production.

Developing against a VM is not something I did before working at Rackspace,
and I find the practice so useful that I will do what I can to continue doing
it no matter where I work in the future.

------
hdeshev
This all VM thing is fine and dandy if you want to have an accessible mirror
of your staging machine and reduce unnecessary stages. Although that in itself
looks fishy - if developers want to stage something just to see if it works,
then something is seriously wrong with your process or developers.

I'm amazed at the level of sophistication in this setup - using virtual
machines and TCP proxies that do text replaces seems too much. They could have
just built the configurability in the damn application. I have a .NET app that
normally gets deployed on three servers and uses various remote resources like
Amazon SQS queues, but I easily built a configuration abstraction similar to
Rails' environments that lets me deploy on a single machine and even test
locally with a debugger attached.

My point is that this problem looks as if it got solved at the wrong level of
abstraction. Which, obviously, made it unnecessarily hard and complex. If I
had to tackle it, I'd make sure developers do the right thing and try to make
the app configurable enough first.

~~~
tomedme
The mind boggles. There's probably a rockstar in there looking to put
something on their CV/resume. Whatever the case, it smells wrong.

------
evilhackerdude
I only skimmed the article but I assume it’s about the benefits of running VMs
to have solid dev environments with zero setup cost.

Let me ask another question: Will it become feasible to move code, tools and
everything else into web services? Imagine you have your favorite editor
(emacs, TM, vim) running in a browser window. Only that it’s live-updating
like, e.g., Ethercodes or Wave. And you have very straight-forward access to
SCM features like staging hunks of code. Also the code deploys straight to
heroku or whatever. It could, of course, also go through closure or sproutcore
in case of a fancy js client app.

Do you think it’ll all evaporate (I refuse to say cloud :) eventually?

The thing is I’m really going through a lot of pain setting up my system and
other’s systems over and over again. Yes, there are ways to automate that but
the effort is too high for most or we would all be doing it right now.

~~~
die_sekte
I'm sure it will happen one day, but it's quite a big challenge and you're up
against all those bad habits build over the last 20 years. For example,
developers really like customization. It's quite hard to provide a hosted
environment that is fully customizable.

Also, parenthesis mismatch in paragraph 3.

------
kristianp
It's not clear to me why they don't just run development on their local
machines. I'm not familiar with PHP, though. I'm all for having a build/test
vm, though.

~~~
Legion
Developers aren't server administrators (usually). It's not the best use of
their time.

I wish I had thought of this when we ran into issues with Windows people and
Ruby. Too much of their time was spent fussing with stuff and crying about it.
:)

~~~
evilhackerdude
“So now it should work, right?”

-“I’m getting an error”

“Which error?”

-“Unnamed method in [...]”

Then, a mountain of pain and torture later.

"But now it works, ok?”

-“Ok.”

\---

Ever had to fix someone else’s system via Skype or phone? I personally believe
writing a rails clone in brainfuck is more fun.

------
kemayo
I do acknowledge that this is something that looks very "oh, how obvious!" to
the modern crowd, but deviantART is a 10 year old PHP app...

Of course, even if you're running a modern web framework in
Ruby/Python/somethingtrendy, a VM makes a lot of sense. It lets you run the
_exact_ versions of everything that your production environment is, without
having to worry about it.

~~~
Supermighty
Developing in the exact same environment as you deploy in should be standard
practice.

------
chewbranca
This reminds me of facebook development, where you need to have a live fqdn
for all of your environments. We used to use a staging server and had a
similar problem with huge amounts of small unnatural commits for things you
would normally test locally. I ended up moving to using ssh tunnels to a
server, where each dev has a subdomain redirecting to their dev box through
the tunnel. This has the added benefit that you can work locally from anywhere
you have wifi. Whether you use tunnels or VMs, both are a solid way to do
development.

------
Jach
If commits were an issue before this, they should really have considered using
git or something. You shouldn't aim for fewer commits, but less expensive
commits, i.e. nothing goes over a network. Plus you can squash later if you
really need to.

Anyway, VMs are cool but has anyone tried developing with NixOS? It seems even
more suited for the tasks they want since it has rollbacks and so on:
<http://nixos.org/>

~~~
yycom
Isn't the expense of frequent commits the cost of composing an intelligible
commit message?

------
The_Igor
Great Article. As QA Analyst and Engineer, I know first hand about the pains
of replicating production environment. Thanks for submission.

------
mgrouchy
I do most of my work in python/django so virtualenv essentially solves much of
these problems.

The problem I am running into now is that our production environment is
starting to get complicated enough(mysql, redis, 2 different django projects)
that testing all the components working together gets more and more difficult
all the time.

~~~
fleitz
Start scripting your environment, the more repeatable your deploy procedure is
the better your deploys will be. The easier your deploys are the more
frequently you do them, the more frequently you do them, the better your
deploys will be. Rinse wash repeat.

Once you have your env scripted with EC2 you can setup / teardown environments
easily for testing.

------
prodigal_erik
Having a VM you can trivially reimage is very helpful because you can test
that (re)deploying packages to it is going to work, you didn't miss any
dependencies that happen to be present on your desktop but not on your
production servers, and uninstall won't leave cruft behind.

------
drwxrwxrwt
Another benefit of this approach is that it takes seconds to revert to a good
state if you destroyed your dataset in a mishap. use snapshots or just restart
VM from original image. all while using your preferred code editor (the app
source is mounted from the host OS)

------
tsmith
I wonder how long this took to set up? And why they don't run their production
servers as VMs as well?

Edit: I think it's an awesome idea, btw, and something we try to push to web
service shops at GridCentric.

------
dnaquin
Only use VMs or sandboxes on production-like machines.

Local development doesn't scale.

