The question always seems to be how much extra overhead you incur setting these things up in the first place for a relatively small business or side project. I've been curious about these kinds of automation tools for as long as they've been around, but they always look like more trouble than they're worth for any of the projects and deployment strategies I'm involved with at that smaller end of the scale.
Maybe they work better if you're doing a relatively standard DB + API + front-end kind of setup, which as it happens nothing I work on actually is. Or maybe they just don't really pay for themselves until you're working at a scale where a simple copy/clone step and everyday scripting tools aren't sufficient to run everything routine anyway?
I suppose my question would be why a lot of small systems would even need something like Docker. The theoretical benefits certainly would apply to various projects I work on, but the reality is that businesses at this scale often set up a standard system image and deploy it on a static set of real servers. Once that's done, the foundation might not change for months or years at a time.
There are usually two significant exceptions. One is security updates, which are typically monitored and tested/deployed via a separate process anyway. The rest is the day-to-day development work and deploying new assets, which typically needs no more than some sort of copy/clone job from the VCS to get the data to the servers and then running a single script to deploy/activate everything.
If you're working with hundreds of servers or dynamically reconfiguring things all the time in a Cloud environment, obviously the situation may be different. However, for the simpler cases -- and you can get a long way with them if you're just running a normal business and not trying to be the next unicorn -- the whole process already seems reasonably reliable and efficient anyway.
I'm a software guy rather than an operations specialist, as are the people I work with in almost all cases, so I always have some nagging doubt that we just have a total collective blind spot here. But so far, speaking only about the smaller scale and more static deployments I tend to work with, these kinds of tools seem like solutions to problems we don't often have.
I'm a software guy too (https://github.com/WriteCodeEveryday), but I have spent an entire 3 years of my life building prototypes and software for small businesses, that just aren't technically savvy and don't have large IT departments.
Once I deployed / configured my fifth Postgres database with Rails, I decided enough was enough, I needed to move faster in my Operations setup since I'm the "ops guy". If you're in a company that has dedicated capable ops guys, you don't need a platform, but if you don't have those guys, you will need one desperately.
Flynn's decent for my use cases because those very things you say you need done -> "deploying new assets", "copy/clone job from the VCS", and "running a single script to deploy/activate everything" are wicked simple and supported right out of the box securely, so instead of handing over the SSH keys to a developer, I can give them the Flynn key and they don't get access to the underlying infrastructure, just a basic API for doing development tasks.
Additionally, the backup functionality allows me to backup a single app or a full cluster and upgrade our infrastructure as it's running.
If you've never looked into Docker, or Flynn, take this post, set aside a $60 budget for DigitalOcean and try it out, I can't guarantee it will be perfect for you, but it's perfect for me. All hail /u/_lmars and /u/titanous.
I've looked into Docker, and various other devops-style tools, in recent years. I guess I just don't see what they usefully achieve in the context I'm talking about.
I mean, for all of the projects I'm thinking of here, we already have full version control of all of our assets. We can deploy an update in about upload time plus a few seconds with a single command, and roll it back with a single command as well. Security updates are typically deployed directly from their own upstream repos, again normally with a single command after any due diligence we feel is necessary. Installing the OS and other foundation packages was more substantial in some cases, but mostly because of the various non-standard things any given project might be using in addition to the routine OS + web + DB stack, and we'd surely have to set that up regardless of whether we were making some sort of master filesystem image to clone directly or some sort of image for use in a container system. And we typically run test infrastructure identical to production in both hardware and software, with deployment to that rather than a live system requiring just a couple of simple changes to configuration.
So given the above, how would introducing an extra layer of infrastructure and a bunch more tools really help us? I'd be happy to look into these things in more detail if someone can tell me what I should be looking for and what problem it solves that we don't even realise we have, but none of the (quite a few) resources I've read/watched about them over the past several years has suggested a compelling advantage for smaller, statically hosted infrastructure. Again, the advantages seem pretty clear for people managing, say, hundreds of systems of various types that are spun up on demand and discarded when no longer useful on some sort of Cloud platform, but eliq's original question and my follow-up relate to a different sort of environment.
Flynn's key selling point is easy setup and load-balancing.
You should try it out yourself, and see if it eases some of your pain points.
It also includes rollback of code (right from the UI, and every ENV variable change / worker change is a new release) and continuous deployment options (ensuring your code isn't broken when deploying a new version and deploying while ensuring your service never goes offline)
You should try it out yourself, and see if it eases some of your pain points.
Could you suggest any good places to start, please? I haven't looked at Flynn specifically before, but I did take a look at its website on your recommendation. Unfortunately, the videos and explanatory text I've found so far don't seem to be any better than others I've seen before in this area: plenty of buzzwords, but little real substance or context to help someone understand what the tool offers or how it works. Are there some better sources I could look into?
Thanks, but those are exactly the pages I was looking at before and didn't find very helpful.
Some examples of what got in the way for me, as someone totally new to Flynn:
I don't know what a "Flynn cluster" is, but it's obviously a central concept and mentioned almost immediately on both of those pages.
There are lots of impressive-looking commands doing impressive-looking things quickly in the videos, but there's not a single mention of where you're actually running them, where any other systems they are affecting came from, or what they're actually doing for that matter.
In fact, nothing on those pages tells me what Flynn does. I spent probably an hour or so exploring their site after we swapped posts the other day, but it was an exercise in frustration and I'm still none the wiser about what its scope or feature set actually is. (For comparison, that's longer than the total time it took me to write all the scripts we use to do the automation for one of those projects I mentioned before!)
Anyway, I don't want to take any more of your time on this, so I'll stop there. Thanks for trying to help, even if it didn't work out in the end.
Maybe they work better if you're doing a relatively standard DB + API + front-end kind of setup, which as it happens nothing I work on actually is. Or maybe they just don't really pay for themselves until you're working at a scale where a simple copy/clone step and everyday scripting tools aren't sufficient to run everything routine anyway?