How do you use Nomad? Do you run Docker containers with it? I don't need the scaling, I need the part where new versions of code are autodeployed automatically, services are connected together, and basically things update without having to run ops checklists every time.
Kubernetes feels too heavyweight and deploying machines feels too snowflakey...
We use nomad a few different ways. One is system jobs which run DNS and Fabio (the networking fabric layer) on the host machines. Then, yeah, for all the apps and services we use Docker containers which I've found runs pretty well...with one caveat: make sure if you're spinning jobs up/down a lot you get a fast disk. For instance, we were running on 64gb ebs gpio volumes and Docker was starting to grind to a halt on them (we spin workers for our queue up/down quite a lot). We started using instances with ephemeral SSDs instead which has a bit more operational complexity on init, but overall works really nicely.
We also use Nomad for deployments as well. I wrote some deploy scripts by hand that create the Docker containers and load the templated Nomad jobs ie
I'd say as far as "take this container and run N instances of it and load balance requests at this hostname to them" Nomad has been pretty great. There is definitely some work involved getting the comm fabric all set up exactly how you want (Fabio does make this easier but it's still work). Consul now has Connect (https://www.consul.io/docs/connect/index.html) which I haven't looked at yet which might alleviate a lot of this. I think some of our complexity also came fromt he fact that we do have TCP services we needed to load balance and most fo the service fabric stuff forces HTTP onto everything.
Overall my experience with Nomad has been great. It's capable and really not too difficult to operate for one person who also has tons of other stuff going as well =].
This is super useful, thank you! Do you think all of this is worth doing, now that you can basically get managed Kubernetes for free from providers as long as you use their machines?
It feels like this is much easier than rolling your own Kube, but not easier than using the managed version...
Not sure, honestly. I've never used Kube, just taken a preliminary look at the docs and been scared away by how much abtraction there is. While providers may manage it for us, I'm not sure to what extent they manage it. We're on AWS and I haven't been super happy with the responses/response times of their support, so when dealing with unknowns I'd rather not rely on someone else.
That said, Nomad hasn't been without problems. It's just that the problems seem to be easier for one person to solve. I set all this up almost a year and a half ago and haven't touched it much since, so it's possible both Nomad, Kube, and and managed services have come a long way and now is a good time to re-evaluate.
Kubernetes feels too heavyweight and deploying machines feels too snowflakey...