
Nomad Task Versioning - dradtke
https://damienradtke.com/post/nomad-task-versioning/
======
schmichael
Nomad dev here. Great post! Hopefully I can provide some insights:

> 1\. Templated job files.

While parameterized jobs[0] are a step toward templated jobs, they
unfortunately don't address your use case. We'll be adding `-var` options to
`run` in 0.7 to allow templating normal jobs like you can template jobs.

In the meantime I think what you're doing is pretty good and should integrate
well into a CI/CD pipeline for automating deploys of new builds.

You could consider a specialized templating tool instead of sed like m4, a
jinja-based tool like j2cli, or even consul-template.

A number of users also bypass `nomad run ...` and run jobs through our HTTP
API instead.[1] While this takes more code, it's easiest to build into
existing deployment tooling (CI/CD, chatops, chef, etc) and obviously allows
easy manipulation.

> 2\. Better documentation.

Absolutely. We're acutely aware of the need for more comprehensive docs.
Expect a "Guides" section to appear in the near future to cover common use
patterns like yours!

> 3\. Explicit support for artifact versioning?

Probably not going to happen as it would be little more than an opaque meta
var to Nomad.[2]

Although in the upcoming 0.5.5 release we do support Git and hg URLs for
artifacts, so you could pull down code from a tag.[3]

We do have other artifact features planned for the near future though (like
caching so common artifacts shared between jobs aren't downloaded multiple
times), so perhaps some sort of versioning will be added. Not sure what the
exact semantics will be yet.

[0] [https://www.hashicorp.com/blog/replacing-queues-with-
nomad-d...](https://www.hashicorp.com/blog/replacing-queues-with-nomad-
dispatch/)

[1] [https://www.nomadproject.io/docs/http/job.html#put-
post](https://www.nomadproject.io/docs/http/job.html#put-post)

[2] [https://www.nomadproject.io/docs/job-
specification/artifact....](https://www.nomadproject.io/docs/job-
specification/artifact.html)

[3] [https://github.com/hashicorp/go-getter#git-
git](https://github.com/hashicorp/go-getter#git-git)

------
bogomipz
I was excited to see a write up on Nomad. I wish it got more attention. I have
a question that maybe someone could answer. The author states:

>"Despite its relative immaturity, there are a few things I really like about
it"

Can anyone elaborate on what that immaturity consists of at the moment?

Doesn't Nomad largely leverage Consul for resource discovery and the remote
exec API, Consul being pretty mature.

~~~
schmichael
Nomad dev here. I'd love to hear unbiased answers to your question, but here
are my biased thoughts. :)

> Can anyone elaborate on what that immaturity consists of at the moment?

We're pre-1.0 for a reason. While you're right we leverage Consul for
discovery and health checks and Vault for secrets, there are still quite a few
gaps we plan to fill before hitting 1.0. We need a plugin infrastructure to
support volume and network plugins. There's still resource constraints to be
added (eg gpus) or improved upon (eg storage accounting). I'd personally love
to get secure mTLS bootstrapping via Vault to be as painless as our server
discovery via Consul. Lots to do, so expect regular improvements as we move
toward 1.0.

tl;dr - The immaturity and pre-1.0 version are due to missing features, not
instability or lots-of-breaking-changes.

~~~
bogomipz
Thanks for the reply, good to know.

Is there a current roadmap anywhere I can view? I looked at the Google Group
for Nomad but the last roadmap post I saw was 0.4.

~~~
schmichael
We have an internal one that we keep private mainly because it changes so
often it would be more frustrating than useful to users.

We should still do a better job of making issues for all major work items and
probably publishing some sort of high level list of desired features for 1.0.

~~~
bogomipz
I understand. Maybe you could post the high level list(that's actually what I
meant in my other question) to the Nomad Google group?

Do you have any ball park of how far off a 1.0 release might be? Cheers.

------
Nikkau
The proper way isn't to specify artifact checksum?

When you rebuild your "main", you update the checksum and Nomad should see it
as a change, no?

I can't test right now, I don't know if Nomad pick the job change and download
the new artifact or just fail the check.

~~~
schmichael
Yup, that's one way! Still requires getting that checksum into the job file
somehow (sed in this example). In the future there will be a -var option for
passing template variables in from the command line.

