
Ignition: A new CoreOS machine provisioning utility - polvi
https://coreos.com/blog/introducing-ignition.html
======
brandonbloom
Apologies for the somewhat off-topic remark, but ... Can we please stop
creating new JSON-based config files? Sane config file formats must support
comments.

~~~
pm
Any suggestions for a format that is sane?

~~~
mintplant
TOML.

[https://github.com/toml-lang/toml](https://github.com/toml-lang/toml)

~~~
nikolay
[[ something ]]? No, thank you! Jsonnet [0], yes, please!

[0]: [http://jsonnet.org/](http://jsonnet.org/)

~~~
placeybordeaux
Why?

~~~
nikolay
It's just weird syntax in my view.

------
darren0
The transition away from cloud-init and YAML is really quite odd to me. Nobody
enjoys editing JSON files. Forgetting commas, no multi-line strings, no
comments, escaping characters in strings, etc. Just reading the documentation
of ignition should be enough to illustrate the pain it is to manage multiline
content for unit files in an JSON string.

But why abandon the cloud-init format in general? Again, why would somebody
want to learn a new configuration syntax. Using CoreOS already requires you to
know and use systemd units (most other distros this really isn't required
knowledge), so that adds two steps to users learning/using CoreOS.

~~~
chrissnell
I'm totally with you. We're big users of CoreOS and while there are aspects of
cloud-init that I find janky, it's at least mostly easy to read and
understand. The use of JSON is nuts. Just looking at the one-shots
consolidated to a single line with \n's gives me a headache. This is moving
forwards?

What I would prefer is a compiler. You feed it a directory of unit file drop-
ins and app config templates and it builds a single artifact that can be
served over HTTP and pulled by the server booting up. This could allow for
dynamic configuration and automation but still makes it easy for the admin to
piece the config together.

~~~
robszumski
Check out bootcfg, a server that sends out customized cloud config or ignition
files based on mac address, IP, serial number, etc

[https://github.com/coreos/coreos-
baremetal#bootcfg](https://github.com/coreos/coreos-baremetal#bootcfg)

------
fpoling
JSON is not language that "makes it very easy to write tools to generate new
configs or manipulate existing ones". One cannot realistically consume JSON in
a shell script, and even with tools that have native JSON support manipulating
the config require the knowledge of document semantics as the language itself
does not define how to combine or merge 2 documents into one.

For example, consider an application that comes with a complex JSON (or YAML
as that language also share this shortcomings) that essentially describes
default settings. I cannot just define an extra file that specifies a couple
settings that I alter/delete/add, I have to have my own copy of the original
file with my changes resulting in painful merge when the original is updated.

What I have found is that things like .ini files or config files in style of
ssh_config work much better in practice. It is easy to generate and process
them in any language with a notion of text IO including shell scripts and the
merging functionality can be provided independent of semantics so I can keep
config fragment outside the main file/files.

------
HorizonXP
Alex this looks great and likely addresses a lot of the problems I had with
cloud-config. I was one of those with a Bash script that I used to generate
multiple files, so this is great.

My biggest issue so far is CoreOS' naming of Ethernet interfaces on VMWare
ESXi. It always uses some eno* name for each interface. I have a unique case
where each VM I spin up has up to 10 interfaces.

I've solved this by adding net.ifnames=0 to my grub.cfg. It requires that I
reboot the machine at least once to get it to take.

If I could have predictable interface names using Ignition, then I'm set!

~~~
jsmthrowaway
> My biggest issue so far is CoreOS' naming of Ethernet interfaces on VMWare
> ESXi. It always uses some eno* name for each interface. I have a unique case
> where each VM I spin up has up to 10 interfaces.

That's a systemd decision, not CoreOS, and also impacts the 7.x series of RHEL
derivatives and anything that uses systemd > v197, really. Your way is one of
three to revert it.

[https://www.freedesktop.org/wiki/Software/systemd/Predictabl...](https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/)

~~~
HorizonXP
Ah, fair enough. But it looks like that introduces another problem. When you
spin up a VM in ESXi, it generates a new MAC address for each interface. So it
would be great to create systemd .network files that bind to those MAC
addresses. Except you don't what MAC addresses you need to match against. The
only flow I see working is:

1) Create a VM with your interfaces, but don't boot it. 2) Take the MAC
addresses and enter them into your .network files, created via cloud-
config/Ignition 3) Mount the cloud-config/Ignition file to your VM and boot

Which is a bit painful to do manually when you have 10 VMs, with 5-10
interfaces each. I'd love to automate this, and if you happen to have a
suggestion here, I'd really appreciate it.

Edit: Oh, and when I said it had an eno* number, it's more like eno16777736,
which is not very predictable at all.

~~~
wmf
You don't have to match against MAC addresses, although there's virtually no
documentation about how to do it any other way. In the pre-systemd past I've
successfully created udev rules that match PCI slots so that the configuration
does not depend on MAC addresses. It looks like the same thing should be
possible (with trial and error) with path globbing in systemd.

------
dghubble
Some useful Ignition configs can be found in the
[https://github.com/coreos/coreos-baremetal](https://github.com/coreos/coreos-
baremetal) examples. For those wondering about the format, they're converted
to JSON and served to machines.

------
songgao
I'm curious about how it works. I must be missing something, but if it runs
before the network and file system are setup, how does it pull configure from
a [http://](http://) URL (say on bare metal) or oem:// which is on FS? Does it
run with a default setup for network/fs and let systemd redo everything later?

------
rdtsc
> Running before systemd helps ensure that all services established by
> Ignition are known to and can be managed by systemd when it subsequently
> starts. This allows systemd to do what it does best: concurrently start
> services as quickly as possible.

I think this looks like a slight jab at systemd ;-)

Systemd thinks it does a lot more other things best

~~~
wmf
I think that was poorly explained on their part. Digging deeper into the docs,
it looks like Ignition writes unit files that systemd runs; e.g. networkd is
still used to configure networking.

~~~
thebeardisred
Affirmative. This is just a mechanism to make sure that interfaces can be
properly plumbed, disks formatted with the desired states & filesystems, etc.

