We searched and found cdist. We're very much enjoying it. The reasons he lists for switching against Puppet resonated with me very much.
Plus, it's written in Python3, which might be a turn off for some, but it's a pretty cool demonstration that python3 is making its way in the wild.
Wether this kind of feature should be implemented in the CM tool is another debate :)
> "In cdist we only use a posix shell on the target plus common UNIX tools (like find, rm, grep), as defined by POSIX."
Puppet evolved to solve a lot of different problems for a lot of different people, and I'm continually impressed by the ingenuity of the community, but having to solve all those problems at once means that it's hard to change behavior at all. My intuition is that a collection of more discrete components - one that uses Unix commands instead of ruby libraries, for example - will probably eventually replace Puppet (and Chef, and the others).
In the meantime, though, there's an impressive amount of sysadmin knowledge baked into the Puppet codebase - it can accomplish very diverse tasks on very diverse platforms - and there's no easy way to extract the results of its evolution into other software.
I hope not... There's a huge value in being able to operate on proper data types easily. I can get a hash and output a corresponding .ini-like file in a couple of lines in chef. This would be a nightmare to do in shell.
oh its great and all but it's pretty much like chrome and others: no testing.
this stuff is critical to the infrastructure, and yet again: no testing.
just git pull updates away! yeah, right then push configuration on this untested stuff over thousand of machines and watch it potentially break...
Oh that, and that it's YA configuration management tool, and of course, you don't switch those overnight "just to check out the newest kid on the block".
I'd much rather get puppet enhanced. (but yeah - some of the puppet issues are design issues which aren't going to change)
What I do like:
- push. ffs! pull uses more resources and means you rely a lot on the host to be working properly. with push you get errors back (can't connect, can't do X, etc)
- no ruby. woot. seriously, ruby's cool but its just not default.
- simple and straightforward. nuff said. kiss has been applied.
- since its simple its less likely to break in odd ways like puppet does, or die off because the master just doesnt handle the load. push model also help a lot here.
- easy to run off a laptop compared to puppet.
I'm initially focusing on Ubuntu Server LTS on AWS EC2. As a result, it's all Bash and very simple. I'm still working on the documentation, but would really appreciate any feedback or improvement suggestions you may have.
The only thing I don't like about that is that you're essentially writing code, which - since this is a sysadmin tool - means you're depending on sysadmins to write code. This is usually a bad idea. With restrictive, structured language you can still provide for all the features they've built into their types/manifests/gencode/etc without relying on someone being able to program.
I was maintaining one part of a configuration management tool at a previous gig. The idea was to have a simple key=value file which essentially set identical environment variables. There was support for a few magic words like "REQUIRED" and "OPTIONAL", some very rudimentary logic statements ("if KEY=VAL then KEY2=VAL2") and an optional executable program for each config file to do extra magic that might be needed before the real configuration management kicked in (Cfengine2 at the time). In this way one or two people could write some generic config management modules/scripts and the environment variables would tell them what to do, so anybody could build and deliver configuration with only a key=value file and no programming. Everybody could pick up the format or copy an example because it was dead simple. Best of all: almost no code to maintain.
You're not seriously suggesting to give the management of a complex deployment into the hands of a person who can't program?
With restrictive, structured language you can still provide for all the features they've built into their types/manifests/gencode/etc without relying on someone being able to program.
Sorry, it's delusional to think you could operate puppet without being able to program. In theory the puppet language is said to be "not quite turing complete" or "mostly declarative".
In reality there is no such difference. It's just a (poorly designed) programming language with domain-specific constraints. It does have all the concepts that give non-programmers headache: variables, classes, parametrizable classes, defines, inheritance, etc. And on top of that you better know your ruby, too, because you won't get away without writing custom types, resources and functions. So... your admin is not just using the language - he's also extending it!
No. I'm suggesting giving the maintenance of the "code" portion of a deployment system to a developer, and having that developer craft it in such a way that a trained monkey can write a couple lines of metadata and have it deploy pretty much whatever you need. Which is how it worked for us in the past (and worked quite well).
I'm sorry but I think you misunderstood me; I was never talking about Puppet.
puppet: Making the error messages a bit better and switching to ssh as a transport mechanism would make a lot of people happy.
The error messages are getting much better - we have a newly-formed UX group that is working hard on the command line. If you haven't watched Randall Hansen's UX talk from PuppetConf it's here - http://bit.ly/zg5D0l [video - Youtube] - he goes quite a bit into where we're looking to improve.
SSH as a transport mechanism - can you provide some reasoning as to why that's an issue for you? Network/firewall concerns, access controls, or something else?
Sounds dangerous.. There's a reason you would normally protect your ssh keys with a passphrase. Do you really want leaked file(s) to permit root login to others servers?
Think it should at least:
- Use authorized_keys to limit what can be done with the key
- use ssh_agent instead of keys to somewhat limit the exposure if/when the laptop with cdist is stolen/cloned/hacked.
Password-less login: if your ssh key uses a passphrase, combine with keychain or ssh-agent.
Yes, to some extend it can be mitigated with ssh-agent, but that's not what they do in their tutorial. And even with ssh-agent, can't someone steal all your keys from memory?.
"It's not as crazy as you imply: it's a configuration management system - it has complete control over your machines. Giving it root or not is irrelevant."
This is crazy in its own regard: Giving one piece of untested, unhardened code control over all your machines. At least the code-base is small..
Both rely on the confidentiality of a private key on the master (the server identity if it is a pull, client identity if it is a push), but to fool a pull you also have to subvert the dns or the routing. Using ssh certificates that expire and are tied to the master's address would be a good first step making push safer against key interception.
Technically you don't have to use root anywhere. You can push from a master box as an unpriviledged user and specify any user on the target machine to connect as, and ssh keys (along with an agent) will take care of the rest. The target machine doesn't need to be running your config management as root, you just need to set up a config management account with rights to the users/groups that your services are running. Your service may get compromised but root wouldn't - until they run a local priv escalation 0day.
I think you're confused. Since the tools are using either SSL or SSH these protocols (when properly implemented) prevent mitm or spoofing attacks on things like dns or routing protocols. Your key does not get intercepted, ever, because you're never sending a private key. But some machine, somewhere, has to have keys or you can't log in.
The only thing you have to worry about is someone getting both the keys AND access to a server that's running an agent that you've authed with. If you're super duper paranoid ssh-agent has an option to never cache a given key's authorization so it can ask you for the password for the key every time you use it. I'm pretty sure that would be super annoying, though, so better to just run one agent once for an entire batch push/deployment and kill it once your job is done.
That's a distraction though - gizzlon is right that it is extremely bad practice to recommend people keep unprotected keys to root accounts on a personal computer.
Using passphrases and ssh-agent is a good and widely accepted solution to this problem (along with gizzlon's advice of having someone peer-review the code, again very standard and obvious).
If the master server is compromised, all the other servers might* be as well, but at least you're not spreading root@ ssh keys around.
* Maybe it's possible to predefine functions on the client and only ever run those. In practice that probably won't work because those functions will be either not powerful enough, or easy to misuse.
Telling him to redesign it or use Readability doesn't change that.
I postponed using Puppet/Chef because it didn't seem right that they were Ruby and thus had their own dependency chain.
Seems to me automatic configuration should be as close to whatever is available on a bare install, meaning C/bash.
And what's with the aversion to dependencies? Installing Ruby is just an apt-get invocation away from a bare server.
> Puppet in contrast to other systems emphasised on "define what I want" versus "define what todo", which is a great approach and we've [cdist] shameless cloned this approach.
With cdist and puppet you describe resources by instanciating resource types like Package, CronJob, Service, File, etc. It's very concise and high-level. Types are composable and easy to reuse, so if you see a common pattern in your configuration, you can factor it into a type.
Mac OS 9 and prior was also super frustrating to work with for similar reasons.
Perhaps there were things that disqualified chef in their eyes?
I have used puppet in the past an am now a happy chef user. I find that node attributes, data bags and search queries (from the nodes themselves) are very compelling reasons to use chef.
I agree that installing either chef or puppet on even a somewhat recent distro can be difficult, and this might be one of the reasons that CM is not as widespread as it should (I maintain PRM packages for chef, and it is no so easy...). So I agree with one of the points made by the author. But I have to admit I pretty much discarded his other ideas after reading the article at first, even though I poked around in the documentation.
But then, I thought about the points made and I must say that a CM tool made of small, autonomous, well tested "parts" makes a lot of sense. If you think about it, a CM system's job is to configure UNIX servers (mostly), so why not implement it the UNIX way? It is a bit like the Hubot implementation by Ted Dziuba: back to basics!
You CM tools, wether puppet of chef, must: deliver files to nodes and then execute code to ensure that they are in a proper state (it's a bit more complex than that, but not that much). I'm not sure if it's better to use push of pull for file delivery, but this should be an independent task, that could be implemented either way. Using SSH for security and access control seems like a no brainer. Then comes the compliancy part, where code is executed to ensure that the node is configured the way you want it. Why not have a bunch a small programs, which could be implemented in different programming languages, do it?
Let me give an example. Both puppet and chef work, at the lowest level, with resources. You define them in containers (manifests or recipes) and then make sure they are defined on the nodes. You could have "file", "package, "user" or "template" resources (for example), like:
# this is pseudo chef code for a fictional redis recipe
package "redis" do version "1.0" end
user "redis" do uid 230 end
template "/etc/redis.conf" do
user "root"; group "root"
notifies :restart, "service[redis]"
# in a shell script, distributed by the CM tool
package redis --version 1.0
user redis --uid 230
template /etc/redis.conf --user root --group root --mode 0644 --restart redis
I am convinced you still need attributes, bits of data attached to roles and nodes, for your CM to be efficient, but I don't see any reason this could not be implemented in cdist.
Anyway, this comment is getting really long and I am not sure I made my point. I guess I'm more thinking out loud rather than commenting on the article :)
Do yourself a favor, read the article, put aside the implementation details and website design, and think about the underlying points.