
Habitat — A new approach to automation - Chico75
https://www.habitat.sh/
======
vishvananda
I've been waiting for this announcement for a few weeks. Since the home page
is quite lean on an actual explanation, let me give a rough summary of
interesting points as i understand it. I apologize in advance if I have
anything wrong.

1\. It is a from-scratch source build system written in rust that borrows a
huge amount from nix. This means repeatable software builds with isolated
dependencies.

2\. It attempts to move some configuration management primitives into the
build pipeline so that the deployment config has enough flexibility for real-
world use.

3\. It intends to replace single host process supervision with a distributed
model based on gossip.

This project is most definitely ambitious, and I'm not sure the whole goal is
realizable but some aspects are particularly interesting.

1\. The build process seems quite useful for building containers, especially
if it gets some traction. The main drawback of nix is that no one knows how to
use it. If this gets the backing of the hordes of chef-aware system
administrators, we may have a good solution for isolated repeatable builds,
especially if they focus on deterministic reproducible builds[1] as well.

2\. Distributed systems are notoriously hard to configure in a general way. It
seems like every company has their own magic combination of config management,
monitoring, and scripting for keeping systems like zookeeper running. Its
pretty clear that we don't have the right primitives for sharing this work. I
don't think a container management system has the right primiteves for this
either, because I don't think you can solve for all distributed systems in a
generic way. You really need custom logic for the particular software you are
trying to distribute. I'm not convinced that a gossip-based process-supervisor
is the best approach for this, but it could give us a place to start
collaborating on these primitives.

[1]
[https://wiki.debian.org/ReproducibleBuilds](https://wiki.debian.org/ReproducibleBuilds)

~~~
davexunit
>1\. It is a from-scratch source build system written in rust that borrows a
huge amount from nix. This means repeatable software builds with isolated
dependencies.

Except the builds aren't at all repeatable in a meaningful way, judging by the
example code on the home page. The 'do_build' function runs 'npm install'.
This means that 1) the build container has network access (i.e. it's
nondeterministic by definition) and 2) the declared dependencies do not fully
describe the dependency graph. So, I really don't see how much it could
possibly borrow from Nix if it threw out the most important part of it.

~~~
vishvananda
They used npm install in their example code? Well that was a poor choice.
Having dealt with npm and node apps in my own build system, I have not found a
good solution for deterministically building node apps. Does nix have a
solution for this? Thankfully the prepackaged software with reasonable source
build systems don't do craziness like this. See the redis package for example:

[https://app.habitat.sh/#/pkgs/adam/redis/3.0.7/2016061320525...](https://app.habitat.sh/#/pkgs/adam/redis/3.0.7/20160613205253)

~~~
cwp
The nix solution is npm2nix, which runs npm against a package.json file and
generates a nix expression for all the npm modules you need.

This mostly works, but occasionally you need to override the generated
derivations to add implicit dependencies that npm doesn't capture, or
otherwise tweak packages that rely on quirks of npm.

~~~
jjuhl
So basically it "does not work"...

~~~
jonaf
Well, let's be clear, if it doesn't work, you know at compile time, not at
runtime. I don't care how broken things are at compile time if I'm guaranteed
that everything will work at runtime. That's kind of the whole point of nix.
It's not impossible to write code with errors, but you should not be able to
build as long as the errors exist.

~~~
cwp
Also, it _does_ work. It's just not completely automatic.

------
otoburb
The article link points directly to the Habitat demo. For those of us that
don't already know what Habitat is, a better place to start might be the about
page: [https://www.habitat.sh/about/why-
habitat/](https://www.habitat.sh/about/why-habitat/)

Habitat seems like an entirely new Chef product to enable what they're calling
Application Automation.

Relevant quote: _" [...] automation must travel with the application, rather
than be provided by the platform. Everything the application needs, from build
dependencies, run-time dependencies, configuration, dynamic topologies,
deployment strategies, secrets management, security auditing, and so on
belongs with the application, because we're doing it for the application.
Runtime and infrastructure layers exist to support the application, but the
efficient building, deployment, and management of the application are
decoupled from those layers."_

Incredibly ambitious.

~~~
evgen
Containerization is disrupting a lot of their current market, so they need to
adapt in some fashion. Given the fact that the only hammer they have at hand
is ruby systems automation tooling it is not a big surprise they are aiming at
these targets.

Unfortunately, IMHO, the scope is a mile wide and an inch deep in terms of
what chef brings to the table here. I can walk down that list and either think
immediately of another tool that is dedicated to the specific task or the
needs are so slight that a few ansible or bash scripts take care of the need.
Sounds like they are aiming to occupy some space in the general "tooling &
orchestration" arena, but it will be an uphill climb.

~~~
2461001642
This is an excellent point - kind of goes against my earlier comment. Maybe
they have to move this way because that's their only choice - if they don't
move into automating this, containers will disrupt them (or already has)

------
Cshelton
Found this elsewhere, interesting to note, Habitat looks to be built mostly
with Rust

[https://github.com/habitat-sh/habitat](https://github.com/habitat-sh/habitat)

~~~
holoway
This is Adam - Rust has been amazing to work with, and was the prefect choice
for this platform. It's great.

~~~
bjz_
Could you elaborate a bit more on why you chose Rust, and why you think it was
the right choice for this project?

~~~
jonaf
On the heels of this, I've been exploring Rust a bit lately but haven't been
able to find any solid patterns / style guides for idiomatic Rust. Do any such
resources exist? For example, the official rust book describes Arc, Mutex, etc
but.. Are there more of these in the standard library? What purpose do they
serve? What are some good idioms to apply when using them? I can surely make
them work, but maybe I should be using some other thing?

~~~
bluejekyll
You might want to join users.rust-lang.org.

Like most languages idiom is learned while working with the language.
Something I've found for instance is to try and use Iterators to get around
sharing rules as they can encapsulate sharing patterns in a lot of cases.

The specific modules you mentioned are for multi-thread support, those are
only necessary for sharing memory between threads.

Anyway, I think this is off-topic for this thread.

------
hrayr
Home Automation is what first came to mind when I read the title. With this in
mind, reading the blurb on the website sounded like, oh they're talking about
building home automation software...

"build automation, IT automation, deployment automation, etc" would have saved
me some confusion. Or simply compare yourself to other solutions out there
will help me understand what category this new piece of utility and name fall
into.

~~~
reitanqild
Yep. Would made perfect sense for the name as well.

------
mwcampbell
I wonder if this project is doing too many things. It includes a package
builder _and_ a supervisor. Is there any benefit to integrating the two?

For a project that's smaller in scope but addresses the part about packaging
the automation with the application, Joyent's ContainerPilot [1] looks
interesting.

[1]:
[https://www.joyent.com/containerpilot](https://www.joyent.com/containerpilot)

------
johnbellone
The GitHub project: [https://github.com/habitat-
sh/habitat](https://github.com/habitat-sh/habitat)

------
_greim_
I'm not sure what I'm looking at. What is being "automated" here? The headline
and home page don't give much context.

~~~
otoburb
The main points around automation seem to be[1]:

* Flexible versioning and configuration rollout

* Ability to deploy anywhere

* Ease of management

Concretely, they're touting their gossip-based configuration and topology
management layer as a way to send configuration updates to service groups, and
then letting the service groups apply the changes to all the group members.
Contrast this with a more traditional way of propagating changes across a
network by enumerating every single host/VM/container. I'd imagine another
large component of the application automation would come in the form of
tooling allowing the ability to deploy anywhere (bare metal vs. VM vs.
container vs. IaaS/PaaS providers), but haven't seen that just yet. Maybe it's
being covered in the live announcement.

[1] [https://www.habitat.sh/about/why-package-automation-with-
app...](https://www.habitat.sh/about/why-package-automation-with-app/)

~~~
_greim_
So but am I automating devices in my home/building IoT-style? Or am I
automating deployments of software? Or hardware? "gossip-based configuration
and topology management" means nothing to me unless I know actually what the
overall context is here.

~~~
otoburb
From what I'm reading, I believe this is primarily geared towards trying to
automate software deployments.

------
rmcdermo
I watched the videos, did the tutorial, read the docs, read the tech site
stories but I still don't see why I would use Habitat over the current Docker
workflow and ecosystem (including third party) for building, shipping and
deploying applications. Both are a type of packaging that puts the
application, its dependencies and configuration in a bundle that can run
exactly the same everywhere. Docker has Dockerfiles and docker-compose.yml
files, Habitat has plan files (seems like a shell script, Dockerfiles and
compose files are much simpler). Docker has a central repository/registry
"docker hub", habitat has depots. "docker run redis" == "hab start
core/redis", etc... etc...

The docs/articles say habitat apps can run in containers, but why bother if
your app can already be run in a container that achieves the same thing with
even greater isolation (cgroups). Why put a self contained app in a self
contained container application?

I'm not trying to put down Habitat, I'm trying to understand its role. As I
currently see it, why would I choose this over the more mature Docker
environment with a massive established ecosystem and user base?

Again I'm just trying to understand what I'm missing, if I'm missing something
please point me in the right direction.

------
willejs
This looks really interesting. Im waiting for 8:30PDT for the full
announcement though!

Im curious about how it works under the hood, it seems to solve 'all the
problems' ;)

My skepticism comes from some of chefs past product releases, which have
seemed rather, half baked.

Products like chef metal (provisioning), pushy (push jobs) or delivery...

This is in no way negative as I am a long time chef user and fan of chef.

------
canterburry
Hmm, I very much like the concept but when I see the need for supervisors and
a habitat service, it makes me wonder if this is really what is says it is.

If I need to pre-install all these services on my hosts before any of this
will work, then how is that really different from uniformly deploying all my
applications on an application server such as WebSphere? An EAR too contains
all my configuration data, my bindings, my security settings etc. So...if you
uniformly run WebSphere everywhere in your enterprise, it sort of
accomplishers the same thing.

HOWEVER, if the habitat configuration is capable of talking to my F5 load
balancer to create a resource pool, self register my app with my corporate
DNS, create a DB schema and populate it with my tables, talk to my networking
equipment and open all relevant firewall ports and do so when the application
is deployed to a new environment, as well as undo all these actions if it's
ever moved elsewhere, then you have piqued my interest...

~~~
GrinningFool
There's no pre-installation - the supervisor is bundled with the application
deployment itself.

------
willejs
Theres a live announcement now
[https://bignews.chef.io/](https://bignews.chef.io/)

------
2461001642
I love the Chef ppl, but...

Is it just me or are there a ton of other folks doing this? The idea of
introducing yet another layer of stuff is scary, and there's lots of work left
to do on Chef. In a lot of ways, they feel just like Nomad (coincidentally
also on the front page)... I wish they would spend some time on their core
products versus expanding into an overfilled market.

~~~
rmoriz
That's what I thought, too. Chef (software/ecosystem) needs a bit more love
IMHO.

However it shows that Chef (Company) is still able to design, build and ship a
new product, as an organization. That's a very positive sign. Until recently I
thought when comparing Chef with Hashicorp: Hashicorp is so much faster in
building and releasing usable stuff but they seem to feel the pain in growing
too fast and too broad in terms of products.

So, in the end: It's a great thing, just also don't forget to put more love
into Chef (software) and don't dissipate developers/time/budget :)

------
onion2k
As nice as the 'console' on the page is, the fact the window gets cleared when
you click to go to the second step makes it look like it lost what you did on
the first page. To me that makes it look like it's broken - the steps should
maintain the state and build on each other, not start over each time.

------
the_arun
Is this like Docker?

~~~
holoway
Nope! You can use it with docker, and we think it makes the experience better.
We love docker.

~~~
SEJeff
Is there anywhere that actually lays out the tech that powers habitat? ie:

This thing uses kubernetes or this thing uses mesos This other thing uses
docker/rkt/containerd/lxd This is the set of things we wrote in rust, etc,
etc.

------
BHSPitMonkey
How does this compare with Heroku's buildpack system?

------
usaphp
That jumping header is quote annoying, I guess the developer did not calculate
a proper padding to accomodate for a sticky menu once it becomes fixed
positioned

------
sctb
We updated the link from the live demo page
([https://www.habitat.sh/try/](https://www.habitat.sh/try/)) to the homepage.

------
albasha
Ah, another limiting abstraction layer to learn and to fight.

Bash is plenty good.

~~~
davexunit
But all of the package recipes are just Bash scripts, which I think is just
terrible, btw. Using a real programming language would have been a much better
choice.

~~~
zeveb
> Using a real programming language would have been a much better choice.

The problem is that everyone disagrees on what a tolerable real programming
language is. Some refuse to use Python; others Ruby; others Scheme; others
Lisp.

At least every can tolerate shell more or less.

------
HerpDerpLerp
Can I also buy a sofa? [http://www.habitat.co.uk/](http://www.habitat.co.uk/)

