
Vault – A tool for managing secrets - eloycoto
https://hashicorp.com/blog/vault.html
======
jvehent
There are many solutions out there that attempt to solve the secrets
distribution problem. Vault is the newest one. None of them solve the
fundamental problem of the secret zero: provisioning the _first_ secret on an
untrusted system the automated way is hard, and without that secret zero,
there is no way to authenticate the new system.

With Vault, you still need to figure out how to push secret zero (here a
client authentication token). If you can solve that, why not push the secret
you need to push directly, and bypass Vault entirely?

~~~
mitchellh
We worked with a handful of companies to help us design the "App ID"
authentication backend that does exactly that:
[https://vaultproject.io/docs/auth/app-
id.html](https://vaultproject.io/docs/auth/app-id.html)

It allows you to have a non-sensitive single factor within things like
configuration management, and have the second factor come from a machine-local
location (instance ID, MAC address, TSM module, etc.). The idea is that
another team out of band sets the 2nd factor that developers and config
management never get to see. The result is you have full automation without
secret zero issues.

As an additional security parameter, you can bind the two factors together to
a single CIDR block, so you can have an additional factor restricting access
from that machine's IP.

The URL itself above goes into a lot more detail, but our beta users are
successfully automated in an elastic environment this way.

~~~
kgilpin
Using an instance id or MAC address as a secret 2nd factor seems surprising to
me. This is what banks are asking for? It's nice that there are two factors to
the authentication, but neither one seems to be securely random.

~~~
mitchellh
No, it is usually one of those values salted with a value that is only
available offline.

------
perlgeek
It sounds very exciting.

But, the first example made me flinch:

    
    
        $ vault write secret/foo value=bar
    

Don't ever pass passwords on the command line. On linux, other users can see
your command line (ps aux), and it usually ends up in a shell's history file.

~~~
mitchellh
Absolutely true! We support values coming from the command-line, stdin, and
files: [https://vaultproject.io/docs/commands/read-
write.html](https://vaultproject.io/docs/commands/read-write.html) So you
should use what you feel is most secure given the data you're entering.

~~~
sliverstorm
I think the argument is something like, why support a method that is insecure
nearly all the time and leads less-experienced users to make the wrong
choices.

Good security software gently herds the non-expert to make good choices. Too
many options, especially specialty risky options, don't help that goal.

~~~
zobzu
true most ppl will do it command line..

------
outworlder
> "Vault is the sixth such tool we've built. We've also built Vagrant, Packer,
> Serf, Consul, and Terraform. "

Sigh.

This is the first time that I actually feel overwhelmed. That nagging feeling
that I'm doing something in some way solely because of habits. And that
someone somewhere has created an awesome tool that would have allowed me to go
home two hours earlier.

The amount of new tools cropping in the last couple of years alone is the
'culprit'.

These are awesome times, of course. Still, a part of my brain misses the time
where you could ftp crap away to a server (even if automated by a Capistrano
tool or similar) and spend the rest of the time arguing about the merits of
puppet vs chef.

~~~
IgorPartola
Obligatory quote from that XKCD about automating things. Remember, most of
these tools simply automate what you already do. If you don't spend much time
performing certain things, you don't need the tool. The cargo cult of "I
should use Packer to create my Vagrant boxes, so I can have dev/prod parity"
only works to a point. Instead, look at where you spend lots of time, and then
look for tools that automate that task.

Personally, I try to be pretty conservative with tools that I start relying
on. I know by painful experience that some tools get abandoned, some change
drastically, making it impossible to upgrade, and some just turn out to be
very buggy and horrible once you start putting them through their paces. In
lots of cases, it's better to be critical than permissive when it comes to
your infrastructure (and that's what these tools are aimed at). In almost all
cases, your edge is in knowing the tools you use, and knowing _about_ other
tools. It's not in knowing all and using everything under the sun.

~~~
koffiezet
The main reason for automation is not saving time. It is about being able to
reproduce and test the thing you're trying to automate. It's also about
tracking changes, accountability on a larger scale.

I manage quite a few servers, all running different in-house developed
services. Almost all configurations are completely different, with a few
exceptions, but I am still automating them. Just leaving it like it is, all
manually managed would probably save me a lot of time, so why do I do it? So
it can first be deployed on some test VM's, and when someone changes a
configuration, I can track it in git/svn/.. and see who committed what change.
It would only save me time if I had to deploy service X more than 3 or 4 times
(dev/test/qa/prod), but as it stands, nop. Timesink it is.

~~~
IgorPartola
I would say that if you are not automating something like what you describe,
you would have to spend time manually verifying that it is working correctly,
so it is about time after all.

------
steveb
I'm pretty excited about this release. We just added deploying Vault on top of
consul in the microservice/realtime data platform we're building:

[https://github.com/CiscoCloud/microservices-
infrastructure](https://github.com/CiscoCloud/microservices-infrastructure)

We were already using consul as our service discovery engine and as a data
store for dynamic configurations via consul template. We feel Vault will help
us manage secrets on our infrastructure and inside containers.

~~~
jgrowl
Neat! I'll have to see how you ended up using Vault since your project uses a
lot of the same tech my personal webrtc server project uses. I'm also using
ansible, docker, consul, registrator.

Excuse the mess, I'm still working on it:
[https://github.com/jgrowl/livehq](https://github.com/jgrowl/livehq)

------
rmetzler
I know it is very new, but I would love to read a Vault vs Keywhiz [1]
article.

[1] [https://square.github.io/keywhiz/](https://square.github.io/keywhiz/)

~~~
hbbio
One is implemented in Java (Keywhiz), the other in Go?

~~~
cwp
Surely there's a little more to it than that.

------
dorfsmay
Got to mention blackbox, it's based on gpg, and meant for teams interacting
within version control systems:

[https://github.com/StackExchange/blackbox](https://github.com/StackExchange/blackbox)

~~~
metavida
I'm not terribly familiar with blackbox, but based on the blackbox Readme it
seems that this critique from Vault's website [1] addresses why HashiCorp
thinks Vault is a better solution.

> Chef, Puppet, etc. all solve this in a similar way: single-key encrypted
> storage. Chef has encrypted data bags, Puppet has encrypted Hiera, an so on.
> The encrypted data is always one secret (a password, a key, etc.) away from
> being decrypted, and this secret is generally not well protected since in an
> elastic environment, every server needs to somehow get this secret to
> decrypt the data. Additionally, access to the encrypted data isn't always
> logged, so if there is an intrusion, it isn't clear what data has been
> accessed and by who.

[1]: [https://vaultproject.io/intro/vs/chef-puppet-
etc.html](https://vaultproject.io/intro/vs/chef-puppet-etc.html)

------
mentat
Any reason why AWS dynamic credentials use IAM users instead of STS and role
assumption? That seems to be a somewhat more widely accepted way to do dynamic
credentials / role proxying.

~~~
ukd1
came here to see how it was done; was hoping for STS :(

~~~
mentat
[https://github.com/dump247/docker-
ec2-metadata](https://github.com/dump247/docker-ec2-metadata) and
[https://github.com/AdRoll/hologram](https://github.com/AdRoll/hologram) exist
as examples of this, just FYI.

------
malandrew
What's coming next from hashicorp? The one infrastructure piece for which good
open-source solutions still lack is ACL with solid auditing and variety of
permissions approval models.

There are a bunch of good ideas in this RFC,
[https://tools.ietf.org/html/rfc2904](https://tools.ietf.org/html/rfc2904)

Something like a host-your-own Amazon IAM would be pretty dope. Add support
for 2fa and especially u2f and you've got yourself a winner.

------
dserodio
I haven't finished reading the docs yet, but this part[1] caught my attention:

> When a backend is unmounted, all of its secrets are revoked and its data is
> deleted.

If a backend is similar to a filesystem as mentioned in the docs, that's quite
a surprise. If a backend is like a filesystem, the secrets (files) should
become unavailable when unmounted, but should become available again after
remounting the backend (filesystem).

Are the secrets actually _deleted_ (ie, can't be recovered again), or am I
misunderstanding?

[1] [http://vaultproject.io/intro/getting-started/secret-
backends...](http://vaultproject.io/intro/getting-started/secret-
backends.html)

~~~
mitchellh
I'm responding a little late here, but hopefully you'll still see this.

Yes, when you unmount a backend, all the data is deleted. Vault mostly behaves
like a filesystem, but not in this scenario. The raw data is not preserved. If
you want to preserve the raw data, you can remount (move the mount point).

We didn't finish it for 0.1, but we already built in secret enumeration into
the core. We plan to expose this at some point. If you want to unmount and
preserve your secrets, you can always enumerate them and back them up
elsewhere (unencrypted). This isn't yet available.

------
carllerche
Looks interesting. There is definitely a need for something secure to manage
secrets easily & efficiently.

I'm not super familiar with Shamir's secret sharing. The post says that the
master key is split up across a number of people. One thing that I am confused
about is if Vault goes down (possibly in the middle of the night), does it
require a quorum of humans to boot it back up?

~~~
mitchellh
It does.

The solution to this problem is to run Vault in high availability mode[1].
This will have a set of stand-bys that can take over if the leader goes down.
The idea is to unseal all the stand bys, and if the leader goes down, the
standbys take over, and the sealed one can be unsealed at some other future
point.

Security-sensitive things have been an interesting UX challenge for us, but
this compromise comes at an expense of UX for a much better security promise.
As one of the other comments says around this comment: without this feature
there would be no real security for the stored data.

[1]:
[http://vaultproject.io/docs/concepts/ha.html](http://vaultproject.io/docs/concepts/ha.html)

~~~
SEJeff
Can you call mlock in go to prevent any secrets from being paged to disk?

~~~
mitchellh
We already do this. And yes, you can (using the "syscall" package).

------
MCRed
Can someone explain hashicorps business model to me?

Love all the projects. Are they contracting out features to companies? Selling
support contracts?

Very curious-- I think if people, especially individuals, can make a living
from open source projects, that's great for everybody.

But do you have to be "famous" to do it?

~~~
druiid
To the best of my knowledge it comes from contracting/support and mostly their
Vagrant Cloud (now Atlas) features that are business/enterprise only, which
they charge money for. For example private hosting of Vagrant box images is a
paid feature. The costs for that are per-user/per-month.

------
ankushio
What are some of the existing tools/solutions which people use for managing
secrets in a production environment?

~~~
amenghra
[https://corner.squareup.com/2015/04/keywhiz.html](https://corner.squareup.com/2015/04/keywhiz.html)

------
flowerpot
Neat! I've been working on something similar for about the past six months,
although my approach is a little more minimalistic, but also based on shamir's
secret sharing. I was planning to release a preview the next couple of days.
If any one is interested [https://github.com/duse-io/api#how-it-
works](https://github.com/duse-io/api#how-it-works)

//edit My solution focuses more on the sharing aspect between people, but can
just as well be used like Vault

------
hexedpackets
This is really awesome! I've been passively looking for something like Vault
for a while, can't wait to start integrating.

I just finished tossing together a Chef cookbook to get Vault
installed/configured up to the init/unseal step. Here's the link in case its
useful to anyone: [https://github.com/hexedpackets/chef-
vault](https://github.com/hexedpackets/chef-vault)

------
BuildTheRobots
> Vault is one giant leap forward for practical security in a cloud
> environment.

My biggest security concern with a cloud (or VPS) provider is that they can
very easily read any keys or unencrypted data from memory. Still a worry when
renting a dedicated box, but it's a damn site easier when it's virtualised.

I don't see Vault helping me with this... but then I'm probably both paranoid
and missing the point.

------
AndrewWright
There must be something in the air regarding tools for managing secrets. First
Keywiz, then Credstash, a little python utility for managing secrets using AWS
KMS
[https://github.com/LuminalOSS/credstash](https://github.com/LuminalOSS/credstash)

------
dustinrcollins
The policies look like they'd be a bear to manage since they are path- and not
role-based. In order to implement least privilege with multiple actors you'd
have to be really careful with your paths.

Example: 2 apps. App 1 needs secrets A B and D. App 2 need secrets A B and C.

So we need to set up our paths in a way that App 1 can get A and B and C, but
not D. App 2 needs C but not D. Now when you want to modify secret access from
your apps you have to rethink how your paths are set up.

When you're instead assigning permissions to roles this is a lot easier. An
example: [http://blog.conjur.net/what-is-a-devops-secrets-
server](http://blog.conjur.net/what-is-a-devops-secrets-server)

------
LukeHoersten
I haven't been to HashiCorp's website basically since they released Vagrant. I
use vagrant all the time though.

A few reactions. First: amazing design. Awesome documentation. Second: wow
they have an awesome little suite of high-impact devops tools. I spent all day
learning about them.

The unified style of websites they have organized into "intro" and
"documentation" is awesome and easy to really digest what's going on quickly.
All of the tools seem powerful, well thought out, and complementary.

Vagrant's been awesome to me and I'm excited to start using some of their
tools like Packer, Consul, and Terraform in production!

------
christop
Interesting indeed! But I couldn't find anything on the website or
documentation about _how_ secrets are encrypted?

The security page seems to be a generic one about reporting vulnerabilities:
[http://vaultproject.io/security.html](http://vaultproject.io/security.html)

Edit: Found it under the threat model section here:
[https://vaultproject.io/docs/internals/security.html](https://vaultproject.io/docs/internals/security.html)

~~~
weitzj
See:
[http://vaultproject.io/docs/internals/security.html](http://vaultproject.io/docs/internals/security.html)

~~~
christop
Found it, thanks.

------
saintfiends
Slightly OT, but is there such a tool (to store secrets securely and
accessibly) for personal use that runs well with Linux and can be easily moved
around (portable).

~~~
vruiz
Try [http://www.passwordstore.org/](http://www.passwordstore.org/)

------
2bluesc
Would be cool if there was Yubikey support using Challenge-Response mechanism
to seal the vault.

If only I had time...

------
mikewhy
this may be an obvious question, but how do things like Vault/Keywhiz differ
from Consul/Etcd?

Is it simply that the former is more encrypted than the latter?

~~~
mikewhy
ah, as per
[http://vaultproject.io/intro/vs/consul.html](http://vaultproject.io/intro/vs/consul.html)

> While Consul can be used to store secret information and gate access using
> ACLs, it is not designed for that purpose. As such, data is not encrypted in
> transit nor at rest, it does not have pluggable authentication mechanisms,
> and there is no per-request auditing mechanism.

I guess my question now is why not add E2E encryption on top of Consul/Etcd
instead of a whole new system?

~~~
acveilleux
It's desirable to keep secret managements simple and as self-contained and
self-sufficient as possible. That way, it's much easier to drastically lock it
down without impacting normal users too much. Your organization might have,
say, 30 people who routinely need access to the orchestration / deployment
infrastructure but maybe 3 who need access to cryptovariables.

------
coherentpony
How do I know I can trust this?

~~~
vially
Read the source code:
[https://github.com/hashicorp/vault](https://github.com/hashicorp/vault)

~~~
coherentpony
Thank you. I read the article but it wasn't clear that it was open source.

------
kgilpin
It's pretty amazing to see how many new secrets tools for DevOps and cloud
have suddenly materialized in the last few weeks. At Conjur, we have been in
production in this space for almost two years. If you're wondering how to
stack up these different tools against each other, here are a few facts about
our solution:

* At Conjur, we care deeply about three things : (1) the overall user experience for developers, operations, and security personnel (2) the security of our code (3) high availability

* Conjur implements full-featured role-based access control for both "carbon" (people) and "silicon" (VMs, servers, containers, and other code)

* Conjur solves not just secrets management, but also SSH access management and software-defined firewall (granular access control for microservices)

* The Conjur core services are not open source, but our cryptography is ([https://github.com/conjurinc/slosilo](https://github.com/conjurinc/slosilo)), as are many of our client libraries and connectors (for example: [https://github.com/conjurinc/api-ruby](https://github.com/conjurinc/api-ruby), [https://github.com/conjur-cookbooks/conjur](https://github.com/conjur-cookbooks/conjur)). And Conjur cryptography has been audited by a respected 3rd party.

* Conjur features packaged integrations with Chef ([https://supermarket.chef.io/cookbooks/conjur](https://supermarket.chef.io/cookbooks/conjur)), Puppet ([https://forge.puppetlabs.com/conjur/conjur](https://forge.puppetlabs.com/conjur/conjur)), Docker ([http://developer.conjur.net/product_demo/secrets.html](http://developer.conjur.net/product_demo/secrets.html)), Heroku ([https://github.com/conjurinc/heroku-buildpack-conjur](https://github.com/conjurinc/heroku-buildpack-conjur)), AWS and more. New integrations are easy to build using the Conjur HTTPS and LDAPS APIs ([http://developer.conjur.net/reference/services](http://developer.conjur.net/reference/services))

We have also gone to great lengths to make it easy to try Conjur yourself,
with automatically provisioned, self-service demo appliances featuring the
Conjur server and client, Docker and Git:

[https://demo-factory-conjur.herokuapp.com/request/secrets](https://demo-
factory-conjur.herokuapp.com/request/secrets)

[https://demo-factory-conjur.herokuapp.com/request/ssh](https://demo-factory-
conjur.herokuapp.com/request/ssh)

