Hacker News new | past | comments | ask | show | jobs | submit login
Blackbox: Safely store secrets in Git (github.com/stackexchange)
150 points by akerl_ on Sept 3, 2014 | hide | past | favorite | 43 comments

There are a lot of alternatives in this space (I wrote transcrypt [1] for just this purpose), but I think the killer features here are that you don't need shared credentials since you can have multiple keys, and that it's generic enough to work with multiple VCS's.

With transcrypt, I do the encryption/decryption transparently once a repository has been configured by utilizing Git's clean/smudge filters, but it uses OpenSSL out of the box rather than GPG. A different workflow, and there are certainly pros and cons to both ways.

[1] https://github.com/elasticdog/transcrypt

Ansible handles this issue very cleanly with a feature called "vault": http://docs.ansible.com/playbooks_vault.html

I think it probably works better integrated into the deployment system. The developer can still write {{ DBPASSWORD }} wherever they need and not have to worry that they don't know what the password on production or staging is.

The README.md explains how to do this cleanly with Puppet and Hiera.

Interesting. For storing passwords I use Pass[1] which encrypts everything with GPG and then store it using Git in a private bitbucket repo. Works really well for me. I love the simplicity and ease of use.


One of the things that looked cool about blackbox to me was the support for encrypting the data with multiple keys and managing that.

I know some folks who use Pass and recommend it highly, but it seems focused on the personal use case like what you're doing.

I do still need to set up Pass myself, and was thinking about making an Alfred workflow to let me quickly access credentials. Does such a thing already exist? (searching for "pass" is not exactly effective :))

You can specify multiple gpg ids when using "pass init":

  pass init [-p sub-folder] user1@example.com user2@example.com
This enables you to control access per subfolder. You can keep your personal passwords in one subfolder and then have various shared subfolders. "pass mv" and "pass cp" will reencrypt with the gpg ids for each subfolder as necessary. You can also "pass init" an existing subfolder to add or remove gpg ids for existing files. See the man page for details [1].

[1] http://git.zx2c4.com/password-store/about/

git-crypt[1] also does per file encryption

[1]: https://www.agwa.name/projects/git-crypt/

And the upcoming version (which is overdue--I've been busy) will support encrypting with GPG with multiple recipients, like BlackBox.

If you can sneak a key into keyrings/live/pubring.gpg, and wait until someone runs blackbox_update_all_files without noticing, you then have access to all files.

Perhaps it should keep a signed copy of the last version of pubring and show a some kind of diff between the verified previous version and the current version before going ahead.

I have been looking for a Python solution like this that does not encrypt the entire file, only the actual password entries. I stumbled upon Jaeger a while back which is written in Go. https://github.com/jyap808/jaeger

By encrypting only the password entries, version control systems can still track plain text configuration changes without showing secrets.

Does anyone know if something like Jaeger is around but written in Python?

Why does the HN title say "in git"? This tool seems to be specifically built to handle hg as well. A pleasant surprise for hg users like me. :-)

My apologies; I took the title from the name/tagline of the repo. It won't let me edit the title now, but if a mod wanted to do me the favor and make it say "git/mercurial"...

Oh, I didn't notice the tagline. HN favours the original article's title in most cases, so I suppose a tagline is about as close to a title as we have in this case.

I've corrected the tag line. Thanks for pointing that out!

Ironically Blackbox was originally for a Puppet repo that used Mercurial and only supported Git as a skunkswork project. Now it is devoid of any Puppet dependencies and is mostly used for Git, but the Mercurial support is solid. In fact, you could easily extend it to support other systems. CVS support, anyone?

I find the teams I work with generally end up transmitting secrets over untrusted centralized chat protocols. I'm glad to see folks stepping into this space and popularizing and simplifying this sort of security!

For environment-based configuration values and multi-member teams, I wrote conf_conf [1].

[1] https://github.com/jkassemi/conf_conf

I've slowly been able to train my coworkers to use https://ezcrypt.it/ when sharing secrets rather than putting them directly into untrusted chat. Works great.

Vaguely along the same lines, I use something similar for keeping my journal. Because I'm paranoid. :P

I create/view/edit files and then I push the directory containing them to some git remote.


Usage commands at the bottom.

We used something similar for Puppet: https://github.com/TomPoulton/hiera-eyaml

Really neat thing was that you could inline encrypt just the secretes making the rest of the Puppet module viewable and comprehensible.

Looks like the read me mentions this for strings.

I think this tool is useful, but it could be even more useful if the extra manual steps were automated as part of the tool. I can see myself constantly looking for examples to perform each step, rather than just using an intuitive tool.

Personally, I don't use anything. I just check in my .gpg files and open with Emacs, which will decrypt/reencrypt the files for me. Any other editor worth their salt should have similar functionality.

Good to know!

This was originally written for Puppet files. The Puppet Master needs to have a separate key that permits it to decrypt all the files.

Shameless plug for a very simple tar-like CLI app done in Python:


why not just use openssl or another command line tool?

To encode: openssl aes-256-cbc -a -salt -in secret.zip -out secret.zip.enc

To decode: openssl aes-256-cbc -d -a -in secret.zip.enc -out secret2.zip

That's essentially what I've automated with transcrypt (which uses Git's clean/smudge filters) to do that for you transparently:


and nevermind. The doc explain why (large teams, key sharing, etc)

Maybe when openssl supports gcm or some other authenticated symmetric cipher mode on the command line?

Why not? Why would you want to? Computers exist to make things easier, not harder.

Interesting... Very similar to my golang project.


additional close system: https://github.com/jschauma/jass

this solves the large team problem, too- it fetches membership data and ssh pubkeys from ldap. great if you have it, but only exists in large organizations.

this of course doesn't do the gitwork for you, but it does avoid creation of another set of keys to be mismanaged.

we use https://github.com/ahoward/sekrets at dojo4. love it.

Storing anything of value in the cloud, regardless of encryption, is the very essence of stupidity. How many times does this fact need to be reiterated in the news before people get it.

It would be helpful if you could tell me where you got the impression that I was encouraging use on cloud-storage services. Was it in the README? the code? a post I made on my blog? I ask because I'd like to correct that doc/post/etc. so that others don't get the same impression.

Tom Limoncelli http://the-cloud-book.com http://everythingsysadmin.com

P.S. Blackbox was written to safely store the occasional secret file (SSL certs, mostly) in a Puppet repo stored on an in-company source code server.

Even though I personally knew everyone with access to the repo, root access to the repo server, or access to where the backup tapes are stored... it became essential to encrypt those secrets. Honest, I don't trust the people who handle our backup tapes... one of them is me and I'm the least trustworthy person I know.

Even more importantly, now that I could open up the repo to all my coworkers, it enabled me to collaborate across the company. They could read my code, even submit pull requests, and I didn't have to worry that they had access to SSL certs.

A few years ago, my company was setting up a second office a few hours drive away. They wanted a replicated file share between the two offices. For the initial sync, they drove a NAS from the main office to the satellite. Then they had (openvpn-encrypted) rsyncs on a nightly basis.

But the ITAR directory on the NAS was never rsynced. Instead, they drove the files out there every week using a flash drive.

Seemed like a crazy policy to me, but after Heartbleed, who knows?

You could write properly encrypted data on a billboard in Times Square and it would still be secure.

Sharing passwords or private keys is inherently risky, but simple necessity always prevails in the end. Acknowledging the risks, developing protocols taking it into account intelligently, is all you can do.

Technically, public key cryptography is a (set of) protocols for sharing secret keys.

Storing things of value in the cloud, without inspecting and validating that the underlying tech is secure enough is the essence of stupidity. You store your money in the bank, rather than in your closet right? And they just put that money as a bit in your computerized account on their mainframe, right? How is that any different than a properly secured 'cloud'?

git != "the cloud"

it seems like a lot of people now think that git == github.

When was it ever in the news that something encrypted was compromised?

I remember this story, though I forgot exactly how deep it went. I thought it was just compromising certificate authorities, or sneaking in physically to catch part of the stream that's not encrypted like they did with Google. That's irrelevant here because the only computers that would hold decrypted info in the case of Blackbox are those computers which would anyway hold it decrypted without Blackbox.

So there's still the part about compromising the encryption software. Well, GPG is not proprietary so that's a lot harder.

As far as somehow brute forcing the cyphertext, I guess that may be possible, though they don't give clear details about that here. That being the case, they still don't say anything about GPG. Perhaps the biggest argument of all is that Edward Snowden himself uses it, knowing everything he knows.

Applications are open for YC Summer 2021

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact