
How to Write a New Git Protocol - alecrn
https://rovaughn.github.io/2015-2-9.html
======
cornstalks
This would have been very nice to have a few months ago! At Aspera we wrote a
protocol handler for Git[1] that let us send data over fasp (a reliable UDP
protocol for fast bulk-data transfers). I learned a ton by reading Git's
source code to figure out how to do it, but this blog post is much more pain-
free.

Very good write up!

[1]:
[https://www.youtube.com/watch?v=ECq7dCwixaY](https://www.youtube.com/watch?v=ECq7dCwixaY)

~~~
alecrn
That's very interesting! The alternatives to TCP that can be written in UDP
which are optimized for a certain application are especially interesting to
me. Working at the UDP level can present a lot of interesting performance
options, kinda like working in C instead of Python.

Something I've been curious about is if there are proxies that can convert
protocols. So for instance, on your local machine, you could have a proxy that
turns TCP connections on port 8000 into FASP connections to another machine.
This would let you use an ordinary web browser over FASP.

You could even pipe the proxies, e.g., TCP -> FASP -> MinimaLT [1]. That way
any program could have really fast data transfer over an encrypted tunnel.

[1]
[http://cr.yp.to/tcpip/minimalt-20130522.pdf](http://cr.yp.to/tcpip/minimalt-20130522.pdf)

------
michaelmior
Thanks for sharing this. It's a good walkthrough and potentially quite useful
for anyone wanting to implement a new protocol. It seems like the wrong tool
for the job to me however.

This seems like it would be more easily implemented with smudge/clean
filters[0]. In fact, this is exactly what git-crypt[1] does. The idea is that
you run a filter on every file as it's checked out to do the decryption and
every time a file is staged to do encryption. This requires nothing extra on
the remote repository and you can you git commands as normal.

[0] [http://git-scm.com/book/en/v2/Customizing-Git-Git-Attributes](http://git-
scm.com/book/en/v2/Customizing-Git-Git-Attributes) [1]
[https://github.com/AGWA/git-crypt](https://github.com/AGWA/git-crypt)

~~~
JoshTriplett
Depends on the behavior you want. With smudge/clean filters, the files in the
repository history are encrypted. That means you don't get the benefit of
delta encoding, and any tools you have that look directly at the repository
(such as gitk or anything based on libgit2) cannot operate on the cleartext
data.

On the other hand, git-remote-gcrypt ([https://github.com/joeyh/git-remote-
gcrypt/](https://github.com/joeyh/git-remote-gcrypt/)) encrypts when pushing
and decrypts when pulling, which leaves the local repository unencrypted but
keeps it encrypted on the untrusted remote server.

~~~
michaelmior
That's a fair point :) Although I do like the fact that the smudge/blur
approach means that you can selectively encrypt files as well as still share
the entire repository publicly via any protocol.

~~~
JoshTriplett
True. And it also has the advantage of failing closed (if you accidentally
push to the wrong place) rather than failing open.

------
fiatjaf
For people wanting to store their passwords, personal data and large files in
a git-like fashion, I recommend Git-Annex: [http://git-
annex.branchable.com/](http://git-annex.branchable.com/)

------
fortytw2
Nice job! I had no idea anything like this was even possible.

------
packetized
This seems interesting to me from the perspective of being able to securely &
incrementally back up a media collection to a cloud storage provider (S3, etc)
without fear of it being scanned for copyrighted content, a la Dropbox. There
are probably solutions out there to do this already; I'm just not aware of
them.

Very interesting article, and well-written walkthrough.

~~~
alecrn
That's the reason I wrote this for myself, and why I made
[https://filegrave.com](https://filegrave.com), as I thought it could be
useful to others as well.

Git is already a great tool with a workflow for handling incremental updates
that I happen to be used to working with, so there was no need to reinvent the
wheel.

------
callesgg
git over ssh should be as safe as ssh, right? Does it somehow go outside the
ssh tunnel?

~~~
beagle3
He is worried about data-at-rest after the ssh.

Specifically, he mentions wanting copies of his private keys backed up, but
NOT having copies of his private keys in a github repository -- which would be
the case with a standard push through ssh.

