
Implementing a Tor relay from scratch in golang (2015) - lox
https://tvdw.eu/blog/2015/01/24/implementing-a-tor-relay-from-scratch/
======
tvdw
Hi. I wrote the post, happy to answer questions.

* yes, more modern versions of Go would likely mitigate some of the memory pain * yes, crypto/tls is fast now * no, crypto/tls still has insufficient functionality for implementing this. crypto/tls implicitly assumes you want to authenticate the channel through certificates, which Tor doesn't do * I was using go 1.4 * yes, I tried Rust

~~~
wbl
Did you consider a different concurrency strategy to avoid the deadlocks? With
separate reader-writer threads you don't have the deadlock you mentioned.

Crypto/tls doesn't support renegotation, which Tor needs, but they are getting
rid off.

~~~
tvdw
There are separate reader/writer goroutines, I don't think splitting them up
further would've helped much. The problem is that all connections may end up
needing something from all other connections, and as soon as one of them slows
down (slow network, etc) its channels start filling up, taking other
connections with it :-)

This could've been mitigated by applying backpressure in a bunch of places,
and is ultimately a problem of Tor and not Go, but the nature of Go makes it
hard to build code to do that.

As for renegotiation: my work on the Go version of Tor had some nice side-
effects, and indeed, renegotiation was finally removed :-)
[https://gitweb.torproject.org/tor.git/tree/ChangeLog?id=55c4...](https://gitweb.torproject.org/tor.git/tree/ChangeLog?id=55c468c5211d5b74acb677767f14d91cd0304771#n1654)

------
billhathaway
There were some very significant[0] crypto speed improvements in the Go 1.6
releases. It would be interesting to see how well a pure go implementation
would perform.

[0] Russ Cox mentions a ~ 10x performance in a TLS benchmark between 1.5 and
1.6.2 at
[https://github.com/golang/go/issues/15713](https://github.com/golang/go/issues/15713)

------
danesparza
The author doesn't indicate what version of Go they were attempting to use,
but in the last two years, Go has seen many improvements by their very active
dev team.

Notably, there were several changes to crypto in 1.5:
[https://golang.org/doc/go1.5#minor_library_changes](https://golang.org/doc/go1.5#minor_library_changes)

1.6:
[https://golang.org/doc/go1.6#minor_library_changes](https://golang.org/doc/go1.6#minor_library_changes)

and 1.7:
[https://golang.org/doc/go1.7#minor_library_changes](https://golang.org/doc/go1.7#minor_library_changes)

------
lox
I posted this because I was interested to see if anyone else had experienced
gaps in the golang `crypto/tls` library. The original author is pretty
scathing about Go's implementation, but my impression thus far has been that
it's an amazing quality part of the Go standard lib.

I'd take it any day over openssl, that is for sure. I wish the author had
published methodology on the benchmark comparison, would be interesting to
dissect that.

~~~
gravypod
From what I've seen in everyone's work is that Go's STD libs are amazing...
for the the exact use case that Google has needed.

When you have to deviate from doing anything that Google/Pike would consider
"acceptable" for what they would see as the "plebeian programmer" the STD libs
aren't built for it. I'd not say it's a side effect of the language (not even
the fact that it's lacking generics). I'd just say it's age. It's too young to
be refined.

~~~
lox
I'd say the std library is built exactly for the "plebian programmer" but such
that it scales up well to larger use cases, pretty much exactly how the
language itself is designed. I think how they've responded to vendoring
package management is a great example of how they do take external feedback,
but don't make knee jerk changes, and I really appreciate that.

~~~
jasonlai
Really not a big fan of some of Go's std libraries that come with heavy use of
global states, baked into packages like http and flag.

I know one can use other functions in the same packages to create instances of
HTTP servers or flag parsers. But the default approach coming as a part of the
std libs will more or less make Go newbies to take it for granted as a
programming idiom, which is not the best as the project grows

------
daenney
The article seems to contradict itself, between "the Go implementation broke
the speed record" and "Go makes the product too slow."

Granted the cgo stuff and the memory usage would be something that one needs
to deal with. Did you talk to go-nuts at all? They might've been able to offer
some more insight into all of this, a better way to deal with the cgo related
issue and perhaps even make some changes to handle these kinds of cases
better.

~~~
tvdw
Ha, I see how it can sound like that :-)

Yes, it broke the speed record: a multithreaded application outperformed the
singlethreaded version. But I wasn't happy with the result. It consumed an
order of magnitude more memory, and gc times were potentially harming users
(not a widely researched subject, but gc times in low-latency mixnets can
likely harm user anonymity). Oh, and it would occasionally crash with OOM
errors.

------
IshKebab
> Go has its own TLS implementation called “crypto/tls“, apparently because
> agl__, one of the people working on the language, decided so.

I thought it was rewritten because of the terrible quality of the OpenSSL
code, which turned out to be a very good decision.

------
jasonlai
Have you tried Rust? It interacts with C libraries fairly nicely.

------
keyle
Nim might have been a good language for this. As fast to write as Go, direct C
access.

~~~
nimmer
And it can generate system libraries. An embeddable "Tor as a library" could
be useful for large client applications (e.g. browsers)

------
dimitar9
go is a meteor for sure. C,live long and prosper.

~~~
HowardStark
If you don't mind me asking, why do you think so? It seems like a pretty solid
language.

~~~
lox
Don't feed the trolls :) From the article, the author concludes that it's not
good for anything where you need performance-critical parts of the code
running via cgo. That has certainly not been my experience, but I'd love to
hear other people's experiences.

