
Tor in a safer language: Network team update from	Amsterdam - QUFB
https://lists.torproject.org/pipermail/tor-dev/2017-March/012088.html
======
sfilargi
As a mere average user of computer languages, every time I play around with Go
I start wondering how a language like this became so popular.

It feels like it was invented in a universe where Haskell, OCaml, Erlang,
Smalltalk, Lisp and so many more languages and research in languages never
happened.

~~~
ASinclair
> As a mere average user of computer languages, every time I play around with
> Go I start wondering how a language like this became so popular.

You can pick it up in a weekend. A lower entry bar means more people will try
it out.

> It feels like it was invented in a universe where Haskell, OCaml, Erlang,
> Smalltalk, Lisp and so many more languages and research in languages never
> happened.

It was developed in a large enterprise context not in an academic context. I
think it shows.

~~~
im_down_w_otp
So was Erlang. The language, the runtime, and the standard library were all
purpose-built together to solve a real world problem... to build fault-
tolerant, distributed network applications.

Which makes Go's rapid popularity as a language for solving the same problem
even more peculiar. Especially since Go's surrounding tracing, debugging, and
online code swapping facilities are so much worse.

I'm sure some component of that is Ericsson's complete lack of interest in
evangelism and Erlang Solution's apparent lack of capability to effectively
evangelize, but it still seems like Go isn't so much filling a void as it is
being better at attracting an ecosystem that delights in reinventing a
particular set of wheels.

Java was very much the same way for a very long time.

~~~
mahyarm
Golang is simpler java, and that is it's target. It even has the same perf
profile as java. Erlang does have a worse perf profile.

~~~
fpoling
Golang performance profile is very different from Java. It's speed is closer
to C++, garbage collection optimized for latency, not bandwidth, very low
startup time comparable with typical native code.

~~~
Jweb_Guru
You can actually get a performance profile very much like Go's out of Java...

* Inasmuch as you can make a blanket statement like this about a language, Go's speed is very much comparable to Java's.

* Java has a huge number of garbage collectors available. You're just talking about the default, but there are extremely low-latency GCs.

* Java has a bunch of ahead-of-time compilers; thanks to Android this might be the most common deployment of Java.

Even green threads were tried (and sensibly abandoned) in Java before.

I think people may not be aware of all the options available in the Java
ecosystem, but other than sized types, which are theoretically coming to Java
10, there isn't much performance-wise that Go does and Java doesn't.

------
searchfaster
When I read safer language, 'Rust' came into mind automatically.. Not sure if
'Rust' will ever be as popular as golang but certainly see a future with it
popping up everywhere mission critical / super safe software is required.

~~~
JepZ
I am afraid Rust will become much more popular than Go is atm. I think the C++
crowd will embrace it as soon as it will be mature. I am sure it is better
than C++, but I still like the concepts and syntax of Go much better.

But as C++ programmers obviously never care about readability and simplicity,
I am pretty sure they will take Rust. After all, Rust is a decent language and
as users, we will all benefit from the migration.

~~~
vertex-four
I've... never had a problem reading Rust code. It's generally well-typed, and
makes good use of "automatic" error handling constructs to ensure errors flow
upwards without visually polluting the success case.

Whereas with Go code, I have to filter out all the error handling (which often
takes up 2/3rds of lines of code, even when it's just "if there's an error,
return the error" which it almost always is), wade through the mass of
functions that take interface{} and cast it to something internally, etc etc.

I've tried programming in Go and I find it horrifying - I either get to ignore
errors or spend ~80% of my code doing this, repeatedly, a few times per
function:

    
    
        foo, err := bar()
        if err {
            return nil, err
        }

~~~
JepZ
My problems with the readability of Rust is related to the syntax and not so
much about the code structure. For example, Smalltalk has the most readable
syntax I know, while I find it's code structure average.

I know that error handling in Go can be tedious, but to some extent, it is
also the programmer's job to utilize the features of the language to write
clean code: [https://blog.golang.org/errors-are-
values](https://blog.golang.org/errors-are-values)

Regarding your type problem I find very few cases where I need an empty
interface (mainly container types). Most of the time I use real interfaces and
the most code I have seen used non-empty Interfaces too.

~~~
vertex-four
I tend to find that Rust's syntax warts are primarily in function headers
(primarily due to lifetimes), and less so in the bodies.

> Regarding your type problem I find very few cases where I need an empty
> interface (mainly container types).

I've run into the issue with quite a few "generic" libraries, since Go doesn't
have a generics system. For example,
[https://github.com/manyminds/api2go](https://github.com/manyminds/api2go)
requires a few of them in common cases.

My alternative is writing the same code over and over, and hoping that I've
not missed something in one instance of it.

------
imron
Rust is almost always going to win for any projects that take a 'gradually
replace' approach, simply because it can interact so easily with C code in
both directions (i.e. C calling Rust and Rust calling C) in a way that other
languages can't, or can't without significant efforts/hassle.

The Rust team showed great foresight for bringing about those changes before
the initial 1.0 release and it will pay off in spades.

It's a key step in removing barriers to entry:

[https://www.joelonsoftware.com/2000/06/03/strategy-letter-
ii...](https://www.joelonsoftware.com/2000/06/03/strategy-letter-iii-let-me-
go-back/)

------
nickpsecurity
Since bitexploder asked, I'll add what I wrote on this on other forums. If
it's about secrets or anonymity, make sure you always use a safe language that
supports careful control and reasoning about both memory and CPU time. The
reason is that this enables covert, channel analysis for vulnerabilities that
leak secrets through storage and timing. It's why I wanted Freenet to ditch
Java aside from the obvious reasons. It's also why GC languages such as Go are
better not used. Although, memory management where programmer controls timing
& it's simple to analyze might be used. Reference counting comes to mind.

The other thing you want is proven, successful use in high-assurance systems.
That is, systems that either didn't fail or provably couldn't in certain ways.
These are almost all written in a subset of C or Ada/SPARK. The advantage of
using those is you can combine them with a vast array of proprietary or open-
source tooling to catch about any error you can think of if it's
implementation. There's also formal specification and protocol analysis tools
that combined with expert review can catch the rest. Rust, although a good
choice for increased safety/security, doesn't have such tooling yet. That
means they will get less correctness overall and over time vs MISRA-C or
Ada/SPARK unless similar ecosystem in industry and CompSci emerges for Rust.
That's why I recommend against it for high-assurance security for now.

It does seem good for medium-assurance security where you want to knock out
low hanging fruit in systems code. It will avoid serious errors in C while
providing additional benefits with type system and other features. Ada 2012 +
SPARK 2014 are the standard for safe systems since they systematically
eliminate all kinds of errors with a consistent design and tooling with
decades of field success. I haven't seen a direct comparison with Rust on each
protection to see if it matches it already or not. The main advantages Rust
has over them are its borrow-checker for temporal safety, more usable method
for safe concurrency, and (best for last) highly-active community to provide
libraries or help. Go has similar benefits if its GC works with your use case
but a lower, learning curve & possibly lower efficiency. Due to ecosystem
benefits, these are main two I'm recommending for medium assurance if
Ada/SPARK are too much to learn.

~~~
ianopolous
G'day Nick!

Java is getting an AOT compiler in July (Graal,
[http://openjdk.java.net/jeps/295](http://openjdk.java.net/jeps/295)) that
will let you AOT compile parts, or all, of your program, including the JVM
modules themselves. This would seem to leave the GC as the main source of side
channel vulnerabilities. The GC itself will become more pluggable as well,
with a pure Java implementation. What requirements would you put on a GC for
side channel safety? (Ignoring the obvious just allocate a large heap and
never GC anything, eventually just restarting the process when it runs out of
heap). What if we had a fully concurrent, pauseless GC (e.g. Azul), would that
change things?

What other issues would remain in your opinion?

~~~
Sanddancer
A concurrent GC running on a single processor machine is still going to pause.
There are still other ways to get into situations where an attacker can cause
the gc to kick in in ways where you can get information -- find a few CPU-
heavy functions that nudge the GC to kick in when and where you want it.
Harder, but even a background GC can wiggle into the foreground.

~~~
ianopolous
Even a statically compiled program run on a single core machine will pause
because of the OS scheduler. Assuming we have a multicore cpu then one core
can be dedicated to the GC.

~~~
nickpsecurity
Good you're thinking out the box. That pause might not be a problem, though.
The pauses that are a problem when the malicious app can control the timing
with an external observer seeing those manipulations. The OS scheduler is
usually independent of secret processing. It's just doing it's own thing
causing pauses that don't tell you about the secret itself. Theoretically,
there could be an esoteric attack in some situation where an OS scheduler
would accidentally leak somethinh or be activated to leak something. Not
popping into mind righg now, though.

------
muraiki
How has this community reached the point where the vast majority of comments
on this decision are arguing about Go, which isn't even the language that they
picked?

~~~
concede_pluto
pg's early essays like
[http://www.paulgraham.com/avg.html](http://www.paulgraham.com/avg.html) and
[http://www.paulgraham.com/power.html](http://www.paulgraham.com/power.html)
drew a crowd who want to see language design move in the polar opposite
direction. From that viewpoint, Tor only barely dodged a bullet.

------
Gaelan
This is either a horribly timed announcement or a joke written very seriously.
I have no idea which.

~~~
danieka
How so? I'm honestly wondering why.

~~~
steveklabnik
Today is April Fool's Day.

~~~
danieka
Oh. So I get it is a joke, I just don't get why it is a joke. I program daily
in Go and everytime I read about buffer overflows and a dozen other
preventable security holes I'm glad I'm programming in Go.

But maybe this joke is only funny to those who enjoy programming in C? :)

~~~
masklinn
Doesn't seem to be a joke, the "meeting" was week-long and lasted from the
22nd to the 27th:
[https://trac.torproject.org/projects/tor/wiki/org/meetings/2...](https://trac.torproject.org/projects/tor/wiki/org/meetings/2017Amsterdam)

It's just the ML recap mail which hit a rather poor timing.

~~~
kbenson
The email was time stamped as being prior to April 1st UTC, so it's actually
less an issue of the recap email being poorly times and more an issue of the
post to HN being during a confusing time period.

~~~
Gaelan
I mean, the post could have been posted on April 1. We don't know that it was
posted from a UTC area.

~~~
masklinn
I believe Sebastian is German (or possibly Austrian), I think the CET date
change was last week so we're in CEST (UTC+2) which means it was posted at 30
to midnight local, close enough that it could be just in time for April 1st
local.

------
bitexploder
I am curious why they were advised not to use Go. Probably not a safety
concern.

Edit: cgo != Go. Thanks for the responses. I have done a bit of Go, but just
pure Go.

~~~
geofft
Go and Rust are very different languages. Rust is, by design, well-suited to
Tor's use case, where they have a large C or C++ program and they need to
incrementally rewrite parts of it (and maybe never all of it!) in a better
language.

It turns out (or so I hear) that Google statically links everything in
production, and has been using C++ as a language to implement HTTP endpoints
for a long time. So Go is a better C++ _for what they want out of a better
C++_ ; for the rest of us, it looks more like a compiled language along the
lines of Python/Ruby/etc. with a nice deployment story. If you want that out
of your better C++, Go is great. If you want to reimplement _all of_ Tor from
scratch, Go certainly seems like a reasonable choice.

But as a result of these priorities, Go basically doesn't have
interoperability with the platform ABI as a goal. (For some combination of
historical reasons and the lack of complicated features in C, the platform ABI
on just about every platform these days is a C ABI.) Rust does; it uses a
standard compiler toolchain (LLVM) instead of what's basically a custom one
(Plan 9), and the standard toolchain knows how to generate calls that follow
the C ABI. Rust doesn't have a runtime of its own, and it's safe to directly
call into a Rust program from some arbitrary point in a C program. Rust's
allocator doesn't care if you do stupid things with pointers it allocates, as
long as you give them back eventually. Rust doesn't create threads on its own
unless you ask. Rust functions use the normal stack. Rust on UNIX uses the
platform libc. And so forth.

It's possible to call C code from Go and vice versa, just as it's possible to
call C code from Python and vice versa. But Go is not best tool for this
particular job.

~~~
socmag
Totally agree, and Rust isn't a sane choice either even though the points you
make are valid.

Rust has some interesting features, but being C isn't one of them.

~~~
geofft
Can you expand on why you think Rust isn't a sane choice here?

~~~
socmag
Because again you are deviating from the industry standard completely portable
lingua-franca language that is designed explicitly for precisely these types
of problem spaces, and is perfectly in tune with the OS and existing standard
library.

What would be the advantage, just improved memory safety guarantees for people
working on the project?

If that's the case start again from scratch and the first thing you do is
build a handle based memory management system that everything must go through
Safexxxx() versions of everything and then explicitly enforce no other memory
access patterns.

If you want to do concurrency, build simple threads with input queues that
would just be like channels in Go.

You can use a completely functional actor model in C. Is all this object
oriented dangling pointer stuff that is scary, but there really isn't any need
for that.

Software pipelines with slab allocation or ring buffers virtually guarantee no
leaks and are very scalable and cache friendly.

That would be my personal recommendation.

~~~
geofft
You _can_ do all of these things in C - but at the point that you're enforcing
use of your Safexxxx() functions, you've already given up on the industry
standard. You can't use the normal OS libraries; you've got to route
everything through your special functions.

The advantage of Rust is, honestly, that it has a community of people who are
excited to do this sort of work in a language. At least 50% of the advantage
of any particular language is not the benefits of the language directly, but
the community around it (that derives indirectly from the benefits of the
language, but also from marketing and other things). Some time ago I needed to
write bindings to SCM_RIGHTS / CMSG_*, which is probably the trickiest part of
the libc API (all of cmsg(3) is macros that do stupid things with casts). It
was annoying, but it led to a better interface than C itself offers, and
someone else found a bug in the OS X handling. I couldn't expect that if I
were working on my own custom reimplementation of libc.

~~~
socmag
So basically what it comes down to is the argument for building it in Rust is
there are some excited people.

And is funny you talk about community, since as the guys on slashdot and
others pointed out, the community of people who can use and work in C
effectively is orders of magnitudes more than Rust. Have you any idea how many
Linux Kernel developers there are alone?

[http://m.slashdot.org/story/324469](http://m.slashdot.org/story/324469)

Building API's specific to the use case is part of our jobs as professional
developers.

Abstracting a few things things isn't "rewriting libc", that is just general
practice for most decent size projects.

Anyway, the decision is made, so the whole thing is moot at this point.

~~~
geofft
> _the community of people who can use and work in C effectively is orders of
> magnitudes more than Rust_

The key word here is "effectively." For the purposes at hand, effectiveness
includes memory-safety. Do you know of a single project that implements
cmsg(3) in C or C++ in a memory-safe, well-typed, cross-platform way? Or a
project where I can submit a pull request and expect it to be reviewed, tested
across platforms, and fixed?

I do genuinely believe that the community of people who can use and work in C
effectively, in the sense of effectiveness that I and the Tor Project are
interested in, is orders of magnitudes _smaller_ than Rust.

I have a very good idea of how many Linux kernel developers there are - and
also how many high-severity security bugs there are. I'm a coauthor of a
research paper where we wanted to talk about exploitable security bugs in
Linux, so we sat down and found a local privilege escalation in _hours_
(CVE-2009-0024).

------
eternalban
Not sure why Ada is not taking off these days. It is mature and has all the
safety and concurrency bells and whistles:

[http://www.adacore.com/uploads_gems/07_safe_secure_ada_2005_...](http://www.adacore.com/uploads_gems/07_safe_secure_ada_2005_safe_memory_management.pdf)

[https://en.wikibooks.org/wiki/Ada_Programming/Tasking](https://en.wikibooks.org/wiki/Ada_Programming/Tasking)

[http://courses.cs.vt.edu/cs5204/sp99/Overheads/6UP/6UPCSPand...](http://courses.cs.vt.edu/cs5204/sp99/Overheads/6UP/6UPCSPandADA.pdf)

~~~
bluejekyll
1) Two words: "begin" and "end";

2) Unix, C is so fundamental to building software, that I think any language
that doesn't share syntax with it is doomed. Having a common syntax helps in
learning new languages, IMO, and can also be a launching point for differing
semantics...

~~~
nickpsecurity
I'll give you it's overly verbose even though quite a bit of it was
justifiable. The theory of Ada's designers was that people read software more
than write it. So, the syntax should be designed to facilitate catching errors
in maintenance mode, during extensions, or during integrations. It's done
phenomenal at that per industrial, case studies despite having been invented
in the 80's when lots of language decisions were still being debated.

EDIT: I keep thinking a different language that acts as a front end w/ a
better syntax might be a good idea. It outputs Ada that integrates with the
tooling ecosystem. Also, a seemless FFI for C libraries like Julia's.

~~~
socmag
I totally agree that code should be written to be read.

It certainly wouldn't hurt if people simply used a more literate programming
style no matter what language they choose

Far too many people think code as below is acceptable.

This is C obviously, but pretty much equal horrors around in every language.

This isn't 1994 and the compiler really doesn't care how long your variable
names are, plus EatWhite() is pretty damn fast.

    
    
      const int to_pn  = base_n ^ label_n;
          const int from_p = _array[to_pn].check;
          const int base_p = _array[from_p].base ();
          const bool flag
            = _consult (base_n, base_p, _ninfo[from_n].child, _ninfo[from_p].child);
          uchar child[256];
          uchar* const first = &child[0];
          uchar* const last  = flag ? _set_child (first, base_n, _ninfo[from_n].child, label_n)
            : _set_child (first, base_p, _ninfo

~~~
wott
> This is C obviously

No, that's C++.

~~~
__david__
You can't say that definitively based on that code. It could very well be C
code. There's nothing there syntactically that a straight C compiler would
barf on.

~~~
socmag
Absolutely correct.

------
viraptor
I like their approach of chipping away pieces instead of porting the whole
system. It will result in better modularity and some generic libraries that
can be used by other people. Seems like a win-win.

------
socmag
I live and love in Amsterdam and Golang seems to be the quintessential hipster
language for this quintessentially hipster city.

Booking.com's soup du jour if you will.

Basically, if you aren't using JavaScript in a web shop, and you claim to be a
"full-stack" "ninja", then probably you are using Go around here as a jobbing
programmer.

I know that sounds terribly cynical and obviously a massive generalization but
that is my personal experience.

I don't have a problem with Go per-se, but I do have a problem that a lot of
people seem to go to extreme lengths to defend what someone else mentioned is
frankly a pretty "mundane" language, citing memory safety, but more strangely
portability and performance as its wonderful virtues.

When I meet with the zealots of the Go community around here I often have to
quickly excuse myself with good grace.

Seriously, if that's what you want... go (pardon the pun) use C#, it is a
saner and more expressive language, has a higher performance runtime and is
way more portable with less vendor / career lock-in.

Go... I just don't get it.

Positives... Has a fairly nice package manager like npm.

Isn't made by Microsoft if that's your thing.

Oh and heaven forbid you don't have to think much... until you do because it
is slow.

For a project like Tor though, which is damn slow as it is, I think it is
totally the wrong choice.

Developers, please suck it up, get over it and learn C/C++ and some variation
of Lisp.

Fine, go play with different languages for fun stuff. Use python for ML, try
serious meta-programming in D, or jump into Haskell for kicks.

On the other hand modern C++ really can be a very safe language to work in if
you can be bothered, and you are only deceiving yourself and your project
going with something less sympathetic to the machine itself, especially if you
are working on infrastructure level systems.

Sorry if this offends anyone, and of course it is just one opinion, but I
think I'm being fairly nice as compared to what Linus might have said in
comparison.

Basically stop it with the hobbyist shit. You are horrifying me and probably
many others.

You aren't writing a web page here, so please treat the project seriously.

IMHO

~~~
hamburglar
> Developers, please suck it up, get over it and learn C/C++ and some
> variation of Lisp.

I've got over 20 years of writing production C/C++ under my belt and I know
Lisp. So I've "sucked it up." Am I allowed to like Go now?

It never ceases to amaze me how many people are bothered about other people's
taste in something so mundane. If you don't enjoy programming in Go, don't do
it. I personally think it feels light and easy like a scripting language, but
with more C-like performance.

I like the fact that its stdlib is very complete, so I can sit down and do
pretty much any kind of small project with zero external dependencies, I like
the fact that the core libraries are well-designed so the interfaces are
consistent and easy to learn, I like the fact that it produces statically-
linked binaries so deployment issues are minimal, I like the simplicity of the
CSP approach for a lot of concurrency problems. I could also come up with a
list of things I __don 't __like about Go, but I 'm not going to bother,
because I've decided that on the whole, I really like it as a tool.

Btw: > Positives... Has a fairly nice package manager like npm

One of my biggest complaints about golang is that it really _doesn 't_ have
nice dependency management. It's terrible and probably the one issue that has
made me seriously consider walking away. :)

~~~
eropple
_> If you don't enjoy programming in Go, don't do it._

Ordinarily, I would agree; I don't really care what people do when writing
application code. The way it's filtered into devops tooling makes the choices
of Go peoplea problem for me, though. If I'm going to be stuck with a language
with bad error handling and inexpressive typing, I'd rather it be Python or
Ruby so at least I can leverage dynamic typing instead of bad static typing.

~~~
hamburglar
Well I could make the same complaints about how ruby has "filtered into devops
tooling." If I never see another quasi-DSL that's really just a cute set of
ill-specified ruby methods I will die happy.

~~~
eropple
You're talking a taste thing; I'm talking a _correctness_ thing. The inability
to create an easily-managed DSL doesn't compensate up for Golang being
actively dangerous when it comes to error management. One of these provides
structures for ensuring that you catch errors and can do the right thing, and
the other demands that you if-check them upon every invocation of a method
that can fail.

------
poland2
I hope Linus can develop a new language.

------
dguido
Literally none of this matters. None of the flaws exploited in Tor are memory
corruption flaws. They are entirely architectural and design flaws. No
attacker cares if they move to Rust. _NO ONE_.

~~~
steveklabnik
> None of the flaws exploited in Tor are memory corruption flaws.

[http://www.cvedetails.com/product/5516/TOR-
TOR.html?vendor_i...](http://www.cvedetails.com/product/5516/TOR-
TOR.html?vendor_id=3132)

This list is full of memory safety issues.

~~~
dguido
> None of the flaws exploited in Tor are memory corruption flaws

_exploited_ is the operative word here.

Show me the list where people wrote exploits for the bugs you point out, or
where someone abused them to de-anonymize a Tor user? There aren't any.

------
omidraha
Concurrent thread for this safer language:
[https://news.ycombinator.com/item?id=14013444](https://news.ycombinator.com/item?id=14013444)

------
generic_user
I'm disappointed they did not consider COBAL given its vast superiority and
flawless track record for April first rewrites.

~~~
kilotaras
Fri Mar 31 21:23:27 UTC 2017

~~~
generic_user
I believe there is a crate they can use to prevent date bound out of bounds
fun leaks. (can't find the link atm)

------
najati83
This is exciting not only because of the Tor project itself but because this
will set an example for other projects to follow.

~~~
wklm
This creates also an opportunity to engage Mozilla into contributing

~~~
duneroadrunner
If any of the developers are reading this, converting the existing C code to
SaferCPlusPlus[1] (a memory safe subset of C++) is probably a more expedient
solution (if that's what they're looking for). (And speaking of contributing,
an automatic translation (assistance) tool is in early development, and could
maybe be functional in short order with a little extra motivated talent... :)

[1] shameless plug:
[https://github.com/duneroadrunner/SaferCPlusPlus](https://github.com/duneroadrunner/SaferCPlusPlus)

~~~
tjalfi
I would recommend updating your .gitignore for VS2015. The .vs directory and
*.VC.db should both be excluded from the repo.

~~~
duneroadrunner
Thanks. I'll take all the git/github advice I can get. :)

------
z3t4
we all want to be safe. no one can say no to safety ...

------
m-j-fox
I don't know much about Tor. But I hope I can route all of my home network
traffic through it. That or route everything through VPN. I'll bet you can
guess why I'm suddenly interested.

~~~
seibelj
Founder of [https://easyvpnrouter.com/](https://easyvpnrouter.com/) ask me
anything or how to build one yourself if you want a project

~~~
m-j-fox
I guess the main question about this product (which is the sort of thing I'd
buy -- an hour of saved effort pays for it) is how can I trust you to be less
evil than Comcast?

I guess if you're open-source and a lot of people are paying attention, that
would do it.

~~~
seibelj
It is much more difficult than 1 hour to setup yourself, because of FCC
regulations flashing new routers is much more difficult, you need a TFTP
server, it's pretty annoying. In addition the edge cases of the internet going
out, then when it comes back bringing the VPN back up automatically, being
able to switch PIA accounts / countries easily, it's not a simple thing. I got
into this because I wanted to build one myself, and I have a friggin master's
degree in CS and it took me several weekends. So I figured a business could be
formed :)

