
Show HN: Fast Rust BitTorrent Tracker - greatest-ape
https://github.com/greatest-ape/aquatic
======
greatest-ape
Author here.

I wrote a BitTorrent tracker in Rust a few years ago. A while ago, I found out
about the dashmap crate (providing a concurrent hashmap) and thought that I
could use it for a new implementation, which could be both performant and
simple. I tried using it, but due to blocking, I had a hard time reaching good
performance. My realization was that since BitTorrent tracker requests are
highly skewed towards popular torrents, even a map which only locks on
accesses to the same key would not scale well when used like this.

I experimented and settled on a design with many threads each opening a socket
with SO_REUSEPORT and using mio to wait for requests. The requests are read,
parsed and sent through a crossbeam channel to request workers, which collect
a number of them before locking the mutexes guarding shared state and doing
work required to generate responses. These are sent back to the socket
workers, which encode them and send them back over the network.

This solution is quite performant and during heavy load testing, I couldn’t
even get a lone request worker to reach full CPU utilization. (Adding more
would likely not bring much improvement anyway since they would block each
other most of the time). Results in terms of responses per second indicate
that aquatic scales a lot better with more threads than the established
project opentracker, being able to reach more than twice the throughout at
best, at the price of using many more threads. (Benchmarks are available in
the README file.)

I’d like to thank (in particular) the authors of mio, crossbeam, hashbrown and
indexmap for making this fun project possible.

