A BitTorrent client https://github.com/anacrolix/torrent, and several projects using it. The original idea started in Python which just didn't cope. Things are probably better now in Python with green-threading being a standard concept, but you just can't easily get the throughput you need without minimal overhead, and that overhead is just too high in Python.
Oh yeah, no argument there. Even now with the fantastic nuitka that compiles Python seamlessly to a stand alone executable, you still don't have the cross compilation story go has. And you have to be careful with libc.
But the thread is about highly scalable things, isn't it ?
Yeah in it's early days, the torrent network was small, and peer counts were limited. The original, and many later Python implementations used event loops, which side step concurrency implementation overheads (like threads), and very often do a lot of heavy lifting in C. I'm not sure that Deluge is Python for the torrent part. The standard is to use libtorrent these days.
Python can handle a torrent client with appropriate tools, but you just have to be extra careful about algorithms etc.
Deluge is really bad with more torrents. After 1000 it's basically unusable. Most people who do long-term seeding use multiple instances of Transmission or rTorrent. There are some manager tools to help with that, these usually also do the balancing.
I used to split them up with around 3000 seeding / client which worked fine.
Ah ok. That's a very specific use case though. Regular users won't do that. But i can understand that non asyncio python is not good for that specific usage.
You're associating the word scalable with the typical web-scale definition. Don't do this. The OP didn't specify what kind of scaling, and a torrent client has a lot of scalability (vertical?) in the way it manages connections and DHT.
It can scale to your all 4 cores and burn your battery.. kidding aside most bittorent clients handle hundreds/thosuands of connections and sparse files as the splits and number of running clients increase, so scalability is a concern here
I think it is not the definition of scalability. It just means it is resource efficient. Scalability usually means that a service can handle the load beyond a single node capacity, at least for me.
