
Libtorrent adds support for the WebTorrent protocol - hauxir
https://feross.org/libtorrent-webtorrent/
======
mcjiggerlog
A while back I built out a proof-of-concept distributed web based on
webtorrents - [https://github.com/tom-james-watson/wtp-
ext](https://github.com/tom-james-watson/wtp-ext). Once this change makes its
way into popular clients, it makes the idea a whole lot more viable.

Unfortunately it looks like Mozilla's interest in the distributed web died
when they made redundant the main stakeholders in the project last year -
[https://github.com/mozilla/libdweb/issues/109](https://github.com/mozilla/libdweb/issues/109).

~~~
saurik
When I was last looking at the libdweb stuff they didn't have WebRTC support
working in the service workers to let you use it from a protocol handler (and
they somehow didn't even consider that important as they insisted no one would
even want to use WebRTC, which made no sense to me); did they eventually fix
that so you could do WebTorrent?

~~~
madushan1000
You can implement your own protocol + handler with the functionality exposed
by the library as far as I understood.

It doesn't really matter anyway since you have access anything firefox can do
with webextensions experiments, which was required since libdeweb isn't a part
of firefox(and won't be anytime soon because AFAIK most people who were
working it has been laid off) and to run experiments you need firefox nightly.

~~~
mcjiggerlog
Pretty much this. The code runs as an extension - it registers the protocol
handler in the extension's background script. That means the code for webrtc
has no need to run in a service worker.

~~~
saurik
OK, so as I don't actually do much in-browser development, and because this
was also two years ago, "of course" I managed to get the terminology wrong:
the issue was that WebRTC wasn't supported in a "WebExtension" (not a "service
worker", which happened to be the subject of a massive thread I was reading a
few weeks ago on WebRTC not working there either).

Here's the bug I had referred to the libdweb person at the time in November of
2018 (due to comment #2):

> RTCPeerConnection.createOffer doesn nothing in WebExtension background
> script

[https://bugzilla.mozilla.org/show_bug.cgi?id=1398083](https://bugzilla.mozilla.org/show_bug.cgi?id=1398083)

This bug had already been resolved in September of 2018 as a duplicate of this
bug:

> WebExtension cannot access WebRTC mic from popup.js nor background.js

[https://bugzilla.mozilla.org/show_bug.cgi?id=1278100](https://bugzilla.mozilla.org/show_bug.cgi?id=1278100)

And I now see that bug ended up getting resolved in September of 2019 as a
duplicate of this bug:

> Improve getUserMedia permission model for web extensions a bit.

[https://bugzilla.mozilla.org/show_bug.cgi?id=1579489](https://bugzilla.mozilla.org/show_bug.cgi?id=1579489)

Which was supposedly fixed? I'm honestly a bit concerned still, though, as
this bug slowly got dragged from being about createOffer to being about
permissions dialogs for audio/video, and it kind of sounds like there's some
weird thing where a "foreground page" has to vouch for the extension, which
shouldn't be required. Everyone who works on WebRTC stuff in browsers
conflates it all with getUserMedia, but all of us distributed web people just
want peer-to-peer data channels :/. I take it this is _actually_ fixed then,
and you can now use WebRTC in this context?

------
ThatPlayer
For some fun confusion, there's this libtorrent library, commonly called
libtorrent-rasterbar by distros. And libTorrent by rakshasa, which is used by
rTorrent.

I've considered something like this for a web music player I have, but it
doesn't seem to have an easy setup for downloading individual files from a
torrent. And the latency would add gaps between songs.

~~~
Scaevolus
You can use prioritize_files() combined with the sequential_download flag to
"stream" a track (and prioritize_pieces() if you need the last chunk for an
m4a file).

Schedule your fetches intelligently and it shouldn't be hard to keep a music
player fed. 320kbps is only 40KB/s.

------
joshspankit
Question: why did noone extend the torrent protocol to understand http[s] and
speak to servers that support resume?

edit: I’m just now reading
[https://www.bittorrent.org/beps/bep_0019.html](https://www.bittorrent.org/beps/bep_0019.html)
and that’s exactly what they did. My bad

~~~
ajconway
It is supported, it’s called web seeding.

~~~
black_puppydog
although from how I understood it, web seeding really assumes you can control
the folder structure on the servers.

so you can't just curated a data collection from publicly accessible sources
and freely map torrent contents to files on http for
bootstrapping/reliability.

too bad, really. it seems like a cool thing to just make a torrent from e.g.
public datasets that are hosted on flimsy university servers. make the
torrent, distribute it, the swarm keeps the files available and reduces load
on the university servers, and the server keeps the torrent from dying even if
the dataset is niche.

~~~
crest
There are two BEPs for web seeding. One is defined in terms of base URL and
relative paths the other in terms of pieces and offsets into them. Few clients
implement both web seeding protocol extensions.

------
adrianmonk
Kind of a tangent, but how large or small is the risk of nefarious sites
running torrent nodes without the user's knowledge or consent?

We've seen something similar with using JS to mine bitcoin. That gets you
money, so that probably has a stronger incentive, but someone might want to
gain access to the resources this would allow.

The WebTorrent FAQ ([https://webtorrent.io/faq](https://webtorrent.io/faq))
says this uses WebRTC and further says:

> _WebTorrent clients running on one domain can connect to clients on any
> other domain. No silos! The same-origin policy does not apply to WebRTC
> connections since they are not client-to-server._

Does the browser do anything to protect the user against something like this?
Is there a permission for network access, and is it allowed or denied by
default?

~~~
feross
> Is there a permission for network access, and is it allowed or denied by
> default?

WebRTC data channels are allowed by default.

> Does the browser do anything to protect the user against something like
> this?

A nefarious website can already use the fetch() API to upload or download
massive amounts of data if their goal is waste bandwidth. WebRTC doesn't
introduce any new vulnerability here.

------
dec0dedab0de
Im not up to date on webtorrent, but if it catches on does that mean a good
chunk of the Inter net could be freed from cdns?

~~~
toyg
unlikely. Torrents are slower, in most scenarios, than straight HTTP. With
small files, they generate more traffic for chatting/handshaking than the file
is actually worth.

It could however lower costs for people who distribute large files, where the
final user can tolerate some download latency. I'm thinking browser games that
can download a big archive via torrent on first launch, similarly to what
Steam does now in their client.

~~~
pantaloony
A large majority of web clients must be operating on battery now, too.
Willingness to burn battery participating in Torrent and other distributed
protocols rather than HTTP will be low among those, let alone willingness to
leave such a thing operating as a daemon in the background. It’s why I don’t
see much future for these things (distributed file-sharing protocols) except
in infrastructure or in applications where many users may be expected to be on
desktop.

~~~
toomuchtodo
Conversely, every home router or NAS can act as a node in the mesh.

~~~
bentcorner
With data caps I'm not sure if home users have an appetite for that either.

~~~
rohan1024
Someone needs to create an app so awesome that the only way to join it is to
contribute to it. Hopefully that will happen in my lifetime.

------
toyg
I expect enterprise IT will soon want a way to block this sort of thing, if it
becomes popular. It risks killing WebRTC for serious applications...

~~~
fsflover
One can also forbid Internet or just computers, because one can do a lot of
harm with them...

~~~
toyg
Considering SSH is forbidden in most enterprise networks I've seen, your
attitude is clearly in the minority.

Enterprise-IT people tend to shutdown first and ask questions later, to cover
their ass (and in their shoes I'd probably do the same). If it becomes trivial
to torrent via mainstream browsers, I expect they will lock down whichever
feature is responsible or apply massive pressure on vendors to remove the
feature.

~~~
ekianjo
> Considering SSH is forbidden in most enterprise networks I've seen, your
> attitude is clearly in the minority.

You can't really forbid SSH (on paper only you can) since you can create SSH
tunnels via virtually any port.

~~~
toyg
Of course, but they still make an effort (and a policy that will be applied to
you if found out). They have to, it's their responsibility. If suddenly half
the network is saturated by people using browsers to torrent the latest
movies, you can bet that the answer won't be "we need a bigger pipe"...

------
rwmj
I recently wrote a fun thing using libtorrent-rb:
[https://rwmj.wordpress.com/2020/06/25/nbdkit-with-
bittorrent...](https://rwmj.wordpress.com/2020/06/25/nbdkit-with-
bittorrent/#content) It's a way to download, install and upload Linux distros
all at the same time.

~~~
loeg
I follow your blog and saw that post a few days ago. I found it an
approachable intro to libtorrent-rasterbar — thanks!

------
trizic
This is great news and will help lead to wider adoption. I hope that some
progress will be made for handling large files since currently it is stored
in-memory:
[https://github.com/webtorrent/webtorrent/issues/86](https://github.com/webtorrent/webtorrent/issues/86)

~~~
ss3000
I've been using Webtorrent Desktop for a while now, and really enjoy using it,
but there are some glaring issues around random freezes on fast connections,
plenty of open issues on it with other users reporting the same experience:
[https://github.com/webtorrent/webtorrent-
desktop/issues](https://github.com/webtorrent/webtorrent-desktop/issues)

I've personally been working around this using a custom fork that messes
around with the simultaneous connections settings, and it seems to help quite
a bit, but I still get freezing from time to time when I download a large
number of torrents simultaneously.

I wonder if an alternative implementation in something like Rust on top of
WASM might make those kinds of issues easier to avoid by making more efficient
use of resources.

------
microcolonel
Once WebTorrent is ubiquitous among desktop and headless clients, the networks
will become a lot stronger. Especially for applications like BitChute and for
anyone who copies their approach. It solves a major problem with the cost of
baseline hosting vs. hosting at scale.

~~~
amelius
The problem is that many people will want to run bittorrent over a VPN
connection, while they'd like to run their normal browser data over their
normal (low-latency) internet connection.

~~~
Nextgrid
You're assuming torrents are intended for copyright infringement which is
unfortunately a big part of its usage right now, however if torrents become as
seamless as HTTP (like not requiring a separate client) I can see them
becoming widely used for totally legitimate purposes.

Torrents can be a great, standardized way of delivering updates for example,
and network administrators can easily deploy a platform-agnostic way to
deliver & cache updates on their network by just deploying a torrent client
somewhere and their fleet would automatically discover them (using local peer
discovery) instead of having to deploy multiple proprietary, platform-specific
solutions.

~~~
olah_1
> You're assuming torrents are intended for copyright infringement

I think they're just assuming that you don't want everyone to know exactly
what you're doing online at all times by exposing your IP to everyone and
their brother that uploads a resource.

~~~
gsnedders
This. It's a privacy nightmare. It essentially broadcasts to arbitrary other
internet users find out what resources you've accessed. That's… pretty dire.

------
lovasoa
When opening the javascript codepen demo linked in the article, the video
plays immediately, but I noticed HTTP GET requests to
[https://webtorrent.io/torrents/Sintel/Sintel.mp4](https://webtorrent.io/torrents/Sintel/Sintel.mp4)

Is this normal ? I know nothing about WebTorrent, but isn't the file supposed
to be streamed from pairs, and not requested from a central server in an HTTP
request ?

~~~
xd1936
One of the Webseeds[1] specified in the Sintel torrent is that URL. That HTTP
address basically acts as another peer in the swarm.

1\.
[https://www.bittorrent.org/beps/bep_0019.html](https://www.bittorrent.org/beps/bep_0019.html)

~~~
techntoke
And leaks your IP address and makes it possible to pirate content for simply
visiting a website. You could target someone with all kinds of illegal stuff.

~~~
anyhooo
It's already possible for a website to get your browser to download illegal
content simply by visiting the page, and expose your IP in the process to
whoever they want. You don't even need to use JavaScript to do the downloading
(just a image or video tag, for example), and the uploading can be done with
xmlhttprequest/fetch.

------
black_puppydog
I'm torn on this one.

On the one hand it means the web moves (or can move) closer to decentralized
modes of operation. Bittorrent is tried and tested and Works (TM).

On the other hand, it also means yet one more thing that moves into the
freaking browser. We have enough things nowadays that are shoehorned into
HTML/JS when really they would have been more usable even if written in
tcl/tk.

You win some, you lose some I guess. ¯\\_(ツ)_/¯

~~~
bastawhiz
Alternatively: every bittorrent client I have used (at least on macOS) is
really, really bad. I used to use Deluge, which was buggy and slow to begin
with. Then they stopped distributing an installer for it, so you had to
install it with pip (!) and run it from the command line. God forbid I `brew
upgrade python` or my bittorrent client will probably break.

Transmission crashes for me, and has little activity. Sometimes it just can't
open torrent files.

The web doesn't break. You can load a URL, get some decent software with an
easily-improvable UI—and assuming you're on a modern browser—expect it to work
with no installation, in seconds. And if a site pushes an update, I hit
refresh and get the update without needing to run a 45-minute `pip install`
command.

~~~
black_puppydog
fwiw, I've used qBittorrent on osx and found it to be usable. I'll admit that
was a few years back, so maybe they (read: apple) broke that one in the
meantime.

Oh, also, coming out of an hour long intercom session trying to explain to a
customer rep how "no, I didn't change anything in my firefox, your site just
suddenly doesn't log me in any longer" I'll posit that the web breaks all the
time and all over the place, in many small, annoying ways.

------
ghego1
The amount of word feross has put into P2P systems over WebRTC is astonishing.

Thanks feross for your incredibly great work!

------
mixmastamyk
Is there a good book that explains all the BT technology? I’d like to use or
build a robust client/server but most learning materials seem to be trivial or
daunting.

~~~
feross
Not a book, but I gave a talk at JSConf Asia in 2014 where I explain
BitTorrent (and WebTorrent) from first principles:
[https://www.youtube.com/watch?v=kxHRATfvnlw](https://www.youtube.com/watch?v=kxHRATfvnlw)

I also recommend just reading the BitTorrent spec. It's quite short and
manageable as far as specs go.

