Hacker News new | past | comments | ask | show | jobs | submit login
WebTorrent now works in the browser, end-to-end (twitter.com)
263 points by MzHN on Sept 15, 2014 | hide | past | web | favorite | 83 comments



"instant.io: drop a file or files on the browser, it generates a torrent, and you become the first seeder just send the info hash to a friend and they'll download the file from you using the bittorrent protocol -- swarming works (so multiple people can download efficiently)"

https://botbot.me/freenode/webtorrent/2014-09-11/

(wasn't obvious to me what this was...)


This is a very cool project but it is not compatible with current bittorrent clients. It uses webrtc for the transport instead of UDP. In order for it to be compatible with traditional bittorrent clients, those clients would have to implement communication with webtorrent clients, which hopefully shouldn't be too hard, still as of now it won't work with current clients. Still very cool though and I hope traditional clients will implement webtorrent. The other way to make it work would for browsers to implement UDP which some do such as chrome but only as chrome apps.


Nitpick: traditional bittorrent sits on top of TCP. utp, the protocol to make torrents run smoother, does run on UDP, but it's not the official way to do it (although most clients should do it)


uTP isn't really to make torrents run smoother. It's to allow NAT traversal.


Actually (again, nitpicking because I love Bittorrent) the reason it was created is to allow smooth day-to-day operations of a computer with a long-running bittorrent client and several other shorter-terms network applications, such as web browsers, chat applications, VoIP softwares, multiplayer games...

The goal is to make sure that bittorrent automatically takes full control of the interface when nobody else is using it, yet quickly gives control to whoever needs it when needed, without manual intervention from the user (the notorious "set speeds to 10% to not slow down your web browsing").

That said, uTP being based on UDP gives the nice side effect that NAT traversal is effectively possible.

http://bittorrent.org/beps/bep_0029.html


Yes, this is a typical feature of delay-based congestion control. Especially, when cross-traffic increases the delay because the change in observed queue size.

uTP also implements LEDBAT, which is a scavenger class of congestion control. See CC implementation details: https://tools.ietf.org/html/rfc6817


uTP was developed to work around bufferbloat-challenged DSL/DOCSIS CPEs, smoother is a reasonable characterization for it. It has since gotten some NAT traversal abilities as a bonus.


So it should work with Chrome and Firefox, correct ?

But I though webrtc still needed dedicated servers to punch through NAT ?

http://www.html5rocks.com/en/tutorials/webrtc/infrastructure...


Could you make a 'bridge' somehow between the two (browser extension?)


> WebTorrent is designed to match the BitTorrent protocol as closely as possible, so when the time comes, existing BitTorrent clients can easily add WebRTC support and swarm with web-based torrent clients, "bridging" the web and non-web worlds. [1]

[1] https://news.ycombinator.com/item?id=8280436


For Firefox, probably. I don't think Chrome extensions allow you to use sockets.

EDIT: confirmed: https://code.google.com/p/chromium/issues/detail?id=152875


Feross actually gave a talk about this where a lot of these questions got answered.


How hard would it be to make a torrent:// protocol extension that allows web like content to be stored in torrents? Obviously that would only work for static content (unless the creation of such torrents could be used to preserve state) but this seems like an interesting development.

Decentralized hosting of content that can easily be browsed seems like a useful thing to have.


In terms of accessing small resources (HTML pages), a client-server setup is by far faster than a P2P system. The Web is client-server specifically for this reason. Not to mention that the whole power of a client-server setup is the ability to have clients alter the state of the server. With a P2P system, propagating changes and making sure each peer has the correct copy, requires more time.

Check out this paper to learn more about the Web applied to P2P networks [1].

Li, Ruixuan, et al. "WebPeer: A P2P-based System for Publishing and Discovering Web Services." http://www-inf.it-sudparis.eu/~tata/papiers/sellami/P2P_Disc...


If P2P were a "first-class citizen", the initial handshake wouldn't take longer--it would be faster!

Suppose that my city's newspaper (or TV station) monitored the (socially accepted, widely adopted) P2P networks for reliable mirrors of their own content, then sent each user's browser hrefs to the mirrors, rather than their own site.

The more popular the content, the more likely that your next door neighbor's P2P system can stream it to you.

Since running such software is socially acceptable in this hypothetical future, your P2P software doesn't have to go through the 'discovery' step. One of your hostnames is 1234.main.st.ci.lincoln.ne.us (see http://owen.sj.ca.us/~rk/howto/usdomainname.html) and one of them is mirror5.journalstar.org.ci.lincoln.ne.us. Everybody on Main St already knows your IP address, and if you mirror a lot of content, they might even have an existing active connection... you see where this is going? :)


But a hybrid server/P2P system has multiple potential benefits (P2P acting as a CDN for common libraries, sharing live video or simply uploading 'unhostable' content).


Client-server is faster if the server's upload speed can match the client's download speed, for all clients. Which implies a large, central server.


There are other parameters that are useful when evaluating a configuration besides just speed. Resilience is one of those, as is long term availability. Torrents seem to do quite well in the resilience department, I'm not sure how well they do when it comes to long term availability. Maybe someone has already studied that, it would seem to be the torrent equivalent of 'link rot', and quite possibly it is very high.


Sorry, what I meant by speed is: the ability to quickly access small sized resources such as HTML pages. The initial handshake required for P2P systems makes this process slower. Obviously for larger files where there are lots of peers serving the content, you can get better speeds.


Latency is also a factor.



Something like this already exists in the form of SyncNet.

http://jack.minardi.org/software/syncnet-a-decentralized-web...



A few years back, I saw a proof of concept on PlanetKDE using magnet links in Konqueror to browse. I was unable to find it again.


https://web.archive.org/web/20120213102248/http://whilos.blo...

Unfortunately, I couldn’t find the video. It worked really well though. You could just open the magnet and drag-and-drop copy the files you wanted, or just open them and they’d start streaming.


I'd be concerned of this becoming an additional security risk. You just know someones gonna find a way to inject malware into it. On a home network it's not a big problem, but on a corp it'll be open season.


Surely this isn't a problem with checksums/hashes to verify the integrity of the files?


Which in fact Bittorrent already does as a core part of the protocol.



Also some further details on the instant.io + whiteboard app

https://botbot.me/freenode/webtorrent/2014-09-11/


While that's a neat tech toy I don't think the browser is a good platform for a torrent client.

A significant drawback is that you'll have to keep a tab open for seeding. And no DHT support either.

BT clients are best operated in a daemon-like fashion with access to system facilities such as inotify, udp and tcp sockets alike and as something that does not compete for CPU time/garbage collection/file descriptors/etc. resources with interactive web browsing.


Perhaps you're describing a Browser Extension?


What would be the benefit of a native code extension over using a standalone application with some URL handler / file type integration?

I don't see the advantage of "do it in the browser" here.


What would be the benefit of a native application over using a browser extension?

I don't see the advantage of "do it native" here. Nor do I see the advantage of doing in the browser. They both seem like perfectly good places to write a BitTorrent client. And I can certainly imagine some interesting cases where I would want to allow my site users to send traffic between each other over a well known, advanced P2P syndication system (rather than cobbling up my own).


>They both seem like perfectly good places to write a BitTorrent client.

Except they aren't. Web APIs don't provide the facilities necessary for a complete bittorrent stack. TCP and UDP sockets are essential (this thing uses webrtc, so it's not even bittorrent-compliant). Additional facilities like mmaped files, epoll-based socket IO and inotify make for more efficient implementations. A good bittorrent client on a gigabit connection can push a lot of data around, hashcheck it and so on.

Some torrents also have thousands of files open simultaneously, this can push the number of filehandles the browser process has to keep, possibly conflicting with other files the browser also wants to open, e.g. its cached files.

You also need to talk to local network devices (UPnP routers), enumerate network interfaces to bind ports to the correct one so you can tell other clients your alternative addresses (this is part of ipv4/ipv6 compatbility) and a whole bunch of other sort of low-level stuff.

Browsers simply do not provide the same access to system APIs. They are not a libc-replacement.


A) you don't need a complete bittorrent stack

B) you don't NEED a more efficient implementation

C) many torrents do not have thousands of files open

You're confusing "is a complete replacement for bittorrent" with "has uses." This definitely has uses.

"Worse is better."


>Web APIs don't provide the facilities necessary for a complete bittorrent stack. TCP and UDP sockets are essential (this thing uses webrtc, so it's not even bittorrent-compliant).

While it's only available to privileged and certified apps, Mozilla's TCPSocket Web API[0] gives you (unsurprisingly, given the name) raw TCP sockets.

The W3C also has a TCP and UDP Socket API[1] under development.

Obviously, exposing raw TCP/UDP sockets to the open web is a Bad Idea but Web Apps are more than that.

[0] https://developer.mozilla.org/en-US/docs/Web/API/TCPSocket

[1] http://www.w3.org/2012/sysapps/tcp-udp-sockets/


Chrome also a set of raw socket API [1] covering UDP, TCP client, and TCP server sockets.

[1] https://developer.chrome.com/apps/app_network


Sounds like a whole lot more bellyaching. Why so serious? Dial down your zeal friend.

BitTorrent is extensible: I see you agree with that premise since you listed UDP sockets, formalized by BEP-0029. The objections you level against WebRTC DataChannels make it sound like the particular transports you mention are somehow blessed, that it's not and won't ever be bittorrent unless it's TCP or UDP (and even if it was a hard/fast requirement, one could still benefit from WebTorrent via Firefox extensions and Chrome apps).

Who cares if you're using mmap, epoll, or inotify? Are you intimate with the performance limitations of IndexedDB in all browsers? How versed are you on the performance hit taken by File API? When do these limits become problems? Why would a browser need to keep open a filehandle for every "every" in the first place? That's certainly not at all how https://github.com/js-platform/filer works, on either it's IndexedDB or it's WebSQL backend.

As for your networking dilemmas, I recommend you check out: The network discovery api (which can enumerate UPnP and DNS-SD), WebRTC's TURN/Stun for correctly choosing from available addresses, and a whole bunch of other web stuff that you obviously don't know anything about but think is explicitly the domain of Real Serious Programming With Real Applications.

Not everyone needs to be operating at the fantastical extremes you are concocting, not all use cases are gigantic seedboxes on fat pipes. It's ok to deliver capabilities that someone else might already have, and maybe not even have some quirks or possible points of poor comparison.

You're throwing a whole bunch of technical minutia at the problem you want to have. You're picking a false fight (and one that the web has plenty of good repostes you didn't know/omitted to retort with). The web might not be the best choice for using a gigabit ethernet connection (or maybe it actually does it fine in some conditions!): the act of finding these particular points to dive deep deep into as an excuse to ignore, shun, and spit upon the expansion of human technical capabilities is a close-minded injustice. It's bigotry, simpleminded technical bigotry showing off your inability to open your mind to possibilities, unwilling to open your arms to new improved things becoming available to people who want them and will benefit from their use, it's your own intransigence in the face of the ongoing adaption and mutation going on in the world. You're in for a bumpy ride if you are going to cling so tightly to your notion of what things are for and how they ought be.


It doesn't need to be a native code extension; that's not what I'm talking about. In Chrome, you can write browser extensions that are essentially off-screen web pages (in simple HTML and JS).

Two advantages of "do it in the browser" are, in theory it would run on every platform that browser is installed on with no additional run-time required (like Python or Java), and two, it could run on Chromebooks. Say, in Incognito mode.


But in case of a browser-extension you just substitute "python" with "chrome" as a runtime.


Yeah, you're right. Do you want to compare the install base of Python versus that of browsers like Chrome and Firefox?


More accessible, cross platform with little headaches, people will be more likely to trust it than native binaries.


Seems similar to PaddleOver [1] which was developed by Bittorrent inc a few years ago using their Btapp.js [2] library, unfortunately the PaddleOver site has expired and the code base hasn't been updated in a year.

It would be interesting to know if the developer looked at btapp.js and if so, why did he decide not to use it? Getting this technology web embeddable will go a long way to start making peer-to-peer a convenient method of downloading content vs directly downloading from a server, there's always going to be further hurdles to overcome with reliability in a case where the swarm lifetime is very short, but seeding with a few servers can help in that situation. If a web app could upload the torrent and data to a server to seed, then replicate it in a few locations it could be a decent DCDN.

[1] http://pwmckenna.com/2012/06/29/making-of-paddle-over.html

[2] http://btappjs.com/


Btapp.js is just a JavaScript API that acts like an interface to their closed-source, Windows-and-OSX-only browser plugin Torque: https://github.com/bittorrenttorque/btapp/issues/11

Something that is done purely with cross-browser compatible web technologies is far more interesting and promising.


Thanks for the clarification, additionally I could only find a Chrome plugin for Torque, which should be compatible with other torrent clients out of the box. However I can definitely see the advantage of using only web technologies. If anyone's interested I found the paddleover example on github:

https://github.com/bittorrenttorque/paddleover.com


Does it mean we can finally have web-based popcorn time?

This is REVOLUTIONARY


Finally someone who gets it. Ideally, one could run YouTube on a $5 DO droplet.


This looks fantastic for an idea I've been kicking around: Using BitTorrent as a way for clients to upload multi-gigabyte files to a website without using FTP.

The idea is to leverage BitTorrent's robustness in the face of random disconnections and reconnections as well as the easy-to-use throttling options that most clients expose.

The server would run a special BitTorrent client that never seeds, to prevent malicious users from using the server as a gateway to redistribute content.

So, rather than the traditional cloud of people seeding and leeching, the only participants would be one client seeding, and the server leeching. Once the transfer completes, the user could disconnect and the torrent would cease to exist.


Sounds more like rsync over WebRTC.


I can't get it to work, hope this helps:

Firefox 32:

ICE failed, see about:webrtc for more details

Nightly 35a1:

Mandatory/optional in createOffer options is deprecated! Use {} instead (note the case difference)!

setLocalDescription called without success/failure callbacks. This is deprecated, and will be an error in the future.

ICE failed, see about:webrtc for more details

this._pc is null bundle.js:25705

Chrome 37:

Uncaught, unspecified "error" event. bundle.js:9134


Is the info hash different from a magnet url? can they be translated from one to the other?


As i understand it, a magnet URL is a way of transferring an infohash as a URL scheme, so that it works with other programs. For example, the following magnet URL:

magnet:?xt=urn:btih:e666231c9a34be278f6cfc390099e4084b30e023&dn=The.Giver+2014+720p+REPACK+HDRip.x264+AC3-JYK&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80&tr=udp%3A%2F%2Ftracker.publicbt.com%3A80&tr=udp%3A%2F%2Ftracker.istole.it%3A6969&tr=udp%3A%2F%2Fopen.demonii.com%3A1337

contains first the protocol (magnet:) which tells the OS that this is a magnet URL so give it to some app that can handle that. Since it's in URL form it's all broken down into & separated key/value pairs, so next part:

xt=urn:btih:e666231c9a34be278f6cfc390099e4084b30e023

is the infohash (actual infohash is e666231c9a34be278f6cfc390099e4084b30e02, the xt is just the key for that value, urn:btih probably means something, google it).

The other ones are pretty self-explanatory, the name of the torrent (presumably so the client can display something pretty before it has downloaded the metadata) and some trackers


> urn:btih probably means something, google it

It's a Uniform Resource Name (hence "urn"), btih (BitTorrent Info Hash) is the namespace identifier and the rest is the namespace-specific string, the actual infohash.


Thanks


Please edit or delete this comment. It's stretching the page.


Is the code available on GitHub? I just saw tge demo on my mobile browser.


Why doesn't it work with magnets, too?


Wow!


It's nice to see BitTorrent used for file transfers (user to user). Strange that it is still hard to get files from A to B. You can put them on a webserver if you have one, or use dropbox, but the download can only start when the upload is finished. I usually use scp for this, but it only works when one side is not firewalled, and isn't feasible for non-technical people. This one could be really useful.

It's sad that BitTorrent is otherwise basically dead (as a way to obtain content). Where I live (Germany) you often get immediately subpoenaed (or the German equiv.) if you use it to download something.


> Where I live (Germany) you often get immediately subpoenaed (or the German equiv.) if you use it to download something.

That's absolutely not true. You can use BitTorrent as much as you like, there is absolutely no danger in doing so. If you commit copyright violations however, you might get caught regardless of how you did it.


Well, I thought it was extremely obvious that BitTorrent by itself is perfectly legal, and that it is of course not banned in Germany.

Of course you only get an "Abmahnung" if you use it for copyright violations. But that is by far the largest use case. It can hardly be ignored that most people, especially "non-technical" ones, associate BitTorrent only with illegally downloading media.

What I wanted to say, is that now - as opposed to a few years ago - the probability of getting into trouble when downloading popular episodes via bittorrent has approached ~1.


As I said on another comment to you, you better include "public" somehow in your rhetoric. The probability of getting into trouble when downloading popular episodes via private bittorrent trackers or on I2P is ~0, not that I would do or advocate such thing.


Well, I wouldn't trust private trackers too much, because if I can get access to such a tracker, then the piracy hunters can as well (with private I assume you don't mean really private between friends, but rather trackers you have to register for, or get invited to). I can't comment on I2P because I haven't tried it, I didn't know that it works with decent speed.

My point was basically, the old approach of Joe Average Pirate of 1) installing µTorrent and 2) googling for "Big Bang Theory S05E01 torrent" is as good as dead.

You're right I was a bit sloppy, but that's moot now since I can't edit my post anymore and will probably continue to be downvoted until the text is completely invisible :-(


Out of interest, could a network provider actually tell the difference between say the data to update warcraft (which at least used to use a torrent style download), and the data through uTorrent of a textbook, and so on.


It's easy for the hollywood copyright holders to just load the .torrent/magnet link into a bittorrent client and look at the list of peers.

Now of course, you can make it difficult to join the torrents - for example, by using private trackers - but that makes it as difficult for me as for the hollywood copyright holders.

Then just IP address -> Court order -> ISP -> Your home address and send the threatening letter.


You need deep packet inspection [0] of the traffic. You'll probably need in addition some signature and algorithm to distinguish between different type of content, but this is probably doable for unencrypted content.

http://en.wikipedia.org/wiki/Deep_packet_inspection


> It's sad that BitTorrent is otherwise basically dead (as a way to obtain content). Where I live (Germany) you often get immediately subpoenaed (or the German equiv.) if you use it to download something.

That's just plain wrong. Maybe if you are downloading the latest hollywood flick from TPB you'll get a "Abmahnung" but that's about it. It's nowhere near "dead". If you are not using one of the big public trackers you won't have a problem. There are also free templates to fight these letters.


do they attack you for using btsync between your own PCs?


Of course not, nobody's attacking anyone for using a protocol. The only group targeted are people downloading certain movies or music where the company owning the content instructed another company to grab the peerlist of said torrent (in this case) and to get the info on the peers so they can send them scary letters extorting money.


I live in Germany but this is the first time I hear about this.

The only rule might be: do not down load content ( music) from Sony/BMG because they actively monitor the biggest torrent sites/downloads and might get a letter from their lawyer ( which you can ignore basically).

Movies and music from other labels are relatively safe. I don't download anymore since with 7 euro montly ( two beers) I have Google all access, while spotify is free from mobile.


Wow, I've never heard of this before. I am using torrent as we speak.


I have to say, there's only a problem while illegally downloading stuff (tv shows, films, ...) of course. BitTorrent itself is perfectly legal, and I've seriously used it many times to download Linux ISOs and similar. Also technically, it is the uploading that bittorrent does that is illegal.

The other day I made the mistake of downloading one of the last How I Met Your Mother episodes via torrent (the first time in years). Two weeks later I had a nice invoice over several hundred Euros in my mailbox :-(, and I know a couple other people to whom happened the same. That's why I wouldn't touch Popcorn Time with a ten feet pole.


Can you scan that letter and post it? That's interesting reading (of course you should redact all the details about your identity).


I don't have a scanner ready, but it was a pretty standard letter. If you search for "Abmahnung Waldorf Frommer" you should find some cover letters (in German). Waldorf Frommer is one law firm that is well known for sending those letters.

It was basically one page explaining that one violated copyright, giving the file name, the date, and the IP address; then a few pages detailing the appropriate laws, explaining how utterly bad piracy is, and telling you how much trouble you are in. Then there was an invoice for a one time "license" for the episode, and the legal fees which were more than the license. Following that, there was a kind of contract you had to sign, agreeing to accept the license, and in case of repeat violation, agree to pay a much higher fee. If I remember correctly, there even was a filled out wire-transfer form you just had to sign and bring to the bank in the end.

If you didn't accept their conditions, and refuse to sign, then they would go to court, which is extremely risky. You might get exonerated, or you might have to pay a lot more, so in the end I just payed the few hundred euros.

Given that the reason I pirated in the first place was that I'm a student and currently can't really afford to buy the media, its especially frustrating...


Wow. I would have never ever paid.

But I can see why such a strategy would work on their part.

I wonder how many of those letters they send out and how many people stand up to them in court and what the outcome is of those cases.


Yeah, the recommendation of many lawyers is not to pay or sign. Rather, you get a lawyer, and sign a "modified" cease-and-desist letter in which you 1) admit to no guilt and 2) don't agree to pay anything. By law, you have to react to their letter in some way. If you don't pay, there is a 50/50 chance that they'll let it be, or that they'll take it to court. I'm not really sure how your chances are in court.

Recently, someone argued that their router had a known vulnerability, and the rights holders couldn't prove that their WiFi was not hacked, and that they downloaded the files themselves, so they were acquitted. (http://www.heise.de/newsticker/meldung/Filesharing-Klage-weg...)

In my case, there were two factors that made me pay: First, they were in the right. As annoying as it might be, and whatever my positions on "intellectual property" are, it's hard to argue against that. Secondly, the cable connection belonged to a friend, and I didn't want to expose that person to a couple years of legal uncertainty.

If a cease-and-desist letter is unjustified, or in a grey area, I would definitely stand up against it, because I believe I'd have a good chance in court.


> Secondly, the cable connection belonged to a friend

Sounds like the perfect legal defense that an IP address does not identify you.


What would you have done (in by that, I mean "what do you advise normal people who aren't well-known, don't have a good knowledge of technology, and don't have a ton of money, to do")?


I would have definitely gone to court arguing that my IP address and me are not the same entity.

On top of that I would have used the (firm) German privacy laws to use the discovery process to figure out if they had obtained my (private!) name, address etc in a legal way.

To me this seems like a racket and I highly doubt that the case would have ever gone to court.

I would expect an outcome like this:

https://torrentfreak.com/evidence-against-bittorrent-users-s...

And without a ton of money the German state would likely appoint a legal representative.


great thanks!


You probably used a public torrent which means that you advertised the fact that you downloaded it publically to the world.


you might be interested in torrent-webstorage services such as put.io then.




Registration is open for Startup School 2019. Classes start July 22nd.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: