Hacker News new | past | comments | ask | show | jobs | submit login
The Pirate Bay Will Stop Serving Torrents (torrentfreak.com)
295 points by llambda on Jan 12, 2012 | hide | past | web | favorite | 86 comments

Hey! I invented the magnet link, almost 10 years ago.

Great to see it still evolving and spreading, based simply on its loosey-goosey merits.

Finding this sort of thing always makes me stop short and realize, "Oh yeah, no matter how smart I think I am, there are people a million times more accomplished than me on HN." Not to make me despair so much as take care to avoid blathering the first thing on my mind before reading and considering what others have said.

The 'magnet URI' was merely the right suggestion, for a tiny sliver of shared convention, at the right time.

You can look at the original writeup – http://magnet-uri.sourceforge.net/magnet-draft-overview.txt – and find a goofy ad-hoc mechanism where local software interested in 'magnet' links listened for localhost web hits, in a certain port range, rather than fighting over the 'magnet' URI scheme handler. That could be seen as either a blathering kludge or a very early attempt at 'web intents'.

I don't know if any code other than my demo ever implemented that localhost-handler trick. But the main idea – of a generically-named URI that was descriptive rather than location-centric – had its own legs. It spread from software to software, even without any formal spec stewardship/revisions. A key step for its longevity, when 'magnet' was adopted to wrap the 'btih' identifiers for DHT-based torrent tracking, happened long after my attention moved elsewhere.

I'd like to think the continuing adoption was because the original draft, even though crude and incomplete, had the right mix of a familiar/defensible URI syntax, wide-open descriptor semantics, and a catchy name/icon.

Good ideas don't have inventors so much as they have stewards. Keep trying and one day such a thing will find you. :)

It's weird, I've been seeing it for years but I never understood what it is. I still don't really.

But when I click on the magnets things seem to download immediately. So looks like you built something pretty amazing. Congrats!

It's basically a URI that contains the torrent hash enabling a client to find peers through a decentralized network such as DHT instead of requiring a tracker. I believe you can append trackers and other info to the URI as well, but I'm not sure about that.

Note: DHT = Distributed Hash Table

The other two comments are (correctly) describing what a magnet link is in the sense of its implementation. On the off chance that you wanted a more abstract/basic description, a magnet link is defined by what you're linking to, as opposed to where it is.

This is kind of cool, because a file can change locations (how many times have you seen a dead link/File Not Found?), and with a magnet link, if you have a file that matches the description, you can be reasonably certain that you've found the right file (unlike the URIs you're familiar with, which can change the content stored at each location). On the flip side, actually finding the file from the description is a bit more complicated, but as you can see, it's still pretty easy for the user when implemented correctly.

It's better for TPB, because they're no longer telling you 'where' the file is, just what it 'looks' like. Legally, I'm sure that makes some difference.

AFAIK, it's a torrent for a torrent; a DHT implementation. The URI for the magnet link has all the information it needs to find peers and start downloading the data, the first datum of which is the torrent file.

Ups to you! I think (and hope) that we haven't seen the end of magnet yet. I wish the scientific community would start using magnet links for paper references, instead of archaic bibtex layouts combined with online publishers who charge 30 dollars without adding value.

When multiple downloads of the same file are available it's usually beneficial to see and select the one with the highest number of seeds. If DHTs become the norm, will it still be possible to obtain and use such data?

If I understand magnet protocol correctly, it doesn't matter. If two files are the same, they have the same hash, so the system (DHT) cannot tell them apart.

If the files are different (e.g. 256 and 320 bitrate mp3 files), then that is a completely different problem.

I should have been precise: how does one determine the number of seeds for a given piece of data e.g. when the same movie at different resolutions is available, that is, different hashes?

I think that when you use a magnet link it either includes the tracker (which is the part of BitTorrent that tracks the seeds) in the text of the link or uses a more sophisticated technology called a Distributed Hash Table.

uTorrent 3.1 added support for BEP33 (DHT scrape), so apparently the answer is yes. http://bittorrent.org/beps/bep_0033.html

Expect a knock on your door from the entertainment mafia's logic-focused lawyers.

> Perhaps even better, without the torrent files everyone can soon host a full copy of The Pirate Bay on a USB thumb drive, which may come in handy in the future.

I've been saying this for years: governments playing an arms race with hackers is like playing Whack-a-mole or cutting of heads of hydras. Every time you block one means of communication (starting in this case with Napster), a new, more decentralized, harder-to-combat protocol is going to emerge. Even if you're the MPAA and don't like copyright violations, you have to face that fact. That doesn't mean you (necessarily) have to throw in the towel and abandon the idea of copyright infringement altogether, but it does mean you need to start being creative instead of engaging in a direct legal-technological battle - an arms race.

I wonder how long it will take before the big players realize this and try to figure out a way to use this new playing field (the Internet) to their advantage, instead of trying to squelch any technological development so that they can cling to old models of payment and distribution. It was nice while it happened, but let's face it - we're past the point of no return. Even if they everyone hosted their own copies of TPB on their thumb drives and then they found some way to shut that down, I'm certain there'd be some hacker smart enough (like gojomo above) to come up with something that just makes things even less centralized, more difficult to track, and more difficult to shut down.

The big players are defeated by the new playing field. There is no "their advantage," there's just a major change in how we produce content.

It's away from exploitation of artists and the control of artistic message.

The production, the marketing the, distribution... EVERYTHING has changed for the music industry in about 15 years. They have no future unless the Internet is tightly controlled (which it cannot be.)

The RIAA and other major music industry players may have no future, but there is definitely a profitable future for a "music industry" populated by entities that are willing to adjust to the modification in distribution channels and consumption habits.

As it stands, the old-fashioned fogies are simply in denial and doing their best to prevent the marketplace from changing because they don't know how to use the new technology and they don't know how to succeed in the new marketplace. That doesn't mean that there's no future for the industry as a whole, it's simply the way entrenched players eventually die.

they don't know how to succeed in the new marketplace

Who does, in your opinion?

I'm with you on this one. It's less that the RIAA and major labels 'don't know how to use the new technology and...succeed in the new marketplace' so much as we're currently (still) in an in-between stage of market control.

Take for example Radiohead or Louis CK. Their previous successes allowed them to produce, release and distribute independently of big entertainment — but I don't think we're at the point where a small independent artist can effectively take their own music to market in a manner comparable to using a label. I don't have data to back this opinion, but high profile successes in independent distribution have been biased toward artists who were well-known and previously successful.

I don't think it is fair to compare Radiohead or Louis CK to that of a lesser known artist in this regard, because the bias is all in the media coverage.

Bands have been self releasing (outside of "big entertainment") since well before digital distribution, because signing to a label isn't always feasible (financially or creatively). Digital distribution makes just makes this even easier.

You only know about Louis CK & Radiohead because that is that the media covering. Digital distribution & artists self releasing material is happening and has been for a while, but no one knows because unless they are already into these lesser known bands & record labels. You have to think of it like this: "Lesser known artists can be more successful by releasing their own material digitally than they would by signing with a label." Or "This independent record label can lower their overhead and make things easier for their bands by embracing digital distribution."

" Internet is tightly controlled (which it cannot be.)"

Oh, it can be controlled alright. SOPA, PIPA today, tomorrow FIPA, DIPA and FuckInternetPA. One of them will pass, then more and more.

And then you're sitting on cable tv. Clicked that link? Invoice has been mailed to you. Clicked that link? House search. Clicked that link? Prison cell prepared. Thank you and good bye.

Oh, you will watch what you do online, you will not click on links without knowing before hand what is behind it. Good luck with that. You will not read just about anything, you will not write whatever you want, certainly not where ever. You will censor yourself before they come to censor you. They will produce and you will consume.

Sadly, you're probably right. It won't be long, 10 years tops, before the US is doing deep packet inspection and hijacking sessions as a matter of course.

This site (http://www.ghacks.net/2010/06/05/what-is-a-magnet-link-and-h...) explains what Magnet links are and how trackers have traditionally worked in the past.

The original article had a link to this (http://torrentfreak.com/bittorrents-future-dht-pex-and-magne...) which I found a bit more helpful explaining the concepts.

It is still fascinating how resilient not only Piratebay but also others such as Demonoid have become. A few years ago it looked like they were close to getting shut down. It is impressive how they are now able to withstand all the "forces" and Governments had to shift their focus towards the ISPs.

One drawback to magnet links is that you can not in advance see what files are there.

So if you only want to download some of them, you first have to wait for the magnet to download the torrent, then go back to it and pick the files you want.

A magnet link also makes it hard to check if the link you are looking at is a duplicate of what you have already.

With a torrent you can check the file size and compare to others.

There isn't really a reason the piratebay web interface couldn't display that info for you anyway.

If users only upload the magnet file, the TPB wouldn't even know how big the file is, unless their servers queried the DHT for the .torrent file and extracted the data.

Right, that's what they'd have to do.

Can someone fill me in on the first step a client (BT) would take to find its first peers? That part is a bit magic to me right now. What would be the first IP it would query and how would it know?

There are three main methods (in order of preference):

1. Use a cached list from a previous run. This is ideal in terms of load balancing, and also helps with bootstrap time if your IP hasn't changed, so all major torrent clients have such a cache.

2. Ask peers you're connected to via a traditional tracker and .torrent file. This is the next best thing, but can only be used if the first torrent you download is via a traditional tracker.

3. Use some bootstrap peers hardcoded into the torrent client. Has a single point of failure, but it works.

Each torrent client can use a different set of bootstrap peers, note - they're nothing special, just a DHT peer that has the bandwidth and CPU power to deal with bootstrap requests (which isn't a very high hurdle - you only get hit once, on the initial install, or if the client's been shut down for so long it no longer has any valid peers). Or the user might even supply their own, if need be.

Thanks for the information, this part was always a mystery to me as well.

To clarify, if someone has never downloaded a torrent before and starts with a rarely downloaded file, would the use of only magnet links mean they will probably not be able to find another peer? It sounds like unless you already have torrents downloaded, your list of possible peers to check for the file is very short.

If your client connects to just one peer you essentially gain access to the entire bittorrent DHT network. As a dramatic simplification, your request for the rare torrent would propagate until it found the peers seeding or downloading it.

But what if the users sharing the file are also connecting for the first time? It seems like if my friend wants to seed a file and I want to download it, it only works if we knows peers (who know peers who know peers who...) know peers of me. Is this correct?

In theory (assuming that there are links to everyone, and no way to have separate networks) (it's of course a theory only, but seems very likely that there are "massive users", who have files of "every kind", connecting the different groups), as long as you both connect to one who is connected, it will work.

Although this is quite the extreme case, and not really the purpose for Bittorrent to begin with, I guess. The main interest is load balancing of popular downloads, not finding the one person with the one file for you.

I've actually found torrents to be a great way to share large files with a friend. There's no invasive upload dialog or locking, no practical filesize limitations other than the free space on my friend's computer, you don't have to upload to a public-facing server before your associate can start downloading, port forwarding is handled transparently by UPnP, client software is user-friendly, easily available, and often already running anyway, and it's easy to scale if another friend wants a copy at some point in the future.

Of course, TPB switching to magnets doesn't really affect this. Torrent files are still used to provide information on the download, they're just distributed from within the P2P network instead of TPB itself. If you have a torrent file to share with an individual or small group instead of the whole world, there's no reason to stop using torrent files directly.

For a brief moment, the title sounded like TPB was going to blackout for "stop SOPA day". Wouldn't that be funny.

If the TPB were to actively oppose SOPA, it would just give the SOPA proponents more to work with - "Look, TPB is against it, so it must be a good thing!".

Brilliant point. Pirate Bay's best approach is to laugh at SOPA and make clear it will do nothing to stop them.

Actually, one day the placed a video instead of their logo created by multiple users with messages against SOPA, it was this one: http://www.youtube.com/watch?v=1w6GtwOvnWM

It is a good point, however, I think it would be celebrated by the target audience. People, or the "average person" who wouldn't generally visit TPB, or know what it is, will probably not really care about it's support, or even what SOPA is for that matter.

For those who visit/use TPB, I think it would be akin to a rally call.

So, what if you don't run the client on the same machine you're browsing TPB with, like people who have torrent enabled routers or VPS/seedboxes? I suppose you can copy-paste the link to your client, but clients which had automatic pick-up of .torrent files (like rtorrent) were nice because you could just drop them on a remote directory and have them be downloaded.

I wonder if I could write a small application just to download the torrent file from the magnet link/DHT to copy it to the remote server afterwards.

See the shell script in the top comment here: http://libtorrent.rakshasa.no/ticket/2100

I tested it, and it works as advertised.

For firefox integration, in about:config create a new boolean as:

  network.protocol-handler.expose.magnet := false
For chromium integration, you need to use Xdg-open.

I suspect it'll be easier to patch the torrent client to accept magnet links fed through a named pipe, or in a text file in the drop-off directory, if it already supports magnet links.

Why not set up an email fetch service on the remote box, and email magnet links? Then just throw together a wrapper that looks for magnet links in new email. Viola!

They announced that they were planning on doing this right when magnet links came out. I haven't used .torrent files since I discovered magnet links and I actually find they more convenient than downloading a torrent file.

I've been using mostly magnet links for the past year and haven't experienced any issues. If people understand that they work just like a normal link/torrent file, this won't make the process any more complicated. Hopefully this is another win in the long run, as links are harder to stop than files.

Only slightly as I understand it, since from a legal perspective the intent of someone providing providing links instead of files doesn't change, and that's what counts.

From a technical perspective, and as someone who doesn't understand how magnet works under the hood, I'm slightly concerned that DHTs might be easier to attack in an underhanded manner than an HTTP server would have been.

"Only slightly as I understand it, since from a legal perspective the intent of someone providing providing links instead of files doesn't change, and that's what counts."

Indeed, but a link can be posted anywhere a user can post text. Here, have Pioneer One:


I can see this leading to a block on many sites on all links starting with the "magnet"-protocol, if it's not in place already. It also highlights the issue with SOPA further. Had that series not been allowed to be shared, Hacker News would be in big trouble for hosting the link.

That would be a restriction of freedom of speech and Internet Freedom for blacklisting magnet urls.

Linux Distro excuse comes to mind.

A cynical person might say that the intent would also further kill free content creators, restricting the supply of content; consumers then see only paid content.

Sadly, that hasn't prevented censorship of "neutral" keywords before. See: https://torrentfreak.com/googles-anti-piracy-filter-110712

Wow, I didn't see this before. I can understand filtering certain terms for court requests, but voluntarily? It looks like Google is attacking distribution methods of content with such broad filtering. Probably to make compromises with the anti-Internet activists. Now judges can interpret it to mean that Google can happily censor any results for certain words as it has been done in other anti-Internet regimes. What scumbags all round.

Note that this isn't filtering the terms for search, just filtering them for auto-complete. As in, if I type "lord of the rings" it won't suggest "lord of the rings torrent" even if that is the most common or one of the most common searches beginning with "lord of the rings."

If you actually type "lord of the rings torrent" and hit return, Google will still do that search and return relevant results.

They are doing this because for many pieces of popular media, the torrent search was fairly high in the list of most popular searches, and they don't want to be seen as suggesting that you download the torrent. Likewise, they won't autocomplete obscene or porn related terms, but will still do searches for them.

There is a difference between banning a term entirely from their search engine, and deciding that it's not something that you want to suggest to people who haven't requested it, to avoid suggesting illegal acts, avoid legal trouble, avoid offending people, or the like.

My concern was mostly due to this statement:

"This is something we looked at and thought we could make some narrow and relatively easy changes to our Autocomplete algorithm that could make a positive difference, Cano added."

They believe in filtering broad innocent terms in them as a solution to copyright infringement rather than saying "No no, we can't filter broad terms, especially innocent uses of it. Not only does the language evolve, but the infringers can adopt innocent names like congress:<link to song>. We need to intelligently handle copyright infringement without hurting access to legitimate websites."

It looks like part of doing SEO now is to be aware of avoiding broad innocent terms. I hope the unaware people don't get caught in Google's broad autocomplete filter.

Maybe the next file-sharing protocol should be named `apple`.

I can't see that working - any more than giving your DVD protection scheme the same acronym as a common file format (CSS) to prevent people finding ways to defeat it.

My point was only that it couldn't be censored in the same way - if Google would remove Apple from suggest, it would clearly be viewed as a monopolistic/anti-competitive action.

"I'm slightly concerned that DHTs might be easier to attack..."

The key is in the D. Since it is distributed, there's not a single server that can be attacked.

That would be nice, but unfortunately it's not really the case. It's quite difficult to make a secure distributed system that allows anyone to join (as opposed to a closed darknet).

Here are a couple examples of attacks that DHT is vulnerable to:

http://www.cs.rice.edu/Conferences/IPTPS02/101.pdf http://www.cs.rice.edu/~twngan/doc/infocom06.pdf

Talking about magnet links I remembered KAD with ed2k and eMule. Its almost same. Bittorrent with magnet links just have different chunk size possibility to make faster download with smaller piece size.

According to Wikipedia, magnet links are in fact based on ed2k and Freenet URIs.

From what I understand, DHT has a big trade-off vs BitTorrent: DHTs are crawlable[1] and copyright holders can more easily track who holds copies of what yet it's easier to duplicate search engines like TPB within hours for the same reason.

So the effect seems to be that the RIAA, MPAA, etc. will likely not be able to take down trackers; they'll have to revert back to suing their "customers" (or lobbying to pass absurd legislation for that matter).

[1] http://z.cs.utexas.edu/users/osa/unvanish/papers/vanish-brok...

It seems that TBP just taken another step to push judge and jury's confusion during next trials even further. "Y'know, we're trying to shut down pirates' secret base that... doesn't serve a single file!"

What I’m wondering is whether it’s not yet possible to have a distributed, decentralised torrent database.

You could already put up a copy of the database as a torrent and distribute the magnet link, but you’d need some method for efficiently keeping it up to date.

DHTs/Torrents are great for static data (like a movie) but bad for dynamic data, like a website with a list of movies, their ratings, user comments, etc.

There are, however, other systems that are designed to combat this, like freenet. They tend to be overwhelmingly slow, because you need to pass lots of data around to make it consistent.

Then again, magnet links, titles, and a little html are probably not a lot of information, so it probably could be done. I just haven't heard of any attempts yet. It's tempting to go and write one. Could you make a decentralized, P2P version of reddit with distributed trust? I think it's possible but hasn't been tried.

I think the easiest way is to not make it really "dynamic" but an append-only structure of static content. For example a Reddit thread might be representable as a static list of actions (add reply, upvote, downvote, etc), which the client would process to get the current state.

That way, you could immediately download an earlier version of a resource and then get the updates as they spread through the network.

a la bitcoin.

They have a big problem in their design where currently it works exactly as you described, and at "some unknown point in the future" when the database of all transactions EVER in the history of bitcoin gets too big for each person to have to have a copy of in order to add another transaction, they'll "figure out a way" to make it unnecessary.

Can't they somehow publish the current state of things, let anyone who wants to verify it from the public record building a web of trust, then go from there forward?

Or is there some subtle flaw in that idea?

Freenet isn't really good for this either - it's basically static data, too.

For dynamic content there is I2P.

I hope that before .torrent files are gone, they (or some scraper) will publish them all as a torrent.

Somebody did that last time PB was in trouble, e.g. one of the pieces: magnet:?xt=urn:btih:4232363a47fe29acdf2c77874365a5e3368854b4&

That's a pretty interesting dataset to mine.

Why? What would the point of the .torrent files be if the magnet links are still there?

> This is topical, since this week courts in both Finland and the Netherlands ordered local Internet providers to block the torrent site.

Is there a list of countries where this has been ordered by courts? I know it's already the case in my country (Belgium) as well[1].

[1]: http://torrentfreak.com/belgium-starts-blocking-the-pirate-b...

Since they are doing this big change, is there a way to make it more secure on the user side, too? Like encrypt the traffic and make it impossible for RIAA to track IP's?

Also since they say that it's like every user would have the TPB site on their computer, does it mean blocking the site would be completely useless? And since they are just links, and links are pretty much speech, I figure it would be impossible to turn it into law as well, to specifically target magnet based sites like TPB.

The entire design of Bittorrent is predicated on clients advertising to each other which torrents they're seeding and which pieces are available. You can encrypt your traffic to get it past your ISP's deep packet inspection, but the peer at the other end has to be able to decrypt it. And you have no way of knowing what nefarious organization controls that peer.

Don't the torrent files contain all the hashes for each piece? Doesn't that mean if a single piece is bad, the entire torrent can't be verified? Torrent files contain a lot of useful data that isn't found in a magnet URI. Is this just being ignored?

The hash in a magnet URI is of the info dictionary in the .torrent file. It's used to query the BitTorrent DHT for peers running the same torrent. Once another peer is found, your client will download the full metadata file from it, and from there it works just like any other BitTorrent download.

http://bittorrent.org/beps/bep_0005.html http://bittorrent.org/beps/bep_0009.html

This is definitely a step in the right direction. Not saying this is a good step to increase piracy, but there really is no need for trackers when we have magnets, and this will lead to a healthier bittorent community.

How does this change the process for uploading a new torrent to TPB?

For the time being, nothing. However, once TPB goes full magnet you'll just submit a link instead of a file. It should make the process easier.

Can anyone say what the gist of this article is? Blocked from torrent sites at work.

Piratebay will now serve magnet links by default instead of .torrent files. For the average user, not much will change. For some particular use cases, there may have to be adjustments.

Pirate Bay is the BEST torrent site ever created. It's a pity they are switching to Magnet. :( It would be so cool if they provided both (Magnet and Torrent)...

Registration is open for Startup School 2019. Classes start July 22nd.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact