
Introducing The Paper Bay - david927
http://www.jacquesmattheij.com/introducing-thepaperbay
======
jpallen
Could you allow the requester to provide a link to the pay wall where they get
stuck? People who have access to the paper through a university subscription
could then easily click on that link and then immediately be taken to the
paper. I imagine most people looking for a paper will already have come across
this paywall and so will know where to find it. It would significantly reduce
the overhead of the person fulfilling the request, but may be against the
spirit of the site.

In many cases that I have come across, the DOI can be turned into a link that
takes you to the article. Perhaps that would be enough.

~~~
kerneis
> In many cases that I have come across, the DOI can be turned into a link
> that takes you to the article. Perhaps that would be enough.

This would be a very simple but extremely helpful change. You only need to
prepend <http://dx.doi.org/> in front of the DOI, and make the resulting link
clickable.

~~~
jacquesm
Ok, I'll add that tonight.

edit: done.

~~~
mHORxaq
You could also add the various other efforts in Open Access domain that were
linked on this page to your <http://thepaperbay.com/others/> page. There was
at least Pirate University, articleak, and /r/scholar.

Also, while it's okay to have a somewhat complexe form to fill when making a
request, it's really sad that it's as much work to share a paper, it should be
very easy, in particular when clicking "I have this, send it", the form should
have only one field: the file upload, or at least have the other prefilled
with the request's values.

------
roel_v
With the exception of old and obscure papers (i.e., 15 years or more and not
cited in the last 5 years), I don't really see what the big problem is. I have
yet to mail an author and _not_ receive a pdf of the paper, usually within
days, sometimes within minutes. When you have a reference and Google, you're
seconds away from finding that author's current affiliation & email address. I
once emailed the author of a book that retailed for 150$+ on some Elsevier
subsidiary to ask for a few pages. Turns out the book was basically her PhD
thesis (many, many books are like that) and she send me the pdf of that thesis
a few hours after I emailed her. All this brouhaha over access to papers is a
storm in a glass water if you ask me, instigated mostly by those who don't
even need the papers but rather are looking for a big meany to pick an
ideological fight with. Maybe it differs per field, I don't know.

~~~
Toenex
I think I'm right in saying that by emailing you a copy of the PDF, many
authors are breaking their contract with the publisher. It's a few years since
I published an academic paper but I've certain signed over my rights more
times that I care to recall. So while you can get a copy of the paper from the
author officially that is not allowed. Thankfully most academics are not
dickheads about this.

The key problem with journals is that much of what they do has been made
obsolete over the last couple of decades by the move to electronic document
distribution and the wonders of the WWW. Don't forget we are talking about
private companies restricting the publication of academic research much of
which has been funded out of the public coffers. Even the peer review is done,
for free by academics. All that is left is the management of the process and
distribution of the publications both of which I'd wager could be handled by
the communities themselves as beautifully and repeatedly demonstrated by open
source software projects.

What the journals still retain is essentially kudos, but in a very real sense
as academics are largely measured against their publication performance of
which the 'currency' is the impact factor of the Journal. Academics need to
publish in those Journals with a high impact factor so that their departments
get a good rating and thus get more government funding (at least here in the
UK). The key will be for a community lead Journal to attain a decent impact
factor. Soon.

~~~
roel_v
Sure, but the kind of 'paper sharing' services such as proposed by the OP have
the same problem. Many journals do allow you to share your own article,
although I don't know the details by hard nor would I even know where to start
looking for the conditions of publishing of my own papers.

Your last two paragraphs illustrate my point, or rather the myopic viewpoint
that these 'paper liberators' advocate - nobody outside of the HN crowd _I_
know (and yes, this is limited to a few fields) even _cares_ about this
supposed 'stranglehold'. Well everybody grumbles every now and then, but at
least nobody cares _enough_ to really push for change, and the things people
grumble about (slow editors, idiotic formatting requirements, ...) are mostly
not solved by 'open access' journals. Everybody has access to their
universities' libraries, or contacts the authors themselves, or has some
research assistant dig them up for them. Yes yes the subscriptions cost the
universities real money, but come on, much more money is wasted by
universities on other trivial stuff. Look, I'm not saying the model is
perfect, it's just that (IMO) it's not a big enough problem for actual
researchers (as opposed to 'interwebs poasters with strong opinions about
things they have little use for') to really matter.

~~~
Toenex
I admit I'm not saying that ThePaperBay necessarily offers the right solution
but I am refuting your claim that there isn't really a problem. The issue here
is that there has always been a problem. Academic output should always have
been available to anyone who wanted it _at a fair price_. Until electronic
documents became so ubiquitous you could tollerate the constraints of
publisher ownership because they helped overcome real problems of
communication and physical distribution. The question is haven't we now
reached the point where these problems have gone away?

------
mHORxaq
I submitted articleak, which something very similar a few days ago, but it
didn't get upvoted. Maybe people on Hacker News do not like Tor?

The idea of using Tor is that the site is way harder for authorities to get
down, and even if they do break Tor and get to the server, the users will
still be entirely protected (there can't be any information about them in the
logs or anything).

link: <http://articleak.allalla.com/>

submission: <http://news.ycombinator.com/item?id=5175234>

~~~
SkyMarshal
Whether you get upvoted or not has as much to do with time of day of
submission and random factors as how much people like it.

------
fwr
We have a thriving, helpful community already up and running at
<http://www.reddit.com/r/Scholar/>.

------
stared
While I agree with the "manifesto", the solution seems to be not that much
radical. There are many places where people can upload their papers.

Moreover, a web1.0 interface does not seems to be efficient.

In that line there is already: <http://www.pirateuniversity.org/>

And... judging by it's name (i.e. The Paper Bay) I would expect something much
more radical, actually aiming at getting 75TB of _any_ paper content...

~~~
jacquesm
Give it some time ;)

~~~
stared
Sure. Especially as from links there I'm aware that you are aware of other
solutions. :)

So, a reliable, automated and anonymous (?) way to upload books/papers seems
to be a must. Or do you have other plans?

------
cllns
It's a bit confusing that the site (ThePaperBay.com) has the same theme as
your blog.

~~~
Jasber
This is because they're both using <http://octopress.org/>

~~~
speednoise
That doesn't really make it any less confusing

------
chewxy
Curious... have you seen /r/scholar? I've requested a few papers from
/r/scholar (under throwaways usually), and they've always obliged

~~~
takluyver
Even within acadaemia, if you don't have access to a paper, it's common enough
to ask your friends at other institutions if they can get it for you. We're
mostly happy to share knowledge, and we don't feel much solidarity with the
publishers who charge £30 for one PDF.

------
michaelfeathers
I never knew that Aaron was in the lineage of the guy behind the Mark Williams
C Compiler.

~~~
wglb
Yes.

I was working at MWC when Aaron was born. I think I met Aaron once when he was
two or three, just briefly. I knew him from stories told by others, and
eventually by following his writings.

MWC was quite an interesting company in its own right.

~~~
michaelfeathers
I have fond memories of the compiler. I bought it when my roommate got an
Atari ST in college. Very well documented and I learned a lot from the C
source of emacs that came bundled with it. I wish I had that source to browse
again today.

~~~
wglb
It was a very good example of dogfooding--the document reflected what the guys
working on the compiler and toolchain would want from such a compiler.

I believe the emacs that you are talking about is MicroEmacs, written by Dave
Conroy while working at MWC. The source has since been made available
generally, and the editor is the one that Linus uses. Daniel Lawrence later
took over the distribution and here is one location:
<http://www.aquest.com/emacs.htm>

I used it for quite a while, but then fell into Emacs and didn't look back.

Oh--I did get it to run on my HP200lx and used it there for a while.

------
wicknicks
Reddit has a channel for paper requests. Might be a useful place for 'much
needed features'

<http://www.reddit.com/r/Scholar>

------
revorad
Great effort! Please add <https://peerj.com/> and <http://www.mendeley.com/>
to your list of resources.

~~~
jacquesm
I'll add those tonight, thanks!

Ok. done.

------
buro9
Very good.

But I'd love to have just seen the pirate bay duplicate itself and to create a
site exclusively for academic papers. Seeding of obscure papers might be an
issue, but I'm sure that it would work quite well.

~~~
lambast
When you download a paper from JSTOR or ScienceDirect, it is quite easy for
them to add various watermarks to the PDF, and they already do (mildly). It
might make it harder to scale this concept if the provider can track down who
initially downloaded the paper (and presumably shared it).

~~~
Create
request several incarnations and junk the diff

~~~
jtheory
That's fine if you're technical enough to even know what that means; I'm
worried about the people working outside the tech realm who could be unaware
even of the personal risk when they try to help out.

------
TimReynolds
I got bored reading the post so started to look around the website. I think
the Domains for Sale page (<http://www.jacquesmattheij.com/auction-of-domains-
for-sale/>) only clarifies my initial thoughts on this person.

~~~
p0ppe
Or you could have a look at this page instead;
[http://jacquesmattheij.com/thank-hn-our-friend-is-safe-
and-s...](http://jacquesmattheij.com/thank-hn-our-friend-is-safe-and-sound)

------
Matti
The filling a request process appears to be more complex than it is due to
superfluous form fields. The information about authors, title and DOI has
already been entered by the requester. For the service to be a viable
alternative, it should be (and appear to be) extremely simple to fill a
request.

Tracking down a paper is time consuming enough, even if you are sitting on the
network with access to most journals. Don't make it look like people have to
type in additional information manually.

~~~
mHORxaq
At articleak (see my other comment), we only ask for the link to the paywall,
and then if we could not get it from there (we make the http request via Tor
for obvious reasons so it may fail some times) we ask you for the title of the
paper, and that's all. So in the worst case you have to fill in two fields,
but in most case, only one: the link to the paywall.

However, I would not be as hard as you seem to be with the pirate would made
The Paper Bay. When you really need a paper and you already lost half an hour
searching for it in the web ocean, it's okay to take 2 more minutes instead of
30 seconds to ask for it on such a service.

------
ivan_ah
Your article is very well worded. I like the idea, but I am not sure if it is
reasonable for people "on the inside" to spend time fulfilling requests.

Could there be a js solution to this? A volunteer inside the paywall could
leave a tab open which periodically goes to get requests from the main
paperbay queue and tries to fulfill them (you would need some robot-like
functionality like find the PDF link). If success, it uploads the PDF. If fail
(no subsc?), it can notify paperbay to put the request back in the queue.

With one requests every 10 minutes and 1000 volunteers you could have a 6000
paper/ hour rate of exodus.

BTW? Where are you hosted? How long do you think before they come for you if
it gets big?

________________

PS: The protocol could be extended a bit it we need to handle captchas' as
well. These really piss me off because I can't get the paper via ssh to campus
+ elinks!

R: requestor F: friend (inside paywall)

    
    
       R-->tpb.com     doi:10000x200 PLZ
       tpb.com:        resolve doi:10000x200, prepare scrape recipe.
       tpb.com-->F     could you get jrnl.com/yr/issue/33131/
       F-->jrnl.com    GET ... 
       F<--jrnl.com    CAPTCHA.jpg  +  form el
       R<--tpb.com<--F solve plz ( CAPTCHA.jpg ,  form el )
       R-->tpb.com-->F form ans
       F-->jrnl.com    captcha form submit 
       R<--tpb.com<--F<--jrnl.com    PAPER.pdf

------
abdullahkhalids
Interesting. It would be much more useful if the papers in the 'store' would
be visible to everyone to see and request.

I wonder though how this ' we will deliver the paper on behalf of that email
address.' will be important to challenge any copyright violation claims...

------
megablast
I have trouble with someone who is also a domain squatter:
<http://www.jacquesmattheij.com/auction-of-domains-for-sale/>

~~~
aw3c2
Weird that he didn't use openpapers.com / open-papers.com

------
Create
perhaps it should have magnet links and torrent files like archive.org and an
API to upload, .bib-s and RDF-s in the url bar for the likes of zotero.

<http://arxiv.org/help/api/index>
<http://archive.org/about/faqs.php#Archive_BitTorrents>

~~~
axusgrad
The site does not host or _link_ to copyrighted work. I think he thought of a
way to make this happen without breaking the law (himself).

------
groby_b
That's a great idea - but _way_ to bury the lede there.

(And it just occurred to me that that's journalisms way of saying tl;dr.
Except I did read it, just to find what the heck The Paper Bay is :)

------
famo
Nice idea. Perhaps evolving this to include raw research data could be
interesting... e.g. may I please have a) your research paper and b) the 20Gb
of trial data you used to come to your conclusions. With this I'll run my own
experiments, expanding on your trials and hopefully adding to the wealth of
knowledge you so wonderfully kick started. Cheers.

~~~
famo
Yeah sure, here you go. Sincerely, Ms Kate Awesome, Phd

------
twodayslate
"Store" makes it sound like I should be buying something. Perhaps change the
name to "Upload"

------
joelthelion
Thanks a lot, this is very nice and helpful. However... Aren't you afraid of
going to jail?

------
djt
Is this legal?

~~~
djt
It's just a question guys, not a moral judgement.

------
devopstom
This is a bit of a grey area that we deliberately avoided with PDFTribute.net;
it seems perfectly acceptable to aggregate and index metadata, just as long as
you're not actually _storing_ any data.

------
dia80
First request and got the paper already!

~~~
popmilo2
My first request also and got the paper after an hour ! Who said, good people
are rare to find :)

------
mikevm
I guess this will come in handy: <https://github.com/kanzure/pdfparanoia>

------
deutronium
I'm wondering if you could do something like this automatically, so users with
access to journals could download a little program, and then other users could
send queries via a P2P program, which would end up at nodes with access to
journals.

There could be various settings to ensure each node with access to journals
throttle the amount of downloads.

------
pepr
I kind of hoped for an actual torrent tracker rather than central site that
can be easily taken down. But never mind :)

------
pla3rhat3r
The UI is not very clean but I love the idea. I wish there was a browse
feature. Some of us that aren't in college just enjoy enlightening our brain
and would love to see what's already up on the site.

------
kriro
Good luck :)

------
mr_luc
You're a really good guy, jacques.

And I had no idea about Mark Williams C.

------
jaipilot747
What is stopping the server from being taken down by legal pressure?

------
brownbat
Already featuring a request for "cocaine," no author given.

Relying on the benevolence of others may only go so far...

~~~
jacquesm
Not any more...

Of course the 'abuse angle' was taken into account, it's a bit annoying but
any user facing service will have that aspect.

------
handycapo
Get this on Tot/i2p.

It'll never go down...

~~~
p4bl0
I agree with you on Tor and/or i2p. I don't know if you can be as categorical
as "It'll never go down", but at least it will be significantly harder to get
it down. And it can be anonymous, which is important of course for the people
offering the service, but also to protect the users of the service…

I don't know if such a website can survive to the scientific publishers'
lobby, especially when its builder is publicly known.

Edit: it seems this exists, as another comment point out on this discussion
page.

