
The independent researcher - hardmaru
https://nadiaeghbal.com/independent-research
======
jandrewrogers
I've done an enormous amount of independent (and fairly successful) computer
science research over the last two decades. Computer science is amenable to it
if you are willing to make a large investment of time. The challenge is that
making this research useful to _other_ people requires an even bigger
investment of time, which creates a somewhat adverse incentive.

At least for me, there is one giant gap I've ever never been able to cross. I
am willing to spend an inordinate amount of time solving interesting theory
problems and doing novel algorithm design work _because it is useful to some
cutting edge software problem I am working on_. However, I have almost zero
incentive to spend the significant amount of time to publish any of my
research, whereas academics do. My time is enormously oversubscribed as it is.
My benefit is that I can reduce it to practice and build software that does
things no one else can do (and I enjoy the research generally). I know a few
other people in the same boat; great researchers, no time or incentive to
publish.

I've never been able to reconcile this. There is a similar analogue in open
sourcing software. Open sourcing your software often _increases_ the amount of
time you need to invest in it in practice, which disincentives people with
limited time to open source code (and after they already invested their extra
time creating the code in the first place) that they otherwise would be
amenable to open sourcing.

~~~
radarsat1
I think an important thing to take into account is that a very important part
of what makes a good scientist is not just how good they are at "doing
science", but how good they are at _communicating it_. A good scientist is a
good communicator. Even if that means finding people who you can work with who
are good communicators, it's part of the job. Coding up a really clever
solution and just kind of hoping people will come across it and appreciate is
not really doing science. (Not saying that's what you are doing, I'm just
going on a tangent here.) The point is, ever since the enlightenment, a
strictly critical part of the job of being a scientist has been the ability to
elaborate your ideas to others, and to argue for its merit.

So yeah, doing the work is the fun and motivating part, but a big part of the
reason that the academic community values well-written scientific papers is
because all that work can go to waste if it is not described (and defended!)
in a way that is clear and convincing (to the relevant audience of course).

Grad students may complain about having to learn to write effective papers,
but it's literally just as important as the technical work they are doing ---
maybe moreso, since in the future they may very likely change topics, and the
writing and communication skills is the part that will carry forward
throughout their career.

~~~
l0b0
I'm reminded of the saying that "Publishing research without data is simply
advertising, not science". Not even publishing the research means it doesn't
have a chance at advancing the state of the art. I don't understand how such
research can be successful at anything other than selling a few extra copies
of a software product likely to disappear within a handful of years. Unless
it's implemented as the absolute killer feature of a very successful product
it's very unlikely anyone will ever know there's anything novel in there.

~~~
jandrewrogers
The research does get disseminated, just slowly. The transmission vector is
contact with other people with firsthand knowledge of the research. It becomes
an oral tradition, which makes it difficult to break into and is inefficient.
I've come across many non-publishing research communities in various parts of
computer science that continue to develop and share the state-of-the-art
amongst themselves. The research isn't being lost, it is just
compartmentalized.

For example, there is an elegant cache replacement algorithm I learned from a
database kernel engineer a decade ago, one of those algorithms that is so
obvious in hindsight that you can't believe it never occurred to you. It
solves a key problem with the algorithms everyone knows in literature. I've
used it ever since and taught a bunch of _other_ database kernel engineers the
same algorithm. I have no idea where the person I learned it from got it, but
the literature still treats it like an unsolved problem. Many engineers are
using it in more and more systems, and there are a few different variants in
the wild, but no one has bothered to write it down AFAICT.

The salient point is that academia is not the only community that develops and
defines the state-of-the-art in computer science, but at least they publish.
Without an incentive for non-publishing research communities to publish, it
becomes difficult for an outside academic to break into some fields because
there isn't an obvious entry point.

~~~
d--
Is there a chance you can point us to a code snippet of said cache algorithm,
or elaborate a bit?

If it's simple and elegant in hindsight this insight might be interesting in
other CS applications. Not cross sharing and reinventing wheels is a somewhat
sad implication from the compartmentalization you mention.

------
sktrdie
The true problem is time, at least in fields like CS where expensive
instruments aren’t really needed to do research (not as much as other fields).

You need the freedom of time and being able to fully dedicate yourself towards
a specific study if you want to make any real contribution.

And I’d say that academia is struggling at doing science nowadays also because
you’re stuck doing all kinds of other things with your time rather than actual
research.

It’s hard to tell what the right formula is for being a productive scientist
and surely everybody’s free at doing it.

I have a feeling that something like Universal Basic Income would give this
freedom of time to crowds that can lead towards more people doing science and
hence more discoveries.

~~~
arcanus
> I have a feeling that something like Universal Basic Income would give this
> freedom of time to crowds that can lead towards more people doing science
> and hence more discoveries.

Agreed. When I was in graduate school I remember thinking that if we had UBI,
everyone would become a scientist. I'm sure this was biased by my own
interests, but the work often was truly thrilling. Sadly, it is very difficult
to find opportunities in academia or industry to pursue work that one finds
meaningful without major strings attached.

Most of my graduate school colleagues have left research. I'm a research
scientist in industry, but it is increasingly obvious to me that this is
relatively rare, particularly given the intellectual freedom and institutional
support I am provided. And certainly much of my time is focused on work that
has benefit to the corporation, versus open ended projects.

~~~
p1esk
_if we had UBI, everyone would become a scientist_

Very few people want to be scientists. To see why, just replace "scientist"
with "nerd", because for most people those are synonyms.

Moreover, anyone who wants to do science can become a grad student. That's at
least 5 years of "basic income", and access to mentors/tools/resources. All
you have to do is to find a lab that does what you're interested in.

~~~
rwallace
I never heard of a grad student program that pays you to do _your_ research
project. The ones I've seen, always offer a choice between paying you to work
on someone else's research project, or you get to work on your project but you
don't get paid; you do get access to facilities, but you have to pay an
exorbitant fee for this. Am I missing something?

~~~
p1esk
That's why I said you have to find a lab that does what you're interested in.
I have done just that, and for the last 5 years as a phd student I've been
paid mostly to do what I would do as a hobby otherwise (read/implement deep
learning papers), and try to advance state of the art.

~~~
n1231231231234
from my own experience, this is the exception, though. even in CS, if you pick
some of the more theoretical problems or a not-so-hot area, you might not get
funding - (almost) no matter how good you are. and that's CS. move to other
subjects like sociology, linguistics, history, and you will find that barely
anyone gets funded.

~~~
p1esk
I haven't heard about anyone in any PhD program in US who's not funded one way
or another. Sure, you might need to TA a class, but it's usually not that
hard, and leaves plenty of time to do research.

------
majos
I have mixed feelings about this.

It is true that modern funding of scientific research is flawed. Certainly in
the US and probably everywhere. Almost any working scientist has plenty of
stories about the silliness of bureaucratic requirements for governments,
universities, or nonprofits. Almost any working scientist can name some people
in their field who they think are severely overrated or even unethical (this
seems to be more of an issue in lab sciences, somehow). Almost any working
scientist has peer-review horror stories and admits that the process is noisy
at best.

Still: there are real benefits to being a non-independent researcher. At this
point it’s hard to get to the forefront of research — in pretty much any field
— without extensive contact with the people who are there already, most of
whom are not independent researchers. Failure to do so doesn’t just mean you
want produce hip research that caters to the specific elite tastes of the
establishment. It also drastically increases the likelihood that you’re wrong.
Good research is hard; it’s even harder in isolation. Ther are hundreds and
hundreds of papers about your favorite math/physics/computer science (it does
seem easier to be an independent naturalist or something) conjectures floating
around out there. Many of them sucked away years of their authors’ lives, and
somewhere around 0% of then are correct.

I suppose my two cents for anybody seriously interested in independent
research is to make it as easy as possible for credentialed researchers to
interact with you. Learn what you can, read accepted literature, acquire some
credentials yourself if possible, learn the language on Stackexchange or
Mathoverflow. Definitely do not open your interactions with grandiose claims.
Most researchers actually like their subjects; if you have something useful to
tell them, they want to know. You just gotta make it as easy as possible for
them to see you’re not a crank.

~~~
mindcrime
_I suppose my two cents for anybody seriously interested in independent
research is to make it as easy as possible for credentialed researchers to
interact with you. Learn what you can, read accepted literature, acquire some
credentials yourself if possible, learn the language on Stackexchange or
Mathoverflow. Definitely do not open your interactions with grandiose claims.
Most researchers actually like their subjects; if you have something useful to
tell them, they want to know. You just gotta make it as easy as possible for
them to see you’re not a crank._

That jibes with my experience for sure. The sample size is small, but I've
found "traditional" academics to be quite receptive to engaging with me when
approached the right way.

For example, I cold-emailed one of the two authors of a book I was reading,
and asked for some datasets they had used in their research. The professor in
question quickly replied with the data, a pre-print of a forthcoming paper,
and an invite to keep him informed on how things were going.

I later emailed him with a technical question about something from the book,
and he replied that he didn't know the answer offhand, as the research in
question was done 20 some odd years ago, so he cc'd the other co-author and
solicited his response. It took a few weeks (turns out he was on vacation) but
he also responded, helped address my question, and asked some questions about
what I was working on.

One of the two professors in question said during some of our email exchanges
that they believe there's a real chance to come up with something useful going
down the path I'm going down, which is encouraging.

Anyway, I suspect part of the key to engaging with academics is:

1\. (as you said) don't open with some grandiose claim.

Rather, ask a question, preferably a "good" question (that is, something that
shows you have done your homework, something that is specific enough _to_
answer, etc.). Not "Can you tell me how to get started in X" where you could
have just googled those exact same words and gotten an answer.

2\. Do your homework and make it clear that you've your homework. If you start
off with "I was reading your paper on X, and on page 3, in the equation for
flibblegazz, I don't quite understand the notation you use..." or something,
it shows that you're not just pulling crap out of the air. And if you're
emailing the person who wrote the paper, you're appealing to their ego a bit
in that you're telling them that you care enough about them, and their paper,
to dig in. I suspect that unless someone is a "celebrity researcher" to the
point that they get more requests than they can handle, they'll probably be
inclined to try and help.

3\. Do your homework in the sense of reading the basic texts in your area of
interest, read papers in the area, learn the relevant math (mathoverflow and
various sub-reddits are good places to ask math questions, BTW), etc.

4\. It's related to a previous point, but showing interest in the work someone
else has done is usually a good icebreaker. Asking for a preprint of a paper,
or asking for a dataset, saying you want to try and replicate their work,
things like that.

Do those things, go slowly, and just plain "be nice", and I think you can
build a relationship over time, with the folks in "proper" academia. And that
could be mutually beneficial. For example, you may eventually want to publish
a paper in a journal, and your academic friends can probably help you there.
You may even wind up collaborating to a level that justifies a co-authorship
scenario, who knows?

~~~
ISL
Academic here:

Totally agreed with every point. These are great tips even for someone within
academia interested in collaborating with someone else. It is a combination of
shared interest, impedance-matching, and mutual respect.

------
ronzensci
I am pursuing a self-sponsored PhD at one of India's leading institute of
science and technology. I have about 15 years of work experience, including
building a product which has seen a good amount of commercial success.

Validation of idea's was one of the reasons I decided to enrol in the part-
time doctoral program. Also, vetting of technologies I planned to use in my
work was another reason to become a part of a central govt. funded institute.
It is easier to collaborate with faculty members when you are part of academia
than otherwise.

Having said that, I am now of the opinion that independent research is the
most optimum way to pursue one's own ideas. If you are working in the space of
citizen impact or developmental projects, engaging with journalism editors has
a far more positive impact than engaging with academic faculty. The science
editors are able to articulate and help you present your work better than most
other people.

Another group of people, I feel, might be helpful are "true incubating
mentors". If you are looking at taking your work to the next level, I feel an
incubator/mentor will provide you with the right direction (I've myself not
experienced this but have seen people benefit from this).

------
chrisbennet
This is a great article but if you need a taste:

 _" My first go at independent research wasn’t necessarily easy. I didn’t know
independent research was a viable path, and especially not in Silicon Valley,
land of builders and shippers. I didn’t know whether I was doing something
useful with my time, or whether I was just a crazy person.

A few years later, I’ve realized that the answer to that question doesn’t
really matter. Life is short. Do whatever you can’t stop thinking about.
Documenting your findings in public (regardless of outcomes!) is a worthy
contribution to society, full stop. If you’re doing something new, and you
care about understanding the problem, people will pay attention. What’s more,
they’ll take your ideas and make them better than you’d ever imagined. And
that style of research - living in service to the public - starts to look very
different from the bloated, ivory tower models we’ve been accustomed to."_

~~~
tnecniv
> whether I was doing something useful with my time, or whether I was just a
> crazy person.

This is grad school in a nutshell, but in the classical route, you have a lot
of people to talk with who can tell you which bucket you're in.

------
arcanus
> This is the dark side of independent research. Without external validation -
> “I teach at Stanford” or “I got a grant from NASA”, it’s hard to convince
> people that you’re any good at what you do.

This is one of the reasons double blind peer review can be appealing. It
avoids credentialism, where the institutional reputation overrides honest
evaluations of the paper quality. Even in my own reviews, it's hard not to be
biased when I know the author is a luminary of the field.

~~~
Xcelerate
> This is one of the reasons double blind peer review can be appealing. It
> avoids credentialism, where the institutional reputation overrides honest
> evaluations of the paper quality.

This is one of the things I really like about the top machine learning
conferences at the moment. The reviews are open to all and blind to
credentials. Combine that with the fact that even vast compute resources can
be obtained fairly cheaply (at least for a short amount of time), and CS
research is one of the more "democratic" fields that someone can be a part of
nowadays.

This is in contrast to the research I performed during grad school, which
required access to a billion dollar neutron source, and reviewers generally
knew which institutions the papers were coming from...

~~~
spiderjerusalem
Eh, more often than not, it's pretty clear who at least one of the authors of
the paper is by just looking at what they cite. Double blind review is not a
foolproof system of avoiding creds and appeals to authority.

~~~
Mxtetris
From an article [0] published in last month's CACM, describing attempts by
three 2016 conferences to anonymize papers:

 _We find that anonymization is imperfect but fairly effective: 70%–86% of the
reviews were submitted with no author guesses, and 74%–90% of reviews were
submitted with no correct guesses._

[0]
[https://cacm.acm.org/magazines/2018/6/228027-effectiveness-o...](https://cacm.acm.org/magazines/2018/6/228027-effectiveness-
of-anonymization-in-double-blind-review/fulltext)

------
thallukrish
I just figured out being an independent researcher is the best for me over
time. It gives me the freedom to look at a problem space without noise. I
consider even a partner or a group to be a noise. Also, it is true as said in
this article that you do not need others approval if you can fund yourself and
find a interesting problem to work on. I do like to go beyond just research
for the sake of it and engineer / productise the ideas.

------
cvansiclen
I became an "independent researcher" after I left my physics research job and
moved to the country. With my trusty iMac and Internet access, I write and run
computer codes using Xcode, do graphics and some analysis with Mathematica,
read and get journal articles remotely (electronically) from a friendly
university library, and write up the research using Lyx. I post my papers on
arXiv (at no cost); there they are freely available, searchable, citable, and
forever archived.

~~~
abdullahkhalids
Can I ask for your Arxiv? I am near the end of my physics PhD and also near
the end of my patience with the bureaucracy of academia. Just wondering what
sort of research is possible independently

~~~
mikhailfranco
[https://arxiv.org/search/?searchtype=author&query=Van+Siclen](https://arxiv.org/search/?searchtype=author&query=Van+Siclen)

------
adamnemecek
I’m pursuing this route right now. Sure it has some downsides but like it’s
insane how productive you can get when you actually devote 12 hrs a day to
your research. A PhD student will have to go to meeting, teach classes, spend
time commuting etc. I get 12 hrs a day of pure research.

That being said I’m far from wealthy.

------
btashton
Has anyone here published in a journal independently of a large company or a
university/research institution? I cannot think of any papers I have read that
are attributed as such. I have been doing a bunch of independent research in
some areas in signal processing that I have only recently started thinking
about writing up, and I was wondering how realistic that is. It has been a
long time since I published anything in a journal.

Also since I started working on this, I realize just how nice it was in
university to be able to click a link to an IEEE paper and skim it. I feel
lucky to work so close to a large university library, but I see why people are
so drawn to SciHub.

~~~
solomatov
Here's a good example of influential paper published outside of academia by a
company which was really small at the time:
[http://www.onboard.jetbrains.com/is1/articles/04/10/lop/mps....](http://www.onboard.jetbrains.com/is1/articles/04/10/lop/mps.pdf)

It has a plenty of citations:
[https://scholar.google.com/scholar?cites=1113313642009476352...](https://scholar.google.com/scholar?cites=11133136420094763520&as_sdt=40000005&sciodt=0,22&hl=en)

It all resulted in an innovative product:
[https://www.jetbrains.com/mps/](https://www.jetbrains.com/mps/)

~~~
AndrewKemendo
Doesn't hurt that he has a graduate degree in Mathematics, so he's already
credentialed for research.

------
d_burfoot
I often recommend to young people just out of college that if they want to do
serious intellectual work, their path ought to be:

1\. Get a finance job on Wall Street.

2\. Spend a few years saving up a decent nest egg of a couple million dollars.

3\. Retire to the country side and start doing your intellectual work.

~~~
crazynick4
This is not as easy as it sounds. If it was, everyone would do it. A finance
job on wall street does not guarantee you will become a millionaire in a few
years. I worked for a few years in the finance department of a Wall St company
and made decent money, even above average for someone out of college with a
bachelor's (still not great for NYC), but if you want to make the millions you
have to get with a very close-knit crowd of the dealers/brokers/market makers
who are not open to letting in new people easily. And given the scandals and
fraud which we read about in the news regularly, I can deduce at least one
reason why. Just my n=1 but I knew a significant number of people who worked
with me, even ones who were closer to the higher ups who still ended up with a
nice but not extraordinary salary

------
AndrewKemendo
The discipline of research seems - historically - to be equal parts social
maneuvering and scientific rigor.

Just look at the very public spat between Ian Goodfellow and Jurgen
Schmidhuber about Generative Networks to get a flavor for that.

Chomsky has perpetually raised the flag when his research won't get published
because he's a vocal critic of some institutional priority.

Even back to Einstein, Kant etc... in multiple fields the powers at be thought
these guys were cranks until they blew everything open. If you can't get a
consortium of your peers in research to even take the time to read your work,
then you won't be able to make any changes.

As with all things, Institutional association (as long as it's the right
institution) serve as the social proof that you're doing real important work.

It's unfortunate, and I wish it weren't the case, but it seems to be
intractable.

------
mikhailfranco
Here are a couple of well-known independent physics researchers:

Julian Barbour:
[https://en.wikipedia.org/wiki/Julian_Barbour](https://en.wikipedia.org/wiki/Julian_Barbour)

Garrett Lisi:
[https://en.wikipedia.org/wiki/Antony_Garrett_Lisi](https://en.wikipedia.org/wiki/Antony_Garrett_Lisi)

Lisi runs an informal hostel for physics collaboration at his home in Hawaii:

[http://www.pacificscienceinstitute.org/](http://www.pacificscienceinstitute.org/)

------
varjag
Perhaps part of the issue with people keeping silent about their private
research is the world today is full of very vocal quacks and cranks. The last
thing you want is to be taken for a crackpot.

~~~
mikhailfranco
... the things to avoid:

[http://math.ucr.edu/home/baez/crackpot.html](http://math.ucr.edu/home/baez/crackpot.html)

~~~
varjag
Oldie but goodie.

Jokes aside, lacking significant result, there's no big incentive to publish
filler and "Towards.." type papers if you do something as a hobby. If one does
feel so inclined, it's definitely inching towards crackpot territory. Very
likely countless amateur projects quietly start and die every year without
making a peep.

------
abdullahkhalids
I don't see much discussion on how these people are able to collaborate on
projects with others? For a lot of research, you need partners to check each
other's work, to bounce ideas off, to stop you from making stupid mistakes or
help when you get stuck. This is why in academia people work within research
groups.

Is is possible to create research groups while working as an independent
researcher?

~~~
gpestana
I think independent researchers - at least in the CS context - end up
organically collaborating online through the content they share in github,
blogposts, etc. Maybe there is some space for a more structured community of
independent researchers which help each other and collaborate, but then again,
isn't structure that indie researchers are trying to avoid?

------
fizixer
I want to do independent (computational) research. That's pretty much the only
thing I want to do. Not employment in industry, not teaching, not funding
agency research (where the agency wants you to do only a restricted form of
research; okay maybe I could do that but then funding is hard to come by). But
independent research. And I don't care about money (as long as I can live on
it, and have access to computational resources).

But I don't know how I can make time for it. I worked for a while, saved up
some money, then did my own research using that money for a few months. But
working to raise money is very distracting.

~~~
watwut
Part time job? Work 4 hours a day or 3-4 day a week. Programmers salaries are
such that it is possible. The key is to watch time at work religiously and
ensure you actually work agreed upon time and not more. It works, but you have
to be consistent.

I have seen people doing this. Not with research, but with hobbies, teaching,
religion or family. If it is possible with these, then it will work with
research too.

~~~
fizixer
> work religiously ...

that's my problem with part time work. If I have to spend even two or three
hours on something not related to the research I really want to do, the rest
of the day is ruined.

I'm the kind of person, who does well by dedicating solid chunks of time day
in and day out on one topic, with the only distractions being eating,
sleeping, and going to the restroom.

However if that part time work involves me showing up and just being there, or
following mindless instructions (like burger flipping, or moving containers
from one place to another) then as long as I'm not physically tired, and I can
go back and do research for the rest of that day. But I doubt part time menial
work would raise enough money to pay the bills.

~~~
watwut
Then work 3-4 days and dedicate 1-2 to research and you still have weekend.

I did not meant burger flipping. Burger flipping does not pay enough. I meant
programming or other well paid job where you have good negotiating position.

By "work religiously" was padt of semtemce that meant to watch time. Don't
stay longer "just this one time".

------
blahedo
The article doesn't mention one very important expense for nearly every
research field: getting access to the research other people have published.
Some fields (e.g. computational linguistics) have long had a policy of letting
authors keep copyright, so they can post their own papers online, and the
collections are also free; but the norm is that nearly all publication goes
through publishers that paywall. So just to see what everybody else has been
doing, you need to have institutional support.

Only slightly less important is access to conferences, which are not always
super cheap: travel + lodging + registration is often in the $1000 range even
for the relatively accessible conferences.

~~~
gwillen
Fortunately: [https://sci-hub.tw/](https://sci-hub.tw/)

Yes, technically it's copyright violation, but if you ask any of the actual
authors, or any human being with any _legitimate_ interest in the papers
therein, they will encourage you to go for it.

~~~
ScottBurson
Also: [https://arxiv.org/](https://arxiv.org/)

------
acbart
I do Computing Education research, so being able to access learners is
actually a necessary part. I guess it'd be possible to do this externally, but
it's so hard to roll out new ed tech without quick access to an audience. But
more importantly, I feel like I'd be very misguided if I didn't interact with
students and understand actual classroom needs.

------
drewmassey
I left academia (in the humanities) to pursue work in tech. Most of this rings
true. My experience is that a.) a lot of exciting work is invisible to
academia and b.) to wing it without an institution is certainly a question of
money but primarily of time and focus (disciplines and research projects will
obviously vary on this front).

------
sleepyams
What would a platform for independent researchers (or potential
collaborators!) look like?

~~~
btrettel
A simple forum running something like phpBB would probably be the minimum
viable product. Make many different categories and see what people want to
discuss and what other features they want.

I'm a PhD student who's interested in going independent eventually, and I've
discussed this with several people. My impression is that most people would be
interested in discussing funding and historical examples of independent
researchers (particularly how they funded themselves). Discussion of how to
improve research and/or academia also seems popular. So perhaps the forum part
could focus on topics like those initially.

Later, however, as the platform ages, more important new features of the
platform would emerge.

~~~
sleepyams
A forum would definitely be core functionality. I've also been curious about
the merits of anonymous publishing?

~~~
btrettel
To me, publishing seems outside of the minimum viable product. I would like to
see many changes to publishing, but I think those would be best handled later
when independent research is better established.

Publishing is a chicken or egg type problem. At the moment few will use your
publishing venue unless it has a certain reputation. But you can't develop a
reputation unless your publishing venue is used often.

A publishing venue could be differentiated with unique features, e.g., open
access, anonymous publishing option, more rigorous peer review, or more
favorable copyright terms (I'm in favor of authors keeping copyright and just
licensing the paper), for example. I'm not certain if these features would be
enough to overcome the reputation disadvantage.

Perhaps independent researchers should forget about people who care so much
about reputation (who probably wouldn't go independent anyway) and instead
focus on making a better product.

~~~
sleepyams
I think this is actually a really great point about making a better product.
One could venture to ask: is the reputation system within the scientific
institution intimately linked the notion of natural truth? If so, does
focusing on approaching science from the lens of product development signify a
shift in the notion of what science is about?

I'm a PhD student in math currently and I often wonder what the difference
really is between the creation and discovery of mathematical ideas. It seems
the difference is really a matter of perspective: are we trying to discover
something true or create something useful? Often these two perspectives
intersect.

~~~
btrettel
I don't think academics are incentived to publish truth in general. In my
field (fluid dynamics), researchers seem to focus on producing as many papers
as possible that sound good but may have problems beneath the surface. This
may be different in math, because the standards are much higher.

The distinction between whether science or math creates or discovers truths is
philosophical and not something I'm that interested in.

My goal is to publish papers as likely to be true or useful as possible. So
thinking about this in terms of "product development" is really just about
increasing the chance of discovering the truth. (Or developing a useful model,
which may not be strictly true but is a good approximation of the truth. As an
engineer, what I do is more likely to be in the latter category.)

------
duncancarroll
Great article--couldn't agree more.

Also don't forget Michael Faraday, a bookbinder's apprentice who managed to
figure out much of electromagnetism, among other things.

------
Jhsto
Does Y Combinator fund (independent or not) research projects?

~~~
gwern
You mean [https://ycr.org/](https://ycr.org/) ?

~~~
Jhsto
Kind of -- if I am a researcher and I see a business application, but for
which I would need to go independent but due to the uncertainty of viability
continue my research and still be involved in my peer group in academics, is
it possible to receive funding and return to my lab?

------
Theodores
The ancient Greeks had a society where they could afford to have people who
did nothing other than open ended thinking. Sometimes I use some maths that
was worked out then and I am appreciative of standing on the shoulders of such
ancestral giants. A true measure of a society includes whether such a society
affords people who are given the time and space just to think.

I find it amazing that in today's world there are employers willing to pay for
likewise. It is quite a privilege to be in such a job where you are given some
type of giant Lego set and there are no set goals as to what you come up with.
In such jobs you really can be 'of independent means' and not subject to the
rules that govern how a lot of people work, to value every moment and not be
clock-watching until 5.30 p.m.

I follow quite a few people that work in tech and they don't seem to be
restricted to boring rules but able to pursue their passion. There are no tax
scams for companies to hire such people or other fiddles.

The Greeks may have had slaves and afforded their thinkers their own special
way, today we have companies that can afford thinkers in their own special
way. It is not just at Google where moonshots happen and neither is it just
the Google-me-toos that do this (Facebook). Even small companies have people
that are hired just to come up with stuff.

In previous times, before the 'modern' company, patronage was different and
there were ways and means. Obviously class hierarchy helped but that was not
all of it. Ultimately Darwin went off sailing with the Royal Navy and most of
his work built on the work others had done regarding livestock breeding - the
modern horse/cow/pig was 'invented' then. He may have inherited wealth but his
work was very much about what mattered to society at the time.

How much really comes out of academia? Do 99% of thesis efforts end up
collecting dust to never be read? For the author to get those letters after
their name and expect the world to roll out the red carpet for them? This
happens, further education is a business. There are plenty in further
education who think about themselves and their status anxiety rather than
society on some bigger/deeper level.

It has always been a two way street, no matter what the era is you also need
people who can do the 1% inspiration/imagination and do the 99% perspiration.
Luckily there are people, maybe few and far between, that can excel at that.
Ultimately these people really are 'independent researchers' and they will do
whatever it is whether they have a patron or an inheritance.

Then there are the wannabees that will insist on getting funding before they
lift up a pen. Sometimes this is an excuse, how many investigations have not
happened because there is no funding yet in reality all that is needed is for
someone to put time into quietly studying the facts?

A comical (well not really) instance of this was after 9/11 when the people
doing the 'investigation' said they were 'setup to fail' because they did not
have the funding to do a full investigation. These people being those pork-
barrel fed politicians who can't wipe their own arses unless they get funding
for it.

Once I met a lovely lady on a train who worked for some think tank. Fascinated
I asked about what it was really like working for this highly respected group.
Apart from the guy who founded it and spent all his time telling people how to
live on TV there was no 'research' going on. They just fund raised and paid
for their salaries in some pyramid scheme of research. It was most
disappointing to find out about.

Tinkering in your shed may not be 'independent research' there needs to be
some wider society connection even if research is done in near isolation and
with nobody knowing about it yet. Sometimes you need a whole new generation to
come along to appreciate the new thinking. Darwin had this problem as did that
Galileo chap.

------
nickpsecurity
Amusing that this type of person goes unnoticed when many of us tell people
what we do. I fall into this category as I was describing in another thread.
What she misses is many of us aren’t rich or the work sustainable: many people
in lower to middle classes sacrificing money or status to do deep research and
development in a field that interests them and/or they find necessary. They
might also not like the priorities of commercial or government funding groups.

For instance, I thought figuring out how to make computers that don’t fail or
get hacked was a thing we desperately needed. I believed both livelihoods and
lives were at stake. That we had access to them was a social good that neither
the markets nor FOSS were really serving. It was also an interesting, deep,
rabbit hole of a problem crossing many sub-fields of IT, economics, and
psychology. That she misses people without money doing it altruistically
surprises me more given she wrote the report on FOSS developers working with
little to no money or contributions on critical stuff that mattered to them.
Same kind of thing I think with different work output.

Still a good write-up that will draw attention to the concept. We might get
more people doing it or publishing what they’re doing. I think most of us
don’t publish enough. We should accept some of the troubles of that since the
ideas get out there more. I also like this quote focusing on the obsessive
nature of deep, independent research:

“I understand, then, why researchers flock to the safety of institutions.
Imagine studying something that nobody else is studying, for reasons you can’t
really articulate, without knowing what the outcome of your work will be. For
the truly obsessed person, the need for validation isn’t about ego; it’s about
sanity. You want to know there’s some meaning behind the dizzying mental
labyrinth that you simultaneously can’t escape and also never want to leave.”

Edit: I also noted it gets even lower-end in response to a comment elsewhere
saying you just need money to live and basic, computing equipment. That's true
but maybe not what people picture.

"Even lower barrier than that: many bright folks that were hackers or makers
that I met in rural areas were on food stamps or living with someone
unemployed without a car. They could usually get a Wifi-enabled phone or old
laptop that they could use at nearby McDonalds or something. Many use data
plans, too, just cuz cell service is a necessity to them. At one point, I went
lower having no PC, phone, or car. Designed on paper or dirt depending on
where we were at.

The reading and practicing like you said is what gave us skill. I think peer
review and support is just as important, though. I had lots of it once I got
online. There’s quite a few people out there probably stuck on some subjects,
reinventing wheels, or chasing dead ends just because they can’t talk to
experienced people."

------
Kyragem
With low federal funding rates and every CS Department having doubled in size
it is barely possible to do research IN academia. You spend nearly most of
your time writing grants that don’t get funded. The little federal funding
that is available ends up mostly at elite/ivy schools who dominate NSF panels
and conference program committees :-( kudos if you can do independent research
outside of it though

