
Rabbit holes: Why being smart hurts your productivity - sthatipamala
http://sridattalabs.com/2012/02/06/rabbit-holes-being-smart-hurts-prod/
======
TeMPOraL
Truth is, most of the things I know about the world in general and programming
in particular is due to procrastination and rabbit-holing. Big chunks of my
understanding of physics come from randomly reading stuff on Wikipedia while
procrastinating in high school. I discovered HN and learned Lisp when shying
away from doing an university project in PHP, which in result got me a job in
Erlang and also led to being widely recognized at the university as 'the Lisp
guy'. Here I discovered pg's essays and LessWrong, both of which helped me
grow, and many other things. I owe much of who I am now to rabbit-holing. And
also being a person who always has three solution candidates to any problem in
30 seconds ("you know, there was this startup/library/tool featured on HN last
month...") is a result of that random learning.

I do sometimes feel that it's not the optimal way of learning things. I try to
find the balance between random-walking and systematized learning, but one
thing I'm sure of, it's that the optimum is not on the side of formal
education, at least not for me.

After trying, again and again, to force and fit myself into 'the System' of
traditional, formalized learning I discovered that it is almost impossible for
me, and it always have been. Learning what I'm told to learn for sake of tests
and exams just repells me, causing almost physical pain. So I'm not trying to
force myself anymore, I decided to do what I have to do with minimum effort,
and spend the rest of the time learning what I want, the way I want, and
allowing myself time for rabbit-holing and procrastinating.

~~~
cheatercheater
I have quit university when I realized that the only difference between
reading books on my own and reading books for a uni course was that university
is a place where you learn how to work. You are spoon-fed work discipline and
research methodology. I already had all of that, and so I quit. At that point
it wasn't difficult at all to continue on my own.

~~~
TeMPOraL
I was thinking about quitting university after taking part in Stanford's AI
Class and then realizing that they have around 12 courses planned for the next
semester. I don't believe that any university in my country could beat the
quality level of what's offered now by Stanford, MIT and others. I decided to
stay, but I'm not lying to myself anymore about the reasons. I stayed not to
get knowledge - which I can get faster and of better quality elsewhere - but
because of:

    
    
      - Master degree (as in, formal title; it may come in handy)
      - Access to expensive/difficult to get equipment - for all those
        crazy ideas of mine that involve e.g. a piece of custom hardware
      - Access to people who know some particular area inside out
      - All the smart people who study at the university - there won't be
        any other chance to meet so many amazing people and maybe Build
        Something Amazing with them, and then keep contact for the future.
      - Cheaper tickets for public transport.
    

But as I said, I'm not there for the knowledge. Not anymore.

~~~
indiecore
"I'm not there for the knowledge. Not anymore."

I'd argue that you ARE there for knowledge just not the kind that universities
tend to say they give you.

Rather it's the kind of knowledge that the University system was supposed to
give you before they all turned into job training, especially telling is
"Access to people who know some particular area inside out". That's what
university is supposed to be about.

Of course cheap bus is nice too.

------
jiggy2011
I don't think this is necessarily limited just to smart people but it's just
more a trait of curious people (perhaps there is a correlation there?) or just
the intellectually insecure.

For example I don't consider myself an especially intelligent person (on the
right side of average I _hope_ ), however I know people who _are_ very
intelligent and can talk at length with authority on a vast number of topics
and are seemingly able to absorb and retain information instantly. Also I am
exposed to forums like HN which are full of smart people.

So naturally whenever a conversation or thread comes up where I feel that I
don't have anything to contribute but I know it sounds _fascinating_ I can't
help but to try and learn everything I can about it so I can weigh in next
time, which in turn leads to finding gaps in my prerequisite knowledge of the
subject (and interesting offshoot subjects). Which of course leads to having
12 tabs open.

This is why I think it is better to pick a few subjects that you decide you
will learn in some detail and buy books on the subject.

The great thing about books is that they follow a linear progression and most
importantly _they end_.

The problem with HN is that you can read every article on the front page along
with a bunch of wikipedia, but the next day there will just be _more_.

~~~
_delirium
> The great thing about books is that they follow a linear progression and
> most importantly they end.

That's the usual case, but I've found myself a few times sitting in a
university library reading books in a Wikipedia-esque way. Read a few chapters
of one, find an interesting footnote, grab the book it references off the
shelf and follow up on that, and soon there's a pile of 15 books on my desk...

~~~
kamaal
This is true for reference books.

But not for other kinds of books like Novels, tutorials.

If somebody is reading a Reference book, then they are reading it the way you
describe.

But if you are learning something/trying to get enough information for the
moment you must know what information to neglect else the 'rabbit hole'
problem is inevitable.

------
_delirium
As one great example of productive rabbit-holing, Knuth was frustrated by the
computer-typesetting tools available to write a computer science book, so he
wrote TeX; and then there weren't good fonts for it so he started designing
one; but he needed a font-designing tool so he wrote METAFONT; etc. Eventually
he got back to his book.

But he was: 1) already a tenured professor at the time; and 2) had a
sabbatical year in the middle as well. So he was able to choose to delay one
kind of productivity to follow this other rabbit hole instead, and ultimately
probably end up more productive as a result. The normal productive solution,
of course, would've been to just muddle through with whatever typesetting
system his publisher was using, and stay focused on the book.

~~~
kamaal
Hmm, I think its more scratching the 'itch'. I can think of one more case like
this Larry Wall's 'Perl'.

Perl was invented because of some form of rabit holing. Actually productive
rabit holing can make a good business.

------
there
_On my second day of practice, I felt the urge to find a better Haskell syntax
highlighting extension for Vim. I managed to find one that was distributed in
a package called a "Vimball". I spent 10 minutes trying to figure out how to
install a Vimball and then another 10 customizing the plugin's settings._

This is also referred to as Yak Shaving.

[http://sethgodin.typepad.com/seths_blog/2005/03/dont_shave_t...](http://sethgodin.typepad.com/seths_blog/2005/03/dont_shave_that.html)

~~~
sirclueless
I don't consider them the same at all. Yak Shaving is a social phenomenon
where a group of people get bogged down by minutia and irrelevant decisions.
The Rabbit Hole is private, caused by becoming easily and repeatedly engrossed
in tangential topics.

~~~
psykotic
You're confusing yak shaving with bikeshedding.

------
krig
The whole reason I end up commenting on HN, just like I am doing right now, is
because I'm drifting from my intended research path and start thinking that
maybe someone has posted some interesting story related to what I'm supposed
to be looking at.

Most recently, my intent to look into writing a compiler frontend for LLVM led
me to install a dozen new modes in emacs, explore OCaml and Go, download the
source code for several major projects with the intent of examining their
garbage collection strategies and roughly 60 new papers in my "To Read"
folder.

:(

------
kamaal
I understand the true spirit of this post. But regarding finding the right
tools for the job(which in this case is syntax highlighting) is worth chasing.

I was once helping out a relative of mine. Who was doing some freelance
project in electronics. As a deliverable we had to ship some soldered boards
in four days. We didn't quite have a organized working shop. The first night
we worked it was a disaster. Nothing was in place, and we spent great deal of
time solving confusions and finding scattered tools. Then the next day we
spent all the day getting organized and finished the entire job the very next
day. Getting organized helps.

But I noticed a strange thing during that project. As we were soldering, I
realized apart from talking to each other about some topic intermittently. We
never really got distracted and got off our seats and we were never
distracted. We only took food and rest room breaks nothing more. How did that
happen, I did simple analysis:

    
    
        1. We were not spoiled by choice. Like in software where there are gazillion things and standards to choose from.
        2. Focused task, Solder, pick components, debug, multimeter.
        3. No such thing like a browser to distract.
        4. Ability to measure goals. Something that is difficult to do in software.
        5. Doing and exploring new things have a little higher barrier to entry. Thereby you don't shift tasks until you complete the current one completely.
        6. Information available is to the point and not like software where every other blog has a hello,world program posted. Thereby doing something new requires iterations of focussed reading, implementing, testing.
    

Having said, this model is a little difficult to create in software. I guess
Electronic engineers at a time were plagued with same problems decades back
when they had 50 free PIC projects magazines near every door way. And having
to worry about every new precision voltmeter that comes very other week.

Many of our problems will resolve automatically with some quality
standardization in our industry.

We still quarrel about Text editors, things like syntax highlighting. Most
technology communities should by now figure out that if you are releasing
something cool, you also need to take care of the tooling.

------
ww520
This makes me want to cry. I see myself going down the rabbit holes recently.
I had this great idea that I was investigating. Then I saw one of the
functionality can become a product by itself. Hey, let me do that quickly.
While doing it, I needed to build a library. Hey, let me generalized it
quickly and released it as open source.

And that's how JSODA came into existence,
<https://github.com/williamw520/jsoda>

But I'm still longing at popping the stack two levels up back to the original
idea.

~~~
jacobr
But how do you know you shouldn't switch to a new idea?

~~~
ww520
Good question. I guess it would require at least doing some due diligence
before dismissing an idea. For me the OS library is generic and I can always
reuse for other projects. The other two ideas require much more extensive
tryout.

------
dkarl
I've always felt that life is not about getting as much boring stuff done as
fast as possible, which seems to be a common definition of productivity.

On the other hand, everyone's definition of "boring" is different, and we
should each be true to our own, regardless of anyone else's opinion.

~~~
cinquemb
I agree, but for me, its really hard to figure out what that is, so i just
keep drifting along just picking up information as it piques my interest.

any suggestions? i like CS/engineering but i feel like these days people are
looking for the next app or the next crappy product, and i can't be bother to
want to spend time making one. i want to do something helpful for the world
using my technical/computing expertise. college is a joke, mainly party and
bullshit.

~~~
tikhonj
If you find college to just be a joke, it probably means you're either at the
wrong college, approaching it the wrong way, or both.

My suggestion is to pick a general field you're interested in (e.g. CS), get a
good survey of different specialities in it (like AI, machine learning,
databases, compilers...etc) and choose the one you like best. You'll probably
find it much easier to focus your curiosity on one speciality within CS, and
it will probably be easier to do something interesting and non-trivial within
some speciality like this.

Coincidentally, this is exactly where college is great--it gives you a nice
structured overview of a bunch of different fields _within_ your chosen
subject and also lets you dive more deeply into whatever interests you most.
College also gives you the general background needed to learn about the more
specialized material--you're forced to learn more about subjects like math and
EE than you would learn by yourself, which will help you in doing something
non-trivial.

------
melling
People need to decide if they're staying on top of things or trying to get to
the bottom of something.

Yes, I stole that from Donald Knuth.

[http://lifehacker.com/398112/neo+amish-drop-outs-eschew-
emai...](http://lifehacker.com/398112/neo+amish-drop-outs-eschew-email-web)

<http://www-cs-faculty.stanford.edu/~uno/email.html>

------
rohit89
This happens way too often with me. I just pull the lan cable out. No
internet. It means I probably end up taking about 10-15 minutes to do
something that would take a couple of minutes if I googled but I end up saving
time because I don't just google for what I'm looking for and then close the
tab.

------
bane
HN is a great example of this. Down down down the page goes the thread and my
productivity.

It takes discipline to apply what you've discovered during an intellectual
deep dive. Discipline, by its very nature, is a learned behavior. And it can
be _very_ hard to learn for the easily distracted ultra-curious types that
tend to make up most of the "smart" population.

Take it as a challenge, learn then apply, learn then apply. I frequently find
that I've learned something down a rabbit hole someplace that, when I apply
it, impresses many of my co-workers. When all I think I'm doing is
regurgitating some fact.

Note: don't forget that other people may have gone down different rabbit holes
and have an entirely different set of apply-potential knowledge just waiting
to make the world better.

------
chalst
I heard the phrase "yak shearing" as the term for what this author calls
"rabbit holing".

The essay cites Hamming, "I notice that if you have the door to your office
closed, you get more work done today and tomorrow, and you are more productive
than most. But 10 years later somehow you don't know quite know what problems
are worth working on."

This reminds me of a quote by Rab Butler (British Chancellor of the Exchequer,
1951—1955) "It is more important to be generous than to be efficient — that is
what I learned at the Treasury." It's about more or less the same thing:
generosity of either your time or support gets you involved with other people
and their concerns, and that is very important to making contributions of real
importance.

------
einhverfr
What I have started to do is to write a short to-do list at the start of each
day. If I don't finish it that is fine but it helps me at least set up some
tasks I need to do and get to them as quickly as I can.

However, one thing. Hard problems are FUN. Double for hard problems that have
some bearing on current work, even if it isn't critically important to the
immediate moment. So I think it is important to allow for a certain amount of
productive distractions.

I would also add one more thing. These rabbit holes can be good for another
reason when running a business: they keep you on your toes regarding to what
you need to do to succeed and sometimes the solutions are important ones to
pursue as a business strategy.

~~~
sthatipamala
I totally agree. As a developer in my startup, my job involves cranking out
code. But as a founder in my startup, I need maintain a giant reverse index of
things that are happening in technology. These two goals can be conflicting at
times.

~~~
Mjux
Absolutely agree with you on the task ahead. Sticky notes does the job when
you can actually finish them on time or even complete. There is a complete
discord on what we set out to achieve and what needed to be done - on the
task.

What do you think of trello and task based applications in this case?

------
akg
What seems most insidious is that the tasks that lead you down the rabbit hole
make you feel like you are actually being productive, which is probably why it
is so hard to intrinsically determine when you are heading down that hole in
the first place.

I have noticed that most of these tasks relate to web-browsing and have
experimented with just turning off my wireless while I am working. This
usually means that when I need to do something "unproductive", I need to make
a little bit of an extra effort to get online, which is generally enough to
think to myself, "Hey, I should really get back to coding..."

------
oodalolly
Important to note that this isn't just a issue of digital rabbit holes.
Freshman year I spent two days teaching myself to juggle with the oranges I
was supposed to be growing mold on.

------
benohear
I've found this is the biggest productivity gain of working in pairs
(designing or programming) - it avoids a lot of rabbit hole-ing, Yak shaving,
gold plating, etc

------
rdl
Compare and contrast: rabbit holes vs. yak shaving.

Rabbit holes are basically unproductive-on-current-task lines of research
(working on something, seeing something shiny on wikipedia, ...). Yak shaving
is setting up a really long critical path for the current task (wanting to
install a package, it not being available, building a new physical machine to
host a virtualization environment to install a new OS to install the
packages...)

------
hypr_geek
This sums it up for me, and usually everyone else (smart or not)...

<http://i.imgur.com/h4K5c.gif>

------
character0
I find that I need these periods of intense rabbit-holing, in order to fuel my
work. Even when aimlessly strolling the internet, I'll somehow link that
information to a later project or conversation I have later on. As we know,
this can often become aimless and we feel a certain amount of guilt, or in
extreme cases mental fatigue from focusing on too little. It's a bit of
sensory overload.

On the other hand, if I've been intensely working on a project, I can often
times feel as though I'm burning out; I have trouble finding further
breakthroughs and I lack inspiration to push a project forward.

I certainly feel that I can get stuck in either environment, so often times it
is about being able to know when to transition before reaching that fatigue.
Like others have mentioned, making lists to prioritize my tasks often times
will help me manage this. Staring at a list that has nothing crossed off is a
very effective deterrent from internet use.

------
scoot
So I got halfway through reading the XKCD cartoon but got distracted by the
infinite grid problem, caught myself, and managed to finish reading the
cartoon before going off to google the solution to the infinite grid problem.
It was some time later before I cam back and realised I hadn't actually read
the blog post. Oh the irony!

------
charlieok
I am always thinking about just how productive I'm going to be, and just how
satisfying it will be to commit one improvement after another, gaining speed
as progress leverages earlier progress, once I hit the bottom of my current
giant yak shave and start unrolling it, all the way back up the technology
stack.

------
mturmon
"But they also had the advantage of working at the top of their games at MIT
and Bell Labs, respectively."

Feynman was at mostly at Caltech, of course, although he attended MIT as an
undergrad and was at Cornell and Los Alamos for a time. Presumably the quote
came from his Caltech years.

~~~
sthatipamala
Ahh, sorry. I was aware of this but I have no idea why I might have typed MIT
while I wrote this. Pushing an edit now.

------
fabricode
Emily Litella: "What's wrong with holding rabbits? They're cute and cuddly and
just awfully good pets. Sure they make messes and eat the lettuce out of your
garden. But when you hold that cute little fuzzy bunny with his little
wiggling nose..."

Jane Curtain: "Ms. Litella, that's rabbit holding, not rabbit holing..."

Emily: "Nevermind"

In all seriousness, if you don't allow yourself to go down a few rabbit holes
and be "unproductive", you'll sell yourself short in the future. The place
where "rabbit holing" is dangerous is in meetings where what should be a quick
& efficient get-together gets drawn out into a debate over meaningless
details.

Context matters.

------
scelerat
Oh man I should hand this article to anyone who wonders how I scraped through
school with a lit degree and somehow became a software engineer.

My pinboard solution is intentionally low-tech. A yellow pad of note paper I
can take with me anywhere. First page is the things I want to accomPlish for
the week. Following pages are things I learn or want to add to the list. On
Friday I make a new to do page, consolidating and removing previous pages.

I keep the pad on my desk in front of me. I can look at without opening my
computer or phone. This usually preempts frivolity.

------
calydon
My god, it's like I'm looking in a looking glass.

------
snowwrestler
In college I spent too much time playing around with the (newly created) World
Wide Web instead of studying for my degree. I didn't exactly graduate with
honors, but it doesn't matter since I ended up working on the web. So for me,
there was a great career at the end of that particular rabbit hole.

------
pedrolll
So we procrastinate because we're intelligent? This should massage every
hackers ego on Hacker News :) "Ah, I see, it is because I'm smart that I can't
get things done. It's not that bad after all". To me it seems that most people
procrastinate, whether they're are intelligent or not.

~~~
mokus
Unfortunately, it's often easier to get away with it when you're "smart"
because when someone looks at your screen they don't see a Facebook game -
they see "research".

------
josephjrobison
Wow this is pretty incredible - I guess it's about finding that perfect
balance between hard working and serendipity. So where does social media and
television fall in this equation?

~~~
Estragon
Rabbit-holing and yak shaving happen when you're still committed to working on
the problem domain but struggling with some aspect of it. Social media and TV
happen when you've explicitly given up for the time being.

------
mayank_kh
Sometimes you overestimate yourself.....Thus you inhibit your learning
curve.....You do not mingle with other people.....Thereby limiting your global
perception

------
feynmansawesome
This sounds like ADD.

> Errata: Feynman had a career at Caltech, not MIT.

Obviously you haven't read any of Feynman's great books about his life! Get at
it, quick!

~~~
psykotic
Feynman was an undergraduate at MIT and his graduate studies were at
Princeton. He never held a position at MIT.

------
badalyan
Another great solution: thoughtback.com

------
aragorn84
That's why you need a system for recording all the things you can do, what
context you should do them in, and keep working on your main task. Once the
time is right, you go back to the things you wanted to do.

I can't claim to be an expert at this but I recently watched these vids
thesecretweapon.org and coupled with what I already know from GTD and the
Pomodoro technique its going well.

~~~
creamyhorror
True. That's why I've started using a variant of Autofocus[1] (a task system,
much simpler than GTD) and am intending to try Pomodoro.

Ultimately when you see the various things you need to do on a list, your
intuitive mind automatically prioritises certain things over others. That's
how you can reduce yak shaving and bikeshedding; by attacking tasks with
shorter 'dependency chains' and higher urgency.

[1] [http://lifehacker.com/5151111/autofocus-is-a-single-
paper+ba...](http://lifehacker.com/5151111/autofocus-is-a-single-paper+based-
list-organization-system)

------
cheatercheater
One aspect of the "rabbit hole" problem is when you're doing work and you find
a detail you want to read up on a bit more. You're doing Lisp and you want to
read up on the best way to do a specific thing, then the best way to do a
specific part of that, etc. The good old argument of parsing an infinite tree
(graph) of information: Depth-first or breadth-first? Let's assume "breadth"
means getting things done, and every new level of depth is going into a (maybe
tangential) detail.

The great thing about a teacher is that they can find cycles in such graphs of
related topics. They can use your attention to detail to steer you back on
topic. Reading up on the trigonometric identities leads you to hyperbolic
functions which leads you to holomorphic functions which leads you to linear
algebra which leads you to a simple proof of trigonometric identities. It's a
great thing if you figure something like this on your own.

There are other strategies to following the graph around. In fact, there's a
topic called "computational strategies", which is fairly popular stuff if
you're into supercomputing. Then on the other hand there are organizational
measures like Time-Division Multiplexing, prioritizing, QoS, etc.. It's all
been worked out, we really make computers in our own image. They can teach us
a lot about ourselves.

I can equate reading up on new things (e.g. following HN closely every day) to
juggling. Every time I read something new and interesting there's a new topic
up in the air. It's there for some time until I've made my peace with it and
integrated it into my consciousness.

I find that in my brain there's a certain amount of resource that I can
dedicate to this sort of juggling, after which I gradually start getting
tired, and "burnt out on news". At this point I start doing some fairly
mechanical tasks. It often involves cleaning my flat, sports, shopping. Those
are things which in some way tick boxes and they are a change for me which is
the next step in the cycle. (Compare this to hanging out with friends or
watching a movie or listening to music, which are passive tasks that do not
advance the cycle for me.) After doing some of this practical stuff, I usually
slow down reading up new stuff, forget about HN or whatever I was parsing, get
caught up in real life. At this point I save all links, close it all off, and
shelve it away. One thing that helps is that I have a structured database of
resources (ok, just some dirs in dirs in dirs) where I save notes or webpages.
This helps a lot to offload me from the burden of feeling I _HAVE_ to read
those articles. It's good because in the end most of this "catching up" that I
do on HN is just to cross-reference topics and get new ideas; if two files are
in the same dir (e.g. an "Parsers" dir) then I will end up looking at them two
at the same time the next time I am working with parsers.

I have never been on a drinking binge, but externally - and as a voluntary
social worker I've been a lot around people who have had strong problems with
alcoholism - such a "research cycle" is very similar to a drinking binge. It
starts with one sip, just let me look at the headlines on HN. Then I start
taking shot after shot of reading the articles. Then I do it so much that I
forget about everything. I'm in the zone. I'm in the flow. The world does not
exist, I am _GOD_ , the clarity is amazing. Escapism at its very best. At some
point the high becomes tiring and I am unable to keep up. I get real-lifed. I
start sobering up. By moving into doing something constructive I completely
mitigate the crash.

So the cycle is start -> get infected -> totally binge out until you can't go
further -> sober up -> do real life stuff -> back to start.

Of course, that's just one of the layers of my life, but it's a fairly well
defined one at this point.

The cool thing is that, unlike sucking on the juice, this actually gets me to
places. Maybe we should teach alcoholics Agda as rehab:
<http://i.imgur.com/njf59.png>

