

Ask HN: How do you solve hard problems if you can't make incremental progress? - ntoshev
http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-solvers.html

======
Goladus
Avoid stress and distractions. Do not multitask. Take breaks to eat (healthy
foods), exercise, and sleep. Feel free to discuss the problem with friends and
colleagues but minimize emotional investment with other people. Read books and
articles on the subject. Study existing tools and study similar problems. If
you can't make incremental progress on a solution you can incrementally
increase your chance to solve it all at once by getting smarter and getting
better tools.

No facebook, television, hacker news, video games, twitter, dating, drinking.

I would not say this is a particularly healthy lifestyle in the long run. When
possible incremental progress will make it easier to live a happy fulfilling
life. But if you really need to solve something...

~~~
wisty
Multi-tasking can be OK ... depending on the person. Some people love it, some
fall apart.

But yeah, "Avoid stress and distractions. Do not multitask. Take breaks to eat
(healthy foods), exercise, and sleep. Feel free to discuss the problem with
friends and colleagues but minimize emotional investment with other people."

I'll add:

Make progress, even if it's not progress. I teach english. If my students are
stuck, I tell them to answer the question in their native language. Or just
guess. Or say something that isn't even relevant. There are 3 reasons why I
say this: 1) you look busy, so your boss will be happy, 2) you feel better and
might subconsciously clear the block just by not stressing over it, and 3) you
start filling in the jigsaw pieces around the block, which might make the
solution easier to reach.

~~~
timwiseman
Research has repeatedly shown that virtually no one can multitask effectively
(<http://arstechnica.com/old/content/2006/03/6417.ars> has a laymen's summary
of some for one place).

The closest thing to an exception I know of is listening to music. I will
listen to music while working on hard problems, but then I tend to choose
music I know well and I use it to block out more distracting sounds rather
than pay much attention to it.

~~~
lmkg
I found I can listen to music so long as it doesn't have lyrics (non-lyrical
vocals are OK). When I was in college, I had a special playlist of wordless
electronica I would use for writing essays.

Someone once told me that there was a study showing that most music tends to
decrease work effectiveness, but classic music actually increases it (I do not
have a link). Sadly, I'm an uncultured boor and while classical music sounds
pleasant it's not my favorite by a long shot. I compromised by including some
techno remixes of classical compositions in my essay-writing playlist.

~~~
khafra
Even classical music is rather suspect, although I've lost the original
article: [http://www.overcomingbias.com/2009/02/thoughtful-
music.html#...](http://www.overcomingbias.com/2009/02/thoughtful-
music.html#comment-387795)

~~~
dasil003
Classical music is _more_ suspect because it's attention grabbing. Rhythmic
music is better.

~~~
trafficlight
I've been trying background noise instead of music. At first it can be a
little annoying, but it falls into the background is actually quite nice

<http://simplynoise.com> \- I prefer the brown noise, myself.

------
dwwoelfel
Use Poyla's methods of problem solving described in his book: _How to Solve
It_ (<http://www.amazon.com/o/ASIN/0691023565>)

He deals specifically with mathematical problems, but the same ideas apply to
computer science.

For an outline of the method see this Wikipedia article:

<http://en.wikipedia.org/wiki/How_to_Solve_It>

~~~
RodgerTheGreat
And if you're specifically trying to design an algorithm, you might look into
"How to Solve It by Computer" by R.G. Dromey, which was inspired by the
former:

[http://www.amazon.com/Solve-Computer-Prentice-Hall-
Internati...](http://www.amazon.com/Solve-Computer-Prentice-Hall-
International-Science/dp/0134340019)

------
michael_dorfman
Many (perhaps most) hard problems don't need to be solved from scratch;
rather, it is a matter of applying a well-known solution that somebody has
already found.

The example in the article is a good one: there's absolutely no need to
attempt to design a Sudoku-solving-algorithm from scratch (except as an
exercise), and it is pitiful to see Ron Jeffries flailing around aimlessly in
TDD-land when a simple trip to Google would have done the trick.

The difficult part, the part that comes with experience, is recognizing which
class of problems the specific problem you are grappling with resembles. In
the case of Sudoku, it's an "exact cover problem", and once you realize this,
all kinds of solutions present themselves.

In this way, the "incremental progress" is not made on attempting to solve the
problem incrementally, but by incrementally sifting through probably solution-
spaces for known algorithms.

~~~
blizkreeg
Well said! Few _hard_ problems are solved entirely from scratch, unless it is
to boost one's ego. In almost every field, the key is to identify problems
with similar patterns or constraints.

------
myth_drannon
TRIZ - <http://en.wikipedia.org/wiki/TRIZ> -"TRIZ is a methodology, tool set,
knowledge base, and model-based technology for generating innovative ideas and
solutions for problem solving."

Generations of Soviet engineers were educated on that methodology. Many
western companies use it as well.

~~~
jhg
Woah... "Creativity as an Exact Science". TRIZ needs to be submitted to HN as
a separate item. It can make for one interesting discussion.

~~~
mikexstudios
Good idea. I just submitted it: <http://news.ycombinator.com/item?id=1222192>

------
oomkiller
I'm not sure if I'm unique in this regard, but I seem to have the ability to
fully visualize complex systems and problems. The key for me is understanding
the problem fully, as I have to pull all of the pieces together. I find it
difficult sometimes because other people aren't used to working this way, so
they try to feed you bite-sized pieces, where what I really want is the entire
problem.

~~~
jacquesm
Could you put some lower bound on what you consider 'complex' ?

It seems a bit of an outlandish claim to make without a reference point,
what's complex to you may be simple to others and vice versa. So a bit of
extra data as to the kind of complex systems and problems you visualize would
help.

~~~
oomkiller
Sorry if it seems outlandish, I'm not really sure how quantify "complex". One
personal example though is I designed a national emergency management network
(Utilizing XMPP/PubSub, AMQP, Rails, etc) in my head without writing anything
down. That may seem special or it may not, I just find it rare, at least in my
(diminutive) peer group. I've also designed all of these huge dreamy systems
growing up, I have pictures and diagrams I drew in 5th grade :) They sure are
funny to look back at!

------
jacquesm
Explain the problem to someone that doesn't know anything about it.

Usually that 'clicks' at some point and I realize what I'd been missing all
along. The less the person knows about computers/programming (my usual subject
matter) the better it seems to work.

~~~
andrewljohnson
This "rubber duck" strategy is great for debugging too.

One person plays the rubber duck and sits there and says (at most) "uhuh,
uhuh" while the programmer with the bug explains the problem. Nine times out
of 10, the bugged programmer will do a facepalm and fix the bug immediately
while explaining the problem.

You actually don't need a person at all for this, or a rubber duck... just the
willingness to explain your bug aloud to yourself when you have one.

~~~
jacquesm
That's a really funny name for it. I don't think I'll alert my favorite
'victims' to it though, it might backfire :)

~~~
andrewljohnson
Yeah, it amused and educated me when I first read about it. Not my term:
[http://www.google.com/search?hl=en&q=rubber+duck+program...](http://www.google.com/search?hl=en&q=rubber+duck+programming)

------
diN0bot
norvig actually does make incremental progress in understanding the
problem/solution space in terms of constraints, bounds and correctness.

------
ntoshev
I have a programming problem, and I'm pretty sure I'm aware of the basic
components of the solution... it's just that I get swamped in complexity
pretty quickly when I try out different combinations.

I know this is a very meta question but this is not the only case. I'll solve
it eventually but I'd like to ask for common strategies.

Edit: Some of my strategies are: to absorb more information about similar
stuff while I put solving it on back burner, to try to describe the problem
clearly, to bounce the problem off other people.

\-------------------------

Okay, here is the specific question: I have a map of categories of things (a
few words) and descriptions what these categories mean (longer text). I also
have another list of category names that sometimes uses different words or
abbreviations, and I need to match the second list to the first one precisely.

I know all about tf-idf, vector space similarity between a document and a
term, wordnet as a source of semantic relations between words, and it's ok to
have a human map some of these. The problem is I don't know when a matching
score is good enough and when I need to fall back to a human.

~~~
sandaru1
Use Levenshtein distance - <http://en.wikipedia.org/wiki/Levenshtein_distance>

~~~
ntoshev
Thanks, my boolean approximation for abbreviations so far is:

    
    
      def abbr(short, full):
          return re.match(''.join(c+'.*' for c in short), full)
    

Actually what I'm doing now seems to be working, so far I can't see any
pattern in the things my algo can't match by itself.

Also thanks to ramanujan, who deleted his comment for some reason, but besides
pointing out orgmode which I want to check, his proposition reminded me that
I'm trying to deal with my dataset incrementally while a batch mode might work
better.

------
Tichy
Isn't the Norvig solution completely straightforward? Just the basic back
propagation algorithm?

So maybe in some cases it just helps to read the literature a little bit...

For example at the Google Code Competition (forgot the proper name), I am not
very good, but I learned some things from reading solutions of older
competitions. I think there even is a book dedicated to solving that kind of
problem. It would probably help a lot with writing a Sudoku solver, too.

Edit: not to take away from the Norvig solution, it is probably beautifully
crafted. But come on - it IS the straightforward solution? Or what am I
missing?

------
gfodor
There are multiple meanings of incremental progress. Just fleshing out a test
suite is only one means of defining progress.

------
kvs
How about solving related or similar problems? The way one may approach those
problems may give insight into incremental steps necessary to solve the hard
problem.

------
josh33
This is very high level, but: take a step back and remember who it is you are
trying to serve/help with the problem you are solving. If it is a particular
market/industry, often the answer can be found by trying to focus on how to
help them than by trying to solve a problem. I think there is something about
focusing on a problem that makes it bigger, whereas focusing on a solution can
make the solution more evident.

~~~
gridspy
I often find it instructive to solve a simple example problem by hand. This
can give you the insight needed to solve it computationally

------
thecombjelly
I try to attack the problem persistently and creatively. Often I will walk
around and just ask random people what they think. Sometimes even describing
the problem in non technical terms for the non techies to understand. People
usually have some ideas that you haven't thought of. I've found that any idea
can set you off on a path to solve the problem.

------
kmak
The part I don't understand is, for this particular problem, it is well-known
enough that a simple visit to Wikipedia would've been enlightening.

I remember reading up on it awhile ago, and it seems like the Wikipedia entry
already had the dancing links algorithm by then.

------
leif
For hard (math) problems, which is where I find it hardest to make incremental
progress, I usually just walk away, make myself some tea, or look for another
problem. Anything that stops me from consciously thinking about it allows my
subconscious to sort through the information, and eventually, the answer will
hopefully float to the top. If that doesn't happen after long enough, when I
do go back, I'll at least have gotten out of the circles I had been thinking
myself into.

------
smallhands
Why not pay somebody to solve it!!!!.

~~~
brk
I don't know why this comment is being modded down, this is a good idea (IMO).
Maybe not always possible in every scenario, but worth considering.

I've been doing more things with mturk, elance, RAC, and so on lately. For a
recent tradeshow we did I had someone on elance doing a short video production
for me for $150. The money was a no-brainer and allowed to concentrate on
other stuff. I also hired someone on elance (for $500) to write an xcode
framework for me for an iPhone app idea. I could probably figure it out
myself, but (I'm hoping) it will be easier and more efficient to get started
in iPhone app programming by having someone more knowledgeable than I put
together the basics so I can see how it's done. (Yes, I know there are a ton
of xcode examples on line, I just wanted something specific that I could build
off of).

Depending on the price/problem, outsourcing a solution might be an excellent
use of time/resources, and may also teach you something new seeing how another
person tackled the issue.

~~~
sfk
"I don't know why this comment is being modded down, this is a good idea
(IMO)."

Because some people do not think about everything in monetary terms.

------
vital101
I generally try to step away from the problem for awhile. Even if it's only
for 10 minutes. I come back with a fresh perspective that usually helps me get
through it.

------
msie
The link mentions a Norvig-Jeffries showdown on writing a Sudoku solver. Kinda
unfair since Norvig pretty much wrote the book on it. :P

~~~
andrewcooke
yeah, i think this needs to be better understood. norvig was not "solving"
much of anything there. he was simply writing out a solution for a problem he
had solved many times before.

that doesn't mean he's not a smart guy, but it's wrong to infer, from
comparing those articles, that he doesn't make mistakes.

------
NEPatriot
I look for inspiration from the world around me and focus on trying to always
find the simplest solution with the least resistance.

------
simsam
i sometimes browse this site: asknature.org

it is often a very strange approach and cannot be replicated by humans, but i
like the ideology.

