
The Knapsack Problem - specular
http://bertolami.com/index.php?engine=blog&content=posts&detail=knapsack-problem
======
Domenic_S
Does this actually answer the stated problem of "deciding the subset of items
to pack"?

Looking at the solution table I can see that there are two possible solutions,
and I can see a max _value_ of 6. But I can't see how v[4,3] implies packing
objects 1 and 3.

~~~
Retric
You can read the chart backwards to get an answer.

Check 2 vs 3 @4lb. That's constant so 3 is not in there.

Check 1 vs 2 @4lb. It changes so ob2 is in there. Now 3 weighs 2lb so look at
4lb - 2lb(ob3) = 2lb max weight.

Check 0 vs 1@2lb. That's increasing so object 1 is in there 2lb - 1lb (ob1) =
1lb max.

Check 0 @1lb that's greater than zero so object 0 is also in there.

Thus, the solution is objects 0,1,2 but not 3.

PS: Note, this only finds one solution there might be several. You can quickly
find them with this chart but it's somewhat more complex. You would be looking
for places where the max value does not change from the previous row, but the
max value in the same row - the object weight = the max value - the object
value.

------
mgraczyk
I wrote a solution using this table building DP algorithm a few years ago.

    
    
        https://github.com/mgraczyk/DiscreteOptimization/blob/master/knapsack/speed_up/solve_it.cpp
    

This was part of an assignment for a Coursera Discrete Optimization course
created by The University of Melbourne. It's a great course and I recommend it
to anybody who wants to hone their understanding of dynamic programming and
solving computationally expensive problems.

    
    
        https://www.coursera.org/course/optimization

~~~
hkon
I took this course as well. Really interesting. And a really cool way to
submit programming assignments and get instant feedback if you "passed" or
not. Some years since I was in school, this might be the de-facto way of doing
it now.

------
chris_va
For those of you planning on interviewing...

Having done interviews for a large tech firm, dynamic programming questions
are a favorite for weeding out candidates. Anecdotally, it correlates rather
well with problem solving skills.

~~~
nilkn
Years ago, Joel Spolsky wrote that it was critical for any interview to at
least touch on pointers/references and recursion. I think at many major tech
companies dynamic programming is included in that bag as well.

~~~
JoeAltmaier
And yet, I can count on one finger the number of recursive algorithms I have
ever, ever used in 20 years of programming.

~~~
kasey_junk
And I'm sure you can find people who have programmed extensively for the last
20 years and have not needed to deal with pointers directly.

~~~
nilkn
I'd be pretty impressed if someone had programmed extensively for two decades
and yet had never dealt with references of some kind. It doesn't have to
explicitly be pointers in C.

~~~
kasey_junk
The above referenced Joel On Software article is called "The Perils of
JavaSchools" and does in fact make an explicit difference between references
and C style pointers.

~~~
nilkn
I, however, explicitly mentioned references in both my posts. I was referring
to a general idea, not a specific article. I've also personally interviewed at
Fog Creek in Manhattan and was asked reference-based questions that I was
allowed to answer in Java rather than C.

Regardless, if you want to reference that specific article, and only that
article, I can do that as well. It happens to refute your very own point:

"Now, I freely admit that programming with pointers is not needed in 90% of
the code written today, and in fact, it's downright dangerous in production
code. OK. That's fine. And functional programming is just not used much in
practice. Agreed.

[...]

But beyond the prima-facie importance of pointers and recursion, their real
value is that building big systems requires the kind of mental flexibility you
get from learning about them, and the mental aptitude you need to avoid being
weeded out of the courses in which they are taught. Pointers and recursion
require a certain ability to reason, to think in abstractions, and, most
importantly, to view a problem at several levels of abstraction
simultaneously. And thus, the ability to understand pointers and recursion is
directly correlated with the ability to be a great programmer."

Finally, you tacitly assumed you knew which article I had in mind, and you
were wrong, not that it matters since I wasn't actually trying to reference a
specific article. I was referring to The Guerrilla Guide to Interviewing. He
offers example questions like "reverse a linked list" or "detect loops in a
tree structure" \-- concepts and problems which are clearly not relegated only
to C.

~~~
kasey_junk
If you think that quote refutes my point then I clearly need to clarify what
my point was.

1) Joel believes that pointers and recursion are concepts that require a way
of thinking about higher order abstractions that correlates well with being a
good programming hire. So it would make sense if we agreed with him to ask
questions about pointers & recursion in interviews, even if we don't believe
that pointers or recursion are used often in the settings we are hiring for.
2) Many people have built long careers without using recursion, but that
doesn't refute the premise because it doesn't speak to the higher order
abstractions assumption. Further, in the modern world recursion is no more or
less likely to be used than pointers (which would be shocking at one point in
the industry) as pointers also are not common. 3) The central assumption Joel
basis his argument on is that pointers and recursion are interesting because
reasoning well about them requires a higher level of abstraction. That higher
level of abstraction simply does not exist with Java style references. So
asking about references, while maybe useful in other contexts, doesn't have
the benefits Joel is arguing for in the same way as pointers. 4) To relate
this back to the article, A) is understanding dynamic programming indicative
of some central mental type that will correlate with a good programmer? B)
Does asking questions about the Knapsack problem suss out that understanding.
I suspect, but can't prove that the answers are A) not like pointer/recursion
understand and B) absolutely not.

------
codehero
Off the top of my head I look at it as a graph 2 coloring problem. The graph
starts out with 1 node per item and zero edges. If we think two items will be
in the same set (either in the knapsack or out) we merge the nodes and their
weight and value combine into the new node. We draw edges between two nodes if
their combined weights exceed the knapsack capacity. When we recurse out of
our merge choice, we also draw an edge between the two nodes that composed the
merge. Our recursion depth ends at a clique.

------
hayksaakian
Interesting problem. If you play a game like elder scrolls skyrim or oblivion,
you run into this problem literally.

Without reading, I already knew the solution.

To solve it in your head: sort items by most value/weight efficient to least
efficient.

Then take as much as you can until you're full.

(in a game like elder scrolls oblivion, that means jewelry first, and silver
vases last)

\-----

edit: I am mistaken, as some of the below comments accurately pointed out.

~~~
crazypyro
This is a "false" solution, in the sense that you can create a large number of
scenarios where just taking the most value/weight efficient item, regardless
of the relationship between the items, will result in a non-optimal solution.

e.g. 2 Silver coins worth 4 and weighs 2. 1 Gold coin worth 5 and weighs 3.
Silver slightly more efficient, but if your bag size is 5, the obvious
solution is 1 gold + 1 silver coin worth 9, where as your "algorithm" would
give 2 silver coins weighing 8, worth 8.

~~~
hayksaakian
I guess my solution was more approximate but easier to reason about. You're
right, it falls apart on the borders of max capacity.

------
pramalin
May be relevant: there is a cool game - knapsack on a radial graph, with Scala
source code. [http://krishnanraman.github.io/scala-
js/examples/helloworld/...](http://krishnanraman.github.io/scala-
js/examples/helloworld/helloworld.html)

------
Retric
This still can easily be worse than n^2 if your weight is significantly higher
than your number of objects. Aka max weight of 10,000 and 1,000 objects.

Edit: Note, n^2 is generally far better than 2^n so it's still useful even if
it's slow.

~~~
alta22433
Yeah, seems like the article implies an upper bound of 2^n for a greedy
solution, so your example would be pretty bad with a worst case of 2^1000?

~~~
Retric
My example would be 1000 objects * 10,000 weights = 10,000,000 options. Which
is much better than 2^(1000) ~= 10^300 for calculating every possibility, but
still worse than n^2.

For it to be worse you would need say 10 objects and 10,000 for max weight. so
2^n = 1024 vs n * k = 10 * 10,000 = 100,000

------
alta22433
I was asked this on a recent programming interview. As a recent college
graduate, I'm not sure this is a fair question to ask under that context given
the complexity of the problem. Nice article though.

~~~
tptacek
Unless the role you're being interviewed for is "Algorithm Designer", this is
a really stupid interview question.

~~~
crazypyro
It's one of those stupid questions where you either have seen it before and
use memorization to pull it up or you flounder because you can't come up with
a new algorithm on the spot.

------
lunz
This is a domain where Unconventional Computing seems to work well, e.g., "DNA
computing" through its massively parallel processing capabilities.

