

Genetic Algorithm for Knapsack Problem - signa11
http://kataklinger.com/index.php/ga-knapsack/

======
scubbo
Nitpicking, I know, but I think the author has misinterpreted the XKCD strip.
IMO, " \- as fast as possible, of course. Want something on the Travelling
Salesman?" is not a continuation of the client's request (Give me set of
appetizers, fulfilling these requirements, as quickly as possible), but rather
a response to the waiter ("I have to see to these other tables" \- "and you
have to do so as quickly as possible"). This makes the mention of "Travelling
Salesman" more appropriate (since it deals with optimizing travel time for
certain visits).

Am I misinterpreting the dialogue?

(Not that it matters, of course!)

~~~
raverbashing
Every NP problem can be converted in another NP problem, so you could probably
try to convert your subset problem to a TSP problem (but it wouldn't be useful
in practice)

~~~
a-nikolaev
Nitpicking: In general, an NP problem can be reduced to an NP-hard problem (in
poly-time).

------
D_hemming
I'm mildly annoyed that they don't actually post the solution their algorithm
found.

~~~
tehwalrus
from what I understand from Koza about genetic programs, it will be
indistinguishable from garbage anyway.

That's one of the problems with Genetic Programming in general - you might get
a specific program optimised to a specific problem, but you never _learn_
anything about the problem space from it.

~~~
moron4hire
Genetic Programming is a slightly different thing than a Genetic Algorithm, a
subclass, really. I didn't read the code too closely on account that C++
template code makes me want to dig my eyes out, but he said he was doing a GA.
A GP is a GA where the genes represent Abstract Syntax Trees and the alleles
represent instructions.

~~~
tehwalrus
Oh, so this blogger is misusing the term (I scanned a little way down the
article and thought I'd concluded he was doing GP). In that case this is much
less interesting!

~~~
moron4hire
I don't get the need for the GA library here. It doesn't seem to have
simplified the problem at all. This code looks several orders of magnitude
more complex (or at least just ugly) compared to a hand-written GA that just
used std::vectors as the genes.

------
joakleaf
In reality this is a subset sum problem (unless I overlooked weights on the
appetizers).

Rather than using the GA for this, there are extremely efficient dynamic
programming methods available (in pseudo-polynomial time). References on the
Wikipedia page
([https://en.wikipedia.org/wiki/Knapsack_problem](https://en.wikipedia.org/wiki/Knapsack_problem)).

Since GA is a metaheuristic, it is really great if you don't know how to
tackle a problem more efficiently. But like most metaheuristics, not really an
ideal method for optimization.

To get a better visual sense of what GAs do wrong, take a look at:
[http://www.demo.cs.brandeis.edu/pr/buildable/crane/](http://www.demo.cs.brandeis.edu/pr/buildable/crane/)

You can see that the bridges and cranes have redundant bricks, are not
straight, and just generally messy. However, GA gets the job done.

There is a lot of enthusiasm over GAs, but it is just a refined form of random
search, and not that different from Simulated Annealing.

------
gamegoblin
How does this compare to a the typical branch-and-bound or dynamic programming
solution?

~~~
pkolaczk
It is very likely worse. GAs and ESs are good only if you can't come up with a
dedicated heuristic that understands the structure of the problem it aims to
solve.

------
willvarfar
Can you use the word 'solve' as they do in the article? As this only ever
gives a best-found-so-far answer, is that 'solving' it?

~~~
merlincorey
Also without knowing the details of the library or the implementation given's
interaction with said library whether or not this is capable of getting stuck
on local maximas and minimas (or perhaps already is).

------
drelihan
Are you taking Discrete Optimization in coursera (
[https://class.coursera.org/optimization-001/class/index](https://class.coursera.org/optimization-001/class/index)
)?

------
moron4hire
In my experience, genetic algorithms with a known target goal (i.e. your
fitness function is "how far away from the target is this gene?") are no
better than hill-climbing. The problem is that allele sharing is apparently
useless (I have never witnessed two wrong parents produce a right child, over
numerous experiments with GAs. I get the theory behind it, it just doesn't
seem to happen in the real world) and mutation--in order to provide enough
variation to find the answer in a short period of time--needs to be nearly
100%, at which point you might as well just iterate. With the propensity for
genetic algorithms to fall into local minima and maxima and the time it takes
to tweak all of the parameters of mutation and sharing and number of parents,
etc., you might as well have just hill-climbed the problem and gotten just as
good of an answer in a quarter of the time.

Genetic algorithms are useful if you have three conditions: your parameter
range is over a practically-infinite space, you know that hill-climbing will
end in a local min/max, and your fitness function is open-ended and relative
to previous generations, not an end goal. GAs _can sometimes_ get out of local
min/max on their own, so they offer a slightly better chances than randomly
distributing seeds in a hill climbing algorithm, and a very large parameter
range favors mutation and allele sharing over directed hill climbing. Finally,
lacking a known end point, you may actually be interested in the intermediate
results of the GA, rather than just the end result.

Anyway, GAs are completely worth studying, because they are fascinating tools
for problem solving for the right class of problem, just don't make the same
mistakes I did early in my playing that they are a general problem solving
algorithm.

~~~
mkehrt
I saw a phenomenal talk in grad school about why genetic algorithms are
generally not as good as stochastic hill climbing. Unfortunately, I can't
track down any references to it.

The punch line was that they mathematically proved that genetic algorithms
produce less optimal solutions than hill-climbing.

So why are "genetic algorithms" so common in nature? Well, it turned out that
genetic algorithms were better at producing solutions that were resilient to
changing environmental conditions. They produced a less sharp peak at the
solution they did find, so when a change occurred in the underlying
conditions, the still had a reasonable chance of being at least ok.

~~~
moron4hire
Yes, my personal experience bares that out. If one has the notion of trying a
genetic algorithm, one should first try hill-climbing. The GA is useful
insomuch as the intermediate results are interesting.

------
moron4hire
Well, I felt inspired to do my own GA program
[http://wedusc.com/ga/](http://wedusc.com/ga/)

with a discussion thread here:
[https://news.ycombinator.com/item?id=5909829](https://news.ycombinator.com/item?id=5909829)

