But nature also evolved a bunch of other organisms alongside humans that are fit in their own way. How do you evolve for just the (human level) intelligent entity and not a cockroach or rat?
Point is that you're going to end up with a massive ecosystem of organisms if you could fully simulate an evolutionary history. It's not even clear that human level intelligence would evolve. It's only happened once in our planet's history. We might be flukes.
You might have to run the massive simulation a thousand times and somehow be able to pinpoint the human-level intelligence when it does succeed.
One would have to figure out which evolutionary pressures to apply in order to select for the desired traits. For example, at some point in the past, humans and rats had a common ancestor. That ancestor had two offspring. One offspring's descendants faced a series of evolutionary pressures that led to rats. The other's descendants faced a different series of pressures, leading to humans. The designer of a multi-stage evolutionary algorithm could choose which evolutionary pressures to apply, and when to apply them. The designer could thus shape the evolution of the program in the desired direction.
The trick is knowing what pressures to apply in order to get the desired results. How could one do that? I have no idea! But if someone can figure it out, perhaps they could mimic what nature has blindly done.
EDIT: As for the worry of ending up with a menagerie of programs, there are a couple of things that I think could help. First, the designer must be careful not to take the natural-selection analogy too far. The real world has numerous diverse environments, each with myriad niches; and in each niche, different evolutionary pressures shape organisms. This sort of algorithm need not provide all these diverse niches. It could apply a single set of pressures to all the programs in it, pushing them either in the desired direction or to extinction. Second, if the designer could identify evolutionary states that are a "dead end" and unlikely to evolve in the desired direction, they could remove the dead-end programs. Whether such dead-end states can be reliably identified is anyone's guess.
Point is that you're going to end up with a massive ecosystem of organisms if you could fully simulate an evolutionary history. It's not even clear that human level intelligence would evolve. It's only happened once in our planet's history. We might be flukes.
You might have to run the massive simulation a thousand times and somehow be able to pinpoint the human-level intelligence when it does succeed.