Hacker News new | past | comments | ask | show | jobs | submit login

Could you summarize what exactly interesting insights did you gained by casting this problem as the election system? Unfortunately abstract fails to communicate this (I hate "teaser-only" abstracts!). Also, it might be easy to go from relative metric to sort of absolute by comparing against a "starndard" agent, for example, a random agent.

The #1 thing is it led to an elegant notion of how to compare intelligence (or rather, a parametrized family of intelligence comparators). There's a theorem called Arrow's Theorem, which basically says there's no good way to decide elections between more than 2 candidates. There are a handful of requirements an election-deciding method should have, and no method satisfies all those requirements. But Arrow's theorem has a loophole, discovered in the 1970s: if there are infinitely many voters, then there are methods of deciding the election which satisfy all Arrow's requirements. Economists in the 1970s exactly characterized these methods, in terms of a mathematical device called an "ultrafilter". Taking their characterization, a definition for relative intelligence is immediate--it writes itself.

If you're already familiar with ultrafilters, then the relative intelligence definition is SO elegant you're like, "Whoah. How can a definition of relative intelligence be that simple?" Of course, if you're not familiar with ultrafilters, then the definition is just as complicated as the definition of ultrafilters, which is quite complicated. So the definition of relative intelligence is like a simple computer program which imports from a complicated library in order to achieve that simplicity.

Taking their characterization, a definition for relative intelligence is immediate--it writes itself

A definition, but not a concrete and practical way of determining the relative difference between two agents?

Come on, bring this into reality. We're mostly a bunch of coders here. Stop talking about ultrafilters, concepts that are complex and thus render those of us not familiar with them unable to understand what you're saying.

Answer the question.. How can you take two real agents and, in finite time with finite resources, and in a completely general way, compare their intelligences?

The answer might be "the paper doesn't get us any closer to that." So just say that. Otherwise you're being misleading, because you start out by presenting the problem in a simple way. Then you get complex when you answer it.

The paper isn't intended to be a manual for how to practically compare agents. It will indirectly help there, I hope, by making people realize that they're looking at an election problem, and that there's a big existing literature on that subject. So in the practical case, say you have 10 different benchmarks, and some agents perform better at some of them, and others perform better at others. You could approach the problem from scratch, but it would be helpful for you to realize "oh this is an election with 10 voters and people have been studying how to decide elections for hundreds of years, I probably shouldn't reinvent the wheel". For example, it might take you many ages to essentially rediscover the Condorcet paradox and you might put inordinate effort into futilely trying to "solve" that paradox. Or you could stand on the shoulders of giants and avoid all that! https://en.wikipedia.org/wiki/Condorcet_paradox

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact