If you peruse the Bloomberg wire and keep track of stock analyst predictions, this site could grow into a "forecast analyzer" of stock "experts." Of course, someone needs to login and edit whether the prediction was correct.
Actually, I remember reading sth about a hedge fund that employed just that approach. They had every analyst or broker send in their predictions or collected them from somewhere and kept track of who did well and who did not. The better someone did (the article did not say which measure the fund used), the more business the fund would send their way (in terms of order flow), betting on their recommendations to go long or short. Presumably, their portfolio would have been optimized to account for forecast accuracy, expected gains, and some measure of risk. The difficult question here would be which measure to take as "forecast accuracy", and that also touches on the site. How wrong is wrong, and how right is right? You would need some measure of dispersion to judge whether someone was right. If there is a lot of noise in some variable (say the oil price), then basically anyone will have been right sometime when the forecast horizon is long enough. On the other hand, you couldn't really say someone was "wrong" in his forecast if he said X and, by the measure of dispersion used, the actual number lies within reasonable bounds around X. Another question I already mailed the site's creator is how the variables are actually defined and tracked. Yet another question is whether and how the forecasters can adjust their forecasts. Obviously, you should be able to update your 5y forecast if tomorrow something happens that topples your assumptions. It's totally legit to do that, and I would argue such a mechanism is the real point of the otherwise pointless forecasting game.
EDIT: There is one more question I find interesting. The measure by which forecast quality is judged should also somehow account for time dimension. Concretely, if the process you are tracking conforms 9 times out of 10 with your forecast path and then "explodes" or "collapses", are you good or bad? How far does it have to collapse or explode that you are "bad", how is the situation if the granularity is finer and you can observe 100 times instead of 10 (so the process is 98 or 99 times fine and then explodes)?
I didn't say "one" hedge fund pursues the strategy. I said I read about one. Also, aggregators do the exact opposite of what this hedge fund was doing: averaging. Whereas the point would be to measure their predictive accuracy individually.
