Rich and Co.

CrowdScourcing Beats ESPN Experts

This is likely also applicable to investing and portfolio management.  

Who can build a better fantasy football team – the highly-paid experts at ESPN, or the lowly-paid workers on crowdsourcing websites like Mechanical Turk?  Take a bow, crowd. According to an experiment that ran over the past American football season, the workers beat the pundits hands down.

in almost every other case, the ranking forecast by the crowd was closer to the final result.   The graph below shows the difference between ESPN and the crowd according to a “precision score” – basically a measure of the difference between the forecast and the actual result for different parts of the actual ranking. You can see that the crowd clearly outperforms ESPN for the 25 or so highest-ranked players. The difference narrows after that, but the pundits never overtake the crowd.

Why was the advantage greatest for the better players?  “The simple answer is that:

  • we got answers from a large number of individualsso the influence of one individual’s bias is smaller
  • People who were uninformed would tend to cancel each other out, so any significant trend would be meaningful. We had a much larger pool than ESPN did
  • And because our crowd responded independently of each other, they were less likely to be influenced by groupthink than the ESPN experts.”

Written by Rich and Co.

February 4, 2011 at 12:36 pm

Posted in Uncategorized

%d bloggers like this: