Scott Berkun posts about the tyranny of experts and other smart people, a recurring theme in my life.
From Studies in Intelligence, a declassified CIA journal:
The strengths of expertise can also be weaknesses. Although one would expect experts to be good forecasters, they are not particularly good at making predictions about the future. Since the 1930s, researchers have been testing the ability of experts to make forecasts. The performance of experts has been tested against actuarial tables to determine if they are better at making predictions than simple statistical models. Seventy years later, with more than two hundred experiments in different domains, it is clear that the answer is no. If supplied with an equal amount of data about a particular case, an actuarial table is as good, or better, than an expert at making calls about the future. Even if an expert is given more specific case information than is available to the statistical model, the expert does not tend to outperform the actuarial table.
The full essay talks about the paradox of using teams to balance against expert bias, the role of methedology, etc. The context of the essay is, you guessed it, CIA type intelligence gathering, but in reading this essay much of it applied to any kind of complex work.
The full essay is here.
Reminds me that Google’s hiring one heck of a lot of experts these days!
(Via Berkun blog.)