Friday, August 03, 2007

The Wisdom of Crowds: A Review

Science fiction/Fantasy writer Piers Anthony wrote a series of books call the Incarnations of Immortality, of which the lead book was called On a Pale Horse about a man names Zane, who becomes the incarnation of Death (who rides a pale hourse). About midway through the book, Nature (another incarnation) and Zane carry on a coversation, a lecture really, about the nature of thinking. During the lecture, Nature tells him of sequential thinking, diagrammed like this -----, thinking in parallel --==-, and intuitive thinking --|||--. The discussion becomes important in the novel's climax as Zane tries to overcome the crux of the novel.

In many ways, Juames Surowiecki's book, the Wisdom of Crowds, reminds me of the parallel thinking segment, where working on different parts of a problem can often lead to a solution much faster. Surowiecki's premise, that experts are no more likely and indeed less likely, to arrive at a correct answer than a group of free thinking, but informed individuals. Looking at situations as diverse as picking a President to playing on Who Wants to be a Millionaire to the manner in which we navigate crowed sidewalks, Surowiecki looks at instances where the wisdom of the grouop is much more effective and efficient than a single expert.

In order for a crowd to work effectively, Surowiecki asserts that it needs four specific criteria--and no crowd size is not one of them--Independence from the other members, diversity of opinion (groupthink leads to bad decisions), decentralized thinking and operation and a effective method for aggregating the opinions. throughout the book, while explaining these requisites, Surowiecki discusses, using real life examples, of how the absence of these traits leads to poor decision making processes and outcomes.

The application of Surowiecki's theory is practially limitless, but in the current political context, the application can be useful. There is an old saw that says if you are dumb, surround yourself with smart people and if you are smart, surround yourself with smart people who disagree with you. The current political debate surely could benefit from a few more dissenting opinions, both for the Administration and for Congress. By having more dissenting views in a room when decisions are made can lead to better policy and better governing. Of course, in a partisan atmosphere, such independence of thought in either the White House or the offices of Congress simply is not tolerated and the American people suffer.

Despite our belief in the wisdom of experts, and we have many of them and they often make good judgments, Surowiecki is not necessarily knocking the existence of experts because they provide important insights and information. What Surowiecki argues is that experts should only be a part of some decision making processes and that we should simply defer to them on the basis of their expertise. Although he only hints at it, Surowiecki was actually on the verge of an important point.

Not only is the wisdom of crowds useful for problem solving but also has a wonderful predictative ability. The proliferation of polls, particularly political polling, is not just about assessing the mood, but a well designed poll can give hints as to what the future holds as well. Although many political pundits may dismiss polls (and dismissing poorly designed polls is proper), a well designed poll gives a pretty good snapshot in time of what is happening and what will happen in the near future. Often times, when a crowd turns out to be wrong, it is based upon unforseen circumstances, events that could not be anticipated (9/11 for instance) or a lack of information about the decision. While perfect information is impossible though, the crowd does a pretty good job of determining what is going to happen based upon limited information.

Surowiecki's book can get bogged down in details of a given situation, perhaps its most glaring weakness. At other times, Surowiecki, comfortable with his knowledge, fails to bring the reader along with him. Often times, the descriptions of psychological and economic experiments fail to properly illustrate his point or leave the reader wondering what is happeing.

Many of the situations and institutions discussed in the book lie at the heart of American society. The foundations of our society, that of a representative democracy, capitalism and the open market, public opinions and mores, and other examples of community and collective action, are based on the wisdom of crowds. Indeed, the fundamental precept of our government is the belief in the wisdom of ordinary people to govern themselves. But the true power of the wisdom of crowds is not just parallel thinking, --===--, but a combination of parallel thinking combined with intuitive thinking, that is dozens, hundredes, thousands, even millions of people coming to conclusions with incomplete information using parallel intuitive thinking. That is after all, how we behave on a regular basis, making decisions with imperfect information. A crowd simply aggregates that decision making into a collective wisdom.

No comments: