Penn prof proves wisdom of crowds can prevail
PennFollowJun 21, 2017
By Katherine Unger Baillie
Groups in which everyone has equal influence made better predictions than groups in which a single individual was deemed an opinion leader.
Anyone following forecasting polls leading up to the 2016 election likely believed Hillary Clinton would become the 45th president of the United States. Although this opinion was the consensus among most political-opinion leaders and media, something clearly went wrong with these prediction tools.
Though it may never be known for certain the reasons for the discrepancy between public perception and the electoral reality, new findings from the University of Pennsylvania’s Damon Centola may offer a clue: the wisdom of a crowd is in the network.
The classic “wisdom of crowds” theory goes like this: If we ask a group of people to guess an outcome, the group’s guess will be better than any individual expert. Thus, when a group tries to make a decision, in this case, predicting the outcome of an election, the group does a better job than experts. For market predictions, geopolitical forecasting and crowdsourcing product ideas, the wisdom of crowds has been shown to even outperform industry experts.
That is true — as long as people don’t talk to each other. When people start sharing their opinions, their conversations can lead to social influences that produce “groupthink” and destroy the wisdom of the crowd. So says the classic theory.
But Centola, an associate professor in Penn’s Annenberg School for Communicationand School of Engineering and Applied Science and director of the Network Dynamics Group, discovered the opposite. When people talk to each other, the crowd can get smarter. Centola, along with Ph.D. candidate Joshua Becker and recent Ph.D. graduate Devon Brackbill, published the findings in Proceedings of the National Academy of Sciences. The paper is available for download here: http://ndg.asc.upenn.edu/experiments/collective-intelligence/.
“The classic theory says that if you let people talk to each other groups go astray. But,” said Centola, “we find that even if people are not particularly accurate, when they talk to each other, they help to make each other smarter. Whether things get better or worse depends on the networks.
“In egalitarian networks,” he said, “where everyone has equal influence, we find a strong social-learning effect, which improves the quality of everyone’s judgements. When people exchange ideas, everyone gets smarter. But this can all go haywire if there are opinion leaders in the group.”
An influential opinion leader can hijack the process, leading the entire group astray. While opinion leaders may be knowledgeable on some topics, Centola found that, when the conversation moved away from their expertise, they still remained just as influential. As a result, they ruined the group’s judgment.
“On average,” he said, “opinion leaders were more likely to lead the group astray than to improve it.”
The online study included more than 1,300 participants, who were placed into one of three experimental conditions. Some were placed into one of the “egalitarian” networks, where everyone had an equal number of contacts and everyone had equal influence. Others were placed into one of the “centralized” networks, in which a single opinion leader was connected to everyone, giving that person much more influence in the group. Each of the networks contained 40 participants. Finally, Centola had several hundred subjects participate in a “control” group, without any social networks.
In the study, all of the participants were given a series of estimation challenges, such as guessing the number of calories in a plate of food. They were given three tries to get the right answer. Everyone first gave a gut response.
Then, participants who were in social networks could see the guesses made by their social contacts and could use that information to revise an answer. They could then see their contacts’ revisions and revise their answers again. But this time it was their final answer. Participants were awarded as much as $10 based on the accuracy of their final guess. In the control group, participants did the same thing, but they were not given any social information between each revision.
“Everyone’s goal was to make a good guess. They weren’t paid for showing up,” Centola said, “only for being accurate.”
Patterns began to emerge. The control groups initially showed the classic wisdom of the crowd but did not improve as people revised their answers. Indeed, if anything, they got slightly worse. By contrast, the egalitarian networks also showed the classic wisdom of the crowd but then saw a dramatic increase in accuracy. Across the board, in network after network, the final answers in these groups were consistently far more accurate than the initial “wisdom of the crowd.”
“In a situation where everyone is equally influential,” Centola said, “people can help to correct each other’s mistakes. This makes each person a little more accurate than they were initially. Overall, this creates a striking improvement in the intelligence of the group. The result is even better than the traditional wisdom of the crowd! But, as soon as you have opinion leaders, social influence becomes really dangerous.”
In the centralized networks, Centola found that, when the opinion leaders were very accurate, they could improve the performance of the group. But even the most accurate opinion leaders were consistently wrong some of the time.
“Thus,” Centola said, “while opinion leaders can sometimes improve things, they were statistically more likely to make the group worse off than to help it.
“The egalitarian network was reliable because the people who were more accurate tended to make smaller revisions, while people who were less accurate revised their answers more. The result is that the entire crowd moved toward the more accurate people, while, at the same time, the more accurate people also made small adjustments that improved their score.”
These findings on the wisdom of crowds have startling real-world implications in areas such as climate-change science, financial forecasting, medical decision-making and organizational design.
For example, while engineers have been trying to design ways to keep people from talking to each other when making important decisions in an attempt to avoid groupthink, Centola’s findings suggest that what matters most is the network. A group of equally influential scientists talking to one another will likely lead to smarter judgments than might arise from keeping them independent.
“By designing informational systems where everyone’s voices can be heard, we can improve the judgment of the entire group. It’s as important for science as it is for democracy.” — Damon Centola
He is currently working on implementing these findings to improve physicians’ decision-making. By designing a social network technology for use in hospital settings, it may be possible to reduce implicit bias in physicians’ clinical judgments and to improve the quality of care that they can offer.
Whether new technologies are needed to improve the way the groups talk to each other, or whether we just need to be cautious about the danger of opinion leaders, Centola said it’s time to rethink the idea of the wisdom of crowds.
“It’s much better to have people talk to each other and argue for their points of view than to have opinion leaders rule the crowd,” he said. “By designing informational systems where everyone’s voices can be heard, we can improve the judgment of the entire group. It’s as important for science as it is for democracy.”
The work was supported in part by a grant from the Robert Wood Johnson Foundation.
Katherine Unger Baillie is a science news officer at Penn’s Office of University Communications.
The University of Pennsylvania is one of the oldest universities in America and one of the most prestigious institutions of higher learning in the world.