Online ratings, likes inflated by herd effect, study finds

Users are highly influenced by the opinions of others online, and new research describes "irrational herding" that gives outsized influence to positive ratings -- regardless of the quality of the content.

It may seem obvious that sometimes the number one song or movie is popular not because it's good, but because people say it's good, and that's been true before hype and "social influence bias" moved online.

Lev Muchnik of the Hebrew University of Jerusalem School of Business Administration, Sinan Aral of MIT's Sloan School of Management and Sean J. Taylor of NYU's Stern School of Business collaborated with an unnamed social news site to conduct a large controlled experiment. Their findings are published in Science.

On a social news site described as "similar to Digg.com and Reddit.com," on which users can give links and comments upvotes and downvotes, researchers tracked 101,281 comments made by users over the course of five months.

The comments were viewed more than 10 million times, and rated by other users a total of 308,515 times.

For the experiment, every time a user left a comment, it was automatically given an upvote, a downvote, or no vote at all for the control group.

Comments that automatically received an upvote were 32 percent more likely than the control group to receive another upvote by the first user to see them.

Those comments were more like to accumulate more positive ratings more quickly, and by the end of the experiment, the upvote group averaged a 25 percent higher rating than the control group.

Comments that automatically received a downvote were also more likely than the control group to receive a positive vote from the first person to see it -- in what the researchers describe as a "correction effect."

"People are more skeptical of negative social influence," Aral said. "They're more likely to 'correct' a negative vote and give it a positive vote." This effect actually ends up neutralizing many negative social influences, while positive social influences gather "herd effect" momentum.

"If perceptions of quality are biased by social influence," the authors write, then aggregation of collective judgment could be manipulated, with "dramatic consequences for our markets, our politics, and our health."

Interestingly, the authors found that comments appearing in threads related to business, culture, society and politics exhibited far greater herding effects than those in economics and general news threads.

The herding effect also happens on consumer review sites like Yelp and Amazon, though researchers haven't yet conducted a study of those ratings.

"Our message is not that we should do away with crowd-based opinion aggregation," Aral said. "Our point is that you need solid science under the hood trying to understand exactly how these mechanisms work in a broad population, what that means for the diffusion of opinion, and how can we design the systems to be fair, to have less incentives for manipulation and fraud, and be safe in aggregating opinions."