Content Moderation Case Study: Dealing With Misinformation During A Pandemic (2020)

from the misinfo-is-hard dept

This series of case studies is published in partnership with the Trust & Safety Foundation to examine the difficult choices and tradeoffs involved in content moderation. Learn more »

Summary: In early 2020, with the world trying to figure out how to deal with the COVID-19 pandemic, one of the big questions faced by internet platforms was how to combat mis- or disinformation regarding the pandemic. This was especially complex, given that everyone -- including global health experts were trying to figure out what was accurate themselves, and as more information has come in, the understanding of the disease, how it spread, how to treat it, the level of risk, and much, much, has kept changing.

Given the fact that no one fully understood what was going on, plenty of people rushed in to try to fill the void with information. Most social media firms put in place policies to try to limit or take down misinformation or disinformation using a variety of policies and tactics. But determining what is misinformation as opposed to legitimate truth-seeking can be very tricky in the midst of a pandemic.

In late March, as the pandemic was hitting full swing, an article appeared on the website Medium by Aaron Ginn, a self-described Silicon Valley “growth hacker,” arguing that the response to COVID-19 was overblown and driven by “hysteria.” The piece included many citations of credible data and reports, but also included a few quotes significantly downplaying the risk of COVID-19, including saying that its “transmission rates are very similar to seasonal flu.”

The story started to spread widely, mainly after a number of Fox News hosts started tweeting it. As the story got more and more attention, Carl Bergstrom, a professor of biology at the University of Washington, decided to critique Ginn’s article in great detail via an extended Twitter thread. Bergstrom makes a fairly compelling case that Ginn’s lack of expertise in epidemiology led him to making a number of mistakes in his analysis, in particular, not understanding how viruses spread, and how that information is tracked. He also argued that Ginn cherry-picked certain data to support a thesis. Bergstrom and others started arguing that Ginn’s Medium piece was (perhaps not intentionally) misinformation.

Decisions to be made by Medium:

• Should the piece be allowed to remain on the platform or should it be taken down?
• Are there alternatives to taking the piece down or leaving it as is?
• If the piece is taken down, should the user be banned?
• What policies do a potentially well-meaning, but possibly incorrect, piece about a pandemic violate?

Questions and policy implications to consider:

• How do you distinguish mis- and disinformation from potentially well-meaning, but inexperienced analysis?
• Should earnest but incorrect information be allowed?
• How does that analysis change when human health is at stake?
• Do pieces like this need to be judged against official government and healthcare sources? If so, how is that reconciled with constantly updated information from those sources?
• If articles like this are taken down, does that create more “credibility” for articles that remain on the site?
• If articles like this are left up, does that create more credibility problems for other articles on the site, that may have a much stronger and more competent analysis?
• Does taking down an article like this create political backlash?  If so, does that matter?
• Do the site and its management want to feel responsible if people take bad health advice that was posted to their site, and possibly come to harm from it?

Resolution: Medium chose to quickly take down Ginn’s piece about a day after it went up and 13 hours after it went “viral.” In fact, the article was taken down while Bergstrom was writing out his tweet thread critiquing it. Indeed, Bergstrom ended his thread early upon learning that the article was taken down.

That was not the end of things, however. The article was reposted to the site ZeroHedge, and copies were stored and reposted in other places as well. It also created a short-lived uproar among those who felt that Medium’s moderation decision was unfair. The Wall Street Journal celebrated Ginn, saying that after being “targeted for censorship,” it only made Ginn more influential. Other publications, including the NY Times, the Washington Post, and Slate, all wrote about the dangers of amateurish analysis in the midst of a pandemic.

Ginn, at one point, appeared to be fine with Medium’s decision, saying that internet platforms “are free to associate with whom they want,” (though he has since removed the tweet saying that). He has continued to use other social media to argue that the claims about COVID-19 were overblown.

Filed Under: aaron ginn, carl bergstrom, case study, content moderation, covid-19, misinformation
Companies: medium