Our Latest Techdirt Greenhouse Panel: Broadband In The Age Of Covid

from the life-gives-you-lemons dept

Let's be clear: despite ample rhetoric to the contrary, U.S. broadband has always been a mediocre mess. Despite decades of incalculable industry handouts and political lip service, 42 million Americans (double official FCC estimates) still lack access to any broadband whatsoever. 83 million Americans are trapped under a broadband monopoly. Tens of millions more Americans are stuck with a duopoly, usually a combination of a cable giant and a phone company unwilling to upgrade or even repair aging DSL lines because it's not profitable, quickly enough, for Wall Street's liking.

The triple whammy of limited competition, regulatory capture, and Congressional corruption has resulted in the U.S. being utterly mediocre (or worse) in nearly every major broadband metric that matters. Billions upon billions in state and federal subsidies, tax breaks, and regulatory favors are thrown at massive monopolies that fail to live up to their end of the bargain time and time again. Worse, government efforts to track and report on broadband coverage and pricing data has been substandard for a generation, often by intentional design (after all, you don't have to fix a problem you refuse to measure or even acknowledge).

As a result, U.S. consumers pay some of the highest prices in the developed world for slower broadband than many overseas nations. Telecom industry customer service is ranked among the worst of any industry in America. Marginalized and low-income communities are most often left disconnected, but even in "tech-centric" cities like San Francisco and Seattle, monopolization and substandard service is often the accepted norm. America's a country that likes to crow about its technological supremacy, yet toddlers in the COVID era have been forced to huddle in the dirt outside of fast food restaurants just to get online for class.

Instead of rising to this challenge, we've repeatedly doubled down on the very practices that helped create the problem in the first place. Whether it's the rubber stamping of competition and job-eroding megamergers and consolidation, the steady elimination of meaningful state and federal oversight of telecom monopolies, or simply refusing to adopt policies that drive meaningful competition to market, we're locked in a cycle of dysfunction that repeatedly prioritizes monopoly revenues over public welfare, market health, or technological innovation.

It doesn't have to be this way. As parents, students, and employees alike run face-first into a problem we've simply refused to fix, the COVID crisis creates a unique opportunity to rethink our approach to broadband delivery with an eye on accurate data, affordability, and the public good.

With that as backdrop, we've decided it would be timely to embrace broadband access (or the lack thereof) as our latest Techdirt Greenhouse panel. So we've collected a diverse array of international authors on the front lines of bridging the digital divide, deploying bleeding edge technological and regulatory solutions for a problem decades in the making. I'm hopeful this collection of industry leaders, activists, executives, and others can shine a little light on how we got here and where we need to go, using the urgency of the COVID crisis to drive meaningful, localized solutions to a problem that should have been solved long ago.

As we noted during our last two panels on content moderation and privacy, we want Techdirt Greenhouse to be a conversation, not a unidirectional bullhorn. As such, if you'd like to participate -- or are an expert eager to respond to any of the pieces posted over the coming weeks -- please feel free to reach out.

Filed Under: broadband, covid-19, digital divide

That's A Wrap: Techdirt Greenhouse Content Moderation Edition

from the building-a-better,-more-ethical-internet dept

When we launched Techdirt Greenhouse, we noted that we wanted to build a tech policy forum that not only tackled the thorniest tech policy issues of the day, but did so with a little more patience and nuance than you'll find at many gadget-obsessed technology outlets. After our inaugural panel tackled privacy, we just wrapped on our second panel subject: content moderation. We'd like to thank all of those that participated in the panel, and all of you for reading.

You'd be hard pressed to find a thornier, more complicated subject than content moderation. On one hand, technology giants have spent years prioritizing ad engagement over protecting their user base from malicious disinformation and hate speech, often with fatal results. At the same time, many of the remedies being proposed cause more harm than good by trampling free speech, or putting giant corporations into the position of arbiters of acceptable public discourse. Moderation at this scale is a nightmare. One misstep in federal policy and you've created an ocean of new problems.

Whether it's the detection and deletion of live-streaming violence, or protecting elections from foreign and domestic propaganda, it's a labyrinthine, multi-tendriled subject that can flummox even experts in the field. We're hopeful that this collection of pieces helped inform the debate in a way that simplified some of these immensely complicated issues. Here's a recap of the pieces from this round in case you missed them:

Much like the privacy debate, crafting meaningful content moderation guidelines and rules (and ensuring consistent, transparent enforcement) was a steep uphill climb even during the best of times. Now the effort will share fractured attention spans and resources with an historic pandemic, recovering from the resulting economic collapse, and addressing the endless web of socioeconomic and political dysfunction that is the American COVID-19 crisis. But, much like the privacy debate, it's an essential discussion to have all the same, and we hope folks found this collection informative.

Again, we'd like to thank our participants for taking the time to provide insight during an increasingly challenging time. We'd also like to thank Techdirt readers and commenters for participating. In a few weeks we'll be announcing the next panel; one that should prove timely during an historic health crisis that has forced the majority of Americans to work, play, innovate, and learn from the confines of home.

Filed Under: content moderation, tradeoffs

If Lawmakers Don't Like Platforms' Speech Rules, Here's What They Can Do About It. Spoiler: The Options Aren't Great.

from the no-easy-answers dept

What should platforms like Facebook or YouTube do when users post speech that is technically legal, but widely abhorred? In the U.S. that has included things like the horrific video of the 2019 massacre in Christchurch. What about harder calls – like posts that some people see as anti-immigrant hate speech, and others see as important political discourse?

Some of the biggest questions about potential new platform regulation today involve content of this sort: material that does not violate the law, but potentially does violate platforms’ private Terms of Service (TOS). This speech may be protected from government interference under the First Amendment or other human rights instruments around the world. But private platforms generally have discretion to take it down.

The one-size-fits-all TOS rules that Facebook and others apply to speech are clumsy and unpopular, with critics on all sides. Some advocates believe that platforms should take down less content, others that they should take down more. Both groups have turned to courts and legislatures in recent years, seeking to tie platforms’ hands with either “must-remove” laws (requiring platforms to remove, demote, or otherwise disfavor currently lawful speech) or “must-carry” laws (preventing platforms from removing or disfavoring lawful speech).

This post lays out what laws like that might actually look like, and what issues they would raise. It is adapted from my ”Who Do You Sue” article, which focuses on must-carry arguments.

Must-carry claims have consistently been rejected in U.S. courts. The Ninth Circuit’s Prager ruling, for example, said that a conservative speaker couldn’t compel YouTube to host or monetize his videos. But must-carry claims have been upheld in Poland, Italy, Germany, and Brazil. Must-remove claims, which would require platforms to remove or disfavor currently legal speech on the theory that such content is uniquely harmful in the online environment, have had their most prominent airing in debates about the UK’s Online Harms White Paper.

The idea that major, ubiquitous platforms that serve as channels for third party speech might face both must-carry and must-remove obligations is not new. We have long had such rules for older communications channels, including telephone, radio, television, and cable. Those rules were always controversial, though, and in the U.S. were heavily litigated.

On the must-remove side, the FCC and other regulators have prohibited content in broadcast that would be constitutionally protected speech in a private home or the public square. On the must-carry side, the Supreme Court has approved some carriage obligations, including for broadcasters and cable TV owners.

Those older communications channels were very different from today’s internet platforms. In particular, factors like broadcast “spectrum scarcity” or cable “bottleneck” power, which justified older regulations, do not have direct analogs in the internet context. But the Communications law debates remain highly relevant because, like today’s arguments about platform regulation, they focus on the nexus of speech questions and competition questions that arise when private entities own major forums for speech. As we think through possible changes in platform regulation, we can learn a lot from this history.

In this post, I will summarize some possible regulatory regimes for platforms’ management of lawful but disfavored user content, like the material often restricted now under Terms of Service. I will also point out connections to Communications precedent.

To be clear, many of the possible regimes strike me as both unconstitutional (in the U.S.) and unwise. But spelling out the options so we can kick the tires on them is important. And in some ways, I find the overall discussion in this post encouraging. It suggests to me that we are at the very beginning of thinking through possible legal approaches.

Many models discussed today are bad ones. But many other models remain almost completely unexplored. There is vast and under examined territory at the intersection of speech laws and competition laws, in particular. Precedent from older Communications law can help us think that through. This post only begins to mine that rich vein of legal and policy ore.

In the first section of this post, I will discuss five possible approaches that would change the rules platforms apply to their users’ legal speech. In the second (and to me, more interesting) section I will talk about proposals that would instead change the rulemakers – taking decisions out of platforms’ hands, and putting them somewhere else. These ideas are often animated by thinking grounded in competition policy.

Changing the Rules

Here are five possible approaches that lawmakers could use to change the rules platforms apply to speech that is currently legal, but potentially prohibited under Terms of Service. Many of these models have both must-carry versions (preventing platform removal, demotion, demonetization, etc.) and must-remove versions (requiring those things). Most could be combined with others – in particular, by applying new rules only to the very largest internet platforms.

Ultimately, I am pessimistic about any of these approaches. Some would clearly be unconstitutional in the U.S., because of the new role they would give to the government in regulating protected speech. But I know that others disagree, since all of these models have been raised in recent discussions of platform regulation.

1. “Common Carriage” Rules for Major Platforms

Some must-carry proponents suggest that edge platforms like Facebook or YouTube should be treated like government-operated public forums, or like privately-operated common carriers. On this model, platforms would be bound to deliver, and give equal treatment to, any lawful user speech. This must-carry idea doesn’t really have an analog on the must-remove side.

I’m teeing it up first because I think it is a major red herring, particularly in the U.S. discussion. Plenty of platform critics act like they want common carriage rules, but I don’t think many of them actually do, for reasons I’ll discuss here. In any case, such a massive legal change has little future as a political or constitutional matter. That leaves “common carriage” proponents likely, in practice, supporting one of the other models I’ll discuss next, like “indecency” rules or “fairness” rules.

The argument for common carriage is most compelling when applied to the internet ecosystem as a whole. Huge components of the internet, including major infrastructurel components like CloudFlare or DNS, are in the hands of private companies. If all of them can exclude legal but unpopular speech – and especially if all of them face pressure from customers, advertisers, and investors to do so —where is that speech supposed to go? Is the public interest served, and are users’ rights adequately protected, if “lawful but awful” speech is driven offline entirely? The right answers to these questions probably come from more layer-conscious internet regulation, treating edge platforms differently from infrastructure – as Annemarie Bridy discusses here. To me, concerns about the overall internet ecosystem at most support extending something like net neutrality rules higher up the internet’s technical “stack” to more infrastructure providers like CloudFlare.

If popular platforms like Facebook or YouTube were required to carry every legal post, not many people would like it. Most obviously, turning these popular forums into free speech mosh pits, and confronting users with material they consider obnoxious, immoral, or dangerous, would increase the real-world damage done by offensive or harmful speech.

Examples of legal online speech that have attracted widespread outrage include “History of why Jews ruin the world” and, horrifically, “How to burn Jews.” Examples of speech protected under recent First Amendment case law include signs held by picketers near a soldier’s funeral saying “thank God for IEDs” and “you’re going to hell.” This speech may be legally protected, but it’s not harmless. Very few people want to see it, and few believe that “responsible” platforms should leave it up. There is a reason public-interest groups and internet users typically urge platforms to take down more legal-but-offensive speech—not less.

Requiring platforms to carry speech that most users don’t want to see would also have serious economic consequences. Among other things, platforms would lose revenue from advertisers who do not want their brands associated with hateful or offensive content. Converting platforms from their current, curated state to free-for-alls for any speech not banned under law would be seen by some as tantamount to nationalization. Platforms would almost certainly challenge it as an unconstitutional taking of property.

Finally, the consequences of a pure common carriage regime for free expression aren’t clear-cut. There are speech rights on all sides of the issue. For one thing, platforms have their own First Amendment rights to set editorial policy and exclude content. (Eugene Volokh’s take on this is here, mine is in “Who Do You Sue.”)

For another, platforms sometimes silence one aggressive user—or many—in order to help another user speak. Without the platform’s thumb on the scales, some speakers, like female journalists barraged with not-quite-illegal threats of rape and violence, might be driven offline entirely.

Even many users who want platforms to reinstate their speech are unlikely to want the entire platform turned into an unmoderated free-for-all. After all, those speakers would lose their own online audience if platforms became so unattractive that listeners departed, or advertisers ceased funding platform operations.

If making platforms carry all legal expression seems too extreme, must-carry proponents might argue that platforms should have to carry some of it, or carry all legal expression but only amplify some of it, or otherwise face new laws for content currently regulated by private Terms of Service. This is where must-carry regulatory proposals begin to converge with must-remove proposals. As the next examples show, both depend on making new legal distinctions between kinds of speech, or setting detailed new rules for platforms’ speech management.

2. Indecency Rules

One possible approach would be to create new legally enforced prohibitions, or carriage obligations, for content that is currently governed only by platforms’ Terms of Service. The must-carry version of such a rule would allow platforms to take down highly offensive or indecent content, while requiring them to tolerate more civil or broadly socially acceptable speech. (This seemed to be white supremacist Jared Taylor’s theory when he sued Twitter, saying the platform could remove truly abusive posts, but not “civil” and “respectful” ones like his.) The must-remove version would, like some versions of the UK’s Online Harms proposal, require platforms to take down certain legal but “harmful” material.

Addressing “lawful but awful” speech through new legal indecency rules would be reminiscent of older regulation of broadcast content. But it would be a deeply troubling solution for online speech for several reasons.

First, we are far from a national or even local consensus about what legal content is highly offensive, dangerous, indecent, or otherwise objectionable in the internet context.

Second, rules of this sort would, like their TV and radio equivalents, require substantial and ongoing regulatory intervention and rulemaking to determine which theory of offensive and dangerous speech should prevail. Any attempt to apply such rules to platforms’ rapidly evolving and technically complex ranking algorithms would be particularly challenging.

Third, unlike broadcast regulation, rules limiting online speech would put major new restrictions on ordinary people’s daily communications. That last point is the kicker, as a constitutional matter. In fact, a must-remove rule for indecent speech online would look a lot like the law struck down by the Supreme Court in its seminal internet First Amendment case, Reno v. ACLU. A must-carry rule for decent speech would be little better. It would use state power to pick winners and losers among legal speech – burdening both the speech rights of affected users, and the editorial rights of affected platforms. In the U.S., that would be possible only after a massive re-litigation of current First Amendment law. It is hard to imagine the current Supreme Court blessing “indecency” rules of either sort.

3. Fairness Rules

To avoid new content-based speech regulation, lawmakers might instead let platforms enforce any TOS rules as long as they are “fair” or “neutral.” That has real problems, too. Substantive fairness rules, requiring fair treatment of all viewpoints, are very hard to imagine. Would Twitter, for example, have to give equal treatment to Democrats, Republicans, Socialists, Monarchists, and Anarchists? To people who like creamy peanut butter and people who like crunchy? To people urging us to invest in sketchy tech start-ups and people urging us not to?  

At the extreme, this would simply become a common carriage regime. Other potential rules here would be reminiscent of the ones the FCC applied to older communications channels. The equal-time doctrine, for example, required broadcasters to give equal airtime to qualified candidates for public office. And the fairness doctrine required “fair” coverage for issues ranging from workers’ rights to nuclear power plant construction. Critics – mostly on the political right – charged that the doctrine was unworkable and that it effectively enabled selective enforcement by an unaccountable bureaucracy. The FCC itself eventually decided the doctrine was unconstitutional, and President Reagan vetoed a bill that would have brought it back.

Could fairness rules be content-neutral, and instead simply require platforms’ rules and processes to be transparent and consistently enforced? That’s the basis of a lot of recent proposals in the U.S. It’s hard to imagine how most such rules could be content-neutral in practice, though. If the FTC or FCC got to review platforms’ decisions about individual pieces of content, assessing whether the company applied its own Terms of Service fairly or consistently, the scope for de facto government intervention in speech rules would be enormous.

The operational burden of reviewing even just Facebook’s or Twitter’s daily torrent of appeals would be crushing. In the fantasy world where a government agency assumed that job, we’d just move from politically polarized anger at how Facebook manages that impossible task to polarized anger at how the government manages it.

At the extreme, a fairness-based model could be entirely procedural – requiring platforms to publish clear content rules and let users appeal takedown decisions, without having any government review of the appeals’ outcome. (In the U.S., that’s the PACT Act’s model right now.) That’s unlikely to reassure critics who believe platforms are biased, excluding important voices, or simply lying about their rules, though. And even a purely disclosure-based standard like this one would face real First Amendment pushback, as an interference with editorial discretion or form of compelled speech.

4. Amplification Rules

Another model might target recommended, ranked, or amplified content, applying different rules to different aspects of platforms’ operations. As Tim Lee put it, we could “think of Facebook as being two separate products: a hosting product and a recommendation product (the Newsfeed).” Applying different rules to hosting on the one hand and amplification on the other would allow for somewhat more nuanced legal regimes.

A must-remove (or must-demote, or must-disfavor) version of this rule might prohibit amplification of extreme or offensive, but lawful, content. As a legal matter in the U.S., that would face the same constitutional barriers as any other government rule disfavoring lawful speech. As the Supreme Court put it in U.S. v. Playboy, “[t]he distinction between laws burdening and laws banning speech is but a matter of degree. The Government’s content‐based burdens must satisfy the same rigorous scrutiny as its content‐based bans.”

A must-carry version of this rule might require platforms to keep hosting all legal speech, but let them apply their own rules to ranking, recommendation, or amplification systems. Disfavored speakers would thus not be banished entirely and could in principle be found by other users. This idea has a Communications law flavor as well, as an update of the traditional distinction between infrastructure “network layer” intermediaries, such as ISPs; and user-facing edge or “application layer” services like Facebook or Google. Users of edge services typically want content curation and don’t want must-carry rules—or didn’t use to. The increase in must-carry claims could be taken as calls to rethink the role of major platforms and to start treating them more like essential, network-layer internet infrastructure.

A must-carry rule that effectively applied a common carriage requirement for hosting but left platforms free to set their own amplification rules would have less extreme constitutional problems than some of the other models. Unlike pure common carriage, it would not take away platforms’ editorial discretion entirely. And unlike “indecency” models, it would avoid creating new state-sponsored speech rules.

But it would still require extensive and ongoing regulation, with resulting distortion of market incentives and innovation, to decide what counts as the “infrastructure” and “edge” aspects of any given platform. Would your racist uncle’s tweets simply never appear in your feed but be visible on his profile page, for example? Be hidden from the casual visitor to his profile page but findable through search? If Google dislikes the only web page that has the text string “fifty purple pigeons eat potatoes,” could it rank a hundred other pages above it when users search for those exact words? There are a thousand variants to these questions, all of which will matter to someone. I don’t see clear or easy answers.

5. Rules for Dominant Platforms

As a final variant, any of these approaches might be applied only to the largest platforms. That wouldn’t avoid the problems identified here entirely. But it could change some of the calculations – both about platforms’ ability to comply with burdensome rules, and about the public interest in making them do so.

Special rules for mega-platforms could in theory also conceptually align with some Communications law precedent about competition and the First Amendment, by imposing obligations only on those who control access to “scarce” Communications channels or audiences.  

Lawmakers outside the United States have experimented somewhat with setting different rules for hosting platforms depending on their size. Germany’s NetzDG, for example, holds social networks with more than two million German users to stringent content-removal timelines, as well as higher standards of public transparency. The EU’s 2019 Copyright Directive also includes special obligations for entities hosting “large amounts” of user-generated content, and a (very limited) carve-out for start-ups.

Setting different restrictions depending on size would create problematic incentives for growing start-ups and is generally not a common approach in American law, though it has cropped up in some recent proposals. It is also hard to identify a workable definition of “bigness” that would not inadvertently sweep in complex entities like the thinly staffed, user-managed Wikipedia. (The EU Copyright Directive solves this particular problem, but not the larger need for flexible rules to protect innovation, with a targeted carve-out for “not-for-profit online encyclopedias[.]”)

Changing the Rulemakers

A set of potentially more interesting regulatory models would abandon the effort to dictate platforms’ speech rules, and instead give users more choices among competing rulesets or rulemakers. These approaches are generally grounded more in competition and user autonomy, and less in the idea that we can arrive at better, mutually agreeable rules. Proposals to change the rulemakers will likely appeal more if your primary concern about platforms is their economic dominance, their gatekeeper role in shaping public conversation, or their capacity to “push” unwanted content to users who do not wish to see it. These ideas will probably appeal less if what you want is for platforms to get rid of legal but offensive or dangerous content, or if you are concerned about filter bubbles.

1. Empowering Users

For many proponents of online civil liberties (including myself), one go-to solution for problems of platform content moderation is to give users themselves more control over what they see. Settings on YouTube or Twitter, for example, could include dials and knobs to signal our individual tolerance for violence, nudity, or hateful speech. This isn’t a cure-all, but it’s still a great idea. It’s been around at least since the 1990s, when technologies like the Platform for Internet Content Selection (PICS) were supposed to allow users to choose what web content appeared in their browsers. Both the Supreme Court in Reno and Congress in passing CDA 230 relied in part on the expectation that such technologies would empower consumers.

Today, there remains much to be done to give users more control over their information diet. There is perhaps a chicken-and-egg question about the paucity of end-user content controls today and the rise of major, centralized platforms. Did internet users stop demanding these tools once they found search engines and curated hosting platforms to protect them from the worst of the web? Or did they flock to centralized platforms because good tools for true end-user control did not exist?

It may be that such tools have only limited promise as a technical matter, because they depend on accurate content labeling. A user who wanted to block most racial epithets but retain access to rap lyrics, historical documents, and news reporting, for example, could do so only if people or algorithms first correctly identified content in these categories. That’s more work than humans could do at internet scale, and algorithmic filters have so far proven highly unreliable at tasks of this sort.

Could or should lawmakers actually mandate something like this? I haven’t checked, but I bet that Larry Lessig, Jack Balkin, James Boyle, Julie Cohen, or others of their cohort wrote something smart about this decades ago. My quick take is that truly granular user content controls would be an incredible amount of work to implement, still might not work very well, and could well wind up being ignored as most users simply accept a platform’s default rules. And any law dictating content classifications for user-controlled settings would tread uncomfortably close to creating state-created content mandates, raising inevitable First Amendment questions.

Still, creating end user controls strikes me as a much better approach than replacing Mark Zuckerberg’s speech rules with Mitch McConnell’s or Nancy Pelosi’s. And at least some of the practical limitations – like users not bothering to change complicated settings – might be addressed by introducing competing rulemakers into the system. That’s part of the thinking behind the last idea I’ll discuss: what I call the “Magic APIs” approach.

2. Magic APIs

A final model, and the most ambitious one, is what I think of as the Magic APIs approach. APIs, or application program interfaces, are technical tools that allow one internet service to connect with and retrieve information from another. In this model, platforms would open APIs so that competing providers, or cooperating ones, could offer users alternate rule sets for managing the content held by major platforms. This model has a lot of overlap with the ideas Mike Masnick has spelled out, in much greater detail, in his Protocols Not Platforms piece. Jonathan Zittrain has also written about ideas like this in the past, suggesting that “Facebook should allow anyone to write an algorithm to populate someone’s feed.” I’ve heard other people kick these ideas around for at least a decade. So the idea isn’t entirely new, but it’s also never had a really thorough public airing.

The Magic APIs approach would be broadly analogous to telecommunications “unbundling” requirements. These aim to insert competition into markets subject to network effects by requiring incumbents to license hard-to-duplicate resources to newcomers. In the platform context, this would mean that Google or Facebook opens up access to the uncurated version of its service, including all legal user-generated content, as the foundation for competing user-facing curation services. Competitors would then offer users some or all of the same content, with their own new content ranking and removal policies, and potentially a whole new user interface. Users might choose a G-rated version of Twitter from Disney or a racial justice-oriented lens on YouTube from a Black Lives Matter-affiliated group, for example. As Mike Masnick put it in 2018:

Ideally, Facebook (and others) should open up so that third party tools can provide their own experiences—and then each person could choose the service or filtering setup that they want. People who want to suck in the firehose, including all the garbage, could do so. Others could choose other filters or other experiences. Move the power down to the ends of the network, which is what the internet was supposed to be good at in the first place.

This, too, has some 1990s precedent. Back then, the Internet Content Rating Association (ICRA) proposed something like it for web content. In addition to user-controlled content settings at the browser level, ICRA planned to let users subscribe to add- or block-lists from trusted third parties. That would have spared users the work of setting detailed content preferences, and webmasters the work of creating detailed labels. But “federated” models along these lines, including Mastodon and a content sharing protocol plan announced by Twitter, have prompted renewed interest of late. Cory Doctorow’s recent work on “adversarial interoperability” is relevant here, too. It tackles the problem from the other end, proposing reforms of laws like the Computer Fraud and Abuse Act (CFAA) and anti-circumvention provisions of the DMCA so that new market entrants can proceed directly to offering users services of this sort – with or without platforms’ permission.

Letting users choose among competing “flavors” of today’s mega-platforms would solve some First Amendment problems by leaving platforms’ own editorial decisions undisturbed, while permitting competing editors to offer alternate versions and include speakers who would otherwise be excluded. In old school Competition terms, this would serve a media pluralism goal. But many platforms would object to compulsory Magic APIs on innumerable grounds, including the Constitution’s prohibition on most state takings of property. And some governments might not actually like decentralizing platform power in this way. After all, a more diversified Internet would lack the chokepoints for regulation and informal pressure that incumbents like Google and Facebook provide today. In any case, this approach would also create a slew of new problems—beyond the ordinary downsides of regulatory intervention.

The technology required to make Magic APIs work would be difficult, perhaps impossible, to build well; that’s the “magic” part. (But previously magical things happen all the time. So I’m always curious what platform engineers would say about this if their lawyers weren’t watching.) There are also serious questions about how such a system would interact with the complex, multiplayer technical infrastructure behind online advertising. I haven’t seen any published work addressing this issue, but I suspect it’s a pretty big one.

Having multiple content-rating efforts for the same material would also be massively inefficient. Few competitors will emerge if they all must make redundant investments to translate the same Kurdish-language post, for example, or to identify the local significance of a particular flag, song, or slang term. As I recently told Slate, in an ideal world that more “objective” part of the work could be centralized or conclusions could be shared. But the subsequent exercise of judgment about whether to exclude the content would be spread out, subject to different rules on different services.  

Perhaps most dauntingly, the Magic APIs model has real privacy issues. After all, the entire Cambridge Analytica scandal was created by Facebook APIs passing information about users to a third party service provider. As a technical matter, Magic APIs would be pretty much the same thing. What would need to be (very) different for Magic APIs to succeed would be user consent and privacy protection. The hardest part would be defining the right rules for data about a user’s friends. Users aren’t likely to migrate to competing services or “flavors” of existing services if that means losing touch with, and not seeing content from, the people they know. Unless we can identify the legal and technical framework to reconcile this kind of data sharing with laws like the GDPR, Magic APIs and similar ideas will provide no way forward.

The good news is that lots of smart people are already wrangling with this problem. Discussions about data portability and interoperability have brought together experts in privacy and competition law sort out issues at the intersection of their fields. The Magic APIs model just adds another layer of complexity by incorporating speech concerns.  

Conclusion

It is far from clear to me that any of these regulatory approaches have upsides that outweigh their downsides. But the really interesting ones have not yet been given a good tire-kicking by technical and legal experts, either. I find that encouraging. We have a lot to talk about, especially at the intersections of competition and speech law. (Some thinkers worth namechecking at that intersection are Harold Feld, Barbara van Schewick, Tim Wu, Blake Reid, and Berin Szoka. Importantly, they don’t all agree with each other.)

For the most ambitious proposals, like the Magic APIs model, we need to add privacy law to the mix. And realistically, we can’t stop there. We’ll make bad laws if we don’t factor in issues like equal protection and anti-discrimination. We’ll fail to solve real-world problems if we don’t address laws like the CFAA. And if lawyers have these conversations without technologists in the room, we’re in for a bad time. Getting experts out of our silos to have those conversations won’t be easy, but it may lead to real ways forward. At the risk of going very off-brand, that prospect makes me cautiously optimistic.

Daphne Keller directs the Program on Platform Regulation at Stanford's Cyber Policy Center, and was formerly the Director of Intermediary Liability at CIS. Her work focuses on platform regulation and Internet users' rights. She has published both academically and in popular press; testified and participated in legislative processes; and taught and lectured extensively. Her recent work focuses on legal protections for users’ free expression rights when state and private power intersect, particularly through platforms’ enforcement of Terms of Service or use of algorithmic ranking and recommendations. Until 2015 Daphne was Associate General Counsel for Google, where she had primary responsibility for the company’s search products. She worked on groundbreaking Intermediary Liability litigation and legislation around the world and counseled both overall product development and individual content takedown decisions.

Filed Under: content moderation, intermediary liability, regulations, section 230, speech, speech rules

Intermediary Liability And Responsibilities Post-Brexit

from the the-entire-game-has-changed dept

This is a peculiar time to be an English lawyer. The UK has one foot outside the EU, and (on present intentions) the other foot will join it when the current transitional period expires at the end of 2020.

It is unclear how closely tied the UK will be to future EU law developments following any trade deal negotiated with the EU. As things stand, the UK will not have to implement future EU legislation, and is likely to have considerable freedom in many areas to depart from existing EU legislation.

The UK government has said that it has no plans to implement the EU Copyright Directive adopted in April 2019. Nor does it seem likely that it would have to follow whatever legislation may result from the European Commission's proposals for an EU Digital Services Act. Conversely, the government has also said that it has no current plans to change the existing intermediary liability provisions of the EU Electronic Commerce Directive, or the Directive's approach to prohibition of general monitoring obligations.

Looking across the Atlantic, there is the prospect of a future trade agreement between the UK and the USA. That has set off alarm bells in some quarters that the US government will want the UK to adopt an intermediary liability shield modeled on S.230 Communications Decency Act.

Domestically, the UK government is developing its Online Harms plans. The proposed legislation would impose a legal duty on user generated content-sharing intermediaries and search engines to prevent or inhibit many varieties of illegal or harmful UGC. Although branded a duty of care, the proposal is more akin to a broadcast-style content regulatory regime than to a duty of care as a tort lawyer would understand it. The regime would most likely be managed and enforced by the current broadcast regulator, Ofcom. As matters stand the legislation would not define harm, leaving Ofcom to decide (subject to some specific carve-outs) what should be regarded as harmful.

All this is taking place against the background of the techlash. This is not the place to get into the merits and demerits of that debate. The aim of this piece is to take an educational ramble around the UK and EU legal landscape, pausing en route to inspect and illuminate some significant features.

Liability Versus Responsibilities

The tour begins by drawing a distinction between liability and responsibilities.;

In the mid-1990s the focus was mostly on liability: the extent to which an intermediary can be held liable for unlawful activities and content of its users. The US and EU landmarks were S.230 CDA 1996 and S.512 DMCA 1998 (USA), and Articles 12 to 14 of the Electronic Commerce Directive 2000 (EU).

Liability presupposes the user doing something unlawful on the intermediary's platform. (Otherwise, there is nothing for the intermediary to be liable for.) The question is then whether the platform, as well as the user, should be made liable for the user's unlawful activity – and if so, in what circumstances. The risk (or otherwise) of potential liability may encourage the intermediary to act in certain ways. Liability regimes incentivise, but do not mandate.

Over time, the policy focus has expanded to take in responsibilities: putting an intermediary under a positive obligation to take action in relation to user content or activity.

A mandatory obligation to prevent users behaving in particular ways is different from being made liable for their unlawful activity. Liability arises from a degree of involvement in the primary unlawful activity of the user. Imposed responsibility does not necessarily rest on a user's unlawful behavior. The intermediary is placed under an independent, self-standing obligation – one that it alone can breach.

Responsibilities Imposed By Court Orders

Responsibilities first manifested themselves as mandatory obligations imposed on intermediaries by specific court orders, but still predicated on the existence of unlawful third party activities.

In the US this development withered on the vine with SOPA/PIPA in 2012. Not so in the EU, where copyright site blocking injunctions can be (and have often been) granted against internet service providers under Article 8(3) of the InfoSoc Directive. The Intellectual Property Enforcement Directive requires similar injunctions to be available for other IP rights. In the UK it is established that a site blocking injunction can be granted based on registered trade marks, and potentially in respect of other kinds of unlawful activity.

Limits to the actions that court orders can oblige intermediaries to take in respect of third party activities have been explored in numerous cases: amongst them, at EU Court of Justice level, detection and filtering of copyright infringing files in SABAM v Scarlet and SABAM v Netlog; detection and filtering of equivalent defamatory content in Glawischnig-Piesczek v Facebook; and worldwide delisting in Glawischnig-Piesczek v Facebook. 

Such court orders tend not to be conceptualized in terms of remedying a breach by the intermediary. Rather, they are based on efficiency: the intermediary, as a choke point, should be co-opted as being in the best position to reduce unlawful activity by third parties. In UK law at least, the intermediary has no prior legal duty to assist – only to comply with an injunction if the court sees fit to grant one.

Responsibilities Imposed by Duties Of Care

Most recently the focus on intermediary responsibilities has broadened beyond specific court orders. It now includes the idea of a prior positive obligation, imposed on an intermediary by the general law, to take steps to reduce risks arising from user activities on the platform.

This kind of obligation, frequently labelled a duty of care, is contemplated by the UK Online Harms proposals and may form part of a future EU Digital Services Act.

In the form in which it has been adapted for the online sphere, a duty of care would impose positive obligations on the intermediary to prevent users from harming other users (and perhaps non-users). Putting aside the vexed question of what constitutes harm in the context of online speech, a legal responsibility to prevent activities of third parties is far from the norm. A typical duty of care is owed in respect of someone's own acts, not to prevent acts of third parties.

Although conceptually distinct from liability, an intermediary duty of care can interact and overlap with it. For example, a damages claim framed as breach of a duty of care may in some circumstances be barred by the ECD liability shields. In McFadden the rightsowner sought to hold a Wi-Fi operator liable for damages in respect of copyright infringement by users, founded on an allegation that the operator had breached a duty to secure its network. The CJEU found that the claim for damages was precluded by the Article 12 conduit shield, even though the claim was framed as breach of a duty rather than as liability for the users' copyright infringement as such.

At the other end of the spectrum, the English courts have held that if a regulatory sanction is sufficiently remote from specific user infringements as not to be in respect of those infringements, the sanction is not precluded by the ECD liability shields. The UK Online Harms proposals suggest that sanctions would be for breach of systemic duties, rather than penalties tied to failure to remove specific items of content.

Beyond Unlawfulness

Although intermediary liability is restricted to unlawfulness on the part of the user, responsibility is not. A self-standing duty of care is concerned with risk of harm. Harm may include unlawfulness, but is not limited to that.

The scope of such a duty of care depends critically on what is meant by harm. In English law, comparable offline duties of care are limited to objectively ascertainable physical injury and damage to physical property. The UK Online Harms proposals jettison that limitation in favor of undefined harm. Applied to lawful online speech, that is a subjective concept. As matters stand Ofcom, as the likely regulator, would in effect decide what does and does not constitute harm.

Article 15 ECommerce Directive

A preventative duty of care takes us into the territory of proactive monitoring and filtering. Article 15 ECD, which sits alongside the liability scheme enacted in Articles 12 to 14, prohibits Member States from imposing two kinds of obligation on conduits, caches or hosts: a general obligation to monitor information transmitted or stored, and a general obligation actively to seek facts or circumstances indicating illegal activity.

Article 15 does not on its face prohibit an obligation to seek out lawful but harmful activity, unless it constitutes a general obligation to monitor information. But in any event, for an EU Member State the EU Charter of Fundamental Rights would be engaged. The CJEU found the filtering obligations in Scarlet and Netlog to be not only in breach of Article 15, but also contrary to the EU Charter of Fundamental Rights. For a non-EU state such as the UK, the European Convention on Human Rights would be relevant.

So far, the scope of Article 15 has been tested in the context of court orders. The principles established are nevertheless applicable to duties of care imposed by the general law, with the caveat that Recital (48) permits hosts to be made subject to "duties of care, which can reasonably be expected from them and which are specified by national law, in order to detect and prevent certain types of illegal activities." What those "certain types" might be is not stated. In any event the recital does not on the face of it apply to lawful activities deemed to be harmful.

The Future Post-Brexit

Both the UK and the EU are currently heading down the road of imposing responsibilities on intermediaries, while professing to leave the liability provisions of the ECD untouched. That is conceptually possible for some kinds of responsibilities, but difficult to navigate in practice. Add the prohibition on general monitoring obligations and the task becomes harder, especially if the prohibition stems not just from the ECD (which could be diluted in future legislation) but from the EU Charter of Fundamental Rights and the ECHR.

The French Loi Avia, very much concerned with imposing responsibilities, was recently partially struck down by the French Constitutional Council. Whilst no doubt it will return in a modified form, it is nevertheless a salutary reminder of the relevance of fundamental rights.

As for UK-US trade discussions, Article 19.17 of the US-Mexico-Canada Agreement has set a precedent for inclusion of intermediary liability. Whether the wording of Article 19.17 really does mandate full S.230 immunity, as some have suggested, is another matter. Damian Collins MP, asking a Parliamentary Question on 2 March 2020, said:

"the US-Mexico-Canada trade agreement required the insertion of the section 230 provisions of the United States' Communications Decency Act, which give immunity from liability to the big social media companies."

The Trade Secretary replied:

"I can confirm that we stand by our online harms commitment, and nothing in the US trade deal will affect that."

Although the USMCA agreement uses language that tracks S.230, it does not fully replicate it. Notably, it does not use the magic word 'publisher' that appears in S.230 and which Zeran v America Online interpreted in 1997 as embracing both strict primary publisher liability and knowledge-based secondary publisher (a.k.a. distributor) liability.

Instead, Article 19.17 precludes liability as an "information content provider," defined as "a person or entity that creates or develops, in whole or in part, information provided through the Internet or another interactive computer service" That aptly describes a primary publisher. But if that language does not cover secondary publishers, then if it were to appear in a UK-US trade agreement it would seem not to preclude a hosting liability regime akin to the existing ECD Article 14.

Graham Smith is Of Counsel at Bird & Bird LLP, London, England. He is the editor and main author of the English law textbook Internet Law and Regulation (5th ed 2020, Sweet & Maxwell). The views expressed in this article are the personal views of the author.

Filed Under: brexit, content moderation, digital services act, duty of care, eu, eu copyright directive, liability, online harms, responsibilities

Content Moderation Best Practices for Startups

from the framework-for-the-internet-of-tomorrow dept

To say content moderation has become a hot topic over the past few years would be an understatement. The conversation has quickly shifted from how to best deal with pesky trolls and spammers  —  straight into the world of intensely serious topics like genocide and destabilization of democracies.

While this discussion often centers around global platforms like Facebook and Twitter, even the smallest of communities can struggle with content moderation. Just a limited number of toxic members can have an outsize effect on a community’s behavioral norms.

That’s why the issue of content moderation needs to be treated as a priority for all digital communities, large and small. As evidenced by its leap from lower-order concern to front-page news, content moderation is deserving of more attention and care than most are giving it today. As I see it, it’s a first-class engineering problem that calls for a first-class solution. In practical terms, that means providing:

1. accessible, flexible policies and procedures that account for the shades of gray moderators see day to day; and

2. technology that makes those policies and procedures feasible, affordable, and effective.

Fortunately, this doesn’t have to be a daunting task. I’ve spent years having conversations with platforms that are homes to tens to hundreds of millions of monthly active users, along with advisors spanning the commercial, academic, and non-profit sectors. From these conversations, I’ve created this collection of content moderation and community building best practices for platforms of all sizes.

Content policies

1. Use understandable policies.

This applies to both the policies you publish externally and the more detailed, execution-focused version of these policies that help your moderators make informed and consistent decisions. While the decisions and trade-offs underlying these policies are likely complex, once resolution is reached the policies themselves need to be expressed in simple terms so that users can easily understand community guidelines and moderators can more easily recognize violations.

When the rules aren’t clear, two problems arise: (i) moderators may have to rely on gut instincts rather than process, which can lead to inconsistency; and (ii) users lose trust because policies appear arbitrary. Consider providing examples of acceptable and unacceptable behaviors to help both users and moderators see the application of your policies in action (many examples will be more clarifying than just a few). Again, this is not to say that creating policies is an easy process, there will be many edge cases that make this process challenging. We touch more on this below.

2. Publicize policies and changes.

Don’t pull the rug out from under your users. Post policies in an easy-to-find place, and notify users when they change. How to accomplish the latter will depend on your audience, but you should make a good faith effort to reach them. For some, this may mean emailing; for others, a post pinned to the top of a message board will suffice.

3. Build policies on top of data.

When your policies are called into question, you want to be able to present a thoughtful approach to their creation and maintenance. Policies based on intuition or haphazard responses to problems will likely cause more issues in the long run. Grounding your content policies on solid facts will make your community a healthier, more equitable place for users.

4. Iterate.

Times change, and what works when you start your community won’t necessarily work as it grows. For instance, new vocabulary may come into play, and slurs can be reappropriated by marginalized groups as counterspeech. This can be a great opportunity to solicit feedback from your community to both inform changes and more deeply engage users. Keep in mind that change need not be disruptive  —  communities can absorb lots of small, incremental changes or clarifications to policies.

Harassment and abuse detection

1. Be proactive.

Addressing abusive content after it’s been posted generally only serves to highlight flaws and omissions in your policies, and puts the onus of moderation on users. Proactive moderation can make use of automated initial detection and human moderators working in concert. Automated systems can flag potentially abusive content, after which human moderators with a more nuanced understanding of your community can jump in to make a final call.

2. Factor in context.

Words or phrases that are harmful in one setting, may not be in another. Simple mechanisms like word filters and pattern matching are inadequate for this task, as they tend to under-censor harmful content and over-censor non-abusive content. Having policies and systems that can negotiate these kinds of nuances is critical to maintaining a platform’s health.

3. Create a scalable foundation.

Relying on human moderation and sparse policies may work when your goal is to get up and running, but can create problems down the road. As communities grow, the complexity of expression and behavior grows. Establishing policies that can handle increased scale and complexity over time can save time and money  —  and prevent harassment  —  in the long term.

4. Brace for abuse.

There’s always the danger of persistent bad actors poisoning the well for an entire community. They may repeatedly test keyword dictionaries to find gaps, or manipulate naive machine learning-based systems to “pollute the well.” Investing in industrial-grade detection tooling early on is the most effective way to head off these kinds of attacks.

5. Assess effectiveness.

No system is infallible, so you’ll need to build regular evaluations of your moderation system into your processes. Doing so will help you understand whether a given type of content is being identified correctly or incorrectly  —  or missed entirely. That last part is perhaps the biggest problem you’ll face. I recommend using production data to build evaluation sets, allowing you to track performance over time.

Moderation actions

1. Act swiftly.

Time is of the essence.  The longer an offensive post remains, the more harm can come to your users and your community’s reputation. Inaction or delayed response can create the perception that your platform tolerates hateful or harassing content, which can lead to a deterioration of user trust.

2. Give the benefit of the doubt.

From time to time, even “good” community members may unintentionally post hurtful content. That’s why it’s important to provide ample notice of disciplinary actions like suspensions. Doing so will allow well-intentioned users to course-correct, and, in the case of malicious users, provide a solid basis for more aggressive measures in the future.

3. Embrace transparency.

One of the biggest risks in taking action against a community member is the chance you’ll come across as capricious or unjustified. Regularly reporting anonymized, aggregated moderation actions will foster a feeling of safety among your user base.

4. Prepare for edge cases.

Just as you can’t always anticipate new terminology, there will likely be incidents your policies don’t clearly cover. One recommendation for handling these types of hiccups is a process that triggers the use of an arbiter that holds final authority.

Another method is to imagine the content or behavior to be 10,000 times as common as it is today. The action you would take in that scenario can inform the action you take today. Regardless of the system you develop, be sure to document all conversations, debates, and decisions. And once you’ve reached a decision, formalize it by updating your content policy.

5. Respond appropriately.

Typically, only a small portion of toxic content comes from persistent, determined bad actors. The majority of incidents are due to regular users having an off-day. That’s why it’s important to not apply draconian measures like permanent bans at the drop of a hat. Lighter measures like email or in-app warnings, content removal, and temporary bans send a clear signal about unacceptable behavior while allowing users to learn from their mistakes.

6. Target remedies.

Depending on the depth of your community, a violation may be limited to a subgroup within a larger group. Be sure to focus on the problematic subgroup to avoid disrupting the higher-level group.

7. Create an appeals process.

In order to establish and build trust, it’s important to create an equitable structure that allows users to appeal when they believe they’ve been wrongly moderated. As with other parts of your policies, transparency plays a big role. The more effort you put into explaining and publicizing your appeals policy up front, the safer and stronger your community will be in the long run.

8. Protect moderators.

While online moderation is a relatively new field, the stresses it causes are very real. Focusing on the worst parts of a platform can be taxing psychologically and emotionally. Support for your moderators in the form of removing daily quotas, enforcing break times, and providing counseling is good for your community — and the ethical thing to do.

And if you’re considering opening a direct channel for users to communicate with Trust & Safety agents, be aware of the risks. While it can help dissipate heightened user reactions, protecting moderators here is also critical. Use shared, monitored inboxes for inbound messages and anonymized handles for employee accounts. Use data to understand which moderators are exposed to certain categories or critical levels of abusive content. Lastly, provide employees with personal online privacy-protecting solutions such as DeleteMe.

Measurement

1. Maintain logs.

Paper trails serve as invaluable reference material. Be sure to keep complete records of flagged content including the content under consideration, associated user or forum data, justification for the flag, moderation decisions, and post mortem notes, when available. This information can help inform future moderation debates and identify inconsistencies in the application of your policies.

2. Use metrics.

Moderation is possibly the single most impactful determinant of a community user’s experience. Measurement of its effectiveness should be subject to the same rigor you’d apply to any other part of your product. By evaluating your moderation process with both quantitative and qualitative data, you’ll gain insight into user engagement, community health, and the impact of toxic behavior.

3. Use feedback loops.

A final decision on a content incident need not be the end of the line. Don’t let the data you’ve collected through the process go to waste. Make it a part of regular re-evaluations and updates of content policies to not only save effort on similar incidents, but also to reinforce consistency.

Most importantly, though, your number one content moderation concern should be strategic in nature. As important as all of these recommendations are for maintaining a healthy community, they’re nothing without an overarching vision. Before you define your policies, think through what your community is, who it serves, and how you’d like it to grow. A strong sense of purpose will help guide you through the decisions that don’t have obvious answers — and, of course, help attract the audience you want.

This collection of best practices is by no means the be-all and end-all of content moderation, but rather a starting point. This industry is constantly evolving and we’ll all need to work together to keep best practices at the frontier. If you have any comments or suggestions, feel free to share on this Gitlab repo.

Let’s help make the internet a safer, more respectful place for everyone.

Taylor Rhyne is co-founder and Chief Operating Officer of Sentropy, an internet security company building machine learning products to detect and fight online abuse. Rhyne was previously an Engineering Project Manager at Apple on the Siri team where he helped develop and deploy advanced Natural Language Understanding initiatives.

Filed Under: best practices, content moderation, transparency

Fighting Hate Speech Online Means Keeping Section 230, Not Burying It

from the first-do-no-harm dept

At Free Press, we work in coalition and on campaigns to reduce the proliferation of hate speech, harassment, and disinformation on the internet. It’s certainly not an easy or uncomplicated job. Yet this work is vital if we’re going to protect the democracy we have and also make it real for everyone — remedying the inequity and exclusion caused by systemic racism and other centuries-old harms seamlessly transplanted online today.

Politicians across the political spectrum desperate to “do something” about the unchecked political and economic power of online platforms like Google and Facebook have taken aim at Section 230, passed in 1996 as part of the Communications Decency Act. Changing or even eliminating this landmark provision appeals to many Republicans and Democrats in DC right now, even if they hope for diametrically opposed outcomes.

People on the left typically want internet platforms to bear more responsibility for dangerous third-party content and to take down more of it, while people on the right typically want platforms to take down less. Or at least less of what’s sometimes described as “conservative” viewpoints, which too often in the Trump era has been unvarnished white supremacy and unhinged conspiracy theories.

Free Press certainly aligns with those who demand that platforms do more to combat hate and disinformation. Yet we know that keeping Section 230, rather than radically altering it, is the way to encourage that. That may sound counter-intuitive, but only because of the confused conversation about this law in recent years.

Preserving Section 230 is key to preserving free expression on the internet, and to making it free for all, not just for the privileged. Section 230 lowers barriers for people to post their ideas online, but it also lowers barriers to the content moderation choices that platforms have the right to make.

Changes to Section 230, if any, have to retain this balance and preserve the principle that interactive computer services are legally liable for their own bad acts but not for everything their users do in real time and at scale.

Powerful Platforms Are Still Powering Hate, and Only Slowly Changing Their Ways

Online content platforms like Facebook, Twitter and YouTube are omnipresent. Their global power has resulted in privacy violations, facilitated civil rights abuses, provided white supremacists and other violent groups a place to organize, enabled foreign-election interference and the viral spread of disinformation, hate and harassment.

In the last few months some of these platforms have begun to address their role in the proliferation and amplification of racism and bigotry. Twitter recently updated its policies by banning links on Twitter to hateful content that resides offsite. That resulted in the de-platforming of David Duke, who had systematically skirted Twitter’s rules by linking to hateful content across the internet while following some limits for what he said on Twitter itself.

Reddit also updated its policies on hate and removed several subreddits. Facebook restricted “boogaloo” and QAnon groups. YouTube banned several white supremacists accounts. Yet despite these changes and our years of campaigning for these kinds of shifts, hate still thrives on these platforms and others.

Some in Congress and on the campaign trail have proposed legislation to rein in these companies by changing Section 230, which shields platforms and other websites from legal liability for the material their users post online. That’s coming from those who want to see powerful social networks held more accountable for third-party content on their services, but also from those who want social networks to moderate less and be more “neutral.”

Taking away Section 230 protections would alter the business models of not just big platforms but every site with user-generated material. And modifying or even getting rid of these protections would not solve the problems often cited by members of Congress who are rightly focused on racial justice and human rights. In fact, improper changes to the law would make these problems worse.

That doesn’t make Section 230 sacrosanct, but the dance between the First Amendment, a platform’s typical immunity for publishing third-party speech, and that same platform’s full responsibility for its own actions, is a complex one. Any changes proposed to Section 230 should be made deliberately and delicately, recognizing that amendments can have consequences not only unintended by their proponents but harmful to their cause.

Revisionist History on Section 230 Can’t Change the Law’s Origins or Its Vitality

To follow this dance it’s important to know exactly what Section 230 is and what it does.

Written in the early web era in 1996, the first operative provision in Section 230 reads: “No provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider.”

When a book or a newspaper goes to print, its publisher is legally responsible for all the words printed. If those words are plagiarized, libelous, or unlawful then that publisher may face legal repercussions. In the terms of Section 230, they are the law’s “information content provider[s]”.

Wiping away Section 230 could revert the legal landscape to the pre-1996 status quo. That’s not a good thing. At the time, a pair of legal decisions had put into a bind any “interactive computer service” that merely hosts or transmits content for others. One case held a web platform that did moderate content could be sued for libel (just as the original speaker or poster could be) if that alleged libel slipped by the platform’s moderators. The other case held sites that did not moderate were not exposed to such liability.

Before Section 230 became law, this pair of decisions meant websites were incentivized to go in one of two directions: either don’t moderate at all, tolerating not just off-topic comments but all kinds of hate speech, defamation, and harassment on their sites; or vet every single post, leading inexorably to massive takedowns and removal of anything that might plausibly subject them to liability for statements made by their users.

The authors of Section 230 wanted to encourage the owners of websites and other interactive computer services, to curate content on their websites as these sites themselves saw fit. But back then that meant those websites could be just as responsible as newspapers for anything anyone said on their platforms if they moderated at all.

In that state of affairs, someone like Mark Zuckerberg or Jack Dorsey would have the legal responsibility to approve every single post made on their services. Alternatively, they would have needed to take a complete, hands-off approach. The overwhelming likelihood is that under a publisher-liability standard those sites would not exist at all, at least not in anything like their present form.

There’s an awful lot we’re throwing out with the bathwater if we attack not just the abuses of ad-supported and privacy-invasive social-media giants but all sites that allow users to share content on platforms they don’t own. Smaller sites likely couldn’t make a go of it at all, even if a behemoth like Facebook or YouTube could attempt the monumental task of bracing for potential lawsuits over the thousands of posts made every second of the day by their billions of users. Only the most vetted, sanitized, and anodyne discussions could take place in whatever became of social media. Or, at the other extreme, social media would descend into an unfiltered and toxic cesspool of spam, fraudulent solicitations, porn, and hate.

Section 230’s authors struck a balance for interactive computer services that carry other people’s speech: platforms should have very little liability for third-party content, except when it violates federal criminal law and intellectual property law.

As a result, websites of all sizes exist across the internet. A truly countless number of these — like Techdirt itself — have comments or content created by someone other than the owner of the website. The law preserved the ability of those websites, regardless of their size, to tend to their own gardens and set standards for the kinds of discourse they allow on their property without having to vet and vouch for every single comment.

That was the promise of Section 230, and it’s one worth keeping today: an online environment where different platforms would try to attract different audiences with varying content moderation schemes that favored different kinds of discussions.

But we must acknowledge where the bargain has failed too. Section 230 is necessary but not sufficient to make competing sites and viewpoints viable online. We also need open internet protections, privacy laws, antitrust enforcement, new models for funding quality journalism in the online ecosystem, and lots more.

Taking Section 230 off the books isn’t a panacea or a pathway to all of those laudable ends. Just the opposite, in fact.

We Can’t Use Torts or Criminal Law to Curb Conduct That Isn’t Tortious or Criminal

Hate and unlawful activity still flourish online. A platform like Facebook hasn’t done enough yet, in response to activist pressure or advertiser boycotts, to further modify its policies or consistently enforce existing terms of service that ban such hateful content.

There are real harms that lawmakers and advocates see when it comes to these issues. It’s not just an academic question around liability for carrying third-party content. It’s a life and death issue when the information in question incites violence, facilitates oppression, excludes people from opportunities, threatens the integrity of our democracy and elections, or threatens our health in a country dealing so poorly with a pandemic.

Should online platforms be able to plead Section 230 if they host fraudulent advertising or revenge porn? Should they avoid responsibility for facilitating either online or real-world harassment campaigns? Or use 230 to shield themselves from responsibility for their own conduct, products, or speech?

Those are all fair questions, and at Free Press we’re listening to thoughtful proposed remedies. For instance, Professor Spencer Overton has argued forcefully that Section 230 does not exempt social-media platforms from civil rights laws, for targeted ads that violate voting rights and perpetuate discrimination.

Sens. John Thune and Brian Schatz have steered away from a takedown regime like the automated one that applies to copyright disputes online, and towards a more deliberative process that could make platforms remove content once they get a court order directing them to do so. This would make platforms more like distributors than publishers, like a bookstore that’s not liable for what it sells until it gets formal notice to remove offending content.

However, not all amendments proposed or passed in recent times have been so thoughtful, in our view, Changes to 230 must take the possibility of unintended consequences and overreach into account, no matter how surgical proponents of the change may think an amendment would be. Recent legislation shows the need for clearly articulated guardrails. In an understandable attempt to cut down on sex trafficking, a law commonly known as FOSTA (the “Fight Online Sex Trafficking Act”) changed Section 230 to make websites liable under state criminal law for the knowing “promotion or facilitation of prostitution.”

FOSTA and the state laws it ties into did not precisely define what those terms meant, nor set the level of culpability for sites that unknowingly or negligently host such content. As a result, sites used by sex workers to share information about clients or even used for discussions about LGBTQIA+ topics having nothing to do with solicitation were shuttered.

So FOSTA chilled lawful speech, but also made sex workers less safe and the industry less accountable, harming some of the people the law’s authors fervently hoped to protect. This was the judgment of advocacy groups like the ACLU that opposed FOSTA all along, but also academics who support changes to Section 230 yet concluded FOSTA’s final product was “confusing” and not “executed artfully.”

That kind of confusion and poor execution is possible even when some of the targeted conduct and content is clearly unlawful. But, rewriting Section 230 to facilitate the take-down of hate speech that is not currently unlawful would be even trickier and fundamentally incoherent. Saying platforms ought to be liable for speech and conduct that would not expose the original speaker to liability would have a chilling impact, and likely still wouldn’t lead to sites making consistent choices about what to take down.

The Section 230 debate ought to be about when it’s appropriate or beneficial to impose legal liability on parties hosting the speech of others. Perhaps this larger debate on the legal limits of speech should be broader. But that has to happen honestly and on its own terms, not get shoehorned into the 230 debate.

Section 230 Lets Platforms Choose To Take Down Hate

Platforms still aren’t doing enough to stop hate, but what they are doing is in large part thanks to having 230 in place.

The second operative provision in the statute is what Donald Trump, several Republicans in Congress, and at least one Republican FCC commissioner are targeting right now. It says “interactive computer services” can “in good faith” take down content not only if it is harassing, obscene or violent, but even if it is “otherwise objectionable” and “constitutionally protected.”

That’s what much hate speech is, at least under current law. And platforms can take it down thanks not only to the platforms’ own constitutionally protected rights to curate, but because Section 230 lets them moderate without exposing themselves to publisher liability as the pre-1996 cases suggested.

That gives platforms a freer hand to moderate their services. It lets Free Press and its partners demand that platforms enforce their own rules against the dissemination of hateful or otherwise objectionable content that isn’t unlawful, but without tempting platforms to block a broader swath of political speech and dissent up front.

Tackling the spread of online hate will require a more flexible multi-pronged approach that includes the policies recommended by Change the Terms, campaigns like Stop Hate for Profit, and other initiatives. Platforms implementing clearer policies, enforcing them equitably, enhancing transparency, and regularly auditing recommendation algorithms are among these much-needed changes.

But changing Section 230 alone won’t answer every question about hate speech, let alone about online business models that suck up personal information to feed algorithms, ads, and attention. We need to change those through privacy legislation. We need to fund new business models too, and we need to facilitate competition between platforms on open broadband networks.

We need to make huge corporations more accountable by limiting their acquisition of new firms, changing stock voting rules so people like Mark Zuckerberg aren't the sole emperors over these vastly powerful companies, and giving shareholders and workers more rights to ensure that companies are operated not just to maximize revenue but in socially responsible ways as well.

Preserving not just the spirit but the basic structure of Section 230 isn’t an impediment to that effort, it’s a key part of it.

Gaurav Laroia and Carmen Scurato are both Senior Policy Counsel at Free Press.

Filed Under: cda 230, content moderation, disinformation, hate speech, misinformation, section 230

The Trust & Safety Professional Association: Advancing The Trust And Safety Profession Through A Shared Community Of Practice

from the strength-through-collaboration dept

For decades, trust and safety professionals in content moderation, fraud and risk, and safety — have faced enormous challenges, often under intense scrutiny. In recent years, it’s become even more clear that the role of trust and safety professionals are both critically important and difficult. In 2020 alone, we’ve seen an increasing need for this growing class of professionals to combat a myriad of online abuse related to systemic racism, police violence, and COVID-19 — such as hate speech, misinformation, price gouging, and phishing — while keeping a safe space for connecting people with vital, authoritative information, and with each other.

Despite the enormous impact trust and safety individuals have towards protecting the online and offline safety of people, the professional community has historically been dispersed, siloed, and informally organized. To date — unlike, say, in privacy — no organization has focused on the needs of trust and safety professionals in a way that builds a shared community of practice.

This is why we founded the Trust & Safety Professional Association (TSPA) and the Trust & Safety Foundation Project (TSF) — something we think is long overdue. TSPA is a new, nonprofit, membership-based organization that will support the global community of professionals who develop and enforce principles and policies that define acceptable behavior online. TSF will focus on improving society’s understanding of trust and safety, including the operational practices used in content moderation, through educational programs and multidisciplinary research.

Since we launched in June, we’ve gotten a number of questions about what TSPA and TSF will (and won’t) do. So we thought we’d tackle them right here, and share more with you about who’s included, why we launched now, and what our vision is for the future. You can also hear us talk more about both organizations on episode 247 of the Techdirt podcast. And if you want to know even more, we’re all ears!

Q&A

Q. How do you define trust and safety? Don’t you mean content moderation?

We define trust and safety professionals as the global community of people who develop and enforce policies that define acceptable behavior online.

Content moderation is a big part of trust and safety, and the area that gets the most public attention these days. But trust and safety also includes the people who tackle financial risk and fraud, those who process law enforcement requests, engineers who work on automating these policies, and more. TSPA is for the professionals who work in all of those areas.

Q. What’s the difference between TSPA and TSF?

TSPA is a 501(c)(6) membership-based organization for professionals who develop and enforce principles and policies that define acceptable behavior and content online. Think ABA for lawyers, or IAPP for privacy people, but for those working in trust and safety, who can use TSPA to connect with a network of peers, find resources for career development, and exchange best practices.

TSF is a fiscally sponsored project of the Internet Education Foundation and focuses on research.

The two organizations are complementary, but have distinct missions and serve different communities. TSPA is a membership organization, while TSF has a charitable purpose.

Q. Why are you doing this now?

We first started discussing the need for something like this more than two years ago, in the wake of the first Content Moderation at Scale (COMO) conference in Santa Clara. The conference was convened by one of TSPA’s founders and board members, Santa Clara University law professor Eric Goldman, which you can read about right here. After the first COMO get-together It was clear that there was a need for more community amongst people who do trust and safety work.

Q. Are you taking positions on policy issues or lobbying?

Nope. We’re not advocating for public policy positions on behalf of corporate supporters or anyone else. We do want to help people better understand trust and safety as a field, as well as shed light on the challenges that trust and safety professionals face.

Q. Ok, so you launched. Now what?

For TSPA, we’re in the process of planning some virtual panel discussions that will happen before the end of the year on various topics related to trust and safety. Topics will range from developing wellness and resilience best practices, to operational challenges in the face of current events like the US presidential election and COVID-19. Longer term, we’re working on professional development offerings, like career advancement bootcamps and a job board.

Over at TSF, we partnered with the folks right here from Techdirt to launch with a series of case studies from the Copia Institute that illustrate challenging choices that trust and safety professionals face. We are also hosting an ongoing podcast series called Flagged for Review, with interviews from people with expertise in trust and safety.

We’re also looking for founding Executive Director, who can get TSPA and TSF off the ground. Send good candidates our way.

Q. Sounds pretty good. How do I get involved?

Sign up here so we can share more with you about TSPA and TSF in the coming months as we open our membership and develop our offerings. Follow us on Twitter, too. If you work for one of our corporate supporters, you can reach out to your trust and safety leadership as well to find out more. We’d also love to hear from organizations and people who want to help out, or whose work is complementary to our own. We’re excited to further develop and support the community of online trust and safety professionals.

Filed Under: content moderation, trust and safety
Companies: tsf, tspa

Social Media Can Apply COVID-19 Policies To Reduce the Spread of Election Disinformation

from the do-more,-do-better dept

With less than eighty days until Election Day and a pandemic surging across the country, disinformation continues to spread across social media platforms, posing dangers to public health, voting rights, and our democracy. Time is short and social media platforms need to ramp up their efforts to combat election disinformation and online voter suppression — just as they have with COVID-19 disinformation.

Social media platforms have content moderation policies in place to counter both COVID-19 disinformation and election disinformation. However, platforms seem to be taking a more proactive approach to combating COVID-19 disinformation by building tools, spending significant resources, and most importantly, changing their content moderation policies to reflect the evolving nature of inaccurate information about the virus.

To be clear, COVID-19 disinformation is still rapidly spreading online. However, the platforms’ actions on the pandemic demonstrate they can develop specific policies to address and remove this harmful content. Platforms’ efforts to mitigate election disinformation, on the other hand, are falling short, due to the significant gaps that remain in their content moderation policies. Platforms should seriously examine how their COVID-19 disinformation policies can apply to reducing the spread of election disinformation and online voter suppression

Disinformation on social media can spread in a variety of ways including (1) the lack of prioritizing authoritative sources of information and third-party fact-checking; (2) algorithmic amplification and targeting; and (3) platform self-monetization. Social media platforms have revised their content moderation policies on COVID-19 to address many of the ways disinformation can spread about the pandemic.

For example, Facebook, Twitter, and YouTube all direct their users to authoritative sources of COVID-19 information. In addition, Facebook works with fact-checking organizations to review and rate pandemic-related content; YouTube utilizes fact-checking information panels; and Twitter is beginning to add fact-checked warning labels. Twitter has also taken the further step of expanding its definition on what it considers harmful content in order to capture and remove more inaccurate content related to the pandemic. To reduce the harms of algorithmic amplification, Facebook uses automated tools to downrank COVID-19 disinformation. Additionally, Facebook places restrictions on its advertisement policy to prevent the sale of fraudulent medical equipment and the platform prohibits ads that use exploitative tactics to create a panic over the pandemic as two methods for stopping the monetization of pandemic-related disinformation.

These content moderation policies have resulted in social media platforms taking down significant amounts of COVID-19 disinformation including recent posts from President Trump. Again, disinformation about the pandemic persists on social media. But these actions show the willingness of platforms to take action and reduce the spread of this content.

In comparison, social media platforms have not been as proactive in enforcing or developing new policies to respond to the spread of election disinformation. Platforms’ civic integrity policies are primarily limited to prohibiting inaccurate information about the processes of voting (e.g., misrepresentations about the dates and times people can vote). But even these limited policies are not being consistently enforced.

For example, Twitter placed a warning label on one of Trump’s inaccurate tweets about mail-in-voting procedures but have taken no action on other similar tweets from the president. Further, social media platforms current policies may not be broad enough to take into account emerging voter suppression narratives about voter fraud and election rigging. Indeed, Trump has pushed inaccurate content about mail-in-voting across social media platforms, falsely claiming it will lead to voter fraud and election rigging. With many states expanding their mail-in-voting procedures due to the pandemic, Trump’s continued inaccurate attacks on this method of voting threaten to confuse and discourage eligible voters from casting their ballot.

Platform content moderation policies also contain significant holes that bad actors continue to exploit to proliferate online voter suppression. For example, Facebook refuses to fact-check political ads even if they contain demonstrably false information that discourage people from voting. President Trump’s campaign has taken advantage of this by flooding the platform with hundreds of ads that spread disproven claims about voter fraud. Political ads with election disinformation can be algorithmically amplified or micro-targeted to specific communities to suppress their vote.

Social media platforms including Facebook and Twitter have recently announced new policies they will be rolling out to fight online voter suppression. As outlined above, there are some lessons platforms can learn from their efforts in combatting COVID-19 disinformation.

First, social media platforms should prioritize directing their users to authoritative sources of information when it comes to the election. Authoritative sources of information include state and local election officials. Second, platforms must consistently enforce and expand their content moderation policies as appropriate to remove election disinformation. Like their COVID-19 disinformation policies, platforms should build better tools and expand definitions of harmful content when it comes to online voter suppression. Finally, platforms must address the structural problems that allow bad actors to engage in online voter suppression tactics including algorithmic amplification and targeted advertisements.

COVID-19 – as dangerous and terrifying an experience as it has been – has at least proven that when platforms want to step up their efforts to stop the spread of disinformation, they can. If we want authentic civic engagement and a healthy democracy that enables everyone’s voices to be heard, then we need digital platforms to ramp up their fight against online voter suppression, too. Our voices – and the voices of those in marginalized communities -- depend on it.

Just as combating COVID-19 disinformation is important to our public health, reducing the spread of election disinformation is critical to authentic civic engagement and a healthy democracy. As part of our efforts to stop the spread of online voter suppression, Common Cause will continue to monitor social media platforms for election disinformation and encourages readers to report any inaccurate content to our tip line. At the end of the day, platforms themselves must step up their fight against new online voter suppression efforts.

Yosef Getachew serves as the Media & Democracy Program Director for Common Cause. Prior to joining Common Cause, Yosef served as a Policy Fellow at Public Knowledge where he’s worked on a variety of technology and communications issues. His work has focused on broadband privacy, broadband access and affordability, and other consumer issues.

Filed Under: content moderation, covid-19, disinformation, election disinformation

Content Moderation And Human Nature

from the unavoidably-human dept

It should go without saying that communication technologies don’t conjure up unfathomable evils all by themselves. They are a convenience-enhancer, a conduit, and a magnifying lens amplifying something that’s already there: our deeply flawed humanity. Try as we might to tame it (and boy have we tried), human nature will always rear its ugly head. Debates about governing these technologies should start by making the inherent tradeoffs more explicit.

Institutions

First, a little philosophizing. From the social contract onwards, a significant amount of resources have been allocated to attempting to subdue human nature’s predilection for self-preservation at all costs. Modern society is geared towards improving the human condition by striving to unlearn — or at least overpower — our more primitive responses.

One such attempt is the creation of institutions, with norms, rules, cultures and, on paper, inherently stronger principles than those rooted deep inside people.

It’s difficult to find ideologies that don’t allow for some need for institutions. Even the most ardent of free market capitalists acquiesce to the — limited, in their mindset — benefits of certain institutions. Beyond order and a sense of impartiality, institutions help minimize humans’ unchecked power in consequential choices that can impact wider society.

One ideal posits that institutions (corporations, parties, governments) given unfettered control over society could rid us of the aspects of our humanity that we’ve so intently tried to escape, bringing forth prosperity, equality, innovation, and progress. The fundamental flaw in that reasoning is that institutions are still intrinsically connected to humanity; created, implemented, and staffed by fallible human beings.

However strict the boundaries in which humans are expected to operate, the potential for partial or even total capture is very high. The boundaries are rarely entirely solid, and even if they were, humans always have the option to not comply. Bucking the system is not just an anomaly, it’s revered in a large portion of non-totalitarian regimes as a sign of independence, strong individuality, and as a characteristic of those lauded as mavericks.

The power of institutional norms tasked with guarding against the worst of what humans can offer is proven to be useless when challenged by people for whom self-preservation is paramount. A current and facile example is the rise to power of Donald Trump and his relentless destruction of society-defining unwritten rules.

Even without challenging the institution, a turn towards self-indulgence is easily achievable, forging a path to a reshaping in its image. The most obvious example is that of communism, wherein the lofty goal of equality is operationalized through a party-state apparatus to ostensibly distribute equally the spoils of society’s labor. As history has shown, this is contingent on the sadly unlikely situation wherein all those populating institutions are genuinely altruistic. Invariably, the best-case scenario dissipates, if it ever materialized, and inequality deepens — the opposite of the desired goal.

This is not a tacit endorsement of a rule-less, institution-less dystopia simply because rules and institutions are not adept at a practically impossible task. Instead, this should be read as a cautionary tale for overextending critical aspects of society and treating them as panacea, rather than a suitable and mostly successful palliative.

Artificial Intelligence

Armed with the continuous failure of institutions to overcome human nature, you’d think we would stop trying to remove our imperfect selves from the equation.

But what we’ve seen for more than a decade now has been technology that directly and distinctly promises to remove our worst impulses, if not humans entirely, from thinking, acting, or doing practically anything of consequence. AI, the ultimate and literal deus ex machina, is advertised as a solution for a large number of much smaller concerns. Fundamentally, its solution to these problems is ostensibly removing the human element.

Years of research, experiments, blunders, mistakes and downright evil deeds have led us to safely conclude that artificial intelligence is as successful at eliminating the imperfect human as the “you wouldn’t steal a car” anti-piracy campaign was at stopping copyright infringement. This is not to denigrate the important and beneficial work scientists and engineers have put into building intelligent automation tasked with solving complex problems.

Technology, and artificial intelligence in particular, is created, run and maintained by human beings with perspectives, goals, and inherent biases. Just like institutions, once a glimpse of positive change or success is evident, we extrapolate it far beyond its limits and task it with the unachievable and unenviable goal of fixing humanity — by removing it from the equation.

Platforms

Communication technology is not directly tasked with solving society, it simply is meant as a tool to connect us all. Much like AI, it has seemingly elegant solutions for messy problems. It’s easy to see that thanks to tech platforms, be they bulletin boards or TikTok, distance becomes trivial in maintaining connection. Community can be built and fostered online, otherwise marginalized voices can be heard, and businesses can be set up and grow digitally. Even loneliness can be alleviated.

With such a slew of real and potential benefits, it’s no wonder that we started to ascribe them with increasingly more consequential roles for society; roles these technologies were never built for and are far beyond their technical and ethical capabilities.

The Arab Spring in the early 2010s wasn't just a liberation movement by oppressed and energized populations. It was also an opportunity for free PR for now tech-giants Twitter and Facebook, as various outlets and pundits branded revolutions with their names. It didn't help that CEOs and tech executives seized on this narrative and, in typical Silicon Valley fashion, took to promising things akin to a politician trying to get elected.

When you set the bar that high, expectations understandably follow. The aura of tech solutionism implies such earth-shattering advancements as ordinary.

Nearly everyone can picture the potential good for society these technologies can do. And while we may all believe in that potential, the reality is that, so far, communication technologies have mostly provided convenience. Sometimes this convenience is in fact live-saving, but mostly it’s just an added benefit.

Convenience doesn’t alter our core. It doesn’t magically make us better humans or create entirely different societies. It simply lifts a few barriers from our path. This article may be seen as an attempt to minimize the perceived role of technology in society, in order to subsequently deny it and its makers any blame for how society uses it. But that is not what I am arguing.

An honest debate about responsibility has to fundamentally start with a clear understanding of the actual task something accomplishes, the perceived task others attribute to it, and its societal and historical context. A technology that provides convenience should not be fundamental to the functioning of a society. Convenience can easily become so commonplace that it ceases to be an added benefit but an integral part of life where the prospect of it being taken away is met with screams of bloody murder.

Responsibility has to be assigned to the makers, maintainers and users of communication technology, by examining which barriers are being lifted and why. There is plenty of responsibility there to be had, and I am involved in a couple of projects that try to untangle this complex mess. However, these platforms are not the reason for the negative parts of life, they are merely the conduit.

Yes, a sentient conduit can tighten or loosen its grip, divert, amplify, temporarily block messages, but it isn’t the originator of those messages, or of the intent behind it. It can surely be extremely inviting for messages of hate and division, maybe because of business models, maybe because of engineering decisions, or maybe simply because growth and scale never actually happened in a proper way. But that hate and division is endemic to human nature, and to assume that platforms can do what institutions have persistently failed to do, namely entirely eradicate it, is nonsensical.

Regulation

It is clear that platforms, reaching the size and ubiquity that they have, require updated and smart regulations in order to properly balance their benefits and the risks. But the push (and counter-push) to regulate has to start from a perspective that understands both fundamental leaps: platforms are to human nature what section 230 (or any other national-level intermediary liability law) is to the First Amendment (or any national level text that inscribes the social consensus on free speech).

If your issue is with hate and hate speech, the main thing you have to contend with are human nature and the First Amendment, not just the platforms and section 230. Without a doubt, both the platforms and section 230 are choices and explicit actions built on top of the other two, and are not fundamentally the only or best form of what they could be.

But a lot of the issues that bubble up within the content moderation and intermediary liability space come from a concern over the boundaries. That concern is entirely related to the broader contexts rather than the platforms or the specific legislation.

Regulating platforms has to start from the understanding that tradeoffs, most of which are cultural in nature, are inevitable. To be clear: there is no way to completely stop evil from happening on these platforms without making them useless.

If we were to simply ignore hate speech, we’d eliminate convenience and in some instances invalidate the very existence of these platforms. That should not be an issue if these platforms were still seen as simple conveyors of convenience, but they are currently being seen as much more than that.

Tech executives and CEOs have moved into the fascinating space wherein they have to protect their market power to assuage their shareholders, treat their products as mind-meltingly amazing to gain and keep users, yet imply their role in society is transient and insignificant in order to mollify policy-makers all at the same time.

The convenience afforded by these technologies is allowing nefarious actors to cause substantial harm to a substantial number of people. Some users get death threats, or even have their life end tragically because of interactions on these platforms. Others will have their most private information or documents exposed, or experience sexual abuse or trauma through a variety of ways.

Unfortunately, these things happen in the offline world as well, and they are fundamentally predicated on the regulatory/institutional context and the tools that allow them to manifest. The tools are not off the hook. Their propensity to not minimize harm, online and off, are due for important conversations. But they are not the cause. They are the conduit.

Thus, the ultimate goal of “platforms existing without hate or violence” is very sadly not realistic. Neither are tradeoffs such as being ok with stripping fundamental rights in exchange for a safer environment, or being ok with some people suffering immense trauma and pain simply because one believes in the concept of open speech.

Maybe the solution is to not have these platforms at all, or ask them to change substantially. or maybe it’s to calibrate our expectations, or maybe yet, to address the underlying issues in our society. Once we see what the boundaries truly are, any debate becomes infinitely more productive.

This article is not advancing any new or groundbreaking ideas. What it does is identify crucial and seemingly misunderstood pieces of the subtext and spell it out. Sadly, the fact that these more or less evident issues needed to be said in plain text should be the biggest take-away.

As a qualitative researcher, I learned that there is no way to “de-bias” my work. Trying to remove myself from the equation results in a bland “view from nowhere” that is ignorant of the underlying power dynamics and inherent mechanisms of whatever I am studying. However, that doesn’t mean we take off our glasses when trying to see for fear of the glasses influencing what we see, because that would actually make us blind. We remedy that by acknowledging our glasses as well.

A communication platform (company, tech, product) that doesn’t have inherent biases is impossible. But that shouldn’t mean that we can’t try to ask it to be better, either through regulation, collaboration or hostile action. We just have to be cognizant of the place we’re standing when asking, the context, potential consequences and as this piece hopefully shows, what it can’t actually do.

The conversation surrounding platform governance would benefit immensely from these tradeoffs being made explicit. It would certainly dial down the rhetoric and (genuine) visceral attitudes towards debate as it would force those directly involved or invested in one outcome to carefully assess the context and general tradeoffs.

David Morar, PhD is an academic with the mind of a practitioner and currently a Fellow at the Digital Interests Lab and a Visiting Scholar at GWU’s Elliott School of International Affairs.

Filed Under: content moderation, human nature

Content Moderation Knowledge Sharing Shouldn't Be A Backdoor To Cross-Platform Censorship

from the too-big-of-a-problem-to-tackle-alone dept

Ten thousand moderators at YouTube. Fifteen thousand moderators at Facebook. Billions of users, millions of decisions a day. These are the kinds of numbers that dominate most discussions of content moderation today. But we should also be talking about 10, 5, or even 1: the numbers of moderators at sites like Automattic (Wordpress), Pinterest, Medium, and JustPasteIt—sites that host millions of user-generated posts but have far fewer resources than the social media giants.

There are a plethora of smaller services on the web that host videos, images, blogs, discussion fora, product reviews, comments sections, and private file storage. And they face many of the same difficult decisions about the user-generated content (UGC) they host, be it removing child sexual abuse material (CSAM), fighting terrorist abuse of their services, addressing hate speech and harassment, or responding to allegations of copyright infringement. While they may not see the same scale of abuse that Facebook or YouTube does, they also have vastly smaller teams. Even Twitter, often spoken of in the same breath as a “social media giant,” has an order of magnitude fewer moderators at around 1,500.

One response to this resource disparity has been to focus on knowledge and technology sharing across different sites. Smaller sites, the theory goes, can benefit from the lessons learned (and the R&D dollars spent) by the biggest companies as they’ve tried to tackle the practical challenges of content moderation. These challenges include both responding to illegal material and enforcing content policies that govern lawful-but-awful (and mere lawful-but-off-topic) posts.

Some of the earliest efforts at cross-platform information-sharing tackled spam and malware such as the Mail Abuse Prevention System (MAPS) — which maintains blacklists of IP addresses associated with sending spam. Employees at different companies have also informally shared information about emerging trends and threats, and the recently launched Trust & Safety Professional Association is intended to provide people working in content moderation with access to “best practices” and “knowledge sharing” across the field.

There have also been organized efforts to share specific technical approaches to blocking content across different services, namely, hash-matching tools that enable an operator to compare uploaded files to a pre-existing list of content. Microsoft, for example, made its PhotoDNA tool freely available to other sites to use in detecting previously reported images of CSAM. Facebook adopted the tool in May 2011, and by 2016 it was being used by over 50 companies.

Hash-sharing also sits at the center of the Global Internet Forum to Counter Terrorism (GIFCT), an industry-led initiative that includes knowledge-sharing and capacity-building across the industry as one of its 4 main goals. GIFCT works with Tech Against Terrorism, a public-private partnership launched by the UN Counter-Terrrorism Executive Directorate, to “shar[e] best practices and tools between the GIFCT companies and small tech companies and startups.” Thirteen companies (including GIFCT founding companies Facebook, Google, Microsoft, and Twitter) now participate in the hash-sharing consortium.

There are many potential upsides to sharing tools, techniques, and information about threats across different sites. Content moderation is still a relatively new field, and it requires content hosts to consider an enormous range of issues, from the unimaginably atrocious to the benignly absurd. Smaller sites face resource constraints in the number of staff they can devote to moderation, and thus in the range of language fluency, subject matter expertise, and cultural backgrounds that they can apply to the task. They may not have access to — or the resources to develop — technology that can facilitate moderation.

When people who work in moderation share their best practices, and especially their failures, it can help small moderation teams avoid pitfalls and prevent abuse on their sites. And cross-site information-sharing is likely essential to combating cross-site abuse. As scholar evelyn douek discusses (with a strong note of caution) in her Content Cartels paper, there’s currently a focus among major services in sharing information about “coordinated inauthentic behavior” and election interference.

There are also potential downsides to sites coordinating their approaches to content moderation. If sites are sharing their practices for defining prohibited content, it risks creating a de facto standard of acceptable speech across the Internet. This undermines site operators’ ability to set the specific content standards that best enable their communities to thrive — one of the key ways that the Internet can support people’s freedom of expression. And company-to-company technology transfer can give smaller players a leg up, but if that technology comes with a specific definition of “acceptable speech” baked in, it can end up homogenizing the speech available online.

Cross-site knowledge-sharing could also suppress the diversity of approaches to content moderation, especially if knowledge-sharing is viewed as a one-way street, from giant companies to small ones. Smaller services can and do experiment with different ways of grappling with UGC that don’t necessarily rely on a centralized content moderation team, such as Reddit’s moderation powers for subreddits, Wikipedia’s extensive community-run moderation system, or Periscope’s use of “juries” of users to help moderate comments on live video streams. And differences in the business model and core functionality of a site can significantly affect the kind of moderation that actually works for them.

There’s also the risk that policymakers will take nascent “industry best practices” and convert them into new legal mandates. That risk is especially high in the current legislative environment, as policymakers on both sides of the Atlantic are actively debating all sorts of revisions and additions to intermediary liability frameworks.

Early versions of the EU’s Terrorist Content Regulation, for example, would have required intermediaries to adopt “proactive measures” to detect and remove terrorist propaganda, and pointed to the GIFCT’s hash database as an example of what that could look like (CDT joined a coalition of 16 human rights organizations recently in highlighting a number of concerns about the structure of GIFCT and the opacity of the hash database). And the EARN-IT Act in the US is aimed at effectively requiring intermediaries to use tools like PhotoDNA—and not to implement end-to-end encryption.

Potential policymaker overreach is not a reason for content moderators to stop talking to and learning from each other. But it does mean that knowledge-sharing initiatives, especially formalized ones like the GIFCT, need to be attuned to the risks of cross-site censorship and eliminating diversity among online fora. These initiatives should proceed with a clear articulation of what they are able to accomplish (useful exchange of problem-solving strategies, issue-spotting, and instructive failures) and also what they aren’t (creating one standard for prohibited — much less illegal— speech that can be operationalized across the entire Internet).

Crucially, this information exchange needs to be a two-way street. The resource constraints faced by smaller platforms can also lead to innovative ways to tackle abuse and specific techniques that work well for specific communities and use-cases. Different approaches should be explored and examined for their merit, not viewed with suspicion as a deviation from the “standard” way of moderating. Any recommendations and best practices should be flexible enough to be incorporated into different services’ unique approaches to content moderation, rather than act as a forcing function to standardize towards one top-down, centralized model. As much as there is to be gained from sharing knowledge, insights, and technology across different services, there’s no-one-size-fits-all approach to content moderation.

Emma Llansó is the Director of CDT’s Free Expression Project, which works to promote law and policy that support Internet users’ free expression rights in the United States and around the world. Emma also serves on the Board of the Global Network Initiative, a multistakeholder organization that works to advance individuals’ privacy and free expression rights in the ICT sector around the world. She is also a member of the multistakeholder Freedom Online Coalition Advisory Network, which provides advice to FOC member governments aimed at advancing human rights online.

Filed Under: best practices, censorship, content moderation, cross-platform, gifct, hashes, knowledge sharing, maps