Does An Internet Infrastructure Taxonomy Help Or Hurt?

from the might-create-more-problems dept

We've been running our Greenhouse discussion on content moderation at the infrastructure level for a bit now, and normally all of the posts for these discussions come from expert guest commentators. However, I'm going to add my voice to the collection here because there's one topic that I haven't seen covered, and which is important, because it comes up whenever I'm talking to people about content moderation at the infrastructure level: do we need a new taxonomy for internet infrastructure to better have this discussion?

The thinking here is that the traditional OSI model of the internet layers is somewhat outdated and not particularly relevant to discussions such as this one. Also, it's hellishly confusing as is easily demonstrated by this fun Google box of "people also ask" on a search on "internet layers."

Clearly, lots of people are confused.

Even just thinking about what counts as infrastructure can be confusing. One of my regular examples is Zoom, the video conferencing app that has become standard and required during the COVID pandemic: is that infrastructure? Is that edge? It has elements of both.

But the underlying concern in this entire discussion is that most of the debate around content moderation is about clear edge providers: the services that definitely touch the end users: Facebook, Twitter, YouTube, etc. And, as I noted in my opening piece, there is a real concern that because the debate focuses on those companies, and there appears to be tremendous appetite for policy making and regulating those edge providers, that any new regulations may not realize how they will also impact infrastructure providers, where the impact could be much more seismic.

Given all that, many people have suggested that a "new taxonomy" might be useful, to help "carve out" infrastructure services from any new regulations regarding moderation. It's not hard to understand a concept like "maybe this rule should apply to social media sites, but not to domain registrars" for example.

However, the dangers in building up such a taxonomy greatly outweigh any such benefits. First, as noted earlier, any new taxonomy is going to be fraught with difficult questions. It's not always clear what really is infrastructure these days. We've already discussed how financial intermediaries are, in effect infrastructure for the internet these days -- and that's a very different participant than the traditional OSI model of internet layers. Same with advertising firms. And I've already mentioned Zoom as a company that clearly has an edge component, but feels more like it should be considered infrastructure. Part of that is just the nature of how the internet works, in which some of the layers are merged. Marc Andreessen famously noted that software eats the world, but the internet itself is subsuming more and more traditional infrastructure as well -- and that creates complications.

On top of that, this is an extremely dynamic world. Part of the reason why the OSI model feels obsolete is because it is. Things change, and they can change fairly rapidly on the internet. So any taxonomy might be obsolete by the time it's created, and that's extremely dangerous if the plan is to use it for classifying services for the purpose of regulation.

The final concern with such a taxonomy is simply that it seems likely to encourage regulatory approaches in places where it's not clear if it's actually needed. If the intent of such a taxonomy is to help lawmakers write a law that only puts its focus on the edge players, that's unlikely how it will remain. Once such a mapping is in place, the temptation (instead) will simply be to create new rules for each layer of the new stack.

A new taxonomy may sound good as a first pass, but it will inevitably create more problems than it solves.

Techdirt and EFF are collaborating on this Techdirt Greenhouse discussion. On October 6th from 9am to noon PT, we'll have many of this series' authors discussing and debating their pieces in front of a live virtual audience (register to attend here).

Filed Under: content moderation, infrastructure, regulations, taxonomy

OnlyFans Isn't The First Site To Face Moderation Pressure From Financial Intermediaries, And It Won't Be The Last

from the money-and-moderation dept

In August, OnlyFans made the stunning announcement that it planned to ban sexually explicit content from its service. The site, which allows creators to post exclusive content and interact directly with subscribers, made its name as a host for sexually-oriented content. For a profitable website to announce a ban of the very content that helped establish it was surprising and  dismaying to the sex workers and other creators who make a living on the site.

OnlyFans is hardly the first site to face financial pressure related to the content it publishes. Advertiser pressure has been a hallmark of the publishing industry, whether in shaping what news is reported and published, or withdrawing support when a television series breaks new societal ground.

Publishers across different kinds of media have historically been vulnerable to the demands of their financial supporters when it comes to restricting the kinds of media they distribute. And, with online advertising now accounting for the majority of total advertising spending in the U.S., we have seen advertisers recognize their power to influence how major social media sites moderate, the organization of campaigns like Stop Hate for Profit, or the development of “brand safety” standards for acceptable content.

But OnlyFans wasn’t bowing to advertiser demands; instead, it says it faced an even more fundamental kind of pressure coming from its financial intermediaries. OnlyFans explained in a statement that it planned to ban explicit content “to comply with the requests of our banking partners and payout providers.”

Financial intermediaries are key actors in the online content hosting ecosystem. The websites and apps that host people’s speech depend on banks, credit card companies, and payment processors to do everything from buying domain names and renting server space to paying their engineers and content moderators. Financial intermediaries are also essential for receiving payments from advertisers and ad networks, processing purchases, and enabling user subscriptions. Losing access to a bank account, or getting dropped by a payment processor, can make it impossible for a site to make money or pay its debts, and can result in the site getting knocked offline completely. 

This makes financial intermediaries obvious leverage points for censorship, including through government pressure. Government officials may target financial intermediaries with threats of legal action or reputational harm, as a way of pursuing censorship of speech that they cannot actually punish under the law. 

In 2010, for example, U.S Congressmen Joe Lieberman and Peter King reportedly pressured MasterCard in private to stop processing payments for Wikileaks; this came alongside a very public campaign of censure that Lieberman was conducting against the site. Ultimately, Wikileaks lost its access to so many banks, credit card companies, and payment processors that it had to temporarily suspend its operations; it now accepts donations through various cryptocurrencies or via donations made to the Wau Holland Foundation (which has led to pressure on the Foundation in turn).

Credit card companies were also the target of the 2015 campaign by Sheriff Tom Dart to shutter Backpage.com. Dart had previously pursued charges against another classified-ads site, Craigslist, for solicitation of prostitution, based on the content of some ads posted by users, and had been told unequivocally by a district court that Section 230 barred such a prosecution. 

In pursuing Backpage for similar concerns about enabling prostitution, Dart took a different tack: He sent letters to Visa and MasterCard demanding that they “cease and desist” their business relationships with Backpage, implying that the companies could face civil and criminal charges. Dart also threatened to hold a damning press conference if the credit card companies did not sever their ties with the website. 

The credit card companies complied, and terminated services to Backpage. Backpage challenged Dart’s acts as unconstitutional government coercion and censorship in violation of the First Amendment. (CDT, EFF, and the Association for Alternative Newsmedia filed an amicus brief in support of Backpage’s First Amendment arguments in that case.) The Seventh Circuit agreed and ordered Dart to cease his unconstitutional pressure campaign. 

But this did not result in a return to the status quo, as the credit card companies declined to restore service to Backpage, showing how long-lasting the effects of such pressure can be. Backpage is now offline—but not because of Dart—the federal government seized the site as part of its prosecution of several Backpage executives, which was declared a mistrial earlier this month.

Since that time, the pressures on payment processors and other financial intermediaries have only increased. FOSTA-SESTA, for example, created a vague new federal crime of “facilitation of prostitution” that has rendered many intermediaries uncertain about whether they face legal risk in association with content related to sex work. After Congress passed FOSTA in 2018, Reddit and Craigslist shuttered portions of their sites, multiple sites devoted to harm reduction went offline, and sites like Instagram, Patreon, Tumblr, and Twitch have taken increasingly strict stances against nudity and sexual content. 

So while advertisers may be largely motivated by commercial concerns and brand reputation, financial intermediaries such as banks and payment processors are also driven by concerns over legal risk when they try to limit what kinds of speech and speakers are accessible online. 

Financial institutions, in general, are highly regulated. Banks, for example, face obligations such as the “Customer Due Diligence” rule in the US, which requires them to verify the identity of account holders and develop a risk profile of their business. Concerns over legal risk can cause financial intermediaries to employ ham-handed automated screening techniques that lead to absurd outcomes, such as when Paypal canceled the account of News Media Canada in 2017 for promoting the story “Syrian Family Adopts To New Life”, or when Venmo (which is owned by PayPal) reportedly blocked donations to the Palestine Children’s Relief Fund in May 2021.

As pressures relating to online content and UGC-related businesses grow, some financial intermediaries are taking a more systemic approach to evaluating the risk that certain kinds of content pose to their own businesses. In this, financial intermediaries are mirroring a trend seen in content regulation debates more generally, on both sides of the Atlantic. 

MasterCard, for example, in April announced changes to its policy for processing payments related to adult entertainment. Starting October 15, MasterCard will require that banks connecting merchants to the MasterCard network certify that those merchants have processes in place to maintain age and consent documentation for the participants in sexually explicit content, along with specific “content control measures.”

These include pre-publication review of content and a complaint procedure that can address reports of illegal or nonconsensual content within seven days, including a process by which people depicted in the content can request its removal (which MasterCard confusingly calls an “appeals” process). In other words, MasterCard is using its position as the second largest credit card network in the US to require banks to vet website operators’ content moderation processes—and potentially re-shaping the online adult content industry at the same time.

Financial intermediaries are integral to online content creation and hosting, and their actions to censor specific content or enact PACT Act-style systemic oversight of content moderation processes should bring greater scrutiny on their role in the online speech ecosystem. 

As discussed above, these intermediaries are an attractive target for government actors seeking to censor surreptitiously and extralegally, and they may feel compelled to act cautiously if their legal obligations and potential liability are not clear. (For the history of this issue in the copyright and trademark field, see Annemarie Bridy’s 2015 article, Internet Payment Blockades.) Moreover, financial intermediaries are often several steps removed from the speech at issue and may not have a direct relationship with the speaker, which can make them even less likely to defend users’ speech interests when faced with legal or reputational risk.

As is the case throughout the stack, we need more information from financial intermediaries about how they are exercising discretion over others’ speech. CDT joined EFF and twenty other human rights organizations in a recent letter to PayPal and Venmo, calling on those payment processors to publish regular transparency reports that disclose government demands for user data and account closures, as well as the companies’ own Terms of Service enforcement actions against account holders. 

Account holders also need to receive meaningful notice when their accounts are closed and provided the opportunity to appeal those decisions—something notably missing from MasterCard’s guidelines for what banks should require of website operators.

Ultimately, OnlyFans reversed course on its porn ban and announced that they had “secured assurances necessary to support [their] diverse creator community” (It’s not clear if those assurances came from existing payment processors or if OnlyFans has found new financial intermediaries). But as payment processors, banks, and credit card companies continue to confront questions about their role in enabling access to speech online, they should learn from other intermediaries’ experience: once an intermediary starts making judgments about what lawful speech it will and won’t support, the demands on it to exercise that judgment only increase, and the scale of human behavior and expression enabled by the Internet is unimaginably huge. The ratchet of content moderation expectations only turns one way.

Emma Llansó is the Director of CDT's Free Expression Project, where she works to promote law and policy that support Internet users' free expression rights in the United States, Europe, and around the world.

Techdirt and EFF are collaborating on this Techdirt Greenhouse discussion. On October 6th from 9am to noon PT, we'll have many of this series' authors discussing and debating their pieces in front of a live virtual audience (register to attend here).

Filed Under: content moderation, financial intermediaries, infrastructure, payment systems

A New Hope For Moderation And Its Discontents?

from the decentralization-as-the-new-frontier dept

In his post kicking off this series, Mike notes that, “the biggest concern with moving moderation decisions down the stack is that most infrastructure players only have a sledge hammer to deal with these questions, rather than a scalpel.” And, I agree with Jonathan Zittrain and other contributors that governments, activists, and others will increasingly reach down the stack to push for takedowns—and will probably get them. 

So, should we expect more blunt force infra layer takedowns or will infrastructure companies invest in more precise moderation tools? Which one is even worse?

Given the choice to build infrastructure now, would you start with a scalpel? How about many scalpels? Or maybe something less severe but distributed and transparent, like clear plastic spoons everywhere! Will the moderation hurt less if we’re all in it together? With the distributed web, we may get to ask all these questions, and have a chance to make things better (or worse). How?

Let me backup a moment for some mostly accurate natural history. In the 90s, to vastly oversimplify, there was web 1.0: static, server-side pages that arose, more manual than you'd like sometimes, maybe not so easy to search or monetize at scale, but fundamentally decentralized and open. We had webrings and manually curated search lists. Listening to Nirvana in my dorm room I read John Perry Barlow’s announcement that "We are forming our own Social Contract. This governance will arise according to the conditions of our world, not yours. Our world is different," in a green IRC window and believed. 

Ok, not every feature was that simple or open or decentralized. The specter of content moderation haunted the Internet from early days of email and bulletin boards. In 1978, a marketer for DEC sent out the first unsolicited commercial message on ARPANET and a few hundred people told him to knock it off, Gary! Voila, community moderation. 

Service providers like AOL and Prodigy offered portals through which users accessed the web and associated chat rooms, and the need to protect the brand led to predictable interventions. There's a Rosetta Stone of AOL content moderation guidelines from 1994 floating around to remind us that as long as there have been people expressing themselves online, there have been other people doing their best to create workable rule sets to govern that expression and endlessly failing in comic and tragic ways (“‘F--- you’ is vulgar” but ‘my *** hurts’ is ok”). 

Back in the Lascaux Cave there was probably someone identifying naughty animal parts and sneaking over with charcoal to darken them out, for the community, and storytellers who blamed all the community’s ills on that person.

And then after the new millenium, little by little and then all at once, came Web 2.0—the Social Web. Javascript frameworks, personalization, everyone a creator and consumer within (not really that open) structures we now call "Platforms" (arguably even less open when using their proprietary mobile rather than web applications). It became much easier for anyone to create, connect, communicate, and distribute expression online without having to design or host their own pages. We got more efficient at tracking and ad targeting and using those algorithms to serve you things similar to the other things you liked.

We all started saying a lot of stuff and never really stopped. If you're a fan of expression in general, and especially of people who previously didn't have great access to distribution channels expressing themselves more, that's a win. But let's be honest: 500 million tweets a day? We've been on an expression bender for years. And that means companies spending billions, and tens of thousands of enablers—paid and unpaid—supporting our speech bender. Are people happy with the moderation we're getting? Generally not. Try running a platform. The moderation is terrible and the portions are so large!

Who’s asking for moderation? Virtually everyone in different ways. Governments want illegal content (CSAM, terrorist content) restricted on behalf of the people, and some also want harmful but legal content restricted in ways that are still unclear, also for the people. Many want harmful content restricted, which means different things depending on which people, which place, which culture, which content, which coffee roast you had this morning. Civil society groups generally want content restricted related to their areas of expertise and concern (except EFF, who will party like it's 1999 forever I hope). 

There are lots of types of expression where at least some people think moderation is appropriate, for different reasons; misinformation is different from doxxing is different from harassment is different from copyright infringement is different from spam. Often, the same team deals with election protection and kids eating Tide Pods (and does both surprisingly well, considering). There’s a lot to moderate and lots of mutually inconsistent demand to do it coming from every direction.

Ok, so let’s make a better internet! Web 3 is happening and it is good. More specifically, as Chris Dixon recently put it, “We are now at the beginning of the Web 3 era, which combines the decentralized, community-governed ethos of Web 1 with the advanced, modern functionality of Web 2.” Don’t forget the blockchain. Assume that over the next few years, Web 3 infrastructure gets built out and flourishes—projects like Arweave, Filecoin, Polkadot, Sia, and Storj. And applications eventually proliferate; tools for expression, creativity, communication, all the things humans do online, all built in ways that embody the values of the DWeb. 

But wait, the social web experiment of the past 15 years led us to build multi-billion dollar institutions within companies aimed at mitigating harms (to individuals, groups, societies, cultural values) associated with online expression and conduct, and increasingly, complying with new regulations. Private courts. Private Supreme Courts. Teams for safeguarding public health and democratic elections. Tens of thousands poring over photos of nipples, asking, where do we draw the line? Are we going to do all that again? One tempting answer is, let’s not. Let’s fire all the moderators. What’s the worst that could happen?

Another way of asking this question is -- what do we mean when we talk about “censorship resistant” distributed technologies? This has been an element of the DWeb since early days but it’s not very clear (to me at least) how resistant, which censorship, and in what ways.

My hunch is that censorship resistance—in the purest sense of defaulting to immutable content with no possible later interventions affecting its availability—is probably not realistic in light of how people and governments currently respond to Web 2.0. The likely outcome is probably quick escalation to intense conflict with the majority of governments. 

And even for people who still favor a marketplace-of-ideas-grounded “rights” framework, I think they know better than to argue that the cure for CSAM is more speech. There will either have to be ways of intervening or the DWeb is going to be a bumpy ride. But “censorship resistant” in the sense of, “how do we build a system where it is not governments, or a small number of powerful, centralized companies that control the levers at the important choke points for expression?” Now we’re talking. Or as Paul Frazee from Beaker Brower and other distributed projects put it: “The question isn't ‘how do we make moderation impossible?’ The question is, how do we make moderation trustworthy?”

So, when it comes to expression and by extension content moderation, how exactly are we going to do better? What could content moderation look like if done consistent with the spirit, principles, and architecture of Web 3? What principles can we look to as a guide? 

I think the broad principles will come as no surprise to anyone following this space over the past few years (and are not so different from those outlined in Corynne McSherry’s post). They include notice, transparency, due process, the availability of multiple venues for expression, and robust competition between options on many axes—including privacy and community norms, as well as the ability of users to structure their own experience as much as possible. 

Here are some recurring themes:

• Transparency: Clear, broadly available, and auditable information about what interventions are being made, including at the infrastructure level, is a precondition to empowering more people on the web. Rather than building in transparency as an adjunct to moderation (like the essential Lumen database), what if we baked it in from the start? With accommodations and exceptions for privacy where needed; a moderation system that defaults to publicly viewable records as part of every act of intervention.

• Multiplicity: More nodes with smaller power differentials between. More storage providers, more ISPs, more ways of accessing expression, more contexts for encountering it, and more communities with attendant norms, controlling more layers of the stack down to the infra.

• Subsidiarity: How do we give users as much control over their own experience as possible? Allow users to customize their applications so that, if desired, certain infrastructure is favored or disfavored (as opposed to blocked or not blocked), through tools like shareable open source lists of disfavored files, actors, institutions, with reliable information about their provenance. Push decisions to the edge.

• Identity & Reputation: can we structure more experience based on a person’s chosen, persistent, potentially pseudonymous identity? Ideally, assembled from multiple, decentralized sources, and multiple implementations parsing that data rather than one or a few arbiters who store data and make a broadly effective determination. 

We’re in the early days, so the question of how we build decentralized systems resilient to broad attempts to censor—in ways that are familiar from Web 2.0—is still in progress. But the applications are here. The Mastodon project and Element have been at it since 2016. Twitter’s Blue Sky project aims to build decentralized standards for social media with a leader in Jay Graber and a rocking Discord. If Web 2.0 has got you down, consider the possibility that Web 3 might be worth another shot?

Time around let’s not have a small number of actors (commercial or otherwise) making decisions for a large number of people about what expression they can or can’t publish or access. Let’s have more of everything. More decision makers, more venues for expression, more chances for users to make small decisions about what they want to see or not see, more civil society stakeholders with real influence (operationalized through say, governance tokens or whatever other means are at hand). 

At the infrastructure level we need more nodes, more actors, more storage providers, more mechanisms for choosing what is available to more disparate groups, and more ways to make it all visible. With all that and some luck, it may add up to fewer and fewer decisionmakers able to decide what comes down at scale, with a sledgehammer, a vaccine for capital C Censorship by the radical proliferation of small m moderators?

Alex Feerst is a legal & tech executive, trust & safety expert and currently is CEO Murmuration Labs whose clients include the Filecoin Foundation.

Techdirt and EFF are collaborating on this Techdirt Greenhouse discussion. On October 6th from 9am to noon PT, we'll have many of this series' authors discussing and debating their pieces in front of a live virtual audience (register to attend here).

Filed Under: content moderation, cryptocurrency, decentralized, decentralized services, infrastructure, web3

Infrastructure And Content Moderation: Challenges And Opportunities

from the complexity-is-messy dept

The signs were clear right from the start: at some point, content moderation would inevitably move beyond user-generated platforms down to the infrastructure—the place where services operate the heavy machinery of the Internet and without which user-facing services cannot function. Ever since the often-forgotten incident when Amazon stopped hosting Wikileaks after US political pressure took place in 2010, there has been a steady uneasiness regarding the role infrastructure providers could end up playing in the future of content moderation. 

A glimpse of what this would look like came in 2017, when companies like Clouldflare and GoDaddy took affirmative action against content they considered problematic for their business models, in this case white supremacist websites that had been the subject of massive public opprobrium. Since then, that future has become the present reality as the list of infrastructure companies performing content moderation functions keeps growing. 

Content moderation has two inherent qualities that provide important context. 

First, content moderation is generally complex in real-world process design and implementation. There are a host of conflicting rights, diverse procedural norms and competing interests that come into play every time content is posted on the Internet; each case is unique and on some level so it should be treated. 

Second, content moderation is messy because the world is messy: the global nature of the Internet, economies of scale, societal realities and cultural differences create a multi-layered set of considerations that are difficult to reconcile. 

The bright spot in all this messiness and complexity is the hope of due process and the rule of law. The theory is that, in healthy and competitive markets, users have choice and therefore it becomes more difficult for any mistakes to scale. So, if a user’s post gets deleted on one platform, the user should have the option of posting it someplace else.

Of course, such markets are difficult to accomplish and the current Internet market is certainly not in this category. But, the point here is that it is one thing to have one of your postings removed from Facebook and it is another to go completely offline if Cloudflare stops providing you their services. The stakes are completely different. 

For a long time, infrastructure providers were smart enough to stay out of the content business. The argument was that the actors who are responsible for the pipes of the Internet should not concern themselves with the kind of water that runs through them. Their agnosticism was encouraged because their main focus was to provide other services, including security, network reliability and performance.

However, as the Internet evolved, so did the infrastructure providers’ relationship with content. 

In the early days of content moderation, what constituted infrastructure was more discernible and structured. People would usually refer to the Open System Interconnection (OSI) model as a useful analogy, especially with policy makers who were trying to identify the role and responsibilities various companies held in the Internet ecosystem. 

The Internet of today, however, is very different from those days. The layers of the Internet are not distinguishable any longer and, in many cases, participating actors are not just operating at the infrastructure or the application layers. At the same time, and as applications in the Internet were gaining in popularity and use, innovation started moving upstream. 

“Infrastructure” is now being nested on top of other “infrastructure” all within just layer 7 of the OSI stack. Things are not as clear-cut.

This indicates that, in some ways, we should not be surprised that the content moderation conversations seem to gradually be moving downstream. A cloud provider that provides support to a host of different websites, platforms, news outlets or businesses, will inevitably have to deal with issues of content. 

A content delivery network (CDN) will unquestionably face, at some point, the moral dilemma of providing its services to businesses that walk a tightrope with harmful or even illegal content. It really comes down to a simple equation: if user-generated platforms don’t do their job, infrastructure providers will have to do it for them. And, they do. Increasingly often. 

If this is the reality, the question becomes what is the best way for infrastructure providers to do moderation considering current practices of content moderation, the significant chilling effects, and the often-missed trade-offs. 

If we are to follow the “framework, tools, principles” triad, we should be mindful to not reinvent any existing ecosystem. Content moderation is not new and, over the years, a combination of laws and self-regulatory norms ensures a relatively consistent, predictable and stable environment—at least most of the time. 

Section 230 of the CDA in the US, the eCommerce Directive in Europe, Marco Civil in Brazil and other laws around the world have succeeded in creating a space where users and businesses could manage their affairs effectively and know that judicial authorities would treat their cases equally. 

For content moderation at the infrastructure level, a framework based on certainty and consistency is even more of a priority. Legal theory instructs that lack of consistency can diminish the development of norms or it can undermine the way existing ones can manifest themselves. In a similar vein, lack of certainty means the inability to get organized in such a way that complies with the law. For infrastructure providers that support basic and day-to-day functions of the Internet, such a framework becomes indispensable. 

I often say that the Internet is not a monolith. This is not only to demonstrate how the Internet was never meant to perform one single thing, but also to show the importance of designing a legal framework that behaves the same. When we talk about predictability and certainty, we must be conscious of putting in place requirements of clarity, stability and intelligibility so that participating actors can make calculated and informed decisions about the legal consequences of their actions. That’s what made Section 230 a success for more than two decades.

Frameworks without appropriate tools to implement and assess them, however, mean little. Tools are important as they can help maximize the benefits of processes, ultimately increasing flexibility, reducing complexity, and ensuring clarity. Content moderation has consistently been suffering from lack of tools that could clearly exhibit the effects of moderation. Think, for instance, all these times content is taken down and there is no way to say what the true effect is on free speech and on users.

In this context, we need to think of tools as things that would allow us to better understand the scale and chilling effect that content moderation in the infrastructure causes. Here is what I wrote about this last year: 

“A starting point is to perform a regulatory impact assessment for the Internet. It is a tested method of policy analysis, intended to assist policy makers in the design, implementation and monitoring of improvements to the regulatory system; it provides the methodology for producing high quality regulation, which can, in turn, allow for sustainable development, market growth and constant innovation. A regulatory impact assessment constitutes a tool that ensures regulation is proportional (appropriate to the size of the problem it seeks to address), targeted (focused and without causing any unintended consequences), predictable (it creates legal certainty), accountable (in terms of actions and outcomes) and, transparent (on how decisions are made).”

In this sense, tools and frameworks are co-dependent. 

Finally, the legitimacy of any framework and of any tools depends on the existence of principles. In content moderation, it is not the lack of principles that is the problem; on the contrary, it is the abundance of them. Big companies have their own Terms of Service (ToS), states operate within their own legal frameworks, and then there is the Internet, which is designed under its own set of principles. Too many principles inevitably create too many conflicts and, therefore, consensus becomes important. 

The Santa Clara principles on transparency and accountability in content moderation have that consensus. Negotiated through a collaborative and inclusive process, they offer a roadmap to content moderation and remove certain obstacles in the process. Their strength lies in their simplicity and straightforwardness. In a similar vein, the properties of the Internet constitute a solid guide in understanding the potential unintended consequences content moderation by infrastructure providers can have. 

The design of the Internet is very specific and adheres to some normative principles that have existed ever since its inception. In fact, the Internet’s blueprint has not changed much since it was sketched a few decades ago. Ensuring that these principles become part of the consideration process is key. 

There are still plenty of unknowns in content moderation at the infrastructure layer. But, there are also quite a few things we do know: the first thing is that scale plays a significant factor. Moderating content down the stack is not just about speech; in many cases, it can be about being present on the Internet. 

The second thing we know is that the general principle of infrastructure actors being allowed to provide their services agnostically of the content they carry should continue to hold as the default. There might be cases when they may need to engage but that should be the exemption rather the rule and it should abide by the “frameworks, tools and principles” identified above. 

Finally, the third thing we know is that content moderation is evolving in ways that could now directly affect the future of the Internet. Ensuring that the role and responsibilities of infrastructure providers is appropriately scoped will be content moderation’s greatest challenge yet! 

Konstantinos Komaitis is a veteran of developing and analysing Internet policy to ensure an open and global Internet. Konstantinos has spent almost ten years in active policy development and strategy as a Senior Director at the Internet Society, and is currently a policy fellow at the Brave New Software Foundation. Before that, he spent 7 years researching and teaching at the university of Strathclyde, Glasgow, UK.

Techdirt and EFF are collaborating on this Techdirt Greenhouse discussion. On October 6th from 9am to noon PT, we'll have many of this series' authors discussing and debating their pieces in front of a live virtual audience (register to attend here).

Filed Under: complexity, content moderation, impact assessment, infrastructure

Against 'Content Moderation' And The Concentration Of Power

from the power-and-moderation dept

Content moderation frameworks and toothless oversight boards legitimize the concentration of power in the hands of infrastructure providers and platforms. This gives them, and not democratic processes and structures, the discretion to egregiously shape the public debate.

In 1964 Marshall McLuhan wrote that content is a “juicy piece of meat carried by the burglar to distract the watchdog of the mind” (McLuhan 2013). I will argue that today this is more true than ever. If we want to solve the issue of human rights violating content, we will need to look at the structures that allow for the production of it. Therefore, I will argue that “content” is a false category, and that infrastructure is often misunderstood as a largely material object whereas it is a complex assemblage of people, practices, institutions, cultures, and devices. To address the false premises on which the concept of “infrastructural content moderation” is based, I propose an analytical framework that does not separate the context from the content but rather offers an integrative approach to address online discourse production. 

Aristotle famously wrote that there is no matter without form and no form without matter. Similarly, Bergson said that color does not exist as an abstract category, but only as a quality of a substance. The same holds true for content on the Internet. A Facebook post is something different than a post on Tiktok, a blog post, a tweet, or a YouTube comment. One understands these messages differently. Just like one understands a sentence spoken in a comic club differently than one spoken in parliament, and a sentence uttered in a forest is different from one in a theater.

It has taken centuries for legal and social rules for public and private spaces to develop. The Internet is a relatively new space that practically is largely private, but feels like the world's largest public space. It will take time for rules to sediment for this space. In the development of new rules, one should commence the interrogation of different possibilities with a simple question: Cui bono? Who profits? 

Julie Cohen describes in her book ‘Between Truth and Power’ that the shift in the image from the Internet as an “electronic superhighway” to a “cloud” should by no means be taken lightly. At least a highway has rules, a cloud has none. In the image that the Internet infrastructure industry has shown us, the Internet infrastructure is a given. A modular space on which things can be built, a neutral platform for economic growth and development, that would only suffer from regulation. 

But Keller Easterling explains that “infrastructure sets the invisible rules that govern the spaces of our everyday lives” and that “changes to the globalising world are being written, not in the language of law and diplomacy, but rather in the language of infrastructure.” She describes the practice of the development, implementation, and operation of these infrastructures as “extrastatecraft,” because these powers used to belong to nation states, but are now taken up by transnational corporations. 

The development, standardization, and implementation of Internet infrastructure is inherently political. Janet Abbate particularly says that: ‘the debate over network protocols illustrates how standards can be politics by other means. Denardis’ 2014 book ‘Protocol politics’ furthers the work by Abbate and showcases how “debates over protocols bring[ing] to light unspoken conflicts of interest.” Whereas the work of DeNardis focuses mostly on Internet protocols, she does emphasize that “politics are not external to technical architecture.”

When one looks at the infrastructure that undergirds the exchange of discourse, we should not see it as a neutral foundation for platforms and services, but rather as a shaping force that has both direct and indirect power. This shaping power is what sets the rules for everything that happens on top of it, which is more influential than the haphazard removal of a particular user or group. This shaping power is deeply entrenched in the standardization and governance bodies where the Internet infrastructure is produced. 

Upon interrogation of these standards and governance bodies, one cannot help but notice, as the research by Corinne Cath-Speth shows, that the bodies can be characterized by a laissez-faire approach to technology development and defy any strong accountability measures. This culture is characterized by a libertarian, American, masculine approach that values individualism. It is exactly these qualities that perpetuate the idea that regulation will “break the Internet” and that individual choice and responsibility is the only way forward for the Internet infrastructure. 

This attitude is deeply ironic because for the first half of its existence the Internet was heavily funded by states, and the second half has been characterized by oligopolies. However, this sense of individual engineering pride keeps the status quo intact, which means a continuous exclusion of those who do not want to succumb to this culture, mostly women, people of color, and those from outside of Europe and the United States. This in turn strengthens a network topology that reinforces power structures of dominance and extraction based in the United States and Europe. Submarine cables now cover the whole world, but network traffic still largely centers in Europe and the United States, maps that very much resemble those of colonial trade routes. 

The Internet infrastructure and its standardization and governance regime exist to increase interconnection between transnational corporations, largely based in the United States and Europe. Expanding the data flows to and through these networks is what these networks and their governance is optimized for. This has transformed the Internet from a medium of connection to a medium of extraction. Solely focusing on the outgrowths of this culture and regime by focusing on content moderation would be naive at best, and legitimizing an extractive practice at worst. 

Reflections on the practice of content moderation should not solely focus on the content that should, or should not be, moderated, but rather on the structures that incentivize and perpetuate such speech. It is the responsibility of communication infrastructure providers to meaningfully engage with the human rights impact of their actions, and their chain responsibility. Thus far, hardly any Internet infrastructure provider has done so sufficiently. The industry’s lack of meaningful adoption and integration of the United Nations Guiding Principles for Business and Human Rights reminisce of the tobacco industry’s opposition against health codes, and their lobbying budgets reflect the same fear for regulation. 

Civil society should not be afraid to present strong alternative network ideologies that rely on free association and self-determination by end users. The priority of the networking and content provision industry should be to address problems of inequity and inequality, not to extract more private data to be sold to advertising and surveillance companies (which are anyhow based on flawed premises). The Internet is the public square of the world, we should better reimagine it as one. This means that the strongest actors should live up to their responsibilities, and not seek to wait for civil society to organize themselves and demand accountability, and fix their problems. Here we can only refer back to Spiderman: with great power, comes great responsibility. It is high time that the Internet infrastructure sector lives up to that.

Niels ten Oever is a postdoctoral researcher with the ‘Making the hidden visible project at the Media Studies department at the University of Amsterdam.

Techdirt and EFF are collaborating on this Techdirt Greenhouse discussion. On October 6th from 9am to noon PT, we'll have many of this series' authors discussing and debating their pieces in front of a live virtual audience (register to attend here).

Filed Under: content moderation, power

Social Media Regulation In African Countries Will Require More Than International Human Rights Law

from the isps-and-human-rights dept

There has been a lot of focus on moderation as carried out by platforms—the rules social media companies base their decision on what content remains online. There has however been limited attention on how actors other than social media platforms, in this case governments, seek to regulate these platforms. 

Focusing more on African governments, they carry out this regulation primarily through laws. These laws can be broadly divided into two: direct and indirect regulatory laws. The direct regulatory laws can be seen in countries like Ethiopia and more recently in Nigeria. They are similar to Germany’s Network Enforcement Act and France’s Online Hate Speech Law that directly place responsibilities on platforms and require them to remove online hate speech within a specific time and failure of which attracts heavy sanctions. 

Section 8 of Ethiopia’s Hate Speech and Disinformation Prevention and Suppression Proclamation 1185/2020 provides for various responsibilities for social media platforms and actors. These responsibilities include the suppression and prevention of disinformation and hate speech content by social media platforms and a twenty-four window within which such content must be removed from their platforms. It also provides that they should bring their policies in line with the first two responsibilities. 

The Proclamation further vests the reporting and public awareness responsibilities on the compliance of social media platforms in the Ethiopian Broadcasting Authority—a body empowered by law to regulate broadcasting services. The Ethiopian Human Rights Commission (EHRC), Ethiopia’s National Human Rights Institution (NHRI), also has responsibilities on public awareness. But it is the Council of Ministers that’s responsible for implementing laws in Ethiopia that may give further guidance on the responsibilities of social media platforms and other private actors.

In Nigeria, the legislative proposal, the Protection from Internet Falsehoods, Manipulation and Other Related Matters bill, is yet to become law. The bill seeks to regulate disinformation and coordinated inauthentic behaviour online. The law is similar to that of Singapore which has been criticised by the current United Nations Special Rapporteur on the promotion and protection of the right to freedom of opinion and expression for the threats it poses to online expression and online rights in general. 

Major criticisms against these laws include how they are opaque and pose threats to online expression. For example, the Ethiopian law defines hate speech broadly and does not include the contextual factors that must be considered in categorising online speech as hateful. With respect to the Nigerian bill, there are no clear oversight, accountability or transparency systems in place to check the government's unlimited powers to decide what constitutes disinformation. 

The indirect regulatory laws are those used by governments through their telecommunications regulatory agencies to compel Internet Service Providers (ISPs) to block social media platforms. This type of regulation requires ISPs to block social media platforms based on public emergencies or national interests. What constitutes these emergencies or interests are vague and in many instances are examples of voices or platforms critical of government policies. 

In January 2021, the Ugandan government ordered ISPs to block Facebook, Twitter, WhatsApp, Signal and Fiber. The order was issued through the communications regulator. The order came a day after Facebook’s announcement that it will close pro-government accounts sharing disinformation. 

In June 2021, the Nigerian government ordered ISPs to block access to Twitter stating that the latter’s activities constituted threats to Nigeria’s corporate existence. However, there have been contrary views that the order was as a result of both remote and immediate causes. The remote cause was the role Twitter played in connecting and rallying publics during the #EndSARS protests against police brutality while the immediate cause was attributed to Twitter’s deletion of President Muhammadu Buhari’s tweet which referred to the country’s civil war, contained veiled threats of violence, and violated Twitter’s abusive policies. 

In May 2021, Ethiopia had just lifted the block on social media platforms in six locations in the country. Routine shutdowns like these have become a thing for African governments and this often occurs during elections or a major political development. 

On a closer look, the cross-cutting challenge posed by both forms of regulation is the lack of accountability and transparency especially on the part of governments on how they enforce these provisions. Social media platforms are also complicit as there is little or no information on the nature of pressure they face from these government actors. 

Alongside the mainstream debates on how to govern social media platforms, it is time to also consider wider forms of regulation especially on how they manifest outside Western systems and the threats such regulation poses to online expression. 

One solution that has been suggested but also severely criticised is the application of international human rights standards to social media regulation. This standard has been argued to be the most preferred because of its customary application across contexts. However, its biggest strength also seems to be its biggest weakness—how does this standard apply in local contexts given the complexity of governing online speech and the myriad of actors involved?

In order to work towards effective solutions, we will need to re-imagine and re-purpose traditional governance roles of not only governments and social media platforms, but also ISPs, civil society, and NHRIs. For example, the unchecked powers of most governments to determine what constitutes online harms must be revisited to ensure that there are judicial reviews and human rights impact assessments (HRIAs) of proposed government social media bans. 

ISPs must also be encouraged to jump into the fray, choose human rights, and not to roll over each time governments make such problematic demands to block social media platforms. For example, they should begin to join other actors like the civil society and the academia to lobby for laws and policies that make judicial reviews and HRIAs requirements before entertaining governments request for blocking of platforms or even content. 

The application of international human rights standards to social media regulation is not where the work stops, but is where it begins. For a start, proximate actors involved in social media regulation like governments, social media platforms, private actors, local and international civil society bodies, and treaty-making bodies like the United Nations and the African Union, NHRIs must come up with a typology of harms as well as actors actively involved in such regulation. In order to ensure that these addresses the challenges posed by these kinds of regulation, the responsibilities of such actors must be anchored on international human rights standards but in such a way that these actors actively communicate and collaborate.

Tomiwa Ilori is currently a Doctoral Researcher at the Centre for Human Rights, Faculty of Law, University of Pretoria. He also works as a Researcher for the Expression, Information and Digital Rights Unit of the Centre.

Techdirt and EFF are collaborating on this Techdirt Greenhouse discussion. On October 6th from 9am to noon PT, we'll have many of this series' authors discussing and debating their pieces in front of a live virtual audience (register to attend here).

Filed Under: africa, content moderation, ethiopia, human rights, isps, nigeria, uganda