Parler Was Allowed Back In The Apple App Store Because It Will Block 'Hate Speech,' But Only When Viewed Through Apple Devices

from the fracturing dept

Last month we noted that Apple told Congress that it was allowing Parler's iOS app to return to its app store, after the company (apparently) implemented a content moderation system. This was despite Parler's then interim CEO (who has since been replaced by another CEO) insisting that Parler would not remove "content that attacks someone based on race, sex, sexual orientation or religion." According to a deep dive by the Washington Post, the compromise solution is that such content will be default blocked only on iOS devices, but will be available via the web or the sideloaded Google app, though they will be "labeled" as hate by Parler's new content moderation partner, Hive.

Posts that are labeled “hate” by Parler’s new artificial intelligence moderation system won’t be visible on iPhones or iPads. There’s a different standard for people who look at Parler on other smartphones or on the Web: They will be able to see posts marked as “hate,” which includes racial slurs, by clicking through to see them.

Hive is well known in the content moderation space, as it is used by Chatroulette, Reddit and some others. Hive mixes "AI" with a large team of what it refers to as "registered contributors" (think Mechanical Turk-style crowdsourced gig work). Of course, it was only just last year that the company announced that its "hate model" AI was ready for prime time, and I do wonder how effective it is.

Either way, this is interesting for a variety of reasons. One thing we've talked about in the past with regards to content moderation is that one of the many problems is that different people have different tolerances for different kinds of speech, and having different moderation setups for different users (and really pushing more of the decision making to the end users themselves) seems like an idea that should get a lot more attention. Here, though, we have a third party -- Apple -- stepping in and doing that deciding for the users. It is Apple's platform, so of course, they do get to make that decision, but it's a trend worth watching.

I do wonder if we'll start to see more pressure from such third parties to moderate in different ways to the point that our mobile app experiences and our browser experiences may be entirely different. I see how we end up in such a world, but it seems like a better solution might be just pushing more of that control to the end users themselves to make those decisions.

The specific setup here for Parler is still interesting:

Parler sets the guidelines on what Hive looks for. For example, all content that the algorithms flag as “incitement,” or illegal content threatening physical violence, is removed for all users, Peikoff and Guo said. That includes threats of violence against immigrants wanting to cross the border or politicians.

But Parler had to compromise on hate speech, Peikoff said. Those using iPhones won’t see anything deemed to be in that category. The default setting on Android devices and the website shows labels warning “trolling content detected,” with the option to “show content anyway.” Users have the option to change the setting and, like iOS users, never be exposed to posts flagged as hate.

Peikoff said the “hate” flag from the AI review will cue two different experiences for users, depending on the platform they use. Parler’s tech team is continuing to run tests on the dual paths to make sure each runs consistently as intended.

Of course, AI moderation is famously mistake-prone. And both Parler and Hive execs recognize this:

Peikoff said Hive recently flagged for nudity her favorite art piece, the “To Mennesker” naked figures sculpture by Danish artist Stephan Sinding, when she posted it. The image was immediately covered with a splash screen indicating it was unsafe.

“Even the best AI moderation has some error rate,” Guo said. He said the company’s models show that one to two posts out of every 10,000 viewed by the AI should have been caught on Parler but aren’t.

I do question those percentages, but either way it's another interesting example of how content moderation continues to evolve -- even if Parler's users are angry that they won't be able to spew bigotry quite as easily as previously.

Filed Under: content moderation, hate speech, ios, platforms
Companies: apple, hive, parler

Twitch Yanks Advertising Revenue From Popular 'Hot Tub Streamer' With No Warning Or Dialogue

from the fast-twitch dept

We've covered a variety of issues Twitch is facing as a platform over the past several months, but there has also been a theme to all of these issues. Whether it's been Twitch's decision to simply nuke a bunch of creator content due to DMCA claims it received for them, its tone-deaf attempt to redirect the focus onto a dumb emoji, changes to its affiliate program, or how it chooses to roll out, or not, tools for creators to respond to the DMCApocalypse it kicked off, those stories all have one thing in common: they demonstrate that Twitch does a brutally terrible job of communicating to its most valuable asset, its own creative community.

When changes happen at Twitch, they often come as a shock to Twitch streamers. If you think that streaming on a platform that is constantly pulling the rug out from under you sounds like a terrible idea, well, I agree with you. And it just keeps happening. The latest is one streamer, who at least partially made a name for herself doing so-called "hot tub meta" streams, suddenly having her ability to make advertising revenue stripped from her without warning or dialogue.

Kaitlyn “Amouranth” Siragusa is one of the most popular female streamers on Twitch. As of now, she can no longer make money off ads on her channel. Siragusa, already one of the more recognizable names on Twitch, has benefited tremendously from the so-called “hot tub meta,” in which swimsuit-clad streamers talk to their chats, play games, and perform other activities from hot tubs. This has proven controversial despite Twitch noting to Kotaku and others that it’s not against the platform’s rules. Now Twitch has taken a different, arguably more damaging sort of action against the biggest streamer to participate in the meta.

“Yesterday I was informed that Twitch has indefinitely suspended advertising on my channel,” Siragusa wrote on Twitter today. “Twitch didn’t reach out in any way whatsoever. I had to initiate the conversation after noticing, without any prior warning, all the ads revenue had disappeared from my channel analytics.”

Let's get this out of the way: you may not like the idea of streams that are essentially women in bathing suits in inflatable pools chatting with viewers and doing random things... and that doesn't matter for this story in the slightest. Instead, what matters is that once again we have Twitch making important changes that effect streamers on its platform with both no warning at all and with no clear standard for the rules streamers are supposed to follow. In fact, as Kotaku notes, Twitch has previously indicated that these "hot tub meta" streams don't violate its rules. So, then, why strip the advertising revenue?

Frankly, nobody seems to know.

“Many people complain about [Twitch’s terms of service] being ‘unclear,’ but at least there’s something to go by,” she wrote. “There is no known policy for what results in a streamer being put on this blacklist. With characteristic opacity, the only thing Twitch made clear is that it is unclear whether or when my account can be reinstated.”

“The issue isn’t Twitch removing ads. It’s them doing so without any clarification of what their guidelines are,” she said. “We saw this coming. Everyone expected it. No one expected it without communication, though. Just, like, a stealth removal.”

She also attempted to manually run an ad from her Twitch dashboard as an experiment, but when she clicked the button, nothing happened.

She's exactly right, frankly. Say what you want about the sort of streaming she does on Twitch, but ripping away very real money and sources of income without warning is not a creator-friendly action for a platform to take. And, as I noted, this fits with the ongoing theme of Twitch not giving a hot damn about its creative community. Were it otherwise, the platform might actually attempt to communicate to and with creators, working with them to avoid this kind of policy-by-surprise system it appears to have set up for itself.

Twitch has built up quite a community for itself over the years, but lately its actions have called into questions whether it even deserves that community.

Filed Under: content moderation, kaitlyn siragusa, monetization, rule changes, streaming
Companies: twitch

New Jersey State Legislators Think They Can Get Trump Back On Facebook By Passing A Stupid Social Media Moderation Bill

from the keep-in-mind-the-only-legacy-you-can-really-shit-on-is-your-own dept

Facebook recently announced its decision to keep Donald Trump banned -- a decision supported by its new oversight board. That decision -- just like the original ban -- made certain people very angry and very sure a platform's decision to ban someone infringes on the First Amendment.

One of those deciding they should be the ones violating the First Amendment is New Jersey state senator Michael Testa. Announcing his ignorance on multiple platforms, Testa and a couple of like-minded state legislators declared they were finally going to do something about all of this not-actually-censorship.

Following the announcement that Facebook will uphold their ban of former President Donald Trump, Senator Michael Testa, Assemblymen Erik Simonsen and Antwan McClellan (All R-1) are introducing legislation that places penalties against social media platforms for the censorship and banning of conservative political and journalist voices.

“The muzzling of freedom of speech by social media sites, such as Facebook and Twitter, have shown that prominent conservative voices are not welcomed on the world’s leading social media platforms,” Senator Testa stated. “Today’s announcement of Facebook’s six month extension of their ban on former President Donald Trump exposes that social media has and always will control the narrative that they see fit – unless leaders take action.

It goes on for a bit more in that same vein, calling for New Jersey centric efforts to prevent the "silencing" of former presidents named Donald Trump and anyone else "King Zuckerberg does not agree with." You know, the usual idiocy that comes from people who are at their most ineffective when they're angry about stuff they don't actually understand.

There's also whatever this is:

“This is not a Republican issue nor a Democrat issue, this is an issue of protecting everyone’s First Amendment right of freedom of speech,” Assemblyman McClellan said. “No individual or corporation should have complete power to silence those who share opposing viewpoints. Our nation was built on the foundation that all Americans, regardless of race, religion, or political opinion should have the ability to freely voice their ideals and beliefs.”

Yeah, but it's mainly a Republican issue, if we're being more honest than these three New Jersey lawmakers. And making social media companies do moderation stuff with government mandates backed by the threat of fines, fees, and other punishments is the exact opposite of "protecting everyone's First Amendment rights."

The bill [PDF] has arrived and it's as ridiculous as you'd think. The new law would make social media companies engage in a whole lot more moderation related BS under the apparent theory that burying them in compliance paperwork will make them less likely to boot the Trumps of the internet off their services.

It shall be an unlawful practice and violation of the New 32 Jersey consumer fraud act, P.L.1960, c.39 (C.56:8-1 et seq.), if a social media website fails to comply with the requirements set forth in this section.

A social media website shall:

a. publish the standards, including detailed definitions, it uses or has used for determining how to censor, selectively suspend, or implement anonymous content or user ban;

b. apply censorship, selectively suspend, and anonymous content or user banning standards in a consistent manner among its users on the platform;

c. notify users about any changes to user rules, terms, and agreements prior to implementing such changes;

d. not censor a user's content or material or selectively suspend a user from the social media website without providing notification to the user who posted or attempted to post the content or material, or in a way that violates State deceptive and unfair trade practices;

e. provide a mechanism for users to request the number of other users who were actually provided or shown the requesting user’s content or posts and provide users with the number of other users who were actually provided or shown that content or posts upon request;

f. categorize algorithms used for post prioritization and anonymous content or user banning and allow users to opt-out of post prioritization and anonymous content or user banning algorithm categories to allow sequential or chronological posts and content;

g. provide users with an annual notice on the use of algorithms used for post prioritization and anonymous content or user banning and reoffer annually the opt-out opportunity provided in subsection f. of this section…

In digital triplicate, one imagines. This is a long list of nuisances that will only help ensure the dominant players in the social media marketplace remain dominant. It doesn't go so far as to compel speech by forcing platforms to retain users they'd rather eject, but it does make it a challenge to comply with the law. And that's the intent, to be sure. You can tell that by all the times the legislators use the word "censor" rather than the word "moderate."

And it doesn't do anything to address the issue it claims to be addressing: namely, the banning of Trump by Facebook. If anything, Facebook's first ban and the oversight board's decision to extend the ban comply with most of these directives in terms of notifications and explanations for the moderation action.

Did I say there was no compelled speech -- one of the big no-nos of First Amendment compliance? That's only because we haven't gotten to that yet. There will be plenty of compelled speech, but only for those most deserving of uninterrupted service from social media platforms: politicians.

h. not apply or use post prioritization or anonymous content or user banning algorithms for content and material posted by or about a user who is known by the social media website to be a candidate for office from the date of qualification through the date of the election or the date the candidate ceases to be a candidate before the date of election...

There it is. It's another quasi-Fairness Doctrine -- one based on the faulty assumption that social media services go out of their way to bury posts from politicians "they" don't agree with. First off, most platforms allow users to sort their posts chronologically and/or make their own moderation decisions about the content they want to see. (See stipulation f. above.) But this attempt to address something that really isn't a problem for anyone who isn't suffering from a persecution complex allows the state to collect up to $100,000 per instance of "unfair" post sorting until platforms satisfactorily explain their algorithms and/or user preference settings to lawmakers who seem entirely uninterested in listening to them.

Finally, there's this word salad that, if you squint at it conspiratorially enough, might be a response to Twitter and Facebook banning links to the New York Post's bizarre "smoking gun" article about Hunter Biden's laptops.

not knowingly take any action to censor, selectively suspend, or anonymous content or user ban a journalistic enterprise based on the content of its publication or broadcast.

If you're curious as to wtf "anonymous content or user ban" means in the numerous places it's used in this bill, it refers to any act of banning that is "not readily apparent to a user." So, shadow-banning and things like that, although it's not clear why these reps felt compelled to make up a new term for it.

So, it's a bunch of different forms of compelled speech masquerading as a concerned-about-consumers bill. It invokes the state's consumer fraud act meaninglessly, equating actual fraud with moderation efforts by private companies -- companies that are free to choose who they do business with without committing a single act of fraud.

And if this does become a law, how would these legislators like social media companies to respond? By gathering even more identifying info to make sure New Jersey residents get all the additional notifications and faux protections the law puts in place? Or maybe suspected Jersey locals will just get a splash page notifying them they're no longer able to use these services because platforms are unable to comply with the ridiculous demands of grandstanding Trump loyalists. Whatever the outcome, it won't do what these legislators want it to do. It will only make these services less likely to welcome new users and more likely to ban content and the third parties creating it.

Filed Under: 1st amendment, compelled speech, content moderation, donald trump, michael testa, new jersey, section 230, social media
Companies: facebook

The Flopping Of Trump's Blog Proves That It's Not Free Speech He's Upset About; But Free Reach

from the it's-the-audience dept

A week ago, we wrote about Trump's new blog, which was designed to look vaguely tweet-like, noting that this proved that he never needed Twitter or Facebook to speak freely. He's always been able to speak on his own website. NBC News has an interesting story now, suggesting that the blog just isn't getting that much attention.

A week since the unveiling, social media data suggests things are not going well.

The ex-president’s blog has drawn a considerably smaller audience than his once-powerful social media accounts, according to engagement data compiled with BuzzSumo, a social media analytics company. The data offers a hint that while Trump remains a political force, his online footprint is still dependent on returning to Facebook, Twitter and YouTube.

The Desk of Donald J. Trump is limited — users can’t comment or engage with the actual posts beyond sharing them to other platforms, an action few people do, according to the data.

Some have been using this to argue that Twitter and Facebook's bans on the former president were attacks on his "free speech." But it actually demonstrates something different -- and important. Everyone complaining about the removal of Trump's account are not actually mad about the "free speech" part of it. They're really mad about the "free reach." (Hat tip to Renee DiResta for making this point years ago).

Being kicked off these platforms by the platforms (as opposed to, say, the government) is not an attack on your ability to speak. There are lots of places to do that. It is, instead, an attack on having easy access to an audience on those platforms. And, as far as I can tell, there is no right to having as large an audience as possible. Thus, in the same sense that I can't demand a million followers on any of these platforms, the former president similarly can't demand that they supply him with the audience of their users.

Filed Under: access, audience, content moderation, donald trump, free reach, free speech
Companies: facebook, twitter

Steam Still Can't Seem To Keep Its Hands Off Some 'Sex Games' Despite Hands Off Policy

from the pron-store dept

It's been three or so years since Valve announced a new "hands off" approach towards approvals for games on its dominant Steam storefront. This new "policy" was unfortunately rolled out in an extremely Steam-like manner: vague and largely indecipherable, full of holes, and all with a caveat baked in that Steam could still do basically whatever it wants. Later, the company clarified that the chief goal with all of this was to allow for more adult-oriented games while still giving Steam the ability to disallow "troll games", as though that actually clarified anything. Predictably, this new policy set off confusion all over the place, and even years into the change its application appears to be aggressively inconsistent.

Three years in and it's still a problem. The developers of Holodexxx, a VR sex game featuring VR-rendered real-life porn stars, has expended thousands of dollars to try to comply with Steam's policies only to find the game banned from the platform. Interestingly, the developers of the game appear to have intended this to be less of a gross or trollish look at a sex video game and more as something that is both adult-oriented but a "sex positive" experience.

Holodexxx is a game in which simulated versions of real adult performers interact with the player in virtual reality, with AI guiding elements of the performance. Its creators bill it as an ethical, sex-positive game being made in conjunction with and featuring real sex workers. Steam, at this point, carries a plethora of games that include adult content—some of which venture into much dicier thematic territory than Holodexxx. But that didn’t stop Valve from chasing Holodexxx off its holodeck.

On pre-2018 Steam, this is a game that would absolutely not be allowed on the platform. But the whole point of the change in policy was supposed to be to allow for more content of this nature, so long as it didn't dive into "troll" territory. And, to some extent, that policy shift achieved its goal. There are a great many adult sex games on Steam these days, with the level of raunchiness in those games playing across a spectrum for that sort of thing. But in this case, the game has been rejected several times, no matter what the developer tries to do to comply with Steam's policy.

In a recent lengthy blog, the game’s developers outlined everything they’ve tried over the course of multiple months. To begin, they submitted a “PG-13 experience” to Steam starring a clothed version of adult film actress Riley Reid, along with a censored video of live adult stars. Valve, say Holodexxx’s developers, blocked the submission “with a boiler-plate explanation that video pornography was not allowed on Steam.” So then the developers spent additional time creating a new demo without video of adult stars, in which the player could instead look at a model of adult film actress Marley Brinx in a virtual environment. Again, Valve blocked it on the basis that it was “pornography.”

At that point, Holodexxx’s developers went back to the drawing board and spent “months” on Holodexxx Home, a more elaborate interactive experience with dialogue systems and direct physical interactions. It involves undressing a character, but so do other games on Steam, so in theory, it doesn’t necessarily activate any of Valve’s tripwires. However, you can probably guess what happened next.

“We submitted Home and waited a few weeks,” wrote Holodexxx’s developers. “After poking Steam via help tickets, our build was reviewed two days later... and banned. The explanation was again, that Steam does not allow ‘pornography’ on their platform.”

In all, the developer claims that it has spent something like $20,000 just trying to get a version in place that would meet Steam's guidelines. Unfortunately, that effort appears to have failed as that last rejection once again simply regurgitated the anti-pornography stance previously iterated.

Now, let's be clear on a couple of things. First, you may not like the idea of adult games showing up on Steam. That doesn't really matter for this story. Steam shifted policies to allow for more of them and they can run their platform as they choose. Secondly, I'll reiterate that point: Steam can block this game from its platform if it wants.

But the problem here is the lack of clarity in the policy and the inconsistency with which it is applied. If you need this broken down further, I'll put it this way: when a developer can spend five-figures trying to comply with a platform's policy and can't manage to do so, the problem is on the platform's end. Either the policy isn't clear enough for developers (check!), hasn't been applied in a way that's consistent enough to allow it to be interpreted (check!), or the communication in feedback to developers about the policy as it applies to their specific games hasn't been handled well (check!).

In this case, Steam has failed at all three levels. Again, the platform can do as it pleases with its property, but if the platform is going to have a policy, it might as well make it one that actually works.

Filed Under: adult content, content moderation, holodexx, steam
Companies: valve

Why Is Wired So Focused On Misrepresenting Section 230?

from the it's-bizarre dept

We've already highlighted our concerns with Wired's big cover story on Section 230 (twice!). The very same day that came out, Wired UK published a piece by Prof. Danielle Citron entitled Fix Section 230 and hold tech companies to account. Citron's proposal was already highlighted in the cover story and now gets this separate venue. For what it's worth, Citron also spent a lot of energy insisting that the Wired cover story was the "definitive" article on 230 despite all of its flaws, and cheered on and liked tweets by people who mocked my arguments for why the article is just not very accurate.

Over the last few years, we've also responded multiple times to Citron's ideas, which are premised on a completely false narrative: that without a legal cudgel, websites have no incentive to keep their sites clean. That's clearly not true at all. If a company doesn't moderate, then it turns into a garbage dump of spam, harassment, and abuse. They lose users. They lose advertisers. It's just not good business. There are plenty of incentives to deal with bad stuff online -- though Citron never seems to recognize any of that, and insists, instead, that every site is a free-for-all because 230 does not provide them legal liability. This latest piece in Wired is more of the same.

American lawmakers sided with the new inventors, young men (yup, all men) who made assurances that they could be trusted with our safety and privacy. In 1996, US Congress passed Section 230 of the Communications Decency Act, which secured a legal shield for online service providers that under- or over-filtered third-party content (so long as aggressive filtering was done in good faith). It meant that tech companies were immune to lawsuits when they removed, or didn’t remove, something a third party posted on their platforms.

That's only partially accurate. The "good faith" part only applies to one subsection of the bill, and is not the key reason why sites can be "aggressive" in filtering. That's the 1st Amendment. The immunity part is a procedural benefit that prevents abusive litigation by those seeking to waste the court's time by filing frivolous and expensive lawsuits -- or, more likely, threaten to do so, if websites won't remove perfectly legal content they just don't like.

But, thanks to overbroad court rulings, Section 230 ended up creating a law-free zone. The US has the ignominious distinction of being a safe haven for firms hosting illegality.

This is just wrong. And Citron knows it's wrong, and it's getting to be embarrassing that she (and Wired) would repeat it. First, Section 230 has no impact on federal criminal law, so anything that violates federal criminal law is not safe. Second, there are almost no websites that want "illegal" content on their site. Most have teams of people who deal with such reports or court orders. Indeed, the willingness of websites to quickly remove any content deemed illegal has been abused by reputation management firms to get content removed via faked judicial orders, or through a convoluted scheme involving fake defendants who "settle" lawsuits just to get a court order out of it.

This isn’t just an American pathology: Because the dominant social media companies are global, illegality they host impacts people worldwide. Indeed, safety ministers in South Korea and Australia tell me that they can help their citizens only so much, since abuse is often hosted on American platforms. Section 230 is to social media companies what the Cayman Islands has long been to the banking industry.

Over and over again we've seen the exact opposite of this, in two separate, but important ways. First, many of these companies still are more than willing to geoblock content if it's found to violate the law in a certain country. However, much more importantly, the ability of US-based websites to keep content up means that threatened, marginalized, and oppressed people are actually able to get their messages out. Oppressive governments around the world, including in places like Turkey and India have sought to force websites to take down content that merely criticizes their governments.

Any reasonable discussion on this needs to take that into account in demanding that "illegal" content must automatically be taken down. And when weighed with the fact that most companies don't want to host truly illegal and problematic content, most of the content that is likely to be removed without those protections is exactly the kind of authoritarians-suppressing-speech content that we're concerned about.

Tech companies amplify damaging lies, violent conspiracies and privacy invasions because they generate copious ad revenue from the likes, clicks and shares. For them, the only risk is bad PR, which can be swiftly dispatched with removals, bans and apologies.

This is stated without anything backing it up and it's garbage. It's just not true. All of the big companies have policies in place against this content, and they (unlike Citron) recognize that it doesn't "generate copious ad revenue from the likes, clicks and shares" (likes, clicks and shares don't directly generate ad revenue...). These companies know that the long-term health of their platforms is actually important, and losing advertisers and users because of garbage is a problem. This is why Facebook, Twitter, and YouTube all have teams working on these issues and trying to keep the platforms in better shape. They're certainly not perfect at it, but part of that is because of the insane scale of these platforms and the ever changing nature of the problematic content on those platforms.

I know that among a certain set it's taken on complete faith that no one at these companies cares, because they just "want clicks" and "clicks mean money." But that shows an astounding disconnect from what the people at these companies, and those setting and enforcing these policies actually think. It's just ivory tower nonsense, completely disconnected from reality.

For individuals and society, the costs are steep. Lies about mask wearing during the Covid-19 pandemic led to a public health disaster and death.

Which spread via cable news more than on social media, and included statements from the President of the United States of America. That's not a Section 230 problem. It's also not something that changing Section 230 fixes. Most of those lies are still Constitutionally protected. Citron's problem seems to be with the 1st Amendment, not Section 230. And changing Section 230 doesn't change the 1st Amendment.

Plans hatched on social media led to an assault on the US Capitol. Online abuse, which disproportionately targets women and minorities, silences victims and upends careers and lives.

These are both true, but it's an incredible stretch to say that Section 230 was the blame for either of these things. The largest platforms -- again, Facebook, YouTube, Twitter, etc. -- all have policies against this stuff. Did they do a bad job enforcing them? Perhaps! And we can talk about why that was, but I can assure you it's not because "230 lets us ignore this stuff." It's because it's not possible to magically make the internet perfect.

Social media companies generally have speech policies, but content moderation is often a shell game. Companies don’t explain in detail what their content policies mean, and accountability for their decisions isn’t really a thing. Safety and privacy aren’t profitable: taking down content and removing individuals deprives them of monetizable eyes and ears (and their data). Yes, that federal law gave us social media, but it came with a heavy price.

This is the only point in which Citron even comes to close to acknowledging that the companies actually do make an effort to deal with this stuff, but then immediately undermines it by pretending they really don't care about it. Which is just wrong. At best it could be argued that the platforms didn't care enough about it in 2010. But that was a century ago in internet years, and it's just wrong now. And, "taking down content and removing individuals deprives them of monetizable eyes and ears (and their data)" only if those particular eyes and ears aren't scaring off many more from their platform. And every platform now recognizes that the trolls and problem makers do exactly that. Citron, incorrectly again, completely misses that these companies now recognize that not all users are equal, and trolls and bad actors do more damage to the platform than they're worth in "data" and "ad revenue."

It feels like Citron's analysis is stuck in the 2010 internet. Things have changed. And part of the reason they've changed is that Section 230 has allowed companies to freely experiment with a variety of remedies and solutions to best deal with these problems.

Are there some websites that focus on and cater to the worst of the worst? There sure are. And if she wanted to focus in on just those, that would be an interesting discussion. Instead, she points to the big guys, who are not acting the way she insists they do, to demand they do... what they already do, and insists we need to change the law to make that happen, while ignoring all of the actual consequences of such a legal change.

The time for having stars in our eyes about online connectivity is long over. Tech companies no longer need a subsidy to ensure future technological progress.

It's not a subsidy to properly apply legal liability to the actual problematic parties. It's a way of saving the judicial system from a ton of frivolous lawsuits and avoiding the ability to censor by proxy by giving aggrieved individuals the ability to silence critics by mere threats of litigation to third party platforms.

If anything, that subsidy has impaired technological developments that are good for companies and society.

Uh, no. 230's flexibility has allowed a wide range of different platforms to try a variety of different approaches, and to seek out the best approaches for that kind of community. Wikipedia's approach is different from Facebooks which is different from Reddit's which is different from Ravelry's which is different from Github's. That's because we have 230 that allows for these different approaches. And all of those companies are trying to come up with solutions that are "good for society" because if they don't, their sites turn into garbage dumps and people will seek out alternatives.

We should keep Section 230 – it provides an incentive for companies to engage in monitoring – but condition it on reasonable content moderation practices that address illegality causing harm. Companies would design their services and practices knowing that they might have to defend against lawsuits unless they could show that they earned the federal legal shield.

The issue with this is that if you have to first prove "reasonableness" you end up with a bunch of problems, especially for smaller sites. First, you massively increase the costs of getting sued (and as such, you vastly increase the ability of threats to have their intended effect to take down content that is perfectly legal). Second, in order to prove "reasonableness" many, many, many lawyers are going to say "just do what the biggest companies do" because that will have been shown in court to be reasonable. So, instead of getting more "technological developments that are good for companies and society" you get homogenization. You lose out on the innovation. You lose out on the experimentation for better models, because any new model is just a model that hasn't been tested in court yet and leaves you open to liability.

For the worst of the worst actors (such as sites devoted to nonconsensual porn or illegal gun sales), escaping liability would be tough. It’s hard to show that you have engaged in reasonable content moderation practices if hosting illegality is your business model.

This is... already true? Various nonconsensual porn sites have been taken down by both civil lawsuits and criminal prosecution over the years. Companies entirely engaged in illegal practices still face federal criminal prosecution as well without 230's protections. On top of that, courts themselves have increasingly interpreted 230 to not shield those worst of the worst actors.

Over time, courts would rule on cases to show what reasonableness means, just as courts do in other areas of the law, from tort and data security to criminal procedure.

Right. And then anyone with a better idea on how to build a better community online would never dare to risk the liability that came with having to first prove it "reasonable" in court.

In the near future, we would see social media companies adopt speech policies and practices that sideline, deemphasize or remove illegality rather than optimise to spread it.

Again, no mainstream site wants "illegality" on their site. This entire article is premised on a lie, backed up with misdirection and a historical myth.

There wouldn’t be thousands of sites devoted to nonconsensual porn, deepfake sex videos and illegal gun sales. That world would be far safer and freer for women and minorities.

Except there's literally no evidence to support this argument. We know what happened in the copyright space, which doesn't have 230 like protections, and does require "reasonable" policies for dealing with infringement. Infringement didn't go away. It remained. As for "women and minorities" it's hard to see how they're better protected in such a world. The entire #MeToo movement came about because people could tell their stories on social media. Under Citron's own proposal here, websites would face massive threats of liability should a bunch of people start posting #MeToo type stories. We've already seen astounding efforts by those jackasses who were exposed during #MeToo to silence their accusers. Citron's proposal would hand them another massive weapon.

The bigger issue here is that Citron refuses to recognize how (and how frequently) those in power abuse tools of content suppression to silence voices they don't want to hear. She's not wrong that there's a problem with a few narrow areas of content. And if she just focused on how to deal with those sites, her argument would be a lot more worth engaging with. Instead, she's mixing up different ideas, supported by a fantasy version of what she seems to think Facebook does, and then insisting that if they just moderated the way she wanted them to, it would all be unicorns and rainbows. That's not how it works.

Filed Under: content moderation, content suppression, danielle citron, section 230, tradeoffs