Content Moderation Case Study: Time Warner Cable Doesn't Want Anyone To See Critical Parody (2013)

from the can-we-make-things-worse? dept

Summary: In 2013, two comedians named Jason Selvig and Davram Stiefler, who performed as “The Good Liars,” got some attention for mocking a particular popular target of mockery: poor service from your broadband provider. For Selvig and Stiefler, their target was Time Warner Cable. In late March of that year, they released a video on YouTube in which they pretended to be Time Warner Cable employees interviewing people on the street about how TWC could make its service even worse.

To support the initial viral attention that the video was receiving, the two also set up a series of parody Time Warner Cable “customer support” accounts that would respond -- just like the real TWC customer support Twitter account -- to people complaining about their service, again asking how they could make things worse.

However, just as the video was getting more momentum, the entire YouTube channel set up by Selvig and Stiefler was taken down, as were most of the fake Twitter accounts, even though they were all clearly labeled as parody accounts, and despite policies that said that parody was allowed on these services.

Time Warner Cable, in a statement to the Daily Dot, said that it had no problem with parodies of its service in general, but was opposed to parodies that used the name of its CEO:

“We’re a big company and so we’re not at all opposed to a good parody or satire,” Bobby Amirshahi, a TWC representative, told the Daily Dot. The two crossed the line, he said, by choosing “Glenn Britt,” the company’s CEO, as their username. The issue was “posting as though it was from the CEO, i.e. impersonation,” Amirshahi said. “Otherwise, no action would be taken.”

TWC also convinced GoDaddy to remove the website that Selvig and Stiefler had used as a central hub for all of its TWC mockery, twcustomerservice.com.

Decisions to be made by YouTube/Twitter/GoDaddy:

• Where do you draw the line regarding what is acceptable parody and unacceptable impersonation?
• Is the use of TWC’s CEO enough to make it no longer acceptable?
• Should there be different rules when the parody is about a large company rather than an individual?

Questions and policy implications to consider:

• Parody and satire are often important ways to speak out against the powerful. Will clamping down on parodies in this manner suppress commentary and criticism?
• Can rules against impersonation allow powerful individuals and companies to silence criticism?

Resolution: The various takedowns remained in place, and very little is left online of the Good Liars’ campaign to mock Time Warner Cable. There was just one of the Twitter accounts @TWCCareNYC that was not removed and while the account is still live, none of the tweets remain.

Time Warner Cable itself no longer exists. Charter Communications bought Time Warner Cable in 2016 and has rebranded most former Time Warner Cable services under its “Spectrum” brand name.

Originally posted to the Trust & Safety Foundation website.

Filed Under: parody, satire, the good liars
Companies: time warner cable, twitter

Content Moderation Case Studies: Twitter Clarifies Hacked Material Policy After Hunter Biden Controversy (2020)

from the clarification-needed dept

Summary: Three weeks before the presidential election, the New York Post published an article that supposedly detailed meetings Hunter Biden (son of presidential candidate Joe Biden) had with a Ukrainian energy firm several months before the then-Vice President allegedly pressured Ukraine government officials to fire a prosecutor investigating the company.

The "smoking gun" -- albeit one of very dubious provenance -- provided ammo for Biden opponents, who saw this as evidence of Biden family corruption. The news quickly spread across Twitter. But shortly after the news broke, Twitter began removing links to the article.

Backlash ensued. Conservatives claimed this was more evidence of Twitter's pro-Biden bias. Others went so far as to assert this was Twitter interfering in an election. The reality of the situation was far more mundane.

As Twitter clarified -- largely to no avail -- it was simply enforcing its rules on hacked materials. To protect victims of hacking, Twitter forbids the distribution of information derived from hacking, malicious or otherwise. This policy was first put in place in March 2019, but it took an election season event to draw national attention to it.

The policy was updated after the Hunter Biden story broke, but largely remained unchanged. The updated policy explained in greater detail why Twitter takes down links to hacked material, as well as any exceptions it had to this rule.

Despite many people seeing this policy in action for the first time, this response was nothing new. Twitter had exercised it four months earlier, deleting tweets and suspending accounts linking to information obtained from law enforcement agencies by the Anonymous hacker collective and published by transparency activists Distributed Denial of Secrets. The only major difference was this involved acknowledged hackers and had nothing to do with a very contentious presidential race.

Decisions to be made by Twitter:

• Does the across-the-board blocking of hacked material prevent access to information of public interest?
• Does relying on the input of Twitter users to locate and moderate allegedly hacked materials allow users to bury information they'd rather not seen made public?
• Is this a problem Twitter has handled inadequately in the past? If so, does enforcement of this policy effectively deter hackers from publishing private information that could be damaging to victims? 

Questions and policy implications to consider:

• Given the often-heated debates involving releases of information derived from hacking, does leaving decisions to Twitter moderators allow the platform to decide what is or isn't newsworthy?
• Is the relative "power" (for lack of a better term) of the hacking victim (government agencies vs. private individuals) factored into Twitter's moderation decisions? 
• Does any vetting of the hacked content occur before moderation decisions are made to see if released material actually contains violations of policy?

Resolution: The expanded version of Twitter's rules on hacked material remain in force. The additions to the policy in response to questions about its takedown of the Post article more clearly state what is or isn't allowed on the platform. The expanded rules presumably also make it easier for moderators to make informed decisions, rather than simply remove any information that may appear to be the result of hacking.

Originally posted to the Trust & Safety Foundation website.

Filed Under: content moderation, elections, hacked materials, hunter biden, journalism, laptop
Companies: twitter

Content Moderation Case Study: Kik Tries To Get Abuse Under Control (2017)

from the kids-will-be-kids dept

Summary: The messaging service Kik was founded in 2009 and has gone through multiple iterations over the years. However, it seemed to build a large following for mostly anonymous communication, allowing users to create many new usernames not linked to a phone number, and to establish private connections via those usernames. This privacy feature has been applauded by some as being important for journalists, activists and at-risk populations.

However, the service has also been decried by many as being used in dangerous and abusive ways. NetNanny puts it as the number one dangerous messaging apps for kids, saying that it “has had a problem with child exploitation” and highlighting the many “inappropriate chat rooms” for kids on the app. Others have said that, while the service is used by many teenangers, many feel that it is not safe for them and full of sexual content and harassment.

Indeed, in 2017, a Forbes report detailed that Kik had a huge “child exploitation problem.” It described multiple cases of child exploitation that we found on the app, and claimed that it did not appear that the company was doing much to deal with the problem, which seemed especially concerning given that over half of its users base was under 24 years of age.

Soon after that article, Kik began to announce some changes to its content moderation efforts. It teamed up with Microsoft to improve its moderation practices. It also announced a $10 million effort to improve safety on the site and named some high profile individuals to its new Safety Advisory Board.

A few months later the company announced updated community standards, with a focus on safety, and a partnership with Crisis Text Line. However, that appeared to do little to stem the concerns. A report later in 2018 said that, among law enforcement, the app that concerned them most was Kik, with nearly all saying that they had come across child exploitation cases on the app, and that the company was difficult to deal with.

In response, the company argued that while it was constantly improving its trust & safety practices, it also wanted to protect the privacy of its users.

Decisions to be made by Kik:

• How can a company that promotes the privacy-protective nature of its messaging also limit and prevent serious and dangerous abusive practices?
• How closely should Kik work with law enforcement when they find evidence of crimes on the platform?
• Are there additional tools and features that can be implemented that would discourage those looking to use the platform in abusive ways?

Questions and policy implications to consider:

• Are there ways to retain the benefits for journalists, activists, and at-risk groups that do not put others -- especially children -- at risk?
• What are the tradeoffs between enabling useful private communications and making sure such tools are not used in abusive or dangerous ways?

Resolution: Despite the claims from Kik that it was improving its efforts to crack down on abuse, reports have continued to suggest that little has changed on the platform. A detailed report from early 2020 -- years after Kik said it was investing millions in improving the platform -- suggested that it was still a haven for sketchy content, even noting that just posting a Kik address publicly (on Twitter) resulted in near immediate abuse.

Despite an announcement in late 2019 that the company was going to shut down the messaging service to focus on a new cryptocurrency plan, it reversed course soon after and sold off the messenger product to a new owner. In the year and half since the sale, Kik has not added any new content to its safety portal, and more recent articles still highlight how frequently child predators are found on the site.

Originally published on the Trust & Safety Foundation website.

Filed Under: content moderation, messaging, privacy, safety
Companies: kik

Content Moderation Case Study: Newsletter Platform Substack Lets Users Make Most Of The Moderation Calls (2020)

from the newsletter-moderation dept

Summary: Substack launched in 2018, offering writers a place to engage in independent journalism and commentary. Looking to fill a perceived void in newsletter services, Substack gave writers an easy-to-use platform they could monetize through subscriptions and pageviews.

As Substack began to attract popular writers, concerns over published content began to increase. The perception was that Substack attracted an inordinate number of creators who had either been de-platformed elsewhere or embraced views not welcome on other platforms. High-profile writers who found themselves jobless after crafting controversial content appeared to gravitate to Substack (including big names like Glenn Greenwald of The Intercept and The Atlantic's Andrew Sullivan), giving the platform the appearance of embracing views by providing a home for writers unwelcome pretty much everywhere else.

A few months before the current controversy over Substack's content reached critical mass, the platform attempted to address questions about content moderation with a blog post that said most content decisions could be made by readers, rather than Substack itself. Its blog post made it clear users were in charge at all times: readers had no obligation to subscribe to content they didn't like and writers were free to leave at any time if they disagreed with Substack's decisions.

But even then, the platform's moderation policies weren't completely hands off. As its post pointed out, the platform would take its own steps to remove spam, porn, doxxing, and harassment. Of course, the counterargument raised was that Substack's embrace of controversial contributors provided a home for people who'd engaged in harassment on other platforms (and who were often no longer welcome there).

Decisions to be made by Substack:

• Does offloading moderation to users increase the amount of potentially-objectionable content hosted by Substack?
• Does this form of moderation give Substack the appearance it approves of controversial content contributed by others?
• Is the company prepared to take a more hands-on approach if the amount of objectionable content hosted by Substack increases? 

Questions and policy implications to consider:

• Does a policy that relies heavily on users and writers to enforce allow users and contributors to shape Substack's "identity?"
• Does limiting moderation by Substack attract the sort of contributors Substack desires to host and/or believes will make it more profitable?
• Does the sharing of content off-platform undermine Substack's belief that others have complete control over the kind of content they're seeing?

Resolution: The controversy surrounding Substack's roster of writers continued to increase, along with calls for the platform to do more to moderate hosted content. Subtack's response was to retirate its embrace of "free press and free expression," but also offered a few additional moderation tweaks not present in its policies when it first received increased attention late last year.

Most significantly, it announced it would not allow "hate speech" on its platform, although its definition was more expansive than policies on other social media services. Attacks on people based on race, ethnicity, religion, gender, etc. would not be permitted. However, Substack would continue to host attacks on "ideas, ideologies, organizations, or individuals for other reasons, even if those attacks are cruel and unfair."

Originally posted to the Trust & Safety Foundation website.

Filed Under: content moderation, controversy, email, newsletters
Companies: substack

Content Moderation Case Study: Knitting Community Ravelry Bans All Talk Supporting President Trump (2019)

from the knitting-politics dept

Summary: When people think of content moderation and political debates, they may not think about knitting. However, the knitting community at the online site Ravelry has become a fascinating place to explore content moderation questions. This actually goes back many years, as Ravelry’s content moderation practices (handled by dozens of volunteer moderators) were studied for a PhD dissertation by Sheila Saden Pisa that was published in 2013, entitled: “In search of a practice: large scale moderation in a massive online community.”

Knitting and Ravelry have also been quite political at times. All the way back in 2009, a blog post was written by someone who was kicked off of Ravelry, and she believed it was because of her conservative political views. After the election of Donald Trump in 2016, Ravelry was where the initial plans for the now famous “pussyhats” (for the Women’s March protesting Trump’s Presidency) were first released and shared. Ioana Literat and Sandra Markus studied Ravelry’s role in online participation, civic engagement and “craftivism.”

Still, it caught many people by surprise, in late 2019, when Ravelry declared a new policy, saying that it would no longer allow any posts supporting Donald Trump. From the announcement:

We are banning support of Donald Trump and his administration on Ravelry.

This includes support in the form of forum posts, projects, patterns, profiles, and all other content. Note that we will not destroy project notebook data. If a project needs to be removed from the site, we will make sure that you have access to your data. Even if you are permanently banned from Ravelry, you will still be able to access any patterns that you purchased. Also, we will make sure that you receive a copy of your data.

We cannot provide a space that is inclusive of all and also allow support for open white supremacy. Support of the Trump administration is undeniably support for white supremacy.

The Community Guidelines have been updated with the following language: “Note that support of President Trump, his administration, or individual policies that harm marginalized groups, all constitute hate speech.”

The company noted that this was not a statement of support for other candidates, nor was it saying that it would ban people who (outside of Ravelry) supported Trump. It also made clear that it was not banning other political topics or statements in support of other candidates. Instead, it said: “We are definitely not banning conservative politics. Hate groups and intolerance are different from other types of political positions.” The decision created quite a lot of attention with many supporters and detractors.

Decisions to be made by Ravelry:

• How do you decide when one politician’s positions are so problematic to your community that you ban any support of that candidate?
• How will this policy be enforced? Should it apply to earlier statements of support or just future ones?
• How will attempts to get around the ban (such as with hints or euphemisms) be dealt with?
• Can volunteer moderators be supporters of Trump?

Questions and policy implications to consider:

• What are the pros & cons of banning support for Trump v. banning all talk of politics?
• Would other approaches -- such as moving all political talk, or all talk about Trump, to a specific area -- work as effectively?
• If, as has been suggested in some Section 230 reform bills, the laws change to require “political neutrality” in content moderation, how will Ravelry’s moderation practices be impacted?

Resolution: After Donald Trump left the White House on January 20th, Ravelry reiterated that its policy remained the same, even though Trump was no longer President. A year and a half after Ravelry’s decision, the New Yorker published a long, and detailed article about the decision to ban Trump support on the site and how it is going, entitled “How Politics Tested Ravelry and the Crafting Community.”

On the day of the ban, Kim Denise, one of the volunteer moderators, told me, “I was, like, I’m so psyched. I’m so proud to be part of Ravelry.” Then the ban happened. “And it was, like, Oh, my God. I wish we’d thought this through.” Right-wing trolls began signing up for Ravelry accounts and spamming threads with anti-Ravelry or pro-Trump sentiment. Denise described it as “hordes of screaming people lining up to sling feces at us. . . . It was terrible.” Users scurried to help moderators by flagging posts for deletion. They recruited a retired moderator to help deal with trolls. Within a couple of months, most of the activity generated by the Trump ban had subsided. Conservative users banded together, in a movement hashtagged #RavelryExodus, deleting their accounts and shifting to other platforms to sell patterns.

The company’s founder also admitted that after the ban was announced, she realized the difficulty in figuring out the exact boundaries of enforcement:

Jessica admitted that Ravelry has struggled to pinpoint exactly what constitutes inappropriate content. “Some of this stuff is so nuanced,” she said. “Think about what tweet got Trump banned. It was not about attending the Inauguration.” She went on, “We went through some pretty crazy rabbit holes: ‘O.K., this is an eagle, but it isn’t really the Nazi eagle. Or is it?’ It’s just, like, ugh.”

Originally posted to the Trust & Safety Foundation website.

Filed Under: content moderation, donald trump, knitting, politics
Companies: ravelry

Content Moderation Case Study: YouTube's New Policy On Nazi Content Results In Removal Of Historical And Education Videos (2019)

from the godwin-in-effect dept

Summary: On June 5, 2019, YouTube announced it would be stepping up its efforts to remove hateful content, focusing on the apparent increase of white nationalist and pro-Nazi content being created by users. This change in algorithm would limit views of borderline content and push more viewers towards content less likely to contain hateful views. The company's blog post specifically stated it would be removing videos that "glorified Nazi ideology."

Unfortunately, when the updated algorithm went to work removing this content, it also took down content that educated and informed people about Nazis and their ideology, but quite obviously did not "glorify" them.

Ford Fischer -- a journalist who tracks extremist and hate groups -- noticed his entire channel had been demonetized within "minutes" of the rollout. YouTube responded to Fischer's attempt to have his channel reinstated by stating multiple videos -- including interviews with white nationalists -- violated the updated policy on hateful content.

A similar thing happened to history teacher Scott Allsop, who was banned by YouTube for his uploads of archival footage of propaganda speeches by Nazi leaders, including Adolph Hitler. Allsop uploaded these for their historical value as well as for use in his history classes. The notice placed on his terminated account stated it had been taken down for "multiple or severe violations" of YouTube's hate speech policies.

Another YouTube user noticed his upload of 1938 documentary about the rise of the Nazi party in Germany had been taken down for similar reasons, even though the documentary was decidedly anti-Nazi in its presentation and had obvious historical value.

Decisions to be made by YouTube:

• Should algorithm tweaks be tested in a sandboxed environment prior to rollout to see how often they're flagging content that doesn't actually violate policies?
• Given that this sort of mis-targeting has happened in the past, does YouTube have a response plan in place to swiftly handle mistaken content removals?
• Should additional staffing be brought on board to handle the expected collateral damage of updated moderation policies? 

Questions and policy implications to consider:

• Should there be a waiting period on enforcement that would allow users with flagged content to make their case prior to being hit by enforcement methods like demonetization or bans?
• Should YouTube offer some sort of compensation to users whose channels are adversely affected by mistakes like these? 
• Should users whose content hasn't been flagged previously for policy violations be given a benefit of a doubt when flagged by automated moderation efforts?

Resolution: In most cases, content mistakenly targeted by the algorithm change was reinstated within hours of being taken down. In the case of Ford Fischer, reinstatement took longer. And he was again demonetized by YouTube in early 2021, apparently over raw footage of the January 6th riot in Washington, DC. Within hours, YouTube had reinstated his account, but not before drawing more negative press over its moderation problems.

Originally published to the Trust & Safety Foundation website.

Filed Under: content moderation, disinformation, education, educational videos, hate speech, history, nazis
Companies: youtube

Content Moderation Case Study: Google Removes Popular App That Removed Chinese Apps From Users' Phones (2020)

from the app-cleansing dept

Summary: An app that allowed users to moderate content residing on their own phones was given the boot by Google after it was determined to be in violation of Play Store rules.

The self-explanatory "Remove China Apps" app was developed by Indian engineers residing in Jaipur, India in collaboration with One Touch App Labs. The app was created in response to growing backlash against China during the early days of the COVID-19 pandemic, after early reports showed visitors to a seafood market in Wuhan, China had contributed to the spread of the virus.

India's proximity to China intensified this backlash. How removing apps developed in China was supposed to stop the spread of the virus is best left to the possibly literally-fevered imaginations of the app developers and the millions of Indian users who downloaded the app.

However questionable the motivation for the development and deployment of the app, it did allow Android users to easily identify apps developed by Chinese developers and remove them from their phones. However, this secondhand act of personal content moderation was soon hampered by Google, which dumped the app from its Play store, citing violations of its policies. Specifically, Google pointed to its "deceptive behavior" policy. App developers are forbidden from uploading apps that "encourage or incentivize users to remove or disable third-party apps."

Decisions to be made by Google:

• Should Google control how Android phone purchasers choose to use their phones?
• Should Google be more concerned with possible exploitation of permissions to compromise phone users, rather than the ability of users to more closely moderate the content of their devices?
• Is an app that openly states it will remove other apps actually "deceptive?" 

Questions and policy implications to consider:

• Could apps like these serve a useful purpose, like giving Google a heads up on questionable apps/developers?
• Does maintaining a blocklist for devs/users achieve the same objective without harming developers who rely on crowdfunding? 
• Does pushing Android users towards sideloading apps do less to protect users than removing questionable apps that run afoul of rules rarely broken by app developers?

Resolution: Google has refused to reinstate the app. Android users are still able to sideload the app if they wish. The popularity of the app went further than India and the county's kneejerk reaction to developments in Wuhan, China.

But Google still has a battle ahead of it. With it commanding nearly 95% of the Indian market, the demand for apps that (correctly or incorrectly) "punish" Chinese app developers remains a growth market.

Originally posted to the Trust & Safety Foundation website.

Filed Under: apps, china, content moderation, covid, deceptive behavior, india, removing apps, security
Companies: google

Content Moderation Case Studies: How To Moderate World Leaders Justifying Violence (2020)

from the leaders-acting-badly dept

Summary: There is an inherent tension in handling content moderation of world leaders -- especially more controversial ones. If those leaders break the rules on social media, some reasonably call for the content, or the accounts, to be removed for violating policies. Others, however, point out that it is important for the public to be aware of what world leaders are saying, rather than removing and hiding the speech.

Twitter has had a public interest exception for tweets from world leaders since at least 2019. Under that policy, Twitter may choose to leave up some content from a world leader that the company admits violates its rules, under the belief that it is more important that the world know what that leader has said. Since 2019, Twitter announced that when it found such content, it would label it clearly -- publicly noting that it violated the company’s policies, but was being kept up due to the public interest.

The policy was put to the test in October 2020, following the murder of a teacher in a Paris suburb, after the teacher had shown students cartoons of the Prophet Muhammad while discussing the controversy over such drawings. A week later, three people were stabbed in Nice, in southern France. French President Emmanuel Macron described both attacks as “Islamist terrorist attacks."

Soon after the latter attack, former prime minister of Malaysia Mahathir Bin Mohamad posted a Twitter thread discussing both attacks. While the thread touched on a variety of points, urged people not to scapegoat entire religions, and said he did not approve of the killings, the twelfth tweet raised many concerns by stating: "Muslims have a right to be angry and kill millions of French people for the massacres of the past."

Twitter posted its public interest notice on this particular tweet, noting that it violated Twitter’s rules about glorifying violence, but Twitter felt that it “may be in the public’s interest for the tweet to remain accessible.”

Many disagreed with this decision, including French officials. France’s digital minister, Cédric O, claimed that if Twitter did not remove the tweet, it would make the company an “accomplice to a formal call for murder.”

Decisions to be made by Twitter:

• What qualifies a tweet from a foreign leader that violates policies to remain up under a “public interest” exception?
• Under what conditions would Twitter reverse this policy and remove tweets?
• How much context should Twitter take into account regarding the tweets? That is, how much should the attacks in France play into the decision regarding this tweet?

Questions and policy implications to consider:

• Whether or not Twitter removes this particular tweet, it is likely to get attention and news coverage. How much does it matter whether or not Twitter removes or labels the particular tweet?
• Should world leaders get special treatment by nature of their position and the fact that what they say can impact world events? 

Resolution: Twitter only kept the tweet up for a few hours before reversing course and deciding to remove the tweet entirely for violating its rules. Twitter did not comment on why it changed its position on this particular tweet, only telling the media that the tweet was removed for violating its policies on glorifying violence. The company chose not to explain why it initially qualified for a public interest exception, only to be changed later.

Originally posted to the Trust & Safety Foundation website.

Filed Under: content moderation, france, mahatir bin mohamad, newsworthiness, paris, violence, world leaders
Companies: twitter

Content Moderation Case Study: Apple Blocks WordPress Updates In Dispute Over Non-Existent In-app Purchase (2020)

from the ok-landlord dept

Summary: Apple controls what apps get onto iPhone and iPads via its full control over the iOS App Store. Every app (and its updates) need to be reviewed by Apple staff before it’s allowed in the store -- and Apple puts in place its own rules for what is and what is not allowed.

One of those rules is that Apple takes a 30% cut of any sales. That fee has become somewhat controversial, especially among service providers who don’t rely on the App Store for discovery, but whose customers likely come on their own -- including Spotify and Epic Games. Spotify, in particular, has urged users to subscribe directly, to avoid having to pay the additional amount per month to cover Apple’s fees. In response, Apple forbade Spotify from even mentioning that it’s cheaper to subscribe outside of the App Store, which is now a central piece of an antitrust fight that is ongoing in the EU.

Perhaps because of all of this, Apple has had to make decisions about whether or not to allow apps in the App Store that seek to avoid paying Apple’s cut of the fees. In August of 2020, Matt Mullenweg, the CEO of Automattic, and the founder/lead developer of the WordPress content management system, announced that the iOS app for WordPress had been frozen by Apple. The given reason was that Apple believed that WordPress was trying to avoid the fees for in-app purchases.

This was the cause of much confusion, as many people noted that the app did not actually sell anything. While WordPress.com does offer paid hosting plans (and domain reselling), that was not a part of the WordPress app. However, as Mullenweg’s tweet showed, Apple was noting that because somewhere else in WordPress.com’s business, it sold things, that meant that WordPress had to pay it a 30% cut of those sales (even though they were outside of the app itself) in order to keep the app in the App Store.

Decisions to be made by Apple:

• How thoroughly should the company be reviewing the business models of apps in the App Store to determine whether they can be included?
• What actually constitutes an attempt to get around the App Store fee?
• Will app developers take advantage of exceptions to the rules if Apple does not follow them closely?
• Should the company allow alternative ways of getting apps on the phone outside of the App Store?

Questions and policy implications to consider:

• When a company builds an entire device ecosystem, should it be able to set its own rules for what apps are allowed on the device?
• Can content moderation decisions raise antitrust concerns?
• Are there policy implications of a single entity reviewing what apps are allowed on a device?

Resolution: As this story got more attention, Apple apologized and restored the WordPress developer account. However, its statement on the matter implied that WordPress had “removed” an option in the app to pay for hosting plans:

We believe the issue with the WordPress app has been resolved. Since the developer removed the display of their service payment options from the app, it is now a free stand-alone app and does not have to offer in-app purchases. We have informed the developer and apologize for any confusion that we have caused.

But users of the app say it never had any in-app purchases at all. The only thing it had were descriptions of WordPress.com Premium offerings, but no way to buy them. Mullenweg said that, before going public, he had asked Apple if removing those mentions would restore the account, and Apple had said it would not.

The reinstatement appeared to take Mullenweg by surprise.

In January of 2021, Apple also moved to lower the cut it took for in-app payments from “small” developers (those making less than $1 million a year in annual sales) to 15%. It was also revealed that Apple quietly cut a special deal with Amazon to charge the retailer a 15% cut for Amazon’s Prime Video app.

Originally published on the Trust & Safety Foundation website.

Filed Under: app store, content moderation, fees, in-app purchases, matt mullenweg, wordpress
Companies: apple, automattic

Content Moderation Case Study: Google Refuses To Honor Questionable Requests For Removal Of 'Defamatory' Content (2019)

from the reputation-management-through-lying dept

Summary: Google has long been responsive to court orders demanding the removal of content, if they're justified. Google has fought back against dubious orders originating from "right to be forgotten" demands from outside the US, and has met no small amount of DMCA abuse head on. But, generally speaking, Google will do what's asked if there's a legal basis for the asking.

But not everyone approaching Google acts in good faith. First, there are any number of bad actors hoping to game the system to juice their Google search rankings.

And, beyond that, there are any number of shady "reputation management" firms willing to defraud courts to obtain orders demanding Google remove content that reflects poorly on their clients.

For a couple of years, these bad actors managed to make some search engine optimization (SEO) inroads. They were able to fraudulently obtain court orders demanding the removal of content. The worst of these companies didn't even bother to approach courts. They forged court orders and sent these to Google to get negative listings removed from search results.

This new system opportunistically preyed on two things: Google's apparent inability to police its billions of search results and the court system's inability to vet every defamation claim thoroughly.

But the system -- not the one operated by the US government or Google -- prevailed. Those targeted by bogus takedown demands fought back, digging into court dockets and the people behind the bogus requests. Armed with this information, private parties approached the courts and Google and asked for content that had been removed illicitly be reinstated.

Decisions to be made by Google:

• Should Google act as an intercessor on behalf of website operators or should it just act as "dumb" pipe that passes no judgment on content removal requests?
• Does manual vetting of court orders open Google up to additional litigation?
• Does pushing back against seemingly questionable court orders allow Google to operate more freely in the future?

Questions and policy implications to consider:

• Given the impossibility of policing content delivered by search results, is it wrong to assume good faith on behalf of entities requesting content removal?
• Is it possible to operate at Google's scale without revamping policies to reflect the collateral damage it can't possibly hope to mitigate?
• If Google immunizes itself by granting itself more discretion on disputed content, does it open itself up to more direct regulation by the US government? Does it encourage users to find other sources for content hosting?

Resolution: Google chose to take more direct action on apparently bogus court orders fraudulently obtained or created by reputation management firms. It took more direct action on efforts to remove content that may have been negative, but not defamatory, in response to multiple (private) investigations of underhanded actions taken by those in the reputation management field. Direct moderation -- by human moderators -- appears to have had a positive effect on search results. Since this outburst back in 2016, shadier operators have steered clear of manipulating search results with bogus court orders.

Originally posted to the Trust & Safety Foundation website.

Filed Under: content moderation, defamation, fake lawsuits, fraud, search engine optimization, search rankings, seo, takedowns
Companies: google