Trumpist Gettr Social Network Continues To Speed Run Content Moderation Learning Curve: Bans, Then Unbans, Roger Stone

from the welcome-to-the-content-moderation-game dept

Remember Gettr? That's the Trumpist social network run by former Trump spokesperson (and vexatious lawsuit filer) Jason Miller that promised to be supportive of "free speech." As we point out what happens with every new social network that jumps into the space with promises to "support free speech!" and "not censor!" before long they will begin to realize content moderation is required to keep your site running -- and soon they discover that content moderation will involve difficult choices. And, sometimes, it involves making mistakes.

Of course, whenever Twitter, Facebook, Instagram or whoever else note that they made a "mistake" with a content moderation decision and reverse it, there are always some people who insist it couldn't possibly be a mistake and must be [insert conspiratorial reason here]. So I find it hilarious that on Wednesday, Gettr got to experience all this as well. First, Trump buddy and admitted "dirty trickster" Roger Stone went on Gab -- another such social network -- to whine about how he was "censored" by Gettr, claiming it was because he had made what he believed were disparaging remarks about Miller (and Steve Bannon).

This graphic alone is pretty hilarious, given all the people who insisted that Gettr doesn't do moderation.

A few hours later, Jason Miller told Salon reporter Zachary Petrizzo, that it was all a mistake, and blamed the problem of a bunch of fake Stone accounts that the site was trying to get rid of. In the process, Miller claimed, they "accidentally" deleted the real Stone's account. The timing coming right after Stone made remarks about Miller was, apparently, entirely coincidental.

"Multiple fake Roger Stone accounts were suspended following user complaints, but his real Gettr account was inadvertently suspended too," Miller told Salon via email. "His correct account is currently active, and the imposter accounts have all been removed."

It's almost like... running a social media account with malicious actors are trying to abuse requires content moderation, and that will often involve making mistakes, rather than, say, "bias" or desires to silence particular people.

Who knows why Gettr actually suspended Stone? These kinds of things happen all the time on all the other social media sites too. It doesn't mean it's malicious or directed at silencing a critic (though, honestly, there's a higher likelihood that is true with a site like Gettr than with Twitter or Facebook, that actually have real policies in place and a history of training staff to moderate judiciously). It could actually be a mistake. Just like Twitter and Facebook sometimes make mistakes.

I'm still waiting for Miller and his fans to realize that all of these challenges that Gettr is facing are no different than the ones that other social media apps faced -- and that maybe all their earlier complaints of "censorship" were just as bullshit then as the claims about Gettr being censorial today are. I get the feeling that I'll be waiting a long, long time for that recognition to set in.

Filed Under: anti-conservative bias, content moderation, jason miller, mistakes, roger stone
Companies: gab, gettr

Oversight Board Tells Facebook It Needs To Shape Up And Be More Careful About Silencing Minorities Seeking To Criticize The Powerful

from the pay-attention-to-this dept

Tomorrow, the Oversight Board is set to reveal its opinion on whether Facebook made the right decision in banning former President Trump. And that will get tons of attention. But the Board came out with an interesting decision last week regarding a content takedown in India, that got almost no attention at all.

Just last week, we wrote about an ongoing issue in India, where the government of Prime Minister Narendra Modi has failed in almost every way possible in dealing with the COVID pandemic, but has decided the best thing to focus on right now is silencing critics on Twitter. That backdrop is pretty important considering that the very next day, the Oversight Board scolded Facebook for taking down content criticizing Modi's government.

That takedown was somewhat different and the context was very different. Also, it should be noted that as soon as the Oversight Board agreed to take the case, Facebook admitted it had made a mistake and reinstated the content. However, this case demonstrates something important that often gets lost in all of the evidence free hand-wringing about "anti-conservative bias" from people who wrongly insist that Facebook and Twitter only moderate the accounts of their friends. The truth is that content all across the board gets moderated -- and often the impact is strongest on the least powerful groups. But, of course, part of their lack of power is that they're unable to rush onto Fox News and whine about how they're being "censored."

The details here are worth understanding, not because there was some difficult decision to make. Indeed, as noted already, Facebook realized it made a mistake almost immediately after the Oversight Board decided to look into this, and when asked why the content was taken down, basically admitted that it had no idea and that it was a complete and total mistake. Here was the content, as described by the Oversight Board ruling:

The content touched on allegations of discrimination against minorities and silencing of the opposition in India by “Rashtriya Swayamsevak Sangh” (RSS) and the Bharatiya Janata Party (BJP). RSS is a Hindu nationalist organization that has allegedly been involved in violence against religious minorities in India. “BJP” is India’s ruling party to which the current Indian Prime Minister Narendra Modi belongs, and has close ties with RSS.

In November 2020, a user shared a video post from Punjabi-language online media Global Punjab TV and an accompanying text. The post featured a 17-minute interview with Professor Manjit Singh, described as “a social activist and supporter of the Punjabi culture.” In its post, Global Punjab TV included the caption “RSS is the new threat. Ram Naam Satya Hai. The BJP moved towards extremism.” The media company also included an additional description “New Threat. Ram Naam Satya Hai! The BJP has moved towards extremism. Scholars directly challenge Modi!” The content was posted during India’s mass farmer protests and briefly touched on the reasons behind the protests and praised them.

The user added accompanying text when sharing Global Punjab TV’s post in which they stated that the CIA designated the RSS a “fanatic Hindu terrorist organization” and that Indian Prime Minister Narendra Modi was once its president. The user wrote that the RSS was threatening to kill Sikhs, a minority religious group in India, and to repeat the “deadly saga” of 1984 when Hindu mobs attacked Sikhs. They stated that “The RSS used the Death Phrase ‘Ram naam sat hai’.” The Board understands the phrase "Ram Naam Satya Hai" to be a funeral chant that has allegedly been used as a threat by some Hindu nationalists. The user alleged that Prime Minister Modi himself is formulating the threat of “Genocide of the Sikhs” on advice of the RSS President, Mohan Bhagwat. The accompanying text ends with a claim that Sikhs in India should be on high alert and that Sikh regiments in the army have warned Prime Minister Modi of their willingness to die to protect the Sikh farmers and their land in Punjab.

The post was up for 14 days and viewed fewer than 500 times before it was reported by another user for “terrorism.” A human reviewer determined that the post violated the Community Standard on Dangerous Individuals and Organizations and took down the content, which also triggered an automatic restriction on the use of the account for a fixed period of time. In its notification to the user, Facebook noted that its decision was final and could not be reviewed due to a temporary reduction in its review capacity due to COVID-19. For this reason, the user appealed to the Oversight Board.

So, you had an ethnic minority -- one who had been attacked in the past -- warning about those currently in power. And Facebook took it down, refused to review the appeal... until the Oversight Board turned its eye on it, and then admitted it was a mistake, and basically threw its hands in the air and said it had no idea why it had been taken down in the first place.

According to Facebook, following a single report against the post, the person who reviewed the content wrongly found a violation of the of the Dangerous Individuals and Organizations Community Standard. Facebook informed the Board that the user’s post included no reference to individuals or organizations designated as dangerous. It followed that the post contained no violating praise.

Facebook explained that the error was due to the length of the video (17 minutes), the number of speakers (two), the complexity of the content, and its claims about various political groups. The company added that content reviewers look at thousands of pieces of content every day and mistakes happen during that process. Due to the volume of content, Facebook stated that content reviewers are not always able to watch videos in full. Facebook was unable to specify the part of the content the reviewer found to violate the company’s rules.

Got that? Facebook is basically saying "yeah, it was a mistake, but that was because it was a long video, and we just had one person reviewing who probably didn't watch the whole video."

Here's the thing that the "oh no, Facebook is censoring people" don't get. This happens all the time. And none of us hear about it because the people it happens to often are unable to make themselves heard. They don't get to run to Fox News or Parler or some other place and yell and scream. And, this kind of "accidental" moderation especially happens to the marginalized. Reviewers may not fully understand what's going on, or not really understand the overall context, and may take the "report" claim at face value, rather than having the ability or time to fully investigate.

In the end, the Oversight Board told Facebook to put back the content, which was a no-brainer since Facebook had already done so. However, more interesting were its policy recommendations (which, again, are not binding on Facebook, but which the company promises to respond to). Here, the Oversight Board said that Facebook should make its community standards much more accessible and understandable, including translating the rules into more languages.

However, the more interesting bit was that it said that Facebook "should restore human review and access to a human appeals process to pre-pandemic levels as soon as possible while fully protecting the health of Facebook’s staff and contractors." There were some concerns, early in the pandemic, about how well content moderation teams could work from home, since a lot of that job involves looking at fairly sensitive material. So, there may be reasons this is not really doable just yet.

Still, this case demonstrates a key point that we've tried to raise about the impossibility of doing content moderation at scale. So much of it is not about biases, or incompetence, or bad policies, or not wanting to do what's right. A hell of a lot of it is just... when you're trying to keep a website used by half the world operating, mistakes are going to be made.

Filed Under: appeals, content moderation, free speech, india, minorities, mistakes, review, takedowns
Companies: facebook, oversight board

Appeals Court Says Address Mistakes On Warrants Are Mostly Harmless, Not Worth Getting Excited About

from the what-even-the-fuck dept

In a case involving a drug bust utilizing a warrant with erroneous information, the Sixth Circuit Court of Appeals had this to say [PDF] about the use of boilerplate language and typographical errors:

Challenges to warrants based on typographical errors or factual inaccuracies typically fall under this Circuit’s clerical error exception. We have consistently found that inadvertent drafting mistakes, for instance transposing a number in a street address or listing an incorrect nearby address, do not violate the Fourth Amendment’s prohibition on unreasonable searches and seizures. That is because those errors create little risk of a mistaken search or a general warrant granting police an unconstitutionally broad authority to conduct searches.

The order to search listed the wrong address. Here's why:

This description at the beginning of the warrant correctly directed officers to Abdalla’s precise New Hope Road address in DeKalb County. But the warrant’s final paragraph “commanded” officers “to search the . . . premises located at 245 Carey Road, Hartsville, Trousdale Tennessee.” (R. 20-1, Search Warrant, Page ID # 59.) Agent Gooch testified that the Carey Road address came from using a previous warrant as a template. Although Judge Patterson had jurisdiction over Abdalla’s residence in DeKalb County, he lacked jurisdiction in Trousdale County, which encompassed the Carey Road property listed on the warrant’s final page.

The court says this is harmless. Rather than suppress evidence in hopes that cops won't just copy-paste "sworn statements" before running them by a judge, the Appeals Court says this creates "little risk" of "mistaken searches." Perhaps in this case the risk was minimal. The rest of the warrant correctly described the residence and how to locate it. But to pretend careless warrant crafting rarely results in "mistaken searches" ignores how often it happens -- and how often this supposed low-risk "mistake" results in real harm.

"Little risk?" Here's what's actually happening in the Sixth Circuit, which covers Kentucky, Michigan, Ohio, and Tennessee.

Oak Park, Michigan (November 2019): Police raid the wrong side of a duplex, breaking windows and the front door before realizing their mistake.

Flint, Michigan (October 2014): Troopers go to the wrong house to locate a fugitive, shoot family's dog in the face.

Detroit, Michigan (May 2017): After conducting a one-day(!) human trafficking investigation, a SWAT teams raids the wrong house, handcuffs everyone present (including two children) before discovering their mistake.

Detroit, Michigan (September 2017): DEA agents raid two(!) wrong addresses. The forty officers(!!) recover no drugs. Search warrants and property receipts left at the properties by the feds were blank, according to this report.

Detroit, MIchigan (April 2017): Police raid wrong house, kill homeowner's dog.

Nashville, Tennessee (August 2020): Three cops raid wrong house, traumatizing the resident and two young children. Officers predicated the search on housing information that hadn't been updated since November 2018.

Lebanon, Tennessee (January 2006) - Officers raid wrong house, kill 61-year-old man while his wife is handcuffed in another room.

Louisville, Kentucky (October 2018) - Officers (three of whom shot and killed Breonna Taylor during another botched raid) using outdated information raid a house looking for someone who had moved out four months earlier.

Bowling Green, Kentucky (July 2016) - Police raid the wrong house looking for a Black suspect. Officers handcuff and question the homeowner, who weighs 100 pounds less than the suspect they're looking for. The interrogated homeowner is also one foot taller than the suspect. He's also white.

Louisville, Kentucky (January 2020) - Officers enter the wrong house seeking a shooting suspect, handcuffing one of the residents.

Louisville, Kentucky (July 2020) - Cops raid a vacant house looking for a drug suspect who had already been arrested and was in jail. Officers break windows, destroy a door, and handcuff the man hired to paint the interior of the vacant residence.

Cleveland, Ohio (November 2018) - Wrong house raided during a shooting investigation. Cops cause over $8,000 of physical damage to the house and spend an hour interrogating all the residents -- some of whom are disabled -- before realizing their mistake.

Strongsville, Ohio (May 2010) - A man and his 14-year-old daughter are forced out of their house and made to lay face down on the lawn until officers realize they have the wrong address.

Cleveland, Tennessee (May 2018) - DEA and local cops raid wrong house in search of murder suspect. Flashbangs are deployed into the house despite the presence of young children -- something officers should have been able to discern from the number of toys around the front entry of the residence.

This is just a small sampling. And this is just from this circuit, which covers only four of the 50 states. This happens far too frequently for it to be shrugged off by an Appeals Court, even if the facts of the case might lead the court to conclude a mistake in an affidavit doesn't warrant the suppression of evidence.

The Fourth Amendment places the sanctity of the home above all else. And yet, officers continue to perform searches without performing the due diligence required to support a home invasion. Outdated info, unverified claims by informants, minimal investigative work… it all adds up to situations where rights are violated and residents are recklessly subjected to violence and deadly force.

How bad can it get? Here's a true horror story that shows just how little law enforcement agencies care about the people they're supposed to protect and serve:

Embarrassed cops on Thursday cited a "computer glitch" as the reason police targeted the home of an elderly, law-abiding couple more than 50 times in futile hunts for bad guys.

Apparently, the address of Walter and Rose Martin's Brooklyn home was used to test a department-wide computer system in 2002.

What followed was years of cops appearing at the Martins' door looking for murderers, robbers and rapists - as often as three times a week.

Every wrong visit to the house was a chance for officers to respond with deadly force to perceived threats. That these residents managed to survive 50+ incidents with cops looking for violent criminals is a miracle. "Mistaken searches" are not an acceptable outcome. Blanket statements like these issued by courts just give cops more reasons to cut corners before banging their way into someone's home in search of nonexistent criminals or criminal activity.

Filed Under: 4th amendment, 6th circuit, mistakes, police, warrants, wrong address

EFF Highlights Stories Of Bad Content Moderation With New TOSsed Out Site

from the content-moderation-is-impossible dept

We've pointed out for many years that content moderation at scale isn't just hard, it's impossible to do well. At the scale of giant platforms, there needs to be some level of moderation or the platforms and users will get overwhelmed with spam or abuse. But at that scale, there will be a ton of mistakes -- both type I and type II errors (blocking content that shouldn't be blocked and failing to block content that probably should be blocked). Some -- frankly dishonest -- people have used a few examples of certain content moderation choices to falsely claim that there is "anti-conservative bias" in content moderation choices. We've pointed out time and time again why the evidence doesn't support this, though many people insist it's true (and I'll predict they'll say so again in the comments, but when asked for evidence, they will fail to present any).

That's not to say that the big platforms and their content moderation practices are done well. As we noted at the very beginning, that's an impossible request. And it's important to document the mistakes. First, it helps get those mistakes corrected. Second, while it will still be impossible for the platforms to moderate well, they can still get better and make fewer errors. Third, it can help people understand that errors are not because someone hates you or has animus towards a political group or political belief, but because they fuck up the moderation choices all the time. Fourth, it can actually help to find what actual patterns there are in these mistakes, rather than relying on moral panics. To that end, it's cool to see that the EFF has launched a new site, creatively dubbed TOSsed Out to help track stories of bad content moderation practices.

Just looking through the stories already there should show you that bad content moderation choices certainly aren't limited to "conservatives," but certainly do seem to end up impacting actually marginalized groups:

EFF is launching TOSsed Out with several examples of TOS enforcement gone wrong, and invites visitors to the site to submit more. In one example, a reverend couldn’t initially promote a Black Lives Matter-themed concert on Facebook, eventually discovering that using the words “Black Lives Matter” required additional review. Other examples include queer sex education videos being removed and automated filters on Tumblr flagging a law professor’s black and white drawings of design patents as adult content. Political speech is also impacted; one case highlights the removal of a parody account lampooning presidential candidate Beto O’Rourke.

“The current debates and complaints too often center on people with huge followings getting kicked off of social media because of their political ideologies. This threatens to miss the bigger problem. TOS enforcement by corporate gatekeepers far more often hits people without the resources and networks to fight back to regain their voice online,” said EFF Policy Analyst Katharine Trendacosta. “Platforms over-filter in response to pressure to weed out objectionable content, and a broad range of people at the margins are paying the price. With TOSsed Out, we seek to put pressure on those platforms to take a closer look at who is being actually hurt by their speech moderation rules, instead of just responding to the headline of the day.”

As the EFF notes, this is something of a reaction to the White House's intellectually dishonest campaign database building program to get people to report stories of social media bias against conservatives. Unlike the White House, which is focused on pulling some one-sided anecdotes it can misrepresent for political points, the TOSsed out effort is a real opportunity to track what kinds of mistakes happen in content moderation and how to deal with them.

Filed Under: content moderation, mistakes, tossed out, type 1 errors, type 2 errors
Companies: eff

Watchdog Says Australia's Traffic Enforcement System Has Hits Hundreds Of Drivers With Bogus Fines

from the move-fast-and-break-things:-hefty-bureaucracy-edition dept

In Australia, the government has automated traffic enforcement, letting the machines do the work. It's also automated the limited due process procedures, giving aggrieved citizens the chance to have their complaints and challenges ignored at scale, as The Newspaper reports.

At least 397 motorists in Victoria, Australia, lost their right to drive because the state government bungled the handling of speed camera fines. In a report released Wednesday, Victoria Ombudsman Deborah Glass blasted Fines Victoria, the agency responsible for overseeing the handling of citations. The report reviewed 605 complaints from members of the public about how this year-old state agency handled their situations.

"Many complaints were about delay in the processing of nominations, completing reviews and implementing payment plans," Glass said in a statement. "The impact of these issues should not be underestimated. People had their licenses wrongly suspended, or were treated as liable for substantial fines, when they had committed no offense."

Automating the back end and the front end results in this sort of thing -- just one of several personal anecdotes the Ombudsman collected from angry Australians forced to deal with the user-unfriendly system. From the report [PDF]:

Dan contacted the Ombudsman about infringements incurred by his late son who had died in tragic circumstances. Dan said he made multiple attempts to call Fines Victoria, but on each occasion the phone went to an automated service, or he was on hold for a long time.

Dan also said he had sent through multiple complaints online but had not received a response. We contacted Fines Victoria, providing a copy of the Coroner’s ‘confirmation of death certificate’ for Dan’s son. We followed up several times after receiving no response. Fines Victoria said later they had not received the email from us.

Dan received further enforcement letters which he said was extremely stressful and upsetting. He became increasingly concerned the Sheriff would go to his son’s home which he had shared with housemates.

Fines Victoria put all 27 outstanding matters, totalling $7,636.60, on hold pending their withdrawal. The enquiry process, however, went for close to 40 days. This is despite Dan providing all relevant information to Fines Victoria a number of times before contacting us.

That's one of the possible outcomes of Fines Victoria mishandling its end of the complaint process: law enforcement showing up to seize property and auction it off to pay fines. All of this is set in motion automatically via date triggers. From the report, it appears Australian citizens can spend every day of their challenge periods on hold without ever reaching anyone who could help them with a resolution.

Even the lucky few who manage to speak to a live person aren't going to receive much help. The multiple failures detailed in the report received responses from Fines Victoria ranging from "not our fault" to "we'll try not to screw so many people over in the future."

One citizen sums up the Fines Victoria experience succinctly:

When we make a mistake, we pay. When they make a mistake, we pay.

Some of this could have been prevented if Fines Victoria had used all of its allotted time to ensure proper functionality before bringing the system online. Instead, the government decided to rush implementation, leading to a spectacular run of failures that stripped people of their licenses, hit them with bogus fees, and otherwise made their lives miserable.

On 31 December 2017, Fines Victoria began operation with only partial IT functionality. Problems with IT functionality, and related procedural and processing issues, have been apparent since the inception of the agency…

It was not clear why, with a default commencement date of 31 May 2018, the decision was taken to commence earlier than this on 31 December 2017. This decision was perplexing considering Fines Victoria was reporting that significant IT challenges were affecting its operation from the time it commenced.

This rush job led directly to Australians being punished by a system that was much, much worse than the one it replaced.

Complaints received about Fines Victoria in 2018 represented a 74 per cent increase from the number of complaints received the previous year about Civic Compliance Victoria.

All the new system added was "significant IT challenges," significant hold times, and a much lower chance of having problems resolved.

In addition to the rise in the number of complaints, Ombudsman staff observed a change in the nature of the complaints. People were expressing frustration about delays and being unable to make contact with Fines Victoria much more frequently than with the predecessor agency. There was also an increase in complexity about the administration of infringements which meant many complaints could not be resolved through a quick series of enquiries.

It's not just the software. It's also the people. The Ombudsman notes that the problems in the system were made worse by the government employees fielding complaints, who often seemed to go out of their way to push problems back on drivers.

Enforcement at this scale isn't possible without at least some automation. But the government's botched implementation shows it cared more about maximizing revenue than minimizing error. And when everything goes this wrong this quickly, it's the citizens who end up paying for both the end results of a broken system and the costs involved in fixing it.

Filed Under: australia, automated traffic enforcement, fines, mistakes, traffic enforcement, victoria

Washington Prison Management Software Setting People Free Too Early, Keeping Other People Locked Up Too Long

from the on-average,-it-works dept

All this technology is getting in the way of justice being served. For the second time in five years, Washington's Department of Corrections is dealing with issues created by its prisoner management software. Four years ago, this happened:

For three years, state Department of Corrections staff knew a software-coding error was miscalculating prison sentences and allowing inmates to be released early. On Tuesday, Gov. Jay Inslee gave the damning tally: up to 3,200 prisoners set free too soon since 2002.

For thirteen years, officials knew there was a problem with the software but did nothing about it. It wasn't until the state's governor got involved that anyone at the DOC started caring about its malfunctioning code. The code was supposedly fixed but new problems arose, affecting both sides of the jail walls.

A software problem has caused at least a dozen Washington prison inmates to be released too early — or held too long — and has sparked a review of as many as 3,500 cases.

Department of Corrections (DOC) officials are scrambling to determine whether the other inmates’ sentences were miscalculated and are still working to gauge the scope of the problem.

The previous bug miscalculated "good time" credits, resulting in thousands of premature releases. This time around, buggy code is screwing up calculations for inmates who have violated their parole. A few have benefited from the problem. But most of the cases being reviewed involve inmates who have been jailed for too long.

The previous calculation error resulted in two homicides by an inmate who was released too early. This time around, it's far more likely inmates who have served their time aren't being released. Either way, there's life and liberty on the line and the Department of Corrections is showing little sense of urgency when addressing these problems.

If there's a silver lining, it's this: with enough votes, the state's convoluted sentencing laws will be simplified, potentially making the calculation of sentences easy enough any human can do it.

[State Representative Roger] Goodman has proposed legislation, House Bill 1495, creating an 18-member task force that would review and recommend simplifications to the state’s sentencing guidelines, with a final report due by the end of 2020. He said his committee also may ask DOC leaders for a public briefing and explanation of the latest sentence-calculation problems.

Of course, this could take as long to fix as it took the state DOC to fix its original software problem. No answers would be expected for another 18 months, by which point new bugs in the DOC's software may surface and start handing out get out of jail free cards to some inmates and go directly to jail cards to others who've already served their time.

And if it's not working now, there's a good chance the DOC's software will never function properly. Part of the problem is the legislature itself, which complicates sentence calculations by adding new wrinkles with each legislative session. According to the Seattle Times, at least 60 bills in the pipeline could affect sentencing guidelines, increasing the chance of new calculation errors developing.

Software may be the only way to handle a job this complex. But those overseeing the software's deployment have shown they're not too interested in proactive maintenance of this complex system. Problems are eventually solved, years after the fact. That sort of responsiveness is unacceptable when guidelines are being constantly altered by new legislation.

Filed Under: department of corrections, inmates, mistakes, prison management, software, washington

There Was Heavy Tech Lobbying On Article 13... From The Company Hoping To Sell Everyone The Filters

from the lobbyist-influence dept

One of the key themes we've been hearing for years now concerning the EU's awful Article 13 section of the EU Copyright Directive, was that no one should pay any attention at all to the critics of Article 13, because it's all just "big tech lobbying" behind any of the criticism. In the past, we highlighted a few of these claims:

Here's Geoff Taylor from BPI:

The US tech lobby has been using its enormous reach and resources to try to whip up an alarmist campaign...

And here's Richard Ashcroft from PRS for Music:

the Internet giants... have whipped up a social media storm of misinformation about the proposed changes in order to preserve their current advantage.

And how about UK Music's Michael Dugher who really wants to blame Google for everything:

Some absolute rubbish has been written about the EU’s proposed changes on copyright rules.

Amongst the ludicrous suggestions from the likes of Google is the claim that the shake-up will mean the end of memes, remixes and other user-generated content. Some have said that it will mean ‘censorship’ and even wildly predicted it will result in the ‘death of the internet’.

This is desperate and dishonest. Whilst some of the myths are repeated by people who remain blissfully untroubled by the technical but crucially important details of the proposed EU changes, in the worst cases this propaganda is being cynically pedalled by big tech like Google’s YouTube with a huge vested and multi-million-pound interest in this battle.

However, as we wrote about back in December, an analysis that looked at the actual lobbying efforts around copyright in the EU found that it was done overwhelmingly by the legacy copyright industries, and only sparingly by the tech companies. In that post, I went through a spreadsheet looking at the lobbying of the EU Commission, and found that over 80% of the meetings were from the entertainment industry.

However, as is coming out now, there was definitely one "tech" company that was one of the most aggressive lobbyists on Article 13. However, it was lobbying in favor of it, and that's because it knew that Article 13 would lead to an artificial, but highly inflated demand for internet filters. And that's the company known for building the filtering technology behind nearly all of the non-ContentID copyright filters: Audible Magic.

Law professor Annemarie Bridy recently posted a detailed Twitter thread of Audible Magic's lobbying activities regarding Article 13. It's easy to see why the company did so, because the law, if put into effect, would be a huge, huge benefit for Audible Magic, more or less forcing nearly every internet platform of a decent size to have to purchase Audible Magic's technology. Indeed, in the run-up to Article 13, we heard directly from policymakers in the EU who would point to Audible Magic as "proof" that filtering technology was readily available for not much money and that it worked. Neither of these claims are accurate.

On the fees, Audible Magic has a public pricing page that has been frequently pointed out by supporters of Article 13, often with the claim "fees start as low as $1,000 per month." But... that's not accurate. The $1,000 only applies to "on device" databases. Hosted databases start at $2,000 per month, which is already double that... and the $2,000 per month only covers very low levels of usage. Indeed, the usage rates are so low that it's unlikely to think that any company that used Audible Magic at that rate would be making very much (if any) money at all -- meaning that relatively speaking, Audible Magic would be a huge margin killer. And the rates quickly go up from there. Indeed, on Audible Magic's pricing page, as soon as you get to a level that one might consider "sustainable" for a business, the prices become "contact us."

A few years back I spoke with one mid-sized streaming company, who told me Audible Magic was quoting them fees that were between $30,000 to $60,000 per month. An academic paper from 2017 found pricing to be slightly lower than what I had heard, but still quite expensive:

Commercial fingerprinting and filtering services, such as Audible Magic and Vobile, do not publicly release pricing. But we can guess at the ballpark: one medium-sized file hosting service reported that its license for Audible Magic filtering cost $10,000-12,000 per month in 2011 (though this provider was later able to negotiate a reduced rate based on the amount of content flagged through the system). Another estimated that Audible Magic cost its service roughly $25,000 per month. OSPs noted that the licensing fees are just the beginning. Filtering systems, several OSPs noted, are not turnkey services. They require integration with existing systems and upkeep as the OSP takes on new mediation roles between rightsholder and user (such as tracking and managing user appeals).

In other words, those things get really pricey quickly -- such that it becomes untenable for all but the largest of service providers.

And that leads us to the second part, about whether or not they work. As we've been detailing for years, the answer is clearly no. These fingerprinting technologies make both false positive and false negative errors all the time. We probably have a few examples sent our way every single day. Incredibly, the very lobbying video that Bridy points out Audible Magic created as part of its lobbying effort says that Audible Magic's technology is accurate to about 99%.

Last summer, we highlighted that Alec Muffett created a "simulator" that would look at the the error rates on such filtering technology -- and it noted that if you went with an accuracy level of 99.5% (higher than even Audible Magic claims) and ran it across 10 million pieces of content, you'd end up censoring approximately 50,000 pieces of content that were non-infringing. 50,000. And that's assuming the technology is even more accurate than even Audible Magic will claim.

And, of course, that's solely discussing the matching accuracy. It says absolutely nothing about understanding user rights -- like fair use, fair dealing, parody, etc -- none of which Audible Magic takes into account (meaning even more non-infringing works would get censored).

And, yet, as Bridy shows, Audible Magic has been lobbying hard for this:

It's not surprising that they'd lobby for such a thing. I mean, which company wouldn't lobby for a new law that would effectively require thousands of internet companies to buy your product for which there is little to no real competition (oh yeah, which almost certainly means Audible Magic would likely raise prices once the government required everyone to buy its filters).

Bridy highlights that their lobbying claims are complete bullshit as well:

Quite incredible that they highlight the "voluntary" nature of the filters while lobbying for making their technology required under law. And equally ridiculous that they claim that intermediary liability protections some how create "barriers" for online services. That, as Bridy points out, is exactly the opposite of reality. Safe harbors create clear rules that platforms understand so they know what they need to do to set up a legit platform. Removing those rules, as Article 13 does, and requiring expensive (and terrible) technology is a huge barrier, as the cost is prohibitive for most.

There's more in that thread, but as Bridy shows, Audible Magic's own presentation shows that it knows who the "winner" of Article 13 will be: Audible Magic inserting itself to become the de facto "copyright filter" layer of the internet:

So, yeah, there was some "tech" lobbying for Article 13 and its mandatory filters. It was just coming from the biggest supplier of those filters.

Filed Under: article 13, censorship, copyright, eu, eu copyright directive, filters, lobbying, mistakes
Companies: audible magic

Internet Content Moderation Isn't Politically Biased, It's Just Impossible To Do Well At Scale

from the stop-this-dumb-narrative dept

The narrative making the political rounds recently is that the big social media platforms are somehow "biased against conservatives" and deliberately trying to silence them (meanwhile, there are some in the liberal camp who are complaining that sites like Twitter have not killed off certain accounts, arguing -- incorrectly -- that they're now overcompensating in trying to not kick off angry ideologues). This has been a stupid narrative from the beginning, but the refrain on it has only been getting louder and louder, especially as Donald Trump has gone off on one of his ill-informed rants claming that "Social Media Giants are silencing millions of people." Let's be clear: this is all nonsense.

The real issue -- as we've been trying to explain for quite some time now -- is that basic content moderation at scale is nearly impossible to do well. That doesn't mean sites can't do better, but the failures are not because of some institutional bias. Will Oremus, over at Slate, has a good article up detailing why this narrative is nonsense, and he points to the episode of Radiolab we recently wrote about, that digs deep on how Facebook moderation choices happen, where you quickly begin to get a sense of why it's impossible to do it well. I would add to that a recent piece from Motherboard, accurately titled The Impossible Job: Inside Facebook’s Struggle to Moderate Two Billion People.

These all highlight a few simple facts that lots of angry people (on all sides of political debates) are having trouble grasping.

1. If you leave a platform completely unmoderated, it will fill up with junk, spam, trolling and the like, thereby decreasing its overall utility and pushing people away.
2. If you do decide to moderate, you have a set of impossible choices. So much content requires understanding context, and context may be very different, even for the same content when viewed by different people.
3. If you're going to moderate at scale, you're going to need a set of "rules" that thousands of generally low paid individuals will have to be able to put into practice, reviewing pieces of content for just a few seconds (a recent report said that Facebook reviewers were expect to review 5,000 pieces of content per day.
4. It is impossible to make rules like that that can easily be applied to all content. A significant percentage of content falls into gray areas, where it then becomes a judgment call by people in a cubicle in the middle of reviewing 5,000 pieces of content.
5. At that rate, many mistakes are made. It is collateral damage of moderation at scale.
6. People caught in the crossfire of collateral damage will rightly make a big stink about it and the social media companies will look bad.
7. Meanwhile, some of the reasonable moderation decisions will hit trolls hard (see point 1 above) and those trolls will then take to other platforms and make a huge stink about how unfair it all is, and the social media companies will look bad.

“This is the difference between having 100 million people and a few billion people on your platform,” Kate Klonick... told Motherboard. “If you moderate posts 40 million times a day, the chance of one of those wrong decisions blowing up in your face is so much higher.”

Later in the piece, Klonick again makes an important point:

“The really easy answer is outrage, and that reaction is so useless,” Klonick said. “The other easy thing is to create an evil corporate narrative, and that is also not right. I’m not letting them off the hook, but these are mind-bending problems and I think sometimes they don’t get enough credit for how hard these problems are.”

This is why I've been advocating loudly for platforms to move the moderation decisions further out to the ends of the network, rather than doing it in a centralized fashion. Let end users create their own moderation system, or adapt ones put together by third parties. But, of course, even that has problems as well.

No matter what choices are made, there are significant tradeoffs. As the Motherboard article also highlights, what seems like a "simple" rule gets hellishly complex quickly when applied to other situations, and then you've suddenly increased the "error" rate and people get angry all over again and the whole mess gets blown out of proportion again.

“There's always a balance between: do we add this exception, this nuance, this regional trade-off, and maybe incur lower accuracy and more errors,” Guy Rosen, VP of product management at Facebook, said. "[Or] do we keep it simple, but maybe not quite as nuanced in some of the edge cases? Balance that's really hard to strike at the end of the day.”

As the Oremus piece notes, the "bias" of platforms when it comes to moderation is not "liberal" or "conservative," it's Capitalist. Having a platform overrun with spam and trolls is bad for business. Hiring enough people who can adequately review content within the correct context is somewhere between insanely cost prohibitive and impossible. So the platforms muddle by with imperfect review processes. Making moderation mistakes is also bad for business, and the platforms would love to minimize them, but "mistakes" are often in the eye of the beholder as well, again reinforcing that this is an impossible task. For everyone screaming about how Alex Jones should be kicked off platforms, there's a similar number of people screaming about how awful the platforms are that do kick him off. There is no "right" way to do this, and that's what every platform struggles with.

And, if you think that these platforms are unfairly silencing "conservatives" (which is the prevailing narrative right now), it's probably because you're not paying enough attention elsewhere. Black Lives Matter and other civil rights groups have complained about "racially biased" moderation in the opposite direction, saying that minority groups are regularly silenced on these platforms. Indeed, it's not hard to find a ton of reports about black activists having content removed from social media platforms. And for all the talk of Infowars being taken off these platforms, how many people noticed that the Facebook page of the Venezuelan socialist TV station Telesur was recently taken down as well.

Yes, it's fine to point out that these platforms (mainly Facebook, Twitter and YouTube) are really bad at moderating. But, unless you're willing to actually understand the scale at play, recognize how many mistakes are going to be made (and recognize how trolls are going to go nuts over correct decisions), you're playing into a false narrative to argue that any of these platforms are "targeting" anyone. It's not true.

Filed Under: bias, content moderation, errors, filters, mistakes, scale, social media, spam, trolls
Companies: facebook, google, twitter, youtube

NSA Employees Routinely Undermined 'Non-Attributable' Web Access With Personal Web Use

from the ONE-OF-US dept

Another large batch of Snowden docs have been released by The Intercept. The new documents are part of the site's "SID (Signals Intelligence Directorate) Today" collection, a sort of interoffice newsletter featuring discussions of intelligence-gathering efforts the agency has engaged in, as well as more mundane office business.

The one discussed in this Intercept post details some careless opsec by Intelligence Community (IC) employees. Like anyone in any office anywhere, IC employees use their office computers to send personal email, shop online, and fritter away the downtime with some web surfing.

That's where they're running into problems. This SID Today document [PDF] deals with the IC's personal use of company computers -- namely, the "attribution" problem that develops when outside websites are accessed using IP addresses that can be traced back to the NSA and other IC components.

The IC uses a system called AIRGAP to provide internet access for IC employees while supposedly still preventing outsiders from tracing IP addresses back to sensitive locations. Set up in 1998 by "one of the world's largest internet providers," the system was supposed to provide non-attributable access to the outside internet world.

Unfortunately, as is detailed by the SID Today doc, the execution of AIRGAP was lacking.

One early concern about the firewall was that it funneled all internet traffic through a single IP address, meaning that if any activity on the address was revealed to be associated with U.S. spies, a broad swath of other activity could then be attributed to other U.S. spies. More IP addresses were subsequently added, but “occasionally we find that the ISP reverts to one address, or does not effectively rotate those assigned,” Speight wrote.

Speight added that the “greater security concern” was the very intelligence agents the system was designed to protect. “Despite rules and warnings to the contrary, all too frequently users will use AIRGAP for registering on web sites or for services, logging into other sites and services and even ordering personal items from on-line vendors,” Speight wrote in a classified passage. “By doing so, these users reveal information about themselves and, potentially, other users on the network. So much for ‘non-attribution.'”

It's the sort of simple carelessness that's almost unavoidable in large organizations. The NSA's effort to distance itself from its employees' internet use was thwarted by the ISP's funnel and IC employee sloppiness. As The Intercept points out, this mirrors some of the brainlessness exhibited by Russian hackers, who used a system designed to obscure their origin, but constantly undermined that protection by using the same system to log in to personal social media accounts.

The difference between the two is AIRGAP was just there to open a portal out of the IC's closed system. The Russian's system was designed to obscure the source of attacks. But the personal use of the IC's firewall/AIRGAP is still a violation of internal policy, as the document points out.

Rather than work towards preventing the unpreventable (personal web use), the IC set up another system -- OUTPARKS -- which provided more than 200 random IP addresses, all of which would be registered to an ISP, rather than the IC itself. Confusingly, the new system -- put in place in 2005 -- is also referred to as AIRGAP, primarily because IC employees are creatures of habit and referred to OUTPARKS as AIRGAP despite it being an entirely new, NSA-owned operation.

Ultimately, the document shows NSA employees are just like the rest of us: periodically bored and prone to using work computers for personal reasons.

Filed Under: covering tracks, mistakes, nsa, security