Again, Algorithms Suck At Determining 'Bad' Content, Often To Hilarious Degrees
from the peachy dept
A few weeks back, Mike wrote a post detailing how absolutely shitty algorithms can be at determining what is "bad" or "offensive" or otherwise "undesirable" content. While his post detailed failings in algorithms judging such weighty content as war-crime investigations versus terrorist propaganda, and Nazi hate-speech versus legitimate news reporting, the central thesis in all of this is that relying on platforms to host our speech and content when those platforms employ very, very imperfect algorithms as gatekeepers is a terrible idea. And it leads to undesirable outcomes at levels far below those of Nazis and terrorism.
Take Supper Mario Broth, for instance. SMB is a site dedicated to fun and interesting information about Nintendo and its history. It's a place that fans go to learn more weird and wonderful information about the gaming company they love. The site also has a Twitter account, which was recently flagged for posting the following tweet.
Below is a screenshot of the Tweet that mistakenly got this account flagged as containing sensitive content. As you can see, this was an error. I would never post actual sensitive content. This Tweet will be pinned until the situation is resolved. pic.twitter.com/nMnElZF9uj
— Supper Mario Broth (@MarioBrothBlog) March 31, 2018
For the sin of tweeting that image out, the site's entire account was flagged as "sensitive", which means anyone visiting the account was greeted with a warning about how filthy it is. What Twitter's systems thought was offensive about the image, which comes from another video from a costume company that works with Nintendo, is literally anyone's guess. Nobody seems to be able to figure it out. My working theory is that the Princess Peach's lips resemble too closely a more private part of the female anatomy and, when coupled with the flesh-colored face surrounding it sent Twitter's algorithm screaming "Aaah! Vagina!" leading to the flagging of the account. But this is just a guess, because although the "sensitive" flag was eventually removed, SMB never got any response or explanation from Twitter at all.
SMB went as far as to test through dummy accounts whether the image was the entire problem. It was. After posting the image several times from other accounts, each account was flagged within minutes of the posting. It's an algorithm doing this, in other words, and one which seems ill-suited to its task.
What we have here is two related problems. We have a company designed to let speakers speak employing an algorithm to flag offensive content, which it is doing very, very badly. We also have a company with a staff insufficiently capable to correct the errors of its incapable algorithm. This would be annoying in any context other than current reality, which sees rising calls for internet sites to automagically block "bad" content and do so with literally inhuman speed.
That means algorithms. But the algorithms can't do the job. And with sites erring on the side of over-blocking to avoid scrutiny from both the public and governments, that means open communication is the loser in all of this. It's hard to imagine an outcome more anathema to services like Twitter than that.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: algorithms, free speech, hate speech, moderation, propaganda
Reader Comments
Subscribe: RSS
View by: Time | Thread
In any case I think we should stop putting content removal in the hands of specific people or algorithms. I'd like to see a decentralized effort where if people enough flagged a determined content it would issue a warning that it was severely flagged but still be visible and after a very high threshold it would be enabled for review by actual humans and possibly removed. Not a perfect system but at least better than what we have now. One can dream, no?
[ link to this | view in chronology ]
Re:
Sanity self-preservation. If everything about the Stormy Daniels affair and other Donald Trump sexual misconduct allegations were allowed on TV news - from detailed descriptions to infographics to computer animation - this Presidency would resemble Call of Cthulhu.
[ link to this | view in chronology ]
Penises and Vulvas
Have you seen the new Last Week Tonight bit on CPCs? We really have genitalia issues. Big swooping raging engorged issues.
[ link to this | view in chronology ]
Re: Penises and Vulvas
THE goto tactic for all groups to attack another group is to take a smaller not really a big problem and then stigmatize it until its a huge throbbing.... er problem!
This was all brought to you by the Catholic Church over the idea that naked people go to hell and clothed people are saved.
As a Christian, I fully recognize that Christian "institutions" have caused more damage to morals and values than secular ones though they appear to be catching up so they are not outdone.
Once a group of humans institutionalize something regardless of what that something is it becomes are desired point of infection by evil people to work towards their own ends.
[ link to this | view in chronology ]
Re: Re:
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re:
*Humanity* doesn't. Western culture seems to have some issues, North American culture more than some others, but less than some others.
As for the rest of your comment, the problem is that there's a push to make these companies directly and even criminally liable for content that manages to appear on their site even after moderation. A system that would allow questionable material to appear even while moderated would probably fall foul of the laws those people are trying to push through. Similarly, this is why these systems are erring on the side of caution and flagging material is obviously not objectionable - their hand is being forced into making these thing over-zealous rather than let something through.
[ link to this | view in chronology ]
What was the tweet?
[ link to this | view in chronology ]
Re: What was the tweet?
[ link to this | view in chronology ]
Re: Re: What was the tweet?
https://kotaku.com/nintendo-fan-account-flagged-on-twitter-for-image-of-ma-1824999104
[ link to this | view in chronology ]
Re: Re: Re: What was the tweet?
[ link to this | view in chronology ]
Re: Re: Re: Re: What was the tweet?
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: What was the tweet?
I use NoScript myself, and wouldn't allow scripting from such a site just for the purpose of loading images (although I might manually edit page source in Firebug, or potentially do it less-manually with Greasemonkey, to enable myself to view a particular image) - but I haven't followed the links in question, and would have been unlikely to speak up here about the matter anyway. (Although knowing that the linked-to site does have that problem is something that's useful to me.)
[ link to this | view in chronology ]
By Twits, For Twits
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Algorithm problem???
[ link to this | view in chronology ]
I have also reported posts that are even more inflammatory and discriminating against a lot of different groups especially the NRA but others as well. FB has given me responses each time that its not hate speech and that if I don't want to see it then I should just hide it. Yet if I were to post a rant about killing anti-gun people it would be pulled nearly instantly for inciting violence etc. Their algorithm is heavily skewed. I think this more than anything is what Cruz was trying to get at. That the bias of what is allowed leans heavily to liberal view points but is exceedingly heavy handed on conservative viewpoints.
[ link to this | view in chronology ]
Re:
"That the bias of what is allowed leans heavily to liberal view points but is exceedingly heavy handed on conservative viewpoints."
Which conservative viewpoints? Honestly held beliefs about honest conservative values, or the racist, misogynistic, homophobic, etc., viewpoints often espoused by those who claim to be conservative? There's a big difference, and if someone is blocking those viewpoints it's not anti-conservative bias if all the racist, homophobic woman haters happen to be on that side.
Give me examples of non-hatred being blocked and I might agree. But, if the majority of hatred is on your "team", you may wish to examine that "team" rather than complaining about censorship. If 90% of objectionable material is in the hands of one "team", it's not unfair if they get 90% of the blocks.
[ link to this | view in chronology ]
"Protected Classes"
It's entirely off topic, though it is a problem that we have:
Large amounts of accepted hate speech is against classes who aren't yet (or aren't quite) protected, whether countercultures, disabled, obese people, or the biggest one, poor people, even though there are clear trends of discrimination against them. (Trump's new welfare reform serving as a notable recent example.)
It's a problem that we classify one form of bigotry over another rather than defining specific actions that are universally regarded as acts of hatred. As a result, society's level of awareness is always many steps behind.
[ link to this | view in chronology ]
Open Source
[ link to this | view in chronology ]