Content Moderation At Scale Is Impossible: Google Play Bans Video Player App Over ASS File Extension Support
from the pain-in-the-[removed] dept
As you should know by now as readers here, content moderation at scale is impossible to do well. Examples for how and why this is so are extensive on these pages, but the crux of the matter is that scaling moderation for content across huge platforms and a variety of avenues in a way that everyone both agrees is right and that doesn't create false-positives is, well, self-evidently impossible. Not everyone agrees what should be moderated, for starters, nor does anyone trust these platforms to actually get it right. Meanwhile, some massive amount of the public does agree that these platforms should be doing something.
And that's how you get big platforms trying to automate content moderation in a way that makes everyone look incredibly dumb. Yet another example of this is the Google Play store banning a video player application over "sexual content and profanity", but just not for the reasons you're probably already imagining.
This time, the puritan robot overlords that run the Play Store briefly decided that listing support for common subtitle files is enough to get your app banned. The developer for Just (Video) Player posted their story to Hacker News, writing in the app's bug tracker, "After a tiny unrelated description update, Just Player got suspended from the Google Play Store for "Sexual Content and Profanity policy". Google finds issues with following: Full description (en_US): “* Subtitles: SRT, SSA, ASS, TTML, VTT.”"
Yes, just listing standard video player features like support for the "ASS" subtitle format was apparently enough to temporarily earn a suspension. The developer says they "immediately filed an appeal" and today, the app is back up with the ASS subtitle listing still in the description.
While I admit to appreciating that all of this happened so as to create the situation where a publication writes "with the ASS subtitle listing still in the description", this story is in every other way indicative as to why broad, automated moderation is impossible to do well. It also lays bare the fact that there is clearly zero human review or intervention in this process. Instead, the filter that looks for naughty words and content detects "ASS", can't consider the context for why that word or acronym might appear in the description, and then just outright banned the app. Sure, Google reinstated it... but all that shows is that the human intervention that occurs after the fact can and should occur before the ban.
And, while I'm typically loathe to throw a ton of props Apple's way, it's worth noting that Apple's app review process actually does what Google's doesn't.
Saying Google's app review bots "ban first and ask questions later" would actually be an improvement over the current situation, since the bots can't ask questions. The bots ban, send an automated email, and it's up to the developers to figure out why they were banned and jump through hoops to make the bots happy, often without being able to speak to a human.
Google and Apple both collect a percentage of app sales, which the companies categorize as a necessary tax that pays for the infrastructure of the store ecosystem. Apple uses this money partly to hire an army of human app reviewers, a system that Google Play developers often hold up as an example that Google should follow. Instead, Google only has this bot system—or at best, it has an extremely small team of manual reviewers—and developers frequently complain that they are at the mercy of an illogical bot, with no human to speak to even during an appeals process.
Which moves this beyond mere annoyance and into a competitive issues between the two major app stores in the mobile market. Perhaps Google is so big that it can ignore this clear deficiency in its platform moderation... but I doubt it. The more these stories crop up, the more developers will simply decide Google isn't worth the trouble.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: ass, content moderation, google play store, keyword filters, sexual content, video player
Companies: google
Reader Comments
Subscribe: RSS
View by: Time | Thread
If only there were a way for users to get apps from some other source that didn't require the blessing of Apple or Google...
[ link to this | view in chronology ]
Re:
Sideloading is a thing, and it's not difficult on Android.
But it's difficult enough that I don't expect my grandparents to figure out how to do it.
[ link to this | view in chronology ]
Re: Re:
It's difficult enough now, because the only people even considering it are those who are tech savvy enough to want it and have needs beyond the store supplied with their phone. It would be possible to make it easier, there just has to be the incentive to cater to that market.
Even so, chances are people like that aren't going to do it themselves anyway, they'll just ask whichever friend/family members they already have doing their free tech support for them to do it.
[ link to this | view in chronology ]
Re: Re: Re:
Not only that, but the majority of people wouldn't know how to find the apps to sideload them in the first place. If it's not on the pre-configured store, they wouldn't know it exists.
It's not just older folks either, I've encountered younger people who are mostly clueless about technology. To be clear, I do not consider myself an expert in any way, but I've seen people in their 20s and 30s, who grew up around this technology, but who don't know how to do simple things, like change the Desktop screen size on their computer or unpack a Rar file.
[ link to this | view in chronology ]
Re: Re: Re: Re:
That's all true, but most people have a person in their lives who they turn to for help with such things. There's nothing that can be done for the truly and wilfully clueless, they will always need help. But, you also can't pretend sideloading doesn't exist just because some people can't do it on their own. If enough desire exists for apps outside of official channels, there will be the incentive to make this as easy and foolproof as possible, then the remaining fools will just have to ask the same people they ask to do everything else for them.
[ link to this | view in chronology ]
"Perhaps Google is so big that it can ignore this clear deficiency in its platform moderation... but I doubt it"
It's doubtful because this has been a complaint against Google since day one - if you have a problem with one of their services, it's hard to get an actual human being for support. This should not be a surprise to anyone, least of all developers who have tied their income to Google's platform.
[ link to this | view in chronology ]
Thank heavens
Compilers don't produce ASSembly language anymore.
[ link to this | view in chronology ]
Re: Thank heavens
... except those used for updating the Donkey Kong code.
[ link to this | view in chronology ]
Re: Thank heavens
So that’s why my game CALL -151 was rejected by google.
[ link to this | view in chronology ]
Tourettes
I'm reminded of the South Park episode where Cartman has Tourettes Syndrome. HE gets to say what he likes, but if others say it, they get in trouble.
For the record, my audio player uses the Portable Electronic Network Interactive Sound codec. I guess I'll just go delete the codebase and start again.
E
[ link to this | view in chronology ]
Re: Tourettes
Beware, puritans spot penises everywhere: https://jira.atlassian.com/browse/JRASERVER-65811
God. fucking. shit.
[ link to this | view in chronology ]
Re: Re: Tourettes
decades later, and we're still dealing with the Penistone and Scunthorpe problem because some people get upset about some words.
We're doomed.
[ link to this | view in chronology ]
Re: Re: Re: Tourettes
Nah, the Scunthorpe Problem is because it's difficult (or takes a lot of manual intervention) to automate filters in ways that allow obviously acceptable words to get through but not open up loopholes that make the moderation ineffective.
This is about a human idiot who made that same mistake, but those have been around long before Scunthorpe existed, let alone the concept named after them. As long as the response is to correctly laugh at the idiots and not enforce the idiocy on to everyone else, we'll be fine.
[ link to this | view in chronology ]
But if your child can see an ass online they might grow up to be like Ted Cruz... now you're all for it aren't you?
[ link to this | view in chronology ]
Re: Your child traumatized by seeing Ted Cruz
In the Indy 500 an (R) after the driver's name means they're a rookie.
In the US Congress an (R) after your name means you're an ass.
E
[ link to this | view in chronology ]
I wasn't raised to view "ass" as profanity, so I don't get why anyone does. It all seems more about thought police than anything else (as in, "wow, people are really stupid aren't they").
[ link to this | view in chronology ]
Advantages and disadvantages
The nice thing about little or no human review is that your app can go live in minutes or hours, whereas it used to take days on the Apple App Store. Maybe they've improved things recently. It does have its downside though.
[ link to this | view in chronology ]
Re: Advantages and disadvantages
I was just about to say that relying solely on human review causes delays in apps (and also app updates) going live. You're also at the mercy of how the reviewer interprets the App Store rules (and, as noted, human moderators are unable to agree on everything).
[ link to this | view in chronology ]
similar experience
One of my apps (which collects no information) had an accurate privacy policy: "This app collects no information." But... Google banned it... because I did not have a valid privacy policy. I appealed it, and a person finally told me (after I jumped through their robotic hoops) that the problem was that my privacy policy did not mention "privacy." I changed it to "This app protects your privacy by collecting no information," and they accepted that.
73
[ link to this | view in chronology ]