Google CEO Admits That It's Impossible To Moderate YouTube Perfectly; CNBC Blasts Him
from the wait,-but-why? dept
Over the weekend, Google CEO Sundar Pichai gave an interview to CNN in which he admitted to exactly what we've been screaming over and over again for a few years now: it's literally impossible to do content moderation at scale perfectly. This is for a variety of reasons: first off, no one agrees what is the "correct" level of moderation. Ask 100 people and you will likely get 100 different answers (I know this, because we did this). What many people think must be mostly "black and white" choices actually has a tremendous amount of gray. Second, even if there were clear and easy choices to make (which there are not), at the scale of most major platforms, even a tiny error rate (of either false positives or false negatives) will still be a very large absolute number of mistakes.
So Pichai's comments to CNN shouldn't be seen as controversial, so much as they are explaining how large numbers work:
"It's one of those things in which let's say we are getting it right over 99% of the time. You'll still be able to find examples. Our goal is to take that to a very, very small percentage, well below 1%," he added.
This shouldn't be that complex. YouTube's most recent stats say that over 500 hours of content are uploaded to YouTube every minute. Assuming, conservatively, that the average YouTube video is 5 minutes (Comscore recently put the number at 4.4 minutes per video) that means around 6,000 videos uploaded every minute. That means about 8.6 million videos per day. And somewhere in the range of 250 million new videos in a month. Now, let's say that Google is actually 99.99% "accurate" (again, a non-existent and impossible standard) in its content moderation efforts. That would still mean ~26,000 "mistakes" in a month. And, I'm sure, eventually some people could come along and find 100 to 200 of those mistakes and make a big story out of how "bad" Google/YouTube are at moderating. But, the issue is not so much the quality of moderation, but the large numbers.
Anyway, that all seems fairly straightforward, but of course, because it's Google, nothing is straightforward, and CNBC decided to take this story and spin it hyperbolicly as Google CEO Sundar Pichai: YouTube is too big to fix. That, of course, is not what he's saying at all. But, of course, it's already being picked up on by various folks to prove that Google is obviously too big and needs to be broken up.
Of course, what no one will actually discuss is how you would solve this problem of the law of large numbers. You can break up Google, sure, but unless you think that consumers will suddenly shift so that not too many of them use any particular video platform, whatever leading video platforms there are will always have this general challenge. The issue is not that YouTube is "too big to fix," but simply that any platform with that much content is going to make some moderation mistakes -- and, with so much content, in absolute terms, even if the moderation efforts are pretty "accurate" you'll still find a ton of those mistakes.
I've long argued that a better solution is for these companies to open up their platforms to allow user empowerment and competition at the filtering level, so that various 3rd parties could effectively "compete" to see who's better at moderating (and to allow end users to opt-in to what kind of moderation they want), but that's got nothing to do with a platform being "too big" or needing "fixing." It's a recognition that -- as stated at the outset -- there is no "right" way to moderate content, and no one will agree on what's proper. In such a world, having a single standard will never make sense, so we might as well have many competing ones. But it's hard to see how that's a problem of being "too big."
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: content moderation, content moderation at scale, errors, false negatives, false positives, large numbers, sundar pichai
Companies: google, youtube
Reader Comments
The First Word
“By Comparison
For what it's worth, according to the numbers at Wikipedia (https://en.wikipedia.org/wiki/Comparison_of_video_hosting_services) on streaming video sites (total number of videos, probably out of date by about a year), and assuming a 99.99% success rate in moderating videos:
US
Break.com : 2.9 missed videos
Flickr : 53.2 missed videos
Godtube : 22.3 missed videos
Internet Archive : 456.7 missed videos
LiveLeak : 115 missed videos
Metacafe : 21.3 missed videos
Vimeo : 3930 missed videos
YouTube : 340,000 missed videos
Non-US
Dailymotion (FR) : 9420 missed videos
EngageMedia (DE) : 0.8 missed videos
Globo Video (BR) : 271 missed videos
Niconico (JP) : 2650 missed videos
QQ Video (CN) : 1350 missed videos
Rutube (RU) : 384 missed videos
SAPO Vídeos (PO) : 88.3 missed videos
Tudou (CN) : 105 missed videos
tune.pk (PK) : 341 missed videos
Youku (CN) : 802 missed videos
Google is far from the only one with a content moderation problem. They're just the largest. I guess it's time to outlaw Vimeo, Flickr and the Internet Archive now. But then again, going after smaller players doesn't get you votes among your decidedly angry, morally panicked, and underinformed constituents.
ps. Purposefully not including Pornhub here, because, well, that would represent a 100% miss rate in Congress' virgin eyes.
Subscribe: RSS
View by: Time | Thread
So they break Google up...
...and what do we get. A separate search engine company. A separate video platform company. A separate email company. A separate whatever else Google did company.
Assuming YouTube gets to keep their name and users continue to upload at the same rate, how does that fix the problem of moderating YouTube?
So what other excuses can the haters come up with to break up Google, or for that matter any of the other large tech companies (excluding telecom and cable and other ISP companies, we have plenty of excuses to break them up).
[ link to this | view in thread ]
“But what about no moderation for any legally protected speech?” asks a “freeze peach” asshole.
[ link to this | view in thread ]
I really really wish that it was required of every person screaming its easy, just tech it, etc. was forced to provide the magic solution to the rest of us.
We don't live in a tv show (despite the comic relief we put into office) & the real world doesn't work this way.
Australian leadership demanded Math change itself to comply with a law demanding it behave in an impossible way. When called out they doubled down that it had to do that.
The soundbite is better when you just pretend its possible & they are just refusing to fix it.
For every talking head demanding Google can easily accomplish policing YT, Google should run an ad demanding the talking head turn over the magic code they must have & are hiding.
Google is just a handy whipping boy to blame for everything, they can't do that much to Google so they just keep piling on with this stupid shit.
I await the day Google decides to flex the 'fine fuck you' response & takes the ball away for a week.
You told us it was possible, but offer nothing but demands we do the impossible so you can score political points. We decided to take some of those points off the board, explain to your constituents how you are responsible for us taking a vacation.
[ link to this | view in thread ]
Admits? It's cheap tactic trying to dodge responsiblity.
I admit you're a corporatist. And you've admitted here that you're a Google shill, so it's NO wonder that you defend it here as ever:
https://copia.is/wp-content/uploads/2015/06/sponsors.png
Now, I'm not going to re-hash the familiar arguments -- you're welcome -- so all remains is for "Gary" to spam this piece with his usual shrieking. -- Heck, I'll spare him the trouble:
Maybe copyright is the real problem, and ownership culture that keeps our culture locked up.
https://www.techdirt.com/articles/20190618/08315742419/self-described-feminist-loses-lawsuit-aga inst-twitter-banning-her-account.shtml
Heh, heh. That one makes me giggle because "Gary" can't see that our culture IS based on ownership, so he's only made a silly self-referring truism. That appears to be the limit of his creative ability.
See that you've gone on shrieking at me there, "Gary", so here's a new "meme": Techdirt clown shrieks at web-page.
[ link to this | view in thread ]
The real problem is not that YouTube is too big to fix, but rather that the Human race is too large to moderate.
[ link to this | view in thread ]
Re: Admits? It's cheap tactic trying to dodge responsiblity.
That word salad needs some dressing.
[ link to this | view in thread ]
Re:
"the Human race is too large to moderate"
Another fact many refuse to acknowledge.
[ link to this | view in thread ]
Re: Re: Admits? It's cheap tactic trying to dodge responsiblity.
Salad? I thought it smelled of bull
[ link to this | view in thread ]
I don,t think google could even find 1000,s of people to moderate
or check every video uploaded to youtube.
its users are different ages , from different country,s ,
different cultures and people of different religions have different opinions .
A british person making a crude joke or using certain words might be judged as racist or sexist or ageist in some countrys .
Its only in the last 10 years that we had the technology and broadband
speed and device whereas millions of people could watch videos
online and upload their own videos .
Services like twitch have different modes , one can start streaming ,
if you get 1000,s of viewers you can become a partner and recieve ad revenue ,
ads will be shown in your videos .
If you break the rules of twitch you can be banned from the service .
[ link to this | view in thread ]
Re:
Last quarter it was Facebook. This quarter it's Google. Sprinkle in a bit of Twitter-hate here and there for flavor.
What tech will the luddites hate next quarter?
[ link to this | view in thread ]
Re: Re: Admits? It's cheap tactic trying to dodge responsiblity.
He's given up on trying to even assert facts not in evidence and fallen back on purely personal attacks and slander lacking any real substance. He's trying to get a rise out of anyone to bolster his sense of self-importance. For some reason he thinks that people calling him a liar, idiot and whatever else while destroying his baseless arguments somehow means he's a real boy.
Just ignore the troll, flag his posts and move on.
[ link to this | view in thread ]
By Comparison
For what it's worth, according to the numbers at Wikipedia (https://en.wikipedia.org/wiki/Comparison_of_video_hosting_services) on streaming video sites (total number of videos, probably out of date by about a year), and assuming a 99.99% success rate in moderating videos:
US
Break.com : 2.9 missed videos
Flickr : 53.2 missed videos
Godtube : 22.3 missed videos
Internet Archive : 456.7 missed videos
LiveLeak : 115 missed videos
Metacafe : 21.3 missed videos
Vimeo : 3930 missed videos
YouTube : 340,000 missed videos
Non-US
Dailymotion (FR) : 9420 missed videos
EngageMedia (DE) : 0.8 missed videos
Globo Video (BR) : 271 missed videos
Niconico (JP) : 2650 missed videos
QQ Video (CN) : 1350 missed videos
Rutube (RU) : 384 missed videos
SAPO Vídeos (PO) : 88.3 missed videos
Tudou (CN) : 105 missed videos
tune.pk (PK) : 341 missed videos
Youku (CN) : 802 missed videos
Google is far from the only one with a content moderation problem. They're just the largest. I guess it's time to outlaw Vimeo, Flickr and the Internet Archive now. But then again, going after smaller players doesn't get you votes among your decidedly angry, morally panicked, and underinformed constituents.
ps. Purposefully not including Pornhub here, because, well, that would represent a 100% miss rate in Congress' virgin eyes.
[ link to this | view in thread ]
Upvote for third party content moderation.
Great idea.
Let's get it published. The feeding frenzy would be hilarious if bad and wonderful if good.
[ link to this | view in thread ]
Break up the company into what?
Wait, are people really saying google should be broken up because it is too big? That would mean Youtube would be spun off as its own entity. They would still have the exact same issues of scale and fewer resources to deal with it. What reality do these people live in?
[ link to this | view in thread ]
Yes, you do.
[ link to this | view in thread ]
Re:
Which just hilites another issue: the smaller the company, the worse the moderation will be. Small companies simply don't have the resources bigger companies have. If google can only be 99% accurate after pouring tens of millions of dollars into this, how accurate will a company be that can only spare thousands of dollar? 90%? 50%?
[ link to this | view in thread ]
Re:
It's almost like I can hear the spittle hitting his screen.
[ link to this | view in thread ]
Re: By Comparison
It would have been interesting to see the cost of moderation for each video posted per site.
[ link to this | view in thread ]
Re: Break up the company into what?
In the nerd-harder reality, where you just say those magic words and it'll solve all the problems.
[ link to this | view in thread ]
Re: Re: Break up the company into what?
Every problem is easy to solve when you don't have to do it.
[ link to this | view in thread ]
Re: Blues cheap tactic trying to dodge responsiblity.
“Techdirt clown shrieks at web-page.”
That should be your motto.
[ link to this | view in thread ]
Re: By Comparison
You know, we need more comments like this. Ones that provide evidence to back their claim.
[ link to this | view in thread ]
Let’s Summarize This in One Situation
Google CEO: Makes a statement about moderation that actually makes sense
CNBC and Others Who Prefer Screwing People Over: Ready The Canons!
[ link to this | view in thread ]
Re: Break up the company into what?
i think they want YT broken up into smaller... tubes. Or something.
[ link to this | view in thread ]
So after all this..
So basically a business news organization and others as well...who has been trying to convince the public that when the loon president calls what they do false lies and inaccurate....TURNS AROUND...
And goes to a another company and ask “why can’t u do it righ brah?
I am so ##### done my turkey is burning in oven.
[ link to this | view in thread ]
Re: Break up the company into what?
Let this sink in...
YouTube gets broken...
A okey
CNBC CNN etc gets the same treatment somewhere..
Attack on first amendment...
I think these people are not what they say they are..
[ link to this | view in thread ]
Re: By Comparison
Internet Archive?
I thought the whole point of an archive was to preserve what has happened, why should it remove content?
Maybe it would be prudent to flag it as not available for whatever reason but deleting an archive because someone does not like it - I suppose it should be expected considering the crapfest goin on.
[ link to this | view in thread ]
Re: Re: Break up the company into what?
It's not a big truck. It's a series of tubes.
[ link to this | view in thread ]
Re: So they break Google up...
No you break up Youtube into 50. One for each State. The rest of the World "doesn't matter because we are Americans".
Each State can then tax Youtube out of existence with the brides paid to politicians and elected officials. No more copyright breaches.
[ link to this | view in thread ]
Re: Break up the company into what?
I suspect that the people calling for this would be the first to run the fuck away if Pichai shrugged and said, "You're right, we're too big. I'm leaving you to break it up and solve the problem I couldn't."
Yet for some dumbass reason, banks became impossibly big to the point where they caused a global economic crisis over a decade ago, and they were entitled to more golden parachutes. This shit gets you fucking government bailouts, but because alt-right Nazis got their jimmies rustled it's suddenly blood on the streets.
[ link to this | view in thread ]
Re: By Comparison
There's another major issue here: what about false positives? Even with a 99.99% success rate, YouTube will still take down 340,000 videos that don't violate any laws, nor their terms and conditions, effectively by accident, each year.
Yikes.
[ link to this | view in thread ]
Re: Re: By Comparison
Ask Google who went a wee bit insane and deleted evidence of war crimes & historical films that teachers were sharing to give their students a better view of history.
You throw in one guy with a funny mustache looking for kyle, have you seen kyle? he's about this tall and its obviously propaganda we need to delete.
History is written by the victors, but the records can be deleted if it might offend someone.
[ link to this | view in thread ]
Never mind Youtube
According to the President of the United States of America, a large portion of the content produced by trained, professional journalists and curated by even more professional editors in Chief is fake news.
Regardless of who is right here - if the US government and a body of highly trained professionals can't agree on what is right and wrong - how can anybody expect to curate an amount of content several orders of magnitude larger than what small groups of journalists produce?
[ link to this | view in thread ]
Re: Never mind Youtube
You'd be surprised at the amount of determination you can have in the ability for things to get done, when not only is all the hard work performed by somebody else, but you also get all the credit when things go right, and none of the blame when things inevitably fuck up.
[ link to this | view in thread ]
I Can Fix YouTube Kids Content Problem
Here's how to get perfect, scaled content moderation on YouTube (at least, for the kids)
Step One: Link the YTK account to the parent's YouTube account
Step Two: Before a video can show up in the YTK account, the parent account must watch it all the way through and approve it for that YTK account.
Step Three: If their kid sees anything inappropriate, it's the parent's own fault.
It scales perfectly, because, as far as I can tell, people generally have parents.
[ link to this | view in thread ]
Re: I Can Fix YouTube Kids Content Problem
I see one flaw in your proposal, it requires biological parents to be real parents.
[ link to this | view in thread ]
What goes around comes around Comcast Xfinity Universal NBC, MSNBC, CNBC, USA Network, Syfy, NBCSN, E!, DreamWorks Animation, Comcast SportsNet, as well as Golf Channel and NBCSN.
[ link to this | view in thread ]
Re: So they break Google up...
how does that fix the problem of moderating YouTube?
It would mean that YouTube couldn't lean on the combine tech of Alphabet to develop copyright filters. I'm sure that it would be a sharp business decision to drop the whole Content ID thing at that point since it only costs them money and pageviews.
[ link to this | view in thread ]
So CNBC is feeding off the same GOP supplied carcass as WSJ and Fox News. The GOP won't be satisfied until their bigotry, misogyny, homophobia, xenophobia, antisemitism, etc. can go unchecked everywhere with no recriminations. What's next? Concentration Camps for techies?
[ link to this | view in thread ]
Re: Break up the company into what?
They live in the demagogue sphere where facts don't matter - only what serves their interests and sounds good. Reflection breaks their illusions and they hate it.
[ link to this | view in thread ]
Re: Re: I Can Fix YouTube Kids Content Problem
Your solution to a "moral panic of the week" problem is to make parents watch every crappy Let's Play and pop culture analysis video their kids could consume on a daily basis?
Good luck with that.
[ link to this | view in thread ]
Re: Re: Re: I Can Fix YouTube Kids Content Problem
Actually, the true underlying solution is for parents to take responsibility for what their kids are watching.
Still a monumental task to ask of some parents, but, speaking as a parent, it comes with the territory.
[ link to this | view in thread ]
A glaring aspect that is constantly overlooked or skimmed over in passing is that the filteting IS NOT a binary equation. There is this large gray area that is causing issues. That is ignoring all the bad actors on this issue for the moment.
I'll pull an example from past experience here on the start of a solution.
In pharmaceutical small volume injectables there is a quality issue of carmelized or burned particulates occuring when a glass ampule is sealed.
These require 100% inspection. The manual methode is to have each inspected 3 times and any failure is segregated and reinspected again to eliminate any false positives.
The automated system is set up based on the original physical inspections at the highest acceptance limit and failures are given a second pass to eliminate false positives. All rejects go through a third time but with a lower threshold and what passes is then physically inspected and sgregated for latter disposition.
That last bit is all about cost benefit analysis and depends on the numbers and the rusks involved.
If automated filtering was performed the same way then once something passes it should be immune from further challenges and confirmed failures end up black listed. This way we only need to expend energy on the grey zone.
[ link to this | view in thread ]
Re:
Define filter parameters for content, other than fuzzy matches against know bad content.
[ link to this | view in thread ]
out_of_the_blue just detests it when due process is enforced.
[ link to this | view in thread ]
Re: Upvote for third party content moderation.
I think there's definitely a place for this, and it's something I strongly support, but I don't believe it's a complete solution. Part of the problem that YouTube and other sites are struggling to address is control of propaganda, hate speech, etc. For one example, third party filtering might be able to ensure that I don't see anti-vax videos that are resulting in children being sickened, but it does nothing to address the anti-vax movement using YouTube to spread their messages to those who are receptive to it.
[ link to this | view in thread ]
Ban content moderation
The real problem is not that YouTube is too big to fix, but rather that the Human race is too large to moderate.
—Anonymous Coward
I agree. The examples in China, Russa, ... If using circumlocution, alternate ids, etc. are some of the reasons that the task is not possible! If a human can't do it a computer can't either!
3rd party moderation might prove feasible but it seems unlikely. Some of these companies are simply do large that they have become defacto public platforms.
[ link to this | view in thread ]
Google CEO Admits
[ link to this | view in thread ]
Re: By Comparison
Admittedly, this is a highly unscientific analysis. It leaves out way too many variables, like upload rates, etc. Break.com only hosts around 29,000 videos, and has collected that number over the course of 20 years. At an average clip of just under 4 videos per day, one would think that any moderation they needed to do could be easily accomplished by a single person. I would think that their false positive and false negative rates would be much lower than the average.
And if I remember correctly, Internet Archive also hosts Wayback. Who knows what material might end up in there.
[ link to this | view in thread ]
Someone somewhere no matter what you say or do or post will be offended
My answer GO FUCK YOUR SELF
[ link to this | view in thread ]
The End is here
Techdirt has taken a knee to the overlords as
a post has been sent to moderators before being posted
So Sad So Sad
[ link to this | view in thread ]
Re: Re: By Comparison
Internet Archive is intended to archive public domain and CC licensed videos, but occasionally copyrighted stuff gets through. If they don't have the licence to archive a copyrighted video, it's removed.
It's different for things like the Wayback Machine where they archive the state of sites that may be copyrighted, but in terms of storing videos they have to obey the law.
[ link to this | view in thread ]
Re: I Can Fix YouTube Kids Content Problem
My parents are dead, you insensitive clod!
[ link to this | view in thread ]
A B C D Let's sign you up with the best service to the Tee.
Our specialists can sign you and provide you with the best free equipment and rate that you can get!
No need to think twice, take advantage of our free offerings and receive competitive processing rates.
You already have merchant processing? that's totally fine we can perform a rate preview to ensure that
you receive the best rates available.
What are you waiting for! Call us now to schedule an appointments to lower your rates
Call Now! (800) 319-2274
[ link to this | view in thread ]
Re:
Testify, my man, testify!
[ link to this | view in thread ]
Hm... thought Google was already 'fixing' this issue...
... given what was shown in the Project Veritas Video - https://www.youtube.com/watch?v=re9Xp6cdkro - then again, it will be interesting to see what Mike and crew have to say as well.
[ link to this | view in thread ]
The Last Word
“The real problem is not that YouTube is too big to fix, but rather that the Human race is too large to moderate.