Putting Pinners First: How Pinterest Is Building Partnerships For Compassionate Content Moderation
from the rethinking-how-moderation-works dept
Last week, Santa Clara University hosted a gathering of tech platform companies to discuss how they actually handle content moderation questions. Many of the participants in the event have written essays about the questions that were discussed at the event. Last week we published five of those essays and this week we're continuing to publish more of them, including this one.
The way platforms develop content moderation rules can seem mysterious or arbitrary. At first glance, the result of this seemingly inscrutable process is varying guidelines across different platforms, with only a vague hint of an industry standard -- what might be banned on one platform seems to be allowed on another. While each platform may have nuances in the way they create meaningful content moderation rules, these teams generally seek to align with the platform's/company's purpose, and use policies and guidelines to support an overarching mission. Different platforms delivering unique value propositions to users' accounts for variations in content moderation approaches.
At Pinterest, our purpose is clear: we help people discover and do what they love by showing them ideas that are relevant, interesting, and personal. For people to feel confident and encouraged to explore new possibilities, or try new things on Pinterest, it's important that the Pinterest platform continues to prioritize an environment of safety and security. To accomplish that, a team of content policy professionals, skilled in collaborating across different technical and non-technical functions at the company, decide where we draw the lines on what we consider acceptable boundaries for content and behavior. Drawing upon the feedback of Pinterest users, and staying up to date on prevailing discourse about online content moderation, this team of dedicated content generalists brings diverse perspectives to bear upon the guidelines and processes that keep divisive, disturbing, or unsafe content off Pinterest.
We know how impactful Pinterest can be in helping people make decisions in their daily life, like what to eat or what to wear, because we hear directly from the Pinterest community. We've also heard how people use Pinterest to find resources to process illness or trauma they may have experienced. Sometimes, the content that people share during these difficult moments can be polarizing or triggering to others, and we have to strike the right balance of letting people rely on Pinterest as a tool for navigating these difficult issues, and living up to our goal of removing divisive, disturbing, or unsafe content. As a team, we have to consider the broad range of use cases for content on Pinterest. For example, important historical yet graphic images of war can be collected in the context of learning about world events, or to glorify violence. Our team takes different contextual signals into account during the review process in order to make meaningful content moderation choices that ensure a positive experience for our community. If we wish to have the impact we hope to have in people's lives, we must also take responsibility for their entire experience.
To be responsible for the online environment that our community experiences, and to be aware of how that experience connects in a concrete way to their life offline, means we cultivate the humility to realize our team's limitations. We can't claim to be experts in fields like grief counseling, eating disorder treatment, or suicide prevention -- areas that many groups and individuals have dedicated their careers to supporting -- so it's crucial that we partner with experts for the guidance, specialized skills, and knowledge that will enable us to better serve our community with respect, sensitivity, and compassion.
A couple years ago, we began reexamining our approach to one particularly difficult issue - eating disorders - to understand the way our image-heavy platform might contribute to perpetuating unhealthy stereotypes about the ideal body. We had already developed strict rules about content promoting self-harm, but wanted to ensure we were being thoughtful about content offering "thinspiration" or unhealthy diets from all over the internet. To help us navigate this complicated issue, we sought out the expertise of the National Eating Disorder Association (NEDA) to audit our approach, and understand all of the ways we might engage with people using the platform in this way.
Prior to reaching out to NEDA, we put together a list of search queries and descriptive keyword terms that we believed strongly signaled a worrying interest in self-harm behaviors. We limit the search results we show when people seek out content using these queries, and also use these terms as a guide for Pinterest's operational teams to decide if any given piece of self-harm-related content should be removed or hidden from public areas of the service. The subject matter experts at NEDA generously agreed to review our list to see if our bar for problematic terms was consistent with their expert knowledge, and they provided us with the feedback we needed to ensure we were aligned. We were relieved to hear that our list was fairly comprehensive, and that our struggle with grey area queries and terms was not unique. Since beginning that partnership with NEDA, they have developed a rich Pinterest profile to inspire people by sharing stories of recovery, content about body positivity, and tips for self-care and illness management. By maintaining a dialogue with NEDA, the Pinterest team has continued to consider and operationalize innovative features to facilitate possible early intervention on the platform. For example, we provide people seeking eating disorder content with an advisory that also links to specialized resources on NEDA's website, and supported their campaign for National Eating Disorder Awareness Week. Through another partnership and technical integration with Koko, a third party service that provides platforms with automated and peer-to-peer chat support for people in crisis, we're also able to provide people who may be engaging in self-harm behaviors with direct, in-the-moment crisis prevention.
Maintaining a safe and secure environment in which people can feel confident to try new things requires a multifaceted approach and multifaceted perspectives. Our team is well-equipped to grapple with broad online safety and content moderation issues, but we have to recognize when we might lack in-house expertise in more complex areas that require additional knowledge and sensitivity. We have much more work to do, but these types of partnerships help us adapt and grow as we continue to support people using Pinterest to discover and do the things they love.
Adelin Cai runs the Policy Team at Pinterest
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: compassion, content moderation, outside groups
Companies: pinterest
Reader Comments
Subscribe: RSS
View by: Time | Thread
Should call those asshole the facebook of adversiting but the advertising industry already bet the fuck out of everyone for the title of everdoosh!
[ link to this | view in chronology ]
Re:
A marketing scheme that makes people hate your service is a big fail.
[ link to this | view in chronology ]
Re: Re:
Good on them for curating content for people with eating disorders or who are suicidal or whatever. That's a plus I guess.
[ link to this | view in chronology ]
Re: they require a login even if I only want to view images that have popped up during a google image search
[ link to this | view in chronology ]
Re: Re:
My utter hatred for Pinterest knows few limits. I find it astonishing that they actually have the gall to try and pretend they care about self-harming behaviours, when they're easily one of the worst companies for driving any number of internet users into a state of psychotic rage.
For the benefit of any and all Pinterest employees or shareholders reading this message: please die. Seriously, please, just die. If you could arrange to do so while shitting out your bloody, cancer-ridden entrails in front of your terrified, screaming children, as an object lesson to others: that would be helpful, but it's not essential, by any means.
Removing yourself, along with as much of your hateful, malevolent, endlessly-spamming corporate nightmare from the world as you can take with you: that will be more than sufficient.
[ link to this | view in chronology ]
Re: Re: Re:
WHY is this comment not at least hidden? The person specializes in bizarre vulgarity, and explicitly states " My utter hatred for Pinterest knows few limits."
Nothing but HATE.
Oh, right. Fanboy. Their comments are never even hidden.
[ link to this | view in chronology ]
Re: Re: Re: Re: Thanks 'dirters, for saving me time here.
Read again slowly, and NOTHING BUT OFF-TOPIC HATE.
[ link to this | view in chronology ]
Re: Re: Re: Re:
If the community wants to hide the message they will.
Your message, and mine for that matter, are entirely off topic and deserve to be hidden. The previous vitriol is at least on topic if off color.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re:
THAT IS A LIE unless YOU know all about Techdirt's system. And you do not, unless YOU are an administrator and now astro-turfing. Fanboys here ALWAYS talk authoritatively but Techdirt itself NEVER responds. Let alone warns in advance.
I know from first-hand experience that comments are (to be accurate for today, were) hidden within five minutes, and on nearly dead topics.
ANOTHER LIE if you don't have real knowledge. You are just making stuff up to protect the site. -- There is surely an administrator involved. State how many clicks, then. And again, how do YOU know how it works when I don't, having asked for years?
No, they aren't / don't. There's no written guideline here and I'm asking a relevant board question. Not for the first time, but at least the hundredth, after getting MANY of my fine by common law comments hidden today.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re:
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re: Re:
Brilliant response! An "AC" with a one-liner.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re: Re: Re:
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re:
You are incompetent.
[ link to this | view in chronology ]
It might be the gentlest and most compassionate...
I don't think there exists a business that made me want to interact with their product less than pinterest.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
So, here we are, a few days later and the only kind comment about Pinterest on this page still consists exactly of the words:
... and that was by an AC who also says he or she blocks Pinterest from image search results.
Pinterest people, please know and understand: outside of your own established community, no-one cares about your moderation efforts. Almost everyone who has encountered your links hates you passionately.
How you've even managed to build a community is something of a mystery, given the utter contempt you routinely show for everyone else in your marketing methods. I can only assume your users are all either permanently stoned off their tits or masochists who enjoy being abused.
I was sure my earlier and absurdly-hostile comment would get hidden, especially with another poster encouraging it - but at the time of writing, it seems the Techdirt community dislikes Liars far more than they dislike abusive language. While my comment is undoubtedly offensive, it has the virtue of being entirely honest in its shameless, naked hatred for all things Pinterest.
It's a hatred that's well deserved. You are a parasitic troll of a company, damaging users efforts to find what we want on the internet, damaging other companies' efforts to provide useful image search services - and now damaging the truth in this debate, with your ridiculous attempt to pretend you care about anyone other than yourselves.
The best moderation effort anyone, anywhere could possibly make would be to excise you vile, corporate spongers from all of our search results, once and for all.
[ link to this | view in chronology ]
what's that, white guys?
well, that's unique! take this to the presses! apocalypse now!
pinterest - keep on keepin' on
[ link to this | view in chronology ]