Content Moderation Case Studies: Snapchat Disables GIPHY Integration After Racist 'Sticker' Is Discovered (2018)

from the sticker-moderation dept

Summary: Snapchat debuted to immediate success a decade ago, drawing in millions of users with its playful take on instant messaging that combined photos and short videos with a large selection of filters and "stickers." Stickers are graphics that can be applied to messages, allowing users to punch up their presentations (so to speak).

Snapchat’s innovations in the messaging space proved incredibly popular, moving Snapchat from upstart to major player in a few short years. It also created more headaches for moderators as sent messages soared past millions per day to billions.

Continuing its expansion of user options, Snapchat announced its integration with Giphy, a large online repository of GIFs, in February 2018. This gave users access to Giphy's library of images to use as stickers in messages.

But the addition of thousands of images to billions of messages quickly resulted in an unforeseen problem. In early March of 2018, Snapchat users reported a search of the GIPHY image database for the word "crime" surfaced a racist sticker, as reported by Josh Constine for TechCrunch:

“We first reported Instagram was building a GIPHY integration back in January before it launched a week later, with Snapchat adding a similar feature in February. But it wasn’t long before things went wrong. First spotted by a user in the U.K. around March 8th, the GIF included a racial slur.” — Josh Constine, TechCrunch

Both platforms immediately pulled the plug on the integration while they sorted things out with GIPHY.

Company Considerations:

• What measures can be put in place to prevent moderation problems from moving from one platform to another during cross-platform integration?
• What steps should be taken prior to launch to integrate moderation efforts between platforms? 
• What can "upline" content providers do to ensure content moving from their platforms to others meets the content standards of the "downline" platforms? 

Issue Considerations:

• What procedures aid in facilitating cross-platform moderation? 
• Which party should have final say on moderation efforts, the content provider or the content user?

Resolution: Instagram was the first to reinstate its connection with GIPHY, promising to use more moderators to examine incoming content from the image site:

“We’ve been in close contact with GIPHY throughout this process and we’re confident that they have put measures in place to ensure that Instagram users have a good experience” an Instagram spokesperson told TechCrunch.

GIPHY offered its own apology for the racist image, blaming the slipup on a bug in its filters. Here's what GIPHY's spokesperson told Gizmodo:

After investigation of the incident, this sticker was available due to a bug in our content moderation filters specifically affecting GIF stickers.

We have fixed the bug and have re-moderated all of the GIF stickers in our library.

The GIPHY staff is also further reviewing every GIF sticker by hand and should be finished shortly.

Snapchat was the last to reinstate its connection to GIPHY, stating it was working directly with the site to revamp both moderation systems to ensure offensive content would be prevented from being uploaded to GIPHY and/or making the leap to connected social media services.

Originally published to the Trust & Safety Foundation website.

Filed Under: 3rd parties, bigotry, integration, racisms, stickers
Companies: giphy, snapchat