Content Moderation Case Study: Bumble Shuts Down Sharon Stone's Account, Not Believing It's Really Her (2019)

from the bees-and-catfish dept

Summary: Almost any platform that allows users to create accounts eventually has to deal with questions of identity and impersonation. Many platforms set up systems like “verified” or “trusted” users for certain recognizable accounts. Others focus on real name policies, or trying to verify all users. But services often discover challenges that come with celebrity users and verification.

While it’s one thing to do verified accounts on platforms like Twitter, Facebook, or Instagram that are often used for promotion and connection, dating site verification is a bit different and more complicated. Setting up fake personas on dating sites to lure people into misleading relationships (for a wide variety of reasons) is so common that it led to the creation of a whole new term: catfishing. Many dating sites now take user verification quite seriously, not just to avoid catfishing issues, but for the safety and protection of their userbase -- who, by definition, are usually trying to meet someone new with the hope of getting together in person.

Bumble is a popular dating app which was built up around the premise of being safer, and more responsive to the needs of female daters. The site includes a verification feature that requests the user upload selfie poses that match poses in photos sent to the user -- which are then reviewed by a team member. The idea is that if a user were faking images by pulling them from online profiles or generating them via AI, it’s much harder to match the pose.

Apparently, however, this form of verification ran into a problem when the actress Sharon Stone decided to use Bumble to meet potential dates. Users who matched with her, perhaps understandably, had difficulty believing that a famous Hollywood star would be using a dating app like Bumble, and they reported the account. Staff reviewers at Bumble were (again, reasonably) equally suspicious of the account, leading them to suspend it.

Bumble quickly restored the account, and did so in a good natured way, wishing her luck in “finding your honey.”

Decisions to be made by Bumble:

• What systems do you use to verify users are who they say they are?

• How much weight should be given to user reports that people they matched with are not real?

• How do you handle celebrities, whose accounts people may not believe are legitimate?

• What appeals process should there be for blocked accounts that were deemed to be fake?

Questions and policy implications to consider:

• On dating apps in particular, user safety is key, so should sites default towards overblocking, rather than being hesitant?

• Are there other forms of verification that would alleviate problems similar to the one Stone faced here?

• Stone was able to get her account reinstated quickly because of her fame; does the existing appeals process work as well for users who don’t have that pull?

Resolution: Bumble was pretty quick to restore Stone’s account after she tweeted about it, and major news organizations picked up the story. A few months later, Stone admitted that she suspected that she was reported by men who were upset she had turned them down on the platform.

“I think that I said no to a couple of people that thought that it would be a nice way to be not-so-kind back,” she explained. “I think some people don’t like to hear, ‘No, no I don’t want to go out with you.'”

She also noted that she has “made some nice friends” on the site.

In the meantime, some have argued that Bumble purposely chose to block Stone in order to generate publicity. Of course, this would only work if the company knew that Stone would complain about the block publicly, which certainly was not guaranteed.

Originally posted to the Trust & Safety Foundation website.

Filed Under: content moderation, dating, identity, sharon stone
Companies: bumble