Content Moderation Case Study: Profanity Filter Causes Problems At Paleontology Conference (October 2020)
from the f-that dept
Summary: With the COVID pandemic still in full force, the Society of Vertebrate Paleontology moved its annual meeting online. The event was due to run for an entire week, but early issues caused attendees and moderators to question the contents of the pre-packaged content filter provided by Convey Services, which operated the virtual meeting software.
The effort to ensure followup Q&A sessions would be free of profanity and other disruptiveness went awry when terms commonly used by paleontologists got caught in the software's filter. While words like "pubic," "bone," or "hell" might be appropriately blocked elsewhere, the blocklisting of these words disrupted the conference the software was supposed to keep from being disrupted.
“I would hope that actual swears or slurs would be censored, since paleontology is not a field that's immune to racist/sexist jerks,” noted Brigid Christison, a masters' student in biology at Carleton University, in an email.
[...]
“Words like ‘bone,’ ‘pubic,’ and ‘stream’ are frankly ridiculous to ban in a field where we regularly find pubic bones in streams,” Christison said.
More problematically, attendees noted the software blocked "Wang" but left "Johnson" intact, despite both being sexual slang. This appeared to indicate some bias on the part of Convey's blocklist creators -- a bias that effectively erased a surname belonging to more than 90 million Chinese citizens.
Decisions to be made by Convey:
- Are generic ban lists effective enough in most situations to justify less content-specific alterations during more specialized use?
- Given the global reach of the internet, is it still acceptable to block common surnames that can also be deployed as sexual slang?
- Does providing separate keyword sets for different uses reduce the profitability of software sales and licenses?
- Do word ban lists raise many issues in more general use or are problems noticed more often when the software is deployed by entities that deal with specialized subject matter?
- Does allowing licensees to alter keywords as needed eliminate some of the problems caused by non-specific ban lists?
- Is shifting the cost of moderation to customers a solid business decision?
Originally posted to the Trust & Safety Foundation website.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: blocklist, content moderation, paleontology, profanity filter
Reader Comments
Subscribe: RSS
View by: Time | Thread
Hope this continues to happen
I find it funny that the ban list was considered racist or "biased" too. This is what you get when you try to appease everyone by censoring everything that anyone could or might find "offensive". It will get to the point that even breathing would be considered to someone.
[ link to this | view in chronology ]
Scunthorp is only the beginning of your troubles.
Go Gamecocks!
[ link to this | view in chronology ]
This idea that blocking words with certain meanings has been practiced by the Chinese authorities for a good while now.
Know what? They aren't very successful at it either since language is very malleable and flexible.
Words and phrases that have no relation to the banned words pop up to replace the banned ones. Everyone knows what those phrases and words are meaning but they are not banned until the authorities catch on. Then new ones come out as substitutes.
It's just a new wack-a-mole. Unending in trying to prevent people from expressing what they mean. It does far more damage than it does good results as demonstrated by the article. Collateral damage is unending and there always seems to be a work around.
[ link to this | view in chronology ]
It makes you wonder
What goes on at real life palaeontology conferences that makes them think they need a filter online?
[ link to this | view in chronology ]
Re: It makes you wonder
To take your comment seriously for a moment, any time you get a place where students and lecturers and researchers (of any discipline, let alone palaeontology) go to hang out away from home, there will be the inevitable consumption of alcohol and subsequent high jinks. Sometimes even in conference sessions.
As noted by Brigid Christison in the article, palaeontology is not immune from racism, sexism, ageism, or any other -ism that can be a source of insult, offence or downright bullying or assault. These need not even be associated with alcohol.
Therefore, it is not unreasonable for the organisers to be cautious about dealing with these kinds of things before the fact.
That they got it a bit wrong is sad, but a consequence of believing that the computer can be the arbiter without a lot of contextual assistance.
[ link to this | view in chronology ]
Um, wot?
How about this:
1) All attendees are presented with a Code of Conduct (CoC) beforehand that they must agree to as a condition of participation.
2) Attendees can flag any posts/comments they feel are inappropriate.
3) A moderator group reviews the flags and determines if the content violates the CoC. If so, they then delete the comment and either warn the violator or ban them outright based upon the severity.
[ link to this | view in chronology ]