Ohio Government Asks Companies To Snitch On Employees, Gets Hit With Auto-Generated Bogus 'Tips' Instead

from the defeating-a-fraud-portal-with-fraudulent-submissions dept

Asking citizens to snitch on other citizens never seems to work out very well. The federal government has been doing it for years, maintaining "See Something, Say Something" hotlines that have mostly collected tips from people concerned about what their browner neighbors are doing. The same thing happens in the private sector. Ring's proprietary app -- Neighbors -- collects the same sort of garbage, empowering bigots to feel like they're acting on behalf of the common good.

With lockdown orders in effect and social distancing rules in place in several cities and states around the country, local governments are asking residents to pitch in with enforcement efforts by reporting those who are breaking the rules. New York City opened a tip line for reports of social distancing violators and collected a bunch of Hitler-related memes, videos of the mayor going to the gym, extended middle fingers, and dick pics instead.

The state of Ohio is asking for the same trouble. Its unemployment fraud portal is supposed to collect reports from businesses about employees of theirs that are collecting unemployment rather than coming into work. Some employees are opting out of potential infection when employers haven't shown the willingness to protect them by enforcing social distancing rules and/or providing them with personal protective equipment.

The state is now going to have to sift through a whole lot of algorithmically-generated crap to find genuine reports of work shirkers, thanks to the efforts of one anonymous coder.

The script, which began circulating on social media earlier this week, automatically fills out a “fraud reporting” form on the state of Ohio’s unemployment insurance website. State officials created the form to encourage companies to snitch on workers who are refusing to work under unsafe conditions, drawing outrage from workers and labor rights advocates. The script’s creator says the goal is to overwhelm the site with a flood of fake submissions, making it harder to process claims and thus deny people their benefits.

Flooding government websites with garbage data isn't the ideal solution but this will possibly make it more difficult for businesses to punish employees they're putting in harm's way by refusing to protect them from potential infection. The downside is this may also delay processing of legitimate claims from people who've been laid off. But if claims continue to be paid while investigations are still ongoing, it's probably a net win for employees who'd rather not roll the dice on dying while the pandemic runs its course.

The state is now aware of the scripted submissions and has deployed a new CAPTCHA that's a bit more difficult to defeat with a script. But the coder is already working on a way to bypass it so the flooding should resume momentarily. Unfortunately, there's no way to personalize submissions with dick pics or Hitler memes, but it should at least slow the roll of vindictive employers who'd rather see their employees punished than protected.

Filed Under: covid-19, lockdown, ohio, pandemic, safety, snitching, tipline, unemployment, workers

Commercial Content Moderation And Worker Wellness: Challenges & Opportunities

from the challenges-and-opportunities dept

Last week, Santa Clara University hosted a gathering of tech platform companies to discuss how they actually handle content moderation questions. Many of the participants in the event have written essays about the questions that were discussed at the event. Between last week and this week, we're publishing a bunch of these essays, including this one.

After a difficult few weeks of media attention and criticism surrounding the discovery of a spate of disturbing, exploitative videos either aimed at or featuring young children, YouTube’s CEO Susan Wojcicki released a blog post on December 4, 2017 that described the platform’s recent efforts to combat the problem. Central to the proposed plan to remedy the issue of unwanted user-generated content, or UGC, on YouTube was Wojcicki’s announcement of the large-scale hiring of additional human content moderation workers to complement those already working for Google, bringing the total number of such reviewers in Google’s employ to over 10,000¹.

Wojcicki also went on to refer to the platform’s development of its automated moderation mechanisms, artificial intelligence and machine learning as key to its plans to combat undesirable content, as it has in other cases when disturbing material was found on the platform in large amounts. Importantly, however, and as indicated by Wojcicki, the work of those thousands of human content moderators would go directly to building the artificial intelligence required to automate the moderation processes in the first place. In recent months, other major social media and UGC-reliant platforms have also offered up plans to hire new content moderators by the thousands in the wake of criticism around undesirable content proliferating within their products.

These recent public responses that invoke large-scale content moderator hires suggest that social media firms who trade in UGC still need human workers to review content, now and into the foreseeable future, whether that activity is to the end of deleting objectionable material, training AI to do it, or some combination thereof. As most anyone familiar with the production cycle of social media content would likely stipulate, these commercial content moderators – paid, professional workers who screen at scale following internal norms and policies – perform critical functions of brand protection and shield platform and user alike from harm. CCM workers are critical to social media’s operations and yet, until very recently, have often gone unknown to the vast majority of the world’s social media users.

The specifics of the policies and procedures that they follow have also been mostly inaccessible (beyond simplified public-facing community guidelines) and treated as trade secrets. There are several reasons for this, not the least of which being the very real concern on the part of platforms that having detailed information about what constitutes allowable, versus disallowed, content would give people intent on gaming the platform using its own rules ample ammunition to do so. There may be other reasons, however, including the fact that the existence of the problem of disturbing content circulating online is one that most mainstream platforms do not care to openly discuss due to the distasteful nature of the issue and the subsequent questions that could be asked about if, how, and by whom such material is addressed.

CCM work is both rote and complex. It requires engagement with repetitive tasks that workers can come to perform in a routinized way, but often under challenging productivity metrics that require a high degree of both speed and accuracy in decision-making. The job calls on sophistication that cannot currently be fully matched by machines, and so a complex array of human cognitive functions (e.g., linguistic and cultural competencies; quick recognition and application of appropriate policies; recognition of symbols or latent meanings) is needed. A further challenge to CCM workers and the platforms that engage them is the fact that the job, by its very nature, exposes workers to potentially disturbing imagery, videos and text from which mainstream platforms wish to shield their user and remove from circulation, the latter which may even be a legal dictate in many cases.

In other words, in order to make its platforms safe for users and advertisers, platforms must expose its CCM workers to the content that it considers unsuitable for anyone else. It is a paradox that illustrates a primary motivation behind the automation of moderation practices: under the status quo, CCM workers put their own sensitivity, skills and psyches on the line to catch, view it and delete material that may include images of pornography, violence against children, violence against animals, child sexual exploitation, gratuitously graphic or vulgar material, hate speech or imagery and so on. It is work that can lead to emotional difficulty for those on the job, even long after some have moved on.

To this end, industry has responded in a variety of ways. Some workplaces have offered on-site counseling services to employees. The availability of such counseling is important, particularly when CCM workers are employed as contractors who may lack health insurance plans or might find mental health services cost-prohibitive. Challenges are present, however, when cultural barriers or concerns over privacy impede workers from taking full advantage of such services.²

When it comes to CCM worker wellness, firms have been largely self-guiding. Several major industry leaders³ have come together to form the self-funded “Technology Coalition,” whose major project relates to fighting child sexual exploitation online. In addition to this key work, they have produced the “Employee Resilience Guidebook,” now in a second version, intended to support workers who are exposed to child sexual exploitation material. It includes information on mandatory reporting and legal obligations (mostly US-focused) around the encountering of said material, but also provides important information about how to support employees who can be reasonably expected to contend emotionally with the impact of their exposure. Key to the recommendations is beginning the process of building a resilient employee at the point of hiring. It also draws heavily from information from the National Center for Missing and Exploited Children (NCMEC), whose expertise in this area is built upon years of working with and supporting law enforcement personnel and their own staff.

The Employee Resilience Guidebook is a start toward the formation of industry-wide best practices, but in its current implementation it focuses narrowly on the specifics of child sexual exploitation material and is not intended to serve the broader needs of a generalist CCM worker and the range of material for which he or she may need support. Unlike members of law enforcement, who can call on their own professional identities and social capital for much-needed support, moderators often lack this layer of social structure and indeed are often unable to discuss the nature of their work due to non-disclosure agreements (NDAs) and stigma around the work they do. The relative geographic diffusion and industrial stratification of CCM work can also make it difficult for workers to find community with each other, outside of their immediate local teams, and no contracting or subcontracting firm is represented in the current makeup of the Technology Coalition, yet many CCM workers are employed through these channels. Finally, I know of no long-term publicly-available study that has established psychological baselines for or tracked CCM worker wellness over their period of employment or beyond. The Technology Coalition’s guidebook references information from a 2006 study on secondary trauma done on NCMEC staff. To be sure, the study contains important results that can be instructive in this case, but it cannot substitute for a psychological study done specifically on the CCM worker population in the current moment. Ethical and access concerns make such an endeavor complicated. Yet, as the ongoing 2016 Washington state lawsuit filed on behalf of two Microsoft content moderators suggests, there may be liability for firms and platforms that do not take sufficient measures to shield their CCM workers from damaging content whenever possible and to offer them adequate psychological support when it is not.

Although the factors described above currently serve as challenges to effectively serving the needs of CCM workers in the arena of their health and well-being, they are also clear indicators of where opportunities lie for industry and potential partners (e.g., academics; health care professionals; labor advocates) to improve upon the efficacy, well-being and work-life conditions of those who undertake the difficult and critical role of cleaning up the internet on behalf of us all. Opening the dialog about both challenges and opportunities is a key step toward this goal.

1 It is not clear what status the moderators at Google, or most other firms, hold, or if they are actually full employees of the platforms and parent companies, contractors, outsourced workers, or some combination thereof. This information was not given in the blog post and is usually not offered up by social media firms when announcing numbers of CCM workers they have on staff or when describing a new mass hire.

2 In interviews that I have conducted with CCM workers, some have reported feeling that others, including co-workers and supervisors, might negatively judge their engagement with an on-site counselor as evidence of an inability to perform requisite job tasks, so although the counselor would arrive at the worksite, these workers self-selected out of making use of the services when others would see them taking advantage of them.

3 As of late 2017, firms who participate in the Technology Coalition include: Apple; Dropbox; Facebook; GoDaddy; Google; Kik; LinkedIn; Microsoft Corporation; PayPal; Snap; Twitter; Yahoo! Inc. Oath, Adobe. More information can be found here: http://www.technologycoalition.org/coalition-projects/

Sarah Roberts, PhD, is an Assistant Professor at UCLA in the Department of Information Studies

Filed Under: content moderation, moderation workers, worker wellbeing, workers
Companies: facebook, google, twitter, youtube

Worker Fired After Calling In Sick, Then Using Facebook At Home

from the lessons-learned dept

Usually when somebody gets fired for their use of a social-networking site, it's because they say something about their coworkers or boss online. But a worker in Switzerland got fired after merely logging on to Facebook at home. The rub, though: she'd called in sick with a migraine and told her boss she couldn't use a computer and needed to lie in a darkened room. The woman claims she was accessing the site while laying in her bed on her iPhone, and also claims the company's story that a coworker noticed her updates and told her supervisor isn't true. She instead is blaming "a fictitious Facebook persona" created by the company that she became friends with on the site, alleging that its only use was to spy on her. It sort of makes you wonder why she friended the company's account, but regardless, her treatment does seem a little harsh. There's no word if the company has fired other people for playing hooky, online or off, nor is there talk of the employee's previous record, but firing somebody based on their Facebook updates seems a bit over-the-top. Would they have reacted the same way if she'd updated from a doctor's office waiting room as opposed to her home?

Filed Under: fired, social networking, workers
Companies: facebook