ID.me Doesn't Have Enough Humans To Backstop Its AI, Allowed A Guy In A Bad Wig To Illegally Obtain $900,000 In Benefits
from the congrats-on-the-windfall,-McLovin dept
ID.me -- the facial recognition company that has managed to snag several lucrative contracts -- has gotten the brushback from perhaps its most lucrative government partner, the IRS. ID.me promised government agencies better control over distributions of unemployment benefits and other payments to the public, citing its own (unexamined) prowess at recognizing faces as well as an astounding claim that governments have been duped out of $400 billion in unemployment benefits by fraudsters -- a claim it has yet to back up with actual evidence.
That the pitch worked so well isn't a surprise. After all, governments hate to give money to taxpayers and most governments have deficits they'd like to trim down. Anyone promising millions in savings is bound to be given a second, third, or fourth chance even after it's become obvious claims about fraud are, at best, dubious, and that the company can't really do the job it promised to do: eliminate fraud.
Misspending tax dollars is a national pastime. The bizarre embrace of ID.me is no exception. The IRS may have walked back its reliance on ID.me for identity verification, but problems persist. States are still relying on ID.me, even if the feds aren't. And ID.me doesn't seem to have the personnel on hand to backstop questionable calls by its facial recognition tech, as Corin Faife reports for The Verge.
Internal documents and former ID.me employees say the company was beset by disorganization and staffing shortages throughout 2021, as shortcomings in the automated systems created tensions among the company’s workforce, particularly the human verification workers who have to step in when the algorithms fail. Even now, the company plays a central role in how claimants access benefits across the United States — working on behalf of 27 state-level uninsurance employment programs to verify applicants — and the underlying issues are far from settled.
Current and former employees who spoke to The Verge paint a picture of a company described as being in “permanent crisis mode,” changing policies rapidly to keep up with fluctuating demand for its services and fight a slew of negative press. In particular, they say a lack of human review capacity has been a chokepoint for the company, leading to stress, pressure, and a failure to meet quality standards.
This verifies accusations raised earlier by other critics of ID.me -- critics who were forced to become users of faulty systems due to several states making ID.me the barrier between claimants and their benefits. Those locked out of their benefits complained the company offered few options for review of their supplied info. ID.me claimed it was performing reviews on the regular, but social media comments suggested this simply wasn't true. Actual humans were nearly impossible to reach. This report confirms what was suspected: the company simply did not have enough humans employed to deal with the problems generated by its verification processes.
Claimants were given no option but to put all their biometric eggs into one malicious hacker-tempting basket owned and operated by ID.me. When glitches separated people from their payments, the company's CEO blamed users for not being better at using an entirely new verification system. When these problems persisted, the CEO claimed most false negatives were actually the company thwarting fraudsters.
But ID.me can be duped. And it can be duped fairly easily it seems. A Washington Post report shows one person illicitly secured nearly $1 million in unemployment benefits using little more than an extremely obvious wig.
[D]espite the scale of the data gathering by the company, ID.me, revealed in newly released records, the system has been exploited by scammers. Federal prosecutors last month said a New Jersey man was able to verify fake driver’s licenses through an ID.me system in California as part of a $2.5 million unemployment-fraud scheme.
ID.me has pointed to the scam as an example of how well its systems work, noting that it referred the case to federal law enforcement after an internal investigation. But the criminal complaint in the case shows that ID.me’s identification systems did not detect bogus accounts created around the same day that included fake driver’s licenses with photos of the suspect’s face in a cartoonish curly wig.
I mean… [images via DOJ criminal complaint]:
Humans might have been able to shut this fraud down immediately. But it's clear ID.me doesn't have enough humans and is relying on mostly unproven tech to decide who is or isn't entitled to government benefits.
The IRS's walk-back on ID.me use may end up causing at least as many problems as it solves, unfortunately. The IRS also suffers from a shortage of humans and now they will be expected to do more with less outside assistance as tax return season shifts into high gear. By the time the IRS was forced by public and Congressional pressure to make a change, it was already up to its eyeballs in returns. Taxpayers can now expect delays ranging from several weeks to several months at least partially as a result of the agency's regrettable decision to do business with ID.me and its questionable track record.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: ai, facial recognition, government benefits, wigs
Companies: id.me
Reader Comments
Subscribe: RSS
View by: Time | Thread
Their claimed numbers make a lot more sense now
ID.me: 'There is an insane amount of fraud going on!'
Also ID.me: Falls for hilariously obvious fraud.
I think I might have an idea how so much of the claimed fraud is getting through...
[ link to this | view in chronology ]
Re: Their claimed numbers make a lot more sense now
What's obvious about it? Even if it's obvious it's a wig, why does that indicate fraud? There really are people going around in terrible wigs, and he looks the same on the licenses and the other photographs.
I once saw someone on a bus with obviously fake eyebrows drawn, possible tattooed, onto their face. Maybe they were heading downtown to commit fraud, but it seems much more likely that they just made bad life choices--or maybe had some medical condition. Would we really want the DMV to make people remove wigs for their photographs? Maybe remove makeup too, shave moustaches/beards, etc?
Despite the attempt to place all the blame on ID.me, the NY and SC unemployment offices can take some blame too. Do they not have a way to check whether the pictured licenses are real? Or maybe call the named person's last known employer, find out whether they were really let go? Get contact information from some other source and ask the person themselves whether they filed for unemployment? Check whether the bank account used was the same one used for payroll, or has the same name?
Also, we're talking about unemployment offices here, and are they really short on humans? Many require that people be actively looking for work and not turn down offers for no good reason, and they've got a database of people they can make offers to...
[ link to this | view in chronology ]
Re: Re: Their claimed numbers make a lot more sense now
... can't tell if serious or joking, but on the off-chance you're serious it's the exact same guy in different colored wigs. That's it. ID.me is pitching their product as a way to crack down on/prevent 'fraud', if something that blatantly obvious is able to skate right past then clearly their system is in dire need of work before states(or especially federal agencies) rely on it for anything but 'How badly can it screw up today?'-level jokes.
[ link to this | view in chronology ]
Re: Re: Re: Their claimed numbers make a lot more sense now
I think it's the exact same guy in the exact same wigs, with poor color calibration in the images. Look at the guy's skin; some whole pictures are tinted red. But on the licenses, all are the same.
The wig is an easy thing to make fun of, but doesn't seem to have anything to do with anything. Take away the wig, and the guy would still have 4 licenses with 4 matching self-photos. Maybe 5 if he got a non-fake license without a wig. If the guy was smart about fraud, he'd have worn different wigs for each, different clothing, shaved the facial hair differently, add some fake moles, use a different facial expression, etc. (For that matter, if he's only using the fake IDs for online benefit application, why does it even have to be a picture of him? There are billions of pictures and videos online. Grab someone else's and use them.)
There are a lot of people who look similar to each other, e.g. IMDb has occasionally pointed out celebrities who are easily confused. Is there are reason to think facial recognition is accurate enough to tell such people apart while not failing due to ordinary hairstyle changes, aging, etc? The government should know that the advertising is not to be trusted, and proper studies are required.
[ link to this | view in chronology ]
I'm not so sure the feds really aren't yet, though - I went to the IRS site on Sunday to try and check on the various tax credits from last year, and it wanted me to set up a new ID.me account.
[ link to this | view in chronology ]
Re:
They were going to use ID.me, but the backlash has apparently forced them to rethink it. It may still be active, but for how much longer?
https://arstechnica.com/tech-policy/2022/02/irs-stops-requiring-selfies-after-facial-recogni tion-system-is-widely-panned/
[ link to this | view in chronology ]
If you can get $900,000 with a bad wig, imagine how much you could get with a good wig!
[ link to this | view in chronology ]
Re:
And how many people were falsely denied, for getting a haircut.
[ link to this | view in chronology ]
Re: Re:
Why did you shave your beard? What happened to that tooth? What's that scar about? No we won't accept your payment!
[ link to this | view in chronology ]
I want such a sweet deal too!
I really wonder how such a relatively small company that is doing such a bad job is able to land such big deals. If I can get a few millions for screwing up, I would accept the resulting hit to my reputation.
[ link to this | view in chronology ]
Saving money for the crooks
We all knew the government only used ID.me as a quick way to save money by denying legitimate applications. We also know that the crooks always find a way to the money...and that the government is fine with that. Witness the cable companies.
[ link to this | view in chronology ]
IRS
Over the last bunch of years around 10.
The IRS has lost allot of people, from Cuts from the Gov.
I should not mention that computers are old enough to be a grand child of teen age. and the 1 Big computer from the 1960's is still used, even when your cellphone has more power.
[ link to this | view in chronology ]
Look at the pictures.
Now look at me laughing.
Now look at the pictures.
Now look at several states that are going to be sued.
Now look at the pictures.
Now look at several states that should look for the kickbacks.
Now look at the pictures and wonder why anyone is still using this company for anything.
[ link to this | view in chronology ]
IRS is still using it
Not sure where you heard that the IRS quit using id.me, i spent hours yesterday trying to get the system to accept my valid original documents trying to create an IRS login.
By the way I am still waiting for a response from them on how to resolve my issue.
[ link to this | view in chronology ]
Re: IRS is still using it
IRSD is still using a mainframe from the 1960's. Duh.
[ link to this | view in chronology ]
Re: IRS is still using it
Previous article, which is the second link in this one. The relevant quote came from a response to Wyden sending the Treasury Department basically a WTF letter, which prompted the IRS to chime in as follows:
'The IRS announced it will transition away from using a third-party service for facial recognition to help authenticate people creating new online accounts. The transition will occur over the coming weeks in order to prevent larger disruptions to taxpayers during filing season.'
[ link to this | view in chronology ]