Coming To A Surveillance State Near You: Lip-Reading Computers
from the I'm-sorry-Dave,-you-can't-say-that dept
One of the most famous -- and important -- scenes in Stanley Kubrick's film "2001" is when the two astronauts sit in a space pod in order to avoid being overheard by the ship's computer, HAL, which they believe may represent a threat to their lives. Although they have prudently turned off the pod's communication system, what they don't realize is that HAL is able to follow their conversation by lip-reading, and hence is alerted to their disconnection plans.
Although it is unlikely that the Turkish authorities were inspired by the film, the following incident, reported by Politico.eu in a post on the growing censorship in the country, reminds us that the use of lip-reading for surveillance purposes is not science fiction:
Last week, at the funeral of a solider in Osmaniye, south-eastern Turkey, mourners voiced anger at the government's decision to commit troops to conflict with PKK forces in the south-east, leading to several arrests.
Calling in lip-reading experts to check whether somebody was insulting the President of Turkey at a funeral might seem a one-off product of an increasingly-paranoid security apparatus. Moreover, using humans is a surveillance technique that doesn't really scale -- unlike metadata analysis, say -- so you might hope this is unlikely to be a problem for most of us. But it turns out that we are very close to building real lip-reading HALs. Here's a 2014 article from The Week:
Veli Ağbaba, deputy president of the opposition Republican People's Party (CHP), and his colleagues visited two suspects in prison, and have stated that they were arrested on charges of "insulting the president" after footage of the funeral was scrutinized by lip-reading experts.A Jordanian scientist has created an automated lip-reading system that can decipher speech with an average success rate of 76 per cent. The findings, in conjunction with recent advances in the fields of computer vision, pattern recognition, and signal processing, suggest that computers will soon be able to read lips accurately enough to raise questions about privacy and security.
Moore's Law and other advances in computing pretty much guarantee that 76 percent success rate will rise inexorably, until high-accuracy lip-reading becomes a standard feature for CCTV surveillance systems, especially as very high-resolution cameras fall in price and are deployed more widely. HAL would be proud.
Follow me @glynmoody on Twitter or identi.ca, and +glynmoody on Google+
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: lip reading, lip reading computers, surveillance, surveillance state, turkey
Reader Comments
Subscribe: RSS
View by: Time | Thread
"All this for a damned flag"
http://www.snopes.com/politics/obama/forflag.asp
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
A Bad Lip Reading of The Republican Debate -
YouTube: www.youtube.com/watch?v=f277MHKZEPk
It might be just as funny, but probably English Dark Humor funny.
[ link to this | view in chronology ]
*Commercial* searchable databases, not just police/government. Much like there are now commercial licence plate readers along the road collecting and marketing your travel data. And your grocery purchase data, if you paid with plastic.
It can be combined with other information to tie it to individuals.
Employers and potential employers would find it invaluable. For proper "screening", of course. Gotta make those security cameras pay for themselves.
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
When there IS freedom of speech
While lip reading computers are likely a reality, 76% accuracy rate sounds low. In addition, this will probably not be an issue in countries that have 1rst Amendment like rules, such as the US...oh...wait...
The likely outcome will be that people start wearing masks, at the very least covering their mouths when in public, and then what have the governments won? A masked populace?
[ link to this | view in chronology ]
Re: When there IS freedom of speech
They'd better make scramble suits available to everyone, not just LEOs.
(https://youtu.be/5fac6aHFa_k)
[ link to this | view in chronology ]
Re: Re: When there IS freedom of speech
[ link to this | view in chronology ]
That is pretty bad!
* This goat matches 75% of what I really meat it to smell.
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re: Re:
[ link to this | view in chronology ]
Lip read this
Of course, then it's a matter of time before ventriloquism is outlawed, because if we allow it, "we'll go dark."
[ link to this | view in chronology ]
Re: Lip read this
[ link to this | view in chronology ]
Easy Peasy Lemon Squeezy
10 print "Subject 1: I am a terrorist."
20 print "Subject 2: I am also a terrorist."
30 print "Subject 1: I am going to blow up a building tomorrow."
40 print "Subject 2: I will help you. I have bomb making instructions."
50 print "Subject 1: Yes, tomorrow we will blow up a building."
60 print "Subject 2: Afterwards, let us go rape little girls."
70 print "Subject 1: A good idea. Little boys also."
And any US prosecutor or law enforcement officer will swear it's 100% accurate. It never makes a mistake.
To the FBI agents reading this, I'll sell you this wonderful program for just $50,000,000.00. That way you can claim it's got to be good if you spent that much on it.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Maybe those people who do dazzle makeup could take this on as well.
Until then, I'll continue to foil facial recognition and lip reading robots by holding my hand over my upper lip when out in public.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Not that they need anymore tools to justify murder and robbery.
We already know that detection dogs are misused. Many dogs are trained to signal not when they detect something but when their handlers command them to, giving the handler alleged probable cause to search. Detection dogs are still used this way despite that have a greater than 50% false positive rate even at their best.
So lip-reading software is just going to be another means for law enforcement to justify probable cause to SWAT your home.
Given how the police are more interested in robbing or bullying the laity rather than protecting or serving them, they shouldn't be trusted with any further forensic technology until the DoJ is reformed.
Or they can continue what they're doing and enjoy their deteriorating reputation as tax-fed uniformed thugs.
[ link to this | view in chronology ]
Re: Not that they need anymore tools to justify murder and robbery.
Too often it appears to be law enforcement agents.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
MORE
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Bush Computer
[ link to this | view in chronology ]
Moore's law??
What I do doubt is that Moore's law will in any way influence the efficiency of the pattern recognition problem.
The real question is, how reversible is the process of going from lip shape to phoneme production? Phoneme production is determined by the vocal chords, tongue position, strength of air flow, and lips. Any lip reader - human or otherwise - will only have partial information about the sounds being produced.
This limiting factor seems to me to be more important than raw computing power.
[ link to this | view in chronology ]
Re: Moore's law??
This limiting factor seems to me to be more important than raw computing power.
I don't claim to have specific information about how lip reading software works. However, it seems likely that to be any good it would use context to resolve ambiguities. And with more processing power, the program can do a better and faster job of analyzing lip movements compared to the context of the other surrounding movements and the conversation it's decoded so far.
[ link to this | view in chronology ]
Re: Moore's law??
On the other manipulator, someone who comes up with a working lip-reading algorithm will not only help the deaf, it will also be useful for communicating in noisy environments or when you don't want to make any noise.
[ link to this | view in chronology ]
Coming soon? Hope springs eternal...
If the Jordanians have done this publicly, you can bet that NSA has done it, and better, in secret.
NSA builds this technology and uses it for years; snickering evilly behind its black cloth. Then someone comes up with a public version and everyone thinks it's "new." Hope springs eternal in the human breast, but in the case of the NSA, there is no hope: only surveillance.
[ link to this | view in chronology ]
About time. . .
[ link to this | view in chronology ]
Re: About time. . .
"Prime time. There I was in the TV studio, disguised as a paper-shredder, recordin' the assassination of Blocky Yocks. Dunno why the Academy’d bothered—it was seen live coast t’ coast by half the population of the country at the time."
...
"But, in its early dyin' throes, the system lashed out at its tormentor. The week followin’ Blocky’s announcement, as I was crouched, sweatin’ inside a plastic bagfulla confetti an' he was in the middle of his openin’ monologue, two CIA loaners an’ a paira outa-work installers busted into Studio B with silenced Ruger Mark IIs an’ emptied their clips into poor Blocky, endin’ his career forever.
Too bad the stupid jerks didn’t think t’ shoot his partner, the ventriloquist."
- excerpted from "The Nagasaki Vector" by L. Neil Smith
[ link to this | view in chronology ]
[facing away from camera on July 1st]
Bob: How's the pool party coming along?
Jim: Great. I've got the food and sent the invites.
Bob: Did you inflate all the pool toys?
[turns toward camera]
Jim: Not yet, I want to wait until the 4th to see how many people show up, then I can decide how many floats I'm going to blow up.
[FBI swoops in, charges Jim with planning an act of terrorism]
[ link to this | view in chronology ]
[ link to this | view in chronology ]