Coming To A Surveillance State Near You: Lip-Reading Computers

from the I'm-sorry-Dave,-you-can't-say-that dept

One of the most famous -- and important -- scenes in Stanley Kubrick's film "2001" is when the two astronauts sit in a space pod in order to avoid being overheard by the ship's computer, HAL, which they believe may represent a threat to their lives. Although they have prudently turned off the pod's communication system, what they don't realize is that HAL is able to follow their conversation by lip-reading, and hence is alerted to their disconnection plans.

Although it is unlikely that the Turkish authorities were inspired by the film, the following incident, reported by Politico.eu in a post on the growing censorship in the country, reminds us that the use of lip-reading for surveillance purposes is not science fiction:

Last week, at the funeral of a solider in Osmaniye, south-eastern Turkey, mourners voiced anger at the government's decision to commit troops to conflict with PKK forces in the south-east, leading to several arrests.

Veli Ağbaba, deputy president of the opposition Republican People's Party (CHP), and his colleagues visited two suspects in prison, and have stated that they were arrested on charges of "insulting the president" after footage of the funeral was scrutinized by lip-reading experts.
Calling in lip-reading experts to check whether somebody was insulting the President of Turkey at a funeral might seem a one-off product of an increasingly-paranoid security apparatus. Moreover, using humans is a surveillance technique that doesn't really scale -- unlike metadata analysis, say -- so you might hope this is unlikely to be a problem for most of us. But it turns out that we are very close to building real lip-reading HALs. Here's a 2014 article from The Week:
A Jordanian scientist has created an automated lip-reading system that can decipher speech with an average success rate of 76 per cent. The findings, in conjunction with recent advances in the fields of computer vision, pattern recognition, and signal processing, suggest that computers will soon be able to read lips accurately enough to raise questions about privacy and security.
Moore's Law and other advances in computing pretty much guarantee that 76 percent success rate will rise inexorably, until high-accuracy lip-reading becomes a standard feature for CCTV surveillance systems, especially as very high-resolution cameras fall in price and are deployed more widely. HAL would be proud.

Follow me @glynmoody on Twitter or identi.ca, and +glynmoody on Google+

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: lip reading, lip reading computers, surveillance, surveillance state, turkey


Reader Comments

Subscribe: RSS

View by: Time | Thread


  1. identicon
    Anonymous Coward, 15 Sep 2015 @ 2:15pm

    "All this for a damned flag"

    link to this | view in thread ]

  2. identicon
    Anonymous Coward, 15 Sep 2015 @ 2:18pm

    Looks like I am going to have to start walking around like an NFL coach doing play calls.

    link to this | view in thread ]

  3. identicon
    another anonymous coward, 15 Sep 2015 @ 2:23pm

    Makes me wonder how long it will be as the equivalent of the Bad Lip Reading channel on Youtube.

    A Bad Lip Reading of The Republican Debate -
    YouTube: www.youtube.com/watch?v=f277MHKZEPk

    It might be just as funny, but probably English Dark Humor funny.

    link to this | view in thread ]

  4. icon
    Roger Strong (profile), 15 Sep 2015 @ 2:24pm

    Just to be clear: It's wouldn't just be lip reading. It would be lip reading to text, with the text stored permanently in a database. Searchable.

    *Commercial* searchable databases, not just police/government. Much like there are now commercial licence plate readers along the road collecting and marketing your travel data. And your grocery purchase data, if you paid with plastic.

    It can be combined with other information to tie it to individuals.

    Employers and potential employers would find it invaluable. For proper "screening", of course. Gotta make those security cameras pay for themselves.

    link to this | view in thread ]

  5. identicon
    Anonymous Anonymous Coward, 15 Sep 2015 @ 2:31pm

    When there IS freedom of speech

    Based upon the headline I was ready to suggest "stop talking to your computer". Personally, I only scream at it now and then.

    While lip reading computers are likely a reality, 76% accuracy rate sounds low. In addition, this will probably not be an issue in countries that have 1rst Amendment like rules, such as the US...oh...wait...

    The likely outcome will be that people start wearing masks, at the very least covering their mouths when in public, and then what have the governments won? A masked populace?

    link to this | view in thread ]

  6. icon
    radix (profile), 15 Sep 2015 @ 2:44pm

    A 76% failure rate!
    That is pretty bad!

    * This goat matches 75% of what I really meat it to smell.

    link to this | view in thread ]

  7. identicon
    Anonymous Coward, 15 Sep 2015 @ 3:03pm

    Re:

    I'm sure the insurance and banking industries would find it fascinating also.

    link to this | view in thread ]

  8. identicon
    Kurt Mosiejczuk, 15 Sep 2015 @ 3:04pm

    Lip read this

    So.... ventriloquist training anyone?

    Of course, then it's a matter of time before ventriloquism is outlawed, because if we allow it, "we'll go dark."

    link to this | view in thread ]

  9. identicon
    Anonymous Coward, 15 Sep 2015 @ 3:24pm

    Easy Peasy Lemon Squeezy

    At least in the US. All you have to do is something like:

    10 print "Subject 1: I am a terrorist."
    20 print "Subject 2: I am also a terrorist."
    30 print "Subject 1: I am going to blow up a building tomorrow."
    40 print "Subject 2: I will help you. I have bomb making instructions."
    50 print "Subject 1: Yes, tomorrow we will blow up a building."
    60 print "Subject 2: Afterwards, let us go rape little girls."
    70 print "Subject 1: A good idea. Little boys also."

    And any US prosecutor or law enforcement officer will swear it's 100% accurate. It never makes a mistake.

    To the FBI agents reading this, I'll sell you this wonderful program for just $50,000,000.00. That way you can claim it's got to be good if you spent that much on it.

    link to this | view in thread ]

  10. identicon
    Anonymous Coward, 15 Sep 2015 @ 3:31pm

    Guess I need to blow the dust off my Guy Fawkes mask...

    link to this | view in thread ]

  11. identicon
    Anonymous Coward, 15 Sep 2015 @ 3:47pm

    I wonder how well it handles facial hair?

    Maybe those people who do dazzle makeup could take this on as well.

    Until then, I'll continue to foil facial recognition and lip reading robots by holding my hand over my upper lip when out in public.

    link to this | view in thread ]

  12. identicon
    TDR, 15 Sep 2015 @ 4:23pm

    "I'm sorry, Citizen. I can't do that."

    link to this | view in thread ]

  13. identicon
    Anonymous Coward, 15 Sep 2015 @ 5:01pm

    Re: When there IS freedom of speech

    A lot of places have been enacting laws banning masks. Toss in facial recognition, and business & government will know where you are, who you are, and what you say 24/7.

    They'd better make scramble suits available to everyone, not just LEOs.

    (https://youtu.be/5fac6aHFa_k)

    link to this | view in thread ]

  14. icon
    Uriel-238 (profile), 15 Sep 2015 @ 5:42pm

    Not that they need anymore tools to justify murder and robbery.

    So who determines what technologies are usable without a warrant by law enforcement agents?

    We already know that detection dogs are misused. Many dogs are trained to signal not when they detect something but when their handlers command them to, giving the handler alleged probable cause to search. Detection dogs are still used this way despite that have a greater than 50% false positive rate even at their best.

    So lip-reading software is just going to be another means for law enforcement to justify probable cause to SWAT your home.

    Given how the police are more interested in robbing or bullying the laity rather than protecting or serving them, they shouldn't be trusted with any further forensic technology until the DoJ is reformed.

    Or they can continue what they're doing and enjoy their deteriorating reputation as tax-fed uniformed thugs.

    link to this | view in thread ]

  15. identicon
    Anonymous Coward, 15 Sep 2015 @ 5:58pm

    How well do these lip-reading computers function with resistance fighters hurling grenades and molotovs at them, and their operators?

    link to this | view in thread ]

  16. identicon
    Anonymous Coward, 15 Sep 2015 @ 7:29pm

    MORE

    link to this | view in thread ]

  17. identicon
    Anonymous Coward, 15 Sep 2015 @ 8:36pm

    Re: Lip read this

    Big Mustaches and voice throwing , holy hell what is the world coming to ... open the pod bay door please HAL ... I'm getting off of the crazy blue marble..lol

    link to this | view in thread ]

  18. identicon
    Anonymous Coward, 15 Sep 2015 @ 9:31pm

    the whole "trust us we are the government" never works when they are known to make stuff up if they want to go after someone that has done nothing wrong save piss off someone with a bit of power and an easily bruised ego.

    link to this | view in thread ]

  19. identicon
    Anonymous Coward, 15 Sep 2015 @ 9:34pm

    Re: Re: When there IS freedom of speech

    unless of course its a mask used by the police or local home invasion aka SWAT then its expected for them to wear masks

    link to this | view in thread ]

  20. identicon
    Pixelation, 15 Sep 2015 @ 9:39pm

    Bush Computer

    "Read my lips, No new taxes!"

    link to this | view in thread ]

  21. identicon
    A. Lauridsen, 15 Sep 2015 @ 11:43pm

    Moore's law??

    I'm not convinced that Moore's law applies here. I do not doubt that the technology is feasible, lip reading is "just" another pattern recognition problem.

    What I do doubt is that Moore's law will in any way influence the efficiency of the pattern recognition problem.

    The real question is, how reversible is the process of going from lip shape to phoneme production? Phoneme production is determined by the vocal chords, tongue position, strength of air flow, and lips. Any lip reader - human or otherwise - will only have partial information about the sounds being produced.

    This limiting factor seems to me to be more important than raw computing power.

    link to this | view in thread ]

  22. identicon
    Just Another Anonymous Troll, 16 Sep 2015 @ 5:01am

    Re:

    76% success rate. Which isn't all that great, but will certainly improve.

    link to this | view in thread ]

  23. icon
    nasch (profile), 16 Sep 2015 @ 9:04am

    Re: Re:

    Whoosh!

    link to this | view in thread ]

  24. icon
    nasch (profile), 16 Sep 2015 @ 9:06am

    Re: Not that they need anymore tools to justify murder and robbery.

    So who determines what technologies are usable without a warrant by law enforcement agents?

    Too often it appears to be law enforcement agents.

    link to this | view in thread ]

  25. icon
    nasch (profile), 16 Sep 2015 @ 10:09am

    Re: Moore's law??


    This limiting factor seems to me to be more important than raw computing power.


    I don't claim to have specific information about how lip reading software works. However, it seems likely that to be any good it would use context to resolve ambiguities. And with more processing power, the program can do a better and faster job of analyzing lip movements compared to the context of the other surrounding movements and the conversation it's decoded so far.

    link to this | view in thread ]

  26. icon
    Coyne Tibbets (profile), 16 Sep 2015 @ 11:20am

    Coming soon? Hope springs eternal...

    Moore's Law and other advances in computing pretty much guarantee that 76 percent success rate will rise inexorably...

    If the Jordanians have done this publicly, you can bet that NSA has done it, and better, in secret.

    NSA builds this technology and uses it for years; snickering evilly behind its black cloth. Then someone comes up with a public version and everyone thinks it's "new." Hope springs eternal in the human breast, but in the case of the NSA, there is no hope: only surveillance.

    link to this | view in thread ]

  27. identicon
    Justme, 16 Sep 2015 @ 12:24pm

    About time. . .

    They have been looking for an excuse to label ventriloquist's as terrorist's for decades, seriously what other reason is there for talk without moving your lips, except that you have something to hide.

    link to this | view in thread ]

  28. identicon
    Rekrul, 16 Sep 2015 @ 12:24pm

    I'm sure this technology won't be misused...

    [facing away from camera on July 1st]

    Bob: How's the pool party coming along?

    Jim: Great. I've got the food and sent the invites.

    Bob: Did you inflate all the pool toys?

    [turns toward camera]

    Jim: Not yet, I want to wait until the 4th to see how many people show up, then I can decide how many floats I'm going to blow up.

    [FBI swoops in, charges Jim with planning an act of terrorism]

    link to this | view in thread ]

  29. identicon
    Jordan, 18 Sep 2015 @ 9:01am

    It's very hard to hack a post-it note placed over the camera.

    link to this | view in thread ]

  30. icon
    TRX (profile), 20 Sep 2015 @ 6:49am

    Re: Moore's law??

    That's going to be some hefty processing. I'm mostly deaf and despite years of practice, I can't lip-read enough to pick out more than occasional phonemes.

    On the other manipulator, someone who comes up with a working lip-reading algorithm will not only help the deaf, it will also be useful for communicating in noisy environments or when you don't want to make any noise.

    link to this | view in thread ]

  31. icon
    TRX (profile), 20 Sep 2015 @ 6:54am

    Re: About time. . .

    > ventriloquists ... terrorists

    "Prime time. There I was in the TV studio, disguised as a paper-shredder, recordin' the assassination of Blocky Yocks. Dunno why the Academy’d bothered—it was seen live coast t’ coast by half the population of the country at the time."
    ...
    "But, in its early dyin' throes, the system lashed out at its tormentor. The week followin’ Blocky’s announcement, as I was crouched, sweatin’ inside a plastic bagfulla confetti an' he was in the middle of his openin’ monologue, two CIA loaners an’ a paira outa-work installers busted into Studio B with silenced Ruger Mark IIs an’ emptied their clips into poor Blocky, endin’ his career forever.

    Too bad the stupid jerks didn’t think t’ shoot his partner, the ventriloquist."

    - excerpted from "The Nagasaki Vector" by L. Neil Smith

    link to this | view in thread ]

  32. identicon
    Anonymous Coward, 24 Sep 2015 @ 9:42am

    Re:

    In football (soccer), coaches, and even players, do it.

    link to this | view in thread ]


Follow Techdirt
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Discord

The latest chatter on the Techdirt Insider Discord channel...

Loading...
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it
Close

Email This

This feature is only available to registered users. Register or sign in to use it.