Facebook Ups Surveillance Of Users To Keep Tabs On People Who Don't Like Facebook

from the threat-level:-oversharing dept

Tech companies are becoming far more than useful repositories of third party records. They're becoming far more active in terms of surveillance, pivoting from platform providers to private sector Big Brothers, weaponizing their data collection capabilities to keep tabs on customers and users.

Facebook has decided to start scanning its platform for threats. Not threats against the many nations it serves or threats targeting other users, but rather threats against Facebook itself.

One of the tools Facebook uses to monitor threats is a "be on lookout" or "BOLO" list, which is updated approximately once a week. The list was created in 2008, an early employee in Facebook's physical security group told CNBC. It now contains hundreds of people, according to four former Facebook security employees who have left the company since 2016.

Facebook notifies its security professionals anytime a new person is added to the BOLO list, sending out a report that includes information about the person, such as their name, photo, their general location and a short description of why they were added.

[...]

Users who publicly threaten the company, its offices or employees — including posting threatening comments in response to posts from executives like CEO Mark Zuckerberg and COO Sheryl Sandberg — are often added to the list. These users are typically described as making "improper communication" or "threatening communication," according to former employees.

It's not that Facebook shouldn't be on the lookout for credible threats. It's that it's turned its platform into a surveillance tool for its in-house knockoff law enforcement agency. It's not clear whether the company is turning over its internal BOLO list to actual law enforcement, but if it is, that raises even more concerns. Certainly the company should be concerned about legitimate threats. But the company is flagging people simply for expressing their displeasure with Facebook in general.

While some users end up on the list after repeated appearances on company property or long email threats, others might find themselves on the BOLO list for saying something as simple as "F--- you, Mark," "F--- Facebook" or "I'm gonna go kick your a--," according to a former employee who worked with the executive protection team.

This undercuts Facebook's official statements about "rigorous reviews" of detected threats. So does the claim made by former employees that fired employees are automatically added to the BOLO list, despite nearly 100% of fired employees from all vocations posing no threat to their former employers.

And it goes further than simply flagging people (and, apparently, displaying their photos on monitors in the threat detection center). Facebook also tracks listed individuals using their smartphones, thanks to permissions granted to the Facebook app. The app comes pre-installed on most smartphones and most users are unaware how much data Facebook is gathering even when the app isn't in use.

Presumably, if some "F--- you, Mark" person gets too close to the Facebook campus, actual law enforcement is alerted. This sort of situation can only lead to positive outcomes. A person mildly displeased with Facebook's endless fuckery will be greeted by armed officers under the impression a credible threat has been made against the company. Good times.

More good times await. Facebook is also promising to "help" the suicidal by sending the cops after them.

Since 2011, Facebook has allowed users to flag potential suicidal content; reports prompted emails from Facebook urging the poster to call the National Suicide Prevention Lifeline. But starting in 2017, Facebook introduced bots to search out and report potential suicidal content. The bots report suspected cries for help to human moderators, who may then "work with first responders, such as police departments to send help," says CNN.

That's right: Facebook might call the cops on you because a bot thought you seemed sad. Facebook executives think that if a user exhibits signs of depression, it's up to Facebook—not the user's friends, family, or community—to intervene.

Rather than trying to track down friends or family, Facebook is turning this over to "first responders." In most cases, the first responder on the scene is going to be the local PD. Given how often police officers have helped talk people out of suicide by killing them, this effort by Facebook is going to result in more dead suicidal people than simply doing nothing.

I'm not saying Facebook should do nothing about threats against the company or to aid people with suicidal thoughts. But these efforts aren't going to make anything better and they're a misuse of Facebook's vast data collections and moderation efforts. There's an abuse of trust happening here and Facebook's efforts are so scattershot and half-assed they're going to cause a lot of collateral damage.

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: privacy, surveillance, threats
Companies: facebook


Reader Comments

Subscribe: RSS

View by: Time | Thread


  1. icon
    Anonymous Anonymous Coward (profile), 21 Feb 2019 @ 1:32pm

    Add me to the list, here is my opinion of Facebook, in part

    I haven't used Facebook. I don't use Facebook. I won't use Facebook. I haven't, won't, and don't do business with any business who's only access is Facebook. I block Facebook in my HOSTS file (whatever good that does) and in Umatrix. I think Facebook is pointless, scummy, dangerous, and sociopathic. I don't like Facebook, I don't like their attitude, I don't like their position or their ability to have the impacts they have. I don't like they way they spin serious accusations against them as being less nefarious than I think they actually are.

    If Facebook HQ and all of it's servers everywhere in the world were destroyed, irrecoverably, my opinion would be that the world would be better off, and that in the long run the employees of Facebook would be better off as well, as they would likely find employment in an actually useful environment. It would be nice is Zuckerberg 's bank account was accidentally drained at the same time.

    If I haven't yet fully expressed my opinion that the world could not only exist without Facebook, but that it would be a better world for its lack of existence it is due only to a lack of negative adjectives and adverbs at my current disposal. I could get out a thesaurus and go to town, but frankly, Facebook just isn't worth it. Better to just ignore it and pity those who have succumbed to the promise never delivered.

    Other than pity which I express only here, I have nothing against any user of Facebook.

    link to this | view in thread ]

  2. identicon
    Anonymous Coward, 21 Feb 2019 @ 1:51pm

    CELEBRITY: "I was threatened on Twitter!"

    FANS: "So you admit to reading every tweet which mentions you!"

    link to this | view in thread ]

  3. identicon
    Anonymous Coward, 21 Feb 2019 @ 1:52pm

    Re: Add me to the list, here is my opinion of Facebook, in part

    AOL was 100x worse in the mid-1990s, when they had control over not just a website, but your internet connection itself.

    Curse words were like saying "Beetlejuice!" as a GUIDE would enter your chat just to keep things clean. Only a wage lawsuit put an end to that, btw.

    The market fixes this problem much more quickly than the government, but most people are their own tabloid anyway.

    link to this | view in thread ]

  4. icon
    That Anonymous Coward (profile), 21 Feb 2019 @ 2:00pm

    We don't allow people to fly based on fewer facts, where is the problem?

    link to this | view in thread ]

  5. identicon
    Anonymous Coward, 21 Feb 2019 @ 2:16pm

    Rigor

    This undercuts Facebook's official statements about "rigorous reviews" of detected threats.

    Not really. It may be that you're interpreting "rigorous" differently than them. "Rigor" basically means "inflexible" or "harsh"—and a hard rule to list people saying "Fuck Mark" as dangerous certainly qualifies. Hell, maybe they have 98 other employees verify the person really said it.

    Did you think Facebook meant "reasonable"? They didn't say that, and like interpreting NSA press releases, we're well past the point where we can assume good faith.

    link to this | view in thread ]

  6. icon
    Bamboo Harvester (profile), 21 Feb 2019 @ 2:18pm

    I can't believe...

    ...this is a surprise to anyone.

    Talk bad about your boss or the company you work for IN THE OFFICE, and you're going to have problems. What makes the "online world" any different than, in the world of facebook, IRL?

    link to this | view in thread ]

  7. identicon
    Anonymous Coward, 21 Feb 2019 @ 2:26pm

    Re: Rigor

    "Rigor" basically means…

    Please recall that this is Techdirt, and some proportion of the “technology” folks do have a fair amount of semi-advanced mathematical education in their backgrounds.

    INot that engineers can or should ever be confused with ”real mathmaticians” — we know those real mathematicians — they sometimes invite us to their parties — probably just so someone's around to change their lightbulbs for 'em. 'Cept for SWEs, they can't deal with h/w either.

    But the point is that we're familiar with “rigor” as a general concept, and have absorbed some rather strong connotative notions about rigor.

    link to this | view in thread ]

  8. icon
    Matthew Cline (profile), 21 Feb 2019 @ 2:27pm

    Re: I can't believe...

    Talk bad about your boss or the company you work for IN THE OFFICE, and you're going to have problems.

    Problems which shouldn't extend to being labelled a potentially dangerous person.

    link to this | view in thread ]

  9. identicon
    Personanongrata, 21 Feb 2019 @ 2:39pm

    Will Facebook Face-Plant?

    Facebook Ups Surveillance Of Users To Keep Tabs On People Who Don't Like Facebook

    As the hour hands of time slowly flow on like molasses on a cold winter day it would appear as if Facebook is going to Face-plant.

    Like the internet behemoths (ie CompuServe, MySpace, Yahoo, AOL, etc) that preceded it's rise Facebook may find it is now on a seemingly unstoppable downward trajectory toward intertube irrelevance.

    https://www.forbes.com/sites/ninaangelovska/2019/01/07/facebook-loosing-users-to-pinterest-youtube- and-twitter-market-share-by-region/#48bd464d7746

    https://techcrunch.com/2018/07/26/facebook-official ly-loses-123-billion-in-value/

    https://www.urbandictionary.com/define.php?term=intertubes

    link to this | view in thread ]

  10. identicon
    Anonymous Coward, 21 Feb 2019 @ 2:41pm

    It now contains hundreds of people, according to four former Facebook security employees who have left the company since 2016.

    Well, I know who some of those people are: I'm pretty sure the list includes four former Facebook security employees and Mike Masnick. It possibly includes me, despite my blackholing of FB assets wherever possible.

    Not because of threatening behavior mind you, but because of "improper communication".

    link to this | view in thread ]

  11. identicon
    Anonymous Coward, 21 Feb 2019 @ 2:43pm

    Re: Re: I can't believe...

    Anyone who may limit Facebook's profitability or ability to traffic in PII is de facto a dangerous person. There's more than one way to 1984.

    link to this | view in thread ]

  12. identicon
    Glenn, 21 Feb 2019 @ 3:07pm

    Is it alright for me to suggest that facebook seems to itself be committing suicide now? (from the death-by-idiocy dept.)

    link to this | view in thread ]

  13. icon
    Bamboo Harvester (profile), 21 Feb 2019 @ 3:15pm

    Re: Re: I can't believe...

    That all depends on who you work for.

    link to this | view in thread ]

  14. identicon
    Anonymous Coward, 21 Feb 2019 @ 3:19pm

    Re: I can't believe...Zuckerberg

    ... does Zuckerberg really know what's going on in the daily operations of his company -- or is he just a figurehead ??

    link to this | view in thread ]

  15. identicon
    Anonymous Coward, 21 Feb 2019 @ 3:46pm

    Re: Re: Rigor

    But the point is that we're familiar with “rigor” as a general concept, and have absorbed some rather strong connotative notions about rigor.

    Do you think Facebook doesn't know the difference between how people are likely to interpret their statements, and what can be shown to have been technically not a lie?

    link to this | view in thread ]

  16. identicon
    Anonymous Coward, 21 Feb 2019 @ 4:16pm

    Re:

    Bury enough bodies in your wake and you spend all your time trying to prevent them from climbing back out for the world to see.

    link to this | view in thread ]

  17. identicon
    Anonymous Coward, 21 Feb 2019 @ 4:41pm

    More good times await. Facebook is also promising to "help" the suicidal by sending the cops after them.

    Wasn't Masnick screeching a while back about how this was a great thing and how GDPR was bad because it might prevent Facebook from "helping" people in this way?

    link to this | view in thread ]

  18. identicon
    Anonymous Coward, 21 Feb 2019 @ 5:58pm

    Re:

    Sending the cops to a suicidal person is generally not helpful. Especially since that's not really their area of expertise or purpose.

    link to this | view in thread ]

  19. identicon
    Anonymous Coward, 21 Feb 2019 @ 6:03pm

    Re: I can't believe...

    Talk bad about your boss or the company you work for IN THE OFFICE, and you're going to have problems.

    Do American workers not have the right to communicate to improve their working conditions, without fear of retaliation?

    link to this | view in thread ]

  20. identicon
    Anonymous Coward, 21 Feb 2019 @ 9:18pm

    Re:

    As usual, more lying.

    You can read the article you're talking about here: https://www.techdirt.com/articles/20180128/21505439105/unintended-consequences-eus-new-interne t-privacy-rules-facebook-wont-use-ai-to-catch-suicidal-users.shtml

    There is no "screeching" present.

    link to this | view in thread ]

  21. identicon
    Anonymous Coward, 21 Feb 2019 @ 11:38pm

    Re: Why must you turn this forum into a house of lies?

    link to this | view in thread ]

  22. identicon
    Anonymous Coward, 22 Feb 2019 @ 1:29am

    Scattershot is right! I'm still a 1st amendment kind of person. They will probably not end with just their users, but the entire internet! So, obligatory once again.. fuck facebook.

    link to this | view in thread ]

  23. identicon
    Anonymous Coward, 22 Feb 2019 @ 7:16am

    Re: Add me to the list, here is my opinion of Facebook, in part

    So glad I am not the only one who has never used fb or likes buttons. I have a friend who will never step foot into a walmart. But he loves fb. I go to walmart every couple of weeks. We are good friends anyway!

    link to this | view in thread ]

  24. identicon
    Anonymous Coward, 22 Feb 2019 @ 7:22am

    Re: Will Facebook Face-Plant?

    Maybe fb will want to pick up 'faceplant' tradename just in case!

    link to this | view in thread ]

  25. identicon
    Anonymous Coward, 22 Feb 2019 @ 7:37am

    Re: Re: Why must you turn this forum into a house of lies?

    I know what having comments anonymously flagged at Techdirt feels like. It feels like.. death. I think that qualifies me to make a few points about seeing some corporations going out of their way to fuck with the public with virtually no fears of reprisal from governments.

    link to this | view in thread ]

  26. identicon
    Anonymous Coward, 22 Feb 2019 @ 8:03am

    Re: Re: Re: Why must you turn this forum into a house of lies?

    I know what having comments anonymously flagged at Techdirt feels like. It feels like.. death.

    Then I suggest finding some new hobbies and finding your self-worth apart from whether or not people like your comments. If that's all it takes to mess up your day then you need help. Fast. Because life is going to throw a whole lot worse at you than some anonymous internet citizens disagreeing with your opinion and hiding your comments.

    link to this | view in thread ]

  27. identicon
    Rog S., 22 Feb 2019 @ 8:37am

    I told you so.....

    Corporate gang stalking DOES involve the police, at many levels, including slander, and actual, chronic surveillance.

    The BOLO originates in many places ranging from corporations, to local LEOs, to DHS private contractors, and is even given to Paul Blart the Mall Cop by police, etc.

    The Association of Threat Assessment Professionals (this eras forensic science / junk science fraud ) calls these collusions the "colliding parallel investigation "

    https://c.ymcdn.com/sites/www.atapworldwide.org/resource/resmgr/2018-ATAP-Preliminary-TMC-Pr. pdf

    ATAP is the primary crisis PR mouthpiece that weaponizes psychological psychobabble, and paints liostick on high heeled pigs so that these agencies can continue slurping at the trough.

    The linked. pdf even describes it in an attempt to legitimize the "colluding parallel investigation " which is colloquially known as gang stalking.

    link to this | view in thread ]

  28. identicon
    Anonymous Coward, 22 Feb 2019 @ 8:53am

    Re: I told you so.....

    psychological psychobabble

    You mean like every single one of your comments?

    link to this | view in thread ]

  29. identicon
    Anonymous Coward, 22 Feb 2019 @ 9:15am

    Re: Re: Re: Re: Why must you turn this forum into a house of lie

    Why you trying to hurt me sis?

    link to this | view in thread ]

  30. identicon
    Dave P., 22 Feb 2019 @ 10:18am

    Re: Add me to the list, here is my opinion of Facebook, in part

    Just one word........Agree!!!!!!!!

    link to this | view in thread ]

  31. identicon
    Rog S., 22 Feb 2019 @ 11:19am

    Re: Re: I told you so.....

    I cant even imagine what kind of person you are.

    But basically, you are human garbage. I wouldnt shit on you if flies ruled the world.

    But you do look like a king right now.

    Surely your cowardly, anonymous, cowardly minions will swarm to you.

    And, your defense contractor friends.

    link to this | view in thread ]

  32. identicon
    Rog S., 22 Feb 2019 @ 11:20am

    your types are losing the war against due process right now.

    Troll harder, piggy.

    link to this | view in thread ]

  33. identicon
    Rog S., 22 Feb 2019 @ 11:26am

    Re: Re: Re: Re: Re: Why must you turn this forum into a house of

    There are actually military contractors who have “kill books ” who harass, and stalk people to suicide, and worse.

    Google "Dan Love and Kill Book "

    These shitbag progressive psycopaths are so far outside of due process, they make John Ashcroft look sane, and liberal.

    link to this | view in thread ]

  34. icon
    btr1701 (profile), 22 Feb 2019 @ 11:41am

    That's right: Facebook might call the cops on you because a bot thought you seemed sad. Facebook executives think that if a user exhibits signs of depression, it's up to Facebook—not the user's friends, family, or community—to intervene.

    This happened to me with Twitter. Not the cops thing, but I got a message from Twitter one day about a month ago saying that a recent post I'd made raised concern that I might be in danger of self-harm and it provided me a link to suicide prevention resources.

    I scoured my posting history and for the life of me I couldn't find anything I'd posted that remotely suggested any kind of self-harm, not even a joke response of some kind. And of course Twitter provides no ability for the user to respond and ask what the hell they're talking about.

    link to this | view in thread ]

  35. icon
    btr1701 (profile), 22 Feb 2019 @ 11:44am

    Re: Add me to the list, here is my opinion of Facebook, in part

    I haven't used Facebook. I don't use Facebook. I won't use Facebook. I haven't, won't, and don't do business with any business who's only access is Facebook. I block Facebook in my HOSTS file (whatever good that does) and in Umatrix. I think Facebook is pointless, scummy, dangerous, and sociopathic. I don't like Facebook, I don't like their attitude, I don't like their position or their ability to have the impacts they have. I don't like they way they spin serious accusations against them as being less nefarious than I think they actually are.

    But will you use Facebook with green eggs and ham?

    link to this | view in thread ]

  36. icon
    Max (profile), 22 Feb 2019 @ 2:42pm

    I'm finding less and less incentive to read comments I can't downvote (some of them x1000 - well, one can dream...) on a site where absolute, utter lunacy seems to be the norm in the comments. But hey, that's just me...

    link to this | view in thread ]

  37. identicon
    Anonymous Coward, 22 Feb 2019 @ 3:46pm

    Re:

    for the life of me

    I've notified the authorities. Help is on the way. Make sure there is nothing in your hands and keep them where they can be seen at all times.

    link to this | view in thread ]

  38. identicon
    Anonymous Coward, 22 Feb 2019 @ 9:01pm

    Re: Re: Add me to the list, here is my opinion of Facebook, in p

    I do not like green eggs and ham.

    link to this | view in thread ]

  39. identicon
    Rog S., 22 Feb 2019 @ 9:50pm

    Poor Max

    Take your medication, snowflake, and follow the other posters advice to “keep your hands where the cops and paranedics can see them ”when we get you 5150d....for your own good, of course, and so that the mean words stop hurting you so much.

    Surely the comments section is out to get you, just you, personally.

    link to this | view in thread ]

  40. identicon
    Rog S., 22 Feb 2019 @ 9:57pm

    Re: Fusion Center Twitter Firehose

    I got a lot of those after the 2016 election, saying they were cautioning me that I had reposted Fancy Bears circus routine, or some other Russian troll farm stuff.

    But then again, purported FBI whistle blower Sibel Edmonds, and half of Israeli Squad 3200 are my “followers ” aka cyberstalkers.

    And that, verified with empirical evidence.

    It seems the icky, bad Russian spies, hackers, and ideologues inside and outside of Twitter are somehow more icky and bad than British / Israeli/US spies, hackers, and ideologues inside and outside of Twitter.

    link to this | view in thread ]

  41. identicon
    Anonymous Coward, 25 Feb 2019 @ 8:49am

    Re: Re: Re: I told you so.....

    That's the best you've got? Just more psychological psychobabble from a Rog S-tered troll.

    link to this | view in thread ]

  42. identicon
    Anonymous Coward, 25 Feb 2019 @ 8:50am

    Re: Re: Re: Re: Re: Re: Why must you turn this forum into a hous

    And this psychological psychobabble has what to do with anything? Including the comment you replied to?

    link to this | view in thread ]


Follow Techdirt
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Discord

The latest chatter on the Techdirt Insider Discord channel...

Loading...
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it
Close

Email This

This feature is only available to registered users. Register or sign in to use it.