DHS Oversight Says Social Media Scanning Program Is Badly Implemented And Agency Doesn't Even Know If It Works

from the 'do-something:'-the-algorithm dept

The DHS and CBP have both taken a healthy interest in travelers' social media posts. The DHS head even suggested withholding this information would no longer be an option -- that demands for account passwords were on the way. (Considering the government can search every person and their electronic devices at the border, demands for social media info would seem to be mostly redundant...) The underlying premise is this would give the US a jump on incoming terrorists by checking travelers' posts against a list of troublesome terms.

This isn't a welcome development, but the federal government continues to be its own worst enemy. You can't fear what can't be deployed competently. The DHS isn't going to stop trying to hoover up social media posts as part of the vetting process, but as a just-released Inspector General's report [PDF] points out, it may be several years before this vetting program operates in any sort of useful fashion. (via The Register)

[T]hese pilots, on which DHS plans to base future department-wide use of social media screening, lack criteria for measuring performance to ensure they meet their objectives. Although the pilots include some objectives, such as determining the effectiveness of an automated search tool and assessing data collection and dissemination procedures, it is not clear DHS is measuring and evaluating the pilots’ results to determine how well they are performing against set criteria.

It appears the DHS has only a vague grasp on what it's looking for in a social media harvesting program. Combining this with a lack of useful metrics means the agency has been throwing algos at the wall and hoping one sticks. Of course, deciding which one has "stuck" also appears to be out of the agency's technical reach.

USCIS started a pilot in December 2015 to screen the social media accounts of [REDACTED] and [REDACTED] applicants for [REDACTED] status. The pilot’s objective was to examine the feasibility of using social media screening with an automated search tool called [REDACTED] and determine whether useful information for adjudicating refugee applications could be obtained. Although the pilot had an objective, it did not define what would constitute a successful outcome…

As the OIG points out, the absence of any metric meant there was no way to know if the program was successful or not. All the DHS determined is that a redacted number of those screened had "confirmed social media accounts," something the agency could likely have achieved without deploying the unnamed "automated search tool." [Google?]

The next pilot program went live in April 2016. It, too, had the same lack of quantifiable results or stated goals.

The applicants were asked to voluntarily give their social media user names. USCIS then screened the user names against [REDACTED] using the [REDACTED] tool; USCIS also manually screened the user names against [REDACTED]. USCIS assessed identified accounts to determine whether the refugees were linked to derogatory social media information that could impact their eligibility for immigration benefits or admissibility into the United States. Using the tool and manual screening, USCIS identified [REDACTED] individuals with confirmed social media accounts and [REDACTED] individuals with unconfirmed accounts. In reviewing the pilot, USCIS concluded that the tool was not a viable option for automated social media screening and that manual review was more effective at identifying accounts.

USCIS said this tool delivered results with "low match confidence," but did not bother measuring the program's success or lack thereof against anything that might have helped choose an algorithmic successor. Meanwhile, ICE was testing its own search tool. Like the rest of the agencies, it also failed to implement anything that might have quantified the tool's usefulness. While it did draft up some prerequisites and metrics, it failed to develop a plan for moving the program forward or even apply the metrics to the pilot program's results. ICE's tool, however, sounds more invasive than the others discussed in the report. Not only would this be used to screen applicants, but would provide post-screening "monitoring" of flagged accounts.

The OIG recommends these agencies do all the things they're not currently doing, instead of wasting time and money deploying software solutions without any apparent attempt to determine if they're capable of solving the government's social media "problem." This doesn't mean social media snooping is on hold. Lord no. It just means it's being done badly by multiple agencies, all of them more interested in the snooping than the snooping's usefulness.

Filed Under: cbp, dhs, social media, social media scanning