New Report Details Massive Mysterious Influence Campaign On Twitter
from the something's-happening dept
Over the weekend, the group Guardians.ai released a fascinating report detailing what appears to be a massive influence campaign taking shape on Twitter. By way of disclosure, one of the three key authors of the report, Brett Horvath, is also one of the key people behind the election simulation game that we helped create and run, though I have nothing to do with this new report. The report is fascinating, and if you don't feel like reading the whole thing, Bloomberg also has a write up.
The key to the report is that they have identified some truly fascinating patterns that they've spotted among a cluster of users on Twitter, who, at the very least appear to be acting in a manner that suggests some attempt to influence others. I should note that unlike other such reports that jump to conclusions, the authors of this report are very, very, very clear that they're not saying these are "bots." Nor are they saying these are Russian trolls. Nor are they saying that a single source is controlling them. Nor are they saying that everyone engaged in the activity they spotted is officially part of whatever is happening. They note it is entirely possible that some very real people are a part of what's happening and might not even know it.
However, what they uncovered does appear strange and notable. It certainly looks like coordinated behavior, at least in part, and it appears to be designed to boost certain messages. The report specifically looks at statements on Twitter about voter fraud using the hashtag #voterfraud, but it appears that this "network" is targeting much more than that. What made the report's authors take notice is that in analyzing instances of the use of the tag #voterfraud, they noticed that it appeared to have a "heartbeat." That is, it would spike up and down on a semi-regular basis, based on nothing in particular. There wasn't a specific news hook why this entire network would suddenly talk about #voterfraud, and they wouldn't talk about it all the time. But... every month or so there would suddenly be a spike.
From there, they started digging into the accounts involved in this particular activity. And they found a very noticeable pattern:
We wanted to know how these accounts were coming onto Twitter and gaining mentions at such a high velocity — what was leading accounts to gain influence, so quickly? So we took a sample set of accounts from a group of suspicious Voter Fraud accounts and started looking at their activity day-by-day, starting at day one. What we began to notice is a pattern for how the influence machine might be working, and how coordination could be happening.
Here's the consistent network pattern we saw:
- User signs up for an account.
- User starts replying to multiple accounts—some known verified Twitter users and many other accounts that are also on our list of actors, or that fit a similar profile.
- The replies tend to contain: text, memes, hashtags, and @mentions of other accounts, building on common themes.
- At some point the pattern shifts from being all replies to original tweets. Those original tweets contain the same types of content as their replies do.
- It appears that this pattern cycles and repeats when the next batch of new accounts come online. The next batch starts replying to the existing, newly influential accounts, and carry on with the same sequence of events for gaining influence.
The report highlights this pattern with a few example accounts, though the full study looked at (and continues to look at) many, many more. What you see over and over again are Twitter feeds of people who seem to do little other than constantly tweet pro-Trump memes and disinformation, and yet magically get thousands and tens of thousands of retweets, often coming out of nowhere. Here's one example:
The gray line at the bottom is the number of tweets. The black line is the number of mentions from others. Notice how it goes from nothing to around 10,000 in no time? Sometimes the accounts are more or less dormant for a while, before suddenly becoming massively popular for no clear reason at all:
Again, as the report makes clear, these aren't necessarily bots (though, they may be). They aren't necessarily even aware that they're a part of something. But the patterns seen over and over and over and over again are uncanny. And it certainly provides strong circumstantial evidence of some sort of influence operation -- and it's one that appears to continue to grow and grow.
As the report notes:
We don’t know why this activity is occurring, or who is behind it. However, the best we can do is look at the data around what’s actually happening. What we've discovered along the way is that there are overlapping patterns of behavior, demonstrating some form of coordination.
We think it's possible that some of these accounts don't realize that they're coordinating or part of a larger influence network. For example, one of these sample accounts might genuinely care about Voter Fraud. A bad actor, coordinating large numbers of accounts could find this person’s tweets useful, then amplify those tweets through thousands of @mentions and replies.
By focusing on the hard data around coordination, we can better understand how public conservations are being distorted and how it affects society. Whatever your views are on Voter Fraud, these accounts and the accounts that amplify them are rapidly accelerating their activity in the lead-up to Election Day.
Similarly, of course, it's not clear that this is actually having any impact on anyone's views. But it's at least worth looking at what happens when there is what appears to be massively coordinated activity, mostly focused around spreading disinformation regarding the election and more. The full methodology of the report is available on the site, as are the names of 200 of the accounts studied.
What's fascinating, of course, is the sheer size of what's happening, and the level of coordination necessary to make it happen. Twitter's response to the report (as noted in the Bloomberg article) is pretty much what you'd expect Twitter to say:
“While we prohibit coordinated malicious behavior, and enforce accordingly, we’ve also seen real people who share the same views organize using Twitter,” the company’s statement said. “This report effectively captures what often happens when hot button issues gain attention and traction in active groups.”
Indeed, that's part of what's so tricky here. Could this kind of thing happen organically? Well, certainly much of it can. Lots of people who share the same views on any particular subject often will see surges in conversations around those topics, including lots of retweets, mentions and replies. But the pattern here definitely looks different. When these things happen organically, they tend to have a fairly different rhythm, either a lot more sustained, or the spikes are much more spread out and explainable (e.g., there was some news event that tied to the topic). Similarly, it is hard to see how so many pseudononymous people, who no one else really knows, magically all jump up to thousands or tens of thousands of mentions with no clear explanation for their sudden and sustained fame.
But this is also why Twitter is put in an impossible position if it's expected to spot all of this. Even with so much evidence, it's still possible that what Guardians.ai spotted was organically formed. It may seem unlikely, but how can you tell? And you can bet that there are some with less than virtuous intent, who are actively figuring out ways to increasingly make all of this activity look organic. Expecting that Twitter, or any company, can always magically determine what is and what is not "authentic" behavior online, is an impossible task. And the very fact that it might sweep up some perfectly innocuous accounts in the process also makes it troubling to expect that the platform should be in charge of sorting out who's who and who's real in these kinds of situations. But, then again, if these kinds of disinformation campaigns truly are having an impact on influencing the public, that too should be a concern. Either way, as the report highlights, there is still much work to be done in analyzing how social networks are being used to influence the public.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: bots, campaigns, coordination, influence, politics, social media
Companies: twitter
Reader Comments
Subscribe: RSS
View by: Time | Thread
So, like, 30 people.
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
Re: Re:
Is 30 people not enough for you?
30 is plenty to justify blocking the polls to thousands! /s
(If you love voter suppression, you might be racist scum.)
[ link to this | view in thread ]
Re: Re:
[ link to this | view in thread ]
Re: Re:
[ link to this | view in thread ]
Re: Re:
And yeah, I mean, hypothetically Mexico or China could send hundreds of thousands of people to live in a county just to try and swing the election that way, but... really?
[ link to this | view in thread ]
Isn't that the principal purpose of basically all communication, though?
[ link to this | view in thread ]
Re: Re: Re:
Ah, but in China's case, they don't send hundreds of thousands of people expressly for this purpose; they just attempt to exploit the hundreds of thousands who already emigrated but still have ties (business, family, etc.) back to the mainland.
Recent example: https://globalnews.ca/news/4545091/bc-election-fraud-allegations/ -- one of the mayoral candidates in Richmond appeared to be hand-picked by the Chinese government.
China appears to be doing this mostly at the local level; once they've got a foothold there's nothing to prevent them moving it up to a state/province level though. And the techniques aren't just limited to influencing Chinese ex-pats; they'll also scoop up anyone wanting to make a quick buck or get some cheap flights.
[ link to this | view in thread ]
Sobering analysis
[ link to this | view in thread ]
[ link to this | view in thread ]
Re: Re: Re: Re:
On the other hand, it doesn't report on the result/success rate/which candidate they were trying to influence, which makes it hard to see how big of a deal it was.
I think that should be considered, if only to determine whether the actual danger is affecting the results of elections (A few hundred votes seems unlikely to be telling, though it's certainly alarming) or the reputation of free and fair elections and the integrity of the voting process. (Russia and China both have motivation to try and undermine the latter and dissillusion people with the system.)
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
Re: Re: Re:
In my middle sized burg: You must be an American citizen. A resident of the burg and at least 16 to register then 18 by the election. Otherwise they might as well stay in China.
Or in the magical thinking land of Da Trump and his Ettes.
[ link to this | view in thread ]
[ link to this | view in thread ]
Well, what have we here?
Interesting, no pictures of themselves, their family or their pets. No shots of sunsets, sunrises, the family car. The pictures are repeated hundreds of times, all post many, many pictures of their photo set.
Nothing but processed (photo shopped or done professionally) GOP/Trump-think images. And the ever believable, "I am not a racist" posts. Shortly after calling Dems commies, or being a proud nationalist.
Oddly enough they sound exactly like a racist, act like a racist, bitch like a racist. Me thinks the racist does protest too much.
[ link to this | view in thread ]
Re: Re: Re: Re:
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
Re: Re: Re: Re: Re:
Actually, he was suggesting that if someone lives in an area and cares enough to want to vote, maybe we should let them; there are plenty of qualified citizens who don't want to, after all.
[ link to this | view in thread ]
Social Media Isn't Social Media Anymore
So yeah, you can make a fake PeTa if you want with one of these advocacy farms, but make it for anything. Like your lame candidacy for town fire whatever. And it will have fake wholesome AI bots pretending to be real people chatting up your website visitors and "forming meaningful relationships that lead to support you can use, at the push of a button".
The reason this is relevant is, I noticed something about a week ago when I finally quit social media forever.
Nobody is talking to each other anymore.
They have somehow made made things so bad now, we all just talk to the Ether, waiting for replies (like I am now, Like we all are, Like we all always are) but never directlyi at each other.
We have literally been conditioned to just shout at each other in rooms like this, basking in the attention of a number incrementing on an approval indicator rather than expecting some meaningful conversation.
(because that's more efficient and they make clicks and money faster. Man that boy that thought this up was a genius)
This entire system in place now, with AI controlled comments sections, and AI deciding who to ban and who to keep on Twitter (Oh, unless you think a human being is doing that. Handling the judgement call on hundreds of millions of... anymore. Noob. Still think PEOPLE at the IRS do your taxes? Do you child?
Most of our communication now is THEM to us. That was unheard of even say, 20 years ago. But now a good 90 percent of your TOTAL communication is between you and your interests rather than people. And they got your interests. They got that Game of Thrones. They got that hate boner waiting for you on Google News. Go justify being a hateful loser. The news media supports that now. They got those worldviews you should have this week on CNN. Whatever the wealthy 1 percenters on CNN think you should believe that week. For example, Maybe Warren Buffet has paid for the Media to say nice things about Apple again for that week. Since he had just invested a bunch of money into them, and didn't want them mentioning that they are now just a third place loser in smartphones. They got that new sexy Doctor Who spouting those Buzzfeed talking points. Because that's what the doctor does. Hang out on Earth and handle social justice issues.
Point is, that's not you talking to me. That's not us talking to each other. That's us being nodes. And at some point they won't even need us anymore. It will be most cost effective to handle us with a "pocket AI" that just decides what we spend money on to "keep the economy stable".
If that isn't already how things are. Kinds feels like it huh.
: )
This will probably be my last post here. I've suddenly decided this well is poisoned. I just spent six months following a careful plan to leave social media entirely and replace it with "actual media" where I have "friends" in "real life" again like I used to say, 20 years ago.
I figured out a good 40 percent of what I thought I "liked" was just me trying to appease imaginary judges
Example. "Actually, I kinda hate that TV show. A lot. I just felt I HAD to like it because of all the pressure from twitter". I kept watching a terrible tv show because I was conditioned to believe I was a "bad person" if I did not like that show.
This was a year ago. Then, with a few months, boom, this huge controversy when absolutely terrible content from low rent talent was exposed to have hired the same PR astroturfing firms to make their shows about "sexism" or "racism" to promote them, and that's how terrible content from people with zero talent was suddenly both in the news, and being shame-sold to me by twitter AI bots.
Again, a year ago...
[ link to this | view in thread ]
[ link to this | view in thread ]
Re: Social Media Isn't Social Media Anymore
Yes, it would be nice to have more conversations about topics here on techdirt but conversations take 2 people. I won't for a second assume anyone here is a friend. Most are friendly but of course there are still problem children that try to cause problems. But I also like that I can state my point of view and debate the merits of it with knowledgeable people.
Maybe don't read the comments if they get to you. You can read full articles from the main page.
[ link to this | view in thread ]
Re: Social Media Isn't Social Media Anymore
It seems you are a little lost, but your points are also valid. We're all lost in this mess to various degrees.
[ link to this | view in thread ]
Re: Re: Re: Re:
Some of the Chinese workers have said they do not want to live in Wisconsin. LOL
Foxconn Considers Bringing Chinese Workers to Wisconsin as U.S. Labor Market Tightens
https://www.wsj.com/articles/foxconn-considers-bringing-chinese-personnel-to-wisconsin-as-u- s-labor-market-tightens-1541505600
[ link to this | view in thread ]
Re: Social Media Isn't Social Media Anymore
Anyway, I do agree with quite a bit of your post. I've stopped using most social media as means for interaction a while ago. Nowadays I mostly use instant messengers with people I know personally and I prefer to meet and talk in person than via phones. It's actually healthy for your mind I found out.
[ link to this | view in thread ]
Re: Re: Social Media Isn't Social Media Anymore
Ah. You mean, like, "disagree"? So, when you refer to "having conversations with knowledgeable people" you mean "having others reinforce my virtuous rightness through conversation".
Sure. That's one way to describe it. That's also, minus the problem children, an echo chamber.
[ link to this | view in thread ]
decide
You need to decide whether you support the 1st amendment and quit dicking around. You don't get to go hunting online for people who oppose you politically and start raising red flags about it if you want a 1st amendment. I'm good either way you go. Just a warning.
Jacob
[ link to this | view in thread ]
Re: decide
You need to decide whether you support the 1st amendment and quit dicking around.
How is anything I've written "dicking around"? Especially regarding my support for the 1st Amendment?
You don't get to go hunting online for people who oppose you politically and start raising red flags about it if you want a 1st amendment.
Where the hell have I done that? Literally today I posted something arguing about people suing Trump -- who I oppose not for political reasons, but because he's a fucking disaster -- noting that their arguments go against the 1st Amendment and could do real damage. I've written about many people I disagree with politically and supported their 1st Amendment rights.
Nothing in this post goes anywhere near arguing against anyone's 1st amendment rights.
I'm good either way you go. Just a warning.
WTF?
[ link to this | view in thread ]
--they discovered the pattern that memes follow
--they discovered what happens when you have a silo of like minded individuals
--they discovered that some people tweet once in a while and have lives
-- etc.
[ link to this | view in thread ]