Obvious News Is Obvious: Polls That Only Call Landlines May Be Biased

from the you-just-realized-this? dept

Thu, Oct 14th 2010 1:42pm — Mike Masnick

I remember, way back during the 2004 election, reading stories about how the rising number of people cutting the cord when it came to their landline phone meant that phone-based surveys were not all that accurate any more. So now, six years later, research has come out saying exactly the same thing. It is true that the number of people who have done away with their landline has increased (now over a quarter of the population has ditched their landlines). Apparently, the study found that landline-only election surveys tend to overcount Republican voters and undercount Democratic ones. This doesn't come as a huge surprise as, generally speaking, the older generation skews more Republican and are also the least likely to ditch their landlines.

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: bias, landlines, polls

16 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

Gabriel Tane (profile), 14 Oct 2010 @ 1:53pm

Anyone Suprised?
Nope... not even the pollsters. I think most polls that are hoisted on a pole (no pun intended) for the populace to look at are biased by their very nature. Now, technological evolution is doing what the pollsters were doing themselves: being selective in which 'random samples' to poll. Now that this has been 'discovered', they'll have to go back to strategizing their rollouts (pollouts?) to make sure they get the right random people.

And no, I don't have any hard facts to back this up... I'm just saying that statistics and polling have long been used to swing opinion one way or another by saying "see, it's the voice of the people..."

About the only surveys I put stock in anymore are the ones on Family Feud.
[ link to this | view in thread ]
Brendan (profile), 14 Oct 2010 @ 2:08pm

Water is wet
Film at 11.
[ link to this | view in thread ]
:Lobo Santo (profile), 14 Oct 2010 @ 2:08pm

I have always beleived...
...that all survey results are skewed in the direction of people who will take surveys.
[ link to this | view in thread ]
Haywood (profile), 14 Oct 2010 @ 2:21pm

Re: I have always beleived...
True that. I tell them to screw off, so I almost never get counted. The rare occasion when I'm bored enough to answer a pollsters questions, however will result in the conservative viewpoint, with the exception that I'm ok with abortion, and think the earth is more than 6k years old.
[ link to this | view in thread ]
Christopher Gizzi (profile), 14 Oct 2010 @ 2:39pm

Dewey Defeats Truman
That is all.
[ link to this | view in thread ]
Chronno S. Trigger (profile), 14 Oct 2010 @ 3:00pm

Re: Anyone Suprised?
There's an episode of Penn & Teller that go into why numbers are Bullshit. Part of that episode is dedicated so showing how those polls are made and how the wording if the question is carefully constructed to get the answer they want.
[ link to this | view in thread ]
mrharrysan (profile), 14 Oct 2010 @ 3:20pm

Re: Re: I have always beleived...
In addition to the demographic of who has landlines anymore, I believe there is also a psychological component. As in, who takes the time to actually be polled? I know that most people I know wouldn't.
[ link to this | view in thread ]
Drizzt, 14 Oct 2010 @ 3:25pm

Decline of "landline" connections in the USA?
Hi there,
just out of curiosity: don't you get a free landline number when you sign up for some broadband internet plan? Here (Europe, at least in most western European countries) you get at least on "classic" phone number. Sure they're VoIP but to non-technical people (and billing-wise) it is a landline number.

Cheers,
Drizzt
[ link to this | view in thread ]
MrWilson, 14 Oct 2010 @ 5:29pm

Survey's by nature cannot qualify the data they attempt to get. The most you can honestly say with survey results is that, "out of x number of people surveyed, y number of people answered this way when given these options to this question." Which of course doesn't provide anything meaningful, so you have some biased person use the numbers to support their prejudices while not revealing what actual questions were asked or what the possible answers were.

Honest responses to survey questions are not able to fit within the limitations of any provided possible answers. Not to mention the people answering the survey don't necessarily know what they think anyway.

I'm going to conduct a survey here.

Question: Are you still a cannibal?

Answer 1: Yes.

Answer 2: No.

I'll skip to the projected results. Extrapolating that the x number of people who responded to the survey represent a diverse enough population, I can reliably conclude that 100% of the world's population are cannibals. Imagine that.
[ link to this | view in thread ]
Suzanne Lainson (profile), 14 Oct 2010 @ 5:31pm

But will the cellphone holders vote?
I think it is reasonable to assume that if pollers are only reaching landlines, they are tapping an older, more conservative demographic.

But will the people they aren't reaching get out and vote? If the results are skewed, but those polled are most likely to vote, then their views WILL reflect the final results.

In other words, if you only end up talking to Republicans, but the Republicans are the only ones who vote, then your predictions will be accurate.

So if you feel you are underrepresented in the polls, ignore the polls, but still make sure you vote.
[ link to this | view in thread ]
Anonymous Coward, 14 Oct 2010 @ 7:30pm

Re: Decline of "landline" connections in the USA?
drzzt what your describing, is bundled as an entirely separate service in america, by morons who can't figure out that data=data, (or think we can't either.)
[ link to this | view in thread ]
mrharrysan (profile), 14 Oct 2010 @ 9:05pm

Re: But will the cellphone holders vote?
I agree with your reasoning, but I think the demographic dissonance expressed by the poll figures needs to be scrutinized in detail.
[ link to this | view in thread ]
Suzanne Lainson (profile), 14 Oct 2010 @ 9:47pm

Re: Re: But will the cellphone holders vote?
I agree with your reasoning, but I think the demographic dissonance expressed by the poll figures needs to be scrutinized in detail.

Well, there are at least two different ways to sample if we are talking about political polls.

1. A random sample of all people.
2. A random sample of all potential voters.

If you are doing a random sample of all people and don't have enough cellphone-only people, you don't truly have a random sample.

If, on the other hand, you want a random sample of voters and you find out that cellphone-only people don't vote, then you wouldn't want them anyway. Or, if you did reach them (via cellphone or by mail or in person) and you found out they weren't planning on voting, then you'd toss out their opinions because you were only trying to reach likely voters.

Let me ask you. So what does it mean if current polling under-represents younger, more liberal voters? Do you expect the election results to be more liberal than predicted? Anyone here placing any bets?
[ link to this | view in thread ]
Suzanne Lainson (profile), 14 Oct 2010 @ 10:12pm

Re: Re: Re: But will the cellphone holders vote?
Here's the article that shows the actual results comparing landline only to landline and cellphone.

Cell Phones and Election Polls: An Update - Pew Research Center

Among everyone polled:
For landline only, 46% say Republican and 45% say Democrat.
For landline and cell, 44% say Republican and 47% say Democrat.

HOWEVER, among LIKELY voters:
For landline only, 53% say Republican and 41% say Democrat.
For landline and cell, 50% say Republican and 43% say Democrat.

So while including cellphone voters does increase the percentage of responding Democrats, when you ask who is likely to vote, the Republicans take the lead. The main difference is that it's a 53/41 split based on landline only, while it is a 50/43 spilt when including landline and cell.
[ link to this | view in thread ]
Anonymous Coward, 15 Oct 2010 @ 2:44am

Lies, damn lies and.....
97.37% of statistics are made up on the spot.

Or to quote(ish) the late lamented Mr. Adams, "statistics don't prove anything you didn't already know, except that everyone in the universe has 2.4 legs and own a hyena."

There was a classic tag line in a british cat food add many years ago I always loved "8 out of 10 owners who expressed a preference said their cats preferred it", or to translate, "8 out of 10 people who already actually BUY the stuff find that the cat they feed it to doesn't refuse to eat it."
[ link to this | view in thread ]
antimatter3009 (profile), 15 Oct 2010 @ 6:29am

Re: Anyone Suprised?
I don't disagree in many cases, especially when it comes to influencing elected officials. However, I will say that polls predicting votes and the like can be nicely checked against real results. Most (all?) pollsters and poll analyzers take the error into account when making future predictions and attempt to correct through proper weighting, etc.

Polling is never perfect, and even if you could feasibly poll everybody you'd still see differences as people change their minds or whatever, but some pollsters consistently get pretty close despite these and other issues. That said, as more and more people drop their land lines, I imagine these corrections are going to get more difficult to make and poll results will start to fluctuate more. But hey, maybe not. I'm pretty consistently surprised how close they get even today.
[ link to this | view in thread ]