Embarrassed Pollster Tries To Backtrack On Study That Says The "Wrong" Thing

from the fun-with-numbers dept

Wed, Mar 22nd 2006 2:12am — Mike Masnick

One of the most annoying things about sponsored "studies" is that it's obvious that the companies running these studies set them up in a way to clearly favor those who are paying for the study. It's a big scam, but it works because the press picks up on the results, but rarely digs into the methodology -- and the sponsorship info is often buried or not clearly explained. That's why it was so interesting earlier this week to see Michael Geist dig into a recording industry sponsored study, where the actual results seemed to disprove many of the points the industry keeps repeating concerning file sharing. Obviously, that looks quite bad for the polling firm in question -- who got their money to make the recording industry look good, not bad. So they apparently put out an 11-page rebuttal that not only insults Geist for being "impertinent and presumptuous," but also accuses him of distracting everyone from "the serious business at hand." Of course, many would point out that Geist is actually shining a light on the serious business at hand -- the one that the industry keeps trying to obfuscate with scary stories and misleading studies. Geist wastes no time in countering the points made by the polling firm -- but even more interesting is that more people are digging deeper into the methodology and revealing just how leading the questions were on the study. That's how these studies usually work. Based on how you word the questions, you can get people to "agree" to almost anything. Mix that with selective stats or descriptive words like "many" when talking about fewer than half, and suddenly a study can say anything, even if reality doesn't support it.

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

8 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

giafly, 22 Mar 2006 @ 2:41am

Leading Questions
The sponsored survey appears to have asked biased questions. Here are some amusing examples (via TFA) from the opposite viewpoint.

"Do you agree that Canadian law should be amended so as to allow record companies to sue your teenage children for $150,000 in damages for each song they download onto your home computer? ...

Is it personally important to you that copyright law unduly favour existing business models by suppressing marketplace innovation?

Do you agree or disagree with this statement: Recording companies or other media firms should be allowed to place potentially damaging software in my computer, without my consent, using their CDs, DVDs, or other digital entertainment products."
Copyright Watch
[ link to this | view in chronology ]
Flanker, 22 Mar 2006 @ 2:53am

Pollara could use a proofreader
There are numerous misspellings and grammatical errors in their 11-page response. They saved the best for last, though. The final paragraph states:

POLLARA has been firm in applying a policy that insists all research undertaken in connection with pubic processes, whether flattering to our clients or not, is made public.

Pollara can stay far, far away from my pubic process. Trust me, it's plenty flattering to its clients.
[ link to this | view in chronology ]
CloakedMirror, 22 Mar 2006 @ 5:49am

Just goes to show...
There are lies. There are damned lies. And of course, there are statistics. Reminds me of the mathematical trick, performed by one of my professors, that showed 1 = 0.
[ link to this | view in chronology ]
Wolfger, 22 Mar 2006 @ 6:34am

Lies, Damn Lies, and Statistics
Misleading poll questions, selective analysis, and sometimes even doctored numbers are what I would call common practice. I think the only thing noteworthy here is that somebody actually caught it.
[ link to this | view in chronology ]
discojohnson, 22 Mar 2006 @ 7:04am

Missed point
I think the point is not that Geist was misleading, but that the recording industry got played a fool by having their same misleading statement tactic used against them. It's not about a pollster getting caught and showing that people doctor things, it's about bringing further exposure to how the recording industry operates. So what if he put a slant on it? At least it's another person showing how shitty the recording industry is being to defend an obsolete business model (in their market)
[ link to this | view in chronology ]
a_pseudonym, 22 Mar 2006 @ 9:46am

heh. they seem...
It's kind of odd that they use the same numbers to come to the same conclusions with such emphatically different wording. If 1/3 of the music on my pc is illegal, then 2/3 of it is legal. right?
[ link to this | view in chronology ]
Jared, 22 Mar 2006 @ 11:59am

Fuck those polls
lol ive always hated polls.,it seems retarded to me when you have people come out and say "Hi im a democratic pollster."
[ link to this | view in chronology ]
hal9001, 23 Mar 2006 @ 6:46pm

Lieing with Statistics
One of the prior posters stated that it is easy to lie with statistics. Over 50 years go, there was a book on this topic called "How to Lie with Statistics" (the author's name was Huff). It is still in print (at least as a reprint of the original). Check it out via your local library or Amazon.
[ link to this | view in chronology ]