Why Is The Obama Administration Putting Billions Of People At Risk With Patent Pressure From The USTR?

Can Pundits Actually Prognosticate? Answer: Mostly, No

from the nice-to-see-some-lookback dept

Tue, May 3rd 2011 11:03am — Mike Masnick

I've discussed in the past how loath the tech industry often is to ever look back at analyst "predictions." The big research firms come out with all sorts of ridiculous predictions, and no one ever goes back and figures out how accurate they were. It seems that the same thing is frequently true with political prognosticators, so it's interesting to see a Hamilton College public policy class analyze the predictions of 26 political pundits over the period of 15 months (September 2007 to December 2008) and measure how good the pundits were. It turns out most were "no better than a coin toss." I'm not entirely convinced of the methodology, since it seems to make use of some subjective analysis from the description, but for the politically minded, it's at least interesting to note that the most "accurate" pundits all fall on the left of the traditional political spectrum, while the least successful tended to fall on the right. I do wonder how much of that has to do with the timing (the period covered the financial decline and the Presidential election). It would be interesting to see a similar test run during a different period of time as well. I wonder if a similar analysis, say, prior to the election of a Republican president, would have turned up the opposite results. In other words: was there a fundamental quality in the predictions, or was it just that "the winning team" looks smarter in retrospect? Either way, it's still great that people are going back and looking at how well some of these prognosticators did.

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: politics, predictions

21 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

John Doe, 3 May 2011 @ 11:28am

The real lesson here...
The real lesson here is don't listen to prognosticators.
[ link to this | view in chronology ]
Anonymous Coward, 3 May 2011 @ 11:29am

Shouldn't "actually" be changed to "accurately". That would be more in keeping with the substance of the article.
[ link to this | view in chronology ]
fogbugzd (profile), 3 May 2011 @ 11:29am

I always figured that if an "expert" was correct more than 50% of the time, then it was a win. It is easy for experts to get distracted by some detail and miss the big picture which is why it is much easier to be wrong than right. I watched Bill Gates pretty closely when he was still running Microsoft, and it looked to me that he was right a little over half the time. But that was enough to make him a billionaire.
[ link to this | view in chronology ]
Thomas (profile), 3 May 2011 @ 11:33am

"Reality has a well-know Liberal bias." - Colbert
[ link to this | view in chronology ]
- John Doe, 3 May 2011 @ 12:23pm
  
  Re:
  "Liberalism is a mental disease." - forgot who said it. :)
  [ link to this | view in chronology ]
  - Steven (profile), 3 May 2011 @ 12:29pm
    
    Re: Re:
    "If you're not a liberal at twenty you have no heart, if you're not a conservative at forty you have no brain." - Winston Churchill
    
    And if you haven't figured out by now that neither party is actually looking out for you, you just can't be helped - Me
    [ link to this | view in chronology ]
    - John Doe, 3 May 2011 @ 12:37pm
      
      Re: Re: Re:
      And if you haven't figured out by now that neither party is actually looking out for you, you just can't be helped - Me
      
      Very true. Unfortunately we only have the two parties to choose from so you have to pick one.
      [ link to this | view in chronology ]
      - Anonymous Coward, 3 May 2011 @ 12:53pm
        
        Re: Re: Re: Re:
        No I don't
        [ link to this | view in chronology ]
      - Lawrence D'Oliveiro, 3 May 2011 @ 8:28pm
        
        Countries With Civilized (i.e. Proportional) Representation Systems ...
        ... tend to have lots of parties to choose from.
        [ link to this | view in chronology ]
Steven (profile), 3 May 2011 @ 11:37am

Unskilled and Unaware
I strongly suspect one of the major pitfalls of 'professional journalism' is exactly that they are trained in journalism. They then think this training grants them expertise in nearly all other areas of life. I would love to see a world in which journalism was only offered as a secondary major, and journalists reported on their primary fields.

There is a fascinating paper (pdf) here:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.64.2655&rep=rep1&type=pdf

It shows how people tend to vastly overestimate their abilities in an area they are unskilled. If you've never read it you should.

Of course seeing as how I'm unskilled in the field of journalism, it's entirely possible I've fallen into my own trap.
[ link to this | view in chronology ]
- ChurchHatesTucker (profile), 3 May 2011 @ 11:44am
  
  Re: Unskilled and Unaware
  I strongly suspect one of the major pitfalls of 'professional journalism' is exactly that they are trained in journalism. They then think this training grants them expertise in nearly all other areas of life.
  
  This is so true it hurts.
  
  We need to remember that almost all reporters are generalists, which means they're relying on 'sources' to feed them information. Just about anything you read from mainstream sources should be treated as a press release. Because it usually is.
  [ link to this | view in chronology ]
- Stuart, 3 May 2011 @ 12:59pm
  
  Re: Unskilled and Unaware
  Should be the other way around.
  Journalism should be the primary major.
  To report well on a subject you do not need intimate knowledge. Only need to have a firm grasp of how little you really know.
  [ link to this | view in chronology ]
  - Squirrel Brains (profile), 3 May 2011 @ 1:36pm
    
    Re: Re: Unskilled and Unaware
    Seems like most journalists are loosing that grasp (or at least the visible ones)
    [ link to this | view in chronology ]
  - ChurchHatesTucker (profile), 3 May 2011 @ 6:33pm
    
    Re: Re: Unskilled and Unaware
    It's a lot easier to teach journalism to a chemist than chemistry to a journalist.
    [ link to this | view in chronology ]
A Dan (profile), 3 May 2011 @ 11:43am

Typo
"I've discussed in the past how loathe the tech industry often is"

I think you mean "loath".
[ link to this | view in chronology ]
- Ron Rezendes (profile), 3 May 2011 @ 12:40pm
  
  For those who didn't know....
  Loath or Loathe?
  
  Loath is an adjective meaning "unwilling." It ends with a hard th and rhymes with growth or both.
  
  Loathe is a verb meaning "to hate intensely." It ends with a soft th like the sound in smooth or breathe.
  
  Examples: He was loath to admit that he was included in the deal.
  (He was unwilling)
  
  Alex loathes spiders.
  (Hates them intensely)
  
  Source: http://englishplus.com/grammar/00000238.htm
  Copyright�1997-2006 English Plus, All rights reserved.
  [ link to this | view in chronology ]
  - A Dan (profile), 3 May 2011 @ 1:18pm
    
    Re: For those who didn't know....
    If it's unclear, he's affirming my correction.
    [ link to this | view in chronology ]
- Mike Masnick (profile), 3 May 2011 @ 1:57pm
  
  Re: Typo
  I think you mean "loath".
  
  You are correct. Oops. Fixed.
  [ link to this | view in chronology ]
Anonymous Coward, 3 May 2011 @ 12:20pm

"The big research firms come out with all sorts of ridiculous predictions, and no one ever goes back and figures out how accurate they were."

I predict the world will come to an end in 2012.
[ link to this | view in chronology ]
- Anonymous Coward, 3 May 2011 @ 12:26pm
  
  Re:
  (and if this prediction doesn't come true, then I predict no one will ever go back to figure out how accurate my prediction was).
  [ link to this | view in chronology ]
  - Squirrel Brains (profile), 3 May 2011 @ 1:38pm
    
    Re: Re:
    Sorry, just marked my calendar for January 1st, 2013 to go back and figure out how accurate your prediction was.
    [ link to this | view in chronology ]