Shouldn't Patent QA Specialists Get Things Right More Than 75% Of The Time?

from the even-25%-sounds-high dept

Thu, Dec 18th 2008 12:12pm — Mike Masnick

A recent lawsuit sheds even more light on just how poor quality control is at the US Patent & Trademark Office. The lawsuit specifically was over the firing of a quality assurance specialist, who's supposed to review patent examiner decisions to determine if errors were made in granting or rejecting claims. The guy was fired after it turned out that a random review showed his reviews erred 35% of the time. The guy complained that it was just a random sample rather than looking across his entire body of work, but that's not all that interesting here. What's more interesting is that apparently the "reasonable" cutoff for such QA specialists is a 25% error rate. Considering that their entire job is supposed to be double checking the work of patent examiners, you would think that getting one in four claims reviewed wrong would be ringing some pretty big alarm bells concerning the quality of any patent. No wonder so many patents are adjusted when re-examined. Even worse, the guy claims that his 35% error rate wasn't really that bad, saying that his colleagues often erred 45 to 50% of the time. What sort of QA is it that can barely QA itself?

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: errors, patent office, patents, qa

14 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

Xyro TR1 (profile), 18 Dec 2008 @ 12:18pm

Think about this
If you only scored a 75% on your tests throughout school, and sometimes dipped to 65%, how far would you get in life?
[ link to this | view in chronology ]
- wallaka, 18 Dec 2008 @ 12:35pm
  
  Re: Think about this
  Why, you could become the President of the US!
  [ link to this | view in chronology ]
- Anonymous Coward, 18 Dec 2008 @ 3:23pm
  
  Re: Think about this
  Youd have a 1st Degree from university
  [ link to this | view in chronology ]
- Anonymous Coward, 18 Dec 2008 @ 6:36pm
  
  Re: Think about this
  If you only scored a 75% on your tests throughout school, and sometimes dipped to 65%, how far would you get in life?
  
  You might end up like poster #3.
  [ link to this | view in chronology ]
- Wiley, 18 Dec 2008 @ 10:11pm
  
  Re: Think about this
  How about President?
  [ link to this | view in chronology ]
- boost, 19 Dec 2008 @ 7:04am
  
  Re: Think about this
  Collegiate testing rarely represents any sort of reality.
  [ link to this | view in chronology ]
Jason, 18 Dec 2008 @ 12:30pm

Scores
It's not really comparable to test scores in school, as schools have every class of people in them, and you are not paid for it.

It would be more like if a teacher had a 65-75% success rate at grading papers.
[ link to this | view in chronology ]
Anonymous Coward, 18 Dec 2008 @ 12:43pm

Take a trip over to Patently-O and you will learn that the story is not as sensational as the headline suggests.
[ link to this | view in chronology ]
- Anonymous Coward, 18 Dec 2008 @ 12:52pm
  
  Re:
  I hit submit before I realized I was not finished.
  
  Continuing along...
  
  I agree it is a tedious exercise, but a quick read of the CAFC decision (which, by the way was not a patent case, but a case under the Merit System Protection Board) and a few of the relevant comments seem to suggest that only a very few (16) of specific types of his work were reviewed; i.e., cases in which he agreed that an examiner was correct in allowing an application to pass to issue. Assuming this was in fact the case, the difficulty in situations such as this is that reasonable minds can differ over what we would like to be an objective process, but which in fact is laden with subjectivity. Of course, a small sample doesn't help either.
  [ link to this | view in chronology ]
  - Anonymous Coward, 18 Dec 2008 @ 2:39pm
    
    Re: Re:
    read the full article. Mike mentions that, and then continues to say that isn't the important part, the important part is that the cut-off is 25%. that is pretty low standards for something like this.
    [ link to this | view in chronology ]
    - Anonymous Coward, 18 Dec 2008 @ 3:06pm
      
      Re: Re: Re:
      Merely FYI, I did read the full article, all the comments, and the court decision. As a prior poster noted, quality control reviews are generally limited to those cases in which a patent examiner has determined that one or more claims are allowable and the application can proceed to issue as a patent. As the prior poster also noted, a majority of applications are rejected, and an applicant who disagrees can file an appeal.
      
      The important point to be made is that the review process utilized here to determine if the individual was or was not correct is quite subjective. Given the same set of facts, other persons could quite reasonably come to an opposite conclusion. As nice as it would be for the process of examining an application to be totally objective, that is simply impossible to achieve.
      [ link to this | view in chronology ]
Cipher-0, 18 Dec 2008 @ 1:27pm

Quality control? We've heard of it.
I'm glad they have such good quality control - the quality might get out of control otherwise!
[ link to this | view in chronology ]
Ex-Examiner, 18 Dec 2008 @ 2:05pm

Quality control checks are mostly done for allowed cases. Considering most cases an examiner deals with result in rejections, and only a very limited number of these cases are actually allowed, there may or may not be a problem here.

I think the quality control for rejections is the board of appeals, and rightly so.
[ link to this | view in chronology ]
gene_cavanaugh, 20 Dec 2008 @ 12:31pm

Patent Q&A
First, the IP status in this country is in disrepair; granted. We need to stop catering to the wealthy (the standard "Welfare for the Wealthy" we find with our system; though note, this is a problem with CONGRESS, not the USPTO).
So, there is a sound fundamental reason to be skeptical of the system as it exists.
We need to change to "first to file", and require that any prospective IP grant (especially patents) be subjected to review by the public before grant.
BUT, at least TRYING to improve quality with a QA function, and enforcing standards in such a function, is a GOOD thing! If it is so hard to do that there is a high error rate (and to me, 25 percent is a low error rate), so be it - it is better to try and fail than simply "sucking your thumb".
Further, mindlessly criticizing the effort because of personal bias is unwarranted! Stop "pegging" on your personal peeves!
[ link to this | view in chronology ]