Lawyer Ranking Site Realizes That It Might Need A Better Ranking System
from the for-example... dept
You may recall that recently there was a class action suit against a new website that claimed to "rank" lawyers based on a variety of factors. The rankings were quite controversial for pretty good reasons. It wasn't at all clear how some were ranked, and seeing some Supreme Court Justices with low rankings raised plenty of questions. The company has now said it's revamping its ranking system, saying it will drop the controversial numerical ranking on lawyers it hasn't collected much info on (such as Supreme Court Justices?). Amusingly (and not very convincingly), the company claims that the changes have nothing to do with the lawsuit, along with some blather about how such sites need to stay "dynamic." Of course, it's still not clear what's illegal about the site. It's simply compiling publicly available data and then coming up with a rank -- the same thing that Google does for websites (instead of lawyers). And, as we've seen from a variety of lawsuits, you can't sue just because you don't like your ranking, since that ranking is considered an opinion.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Reader Comments
Subscribe: RSS
View by: Time | Thread
Why a ranking?
I'm all for collecting statistical data but the problems start when you try to put all those different numbers together into a single all telling score.
[ link to this | view in chronology ]
Re: Why a ranking?
[ link to this | view in chronology ]
Re: Re: Why a ranking?
The 'score' is the only 'easy-to-look-at number' on the whole report
The score is exactly what I was talking about. I guess I should have said "score" instead of "report"
[ link to this | view in chronology ]
My Ranking
And here's the clincher-- I'm not even an active member of the bar any more. I went inactive almost ten years ago when I went into federal law enforcement.
Seems like the site treats no data on a lawyer as a positive so if you're like me and haven't even been actively practicing, you get a high score.
[ link to this | view in chronology ]
Is the lawyer ranking site is just a VC game?
[ link to this | view in chronology ]
Add an Error Estimate
There's nothing inherently wrong with giving a ranking based on limited information--statisticians do it all the time. But you do have to give an explicit indication of the amount of uncertainty in the estimate: less information means greater uncertainty. In a chart plot, these uncertainties are shown in the form of error bars.
If their rankings included such uncertainty estimates, that would give you a much better idea of how seriously to take them.
[ link to this | view in chronology ]