It's Time We Talk About Getting Rid Of The Bar Exam. And Here's Why.
from the naked-emperor dept
Thousands of law school graduates sat for the bar exam this week. It has long been questionable whether this semi-annual process of testing lawyers is in any way a valid method of determining whether a lawyer is qualified to practice. But the egregious way bar authorities have altered the administration of the test to respond to the pandemic has made it clear that the answer is no.
In theory, how lawyers become licensed to practice shouldn't really be a technology topic. On the other hand, here at Techdirt we do talk about law, and sometimes that means we need to talk about the role lawyers play in our legal system so that law can be effectively and equitably administered in a way that protects liberty. And sometimes that means that we need to talk about how technology may interfere with that role.
We also regularly chronicle when awful lawyers do actually cause real harm, and we cheer when their bad behavior is finally checked. Lawyer licensure is one way that bad lawyers can be checked, and that's why we need to care when the systems we use to license lawyers themselves need to be checked. Especially when, as now, those systems have deployed technology in a way that just further betrays any credible principle underpinning the licensing system.
Even before the pandemic the system was already straining credulity. While on paper it makes sense for bar admissions authorities to test the competency of new lawyers before admitting them to practice, it has been increasingly apparent that the actual exams being administering generally do no such thing, instead testing them on irrelevant knowledge that will be of little use to them in their jobs, and under circumstances that in no way align with those they are likely to experience as part of the actual practice of law. While there are occasional moments of potential utility, the bar exam, on the whole, has become little more than a formal system of hazing that newbie lawyers must endure before gaining entrance to the profession.
And the pandemic has only exacerbated all of those problems. As the coronavirus upended standard testing practices, it had the effect of pulling back the curtain to reveal the defects of the entire system as bar authorities responded to the moment by doubling-down on the worst of it. Because instead of using it as an opportunity to focus on what could potentially be worthwhile about the exam, bar authorities decided to make it an even more arbitrary and impossible obstacle for new lawyers to hurdle.
To sit for the exam, examinees in many states had no choice but to install insecure, invasive, and unstable remote proctoring software, Examsoft, on their own personal machines (and in some cases expend additional resources to even come up with a machine that would be eligible for the software) in order to lock them down so that they could be used for nothing but writing exam essay answers, and surveilling the test takers while they took the test in order to make sure they did nothing but.
I still had my exam sessions crash several times, and at one point my camera turned off 10 minutes into a session. Luckily I realized it, otherwise I have to imagine I would have been flagged and my score invalidated. @ExamSoft @NCBEX
— Bree Philips (@philibree) July 29, 2021
The result of all this distrust by bar authorities that examinees might cheat on their tests was the complete sabotage of their performance on this test by said bar examiners, who had elected to employ this software despite its previous flaky performance.
There was an @examsoft class action in 2015. I got $90. https://t.co/ieMkI1Dv6H
— Whitney Merrill (@wbm312) July 28, 2021
Speaking of trust, it let examinees be deceived that everything would be fine, only to put them through needless stress when they found out it wasn't.
Lol, the CA bar replied to me on Monday, before the exam, and said to follow ExamSoft's advice and use an external camera. Then they emailed me again today, during my second day of testing, to let me know external cameras are not allowed after all. This exam is đź—‘ https://t.co/z4tNiF0oWh pic.twitter.com/WwzBs7LniC
— andrea 🦎 (@taquitararita) July 28, 2021
Mine crashed and it’s literally a brand new computer. I had to purchase it because Examsoft updated the software a week and half prior to the exam and suddenly my computer would no longer work with it. Getting a new one was their only advice. Complete incompetents.
— Jew the Bartman 🌹 (@halfbreedjew) July 28, 2021
And it put all these test takers through this hell for nothing.
Hey @NCBEX and @ExamSoft, how can you guarantee that the *THOUSANDS* of bar exam takers who experienced crashes didn't use the significant unmonitored down time to look up answers? There is absolutely no way to grade this bar exam fairly at this point.
— Everett S (@CornBreadFan2) July 28, 2021
Horror story after horror story after horror story accumulated during the days of the test as examinees, who in many cases were taking the most important and taxing test of their lives, found themselves with frozen machines, busy support lines, and constant reboots chewing up the finite time they had available to complete the test.
Shoutout to @ExamSoft whose software crashed in the middle of my bar exam and now has my laptop locked in a white screen I can’t escape while I wait for 30 minutes on hold for someone to answer. Thank you for wasting a year of prep and 1000’s of dollars.
— natasha c (@TashaC_93) July 27, 2021
The upshot: for many, the ability to finally earn a living in their new career, despite the immense amount of time and money expended, will be derailed by this fiasco, as many a qualified lawyer will unjustly fail the test thanks to the software failing to work.
"Technical issues" meaning that hundreds of people will need to waste another 6 months and several thousand dollars (not to mention lost income) retaking the exam.
(You know, the one they cram test-specific knowledge for, and that *practicing law* is likely to make them forget.) https://t.co/6ruwpd2XHu
— Kat Walsh (@mindspillage) July 29, 2021
Such an outcome is inexcusable, unnecessary, and bad for the profession at large.
I certainly hope @WAStateBar, @JudgeMaryYu, and the other members of the WA Supreme Court are paying attention to what's happening with @ExamSoft and @NCBEX - we cannot rely on shoddy software and uncaring (and incompetent) overseers to determine who joins our profession.
— Alan Mygatt-Tauber (@AMTAppeals) July 28, 2021
It's bad for the profession because it has the effect of locking out scores of people who could actually be good lawyers. Even pre-pandemic, the bar exam had the effect of keeping out smart, capable people who simply don't test well, or who could not afford the extra cost of exam preparation in terms of time or money. The reality is that few, if any, law students graduate from law school ready to sit for the bar exam. Nearly everyone does a separate bar exam course to prepare, which easily adds several thousand dollars to the cost of an already extremely expensive legal education. (Plus it also costs a lot of money to even sign up to take the test in the first place, let alone take the time off to study for it. Google "bar loan" for what a good chunk of law grads need to take out in order to finance the endeavor.) The profession already suffers from a lack of diversity by driving away those who can't afford the enormous cost of pursuing it. Having additional arbitrary hurdles connected with the test only worsens that lack of diversity, and adding even more, through imposing these insidious remote proctoring requirements, will only exclude even more qualified people, and for no good reason.
Because these proctoring requirements are also unnecessary. Treating examinees like presumptive cheaters, especially at the expense of testing their actual competence, does the profession and public a huge disservice. The reality is that the practice of law in the United States is not a memorization game. In fact, relying solely on memory is likely to be malpractice – good lawyering almost always requires looking things up to be sure. The trick is to have enough familiarity with the law to be able to spot the issues that arise and know how to efficiently look up the specific law that applies to them, which are the skills the exam should be testing for. And that means that a timed, open-book test would do a vastly better job meaningfully assessing law graduates on their readiness to be lawyers than this bizarrely patronizing and punitive system currently does.
And it's inexcusable that this is the current state of affairs. It is bad enough that even in normal times bar authorities so heavily policed test takers that they forbade menstruating women from having sanitary supplies available (or forced them to show them off publicly). But the pandemic protocols have completely forsaken any claim on the bar exam being anything close to a socially valuable exercise. Instead these protocols have revealed it to be little more than an arbitrary exercise of power by bar authorities, who seem to be utterly confused about why we have bar authorities in the first place.
The job of bar authorities is to help shore up the administration of justice, not directly undermine it with pointless displays of authority over people's lives and livelihoods. The ghastly irony here is that the very same stakes that have driven bar authorities to automatically suspect every bar candidate of cheating and to impose these onerous remote proctoring rules are the very same stakes that they have shown themselves to be completely indifferent to by imposing these with these very same onerous remote proctoring rules, with potentially devastating and undeserved results to perfectly capable bar candidates' futures. In the wake of these misplaced priorities it is impossible to believe that these same authorities can be qualified custodians for anyone's interests, including the public's. And by so abdicating their legitimacy as stewards of the profession they make it impossible to credibly act as such, no matter how much the public, or the profession, might need them to.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: bar exam, examsoft, lawyers, standardized testing, tests
Companies: examsoft
Reader Comments
Subscribe: RSS
View by: Time | Thread
bar exam test
i took one of those online example tests for shits and giggles. it was only about 1/4 of what the real test would be. with zero law school classes, no pretest studying, just out of the blue take it and see what kind of score i would get. while some of the questions are confusing and i got most of the hearsay questions wrong. i still managed to pass the test with a low passing score. it does help to know something about the law before you start and i did well for an untrained non-lawyer.
once you figure out how to lookup the laws for where ever (location) and what that law is. it makes it easy to form a proper defense/ offence. the rest is just courtroom procedures and filing paperwork.
[ link to this | view in thread ]
The resistance to open-book tests, in most any field, has always struck me as lazy. When you think of a lawyer's office, what do you imagine? For me, I expect to see a bunch of law books and even other laywers, paralegals, etc. that can help out. If they can get me a useful answer in an hour or two, doesn't matter how they did it.
Same thing for a doctor, right? There are always books around, and I'm not going to hold it against a person if they open a few, or consult with another doctor, or refer me to a specialist. Electricians are pretty much expected to own a copy of the electrical code. Engineers and scientists talk amongst themselves all the time.
But if you're running a class of 100 law students and have to evaluate each one for "would they be a useful addition to a law office?", well, now you'll have to get to know 100 people. It seems fair, given the amount they pay to be there. Not as easy, though, as coming up with 100 rote-knowledge multiple-choice questions out of a textbook, and feeding 100 papers into a grading machine.
Gatekeeping is another rant on its own. You know some places require a diploma just to give a haircut? Unless it's done with a straightrazor, or one is also applying potentially dangerous chemicals like peroxides, that's pretty ridiculous.
[ link to this | view in thread ]
Re: bar exam test
I took an online course a few months back. It was open book, open notes, but it was a timed test. Our class instructor said upfront at the beginning of the course that there was certainly a lot of knowledge on the subject, and it would be difficult to memorize everything. Far better than temporarily memorizing a ton of facts and then forgetting them within 6 months, is HOW to look up stuff and then apply it to a complex question to arrive at an answer within a reasonable amount of time.
Of course, I guess it would have been possible for me to just hire a ringer to sit-in for me. There's probably never going to be a way to eliminate all cheating. But it still seems like a better alternative installing malware on your device in order to sit for an exam.
[ link to this | view in thread ]
It seems that many of these kinds of tests aren't designed to test your knowledge or abilities at all, they are designed for maximum revenue. You end up learning the test and not what you will need in your profession if that makes sense.
[ link to this | view in thread ]
Well, Ms Gellis.
You make an excellent case for reworking how remote proctoring is done. You even make a good case for changing the bar exam itself.
It is misfortunate, then, that the title of the essay invokes getting rid of the bar exam entirely as that seems only weakly argued, and doesn't really appear to be the focus of your essay at all. And unfortunately, I don't have enough personal experience with the bar exam and how good (or whether) it acts as an indicator of basic competency, to opine on whether or how the bar exam might be changed (or eliminated).
But remote proctoring the bar exam seems to have done no better than any other remote proctoring: security concerns, false accusations of cheating, true accusations of cheating, ability of the user's computer to run the proctoring software at all. Horror stories about all of these are easily found online, and certainly not limited to the bar exam. Proctorio, ProctorU, even Dartmouth's Canvas have all appeared here in the past.
And for something completely unrelated to the bar exam: I would be interested in your take on the case recently concluded regarding the Texas Bar Association, mandatory dues, and political advocacy, but I'm not sure how it would make it to these pages.
[ link to this | view in thread ]
My familiarity with the subject is mostly colored by my last year on Twitter and listening to a good portion of LawTwitter be advocated toward the view that the Bar Exam does nothing they thought it did. I was a little surprised that this piece didn't even mention in passing that after all the issues with the last 2 exams during the pandemic, the Bar Exam officials in several places messed up scoring and erroneously said some people passed or didn't pass -- which is just a travesty in such a high-stakes test.
Also, if people want to see someone make some good arguments for abolishing the bar exam -- see @LizCGil on Twitter. She's a strong advocate that brings case and argument for why this thing is awful.
[ link to this | view in thread ]
Re: why an extra academic exam ?
Well, stepping back and taking a fresh look at the ritual Bar Exam seems a very good idea.
Why is a J.D from a state certified law school inadequate for basic entry into that state's Bar ?
Either individual state governments do not trust their official law school certifications ... or the state is imposing extra unnecessary requirements on the practice of law.
What critical value-added component does the Bar Exam provide -- that is not/cannot be already supplied by a certified law school ??
This also logically leads to broader and important issues of Occupational Licensing itself.
[ link to this | view in thread ]
Considering some of the stories on Techdirt regarding the antics of some lawyers, it would probably be a great idea to add in a thorough mental exam to the Bar exam. It might, however, reduce popcorn sales.
[ link to this | view in thread ]
Re: Re: why an extra academic exam ?
Why?
Because it's really another one of those self-declared "official" guilds with quasi-governmental authority designed to take money and keep people out. That's why. (And also do jack shit to police their profession once someone is admitted.)
[ link to this | view in thread ]
Lawyers shouldn't be allowed to enact laws by being legislators. It's a conflict of interest.
[ link to this | view in thread ]
Re:
Actually, the term "lawyer" is best applied to one who makes law(s). The proper sobriquet for one who appears in court on behalf of others is "attorney". But if even the legal profession gets it wrong, who am I to pee in their collective Cheerios?
[ link to this | view in thread ]
"instead testing them on irrelevant knowledge that will be of little use to them in their jobs"
This line could apply to so many different things in education. But if there's one thing lawyers love more than money and writing things average people can't understand it's gatekeeping.
[ link to this | view in thread ]
Re:
The anonymous coward pic looks rather similar to a swastika.
[ link to this | view in thread ]
Re: Re:
Why would lawyer best be applied to people who write/pass laws when the term lawmaker exists for them?
The thesaurus also disagrees with you and lists lawyer and attorney as synonyms. I've literally never heard this argument.
[ link to this | view in thread ]
I remember when I took the bar, one of the moderators stood up and told the room that they were looking for “minimal competence.” That really took the pressure off because I figured, “hell, I’m minimally competent.”
Unfortunately, it looks like ExamSoft can’t even meet that standard.
[ link to this | view in thread ]
Re: Re:
It's a gravatar, generated off your IP/email.
[ link to this | view in thread ]
The bar exam.
I had a friend take in in person test, years ago, and her computer froze. The 'technicians' were even worse than the Geek Squad.
However, studying for the exam and taking the exam does cover some material not covered in the law school classes. The MBE covers a large area of law and are designed to trick you on some of the subtle exceptions. The essay questions actual make you identify the issues, know the rules, and apply reasoning to the rules. They also have a practice test where you are provided a closed library so you don't have to memorize all of the rules for it.
[ link to this | view in thread ]
Huh, I wonder if students can sue the various bars who still use this crap.
IIRC there were multiple test takers who discovered that the testing package left their machines open to hackers who helped themselves to all sorts of PII.
The warnings were so strong there were those taking the test who bought a brand new computer to take the test knowing they would wipe it at the end & never use if themselves because it cost to much to certify it would ever be safe to use.
Given the number of lawyer who represented Trump & his claims across the nation perhaps one should ask if the Bar has become worthless, they do not deter the unfit & only take action to reign them in well after the barns burned down & been rebuilt 3 times.
[ link to this | view in thread ]
Re: Re: Re:
I don't agree with sumgai, but "attorney" and "lawyer" are not true synonyms. "Attorney" refers to a person who has the authority to represent the interests of another, as in "power of attorney". Lawyer generally refers to a person who practices law; when representing a client they would indeed be an "attorney" (though many attorneys are not lawyers).
[ link to this | view in thread ]
Re:
I believe that Examsoft has the standard arbitration and waiver of all liability against them as part of the terms of use of the software.
Where this is required software for the bar exam, one could argue that it is a contract of adhesion and these provisions are not enforceable.
[ link to this | view in thread ]
Distant work area administrations exist in a huge swath of reach and it must be ideally chosen by the prerequisite of the client. https://aminserve.com/blog/buy-rdp-bitcoin The Desktop moreover incorporates various scriptable UIs to create distinctive virtual channels inside the far off work area administration.
[ link to this | view in thread ]
Re: Re: bar exam test
"open book, open notes"
That means that the test you took was intended to test you on your ability to use available documentation, not your ability to recall certain facts. Which is great, that type of testing won't work everywhere. Sometimes you really do just want to test that the subject has gathered a basic understanding of the subject, and you can't do that if they can just copy a description from another source.
[ link to this | view in thread ]
Absolute bunk
I completely disagree with this drivel of a comment. It does nothing but complain. I am an attorney and took the bar. I found that it challenged me to be familiar with the subjects I would likely encounter. The test was grueling - 8 hours/day for 3 days. Finger print and facial recognition proof required to enter and on demand. Any equipment used had to have no memory capacity. I sat with some 9,000 others in a huge hall. A number we’re forced to leave for apparent cheating.
They did the best they could during the pandemic. You did not have to take the exam when and as you did. You could have gone to any state you qualified for. You could have delayed. Stop complaining so much and get on with it.
[ link to this | view in thread ]
Re: Absolute bunk
Is that how we got geniuses like Andrew Crossley and Paul Hansmeier?
[ link to this | view in thread ]
Re: Absolute bunk
That explains what you've been up to since you were Prime Minister years ago.
[ link to this | view in thread ]
If you need to know how worthless the test is, just remember that Frank Abagnale passed it without ever going to law school
[ link to this | view in thread ]
Re: Re: Re: why an extra academic exam ?
Yes. Hence the important overall issue of corrupt Occupational Licensing systems.
[ link to this | view in thread ]
Re: Absolute bunk
... so you attended an inferior law school that did NOT 'familiarize you with the subjects you would likely encounter'.
Must be many lawyers who were cheated like this by law schools -- and needed remedial legal education via state Bar Exams.
Also seems an excellent idea then to ensure all lawyers maintain professional currency by requiring the Bar Exam every 5 years for state licensing renewal.
[ link to this | view in thread ]
Re: Re: why an extra academic exam ?
In fairness, most lawyers (and law students) think law school is a waste of time.
[ link to this | view in thread ]
Re: Re: Absolute bunk
Funny how every adult who fervently defends putting children through harsh examinations suddenly lose all enthusiasm when it's suggested that they sit through the same tests to brush up on their knowledge.
[ link to this | view in thread ]
Re:
"His claim that he passed the Louisiana bar exam and worked for Attorney General Jack P. F. Gremillion was debunked by several journalists in 1978."
https://en.wikipedia.org/wiki/Frank_Abagnale
[ link to this | view in thread ]
Re: Re: Re: bar exam test
No. You still have to issue-spot and know where to look for the rules, which won't work out well with a ticking clock if you don't have enough of a mastery of the subject to quickly know where to look. And you also still need to be able to write a cogent analysis, which open notes won't help you with there either.
[ link to this | view in thread ]
Re:
Yes, and not in the way you think!
IANAL, but my profession does the same exact thing: guarantee a limited supply of practitioners. The stupid thinking goes that if we allow every Tom, Dick and Henrietta in, then there will be too many of us, and that sweet, sweet contracting cash will go down as supply goes up.
The downside is
a) the rent is just too damn high
b) the next breakthrough in the field might not happen because the gatekeepers filtered out the one person who could've done it today
c) debt and misery
d) ???
e) Profit!
[ link to this | view in thread ]