It's Time We Talk About Getting Rid Of The Bar Exam. And Here's Why.

from the naked-emperor dept

Thousands of law school graduates sat for the bar exam this week. It has long been questionable whether this semi-annual process of testing lawyers is in any way a valid method of determining whether a lawyer is qualified to practice. But the egregious way bar authorities have altered the administration of the test to respond to the pandemic has made it clear that the answer is no.

In theory, how lawyers become licensed to practice shouldn't really be a technology topic. On the other hand, here at Techdirt we do talk about law, and sometimes that means we need to talk about the role lawyers play in our legal system so that law can be effectively and equitably administered in a way that protects liberty. And sometimes that means that we need to talk about how technology may interfere with that role.

We also regularly chronicle when awful lawyers do actually cause real harm, and we cheer when their bad behavior is finally checked. Lawyer licensure is one way that bad lawyers can be checked, and that's why we need to care when the systems we use to license lawyers themselves need to be checked. Especially when, as now, those systems have deployed technology in a way that just further betrays any credible principle underpinning the licensing system.

Even before the pandemic the system was already straining credulity. While on paper it makes sense for bar admissions authorities to test the competency of new lawyers before admitting them to practice, it has been increasingly apparent that the actual exams being administering generally do no such thing, instead testing them on irrelevant knowledge that will be of little use to them in their jobs, and under circumstances that in no way align with those they are likely to experience as part of the actual practice of law. While there are occasional moments of potential utility, the bar exam, on the whole, has become little more than a formal system of hazing that newbie lawyers must endure before gaining entrance to the profession.

And the pandemic has only exacerbated all of those problems. As the coronavirus upended standard testing practices, it had the effect of pulling back the curtain to reveal the defects of the entire system as bar authorities responded to the moment by doubling-down on the worst of it. Because instead of using it as an opportunity to focus on what could potentially be worthwhile about the exam, bar authorities decided to make it an even more arbitrary and impossible obstacle for new lawyers to hurdle.

To sit for the exam, examinees in many states had no choice but to install insecure, invasive, and unstable remote proctoring software, Examsoft, on their own personal machines (and in some cases expend additional resources to even come up with a machine that would be eligible for the software) in order to lock them down so that they could be used for nothing but writing exam essay answers, and surveilling the test takers while they took the test in order to make sure they did nothing but.

The result of all this distrust by bar authorities that examinees might cheat on their tests was the complete sabotage of their performance on this test by said bar examiners, who had elected to employ this software despite its previous flaky performance.

Speaking of trust, it let examinees be deceived that everything would be fine, only to put them through needless stress when they found out it wasn't.

And it put all these test takers through this hell for nothing.

Horror story after horror story after horror story accumulated during the days of the test as examinees, who in many cases were taking the most important and taxing test of their lives, found themselves with frozen machines, busy support lines, and constant reboots chewing up the finite time they had available to complete the test.

The upshot: for many, the ability to finally earn a living in their new career, despite the immense amount of time and money expended, will be derailed by this fiasco, as many a qualified lawyer will unjustly fail the test thanks to the software failing to work.

Such an outcome is inexcusable, unnecessary, and bad for the profession at large.

It's bad for the profession because it has the effect of locking out scores of people who could actually be good lawyers. Even pre-pandemic, the bar exam had the effect of keeping out smart, capable people who simply don't test well, or who could not afford the extra cost of exam preparation in terms of time or money. The reality is that few, if any, law students graduate from law school ready to sit for the bar exam. Nearly everyone does a separate bar exam course to prepare, which easily adds several thousand dollars to the cost of an already extremely expensive legal education. (Plus it also costs a lot of money to even sign up to take the test in the first place, let alone take the time off to study for it. Google "bar loan" for what a good chunk of law grads need to take out in order to finance the endeavor.) The profession already suffers from a lack of diversity by driving away those who can't afford the enormous cost of pursuing it. Having additional arbitrary hurdles connected with the test only worsens that lack of diversity, and adding even more, through imposing these insidious remote proctoring requirements, will only exclude even more qualified people, and for no good reason.

Because these proctoring requirements are also unnecessary. Treating examinees like presumptive cheaters, especially at the expense of testing their actual competence, does the profession and public a huge disservice. The reality is that the practice of law in the United States is not a memorization game. In fact, relying solely on memory is likely to be malpractice – good lawyering almost always requires looking things up to be sure. The trick is to have enough familiarity with the law to be able to spot the issues that arise and know how to efficiently look up the specific law that applies to them, which are the skills the exam should be testing for. And that means that a timed, open-book test would do a vastly better job meaningfully assessing law graduates on their readiness to be lawyers than this bizarrely patronizing and punitive system currently does.

And it's inexcusable that this is the current state of affairs. It is bad enough that even in normal times bar authorities so heavily policed test takers that they forbade menstruating women from having sanitary supplies available (or forced them to show them off publicly). But the pandemic protocols have completely forsaken any claim on the bar exam being anything close to a socially valuable exercise. Instead these protocols have revealed it to be little more than an arbitrary exercise of power by bar authorities, who seem to be utterly confused about why we have bar authorities in the first place.

The job of bar authorities is to help shore up the administration of justice, not directly undermine it with pointless displays of authority over people's lives and livelihoods. The ghastly irony here is that the very same stakes that have driven bar authorities to automatically suspect every bar candidate of cheating and to impose these onerous remote proctoring rules are the very same stakes that they have shown themselves to be completely indifferent to by imposing these with these very same onerous remote proctoring rules, with potentially devastating and undeserved results to perfectly capable bar candidates' futures. In the wake of these misplaced priorities it is impossible to believe that these same authorities can be qualified custodians for anyone's interests, including the public's. And by so abdicating their legitimacy as stewards of the profession they make it impossible to credibly act as such, no matter how much the public, or the profession, might need them to.

Filed Under: bar exam, examsoft, lawyers, standardized testing, tests
Companies: examsoft

Why Softbank Patent Troll's Promise Of 'Royalty Free' License On Theranos Patents For COVID-19 Is Bullshit

from the an-offer-everyone-should-refuse dept

As noted earlier this week, a Softbank-owned patent troll, Fortress Investment Group, spun up a shell company Labrador Diagnostics this month, and days after the company came into existence, it sued a company working on COVID-19 diagnostics, claiming that the technology it used for those tests violated the patents it had bought from Theranos -- the sham of a company that went down in flames, and whose founders are still facing federal charges for fraud. Oh, and Fortress/Labrador/Softbank was using the law firm Irell & Manella, the same law firm that once sued on behalf of a monkey over copyright. On a related note, the current head of the US Patent Office, Andrei Iancu, used to be the managing director of that law firm, and to this day insists that there's no such thing as a patent troll. He should perhaps talk to his former partner, Morgan Chu.

As you probably heard (because Irell & Manella made sure to reach out to everyone mocking them for this lawsuit), within hours of my post going viral, "Labrador Diagnostics" (again, a company that literally did not exist a few weeks ago) put out a press release saying that (1) it didn't know that BioFire was working on a COVID-19 test (which is questionable, given that the Wall Street Journal and other publications had mentioned it, but I guess Labrador didn't even exist when that article came out, so...), and (2) that it had "offered" a royalty-free license on COVID-19 tests once it realized. As we pointed out, the details and conditions of that "offer" still have not been made clear.

However, as Josh Landau at Patent Progress notes, even if Labrador/Fortress/Softbank is actually offering such a license, it's completely worthless, because the lawsuit is still claiming patent infringement on the machine BioFire is using to run the diagnostic:

While we can acknowledge that Labrador has made this commitment, it’s difficult to credit the commitment given that the commitment doesn’t seem to be a meaningful one. Suing a company that makes (extremely complicated) “razor handles” and “razor blades”, Labrador has essentially offered to freely license COVID-19 blades—but it’s still trying to block sales of the razor the blades require....

That’s the thing—Labrador’s offer isn’t meaningful in practice.  BioFire’s product, FilmArray, uses a technology called PCR (polymerase chain reaction).  And their product is pretty cool.  Essentially, FilmArray is a two-piece system—a durable test device and a consumable test pouch containing test-specific reagents.  You take the pouch, inject the sample to be tested into the chamber, and stick the pouch into the test device. The test device does some complex mechanical and biomolecular magic—PCR—and delivers a test result for the doctor.  (And as Mike noted, the patent isn’t on the biomolecular magic—it’s on using a barcode on the test pouch to trigger the PCR reaction.)

But in BioFire’s system, no matter what disease you’re testing for, you use the same test device.  The test-specific aspects are entirely contained in the pouch—a respiratory pouch, a gastrointestinal pouch, etc.  Each one tests for a couple dozen diseases. A respiratory pouch might check for COVID-19, common flu strains, and pneumonia all at once.  And what Labrador is seeking to block is the whole system, not just the pouches.

So an offer to allow a royalty-free license for the COVID-19 test appears to be meaningless—they’re still trying to block the technology that’s needed to run the COVID-19 test. A razor blade without a handle to hold it isn’t much use. The offer would seem to be made purely for the sake of making Labrador look less like ghouls trying to benefit from a pandemic—in practice, Labrador’s injunction request would still effectively shut down COVID-19 testing or make it more expensive. Even though it’s theoretically offering a royalty-free license to test for COVID-19.

Got that? Saying that you'll grant a royalty-free license on one part of the system, but not the part needed to make that "royalty-free" part work, is still seeking to block the use of COVID-19 tests. The lawsuit still seeks an injunction on the testing device, even if it will let COVID-19 pouches operate royalty free. Even worse, we don't know (and Labrador has so far refused to reveal the details of the "offer") what the terms of the offer are, and you can almost guarantee that if BioFire "accepts" the deal that would be used against them, as "evidence" that they admit the patents are valid, since they agreed to a "license" for one product. It's sham deal.

As Landau further notes, this is still likely to create a real chilling effect on other test developers as well:

Second, the offer of a license in one instance doesn’t require the patent owner to offer that license to others.  (And as we’ve seen with Qualcomm, even a legal commitment won’t stop a determined patent holder from trying to avoid offering licenses they’ve committed to offer.)  If you’re a medical startup thinking about developing a COVID-19 test, you could invest in that—and hope that Labrador licenses you royalty-free—or you could spend your money somewhere without the risk of a well-financed patent troll coming after you.  That risk will chill investment in public health and innovation, even though this particular case might be licensed.

Separately, Landau highlights another detail in all of this: how Fortress/Softbank got the patents in the first place. It wasn't a straight up "oh we'll buy your patents," but rather a more complicated deal in which Fortress lent money to Theranos, and when the company defaulted, took the patents:

As reported in John Carreyrou’s Bad Blood, Fortress loaned Theranos money to help them wind down, securing the loan by stating that if Theranos failed to make specified payments, then Fortress would take ownership of the patents.

Unsurprisingly, the medical device company which didn’t have a functional medical device failed to make those payments, and Fortress took the patents.

So, no, Morgan Chu and Irell & Manella should not be off the hook for this blatant pandemic profiteering. They're engaging in the worst kind of patent trolling: ones that will create a real chilling effect and potentially block the ability to create COVID-19 tests at a time when millions of lives literally depend on those tests.

Filed Under: covid-19, diagnostics, patent troll, patents, tests
Companies: biofire, fortress investment group, irell & manella, labrador diagnostics, softbank

How To Improve Online Comments: Test Whether People Have Read The Article Before Allowing Them To Respond

from the probably-asking-too-much dept

For a while now, Techdirt has been writing about the decision by some sites to stop allowing readers to make comments on articles. We've pointed out that's pretty regrettable, especially when it's couched in insulting terms of "valuing conversations" or building "better relationships." Dropping comments is a lazy response to a real and challenging problem: how to encourage readers to engage in meaningful ways.

As well as a natural tendency for people to write hurtful or insultings things that they probably wouldn't say to each other face-to-face, there's another problem: the rise of Internet troll factories whose entire purpose is to flood sites with propaganda in the form of comments that espouse a particular viewpoint. As we noted recently, Google is looking to use machine learning technology to help identify and then deal with toxic comments:

a publisher could flag comments for its own moderators to review and decide whether to include them in a conversation. Or a publisher could provide tools to help their community understand the impact of what they are writing -- by, for example, letting the commenter see the potential toxicity of their comment as they write it. Publishers could even just allow readers to sort comments by toxicity themselves, making it easier to find great discussions hidden under toxic ones.

As Google itself admits, the issue is "about more than just improving comments. We hope we can help improve conversations online." A rather clever way to do that has been devised by NRKbeta, the technology site of the Norwegian government-owned radio and television public broadcasting company, NRK. Here's the basic idea (via Google Translate):

a small [on-screen] module is presented to you as a reader with three questions from the article that you must answer in order to be able to contribute to the discussion.

Actually reading the article before you comment on it -- pretty revolutionary, no? NRKbeta realizes that it's not a perfect solution:

We know of course that it is possible to "cheat" with these questions by searching the text above [the on-screen module], and that using this approach it cannot be guaranteed that everyone actually read the article, but we still think it's worth the experiment.

It's hard not to agree, because it tries to tackle one of the root causes of comments that add nothing to the conversation -- a failure to read what the article said -- by making it a pre-requisite before you can add your own thoughts. It also has the virtue of being extensible in various ways. For example, there could be more than three questions in the pop-up box, and your comment's place and prominence in the conversation could be determined by how many you get right. This might allow the thoughts of more engaged readers to bubble naturally to the top of the conversation. The fact that the code for the feature has been released as free software makes experimentation even easier. NRKbeta's idea certainly seems a better approach than simply giving up and removing comments altogether.

Follow me @glynmoody on Twitter or identi.ca, and +glynmoody on Google+

Filed Under: comments, knowledge, tests, trolling
Companies: nrkbeta

Licensing Boards Think Studying For A Test Is Copyright Infringement, Forbid Memorization Of Material

from the 'all-applicants-must-have-brain-wiped-before-leaving-testing-area' dept

Today's copyright-induced stupidity is brought to you by… a whole host of regulatory institutions. An anonymous Techdirt reader sent in a pointer to this ridiculous warning that greets those accessing the National Association of Legal Assistants practice tests. (Press "Sign In" to view the legal threats pop-up.)

These online practice tests and all items contained herein are protected by federal copyright law. No part of this examination may be copied, reproduced or shared in any manner, in part or whole, by any means whatsoever, including memorization or electronic transmission.

While I realize there have been several attempts to broaden the coverage of copyright and extend its length towards the far end of perpetuity, I was unaware that federal copyright law now provides remedies for the creation of infringing memories.

This would be merely inane (but still noteworthy), if this stupidity began and ended with the NALA's stern warning. But a search for that wording finds examples elsewhere.

Here's the National Board of Chiropractic Examiners making the same ridiculous claim:

These sample tests and all items contained herein are protected by federal copyright law. No part of these tests may be copied, reproduced, or shared in any manner, in part or in whole, by any means whatsoever, including memorization or electronic transmission.

Here's Michigan State University going one step further than the previous two and basically stating that studying for a test is copyright infringement.

The examination and the items contained therein are protected by copyright law. No part of this examination may be copied or reproduced in part or whole by any means whatsoever, including memorization, note-taking, or electronic transmission.

Who else thinks copyright law forbids the storage of infringing copies in the original cloud storage system, the human brain?

• American Board of Perianesthesia Nursing Certification
• Texas Pharmacy Technician Certification Board
• American Association of Veterinary State Boards
• Oncology Nursing Certification Corporation
• National Commission for Health Education Credentialing, Inc.

The list goes on from there, but as you can see, anyone offering some sort of expensive testing process/licensing claims you can't memorize their tests. How they aim to prevent this remains unanswered. Presumably, if someone duplicates the test from memory and offers these materials to others, they would pursue this as infringement.

It's clear they're trying to head off students selling questions and answers, something that could undermine the certification system. (Of course, these agencies could change their questions from year-to-year to head this off, but that would be far more difficult than just smacking around applicants with highly-dubious legal language.) But claiming that memorization is forbidden under copyright law is just idiotic. Unauthorized duplication and distribution might be, but creating cached copies in your mind while studying certainly isn't. It's an integral part of studying for tests. Throughout their educational existence, students use memorization to become familiar with the subject matter, and now, when they attempt to apply that in hopes of receiving certification, they're greeted with notices that say this practice is against the law.

Then there's the question of fair use, that awkward part of copyright protection that many rights holders tend to ignore. A few sample questions being reproduced would be an obvious case of fair use, but according to the (literally) "don't even think about it" warnings, this would also be an actionable offense.

A better solution would be to simply strike that asinine wording and deal with actual duplication of test materials that stretch beyond the limits of fair use. Those performing these acts of infringement are generally in it to profit from those who can't be bothered to learn the material. Chances are, most people wouldn't view the edited legal preamble as providing a but-I-can-store-it-in-my-brain loophole. The same goes for note-taking, which a few of these notices also single out. The problem isn't the person's notes, which are still personal. It's the duplication and distribution to the public. You can still keep people from taking notes during the actual tests, but you certainly can't keep them from remembering what they've seen.

Filed Under: copyfraud, copyright, licensing, memorization, studying, tests

DailyDirt: Comparing US Schools To...

from the urls-we-dig-up dept

The US education system is constantly being compared to other school systems -- with the underlying theme that the US might copy some of the best practices and put them to use for our kids. Sure, it's good to take a look around and see if there's a better way to teach kids, but some practices might be difficult to adopt (eg. eliminating long summer vacations). Here are just a few links on what we might learn from foreign school systems.

• Japan's education system may teach us that instituting strict standards testing is unproductive. Teaching to the test and similar practices has been blamed for creating a less innovative culture and students who are less prepared to adapt to changes. [url]
• Another suggestion is to pay teachers more, so that teachers have resources and incentives to become better teachers. Countries like South Korea, Singapore and Finland pay their teachers well and provide social prestige for educators. [url]
• The role of American parents in their kids' primary school education differs somewhat widely versus Chinese parents. One big difference seems to be that Chinese parents blame their kids' lack of effort, but American parents more often blame the lack of ability of the child, teacher or school. [url]

If you'd like to read more awesome and interesting stuff, check out this unrelated (but not entirely random!) Techdirt post via StumbleUpon.

Filed Under: china, education, japan, lessons, parents, schools, south korea, standardized testing, students, teaching, tests

DailyDirt: Can Computers Grade Written Essays?

from the urls-we-dig-up dept

Technology aimed at education could really benefit an incredible number of students by making classes and learning (potentially) a more pleasant and efficient experience. Computers can't replace a really good human teacher, but they can make it easier for good human teachers to reach a vast audience of students. Massively open online courses (MOOCs) promise to change how education works, but there are some technological tools that might be missing. It's pretty straightforward to test students on math problems in an automated way, but grading essays is a much more daunting problem. There have been some calls for automated grading software from various organizations (like the Hewlett Foundation). But at the same time, the National Council of Teachers of English argues that computers simply can't grade essays. Here are just a few more links on this debate over the use of algorithms over English professors (or grad students).

• EdX, the non-profit started by Harvard and MIT, is releasing some software to automagically grade human-written essays. Some see this software as just another tool for educators to use for more immediate feedback to students, while others are worried that these algorithms will be used incorrectly and lead to disastrous educational policies and outcomes. [url]
• There are studies that show algorithms are statistically comparable to humans when it comes to ranking essays on a 5 point scale. There are things machines can do better and things humans do better -- just make sure you know the differences and automated essay grading can be done productively in the right context. [url]
• Automated essay readers can grade 16,000 essays in 20 seconds. The Educational Testing Service is testing out automation, so students may soon be facing algorithmic grading for their college entrance exams. [url]
• Grading a few sentences can be harder than it might look. Professional (human) teachers are obviously better at interpreting the insights and ideas behind the words a student writes, but computers scale much better and never tire of horrible spelling mistakes or misplaced modifiers.... [url]

If you'd like to read more awesome and interesting stuff, check out this unrelated (but not entirely random!) Techdirt post via StumbleUpon.

Filed Under: ai, algorithms, artificial intelligence, automation, education, edx, essays, ets, exams, grading, mooc, nlp, tests
Companies: harvard, hewlett foundation, kaggle, mit

DailyDirt: Making The Grade...

from the urls-we-dig-up dept

Technology can be very useful for helping teachers reach out to more students and for spreading information efficiently among schools. Some grading can be automated, but obviously not all grading can be done with heuristics and strict rules. Here are just a few examples of grading challenges that teachers are already facing that might need some technological improvement.

• Grading on a curve can backfire if all of your students scheme to get the same grade: a zero. Grading policies have adapted to account for this boundary condition, so students beware.... [url]
• Some startups are collecting as much grading data as they can, in hopes of obtaining some of the millions of venture capital directed at the education sector. Now when teachers threaten that students' actions will go on a permanent record, they actually have a database that will back them up. [url]
• Massive open online courses (MOOCs) need to watch out for massive cheating schemes. Test proctoring software is getting more sophisticated, but presumably some students are always trying new ways to cheat. [url]

If you'd like to read more awesome and interesting stuff, check out this unrelated (but not entirely random!) Techdirt post via StumbleUpon.

Filed Under: cheating, classes, education, grading, moocs, online courses, proctoring, software, tests

DailyDirt: Doing Math In Your Head

from the urls-we-dig-up dept

Math might not be the easiest subject for some students, but there might be different ways of teaching it that could make it more tolerable for kids. The more we learn about how our brains process math problems, the better we can teach ourselves how to tackle math education. There's a lot of concern over how Americans can compete in a global economy if our kids don't have some pretty basic math skills. Maybe some of these findings will help students pick up some much needed math skills.

• Learning how to use an abacus could actually be useful. Japanese students have demonstrated that using a mental image of an abacus (no actual abacus needed) can help them perform some incredibly fast mental calculations. [url]
• American kids don't do well on international math tests, but the bright side is that we're slowly learning what might be the best remedies. Understanding *why* kids don't do well on math tests is an important part of coming up with a solution, but looking at the bad test results is pretty scary.... [url]
• Doing a few algebraic word problems before bedtime might help alleviate some math fears in children. Or they could inspire math-related nightmares that haunt kids like Freddie Krueger. You decide. [url]

If you'd like to read more awesome and interesting stuff, check out this unrelated (but not entirely random!) Techdirt post.

Filed Under: abacus, bedtime math, education, math, teaching, tests

DailyDirt: Wuzzle Means To Mix. Sculch Is Junk. Alate Means To Have Wings. A Baloo Is A Bear....

from the urls-we-dig-up dept

There are a lot of standardized tests for kids to take, but it's not always clear what the results of the tests actually mean. If society wants to create a huge population of adults who can memorize some facts or fill out circles with no.2 pencils, then we're doing a pretty good job of it. Here are a few links that question the usefulness of certain kinds of tests.

• The New York state Education Department recently threw out standardized test questions related to a nonsensical story about talking animals and a sleeveless pineapple. Apparently, a lot of 8th graders were confused about the moral of this story, but the larger lesson might be that standardized tests shouldn't be taken too seriously. [url]
• The headmaster of one of NYC's top private schools (Riverdale) doesn't have a high opinion of standardized IQ tests for admissions. "This push on tests ... is missing out on some serious parts of what it means to be a successful human." [url]
• In Florida, it looks like 5th graders are getting their answers marked wrong even when they're correct. Science is so subjective these days. [url]
• Finnish schools don't administer standardized tests until the last year of high school, but somehow Finnish students seem to do well on the PISA (Program for International Student Assessment) exams. Is there something to be learned from the Finnish school system? [url]
• To discover more interesting education-related content, check out what's currently floating around the StumbleUpon universe. [url]  

By the way, StumbleUpon can recommend some good Techdirt articles, too.

Filed Under: education, exams, finland, schools, students, teachers, tests
Companies: riverdale