Dartmouth's Insane Paranoia Over 'Cheating' Leads To Ridiculous Surveillance Scandal
from the this-is-dumber-than-it-looks dept
The NY Times had an incredible story a few days ago about an apparent "cheating scandal" at Dartmouth's medical school. The problem was, it doesn't seem like there was any actual cheating. Instead, it looks like a ton of insane paranoia and an overreliance on surveillance technology by an administration which shouldn't be in the business of educating kindergarteners, let alone med students. We've had a few posts about the rise of surveillance technology in schools, and its many downsides -- and those really ramped up during the pandemic, as students were often taking exams from home.
So much of the paranoia is based on the silly belief that if you don't have everything crammed totally into your head, you haven't actually learned anything. Out here in the real world, it seems like a more sensible realization is that if you teach people how they can look up the necessary details when they need them, you've probably done a good job. Yes, there may be some exceptions and some scenarios where full knowledge is important. But for most things, the ability to know how to find the right answer is a lot more important than making sure trivial details are all remembered and can be regurgitated on an exam. Indeed, studies have shown repeatedly, that trying to cram the details into your head for an exam often means they don't stick in long term memory.
In short, this type of insane test taking tests people on exactly the wrong thing, and instead encourages the kind of behavior that leads to worse outcomes in the long run.
But the situation at Dartmouth is -- believe it or not -- even dumber. 17 Dartmouth medical students have been accused of cheating -- but those accusations were based on a tool that is not designed to spot cheating. It was based on Canvas, a popular platform for professors to post assignments and for students to submit homework through. And here's what happened, according to the NY Times:
To hinder online cheating, Geisel requires students to turn on ExamSoft — a separate tool that prevents them from looking up study materials during tests — on the laptop or tablet on which they take exams. The school also requires students to keep a backup device nearby. The faculty member’s report made administrators concerned that some students may have used their backup device to look at course material on Canvas while taking tests on their primary device.
Geisel’s Committee on Student Performance and Conduct, a faculty group with student members that investigates academic integrity cases, then asked the school’s technology staff to audit Canvas activity during 18 remote exams that all first- and second-year students had taken during the academic year. The review looked at more than 3,000 exams since last fall.
The tech staff then developed a system to recognize online activity patterns that might signal cheating, said Sean McNamara, Dartmouth’s senior director of information security. The pattern typically showed activity on a Canvas course home page — on, say, neurology — during an exam followed by activity on a Canvas study page, like a practice quiz, related to the test question.
“You see that pattern of essentially a human reading the content and selecting where they’re going on the page,” Mr. McNamara said. “The data is very clear in describing that behavior.”
The audit identified 38 potential cheating cases. But the committee quickly eliminated some of those because one professor had directed students to use Canvas, Dr. Compton said.
In emails sent in mid-March, the committee told the 17 accused students that an analysis showed they had been active on relevant Canvas pages during one or more exams. The emails contained spreadsheets with the exam’s name, the test question number, time stamps and the names of Canvas pages that showed online activity.
If you just read that, it might sound like at least some evidence that those students were doing something they weren't supposed to be doing (even if you think the rules are dumb). But, even that seems to not be accurate. There are some of us (and I am guilty of this) who rarely, if ever, shut down tabs that have important tools or information for our work. Plenty of students are the same, and likely leave Canvas open all the time. And that's what many of the students have claimed.
Geisel students said they often had dozens of course pages open on Canvas, which they rarely logged out of. Those pages can automatically generate activity data even when no one is looking at them, according to The Times’s analysis and technology experts.
School officials said that their analysis, which they hired a legal consulting firm to validate, discounted automated activity and that accused students had been given all necessary data in their cases.
But at least two students told the committee in March that the audit had misinterpreted automated Canvas activity as human cheating. The committee dismissed the charges against them.
In another case, a professor notified the committee that the Canvas pages used as evidence contained no information related to the exam questions his student was accused of cheating on, according to an analysis submitted to the committee. The student has appealed.
The school's paranoia over this went further. When it confronted the 17 students, it more or less pressured them into pleading guilty rather than fighting their case:
Dartmouth had reviewed Mr. Zhang’s online activity on Canvas, its learning management system, during three remote exams, the email said. The data indicated that he had looked up course material related to one question during each test, honor code violations that could lead to expulsion, the email said.
Mr. Zhang, 22, said he had not cheated. But when the school’s student affairs office suggested he would have a better outcome if he expressed remorse and pleaded guilty, he said he felt he had little choice but to agree. Now he faces suspension and a misconduct mark on his academic record that could derail his dream of becoming a pediatrician.
“What has happened to me in the last month, despite not cheating, has resulted in one of the most terrifying, isolating experiences of my life,” said Mr. Zhang, who has filed an appeal.
The article notes other students were told they had 48 hours to respond to charges -- and that they weren't provided the evidence the school supposedly had on them, while also being pressured to admit guilt:
They said they had less than 48 hours to respond to the charges, were not provided complete data logs for the exams, were advised to plead guilty though they denied cheating or were given just two minutes to make their case in online hearings, according to six of the students and a review of documents.
There are just layers upon layers of ridiculousness here. Not only is it bad pedagogically to teach this way, it's dangerous to engage in this kind of surveillance (in the middle of a pandemic, no less), and to just build up an entire atmosphere of mistrust.
EFF did a long and detailed post on this in which they note that the data in question could not have shown cheating, and arguing that these students have been denied basic due process.
But after reviewing the logs that were sent to EFF by a student advocate, it is clear to us that there is no way to determine whether this traffic happened intentionally, or instead automatically, as background requests from student devices, such as cell phones, that were logged into Canvas but not in use. In other words, rather than the files being deliberately accessed during exams, the logs could have easily been generated by the automated syncing of course material to devices logged into Canvas but not used during an exam. It’s simply impossible to know from the logs alone if a student intentionally accessed any of the files, or if the pings exist due to automatic refresh processes that are commonplace in most websites and online services. Most of us don’t log out of every app, service, or webpage on our smartphones when we’re not using them.
Meanwhile, the student free speech advocacy organization FIRE has been demanding answers from Dartmouth as well. To make matters worse, FIRE noticed that Dartmouth recently hid its "due process policies" from public view (convenient!):
We also asked why the college appears to have recently password-protected many of its due process policies. Of course, doing so conveniently hides them from the scrutiny of the public and prospective students who might be curious whether they will have rights — and what those rights might be — if they matriculate at Dartmouth.
And, of course, all of this could have been avoided if Dartmouth wasn't so overly paranoid about the idea that medical students might (gasp!) be able to look up relevant information. When I go to a medical professional, I don't necessarily need them to have perfect recall of every possible symptom or treatment. What I hope they're able to do is use their knowledge, combined with their ability to reference the proper materials, to figure out the best solution. Perhaps I should avoid doctors who graduated from Dartmouth if I want that.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: cheating, exams, medical students, surveillance, trust
Companies: canvas, dartmouth
Reader Comments
Subscribe: RSS
View by: Time | Thread
I never understood the point of memorizing formulas. Being able to recite the quadratic formula from memory doesn't mean I'm better at solving an equation.
The ability to use the knowledge properly is what needs to be tested, not simply checking to see if you can regurgitate the text.
[ link to this | view in chronology ]
Re:
Agreed. But the standard is "pick from one of these choices where at least one of them is always guaranteed to be right." There's a non-zero chance of luck that you can ace one of these tests without knowing how to even read it.
Why? Because that's the cheapest and fastest way to assign a random percentage to a random person. Those random percentages then can be added together quickly to give the idiots in charge the perfect set of numbers to quote when congratulating themselves or condemning the failure of others. Those random percentages also tend to be useful for other idiots that make demands for "something to be done." Like pay cuts, firings, and tax revenue allocation. I.e. The whole point is to quickly assign blame and praise for political purposes. Not actually determine what a given kid knows and what they need help with.
When you spend 12+ years educating kids that the answers they need will always be given to them when they need them, you shouldn't be surprised that they do everything they can to maintain that expectation later in life. Or fail miserably when they cannot.
Yet another case of overly-politicizing an issue coming back to bite us...
[ link to this | view in chronology ]
Re:
Exactly this. My high school calculus teacher is the first teacher I had who seemed to understand this and his tests (even the finals) were open book for this reason. He noted that, if we had a job where calculus was necessary, we'd never need to pull formulas out of our heads and this was before the smartphone era where even people who never took a calculus class can simply get Wolfram Alpha to compute derivatives or integrals for them.
Instead of asking 100 multiple choice questions, just ask a half-dozen moderately complex "story" problems and you can find out if the student in question actually understands what they've been taught.
[ link to this | view in chronology ]
Combining online surveillance with the Kafkaesque kangaroo "courts" of universities accusing students of cheating was never going to end well.
[ link to this | view in chronology ]
We also asked why the college appears to have recently password-protected many of its due process policies. Of course, doing so conveniently hides them from the scrutiny of the public and prospective students who might be curious whether they will have rights — and what those rights might be — if they matriculate at Dartmouth.
Nothing says 'We're on the right side of this issue and students can trust that they will be treated fairly if they come here' like trying to hide your own campus policies while pressuring students to 'admit' to cheating based upon evidence not provided to them to refute.
[ link to this | view in chronology ]
Failure of the imagination
This brings to mind a high school math teacher I had who decided the best way to deal with "cheat sheets" on his tests was to simply tell everyone they could bring in a 2x2 square of paper (the cheat sheet) with whatever they wanted on it.
That turned out to be a very effective study aid for the material!
Similarly for the EIT for professional engineers -- you could bring whatever books you wanted. And this was pre-internet.
[ link to this | view in chronology ]
Open Book Tests
The upper-division (junior--senior--M.S. first-year) STEM courses I took generally were open-book. Usually the fact that they were also time-limited kept the book useless if you didn't have a good overall grasp of the subject.
Aside: the only Stanford math graduate I knew had never heard of the quadratic equation--let alone how to look it up. Your mileage may vary.
[ link to this | view in chronology ]
Re: Open Book Tests
"The upper-division (junior--senior--M.S. first-year) STEM courses I took generally were open-book."
Most advanced STEM courses I attended in university had some form of PBL as the biggest part of the test. A problem is posed to the individual or group. They discuss it, debate it, divide the research angles appropriately and present their findings together. Especially in biology, pharmacology and biochemistry the point of the test is to demonstrate research capability not memory.
And to think we get that sort of education for free in much of europe while americans get to pay through the nose for what appears to be last century's teaching methods.
[ link to this | view in chronology ]
The majority of exams I had in (many) years of post-high school education were open book. Good test design makes negates a lot of the usefulness of the book. They were almost all timed with just a little extra time so you either knew the material, or were familiar enough with the book (from studying) that you knew where it was, or you didn’t finish.
Probing questions that force you to prove deeper understanding of what’s in the book also makes extra materials largely irrelevant.
[ link to this | view in chronology ]
But but but it might make us look bad!!!!
See also: Title IX "investigations" where its guilt by penis.
They were so worried about looking bad, they managed to make themselves look tat much worse.
[ link to this | view in chronology ]
Despite what we sometimes see on television, they don't need to be a genius that can solve every problem by themselves either. Good doctors consult with other medical professionals as necessary. The educational system's desire to come up with individual quantified ratings does not reflect reality.
[ link to this | view in chronology ]
There's some rationale when it comes to memorization for people in the medical field, under the assumption that they'd need to make split second decisions and diagnoses - but an obsessive attachment to rote memorization as a method of teaching and learning continues to be prevalent. And it will continue to be so until the advocates of such a system die off. Or until enough students who've slaved their way to the top of the class, only to be replaced by machines or cheaper interns, realize that none of the sacrifices they've made for their "education" have resulted in nothing for their career progression. Sure, some wide-eyed idealist will tell them that it's the "skills" they picked up along the way. It's cold comfort when they're struggling to pay off rent and student debt while some kid makes millions a month with YouTube reaction videos.
[ link to this | view in chronology ]
Re:
That's true for paramedics and people working in emergency rooms, but isn't that what residency is for? If someone bluffs their way into such a position, it's hard to believe none of their co-workers would notice; I'd be inclined to take their opinions over the grades from a standardized test.
[ link to this | view in chronology ]
Out Here in the Real World
Proably a good plan. I say that as a person whose graduate school had a subject-specific library of some three storeys, largely loaded with shelves full of books. In order to increase capacity, some of the shelves actually were on tracks so they could be moved to allow passage between particular shelves while the other shelves sat cheek-to-jowl.
Now being out of school, I work in an office. A substantial portion of that space is devoted to shelves holding a few thousands of volumes. The intent is that if I do not know something, I might go through some few of the volumes and find out.
[ link to this | view in chronology ]
Re: Out Here in the Real World
teach people how they can look up the necessary details when they need them
exactly, i never felt slighted when my former doctor would come in to discuss my complaint (gather evidence) and go back to his office to consult his Physician's Desk Reference...
[ link to this | view in chronology ]
Re: Re: Out Here in the Real World
The fields of medical knowledge are vast -- and not only constantly being updated, but sometimes even changing.
I damn well hope that my doctor is prepared to consult reference materials before proffering a professional opinion or advising treatment.
If there's anything thing scarier than a newly minted, "inexperienced" doctor fresh out of med school, it's an old practitioner who hasn't bothered to keep him or herself current since they graduated med school.
[ link to this | view in chronology ]
Not just the medical field...
"When I go to a medical professional, I don't necessarily need them to have perfect recall of every possible symptom or treatment. What I hope they're able to do is use their knowledge, combined with their ability to reference the proper materials, to figure out the best solution."
A side discussion but the above and this whole article make me think of the nature of problem solving in our internet age.
In the first decade of the 2000's I tried to escape a job at a national laboratory. Most of the interviews I landed were patterned after techniques developed at Microsoft and Google. These entailed programming in realtime on a white board and/or answering impossible but thought provoking questions such as: how would you move mt. Fuji?
These techniques became so popular that books were written about them with potential questions and answers (How Would You Move Mt. Fuji by William Poundstone, Programming Interviews Exposed by Morgan et al, et cetra). If you could memorize all this published material, you could do well in such an interview, and many did. Unfortunately, memory and perfect recall have never been one of my gifts, at least not for questions such as these.
I always wanted to ask the interviewers but lacked the courage: does anyone today really program like this anymore? Directly from memory? Or when needing a common function (for example, some sort of sort routine) do they grab the algorithm and/or code from their "bag of tricks" or an internet repository and copy/paste.
As I got older my value as an employee increased with my ability to "find" answers and "solve" problems, not regurgitate answers from the top of my head. I have no idea if tech still interviews this way or not.
I've come to the conclusion that if I cannot find the answer on the internet (or at least something useful leading to an answer) then I may be asking the wrong question.
[ link to this | view in chronology ]
Tech side informative; but.. lack of situational awareness
Dartmouth is extremely fearful that the valid ethics challenges have snowballed and it's Lord of the Flies. That they tried something stupid is par for the course. The College is led by a sellout with a terrible wife who seethes with envy and greed...also quite reflective of the state of not only Dartmouth but also Harvard and Yale. The Jennifer Show launched with a medical fraud conducted by Dr. Britt Lynes '87 who apparently took a bribe. Similarly a certain Song of Myself mentally ill general who has a professional history of HC finance reverse engineered recovery and first aid. It is vastly exciting for narcissists to see and hear their abuse target scream in pain.
[ link to this | view in chronology ]
Cheaters always prosper
The latest cheat making the rounds is adding dark grey text on a black desktop background. Video compression fails to see it, but human eyes sure do. In my day we would just use nervous foot tapping from the guy in front.
Now that I think about it, cheaters rise to the top of cheating corporations. They continue to cheat the law, taxes and the people. And now Universities are cheating their students. College-level education right here.
[ link to this | view in chronology ]