DailyDirt: Can Computers Grade Written Essays?

from the urls-we-dig-up dept

Technology aimed at education could really benefit an incredible number of students by making classes and learning (potentially) a more pleasant and efficient experience. Computers can't replace a really good human teacher, but they can make it easier for good human teachers to reach a vast audience of students. Massively open online courses (MOOCs) promise to change how education works, but there are some technological tools that might be missing. It's pretty straightforward to test students on math problems in an automated way, but grading essays is a much more daunting problem. There have been some calls for automated grading software from various organizations (like the Hewlett Foundation). But at the same time, the National Council of Teachers of English argues that computers simply can't grade essays. Here are just a few more links on this debate over the use of algorithms over English professors (or grad students). If you'd like to read more awesome and interesting stuff, check out this unrelated (but not entirely random!) Techdirt post via StumbleUpon.
Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: ai, algorithms, artificial intelligence, automation, education, edx, essays, ets, exams, grading, mooc, nlp, tests
Companies: harvard, hewlett foundation, kaggle, mit


Reader Comments

Subscribe: RSS

View by: Time | Thread


  • identicon
    Anonymous Coward, 9 May 2013 @ 5:20pm

    I imagine they'll start using this for grading bar exams soon...

    And we'll have a flood of new lawyers who can convince computers without using any facts at all... yay!

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 9 May 2013 @ 5:21pm

    I think the teachers are arguing they'd like to keep their jobs. I've honestly seen some teachers who think their job is grading rather than teaching. Anyways a well done grading program could grade just fine. I guess the argument would be that nobody has created a great program for grading written papers.

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 9 May 2013 @ 5:50pm

    Yeah, cell phones are really good it, let's apply it to exams.

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 9 May 2013 @ 5:51pm

    Here's an idea!

    Lets combine automated grading software with automated essay composition software and leave humans completely out of the loop.

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 9 May 2013 @ 7:22pm

    Considering the Zero Tolerance stance on.. er... children... in american schools, I'm surprised we aren't seeing computers designed to impose suspensions.

    link to this | view in chronology ]

    • identicon
      Anonymous Coward, 9 May 2013 @ 7:35pm

      Re:

      I have always thought that the way to restore sanity regarding zero tolerance is to threaten to do that.

      Nothing is as influential as a bureaucrat trying to preserve their own job. If it goes through without a return to sanity well you punished the morons behind it by making them unemployed and you aren't worse off. Not to mention the tax revenue saved. It is a heads you lose tails I win situation.

      link to this | view in chronology ]

    • identicon
      Anonymous Coward, 10 May 2013 @ 5:04am

      Re:

      It could be like in that movie - Demolition Man.
      Swearing is illegal and you get a printout for each infraction. I'd better hurry and invest in paper companies before its cool.

      link to this | view in chronology ]

  • identicon
    Anonymous Coward, 9 May 2013 @ 7:30pm

    Grading? Easy

    Grading an essay is actually pretty damn easy, just return a value. Grading it right is the challenge.

    link to this | view in chronology ]

  • icon
    artp (profile), 9 May 2013 @ 7:55pm

    Depends

    Can a student turn in a computer-generated paper that is written to the grading algorithm?

    It would take some smarts to do that, too. Neither would be satisfactory.

    link to this | view in chronology ]

    • identicon
      Anonymous Anonymous Coward, 9 May 2013 @ 8:38pm

      Re: Depends

      Along those lines, isn't the current system just a set of 'facts' (loosely describing the current standardized testing system) and that the way to game the system is to know what is on the final test?

      Reminds me of a segment regarding the author of In Search of Excellence. He was a new professor on his first day, and he handed out the final exam to all of the students. A more senior professor was passing by and noticed this atrocity. He cornered his junior and remonstrated him for giving out the final exam. The new to the campus professor retorted, (to the best of my memory) "Not only am I giving them the final exam on the first day, I intend to spend the rest of the semester giving them the answers."

      As an experienced employer, with at times thousands of employees, my philosophy on hiring is: I don't care what you know, I care what you can do! Education provides knowledge. Knowledge, TRAINING, AND EXPERIENCE provide skill. Skill is what one sells on the marketplace. Either to a new employer, or a customer.

      Unfortunately, other employers are not so enlightened. They care what your grades are and what is on your Facebook page, and if those keywords weren't on the resume, we wouldn't even be talk'n.

      For me, teach the student the basics (read'n, rite'n, and rithmitic) then teach them to think, make decisions, take stances, be critical without animosity, resolve issues, expand creativity (I wish art and music were required, but creativity can come elsewhere, even in math), and to 'do it, try it, fix it.

      I care not what the grades were, I care what they can do, and if they might grow with us.

      link to this | view in chronology ]

      • identicon
        Anonymous Anonymous Coware, 9 May 2013 @ 8:44pm

        Re: Re: Depends

        Tom Peters, damn this early onset....what was I talking about?

        link to this | view in chronology ]

      • identicon
        Anonymous Coward, 9 May 2013 @ 11:47pm

        Re: Re: Depends

        The problem really is that there is a hierarchy of knowledge (use information-gathering mechanism of choice for finding out about Bloom's Taxonomy and related). At the lowest levels, you have things characterised by words like 'recall' and 'recite'. At higher levels, you progress through things like 'understand' and 'apply' through to 'analyse' up to 'create'.

        Evaluating the lower levels ranges from trivially easy to simple. As you look for higher-level skills, you begin to require expert judgment. I'll assert that expert judgment is almost impossible to implement as an algorithm.

        I'm doing a MOOC atm. It is sitting squarely in the lower cognitive domains. It'll replace a bad teacher easily. It won't replace the best a university can do.

        link to this | view in chronology ]

    • identicon
      Anonymous Coward, 10 May 2013 @ 2:48am

      Re: Depends

      Yes. Les Perelman, a director of writing at MIT, did an analysis of e-Rater algorithms last year. He came up with a completely nonsensical and factually incorrect essay that got a top score, and a well-argued and well-written essay that scored lower.

      Here's the NYT article about it:
      http://www.nytimes.com/2012/04/23/education/robo-readers-used-to-grade-test-essays.html

      And here's his top-scoring essay:
      http://www.documentcloud.org/documents/346138-essay-awarded-a-top-grade-by-e-rater.html

      I f nothing else, it's worth skimming through the essay. Among other things, it includes a line from Howl and claims that the union of teaching assistants is more powerful than the Freemasons (although less powerful than the Jedi Knights).

      Textual analysis is hard. It can potentially test for writing style, but it can't test for comprehension or content.

      link to this | view in chronology ]

      • icon
        art guerrilla (profile), 10 May 2013 @ 6:46am

        Re: Re: Depends

        yep, thanks, was going to post about that...
        but it kind of fits with the times: style over substance...

        art guerrilla
        aka ann archy
        eof

        link to this | view in chronology ]

  • identicon
    Anonymous Coward, 9 May 2013 @ 8:36pm

    Need to worry about people gaming the models

    Here's a quick summary of the most insightful comments I've seen about this issue:

    Yes, computers can apparently grade essays written for humans as well as underpaid and overworked graders do (which is badly). But once you start having computers actually grade the students, the students will start writing their essays to fool the computer graders (likely with help of computer programs)...

    link to this | view in chronology ]

  • icon
    timmaguire42 (profile), 11 May 2013 @ 6:02am

    MS Word has an auto-summary feature that is disturbingly good, but just because software can create a summary doesn't mean it can recognize a good one.

    The day they write software that can recognize and appreciate a pun is the day they can dream about writing software that can recognize and appreciate a sound argument.

    This sounds more like the kind of dumbing down we get from standardized testing. And I suppose if they really do want to reduce our children to mindless drones, then this is another step in the right direction

    link to this | view in chronology ]

  • identicon
    assignment writing, 11 Aug 2014 @ 2:22pm

    Assignment writer uk

    This is really nice information which you have shared with us. Please keep sharing that kind of information further as I really like it.

    link to this | view in chronology ]

  • icon
    Juana Dalton (profile), 6 Mar 2015 @ 3:12am

    essay

    At my university essay test worked very attentive teacher. However, using the www.grabmyessays.com/ - service for writing essays I successfully passed the course.

    link to this | view in chronology ]

  • identicon
    swesub filmer, 19 Nov 2015 @ 9:53pm

    good

    hey, The day they write software that can recognize and appreciate a pun is the day they can dream about writing software that can recognize and appreciate a sound argument.
    http://www.dreamfilm.ws/forfattare/edgar-wright

    link to this | view in chronology ]

  • identicon
    123Movies.biz, 5 May 2016 @ 8:13am

    Great

    Massively open online courses (MOOCs) promise to change how education works, but there are some technological tools that might be missing.

    http://www.123movies.biz/423-criminal-full-movie.html

    link to this | view in chronology ]

  • identicon
    Dreamfilm, 28 Jul 2016 @ 7:12pm

    Thloolm

    Massively open online courses (MOOCs) promise to change

    link to this | view in chronology ]

  • identicon
    xmovies-8.cc, 6 Aug 2016 @ 6:28pm

    Thloolm

    link to this | view in chronology ]

  • identicon
    Kolla på LIGHTS OUT Online, 9 Aug 2016 @ 4:36pm

    Thloolm

    Thanx

    link to this | view in chronology ]

  • identicon
    Visit this site, 16 Aug 2016 @ 5:31am

    Thloolm

    thxxxxxxxxxxxxxxxxxx

    link to this | view in chronology ]

  • identicon
    pandemic watch32, 18 Aug 2016 @ 10:19am

    Thloolm

    Txxxxxx

    link to this | view in chronology ]

  • identicon
    putlocker Movies, 25 Aug 2016 @ 6:38pm

    Thloolm

    ThanxThanx

    link to this | view in chronology ]

  • identicon
    free movies online, 27 Aug 2016 @ 5:54pm

    Thloolm mol

    Kooopasi !

    link to this | view in chronology ]

  • identicon
    www.rainiertamayo.tv, 27 Aug 2016 @ 6:33pm

    Thloolm

    Tgabjs

    link to this | view in chronology ]

  • identicon
    Drama swesub, 1 Sep 2016 @ 3:49pm

    Thanks

    Great one !

    link to this | view in chronology ]

  • identicon
    se element, 1 Sep 2016 @ 4:24pm

    Thloolm

    link to this | view in chronology ]

  • identicon
    movie25, 13 Sep 2016 @ 4:22pm

    Thloolm

    link to this | view in chronology ]

  • Thloolm

    Oh yes !

    link to this | view in chronology ]

  • identicon
    Ride Along 2 online hd, 14 Sep 2016 @ 6:10pm

    Thloolm

    yES THX §

    link to this | view in chronology ]

  • identicon
    f movies, 18 Sep 2016 @ 4:57pm

    Thloolm

    Thak !

    link to this | view in chronology ]

  • identicon
    Visit this site, 27 Sep 2016 @ 9:01pm

    Thloolm

    Thxx !

    link to this | view in chronology ]

  • Streama Hundraettåringen som smet från notan och försvann

    Hundraettåringen som smet från notan och försvann swefilmer, Hundraettåringen som smet från notan och försvann swesub streama gratis online

    link to this | view in chronology ]

  • identicon
    www.genvideos.ws, 24 Dec 2016 @ 10:13am

    www.genvideos.ws

    good

    link to this | view in chronology ]

  • identicon
    www.pubfilm.ws, 25 Dec 2016 @ 6:41am

    www.pubfilm.ws

    good post

    link to this | view in chronology ]

  • identicon
    pubfilm.ws, 25 Dec 2016 @ 6:47am

    pubfilm.ws

    This is really nice information which you have shared with us. Please keep sharing that kind of information further as I really like it.

    link to this | view in chronology ]

  • identicon
    www.putlockers.la, 7 Jan 2017 @ 5:07am

    www.putlockers.la

    good site

    link to this | view in chronology ]

  • identicon
    Matt, 12 May 2017 @ 4:05am

    Streama Film

    http://gratis-filmer.tv Streama Film Online

    link to this | view in chronology ]

  • icon
    123movies (profile), 4 May 2020 @ 6:57am

    MS Word has a disturbingly good auto-summary feature, but just because software can build a summary doesn't mean it can recognize a good one.

    The day they're writing software that recognizes and appreciates a joke is the day they can dream of writing software that can recognize and appreciate a sound statement.

    Watch online movies 2020 on 123Movies new site.

    link to this | view in chronology ]


Follow Techdirt
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Discord

The latest chatter on the Techdirt Insider Discord channel...

Loading...
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it
Close

Email This

This feature is only available to registered users. Register or sign in to use it.