Are Automated Essay Graders Sucking The Life Out Of Writing?

from the who-needs-humans? dept

A slightly disturbing story about how English essays are increasingly being graded by machines. The folks behind the program claim that the computer's results have a very high percentage correlation to scores given by human graders. Of course, the machine is scoring the essays more on technical points concerning sentence structure and the use of certain key words. The same company that makes the software for grading exams is also offering a "training" application that students can use to get feedback on their own papers. As someone in the article points out, while it's nice to have a system to help teach students proper writing techniques, there is also the fear that such systems will beat out any creativity from students by focusing exclusively on the technical (rather than the creative) aspects of writing.
Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team


Reader Comments

Subscribe: RSS

View by: Time | Thread


  1. identicon
    larry kniepkamp, 4 Sep 2003 @ 5:51am

    worksaving technology

    Your comments sound very much like the ones made when engineers stopped using slide rules and log tables because they had hand held calculators. Engineers are still creative, they are just more efficient.

    link to this | view in thread ]

  2. identicon
    bastard sam, 4 Sep 2003 @ 6:39am

    No Subject Given

    I think that's an unfair assumption mike. If it correlates to human graders like they say, then creative writing or not should make no difference. Might be interesting to follow this one.

    link to this | view in thread ]

  3. identicon
    AMetamorphosis, 4 Sep 2003 @ 8:11am

    Creativite Writing ...

    If a creative paper is graded by a machine, how can the " creativity " included in the paper be properly assessed ?
    Machines are great for use in grading assignments that have a predetermined outcome. ie: Multiple choice, True & False, or specific words or numerical values.
    This topic greatly intrigues me and I wonder what grade a Shakespearean piece or a poem would garner.

    link to this | view in thread ]

  4. identicon
    AMetamorphosis, 4 Sep 2003 @ 8:26am

    Quality verses Stupidity

    From Article: " E-rater cannot read or judge an essay's quality but uses statistical analysis to determine which linguistic features are characteristic of each human scoring level, typically on a scale from 1 to 6. "
    Clearly, these programs do NOT judge or have the ability to even read the essay's QUALITY.
    I think it would be quite easy to come up with a formulation that would pick enough necessary words, phrases & other items the programs are looking for in order to achieve a high grade ... even though the quality may be abhorrent.

    link to this | view in thread ]

  5. identicon
    Beck, 4 Sep 2003 @ 8:40am

    Teach the Test

    If these become common then it's only a matter of time until private companies develop courses that teach students how to make the machine happy. Instead of teaching writing they will be teaching how to beat the machine.

    I hope these machines have better grammar checking than the green underlines in Microsft Word.

    link to this | view in thread ]

  6. identicon
    digibum, 4 Sep 2003 @ 8:41am

    Look at the upside

    Hey at least our students will be able to get high paying jobs as search engine optimization consultants.
    I think this trend will die out when someone figures out a way to write a total jibberish essay that is somehow graded very highly.

    link to this | view in thread ]

  7. identicon
    Dan Z., 4 Sep 2003 @ 9:04am

    Essay, Schmessay

    Given that automated scoring can't judge quality, why use essay testing in the first place? If you're only looking for certain vocabulary words in certain places, why not just offer a Mad Lib style essay template where students fill in the blanks? At least then students and graders would agree 100% of the time on what belonged where, computers wouldn't be penalizing originality, and they wouldn't be giving reinforcement to poorly written essays and poorly thought out arguments.

    link to this | view in thread ]

  8. identicon
    mark, 4 Sep 2003 @ 9:23am

    my college exit exams

    In order to graduate, I had to take some sort of basic competency exam and thought, man is this stupid. How could anyone get passed through 18 years of school and still not have a grasp of basic grammer and vocabulary?

    Anyway, when it came to the essay segments, I wrote my answers like a 4th grader. Short, simple sentances. See the cat. The cat is blue. I didn't attempt anything sophisticated like using a semicolon. Whoever graded the work must have thought I was a moron, but I didn't care, I thought the test itself was a waste of my time. Might as well use a machine.

    link to this | view in thread ]

  9. identicon
    LittleW0lf, 4 Sep 2003 @ 1:59pm

    Anyone actually read the article?

    If they would have (ok, a couple folks who replied had read it,) they would have realized that nobody is proposing using the system alone to grade. The article specifically mentions that the grade is based both on the computer and the human grader's interpretation of the work, so it isn't likely to be cheated any time soon.

    I think it is a great idea...I write, and often I find it very difficult to find someone who will critique my work. Having a computer do it for me would be extremely helpful, so long as it does a much better job than the normal spell & grammer checkers do. And I can see where this could be a big advantage for the teacher, who can let the software grade the technical aspects of the work while they admire the literary aspects. Teachers who grade the technical aspects of a work often loose the literary aspects, since they spend most of their time grading the technical aspects.

    Sounds like a great idea to me.

    link to this | view in thread ]

  10. identicon
    Anonymous Coward, 4 Sep 2003 @ 2:40pm

    Re: Anyone actually read the article?

    I like your use of moderation and a blending of two for a harmonious outcome ...

    link to this | view in thread ]


Follow Techdirt
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Discord

The latest chatter on the Techdirt Insider Discord channel...

Loading...
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it
Close

Email This

This feature is only available to registered users. Register or sign in to use it.