Academic Journals Seeing Benefits In More Open Peer Review

from the freedom-is-powerful dept

Tue, Aug 31st 2010 12:49am — Mike Masnick

We recently discussed the idea of much more open, online-based peer review processes. In the ensuing discussion some claimed that such things might work for subject areas like math, where concepts could be reviewed and tested by others, but might not work as well in other areas. However, a recent experiment by the Shakespeare Quarterly to experiment with a more open, online peer review system apparently worked quite nicely:

Mixing traditional and new methods, the journal posted online four essays not yet accepted for publication, and a core group of experts -- what Ms. Rowe called "our crowd sourcing" -- were invited to post their signed comments on the Web site MediaCommons, a scholarly digital network. Others could add their thoughts as well, after registering with their own names. In the end 41 people made more than 350 comments, many of which elicited responses from the authors. The revised essays were then reviewed by the quarterly's editors, who made the final decision to include them in the printed journal, due out Sept. 17.

Even one of the authors who was quite skeptical of the program as "entirely won over" by the end, noting that the comments were "more extensive and more insightful" than he was used to receiving on his works.

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: openness, peer review

6 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

B. Nicholson, 31 Aug 2010 @ 2:13am

academic peer review
Elderly peer review is an adequate system for youngsters learning to write. Crowd sourcing may be a nice way to fluff out a review, but only the intensity of an educated mind prepared for surprises can make the breakthroughs upon which humanity must ultimately depend.
Why? Because nobody wants the work of comprehending novelty. Our most foresightful works are published, if at all, in peripheral journals. The birth of radio astronomy is a good example. Our most advanced thinkers can not be published at all. Leonardo was forced to self-publish his folios, which went unrecognized as anything but pretty pictures for more than 500 years.
So, no. Do not count on peer-review to discover anything whatsoever. Instead, wait for all the old men to die upon their golden thrones, then we will have progress, maybe. One cannot climb up upon the shoulders of giant imbeciles without sticking one's buttocks into the faces of our recognized scientific greats. So long as money funds science and science pays a living wage, airheads will get those jobs. Open an area of inquiry and some nit-wit thumpkin will apply for a fraudulent patent or copyright for it.
[ link to this | view in chronology ]
- Richard (profile), 31 Aug 2010 @ 2:56am
  
  Re: academic peer review
  You have something of a point there - although I think you overstate it. Some fields are more tolerant of advanced thinking than others. The theoretical physics community is pretty good in this respect - it tends to tolerate crazy ideas - after all it is fromt his field that the quotation "we are all agreed that you are crazy, what we are arguing about is wheter you are crazy enough to be right."
  [ link to this | view in chronology ]
  - Anonymous Coward, 31 Aug 2010 @ 4:58am
    
    Re: Re: academic peer review
    For now, it was not that way in the past.
    Renewal will guarantee that next generations will have different point of views of the current ones and that is for better or for worst.
    
    The mechanics of social interactions will forge new groups of thought that will focuses on the maladies of their current system and naturally try to fix things, that can change things in unexpected ways.
    [ link to this | view in chronology ]
The other coward, 31 Aug 2010 @ 5:29am

Skeptical
I'm still skeptical if this could work on a large scale. Sure, it will work well for popular subjects or while it's novel, but what happens when there are 100,000+ papers online to be reviewed in 100s of different locations?

Someone has to find it and be interested in it enough to spend the time. With a ton of options, I see many being neglected especially in the more obscure fields of study.

The issues are not insurmountable, but they seem to be frequently ignored or glossed over.
[ link to this | view in chronology ]
- Milo B Farnsworth, 31 Aug 2010 @ 10:01am
  
  Re: Skeptical
  Hmmm, finding people with interest and time...
  
  If only we had some way of contacting or connecting with these people.
  [ link to this | view in chronology ]
- Milo B Farnsworth, 31 Aug 2010 @ 10:01am
  
  Re: Skeptical
  Hmmm, finding people with interest and time...
  
  If only we had some way of contacting or connecting with these people.
  [ link to this | view in chronology ]