DailyDirt: Problems With Peer Reviewed Publications

from the urls-we-dig-up dept

Tue, Apr 14th 2015 5:00pm — Michael Ho

Peer review isn't exactly a sexy topic, but it's an essential part of academic publishing -- and it may need to change a bit to keep up with the times. Peer review is typically a thankless chore that is distributed among academics working in a network of related fields, and sometimes personal politics can enter into the process if the subject matter is obscure enough. Misconduct in peer review doesn't usually get the same kind of coverage as various journalistic scandals (eg. Rolling Stone, Buzzfeed, etc), but the damages done can be even more significant to society.

Peer review processes aren't free of corruption -- with some third party agencies offering services that fake reviews or try to improve a paper's odds of being published in other unsavory ways. The publication system for scientific work doesn't seem to have a great way to deal with this issue besides retracting (instead of correcting) articles published in error. Dozens of papers have been retracted by BioMed Central recently, but the problems with peer review appear to be much more widespread. [url]
Some scientific papers are published for a fee -- with absolutely no quality control whatsoever. The impressively-titled International Journal of Advanced Computer Technology accepted a paper that consisted of nothing but "Get me off your fucking mailing list" repeated over and over. [url]
Paying for expedited peer review sounds sketchy, right? Rubriq's peer-review service promised a review within 3 weeks or your money back -- but perhaps these kinds of services should be subject to yet another round of reviews. [url]
Can publishers try to automate the detection of fake papers and poorly-reviewed articles before they turn into embarrassing mistakes? Artificial intelligence just isn't that good, but perhaps software will make it harder for people to detect shady predatory publications. [url]

After you've finished checking out those links, take a look at our Daily Deals for cool gadgets and other awesome stuff.

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: academia, journals, media, natural language processing, peer review, publications, retractions, rubriq, scandals

6 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

Anonymous Coward, 14 Apr 2015 @ 5:56pm

Public funded research is thankfully being required to be published in open access journals.
[ link to this | view in thread ]
Anonymous Coward, 15 Apr 2015 @ 5:16am

"Public funded" research to be published in "open access".

I bet there is some leeway, some interpretation, definitely will be continued corruption.
[ link to this | view in thread ]
Gene Cavanaugh (profile), 15 Apr 2015 @ 11:45am

On peer review
You say perhaps AI will make it HARDER to detect shady papers? Why would we use it, then? Don't you mean "easier"?
[ link to this | view in thread ]
Michael Ho (profile), 15 Apr 2015 @ 11:57am

Re: On peer review
Gene, I meant more difficult... analogous to the ways spammers use software to get around spam filters. Legit publishers won't use it, but sketchy journals that are trying to sound legit would.
[ link to this | view in thread ]
Coyne Tibbets (profile), 15 Apr 2015 @ 10:43pm

There used to be peerless scientific journals; now we have peer-less.
[ link to this | view in thread ]
Lepton x, 16 Apr 2015 @ 7:28pm

Review
Peer review is not an objective rule based process. It's subjective as the reviewer is asked to gauge the work's novelty as well as its impact and authenticity.

An article may be rejected for any one of the above based on the journals own internal, 'floating', standards. Researchers know which journals will likely publish their work just by reading comparable papers previously published (i.e. don't try to publish your global cooling hypothesis in a journal that only published global warming papers)

Every journal has its own biases, and some may not like you or your area or even what country you reside.
There are often cabals of reviewers with a particular view who span several journals.

The whole process is biased, but not totally corrupt. Journals are in the business of selling overpriced subscriptions though and there is MASSIVE competition due to an over-abundance of journals.

It's also a fact most people with things to do don't want to serve as reviewers (Ala jury duty) and so you may get a particular type of wanker who enjoys sadistic rejection.
It's also true a reviewer is unlikely to approve a paper that contradicts his or her own research.

The process is marginally fair if you do your due diligence in selecting the right journal.

P.S- people who publish crap may get away with it once or twice, but then the plug will be pulled.
[ link to this | view in thread ]