Move By Top Chinese University Could Mean Journal Impact Factors Begin To Lose Their Influence
from the and-no-bad-thing,-either dept
The so-called "impact factors" of journals play a major role in the academic world. And yet people have been warning about their deep flaws for many years. Here, for example, is Professor Stephen Curry, a leading advocate of open access, writing on the topic back in 2012:
I am sick of impact factors and so is science.
The impact factor might have started out as a good idea, but its time has come and gone. Conceived by Eugene Garfield in the 1970s as a useful tool for research libraries to judge the relative merits of journals when allocating their subscription budgets, the impact factor is calculated annually as the mean number of citations to articles published in any given journal in the two preceding years.
The rest of that article and the 233 comments that follow it explain in detail why impact factors are a problem, and why they need to be discarded. The hard part is coming up with other ways of gauging the influence of people who write in high-profile publications -- one of the main reasons why many academics cling to the impact factor system. A story in Nature reports on a bold idea from a top Chinese university in this area:
One of China's most prestigious universities plans to give some articles in newspapers and posts on major social-media outlets the same weight as peer-reviewed publications when it evaluates researchers.
It will work like this:
articles have to be original, written by the researcher and at least 1,000 words long; they need to be picked up by major news outlets and widely disseminated through social media; and they need to have been seen by a large number of people. The policy requires an article to be viewed more than 100,000 times on WeChat, China's most popular instant-messaging service, or 400,000 times on news aggregators such as Toutiao. Articles that meet the criteria will be considered publications, alongside papers in peer-reviewed journals.
The university has also established a publication hierarchy, with official media outlets such as the People's Daily considered most important, regional newspapers and magazines occupying a second tier, and online news sites such as Sina, NetEase or Sohu ranking third./blockquote>
One of the advantages of this idea is that it recognizes that publishing in non-academic titles can be just as valid as appearing in conventional peer-reviewed journals. It also has the big benefit of encouraging academics to communicate with the public -- something that happens too rarely at the moment. That, in its turn, might help experts learn how to explain their often complex work in simple terms. At the same time, it would allow non-experts to hear about exciting new ideas straight from the top people in the field, rather than mediated through journalists, who may misunderstand or distort various aspects.
However, there are clear risks, too. For example, there is a danger that newspapers and magazines will be unwilling to accept articles about difficult work, or from controversial academics. Equally, mediocre researchers that hew to the government line may benefit from increased exposure, even resulting in them being promoted ahead of other, more independent-minded academics. Those are certainly issues. But what's interesting here is not just the details of the policy itself, but the fact that it was devised and is being tried in China. That's another sign that the country is increasingly a leader in many areas, and no longer a follower.
Follow me @glynmoody on Twitter or identi.ca, and +glynmoody on Google+
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: academic research, china, impact factors, journals, open access, peer review
Reader Comments
Subscribe: RSS
View by: Time | Thread
Newspapers just republish things without checking them in any form.
How about the time that Kim Jong-un's uncle was killed by turning dogs loose on him? One newspaper made up that story, and it spread through the world's newspapers like a wild fire?
The Onion's articles that get taken for real?
Climate denier's research being picked up by all those news outlets that like to push that agenda?
And THAT is the filter that is going to determine whether something is a reputable scientific research? Pfff. Bad idea.
[ link to this | view in chronology ]
web pages creation company in athens
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Errr... wha?
While it certainly is a worthy goal to encourage scientists to write articles that can be understood by mere mortals, page-views are a piss-poor way of deciding if an article is valid or not. In fact, an crackpot with startling (but entirely bogus) conclusions is likely to get a LOT more pageviews than another scientist publishing correct information that is inherently less dramatic.
China already has enough difficulty with scientific integrity; putting the popular (non-peer-reviewed, and often utterly ignorant) press on the same level as even that most-basic level of quality control is not likely to help with that problem. Though I suppose it'll help confine the bogus research to China, instead of causing issues with academia at-large.
[ link to this | view in chronology ]
Attribute, unit of measure?
[ link to this | view in chronology ]
Wow
I'm disappointed that techdirt saw fit to report it as in any way positive.
The key problem here is that it is impossible to truly judge the value of any research until so much time has passed that the judgement itself is of little value.
As for China being a leader - well they are certainly taking over the infrastructure of research on the back of sheer numbers and money - but (probably because of the nature of the Chinese educational system) we have yet to see much genuine originality coming from there.
[ link to this | view in chronology ]
Re: Wow
It really comes down to the fact that with the media becoming more focused on being entertaining for greater profits instead of reporting accurately on important events, people all over the world have fewer reliable sources of information in general.
[ link to this | view in chronology ]
Can't wait for the first citation of the Time Cube, or the Chinese government admitting their space program is a sham and that all the rockets they sent up actually exploded upon collision with the firmament.
Honestly, i can see this abused for the political agenda, though.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Terrible
The dumbing down of our populace is expanding at a increasing rate every year. Here's a great way to help it increase. Fortunately, the grown-ups will continue with actual, useful ways to gauge impact factors for both commercial and open-access journals.
The fact that the author here opens with a mention of "so-called 'impact factors'" speaks volumes about his own education level.
[ link to this | view in chronology ]