Individual researchers benefit from online impact assessment, but Online measuring of impact not yet suited for research assessment exercises

Via Scoop.itDual impact of research; towards the impactelligent university

Individual researchers are very interested in evidence of the impact of their publications. Research institutes and independent organisations assessing research have a special interest when comparing groups and organisations for research assessment. Thanks to the possibilities of web based publishing it is now possible to gauge the impact of some publications under certain conditions. New information filters and tools are helping researchers to assess their own progress and to find responses of others to their publications.   The report Users, Narcissism and Control, which was commissioned by SURF, offers a comprehensive overview of the current tracking tools of online publications. The report shows to what extent it is possible to follow in real-time how research results are being downloaded, read, cited, and applied.   Stricter protocols required
The fact that researchers can use these tools does not necessarily mean that this technology is also a legitimate source of information for research assessment. For this application, they need to adhere to a far stricter protocol of data quality and indicator reliability and validity (for example; what does a download imply on the use of the research results). Most new tools do not yet comply with these stricter quality criteria required for them to be used in research assessments.   Tracking tools
The authors of the report ‘Users, narcissism and control’, Paul Wouters and Rodrigo Costas (CWTS), explore the explosion of tracking tools that has accompanied the surge of web based information instruments. A total of 16 quite different tools have been assessed. The authors also highlight the potential risks and disadvantages of the new tracking tools. Just some of the tools discussed are: Google Scholar, Microsoft Academic Research, Total-Impact, PlosONE altmetrics, en F1000.   Frank van Harmelen, professor of Knowledge Representation and Reasoning at the VU University Amsterdam writes on the report: “New web-based metrics for scientific impact will make it possible to observe the developments in science in near real-time. I’m very happy to see that such web-based measures of scientific impact are now being considered by leading scholars involved in the science of measuring and analysing science (Scientometrics)”.   Research assessment exercises
The report also recommends to start a concerted research programme investigating the dynamics, properties, and potential use of new web based metrics. This programme should relate these new measures to the already established indicators for scientific and scholarly impact. It can provide insight in how these developing metrics could be applied in research assessment exercises.   source: Users, narcissism and control – tracking the impact of scholarly publications in the 21st century
SURFfoundation   Paul Wouters – CWTS, Rodrigo Costas – CWTS   Contributors
Wouter Boon – Rathenau Institute
Jeroen Bosman- Utrecht University
Mariken Elsen – Netherlands Organisation for Scientific Research (NWO)
Gert Goris – Erasmus University Rotterdam
Paul Groth – VU University Amsterdam
Henk van den Hoogen – Maastricht University
Wilfred Mijnhardt – Erasmus University Rotterdam
Ronald Snijder – Amsterdam University Press
Maurice Vanderfeesten – SURFfoundation
Ludo Waltman – CWTS, Leiden University   Editors
Marnix van Berchum – SURFfoundation
Keith Russell – SURFfoundation

The fulltext report is here:


About Wilfred Mijnhardt
RMIMR is my virtual playground, a place to reflect on issues from my professional context, my job as Policy Director at Rotterdam School of Management, Erasmus University (RSM). RSM is the international university based business school at Erasmus University Rotterdam. More info here: Here is my list of relevant publications on the topic of my RMIMR weblog: The rss feed for my RMIMR collection is here: Here is my other weblog on impact of research:

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

RSS Business School News

  • An error has occurred; the feed is probably down. Try again later.
Locations of Site Visitors
%d bloggers like this: