Making research count: applying H-indexes within and between disciplines to analyse publication cultures

See on Scoop.itDual impact of research; towards the impactelligent university


This paper demonstrates that it is possible both to examine H-index scores within disciplines, and to  create discipline-normalized H-indexes for interdisciplinary comparisons. The same approach can be applied to other bibliometric measures and be used to improve comparative metrics.


This is especially important for several common types of bibliometric comparisons. The first is international rankings, such as the Shanghai Jiao Tong Academic Ranking of World Universities (ARWU, or Shanghai Rankings, as it is colloquially known). In the 2011 edition, 20% of this ranking was determined by research output. This was measured by number of papers cited in Nature and Science, and papers indexed in Science Citation Index-expanded, and Social Science Citation Index (which were given double the weight of papers indexed in Science Citation Index-expanded). Another 20% was determined by “highly cited researchers in 21 broad subject categories.”20 The absence of discipline-normalization means that this ranking does not control for publication cultures and the biases that they can generate. As a consequence, institutions with more faculty members in highly productive disciplines have an advantage. This issue is not unique to ARWU – while some ranking systems break out their comparisons by field of study, none to date have implemented valid discipline-normalization.


Secondly, comparisons within institutions can be strengthened by discipline normalization. When institutions assess the relative performance of academic departments or units, often as part of unit
review processes, using accurate discipline-normalization to control for publication culture provides a more valid comparison.


Lastly, bibliometrics are increasingly applied to hiring decisions within universities. When candidates’ fields of study differ (as would be the case for a dean of science, for example), discipline normalization helps to ensure that the hiring committee understands differences in publication records (and corresponding H-indexes) in the context of differences in publication cultures.


It’s time for applications of bibliometrics to become more sensitive to publication culture. Just as importantly, users of bibliometrics need to be aware of the dramatic biases that publication culture can generate. We hope that this paper inspires both scholars and professionals to use accurate and normalized measures when applying bibliometrics to their research, assessment, and decision-making activities.




Jarvey, P., Usher, A. and McElroy, L. 2012. Making Research Count: Analyzing Canadian
Academic Publishing Cultures. Toronto: Higher Education Strategy Associates.



See on


About Wilfred Mijnhardt
RMIMR is my virtual playground, a place to reflect on issues from my professional context, my job as Policy Director at Rotterdam School of Management, Erasmus University (RSM). RSM is the international university based business school at Erasmus University Rotterdam. More info here: Here is my list of relevant publications on the topic of my RMIMR weblog: The rss feed for my RMIMR collection is here: Here is my other weblog on impact of research:

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

RSS Business School News

  • An error has occurred; the feed is probably down. Try again later.
Locations of Site Visitors
%d bloggers like this: