
Management researchers love journal rankings. A new way to get the judgement of thousands of voters was launched recently by Teppo Felin, Brigham Young University. After a few weeks already 80.000 votes were submitted to rank more than 100 journals in the field of management. Voters can submit a vote as many times as they like. After filtering out about 20.000 automated robot votes, an impressive amount of 60.000 votes remained.
There are already some blogposts on this ranking where peers discuss the pro’s and cons of performing the ranking this way.
The crowd initiative is here @ All Our Ideas: http://www.allourideas.org/management
Besides the obvious trouble to compare all these journals (a nearly impossible task), one can ask what the ‘wisdom of the crowds’ adds to already existing lists of the top journals in the field.
A lot of normal suspects turn up in the top 25: (as of today 22/01/2011, with 80.000 votes, the results before the cleaning).
A score of 100 means a journal is expected to win all the time when compared with another journal in the contest and a score of zero means the journal is expected to lose all the time. There are no zero scores in this ranking (yet).
- JOURNAL OF APPLIED PSYCHOLOGY (score: 81)
- ADMINISTRATIVE SCIENCE QUARTERLY (76)
- ACADEMY OF MANAGEMENT REVIEW (74)
- ORGANIZATION SCIENCE (73)
- ACADEMY OF MANAGEMENT JOURNAL (72)
- JOURNAL OF MANAGEMENT (71)
- MIS QUARTERLY (70)
- STRATEGIC MANAGEMENT JOURNAL (68)
- ORGANIZATIONAL BEHAVIOR AND HUMAN DECISION PROCESSES (67)
- MANAGEMENT SCIENCE (66)
- DECISION SCIENCES (65)
- ORGANIZATIONAL RESEARCH METHODS (64)
- JOURNAL OF ORGANIZATIONAL BEHAVIOR (63)
- JOURNAL OF MANAGEMENT STUDIES (63)
- PERSONNEL PSYCHOLOGY (63)
- JOURNAL OF INTERNATIONAL BUSINESS STUDIES (60)
- RESEARCH IN ORGANIZATIONAL BEHAVIOR (60)
- ACADEMY OF MANAGEMENT PERSPECTIVES (60)
- HUMAN RELATIONS (60)
- LEADERSHIP QUARTERLY (59)
- ORGANISATION STUDIES (59)
- JOURNAL OF OPERATIONS MANAGEMENT (59)
- HARVARD BUSINESS REVIEW (58)
- MIT SLOAN MANAGE REVIEW (57)
- HUMAN RESOURCE MANAGEMENT-US (57)
The initiator of the survey reported today (29-jan-2011) on the results:
So, what can a crowdsourced ranking tell us? Nothing definitive is my guess, though I’m sure other rankings don’t necessarily give us a definitive signal either. Though I do think that aggregated perceptions perhaps give us another data point when evaluating and comparing journals (along with impact and influence factors and other, more “objective” measures). These rankings can of course mirror extant rankings (raising causal questions). But they might also capture more up-to-date anticipations of future performance. For example, the UC Irvine Law School (established in 2008) has not graduated a single student, though the school is already well within the top 50 in the crowdsourced ranking.
Lots of other questions can be raised, specifically related to a management journal ranking like this. For example, should micro and macro journals be lumped together like this? And certainly disciplinary journals play a large role in management – should they be included (sociology, psychology, economics)?
Strategic “gaming” of the results of course can also occur. For example, I ended up having to delete some 25,000+ automatically generated votes (it looked like a computer script was created to throw the ranking off), votes that were explicitly cast to sabotage the effort (the African Journal of Management beat all the top journals according to this mega, robo-voter). Though, it is interesting to see how the “crowd” essentially plays a role in averaging bias and in putting a check on strategic voting.
Ironically, I’m actually not one to necessarily really care about journal rankings like this. I wonder whether article-effects trump journal-effects? (I believe Joel Baum has a provocative paper on this.) Of course I read and submit to “top” journals, but there are many “lesser” (ahem) journals that are just as much on my radar screen, for example Industrial and Corporate Change, Managerial and Decision Economics andStrategic Organization. Obsessions with journal standing can detract from ideas.
Pragmatically, yes, journal rankings matter: promotions indirectly depend on it, as do resource flows etc. So, perhaps a “democratic,” crowdsourced ranking like this can provide additional information for decision-makers and scholars in the field.
The top 15 list after filtering out the 20.000 automated robot-votes: the picture is now even more ‘traditional’.
|
Management Journals |
% |
1 |
Administrative Science Quarterly |
90.43% |
2 |
Academy of Management Journal |
90.41% |
3 |
Academy of Management Review |
88.94% |
4 |
Organization Science |
88.31% |
5 |
Strategic Management Journal |
84.42% |
6 |
Journal of Applied Psychology |
84.00% |
7 |
Management Science |
82.56% |
8 |
Journal of Management |
82.46% |
9 |
OBHDP |
78.93% |
10 |
Organizational Research Methods |
74.06% |
11 |
Journal of Organizational Behavior |
72.79% |
12 |
Personnel Psychology |
71.93% |
13 |
Journal of Management Studies |
71.10% |
14 |
Research in OB |
70.37% |
15 |
Organization Studies |
69.68% |
Where do all these votes come from?
The whole picture is US en EU dominated. For example: only 16 votes from Shanghai, China are recorded.

The complete ranking is here as PDF file of the ranking website (22/01/2011): https://rmimr.files.wordpress.com/2011/01/all-our-ideas-crwodsourcing-management-journals-20110122.pdf
51.922690
4.470787
Youre invited to share my content:
Like this:
Like Loading...