ESI: only 20 ISI journals in Economics & Business receive on average at least 20 citations per article over the last ten years

Essential Science Indicators has been updated as of September 1, 2012 to cover a 10-year plus 6-month period, January 1, 2002-June 30, 2012.

The field of economics & Business covers 289 journals in the essential science Indicators of Web of Science. Only about 20 journals received on avarage at least 20 citations per paper in the period covered by the ESI.

Here is the list:

Journal	Papers	Citations	Citations Per Paper
QUART J ECON	397	16894	42,55
ACAD MANAGE REV	377	15731	41,73
ADMIN SCI QUART	191	7854	41,12
J ECON LIT	200	8111	40,55
MIS QUART	380	14346	37,75
ACAD MANAGE J	631	23325	36,97
J FINAN	873	28216	32,32
J MARKETING	469	14346	30,59
J POLIT ECON	378	11526	30,49
STRATEG MANAGE J	717	20049	27,96
J ECON PERSPECT	457	11199	24,51
ORGAN SCI	613	14996	24,46
ECONOMETRICA	628	15223	24,24
J MANAGE	488	11413	23,39
J ECON GROWTH	132	3038	23,02
J FINAN ECON	932	20463	21,96
REV ACC STUD	57	1142	20,04
J ACCOUNT ECON	351	7028	20,02
J OPER MANAG	465	9304	20,01

Crowdsourced ranking of journals in Management; any news?

Management researchers love journal rankings. A new way to get the judgement of thousands of voters was launched recently by Teppo Felin, Brigham Young University. After a few weeks already 80.000 votes were submitted to rank more than 100 journals in the field of management. Voters can submit a vote as many times as they like.  After filtering out about 20.000 automated robot votes, an impressive amount of 60.000 votes remained.

There are already some blogposts on this ranking where peers discuss the pro’s and cons of performing the ranking this way.

The crowd initiative is here @ All Our Ideas: http://www.allourideas.org/management

Besides the obvious trouble to compare all these journals (a nearly impossible task), one can ask what the ‘wisdom of the crowds’ adds to already existing lists of the top journals in the field.

A lot of normal suspects turn up in the top 25: (as of today 22/01/2011, with 80.000 votes, the results before the cleaning).

A score of 100 means a journal is expected to win all the time when compared with another journal in the contest and a score of zero means the journal is expected to lose all the time. There are no zero scores in this ranking (yet).

 

1. JOURNAL OF APPLIED PSYCHOLOGY (score: 81)
2. ADMINISTRATIVE SCIENCE QUARTERLY (76)
3. ACADEMY OF MANAGEMENT REVIEW (74)
4. ORGANIZATION SCIENCE (73)
5. ACADEMY OF MANAGEMENT JOURNAL (72)
6. JOURNAL OF MANAGEMENT (71)
7. MIS QUARTERLY (70)
8. STRATEGIC MANAGEMENT JOURNAL (68)
9. ORGANIZATIONAL BEHAVIOR AND HUMAN DECISION PROCESSES (67)
10. MANAGEMENT SCIENCE (66)
11. DECISION SCIENCES (65)
12. ORGANIZATIONAL RESEARCH METHODS (64)
13. JOURNAL OF ORGANIZATIONAL BEHAVIOR (63)
14. JOURNAL OF MANAGEMENT STUDIES (63)
15. PERSONNEL PSYCHOLOGY (63)
16. JOURNAL OF INTERNATIONAL BUSINESS STUDIES (60)
17. RESEARCH IN ORGANIZATIONAL BEHAVIOR (60)
18. ACADEMY OF MANAGEMENT PERSPECTIVES (60)
19. HUMAN RELATIONS (60)
20. LEADERSHIP QUARTERLY (59)
21. ORGANISATION STUDIES (59)
22. JOURNAL OF OPERATIONS MANAGEMENT (59)
23. HARVARD BUSINESS REVIEW (58)
24. MIT SLOAN MANAGE REVIEW (57)
25. HUMAN RESOURCE MANAGEMENT-US (57)

The initiator of the survey reported today (29-jan-2011) on the results:

So, what can a crowdsourced ranking tell us?  Nothing definitive is my guess, though I’m sure other rankings don’t necessarily give us a definitive signal either.  Though I do think that aggregated perceptions perhaps give us another data point when evaluating and comparing journals (along with impact and influence factors and other, more “objective” measures).  These rankings can of course mirror extant rankings (raising causal questions). But they might also capture more up-to-date anticipations of future performance. For example, the UC Irvine Law School (established in 2008) has not graduated a single student, though the school is already well within the top 50 in the crowdsourced ranking.

Lots of other questions can be raised, specifically related to a management journal ranking like this.  For example, should micro and macro journals be lumped together like this?  And certainly disciplinary journals play a large role in management – should they be included (sociology, psychology, economics)?

Strategic “gaming” of the results of course can also occur.  For example, I ended up having to delete some 25,000+ automatically generated votes (it looked like a computer script was created to throw the ranking off), votes that were explicitly cast to sabotage the effort (the African Journal of Management beat all the top journals according to this mega, robo-voter).  Though, it is interesting to see how the “crowd” essentially plays a role in averaging bias and in putting a check on strategic voting.

Ironically, I’m actually not one to necessarily really care about journal rankings like this.  I wonder whether article-effects trump journal-effects?  (I believe Joel Baum has a provocative paper on this.)  Of course I read and submit to “top” journals, but there are many “lesser” (ahem)  journals that are just as much on my radar screen, for example Industrial and Corporate Change, Managerial and Decision Economics andStrategic Organization. Obsessions with journal standing can detract from ideas.

Pragmatically, yes, journal rankings matter: promotions indirectly depend on it, as do resource flows etc.  So, perhaps a “democratic,” crowdsourced ranking like this can provide additional information for decision-makers and scholars in the field.

 

The top 15 list after filtering out the 20.000 automated robot-votes: the picture is now even more ‘traditional’.

	Management Journals	%
1	Administrative Science Quarterly	90.43%
2	Academy of Management Journal	90.41%
3	Academy of Management Review	88.94%
4	Organization Science	88.31%
5	Strategic Management Journal	84.42%
6	Journal of Applied Psychology	84.00%
7	Management Science	82.56%
8	Journal of Management	82.46%
9	OBHDP	78.93%
10	Organizational Research Methods	74.06%
11	Journal of Organizational Behavior	72.79%
12	Personnel Psychology	71.93%
13	Journal of Management Studies	71.10%
14	Research in OB	70.37%
15	Organization Studies	69.68%

 

Where do all these votes come from?

The whole picture is US en EU dominated. For example: only 16 votes from Shanghai, China are recorded.

The complete ranking is here as PDF file of the ranking website (22/01/2011): https://rmimr.files.wordpress.com/2011/01/all-our-ideas-crwodsourcing-management-journals-20110122.pdf

Science-metrix develops new journals classification system (and interactive webtool) using both the Web of Science (Thomson Reuters) and Scopus (Elsevier).

A new set of tools recently released (first public release: 2010-12-01 (v1.00)) by Science-Metrix Inc. seeks to improve the way we talk about and understand science. The US/Canada-based research evaluation firm has developed a new, multi-lingual (18 languages!) classification of scientific journals, which is accompanied by an interactive web tool.  The journal classification, which covers 15,000 peer-reviewed scientific journals, is translated by more than 22 international experts who volunteered their time and expertise, making the tools available to a worldwide audience.  The complete journals list is available for download as excel sheet.

The interactive ‘Scientific Journals Ontology Explorer’ allows users to visualise the links between 175 scientific specialties (subject categories) in 18 languages, from Arabic to Swedish.

The visualization contains 3 different views: a circular “Subfield Citation Wheel” (representing both citations as well as references), a “Field Citation Wheel” (showing the links between distinct scientific disciplines) and a network “Map of Science” (revealing similarities between disciplines by relative distance). The goal of this visualization is to show people how science spans a broad universe and how interlinked scientific research actually is.

How is the field of Business & Management covered in the journals list?

The field of  Economics & Business (as part of the domain Economic & Social Sciences) contains 822 journals (this is 5,4% of he total Science metrix list)  in 12 subfields, where Business & Management is included with 222 journals (27%):

Every journal is classified into only one category.

Subfields of ‘Economics & Business’	N	%
Accounting	32	3,9%
Agricultural Economics & Policy	27	3,3%
Business & Management	222	27,0%
Development Studies	42	5,1%
Econometrics	13	1,6%
Economic Theory	13	1,6%
Economics	244	29,7%
Finance	63	7,7%
Industrial Relations	21	2,6%
Logistics & Transportation	49	6,0%
Marketing	61	7,4%
Sport, Leisure & Tourism	35	4,3%
Total Journals	822	100%

I will compare these with our ERIM Journals list, the ISI quartiles and the SJR (scopus) quartiles scores to see how the list is structured in terms of quality layers.

(I wil add these details later this week.)

In the ontology browser, you can create a map of science and learn how the field of business and management is connected to other subject categories. I have selected the closest fields in the screenshot below.

About Science-Metrix:

Science-Metrix is the world’s largest independent firm dedicated to scientometrics, technometrics, and science and technology (S&T) evaluation. The firm’s core business involves supporting evidence-based decision-making with strong empirical data and sound theoretical approaches. This contract research organization combines qualitative and quantitative techniques to deliver high quality program evaluations, performance and outcome assessments, and evaluation frameworks. Every Science-Metrix report is produced by a team of dedicated high-calibre experts and relies on the world-class data found in the Scopus, Web of Science and Questel databases.

How are the stocks (top journals) performing over a period of 28 years?

Sciencewatch has published the latest Journal Performance Indicators (JPI’s) for the field of business.  This chart shows which journals perform in the top over decades.  It shows that the very top of the field is very stable over a longer period.  Publishing in these top journals can be seen as a stable investment for researchers; it increases the possibility of a good citation yield.

The table below compares the citation impact of journals in the field of business as measured over three different time spans. The left-hand column ranks journals based on their 2008 “impact factor,” as enumerated in the current edition of Journal Citations Report. The 2008 impact factor is calculated by taking the number of all current citations to source items published in a journal over the previous two years and dividing by the number of articles published in the journal during the same period–in other words, a ratio between citations and recent citable items published. The rankings in the next two columns show impact over longer time spans, based on figures from Journal Performance Indicators. In these columns, total citations to a journal’s published papers are divided by the total number of papers that the journal published, producing a citations-per-paper impact score over a five-year period (middle column) and a 28-year period (right-hand column).

Journals Ranked by Impact: Business

Rank	2008 Impact Factor	Impact 2004-08	Impact 1981-2008
1	Academy of Management Review (6.13)	Administrative Science Quarterly (9.83)	Administrative Science Quarterly (89.77)
2	Academy of Management Journal (6.08)	Academy of Management Journal (9.79)	Academy of Management Review (66.56)
3	Journal of Retailing (4.10)	Academy of Management Review (8.91)	Journal of Marketing (53.84)
4	Journal of Marketing (3.60)	Journal of Marketing (8.72)	Academy of Management Journal (50.48)
5	Strategic Management Journal (3.34)	Strategic Management Journal (7.07)	Strategic Management Journal (46.01)
6	Marketing Science (3.31)	Journal of Management (6.35)	Journal of Consumer Research (39.30)
7	Journal of Management (3.08)	Marketing Science (6.10 )	Journal of Marketing Research (36.34)
8	Journal of International Business Studies (2.99)	Journal of Consumer Psychology (5.30)	Journal of Management (28.14)
9	Administrative Science Quarterly (2.85)	Journal of International Business Studies (5.05)	Sloan Management Review (20.38)
10	Journal of Consumer Psychology (2.84)	Journal of Organizational Behavior (4.85)	Marketing Science (20.36)

Another new Citation Impact tool on Scopus data: Scimago

Declan Butler, Free journal-ranking tool enters citation market, Nature News, January 2, 2008. Excerpt:

A new [OA] Internet database lets users generate on-the-fly citation statistics of published research papers for free. The tool also calculates papers’ impact factors using a new algorithm similar to PageRank, the algorithm Google uses to rank web pages. The open-access database is collaborating with Elsevier, the giant Amsterdam-based science publisher, and its underlying data come from Scopus, a subscription abstracts database created by Elsevier in 2004.

The SCImago Journal & Country Rank database was launched in December by SCImago,

Thomson is also under fire from researchers who want greater transparency over how citation metrics are calculated and the data sets used. In a hard-hitting editorial published in Journal of Cell Biology in December, Mike Rossner, head of Rockefeller University Press, and colleagues say their analyses of databases supplied by Thomson yielded different values for metrics from those published by the company (M. Rossner et al . J. Cell Biol. 179, 1091–1092 ; 2007). Thomson, they claim, was unable to supply data to support its published impact factors. “Just as scientists would not accept the findings in a scientific paper without seeing the primary data,” states the editorial, “so should they not rely on Thomson Scientific’s impact factor, which is based on hidden data.”

It also includes a new metric: the SCImago Journal Rank (SJR).

The familiar impact factor created by industry leader Thomson Scientific, based in Philadelphia, Pennsylvania, is calculated as the average number of citations by the papers that each journal contains. The SJR also analyses the citation links between journals in a series of iterative cycles, in the same way as the Google PageRank algorithm. This means not all citations are considered equal; those coming from journals with higher SJRs are given more weight. The main difference between SJR and Google’s PageRank is that SJR uses a citation window of three years. See Table 1

I tested some testing on the marketing research subfield of business and management (see screenshot). I ranked the list according to total cites over the last 3 years.

SJR versus JCR:

Let’s take the highest ranked journal form Scimago: Journal of Marketing (sjr 0,107) and compare it with the JCR citation trend. JOM has the higest impactfactor i the ISI subjectcategory Business for 2006. So in general this would mean that the best journals come up equally. But it remains a situation of comparing apples and oranges because the subject categories differ between Scopus and ISI. So the relative position of a journal is different in the two measure systems.

The citation chain: Eigenfactor maps the researchers citation trail and the amount of time he spends at each citation along the track.

Another way of mapping and ranking is developed, which ranks journals much as Google ranks websites; the Eigenfactor. This new concept and facility is developed as part of a non-commercial academic research project sponsored by the Bergstrom lab in the Department of Biology at the University of Washington. They aim to develop novel methods for evaluating the influence of scholarly periodicals and for mapping the structure of academic research. The Eigenfactor score of a journal is an estimate of the percentage of time that library users spend with that journal. The Eigenfactor algorithm corresponds to a simple model of research in which readers follow chains of citations as they move from journal to journal. Imagine that a researcher goes to the library and selects a journal article at random. After reading the article, the researcher selects at random one of the citations from the article. She then proceeds to the journal that was cited, reads a random article there, and selects a citation to direct her to her next journal volume. The researcher does this ad infinitum. The amount of time that the researcher spends with each journal gives us a measure of that journal’s importance within network of academic citations. The amount of time that a researcher spends with each journal gives an estimate of the amount of time that real researchers spend with each journal. The developers of this new concept use mathematics to simulate this process.

Just to try I searched the Business category in the search option of this facility. As a result of this a list of 717 journals was presented. For all the journals the article influence was listed. The following screenshot gives an overview of the result:

Short impact (JCR) and long impact (JPI) of journals

We are updating the ERIM journalslist at the moment and as part of this procedure we are evaluating different measures of status and prestige of journals. The ERIM list is our main vehicle for focus and quality-policy in our filed of Research in Management at Erasmus University. We update this list every two year.

One of the issues is ofcourse the period we take into account when we look at the journal performance. We can use the JCR reports with the impact factors of ISI. This gives an inpression of the last two years. We could also take a longer time horizon for our classifications of journals. The following example takes a five year and a 20+ year perspective on the performance of journals for the subject category of Business. See how the long term quality journals perform here. It helps me to develop a more balanced view on impact and prestige of journals. If only those data were all free available on the web…
source: http://www.in-cites.com/research/2003/july_28_2003-1.html

Journals Ranked by Impact: Business

Rank	2002 Impact Factor	Impact 1998-2002	Impact 1981-2002
1	Acad. Management Rev. (3.70)	Acad. Management Rev. (10.24)	Admin. Science Quart. (51.85)
2	Strategic Mgmt. J. (3.09)	Admin. Science Quart. (8.42)	Acad. Management Rev. (39.91)
3	Sloan Management Rev. (3.04)	Journal of Marketing (5.73)	Journal of Marketing (33.34)
4	Admin. Science Quart. (2.63)	Strategic Mgmt. J. (5.55)	J. Consumer Research (30.76)
5	Acad. Management J. (2.54)	California Mgmt. Rev. (5.46)	Acad. Management J. (30.18)
6	J. Consumer Research (2.31)	Acad. Management J. (5.44)	Strategic Mgmt. J. (25.76)
7	Journal of Marketing (2.29)	J. Consumer Research (4.63)	J. Marketing Research (23.91)
8	Harvard Business Rev. (2.03)	J. Acad. Market. Sci. (4.53)	Journal of Business (16.07)
9	Marketing Science (1.94)	Sloan Management Rev. (4.36)	Journal of Management (15.19)
10	J. Acad. Market. Sci. (1.93)	J. Marketing Research (3.87)	Marketing Science (12.48)

And the same exercise for the subject category Business Finance:

source: http://www.in-cites.com/research/2004/july_19_2004-1.html

Journals Ranked by Impact: Business, Finance

Rank	2003 Impact Factor	Impact 1999-2003	Impact 1981-2003
1	J. Account./Economics (3.84)	Journal of Finance (6.85)	J. Financial Economics (34.68)
2	Journal of Finance (3.27)	J. Financial Economics (5.60)	Journal of Finance (21.93)
3	J. Financial Economics (2.72)	IMF Staff Papers (4.87)	J. Monetary Economics (20.39)
4	Rev. Financial Studies (2.20)	J. Account./Economics (4.24)	J. Account./Economics (15.32)
5	J. Accounting Research (1.52)	Rev. Financial Studies (4.12)	Rev. Financial Studies (14.82)
6	Accounting Review (1.45)	J. Accounting Research (3.16)	J. Finan. Quant. Analys. (9.37)
7	J. Industrial Econ. (1.33)	Mathematical Finance (2.90)	J. Risk & Uncertainty (9.22)
8	World Bank Econ. Rev. (1.27)	World Bank Econ. Rev. (2.83)	Journal of Accounting (8.98)
9	Financial Mgmt. (1.21)	J. Monetary Economics (2.72)	J. Industrial Econ. (8.81)
10	J. Monetary Economics (1.15)	J. Risk & Uncertainty (2.61)	J. Money Credit & Banking (7.80)