Another new Citation Impact tool on Scopus data: Scimago

Declan Butler, Free journal-ranking tool enters citation market, Nature News, January 2, 2008. Excerpt:

A new [OA] Internet database lets users generate on-the-fly citation statistics of published research papers for free. The tool also calculates papers’ impact factors using a new algorithm similar to PageRank, the algorithm Google uses to rank web pages. The open-access database is collaborating with Elsevier, the giant Amsterdam-based science publisher, and its underlying data come from Scopus, a subscription abstracts database created by Elsevier in 2004.

The SCImago Journal & Country Rank database was launched in December by SCImago,

Thomson is also under fire from researchers who want greater transparency over how citation metrics are calculated and the data sets used. In a hard-hitting editorial published in Journal of Cell Biology in December, Mike Rossner, head of Rockefeller University Press, and colleagues say their analyses of databases supplied by Thomson yielded different values for metrics from those published by the company (M. Rossner et al . J. Cell Biol. 179, 1091–1092 ; 2007). Thomson, they claim, was unable to supply data to support its published impact factors. “Just as scientists would not accept the findings in a scientific paper without seeing the primary data,” states the editorial, “so should they not rely on Thomson Scientific’s impact factor, which is based on hidden data.”

It also includes a new metric: the SCImago Journal Rank (SJR).

The familiar impact factor created by industry leader Thomson Scientific, based in Philadelphia, Pennsylvania, is calculated as the average number of citations by the papers that each journal contains. The SJR also analyses the citation links between journals in a series of iterative cycles, in the same way as the Google PageRank algorithm. This means not all citations are considered equal; those coming from journals with higher SJRs are given more weight. The main difference between SJR and Google’s PageRank is that SJR uses a citation window of three years. See Table 1

I tested some testing on the marketing research subfield of business and management (see screenshot). I ranked the list according to total cites over the last 3 years.

SJR versus JCR:

Let’s take the highest ranked journal form Scimago: Journal of Marketing (sjr 0,107) and compare it with the JCR citation trend. JOM has the higest impactfactor i the ISI subjectcategory Business for 2006. So in general this would mean that the best journals come up equally. But it remains a situation of comparing apples and oranges because the subject categories differ between Scopus and ISI. So the relative position of a journal is different in the two measure systems.

A New Era in Citation and Bibliometric Analyses: Web of Science, Scopus, and Google Scholar

Lokman I. Meho and Kiduk Yang
School of Library and Information Science, Indiana University, 2007

Abstract:

Academic institutions, federal agencies, publishers, editors, authors, and librarians increasingly rely on citation analysis for making hiring, promotion, tenure, funding, and/or reviewer and journal evaluation and selection decisions. The Institute for Scientific Information’s (ISI) citation databases have been used for decades as a starting point and often as the only tools for locating citations and/or conducting citation analyses. ISI databases (or Web of Science), however, may no longer be adequate as the only or even the main sources of citations because new databases and tools that allow citation searching are now available. Whether these new databases and tools complement or represent alternatives to Web of Science (WoS) is important to explore. Using a group of 15 library and information science faculty members as a case study, this paper examines the effects of using Scopus and Google Scholar (GS) on the citation counts and rankings of scholars as measured by WoS. The paper discusses the strengths and weaknesses of WoS, Scopus, and GS, their overlap and uniqueness, quality and language of the citations, and the implications of the findings for citation analysis. The project involved citation searching for approximately 1,100 scholarly works published by the study group and over 200 works by a test group (an additional 10 faculty members). Overall, more than 10,000 citing and purportedly citing documents were examined. WoS data took about 100 hours of collecting and processing time, Scopus consumed 200 hours, and GS a grueling 3,000 hours.

Conclusions by the authors:

The study found that the addition of Scopus citations to those of WoS could significantly alter the ranking of scholars. The study also found that GS stands out in its coverage of conference proceedings as well as international, non-English language journals, among others. GS also indexes a wide variety of document types, some of which may be of significant value to researchers. The use of Scopus and GS, in addition to WoS, reveals a more comprehensive and accurate picture of the extent of the scholarly relationship between LIS and other fields, as evidenced by the unique titles that cite LIS literature (e.g., titles from Cognitive Science, Computer Science, Education, and Engineering, to name only a few). Significantly, this study has demonstrated that:

1. Although WoS remains an indispensable citation database, it should not be used alone for locating citations to an author or title, and, by extension, journals, departments, and countries; Scopus should be used concurrently.
2. Although Scopus provides more comprehensive citation coverage of LIS and LIS-related literature than WoS for the period 1996-2005, the two databases complement rather than replace each other.
3. While both Scopus and GS help identify a considerable number of citations not found in WoS, only Scopus significantly alters the ranking of scholars as measured by WoS.
   Although GS unique citations are not of the same quality as those found in WoS or Scopus, they could be very useful in showing evidence of broader international impact than could possibly be done through the two proprietary databases.
4. GS value for citation searching purposes is severely diminished by its inherent problems. GS data are not limited to refereed, high quality journals and conference proceedings. GS is also very cumbersome to use and needs significant improvement in the way it displays search results and the downloading capabilities it offers for it to become a useful tool for large-scale citation analyses.
5. Given the low overlap or high uniqueness between the three tools, they may all be necessary to develop more accurate maps or visualizations of scholarly networks and impact both within and between disciplines (Börner, Chen, & Boyack, 2003; Börner, Sanyal, & Vespignani, 2006; Small, 1999; White & McCain, 1997).
6. Each database or tool requires specific search strategy(ies) in order to collect citation data, some more accurately and quickly (i.e., WoS and Scopus) than others (i.e., GS).

(Accepted for publication in the Journal of the American Society for Information Science and Technology)

The Rise and Rise of Citation Analysis

Meho, Lokman I. (2007) The Rise and Rise of Citation Analysis.

Full text available as:PDF -.

Abstract:

With the vast majority of scientific papers now available online, this paper (accepted for publication in Physics World) describes how the Web is allowing physicists and information providers to measure more accurately the impact of these papers and their authors. Provides a historical background of citation analysis, impact factor, new citation data sources (e.g., Google Scholar, Scopus, NASA’s Astrophysics Data System Abstract Service, MathSciNet, ScienceDirect, SciFinder Scholar, Scitation/SPIN, and SPIRES-HEP), as well as h-index, g-index, and a-index.

The author shows his awareness with the new dimensions of publishing:

Scientists now need to make it their job to disseminate their work on as many platforms and in as many different ways as possible, such as publishing in open access and high-impact journals, and posting their work in institutional repositories, personal homepages and e-print servers, if they want their peers to be aware of, use and ultimately cite their work. Publishing a journal article is now only the first step in disseminating or communicating one’s work; the Web provides a multitude of methods and tools to publicize its scholarly worth.

Author Affiliation Index (AAI); the pattern of authorship/coauthorship across journals

Although this recent study by Chen en Huan (Journal of Corporate Finance 2007) is focused on the field of Finance, the concept of AAI is valuable for all fields of management research. The AAI is calculated as the ratio of articles authored by faculty at the world’s top 80 finance programs divided by the total number of articles by all authors. It provides provides academics with a credible alternative measurement of journal quality, in ddition to the traditional survey-based and citation-based journal ratings.

Abstract:

In this paper we use a new method to rank finance journals and study the pattern of authorship/coauthorship
across journals. Defined as the ratio of articles authored by faculty at the world’s top 80 finance
programs to the total number of articles by all authors, the Author Affiliation Index is a cost-effective and
intuitively easy-to-understand approach to journal rankings. Forty-one finance journals are ranked
according to this index. If properly constructed, the Author Affiliation Index provides an easy and credible
way to supplement the existing journal ranking methods. Our ranking system reveals the journal–researcher
clientele, and we find that collaboration (co-authoring) between faculty within elite programs exists only in
top-tier and near-top-tier journals. Publications in lower-tier journals by researchers of elite programs are
driven by their co-authors. Collaboration between faculty in elite and non-elite programs, however, is more
prevalent than that within elite programs across all tiers of journals. Co-authorship among top 80 programs,
nevertheless, is more common in top-tier journals, while co-authorship between top 80 and other programs
is more dominant in lower-ranked journals.

Annual Dutch citation-topparade (2000-2004) and tipparade (2004) for economists; not the complete picture

The ESB journal has published it’s list of top economists (esb-economentop_20070405.pdf) again this week. The topparade measures the citation in the Web of science in the period 2000-2004 for articles published in the period 1988-2004. This is a moving time window. The six ‘big’ economics faculties in the Netherlands (EUR, RUG, UM, UvA, UvT, VU) each submit 20 of their best researchers to the ranking, while the relatively smaller institutes (UU, WUR, DNB, TUD) select each 5 names. The ranking does not cover all of he faculties in the Netherlands, because some choose not to participate and the faculties of management sometimes do not participate. Still the ranking gives an indication of where the best academic research in the Netherlands is done. The ranking also gives an overview of the institutions where the best economists are affiliated. I’m proud to mention that Erasmus University Rotterdam has jumped Tilburg University to the first place in the institutions list this year.

The citation chain: Eigenfactor maps the researchers citation trail and the amount of time he spends at each citation along the track.

Another way of mapping and ranking is developed, which ranks journals much as Google ranks websites; the Eigenfactor. This new concept and facility is developed as part of a non-commercial academic research project sponsored by the Bergstrom lab in the Department of Biology at the University of Washington. They aim to develop novel methods for evaluating the influence of scholarly periodicals and for mapping the structure of academic research. The Eigenfactor score of a journal is an estimate of the percentage of time that library users spend with that journal. The Eigenfactor algorithm corresponds to a simple model of research in which readers follow chains of citations as they move from journal to journal. Imagine that a researcher goes to the library and selects a journal article at random. After reading the article, the researcher selects at random one of the citations from the article. She then proceeds to the journal that was cited, reads a random article there, and selects a citation to direct her to her next journal volume. The researcher does this ad infinitum. The amount of time that the researcher spends with each journal gives us a measure of that journal’s importance within network of academic citations. The amount of time that a researcher spends with each journal gives an estimate of the amount of time that real researchers spend with each journal. The developers of this new concept use mathematics to simulate this process.

Just to try I searched the Business category in the search option of this facility. As a result of this a list of 717 journals was presented. For all the journals the article influence was listed. The following screenshot gives an overview of the result:

Reflections on Google Scholar and Hirsch index

Anne Will Harzing reflects on these citation developments form the perspective of the business and management field. She matches these sources with the Publish or Perish software. This website is one of the soources very relevant to stay in touch with. Two new white papers were added: Reflections on Google Scholar and Reflectionson the h-index. These papers discuss the validity, assumptions, and limitationsof the underlying sources and methods used by Publish or Perish.

[http://www.harzing.com/pop_gs.htm]
[http://www.harzing.com/pop_hindex.htm]
[http://www.harzing.com/pop.htm]

Competitors for the Web of Science?

Dana L. Roth has published an overview of the current alternatives. Services currently offering cited reference searching include (and I give the complete list to have a good look at it):

1. Chemical Abstracts/SciFinder/SciFinder Scholar
2. NASA Astrophysics Data System Abstract Service
3. Amazon.com’s ‘Search Inside this Book’ program
4. Scopus
5. Scitation/Spin Web
6. PROLA (Physical Review Online Archive)
7. Citation Bridge (US Patents)
8. US Patent and Trademark Office
9. Google Scholar
10. Optics InfoBase
11. CiteSeer
12. Science Direct
13. PsycINFO
14. IEEE Xplore
15. Spires HEP
16. IOP (Institute of Physics)
17. CrossRef

She concludes her overview:

… Recent developments of ‘competitors’ to the WoS, while interesting and useful for quick links to some citing references, are clearly not a substitute for a comprehensive citation search. WoS currently indexes ~ 8000 journals from the sciences, engineering, social sciences and the humanities, and clearly remains the primary resource for citation searching….

Source: Dana L. Roth,The emergence of competitors to the Science CitationIndex and the Web of Science (pdf 6pp), Current Science Online, Vol. 89,No. 9, 10 November 2005,[http://www.ias.ac.in/currsci/nov102005/1531.pdf]

Short impact (JCR) and long impact (JPI) of journals

We are updating the ERIM journalslist at the moment and as part of this procedure we are evaluating different measures of status and prestige of journals. The ERIM list is our main vehicle for focus and quality-policy in our filed of Research in Management at Erasmus University. We update this list every two year.

One of the issues is ofcourse the period we take into account when we look at the journal performance. We can use the JCR reports with the impact factors of ISI. This gives an inpression of the last two years. We could also take a longer time horizon for our classifications of journals. The following example takes a five year and a 20+ year perspective on the performance of journals for the subject category of Business. See how the long term quality journals perform here. It helps me to develop a more balanced view on impact and prestige of journals. If only those data were all free available on the web…
source: http://www.in-cites.com/research/2003/july_28_2003-1.html

Journals Ranked by Impact: Business

Rank	2002 Impact Factor	Impact 1998-2002	Impact 1981-2002
1	Acad. Management Rev. (3.70)	Acad. Management Rev. (10.24)	Admin. Science Quart. (51.85)
2	Strategic Mgmt. J. (3.09)	Admin. Science Quart. (8.42)	Acad. Management Rev. (39.91)
3	Sloan Management Rev. (3.04)	Journal of Marketing (5.73)	Journal of Marketing (33.34)
4	Admin. Science Quart. (2.63)	Strategic Mgmt. J. (5.55)	J. Consumer Research (30.76)
5	Acad. Management J. (2.54)	California Mgmt. Rev. (5.46)	Acad. Management J. (30.18)
6	J. Consumer Research (2.31)	Acad. Management J. (5.44)	Strategic Mgmt. J. (25.76)
7	Journal of Marketing (2.29)	J. Consumer Research (4.63)	J. Marketing Research (23.91)
8	Harvard Business Rev. (2.03)	J. Acad. Market. Sci. (4.53)	Journal of Business (16.07)
9	Marketing Science (1.94)	Sloan Management Rev. (4.36)	Journal of Management (15.19)
10	J. Acad. Market. Sci. (1.93)	J. Marketing Research (3.87)	Marketing Science (12.48)

And the same exercise for the subject category Business Finance:

source: http://www.in-cites.com/research/2004/july_19_2004-1.html

Journals Ranked by Impact: Business, Finance

Rank	2003 Impact Factor	Impact 1999-2003	Impact 1981-2003
1	J. Account./Economics (3.84)	Journal of Finance (6.85)	J. Financial Economics (34.68)
2	Journal of Finance (3.27)	J. Financial Economics (5.60)	Journal of Finance (21.93)
3	J. Financial Economics (2.72)	IMF Staff Papers (4.87)	J. Monetary Economics (20.39)
4	Rev. Financial Studies (2.20)	J. Account./Economics (4.24)	J. Account./Economics (15.32)
5	J. Accounting Research (1.52)	Rev. Financial Studies (4.12)	Rev. Financial Studies (14.82)
6	Accounting Review (1.45)	J. Accounting Research (3.16)	J. Finan. Quant. Analys. (9.37)
7	J. Industrial Econ. (1.33)	Mathematical Finance (2.90)	J. Risk & Uncertainty (9.22)
8	World Bank Econ. Rev. (1.27)	World Bank Econ. Rev. (2.83)	Journal of Accounting (8.98)
9	Financial Mgmt. (1.21)	J. Monetary Economics (2.72)	J. Industrial Econ. (8.81)
10	J. Monetary Economics (1.15)	J. Risk & Uncertainty (2.61)	J. Money Credit & Banking (7.80)

Measures of Value of a Journal Beyond the Impact Factor

Anita Coleman gives a nice overview of possible multi-dimensional measure of the value of academic journals next to the traditional impact factor. To measure the value of a journals Coleman first selects three measures, namely journal attraction power, author associativity, and journal consumption power; she redefines two of them as journal measures of affinity (the proportion of foreign authors) and associativity (the amount of collaboration), and calculate these as objective indicators of journal value. To illustrate the multi-dimensional identity of the value of a journal I selected the following list of possible measures from Coleman’s article (in alfabetical order):

1. Acceptance and Rejection rates
2. Adjusted Impact Factor,
3. Article Quality,
4. Author Reputation Score,
5. Average Ranking Position,
6. Circulation size,
7. Citation Rate,
8. Citing Half-Life,
9. Degree of specialization,
10. Disciplinary Impact Factor,
11. Editorial board,
12. Editorial standards,
13. Immediacy Index,
14. Impact Factor,
15. Importance Index
16. Influence Weight,
17. Journal Age,
18. Journal origin and orientation,
19. Mean Response Time,
20. Popularity Factor
21. Readership
22. References per Paper,
23. Reprint distribution,
24. School Reputation Score,
25. Self-citedness,
26. Standing,
27. Type of research covered,
28. Uncitedness,