University Research Rankings’ Landscape Through the eyes of a library professional Based on Ruth’s Ranking in e-Access
For Library Association of Singapore
Ruth A Pagell, Singapore November 2015
1
HKUST surges to no. 28 worldwide and no. 1 in Hong Kong in QS World University Rankings 2015-20 16 SMU LEE KONG CHIAN D \SCHOOL OF BUSINESS DEBUTS IN FT EMBA RANKING 2015
May 14, 2015 QS World University Ranking by Subject Peking U ranks top among Mainland China universities [in].. Philosophy, Modern Languages, History, Linguistics, English Language & Literature, Psychology http://www.ntu.edu.sg/Pages/home.aspx https://www.facebook.com/hkust/posts/10153500031624473 http://english.pku.edu.cn/news_events/news/global/3640.htm http://www.smu.edu.sg/
Drivers for Bibliometrics Research Citations (SCI – 1963)
Faculty comparisons
Institutional benchmarking
National Policy Initiatives (Malaysia, 2015) © Ruth A. Pageli
3
The Players Authors Publishers Institutions Aggregators of articles - sources
© Ruth A. Pageli
4
Sources for Bibliometrics COMMERCIAL Aggregatprs
THOMSON REUTERS Web of Science (WOS) SSCI, SCI-e, A & H INCITES; Journal Citation Reports (JCR); ESI ELSEVIER Scopus SciVal PURE
SPECIALIZED SITES Google Scholar Websites Social Media Repositories Research Gate Mendeley Specialized scholar cites
©Ruth A. Pagell
5
The Players Authors Publishers Aggregators of articles
Institutions
Bibliometricians/Scientometricians Rankers Users(Stakeholders)
© Ruth A. Pageli
6
IREG Stakeholders – Rankings Users
http://www.ireg-observatory.org/en/index.php/initiatives International Ranking Expert Group
The Players Authors Publishers Aggregators of articles
Institutions
Bibliometricians/Scientometricians Rankers Users(Stakeholders)
Information Professionals © Ruth A. Pageli
8
Topics Behind the Rankings Evolution of Rankings – History (RR 1) International and national Policies (RR2) The metrics What we count; How we count (RR3, 13) Providers, Aggregators (RR4, 10, 14, 15) Rankers (5-9, 1,12,15) How we interpret From Ruth’s Rankings in Access http://librarylearningspace.com/ruthsrankings-1/ © Ruth A. Pagell
9
What is the correct ranking? The most relevant? 161 151-200 186 201-250 207 314 327 330
• • • • • • • •
U.S. News U.S. -200 ARWU - 500 NTU - 500 THE - 800 Leiden - 750 U.S. News - 750 QS - 700 SIR - 2838 © Ruth A. Pageli
10
RANKING SINGAPORE TOTAL SG
TYPE NUS
NTU
SMU
ARWU
500
2
Schl
101-150 151-200
NTU
500
2
Schl
42
107
QS (1)
700
3
Multi
12
13
THE
800
2
Multi
26
55
U.S. NEWS
750
3
Multi
49
74
LEIDEN (2)
750
2
Schl
144/29
85/55
SIR (3)
2839
5 /25
Multi
436
350
230
46
Multi
89
103
1095
COMPOSITE
Subject
Subject
BY METRIC
RANKING WEB > 23,800
1:Registration;;2. Default - Size Independent top 10% /Size dependent 3. Permission 11 required; for research; ranking impact; with non-unis; Ranking Web includes Polys
Topics for Discussion Basic Variables
Factors impacting rankings
Institutions
Authors
1. 2. 3. 4. 5.
Metrics
6.
Publications
Size Discipline Publication type Data base coverage-geography Manipulation of data Of rankings interfaces Time (and Age)
SciVal Metrics Guidebook 2014
http://www.elsevier.com/__data/assets/pdf_file/0020/53327/sci val-metrics-guidebook-v1_01-february2014.pdf ©Ruth A. Pagell
12
VARIABLES FOR Basic Units • Institutions –types Universities, research institutes, government
Systems or individual units; number of publications; Peer review
• Publication- types Articles, reviews, books, proceedings
Total output Output per faculty Articles in top n%
• Authors Matching authors to publications, institutions & countries
Disambiguation of authors’ names Assigning institutional credit for multiple authors © Ruth A. Pageli
13
Assigning Credit This is a sample article. Who gets credit? BY Kumar [4]; Page [2]; Wang[3]; Tan [1]
:
COUNTING: Authors and Institutions: WOS 4 – one for each; Leiden ¼ (fractionalized) Reprint Address L Tan Singapore U. School of …… Only Kumar; only Tan Addresses: [1] Singapore University [2] U.S. University 1 [3] U.S. University 2 [4] Indian University
Countries WOS – 3 Leiden= 1/3 Ruth A. Pagell
14
Not-so Basic Metrics – Citations SNOWBALL METRICs Citation count for fixed • Outputs in top time period percentiles – 1%; Citations per FTE 10%... Citations per output Per FTE h-index – for institutions Per publication by discipline • Outputs in top Field weighted citation journals impact http://www. ballmetrics.com/wp-content/uploads/snowball-recipe-book_HR.pdf ©Ruth A. Pagell
15
Determining Rankings Composite score with weighted metrics
Rank only Rank and score Rank, score and raw data (Leiden) Re-rank by metrics
No composite score
Rank by individual metric Recommended metric, i.e. field weighted citation © Ruth A. Pageli
16
Rank and Scoring Distance ARWU 2015
http://www.shanghairanking.com/ARWU2015.html © Ruth A. Pageli
17
SAME RANK- SAME SCORE World Rank 29 Social Sciences THE from 2014-2015 University
Teach InterResearch Citanational tions ing Outlook
Industry SCOR Income E
NUS
67.5
83.6
70.7
53.1
34.9
65.3
U HK
55.7
87.9
70.6
61.1
92.8
65.3
7.5% 3
30% 3*
30%
2.5%
Weight & # 30% of Metrics 5* * Includes reputation
© Ruth A. Pageli
18
Can you modify the rankings? Filter rankings by: Geography – Asia-Pacific, Asia, Middle East
Subject / field – 5 broad fields to >200 individual disciplines Time – change the time period © Ruth A. Pageli
19
Size Matters –Leiden SIZE DEPENDENT NTU -80%
SIZE INDEPENTENT U.S. News 45%
©Ruth A. Pagell from Leiden 2015
20
Field matters
Field
PUBLICATIOS
CITATIONS
% Cited
Neurosciences
222,000
1.877,000
69%
History
122625
31,164
9%
© Ruth A. Pageli
21
Broad Subject Area Matters Citations from Scopus Life Science and Medicine
49.1%
Natural Sciences
27.6%
Engineering & Technology
16.5%
Social Sciences
5.8%
Arts & Humanities
1%
Sowter, Ben (July 27, 2015) Intelligence Unit Blog http://www.iu.qs.com/2015/07/potentialrefinements-in-the-qs-world-university-rankings-2015/ Ruth A. Pagell
22
Publication Type and Citations
A. Pageli From SciVal Metrics Guidebook© Ruth Table 3.2.3
23
Geography /Language Database journal coverage
US and UK English
Language
Localized research
Social Sciences Arts Humanities
WOS - Emerging Sources Citation Index
Regional publications and emerging topics
© Ruth A. Pageli
24
Geography of Research SciVal
COUNTRIES USA CHINA INDIA RUSSIA IRAN
©SciVal Benchmarking Nov 2015l
25
Manipulation of Data Increase output
Report by system * U H System -10,445 U H Manoa – 4779 U H Hilo - 700
Increase citation count
Increase self-citations Groups who cite each other
© Ruth A. Pageli
26
“Time” and Age Date range of the publications
Subject differences in “half life “ of citations
17 of 19 “top 10 world” universities founded before 1900
Longer time span, better for larger, older world institutions
Does University Age Count? http://meta.umultirank.org/fileadmin/downloads/Newsletter/2015/Nove mber/Readymade_Rankings_by_University_Age_Story.pdf
Ruth A. Pagell
27
Deconstructing Rankings –NTU-TW
NUS
NTU
World Rank 42 11 yrs Articles 67 Current Articles 74.7 11 years Cites 63.6 Current Cites 71.9 h Index 66.4 Highly cited 64.8 Ref Rank 70 normalized by # of faculty
107 61.2 63.6 53.8 65.2 67.1 59.1 126
© Ruth A. Pageli
28
h- index - number of cites that match the nth article Author A Author B
h index 12 h index 1
Author C h index 3 Singapore h index 111
Web of Science Core Collection Singapore subject engineering
34 articles 792 total cites 2 articles 19 total cites from 1 article 9 articles 27 total cites 471 articles 4706 cites
For current 11 years
© Ruth A. Pageli
29
Highlights and Low Lights of Rankings RANKER ARWU – 500 NTU - 500
Strengths OR Weaknesses Uses Nobel prize winners; 100 individual ranks 80% size dependent; re-rank by individual metric ; >495 ranked THE – 800 33% reputation; 200 individual ranks QS - 700 40% reputation; 400 ranked U.SNews - 750 25% reputation; displays only rank & score; all ranked; 45% size independent Leiden - 750 All ranked for each indicator; size independent & dependent SIR – 5054 Requires permission; 2838 higher ed; unusual categories and results © Ruth A. Pageli
30
What you can do to add value CHECK THE RANKINGS’ METHODOLOGY KNOW YOUR INSTITUTION’S GOALS AND OBJECTIVES
GET INVOLVED Ruth A. Pagell
31
There is “A role for the library in awakening to the power and potential of institutional metrics for research” “Where should the expertise to mine, collect, analyze and interpret the data reside?” That may be the library. Also, the library may be a neutral and central resource among the various data generators, including the Provost’s Office, Institutional h Office (where pre- and post-award information may be in separate subdivisions), the Foundation, the Technology Transfer Office, the International Affairs Office, and the list goes on. (Prema Arasu, CEO and Vice Provost K-State Olathe) http://www.snowballmetrics.com/wp-content/uploads/LCN_Arasu_Dec-8-2014.pdf
© Ruth A. Pageli
32
MAHALO
Contact:
[email protected] SEE: Ruth’s Rankings articles in Access http://librarylearningspace.com/ruths-rankings-1/ Chapter 34 (with same title) in Emerging Trends and Issues in Scientometrics, Informetrics and Webometrics http://anebooks.com/bookdetails.asp?id=1989 Ruth A. Pagell
33
