University Research Rankings' Landscape

0 downloads 0 Views 702KB Size Report
May 14, 2015 - http://www.ireg-observatory.org/en/index.php/initiatives .... h-index – for institutions by discipline .... http://anebooks.com/bookdetails.asp?id=1989.
University Research Rankings’ Landscape Through the eyes of a library professional Based on Ruth’s Ranking in e-Access

For Library Association of Singapore

Ruth A Pagell, Singapore November 2015

1

HKUST surges to no. 28 worldwide and no. 1 in Hong Kong in QS World University Rankings 2015-20 16 SMU LEE KONG CHIAN D \SCHOOL OF BUSINESS DEBUTS IN FT EMBA RANKING 2015

May 14, 2015 QS World University Ranking by Subject Peking U ranks top among Mainland China universities [in].. Philosophy, Modern Languages, History, Linguistics, English Language & Literature, Psychology http://www.ntu.edu.sg/Pages/home.aspx https://www.facebook.com/hkust/posts/10153500031624473 http://english.pku.edu.cn/news_events/news/global/3640.htm http://www.smu.edu.sg/

Drivers for Bibliometrics Research Citations (SCI – 1963)

Faculty comparisons

Institutional benchmarking

National Policy Initiatives (Malaysia, 2015) © Ruth A. Pageli

3

The Players Authors Publishers Institutions Aggregators of articles - sources

© Ruth A. Pageli

4

Sources for Bibliometrics COMMERCIAL Aggregatprs

THOMSON REUTERS Web of Science (WOS) SSCI, SCI-e, A & H INCITES; Journal Citation Reports (JCR); ESI ELSEVIER Scopus SciVal PURE

SPECIALIZED SITES Google Scholar Websites Social Media Repositories Research Gate Mendeley Specialized scholar cites

©Ruth A. Pagell

5

The Players Authors Publishers Aggregators of articles

Institutions

Bibliometricians/Scientometricians Rankers Users(Stakeholders)

© Ruth A. Pageli

6

IREG Stakeholders – Rankings Users

http://www.ireg-observatory.org/en/index.php/initiatives International Ranking Expert Group

The Players Authors Publishers Aggregators of articles

Institutions

Bibliometricians/Scientometricians Rankers Users(Stakeholders)

Information Professionals © Ruth A. Pageli

8

Topics Behind the Rankings Evolution of Rankings – History (RR 1) International and national Policies (RR2) The metrics What we count; How we count (RR3, 13) Providers, Aggregators (RR4, 10, 14, 15) Rankers (5-9, 1,12,15) How we interpret From Ruth’s Rankings in Access http://librarylearningspace.com/ruthsrankings-1/ © Ruth A. Pagell

9

What is the correct ranking? The most relevant? 161 151-200 186 201-250 207 314 327 330

• • • • • • • •

U.S. News U.S. -200 ARWU - 500 NTU - 500 THE - 800 Leiden - 750 U.S. News - 750 QS - 700 SIR - 2838 © Ruth A. Pageli

10

RANKING SINGAPORE TOTAL SG

TYPE NUS

NTU

SMU

ARWU

500

2

Schl

101-150 151-200

NTU

500

2

Schl

42

107

QS (1)

700

3

Multi

12

13

THE

800

2

Multi

26

55

U.S. NEWS

750

3

Multi

49

74

LEIDEN (2)

750

2

Schl

144/29

85/55

SIR (3)

2839

5 /25

Multi

436

350

230

46

Multi

89

103

1095

COMPOSITE

Subject

Subject

BY METRIC

RANKING WEB > 23,800

1:Registration;;2. Default - Size Independent top 10% /Size dependent 3. Permission 11 required; for research; ranking impact; with non-unis; Ranking Web includes Polys

Topics for Discussion Basic Variables

Factors impacting rankings

Institutions

Authors

1. 2. 3. 4. 5.

Metrics

6.

Publications

Size Discipline Publication type Data base coverage-geography Manipulation of data Of rankings interfaces Time (and Age)

SciVal Metrics Guidebook 2014

http://www.elsevier.com/__data/assets/pdf_file/0020/53327/sci val-metrics-guidebook-v1_01-february2014.pdf ©Ruth A. Pagell

12

VARIABLES FOR Basic Units • Institutions –types Universities, research institutes, government

Systems or individual units; number of publications; Peer review

• Publication- types Articles, reviews, books, proceedings

Total output Output per faculty Articles in top n%

• Authors Matching authors to publications, institutions & countries

Disambiguation of authors’ names Assigning institutional credit for multiple authors © Ruth A. Pageli

13

Assigning Credit This is a sample article. Who gets credit? BY Kumar [4]; Page [2]; Wang[3]; Tan [1]

:

COUNTING: Authors and Institutions: WOS 4 – one for each; Leiden ¼ (fractionalized) Reprint Address L Tan Singapore U. School of …… Only Kumar; only Tan Addresses: [1] Singapore University [2] U.S. University 1 [3] U.S. University 2 [4] Indian University

Countries WOS – 3 Leiden= 1/3 Ruth A. Pagell

14

Not-so Basic Metrics – Citations SNOWBALL METRICs Citation count for fixed • Outputs in top time period percentiles – 1%; Citations per FTE 10%... Citations per output Per FTE h-index – for institutions Per publication by discipline • Outputs in top Field weighted citation journals impact http://www. ballmetrics.com/wp-content/uploads/snowball-recipe-book_HR.pdf ©Ruth A. Pagell

15

Determining Rankings Composite score with weighted metrics

Rank only Rank and score Rank, score and raw data (Leiden) Re-rank by metrics

No composite score

Rank by individual metric Recommended metric, i.e. field weighted citation © Ruth A. Pageli

16

Rank and Scoring Distance ARWU 2015

http://www.shanghairanking.com/ARWU2015.html © Ruth A. Pageli

17

SAME RANK- SAME SCORE World Rank 29 Social Sciences THE from 2014-2015 University

Teach InterResearch Citanational tions ing Outlook

Industry SCOR Income E

NUS

67.5

83.6

70.7

53.1

34.9

65.3

U HK

55.7

87.9

70.6

61.1

92.8

65.3

7.5% 3

30% 3*

30%

2.5%

Weight & # 30% of Metrics 5* * Includes reputation

© Ruth A. Pageli

18

Can you modify the rankings? Filter rankings by: Geography – Asia-Pacific, Asia, Middle East

Subject / field – 5 broad fields to >200 individual disciplines Time – change the time period © Ruth A. Pageli

19

Size Matters –Leiden SIZE DEPENDENT NTU -80%

SIZE INDEPENTENT U.S. News 45%

©Ruth A. Pagell from Leiden 2015

20

Field matters

Field

PUBLICATIOS

CITATIONS

% Cited

Neurosciences

222,000

1.877,000

69%

History

122625

31,164

9%

© Ruth A. Pageli

21

Broad Subject Area Matters Citations from Scopus Life Science and Medicine

49.1%

Natural Sciences

27.6%

Engineering & Technology

16.5%

Social Sciences

5.8%

Arts & Humanities

1%

Sowter, Ben (July 27, 2015) Intelligence Unit Blog http://www.iu.qs.com/2015/07/potentialrefinements-in-the-qs-world-university-rankings-2015/ Ruth A. Pagell

22

Publication Type and Citations

A. Pageli From SciVal Metrics Guidebook© Ruth Table 3.2.3

23

Geography /Language Database journal coverage

US and UK English

Language

Localized research

Social Sciences Arts Humanities

WOS - Emerging Sources Citation Index

Regional publications and emerging topics

© Ruth A. Pageli

24

Geography of Research SciVal

COUNTRIES USA CHINA INDIA RUSSIA IRAN

©SciVal Benchmarking Nov 2015l

25

Manipulation of Data Increase output

Report by system * U H System -10,445 U H Manoa – 4779 U H Hilo - 700

Increase citation count

Increase self-citations Groups who cite each other

© Ruth A. Pageli

26

“Time” and Age Date range of the publications

Subject differences in “half life “ of citations

17 of 19 “top 10 world” universities founded before 1900

Longer time span, better for larger, older world institutions

Does University Age Count? http://meta.umultirank.org/fileadmin/downloads/Newsletter/2015/Nove mber/Readymade_Rankings_by_University_Age_Story.pdf

Ruth A. Pagell

27

Deconstructing Rankings –NTU-TW

NUS

NTU

World Rank 42 11 yrs Articles 67 Current Articles 74.7 11 years Cites 63.6 Current Cites 71.9 h Index 66.4 Highly cited 64.8 Ref Rank 70 normalized by # of faculty

107 61.2 63.6 53.8 65.2 67.1 59.1 126

© Ruth A. Pageli

28

h- index - number of cites that match the nth article Author A Author B

h index 12 h index 1

Author C h index 3 Singapore h index 111

Web of Science Core Collection Singapore subject engineering

34 articles 792 total cites 2 articles 19 total cites from 1 article 9 articles 27 total cites 471 articles 4706 cites

For current 11 years

© Ruth A. Pageli

29

Highlights and Low Lights of Rankings RANKER ARWU – 500 NTU - 500

Strengths OR Weaknesses Uses Nobel prize winners; 100 individual ranks 80% size dependent; re-rank by individual metric ; >495 ranked THE – 800 33% reputation; 200 individual ranks QS - 700 40% reputation; 400 ranked U.SNews - 750 25% reputation; displays only rank & score; all ranked; 45% size independent Leiden - 750 All ranked for each indicator; size independent & dependent SIR – 5054 Requires permission; 2838 higher ed; unusual categories and results © Ruth A. Pageli

30

What you can do to add value CHECK THE RANKINGS’ METHODOLOGY KNOW YOUR INSTITUTION’S GOALS AND OBJECTIVES

GET INVOLVED Ruth A. Pagell

31

There is “A role for the library in awakening to the power and potential of institutional metrics for research” “Where should the expertise to mine, collect, analyze and interpret the data reside?” That may be the library. Also, the library may be a neutral and central resource among the various data generators, including the Provost’s Office, Institutional h Office (where pre- and post-award information may be in separate subdivisions), the Foundation, the Technology Transfer Office, the International Affairs Office, and the list goes on. (Prema Arasu, CEO and Vice Provost K-State Olathe) http://www.snowballmetrics.com/wp-content/uploads/LCN_Arasu_Dec-8-2014.pdf

© Ruth A. Pageli

32

MAHALO

Contact: [email protected] SEE: Ruth’s Rankings articles in Access http://librarylearningspace.com/ruths-rankings-1/ Chapter 34 (with same title) in Emerging Trends and Issues in Scientometrics, Informetrics and Webometrics http://anebooks.com/bookdetails.asp?id=1989 Ruth A. Pagell

33