Tracking the development of disciplinary structure in China's top ...

Research Evaluation 24 (2015) pp. 312–324 Advance Access published on 10 June 2015

doi:10.1093/reseval/rvv013

Tracking the development of disciplinary structure in China’s top research universities (1998–2013) Feng Li1,*, Yajun Miao2 and Jing Ding3 1

School of Public Administration, Hohai University, No.8 Fochengxi Road, Nanjing, Jiangsu, China, 211199 2 Business School, Hohai University, No.8 Fochengxi Road, Nanjing, Jiangsu, China, 211199 3 School of Management, Hefei University of Technology, No.193 Tunxi Road, Hefei, Anhui, China, 230009 *Corresponding author. Email: [email protected]

This study focuses on the development of disciplinary structure in China’s top research universities. Using the science structures of Mainland China, the world, and the world’s top 10 universities as three reference standards, we measure the disciplinary structure ‘distance’ between the Chinese ‘985 universities’ and the references, and analyze whether the 985 universities are moving toward or away from the reference groups (RG) in terms of disciplinary structure. The results show that most of the 985 universities’ disciplinary structures are very similar to China’s science structure. The disciplinary structure distances between the 985 universities and both the world reference and the world’s top universities reference have narrowed recently, but the overall degrees of similarity are still low. Another important finding is that the rise of biomedical science has played an important role in the 985 universities’ move from a native ‘sci-engineering focused’ structure to an international ‘bio-science focused’ structure. Keywords: disciplinary structure; similarity; research university; China.

1. Introduction Initiatives for creating world-class universities in China were officially started in 1998, when the government launched the 985 Project (Zhang et al. 2013). Thirty-nine of China’s top research universities have been selected for the Project, and according to China’s Minister of Education (MOE), the Project has now closed the door on new universities (Yuan and Zhang 2011). Although national policy has shifted several times during the Project’s history, cultivating world-class disciplines has always been one of the fundamental tasks of the policy. During the first phase of the 985 Project (1998–2001), the strategy was to pool limited national financial resources and concentrate on building key disciplines, especially those disciplines that were close to achieving and qualifying for a world-class level (MOE 1998). State funding was distributed directly to sponsored universities without any conditions and restrictions on the use of the money (Cheng

2011). Therefore, universities had a high level of autonomy in deciding which discipline(s) to support. The second phase of the Project (2004–7) was adjusted to focus on the building of innovative platforms for science and technology, as well as the humanities and social sciences (MOE 2004; Zhou 2004). Unlike in the first phase, the funding in the second phase had a direct purpose, and most of the money was used to build innovative platforms (Cheng 2011). The third phase of the 985 Project (2010–3), also called the new 985 Project, was implemented on the basis of continuously strengthening key disciplines and innovative platforms built up during the last two phases (MOE 2010a, b). A more competitive funding model was adopted during the third phase, and universities were encouraged to carry out S&T management reforms. As the government has tried to allocate funding based on the research performance of universities, 985 Project-sponsored universities (hereafter referred to as 985 universities) have become extremely concerned about particular performance

ß The Author 2015. Published by Oxford University Press. All rights reserved. For Permissions, please email: [email protected]

Tracking the development of disciplinary structure indicators, such as productivity and the impact of research publications in internationally peer-reviewed journals. Some scholars have used scientometric indicators to evaluate and analyze the research performance of the 985 universities (Zhu et al. 2004; Li et al. 2012; Zhang et al. 2013), while the authors of this article are particularly interested in the development of disciplines or disciplinary structures in the 985 universities. Institutional disciplinary structure has not been as comprehensively studied as national disciplinary structure. In scientometric literature, three measures of national disciplinary structure have mainly been studied, namely, ‘variety’, ‘evenness’, and ‘disparity’ (Zhou et al. 2012). ‘Variety’ measures the number of categories into which different national disciplinary structures can be divided. Cluster analysis is commonly used to measure the variety of national disciplinary structures (Yang et al. 2012; Moya-Anego´n and Herrero-Solana 2013; Harzing and Giroud 2014). ‘Evenness’ measures the concentration or dispersion of a country’s disciplinary structure. If a country has a dispersed distribution of research outputs among disciplines, this country is described as having a balanced disciplinary structure. The evenness of national disciplinary structure is usually measured by Gini coefficient (Yang et al. 2012). ‘Disparity’ measures the similarity or dissimilarity of disciplinary structures of different countries. The core aim of measuring disparity is to measure the ‘distance’ between national disciplinary structures. Various methods can be used to measure this distance, such as the Activity Index (AI; Frame 1977; Schubert and Braun 1986), the further developed Relative Specialization Index (RSI; Gla¨nzel 2000; Aksnes et al. 2014) and the Cosine Index (Kozlowski et al. 1999; Zhou et al. 2012). Most of these indicators are relative indicators in the application of reference standards (Vinkler 2010). Besides this, by combining cluster analysis with multidimensional scaling, or correspondence factor analysis, some studies have made the relative distances and locations of national disciplinary structures more visual (Dore´ and Ojasoo 2001; Zhou et al. 2012). As with the above, most studies of institutional disciplinary structure also focus on ‘variety’, ‘evenness’, or ‘disparity’. Thijs and Gla¨nzel (2008) have used cluster analysis to classify universities in Europe based on their disciplinary structures, and then applied the AI to measure the research profile distance between university categories and European countries. Some other scholars have used Gini coefficients to investigate the degree of concentration of research performance among universities in Italy (Abramo et al. 2012), Spain (Lo´pez-Illescas et al. 2011), and the Netherlands (Moed et al. 1999). These studies mainly deal with universities in Europe. Little work has been done on the disciplinary structures of the 985 universities. In a study by Cheng and Liu (2008), the disciplinary structures of the original nine 985 universities (known as the ‘C9 League’) was studied. Using publication

.

313

data ranging from 1997 to 2005, the authors examined the balance of publication distribution among major subject fields and found that most universities from the C9 League were becoming more balanced in disciplinary structure. Cheng and Liu’s (2008) study only revealed a fraction of how the 985 universities had evolved their disciplinary structures. Given the preferred policy of developing world-class disciplines and universities, there are several questions requiring answers: How do the 985 Project policies affect the development of the 985 universities’ disciplinary structures? Are institutional structures becoming more domestically oriented in response to national science structure, or more internationally oriented in response to the world’s science structure? Or more directly, are institutional structures becoming more similar or dissimilar to those of world-class universities? Focusing on the questions above, we use institutional publication profiles to measure the disciplinary structures of the 985 universities, and examine the development of their disciplinary structures for the period 1998–2013. By measuring the disciplinary structure distance between each 985 university and (1) the national reference, (2) the world reference, and (3) the world’s top universities reference, we aim to find out how the 985 universities evolve their disciplines in response to national and international science structure.

2. Methods 2.1 University samples and data resource For this article, we selected thirty-seven 985 universities (see Table 1) and removed Minzu University of China and Renmin University of China because their low volume of scientific publications did not match up with other universities. The publication data for each university between 1998 and 2013 was extracted from the InCites Database (InCitesTM 2012). All the data were organized into 22 fields (see Table 2), as defined by Thomson Reuters’ Essential Science Indicators (ESI). InCites is a commonly used evaluation tool which allows the assessment and benchmarking of the research performance of individual researchers, institutions, and countries (Bornmann and Leydesdorff 2013b). It should be noted, however, that the InCites Database is based on Thomson Reuters’ Web of Science (WoS), in which the coverage of research publication for some disciplines may be underestimated (Harzing and Giroud 2014; Tang 2013). Two disciplines that may disturb our study are engineering and social sciences. According to Moed’s (2005) research, engineering conference proceedings and social science publications are not well covered by WoS, and both of these are important sources of research output for these two disciplines. Moreover, Chinese scholars often publish academic papers and books in Chinese, which are also not well covered by WoS (Nederhof 2006; Tang and Hu 2013). As the Chinese government encourages research

314

.

F. Li et al.

Table 1.

List of the selected 985 universities and their abbreviations

Institution

Abbreviation

Institution

Abbreviation

BEIHANG UNIV BEIJING INST TECHNOL BEIJING NORMAL UNIV CENT S UNIV CHINA AGR UNIV CHONGQING UNIV DALIAN UNIV TECHNOL E CHINA NORMAL UNIV FUDAN UNIV HARBIN INST TECHNOL HUAZHONG UNIV SCI & TECHNOL HUNAN UNIV JILIN UNIV LANZHOU UNIV NANJING UNIV NANKAI UNIV NATL UNIV DEF TECHNOL NORTHEASTERN UNIV-CHINA NORTHWESTERN POLYTECH UNIV

BUAA BIT BNU CSU CAU CQU DLUT ECNU FUDAN HIT HUST HNU JLU LZU NJU NANKAI NUDT NEU NWPU

NW A&F UNIV OCEAN UNIV CHINA PEKING UNIV S CHINA UNIV TECHNOL SHANDONG UNIV SHANGHAI JIAO TONG UNIV SICHUAN UNIV SOUTHEAST UNIV SUN YAT SEN UNIV TIANJIN UNIV TONGJI UNIV TSING HUA UNIV UNIV ELECT SCI & TECHNOL CHINA UNIV SCI & TECHNOL CHINA WUHAN UNIV XIAMEN UNIV XIAN JIAOTONG UNIV ZHEJIANG UNIV

NWSUAF OUC PKU SCUT SDU SJTU SCU SEU SYSU TJU TONGJI TSINGHUA UESTC USTC WHU XMU XJTU ZJU

Table 2.

List of ESI fields and their abbreviations

ESI fields

Abbreviation

ESI fields

Abbreviation

ESI fields

Abbreviation

Agricultural Sciences Biology & Biochemistry Chemistry Clinical Medicine Computer Science Economics & Business Engineering Environment/Ecology

AGR BIO CHE CLI COM ECO ENG ENV

Geosciences Immunology Materials Science Mathematics Microbiology Molecular Biology & Genetics

GEO IMM MAS MAT MIC MOL

Neuroscience & Behavior Pharmacology & Toxicology

NEU PHA

Multidisciplinary

MUL

Physics Plant & Animal Science Psychiatry/Psychology Social Sciences, general Space Science

PHY PLA PSY SOC SPA

universities to publish in internationally peer-reviewed journals, the quality and quantity of international publications covered by WoS (especially its Science Citation Index) have become important measures of the 985 universities’ research performance. As a result, the disciplinary structure of Chinese universities’ international publications retrieved from the InCites Database echoes, to a large extent, the disciplinary structure of the universities’ overall publications. Also, the InCites data has proved reliable in research evaluation in China at both a national and disciplinary level (Bornmann and Leydesdorff 2013a; Ding et al. 2013). Another point of note is that the institutional data covered by InCites is consistent over time despite the mergers that have occurred among some of the 985 universities (Xiong et al. 2011). For example, Shanghai Jiao Tong University (SJTU) merged with Shanghai Second Medical University (SSMU) in 2005, and the latter became the School of Medicine at Shanghai Jiao Tong University (SJTUSM). In the InCites Database, not only the post-2005 publication data from SJTUSM

(formerly SSMU) has been included as part of the SJTU publications, but the publication data from SSMU before 2005 has been included as well. The same is true for the 21 other 985 universities with mergers after 1998. 2.2 Measures of distance between institutional disciplinary structures In order to investigate the disciplinary structure distances between the 985 universities and the RG at institutional levels as well as the disciplinary level, we combined Disciplinary Balancing Index (DBI) and RSI in our study. Using the squared Euclidean distance metric, DBI measures the overall distance between each of the 985 universities and a selected reference group. The fomulation of DBI is presented as follows (Cheng and Liu 2008): X 2 Pij  pj ð j : 22 ESI FieldsÞ DBIi ¼ j

Where DBIi denotes the DBI value for unversity i, Pij denotes the share of scientific publications produced by

Tracking the development of disciplinary structure

.

315

Degree of similarity according to hierarchy of DBI value

Table 3. DBI value

(0, 0.05)

(0.05, 0.10)

(0.10, 0.15)

(0.15, 0.20)

(0.20, 1)

Degree of similarity (abbreviation)

Very high (VH)

High (H)

Normal (N)

Low (L)

Very low (VL)

university i’s discipline j in university i’s total publications, and pj denotes the share of publications produced by a reference group’s discipline j in the total publications produced by this reference group. Furthermore, the degree of similarity is defined as based on the DBI value. As shown in Table 3, we use 0.05, 0.10, 0.15, and 0.20 as thresholds to classify the degree of similarity into five ordered levels. We use RSI to measure the relative individual discipline distance between each 985 university and a selected reference group. RSI was, according to Gla¨nzel (2000), first proposed by the European Commission (1997). In order to get RSI, the AI at the disciplinary level is defined as follows: the share of the given discipline in the publications of the observed university AI ¼ the share of the given discipline in the total publication of the reference group Then, RSI is calculated as follows: RSI ¼

AI  1 AI+1

The value of RSI is limited to the range [1, 1]. If RSI = 0, the given discipline of the observed university is benchmarked against the reference group. If RSI > 0, the given discipline of the observed university is relatively more active in research than the reference group. If RSI < 0, the given discipline of the observed university is less active than the reference group. 2.3 Reference groups Unlike the traditional calculation of RSI (or AI) and DBI at the national level, we introduce three different RG instead of using the world average as the only reference. The description of selected references and the reasons why we choose them are presented as follows: RG1: The total research publications produced by mainland Chinese researchers. The disciplinary structure of RG1 represents the average structure of China’s science system. As mentioned by Zhang et al. (2013), Chinese research universities play a very important role in the national innovation system.

The majority of the 985 universities’ research funding is from the government, and as a result, the research activities carried out by these universities are most likely to be influenced by national science policy, which is the main driver of the development of national science structure. Measuring the disciplinary structure distances between the 985 universities and RG1 gives a possible method of examining how Chinese research universities are influenced by national science policy. RG2: The world’s total research publications. The disciplinary structure of RG2 represents the average structure of the world’s science system. RG2 is the original reference used in the calculation of AI and RSI (Gla¨nzel 2000; Aksnes et al. 2014). Based on the publication data from InCites, the institutional disciplinary structure we measure in this article is actually the structure of a university’s publications in internationally peer-reviewed journals. Chinese universities are facing competition worldwide to publish internationally. RG2 provides a baseline for international publications across disciplines. RG3: The total research publications of the world’s top 10 universities. The disciplinary structure of RG3 represents the average structure of elite unversities. Since the fundamental mission of the 985 policy is to create world-class universities, it is important to see whether the disciplinary structures of the 985 universities are moving toward or away from those of world-class universities. We selected the top 10 universities from the 2013 to 2014 editions of the three most popular world unversity rankings, as produced by Shanghai Jiao Tong University (http://www.shanghairanking.com/ ARWU2013.html), Times Higher Education (http://www. timeshighereducation.co.uk/world-university-rankings/201314/world-ranking), and Quacquarelli Symonds Limited (http://www.topuniversities.com/university-rankings/worlduniversity-rankings/2013). A total number of 13 unversities were chosen, and they are all in at least one of the top 10 lists. These unversities are Harvard University, Stanford University, the University of California at Berkeley, MIT, the University of Cambridge, the California Institute of Technology, Princeton University, Columbia University, the University of Chicago, the University of Oxford, Imperial College London, University College London, and Yale University.

316

.

F. Li et al.

Figure 1. in 1998.

The Euclidean distance between one group’s disciplinary structure in each year and the reference group’s original structure

Figure 2. 2009–3.

Share of each discipline’s publications in total publications produced by different groups over the periods 1998–2002 and

3. Analysis and results 3.1 Disciplinary structures of the RG and the 985 universities group We measured the distance between each reference group’s disciplinary structure in 1 year and the same reference’s original structure in 1998, using the Euclidean distance metric. The result in Figure 1 shows that both RG2 and RG3 have more stable structures than RG1. RG1’s disciplinary structure is changing from its 1998 structure over time (the Euclidean distance for RG1 rises rapidly, especially after 2007), while the disciplinary structures of RG2 and RG3 change slowly from their 1998 structures. Since the total research publications of the 985 universities account for about half of China’s total publications, the development of the 985 universities’ average structure mirrors that of China’s average structure. The radar charts in Figure 2 show further information about how each ESI field’s publication share changes in the RG and the group of 985 universities in two different

time periods (1998–2002 and 2009–13). Only a few fields show significant fluctuations in publication shares. Both the 985 universities group and RG1 have considerable decreases in the percentages of scientific papers produced by CHE and PHY (see the full names of the disciplines in Table 2, and likewise for all disciplines hereafter), and increases in those from CLI and ENG. RG2 has slight reductions in the publication shares of some fields such as PHY and BIO, while the fields with minor increases in publication shares are SOC, MAS, and so on. RG3 has similar changes in the publication shares of PHY, BIO, and SOC to those of RG2. Overall, RG2 and RG3 have stable disciplinary structure patterns over time, while the average disciplinary structure patterns of RG1 and the 985 universities change significantly. In addition to the above, the observed groups have different combinations of high yielding disciplines. Figure 2 shows that the predominant discipline of RG1 and the 985 universities group is CHE, accounting for about one fifth of the total publication volume in the period 2009–13,

Tracking the development of disciplinary structure Table 4.

317

Distribution of the 985 universities in five different levels of similarity in three time periods, 1998–2001, 2004–7, and 2010–3

Similarity degree

VH H N L VL

.

RG1

RG2

RG3

1998–2001

2004–7

2010–3

1998–2001

2004–7

2010–3

1998–2001

2004–7

2010–3

18 5 6 3 5

23 6 6 2 0

26 5 6 0 0

0 7 10 11 9

7 12 7 8 3

11 11 11 4 0

0 2 9 8 18

3 5 11 7 11

7 8 8 11 3

while the predominant discipline of RG2 and RG3 is CLI, taking a share of roughly one fifth of the overall publications. The other high-yielding disciplines of RG1 and the 985 universities group include PHY, MAS, and ENG, all accounting for more than 10% of total publications. Both RG2 and RG3 have only one follow-up discipline with a similar percentage of publications in the period 2009–13 (CHE for RG2 and PHY for RG3). Overall, the research structures of both China and the 985 universities group are ‘sci-engineering focused’, and are more concentrated on a few disciplines. Somewhat differently, both the world and the world’s top universities average structures are ‘bioscience focused’, and are more dispersed in terms of other disciplines. It is important to note that China is reducing the relative production of CHE and PHY and increasing that of CLI, which shows that China’s science structure is moving toward that of the world structure. 3.2 DBI values and institutional similarity degrees Based on the DBI values of the 985 universities with respect to each reference group’s average disciplinary structure, universities are coded with different levels of similarity (see Appendix A), and the distribution of the 985 universities in each category of similarity degree is presented in Table 4. The results are explained as follows: (1) The disciplinary structures of most of the 985 universities have a very high degree of similarity to China’s average structure, and the number of universities in the VH group climbs from 18 during the period 1998–2001 to 26 during the period 2010–3. Universities that have a disciplinary structure with low or very low degrees of similarity to China’s average structure have disappeared in recent time periods. Besides this, the number of universities in groups H and N stays stable over time. (2) The number of universities that have a disciplinary structure with a very high degree of similarity to the world reference increases from 0 to 11 over the periods, while that with a very low degree of similarity decreases from 9 to 0. Unlike the results using RG1 as the reference, there are more universities in groups H and N than in group VH most of the time.

In addition, the number of universities in groups H and N fluctuates during some periods, as a few universities (e.g. NWSUAF and UESTC) appear to have unstable disciplinary structure development. (3) Only a few of the 985 universities have a disciplinary structure with a very high degree of similarity to the world’s top universities. There are no universities in group VH during the period 1998–2001, while the number increases slowly to seven during the period 2010–3. These seven universities are SYSU, FUDAN, PKU, HUST, SJTU, WHU, and SDU. A detailed analysis of these universities will be presented in the following section. The number of universities in group VL decreases greatly from 18 to just 3 universities. As with the RG2 scenario, the trends for the number of universities in groups N and L are highly erratic, and universities that have disciplinary structures with a low degree of similarity to the world’s top universities reference appear to be the largest group in recent periods. (4) Overall, the number of 985 universities with disciplinary structures showing a very high degree of similarity to each of the three RG increases. Moreover, most universities are moving toward the average disciplinary structure of each reference group. Only the distribution of universities in the five levels of similarity degree are different with respect to different references. It appears that most universities are in group VH when using RG1 as a reference, and in groups H and N when using RG2 as a reference; while for RG3, most universities are in groups L and N. In other words, the average degrees of similarity between the 985 universities and the RG follow the rule: RG1 > RG2 > RG3. Furthermore, more universities are grouped into the same level of similarity during the three periods when using RG1 as the reference than when using RG2 and RG3 as the reference (18 for RG1 compared to 2 for RG2 and 5 for RG3), which indicates that the disciplinary structure distances between the 985 universities and China’s average are more stable than those between the 985 universities and the world average, and also the world’s top universities average.

318

.

F. Li et al.

Table 5.

Distribution of non-yielding disciplines over the periods

Time period

IMM

PSY

SPA

ECO

MIC

NEU

AGR

PLA

SOC

MUL

PHA

1998–2001 2004–7 2010–3 Time period 1998–2001 2004–7 2010–3

22 8 1 MOL 7 1 0

22 13 3 CLI 6 0 0

18 2 3 BIO 4 0 0

16 5 1 ENV 3 0 0

15 4 1 GEO 3 0 0

12 4 0 COM 2 0 0

11 2 1 MAS 1 0 0

9 2 0 MAT 1 1 0

9 2 0 CHE 0 0 0

8 4 0 ENG 0 0 0

8 1 0 PHY 0 0 0

Table 6.

Distribution of disciplines over the periods (using RG1 as the reference)

Discipline

AGR BIO CHE CLI COM ECO ENG ENV GEO IMM MAS MAT MIC MOL MUL NEU PHA PHY PLA PSY SOC SPA

Least active disciplines

Most active disciplines

Most similar disciplines

1998–2001

2004–7

2010–3

1998–2001

2004–7

2010–3

1998–2001

2004–7

2010–3

9 3 2 11 1 7 2 6 16 1 3 1 1 1 3 7 6 5 10 4 7 5

8 1 0 16 1 5 1 3 10 2 5 0 2 4 1 4 4 2 13 9 9 11

18 0 0 14 1 2 0 1 7 8 4 1 3 0 0 5 1 2 17 9 1 17

5 1 12 3 10 0 18 4 3 2 12 7 5 2 1 2 5 9 5 1 0 4

5 1 6 3 12 1 15 5 4 4 14 6 5 4 6 4 3 4 4 1 0 4

4 0 4 7 13 5 16 4 4 5 9 5 4 3 1 5 1 7 4 5 1 4

0 6 8 3 10 2 3 8 4 2 7 18 4 1 5 3 4 15 3 3 1 1

2 6 10 1 7 1 4 4 5 4 6 15 2 2 7 4 6 15 3 1 4 2

2 11 7 1 5 6 1 5 1 6 9 11 4 5 12 2 6 8 2 3 1 3

3.3 RSI values and disciplinary distances Four types of disciplines have been recognized according to their RSI values. They are defined as non-yielding disciplines, the least active disciplines, the most active disciplines, and the most similar disciplines. Specifically, nonyielding disciplines have no publications in the given time period, and their RSI values equal 1. The least active disciplines have the three smallest RSI values among all the disciplines at a given university, and their RSI values are close to but not equal to 1. The most active disciplines have the three largest RSI values in a given university, while the most similar disciplines have the three RSI values closest to 0. The distributions of these four disciplinary categories over the periods are

presented in Tables 5–8, and the results are explained as follows: (1) The number of non-yielding disciplines has decreased dramatically over the periods (see Table 5). During the first phase of the 985 Project, most of the 985 universities had a polarized distribution of research publications among their disciplines. There were a total number of 177 non-yielding disciplines, and each 985 university had nearly five non-yielding disciplines on average. The most frequently reported non-yielding disciplines include IMM, PSY, SPA, ECO, and MIC. About 60% (22 out of 37) of the 985 universities had no publications in IMM or PSY. During the third phase of the 985 Project,

Tracking the development of disciplinary structure Table 7.

319

Distribution of disciplines over the periods (using RG2 as the reference)

Discipline

AGR BIO CHE CLI COM ECO ENG ENV GEO IMM MAS MAT MIC MOL MUL NEU PHA PHY PLA PSY SOC SPA

.

Least active disciplines

Most active disciplines

Most similar disciplines

1998–2001

2004–7

2010–3

1998–2001

2004–7

2010–3

1998–2001

2004–7

2010–3

10 4 1 14 0 8 1 5 6 5 1 1 5 4 0 10 3 2 7 6 15 3

6 1 0 19 0 11 0 0 0 3 0 0 1 4 0 7 2 1 9 21 20 6

11 0 0 13 0 4 0 0 2 8 0 0 2 0 0 4 1 0 12 19 20 15

2 0 19 0 1 0 16 1 3 0 22 23 1 0 9 0 1 9 2 0 0 2

2 0 17 0 13 0 13 1 5 0 22 18 3 0 3 0 1 9 3 0 0 1

2 0 13 1 11 0 19 3 5 0 20 7 2 3 2 1 3 13 3 0 0 3

2 7 14 0 13 0 7 4 4 0 12 3 1 2 16 1 7 14 2 1 0 1

3 9 13 3 12 0 8 10 5 1 7 8 1 1 8 0 5 12 3 1 0 1

3 11 11 2 5 0 3 5 3 1 4 15 4 5 17 5 7 6 2 1 0 1

only 10 disciplines had no research publications, and these disciplines belonged to six universities, namely, HNU, NUDT, NEU, TJU, UESTC, and NWSUAF. (2) Compared to China’s average structure, the 985 universities tended to be least active in the fields of GEO, CLI, and PLA, and most active in the fields of ENG, MAS, CHE, and COM during the first phase of the 985 Project (see Table 6). In the second phase, more of the 985 universities were reported to be least active in CLI, PLA, and SPA, and most active in COM and MAS. There were noticeable decreases in the number of 985 universities which were least active in GEO or most active in CHE and ENG. In the third phase, the most frequently reported least active disciplines were AGR, PLA, SPA, and CLI, while the most active disciplines were mainly distributed in ENG and COM. Besides this, the most similar disciplines with the closest publication shares to China’s average structure changed from the major fields of MAT, PHY, and COM during the period 1998–2001 to the fields of MUL, MAT, and BIO during the period 2010–3. (3) There are some similar trends in the distributions of the least active disciplines and most active disciplines when using the world reference compared to the national reference, such as for AGR, CLI, PLA, and SPA in the group of least active disciplines, and for CHE, COM, ENG, and MAS in the group

of most active disciplines. The 985 universities have different distributions and trends in the fields of ECO, PSY, SOC, MAT, and PHY. More universities have ECO, PSY, and SOC as their least active disciplines, and have CHE, MAS, MAT, and PHY as their most active disciplines over the periods (see Table 7). As for the most similar disciplines, more 985 universities have gotten closer to the world reference in the fields of BIO and MAT, while fewer universities have done so in the fields of COM, MAS, and PHY. (4) Compared to the world’s top universities, more 985 universities appear to have been least active in the fields of IMM, PSY, and SPA, while fewer universities were so in the fields of ECO, MOL, and NEU over the periods (see Table 8). In the period 2010–3, 25 universities were reported to be least active in PSY, which accounted for about 64% of the 985 universities. According to the distribution of the most active disciplines, most 985 universities had relatively higher publication shares in the fields of MAS, CHE, MAT, and ENG in the first phase of the 985 Project, but the number of universities being most active in the fields of CHE and MAT shows a decreasing tendency over the periods. As for the most similar disciplines, a large proportion of 985 universities were reported to be close to the world’s top universities reference in the

.

320

F. Li et al.

Table 8.

Distribution of disciplines over the periods (using RG3 as the reference)

Discipline

AGR BIO CHE CLI COM ECO ENG ENV GEO IMM MAS MAT MIC MOL MUL NEU PHA PHY PLA PSY SOC SPA

Least active disciplines

Most active disciplines

Most similar disciplines

1998–2001

2004–7

2010–3

1998–2001

2004–7

2010–3

1998–2001

2004–7

2010–3

0 5 1 13 0 11 1 3 6 6 0 1 4 11 1 12 1 2 3 10 13 7

0 1 0 16 0 11 0 1 0 8 0 0 0 6 2 9 0 1 0 22 18 16

0 0 0 13 0 4 0 0 3 11 0 0 1 2 1 7 0 0 3 25 17 24

5 0 26 0 1 0 15 0 1 0 30 22 0 0 1 0 2 3 5 0 0 0

11 0 21 0 12 0 15 1 1 0 29 14 1 0 0 0 2 1 3 0 0 0

12 0 19 0 9 0 18 3 1 0 30 6 2 0 0 0 6 1 4 0 0 0

8 3 8 0 18 0 12 9 5 0 1 4 1 0 10 0 6 21 2 0 0 3

8 11 6 1 11 0 12 9 7 0 2 5 2 1 3 0 7 22 4 0 0 0

8 21 6 3 5 1 9 9 7 0 1 8 5 1 0 1 5 17 3 1 0 0

field of PHY over the periods. There is a great increase in the number of universities having BIO as the most similar discipline, while decreases have been reported in the number of universities having COM and ENG as the most similar disciplines over the periods.

Secondly, most disciplines tended to move closer to the world’s top universities reference in publication shares from the period 1998–2001 to the period 2010–3. All seven universities (or most of them) had adjustments in publication shares toward the reference in 14 disciplines, which can be classified into four groups as follows, based on the level of variations in their RSI values over the periods: . Group 1 includes PHY, MAT, PSY, GEO, PLA, SOC,

3.4 Seven universities with the highest degree of similarity to RG3 There are seven universities with the highest degree of similarity to the world’s top universities reference in the period 2010–3. Three of them are C9 league universities. According to the RSI values of 22 ESI fields in these universities as presented in Figure 3, some common characteristics of the development of these universities’ disciplinary structures are summarized as follows: Firstly, the number of disciplines with a very close publication share to the world’s top universities reference increased vastly from the period 1998–2001 to 2010–3. During the period 1998–2001, seven universities had just 10 disciplines with RSI values extremely close to 0 (between 0.1 and 0.1). These disciplines were mainly ENG, ENV, and MUL. During the period 2010–3, the number of disciplines close to the world’s top universities reference increased to 21, and BIO became the discipline closest to the reference at seven universities.

and CHE. The RSI values of these disciplines moved toward ‘0’ at a moderate rate over the periods. Most of the variations in these disciplines’ RSI values are smaller than 0.4. Unlike Group 1, the disciplines included in Groups 2–4 had significant adjustments in publication shares toward the reference over the periods. . Group 2 includes IMM, MIC, and BIO. The variations in RSI values in these disciplines are observable primarily from the period 1998–2001 to the period 2004– 7. . ECO is the only discipline included in Group 3. The variation of its RSI value mainly occurs between the period 2004–7 and the period 2010–3. . Group 4 includes NEU, MOL, and CLI. Universities have developed differently in these disciplines. Some variations of the RSI values in these disciplines happen mainly between the period 1998–2001 and 2004–7, such as with PKU’s NEU and MOL, SYSU’s NEU and MOL, SDU’s CLI, WHU’s CLI

Figure 3. The RSI values of seven selected 985 universities in 22 ESI fields in three periods, 1998–2001, 2004–7 and 2010–3. Note: The RSI values are calculated using RG3 as a reference.

Tracking the development of disciplinary structure .

321

322

.

F. Li et al.

as well as FUDAN’s MOL; while for others between the period 2004–7 and the period 2010–3, such as with SDU’s NEU and MOL, SJTU’s CLI, NEU, and MOL, WHU’s NEU and MOL, HUST’s NEU as well as FUDAN’s CLI. Thirdly, most universities have rising positive RSI values in the fields of PHA and ENG, and decreasing negative RSI values in MUL. In other words, most universities are moving away from the top universities reference in the fields of PHA, ENG, and MUL. While in other fields, such as ENV, COM, AGR, MAS, and SPA, universities have different trends in RSI values.

4. Discussion and conclusions We have tracked the development of the disciplinary structures of the 985 universities by measuring their distances from the average structure of China’s total publications, worldwide publications, and also the world’s top 10 universities’ publications. The results show that the majority of the 985 universities have very similar disciplinary structures to China’s research structure. Although there are signs of the gap narrowing between the 985 universities and the world reference, most universities’ disciplinary structures are dissimilar to the world or the world’s top universities reference. Within the 985 Project, all of the 985 universities have been required to create a world-class university or a worldrenowned university with some priority fields achieving a world-class status. The development of the 985 universities’ disciplinary structures is strongly affected by national science policy. In our study, most of the 986 universities are reported to share the same most active disciplines, such as CHE, ENG, COM, MAS, and MAT (see Tables 6–8), which is consistent with the ‘sci-engineering focused’ national science structure. Over the periods, some 985 universities have reduced their publication shares in some hard science fields (such as CHE, MAT), while more universities have become most active in the engineering sciences (such as ENG, COM, and MAT). The concentration of the 985 universities’ science publications in the engineering sciences is a reflection of China’s need to consolidate defense and industry (Kostoff et al. 2007). The rise of biomedical sciences in the 985 universities has been astonishing, which is a good sign that some Chinese research universities are moving from a native ‘sci-engineering focused’ structure to an international ‘bio-science focused’ structure. More universities are reported to be very close to the world and world’s top universities references in the fields of BIO, MIC, MOL, NEU, and CLI (see Tables 7 and 8). BIO is the discipline with the fastest development of all. The number of 985 universities having BIO as their most similar discipline to the world’s top universities reference has increased from 3 in the period 1998–2001 to 21 in the period 2010–3. The

merging phenomenon of the 985 universities and medical colleges (or universities) around the year 2000 has helped a lot in increasing the publication shares of biomedical sciences among the 985 universities. Thirteen 985 universities merged with at least one medical college or university after 1998. However, it is interesting to see that CLI is not growing as fast as BIO. Moreover, CLI and IMM have remained the least active disciplines over the periods compared to the world reference. In the analysis of the seven universities with the most similar disciplinary structures to the world’s top universities reference, we found that about two thirds of the observed disciplines are moving toward the top universities reference, and the disciplines with significant adjustments toward the reference’s structure are mainly in the field of biomedical sciences (such as IMM, MIC, BIO, MOL, and CLI). Furthermore, the majority of the adjustments in these disciplinary structures were made from the period 1998–2001 to the period 2004–7.

Funding This study is supported by the Fundamental Research Funds for the Central Universities (Grant No.2013B33214) and the National Natural Science Foundation of China (Grant Nos. 71403079 and 71303147). The authors would like to thank anonymous referees for their valuable comments. We also thank Christopher John Hogan for his proofreading.

References Abramo, G., Cicero, T., & D’Angelo, C. A. (2012) ‘The Dispersion of Research Performance Within and between Universities as a Potential Indicator of the Competitive Intensity in Higher Education Systems’, Journal of Informetrics, 6/2, 155–68. Aksnes, D. W., van Leeuwen, T. N., & Sivertsen, G. (2014) ‘The Effect of Booming Countries on Changes in the Relative Specialization Index (RSI) on Country Level’, Scientometrics, 101, 1391–401. Bornmann, L. & Leydesdorff, L. (2013a) ‘Macro-indicators of Citation Impacts of Six Prolific Countries: InCites Data and the Statistical Significance of Trends’, PloS One, 8/2, e56768. ——. (2013b) ‘The Validation of (Advanced) Bibliometric Indicators Through Peer Assessments: A Comparative Study Using Data from InCites and F1000’, Journal of Informetrics, 7/2, 286–91. Cheng, Y. (2011) ‘A Reflection on the Effects of the 985 Project’, Chinese Education and Society, 44/5, 19–30. Cheng, Y. & Liu, N. (2008) ‘Tracking the Development of China’s Top Universities Using Scientometric Indicators, 1997-2005’, Frontiers of Education in China, 3/3, 415–28. Ding, Z. Q., Ge, J. P., Wu, X. M., & Zhang, X. N. (2013) ‘Bibliometrics Evaluation of Research Performance in Pharmacology/Pharmacy: China Relative to Ten Representative Countries’, Scientometrics, 96/3, 829–44. Dore´, J. C. & Ojasoo, T. (2001) ‘How to Analyze Publication Time Trends by Correspondence Factor Analysis: Analysis of Publications by 48 Countries in 18 Disciplines Over 12

Tracking the development of disciplinary structure Years’, Journal of the American Society for Information Science and Technology, 52/9, 763–9. European Commission. (1997) Second European Report on S&T Indicators 1997. Brussels – Luxembourg: EUR 17639. Frame, J. D. (1977) ‘Mainstream Research in Latin America and the Caribbean’, Interciencia, 2/3, 143–8. Gla¨nzel, W. (2000) ‘Science in Scandinavia: A Bibliometric Approach’, Scientometrics, 49/2, 357. Harzing, A. W. & Giroud, A. (2014) ‘The Competitive Advantage of Nations: An Application to Academia’, Journal of Informetrics, 8/1, 29–42. InCitesTM, T. R. (2012) Report Created: 2015-1-12; Data Processed October 1, 2014; Data Source: Web of ScienceÕ This Data is Reproduced Under a License from Thomson Reuters. Kostoff, R. N., Briggs, M. B., Rushenberg, R. L., Bowles, C. A., Icenhour, A. S., Nikodym, K. F., Barth, R. B., & Pecht, M. (2007) ‘Chinese Science and Technology—Structure and Infrastructure’, Technological Forecasting and Social Change, 74/9, 1539–73. Kozlowski, J., Radosevic, S. & Ircha, D. (1999) ‘History Matters: The Inherited Disciplinary Structure of the Postcommunist Science in Countries of Central and Eastern Europe and its Restructuring’, Scientometrics, 45/1, 137–66. Li, F., Yi, Y., Guo, X., & Qi, W. (2012) ‘Performance Evaluation of Research Universities in Mainland China, Hong Kong and Taiwan: Based on a Two-dimensional Approach’, Scientometrics, 90/2, 531–42. Lo´pez-Illescas, C., Moya-Anego´n, F., & Moed, H. (2011) ‘A Ranking of Universities Should Account for Differences in Their Disciplinary Specialization’, Scientometrics, 88/2, 563–74. MOE (1998) Action Plan for Revitalizing Education of the Twenty-First Century. accessed 9 Jan 2015 ——. (2004) Rejuvenating Education 2003-2007. http://www. moe.gov.cn/publicfiles/business/htmlfiles/moe/moe_177/ 200407/2488.html accessed 9 Jan 2015. ——. (2010a) On the Acceleration of the Construction of the World First-class University and High Level University. http://kejichu. sjtu.edu.cn/info/1363/8226.htm accessed 9 Jan 2015. ——. (2010b) Outline of China’s National Plan for Medium and Long-term Education Reform and Development (2010-2020). http://www.moe.edu.cn/publicfiles/business/htmlfiles/moe/ moe_838/201008/93704.html accessed 9 Jan 2015. Moed, H. F. (2005). Citation Analysis in Research Evaluation. New York: Springer-Verlag. Moed, H. F., Van Leeuwen, T. N., & Visser, M. S. (1999) ‘Trends in Publication Output and Impact of Universities in the Netherlands’, Research Evaluation, 8/1, 60–7.

.

323

Moya-Anego´n, F. & Herrero-Solana, V. (2013) ‘Worldwide Topology of the Scientific Subject Profile: A Macro Approach in the Country Level’, PloS One 8/12, e83222. Nederhof, A. J. (2006) ‘Bibliometric Monitoring of Research Performance in the Social Sciences and the Humanities: A Review’, Scientometrics, 66/1, 81–100. Schubert, A. & Braun, T. (1986) ‘Relative Indicators and Relational Charts for Comparative Assessment of Publication Output and Citation Impact’, Scientometrics, 9/5–6, 281–91. Tang, L. (2013) ‘Does ‘Birds of a Feather Flock Together’ Matter: Evidence from a Longitudinal Study on the USChina Scientific Collaboration’, Journal of Informetrics, 7/2, 330–44. Tang, L. & Hu, G. (2013) ‘Tracing the Footprint of Knowledge Spillover: Evidence from U.S.-China Collaboration in Nanotechnology’, Journal of the American Society for Information Science and Technology, 64/9, 1791–801. Thijs, B. & Gla¨nzel, W. (2008) ‘A Structural Analysis of Publication Profiles for the Classification of European Research Institutes’, Scientometrics 74/2, 223–36. Vinkler, P. (2010) The Evaluation of Research by Scientometric Indicators. Oxford: Chandos Publishing. Xiong, Q., Zhang, D., & Liu, H. (2011) ‘Governance Reform at China’s “985 Project” Universities’, Chinese Education and Society, 44/5, 31–40. Yang, L. Y., Yue, T., Ding, J. L., & Han, T. (2012) ‘A Comparison of Disciplinary Structure in Science between the G7 and the BRIC Countries by Bibliometric Methods’, Scientometrics, 93/2, 497–516. Yuan, C. & Zhang, G. (2011) The Ministry of Education: 985 and 211 Projects have Closed their Door. accessed 9 Jan 2015, China Youth Online. Zhang, H., Patton, D., & Kenney, M. (2013) ‘Building Globalclass Universities: Assessing the Impact of the 985 Project’, Research Policy, 42/3, 765–75. Zhou, J. (2004) The Speech on the Working Conference of the Phase II Construction of 985 Project. http://www.moe.edu.cn/ publicfiles/business/htmlfiles/moe/moe_155/200410/4074. html> accessed 9 Jan 2015, MOE Bulletin. Zhou, Q., Rousseau, R., Yang, L., Yue, T., & Yang, G. (2012) ‘A General Framework for Describing Diversity Within Systems and Similarity between Systems with Applications in Informetrics’, Scientometrics, 93/3, 787–812. Zhu, X., Wu, Q., Zheng, Y., & Ma, X. (2004) ‘Highly Cited Research Papers and the Evaluation of a Research University: A Case Study: Peking University 1974–2003’, Scientometrics, 60/2, 237–347.

324

.

F. Li et al.

Appendix A.

Detailed results of DBI values and similarity degrees

Institute

OUC TSINGHUA CQU SEU ECNU USTC LZU JLU DLUT SCUT BIT HNU UESTC CAU TJU NWSUAF BUAA HIT NANKAI BNU SCU NJU XJTU CSU TONGJI ZJU XMU NUDT NEU NWPU SDU FUDAN SJTU WHU SYSU HUST PKU

RG1 1998–2001

2004–7

0.111 0.032 0.037 0.068 0.036 0.051 0.063 0.048 0.030 0.027 0.033 0.104 0.127 0.252 0.049 0.252 0.146 0.165 0.183 0.049 0.016 0.042 0.109 0.250 0.037 0.023 0.160 0.126 0.212 0.216 0.037 0.015 0.042 0.076 0.029 0.061 0.014

0.059 0.023 0.044 0.043 0.028 0.046 0.030 0.046 0.017 0.027 0.031 0.017 0.102 0.193 0.051 0.198 0.104 0.096 0.085 0.038 0.014 0.022 0.059 0.105 0.019 0.004 0.064 0.148 0.135 0.147 0.011 0.013 0.037 0.011 0.023 0.043 0.017

(N) (VH) (VH) (H) (VH) (H) (H) (VH) (VH) (VH) (VH) (N) (N) (VL) (VH) (VL) (N) (L) (L) (VH) (VH) (VH) (N) (VL) (VH) (VH) (L) (N) (VL) (VL) (VH) (VH) (VH) (H) (VH) (H) (VH)

RG2 2010–3

(H) (VH) (VH) (VH) (VH) (VH) (VH) (VH) (VH) (VH) (VH) (VH) (N) (L) (H) (L) (N) (H) (H) (VH) (VH) (VH) (H) (N) (VH) (VH) (H) (N) (N) (N) (VH) (VH) (VH) (VH) (VH) (VH) (VH)

Note: Similarity degrees are coded in parentheses.

0.060 0.029 0.039 0.030 0.024 0.058 0.021 0.023 0.034 0.034 0.041 0.037 0.109 0.122 0.035 0.140 0.086 0.066 0.044 0.041 0.022 0.019 0.035 0.041 0.019 0.002 0.015 0.123 0.110 0.126 0.008 0.033 0.038 0.006 0.057 0.027 0.017

(H) (VH) (VH) (VH) (VH) (H) (VH) (VH) (VH) (VH) (VH) (VH) (N) (N) (VH) (N) (H) (H) (VH) (VH) (VH) (VH) (VH) (VH) (VH) (VH) (VH) (N) (N) (N) (VH) (VH) (VH) (VH) (H) (VH) (VH)

1998–2001

2004–7

0.103 0.112 0.098 0.125 0.123 0.163 0.182 0.179 0.124 0.138 0.158 0.244 0.194 0.197 0.159 0.220 0.213 0.246 0.348 0.124 0.095 0.147 0.167 0.271 0.100 0.121 0.313 0.194 0.299 0.305 0.159 0.052 0.063 0.185 0.092 0.097 0.075

0.071 0.084 0.099 0.077 0.091 0.129 0.111 0.153 0.096 0.122 0.133 0.108 0.147 0.160 0.167 0.165 0.161 0.163 0.202 0.081 0.062 0.087 0.081 0.136 0.060 0.043 0.155 0.180 0.215 0.227 0.078 0.034 0.046 0.048 0.019 0.035 0.031

(N) (N) (H) (N) (N) (L) (L) (L) (N) (N) (L) (VL) (L) (L) (L) (VL) (VL) (VL) (VL) (N) (H) (N) (L) (VL) (N) (N) (VL) (L) (VL) (VL) (L) (H) (H) (L) (H) (H) (H)

RG3 2010–3

(H) (H) (H) (H) (H) (N) (N) (L) (H) (N) (N) (N) (N) (L) (L) (L) (L) (L) (VL) (H) (H) (H) (H) (N) (H) (VH) (L) (L) (VL) (VL) (H) (VH) (VH) (VH) (VH) (VH) (VH)

0.074 0.077 0.092 0.067 0.064 0.119 0.069 0.072 0.100 0.101 0.111 0.110 0.156 0.119 0.106 0.130 0.140 0.125 0.102 0.060 0.029 0.053 0.059 0.046 0.027 0.018 0.051 0.171 0.177 0.197 0.021 0.013 0.019 0.014 0.018 0.025 0.016

(H) (H) (H) (H) (H) (N) (H) (H) (N) (N) (N) (N) (L) (N) (N) (N) (N) (N) (N) (H) (VH) (H) (H) (VH) (VH) (VH) (H) (L) (L) (L) (VH) (VH) (VH) (VH) (VH) (VH) (VH)

1998–2001

2004–7

0.137 0.145 0.135 0.162 0.168 0.184 0.242 0.237 0.172 0.193 0.201 0.320 0.219 0.244 0.219 0.280 0.250 0.284 0.425 0.140 0.136 0.175 0.193 0.304 0.129 0.176 0.390 0.212 0.346 0.348 0.205 0.075 0.082 0.248 0.136 0.112 0.103

0.117 0.123 0.140 0.113 0.132 0.158 0.169 0.219 0.145 0.188 0.195 0.165 0.170 0.211 0.242 0.218 0.201 0.212 0.269 0.104 0.106 0.122 0.114 0.176 0.099 0.089 0.225 0.201 0.273 0.275 0.124 0.062 0.076 0.093 0.047 0.050 0.049

(N) (N) (N) (L) (L) (L) (VL) (VL) (L) (L) (VL) (VL) (VL) (VL) (VL) (VL) (VL) (VL) (VL) (N) (N) (L) (L) (VL) (N) (L) (VL) (VL) (VL) (VL) (VL) (H) (H) (VL) (N) (N) (N)

2010–3 (N) (N) (N) (N) (N) (L) (L) (VL) (N) (L) (L) (L) (L) (VL) (VL) (VL) (VL) (VL) (VL) (N) (N) (N) (N) (L) (H) (H) (VL) (VL) (VL) (VL) (N) (H) (H) (H) (VH) (VH) (VH)

0.118 0.118 0.144 0.111 0.104 0.147 0.111 0.118 0.161 0.169 0.171 0.175 0.191 0.172 0.171 0.177 0.189 0.176 0.153 0.082 0.054 0.081 0.091 0.071 0.057 0.053 0.097 0.204 0.239 0.252 0.045 0.020 0.032 0.042 0.021 0.040 0.025

(N) (N) (N) (N) (N) (N) (N) (N) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (H) (H) (H) (H) (H) (H) (H) (H) (VL) (VL) (VL) (VH) (VH) (VH) (VH) (VH) (VH) (VH)