Science from the periphery: Collaboration, Networks and ‘Periphery Effects’ in the Citation of New Zealand Crown Research Institutes Articles 1995-2000
Shaun Goldfinch Centre for National University Finance 2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo, 101-0003, Japan and Department of Political Science University of Canterbury Private Bag 4800 Christchurch New Zealand e-mail:
[email protected] Tony Dale Department of Computer Science University of Canterbury Karl DeRouen Jr Department of Political Science University of Canterbury
Abstract. While collaboration is associated with higher article citation rates, a body of research has suggested that this is, in part, related to the access to a larger social network and the increased visibility of research this entails, rather than simply a reflection of greater quality. We examine the role of networks in article citation rates by investigating article publication by the nine New Zealand Government-owned Crown Research Institutes (CRIs), drawing on the Science Citation Index. We analyse an aggregate data set of all CRI publications with duplicates removed, and, in addition, investigate each CRI. We find that a greater number of authors, countries and institutions involved in co-publication increases expected citation rates, although there are some differences between the CRIs. However, the type of co-publication affects the expected citation rates. We discover a ‘periphery effect’ where greater levels of co-publication with domestic institutions decreases expected citation rates. We conclude that scientists working on the periphery looking to increase the visibility of their research should strive to link their research to the international research community, particularly through co-publication with international authors.
February, 2003. Forthcoming in Scientometrics,
Science from the periphery: Collaboration, Networks and ‘Periphery Effects’ in the Citation of New Zealand Crown Research Institutes Articles 1995-2000
While citations have often been seen a proxy for the quality of published research, especially in the United States, this has been challenged by a body of research (Baldi 1998; Phelan 1999). Instead, citations might be better seen as a measure of ‘visibility’, ‘influence’, ‘importance’, ‘communication’ or, as Martin (1996) terms it, ‘impact’. To some extent, citations per article are an effect of collaborative behaviour. Collaboration may allow researchers to plug into greater scientific networks. As Katz & Martin (1997, 15) note: an individual researcher may have good contacts with 50 or 100 other researchers in his or her field around the world with whom he or she can contact for information or advice. By collaborating with others in another institution or country, the individual can greatly extend that network.
Networks can be important in maintaining links and learning of new directions and discoveries within the wider discipline (Baldi 1998). A greater network may also increase the likelihood of future collaborations, by the forging of new research links and other relationships. Working with two heads, or more, may allow for the utilisation of differing and complementary skills, knowledge and equipment, the greater scrutiny of work (an ‘internal refereeing’ process) and the pointing out of errors.
While collaboration may increase quality, it may be that citation rates for an article are not simply a reflection of quality, but to some extend reflect the access to greater social networks that co-publication can allow. That is, articles are not necessarily cited only because of greater quality, but because the article is brought to the attention of the citer through personal contacts, and through the many authors involved in producing the work and citing the work themselves. This is at least suggested by a number of studies. The number of authors of a paper is associated with impact; in cancer research (Lawani 1986) and astrophysics (Baldi 1998) the likelihood of being cited increases with a number of authors. Internationally coauthored papers are twice as likely to be cited than single country papers (Narin & Whitlow 1990). As some of these authors note, this is probably related to the access
that co-publication gives to a larger social network and the increased visibility of research this entails.
New Zealand’s peripheral status to the science world allows the further testing of the role of networks in citation behaviour.1 The research published so far suggests that the greater New Zealand scientists can ties themselves into international networks, the greater likelihood their work will be more widely cited. This paper seeks to undertake examine whether this is the case across the nine Crown Research Institutes (Table 1). The Crown Research Institutes are government owned limited liability companies accounting for 86.3 percent of the government spending on R&D during 1997/98 (MORST 2001). Each focuses on broad research areas (Table 1). They are strongly linked to the international scientific world, with high levels of copublication with institutions outside New Zealand, with New Zealand institutions and with each other (Figures 1, 2, 3). Their citation rates measured 1995-2000 are good by international standards, with average citations per article as good or better than citation rates in similar fields in the Australian main public science body, the CSIRO (Table 2; Butler 1997). Their productivity, measured by articles per research staff member, are highly variable between CRIs, reflecting to some extent the different roles played by the agencies (Figure 4). For example, some such as ESR, are primarily service organisations, while others have a stronger research focus.
To measure article publication rates and citations, we draw on the ISI Web of Science, downloading and sorting the data using software we have developed. We have two datasets: First, one containing all publications by the CRIs, but with duplicates (where articles have been co-authored by 2 or more CRIs) removed. We name this All CRIs. Second, a dataset for each CRIs, with all publications. Our datasets gives number of authors per article, number of citations as measured by the Science Citation Index, number of countries involved in co-authorship, number of institutions involved in co-authorship, and number of domestic and foreign institutions involved in co-authorship, amongst other things.2 The All CRIs dataset has a total of 5726 articles included. While the Web of Science allows access to the Humanities Citation Index and the Social Science Citation Index, these reported very few or no publications for the CRIs. For this reason, and because of the physical
sciences focus of the CRIs since the demise of the NZ Institute of Social Research and Development in 1994, it was decided to focus only on the Science Citation Index.
From our general assumption that citations per article for a peripheral science nation are party a function of the ability to tie published research into international networks, we propose the following hypotheses:
Hypothesis 1. If citations rates are in a part an outcome of access to social networks then citations rates should increase with number of authors for each article.
Hypothesis 2. If citations rates are in part an outcome of access to social networks then citations rates should increase with number of countries involved in coauthorship.
Hypothesis 3. If citations rates are in part an outcome of access to social networks then citations rates should increase with number of institutions involved in producing an article. It is assumed that each institution will provide access to wider social networks, both within and outside the institution.
Hypothesis 4. If citations rates are in part an outcome of access to social networks then citations rates should increase with number of institutions involved in producing an article. However, this will depend on the country base of the institutions. Institutions based in a small peripheral country are unlikely to increase access to significantly greater social networks, and so a greater number of domestic institutions involved in co-publication of an article are not likely to produce a greater citation rate.
Methods Setting citations as the dependent variable, we test a number of models for the All CRIs dataset and for the 9 individual CRIs. For the All CRIs dataset we test the following four models. First, number of authors and number of New Zealand institutions involved in co-authorship. Second, number of authors and number of countries of authorship origin involved in co-authorship. Third, total number of institutions involved in co-authorship. Fourth, number of overseas institutions involved in co-authorship. The results for Models 1-4 are presented in Table 3. For
the individual CRIs we test the following three models: First, number of authors and number of New Zealand institutions involved in co-authorship. Second, number of authors and number of countries of authorship origin involved in co-authorship. Third, number of overseas institutions involved in co-authorship. These results are presented in Table 4. We also run a number of simulations using marginal effects. We use a negative binomial regression events count model because of the nature of the dependent variable. Citations cannot take a negative value and can best be considered as counts. As such, ordinary least squares is inappropriate. We chose negative binomial over Poisson based on the assumption that the events (citations) are not independent and because of the significant alpha parameters in all cases (King 1989; StataCorp 2001). Our model selection is designed to avoid problems of multicollinearity. We kept highly correlated variables out of the same equations.
Findings and Conclusion As Table 3 shows, our findings for the All CRIs dataset strongly confirm the hypotheses. The number of authors is highly significantly associated with citation rates, as is number of overseas countries, number of total institutions and number of foreign institutions. Probably of most interest however, is that increasing the number of domestic institutions involved in co-authorship significantly reduces expected average citation rates per article (Table 3). While we expected that increasing domestic institutions would not increase citation rates, we were surprised that it is likely to reduce them. So while collaboration in general increases article impacts, this depends on the type of collaboration. Scientists may be more likely to cooperate with others that are in geographical proximity, as this allows for more informal communication (Kraut & Egido 1988). However, a scientist working on the periphery wishing to increase the impact of her work, should look to collaborate with international individuals and institutions, rather than simply domestic ones. We also ran a number of simulations based on manipulating the marginal effects. With number of authors set at the mean of 3.47 and NZ institutions set at the mean of 1.42, the negative binomial model predicted 5.75 citations. Increasing the number of authors to 5 and decreasing NZ institutions to 1, increases predicted citations to 7.26 (Model 1). In Model 2, increasing the number of authors from the mean of 3.47 to 5 and the number of overseas countries from 0.44 to 1, increases the predicted citation
rate from 5.73 to 7.23. Increasing the number of total institutions from the mean of 2 to 3 (Model 3) and foreign institutions from the mean of 0.58 to 1 (Model 4) increases predicted citations from 5.84 to 6.89, and 5.83 to 6.34, respectively (Table 3).
Using the individual datasets for each CRI, we find significant variation across the different CRIs. AgResearch shows more-or-less similar findings to the aggregated dataset (Table 4). The number of authors is again a highly significant predictor of citation rates, as is number of overseas countries, number of total institutions and number of foreign institutions involved in co-authorship. A greater number of domestic institutions involved in co-publication again significantly reduces the number of expected citations per article. Crop and Field show similar results, although domestic institutions is not significant (Table 4). The effect of increasing the number of domestic institutions on citation rates is not significant for ESR, while number of countries decreases citation rates, but not at a significant level. The most divergent findings are given by the Forest Research Institute (Table 4). Increasing the number of authors is not a significant factor in increasing citation rates, while increasing the number of domestic institutions in co-publication significantly increases expected citation rates. Number of overseas countries is not significant in increasing citation rates, nor is increasing the number of international institutions. For HortRes, findings are similar to the aggregated dataset, although increasing the number of NZ institutions does decrease citation rates, but is not statistically significant. IGNS’s citation rates are not significantly affected by NZ institutions, while increasing the number of countries increases citations, but this is not a significant finding. Industrial Research Limited has similar findings to the aggregated dataset, all of which are significant expect for domestic institutions, while Landcare differs by finding that increasing the number of overseas countries decreases citation rates, while increasing the number of NZ institutions increases it, although neither is significant (Table 4). NIWA show similar results to the aggregate model, but number of authors is not significant when number of overseas countries is included in the model (Table 4).
In sum, our findings are generally as might be expected from the existing research on collaboration and as predicted by out hypotheses. While there are some differences across the CRIs, possibly reflecting disciplinary differences, in general,
increasing numbers of authors, number of countries and number of institutions involved in co-authorship, increases the expected citation rates. That citation rates are higher for articles with higher levels of collaboration and greater geographical and institutional spread of co-authors may be a function of a better quality of research produced through collaboration. However, we also suspect that this higher citation rate is at least partly a function of scientists on the periphery tying their research into international research communities and networks, and thus increasing its visibility. That there is some type of ‘periphery effect’ in citation rates is strongly suggested by our most interesting finding: that the type of collaboration affects the citation rates. Greater levels of collaboration with domestic institutions are likely to lead to lower citation rates. It may be that New Zealand scientists are just not as good as international scientists, and that international linkages led to better research. However, we are not convinced this is necessarily the case. New Zealand, while a small contributor to the international science community, is not noticeably worse and in some cases better, than would be expected given its size and income (ISI 2001; ISI 2002). Even if international collaboration did lead to greater quality and higher citation rates were a reflection of this, it would need to be explained why greater levels of domestic collaboration leads to lower citation rates. It seems highly unlikely that greater collaboration with domestic institutions lowers the quality of articles. The exception to this ‘periphery effect’ is Forest Research, which diverges in that greater international collaboration does not always lead to greater citation rates, while the number of domestic institutions is significantly associated with increasing citation rates. However, in general, the periphery effect seems to hold. Scientists on the periphery and the institutions they work for, if they wish to increase the impact of their research and the benefits this might entail, either in status or commercial terms, should look to tie their research as strongly as they can to the international community.
Table 1 The New Zealand Crown Research Institutes1 NZ Pastoral Agriculture Research Institute Ltd (AgResearch) Horticulture and Food Research Institute of NZ Ltd (HortResearch) NZ Institute for Crop &Food Research Ltd (Crop&Food Research) NZ Forest Research Institute Ltd (Forest Research) Industrial Research Ltd (IR) Institute of Environmental Science & Research Ltd (ESR) National Institute of Water and Atmospheric Research Ltd (NIWA) Institute of Geological & Nuclear Sciences Ltd (IGNS) Landcare Research NZ Manaaki Whenua (Landcare Research) Notes. 1.
The NZ Institute of Social Research and Development was disestablished in 1994.
Table 2 CRI Average Citation Per Article 1995-20001 CRI
Number of Articles
AgResearch Crop&Food ESR Forest Research HortResearch IGNS IR Landcare NIWA All CRIs2 Notes. 1. 2.
1363 476 183 245 719 501 708 888 975 5726
Number of Citations
CPP
8751 2290 1254 1143 3907 2856 4641 5524 6532 34512
6.29 4.81 6.85 4.67 5.43 5.70 6.56 6.22 6.70 6.03
Accessed from ISI Web of Science, Science Citation Index, late 2002 All article publication by CRIs, but with duplicates (where articles co-published by two or more CRIs) removed.
Figure 4 Productivity Measured by Article Per Research Staff Member1
Article/ Research Staff
0.7 0.6 1995
0.5
1996
0.4
1997
0.3
1998
0.2
1999
0.1 0 NIWA
LANDCARE
IR
IGNS
HORTRES
FOREST
ESR
CROP
AGRES
CRI
Notes. 1. Article numbers from ISI Web of Science. Staff numbers from various CRI’s annual report and other documents and Official Information Act 1982 requests.
Figure 1 Co-publication Between CRIs By Number of Articles 1995-20001
CROP 2 ESR 40 7 9
AGRES
6 6
8 15
FOREST
58
19
91
13
29
3 27
5
40
43
89
8
18
4
HORTRES
NIWA
14 13 43
8
2
LANDCARE
13 14 16
IGNS 26 IR
Notes 1.
Network diagrams produced using LaTeX with psnode and pstcol software packages.
Figure 2 Co-publication Between CRIs and Domestic Institutions by Number of Articles 199520001
DEPT CONSERVATION
51
140
UNI CANTERBURY
NIWA
LANDCARE
3 WOODWARD CLYDE NZ LTD 3 MUSEUM AUCKLAND 141
VICTORIA UNI
3 MIRINZ
IR
3 FDN RES SCI TECHNOL 4 NZ PHARMACEUT LTD 183
LINCOLN UNI
IGNS
4 MINISTRY HEALTH 5 CHRISTCHURCH SCH MED 5 CANTERBURY HLTH
190
UNI WAIKATO
HORTRES
5 AUCKLAND HEALTHCARE 6 LIVESTOCK IMPROVEMENT 6 ECOL RES ASSOCIATES
UNI OTAGO
238
FOREST
7 WELLINGTON SCH MED 9 TE PAPA 10 CARINA CHEM LABS
UNI AUCKLAND
293
ESR
11 OTAGO SCH MED 11 CENT ANIM HLTH LAB 15 MAF
MASSEY UNI
OTHERS
Notes. 1. 2.
357
430
CROP
22 CAWTHORN INST
AGRES
47 DAIRYING RES CORP
Numbers in Circles show total articles published with all CRIs. Network diagrams produced using LaTeX with psnode and pstcol software packages.
Figure 3 Co-publication between CRIs and Overseas Regions By Number of Articles 199520001
6
Mid.East 1
1
3
1
IR
HORTRES
5
Paci c
ESR
46
10
51
4
2
65
1
3
10
43
2
1
SE.Asia 6 19
4
16
13
74
3
AGRES 16
4
5
20
1
92
10 13
33 4
17
198
46
89
10
21 137
5
6 1
Europe
6
6 84
62
FOREST
38
1
N.America
LANDCARE
27
82
53 27
Notes 1.
45
2 30
6 UK
32
CROP 2
Africa
74
34
70
59 32
74
4 3
4
77
1
82
Asia
2
24
68 50
3
4
2
2
NIWA
2
Australia
122
S.America 5
IGNS
Network diagrams produced using LaTeX with psnode and pstcol software packages.
References BALDI S. (1998). Normative versus social constructivist processes in the allocation of citations: A network-analytic model. American Sociological Review 63: 829-46 BUREAU OF INDUSTRY ECONOMICS. (1996). Australian Science. Performance from Published Papers. Report 96/3. Canberra: Australian Government Publishing Service BUTLER L, BOURKE, P. AND BIGLIA, B. (1997). CSIRO: Profile of Basic Research. Canberra: Australian National University ISI. (2001). Science in New Zealand. At web cite: http://www.in-cites.com/countries/world-map.html. ISI. (2002). National Research Concentration. At web cite: http://www.in-cites.com/countries/worldmap.html KING, G. (1989). Unifying Political Methodology. Cambridge: Cambridge University Press. KRAUT R, C. EGIDO. (1988). Patterns of contact and communication in scientific research collaboration. Presented at Proceedings of the Conference on Computer-Supported Cooperative Work, 26 -- 28 September, Portland, Oregon LAWANI S. M. (1986). Some bibliographic correlates of quality in scientific research. Scientometrics 9: 13 -- 25 MARTIN B. R. (1996). The use of multiple indicators in the assessment of basic research. Scientometrics 36: 343 - 62 MORST (Ministry of Science, Research and Technology) (2001). New Zealand Research and Development Statistics 1997-8. at web cite:http://www.morst.govt.nz/publications/rdstats_98/gvmt.htm NARIN F, WHITLOW E. S. (1990). Measurement of Scientific Collaboration and Co-Authorship in CEC-related Areas of Science. Report EUR 14581, Office for Official Publications of the European Communities, Luxembourg PHELAN T. J. (1999). A Compendium of Issues for Citation Analysis. Scientometrics 45: 117 – 36 STATACORP (2001). Stata Statistical Software: Release 7.0, Reference H-P. College Station, TX: Stata Corporation.
Table 3. Negative binomial regressions coefficients and marginal effects: All CRIs and Citations, 1995-2000 MODEL VARIABLE Constant Number of Number of Number of Number of Number of
(1)
authors NZ institutions overseas countries total institutions foreign institutions
PREDICTED NUMBER OF CITATIONS Baseline Model Simulation 1 Number of obs Wald chi2(2) Log likelihood Alpha
1.43** 0.13** -0.09*
(2)
1.36** 0.09**
(3)
1.43**
((4)
1.64**
0.17** 0.17** 0.20**
5.75 7.26 5726.00 68.78** -16147.10 1.32**
5.73 7.23 5726.00 72.84** -16127.36 1.31**
5.84 6.89 5726.00 37.15** -16207.39 1.35**
5.83 6.34 5726.00 44.43** –16194.52 1.34**
Notes. Dependent variable is number of times a CRI article is cited as measured by the Web of Science, SCI. Robust standard errors adjusted for clustering on CRI. Baseline model based on all independent variables set to mean. Simulation 1 based on Number of authors=5, Number of NZ institutions=1, Number of overseas countries=1, Number of total institutions=3; Number of foreign institutions=1. * sig at .05 (two-tailed tests) ** sig at .001
Table 4. Negative binomial regression coefficients: CRI and Citations, 1995-2000 MODEL VARIABLE/CRI Constant Number of authors Number of NZ institutions Number of overseas countries Number of foreign institutions Predicted number of citations (baseline model) Number of obs Wald chi2 Log-likelihood Alpha VARIABLE/CRI Constant Number of authors Number of NZ institutions Number of overseas countries Number of foreign institutions Predicted number of citations (baseline model) Number of obs Wald chi2 Log-likelihood Alpha
(1) AG RESEARCH 1.57** 0.14** -0.22**
(2) 1.33** 0.11**
(3) 1.74**
(1) FOREST RESEARCH 1.00 0.04 0.25*
0.11 5.92 1363.00 41.68** -3865.14 1.44** CROP & FOOD 1.31 0.12** -0.12
5.97 1363.00 22.50** -3872.17 1.45**
1.14 0.08*
476.00 11.10* -1259.94 1.21**
4.64 476.00 19.98** -1253.31 1.17**
(3)
(1)
1363.00 5.55* -3897.50 1.52**
1.44
1.15 0.10*
1.52
1.15 0.24** -0.14
476.00 14.68** -1257.61 1.20**
(3)
1.11 0.15**
1.55
0.26* 0.05
4.50
4.56
245.00 14.21** -637.32 1.28**
245.00 7.94* -639.41 1.31**
HORT RESEARCH 1.32 0.10** -0.01
1.32 0.08*
5.91 245.00 0.29 -642.85 1.36**
1.58
5.31 719.00 19.79** -1979.63 1.19**
5.27 719.00 21.20** -1976.02 1.32**
5.83
708.00 52.26** -1984.40 1.72**
708.00 53.76** -1979.37 1.69**
LANDCARE 1.35 0.10** 0.07
1.42 0.13**
0.20* 0.24** 4.70
(2)
IR
0.02 0.02* 6.16
0.29** 4.73
(2)
0.39** 6.00 708.00 32.84** -1989.70 1.75**
1.76
-0.07 0.23* 5.33 719.00 7.93* -1982.00 1.20**
6.00 888.00 51.84** -2545.24 1.11**
5.97 888.00 51.60** -2545.47 1.11**
0.09** 6.15 888.00 10.62* -2565.71 1.18**
VARIABLE/CRI ESR IGNS NIWA Constant 1.27 1.15 1.69 1.29 1.30 1.52 1.70 1.61 1.72 Number of authors 0.14* 0.16** 0.11** 0.09* 0.10** 0.03 Number of NZ institutions -0.04 0.01 -0.12 Number of overseas countries -0.09 0.12 0.28** Number of foreign institutions 0.15* 0.20** 0.18** Predicted number of citations 6.00 5.97 6.32 5.48 5.46 5.45 6.47 6.37 6.44 (baseline model) Number of obs 183.00 183.00 183.00 501.00 501.00 501.00 975.00 975.00 975.00 Wald chi2 9.67* 14.43** 8.46* 23.12** 23.71** 31.76** 28.84** 34.47** 28.56** Log-likelihood -525.69 -525.24 -534.22 -1392.96 -1391.67 -1391.54 -2866.59 -2854.45 -2862.97 Alpha 1.02** 1.01** 1.13** 1.29** 1.24** 1.23** 1.12** 1.09** 1.11** Notes. Dependent variable is number of times a CRI article is cited as measured by the Web of Science, SCI. Robust standard errors. Baseline model based on all independent variables set to mean. * sig at .05 (two-tailed tests) ** sig at .001.
1
During 1981 to 1994, New Zealand was ranked 27th in number of papers published, 22nd on citations and 25th in terms of papers cited at least once. Its share of world citations and published papers was 0.004 percent (Bureau of Industry Economics 1996). On average citations per paper, New Zealand ranked 9th (Bureau of Industry Economics. 1996). During 1994-1998, New Zealand New Zealand has a particular concentration in agricultural science, with 7% of publications devoted to agriculture ranking, ‘producing 3 times as many papers relative to its size than the world percentage in agriculture’(ISI. 2002). Spending on science is low by developed country standards, both in terms of absolute amounts and as a percentage of GDP – in 1997/98, an estimated $NZ 1,107.4 million, was spent, compared to $889.5 million in 1995/96. R&D as a percentage of GDP rose to 1.1 percent in 1997/98, still low compared to the OECD average of 2.10 percent for OECD countries. The proportion of R&D funded by business is low by OECD standards, with New Zealand at 30 percent compared to the OECD average of 60 percent, while government funding as a proportion is high, at 53 percent compared to the OECD average of 30 percent. The other 17 percent are provided universities, overseas and non-profit organisations. While steadily increasing in recent years, the number of research search scientists as a proportion of the workforce force is also low by OECD standards, with 4.4 R&D research scientists per 1,000 workers in 1997/98, compared to 5.5 in other OECD countries (MORST 2001). 2
It is not possible to link author name with institution, or author name with country due to limitations in the Web of Science database.
