Educ Psychol Rev DOI 10.1007/s10648-016-9360-8 REFLECTION ON THE FIELD
Individual and Institutional Productivity in Educational Psychology Journals from 2009 to 2014 Hannah Greenbaum 1 & Lisa Meyer 1 & M Cecil Smith 1 & Amanda Barber 1 & Heather Henderson 1 & David Riel 1 & Daniel H. Robinson 2
# Springer Science+Business Media New York 2016
Abstract This article examines the productivity of both individuals and institutions, indexed through an examination of five educational psychology journals (Cognition and Instruction, Contemporary Educational Psychology, Educational Psychologist, Educational Psychology Review, and Journal of Educational Psychology) from 2009 to 2014. These results are discussed relative to four previous studies (Hsieh et al. Contemporary Educational Psychology, 29, 333–343, 2004; Jones et al. Contemporary Educational Psychology, 35, 11–16, 2010; Smith et al. Contemporary Educational Psychology, 23, 173–181, 1998; Contemporary Educational Psychology, 28, 422–430, 2003). Vanderbilt University and Fred Paas replaced the University of Maryland and Richard E. Mayer as the top research institution and author, respectively. Sixteen of the top 19 researchers’ institutions were outside the USA, compared to only 10 of the top 32 during 2003–2008 and three of the top 20 during 1991–1996. Educational psychology research continues the trend of becoming more international. Keywords Productivity . Educational psychology . Journals . Authors The scholarly productivity of an academic program’s faculty is one of several important considerations for graduate students when choosing an institution and an advisor. Many students’ horror stories recount poor selection of an institution, program of study, or advisor—resulting in years of unproductivity and failure to progress. Publications in refereed journals are also an important criterion when considering new Ph.D.s for faculty positions and for merit rewards to faculty members. At some state-supported institutions, faculty publications may be considered as one among several benchmarks of institutional Bquality^ and, thereby, can influence legislative priorities for institutional or program-level funding. * Daniel H. Robinson
[email protected]
1
West Virginia University, Morgantown, WV, USA
2
Learning Sciences, The University of Texas at Austin, Mail code G2100, Austin, TX 78713, USA
Educ Psychol Rev
Productivity studies have been conducted in various fields (e.g., school psychology, Davis et al. 2005; developmental psychology, Pahlke et al. 2006; and discipline-specific journals, de la Cruz et al. 2006). Our research group has previously conducted studies examining individual and institutional productivity in five journals: Cognition and Instruction, Contemporary Educational Psychology, Educational Psychologist, Educational Psychology Review, and Journal of Educational Psychology (Hsieh et al. 2004; Jones et al. 2010; Smith et al. 1998, 2003) over the period from 1991 through 2008. The present study updates this literature by examining productivity during 2009– 2014. We have employed two different approaches in identifying the most productive authors. The first two studies used a formula that gives weighted credit to authors depending on the order of authorship, whereas the last two studies also counted the number of articles for each author. The formula method is clearly preferred for defining institutional productivity (simple article counts per author would be misleading if there were multiple authors on articles at the same institution). The same method can also be an effective deterrent to the questionable behavior of gratuitous, reciprocal authorship add-ons designed to increase individual publication numbers (see Bishop 2015). However, the formula method essentially punishes collaboration and instead rewards both sole and first authorships. Especially when considering the importance of mentoring graduate students by having them serve as first author on collaborative efforts, the formula method could discourage such behavior if senior authors seek recognition for their individual productivity. Acknowledging the increasing importance of collaboration based on the increasing number of authors per article, the last two studies ranked the top authors based on total number of articles, regardless of author position. In the present study, we updated data from the previous studies by again creating two individual lists using both methods. We also examined the productivity of institutions, numbers of authors per article, and articles authored by international scholars.
Method We examined all articles (except errata/corrigenda) published in the five journals from 2009 to 2014. We coded author names and institutions, placement in authorship order, and number of authors for each article. We calculated both total articles and points for each individual author. Consistent with our previous research, we employed a scoring system based on total number of publications and authorship order for each article, taken from the Howard et al. (1987) method: A single-authored article is awarded 1 point to the author (and, by association, the author’s institution). Credit is given to authors (and institutions) on a proportional basis for multipleauthor papers (e.g., first authorship for a co-authored paper = 0.60, second authorship = 0.40, total = 1.00). Institutional scores are determined by summing the scores of all authors’ affiliations with institutions. The formula, used previously by Smith et al. (1998, 2003), is as follows: credit ¼ ð1:5n−1Þ=
n X i¼1
1:5i−1
!
where n is the total number of authors for a particular article and i is the individual author’s ordinal position. We calculated individual productivity by sorting the database by author and counting the number of articles regardless of number of authors for each article or author position. Using the same database, we also calculated individual productivity based on the point system described
Educ Psychol Rev
earlier. We included any published work that appeared in the journal. BNon-research articles^ included book reviews, editorials, essays, introductions, and interviews. BTotal articles^ included any article appearing in the five journals, whereas Bother articles^ were articles from 2009 to 2014 in any journal other than the selected five. To gather more information about the top authors’ productivity and to verify our findings, we e-mailed the authors requesting their current vita. The authors were also asked to identify, for the articles that they authored in the five journals from 2009 to 2014, those co-authors who were graduate students at the time that the study was conducted. For each of the top authors, the average number of authors per article was calculated, as well as a total number of graduate student co-authors. We also included the number of articles that were classified as non-research (e.g., editorials, essay, introductions, interviews, book reviews, and memorandums). From their vita, we counted the number of journal articles that the top authors published from 2009 to 2014 in any journal other than the selected five, to determine whether authors targeted mainly the educational psychology journals or published more in outside journals. Institutional productivity was calculated using the point system. The data were sorted by institutional affiliation, point totals for each were calculated, and the top institutions were listed. Finally, we calculated the numbers of institutional affiliations outside the USA to assess the involvement of international authors.
Results and Discussion Individual Productivity A total of 931 articles were published in the five journals (CEP, C&I, EP, EPR, and JEP) from 2009 to 2014 with 2123 authors. Productivity results are shown in Tables 1, 2, and 3 for the top authors and institutions. The top 19 most productive individuals according to numbers of articles produced a total of 188 articles– of which only 11 (6 %) were single-authored papers, written by five of these individuals. Thus, collaborative papers (i.e., two or more authors) clearly dominated among the papers published by the top-producing authors and their coauthors. Based on total articles, Fred Paas was the most productive author. Paas is an emerging star as he was not listed among the top authors from 1991 to 1996, 1997 to 2001, or 1997 to 2002 and was tied for 20th from 2003 to 2008. The top two authors in the three previous studies, Herbert Marsh and Richard Mayer, Bslipped^ to third and tenth, respectively. Remarkably, these two authors have continued their high productivity level in these five journals over the past 24 years. Paas and Marsh both work at institutions outside the USA. Increasingly, the top authors represent such non-US institutions, which may indicate an increasingly global representation in our field. In 1991–1996, only three of the top 20 authors were outside the USA, whereas in 2009–2014, 16 of the top 19 authors worked at institutions outside the USA. As for the top authors according to numbers of points, Andrew Martin had the most points, followed by Paas. Marsh and Mayer have appeared in the point lists in all four periods. Similar to the findings of Hsieh et al. (2004) and Jones et al. (2010), there was considerable overlap in the numbers and point lists, with 13 persons appearing in both lists. Interestingly, the point total for the top author has decreased in each period with Marsh receiving 10.14 in 1991–1996 and Mayer had 8.67 and 8.23 from 1997 to 2001 and 2003 to 2008, respectively, and Martin receiving 5.25 in 2009–2014. Perhaps, this declining trend reflects a decreasing importance of being first or sole author. Similar to the number of articles system, there is increasing
6
14
3
4
4
4
nr
1
nr
20
nr
nr
nr
nr
20
10
nr
9
9
9
12
12
12
12
12
12
12
12
nr
2
3
nr
3
2
8
20
1
4
Rank 2003–2008
Rank 2009–2014
nr
12
9
nr
nr
nr
nr
nr
nr
1
nr
nr
nr
nr
nr
nr
2
nr
nr
Rank 1991–2002
Maarten Vansteenkiste
John Sweller
Daniel Robinson
Johnmarshall Reeve
Benjamin Nagengast
Alexandre Morin
Mareike Kunter
Avi Kaplan
Reinhard Pekrun
Richard Mayer
Thomas Goetz
Andrew Martin
Tamara van Gog
Ulrich Trautwein
Alexander Renkl
Lynn Fuchs
Herbert Marsh
Oliver Ludtke
Fred Paas
Top authors
2005
1972
1993
1986
2009
2005
2004
1997
1982
1973
2002
1999
2006
2002
1991
1981
1974
2004
1993
Year PhD
Ghent
New South Wales
Texas
Korea
Tubingen
Tubingen
Goethe
Ben-Gurion
Munich
California-Santa Barbara
Konstanz
Sydney
Erasmus
Tubingen
Freiburg
Vanderbilt
Oxford
Max Planck Institute
Erasmus/Wollongong
2014 institution
Table 1 Rankings of individual educational psychologists using the count system
8
8
8
8
8
8
8
8
9
9
9
10
11
11
11
11
13
14
16
Total articles
2
2
3
4
2
0
2
3
4
1
2
7
4
4
1
6
5
1
3
First-author articles
0
2
1
2
0
0
0
0
0
3
0
3
0
0
0
0
0
0
0
Single-author articles
0
0
2
0
0
0
0
2
1
2
0
0
2
0
0
0
0
0
3
Non-res articles
4.9
2.6
3.9
2.3
5.4
5.9
4.4
2.8
4.6
4
5.1
3.5
3.5
5.1
3.1
7
5.6
5
3.3
Mean authors per article
4
1
15
2
7
2
5
4
12
14
4
2
5
8
8
10
na
6
5
Grad student authors
35
47
10
74
23
23
57
21
127
23.6
61
21
41
50
31
13.5
24.5
50
46
Median citations per article
70
20
25
15
27
62
33
9
37
51
15
58
59
62
78
57
17
65
47
Other articles
Educ Psychol Rev
18
8
25
nr
2
nr
nr
nr
16
17
17
19
20
14
15
13
1
13
nr
nr
7
12
nr
4
6
nr
7
5
17 nr
nr
4
9
nr
3
10 11
nr
nr
2
nr
nr
nr
9
nr
nr
1
nr
nr
nr nr
nr
5
2
nr
nr
nr
nr
19
1
Rank 1997–2001
Rank 2003–2008
Rank 2009–2014
nr
nr
nr
8
nr
nr
2
nr
nr
nr nr
nr
nr
nr
1
4
nr
nr
nr
nr
Rank 1991–1996
Lisa Linnenbrink-Garcia
Jennifer Cromley
Reinhard Pekrun
Patricia Alexander
Oliver Ludtke
Alexander Renkl
Richard Mayer
Ulrich Trautwein
Robert Klassen
John Sweller H. Lee Swanson
Avi Kaplan
Lynn Fuchs
Tamara van Gog
Herbert Marsh
Gregory Schraw
Jeffrey Greene
Johnmarshall Reeve
Fred Paas
Andrew Martin
Top authors
Table 2 Rankings of individual educational psychologists using the point system
2002
2005
1982
1981
2004
1991
1973
2002
2003
1972 1976
1997
1981
2006
1974
1990
2007
1986
1993
1999
Year PhD
Michigan
Illinois
Munich
Maryland
Max Planck
California-Santa Barbara Freiburg
Tubingen
York
New South Wales California Riverside
Ben-Gurion
Vanderbilt
Erasmus
Oxford
UNLV
Maryland
Erasmus and Wollongong Korea
Sydney
2014 institution
2.62
2.64
2.66
2.66
2.77
2.78
2.93
2.99
3.16
3.42 3.17
3.44
3.56
3.77
3.84
4.03
4.16
4.41
4.56
5.25
Point total
7
7
10
6
14
10
9
10
7
8 5
9
11
11
13
7
7
8
14
10
Total articles
0
0
0
1
0
0
0
0
0
2 2
0
0
0
0
2
1
2
0
3
Single-author articles
Educ Psychol Rev
Educ Psychol Rev Table 3 Rankings of institutions, by educational psychology faculty productivity 2009–2014
2003–2008
1997–2001
1991–1996
Institution
Score
1
2
16
35
Vanderbilt University
19.43
2
1
1
1
University of Maryland
18.80
3
14
nr
3
Arizona State University
16.31
4 5
7 nr
22 nr
nr nr
University of Texas Erasmus University (Netherlands)
15.52 14.88
6
nr
nr
nr
Temple University
13.20
7
nr
12
13
Purdue University
12.89
8
4
2
9
University of Michigan
12.41
9
nr
35
7
University of Pittsburgh
11.48
10
5
11
25
University of Illinois
11.31
11
nr
nr
nr
University of Freiburg (Germany)
10.25
12 13
nr 6
30 3
14 5
Florida State University University of California-Santa Barbara
14
nr
nr
nr
University of Sydney (Australia)
9.25
15
nr
nr
nr
McGill University (Canada)
9.23
16
nr
32
34
University of North Carolina
9.01
17
nr
nr
nr
Duke University
8.92
18
3
nr
nr
University of Nevada-Las Vegas
8.87
19
15
6
12
University of California-Los Angeles
8.47
20
17
25
nr
University of New South Wales (Australia)
8.05
9.65 9.62
representation of non-US authors among the top 20. In 2009–2014, 11 of the top 20 authors, according to the point system, worked at institutions outside the USA – compared with only three in 1991–1996. Similar to the 2003–2008 findings, the number of non-research articles was low compared to an author’s total articles. With regard to graduate student co-authors, the three authors from US institutions averaged over 12, whereas 15 non-US authors averaged just under five over the 6-year period. This result may need further investigation regarding whether it is an accurate measure of either the importance or willingness of non-US researchers to work with graduate students and whether cultural differences exist. Only four authors averaged more than one graduate student co-author per article, compared to eight during 2003–2008. Again, this result deserves further investigation as to whether a trend exists in terms of declining graduate student co-authorship with top authors. Similar to 2003–2008, there is a broad range of experience among the top authors—ranging from just a few years in the field (Nagengast) to more than 40 years (Sweller, Mayer, and Marsh). Four authors received their doctorate within the past 10 years (Morin, Nagengast, Van Gog, & Vansteenkiste). In contrast, nine authors from both tables received their doctorate over 20 years ago. Publishing in journals other than the educational psychology journals is becoming more common among the top authors. Compared to 2003–2008, when nine authors published more articles in the five educational psychology journals than in other journals, none of the present top authors did so. In terms of the total number of articles authored during the 6year period, Alexander Renkl had the most with 89, whereas Avi Kaplan had 17. Citations are considered by many to provide a measure of the impact of one’s work. Similar to Jones et al. (2010), we calculated the median number of citations per article for the top
Educ Psychol Rev
authors according to Google Scholar. Recording the data on November 30, 2015, the median number of citations for each of the top authors’ educational psychology journal articles is displayed in Table 1. The medians ranged from less than one (two authors) to 38 (Sweller). Of course, if most of an author’s articles were published earlier in the time period than later, the number of citations is likely to be higher.
Institutional Productivity Table 3 lists the 20 most productive institutions for educational psychology articles from 2009 to 2014, as well as previous rankings. Among the 522 institutions represented, Vanderbilt University replaced the University of Maryland (second place) as the most productive. Arizona State University moved back up to the number 3 spot it occupied in 1991–1996, after a considerable drop. The University of Texas was number 4, showing a steady upward trend after not being ranked in 1991–1996. Perhaps, the most obvious trend is the increasing number of institutions outside of the USA. In 1991–1996, none of the top 20 institutions was outside the USA. This number increased to three in 1997–2001, four in 2003–2008, and now five in 2009–2014. As for the relationship between productive individuals and institutions, only 11 of the top 20 most productive institutions employ top authors in either the articles or point lists. Similarly, institutions that did not make the top 20 most productive employed 12 authors who were listed as most productive. Thus, simply having a productive faculty member does not ensure that an institution will be highly productive, nor does having a productive institution ensure that one of its faculty will be highly productive.
International Involvement Given that five of the top 20 institutions were international, we examined the numbers of institutions from non-US countries represented in our 2009–2014 articles. Smith et al. Table 4 Representation of non-US institutions in educational psychology journals, 2009–2014
# Institutions
Country
45
Germany
30
Canada
25
UK
18
France
16
Australia
15 13
China Netherlands
8
Israel; South Korea
7
Spain
6
Cyprus; Finland; Taiwan
4
Belgium; Singapore; Switzerland
3
Greece; Japan; Portugal; Turkey
2
Argentina; Austria; Hungary; Italy; New Zealand; Norway; Saudi Arabia; Sweden
1
Denmark; Iceland; Luxembourg; Macau; Russia; South Africa; United Arab Emirates
Educ Psychol Rev
(2003) found that 22 % of the published authors in 1997–2001 represented foreign institutions. We found that, by 2014, non-US authors were affiliated with 255 of the 511 (50 %) of the schools, colleges, research institutes, and universities. Table 4 shows great diversity of non-US institutions that are now represented in educational psychology journals.
Conclusion Our purposes for this study were simply to update the 2009–2014 rankings of the most productive authors and institutions as well as to analyze educational psychology article trends in terms of collaboration and international involvement. Our findings revealed that individual and institutional productivity in educational psychology journals has been both consistent (e.g., Richard E. Mayer, University of Maryland) and changing (e.g., Fred Paas, University of Texas) in terms of the top performers over the past 24 years. At the same time, some institutions have recently appeared (e.g., Erasmus University) and disappeared (e.g., the University of Nebraska), perhaps due in part to the addition and loss of productive individuals. Other institutions (e.g., Arizona State University, the University of Michigan) remain highly productive without having any obvious Bstars^ in terms of individual productivity in the five journals. Of course, defining institutional and individual productivity in terms of publishing in only five targeted journals is an admittedly narrow view. Finally, although a few individuals continue to be highly productive, some individuals have appeared in this most recent ranking, whereas others have disappeared. Consistent with previous studies (Hsieh et al. 2004; Jones et al. 2010; Smith et al. 1998, 2003), we hope that the findings may be useful to potential graduate students who are considering where they would like to pursue a degree. In terms of collaboration, it is easier for faculty members who seldom publish with graduate students to rank among the most productive authors in terms of points than it is for those who frequently publish with graduate students and are rarely first or sole authors. For that reason, we also calculated productivity in terms of numbers of articles authored to recognize the latter type of faculty member. Because our field is becoming more collaborative (Jones et al. 2010), it makes sense to define productivity in ways that do not discourage collaboration. We hope that these results may prove to be a valuable resource, in addition to others, to those in search of an institution or department that meets their professional and scholarly needs. Finally, our field has witnessed increased involvement by authors outside of the USA. We believe that this is a positive development for educational psychology, suggesting appreciation for a diversity of phenomena and problems that interest educational psychologists, an embrace of emerging investigative methods, theories, and analytic approaches, and a more global and inclusive perspective. Our hope is that these trends will encourage continued growth in both collaboration and international involvement within educational psychology, which will serve to strengthen our field. Compliance with Ethical Standards Conflict of Interest The authors declare that they have no conflict of interest.
Educ Psychol Rev
References Bishop, D. (2015). Editors behaving badly? Retrieved 21 Jul 2015 from http://deevybee.blogspot.co.uk/2015_ 02_01_archive.html Davis, K. S., Zanger, D., Gerrard-Morris, A., Roberts, G., & Robinson, D. H. (2005). Productivity and collaboration of authors in school psychology journals, 1991–2003. The School Psychologist, 59(4), 129– 133. de la Cruz, B., Cannella-Malone, H. J., Edrisinha, C., Sigafoos, J., Robinson, D., & Son, S.-H. (2006). Author productivity and publication trends in autism-specific journals from 1997 to 2004. Focus on Autism and Other Developmental Disabilities, 21(4), 245–250. Howard, G. S., Cole, D. A., & Maxwell, S. E. (1987). Research productivity in psychology based on publication in the journals of the American Psychological Association. American Psychologist, 42, 975–986. Hsieh, P., Acee, T., Chung, W., Hsieh, Y., Kim, H., Thomas, G. D., You, J., & Robinson, D. H. (2004). An alternate look at educational psychologist’s productivity from 1991 to 2002. Contemporary Educational Psychology, 29, 333–343. Jones, S., Fong, C., Torres, L., Yoo, J., Decker, M., & Robinson, D. H. (2010). Productivity in educational psychology journals from 2003–2008. Contemporary Educational Psychology, 35, 11–16. Pahlke, E., Lamb, L. M., Wilson, A., Cooper, C. E., Green, V. A., & Robinson, D. H. (2006). An examination of developmental psychologists’ productivity from 1995–2004. E-News, 24, 6–7. Smith, M. C., Locke, S. G., Boisse, S. J., Gallagher, P. A., Krengel, L. E., Kuczek, J. E., et al. (1998). Productivity of educational psychologists in educational psychology journals, 1991–1996. Contemporary Educational Psychology, 23, 173–181. Smith, M. C., Plant, M., Carney, R. N., Arnold, C. S., Jackson, A., Johnson, L. S., et al. (2003). Further productivity of educational psychologists in educational psychology journals, 1997–2001. Contemporary Educational Psychology, 28, 422–430.
