.............................................................................................................................................................................................................................................................
The Profession .............................................................................................................................................................................................................................................................
Political Science Journals in Comparative Perspective: Evaluating Scholarly Journals in the United States, Canada, and the United Kingdom James C. Garand, Louisiana State University Micheal W. Giles, Emory University André Blais, Université de Montréal Iain McLean, Nuffield College, Oxford University
In this article we report the results from a new survey of political scientists regarding their evaluations of journals in the political science discipline. Unlike previous research that has focused on data from the United States, we conducted an Internet survey of political scientists in the United States, Canada, and the United Kingdom. We present data on journal evaluations, journal familiarity, and journal impact, both for our entire sample (N =1,695) and separately for respondents from each of the three countries. We document the overall hierarchy of scholarly journals among political scientists, though we find important similarities and differences in how political scientists from these three countries evaluate the scholarly journals in the discipline. Our results suggest that there is a strong basis for cross-national integration in scholarly journal communication, though methodological differences among the three countries may be an impediment. ABSTRACT
P
olitical scientists communicate through a wide variety of scholarly media, including books, journal articles, convention papers, and electronic papers. Among these, books and journal articles hold a preeminent place in scholarly communication networks,
but it is also the case that not all book publishers and scholarly journals are created equal. Rather, there is a well-documented hierarchy among both book publishers (Goodson, Dillman, and Hira 1999) and scholarly journals (Giles and Wright 1975; Giles, Mizell, and Patterson 1989; Garand 1990, 2005; Garand
James C. Garand is the Emogine Pliner Distinguished Professor and the R. Downs Poindexter Professor of Political Science at Louisiana State University. His research interests cover varying aspects of American politics, and his current research includes projects relating to the sociology of the political science discipline, public opinion on immigration, and the political implications of income inequality. He is former editor of the American Politics Quarterly and former president of the Southern Political Science Association. In 2006 he was named the LSU Distinguished Research Master, which is the highest career award for research given to faculty at LSU. (E-mail:
[email protected]). Micheal W. Giles is the Goodrich C. White Professor of Political Science at Emory University. In addition to a continuing interest in the sociology of the discipline, his research focuses primarily on courts and judicial behavior. His current work focuses on the institution of en banc rehearing in the U.S. Courts of Appeals. He has served the profession in many positions including as president of the Southern Political Science Association, chair
of the Law and Courts Section of the American Political Science Association, and editor of the Journal of Politics. André Blais is professor in the department of political science at the Université de Montréal. He holds a Canada Research Chair in Electoral Studies and he is a fellow of the Royal Society of Canada. His main research interests are electoral behavior, turnout, and electoral systems. He is the director of Making Electoral Democracy Work: Voters, Parties, and the Rules of the Game (www.electoraldemocracy.com). He can be reached at:
[email protected]. Iain McLean is a professor of politics at the University of Oxford and Official Fellow in Politics at Nuffield College. His research interests include devolution, the design of electoral systems, and the application of rational choice models to political history. He is the author of Rational Choice and British Politics (OUP, 2001) and of a forthcoming book titled What’s Wrong with the British Constitution? (OUP, 2009). He is a fellow of the British Academy.
doi:10.1017/S1049096509990205
PS • October 2009 695
The Profession: Political Science Journals in Comparative Perspective
.............................................................................................................................................................................................................................................................
and Giles 2003; Crewe and Norris 1991), with some scholarly outlets viewed as more significant than others. Some political scientists differentiate the contributions and importance of various scholarly journals using so-called objective criteria such as citation counts and impact factors (Christenson and Sigelman 1985; Hix 2004; Giles and Garand 2007), while others use subjective self-reports based on surveys of political scientists (Giles, Mizell, and Patterson 1989; Garand 1990; Garand and Giles 2003). There is evidence that, properly conceived, subjective and objective indicators of journal impact are at least moderately highly correlated (Giles and Garand 2007). What do we know about how political scientists evaluate scholarly journals in the discipline? First, most of the work on subjective journal evaluations has been conducted in the United States, and there is a discernible pattern of journal rankings among American political scientists. Some journals are regularly awarded with elite status; these are journals that are familiar to high proportions of political scientists and receive high marks for the rigor and quality of research that they publish. Other journals are targeted to more specialized audiences (and hence are not broadly familiar to political scientists), are perceived as less rigorous in their scholarly standards, or both, and these journals typically receive less favorable evaluations from American political scientists. Second, there is some systematic variation among American political scientists in how they evaluate scholarly journals. Political scientists differ by subfield specialization and methodological orientation in their familiarity with and evaluations of scholarly journals (Garand and Giles 2003; Garand 2005). However, these differences occur primarily in how scholars rank the elite disciplinary journals, with political scientists shuffling the order of highly ranked journals depending on their subfield and methodological approach. When one considers subfield and methodological differences in evaluations, familiarity, and impact for a broad set of political science journals, one finds that there are more similarities than differences. While scholars who concentrate on American politics may differ from those who study international relations, comparative politics, or political theory in the journals that they tout as being the leading journals in the discipline, there are similarities in evaluations across the broad spectrum of journals (Garand 2005). Perhaps the biggest shortcoming in this literature is its almost exclusive focus on journal evaluations by American political scientists. While we know a fair amount about how American political scientists evaluate scholarly journals, we know little about patterns of journal evaluations among political scientists in other countries. Simply, very little work has been done in terms of comparative, cross-national research on how political scientists evaluate scholarly journals. The most noteworthy exception is Crewe and Norris (1991), who replicate the work of Garand (1990) and Giles, Mizell, and Patterson (1989) for a sample of British political scientists. They find considerable differences in how British and American political scientists evaluate political science journals. There are some journals that emerge on both British and American lists, but British political scientists evaluate several British and/or subfield journals very differently than do American political scientists. This work provides tantalizing evidence of country effects in how scholars evaluate the professional media in political science. Given the globalization of the discipline of political science, it is important to describe and understand how polit696 PS • October 2009
ical scientists in different countries evaluate the scholarly media in which they communicate with other scholars. In this article we build on previous research on subjective journal evaluations and respond to the need for cross-national comparison. We replicate previous work by Giles and Garand (Giles and Wright 1975; Giles, Mizell, and Patterson 1989; Garand 1990, 2005; Garand and Giles 2003) by reporting results based on an online survey in which we ask political scientists to evaluate a wide range of political science journals. Hence our results are directly comparable to those reported in previous research by Giles and Garand and their collaborators. We use both political scientists’ evaluations of scholarly journals and their familiarity with these journals to create a measure of “journal impact” that is comparable to those reported in previous studies. However, in addition to respondents from the United States, we include a sample of respondents from the United Kingdom and Canada, and this permits us to make explicit comparisons among political scientists from these three countries. The result is a broader picture of how political scientists evaluate the major journals through which they communicate the findings of their research. DATA AND METHODS
The present study is based on responses to an Internet survey of political scientists conducted in the United States, Canada, and United Kingdom during 2007. We started with the questionnaire that was the basis for the work of Garand and Giles (2003) on journal evaluations among American political scientists. Garand and Giles asked political scientists about their evaluations of 115 political science journals, as well as a variety of demographic, career, and attitudinal variables. In the present survey we ask respondents to evaluate 92 political science journals, and we adapt other components of the Garand-Giles survey to reflect differences across the three countries. We use English versions of the survey in each country, and in Canada we provide respondents with the option of answering the questionnaire in French. Overall, these data permit us to provide ratings of political science journals on three important dimensions (i.e., evaluation, familiarity, and impact) and to explore the key determinants of these ratings. Sampling Issues Our population of interest is political scientists who are affiliated with political science departments housed in universities that grant the Ph.D. in political science. This stands in some contrast to the work of Garand and Giles, who included a large subsample of faculty from Ph.D.-granting departments but also a subsample of political scientists from departments that do not grant the Ph.D. For the United States, we obtained a list from the American Political Science Association (APSA) of members affiliated with political science departments that offer the Ph.D. in political science. The total number of American political scientists on the list was 3,486; of these, 1,134 provided usable responses, for a response rate of 32.53%. For Canada, where many political scientists are not members of the Canadian Political Science Association (CPSA), we could not rely on CPSA membership lists. Instead, we started with a list of all Ph.D.-granting political science departments in Canada, consulted the Web site for each department to compile an initial faculty list, and followed up by contacting all departments to see
.............................................................................................................................................................................................................................................................
whether our list needed to be updated. Based on this procedure we generated a list of 607 names of faculty at Canadian Ph.D.granting departments. Of these, we received usable responses from 196, for a response rate of 32.3%. For the United Kingdom, the population of interest is defined by the annual Political Studies Association directory. From data supplied by heads of departments, the directory lists all academic staff in political science and cognate departments in the UK, whether or not they are PSA members. It also lists PSA (and British International Studies Association) members in institutions outside of political science departments. As a result, the Directory likely overstates the true population of political scientists in Ph.D.granting departments, making difficult the calculation of a firm response rate. Based on the Directory we identified 1,943 political scientists in Ph.D.-granting departments, of whom 432 responded for a response rate of 22.2%. However, because the denominator likely includes a large number of non-political scientists who are unlikely to respond to our survey, this figure understates the actual response rate. By comparison, about 1,000 political scientists are entered by the universities as research-active political scientists in the 2008 Research Assessment Exercise, so the true unobservable population lies between 1,000 and 1,943. The 22.2% figure is hence a lower boundary on the response rate for the UK sample. Our total sample consists of 1,695 respondents, with 64% from the United States, 11% from Canada, and 24% from the United Kingdom. Survey Administration The present expert survey is the fifth in a series initiated in 1975, with the most recent installment published in 2003 by the two U.S. authors of this article (Giles and Wright 1975; Giles, Mizell, and Patterson 1989; Garand 1990; Garand and Giles 2003). In these works Giles and Garand conducted surveys of political scientists in the U.S., with respondents asked to evaluate the quality of journals on a scale ranging from 0 (poor quality) to 10 (outstanding). Garand (1990) and Garand and Giles (2003) combined data on mean journal evaluations and the proportion of respondents who were familiar with each journal to create a measure of journal impact, measured as: Journal Impact = Journal Evaluation + (Journal Evaluation * Journal Familiarity) This measure has a theoretical range of 0 (poor mean evaluation and no journal familiarity) to 20 (perfect mean evaluation and perfect familiarity). The authors reason that the most important journals in political science are those that are both (1) highly regarded for the quality of the work that they publish and (2) highly visible to the broadest group of political scientists. By combining quality and familiarity measures into a single scale, Garand and Giles created an impact measure that has a high level of face validity and that is fairly highly correlated (r = 0.656) with citationbased measures of journal impact.1 Our survey was administered by the Public Policy Research Laboratory, an academic survey-research center located at Louisiana State University. Respondents from the United States, United Kingdom, and Canada were sent e-mails with a link to the survey, which was tailored to the language and academic customs of each country. After an initial period of receiving responses from our sample, we sent a second reminder e-mail to respondents.
The survey was divided into four sections. First, all respondents received in the e-mail solicitation a cover letter that included a brief description of the project, a confidentiality statement, and a statement relating to human-subjects review by the institutional review board (IRB) at Louisiana State University. Second, we included a series of questions designed to gather descriptive information, including country of origin, highest degree received, age, sex, academic rank, field and subfield interests, and methodological approaches. Third, we included a section with openended questions in which respondents could identify journals (1) to which respondents would submit “a very strong paper on a topic in your area of expertise,” and (2) that respondents “read regularly or otherwise rely on for the best research in your area of expertise.” This provides an alternative way of measuring respondents’ evaluations of political science journals, as developed by Garand and Giles (2003). Finally, we included a section in which we asked respondents to evaluate “journals in terms of the general quality of the articles it publishes.” We used a scale from 0 (poor) to 10 (outstanding) and asked respondents to evaluate each of 92 journals with which they might be familiar. We also asked respondents specifically to indicate if they were familiar with each of these journals, as well as whether or not they have ever published an article in each journal. Journal List The selection of journals for the survey required a balance between (1) including sufficient journals to achieve adequate coverage across journal types, specialty areas, and countries of origin and (2) not including so many journals as to burden survey respondents and reduce the response rate below acceptable levels. This was a difficult task, because there are many more journals of interest to political scientists than there is room in a survey of manageable length. We approached this task in four steps. First, we included all of the journals in the Garand and Giles (2003) survey that were also classified as political science, public administration, or international studies by the Institute for Scientific Information (ISI). Second, given the emphasis of the Garand and Giles list on American journals, we included all of the journals included in Hix’s (2004) UK-originated bibliometric study. This resulted in the inclusion of 84 journals: (1) 45 journals that were in the appropriate ISI categories and included in the studies of both Garand and Giles and Hix; (2) 19 journals that were only included in the Hix study; and (3) 20 journals that were not included in the Hix study but were in the appropriate ISI category and included in the Garand and Giles study. Third, we included all journals that were familiar to at least 20% of the respondents to the Garand and Giles study regardless of their ISI categorization. This step added only four journals to the list.2 Finally, each of the authors was allowed to nominate journals for inclusion but agreement of three of the four authors was required for a nominated journal to be added to the list. Only three journals were added to the list through this procedure.3 This procedure resulted in the exclusion of several new journals that are already well recognized, as well as some other journals with strong reputations but that do not meet the criteria to which we agreed at the outset of the study. In some cases we received inquiries from journal editors or other interested scholars, asking why a given journal was excluded from our study. However, our goal was to establish clear criteria for selecting journals and follow those criteria with only very limited exceptions. We PS • October 2009 697
The Profession: Political Science Journals in Comparative Perspective
.............................................................................................................................................................................................................................................................
did give respondents the opportunity to list additional journals, both at the end of the closed-answer section of the survey and in the open-ended questions about journal submission and journalreading preferences. EMPIRICAL RESULTS
Consistency of Journal Impact, Evaluation, and Familiarity Measures over Time As a starting point, we consider the degree to which there is stability in our measures of journal impact, evaluation, and familiarity from one survey to the next. Garand and Giles (2003) find a considerable amount of consistency in these three measures from the 1989 Giles, Mizell, and Patterson study (Garand 1990) to the 2001 survey conducted by Garand and Giles (2003), and our findings are very similar. We examined the scatterplots and estimated correlation and regression coefficients for the relationships between 2001 and 2007 measures of journal impact, journal evaluations, and journal familiarity for the 69 journals that are included on both the 2001 and 2007 surveys. Since the 2001 survey was based on a sample drawn only from the United States, we include here only the U.S. respondents to our 2007 survey. Based on these scatterplots—which are available in Online Appendix Figures 1–3 4 —as well as the correlation and regression coefficients, it is clear that there is considerable stability over time in the relative positions of journals in their impact, evaluations, and familiarity. The relationship between 2001 and 2007 measures of journal impact is very strong (r = 0.950, b =1.226, t = 25.00), suggesting that how political scientists assess the impact of scholarly journals in 2001 is closely aligned with how they evaluate the impact of scholarly journals in 2007. The same can be said for journal evaluations, where the correlation is slightly smaller but still very high (r = 0.890, b = 1.146, t = 16.00). There is also a very strong relationship between the 2001 and 2007 journal familiarity measures (r = 0.939, b =1.081, t = 22.25). Simply, these results reveal that the highest ( lowest) journals in terms of evaluation, familiarity, and overall impact in 2007 were also the highest ( lowest) impact, evaluation, and familiarity journals in 2001. Journal Impact in Three Countries In Table 1 we report the relative journal impact scores and rankings for respondents from all three countries combined, as well as separately for respondents from the United States, United Kingdom, and Canada. The relative impact score is the impact score for each journal, divided by the impact score for the highest-ranked journal in each relevant comparison group. A more detailed summary of mean journal evaluations, journal familiarity, and journal impact for all respondents is found in Online Appendix 1, and separate detailed country-specific summary tables are found in Online Appendix 2. Several important findings emerge from these results. First, the elite journals among political scientists from all three countries combined are similar to those found in previous studies of political scientists in the United States. This no doubt reflects the fact that almost two-thirds of respondents are from the United States, and we have already documented the continuity in ratings for this group. Once again, the American Political Science Review, American Journal of Political Science, and Journal of Politics emerge as the top three journals in the discipline, with the British Journal of Political Science ranked a close fourth. These are 698 PS • October 2009
general readership journals that are both highly regarded for the quality and rigor of the research that they publish and broadly familiar to a wide range of political scientists. The remainder of the top 10—including International Organization, World Politics, Comparative Politics, Comparative Political Studies, International Studies Quarterly, and the Journal of Conflict Resolution—comprises journals that represent the leading general journals in the fields of comparative politics and international relations. When one moves to the second tier, one finds journals of different stripes, including general journals (e.g., Perspectives on Politics, Political Research Quarterly, PS: Political Science and Politics), broad-based subfield journals (e.g., Foreign Affairs, Political Theory, International Security), and well-regarded specialty journals (e.g., Legislative Studies Quarterly, Electoral Studies). Second, careful readers will note that there is a gradual dropoff in relative impact scores for journals across the distribution. While there seems to be a strong consensus that there are certain journals that are among the elite journals in political science, in many cases the journal impact scores of lower-ranked journals are not very far removed from the scores for high-ranked journals. For instance, 53 journals have relative impact scores that are at least 50% of the impact score for the American Political Science Review, the journal ranked first among the combined respondents from the three countries in this study. As Giles and Garand (2007) suggest, measures of journal impact based on citations tend to have a small number of elite journals accompanied by a rapid drop-off, with a small number of journals being cited a great deal and most journals with small citation counts. However, the more gradual drop-off in relative journal impact suggests that publications in many journals are viewed by political scientists in favorable terms, even if articles in those moderately ranked journals are not cited extensively by other scholars. Third, there is considerable similarity in how political scientists in the United States, United Kingdom, and Canada perceive the impact of scholarly journals in the discipline, though there are also some noteworthy differences. Among the top 10 journals for all respondents combined, nine are found on the American top-10 list, seven are found on the UK list, and eight are found on the Canadian list. For American political scientists, Perspectives on Politics moves into the top 10, replacing International Studies Quarterly, which is still ranked eleventh. Among political scientists from the United Kingdom, three journals in their top 10—European Journal of Political Research (eighth), Political Studies (third), and the European Journal of International Relations (tenth)—are not found in the overall top 10; UK political scientists downgrade the rankings of the Journal of Politics (from third to thirteenth), International Studies Quarterly (from nineth to seventeenth), and the Journal of Conflict Resolution (from tenth to thirty-first). For Canadian political scientists, the International Studies Quarterly and Journal of Conflict Resolution are replaced in the top 10 by the European Journal of Political Research (ranked tenth by Canadian political scientists) and the Canadian Journal of Political Science (ranked seventh). Overall, there is a fair amount of agreement among political scientists in these three countries in terms of the journals that they place in the elite category. One can observe the similarity of journal rankings for political scientists across country by examining the scatterplots of relative journal rankings for each pair of countries. In Figures 1–3 we present scatterplots of relative journal impact for the
.............................................................................................................................................................................................................................................................
Ta b l e 1
Evaluations, Familiarity, and Impact Ratings for Political Scientists from the United States, United Kingdom, and Canada, 92 Selected Journals, 2007 ALL COUNTRIES JOURNAL
Relative Impact
American Political Science Review @*#
1.000
American Journal of Political Science Journal of Politics
UNITED STATES Rank
Relative Impact
1
1.000
0.930
2
0.882
3
British Journal of Political Science
0.872
International Organization
0.845
World Politics Comparative Politics Comparative Political Studies
0.766
8
International Studies Quarterly
0.697
9
Journal of Conflict Resolution
0.672
10
Perspectives on Politics @*#
0.666
11
Legislative Studies Quarterly
0.638
Political Analysis
0.637
Political Theory Foreign Affairs
CANADA Rank
Relative Impact
1
1.000
0.932
2
0.911
3
4
0.820
5
0.805
0.798
6
0.766
0.768
7
0.713
UNITED KINGDOM Rank
Relative Impact
Rank
1
0.969
2
0.900
5
0.910
5
0.832
9
0.771
14
4
0.921
3
1.000
1
5
0.935
2
0.913
4
6
0.885
6
0.846
7
8
0.910
4
0.857
6
0.725
7
0.878
7
0.830
9
0.670
10
0.749
11
0.742
17
0.670
11
0.665
26
0.655
32
0.688
9
0.706
20
0.554
53
12
0.660
12
0.568
54
0.574
48
13
0.646
14
0.594
43
0.612
38
0.635
14
0.597
16
0.705
21
0.715
24
0.632
15
0.585
18
0.744
12
0.717
23
Political Research Quarterly
0.629
16
0.653
13
0.597
42
0.534
61
European Journal of Political Research
0.618
17
0.507
36
0.761
10
0.843
8
PS: Political Science and Politics @*#
0.606
18
0.591
17
0.743
13
0.577
46
Electoral Studies
0.601
19
0.550
22
0.655
28
0.725
21
International Security
0.600
20
0.548
23
0.716
16
0.704
26
Public Opinion Quarterly
0.598
21
0.601
15
0.581
48
0.594
42
Political Studies @*#
0.596
22
0.433
49
0.709
19
0.939
3
Philosophy and Public Affairs
0.585
23
0.520
33
0.669
25
0.727
20
Politics and Society
0.579
24
0.530
29
0.729
14
0.650
33
Journal of Theoretical Politics
0.575
25
0.557
21
0.574
50
0.628
36
Political Behavior
0.565
26
0.565
20
0.571
51
0.566
51
Journal of Democracy
0.564
27
0.515
35
0.683
24
0.648
34
European Journal of International Relations
0.560
28
0.444
47
0.717
15
0.783
10
American Politics Research
0.554
29
0.577
19
0.489
78
0.474
79
Political Science Quarterly
0.554
30
0.534
28
0.614
38
0.591
43
Canadian Journal of Political Science @*#
0.552
31
0.484
39
0.868
8
0.586
44
Public Administration Review
0.548
32
0.534
27
0.620
34
0.546
57
History of Political Thought
0.544
33
0.473
41
0.626
33
0.699
27
Party Politics
0.543
34
0.455
43
0.641
31
0.738
18
Public Choice
0.543
35
0.541
24
0.554
59
0.545
58
Review of International Studies
0.542
36
0.415
55
0.692
23
0.777
12
Journal of Law and Economics
0.540
37
0.540
25
0.584
46
0.517
65
Journal of Peace Research
0.532
38
0.495
37
0.578
49
0.621
37
Political Psychology
0.530
39
0.524
30
0.555
57
0.538
60
West European Politics
0.530
40
0.412
56
0.592
45
0.778
11
Polity
0.528
41
0.523
32
0.582
47
0.507
69
Journal of Common Market Studies
0.527
42
0.404
62
0.617
37
0.767
15 ~continued !
PS • October 2009 699
The Profession: Political Science Journals in Comparative Perspective
.............................................................................................................................................................................................................................................................
Ta b l e 1 (c o n t i n u e d ) ALL COUNTRIES JOURNAL
Relative Impact
UNITED STATES
Rank
Relative Impact
CANADA
UNITED KINGDOM
Rank
Relative Impact
Rank
Relative Impact
Rank
Social Science Quarterly
0.525
43
0.540
26
0.525
69
0.493
74
Governance
0.520
44
0.425
50
0.713
18
0.689
28
Annals of the American Academy
0.519
45
0.483
40
0.645
29
0.572
49
Government and Opposition
0.519
46
0.408
57
0.603
41
0.771
13
Journal of European Public Policy
0.517
47
0.394
66
0.609
40
0.736
19
International Political Science Review
0.516
48
0.438
48
0.714
17
0.633
35
Law and Society Review
0.516
49
0.517
34
0.568
55
0.463
83
International Affairs
0.514
50
0.392
68
0.613
39
0.763
16
Review of International Political Economy
0.514
51
0.418
53
0.695
22
0.671
30
Studies in American Political Development
0.514
52
0.524
31
0.493
77
0.413
87
Millennium
0.509
53
0.398
64
0.664
27
0.721
22
Journal of Policy Analysis and Management
0.500
54
0.493
38
0.539
66
0.457
84
Publius
0.495
55
0.469
42
0.645
30
0.510
67
Public Administration
0.488
56
0.366
80
0.620
35
0.707
25
Journal of Public Policy
0.477
57
0.420
51
0.562
56
0.599
41
Democratization
0.475
58
0.391
69
0.488
79
0.678
29
Review of Politics
0.471
59
0.445
46
0.536
67
0.517
66
Studies in Comp. and Int. Development
0.469
60
0.455
44
0.499
74
0.473
80
Political Communication
0.468
61
0.450
45
0.511
71
0.500
72
Political Geography
0.468
62
0.419
52
0.553
60
0.571
50
European Union Politics
0.461
63
0.391
70
0.500
72
0.611
39
Security Studies
0.454
64
0.405
61
0.544
64
0.558
52
Nations and Nationalism
0.450
65
0.344
83
0.592
44
0.610
40
American Journal of International Law
0.447
66
0.393
67
0.541
65
0.581
45
Political Quarterly
0.445
67
0.337
85
0.549
61
0.656
31
Latin American Politics and Society
0.444
68
0.408
58
0.569
53
0.502
71
Policy Studies Journal
0.442
69
0.416
54
0.544
63
0.478
77
Women and Politics
0.431
70
0.406
60
0.545
62
0.479
76
Journal of Legislative Studies
0.428
71
0.383
72
0.494
76
0.548
55
Review Francaise de Science Politique
0.428
72
0.367
78
0.521
70
0.547
56
Scandinavian Political Studies
0.426
73
0.376
76
0.480
80
0.534
62
Australian Journal of Political Science
0.420
74
0.343
84
0.569
52
0.542
59
Political Science
0.420
75
0.365
81
0.554
58
0.509
68
Conflict Management and Peace Science
0.418
76
0.407
59
0.479
81
0.408
89
Journal of Strategic Studies
0.418
77
0.368
77
0.470
84
0.525
63
Administration and Society
0.416
78
0.401
63
0.469
85
0.421
86
American Review of Public Administration
0.415
79
0.389
71
0.477
82
0.470
81
Europe-Asia Studies (Soviet Studies)
0.414
80
0.334
86
0.497
75
0.574
47
Politische Vierteljahreschrift
0.413
81
0.377
75
0.431
89
0.495
73
Cooperation and Conflict
0.406
82
0.323
89
0.499
73
0.553
54
Policy Sciences
0.406
83
0.366
79
0.535
68
0.464
82
East European Politics and Societies
0.401
84
0.350
82
0.471
83
0.517
64
Public Interest
0.399
85
0.379
74
0.450
88
0.480
75
Presidential Studies Quarterly
0.393
86
0.398
65
0.462
87
0.411
88 ~ continued )
700 PS • October 2009
.............................................................................................................................................................................................................................................................
Ta b l e 1 (c o n t i n u e d ) ALL COUNTRIES JOURNAL
Relative Impact
UNITED STATES Rank
Relative Impact
CANADA
UNITED KINGDOM
Rank
Relative Impact
Rank
Relative Impact
Rank
International Interaction
0.391
87
0.382
73
0.423
90
0.400
90
Post-Soviet Affairs
0.382
88
0.327
87
0.463
86
0.506
70
Canadian Public Administration
0.366
89
0.280
91
0.617
36
0.372
91
Canadian Public Policy
0.363
90
0.269
92
0.629
32
0.327
92
Acta Politica
0.361
91
0.319
90
0.379
91
0.474
78
Rivista Italiana di Scienza Politica
0.356
92
0.324
88
0.374
92
0.444
85
@*# Denotes journals associated with one of the three national political science associations for the countries from which our sample was selected: American Political Science Association ~USA!, Canadian Political Science Association ~Canada!, and Political Studies Association ~UK!.
Figure 1
Scatterplot for Relationship between USA Journal Impact and Canada Journal Impact
countries is quite high (␣ = 0.869), and a principal components factor analysis of these three country-specific scores reveals a single factor (eigenvalue = 2.398, variance explained = 0.799), with all factor loadings in excess of 0.85. Hence we find considerable similarity in journal impact across these three countries, even when there is some different ordering of journals, particularly at the highest levels. Journal Evaluations
r = 0.737
USA-Canada, USA-UK, and UK-Canada pairings, respectively. As one can readily see, there is a strong relationship among the journal impact measures for country pairings, with the correlations ranging from a low of 0.567 (for the United Kingdom and United States) to 0.787 (for the United Kingdom and Canada). There are some clear outliers in these scatterplots. The Canadian Journal of Political Science has a much lower impact score in the United States than in Canada (Figure 1), and Political Studies has a much lower impact score in the United States than in the United Kingdom (Figure 2). In comparing journal impact in the United Kingdom and Canada (Figure 3), we find that Canadian political scientists give much higher scores to the Canadian Journal of Political Science, Canadian Public Policy, and Canadian Public Administration, while political scientists from the United Kingdom give higher impact scores to Political Studies and Democratization than their Canadian counterparts. Despite these outliers, for the most part there is a reasonably strong relationship between journal impact scores among the three countries. In fact, the reliability for an additive scale based on the separate impact scores from these three
We remind the reader here that the impact scores that we described in the foregoing section are based on two components: (1) journal evaluations, based on respondents’ evaluations of 92 journals on a scale ranging from 0 (poor) to 10 (outstanding), and (2) journal familiarity, based on the proportion of respondents who indicate that they are familiar enough with a given journal to evaluate it. The cross-national differences and similarities in journal impact documented in the previous section can be decomposed into evaluation and familiarity components, so it is important to explore country-specific patterns on these two variables to consider the basis for cross-national differences in journal impact. We begin with journal evaluations. In Table 2 we report the mean evaluation scores for the top 30 journals, and in Figure 4 we present a histogram and kernel density plot of the distribution of mean evaluation scores for all 92 journals. A complete table of mean evaluations for all 92 journals can be found in Online Appendix 3. Turning first to Figure 4, we find that the distribution of mean journal evaluations is skewed to the right, with a relatively small number of journals evaluated very highly (i.e., above a mean of 7.0), a small number of journals evaluated as below average (i.e., below 5.0), and most journals falling into the mid-range (between 5.0 and 7.0). Political scientists in these three countries have identified an elite set of journals that are evaluated particularly favorably for the quality of their contributions to the scholarly literature in political science. What journals are evaluated most favorably? In previous studies Garand (1990) and Garand and Giles (2003) have found that some journals that are familiar to only narrow subsets of PS • October 2009 701
The Profession: Political Science Journals in Comparative Perspective
.............................................................................................................................................................................................................................................................
of Political Science (third), Journal of Politics (fourth), and the British Journal of Political SciScatterplot for Relationship between USA Journal ence (sixth)—and the leading subfield journals— Impact and UK Journal Impact International Organization (second), World Politics (fifth), Political Theory (seventh), and Comparative Political Studies (nineth). But the list also includes some journals that are ranked lower in terms of overall impact, especially Philosophy and Public Affairs and Political Analysis; these journals are thought of highly by the relatively small proportion of respondents who are familiar enough to provide an evaluation, but they are somewhat less familiar to political scientists than some other journals on the list. It is particularly interesting to note that the previous evaluation of the American Political Science Review reported by Garand and Giles (2003) was considerably lower than that reported in Table 2. In the 2002 survey, the APSR was ranked seventeenth in terms of its evaluation by American political scientists; for many political scientists this was a shockingly low mean evaluation for what is generally viewed as the flagship jourr = 0.563 nal of the discipline. However, in these results based on our 2007 survey, the APSR is ranked first in terms of its mean evaluation. Perhaps the relatively lower ranking in 2002 reflected critiFigure 3 cisms of the APSR that prevailed at the time Scatterplot for Relationship between Canada Journal among some political scientists, most notably the Impact and UK Journal Impact adherents to the Perestroika movement within the political science discipline (see, for example, Smith 2002). The remainder of the top 30 list includes a mix of journals that are both favorably evaluated and familiar to a large proportion of political scientists (e.g., International Studies Quarterly, Journal of Conflict Resolution), as well as journals that are highly evaluated for the quality of their published work but by relatively narrower readerships (e.g., History of Political Thought, Journal of Policy Analysis and Management, Political Behavior). For some journals, political scientists are both familiar with the journal and evaluate the quality of its contributions very favorably; these journals score highly on impact, evaluation, and familiarity measures. In other cases, a relatively small number of political scientists are familiar with a given journal, but the narrow readership evaluates the quality of scholarly contributions r = 0.782 in favorable terms; here the impact of the journal on the broader political science profession is limited by the small number of political scientists respondents earn surprisingly strong mean evaluations from these who are familiar with the journal. small groups of readers, and the result is that using only mean What about the effects of country on the evaluations of politevaluations overstates the discipline-wide impact of these jourical science journals? Do political scientists from the United States, nals. The results in Table 2 are closer to the conventional wisCanada, and the United Kingdom evaluate political science jourdom, though as before there are also some surprise results. First, nals differently? In Table 3 we report the mean evaluations for the journals among the top 10 are generally those that one would each of the 25 political science journals, ranked by journal impact expect to see in such a list, including the leading general journals— as reported in Table 1. The list provides a mix of general, subfield, American Political Science Review (ranked first), American Journal specialty, and country-specific journals that earn high journal Figure 2
702 PS • October 2009
.............................................................................................................................................................................................................................................................
Ta b l e 2
Political Scientists’ Subjective Evaluations, Top 30 Highest-Ranked Journals, All Countries Combined and Separately for Each Country, 2007 ALL COUNTRIES JOURNAL
Relative Impact
American Political Science Review
7.688
International Organization American Journal of Political Science
UNITED STATES Rank
Relative Impact
CANADA Rank
Relative Impact
UNITED KINGDOM Rank 9
Relative Impact
Rank
7.022
6
1
7.973
1
7.159
7.546
2
7.498
4
8.000
1
7.471
2
7.381
3
7.598
2
6.796
21
6.938
8
Journal of Politics
7.328
4
7.537
3
6.975
16
6.581
25
World Politics
7.186
5
7.161
5
7.522
2
7.121
4
British Journal of Political Science
7.135
6
7.145
6
7.147
10
7.131
3
Philosophy and Public Affairs
7.111
7
6.826
10
7.467
3
7.538
1
Political Theory
7.104
8
7.110
7
7.367
4
7.015
7
Comparative Political Studies
6.944
9
6.953
8
7.287
5
6.759
13
Political Analysis
6.869
10
6.852
9
7.000
15
6.866
10
International Studies Quarterly
6.775
11
6.762
11
7.058
11
6.683
19
History of Political Thought
6.754
12
6.491
14
7.047
12
7.025
5
Comparative Politics
6.747
13
6.659
12
7.197
7
6.753
16
Journal of Conflict Resolution
6.517
14
6.573
13
6.568
34
6.238
32
European Journal of Political Research
6.415
15
6.003
27
7.044
13
6.718
17
Legislative Studies Quarterly
6.384
16
6.424
15
6.395
39
6.108
37
International Security
6.376
17
6.128
21
7.186
8
6.755
15
Journal of European Public Policy
6.358
18
5.566
38
6.761
23
6.756
14
Journal of Policy Analysis and Management
6.328
19
6.403
16
6.500
36
5.586
69
European Journal of International Relations
6.322
20
5.684
33
7.250
6
6.782
12
Public Opinion Quarterly
6.311
21
6.276
19
6.288
48
6.597
24
Journal of Law and Economics
6.307
22
6.269
20
6.879
19
6.229
33
Studies in American Political Development
6.292
23
6.356
17
6.313
46
5.158
84
Public Administration Review
6.272
24
6.313
18
6.911
18
5.835
51
Political Studies
6.268
25
5.452
47
6.636
29
6.874
9
Review of International Studies
6.250
26
5.470
43
7.000
14
6.842
11
Politics and Society
6.184
27
6.063
26
6.747
26
6.178
34
Political Behavior
6.180
28
6.118
22
6.579
33
6.385
28
Review of International Political Economy
6.127
29
5.548
39
6.917
17
6.626
23
Journal of Theoretical Politics
6.114
30
6.108
23
6.370
43
6.085
39
Note: Figures represent the mean evaluation score on a scale from 0 ~poor! to 10 ~outstanding!.
impact rankings. We report the overall mean evaluation for each of these journals, the mean evaluation for respondents from each of the three countries in our study, and the results of an analysis of variance (ANOVA) analysis of journal evaluations. Our ANOVA analysis has the 11-point journal evaluation scale as the dependent variable, with country serving as the independent variable. We find a wide variety of country-specific effects, ranging from strong country effects to null effects. First, the three highestranked journals—the American Political Science Review, American Journal of Political Science, and Journal of Politics—are generalinterest journals that are highly regarded by scholars from all three
countries. However, there is a significant difference in how political scientists from these three countries evaluate these three journals. In each case, the mean evaluation is highest for political scientists from the United States, and the result is an F test signifying that the differences in means for these three countries are statistically significant. Even though these journals are highly regarded for political scientists from all three countries, all three journals are associated with national and regional political science associations from the United States, and this results in a significantly higher evaluation for American political scientists. Second, there are several journals on our list that are associated with a specific country or that have a European identity, PS • October 2009 703
The Profession: Political Science Journals in Comparative Perspective
.............................................................................................................................................................................................................................................................
We are struck by the journals for which systematic differences among political scientists Distribution of Evaluations for 92 Political Science from different countries are not observed. In addiJournals on 11-point Scale, All Respondents tion to the British Journal of Political Science, the differences among political scientists are not statistically significant for World Politics, International Studies Quarterly, Journal of Conflict Resolution, Legislative Studies Quarterly, Political Theory, Political Analysis, Political Research Quarterly, Public Opinion Quarterly, and Journal ofTheoretical Politics. The intellectual and scholarly appeal of these journals is broad based and similar for political scientists from all three countries. Finally, how are journal evaluations among political scientists in one country related to journal evaluations among political scientists in other countries? We inspected the scatterplots of mean journal evaluations for the USA-Canada, USAUK, and UK-Canada pairings, respectively (see Online Appendix Figures 4–6). As is the case for journal impact, there is a strong relationship among the journal evaluation measures for each country pairing. The correlations are all quite high, ranging from 0.692 to 0.815, and there are Note: The curve overlaying the histogram is a kernel density function. relatively few outliers in each of the scatterplots. American political scientists evaluate the American Political Science Review, American Journal of Political Science, and the Journal of Politics more and political scientists typically differ in their evaluations of these highly than what would be predicted by journal evaluations by journals.The European Journal of Political Science and Political Studpolitical scientists from Canada and the United Kingdom, but for ies stand out in this regard. Political scientists from Canada and the most part journal evaluations in one country can be predicted the United Kingdom evaluate both of these journals more highly reasonably well by knowledge of journal evaluations in the other than do American political scientists, and in both cases these diftwo countries. Here again, the reliability for an additive scale based ferences are highly significant. Another journal that falls into this on the separate journal evaluation scores is high (␣ = 0.888), and category but that is not on the top-25 list is the Canadian Journal of a single factor emerges from a principal components factor analPolitical Science (ranked thirtieth in Table 1); in this case Canadian ysis of these three items (eigenvalue = 2.498, variance explained = political scientists evaluate this journal significantly more favor0.833). Across the full range of journals, we find a high degree of ably than political scientists from the United States and United similarity in how political scientists from the United States, CanKingdom. On the other hand, the British Journal of Political Science ada, and the United Kingdom evaluate these 92 scholarly journals. is evaluated equally by political scientists from each of the three Journal Familiarity countries, so it is not inevitable that a country designation differFigure 4
entiates political scientists in their evaluations of a journal. Third, there is a wide range of other journals for which there are significant differences for political scientists across the three countries. Several journals in the fields of comparative politics and international relations have evaluations that differ from one country to the next. International Organization, Comparative Politics, Comparative Political Studies, Foreign Affairs, and International Security all exhibit significant differences, with Canadian political scientists most likely to evaluate these journals favorably. Interestingly, Perspectives on Politics and PS: Political Science and Politics are evaluated more favorably by Canadian political scientists, exceeding the evaluation score given by political scientists from the United Kingdom and, most surprisingly, the United States. Political scientists from Canada and United Kingdom are more likely to evaluate the European Journal of Political Science, Electoral Studies, Political Studies, and Philosophy and Public Affairs more favorably than American political scientists; of these, only Political Studies is evaluated most favorably by political scientists from the United Kingdom. 704 PS • October 2009
In addition to respondents’ evaluations of the quality of articles published in a given journal, the impact of that journal in the political science discipline is also a function of the degree to which political scientists are familiar with the research found in articles published with a journal. In Figure 5 we present the histogram and kernel density plot for the distribution of familiarity scores for all 92 journals in our study, and in Table 4 we report the proportion of respondents who report being familiar with the top 30 journals. A complete table of journal familiarity results for all 92 journals and for all three countries can be found in Online Appendix 4. The histogram for our journal-familiarity variable reveals that the distribution is heavily skewed to the right, with only a few journals exhibiting familiarity levels in excess of 0.50. The lion’s share of journals is familiar to between 20% and 50% of respondents. It appears that a relatively small number of journals have the potential to make a discipline-wide impact, with most journals having a potential impact on selected subgroups of political scientists based on subfield, methodological approach, or other criteria.
.............................................................................................................................................................................................................................................................
Ta b l e 3
Mean Evaluations of Selected Political Science Journals, by Country of Respondent, 2007 JOURNAL
TOTAL
USA
CANADA
UK
F
American Political Science Review
7.688 ~1584!
7.973 ~1070!
7.159 ~170!
7.022 ~325!
27.08***
American Journal of Political Science
7.381 ~1482!
7.598 ~1023!
6.796 ~152!
6.938 ~289!
14.79***
Journal of Politics
7.328 ~1340!
7.537 ~992!
6.975 ~119!
6.581 ~215!
23.30***
British Journal of Political Science
7.135 ~1385!
7.145 ~887!
7.147 ~143!
7.131 ~337!
0.01
International Organization
7.546 ~1127!
7.498 ~759!
8.000 ~113!
7.471 ~242!
3.66**
World Politics
7.186 ~1106!
7.161 ~753!
7.522 ~115!
7.121 ~224!
1.56
Comparative Politics
6.747 ~1176!
6.659 ~754!
7.197 ~137!
6.753 ~267!
4.59**
Comparative Political Studies
6.944 ~1087!
6.953 ~706!
7.287 ~122!
6.759 ~245!
3.88**
International Studies Quarterly
6.775 ~897!
6.762 ~618!
7.058 ~86!
6.683 ~183!
1.15
Journal of Conflict Resolution
6.517 ~904!
6.573 ~667!
6.568 ~74!
6.238 ~151!
1.88
Perspectives on Politics
5.973 ~1115!
5.957 ~899!
6.632 ~87!
5.592 ~120!
5.53***
Political Analysis
6.869 ~642!
6.852 ~535!
7.000 ~33!
6.866 ~67!
0.09
Legislative Studies Quarterly
6.384 ~823!
6.424 ~679!
6.395 ~43!
6.108 ~93!
1.41
Foreign Affairs
5.476 ~1215!
5.309 ~807!
6.112 ~125!
5.632 ~269!
6.75***
Political Theory
7.104 ~557!
7.110 ~356!
7.367 ~60!
7.015 ~135!
0.57
Political Research Quarterly
6.022 ~938!
6.068 ~765!
6.194 ~62!
5.573 ~103!
3.54**
European Journal of Political Research
6.415 ~733!
6.003 ~365!
7.044 ~91!
6.718 ~259!
20.57***
PS: Political Science and Politics
5.096 ~1301!
4.982 ~954!
5.890 ~136!
5.072 ~197!
10.10***
Electoral Studies
6.071 ~799!
5.792 ~547!
6.820 ~61!
6.575 ~179!
21.55***
International Security
6.376 ~676!
6.128 ~452!
7.186 ~70!
6.755 ~147!
9.90***
Public Opinion Quarterly
6.311 ~692!
6.276 ~561!
6.288 ~52!
6.597 ~72!
1.02
Political Studies
6.268 ~701!
5.452 ~279!
6.636 ~88!
6.874 ~318!
40.83***
Philosophy and Public Affairs
7.111 ~378!
6.826 ~224!
7.467 ~45!
7.538 ~106!
4.22**
Politics and Society
6.184 ~667!
6.063 ~416!
6.747 ~91!
6.178 ~152!
4.69***
Journal of Theoretical Politics
6.114 ~676!
6.108 ~481!
6.370 ~46!
6.085 ~142!
0.48
Note: Figures represent the mean evaluation score on a scale from 0 ~poor! to 10 ~outstanding!. Figures in parentheses are the number of respondents used to calculate the mean evaluation score. *** prob~ F ! < .01 ** prob~ F ! < .05 * prob~ F ! < .10
PS • October 2009 705
The Profession: Political Science Journals in Comparative Perspective
.............................................................................................................................................................................................................................................................
These results reveal considerable differences in journal familiarity for political scientists from Distribution of Levels of Familiarity for 92 Political the United States, United Kingdom, and CanScience Journals, All Respondents ada. Of the 25 top-impact journals, 19 exhibit a significant difference (at the .05 level ) in familiarity levels across countries. This includes the “big three” journals (i.e., American Political Science Review, American Journal of Political Science, and the Journal of Politics), other general journals (e.g., Perspectives on Politics, Political Research Quarterly), various high-impact subfield journals (e.g., International Organization, International Studies Quarterly), and specialty journals (e.g., Legislative Studies Quarterly, Political Analysis). The only high-impact journals that are equally familiar to political scientists from all three countries are the British Journal of Political Science, Comparative Politics, Comparative Political Studies, PoliticalTheory, International Security, and Philosophy and Public Affairs. Three of these journals are explicitly cross-national in focus, and two are in the field of political theory and philosophy. It is noteworthy that among the 19 journals for which there is a significant difference in familNote: The curve overlaying the histogram is a kernel density function. iarity for political scientists from different countries, the level of familiarity is highest among American political scientists for 16 of these journals. The only exceptions are the European JourAmong the 30 journals with which the respondents are most nal of Political Science and Political Studies (both of which are more familiar, we find that there are only 15 journals for which a majorfamiliar to political scientists from the UK and Canada) and Polity of respondents indicate familiarity. The American Political Sciitics and Society (which is more familiar to Canadian political science Review is the journal with the highest level of familiarity, entists). This may partially be explained by a general USA effect, with 93.4% of respondents indicating that they are familiar with whereby the overall level of familiarity is higher among American the flagship journal of the discipline. Four other journals are familpolitical scientists than those from other countries. Across all 92 iar to at least 75% of political scientists—the American Journal of journals, the mean level of familiarity is 34.6%, with political sciPolitical Science (87.5%), British Journal of Political Science (81.6%), entists from the United States showing the highest familiarity Journal of Politics (79.1%), and PS: Political Science and Politics (35.8%), followed by those from Canada (34.4%) and the United (76.7%). These are the journals with the potential to have the stronKingdom (32.1%). Even among the top 25 journals, the overall gest overall impact on the discipline; in fact, four of these five mean familiarity is 47.5%, with political scientists from the United journals are also among the most favorably evaluated journals, so States (49.9%) exhibiting higher familiarity than those from Canthis helps to explain their strong overall impact (Table 1). Among ada (44.5%) and the United Kingdom (43.2%). This USA effect is the journals familiar to over 50% of respondents, there is a mix of likely due to the fact that a large proportion of high-impact jourgeneral interest journals (e.g., Perspectives on Politics, Political nals originates or is published in the United States. Research Quarterly, Political Science Quarterly) and major subfield We also consider the interrelationships for journal-familiarity journals (e.g., Comparative Politics, International Organization, Comlevels among political scientists from the United States, Canada, parative Political Studies). Finally, the remainder of the list comand the United Kingdom. We examined the scatterplots for the prises journals familiar to over 40% of respondents, including relationships between journal familiarity levels for country pairs specialty journals (e.g., Legislative Studies Quarterly, Electoral Stud(see Online Appendix Figures 7–9), and it is here that we begin to ies, Public Choice), a few general interest journals (e.g., Polity, Social observe a slight deterioration in the strength of the relationships Science Quarterly), and journals with an audience targeted to a between countries. The correlations for journal familiarity are specific country (e.g., Canadian Journal of Political Science, Political lower than those for journal impact and journal evaluations, rangStudies). Beyond these journals, most journals are familiar to less ing from a low of 0.453 (for the USA-UK relationship) to a high of than 40% of political scientists in these three countries. 0.734 (for the United Kingdom and Canada). Clearly, political sciHow do levels of journal familiarity vary for political scientists entists from the United States and the United Kingdom do not across the three countries in our study? In Table 5 we report the share the same levels of familiarity for many scholarly journals, proportion familiar for each of the top 25 high-impact political while political scientists from the United Kingdom and Canada science journals (as reported in Table 1), both in total and broken tend to exhibit similar levels of journal familiarity. down by country. Furthermore, we include in Table 5 the likeliFinally, even though the cross-national interrelationships hood ratio (LR) 2 test of the cross-national difference in the proinvolving journal familiarity are not as strong as those involving portion of respondents who are familiar with each journal. journal evaluations and journal impact, the similarity in journal Figure 5
706 PS • October 2009
.............................................................................................................................................................................................................................................................
Ta b l e 4
Political Scientists’ Journal Familiarity, Top 30 Highest-Ranked Journals, All Countries Combined and Separately for Each Country, 2007 ALL COUNTRIES JOURNAL
Proportion Familiar
UNITED STATES
Rank
Proportion Familiar
CANADA
Rank
Proportion Familiar
UNITED KINGDOM Rank
Proportion Familiar
Rank
American Political Science Review
0.935
1
0.984
1
0.950
2
0.806
2
American Journal of Political Science
0.874
2
0.941
2
0.849
3
0.717
5
British Journal of Political Science
0.817
3
0.816
6
0.799
4
0.836
1
Journal of Politics
0.791
4
0.913
3
0.665
10
0.533
16
PS: Political Science and Politics
0.768
5
0.878
4
0.760
6
0.489
19
Foreign Affairs
0.717
6
0.742
7
0.698
8
0.667
6
Comparative Politics
0.694
7
0.694
10
0.765
5
0.663
7
International Organization
0.665
8
0.698
9
0.631
12
0.600
10
Perspectives on Politics
0.658
9
0.827
5
0.486
20
0.298
46
World Politics
0.653
10
0.693
11
0.642
11
0.556
15
Comparative Political Studies
0.641
11
0.649
12
0.682
9
0.608
9
Political Research Quarterly
0.553
12
0.704
8
0.346
36
0.256
49
Journal of Conflict Resolution
0.533
13
0.614
14
0.413
28
0.375
33
Political Science Quarterly
0.532
14
0.614
15
0.402
29
0.382
31
International Studies Quarterly
0.529
15
0.569
18
0.480
21
0.454
24
Legislative Studies Quarterly
0.486
16
0.625
13
0.240
60
0.231
53
Electoral Studies
0.471
17
0.503
23
0.341
37
0.444
26
American Politics Research
0.470
18
0.611
16
0.263
50
0.191
61
Annals of the American Academy
0.467
19
0.516
21
0.458
22
0.335
38
Journal of Democracy
0.458
20
0.473
26
0.497
18
0.404
28
International Political Science Review
0.457
21
0.406
31
0.721
7
0.479
23
Polity
0.452
22
0.542
19
0.374
34
0.253
50
Canadian Journal of Political Science
0.445
23
0.383
32
0.955
1
0.395
29
Social Science Quarterly
0.443
24
0.593
17
0.285
47
0.127
73
European Journal of Political Research
0.432
25
0.336
40
0.508
16
0.643
8
Public Choice
0.424
26
0.520
20
0.324
42
0.223
57
Government and Opposition
0.416
27
0.288
49
0.458
23
0.732
4
Political Studies
0.414
28
0.257
53
0.492
19
0.789
3
Governance
0.412
29
0.367
35
0.542
14
0.481
21
Public Opinion Quarterly
0.408
30
0.516
22
0.291
45
0.179
65
Note: Figures represent the proportion of respondents who are sufficiently familiar with a given journal to provide an evaluation.
familiarity is strong enough that the familiarity variables for the United States, Canada, and the United Kingdom fit together well into a scale. The reliability for an additive scale based on the separate journal-familiarity scores is lower than that for journal impact and evaluation measures, but is still quite high (␣ = 0.823). Moreover, a single factor emerges from a principal components factor analysis of these three items (eigenvalue = 2.235, variance explained = 0.745). Although the interrelationships between journal familiarity levels for the United States, Canada, and the United Kingdom are not as strong as those observed for journal impact and journal evaluation, there remains a fairly high level of simi-
larity in the extent to which political scientists from these three countries are familiar with scholarly journals. The Relationship between Journal Evaluation and Journal Familiarity How closely are journal evaluation and journal familiarity related to one another? Are journals that receive a favorable evaluation from political scientists also the most familiar to political scientists? In Figure 6 we present a scatterplot of the relationship between these two variables for our full sample. The relationship is moderately strong for the combined sample (r = 0.553), and the PS • October 2009 707
The Profession: Political Science Journals in Comparative Perspective
.............................................................................................................................................................................................................................................................
Ta b l e 5
Familiarity with Selected Political Science Journals, by Country of Respondent, 2007 JOURNAL American Political Science Review American Journal of Political Science Journal of Politics British Journal of Political Science International Organization World Politics Comparative Politics Comparative Political Studies International Studies Quarterly Journal of Conflict Resolution Perspectives on Politics Political Analysis Legislative Studies Quarterly Foreign Affairs Political Theory Political Research Quarterly European Journal of Political Research PS: Political Science and Politics Electoral Studies International Security Public Opinion Quarterly Political Studies Philosophy and Public Affairs Politics and Society Journal of Theoretical Politics Total number of observations
TOTAL
USA
CANADA
UK
LR 2
0.935 ~1,584!
0.984 ~1,070!
0.950 ~170!
0.806 ~325!
136.27*** 127.45***
0.874
0.941
0.849
0.717
~1,482!
~1,023!
~152!
~289!
0.791
0.913
0.665
0.533
~1,340!
~992!
~119!
~215!
0.817
0.816
0.799
0.836
~1,385!
~887!
~143!
~337!
0.665
0.698
0.631
0.600
~1,127!
~759!
~113!
~242!
0.653
0.693
0.642
0.556
~1,106!
~753!
~115!
~224!
0.694
0.694
0.765
0.662
~1,176!
~754!
~137!
~267!
0.641
0.649
0.682
0.608
~1,087!
~706!
~122!
~245!
0.529
0.569
0.480
0.454
~897!
~618!
~86!
~183!
0.533
0.614
0.413
0.375
~904!
~667!
~74!
~151!
0.658
0.827
0.486
0.298
~1,115!
~899!
~87!
~120!
0.379
0.492
0.184
0.166
~642!
~535!
~33!
~67!
0.486
0.625
0.240
0.231
~823!
~679!
~43!
~93!
0.717
0.742
0.698
0.667
~1,215!
~807!
~125!
~269!
0.329
0.328
0.335
0.335
~557!
~356!
~60!
~135!
0.553
0.704
0.346
0.256
~938!
~765!
~62!
~103!
0.432
0.336
0.508
0.643
~733!
~365!
~91!
~259!
0.768
0.878
0.760
0.489
~1301!
~954!
~136!
~197!
0.471
0.503
0.341
0.444
~799!
~547!
~61!
~179!
0.399
0.416
0.391
0.365
~676!
~452!
~70!
~147!
0.408
0.516
0.291
0.179
~692!
~561!
~52!
~72!
0.414
0.257
0.492
0.789
~701!
~279!
~88!
~318!
0.223
0.206
0.251
0.263
~378!
~224!
~45!
~106!
0.394
0.382
0.508
0.377
~667!
~416!
~91!
~152!
0.399
0.443
0.257
0.352
~676!
~481!
~46!
~142!
1,695
1,087
179
403
272.63*** 2.35 15.09*** 24.43*** 5.73* 2.68 18.93*** 80.00*** 402.59*** 177.50*** 244.32*** 9.73*** 0.12 288.07*** 122.52*** 233.37*** 16.75*** 3.74 162.23*** 361.30*** 5.56* 7.71** 30.05***
Note: Figures represent the proportion of respondents who are sufficient familiar with a journal to provide an evaluation. Figures in parentheses are the number of respondents used to calculate the mean evaluation score. ***prob~ F ! < .01 **prob~ F ! < .05 *prob~ F ! < .10
708 PS • October 2009
.............................................................................................................................................................................................................................................................
up to three journals to which they would like to submit their best work. In Table 6 we report the Scatterplot for Relationship between Journal Evaluation number of first, second, and third references, as and Journal Familiarity well as the number of total mentions. We also report the number of “ratings points” based on three points for a first reference, two points for a second reference, and one point for a third reference. As one can see from Table 6, there is again a clear-cut ranking of political science journals, with many of the highly ranked journals on this list also found among the elite journals from the close-ended items reported in Table 1. The top five journals to which political scientists prefer to send their best work are the American Political Science Review, American Journal of Political Science, Journal of Politics, International Organization, and World Politics, followed by the International Studies Quarterly, British Journal of Political Science, Comparative Politics, Comparative Political Studies, and Political Theory. Not surprisingly, the journals to which political scientists prefer to send their best work are also the journals that earn high impact ratings in the closer = 0.554 ended items. When political scientists are thinking about where to send their best work, correlations calculated separately for political scientists from each they consider the leading general journals and broad-based subcountry range from 0.376 (for Canada) to 0.624 (for the United field journals. States). (See Online Appendix Figures 10–12.) Overall, it appears Two other aspects of this table warrant attention. First, we that the journals that are most favorably evaluated are typically point out that political scientists vary considerably in how they the ones that are most familiar to political scientists in the United prioritize political science journals as possible outlets of their work. States, Canada, and the United Kingdom. Respondents mentioned 201 different journals as their first choice There are some important outliers, however. Some journals to submit their best work, with 277 different journals mentioned are well known to political scientists but have only moderate evalas their second choice and 323 different journals mentioned third. uations; PS: Political Science and Politics, Foreign Affairs, PerspecMany of these journals are mentioned only once or twice, but the tives on Politics, Presidential Studies Quarterly, and Political Science fact remains that political scientists see a wide variety of possible are journals with mean evaluations that are below what one would outlets for their best research papers. predict based on their level of familiarity to political scientists. Second, we note the dominant position of the American PolitThese journals earn higher impact scores based on their wideical Science Review, American Journal of Political Science, and (to a spread familiarity among political scientists, rather than on politlesser extent) the Journal of Politics on this list. The second-place ical scientists’ evaluations of the importance and quality of journal, the American Journal of Political Science, has only 54.1% scholarly work that they publish. On the other hand, some jourof the rating points as the American Political Science Review, and nals are familiar only to moderate numbers of political scientists the Journal of Politics has only 34.7% of the rating value of the but have mean evaluations that are above what would be preAPSR. The journals International Organization and World Politics dicted by their level of familiarity; these journals include Political have 22.3% and 21.0% of the rating value of the APSR, but no Analysis, Political Theory, History of Political Thought, and Philosother journal approaches 20% of the rating points of the APSR. ophy and Public Affairs. Journals in this category are known to When political scientists think about the highest-priority outlets more specialized audiences who give them high marks for the for their best work, the American Political Science Review, Ameriquality of the work that they publish. can Journal of Political Science, Journal of Politics, International Organization, and World Politics dominate the list. In fact, these five Preferred Journal Submissions journals represent 55% (886 of 1,597) of the first-choice journals In addition to closed-ended questions, Garand and Giles (2003) to which scholars prefer to send their high-quality work, and the report the results from open-ended questions in which they ask top 20 journals represent 75.5% (1,205 of 1,597) of the most prerespondents to which journal they would submit a “very strong ferred outlets for their best work. While political scientists cite paper” that they had written in their field of expertise. This quesquite a few different journals as journals to which they prefer to tion is designed to give respondents an alternative way of thinksubmit high-quality manuscripts, the fact is that there are a small ing about journal evaluations by eliciting from them information number of journals that are cited by a high percentage of politiabout how they prioritize the journals in which they would like to cal scientists. see their own research published. We include a similar item in our Political scientists from our three countries exhibit some inter2007 survey, and we gave respondents the opportunity to indicate esting differences in terms of the journals to which they prefer to Figure 6
PS • October 2009 709
The Profession: Political Science Journals in Comparative Perspective
.............................................................................................................................................................................................................................................................
Ta b l e 6
Political Scientists’ Preferences for Journal Submissions of High-Quality Manuscript, All Countries Combined, 2007 JOURNAL
1ST
2ND
3RD
TOTAL MENTIONS
RATING POINTS
RELATIVE RATING
1. American Political Science Review
595
80
40
715
1985
1.000
2. American Journal of Political Science
109
343
71
523
1084
0.546
3. Journal of Politics
46
142
265
453
687
0.346
4. International Organization
89
72
42
203
453
0.228
5. World Politics
87
58
47
192
424
0.214
6. International Studies Quarterly
35
54
53
142
266
0.134
7. British Journal of Political Science
33
44
52
129
239
0.120
8. Comparative Politics
30
49
37
116
225
0.113
9. Comparative Political Studies
18
46
69
133
215
0.108
10. Political Theory
43
30
18
91
207
0.104
11. Political Studies
29
33
24
86
177
0.089
12. European Journal of International Relations
17
24
18
59
117
0.059
13. Review of International Studies
18
19
25
62
117
0.059
14. International Security
28
10
11
49
115
0.058
15. Journal of Common Market Studies
20
10
8
38
88
0.044
16. Journal of Political Philosophy
12
16
19
47
87
0.044
17. Public Administration Review
19
9
7
35
82
0.041
9
19
12
40
77
0.039
18. European Journal of Political Research 19. Ethics
14
13
6
33
74
0.037
20. Perspectives on Politics
11
12
16
39
73
0.037
203
282
328
Total number of journals receiving one or more mentions
Note: The entries represent the number of respondents who report the journal as their first, second, or third preferences for submission of a high-quality manuscript. Rating points is based on giving three points for a first rating, two points for a second rating, and one point for a third rating. Relative rating is the rating points for each journal as a share of the rating points for the American Political Science Review.
send their best work. In Table 7 we report the top five journalsubmission preferences for political scientists from the United States, United Kingdom, and Canada. As one can see, the ranking for political scientists from the United States closely tracks the overall impact ranking presented in Table 1. American political scientists prefer to send their best work to the American Political Science Review, American Journal of Political Science, Journal of Politics, International Organization, and World Politics; the only journal missing from those with the highest impact (Table 1) is the British Journal of Political Science. For political scientists from the United Kingdom and Canada, there are some differences. Canadian political scientists drop from their top five the Journal of Politics, add the Canadian Journal of Political Science, and shuffle the order for other journals, though the American Political Science Review is also the journal to which Canadian political scientists would prefer to send their best work. Political scientists from the United Kingdom are the outliers here. The American Political Science Review and International Organization are ranked third and fourth as preferred scholarly outlets. However, Political Studies and the British Journal of Political Science earn the top two positions, and the Review of International Studies captures the fifth position. Overall, it appears that political scientists from Canada and the United States have similar preferences about where they should 710 PS • October 2009
send their best work, but political scientists from the United Kingdom have less overlap with their colleagues from across the Atlantic. Preferred Reading Sources An alternative way of determining political scientists’ journal priorities is to ask respondents to identify the journals that they “read regularly or otherwise rely on for the best research” in their areas of scholarly interest. We included a question in our 2007 survey to capture these journal-reading priorities. In Table 8 we report the results for this item, coding up to five responses for each survey respondent. We also report the number of total mentions, as well as the number of rating points based on five points for a first reference, four points for a second, reference, and so on. Here again, we see a fairly clear pattern of journal ratings. The ranking of the first five journals on this list—the American Political Science Review, American Journal of Political Science, Journal of Politics, International Organization, and World Politics—is identical to the ranking for the journal-submission preferences, and the second five journals are the same, albeit with a slightly different ranking. There is clearly a strong similarity in the results for journal-submission and journal-reading preferences. Indeed, for
.............................................................................................................................................................................................................................................................
Ta b l e 7
Political Scientists’ Preferences for Journal Submissions of High-Quality Manuscript, Top Five by Country, 2007 JOURNAL
1ST
2ND
3RD
TOTAL MENTIONS
RATING POINTS
RELATIVE RATING
United States 1. American Political Science Review
523
73
37
633
1,752
1.000
2. American Journal of Political Science
92
310
68
470
964
0.550
3. Journal of Politics
42
132
249
423
639
0.365
4. International Organization
71
41
39
151
334
0.191
5. World Politics
51
55
34
140
297
0.170
Canada 1. American Political Science Review
30
5
0
35
100
2. International Organization
17
8
4
29
71
3. Canadian Journal of Political Science
13
10
8
31
67
0.670
4. American Journal of Political Science
7
15
1
23
52
0.520
5. World Politics
6
8
2
16
36
0.360
1.000 0.710
United Kingdom 1. Political Studies
27
30
21
78
162
1.000
2. British Journal of Political Science
30
21
17
68
149
0.920
3. American Political Science Review
38
2
2
42
120
0.741
4. International Organization
21
9
3
33
85
0.525
5. Review of International Studies
12
13
19
44
81
0.500
Note: The entries represent the number of respondents who report the journal as their first, second, or third preferences for submission of a high-quality manuscript. Rating points is based on giving three points for a first rating, two points for a second rating, and one point for a third rating. Relative rating is the rating points for each journal as a share of the rating points for top-rated journal in each country ~i.e., the American Political Science Review for the United States and Canada, and Political Studies for the United Kingdom!.
the 64 journals with at least five references on each of the journalsubmission and journal-reading items, the correlation is almost perfect (r = 0.985). Once again we find that political scientists cite a large number of journals for their first, second, third, fourth, or fifth preferences for journal reading. Political scientists rely on many different journals for the best work in their fields of study. However, many of these journals are cited by only one or a few political scientists. As is the case for journal-submission preferences, there is a small group of journals that dominate the list of journal-reading preferences, though there is a wider distribution of journals that political scientists read than to which they submit their work. We find that the top five journals make up 41.7% (662 of 1,587) of the first preferences, and the top 20 journals make up 62.0% (984 of 1,587) of the first preferences. There is also a bit smaller drop-off as one moves from the American Political Science Review to lower-ranked journals. The American Journal of Political Science has 68.1% of the rating value of the APSR, and the Journal of Politics has 47.5% of the APSR rating value. Other journals among the top 20 journals have a journal-reading rating value that is higher than the journal-submission rating value reported in Table 6. The bottom line is that political scientists spread their journal-reading preferences around just a bit more than their journal-submission preferences, though there is a discernible rank order of journals in terms of political scientists’ reading preferences. Political scientists are somewhat more “elitist” in terms of where they would submit their best manuscripts than in where they read for the best work in their fields of study.
Finally, in Table 9 we report the top five journal-reading preferences for political scientists from the United States, United Kingdom, and Canada, and once again we find that political scientists differ across country in the journal-reading preferences. The ranking for scholars from the United States is nearly identical for journal-submission and journal-reading preferences; the only difference is that World Politics and International Organization switch positions in Table 9. Canadian political scientists retain the same top five journals but shuffle the positions of the top three; the Canadian Journal of Political Science is the most read journal by Canadian political scientists, followed by the American Political Science Review and International Organization. Finally, for political scientists from the United Kingdom, the Journal of Common Market Studies replaces the International Organization in the fourth position, with the other four journals retaining their lofty rankings. Here again, political scientists from the United Kingdom are outliers, sharing only two of their five top journals in terms of reading preferences with their American and Canadian counterparts. A Multivariate Analysis of Journal Evaluations What explains variation in individuals’ evaluations of political science journals? One of the questions left unresolved in this analysis is whether country differences (or similarities) are really due to the effects of other variables. For instance, Blais et al. (2008) find that there are systematic differences among political scientists from different countries in terms of the methodological approach that they adopt for their research and professional work. Garand and Giles (2003) find that methodological approach is PS • October 2009 711
The Profession: Political Science Journals in Comparative Perspective
.............................................................................................................................................................................................................................................................
Ta b l e 8
Political Scientists’ Preferences for Journal Reading, All Countries Combined, 2007 JOURNAL 1. American Political Science Review
1ST
2ND
3RD
4TH
5TH
TOTAL MENTIONS
RATING POINTS
RELATIVE RATING
399
98
81
74
59
711
2,837
1.000
91
273
87
48
37
536
1,941
0.684
3. Journal of Politics
52
95
191
45
42
425
1,345
0.474
4. International Organization
85
54
42
17
29
227
830
0.293
2. American Journal of Political Science
5. World Politics
70
57
47
21
22
217
783
0.276
6. International Studies Quarterly
48
45
38
39
25
195
637
0.225
7. Comparative Political Studies
26
41
51
59
29
206
594
0.209
8. British Journal of Political Science
19
27
58
49
35
188
510
0.180
9. Comparative Politics
24
34
49
22
20
149
467
0.165
10. Political Theory
46
23
16
14
11
110
409
0.144
11. Political Studies
29
16
27
30
18
120
368
0.130
6
10
28
61
41
146
317
0.112
12. Political Research Quarterly 13. International Security
31
17
14
19
11
92
314
0.111
14. Legislative Studies Quarterly
16
24
10
41
17
108
305
0.108
15. European Journal of International Relations
21
20
20
15
11
87
286
0.101
16. Journal of Conflict Resolution
16
22
19
18
12
87
273
0.096
17. Review of International Studies
14
23
20
11
14
82
258
0.091
18. Perspectives on Politics
11
15
27
15
15
83
241
0.085
19. Journal of Common Market Studies
17
18
12
10
7
64
220
0.078
20. Journal of Political Philosophy
10
17
18
13
2
60
200
0.071
250
309
337
349
337
Total number of journals receiving one or more mentions
Note: The entries represent the number of respondents who report the journal as their first, second, or third preferences as one that they “read regularly or otherwise rely on for the best research” in their areas. “Rating points” is based on giving five points for a first rating, four points for a second rating, three points for a third rating, and so on. “Relative rating” is the rating points for each journal as a share of the rating points for the American Political Science Review.
related to the evaluations that political scientists give to some journals, particularly those that adopt a clear methodological identity. It is possible that country differences in journal evaluations may actually reflect the distribution of methodological preferences among political scientists from those countries. Other confounding effects may be possible as well, so it is important to explore the influence of country on journal evaluations, controlling for the effects of other variables. We developed and tested a regression model for a small, select group of political science journals. Obviously, we cannot report regression results for all 92 journals, so we selected journals for inclusion in this analysis to represent a mix of general, subfield, specialty, and country-specific journals. Our models include several sets of independent variables: (1) subfield variables, representing political scientists’ subfield designations, with comparative politics the excluded category; (2) methodological-approach variables, including quantitative methods, qualitative methods, and formal theory, with normative theory the excluded comparison category; (3) demographic attributes of age and gender; and (4) country variables, represented by separate binary variables for the United Kingdom and Canada and with the United States the excluded category. This model permits us to estimate the effects of country on journal evaluations, controlling for other variables found by Garand and Giles (2003) to be related to general journal evaluations. 712 PS • October 2009
In Table 10 we report the results for our regression models for 14 journals, including the American Political Science Review, American Journal of Political Science, Journal of Politics, British Journal of Political Science, International Organization, World Politics, Comparative Politics, Comparative Political Studies, International Studies Quarterly, Journal of Conflict Resolution, Perspectives on Politics, Political Theory, Canadian Journal of Political Science, and Political Studies. Turning first to the country-specific variables, we find that only 25% (seven of 28) of the country coefficients are statistically significant at conventional levels. Hence there is relatively little evidence that political scientists from the United States, Canada, and the United Kingdom differ in their evaluations of these political science journals, once we control for the effects of other variables in our model. British political scientists are slightly more favorably oriented to the American Journal of Political Science and, as one might expect, Political Studies, and they are less favorable toward the Journal of Politics, World Politics, Comparative Political Studies, Perspectives on Politics, Political Theory, and the Canadian Journal of Political Science. Canadian political scientists do not differ from American political scientists in their evaluations of any of our 14 journals, though the Canada coefficient for the Political Studies is positive and significant at a more relaxed .10 level of significance. These results suggest that any simple differences that are observed in how political scientists from the United States, Canada, and the United Kingdom evaluate scholarly journals are
.............................................................................................................................................................................................................................................................
Ta b l e 9
Political Scientists’ Preferences for Journal Reading, Top Five by Country, 2007 JOURNAL
1ST
2ND
3RD
4TH
5TH
TOTAL MENTIONS
RATING POINTS
RELATIVE RATING
United States 1. American Political Science Review
351
82
64
60
48
605
2,443
1.000
2. American Journal of Political Science
80
244
76
38
30
468
1,710
0.700
3. Journal of Politics
49
85
181
37
35
387
1,237
0.506
4. World Politics
58
42
34
16
16
166
608
0.249
5. International Organization
51
39
36
11
20
157
561
0.230
Canada 1. Canadian Journal of Political Science
20
10
8
7
8
53
186
1.000
2. American Political Science Review
20
7
10
5
6
48
174
0.935
3. International Organization
19
9
1
1
3
33
139
0.747
4. American Journal of Political Science
6
13
3
3
2
27
99
0.532
5. World Politics
6
7
5
4
3
25
84
0.452
United Kingdom 1. Political Studies
27
14
22
21
17
101
316
1.000
2. British Journal of Political Science
14
12
19
12
7
64
206
0.652
3. American Political Science Review
23
9
7
9
5
53
195
0.617
4. Journal of Common Market Studies
15
15
9
8
6
53
184
0.582
5. Review of International Studies
11
18
11
6
9
55
181
0.573
Note: The entries represent the number of respondents who report the journal as their first, second, or third preferences as one that they “read regularly or otherwise rely on for the best research” in their areas. “Rating points” is based on giving five points for a first rating, four points for a second rating, three points for a third rating, and so on. Relative rating is the rating points for each journal as a share of the rating points for top-rated journal in each country ~i.e., the American Political Science Review for the United States, Political Studies for the United Kingdom, and the Canadian Journal of Political Science for Canada!.
due primarily to the effects of other variables, such as subfield or methodological orientation. We observe quite a few significant coefficients for the subfield variables in our models, though on balance only 28.6% (48 of 168) of the subfield coefficients achieve statistical significance at the .05 level. Scholars who list international politics as a subfield are more favorable than comparative politics scholars (who comprise the excluded comparison group) in how they evaluate International Organization, International Studies Quarterly, and the Journal of Conflict Resolution, three international relations journals that are among the top 10 journals in terms of overall impact; on the other hand, they are less favorable in their evaluations of the Journal of Politics, the British Journal of Political Science, and World Politics. We find that American politics scholars tend to evaluate the Journal of Politics in favorable terms but are significantly less positive in their evaluations of International Organization and Comparative Political Studies. Scholars who list Canadian politics and British politics as one of their subfields of interest are more likely to evaluate the Canadian Journal of Political Science and Political Studies in positive terms, and scholars specializing in Canadian politics give more positive evaluations to the British Journal of Political Science, Comparative Politics, and the International Studies Quarterly. Moreover, area studies and comparative politics scholars differ in their evaluations of some journals; specifically, areastudies scholars are significantly less favorable in their evaluations of the American Political Science Review but are more favorable in their evaluations of World Politics, Comparative Politics, and Comparative Political Studies. Finally, political theorists exhibit quite a few significant differences from comparative politics scholars; theorists give less favorable evaluations to the American Journal of Polit-
ical Science, International Organization, Comparative Political Studies, and Journal of Conflict Resolution, but they are more favorably oriented toward Perspectives on Politics, Political Theory, the Canadian Journal of Political Science, and Political Studies. Of the four clusters of independent variables in our models, our methodological-approach variables have the most consistent effects on journal evaluations, with 50% (21 of 42) of the coefficients surpassing standard levels of statistical significance. Further, at least one methodological-approach variable achieves statistical significance for 12 of the 14 journal models in Table 10. There are some interesting patterns of coefficients for our methodological-approach variables, and these coefficients can help to map the methodological profile of our various journals. We find that quantitative scholars give systematically more favorable evaluations to the American Political Science Review, American Journal of Political Science, Journal of Politics, and British Journal of Political Science than do normative theorists, who comprise the excluded (comparison) group; these general-interest journals are significantly more positively evaluated by scholars with some quantitative orientation than other scholars. Qualitative scholars differ significantly from normative theorists in how they evaluate several journals. Scholars adopting a qualitative research orientation are significantly less favorable in their evaluations of the American Political Science Review, American Journal of Political Science, Journal of Politics, British Journal of Political Science, and the Journal of Conflict Resolution; each of these journals publishes quite a bit of quantitative research, and this emerges in how qualitative scholars evaluate these journals. On the other hand, qualitative scholars give more favorable evaluations to World Politics, Comparative Politics, and Comparative Political Studies, at least PS • October 2009 713
The Profession: Political Science Journals in Comparative Perspective
.............................................................................................................................................................................................................................................................
Ta b l e 1 0
OLS Regression Estimates for Models of Journal Evaluations, Selected Journals, All Countries Combined, 2007 APSR
Intercept
AJPS
JOP
BJPS
b
t
b
t
b
t
b
t
7.824
24.28***
7.187
22.92***
7.936
27.83***
6.491
23.84***
3.37***
−0.237
Subfield American politics
−0.081
−0.55
0.184
1.30
0.421
Canadian politics
0.041
0.14
0.117
0.42
−0.048
−0.19
0.689
British politics
0.142
0.55
0.121
0.48
0.258
1.07
0.318
1.63
Area studies
−0.338
−2.12**
−0.249
−1.58
−0.289
−1.92*
0.026
0.19
International politics
−0.159
−1.11
−0.086
−0.60
−0.390
−2.94***
0.342
2.52**
Political behavior
0.338
2.07**
0.355
2.27**
−1.92* 2.87***
−0.377
−3.08***
0.307
2.35**
Judicial politics
−0.172
−0.86
−0.149
−0.78
−0.114
−0.68
−0.348
−2.04**
Political theory
−0.000
−0.00
−0.478
−2.78***
−0.193
−1.22
−0.060
−0.40
0.084
0.55
0.002
0.01
0.110
0.85
0.168
1.34
−0.333
−1.51
0.002
0.01
−0.052
−0.27
−0.395
−2.13**
Research methodology Public administration Public policy Other subfield
0.290
1.91*
−0.238
−1.63
0.205
1.38
−0.273
−1.91*
0.279
2.09**
−0.211
−1.59
0.230
1.87*
0.032
0.27
Methodological Approach Quantitative methods Qualitative methods Formal theory
1.072
7.14***
1.497
10.17***
0.813
6.07***
0.548
−0.502
−3.55***
−0.851
−6.22***
−0.707
−5.85***
−0.236
−2.01**
0.412
2.07**
1.72*
−0.506
−2.95***
−0.155
−0.97
−3.71***
0.324
4.31***
Demographic Attributes Age
−0.007
−1.27
−0.007
−1.47
−0.017
Gender
−0.031
−0.24
0.101
0.79
0.024
United Kingdom
−0.398
−2.38**
Canada
−0.379
−1.76*
0.20
0.009
2.11**
0.192
1.74*
Country Variables
N R
0.284
1,535
2
b Intercept
−0.21
−0.422
−2.63***
−0.021
−0.11
1,432
0.163
F
1.70*
−0.046
1,345 0.095
0.272
0.257
27.81***
23.24***
IO
WP
CP
t
b
7.28***
CPS
t
b
t
18.87***
6.772
20.65***
24.36***
7.209
American politics
−0.618
−3.93***
−0.282
−1.61
−0.163
Canadian politics
0.557
1.63
0.419
1.16
0.633
British politics
0.269
0.93
0.048
0.14
−0.152
−0.051
−0.34
1.09 −0.25
1,295
15.48***
8.254
0.149 −0.047
b
t
7.594
24.58***
−0.617
−4.35***
Subfield
Area studies International politics Political behavior
0.423
3.12***
0.825
4.94***
−0.442
−2.90***
0.334 −0.131
−1.10 2.38** −0.63 2.30** −0.95
0.455 −0.117 0.358 −0.221
1.82* −0.52 2.61** −1.69*
0.023
0.13
0.095
0.48
0.169
1.04
0.161
1.06
Judicial politics
−0.282
−1.25
−0.278
−1.11
−0.228
−1.06
−0.261
−1.24
Political theory
−0.593
−3.32***
Research methodology Public administration Public policy Other subfield
0.065
0.41
−0.379
−1.82*
−0.023
−0.13
−0.498
−2.76***
−0.401
−2.27**
−0.302
−1.97**
−0.149
−1.07
−1.09
−0.244
−1.00
−0.347
−1.56
0.236
1.55
0.064
0.45
−0.041
−0.30
−0.586
−2.20**
−0.320
0.219
1.36
0.535
−0.207
−2.06**
−0.368
−2.15**
−0.299
−2.08**
−0.132
−0.92
2.92***
Methodological Approach Quantitative methods
0.184
1.23
0.012
0.07
Qualitative methods
0.062
0.41
0.492
2.91***
−0.63
−0.075
Formal theory
−0.121
−0.34
0.636 −0.432
4.39*** −2.18**
0.252
1.85*
0.346
2.54**
−0.156
−0.84 ~continued !
714 PS • October 2009
.............................................................................................................................................................................................................................................................
Ta b l e 1 0 (c o n t i n u e d ) IO
WP
b
t
CP
b
t
CPS
b
t
b
t
Demographic Attributes Age Gender
−0.016
−0.003
−0.51
−0.006
0.060
−2.93*** 0.44
0.106
0.69
0.404
−0.234
−1.46
−0.392
−2.12**
0.143
0.66
0.060
−1.23 3.10***
−0.016
−3.19***
0.074
0.60
Country Variables United Kingdom Canada
N R
2
F
−0.255
−1.58
−0.436
−2.88***
0.001
0.00
−0.075
−0.37
1095
1072
1138
1051
0.085
0.101
0.100
0.092
5.27***
6.18***
6.50***
5.49***
ISQ
Intercept
0.25
JCR
POP
PT
b
t
b
t
b
t
b
t
6.886
17.36***
6.387
16.29***
5.592
15.01***
8.385
16.18***
Subfield American politics
−0.134
Canadian politics
0.934
−0.73 2.40**
−0.311
−1.78*
0.298
1.76*
−0.104
−0.44
0.655
1.62
0.382
1.05
0.690
1.47
British politics
−0.019
−0.05
0.087
0.24
0.080
0.22
0.410
1.02
Area studies
−0.288
−1.71*
0.125
0.70
0.283
1.44
−0.332
−1.26
0.570
3.54***
0.196
1.00
International politics Political behavior
1.007
6.45***
−0.039
−0.18
Judicial politics
−0.221
−0.81
Political theory
−0.360
−1.74*
Research methodology
−0.019
−0.11
Public administration
−0.233
−0.71
Public policy Other subfield
0.345
1.77*
−0.166
−0.95
0.037
0.15
−0.700
−3.31***
0.266
−0.309
−1.76*
−0.130
−0.57
0.022
0.12
−0.184
−0.62
0.003
0.01
−0.516
−1.85*
0.722
3.41***
1.60
−0.031
−0.19 −0.43
0.731
3.23***
−0.646
−2.50**
0.072
0.25
−0.126
−0.681
−1.75*
0.177
0.95
0.463
2.53**
−0.079
−0.31
0.156
0.90
0.054
0.29
−0.089
−0.38
−1.55
Methodological Approach Quantitative methods
0.176
1.01
0.276
1.58
−0.539
−3.05***
−0.389
Qualitative methods
−0.016
−0.10
−0.615
−3.71***
0.786
4.86***
0.064
Formal theory
−0.643
−2.85***
−0.131
−0.62
−0.889
−3.93***
−0.900
−2.77***
−0.010
−1.70*
0.001
0.13
0.001
0.11
−0.020
−2.42**
0.137
0.87
0.264
1.62
0.623
4.08***
0.439
2.02**
−3.30***
0.30
Demographic Attributes Age Gender Country Variables United Kingdom
−0.222
−1.20
−0.254
−1.29
−0.775
Canada
−0.205
−0.83
−0.005
−0.02
0.236
0.86
−0.676
−2.76***
−0.511
−1.56
N
865
876
1080
547
R2
0.100
0.079
0.156
0.140
F
4.93***
3.87***
10.29***
4.50*** ~ continued !
two of which are known as outlets for publishing qualitative research. For formal-theory scholars, the American Political Science Review draws favorable evaluations, no doubt because it publishes highly regarded work in formal theory; on the other hand, we note that formal-theory scholars give less positive marks to the Journal of Politics, Comparative Politics, International Studies Quarterly, Political Theory, and Political Studies.
The cases of International Organization and Perspectives on Politics provide an important contrast. On one hand, for International Organization none of the methodology coefficients is statistically significant, suggesting that scholars from all methodological traditions evaluate this journal in similar terms. On the other hand, Perspectives on Politics represents a stark case of the methodological divide in political science. This journal is PS • October 2009 715
The Profession: Political Science Journals in Comparative Perspective
.............................................................................................................................................................................................................................................................
Ta b l e 1 0 . (c o n t i n u e d ) CJPS
Intercept
POLITICAL STUDIES
b
t
b
t
5.141
12.71***
5.583
12.52***
Subfield American politics
−0.068
Canadian politics
1.754
7.21***
0.704
2.07**
British politics
0.625
2.24**
0.764
3.13***
Area studies
0.170
0.84
0.350
International politics
0.121
0.69
−0.104
Political behavior Judicial politics Political theory Research methodology
−0.37
−0.370
0.212
1.06
0.104
−0.195
−0.80
−0.617
−1.69*
1.79* −0.56 0.45 −2.14**
0.712
3.54***
0.968
4.76***
−0.572
−3.03***
−0.729
−3.07*** −1.54
Public administration
0.021
0.08
−0.427
Public policy
0.233
1.26
0.492
2.43**
Other subfield
0.035
0.20
0.073
0.40
−0.002
−0.01
−0.306
−1.54
0.224
1.34
0.133
−0.168
−0.68
−0.940
−3.04***
Age
0.002
0.27
−0.001
−0.12
Gender
0.209
1.28
0.192
1.05
Methodological Approach Quantitative methods Qualitative methods Formal theory
0.70
Demographic Attributes
ing with the perception by many political scientists that Perspectives on Politics was created to appeal to (mostly qualitative) critics of the American Political Science Review. Finally, we also include two demographic characteristics, age and gender, as independent variables in our models. The gender coefficient is significant for only three journals. Controlling for the effects of other variables, women political scientists are more positive in their evaluations of Comparative Politics, Perspectives on Politics, and Political Theory. Women are indistinguishable from men in how they evaluate the other political science journals in Table 10. The age variable was expected to capture generational influences on journal evaluations, and we find that a small number of the age coefficients achieve conventional levels of statistical significance. Older political scientists are less favorably oriented toward the Journal of Politics, International Organization, Comparative Political Studies, and Political Theory. There does not appear to be a systematic factor that ties these negative evaluations together, since this list includes journals from a variety of subfields and methodological approaches. CONCLUSION
The purpose of this article is to build on previous research relating to how political scientists evalUnited Kingdom −0.382 −1.90* 0.808 4.23*** uate the quality and impact of scholarly journals Canada −0.173 −0.86 0.531 1.87* in the discipline. Scholars who study journal impact in the social sciences often differentiate N 726 678 objective (i.e., bibliographic or citation-based) R2 0.133 0.219 approaches and subjective (i.e., survey-based) F 5.71*** 9.73*** approaches for evaluating how various scholarly *** prob~ t ! < .01 journals influence intellectual discourse in the political science profession. As Giles and Garand ** prob~ t ! < .05 (2007) point out, these two approaches provide * prob~ t ! < .10 complementary information about journal Note: Comparative politics is the excluded group among the subfield variables, and normative theory is the excluded group among the methodological approach variables. impact, and ultimately there is a moderately Key: strong relationship between the rankings of jourAPSR American Political Science Review nals using these two approaches. AJPS American Journal of Political Science This article clearly adopts the subjective JOP Journal of Politics approach, but with an important twist. In most BJPS British Journal of Political Science previous studies scholars have conducted surIO International Organization WP World Politics veys of political scientists from the United States, CP Comparative Politics so relatively little is known about how American CPS Comparative Political Studies political scientists differ from those from other ISQ International Studies Quarterly countries in how they evaluate political science JCR Journal of Conflict Resolution POP Perspectives on Politics journals. Here we report the results of a survey PT Political Theory that we conducted of political scientists from the CJPS Canadian Journal of Political Science United States, Canada, and the United KingPolitical Studies Political Studies dom, and this permits us to make explicit comparisons among political scientists from these evaluated significantly less positively by quantitative and formalthree countries. Moreover, our data permit us to consider how theory political scientists, with qualitative scholars giving signifjournal evaluations in 2007 are similar to those observed in preicantly more favorable evaluations. Taken as a whole, it appears vious studies. that Perspectives on Politics is favorably evaluated by qualitative What do we learn about journal evaluations, familiarity, and and normative-theory scholars, but is much less favorably evaluimpact? First, as is the case in previous research, we find that ated by quantitative and formal-theory scholars. This is in keepthere is a fairly clear ranking of political science journals. This Country Variables
716 PS • October 2009
.............................................................................................................................................................................................................................................................
ranking is fairly robust across the different types of measures that we use throughout this study. There is a small group of elite journals that are both broadly familiar to and evaluated very favorably by political scientists, and these journals earn high marks for their subjective impact on the political science discipline. These journals include the three major general journals from the United States (i.e., the American Political Science Review, American Journal of Political Science, and Journal of Politics), a major general journal from the United Kingdom (i.e., British Journal of Political Science), and several broad-based subfield journals, such as International Organization, Comparative Political Studies, International Studies Quarterly, among others. It is these journals that draw considerable attention in the political science discipline, and publications in these journals are more likely to be read and cited and to become part of the scholarly discourse in the profession. Other journals have some combination of high familiarity or strong positive evaluations, but not both, and these journals form a second tier of scholarly journals on our list. Many of these journals are highly regarded specialty journals (e.g., Legislative Studies Quarterly, Electoral Studies) or well-known general journals with moderate evaluations (e.g., Perspectives on Politics, Political Research Quarterly, or Foreign Affairs). Finally, there are other journals that are at the periphery of political science, insofar as they are not widely read and/or not highly evaluated. Second, there are some differences in how political scientists from the United States, Canada, and the United Kingdom perceive the quality and impact of scholarly journals, but most of these differences are fairly modest. If one looks at the top-10 lists for each of these three countries, there is a fair amount of similarity in the journals that make the list, though there are notable exceptions. Everyone thinks highly of the American Political Science Review, American Journal of Political Science, British Journal of Political Science, International Organization, and a small number of other journals, but there are also some journals (e.g., European Journal of Political Research, Political Studies, Perspectives on Politics, Canadian Journal of Political Science, Journal of Common Market Studies) that have country-specific appeal. Taken as a whole, there is more broad similarity in journal evaluations than differences for political scientists across these three countries. The correlation for journal-impact measures among these three countries is fairly high. Finally, we estimated a series of regression models for a small subset of 14 journals and find little systematic country differences in how political scientists evaluate these journals. Although there are some mean differences across countries that emerge in our ANOVA analyses, many of these differences disappear when we include control variables representing subfield specialties, methodological approaches, demographic attributes, and publication status. It appears that most of the differences among political scientists from the United States, Canada, and the United Kingdom are due to methodological differences across these three countries. Political scientists’ methodological approach has been found to relate to evaluation of some journals (Garand and Giles, 2003), and there is clear evidence of substantial methodological differences for political scientists from the United States, Canada, and the United Kingdom (Blais et al. 2008). When the methodological differences among political scientists from the United States, Canada, and the United Kingdom are filtered out, the coefficients representing country-specific effects become statistically non-
significant. The underlying basis for differences in political scientists’ methodological approaches across these three countries is the subject of ongoing research. All of this raises some questions about the possibility of greater cross-national integration of political science. While there is a broad-based consensus about journal evaluation, familiarity, and impact among political scientists from all three countries, there are some notable differences, and the methodological divide for political scientists from the United States, Canada, and the United Kingdom could possibly create a barrier to further integration of cross-national research. How political scientists from different countries but working on similar topics can be brought to communicate with each other may have geographic underpinnings, but it is also possible that methodological differences may serve as the greatest barrier. 䡲 NOTES 1. When we remove the journals representing the field of international relations, which appear to be operating in a different citation environment/culture, the correlation between the bibliometric and reputational ranks of the 76 remaining journals is 0.795. 2. These journals are Law and Society Review, Presidential Studies Quarterly, Review of Politics, and Social Science Quarterly. We do not include in this step journals familiar to 20% or more in the Garand and Giles study but central to another discipline, that is, American Economic Review, American Sociological Review, and American Journal of Sociology. 3. These journals are Political Analysis, Canadian Public Administration, and Canadian Public Policy. 4. “For the sake of brevity, we have created an online appendix in which we present tables and figures representing various auxiliary analyses. This online appendix can be found at the Cambridge University Press Web site: http://journals.cambridge.org/action/displayJournal?jid=PSC.
REFERENCES Blais, Andre, Iain McLean, James C. Garand, and Micheal W. Giles. 2008. “The Methodological Divide in Political Science.” Paper presented at the 2008 annual meeting of the Canadian Political Science Association, Vancouver, British Columbia, June 4–6. Christenson, James, and Lee Sigelman. 1985. “Accrediting Knowledge: Journal Stature and Citation Impact in Social Science.” Social Science Quarterly 66: 964–75. Crewe, Ivor, and Pippa Norris. 1991. “British and American Journal Evaluation: Divergence or Convergence.” PS: Political Science and Politics 24: 524–30. Garand, James C. 1990. “An Alternative Interpretation of Recent Political Science Journal Evaluations.” PS: Political Science and Politics 23: 448–51. _. 2005. “Integration and Fragmentation in Political Science: Exploring Patterns of Scholarly Communication in a Divided Discipline.” Journal of Politics 67: 979–1005. Garand, James C., and Micheal W. Giles. 2003. “Journals in the Discipline: A Report on a New Survey of American Political Scientists.” PS: Political Science and Politics 36: 293–308. Giles, Micheal, Francie Mizell, and David Patterson. 1989. “Political Scientists’ Journal Evaluations Revisited.” PS: Political Science and Politics 22: 613–17. Giles, Micheal, and Gerald Wright. 1975. “Political Scientists’ Evaluations of SixtyThree Journals.” PS: Political Science and Politics 8: 254–57. Giles, Micheal, and James C. Garand. 2007. “Ranking Political Science Journals: Reputational and Citational Approaches.” PS: Political Science and Politics 40: 741–51. Goodson, Larry, Bradford Dillman, and Anil Hira. 1999. “Ranking the Presses: Political Scientists’ Evaluations of Publisher Quality.” PS: Political Science and Politics 32: 257–62. Hix, Simon. 2004. “A Global Ranking of Political Science Departments.” Political Studies Review 2: 293–313. Smith, Rogers. 2002. “Should We Make Political Science More of a Science or More About Politics?” PS: Political Science and Politics 35: 199–201.
PS • October 2009 717