Political Science Journals in Comparative Perspective - Cambridge ...

.............................................................................................................................................................................................................................................................

The Profession .............................................................................................................................................................................................................................................................

Political Science Journals in Comparative Perspective: Evaluating Scholarly Journals in the United States, Canada, and the United Kingdom James C. Garand, Louisiana State University Micheal W. Giles, Emory University André Blais, Université de Montréal Iain McLean, Nuffield College, Oxford University

In this article we report the results from a new survey of political scientists regarding their evaluations of journals in the political science discipline. Unlike previous research that has focused on data from the United States, we conducted an Internet survey of political scientists in the United States, Canada, and the United Kingdom. We present data on journal evaluations, journal familiarity, and journal impact, both for our entire sample (N =1,695) and separately for respondents from each of the three countries. We document the overall hierarchy of scholarly journals among political scientists, though we find important similarities and differences in how political scientists from these three countries evaluate the scholarly journals in the discipline. Our results suggest that there is a strong basis for cross-national integration in scholarly journal communication, though methodological differences among the three countries may be an impediment. ABSTRACT

P

olitical scientists communicate through a wide variety of scholarly media, including books, journal articles, convention papers, and electronic papers. Among these, books and journal articles hold a preeminent place in scholarly communication networks,

but it is also the case that not all book publishers and scholarly journals are created equal. Rather, there is a well-documented hierarchy among both book publishers (Goodson, Dillman, and Hira 1999) and scholarly journals (Giles and Wright 1975; Giles, Mizell, and Patterson 1989; Garand 1990, 2005; Garand

James C. Garand is the Emogine Pliner Distinguished Professor and the R. Downs Poindexter Professor of Political Science at Louisiana State University. His research interests cover varying aspects of American politics, and his current research includes projects relating to the sociology of the political science discipline, public opinion on immigration, and the political implications of income inequality. He is former editor of the American Politics Quarterly and former president of the Southern Political Science Association. In 2006 he was named the LSU Distinguished Research Master, which is the highest career award for research given to faculty at LSU. (E-mail: [email protected]). Micheal W. Giles is the Goodrich C. White Professor of Political Science at Emory University. In addition to a continuing interest in the sociology of the discipline, his research focuses primarily on courts and judicial behavior. His current work focuses on the institution of en banc rehearing in the U.S. Courts of Appeals. He has served the profession in many positions including as president of the Southern Political Science Association, chair

of the Law and Courts Section of the American Political Science Association, and editor of the Journal of Politics. André Blais is professor in the department of political science at the Université de Montréal. He holds a Canada Research Chair in Electoral Studies and he is a fellow of the Royal Society of Canada. His main research interests are electoral behavior, turnout, and electoral systems. He is the director of Making Electoral Democracy Work: Voters, Parties, and the Rules of the Game (www.electoraldemocracy.com). He can be reached at: [email protected]. Iain McLean is a professor of politics at the University of Oxford and Official Fellow in Politics at Nuffield College. His research interests include devolution, the design of electoral systems, and the application of rational choice models to political history. He is the author of Rational Choice and British Politics (OUP, 2001) and of a forthcoming book titled What’s Wrong with the British Constitution? (OUP, 2009). He is a fellow of the British Academy.

doi:10.1017/S1049096509990205

PS • October 2009 695

The Profession: Political Science Journals in Comparative Perspective

.............................................................................................................................................................................................................................................................

and Giles 2003; Crewe and Norris 1991), with some scholarly outlets viewed as more significant than others. Some political scientists differentiate the contributions and importance of various scholarly journals using so-called objective criteria such as citation counts and impact factors (Christenson and Sigelman 1985; Hix 2004; Giles and Garand 2007), while others use subjective self-reports based on surveys of political scientists (Giles, Mizell, and Patterson 1989; Garand 1990; Garand and Giles 2003). There is evidence that, properly conceived, subjective and objective indicators of journal impact are at least moderately highly correlated (Giles and Garand 2007). What do we know about how political scientists evaluate scholarly journals in the discipline? First, most of the work on subjective journal evaluations has been conducted in the United States, and there is a discernible pattern of journal rankings among American political scientists. Some journals are regularly awarded with elite status; these are journals that are familiar to high proportions of political scientists and receive high marks for the rigor and quality of research that they publish. Other journals are targeted to more specialized audiences (and hence are not broadly familiar to political scientists), are perceived as less rigorous in their scholarly standards, or both, and these journals typically receive less favorable evaluations from American political scientists. Second, there is some systematic variation among American political scientists in how they evaluate scholarly journals. Political scientists differ by subfield specialization and methodological orientation in their familiarity with and evaluations of scholarly journals (Garand and Giles 2003; Garand 2005). However, these differences occur primarily in how scholars rank the elite disciplinary journals, with political scientists shuffling the order of highly ranked journals depending on their subfield and methodological approach. When one considers subfield and methodological differences in evaluations, familiarity, and impact for a broad set of political science journals, one finds that there are more similarities than differences. While scholars who concentrate on American politics may differ from those who study international relations, comparative politics, or political theory in the journals that they tout as being the leading journals in the discipline, there are similarities in evaluations across the broad spectrum of journals (Garand 2005). Perhaps the biggest shortcoming in this literature is its almost exclusive focus on journal evaluations by American political scientists. While we know a fair amount about how American political scientists evaluate scholarly journals, we know little about patterns of journal evaluations among political scientists in other countries. Simply, very little work has been done in terms of comparative, cross-national research on how political scientists evaluate scholarly journals. The most noteworthy exception is Crewe and Norris (1991), who replicate the work of Garand (1990) and Giles, Mizell, and Patterson (1989) for a sample of British political scientists. They find considerable differences in how British and American political scientists evaluate political science journals. There are some journals that emerge on both British and American lists, but British political scientists evaluate several British and/or subfield journals very differently than do American political scientists. This work provides tantalizing evidence of country effects in how scholars evaluate the professional media in political science. Given the globalization of the discipline of political science, it is important to describe and understand how polit696 PS • October 2009

ical scientists in different countries evaluate the scholarly media in which they communicate with other scholars. In this article we build on previous research on subjective journal evaluations and respond to the need for cross-national comparison. We replicate previous work by Giles and Garand (Giles and Wright 1975; Giles, Mizell, and Patterson 1989; Garand 1990, 2005; Garand and Giles 2003) by reporting results based on an online survey in which we ask political scientists to evaluate a wide range of political science journals. Hence our results are directly comparable to those reported in previous research by Giles and Garand and their collaborators. We use both political scientists’ evaluations of scholarly journals and their familiarity with these journals to create a measure of “journal impact” that is comparable to those reported in previous studies. However, in addition to respondents from the United States, we include a sample of respondents from the United Kingdom and Canada, and this permits us to make explicit comparisons among political scientists from these three countries. The result is a broader picture of how political scientists evaluate the major journals through which they communicate the findings of their research. DATA AND METHODS

The present study is based on responses to an Internet survey of political scientists conducted in the United States, Canada, and United Kingdom during 2007. We started with the questionnaire that was the basis for the work of Garand and Giles (2003) on journal evaluations among American political scientists. Garand and Giles asked political scientists about their evaluations of 115 political science journals, as well as a variety of demographic, career, and attitudinal variables. In the present survey we ask respondents to evaluate 92 political science journals, and we adapt other components of the Garand-Giles survey to reflect differences across the three countries. We use English versions of the survey in each country, and in Canada we provide respondents with the option of answering the questionnaire in French. Overall, these data permit us to provide ratings of political science journals on three important dimensions (i.e., evaluation, familiarity, and impact) and to explore the key determinants of these ratings. Sampling Issues Our population of interest is political scientists who are affiliated with political science departments housed in universities that grant the Ph.D. in political science. This stands in some contrast to the work of Garand and Giles, who included a large subsample of faculty from Ph.D.-granting departments but also a subsample of political scientists from departments that do not grant the Ph.D. For the United States, we obtained a list from the American Political Science Association (APSA) of members affiliated with political science departments that offer the Ph.D. in political science. The total number of American political scientists on the list was 3,486; of these, 1,134 provided usable responses, for a response rate of 32.53%. For Canada, where many political scientists are not members of the Canadian Political Science Association (CPSA), we could not rely on CPSA membership lists. Instead, we started with a list of all Ph.D.-granting political science departments in Canada, consulted the Web site for each department to compile an initial faculty list, and followed up by contacting all departments to see

.............................................................................................................................................................................................................................................................

whether our list needed to be updated. Based on this procedure we generated a list of 607 names of faculty at Canadian Ph.D.granting departments. Of these, we received usable responses from 196, for a response rate of 32.3%. For the United Kingdom, the population of interest is defined by the annual Political Studies Association directory. From data supplied by heads of departments, the directory lists all academic staff in political science and cognate departments in the UK, whether or not they are PSA members. It also lists PSA (and British International Studies Association) members in institutions outside of political science departments. As a result, the Directory likely overstates the true population of political scientists in Ph.D.granting departments, making difficult the calculation of a firm response rate. Based on the Directory we identified 1,943 political scientists in Ph.D.-granting departments, of whom 432 responded for a response rate of 22.2%. However, because the denominator likely includes a large number of non-political scientists who are unlikely to respond to our survey, this figure understates the actual response rate. By comparison, about 1,000 political scientists are entered by the universities as research-active political scientists in the 2008 Research Assessment Exercise, so the true unobservable population lies between 1,000 and 1,943. The 22.2% figure is hence a lower boundary on the response rate for the UK sample. Our total sample consists of 1,695 respondents, with 64% from the United States, 11% from Canada, and 24% from the United Kingdom. Survey Administration The present expert survey is the fifth in a series initiated in 1975, with the most recent installment published in 2003 by the two U.S. authors of this article (Giles and Wright 1975; Giles, Mizell, and Patterson 1989; Garand 1990; Garand and Giles 2003). In these works Giles and Garand conducted surveys of political scientists in the U.S., with respondents asked to evaluate the quality of journals on a scale ranging from 0 (poor quality) to 10 (outstanding). Garand (1990) and Garand and Giles (2003) combined data on mean journal evaluations and the proportion of respondents who were familiar with each journal to create a measure of journal impact, measured as: Journal Impact = Journal Evaluation + (Journal Evaluation * Journal Familiarity) This measure has a theoretical range of 0 (poor mean evaluation and no journal familiarity) to 20 (perfect mean evaluation and perfect familiarity). The authors reason that the most important journals in political science are those that are both (1) highly regarded for the quality of the work that they publish and (2) highly visible to the broadest group of political scientists. By combining quality and familiarity measures into a single scale, Garand and Giles created an impact measure that has a high level of face validity and that is fairly highly correlated (r = 0.656) with citationbased measures of journal impact.1 Our survey was administered by the Public Policy Research Laboratory, an academic survey-research center located at Louisiana State University. Respondents from the United States, United Kingdom, and Canada were sent e-mails with a link to the survey, which was tailored to the language and academic customs of each country. After an initial period of receiving responses from our sample, we sent a second reminder e-mail to respondents.

The survey was divided into four sections. First, all respondents received in the e-mail solicitation a cover letter that included a brief description of the project, a confidentiality statement, and a statement relating to human-subjects review by the institutional review board (IRB) at Louisiana State University. Second, we included a series of questions designed to gather descriptive information, including country of origin, highest degree received, age, sex, academic rank, field and subfield interests, and methodological approaches. Third, we included a section with openended questions in which respondents could identify journals (1) to which respondents would submit “a very strong paper on a topic in your area of expertise,” and (2) that respondents “read regularly or otherwise rely on for the best research in your area of expertise.” This provides an alternative way of measuring respondents’ evaluations of political science journals, as developed by Garand and Giles (2003). Finally, we included a section in which we asked respondents to evaluate “journals in terms of the general quality of the articles it publishes.” We used a scale from 0 (poor) to 10 (outstanding) and asked respondents to evaluate each of 92 journals with which they might be familiar. We also asked respondents specifically to indicate if they were familiar with each of these journals, as well as whether or not they have ever published an article in each journal. Journal List The selection of journals for the survey required a balance between (1) including sufficient journals to achieve adequate coverage across journal types, specialty areas, and countries of origin and (2) not including so many journals as to burden survey respondents and reduce the response rate below acceptable levels. This was a difficult task, because there are many more journals of interest to political scientists than there is room in a survey of manageable length. We approached this task in four steps. First, we included all of the journals in the Garand and Giles (2003) survey that were also classified as political science, public administration, or international studies by the Institute for Scientific Information (ISI). Second, given the emphasis of the Garand and Giles list on American journals, we included all of the journals included in Hix’s (2004) UK-originated bibliometric study. This resulted in the inclusion of 84 journals: (1) 45 journals that were in the appropriate ISI categories and included in the studies of both Garand and Giles and Hix; (2) 19 journals that were only included in the Hix study; and (3) 20 journals that were not included in the Hix study but were in the appropriate ISI category and included in the Garand and Giles study. Third, we included all journals that were familiar to at least 20% of the respondents to the Garand and Giles study regardless of their ISI categorization. This step added only four journals to the list.2 Finally, each of the authors was allowed to nominate journals for inclusion but agreement of three of the four authors was required for a nominated journal to be added to the list. Only three journals were added to the list through this procedure.3 This procedure resulted in the exclusion of several new journals that are already well recognized, as well as some other journals with strong reputations but that do not meet the criteria to which we agreed at the outset of the study. In some cases we received inquiries from journal editors or other interested scholars, asking why a given journal was excluded from our study. However, our goal was to establish clear criteria for selecting journals and follow those criteria with only very limited exceptions. We PS • October 2009 697


.............................................................................................................................................................................................................................................................

did give respondents the opportunity to list additional journals, both at the end of the closed-answer section of the survey and in the open-ended questions about journal submission and journalreading preferences. EMPIRICAL RESULTS

Consistency of Journal Impact, Evaluation, and Familiarity Measures over Time As a starting point, we consider the degree to which there is stability in our measures of journal impact, evaluation, and familiarity from one survey to the next. Garand and Giles (2003) find a considerable amount of consistency in these three measures from the 1989 Giles, Mizell, and Patterson study (Garand 1990) to the 2001 survey conducted by Garand and Giles (2003), and our findings are very similar. We examined the scatterplots and estimated correlation and regression coefficients for the relationships between 2001 and 2007 measures of journal impact, journal evaluations, and journal familiarity for the 69 journals that are included on both the 2001 and 2007 surveys. Since the 2001 survey was based on a sample drawn only from the United States, we include here only the U.S. respondents to our 2007 survey. Based on these scatterplots—which are available in Online Appendix Figures 1–3 4 —as well as the correlation and regression coefficients, it is clear that there is considerable stability over time in the relative positions of journals in their impact, evaluations, and familiarity. The relationship between 2001 and 2007 measures of journal impact is very strong (r = 0.950, b =1.226, t = 25.00), suggesting that how political scientists assess the impact of scholarly journals in 2001 is closely aligned with how they evaluate the impact of scholarly journals in 2007. The same can be said for journal evaluations, where the correlation is slightly smaller but still very high (r = 0.890, b = 1.146, t = 16.00). There is also a very strong relationship between the 2001 and 2007 journal familiarity measures (r = 0.939, b =1.081, t = 22.25). Simply, these results reveal that the highest ( lowest) journals in terms of evaluation, familiarity, and overall impact in 2007 were also the highest ( lowest) impact, evaluation, and familiarity journals in 2001. Journal Impact in Three Countries In Table 1 we report the relative journal impact scores and rankings for respondents from all three countries combined, as well as separately for respondents from the United States, United Kingdom, and Canada. The relative impact score is the impact score for each journal, divided by the impact score for the highest-ranked journal in each relevant comparison group. A more detailed summary of mean journal evaluations, journal familiarity, and journal impact for all respondents is found in Online Appendix 1, and separate detailed country-specific summary tables are found in Online Appendix 2. Several important findings emerge from these results. First, the elite journals among political scientists from all three countries combined are similar to those found in previous studies of political scientists in the United States. This no doubt reflects the fact that almost two-thirds of respondents are from the United States, and we have already documented the continuity in ratings for this group. Once again, the American Political Science Review, American Journal of Political Science, and Journal of Politics emerge as the top three journals in the discipline, with the British Journal of Political Science ranked a close fourth. These are 698 PS • October 2009

general readership journals that are both highly regarded for the quality and rigor of the research that they publish and broadly familiar to a wide range of political scientists. The remainder of the top 10—including International Organization, World Politics, Comparative Politics, Comparative Political Studies, International Studies Quarterly, and the Journal of Conflict Resolution—comprises journals that represent the leading general journals in the fields of comparative politics and international relations. When one moves to the second tier, one finds journals of different stripes, including general journals (e.g., Perspectives on Politics, Political Research Quarterly, PS: Political Science and Politics), broad-based subfield journals (e.g., Foreign Affairs, Political Theory, International Security), and well-regarded specialty journals (e.g., Legislative Studies Quarterly, Electoral Studies). Second, careful readers will note that there is a gradual dropoff in relative impact scores for journals across the distribution. While there seems to be a strong consensus that there are certain journals that are among the elite journals in political science, in many cases the journal impact scores of lower-ranked journals are not very far removed from the scores for high-ranked journals. For instance, 53 journals have relative impact scores that are at least 50% of the impact score for the American Political Science Review, the journal ranked first among the combined respondents from the three countries in this study. As Giles and Garand (2007) suggest, measures of journal impact based on citations tend to have a small number of elite journals accompanied by a rapid drop-off, with a small number of journals being cited a great deal and most journals with small citation counts. However, the more gradual drop-off in relative journal impact suggests that publications in many journals are viewed by political scientists in favorable terms, even if articles in those moderately ranked journals are not cited extensively by other scholars. Third, there is considerable similarity in how political scientists in the United States, United Kingdom, and Canada perceive the impact of scholarly journals in the discipline, though there are also some noteworthy differences. Among the top 10 journals for all respondents combined, nine are found on the American top-10 list, seven are found on the UK list, and eight are found on the Canadian list. For American political scientists, Perspectives on Politics moves into the top 10, replacing International Studies Quarterly, which is still ranked eleventh. Among political scientists from the United Kingdom, three journals in their top 10—European Journal of Political Research (eighth), Political Studies (third), and the European Journal of International Relations (tenth)—are not found in the overall top 10; UK political scientists downgrade the rankings of the Journal of Politics (from third to thirteenth), International Studies Quarterly (from nineth to seventeenth), and the Journal of Conflict Resolution (from tenth to thirty-first). For Canadian political scientists, the International Studies Quarterly and Journal of Conflict Resolution are replaced in the top 10 by the European Journal of Political Research (ranked tenth by Canadian political scientists) and the Canadian Journal of Political Science (ranked seventh). Overall, there is a fair amount of agreement among political scientists in these three countries in terms of the journals that they place in the elite category. One can observe the similarity of journal rankings for political scientists across country by examining the scatterplots of relative journal rankings for each pair of countries. In Figures 1–3 we present scatterplots of relative journal impact for the

.............................................................................................................................................................................................................................................................

Ta b l e 1

Evaluations, Familiarity, and Impact Ratings for Political Scientists from the United States, United Kingdom, and Canada, 92 Selected Journals, 2007 ALL COUNTRIES JOURNAL

Relative Impact

American Political Science Review @*#

1.000

American Journal of Political Science Journal of Politics

UNITED STATES Rank

Relative Impact

1

1.000

0.930

2

0.882

3

British Journal of Political Science

0.872

International Organization

0.845

World Politics Comparative Politics Comparative Political Studies

0.766

8

International Studies Quarterly

0.697

9

Journal of Conflict Resolution

0.672

10

Perspectives on Politics @*#

0.666

11

Legislative Studies Quarterly

0.638

Political Analysis

0.637

Political Theory Foreign Affairs

CANADA Rank

Relative Impact

1

1.000

0.932

2

0.911

3

4

0.820

5

0.805

0.798

6

0.766

0.768

7

0.713

UNITED KINGDOM Rank

Relative Impact

Rank

1

0.969

2

0.900

5

0.910

5

0.832

9

0.771

14

4

0.921

3

1.000

1

5

0.935

2

0.913

4

6

0.885

6

0.846

7

8

0.910

4

0.857

6

0.725

7

0.878

7

0.830

9

0.670

10

0.749

11

0.742

17

0.670

11

0.665

26

0.655

32

0.688

9

0.706

20

0.554

53

12

0.660

12

0.568

54

0.574

48

13

0.646

14

0.594

43

0.612

38

0.635

14

0.597

16

0.705

21

0.715

24

0.632

15

0.585

18

0.744

12

0.717

23

Political Research Quarterly

0.629

16

0.653

13

0.597

42

0.534

61

European Journal of Political Research

0.618

17

0.507

36

0.761

10

0.843

8

PS: Political Science and Politics @*#

0.606

18

0.591

17

0.743

13

0.577

46

Electoral Studies

0.601

19

0.550

22

0.655

28

0.725

21

International Security

0.600

20

0.548

23

0.716

16

0.704

26

Public Opinion Quarterly

0.598

21

0.601

15

0.581

48

0.594

42

Political Studies @*#

0.596

22

0.433

49

0.709

19

0.939

3

Philosophy and Public Affairs

0.585

23

0.520

33

0.669

25

0.727

20

Politics and Society

0.579

24

0.530

29

0.729

14

0.650

33

Journal of Theoretical Politics

0.575

25

0.557

21

0.574

50

0.628

36

Political Behavior

0.565

26

0.565

20

0.571

51

0.566

51

Journal of Democracy

0.564

27

0.515

35

0.683

24

0.648

34

European Journal of International Relations

0.560

28

0.444

47

0.717

15

0.783

10

American Politics Research

0.554

29

0.577

19

0.489

78

0.474

79

Political Science Quarterly

0.554

30

0.534

28

0.614

38

0.591

43

Canadian Journal of Political Science @*#

0.552

31

0.484

39

0.868

8

0.586

44

Public Administration Review

0.548

32

0.534

27

0.620

34

0.546

57

History of Political Thought

0.544

33

0.473

41

0.626

33

0.699

27

Party Politics

0.543

34

0.455

43

0.641

31

0.738

18

Public Choice

0.543

35

0.541

24

0.554

59

0.545

58

Review of International Studies

0.542

36

0.415

55

0.692

23

0.777

12

Journal of Law and Economics

0.540

37

0.540

25

0.584

46

0.517

65

Journal of Peace Research

0.532

38

0.495

37

0.578

49

0.621

37

Political Psychology

0.530

39

0.524

30

0.555

57

0.538

60

West European Politics

0.530

40

0.412

56

0.592

45

0.778

11

Polity

0.528

41

0.523

32

0.582

47

0.507

69

Journal of Common Market Studies

0.527

42

0.404

62

0.617

37

0.767

15 ~continued !



.............................................................................................................................................................................................................................................................

Ta b l e 1 (c o n t i n u e d ) ALL COUNTRIES JOURNAL

Relative Impact

UNITED STATES

Rank

Relative Impact

CANADA

UNITED KINGDOM

Rank

Relative Impact

Rank

Relative Impact

Rank

Social Science Quarterly

0.525

43

0.540

26

0.525

69

0.493

74

Governance

0.520

44

0.425

50

0.713

18

0.689

28

Annals of the American Academy

0.519

45

0.483

40

0.645

29

0.572

49

Government and Opposition

0.519

46

0.408

57

0.603

41

0.771

13

Journal of European Public Policy

0.517

47

0.394

66

0.609

40

0.736

19

International Political Science Review

0.516

48

0.438

48

0.714

17

0.633

35

Law and Society Review

0.516

49

0.517

34

0.568

55

0.463

83

International Affairs

0.514

50

0.392

68

0.613

39

0.763

16

Review of International Political Economy

0.514

51

0.418

53

0.695

22

0.671

30

Studies in American Political Development

0.514

52

0.524

31

0.493

77

0.413

87

Millennium

0.509

53

0.398

64

0.664

27

0.721

22

Journal of Policy Analysis and Management

0.500

54

0.493

38

0.539

66

0.457

84

Publius

0.495

55

0.469

42

0.645

30

0.510

67

Public Administration

0.488

56

0.366

80

0.620

35

0.707

25

Journal of Public Policy

0.477

57

0.420

51

0.562

56

0.599

41

Democratization

0.475

58

0.391

69

0.488

79

0.678

29

Review of Politics

0.471

59

0.445

46

0.536

67

0.517

66

Studies in Comp. and Int. Development

0.469

60

0.455

44

0.499

74

0.473

80

Political Communication

0.468

61

0.450

45

0.511

71

0.500

72

Political Geography

0.468

62

0.419

52

0.553

60

0.571

50

European Union Politics

0.461

63

0.391

70

0.500

72

0.611

39

Security Studies

0.454

64

0.405

61

0.544

64

0.558

52

Nations and Nationalism

0.450

65

0.344

83

0.592

44

0.610

40

American Journal of International Law

0.447

66

0.393

67

0.541

65

0.581

45

Political Quarterly

0.445

67

0.337

85

0.549

61

0.656

31

Latin American Politics and Society

0.444

68

0.408

58

0.569

53

0.502

71

Policy Studies Journal

0.442

69

0.416

54

0.544

63

0.478

77

Women and Politics

0.431

70

0.406

60

0.545

62

0.479

76

Journal of Legislative Studies

0.428

71

0.383

72

0.494

76

0.548

55

Review Francaise de Science Politique

0.428

72

0.367

78

0.521

70

0.547

56

Scandinavian Political Studies

0.426

73

0.376

76

0.480

80

0.534

62

Australian Journal of Political Science

0.420

74

0.343

84

0.569

52

0.542

59

Political Science

0.420

75

0.365

81

0.554

58

0.509

68

Conflict Management and Peace Science

0.418

76

0.407

59

0.479

81

0.408

89

Journal of Strategic Studies

0.418

77

0.368

77

0.470

84

0.525

63

Administration and Society

0.416

78

0.401

63

0.469

85

0.421

86

American Review of Public Administration

0.415

79

0.389

71

0.477

82

0.470

81

Europe-Asia Studies (Soviet Studies)

0.414

80

0.334

86

0.497

75

0.574

47

Politische Vierteljahreschrift

0.413

81

0.377

75

0.431

89

0.495

73

Cooperation and Conflict

0.406

82

0.323

89

0.499

73

0.553

54

Policy Sciences

0.406

83

0.366

79

0.535

68

0.464

82

East European Politics and Societies

0.401

84

0.350

82

0.471

83

0.517

64

Public Interest

0.399

85

0.379

74

0.450

88

0.480

75

Presidential Studies Quarterly

0.393

86

0.398

65

0.462

87

0.411

88 ~ continued )

700 PS • October 2009

.............................................................................................................................................................................................................................................................

Ta b l e 1 (c o n t i n u e d ) ALL COUNTRIES JOURNAL

Relative Impact

UNITED STATES Rank

Relative Impact

CANADA

UNITED KINGDOM

Rank

Relative Impact

Rank

Relative Impact

Rank

International Interaction

0.391

87

0.382

73

0.423

90

0.400

90

Post-Soviet Affairs

0.382

88

0.327

87

0.463

86

0.506

70

Canadian Public Administration

0.366

89

0.280

91

0.617

36

0.372

91

Canadian Public Policy

0.363

90

0.269

92

0.629

32

0.327

92

Acta Politica

0.361

91

0.319

90

0.379

91

0.474

78

Rivista Italiana di Scienza Politica

0.356

92

0.324

88

0.374

92

0.444

85

@*# Denotes journals associated with one of the three national political science associations for the countries from which our sample was selected: American Political Science Association ~USA!, Canadian Political Science Association ~Canada!, and Political Studies Association ~UK!.

Figure 1

Scatterplot for Relationship between USA Journal Impact and Canada Journal Impact

countries is quite high (␣ = 0.869), and a principal components factor analysis of these three country-specific scores reveals a single factor (eigenvalue = 2.398, variance explained = 0.799), with all factor loadings in excess of 0.85. Hence we find considerable similarity in journal impact across these three countries, even when there is some different ordering of journals, particularly at the highest levels. Journal Evaluations

r = 0.737

USA-Canada, USA-UK, and UK-Canada pairings, respectively. As one can readily see, there is a strong relationship among the journal impact measures for country pairings, with the correlations ranging from a low of 0.567 (for the United Kingdom and United States) to 0.787 (for the United Kingdom and Canada). There are some clear outliers in these scatterplots. The Canadian Journal of Political Science has a much lower impact score in the United States than in Canada (Figure 1), and Political Studies has a much lower impact score in the United States than in the United Kingdom (Figure 2). In comparing journal impact in the United Kingdom and Canada (Figure 3), we find that Canadian political scientists give much higher scores to the Canadian Journal of Political Science, Canadian Public Policy, and Canadian Public Administration, while political scientists from the United Kingdom give higher impact scores to Political Studies and Democratization than their Canadian counterparts. Despite these outliers, for the most part there is a reasonably strong relationship between journal impact scores among the three countries. In fact, the reliability for an additive scale based on the separate impact scores from these three

We remind the reader here that the impact scores that we described in the foregoing section are based on two components: (1) journal evaluations, based on respondents’ evaluations of 92 journals on a scale ranging from 0 (poor) to 10 (outstanding), and (2) journal familiarity, based on the proportion of respondents who indicate that they are familiar enough with a given journal to evaluate it. The cross-national differences and similarities in journal impact documented in the previous section can be decomposed into evaluation and familiarity components, so it is important to explore country-specific patterns on these two variables to consider the basis for cross-national differences in journal impact. We begin with journal evaluations. In Table 2 we report the mean evaluation scores for the top 30 journals, and in Figure 4 we present a histogram and kernel density plot of the distribution of mean evaluation scores for all 92 journals. A complete table of mean evaluations for all 92 journals can be found in Online Appendix 3. Turning first to Figure 4, we find that the distribution of mean journal evaluations is skewed to the right, with a relatively small number of journals evaluated very highly (i.e., above a mean of 7.0), a small number of journals evaluated as below average (i.e., below 5.0), and most journals falling into the mid-range (between 5.0 and 7.0). Political scientists in these three countries have identified an elite set of journals that are evaluated particularly favorably for the quality of their contributions to the scholarly literature in political science. What journals are evaluated most favorably? In previous studies Garand (1990) and Garand and Giles (2003) have found that some journals that are familiar to only narrow subsets of PS • October 2009 701


.............................................................................................................................................................................................................................................................

of Political Science (third), Journal of Politics (fourth), and the British Journal of Political SciScatterplot for Relationship between USA Journal ence (sixth)—and the leading subfield journals— Impact and UK Journal Impact International Organization (second), World Politics (fifth), Political Theory (seventh), and Comparative Political Studies (nineth). But the list also includes some journals that are ranked lower in terms of overall impact, especially Philosophy and Public Affairs and Political Analysis; these journals are thought of highly by the relatively small proportion of respondents who are familiar enough to provide an evaluation, but they are somewhat less familiar to political scientists than some other journals on the list. It is particularly interesting to note that the previous evaluation of the American Political Science Review reported by Garand and Giles (2003) was considerably lower than that reported in Table 2. In the 2002 survey, the APSR was ranked seventeenth in terms of its evaluation by American political scientists; for many political scientists this was a shockingly low mean evaluation for what is generally viewed as the flagship jourr = 0.563 nal of the discipline. However, in these results based on our 2007 survey, the APSR is ranked first in terms of its mean evaluation. Perhaps the relatively lower ranking in 2002 reflected critiFigure 3 cisms of the APSR that prevailed at the time Scatterplot for Relationship between Canada Journal among some political scientists, most notably the Impact and UK Journal Impact adherents to the Perestroika movement within the political science discipline (see, for example, Smith 2002). The remainder of the top 30 list includes a mix of journals that are both favorably evaluated and familiar to a large proportion of political scientists (e.g., International Studies Quarterly, Journal of Conflict Resolution), as well as journals that are highly evaluated for the quality of their published work but by relatively narrower readerships (e.g., History of Political Thought, Journal of Policy Analysis and Management, Political Behavior). For some journals, political scientists are both familiar with the journal and evaluate the quality of its contributions very favorably; these journals score highly on impact, evaluation, and familiarity measures. In other cases, a relatively small number of political scientists are familiar with a given journal, but the narrow readership evaluates the quality of scholarly contributions r = 0.782 in favorable terms; here the impact of the journal on the broader political science profession is limited by the small number of political scientists respondents earn surprisingly strong mean evaluations from these who are familiar with the journal. small groups of readers, and the result is that using only mean What about the effects of country on the evaluations of politevaluations overstates the discipline-wide impact of these jourical science journals? Do political scientists from the United States, nals. The results in Table 2 are closer to the conventional wisCanada, and the United Kingdom evaluate political science jourdom, though as before there are also some surprise results. First, nals differently? In Table 3 we report the mean evaluations for the journals among the top 10 are generally those that one would each of the 25 political science journals, ranked by journal impact expect to see in such a list, including the leading general journals— as reported in Table 1. The list provides a mix of general, subfield, American Political Science Review (ranked first), American Journal specialty, and country-specific journals that earn high journal Figure 2


.............................................................................................................................................................................................................................................................

Ta b l e 2

Political Scientists’ Subjective Evaluations, Top 30 Highest-Ranked Journals, All Countries Combined and Separately for Each Country, 2007 ALL COUNTRIES JOURNAL

Relative Impact

American Political Science Review

7.688

International Organization American Journal of Political Science

UNITED STATES Rank

Relative Impact

CANADA Rank

Relative Impact

UNITED KINGDOM Rank 9

Relative Impact

Rank

7.022

6

1

7.973

1

7.159

7.546

2

7.498

4

8.000

1

7.471

2

7.381

3

7.598

2

6.796

21

6.938

8

Journal of Politics

7.328

4

7.537

3

6.975

16

6.581

25

World Politics

7.186

5

7.161

5

7.522

2

7.121

4


7.135

6

7.145

6

7.147

10

7.131

3


7.111

7

6.826

10

7.467

3

7.538

1

Political Theory

7.104

8

7.110

7

7.367

4

7.015

7

Comparative Political Studies

6.944

9

6.953

8

7.287

5

6.759

13

Political Analysis

6.869

10

6.852

9

7.000

15

6.866

10


6.775

11

6.762

11

7.058

11

6.683

19

History of Political Thought

6.754

12

6.491

14

7.047

12

7.025

5

Comparative Politics

6.747

13

6.659

12

7.197

7

6.753

16


6.517

14

6.573

13

6.568

34

6.238

32


6.415

15

6.003

27

7.044

13

6.718

17


6.384

16

6.424

15

6.395

39

6.108

37


6.376

17

6.128

21

7.186

8

6.755

15

Journal of European Public Policy

6.358

18

5.566

38

6.761

23

6.756

14

Journal of Policy Analysis and Management

6.328

19

6.403

16

6.500

36

5.586

69

European Journal of International Relations

6.322

20

5.684

33

7.250

6

6.782

12


6.311

21

6.276

19

6.288

48

6.597

24

Journal of Law and Economics

6.307

22

6.269

20

6.879

19

6.229

33

Studies in American Political Development

6.292

23

6.356

17

6.313

46

5.158

84

Public Administration Review

6.272

24

6.313

18

6.911

18

5.835

51

Political Studies

6.268

25

5.452

47

6.636

29

6.874

9

Review of International Studies

6.250

26

5.470

43

7.000

14

6.842

11


6.184

27

6.063

26

6.747

26

6.178

34

Political Behavior

6.180

28

6.118

22

6.579

33

6.385

28

Review of International Political Economy

6.127

29

5.548

39

6.917

17

6.626

23


6.114

30

6.108

23

6.370

43

6.085

39

Note: Figures represent the mean evaluation score on a scale from 0 ~poor! to 10 ~outstanding!.

impact rankings. We report the overall mean evaluation for each of these journals, the mean evaluation for respondents from each of the three countries in our study, and the results of an analysis of variance (ANOVA) analysis of journal evaluations. Our ANOVA analysis has the 11-point journal evaluation scale as the dependent variable, with country serving as the independent variable. We find a wide variety of country-specific effects, ranging from strong country effects to null effects. First, the three highestranked journals—the American Political Science Review, American Journal of Political Science, and Journal of Politics—are generalinterest journals that are highly regarded by scholars from all three

countries. However, there is a significant difference in how political scientists from these three countries evaluate these three journals. In each case, the mean evaluation is highest for political scientists from the United States, and the result is an F test signifying that the differences in means for these three countries are statistically significant. Even though these journals are highly regarded for political scientists from all three countries, all three journals are associated with national and regional political science associations from the United States, and this results in a significantly higher evaluation for American political scientists. Second, there are several journals on our list that are associated with a specific country or that have a European identity, PS • October 2009 703


.............................................................................................................................................................................................................................................................

We are struck by the journals for which systematic differences among political scientists Distribution of Evaluations for 92 Political Science from different countries are not observed. In addiJournals on 11-point Scale, All Respondents tion to the British Journal of Political Science, the differences among political scientists are not statistically significant for World Politics, International Studies Quarterly, Journal of Conflict Resolution, Legislative Studies Quarterly, Political Theory, Political Analysis, Political Research Quarterly, Public Opinion Quarterly, and Journal ofTheoretical Politics. The intellectual and scholarly appeal of these journals is broad based and similar for political scientists from all three countries. Finally, how are journal evaluations among political scientists in one country related to journal evaluations among political scientists in other countries? We inspected the scatterplots of mean journal evaluations for the USA-Canada, USAUK, and UK-Canada pairings, respectively (see Online Appendix Figures 4–6). As is the case for journal impact, there is a strong relationship among the journal evaluation measures for each country pairing. The correlations are all quite high, ranging from 0.692 to 0.815, and there are Note: The curve overlaying the histogram is a kernel density function. relatively few outliers in each of the scatterplots. American political scientists evaluate the American Political Science Review, American Journal of Political Science, and the Journal of Politics more and political scientists typically differ in their evaluations of these highly than what would be predicted by journal evaluations by journals.The European Journal of Political Science and Political Studpolitical scientists from Canada and the United Kingdom, but for ies stand out in this regard. Political scientists from Canada and the most part journal evaluations in one country can be predicted the United Kingdom evaluate both of these journals more highly reasonably well by knowledge of journal evaluations in the other than do American political scientists, and in both cases these diftwo countries. Here again, the reliability for an additive scale based ferences are highly significant. Another journal that falls into this on the separate journal evaluation scores is high (␣ = 0.888), and category but that is not on the top-25 list is the Canadian Journal of a single factor emerges from a principal components factor analPolitical Science (ranked thirtieth in Table 1); in this case Canadian ysis of these three items (eigenvalue = 2.498, variance explained = political scientists evaluate this journal significantly more favor0.833). Across the full range of journals, we find a high degree of ably than political scientists from the United States and United similarity in how political scientists from the United States, CanKingdom. On the other hand, the British Journal of Political Science ada, and the United Kingdom evaluate these 92 scholarly journals. is evaluated equally by political scientists from each of the three Journal Familiarity countries, so it is not inevitable that a country designation differFigure 4

entiates political scientists in their evaluations of a journal. Third, there is a wide range of other journals for which there are significant differences for political scientists across the three countries. Several journals in the fields of comparative politics and international relations have evaluations that differ from one country to the next. International Organization, Comparative Politics, Comparative Political Studies, Foreign Affairs, and International Security all exhibit significant differences, with Canadian political scientists most likely to evaluate these journals favorably. Interestingly, Perspectives on Politics and PS: Political Science and Politics are evaluated more favorably by Canadian political scientists, exceeding the evaluation score given by political scientists from the United Kingdom and, most surprisingly, the United States. Political scientists from Canada and United Kingdom are more likely to evaluate the European Journal of Political Science, Electoral Studies, Political Studies, and Philosophy and Public Affairs more favorably than American political scientists; of these, only Political Studies is evaluated most favorably by political scientists from the United Kingdom. 704 PS • October 2009

In addition to respondents’ evaluations of the quality of articles published in a given journal, the impact of that journal in the political science discipline is also a function of the degree to which political scientists are familiar with the research found in articles published with a journal. In Figure 5 we present the histogram and kernel density plot for the distribution of familiarity scores for all 92 journals in our study, and in Table 4 we report the proportion of respondents who report being familiar with the top 30 journals. A complete table of journal familiarity results for all 92 journals and for all three countries can be found in Online Appendix 4. The histogram for our journal-familiarity variable reveals that the distribution is heavily skewed to the right, with only a few journals exhibiting familiarity levels in excess of 0.50. The lion’s share of journals is familiar to between 20% and 50% of respondents. It appears that a relatively small number of journals have the potential to make a discipline-wide impact, with most journals having a potential impact on selected subgroups of political scientists based on subfield, methodological approach, or other criteria.

.............................................................................................................................................................................................................................................................

Ta b l e 3

Mean Evaluations of Selected Political Science Journals, by Country of Respondent, 2007 JOURNAL

TOTAL

USA

CANADA

UK

F


7.688 ~1584!

7.973 ~1070!

7.159 ~170!

7.022 ~325!

27.08***

American Journal of Political Science

7.381 ~1482!

7.598 ~1023!

6.796 ~152!

6.938 ~289!

14.79***

Journal of Politics

7.328 ~1340!

7.537 ~992!

6.975 ~119!

6.581 ~215!

23.30***


7.135 ~1385!

7.145 ~887!

7.147 ~143!

7.131 ~337!

0.01


7.546 ~1127!

7.498 ~759!

8.000 ~113!

7.471 ~242!

3.66**

World Politics

7.186 ~1106!

7.161 ~753!

7.522 ~115!

7.121 ~224!

1.56


6.747 ~1176!

6.659 ~754!

7.197 ~137!

6.753 ~267!

4.59**


6.944 ~1087!

6.953 ~706!

7.287 ~122!

6.759 ~245!

3.88**


6.775 ~897!

6.762 ~618!

7.058 ~86!

6.683 ~183!

1.15


6.517 ~904!

6.573 ~667!

6.568 ~74!

6.238 ~151!

1.88

Perspectives on Politics

5.973 ~1115!

5.957 ~899!

6.632 ~87!

5.592 ~120!

5.53***

Political Analysis

6.869 ~642!

6.852 ~535!

7.000 ~33!

6.866 ~67!

0.09


6.384 ~823!

6.424 ~679!

6.395 ~43!

6.108 ~93!

1.41

Foreign Affairs

5.476 ~1215!

5.309 ~807!

6.112 ~125!

5.632 ~269!

6.75***

Political Theory

7.104 ~557!

7.110 ~356!

7.367 ~60!

7.015 ~135!

0.57


6.022 ~938!

6.068 ~765!

6.194 ~62!

5.573 ~103!

3.54**


6.415 ~733!

6.003 ~365!

7.044 ~91!

6.718 ~259!

20.57***

PS: Political Science and Politics

5.096 ~1301!

4.982 ~954!

5.890 ~136!

5.072 ~197!

10.10***

Electoral Studies

6.071 ~799!

5.792 ~547!

6.820 ~61!

6.575 ~179!

21.55***


6.376 ~676!

6.128 ~452!

7.186 ~70!

6.755 ~147!

9.90***


6.311 ~692!

6.276 ~561!

6.288 ~52!

6.597 ~72!

1.02

Political Studies

6.268 ~701!

5.452 ~279!

6.636 ~88!

6.874 ~318!

40.83***


7.111 ~378!

6.826 ~224!

7.467 ~45!

7.538 ~106!

4.22**


6.184 ~667!

6.063 ~416!

6.747 ~91!

6.178 ~152!

4.69***


6.114 ~676!

6.108 ~481!

6.370 ~46!

6.085 ~142!

0.48

Note: Figures represent the mean evaluation score on a scale from 0 ~poor! to 10 ~outstanding!. Figures in parentheses are the number of respondents used to calculate the mean evaluation score. *** prob~ F ! < .01 ** prob~ F ! < .05 * prob~ F ! < .10



.............................................................................................................................................................................................................................................................

These results reveal considerable differences in journal familiarity for political scientists from Distribution of Levels of Familiarity for 92 Political the United States, United Kingdom, and CanScience Journals, All Respondents ada. Of the 25 top-impact journals, 19 exhibit a significant difference (at the .05 level ) in familiarity levels across countries. This includes the “big three” journals (i.e., American Political Science Review, American Journal of Political Science, and the Journal of Politics), other general journals (e.g., Perspectives on Politics, Political Research Quarterly), various high-impact subfield journals (e.g., International Organization, International Studies Quarterly), and specialty journals (e.g., Legislative Studies Quarterly, Political Analysis). The only high-impact journals that are equally familiar to political scientists from all three countries are the British Journal of Political Science, Comparative Politics, Comparative Political Studies, PoliticalTheory, International Security, and Philosophy and Public Affairs. Three of these journals are explicitly cross-national in focus, and two are in the field of political theory and philosophy. It is noteworthy that among the 19 journals for which there is a significant difference in familNote: The curve overlaying the histogram is a kernel density function. iarity for political scientists from different countries, the level of familiarity is highest among American political scientists for 16 of these journals. The only exceptions are the European JourAmong the 30 journals with which the respondents are most nal of Political Science and Political Studies (both of which are more familiar, we find that there are only 15 journals for which a majorfamiliar to political scientists from the UK and Canada) and Polity of respondents indicate familiarity. The American Political Sciitics and Society (which is more familiar to Canadian political science Review is the journal with the highest level of familiarity, entists). This may partially be explained by a general USA effect, with 93.4% of respondents indicating that they are familiar with whereby the overall level of familiarity is higher among American the flagship journal of the discipline. Four other journals are familpolitical scientists than those from other countries. Across all 92 iar to at least 75% of political scientists—the American Journal of journals, the mean level of familiarity is 34.6%, with political sciPolitical Science (87.5%), British Journal of Political Science (81.6%), entists from the United States showing the highest familiarity Journal of Politics (79.1%), and PS: Political Science and Politics (35.8%), followed by those from Canada (34.4%) and the United (76.7%). These are the journals with the potential to have the stronKingdom (32.1%). Even among the top 25 journals, the overall gest overall impact on the discipline; in fact, four of these five mean familiarity is 47.5%, with political scientists from the United journals are also among the most favorably evaluated journals, so States (49.9%) exhibiting higher familiarity than those from Canthis helps to explain their strong overall impact (Table 1). Among ada (44.5%) and the United Kingdom (43.2%). This USA effect is the journals familiar to over 50% of respondents, there is a mix of likely due to the fact that a large proportion of high-impact jourgeneral interest journals (e.g., Perspectives on Politics, Political nals originates or is published in the United States. Research Quarterly, Political Science Quarterly) and major subfield We also consider the interrelationships for journal-familiarity journals (e.g., Comparative Politics, International Organization, Comlevels among political scientists from the United States, Canada, parative Political Studies). Finally, the remainder of the list comand the United Kingdom. We examined the scatterplots for the prises journals familiar to over 40% of respondents, including relationships between journal familiarity levels for country pairs specialty journals (e.g., Legislative Studies Quarterly, Electoral Stud(see Online Appendix Figures 7–9), and it is here that we begin to ies, Public Choice), a few general interest journals (e.g., Polity, Social observe a slight deterioration in the strength of the relationships Science Quarterly), and journals with an audience targeted to a between countries. The correlations for journal familiarity are specific country (e.g., Canadian Journal of Political Science, Political lower than those for journal impact and journal evaluations, rangStudies). Beyond these journals, most journals are familiar to less ing from a low of 0.453 (for the USA-UK relationship) to a high of than 40% of political scientists in these three countries. 0.734 (for the United Kingdom and Canada). Clearly, political sciHow do levels of journal familiarity vary for political scientists entists from the United States and the United Kingdom do not across the three countries in our study? In Table 5 we report the share the same levels of familiarity for many scholarly journals, proportion familiar for each of the top 25 high-impact political while political scientists from the United Kingdom and Canada science journals (as reported in Table 1), both in total and broken tend to exhibit similar levels of journal familiarity. down by country. Furthermore, we include in Table 5 the likeliFinally, even though the cross-national interrelationships hood ratio (LR) ␹ 2 test of the cross-national difference in the proinvolving journal familiarity are not as strong as those involving portion of respondents who are familiar with each journal. journal evaluations and journal impact, the similarity in journal Figure 5


.............................................................................................................................................................................................................................................................

Ta b l e 4

Political Scientists’ Journal Familiarity, Top 30 Highest-Ranked Journals, All Countries Combined and Separately for Each Country, 2007 ALL COUNTRIES JOURNAL

Proportion Familiar

UNITED STATES

Rank

Proportion Familiar

CANADA

Rank

Proportion Familiar

UNITED KINGDOM Rank

Proportion Familiar

Rank


0.935

1

0.984

1

0.950

2

0.806

2

American Journal of Political Science

0.874

2

0.941

2

0.849

3

0.717

5


0.817

3

0.816

6

0.799

4

0.836

1

Journal of Politics

0.791

4

0.913

3

0.665

10

0.533

16

PS: Political Science and Politics

0.768

5

0.878

4

0.760

6

0.489

19

Foreign Affairs

0.717

6

0.742

7

0.698

8

0.667

6


0.694

7

0.694

10

0.765

5

0.663

7


0.665

8

0.698

9

0.631

12

0.600

10

Perspectives on Politics

0.658

9

0.827

5

0.486

20

0.298

46

World Politics

0.653

10

0.693

11

0.642

11

0.556

15


0.641

11

0.649

12

0.682

9

0.608

9


0.553

12

0.704

8

0.346

36

0.256

49


0.533

13

0.614

14

0.413

28

0.375

33

Political Science Quarterly

0.532

14

0.614

15

0.402

29

0.382

31


0.529

15

0.569

18

0.480

21

0.454

24


0.486

16

0.625

13

0.240

60

0.231

53

Electoral Studies

0.471

17

0.503

23

0.341

37

0.444

26

American Politics Research

0.470

18

0.611

16

0.263

50

0.191

61

Annals of the American Academy

0.467

19

0.516

21

0.458

22

0.335

38

Journal of Democracy

0.458

20

0.473

26

0.497

18

0.404

28

International Political Science Review

0.457

21

0.406

31

0.721

7

0.479

23

Polity

0.452

22

0.542

19

0.374

34

0.253

50

Canadian Journal of Political Science

0.445

23

0.383

32

0.955

1

0.395

29

Social Science Quarterly

0.443

24

0.593

17

0.285

47

0.127

73


0.432

25

0.336

40

0.508

16

0.643

8

Public Choice

0.424

26

0.520

20

0.324

42

0.223

57

Government and Opposition

0.416

27

0.288

49

0.458

23

0.732

4

Political Studies

0.414

28

0.257

53

0.492

19

0.789

3

Governance

0.412

29

0.367

35

0.542

14

0.481

21


0.408

30

0.516

22

0.291

45

0.179

65

Note: Figures represent the proportion of respondents who are sufficiently familiar with a given journal to provide an evaluation.

familiarity is strong enough that the familiarity variables for the United States, Canada, and the United Kingdom fit together well into a scale. The reliability for an additive scale based on the separate journal-familiarity scores is lower than that for journal impact and evaluation measures, but is still quite high (␣ = 0.823). Moreover, a single factor emerges from a principal components factor analysis of these three items (eigenvalue = 2.235, variance explained = 0.745). Although the interrelationships between journal familiarity levels for the United States, Canada, and the United Kingdom are not as strong as those observed for journal impact and journal evaluation, there remains a fairly high level of simi-

larity in the extent to which political scientists from these three countries are familiar with scholarly journals. The Relationship between Journal Evaluation and Journal Familiarity How closely are journal evaluation and journal familiarity related to one another? Are journals that receive a favorable evaluation from political scientists also the most familiar to political scientists? In Figure 6 we present a scatterplot of the relationship between these two variables for our full sample. The relationship is moderately strong for the combined sample (r = 0.553), and the PS • October 2009 707


.............................................................................................................................................................................................................................................................

Ta b l e 5

Familiarity with Selected Political Science Journals, by Country of Respondent, 2007 JOURNAL American Political Science Review American Journal of Political Science Journal of Politics British Journal of Political Science International Organization World Politics Comparative Politics Comparative Political Studies International Studies Quarterly Journal of Conflict Resolution Perspectives on Politics Political Analysis Legislative Studies Quarterly Foreign Affairs Political Theory Political Research Quarterly European Journal of Political Research PS: Political Science and Politics Electoral Studies International Security Public Opinion Quarterly Political Studies Philosophy and Public Affairs Politics and Society Journal of Theoretical Politics Total number of observations

TOTAL

USA

CANADA

UK

LR ␹ 2

0.935 ~1,584!

0.984 ~1,070!

0.950 ~170!

0.806 ~325!

136.27*** 127.45***

0.874

0.941

0.849

0.717

~1,482!

~1,023!

~152!

~289!

0.791

0.913

0.665

0.533

~1,340!

~992!

~119!

~215!

0.817

0.816

0.799

0.836

~1,385!

~887!

~143!

~337!

0.665

0.698

0.631

0.600

~1,127!

~759!

~113!

~242!

0.653

0.693

0.642

0.556

~1,106!

~753!

~115!

~224!

0.694

0.694

0.765

0.662

~1,176!

~754!

~137!

~267!

0.641

0.649

0.682

0.608

~1,087!

~706!

~122!

~245!

0.529

0.569

0.480

0.454

~897!

~618!

~86!

~183!

0.533

0.614

0.413

0.375

~904!

~667!

~74!

~151!

0.658

0.827

0.486

0.298

~1,115!

~899!

~87!

~120!

0.379

0.492

0.184

0.166

~642!

~535!

~33!

~67!

0.486

0.625

0.240

0.231

~823!

~679!

~43!

~93!

0.717

0.742

0.698

0.667

~1,215!

~807!

~125!

~269!

0.329

0.328

0.335

0.335

~557!

~356!

~60!

~135!

0.553

0.704

0.346

0.256

~938!

~765!

~62!

~103!

0.432

0.336

0.508

0.643

~733!

~365!

~91!

~259!

0.768

0.878

0.760

0.489

~1301!

~954!

~136!

~197!

0.471

0.503

0.341

0.444

~799!

~547!

~61!

~179!

0.399

0.416

0.391

0.365

~676!

~452!

~70!

~147!

0.408

0.516

0.291

0.179

~692!

~561!

~52!

~72!

0.414

0.257

0.492

0.789

~701!

~279!

~88!

~318!

0.223

0.206

0.251

0.263

~378!

~224!

~45!

~106!

0.394

0.382

0.508

0.377

~667!

~416!

~91!

~152!

0.399

0.443

0.257

0.352

~676!

~481!

~46!

~142!

1,695

1,087

179

403

272.63*** 2.35 15.09*** 24.43*** 5.73* 2.68 18.93*** 80.00*** 402.59*** 177.50*** 244.32*** 9.73*** 0.12 288.07*** 122.52*** 233.37*** 16.75*** 3.74 162.23*** 361.30*** 5.56* 7.71** 30.05***

Note: Figures represent the proportion of respondents who are sufficient familiar with a journal to provide an evaluation. Figures in parentheses are the number of respondents used to calculate the mean evaluation score. ***prob~ F ! < .01 **prob~ F ! < .05 *prob~ F ! < .10


.............................................................................................................................................................................................................................................................

up to three journals to which they would like to submit their best work. In Table 6 we report the Scatterplot for Relationship between Journal Evaluation number of first, second, and third references, as and Journal Familiarity well as the number of total mentions. We also report the number of “ratings points” based on three points for a first reference, two points for a second reference, and one point for a third reference. As one can see from Table 6, there is again a clear-cut ranking of political science journals, with many of the highly ranked journals on this list also found among the elite journals from the close-ended items reported in Table 1. The top five journals to which political scientists prefer to send their best work are the American Political Science Review, American Journal of Political Science, Journal of Politics, International Organization, and World Politics, followed by the International Studies Quarterly, British Journal of Political Science, Comparative Politics, Comparative Political Studies, and Political Theory. Not surprisingly, the journals to which political scientists prefer to send their best work are also the journals that earn high impact ratings in the closer = 0.554 ended items. When political scientists are thinking about where to send their best work, correlations calculated separately for political scientists from each they consider the leading general journals and broad-based subcountry range from 0.376 (for Canada) to 0.624 (for the United field journals. States). (See Online Appendix Figures 10–12.) Overall, it appears Two other aspects of this table warrant attention. First, we that the journals that are most favorably evaluated are typically point out that political scientists vary considerably in how they the ones that are most familiar to political scientists in the United prioritize political science journals as possible outlets of their work. States, Canada, and the United Kingdom. Respondents mentioned 201 different journals as their first choice There are some important outliers, however. Some journals to submit their best work, with 277 different journals mentioned are well known to political scientists but have only moderate evalas their second choice and 323 different journals mentioned third. uations; PS: Political Science and Politics, Foreign Affairs, PerspecMany of these journals are mentioned only once or twice, but the tives on Politics, Presidential Studies Quarterly, and Political Science fact remains that political scientists see a wide variety of possible are journals with mean evaluations that are below what one would outlets for their best research papers. predict based on their level of familiarity to political scientists. Second, we note the dominant position of the American PolitThese journals earn higher impact scores based on their wideical Science Review, American Journal of Political Science, and (to a spread familiarity among political scientists, rather than on politlesser extent) the Journal of Politics on this list. The second-place ical scientists’ evaluations of the importance and quality of journal, the American Journal of Political Science, has only 54.1% scholarly work that they publish. On the other hand, some jourof the rating points as the American Political Science Review, and nals are familiar only to moderate numbers of political scientists the Journal of Politics has only 34.7% of the rating value of the but have mean evaluations that are above what would be preAPSR. The journals International Organization and World Politics dicted by their level of familiarity; these journals include Political have 22.3% and 21.0% of the rating value of the APSR, but no Analysis, Political Theory, History of Political Thought, and Philosother journal approaches 20% of the rating points of the APSR. ophy and Public Affairs. Journals in this category are known to When political scientists think about the highest-priority outlets more specialized audiences who give them high marks for the for their best work, the American Political Science Review, Ameriquality of the work that they publish. can Journal of Political Science, Journal of Politics, International Organization, and World Politics dominate the list. In fact, these five Preferred Journal Submissions journals represent 55% (886 of 1,597) of the first-choice journals In addition to closed-ended questions, Garand and Giles (2003) to which scholars prefer to send their high-quality work, and the report the results from open-ended questions in which they ask top 20 journals represent 75.5% (1,205 of 1,597) of the most prerespondents to which journal they would submit a “very strong ferred outlets for their best work. While political scientists cite paper” that they had written in their field of expertise. This quesquite a few different journals as journals to which they prefer to tion is designed to give respondents an alternative way of thinksubmit high-quality manuscripts, the fact is that there are a small ing about journal evaluations by eliciting from them information number of journals that are cited by a high percentage of politiabout how they prioritize the journals in which they would like to cal scientists. see their own research published. We include a similar item in our Political scientists from our three countries exhibit some inter2007 survey, and we gave respondents the opportunity to indicate esting differences in terms of the journals to which they prefer to Figure 6



.............................................................................................................................................................................................................................................................

Ta b l e 6

Political Scientists’ Preferences for Journal Submissions of High-Quality Manuscript, All Countries Combined, 2007 JOURNAL

1ST

2ND

3RD

TOTAL MENTIONS

RATING POINTS

RELATIVE RATING

1. American Political Science Review

595

80

40

715

1985

1.000

2. American Journal of Political Science

109

343

71

523

1084

0.546

3. Journal of Politics

46

142

265

453

687

0.346

4. International Organization

89

72

42

203

453

0.228

5. World Politics

87

58

47

192

424

0.214

6. International Studies Quarterly

35

54

53

142

266

0.134

7. British Journal of Political Science

33

44

52

129

239

0.120

8. Comparative Politics

30

49

37

116

225

0.113

9. Comparative Political Studies

18

46

69

133

215

0.108

10. Political Theory

43

30

18

91

207

0.104

11. Political Studies

29

33

24

86

177

0.089

12. European Journal of International Relations

17

24

18

59

117

0.059

13. Review of International Studies

18

19

25

62

117

0.059

14. International Security

28

10

11

49

115

0.058

15. Journal of Common Market Studies

20

10

8

38

88

0.044

16. Journal of Political Philosophy

12

16

19

47

87

0.044

17. Public Administration Review

19

9

7

35

82

0.041

9

19

12

40

77

0.039

18. European Journal of Political Research 19. Ethics

14

13

6

33

74

0.037

20. Perspectives on Politics

11

12

16

39

73

0.037

203

282

328

Total number of journals receiving one or more mentions

Note: The entries represent the number of respondents who report the journal as their first, second, or third preferences for submission of a high-quality manuscript. Rating points is based on giving three points for a first rating, two points for a second rating, and one point for a third rating. Relative rating is the rating points for each journal as a share of the rating points for the American Political Science Review.

send their best work. In Table 7 we report the top five journalsubmission preferences for political scientists from the United States, United Kingdom, and Canada. As one can see, the ranking for political scientists from the United States closely tracks the overall impact ranking presented in Table 1. American political scientists prefer to send their best work to the American Political Science Review, American Journal of Political Science, Journal of Politics, International Organization, and World Politics; the only journal missing from those with the highest impact (Table 1) is the British Journal of Political Science. For political scientists from the United Kingdom and Canada, there are some differences. Canadian political scientists drop from their top five the Journal of Politics, add the Canadian Journal of Political Science, and shuffle the order for other journals, though the American Political Science Review is also the journal to which Canadian political scientists would prefer to send their best work. Political scientists from the United Kingdom are the outliers here. The American Political Science Review and International Organization are ranked third and fourth as preferred scholarly outlets. However, Political Studies and the British Journal of Political Science earn the top two positions, and the Review of International Studies captures the fifth position. Overall, it appears that political scientists from Canada and the United States have similar preferences about where they should 710 PS • October 2009

send their best work, but political scientists from the United Kingdom have less overlap with their colleagues from across the Atlantic. Preferred Reading Sources An alternative way of determining political scientists’ journal priorities is to ask respondents to identify the journals that they “read regularly or otherwise rely on for the best research” in their areas of scholarly interest. We included a question in our 2007 survey to capture these journal-reading priorities. In Table 8 we report the results for this item, coding up to five responses for each survey respondent. We also report the number of total mentions, as well as the number of rating points based on five points for a first reference, four points for a second, reference, and so on. Here again, we see a fairly clear pattern of journal ratings. The ranking of the first five journals on this list—the American Political Science Review, American Journal of Political Science, Journal of Politics, International Organization, and World Politics—is identical to the ranking for the journal-submission preferences, and the second five journals are the same, albeit with a slightly different ranking. There is clearly a strong similarity in the results for journal-submission and journal-reading preferences. Indeed, for

.............................................................................................................................................................................................................................................................

Ta b l e 7

Political Scientists’ Preferences for Journal Submissions of High-Quality Manuscript, Top Five by Country, 2007 JOURNAL

1ST

2ND

3RD

TOTAL MENTIONS

RATING POINTS

RELATIVE RATING

United States 1. American Political Science Review

523

73

37

633

1,752

1.000


92

310

68

470

964

0.550


42

132

249

423

639

0.365


71

41

39

151

334

0.191

5. World Politics

51

55

34

140

297

0.170

Canada 1. American Political Science Review

30

5

0

35

100


17

8

4

29

71

3. Canadian Journal of Political Science

13

10

8

31

67

0.670


7

15

1

23

52

0.520

5. World Politics

6

8

2

16

36

0.360

1.000 0.710

United Kingdom 1. Political Studies

27

30

21

78

162

1.000


30

21

17

68

149

0.920


38

2

2

42

120

0.741


21

9

3

33

85

0.525


12

13

19

44

81

0.500

Note: The entries represent the number of respondents who report the journal as their first, second, or third preferences for submission of a high-quality manuscript. Rating points is based on giving three points for a first rating, two points for a second rating, and one point for a third rating. Relative rating is the rating points for each journal as a share of the rating points for top-rated journal in each country ~i.e., the American Political Science Review for the United States and Canada, and Political Studies for the United Kingdom!.

the 64 journals with at least five references on each of the journalsubmission and journal-reading items, the correlation is almost perfect (r = 0.985). Once again we find that political scientists cite a large number of journals for their first, second, third, fourth, or fifth preferences for journal reading. Political scientists rely on many different journals for the best work in their fields of study. However, many of these journals are cited by only one or a few political scientists. As is the case for journal-submission preferences, there is a small group of journals that dominate the list of journal-reading preferences, though there is a wider distribution of journals that political scientists read than to which they submit their work. We find that the top five journals make up 41.7% (662 of 1,587) of the first preferences, and the top 20 journals make up 62.0% (984 of 1,587) of the first preferences. There is also a bit smaller drop-off as one moves from the American Political Science Review to lower-ranked journals. The American Journal of Political Science has 68.1% of the rating value of the APSR, and the Journal of Politics has 47.5% of the APSR rating value. Other journals among the top 20 journals have a journal-reading rating value that is higher than the journal-submission rating value reported in Table 6. The bottom line is that political scientists spread their journal-reading preferences around just a bit more than their journal-submission preferences, though there is a discernible rank order of journals in terms of political scientists’ reading preferences. Political scientists are somewhat more “elitist” in terms of where they would submit their best manuscripts than in where they read for the best work in their fields of study.

Finally, in Table 9 we report the top five journal-reading preferences for political scientists from the United States, United Kingdom, and Canada, and once again we find that political scientists differ across country in the journal-reading preferences. The ranking for scholars from the United States is nearly identical for journal-submission and journal-reading preferences; the only difference is that World Politics and International Organization switch positions in Table 9. Canadian political scientists retain the same top five journals but shuffle the positions of the top three; the Canadian Journal of Political Science is the most read journal by Canadian political scientists, followed by the American Political Science Review and International Organization. Finally, for political scientists from the United Kingdom, the Journal of Common Market Studies replaces the International Organization in the fourth position, with the other four journals retaining their lofty rankings. Here again, political scientists from the United Kingdom are outliers, sharing only two of their five top journals in terms of reading preferences with their American and Canadian counterparts. A Multivariate Analysis of Journal Evaluations What explains variation in individuals’ evaluations of political science journals? One of the questions left unresolved in this analysis is whether country differences (or similarities) are really due to the effects of other variables. For instance, Blais et al. (2008) find that there are systematic differences among political scientists from different countries in terms of the methodological approach that they adopt for their research and professional work. Garand and Giles (2003) find that methodological approach is PS • October 2009 711


.............................................................................................................................................................................................................................................................

Ta b l e 8

Political Scientists’ Preferences for Journal Reading, All Countries Combined, 2007 JOURNAL 1. American Political Science Review

1ST

2ND

3RD

4TH

5TH

TOTAL MENTIONS

RATING POINTS

RELATIVE RATING

399

98

81

74

59

711

2,837

1.000

91

273

87

48

37

536

1,941

0.684


52

95

191

45

42

425

1,345

0.474


85

54

42

17

29

227

830

0.293


5. World Politics

70

57

47

21

22

217

783

0.276

6. International Studies Quarterly

48

45

38

39

25

195

637

0.225

7. Comparative Political Studies

26

41

51

59

29

206

594

0.209


19

27

58

49

35

188

510

0.180

9. Comparative Politics

24

34

49

22

20

149

467

0.165

10. Political Theory

46

23

16

14

11

110

409

0.144

11. Political Studies

29

16

27

30

18

120

368

0.130

6

10

28

61

41

146

317

0.112

12. Political Research Quarterly 13. International Security

31

17

14

19

11

92

314

0.111

14. Legislative Studies Quarterly

16

24

10

41

17

108

305

0.108

15. European Journal of International Relations

21

20

20

15

11

87

286

0.101

16. Journal of Conflict Resolution

16

22

19

18

12

87

273

0.096


14

23

20

11

14

82

258

0.091

18. Perspectives on Politics

11

15

27

15

15

83

241

0.085


17

18

12

10

7

64

220

0.078

20. Journal of Political Philosophy

10

17

18

13

2

60

200

0.071

250

309

337

349

337

Total number of journals receiving one or more mentions

Note: The entries represent the number of respondents who report the journal as their first, second, or third preferences as one that they “read regularly or otherwise rely on for the best research” in their areas. “Rating points” is based on giving five points for a first rating, four points for a second rating, three points for a third rating, and so on. “Relative rating” is the rating points for each journal as a share of the rating points for the American Political Science Review.

related to the evaluations that political scientists give to some journals, particularly those that adopt a clear methodological identity. It is possible that country differences in journal evaluations may actually reflect the distribution of methodological preferences among political scientists from those countries. Other confounding effects may be possible as well, so it is important to explore the influence of country on journal evaluations, controlling for the effects of other variables. We developed and tested a regression model for a small, select group of political science journals. Obviously, we cannot report regression results for all 92 journals, so we selected journals for inclusion in this analysis to represent a mix of general, subfield, specialty, and country-specific journals. Our models include several sets of independent variables: (1) subfield variables, representing political scientists’ subfield designations, with comparative politics the excluded category; (2) methodological-approach variables, including quantitative methods, qualitative methods, and formal theory, with normative theory the excluded comparison category; (3) demographic attributes of age and gender; and (4) country variables, represented by separate binary variables for the United Kingdom and Canada and with the United States the excluded category. This model permits us to estimate the effects of country on journal evaluations, controlling for other variables found by Garand and Giles (2003) to be related to general journal evaluations. 712 PS • October 2009

In Table 10 we report the results for our regression models for 14 journals, including the American Political Science Review, American Journal of Political Science, Journal of Politics, British Journal of Political Science, International Organization, World Politics, Comparative Politics, Comparative Political Studies, International Studies Quarterly, Journal of Conflict Resolution, Perspectives on Politics, Political Theory, Canadian Journal of Political Science, and Political Studies. Turning first to the country-specific variables, we find that only 25% (seven of 28) of the country coefficients are statistically significant at conventional levels. Hence there is relatively little evidence that political scientists from the United States, Canada, and the United Kingdom differ in their evaluations of these political science journals, once we control for the effects of other variables in our model. British political scientists are slightly more favorably oriented to the American Journal of Political Science and, as one might expect, Political Studies, and they are less favorable toward the Journal of Politics, World Politics, Comparative Political Studies, Perspectives on Politics, Political Theory, and the Canadian Journal of Political Science. Canadian political scientists do not differ from American political scientists in their evaluations of any of our 14 journals, though the Canada coefficient for the Political Studies is positive and significant at a more relaxed .10 level of significance. These results suggest that any simple differences that are observed in how political scientists from the United States, Canada, and the United Kingdom evaluate scholarly journals are

.............................................................................................................................................................................................................................................................

Ta b l e 9

Political Scientists’ Preferences for Journal Reading, Top Five by Country, 2007 JOURNAL

1ST

2ND

3RD

4TH

5TH

TOTAL MENTIONS

RATING POINTS

RELATIVE RATING

United States 1. American Political Science Review

351

82

64

60

48

605

2,443

1.000


80

244

76

38

30

468

1,710

0.700


49

85

181

37

35

387

1,237

0.506

4. World Politics

58

42

34

16

16

166

608

0.249


51

39

36

11

20

157

561

0.230

Canada 1. Canadian Journal of Political Science

20

10

8

7

8

53

186

1.000


20

7

10

5

6

48

174

0.935


19

9

1

1

3

33

139

0.747


6

13

3

3

2

27

99

0.532

5. World Politics

6

7

5

4

3

25

84

0.452

United Kingdom 1. Political Studies

27

14

22

21

17

101

316

1.000


14

12

19

12

7

64

206

0.652


23

9

7

9

5

53

195

0.617


15

15

9

8

6

53

184

0.582


11

18

11

6

9

55

181

0.573

Note: The entries represent the number of respondents who report the journal as their first, second, or third preferences as one that they “read regularly or otherwise rely on for the best research” in their areas. “Rating points” is based on giving five points for a first rating, four points for a second rating, three points for a third rating, and so on. Relative rating is the rating points for each journal as a share of the rating points for top-rated journal in each country ~i.e., the American Political Science Review for the United States, Political Studies for the United Kingdom, and the Canadian Journal of Political Science for Canada!.

due primarily to the effects of other variables, such as subfield or methodological orientation. We observe quite a few significant coefficients for the subfield variables in our models, though on balance only 28.6% (48 of 168) of the subfield coefficients achieve statistical significance at the .05 level. Scholars who list international politics as a subfield are more favorable than comparative politics scholars (who comprise the excluded comparison group) in how they evaluate International Organization, International Studies Quarterly, and the Journal of Conflict Resolution, three international relations journals that are among the top 10 journals in terms of overall impact; on the other hand, they are less favorable in their evaluations of the Journal of Politics, the British Journal of Political Science, and World Politics. We find that American politics scholars tend to evaluate the Journal of Politics in favorable terms but are significantly less positive in their evaluations of International Organization and Comparative Political Studies. Scholars who list Canadian politics and British politics as one of their subfields of interest are more likely to evaluate the Canadian Journal of Political Science and Political Studies in positive terms, and scholars specializing in Canadian politics give more positive evaluations to the British Journal of Political Science, Comparative Politics, and the International Studies Quarterly. Moreover, area studies and comparative politics scholars differ in their evaluations of some journals; specifically, areastudies scholars are significantly less favorable in their evaluations of the American Political Science Review but are more favorable in their evaluations of World Politics, Comparative Politics, and Comparative Political Studies. Finally, political theorists exhibit quite a few significant differences from comparative politics scholars; theorists give less favorable evaluations to the American Journal of Polit-

ical Science, International Organization, Comparative Political Studies, and Journal of Conflict Resolution, but they are more favorably oriented toward Perspectives on Politics, Political Theory, the Canadian Journal of Political Science, and Political Studies. Of the four clusters of independent variables in our models, our methodological-approach variables have the most consistent effects on journal evaluations, with 50% (21 of 42) of the coefficients surpassing standard levels of statistical significance. Further, at least one methodological-approach variable achieves statistical significance for 12 of the 14 journal models in Table 10. There are some interesting patterns of coefficients for our methodological-approach variables, and these coefficients can help to map the methodological profile of our various journals. We find that quantitative scholars give systematically more favorable evaluations to the American Political Science Review, American Journal of Political Science, Journal of Politics, and British Journal of Political Science than do normative theorists, who comprise the excluded (comparison) group; these general-interest journals are significantly more positively evaluated by scholars with some quantitative orientation than other scholars. Qualitative scholars differ significantly from normative theorists in how they evaluate several journals. Scholars adopting a qualitative research orientation are significantly less favorable in their evaluations of the American Political Science Review, American Journal of Political Science, Journal of Politics, British Journal of Political Science, and the Journal of Conflict Resolution; each of these journals publishes quite a bit of quantitative research, and this emerges in how qualitative scholars evaluate these journals. On the other hand, qualitative scholars give more favorable evaluations to World Politics, Comparative Politics, and Comparative Political Studies, at least PS • October 2009 713


.............................................................................................................................................................................................................................................................

Ta b l e 1 0

OLS Regression Estimates for Models of Journal Evaluations, Selected Journals, All Countries Combined, 2007 APSR

Intercept

AJPS

JOP

BJPS

b

t

b

t

b

t

b

t

7.824

24.28***

7.187

22.92***

7.936

27.83***

6.491

23.84***

3.37***

−0.237

Subfield American politics

−0.081

−0.55

0.184

1.30

0.421

Canadian politics

0.041

0.14

0.117

0.42

−0.048

−0.19

0.689

British politics

0.142

0.55

0.121

0.48

0.258

1.07

0.318

1.63

Area studies

−0.338

−2.12**

−0.249

−1.58

−0.289

−1.92*

0.026

0.19

International politics

−0.159

−1.11

−0.086

−0.60

−0.390

−2.94***

0.342

2.52**

Political behavior

0.338

2.07**

0.355

2.27**

−1.92* 2.87***

−0.377

−3.08***

0.307

2.35**

Judicial politics

−0.172

−0.86

−0.149

−0.78

−0.114

−0.68

−0.348

−2.04**

Political theory

−0.000

−0.00

−0.478

−2.78***

−0.193

−1.22

−0.060

−0.40

0.084

0.55

0.002

0.01

0.110

0.85

0.168

1.34

−0.333

−1.51

0.002

0.01

−0.052

−0.27

−0.395

−2.13**

Research methodology Public administration Public policy Other subfield

0.290

1.91*

−0.238

−1.63

0.205

1.38

−0.273

−1.91*

0.279

2.09**

−0.211

−1.59

0.230

1.87*

0.032

0.27

Methodological Approach Quantitative methods Qualitative methods Formal theory

1.072

7.14***

1.497

10.17***

0.813

6.07***

0.548

−0.502

−3.55***

−0.851

−6.22***

−0.707

−5.85***

−0.236

−2.01**

0.412

2.07**

1.72*

−0.506

−2.95***

−0.155

−0.97

−3.71***

0.324

4.31***

Demographic Attributes Age

−0.007

−1.27

−0.007

−1.47

−0.017

Gender

−0.031

−0.24

0.101

0.79

0.024

United Kingdom

−0.398

−2.38**

Canada

−0.379

−1.76*

0.20

0.009

2.11**

0.192

1.74*

Country Variables

N R

0.284

1,535

2

b Intercept

−0.21

−0.422

−2.63***

−0.021

−0.11

1,432

0.163

F

1.70*

−0.046

1,345 0.095

0.272

0.257

27.81***

23.24***

IO

WP

CP

t

b

7.28***

CPS

t

b

t

18.87***

6.772

20.65***

24.36***

7.209

American politics

−0.618

−3.93***

−0.282

−1.61

−0.163

Canadian politics

0.557

1.63

0.419

1.16

0.633

British politics

0.269

0.93

0.048

0.14

−0.152

−0.051

−0.34

1.09 −0.25

1,295

15.48***

8.254

0.149 −0.047

b

t

7.594

24.58***

−0.617

−4.35***

Subfield

Area studies International politics Political behavior

0.423

3.12***

0.825

4.94***

−0.442

−2.90***

0.334 −0.131

−1.10 2.38** −0.63 2.30** −0.95

0.455 −0.117 0.358 −0.221

1.82* −0.52 2.61** −1.69*

0.023

0.13

0.095

0.48

0.169

1.04

0.161

1.06

Judicial politics

−0.282

−1.25

−0.278

−1.11

−0.228

−1.06

−0.261

−1.24

Political theory

−0.593

−3.32***

Research methodology Public administration Public policy Other subfield

0.065

0.41

−0.379

−1.82*

−0.023

−0.13

−0.498

−2.76***

−0.401

−2.27**

−0.302

−1.97**

−0.149

−1.07

−1.09

−0.244

−1.00

−0.347

−1.56

0.236

1.55

0.064

0.45

−0.041

−0.30

−0.586

−2.20**

−0.320

0.219

1.36

0.535

−0.207

−2.06**

−0.368

−2.15**

−0.299

−2.08**

−0.132

−0.92

2.92***

Methodological Approach Quantitative methods

0.184

1.23

0.012

0.07

Qualitative methods

0.062

0.41

0.492

2.91***

−0.63

−0.075

Formal theory

−0.121

−0.34

0.636 −0.432

4.39*** −2.18**

0.252

1.85*

0.346

2.54**

−0.156

−0.84 ~continued !


.............................................................................................................................................................................................................................................................

Ta b l e 1 0 (c o n t i n u e d ) IO

WP

b

t

CP

b

t

CPS

b

t

b

t

Demographic Attributes Age Gender

−0.016

−0.003

−0.51

−0.006

0.060

−2.93*** 0.44

0.106

0.69

0.404

−0.234

−1.46

−0.392

−2.12**

0.143

0.66

0.060

−1.23 3.10***

−0.016

−3.19***

0.074

0.60

Country Variables United Kingdom Canada

N R

2

F

−0.255

−1.58

−0.436

−2.88***

0.001

0.00

−0.075

−0.37

1095

1072

1138

1051

0.085

0.101

0.100

0.092

5.27***

6.18***

6.50***

5.49***

ISQ

Intercept

0.25

JCR

POP

PT

b

t

b

t

b

t

b

t

6.886

17.36***

6.387

16.29***

5.592

15.01***

8.385

16.18***


−0.134

Canadian politics

0.934

−0.73 2.40**

−0.311

−1.78*

0.298

1.76*

−0.104

−0.44

0.655

1.62

0.382

1.05

0.690

1.47

British politics

−0.019

−0.05

0.087

0.24

0.080

0.22

0.410

1.02

Area studies

−0.288

−1.71*

0.125

0.70

0.283

1.44

−0.332

−1.26

0.570

3.54***

0.196

1.00

International politics Political behavior

1.007

6.45***

−0.039

−0.18

Judicial politics

−0.221

−0.81

Political theory

−0.360

−1.74*

Research methodology

−0.019

−0.11

Public administration

−0.233

−0.71

Public policy Other subfield

0.345

1.77*

−0.166

−0.95

0.037

0.15

−0.700

−3.31***

0.266

−0.309

−1.76*

−0.130

−0.57

0.022

0.12

−0.184

−0.62

0.003

0.01

−0.516

−1.85*

0.722

3.41***

1.60

−0.031

−0.19 −0.43

0.731

3.23***

−0.646

−2.50**

0.072

0.25

−0.126

−0.681

−1.75*

0.177

0.95

0.463

2.53**

−0.079

−0.31

0.156

0.90

0.054

0.29

−0.089

−0.38

−1.55

Methodological Approach Quantitative methods

0.176

1.01

0.276

1.58

−0.539

−3.05***

−0.389

Qualitative methods

−0.016

−0.10

−0.615

−3.71***

0.786

4.86***

0.064

Formal theory

−0.643

−2.85***

−0.131

−0.62

−0.889

−3.93***

−0.900

−2.77***

−0.010

−1.70*

0.001

0.13

0.001

0.11

−0.020

−2.42**

0.137

0.87

0.264

1.62

0.623

4.08***

0.439

2.02**

−3.30***

0.30

Demographic Attributes Age Gender Country Variables United Kingdom

−0.222

−1.20

−0.254

−1.29

−0.775

Canada

−0.205

−0.83

−0.005

−0.02

0.236

0.86

−0.676

−2.76***

−0.511

−1.56

N

865

876

1080

547

R2

0.100

0.079

0.156

0.140

F

4.93***

3.87***

10.29***

4.50*** ~ continued !

two of which are known as outlets for publishing qualitative research. For formal-theory scholars, the American Political Science Review draws favorable evaluations, no doubt because it publishes highly regarded work in formal theory; on the other hand, we note that formal-theory scholars give less positive marks to the Journal of Politics, Comparative Politics, International Studies Quarterly, Political Theory, and Political Studies.

The cases of International Organization and Perspectives on Politics provide an important contrast. On one hand, for International Organization none of the methodology coefficients is statistically significant, suggesting that scholars from all methodological traditions evaluate this journal in similar terms. On the other hand, Perspectives on Politics represents a stark case of the methodological divide in political science. This journal is PS • October 2009 715


.............................................................................................................................................................................................................................................................

Ta b l e 1 0 . (c o n t i n u e d ) CJPS

Intercept

POLITICAL STUDIES

b

t

b

t

5.141

12.71***

5.583

12.52***


−0.068

Canadian politics

1.754

7.21***

0.704

2.07**

British politics

0.625

2.24**

0.764

3.13***

Area studies

0.170

0.84

0.350

International politics

0.121

0.69

−0.104

Political behavior Judicial politics Political theory Research methodology

−0.37

−0.370

0.212

1.06

0.104

−0.195

−0.80

−0.617

−1.69*

1.79* −0.56 0.45 −2.14**

0.712

3.54***

0.968

4.76***

−0.572

−3.03***

−0.729

−3.07*** −1.54

Public administration

0.021

0.08

−0.427

Public policy

0.233

1.26

0.492

2.43**

Other subfield

0.035

0.20

0.073

0.40

−0.002

−0.01

−0.306

−1.54

0.224

1.34

0.133

−0.168

−0.68

−0.940

−3.04***

Age

0.002

0.27

−0.001

−0.12

Gender

0.209

1.28

0.192

1.05

Methodological Approach Quantitative methods Qualitative methods Formal theory

0.70

Demographic Attributes

ing with the perception by many political scientists that Perspectives on Politics was created to appeal to (mostly qualitative) critics of the American Political Science Review. Finally, we also include two demographic characteristics, age and gender, as independent variables in our models. The gender coefficient is significant for only three journals. Controlling for the effects of other variables, women political scientists are more positive in their evaluations of Comparative Politics, Perspectives on Politics, and Political Theory. Women are indistinguishable from men in how they evaluate the other political science journals in Table 10. The age variable was expected to capture generational influences on journal evaluations, and we find that a small number of the age coefficients achieve conventional levels of statistical significance. Older political scientists are less favorably oriented toward the Journal of Politics, International Organization, Comparative Political Studies, and Political Theory. There does not appear to be a systematic factor that ties these negative evaluations together, since this list includes journals from a variety of subfields and methodological approaches. CONCLUSION

The purpose of this article is to build on previous research relating to how political scientists evalUnited Kingdom −0.382 −1.90* 0.808 4.23*** uate the quality and impact of scholarly journals Canada −0.173 −0.86 0.531 1.87* in the discipline. Scholars who study journal impact in the social sciences often differentiate N 726 678 objective (i.e., bibliographic or citation-based) R2 0.133 0.219 approaches and subjective (i.e., survey-based) F 5.71*** 9.73*** approaches for evaluating how various scholarly *** prob~ t ! < .01 journals influence intellectual discourse in the political science profession. As Giles and Garand ** prob~ t ! < .05 (2007) point out, these two approaches provide * prob~ t ! < .10 complementary information about journal Note: Comparative politics is the excluded group among the subfield variables, and normative theory is the excluded group among the methodological approach variables. impact, and ultimately there is a moderately Key: strong relationship between the rankings of jourAPSR American Political Science Review nals using these two approaches. AJPS American Journal of Political Science This article clearly adopts the subjective JOP Journal of Politics approach, but with an important twist. In most BJPS British Journal of Political Science previous studies scholars have conducted surIO International Organization WP World Politics veys of political scientists from the United States, CP Comparative Politics so relatively little is known about how American CPS Comparative Political Studies political scientists differ from those from other ISQ International Studies Quarterly countries in how they evaluate political science JCR Journal of Conflict Resolution POP Perspectives on Politics journals. Here we report the results of a survey PT Political Theory that we conducted of political scientists from the CJPS Canadian Journal of Political Science United States, Canada, and the United KingPolitical Studies Political Studies dom, and this permits us to make explicit comparisons among political scientists from these evaluated significantly less positively by quantitative and formalthree countries. Moreover, our data permit us to consider how theory political scientists, with qualitative scholars giving signifjournal evaluations in 2007 are similar to those observed in preicantly more favorable evaluations. Taken as a whole, it appears vious studies. that Perspectives on Politics is favorably evaluated by qualitative What do we learn about journal evaluations, familiarity, and and normative-theory scholars, but is much less favorably evaluimpact? First, as is the case in previous research, we find that ated by quantitative and formal-theory scholars. This is in keepthere is a fairly clear ranking of political science journals. This Country Variables


.............................................................................................................................................................................................................................................................

ranking is fairly robust across the different types of measures that we use throughout this study. There is a small group of elite journals that are both broadly familiar to and evaluated very favorably by political scientists, and these journals earn high marks for their subjective impact on the political science discipline. These journals include the three major general journals from the United States (i.e., the American Political Science Review, American Journal of Political Science, and Journal of Politics), a major general journal from the United Kingdom (i.e., British Journal of Political Science), and several broad-based subfield journals, such as International Organization, Comparative Political Studies, International Studies Quarterly, among others. It is these journals that draw considerable attention in the political science discipline, and publications in these journals are more likely to be read and cited and to become part of the scholarly discourse in the profession. Other journals have some combination of high familiarity or strong positive evaluations, but not both, and these journals form a second tier of scholarly journals on our list. Many of these journals are highly regarded specialty journals (e.g., Legislative Studies Quarterly, Electoral Studies) or well-known general journals with moderate evaluations (e.g., Perspectives on Politics, Political Research Quarterly, or Foreign Affairs). Finally, there are other journals that are at the periphery of political science, insofar as they are not widely read and/or not highly evaluated. Second, there are some differences in how political scientists from the United States, Canada, and the United Kingdom perceive the quality and impact of scholarly journals, but most of these differences are fairly modest. If one looks at the top-10 lists for each of these three countries, there is a fair amount of similarity in the journals that make the list, though there are notable exceptions. Everyone thinks highly of the American Political Science Review, American Journal of Political Science, British Journal of Political Science, International Organization, and a small number of other journals, but there are also some journals (e.g., European Journal of Political Research, Political Studies, Perspectives on Politics, Canadian Journal of Political Science, Journal of Common Market Studies) that have country-specific appeal. Taken as a whole, there is more broad similarity in journal evaluations than differences for political scientists across these three countries. The correlation for journal-impact measures among these three countries is fairly high. Finally, we estimated a series of regression models for a small subset of 14 journals and find little systematic country differences in how political scientists evaluate these journals. Although there are some mean differences across countries that emerge in our ANOVA analyses, many of these differences disappear when we include control variables representing subfield specialties, methodological approaches, demographic attributes, and publication status. It appears that most of the differences among political scientists from the United States, Canada, and the United Kingdom are due to methodological differences across these three countries. Political scientists’ methodological approach has been found to relate to evaluation of some journals (Garand and Giles, 2003), and there is clear evidence of substantial methodological differences for political scientists from the United States, Canada, and the United Kingdom (Blais et al. 2008). When the methodological differences among political scientists from the United States, Canada, and the United Kingdom are filtered out, the coefficients representing country-specific effects become statistically non-

significant. The underlying basis for differences in political scientists’ methodological approaches across these three countries is the subject of ongoing research. All of this raises some questions about the possibility of greater cross-national integration of political science. While there is a broad-based consensus about journal evaluation, familiarity, and impact among political scientists from all three countries, there are some notable differences, and the methodological divide for political scientists from the United States, Canada, and the United Kingdom could possibly create a barrier to further integration of cross-national research. How political scientists from different countries but working on similar topics can be brought to communicate with each other may have geographic underpinnings, but it is also possible that methodological differences may serve as the greatest barrier. 䡲 NOTES 1. When we remove the journals representing the field of international relations, which appear to be operating in a different citation environment/culture, the correlation between the bibliometric and reputational ranks of the 76 remaining journals is 0.795. 2. These journals are Law and Society Review, Presidential Studies Quarterly, Review of Politics, and Social Science Quarterly. We do not include in this step journals familiar to 20% or more in the Garand and Giles study but central to another discipline, that is, American Economic Review, American Sociological Review, and American Journal of Sociology. 3. These journals are Political Analysis, Canadian Public Administration, and Canadian Public Policy. 4. “For the sake of brevity, we have created an online appendix in which we present tables and figures representing various auxiliary analyses. This online appendix can be found at the Cambridge University Press Web site: http://journals.cambridge.org/action/displayJournal?jid=PSC.

REFERENCES Blais, Andre, Iain McLean, James C. Garand, and Micheal W. Giles. 2008. “The Methodological Divide in Political Science.” Paper presented at the 2008 annual meeting of the Canadian Political Science Association, Vancouver, British Columbia, June 4–6. Christenson, James, and Lee Sigelman. 1985. “Accrediting Knowledge: Journal Stature and Citation Impact in Social Science.” Social Science Quarterly 66: 964–75. Crewe, Ivor, and Pippa Norris. 1991. “British and American Journal Evaluation: Divergence or Convergence.” PS: Political Science and Politics 24: 524–30. Garand, James C. 1990. “An Alternative Interpretation of Recent Political Science Journal Evaluations.” PS: Political Science and Politics 23: 448–51. _. 2005. “Integration and Fragmentation in Political Science: Exploring Patterns of Scholarly Communication in a Divided Discipline.” Journal of Politics 67: 979–1005. Garand, James C., and Micheal W. Giles. 2003. “Journals in the Discipline: A Report on a New Survey of American Political Scientists.” PS: Political Science and Politics 36: 293–308. Giles, Micheal, Francie Mizell, and David Patterson. 1989. “Political Scientists’ Journal Evaluations Revisited.” PS: Political Science and Politics 22: 613–17. Giles, Micheal, and Gerald Wright. 1975. “Political Scientists’ Evaluations of SixtyThree Journals.” PS: Political Science and Politics 8: 254–57. Giles, Micheal, and James C. Garand. 2007. “Ranking Political Science Journals: Reputational and Citational Approaches.” PS: Political Science and Politics 40: 741–51. Goodson, Larry, Bradford Dillman, and Anil Hira. 1999. “Ranking the Presses: Political Scientists’ Evaluations of Publisher Quality.” PS: Political Science and Politics 32: 257–62. Hix, Simon. 2004. “A Global Ranking of Political Science Departments.” Political Studies Review 2: 293–313. Smith, Rogers. 2002. “Should We Make Political Science More of a Science or More About Politics?” PS: Political Science and Politics 35: 199–201.
