Supporting Information © Wiley-VCH 2008 69451 Weinheim, Germany
The Effectiveness of the Peer Review Process: Inter-Referee Agreement and Predictive Validity of Manuscript Refereeing at Angewandte Chemie – Supporting Information –
Lutz Bornmann and Hans-Dieter Daniel
Dr. Lutz Bornmann (correspondence author and first author) ETH Zurich Professorship for Social Psychology and Research on Higher Education Zähringerstr. 24 CH-8092 Zurich Tel.: +41 (0)44 632 48 25 Fax: +41 (0)44 632 12 83 E-mail:
[email protected]
Prof. Dr. Hans-Dieter Daniel (1) ETH Zurich, Professor of Social Psychology and Research on Higher Education (2) University of Zurich, Director of the Evaluation Office Mühlegasse 21 CH-8001 Zurich Tel.: +41 (0)44 634 23 13 Fax: +41 (0)44 634 43 79 E-mail:
[email protected]
1
1.
The fate of Communications rejected by Angewandte Chemie The search for the Communications (Zuschriften) rejected for publication by
Angewandte Chemie was conducted using two research literature databases, Web of Science (Thomson Reuters, Philadelphia, PA, USA) and Chemical Abstracts (Chemical Abstracts Services, Columbus, Ohio, USA). The search was carried out by a Ph.D. chemist. The results of the investigation revealed that of the total of 1,021 Communications rejected for publication by Angewandte Chemie, 959 were published as contributions (93.9%) in other journals, seven as patents, and two as contributions to anthologies. No publication information was found for 53 (5.2%) of the rejected Communications, whereby for 26 publications no thorough searching could be done because only the title of the Communication and the name of the correspondence author was available and not the rejected Communication itself. As early as 1980 Abelson[1] reported similar a finding – that almost all of the manuscripts rejected by a journal were published later in other journals – for the journal Science. Other studies on the fate of manuscripts rejected by a journal report percentages ranging from 28% to 85% for manuscripts later published elsewhere.[2] For Communications rejected by Angewandte Chemie in the year 1984, Daniel[3] determined a percentage of 71%. Where investigation of the fate of rejected Communications established that a manuscript had been subsequently published elsewhere, the Ph.D. chemist in our research group determined the extent to changes had been made to the manuscript. Table 1 shows the results of the search for the Communications rejected by Angewandte Chemie but published later elsewhere, and the findings on the extent of changes to the manuscripts. The 959 Communications that were rejected by Angewandte Chemie and later published in other journals were published in 136 different journals. Table 1 shows the 21 journals in each of which more than nine rejected Communications (a total of 723 Communications) were published (fewer than 10 Communications each were published in 2
115 journals, totaling 236 rejected Communications). Fifty or more rejected Communications each were published in the journals Chemical Communications (n=119), Organic Letters (n=91), Journal of the American Chemical Society (n=70), Tetrahedron Letters (n=60), and Organometallics (n=50). Daniel[3] determined the journals in which Communications that were rejected by Angewandte Chemie in the year 1984 were later published: Of the total of 115 Communications, 10 were published in Tetrahedron Letters and 8 each in Inorganica Chimica Acta, Journal of Organometallic Chemistry, and Chemical Communications. The results of both studies indicate that certain journals, like Tetrahedron Letters and Chemical Communications, have always published a large number of Communications rejected by Angewandte Chemie. Six of the total of 21 journals that are listed in Table 1 can be called “sister journals” of Angewandte Chemie, as they are published by the same publisher: Chemistry – a European Journal (n=46), European Journal of Inorganic Chemistry (n=27), Advanced Materials (n=15), ChemPhysChem (n=15), European Journal of Organic Chemistry (n=14), and ChemBioChem (n=12). With the exception of Advanced Materials, these journals are also owned (or part-owned) by the German Chemical Society (GDCh) or another chemical professional association. The publication of a manuscript rejected by Angewandte Chemie in another journal can be attributed to either the initiative of the author (who submits the manuscript to another journal) or, according to information provided by Dr. Peter Gölitz, Editor-in-Chief at Angewandte Chemie, to the initiative of an editor at Angewandte Chemie (who forwards the manuscript to another journal): “For manuscripts that are deemed too specialized or much too long for Angewandte Chemie it is also extremely important to assure a smooth transfer to the likewise prestigious sister journals, which target a more specific audience and/ or publish Full Papers.”[4] And accordingly, letters written by editors at Angewandte Chemie notifying authors that a Communication has not been accepted for publication contain comments such as, “a full paper that took into consideration the referees’ 3
criticism would be accepted almost instantly at Chemistry – A European Journal,” or “given the comments of the referees and the length of the paper the best option may be to revise the paper in accord with the comments and submit it directly as a full paper, for example, to Chemistry – A European Journal.” As Table 1 shows, a total of 16 rejected Communications were published in Angewandte Chemie after re-submission.
2.
Extent of changes to the Communications rejected by Angewandte Chemie and later published elsewhere As Table 1 shows (Total), no changes were made to 30.6% of the Communications
and minor changes were made to 42.9% of the Communications for publication elsewhere (total: 73.5%). A similar percentage was reported by another study: “In approximately 80 percent of … cases, the manuscript had not been changed appreciably. Of the remaining 20 percent, most were altered only moderately.”[5] For 23.6% of the Communications, a medium extent of changes or major changes were made or the content of the Communication was published in connection with other research results (for 21 Communications the extent of changes for publication elsewhere could not be determined, because the Communications were not available). The result of the test statistics (see Table 1) shows a highly significant difference with regard to the extent of changes for the Communications published in the different journals (and rejected by Angewandte Chemie). According to the results, some journals (such as Chemical Communications, Organic Letters, and Tetrahedron Letters) characteristically publish Communications rejected by Angewandte Chemie largely unchanged, whereas other journals (Journal of the American Chemical Society, Chemistry – a European Journal, Langmuir, and Journal of Physical Chemistry Part A) publish considerably changed rejected Communications. The changes that the authors made to the rejected manuscripts for publication 4
elsewhere resulted from, for one, extending the manuscript for publication as a full paper (for example, in Chemistry – a European Journal). For another, a number of the manuscripts had been submitted to Angewandte Chemie that were too lengthy for a Communication. If these manuscripts are rejected and then published in Chemical Communications, they must be shortened for publication. This means that a change to the length of rejected manuscripts can indicate that a Communication was either shortened or lengthened.
5
References [1] [2] [3]
[4] [5]
P. H. Abelson, Science 1980, 209, 60. A. C. Weller, Editorial peer review: its strengths and weaknesses, Information Today, Inc., Medford, NJ, USA, 2002. H.-D. Daniel, Guardians of science. Fairness and reliability of peer review, WileyVCH. Published online 16 July 2004, Wiley Interscience, DOI: 10.1002/3527602208, Weinheim, Germany, 1993/2004. Peter Gölitz, Angew. Chem. 2006, 118, 5152; Angew. Chem. Int. Ed. 2006, 45, 5030. A. S. Relman, in Coping with the biomedical literature explosion: a qualitative approach (Eds.: W. Goffman, J. T. Bruer, K. S. Warren), The Rockefeller Foundation, New York, NY, USA, 1978, pp. 54.
6
Table 1. Journals in which more than nine Communications each were published that had been rejected by Angewandte Chemie, by extent of changes that the authors made to the (rejected) Communications for publication elsewhere (row percentage). Journals in which Communications rejected by Angewandte Chemie were published Chemical Communications
Number of Communications
No changes (n=221)
119
31.1
Organic Letters Journal of the American Chemical Society Tetrahedron Letters
91
+
70
-
60
26,7
Organometallics Chemistry - a European Journal Inorganic Chemistry Journal of Organic Chemistry European Journal of Inorganic Chemistry Chemistry Letters
50
34.0
46
26.1
43
16.3
40
Extent of changes:* Medium Minor Major extent of changes changes changes (n=310) (n=35) (n=136) + .8 58.8 7.6
63.7
29.7
15.7
45.7
-
Not determined (n=21) 1.7
2.2
2.2
2.2
31.4
7.1
.0
63,3
8,3
1,7
,0
36.0
22.0
6.0
2.0
43.5
8.7
6.5
46.5
30.2
7.0
.0
40.0
32.5
20.0
5.0
2.5
27
29.6
33.3
29.6
3.7
3.7
24
16.7
62.5
12.5
8.3
.0
Synlett
20
40,0
55,0
5,0
,0
,0
Chemistry of Materials
16
25.0
31.3
31.3
12.5
.0
Angewandte Chemie
16
-
,0
68,8
25,0
,0
6,3
Advanced Materials
15
46.7
46.7
6.7
.0
+
+
-
15.2
+
.0 +
ChemPhysChem
15
46.7
13.3
6.7
.0
Chemical Physics Letters European Journal of Organic Chemistry ChemBioChem
14
14.3
35.7
35.7
14.3
.0
14
21.4
28.6
35.7
7.1
7.1
12
16.7
25.0
25.0
.0
Dalton Transactions
11
9.1
36.4
45.5
Langmuir Journal of Physical Chemistry Part A Total
10
10.0
70.0
10
.0
20.0
723
30.6
42.9
33.3
+
33.3
9.1
.0
.0
+
20.0
.0
50.0
+
30.0
.0
4.8
2.9
+
18.8
Note. The journals in which the Communications rejected by Angewandte Chemie were published differed statistically significantly with regard to the extent of changes that were made to the Communications for publication; χ2Test: χ2 (80, n=723) = 310.3, p
