1
Clinical Hemorheology and Microcirculation xx (20xx) x–xx DOI 10.3233/CH-141820 IOS Press
3 4 5 6 7 8 9 10
roo f
or P
2
The last bite was deadly – About responsibility in scientific publishing Dragan Pavlovica,∗,1 , Taras Usichenkob and Christian Lehmmanc,d,e a
Ernst-Moritz-Arndt Universit¨at, Greifswald, Germany Klinik f¨ur An¨asthesiologie und Intensivmedizin, Universit¨atsmedizin Greifswald, Greifswald, Germany c Departments of Anesthesiology, Pharmacology, Microbiology and Immunology, Computer Science, Dalhousie University, Halifax, NS, Canada d Anesthesiology, Humboldt-University, Charit´e, Universit¨atsmedizin, Berlin, Germany e Department of Anesthesia, Pain Management and Perioperative Medicine QE II Health Sciences Centre, Halifax, NS, Canada b
uth
1
24
1. Introduction
14 15 16 17 18 19 20 21 22
25 26 27 28
Un
co
23
cte
13
rre
12
dA
24
Abstract. Some open access journals are believed to have devaluated the highly respected image of the scientific journal. This has been, it is claimed, verified. Yet the project we believe failed and we show why we think that it failed. The study itself was badly conducted and the report, which Science published, was itself a perfect example of “bad science”. If the article that was published in Science were to be taken as one of the “test” articles and Science as a victim journal (a perfect control though), the study would show the opposite of what author concluded in his paper: 100% of the controls (normal non-open access journals, in the present study this was Science) the “bait” paper for publication, while in the experimental group only about 60% (open access journals) accepted the bait paper for publication. The conclusion is that, with respect to non-open access and open access, the probability of accepting pseudoscience is well in favor of this being done by a non-open access journal. Since this interpretation is based on some facts that were not included in the project itself, the only warranted result of this study would be that nothing could be concluded from it. It is concluded that the method that Bohannon used was heavily flawed and in addition immoral; that the report that was published by Science was inconclusive and that the act of publishing such report cannot be morally justified either. Various methods to improve the quality of published papers exist but scientific fraud with “good intentions” as a method to promote scientific publishing should be avoided. Keywords: Open access, peer review, scientific publishing, morality of publishing, Bohannon, 2
11
It appears that scientists are under the permanent pressure of a never-ending tide of irrelevant information and its negative secondary effects. The “publish or perish” tendency sharpens; unfair competition among scientists is intense, and there is a perennial challenge posed by competitive ideologies [6, 8, 11]. Scientific publishing is an adventurous and highly responsible enterprise that makes science valuable and 1
Current address: Dragan Pavlovic, 5, Place de la Sorbonne, 75005 Paris, France. Corresponding author: Dragan Pavlovic, Former Research Director, Ernst-Moritz-Arndt Universit¨at, Greifswald, Germany. Tel./Fax: +33 1 56 24 90 99; E-mail:
[email protected]. ∗
1386-0291/14/$27.50 © 2014 – IOS Press and the authors. All rights reserved
2
D. Pavlovic et al. / Responsibility in scientific publishing
34
2. What happened and how
31 32
or P
30
roo f
33
available through a variety of modern media. The scientists who publish frequently have been aware for some time that the number of scientific journals is not only exploding but that there are scientific journals that publish without much control over the content of the papers they publish [2]. Some open access journals (OA) [3] are believed to have devaluated the highly respected image of the scientific journal once established by great journals such as Nature or Science. This has been, it is claimed, verified.
29
56
3. What was the result of the study
41 42 43 44 45 46 47 48 49 50 51 52 53 54
57 58 59 60 61 62 63 64 65 66 67 68
dA
40
cte
39
rre
38
co
37
The final result was much worse than expected. Bohannon’s article, which Science published [11], was itself a perfect example of “bad science” as he himself had defined it, and this was impossible to hide, i.e. the intentional flaws of Bohannon’s bait were: bad controls for the experimental groups, a wrong legend for one graph, non idiomatic English, and the conclusion contained unjustified and therefore “fallacious” recommendations. The very article by Bohannon published in Science has itself more than similar weaknesses i.e.: there is no randomization of his “experimental group”, there are no controls at all (he could have compared the OA journals to the ordinary big journals), there was elimination of non–responders and there was no application of the intention to treat principle in the analysis. Then, even worse: there were no inferential statistics and no references. Indeed, the main drawback of the bait articles was that the introduction and discussion were extremely modest. But the alleged “flaws” of the bait articles which the journal victim should not have missed, were not pertinent enough. That Fig. 1 does not really show dose – response could be excused by the fact that the agent was probably specific so
Un
36
uth
55
Recently, an article was published in Science whose author claimed to have done a study and to have identified numerous culprits, guilty of such behavior, in open access (OA) journals [3]. It looked like quite a good idea to at least have something that can help us distinguish science from pseudoscience in publishing. Yet the project we think, as do a number of others, failed [9]. Not only did the text fall short of its target, it compromised the famous journal Science, morally and scientifically. What happened and how? The plan, we presume, was to produce a text (bait) which would look like a scientific study but would be recognizably flawed and, if accepted by an open access journal (OA) (victim), would expose that OA journal as unscientific. Therefore a professional journalist, John Bohannon, was hired, and he performed the “study” that was, when finished, published as an article in Science. We describe below in some detail the bait and the text that was published to expose the OA journals. After publication, this “plot” would be still problematic because it contained a contradiction: an inference that something is the case that could not be because based on an unscientific foundation, as Bohannon’s paper was planned to be. Scientists would immediately see the problem and the discussion after the publication might become quite pernicious. Science planned therefore, to declare the paper to be merely a news item. Thereby the eventual critique could be prevented. And, we presume, no printed comments were to be published, or if some were to be allowed, they would merely be moderate critique. If some harsh critique were by accident to surface, it would be better to divert it to online comments, and not to expose it in the printed form. And then, online comments would be allowed to explode in all possible directions and this would obfuscate the result. Otherwise, the risky project could yield quite unfavorable results for Science. And this is what I at least believe to have unfortunately happened.
35
3
dA
uth
or P
roo f
D. Pavlovic et al. / Responsibility in scientific publishing
cte
Fig. 1. Ouroboros drawing from a late medieval Byzantine Greek alchemical manuscript. Fol. 196 of Codex Parisinus graecus 2327, a copy (made by Theodoros Pelecanos (Pelekanos) of Corfu in Khandak, Iraklio, Crete in 1478) of a lost manuscript of an early medieval tract which was attributed to Synosius (Synesius) of Cyrene (d. 412). The text of the tract is attributed to Stephanus of Alexandria (7th century). From http://en.wikipedia.org/wiki/File:Serpiente alquimica.jpg (accessed 13th Febr., 2014) .
85
4. The last bite was deadly
72 73 74 75 76 77 78 79 80 81 82 83
86 87
co
71
Un
70
rre
84
that it already had its maximal effect by the first concentration tested. Then, the objection about artefacts from ethanol does not hold, because the concentrations of ethanol were too small (0.66% ethanol). Also, an increase of sensitivity to radiation (Fig. 2) may be present since the response does not start from 100% but much lower. Finally, it is not a “fatal flaw” to conclude that the agent should be examined “in animal and human”. Therefore nothing could be concluded from the text published in Science. Bohannan’s bait articles were withdrawn from OA journals (victims), so unfortunately we do not know what type of articles they would have been if published (brief report, pilot study, field report, special report, news or something else). In the discussions that followed its publication in Science – Bohannon’s text is presented not as simple interesting “news” but as a scientific study. This is indirectly suggested also in the subtitle. What follows from Bohannon’s text is that the journal where Bohannon’s text itself was published (Science) could belong also to the “victim” group of journals characterized by Bohannon as journals that publish weak science. Science was sufficiently clever to publish Bohannon’s article as “news”, which apparently liberated the study from applying scientific method. Scientific method exists to try to establish what is and what is not, and even when it is used correctly, the conclusions are uncertain. As it is, this text would therefore imply no scientific consequences. Nevertheless, we think that it has some.
69
We must correct ourselves and the other commentators on one point. What a number of people claimed about the lack of the controls in Bohannon’s study, is not entirely true. There was a control, though
4
D. Pavlovic et al. / Responsibility in scientific publishing
117
5. Morals for the end
96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115
118 119 120 121 122 123 124 125 126 127 128 129
or P
95
uth
94
dA
93
cte
92
rre
91
co
90
In addition, and this is the greatest problem. The act of producing and sending fraudulent articles to scientific journals is a moral failure. Such an immoral strategy to expose bad science is not necessary. (i) The scientific community already knows that such failings exist, that even „normal“ and respected journals, suffer from bad reviewing. (ii) The so called „Socal Affaire“ [12] from 1996 consisted in a similar intentional fraud to expose bad science and there was no need to repeat such an act. (iii) The very recent case involving the Romanian journal Metalurgia International has already been received without much enthusiasm [5, 10], and (iv) Scientific publications are perennially exposed to fraudulent research. Therefore, to produce a fraudulent text with “good intentions” cannot be justified in principle. It is impossible not to condemn the act on deontological grounds. Or, the principles of utility [14] would have to show that submitting 304 fraudulent articles can somehow be made acceptable because: 1) The intention was noble. 2) This is the only means through which pseudoscientific journals can be discredited and 3) The means justify the ends: Identification and elimination of pseudoscience from
Un
89
roo f
116
a single one but nevertheless a perfect control. Here is how. Bohannon claimed that he sent 304 sting papers (baits) to open access (OA) journals (victims) and analyzed 255 responses. In fact there were 255 plus one control journal, which was the last journal that was “tested”. In that “last bite” offered to one non-OA control journal, there was no randomization, no controls at all, there was elimination of non –responders, no application of the intention to treat principle, no inferential statistics and no references. So this was somehow extreme bait, the most pseudoscientific of all bait articles so far in Bohannon’s study. The victim, which was in fact a control journal, was now non-OA, a scientific top journal, and it not only accepted the bite, but it published it. Thereby it was more than an ideal control journal – it was also an extreme victim, but in the other sense: one of the most resistant to publishing pseudoscience among normal non-OA journals. Needless to say: what was unconsciously submitted as a bait paper was Bohannon’s very report about his “study” and the victim journal that published it was Science. This is also a perfect example of how “retrograde causation” [4, 7] may work in real life: by publishing the mentioned paper, Science changed the nature of the Bohannon study that was already terminated: the Bohannon “study” had in the end a control group. The implications of the study changed as well, the study now showed the opposite of what Bohannon concluded in his paper: 100% of the controls (normal non-OA journals, in the present study this was Science) accepted the bite paper for publication, while in the experimental group only about 60% (OA journals) accepted to publish bait paper. Some would say that since the control sample was n = 1 it is not really enough to draw such a conclusion. However, since both, the bait and the control journal, were pushed to the extremes, this control experiment is much more sensitive than it would have been if an ordinary bait and an average non-OA journal were used. This super control experiment, even with n = 1 had much more weight than an ordinary single control experiment. The conclusion is that, as between non-OA and OA, the probability of accepting pseudoscience is well in favor of this being done by the non-OA journal. A result quite unexpected by the investigators. Indeed, in principle, an article either states how some matters of fact are, in the world, and thereby claims to be scientific. Or else, it is cognitively neutral and thereby just part of some form of entertainment. Since this interpretation is based on some facts that were not included in the project itself, only warranted result of this study would be that nothing could be concluded from it. Here it will be on the scientific community to decide either for the above given conclusion or for the entertainment. We would opt for the latter.
88
D. Pavlovic et al. / Responsibility in scientific publishing
5
136
6. Post script
132 133 134
or P
131
roo f
135
scientific publishing. Unfortunately, none of this was the case. The project intended to demonstrate something that was previously known, other more decent measures for the same purpose were available, and, most importantly, the act itself was an exceedingly unfair means employed to achieve doubtful end. To justify the act by using the doctrine of double effect (DDE) [1, 13, 15] would also involve the above given justifications for the use of multiple scientific frauds as sheer means, which, as we said, are inadequate.
130
149
References
143 144 145 146 147
150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172
dA
142
cte
141
[1] E. Anscombe, Medallist’s Address: Action, Intention and ‘Double Effect’. In: Proceedings of the American Catholic Philosophical Association 56 (1982), 12–25; Washington, D.C.: American Catholic Philosophical Association, reprinted in Woodward (ref. 15). [2] Beall’s list of predatory publishers. http://scholarlyoa.com/publishers (accessed 2013 Nov 15). For a list of the open access journals see at: http://www.doaj.org/doaj?func=byPublicationFee&uiLanguage=en (accessed 2013 Nov 15). [3] J. Bohannon, Who’s Afraid of Peer Review? A spoof paper concocted by Science reveals little or no scrutiny at many open-access journals, Science 342 (2013), 60–65. [4] M. Dummett, Can an Effect Precede its Cause”, Proceedings of the Aristotelian Society 28(Supplement) (1954), 27–44. [5] Z.D. Duri´c, B. Delilbaˇsi´c and S. Radisic, Evaluation of transformative hermeneutics heuristics for processing random data, Metalurgia International 18(6) (2013), 98–102, Free PDF available at: http://fr.scribd.com/doc/167706815/EVALUA TION-OF-TRANSFORMATIVE-HERMENEUTIC-HEURISTICS-FOR-PROCESSING-RANDOM-DATA (accessed 2013 Dec 11). [6] D.M. Eisenberg, R.B. Davis and S.L. Ettner, et al., Trends in alternative medicine use in the United States, 1990-1997: Results of a follow-up national survey, JAMA 280 (1998), 1569–1575. [7] A. Flew, Can an Effect Precede its Cause. In: Proceedings of the Aristotelian Society 28(Supplement) (1954), 45–62. [8] P.B. Fontanarosa and G.D. Lundberg, Alternative medicine meets science, JAMA 280 (1998), 1618–1619. 10 [9] Science online: http://comments.sciencemag.org/content/10.1126/science.342.6154.60/, (accessed 2013 Nov 11). [10] A Serbian Sokal? Authors spoof pub with Ron Jeremy and Michael Jackson references, Retraction Watch: http://retraction watch.com/2013/09/23/a-serbian-sokal-authors-spoof-pub-with-Ron-Jeremy-and-Michael-Jackson-references/ (accessed 2013 Dec 11). 11 [11] R. Sheldrake, The Science Delusion: Freeing the Spirit of Enquiry, Hodder and Stoughtton Ltd, (2012). [12] A. Sokal, Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity, Social Text 46/47 (1996), 217–252.
rre
140
co
139
Un
138
uth
148
The idea of measuring the quality of the reviewing of open access journals was good, the motive was probably justified, the performance shabby; the result is a moral failure. Improving the reviewing procedure would most probably improve on the quality of published science. The big journals that attract through their reputation the best reviewers have lesser problems with the scientific quality of the texts that they publish. As we can see from the above discussed example, knowledge and science remain chimeras that escape most of our efforts to impose on them our rules and rigorous standards. Standards of certification for reviewers or some mechanisms of feedback that would evaluate the reviewing quality might improve the reviewing procedure. It would also help if the scientific community would pay less attention to the fame of a journal or to indicators such as the impact factor or similar indices, but would consider more directly the quality of the research itself. This could introduce some security against scientific misconduct and help improve on the quality of scientific journals. Open access journals fulfill in a certain way the dream of free knowledge. We should not spoil the dream - perhaps merely wake up.
137
6
roo f
or P uth dA
177
cte
176
rre
175
[13] Thomas Aquinas, Summa Theologica (II-II, Qu. 64, Art.7). [14] B. Williams, A Critique of Utilitarianism. In: Utilitarianism: For and Against, by Smart JJC and Williams B. (1973), pp. 77–150; Cambridge: Cambridge University Press. [15] P.A. Woodward (ed.) The Doctrine of Double Effect: Philosophers Debate a Controversial Moral Principle. Notre Dame, University of Notre Dame Press, (2001), pp. 50–66.
co
174
Un
173
D. Pavlovic et al. / Responsibility in scientific publishing
