Exploring Why and How Journal Editors Retract ... - Semantic Scholar

Sci Eng Ethics (2013) 19:1–11 DOI 10.1007/s11948-011-9292-0 ORIGINAL PAPER

Exploring Why and How Journal Editors Retract Articles: Findings From a Qualitative Study Peter Williams • Elizabeth Wager

Received: 7 April 2011 / Accepted: 5 July 2011 / Published online: 15 July 2011 Springer Science+Business Media B.V. 2011

Abstract Editors have a responsibility to retract seriously flawed articles from their journals. However, there appears to be little consistency in journals’ policies or procedures for this. In a qualitative study, we therefore interviewed editors of science journals using semi-structured interviews to investigate their experience of retracting articles. We identified potential barriers to retraction, difficulties in the process and also sources of support and encouragement. Our findings have been used as the basis for guidelines developed by the Committee on Publication Ethics. Keywords

Retractions Reasons Medical research

Introduction Since peer-reviewed journals act as a repository for research findings, their editors have a duty to ensure the correctness of published material. Therefore, when published material turns out to be seriously flawed, for whatever reason, it should be retracted so that future readers are not misled. The Committee on Publication Ethics (COPE) Code of Conduct for editors (COPE, 2011, p. 2) states that ‘Whenever it is recognised that a significant inaccuracy, misleading statement or distorted report has been published, it must be corrected promptly and with due prominence. If, after an appropriate investigation, an item proves to be fraudulent, it should be retracted. The retraction should be clearly identifiable to readers and indexing systems.’

P. Williams (&) University College London, Gower Street, London WC1E 6BT, UK e-mail: [email protected] E. Wager Sideview, 19 Station Road, Princes Risborough HP27 9DE, UK e-mail: [email protected]

123

2

P. Williams, E. Wager

However, retracting a published article may not be straightforward and few guidelines are available for editors to determine when a retraction (rather than a correction) is warranted—a rare example being the Council of Science Editors’ White Paper on Promoting Integrity in Scientific Journal Publications (CSE 2009). Retraction is one of the most serious sanctions journals can take against authors in cases of misconduct, and can cause permanent damage to an academic career. However, journal policies and practices are not consistent. For example it appears that some journals will always retract in cases of redundant publication, while others will issue a notice of duplicate publication or do nothing. Even in well-publicised fraud cases, some journals have not retracted the affected articles (Sox and Rennie 2006). Discussion of cases at COPE indicates that editors or publishers are sometimes reluctant to retract articles. Reasons for such reluctance may include beliefs that retractions may be made only by the authors; author disputes in which some authors request retraction while others oppose it; and concerns about, or actual threats of, litigation from authors. We therefore decided to examine retractions on Medline to gain an accurate picture of how retractions take place, and to invite a sample of editors who had retracted articles recently to take part in qualitative research to learn more about their experience of retracting an article and the potential barriers to retraction. The latter issue is addressed in the current paper, but a more comprehensive consideration, including the results of a content-analysis of 312 retractions has been published (Wager and Williams 2011). For that study we assessed retracted Medline articles (1988–2008), choosing this source because Medline is the only database that indexes retractions so that they can be searched. Although predominantly biomedical, it also includes some general journals (such as Nature and Science) so, for this qualitative study requiring only a small sample, we were able to select a range of disciplines. The analysis indicated that editors’ practices are not uniform and that there are inconsistencies in what information is provided about the reasons for retractions. In the part of the study being reported in this paper, we interviewed a convenience sample of editors who had retracted papers recently to gain insights into their decision making processes and to better appreciate what factors may have contributed to their decision to retract. We thus hoped to identify factors that might have lead to the noted inconsistencies in our earlier (Wager and Williams 2011) analysis.

Study Objectives Analysis of COPE cases and anecdotal evidence indicates that editors do not behave consistently with regard to retractions and might therefore appreciate guidance about appropriate actions. We therefore aimed to explore editors’ decision-making processes and to identify potential barriers and facilitating factors which might decrease or increase the likelihood of an editor deciding to retract an article. Previous studies have analysed retraction statements from the bibliographic database Medline to determine the stated reasons for retraction (Budd et al. 1998;

123

Exploring Why and How Journal Editors Retract Articles

3

Nath et al. 2006; Wager and Williams 2011) but we wanted to explore editors’ decision-making processes and to identify potential barriers and facilitating factors which might decrease or increase the likelihood of an editor deciding to retract an article.

Methods The exploratory nature of the study—in which we were seeking accounts of individual experiences from particular cases, from which we hoped to elicit the issues inherent in the topic—suggested that a qualitative depth-interview approach was the most appropriate research method. Interviews were carried out, based on the particular retraction we had discovered from our literature search. These were semistructured in nature, during which as far as possible all the areas of research interest were covered, but enabling respondents to answer in their own words, without being forced into answering a pre-defined questionnaire which might constrain their responses. The broad outline of the (ideal) interview is shown in Box 1. How the information was elicited depended greatly on the direction the interviewee took. Also, of course, time and other constraints sometimes precluded a discussion of all the topics itemised in the schedule. Interview transcripts were ‘framework’ analysed (Richie and Spencer 1994). This approach has been previously used successfully in studies undertaken by the present writers (Williams 2005; Williams et al. 2009) in a variety of settings, and involves a systematic process of ‘sifting, charting and sorting material according to key issues and themes.’ (Richie and Spencer1994, p. 177) Once such key themes were established, including a priori topics informed by the research aims, together with issues raised by the interviewees, the original notes were thematically indexed and

Box 1 Topics covered in semi-structured interview Who initiated this particular retraction process? (how did you find out about the problem/who contacted you?) What did you do next? How much work was involved? And how long did it take? How did you decide what to do? (did you consult any guidelines? ask colleagues?) Does your journal/publisher/society have a standard procedure on retractions to help your decision process? How easy/difficult was the decision? Did you have any concerns about issuing a retraction? (e.g. effect on authors, effect on readers, fear of legal action?) How confident did you feel about the process? Is there anything that would have made the process easier? If so, what? Were there any repercussions? (e.g complaints from author, threat of legal action) What lessons did you learn/is there anything you would do differently another time?

123

4


‘charted’. This operation requires data to be lifted from their original context and collated according to their thematic indexing. These ‘charted’ data were further examined to complete the research, by eliciting concepts, finding associations, assessing the strength and extent of elicited views and behaviour.

Interviewees Participants were identified from Medline searches (2007–2008) which indicated they had retracted an article within the previous 2 years. From the 369 retractions in that time period, a purposive sample was selected, including editors from biomedical and other disciplines and retractions for a variety of reasons (including misconduct and honest errors). Of 11 editors approached, five agreed to be interviewed during the study timeframe. The editors, including an executive editor and a subject editor in an editorial team, were based in the UK (three) or the USA (two). Three were employed full-time as editors and the others fulfilled their editorial roles in addition to academic positions. The interviewees between them worked at four journals, two of which covered medicine and the others general science. The group was selected for variety rather than homogeneity (i.e. large and small journals, full-time and part-time editors) so we feel that, although we only conducted 5 interviews these editors were broadly representative of the entire population (not just the 11 initially identified).

Cases Described Seven cases were discussed, since two of the five interviewees each recounted two cases. To preserve anonymity, only brief outlines of the cases are shown below. •

•

• • •

•

Case 1: A paper reporting an aspect of hospital management, which was found to be extremely similar to a previous paper on the same topic by different writers. Case 2: A paper involving a particular scientific calculation. When a new, independent code was used for the calculations on which the conclusions were based, the results could not be reproduced. The new code was considered more reliable than the one used in the article. Case 3: A paper on chemistry which was found to be very similar to an earlier paper published by the same authors. Case 4: A materials science paper, which was found to be copied in part, ‘almost word for word’ from an article by different authors. Case 5: A paper on clinical medicine which was withdrawn after the authors admitted that an identical dataset had been published earlier by other authors and that they did not have permission to publish these findings. Case 6: A paper on the treatment of an infectious disease in which the results could not be replicated.

123


•

5

Case 7: A paper about contamination associated with genetic modification (GM) in which the methodology was questioned and the reliability of the results brought into question.

Results Who Initiated the Retraction Process? A mix of readers, aggrieved authors (i.e. victims of plagiarism) and authors of the papers in question initiated the retraction process. Readers alerted editors to cases of duplication or plagiarism. Researchers also informed the journal when they had attempted the same experiment or calculation as described and had not been able to reproduce results. To give an illustrative example of each: A UK reader of a US journal alerted the editor to overlap in a systematic review, stating that there had been a similar review of the topic published a couple of years before. This was later found to be correct, leading eventually to the retraction. The author of a paper contacted the journal, saying that since he published his original paper, he had used different simulation tools on a further analysis of the phenomenon he was investigating, but could not reproduce the same results as he did by using a modified version of another model. The original results showed certain effects that were subtle but had important consequences—these subtle changes did not happen when the test was run on the later tool. This was an honest error made by the author. The peer reviewers did not detect the problem, and there was no ‘a priori’ reason why the results should have been doubted. An author who felt he had been plagiarised contacted the journal and said the later article that had appeared had: • • •

used his data without acknowledgement published essentially the same material twice previously (i.e. so it finally appeared three times!) hadn’t cited the previous papers.

Thus, this was a case of plagiarism and duplication. Interestingly, in none of the selected cases did the retraction arise as a required action following an institutional inquiry into misconduct. While cases of duplicate publication or plagiarism may be handled by the editor without consulting the author’s institution, since the evidence of the misconduct is in the editor’s hands, other types of misconduct usually require the institution or employer to investigate. Since this was not the case in the retractions included in this study, we did not gain any insights into the interaction between editors and institutions. The Retraction Procedure Each case was, of course, very different. Indeed, we deliberately selected a wide range of cases to reflect as many different scenarios as possible, and from a wide

123

6


range of journals and disciplines. Thus, it would be inappropriate to generalise. What did emerge was the finding that the course of the process appeared to depend very much on: • • • •

the status/role of the initiator the grounds on which retraction had to be considered how the communication between the parties developed the experience of each journal (i.e. past experiences in retracting papers; precedent; policies, etc.).

Of course, in every case the same actors were involved—the paper’s authors, the ‘complainant’ (the person initiating the process), and the other journal involved (in cases of plagiarism or duplication). In the case of larger, well-resourced journals, legal experts were also involved and peer reviewers to evaluate the evidence in the case. To quote one of the interviewees: ‘the first thing is to establish the validity of the concerns’. This generally required a formal response from the author, although in many cases the first step would be to examine papers (such as in cases of duplication or plagiarism) to see whether it was necessary to take the case to that stage. The time and effort involved in each retraction was generally a function of the attitudes of the various parties—particularly the authors. There were cases where authors had moved jobs and thus did not receive emails, and others where there was a dispute, either between the author and the journal editor, or sometimes between the authors. The latter cases involved co-authors who did not know they were even attributed on a paper and an author who did not accept an ‘honest error’ which fellow authors had admitted. In the case of one major journal, the process of peer reviewing correspondence—and hence—communication with the reviewers, also took considerable time. However, another editor reported relying on the journal publisher for much of the work, as it has a department that deals with ethical issues, and helps in such cases. It is worth recounting one particular case as indicative of some of the issues involved and the time taken to carry out the process. This was a case of duplication, initially discovered by a reader who contacted the journal which had published the second version of the paper. The procedure from that point unfolded as follows: •

•

•

•

The editor looked at both articles, and also had an expert evaluate the content and similarity of content (the expert being a member of the journal’s editorial board). The expert reported that there was clearly a significant overlap in content. The journal which had published the original paper was contacted, and a series of ‘conversations’ ensued by telephone and email (suggesting, as was admitted in the interview, that there was no procedure in place to handle this situation). It was decided that the original journal should contact the authors. This too, however, necessitated further communication, as a form of wording had to be agreed by both journals’ editors; The lead author was thus contacted, replied within 5 days, and admitted the mistake.

123


7

Up to this point the editor felt that it appeared as a straightforward copyright violation. However, it transpired that the authors had transferred copyright to the second journal by signing and sending the copyright form to that one first. Thus, although the article was published first in one journal, the other journal held copyright. This led to a further round of activity: • • •

• •

A series of communications took place, ‘back and forth’, between the journal editors, who agreed to publish identical retraction statements in their journals. The authors were alerted to the retraction. The lead author replied, apologising again and saying that he had not been very organised due to moving labs. He explained that another paper should have been submitted, so the paper that was retracted was submitted in error. One co-author, however, said he had no knowledge of the second paper, and was only alerted when the editors contacted him. Four months later, a second co-author replied saying he also had no knowledge of the second paper. His reply was delayed because he was no longer in academia and did not use the email address known to the editor.

Clearly, this story raises some interesting points about the retraction process, first regarding that of copyright and ownership of published material. The COPE guidelines resulting from this study, about which more below, reflect this. The case also reveals problems in the relationship between authors, and in particular the awareness of various co-authors with regard to the attribution of articles. Clearly, being associated with a retraction can be viewed very negatively, and authors who are caught up in the process through no fault of their own have a justifiable grievance. In other cases it might be possible to include in the notice words to the effect that one (or more) of the authors was unaware of the publication. Here, however, the retraction had been issued before the journal received correspondence from the author. It is worth noting here that one of the respondents felt that, in a retraction, where there is a dispute between authors, it is important in every case to ‘spell out’ who doesn’t agree and about what. Similarly, credit should be given to the person who noticed any error and brought the case to the attention of the editors. Policies and Procedures Used Two complementary findings emerged with regard to procedures and policies adopted by journal editors. First, only one journal had clear-cut procedures in place, although even here, the editor considered the process to be ‘standard’ only ‘to an extent’. The reason for this leads to the second major finding, which is that every retraction is a different case, and so it is not possible to follow one particular system. There is a huge difference between cases where an author retracts an article and where the journal has to retract, for example. One respondent observed that each case was ‘painfully peculiar’, and others stated that the unique nature of some of the cases made it extremely difficult to develop even a framework procedure from which to work. It is up to the editor to take each case forward.

123

8


At the time of the study, little general guidance was available and several editors expressed interest in the goal of the project to develop guidelines, with one respondent feeling that this was ‘an excellent idea’. It is worth noting, too, that preexisting COPE flowcharts, although not specifically dealing with retractions, were used by one respondent in deciding what to do. One of the journals had written guidelines for reviewers, which states the ‘dim view’ the journal takes of publishing fraud and duplicate publishing. Another respondent pointed out how difficult it was to police cases of plagiarism and duplication, since editors had to rely largely on peer reviewers to spot such cases from their expert knowledge of the literature which might be better or more specialised than that of the editorial board. Confidence in the Process The cases examined tended to be rather clear-cut, and generally much confidence was expressed in the process and the outcome of their own particular cases. One interviewee, however, did add a cautionary note. He said that there can never be total confidence that the process in any particular case was 100% accurate, because journal editors do not have the legal powers to seize or peruse lab notes or any other raw data that is not voluntarily submitted by the authors. However, he added that the important thing is that any concerns raised are taken seriously and investigated ‘to the degree appropriate’, and ‘to the best of one’s ability’. Journals do, of course, have the power to retract articles, but it is rare that this is done without the author’s consent. What needs to be taken into account is the extent of the concerns, and who has raised them. Another problem raised is that the act of ‘retraction’ has negative connotations, and so it was very important to be clear about the reason for taking this step. In some cases, it was pointed out, ‘correction’ notices could be used instead, which may have less negative effects on authors. Another interviewee made the point that although papers may be retracted, many of them still contained useful research and, of course, the paper still exists and therefore may be read and cited. For these reasons it is very important to choose one’s words very carefully in a retraction, and to emphasise what is correct in the paper. Lessons Learned When asked if anything different would have been done with hindsight, the editors made a number of important comments. One general point made was that one editor would seriously consider and plan for all the possible repercussions in future. In cases where there had been considerable ramifications, these had been something of a surprise. The editor confirmed that the journal would still retract, but would ‘think it through more carefully’ in the future. In another case it was felt that all the authors should have been contacted about the retraction—regardless of the fact that only one was usually designated as the ‘corresponding’ author. This would allow any authorship disputes or different author experiences (such as one author not being aware of a submission) to be dealt

123


9

with at an early stage and to prevent all the authors being blamed for any ‘misdemeanour’. Finally, one editor felt so strongly about a particular case of plagiarism that he wanted to ban the infringing author from publishing in the journal for 3 years. He took the case to COPE, who said that further punishment (i.e. beyond the retraction) was unnecessary, and reluctantly abided by this recommendation. Other Issues Another issue is that of discovering ‘historic’ cases of misconduct, i.e. cases that remain undetected for several years. One case was recounted of an author who, it was claimed, lifted 85% of the contents of a previous paper published by someone else and slotted in his own data. The interesting point about this case is that it remained undetected for 17 years and was only spotted when anti-plagiarism software was used to examine Medline abstracts (Errami et al. 2008). The creators of eTBLAST (http://invention.swmed.edu) and the De´ja` vu website (http://spore. swned.edu/dejavu), alerted the editor to the similarities between the papers. At the time of the interview, the editor was investigating the case, but initial examination suggested this was a clear case of plagiarism, and the editor considered it highly likely that the journal would issue a retraction—even after this length of time. An interesting point is that the age of the case has added to the problem of locating the author. He produced six single-authored papers, the last of which was published in 1996, so he is probably retired by now. There is now, of course, speculation that he may have plagiarised others’ work for those articles too. Still on the subject of past research, one editor said that her journal often receives complaints from readers about articles that are years old, to the effect that the results or methods described are no longer valid. However, the editors consider that there is a distinction between a paper containing errors and one that is superseded in later years by further scientific or technological advances. As long as the paper took into account everything that was known at the time, the paper is considered legitimate.

Study Limitations Interviews are the most utilised data collection method in qualitative research (Rogers and Bouey 1996) since they allow a tremendous amount of data to be gathered by talking to people, exploring their experiences and allowing them a degree of latitude (Silverman 1999; May 1997). However, in a small study such as this, with a purposive sample, it is not possible to guarantee that the respondents are truly a representative sample. However, we included both full-time (professional) and part-time (academic) editors working for prestigious journals published by major publishers and smaller titles produced by smaller companies. We also included a range of disciplines, and editors from Europe and America, but were struck by the similarities rather than the differences in the editors’ views and experiences, regardless of the subject matter of the retracted article or the editorial base of the journal.

123

10


Conclusion Retracting an article is a serious step for an editor to take and may have serious repercussions for the journal. Because retractions are, thankfully, relatively rare events, most editors cannot rely on previous experience to guide them since they may have to deal with only one or two during their editorial career. Journal owners, publishers or organizations such as COPE therefore play an important role in recording the collective experience and ensuring lessons learned are translated into corporate knowledge and helpful guidance. Although the different circumstances of each retraction, and the reactions of the players, make it difficult to create detailed processes for every situation, COPE has developed specific guidelines on retractions (Wager et al. 2009) based on the research reported here and our other study (Wager and Williams 2011). Two examples of the guidelines that were informed by our findings may be instructive. First, the guidelines dealt with multiple authorship by stating that: ‘‘If retraction is due to the actions of some, but not all, authors of a publication, the notice of retraction should mention this. However, most editors consider that authorship entails some degree of joint responsibility for the integrity of the reported research so it is not appropriate for authors to dissociate themselves from a retracted publication even if they were not directly culpable of any misconduct.’’ However, if an individual’s name has been included without his knowledge and if the individual had nothing to do with the research, this should also be indicated. Second, our study highlighted a complex problem with copyright. The COPE guidelines therefore recommend: ‘‘If an article is submitted to more than one journal simultaneously, and is accepted and published in both journals (either electronically or in print) at the same time, precedence may be determined by the date on which a licence to publish or a copyright transfer agreement was signed by the authors.’’ To conclude, our interviews helped explain some of the reasons why editors may be reluctant to retract articles and why the retraction process may take several months. The research also highlighted the complexities of some of the cases and of the resulting considerations to be taken into account. Nevertheless, COPE has used this study and its experience in the field to produce guidelines that it hopes will aid editors handle the difficult and time-consuming, but essential, task of retracting unreliable articles from their journals to ensure the integrity of the research literature. Acknowledgements This project was funded by COPE (the Committee on Publication Ethics). We thank the editors who agreed to be interviewed and gave so generously of their time and their insights.

References Budd, J. M., Sievert, M. E., & Schulz, T. R. (1998). Phenomena of retraction. Reasons for retraction and citations to the publications. Journal of American Medical Association, 280, 296–297. Council of Science Editors (CSE). (2009). White paper on promoting integrity in scientific journal publications. Available online at: http://www.councilscienceeditors.org/i4a/pages/index.cfm? pageid=3354#2.1.7. Accessed June 14, 2011.

123


11

Committee on Publication Ethics (COPE) (2011). Code of conduct for editors London: COPE Available online at: http://publicationethics.org/files/Code%20of%20conduct%20for%20journal%20editors_0. pdf. Accessed Feb 18, 2011. Errami, M., Hicks, J. M., Fisher, W., Trusty, D., Wren, J. D., Long, T. C., et al. (2008). De´ja` vu–a study of duplicate citations in medline. Bioinformatics, 24, 243–249. May, T. (1997). Social Research: Issues, Methods and Process. Buckingham: OUP. Nath, S. B., Marcus, S. C., & Druss, B. G. (2006). Retractions in the research literature: Misconduct or mistakes? Medical Journal of Australia, 185, 152–154. Richie, J., & Spencer, L. (1994). Qualitative data analysis for applied policy research. In A. Bryman, R. G. Burgess (Eds.), Analyzing qualitative data (pp. 173–194). London: Routledge. Rogers, G., & Bouey, E. (1996). Collecting your data. In L. M. Tutty, M. Rothery, R. M. Grinnell, Jr. (Eds.), Qualitative research for social workers: Phases, steps, and Tasks, 4th ed. (pp. 50–87). Boston: Ally & Bacon. Silverman, D. (1999). Doing qualitative research. A practical handbook. London: Sage. Sox, H. C., & Rennie, D. (2006). Research misconduct, retraction, and cleansing the medical literature: Lessons from the Poehlman case. Annals of Internal Medicine, 144, 609–613. Wager, E., Barbour, V., Yentis, S., & Kleinert, S. (2009). Retractions: Guidance from the committee on publication ethics (COPE) Available at: http://publicationethics.org/files/u661/Retractions_COPE_ gline_final_3_Sept_09__2_.pdf. Accessed March 21, 2011. Williams, P. (2005). Using information and communication technology with special educational needs students: The views of frontline professionals. Aslib Proceedings, 57(6), 539–553. Williams, P., Stevenson, I., Nicholas, D., Watkinson, A., & Rowlands, I. (2009). The role and future of the monograph in arts and humanities research. Aslib Proceedings, 61(1), 67–82. Wager, L., & Williams, P. (2011). Why and how do journals retract articles? An analysis of medline retractions 1988–2008. Journal of Medical Ethics (to be published summer 2011). doi:10.1136/ jme.2010.040964 (in-press).

123