From conference to journal publication: How ... - Wiley Online Library

From Conference to Journal Publication: How Conference Papers in Software Engineering are Extended for Publication in Journals

Michela Montesi and John Mackenzie Owen Archive and Information Science Department, University of Amsterdam, Turfdraagsterpad 9, 1012 XT Amsterdam. E-mail: {m.montesi, mackenzie}@uva.nl

In software engineering (SE) and in the computing disciplines, papers presented at conferences are considered as formal papers and counted when evaluating research productivity of academic staff. In spite of this, conference papers may still be extended for publication in academic journals. In this research, we have studied the process of extension from conference to journal publication, and tried to explain the different purposes these two forms of publication serve in the field. Twenty-two editors in chief and associate editors in chief of major publications in SE and related fields were interviewed, and 122 authors of extended versions of conference papers answered a Web questionnaire regarding the extension of their papers. As a result, the process of extending conference papers for journal publication in SE is recorded. In the conclusion, we comment on the following: (a) the role of the conference in the development of the research work; (b) the review process at the conference and at the journal stage; and (c) the different purposes conference and journal publication fulfill in SE.

Introduction and Purpose Generally, in software engineering (SE) and the computing disciplines, papers presented at conferences are counted as formal papers valid for assessing research productivity and for tenure decisions. Participants in prestigious SE conferences, such as the International Conference on Software Engineering (ICSE), or the joint meeting of the European Software Engineering Conference and the ACM Sigsoft Symposium on the Foundations of Software Engineering (ESEC/FSE) can submit papers of a maximum of 10 pages, which are later published in the corresponding conference proceedings. Authors are thus left sufficient space to report their research, and the report is formally published and taken into account for assessing research performance. All the same, Received November 2, 2006; revised October 2, 2007; accepted November 14, 2007

•

© 2008 ASIS&T Published online 31 January 2008 in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/asi.20805

some conference papers are further expanded to become journal papers. Many SE journals refer to them as “extended versions of conference papers,” i.e., papers that have been presented initially at a conference, workshop, seminar, or the like, and have later been adapted for publication in a journal. It is known that, in many disciplines, several stages of informal dissemination of research work, including conference presentations, precede publication in journals (Garvey, 1979). However, previous work in scientific communication has pointed out that the publication of “amplified versions” of conference papers is more typical of SE and the computing disciplines compared with other disciplines (Kling & McKim 1999; Montesi & Mackenzie Owen, in press). This process hasn’t been explained yet; an investigation of the process may be revealing of specific patterns of research and communication within SE and the other computing disciplines. Scientific communication is an area of research in information science and knowledge organization (Hjørland, 2003). However, when studying scientific communication, information scientists tend to favor articles published in scholarly journals, as pointed out by Drott (1995). He claimed that conference papers should be seen as having a role of their own, complementary to that of journal publications. Nonetheless, few studies have built on Drott’s conclusions, an exception being the citation analysis conducted by Goodrum, McCain, Lawrence, and Giles (2001) on citation practices within the computing sciences. The interaction between scientists as mediated by technology, on the other hand, has recently received the most attention (Kling, McKim, & King, 2003; Barjak, 2006). Interactions taking place at conferences are studied less, and so in general the role of conference publication versus journal publication is studied. Garvey discusses the role of meetings and conferences in the dissemination of research results, though he comments only briefly on the role of the conference in the production of the journal paper (1979, pp. 209–210). The topic of conference papers versus journal papers and of the process leading from one to the other still remains largely unexplored.

JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, 59(5):816–829, 2008

Within information science, bibliometricians are paying increasing attention to the area of SE. A few studies focus on the definition of the field. For example, Marion and McCain (2001), through co-citation analysis and analysis of classification terms, define the field of SE as interdisciplinary, inclined to application, and mirroring the software-development life cycle. McCain, Verner, Hislop, Evanco, and Cole (2005), on the other hand, define the discourse community of SE through citation patterns and perceptions of citing authors about cited authors. Some research effort has been devoted to the question of assessing research performance in the computing disciplines. In this regard, bibliometricians emphasize the importance that the computing disciplines assign to publication in conference proceedings, which is not properly conveyed in current bibliometric methods of assessing research performance. Official systems providing data for research-assessment purposes, such as the Science Citation Index (SCI) and Web of Science (WoS) published by Thomson Scientific, rely almost exclusively on journal publication. This may be inappropriate for SE and the computing disciplines, and not only because conference publication isn’t taken into account to the needed extent. Among the other limitations of current systems that perform citation analysis, Sidiropoulos and Manolopoulos (2005) note that, the SCI, for example, considers neither books nor technical reports, apart from conference proceedings. Another limitation of the SCI is the inability, in their opinion, to follow the dramatic evolution of the computing disciplines. The fact that the area of computing is evolving at a very fast pace is also confirmed by Coulter, Monarch, and Konda (1998), who performed a co-citation analysis of SE publication descriptors in the period 1982–1994. Also, by relying mainly on journal publication to measure scholarly achievement, theoretical approaches may be favored instead of experimental ones (Computing Research Association, 1999). Experimentalists tend to prefer conference publication to journal publication because of the shorter time to print, the opportunity to present the research to their peers face-toface, and other reasons. On the other hand, studying citations from published literature and the publicly indexable Web, Goodrum et al. (2001) found that what computing researchers cite the most are books and book chapters, followed by journal articles, whereas conference proceedings come only in third place. Moed and Visser (2007) have explored the possibility of expanding the WoS database to include some of the leading conferences for computer science and related disciplines. They concluded that it is definitely possible to expand the WoS database by incorporating conference proceedings, provided that some technical problems are solved. One of these is dealing with different versions of the same publication (journal article, conference article, book chapter, technical report, etc.). They have suggested two ways of approaching this problem: either by considering the individual publication— the classical approach—or by using the concept embodied in a series of publications, which is an alternative, still unexplored approach. Finally, if the need to reform current systems

to measure research performance in the computing sciences is largely recognized by many parties, the area still deserves further attention. This research may cast some light on the problems just commented on. Looking at the extended versions of conference papers from the point of view of SE, a better knowledge of how these papers are extended would provide better guidance to authors, on a basis of solid evidence. It is a requirement of many scientific journals that extended versions of conference papers include new information. Some journals require that at least one third of the information contained in the paper must be new. For example, several IEEE publications, such as the IEEE Transactions on Software Engineering (TSE Author Center, 2006), require that, if a preliminary version is available, authors should submit it together with the paper and a description of the major changes made. Reviewers, on their part, must compare the two versions and make sure that authors comply with what is sometimes referred to as the novelty requirement (Referee Guidelines, 2006). However, most SE publications do not explain precisely what new information authors should include, and no information is available on how or what authors actually change when extending or revising their conference papers. A better understanding of what changes from the conference to the journal paper, and how this unfolds, may be of help to different actors in SE, as well as to information services that specialize in the subject. In these cases, reviewers of computing papers would have more documented data to carry out their job. Differences in the judgment of program committee members have been noted, and a better knowledge of article types has been advocated to address this problem (Wieringa, Maiden, Mead, & Rolland, 2006). Readers and searchers may also benefit from this study, because, when reading and selecting literature, they can base their expectations on solid evidence about these types of papers. Other types of papers, such as surveys or experimental papers, are well known and have been documented (Goldberg, 1982; Kitchenham et al., 2002; Sjøberg et al., 2005), and it would be desirable to better document extended versions of conference papers as well, to the readers’ and searchers’ advantage. Well-grounded expectations about article types may be a valid selection criterion when searching for relevant literature (Montesi & Mackenzie Owen, in press). In addition, it is unclear how extended versions of conference papers are currently treated in databases specializing in the computing disciplines, such as the IEEE Computer Society Digital Library (http://www.computer.org/portal/site/csdl/index.jsp) and the ACM Digital Library (http://portal.acm.org/dl.cfm). The latter offers the option to trace citing papers for each item; this is probably the only way searchers can link a conference version of a paper with the corresponding extended version. It is indeed difficult for these information services to adopt any clear policy, because the different functions that conference papers and journal papers are supposed to accomplish are not precisely documented. A switch in the communicative function from conference to journal publication is also expected, and this should be reflected in the extension

JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY—March 2008 DOI: 10.1002/asi

817

process. In summary, this research should make a contribution to addressing all these issues.

TABLE 1.

Methodology

Interviews

The results reported in this paper come from two sources. First, we interviewed a sample of 22 editors of major publications in SE, either editors in chief or associate editors in chief. Second, we surveyed a sample of 122 authors using a Web questionnaire. The interviews should help to put the extended versions of conference papers in the context of the discipline, and understand their role in knowledge building, as well as highlighting the most important quality requirements of such papers. The main purpose of the author survey was to ascertain if the conference presentation and discussion had an influence on the way authors revise their papers, and how the extension/revision process had developed. Interviews With Editors Semistructured interviews were held with a sample of 22 editors, either editors in chief or associate editors in chief, from major publications in SE and some related fields. In the text, we will refer to them simply as “editors” regardless of their more specific function on the editorial board. Possible participants’ names were located either on journal Web pages or through personal recommendations of interviewees. All journal titles were chosen from the list of the JCR, “Computer Science—Software Engineering,” April 2006, which among the most cited publications also included some of the related fields, such as database systems and computer science. Eighteen editors agreed to give the interview, whereas four editors asked to answer the interview questions in writing. Most interviews (16) were given over the telephone, but in two cases a face-to-face interview was held. The duration varied from approximately 10 to 28 minutes. All interviews took place and were recorded between June 7 and August 17, 2006. The interviews were immediately transcribed. The journal titles about which the editors were contacted are provided in Table 1, and the countries of affiliation of the editors are given in Table 2. Author Questionnaire For the author questionnaire, we drew on the same list of journals from the JCR “Computer Science—Software Engineering,” April 2006. In order to gather a sample of possible respondents, we located recently published papers based on preliminary conference versions or extended abstracts, and contacted their respective authors. Only in a few cases did the journals mention the preliminary conference version in a note on the first page. In fact, most journals do not report any explicit reference to the preliminary version. It was suggested by some editors that we look in the reference section for a similar title published by the same authors, or that we simply try to search for the title using a search engine. Though it may seem obvious, we noted that it isn’t easy to establish 818

Editors taking part in the research: Journal titles. Journal Title

In writing

ACM Transactions on Software Engineering and Methodology ACM Transactions on Database Systems Computer Computer Graphic Forum IEEE Transactions on Software Engineering Information and Computationa Journal of Functional Programming Journal of Visual Languages and Computing Science of Computer Programming

3 3 1 1 4 1 1 1 3

IEEE Computer Graphics and Applications IEEE Software IEEE Transactions on Software Engineering Mathematical Programming

1 1 1 1

a This is the only publication not reported in the JCR list consulted. The name of the corresponding editor was suggested by a previous interviewee.

TABLE 2. Editors taking part in the research: Country of affiliation. Journal Title Interviews

In writing

Denmark Germany Greece Italy Switzerland The Netherlands UK USA

1 1 1 1 1 2 6 5

Canada UK USA

2 1 1

when a paper can be considered an extension of a previous one, especially without an adequate knowledge of the area. Sometimes one paper may simply be a complementary paper or the logical development of the research reported in a previous one. Therefore, in order to avoid any subjective interpretation on our part, we decided to limit our sample exclusively to those papers explicitly labeled as “extended versions.” This means that the papers selected were published in a limited set of journals, namely, those reported in Table 3. The best papers from conferences, i.e., conference papers invited for publication in a journal as high-quality contributions, were not included because it isn’t clear whether they have been extended. All six publications appear in the list of the JCR, “Computer Science—Software Engineering.” The number of extended versions published as journal articles accounts for approximately one third of all research articles for Algorithmica, ACM TOPLAS, and ACM TOSEM. For the other journals it varies greatly. However, this data must be regarded with caution. The period that has been considered is quite short, and some journal articles/issues were not counted, as indicated in the notes below Table 3. The median of the time lag between the conference publication and the journal publication is also reported, and shows variation


TABLE 3.

Data on extended versions of conference papers published in five journals. Extended versions published in:

Journal title

2003

2004

2005

2003–2005

Research papers published 2003–05

Extended versions 2003–2005 (%)

Median time lag (years)

2 8

4 9

8 8

14 25

64 79

21.9% 31.6%

2 4

ACM Transactions on Database Systemsa ACM Transactions on Programming Languages and Systems ACM Transactions on Software Engineering and Methodology Algorithmica Journal of Systems and Software Journal of the ACMb

4

4

3

11

34

32.3%

3

19 5 16

17 3 18

23 5 16

59 13 50

164 344 86

35.9% 3.8% 58.1%

2 2 3

Total

54

55

63

172

a

Best papers from international conferences were not counted. Articles published in a special issue for the 50th anniversary of the journal (50(1), 2003) are not included.

b

across the journals, spanning from two years for ACM TODS, Journal of Systems and Software, and Algorithmica, to four years for ACM TOPLAS. Extended versions of conference papers are indicated in different ways, such as “The results appeared in preliminary form in the Proceedings . . . ,” or “A preliminary version of this article was presented at . . . ,” etc. We didn’t consider relevant the fact that authors alluded in different ways to the conference version of their paper. The first and, when available, the second author of each of the 172 extended versions were included in our sample. The total number of authors contacted was 265. It was only after several invitations that a satisfactory number of answers could be gathered. We based our questionnaire on a TABLE 4.

Web-survey population: Contacted authors and response rate. Responded

Declined

No response

Contacted/Total

57 65 122

28 22 50

56 37 93

141 124 265

First Authors Second Authors Total

TABLE 5.

commercial Web site designed for conducting Web surveys, (www.surveymonkey.com), and often our e-mail invitations were automatically classified as spam. Out of 265 authors contacted, 122 responded and 55 declined our invitation. However, the relatively low response rate (46%) was compensated for by the qualitative data, gathered thanks to a number of open-ended questions. The response rate was higher among second authors than first authors (see Table 4). The geographical affiliation of the responding authors at the time of the journal publication is reported in Table 5, as is the number of responding authors per journal. Response rates were lower for authors from Algorithmica and ACM TOPLAS, but their actual contribution to the corpus of responding authors is still notable. The contrary is true for

Response rate 40.4% 52.4% 46%

Web-survey population: Responding authors.

ACM TODS USA Italy Israel Canada Germany India France Othera Total Response rate

6 – – – 2 1 – 3 12 57.1%

ACM TOPLAS 10 2 1 1 – 1 – 2 17 38.6%

ACM TOSEM 4 6 1 1 – – – – 12 57.1%

Algorithmica 11 2 6 2 5 – 1 7 34 40.4%

Journal of Systems and Software 3 1 – 4 – – 1 4 13 47.2%

Journal of the ACM

Total

13 6 5 1 1 2 2 4

47 17 13 9 8 4 4 20

34 56%

122

a

Includes authors from Spain, Austria, Switzerland, Hong Kong, UK, Japan, among others.


819

ACM TODS and ACM TOSEM, whose response rates were higher, but their actual contribution was limited to 12 responding authors each. Results Results From Interviews With Editors The answers given by the editors to the nine interview questions, which are reported in the Appendix, are summarized under the following headings: Handling extended versions of conference papers Papers with a conference history The conference and the production of the journal paper Other factors influencing the extension Advantages and disadvantages of extended versions Comparing the two versions Summary: How authors are expected to extend their papers

All text in quotation marks (below) corresponds to direct statements of the editors. Handling extended versions of conference papers. When editors receive a submission of an extended paper, the previous version may or may not be an immediate concern. Those editors that need to be informed directly of the conference version appeal to the ethical concerns of the journal. On the other hand, those editors who, in principle, do not treat extended papers differently from ordinary submissions are mostly concerned with the overall quality, and leave the comparison of the two versions for a later stage. When comparing the two versions, editors try to ascertain whether there is a “meaningful extension” and a “significant scientific contribution above the conference version,” which is not without difficulty. It is especially difficult to say whether the extension is meaningful enough, and whether it accounts for the required 25–30% new material. New items of information that constitute a meaningful extension may vary depending on the type of paper. For example, in theoretical papers, these items may be new or stronger theorems, and proofs; in more empirical papers, on the other hand, the items can include additional results, an additional case study, an additional empirical study, or an incremental evolution of the work. It is ultimately up to the reviewers to make the conclusive decision. Papers with a conference history. According to the editors, the name of the conference where the papers were initially presented may raise different types of expectations about the contribution. However, the individual merits and the quality of the paper count for more than its conference history. It is especially important that the contribution meets the standards of the journal, no matter where it was published earlier. Besides, practically all papers in SE have been published at a conference previously, since exclusive journal publication 820

is reserved for specific types of papers, such as extended reviews or deep analysis of problems, which are not amenable to conference publication. All the same, the conference history of the paper does not go unnoticed. Conferences can differ in terms of acceptance rate and standard quality of the contributions, in terms of size, and finally in terms of focus and orientation. Editors expect these differences to be somehow reflected in the corresponding extended papers. The refereeing process together with a stringent acceptance rate is seen, in a certain sense, as a guarantee of quality. An editor commented, “Typical people evaluating such a paper would have a sense of the typical acceptance rate of the conference, the typical research quality, the significance of the results that are typically presented at the conference, the rigor of the peer-review process, the program committee process, and those sorts of things.” All conference papers can be expected to have a certain degree of quality, because they have already passed through a first quality filter. Depending on how well known the conference is, there may be differences in the expectations and requirements for extension. Major conferences in contrast to small conferences guarantee that the paper has already been judged in terms of innovation. It is also likely that papers presented at major conferences have been widely read in the community, and the journal must take extra care in assessing that the differences between the two versions are meaningful enough. Another class of expectation raised by conferences regards their focus and orientation. Some conferences are very academic, whereas others have a more practical orientation. Some may be highly focused and specialized, and others may be broader. Journals usually address a broad audience, and, in the case of extended versions coming from highly specialized conferences, editors must make sure that the paper is comprehensible enough, even to a nonspecialized audience. The conference and the production of the journal paper. All editors agreed that the whole conference process influences the way a paper is extended and revised, far more than the simple discussion taking place after the presentation. Comments made in the discussion following the presentation tend to be “erratic”; they can lead to new applications/collaborations, but hardly ever do they change the actual content of the paper. Questions or comments made at the conference discussion may also be very specific and specialized, with little influence on the journal version, which is often addressed to a more general audience. An attempt to have the discussion included can be made when the people who attended the conference are in charge of the review for some special issue. It may happen that the person who asked a critical question is chosen to do the journal review. The time available for questions after the presentation is often limited, but later on in the conference, people can approach the authors and ask them about the paper. Talking about the research helps one to think of one’s work in a different manner, explain unclear points, and add new viewpoints: “Whenever you have an idea, you can have a better idea . . . even just


talking about it, you can think of something differently,” said one editor. At the conference venue, authors get “a sense for which open problems the community is finding most interesting and more compelling.” Consequently, they can give these problems a priority in future work, including the possible extension. In general, authors can receive ideas for future work. Authors may also get to know some related work, either work that was not yet published when the conference paper was written, or that they had simply missed, and so better place their work in the context of earlier research. Finally, in rare cases, they may even discover that something they have done is wrong. Thus all these conference factors contribute to the extension of the paper. Other factors influencing the extension. Apart from the twostage review process (for the conference and the journal), the discussion at the conference, and the reactions of the colleagues who have read the conference proceedings, editors have also mentioned other factors influencing the extension. These can be classified into journal-related, time-related, thought-related, author-related, and related to the type of paper.

•

•

•

•

Journal-related factors: As for the influence exerted by the reviewers, it was stressed that revision after journal review is compulsory. In contrast, after conference review, authors may or may not apply changes, and often, because of time limitations, they don’t. Besides, when writing for a journal, authors have far more space to express their ideas, and no longer have stringent page limitations. Time-related factors: Time moves on after a paper is accepted at a conference, and authors may have refined their work and found some minor problems, thus being able to present a more robust work. But in the meantime, others may have been working on the same subject, and the paper may no longer be so innovative. Also, depending on the body of research accomplished before the conference submission, the research may have progressed and there may be more data available to add. Thought-related factors: Having more time gives authors the opportunity to keep thinking about their work. Authors “are working after the publication of the conference paper, further expanding, further studying the field in depth, and the factor influencing extension is the continuing research.” Reading about the same subject or about similar subjects can positively influence the extension, though what really counts is that authors continue to think about their work. Author-related factors: Author-related factors may influence extensions, positively or negatively: “external drivers, for example, need for tenure, or research assessment exercise”; “the desire for the author to make the work more enduring, that it has a longer life”; the reluctance to put in an extra effort after the paper has been accepted at a competitive conference, or, conversely, the wish to make their ideas clearer; their understanding of the need to be significantly different, and the need to address a different audience; their individual writing skills.

•

Factors related to the type of paper: Finally, extension can depend a lot on the type of paper one has written. “Even within SE there is a complete gamut of different styles and kinds of papers, from more implementation-oriented to more theoretical,” each of which requires or allows for different extensions. For an empirical work, extension may be faster, as a qualitative presentation of the data, plots and formulas, a discussion of data, and so forth do not usually fit in a tenpage conference paper. Conversely, papers that are more theoretical require more additional work. Extension is also different, depending on the number of papers to be merged into the journal version. Finally, some papers are just not suitable for extension: “It very much depends upon the paper, there are lots of conference papers which are tackling a very specific problem and they solve it. That sort of papers simply will not become a journal paper, no matter how hard you try. A journal paper is supposed to have longer, broader impact on the community,” commented one editor.

Advantages and disadvantages of extended versions. The factors influencing the extension process just described do not necessarily have a positive impact on the extended version and on the community. The editors mentioned both advantages and disadvantages of this way of producing journal papers. For example, despite the work done on them, extended papers may not be different enough from their conference version. Proofs and more details may be added on, but not necessarily new ideas, nor any really new material. In other words, authors may fail to think of their work in a different way. Extending a conference paper is not just a matter of revision, and some authors are unable to adequately extend their papers. Papers may be overextended, loose balance and cohesion, and “fail to report a logical whole.” Authors may also fail to rewrite the paper in a style appropriate for the different audience a journal is supposed to address. It is indeed a very hard and long process to convert a conference paper into a journal paper, because it is difficult to make the work sufficiently novel, and to communicate the information in a different manner to a different audience. With this in mind, some editors question the practice of extending conference papers for publication in journals. With conference proceedings available in digital libraries, and the difference in page length between journals and conference proceedings getting smaller, “this strategy encourages double-dipping.” “It does not make much sense when so many people are competing for print space and journals are backlogged, suffering long publication delays . . . I am afraid the only factor, and incentive, for the authors is to increase their publication count for tenure/promotion purposes.” In contrast, for other editors, it is a shame that relatively few conference papers become journal papers, because “journal papers require you to reflect for a longer period of time on the work,” and to clearly place your work in the context of the field. However, not many authors take the trouble to extend their conference papers, since they are not rewarded. Simply getting the paper into a conference with a very low acceptance rate may be enough for many of them.


821

Not applying a meaningful extension often results in many submissions ending in a rejection. This is particularly undesirable because extended versions of conference papers can be more labor-intensive than standard contributions from the point of view of the review process, and often cause an unpleasant waste of time. In general, it can also be difficult to find the right people to do the journal review. While conference reviewing is seen as an honor, because it publicly shows the community that the reviewer is “an expert, someone who has expertise and good judgment,” journal reviewing is primarily performed as a professional service, and the reviewer’s contribution to the improvement of the paper remains anonymous. However, depending on the circumstances, the review of extended versions may even be faster. If the paper has been selected as one of the best papers at a conference, and has been invited for publication in a journal, the conference committees will help to find the appropriate reviewer, which is often “the major burden now for journal editors.” Consequently, the chances of getting published quickly will increase. These referees will also be more familiar with the contents of the paper, and probably more interested in it. Therefore as a result of this, the publication process for papers coming from conferences may be shorter. Extended versions offer advantages to the community as well. As mentioned, the quality and value of the contribution has already been recognized and “tested,” especially if the paper was presented at a highly selective conference. Not only have extended versions passed through a first reviewing process, but by the time a paper is submitted to a journal, it has already been rereviewed—maybe several times— rewritten, and restructured, and the journal reviewers can focus on the “bigger” questions about the scientific contribution of the paper, instead of the “lower” questions: “Is it a significant contribution to the disciplines? Or, is it breaking/opening a new area of research? Or, is there a major insight into a problem that has not been made before?” On their part, readers can benefit from a more definitive version of the work, especially when the work has been extended to a version that “is more appropriate for a nonexpert in a subfield, because it is usually explained in more detail.” Comparing the two versions. Most editors defined the journal version of the research work as more satisfactory than the conference version, with some caveats. There are basically four reasons that the journal version can be considered more satisfactory. First, the journal article is complete, or, at least, more complete, having solved, in theory, any problems raised at the conference discussion. The language presentation is expected to be better, and concepts should have reached maturity and full clarification. Second, as a consequence of their completeness, journal versions should allow other researchers to use the work either “to recreate the results or to build off of them,” or as a basis for students’ work. Third, journal versions have passed through at least two rounds of review, and a strict editing process. Finally, authors do not have to deal with page limitations, and thus can explain their work better. 822

On the other hand, several editors pointed out that both the conference version and the journal version are equally useful, and they accomplish different purposes in the discipline. In this perspective, conference papers serve to sketch out one’s ideas, always backed up by some evidence. They may also help to highlight the topics of the conference, because they contain the “exciting part” of the paper, the major innovations, and are also shorter, sharper, and denser than most journal papers. One editor differentiated the point of view of the readers from that of the authors. For an author, a journal paper is indeed “a more satisfying thing to produce.” Then again, it can take too long for a reader to find the core of the subject in a journal paper, unless it is really well written. Conference papers are more immediate, and authors get quick feedback and a quick release of their results in the community, and in this way, discoveries can be communicated. Publishing in the faster track of conferences, “authors establish precedence and ownership of ideas in the field,” and, competing in getting there first, many of them won’t bother to do journal publication. It is in the journal version, however, where one can see if the ideas are good and the results really useful. Despite the journal version being regarded as more satisfactory, according to most editors interviewed, there is a regrettable tendency to publish more and more in conferences and less and less in journals. This tendency seems to be more noticeable among young researchers, both from academia and industry, and deprives the discipline of carefully reviewed work, which is “explained fully enough so that even a nonexpert in that subfield can gain an understanding of what this work represents.” Lack of funding, as well, may prevent authors from further working on their paper. This is the case especially for authors coming from industry: “For industrial groups it is important to show their face, and less important that other people know in detail what they do,” commented one editor. Summary: How authors are expected to extend their papers. Most editors complained that authors are too often content with their conference submission and do not bother to make the extra effort that writing a journal article requires. Editors also admitted that not all conference papers can or should be extended for publication in a journal. In fact, the first concern of an author should be to establish if the paper is amenable to extension, and if they have significant new material to add on; otherwise the attempt will probably be unsuccessful. When the results of the conference paper do not allow for replication, for example, it is definitively necessary to expand. Once it is determined that the work can be extended, authors should concentrate on its foundations and try to “provide scientific answers” to those problems they decide to expand on. Taking care with their new list of references is pivotal, because the technical baseline may have advanced since the conference version, and because in this way they can “really place their contribution in the context of the related technique.” In this context, it is important that authors follow the referees’ suggestions, as they can point to


relevant research and come up with interesting ideas for expansion. Other conference contributions on the same or related topics may help in extending the paper as well. Authors should also be ready to entirely rewrite their papers. Instead of adding a few sections here and there, a proper extension must be adequately structured and consciously addressed to a different audience. The journal paper is intended to be archival, there is no personal interaction, and one must be able to refer to it as a complete and finished work. In doing so, however, they should take care not to overwrite. Authors should say clearly what is new in their journal version, especially for the benefit of those searching for relevant literature. Results From the Authors Survey The author survey provided information on what authors actually did when extending their conference papers. The results of the Web survey are illustrated in Figures 1–4. In the first part of the questionnaire, we asked questions regarding the conference version of the paper, and in particular we tried to find out if the discussion following the conference presentation had had some influence on the extension. In the second part of the questionnaire, authors were asked to indicate what they changed/added, and their opinion about the two versions of their work. Results are summarized in Figures 1–4, and the questionnaire questions are reported in the Appendix. Figure 1 shows how authors evaluated the contribution of the discussion to the extension of their paper. Even though the single greatest percentage of authors (37%) responded that there was a partial influence of the discussion on the extension process, the authors considering the influence of the discussion irrelevant and nonexistent together account for 50% of the total respondents. Explaining their evaluation of the role of the discussion in the extension process, authors often provided similar answers to the subsequent open-ended question, regardless of the evaluation they had given in the previous question. Only an increase in the emphasis put on each choice is noticeable. When authors chose “not at all” or “not applicable” in Question 1, they meant that the discussion was neither significant nor deep, whereas when they chose “not relevantly,” they often meant that the discussion was short or did not take place. Even if short, however, the discussion may have helped them

FIG. 1. Authors’ responses the question, “Did the conference discussion following your presentation influence the way you revised the paper?”

clarify some points, which is still evaluated as “not relevant.” This clarifying role of the discussion is more strongly stressed by those answering “yes partly,” who also insist on other contributions having been derived from the conference discussion: pointers to related works, new viewpoints, emphasis on some aspects, and so forth. Finally, authors choosing “yes very much” indicate the importance of the conference to get ideas and start a thinking process, but also mention that the discussion pointed to some technical problems. The audience authors addressed at their presentation was most often formed by only academics (55.2%), or sometimes by academics in combination with students (18.4%), or with practitioners (7%), or with students and practitioners (4.4%), as shown in Figure 2. The composition of the audience has no effect on the perceived influence of the discussion on the extension of the following paper. We applied a chi-square test to verify if the two variables were related, and in particular if a composite audience has a more positive effect on the extension than an audience constituted only by academics. The answers to the first question were combined in two groups, positive and negative, whereas the answers of the second question were reduced to two options: academics and composite audience. The Pearson chi-square test produced a score of 2,419 with 2 degrees of freedom, and a not statistically significant p-value of .298. Asking about the type of feedback received, we differentiated between questions, comments, suggestions, and problems (Figure 3). The most typical feedback received was questions, either alone (22.4%) or in combination with comments (19%). Problems are hardly ever mentioned and appear usually in combination with another type of feedback. One would expect that the more differentiated the feedback received, the stronger the effect the discussion has on the extension of the paper. We elaborated on the answers, reducing them to three options: questions alone, questions and other feedback, and comments and/or suggestions. The Pearson chi-square test produced a statistically significant score of 35,364 with 3 degrees of freedom, pointing to a relationship between the type of feedback received and the influence of the discussion on the extension of the paper. The more diversified the feedback, the greater the influence of the discussion on the revision process. In Question 5, authors indicated not only which sections of the paper they changed, but also whether such change was complete, considerable, or weak (Figure 4). The main body and the discussion are “considerably” changed more often

FIG. 2. Composition of the audience attending a conference presentation.


823

FIG. 3.

FIG. 4.

Feedback received at the presentation.

Parts of the manuscript changed for extension to a journal article.

than any other sections. On the other hand, the background and conclusions are more often “barely” changed than the others. Interestingly, the methodology section was often ticked as not applicable, maybe pointing to the frequent lack of this section in SE papers, as already shown by Posteguillo (1999). Items of new information added to the extended versions are reported in Table 6. Several additions contribute to better and more clearly communicating the message, such as examples, definitions, graphical material, and the reshaping of the presentation and/or exposition. Authors tended to mention the results section more often than any other when asked what sections were modified/augmented/inserted. Out of 90 authors answering this question, 34 mentioned the results, 8 the discussion, 4 the conclusion, and 2 the introduction. Finally, a special emphasis is placed on the paper becoming better placed in the context of the problem at issue or of the discipline in general (see Relationships in Table 6). 824

Only 19 responding authors (17.9%) deleted information when extending their papers, whereas 87 (82.1%) claimed that they had deleted no information. There is no statistically significant relationship between this variable and any of the sections of the paper changed. In the related open-ended questions, the authors explained that, by deleting information, they wanted to make their paper more focused. When it comes to comparing the two papers, the percentage of authors considering the journal version of their work more satisfactory is strikingly higher (84.1%) than the percentage considering both versions equally satisfactory (15.9%). Statistically, there is a significant relationship between this variable and the variable corresponding to the changes applied to the main body ( p .049). It is basically as a consequence of the changes and additions applied that the journal version tends to be more satisfactory to the authors. Thus, we will not repeat our discussion from the previous paragraphs. Other factors influencing the


TABLE 6.

Summary of the changes/additions made to the extended version.

Algorithms

Algorithms included in the journal version are qualified as new, simpler, more generalized so that they can “work for a broader class of problems,” and more detailed.

Analysis

A deeper and more detailed analysis is carried out, sometimes supported by graphics and mathematical tools. Thoughts are developed, whereas ideas are improved and extended. As a consequence, authors may gain additional insights.

Applications

More applications are described, with details, implications for practitioners, practical exploitations, added implementations, and related problems.

Corrections

Mistakes are corrected.

Definitions & Explanations

Definitions are provided of terms and controversial elements, sometimes a new section is added to leave room these. Explanations about the problem at hand and the paper are provided, sometimes improved and differentiated.

Examples

More or new examples are given or presented in a more detailed way.

Exposition & Presentation

The exposition and presentation of the paper are improved, extended, and/or reorganized. Sometimes, the presentation can be completely rewritten, and the structure changed.

Graphical material

Diagrams, more figures, pictures, and graphics are added, sometimes to support data analysis.

Merging

Two or more papers may be merged.

Motivation

Motivations and motivational material or sections/parts are added. Some choices may be justified.

No major changes

The two versions are practically identical.

Omissions

If some sections or some results are omitted, authors do so to achieve focus.

Proofs

The proofs included in an extended version of a conference paper are clearer, simplified, full, significantly revised, elaborated, and/or expanded.

Relationships

A diversified range of relations may be established, and in particular comparisons, relations with context, and relation to other work. a. Comparisons Comparisons are made with other techniques, alternate approaches or methods, new examples, other problems, products/techniques discovered after the conference paper (such as new algorithms), and related works. b. Context The context is broadened, more contextual information is provided, techniques may be set in a wider context. c. References & Related Work More references are included, either to subsequent and new related work, or to existing studies with more details and discussion.

Sections

New sections may be added, or reshaped and augmented. a. Conclusions In this section authors mentioned concluding remarks, concluding elements to better define the limits of the study, or more contextual information for conclusions “that were thinly commented in the conference paper due to page limitations.” b. Discussion A discussion section is included/expanded for assessment, implications of the results, and related work. The novelty and interest of the results are discussed in more detail. c. Introduction Additional research questions may be added. d. Methodologya In the case of empirical studies in particular the methodology section is extended with experimental validations, new or more experiments, or more extensive empirical studies. The methodology, which in this case is probably understood also as the object of the research, can be better formalized, completely rewritten, more detailed, explained, revised, and refined. e. Results The journal results are more precise, selected (those not directly relevant have been removed), augmented, analyzed in a more complicated framework, improved, more detailed and elaborated upon, and strengthened. Their presentation is modified and made more readable.

Technical Detailsa

Technical details and explanations of techniques are provided, sometimes as a consequence of the author “obtaining a more clear insight.”

Title

Authors assign a new title.

judgment of a paper as more satisfactory include suggestions and comments made by referees and other members of the community, or comments received at the conference itself, or at seminars and meetings where the authors presented their work later. Time and space are also crucial factors. With less page limitations, the work is presented in a more detailed, comprehensive, and complete way. Some even consider time as more of an influence than the conference discussion. Authors spend more time on the work and therefore

are able to look at it with more perspective, resulting in the journal paper being a more mature and updated version of the work. In the process of extension, new ideas may come up, and ideas presented in the conference version may be proved to be correct. With better control over their work, authors can also simplify and clarify proofs, formalizations, and the presentation of the paper or parts of it, especially the results. This results in the whole paper or parts of it being more general, understandable, and readable. The conference


825

version, on the other hand, is a rougher, preliminary presentation of the work, which may also leave out important points. It presents intuitions and ideas, and may contain proofs that are only outlined or difficult to understand. In conclusion, “There is no relevant information in the conference paper that cannot be found in the journal version. The opposite is not true.” When both versions are considered satisfactory, authors recognize that they accomplish two different functions, or that there are no major differences between the two versions. An advantage of the conference paper over the corresponding journal paper is its conciseness. It is therefore a good choice for a casual reader, looking for a quick overview of the topic. Those who want to reconstruct the results in a different context or for a different purpose would be better off starting with the journal version. Conferences, finally, offer a much faster publication venue. Discussion It is not known how many SE conference papers become journal papers. However, from this research it has emerged that the process of extension, and sometimes merging, of conference papers into journal papers follows certain established patterns of research in the field, recognized by both authors and journal editors. After analyzing the extensive information obtained from our research, we can comment on some characteristics of this process. First, it seems that the opportunity for personal and direct communication with colleagues offered by conferences is seen as the most effective way of getting the information needed for completing/improving research work. The importance of oral communication in all engineering fields is well known in information science, since Allen’s seminal work (1977). More recently, Yitzhaki and Hammershlag (2004), studying Israeli computer scientists and software engineers, found that among the most used sources of information “oral discussions with colleagues/experts” within one’s organization topped all lists of accessibility and use. It did not matter if the subjects surveyed worked in academia or in industry. Hertzum and Pejtersen (2000) also note that engineers often turn to colleagues for obtaining solutions to their information problems and gaining access to written documentation. Written sources may lack adequate information about the reason for adopting certain solutions or about the context of the design process, which often has implications in different domains. In addition, direct contact with other engineers can allow researchers to get answers tailored to their specific needs. Fidel and Green (2004) argue that human resources are paramount for engineers, who generally choose their human sources by applying other criteria than those applied to select written sources. Finally, Sharp, Robinson, and Woodman (2000), in a work on community and culture of SE, stress the importance of human factors in the field, and in particular of personal opinions, which very often are preferred to objective evidence. From this point of view, then, conference presentations and discussions could 826

be seen as a phase in the research process in which authors look for information and opinions from peers to strengthen the foundations of their work. Interaction with peers and/or students/practitioners revolves around the conference and takes place at various stages, before, during, and after the event, with all stages being equally significant. Before the conference, authors receive comments and opinions from referees. During the conference, they interact with colleagues both at the presentation and informally, and probably with journal representatives, if their paper is chosen for publication in a special issue. After the conference, they may interact with journal referees, colleagues who have read the conference proceedings, or with new research partners met at the conference. The information authors get from the different stages of the conference includes specialized knowledge, opinions and viewpoints, and references to similar/related work, either published or being done. Another element intrinsic to the extension process is that in a fast-evolving area such as SE, authors need to establish precedence and ownership of ideas as soon as possible. The conference venue allows authors to publish their ideas in a much faster way than journal publication, which is often hindered by long reviewing times. Review, on its part, characterizes extended versions of conference papers in that it takes place in a double round, and can have a determining role on the direction of the research. Though some authors complained about journal review taking up too much time, several editors highlighted the role of reviewers, both at the conference and the journal stage, in improving the paper. It may be that reviewing has a different character in SE than in other areas, in the sense that conference reviewers especially may influence not only the paper but also the future development of the work, if the paper is extended. Indeed, conference reviewing seems to be more rewarding in terms of recognition of one’s work, because it is not completely anonymous, and may cause a stronger involvement on the part of the reviewers. In this sense, it has an advantage over traditional peer review as described in Bence and Oppenheim (2004). In any case, the reviewers’ contribution to the final version of the work can be considerable, making a difference in the role of meetings in the production of journal papers described by Garvey (1979). In Garvey’s research, meetings and conferences served as filters in the scientific information flow, eliminating over one third of research from further dissemination because it was proved unimportant or mistaken after the presentation (1979, p. 51). In this research, exclusion of unimportant or mistaken research takes place mostly before the conference presentation, through the refereeing process, while conference referees can have a say in the future development of the research. It is probably due also to the long review time for journal publication that many authors in SE don’t bother extending their conference papers. The median time lag (2–4 years) between conference publication and journal publication we calculated for our corpus is evidence of this common complaint, and can also support the claims of inadequacy of the SCI system for SE and the computing disciplines. Research


performance is calculated by the SCI on the basis of journal publication, but what is published in journals may have appeared originally 2–4 years before. In fast-evolving fields, these calculations may fail to transmit actual research dynamics. Conference and journal publication serve two different and complementary functions in SE. The editors interviewed in this research consider conference and journal publication both indispensable. Certain recognition has now been obtained for conference publication, which is taken into account for promotion and tenure of SE academics, though it is a general complaint that many, mostly younger, researchers nowadays do not bother to extend their conference papers for publication in a journal. One wonders, then, what functions these different types of publications should fulfill, especially now that conference proceedings are as easily accessible as journal articles. If not all papers are amenable to extension, and some will remain conference papers and be published only in conference proceedings, why is it in other cases necessary to extend them? Sometimes, it may be the type of the paper that determines whether it is necessary to extend it or not, as pointed out by some editors. But there is also a shift in the communicative purpose of these two different forms of publication. Based on editors’ interviews and authors’ questionnaire answers, we present in Table 7 an outline of the functions conference papers and journal papers are supposed to accomplish. The opinions of the authors surveyed in this research represent only those authors who have extended their conference paper. Editors’ opinions, conversely, are more general but they still represent journals. Our sources could be biased in the sense of favoring journal publications. However, editors and authors play different roles within the community; depending on the circumstance, they may also be conference attendees, referees, or conference authors. Besides, the degree of agreement between the editors and authors taking part in the research is quite high. Among other things, they agree on the judgment that a journal paper tends to be a more satisfactory piece of work than a conference paper, but nobody denies that conference publication also has an indispensable role. Authors’ answers about what they change when extending their conference paper support the data reported in Table 7.

TABLE 7.

That conference papers often report only part of the work is confirmed by the main body being the section most often “considerably” changed. Changes to the main body are also related to the authors’satisfaction with the two versions of their work. Further evidence supporting this also comes from the authors’ rarely deleting information from the conference to the journal version. The discussion, the section where authors usually discuss the significance of the work in the context of the discipline, is also often considerably changed, and authors also claimed that they established a differentiated set of relationships, as reported in Table 6. Then again, the introduction, the background, and the related work sections, where authors usually present and discuss the problem from which the research originated, are most often only “barely” changed. In other words, the central idea or problem tends to remain unchanged from the conference to the journal, while the work is more definitively placed in the context of the discipline in the journal article. Of course, all these changes/additions are applied only to those conference papers that are extended. In any case, bibliographic databases specialized in SE must take this into account and, whenever possible, provide a link from the conference to the journal version. Conference papers may often be incomplete. Comparing SE with formal and empirical sciences, Marcos observes that whereas the latter “deal with the study of existing phenomena and objects, engineering sciences deal with the study of the methods and techniques necessary for the creation of new objects and even with the creation of such methods and techniques.” (2005, p. 1). Similarly, March and Smith (1995) point out that a key aspect of IT research is that it deals with artificial phenomena and human creations instead of natural phenomena. There is thus a creative effort in the SE research process, which may account for the importance attached to new ideas and the ability to innovate. In this perspective, conference publication could be seen as innovation-laden, and journal publication as intended to settle a knowledge basis. This may clarify why conference papers are considered valuable for productivity, though they are at the same time preliminary. From this perspective, the conference appears the best venue to establish the degree of innovation of certain research because it often gathers the specialists in a certain area.

Functions of conference papers versus journal papers. Conference Paper

1.

Contains the exciting part of the paper, the major novelty and innovation.

2. 3. 4. 5.

Is shorter and sharper, and contains all the essence of the contribution. Is a snapshot of the work, and serves to sketch out one’s ideas and present one’s intuitions. Helps one keeping abreast of research in the field. Is often intended for a very specialized audience.

6. 7.

The publication process is quicker. The work may have been further developed in subsequent articles.

Journal Paper Contains the details that allow someone to replicate the results or to fully understand the results. Is longer and rounds out the subject. Is the complete, mature report of the research. Is archival and intended to last for some time. Is often intended for a more general audience, and can be the basis of students’ work. The publication process is often very long. The work is fully described and the article is self-contained.


827

As a matter of fact, innovation is not a stringent requisite for publication of extended versions of conference papers. Those journals that regulate the publication of this type of paper insist on the need to add new material, but not to innovate. The IEEE Transactions on Visualization and Computer Graphics (TVCG Author Center, 2006), for example, doesn’t require new results, but expects key ideas, examples, and further elaboration. The editors interviewed, besides, pointed out that a good reason for extending conference papers is to make them accessible to a wider audience. In these terms, the issue of republication doesn’t concern exclusively SE, but affects all disciplines, when, for example, a previous version of a work has already been published on the Internet (Kling & McKim, 1999). In this case, Kling and McKim recommend considering to what extent “the manuscript has been effectively published within the relevant readership communities.” In other words, the novelty of a contribution depends also on the audience it is addressed to. The limitations of current systems to measure research productivity and journal impact factors are well known, and research is underway to develop better ways to account for research productivity (Moed & Visser, 2007). The results of this study thus may be of interest to bibliometricians, especially in connection with the problem of dealing with different versions of the same research work when performing citation analysis. One of the alternatives proposed by Moed and Visser is a conceptual approach to citation analysis, considering all references to different versions of the same research work together. This research may have cast some light on this still unexplored issue. Bibliometricians should also be aware of the fact that one or more citations to conference papers may be different from others because of reference to the author’s previous work, and because that work is included in the same article. Similarly, when publications are counted for research productivity evaluation, the different functions of conference papers and journal papers should be taken into account as much as possible. Conclusions and Future Research In this article we have reported the results of a study on a type of paper published in SE, specifically, the extended version of a conference paper. We have interviewed editors in chief and associate editors in chief from major publications in the field, and surveyed authors of extended versions of conference papers. The results show that extending conference papers for publication in journals is a common way of carrying out research work in SE, though this practice has not yet been quantified. Several issues related to this research pattern have been discussed, and other issues have been commented on, such as the role of reviewers in the extension process and the functions that conference and journal papers have. The results achieved can be useful to those bibliometricians studying research assessment in the computing disciplines. Considering the exploratory nature of this research, we can finally comment on some problems that future research could address. 828

There are studies on the literature citation in SE, but, as far as we know, no study has addressed the amount of conference publication in SE in comparison with journal publication. It would also be interesting to know what percentage of conference papers become journal papers, and to better quantify the time lag between conference publication and journal publication. Since both conference and journal papers are considered indispensable, we have tried to explain what functions they accomplish. But our tentative explanation that conference publication can measure the ability to innovate and propose new ideas, whereas journal publication can more strongly contribute to building a knowledge basis, needs to be confirmed or disproved by further research. Acknowledgments This research has been funded with a grant from the Spanish Ministry of Education and Culture. I would like to thank the three anonymous reviewers for their valuable suggestions for improvement, and Miss Vanessa Fairbank for revising the text of the paper. References Allen, T.J. (1977). Managing the flow of technology: Technology transfer and the dissemination of technological information within the R&D Organization. Cambridge, MA: MIT Press. Barjak, F. (2006). The role of the Internet in informal scholarly communication. Journal of the American Society for Information Science and Technology, 57(10), 1350–1367. Bence, V., & Oppenheim, C. (2004). The influence of peer review on the research assessment exercise. Journal of Information Science, 30(4), 347–368. Computing Research Association (1999, September). Best practices memo: Evaluating computer scientists and engineers for promotion and tenure. Computing Research News. Retrieved August 29, 2006, from http:// www.cra.org/reports/tenure_review.html Coulter, N., Monarch, I., & Konda, S. (1998). Software engineering as seen through its research literature: A study in co-word analysis. Journal of the American Society for Information Science, 49(13), 1206–1223. Drott, M.C. (1995). Reexamining the role of conference papers in scholarly communication. Journal of the American Association for Information Science, 46(4), 299–305. Fidel, R., & Green, M. (2004). The many faces of accessibility: Engineers’perception of information sources. Information Processing & Management, 40(3), 563–581. Garvey, W.G. (1979). Communication: The essence of science. Facilitating information exchange among librarians, scientists, engineers and students. Oxford: Pergamon. Goldberg, A.J. (1982). Editorial policy. ACM Computing Surveys, 14(2), 151–157. Goodrum, A.A., McCain, K.W., Lawrence, S., & Giles, C.L. (2001). Scholarly publishing in the Internet age: A citation analysis of computer science literature. Information Processing & Management, 37(5), 661–675. Hertzum, M., & Pejtersen, A.M. (2000). The information seeking practices of engineers: Searching for documents as well as for people. Information Processing & Management, 36(5), 761–778. Hjørland, B. (2003). Fundamentals of knowledge organization. Knowledge Organization, 30(2), 87–111. ISI Web of Knowledge (2006). Journal Citation Reports, Retrieved July 21, 2006, from http://portal.isiknowledge.com/portal.cgi?DestAppJCR&Func Frame Kitchenham, B.A., Pfleerger, S.L., Pickard, L.M., Jones, D.C., Hoaglin, D.C., El Emam, K., et al. (2002). Preliminary guidelines for empirical research in


software engineering. IEEE Transactions on Software Engineering, 28(8), 721–734. Kling, R., & McKim, G. (1999). Scholarly communication and the continuum of electronic publishing. Journal of the American Association for Information Science and Technology, 50(10), 890–906. Kling, R., McKim, G., & King, A. (2003). A bit more to it: Scholarly communication forums as socio-technical interaction networks. Journal of the American Society for Information Science and Technology, 54 (1), 47–67. March, S.T., & Smith, G.F. (1995). Design and natural science research on information technology. Decision Support Systems, 15(4), 251–266. Marcos, E. (2005). Software engineering research versus software development. ACM SIGSOFT Software Engineering Notes, 30(4), 1–7. Marion, L.S., & McCain, K.W. (2001). Contrasting views of software engineering journals: Authors cocitation choices and indexer vocabulary assignments. Journal of the American Society for Information Science and Technology, 52(4), 297–308. McCain, K.W., Verner, J.M., Hislop, G.W., Evanco, W., & Cole, V. (2005). The use of bibliometric and knowledge elicitation techniques to map a knowledge domain. Scientometrics, 65(1), 131–144. Moed, H.F., & Visser, M.S. (2007). Developing bibliometric indicators of research performance in computer science: An exploratory study. Research Report to the Council for Physical Sciences of the Netherlands Organization for Scientific Research (NWO), CWTS Report 2001-01. Montesi, M., Mackenzie Owen, J. (2008). Research journal articles as document genres: Exploring their role in knowledge organization. Journal of Documentation, 64(1). Posteguillo, S. (1999). The schematic structure of computer science research articles. English for Specific Purposes, 18(2), 139–160.

Referee Guidelines. (2006). ACM Transactions on Database Systems. Retrieved August 29, 2006, from http://www.acm.org/tods/Referees.html# Guidelines Sharp, H., Robinson, H., & Woodman, M. (2000). Software engineering: Community and culture. IEEE Software, 17(1), 40–47. Sidiropoulos, A., & Manolopoulos, Y. (2005). A new perspective to automatically rank scientific conferences using digital libraries. Information Processing & Management, 41(2), 289–312. Sjøberg, D.I.K., Hannay, J.E., Hansen, O., Kampenes, V.B., Karahasanovic´, A., Liborg, N., & Rekdal, A.C. (2005). A survey of controlled experiments in software engineering. IEEE Transactions on Software Engineering, 31(9), 733–753. TSE Author Center. (2006). IEEE Transactions on Software Engineering. Retrieved August 29, 2006, from http://www.computer.org/portal/site/ transactions/menuitem.eda2ca84d8d67764cfe79d108bcd45f3/index.jsp ?&pNametransactions_level1&pathtransactions/tse/mc&fileauthor. xml&xslarticle.xsl& TVCG Author Center. (2006). IEEE Transactions on Visualization and Computer Graphics. Retrieved August 29, 2006, from http://www.computer .org/portal/site/transactions/menuitem.eda2ca84d8d67764cfe79d108bcd 45f3/index.jsp?&pNametransactions_level1&pathtransactions/tvcg/ mc&fileauthor.xml&xslarticle.xsl& Wieringa, R., Maiden, N., Mead, N., & Rolland, C. (2006). Requirements engineering paper classification and evaluation criteria: A proposal and a discussion. Requirements Engineering, 11(1), 102–107. Yitzhaki, M., & Hammershlag, G. (2004). Accessibility and use of information sources among computer scientists and software engineers in Israel: Academy versus industry. Journal of the American Association for Information Science and Technology, 55(9), 832–842.

Appendix Questions From the Semistructured Interviews With Editors 1. 2. 3. 4. 5. 6. 7. 8. 9.

What’s the first thing you would do, if you had to judge an extended version of a conference paper? Does the name of the conference mean anything to you? Do extended versions of conference papers have typical pitfalls or problems? Do extended versions of conference papers have any advantage/disadvantage in comparison with papers written exclusively for publication in a journal? When a paper is presented at a conference, there is usually a discussion that takes place afterwards. Do you think this discussion may influence the way authors revise their papers? Which other factors may influence the way authors extend their papers? Comparing the conference version with the revised version, could you tell that generally one is more satisfactory than the other? What would you recommend to authors needing to extend their conference paper for publication in a journal? Do you have anything else to say on the topic, that you think is especially important?

Questions From the Author Survey 1. 2. 3. 4. 5. 6. 7. 8. 9.

Did the conference discussion following your presentation influence the way you revised the paper? Why? How? Who took part in the discussion? What kind of feedback did you receive? Which parts of the paper did you change? What kind of new information did you add? Did you delete information? What is your opinion about the two versions of your paper? Could you explain briefly the reasons for your answer to the previous question?


829