BULETINUL ŞTIINŢIFIC al Universităţii “POLITEHNICA” din Timişoara, România, Seria AUTOMATICĂ ŞI CALCULATOARE SCIENTIFIC BULLETIN of The “POLITEHNICA” University of Timişoara, Romania, Transactions on AUTOMATIC CONTROL and COMPUTER SCIENCE, Vol. xx (yy), No. z, 20vv, ISSN 1224-600X
Final preprint submitted for publication
Using Qualitative Research to Explore Automation Level of Software Change Request Process: A Study on Very Small Software Companies Zeljko Stojanov* *
University of Novi Sad, Technical Faculty “Mihajlo Pupin”, Djure Djakovica BB, 23000 Zrenjanin, Serbia
[email protected]
Abstract – Software change request process is an essential sub-process in software maintenance that deals with modifications of software systems. This paper present an empirical study that uses qualitative research methods for investigating automation level of software change request process in local very small software companies. Findings of qualitative research provide a grounded understanding of the practice that is based on opinions of software experts. In research participated fifteen software experts from local very small software companies with various level of experience. Software experts were interviewed individually in their companies or participated in organized focus groups. Constructivist grounded theory methods were used as a suitable choice for discovering common practice and for creating explanations that are grounded in empirical data. Better understanding of the study is achieved through description of the research context and procedures. Research findings are presented with the focus on inductive analysis that led to identification of automation level as a property of software change request process. Analysis of empirical data reveals that very small software companies do not have automated software change request process, or have automated only parts of the process. In the paper are also considered validation and constraints of the study. Despite identified constraints, this study helps in building deeper understanding of current practice in very small software companies, helps in identifying problems in the practice as potential staring points for further improvements, and contribute to the body of empirical knowledge in software engineering.
engineering’s research is rather directed toward providing prescriptions to practitioners what they ought to be doing, then to understanding what practitioners actually are doing [2]. According to Glass, this is the consequence of the fact that researchers too often simply don’t know what goes on in practice, while practitioners often have their noses buried so deeply in today’s projects. This standing can be illustrated with the following quotation taken from [3]: “The first thing that struck us when we entered the work place was that we did not know exactly what it was that the software engineers did on a day-to-day basis. That is, we knew neither the kinds of activities they performed, nor the frequency with which these various activities took place. … Consequently, we decided to begin our study of work practices by finding out what it is that software engineers do when they do their work.” A serious problem with research and practice in software engineering is related to understanding of the specific factors that cause tools and methods to be more or less cost-effective in the practice [4]. This further leads towards the gap between research and practice. The main reason for this gap is the fact that practitioners are deeply involved in today’s projects, while researchers often do not know what actually happens in practice. The problem of understanding between researchers and practitioners is bi-directional. While researchers are focused on new development of paradigms, algorithms, methods and tools, practitioners have difficulty putting these innovations to work. In contrast, researchers have difficulty in addressing practitioners’ problems faced in real settings in their researches. Practitioners will perceive researchers’ solutions as helpful only if researchers recognize that practitioners live in an imperfect world and understand issues that constrain their daily activities [5]. The gap between the state of the research and the state of the practice could be overcome through better understanding between researchers and practitioners. That means that researchers must place more emphasis on the problems identified in industry, while practitioners must frame their problems so that researchers can address the underlying scientific issues. According to Rainer [6], researchers need
Keywords: software changer request, process automation level, very small software companies, qualitative research. I. INTRODUCTION Software engineering research has been recently more oriented towards real industrial problems and practice [1]. This means that solutions should be simple enough in order to be adopted in practice. However, most of the software 1
to strengthen collaborations with industry, because industry is both the most appropriate source of empirical evidence for developing understanding of the practice and the intended target for action. Higher quality, greater relevance of empirical studies, and better transfer of research results can be achieved by using the software industry as a laboratory [7].
is typically perceived as being expensive and ineffective [20]. Therefore, for the investigation is selected software change request process as an essential sub-process in software maintenance [21]. In maintenance phase of software life cycle software change request process is usually initiated by software users that report problems or request new feature in software products they use.
Small companies are dominant in economies across the globe [8]. Research focused on small companies is important because of their importance to economic activity, employment, innovation and wealth creation in many countries [9]. U.S. Census Bureau’s “1995 County Business Patterns” pointed that the vast majority of software and data processing companies are small, and that those with more than 50 employees comprise only a few percent of the total number [10]. Coleman and O'Connor [11] reported that 61% out of a total of 630 indigenous software companies in Ireland employed 10 or fewer people. Laporte et al. [12] reported that in Europe, 85% of IT sector companies have between 1 and 10 employees, while in Montréal area in Canada over 50% have fewer than 10 employees. Organizations that have less than 25 employees are defined as very small enterprises (VSEs) [12][13][14].
Software engineering is a social activity with a number of unique management and organizational issues, or people problems [22]. In empirical software engineering are dominant quantitative studies that usually test hypotheses through experimentation and statistical analyses of the results. Research that emphasizes how software practitioners make sense of methods in their daily work is well suited for qualitative research methods [23]. Comparing to quantitative research, qualitative research is more flexible but less controllable. The findings from qualitative studies provide insight into the social dimensions that are an intrinsic part of software engineering practice. Silverman [24] suggested the view of qualitative method analysis as how people do things, rather than how people see things. Qualitative research also provides findings and knowledge that is an essential prerequisite for the generation and testing of theories and hypotheses and for interpreting the results of such empirical studies [22].
The maturity levels of processes within the very small companies have broad range from very good processes to low level processes. Due to the small number of people involved in the projects and the organization, most of the processes are performed through an informal way ( ad-hoc) and without any documentation. The findings of the study [15] revealed that all respondents did not accredit with any type of software quality certification and 60% of them do not plan to adopt any kind of standard in the near future. However, in spite of this fact, scientific and public research attention has been mainly focused on large companies. The aim of empirical research is to explore, describe, predict, and explain natural, social, or cognitive phenomena through evidence based on observation or experience. According to Segal, understanding the nature of software engineering practice should be central to the discipline of empirical software engineering [16]. Understanding software engineering practice requires studying people software practitioners as they solve real software engineering problems in real environments [17]. Without assessments of current perception and practice empirical software engineers will find it difficult to identify and promote potential improvements in practice [18]. Assessment of current practice and identification of potential areas and barriers for practice improvement is the way software companies should follow in order to improve themselves.
When the research goal is to investigate common industrial practice the one of suitable research methods is grounded theory that is regularly used in qualitative research. Grounded theory method consists of a set of systematic, but flexible, guidelines for conducting inductive qualitative inquiry with the main aim to construct a theory [25]. Grounded theory can also be used as qualitative research method, in the cases when the result of the research is not developed theory but grounded explanation of the practice [26]. Because of its flexibility, grounded theory was in this research used as a method for exploring software engineering practice in very small software companies. This research used constructivist grounded theory research procedures proposed by Charmaz [26]. Constructivist grounded theory is based on creative writing as a form of expression that has the potential to reveal how participants construct their worlds [27]. This approach is pragmatic and assumes that researcher has some knowledge of the studied phenomenon. This practically means that researcher may have many years of experience in the field and/or is familiar with the relevant disciplinary literature. Preliminary literature review provides insight into the previous research and identification of gaps that should be investigated [26]. This can positively affect the research of the practice.
Literature review reveals that software maintenance is the most costly part of the software life cycle [19]. Despite that fact, software maintenance is in literature considered as the last phase in software life cycle with little attention comparing to software development. Software maintenance
In this paper is presented the process of investigating the automation level of software change request process in very small software companies. In the background section is briefly discussed the work related to qualitative research and grounded theory studies in software engineering. After
2
that are described research context and procedures in order to provide better understanding of the study. The inductive analytical process supported by grounded theory methods and research findings are outlined in the fourth section. Some constraints and validity of the study are discussed in the fifth section, while some ethical issues are outlined in the sixth section. The last section of the paper contains concluding remarks and further research directions.
The basic components of grounded theory practice identified by Glaser and Strauss [31] are:
II. BACKGROUND
Detailed understanding or exploration of a phenomenon or a problem can be achieved only if the research is conducted in the context where it occurs. This is even more important if the research goal is to investigate human behavior in the real context or setting and to get their opinions about studied problem. Qualitative research techniques are appropriate choice when the research goal is to understand the context or settings in which problems or issues occur. Morse stated that [28]:
Simultaneous involvement in data collection and analysis, Constructing analytic codes and categories from data, not from preconceived logically-deduced hypotheses, Using the constant comparative method, which involves making comparisons during each stage of the analysis, Advancing theory development during each step of data collection and analysis, Memowriting to elaborate categories, specify their properties, define relationships between categories, and identify gaps, Sampling aimed toward theory construction, not for population representativeness, Conducting the literature review after developing an independent analysis.
These components are in the core of grounded theory inquiry although some authors like Corbin and Strauss [32] and Charmaz [26] disagree about the proposition related to conducting literature review. The aim of delaying literature review is to avoid preconceived ideas and knowledge before starting inquiry. However, this is very problematic issue because all researchers bring to the inquiry considerable experience and background in disciplinary literature that affect researchers approach to inquiry. This issue is especially problematic when the research is conducted in technical sciences, like software engineering, and when researcher possesses many years of experience in the field.
“…qualitative methods not only provide us with the means to explore such complex and chaotic real-life situations, but also provide us with methodological choices—multiple options about how to tackle such a setting according to one’s identified research problem and long-term research goals.” (p. 393) The main characteristic of qualitative research is that qualitative researcher studies things in their real settings and attempts to interpret phenomena based on information obtained from people. From the perspective of sociologist [24], the particular strength of qualitative research, for both researchers and practitioners, is its ability to focus on actual practice in situ, looking at how organizations are routinely enacted.
Although grounded theory originators and principal users to date are sociologists, it has been found useful by social scientists from other disciplines, as well as researchers in education, public health, social work, and nursing. Recently grounded theory gain popularity in technical sciences, especially in the fields of information systems research and in last few years in software engineering. Grounded theory is useful because it provides general style and methodological framework for doing analysis that does not depend on particular disciplinary perspectives [33]. Grounded theory method is applicable to any research focused on people’s actions and interpretations in organizational and other social contexts [34].
Number of approaches exists for undertaking qualitative research [29][30]. One of the most popular research methodology used by qualitative researchers in the social sciences is grounded theory proposed by Glaser and Strauss [31]. According to Glaser and Strauss, grounded theory is discovering of theory from data systematically obtained from research that involves humans. The main difference between grounded theory and other approaches for qualitative research is that grounded theory approach does not use any a priory assumptions, while other qualitative methods try to test and verify theories. The main aim of grounded theory is to construct theory that fit and work for the research context. However, Charmaz suggested that grounded theory methods could be used for development of both grounded theories and grounded descriptions of experience [26]. Anyway, researcher should enter the phenomenon to discover what is significant from the viewpoints and actions of people who experience it. In that case researcher will better appreciate what is happening in a real setting and what things mean to participants.
Grounded theory methods have recently gained attention in empirical software engineering. It is used in studies with the focus on human aspects of software engineering. These methods have been used to discover a theory, or to provide grounded description of an investigated phenomenon from the practice [35]. Seaman advocated the usage grounded theory research methods to study software development and maintenance in real settings and to develop theories that insightfully and richly describe a phenomenon. [22].
3
Grounded theory have been used in many areas of software engineering such as cognitive dimensions in learning software modeling [36][37], understanding of software process improvement [38][11], installation and use of an electronic process guide within a small-to-medium software development company [39], descriptive process modeling [40], understanding the social processes that influence software team performance and the influence software methods have on those processes [41], the impact of background and experience on software inspections [35], investigation of agile user-centered design in practice [42], management of software change tasks using a state-of-thepractice IDE to a medium-sized open source software system [43], self-organization of agile teams [44], the importance of adequate customer involvement on agile projects [45], the rationale for a software architecture description [46], bridging cultural differences in distributed software development [47], examining how a software process is tailored for a project [48], or analysis of pair programming [49].
used as self-contained method for collecting data or in conjunction with other methods (individual interviews, surveys, experiments, or participant observations) [54]. In this inquiry focus group are combined with individual indepth interviews. In-depth descriptive field notes [25, p. 341-343] about the context, participants and the process of the research were written by the researcher as a complement to the collected qualitative data. Field notes contain observations and researchers’ personal reflections. Some quantitative data were extracted from the collected text and used as numerical evidence [55]. It is important to note that the final output of this study is not fully developed grounded theory, but rather grounded description of daily practice. A. Research Context In research participated software practitioners from very small software companies in Zrenjanin (Serbia) and Timisoara (Romania). The total number of software experts that participated in the research is 15. Experience levels of participants varied between two and twenty four years with an average of 7,47 years of experience in the practice (see Fig. 1). Software experts’ years of experience are recorded in field notes during the interviews. All software experts work in small software companies with less than ten employees.
III. RESEARCH METHODS In this research is used a tailored version of techniques that are common practice in constructivist grounded theory research [26]. Two types of interviews were used in the research as methods for collecting empirical data: focus groups and individual interviews [50][25][51]. The accepted rule of thumb is to plan three or four focus groups with less than ten participants in order to rich saturation of newly collected data [51]. However, in this research were conducted two focus groups in conjunction with in-depth individual interviews. Analysis of data collected in focus groups is usually based on traditional qualitative methods that include grounded theory, phenomenological approaches, and thematic analysis [52]. All interviews were semi-structured with focused open-ended questions. Practically, some interview questions were predetermined but during the interviews appeared additional open-ended questions. Open-ended questions provide the opportunity to research participants to choose the terms with which they want to construct their descriptions. The purpose of interviews was to gather information about research topic through a set of focused open-ended questions. Responses to open-ended questions enable the researcher to understand and capture the points of view of other people without predetermining those points of view through prior selection of questionnaire categories [53, p. 21].
Number of experts
Softw are experts' experience 5 4 3 2 1 0 1
2 3
4 5
6 7
8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Years of experience
Number of experts
Fig. 1. Software experts’ years of experience
Two focus groups and three individual in-depth interviews were organized during the research. The first focus group was organized with four participants in Business incubator in Zrenjanin. The second focus group with eight participants was conducted in Timisoara software business incubator. Focus groups’ participants were invited in the groups through the announcements on the local incubators’ web sites. Therefore, the participants in focus groups had non been known before the groups were conducted. All software practitioners participated in the focus groups on the voluntary basis. The focus group in Zrenjanin was conducted in Serbian, while the focus group in Timisoara was conducted in English.
The general research goal of this study is to develop a broad and deep understanding of the practice, rather than a quantitative summary. Participants’ responses were collected in two ways. All individual interviews were recorded and transcribed. In focus groups, participants were asked to write their answers on the paper after short discussion on the set of questions. The focus groups are effective and cheap way of collecting large amount of qualitative empirical data. In practice, focus groups can be
Other three software practitioners were individually contacted, and after that interviewed in their companies in Zrenjanin. Individual interviews were conducted in Serbian. All participants work in software companies that
4
develop software for both local clients and clients from other countries. Interviews were conducted from November 2009 to February 2010. The last two interviews were individual in participants companies.
develop ideas of basic concepts, explore relationships between them, and relate written material to available literature. In this research, memos were used to describe identified concepts, categories and their properties, relationships between categories, and the process of developing concepts and categories. Clear repetitions within the data [26, chap. 5] indicated the completion of data collection process, which happened after analysing the response of fifteenth expert. Data collection was terminated although there is a possibility that some additional concepts could be identified in few additional interviews with software experts. These new concepts could be also identified through different sampling strategy of software experts (different choice of experts for interviewing). For now, this can be treated as a limitation of this study.
B. Research Procedure The research started with an initial individual in-depth interview. The initial interview was conducted with a software expert with 21 year of working experience in small software companies. The expert was chosen because of his experience in solving unexpected problems and accumulated knowledge and skills that have a positive impact on day-to-day software maintenance tasks [56]. Analysis of collected data during the initial interview streamlined further data collection and helped in structuring focus group interviews and other individual interviews. After that, focus groups were organized in Zrenjanin and in Timisoara. The collection of data ended after two focus groups followed with two individual indepth interviews.
IV. RESEARCH FINDINGS Two sources of data were used in the research: participants’ answers on open-ended questions, and field notes recorded by the researcher during the interviews and in informal discussions with participants.
All interviews (focus groups and individual) lasted between 45 and 60 minutes. Individual interviews were recorded, while answers in focus groups were collected in the paper form. Each interview began with a short presentation of the research goal (practice exploration) and the research topic (software change request process in maintenance phase). After that, a short discussion about the current practice on collecting and processing software change requests from customers was conducted in focus groups. During these discussions were written field notes by the researcher. After the discussions, participants in focus groups wrote answers on open-ended questions. During the writing responses on open-ended questions, participants were also allowed to ask questions that helped them to provide appropriate answers.
Common practice and problems were identified and derived from available empirical data during the coding techniques described by Charmaz [26, chap. 3]. Participant’s answers were first coded with initial and after that focused coding techniques. Further analysis of data and codes provides the basis for defining concepts. Concepts were further raised to categories and their properties that describe common practice about software change request process in very small software companies. Data collection process was completed after the analysis of data from fifteenth interview because of the clear repetitions within the data. The repetition within the collected data is called saturation [26, chap. 5]. This saturation within the data also denotes the development of categories in terms of their properties and dimensions [32, p. 143]. All identified concepts during the coding process, and developed categories and their properties were described in written memos as it is proposed in [26, chap. 4]. Each memo has been changed several times during the analytic research process.
Analytic activities that form the core of grounded theory study are data collecting, coding and writing theoretical memos. Data collecting leads quickly to coding, and this further may lead to writing memos. Collected empirical data were carefully read and coded with techniques presented by Charmaz [26, chap. 3]. Grounded theory coding is iterative, inductive process aimed to organize data from which the researcher can construct themes, essences, descriptions and theories [57]. Constant comparison of collected data and identified codes against each other for similarities and differences lead towards identification of concepts as basic units for further analysis and development of grounded theory or description [58]. From the early beginning of the inquiry, the coding was frequently interrupted in order to write theoretical memos. Writing memos helps to clarify what is happening in the field and in collected data. Written memos are an efficient tool for analyzing data and codes, developing ideas and concepts [26, chap. 4]. Memos form the core of developed grounded theory, or serve as intermediate step towards generating research reports. Memos were regularly used to
As a core category was identified concept named software change request process. The core category represents the central phenomenon of the study and it emerged from among the categories already identified [58]. All other categories, such as Business policy of Software Company or Knowledge and experience of software experts, are directly or indirectly related to the core category. Although other categories influence the core category and its properties, they will not be discussed in more details in this paper. Categories were developed from memos, which are based on codes extracted from empirical data. Memos contain detailed descriptions of derived codes, concepts and categories. The automation level of the process is classified as a property of the core category. From the collected data
5
were also derived other properties of the core category [59]. The level of process automation in SCR specification practice is based on the responses provided by software experts. Sample experts’ responses are presented in the paper [60].
the Fig. 2 are presented only responses and initial codes from Table 1. It should be noted that all codes are related to using some kind of software applications (internal or web). Various phrases used by software experts associate on the existence and usage of software applications that automate some tasks in software change request process. Some of them are:
In Table 1 is presented initial coding of collected responses initially presented in the paper at SACI 2011 conference [60]. These responses denote some degree of process automation. In the left column are presented experts responses with underlined text used in initial coding, while in the right column are presented initial codes derived from presented responses. Some of the initial codes are the terms widely used by experts in day-to-day practice. Since these terms are used frequently they are adopted in the research as in vivo codes [26][33].
Phrase 1: “All of us in the company can receive requests, and each programmer tracks his requests in the internal application.” Phrase 2: “Programmer can see request, and if he does not want to accept it, he must explain rejection and send it to more relevant programmer. This way we can track all requests, and identify all unassigned.”
TABLE 1. Initial coding of sample responses. Expert’s responses A client submits a request usually by phone. Our team member records the request through an internal Web application. After that, the request is analysed as soon as possible and assigned to a programmer. This assignment is in internal application represented as a task. After implementing the requested change, the evidence about the change is recorded in the application. There is no feedback to the client about the status of the request in the process. Some segment of our process should also be supported by internal application through development of additional modules. When we finish the work on the request we prepare and send invoice to a client who submitted a request. Customers submit change requests by phone or by filling a web form on our web site. If the data submitted for change request are incomplete, the customer is contacted by phone or email to provide additional data. It usually takes 2-3 days to specify change request. The implementation of change depends on its complexity and feasibility. The steps in the process are not recorded internally.
Phrase 3: “We simply record such request in our application, and mark it as “on standby”. This can be a request that we do not processed immediately, and client forget on it.”
Initial codes recording request in an internal application
Phrase 4: “User can send a request or report problem directly from application by using email, or can call by phone.”
assigning request to a programmer
Using software applications or tools helps in automating some steps in the process, and this reasoning lead to development of the concept Automation level.
evidencing change completion no feedback improving internal application
Initial codes
Focused codes
Concepts
Recording request in an internal application
preparing and sending invoice to client
Evidencing change completion
Using internal application
submitting by phone or web application
Improving internal application
identifying incomplete requests contacting client again completing specification
Filling web form
Using web application
OTHER INITIAL CODES
OTHER FOCUSED CODES
influencing change implementation without recording steps
Automation level
Fig. 2. Analytical development of the concept Automation Level
Insight in the answers reveals that four experts reported partially automated SCR process in their companies. This quantitative data is extracted by counting the number of collected responses where it is noted the existence of partially automated process in the company. Other experts reported that there is no automated software change request process in their companies.
It should be noted that initial codes in Table 1 are short analytic labels in the form of gerunds (according to Charmaz [26, p. 49]) that identify specific processes and treat them theoretically. Other experts’ responses are initially coded in the same way. From the list of all initial codes were extracted through focused coding the concepts that are the most frequent and the most descriptive. In Fig. 2 is presented analytic process of developing concept Automation level that is further raised as a property of the core category. In
The main research finding is that the majority of local very small software companies does not have standardized and/or automated process for collecting and handling change requests. This is outcome of the fact that small
6
software companies have small teams that are task oriented. Furthermore, these findings agree with the discussion that can be found in literature about small software companies regarding other aspects of practice.
the nature and the consequences of the research in which they were involved through a document called Research Informed Consent. This document was presented to all participants, and after that signed by the researcher and each participant. This document ensured that subjects agreed voluntarily to participate in the research based on open information [66]. Participants agreed to participate in the research by signing the document. Participants anonymity was preserved in two ways: (1) they had possibility to sign the Research Informed Consent document with the personal name or just to write accept participation or participant, and (2) in the Research Informed Consent document is stated that participants names and the names of their companies will not be used in any published document that presents research findings [66][67][68]. On the other hand, we get agreement from all organizations that supported this research (business incubators) to use their names. This is in compliance with their business policies, and also helps in their promotion.
V. VALIDITY AND CONSTRAINTS OF THE STUDY It is adopted by researchers that validity refers broadly to the goodness or soundness of a study [25]. Research should use the conception of validity that is appropriate for the inquiry paradigms being engaged [61]. In this research, validity is achieved by using prescribed and wellentrenched grounded theory procedures and techniques proposed by Charmaz [26] and Corbin and Strauss [32]. Implementation of these procedures and techniques ensures that descriptive validity (accuracy of the collected data), interpretive validity (how well the researcher reports the participants meaning) and theoretical validity (theoretical constructions that the researcher brings to, or develops) proposed by Maxwell [62] are satisfied.
VII. CONCLUDING REMARKS It should be noted here, that this paper reports only a segment of the whole research that is conducted in order to explore software change request practice in local very small software companies. Therefore, the research finding presented in this paper is not a fully developed grounded theory about the practice, but rather a segment of a grounded description of practice. Grounded theory research methods ensure correctness of qualitative research does not matter if the result of the research is developed grounded theory [26][32].
The main research goal of the presented study is to explore the automation level of software change request process in very small software companies. This process is selected as an essential process during the maintenance phase of software life cycle. It is important to note that this is just a snippet of the whole study that explores more aspects of software change request process in local very small software companies. In the study were used methods and techniques of constructivist grounded theory approach proposed by Charmaz [26]. Qualitative data were collected through individual interviews or organized focus groups with software professionals from local very small software companies. Analyzes of collected data revealed that local small software companies do not have well established and automated process for collecting and handling software change requests that originate from their clients. The process is rather tailored to a specific client or current situation. Only four practitioners reported that they implemented partially automated process in their companies.
External validity is important quality attribute of empirical studies [63]. This validity corresponds to generalizability of qualitative studies proposed by Maxwell [62]. Without confirming the research findings of empirical studies with humans through additional external replications, they should be treated with caution [64]. The main characteristic of qualitative research is that it is concerned with the specific context and group of participants. Therefore, the findings of this study can be applicable to similar contexts and similar groups of people. The design of this study can be implemented in various research contexts where the research goal is to discover common practice by investigation of practitioners’ opinions. However, due to the specific business context in the region it is hard to draw any conclusions on how the results of this study can be generalized to other regions and countries without deeper understanding of their business, political and cultural circumstances.
This evidence about the state of the practice is additionally supported by the random usage of many approaches for collecting change requests from clients (by phone, by email, via web site, direct contact). Empirical evidence based on collected data in this study confirmed the state of the practice reported in literature about small software companies (although reported for other aspects of practice).
VI. ETHICAL ISSUES Despite identified constraints, this study also has some contributions. These are: Deeper understanding of current practice in local very small software companies,
Empirical research that involves human subjects should consider ethical issues related to researchers, participants, other stakeholders, and all affected institutions and research locations [65][66][67]. Participants were informed about
7
Identification of issues that should be investigated in more details by using both quantitative and qualitative research methods, Identification of problems in practice as potential staring points for further improvements, and Contribution to the body of empirical knowledge in software engineering.
[5]
[6]
Presented findings are outcome of the fact that small software companies have small teams that are task oriented, and have a little time to improve existing practice related to maintenance activities. Having that as a state of the practice, the following research directions are challenging: improvements of the current practice through automation of maintenance processes, improvements of communication with clients, and closer cooperation with researchers from universities. In practice, these further researches should improve communication procedures in companies and with clients, and enable development of tolls or services that automate software change request process and other maintenance processes.
[7]
[8]
[9]
[10]
Important research direction is related to additional empirical research of the software change request process in very small companies that will include response from their clients. Process investigation and assessment in small software companies by using quantitative methods or mixed methods will also provide insight into the practice from different, more technical point of view [69]. In that case, the picture of the practice will be more relevant, and triangulation will increase the validity of the research findings [70][71].
[11]
[12]
[13]
ACKNOWLEDGMENTS
[14]
The author would like to thank Bojan Ljutic director of Business incubator in Zrenjanin and Radu Ticiu executive director of Software business incubator in Timisoara for their help and support in organizing focus groups. Ministry of Education and Science, Republic of Serbia, supports this research under the project “The development of software tools for business process analysis and improvement”, project number TR32044, 2011-2014.
[15]
[16]
REFERENCES [1]
[2]
[3]
[4]
A. Finkelstein and J. Kramer, “Software engineering: a roadmap”, In Proceedings of the Conference on The Future of Software Engineering (ICSE '00), ACM, New York, NY, USA, 3-22, 2000, DOI: 10.1145/336512.336519. R. L. Glass, "Guest Editor's Introduction: The State of the Practice of Software Engineering," IEEE Software, Vol. 20, No. 6, pp. 20-21, 2003, DOI: 10.1109/MS.2003.1241361. J. Singer, T. Lethbridge, N. Vinson and N. Anquetil, “An examination of software engineering work practices”. In CASCON First Decade High Impact Papers (CASCON '10). ACM, New York, NY, USA, 174-188. DOI=10.1145/1925805.1925815. D. E. Perry, A. A. Porter, and L. G. Votta, “Empirical studies of software engineering: a roadmap”, In Proceedings of the Conference
[17]
[18]
[19]
[20]
8
on The Future of Software Engineering (ICSE '00), ACM, New York, NY, USA, 345-355, 2000, DOI: 10.1145/336512.336586. D. J. Reifer, "Is the Software Engineering State of the Practice Getting Closer to the State of the Art?," IEEE Software, Vol. 20, No. 6, pp. 78-83, 2003, DOI: 10.1109/MS.2003.1241370. A. Rainer, “The value of empirical evidence for practitioners and researchers”, In: Basili, V., Rombach, D., Schneider, K., Kitchenham, B., Pfahl, D., Selby, R. (Eds.), Empirical Software Engineering Issues. Critical Assessment and Future Directions, Vol. 4336 of Lecture Notes in Computer Science, Springer Berlin / Heidelberg, pp. 24–24, 2007, DOI: 10.1007/978-3-540-71301-2_8. D. I. K. Sjoberg, T. Dyba, and M. Jorgensen, “The Future of Empirical Methods in Software Engineering Research”, In 2007 Future of Software Engineering (FOSE '07), IEEE Computer Society, Washington, DC, USA, 358-378, 2007, DOI: 10.1109/FOSE.2007.30. I. Richardson and C. G. von Wangenheim, "Guest Editors' Introduction: Why are Small Software Organizations Different?," IEEE Software, Vol. 24, No. 1, 2007, pp. 18-22, DOI: 10.1109/MS.2007.12. J. Bell, D. Crick and S. Young, “Small Firm Internationalization and Business Strategy: An Exploratory Study of 'Knowledge-Intensive' and 'Traditional' Manufacturing Firms in the UK”, International Small Business Journal, Vol. 22, No. 1, February 2004, pp. 23-56, doi:10.1177/0266242604039479 M. E. Fayad, M. Laitinen and R. P. Ward. “Thinking objectively: software engineering in the small”, Communications of the ACM, Vol. 43, Issue 3, pp. 115-118, 2000, DOI=10.1145/330534.330555. G. Coleman and R. O'Connor, “Investigating software process in practice: A grounded theory perspective”, Journal of Systems and Software, Volume 81, Issue 5, Software Process and Product Measurement, pp. 772-784, May 2008, DOI: 10.1016/j.jss.2007.07.027. C. Y. Laporte, A. Renault, S. Alexandre, T. Uthayanaka, “The Application of ISO/IEC JTC 1/SC7 Software Engineering Standards in Very Small Enterprises”, ISO Focus, International Organisation for Standardisation, pp. 36-38, September 2006. N. Habra, S. Alexandre, J-M. Desharnais, C. Y. Laporte, A. Renault, “Initiating software process improvement in very small enterprises: Experience with a light assessment tool”, Information and Software Technology, Vol. 50, Issues 7-8, pp. 763-771, June 2008, DOI: 10.1016/j.infsof.2007.08.004. V. Ribaud, P. Saliou, R. V. O’Connor and C. Y. Laporte, “Software Engineering Support Activities for Very Small Entities”, Systems, Software and Services Process Improvement, Communications in Computer and Information Science, 2010, Vol. 99, pp. 165-176, Springer-Verlag Berlin Heidelberg, 2010, DOI: 10.1007/978-3-64215666-3_1. S. Basri and R. V. O’Connor, “Understanding the Perception of Very Small Software Companies towards the Adoption of Process Standards”, Systems, Software and Services Process Improvement, Communications in Computer and Information Science, CCIS Vol. 99, Springer-Verlag, 153-164, 2010, DOI: 10.1007/978-3-64215666-3_14. J. Segal, “The Nature of Evidence in Empirical Software Engineering”, In Proceedings of the Eleventh Annual International Workshop on Software Technology and Engineering Practice (STEP '03), IEEE Computer Society, Washington, DC, USA, 40-47, 2003, DOI: 10.1109/STEP.2003.33. T. C. Lethbridge, S. E. Sim and J. Singer, “Studying Software Engineers: Data Collection Techniques for Software Field Studies”, Empirical Software Engineering, Vol. 10, Issue 3, 311-341, 2005, DOI=10.1007/s10664-005-1290-x. C. Y. Laporte, “Process Improvement and the Management of Change”, Proceedings - 4th IEEE Computer Society Workshop on Software Engineering Technology Transfer, pp. 213-216, 1994. F. J. Pino, F. Ruiz, F. García and M. Piattini, “A software maintenance methodology for small organizations: Agile_MANTEMA”, Journal of Software Maintenance and Evolution: Research and Practice, doi: 10.1002/smr.541 A. April, A. Abran, “A Software Maintenance Maturity Model (S3M): Measurement Practices at Maturity Levels 3 and 4”,
[21]
[22]
[23]
[24]
[25] [26]
[27]
[28]
[29]
[30]
[31] [32]
[33] [34]
[35]
[36]
[37]
[38]
[39]
Electronic Notes in Theoretical Computer Science, Volume 233, Proceedings of the International Workshop on Software Quality and Maintainability (SQM 2008), 27 March 2009, pp. 73-87, DOI: 10.1016/j.entcs.2009.02.062. A. Abran, P. Bourque, R. Dupuis, J. W. Moore, L. L. Tripp, Guide to the Software Engineering Body of Knowledge - SWEBOK, 2004th Edition, IEEE Press, Piscataway, NJ, USA, 2004. C. B. Seaman, “Qualitative methods in empirical studies of software engineering”, IEEE Transactions on Software Engineering, Vol. 25, No. 4, pp.557-572, Jul/Aug 1999, DOI: 10.1109/32.799955. Y. Dittrich, M. John, J. Singer, B. Tessem, “For the Special issue on Qualitative Software Engineering Research”, Information and Software Technology, Vol. 49, Issue 6, Qualitative Software Engineering Research, Pages 531-539, June 2007, DOI: 10.1016/j.infsof.2007.02.009. D. Silverman, “Qualitative research: meanings or practices?”, Information Systems Journal, Vol. 8, Issue 1, Pages 3 - 20, 1998, DOI: 10.1046/j.1365-2575.1998.00002.x. L. M. Given, The SAGE Encyclopedia of Qualitative Research Methods, Sage Publications, Thousand Oaks, 2008. K. Charmaz, Constructing grounded theory: A practical guide through qualitative analysis, 1st edition, Sage Publications, London, UK, 2006. J. Mills, A. Bonner and K. Francis, “The development of constructivist grounded theory”, International Journal of Qualitative Methods, Vol. 5, No 1, pp. 25-35, 2006. J, M. Morse, “Qualitative Methods: The State of the Art”, Qualitative Health Research, Vol. 9, No. 3, 393-406, 1999, doi:10.1177/104973299129121938. N. K. Denzin and Y. S. Lincoln, “Introduction: The discipline and practice of qualitative research”. In N. K. Denzin and Y. S. Lincoln (Eds.), The Sage Handbook of Qualitative Research (3rd ed.), pp. 132, Sage Publications, Thousand Oaks, CA, USA, 2005. J. W. Creswell, Qualitative inquiry & research design: Choosing among five approaches, Second Edition, Sage Publications, Thousand Oaks, California, USA, 2007. B.B. Glaser and A.L. Strauss, The Discovery of Grounded Theory, Aldine Publishing Company, Chicago, USA, 1967. J. Corbin and A. Strauss, Basics of qualitative research: Techniques and procedures for developing grounded theory, 3rd edition, Sage Publications, Thousand Oaks, USA, 2008. A. L. Strauss, Qualitative Analysis for Social Scientists, Cambridge University Press, Cambridge, UK, 1987. A. Bryant, "Re-grounding Grounded Theory", The Journal of Information Technology Theory and Application (JITTA), Vol. 4, Issue 1, 25-42, 2002. J. Carver, “The Impact of Background and Experience on Software Inspections”, Empirical Software Engineering, Volume 9, Number 3, pp. 259-262, 2004, DOI: 10.1023/B:EMSE.0000027786.04555.97. R. Razali, C. Snook, M. Poppleton and P. Garratt, “Usability Assessment of a UML-Based Formal Modeling Method Using a Cognitive Dimensions Framework”, Human Technology: An Interdisciplinary Journal on Humans in ICT Environments, Volume 4, Number 1, 26-46, May 2008. G. Arcs, R. Razali, “Cognitive dimensions and grounded theory in learning software modeling”, Procedia - Social and Behavioral Sciences, Volume 1, Issue 1, World Conference on Educational Sciences, Nicosia, North Cyprus, 4-7 February 2009 - New Trends and Issues in Educational Sciences, pp. 1884-1888, 2009, DOI: 10.1016/j.sbspro.2009.01.331. G. Coleman, R. O'Connor, “Using grounded theory to understand software process improvement: A study of Irish software product companies”, Information and Software Technology, Volume 49, Issue 6, Qualitative Software Engineering Research, pp. 654-667, June 2007, DOI: 10.1016/j.infsof.2007.02.011. L. Scott, L. Carvalho, R. Jeffery, J. D'Ambra, U. Becker-Kornstaedt, “Understanding the use of an electronic process guide”, Information and Software Technology, Volume 44, Issue 10, pp. 601-616, 2002, DOI: 10.1016/S0950-5849(02)00080-0.
[40] L. Carvalho, L. Scott, R. Jeffery, “An exploratory study into the use of qualitative research methods in descriptive process modeling”, Information and Software Technology, Volume 47, Issue 2, pp. 113127, 2005, DOI: 10.1016/j.infsof.2004.06.005. [41] S. Adolph, W. Hall and P. Kruchten, “Using grounded theory to study the experience of software development”, Empirical Software Engineering, Volume 16, Number 4, 487-513, DOI: 10.1007/s10664-010-9152-6. [42] Z. Hussain, W. Slany and A. Holzinger, “Investigating Agile UserCentered Design in Practice: A Grounded Theory Perspective”, HCI and Usability for e-Inclusion, Lecture Notes in Computer Science, Volume 5889/2009, 279-289, 2009, DOI: 10.1007/978-3-64210308-7_19. [43] J. Sillito, K. De Voider, B. Fisher, G. Murphy, "Managing software change tasks: an exploratory study," isese, 2005 International Symposium on Empirical Software Engineering, 2005. [44] R. Hoda, J. Noble and S. Marshall, “Organizing self-organizing teams”. In Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 1 (ICSE '10), Vol. 1, pp. 285-294, 2010, DOI=10.1145/1806799.1806843. [45] R. Hoda, J. Noble and S. Marshall, “The impact of inadequate customer collaboration on self-organizing Agile teams”, Information and Software Technology, Volume 53, Issue 5, Special Section on Best Papers from XP2010, May 2011, pp. 521-534, DOI: 10.1016/j.infsof.2010.10.009. [46] K. Smolander and T. Paivarinta, “Describing and Communicating Software Architecture in Practice: Observations on Stakeholders and Rationale”, Advanced Information Systems Engineering, Lecture Notes in Computer Science, 2006, Volume 2348/2006, pp. 117-133, DOI: 10.1007/3-540-47961-9_11. [47] S. Dorairaj, J. Noble, and P. Malik, “Bridging cultural differences: a grounded theory perspective”, In Proceedings of the 4th India Software Engineering Conference (ISEC '11), 3-10, 2011, DOI=10.1145/1953355.1953357. [48] P. Xu and R. Balasubramaniam, “Software Process Tailoring: An Empirical Investigation”, Journal of Management Information Systems, Volume 24, Issue 2, pp. 293-328, 2007, DOI: 10.2753/MIS0742-1222240211. [49] S. Salinger, L. Plonka and L. Prechelt, “A Coding Scheme Development Methodology Using Grounded Theory for Qualitative Analysis of Pair Programming”, Human Technology: An Interdisciplinary Journal on Humans in ICT Environments, Volume 4, Number 1, 9-25, May 2008. [50] D. W. Stewart, P. N. Shamdasani and D. W. Rook, Focus Groups: Theory and Practice, Second Edition, SAGE Publications, London, UK, 2007. [51] R. A. Krueger and M. A. Casey, Focus Groups: A Practical Guide for Applied Research, Fourth Edition, SAGE Publications, London, UK, 2009. [52] O. T. Massey, “A proposed model for the analysis and interpretation of focus groups in evaluation research”, Evaluation and Program Planning, Vol. 34, Issue 1, pp. 21-28, February 2011, DOI: 10.1016/j.evalprogplan.2010.06.003. [53] M. Q. Patton, Qualitative research and evaluation methods, 3rd edition, Sage Publications, Thousand Oaks, USA, 2001. [54] P. Yuhas Byers and J. R. Wilcox, “Focus Groups: A Qualitative Opportunity for Researchers”, Journal of Business Communication, Vol. 28, No. 1, pp. 63-78, January 1991, doi:10.1177/002194369102800105. [55] J. A. Maxwell, “Using Numbers in Qualitative Research”, Qualitative Inquiry, Vol. 16, No 6, pp. 475-482, 2010, doi: 10.1177/1077800410364740. [56] M. Jorgensen, D. I. K. Sjoberg, “Impact of experience on maintenance skills”, Journal of Software Maintenance and Evolution: Research and Practice, Vol. 14, Issue 2, pp. 123-146, March/April 2002, DOI: 10.1002/smr.248. [57] D. Walker and F. Myrick, “Grounded Theory: An Exploration of Process and Procedure”, Qualitative Health Research, 2006, Vol. 16, No. 4, pp. 547-559, DOI: 10.1177/1049732305285972. [58] J. Corbin and A. L. Strauss, “Grounded Theory Research: Procedures, Canons, and Evaluative Criteria”, Qualitative Sociology, Vol. 13, No.1, pp. 3-21, 1990.
9
[59] Z. Stojanov, D. Dobrilovic and V. Jevtic, “Identifying properties of software change request process: Qualitative investigation in very small software companies”, Proceedings of the IEEE 9th International Symposium on Intelligent Systems and Informatics (SISY 2011), pp. 47-52, September 8-10, 2011, Subotica, Serbia. [60] Z. Stojanov, “Discovering automation level of software change request process from qualitative empirical data”, Proceedings of the 6th IEEE International Symposium on Applied Computational Intelligence and Informatics (SACI 2011), pp. 51-56, May 19-21, Timisoara, Romania, DOI: 10.1109/SACI.2011.5872972. [61] J. Cho and A. Trent, “Validity in qualitative research revisited”, Qualitative Research, Vol. 6, No. 3, pp. 319-340, August 2006, DOI:10.1177/1468794106065006. [62] J. A. Maxwell, “Understanding and Validity in Qualitative Research”, Harvard Educational Review, Vol. 62, No. 3, pp. 279300, 1992. [63] B. A. Kitchenham, S. L. Pfleeger, L. M. Pickard, P. W. Jones, D. C. Hoaglin, K. El Emam and J. Rosenberg, “Preliminary Guidelines for Empirical Research in Software Engineering”, IEEE Transactions on Software Engineering, Vol. 28, No. 8, pp. 721-734, 2002, DOI: 10.1109/TSE.2002.1027796. [64] A. Brooks, M. Roper, M. Wood, J. Daly, and J. Miller, “Replication's role in software engineering”, In Guide to Advanced Empirical Software Engineering, F. Shull, J. Singer and D. I. K. Sjøberg, Eds., Springer London, Section III, 365-379, 2008, DOI: 10.1007/978-1-84800-044-5_14.
[65] J. E. Sieber, “Protecting Research Subjects, Employees and Researchers: Implications for Software Engineering”, Empirical Software Engineering, Vol. 6, Issue 4, pp. 329-341, 2001, DOI: 10.1023/A:1011978700481. [66] C. G. Christians, “Ethics and politics in qualitative research”. In The Sage Handbook of Qualitative Research, N. K. Denzin and Y. S. Lincoln, Eds., Sage Publications, Chapter 6, 2005, pp. 139–164. [67] N. G. Vinson, J. Singer, “A Practical Guide to Ethical Research Involving Humans”, In Guide to Advanced Empirical Software Engineering, F. Shull, J. Singer and D. I. K. Sjøberg, Eds., Springer London, 2008, Section II, 229-256, DOI: 10.1007/978-1-84800-0445_9. [68] K. M. Guenther, “The politics of names: rethinking the methodological and ethical significance of naming people, organizations, and places”, Qualitative Research, Vol. 9, No 4, pp. 411-421, 2009, DOI:10.1177/1468794109337872. [69] A. Bryman, “Integrating quantitative and qualitative research: how is it done?”, Qualitative Research, Vol. 6, No. 1, pp. 97-113, February 2006, doi:10.1177/1468794106058877. [70] L. Bratthall and M. Jørgensen, “Can you Trust a Single Data Source Exploratory Software Engineering Case Study?”, Empirical Software Engineering, Vol. 7, No 1, pp. 9-26, 2002, DOI: 10.1023/A:1014866909191. [71] K. T. Konecki, “Triangulation And Dealing With The Realness of Qualitative Research”, Qualitative Sociology Review, Vol. IV, Issue 3, pp. 7-28, 2008.
10
