Supporting Experimentation Processes Configuration in Experimental Software Engineering Environments Paulo Sérgio M. dos Santos, Artur Barbalho de O. Souza, Vitor Pires Lopes, Guilherme Horta Travassos COPPE / UFRJ – Systems Engineering and Computer Science Department Caixa Postal 68.511, CEP 21945-970, Rio de Janeiro – Brazil
[email protected], {pasemes, vitorlopes, ght}@cos.ufrj.br
Abstract. Currently most results about new software technologies (processes, methods, techniques and tools) are based on personal opinion or marketing. Experimentation can represent a systematic, disciplined and controlled approach to evaluate software processes and human technologies. Usually, executing experimental studies consumes time and produces a huge amount of information and knowledge. This scenario motivated the building of an experimental Software Engineering Environment: a computerized infrastructure based on e-services to support large-scale experimentation in Software Engineering. This paper describes a set of tools to support the activities regarding the configuration of experimentation processes and allocation of e-services in the environment. Keywords. Experimental Software Engineering experimentation process, e-services, e-science.
Environments,
1. Introduction Software Engineering aims at the use of software technologies (processes, methods, techniques and tools) to improve product quality and team productivity. Although it represents an important and strategic knowledge area, it’s still common the proposing of new software technologies based on opinion or commercial interest. Scientific research, however, should not be directed or based on opinion neither speculation. Scientific investigations are represented by studies based on observations of/or experimentation with the real world and its measurable behaviors. This should not be different when considering the construction and evaluation of software [Juristo and Moreno, 2001]. Experimental studies have been used as a mechanism to acquire knowledge through a scientific methodology based on measurement of phenomena in different Software Engineering areas. Its benefits are related to the attainment of results that justify the use or not of a technology based on some indication that such technology can contribute for quality improvement. Thus, the results of experimental studies executed in different development processes scenarios can be used as starting points to define a criteria set to support the decision making process regarding software technologies usage. Looking for this scientific support in Software Engineering it can be observed an increasing interest in the experimentation field [Tichy, 1998; Zelkowitz and Wallace, 1998]. However, experimental research in Software Engineering must take into account a large number of problems, context variables and characteristics, being a complex activity as shown in [Basili et al., 1999]. In this manner, many researchers are investing efforts in technology definition to support application of experimentation in software
45
engineering [Arisholm et al., 2002, Jedlisthka and Pfhal, 2005]. Thus, providing support to experimental studies planning and execution becomes a research target in the field. Into this context, the Experimental Software Engineering group at COPPE/UFRJ (http://www.cos.ufrj.br/~ese) has been working on the development of a computerized infrastructure to support large-scale experimentation in Software Engineering, called eSEE (experimental Software Engineering Environment) [Santos and Travassos, 2007]. The eSEE infrastructure is basically organized in three levels: meta-eSEE, configuredeSEE and instantiated-eSEE. In this paper, we will describe our experience on building the set of facilities to support the activities regarding the configured-eSEE level, mainly those concerned with the experimentation process configuration and allocation of eservices. This paper is composed by four sections. The first one comprises this introduction. Section 2 discusses the experimental Software Engineering Environment, describing the basic concepts of its infrastructure. Next, Section 3 presents the tools regarding Instantiation Environment: Process Configuration and Association Maps Definition tools. Finally, Section 4 summarizes and concludes the paper.
2. The experimental Software Engineering Environment In order to support experimentation in Software Engineering, an engineering environment must provide a set of facilities, allowing the tasks execution by individuals playing different roles in the Experimentation Process. Chapetta et al. (2004) performed the requirements elicitation to define the functionalities for SE experiments monitoring, executing and packaging activities based on Wohlin et al. (2000), Amaral (2003) and Costa et al. (2004) experimental processes, what includes: (1) Having integrated experimentation support tools, performing similarly as a Software Development Environment (SDE); (2) Being a Web System, allowing its use on different localities and interinstitutional researches; (3) Using of eservices based paradigm, and; (4) Providing Knowledge Management mechanisms, since Experimental Software Engineering is a knowledgeintensive area.
Figure 1 - Basic Concepts of eSEE’s infrastructure
Based on these requirements, Dias Neto et al. (2004) defined the architecture for eSEE composed by three distributed macro-components. This architecture aims at to describe a solution for experimental Software Engineering processes modeling, instantiation and executing. These distributed components are represented by: Meta-Configurator (MC), Instantiation Environment (IE) and Execution Environment (EE) (Figure 1). The Meta-Configurator, as illustrated in Figure 1, deals with meta-descriptions activities: definition of process and document models and service configuration [Kim et al., 2002]. The process and document models are defined following meta-models, which aggregate knowledge from various experimental studies. Each element is stored in a
46
repository for further linkage among them. The Process Modeling Component has been improved by Souza (2007) in order to facilitate the reuse of other process models and knowledge regarding previous experiments. A software engineer with experience formalizes knowledge through the definition of questions, which involve decisions of including or excluding tools, roles and activities regarding the experimental study. The Instantiation Environment macro-component provides support for the definition and instantiation of an Execution Environment, based on the process, documents and services defined previously by the MC macro-component. According to improvements done to the MC [Souza, 2007], a process configuration component was created. It is responsible for analyzing the configurable process model, defined by MC, and configuring the model for a specific experimental study. The Associations Mapping Component aims at to establish the relationships among the experimentation process’s activities produced/consumed documents and configured services to accomplish a task. This set of relationships is called an Association Map. It is stored in a specific repository, from which the instantiation component is able to use when instantiating an EE. The Execution Environment is responsible for the enactment of the experimentation process. It allows the monitoring, controlling and execution of the activities, based on the restrictions defined in the chosen Experimentation Process. Further information regarding eSEE’s IE and EE can be obtained in Dias Neto et al. (2004) and Mian et al. (2005). The current eSEE´s facilities allow the instantiation of experimental environments to support in vitro studies for geographically distributed research teams. Some additional research efforts have been invested to make eSEE to deal with in vivo, in virtuo and in silico experimental studies [Travassos and Barros, 2003]. Moreover, besides Software Engineering, researchers in other areas can try eSEE to support experimentation. To evaluate the feasibility of eSEE’s infrastructure development, a prototype has been built [Dias Neto et al. 2004]. In this first prototype, simple documents and processes models were produced without tools to support it, and the service configuration activities were not considered due to the lack of e-services to use in this context. Later, Santos (2007) improved the first prototype towards the implementation of the Associations Mapping Component. Besides, Souza (2007) implemented the Process Configuration Component, allowing the configuration of the process model, based on answers given by the experimental software engineer.
3. eSEE’s Instantiation Environment Tools The Instantiation Environment contains knowledge regarding specific experiments types. This level is responsible for the eSEE’s instantiation accomplishment by choosing the experimental study type and configuring it according to specific characteristics. The instantiated environment, then, is generated with the specific knowledge to that type of study [Mian et al., 2005]. Currently, the Instantiation Component can not be considered completely implemented because it needs to evolve, following the on going definitions concerned with the Execution Environment.
3.1. Process Configuration Tool Based on Vasconcellos and Werner (1998), the improvements made in the Process Modeling Component [Souza, 2007] provided a mechanism to create a configurable process model. A process model is composed by activities that are executed by people who play specific roles and can be supported by tools, such as a text editor. A
47
configurable process model, in turn, is characterized as a process model that, besides activities, people, roles and tools, defines questions related to specific studies. These questions are created by experienced researches in Experimental Software Engineering during the metaconfiguration activities. Each question can have two or more answers with correspondingly actions that modify the initial process model. Thus, knowledge formalized into these questions can be available for others researchers, even novices. Figure 2 – Question answering for process model configuration This tool implements the Process Configuration Component. Using a configurable process model defined in the MC, it is responsible for providing an answering filling form for the model’s questions (Figure 2) and, based on the answers provided by the researcher, to configure a process model for the specific experimental study [Souza, 2007].
The configuration of process models is based on actions associated to each possible answer for a specific question. An action can insert or remove activities, tools and roles. So, the configuration of a process model is the result of the selected actions applied to the configurable process model. After the configuration of the process model, this tool publishes a new process model which is not configurable. Non-configurable process models are the only ones accessed by the Association Mapping Component.
3.2. Association Maps Definition Tool The Association Mapping Component provides a way to instantiate an EE, based on the configured process and document models previously defined. It also allows the web services allocation, which will work as external experimentation tools supporting the tasks accomplishment when the correspondingly EE enacts the experimentation process. Figure 3 shows the tool interface for defining associations between these elements. Note that an activity can have sub-activities and a document can have sections which can have sub-sections. An association is composed by a process activity, a document or a document section and a web service. Each association contains all information related to when (which activity) a document or section will be filled and how (which web service) it will be filled. The set of associations is called an association map. The main difference between an association map and process model is that an association map binds an activity to the computerized representation of the document produced or consumed by the activity, whereas a process model contains information describing the produced document. This separation is important for the EE because it allows the using of appropriate tools for each type of document.
48
Figure 3 – Association map creation
4. Conclusion This paper described how eSEE supports the configuration of experimentation processes. The environment is currently under construction. IE and EE are partially constructed, although MC has already been built [Chapetta et al., 2005]. New types of experimental studies are being described in order to be supported by the infrastructure. These include in virtuo and in silico primary studies [Araujo and Travassos, 2005], as well as secondary studies, most noticeably systematic reviews [Biolchini et al., 2005]. In order to continue constructing eSEE, research has been conducted mainly in three areas: 1) Definition and construction of a first version of the Service Manager Component; 2) Construction of scientific research ontology in Software Engineering [Biolchini et al., 2007] to facilitate the creation, access and reuse of knowledge and integration of tools into the environment. 3) Definition of the requirements to support the instantiation of in vivo, in virtuo and in silico experimental studies environments. Specific activities must be considered for each type of these experimental studies. For instance, in silico studies demand activities to create computerized models of participants, environment and the object under study, while in vivo ones need a specific planning involving how to conduct the experiment under real circumstances of software development process. Such activities will make processes different from each other. eSEE’s architecture is designed to allow the configuration and instantiation of different experimental processes. Thus, the current solution may be able to hold the changes to support the instantiation of in vivo, in virtuo and in silico study environments.
Acknowledgments The eSEE project is part of CNPq project – Infrastructure for Experimentation in Software Engineering (302154/2004-3) and FAPERJ. eSEE is the result of ESE Group´s work, which the authors would like to thank.
References Amaral, E. A. G. G. (2003) “Empacotamento de Experimentos em Engenharia de Software”, Master Dissertation, COPPE/UFRJ, Engenharia de Sistemas e Computação, RJ, Brazil.
49
Araujo, M. A. P., Travassos, G. H. (2005) “Aplicação das Leis de Evolução de Software em Manutenção Evolutiva”. Revista Tecnologia da Informação, Brasília, v. 5, n. 2. Arisholm, E., Sjøberg, D.I., Carelius, G.J., Lindsjørn, Y. (2002). "A Webbased Support Environment for Software Engineering Experiments", In: Nordic Journal of Computing, v. 9, n. 3 (September), pp. 231-247. Basili, V.R., Shull, F., Lanubile, F. (1999). “Building Knowledge through Families of Experiments”, In: IEEE Trans. on Software Engineering, vol. 25, No. 4. Biolchini, J. ; Mian, P. ; Conte, T. U. ; Natali, A. C. C. ; Travassos, G. H. (2007) “Scientific research ontology to support systematic review in Software Engineering”, Advanced Engineering Informatics, v. 21, p. 133-151. Biolchini, J., Mian, P.G., Natali, A.C.C., Travassos, G.H. (2005). “Systematic Review in Software Engineering”. Technical Report RT – ES 679/05. COPPE/UFRJ. Chapetta, W. A., Santos, P.S.M. And Travassos, G. H. (2005) “Supporting Meta- Description Activities in Experimental Software Engineering Environments”, 2nd Experimental Software Engineering Latin American Workshop, Brazil. Chapetta, W. A.; Conte, T. U.; Travassos, G. H. (2004). “Requisitos para um Ambiente para Experimentação em Engenharia de Software”, Technichal Report, ESE, Engenharia de Sistemas e Computação, COPPE/UFRJ, RJ, Brazil. Costa, H.R.; Mian, P.G. Travassos, G.H. (2004) “Estendendo um Modelo de Processo e de Empacotamento de Estudos Experimentais”. Relatório Técnico do eSEE. COPPE/UFRJ. Dias Neto, A. C., Barcelos, R., Chapetta W. A., Santos, P. S. M., Mafra, S. N., Travassos, G. H. (2004). “Infrastructure for SE Experiments Definition and Planning”, ESELAW'04, Brasília, Brazil. Jedlitschka, A., Pfahl, D. (2005). “Reporting Guidelines for Controlled Experiments in Software Engineering”, In: Proceedings the 4th IEEE/ACM International Symposium on Empirical Software Engineering. Australia, November, 2005. Juristo, N., Moreno, A. (2001). “Basics of Software Engineering Experimentation”, Kluwer Academic Press, 1st edition. Kim, W., Graupner, S., Sahai, A., Lenkov, D., Chudasama, C., Whedbee, S., Luo, Y., Desai, B., Mullings, H., Wong., P. (2002). "Web E-speak: facilitating Web-based e-Services", Multimedia,IEEE, v. 9, n. 1 (Jan-Mar), pp. 43-55. Mian, P. G., Travassos, G. H., Rocha, A. R. C. (2005). “A Computerized Infrastructure for Supporting Experimentation in Software Engineering”, ESELAW’05, Uberlândia, Brazil. Santos, P.S.M. (2007). “Ferramenta para Alocação de Serviços WEB em Ambientes de Engenharia de Software Experimental”, B.Sc. Final Project Dissertation, DCC/UFRJ, Rio de Janeiro, RJ, Brazil. Santos, P.S.M. and Travassos, G.H. (2007). “eSEE – Ambiente de Apoio a Experimentação em Larga Escala em Engenharia de Software”, Proceedings of the 1st Brazilian e-Science Workshop, João Pessoa, PB, Brazil. October, 2007. Souza, A.B.O. (2007). “Configurando Modelos de Processos de Experimentação em Engenharia de Software”, XXIX Jornada Giulio Massarani de Iniciação Científica, Artística e Cultural (JIC 2007), DCC/UFRJ, October, Rio de Janeiro, RJ, Brazil. Tichy, W.F. (1998) “Should Computer Scientists Experiment More?” In: IEEE Computer, 31 (5), p. 32-39. Travassos, G.H., Barros, M.O. (2003). “Contributions of In Virtuo and In Silico Experiments for the Future of Empirical Studies in Software Engineering”, Proc. of the WSESE03, Fraunhofer IRB Verlag, Roma. Vasconcelos Jr., F. M., Werner, C. M. L. (1998). “Organizing the Software Development Process Knowledge: an Approach Based on Patterns”, Int. Journal of Software Engineering and Knowledge Engineering, Vol. 8 No. 4, p. 461-482. Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., Wesslén, A. (2000). “Experimentation in Software Engineering: an Introduction”, Kluwer Academic Publishers, Massachusetts. Zelkowitz, M.V, Wallace, D.R. (1998). “Experimental Models for Validating Technology”, In: IEEE Computer, 31 (5), p. 23-31.
50
