A web based simulation game as learning tool for the design process of complex systems Francesco BIANCONI, Stefano SAETTA, Lorenzo TIACCI* Dipartimento Ingegneria Industriale, Università degli Studi di Perugia Via G. Duranti 1/A-4, 06125 Perugia (ITALY) *corresponding author:
[email protected] Abstract While relational simulation games have been widely used for learning activities in areas such as business, management and communication, their development in the engineering field is still very limited. In the paper a web based simulation game was designed in order to fill this existing gap in the development of relational simulation games related to engineering aspects, and in particular to the design process of complex systems. Nowadays the design process becomes more and more complex for both technical and relational aspects and the possibility to have a learning tool such as the simulation game here presented is very appealing. The web based simulation game here presented is a teaching tool aimed at creating a simulation environment where players participate in a competitive design process of a complex system. In this manner it is possible to provide a playground where students can apply techniques for the design and management of complex systems (such as Design of Experiments, Genetics Algorithms and others) and learn how to face personal, interpersonal and group relationships. The specific application, in particular, has been implemented to simulate a competitive tender for the design of a military aircraft. The implementation of the web-based application relies on the classical three-tier architecture: presentation layer, business layer and persistence layer. The system has been developed on a pure java platform using the JavaServer Faces© technology for the presentation layer and Hibernate/HSQL© for the persistence layer. An evaluation of a first web trial competition, carried out by questionnaires, shows a satisfactory achievement of the proposed learning tool objectives. Keywords: learning tools, engineering-design education, complex systems, web-based games, simulation.
1. Introduction Web Based Simulation Games (WBSG) can effectively integrate engineering teaching tools providing the possibility to stimulate the learning process in a competitive web environment. The most adopted teaching tool in academia is frontal lesson, both in engineering courses and in other scientific or technical curricula. In general this approach enables the teacher to select and to expose those topics that he considers of primary importance for the specific subject he teaches. In addition to traditional frontal lessons there are other teaching activities that would invite students to autonomously reorganize and apply the concepts they are taught to case studies, in order to stimulate their relational and practical capabilities. Such teaching actions, which we could conveniently call “active”, usually fall into three categories (Vaccani, 1997). The first category is room research, sometimes referred to as the inductive method, where the teacher partially introduces the topic of a lesson and then stimulates a discussion among the various students, who are invited to propose possible solutions to a given problem. The second category is case studies, where real or very similar scenarios are proposed to the students, with the main objective of developing their capabilities in elaborating management strategies for situations that are likely to occur in real circumstances. The third category, to which WBSG belongs, is simulation, that can probably be considered the most “active” one, since students are supposed to play a primary role in a complex situation which usually requires decision making and management of complicated relationships. Simulation, i.e. a situation which implies physical and mental involvement, can stimulate emotional intelligence, recognized as an essential component to the success of human
learning processes (Salovey and Mayer, 1990; Goleman, 1995). At the same time, however, the fact of being an artificial scenario protects one from possible drawbacks such as fear of failure and anxiety which would otherwise exist in reality. This is undoubtedly a considerable advantage, since it is well known that fear of failure (McCelland, 1985) and anxiety (Hembree, 1998; Spielberger, 1980) can significantly hinder the learning process. The “protected” environment of simulation allows learning through errors without the considerable drawbacks that would arise in real situations. In order to motivate the learning process, simulation tools are often implemented as simulation games. With simulation games, the competitive environment opposes the natural trend, which can be seen in each learning activity, where the emotional aspects of learning lessen as the student’s knowledge increases. Internet has brought a significant contribution in the evolution of simulation games, since it is possible to play located in different places and at different times. Such consideration suggested that a web-based architecture for the development of the WBSG here presented be adopted. One of the most interesting and promising fields of application of WBSG is the design process of complex systems. Complex systems in engineering problems consist of systems with such a large number of variables interacting with each other that it is not easy to predict and control their behaviour: airplanes, ships, industrial and power plants and cars are just some examples. In order to simulate their design process, we must include the aspects typical of technological simulations and relational simulations. Technological simulations are usually focused on the acquisition of technical skills. Simulations which show the behaviour of mechanical or electric systems, for instance, pertain to this group. Simulations that train people in the use of production systems, in the conduction of industrial plants, in emergency in nuclear or chemical plants or in the management of information systems also pertain to this category. Considering design processes, however, we also need to take into account those aspects not directly related to technical problems, but, perhaps, more connected to individual aspects such as personal opinions and human relationships. Relational simulations are usually focused on the development of relational skills such as: personal relationships (such as the decision making process); interpersonal relationships (such as employer-employee, customer-provider, customerseller, client-consultant and so on); group relationships (such as team work, team design, coordination activities, meeting and similar) In order to clarify how both technological and relational aspects play an important role in the design process of complex systems, we can describe the situation to which the WBSG proposed refers. A customer who needs a particular product typically asks different companies for the development of a prototype and then decides, based on quality and proposed prices, which company to entrust. Let’s suppose, then, that a team is requested to design a complex system with specific requirements and limited resources available in terms of time, budget etc. Since the design team is enforced to respect such constraints, it will probably adopt the design techniques that will quickly and effectively enable the development of prototypes which are compliant with the required specifications. In order to do this, it is likely that the team will use product modeling and simulation to handle the set of parameters that are considered to be significant. Thus this part of the team work pertains to technological simulation. On the other side the team will also be required to take important decisions on which strategy to choose in the design process. Such decisions can usually be guided by not just purely technical motivations. The simulation of such decision making process is an example of relational simulation. In this paper we propose a WBSG with the following overall aims: 1) to provide a simulation environment where the participants are supposed to take part in a competitive design process of a complex system (a military aircraft, as detailed later);
2) to provide a playground where the players can apply techniques for the management of complex systems (such as, for instance, Design of Experiments and others); 3) to provide a playground in which personal, interpersonal and group relationships play a substantial role. In the remainder of the paper the related research shows that, despite the fact that the relational approach to simulation has been investigated far and wide, the development of gaming and simulation systems more related to engineering aspects - such as design of complex systems -, to the best of our knowledge, has not been studied so far. Then we describe how the WBSG proposed tries to fill this gap, which is by integrating some of the concepts typical of relational simulations with specific engineering issues related to the design process. In this paragraph game setup and main rules are also explained in detail. The particular design process considered consists in the design of a military aircraft. Implementation issues are then discussed, by giving motivations on how choices about architecture and frameworks have been done. The system is implemented in a pure Java language and web architecture follows the three-tier approach, with presentation, business and persistent layers. Finally, preliminary results on the use of the game in an academic domain are presented and critically discussed. 2. Related research Background research related to the work presented in this paper involves both pedagogical and philosophical aspects associated with the use of gaming and simulation systems in teaching, and technical issues connected to the development of web-based gaming and simulation environments. The topic of web-based simulations for computer-assisted learning has been addressed by various authors during the last years. In an editorial article Bruzzone, in 2001, foresaw that Web-based technologies would totally alter the way we conceive, develop, manipulate and apply simulation as a problem-solving technique and decision support system (Bruzzone, 2001). Several systems have been proposed to enhance traditional teaching through simulation environment in specific topics. As stated in the introduction, there are two groups of simulations: the first one follows a technical approach that simulates, through a computer program (internetbased or not), the behavior of a certain system (for instance an electric circuit, a living cell, and so on), while the second one follows the relational approach that simulates the interaction between various users in a complex environment. In the first case the user is typically supposed to feed the simulation program with some input values, to gather the output and to see how the simulated system evolves. In the second case the user normally plays an active role in the simulation, because the user is a part of the simulation environment. The Web-Based CAL system (Masson, 1999), for instance, falls in the first group. It is a framework designed to simulate the behavior of electric and electronic circuits. The interface of the system offers some types of circuits where the user can alter the input values in the form boxes (i.e. resistance values and power supply voltage) and see the results in terms of circuit current. The Web-Based CAL also allows the study of digital circuits, enabling the user to select a combination from an input truth table and see the resulting logic circuit that would give the required output. The system was implemented following the HTML/CGI architecture, using the C language to implement server side scripts. Another example of this group is represented by the iCell modeling tool (Demir, 2003), developed to integrate research and education for electrophysiology training. The software, implemented as a client-side program through Java applet, provides various models of cardiac cells and neurons, and offers simulation data of their bioelectric activity. Approaches have also been proposed which allow the user to design their own simulation environment. The CSLab (Chatterjee, Paramasivam and Yakowenko, 1997) is, for instance, a webbased simulation environment which allows users to set up their own experiments. This system is
based on modules, which appear to the user as labeled boxes with input and output buttons performing specific computations. Each module consumes some inputs and produces some outputs. Virtual experiments (simulations) can be configured by connecting modules so that the outputs of one are the inputs of another. The CSLab system is implemented in Java. It is not detailed if it runs as a client-side application or as a web-accessible server side module. Role play simulation and business simulation pertain to the second group. In the first case the user is asked to imagine he is someone else, this someone else being a character in a simulation that a group of people play as part of a university course or in a different context (Linser, 2004). The user then confronts himself as a character in a scenario trying to answer questions such as: what am I to do ? What are my obligations ? What strategies would best suit an outcome that is best for staff, parents and children ? An example of such a framework is represented by the Fablusi system (Ip, Linser and Naidu, 2001). Business games are simulations of the lives of several companies competing in a single market. In the classical approach (Esposito, Michelot and Thomas, 2002) all the companies start with the same financial structure and produce the same product. Business simulation, for instance, has been investigated for years as a useful system to improve management training (Cadotte, 1995). It is commonly accepted that simulations can go farther than traditional methods in bridging the gap between the classroom and the world of real-life business and decision making. It is commonly accepted that the use of business games in teaching fulfill several aims, such as giving students the opportunity to learn how companies work and giving them a live opportunity to experience group work. Various business games have been proposed since 1955, when the Rand Corporation introduced Monopologs, a simulation the main objective of which was to train personnel of the U.S. Air Force in Logistics. Since then several business games have been proposed and used as training tools in famous companies, such as the Top Management Decision Simulation developed by the American Management Association, the Business Management Game by McKinsey, the Carnegie Tech Management Game developed at the Carnegie Mellon University. At the time of writing an important business game, the Global Management Challenge, sponsored by various companies, is ongoing. The technology adopted to develop web-based modeling and simulation environments has also received continuous improvement since the late nineties. Several research efforts have been focused on exploiting the web infrastructure to build models and execute simulation remotely on the web. According to Narayanan (2000), the research activity has been focused on these four main aspects: 1) the use of the worldwide web to retrieve documentation on existing simulations; 2) the development of client-side modeling systems in order to enable designers to run simulations on their desktop; 3) the development of server-side simulation systems where designers can access the simulation program remotely through a web browser and execute it on a server platform, and 4) the development of distributed-design systems where multiple users can interconnect with the same simulation from different locations and at different levels of abstraction. Architectures for web-based and web-accessible simulation environments have been proposed and discussed by various authors. An interesting web-based simulation tool (Websim) has been described by Leong, Ali, Parkash and Nordin (2000). Here the main objective is to develop a system that allows the creation of a web-based simulation on the fly. The basic idea is that the system administrator uploads the simulation engine through a (.exe) file which follows a certain format required by the system. Once this is done the simulation library is deployed into the server. A user who connects to the server may select a pre-installed simulation project and input the required parameters. After the simulation is done, the results are plotted through an independent plotting program which takes as input the simulation results saved in an ASCII file and produces a (.gif) image as output. Websim was implemented using CGI Perl scripts. A reliable architecture for web-based simulation environments has also been introduced by Kumara et al. (2002). This approach is based on a three-tier client/server framework which consists of three levels: database, execution and user interface. In the author’s opinion there are three
primary reasons for creating the three-tiered architecture: 1) applications can be partitioned in a way that best fits the organizational computing needs; 2) data analysis resides on a powerful application server; 3) it is possible to partition applications in different ways. Other authors (Ramirez, 2002) have also stressed significant additional advantages related to the use of a three-tier architecture, such as capability of modifying or replacing one tier without affecting the others, separation between application and database functionality and possibility of enforcing adequate security policies at server side. We share the same opinion of the authors: we believe that the three-tier approach is, at present, a very reliable system for the implementation of a web-based distributed simulation environment. For this reason we adopted this approach to implement the system discussed in this paper. 3. Description of the Web-Based Simulation Game The design process considered in the WBSG here proposed is about a military aircraft, that represents a complex system with a very large number of variables interacting with each other. As stated in the introduction, both technological and relational aspects of simulation have to be considered in order to include all the facets of the design process. Regarding the technological aspects, the system has been implemented with the goal of simulating a design process based on the development of a set of military aircraft prototypes. The game is divided into phases: in each phase each team can do a set of virtual prototypes, by assigning a set of values to the input parameters. Figure 1 briefly summarizes the architecture of the simulation environment, from the technological point of view. The model of the aircraft is made up of input and output engineering parameters such as thrust, speed, length, weight, etc. The set of the variables which defines an aircraft cannot be changed; the designer can only change the values of the input variables. The result of each test is a virtual prototype of an aircraft, with its associated performance. The requirements establish a set of minimal values for some output parameters that the designed aircraft should satisfy. If an aircraft does not comply with the requirements, its evaluation is negatively affected. Each test has its own cost, so the number of tests a team can do is related to the allotted initial budget. As detailed below, the cost of each test varies according to some parameters such as, for instance, the phase of the game. This part of the game represents a means to test and develop the competence of the students in evaluating experimental results and in proposing modified models starting from the previously acquired tests. In this manner the students can be encouraged to adopt the design techniques that can be applied to the management of complex models that cannot be represented through an explicit mathematical model, such as, for instance: Design of Experiments, Neural Networks, Genetic Algorithms and so forth. .
Figure 1 – Description of the technological aspects of the WBSG
Regarding the relational part of the game the design of the military aircraft is usually carried out by a design team. The complexity of the system suggests that it can be divided into various submodules, which, in the case of the aircraft, can be engine, avionics and cell. The division into subsystems should encourage a certain degree of parallelization in the work of each team. In such an environment it is possible to simulate personal and interpersonal relationships such as the choice or the election of the project leader of each sub-module. The coupling between the performance of the various sub-modules, and their influence on the global performance of the aircraft, are elements that enable the development of interpersonal and team relationships among the components of a team, as shown in fig.2.
Figure 2 - Description of the relational aspects of the WBSG
The game has been thought as a competitive tender for the design of an aircraft, where various teams are in competition. The teams which participate in the game are required to design a new military aircraft model requested by a client company. The overall aim is to design the best aircraft in order to win the tender for the supply of the airplane. The competitive environment is shown in figure 3.
Figure 3 – WBSG competitive environment
.
In the remainder of the paragraph the aircraft model is shown, tests are described, and the budget of teams and tests costs are discussed. Finally the requirements the prototypes must satisfy are detailed and the evaluation criteria for team ranking in competition are explained. 3.1 Aircraft model The simulated aircraft model is composed of a set of input and output variables. The input variables are divided into three groups: cell, engine and avionics. The output variables, which represent the performance of the aircraft, are also divided into the following groups: cell, engine, avionics and overalls. The output variables are computed through empirical relations available in literature. Such relations may use the input value without alterations as in the following example: ETC 2 2.7
IC e IC
(1)
where ETC represents the takeoff coefficient and IC the inversion coefficient of the engine. In other cases the input value is normalized between the maximum and the minimum allowed values The simulated design process, which represents the central point of the simulation environment discussed here, assumes that a model of the aircraft, with a certain degree of approximation to reality, is available to the designers. All the participants, in fact, are provided with a model of the aircraft implemented in a spreadsheet. The model, which is the same for all the participants, is a set of mathematical formulas which link the input and output variables of the aircraft. Of the formulas that are given to the participants, some are the same as those implemented in the simulation engine of the web-based system, others have some differences. This tries to simulate that a mathematical model is always an approximation of the real system, and sometimes it is necessary to correct it through the evaluation of experimental results. Such approach should encourage the participants to confront the results of the theoretical model, provided through the spreadsheet, with the results of experiments conducted on the real model, which come out from the web-based system. This simulated design process, which involves both a theoretical and a real model, may stimulate, in the participants, the following actions: the use of the theoretical model to conduct factorial experiments (DOE) in order to determine which input parameters have significant influence on a certain set of output values; to check the validity of the theoretical model through a set of experimental tests and, if necessary, to correct it; the use of both theoretical and experimental results through techniques that are commonly adopted for the management of complex systems (artificial neural networks, fuzzy logic, optimization techniques and others). 3.2 Tests Each team is given the possibility to carry out, through the web-based system, a set of tests. Such tests simulate real experiments conducted on aircraft prototypes. In each test the performance of an aircraft is evaluated as a response of the inputs issued by the team designers. Each test is recorded in the database and can be retrieved at any time. A sample of a test result is reported in figure 4. During the game each team has to choose which prototype to present for the comparative evaluation. At each phase a team can display the general situation of the previous phase, as depicted in figures 5 and 6. The selected airplane can be changed throughout the game.
Figure 4 - Screenshot of an experimental test
In each test the performance of the aircraft is computed through a set of formulas which, as stated before, do not coincide, completely, with the formulas that the participants are given through the spreadsheet. In addition, since each test represents an experimental assessment, noise is added to the computed outputs in order to simulate non-repeatability. Regarding uncertainty, the outputs have been divided into two groups, according to the degree of confidence that can be associated to the specific parameter: low confidence parameters and high confidence parameters. The set of parameters that are difficult to measure, such as efficiency in air battles and in ground attack mission, as well as aircraft maneuverability, pertains to the first group. The second group includes parameters the measurement of which is easier, such as weight and consumption. A uniform noise of respectively ±7% and 3% is added to the two groups.
Figure 5 - General situation text report (absolute values)
Figure 6 - General situation (graph report with normalized values)
3.3 Budget and costs Each test has its own cost. At the beginning the budget of each team is 200 million dollars. The game is divided into three subsequent phases, and the cost of each test increases as the phase goes on. This tries to simulate a real design scenario where late changes in design are more likely to have higher costs. The cost of each new test is composed of a fixed amount (5 M $, 10 M $ and 15 M $ respectively for the first, second and third phase) plus a variable amount which depends on how the independent variables vary with respect to previously done tests. The variable cost is proportional to a coefficient s computed as in equation (2):
k Pn ,i Pa ,i s min aA k i 1
(2)
where A is the set of the previously done tests, Pn,i the value of the ith input parameter of the new airplane, Pa,i the value of the ith input parameter of the ath prototype existing in the database. The input values are normalized between their maximum and minimum. The variable cost tries to simulate the situation of a real scenario where the development of a prototype, which differs a lot from existing ones, is more expensive. 3.4 Requirements The aircraft which the participating teams select as the one to be evaluated as the winner of the tender is required to satisfy a predefined specification. Such requirements are given both as absolute values (requirements of type A - equation 3) and as relative values with reference to a given aircraft (requirements of type B - 4).
max_ speed action _ range max_ weapins _ weight thrust _ weight _ ratio cos t plane _ weight _ ready t arg ets AA _ eff AG _ eff
a b c d
(3)
f
g h
1.1 AA _ eff _ ref
1.1 AG _ eff _ ref
(4)
The relative requirements (equation 4) state that the efficiency of the aircraft in air battles (AA_eff) and in ground attack missions (AG_eff) needs to be at least 10% higher than the corresponding efficiency of an aircraft that is given as reference. The aim of this constraint is to simulate real situations where there could be a need of developing an aircraft the characteristics of which are better than those of rival countries. 3.5 Evaluation criteria The aircraft selected by each team is evaluated by taking into account both the overall results obtained at the end of the project (third phase) and the partial results reached at the end of the first and the second phase. The global score is computed using the values listed in tables 1, 2 and 3. Table 1 - Parameters evaluated at the end of the project Parameter Points Requirements satisfaction (A and B) 42 Lowest development cost 4 Lowest aircraft cost 7 Best AA efficiency 8 Best AG efficiency 5 Best stealth 5 Shortest runway 4 Highest speed Hi 3 Highest speed Low 3
Table 2 - Parameters evaluated at the end of the first phase Parameter Points Requirements satisfaction (A) 6 Highest speed Hi 3 Highest speed Low 3
Table 3 - Parameters evaluated at the end of the second phase Parameter Points Requirements satisfaction (A) 6 Highest speed Hi 3 Highest speed Low 3
Points are given to requirements satisfaction at the end of the first and second phase to simulate that, in real scenarios, it is good to comply with the requirements from the early phases of the design process. 4. Implementation As pointed out by Conallen (2000), a web application has an architecture similar to that of a client/server system, but with few notable distinctions: 1) no special software or configuration is required on the part of the client; 2) the principal communication protocol of a web application is HTTP. During the last years, the global trend in the development of web-based systems was to move as much code as possible to the server-side. This is mainly due to the following reasons: 1) there is the great advantage that the application can be accessed through a simple web browser without being compelled to download programs, plug-ins or applets; 2) the development of clientside web GUIs based on dynamically generated HTML pages, that was cumbersome up to some years ago (it was usually carried out using techniques such as Servlets and/or CGI scripts), is now much easier thanks to recently released frameworks such as JavaServer Faces, Struts, and others. The above stated considerations suggested that we adopt a web-based architecture where the final users need only a web browser to connect to the system. 4.1 System architecture In general the most adopted architecture to implement such frameworks is the three-tier approach. This method usually presupposes that the application data are usually stored in a database, presented to the users through a GUI, and modified by the user through a set of events generated through the GUI. Sometimes these tiers are referred to as persistence layer, presentation layer and business layer. The three-tier structure follows, to a great extent, the Model-View-Controller (MVC) pattern, as described by Gamma, Helm, Johnson and Vlissides (1995). The basic key feature of the MVC pattern, recently referred to as front controller, is that all requests into the application flow through a common application level controller (McClanahan, 2005).
Figure 7 – Implementation architecture Figure 8 - Simplified UML diagram
According to various authors (Kassem, 2000; Hadjerrouit, 2001) the MVC design pattern provides a host of design benefits, and fits well to web-based application. MVC separates design concerns (data persistence and behavior, presentation, and control), decreasing code duplication, centralizing
control, and making the application more easily scalable or modifiable. MVC also helps developers with different sets of skill to focus on their core skills and collaborate through clearly defined interfaces. In figure 7 the three tiers are shown. In the presentation layer all the GUIs are implemented, such as team login, input and output masks, tests results, masks for retrieving tests data, etc. Presentation layer is a server which generates HTML dynamic pages sent to the user. The model of the aircraft is implemented in the business layer together with the following: equations linking inputs to outputs, noise generation in the tests, calculations of overalls and team ranking. Business layer is server resident. In the persistent layer all data concerning teams such as team members, tests done and aircraft prototypes, are stored in the database. Persistent layer is in the server. 4.2 Adopted frameworks At present various frameworks can be adopted for the development of web-based applications, such as: PHP/MySQL, .NET, J2EE and others. In this work, in order to develop a scalable and portable system we decided to implement the hole framework on a pure Java platform. We decided to adopt this solution for the following reasons: the Java language is intrinsically clear and concise; it is easy to integrate modules developed by different teams; the configuration work required to set up the server and the database is minimal; the application can be easily distributed (deployed) through a single file (.war) One of the main reasons that made this implementation possible was the absence of any legacy software to deal with. As depicted in figure 7, the three levels of the framework rely, respectively, on the following tools: JavaServer Faces (presentation layer), Java (businees layer) and Hibernate/HSQLDB (persistence layer). Figure 8 shows a simplified UML class diagram of the system architecture. The presentation layer is based on the UserInterfaceBean class which acts as the backing bean for the JSF pages. JSF allows the development of user interfaces in a similar way as they are developed in stand-alone applications. It is recognized that JSF provides the following advantages (Hightower, 2005): clean separation between behavior and presentation, event-driven development, the possibility to develop reusable GUI components. The core of the business layer is represented by the Team class, which models a team that participates in a game. User-generated events are sent to the Team methods through the UserInterfaceBean. There are several ways to persist data with Java. The persistence layer is one of the most important layers in a Java application. In most cases the persistence store is a relational database. In this application the persistent layer is based on the Hibernate technology (King and Bauer, 2005). Some of the features Hibernate offers are the following: support for OO features (inheritance, polymorphism, composition and the Java collections), automatic support for optimistic locking and no build-time source or byte code generation. Another powerful feature of Hibernate is that it provides a layer of independence with respect to the underlying database. This makes the business layer independent of the database. In this specific application we adopted HSQLDB, a free SQL relational database engine written in Java. The entire application can be deployed in a (.war) file and run in a J2EE compliant server (the framework has been tested with Jboss 1.4), without any additional configuration. 4.3 Game development and interactions with three-tiers implementation. The administrator of the game can connect to a restricted area of the site with a special login. He can create a tournament by assigning a username and password to each team and decide the
duration of the game phases. In this case the presentation layer sends administrator inputs to business layer that save them in the persistent layer. At the end of each phase general classifications are made in the business layer and stored in the persistent layer. A team member opens the homepage game (http://dismac.dii.unipg.it/webgame) through a browser. Then he logins with team username and password. All teams can login simultaneously and also different members of the same team. In the latter case, to avoid conflicts in writing an optimistic locking to the database, where the first modification is persisted and the member submitting the second change receives an error message, was adopted. During an active login session a team member can make a new test generating a new aircraft prototype. Here the presentation layer interacts with the business layer by sending it the input data. The business layer makes calculations and saves in the persistent layer data tests inputs and outputs. A member can also retrieve previous tests and prototypes, can visualize the general classification of ended phases and current budget situation. In this case the presentation layer asks data to the business layer that in turn interrogates the persistent layer and sends results to the presentation layer. 5. Evaluation questionnaire and experimental feedback In order to measure the degree of fulfillment of the three overall aims stated in the introduction an evaluation questionnaire addressed to the participants of the game was developed (Table 4). Regarding the first aim, two sets of questions were prepared. Evaluation of the simulation game set inquires the usability of the system (clarity of rules and goals, user friendliness of the interface and of data insertion and reading). Adopted strategies set wants to point out if competition stimulates players in looking for particular winning strategies. Regarding the second aim, two sets of questions were defined. First set (Adopted tools) identifies the software tools that could be used for supporting the design process. Second set (Adopted techniques) investigates which techniques for the design of the complex system were applied in the game. Regarding the third aim, the Team organization set of questions was written in order to see how much relational aspects of the simulation game played a part in the game course. At the end, two general questions concerning an overall evaluation of the proposed game were added. The answer to each point of the questionnaire is an integer value ranging from 0 to 3 (0=nothing, 1=little, 2=some, 3=much). Note that the questionnaire has been designed referring to a generic complex system and not specifically to the aircraft model. The questionnaire represents the means by which it will collect a statistical relevant number of observations in order to reach a meaningful evaluation of the WBSG. For a first test, a trial competition involving three universities has been carried out. The experimental test was conducted involving six teams, each one composed of four players. The teams which participated in the test were all located in Italy: four in Perugia, one in Terni and one in Genova. The final situation of the competition is reported in figures 5 and 6. The teams were mainly composed of postgraduate and doctoral students that had not yet a specific background in design of complex systems techniques. Each of the three phases of the game lasted 24h. At the end of the competition the participants were asked to fill in the questionnaire. All 24 participants in the game sent back the form filled in. Results are shown in table 4. Table 4 - Results of the evaluation form Evaluation of the simulation game Are the rules clear? Is the goal of the game clear? Is the system user-friendly? Is data insertion easy?
Avg (std dev) 2,1 (0,4) 2,9 (0,3) 2,7 (0,5) 2,5 (0,8)
0 1 2 3 (not at all) (little) (some) (much) 0 1 19 4 0 0 2 22 0 1 5 18 0 4 5 15
Is data reading easy? Adopted strategy Performance maximization Cost minimization Fulfillment of the requirements Fulfillment of the requirements in the first two phases Adopted tools Spreadsheet (Excel, ecc.) Computing environments (Matlab, ecc.) Programming languages (Visual Basic, C/C++, Java) Adopted techniques Optimization techniques Game theory Fuzzy logic Design of Experiments Artificial intelligence (artificial neural networks, etc.) Team organization All the members actively participated in the project A team leader spontaneously emerged The tasks were efficiently distributed among the members There have been frequent discussions and brainstorming Overall system evaluation Usefulness of the specific system General usefulness of simulation games in engineering learning
2,7 (0,6)
0
2
4
18
2,9 (0,3) 1,5 (0,8) 2,9 (0,3) 1,8 (1,2)
0 0 0 6
0 15 0 3
3 5 2 5
21 4 22 10
3,0 (0,0) 0,3 (0,6) 0,3 (0,6)
0 20 20
0 2 2
0 2 2
24 0 0
0,3 (0,6) 0,0 (0,0) 0,0 (0,0) 0,8 (1,0) 0,0 (0,0)
20 24 24 14 24
2 0 0 1 0
2 0 0 9 0
0 0 0 0 0
2,9 (0,3) 1,8 (0,8) 1,2 (0,7) 2,9 (0,4)
0 4 2 0
0 0 18 1
3 18 2 1
21 2 2 22
2,8 (0,4) 2,9 (0,3)
0 0
0 0
4 2
20 22
The results show that the general look of the system was appreciated by the participants. The fact that specific strategies were adopted could be a sign of the competition level among teams. The main adopted strategy, for instance, was performance optimization and fulfillment of the requirements. Regarding the adopted tools and techniques, we can notice that systematic procedures (optimization techniques, design of experiments and similar) did not received great attention, this could be because the student did not know the techniques. Nevertheless, the high average values in "Overall system evaluation" are due mainly to the fact that after the game participants are motivated in studying such techniques. In relation to team organization we can see that the simulation environment encouraged discussion and brainstorming, even if the distribution of the work load among the team members was not very satisfactory. In this case it seems that the teams that did not efficiently distribute tasks among its components are those that did not exploit the possibility to make multiple login. 6. Conclusions and further developments The WBSG here presented was designed in order to fill the existing gap in the development of relational simulation games related to engineering aspects, such as the design process of complex systems. In order to cover this gap a game was implemented based on the three aims stated in the introduction. All decisions made in the design of the game were functional to the fulfillment of these aims. From the first results it seems that the proposed game represents a meaningful step in the right direction. The proposed application can be used as an active teaching tool where students can be stimulated to develop their relational and practical capabilities. In particular the scholars can be encouraged to adopt the design techniques that can be applied to the management of complex models. The trial competition has shown a good feedback from the students who participated. Future application in engineering courses may integrate the teaching of structured approaches in the design process with the simulation environment presented in this paper.
Future developments will include the extension of the framework to other engineering contexts, such as any complex system (other complex products, maintenance services, industrial plants etc.). In effect, although the acquisition of specific know-how in the design of a particular complex system is not within the goals of the presented game, the adoption of very accurate models would be still desirable. In this way the possibility to familiarize with the specific product design will provide an additional skill to participants. At present both the rules of the game and the product model are hard-coded in the business layer. A significant improvement of the system will be the development of a reconfigurable environment, where the game rules and the simulation model can be easily modified through the upload of configuration files. Acknowledgements This work was supported by the MISS (McLeod Institute of Simulation Sciences), Perugia Center. The authors thank Prof. Agostino G. BRUZZONE of the University of Genova for his valuable suggestions. The authors also thank the anonymous reviewers and the editor for their valuable suggestions which significantly improved the quality of the paper. References Bruzzone, A. G. (2001) Web-based simulation: Best of Websim99. Future Generation Computer Systems, 17 (5), 501-502. Cadotte, E.R. (1995) Business Simulations: The Next Step in Management Training. Selections, Graduate Management Admission Council. Autumn, pp. 8-19. Chatterjee, S., Paramasivam, M. and Yakowenko, W.J. (1997) Architecture for a Web-Accessible Simulation Environment. IEEE Computer, 30 (6), 88-91. Conallen, J. (2000) Web Application Architectures with UML. Communications of the ACM, 42 (10), 63-70. Demir, S.S. (2003) iCell: an interactive web resource for simulation-based teaching and learning in electrophysiology training. Proceedings of the 25th Annual International Conferenceof the IEEE EMBS, Cancun, Mexico, pp. 3501-3504. Esposito, E., Michelot, C. and Thomas, G. (2002) Using business games to enhance engineering students capacity for individual and social awareness. Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Hammamet, Tunis, pp. 560-563. Gamma, E., Helm, R., Johnson, R. and Vlissides, J. (1995) Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley. Goleman, D. (1995) Emotional intelligence. New York, Bantam Books. Hadjerrouit, S. (2001) Web-based application development: a software engineering approach. ACM SIGCSE Bulletin, 33 (2). Hembree, R. (1998) Correlates, causes, effects, and treatment of test anxiety. Review of Educational Research, 58, 47-77.
Hightower, R. (2005) JSF for nonbelievers: clearing the FUD about JSF. Available at http://www106.ibm.com/developerworks/library/j-jsf1/. Ip, A., Linser, R. and Naidu, S. (2001) Simulated Worlds: Rapid Generation Of Web-Based RolePlay. Proceeding of the Seventh Australian World Wide Web Conference, Coffs Harbour, Australia. Kassem, N. (2000) Designing Enterprise Applications with the Java(TM) 2 Platform (Enterprise Edition). Addison-Wesley. King, G. and Bauer, C. (2005) Hibernate in Action. Greenwich, Manning. Kumara, R.T., Lee, Y., Tang, K., Dodd, C., Tew, J., and Lee, S. (2002) Simulation anywhere any time: web-based simulation implementation for evaluating order-to-delivery systems and processes. Proceedings of the 2002 Winter Simulation Conference. San Diego, CA, pp. 1251-1259. Leong, T.K., Ali B.M., Prakash V. and Nordin N.K. (2000) A prototype of Web-Based Simulation Environment: Using CGI and Javascript. Proceedings of TENCON 2000. Kuala Lumpur, Malaysia, pp. 357-360. Linser, R. (2004) Suppose you were someone else… The learning environment of a web-based roleplay simulation. Proceedings of the SITE 2004 International Conference. Atlanta, GE. Masson, A.M. (1999) Web-based simulations for computer-assisted learning in the higher education sector, Engineering Science and Education Journal. 6, pp.107-114. McCelland, D.C. (1985) Human Motivation. Glenview, Scott, Foresman and Co. McClanahan, C. (2005), The Evolution of Web Application Architectures. O'Reilly Open Source Convention, Portland, USA. Narayanan, S. (2000) Web-based Modeling and Simulation. Proceedings of the 2000 Winter Simulation Conference, Orlando, FL, pp. 60-62. Ramirez, A. (2002) Three Tier Architecture. Linux Journal, 74. Salovey, P. and Mayer, J.D. (1990) Emotional intelligence. Imagination, Cognition, and Personality, 9, pp.185-211. Spielberger, C. D. (1980) Test Anxiety Inventory. CA, Consulting Psychological Press Inc. Vaccani R. (1997) Le metodologie didattiche. Progettare la formazione. Guida metodologica per la progettazione del lavoro in aula. ed M. Castagna, pp. 142-144. Franco Angeli, Milano.
