Using Immersive Virtual Reality to Reduce Work

Visual Computing and the Progress of Developing Countries

Using Immersive Virtual Reality to Reduce Work Accidents in Developing Countries Luciana Nedel, Vinicius Costa de Souza, and Aline Menin ■ Federal University of Rio Grande do Sul Lucia Sebben ■ Sebben Business Consulting Jackson Oliveira ■ AES Sul Frederico Faria ■ Nexo Capacitação Digital Anderson Maciel ■ Federal University of Rio Grande do Sul

B

razil’s Ministry of Social Security officially registers more than 700,000 work accidents every year. Approximately 2,800 workers die as a consequence of these accidents, while 15,000 become permanently disabled. These numbers do not account for many other cases not officially reported. Besides Immersive VR simulators can the intangible cost in lives, the help reduce the number of government and large companies accidents in the workplace. spend more than US$30 billion a Far more effective than year to address the consequences traditional training safety of such kinds of accidents. In this measures, the proposed context, the industrial sector and utility companies have been insystem simulates day-to-day vesting in innovative projects to situations and analyzes user improve work safety. The use of reactions to detect behavioral virtual environments (VEs) for patterns that may lead to an increased predisposition to risk training safe procedures is continuously spreading. In many exposure. areas, however, more than welldesigned procedures and a well-trained staff wearing individual protection equipment (IPE), a number of human factors are crucial for safe behavior. In terms of human factors, the ability to perceive risks varies among individuals and is affected by their beliefs, motivations, and relations with other 50

March/April 2016

people. In other words, people react according to their mental models of a potentially risky situation rather than to the real risk itself.1 For example, a trash bin dropped in a hallway or on a sidewalk represents a risk for some people. However, many might not even notice the object’s presence until they trip over it or will simply walk around it. In the latter case, there may be no memory of the encounter at all. Virtual reality (VR) simulators have been used in many training areas, including the ability to perceive dangerous situations. Typical examples are flight, surgery, and driving simulators. In such cases, immersive VEs are created to reproduce the real environments as accurately as possible to impose on the immersed subject the feeling of presence, which in turn elicits behavioral responses that are faithful or at least plausible in relation to the person’s behavior in a real environment. In this article, we propose the design and use of VR simulators to continuously assess capability for safe behavior among workers. Here, we introduce our framework for risk perception simulation, discuss how we intend to use it to build psychosocial behavior profiles, and share our results from an experimental evaluation we carried out with real users.

Published by the IEEE Computer Society

0272-1716/16/$33.00 © 2016 IEEE

Risk-Related Works in VR Computer-based simulations have been used for many purposes, but only a limited number can be linked to risk and safe behavior. The training of medical doctors in a virtual battlefield, for example, facilitates the recognition and understanding of injured soldiers in extreme situations.2 Simulators have also been used to train police officers to better evaluate crime risks, develop contingency plans,3 analyze crime scenes, and improve efficiency and accuracy in the judgment of forensics analysis experts.4 VR-based solutions have also been used to develop a system that trains a team of workers in electricity-line maintenance.5 That system is based on a large display and the use of Wiimotes (Nintendo Wii remotes) for interaction. Simulators have also been used for training in the operation and maintenance of electrical substations. For example, Zhang Bingda and his colleagues developed a distributed system that can simulate eight different kinds of failures that, combined, can generate more complex failures.6 However, all these systems provide user interaction in a rather conventional manner, using menus and text boxes associated with keyboard and mouse devices, as is common in nonimmersive 3D games. These simulators do not match the minimum requirements of VR applications. As a consequence, the level of subjective presence in the simulated environment is low, causing the users to behave differently than in real life.7 The training of a worker should involve different situations and equipment.6,8 VR fits these needs well and allows for faster and lower-cost development of such scenarios and tools. Additionally, invisible substances in the real world, such as radiation, electricity, and gas may be visualized in VEs.8 In addition, research has shown that the knowledge acquired during a VR training session is transferred to the real environment and that the use of VR offers a challenging environment, which improves learning.8 VR researchers have already determined that the use of VR can positively affect human sensorial perception. This issue is similar to the teleoperation of machines and robots, where viewing the environment through cameras and the use of unnatural controllers to accomplish tasks changes the way we perceive and act. These scenarios impose a narrower field of view, different lighting conditions, and variable refresh rates. Strategies used to mitigate these problems are constantly being explored. Some approaches include 3D displays with a wide field of view, data gloves, and other body trackers.

A Framework for Risk Perception Simulation Based on the current state of the field, we created a framework for the development of simulators for risk perception assessment and training. Our framework was first used (as we described in a previous work9) to build a simulator for risk perception analysis. That first simulator was designed to periodically evaluate a worker’s capability to perceive risks in different scenarios. Figure 1 illustrates some of the simulated scenarios. In a typical simulation session, the user wears a head-mounted display (HMD) and walks around using a game controller. When users detect elements of potential risk in the scenario, they must select them with the controller or a gesture. At the end of the session, a report is issued indicating the selected objects, the undetected risks, and the path traveled. The simulator also allows the development of new metrics for risk perception and task performance, contributing both to the safety and efficiency of the activities. We performed a number of experimental simulations with several groups of subjects to refine the tools and methods. This helped us to generalize the simulator in the form of a framework that serves as a basis for developing a plurality of simulators focused on other risk-related issues. The framework is based on several possible interface equipment options, such as a HMD, data gloves, gesture sensors, a gamepad, and weight sensors. We were interested in a system that was fully immersive, portable, and simple to integrate. With these assumptions in mind, we chose devices and programming environments that could provide such characteristics while minimizing the need for development resources. One of these is a game engine. Although computer games are usually not fully immersive, they are by definition interactive. Game engines, then, provide many features to simplify VE development. We found that workplace goals can often be translated into game goals. Also, the computer graphics required to create several visual effects are easily deployed using the average game engine. When compared with 3D graphics engines, game engines have the advantage that they map several events, including those that are not always available in more general graphics engines (such as the events generated by VR input devices) in a clean way. These events provide a way to quickly create simple games with predefined goals. We have successfully implemented the framework using both the Unreal Development Kit (UDK) and Unity3D. IEEE Computer Graphics and Applications

51


(a)

(b)

(c) Figure 1. Three scenarios used in the risk analysis simulator for electric utility companies. (a) Office environment, (b) lightning rod replacement, and (c) substation operation.

The use of gesture sensors, such as the Microsoft Kinect, is strategic as an affordable solution to track users’ limbs and apply their motions to their avatars. During the system development and user experiments, many users seemed to enjoy the experience of having control over their virtual limbs using natural body motions, which increases the feeling of presence. For more precise hand movements, the framework also supports data gloves. However, in practice, the target users may not adjust to tight, cumbersome data gloves. Until now, the simulators we developed did not involve situations requiring precise hand manipulations, so we have avoided the use of data gloves. For locomotion, we use either a standard gamepad, which is readily supported by engines and has an accurate response to trigger actions, or a walking-in-place strategy. The gamepad provides 52

March/April 2016

constant speed locomotion, and users can select or deselect items by pushing any of the gamepad buttons. The walking-in-place solution, in turn, offers a more natural interface. To detect feet activity, we used a weight sensor platform implemented using four Wiimote balance boards. Our algorithm counts the user’s steps and estimates the direction and speed of walking. The framework also offers various display options. The developer or end user can choose between using a regular monitor display, HMD, or mini-CAVE. The HMD was our first choice because it allows for greater portability. However, we also developed a mini-CAVE based on three angled 55-inch stereo TV sets that also provides high immersion. The mini-CAVE is portable and avoids most of the cybersickness caused by current HMDs. This last setup was preferred by the users in the user tests we conducted.

Table 1. Behavioral group characteristics. ID

Group

Behavior

G1

Quality

Corresponds to the philosophy of continuous improvement. Every task should be done correctly, to the best of one’s ability. Learned procedures and rules are followed.

G2

Health

Involves a person’s carefulness in terms of mental and physical health aspects, his/her preventive attitudes, and care with his/her own integrity and life.

G3

Environment

Includes attitudes toward environmental preservation and actions that promote a healthy and sustainable life for the collective community.

G4

Emotional balance

Involves an individual’s capability to withstand pressure, manage stress, minimize wear, control impulses and emotions, have a sense of urgency, and maintain balance and stability in relation to internal demands and external requirements.

Building Psychosocial Behavior Profiles Psychologists use several methods to build profiles. Based on the target application, methods are chosen to measure cognitive skills, social skills, competences, personality, and so forth. For psychosocial profiles, one method currently in practice in Brazil is based on four behavioral groups oriented to quality, health, environment, and emotional balance. Table 1 briefly lists the characteristics of each group. In the context of a cross-disciplinary endeavor to effectively reduce the number and gravity of work accidents, we developed a new conceptual application for a VR simulator. The goal was to help recruiters assess the psychological profile of job candidates. The simulator aims specifically at classifying users in terms of their tendency to misbehave when confronted with psychosocial risks. It was, again, developed in the context of an electric utility company that wished to measure, at an early stage, a job applicant’s propensity to become involved in an accident if he or she is assigned to certain functions. The safety specialists assume that the process of preventing serious injuries and loss of life starts with recruiting employees who demonstrate psychological, emotional, and attitudinal conditions appropriate for risky environments. In other words, some psychological aspects of professionals’ profiles can define their longterm susceptibility to expose themselves to danger even before any in-house safety training. After validation, the simulator will be inserted into the recruitment process. In this new process, candidates will be invited to take part in an immersive VR experience, where they navigate in a virtual scenario representing ordinary day-to-day situations, some of which may involve some kind of danger or unexpected events. The following synopsis describes the plot of the simulation: The simulation starts in an urban scenario with the participant standing at a central place. Then a cell phone rings. When the participant answers, a voice instructs him

or her to go to the parking lot across the avenue to look for a wallet that was forgotten in the car. On the way to the car, the participant faces situations such as finding a safe way to cross the street. Without being careful, the participant cannot avoid getting hit by a cyclist. At a certain point, before the participant reaches the car, a traffic accident occurs in which a person is injured. The participant can choose to ignore the accident or intervene by calling an ambulance or the police (with the cell phone) and trying to help a child and/or comfort a woman involved with the accident. In the sequence, the cell phone provides new instructions. After executing each given task, another task is proposed, until the end of the simulation. Other key elements are a bank ATM where the participant must wait in line and deal with an out of service message, choose and pay for a product at a store, pay a bill and choose between a regular or priority line at a lottery retailer, and find a free table in a restaurant, where he or she must perform basic hand washing and throw away garbage actions. Figure 2 illustrates three of the situations that the user faces in the virtual world. At the end of each session, the system outputs a report with a summary of the user’s decisions and the actions taken given the experiences in the virtual world. Every setting in the virtual environment was carefully designed to verify which among a set of possible reactions the participant took. Depending on the reactions logged, the candidate is classified according to the four behavioral groups described in Table 1. The simulator was designed using the framework we described in the last section and developed on Unity3D. The display options include a HMD and a mini-CAVE. Figure 3 depicts real users in action with the simulator. IEEE Computer Graphics and Applications

53


■■

■■

Do employees previously recruited by the traditional selection method obtain consistent results with the simulator? Do the subjects feel present during the simulation?

The first question is important because being able to reproduce the results of the traditional methods being simulated is the first milestone to be achieved by VR assessment simulators. Then, presence is crucial to ensure the credibility of the users’ actions—that is, would they act the same way in real life? If both answers are positive, CEOs and funding agencies will be more likely to support a wider study on the impact of the simulator on long-term accident mitigation. For this experiment, we invited employees from an electric utility company that currently provides service in southern Brazil. To register their perceptions about the use of the system and subjective presence, we use a typical experimental protocol with the following steps: introduction, informed consent form, profile questionnaire, practice, actual recorded simulation, and postquestionnaire. In addition to system logs about behavioral groups and user performance, we also recorded video (user and screen) and heart rate (HR).

(a)

(b)

Materials

(c) Figure 2. Three of the scenarios used in the simulator for profile evaluation. (a) Bicycle accident, (b) car accident, and (c) robbery.

Although the system instructions guide the users’ actions, the evaluation of the participants’ psychosocial profile is based on their decisions and their relations with the behavioral groups. Table 2 presents the set of actions that are verified by the simulator and reported to the evaluator at the end of the simulation.

User Evaluation To measure the effectiveness of our psychosocial risk VR simulator in helping recruiters assess the psychological profile of candidate workers, we carried out a user evaluation study. Our approach depends on two parallel analyses that answer the following questions: 54

March/April 2016

The VR simulator was implemented on Unity3D and run on a PC equipped with a 3.4 GHz Intel Core i7 processor, 8 Gbytes of RAM, and an NVidia GTX 680 graphics card. For the visualization, we developed a mini-CAVE based on three angled TV sets that provides high immersion, is portable, and avoids most of the cybersickness caused by current HMDs. We used three 55-inch 3D full HD LG 55LM6200 TVs with stereo glasses and an XBox controller for rotation and locomotion (see Figure 3b). One of the sticks allows the user to move forward and back, while the other one is used to look around, up, and down. The X button on the controller is used to call the ambulance or the police, and the A button is used to pick-up items.

Participants The evaluation involved 15 employees (nine male and six female) of AES Sul, an electric utility company. The employees were from different departments: administrative, human resources, information technology, technical services (electrician), safety engineering, and sales. The average age of the participants was 29 years (with a standard deviation of 6.08). In terms of education, 46.66 percent of them have a high school degree

(2)

(1)

(3) (4)

(a)

(b)

Figure 3. Alternate displays. (a) The HMD was used in pilot evaluations with a significant level of cybersickness. (b) The final setup used in the user evaluation employed (1) a TV-based mini-CAVE, (2) shutter glasses, and (3) a gamepad. (4) The webcam below the screens records the user’s actions for subjective evaluation.

and 53.33 percent have college degrees. One subject (female) dropped out before concluding the exercise due to cybersickness. The experiments were conducted in a dedicated room at the company office, which was reserved exclusively for the use of the simulator. We profiled participants in terms of the frequency they play videogames on a five-point scale, where 1 was never, 2 was a few times, 3 was about once a month, 4 was about once a week, and 5 was every day. Two subjects reported they had never played video games, 10 had played a few times, two played once a week, and one played every day. We also questioned subjects about how often they see 3D movies. Five subjects reported that they had only seen one, eight had seen more than one, and two had seen a 3D movie more than five times.

Table 2. Behavioral variables and their relation to the behavioral groups.* ID

Variable

G1

V1

Paid for goods purchased in stores.

Behavior

G1

V2

[Did not] use the priority queue at the lottery retailer.

G1

V3

In the restaurant, sat down at a clean table.

G2

V4

Used the pedestrian lane.

G2

V5

[Did not] try to confront the thief during the robbery.

G2

V6

Called the ambulance after the accident.

G3

V7

Remained in line at the ATM waiting for their turn.

G3

V8

Asked the police officer for help during the robbery.

G3

V9

Called the police officer by cell phone.

G3

V10

[Did not] ignore the robbery.

G4

V11

Tried to help the child in the accident.

G4

V12

Tried to calm the woman in the accident.

*Some variables are negated to ease the comparison.

Measures Participants wore a chest strap sensor (with a Polar transmitter) to record their HRes. HR is increasingly used in studies of presence10 as an index of sympathetic arousal. That is, an increase in HR indicates increased sympathetic influence, which is related to increased stress level. Before participants tried the simulator, we measured their HR baseline values (the HR value observed when a participant is in a resting state). When analyzing physiological data, the participant’s baseline value is subtracted from the data recorded during the experimental condition. This helps to separate the physiological responses to experimental stimuli from the intrinsic biological differences among participants. At the end of each simulated session, the system outputs a report with a summary of the decisions made and the actions taken in face of the experiences the users experienced in the virtual world. Table 2 lists the set of actions that are verified by

the simulator and reported to the evaluator at the end of the simulation. After the simulation session, the subjects filled out a SUS (Slater-Usoh-Steed) presence questionnaire11 to evaluate the level of subjective presence during each task. We included one questionnaire for each part of the simulator: central place, traffic accident, parking lot, robbery, bank ATM, shopping store, lottery retailer, and restaurant. The questionnaires consisted of six seven-point Likerttype questions, where 1 corresponded to not feeling there at all and 7 to the highest sense of being there (as experienced in the real world).

Procedure First, the participants signed a consent form for participation and for recording physiological data, video, and answers to the initial questionnaire. They were also informed that they could refrain from continuing the experiment at any IEEE Computer Graphics and Applications

55

Visual Computing and the Progress of Developing Countries 7

30

6

25

(a)

nt ura sta Re

Lo tte ry

Sto re

ery

AT M nk Ba

Ro bb

lot ng rki

al ntr Ce

Pa

pla ce

nt ura sta Re

Lo tte ry

nk Ba

Ro bb

lot

de

ng rki Pa

e

cci

fic a

al

pla c

Tra f

ntr Ce

Sto re

0

AT M

0

ery

5

nt

1

nt

10

de

2

15

fic a

3

20

Tra f

4

cci

Variation (%)

5

(b)

Figure 4. Dependent variables used to measure presence from the user experiment. (a) SUS mean values from the questionnaire and (b) HR means above each individual’s baseline.

time without providing a reason to the experimenter. After participants gave their consent to participate in the experiment, they filled out an initial questionnaire that included their age, gender, level of education, job position, frequency of game use, and how often they watch 3D movies. After that, the participants were invited to wear the chest strap HR monitor and were helped by the experimenter to adjust it until they felt comfortable. Then, the experimenter asked the subjects to sit in a chair and relax for three minutes, during which their HR baselines was recorded. After the HR baseline acquisition, the experimenter handed the 3D TV shutter glasses and the game controller to the participant and explained the controls. Participants used the game controller with their preferred hand and tried the controls on a simple VE (a virtual office room) until they fully understood how to navigate and select actions. More precisely, the experimenter invited each participant to look around, move forward, briefly explore, and select actions. All participants quickly understood the controls. The actual test then occurred (experimental condition). After the test, the chest strap HR monitors were removed, and the participants filled out the postquestionnaires about their sense of presence.

Presence Assessment Presence is still a concept that is often misunderstood. It is also difficult to measure because it may be influenced by factors as varied as user cultural background and graphics rendering quality. We applied a two-fold approach to measure both the subjective feeling of being there and other more objective parameters, such as the physiological response of the sympathetic branch of the au56

March/April 2016

tonomic nervous system,12 which translates into HR variations.

Subjective Presence We collected responses to the six SUS presence questions associated with each of the simulated scenarios (central place, traffic accident, parking lot, robbery, bank ATM, store, lottery retailer, and restaurant). These helped us to observe how the subjective presence varies among scenarios (see Figure 4a). We also calculated the SUS mean score across the six questions, which has already been described in previous studies as an effective tool to measure presence.11 Table 3 lists the mean values for each location. Notice that all responses are positive, with mean values above 4.2 and standard errors below 0.3.

Objective HR We recorded the participants’ HRs associated with each simulator scenario once every second. Figure 4b presents the mean values for HR as a percentage above each individual’s baseline. This measure of the variation above the baseline is necessary to normalize between users with different baselines. We must be careful in directly relating HR with presence, however. There are many competing physiological factors pushing and pulling the HR. One observation of the results in Figures 5 and 4b is that the HR increases throughout the experiment, lowering at the end. This second-order tendency is mostly due to the stress of being tested, although it can also be interpreted as the user becoming more present over time. More interestingly, we can see that, for some of the situations, when user attention and motor activity are in more demand, the HR is locally higher. This aligns

Table 3. SUS presence questionnaire responses for each simulated scenario.* Central place

Traffic accident

Parking lot

Robbery

4.80

4.67

5.40

4.67

(0.44)

(0.44)

(0.25)

2. Rate your feeling that the simulated scenario was real.

4.80 (0.45)

4.80 (0.40)

3. How real do you remember the simulated scenario?

3.80 (0.49)

4. Rate your feeling of being inside simulated scenario or observing it.

4.20

Question 1. Rate your sense of being in the simulated scenario.

5. How is your memory of the scenario similar to being in real places? 6. Did you think you were really in the simulator situations? SUS means

(0.50) 4.60 (0.42) 3.93 (0.50) 4.36 (0.18)

Bank ATM

Store

Lottery retailer

6.07

5.80

6.20

5.60

(0.51)

(0.25)

(0.26)

(0.24)

(0.25)

5.20 (0.33)

4.93 (0.47)

5.80 (0.40)

5.33 (0.36)

5.93 (0.41)

3.73 (0.61)

5.33 (0.41)

3.40 (0.58)

5.60 (0.40)

5.33 (0.41)

5.60 (0.46)

4.80 (0.50)

4.00 (0.56)

4.47 (0.56)

4.93 (0.54)

4.53 (0.53)

5.07 (0.56)

4.33 (0.50)

5.00 (0.49)

4.60 (0.51)

5.33 (0.47)

4.60 (0.49)

5.00 (0.52)

3.67 (0.61)

4.07 (0.50)

3.53 (0.61)

4.60 (0.50)

4.27 (0.45)

4.80 (0.54)

4.33 (0.21)

4.83 (0.26)

4.27 (0.26)

5.39 (0.22)

4.98 (0.24)

5.43 (0.23)

Restaurant

5.47 (0.38) 4.93 (0.53) 4.33 (0.57) 5.47 (0.42) 4.47 (0.54) 5.04 (0.22)

*Data lists the mean score and the standard error of the mean in parentheses. Scores are based on a seven-point Likert scale, where a 7 indicates the highest sense of being there (as experienced in the real world).

Central place

Traffic accident

Parking lot

Robbery

Bank ATM

60 sec.

26 sec.

128 sec.

Lottery

Restaurant

120

BPM

100

80

60 27 sec.

48 sec. Store

120 Users

BPM

100

80

User 1

User 7

User 13

User 2

User 8

User 14

User 3

User 9

User 15

User 4

User 10

User 5

User 11

User 6

User 12

60 83 sec.

98 sec.

18 sec.

Figure 5. Participants’ heart rates during each of the simulated scenarios. These values are provided as a percentage above each individual’s baseline in seconds for each scenario.

IEEE Computer Graphics and Applications

57


Both the subjective and objective elements signal a significant level of presence, and a correlation can also be observed between them. Note in Figure 4 that both the SUS presence questionnaire responses and HR values are higher in the second half of the experiment (ATM, shop, lottery retailer, and restaurant). Although these two results are weak when viewed separately, together they indicate that the sense of presence has grown with the use of the system.

Effectiveness for Psychosocial Risk Profiling

Figure 6. Three scenarios where the demand for user manipulation is higher. In these situations, the participants reported a higher sense of presence.

with previous literature on presence that relates manipulation activities with a higher feeling of presence than merely observational activities. This is even more evident when correlated with the SUS presence questionnaire responses in Figure 4a. For example, the parking lot, bank ATM, and lottery retailer situations present a higher subjective presence than the other situations. Figure 6 illustrates these scenarios. Moreover, Figure 7 shows that situations demanding manipulation induce a higher oscillation of the HR (a wider range of HRs per time slice). This can be associated with the user’s body trying to accommodate the feeling that the situation is real and the knowledge that it is not (sympathetic versus parasympathetic). 58

March/April 2016

Figure 8 gives a summary of the participants’ decisions and actions in the virtual world. Observe that V3, V7, and V8 are undesirable behaviors, so they have been negated to allow a more meaningful presentation of the results. Because our population was selected from people previously evaluated by psychologists as suitable in terms of psychosocial risk and who had already received training, we expected high values for all tested variables. Nevertheless, the reactions were not all the same, indicating that the simulator was able to maintain some power of discrimination. The results in Figure 8 demonstrate that for most situations the participants reacted positively, but not equally. This demonstrates the effectiveness of the simulation, addressing the first question we posed in our analysis: Do employees previously recruited by the traditional selection method obtain consistent results with the simulator? Two variables related to behavioral group 4 (emotional balance), however, did not perform well: V9 and V11. This points to a limitation of the simulator that may have a number of different explanations. We believe that the animated virtual humans’ lack of facial expressions and meaningful gestures is mostly to blame. These two variables are closely linked to human-human interaction, and humans are very picky when it comes to imperfections in the virtual representation of humans.

H

alf a decade ago, we accepted the challenge of designing and implementing an affordable and portable simulation system to assess a person’s ability to avoid risks in the work environment. Although reducing workplace accidents is a problem with multiple factors, especially in developing countries, our approach focuses on the psychosocial ability to deal with and perceive risks. Computer scientists, psychologists, work safety engineers, and designers were involved in this project. Our simulations provide an immer-

Users 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Acknowledgments We thank all the AES Sul employees who took part in the user experiments. Acknowledgements are also due to Aneel-P&D and CNPq-Brazil process 305071/2012-2 for funding.

References 1. Y. Asnar and N. Zannone, “Perceived Risk Assessment,” Proc. 4th ACM Workshop Quality of Protection, 2008, pp. 59–64. 2. R. Kuskuntla, E.S. Imsand, and J.A. Hamilton Jr., “Enhanced Expert Field Medical Training Simulations and Their Effect on the Modern Combat

20

nt ura Re

sta

Lo tte ry

Sto re

Ba

nk

AT M

ery Ro bb

lot ng rki Pa

de cci fic a

Tra f

Ce

ntr

al

pla c

nt

e

10

Figure 7. Mean, deviation, and individual HR range in each simulated scenario in beats per minute. Arrhythmia is higher in situations where the users must perform active manipulation than in those where they are observers instead of actors. 15

Participants

sive VE where a multitude of scenarios can be presented to users that are invited to perform simple tasks. Their small reactions and overall behavior are monitored by the simulator, which outputs a report of fitness to face risk. Long-term studies are necessary to assess the effectiveness of the proposed simulators in reducing accidents. The user study we present here shows that this approach is practical, that the feeling of presence is high despite the simplicity of the equipment used, and that the simulator reports results consistent with traditional psychology tests. Our experiments also show that the simulator failed in evaluating one of the four main behavioral groups (emotional balance), which remains an issue for future work. Before the experiment described here, we conducted hundreds of hours of experimental sessions with more than 200 volunteer users. Those experiments helped us refine the interaction techniques and the user tasks. The mini-CAVE display was used by 30 percent of the participants, while the others used an HMD. Half of the HMD users dropped out before concluding the study due to cybersickness, however, which is the main reason we chose the mini-CAVE display for the evaluation reported here. In addition to system logs about behavioral groups and user performance, we also recorded video and HR for posterior analysis. Among the collected data, we analyzed the correlation of HR with the feeling of presence. This and other physiological measurements should help to build a framework to evaluate presence more objectively. In the future, we will continue to analyze these data to try to correlate a number of other dimensions. In the long term, such study will help to quantify the impact of this kind of technology on the reduction of the work accidents to attract investments and legal interest.

Range (BPM)

30

10

5

0

V1

V2

Quality

V3

V4

V5

Health

V6

V7 V8 V9 V10

V11

Environment

Emotional balance

V12

Figure 8. Variables measured from participants’ reactions to the simulated events arranged by behavioral group. Table 2 provides a description of each variable and group. Higher values indicate the participant is more suitable in terms of psychosocial risk. Life Saver Training Procedures,” Proc. 2010 Spring Simulation Multiconference, 2010, p. 49. 3. M. Kavakli, “Training Simulations for Crime Risk Assessment,” Proc. 7th Int’l Conf. Information Technology Based Higher Education and Training (ITHET), 2006, pp. 203–210. 4. X. Feng, S. Daguo, and Y. Hongchen, “Simulation Research of Crime Scene Based on UDK,” Proc. 2nd Int’l Conf. Information Science and Engineering (ICISE), 2010, pp. 1–4. 5. M. Rosendo et al., “Towards the Development of a 3D Serious Game for Training in Power Network Maintenance,” Proc. 3rd Int’l Conf. Games and Virtual Worlds for Serious Applications (VS-GAMES), 2011, pp. 16–23. 6. Z. Bingda, X. Shiwei, and W. Dong, “The Training Simulation System for Substation Based on Network IEEE Computer Graphics and Applications

59


and Expert System,” Proc. Int’l Conf. Advances in Power System Control, Operation and Management (APSCOM), vol. 2, 2000, pp. 532–536. 7. L. Chittaro and F. Buttussi, “Assessing Knowledge Retention of an Immersive Serious Game vs. a Traditional Education Method in Aviation Safety,” IEEE Trans. Visualization and Computer Graphics, vol. 21, no. 4, 2015, pp. 529–538. 8. A. Sebok, E. Nystad, and A. Droivoldsmo, “Improving Safety and Human Performance in Maintenance and Outage Planning through Virtual Reality-Based Training Systems,” Proc. 7th IEEE Conf. Human Factors and Power Plants, 2002, pp. 8–14. 9. V.A.M. Jorge et al., “Interacting with Danger in an Immersive Environment: Issues on Cognitive Load and Risk Perception,” Proc. 19th ACM Symp. Virtual Reality Software and Technology (VRST), 2013, pp. 83–92. 10. M. Slater et al., “Analysis of Physiological Responses to a Social Situation in an Immersive Virtual Environment,” Presence: Teleoperators and Virtual Environments, vol. 15, no. 5, 2006, pp. 553–569. 11. M. Usoh et al., “Using Presence Questionnaires in Reality,” Presence, vol. 9, no. 5, 2000, pp. 497–503.

12. J. Taelman et al., “Influence of Mental Stress on Heart Rate and Heart Rate Variability,” Proc. IFMBE 4th European Conf. Int’l Federation for Medical and Biological Eng., vol. 22, J. Vander Sloten et al., eds., Springer, 2009, pp. 1366–1369. Luciana Nedel is an associate professor in the Institute of Informatics at the Federal University of Rio Grande do Sul (UFRGS). Her research interests include VR, interactive visualization, and non-conventional interaction. Nedel has a PhD in computer science from the Swiss Federal Institute of Technology (EPFL). Contact her at [email protected]. Vinícius Costa de Souza is a PhD candidate in computing in the Institute of Informatics at UFRGS and is an assistant professor at Universidade do Vale do Rio dos Sinos (UNISINOS). His research interests include VR, human computer interaction, accessibility, and software process. Contact him at [email protected] Aline Menin is a master’s student in computer science in the Institute of Informatics at UFRGS. Her research interests include VR, nonconventional interaction techniques and devices, serious games, and computer graphics. Menin has a BS in computer science from the Federal University of the Southern Border (UFFS). Contact her at aline.menin@inf .ufrgs.br. Lucia Sebben is CEO of Sebben Business Consulting. She has more than 20 years of experience developing human resources in public and private companies of all sizes. Sebben has a BS in psychology from the Pontifical Catholic University of Rio Grande do Sul (PUCRS). Contact her at lucia @sebbenconsultoria.com.br. Jackson Oliveira is a manager of work safety, health, and environment at AES Sul. Oliveira has an MBA in environment and sustainability from Fundação Getúlio Vargas (FGV). Contact him at [email protected]. Frederico Faria is CEO of Nexo Capacitação Digital, a digital training company that provides online courses to the safety market and simulators to reduce the number of workrelated accidents. Contact him at [email protected]. Anderson Maciel is an associate professor in the Institute of Informatics at UFRGS. His research interests include soft tissue and surgery simulation, VR, and human-computer interfaces. Maciel has a PhD in computer science from EPFL. Contact him at [email protected].

Selected CS articles and columns are also available for free at http://ComputingNow.computer.org. 60

March/April 2016