Workload From Nuclear Power Plant Task Types

Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting - 2014

210

Workload from Nuclear Power Plant Task Types Across Repeated Sessions Rebecca Leis, Lauren Reinerman-Jones, Joseph Mercado, Daniel Barber, and Brandon Sollins University of Central Florida (UCF), Institute for Simulation and Training (IST) Nuclear Power Plant (NPP) operators complete multiple types of tasks within Emergency Operating Procedures (EOPs). Due to the potential serious consequences of committing an error, it is important to determine if the workload (WL) demands operators encounter are at acceptable levels. This study investigates whether there are workload differences are distinct between task types and if there is a difference between each task type over multiple sessions in a simulated environment. Previous research supports that EEG, ECG, and the NASA-TLX are sensitive to changes in WL. The present preliminary experiment sought to investigate WL changes for experienced participants over a number of sessions and task types. During each session, participants completed tasks derived from a combination of EOPs and subject matter expert input that consisted of checking, detection, and response implementation task types. WL changes were measured through EEG, ECG, and NASA-TLX responses. The results indicate that WL differences were found among the different task types, but not sessions. The implications for these findings are discussed in detail.

Copyright 2014 Human Factors and Ergonomics Society. DOI 10.1177/1541931214581044

INTRODUCTION Nuclear Power Plant (NPP) Main Control Rooms (MCR) require multiple members to operate, including Reactor Operators (ROs) and Senior Reactor Operators (SROs). Some typical tasks associated with an RO role include locating controls on NPP MCR panels, physically identifying the control by pointing to the indicator, accurately determining the state of a given control indicator, changing the state of the control when necessary, and openly communicating control status and changes. The role of the SRO is often to monitor the progress of the plant, maintain communication crucial to completing tasks successfully, and provide the ROs with Operating Procedures (OPs) when necessary. Operators need to use clear communication techniques and coordination skills to respond to instructions while navigating through the NPP MCR OPs quickly and accurately to respond to off-normal conditions of the plant in order to maintain public safety and health. The acquisition of techniques and skills required to complete these tasks and objectives come with experience. Therefore, it is important to distinguish experienced participants from experts. Ericsson and Charness (2013), define expert performance as a mastery of all knowledge and skills relevant to the domain. An NPP operator is an expert. Expert performers are required to handle dynamic situations quickly and accurately. Performers attain experience through deliberate practice over long periods (Ericsson & Charness, 2013). Research has investigated effects of deliberate practice on performance in experienced and expert populations, but little is known about its relationship with WL, especially over time. However, for the purposes of studying WL in the NPP domain, it is often inconvenient, expensive, and unpractical to retain an operator population over long periods. Thus using an “experienced” population is beneficial to begin understanding the effects of taskload on perceived WL in NPP task types. For the present experiment, experienced participants are nonoperators with little to no initial knowledge of the domain that acquire some general domain skills and knowledge through

extensive training and practice during the beginning stages of experimentation in order to complete NPP task types. This means that after training the participant should be able to complete some common NPP task types. To investigate specific components in NPP procedures, NPP tasks have been divided into different categories. Previous research and task analyses have determined that NPP tasks include monitoring and detection, situational assessment, response planning, and response implementation (O’Hara, et al. 2008; Reinerman-Jones, Guznov, Mercado, & D’Agostino, 2013). O’Hara and colleagues (2008) group monitoring and detection as similar task types but define them separately. As a result, ReinermanJones, et al. (2013) separate monitoring and detection, stating that monitoring happens when an operator observes the MCR to determine the state of the plant, whereas detection is defined as the recognition and acknowledgement of the change in the state of the plant. For the purposes of this paper, three task types associated with NPP operation are highlighted: checking, detection, and response implementation. However, notice the difference in task types and verbiage. While detection and response implementation remain (as according to Reinerman-Jones, et al., 2013), a new task type is identified by Subject Matter Experts (SMEs) called “checking.” Checking as a discrete, one time, inquiry of a particular control. Though not a regulatory requirement, NPP operators often utilize a particular communication technique called 3way communication. Three-way communication is a method for relaying information and checking for understanding between team members by clearly and simply expressing all components of the communication and confirming instructions. Each step of the procedure consists of two instances of 3-way communication: the instruction and the reply. It is essential for teams to first initiate the instruction or reply, repeat back, and close with a confirmation. An example of 3-way communication is detailed below.

Downloaded from pro.sagepub.com by guest on October 28, 2015


Instruction – SRO: R-O-1, verify R-T-A is open. – RO1: S-R-O, understood verifying R-T-A is open. – SRO: R-O-1, that is correct. Reply – RO1: S-R-O, R-T-A is open. – SRO: R-O-1, understood R-T-A is open. – RO1: S-R-O, that is correct. The intent of three-way communication is to ensure that operators are clearly communicating essential information in this complex domain. These communications procedures are used for each task type, but the instructions are distinct and therefore, might impact overall workload for task types. The complexity of the task types and accuracy required to complete NPP procedures can be demanding. Thus, operators might experience various levels of WL for different task types associated with distinct factors and processing requirements. WL is the result of the perceived experience and the response associated with multiple task demands, often compounded by additional factors such as severe time constraints, task difficulty, task complexity, and information flow required to complete the task (Huey & Wickens, 1993; Reinerman-Jones, Guznov, Tyson & D’Agostino, 2013. For the purposes of this paper, WL is defined as “the operators perceived evaluation and accompanying physiological response to the experience imposed by the task demands rather than a direct reflection of the task demands themselves” (Abich, 2013). Objective and subjective measures have been used to assess WL including electroencephalogram (EEG), electrocardiogram (ECG), and National Aeronautics and Space Administration-Task Load Index (NASA-TLX). Research specific to completing Emergency Operating Procedures (EOPs) in the NPP domain indicated that an increase in global alpha and global theta is apparent when higher levels of taskload are introduced (Moon et al., 2002). Decreases in interbeat-intervals (IBI) and increases in heart rate have been associated with increases in WL (ReinermanJones, et al., 2013). The NASA-TLX has been used to identify various types of taskload associated with the different subscales across many domains (Hart, 2006). The aim for the present experiment was to determine if WL differences exist between task types and sessions for experienced participants in a simulated NPP environment. METHOD Participants Three participants (1 male, 2 females, M = 28.66, SD = 8.96) were compensated with monetary payment equal to the participant’s hourly wage. Participants were required to abstain from ingesting nicotine two hours prior to each experimental session and alcohol, sedatives, and/or tranquilizers 24 hours prior to each experimental session. Participants were also required to have normal or corrected to normal vision. None of the three participants tested as colorblind using the Ishihara color-blind test.

211

Apparatus Simulator. Investigators utilized the desktop based GSE Generic Pressurized Water Reactor (GPWR) NPP simulator. The Simulator was presented using a 6.4GT/s, Intel Xeon™ 5600 standard desktop with a dual monitor setup using 24”, 16:10 aspect ratio monitors. A customized in-house software program called Panel Viewer was used to introduce experimental control to the physics of the simulator with the use of pop up dialogs. Independent Variables Session. For this preliminary experiment, participants’ objective and subjective responses were measured over five individual sessions. For the present experiment, the second, third, fourth, and fifth sessions were investigated due to missing data in the first session for one participant. Task Type. The EOP in the present experiment required participants to complete checking, detection, and response implementation task types. For all task types, the participant located and clicked on the correct control causing a zoomed in pop-up version of the control to appear. In the present experiment, participants were trained to understand checking as a one-time identification by physically clicking on the intended control and checking the status light indicator. Participants were instructed to select the indicator using the mouse cursor in order to simulate the act of pointing at the item of focus. Participants were trained to understand detection as identifying the control, physically clicking on the control to activate the gauge pop-up, and acknowledging gauge level changes by clicking on an “acknowledge” button located under the control. Detection gauges always displayed for five minutes and 60 gauge level changes occurred for each gauge during the five minutes before reaching a reporting level. Participants were instructed that response implementation was the identification, selection, and the completion of the appropriate action needed in order to change the state of the control (open or shut dependent upon the specific step within the required task). Dependent Variables EEG/ECG. Researchers employed the X10 EEG/ECG system (nine EEG channels and one ECG channel) produced by Advance Brain Monitor (ABM). Researchers specifically examined nine channels (C3, Cz, C4, F3, Fz, F4, P3, POz, and P4) at the standard bandwidths of alpha (813Hz), beta (13-30 Hz), and theta (4-8 Hz) using the international standard 10-20 system at a sampling rate of 256Hz (Wilson, 2002). Furthermore, investigators collected hear rate (HR), heart rate variability (HRV), and inter-beat interval (IBI) using the “So and Chan” QRS detection method, which maximizes the amplitude of the R-wave (Chiu, C., Lin, T., Liau, B., 2005; Taylor, Reinerman-Jones, Consenzo, & Nicholson, 2012).



NASA-TLX. The NASA-TLX is a post-task subjective measure of WL consisting of six subscales including mental demand, physical demand, temporal demand, performance, effort, and frustration (Hart & Staveland, 1988). The unweighted version was used for the present experiment and the average of these six sub-scales yielded a global score (Hallbert, Sebok, & Morisseau, 2000). Procedure Participants initially completed seven training sessions that included procedures similar to and actually completed during the experimental sessions. After training, participants completed five sessions of the same experiment. Training and experimentation occurred over a three-month period. The justification for using multiple training sessions and experimental sessions as a basis for investigation is because an operator completes an extensive amount of simulated training procedures prior to starting his/her job at any plant and he/she is required to complete periodic requalification tests. The intention was to simulate a simplified version of this process. EEG and ECG sensors were attached to measure participants’ physiological responses during the experimental session. Participants then completed a short training scenario followed by a five minute resting baseline. Each experimental session required the participant to complete steps derived from a combination of EOPs and subject matter expert inputs that consisted of checking, detection, and response implementation task types. Each participant performed the role of an RO, while the researcher acted as the SRO to distribute instructions via 3-way communication. The three task types were partially counterbalanced to maintain realism. In an actual NPP EOP, checking always precedes response implementation, but detection can occur at any point. Thus, the task type combinations possible included (1) checking, response implementation, and detection, (2) checking, detection, and response implementation, and (3) detection, checking, response implementation. Additionally, investigators modified NPP MCR panels and names of task relevant instrumentation and controls. Experimenters consulted with a SME on the panel and control simplifications, step order modifications, and the realism of the steps in order to determine an appropriate adaptation for participants. Investigators implemented these adjustments in order to ensure that participants could practice and master the modified steps and gather sufficient experience within a three-month period (see Reinerman-Jones, Guznov, Mercado, & D’Agostino, 2013 for a detailed description of panel and control modifications methodology).

212

sphericity assumption was not met. No significant main effects were found for the ECG, EEG, or NASA-TLX for the Session variable. Additionally, no significant interaction effect was found between Session and Task Type. The following results were found for the Task Type variable only. Little research exists in the public arena for the NPP domain that used physiological measures. Therefore, as an exploratory effort, EEG measures of alpha, beta, and theta were collected for each participant from nine individual sensor sites (C3, Cz, C4, F3, Fz, F4, P3, POz, P4; see Figure 1.), Significant main effects in the alpha band were found for sensor site C3 (F(2, 16) = 3.703, p=.048, ɳ2= .316), Fz (F(2, 16) = 5.945, p=.012, ɳ2= .426), F4 (F(1.26, 10.092) = 5.387, p=.037, ɳ2= .402), and P3 (F(2, 16) = 7.221, p=.006, ɳ2= .474). In each of the cases, the checking and response implementation task types resulted in the greatest decreases from baseline for alpha. Significant main effects in the theta band were found for the sensor site C4 (F(1.227, 9.815) = 4.779, p=.049, ɳ2= .374), Fz (F(2, 16) = 7.332, p=.005, ɳ2= .478), F4 (F(2, 16) = 6.788, p=.007, ɳ2= .459), and P3 (F(2, 16) = 11.521, p=.001, ɳ2= .590). For the P3 and Fz sensors, the detection task types resulted in the greatest increases from baseline for theta. For the F4 sensor, the checking and response implementation task types resulted in the greatest decreases from baseline for theta (Figure 2). No significant main effects were found for the beta band.

Figure 1: Nine sensor node positions

RESULTS A 3 (Task Type) x 4 (Session) repeated measures ANOVA was conducted to evaluate participants’ physiological mean differences from baseline and subjective responses during the task types and sessions. GreenhouseGeisser corrections were applied to those cases in which the

Figure 2: EEG Mean Difference from Baseline Changes. ECG shows significant main effects were found for HR (F(2, 16) = 4.444, p=.029, ɳ2= .357), and IBI (F(2, 16) =5.807, p=.013, ɳ2= .421). The greatest increase from baseline for IBI



was found during the detection task type, while the checking task type resulted in the greatest increase from baseline for HR (Figures 3 and 4). No significant main effects were found for HRV.

Figure 3: HR Mean Difference from Baseline Changes

Figure 4: IBI Mean Differences from Baseline Changes The NASA-TLX showed a significant main effect was found for the physical demand subscale, F(1.12, 8.98) = 5.486, p=.041, ɳ2= .407, such that detection yielded the highest perceived ratings (Figure 5).

Figure 5: NASA-TLX Physical Demand Mean Subjective Scores DISCUSSION

213

The results from the EEG data showed significant main effects in the alpha analysis in sensor sites C3, Fz, F4, and P3. Alpha levels decreased from baseline during each task type across all sensor sites, with the checking and response implementation task types resulting in the greatest decreases from baseline. Additionally, theta analysis for EEG showed significant main effects in sensor sites C4, Fz, F4, and P3. Theta from the P3 and Fz sensors resulted in decreases from baseline for the checking and response implementation task types, and increases from baseline for the detection task type. For the C4 sensor, all three task types demonstrated increases in theta from baseline, with the checking and response implementation task types resulting in the greatest increases. Previous research indicates that alpha decreases and theta increases with increases in task load (Boytsova, & Danko, 2010). Therefore, the detection task type shows the strongest pattern for increased workload, but the checking and response implementation task types are not without workload. Instead the differences reflect the nature of each task type. The areas of the brain showing alpha changes are associated with higher order processing, decision making, and verbal tasks. The checking and response implementation task types do require an immediate decision delivered by a verbal response, whereas the detection task type necessitates multiple points for a decision via mouse click followed by a verbal response. Therefore, the detection task type seems to require more resources to execute. The C4 site is over the motor cortex in the right hemisphere and theta findings indicate that physical demand was present for all three visual task types, but was the most taxing for the detection task type, which required multiple mouse clicks over four 5-min steps. Theta from the P3 and Fz sensors resulted in decreases from baseline for the checking and response implementation task types, and increases from baseline for the detection task type. Although increases in theta have been linked to increases in cognitive processing, previous research has associated increases in theta with focused attention, particularly in the frontal regions of the brain (Takahashi, Murata, Hamada, Omori, Kosaka, Kikuchi, et al., 2005). This finding is consistent with the nature of the detection task type in which the participant completes four 5-min steps acknowledging indicator level changes. The detection task type lasted for a minimum of 20 minutes versus checking and response implementation, which usually lasted for approximately five minutes for four steps. Thus, in this context, the detection task type would be expected to elicit similar responses to vigilance tasks compared to checking and response implementation task types. The results from the present study using experienced participants supported Moon et al. (2002) who described similar alpha and theta results during an NPP EOP experimental scenario using the operator population. However, as noted by Moon et al. (2002) these findings are not consistent with much of the previous research outside of the NPP domain. This inconsistency suggests that further research is needed specifically in the NPP domain to understand why these inconsistencies exist.



The results from ECG indicate that greater increases from baseline were found for HR for the checking and response implementation task types compared to the detection task type. Additionally, greater decreases from baseline were found for IBI for the checking and response implementation task types compared to the detection task type. According to previous research, increases in HR and decreases in IBI have been linked to increases in WL (Dussault et al. 2005; Veltman & Gaillard, 1996). These findings are somewhat inconsistent with the EEG results and suggest that checking and response implementation task types elicited higher cognitive demand. However, this difference between the EEG and ECG findings might be explained by the immediate execution and back-toback communications required by the checking and response implementation task types, thus reflecting the body’s sympathetic response to demands, whereas the detection task type would likely resemble that of sustained attention tasks that induce longer term stress effects, not that of the fight-orflight mechanisms. The results of the NASA-TLX show that there was a significant main effect for physical demand for task type indicating that the detection task type elicited the highest perceived physical demand. This is most likely due to the nature of the detection task type requiring more mouse clicks to “acknowledge” gauge level changes (12 changes per minute). Checking and response implementation task types only required a few clicks to activate the pop-up and/or to flip a switch on a valve. Results of both the session and the interaction between session and task types failed to yield any significant effects suggesting that the WL does not change with more experience. Although Ericsson & Charness, (2013) emphasize the importance of long periods of deliberate practice on performance, there has been little research on how deliberate practice effects WL over time. In the present experiment, one level of experience was investigated. Future research should examine physiological differences between novice, experienced, and expert populations and individual differences of each group. Future research should also investigate experienced participant workload and performance over time with more sessions, as four sessions might not be representative of gaining expertise even with extensive training and a simplified environment. The present experiment indicates that while demand is at a constant high for detection, it is still present in checking and response implementation, but different. Future research is necessary to better understand the findings in the present paper, but particularly important is teasing apart when errors might occur. ACKNOWLEDGEMENT

214

represents that its use by such third party would not infringe privately owned rights. The views expressed in this paper are not necessarily those of the U.S. Nuclear Regulatory Commission.

REFERENCES Abich, J. IV. (2013). Investigating the universality and comprehensive ability of measures to assess the state of workload. (Doctoral Dissertation). University of Central Florida. Boytsova, Y. A., & Danko, S. G. (2010). EEG differences between resting states with eyes open and closed in darkness. Human Physiology, 36(3), 367-369. Chiu, C., Lin, T., Liau, B., (2005) Using Correlation Coefficient in ECG Waveform For Arrhythmia Detection. In Biomedical Engineering Applications, Basis & Communications. Dussault, C., Jouanin, J., Phillippe, M., & Guezennec, C. (2005). EEG and ECG changed during simulator operation reflect mental workload and vigilance. Aviation, Space, and Environmental Medicine, 76(4), 344351. Ericsson, K. A., & Charness, N. (1994). Expert Performance. American Psychologist, 49(8), 725-747. Hallbert, B. P., Sebok, A., & Morisseau, D. (2000). A Study of Control Room Staffing Levels for Advance Reactors (No. NUREG- IA-0137) Nuclear Regulatory Commission, Washington D.C. Hart, S. G. (2006). NASA-Task Load Index (NASA-TLX); 20 years later. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 50(9), 904-908. Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P. A. Hancock & N. Meshkati (Eds.), Human Mental Workload (pp. 139-184). North-Holland: Elsevier Science Publishers. Huey, B. & Wickens, C.D. (Eds.). (1993). Workload Transition. Panel on Workload Transition. Wachinton, DC: Notional Academic Press. Moon, B. S., Lee, H. C., Lee, Y. H., Park, J. C., Oh, I. S., & Lee, J. W. (2002). Fuzzy systems to process ECG and EEG signals for quantification of the mental workload. Information Sciences, 142(2002), 23-35. O’Hara, J. M., Higgins, J. C., & Brown, W. S. (2008). Human factors considerations with respect to emerging technology in nuclear power plants. US Nuclear Regulatory Commission, Office of Nuclear Regulatory Research. Reinerman-Jones, L., Guznov, S., Mercado, J., & D’Agostino, A. (2013). Developing Methodology for Experimentation Using a Nuclear Power Plant Simulator. In Foundations of Augmented Cognition (pp. 181-188). Springer Berlin Heidelberg. Reinerman-Jones, L.E., Guznov, S., & Tyson, J. A. (2013). Workload,Situation Awareness, and Teamwork. (No. NUREG-under review) Nuclear Regulatory Commission, Washington D.C. Takahashi, T., Murata, T., Hamada, T., Omori, M., Kosaka, H., Kikuchi, M., et al.. (2005). Changes in EEG and autonomic nervous activity during meditation and their association with personality traits. International Journal of Psychophysiology, 55, 199–207. Taylor, G., Reinerman-Jones, L.E., Consenzo, K., & Nicholson, D. (2010). Integrating multiple sensors to classify operator state in adaptive systems. Proceedings of the 54nd Annual Meeting of the Human Factors and Ergonomics Society (HFES). Veltman, J.A., & Gaillard, A. W. K., (1996). Physiological workload reactions to increased levels of task difficulty. TNO Human Factors Research Institute TNO-Report No. TM-96-A026. Wilson, G. F. (2002). An Analysis of Mental Workload in Pilots During Flight Using Multiple Psychophysiological Measures. The International Journal of Aviation Psychology, 12(1), 3–18.

This report was prepared as an account of work sponsored in part by an agency of the U.S. Government. Neither the U.S. Government nor any agency thereof, nor any of their employees, makes any warranty, expressed or implied, or assumes any legal liability or responsibility for any third party’s use, or the results of such use, of any information, apparatus, product, or process disclosed in this report, or Downloaded from pro.sagepub.com by guest on October 28, 2015