Individual Differences in Dynamic Multitasking ...

Proceedings of the Human Factors and Ergonomics Society 2016 Annual Meeting

1255

Individual Differences in Dynamic Multitasking Performance Kyle A. Bernhardt, Kathryn A. Salomon, F. Richard Ferraro, RaeEllen J. Crockett, Heather K. Terrell, Thomas Petros, and Joseph J. Vacek University of North Dakota

Copyright 2016 by Human Factors and Ergonomics Society. DOI 10.1177/1541931213601292

The current study examined whether individual differences in memory, attention, and visuospatial ability are significant predictors of multitasking ability (multitasking performance baseline level) and multitasking adaptability (the capacity to adapt to dynamic changes in task demands). Participants were administered a neuropsychological battery to measure individual differences in cognitive abilities. Then, participants performed the Multi-Attribute Task Battery-II (MATB) comprising of three workload conditions. Results indicated that participants scoring higher on measures of attention performed better on the MATB during the baseline condition (ability). However, higher scores on measures of delayed memory predicted better performance during more demanding MATB conditions (adaptability), while visuospatial ability predicted worsened performance during more demanding MATB conditions. Additionally, higher global neuropsychological functioning predicted better MATB performance during all conditions.

The capacity to multitask is becoming an important aspect for many operators in the workplace. Complex and information-rich work settings, such as those experienced by pilots and air traffic controllers (Seamster, Redding, Cannon, Ryder, & Purcell, 1993), contain multiple systems which must be attended to in rapid succession. Thus, individuals are required to employ higher order cognitive functions (Deprez et al., 2013; Strayer, Medeiros-Ward, & Watson, 2013) and recruit additional cognitive resources to maintain performance (Wickens, 2008). The performance decrements observed when individuals multitask is a well-studied phenomenon (e.g., Tombu & Jolicoeur, 2004; Watson & Strayer, 2010). However, some individuals have a higher propensity for performing exceedingly well during multitasking situations (Watson & Strayer, 2010). Thus, the evaluation of individual differences (IDs) relating to multitasking performance provides valuable insight into the underlying mechanisms associated with successful multitasking performance. Research has revealed several IDs that contribute to multitasking performance. For example, individuals high in the personality traits of neuroticism and conscientiousness tend to perform worse in automated multitasking environments (Salomon, Ferraro, Petros, Bernhardt, & Rhyner, 2015). Furthermore, cognitive factors such working memory, fluid intelligence, and attention have been shown to predict better multitasking performance (König, Bühner, & Mürling, 2005). An important factor to consider when evaluating multitasking performance is how one copes with variable task demands and associated increases in cognitive workload. Morgan et al. (2013) made a distinction between multitasking ability and multitasking adaptability (MULT-AD). Morgan and colleagues defined multitasking ability as multitasking performance on a task at a baseline level. MULT-AD, however, is the

capacity to adapt to dynamic changes in task demands relative to baseline performance. To clarify, MULT-AD is the potential for individuals to cope with fluctuations in task demands and associated workload. MULT-AD can be observed in the profession of air traffic controlling where controllers must cope with surges in air traffic volume. This often requires controllers to perform beyond routine workload levels. Thus, this type of multitasking performance is considered dynamic and requires adaptation. Currently, little research has been conducted on the IDs that contribute to MULT-AD (Morgan et al., 2013). However, a seminal study by Morgan et al. (2013) provided evidence for multitasking ability and adaptability being two distinct constructs. Utilizing a generalized aviation simulator, Morgan and colleagues demonstrated that the IDs predicting multitasking ability are different from those that predict MULT-AD. Specifically, scholastic aptitude and working memory capacity were the best predictors of multitasking ability. However, after controlling for baseline multitasking ability, spatial manipulation was the only predictor of adaptation performance to changing task conditions. These findings suggest that the tendency for adapting to changes in task demands relies on a different set of cognitive factors than when performing at a static level. The purpose of the current study was to further examine the IDs that contribute to multitasking ability and MULT-AD using a neuropsychological battery to measure IDs in memory, attention, visuospatial ability, and global neuropsychological functioning. These IDs were then used to predict multitasking performance on a computerized aviation simulator consisting of varying workload conditions. From the previous literature, two hypotheses were formed to replicate findings. First, attentional capacity would predict multitasking performance at a baseline level (ability). Second, multitasking

Downloaded from pro.sagepub.com at UNIV OF NORTH DAKOTA on October 10, 2016

Proceedings of the Human Factors and Ergonomics Society 2016 Annual Meeting

performance during higher workload conditions (adaptability) would be predicted only by measures of visuospatial ability (Morgan et al., 2013). Additionally, it was hypothesized that higher global neuropsychological functioning would predict higher multitasking ability and adaptability. This hypothesis stems from the inference that those with higher general cognitive functioning are better able to allocate cognitive resources (Ben-Shakhar & Sheffer, 2001). METHOD Participants A total of 97 undergraduate students participated in this study and received course credit for their participation. Twenty-three participants were excluded due to reported visual abnormalities (n = 12), incomplete measures (n = 6), or extremely high task error rates indicative of not comprehending task instructions (n = 5). After exclusions, a final sample size of 74 participants was obtained (18 men and 56 women) with a mean age of 19.26 (SD = 2.41) years. All participants reported normal-to-corrected 20/20 vision and good health. Materials Individual differences measure. The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; Randolph, 1998) is a neuropsychological battery that evaluates five cognitive dimensions via 12 subtests. These 12 subtests are aggregated into five Index scores representing each dimension (see Randolph, 1998 for descriptions of the Index subtests). A global score is then calculated from the Index scores. For this study, the Immediate Memory Index (IMI), Visuospatial/Constructional Index (VI), Attention Index (AI), Delayed Memory Index (DMI), and global scores were used as predictors. Multitasking apparatus. The Multi-Attribute Task Battery-II (MATB; Santiago-Espada, Myer, Latorella, & Comstock Jr., 2011) was used to evaluate participant multitasking performance. Composed of four simultaneously occurring subtasks (systems monitoring, communications, tracking, and resource management), the MATB is a computerized multitasking simulator that mimics the basic cognitive tasks experienced by pilots. Consult Santiago-Espada et al. (2011) for complete subtask descriptions. The traditional communications task was not used in this study. Instead, the communications task interface was retrofitted to present a listening span task (Daneman & Carpenter, 1980). When presented, participants were instructed to listen to six sets of prerecorded sentences. Each set consisted of four

1256

sentences. After each sentence in the set was presented, participants had to state the validity of the sentence (i.e., if the sentence made logical sense). Then, participants were prompted to recall the last word of each of the four sentences once all four sentences were presented. This task was used to vary MATB simulation demands without compromising tracking task performance. Additionally, the MATB contained the Workload Rating Scale (WRS). The WRS is a subjective measure of workload based on the NASA-TLX (Hart & Staveland, 1988) and computes a composite workload score from 0 (low) to 100 (high) from six subscales (mental demand, physical demand, temporal demand, performance, effort, and frustration). Procedure After providing informed consent, participants completed a brief demographics questionnaire. Participants were then administered the RBANS via paper and pencil method by a trained research assistant. Neuropsychological testing lasted approximately 20min. Then, participants were seated at a color computer monitor with a keyboard and joystick for MATB testing. MATB sequence. Once seated at the computer, a researcher explained each of the MATB subtasks according to a script. Participants then completed a 9min researcher-facilitated practice trial of the MATB. During the practice trial, each subtask was completed individually for 3-min. The researcher provided feedback to ensure that participants understood each subtask. As with Morgan et al. (2013), participants did not engage in all three subtasks simultaneously prior to testing. After the MATB practice trial, participants were administered a practice listening span set. Then, participants completed a 15-min MATB testing session containing three workload conditions at 5-min intervals. The testing session progressed through a low (baseline), medium, and high workload sequence with the WRS presented at the end. The workload conditions were created by manipulating the MATB programming script to increase system monitoring event density, tracking task reticle refresh rate, and resource management pump failures. The listening span task was present only in the high condition. Table 1 displays the specific subtask parameters. MATB scoring. The MATB does not provide an overall performance score. Therefore, the following method was utilized in order to gain a holistic picture of multitasking performance. First, averages for each subtask within the workload levels were computed. Then, each subtask was standardized using z-scores within each workload condition. These z-scores were then weighted equally and summed to create a composite score. In the high workload condition, the listening span

Downloaded from pro.sagepub.com at UNIV OF NORTH DAKOTA on October 10, 2016

Proceedings of the Human Factors and Ergonomics Society 2016 Annual Meeting

task was scored by subtracting the number of word intrusions from the number of correct words recalled. The resulting score was then standardized, weighted equally with the MATB scores in the high condition, and summed. Composite MATB scores were then multiplied by negative one so higher scores indicated better performance. Table 1. MATB Workload Condition Parameters Task Resource Management (pump Workload failures/min) Low 2 Medium 1 Medium 7 High 1-2 High* 7 High 2-3 Note. * Indicates the presence of the listening span task. System Monitoring (events/min)

Tracking (reticle movement)

Pilot Study A pilot study was conducted to ensure workload differed between each MATB condition. Participants (N = 16) completed the conditions in either a low, medium, high sequence (n = 8) or a low, high, medium sequence (n = 8) for 5-min each. The WRS was presented after each condition to measure workload. A 3 (condition) x 2 (order) mixed ANOVA analyzing composite WRS scores with condition as the within-subjects factor and order as the between-subjects factor revealed a significant main effect for condition, F(2, 28) = 54.19, p < .001, η2 = .80. Pairwise comparisons using Bonferroni correction revealed significant differences between all conditions, low (M = 50.79, SD = 12.69), medium (M = 56.76, SD = 13.76), and high (M = 71.57, SD = 12.99). The main effect for order and the order by condition interaction were both nonsignificant, Fs < 1, p > .05 in all cases. RESULTS Data were first screened for outliers using Mahalanobis distances and Cook’s D. A Mahalanobis distance critical value of 18.43 was determined utilizing the table provided by Barnett and Lewis (1978). No cases exceeded this critical value and all Cook’s D values were less than .25. Overall, when tested against the midpoint of the WRS composite scale, participants (M = 63.97, SD = 11.88) found the simulation to be high in workload, t(73) = 10.11, p < .001, d = 1.23. Correlations between the MATB conditions, low-medium (r = .74), low-high (r = .67), and medium-high (r = .80), were all significant (p < .001). Those with higher baseline performance tended to perform better during the more demanding MATB conditions. Multiple regression was then used to predict multitasking ability at the low (baseline) workload

1257

condition from the four RBANS Index scores (IMI, VI, AI, and DMI). The four Index scores were entered into a single block using the enter method. The overall model was significant, F(4, 68) = 2.98, p = .03, R2 = .15, R2adj = .10. The AI was the only significant predictor of performance (see Table 2). Participants that scored higher on the AI tended to perform better during the baseline MATB condition. Table 2. Coefficients for MATB Performance at Baseline B β t p .01+ .06 .48 .63 Immediate Memory Attention .01 .28 2.30* .03 Visuospatial .01+ -.01 -.08 .94 Delayed Memory .02 .14 1.02 .31 Note. *Indicates significance at p < .05. +Indicates values < .01.

Next, two hierarchical multiple regression models were computed to examine MULT-AD in the medium and high workload conditions. Similarly to Morgan et al. (2013), MATB baseline performance was entered into the first block in order to control for multitasking ability. Then, the four RBANS Index scores were entered into the next block. The overall Step 2 model for the medium workload condition was significant, F(5, 68) = 20.19, p < .001, R2 = .60, R2adj = .57. However, the model did not produce a significant change in R2 from the initial hierarchical step (ΔR2 = .05, p = .09). None of the RBANS Index scores were significant predictors (see Table 3). Table 3. Coefficients for Adaptation to the Medium Condition B β t p Step 1 Baseline Performance .80 .74 9.35**