model was designed to prioritize robotic vehicle control over answering SA .... exhibited much more dispersed pattern for the cone collision metric than for the ...
Human-robot interaction modeling and simulation of supervisory control and situational awareness during field experimentation with military manned and unmanned ground vehicles Tony Johnson, Jason Metcalfe, Benjamin Brewster, Christopher Manteuffel, Matthew Jaswa DCS Corporation, 6909 Metro Park Drive, Alexandria, VA, USA 22310 Terrance Tierney US Tank Automotive Research, Development and Engineering Command, ATTN: (AMSRD-TAR-R/MS: 264), 6501 E. 11 Mile Road Warren, MI 48397-5000 ABSTRACT The proliferation of intelligent systems in today’s military demands increased focus on the optimization of human-robot interactions. Traditional studies in this domain involve large-scale field tests that require humans to operate semiautomated systems under varying conditions within military-relevant scenarios. However, provided that adequate constraints are employed, modeling and simulation can be a cost-effective alternative and supplement. The current presentation discusses a simulation effort that was executed in parallel with a field test with Soldiers operating military vehicles in an environment that represented key elements of the true operational context. In this study, “constructive” human operators were designed to represent average Soldiers executing supervisory control over an intelligent ground system. The constructive Soldiers were simulated performing the same tasks as those performed by real Soldiers during a directly analogous field test. Exercising the models in a high-fidelity virtual environment provided predictive results that represented actual performance in certain aspects, such as situational awareness, but diverged in others. These findings largely reflected the quality of modeling assumptions used to design behaviors and the quality of information available on which to articulate principles of operation. Ultimately, predictive analyses partially supported expectations, with deficiencies explicable via Soldier surveys, experimenter observations, and previously-identified knowledge gaps. Keywords: Intelligent Systems, Human-Robot Interaction, Human Performance Models, Modeling and Simulation
1. INTRODUCTION In support of the Tank Automotive Research, Development and Engineering Center (TARDEC) and Army Research Laboratory (ARL) initiative to research, develop, and transition robotic control interfaces, the Robotics Collaboration (RC) Army Technology Objective (ATO) specified an objective to “develop advanced models, metrics and design guidelines for optimal mounted and dismounted Soldier-robotic performance.”1 A significant component of the RC ATO research focused on shared control schemes, in which human operators and automated mobility systems jointly controlled unmanned ground vehicles during a mission conducted in an operationally-relevant environment. Pursuant to its objectives, the RC ATO conceived a suite of driving aids for which prototypes were developed and tested via experimentation or demonstration in the field. The RC Capstone experiment, conducted at Fort Bliss, Texas in 2008, studied one such driving aid, the “steerable waypoint”, that allowed human operators to influence the direction of an Experimental Unmanned Vehicle (XUV) operating under an autonomous mobility system, following a pre-planned route and using Laser Detection and Ranging (LADAR) sensors for obstacle detection and avoidance. This aid allowed the operator to change the target waypoint by using a control device; thus the autonomous mobility system, in the course of continuously re-planning the optimal route to the next waypoint, was influenced by the direction in which the current waypoint was “steered” by the operator. Thus, the objective of the RC Capstone experiment was to evaluate the effect of steerable waypoint on autonomous vehicle control.
As a corollary to this experiment, a predictive analysis was conducted in parallel via modeling and simulation, using a Robotic Technician (Robotech) human performance model that was designed to reflect the expected behaviors of a Soldier participating in the field experiment. The virtual Soldier model was developed and exercised using the RC ATO Intelligent Systems Behavior Simulator (ISBS), developed by TARDEC in part to “drive development of intelligent agents that decrease Soldier workload and reduce and/or automate controlling tasks across mounted and dismounted systems”.1 Herein, the modeling and simulation experiment is described with respect to performance across multiple vehicle control modes and with respect to test results collected from the field experiment using active duty Soldiers.
2. METHODOLOGY Using the RC Capstone research protocol2 to derive requirements, a constructive operator was designed to model the expected behavior of a competent but otherwise nominal test participant. The virtual Soldier was comprised of task networks that modeled behaviors such as controlling a robotic vehicle and maintaining situational awareness. The modeling component of the test occurred via the ISBS, which “provides the ability to define behaviors, allocate behaviors to human or computer (software agent) deployments, and analyze deployment effectiveness within specified scenarios in accordance with a set of metrics”3. In order to collect Human Performance Model (HPM) metrics representative of the RC Capstone field test, the ISBS was interfaced with a high-fidelity simulation environment providing the operational setting. A terrain database representing the Fort Bliss test range was constructed and populated with vehicle obstacles and mission events, providing a virtual environment representative of the field experiment. During the data collection phase multiple missions were run in each test condition, including teleoperation only (TEL), autonomous mobility with teleop intervention (ATL), and autonomous mobility with steerable waypoint intervention (ASW). This practice was consistent with data collection methods used with the Soldiers. Thus, the results of the ISBS data collection formed the basis for a predictive analysis of the RC Capstone field experiment. 2.1 Model design The model design consists of the set of behaviors necessary for the constructive operator to successfully complete the requirements of the ISBS version of the RC Capstone experiment, as well as the human performance modeling principles which guided the model execution and data collection. 2.1.1 Model behavior hierarchy A top-down task decomposition process was applied to the derived requirements to achieve the design for the ISBS Robotech. The constructive operator behaviors were classified hierarchically such that lower level behaviors were reusable. Higher level behaviors composed lower level sequential tasks into processing loops that contained sequential and parallel execution. Soldier behaviors were characterized in the ISBS according to the pyramid illustrated in Figure 1.
Figure 1. RC Capstone Soldier Behavior Hierarchy
2.1.1.1 Operational behaviors Operational behaviors represented the entire scope of a military operation; in other words, all of the Soldier behaviors required to complete the assigned mission. For the RC Capstone experiment, there was one behavior defined at the operational level: Perform XUV Control Experiment. This behavior was comprised entirely of tactical behaviors. 2.1.1.2 Tactical behaviors Behaviors defined at the tactical level were designed such that they could easily be redeployed from one Soldier to another or to an automated intelligent system. The tactical behaviors defined for the RC Capstone experiment were Control Robot and Receive SA Message. These tasks were modeled using aggregations of functional behaviors. 2.1.1.3 Functional behaviors Functional behavior attempted to characterize a single, well-defined action that represented one step of many in a tactical-level task. Functional behaviors could consist of other functional behaviors, but relied ultimately upon elemental behaviors as the core building blocks. The list of functional behaviors defined for RC Capstone includes: Acquire SA Data, Adjust Speed Teleop, Adjust Steering Teleop, Adjust Steering SWP, Answer SA Query, Begin Steerable Waypoint Intervention, Begin Teleop Intervention, Consider SA Factors, Control Robot Steerable Waypoint, Control Robot Teleop, End Steerable Waypoint Intervention, End Teleop Intervention, End Teleop Intervention Head Eyes, Estimate Course Deviation, Find SMI Button, Halt Robot, Listen To SA Query, Monitor Robot, Observe SA Map, Observe View (driving), and Perform SA. 2.1.1.4 Elemental behaviors Elemental behaviors quantified workload and time for a unary task, and were based upon the four modalities (i.e. modal resources) which underpin the VACP workload theory, as described by McCracken & Aldrich, to consist of Visual, Auditory, Cognitive, and Psychomotor modalities4. The workload values defined for each of the elemental tasks used in the virtual operator experiment, listed in Table 1, were derived from scales developed by Bierbaum, Szabo and Aldrich 5. Table 1. Summary of elemental behaviors
Elemental Behavior Consider Level One SA Consider Level Two SA Consider Level Three SA Decide Evaluate Fixate Eyes Locate Align Interpret Semantic Content Move Eyes Locate Align Move Eyes Visually Scan Move Foot Position Move Hand Position Move Head Visual Stimulus Push Button
V 7.0 7.0 7.0 0.0 0.0 5.0 0.0 5.0 7.0 0.0 0.0 5.0 5.4
A 0.0 0.0 0.0 0.0 0.0 0.0 4.9 0.0 0.0 0.0 0.0 0.0 0.0
C 3.7 4.6 7.0 1.2 4.6 3.7 1.0 3.7 4.6 1.0 1.0 3.7 1.0
P 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.6 2.6 0.0 2.2
Recall Memory / Store Memory
0.0
0.0
5.3
0.0
Speak
0.0
0.0
1.0
1.0
Task Time Factors to consider × 480 ms Factors to consider × 620 ms Factors to consider × 620 ms + 1000 ms 70 ms (Factors to consider × 167 ms) + 70 ms 230 ms ((Words / 2.4) × 1000 ms) + 480 ms 100 ms 70 ms lM log2(D/S × 0.5) [Fitt’s law] + 170 ms lM log2(D/S × 0.5) [Fitt’s law] + 170 ms 200 ms (assumes < 30 degrees movement) 400 ms Digits = 33 ms, Colors = 38 ms, Words = 47 ms, Geometric Shapes = 50 ms, Random Forms = 68 ms, Nonsense Syllables = 73 ms, Letters = 40 ms, Dot Patterns = 46 ms, 3D Shapes = 68 ms, Default/nominal value = 70 ms ((Words / 3.4) × 1000 ms) + 550 ms
The formulas used to calculate task execution times were derived from the human performance micromodels specified in Appendix B of the Improved Performance Research Integration Tool (IMPRINT) User’s Guide6, which further references the original research and appropriate sources for each formula.
The micromodels represent four independent neurophysiological channels for which processing time is specified, including perceptual, motor, speech, and cognitive. The established method for elemental behavior specification within ISBS human performance models was to analyze the contributing factors for a behavior and attempt to account for all components of time by summing the tasks times for each channel. An exemplar of this rationale can be found in Table 2. Table 2. HPM specification for the elemental behavior “speak”
Factor Value Visual Workload 0.0 Auditory Workload 0.0 Cognitive Workload 1.0 Psychomotor Workload 1.0 Perceptual Task Time 100 ms Motor Task Time 70 ms Speech Task Time (Words/3.4) ×1000 ms Cognitive Task Time 480 ms
Design Rationale Visual workload scale: No visual workload Auditory workload scale: No auditory workload Cognitive workload scale: Automatic (simple association) Psychomotor workload scale: Speech Speaking involves a basic perceptual process Speaking involves a basic Motor process The task time for speaking depends on the number of words Speaking was deemed the equivalent of a Reaction Name Match in the cognitive micromodel table by estimating that the process of completing an utterance was equivalent to matching words in a person’s vocabulary with the concepts to be communicated.
2.1.2 Human performance modeling considerations In determining the extent to which tasks could execute in parallel, the ISBS adopted the paradigm used for the prediction of workload in ARL modeling tools7; namely Wickens’ Multiple Resource Theory (MRT), which posits that the performance of concurrent, multiple tasks by humans is resource-constrained, leading to task interference and performance degradation8,9. The discrete event simulation engine used for executing the ISBS models, the Command Control, and Communications Human Performance Model (C3HPM)10, performed real-time workload status verification to prevent the virtual Soldier from accepting additional tasks when doing so would cause overload; i.e., an excess of the maximum allowable workload in any modality. The upper limit for workload was determined to be 7.0 for each modal resource based on the VACP scales. Consequently, execution of the constructive operator exhibited both task interference and task degradation, by virtue of deferring tasks (i.e., processor time) which would otherwise place the operator in an overload state. Additionally, each task was assigned a priority. Tasks with higher priority were allocated before tasks with lower priority; however waiting tasks of equal priority were selected on the basis of time spent in the queue. The Robotech model was designed to prioritize robotic vehicle control over answering SA queries during complex maneuvers. When not performing a maneuver, performing SA had priority over driving. Finally, it is important to note that the ISBS modeling and simulation environment was designed to be deterministic, in that behaviors were intended to react to environmental stimuli in a consistent manner. There were no probabilistic components within this model design. Any variation within ISBS mission results represent a by-product of the real-time distributed simulation, in which the virtual operator interacted with the virtual environment via control theory guidelines. 2.2 Modeling and simulation environmental testbed The environmental testbed used for the construction and execution of the virtual Robotech model was comprised of several distributed platforms and spanned multiple engineering phases, including model and simulation development, model translation, and model execution. Figure 2 illustrates the system components and their function within each phase of the experimental process.
Figure 2. RC Capstone modeling and simulation environmental testbed
2.2.1 Modeling and simulation development The Robotech model design was specified using the Unified Modeling Language (UML) and the behaviors within the hierarchy were implemented via a customized syntax developed for the C3HPM known as the Mission Thread Intermediate Language (MTIL). The language rules which defined MTIL were analogous to the constructs available in the ontology editor previously used in ISBS model devleopment3, thus providing an alternative implementation scheme. The scenario was generated using a single test course with six designated reconnaissance areas (only one of which would be used per mission). The a priori route was developed using the route planning component of the Warfighter Machine interface (WMI). The OneSAF Testbed Baseline (OTB) was used to establish event zone boundaries in order to facilitate the analysis of operator performance with respect to a single obstacle. Trip lines were placed across the vehicle route to mark the entry and exit points for planned maneuvers. The test course required navigation of the following eleven obstacles during each mission run; cone narrows, cone slalom, hairpin turn, minefield bypass, barrel narrows, blocked path, obstructed gate, 90-degree turn, barrel slalom, buttonhook, and one named are of interest reconnaissance. The OTB was also used to establish trip lines that, when crossed by the vehicle, prompted SA queries within the Event Server. An important consideration in designing the predictive analysis was replicating the test environment that Soldiers would experience in the field; optimally, the simulation terrain database would match, as closely as possible, the actual terrain. In constructing the terrain database, elevation data for the test range (Ft. Bliss, TX) was obtained, followed by the procurement of texture source data from Global Mapper that was used to register and geo-reference the data. Finally, a single database was then generated using Terra Vista software. 2.2.2 Modeling translation and code generation The MTIL source file was used in conjunction with a custom plug-in application within the Protégé Ontology Editor to generate an ontology representing the behaviors specified in the original UML model. Using the same plug-in, the ontology was translated to C++ source code and a task network. The executable file generated from compiling the autogenerated code, when combined with the task network, comprised the C3HPM component of the ISBS. 2.2.3 Modeling execution and data collection The simulation was executed via the real-time interaction of several distributed components. The Embedded Simulation System (ESS) modeled the vehicle dynamics and the terrain, and provided vehicle state information to other simulation components. The OTB modeled obstacles (cones and barrels) on the route and provided their locations to the rest of the simulation. The Event Server monitored the vehicle location to notify other simulation components when the vehicle entered or exited a mission event zone and whenever a situational awareness query was posed to the operator. The simulation environment was rendered in real-time via the Image Generation component, with both first-person and overhead views available, allowing the researchers to observe the vehicle’s progress during the test. Thus, the simulation environment provided the stimuli for the virtual operator and a real-time experiment oversight capability. The C3HPM modeled the Robotech controlling a vehicle following a predetermined route while periodically probed for situational awareness and understanding. In fulfilling the Control Robot tactical behavior, the Robotech model reacted to changes in the vehicle’s state (via supervisory control), and made steering, accelerator and brake adjustments as necessary to maneuver the obstacles within the experimental test course. The Receive SA Message tactical behavior required the Robotech model to “listen” for SA queries, and then invoke all appropriate behaviors necessary to determine the correct answers and deliver them by way of a text-to-speech interface. The C3HPM interacted with the WMI to control the vehicle, simulating button presses to accomplish operator control tasks such as switching between the different levels of autonomy. When the vehicle was operated in autonomous or steerable waypoint mode, the Auto Mobility System (AMS) used terrain and obstacle data to navigate the vehicle around obstacles it could detect such as hills and barrels, while attempting to follow the a priori route. During data collection the mission logs were extracted from the ISBS and the WMI. The ISBS recorded all relevant data relating to the state of the vehicle and the Robotech, and the WMI recorded all button presses received from the C3HPM. From these data, performance measures and workload measures were derived in the data reduction phase.
3. RESULTS Following the RC Capstone field experiment data collection and reduction phases, a qualitative comparison of the Soldier data from the RC Capstone field experiment and the HPM data from the predictive analysis was conducted to examine the areas in which the cognitive modeling performed within expectations and where there were deviations. A relative comparison of operator workload was also performed, using Soldier surveys and the HPM workload results. 3.1 Performance data The performance metrics included for the qualitative analysis were selected based on the assumption that the deterministic behaviors which comprised the Robotech HPM could yield a predictable result when compared to the Soldier behaviors. For example, the Soldiers did not correctly answer all situational awareness queries across all missions; this was expected, yet predicting how many answers (or, which specific answers) would be incorrect would not have been possible without introducing an element of probability into the decision logic of the modeled behaviors. Since this conflicted with the modeling paradigm, it was determined that all SA queries would be answered correctly, yet the goal was to account for all of the time necessary to derive the answers from crew station displays. For the selected measures, the method of qualitative comparison was to derive the effect size by normalizing the difference between the Soldier data mean and the HPM data mean against the standard deviation which represented the Soldier population. This process is denoted mathematically in Equation 1, with the result called “∆ STD DEV”.
Equation 1. Calculation of Qualitative Analysis Effect Size This representation sought to account for the variable range of performance among the Soldier population, which was comprised of 13 active-duty Soldiers from the 2nd Combined Arms Battalion 5th Brigade (Army Experiment Task Force). This method also normalized variance due to the different experiment measures. For example, the Soldier data exhibited much more dispersed pattern for the cone collision metric than for the mean vehicle speed metric (coefficient of variance of 1.22 versus 0.148, respectively). It is also worth noting that Equation 1 maintained a signed magnitude, and thus represented whether the HPM underestimated, or overestimated the expected value of each performance metric. An additional level of comparison was to verify whether or not the HPM mean was within the Soldier performance range, or completely outside the observed Minimum-Maximum values for a specific measure from the field experiment. Thus, in identifying appropriate candidate experiment metrics and calculating effect size for each one, the qualitative analysis yielded the results listed in Table 3. The objective for the performance of the HPM was to be within the Soldier range of performance and less than one standard deviation of the Soldier mean for a given measure, with significant deviations explicable (and potentially correctable) via data analysis and experimental observation. Overall, the predictive analysis conducted via the Robotech HPM was within the Soldier range of performance for 75.8% of the compared measures, and within one standard deviation for 48.3% of the measures. When analyzed by experiment condition, these data indicate that performance indexing semi-autonomous control behaviors were more difficult to predict, with steerable waypoint being the least predictable; only 54.5% of the Robotech performance measure means fell within the Soldier range of performance. Essentially, the instances in which HPM mean values fell outside the Soldier range of performance indicate that the model did not match the soldier behaviors as well as expected, even allowing for individual differences among the 13 Soldiers in the field experiment. Some of the differences in results exhibited a systemic effect, in which modeling decisions for a single behavior affected multiple metrics. For example, the mean speed of the vehicle in the predictive analysis, at approximately 13 kilometers per hour, was significantly higher than the 10 kilometers per hour observed in the field experiment. Thus, the modeling decision for assigned vehicle speed affected the ∆ STD DEV for several measures, including mean vehicle speed, obstacle performance (i.e., how many maneuvers successfully navigated in the intended manner), cone and barrel collisions, and the rate of overlap between vehicle obstacles and SA queries. In theory, reducing the assigned vehicle speed to match the field experiment mean values would have facilitated a convergence in the previously stated metrics.
Table 3. Results of qualitative analysis, comparing Soldier data with HPM data
The Robotech HPM performed particularly well in approximating the overall route deviation in all test conditions. This metric captured the mean difference between the actual vehicle route extracted during data collection, and an “ideal run”, that represented a path that would successfully navigate the course as correctly, and efficiently, as possible. Thus, the close correlation between Soldier results and ISBS results for the route deviation metric revealed a detailed understanding of the conditions under which when the operator would intervene to control the vehicle, and more specifically, when the operator would deviate from the a priori route. Conversely, the Robotech HPM suffered a much higher rate of collisions with barrels than Soldiers experienced in the field test, when considering the conditions involving the autonomous mobility system. Part of this owes to the fact that the actual XUV tended to avoid attempting maneuvers altogether if a barrel collision appeared to be imminent, thus eliminating the possibility of a collision. There were two obstacles, the barrel narrows and the buttonhook, for which this happened regularly. The Robotech HPM, however, always elected to attempt the obstacles, and thus “clipped” some of the barrels in doing so. A more robust behavior model might have included an option to skip a maneuver if the projected obstacle clearance did not meet a sufficient threshold. Overall, the median value for ∆ STD DEV when considering all experiment measures listed in Table 3 was 0.918. This meta-statistic indicates that the initial objective of attempting to achieve behavioral modeling results that fell within one standard deviation of the data collected from the Soldiers (i.e., approximately 68% of the test population), was successful, yet left room for improvement in some areas.
3.2 Workload data The VACP workload data collected for the Robotech HPM were analyzed at the mission level as well as at the mission event level. When considering aggregate workload, it was possible to compare the results from the predictive analysis with Soldier survey data to identify areas of correlation or disagreement. For the event-level workload data, a case study was developed to examine VACP changes based on specific stimuli. 3.2.1 Mission summary workload The HPM workload summary depicted in Figure 3A reveals multiple benefits of automation. In the TEL condition, the psychomotor (P) workload was significantly higher than that observed for the two conditions that featured purely autonomous mobility for some segments of the mission. This was due largely to the fact that the scenario featured only 23% of the overall mission time in obstacle zones, with the remaining 77% of the time spent in transit between event zones. The transit areas were easily navigated by the AMS, requiring no manual input from the operator and thus, low psychomotor workload.
Figure 3. HPM measures, including A) overall mission workload and B) multi-tasking summary
Similarly, the demands of manual driving increased workload in the cognitive modality due to the frequent need to adjust the steering and speed actuators. Thus the TEL condition yielded the highest total workload (i.e., sum of the VACP workload values) of all of three conditions. This conclusion is further supported by the task execution summary (see Figure 3B), which indicates the mean number of tasks executing in parallel during the mission. Vehicle control through pure teleoperation required, on average, more than two tasks to be executing in parallel at all times. By comparison, the conditions that utilized the AMS showed a substantial decrease in the number of tasks required to accomplish the same mission. Within the semi-autonomous conditions, the ATL condition performed slightly better than ASW condition. Workload across all modalities were slightly lower in the ATL condition, and the average number of tasks executing in parallel was slightly less, although very close to that produced in the ASW condition. The workload differences for these two schemes were attributable to the model design as well as the expected frequency of operator interventions. Since the vehicle had to be stopped in order to transition from AMS control to teleoperation
control in the ATL condition, the anticipated usage of manual driving was for negotiating the scripted maneuvers only and not for minor course corrections. Since the steerable waypoint driving aid could be engaged “on the fly”, the HPM included a behavior to intervene if the vehicle drifted off course. Therefore, the vehicle control tasks for the steerable waypoint were executed more often, resulting in higher workload. The effect on performance was predictable in that the mean route deviation for the ATL condition was 3.212 meters, and for the ASW condition it was 2.821 meters. A similar pattern was observed in the Soldier workload survey data (see Figure 4), in that the TEL condition required more mental and physical workload than were required in the ATL condition; i.e. the addition of the AMS provided some measure of workload relief. However, the steerable waypoint seemed to add workload in some cases.
Figure 4. Soldier subjective workload
According to the usability survey, the Soldiers responded positively to the steerable waypoint concept, yet the implementation that was tested in the RC Capstone field experiment was not regarded as a useful technology. This is confirmed in the Soldier workload survey in the areas of mental, physical, and temporal workload, and overall effort required. The last finding is consistent with the Robotech HPM workload values, yet the enhanced route conformance present in the HPM results was not observed in the field test, with the Soldiers averaging a route deviation of 2.895 meters in the ATL condition, and 5.720 meters in the ASW condition. 3.2.2 Mission event workload case study: the cone slalom The cone slalom measured the ability of the participant to maneuver the XUV in a weaving pattern through a series of cones that were spaced 15 meters apart, as represented in Figure 5.
Figure 5. Cone Slalom Mission Event
The Robotech was solely responsible for obstacle detection and avoidance when negotiating this obstacle since the cones were too short for the XUV LADAR to sense, and the a priori route would otherwise guide the robot through the middle of the slalom when navigating. Thus, in all test conditions the AMS did not assist with obstacle detection for this event. The differences in operator workload revealed in the predictive analysis case study again point to a reduction in required resources when the Robotech operated in the one of the semi-autonomous control conditions. Figure 6 depicts a comparison of the VACP workload for the cone slalom and the surrounding inter-zone transit areas.
Figure 6. Workload for the cone slalom maneuver in A) TEL condition, B) ATL condition, and C) AWS condition
The psychomotor value of 2.6 was an indicator of how frequently the operator adjusted the vehicle speed and steering when responsible for manual control of the vehicle. Psychomotor values of 2.2 reflected WMI button presses related to the manual component of a semi-autonomous control mode in the ATL and ASW conditions, marking the beginning or ending of an intervention. Overall, there is an observable reduction in the amount of psychomotor workload required to control the vehicle as it approaches the cone slalom, yet in all conditions the operator maintained full control, or shared control of the vehicle throughout the slalom. The reduction in manual vehicle control required in the transit areas between events also impacted the cognitive workload, showing a marked decrease in the cognitive resources required. However, the visual workload required for all test conditions remain consistently high. In the TEL condition, there are periodic occurrences of visual workload of 5.0 that reflect the constructive operator behavior of checking the vehicle gauges on the WMI. While this behavior was far less necessary in the ASW condition since the operator could not control the vehicle speed, there were several invocations of the Push Button elemental behavior, necessary to enable and disable the steerable waypoint driving aid. This behavior held a 5.4 visual workload. Thus, the ATL condition produced the smallest average visual workload of the three conditions since interventions were less frequent, therefore requiring less time scanning the WMI to read gauges or to change the vehicle control mode. The cone slalom workload case study thus supports the overall findings of the aggregate workload analysis, yet also facilitates a mapping of VACP workload values to the HPM design at selected times during the mission.
4. CONCLUSION The modeling effort described in the current paper was conceived as a predictive analysis aimed at constructing and exercising a Human Performance Model to assess the effect of a shared control mobility scheme on Human-Robot Interactions (HRI). In the main, the present study was successful in capturing some of the essential elements driving performance differences between the conditions used in the field experiment. Further, insights into the dynamic aspects of workload were achieved as evidenced in a case study. Specifically, the workload analysis indicated that the use of an autonomous mobility system yielded a clear reduction in operator workload associated with the cognitive and psychomotor modalities when compared to full-time teleoperation of an unmanned vehicle operating in a test environment that was designed to exploit AMS limitations. Additionally, there was an overall reduction in the amount of multi-tasking required with the AMS, although some of the experiment measures indicated slight performance degradation when compared to the teleoperation condition. These findings suggest that a Soldier sharing unmanned vehicle control responsibility with an AMS may benefit from reduced workload and tasking in terms of potentially being able to fulfill additional roles. The qualitative comparison of HPM data with Soldier data revealed substantial difficulty in modeling Soldier-robot interactions in the shared control scheme since it was difficult to anticipate how proficiently an operator would utilize the steerable waypoint. Soldiers seemed to have difficulty adapting to the vehicle navigation effects of combing the operator-controlled steerable waypoint distance and direction with the AMS calculated path, and this factor was not modeled in behaviors contributing to the predictive analysis. As such, we conclude that there are significant benefits of automation to unmanned vehicle navigation, although the specifics of the implementation are critical to user acceptance of the technology. Furthermore, these concepts are conducive for study using detailed Human Performance Models, given sufficient simulation support. Thus, field test capabilities can be complimented, or augmented through modeling and simulation.
REFERENCES [1] Department of the Army (Editor), [U.S. Army Weapons Systems 2009], Skyhorse Publishing Inc., New York, NY, 273 (2008). [2] U.S. Army Research Laboratory Human Research and Engineering Directorate, “Research Protocol: Robotics Collaboration Capstone Experiment: Unmanned System Mobility and Soldier-Robot Teaming”, Unpublished human use protocol, 3-6 (2008). [3] Johnson, T., Manteuffel, C., Brewster, B., and Tierney, T., “Integration of an Intelligent Systems Behavior Simulator and a Scalable Soldier-Machine Interface”, Proc. SPIE 6561, 3 (2007). [4] McCracken, J.H., & Aldrich, T.B., “Analyses of Selected LHX Mission Functions: Implications for Operator Workload and System Automation Goals.” Technical Note ASI479-024-84, Fort Rucker, AL: U.S. Army Research Institute Aviation Research and Development Activity, 4-8 (1984). [5] Bierbaum, C.R., Szabo, S.M., & Aldrich, T.B. “Task analysis of the UH-60 mission and decision rules for developing a UH-60 workload prediction model, Volume I: Summary Report”, AD-A210 763, Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences, 15-16 (1989). [6] US Army Research Laboratory Human Research and Engineering Directorate, “Improved Performance Research Integration Tool (IMPRINT) User’s Guide, Version 7”, Micro Analysis and Design, Inc., 279-282 (2005). [7] Mitchell, D., “Mental Workload and ARL Workload Modeling Tools.” Technical Note ARL-TN-161, Aberdeen Proving Ground, MD: U.S. Army Research Laboratory Human Research and Engineering Directorate, 2-3 (2000). [8] Horry, W., and Wickens, C., “Multiple resource modeling of task interference in vehicle control, hazard awareness and in-vehicle task performance”, Proceedings of the Second International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design, 1-5 (2003). [9] Wickens, C.D., “Multiple resources and performance prediction”, Theoretical Issues in Ergonomics Science 3, 159177 (2002). [10] National Research Council of the National Academies, [Behavior Modeling and Simulation: From Individuals to Societies], National Academies Press, Washington, DC, 62 (2008).
