Incidents of motion sickness for indirect vision driving experiments .... symptoms such as pallor, drowsiness, salivation, sweating, nausea and finally vomiting.
Driving Performance of the Vetronics Technology Test-bed (VTT) Vehicle Christopher C. Smyth MAJ Dennis Gaare James W. Gombash Christopher C. Stachowiak Army Research Laboratory Human Research & Engineering Directorate
ABSTRACT In June of 2001, Human Factors Engineers from the Army Research Laboratory, Human Research and Engineering Directorate (ARL-HRED) assessed the driving performance of the Vetronics Technology Testbed (VTT). The VTT is an advanced technology demonstrator of crew automation technology for armored ground vehicles that is being developed by the U.S. Army Tank Automotive Research Development & Engineering Center (TARDEC). The VTT is a modified M2 Bradley Fighting Vehicle (BFV) chassis driven with an indirect vision system and yoke and foot pedal controls. The vision system consists of forward looking, side, and rear vehicle mounted camera arrays and scene displays mounted in two identical crew stations within the hull of the vehicle. A safety driver was retained in the BFV’s driver’s position. The assessment of the VTT was conducted at the Camp Grayling Military Reservation, Michigan, with a series of experiments using visual tasks and maneuvers that subjected the system to a variety of military driving conditions. In this study, seven military participants negotiated close-in obstacles at low speeds, interacted with ground guides, conducted operations on unimproved roads and drove on cross-country trails, the latter with daytime and FLIR cameras. The VTT allowed the driver to perform most driving tasks without physical difficulty or excessive stress. The visual acuity with the indirect vision system was less than direct vision but likely did not significantly impact performance. The speed at which most tasks were accomplished was much slower than would be expected with a crew operating a standard BFV in the open hatch mode. The accuracy of maneuvering around close in obstacles was also less than what would be expected. Under the current configuration, the system may only achieve a performance level comparable to a standard BFV operating in a limited mode such as at night, buttoned up, or while the crew is in protective masks. These are the opinions of a Subject Matter Expert, since no comparison experiments were conducted with a BFV. The experiments provide descriptive statistics on the performance of the VTT in representative situations; however, the results are limited in significance because of the small sample size. This was true for all but the crosscountry trials where driving with the daytime cameras was compared to that with the FLIR system. Here, driving was significantly faster with the daytime cameras although operationally the difference was slight. Although some participants reported symptoms of motion sickness, still sickness was not an apparent problem under the restricted experimental conditions. Establishment of a performance baseline and further experimentation under increased levels of workload and vehicle performance is recommended.
2
ACKNOWLEDGEMENTS The authors would like to express appreciation for the support and interest in this experiment by Mr. Bruce Brendle of the Vetronics Technology Area at the U.S. Army Tank Automotive Research, Development and Engineering Center (TARDEC) in Warren, MI. Our thanks to the various members of Bruce’s group, who supported this project including Mr. Brian Novak of the Applications Team and Mr. Joseph Warner who served as safety driver. Our thanks to Mr. Chris Geeding of the Ft. Knox BattleLab who helped manage the experiment and served as ground guide. Also, our thanks to the engineers and programmers from General Dynamics Land Systems and DCS Corporation who performed the task of maintaining the VTT during the experiments. Essential to the success were the soldiers who volunteered as participants from the Michigan National Guard and US Army Reserve based in Minnesota. Our thanks to Ms. Melissa J. Karjala of TARDEC for coordinating troop and range support. Mr. Don Sama of TARDEC was kind enough to provide photographs of this operation for this report. Again, this experiment would not have been possible without the support of our technical personnel, Nicky Keenan and Dennis Hash for fixture construction and assembly area integration.
3
TABLE OF CONTENTS
Abstract Acknowledgments Background Indirect vision systems Camera field-of view Display selection Optimal crew performance Motion sickness Emotional stress Objective Methodology Equipment Participants Driving courses Questionnaires Experimental design Procedure Experimental Studies Close in maneuver assembly area Visual acuity for sign reading Judging clearance Parking and backing Tactical road march operations Convoy following Obstacle avoidance Tactical road following Questionnaires Mental workload Exit evaluations Discussion Operational performance Technical evaluations Mental workload and stress Motion sickness comparison Conclusions Recommendations References Appendix A. Volunteer Agreement Affidavit Appendix B. Mental Workload Questionnaires Appendix C. Exit Evaluation
4
Tables 1. Participants’ Demographics 2. Descriptive statistics for visual acuity readings 3. Descriptive statistics for the number of hand signals 4. Descriptive statistics for guide times (minutes) 5. Descriptive statistics for the separation distances (meters) 6. Descriptive statistics for the road speed (km/hr) 7. Descriptive statistics for the course times (minutes) 8. Equivalent road course speed (km/hr) 9. Frequency distributions of scores for allotting task attention 10. Frequency distributions of scores for NASA task load index 11. Frequency distributions of situational awareness rating scores 12. Frequency distributions of cognitive compatibility component scores 13. Frequency distributions of motion sickness symptoms and total severity 14. Incidents of motion sickness for indirect vision driving experiments Figures 1. Crew Integration and Automation Testbed (CAT) Advanced Technology Demonstrator (ATD) concept vehicle. 2. Experimental crew station displays and controls 3. Top view of visual acuity test site 4. Box-plots for visual acuity readings 5. Clearance test 6. VTT entering clearance test site 7. Side view of camera deployment 8. Top view of camera deployment 9. Visibility plots 10. VTT positioned at post set # 6. 11. Assembly test area dimensions 12. Assembly area maneuver obstacles 13. Assembly area maneuvers 14. VTT parked with ground guide 15. Assembly area “B” maneuvers 16. Hand signals at guide stations 17. Times at guide stations 18. Vehicle following test course 19. VTT following lead vehicle 20. Separation distance 21. Average road speed 22. Road obstacle negotiation test 23. Speed of travel 24. Cross country driving course 25. VTT on tactical endurance course 26. Course travel times 27. Subjective stress scores for assembly area and road operation study phases 28. Visual task attention loading scores for assembly area and road operation study phases 29. Cognitive task attention loading scores for assembly area and road operation study phases 30. Auditory task attention loading scores for assembly area and road operation study phases 31. Psychomotor task attention loading scores for assembly area and road operation study phases 32. Task load index demand and interaction component scores for assembly area and road operation study phases 33. Task load index dimension scores for assembly area and road operation study phases 34. Situational awareness rating component scores for assembly area and road operation study phases 35. Situational awareness rating dimension scores for assembly area and road operation study phases 36. Cognitive Compatibility component scores for assembly area and road operation study phases 37. Motion sickness symptom scores for assembly area and road operation study phases 38. Factorial component plot in rotated space of mental workload measures 39. Exit questionnaire responses
5
BACKGROUND The Army needs combat vehicles that are smaller, lighter, more lethal, survivable, and more mobile to support a rapidly deployable force. Combined with the need to assimilate and distribute more information to, from, and within the vehicle as the Army moves toward a digital battlefield, there is the need for an increase in vehicle and C4I systems integration and performance. Consequently, the Army will need sophisticated, highly integrated crew stations for these future combat vehicles. In support of this effort, the Tank Automotive Research Development and Engineering Center (TARDEC) is developing the Crew Integration and Automation Test-bed (CAT) Advanced Technology Demonstrator (ATD). The purpose of the CAT ATD is to demonstrate crew interfaces, automation, and integration technologies that are required to operate and support future combat vehicles. In the present design, the crew consists of two members each of which uses a console with five flat panel LCD displays showing the outdoor scene, a digital map display, and communications and system status displays. For driving, the crew uses an indirect vision system consisting of a vehicle mounted camera array and the scene displays. The U. S. Army Research Laboratory (ARL) is providing Human Factors expertise in determining the effect of these new crew station technologies on system performance through a continuing series of studies and investigations. As part of this effort, ARL in June 2001, participated with TARDEC in a demonstration of the Vetronics Technology Testbed (VTT) vehicle, an initial version of the ATD in a modified M2 Bradley chassis, at the Camp Grayling Military Reservation, Michigan. During the demonstration, researchers from HRED conducted several experiments on the soldier performance with the driving system, as well as collected subjective data on the stress, workload, motion sickness, and situational awareness from questionnaires. In this study, the crew was individually trained and experimentally studied by HRED on their skills in tactical assembly, road march, and day and night tactical driving. Following the measurement of the visual resolution of the vision system, HRED studied the ground guide interactions, precision driving (parking and backing), and judging clearances in the assembly area operations. In the road march segment, vehicle following and roadway obstacle avoidance were studied. This was followed by tactical operations during which navigating a tree-covered route across country in day and night was studied with both daytime and FLIR camera systems. Here, we report on the experimental methodology and the statistical results. However, first we review some human factors issues of interest to the study. Indirect Vision Systems To satisfy Army requirements for reduced gross weight and lower silhouette, as well as the need for increased crew protection against both ballistic and directed energy threats, designers of future armored combat vehicles will place the crew stations deep within the hull of the vehicle. For protection against direct and indirect fire as well as chemical and biological agents, the crews will operate with their hatches closed and sealed. High intensity combat lasers that can penetrate direct vision blocks may force the crew to operate on the battlefield with indirect vision systems for driving, and target search and engagement. In these vision systems, the conventional optics consisting of periscopic vision blocks and optical sights will be replaced by displays at each crew station and externally mounted cameras arrays on the vehicle. These vision systems will show computerized digitized images acquired by the camera arrays. The crewmember will see a selected portion of the computerized display buffer that depends upon his or her role and viewing direction. No doubt future vision systems will appear to the user as “see-through armor” by incorporating virtual reality components for the seemingly direct viewing of the external scene. However, before indirect vision systems can be considered for future vehicle designs, combat and materiel developers will need to know the potential impact upon the crew's combat performance. During night operations, replacing the vision blocks with infrared thermal viewers improves the crew performance by enhancing visibility at low light levels (McCarley & Krebs, 2000). In daylight conditions, however there are several factors that may affect performance and the use of indirect vision may cause a reduction in visual performance and situational awareness. This is because of the decrease in visual resolution and field of view of the current sensors and displays as compared to vision with the human eye through vision blocks. This reduction in visual performance may reduce overall combat performance. Further, the choice of camera configuration and placement on the vehicle can influence performance. For example, the use of a single camera for driving instead of a convergent dual camera array will deprive the driver of the near depth perception that is needed to avoid obstacles. This is true since the scene will appear bi-ocular (i.e., same image to both eyes) instead of the binocular needed for stereopsis. However, distant depth perception is still apparent from motion stereoscopic and terrain features. Visual Field-of-view One area of interest is the effect of the choice of visual field-of-view (FOV) upon driving performance for panoramic panel displays. This would be the case for a driver operating an armored vehicle with a video display and camera array in place of
6
direct vision from an open hatch or through vision blocks. The choice of FOV may depend upon the task being performed. The driver may accept a limited view for driving along a known route since the view allows the perception of potential road hazards. On the other hand, the driver may prefer a wider view at road turns for route selection. Some prefer to see the sky and the front of the vehicle’s hood in the scene (Smyth, Gombash, & Burcham, 2000). Display Selection The display selection will influence the vehicle design. The design may use a set of panel-mounted displays, either cathode ray tube (CRT) or flat panel liquid crystal displays (LCD), which are fixed in a panoramic arrangement about the crew member's station. As well as LCDs, Plasma and Electroluminescence are suitable candidates for flat panel displays because of their rugged sturdiness. The form of computerized aiding used with automation for the armored crew station, such as the Crewmember’s Electronic Associate Program, may be influenced by the display design. A panoramic design of panel displays for a two-man crew seated together may facilitate team interaction and performance. Optimal Crew Performance From the human factors viewpoint, there are three criteria for optimal driving performance: (1) proper design choice for the camera and display system, (2) a well designed vehicle control system, and (3) a supportive ambient environment in the cab area. Human operators can however, perform well with less than optimal systems by increasing their efforts to meet the more demanding workload. The problem is that over time excessive workload can lead to fatigue and increased errors. Furthermore, the increased flow of information and tasks may result in a loss of situational awareness; this is because the ability of humans to process information is innately limited. As noted by Endsley (1993), situational awareness is a precursor to optimal performance, since a loss in awareness impacts decision making and leads to a risk of performance error. For this reason, a fourth criteria is that the system design reduce excessive task workload and increase situational awareness as well as demonstrate improved performance. Motion Sickness Another issue influencing crew performance is the possibility of motion sickness, which can occur in an enclosed cab area with spatial disorientation. As noted by Yardley (1992), motion sickness is provoked by sensory conflict between the visual and sensorimotor activities, which involve the vestibular system through head movements. Associated with motion sickness are a constellation of mainly autonomic symptoms such as pallor, drowsiness, salivation, sweating, nausea and finally vomiting in the more severe cases of sickness. Although some individuals may eventually adapt to situations that initially provoke sickness (Yardley, 1992), the occurrence may be severe enough to arrest task performance until the symptoms subside. Emotional Stress Another research issue of interest is the effect of the system design upon the cognitive functioning and the emotional stressstate of the operator. This is important because the commander in the two-man armored vehicle design may have the additional role of being the driver. The commander may be expected to cognitively process information acquired visually from data displays for decision making such as for target engagement, and to select routes of approach from digital map displays. These cognitive functions may be composed of such basic elements as mathematical, semantic, logical and spatial reasoning, as well as higher level functions such as planning. The ability and desire to acquire and process information may be influenced by the stress-state of the commander.
OBJECTIVE The objective of this work was to determine the driving performance for the VTT with indirect vision for tasks found across a spectrum of operations including tactical assembly area, road march, and day and night cross country tactical driving. An additional objective was to collect subjective measurements on the mental workload for these tasks. The benefit of this study to the US Army is a better understanding of the advantages and limitations of the novel technologies incorporated in the CAT ATD for use in the Future Combat System (FCS).
7
METHODOLOGY The equipment, participants, driving courses, questionnaires, experimental design, and procedures used in this study are as follows. Equipment: The experimental apparatus is an M2 Bradley Fighting Vehicle (BFV) chassis, modified for the study by General Dynamics Land Systems (GDLS) with camera arrays attached to the roof of the vehicle and two additional drivers’ positions located in the troop compartment. These experimental driving stations can be rearranged from a side by side to a tandem configuration as required, but were kept in a tandem configuration for this study. The forward-looking camera arrays consist of five monocular charge-coupled device (CCD) color NTSC cameras that together cover roughly a 183-degree horizontal field of view (HFOV). Three of the cameras are grouped together in a front camera array and one camera is placed on each side of the vehicle. There is also a provision for a rear looking camera for driving in reverse. All cameras including the rear one have a scene magnification (defined by the ratio of the camera FOV to display FOV), of near-unity. The three central and rear cameras are 33.4 degrees HFOV; the far side cameras are 42-degrees HFOV. The central camera array has a 5.5-degree downward tilt. As well as daytime cameras, there are forward-looking infrared (FLIR) cameras for nighttime and low light level operation, with the latter being the Driver’s Vision Enhancer AN/VAS-5 (V) sensor assembly. An onboard safety driver is in the standard M2 driving position with a direct forward view and access to vehicle controls at all times. The vehicle is fitted with a Global Position system (GPS) and recorder for data logging. See Figure 1 for a photograph of the VTT concept vehicle.
Safety Driver Front camera array Rear camera
Side camera
Side camera
Figure 1. Crew integration and Automation Testbed (CAT) Advanced Technology Demonstrator (ATD) concept vehicle
The camera outputs are seen on five fixed flat-panel video displays that are mounted across the top of each of the experimental driving stations. As shown in Figure 2, the displays for the front camera array are arranged with a central display directly in front of the driver, and left and right side displays. The central camera of the front camera array drives the central display and the left and right side cameras of the array drive the corresponding side displays. The side displays are pivoted inwards to match the angles of the array side cameras. The forward and side displays are each 10.4-inches horizontal by 7.8-
8
inches vertical (13-inch diagonal), with 60 Hz non-interlaced RGB and a screen resolution of 1280 pixels by 1024 pixels scalable to the NTSC camera returns. The displays are positioned 17 to 21 inches from the experimental crew station design eye point. In addition, the views from the vehicle side cameras are seen on two wing displays positioned to the far sides of the crew station. The crewmember manually drives the vehicle from the experimental crew station with a hand yoke controller and a foot pedal brake and accelerator. A lower bank of three graphic displays is used for displaying a tactical map, and communications and system status. See Figure 2 for the layout of the displays and controls at the crew station. Participants wore standard Bradley crewmember military attire including helmets and boots for safety reasons.
Driving displays
yoke
tactical displays pedals
Figure 2. Experimental crew station displays and controls. Participants: Seven (7) military males and one civilian male who according to acuity measurements had 20/20 – 20/30 (corrected) vision and good hearing, served as participants. The military participated in all study phases. Six military participants were from the US Army Reserve based in Minnesota and one was from the Michigan National Guard. The civilian volunteer was from TARDEC and participated only in the cross-country trials. All had some experience driving tracked vehicles; in addition, the Army Reserve participants had extensive crew experience in a Bradley. The demographics of the participants are listed in Table 1 below. The table lists the age in years, the highest education level attained (HS: high school, BS: Bachelors of Science, College), Military Occupational Skill (MOS) and description, years of military service, type of tracked vehicles driven, and years of experience as a driver, gunner, or vehicle commander. As can be seen from the table, the experience is varied extending from personnel carriers to tanks as well as the BFV. The one civilian from TARDEC had extensive experience with the BFV through their effort in developing the VTT. Driving Courses: The participants drove on a close-in maneuver course replicating first a tactical assembly area, then a road course, and finally a cross-country trail in a controlled training area at Camp Grayling Military Reservation, Michigan. The dedicated area included ranges, range support facilities and field training areas that were made available to HRED for training and experimentation. The area used for close-in maneuver and assembly area operations is roughly 50 by 75 meters. A two kilometer (KM) unimproved road is available for road march and a 2.2 KM trail within a six KM-square adjacent area is available for cross-country tactical driving.
9
Table 1. Participants’ Demographics Participant S1 S2 S3 S4 S5 S6 S7 S8
Age 30 43 33 27 20 29 29 33
Rank SSG SSG SFC E4 E4 SGT E4 GS13
Edu HS HS HS BS HS HS HS BS
MOS 19K20 88M 11M40 11M10 51M 11M 11M n/a
Description Service Tracked vehicle Driver Gunner CO M1 Crew 12 M1, M113 1 3 1 Supply 1 M113, BFV 1 Gunner 6 BFV, M113 2 4 MechInf 5 BFV 1 Rescue 2 PLDC 1 MechInf 8 BFV/M113 8 2 MechInf 10 BFV 7 . n/a n/a BFV 1
Questionnaires: As well as the Volunteer Agreement Affidavit, the participants completed questionnaires for workload and technical evaluations as follows: Workload – The participant completed a battery of questionnaires for measuring the subjective effects upon different aspects of the task workload, included the following: Allotment of task attention, NASA Task Load Index, Motion Sickness, Situation Awareness Rating Technique, Cognitive Compatibility, and the Subjective Stress scale. These are described in further detail below. (a) Allotment of task attention - A questionnaire is used for rating the attention to the visual, cognitive, and motor processing channels of the human operator according to loading factors (McCracken & Aldrich, 1984). The questionnaire consists of a set of four 7-point, bipolar scales for rating the attention loading on each channel, with verbal anchors for corresponding activities overlaid on the scales. (b) NASA TLX - The NASA Task Loading Index (TLX) questionnaire (Hart & Staveland, 1988) is used for rating the perceived workload in terms of task demand and interaction. The NASA TLX is a multidimensional rating procedure for the subjective assessment of workload. Workload has been defined as a hypothetical construct that represents the cost incurred by the human operator to achieve a specific performance level. The construct is composed of behavioral, performance, physiological, and subjective components, which result from the interaction between a specific individual and the demands imposed by a particular task. The questionnaire consists of six scales that relate to the demands imposed on a subject and the interaction of the subject with the task. The Mental, Physical, and Temporal scales measure the demands, while the Effort, Frustration, and Performance scales relate to the interaction with the task. (c) Motion Sickness - A Motion Sickness questionnaire is used for the subjective estimation of motion sickness (Kennedy, Lilienthal, Berbaum, Baltzley, & McCauley, 1989). The questionnaire consists of 4-point, bipolar rating scales consisting of verbal descriptors for 26 symptoms of motion sickness, for example, general discomfort, eyestrain, dizziness, and nausea, among others. Based on data from a factor analysis of simulator sickness experiences, a procedure has been developed for reducing the scores to subscales for the symptomatic components of oculomotor stress (eyestrain), nausea, and disorientation, and a measure of total severity (Kennedy, Lane, Lilienthal, Berbaum, & Hettinger, 1992). (d) SART- The Situation Awareness Rating Technique (SART) questionnaire is used for rating the ability to maintain situational awareness (Taylor, 1989; Taylor & Selcon, 1994). The SART questionnaire (Selcon, Taylor & Koritsas, 1991) was designed to measure subjective ratings of non-attention factors such as domain knowledge or schemata and experience, the cognitive nature of the information received while performing the task, and the workload needed to process the information. The questionnaire uses ten independent seven-point bipolar dimensions, which are in turn classified into three major domains of situation demand, supply, and understanding. The ten dimensions of the questionnaire are: Instability, Variability, and Complexity of the situation for the demand domain; the Arousal, Spare Mental Capacity, Concentration, and Division of Attention for the supply domain; and the Quantity, Quality, and Familiarity of the information for the understanding domain.
10
(e) CC SART - A complementary measure to the SART, the questionnaire is used to rate the cognitive compatibility of a task (Taylor, 1995) requiring responses to situational information on such dimensions as depth of processing, stimulusresponse compatibility, ease of reasoning and understanding, and degree of knowledge. (f) The Subjective Stress scale - The Subjective Stress Scale (Kerle & Bialek, 1958) detects significant affective changes in one's emotional state. Participants select one word from a list of 15 adjectives that best describes how they feel during a specific time period or event. The scale is thought to be a measure of the overall "anxiety" or "worry" state due to stressful situations. Technology Evaluation - As well as an exit evaluation, the participant completed a battery of questionnaires to evaluate the VTT in performance of the different driving skills needed for assembly area, road march, and tactical operations. Each questionnaire consists of interview questions and sets of 7-point, bipolar-scaled questions in formats appropriate for rating the topics of test interest. Experimental Design: As mentioned above, the driving tasks were studied in the three separate phases of assembly area, road march, and tactical road operations. Since the time available for the study was limited, we were only able to operate the VTT in the indirect vision mode with the day camera during the assembly area and road-march operations and no comparison data was collected with the FLIR camera or from the open hatch position. For these phases, no experimental design applies and the data collected were used for descriptive statistics for the tasks. The data collected were the task times, accuracy, and answers to the questionnaires that were particular to the driving task studied. In addition, the dependent measures include the data from the workload questionnaires, which were applied at the end of the assembly area operations and again at the end of the tactical cross-country driving. Further dependent data is obtained from the exit evaluation of the technology. In contrast, the study of the tactical operations followed an experimental design in which the daytime and FLIR camera systems were compared in a counterbalanced scheme. Here, we conducted a within subjects single fixed factorial experiment with repeated measures on the day and FLIR cameras, and the participants as a random factor. For this task, the independent variables were the two camera systems and the null hypothesis was that there is no difference in performance by camera treatments. Procedure: As explained above, for ease of operation the experiment was performed in a set schedule of consecutive study phases that corresponded to normal tactical operations, consisting of a sequence of first assembly area tasks, then road march, and finally tactical functions. This is reasonable, since the operations for the separate phases demand different skill areas for proper functioning. For example, assembly area operations call upon depth perception, hand and eye coordination, and surround spatial situational awareness. Road-march skills involve speed and distance judgments. Finally, tactical functions involve orienteering, land navigation, and tactical situational awareness. While one study phase trains in the fundamentals for the sequential phase, they do not constitute continual training of the same activities across phases. In particular, the activities of the assembly area consist of task elements that are basic to those activities of the following phases. For this reason, the participants were trained and tested on the assembly area tasks, and then tested on the road march, and finally on the tactical functions. Since these are driving tasks, the participants were individually trained and tested alone with only one participant in the compartment at a time; here, the participant drove from the forward experimental crew station position. During all trials, a safety driver was present in the standard M2 driving position in the front of the vehicle, and an experimenter was present in the rear experimental crew station position. Training- The participant was briefed on the experiment, read and signed the consent agreement (see Appendix A), had his vision and hearing acuity tested, and completed a questionnaire on demographics. Next, the participant was trained by TARDEC to drive the VTT. The training was conducted on a site separate from the experimental sites in the Grayling area and training continued until TARDEC judged that the participant was prepared. Following training, he practiced filling out the workload evaluation questionnaires and was allowed a rest break. Experimental Studies- The participant performed the assembly area, road march, and tactical tasks in the following order. Following measurement of the visual resolution of the vision system, the clearance judgments of the participant and his parking and backing performance while under the control of a ground guide, was determined during the assembly area phase. In the road march, we studied his ability to follow another vehicle in a convoy and avoid roadway obstacles. This was followed by a tactical operation during which the participant navigated a tree-covered route across country with day and FLIR cameras.
11
The participant completed the technical evaluations at the end of each task exercise. The workload questionnaires were completed at the end of the assembly area exercises and the end of the tactical exercises. In the following section, we describe each of the experimental studies in detail, and in the process elaborate upon the procedures, dependent variables, and results.
12
EXPERIMENTAL STUDIES The results are presented for the tasks of the assembly area, road march, and tactical operations. Also presented are the experimental purpose, method, facilities, dependent measures, procedures, instructions to participants, evaluations by the participants, statistical results, and discussions of the results. Close In Maneuver Assembly Area The results for the visual acuity, judging clearances, and parking and backing are presented as separate studies. Visual Acuity for Sign Reading As a measure of the ability to perform tasks associated with assembly area, on road, and built up area operations; sign reading with the camera system on the Vetronics Technology Testbed (VTT) was measured on at a static test site. Purpose: The purpose of the study was to measure the ability to see detail and read road and signs with the camera system. Method: The participant read from Snellen equivalent vision charts that were placed in front of the vehicle cameras. Facility: Five standard letter and E-type vision charts were attached on vertical standing sign boards placed at 45-degree intervals about a central point of a semicircular arc with a 20-foot radius. Each chart could be seen by only one camera when the VTT was parked in the correct orientation at the central point. Figure 3 is a schematic of the facility and shows the distances from the viewing cameras to the corresponding signs for a parked vehicle as well as the approximate true Northdirection. The figure shows the assignment of letter and E-charts to the sign positions. The charts were fixed in location throughout the experiment because of the difficult in replacing charts and the limited time available for the trials. Measures: The dependent measures were the lowest acuity level correctly read and the answers to the questionnaires. Procedure: The VTT was driven to the front of the visual acuity test station by the safety driver and centered on alignment posts to ensure alignment with the facility. Once the vehicle was parked, the experimenter asked the participant to read the letters out loud on each of the charts starting with the largest letter. The experimenter recorded the observations until two errors were made in succession, where upon the participant was directed to proceed to the next sign. The participant reported the orientation (left, right, up, or down) of the letters on the E-chart. The chart reading order was randomized across participants. The acuity test was performed at the start of the testing day about 9Am during June 2001, following briefing of the participants and the measurement of their natural visual acuity with a Titmus vision tester. Instructions to Participants: From the driver’s station in the vehicle, you will view five visual acuity eye charts in turn, reading from the chart starting from the top and going from left to right. Note that there are two types of charts: The first set has letters in rows of different sizes and the second type has an “E” pattern in four different orientations in rows of different sizes. The experimenter will tell you to read one of the eye charts by location relative to the vehicle (“far-left,” near left,” “front,” near right,” and “far right”). For the letter chart, please speak each letter that you can read to the experimenter starting with the top most and reading from left to right and top to bottom. Tell the experimenter when you can no longer recognize a letter. For the E-pattern chart, please tell the experimenter the direction that the E-pattern is pointing, up, down, left, or right, starting with the top most “E” and reading left to right and top to bottom. Tell the experimenter when you can no longer recognize the character. The experimenter will ask you to repeat this procedure for each of the target boards in turn by board location. He will record the letters or orientations that you read. Statistical Results: Figure 4 is an Exploratory Data Analysis (EDA) box-and-whisker plot (Velleman & Payne, 1992) of the chart readings for the cameras. This follows corrections of the raw data to the standard 20-foot reading distance for the Snellen charts. The figure includes the near and far distance visual acuity readings for the participants that were measured by the experimenters with the Titmus Vision Tester prior to the trials. Note that normal acuity is considered to be 20/20 as shown on the figure. The near visual acuity is of interest since the accommodation distance for the tester is 14-inches and the vehicle displays for the cameras are placed about 17-inches from the viewing point of the driver. The boxplot figure shows the median, the “hinges” (first and third quartiles), and the maximum and minimum values that are not outliers, for the distribution of each viewing treatment. Values more than 1.5 times the box-lengths (interquartile range) from the quartiles are designated as outliers and values more than 3 box-lengths as extremes. The figures show that the reading distributions are far from normal
13
since they are largely skewed with two extremes marked by asterisks ( ). Table 2 lists the corresponding statistics for reference, including those for all cameras combined together. A nonparametric, repeated measures Friedman test by ranks shows statistically significant differences among the distributions for the near visual acuity and the camera readings (Chisquared value = 25.37, N =7, df =5, p < 0.001). A Scheffe’ family-wise comparison test shows that the near visual acuity is significantly different from the readings for the cameras (Scheffe’ test value = 15.00, df = 6, p < 0.025), but that the readings for the cameras are not significantly different. This is even true for a comparison of the readings for the letter-charts to those for the E-charts since these distributions largely overlap in range. Table 2 shows that for the median near visual acuity is 20/30 and the median across all camera readings is 20/60.606 for this set of participants; that is, the median natural acuity is 2.020 times that seen through the camera systems. Table 2. Descriptive statistics for visual acuity readings.
Condition Treatment Titmus Near Visual acuity Camera Far left Near left Center Near right Far right All cameras combined
25th 22.000 60.606 60.431 59.154 47.059 66.667 47.059
Percentile 50th (Median) 30.000 60.606 60.431 59.154 47.059 66.667 60.606
75th 40.000 84.848 172.659 84.506 47.059 93.333 84.506
Participants’ Evaluation: The participants were asked to rate the vision system for the task of sign reading. The ratings are on a 1 to 7 point, bipolar scale with the verbal end anchors of “less” for the 1 rating and “more” for the 7 rating. Considering the small sample size of six participants who elected to answer the questionnaire, we summarize the results by reporting the rating average and range. The participants rated the letters as being relatively easy to read (average rating: 3.67, range: 3-4). There were some distortions of the letters as seen from the displays (average: 4.33, range: 1-7). They experienced little eyestrain while reading the letters (2.25, 1-3). They had to concentrate somewhat to read the letters (3.5, 1-5), especially the smaller ones. Interesting enough, while all but one reported no feelings of motion sickness, that one participant rated motion sickness on a 3 level even though the vehicle was stationary. One participant said that the front sign was difficult to see while reporting that his eyesight was less than 20/20. Some participants mentioned that sunlight reflected from the center signs made some hard to read. Another reported that the center and left-front signs were difficult to read because of the sunlight. Furthermore, the center display was not sharp causing some eyestrain. One reported the camera on the left side as being the clearest. Discussion: While the acuity readings with the cameras are statistically equivalent, the lowest median reading was with the chart at the near right camera. The fact that all participants reported the same acuity for this chart suggests that the actual values were less for the viewing conditions as determined by the outside ambient lighting, and the camera lens and focus setting. Considering that the next lower reading for a letter-chart of 20/40 was not recorded, the visual acuity value must be greater than that for all participants. Correcting for the chart distance, the lowest acuity for the participants must be greater than 20/37.64; here, the median natural acuity is 1.26 times our estimate of the lower bound to the acuity as seen through the camera systems. The acuity ratio is an indication of the reduction in visibility with the camera system for driving and navigating. For example, a road sign would only be readable on the average when located at a distance that is 49.50% of that for natural vision (less than 79.68% for best camera). That is, a sign naturally read at 20-feet could only be read at 9.90-feet with the camera system (15.94-feet best camera). Also, the reduction in acuity reduces the resolution of the natural roadway image on the display and there is reason to believe that the driver would be compelled visually to drive at 79% of his natural road speed (92.58% best camera). This is true even with a unity display because of the inverse relation of road speed to the cube root of the display resolution in driving with camera systems (Smyth, Gombash, & Burcham, 2000). While the 13.0 inch diagonal AMICD flat panel displays in the VTT are 1280 by 1024 SXGA pixel resolution, they scale the images to the 460 by 400 TV resolution of the NTSC return from the PULNIX TMC-73M cameras (1/3” imager, 768 by 494 pixel resolution). In summary, the resolution is limited by the cameras, and using a higher resolution camera with say 630 by 548 TV lines, may allow viewing with natural acuity (547 by 476 TVL for best viewing conditions). The resulting camera TVL resolution
14
remains the limiting factor when used with the SXGA display. This remains true for the VTI with the 1200 by 900 pixels of the ASV Technology UXGA display dedicated to the driving scene.
N Center 20 ‘
Near left
23 ‘ 8”
Near right
21 ‘ 3”
23 ‘ 2”
Far left
Far right 15 ‘ 0”
16 ‘ 6”
Camera arrays VTT Bradley rear
Figure 3. Top view of visual acuity test site
15
Note: * - extreme value
Visual Acuity (Snellen equivalents)
20/200
20/150
20/100
20/50
20/20 0 Far
Near
letters
E-chart
E-chart
letters
Far left
Near left
center
Near right
Titmus Vision Test
VTT camera disposition
Figure 4. Box-plots for visual acuity readings.
16
E-chart Far right
Judging Clearances: As a measure of the ability to perform tasks associated with assembly area operations, operations in built up areas, and deal with road hazards, the quality of the visual and spatial clues needed to maneuver in close to objects with the camera system on the Vetronics Technology Testbed (VTT) was measured on a test site. Purpose: The purpose of the study was to determine the ability to judge the clearance of obstacles along a well marked lane of travel, in particular when the obstacles can be seen in the front camera but not the sides as would be experienced when crossing a bridge. Method: The participant drives the VTT down a converging roadway until he estimates that he can no longer move forward without striking one of the side markers. Facility: A 85.75 foot straight dirt roadway marked with a converging set of 7 pairs of 3.5” diameter PVC pipe spaced along the roadway nominally every 14.3-feet, was used as the test course. The posts were emplaced in dirt alongside the road and extend 4-feet above ground. The posts of the pair at the entrance to the course are 16.83 feet apart while those at the exit are separated by 9-feet. See Figure 5 for a diagram with the actual pair spacing along the course. Measurements from a M2 Bradley show a 9.75 feet track width, an overall 10.42 feet width, a 2.167 feet skirt height, and roughly a 21.5 feet length. As shown in the figure, the VTT M2 Bradley chassis with the 10.42-foot width could be driven up to the next to last pair of markers (pair #6) before hitting a post. This is true since while pair #6 are separated by only 10.27-feet, the pair behind those (pair #5) are separated by 11.57-feet, a distance which allows a 7-inch clearance on each side between the vehicle and post. A discerning driver capable of judging the roadway sides from the post placements could drive the vehicle to within 15.5-feet of the course end to meet the test criteria. A M2 Bradley was driven along the course to leave a track print before the test started. Figure 6 shows a picture of the clearance test site as seen looking toward the entrance with the VTT entering the site. Measures: The dependent measures were the course times, obstacles hit, and the technical questionnaire ratings. Procedure: The vehicle was positioned in the center of the lane at the front of the course by the safety driver and headed down course. The participant was directed to drive forward until he estimates that he can no longer move forward without striking one of the side markers. The trial was stopped if he struck any side marker. Recorded were the time to complete the course, the distance traveled, and the identities of the posts struck, if any. Any dislocated posts were repositioned. Instructions to Participants: At the safety officer’s command, slowly drive the vehicle forward until you feel that you can go no further without striking a post with the vehicle. At this point stop, put the vehicle in park, and inform the safety officer of your decision. If you unknowingly strike a post the safety officer will instruct you to stop and park. Statistical Results: None of the seven participants met the clearance criteria since three struck a post of pair #4 (12.7-feet separation) while four pushed pass the criteria line to strike pair #2. While two participants ran into a post of pair #4, one went slightly pass pair #5 and struck a post of pair #4 sideways when making a steering correction. Based on this data, the average additional spacing needed for vehicle passage was 9.4 inches (4.7 inches on a side); however, 28% of the participants needed spacing greater than 2.25 feet (1.13 feet on a side). The average speed of travel was a minimal 2.16 km/hr (0.93 km/hr minimum, 3.46 km/hr maximum) or 1.34 MPH, that is, roughly a foot per second. There was no significant correlation among the speed of travel, the ending offset, and the near distance visual acuity of the participants. There is no trend of changes in ending offset with test order. Participants’ Evaluation: The participants were asked to rate the VTT for the task of judging clearances. The ratings are on a 1 to 7 point, bipolar scale with the verbal end anchors of “less” for the 1 rating and “more” for the 7 rating. Considering the small sample size of six participants who elected to answer the questionnaire, we summarize the results by reporting the rating average and range. The participants reported that while it was relatively easy to track the lane poles on the displays (average ease: 5.5, range: 3-7), it was not as easy to judge the clearance to the front (average: 4.5, range: 2-6) and somewhat difficult to judge clearance on the sides (average: 3.33, range: 2-7). The participants stated that this was because they had difficulty in judging the location of the front and sides of the vehicle. They stated that the speed of the vehicle was easy to judge from the displays, although one participant felt that the perceived speed was faster than his actual motion. Another participant reported that he could see the poles in his front displays, but that he did not use his far side displays. One participant reported slight feelings of motion sickness at the end of the trial. Further, the reaction time of a M2 chassis needed to stop or turn makes it somewhat difficult to maneuver in tight areas.
17
Analysis: A question of interest is how well can we expect the participants to drive the course with the cues that were provided by the posts and ground elements such as the road surface and track print. To answer this question, we consider the visibility plots for the vehicle as determined from the geometry of the cameras. As shown in Figure 7 for a side view of the vehicle, the three forward looking cameras on the Bradley are placed 6.3 feet above the ground and tilted down 5.5 degrees from the horizon. The cameras have a 25.37 degrees vertical field-of view (VFOV) and the ground intercept point is 19.01feet from the cameras. This is the point on the ground that is seen at the lower edge of the display. Similarly, the top of a four feet high post is just visible at a distance of 6.84 feet from the cameras. Considering now Figure 8 for a top view of the vehicle, the three display videos from the cameras subtend about 115degrees centered on the forward direction. This includes the 33.4 degrees horizontal field-of view (HFOV) for each of the camera returns to the displays and the display separations. Our measurements showed that the displays were separated by about a 2.25 inches wide frame, which at a viewing distance of 17-inches causes a 7.47 degrees wide blind spot between the displays. The 10.2-inch wide displays (13-inches diagonal) subtend 33.4 degrees at 17-inches viewing distance. The figure shows that the posts of set #6 are visible in the central display for a distance greater than 17.12 feet. The posts are in the visible blind spot generated by the display frames when the posts are between 11.44 and 17.12 feet from the vehicle. Finally, the posts are visible in the side displays when they are between 11.44 feet and 6.84 feet where they pass below the camera VFOV. Figure 9 shows visibility plots for the post set #6 on the clearance course as determined from the calculations for Figures 7 and 8. As noted in the facility section, since the separation of the posts is less than the width of the vehicle the participant must stop before this set to meet the experimental criteria. If he reaches this set, he has gone too far. Figure 10 shows the VTT at the post set # 6 with the posts pushed aside by the passage of the vehicle. As shown in Figure 8, the participant could no longer see the ground at the post set when he was within 19-feet; this occurred between the post sets # 4 and 5. The posts passed out of his view when he was within 17 feet and reappeared in the side displays at about 11 feet. These events occurred about the set #5. Finally, the posts passed below view when he was within 6’ 10” of the posts, which occurred when he was well pass set #5. However, he could continue to guide himself by his view of set #7, which was visible up within a foot of set #6. This is because the set #7 with a 9-feet separation between posts is visible to within 15-feet in the central display. Discussion: The extended view used for driving and the inability to pan and tilt the camera system limits the ability to maneuver in close areas. The participant estimated his course from the limited cues on the display scene. While the ground track was not visible to his immediate front, he could guide his course from the post markers. Although he would lose sight of a marker set as he approached, still the next set remained in view. As he approached, the posts would briefly reappear in his side displays for several seconds; however, the view was too brief to be helpful. The participant estimated his passage from the scene of the posts and the extended track print on the central display. However, the process increases the mental load since the participant had to extrapolate the clearance from the separation of the posts in front of him. In a conventional BFV or similar vehicle, close in maneuvering is usually facilitated by a ground guide and or the commander’s look down view from the turret. Currently this support to the driver’s limited view is only unavailable when the risk of exposure of crewmembers is greater than the risk associated with a collision. In a future vehicle configured as the VTT is, it is likely that driver and other crewmembers will have the same visual information available through the same cameras and augmented views from a better crew position or the ground may be unavailable. The inability of the VTT to provide the visual and spatial clues needed to deal with the close in obstacles presented by this test is an area that needs consideration.
18
Pole pair #
16’ 10” 1 15’ 1” 2 14’ 2” 3 14’ 2” 85’ 9”
4 12’ 2”
rear
5 VTT 14’ 2” 6 15’ 6”
13’ 11” 7 9’
Figure 5. Clearance test.
19
Figure 6. VTT entering clearance test site.
20
25.4 degrees Vertical FOV
Central camera
horizon 5.5 degrees Centerline 6.25 feet height
4 foot post
Intercept
ground
6.84 feet 19.01 feet
Figure 7. Side View of Camera Deployment
21
Left side camera 33.4 degrees Horizontal FOV 7.47 degree blind spot
Central camera 33.4 degrees Horizontal FOV
10.27’
Post set #6 7.47 degree blind spot Right side camera 33.4 degrees Horizontal FOV
11.44’
17.12’
Figure 8. Top View of Camera Deployment
22
Entrance 16’ 10”
Visibility of post set #6
2
3
85’ 9”
Distance from post # 6 (feet)
1
Sees all of post in central display
4 19’ 0” rear
17’ 1” 5
Cannot see ground at post Cannot see post
11’ 5”
VTT
6’ 10”
Sees post in side display Cannot see post
0’ 0”
6
7 9’
Figure 9. Visibility Plots
23
Figure 10. VTT positioned at post set #6.
24
Parking and Backing: As a measure of the ability to perform tasks associated with maneuvering close to obstacles such as assembly area operations, the maneuverability of the Vetronics Technology Testbed (VTT) under the direction of a ground guide was evaluated. Purpose: The purpose of this study was to determine the ability of the participant to follow the hand signal commands of a ground guide through a sequence of parking and backing maneuvers with the vehicle. Method: The times and number of hand signals used by a ground guide to direct the vehicle were measured along with the number of barrier strikes. Facility: Figure 11 shows a layout of the assembly area experimental facility that was established on a large ground level concrete apron roughly 170 feet by 70 feet that was normally used for vehicle parking. A holding box is shown in the upper right corner of the layout. The box was painted on the apron to the dimensions of a Bradley vehicle. Similarly, a starting box the same size and shape as the holding box was painted in the central right of the apron. Several wooden fixtures were built on the apron out of perforated wood panel sections in a latticed design (8-feet long by 6.5 feet high), painted a semi-gloss black. An enclosure open at one end for access is shown at the central top of the layout. The enclosure could be interpreted as a parking space between two other vehicles. Finally, two shed size structures were built parallel to each other as obstacles. Again, these structures could be interpreted as parked vehicles or buildings that one must maneuver around. Figure 12 is a photograph of the shed structures. As can seen from the figures, the structures were well spaced apart to reduce the chance of damage to the wooden paneling. Measures: The dependent measures are the times to reach the parking positions, the number of ground guide’s hand signals, and the technical questionnaire ratings. Procedure: For safety reasons, the safety officer directed the vehicle to the holding box and released the vehicle to the participant once the ground guide was in position. The ground guide then directed the vehicle to the starting box in the central right of the layout. All trials were started from the starting box. The experiment consisted of two trials. These trials are described in greater detail in the following paragraph. Figure 13 shows the maneuvers for the first trial. In this trial, the ground guide directs the vehicle from a starting position forward and then back into a parking position between wooden enclosures while the experimenter records the time and number of hand signals. The circled position marked “1A” shows the position of the ground guide at the start of the trial with the vehicle parked in the starting box, here marked “Start-A.” Walking in front of the vehicle, the ground guide directs the vehicle through a series of hand signals to the parking position marked “Park-1A”. Again, placing himself at the position marked “2A,” the ground guide proceeds to direct the vehicle backwards in a right hand turn to the parking position marked “Park-2A,” while walking in front of the vehicle. Figure 14 is a photograph of the parked VTT with the ground guide. The vehicle is then directed back to the starting box. Figure 15 shows the maneuvers for the second trial. In this trial, the ground guide directs the participant forward to a parking position, then around a barrier to another parking position, and finally backwards retracing the route to the original parking position. Again, the circled position marked “1B” shows the position of the ground guide at the start of the trial with the vehicle parked in the starting box, here marked “Start-B.” Walking in front of the vehicle, the ground guide directs the vehicle through a series of hand signals to the parking position marked “Park-1B.” Again, placing himself at the position marked “2B,” the ground guide proceeds to direct the vehicle forward in a left hand turn to the parking position marked “Park2B,” while walking in front of the vehicle. Again, placing himself at the position marked “3B,” the ground guide proceeds to direct the vehicle backwards in a left hand turn to the original parking position now marked “Park-3B,” while walking in front of the vehicle. Instructions to Participants: Following the safety officer’s command, you will drive the vehicle up to the holding position. At all times during this and other exercises, the safety officer has overriding control. You will follow the hand signals of the ground guide. He will first direct you to drive the vehicle to a starting “box” painted on the cement apron where you will stop and wait for the next command. The ground guide will then direct you to drive the vehicle forward past a set of wooden enclosures to a stopping position. He will then direct you backwards into a right rear turn between the wooden enclosures to a parking
25
position. The experimenter will record the time that you take to reach the parking position and the number of hand signals the guide used to direct you. The ground guide will then direct you out of the enclosure and back to the starting box. At the command of the ground guide, you will drive the vehicle forward to the side of a wooden barrier and stop. Then again following the commands of the ground guide, you will the vehicle around the barrier and stop on the other side. Again, at the command of the ground guide, you will drive backwards retracing your path around the barrier and stop on the other side. The experimenter will measure the time that you take to park the vehicle and the number of hand signals. Statistical Results: The statistics for the number of hand signals and the time needed to guide the vehicle in each maneuver are presented below. A correlation analysis shows significant correlation between the number of signals and the maneuver time for the ground guide station “1A” (Pearson correlation = 0.890, p < 0.017) and those for station “3B” (Pearson correlation = 0.943, p < 0.005), but none other. Two of the seven participants struck the structures with the vehicle when performing the backing turn at station “2A;” no other strikes were made. We describe the statistics for the hand signals and maneuver times in greater detail below: (a) Guide Hand Signals: Figure 16 is an Exploratory Data Analysis (EDA) box-and-whisker plot (Velleman & Payne, 1992) of the number of hand signals needed to guide the vehicle at each station. The boxplot figure shows the median, the “hinges” (first and third quartiles), and the maximum and minimum values that are not outliers, for the distribution of each station. Values more than 1.5 times the box-lengths (interquartile range) from the quartiles are designated as outliers and values more than 3 box-lengths as extremes. The figures show that the distributions are far from normal since they are largely skewed with occasional outliers and an extreme. Here, the figure shows the outlier values marked by an open circle ( ) and the extreme marked by an asterisk ( ). Table 3 lists the corresponding statistics for reference. A nonparametric, repeated measures Friedman test by ranks shows statistically significant differences among the distributions (Chi-squared value = 13.826, N =7, df =4, p < 0.008). A Scheffe’ family-wise comparison test shows that the rankings of the hand signal counts for station 2A and 1B together are significantly different from those for the remaining stations (Scheffe’ test value = 10.75, df = 4, p< 0.050).
Table 3. Descriptive statistics for the number of hand signals. 25th Station 1A 6.0 2A 12.0 1B 13.0 2B 8.0 3B 8.0 All combined 8.0
Percentile 50th (Median) 7.0 21.0 17.0 13.0 12.0 12.5
75th 10.0 23.0 19.0 15.0 14.2 17.5
(b) Guide Times: Figure 17 is an Exploratory Data Analysis (EDA) box-and-whisker plot for the time needed to guide the vehicle at each station. Table 4 lists the corresponding statistics for reference. A nonparametric, repeated measures Friedman test by ranks shows statistically significant differences among the distributions (Chi-squared value = 13.257, N =7, df =4, p < 0.010). However, a Scheffe’ family-wise comparison test shows no significantly different among the rankings for the station times.
Table 4. Descriptive statistics for guide times (minutes).
Station 1A 2A 1B 2B 3B All combined
Percentile 25th 50th (Median) 0.413 0.615 1.857 2.370 1.683 1.883 1.183 1.450 1.423 1.543 1.157 1.688
75th 1.002 2.660 2.918 1.693 1.922 2.141
26
Participants’ Evaluation: The participants were asked to rate the VTT for the task of close quarters maneuvers. The ratings are on a 1 to 7 point, bipolar scale with the verbal end anchors of “less” for the 1 rating and “more” for the 7 rating. Considering the small sample size of seven participants who elected to answer the questionnaire, we summarize the results by reporting the rating average and range. Regarding the maneuver guidance, the participants rated the ground guide as being very visible (average rating: 6.14, range: 3-7); however, the hand commands were less so (average: 5.86, range: 3-7). The participants could readily follow the commands of the ground guide (6.43, 5-7) and felt confidence that they could perform their task doing so (5.42, 2-7). They reported a high degree of interaction between the ground guide and themselves as the driver (6.14, 4-7). They felt that the ground guide was safe during the exercise (6.71, 6-7). One participant reported that he was able to understand all hand commands, but tended to reverse the commands for left and right. He was able to see the ground features and did not feel isolated. He felt confident about his interaction with the ground guide. Another participant reported that he was completely dependent on the guide since he could not tell from the displays where he was in the assembly area. At times he would lose sight of the guide when he moved between cameras. Still another reported that the guide was not always visible because of blind spots between the cameras and sun light glare. He had trouble following the commands because the guide would stand too far away and for this reason he lacked feedback from the guide. He reported that his vision was limited by the lack of depth perception. In contrast, another participant reported that the hand commands were highly visible, that he could follow the guide without difficulty, and that the guide seemed aware of his responses. Other statements were that it was difficult to tell clearance from the rear camera, the M2 chassis was difficult to control when accelerating from a stop, there was a blind spot between the front and near side cameras, the perceived speed seemed faster than the actual speed, and that one participant felt slightly motion sick toward the end of the trial. Finally, one participant reported that the guide was unsafe when he took too long to brake while the guide was standing in front of the vehicle. Regarding the parking task, they reported that they could readily see the partitions through the cameras (6.14, 4-7). However, while they were readily able to follow the approach path (6.0, 5-7), they were less able to locate the marked parking position (5.14, 2-7). They had difficulties in judging the parking confines (3.83, 1-6) and were not necessarily satisfied with how well they parked the vehicle (4.71, 1-7). Finally, they were not necessarily satisfied with the placement of the cameras for the task of parking the vehicle (5.24, 1-7). Mostly, the participants reported that they parked the vehicle by following the ground guide and that they could not see enough of the assembly area to park the vehicle by themselves using the cameras alone. One reported that the yoke control felt different from that for a BFV. In an evaluation of the cameras, displays and controls, the participants rated the camera placement as being fairly effective for this task (5.57, 3-7). Similarly, the placement of the displays was rated fairly effective (5.57, 2-7) as was the resolution (5.0, 3-7) and the brightness and colors (5.24, 3-6) of the displays. There were few distortions in the display scene (2.28, 1-6). However, some had difficulty with the lack of depth perception (4.43, 1-7) and display vibrations (2.71, 1-5). The steering controls were effective (6.0, 5-7); however, some rated the accelerator and brake pedals as less so (4.28, 2-7). Most did not feel that they had to think about the task (2.86, 1-5) and were fairly aware of the surrounding situation (5.71, 2-7). However, while four reported no motion sickness, two did report slight motion sickness (1.86, 1-5). Comments mentioned small blind spots between cameras, the scene on the displays looked natural, that the front display vibrated when accelerating, and too much sensitivity and delayed response. Discussion: When compared to subjective opinions based on experience ground guiding military vehicles, the median number of 12.5 hand signals and 1.688 minutes guide time per maneuver appears excessive for the given maneuvers. A number of elements go into making an efficient guide-driver team. Theoretically a driver should only be reacting to the signals of the guide, however in reality the driver is facilitating the process by making decisions based on other visual clues provided by his environment. Additionally the ground guide is making decisions based not only on the movement of the vehicle but also on nonverbal feedback gained from looking at the driver. When the drivers’ visual clues are diminished and face-to-face contact with the guide is lost, such as at night, performance is understandably degraded. In summary, under the current configuration the performance of the system may only achieve a level comparable to a vehicle operating in a degraded mode such as night, buttoned up or having the crew in protective masks. One source of difficulty may be the resolution of the cameras on the VTT. This would make the determination of a hand signal difficult causing slower responses and the repetition of signals.
27
N 15’ 4”
16’ 5” 8’
8’
10’ 6”
24’ 0”
25’ 0”
26’
20’ 4”
16’ 8” 8’
10’ 6”
45’ 20’ 4”
16’ 3” 16’ 8”
Figure 11. Assembly test area dimensions
28
Figure 12. Assembly area maneuver obstacles.
29
Park 2A
Backup Enter
2A Forward Park 1A
1A Start-A
Figure 13. Assembly area maneuvers
30
Figure 14. VTT parked with ground guide.
31
Park 3B Park 1B
2B
1B Backup
Forward
Start-B
3B Park 2B
Figure 15. Assembly area “B” maneuvers
32
Note: * - extreme value O – outlier value
Number of hand signals
40
30
20
10
0 1A
2A
1B
2B
Figure 16. Hand signals at guide stations.
33
3B
Guide station
Time (minutes)
Note: * - extreme values O – outlier values 4
3
2
1
0 1A
2A
1B
2B
Figure 17. Times at guide stations.
34
3B
Guide station
Tactical Road March Operations The results for road march (vehicle following and obstacle avoidance) and tactical driving are presented as separate studies. Vehicle Following: As a measure of the ability to perform tasks associated with tactical road march operations, the convoy following ability of the Vetronics Technology Testbed (VTT) was evaluated. Purpose: The purpose of this study is to test the ability to maintain a separation distance while following another vehicle, such as when in a convoy. Method: Follow a HMMWV at a distance of 50 meters. Facility: A course was laid out on a 1200-meter stretch of dirt roadway for convoy road following. See Figure 18 for a map of the test area with an overlay of the course. The starting position was marked with a sandbag. The 50-meter position for the lead vehicle was marked with a second sandbag. The roadway ran in the East direction to the 200-meter mark where following a bend, the roadway ran in a straight line in the North East direction for the next 1000 meters. Starting with marker 04 at 200 meters, the roadway was marked with numbered signboards every 50 meters. As can be seen from the figure, the lead vehicle accelerated slowly to 15 mph (24.13 km/hr) for the first 150 meters, then held that speed for the next 350 meters, accelerated quickly to 25 mph (40.22 km/hr) and held that speed over the next 350 meters, then dropped to 10 mph (16.09 km/hr) for the remaining 300 meters. Measures: The dependent measures are the course time, the safety officer’s estimate of minimum and maximum separation distances, and the technical questionnaire ratings. Procedure: At the start of the trial, the lead vehicle was moved to the 50-meter marker and the test vehicle to the zero-meter marker. The participant was allowed to study the lead vehicle in the display for about 30 seconds to develop a sense of the 50meter separation distance. The safety officer (SO) then moved the test vehicle to within 10 meters of the lead vehicle, where upon the lead vehicle moved out slowly accelerating to 15 mph (24.13 km/hr) by marker 04. The participant was told to follow the lead vehicle and maintain a 50-meter separation distance. The lead vehicle changes speed to 25 mph (40.22 km/hr) at marker 11 and 10 mph (16.09 km/hr) at marker 18. Throughout the test, the safety driver uses the 50-meter markers to judge the actual distance. The lead vehicle stops at market 24. The experimenter records the minimum and maximum distances and the route time. Figure 19 is a photograph of the VTT following the lead vehicle. Instructions to Participants: As directed by the SO, you will drive the VTT to a starting position marked by a white marker on the roadway. A HMMWV will be parked 50 meters ahead. Study the apparent size of the vehicle at that distance. After 30 seconds, you will be directed by the SO to drive the VTT to a position directly behind the HMMWV. Shortly there after, the HMMWV will slowly start to drive away from you down the road. At the command of the SO, proceed to follow the HMMWV maintaining 50 meters separation as you would in a convoy. The HMMWV will slowly alter speed of travel speeding up and slowing down; you should adjust your speed accordingly to maintain 50 meters separation distance. The SO will tell you to come to a halt at the end of the test. The experimenter will measure the time of travel and estimate your separation distance. Statistical Results: Figure 20 is an Exploratory Data Analysis (EDA) box-and-whisker plot (Velleman & Payne, 1992) of the road convoy separation distance between the vehicles constructed from the minimum and maximum distances recorded. The boxplot figure shows the median, the “hinges” (first and third quartiles), and the maximum and minimum values that are not outliers, for the range distributions. Values more than 1.5 times the box-lengths (interquartile range) from the quartiles are designated as outliers. The figure show that the distribution is fairly normal with an outlier. The figure shows the specified 50meter separation distance for comparison. Table 5 lists the corresponding statistics for reference.
35
Figure 21 is a box-and-whisker plot of the average speed of travel. The course speeds were computed from the recorded times for traveling the 1150 meter course (50-meter point to marker 24). The figure shows a normal distribution with a 14.3 mph (23.00 km/hr) median speed, and 13.32 mph (21.43 km/hr) first quartile and 16.39 mph (26.37 km/hr) third quartile; this is a fairly tight distribution considering the changes in the speed of the lead vehicle from 15 mph (24 km/hr) to 25 mph (40 km/hr), then back down to 10 mph (16 km/hr). As indicated by the broken line on the figure, the average road speed is 13.16 mph (21.18 km/hr) for the lead vehicle. There is no significant correlation among the road speeds and the separation distances.
Table 5. Descriptive statistics for the separation distances (meters)
Distance
Percentile 25th 50th (Median) 75th 49.17 67.50 78.54
Participants’ Evaluation: The participants were asked to rate the VTT for the task of vehicle following. The ratings are on a 1 to 7 point, bipolar scale with the verbal end anchors of “less” for the 1 rating and “more” for the 7 rating. Considering the small sample size of seven participants who elected to answer the questionnaire, we summarize the results by reporting the rating average and range. The participants reported that they could see the lead vehicle moderately well (average rating: 5.0, range: 4-7), and judge the distance fairly accurately (average: 5.14, range: 4-6). They had some difficulty noticing changes in the speed of the lead vehicle (4.57, 2-7). However, they felt safe following the lead vehicle (6.43, 5-7). Most commented that dust from the lead vehicle obscured the driving scene. One participant commented on sun glare with the cameras. Some commented on a lag time in noticing changes in the vehicle speed. Two participants reported slight motion sickness with one reporting a headache caused by the vibrations and interior heat. Discussion: Precise vehicle separation in convoys is difficult to achieve with direct vision. However, considering the limited and controlled environment of the test course and the demonstrated ability of a BFV to maintain a specified interval during the pilot study establishing this course, a driver should have been able to maintain his assigned interval. The data points out that the participants did not keep up with the lead vehicle. The median road speed was 14.29 mph (23 km/hr), which is greater than the average speed that the lead vehicle maintained over the course. This would explain the large range in separation distances varying over 75 meters; however, the median separation of 67.5 meters is 17.5 meters greater than the test specification. This may be within the accuracy of our measurements, which were limited by the 50-meter spacing between the markers. The indirect vision systems could be degrading the ability of the driver to judge distance and react to speed changes. The participants may have had difficulty detecting the speed changes either due to the display resolution or perhaps dust generated from the lead vehicle. Similarly the participants may have had trouble judging distances because of the differences in the judgment relationship clues provided by the system compared to direct vision.
36
Camp Grayling Test Area 24
18
25 mph 11
04 0 Meter Sandbag
50 Meter Sandbag
Accelerate slowly to 15 mph
Figure 18. Vehicle Following test course
37
15 mph
10 mph
Figure 19. VTT following lead vehicle
38
Note: * - extreme value
Distance (meters)
160
140
120
100
80
60 50-meter separation 40
20 0
Figure 20. Vehicle separation distance
39
Average speed (km/hr)
30
28
26
24
22 Lead vehicle 20
Figure 21. Average road speed
40
Obstacle Avoidance: As a measure of the ability to perform tasks associated with tactical road march operations, the Vetronics Technology Testbed’s (VTT) ability to allow the drive to detect and maneuver around obstacles encountered on the road was tested. Purpose: The purpose of this study was to test the ability of the VTT to navigate about roadway obstacles. Method: Determine the speed at which the participant navigates a set of roadway barriers. Facility: An obstacle to roadway passage was made by placing three sets of two 55-gallon barrels to the sides of a 4.7-meter wide dirt roadway at 25-meter intervals with the middle set on the opposite side of the end sets. The barrels are about 0.58 meters in diameter. Figure 22 shows a schematic of the obstacle course with reference to a map of the test area. The participant needed to steer in an “S” pattern to navigate between the barrel pairs and continue on the road. The foremost pair was placed at road marker 32 and the last pair at marker 33. The course was located a distance of 400 meters from the end point of the convoy road following test. The participant approached the barriers after making a roadway turn just after road marker 24. Measures: The dependent measures are course time, obstacle passage time, number of obstacles struck, and technical questionnaire ratings. Procedure: From the end of the vehicle following test (marker 24) the safety officer told the participant driver to move out down the road at 20 mph. When the driver encountered the obstacle he performed an “S” turn to avoid the barrels and continue on the road. As necessary the safety officer warned the participant when the obstacle was in sight and advised him on turn maneuvers to keep him on the road and avoid a collision. An experimenter on the ground at the site used a stopwatch to time the vehicle entering and exiting the obstacle and record barrel strikes, if any. The vehicle was stopped at the range marker 34. Instructions to Participants: Move out down the road at 20 mph (32.18 km/hr), and staying on the road avoid any obstacles that you may encounter. Statistical Results: Figure 23 is an Exploratory Data Analysis (EDA) box-and-whisker plot (Velleman & Payne, 1992) of the speed of travel. The course speeds were computed from the recorded times for traveling the 500-meter course (marker 24 to 34) and the obstacle speeds from the times for traversing the 50 meter distance between the end pairs of obstacles. The obstacle time was subtracted from the course time to compute the approach speed. The boxplot figure shows the median, the “hinges” (first and third quartiles), and the maximum and minimum values that are not outliers, for the distribution of the approach and obstacle speeds. The figures show that the distributions are fairly normal. Table 6 lists the corresponding statistics for reference. There is no significant correlation between the approach speed and that for the obstacles. One barrel strike was recorded.
Table 6. Descriptive statistics for the road speed (km/hr).
Station Approach Obstacles
Percentile 25th 50th (Median) 16.40 17.90 10.00 14.40
75th 18.35 19.57
A nonparametric, repeated measures Wilcoxon Signed Ranks Test shows no statistically significant differences among the distributions. A nonparametric test was applied because of the large differences in the size of the variances for the two distributions. Of the seven participants, five slowed down when approaching the obstacles and two speeded up. Applying a parametric paired sample t-test as a check on the results shows no significant differences in the two distributions. Participants’ Evaluation: The participants were asked to rate the VTT for the task of obstacle avoidance. The ratings are on a 1 to 7 point, bipolar scale with the verbal end anchors of “less” for the 1 rating and “more” for the 7 rating. Considering the small sample size of six participants who elected to answer the questionnaire, we summarize the results by reporting the rating average and range. The participants rated their ability to see the road ahead as high (average rating: 5.80, range: 5-6), as was
41
their ability to notice changes in the roadway (average: 5.17, range: 4-6); however, they were not as able to see the road edges (4.6, 3-6) although the sides of the road were fairly visible (4.83, 4-6). They rated their ability to judge the distance to the obstacles as moderate (4.5, 1-6), as was their ability to see the obstacles as they passed them (5.33, 3-7). One participant recommended a safe road speed of 15 MPH while another 20 to 25 MPH to reduce vibrations. Several reported a lag in control response time that caused over steering. Another reported that the lack of depth perception made it difficult to tell distance to the obstacles. Two participants reported symptoms associated with motion sickness, one a slight headache because of the heat and camera vibrations, and the other general discomfort. Discussion: Tactical vehicles operate for a significant amount of time on the road and among local and military traffic. They must be able to deal with this traffic and other obstacles they encounter at a level of safety and efficiency at least consistent with current military vehicles. Although the data show that the participants on the average slowed down to 85% of their road course speed as they passed through the obstacle course, the differences are not statistically significant. Noteworthy is the fact that in seven runs one barrel was hit. This would be an unacceptable 14.3% accident rate per obstacle encountered. The test course was designed as a static representation of what is likely encountered on the highway. The performance would likely be worse if the obstacles were moving. In summary, the system allowed the driver to detect and react to perform maneuvers around the static obstacles. The current configuration of cameras and displays may provide insufficient situational awareness for a driver to operate at an acceptable level of safety on a road populated with static and dynamic obstacles such as local or military traffic. Under the current configuration the system may only achieve a level of performance comparable to a vehicle operating in a mode such as buttoned up or having the crew in protective masks.
42
Camp Grayling Test Area
34
Range Gate
33
End Point
25 Meters
55 gallon barrels
32
25 Meters
Start Point 4.7 Meters 24
Test Vehicle
Figure 22. Road obstacle negotiation test
43
24
Speed (km/hr)
22
20
18
16
14
12
10
8 6 Obstacle
Approach Course condition Figure 23. Speed of travel.
44
Tactical Road Following As a measure of the ability to perform tactical operations, the driving speed of the Vetronics Technology Testbed (VTT) was determined in both the daytime and nighttime FLIR cameras modes. Purpose: The purpose of this study was to measure the speeds at which the participants navigated a cross-country treecovered dirt road with the camera systems, and to determine if there is a statistically significant difference between the systems. Method: The participant drives the VTT on the course in one direction with the daytime camera and in the opposite direction with the FLIR camera, in a manner counterbalanced across participants. Facility: A winding dirt trail through a forest area was used as a tactical course. The course is 2.15 kilometers long; markers were placed at the both ends of the course. See Figure 24 for a map section of the course. Figure 25 is a photograph of the VTT on the course. Experimental Design: The design is a within subjects single fixed factorial experiment with repeated measures on the day and FLIR cameras, and the participants as a random factor. Measures: The independent measures were the daytime and FLIR camera systems; the dependent measures were the course times for day and FLIR cameras, and technical questionnaire ratings. Procedure: The participant was allowed to familiarize himself with the route using a map and aerial photo; however, he was not allowed to practice on the test route. A HMMWV was driven over the course to ensure that no other traffic was present. The safety officer positioned the test vehicle at the start point marker and ordered the participant to move out as fast as he wanted to without exceeding 20 MPH using the designated camera system. The participant was not being evaluated on land navigation and the safety officer provided enroute navigation information to the driver but let him make all driving decisions unless the situation approached an unsafe condition. The safety officer stopped the vehicle when it reached the end of course. After a brief rest, the safety officer turned the vehicle around and the participant was directed to drive back to the starting point with the other camera system. The onboard experimenter recorded the time to drive the course. The participants drove the VTT on the course in one direction with the daytime camera and in the reverse direction with the FLIR camera. The order in which the cameras were used was counterbalanced across all participants. That is, while the first participant of a set of two drove from start to finish first with the daytime camera and then in the reverse direction with the FLIR, the second participant drove first with the FLIR and then in the reverse direction with the daytime camera. Instructions to Participants: At the command of the safety officer, drive the roadway as fast as you safety can; however, do not exceed 20 MPH. The safety officer will direct you to the correct path at road turns. Continue to drive until the safety officer tells you to stop. The experimenter will measure the route time. Statistical Results: Figure 26 is an Exploratory Data Analysis (EDA) box-and-whisker plot (Velleman & Payne, 1992) of the road course time in minutes for the two camera systems. The box plot figure shows the median, the “hinges” (first and third quartiles), and the maximum and minimum values, for the distribution of each camera. The figures show that the distributions are fairly normal without outliers or extreme values. Table 7 lists the corresponding statistics for reference. A parametric, paired sample t-test shows statistically significant differences among the distributions (t-value = -2.340, N = 8, df =7, p < 0.052). Application of a natural logarithmic transformation to the time data improves the distributions further and strengthens the statistically significant differences (t-value = -2.373,N = 8, df =7, p < 0.049). Participants’ Evaluation: The participants were asked to rate the VTT for the task of tactical road following. The ratings are on a 1 to 7 point, bipolar scale with the verbal end anchors of “less” for the 1 rating and “more” for the 7 rating. Considering the small sample size of eight participants who elected to answer the questionnaire, we summarize the results by reporting the rating average and range. We report the ratings first for the day cameras and then those for the FLIR cameras. For the day cameras, the route followed was rated as moderately apparent as seen on the displays (average rating: 4.62, range: 3-6). They felt that were moderately successful in navigating through the turns (average: 4.5, range: 3-6); however, there were a few turns that were particularly difficult to navigate (3.5, 2-5). They felt moderately safe driving with the camera system (5.25, 3-7). In contrast, the ratings were higher for the FLIR cameras. For example, the route followed was rated as more apparent on the FLIR displays (average rating: 5.8, range: 4-7). They felt that were more successful in navigating through the turns (average:
45
Table 7. Descriptive statistics for the course times (minutes).
Camera Day FLIR
25th 6.39 6.89
Percentile 50th (Median) 6.87 8.44
75th 7.73 9.48
5.33 , range: 4-6); however, there were more turns that were particularly difficult to navigate (5.0, 4-6). They felt safer when driving with the FLIR camera system (6.0, 5-7). Several participants mentioned that the visibility with the day camera was decreased when passing through the woods since the combination of sunlight and shade in the image made objects hard to see. This was caused by the automatic adjustment of the iris in the cameras in bright sunlight. At one point during a trial, a participant had to stop the vehicle to study the scene before proceeding slowly forward because the sunlight filtering through the overhead branches washed out the image of the roadway in the tree shadows. This was not a problem with the FLIR camera and the terrain was easy to discern. One participant reported that he needed to travel at a slightly slower speed with the FLIR since the resolution was lower than with the day camera. At certain speeds, the vehicle vibrations made the displays hard to read with the day cameras but not the FLIR. The steering was jerky pulling to the left and the response lag induced over steering. Again, three participants reported mild symptoms of motion sickness including a headache and stomach awareness, which occurred mostly during the turns and near the end of the trial. In a follow on evaluation of the cameras, displays and controls, the participants rated the camera placement as being relatively effective for this task (5.83, 4-7), a rating compatible with those for the assembly area operations. Similarly, the placement of the displays was rated fairly effective (5.83, 3-7) as was the resolution (5.17, 3-6) and the brightness and colors (5.83, 5-6) of the displays. There were some distortions in the display scene (3.50, 2-5). However, the difficulty with the lack of depth perception (2.67, 2-4) was less than that for the assembly area, while that with the display vibrations (3.83, 1-6) was more. The steering controls were as effective here as for the assembly area (5.33, 5-7); while the accelerator and brake pedals were more so (5.33, 4-6). As with the assembly area, most did not feel that they had to think about the task (3.0, 2-5) and were fairly aware of the surrounding situation (5.83, 5-6). However, while four reported no motion sickness, two did report slight to moderate motion sickness (1.83, 1-5). As with the assembly operations, comments mentioned small blind spots between cameras, that the front display vibrated when accelerating, and too much sensitivity and delayed response. In addition, scene continuity was broken by bezels between displays and transition of the view across displays was difficult. While the brightness and colors of the displays were satisfactory, the shadows and sunlight glare made navigation difficult. The perceived speed as seen from the displays was faster than the actual speed. The lack of a vehicle reference in the camera view made it hard to judge the closeness of objects, such a trees along the roadside when in a turn. As with any M2 chassis the vehicle would vibrate in resonance in the 14-18 MPH range and all displays would vibrate making viewing difficult. The steering controls exhibited a response lag between turn execution and the movement of the vehicle. Further, the pedals were not as sensitive as they should be. Finally, one participant reported a feeling of isolation caused by the engine noise and dissociation from the outside. Discussion: Table 8 lists the equivalent speed of travel in km/hr given a 2.15 kilometers long course. While the course times are significantly different statistically for the two camera systems, there is little operational difference in speed of travel. For example, the median speed with the FLIR camera is 3.5 km/hr slower (2.2 MPH) than that for the daytime cameras. This slower speed may have been due to the lower resolution of the AN/VAS-5(V) Driver’s Vision Enhancer FLIR system (320H by 240V, ¼ VGA) and the decreased brightness contrast (8 levels).
Table 8. Equivalent road course speed (km/hr).
Camera Day FLIR
25th 20.18 18.72
Percentile 75th 50th (Median) 18.78 16.69 15.28 13.61
46
What is also noteworthy is that in neither treatment did the subjects approach the 32.18 km/hr (20 MPH) safety speed limit. A driver who had driven the course many times demonstrated that the vehicle could go this fast. For example, one of the safety officers drove the course at 32.18 km/hr (20 MPH) in 4 minutes from his position in the driver’s seat of the VTT with the hatch open. Whether the limited speeds used to traverse the course are a significant function of the indirect driving system, or merely a lack of familiarity with the course, or some other irrelevant limiting factor, cannot be determined by the available data. In summary, under the limiting conditions of speed and workload imposed in this experiment, the system provided the crew an ability to maneuver across country without physical or mental discomfort. However, the reduced resolution of the indirect vision system may be causing the crew to limit their speed of travel.
47
Camp Grayling Test Area
Point A
Range Gate
32
400M 24
Point B
Figure 24. Cross Country Driving Course
48
Figure 25. VTT on tactical endurance course.
49
Route travel time (minutes)
11
10
9
8
7
6
5 DAY
FLIR
Figure 26. Course travel times
50
Cameras
QUESTIONNAIRES The results from the application of the workload questionnaires and the exit questionnaires including the technological evaluations are reviewed in this section. Workload Questionnaires Here, the results are reviewed from a battery of questionnaires for measuring the subjective effects upon different aspects of the task workload, included the following: Subjective Stress scale, Allocation of task attention, NASA Task Load Index, Situation Awareness Rating Technique, Cognitive Compatibility, and Motion Sickness. These questionnaires were applied two times, at the end of the assembly area phase and following the tactical operations. We present the results for each of these questionnaires; however, we first discuss the limitations on statistical analysis. The subject population is too small in number for valid analysis of the workload differences between the assembly area tasks and those of the road operations. This is true since one participant failed to complete any of the workload questionnaires and another participant, while completing those for the assembly area, did not complete those for the tactical operations. The result is that questionnaires from only six participants were available for a within subjects statistical analysis. The small sample size leads to a low statistical power. This is especially true, since the distributions tend to be nonparametric and the statistical test that would be applied is the Wilcoxon Signed Ranks Test for two related samples. Furthermore, the application of the questionnaires was not counterbalanced across participants because of the nature of the experimental procedures, and any statistical differences may be caused to learning or fatigue effects over time and not just the different natures of the tasks. For this reason, we summarize the statistics for the two study regimes and draw comparisons rather than attempt a statistical test. Here, we are summarizing the scores reported by seven participants for the assembly area and those reported from six of those participants for the road operations. Subjective Stress scale: Although the subjective stress was low in both operations, the assembly area sustained a slightly higher reported stress than did the road operations. Following the Kerle and Bialek (1958) scoring of subjective stress, the median score for the assembly area is 17 with a corresponding verbal anchor of “comfortable,” and the quartile scores extend from 9 for “fine” to 48 for “indifferent.” Similarly, the median score for the road operations is 9 (“fine”), and the quartile scores extend from 6.75 (better than “fine”) to 52 (approaching “timid”). See Figure 27 below for box plots.
Verbal Anchors
70
“unsteady”
Subjective Stress Score
60
“timid”
50
“indifferent”
40
“not bothered”
30
“steady”
20
“comfortable”
10
“fine” “wonderful”
0 Road operations
Assembly area Study Phase
Figure 27. Subjective stress scores for assembly area and road operation study phases
51
Allotment of task attention: As can be seen from Table 9, the median scores for allotting the task attention to the processing channels are relatively consistent across study phases for all channels except the psychomotor, which has a higher score for the assembly area than the road operations. Referring to the corresponding verbal anchors for the scores, the median visual attention corresponds to that demanded by a “locating and alignment” activity, the cognitive attention to “evaluation and judgment,” and the auditory attention to a “selective orientation to sound,” which would be in response to the instructions from the safety driver. However, the median motor attention for the assembly area corresponds to a “discrete activation,” while that for the road operations corresponds to a less demanding manual “manipulation.” This is reasonable since in the assembly area operations the driver made discrete adjustments in response to the hand signals of the ground guide, while in the road operations the driver was continuously adjusting the controls in response to changes in the road scene and occasional directives from the safety driver. Figures 29 through 31 are box plots for the task attention loading scores for the visual, cognitive, auditory, and psychomotor channels as rated by the participants for the assembly area and road operation study phases.
Table 9. Frequency distributions of scores for allotting task attention
Visual attention loading score
25% quartile median 75% quartile
6.5
Visual Assembly Road 4.000 4.750 5.000 5.100 5.900 5.475
Processing Channel Cognitive Auditory Assembly Road Assembly Road 3.000 3.925 4.000 2.975 4.400 4.000 4.000 4.250 4.900 5.082 4.500 5.000
Motor Assembly Road 4.000 3.775 5.600 4.750 5.850 5.250
Note: * - extreme value O – outlier value
Verbal Anchors 6.0
“read”
5.5
“track”
5.0
“locate”
4.5
“inspect”
4.0
“discriminate”
3.5 Road operations
Assembly area Study Phase
Figure 28. Visual task attention loading scores for assembly area and road operation study phases
52
Cognitive attention loading
6.5
Note: * - extreme value O – outlier value
Verbal Anchors
6.0 5.5 “recall”
5.0 “evaluate”
4.5 4.0
“recognize”
3.5 3.0 2.5 Road operations
Assembly area Study Phase
Figure 29. Cognitive task attention loading scores for assembly area and road operation study phases
Note: * - extreme values O – outlier values
Auditory attention loading
6
Verbal Anchors “interpret”
5
“verify” “attend”
4
3
2
“orient”
1
“detect”
0 Road operations
Assembly area Study Phase
Figure 30. Auditory task attention loading scores for assembly area and road operation study phases
53
Motor attention loading score
Verbal Anchors “writing”
6.5 6.0
“discrete activation”
5.5 5.0 “manipulate”
4.5 4.0 3.5 3.0 “continuously adjust”
2.5 Road operations
Assembly area Study Phase
Figure 31. Psychomotor task attention loading scores for assembly area and road operation study phases
NASA Task Load Index: As can be seen from Table 10, the median scores for the NASA task load index are relatively consistent across study phases. This is true for the total scores and also the scores for the demand and interaction components. Figure 32 shows box plots for loading scores for the components as a function of the assembly area and road operation study phases. The figure shows consistent median values with roughly the same variances across the study phases. Similar comments apply to the dimensions making up the components, except that the mental demand and interactive effort are higher for the assembly area, while the physical demand is lower. However, these differences are minor, and the implication is that the participants felt that they had experienced the same low workload in both study phases. For reference, Figure 33 shows box plots for the dimensions loading scores as a function of the assembly area and road operation study phases.
Table 10. Frequency distributions of scores for NASA task load index
25% quartile median 75% quartile
Total Assembly Road 5.800 6.575 9.000 10.250 31.800 23.225
Components . Demand Interactive Assembly Road Assembly Road 2.600 3.375 3.200 3.375 5.000 4.400 4.900 6.000 17.800 10.600 14.000 12.625
54
Task load index component scores
Note: * - extreme value O – outlier value
30 maximum
Interaction
Demand 20
10
0 Road operations
Assembly area
Road operations
Assembly area
Study Phase
Study Phase
Figure 32. Task load index component scores for assembly area and road operation study phases
mental
Task load index dimension
INTERACTION
DEMAND
10
physical
temporal
performance
effort
frustration
maximum 8
Note: * - extreme values O – outlier values
6
4
2
0 area
road
area
road
area
road
area
road
area
road
area
road
Study phases
Figure 33. Task load index dimension scores for assembly area and road operation study phases
55
Situation Awareness Rating Technique: As can be seen from Table 11, the median scores for the situational awareness are higher for the assembly area. Here, in agreement with Taylor and Selcon (1994), situational awareness is computed as the sum of the supply and understanding components minus the demand component. Similarly, the supply and understanding scores are higher for the assembly area, while the demand is lower. Figure 34 shows box plots for loading scores for the components as a function of the assembly area and road operation study phases. Similar comments apply to the dimensions making up the components with higher instability and complexity but lower variability for the assembly area along with more concentration and familiarity with the information received. However, these differences are insignificant, and the implication is that the participants felt that they had experienced the same awareness ratings in both study phases. For reference, Figure 35 shows box plots for the dimensions rating scores as a function of the assembly area and road operation study phases.
Table 11. Frequency distributions of situational awareness rating scores
25% quartile median 75% quartile
Total Assembly Road 24.000 21.750 26.000 23.500 31.000 24.750
Demand Assembly Road 7.000 8.500 9.000 10.000 12.000 14.500
Components Supply Assembly Road 12.000 17.000 21.000 18.500 24.000 21.750
. Understanding Assembly Road 17.000 12.750 19.000 14.000 19.000 19.000
30 Situational Awareness rating component scores
maximum
Note:
O – outlier value
maximum
maximum 20
10
Supply
Demand
Knowledge
0 Assembly Study Phase
Road
Assembly
Road
Study Phase
Assembly
Road
Study Phase
Figure 34. Situational awareness rating component scores for assembly area and road operation study phases
56
DEMAND Instability
SUPPLY
UNDERSTANDING
Capacity Concentrate Division
Variability Complexity Arousal
Quantity
Quality
Familiarity
Situational Awareness rating dimension
7
maximum
6
5
4
3 Note: O – outlier value
2
1 area
road
area road
area road
area road
area road area road area
road area road
area road area
road
Study phases
Figure 35. Situational awareness rating dimension scores for assembly area and road operation study phases
Cognitive Compatibility: As can be seen from Table 12, the median scores for the cognitive compatibility components are higher for the assembly area at least for processing and knowledge activation; where the components are the sum of the dimensions aligned for the speed of response. That is, the level of processing is scored higher for a more natural, automatic, and intuitive response; the ease of reasoning is higher for a straightforward situation; and the knowledge activation is higher for a familiar setting. The implication is that the structure of the assembly area supported cognitive processing more than did the more fluid setting of the changing roadway. For reference, Figure 36 shows the box plots of the cognitive compatibility component scores for the assembly area and road operation study phases.
Table 12. Frequency distributions of cognitive compatibility components
25% quartile median 75% quartile
Processing Assembly Road 22.000 21.750 29.000 27.500 33.000 30.250
Reasoning Assembly Road 16.250 18.500 22.500 23.000 29.000 32.750
57
Knowledge Assembly Road 12.000 14.000 18.000 15.000 21.000 19.750
Cognitive compatibility components scores
40 maximum
maximum Note: * - extreme value O – outlier value
30
maximum
20
10
Processing
Knowledge
Reasoning
0 Assembly
Road Study Phase
Road
Assembly
Study Phase
Assembly
Road
Study Phase
Figure 36. Cognitive Compatibility component scores for assembly area and road operation study phases Motion Sickness: As can be seen from Table 13, the median scores for the total severity and the oculomotor symptom are higher than zero for the assembly area; in turn, the median scores are zero for the nausea and disorientation symptoms in the assembly area and for all symptoms in the road operations. The implication is that on the average, the participants experienced an oculomotor based symptom of motion sickness during the assembly area operation. The source of this effect may have been the need to closely study the video display to determine the hand signals of the ground guide and to navigate the vehicle in close quarters. However, the scores are relatively slight since the maximum scores possible are 120 for total severity, 200 for the nausea symptom, 160 for the oculomotor symptom, and 300 for the disorientation symptom (Kennedy, Lilienthal, Berbaum, Baltzley & McCauley, 1989). Some participants experienced no motion sickness while for others all symptoms were experienced in both operations. That the symptoms are a measure of motion sickness is shown by the strong correlation with an independent rating by the participants of motion sickness on a 1 to 7 point, bipolar scale with verbal end anchors. For the assembly phase, the nausea symptom is significantly correlated with the ratings (Pearson Correlation = 0.772, N = 7, p = 0.042); while for the road phase, the total severity and all symptoms are significantly correlated with the ratings (total severity: Pearson Correlation = 0.954, N = 6, p = 0.003; nausea: Correlation = 0.965, p = 0.002; oculomotor: Correlation = 0.915, p = 0.011; and disorientation: Correlation = 0.915, p = 0.011).
Table 13. Frequency distributions of motion sickness symptoms and total severity
25% quartile median 75% quartile
Total Severity Assembly Road 0.000 0.000 5.740 0.000 15.480 31.015
Nausea Assembly Road 0.000 0.000 0.000 0.000 38.160 66.780
58
Components Oculomotor Assembly Road 0.000 0.000 15.160 0.000 30.320 4.750
. Disorientation Assembly Road 0.000 0.000 0.000 0.000 27.840 4.750
For reference, Figure 37 shows the box plots of the sickness symptom scores for the assembly area and road operation study phases. A close study of the data shows that five participants reported some symptoms for the assembly area and three of those five symptoms for the road operation. These symptoms include general discomfort, eyestrain, difficulty focusing, sweating, nausea, difficulty concentrating, fullness of head, and stomach awareness, among others. Although for some there may have been other causes, still these symptoms may lead to motion sickness. However, the symptoms were not severe enough to cause the participants to suspend operations.
Motion sickness scores
80 Nausea
Oculomotor
Disorientation
60
40
20
0 Assembly
Road
Study Phase
Road Assembly Study Phase
Assembly
Road
Study Phase
Figure 37. Motion sickness symptom scores for assembly area and road operation study phases
Correlation: Application of the Pearson Correlation bivariate correlation analysis shows statistically significant correlation between the total severity of motion sickness and the nausea symptom (Pearson Correlation = 0.935, N = 13, p < 0.001), and between the oculomotor and disorientation symptoms (Pearson Correlation = 0.665, N = 13, p = 0.013). Similarly, there is significant correlation between the total severity and the nausea symptom, with the stress (Pearson Correlation = 0.925, N = 13, p < 0.001), total task load index (Pearson Correlation = 0.656, N = 13, p = 0.015) and the demand (Pearson Correlation = 0.666, N = 13, p = 0.013) and interaction (Pearson Correlation = 0.620, N = 13, p = 0.024) components, and the situational awareness demand (Pearson Correlation = 0.693, N = 13, p = 0.009). In turn, the stress, total task load and components, and the awareness demand are all significantly correlated with each other (p < 0.004 level). The awareness demand is significantly correlated with the supply (Pearson Correlation = 0.781, N = 13, p = 0.002), but neither with the understanding. The processing component of cognitive compatibility is significantly correlated with the knowledge component (Pearson Correlation = 0.745, N = 10, p = 0.013), and with the task visual attention (Pearson Correlation = -.638, N = 11, p = 0.035) and motor attention (Pearson Correlation = -.700, N = 11, p = 0.016). Finally, the reasoning component of cognitive compatibility is significantly correlated with the total task load index (Pearson Correlation = -.676, N = 10, p = 0.032), through the interaction component (Pearson Correlation = -.696, N = 10, p = 0.026), and with the SART supply component (Pearson
59
Correlation = -.723, N = 10, p = 0.018). The correlation matrix is used in the calculation of the factorial analysis described in the next paragraph. Factorial analysis component space: Although the number of participants is limited, still it is instructive to perform a factorial analysis on the mental workload measures. Figure 38 is a component plot in rotated space for reducing the mental measures to two components using principal component analysis as the extraction method (58.31% total variance explained), and Varimax rotation with Kaiser normalization. Interesting enough, the distribution of the measures suggests a form of cognitive processing space with the task attention focus well separated from the cognitive demand loading and supply. The task focus consists of the attention allocated for visual, cognitive, auditory, and motor processing. In turn, the cognitive demand clusters together the task load index demand and interaction components with the SART demand, the total severity of motion sickness, and the stress. Although located well outside the region of task focus, the cognitive support consisting of the cognitive compatibility component and the SART supply and understanding components, is more widely dispersed over the cognitive processing space. Other studies with larger data pools have shown that the cognitive region takes a more definite shape with a three-component space (Smyth, 2002). 1.0 cognitive
Allocated attention
Component 2
motor
Motion sickness visual
TLX SART stress
auditory
.5
CC-SART
Task focus
supply interaction 0.0 demand demand understanding knowledge
severity
reasoning -.5
Cognitive demand
processing
Cognitive support
stress
SA
-1.0 -1.0
-.5
0.0
.5
1.0 Component 1
Figure 38. Factorial component plot in rotated space of mental workload measures
In the next section, we review the results of the exit questionnaire and the final technological evaluation of the VTT and the components.
Exit Evaluation The participants evaluated the VTT in an exit questionnaire in which they rated questions on the quality of the display images from the daytime and FLIR cameras, the usability of the control handles and pedals, the amount of manual activity
60
required, the driver’s mental workload, and the task performance of the vehicle. The ratings are on a 1 to 7 point, bipolar scale with verbal end anchors of “less” for a 1 rating and “more” for a 7 rating. The manual activity included the eye, head, steering, and pedal activity. The mental workload covered the perceived mental loading, stress, motion sickness, and situational awareness. A rating for the separate categories was obtained by averaging the ratings for the applicable questions. Figure 39 shows the box-plot distributions for the separate categories. In general, the ratings are favorable since the vehicle display image quality and controls are highly rated along with the vehicle performance. In contrast, the mental workload is low and while driving required more activity, this may have been because of the assembly area operations. A study of the ratings for the individual questions shows these ratings are consistent for both the daytime and FLIR camera systems. For the controls, the participants rated the control response to be lower than the ease of use. In turn, the physical activity required to operate the controls and displays shows that while little head movement was used, the amount of eye, arm, and foot movements was higher. The implication is that they operated mostly from the scene on the central display. A study of the mental workload shows that while little motion sickness was felt, the situational awareness was judged high possibly because of the wide field of view provided by the displays; furthermore, the mental loading and stress were low. Finally, while the vehicle speed was rated low, the accuracy and performance were rated high. However, the speed for the assembly area was intermittent because of the control by the ground guide and that for the road operations restricted by range safety.
Note: O – outlier value
maximum
Average ratings
7
6
5
4
3
2
1 Day cameras
FLIR
controls
physical
mental
Driver’s workload
Vehicle displays & controls
Task performance
Figure 39. Exit Questionnaire responses
Technological Evaluations The participants were asked to rate the VTT on such factors as communications, environmental awareness, maneuverability, crew interaction, and the crew station design. In addition, they were asked to compare the VTT to the BFV for maneuvers, road convey, and tactical operations. The ratings are on a 0 to 7 point, bipolar scale with verbal end anchors. For example, a usefulness scale extends from “low” at 0 to “high” at 7, and an ease-of-use scale extends from “difficult” at 0 to “easy” at 7. Considering the small sample size of seven participants, we summarize the results by reporting the rating average and range.
61
Communications: The participants rated the ability to communicate verbally and visually within the vehicle and with dismounted personnel within the vicinity. They rated the internal verbal communication as highly useful (average 6.0, range 2 to 7) and easy to use (5.8, 2 - 7). However, the ratings for the ability to communicate visually within the vehicle and with outside personnel were more evenly spread over the range. The ability to communicate visually within was moderately useful (4.0, 0-7) and not necessarily easy to use (4.3, 0-7). Similarly, the ability to communicate with external personnel was not necessarily easy to use (4.0, 0-7). Actually, during trials the internal voice communication was with the safety driver over the intercom of the vehicle; the participant could not see the safety driver in the M2 driving position or the experimenter in the back since the crew stations were in line. The experimenter would move forward to the participant’s driving station between trials to apply questionnaires. The only way that the driver could communicate with external personnel was by seeing them on his visual display as for example, when observing the hand signals of the ground guide. Environmental Awareness: The participants rated the ability to determine the vehicle’s location, orient the vehicle, and navigate a road course. The ability to determine the location was rated highly useful (5.8, 3-7) and fairly easy (5.7, 3-7) to perform. In turn, the ability to orient the vehicle was rated highly useful (6.1, 3-7) and very easy to perform (6.0, 3-7). Finally, the ability to navigate the course was rated highly useful (6.1, 4-7) and easily performed (5.8, 4-7). The participant could determine the location from the onboard digital terrain map, and could orient the vehicle from the map and from a visual compass overlaid on the video displays. He navigated from the road scene on the displays. Maneuverability: The participants rated the ability to maneuver, see the terrain, control the vehicle, receive feedback, and on the design of the control handles. The ability to maneuver was moderately useful (5.1, 3-6) and somewhat easy (4.7, 3-7) to perform. They rated the ability to see the terrain as moderately useful for maneuvering (5.1, 3-6) and relatively easy (4.5, 3-6). The ability to control the vehicle was moderately useful (5.6, 4-7) and relatively easy (4.1, 3-7). The feedback was moderately useful (5.7, 3-7) and easy (5.3, 3-7) to perform. Finally, the control handle design was rated moderately useful (5.7, 4-7) and relatively easy (5.0, 4-6) to use. The feedback was limited to the visual scene, the vehicle vibrations through the crew station seat, and the engine noise. There was no force feedback through the control handle since the control system was based on a drive by wire mechanism. Crew Interaction: The participants rated the team communications and crew burden for the assembly area operation. The communications were somewhat effective (5.4, 2-7), where the rating range is from “ineffective” at 0 and “effective” at 7. The crew burden was rated somewhat light for the ground guide (4.8, 4-6) and the driver (4.5, 3-6). One observer mentioned that the sound quality seemed adequate since he was able to hear both of the other people in the vehicle and the auditory alerts. In his opinion, the “push to talk” button on the helmet was not an optimal placement for that control and it should be placed somewhere easy to get to, perhaps on the steering yoke. Crew Station Design: The crew station was rated on the functionality of the controls and displays, and on the quality of the seats and ventilation. The participants rated the controls as being very easy to reach (6.8, 6-7), they could easily see all displays (6.4, 4-7), and the controls and displays were well grouped by function (6.5, 6-7) and sequence of operations (6.3, 57). Considering the map and navigational features, the map display had good resolution (6.0, 4-7), as did the text on the map (6.0, 5-7). The map scale was easily controlled (6.3, 5-7), and the screen control buttons were place properly (6.0, 6-6) and easy to use (6.2, 6-7). The information displayed is of value (5.8, 5-7) and the map displays have a high correlation with the driving scene (5.7, 6-7). Considering the displays, the driving screen was highly useful (6.3, 4-7) and easy to use (6.1, 4-7), while the system status display was moderately useful (6.1, 5-7) and fairly easy to use (6.0, 5-7). The task burden was fairly light for driving (5.0, 0-7), communications (5.7, 4-7), and navigating (4.7, 2-7), where the “light” is 7.0 and “heavy” is 0. The participants rated themselves as being fairly confident that they could operated the vehicle (5.3, 4-7),and felt fairly comfortable in doing so (5.6, 3-7). They rated the ride as being comfortable (6.0, 5-7), and the displays as being easily read while on the move (5.8, 5-7). While two of the participants rated the motion sickness as being fairly acute, five reported no motion sickness. Similarly, three reported moderate eye-strain, while four reported minimal or none. The crew station lighting was rated good (4.8, 4-7); however, the air quality was moderate to poor (2.8, 0-5). The seating arrangement was rated highly useful (5.4, 4-6), easily used (6.0, 4-7), and very comfortable (5.7, 5-6); however, with moderate vibration (4.6, 2-7) where high vibration was 0 and low was 7. Finally, the crew station was rated as being moderately effective on the battlefield (5.3, 37). Comparison of VTT to the BFV: While three of the participants preferred the VTT for maneuvers and road convoy operations, the remaining four preferred the BFV. In contrast, four preferred the VTT for tactical operations while three
62
preferred the BFV. One participant preferred the BFV for all operations, while another preferred the VTT. Comments made by the participants follow: (1) The participant preferring the BFV for all operations stated that this was because of his familiarity with the vehicle; however, he liked the displays, digital map display, and the FLIR camera displays for nighttime operations in the VTT. (2) In contrast, the participant who preferred the VTT for all operations did so because the vehicle was safe with few flaws and had high speed. He liked the digital map, the nighttime displays, and the fact that the driver was inside without his head out. (3) Another participant preferred the BFV in maneuvers because the BFV handled better and has mechanical feedback. However, the VTT is preferred for road convoys because of the wider field of view. Again, he preferred the vehicle for tactical operations because the VTT has a low silhouette. He disliked the lack of depth perception and reduced resolution, and the fact that the corners of the vehicle were out of view. (4) One participant preferred the controls and depth perception of the BFV for maneuvers and road convoy, but the FLIR displays of the VTT for tactical operations. As well as the FLIR displays, he liked the displays, navigation systems, and drive selection controls on the VTT. However, the VTT steering was jerky with over correction. He would like both front camera and rear cameras in view when in reverse. Also, the displays were distorted at times because of vehicle vibrations. (5) One participant preferred the BFV for maneuvers and road convoys because the vehicle was more responsive and has a greater field of view. However, he preferred the VTT for tactical operations because it was easier to judge distances. He liked the responsiveness of the VTT controls, the ride comfort, and the wide field of view. However, he disliked the fact that he experienced motion sickness during the tactical operations. He stated that the VTT technology is great, but expressed concern about the endurance on the battlefield. (6) One participant appreciated the fact that the crew is totally enclosed at all times in the VTT. However, he had difficulty driving the vehicle because of the delay between the control handle action and the vehicle response. (7) Finally, a participant preferred the VTT for maneuvers and the road convoy because of the quick reaction to command and the vehicle comfort. However, the BFV would be better for tactical operations because he had difficulty seeing through the woods with the VTT displays. Although the VTT seating and displays were comfortable, the vehicle needs to be air-conditioned. Climate conditions were poor inside the vehicle. The vehicle gets very warm with all of the electronics running. The displays vibrate when shifting gears and may be hard to read for that reason.
63
DISCUSSION While largely statistically insignificant because of the small sample size, the results of this study suggest implications for the technical design as demonstrated by the effects on the operational performance, mental workload and stress, and motion sickness. In the following, we discuss each of these topics in greater detail. Operational Performance: The VTT allowed the crew to perform most driving tasks without physical difficulty or excessive stress. The visual acuity with the indirect vision system was less than direct vision but likely did not significantly impact performance. The speed at which most tasks were accomplished was much slower than would be expected with a crew operating a standard BFV in the open hatch mode, at least in the opinion of a Subject matter Expert. The accuracy of maneuvering around close-in obstacles was also less than what would be expected. Some comments about the displays, controls, and seating follow. Displays- The need to scale the high-resolution video displays to the NTSC camera returns reduced the scene resolution and color contrast, at least for the daytime cameras. This was apparent in the scene image, which appeared slightly washed out and of medium resolution. Furthermore, vehicle vibrations are a likely source of image degradation. In the VTT, the picture quality of the displays was degraded between the speeds of 12 to 17 MPH by harmonic vibrations. At these speeds the displays and perhaps the cameras vibrated and shook so severely that it was difficult to drive. The picture was sufficiently stable at idle and low speeds under 10 MPH, and above 18 MPH. One observer noted that during vehicle vibrations, he “tunneled” his vision by concentrating on a very small part of the center display because of the poor picture quality. This action allowed him to follow the well-defined course road but would not have been safe for off road travel. He further reported that after his exposure he had a sort of motion sickness related eyestrain. For example, after leaving the vehicle, he had a hard time focusing his vision on a fixed object. When going inside a building, he had a hard time focusing on text on a computer screen. He reported a slight feeling of “fogginess” as sometimes accompanies a head cold, which lasted the rest of the day. Vibration analysis of the vehicle should allow the selection of vibration absorbing camera mounts or possibly active vibration dampening could be employed. Steering- The drive by wire control system using the steering yoke controller has certain limitations as implemented in the VTT possibly because steering in the M2 chassis is done with the transmission. There is no manual feedback and very little resistance to turning with the yoke. At very slow speed the steering action must be balanced with throttle to produce movement. When traveling faster the steering response is nonlinear and hard to judge since a slight movement of the yoke causes a slight steering change, but more yoke movement causes a disproportionately greater amount of vehicle steering change. This makes the driver over compensate and begin an oscillatory pattern. This relationship between the throttle and the steering must be learned. Since this a drive-by-wire system, it may be possible to develop a better steering algorithm that could be augmented by sensing the separate track speeds. This could give the driver a more linear control. A spring system could be employed on the yoke to provide the feeling of force feedback. Seating- In this study, the participants drove the vehicle from the front crew station, while the experimenter sat in the rear station. Outside of the natural harmonic speed range, the M2 chassis has a relatively smooth ride as far as shock is concerned. For this reason, the front driving seat had some vibration but it was not disturbing, and there is little shock and little “swing” movement. Since the front seat is close to the center of the vehicle, all major movements happen about the point of the operator and the seat is exposed to turns about the axis. In contrast, the ride in the rear seat was very different. The vibration was much more pronounced, to the point of being disturbing and the shock and swing movements were much greater. This is because the rear seat is almost at the very back of the vehicle. The vibration is amplified and instead of the vehicle rotating about the driver, it is swinging the driver in all directions. Measuring the vibration and accelerations at all of the proposed seating locations would aid in the selection of seating positions. Mental Workload and Stress: The mental workload and subjective stress were relatively low for these operations. Although statistically insignificant, the trends in the ratings suggest that the mental loading was higher during the assembly area than the road operations. For example, although the subjective stress was low in both operations, the stress is insignificantly higher for the assembly area than the road operations. As a measure of workload, the task load index ratings for the mental demand and interactive effort are higher for the assembly area, while the physical demand is lower. Further, the allotment of task attention to the psychomotor task channel has an insignificant higher score for the assembly area. The situational awareness is insignificantly higher for the assembly area; similarly, the supply and understanding scores are higher for the assembly area, while the demand is lower. The cognitive compatibility components are higher for the assembly area at least for processing information and the activation of knowledge. The implication is that the structure of the assembly area supported cognitive
64
processing more than did the fluid setting of the changing roadway. The total severity of motion sickness and the oculomotor symptom are higher for the assembly area; the implication is that the participants experienced an oculomotor based symptom of motion sickness during the assembly area operation. Finally, this implication is supported by the high correlation among the subjective stress, workload index demand and interaction, situational awareness demand, and total severity of motion sickness. Motion Sickness Comparison: Of interest is a comparison of the occurrence of motion sickness in this study to those in other studies on indirect vision driving that we have reported. Table 14 lists the vehicle type, display characteristics, horizontal fieldof view (HFOV), camera video return and downwards tilt angle, road course conditions and average speed, the number of incidences of critical motion sickness, and the prior experience of the participants, for two earlier studies as well as this study. A critical sickness incident causes the participant to abort the driving trial. In the first study listed, conducted by HRED at Aberdeen Proving Ground (Smyth & Whittaker, 1998; Whittaker & Smyth, unpublished), one of the eight participants was sick enough to abort a trial run. In that study, the participants drove a HMMWV around a serpentine course using a low resolution, liquid crystal helmet mounted display with a low video update rate and limited field-of view. The cameras were tilted downward 13-degrees from the horizon and had a NTSC video return. In the second study, again conducted by HRED at APG (Smyth, Gombash, & Burcham, 2000), two of nine participants were incapacitated by motion sickness. The participants drove a HMMWV around a serpentine course using medium resolution liquid crystal panel displays with a wide field-of view, but a low update rate. Again, the cameras were tilted downward 13-degrees and had a NTSC video return. In the third study (the current study, conducted at Grayling, MI and described in this report), while the participants because of their military background had some experience as drivers and maintainers of tracked vehicles, they had limited experience with the experimental configuration. The displays were high resolution with a wide field-of view and a high update rate. They were scaled to match the NTSC video return and the cameras were tilted downward at a 5-degree angle. The participants drove over relatively controlled roads with natural turns that you would find on a rural road network. While no critical incidences of motion sickness occurred, symptoms of motion sickness were reported in at least three of the eight participants. While the symptoms were mild, this is the frequency that was reported in the first and second studies. That the symptoms are a measure of motion sickness is shown by the strong correlation with the independent rating by the participants of motion sickness on a 1 to 7 point, bipolar scale that was reported above for this study (see “Questionnaires: Motion Sickness”). A research question remains as to the source of this effect. In the remainder of this section, we describe some potential sources that may apply to these studies.
Table 14. Critical Incidents of Motion Sickness for Indirect Vision Driving Experiments
Vehicle
Display Type Resolution Update Rate
. Camera Interlaced HFOV Type Tilt
Road . Conditions Speed
Critical Incidents*
Study 1. HRED @ APG, MD Summer 1996HMMWV HMD 185K pixels 30HZ Participants: Occasionally HMMWV drivers
yes
27°
NTSC -13°
berms, S-turns 10 MPH
1/8
Study 2. HRED @ APG, MD Summer 1999HMMWV panels 307K pixels 30HZ Participants: Occasionally HMMWV drivers
yes
110°
NTSC -13°
berms, S-turns 11 MPH
2/9
roadways
0/8
Study 3. TARDEC/HRED @ Grayling, MI Summer 2001VTT panels 1310K scaled 60HZ no 180° NTSC -5° Participants: Experienced Army Reserve & National Guard Bradley drivers
11 MPH
*Note: Number of sickness incidents causing driving to be aborted over total number of participants.
The prevalent theory of the cause of motion sickness is that of sensory conflict, where the visual system, the vestibular system, and the proprioceptors are in conflict with each other or with what is expected on the basis of previous experience. There are two main categories of sensory conflict: Either the information from the visual system and that from the vestibular system are incompatible with each other, or the information from the canals and otoliths within the vestibular system provide conflicting signals (Pausch, Crea, & Conway, 1992). The particular mechanics of indirect vision driving that may have
65
.
induced motion sickness are the display field-of view, image motion blurring, and the task environment, all which are described in greater detail below. Display characteristics: The wide horizontal field-of view used with the panel displays in the second and third studies, may have induced motion sickness because of motion in the peripheral vision. An analogous experience, simulator sickness generally occurs more frequently and intensely with a wider field-of view display since it provides more ocular stimulation (Kennedy, Fowlkes, & Hettinger, 1989; Scribner & Gombash, 1998). Another source of motion sickness may have been the lack of natural binocular stereovision and the accompanying depth perception, which causes a discrepancy between the scene and that expected from direct viewing (Pausch, Crea, & Conway, 1992). However, binocular stereovision that is artificially induced by binocular rivalry between offset images presented to both eyes, has been reported to increase simulator sickness in teleoperations (Scribner & Gombash, 1998). Presumably, this is because the stereo-optics is slaved to the vehicle and not to movements of the driver’s head as in natural vision. Image motion blurring: In the second study listed in Table 14, several participants reported incidences of motion sickness during a rapid turn or when going over the berms. This may be due to the block crystal realignment method of image refresh that is employed in Liquid Crystal displays. The display refresh used in that study could not keep up with the changing scene during a rapid turn and while going over the berms on the course. This resulted in the display appearing out of focus due to the temporary motion blurring of the video return with the accompanying loss of dynamic resolution. In some participants, this may have induced a lack of convergence accommodation that resulted in blur driven asthenopia symptoms. As reported in the literature, motion sickness especially asthenopia symptoms (Ebenholtz, 1992), can be produced by insufficiencies in visual stimulation. These symptoms can arise from lack of binocular convergence, inappropriate accommodative responses to blurred images, unequal image sizes in the two eyes, unequal focusing capability in each eye, and from inadequate fixation or pursuit responses. Furthermore, visual after effects consisting of illusory and unstable perception after exposure, have been reported to follow asthenopia symptoms (Ebenholtz, 1992). However, the update rate employed in the third study may have been high enough to overcome this effect for most of the participants. Task environment: The task environment for the indirect vision studies tends to increase sensitivity to motion sickness due to sensory deprivation while riding in an enclosed compartment. As noted above, the driver experiences physical isolation, darkness, heat, and noise with the indirect vision systems. The imposition of these conditions upon the participant may have been interpreted as a loss of control, a condition that increases the susceptibility to motion sickness (Pausch, Crea, & Conway, 1992). In summary, while the symptoms reported for the Grayling MI study were mild, motion sickness continues to be a problem for indirect vision operations. At the 1310K screen resolution (scaled to the NTSC video return) and 60 HZ update rate used in this study, LCD displays may still induce motion sickness, which in turn increases subjective stress. As well as vestibular system conflicts due to differences between the visual scene and head movements, another source of motion sickness may be a loss of convergence accommodation. This loss may be induced by temporal motion blurring of the video display during rapid changes in the scene such as occur during road turns. Associated with the indirect vision is an increase in both the workload and the demand on situation awareness. These increases are caused by an increase in mental and temporal demands on the cognitive facilities of the human driver. In turn, these may be due to the physical differences between the natural viewing and indirect vision with the decreased resolution of the displays, and lack of stereooptics. Over time, the increase in workload can lead to fatigue and errors.
66
CONCLUSIONS The crew of the VTT was able to perform driving tasks without physical difficulty or excessive stress. However the visual acuity with the camera system was less than that for direct vision, and for this reason the cameras degraded performance. Due to a combination of factors including lack of depth perception and limited camera view, the speed with which most tasks was performed was much slower than what would be expected out of a crew (commander and driver), operating a standard BFV in the open hatch mode. The accuracy of maneuvering around close-in obstacles was also less than what would be expected, at least in the opinion of a Subject Matter Expert. The performance with the VTT is comparable to that with a BFV operated in a night mode, buttoned up or with the crew wearing protective masks. The visual acuity study shows that the cameras limit the display resolution and using a higher resolution camera with 630 by 548 television lines (TVL), will allow viewing that is closer to natural acuity. The SXGA displays presently used have to be scaled down to match the low resolution of the NSTC cameras. The use of a higher resolution camera should improve scene resolution enabling road signs and scene details to be detected and processed at a farther viewing distance and greater speed. The placement and configuration of the driving cameras in the present design may not support the full spectrum of driving environments. The extended view used for driving and the inability to pan and tilt the camera system limits the ability to maneuver in close areas. For example, in the clearance judgment study the ground track was not visible to the immediate front and the participant had to estimate the course from the post markers. Although he would lose sight of a marker set as he approached them, the next set remained in view. As he closed on them, the posts would briefly reappear in his side displays for several seconds; however, the view was too brief to be helpful. The participant estimated his passage from the movement of the posts across the scene and the extended track print on the central display. However, the process increases the mental load since the participant had to extrapolate the clearance from the separation of the posts in front of him. The addition of side cameras mounted at the rear of the vehicle and pointing along the sides and slightly downward would provide a view of the tracks on the vehicle. The driver could then switch from a forward view to the view of his tracks on the side displays for driving in close quarters. During close-in maneuvers such as parking and backing, the ability of the driver to see the ground guide and his hand signals through the vision system is limited. One source of difficulty may be the reduced resolution of the cameras on the VTT. This makes the determination of a hand signal by the driver difficult causing slower responses and the repetition of signals. The use of luminous gloves or wands by the ground guide may increase the visibility of the hand signals. Another option is the use of a wireless intercom system that would allow the driver to communicate verbally with the ground guide. The ability of the driver to determine separation distance when in a convoy is limited. The indirect vision system may degrade the ability of the driver to judge distance and react to speed changes. The participants may have had difficulty detecting the speed changes either due to the reduced display resolution or perhaps because of dust generated from the lead vehicle. Similarly the participants may have had trouble judging distances because of the differences in the judgment relationship cues of the camera system as compared to those with direct vision. One possible solution for maintaining separation distances in dust conditions is to use the FLIR cameras. The FLIR system may also be used to detect turn signals, brake and taillights. In the obstacle avoidance study, the vision system allowed the participants to detect and maneuver around static obstacles in the roadway. However, the configuration of cameras and displays may provide insufficient situational awareness for the driver to operate at an acceptable level of safety on a road populated with both static and dynamic obstacles such as local or military traffic. System designers must consider the need for the system to operate in a full spectrum of environments to include on the road with noncombatant traffic. Important is the ability to detect and maneuver around obstacles under a greater variety of situations such as at night and low visibility due to obscuration, with both the daylight and FLIR systems. It is also important to test the ability to detect and maneuver around obstacles in a dynamic environment such as with moving highway traffic and the task of passing another vehicle. Although the difference between the road speeds is statistically significant, the average speed is practically about the same for the daytime camera and the FLIR camera operated in daytime. The system provided the crew an ability to move across country without difficulty in maneuver or physical or mental discomfort, under the limited speed and workload conditions of this experiment. However, some function of the indirect vision system may be limiting the speeds the crews choose to use. In the cross-country course, the vehicle obtained roughly one-half the speed with indirect vision that was obtained by the safety officer with the open hatch.
67
Although symptoms of motion sickness occurred for several participants, the occurrences were apparently not a problem under the experimental conditions. This may have been because of the use of relatively high-resolution LCD displays with a fast update rate (scaled for the NTSC camera video return), a wide panoramic field of view, an experienced pool of subjects, and possibly the benign experimental driving conditions. The test regime used to assess the VTT was designed to evaluate the ability to operate under the widest variety of situations within the given time and resource limitations. Because of the small sample sizes and the lack of a baseline measurement using a standard BFV, this study is a set of pilot studies from which definitive statistical conclusions cannot be made. However, the results may be useful for designing future experiments about the performance of the system since the data collected can be used to make estimates of the statistical power and the corresponding sample size that is needed for statistical validity. The variety of studies performed subjected the VTT and its drivers to conditions that researchers have found to be areas of human performance concern. Depth perception, situational awareness, motion sickness, workload, and visibility are a few of these issues. The experimental regime touched on all of these in connection with the single aspect of the driving task. Although the driving task was evaluated under a variety of conditions, they were at the low end of the speed, workload, and stress range. For these reasons, the conclusions from this data cannot be applied to the higher end of the operational range.
68
RECOMMENDATIONS The following recommendations are made to designers of future military vehicles: Improve the indirect driving vision system by using higher resolution cameras having 630 by 548 television lines. Consider side cameras with views adjustable through pan and tilt mechanisms. Investigate design options for the reduction of motion sickness a phenomena that occurs in some drivers with indirect vision systems.
69
REFERENCES Ebenholtz, S. M. [1992]. "Motion sickness and oculomotor systems in virtual environments." Presence, 1(3), 302-305. Endsley, M. R. [1993]. “Situation awareness and workload: Flip sides of the same coin.” Proceedings of the 7 th International Symposium on Aviation Psychology, April. Hart, S.G. & Staveland, L. [1988]. “Development of the NASA Task Load Index (TLX): Results of empirical and theoretical research.” In P.A. Hancock and N. Meshkati (Eds.) Human Mental Workload, pp. 139-183. Amsterdam: North-Holland. Kennedy, R. S., Fowlkes, J. E., and L. J. Hettinger [1989]. Review of simulator sickness literature. NTSC TR 89-024, p. 51, Orlando, FL, 4 September. Kennedy, R. S., Lane, N. E., Lilienthal, M. G., Berbaum, K. S., & L. J. Hettinger [1992]. “Profile analysis of simulator sickness symptoms: Application to virtual environment systems.” Presence, 1(3), 295-301. Kennedy, R. S., Lilienthal, M. G., Berbaum, K. S., Baltzley, D. R. & M. E. McCauley [1989]. “Simulator sickness in U.S. Navy flight simulators.” Aviation, Space, and Environmental Medicine, 60: 10-16. Kerle, R. H. & H. M. Bialek [1958]. The construction, validation, and application of a Subjective Stress Scale (Staff Memorandum Fighter IV, Study 23). Presidio of Monterey, CA: US Army Leadership Research Unit. McCracken, J. H. & T. B. Aldrich [1984]. Analysis of selected LHX mission functions: Implications for operator workload and system automation goals. Technical Note ASI 479-024-84. Fort Rucker, AL: Army Research Institute Aviation Research and Development Activity. McCarley J.S. & W.K. Krebs [2000]. “Visibility of road hazards in thermal, visible, and sensor-fused night-time imagery”, Applied Ergonomics, 31(5), 523-530. Pausch, R., Crea, T., and M. Conway [1992]. "A literature survey for virtual environments: Military flight simulator visual systems and simulator sickness.", Presence, 1(3), 344-362. Scribner, D. R. & J. W. Gombash [1998]. The effect of stereoscopic and wide field of view conditions on teleoperator performance. ARL-TR-1598, Aberdeen Proving Ground, MD: Army Research Laboratory. Selcon, S. J., Taylor, R. M., and E. Koritsas [1991]. “Workload or situational awareness?: TLX vs. SART for Aerospace systems design evaluation.” Proceedings of the Human Factors Society, 35th Annual Meeting, 62-66. Smyth, C.C., Gombash, J.W., and P.M.Burcham [2000]. Indirect vision driving with fixed flat-panel displays for near-unity, wide and extended fields of camera view. ARL-TR-2511, Aberdeen Proving Ground, MD: U.S. Army Research Laboratory, Human Research and Engineering Directorate. Smyth, C. C. and R. G. Whitaker [1998]. “Indirect vision driving study.” Proceedings of the 21 st Army Science Conference, 15-17 June, 497-502. Taylor, R. M. [1989]. “Situational awareness rating technique (SART): The development of a tool for aircrew systems design.” In Proceedings of the AGARD AMP symposium on situational awareness in aerospace operations, CP478, Seuillysur Seine: NATO AGARD. Taylor, R. M. [1995]. CC-SART: The development of an experimental measure of cognitive compatibility in system design. TTCP Technical Panel UTP-7, Human Factors in Aircraft Environments. Minutes of the Annual Meeting, 13-16 June 1995, Annex I, Appendix 2, pp 100-112. Toronto, Canada: DCIEM. Taylor, R. M., and S. J. Selcon [1994]. “Situation in mind: Theory, applications, and measurement of situational awareness.” In R, D. Gilson, D. J. Garland, and J. M. Koonce (Eds.), Situational Awareness in Complex Systems, Embry-Riddle Aeronautical University Press.
70
Velleman, P. F. & D. E. Payne [1992]. Applications, Basics, and Computing of Exploratory Data Analysis. Boston, MA: Duxbury Press. Whittaker, R. and C. C. Smyth. Indirect vision driving performance with a helmet mounted display. Manuscript submitted for publication. Aberdeen Proving Ground, MD: U.S. Army Research Laboratory, Human Research and Engineering Directorate. Yardley, L. (1992). “Motion sickness and perception: A reappraisal of the sensory conflict approach.” British Journal of Psychology, 83, 449-471.
71
72
