Using Embedded Pedagogical Tools for Optimizing Within Visual Range Combat Training – the Initial Experience Staffan Nählinder, Jonathan Borgvall, Erland Svensson FOI – Swedish Defence Research Agency P.O. Box 1165 SE-581 11 LINKÖPING, Sweden
[email protected]
Abstract Many of today’s combat flight simulators have a high level of realism regarding most important levels of fidelity. Unfortunately, most often, they suffer regarding explicit training tools or “pedagogical tools”. Even though the computer evolution has made it possible, very few, if any, embedded pedagogical tools are available in the Swedish military simulator training facilities of today. In a simulated environment, it is possible to embed tools and visualizations that effectively can enhance training efficiency in ways not possible in real world situations. When developing a new simulator, it might in fact be a very effective strategy to focus (some) resources towards pedagogical aspects rather than just continue pursuing the impossible goal of achieving perfect realism. The authors believe that by relaxing the realism - and instead stressing the pedagogical possibilities - a much more effective training environment can be accomplished. In the development of the present “WVR Demonstrator” (Within Visual Range), such a strategy has been chosen. The simulator is a second version of the “WVR Illustrator” (presented at SAWMAS 2002). It is much improved and now has several embedded pedagogical tools for training. It is now possible for the pilots to see a visualization of several important combat parameters in real time in order to maximize the usefulness and training efficiency of the simulator. During a combat situation, these tools can be used individually or together according to the preference of the pilot and instructor. In the present report, a study of the training effects of such tools is presented. Swedish military pilots have answered several questions regarding (among other things) the usefulness of the pedagogical tools in the “WVR Demonstrator”. Preliminary results from the study suggest that the pedagogical tools are most probably very useful, at least in early stages of combat flight training. Pilots in the study believe that, if used correctly, the embedded pedagogical tools can make WVR-combat training more effective. The results are yet preliminary; however, they clearly show an effective step forward in the development of training simulators.
Introduction Debriefing as a method for learning and improving has historical roots (Pearson & Smith, 1986) and could be closer described as a discussion performed after a mission with the purpose of improving future performance. One definition is that debriefing is the process of learning through the reflection of an experience (Thatcher, 1986; Pearson & Smith, 1986; Lederman & Stewart, 1986). Debriefings specifically used in training situations are often referred to as after action-reviews (Downs, Johnson & Fallesen, 1987).
The normal debriefing procedure in combat flight training today is that the student pilot is debriefed by the instructor after a flight. The obvious problem with this is that both of them have to remember and recall the situations and manoeuvres, often after several air-to-air contacts. Even if they could remember the situations correctly it is impossible to repeat exactly the same situation at a later point in time (Danielsson, Svensson & Jenvald, 2002). That is, the learning on how to handle a specific situation might be restricted to the feedback given by an instructor based on what is remembered after very few encounters of that particular situation. This is an important matter since debriefing is an important part of experience-based
learning processes (Lederman, 1992). Research by Stammers in 1983 support this idea, since he isolated the provision of extrinsic feedback as a key training variable. However, if appropriate feedback for learning of the task is not present in the real system, some form of simulator practice is desirable (Stammers, 1983). Any task that one wants to master should be repeated many, many times in order to optimize the learning of the task (Ericsson and Simon, 1993; Fisk and Lloyd, 1988). Research suggest that experience play a key role in performance among fighter pilots (Angelborg-Thanderz, 1990). According to Stedmon and Stone (2001), virtual environments offer improved effectiveness, reduced reliance and enhanced training methods over and above traditional methods. This possibility is one important reason for integrating comprehensive pedagogical tools in the Demonstrator. The pedagogical problems discussed above and the increasing potential of new technology has raised the issue of developing a high-fidelity simulator for air-to-air combat training with embedded extensive pedagogical tools for feedback during and after combat (Svensson, Nählinder, Danielsson & Jenvald, 2002). During the autumn of 2001, the development of a first prototype started off in cooperation between the Swedish Armed Forces Headquarters (HKV) and the Department for ManSystem Interaction (MSI) at the Swedish Defence Research Agency (FOI). The system was called the Illustrator and the main purpose was to develop the prototype in a short time and to a low cost. No pedagogical tools were integrated at this stage. In March 2002 the Illustrator was delivered to FOI-MSI for evaluation using fighter pilots from the Swedish Air Force, who identified some major weaknesses of the system. This simulator evaluation was presented at SAWMAS 2002 (Svensson, Nählinder, Danielsson & Jenvald, 2002). However, the results of the evaluation also indicated a great potential for this type of system (Borgvall, Nählinder & Andersson, 2002) and a decision was made to continue the project by developing a more advanced prototype. The extensive goal of the project was yet to develop a simulator prototype with comprehensive pedagogical tools for WVR-combat fighter pilot training (Danielsson, Svensson & Jenvald, 2002; Svensson, Nählinder, Danielsson & Jenvald, 2002). The guidelines that the evaluation of the Illustrator generated were used during the specification of the new prototype. The second prototype is called the Demonstrator and it will complement the present simulation facilities in the Swedish Air Force. To conclude the discussion of the pedagogical tools, there are a range of military studies that have clearly demonstrated that computer-based training increased performance improvement, even in comparison with hands-on training with real equipment (e.g. Stedmon & Stone, 2001; Stone, 2001).
WVR Combat The WVR-Demonstrator is a flight simulator specifically built for within-visual range (WVR) combat training.
Figure 1. A simulator cabin in the WVR Demonstrator The Swedish Air Force has several simulators for fighter pilot training. Concerning the visual presentation, those systems are optimized for beyond visual range combat (BVR-combat) simulation. During this type of combat, the fighting aircrafts have no or at least very limited visual contact. The activities related to BVR-combat are therefore primarily based on instrument flying. This means that the demands on the visual presentation of the surrounding environment are restricted in contrast to within visual range combat (WVR-combat). During a WVR-combat situation (often referred to as dogfight), the fighting aircrafts have a maximum distance of approximately 15 kilometres, depending on weather conditions. During WVR-combat, the combatants try to maintain visual contact to be able to perform the smartest manoeuvres in order to reach a location where it is possible to fire at the enemy, or at least make sure that the enemy can not fire back. Most often, the best location is to end up directly behind the enemy aircraft. The most optimal position depends on what weapons are available and what weapons the enemy carries. One of the challenges when learning how to master WVR-combat, is to understand the importance of having a superior energy level. The energy level is the sum of energy from the velocity and the energy from the altitude. That is with high altitude and high speed you are more likely to win than with low altitude and low speed. The geometrical relationships between the aircraft must be understood as well as the relationship between manoeuvring and maintaining speed.
In WVR-combat the instruments are almost exclusively used for controlling the status of the own aircraft (height, speed, altitude). In many simulators, the visual presentation is static in the sense that the operators see a limited field of what is in front of the aircraft and the view does not change according to the operators’ head movements. The exceptions are domes and head-mounted displays with head-tracking systems. For BVR-combat simulation, this is not believed to be necessary, but for WVR-combat it is. In WVR-combat, the operator has to be able to follow the enemy visually in the simulated environment. Domes and HMD´s offer this possibility, with HMD´s as the cheaper alternative. However, one major question is whether the HMD´s available today are of sufficient quality for high fidelity air-to-air combat simulation. In the Illustrator evaluation, one of the key results was that the HMD’s used in that simulator had a much too narrow field-of-view. The fighter pilots that tested the Illustrator all agreed that the limited field-of-view was the single most limiting feature regarding its use as a training simulation. The importance of a large field-of-view is further discussed in Burki-Cohen, Tiauw & Longridge (2001).
Figure 2. Head mounted display in the WVR Demonstrator Normally, the scenario is set up that the two pilot stations will compete against each other trying to manoeuvre into a favourable position behind the opponent aircraft. However, it is also possible to join forces and to fight simulated enemies together. Each pilot station can be run separately for instance while training advanced flying skills.
The WVR Demonstrator The WVR Demonstrator was built in 2003 and consist of two pilot stations (cabins), see Figure 1, and a instructor station. The instructor can manipulate the scenario and can also control the pedagogical tools. In the pilot stations, the head down displays, the flight instruments, are presented on a computer screen inside the cabin. The world is displayed inside head mounted displays which offer a quite large field of view, see Figure 2. The HMD’s used in the Demonstrator has a much larger field-of-view than the ones used in the Illustrator. The image in the head mounted displays are updated according to the pilots head movements, that is, if the pilot turns his or her head towards the left, the HMD will also display the view looking out to the left of the aircraft. The head movements are measured in six dimensions, so not only the head direction is detected but also the head position. As a total, the HMDs generate a very compelling out-of-the-window feeling. One can almost lean out of the cockpit and look below the aircraft! The pilots communicate with the instructor through a headset.
Pedagogical tools in the Demonstrator Most simulators today have several imbedded pedagogical tools, such as the possibility to freeze the scenario, or to manipulate environmental values, such as visibility, wind and the time of day. Many simulators also support (at least limited) after-action-review. However, most often, the range of pedagogical tools is limited. However, today with the overwhelming and constantly improving power of computers, there is much more that can be done!
Spatial tools In the WVR “Demonstrator”, it is possible to enhance the visual understanding of the geometrical relationships between the aircraft by seeing the scene from any position of the simulated world. For instance, if one has trouble understanding the opponents’ point of view, one can simply freeze the running combat and then move ones eyes placing them inside the enemies’ aircraft. This way, the pilot in the first aircraft can see exactly what the opponent sees and understand his or her situational awareness. One can also place ones eyes at any other position in the simulated environment. For instance it is possible to see the scene from a God’s eye position in the sky! The pilot can control the position of the eyes from his cockpit. To further enhance the geometrical understanding of the aircraft, it is possible to add 3D tapes behind each aircraft showing the latest path of the aircraft, see Figure 3. The visual appearance of the aircraft can be improved by scaling up the aircraft.
Other analysis tools include comparisons of speeds, altitudes and energy loss due to extensive maneuvering. Several other analysis tools are also available especially concerning analysis of relative positions between the aircraft. These combat analyses graphs all help to understand who won the combat and more importantly why.
Evaluation
Figure 3. 3D bands showing the aircrafts path. It is also possible to switch aircraft with the opponent in mid flight without leaving the cabins. Or both pilots can be placed in the same aircraft (however only one at a time can handle the controls).
Time tools There is a feature to rewind and redo a situation. By simply moving back the time, the two aircraft can redo the exact same situation again. Further either none, one or both pilots can choose to replay whatever happened last time, or to do a fly-out from the same initial condition but fly differently. If replay is chosen, the aircraft will fly the exact same path as last time, and the pilot can literally lean back and enjoy the ride! After several different fly outs, it is possible to display all the flight paths in the air, creating a sun feather effect displaying each of the flights overlaying each other. This supports the pedagogical effect of directly comparing the outcomes of different flight strategies. It is also possible to further manipulate time. For instance, time can be played backwards, or forwards at either normal speed, below normal speed or above normal speed. Some studies show that by training at above-speed simulation, real flight then becomes less stressful than if the training was done at normal speed. On the other hand, training at below normal speed increases the feeling of situational awareness and thus might also facilitate in the learning process.
Combat analyses Besides the embedded pedagogical tools for manipulating space and time, the Demonstrator also includes possibilities of real-time analyses of important combat parameters. For instance, a direct comparison of aircraft energies is possible using graphs that display the energy history of the two aircraft. This makes it intuitively easy to see what aircraft can maintain an energy advantage over his or her opponent.
To evaluate the usefulness of the pedagogical tools in the WVR Demonstrator, some initial qualitative data regarding the potential future training value of the pedagogical tools was collected through the use of questionnaires. When collecting such data it is important to use the future users of the system to answer the questionnaires. Otherwise it will be difficult to generalize the results to the relevant population. As of today, only limited data has been collected regarding the end users ratings of the pedagogical features mentioned above. Four experienced fighter pilots have scrutinized the tools and subjectively evaluated the perceived usefulness of each tool. They all agree that the tools will be very useful, especially in early combat flight training. In the short time future, a more extensive data collection will be done asking several more expert fighter pilots about the usefulness of the pedagogical tools. The best way to evaluate the usefulness of the pedagogical tools would be to perform a test of the transfer-of-training effect. In such a study, one could take a group of relatively new pilot students, and divide the group into two. Both groups would train their combat flight skills in the Demonstrator, but only the first group would be allowed to use the pedagogical tools. After extensive training, one would let each student from one group compete against a student from the other group in a real-flight combat situation. If the students from the first group would win significantly more often than the second group, the pedagogical tools would most probably be useful! Obviously, such studies can rarely be done. There are many issues to account for before it could possibly be done. Therefore, with the present Demonstrator, subjective questionnaire data will be collected instead, focusing on the perceived usefulness of the pedagogical tools at hand. When preparing for such a study, one must always remember what the purpose of the simulator is. The key issue must always be operative performance! It doesn’t matter if it looks good or feels good: if it doesn’t improve performance in the real world, it’s of no use. Operative performance is quite difficult to measure. Often a particular behaviour can be affective even though it is difficult to quantify. For instance in an intercept mission, a good performance could be to scare off the enemy even without firing missiles. However, in other situations the
only good performance is to shoot down the enemy before he or she shoots down you!
References
Conclusions
Angelborg-Thanderz, M. (1990). Military Flight Training at a Reasonable Price and Risk. Swedish National Defense Research Establishment report C 50083-5.1.
The main study on the usefulness of the WVR Demonstrator is yet to come. Swedish Air Force pilots will be used to answer questionnaires regarding many different aspects of the simulator. Primarily, the usefulness of the pedagogical tools will be analyzed. Besides evaluating the pedagogical tools, a fidelity evaluation (Roscoe, Wilkinson & VanderVliet, 2001) will also be made similar to the one performed on the Illustrator. User acceptance will be measured concerning different aspects of the simulator (see Bell & Waag’s, 1998 regarding the importance of user acceptance; Salas, Bowers and Rhodenizer, 1998). The main study is scheduled to be performed in the beginning of 2004. So far however, it seems that the pedagogical tools can in deed be very useful, especially when it comes to understanding the importance of maintaining superior geometrical location relative to your opponent. Also, the usefulness of the pedagogical tools might be largest when teaching fresh pilot students WVR combat. Initially, many pilot students have a hard time understanding the geometrical relationships between two fighting aircraft and the importance of avoiding certain situations. With the tools described above, this understanding can be achieved much earlier in the training, and hence make combat flight training more effective. The results so far seem to show that the implementation of pedagogical tools in the simulator might make training more effective. Future simulator development projects should consider these findings and thoroughly investigate what tools to embed. The pedagogical tools described above only start to illustrate what possibilities lay in training simulators. Through extensive research and cognitive task analysis it should be possible to find similar useful pedagogical tools in any training situation, not only in flight simulators. Since the study has only just begun, the results should be interpreted with great care. However, the authors believe that the pedagogical tools are a very useful way forward and an important step in future training simulator development projects!
Bell, H. H., & Waag, W. I. (1998). Evaluating the Effectiveness of Flight Simulators for Training Combat Skills: A Review. The International Journal of Aviation Psychology, 8(3), 223-242. Borgvall, J., Nählinder, S., & Andersson, J. (2002). WVRIllustrator Evaluation: Using Pilot Expertise for Future Development (Scientific FOI-R--0710--SE). Linköping: Department for Man-System Interaction. Borgvall, J., Nählinder, S., & Andersson, J. (2003, June 24). Using Pilot Expertise for the Evaluation of a WithinVisual-Range Air-to-Air Combat Simulator Prototype. Paper presented at the The 22nd European Conference on Human Decision Making and Manual Control, Linköping. Burki-Cohen, J., Tiauw, G., & Longridge, T. (2001). Flight Simulator Fidelity Considerations for Total Airline Pilot Training and Evaluation (A01-37337). Montreal, Canada: AIAA. Danielsson, B., Svensson, E., & Jenvald, J. (2002). Pedagogical Tools for Within Visual Range (WVR) Fighter Pilot Training (Reprint FOI-S--0453--SE). Linköping: Division of Command and Control. Downs, C. W., Johnson, K. M. and Fallesen, J. J. (1987). Analysis of Feedback in After Action-Reviews. AD A188 336. U.S. Army Research Institute for Behavioural and Social Sciences, Alexandria, Virginia, USA. Ericsson, K. A. & Simon, H. A. (1993). Protocol Analysis (Second Ed.). Cambridge: MIT Press. Fisk, A. D. & Lloyd, S. J. (1988). The Role of Stimulus-toRule Consistency in Learning Rapid Application of Spatial Rules. Human Factors, 30, 35-49. Lederman, L. C. and Stewart, L. P. (1986). Instructional Manual for THE MARBLE COMPANY: A Simulation Board Game. New Brunswick, New Yersey: SCILS. Lederman, L. C. (1992). Debriefing: Toward a Systematic Assessment of Theory and Practice. Simulation & Gaming, 23(2), 145-160. Pearson, M. and Smith, D. (1986). Debriefing in Experience-Based Learning. Simulations/Games for Learning, 16(4), 155-172. Roscoe, M. F., Wilkinson, C. H., & VanderVliet, G. (2001). The Use of ADS-33D Useable Cue Environment Techniques for Defining Minimum Visual Fidelity Requirements (A01-37380). Montreal: AIAA.
Salas, E., Bowers, C. A., & Rhodenizer, L. (1998). It Is Not How Much You Have but How You Use It: Toward a Rational Use of Simulation to Support Aviation Training. The International Journal of Aviation Psychology, 8(3), 197-208. Stammers, R. B. (1983). Simulators for training. In T. O. Kualseth (Ed.), Ergonomics of Workstation Design (pp. 229-242). London: Butterworths. Stedmon, A. W., & Stone, R. J. (2001). Re-viewing reality: human factors of synthetic training environments. International Journal of Human-Computer Studies, (55), 675-698. Stone, R. (2001). Virtual Reality for Interactive Training: An Industrial Practitioner´s Viewpoint. International Journal of Human-Computer Studies, (55), 699-711. Svensson, E., Nählinder, S., Danielsson, B., & Jenvald, J. (2002, October 30-31). Pedagogical Tools for Within Visual Range (WVR) Fighter Pilot Training - Another Step Forward. Paper presented at the Swedish American Workshop on Modelling and Simulation SAWMAS-2002, Orlando, FL, USA. Thatcher, D. (1986). Promoting Learning Through Games and Simulation. Simulation/Games for Learning, 16(4), 144-154.
