Intelligent tutoring issues for air traffic control training - IEEE Xplore

IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 1, NO. 3, SEPTEMBER 1993

204

Intelligent Tutoring Issues for Air Traffic Control Training Andrew I. Komecki, Thomas B. Hilbum, Thomas W. Diefenbach, and Massood Towhidnejad

Abstract-This paper describes a system designed for air traffic control (ATC) training. The system consists of a computer simulation of an ATC radar workstation, a computer model of an ATC expert and other components that allow for automatic evaluation and “coaching” of an ATC student. The rationale for such a system is presented, along with a discussion of computer “intelligent” training methodologies. The architectural design of the system is described and those design features implemented in the current version of the system are discussed. Finally, there is a discussion of current and future research and system development: ideas for incorporating intelligent tutoring into the system software, techniques to be used by the system in evaluating student performance, and parameters and metrics to be used in generating ATC training exercises.

I. INTRODUCTION

T

HE Federal Aviation Administration (FAA) is responsible for the management of the National Airspace System (NAS). The air traffic control system (ATCS) represents the critical component of the NAS. The Advanced Automation System (AAS) is a new, highly automated ATC system that will be implemented in late 1990’s [l]. The AAS requires the development of appropriate techniques and tools to improve the screening, training, and work of air traffic controllers. Projected increases in air traffic will required a significant improvement in the capacity and quality of the ATCS. The FAA supports activities leading to the creation of a system capable of working well into the next century. Such a system includes three factors: hardware, software, and human operators. Modern techniques and methods of computer science, including artificial intelligence, software engineering, real time computing, simulation, and computer graphics, are being used in the development of the AAS. However, humans will still be crucial as air traffic specialists. To produce high-quality specialists, it is necessary to motivate them at each stage of the training process. They must be familiar with the system, understand their future duties, and be able to apply extensive spatiotemporal skills in complex decision-making situations. It is important for aviation and education professionals involved in such training to identify all appropriate tools, techniques, and methods. Current FAA regulations require all future air traffic controllers to attend the FAA ATC Academy in Oklahoma City, to become thoroughly acquainted with methods, procedures, Manuscript received April 1, 1993; revised June 11, 1993. The authors are with the Department of Aviation Computer Science, Embry Riddle Aeronautical University, Daytona Beach, FL 321 14. IEEE Log Number 9212011.

terminology, phraseology, and controlling techniques. During this initial phase, on-the-job training is not recommended. Simulation training, which uses a graphical presentation of dynamic situations, remains the fundamental training paradigm. It is the only widely accepted way to teach and test the trainee’s spatiotemporal skills, understanding of ATC procedures, and work attitudes in a semi-realistic environment. Simulation training is carried out under instructor supervision. Because of the need for close observation of training performance, ATC simulation training is carried out with one instructor per trainee. Often another operator plays the role of a pilot flying through the sector (and/or as an adjacent controller). This method of training is very expensive and requires large amounts of instructor time. After graduating from the ATC Academy, the controller trainee goes through on-the-job training at a specific facility and on a particular sector. During this phase the controller becomes familiar with the sector, facility operating procedures, traffic flows, etc. This basic part of the training is carried out using a full performanFe dynamic simulation (DYSIM) system, which simulates an operational radar console. Again, the training depends on a great deal of one-on-one instructor time. Rapid progress in computer technologies, especially application of currently available computer graphics with interactive and user-friendly interfaces, makes it possible to realistically depict an ATC radar screen on a computer display. The basic dynamic elements in such a simulation are aircraft flying through an ATC sector. They are represented by targets and data blocks on the radar screen. Each aircraft can be specified as an object with a set of dynamically updated attributes. Interactive multi-window screen operation, with mouse and menu selections, permits the user to operate the simulation in a user-friendly fashion and incorporate most of the standard controlling activities [2]. Recent progress in instructional design, computer-assisted instruction, and artificial intelligence provide a basis for the development of instructional systems that will reduce the instructor-student ratio without compromising the quality of instruction. The basic principle is to allow trainees to advance at their own pace, to use appropriate dynamic scenarios, and to continuously monitor trainee performance. The system presented in this paper is based on these ideas. 11. INTELLIGENT TUTORING SYSTEMS Much of the research in the field of educational software has addressed computer-aided instruction (CAI) which is com-

1063-6536/93$03.00 0 1993 IEEE

__

KORNECKI et al.: INTELLIGENT TUTORING ISSUES

205

monly used to indicate the use of computers in education. The the success of an ITS depends significantly on the quality of goal of CAI research is to build instructional programs that the usedstudent interface. An ITS uses a specific tutoring strategy to tutor students incorporate well-prepared course materials in lessons which are optimized for each student. Most of them are either in a particular domain. Most commonly, an ITS takes the electronic page-turners or drill-and-practice monitors. form of a computer-based problem-solving tutor, a coach, a Intelligent CAI (ICAI) programs are an improvement over laboratory instructor or a consultant. For the development of the earlier CAI systems. Through the incorporation of artificial an ITS for ATC, the most suitable tutoring strategy seems intelligence (AI) techniques, the system is capable of drawing to be “coaching” or “guided-discovery learning.” In this type conclusions from the student’s problem-solving behavior and of ITS, the student is in full control of the activity. The supervising the sequence in which material is presented to the only way for the system to direct the course of action is student. More recently, intelligent tutoring systems (ITS) have by modifying the environment. In most coaching ITS’s, the been used in place of ICAI in order to emphasize that the ongoing activity is a game. Game activities are chosen because significance of the shift in research methodology goes beyond of their conceptual simplicity and intrinsic motivational value, and also because a game provides an attractive context for the addition of an “I” to CAI [3]. ITS’s are more sophisticated in their basic design and discovery learning. If a student reaches a dead end in a game, more effective in their role as a tutor. They are intended to the coach can advise the student about other possibilities. supplement the classroom environment by helping individual Thus, the task of the coach is not lecturing but fostering students identify their specific weaknesses and rectify them and learning inherent in the activity itself by pointing out in an effective manner. ITS’s are designed to be sensitive existing learning opportunities and by transforming failures to the student’s strengths, weaknesses, and preferred style of into learning experiences [4]. Coaching seems to have great potential, especially when learning. Moreover, it is the objective of some ITS developers to produce ITS’s which are capable of entirely autonomous embedded in an interactive, real-time simulation. However, pedagogical reasoning that is based solely on primitive prin- most of the previously developed simulation-based ITS’s are ciples in the domain knowledge and on pedagogical expertise strong in their simulation capabilities but rather weak in employing an effective tutoring strategy. A good example [41. Most researchers today seem to agree that an ITS consists is the ITS called STEAMER [4], which is used for trainof the following four main components: the domain-expert ing engineers to operate steam propulsion plants on large module, the student-model module, the tutoring module and ships. STEAMER’Senvironment allows a student to learn and practice all important procedures by manipulating a simulated the communication module [5]. The domain-expert module consists of a representation plant and to practice all kinds of emergency procedures that of the knowledge that should be presented and taught to are typically taught in the abstract. Nevertheless, STEAMER the student. This expertise is also used for evaluation of does not evaluate student performances to point out any the student’s solution and overall progress. To achieve this misconceptions or “bad habits.” This paper describes the design and partial implementation objective, most systems generate, or store in some form, all feasible solutions to the problems in the same context as the of an ITS for ATC. The sophisticated learning environment student does, so that their respective answers can be compared. is established through an interactive, real-time simulation of The student-model module contains information about the the air traffic controller’s work environment and is further student’s understanding of the material to be learned. This enhanced through an intelligent coach, which critiques the type of knowledge has only recently been used to improve trainees’ actions in order to improve their ATC skills. the quality of hypotheses about the student’s misconceptions and suboptimal performance strategies. As in human tutoring, 111. ATC-ITS ANALYSIS AND DESIGN this knowledge about the student is extremely important in This section describes the architectural design for an ITS to the decision-making process affecting the choice of further be used for ATC instruction (hereafter referred to as ATC-ITS). tutoring strategies. The tutoring module is the core of the system. It interacts The four major objectives ofthe ATC-ITS are as follows: with the student through the communication module. With its provide the student with an interactive, simulated ATC knowledge about teaching methods and its ability to access domain; the expert and the student model, the tutoring module is able create training scenarios that provide meaningful training to select a problem and to monitor and criticize the student’s exercises and match the student’s performance level; performance in a humanlike way. The critical questions this coach the student through the scenarios; module address are: when to give a hint, how far to let the evaluate student performance. student continue in case of an error, etc. Because many of these questions cannot be answered clearly, even by human tutors, the decision-making process used within an ITS is, to A. Analysis and Requirements Collection a certain extent, based on the results of experiments [6]. The development of the ATC-ITS began about four years Finally, the communication module, also known as the user ago with an initial study of the problem of training air traffic interface, controls the flow of communication between the controllers. The initial research goal was to gain familiarity student and the tutoring system. Research [7] has shown that with ATC operations and the controller’s environment and

206


work. The primary sources of information for this study were FAA documents related to both ATC operations and ATC training (regulation manuals and training and task analysis documents). The first major task was to build a knowledge base representing the domain expertise of an en-route air traffic controller. The principal methods used were observations (field trips to an ATC center), informal and formal interviews, operational task analysis, simulator operation, and research team discussions. A key element of gaining knowledge about ATC training was the consultation and advice provided by air traffic controllers with both operational experience in en-route ATC environment and ATC training expertise. The initial work concentrated on the development of an expert system and its interface with a realistic simulation. More recent system analysis efforts have concentrated on the training component of the system and consisted of study of literature concerned with intelligent tutoring systems and ATC training; discussions with air traffic controllers about FAA ATC training methods, ATC trainee characteristics, and ATC controller characteristics, capabilities, and skills; formulation of an initial set of ATC training objectives for the ATC-ITS; development of training strategies for meeting the training objectives; preparation of a formal description of the ATC-ITC architecture system, including functional descriptions of an ATC tutoring module and an ATC student-model module; proposal of assessment techniques for evaluation of trainee actions during a training session. B. Design Considerations The ATC-ITS provides training for en-route air traffic controller students. It is assumed that the trainee has control over all aircraft within a specified training sector. The trainee is primarily responsible for ensuring the proper separation of all aircraft within the training sector. In addition, the trainee is responsible for receiving aircraft which enter the training sector from an adjacent sector and for transferring aircraft which are departing the training sector into an adjacent sector. The transfers should conform to predefined “standard operating procedures” describing the specific altitudes for a specific aircraft (related to the aircraft’s destinations). The route through the sector should follow a filed flight plan with appropriate in-sector flight plan deviation under the control of the trainee. Finally, trainees should, in general, adhere to proper communication standards, use correct ATC phraseology, promote the efficient and orderly movement of aircraft through the sector, and provide appropriate response to pilot requests. Even though the goal of the ATC-ITS is to train students as air traffic controllers in all aspects of en-route ATC, the initial (current) version of ATC-ITS focuses on a selected set of subskills. This approach has allowed the development of a prototype that can be used to test the essential functionality

of the system. This prototype forms the basis for future enhancement and extension of the system. Some of the major simplifications include the omission of significant weather cells and the lack of aircraft entering the sector with an urgency or emergency status. Eventually, the system will be expanded to cover these and other situations. As in real-world ATC training, the student controls only one or two fixed sectors. In our current versions of ATC-ITS, only high-altitude sectors are used, because they are the least complex and thus the easiest to implement. It is anticipated that future versions of the system will also include low-altitude sectors and terminal airspace. A prerequisite for use of the ATC-ITS is familiarity with ATC terminology. Also, students must have some basic knowledge about the controller’s work place and responsibilities. Hence, the initial lessons on the ATC-ITS provide students with basic information, such as how to operate the radar screen, how to communicate with both pilots and controllers, and how to handle flight strips. This part of ATC-ITS is designed using a traditional CBI format [3]. C. Architectural Design

Although there is no generally agreed upon methodology for developing an ITS system, a general purpose architecture for building such a system has been proposed in [8]; its components include a user interface, an ATC expert model, a training system manager, a training scenario generator, and a student model. All of these are coordinated by a communication component referred to as a “blackboard.” Using this approach any of the instructional components can place information on the blackboard and then that information is available to all the other components. The design of the ATC-ITS is based upon such an architecture. Fig. 1 depicts a logical model for the ATC-ITS architecture [9]. The ATC-ITS architecture provides for modular decomposition of the functional units of intelligent ATC training along with effective communication between all components through the BLACKBOARD. Such a design makes it easy to accomplish the following: integrate separately developed modules; modify the system to add new functionality; and implement and test the individual modules prior to completion of the entire system. The user interface permits student access to ATC-ITS; it also provides access for a system manager and ATC instructors. The ATC simulation provides a realistic realtime simulation that produces an orderly presentation of aircraft targets, data-block displays, flight paths, and flight strips. The simulation also handles controller-to-pilot and controller-to-controller communications. The student, playing the role of an air traffic controller, interacts with the simulation during training sessions. The ATC student model contains history of the individual student’s interactions with the system and information about the student’s current training level. The air traffic expert controller (ATEC) is an implementation of a rule-based ATC expert model. The ATEC, through information provided by the situation analyzer (SA), can provide

KORNECKI

et

al.: INTELLIGENT TUTORING ISSUES

207

+

ATC Intelligent Tutoring System

TRAININGHANAGER

I

I

AT0 llyUUTloN

Fig. 1. ATC-ITS architecture.

expert knowledge and guidance relative to ATC decision making. ATEC assesses the alternative actions identified by the SA and uses the ATC rules and regulations represented in its knowledge base to arrive at the “best” decision. The actual tutoringkoaching ATC instructions are carried out by the training manager (TM). The TM guides and assists a student in the use of the ATC simulation and the ATEC to meet the ATC training objectives. The TM provides for three functions: creating and initiating training scenarios for the student, evaluating student performance, and providing advice and help. Training Scenario Generator (TSG)-The TSG uses information about the student’s training level to construct realistic ATC situations (called scenarios) appropriate for the training session. Training Evaluator (TE)--The TE assesses the student’s training level using information which results from the student’s interaction with a training scenario and assessments made by the ATEC. The TE also has a report generation feature that provides information about each student training session along with a description of the student’s current training level. Training Advisor (TA)-The TA provides help and advice to the student during an “assisted” mode of operation. The TA also provides introductory and explanatory information to the student about operation of the ATC simulation and its supporting modules.

IV. CURRENT STATEOF ATC-ITS IMPLEMENTATION This section describes the state of the implementation of the ATC-ITS system. A. ATC Simulation and Situation Analyzer The basic elements of the ATC simulation component of the current ATC-ITS training station are an ATC simulation program; a graphics display (simulated radar scope); data displays (simulated flight strips); input devices (keyboardmouse). The simulation is implemented using an object-oriented paradigm in the MODSIM I1 language. The simulation soft-

ware is organized into modules. Each module contains object definitions, variable declarations (including objects), and module procedures. Example objects include aircraft, pilots, controllers, airways, flight strips, etc. [lo]. The ATC simulation updates aircraft targets according to a predefined scenario, resulting in changes of both target position and data block entries on the screen. It tracks each aircraft using data simulating the current route, ground speed and altitude. Also, depending on conditions, the system changes target depiction resulting in changes of target shape and data block appearance (“on-plan” or “off-plan,’’ “hand-off initiated,” “hand-off accepted,” “emergency,” “under control,” “out of control,” “inside-sector,” or “outside-sector”). The system is responsible for all standard and nonstandard pilot and adjacent controller communications as well as the simulated aircraft responses to the trainee’s clearances. These responses include modifying an aircraft’s status (with an appropriate communication), resulting in a possible change of the simulated flight path. The graphics display in the current ATC-ITS simulation is shown on the left side of Fig. 2; it represents an enroute radar scope. It displays sector boundaries, airways, fixes, restricted areas, targets, data blocks in different forms (limited, full, separation violation, hand-off initiated, hand-off accepted, radio failure, emergency, hijack, etc.). The graphics display responds to mouse operations such as hand-off initiation, handoff acceptance, data block move, and altitudehoute change for selected aircraft. The bottom left of Fig. 2 represents a communication window which simulates the radio communication between the controller and simulated pilots. The data display, shown on the right side in Fig. 2, represents an electronic flight strip area. It displays flight data for aircraft as either pending or active in the sector, in a format similar to one used in an actual ATC en route sector. The data display responds to mousekeyboard inputs to update fields on the strips, tag a strip to show some special status, change the sequential positions of the strips, or remove a flight strip. The input devices are a standard keyboard and mouse. They allow for operations representing a trainee controller’s entries to the system, resulting in corresponding changes to the graphics display and data display. The input devices also provide for operations representing a controller’s communications to pilots and to adjacent controllers. In the current version of ATC-ITS, the voice communication channel “from controller” is represented by a communication window on the bottom left of the graphic display, which allows the trainee controller to issue commandsklearances through a menudriven input interface. The voice communication channel “to controller” (simulating communication to the trainee) is represented by a communication window with text on the graphics display. The situation analyzer is responsible for monitoring the simulation and generating the triggering and situation facts which will be used by the ATEC. In the current implementation of the ATC-ITS system, the situation analyzer is physically located in the simulation module. Minor modification to the simulation module is required in order to expand the current system to accommodate training in a terminal area.

208


Fig. 2. ATC-ITS graphics display.

B. Air Traflc Expert Controller In real ATC sectors, a controller’s actions are based on a subjective evaluation of the situation and a rough mental projection of the aircraft courses and computation of a future possible separation violation. There is no extensive or precise arithmetic computation involving the geometric relation between objects in three-dimensional space. In interviews with ATC experts we have found that each action is triggered by some primary information, e.g., a radar contact, a pilot request, a predicted separation violation, or the need for a hand-off. The actual controller action selected depends on other secondary. information describing the situation at the moment of decision, such as the type of aircraft, the aircraft’s flight plan, weather, airport situation, other traffic, specific knowledge of the area, etc. [ll]. Based on this information and the analysis of ATC operational manuals, one can see that there are a lot of controller actions which conform to a cause-effect paradigm. Hence, the decision was made to represent the ATC operational knowledge in the form of a set of rules. These rules represent the regulations that an air traffic controller is expected to follow to achieve the efficient and safe flow of air traffic. Therefore, ATEC design is based on a rule-based knowledge representation paradigm associated with the area of en-route air traffic control [ 121. There are two parts in a rule. The left-hand side represents a set of conditions (causes) which need to be satisfied in

order for the right-hand side of the rule, which represents the conclusion (effect), to become true. Currently there are more than 150 rules in the ATEC knowledge base. The ATEC component of ATC-ITS is implemented in CLIPS-a C-based expert system shell which uses a forward chaining inference engine to reason about each particular situation as the situation changes. The ATEC receives the current information about the aircraft and the airspace situation from the ATC simulation via the situation analyzer. Each individual piece of information is classified as either a triggering fact or a situation fact. The triggering facts are determined from the current status of the simulation: the location of aircraft, the distance between the aircraft, and pilot requests. The situation facts represent other aspects of the situation that constrain the type of actions that the pilot could take in order to remedy the problems identified by the triggering facts. For example, if a separation violation has been detected, can the plane climb to another altitude and avoid the conflict, and at the same time not generate a second conflict with another aircraft? The triggering and situation facts are used in the left-hand side of the rules in the knowledge base. Based on these facts, the ATEC identifies possible problems and suggests corrective actions. This information is represented as action facts, which are presented on the right hand side of the rules. Upon request the ATEC explains a situation and displays a set of correct ATC actions from which to choose.

209

KORNECKl et al.: INTELLIGENT TUTOFUNG ISSUES

To use the ATC-ITS system in the terminal environment a completely different ATEC module is required. The new ATEC module needs to represent the ATC procedures which are used in the terminal area.

C. Training Manager and Student Model A simplified version of the training manager for the ATCITS system has been implemented. This section describes the capabilities of the current training manager module. Future development of this module is discussed in the next section. The current TA is located in the ATEC module; it monitors the current situation in the airspace and, upon request, provides advisory information to the trainee. The advisory information describes the current situation at hand and the required actions which the trainee needs to take. The TE module uses a rudimentary approach in evaluating ATC trainees. The TE assesses the training level of a trainee using information about the trainee’s interaction with a training scenario, and assessments made by the ATEC and the situation analyzer. A default score for a trainee is set at 100. Points are deducted for each time the trainee misses a separation violation or is late in accepting or initiating a handoff. The TE is capable of generating a report which provides the TE assessment for each student training session. Currently, there is no automatic intelligent scenario generation withimthe ATC-ITS system. There is a series of scenarios with different levels of difficulty which are available to both the trainee and the instructor. These scenarios are selected (mouse clicked) by the user. There is also a provision for running the simulation with a randomly generated scenario. The scenario files can be modified by an ATC instructor to generate a new scenario. There is a very basic student model module which keeps track of the student’s progress and performance, and which can generate reports to both student and instructor. Each student has a file which represents individual training progress on the system. This file contains information such as how far a student has progressed through a lesson and his or her score. These files are protected: only the student and his or her instructor can read its content. The instructor can write information to the files of students who are under his or her supervision. V. FUTURE IMPLEMENTATIONISSUES This section describes three critical components of a complete ITS which are not yet fully implemented. A discussion and description of tutoring concepts, evaluation paradigms, and methods of assigning ATC scenarios is induded. A. Tutoring Concepts

The ATC-ITS coach is to be embedded in the TM. Its primary task is to continuously compare a student’s actions at any point with those that an experienced controller would make. Based on this comparison, the coach can provide a brief description of what an “expert” might do at that moment, together with reasoning of why that action would be appropriate. If a series of very poor judgments by a student are detected, the coach intervenes with advice, suggestions, and a

reasoning description. Additionally, the student can request help from the coach in this same format whenever needed. In the ATC-ITS, the TM will first evaluate the trainee’s performance (the TE function), then formulate tutoring decisions (the TA function) based on the number and type of errors a student makes in response to the events presented in the simulation. If the student takes an action which resolves a conflict and if the action was accomplished within the time-frame for an acceptable response, then increasingly more complex situations will be presented. When a response does not result in conflict resolution or if the student causes additional conflicts, the TM will attempt first to analyze the trainee’s understanding of what is the correct response. If the trainee does not identify the correct response, instructions will be given to clarify any misconceptions, and the TM will ensure that upcoming scenarios incorporate the particular topic again. However, if testing confirms that a trainee is aware of the correct response, then the TM will try to determine whether or not the trainee is able to recognize the existence of the conflict situation. If the trainee does not appear to recognize the existence of the conflict, the TM will add more salient features to the situation which will enable the trainee to recognize the problem more easily. As improvement in recognizing the problems proceeds, additional salient features will be reduced until the student is able to recognize the problem in the original situation without the help of these other prompts. Finally, if the student demonstrates both knowledge of the correct response and the ability to identify the situation calling for that response, but still does not make the correct response, then the TM will test the trainee’s knowledge of the consequences of an incorrect response by adding features to the resulting situation. As student recognition of the consequences of the incorrect response improves, saliency and immediacy of the depicted consequences will decrease those normally experienced by full-performance controllers. B. Evaluation Issues During the development of the ATC-ITS system, it was established that it is not practical to generate just one correct controller action for any situation or triggering event. Our initial idea that trainee actions could be matched against single precomputed expert actions was rejected by our air traffic controller consultants as unrealistic. As a result, the ATEC has been designed to generate a set of correct controller actions for any given situation. For example, if there is a conflict between two aircraft, it can be resolved by changing either the altitude or course of either of the aircraft. Eaclfchoice produces a different scenario, resulting from differences in triggering, situation, and action facts. For the design of the TE two criteria for evaluating trainee progress have been established: the trainee’s ability to properly respond to triggering facts and the trainee’s ability to prioritize controlling actions. Regarding the first evaluation criteria, the trainee response may cause some triggering facts to be eliminated (e.g., a conflict disappears) or may cause some to be added (e.g., another conflict appears). The elimination of a triggering fact

210

IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY,VOL. 1, NO. 3, SEPTEMBER 1993

will be treated as a correct action, while the lack of elimination of a trigger will be treated as an incorrect action. Elimination of a trigger with creation of another for the same object will be treated as marginal. Such an approach can be used under the not quite realistic assumption that all triggering facts are known in advance for the whole session. The TE will analyze each action of the trainee, based on the specific scenarios presented. For the first evaluation criteria, the key question is whether or not the student is able to recognize situations that require actions; that is, can the student recognize triggering facts such as separation conflicts and violations of flight rules? This question is difficult to answer because the ATC-ITS has no direct understanding of the trainee’s mental “big picture” of ongoing events. The TE component will assess a trainee’s performance in the following categories: ability to recognize the existence of a triggering fact; ability to make the appropriate response to the triggering and situation facts; understanding of the consequences of an inappropriate response. The TM will track the types of events resulting in marginal/substandard trainee response. For the second evaluation criterion, concerned with prioritizing actions, the TE will consider the following perspectives All the trainee’s actions should be accomplished in order of the urgency-level of the situations. Related conflicts should be handled “efficiently.” For example, if aircraft A and B, as well as aircraft A and C, are in imminent separation conflicts, it may be more efficient to change the path of aircraft A only and thus resolve two conflicts with one action. If a number of conflicts are expected at approximately the same time, students should start resolving these conflicts early enough so that sufficient time is available for resolution while the conflicts are still imminent. For this second evaluation criterion, the TE will use a quantitative approach based on an arbitrary performance index representing a count of the number of imminent conflicts, the number of correct controller-to-pilot communications (aircraft maneuvers), the number of correct and timely handoffs, conformance with ATC standard operating procedures (the TE will be able to detect violations of ATC regulations), and air traffic time delays in the sector. The TE will record student scores in each of these categories. C. ATC Scenario Selection The purpose of the training scenario generator (TSG) function of the TM is to use information about the training level of the student to construct specific air traffic scenarios to provide realistic and appropriate training activities. Training scenarios will be designed to exercise four categories of possible controller actions Issue a clearance to a pilot: turn (vector, direct route), climb/descend (change altitude), decrease/increase/resume (change speed), hold, switch frequency to the receiving sector.

Request information from a pilot: altitude, heading, identification, squawk code, estimated time/distance to a fix. Acknowledge communication from a pilot: radar contact, standby. Communicate with an adjacent sector controller: initiate handoff, accept handoff, coordinate actions, request control of aircraft in adjacent sector’s airspace. The actions listed above were identified by ATC controllers as critical for “what controllers do.” A single critical parameter in any training exercise is the number of aircraft involved in the exercise. Elementary ATC operations to be used in constructing training scenarios in the ATC-ITS include: handoff, standard procedural altitude clearance, potential separation conflict (in various configurations of altitude and route), coordination or control request from adjacent controllers, and pilot request. For scenario design, each of the above ATC operations is assigned a value indicating the percentage of aircraft that will be involved in the given ATC operation. Thus, the initial design of a scenario consists of selecting values for the number of aircraft and the percentages assigned to each elementary ATC operation. Therefore, a basic complexity metric would be a weighted sum representing the total number of ATC operations involved in the scenario. Other components of scenario design include route selection for each aircraft and the level of cooperation of simulation objects (manner and fidelity of response to the trainee requests/commands). Research continues on the TSG. For example, a more accurate metric could be obtained by using “complexity” weights (say 0.0-1.0) for the ATC operations, to indicate different levels of difficulty associated with them. The “complexity” could be simulated by the behavior of simulation objects and the level of their responsiveness. As an example, an adjacent controller could approve or deny some or all coordination and control requests. If the requests are denied scenario complexity increases, forcing the trainee to repeat the message and/or seek another solution. A training scenario must include activities that advance either the type or level of a skill, independently or in combination with other skills. In addition, in order for a scenario to be meaningful to a given trainee at a given time in training, the difference between the level of complexity of the scenario and the scenarios which were recently completed successfully by the trainee must be small. That is, complexity should be added in small doses, so that the training schedule exhibits sufficient continuity and focus, and the trainee experiences a feeling of progress and the evaluation process gives useful results. It is essential that scenario desigdselection be based both on the event-driven evaluation of scenario activities and the overall evaluation of previous training exercises.

VI. CONCLUSION Embry Riddle Aeronautical University has been involved in the development of an intelligent training system for ATC for several years. The major tasks of this system are to let the student interact with the simulated ATC console, to evaluate

KORNECKI et al.: INTELLIGENT TUTORING ISSUES

and coach the performance of the student, and to tailor the training scenarios to student performance. The current prototype focuses on providing a simulationbased training system (with coaching provided by an ATC expert) in a high-altitude en-route environment. Recent research in the ATC-ITS has centered around tutoringltraining strategies, trainee evaluation issues, and training scenario composition. Future development may expand to incorporate low-altitude and terminal environments as well as a increased realism in the sector environment (e.g., significant weather cells, urgency/emergency events, etc.). In addition, the work on the ATC-ITS has brought to light numerous interesting and challenging research problems associated with ATC training: identifying and modeling different levels of ATC problem-solving capabilities, individual versus team training stratagems, and the creation and evaluation of dynamically generated training scenarios. REFERENCES [I] US Department of Transportation, Federal Aviation Administration, Advance Automation System: System Level Description, Govemment Printing Office, Washington, DC, 1987. [2] V. P. Galotti and A. J. Kornecki, “ATC simulators do not have to replicate operational systems physically,” ZCAO J., pp. 6-8, May 1991. [3] T. W. Diefenbach, D. R. Carl, and M. Towhidnejad, “Intelligent tutoring and air traffic control training,” in Proc. 37th Air Traffic Control Assoc., Nov. 1992. [4] E. Wenger, Artificial Zntelligenceand Tutoring Systems. Los Altos, CA: Morgan Kaufmann, 1987. [5] H. Mandl and A. Lesgold, Learning Issues for Intelligent Tutoring Systems. New York Springer-Verlag, 1988. [6] J. M. Keller, “Development and use of the ARCS model of instructional design,” J. Instructional Devel., vol. 10, 3, pp. 2-10, 1987. [7] L. Barfield, The User Interface: Concepts and Designs. Reading, MA: Addison-Wesley, 1993. [8] R. B. Loftin and R. T. Savely, “Applications of intelligent computeraided training,” American Inst. Aeronautics and Astronautics, Washington, DC, 1991. [9] T. B. Hilburn, M. Towhidnejad, and T. W. Diefenbach, “The architectural design of an air traffic control intelligent tutoring system,’’ in Proc. 37th Air Traffic Control Assoc., Nov. 1992, [lo] A. J. Kornecki, “Object-oriented simulation for air traffic control training,” in Proc. 26th Annu. Simulation Symp., Mar. 1993. [ 111 “Simulation and AI as tools in aviation education,” in Pmc. SCS Con$ AI and Simulation, 1988. [12] A. J. Gonzalez, A, J. Komecki, A Ransom, P. Bauert, and R. Phinney, “A simulation based expert system for training air traffic controllers,” in Proc. Florida Artificial Intell. Res. Symp., May 1988.

-.

Andrew J. Kornecki, for a photograph and biography, please see page 136 of this issue.

21 1

Thomas B. Hilburn received the Ph.D. degree in mathematics from Louisiana Tech University in 1973. He is a Professor of Computer Science at Embry Riddle Aeronautical University. His current research interests include software engineering and the Ada programming language, fuzzy cluster analysis, and undergraduate computer science education. He has worked with the MITRE Corporation on a problem involving the analysis of airspace complexity and has recently been involved in a joint project between the University of Central Florida, General Electric Compa&-and Embry Riddle Aeronautical University. The objective of this project is to develop an Air Traffic Control Intelligent Simulation Training System.

Massood Towhidnajad received the M.S. degree in electrical engineering in 1986, and the Ph.D. degree in computer engineering in 1990 from the University of Central Florida. He is an Associate Professor in the Department of Aviation Computer Science at Embry Riddle Aeronautical University. He has expertise in such areas as computer architecture, computer networking, and artificial intelligence. He has been involved in the development of automatic knowledge acquisition for NASA’s diagnostic system. His current research includes the develop:me nt of the Air Traffic Control Intelligent Simulation Training system, and Int elligent Multimedia Maintenance Training System.

Thomas W. Diefenbach received the Vordiplom (B S . degree equivalent) from the Johann Wolfgang von Goethe Universitat in 1984 and the M.S. and Ph.D. degrees in computer science from Florida State University in 1989 and 1991, respectively. He is an Assistant Professor at Embry Riddle Aeronautical University in Daytona Beach, Florida. He has authored or coauthored several articles and a chapter in a book. His principal areas of research are intelligent tutoring systems, human factors, and artificial intelligence, especially as they relate to aviation issues such as air traffic control, general aviation accident investigation, and GPS satellite navigation. He also serves as a Senior Consultant for Aviation Consulting Services Ltd. in Napa, CA. Dr. Diefenbach is a member of the Association for Computing Machinery, the IEEE Computer Society, the American Association of University Professors, and the Aircraft Owners and Pilots Association.