In term of task distribution, this induces a swapping of roles: flight crew ... of function re-allocation (or tasks re-distribution) among the existing "components" of ...
International Workshop on Human Error, Safety and System Development (HESSD) 11-12 June, Linköping, Sweden
Implementing changes in controller-pilot tasks distribution: the introduction of limited delegation of separation assurance Laurence Rognin, Isabelle Grimaud, Eric Hoffman, Karim Zeghal Eurocontrol Experimental Centre BP 15, 91222 Bretigny, France {laurence.rognin; isabelle.grimaud; eric.hoffman; karim.zeghal}@eurocontrol.fr
Introduction Researches in the dependability domain identify various approaches and means contributing to safetycritical systems dependability (see for example Laprie et al, 1995). While the objectives of “zero default” has proven to be an utopia, new approaches insist on the unavoidable nature of errors, and suggest the design of fault-tolerant systems. These fault-tolerant systems are designed with the underlying assumption that errors will occur, and therefore dependability means are introduced in the system, aiming to provide support for the errors detection and recovery. Similar approaches in sociology (Rochlin, 1996) evoke high reliability organisation and describe socio-technical systems as globally reliable, mainly thanks to the emergence of control loops, supporting error detection and recovery within an hybrid system. More and more recent studies now envisage not only the risks associated with human presence in systems, but also the human positive contribution to the sociotechnical safety, in particular through the implementation of dependability means such as error detection and recovery. Indeed, the co-presence of various agents in a same given system could on the one hand have negative impacts (inducing errors, overload, …) and on the other hand provide redundancies and diversification between sources of information, multiple support for confrontation and sharing of mutual knowledge which happen to support efficiently the implementation of such control loops. In the air traffic control domain, studies highlight how co-operation between controllers, combined with an appropriate use of the environmental resources provided (e.g. looking at radar display, paper strips, overhearing radio communications) contribute to ensuring a globally safe system, where errors of either technical or human origins are usually detected and recovered before they lead to disastrous failures (Rognin et al, 2000). Today’s challenge in the domain of air traffic control is the foreseen increase of traffic load. While the capacity of the system is expected to double within the next 10 years, its level of safety has to be at least maintained. Therefore, solutions proposed to enable more traffic to be controlled in given areas investigate if and how current controllers could handle more traffic. The study presented here appears as an alternative to a previous study based on the “free-flight” paradigm in which the whole separation assurance lies to flight crews. In term of task distribution, this induces a swapping of roles: flight crew becomes the primary actor for separation assurance whereas the controller is supposed to intervene as a last resort in case of failure. The controller is thus in a position of acting by exception, which raises the key issue of human performance (Wickens 1997, Billings 1997, Corker 1999, Dekker & Woods, 1999). Rather than following some technical driven approaches, exploring how safety-critical tasks could be taken in charge by automated systems, we consider how the current system might be improved in terms of function re-allocation (or tasks re-distribution) among the existing "components" of the system. From our point of view, limited delegation is expected to contribute to the safety in enabling controllers and pilots to maintain an up to date picture of the relevant situation (situation awareness of the current delegation), as well as a shared cognitive environment (i.e. information used as resources). Controllers and pilots should then share a consistent representation of the on-going delegation. The present paper describes the concept of limited delegation as an approach enabling both the global existing level of safety to be at least maintained (if not increased), and the local performances of controllers and pilots to be improved, through a reduction of verbal exchanges, smoother aircraft trajectories, an increase of controllers and pilots’ situational awareness and availability. This increased availability is then expected to improve sector capacity and safety. In the first section, we address the method followed when designing the concept itself. In the second section, we briefly describe the experimental set-up defined in order to evaluate the benefits of delegation. In the third section, we present results related to the operational acceptance of the limited delegation. In the fourth section, we discuss the results related to safety issues. We conclude this paper with new issues to investigate during future experiments.
1
International Workshop on Human Error, Safety and System Development (HESSD) 11-12 June, Linköping, Sweden
Design of the limited delegation concept Systemic approach The approach implemented in the context of our project aimed at considering the whole socio-technical system, combining both a macroscopic (overall air-ground system) and microscopic (local practices) perspectives. The microscopic level consisted of identifying the current respective roles and functions of each component of the system, as well as the current human practices. The macroscopic level refers to a global vision of the system, pointing out existing informational exchanges, as well as loops of control between sub-systems. While resources are provided in order to support the existence of such loops, it is interesting to notice the emergence of unexpected reliability mechanisms, often enabling the detection and the recovery of errors. Envisaging a new allocation of functions between controllers and pilots requires us to anticipate potential consequences of local modifications on the overall system. Following some principles of cognitive systems engineering (Hollnagel & Woods, 1983) we considered the “joint cognitive systems” composed of controllers, pilots and the technical systems supporting their interactions. Therefore, in our study, we are investigating the respective functions of resources involved in the air traffic control, both technical and human, as well as both on ground and air sides. The figure 1 proposes a schematic simplified description of the components of the air traffic control system. Similar descriptions were proposed in past studies (see for example Billings, op. cited, p36-37; Haraldsdottir et al, 1998). On our figure, we represent existing and expected loops. Separation assurance
Accept/ Report
Aircraft
Aircraft
CRM
CRM
Pilot Flying
Pilot Non Flying
Pilot Flying
Pilot Non Flying
Existing explicit
Party line
Existing implicit Expected explicit
ATC Ground system (Radar, Strips) Executive
Planning
Controller
Controller
Over hearing
TRM: Team Resource Management CRM: Crew Resource Management
TRM Figure 1: Existing and new control loops within the air-ground system.
Participative and iterative approach The design of the delegation is participative in the sense that future users (i.e. controllers and pilots) are involved in the whole cycle of design, from early description of their practices, to the design activities themselves (i.e. the proposition of potential solutions). Indeed, the design is built upon an analogy with current practices. Expert controllers who took part in the design, contributed to the definition of the concept (including procedures and phraseology). The method implemented to support the evaluation of the delegation concept investigated gradually the overall air-ground system. As illustrated on the figure 2, three steps marked the approach out: initial small scale simulation, larger scale simulation focusing on the ground system and lately a second large scale integrated simulation, connecting together ground and air systems. Each time, the same process was applied, that is first the definition of users requirements turned into specifications (of systems,
2
International Workshop on Human Error, Safety and System Development (HESSD) 11-12 June, Linköping, Sweden
procedures, phraseology), then a prototyping and finally the evaluation of these prototypes by users (controllers and pilots). This evaluation informs at each iteration the users requirements, which lead to new specifications and new prototypes to be tested. At each step, the two sides (air and ground) were considered and progressively integrated in the evolving experimental set-up. Floyd et al (1984) differentiated incremental development (defined as stepwise implementing of a prespecified overall design) from evolutionary development (where the overall design changes through a series of cycles of design, implementation, and evaluation). Thus, our approach and the underlying concept we designed can be described as an evolutionary one, as opposed to a revolutionary concept such as free flight.
Users requirements Pilots needs Pseudo pilots needs Controllers needs Controllers needs Pseudo pilots needs
Analysis
Define concept Identify context Identify procedures
Prototyping Observation Observation Debriefing Debriefing Questionnaires Observation Questionnaires Log files analysis Debriefing Log files analysis Questionnaires Log files analysis
Simulation 1
Evaluation
Traffic sample Phraseology Traffic sample Procedures Procedures Phraseology Pseudo pilots HMI Multi Cockpit simulator Pseudo pilot HMI ATCo marking functions
Simulation 2 Simulation 3
Figure 2: Iterative design process
Resulting concept As described in the previous section, the proposed task distribution was actually designed around the human actors involved – controllers and flight crews. Taking as starting point existing human roles and activities, it is based upon the following key elements (Hoffman et al, 1999): –
Some separation assurance related tasks are delegated to flight crews. This is always initiated by the controller who decides to delegate if appropriate and helpful.
–
The delegation is limited since the controller can only delegate “low level” tasks (e.g. implementation and monitoring tasks) to the flight crew as opposed to higher level tasks (e.g. definition of strategy). In addition, only one flight parameter is delegated at a time.
–
The delegation is flexible since the controller has the ability to select for each situation the level of task to be delegated from monitoring up to implementation.
The delegation covers two classes of application: crossing and passing applications in en-route air space, sequencing operations in extended Terminal Manoeuvring Area (TMA). For each class, three delegation levels are proposed corresponding to reporting, maintaining, and providing separation. This form of distribution is expected to maintain unchanged core roles and working methods of controllers and flight crews. Typically, the delegation does not impact on controller’s decision-making: situation analysis, identification of problems (e.g. conflict detection), definition of solutions, and decision of delegation are part of controllers’s role and responsibility. Consequently, the controller keeps the initiative and overall authority on traffic management. In addition, the delegation does not impose any change of responsibility since it can be considered as a new clearance. Thus, the controller would be responsible for providing the appropriate clearance (i.e. which ensures the separation and is flyable for the flight crew) while the flight crew would be responsible for following this clearance,
3
International Workshop on Human Error, Safety and System Development (HESSD) 11-12 June, Linköping, Sweden
once they have accepted it. Indeed, pilots can refuse delegation in case of unability to comply with the instructions. The controller should be able to anticipate pilot future actions and to predict future aircraft trajectories as today through: (1) the delegation of no more than implementation task; (2) the restriction to only one parameter delegated at a time; and (3) the use of appropriate pilot reports. The flexible aspect of the delegation would provide flexibility to use the delegation under different conditions such as traffic density, airspace constraints, and practice level. It would also enable a gradual confidence building in the delegation. Indeed, the levels of delegation reflect increased use in practice. Each controller should thus experience his/her own “trade-off”. From a controller perspective, a potential reduction of the workload is expected, while from a flight crew perspective, the delegation is expected to allow for more anticipation (less time-critical instructions to follow), to increase situational awareness, and to enable an optimisation of trajectories. The delegation may however induce modifications in activities. Compared to visual clearances, the delegation is expected to be used more intensively, over a longer time period, and also will impose larger alterations (to ensure radar-based separation instead of staying visually apart). Therefore, from a controller perspective, one of the main issues is the capability to maintain an adequate mental picture of the traffic with delegated aircraft, while from a flight crew perspective, the delegation raises among others the question of the workload in the cockpit. In terms of technology, no change on controller working positions, no controller-pilot data-link communication, no intent information from aircraft, no automation on-board, no coupling to the auto-pilot nor to the flight management system are required – although they could be of interest. Only a subset of the air traffic situation is needed on board aircraft along with appropriate display cues (e.g. the use of a CDTI with TIS-B or ADS-B).
Evaluate impact of this new concept The scope of the study is to define a concept and assess its operational acceptability and its impact on roles, activities, safety, and flight efficiency. While many studies have investigated in the past only part of the air-ground system (e.g. Zeitlin et al., 1998; Casaux & Hasquenoph, 1997; Pritchett at al., 1999; Johnson et al, 1999), we carry out integrated experiments assessing the acceptability of the delegation concept from controllers and pilots points of view. On the one hand, from the controllers’ side we have to make sure controllers’ situational awareness is preserved and that performance (not only in terms of capacity but rather safety) is not diminished. On the other hand, from pilots’ perspective, the experiment aims to measure the impact delegation has on pilots’ workload, situational awareness and performance. Yet, it is essential to clarify that our experiments do not consist of running two parallel studies, but rather a global simulation of the system, composed of air-traffic controllers and pilots.
Iterative evaluation The iterative approach enables a continuous improvement of the concept (e.g. procedures, phraseology) and the simulation material (e.g. traffic samples, training). Even though the similarities between experimental set up was essential in terms of comparability of results, new functionality were implemented all along the project. Step 1: Assess the operational feasibility and identify user needs A first small-scale real-time experiment took place in June 1999 (two sessions of 1 week each). The main objective was to collect feedback from both controllers and pilots, in order to assess the operational feasibility and potential interest of the concept. Beyond, the objective was also to identify needs, other possible applications and evolutions, as well as indexes of evaluation for future experiments. The simulation environment consisted of one “measured” sector, two cockpit simulators equipped to receive delegations, background traffic not equipped and low traffic sample. 5 controllers (2 English, 2 French and 1 Spanish), 2 airline pilots and 2 pseudo-pilots took part in the 2 weeks simulation. Due to the assumptions made – simple ATC environment, small number of participants and limited occurrences of potential delegations – no quantitative measures were searched for. The results were qualitative indications gathered through questionnaires and debriefings, with an inherent subjective component in controller and pilot responses. Step 2: Validate the concept in a more realistic environment A second larger scale experiment was carried out in June 2000 (Zeghal et al., 2000). The objectives were to validate the concept in a more realistic environment and to evaluate the initial benefits with a
4
International Workshop on Human Error, Safety and System Development (HESSD) 11-12 June, Linköping, Sweden
focus on the ground side. The other objective was to set up a “final” version of procedures, in particular in terms of phraseology and co-ordination. The simulation environment consisted of two measured sectors with all traffic equipped, thus offering maximum opportunities to use delegation. The airspace simulated was part of the Paris terminal area, and was thought to be representative of a dense area and generic enough to allow an easy assimilation by the controllers. The working environment replicated the current one (paper strips, no advanced tool). Thus, controllers were able to get rapidly familiar with airspace and traffic flows, and were able to concentrate on the delegation concept. 6 controllers (2 Italian, 2 Portuguese and 2 English) and 4 pseudo-pilots participated in the 2 weeks simulation. The first week covered extended terminal manoeuvring area (from cruise to initial approach fix) while the second week focused on en-route airspace. In addition to questionnaires, system recordings were performed. To allow for a relevant comparison, each exercise was simulated twice: with and without delegation. During the measured exercises, controllers were familiar enough with the airspace, the traffic flows and the delegation. Step 3: Integrated simulation extending the concept assessment to the air system A third experiment took place in November 2000. In addition to the objective of previous session, we addressed workload issues through physiological measurements, flight deck issues through the connection of a cockpit simulator during three days and usability of controllers’ interface. The simulation environment was the same as in June (airspace, traffic flows, …). The working environment was a stripless environment with marking capabilities for reminder purposes. The delegation procedures and phraseology were identical. 6 controllers (3 French, 2 Irish and 1 Italian), 5 pilots and 4 pseudo-pilots took part in the 2 weeks simulation. Though we introduced a new interface, no specific training was planned. Indeed, we thought that either controllers were familiar enough with stripless environment, or they could easily get familiar with it. Some of the controllers were not familiar with the type of flows and work (sequencing in extended terminal area) simulated. In addition, we encountered technical problems with the interface. Consequently, during the measured exercises, controllers did not manage to get familiar enough with the delegation procedures and this impacted the results.
Definition of indicators and associated data analysis Whereas the ultimate goal of our project is to assess the relevance of delegation for both ground and air sides, we have not yet investigated at the same microscopic level both sides. The results presented here only concern the ground side. The evaluation of the concept of limited delegation aimed at answering the following general questions related to theoretical and operational acceptance, impact on efficiency and on safety. In order to answer these questions, indicators have been identified. •
Is the concept of delegation acceptable? Is the delegation usable and used?
Answers to that question were mainly collected through controllers’ subjective feedback. This feedback was collected through three main means, which were individual answers to questionnaires, collective discussions during debriefing and observation and recording of controllers’ comments while working. This “theoretical” acceptance is essential, especially in terms of integration and appropriation of the concept. Whereas negative feedback on operational aspects could be corrected with modification of procedures for example, a negative feedback on the concept itself would be disastrous because it might lead to its withdrawal. In addition, we also considered as indicators of delegation usage, the number of aircraft delegated. •
How did the controllers use it? Did it impact on the controllers’ activity?
Therefore, in addition to analysing the use of delegation per se, we also tried to understand how it is used, as well as its impact on activity. This was done typically through analysing communications (number and duration) and instructions given (number and types). We also investigated if delegation had an impact on the strategies of control, in comparing in particular the building of sequences in terminal areas and the aircraft trajectories.
5
International Workshop on Human Error, Safety and System Development (HESSD) 11-12 June, Linköping, Sweden
Another indicator we considered is the distribution of instructions, according to two different perspectives, first the spatial one, second the temporal one. The spatial perspective consisted in investigating first if instructions were gathered in specific geographical areas and second if delegation did modify such a gathering. Our assumption was that delegation would not only enable an early distribution of instructions, but also group them. Indeed, one of our expectations was that single delegation instructions would replace successive standard clearances (e.g. “remain behind target” replacing “reduce to 250kts (…) reduce to 220kts (…) maintain speed (…) reduce to 200kts”). The temporal perspective aimed at analysing if delegation would introduce a smoothing of activity. Our basic assumption was that delegation would enable controllers to anticipate peaks of activities and therefore smooth their activity over the exercise period. Last of all, especially during the latest simulation, we measured subjective workload, using questionnaire items, NASA-TLX and Instantaneous Self-Assessment device (ISA), enabling controllers to report their perceived level of workload every 2 minutes all along the exercise. During the simulation 2, only answers to questionnaires informed about the controllers’ perceived workload. •
Does the concept contribute to flight efficiency?
Though the major expected benefit relies on the controller side, it is essential to check that delegation does not impact negatively on flight efficiency. The objective indicators used are time and distance flown and fuel consumption (based not only on distance, but also on speed and altitude changes). Subjective indications are provided through answers to the questionnaire item. •
How does delegation impact on safety?
In addition to an increased availability for the controller, it is expected that safety would be improved or at least maintained, typically through the redundant separation monitoring ensured by controller and flight crews. Indications of the impact on safety will be captured through both subjective and objective data. Subjective feedback is provided by controllers and pilots through their comments, either provided in the questionnaires, or during debriefing sessions. The objective indicators used when analysing the system recordings are: separation infringements, Aircraft Proximity Index (API1 as defined by the FAA, e.g. Paul, 1990) and transfer conditions (sequencing separation infringements). From these diverse observations, we identified major types of errors.
Operational acceptance of the concept The results presented in this section are based on a combination of quantitative and qualitative set of data provided during the last two simulations. After a presentation of the initial feeling collected at the end of the simulation 1, we will describe in more details the results collected after simulations 2 and 3. In particular, indicators of safety will be discussed.
Initial feeling After the initial experiment (addressing the feasibility of the concept), the overall feeling on the concept was “promising with a great potential”. According to controllers and pilots interviewed, delegation could allow an increase of controller availability. The flexible use of delegation could provide opportunities to use the method under different conditions – traffic, airspace, practice level – however identifying the appropriate level of delegation did not seem easy. To ensure a safe and beneficial delegation, the procedures impose applicability conditions that shall be respected. It has been observed that the non-respect of these conditions could result in an increase of workload, communication, and stress. This appeared as a potentially critical problem and was thus considered as an important aspect to be stressed on during the training in future experiments. This experiment allowed us to refine and restrict some of the applications. Typically, for the sequencing applications, the highest level of delegation that was proposed allowed the flight crews to create themselves their own separation through the modification of two flight parameters (heading and speed). This was hardly acceptable on both sides: controllers could not foresee flight crew actions which is critical in terminal
1 API is a weighted measure of conflict intensity where 100 is a mid-air collision and 1 is a minor violation of the separation standards
6
International Workshop on Human Error, Safety and System Development (HESSD) 11-12 June, Linköping, Sweden
areas, and flight crews were unable to select the appropriate sequence of manoeuvres. The question of responsibility was raised as a major concern.
Concept acceptance The controllers’ feedback, provided in questionnaires is positive. Indeed, June and November controllers understood the concept and found it satisfying. They stressed benefits in terms of workload, anticipation and quality of control. Compared to the actual use of delegation, it appeared that between 60% (November) and 77% (June) of the traffic was delegated. Whereas the use of delegation was constant in June (between 50% and 60%), it was more progressive in November (from 30% to 55%). This could be related to the required familiarisation with not only the delegation concept, but also with a new HMI (stripless working environment and for some, with the type of traffic). From the pilots’ perspective, the concept of delegation is also acceptable, if sufficient support is provided to the pilots, especially in terms of actions definition (e.g. calculation of speed, time to resume).
Impact on flight efficiency Delegation had a minor positive impact on efficiency, which is important for us. Indeed, one major issue is to prove that at least delegation does not impact negatively on the flight efficiency, nor increases the cost of flights (in terms of fuel consumption, derived from distance flown).
Impact on controllers activities Strategies of control The delegation led to a drastic reduction of standard instructions, especially during the second simulation where a 35% reduction of manoeuvring instructions was observed (and 20% reduction when including non-manoeuvring instructions such as “select target” in the total number). The manoeuvring instructions seriously impacted by delegation are the speed (80% reduction in simulation 2) and the heading/direct (50% reduction in simulation 2) instructions. Type of instructions/sessions
Total number of instructions 600
3000
Tot_June_No_Del
No Del Delegation
2500 2000
500
Tot_June_Del Tot_Nov_No_Del
400
Tot_Nov_Del
300
1500
200
1000
100
500 ng
l
ad i He
Le ve
D
ire
ct
d Sp ee
R
HD G
l ta To
ov l_ N
ne l_ ju
ta To
AR
ta To
ta
l_
AO to
l_ ta to
Figure 3: Total number of instructions, by sector and by simulation
em ai n _r em ai n M er ge HD G _m er ge
0
0
Figure 4: Distribution of instructions by type (standard and delegation) and session
The analysis of elaborated sequences and aircraft trajectories shows no impact of delegation on controllers’ strategies. Indeed, even when different solutions were implemented, they both appeared equivalent. Workload Both answers to questionnaires and objective measurements (performed only in November) show a workload reduction in delegated situations. Indeed, 11 controllers out of 12 report a perceived reduction of their workload with delegation. However the continuous assessment of the individual
7
International Workshop on Human Error, Safety and System Development (HESSD) 11-12 June, Linköping, Sweden
workload (ISA) performed in November shows an increase in the reported workload in one of the sectors, and more specifically for the planning controller (e.g. figures 5 and 6). This could be related to the fact that in this sector the traffic converges soon after entering the sector. The controller therefore has to rapidly organise his/her sequence. While in the other sector, the controllers had more time to decide the sequence order.
Exercice=151100B
12
12
13 12
12 11
11 10
Nb . o f a i rc ra f t
10
A S A S
9
9
12
13 12
11
11
10
9 8
8
50 8
A S A S
8 7
40 6
6
30
5
20
3
10
10 9
12
70
13 12
12
12
11
60
11
10
9
10
50
9 8
8
8
8
8
8
40 7
7
6
6
30 5
5
20
4
2
3
10
1 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 1 1 2 2 2 2 2 3 3 3 3 3 4 4 4 4 4 5 5 5 5 5 0 0 0 0 0 1 1 1 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4
Exercice=161100B 14 14 14 13
AR/EXC
10
10 10 10
A S A S
60
9
50
8
8
7
7
7
7
40
6 5
AO/EXC
30
10
4
20
12
70 10
10
10 10 10
60
9 8
8
50 7
7
7
7
40
6
6 5
30
3
2
10
1
0
10 0
0
5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 6 : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 1 1 2 2 2 2 2 3 3 3 3 3 4 4 4 4 4 5 5 5 5 5 0 0 0 0 0 1 1 1 1 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6
20
2
2
1
0 AO/EXC Very High
High
AO/PLC Normal
AR/EXC Low
Very Low
0
0
5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 6 : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 1 1 2 2 2 2 2 3 3 3 3 3 4 4 4 4 4 5 5 5 5 5 0 0 0 0 0 1 1 1 1 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6
AR/PLC No Answer
Figure 5: Executive controller workload (without delegation on top, with delegation below)
AR/PLC
80
12 12 12 12 12
4
3
AR/EXC
14 13
12 12 12
10
AO/PLC
% 100
11
11 10
0
0
90
14 14 14 13
12
70
11 10
0
Exercice=161100B 14 13
12
0
5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 1 1 2 2 2 2 2 3 3 3 3 3 4 4 4 4 4 5 5 5 5 5 0 0 0 0 0 1 1 1 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4
80
12 12 12 12 12
0
0
AR/PLC
Nb . o f a i r c r a f t
10
AO/PLC
13
12 12 12 11
11
14
13
12 12
12
AO/EXC
90
14 13
1
0 % 100
10
2
0
Nb . o f a i rc ra f t
11
11
7
10
12
12
60
4
A S A S
13
13
10
9 8
8
14 14
14
N o
12
80
15
15
70 12
90
Exercice=151100B
80
14 14
13
13
N o
15
14
14
Estimated Workload (ISA) for Sector AR Position PLC Traffic=1AMHIGH1, Static=S1 Measured Period = 5:16/ 6:14
90
Nb . o f a i r c ra f t
15
% 100
% 100
Estimated Workload (ISA) for Sector AR Position EXC Traffic=1AMHIGH1,Static=S1 Measured Period = 5:16/ 6:14
0 AO/EXC Very High
AO/PLC High
Normal
AR/EXC Low
AR/PLC Very Low
No Answer
Figure 6: Planning controller workload (without delegation on top, with delegation below)
The NASA-TLX shows a reported higher workload with delegation, especially for the planning controller. This could be related to the task sharing proposed for the delegation, in which the planning controller is supposed to prepare the delegable situations and therefore is more involved in the traffic management. Two objective measurements (pupil diameter and heart rate) show a workload reduction, while the third one (dwell time) shows a similar workload. In the absence of warning functions (e.g. alert in case of loss of separation), the controllers still have to closely monitor each aircraft. It is expected that the introduction of some warning devices could reduce even more the perceived workload. Overall activity Controllers felt delegation had a positive impact on their activity, enabling them to anticipate decisions and actions, as well as reducing their workload. The analysis of the geographical distribution of instructions in the sector shows that delegation enables instructions to be given earlier. Similar peaks were observed in situations without and with delegation, corresponding to the building of sequences, and then delegation replaces successive instructions given all along the sectors with no more manoeuvring instructions (figure 7). 60
60
210600A
1. Standard
210600B 1. Standard
Distance repartition of instructions from LFPG IAF (nm)
2. Delegation
Distance repartition of instructions from LFPG IAF (nm) 50
50
Number of observations
Number of observations
45
40 36
30 28
24
24
20
40 12
12
30 26
20
20 7
14 11
10
11
10
1 8
8
1 4 1
0
1
Distance to IAF (nm)
Figure 7: Distribution of instructions (distance from exit point) without delegation (left) and with delegation (right)
8
International Workshop on Human Error, Safety and System Development (HESSD) 11-12 June, Linköping, Sweden
In order to make sense of the geographical distribution of instructions, we investigated the location over the geographical map of sectors. The figure 8 shows the differences between the number and location of instructions without and with delegation. In addition to highlighting a drastic reduction of their number, the figures also show how the two different types of sequences are elaborated in the two sectors, that is over which areas (usually around a merging waypoint) the sequences of aircraft are elaborated. EACAC GEMRA 220600A
CTL
PON LFPG LFPB
BAMES
EACAC
GTQ
GEMRA 220600B
UTELA CLM LFPN OL LFPO BEGEL SOTOR RBT BARAK
VAMDA VADOM CHW
TOLPA ETAMP MELEE
VATRI BUBLI KOTUN
SONAT FIJAC
OBORN
CHW
TOLPA ETAMP MELEE
ARSIL GELTA
MANAG LUREN
MOTAL
HR TORPA
VERDI ALOGA
PENDU
LUL DELOX
TINIL
AVLON
BLM HR
REKLA
TORPA
HOC LASON BASUD
MOROK ARPUS
TIRSO IXILU
BLM
REKLA
DIJ
BEGAR RESPO
CHABY OSKIN
LUL DELOX
TINIL
RLP
TUNOR
TIRSO IXILU
OSKIN
PILON
OKRIX BEGAR
RESPO
AVLON
LASAT
VERIX
CDN PILON
RLP
TUNOR CHABY
DIJ
NEV
STR
EPL
LUVAL
LAULY
MANAG LUREN
OBORN POGOL
MELKO TRO ORVEN ROTSI
BRY
OKRIX
MOTAL
FIJAC
BERAP MEL TELBO
LASAT
VERIX
VATRI BUBLI KOTUN
SONAT
EPL
LUVAL
LAULY CDN
VADOM
STR
GTQ
LFPN OL LFPO BEGEL SOTOR RBT BARAK
POGOL MELKO TRO ORVEN ROTSI
BRY
LFPG LFPB UTELA CLM
VAMDA
ARSIL GELTA
BERAP MEL TELBO
CTL
PON BAMES
VERDI ALOGA
PENDU
HOC LASON BASUD
MOROK ARPUS
NEV CACHI
RIGNI
ROMTA
BENIP
LISMO TUROM
ATN BAGOL
MOU
CACHI
RIGNI ROMTA
BENIP
MENOX AMORO
SAUNI LIRKO KELUK
ALURA GERBI
MENOX AMORO
MOLUS
ROA
BULOL
CERVI
MADOT
LSA GIPNO TDP
VANAS
ETREK
CERVI
BANKOAOSTA
CMF
LSA GIPNO TDP
PUNSA ROLAV
DANBO
BANKOAOSTA MADOT
MOLUS
GVA
PUNSA ROLAV
DANBO CMF
SOSAL SPR
MILPA PAS
LOGNI
GUERE
BULOL
SAUNI LIRKO KELUK
ALURA GERBI
BUSIL
GVA ROA
GALBIBIBOT LUSAR
MILPA PAS
LOGNI
GUERE
BAGOL
MOU SOSAL
SPR
BUSIL
LISMO TUROM
ATN
GALBI BIBOT LUSAR
VANAS
ETREK
Figure 8: Geographical distribution of instructions without delegation (left) and with delegation (right)
The analysis of the distribution of instructions in time does not show any impact of delegation on the smoothing of controllers’ activity. Yet, whereas delegation does not induce a smooth distribution of tasks over the exercise, it does introduce longer periods without instructions giving.
Discussing the impact of delegation on safety From traditional indicators to new ones While other studies focus on technical safety, we address in the context of our experimentations the human contribution to safety, in investigating how the operational use of delegation impacts on the overall system safety. In particular, we aimed to assess if delegation enables safe separations to be maintained between aircraft. As described previously, we approached safety at three levels: the controllers’ self-assessment, an automatic calculation of unsafe events (losses of separation) and the analysis of the recorded sessions. An important feature of this analysis is the multidisciplinary team involved, composed of an expert controller, a simulation analyst, two aeronautical engineers and a human factor expert. Globally, the controllers have a positive feeling about the impact of delegation on safety. According to them delegation should contribute to a reduction of their workload as well as enable separation distance to be more accurate. Moreover, they feel delegation should increase pilots’ situation awareness and therefore enhance their contribution to the safety of the overall system. Yet, one issue raised by controllers is the risk of lesser involvement in delegated situations. It should be noticed however that the experimental situation (e.g no real warning or refusals from pseudo-pilots, no interaction with controllers on feed sectors, no alerting tools on the radar screen) might have "encouraged" this form of disengagement. In addition to the issue of responsibility, controllers questioned the applicability of delegation to non-nominal situations and mixed equipages (aircraft equipped and non-equipped). The main potential negative impact of delegation is rather related to their doubts regarding the pilots’ performances, and in particular to the issue of responsibility, combined with pre-existing ideas about pilots attitudes. The analysis of objective data (loss of separation, API) shows cases, both without and with delegation, when the safety is endangered. Yet, we felt that these figures were not sufficient to understand the impact of delegation. Therefore it appeared essential to identify complementary indicators to investigate further in order to measure and qualify this impact. The first indicator that seemed relevant was the conditions of transfer. Indeed, observing controllers during the experiments enabled us to detect unstable situations, when aircraft transferred to the next sector might not remain separated. Mainly because of differences in speed, it happened in situations
9
International Workshop on Human Error, Safety and System Development (HESSD) 11-12 June, Linköping, Sweden
without delegation that traffic was transferred to the approach in catching up situation. Yet, it is important to clarify that the next sectors were not controlled in our simulations. This could explain a lack of consideration of these conditions of transfer. It is interesting to notice that delegation contributed to an improvement of these conditions of transfer. Indeed, with delegation, less situations of “catching up on target” were observed. Regarding conditions of transfer, while controllers did understand and respect applicability conditions while initiating the delegation, they usually fail to continuously update the assessment of these conditions. While it is obvious for them that conditions evolve with time (e.g. closure rate, or aircraft speed are impacted by changes in aircraft level), it seems as if they forget that once established the delegation conditions might change. Our assumption to explain this is related to the controllers’ overconfidence and excessive expectations from the limited delegation. It seems as if they transferred to the flight crew not only the implementation of the task, but also its maintaining despite unsuited conditions. The microscopic analysis of controllers’ performances (using replay tools) enabled us to identify various cases when the applicability conditions were not respected. These cases can be split in three categories, in relation to underlying mechanisms: -
Failure of expertise: the controllers did know the conditions, but did not consider the situation properly, misidentifying it. For example, in some cases, whereas they assessed the initial conditions, they did not update this assessment and fail to anticipate the consequences of changing aircraft level.
-
Lack of expertise: because of the short training provided, it happened that controllers did not fully integrate these conditions, and misuse delegation in context where it was not appropriate.
-
Overconfidence: with time and increased practice, the controllers assess the relevance of delegation, and especially in the simulation context, its effectiveness. Then, they gradually expect more and more from delegation. They finally expect too much from delegation, using it in more and more extreme situations. They tend to forget one of the initial principles of delegation, that is no change in current practices and the fact that delegation does not provide better aircraft performances.
From error types to design requirements In our simulations, safety is analysed at two levels: the error production (causes or origins of errors) and the error consequences (failures observed, either under the form of losses of separation, or dangerous conditions of transfer). Among various origins to observed failures we can identify for example, the choice of erroneous instructions, the non-respect of the applicability conditions, or the selection of a wrong target. Observations of simulations as well as the analysis of debriefing items stress some potential sources of errors, introducing unsafe events. In reference to existing error taxonomy (Reason, 1990; Norman, 1981; Hollnagel, 1993), these sources of errors can be described in some cases in terms of slips (erroneous execution) and in other cases in terms of mistakes (errors in planning actions). The following table summarises the diverse errors we have identified (some observed, other suspected) and classifies them according to the slips (errors in the execution) and mistakes (errors in the planning, either due to a failure or to a lack of expertise) distinction. In each case, we try to identify events or reasons that could cause the occurrence of these errors, and when possible we suggest possible means supporting their prevention or tolerance. It is worth noticing that most of the means envisaged for error tolerance rest on the use of existing interactions between controllers and pilots. It is proposed to introduce safety means in the system, in not only maintaining existing redundancies, but also introducing automatic support for error detection and recovery. Because we insist in keeping close interactions between controllers and pilots, we suggest to use these interactions as a support for detection.
10
International Workshop on Human Error, Safety and System Development (HESSD) 11-12 June, Linköping, Sweden
Table 1 : Types of errors identified, associated to supports envisaged for their prevention and tolerance Error Pilots
Possible causes
Slip in entering target code Mistake in entering target code
Possible means to prevent and/or tolerate Position target (detect if target appearing on cockpit displays is where expected target should be)
Misunderstanding (quality of communication) Erroneous controller instruction
Cockpit display informing about unknown target or alerting for inconsistent target-own configuration Position target
Mistake in selecting instructions (on interface)
Misunderstanding instruction (e.g. remain “at least”)
Mistake in identifying required action (speed/heading instructions)
Overload (cognitive, physical)
Mistake in performing actions : omission of resume action
Overload
Pressing wrong key
Failure of expertise
Clear and unambiguous phraseology Usable interface and possibility to undo actions System suggestion of appropriate action System detection and information if selected action not consistent with given constraints
Lack of situation awareness
System detection of point of resume and alert in case of overshooting
Failure / Lack of expertise Slip in performing required action Controllers
System detection if selected actions different from suggested one, and request for confirmation
Mistake in reading target code
Overlapping labels
Efficient Anti-overlapping system
Mistake in selecting target
Overlapping labels
Efficient Anti-overlapping system
Failure of expertise (e.g. faster aircraft used as delegated and slower as target)
Alerting function for inconsistent target-own configuration (on ground and on board)
Mistake in identifying appropriate instruction (strategy)
Failure of expertise: erroneous assessment of applicability conditions
Airborne system detecting inability to comply with instructions
Redundant availability of target code (radar label, delegation menu and aircraft list)
Lack of expertise: absence of knowledge of the applicability conditions Slip in giving instruction
Misspelling
Controller or pilot detection according to situation
Failure to detect loss of separation
Lack of situation awareness
Alert support systems (both on ground and on board)
Overconfidence Overload
11
International Workshop on Human Error, Safety and System Development (HESSD) 11-12 June, Linköping, Sweden
Conclusion This initial investigation of the concept of limited delegation highlights its operational acceptance, even though it requires complementary studies, focusing in particular on the joint air and ground system. It is planned to perform first microscopic analysis of the controllers and pilots tasks in order to understand local impact of delegation. Second, a joint analysis at a macroscopic level should inform on the impact of delegation on the overall system. The realism of the experiments should also be improved through the development of pseudo-pilots “unable” messages, controllers monitoring aids, and the increasing of the feed sector role. One of our assumptions in terms of safety is that monitoring tasks should be modified with the introduction of delegation. Indeed, while we expect controllers’ monitoring tasks to be reduced, we simultaneously expect the pilots’ ones to be increased. In particular, we assume that the rent role played by pilots in the monitoring of traffic (typically using party line information) will become more explicit than it is today. From a safety perspective, it is necessary to foresee not only the benefits of such an increased role but also its negative impact (e.g. information overload). One way to address this issue of monitoring tasks will consist of setting up experiments investigating controllers and pilots’ situation awareness in both situations. The process of analysing data enabled us to identify the need for complementary indicators informing about the impact of delegation on safety. Yet, their current measure is not optimised, in the sense that it is time consuming for the analysts (e.g. replaying and analysing every sequence) and requires combined expertise in different domains (human factors, ATC, safety). Even though the involvement of expert is absolutely not questioned, especially in phases of interpretation, we suggest that some indicators could be automatically calculated from the system recording. Conditions of transfer for example might be calculated (e.g. comparing aircraft respective speed). Similarly, it might be possible to detect violations of the applicability conditions. If such calculations were possible, then the experts could focus on the detailed analysis of unsafe events rather than initially replay every exercise in search of such events.
References Billings, C. E. (1997). Aviation automation. The search for a Human-Centered Approach. L. Erlbaum Publishers. NJ. Casaux, F. & Hasquenoph, B. (1997) Operational use of ASAS, USA/Europe ATM R&D Seminar, Saclay, France. Corker, K., Flemming, K. & Lane, J. (1999). Measuring controller reactions to free flight in a complex transition sector. Journal of ATC, Oct.–Dec. De Keyser, V. (1991). Work analysis in French language ergonomics: origins and current research trends. Ergonomics, 34 (6): 653-669, 1991. Dekker, S. & Woods, D. (1999). To intervene or not to intervene: the dilemma of management by exception. Cognition, Technology & Work, 1, 86-96. Floyd, C. (1984). A Systematic Look of Prototyping. In R. Budde, K. Kuhlenkamp, L. Mathiassen, & H. Züllighoven (eds.), Approaches to Prototyping. Berlin: Springer Verlag, 1984, pp. 1-18. Gould, J.D. (1988). How to design usable systems. In M. Helander (ed) Handbook of HumanComputer Interaction (pp. 757-789). Elsevier: North Holland. Gould, J.D. & Lewis, C. (1985). Designing for usability: Key principles and what designers think. Communications of the ACM, 28, 300-311. Grimaud, I., Hoffman, E, Rognin, L. & Zeghal, K. (2001). Involving pilots and controllers in the evaluation of the limited delegation concept. 11th International Symposium on Aviation Psychology, Columbus, USA. Haraldsdottir, A., Schwab, R.W. & Alcabin, M.S. (1998). Air Traffic Management capacity-driven operational concept through 2015. 2nd USA/Europe ATM R&D Seminar, Orlando. Hoffman, E., Zeghal, K., Cloerec, A., Grimaud, I. & Nicolaon, J.P. (1999). Operational concepts for limited delegation of separation assurance to the cockpit. AIAA Guidance, Navigation and Control Conference, Portland, Oregon. Hollnagel, E. (1993). Human Reliability Analysis. Context and Control. London: Academic Press.
12
International Workshop on Human Error, Safety and System Development (HESSD) 11-12 June, Linköping, Sweden
Hollnagel, E. & Woods, D. (1983). Cognitive Systems Engineering: New wine in new bottles. International Journal of Man-Machine Studies, 18,583-600. Hughes, J.A., Randall, D., & Shapiro, D. (1992). Faltering from Ethnography to Design. In J. Turner & R. Kraut (Ed.), Conference of Computer Supported Cooperative Work, (pp. 115-122). Oct 31 -Nov 4, Toronto, Canada. ACM Press, NY. Johnson, W., Battiste, V. & Holland, S. (1999). A Cockpit display designed to enable limited flight deck separation responsibility. SAE/AIAA World Aviation Congress, San Francisco. Laprie, J.-C. (1995). Dependability- Its attributes, impairments and means. In B. Randell, JC. Laprie, H. Kopetz, B. Littlewood (eds), Predictably Dependable Computing Systems. Springer, N°ISBN 3540-59334-9, pp.3-24 Monk, A., Wright, P., Haber, J. & Davenport, L. (1993). Improving your human-computer interface : A practical technique. Prentice Hall. Norman, D.A. (1981). Categorization of actions slips. Psychological Review, 88 , 1-15. Norman, D. A. & Draper S. W. (1986). User Centered Systems Design. L. Erlbaum Assoc. NJ. Paul, L. (1990). Using simulation to evaluate the safety of proposed ATC operations and Procedures (DOT/FAA/CT-TN90/22). Atlantic City, NJ: DOT/FAA Technical Center. Perlman, G. (1988). Software tools for user interface development. In M. Helander (ed) Handbook of Human-Computer Interaction (pp. 819-833). Elsevier: North Holland. Pritchett, A.R., Yankosky, L.J. & Haserueck, D. (1999). Simultaneous design of cockpit display of traffic information and air traffic control procedures. In 10th International Symposium on Aviation Psychology (pp.88-94), Columbus, USA. Reason, J. (1990). Human Error. Cambridge University Press. Rochlin, G. (1996). The Social Construction of Safety. In 2nd International Conference on Nuclear Power Operations (ICNPO), November, Berlin, Germany. Rognin, L., Salembier, P., & Zouinar, M. (2000). Cooperation, reliability of socio-technical systems and allocation of function. International Journal of Human-Computer Studies, 52 (2), pp.357-379. Schneiderman, B. (1992). Designing the User Interface. Addison-Wesley. Schuler, D. & Namioka, A. (1993), Participatory design: principles and practices. L. Erlbaum Assoc. NJ. Wickens, C. D., Mavor, A. S. & McGee J. P. (1997). Flight to the Future: Human Factors in Air Traffic Control. National Academy Press. Zeghal, K., Grimaud, I. & Hoffman, E. (2000). Evaluation of delegation of sequencing operations to the flight crew from a controller perspective – Preliminary results. SAE/AIAA World Aviation Congress, San Diego. Zeitlin, D., Hammer, J., Cieplak, J. & Olmos, B.O. (1998). Achieving early CDTI capability with ADS-B, USA/Europe ATM R&D Seminar, Orlando.
13
