Airport Capacity Prediction with Explicit Consideration of Weather

JOURNAL OF AIR TRANSPORTATION Vol. 24, No. 1, January 2016

Airport Capacity Prediction with Explicit Consideration of Weather Forecast Uncertainty Rafal Kicinger∗ and Jit-Tat Chen† Metron Aviation, Dulles, Virginia 20166 and Matthias Steiner‡ and James Pinto§ National Center for Atmospheric Research, Boulder, Colorado 80301 DOI: 10.2514/1.D0017 This paper describes a stochastic analytical model for predicting airport capacity with a look-ahead horizon suitable for strategic traffic flow management. The model extends previous research on airport capacity estimation by explicitly integrating the impact of terminal weather and its uncertainty. Different types of weather forecast inputs are explored, including deterministic forecasts, deterministic forecasts with forecast error models, and ensemble forecasts, to produce distributions of predicted arrival and departure capacity for each runway configuration at an airport. The paper introduces a mathematical capacity prediction model and weather data sources supported by a proof-of-concept prototype implementation, including results of validation studies at Hartsfield–Jackson Atlanta International Airport. These results are compared with standard airport benchmark capacities and actual observed throughputs. Results show that our analytical airport capacity model accurately predicts maximum available airport capacity for Hartsfield– Jackson Atlanta International Airport for different weather conditions. The validation studies reveal a limited impact of forecast uncertainty representation on the accuracy of airport capacity predictions.

management [1,2]. Achieving accurate airport capacity prediction, however, is difficult as it depends on many interrelated factors, including operational standards and procedures, runway configuration and status, meteorological conditions, and expected air traffic demand mix [3]. Accurate capacity prediction becomes more challenging for long look-ahead times required for strategic planning (i.e., for time horizons of 4 to 8 h and beyond) as the uncertainty inherent in many factors affecting airport capacity (most notably weather) is increased significantly. Terminal weather conditions, including ceiling and visibility (C&V), winds, precipitation, and convective storm activity, are among the key factors having direct (e.g., available runways and airport meteorological conditions [MCs]) and indirect (e.g., separation standards) impacts on airport operations [4,5]. Most of the existing analytical airport capacity models [6–9] only implicitly consider weather impacts, typically achieved by assuming the MCs at an airport [i.e., instrument meteorological conditions (IMCs), marginal visual meteorological conditions (MVMCs), or visual meteorological conditions (VMCs)] as an input to the model. Those implementations of airport capacity models that do consider weather impacts (e.g., in [7]), so far, have used deterministic weather information only. Therefore, such airport capacity predictions do not account for forecast uncertainty. The integrated airport capacity model (IACM) [10] was developed specifically to address this gap. The IACM explicitly integrates convective storm, C&V, and surface wind forecasts to produce probabilistic arrival and departure capacity predictions for each runway configuration at an airport. Initially, the IACM incorporated deterministic weather forecasts and associated forecast error models derived using historical weather data [10]. The IACM was subsequently extended by Kicinger et al. [11] to use ensemble forecasts, an emerging class of weather prediction methods that aims to generate a representative sample of the possible future states of the atmosphere [12]. This paper describes results of qualitative and quantitative validation studies for Hartsfield–Jackson Atlanta International Airport (ATL), contrasting the IACM’s airport capacity predictions with actual observed arrival and departure throughputs. The paper also discusses results of studies comparing the accuracy of airport capacity predictions as a function of different types of weather forecast inputs, including deterministic forecasts, deterministic forecasts with forecast error models, and ensemble forecasts.

Nomenclature Ai bDEP bMIT bREL

= = = =

D Pi

= =

Si;j

=

T Tc Tm Ts Vi μ

= = = = = =

actual hourly traffic count separation buffer for consecutive departure release, s final approach separation buffer, n mile separation buffer for departure release between two successive arrivals, n mile length of common approach path, n mile predicted hourly traffic rate computed by Integrated Airport Capacity Model minimum separation requirements between leading aircraft of type i and following aircraft of type j, n mile Theil inequality coefficient Theil inequality coefficient incomplete covariation Theil inequality coefficient inequality proportion Theil inequality coefficient error in trend speed of aircraft i, kt minimum time separation between two consecutive arrivals, s

Introduction

A

IRPORT capacity estimates are among the key inputs critical for implementing many of today’s traffic management initiatives (TMIs). Inaccurate capacity predictions, both over- and underpredictions, result in unnecessary delays, either airborne or on the ground, and contribute to inefficiencies in today’s air traffic

Received 28 January 2015; accepted for publication 8 December 2015; published online 29 February 2016. Copyright © 2015 by Metron Aviation, UCAR, and STAR. Published by the American Institute of Aeronautics and Astronautics, Inc., with permission. Copies of this paper may be made for personal and internal use, on condition that the copier pay the per-copy fee to the Copyright Clearance Center (CCC). All requests for copying and permission to reprint should be submitted to CCC at www.copyright.com; employ the ISSN 2380-9450 (online) to initiate your request. *Principal Analyst, 45300 Catalina Court, Suite 101. Member AIAA. † Lead Analyst, 45300 Catalina Court, Suite 101. Member AIAA. ‡ Deputy Director, Aviation Applications Program, 3450 Mitchell Lane; also Consultant, Science and Technology in Atmospheric Research (STAR) Institute, 3125 Sterling Circle, Suite 107, Boulder, CO 80301. Member AIAA. § Project Scientist, Aviation Applications Program, 3450 Mitchell Lane; also Consultant, Science and Technology in Atmospheric Research (STAR) Institute, 3125 Sterling Circle, Suite 107, Boulder, CO 80301. 18

KICINGER ET AL.

Fig. 1 Schematic tradeoff between the AAR and ADR for a single runway.

Airport Capacity Prediction Airport capacity can be expressed in terms of two interdependent capacities: arrival capacity, expressed by the airport arrival rate (AAR), and departure capacity, expressed by the airport departure rate (ADR). The relationship between the AAR and ADR for a single runway is nonlinear [9,13,14] and typically characterized by a function defined by three or more points, each describing the relationship between AAR and ADR, depending on how a runway is used operationally, i.e., exclusively for arrivals, exclusively for departures, or mixed use when arrival and departure operations are allowed from the same runway (Fig. 1). Various airport capacity models, both analytical and simulation based, have been developed [7,9,14–17]. Analytical models use mathematical representations of airport operations and derive estimates of its capacity [18]. Simulation-based models determine airport capacities by creating virtual components of airport operations and simulating their flow. In general, analytical models are typically macroscopic. They use abstract, simplified, and aggregate quantities of interest, and they model airport operations at a relatively low level of detail. Hence, the estimates produced by these models are approximations and more suitable for strategic rather than for tactical decision support. In contrast, simulation-based models provide a much greater level of detail in modeling airport operations, typically at the expense of significantly longer computational time. Examples of analytical airport capacity models include the Federal Aviation Administration (FAA) Airfield Capacity Model [9], the Logistics Management Institute Airport Capacity Model [7,19], the Boeing Airport Capacity Constraints Model [6], a class of integrated analytical airport capacity and delay models called MANTEA Airfield Capacity And Delays (MACAD) [8], and the IACM [10].

Fig. 2

19

The IACM was used in the validation studies reported in this paper. It combines surface, terminal, and transition airspace models into an analytical stochastic airport capacity model that provides probabilistic airport capacity estimates (see Fig. 2). The model works with various types of weather forecast inputs, including deterministic forecasts, deterministic forecasts with forecast error models, and ensemble forecasts. The IACM consists of three major components shown in Fig. 2: 1) the terminal capacity model (TCM), 2) the airfield capacity model (ACM), and 3) the multicriteria capacity forecast integrator (MCFI). The TCM takes into account predicted weather conditions in the terminal airspace, including precipitation and echo tops forecasts, and produces probabilistic capacity estimates (one probability distribution per look-ahead hour) for the terminal airspace. The ACM computes probabilistic capacity estimates of the runway system based on predicted weather conditions at the airport surface (i.e., C&V and winds). These independently obtained capacity distributions from the TCM and ACM are subsequently combined by the MCFI component, which produces an integrated probabilistic airport capacity forecast for each look-ahead hour. Figure 2 shows that the IACM incorporates the following major groups of inputs: 1) weather forecasts that include C&V, surface winds, precipitation, and echo tops; 2) predicted demand data, providing information on the number of arrivals and departures, and their weight class mix; 3) airport adaptation data, defining layout and geometry of an airport, historically used runway configurations, expected runway occupancy times (ROTs), and approach speeds as well as their variations; and 4) operational standards and procedures that define constraints on minimum aircraft separation in the terminal airspace and along the common approach path (these standards and procedures are obtained from the Air Traffic Control Handbook [20], which provides required arrival and departure separations as well as wake vortex separations). The first two groups of inputs (i.e., weather forecasts and predicted demand) can be provided to the IACM in real time when the IACM is set to compute capacity predictions with respect to the current time or historical weather forecast, and demand data can be used when the baseline time for the IACM capacity predictions is set in the past (as was the case with the validation studies reported in this paper). The TCM produces capacity estimates for the terminal airspace around an airport. It uses the convective weather avoidance model [21], an extension of the Maximum Flow, Minimum Cut Theory [22], and computational geometry algorithms [22,23] to identify airspace blockages or no-fly zones for departing and arriving flights in the terminal airspace. The no-fly zones determined by weather avoidance fields are then used to determine if a particular arrival or departure fix is deemed to be “open” or “closed” because of weather blockage [10].

Major IACM components and inputs.

20

KICINGER ET AL.

Fig. 3

Flow of information and major processing steps for the airfield capacity model.

The ACM computes capacity estimates at the airport surface as a function of projected demand mix and forecasted weather conditions, specifically surface winds, cloud ceiling height, and visibility. C&V conditions directly affect the flight rules at the airport, determining the appropriate separation rules. Wind conditions impact both crosswind and tailwind components upon aircraft, influencing how runway operations are configured. The flow of information and major processing steps for the ACM are shown in Fig. 3. The figure shows that the ACM is composed of two components, the runway configuration estimator and runway capacity model (RCM). The former uses a set of historically used runway configurations (provided as part of airport adaptation data) and either forecast or simulated (Monte Carlo) surface wind conditions to determine usability of each runway in a given configuration. The usability is determined considering crosswind and tailwind restrictions; when either the crosswind or tailwind component exceeds the corresponding operational thresholds, the runway is assumed unusable. The RCM component computes a capacity estimate for each usable runway of a given runway configuration, taking into consideration forecast or simulated (Monte Carlo) C&V conditions. The underlying analytical model used by the IACM to compute runway capacity is based on a slightly modified version of the MACAD model [8]. In particular, modifications to the original MACAD model include incorporating controller separation buffers that are frequently employed by air traffic controllers in today’s

operations to account for weather (e.g., wind and storms) on the final approach course (Fig. 4), arrival compression, aircraft-type speed variance, differences in pilot behavior, etc. [24]. Similarly to MACAD, the RCM considers three cases for defining controller separation constraints: 1) Case 1 is the separation between successive arrivals at the start of a common approach path. 2) Case 2 is the same as Case 1 but also allows for at least one departure between these two arrivals. 3) Case 3 is the separation between successive departures at the runway threshold. The controller separation between successive arrivals at the start of a common approach path (Case 1) must satisfy two constraints: A) Constraint 1 is the miles-in-trail (MIT) separation requirement: aircraft must maintain the MIT separation throughout the entire length of the common approach path (Fig. 4). B) Constraint 2 is the single runway occupancy requirement: the following aircraft (follower) does not cross the runway threshold until the leading aircraft (leader) has exited the runway. Because aircraft maintain constant speed during their final approach, the MIT separation requirement needs to consider two cases based on the approach speed of the follower V F and the speed of the leader V L : 1) For the follower gaining on the leader, V F ≥ V L , the smallest distance between the follower (gainer) and leader occurs just before the leader leaves the common approach path.

Fig. 4 Miles-in-trail separation requirements between two arriving aircraft along a common approach path.

21

KICINGER ET AL.

2) For the follower losing the leader, V F < V L , the smallest distance between two arriving aircraft occurs at the entry point of the common approach path. The MIT separation requirement incorporating separation buffer bMIT (Fig. 4) for the case of the follower of type j gaining on the leader of type i is defined as
 D D −
V j  δV j  · − μ ≥ Si;j  bMIT (1) V i  δV i where D is the length of the common approach path, V i denotes the speed of the leading aircraft i, δV i denotes the uncertainty in the approach speed of the leading aircraft i (assumed to be a normal random variable with mean zero and standard deviation σ Vi ), V j denotes the speed of the following aircraft j, δV j denotes the uncertainty in the approach speed of the following aircraft j (assumed to be a normal random variable with mean zero and standard deviation σ Vj ), Si;j defines the minimum separation requirements between the leading aircraft of type i and the following aircraft of type j, and μ defines the minimum time separation imposed by controllers between two consecutive arrivals. Note that the time a leader spends in the common approach path is D∕
V i  δV i . Thus, the time that a follower is in the common approach path before the leader enters the runway is D∕
V i  δV i  − μ. Equation (1) leads to the formula for μ, μ≥

D D −
Si;j  bMIT  − Vj Vi

with 95% probability

(2)

where a 95% confidence level is assumed because the approach speed for the leading and following aircraft are modeled as random variables. Similarly, the constraint for the case in which the speed for a leader of type i is higher than the speed for a follower of type j is defined by μ
V i  δV i  ≥ Si;j  bMIT

(3)

which leads to a similar constraint on the minimum time separation μ: μ≥

Si;j  bMIT Vi

with 95% probability

(4)

Similar separation buffers were introduced for the two remaining cases defining controller separation constraints, i.e., a separation buffer for departure release bREL between two successive arrivals, which is added to the minimum required separation of the arriving aircraft from the runway threshold, and a separation buffer for consecutive departure release bDEP . The capacity of a runway configuration computed by the ACM under the considered terminal weather conditions is obtained by aggregating the capacities of all usable runways. All arrival runways for a given runway configuration contribute to the AAR only, all departure runways contribute to the ADR, and contributions of shared runways to the AAR and ADR are computed using each of the four capacity envelope points shown in Fig. 1. The IACM was used in several studies focused on estimating and validating weather-dependent capacities at three major U.S. airports, including the ATL, Dallas–Fort Worth International Airport, and Chicago O’Hare International Airport. The model was also applied to conduct airside capacity enhancement studies for several South African airports, including O.R. Tambo International Airport, King Shaka International Airport, and Cape Town International Airport [25].

Hartsfield–Jackson Atlanta International Airport The analytical studies described in this paper focused on validating the IACM airport capacity predictions for ATL. This airport was selected because it is one of the busiest airports in the U.S. and has a relatively simple layout, enabling detailed studies and sensitivity

Table 1

Maximum and average top-100 hourly demands at ATL

Year Maximum hourly demand Average top-100 hourly demand 2007 236 226 2008 253 218 2011 205 194

analysis of weather impact on airport capacity. In particular, ATL’s runway system consists of five parallel runways (i.e., 8L-26R, 8R26L, 9L-27R, 9R-27L, and 10-28) that are sufficiently separated from each other so that their operations are not interdependent. The set of historically used runway configurations at ATL, used by the IACM as inputs, was extracted from the National Traffic Management Log. This set was large and included 35 runway configurations since we considered all historically used configurations even though their operational usage may have been marginal. Other required IAMC inputs, namely, ROTs and approach speeds, were determined by analysis of Airport Surface Detection Equipment Model X surface data as described in [26]. The C&V minima determining VMCs, MVMCs, and IMCs for ATL were taken from the 2004 Airport Capacity Benchmark Report [27]. Following recent analysis of crosswind and tailwind impacts reported in [28], crosswind and tailwind thresholds for determining runway usability were assumed equal to 30 and 20 kt, respectively. Wake vortex separations were assumed the same as in previous work [10], i.e., consistent with current operational requirements defined in the Air Traffic Control (ATC) handbook [20]. Since the IACM computes the maximum available airport capacity, we focused on analyzing days and specific hours at ATL that had the highest throughput for each of the three considered years (i.e., 2007, 2008, and 2011). In particular, we conducted a comparative analysis of the maximum hourly demand and average top-100 hourly demand (see Table 1) for these three years and selected top-10 demand hours for each year for conducting sensitivity analyses. Table 1 shows that the top hourly demand in 2011 was 15–20% lower when compared to 2007 and 2008.

Weather The IACM explicitly integrates the impact of terminal weather and its uncertainty. In our studies, we explored different types of weather inputs for the IACM, including observations, deterministic forecasts, deterministic forecasts with forecast error models, and ensemble forecasts. The latter two weather inputs were used to produce distributions of predicted arrival and departure capacity for each runway configuration at ATL. In particular, Meteorological Aerodrome Report (METAR) observations were used as “perfect forecasts” for conducting a sensitivity analysis and calibration of the IACM parameters. We also considered deterministic forecasts produced by the high-resolution rapid refresh (HRRR) model [29], which runs at a spatial resolution of 3 km, uses advanced data assimilation to include observational METAR cloud base information and radar reflectivity, and nominally produces a new 15 h forecast hourly. Forecast error models can be used to generate probabilistic forecast information from a single deterministic run. Forecast error models were developed by finding relationships between HRRR forecasts of each aviation hazard (i.e., wind and C&V) and the METAR observations at ATL (see the Appendix). These forecast error models are used by the IACM in Monte Carlo simulations to perturb a single bias-corrected deterministic HRRR forecast with known error distributions to represent forecast uncertainty. Probabilistic forecast information can also be generated using ensemble forecasts that can be generated in a variety of ways [12]. In our work, we used time-lagged ensemble forecasts created by aligning multiple HRRR forecasts (of varying issue and lead times) available for a given valid time. The validation studies focused on computing and analyzing capacity estimates of an airport’s runway system as a function of terminal

22

KICINGER ET AL.

Fig. 5

Available case days for IACM validation studies.

weather conditions. Therefore, only surface weather conditions were considered, including C&V and surface wind forecasts. Terminal convection forecasts, albeit supported by the IACM, were not explicitly included in the analysis, except in cases in which convection was forecast to impact winds and C&V at the airport. Based on available traffic demand and weather forecast data (HRRR forecasts were available for 2011 only), we used a set of 22 scenario days spanning different types of weather impacts at ATL that were obtained from the 2011 data. The case days that were selected include a number of clear and calm days (6) and weather-impacted days (16) that had either wind constraints and/or C&V constraints as shown in Fig. 5.

Experiments We divided the experiments into two major groups: 1) sensitivity analysis aimed at calibrating IACM parameters and 2) validation studies contrasting IACM airport capacity predictions against the actual throughput at ATL. We applied a modified version of the methodology for validating runway capacity models described in [30] for conducting both groups of experiments. This methodology consists of qualitative validation with scatter plots and quantitative validation involving the Theil inequality coefficient T [31], P T

i


Pi − Ai 2 P 2 i Ai

(5)

where Pi and Ai denote a predicted hourly rate (AAR or ADR) computed by the IACM and actual hourly count obtained from the Aviation System Performance Metrics (ASPM), respectively. Sensitivity Analysis

The objectives of this group of experiments included determining the sensitivity of the IACM predictions to key parameters and calibrating the model for conducting validation studies for ATL. The sensitivity studies focused on the following parameters: 1) the final approach separation buffer bMIT (Fig. 4), 2) the separation buffer for departure release bREL , and 3) the separation buffer for consecutive departure release bDEP . As described earlier, these parameters define buffers that are added to the minimum required separation requirements defined for each MC and used as inputs by the IACM. In the context of the IACM modeling framework, these buffers also help in calibrating the model for a specific airport by accounting for inaccuracies and simplifications of the model in capturing the complexities of airport operations. A wellcalibrated model will have relatively small values of these buffers; on the other hand, excessively large buffer values may indicate inappropriate modeling assumptions or other modeling errors.

In the sensitivity analysis studies for ATL, we determined the range of values for a given parameter, run the model for each studied parameter with values sampled across this range, and subsequently prepared scatter plots and computed Theil inequality coefficients based on model outputs. The optimal parameter values, i.e., the ones associated with the smallest values of T, were subsequently used for conducting detailed model validation studies, as reported below. In an effort to control external biases and variability in model inputs, we used METAR observations (i.e., perfect forecasts) as the only type of weather input in this sensitivity analysis. Figure 6a shows the results of a sensitivity analysis for the final approach separation buffer bMIT stratified by year (2007, 2008, and 2011). In this analysis, the value of the separation buffer was varied from 0 to 3 n mile in 0.5 n mile increments. Clearly, the accuracy of arrival capacity predictions for VMC, as measured by the Theil coefficient, exhibits significant sensitivity to the separation buffer bMIT . Best predictions are generated when bMIT was equal to 1 n mile for the 2007 and 2008 data sets, while the optimal value of bMIT reached 2 n mile for the 2011 data set. This 1 n mile buffer value compares well with actual separation buffers employed by ATC at ATL as measured by the spacing efficiency tool, an FAA automation tool that measures the distance between subsequent arrivals on final approach and compares actual to required separation. Figure 6b shows a similar sensitivity analysis for the consecutive departures separation buffer bDEP. In this case, the value of the separation buffer was varied from −30 to 50 s in 10 s increments. The results show the impact of bDEP on the accuracy of departure capacity predictions for VMCs. Interestingly, for 2007, the optimal value of the bDEP was negative. This can be explained by the fact that in cases of high departure demand ATC is allowed to reduce minimum required separation between aircraft when the successive departure paths are separated by 15 deg or more [20]. In general, good predictions for 2007 and 2008 were achieved when bDEP was equal to zero s (no buffer), and hence we set bDEP  0 s as the optimal parameter value for validation studies. Finally, a sensitivity analysis of the separation buffer for departure release bREL (not shown) found the IACM’s performance to be quite insensitive to its value. A value of bREL  0.2 n miles was assumed in the remainder of the runs. Figure 6 reveals potential pitfalls of calibrating the model based on historical throughput data. In particular, because of a reduced demand in 2011 compared to 2007 and 2008 (Table 1), the derived separation buffers are significantly higher and may cause model overfitting that affects the accuracy of IACM airport capacity predictions. Since the IACM estimates the maximum airport capacity, and peak hourly demand in 2007 and 2008 at ATL was close to or at capacity, the separation buffers derived based on 2007 and 2008 data define a better calibration of model parameters than that obtained for 2011. At the same time, the impact of a reduced demand in 2011 must be

23

KICINGER ET AL.

,

,

a)

b) Fig. 6 Sensitivity of final approach and consecutive departure separation buffers on the accuracy of arrival and departure capacity predictions for VMCs.

Fig. 7

a) b) Predicted a) arrival and b) departure capacities based on using deterministic forecasts vs actual traffic counts for selected scenario days in 2011.

accounted for when assessing the accuracy of IACM predictions against the actual throughput data as discussed in what follows. Validation Studies

In the second group of studies, IACM airport capacity predictions were compared with actual airport throughputs for data collected in 2011. The examination focused on how well deterministic and probabilistic capacity predictions may reflect the observed airport throughput and what difference the choice in weather uncertainty representation might make. The following aviation data sources were considered in these analyses: 1) First is scheduled hourly counts of arriving and departing flights at ATL obtained from the ASPM. These counts were further subdivided into the following aircraft weight class categories: small, large, heavy, and other. The ratios of aircraft weight class categories were used as inputs for IACM airport capacity predictions. 2) Second is a set of runway configurations at ATL obtained from the ASPM database. The ASPM determined one airport configuration for each hour, and we used the IACM to generate airport capacity predictions for this runway configuration and subsequently compared it to the actual throughput. 3) Third is actual hourly counts of arriving and departing flights at ATL obtained from the ASPM and also subdivided into aircraft weight class categories.

In this group of experiments, we used three types of weather inputs: deterministic forecasts, deterministic forecasts with forecast error models, and ensemble forecasts. We identified hours with significant throughput at ATL for each of the considered scenario days (see Fig. 5). For each hour i, we used the IACM to generate capacity predictions PAi and PD i for arrivals and departures, respectively, for the runway configuration that was actually used at ATL during this hour as reported in the ASPM. We subsequently conducted a qualitative analysis using scatter plots and quantified the prediction accuracy using the Theil inequality coefficient T and rootmean-square error (RMSE) values, where lower values of the Theil inequality coefficient define better predictive capability. Figure 7 shows scatter plots comparing predicted airport capacities (arrivals and departures) to actual traffic counts generated using deterministic forecasts.¶ The vertical axes show capacity predictions, whereas the horizontal axes define actual throughput counts. The scatter plots include data from all days in Fig. 5. Different markers in Fig. 7 indicate different meteorological conditions at the airport (IMCs, MVMCs, and VMCs).

¶ Qualitatively similar plots were obtained for deterministic forecasts with forecast error models and ensemble forecasts, and hence these plots have been omitted.

24

KICINGER ET AL.

Table 2 Average top 100 hourly demands from 2007 to 2008 for different meteorological conditions Type VMCs MVMCs IMCs Arrival 118 110 95 Departure 110 116 124 Total 228 226 219

The diagonal line corresponds to an idealized scenario in which predicted counts exactly match the actual counts. This idealized line is only valid for analyzing IACM prediction errors for the cases in which demand at the airport exceeds available capacity. Otherwise, offsets from the diagonal line may be caused by IACM prediction errors and/or lack of sufficient demand to use available airport capacity. Since the data sample shown in Fig. 7 comes from 2011 when ATL experienced a reduced traffic demand, most of the analyzed hours correspond to cases in which the demand was less than the available capacity. Hence, another approach to validation was taken in which the predicted hourly capacity Pi value generated by the IACM for each hour i was scaled using a scaling factor derived based on the ratio of actual hourly demand Ai in 2011 and baseline actual hourly demand Abase derived based on analysis of top-100 hourly demands from 2007 to 2008 shown in Table 2.

a) Fig. 8

The scaled predicted values Pi;s are computed using Eq. (6): Pi;s 

Ai P Abase i

(6)

Figure 8 shows the scaled version of a) arrival and b) departure airport capacities presented earlier in Fig. 7. The scaled predictions match the corresponding actual capacities much better than the unscaled predictions as the scatter plots are more closely aligned with the idealized diagonal line. The arrival capacities are slightly overpredicted for VMCs and slightly underpredicted for MVMCs and IMCs. A similar analysis of departure capacities (Fig. 8b) shows that the departure capacities are consistently underpredicted for all meteorological conditions. Both scatter plots in Fig. 8 contain several outliers. In particular, a group of four arrival capacity predictions (Fig. 8a) are significantly below the main points clustered around the diagonal 1∶1 correspondence line. Similarly, a few departure capacity predictions are located significantly above the diagonal line in the right scatter plot. Upon further inspection of the data, we discovered that all of them correspond to cases in which ATL used a configuration with a shared runway, i.e., the runway was used for both departures and arrivals. Figure 9 illustrates how the IACM generates arrival and departure capacity predictions using different assumptions about the relationship between the AAR and ADR for shared runways. If the

b) Same as Fig. 7, but scaled for reduced traffic demand in 2011.

a) b) Fig. 9 Scatter plots of scaled predicted a) arrival and b) departure airport capacities for VMCs generated with different assumptions on the relationship between the AAR and ADR for shared runways.

KICINGER ET AL.

Table 3

RMSEs and Theil coefficients for arrival capacity predictions

Weather Input Deterministic Deterministic  error model Ensemble

Ts Tc RMSE T Tm 3.61 0.0020 0.2885 0.0657 0.6458 2.00 0.0006 0.3259 0.0574 0.6167 3.41 0.0018 0.2809 0.0601 0.6590

25

analysis (using 2007 and 2008 data). During operational use, the IACM does not require a demand-level scaling, as it will be used to predict the maximum capacity for an airport, given meteorological conditions and demand mix. If the predicted capacity is below demand, TMIs may be needed to resolve the imbalance. Statistical Assessment

Table 4

RMSE and Theil coefficients for departure capacity predictions

Weather Input Deterministic Deterministic  error model Ensemble

Ts Tc RMSE T Tm 7.80 0.0104 0.8770 0.0188 0.1042 8.91 0.0136 0.9094 0.0283 0.0623 8.07 0.0112 0.8735 0.0221 0.1044

assumption is made that a shared runway is used exclusively for arrivals (see Point 1 in Fig. 1), then the IACM predictions of the AAR will be increased, while there will be no contribution of this runway to the predicted ADR. Conversely, if we assume that a shared runway is used exclusively for departures (see Point 4 in Fig. 1), then this runway’s contributions to the predicted AAR and ADR will be reversed. Assuming alternating arrivals and departures (see Point 3 in Fig. 1) will yield equal contributions of this runway to the predicted AAR and ADR, whereas assuming freely inserted departures (see Point 2 in Fig. 1) will contribute the same AAR as for all arrival points while also adding a small contribution to the predicted ADR allowed by accommodating departures in sufficient interarrival spacing gaps. The analysis of cases with shared runways for ATL showed that the best match is achieved assuming freely inserted departures; i.e., ATL typically uses shared runways for arrivals and departures that are allowed only when there is sufficient spacing between successive arrivals. This usage is illustrated in Fig. 9, which shows gray squares corresponding to freely inserted departures envelope points located much closer to the diagonal 1∶1 correspondence line as compared to other envelope points (i.e., all departures, alternating arrivals and departures, and all arrivals) denoted by gray triangles. With the assumption of using freely inserted departures envelope points to model shared runways, the outliers have now been eliminated by properly accounting in the IACM predictions for the operational procedures used at ATL. The analyses thus far have demonstrated that IACM is capable of predicting potential maximum arrival and departure rates. Moreover, initial differences between the predicted and actual observed throughput have been properly explained by deviations in traffic demand and runway usage. Therefore, the IACM model has been validated along with the buffer values determined through sensitivity

We are now in a position to evaluate differences in IACM predictions based on using varied weather forecasts and associated uncertainty. We computed Theil inequality coefficient T values (and its components T m , T s , and T c ) as well as RMSE values to quantitatively assess the differences in airport capacity prediction accuracy for different types of weather inputs used by the IACM. Table 3 reports T and RMSE values for arrival capacity predictions, whereas Table 4 shows similar results for departure capacity predictions. Note that lower RMSE and T values correspond to better airport capacity predictions. Table 3 shows that the coefficient T and RMSE for all arrival predictions are low and that they vary minimally between weather input types. The incomplete covariation T c represents the largest error subtype, followed by the inequality proportions T m . This finding suggests that point-to-point correspondence and offset errors mostly contribute to the overall prediction error for arrival, while the errors in trend T s are low. Table 4 shows that the coefficient T and RMSE for departure predictions are higher than for arrival predictions. This difference also shows that the offset error T m is a dominant subtype of error for departure capacity prediction, as illustrated in Fig. 9. Figure 10 compares mean prediction errors and their standard deviations for different types of weather inputs. The left plot shows prediction errors for arrivals, while the right plot shows the ones for departures. The smallest arrival prediction error and its variance were obtained for deterministic forecasts with the forecast error model, whereas departure prediction errors were essentially the same across all types of weather inputs. Even in the case of arrival prediction errors, the differences among different weather inputs are not statistically significant. The ensemble-based approach, however, can provide operationally relevant information about possible outcomes (scenarios) in ways that neither of the other two approaches can achieve. For example, Fig. 11 shows ATL airport capacity distributions for runway configuration 8L 9R 10 | 8R 9L produced by the IACM using time-lagged ensemble forecast on 30 March 2011 when ATL experienced low ceilings and significantly reduced visibility. For the first look-ahead hour, all valid members of the ensemble predicted C&V within the range of MVMCs and thus yielded an AAR of 140 (as compared to 166 for VMCs) and an ADR of 93. From look-ahead hours 2 through 6, however, some members of the ensemble predicted C&V values for MVMCs, whereas other members are

a) b) Fig. 10 Mean and variance (standard deviation) of errors in predicted a) arrival and b) departure airport capacities by type of weather input.

26

KICINGER ET AL.

Fig. 11 Probabilistic arrival (right) and departure (left) capacity estimates for ATL on 30 March 2011, 10:00Z, for runway configuration 8L 9R 10 | 8R 9L.

translated into IMCs. Moreover, Fig. 11 shows that the likelihood of the AAR reduced to 109 (IMCs) increased with look-ahead time, suggesting that weather conditions were expected to worsen, and traffic management regulation may have been necessary to address demand–capacity imbalances caused by significantly reduced capacity. Finally, for look-ahead hours 5 and 6, there was a small probability that the impact of surface winds (tailwinds and/or crosswinds) would reduce the available capacity to zero, necessitating the change to a different runway configuration. In summary, the results of the statistical analysis show that no type of weather forecast input produces overall the best results; on the contrary, in some cases, ensemble forecasts were superior, and in other cases, the deterministic forecast or deterministic forecast with error model produced the best results. This finding suggests that in some cases deterministic forecast information may be sufficient to generate predictions of airport capacity. On the other hand, ensemble forecasts may become important when there is a potential for significantly different outcomes of predicted operational impacts (e.g., the timing of wind shift, MVMCs vs IMCs, etc.). An analysis of a much larger sample of days and hours is planned as part of future work that will further investigate this issue.

Conclusions This paper discussed the integrated airport capacity model (IACM), a stochastic analytical model for generating probabilistic maximum airport capacity predictions, and described results of calibration and validation studies for Hartsfield–Jackson Atlanta International Airport (ATL). The model explicitly integrates weather information and its uncertainty to estimate airport capacity. The studies show that IACM maximum capacity predictions agree well with traffic counts at this airport. This is especially true when the demand approaches the airport’s capacity, as was the case in 2007 and 2008 when ATL exceeded 250 operations (arrivals and departures) per hour. Extensive validation of the IACM prediction against actual throughput at ATL in 2011 showed that the IACM can accurately predict arrival and departure rates for ATL under different weather conditions, if the traffic demand is appropriately scaled and proper runway configuration and usage are applied. During operational use, the IACM does not require a demand-level scaling, as it will be used to predict maximum capacity for an airport given the meteorological conditions and demand mix. If the predicted capacity is below the demand, TMIs may be needed to resolve the imbalance. The studies did not reveal statistically significant differences of forecast uncertainty representation on IACM performance. In particular, the accuracy of airport capacity predictions was comparable for all types weather forecasts studied, i.e., deterministic forecasts, deterministic forecasts with forecast error models, and ensemble forecasts. Using a true weather forecast ensemble,

however, may convey operationally relevant information on the possibility of different scenarios that the other two approaches cannot provide. Further analysis involving a much larger set of scenario days is recommended to investigate and confirm these findings. Future work will extend validation studies to include other factors impacting airport capacity, such as fleet mix, terminal weather conditions, operational standards and procedures, and more complex runway geometries as found at other airports. Future studies will also work with subject matter experts to investigate operational factors not yet included in the model that can potentially impact the accuracy of capacity predictions (e.g., wet or otherwise contaminated runways) and to refine existing models and algorithms to better reflect operational constraints. Finally, a prototype with a web-based interface could be developed to provide close to real-time airport capacity predictions for key airports in the National Airspace System. Such a tool could illicit valuable feedback for development efforts and aid in the integration of the IACM with decision support tools.

Appendix A: Surface Wind Forecast Error Model Comparison of predicted HRRR wind values against METAR observations** at ATL showed that the mean error in the u and v components is generally small because of the cancellation of positive and negative values. Detailed analyses, however, revealed that the forecast bias was a function of the component magnitude and correctable as Uc  αu
τU0  βu
τ

(A1)

V c  αv
τV 0  βv
τ

(A2)

where U0 and V 0 are the uncorrected model forecast values for the uand v-wind components and αi and βi are the corresponding biascorrection factors that are a function of the month. Bias corrections shown in Table A1 have been applied universally to all forecast lead times since lead time dependencies in the wind component biases were found to be small. In addition to the bias correction factor for winds, a model that accounts for random errors in the predicted winds was developed. The wind component error frequency distributions can be fitted using a Gaussian function of the form **Data from four months in 2011 (i.e., March, August, October, and December) were used in these analyses and selected in such a way as to span all seasons.

27

KICINGER ET AL.

Table A1 Coefficients of parameterized bias corrections for uand v-wind components for ATL

conditional probabilities that a forecast of a specific aviationimpacting event will be correct. In other words, the forecast uncertainty is modeled using a categorical forecast accuracy model. Several observed categories are considered for both ceiling and visibility, as listed in Table B1. For each forecast value, the probability of the observed (actual) category is found using logistic regression. The logistic regression function is given by

Wind u Wind v component component Month
τ αu
τ βu
τ αv
τ βv
τ March 0.3 1.5 −0.7 1.5 August −0.1 1.4 −0.8 1.1 October −0.2 1.4 −0.1 1.4 December −0.1 1.5 −0.3 1.5

P
x 

Table A2 Coefficients of parameterized error distributions for u- and v-wind components for ATL Component u v

A0 0.0828 0.0850

Coefficients A1 (m · s−1 ) A2 (m · s−1 ) −0.176 2.376 −0.102 2.258

2 ∕2

(A3)

z 
x − A1 ∕A2

(A4)

f
x  A0 e−z

where z is the magnitude of the error, A1 is the sample mean error, A2 is the standard deviation in the sample error distribution, and A0 is an empirical parameter. The value of each coefficient is listed in Table A2. The same error model is used for all forecast lead times and seasons, since variations in the distribution of errors was found to be small.

Appendix B: Ceiling and Visibility Forecast Error Models Following the approach in [11], uncertainty in the deterministic HRRR forecasts of C&V can be accounted for by computing

Table B1

Coefficient β0 β1 Coefficient β0 β1

Regression coefficients for C&V forecast error models Observed ceiling categories