Modeling and prediction of the magnetospheric dynamics during

JOURNAL OF GEOPHYSICAL RESEARCH, VOL. 111, A04209, doi:10.1029/2005JA011359, 2006

Modeling and prediction of the magnetospheric dynamics during intense geospace storms Jian Chen1 and A. Surjalal Sharma1 Received 10 August 2005; revised 15 November 2005; accepted 29 December 2005; published 14 April 2006.

[1] The nonlinear dynamical models of the coupled solar wind-magnetosphere system

derived from observational data have been used to yield efficient forecasts of the magnetospheric conditions. These data-derived models have the advantage of capturing the essential features inherent in the data and thus can be improved as larger and better databases become available. Current models are based largely on the data of periods when the magnetosphere is weakly driven. For example, the widely used Bargatze et al. (1985) data set corresponds to a declining phase of the solar cycle and contains only a few weak storms. A correlated database of solar wind and magnetospheric time series data for the last solar cycle near its peak (year 2001) is compiled and used to model the magnetospheric dynamics under strong driving. The solar wind variables are obtained from ACE spacecraft, while the magnetospheric response consists of geomagnetic indices as well as magnetic perturbations measured by ground magnetometers. The dynamical models of the magnetosphere during superstorms developed with this database are used to forecast the geospace storms of October–November 2003 and April 2002 and yields improved forecasts of the intense storms. A new technique is introduced for data-derived modeling based on the distances of the nearest neighbors from the current state. In this technique the contributions of the nearest neighbors are weighted by factors inversely proportional to the distances in the reconstructed phase space. This yields better predictions, especially during the strongly driven periods. The comparison of the models based on the year 2001 and Bargatze et al. (1985) databases show the similarities and differences in the magnetospheric states during solar maximum and minimum periods. Citation: Chen, J., and A. S. Sharma (2006), Modeling and prediction of the magnetospheric dynamics during intense geospace storms, J. Geophys. Res., 111, A04209, doi:10.1029/2005JA011359.

1. Introduction [2] The solar wind-magnetosphere coupling is enhanced when the interplanetary magnetic field (IMF) turns southward, leading to geospace storms and substorms. The magnetosphere is a highly dynamic system under these conditions and the magnetospheric substorms are the elementary disturbances with a typical timescale of an hour. When the IMF remains southward for an extended interval, the ring current grows under the influence of the solar wind variations, leading to geospace storms with typical timescales of days, and are characterized by strong decreases in the surface magnetic field. [3] The Earth’s magnetosphere is a nonautonomous dynamical system, driven by the solar wind. Studies of the magnetospheric dynamics using models derived from the correlated database of the solar wind-magnetosphere system have enhanced our understanding of the complex behavior of the magnetosphere. The advantage of this approach is the 1 Department of Astronomy, University of Maryland, College Park, Maryland, USA.

Copyright 2006 by the American Geophysical Union. 0148-0227/06/2005JA011359

ability to yield the dynamics, inherent in observational data, independent of modeling assumptions. There has been considerable progress in the modeling and forecasting of the solar wind-magnetosphere coupling as an input-output system by linear and nonlinear approaches. One of the earliest methods used to analyze the solar wind-magnetosphere coupling is the linear prediction filter, which assumes a linear relationship between the input time series I(t)(solar wind convective electric field VBs) and output time series O(t) (geomagnetic activity index AE or AL). [4] The linear prediction filter technique was used to obtain the response time of the magnetosphere from the AL-VBs database [Bargatze et al., 1985, hereinafter referred to as the BBMH data set]. This database spans the period from November 1973 to December 1974 and has 2.5 min resolution. The response functions from this analysis have been used to interpret how the magnetospheric response to the solar wind driver varies with changes in the activity level, indicating nonlinearity. These response functions exhibited two timescales, corresponding to the directly driven and loading-unloading processes. [5] The modeling of magnetospheric substorms as a low dimensional system using the time series data of the electrojet indices, AL or AE, to reconstruct its dynamics

A04209

1 of 14

A04209

CHEN AND SHARMA: MODELING MAGNETOSPHERE DYNAMICS

has shown its low dimensionality and the nonlinear nature of the magnetosphere [Vassiliadis et al., 1990; Shan et al., 1991a, 1991b; Sharma et al., 1993]. The reconstructed phase space show clear evidence that the dynamical system follows a pattern in the reconstructed phase space [Sharma, 1995, 1997]. This implies that the magnetosphere is predictable and has stimulated the study of forecasting substorms [Vassiliadis et al., 1995] and storms [Valdivia et al., 1996]. Vassiliadis et al. [1995] used the local-linear technique on the BBMH data set, with the solar wind convective electric field VBS as the input and the AL index as the output, and obtained good predictions. These predictions gave strong evidence that nonlinear models can be used to develop accurate and reliable forecasting tools for space weather. Recent studies using time series data have shown that the coherence on the global magnetospheric scale can be obtained by averaging over the dynamical scales. A model for the global features can be obtained by a mean field technique of averaging outputs corresponding to similar states of the system in the reconstructed phase space [Ukhorskiy et al., 2002, 2004]. With such a mean-field model, accurate iterative long-term predictions can be obtained, as the model parameters need not be changed during the prediction. [6] During October – November 2003, nearly 2 years after the last solar maximum, two extremely big geospace storms occurred. Two strong coronal mass ejections (CME) occurred on 28– 29 October 2003 with velocities of more than 2000 km/s near the Sun and reached the Earth after a travel time of only 19 hours. The arrival of the first coronal mass ejection from the center of the solar disc led to an immediate start of a G5 storm, an extreme geomagnetic storm on the NOAA space weather scale that runs G1 to G5. During 29– 31 October 2003, the Advanced Composition Explorer (ACE), located at 220 RE upstream from Earth, detected extreme conditions in the solar wind. The solar wind speed reached as high as
2000 km/s and the maximum of the southward component of the interplanetary magnetic field (IMF) was -58.31 nT, leading to the geomagnetic field index AL values as low as 2778 nT. On 19 April 2002 and 20 November 2003, another two G5 extreme geomagnetic storms were driven by the solar wind with velocities of 696 km/s and 766 km/s, and the southward IMF of 32.03 nT and 53.02 nT, measured by ACE, and these led to the AL index values of 1851 nT and 2499 nT, respectively. These three intense geospace storms provide interesting opportunities for the study of nonlinear phase space reconstruction under extreme conditions. In order to model and predict such intense storms, a correlated database of the solar wind and magnetospheric variables of the year 2001, which is close to the peak period of 11-year solar cycle, is compiled.

2. Correlated Database of Solar WindMagnetosphere Coupling Under Strong Driving 2.1. Year 2001 Storm Database for Solar Maximum Period [7] During the period of maximum solar activity, the magnetosphere is strongly driven and the year 2001 near the last solar maximum is chosen for compiling a database for such an epoch. This database contains solar wind flow

A04209

speed V, the north-south component BZ of the IMF, and the AL index for the 11 months of 2001 (January to November). The solar wind data were obtained from ACE at 64 s and 4 min time resolution in the geocentric solar magnetospheric (GSM) coordinates, and the AL index data were obtained from the World Data Center (WDC), Kyoto. The ACE data, obtained from Coordinated Data Analysis Web (CDAWEB), and the AL data were averaged to 5 min resolution. The ACE satellite is located at the L1 point, about 218 – 248 RE from the Earth and with a solar wind bulk speed of 420 km/s along the Sun-Earth direction, it gives a 60 min lead time. So in compiling the correlated database of the geomagnetic field measurements (AL), and the solar wind (VBZ), a time lag of 1-hour is used. The location of ACE at a fixed position largely removes the problem of propagating the solar wind data from the spacecraft to the magnetopause. [8] The solar wind data for 2001 were compiled for a set of data intervals, each defined as any continuous data longer than 12 hours with no more than half-hour data gap. The data set contains 81 intervals with periods 12 hours to 3 days long. The data gaps in these intervals were interpolated linearly to satisfy the data continuity requirement. Two criteria, similar to those used by Bargatze et al. [1985], were used to select the data segments: (1) A data interval bounded at both ends by around 1 hour segment of weak, nearly zero, solar wind input and small values of the AL index. (2) If more than one data segment with a day-long time data interval satisfy the first criterion, the interval is separated into different intervals. [9] During January– November 2001, there were 81 such data intervals containing 33,931 data points at 5-min resolution, satisfying the above conditions. The correlated solar wind induced electric field VBZ and the auroral electrojet index AL for 81 intense storm intervals during year 2001 are shown on Figure 1. During this period of strong solar activity, intense substorms and storms were triggered with higher frequency. If we define a strong geomagnetic storm as having Dst less than 100 nT, we find that there are 12 such storms in 2001 compared with four such storms in 1995 and one in 1996. Thus the 2001 database is appropriate for studying the properties of geomagnetic activity during a solar maximum. The selected 81 events are separated into three activity levels by the average values of VBS: medium (hVBSi  1500 nTkm/s), high (1500 nTkm/s  hVBSi  2500 nTkm/s), and super (hVBSi  2500 nTkm/s). To model a specific event, we choose the corresponding activity level to which it belongs and use it as a reference database. [10] A comparison of the characteristic parameters of the year 2001 and the BBMH is given in Table 1 and Figure 2. From the definitions of the activity levels and Table 1, we find that most data intervals of the BBMH database are in the medium activity level, and a small number of data intervals are in high activity level. However, for the 2001 data set, only 2/5 of the data intervals are in the medium level group, and 3/5 of the data intervals are in the high and super activity levels. The counts of VBS and AL within the proper bin sizes for the BBMH and the 2001 datasets are shown on Figure 2. It clearly shows that the 2001 database has much more events with VBS bigger than 1000 nTkm/s and AL smaller than 200 nT. And also there are many cases with VBS bigger than 10,000 nTkm/s and AL smaller

2 of 14

A04209

CHEN AND SHARMA: MODELING MAGNETOSPHERE DYNAMICS

A04209

Figure 1. The correlated solar wind induced electric field (VBZ) and the auroral electrojet index AL for 81 intense storm intervals during year 2001. The geomagnetic activity in these intervals during the peak of the last solar cycle is very high and corresponds to strong driving by the solar wind. (a) VBZ (b) AL. than 1000 nT (belonging to the super group) in the 2001 database but none in the BBMH database. It may be noted that BS = BZ for southward IMF so that VBS is positive. We calculated the mean value of solar wind input for each data intervals. Among these mean value of each event, the maximum one of VBS for the year 2001 and the BBMH databases are 13116 nTkm/s and 2989 nTkm/s, respec-

tively, showing the strong driving of the magnetosphere in 2001. 2.2. Intense Geospace Storms During October – November 2003 and April 2002 [11] During 2002 –2003 there were three intense storms, occurring in April 2002, October 2003, and November

Figure 2. Statistics of the events in the BBMH and Year 2001 databases. (a) VBS (bin size = 100) (b) AL (bin size = 20). The events represent averages over the indicated bin sizes. The differences in the magnetospheric response in the two databases are not as pronounced as in the solar wind variable. 3 of 14

A04209

CHEN AND SHARMA: MODELING MAGNETOSPHERE DYNAMICS

Table 1. Comparison Between BBMH [Bargatze et al., 1985] and Year 2001 Databases

BBMH Database Year 2001 Database

Level 1 (Medium)

Level 2 (High)

Level 3 (Super)

26 33

7 32

1 16

2003. The solar wind data from ACE through CDAWEB and the corresponding geomagnetic field index AL were compiled for these storms. However, for the October 2003 event there were some problems with the solar wind velocity data during 28 – 30 October 2003, the critical period for the interaction of the violent solar wind with the geomagnetic field. For this period we adopt the modified solar wind velocity data [Skoug et al., 2004] from Solar Wind Electron Proton Alpha Monitor (SWEPAM) [McComas et al., 1998], whose measurement capability was pushed to the limit during the event. The SWEPAM ion instrument collects data in two modes, normal (tracking) mode and search mode, each of which required 64 s for a full measurement. During the period beginning at 1241 UT on 28 October and ending at 0051 UT on 31 October, the high background noise levels due to the penetrating radiation from the intense solar energetic particle caused the solar wind tracking algorithm to fail in recording the solar wind during the high-speed events. However, the search mode data with 33-min resolution are available during this highspeed solar wind period and are used in compiling the database. [12] For the November 2003 and April 2002 events, the solar wind velocity and IMF data are from ACE obtained through CDAWEB. In the correlated databases, 1-hour time delays were applied on all these events, as in the 2001 database.

3. Nonlinear Dynamical Modeling Using Correlated Data 3.1. Input-Output Modeling of the Magnetosphere [13] The driven nature of the magnetosphere is inherent in many input-output models, such as the linear prediction filter technique, in which the input to a physical system and its response or output are used to describe the coupling characteristics in term of linear filters. Neural networks [Hernandez et al., 1993] and local linear filters [Vassiliadis et al., 1995] have been used to model the dynamical characteristics of the solar wind (input) driving the geomagnetic activity (output). In these approaches, the magnetospheric coupling characteristics are obtained directly from observational data and have the advantage of being unencumbered by presumed processes. [14] The magnetosphere has been shown to exhibit the features of a nonlinear dynamical system, and its global features have been modeled by a few variables [e.g., Baker et al., 1990]. This remarkable property arises from the inherent property of phase space contraction in dissipative nonlinear systems. A dynamical input-output model can be constructed based on local-linear filters, which represent the relationship between the input I(t) and the output O(t) of the system. It is assumed that the time series data contain the

A04209

information necessary to determine the system evolution. Thus if a phase space is large enough to unfold the dynamical attractor, the phase space reconstructed from the time series is appropriate for modeling the evolution of the system. Consequently, the future states can be derived from the known evolution of the similar states. [15] The time delay embedding technique is an appropriate method for the reconstruction of the phase space and for obtaining its characteristic properties [Gershenfeld and Weigend, 1993; Strogatz, 2000]. In this technique, a m component phase vector Xi is constructed from this time series x(t) as: Xi ¼ fx1 ðti Þ; x2 ðti Þ;    ; xm ðti Þg;

ð1Þ

where xk(ti) = x(ti  (k  1)T) and T is a time delay. If the embedding procedure is properly performed, the dynamical attractor underlying the observed time series will be completely unfolded, and the constructed states have one to one correspondence with the states in the original phase space. Appropriate values of the time delay T and the embedding dimension m can be obtained by using techniques such as the average mutual information and the correlation integral [Abarbanel et al., 1993]. [ 16 ] In an input-output model of the solar windmagnetosphere system during substorms, the solar wind convective electric field VBS is commonly used as the input and the geomagnetic activity index AL or AE as the output. Thus the input-output vector in the 2m dimensional embedding space can be constructed as Xi ¼ ðI1 ðti Þ; I2 ðti Þ;    ; IM I ðti Þ; O1 ðti Þ;

O2 ðti Þ;    ; OMo ðti ÞÞ; ð2Þ

where MI = MO = m. The 2m-dimensional state vector Xi at t = t1, t2, . . . tN, can now be used to construct a trajectory matrix for the dynamics of the system as: 2

X ¼

I1 ðt1 Þ 6 6 6 I1 ðt2 Þ 6 N 1=2 6 6 . 6 .. 6 4

I2 ðt1 Þ



Im ðt1 Þ

O1 ðt1 Þ

O2 ðt1 Þ



I2 ðt2 Þ



Im ðt2 Þ

O1 ðt2 Þ

O2 ðt2 Þ



.. .

..

.. .

.. .

.. .

..

.

I1 ðtN Þ I2 ðtN Þ   

Im ðtN Þ

.

O1 ðtN Þ O2 ðtN Þ   

Om ðt1 Þ

3

7 7 Om ðt2 Þ 7 7 7 .. 7 . 7 7 5 Om ðtN Þ

N 2m

ð3Þ

where N is the number of vectors. This N 2m matrix contains all the dynamical features of the system contained in the data and yields its evolution in the reconstructed phase space. [17] An estimate of the embedding dimension is given by the number eigenvalues above the noise level, obtained by singular spectrum analysis [Broomhead and King, 1986]. However, it turns out that real systems do not always yield a clear convergence to a value of this dimension. Sitnov et al. [2000] have shown that the power spectral shape of VBS-AL time series is retained over a wide range of scales with no clear sign of a noise floor. Vassiliadis et al. [1995] and Ukhorskiy et al. [2002] considered m as a free parameter and in the latter the proper embedding space dimension was chosen as corresponding to the minimum error between the

4 of 14

A04209

CHEN AND SHARMA: MODELING MAGNETOSPHERE DYNAMICS

A04209

predicted and real time series. The prediction accuracy is quantified by normalized mean square error (NMSE):

nonlinear filters for short-term and long-term predictions of auroral indices [Ukhorskiy et al., 2002]. Thus

vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u N 1u 1 X 2 ðOi  O*i Þ ; h¼ t so N i¼1

  Onþ1 ¼ Onþ 1 NN þ AdIn þ BdOn

ð4Þ

ð6Þ

1 XNN ~k ~k  I ; On . Ukhorskiy et al. k¼1 n NN [2002] pointed out that for a long-term prediction of AL time series, the terms other than the center of mass consist of the higher-order terms of the filter for locallinear ARMA filter. These terms cannot be modeled in a consistent manner and should be ignored in developing a global model. The prediction procedure then reduces to a search of the average response of the system. In this forecasting method the choice of the nearest neighbors (NN) and the embedding dimension (2m) are critical. A large number of neighbors are likely to lead to a smoothing of the variations while a small number may lead to wide variations among the chosen states. The choice of the embedding dimension of the input (output) m is based on the unfolding of the dynamics in the reconstructed phase space. For a state Xn, if the dimension 2m and its NN nearest neighbor represent the system properly, the average over NN nearest neighbor defines the smooth manifold of dimension 2m on which its dynamics can be predicted. The prediction using the mean field approach [Ukhorskiy et al., 2004] is

in which hOn+1iNN = where Oi and O*i are the observed and predicted data, respectively, and so is the standard deviation of Oi. 3.2. Local-Linear and Mean Field Filters [18] The local-linear filters, as the name implies, are essentially linear prediction filters and they take into account of the nonlinearity, as they are linear only locally in the reconstructed phase space. If a proper embedding dimension is achieved, the attractor of the system is completely unfolded and the reconstructed phase space has oneto-one correspondence with the states in the original phase space, thus making the prediction of the dynamical system possible. The main idea of this method is the use of the trajectories in the neighborhood of the state at time t to predict its location at the next time step. Knowing how the neighboring trajectories evolve, the location of the current state x(t) at next time step t + T can be predicted. The procedure is locally linear but is essentially nonlinear as the features of the neighboring trajectories are taken into account by considering a small neighborhood. For a given time series, a proper embedding dimension is obtained when two states xk and xn, which are close together in the embedded phase space, yield the next states of Ok+1 and On+1 which are also close together [Ukhorskiy et al., 2002]. [19] The predicted output On+1 can be described as a nonlinear function of the input In and current output On as Onþ1 ¼ F ðIn ; On Þ:

ð5Þ

In this model, the geomagnetic activity is represented by a state vector and the function F governing the evolution of the magnetospheric state On depends on both the input and the previous states. A Taylor expansion of F up to the linear terms gives: Onþ1

M 0 1 I 1 X

MX  F ð0Þ Inc ; Ocn þ Ai dIiþ1;n þ Bj dOjþ1;n i¼0



¼ F ð0Þ Inc ; Ocn þ AdIn þ BdOn :

j¼0

The zero-order term F(0) is a function of (Icn, Ocn), the center of expansion, while dOn and dIn are small deviations from the center. All the three parameters F(0), dOn, dIn can be obtained from the known data, which is referred to as the training set. Given the current state, the states similar to it in the training set are selected as the first step. The similarity of the current state with any other state in the training set is quantified by the Euclidean distance between them in the embedding space. The states within a specified distance of the current state are referred to as the nearest neighbors (NN). [20] The average value of the state vectors of the nearest neighbors is usually defined as the center of the expansion, referred as the center of the mass and is used in defining the

Onþ1 ¼

NN 1 X Xk : NN k¼1

ð7Þ

The local linear ARMA and the local linear mean field filters have been used with the correlated BBMH database of solar wind and geomagnetic activity time series [Ukhorskiy et al., 2002, 2004]. Both these techniques yield good results with the small or medium values of the dimension, indicating that the global aspects of the magnetospheric dynamics can be modeled as a low-dimensional system. 3.3. Weighted Mean Field Filter [21] In the mean field model discussed above, all the states in the specified neighborhood, the NN nearest neighbors, were used to obtain the center of mass by a simple averaging procedure. It is, however, clear that the prediction will be improved if the states close to the current state contribute more than those farther away. On the basis of this recognition, a new filter based on the mean field filter is proposed to improve the accuracy and efficiency of predictions. This weighted filter takes into account the distance of the nearest neighbors and is not just a simple average over the NN nearest neighbors. Since some of the nearest neighbors are far away from each other and also farther away from the center of mass, a set of weight factors g which depend inversely on the distances of each nearest neighbor from the mass center is introduced as

5 of 14

1 gk ¼ 2 dk

,

NN X 1 ; 2 d i¼1 i

ð8Þ

CHEN AND SHARMA: MODELING MAGNETOSPHERE DYNAMICS

A04209

where di is the Euclidean distance of the ith nearest neighbor from the center of mass. The predicted output that includes this weighting of the neighbors is Onþ1 ¼

NN 1 X Xk  gk : NN k¼1

ð9Þ

In the limit all the nearest neighbors have the same distance from the center of mass, the weighted mean-field filter will yield the same prediction as the mean field filter. However, if the NN nearest neighbors have a wide range of the distances, the nearest neighbors closer to the center of mass will dominate the prediction. The inclusion of nearest neighbors farther away should not affect the prediction significantly as these will have smaller weights, thus making the predictions less sensitive on NN.

4. Modeling and Prediction During Superstorms 4.1. October– November 2003 Superstorms [22] The weighted mean field filter is used to model the solar wind-magnetosphere coupling during the superstorms of October–November 2003. The earlier works by Vassiliadis et al. [1995] and Ukhorskiy et al. [2002] used VBS as the model input and applied the nonlinear moving-average filter and mean field average filter, respectively. It was shown that using appropriate parameters the filter responses are stable and yield good long-term predictions. The best filter parameters were chosen to be those yielding the minima of the prediction errors. In these studies, the solar wind input was taken to be VBS, the product of the solar wind velocity and southward component of the IMF; consequently the input becomes zero when the IMF is northward. [23] In the studies discussed in this section, we take the solar wind input as VBZ, which includes both the northward and southward components of IMF. This enables the model to include the effects of northward IMF, although these may be weaker than those of the southward IMF. [24] In order to obtain the optimal nonlinear weighted mean field filter for superstorms, the following steps are adopted. First, the activity level of the solar wind driving is computed by averaging the southward component of VBZ. Then both the input (VBZ) and output (AL) of the time interval corresponding to the same activity level of the magnetospheric activity from the 2001 database are selected as the training set. For the two superstorms, the super level (hVBSi  2500 nTkm/s) of the 2001 database is selected. Second, using all of the selected data interval of input (VBZ) and its corresponding (AL) as a training set, the index AL is predicted for the superstorms using the weighted mean filter discussed above. The normalized mean square error (NMSE) is used to determine the optimal parameters for the prediction by comparing the predicted and actual AL. In this model, the time resolution (5 min) of the training set is chosen as the time delay T, and the other three free parameters are used to minimize the NMSE. The first two parameters are the embedding dimensions MI and MO, and as in the previous studies, we take m = MO = MI, which determines the vector length in the phase space to be 2m. The third parameter is the number of nearest neighbors NN. A wide range of values of these parameters are used in the

A04209

model to obtain the optimal predictions for the superstorms of 19– 23 November and 26 October to 1 November of 2003 and these are shown in Figure 3. The solar wind convective electric filed (VBZ) for these events are shown on Figure 3a and 3c. There is a sudden enhancement of the solar wind convective electric field VBS in the early part of these events and this drives the geospace storms. The predicted and real AL are plotted in Figures 3b and 3d. The solid lines represent the real AL and the dotted lines represent the predicted AL. Iterative predictions of the November 2003 storm were carried out for 7500 min (125 hours) with a minimum NMSE of 0.792 and the maximum correlation coefficient of 0.758. Also for the predictions of the October 2003 storm, made for 10,000 min (167 hours), yielded a minimum NMSE of 0.911 and a maximum correlation coefficient of 0.714. In these figures the model output closely reproduces the large-scale variations of AL and captures some of the most abrupt changes. Also preceding the AL minima, there are sharp jumps with positive VBZ, corresponding to the abrupt enhancements of the northward IMF. However, the southward IMF is the main driver of the geomagnetic storms, and it is not clear how well the model captures the effects of positive IMF enhancements. [25] In the earlier studies using the BBMH data set [e.g., Vassiliadis et al., 1995; Ukhorskiy et al., 2002, 2004], a major part of the data set was used as the training set and the predictions were made for the remainder of the data set. Consequently, there were many similar states in the phase space. However for the two superstorms of 2003, it is hard to find so many similar big substorms in the available databases, such as that of year 2001. The nearest neighbor searches in these cases yields only a few states close to the superstorms. If we use a large number of nearest neighbors and a simple arithmetic averaging, the output of the model is smoothed over these and cannot capture the peak of the substorms. In such cases the weight factor g plays an important role and the averaging procedure yields improved predictions. [26] The dependence of the prediction errors on the filter parameters is shown in Figure 4, in which Figures 4a – 4d shows the surface plots of NMSE as functions of NN and m for the November and October 2003 storms. For the November 2003 storm, the NMSE surface plots have a significant drop from 1.2 to 0.75 when m is increased from 1 to 3. With a further increase of m, NMSE returns back to 0.9 and has similar values until m reaches 11. With m in the range 12– 14, the NMSE surface reaches another minimum, independent of NN. With further increase of m, NMSE starts to grow again. The overall structure of the surface plot of NMSE is a ‘‘canyon’’ with a broader range of m values for all values of NN than in the earlier models [Vassiliadis et al., 1995; Ukhorskiy et al., 2002]. The prediction results have similar correlations and prediction error surfaces when m varies in the range 12– 14, independent of the number of nearest neighbors NN. This also indicates the dimension of the magnetosphere phase space to be 12– 14, which indicates the presence of the multiscale component of the solar wind-magnetospheric activity [Ukhorskiy et al., 2002]. [27] However, for the October 2003 storm, the NMSE surface is more complicated. The valley in m around 2 – 4, shown in Figures 4c and 4d, is much deeper than that

6 of 14

A04209

CHEN AND SHARMA: MODELING MAGNETOSPHERE DYNAMICS

A04209

Figure 3. The weighted mean-field predictions for storms: (a – b) 19– 26 November 2003, (c –d) 26 October to 3 November 2003, and (e – f) 15– 24 April 2002. The left panel is VBZ, and the right panel is the real AL (solid line) and predicted AL (dotted line). around 10– 12 and yields better predictions. During 29– 30 October 2003, SWEPAM measured solar wind speed in excess of 1850 km/s, almost the highest speed directly measured in the solar wind. However, despite the unusual

high speed many of the other solar wind parameters were not particularly unusual in comparison with other large events [Skoug et al., 2004]. Comparing with the solar wind speed at 750 km/s on 20 November 2003, the October 2003

7 of 14

A04209

CHEN AND SHARMA: MODELING MAGNETOSPHERE DYNAMICS

A04209

Figure 4. The normalized mean square error (NMSE) of the predictions plotted against the number of nearest neighbors NN and the embedding dimension m. (a– b) November 2003 storm; (c – d) October 2003 storm; (e – f) April 2002 storm. Plots on right are the two-dimensional projection from the left panel. Each line represents a particular value of NN.

high-speed events in which the solar wind speed was 2 – 3 times faster than those typical for large solar wind transient events did not produce an unusually larger geomagnetic storm. This may be due to the short-lived BS, although large in value, and so the storm of October 2003 is driven partially by the dynamic pressure and large positive BZ. The phase space in this case is expected to be signif-

icantly different from those of the storms caused by southward IMF. [28] The role of the weight factor g and the dominance by the first few nearest neighbors can be examined by using the values computed from the data set. For the November 2003 storm, a total of eight nearest neighbors are considered in the model. Out of these the weight factors for the first three

8 of 14

A04209

CHEN AND SHARMA: MODELING MAGNETOSPHERE DYNAMICS

A04209

Figure 5. The normalized mean square error (NMSE) of the predictions of the November 2003 using mean-field filter model. (a) NMSE surface plot; (b) two-dimensional projection of Figure 5a. accounts for 48% on average, with the maximum reaching as high as 85%. Thus the first three nearest neighbors dominate the predictions, which enhance the ability of such models to make better predictions under extreme conditions when the number of similar events in the database is small. To further understand the effects of the g factor, we use the mean field average filter without the weight factors to predict the substorms during November 2003 using the year 2001 database. The NMSE surface plots of predicted AL are shown in Figure 5, following the same format as in Figure 4. The best prediction results are almost the same as those of the weighted mean field filter. The NMSE surface plots for the two cases, with and without the weight factors, are similar for NN values in the range 1 – 11, and when NN is larger than 12, the bottom of the NMSE profile become wider and higher in Figure 5. This means that the predictions become worse with the increase of NN when it is bigger than 12. Thus the weight factor g is effective in controlling the quality of the predictions by reducing the contributions of the nearest neighbors which are farther away when their distances are distributed over a wide range. Also since the prediction efficiency drops when NN is larger than 12, better predictions are made with NN less than 12. Among the predictions with NN less than 12, we can define a narrow canyon when m is around 12– 14 so that the whole phase space is unfolded well and yields good predictions. The resulting predictions are similar to those obtained using the weighted mean filter prediction. 4.2. April 2002 Superstorm [29] The intense storm of 17 – 20 April 2002 has a minimum value of Dst of 154.2 nT, occurring around

0900 UT on 20 April 2002, and is a good case for modeling using the 2001 database for intense geomagnetic activity. During this storm the AL index reached a minimum of ALmin at 1851 nT and VBZmax reached a peak value of 16652 nTkm/s. A comparison of key parameters of the storms is given in Table 2. Compared with the AL index minima of 2778 nT and 2499 nT, and VBZ maxima of 69330 nTkm/s and 32640 nTkm/s, respectively, for the October and November 2003 storms, it is clear that the storm of April 2002 is the least active of the three storms. Also from the two databases, it is clear that the biggest events during the year 1973 [Bargatze et al., 1985] and the year 2001 are widely different. The biggest storm in year 2001 has the maximum and mean of solar wind input value of 32,015 nTkm/s and 13,116 nTkm/s, respectively, and the corresponding AL index values are 2701 nT and 450 nT. On the other hand, the biggest storms during 1973 has only 14,291 nTkm/s as the maximum of the solar wind input and 2989 nTkm/s as its mean value, which are about 30– 50% of the corresponding values for the year 2001. Also the corresponding AL index values of 1867 nT and 413 nT are much smaller than those in the year 2001 database. From Table 2, we also note that the April 2002 storm has intensity similar to the biggest event of the BBMH database, and October and November 2003 storms have intensities similar with the biggest one in the year 2001 database. [30] The weighted mean field filter technique was applied to the April 2002 storm period using the 2001 database as the training set. The prediction results and NMSE surface plots are shown in Figures 3e and 3f and Figures 4e and 4f.

Table 2. Intensities of the October 2003, November 2003, and April 2002 Storms

Maximum VBZ, nTkm/s Minimum AL, nT Mean of VBZ, nTkm/s Mean of AL, nT

October 2003

November 2003

April 2002

Biggest Event in BBMH Database

69,330 2778 4619 441

32,640 2499 3560 315

16,652 1851 2511 316

14,291 1867 2989 413

9 of 14

Biggest Event in Year 2001 Database 32,015 2701 13,116 450

A04209

CHEN AND SHARMA: MODELING MAGNETOSPHERE DYNAMICS

A04209

Figure 6. The weighted mean field predictions on November 2003 storm using the BBMH, Year 2001 and combined databases. The solid line is real AL data, dotted line is predicted results. (a) BBMH database, (b) Year 2001 database, (c) combined database, (d) NMSE for 250-min segments, solid line represent BBMH database, dotted line represent Year 2001 database, dashed line represent combined database. The iterative predictions of the April 2002 storm were carried out for 10 days; they yield a minimum NMSE of 0.748 and a maximum correlation of 0.831. These figures show that the model also reproduces the large-scale variations of AL very well. Also the NMSE surface plots (Figure 4) show a single valley around m
2 – 4 independent of the value of NN, in contrast to the two mimina in the cases of October and November 2003 storms. 4.3. Comparison of Predictions Using Bargatze et al. [1985] and Year 2001 Databases [31] In order to compare the predictions using different databases as the training set, the storms of November 2003 and April 2002 are predicted using the BBMH database. To highlight the differences clearly, the periods of quiet and low activity before and after the main phase of the storms are neglected. The results of the storm of November 2003 are shown in Figure 6a. It is clear that the peaks of AL cannot be predicted, mainly due to the absence of similar strong substorms in the BBMH database. The overall predictions have an NMSE of 0.847 and a correlation coefficient of 0.772. The predictions of for the same period using the year 2001 database and the combined database of year 2001 and BBMH are shown on Figures 6b and 6c, respectively. A comparison of these predictions, Figures 6a – 6c, shows the substantial improvement with the inclusion of the year 2001 database, either as the complete training set or

as a part of a bigger training set. This is clearly due to the presence of many events in the year 2001 database similar to those in the November 2003 storm. It may be noted that the correlations among the predicted and actual AL in the three cases are not substantially different in spite of the very different qualities of the predictions, e.g., between Figure 6a and Figure 6b or Figure 6c. This brings into focus the inability of the linear correlation functions to provide an adequate measure of predictive capability. In order to compare the predictability for different segments of the database, the November 2003 event was separated into smaller segments of 250 min or 50 data points each. The comparisons of the NMSE for the different segments are shown in Figure 6d. It is clear that the NMSE for the data segments with large values of AL in the 2001 data set are much smaller than those of the similar segments in the BBMH data set. Also the NMSE of the segments of the year 2001 database are very similar to those of the BBMH and 2001 data sets combined. Further, except for the segments containing the large values of AL, the NMSE for the other segments have similar values for both the data sets. [32] The BBMH database is expected to yield better predictions for the storm of April 2002, a weaker storm compared to the November 2003 storm. The predictions and their NMSE are shown in Figures 7a – 7c. The predictions are found to be almost the same when the three databases, namely, BBMH, year 2001, and the two combined, are used

10 of 14

CHEN AND SHARMA: MODELING MAGNETOSPHERE DYNAMICS

A04209

A04209

Figure 7. The weighted mean field predictions on April 2002 storm using the BBMH, Year 2001 and combined databases. The solid line is real AL data, dotted line is predicted results. (a) BBMH database, (b) Year 2001 database, (c) combined database, (d) NMSE for 250-min segments, solid line represent BBMH database, dotted line represent Year 2001 database, dashed line represent combined database. as the training sets. Also the NMSE values for 250 min intervals are shown in Figure 7d, and the NMSE have similar values in most of the segments. [33] The NMSE values for the predictions of all three storms, using the BBMH, Year 2001 and combined databases as the training sets, are given in Table 3. This table compares not only the NMSE for the predictions for the whole storm period but also those of the main phase of the storms. The errors in the predictions of the storm main phases are systematically higher than those for the whole storm period. This could be expected, at least partly, as the periods of intense activity are harder to predict than the less active periods. However it should be noted that the NMSE values depend on the number of points N, as given by equation (4), and it can yield smaller values for larger N. Thus the NMSE values yield a useful comparison when the data sizes are similar and caution is essential otherwise. For

the predictions for the entire intervals of October and November 2003 storms, NMSEs when the BBMH database alone is used are 1.003 and 0.847, respectively, much larger than the corresponding values of 0.911 and 0.792 for the year 2001 database. However, NMSE for the year 2001 database alone are 0.911 and 0.792, which are very close to those for the combined database, namely, 0.939 and 0.792, respectively. The same results are obtained on the NMSE values for the main phases of the October and November 2003 storm predictions. For the main phase of the October event, NMSE is 1.181 when the BBMH database is used, 1.064 for the 2001 database, and 1.101 for the two databases combined. For the November 2003 event, NMSE is 1.059 for the BBMH database, 0.979 for the 2001 database, and 0.955 for the combined database. [34] In the case of the April 2002 storm, all the NMSE values obtained using different databases are similar, indi-

Table 3. Normalized Mean Square Error (NMSE) of the Predictions of All Three Storms Using the BBMH, Year 2001, and Combined Databasesa BBHM Database October 2003 November 2003 April 2002

2001 Level 3 Database

BBHM+2001 Level 3 Database

Whole

Main Phase

Whole

Main Phase

Whole

Main Phase

1.003 0.847 0.727

1.181 1.059 0.924

0.911 0.792 0.748

1.064 0.979 0.952

0.939 0.792 0.747

1.101 0.955 0.943

a

Both the NMSE of the whole event and of the main phases are shown.

11 of 14

A04209

CHEN AND SHARMA: MODELING MAGNETOSPHERE DYNAMICS

Figure 8. Comparison of real AL and predicted AL under the BBMH and Year 2001 databases. (a– b) November 2003 storm; (c –d) October 2003 storm; (e– f) April 2002 storm. The right panels are the averaged values at bin size = 20 of the events in the left panel.

12 of 14

A04209

A04209

CHEN AND SHARMA: MODELING MAGNETOSPHERE DYNAMICS

A04209

combination of these two databases under different solar activities provides a comprehensive database for improved modeling and prediction of magnetospheric activity under a wide range of solar wind conditions.

5. Conclusion

Figure 9. The averaged NMSEs over an ensemble of nearest neighbors (NN). Three curves correspond to averages of curves in Figures 4b, 4d, and 4f, respectively. All three cases have two minima: the first minimum at m = 2 – 4 is clear, while the second one at m = 12– 14 is clear for the November 2003 storm and less so for the other two storms. cating that the BBMH and the 2001 databases yield similar predictions. However the 2001 database is a better choice for the October – November 2003 storms, as the comparisons in Figures 6 and 7 indicate. The remaining parts quieter periods of the October – November 2003 and the whole of April 2002 storms can be predicted very well using both the BBMH and Year 2001 databases as the training sets. [35] To further compare the predictions using the BBMH and year 2001 databases, the observed and predicted AL are plotted in Figure 8. The left panels are the plots of the observed and predicted AL index for the three storms. In order to examine the comparison in more detail, the AL values are averaged over a bin size of 20 nT, and this yields the three right panels. These plots show the similarities and differences between the two databases. For the April 2002 storm, the predictions, (Figures 8e and 8f), show no significant difference for the two data sets. However, for the November 2003 (Figures 8a and 8b), and the October 2003 (Figures 8c and 8d) storms, the predictions using the 2001 database are much better than those using the BBMH database. On the other hand, for the low activity periods of these two storms, the predictions are similar. Also for all three storms, the low activity periods show most points along the 45-degree line, implying predictions with high accuracy. The predictions for whole of April 2002 storm and low activity periods of 2003 October – November storms show similarities. The differences in the predictions are mainly in the high activity parts of 2003 October – November storms. These similarities and differences indicate that the magnetospheric responses to similar solar wind input are similar. The differences are likely to be due to the lack of the big events in the BBMH database, rather than a fundamental difference in the magnetospheric response. [36] The analysis of the storms with different intensities and using different databases indicates that the geomagnetic response during the solar minimum and solar maximum periods have similar behavior. The 2001 and BBMH databases can thus be considered to complement each other. The

[37] The modeling of magnetospheric response to strong driving by the solar wind is important not only for a better understanding of the solar wind-magnetosphere coupling and but also for developing our capability to forecast extreme conditions. During the last solar maximum there were many intense geospace storms and the existing models had limited success in forecasting these accurately. In order to develop better models and improve forecasting capability, a correlated database of the solar wind and the magnetospheric response is compiled for the year 2001 during the peak of the last solar cycle. In this database, the solar wind variable is the induced electric field and the magnetospheric response is the auroral electrojet index AL. This database is particularly well-suited for modeling using the phase space reconstruction techniques. [38] The mean field approach to the modeling of the global magnetospheric dynamics [Ukhorskiy et al., 2002, 2004] is used to develop nonlinear dynamical models of the magnetospheric response from the year 2001 database. These predictions are then compared with the models based on the Bargatze et al. [1985] database, corresponding to a solar minimum period (1973 – 1974). The predictions for the big storms of October and November 2003 and April 2002 yields improved forecasts, especially for the intense storms. [39] The mean field approach has the advantage of yielding iterative predictions without having to fix model parameters, in particular the number of nearest neighbors NN and the dimension of the embedding space m [Ukhorskiy et al., 2002, 2004]. However, during intense storms the number of similar events is usually small and this limits the ability to predict big events. In order to improve the predictability in such situations, the mean field approach is modified by assigning weights to each of the nearest neighbors. These weights are inversely proportional to the square of the distance and leads to improvements in the predictions. The forecasting capability of the model is quantified in terms of a normalized mean square error (NMSE) computed from the predicted and actual AL values. [40] For a nonlinear reconstructed phase space, the evolutions of the system variables are closely related to the corresponding number of embedding dimension. The dynamical model of the solar wind-magnetosphere coupling specifies its evolution in the reconstructed input-output phase space of dimension 2m. The prediction error surfaces obtained from NMSE values, averaged over different nearest neighbors NN, and are shown on Figure 9 for all three storms. The prediction error surfaces have minima for m
2– 4, indicating proper embedding and this is in agreement with the results of Vassiliadis et al. [1995] based on the modeling using the BBMH database. These results show the low dimensional nature of the coupled solar wind-magnetosphere system on the global scale, corresponding to the length of the best filter m * T
10– 20 min, which is in agreement with the loading timescales. However, for the October and November 2003 events, NMSE surface yield

13 of 14

A04209

CHEN AND SHARMA: MODELING MAGNETOSPHERE DYNAMICS

another minima at m
12– 14 corresponding the timescale of 60 – 70 min lengths, which is close to the typical unloading timescale. These values of the embedding dimension are in agreement with the results of Ukhorskiy et al. [2002], based on the modeling using the BBMH database. The averaged NMSE for April 2002 storm (Figure 9), indicate a local minimum for m
12 –14, although it has lower value when m is over 20. Figure 9 shows that there are two minima in averaged NMSE, and these may be associated with directly driven and unloading timescales. Considering that the October and November 2003 storms are much more intensive storms than April 2002 storm, the differences in NMSE minima indicate a more complicated phase space for the superstorms compared to that of the moderate storms. Further analysis is needed to draw clear conclusions on the nature of the reconstructed phase spaces under different solar activity conditions. However, it is clear that the embedding dimension m
2 – 4, corresponding to timescales of 10– 20 min, yields minimum errors in all cases. The second minima in NMSE at m
12– 14 is more prominent in the case of more intense storms. For modeling purpose, embedding dimension of m
12 – 14 is an appropriate for a proper reconstruction of the phase space. [41] Acknowledgments. We thank R. M. Skoug for the solar wind SWEPAM data, G. Lu for the AL data, and both for many fruitful discussions. We also thank the referees for many constructive comments. The research is supported by NSF grants ATM-0318629 and DMS0417800. [42] Arthur Richmond thanks Alex Klimas and James Wanliss for their assistance in evaluating this paper.

References Abarbanel, H. D., R. Brown, J. J. Sidorowich, and L. S. Tsimring (1993), The analysis of observed chaotic data in physical systems, Rev. Mod. Phys., 65, 1331. Baker, D. N., A. J. Klimas, R. L. McPherron, and J. Buchner (1990), The evolution from weak to strong geomagnetic activity: An interpretation in terms of deterministic chaos, Geophys. Res. Lett., 17, 41. Bargatze, L. F., D. N. Baker, R. L. McPherron, and E. W. Hones (1985), Magnetospheric impulse response for many levels of geomagnetic activity, J. Geophys. Res., 90, 6387. Broomhead, D. S., and G. P. King (1986), Extracting qualitative dynamics from experimental data, Physica D, 20, 217. Gershenfeld, N. A., and A. S. Weigend (1993), The future of time series: Learning and understanding, in Time Series Prediction: Forecasting the

A04209

Future and Understanding the Past, Santa Fe Inst. Stud. in the Sci. of Complexity, vol. XV, edited by A. S. Weigend and N. A. Gershenfeld, pp. 1 – 70, Addison-Wesley, Reading, Mass. Hernandez, J. V., T. Tajima, and N. Horton (1993), Neural net forecasting for geomagnetic activity, Geophys. Res. Lett., 20, 2707. McComas, D. J., S. J. Bame, P. Baker, W. C. Feldman, J. L. Philips, P. Riley, and J. W. Griffee (1998), Solar Wind Electron Proton Alpha Monitor (SWEPAM) for the Advanced Composition Explorer, Space Sci. Rev., 86, 563. Shan, L. H., P. Hansen, C. K. Goertz, and R. A. Smith (1991a), Chaotic appearance of the AE index, Geophys. Res. Lett., 18, 147. Shan, L. H., C. K. Goertz, and R. A. Smith (1991b), On the embeddingdimension analysis of AE and AL time series, Geophys. Res. Lett., 18, 1647. Sharma, A. S., D. Vassiliadis, and K. Papadopoulos (1993), Reconstruction of low-dimensional magnetospheric dynamics by singular spectrum analysis, Geophys. Res. Lett., 20, 335. Sharma, A. S. (1995), Assessing the magnetosphere’s nonlinear behavior: Its dimensional is low, its predictability is high, Rev. Geophys., 33, 645. Sharma, A. S. (1997), Nonlinear dynamical studies of global magnetospheric dynamics, in Nonlinear Waves and Chaos in Space Plasmas, edited by T. Hada and H. Matsumoto, pp. 359 – 389, chap. 11, Terra Sci., Tokyo. Sitnov, M. I., A. S. Sharma, K. Papadopoulos, D. Vassiliadis, J. A. Valdivia, A. J. Klimas, and N. D. Baker (2000), Phase transition-like behavior of the magnetosphere during substorms, J. Geophys., Res., 105, 12,955. Skoug, R. M., J. T. Gosling, J. T. Steinberg, D. J. McComas, C. W. Smith, N. F. Ness, Q. Hu, and L. F. Burlaga (2004), Extremely high speed solar wind: 29 – 30 October 2003, J. Geophys. Res., 109, A09102, doi:10.1029/ 2004JA010494. Strogatz, S. H. (2000), Nonlinear Dynamics and Chaos With Applications to Physics, Biology, Chemistry, and Engineering, Westview, Cambridge, Mass. Ukhorskiy, A. Y., M. I. Sitnov, A. S. Sharma, and K. Papadopoulos (2002), Global and multiscale aspects of magnetospheric dynamics in local-linear filters, J. Geophys. Res., 107(A11), 1369, doi:10.1029/2001JA009160. Ukhorskiy, A. Y., M. I. Sitnov, A. S. Sharma, and K. Papadopoulos (2004), Global and multi-scale features of solar wind-magnetosphere coupling: From modeling to forecasting, Geophys. Res. Lett., 31, L08802, doi:10.1029/2003GL018932. Valdivia, J. A., A. S. Sharma, and K. Papadopoulos (1996), Prediction of magnetic storms by nonlinear models, Geophys. Res. Lett., 23, 2899. Vassiliadis, D. V., A. S. Sharma, T. E. Eastman, and K. Papadopoulos (1990), Low dimensional chaos in magnetospheric activity from AE time series, Geophys. Res. Lett., 17, 1841. Vassiliadis, D. V., A. J. Klimas, D. N. Baker, and D. A. Roberts (1995), A description of the solar wind-magnetosphere coupling based on nonlinear filters, J. Geophys. Res., 100, 3495.



J. Chen and A. S. Sharma, Department of Astronomy, University of Maryland, College Park, MD 20742, USA. ([email protected])

14 of 14