River flow forecasting using different artificial neural network ...

26

River flow forecasting using different artificial neural network algorithms and wavelet transform Turgay Partal

Abstract: In this study, the wavelet–neural network structure that combines wavelet transform and artificial neural networks has been employed to forecast the river flows of Turkey. Discrete wavelet transforms, which are useful to obtain to the periodic components of the measured data, have significantly positive effects on artificial neural network modeling performance. Generally, the feed-forward back-propagation method was studied with respect to artificial neural network applications to water resources data. In this study, the performance of generalized neural networks and radial basis neural networks were compared with feed-forward back-propagation methods. Six different models were studied for forecasting of monthly river flows. It was seen that the wavelet and feed-forward back-propagation model was superior to the other models in terms of selected performance criteria. Key words: wavelet transform, feed-forward back-propagation neural network, generalized regression neural network, radial basis neural network, monthly flow, forecasting. Re´sume´ : Une structure de re´seau de neurones a` ondelettes, qui combine une transforme´e par ondelettes et des re´seaux de neurones artificiels, a e´te´ utilise´e dans cette e´tude pour pre´dire des de´bits de rivie`res en Turquie. Des transforme´es discre`tes par ondelettes, utiles pour obtenir les composantes pe´riodiques des donne´es mesure´es, pre´sentent des effets tre`s positifs sur le rendement de la mode´lisation par re´seau de neurones artificiels. De manie`re ge´ne´rale, la me´thode par re´seaux neuronaux a` re´tropropagation non re´currents a e´te´ e´tudie´e par rapport aux applications de re´seau de neurones artificiels sur des donne´es de ressources en eau. Dans la pre´sente e´tude, le rendement des re´seaux de neurones artificiels et les re´seaux a` fonctions de base radiales ont e´te´ compare´s aux me´thodes par re´seaux neuronaux a` re´tropropagation non re´currents. Six diffe´rents mode`les ont e´te´ e´tudie´s afin de pre´dire les de´bits mensuels des rivie`res. Le mode`le par ondelettes et re´seaux neuronaux a` re´tropropagation non re´currents s’est ave´re´ supe´rieur aux autres mode`les en termes des crite`res de rendement choisis. Mots-cle´s : transforme´e par ondelettes, re´seaux neuronaux a` re´tropropagation non re´currents, re´seau neuronal a` re´gression, re´seaux a` fonctions de base radiales, de´bit mensuel, pre´vision. [Traduit par la Re´daction]

Introduction Monthly river flow forecasting is very important for water resources system planning and management problems such as dam construction, reservoir operation, flood control, and wastewater disposal (Nagesh Kumar et al. 2004). The purpose of forecasting is to reduce the risk in a decision at any given point of interest. River flow forecasting has been studied by various researchers during the past few decades. Generally, river flow models can be grouped into the two main techniques: physical-based models and black-box models (Sivakumar et al. 2002). Physical models usually contain simplified shapes of physical laws and are dependent on parameters that represent basin characteristics (Hsu et al. 1995). However, physical models have some difficulties as they require a Received 14 August 2007. Revision accepted 19 August 2008. Published on the NRC Research Press Web site at cjce.nrc.ca on 10 December 2008. T. Partal. Civil Engineering Department, Engineering Faculty, Dumlupinar University, Kutahya 43100, Turkey (email: [email protected]). Written discussion of this article is welcomed and will be received by the Editor until 31 May 2009. Can. J. Civ. Eng. 36: 26–39 (2009)

significant amount data — a large number of parameters representing the physical dynamics of a river. The artificial neural network (ANN) method, which is a nonlinear blackbox model, has been successfully studied with respect to forecasting various hydrological variables. Maier and Dandy (2000) presented a detailed review about ANN applications on hydrological variables such as flow and precipitation. The ANN model was employed for rainfall-runoff forecasting (Hsu et al. 1995; Minns and Hall 1996; Tokar and Johnson 1999; Jain et al. 2004), for river flow prediction (Campolo et al. 1999; Lauzon et al. 2000; Sivakumar et al. 2002; Cigizoglu 2003a, 2003b, 2004, 2005b; Kis¸i 2004; Cigizoglu and Kis¸i 2005; Nagesh Kumar et al. 2004), for precipitation forecasting (Hall et al. 1999; Bodri and Cernak 2000; Silverman and Dracup 2000; Freiwan and Cigizoglu 2005), for sediment prediction (Cigizoglu 2004; Cigizoglu and Alp 2006; Cigizoglu and Kis¸i 2006; Ard
c¸l
og˘lu et al. 2007), and for modeling of a water treatment process (Baxter et al. 2002). Baawain et al. (2005) employed artificial neural networks for the prediction of El Nino southern oscillation (ENSO) using some of its indicators. Generally, in most of the ANN applications, the feed-forward backpropagation method (FFBP) was employed to train neural networks (Cigizoglu 2005a). However, the FFBP algorithm has some drawbacks, as it is very sensitive to the selected

doi:10.1139/L08-090

Published by NRC Research Press

Partal

27

Fig. 1. Typically layered feed-forward neural network.

initial weight values and may show performances differing from each other excessively (Cigizoglu 2005a). Different ANN methods, such as radial basis function (RBF) and generalized regression neural network (GRNN), have been used for a few applications in the water resources area (Cigizoglu 2005a, 2005b; Cigizoglu and Alp 2004, 2006). Although the ANN has been extensively accepted as a useful tool for hydrologic processes modeling, it has some criticism. It does not use any the mathematical definitions of the physical model processes. Also, the successful performance of the model is significantly dependent on the user’s ability and knowledge (Cigizoglu 2004). At this point, wavelet transform, which provides considerable information about the structure of the physical process to be modeled, has positive contributions on modeling performance. In the last few years, new hybrid models on wavelet transform processes have been improved for forecasting purposes (Li et al. 1999; Yao et al. 2000; Zhang and Dong 2001; Zheng et al. 2000; Partal and Kis¸i 2007). The wavelet technique was applied to simulate streamflow (Bayaz
t et al. 2001; Bayaz
t and Aksoy 2001). Observed time series are decomposed into various components by a wavelet transform so that new time series can be used as inputs for the ANN network. The wavelet–ANN model has been studied by researchers in hydrology and water resources in the pastfew years. Wang and Ding (2003) suggested a wavelet network model and used this model for monthly groundwater level and daily discharge forecasting. They used decomposed components of the observed data as inputs for the FFBP network. They showed that a discrete wavelet transform can be successfully combined with artificial neural networks. Kim and Valdes (2003) employed a FFBP neural network model combined with discrete wavelet transforms for monthly and annual flow and rainfall forecasting. They used the neural network model in two stages. Neural networks were firstly employed to forecast the signals decomposed by wavelet transform in various resolution levels and later the forecasted decomposed signals were reconstructed into the original time series. They showed that the model increased the neural network forecasting success. In another study, Anctil and Tape (2004) used a wavelet–neural network model for oneday-ahead rainfall-runoff forecasting. The time series was

decomposed by wavelets into three subseries: short, intermediate, and long wavelet periods. Then, the ANN forecasting model was trained for each wavelet-decomposed subseries and later forecasted decomposed signals were reconstructed into the original time series. Cannas et al. (2006) conducted a study to predict monthly river flow data using wavelet analysis and neural networks. Two different approaches using wavelet and neural networks were used for river flow prediction. Firstly, the ANN method was employed using wavelet coefficients of observed data as inputs and the same coefficients one month ahead as the output. Then, the ANN model was again employed to reconstruct the flow values. Secondly, the ANN model was employed using wavelet coefficients of observed data as inputs and flow one month ahead as the output. They used only one month preceding values of wavelet coefficients. They found that the neural network model with decomposed wavelet components has better performance than a conventional neural network with original values. In all of the mentioned studies, a feed-forward back-propagation algorithm was employed to train neural networks. The objective of this research is to study the potential of wavelet and different neural network structures (FFBP, RBF, GRNN) when used for monthly river flow forecasting. Firstly, data was decomposed into wavelet components (DWs) by discrete wavelet transform (DWT). Then, wavelet–ANN configurations were constructed with appropriate (selected) wavelet components as inputs and original data as the output. In this study, correlation coefficients between each wavelet component of the observed flow data and observed flow series were evaluated for the selection of dominant components. Finally, employment of a wavelet–FFBP in river flow forecasting is compared with the wavelet– GRNN and wavelet–RBF performances. Wavelet transforms on a radial basis function and on generalized regression neural networks for river flow forecasting are used in this study. Also, the performance of wavelet–ANN methods is compared with the conventional ANN methods.

Methods Feed-forward back-ropagation The FFBP is the most known ANN method in water resourPublished by NRC Research Press

28

Can. J. Civ. Eng. Vol. 36, 2009

Fig. 2. Structure of radial basis neural network with input, hidden, and output units.

Fig. 3. Structure of generalized regression neural network.

and spread (s) parameters; K refers to exponential function. Synaptic weights (wij) are only between hidden and output layers (Sudheer and Jain 2003). For the xj input pattern, the response of the jth node in the hidden layer is zj. The output of the network at the jth output is given by ½1


yL ¼

L X

zj wij

j¼1

Different spread constants were tried in the study.

ces published literature. An FFBP neural network structure is showed in Fig. 1. An FFBP network structure has one input layer, one output layer, and at least one hidden layer with hidden neurons. The connections between neurons in different layers are supplied by adjustmed weighting values. Each neuron is connected only with neurons in following layers (Cigizoglu 2004). Each neuron sums its inputs and later produces its output using an activation function. In this study, a tangent sigmoid function is used as the neuron transfer function. Predicted output values are always different from observed values. The weight of connections is modified based on the differences between the computed values and the observed values at the output layer. This is the backpropagation process. After that, a feed-forward process is again formed until a target total error or number of prescribed iterations is reached (Kis¸i 2005).

Generalized regression neural networks The GRNN, which does not need a training procedure as in the back-propagation method, has four layers (input layer, pattern layer, summation layer, and output layer) (Fig. 3). In the first layer, there are input parameters (x1, x2,...xn) that are completely connected to the second layer, which is the pattern layer. The pattern units are connected to the summation layer. The spread parameter of the transaction function is determined by trial and error (Cigizoglu and Alp 2004). The GRNN defines any arbitrary function between input and output nodes, and is more useful for the estimation of continuous variables (as in standard regression techniques). It is based on a standard statistical technique called kernel regression (Cigizoglu 2005b). By definition, the regression of a dependent variable y on an independent x, estimates the most probable value for y, given x and a training set. The regression method will produce estimated value of y, which minimizes the mean-squared error. The estimate Y can be visualized as a weighted average of all observed values. More details on GRNN networks can be seen in Cigizoglu (2005b).

Radial basis function-based neural networks Radial basis function-based neural networks were first developed by Bromhead and Lowe (1988). The RBF neural network model is inspired by the locally turned response observed in biological neurons (Fig. 2). Theoretically, an RBF network is similar to a FFBP network. An RBF network uses a radial symmetric transfer function on the hidden layer. A radial symmetric transaction consists of centers (m)

Wavelet transform The wavelet transform is a strong mathematical tool that presents nonstationary variance analysis at many different scales (periods) in a time series (Daubechies 1990; Smith et al. 1998). Recently, wavelet transforms have been successfully studied for analysis of various hydrological and meteorological variables (Smith et al. 1998; Torrence and Compo 1998; Park and Mann 2000; Drago and Boxall 2002; Kucuk Published by NRC Research Press

Partal

29

Table 1. Related information for the flow data. Station K
lay
k Ru¨stu¨mko¨y

Longitude 38812’E 29846’E

Latitude 38819’N 40815’N

Drainage area (km2) 277.6 2021.0

xmean (m3/s) 1.78 19.11

xmin (m3/s) 0.2 0.2

xmax (m3/s) 18.1 90.3

sx (m3/s) 2.45 17.37

Csx 3.44 1.35

r1 0.56 0.75

r2 0.12 0.43

Note: Sx, standard deviation; Csx, skewness; r1, lag-1; r2, lag-2.

Fig. 4. Plot of the flow data.

2004; Yan et al. 2004). Wavelet transform, developed during the past few decades, appears to be a more effective tool than the Fourier transform (FT), which does not provide an accurate time–frequency analysis for nonstationary signals (Coulibaly and Burn 2004). The continuous wavelet transform provides an ideal opportunity to examine the process of energy variations in terms of where and when hydrological events occur (Kucuk and Agiralioglu 2006). Given a continuous time series x(t), t [ [?, –?], the wavelet function j (h) that depends on a nondimensional time parameter h can be acquired as below
t  t  ½2
jðhÞ ¼ jðt; sÞ ¼ s1=2 j s where t is time, s is wavelet scale, and t is the time step in which the window function is iterated; j(h) should have a zero mean and be localized both with respect to time and Fourier space (Meyer 1993). The successive wavelet transform of x(t) is defined as ½3


Wðt; sÞ ¼ s1=2

Zþ1

xðtÞj


t  t  s

dt

1

where * indicates the conjugate complex function; W(t, s)

presents a two-dimensional picture of wavelet power under a different scale. However, computing the wavelet coefficients at every possible scale (resolution level) generates a large amount of data. If wavelet coefficients are only obtained at defined scales, the wavelet analysis will be much more efficient and useful. The most common wavelet scales are dyadic scales. This transform is called discrete wavelet transform as described below  
t  t  t  nt 0 sm m=2 0 ¼ s0 j ½4
jm;n sm s 0 where m and n are integers that control, respectively, the scale and time; s0 is a specified fixed dilation step greater than 1; and t0 is the location parameter and must be greater than zero. The most general choice for the parameters s0 and t0 is 2 and 1 (time steps), respectively. Discrete wavelet transforms that present the power of two logarithmic scalings of the translations are the most efficient solution for practical purposes (Mallat 1989). For a discrete time series xi, where xi occurs at discrete time i (i.e., integer time steps are used here), the discrete wavelet transform becomes Published by NRC Research Press

30

Can. J. Civ. Eng. Vol. 36, 2009

Fig. 5. Discrate wavelet (DW) components for K
lay
k flow data. y-axis values refers to the wavelet powers of the time series.

Published by NRC Research Press

Partal

31

Table 2. Correlation coefficients between discrete wavelet components and observed flow data for the K
lay
k station. Previous monthly flow Qt–1 Qt–2 Qt–3 Qt–4 Qt–5

DW1 0.13 0.13 0.13 0.13 0.13

DW2 0.27 –0.32 –0.42 –0.13 0.13

DW3 0.55 0.29 –0.04 –0.33 –0.51

DW4 0.37 0.30 0.18 0.05 –0.06

DW5 0.28 0.27 0.26 0.25 0.13

DW6 0.26 0.26 0.25 0.24 0.23

DW7 0.20 0.20 0.20 0.21 0.21

DW8 0.13 0.13 0.13 0.13 0.13

App. 0.12 0.12 0.12 0.12 0.12

Correlation of summed DW components 0.68 0.46 0.37 0.34 0.28

Table 3. Correlation coefficients between discrete wavelet components and observed flow data for the Ru¨stu¨mko¨y station. Previous monthly flow Qt–1 Qt–2 Qt–3 Qt–4 Qt–5

½5


DW1 –0.12 –0.06 0.03 0.01 0.01

Wm;n ¼ 2m=2

N 1 X

DW2 0.30 –0.08 –0.22 –0.14 –0.08

DW3 0.73 0.41 –0.01 –0.42 –0.71

DW4 0.47 0.35 0.19 0.02 –0.14

DW5 0.27 0.26 0.24 0.21 0.08

xi jð2m i  nÞ

i¼0

where Wm,n is the wavelet coefficient for the discrete wavelet of scale s = 2m and location t = 2mn, and N is the an integer power of 2, N = 2m. The discrete wavelet transform provides a means of obtaining one or more detailed series and an approximation at different scales.

Case study In this study, forecasting is performed on two stations. River flow data belonging to K
lay
k station (station No. 2131) on the Beyderesi river at the F
rat basin in the east Anatolia region of Turkey and Ru¨stu¨mko¨y station (station No. 1222) on Kocasu river at the Sakarya basin in the Marmara region of Turkey have been employed. Related information for the flow data is presented in Table 1. The data has a length of 33 years (monthly average data of daily flow) covering a time period between 1962 and 1994. The data was found to be homogeneous for this period. The plot of the whole records is illustrated in Fig. 4. Some of the statistical properties of the monthly flow data are presented in Table 1. The observed series of the K
lay
k station have quite high skewness values (3.44), whereas the flow data of the Ru¨stu¨mko¨y station have a lower skewness coefficient (1.43). The lag-1 auto-correlation of the records has a significant value of r1 = 0.56 for the K
lay
k station, whereas lag-2 auto-correlations are closer to zero (r2 = 0.12 for the K
lay
k station).

Results Wavelet decomposition of observed time series Discrete wavelet transform provides decomposed components (wavelet coefficients time series) at determined scales. This enables the study of components at different scales or periods. Therefore, wavelet coefficients time ser-

DW6 0.19 0.19 0.19 0.19 0.18

DW7 0.19 0.19 0.19 0.19 0.19

DW8 0.15 0.15 0.15 0.15 0.15

App. 0.13 0.13 0.13 0.13 0.13

Correlation of summed DW components 0.81 0.52 0.43 0.30 0.25

ies (DWs) of flow data were obtained by DWT. Each of the wavelet coefficients time series or wavelet components of observed flow data plays a distinct role in the original time series. The flow data was decomposed into an approximation series and an eight-detail components series. For decomposition, the algorithm proposed by Mallat (1989) was used. For the K
lay
k flow data, an eight-detail series (2, 4, 8, 16, 32, 64, 128, 256 months) and one approximation series are presented in Fig. 5. DW1 shows the highest frequency component, while DW8 shows the lowest frequency component. DW3 (8 month mode) and DW4 (16 month mode) components represent a nearly annual periodicity of monthly flow series. Each of components present different characteristic of the observed time series. Therefore, some wavelet components could be more efficient with respect to the observed time series. In this case, appropriate DW components have a highly positive effect on the ANN modeling ability. The correlation coefficients between each wavelet components of observed flow data and observed flow series are calculated and presented in Tables 2 and 3. These correlation tables are used for the determination of appropriate wavelet components for ANN forecasting. Among the correlations between 1-month-preceding subtime series and the original flow series, the DW3 component shows the highest correlation (equal to 0.55 and 0.73 for K
lay
k station and Ru¨stu¨mko¨y stations, respectively). DW2, DW4, DW5, DW6, and DW7 also show high correlation coefficients. Among 2-months-preceding subtime series and observed flow series, DW3, DW4, DW5, DW6, and DW7 components show slightly higher positive correlation than the others. For K
lay
k station, DW2 demonstrates negative high correlation (its value is –0.32). The correlation analysis between the periodic components and observed data reveals which component is more effective on the original flow data. The selection of the efficiency wavelet components has a significantly positive effect on the ANN modeling ability. Instead of using each DW component of original data individually, employment of the added suitPublished by NRC Research Press

32

Table 4. Correlation coefficients (R) and mean square error (MSE) for each forecast model-testing period for K
lay
k station. 1 input Model FFBP RBF GRNN Wavelet–FFBP Wavelet–RBF Wavelet–GRNN

R 0.45 0.22 0.42 0.56 0.56 0.51

2 input MSE 4.94 10.80 4.98 3.70 3.81 3.71

R 0.65 0.52 0.56 0.69 0.59 0.53

3 input MSE 2.94 3.94 4.35 3.31 3.88 4.16

R 0.72 0.61 0.54 0.92 0.91 0.89

4 input MSE 2.50 3.24 5.05 0.89 0.90 1.05

R 0.68 0.62 0.55 0.95 0.92 0.91

5 input MSE 2.77 3.13 4.98 0.45 0.80 0.91

R 0.56 0.51 0.45 0.93 0.91 0.88

MSE 3.72 3.78 4.62 0.88 0.91 1.22

Note: Bold numbers indicate the best correlations. FFBP, feed-forward back-propagation method; RBF, radial basis function; GRNN, generalized regression neural network.

Table 5. Correlation coefficients (R) and mean square error for each forecast model-testing period for Ru¨stu¨mko¨y station. 1 input R 0.43 0.41 0.33 0.80 0.77 0.76

MSE 110.11 127.26 145.5 36.76 48.21 48.33

R 0.76 0.41 0.35 0.90 0.89 0.89

3 input MSE 75.43 125.67 143.01 31.41 33.46 35.17

R 0.77 0.75 0.66 0.90 0.90 0.89

4 input MSE 70.3 80.4 134.3 27.15 31.51 35.46

R 0.73 0.70 0.63 0.91 0.91 0.85

5 input MSE 82.74 100.36 115.2 24.72 24.29 46.07

R 0.73 0.71 0.62 0.91 0.91 0.81

Note: Bold numbers indicate the best correlations. FFBP, feed-forward back-propagation method; RBF, radial basis function; GRNN, generalized regression neural network.

MSE 87.43 100.78 111.22 23.49 23.64 55.86

Can. J. Civ. Eng. Vol. 36, 2009

Published by NRC Research Press

Model FFBP RBF GRNN Wavelet–FFBP Wavelet–RBF Wavelet–GRNN

2 input

Partal

33

Fig. 6. Monthly flow forecast at K
lay
k station for (a) feed-forward back-propagation method (FFBP) models with 3 inputs; (b) radial basis function (RBF) and (c) generalized regression neural network (GRNN) model with four inputs.

able DW components is more useful and increases the performance of ANN models. Also, correlations between the summed DW series and observed flow series must show the highest value. For instance, the correlation between the 1-day-preceding DW3 series of the observed flow series and the observed flow series has the highest value (equal to 0.73, Table 3). However, for the new summed series, obtained by adding dominant DW components to each other (DW2+DW3+DW4+DW5+DW6+DW7), the corresponding correlation value jumps to 0.81 (see Table 3). It

is important to say that the number of the components to be included in the ANN input layer is actually dependent on the user’s preference. However, the determination of a limit for the correlation value may be quite helpful for this objective. According to this, the following components of detailed series, in light of the correlation analysis (Tables 2 and 3), were selected for ANN models inputs:

DW2 þ DW3 þ DW4 þ DW5 þ DW6 þ DW7 for Qt1 DW3 þ DW4 þ DW5 þ DW6 þ DW7 for Qt2 DW4 þ DW5 þ DW6 þ DW7 for Qt3 DW5 þ DW6 þ DW7 for Qt4 DW6 þ DW7 for Qt5 The new series obtained by adding appropriate DW components were used for forecasting. Published by NRC Research Press

34

Can. J. Civ. Eng. Vol. 36, 2009

Fig. 7. Monthly flow forecast for (a) wavelet–FFBP, (b) wavelet–RBF, and (c) wavelet–GRNN model with four inputs at K
lay
k station. FFBP, feed-forward back-propagation method; RBF, radial basis function; GRNN, generalized regression neural network.

Table 6. Artificial neural network model structures for each input configuration. Structures Inputs 1 2 3 4 5

FFBP (1,2,1) (2,3,1) (3,4,1) (4,4,1) (5,4,1)

RBF (1, s = (2, s = (3, s = (4, s = (5, s =

0.5,1) 0.7,1) 0.9,1) 0.9,1) 0.9,1)

GRNN (1, s = 0.2,1) (2, s = 0.3,1) (3, s = 0.5,1) (4, s = 0.4,1) (5, s = 0.4,1)

River flow forecasting Three different ANN models (FFBP, RBF, GRNN) were firstly evaluated for forecasting monthly flow on original data. Secondly, three ANN models with wavelet subseries (wavelet–FFBP, wavelet–RBF, wavelet–GRNN) were eval-

uated for flow forecasting. Therefore, six different forecast models were studied for flow forecasting. The data from 1964 to 1987 (312 values) were chosen for the training periods. The data from 1988 to 1994 (84 values) were employed for forecasting using ANN methods. The inputs represent the previous monthly flows (t–1,. . .,t–5) and the output layer node corresponds to the flow at time t. In the study, the following combinations of input data of flow were evaluated: (1) (2) (3) (4) (5)

Qt–1 Qt–1 and Qt–2 Qt–1, Qt–2, and Qt–3 Qt–1, Qt–2, Qt–3, and Qt–4 Qt–1, Qt–2, Qt–3, Qt–4, and Qt–5

The performance evaluation measures are the mean Published by NRC Research Press

Partal

35

Fig. 8. Monthly flow forecast for artificial neural network models with three inputs at Ru¨stu¨mko¨y station. (a) feed-forward back-propagation method (FFBB), (b) radial basis function (RBF), and (c) generalized regression neural network (GRNN).

square error (MSE) and correlation coefficient (R) between forecasted and observed flows. The forecasting results are summarized in Tables 4 and 5. The number of hidden layer nodes and spread parameter value are found by trial and error. The most appropriate model configuration was shown in Table 6. Tables 4 and 5 show that, generally, forecasts with 3 month, 4 month, and 5 month previous flows provided the best performance. Table 4 shows that for K
lay
k Station the wavelet–FFBP model with four inputs having the configuration (4,4,1) provided the best performance criteria [MSE = 0.45 (m3/s)2; R = 0.95]. The R value shows a very strong relationship between forecasted and observed values. Whereas correlation coefficients obtained by the conventional ANN method are up to 0.7, with the wavelet–ANN model these values increase to 0.9. The best correlation coefficient found by the FFBP model is 0.72. For the FFBP model, the MSE of the best simulation was found to be 2.50 (m3/s)2. For four inputs, the wavelet–RBF model shows 0.92 correlation coefficient values, whereas the RBF model shows 0.62 correla-

Table 7. Root mean square error (RMSE) for observed peak flows and corresponding forecasts by the best forecast model for the testing period.

Model FFBP RBF GRNN Wavelet–FFBP Wavelet–RBF Wavelet–GRNN

Ru¨stu¨mko¨y station

K
lay
k station

Inputs 3 3 3 4 4 3

Inputs 3 4 4 4 4 4

RMSE 12.18 12.82 17.02 6.99 7.16 7.62

RMSE 4.83 5.25 6.55 2.43 2.62 2.57

Note: FFBP, feed-forward back-propagation method; RBF, radial basis function; GRNN, generalized regression neural network.

tion coefficient values. Similarly, while the GRNN model show 0.55 correlation coefficient values, the wavelet-GRNN model show 0.91 correlation coefficient values. Generally, Published by NRC Research Press

36

Can. J. Civ. Eng. Vol. 36, 2009

Fig. 9. Monthly flow forecast for (a) wavelet–FFBP, (b) wavelet–RBF models with five inputs, and (c) wavelet–GRNN model with two inputs at Ru¨stu¨mko¨y station. FFBP, feed-forward back-propagation method; RBF, radial basis function; GRNN, generalized regression neural network.

GRNN models have the lowest correlation coefficient and MSE values. Table 5 shows that the wavelet–FFBP model with five inputs having the configuration (5,3,1) provided the best performance criteria [MSE = 23.49 (m3/s)2; R = 0.91] for Ru¨stu¨mko¨y station. For the FFBP model, the MSE of the best simulation was equal to 70.73 (m3/s)2 and the correlation coefficient was found to be 0.77. However, the wavelet–FFBP model and the wavelet–RBF model show almost same forecast performance. The MSE and correlation coefficients values by the wavelet–RBF model are 23.64 and 0.91, respectively. Here, it is obvious that the wavelet– FFBP model and the wavelet–RBF model show the best performance in terms of determination coefficient for both stations. For the GRNN model, the MSE of the best simulation was equal to 11.22 (m3/s)2 (for five inputs) and the

correlation coefficient was found to be 0.66 (for three inputs). However, the wavelet–GRNN model has superior performance [MSE = 35.17 (m3/s)2; R = 0.89]. The results clearly show that wavelet transform significantly improved the performance of ANN models with respect to river flow forecasting. Figure 6, 7, 8, and 9 contain scatter diagrams and a plot of the output (using six different forecasting methods) and observed flows of both stations for the testing period. For K
lay
k station, the forecasted flows are compared with observed ones in the hydrograph and scatter plot in Fig. 6. Figure 7 shows that both low parts and high parts of the observed flow were forecasted more approximately by the wavelet–FFBP model. Scatter plots show that the most accurate forecast was obtained by the wavelet–FFBP method. Published by NRC Research Press

Partal

Figure 8 shows that the ANN model (FFBP, RBF, GRNN) outputs are close to the observed ones for Ru¨stu¨mko¨y station. However, Fig. 9 shows that the wavelet–ANN outputs are smoother and closer to the ANN model output for the observed data. On the scatter plot in Fig. 9, it can be clearly seen that the values are denser in the neighborhood of the straight line. As the result, Figs. 6–9 generally show that ANN and wavelet–ANN model approximately forecast the general behavior of the observed data. However, examination of these figures shows that the wavelet–ANN models (wavelet– FFBP, wavelet–RBF, wavelet–GNN) show a fairly better match with observed data. The forecasting performances of the peak flows in terms of root mean square error (RMSE) are presented in Table 7. The wavelet–FFBP model has the best performance criteria (RMSE = 6.99 m3/s). The peaks in the testing period were estimated better by the wavelet– ANN models. In Figs. 6–9, it is seen that wavelet–ANN model forecasts are smoother than ANN model forecasts. It is known that discrete wavelet transform (DWTs) decompose into different wavelet components to observed signal. The DWTs allow a detection noise component in the data. The noise component is mainly responsible for roughness in the forecasting. That is to say, removing the noise component from the data provided smoother and more efficient forecasting results.

Conclusions The aim of this study was to investigate the ability of different neural networks and wavelet transforms to forecast flow. Six different forecasting methods — FFBP, RBF, GRNN, wavelet–FFBP, wavelet–RBF, and wavelet–GRNN — have been employed for forecasting monthly river flows. This is the first study on river flow forecasting using wavelet–RBF and wavelet–GRNN methods. The wavelet–FFBP model is compared, for the first time in study, with other wavelet– ANN models (wavelet–RBF and wavelet–GRNN). The performance of these different forecast models have been compared in terms of the selected performance criteria. For wavelet–ANN model forecasting, observed flow data was decomposed into periodic components by DWT. Periodic components are wavelet coefficient time series at different scales. One of the greatest difficultiess in ANN modeling lies in determining the appropriate model inputs for such a problem. So, appropriate components were determined using the correlation coefficients between each of DWs and observed flow series efficiencies. Using the efficiency components on observed flow data significantly improved the forecasting performance. The results show that the best performance in terms of performance criteria was obtained by the wavelet–FFBP model. Generally, GRNN models show the lowest correlation coefficient, but it generally results in the highest MSE values. It can be seen that the wavelet–ANN models have a significant positive effect on forecasting performance. For instance, for the best simulations, whereas correlation coefficients obtained by FFBP are around 0.7, the wavelet–FFBP method provided a correlation coefficient over 0.9. It is important to stress that generally, the forecast with 3, 4, and 5 month previous flows provided the best performance.

37

Wavelet–ANN models generally show similar performances. However, hydrographs of observed and forecasted values show that both low parts and high parts of the observed flow were forecasted more accurately by the wavelet–FFBP model. The time series of wavelet coefficients presents the evolution in time of the signals at each resolution level (Drago and Boxall 2002). Some wavelet components better represent the original signal than others. At each step back in time, the different wavelet components could be more efficient with respect to the observed signal. It is seen that at the each time step t back i ntime, the component belonging to the low scale must be omitted for more successful forecasting performance. Namely, for each of the previous monthly flows (Qt–1, Qt–2,. . .), the ANN model was employed by adding different wavelet components. Forecasting performance increased using this procedure. As a result, wavelet transforms positively affect ANN forecasting ability. Monthly flows were estimated much more accurately by wavelet–ANN models as compared with conventional ANN models. The wavelet–ANN model is a useful tool for flow forecasting.

References Anctil, F., and Tape, D.G. 2004. An exploration of artificial neural network rainfall-runoff forecasting combined with wavelet decomposition. Journal of Environmental Engineering and Science, 3(S1): S121–S128. doi:10.1139/s03-071. ¨ ., and Haktan
r, T. 2007. Suspended sediArd
c¸l
og˘lu, M., Kis¸i, O ment prediction using two different feed-forward back-propagation algorithms. Canadian Journal of Civil Engineering, 34(1): 120–125. doi:10.1139/L06-111. Baawain, M.S., Nour, M.H., El-Din, A.G., and El-Din, M.G. 2005. El Nin˜o southern-oscillation prediction using southern oscillation index and Nin˜o3 as onset indicators: Application of artificial neural networks. Journal of Environmental Engineering and Science, 4(2): 113–121. doi:10.1139/s04-047. Bayaz
t, M., and Aksoy, H. 2001. Using wavelets for data generation. Journal of Applied Statistics, 28(2): 157–166. doi:10.1080/ 02664760020016073. Baxter, C.W., Stanley, S.J., Zhang, Q., and Smith, D.W. 2002. Developing artificial neural network models of water treatment processes: a guide for utilities. Journal of Environmental Engineering and Science, 1(3): 201–211. doi:10.1139/s02-014. ¨ no¨z, B., and Aksoy, H. 2001. Nonparametric streamBayaz
t, M., O flow simulation by wavelet or Fourier analysis. Hydrological Sciences Journal, 46(4): 623–634. Bodri, L., and Cernak, V. 2000. Prediction of extreme precipitation using a neural network: application to summer flood occurrence in Moravia. Advances in Engineering Software, 31(5): 311–321. doi:10.1016/S0965-9978(99)00063-0. Bromhead, D., and Lowe, D. 1988. Multivariable functional interpolation and adaptive networks. Complex Systems, 2(3): 321–355. Campolo, M., Andreussi, P., and Soldati, A. 1999. River flood forecasting with a neural network model. Water Resources Research, 35(4): 1191–1197. doi:10.1029/1998WR900086. Cannas, B., Fanni, A., See, L., and Sias, G. 2006. Data processing for river flow forecasting using neural networks: Wavelet transforms and data partitioning. Physics and Chemistry of the Earth, Parts A/B/C, 31(18): 1164–1171. Cigizoglu, H.K. 2003a. Estimation, forecasting and extrapolation of flow data by artificial neural networks. Hydrological Sciences Journal, 48(3): 349–361. doi:10.1623/hysj.48.3.349.45288. Published by NRC Research Press

38 Cigizoglu, H.K. 2003b. Incorporation of ARMA models into flow forecasting by artificial neural networks. Environmetrics, 14(4): 417–427. doi:10.1002/env.596. Cigizoglu, H.K. 2004. Estimation and forecasting of daily suspended sediment data by multi layer perceptrons. Advances in Water Resources, 27(2): 185–195. doi:10.1016/j.advwatres. 2003.10.003. Cigizoglu, H.K. 2005a. Application of the generalized regression neural networks to Iitermittent flow forecasting and estimation. Journal of Hydrologic Engineering, 10(4): 336–341. doi:10. 1061/(ASCE)1084-0699(2005)10:4(336). Cigizoglu, H.K. 2005b. Generalized regression neural network in monthly flow forecasting. Civil Engineering and Environmental Systems, 22(2): 71–84. doi:10.1080/10286600500126256. Cigizoglu, H.K., and Kis¸i, O. 2005. Flow prediction by three back propagation techniques using k-fold partitioning of neural network training data. Nordic Hydrology, 36(1): 1–16. Cigizoglu, H.K., and Kis¸i, O. 2006. Methods to improve the neural network performance in suspended sediment estimation. Journal of Hydrology, 317(3–4): 221–238. Cigizoglu, H.K., and Alp, M. 2004. Rainfall-runoff modelling using three neural network methods. In Proceedings of the 7th International Conference on Artificial Intelligence and Soft Computing - ICAISC 2004, Zakopane, Poland, 7–11 June 2004. Lecture Notes in Artificial Intelligence, subseries of Lecture Notes in Computer Science. Edited by J. G. Carbonell and J. Siekmann. Springer-Verlag, Berlin/Heidelberg, Germany. Vol. 3070. pp. 166–171. Cigizoglu, H.K., and Alp, M. 2006. Generalized neural network in modelling river sediment yield. Advances in Engineering Software, 37(2): 63–68. doi:10.1016/j.advengsoft.2005.05.002. Coulibaly, P., and Burn, D.H. 2004. Wavelet analysis of variability in annual Canadian streamflows. Water Resources Research, 40(3): W03105.1–W03105.14. Daubechies, I. 1990. The wavelet transform, time-frequency localization and signal analysis. IEEE Transactions on Information Theory, 36(5): 961–1005. doi:10.1109/18.57199. Drago, A.F., and Boxall, S.R. 2002. Use of the wavelet transform on hydro-meteorological data. Physics and Chemistry of the Earth, Parts A/B/C, 27(32): 1387–1399. Freiwan, M., and Cigizoglu, H.K. 2005. Prediction of total monthly rainfall in Jordan using feed forward backpropagation method. Fresenius Environmental Bulletin. 14(2): 142–151. Hall, T., Brooks, H.E., and Doswell, C.A., III. 1999. Precipitation forecasting using a neural network. Weather and Forecasting, 14(3): 338–345. doi:10.1175/1520-0434(1999)0142.0.CO;2. Hsu, K., Gupta, H.V., and Sorooshian, S. 1995. Artificial neural network modelling of the rainfall runoff process. Water Resources Research, 31(10): 2517–2530. doi:10.1029/95WR01955. Jain, A., Sudheer, K.P., and Srinivasulu, S. 2004. Identification of physical processes inherent in artificial neural network rainfallrunoff models. Hydrological Processes, 18(3): 571–581. doi:10. 1002/hyp.5502. Kim, T.W., and Valdes, J.B. 2003. Nonlinear model for drought forecasting based on a conjunction of wavelet transforms and neural networks. Journal of Hydrologic Engineering, 8(6): 319– 328. doi:10.1061/(ASCE)1084-0699(2003)8:6(319). ¨ . 2004. River flow modeling using artificial neural netKis¸i, O works. Journal of Hydrologic Engineering, 9(1): 60–63. doi:10. 1061/(ASCE)1084-0699(2004)9:1(60). ¨ . 2005. Daily river flow forecasting using artificial neural Kis¸i, O networks and auto-regressive models. Turkish Journal of Engineering and Environmental Sciences, 29(1): 9–20.

Can. J. Civ. Eng. Vol. 36, 2009 Kucuk, M. 2004 Dalgac
k do¨nu¨s¸u¨m teknig˘i kullanarak ak
m serilerinin modellenmesi. Ph.D. thesis, Istanbul Technical University, Instanbul, Turkey. [In Turkish.] Kucuk, M., and Agiralioglu, N. 2006. Wavelet regression techniques for streamflow predictions. Journal of Applied Statistics, 33(9): 943–960. doi:10.1080/02664760600744298. Lauzon, N., Rousselle, J., Birikundavyi, S., and Trung, H.T. 2000. Real-time daily flow forecasting using black-box models, diffusion processes, and neural networks. Canadian Journal of Civil Engineering, 27(4): 671–682. doi:10.1139/cjce-27-4-671. Li, X., Ding, J., and Li, H. 1999. Combing neural network models based on wavelet transform. Journal of Hydraulic, 2: 1–4. [In Chinese.] Maier, H.R., and Dandy, G.C. 2000. Neural networks for the prediction and forecasting of water resources variables: A review of modelling issues and applications. Environmental Modelling & Software, 15(1): 101–124. doi:10.1016/S1364-8152(99) 00007-9. Mallat, S.G. 1989. A theory for multi resolution signal decomposition: The wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(7): 674–693. doi:10. 1109/34.192463. Meyer, Y. 1993. Wavelets algorithms and applications. Society for Industrial and Applied Mathematics, Philadelphia, Pa. Minns, A.W., and Hall, M.J. 1996. Artificial neural networks as rainfall-runoff models. Hydrological Sciences Journal, 41(3): 399–417. Nagesh Kumar, D., Raju, K.S., and Sathish, T. 2004. River flow forecasting using recurrent neural network. Water Resources Management, 18(2): 143–161. doi:10.1023/B:WARM. 0000024727.94701.12. Park, J., and Mann, M.E. 2000. Interannual temperature events and shifts in global temperature: a ‘‘multiwavelet’’ correlation approach. Earth Interactions, 4(1): 1–36. Partal, T., and Kis¸i, O. 2007. Wavelet and neuro-fuzzy conjunction model for precipitation forecasting. Journal of Hydrology, 342(1–2): 199–212. doi:10.1016/j.jhydrol.2007.05.026. Silverman, D., and Dracup, J.A. 2000. Artificial neural networks and long-range precipitation prediction in California. Journal of Applied Meteorology, 39(1): 57–66. doi:10.1175/15200450(2000)0392.0.CO;2. Sivakumar, B., Jayawerdana, A.W., and Fernando, T.M.K.G. 2002. River flow forecasting: use of phase-space reconstruction and artificial neural networks approaches. Journal of Hydrology, 265(1–4): 225–245. doi:10.1016/S0022-1694(02)00112-9. Smith, L.C., Turcotte, D.L., and Isacks, B. 1998. Stream flow characterization and feature detection using a discrete wavelet transform. Hydrological Processes, 12(2): 233–249. doi:10.1002/ (SICI)1099-1085(199802)12:23.0.CO;2-3. Sudheer, K.P., and Jain, S.K. 2003. Radial basis function neural network for modeling rating curve. Journal of Hydrologic Engineering, 8(3): 161–164. doi:10.1061/(ASCE)1084-0699(2003) 8:3(161). Tokar, A.S., and Johnson, P.A. 1999. Rainfall-runoff modelling using artificial neural networks. Journal of Hydrologic Engineering, 4(3): 232–239. doi:10.1061/(ASCE)1084-0699(1999) 4:3(232). Torrence, C., and Compo, G.P. 1998. A practical guide to wavelet analysis. Bulletin of the American Meteorological Society, 79(1): 61–78. doi:10.1175/1520-0477(1998)0792.0.CO;2. Wang, W., and Ding, J. 2003. Wavelet network model and its application to the prediction of hydrology. Nature and Science, 1(1): 67–71. Published by NRC Research Press

Partal Yan, Z., Tsimplis, M.N., and Woolf, D. 2004. Analysis of the relationship between the North Atlantic oscillation and sea level changes in northeast Europe. International Journal of Climatology, 24(6): 743–758. doi:10.1002/joc.1035. Yao, S.J., Song, Y.H., Zhang, L.Z., and Cheng, X.Y. 2000. Wavelet transform and neural networks for short term electrical load forecasting. Energy Conversion and Management, 41(18): 1975–1988. doi:10.1016/S0196-8904(00)00035-2.

39 Zhang, B.L., and Dong, Z.Y. 2001. An adaptive neural-wavelet model for short term load forecasting. Electric Power System Research, 59(2): 121–129. doi:10.1016/S0378-7796(01)00138-9. Zheng, T., Girgis, A.A., and Makram, E.B. 2000. A hybrid wavelet kalman filter method for load forecasting. Electric Power System Research, 54(1): 11–17. doi:10.1016/S0378-7796(99)00063-2.

Published by NRC Research Press