Weather Forecasting Using ANN with Error Backpropagation Algorithm Meera Narvekar, Priyanca Fargose and Debajyoti Mukhopadhyay
Abstract Weather forecasting is very necessary for making strategic plans and executing daily tasks in multiple application areas like airport, agriculture, electricity, water reservoir, tourism, and daily life. Accurate prediction of weather conditions is the challenge in front of meteorological departments. Many techniques such as data mining, k-means, decision tree, etc. have been used to predict the weather. The technique presented in the paper uses artificial neural network with error back propagation. Here we generate weather forecast for next day. The results are obtained in acceptable range of accuracy. ANN suits best classification technique that works on complex systems like Weather forecasting, in which datasets are nonlinear. The forecasting model used is limited to only certain geographical area which comes under Mumbai, India. Keywords Artificial neural network Weather forecasting ANN
⋅
⋅
Error backpropagation algorithm
⋅
1 Introduction Weather forecasting is a probabilistic system, where the future outcome is difficult to state in advance. The weather depends on multiple climatic conditions like temperature, air pressure, humidity, wind flow speed and direction, cloud height and density, and rainfall. Change in any of this parameter can have more or less impact on the future weather. Meera Narvekar (✉) ⋅ Priyanca Fargose D. J. Sanghvi College of Engineering, Mumbai, India e-mail:
[email protected] Priyanca Fargose e-mail:
[email protected] Debajyoti Mukhopadhyay Maharashtra Institute of Technology, Pune, India e-mail:
[email protected] © Springer Science+Business Media Singapore 2017 S.C. Satapathy et al. (eds.), Proceedings of the International Conference on Data Engineering and Communication Technology, Advances in Intelligent Systems and Computing 468, DOI 10.1007/978-981-10-1675-2_62
[email protected]
629
630
Meera Narvekar et al.
The weather forecasting is a real time system used in several applications such as airport, agriculture, electricity, water reservoir, and tourism. Thus, accuracy of the weather forecasts is of prime importance for decision-making. Many classification techniques are such as support vector machine, K-nearest neighbor, naive Bayes classifier, decision tree, learning vector quantization, artificial neural network (ANN), etc., are used to classify whether in classes such as sunny, partly cloudy, thunderstorm etc. Out of these ANN uses artificial intelligence to understand the complex and nonlinear input data and generates the frequent patterns of similar samples together which associate with one class of output parameter. Artificial neural network with error back propagation technique is used for classification of weather conditions. The success of the model depends on multiple criteria like set of input data, structure of the model, activation function of each layer, number of input and output parameters, number of hidden layers and neurons in each layer. A proper arrangement of all these parameters will give more accurate results. The meteorological centers record the observations in terms of different Input parameters. Each of the parameter is different than the other one and has different set of values. Handling such nonlinear data is a very challenging task. ANN is a special classification technique that deals efficiently with the nonlinear data. This paper uses huge dataset of seven years with an interval of 15 min for training and testing the ANN Model. The tansig and purelin activation functions are used in hidden layers. Use of Back propagation technique will help the model to learn and adapt the necessary adjustments in weight and bias value depending on the error generated in each epoch. Thus, after finishing with the training process, the model will generate more accurate forecasts. The model is created considering only a specific geographical area. Scope of the model is limited to weather forecasting in Mumbai city only. Following section covers previous work done on Weather Forecasting using multiple classification models.
2 Related Work Many researchers have used different classification techniques for multiple purposes. Lai et al. [1] describes a methodology to short-term temperature and rainfall forecasting over the east coast of China based on some necessary data preprocessing technique and the dynamic weighted time delay neural networks (DWTDNN), in which each neuron in the input layer is scaled by a weighting function that captures the temporal dynamics of the biological task. Results have shown that neural networks with as few as a single hidden layer and an arbitrary
[email protected]
Weather Forecasting Using ANN with Error …
631
bounded and nonconstant activation function can approximate weather forecast prediction for rainfall and temperature. The next step aims to show that it can also predict wind direction. Shuxia [2] forecasted power consumption using a nonlinear network model between power consumption and the affected factors were obtained through training the relative data of power consumption from 1980 to 2005 in China. The result shows that the BP neural network with immune algorithm is more accurate than optimized neural network by genetic algorithm. Marzi et al. [3] studies solutions for forecasting option prices in a volatile financial market. It reviews a mathematical model based on traditional Black-Scholes parametric solution. The neural network was trained using the Levenberg–Marquardt backpropagation algorithm. When the NNHybrid model was compared against both the Black-Scholes model and the NNSimp model, a number of observations were made. The first outcome was that the NNHybrid’s Mean Absolute percentage error for the entire data set was better than both the Black-Scholes and NNSimp models. Gao et al. [4] performs sales forecasting for accurate and speedy results to help e-commerce companies solve all the uncertainty associated with demand and supply and reduce inventory cost. The model is proposed with extreme learning machine. By determining the coefficient of ELM, that is the number of hidden nodes, and conducting the experiments with different variables, the proposed method does have a certain effect for reducing the forecasting RMSE and improving the accuracy and speed. Harshani et al. in [5] has used Ensemble neural network where, finite numbers of ANN are trained for the same task and their results are combined. The performance is compared with Back propagation neural network (BPN), radial basis function network (RBFN), and general regression neural network GRNN). The results show that, ENN model predicts rainfall more accurately than individual BPN, RBFN and GRNN. The weakness of the technique proposed in paper is that it predicts only the rainfall. Hayati et al. in [6] has used ANN for “one day ahead” (the next day) temperature prediction. The training was done on 65 % of dataset. Testing performed on 35 % of dataset. Network has one hidden layer and sigmoid transfer function is used in that layer. Number of neurons and epochs were decided by trial and error method. The strengths of this technique are, the structure used has minimum prediction error, good performance and reasonable prediction accuracy. The model is limited to forecast only temperature. Hall et al. [7] developed a neural network and initially two years data was used consisting nineteen variables. In this application, two networks are created, a QPF network for predicting amount of precipitation and a PoP network for probability or confidence in the forecast. This technique improves precipitation forecast dramatically, particularly for applications requiring accurate results. The main drawback of this model is, it focused only on precipitation forecasting. Above survey shows that ANN is most suitable to predict the weather conditions.
[email protected]
632
Meera Narvekar et al.
3 Techniques Used This section covers the detailed information about the techniques used in this model to forecast weather. First section covers ANN and second section describes error backpropagation algorithm.
3.1
Artificial Neural Network
An artificial neuron is a computational model inspired in the natural neurons. For each input received, the neuron is activated and emits a signal. This signal might activate other neurons. ANN basically consists of inputs, which are multiplied by weight and then computed by a mathematical function which determines the activation of the neuron. Depending on the weights, the computation of the neuron will be different [8]. By adjusting the weights of an artificial neuron we can obtain the output we want for specific inputs.
3.2
Error Backpropagation Algorithm
The backpropagation algorithm uses supervised learning, which means that we provide the algorithm with examples of the inputs and outputs we want the network to compute, and then the error is calculated as difference between actual and expected results. The idea of the backpropagation algorithm is to reduce this error, until the ANN learns the training data. The training begins with random weights, and the goal is to adjust them so that the error will be minimal [8].
4 System Design The system is designed with proper arrangement of sub modules working together to achieve the final output, that is the weather forecast. Following Fig. 1 shows the block diagram for system; followed by the detailed explanation on working of system.
4.1
Data Preprocessing and Cleaning
The acquired data needs to be preprocessed. The input data is revised in various ways for each input parameter as mentioned further. The values for each parameter
[email protected]
Weather Forecasting Using ANN with Error …
633
Fig. 1 Block diagram of the system
are checked for allowed range of values in that parameter and the outliers which do not fit in the range are removed. Records with missing data are removed. Also the nonnumeric data is converted to numerical acceptable data.
4.2 4.2.1
Initialization of Artificial Neural Network Architecture
The architecture of neural network is of 28-10-10-5 type. That means, the network has 28 input parameters, two hidden layers each with 10 neurons and 5 output neurons.
4.2.2
Weights and Bias
The values of bias and weights connecting individual neurons in all layers are assigned randomly using a random function with a seed value. Change in seed value for random function also has a great impact on the efficiency of system. Following Table 3.2 shows the summary of results achieved using different seed values for the random function used for assigning values to weights.
4.2.3
Activation Function
The transfer functions normally used in multilayer neural network are tansig, logsig, and purelin. Tansig is a neural transfer function used to calculate a layer’s output from its net input. It is used because meteorological observed data is nonlinear in nature, from which it is difficult to generate the frequent patterns of similar type of
[email protected]
634
Meera Narvekar et al.
data related to one of the output class. This activation function gives output from −1 to +1 [9]. Purelin is a neural transfer function used in fitting problems. This activation function is used in the middle layer between two hidden layers [10].
4.2.4
Learning Rate
Learning rate leads training of network with a given constant factor. It is used to adjust the weights in network. Smaller learning rate leads to better results with more accuracy. But is slows down the performance. Default learning rate selected by matlab is 0.01. The learning rate selected here for the network is 0.5. It gives same accuracy as given by value 0.1, but the number of epoch increases when learning rate is kept low.
4.2.5
Number of Epochs
In neural networks numerous iterations(epochs) of the input and output processes are carried out by adjusting internal weights and bias till desired target values and output is acceptable. Maximum numbers of epochs set for the network are 1000. Less number of epochs such as 500 may limit the training process in case of high amount of dataset.
4.3
Training Neural Network
The model learns by itself using backpropagation algorithm where, the achieved output is compared with desired output and error is calculated from which the weights of individual links to neurons are updated. Thus, for the system, 70 % of the input dataset is given as input to the network. Also, the network is provided with target output for the given input dataset for applying backpropagation algorithm with supervised learning.
4.4
Testing Neural Network
Once training of neural network with training data is done, the model needs to be tested and validated. Thus, remaining 30 % of the input dataset is used for testing. Here, the target output is not provided to the network. The network produces the output for the given dataset of testing, and then the error is calculated using the mean squared error (MSE) formula. The MSE measures the average of the squares of errors that is, the difference between the actual output and the predicted output of the model.
[email protected]
Weather Forecasting Using ANN with Error …
635
The estimated MSE of the predictor is given as follows: MSE =
1 N ∑ ðY ̂i − Yi Þ2 N I =1
where n is number of epochs, Y ̂ is the vector of n predictions (output), and Y is vector of the true values. Lesser the MSE value of the model, more accurate the results are. The network is now ready to forecast the weather. Thus a random unseen sample with all 28 input parameters from same area is given to the network, and its output is displayed as the weather forecast for the next day.
5 Performance and Results The performance of the system is calculated using mean squared error, accuracy of output and time taken for execution. We divided the data collected from various sources and divided them into six datasets. Each set varied in number of parameters, size of data. This was done to identify the most influential parameters to determine accuracy of the system. The above Table 1 shows that, the dataset number six is giving better performance (mean squared error as 0.0786 and accuracy as 90.6351 %) as compared to other datasets. It has 8 years data with 90 daily readings which makes more than 2.5 lakhs number of records. Thus, it is observed that more number of records and more number of input parameters used for training increases the performance of the system. Above Table 2 shows the testing performed on various combinations of ANN structure that are further compared with respect to their performance. Here, the fourth case with 2 hidden layers with 10 neurons each is more feasible; thus it is selected to be used in the system. The values of bias and weights connecting individual neurons in all layers are assigned randomly using a random function with a seed value. Change in seed value for random function also has a great impact on the efficiency of system. Table 1 Performance comparison with respect to various dataset Number of attributes
Number of years
Daily observations
Total records
Mean squared error
Accuracy
28 05 28 18 18 07
1.5 1.5 0.8 summer 05 20 08
08 08 08 01 01 90
4320 4320 1920 1825 7300 262800
0.6552 2.8047 0.8323 1.6999 0.2461 0.0786
54.5362 38.5942 21.2029 47.6349 67.2139 90.6351
[email protected]
636
Meera Narvekar et al.
Table 2 Performance comparison for different number of hidden layer and neurons Number of hidden layers
Number of neurons
Mean squared error
Accuracy
1 1 2 2 2
5 10 5–5 10–10 15–15
0.0850 0.0802 0.0827 0.0757 0.0759
90.3640 91.9435 90.0585 92.0249 91.3160
Following Table 3 shows the summary of results achieved using different seed values for the random function used for assigning values to weights. The most accurate seed value is chosen after comparing the efficiency with other seed values. In following Table 4, the seed value 491218382 has more accuracy and less MSE. Table 4 shows the comparison among all combinations of activation functions. The best combination which gives lowest MSE and highest accuracy is selected. This system uses tansig function at hidden layer and purelin function at output layer between the layers because of their special functionality. Learning rate leads training of network with a given constant factor. It is used to adjust the weights in network. Smaller learning rate leads to better results with more accuracy. But is slows down the performance. Table 3 Summary of results achieved using various seed values used for random function Seed value for random function
Number of epochs
Running time (hh:mm:ss)
Mean squared error
Accuracy
101 4912 456457 491218382 738745463523
10 290 215 127 11
00:00:57 01:11:11 00:19:13 00:11:54 00:01:09
12.756 0.0773 0.0779 0.0774 6.3146
31.7144 90.6023 91.4178 90.6351 37.7523
Table 4 Performance comparison for various combinations of activation functions Activation function at hidden layer
Activation function at output layer
Running time (hh:mm:ss)
Number of epochs
Mean squared error
Accuracy
Purelin Purelin Purelin Logsig Logsig Logsig Tansig Tansig Tansig
Purelin Logsig Tansig Purelin Logsig Tansig Purelin Logsig Tansig
00:00:30 00:05:17 00:05:23 00:22:00 00:21:43 00:12:56 00:19:29 00:22:34 00:15:47
5 55 59 225 225 123 210 262 169
0.1632 0.0773 0.0773 0.0757 0.0757 2.9830 0.0767 4.6776 0.0787
88.7553 91.4280 92.0695 92.0249 92.0249 62.7199 91.9002 35.5925 91.2129
[email protected]
Weather Forecasting Using ANN with Error … Table 5 Performance comparison for various learning rates
637
Learning rate
Training time
0.01 0.1 0.3 0.5 0.8 1
00:11:30 00:11:39 00:12:50 00:11:22 00:13:09 00:14:53
Fig. 2 Forecasted result for unseen dataset
Following Table 5 shows the comparison among various learning rate within the range of 0 to 1. Default learning rate is 0.01. The learning rate selected here for the network is 0.5. It gives same accuracy, mean squared error and number of epochs as given by other learning rates. Finally the model is selected with proper values for each part of it. The 70 % of the input dataset is selected as training dataset and remaining 30 % is used as testing and validation dataset. The model is later used for forecasting the weather conditions of next day when the current day’s weather is given as input. Test case of 25 unseen samples each is used to forecast the weather for day next to those days. Results denoting accuracy of the algorithm are tabulated below in Fig. 2. The parameters of false positives and true positives are used as references for comparison as well as for benchmarking accuracy levels. Accuracy levels are checked for forecasting next day weather conditions.
6 Conclusion The weather forecasting system using artificial neural network with error backpropagation algorithm has been implemented. The system forecasts next day weather by accepting input parameters of previous day. The output accuracy obtained is high. The seed value used for random function should be chosen properly to get the more accurate results, because the seed value defined for random
[email protected]
638
Meera Narvekar et al.
function is used to assign the weights to the neurons of each layer of the ANN. The initial weights used in the model have a great impact on the performance of the system. The activation function used in the middle layers also has an impact on the performance. The tansig and purelin are best suited for the forecasting problems. The learning rate directly doesn’t affect the results of the system, but it is responsible for delaying or speeding up the training time. The implemented system gives the error rate at 0.0773 and the accuracy as 90 % which is very near to accurate results. Thus, this system can be used to forecast the next day weather for Mumbai city.
7 Future Work The model can be enhanced by covering entire geographical area by collecting the historical data of many years for entire globe. The impact of global warming and other environmental factors can be incorporated to predict the weather more accurately. Inclusion of RADAR and satellite data in model input can also improvise the performance of weather forecast especially the clouds report in such images can help in rainfall prediction. Acknowledgment We express our sincere thanks to the Indian meteorological center, Mumbai, India. The center has collected the data under harsh climatic conditions and makes it available for us.
References 1. L.L. Lai, H. Braun, Q.P. Zhang, Q. Wu, Y.N. Ma, W.C. Sun, L. Yang, “Intelligent Weather Forecast”, Proceedings of the Third International Conference on Machine Learning and Cybernetics, Shanghai, 26–29 August 2004. 2. Yang Shuxia, “Power Consumption Forecast by Combining Neural Networks with Immune Algorithm”, Eighth ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing. 3. Hosein Marzi, Senior Member, IEEE, Mark Turnbull, and Elham Marzi, “Use of Neural Networks in Forecasting Financial Market”, 2008 IEEE Conference on Soft Computing in Industrial Applications (SMCia/08), June 25–27, 2008, Muroran, JAPAN. 4. Ming Gao, Wei Xu, Hongjiao Fu, Mingming Wang, Xun Liang, “A Novel Forecasting Method for Large-scale Sales Prediction Using Extreme Learning Machine”, 2014 Seventh International Joint Conference on Computational Sciences and Optimization. 5. Ch. Jyosthna Devi, B. Syam Prasad Reddy, K. Vagdhan Kumar, B. Musala Reddy, N. Raja Nayak, “ANN Approach for Weather Prediction using Back Propagation,” International Journal of Engineering Trends and Technology- Volume 3Issue1- 2012. 6. Mohsen hayati and Zahra mohebi, “Temperature Forecasting based on Neural Network Approach”, World Applied Sciences Journal 2(6): 613–620, 2007, ISSN 1818-4952, IDOSI Publications, 2007.
[email protected]
Weather Forecasting Using ANN with Error …
639
7. Tony Hall, Harold E. Brooks, Charles A. Doswell,” Precipitation Forecasting Using a Neural Network”, Weather and Forecasting, Volume 14, June 1999, pp 338–345. 8. Carlos Gershenson, “ArtificialNeural Networks for Beginners”,
[email protected]. 9. http://in.mathworks.com/help/nnet/ref/tansig.html. 10. http://in.mathworks.com/help/nnet/ref/purelin.html.
[email protected]
