Neural models for ambient temperature modelling

Neural models for ambient temperature modelling Ceravolo F. , Di Pietra B. , Pizzuti S. , Puglisi G. Energy New technologies and Environment Agency (ENEA) ‘Casaccia’ R.C. – Via Anguillarese, 301 , 00123 Rome, Italy Phone: +39-06-30484411, Fax: +39-06-30484811, Email: {francesco.ceravolo, biagio.dipietra, stefano.pizzuti, giovanni.puglisi}@casaccia.enea.it

Abstract –In this work we show how to model ambient temperature through neural models. In particular we tried feed forward and fully recurrent architectures, trained with the back-propagation and evolutionary algorithms, to estimate the monthly average temperature and compared the results to the nearest neighbor approach. Therefore, the best neural model has been tested to get hourly estimations. We compared the outcomes to a well known tool which doesn’t have such an estimation capability and results show that the proposed approach clearly outperforms the traditional ones. Keywords – ambient temperature models, feed forward neural networks, fully recurrent neural networks, evolutionary algorithms.

I. INTRODUCTION The design of efficient solar based energy production systems and eco-sustainable buildings strongly depends on simulations where the accuracy of the hourly computation of several environmental values is critical. The two main problems when facing this task are that often the only available data are monthly averages and, secondly, that only the data of few localities, mainly those with an airport, are existing. For the first problem there are methods [5] allowing to estimate reliable hourly values given the monthly ones. The second problem is slightly tougher. At present, tools like TRNSYS [16] performing this task are based on fetching data from historical databases and if a location not present in the database is requested then the data of the nearest one known (the nearest neighbor) will be given as estimation of the unknown one. This is a very simple approach, but, since climate is highly non linear and dependent on a large number of variables, often it is not true that locations close to each other have similar environmental behaviors (for example temperature). In this context we try to carry out a new approach in order to overcome this problem and get more precise estimation tools. Other approaches involve different interpolation techniques. Spatial interpolation makes it possible to estimate any meteorological characteristic (such as a maximum temperature) at locations away from those for which direct measurements exist. In this way, estimates can be made for scales up to continents and grid spatial resolution is typically in the order of several kilometres. The interpolation methods vary in complexity and accuracy, from simple Thiessen tessellation and inverse square distance [7] to more complex methods such as Truncated Gaussian Filter [15][22], kriging

and co-kriging methods [15] and variations of spline interpolation [17][14][9]. The choice of methods is partly determined by the speed of computation required and nature of the modelled phenomena; whereas methods such as Thiessen polygon methods are very fast, kriging and multivariate splines require more computational effort. Applications range from [12], where meteorological stations were spatially interpolated over the whole Europe using a multidimensional Regularized Spline with Tension (RST) [9] in order to get daily temperature profiles, to [20], where a procedure based on sets of equations to predict monthly-mean hourly values of relative humidity, ambient temperature and wind velocity for Indian locations is presented, to [13], where a method based on probability density functions to estimate daily temperature profiles is presented and applied to a few Australian localities. Moreover, in the last decades there has been a raised interest in black box methodologies, in which the mathematical approach is replaced by an empirical study based on input-output signals. In this framework the theory of Artificial Neural Networks (ANN)[8] have proven to be powerful tools to solve complex modelling problems for nonlinear systems [19]. Furthermore, since the late 70's a new branch of theory has been introduced in the evolutionary system research: the Evolutionary Algorithms (EA)[6][10]. In these approaches the algorithm structure is able to optimize a fitness function, or to optimize a winning strategy simulating some mechanisms of the genetic dynamics of chromosomes (reproduction, recombination, mutation, selection). These algorithms have been successfully applied in many technological and engineering problems, in order to solve optimization or design problems. Therefore, ANN and EA are both abstractions of natural processes. They are formulated into a computational model so that the learning power of neural networks and adaptive capabilities of evolutionary processes can be harnessed in an artificial life environment. “Adaptive learning”, as it is called, produces results that demonstrate how complex and purposeful behavior can be induced in a system by randomly varying the topology and the rules governing the system. Evolutionary algorithms can help determine optimized neural network architectures giving rise to a new branch of ANN known as Evolutionary Neural Networks [23] (ENN). It has been found [1] that, in most cases, the combinations of

evolutionary algorithms and neural nets perform equally well (in terms of accuracy) and were as accurate as hand-designed neural networks trained with Back-Propagation(BP) [3]. Moreover, using EA easily allows to train neural networks with more complex topologies, like the Fully-Recurrent (FRENN) architecture. What’s new in this work is the use of different neural models (ANN, ENN, FRENN) in order to model ambient temperatures. Experimentation has been carried out on Italian localities. The main advantage of this approach is that we have a non linear interpolation tool capable to provide a reliable daily estimation and to deal with unknown localities. This paper will be structured as follows; In Section II we will describe the approaches; In section III we will show how applying the proposed approach yields meaningful advantages over a traditional one. II. APPROACHES

A. Neural Models In our study we set up two different neural architectures, multi-layer Feed-Forward (FF) and Fully-Recurrent Neural Networks (FRNN), and used two training algorithms: a classic BP implementation applied to FF topologies, implementing the ANN paradigm, and an EA applied both to FF and FR topologies implementing the ENN and FRENN models. As BP implementation we used the MATLAB Neural Network Toolbox based on the gradient descent algorithm (traingdx) with constant momentum set to 0.9, learning rate to 0.01. The implemented EA is reported in [2]. The data, taken from [11], correspond to the monthly average temperatures of 193 Italian cities and towns, therefore the total amount of records is 193 x 12=2316. Each record has the following information available : latitude , longitude, height above sea level, month of the year and the average monthly temperature. Therefore, the topology of each neural network is made of four inputs (the three geographical coordinates and the day of the year), one output (the average monthly temperature) and four hidden neurons. Therefore the difference between the FF and the FR models is only the connection among the nodes. The FRNN models, see for example [21], can be considered as a generalization of Recurrent Neural Networks (RNN) [4]. RNN are used in situations when we have current information to give the network, but the sequence of inputs is important, and we need the neural network to somehow store a record of the prior inputs and factor them in with the current data to produce an answer. In RNN, information about past inputs is fed back into and mixed with the inputs through recurrent or feedback connections for hidden or output units. In this way, the neural network contains a memory of the past inputs via the activations.

Fig. 1. FF (up) and RNN neural topologies.

FRNN, as their name suggests, provide two-way connections between all processors in the neural network. A subset of the units is designated as the input processors, and they are assigned or clamped to the specified input values. Because of the fully connected recurrent structure, each connection has a minimum time delay of one time step. With each input pattern the activities of all neurons are updated and an output pattern is emitted.

Fig. 2. FRNN neural topology.

B. Hourly temperature estimation The neural models can be run over all the 365 days of a year providing a daily estimation of the ambient temperature. Such a result is therefore given to a module which calculates the hourly temperature according the following relation [5] : Th − T

A = 0 . 4632 cos( t * − 3 . 805 ) + 0 . 0984 cos( 2 t * − 0 . 36 ) +

(1)

+ 0 . 0168 cos( 3 t * − 0 . 822 ) + 0 . 0138 cos( 4 t * − 3 . 513 )

where Th is hourly environmental temperature, T is the daily average temperature provided by the neural model, A is the amplitude of the variation (peak to peak) and t* is t * = 2π (t − 1) 24 , where t is the hour of the day ranging from 1 to 24. The whole system, named Neural Temperature Generator (NTG), has been implemented in Simulink.

III.

EXPERIMENTATION

Experimentation concerned the monthly and hourly estimation. The whole data set has been then split into training (109 cities) and testing set (84 towns). Over the last one we performed a comparison of three neural models (ANN, ENN, FRENN) with the classical “Nearest Neighbor (NN)” approach, which in this case consisted in matching the temperature of each town of the testing set, theoretically unknown, with the temperature of nearest city of the training set. The experimental training set-up of the neural models consisted of 30000 performance requests. C. Monthly estimation In table 1 we report the testing accuracy as the mean and maximum absolute error in Celsius degrees of each of the four methodologies. Moreover, in figure 3 we show a graph comparing the behavior of the best neural model (FRENN) to the NN approach over eight different towns.

climatic areas and for different seasons) and ensembling methods combining different techniques, like Radial Basis Function networks (RBF) and Support Vector Machines (SVM), might get better estimations. Lastly, since the method is general, working with a higher sampling, for instance daily averages rather than monthly, will enhance the accuracy of the models. D. Hourly estimation In table 2 we compare the hourly estimation, referring to the city of Rome, of the overall computation provided by (1) using the best neural module (FRENN) as daily temperature estimator (Neural Temperature Generator : NTG) to that obtained by the TRNSYS tool [16] which uses a NN estimation approach. Table 2. Accuracy comparison (absolute error in Celsius degrees) experimented for the city of Rome with respect to real values.

Table 1. Testing absolute error accuracy (°C).

ANN 1.2 6.49

Average Maximum

ENN 1.5 9.29

FRENN 1.15 5.34

NN 1.58 7.9

TRNSYS 3.49 14.55

Average Maximum

NTG 2.78 9.71

Difference -0.71 (-20%) -4.84 (-33%)

Difference -0.43 (-27%) -2.56 (-32%)

30.00

25.00

temperature (C)

20.00

Figure 4 : NTG hourly ambient temperature for the city of Rome (2000 hours – 1 year)

15.00

10.00

25

20

5.00

ENN(fc)

115

112

109

106

97

103

94

100

91

88

85

82

79

76

73

70

67

64

61

58

55

52

49

46

43

40

37

34

31

28

25

22

19

16

7

13

4

10

1

Real

NN

Figure 3 : techniques comparison (testing set)

Temperature [°C]

15

0.00

10

5

0

-5 0

200

400

600

800

1000

1200

1400

1600

1800

2000

Time[h]

Figure 5 : TRNSYS hourly ambient temperature for the city of Rome (2000 hours –1 year) 20

45

18

40

16

35

14

30 12

25 10

20 8

15 6

10 4

5 2

0 1 215 429 643 857 1071 1285 1499 1713 1927 2141 2355 2569 2783 2997 3211 3425 3639 3853 4067 4281 4495 4709 4923 5137 5351 5565 5779 5993 6207 6421 6635 6849 7063 7277 7491 7705 7919 8133 8347 8561

0 1 48 95 142 189 236 283 330 377 424 471 518 565 612 659 706 753 800 847 894 941 988 1035 1082 1129 1176 1223 1270 1317 1364 1411 1458 1505 1552 1599 1646 1693 1740 1787 1834 1881 1928 1975

Experimentation show promising results. In particular the model which performs better is the one with a fully-recurrent architecture trained with EA which clearly outperforms (about a 30% improvement) the outcome of the nearest neighbor approach. Nevertheless, we can see the maximum error is still pretty high. This is mainly due to the fact that some important environmental data, like direction and speed of the wind, humidity, etc., are missing and such a lack of information severely affects the temperature estimation for those localities where this missing information is relevant. Such results might be improved in several ways. Firstly, since ambient temperature is highly non-linear and dependent on many variables, the data set size is important. Therefore, larger data sets are likely to improve the results. Secondly, using a modular approach (different models for different

-5

Figure 6 : real hourly ambient temperature for the city of Rome (2000 hours - 1 year)

As we can see from the graphs the signal we get with the proposed methodology is pretty clear and smooth. This is due to the fact that we are not taking care of the cloudy conditions, while TRNSYS does with a random noise. IV.

CONCLUSION

In this paper we proposed a new approach based on neural networks in order to model ambient temperature given the three geographical coordinates of a locality and the day of the year. We trained the models on several Italian localities and compared the outcome of the testing set to the nearest neighbor algorithm and the TRNSYS tool on two different time scales : monthly and hourly. Results showed a remarkable improvement, about 20%-27% on the average error and about 33% on the maximum. The advantages of the proposed method are mainly two : the possibility to reliably estimate the temperature of unknown localities and the applicability to different time scales. Although results are encouraging, improvements can be done. For instance, using larger data set with a small sampling, using modular models with respect to geographical areas and seasons, applying different models (like RBF and SVM) and ensembling techniques. REFERENCES [1]

[2]

[3]

[4] [5] [6] [7] [8] [9] [10] [11] [12] [13]

Alander, J. T., “An indexed bibliography of genetic algorithms and neural networks”, Technical Report 94-1-NN, University of Vaasa, Department of Information Technology and Production Economics, 1998 Annunziato M. , Pizzuti S. , “Adaptive Parameterisation of Evolutionary Algorithms and Chaotic Population Dynamics”, International Journal of Knowledge-Based Intelligent Engineering Systems, 170-177, ISSN: 1327-2314, 2002 Cant-Paz, E. and Kamath, C. , “An empirical comparison of combinations of evolutionary algorithms and neural networks for classification problems”, IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics, 915-927, 2005. Elman J. L. , “Finding structure in time”, Cognitive science, Vol.14, pp. 179-211, 1990 Erbs D.G., Klein S.A., Beckman W.A., “Estimation of degree-days and ambient temperature bin data from monthly-average temperatures”, Ashrae Journal, 60-65, June 1983 Goldberg D. E., “Genetic Algorithms in Search, Optimisation and Machine Learning”, Addison Wesley, USA, 1989 Goovaerts P., “Geostatistical approaches for incorporating elevation into the spatial interpolation of rainfall”. Journal of Hydrology 228, pp. 113–129, 2000 Haykin S. “Neural Networks, a comprehensive foundation”. Prentice Hall. New Jersey, 2nd edition. ISBN 0-13-273350-1, 1999. Hofierka J., Parajka J., Mitasova H. and Mitas L., “Multivariate interpolation of precipitation using regularized spline with tension”, Transactions in GIS 6, pp. 135–150, 2002 Holland J. H. , “Adaption in Natural and Artificial Systems”, MIT Press, Cambridge/MA, USA, 1975 http://clisun.casaccia.enea.it/Pagine/Index.htm. Huld T.A., Šúri M., Dunlop E.D., Micale F., “Estimating average daytime and daily temperature profiles within Europe”. Environmental Modelling & Software, 21, 12, 1650-166, 2006 Hunter A.J., “A Climatic Model for the Prediction of Percentile Statistics for Ambient Temperature”. American Meteorological Society Journal, Volume 109, Issue 4 (April 1981), pp. 722–728

[14] Jeffrey S.J., Carter J.O., Moodie K.B. and Beswick A.R., “Using spatial interpolation to construct a comprehensive archive of Australian climate data”. Environmental Modelling and Software 16, pp. 309–330, 2001. [15] Jolly W.M., Graham J.M., Michaelis A., Nemani R. and Running S.W., “A flexible, integrated system for generating meteorological surfaces derived from point sources across multiple geographic scales”. Environmental Modelling and Software 20, pp. 873–882, 2005 [16] Klein S.A., Beckman W.A., Mitchell J.W., Duffie J.A., Duffie N.A., Freeman T.L., Mitchell J.C., Braun J.E., Evans B.L., Kummer J.P., Urban R.E., Fiksel A., Thornton J.W., Blair N.J., Williams P.M., Bradley D.E., “TRNSYS, a Transient System Simulation Program”, Solar Energy Lab, Univ. of Wisconsin-Madison, 2000 [17] Mitas L. and Mitasova H., “Spatial interpolation”. In: P. Longley, M.F. Goodchild, D.J. Maguire and D.W. Rhind, Editors, Geographical Information Systems: Principles and Applications, Wiley, pp. 481–492, 1999. [18] Narendra K.S., Parthasarathy K., “Identification and control of dynamical systems using neural networks”. IEEE Trans. Neur. Netw., Vol. 1, No. 1, pp. 4-27, 1990. [19] Nikravesh M., Farell A.E., Stanford T.G., “Model identification of non linear time variant processes via artificial neural network”. Computes and Chemical Engineering, Vol. 20, No. 11, pp. 1277-1290, 1996. [20] Parishwad G.V., Bhardwaj R.K., Nema V.K. “Prediction of monthlymean hourly relative humidity, ambient temperature, and wind velocity for India”. Renewable Energy, Volume 13, Number 3, March 1998 , pp. 363-380(18) [21] Reininger K.K.H., Wolf D., Wüst H. , “Fully Recurrent Neural Networks for Phoneme Based Speech Recognition”, http://citeseer.ist.psu.edu/226521.html [22] Thornton P.E., Running S.W. and White M.A., “Generating surfaces of daily meteorological variables over large regions of complex terrain”. Journal of Hydrology 190, pp. 214–251, 1997. [23] Yao, X., “Evolving Artificial Neural Networks”, Proceedings of the IEEE, 87(9), 1423-1447, 1999