Energy consumption control automation using

Applied Energy 165 (2016) 60–71

Contents lists available at ScienceDirect

Applied Energy journal homepage: www.elsevier.com/locate/apenergy

Energy consumption control automation using Artificial Neural Networks and adaptive algorithms: Proposal of a new methodology and case study Miriam Benedetti a,⇑, Vittorio Cesarotti b, Vito Introna b, Jacopo Serranti a a b

Department of Industrial Engineering, ‘‘Tor Vergata” University of Rome, Rome, RM, Italy Department of Enterprise Engineering, ‘‘Tor Vergata” University of Rome, Rome, RM, Italy

h i g h l i g h t s A methodology to enable energy consumption control automation is proposed. The methodology is based on the use of Artificial Neural Networks. A method to control the accuracy of the model over time is proposed. Two methods to enable automatic retraining of the network are proposed. Retraining methods are evaluated on their accuracy over time.

a r t i c l e

i n f o

Article history: Received 24 September 2015 Received in revised form 15 December 2015 Accepted 17 December 2015

Keywords: Energy model Energy management Artificial Neural Networks Adaptive algorithms

a b s t r a c t Energy consumption control in energy intensive companies is always more considered as a critical activity to continuously improve energy performance. It undoubtedly requires a huge effort in data gathering and analysis, and the amount of these data together with the scarceness of human resources devoted to Energy Management activities who could maintain and update the analyses’ output are often the main barriers to its diffusion in companies. Advanced tools such as software based on machine learning techniques are therefore the key to overcome these barriers and allow an easy but accurate control. This type of systems is able to solve complex problems obtaining reliable results over time, but not to understand when the reliability of the results is declining (a common situation considering energy using systems, often undergoing structural changes) and to automatically adapt itself using a limited amount of training data, so that a completely automatic application is not yet available and the automatic energy consumption control using intelligent systems is still a challenge. This paper presents a whole new approach to energy consumption control, proposing a methodology based on Artificial Neural Networks (ANNs) and aimed at creating an automatic energy consumption control system. First of all, three different structures of neural networks are proposed and trained using a huge amount of data. Three different performance indicators are then used to identify the most suitable structure, which is implemented to create an energy consumption control tool. In addition, considering that huge amount of data are not always available in practice, a method to identify the minimum period of data collection to obtain reliable results and the maximum period of usability is described. The general purpose of the work is to allow the automatic utilization of this kind of tools, so a method to identify a lack of accuracy in the model and two different retraining methods are proposed and compared (Mobile Training and Growing Training). The whole approach is eventually applied to the case study of a tertiary building in Rome (Italy). Ó 2015 Elsevier Ltd. All rights reserved.

1. Introduction ⇑ Corresponding author. E-mail address: [email protected] (M. Benedetti). http://dx.doi.org/10.1016/j.apenergy.2015.12.066 0306-2619/Ó 2015 Elsevier Ltd. All rights reserved.

Over the last years energy management has gained an always more prominent role in performance controlling and

M. Benedetti et al. / Applied Energy 165 (2016) 60–71

competitiveness improvement efforts of any company. The importance of energy management is enhanced by three main drivers: the growing energy costs, the always more restrictive environmental regulations (also introducing additional costs related to CO2 emissions), and finally the ‘‘green” customers’ purchasing behaviour as regards products and services [1,2]. Energy management systems are consequently gradually being introduced in many companies, together with information systems assisting data gathering activities, which are one of their main pillars [3]. In fact, measuring and controlling energy consumptions are essential activities to accomplish the objectives of reducing energy wastes and raising energy efficiency awareness, and Internet of Things technologies are nowadays fostering these activities, making a huge amount of data easily available for analyses. Thus, there is a substantial need of methodologies and tools that enable simplified energy performance measurement and analysis and consumption control [3,4]. For all these reasons, an energy consumption control system is nowadays an indispensable tool for any energy intensive company [5]. These systems are in fact able to identify a reliable energy consumption baseline and to compare actual consumption values with it, allowing energy managers to evaluate performances, identify malfunctions or energy efficiency opportunities and promptly undertake corrective actions [6]. Moreover, after an energy efficiency project is concluded, a control system can be used to compare previous to current consumptions, calculating the obtained energy saving [7]. Anyway controlling over time the energy behaviour of complex systems (i.e. systems presenting highly variable consumptions and a huge number of drivers influencing energy performance, and also requiring a high frequency data update to make the control over time effective) can be quite difficult and time consuming, and might require a huge effort [6,8,9]. Statistical definition of energy baseline is often required [10] and these kind of models have to be constantly updated and maintained, verifying their accuracy at fixed intervals (energy using systems often undergo structural changes and dramatic configuration modifications, and baseline models’ validity might fall [11]). In addition, considering multisite companies, generally characterized by a huge number of buildings or energy systems to be managed by a centralized Energy Management Office with no or partial knowledge of their actual configurations and unaware of possible ongoing modifications, the work can become even heavier. Thus, automation can play a significant role in simplifying these activities and enhancing the diffusion of energy control systems. To this purpose Artificial Neural Networks, self-learning tools that are able to approximate nonlinear relationships between input and output variables of complex systems [11], can be undoubtedly taken into account. They have already been often implemented to characterize energy systems’ behaviour, as next paragraph will demonstrate, but entail two main criticalities when it comes to energy consumption control automation: the huge amount of historical data that is required in order to create a reliable baseline model is not often available (the introduction of information systems in the energy management field is still an ongoing, incomplete activity) [11–16], and even when it is available the probability that changes occur to the system and therefore that the baseline model becomes poorly accurate after a short time is high [11,13,15–20]. In particular, the identification of a lack of accuracy in the baseline model is of particular interest in energy consumption control system, as it allows distinguishing between a real energy waste (anomalous energy behaviour) and a simple maintenance need of the control system itself. In the present paper, a new methodology to overcome both criticalities and automate energy consumption control is proposed. Artificial Neural Networks are used to allow an easy complex systems’ modelling, and performance indexes are compared to choose

61

the best network’s structure. An adaptive algorithm is proposed to allow an automatic identification of lack of accuracy in the model (retraining need) and two different retraining methods (mobile training and increasing training) are presented and compared even in terms of their reliability over time. When a lack of accuracy appears (for example after a structural change is made to the system), retraining methods are used to allow keep on effectively monitoring the efficiency of the system even without having a huge amount of data (or more reliable data referred to similar seasons) to set the baseline. Eventually, in order to make the proposed methodology as widely applicable as possible, a method to identify the minimum period of data collection to obtain reliable results and the maximum period of usability is described, considering that huge amount of data are not so often available in practice. The outcomes of this methodology (in terms of ANN’s structures, accuracy control method and retraining methods) are meant to allow an easy creation and automatic maintenance over time of energy consumption control systems so as to foster their spread even where they had not yet been implemented, and so helping the energy waste reduction, the individuation of possible energy efficiency opportunities and the raise of energy efficiency awareness in any sector. The methodology and tools proposed can be applied to any energy consuming system and used to build automatically updatable control tools such as deviations charts and CuSum charts [5,6]. The whole approach is eventually applied to the case study of a tertiary building in Rome (Italy), which represents a perfect example of a complex system whose energy consumptions are variable according to several drivers and is likely to undergo several structural changes over time. 2. Background 2.1. Approaches to the characterization and prediction of energy consumption for complex systems Different approaches to energy consumption modelling, forecast and control of complex systems are available in literature. Energy characterization models can be classified basing on the methods and tools used to develop them. First of all, it is possible to distinguish between engineering approaches and data-driven approaches [21–23], see the schematic representation given in Fig. 1. Engineering approaches use physical and thermodynamic functions to exactly derive theoretical energy consumption of processes and systems; they produce accurate results but also require detailed inputs [21,24]. They can be further divided into simplified and elaborate approaches basically according to the number of equations and variables considered in the model. Data-driven approaches, instead, use historical data to individuate the relation between energy consumption and its drivers (hereinafter called energy drivers) [21–29]. They can be structured as a ‘‘white-box” [23,25,29] if the individuated relation is mathematical and explicitly given in the form reported in Eq. (1):

E ¼ E0 þ f 1 ðx1 Þ þ f 2 ðx2 Þ þ . . . þ f n ðxn Þ

ð1Þ

(where E represents the total energy consumption, E0 the constant portion of the energy consumption, fi the portion of the energy consumption which is variable according to the energy driver xi) or as a ‘‘black-box” if the relation is not clearly explicit (they predict the value of energy consumptions basing on input variables’ values). Black-boxes are typically created using machine learning techniques such as Artificial Neural Networks (further discussed in the followings) or Support Vector Machines [26,27,29], or by the means of Decision trees [23,28].

62


Fig. 1. Schematic representation of different energy consumption characterization applied to complex systems in literature.

All of the approaches that have been here briefly recalled have been tested, analysed and their results have been compared in literature. The comparison amongst these approaches has been conducted considering the complexity of the models used, their usability, the running speed, the number and typology of inputs needed and the results’ accuracy [21]. Considering that historical data are the perfect input for a completely automated system (as a deep understanding of the system’s behaviour is not a priori necessary) and that ‘‘black-box” approaches allow to achieve high accuracy and effective results with basically no interaction with the final user of the tool (it is possible to have a high quantity of input data and to consider a lot of different hypothetical energy drivers without having to distinguish those significant from those negligible), Artificial Neural Networks have been chosen for the present work. In the following paragraph, a brief literature review regarding the use of Artificial Neural Networks in the energy management sector is given and similar applications to the one here presented are described, in order to further support the effectiveness of this choice. In addition, main differences between similar applications and methodologies and this work will be highlighted. 2.2. Application of Artificial Neural Networks to energy management Artificial Neural Networks are a technology that is nowadays widely diffused as that is able to effectively solve complex problems, also those problems that are barely definable. They learn from examples, are fault tolerant (able to handle noisy and incomplete data), are able to deal with non-linear problems and, once trained, can perform prediction and generalization at high speed [30]. They have been used in diverse applications in control, robotics, pattern recognition, forecasting, medicine, power systems, manufacturing, optimization, signal processing and social/ psychological sciences [31]. Their application to energy management purposes is nowadays starting to be quite common thanks to the proliferation of onsite energy production systems and renewable energy, as they have often been implemented to study the behaviour and control this kind of systems. Once the advantages derived from their use have been known to most of the energy management sector, they have started to be implemented to many different purposes. In particular, main applications found in literature are summarized in the Table 1, and most relevant papers associated to the single applications are specified.

In the present paper, an application of neural networks to energy consumption control is presented, providing a methodology to automatically identify lacks of accuracy in the model used to predict energy consumption (the baseline against which energy consumption values are compared over time to control purposes) and to retrain the neural network. The methodology created is based on relevant research literature on adaptive algorithms for ANNs, but is highly innovative from several point of view. In fact retraining methods, even if very rare, have already been proposed in literature [11,17–20]. Some authors presented algorithms based on a sliding windows approach (very similar to the Mobile Training here presented) [17–20], and a very few authors presented algorithms based on an accumulative approach (very similar to the Growing Training here presented) [11]. All of these approaches tend to set the amount of data needed to retrain the network in advance, and to use always that very amount of data to retrain any network. The methodology here proposed, instead, allows defining a minimum training data set to have reliable results and a proper amount of data to retrain the network, tailored to each specific application; in this way, it is possible to avoid using an excessively high amount of data to retrain the network when not necessary, making the process much more rapid and efficient. In addition, the continuous control of the accuracy allows redefining this amount of data at any time if reliability deficiencies occur; this continuous control is carried on by the means of three different performance indicators (while only one or two indicators are usually employed in literature [17–20]) so as to be more effective and reliable and avoid retraining the network when not actually necessary. As regards the comparison between the different retraining methods, the present paper employs different indexes already used in literature [11], but also introduces new criteria. In fact, in order to allow choosing the most suitable retraining method on the basis of dynamic criteria, the maximum usage period, meaning the time slot during which the networks will remain reliable giving acceptable results, has been introduced, so that results are judged considering their quality but also their stability and reliability over time.

3. Methodology The methodology proposed in this paper is summarized in Fig. 2.

63

M. Benedetti et al. / Applied Energy 165 (2016) 60–71 Table 1 Summary of literature review on the application of ANNs to energy management. Application

Description

References

Benchmarking of the energy performance of different plants, industries or nations

In this case, they are generally used to ‘‘normalize” the energy performance indicators according to external energy drivers, in order to compare very different entities in a reliable and fair way ANNs are applied to forecast the energy production of renewable power plants in particular. In fact, this kind of plants is characterized by highly variable performances with many, often unpredictable drivers. In addition, they are often implemented to forecast the amount of solar radiation, wind or water flow in a determined area so as to predict the productivity of the power plant (to control or feasibility study purposes) Accurate loads forecasting is very important in case demand side management is applied. Loads are very difficult to predict for residential complexes and urban contexts in particular, where human behaviour influences energy consumption. ANNs are successfully applied to this field ANNs have been applied to forecast the energy consumption of industrial plants, transports and buildings to energy control purposes. These systems have been modelled through the application of ANNs and obtained values have generally been compared to actual consumption data in order to identify significant deviations

[1,32–37]

Forecasts of energy produced in power plants

Forecasts of energy loads to demand side management and network balancing purposes Energy consumption forecasts for industrial plants, transports and buildings

[38–52

[12–14,53–57]

[15,16,23,58–72]

It is important to highlight that MSE (the mean value of a quadratic deviation summation) gives more weight to large errors, becoming extremely useful when large errors might occur whose negative consequences are proportionally much bigger than those of smaller ones. However, this characteristic can be seen as a disadvantage considering that the index is highly influenced by extreme, isolated values. The choice of MSE is related to the training algorithm used, as the Levenberg–Marquardt is an optimization algorithm aimed at minimizing sums of square functions. 3.2. Data preparation and selection

Fig. 2. Schematic representation of different steps of the methodology proposed.

In order to make it as general and widely applicable as possible, a choice between three different ANNs’ structures is proposed, to be conducted basing on three different performance indexes. Once the most suitable structure has been individuated, data sets have to be selected and gathered in order to train the network and then test its effectiveness and accuracy. In order to allow an application of the present methodology also when data sets are of a small entity, a method to define minimum training data and the maximum usage period is proposed. Eventually, in order to automate energy consumption control, a method to individuate losses of accuracy and two different retraining methods are presented. 3.1. Performance indexes used to accuracy evaluation Accuracy is one of the main concerns when creating an energy consumption model. In order to estimate accuracy, the analysis of only one performance index is limiting, and a more complex set of statistical parameters is needed to increase the reliability of the results. In the present work, three different performance indexes already known in literature [21–25,73] have been used to accuracy estimate purposes: the mean square error (MSE), the mean absolute percentage error (MAPE) and the coefficient of determination (R2). Each model is analysed using all of these parameters and compared with other models by the means of them.

Data used to create and test the models can be divided into two groups: input data and output data. Input data include all energy drivers influencing energy consumption of the analysed system in a defined time period and with a define time step. Output data, instead, are the different values of the energy consumptions in the same time period and with the same time step. The time step used (i.e. hourly, daily, monthly, etc.) mainly depends on the type of monitoring system adopted, while the number of observations (time period) depends on both the time step and the length of the monitoring period. Thus, the input data set is a matrix with the time steps on the rows the energy drivers on the columns (columns’ length is equal to the number of observations), while the output data set is a matrix with the time steps on the rows the different types of energy consumptions that have to be controlled on the columns (again, columns’ length is equal to the number of observations). Input–output pairs are then further divided into two groups: the training set, used to train the model, and the test set, used to test the model behaviour after the training. Before training the network, the training data set is again divided into three groups: data used to actually train the network, data used to validate the network and data used to check the network. The percentage of the random division of the training set is respectively 75% for actual training, 15% for validation and 15% for checking. An evaluation of a possible multicollinearity between the identified energy drivers is conducted before creating the model, in order to simplify it. The Pearson correlation coefficient (r) is used to identify a linear correlation between the different pairs of variables (values of the Pearson correlation coefficient higher than 0.60 and lower than 0.60 indicate that the pair of variables considered might be characterized by a linear correlation, and this possibility has to be further verified) [74].

64


3.3. Artificial neural network structures Since ANNs’ structure is generally quite tied to the particular application, and a single structure is not usually suitable for any type of model, three different models based on three different structures are here proposed. All of these three models are considered valid to energy consumption control purposes, and the choice between them (to be made on the basis of the introduced performance indexes) is left to the user. Typically, energy consumption control and forecasting models assume a layered structure called Multilayer feedforward Perceptron (MLP) [75–78]. In this structure each neuron of a single layer is linked only to neurons of the previous and the following layer. A MLP has three types of layer: one input layer where input variables are introduced, one output layer that produces the network’s output and a variable number of hidden layers used for internal calculations. For energy models, only one or two hidden layers are generally used [53] (only one hidden layer is sufficient for approximation of any measurable function [79]). Considering what previously stated, the first model proposed is the typical neural network with only one hidden layer, while the other two models are based on neural networks with a two hidden layers structure. The first structure is a very simple one: one input layer, one output layer and one hidden layer. The number of inputs is defined by the number of chosen variables. The number of neurons in the output layer is defined by the different kinds of energy consumption we want the model to control (i.e. electric consumption, gas consumption, gasoline consumption, etc.). A multilayer feed forward network with only one hidden layer is capable of approximating any function with the desired degree of accuracy, so the number of hidden neurons is not a priori defined and is linked to the desired degree of accuracy [79]. Increasing of the neurons’ number might help improving the neural network performance but it might also entail over-fitting and an increased training time. Thus, the neural network model with characterized by this structure is trained with different numbers of hidden neurons, from 2 to 200. After each training the performance indexes’ values obtained with both the training data set and the test data set are saved and plotted versus the number of hidden neurons used. Analysing the three plots obtained (one for each index) it is possible to identify the number of neurons in the hidden layer that minimizes the quadratic and percentage error and maximizes the coefficient of determination. The other two models are based on two hidden layers structure. These structures have a number of hidden neurons variable with the number of inputs. In the second structure the number of neurons in the first hidden layer is equal to the number of inputs, while the number of neurons in the second hidden layer is equal to 2 (inputs + 1) [80]. The third structure has instead a number of neurons in the first hidden layer equal to 2 (2 inputs + 1) and a number of neurons in the second hidden layer equal to (2 inputs + 1). Fig. 3 shows the three structures proposed. In the circles the number of neurons in each layer is represented (x is the number of inputs of each structure, u is the number of outputs and n is the number of neurons in the hidden layer of the first structure). When all the three structures are completely defined, each model is trained with a training data set and tested with a test data set. The training algorithm that has been used in the present work is the Levenberg–Marquardt algorithm, which is one of the most used [81]. Results of the two data sets for each model are compared to identify the best structure for the analysed case study. The following steps of the methodology will use the model based on the chosen structure.

Fig. 3. The three proposed structures.

3.4. Minimum training data set estimation In the previous step a neural network model has been created giving the availability of a huge amount of data from a long monitoring period for grant. It is a fact that nowadays most companies do not have such a huge amount of historical data and starting a monitoring campaign is usually time-consuming and resourcesconsuming, besides requiring a long time period. Thus, in order to make the methodology usable into practice, the minimum training data set allowing the creation of a sufficiently accurate model has to be defined. In order to achieve this objective, it is advisable to test the results obtained by at least two different models created using small amounts of data. The results obtained by these models can be compared by the means of the already defined three performance indexes (MSE, MAPE and R2). An additional accuracy test might be carried on by calculating the differences between consumption data estimated by the models and actual consumption data; these differences can be analysed by plotting them over time and therefore building a deviations chart. Models created with small amount of data will obviously be sufficiently accurate for a short time period, after which a new training might be required. Next step of the proposed methodology will be to determine the length of this time period.

3.5. Maximum usage period estimation As previously said, it is necessary to estimate how long a model created with the minimum training data set remains reliable and stable. Possible causes of an accuracy loss for energy models could be the following: 1. The analysed system changes its energy behaviour (structural changes). 2. A sudden change of external conditions. 3. A relatively long period after the training has passed. Models trained with small amount of data are mainly affected by the second and the third causes. Models created with the minimum training data set have therefore to be tested in different periods characterized by different durations and results should be


evaluated (MSE, MAPE and R2 can once again be used as performance indexes). 3.6. Accuracy control method A method to immediately and automatically point out when the model is no longer able to accurately reproduce the energy behaviour of the system is necessary in order to automate energy consumption control. The three performance indexes are used to evaluate the accuracy of the model and to decide whether a new training is required. These indexes have to be monitored at defined time steps and when all of them excide a previously identified threshold, the model has to be considered no longer accurate. The identification of the correct time step to control the accuracy of the model is a critical activity; in fact, the frequency of this control should be high enough to avoid late warnings, but not too high, in order not to take an isolated data anomaly for an alarming loss of efficiency. For a daily energy consumption control model, for example, a daily control might be highly affected by any hourly out-ofcontrol; a weekly control, instead, might more correctly highlight a significant loss of accuracy. In this case, if for a whole week the model does not provide accurate results, a new training will clearly be necessary. As regards the determination of the threshold values for the performance indexes, it should be carried on by considering the values obtained during the previous training. It is in fact possible to a priori establish a threshold value only for the coefficient of determination (75% is a threshold value already adopted in literature [25,28,73]; then, the threshold values for MSE and MAPE can be determined by considering the percent difference between the actual R2 value and its threshold during the previous training. In this way, the ratios of indexes’ values obtained during the training and their threshold values will be the same for all performance indexes. However, analyses presented until now do not allow any consideration about the causes of accuracy losses, but only on the moment they appear. The individuation of the causes is essential in order to take the most appropriate corrective action, as, obvi-

65

ously, a change in the energy performance and energy behaviour and a change in the energy drivers’ values entail the need of two different actions. The minimum, maximum and mean values of each energy driver in the time period when an accuracy loss has occurred and in the time period when the accuracy was acceptable can be analysed in order to identify these causes. If these three values are similar, the model might have changed its energy behaviour and the energy consumption control system might need a new data collection in order to update the model.

3.7. Retraining methods When accuracy losses occur, a new training with a new data set is required. In order to avoid interrupting energy consumption control and gathering new data, two different retraining methods are proposed: 1. Mobile Training: Having defined the minimum training data set, the new training data set is collected in a period of the same length before the moment the accuracy loss has occurred (see Fig. 4). 2. Growing Training: In the time interval between the end of the first training and the moment the accuracy loss occurs, new data are collected and added to the previous training data set, creating a larger data set for the new training (see Fig. 4). In Mobile Training data used for training are close to the moment in which the accuracy loss has occurred; thus, this method has the advantage of allowing the network to learn a situation that is very similar to the current one. However, the maximum usage period of a model retrained in this way is relatively short, and a sudden change of the external conditions might require a new training in the short term. Growing Training, instead, allows training the network with a huger amount of data; in this way the model has a longer maximum usage period.

Fig. 4. Mobile Training and Growing Training.

66


4. Case study The proposed methodology has been applied to the case study of a tertiary building in Rome (Italy). In the followings, the case study will be presented and discussed, as well as main findings and results. 4.1. The analysed building The building has a total area of 4968 m2 and 8 floors. The maximum number of occupants is 390. The building is open from 8 a.m. until 7 p.m. for five days a week (from Monday to Friday), and from 8 a.m. until 1 p.m. on Saturday. Lunch break is usually from 12 a.m. to 2:30 p.m. Since the building is located in Rome, its climatic type will be ‘‘Csa” according to Köppen-Geiger climate classification [82]. The exposition of the building is north-west, so even during summer it does not receive a strong solar radiation. The analysed building is equipped with the typical facilities of a tertiary building. An air handling unit (AHU) controls air exchanges and dehumidification in each room and in the corridors. Fan-coils are used for air conditioning, supplied by chillers during summer and by boilers (fuelled with gas) during winter. Each group of toilets is equipped with a small electric boiler for hot water production and there are pumps for water circulation in the whole building.

3. Air conditioning. 4. Water pumps and boilers. 5. Lifts. The monitoring system also registers the following data related to external conditions: 1. External temperature (°C). 2. Relative humidity (%). 3. Illuminance (lux). The main energy driver of the building’s consumption is the presence of people in the offices, which is not an easily measurable variable. In order to get an estimation of this datum as accurate as possible, the hourly electric consumption of PCs and informatics equipment has been analysed (assuming that this kind of equipment in turned on when people are in the offices, as the building has an automatic switch-off system when they are not in use). In Fig. 7, the hourly electric consumption of the informatics devices in the whole building and the estimated hourly trend of presences within the building during a working day (9th of January 2013) are shown. 4.4. Input data selection The first variables that have been taken into consideration as energy drivers are those representative of the system’s usage:

4.2. Available consumption data Building’s main energy source is electricity (natural gas is consumed only during winter). The neural network model has therefore been built to reproduce electric energy consumptions only. In Fig. 5 the whole daily electric consumption of the building from 1st January 2013 to 1st December 2014 is represented. The building is provided of a monitoring system that collects consumption data with a 15 minutes frequency. The data frequency chosen for the present analysis is hourly, which, from a first analysis of the data, has appeared to be the most significant. In Fig. 6 the electric consumption hourly profile of two different days is represented. Both days are working days, but belonging to two different seasons.

1. 2. 3. 4.

Hour of the day [0–23]. Day of the year [1–325]. Whether the hour of the day is an opening or closing hour [1; 0]. Whether the day of the year is an opening or closing day [1; 0].

4.3. Monitored variables Main energy uses monitored by the energy monitoring system are: 1. Personal computers and other informatics devices. 2. Lighting. Fig. 6. Electric energy consumption profile of two different days.

Fig. 5. Daily electric energy consumption from 1st January 2013 to 1st December 2014.

67


4. u – Relative humidity (%). 5. p – Number of people inside the building. Multicollinearity has been estimated by calculating the Pearson correlation coefficient for each pair of input variables. The results are shown in Table 2. The relation between Illuminance (I) and relative humidity (u) is characterized by a moderate high value of the coefficient, 0.655 (showing a moderate strong negative linear relation). Anyway, the scatterplot of the two variables shows a consistent dispersion of the points (Fig. 8), and the weakness of the correlation is confirmed by the low value of the R2 (0.43); thus, none of the two variables has been excluded from the analysis. Fig. 7. Hourly trend of informatics devices consumption and hourly trend of presences.

5. Number of people in the building. Subsequently, variables representing weather conditions have been considered. Energy models usually envisage the following weather variables [21]: 1. 2. 3. 4. 5.

External temperature (°C). Absolute humidity (%). Visibility (km). Illuminance (lux). Windiness (m/s).

An energy model created using an Artificial Neural Network requires a limited number of variables in order not to be perfectly able to describe training data but completely unable to predict new values. Thus, energy drivers have to be selected and only the most significant have to be considered in the energy model. The variable ‘‘Number of people in the building”, which, as already said, is the most influencing, allows to automatically take into account variables related to opening/closing conditions (variables 3 and 4 in the previous bullet point), which can be therefore excluded. In tertiary buildings, the number of people in the building is usually estimated based on the number of badges presented at the entrance by workers (automatically registered at the barriers). In the present case, this was not possible because the barriers registered the entrance of workers in several different buildings. Thus, data referring to the energy consumption of personal computers have been used to estimate it, as they were measured apart from all the other energy consumption of the building, and considered a significant indicator of workers’ presence. Furthermore, the analysis of the consumption profiles has allowed to state that the hourly profile is almost constant during the year, so that the categorical variable ‘‘Hour of the day” can be considered significant, while the categorical variable ‘‘Day of the year” has to be excluded. As regards absolute humidity, that variable was not available nor automatically registered by the measuring system of the analysed building, and the nearest meteorological station measuring it was too far from the building not to introduce a significant error in the model by using its acquired data. Thus, relative humidity was taken into account in this case, to allow somehow considering humidity and its influence on comfort conditions and therefore on energy consumption. In conclusion, the input data set is composed by the following variables 1. h – Hour of the day. 2. T – External temperature (°C). 3. I – Illuminance (lux).

4.5. Neural network structure selection The training data set is referred to the period from 1st January 2013 to 15th September 2014, while the test data set is referred to the period from 16th September 2014 to 1st December 2014. Results obtained from the models characterized by the three different structures are shown in Table 3. Results obtained implementing the three different structures are very similar one to each other. However, the third structure has the best values of performance indexes. As regards the results of the test phase, the difference between the models is even more negligible, and they can be considered substantially similar. In addition, Structures 1 and 2 present an anomaly in performance estimation as regards MSE index. In fact, these two structures, according to this index, perform better in the test phase rather than in the training phase, which is obviously not possible into practice. That is partly due to the formulation of the MSE index, which highly takes into account the presence of ‘‘big” errors not considering an ‘‘average” value of the error over the analysed time (as MAPE and R2 do), and partly to the data set, which includes many holidays (and therefore many outliers). Thus, structure number three has been chosen to create the energy consumption control model.

4.6. Minimum training data set estimation The first training data set used to train the neural network energy model and estimate the minimum training data set is composed of data referred to a period shorter than one month: from 8th January 2013 to 31st January 2013. The first week of January is not considered in the training data set because in this period the analysed building has a non-standard behaviour. The trained neural network is then tested with a new data set referred to a period of sixty days (from 1st February 2013 to 2nd April 2013). Table 4 shows the training and test results. The training results demonstrate that the model is perfectly able to reproduce the behaviour observed in the training period. Unfortunately, it has an extremely poor performance with a new data set. Less than one month of training data is therefore to be considered not sufficient to create a reliable neural network model.

Table 2 Pearson correlation coefficient of input variables.

h T I u p

h

T

I

u

p

1 0.198 0.117 0.358 0.115

0.198 1 0.468 0.494 0.141

0.117 0.468 1 0.655 0.533

0.358 0.494 0.655 1 0.346

0.115 0.141 0.533 0.346 1

68


Fig. 8. Correlation between illuminance and relative humidity.

Table 3 Results for structures 1, 2 and 3. Structure 1

MSE MAPE R2

Structure 2

Structure 3

Training

Test

Training

Test

Training

Test

106.69 kW h2 12.87% 86.30%

94.90 kW h2 14.69% 84.71%

111.47 kW h2 12.80% 85.68%

101.21 kW h2 14.81% 83.69%

91.70 kW h2 11.60% 88.22%

99.71 kW h2 14.51% 83.93%

A new training data set is chosen to train the model again. Data are referred to a longer period of approximately two months: from 8th January 2013 to 1st March 2013. The trained model is then tested with a data set collected in a period of sixty days from 2nd March 2013 to 30th April 2013. In Table 4 results obtained are shown. Training results are very similar to those previously obtained, but test results show that the new model is perfectly able to reproduce the behaviour characterizing new sets of data. Fig. 9 shows the comparison between actual consumptions and model’s forecasts. A collecting period of about two months for the training data set is therefore to be considered sufficient to create a reliable hourly energy model.

4.7. Maximum usage period estimation The neural network trained in paragraph 5.2 is now used to reproduce the building’s energy behaviour for a period of 30 days after the first test period. Results obtained are shown in Table 5. Performance indexes show a consistent decrease of model’s performance, and it is possible to conclude that a model trained with a data set of two months is able to accurately reproduce con-

sumptions for 60 days after the training period, but loses its accuracy in the following 30 days. Fig. 10 shows the comparison between actual consumptions and model’s forecasts. 4.8. Accuracy control As explained in paragraph 3, threshold values for the three indexes have to be determined; the threshold of the coefficient of determination is fixed at 75%, then the others threshold values assumed are given in Table 6. The model is created using data collected in the period from 8th January 2013 to 1st March 2013 and it is then applied to forecast energy consumption in the period from 2nd March 2013 to 25th June 2013. At the end of each week all the performance indexes are calculated. Results obtained are shown in Fig. 11, where grey circles indicate the moment in which all three performance indexes simultaneously exceed the respective thresholds. The moment the model loses its accuracy is identified in the week that starts the 22nd April 2013, thus the following week (the one that

Table 4 Results obtained with less than one month training data set (first data set) and approximately two months training data set (second data set). First data set

MSE MAPE R2

Second data set

Training

Test

Training

Test

11.40 kW h2 6.47% 96.59%

97.41 kW h2 16.83% 59.32%

12.21 kW h2 6.91% 95.82%

20.14 kW h2 9.53% 89.29%

Fig. 9. Real and estimated consumptions in the first test period.

69

M. Benedetti et al. / Applied Energy 165 (2016) 60–71 Table 5 Results obtained for 30 days after the first test period. Training MSE MAPE R2

12.21 kW h 6.91% 95.82%

Test 2

42.21 kW h2 10.10% 73.81%

Fig. 10. Real and estimated consumptions in the second test period.

Table 6 Training results and thresholds values.

MSE MAPE R2

Training

Threshold

12.21 kW h2 6.91% 95.82%

15.60 kW h2 8.83% 75%

begins with 29th April 2013) the network should be retrained with a new training data set. This result is coherent with what emerged from the estimation of the maximum usage period. In order to better understand the causes of the accuracy loss, in Table 7 the minimum, maximum and mean value of four input variables are reported for three different periods: the training period (January–February), the second test period (May) and the third test period (June). Comparing January–February to May, only one variable has the same extreme and mean values: the number of people inside the building (p). The other inputs show different extreme values and mean value. Thus, it is possible to conclude that in the second test period the model has to be updated because the external conditions are sensitively changed. Fig. 12 shows both the hourly actual electric consumption and model’s forecasts in the months of May and June 2013. In this case the model is not able to correctly reproduce consumptions, despite the boundary conditions are roughly unchanged. The analysis of performance indexes compared with the threshold values (Fig. 12) highlights a drastic drop of accuracy in June. However, it does not investigate the reasons of this reduction, which in this case is not linked to a variation of the external conditions, but to a stable change of the energy behaviour (the air conditioning system is turned on). In order to give a general idea of a possible application of the results obtained from the ANN model, the CuSum chart obtained by the two sets of data plotted in Fig. 12 is represented in Fig. 13. As it is possible to observe, the high difference between model values and actual values due to the different system’s behaviour is clearly indicated by the CuSum chart, as well as the moment the behavioural change begins.

Fig. 11. Performance indexes and their threshold values.

Table 7 Minimum, maximum and mean values of input variables. T (°C)

I

u (%)

p

January–February Min 0.6 Max 18.6 Mean 6.4

6.2 lux 662.3 lux 87.7 lux

23.7 100 75.2

0 353 166

May Min Max Mean

8.4 24.2 15.7

6.2 lux 1086.8 lux 244.2 lux

27.5 99.5 69.0

0 339 168

June Min Max Mean

11.0 27.8 18.1

6.2 lux 1063.2 lux 328.2 lux

26.8 99.7 61.8

0 307 153

Fig. 12. Real and estimated consumptions in the period May–June.

70


Fig. 13. CuSum chart in the period May–June.

Table 8 Mobile Training and Growing Training results. Mobile Training

MSE MAPE R2

Growing Training

Training

Test

Training

Test

6.89 kW h2 5.84% 96.33%

26.49 kW h2 6.77% 83.22%

7.86 kW h2 5.61% 96.72%

26.84 kW h2 6.49% 82.93%

use of different ways to estimate the number of people in the building or of the absolute humidity instead of relative humidity). Eventually, other machine learning techniques, such as support vector machines could be applied to energy consumption control and results obtained will be compared to the ones obtained by using ANNs. In this way, the most suitable tool will be unequivocally determined.

4.9. Retraining

References

Once the energy model is no longer able to reproduce the energy consumption of the system, a new data set to retrain the neural network has to be chosen. As already said, the proposed retraining methods are the Mobile Training and the Growing Training; for both methods the test period will be May 2013, but the training data are collected in two different time periods (the new training data are collected from 1st March 2013 to 30th April 2013 for Mobile Training and from 8th January 2013 to 30th April 2013 for Growing Training). Table 8 shows results obtained with Mobile Training and results obtained with Growing Training. Results obtained are extremely similar and the two retraining methods can therefore be considered equivalent in this case.

[1] Bunse K, Vodicka M, Schönsleben P, Brünlhart M, Ernst FO. Integrating energy efficiency performance in production management – gap analysis between industrial needs and scientific literature. J Clean Prod 2011;19:667–79. [2] Benedetti M, Cesarotti V, Holgado M, Introna V, Macchi M. A proposal for energy services’ classification including a product service systems perspective. Procedia CIRP 2015;30:251–6. [3] Introna V, Cesarotti V, Benedetti M, Biagiotti S, Rotunno R. Energy management maturity model: an organizational tool to foster the continuous reduction of energy consumption in companies. J Clean Prod 2014;83:108–17. [4] Benedetti M, Cesarotti V, Introna V. Improving Energy Efficiency in manufacturing systems: literature review and analysis of the impact on the energy network of consolidated practices and upcoming opportunities. In: Eissa M, editor. Energy efficiency improvements in smart grid components. InTech; 2015. p. 41–68. [5] Carbon Trust. CTG054 – Energy management – a comprehensive guide to controlling energy use. Great Britain; 2011. [6] Morvay ZK, Gvozdenac DD. Applied industrial energy and environmental management. Chichester (UK): John Wiley and Sons; 2008. [7] EVO, Efficiency Valuation Organization. International performance measurement and verification protocol; 2011. [8] Goldstein DB, Almaguer JA. Developing a suite of energy performance indicators (EnPIs) to optimize outcomes. ACEEE Summer Study on Energy Efficiency in Industry; 2013. [9] Van Gorp JC. Using key performance indicators to manage energy costs. In: Proc 27th Industrial Energy Technology Conference, New Orleans, LA; May 10– 13, 2005. [10] Swan LG, Ugursal VI. Modeling of end-use energy consumption in the residential sector: a review of modeling techniques. Renew Sustain Energy Rev 2009;13:1819–35. [11] Yang J, Rivard H, Zmeureanu R. On-line building energy prediction using adaptive artificial neural networks. Energy Build 2005;37:1250–9. [12] Chaturvedi DK, Sinha AP, Malik OP. Short term load forecast using fuzzy logic and wavelet transform integrated generalized neural network. Electr Power Energy Syst 2015;67:230–7. [13] Friedrich L, Afshari A. Short-term forecasting of the Abu Dhabi electricity load using multiple weather variables. Energy Procedia 2015;75:3014–26. [14] Macedo MNQ, Galo JJM, deAlmeida LAL, Lima ACdeC. Demand side management using artificial neural networks in a smart grid environment. Renew Sust Energy Rev 2015;41:128–33. [15] Karatasou S, Santamouris M, Geros V. Modeling and predicting building’s energy use with artificial neural networks: methods and results. Energy Build 2006;38:949–58. [16] Kumar R, Aggarwal RK, Sharma JD. Energy analysis of a building using artificial neural network: a review. Energ Build 2013;65:352–8. [17] Djukanovic MRS, Babic B, Sobajic DJ, Pao YH. A neural-based short term load forecasting using moving window procedure. Electr Power Energy Syst 1995;17(6):391–7. [18] Khotanzad A, Hwang RC, Abaye A, Maratukulam AD. An adaptive modular artificial neural network hourly load forecaster and its implementation at electric utilities. IEEE Transactions Power Syst 1995;10(3):1716–22.

5. Conclusions In the present paper, a new methodology to automate energy consumption control has been proposed. The methodology is based on the use of Artificial Neural Networks to reproduce the energy consumptions of a system, allowing the comparison between baseline and actual consumptions data and therefore the individuation of occurring energy behaviour anomalies or energy wastes and efficiency opportunities and also enabling the use of automatically updatable control tools such as deviations chart and CuSum charts [5,6]. The methodology proposed does not only allow to create an accurate model by selecting the most suitable ANN structure, but also to automate the mechanism by which a loss of accuracy of the model is individuated and its causes are investigated. In addition, two different methods to automatically retrain the network in case a permanent loss of accuracy occurs are illustrated. Next steps of this research will be to create an integrated tool that allows further testing the methodology by adding a proper consumption control phase, including the automatic creation of control charts and the verification of their effectiveness to consumption control purposes. In addition, other case studies will be conducted in order to test the methodology on even more complex systems and with different sets of variables (for example, it could be interesting to evaluate the impact on model’s accuracy of the

M. Benedetti et al. / Applied Energy 165 (2016) 60–71 [19] Mohammed O, Park D, Merchant R, Dinh T, Tong C, Azeem A, et al. Practice experience with an adaptive neural networks short-term load forecasting system. IEEE Transactions Power Syst 1995;10(1):254–65. [20] Charytoniuk W, Chen MS. Very short-term load forecasting using artificial neural networks. IEEE Trans Power Syst 2000;15(1):263–8. [21] Zhao H, Magoulès F. A review on the prediction of building energy consumption. Renew Sust Energy Rev 2012;16:3586–92. [22] Ahmad AS, Hassan MY, Abdullah MP, Rahman HA, Hussin F, Abdullah H, et al. A review on applications of ANN and SVM for building electrical energy consumption forecasting. Renew Sust Energy Rev 2014;33:102–9. [23] Tso GKF, Yau KKW. Predicting electricity energy consumption: a comparison of regression analysis, decision tree and neural networks. Energy 2007;32:1761–8. [24] Al-Homoud MS. Computer-aided building energy analysis techniques. Build Environ 2001;4:421–33. [25] Cam E, Hardalac F, Kaytez F, Taplamacioglu MC. Forecasting electricity consumption: a comparison of regression analysis neural networks and least squares support vector machines. Electr Power Energy Syst 2015;67. [26] Stathakis E, Papadimitriou T, Gogas P. Forecasting energy markets using support vector machines. Energy Econ 2014;44:135–42. [27] Magoulès F, Zhao H. Parallel support vector machines applied to the prediction of multiple buildings energy consumption. J Algorithms Comput Technol 2010;4:231–49. [28] Mingers J, Curram SP. Neural networks, decision tree induction and discriminant analysis: an empirical comparison. J Oper Res Soc 1994;45:440–50. [29] Junga HC, Kimb JS, Heoa H. Prediction of building energy consumption using an improved real coded genetic algorithm based least squares support vector machine approach. Energy Build 2015;90:76–84. [30] Kalogirou SA. Applications of artificial neural networks in energy systems. Energy Convers Manage 1999;40:1073–87. [31] Kalogirou SA. Artificial neural networks in renewable energy systems applications: a review. Renew Sust Energy Rev 2001;5:373–401. [32] Gao X, Malkawi A. A new methodology for building energy performance benchmarking: an approach based on intelligent clustering algorithm. Energy Build 2014;84:607–16. [33] Ke J, Price L, McNeil M, Khanna NZ, Zhou N. Analysis and practices of energy benchmarking for industry from the perspective of systems engineering. Energy 2013;54:32–44. [34] Antanasijevic´ D, Pocajt V, Ristic´ M, Peri-Grujic´ A. Modeling of energy consumption and related GHG (greenhouse gas) intensity and emissions in Europe using general regression neural networks. Energy 2015;84:816–24. [35] Azadeh A, Babazadeh R, Asadzadeh SM. Optimum estimation and forecasting of renewable energy consumption by artificial neural networks. Renew Sust Energy Rev 2013;27:605–12. [36] Kankal M, Akpınar A, Kömürcü MI, Özsßahin TS. Modeling and forecasting of Turkey’s energy consumption using socio-economic and demographic variables. Appl Energy 2011;88:1927–39. [37] Kialashaki A, Reisel JR. Development and validation of artificial neural network models of the energy demand in the industrial sector of the United States. Energy 2014;76:749–60. [38] Olivencia Polo FA, Ferrero Bermejo J, Gomez Fernandez JF, Crespo Marquez A. Failure mode prediction and energy forecasting of PV plants to assist dynamic maintenance tasks by ANN based models. Renew Energy 2015;81:227–38. [39] Anvari S, Taghavifar H, Khoshbakhti Saray R, Khalilarya S, Jafarmadar S. Implementation of ANN on CCHP system to predict trigeneration performance with consideration of various operative factors. Energy Convers Manage 2015;101:503–14. [40] Hossain R, Maung Than Ooa A, Shawkat Ali ABM. Historical weather data supported hybrid renewable energy forecasting using artificial neural network (ANN). Energy Procedia 2012;14:1035–40. [41] Amrouche B, Le Pivert X. Artificial neural network based daily local forecasting for global solar radiation. Appl Energy 2014;130:333–41. [42] Dorvlo ASS, Jervase JA, Al-Lawati A. Solar radiation estimation using artificial neural networks. Appl Energy 2002;71:307–19. [43] Vakili M, Sabbagh-Yazdi SR, Kalhor K, Khosrojerdi S. Using artificial neural networks for prediction of global solar radiation in Tehran considering particulate matter air pollution. Energy Procedia 2015;74:1205–12. [44] Yadav AK, Malik H, Chandel SS. Application of rapid miner in ANN based prediction of solar radiation for assessment of solar energy resource potential of 76 sites in Northwestern India. Renew Sust Energy Rev 2015;52:1093–106. [45] Ata R. Artificial neural networks applications in wind energy systems: a review. Renew Sust Energy Rev 2015;49:534–62. [46] Jung S, Kwon SD. Weighted error functions in artificial neural networks for improved wind energy potential estimation. Appl Energy 2013;111:778–90. [47] Li G, Shi J. On comparing three artificial neural networks for wind speed forecasting. Appl Energy 2010;87:2313–20. [48] Thiaw L, Sow G, Fall SS, Kasse M, Sylla E, Thioye S. A neural network based approach for wind resource and wind generators production assessment. Appl Energy 2010;87:1744–8. [49] Tu YL, Chang TJ, Chen CL, Chang YJ. Estimation of monthly wind power outputs of WECS with limited record period using artificial neural networks. Energy Convers Manage 2012;59:114–21. [50] Velázquez S, Carta JA, Matías JM. Comparison between ANNs and linear MCP algorithms in the long-term estimation of the cost per kW h produced by a

[51] [52]

[53]

[54]

[55]

[56]

[57]

[58] [59]

[60]

[61]

[62] [63]

[64]

[65]

[66]

[67]

[68] [69]

[70] [71]

[72] [73] [74] [75] [76] [77] [78] [79] [80] [81] [82]

71

wind turbine at a candidate site: a case study in the Canary Islands. Appl Energy 2011;88:3869–81. Zjavka L. Wind speed forecast correction models using polynomial neural networks. Renew Energy 2015;83:998–1006. Yeo IA, Yee JJ. A proposal for a site location planning model of environmentally friendly urban energy supply plants using an environment and energy geographical information system (E-GIS) database (DB) and an artificial neural network (ANN). Appl Energy 2014;119:99–117. Hippert HS, Pedreira CE, Souza RC. Neural networks for short-term load forecasting: a review and evaluation. IEEE Transaction Power Syst 2001;16:44–55. Wang S, Fan C, Xiao F. Development of prediction models for next-day building energy consumption and peak power demand using data mining techniques. Appl Energy 2014;127:1–10. Kialashaki A, Reisel JR. Modeling of the energy demand of the residential sector in the United States using regression models and artificial neural networks. Appl Energy 2013;108:271–80. Ghofrani M, Ghayekhloo M, Arabali A, Ghayekhloo A. A hybrid short-term load forecasting with a new input selection framework. Energy 2015;81: 777–86. Matallanas E, Castillo-Cagigal M, Gutiérrez A, Monasterio-Huelin F, CaamañoMartín E, Masa D, et al. Neural network controller for active demand-side management with PV energy in the residential sector. Appl Energy 2012;91:90–7. Siami-Irdemoosa E, Dindarloo SR. Prediction of fuel consumption of mining dump trucks: a neural networks approach. Appl Energy 2015;151:77–84. Turhan C, Kazanasmaz T, Erlalelitepe Uygun I, Evren Ekmen K, Gokcen Akkurt G. Comparative study of a building energy performance software(KEP-IYTEESS) and ANN-based building heat load estimation. Energ Build 2014;85:115–25. Mahmoud MA, Alajmi AF. Quantitative assessment of energy conservation due to public awareness campaigns using neural networks. Appl Energy 2010;87:220–8. Safa M, Samarasinghe S. Determination and modelling of energy consumption in wheat production using neural networks: a case study in Canterbury province, New Zealand. Energy 2011;36:5140–7. Ben-Nakhi AE, Mahmoud MA. Energy conservation in buildings through efficient A/C control using neural networks. Appl Energy 2002;73:5–23. Buratti C, Barbanera M, Palladino D. An original tool for checking energy performance and certification of buildings by means of Artificial Neural Networks. Appl Energy 2014;120:125–32. González PA, Zamarreño JM. Prediction of hourly energy consumption in buildings based on a feedback artificial neural network. Energy Build 2005;37:595–601. Hernandez Neto A, Sanzovo Fiorelli FA. Comparison between detailed model simulation and artificial neural network for forecasting building energy consumption. Energy Build 2008;40:2169–76. Platon R, Dehkordi VR, Martel J. Hourly prediction of a building’s electricity consumption using case-based reasoning, artificial neural networks and principal component analysis. Energy Build 2015;92:10–8. Walger da Fonseca R, Leite Didoné E, Ruttkay Pereira FO. Using artificial neural networks to predict the impact of daylighting on building final electric energy requirements.. Energy Build 2013;61:31–8. Wong SL, Wan KKW, Lam TNT. Artificial neural networks for energy analysis of office buildings with daylighting. Appl Energy 2010;87:551–7. Cortés O, Urquiza G, Hernández JA. Optimization of operating conditions for compressor performance by means of neural network inverse. Appl Energy 2009;86:2487–93. Ghorbanian K, Gholamrezaei M. An artificial neural network approach to compressor performance prediction. Appl Energy 2009;86:1210–21. Zeng C, Wu C, Zuo L, Zhang B, Hu X. Predicting energy consumption of multiproduct pipeline using artificial neural networks. Energy 2014;66: 791–8. Hill T, Marquez L, O’Connor M, Remusa W. Artificial neural network models for forecasting and decision making. Int J Forecast 1994;10:5–15. Makridakis S, Hibon M. Evaluating accuracy (or error) measures. Fointanbleau: Insead; 1995. Heiman GW. Basic statistics for the behavioral sciences. 6th ed. Cengage Learning; 2010. Evans JD. Straightforward statistics for the behavioral sciences. Brooks/Cole Pub Co; 1995. Haykin SS. Neural network a comprehensive foundation. Prentice Hall International Inc.; 1999. Horn BG, Hush DR. Progress in supervised neural networks. IEEE Signal Process Mag 1993(1):8–39. Kröse B, van der Smagt P. An introduction to neural networks. 8th ed. University of Amsterdam; 1996. Hornik K. Multilayer feedforward network are universal approximators. Neural Networks 1989;2:359–66. Ismailov VE. On the approximation by neural networks with bounded number of neurons in hidden layers. J Math Anal Appl 2014;417:963–9. Hagan MT, Demuth HB, Beale MH. Neural network design. Boston (MA): PWS Publishing; 1996. Peel MC, Finlayson BL, McMahon TA. Updated world map of the KöppenGeiger climate classification. Hydrol Earth Syst Sci 2007;11:1633–77.