Integrating Artificial Neural Networks and Cluster

Proceedings of the 2014 Industrial and Systems Engineering Research Conference Y. Guan and H. Liao, eds.

Integrating Artificial Neural Networks and Cluster Analysis to Assess Energy Efficiency of Buildings Faisal Aqlan Industrial and System Engineering Program Department of Mechanical, Civil, and Environmental Engineering University of New Haven West Haven, CT 06516 Abdulaziz Ahmed, Krishnaswami Srihari and Mohammad T. Khasawneh Department of Systems Science and Industrial Engineering State University of New York at Binghamton Binghamton, NY 13902 Abstract Energy consumption of buildings worldwide has steadily increased over the past couple of decades. Furthermore, energy performance of buildings is one of the factors that contribute to energy waste and its perennial adverse impact on the environment. This paper presents a data mining approach for assessing the heating and cooling requirements of residential buildings. The proposed approach combines Artificial Neural Networks (ANNs) and cluster analysis to assess and predict the heating and cooling energy efficiency of residential buildings. The ANNbased model uses eight input variables (i.e., relative compactness, surface area, wall area, roof area, overall height, orientation, glazing area, and glazing area distribution) to predict both the heating and cooling loads of residential buildings. Buildings are then clustered based on the output variables using the K-means clustering method. The proposed approach is used to assess and evaluate 768 diverse residential buildings based on simulated literature data. The research results showed that the proposed approach can effectively predict the heating and cooling requirements of residential buildings based on the input variables considered with a very high level of accuracy.

Keywords Energy efficiency, heating requirements, cooling requirements, data mining, neural networks, cluster analysis

1. Introduction The rapidly growing energy consumption worldwide has raised the concerns about supply difficulties in the last few decades. The global contribution from buildings towards energy consumption has steadily increased reaching 32% of total final energy consumption, as estimated by the International Energy Agency (IEA). The increase in building energy consumption is caused by several factors, including growth in population, rise in time spent inside buildings, and increasing demand for building services and comfort levels [1]. Studying the factors that affect building energy consumption can help improve the efficiency of building, reduce amount of wasted energy, and reduce the financial and environmental effects. Generally, many factors affect the energy consumption of building, such as climate, building characteristics, and the required quality of indoor environment. To address the topic of building energy consumption, different approaches have been proposed, such as linear and non-linear regression, decision tree analysis, artificial neural networks, and support vector machines. In this paper, the factors that affect energy consumption of buildings are studied considering heating and cooling energy efficiency. Therefore, the objective of this research is to use artificial neural networks and cluster analysis to build a prediction model based on the input data. 1.1 Artificial Neural Networks (ANNs) ANNs have been proven to be an efficient approach in many areas, such as aerospace, automotive, mathematics, engineering, medicine, economics, meteorology, psychology, neurology, and many others [2]. There are many learning algorithms used for ANNs. The most popular algorithm is Backpropagation (BP) neural network as shown in Figure 1. Fausett [3] stated that a BP neural network is also called Multi-Layer Perceptrons (MLPs) and is based

Aqlan, Ahmed, Srihari and Khasawneh

on gradient descent optimization approach in which the total squared error of the output signals is minimized. Also, he stated that the training process of a network by this method involves three main steps which are feed forward, backward propagation, and weights update. In the forward propagation, the signals of each input unit are fed in a forward direction to the next layer (hidden layer). The weights of that signal are then computed and the signal is forwarded to the next layer (output layer). During backward propagation step, the errors at the output units are computed be making a comparison between output target and output signals activation. The errors are computed in a form of factor called delta which is propagated backwards to the previous layer (hidden layer) and utilized afterwards to update the weights between the output layer and previous layer (hidden layer). In the same way, the factor delta corresponding to the hidden layer is computed and propagated backwards to the previous layers (input) and also used afterwards to update the weights between the hidden layer and input layer. After computing the factor delta (error), the weights and bias of both hidden and output layers are modified. Once the errors are computed, the weights of all layers are updated. The main advantage of this method over the other training algorithms (such as Perceptorn, Hebb, and Delta rule) is that the error is computed by comparing the actual output with the target.

Figure 1: Typical BP neural network 1.2 K-Means Clustering The goal of clustering is to assign a set of observations into groups (called clusters) so that the observations in each cluster are similar in some sense. One of the most common clustering methods is -means clustering. -means clustering is an algorithm mainly used for grouping different subjects based on various attributes or feature into groups where is a positive integer number [4]. The grouping is performed in a way that the distance between the data points in the same group is minimum corresponding to the centroid while the distance between the centroid of different group is maximum [4]. A -means problem can be formulated by using an optimization model that minimizes the sum of squared error over all clusters, as follows [4]: K

 

min

1 , 2 ,...,

k

k 1 xi ck

xi  k

2

(1)

where is cluster index, is the cluster set, equals to the mean of cluster , refers to total number of clusters, and xi is the individual data point. The objective of -means is to minimize the sum of squared error over all clusters. Data is normalized to eliminate the effect of the different scales of pick frequency and part age. means starts with an initial partition with clusters and assign patterns to clusters to minimize the squared error [5]. -means, which is a greedy algorithm, repeatedly adapts the clusters centroid locations to reduce the Euclidean distance [5]. The -means algorithm requires three user-specified parameters, number of clusters , distance metric, and cluster initialization [5]. -means clustering used in many applications such as unsupervised learning of neural network, Pattern recognitions, Classification analysis, Artificial intelligent, image processing, machine vision, etc. [5]. In this paper, -means clustering was used to group the building into different clusters based on the heating and cooling load. The remainder of this paper is organized as follows: Section 2 discusses a literature review on energy consumption of buildings and the use of ANN and cluster analysis to predict energy efficiency. Section 3 presents the proposed methodology. Section 4 discusses the results. Finally, Section 5 concludes and summarizes the study.

Aqlan, Ahmed, Srihari and Khasawneh

2. Related Literature In the past few years, several studies have investigated the building energy consumption and the factors affecting it. In [6], the authors studied the effects of climate change on heating and cooling load of building in China using Principal Component Analysis and Regression. In [7], the effects of the factors related to the building characteristics, such as structure type, orientation, and window-to-wall ratio to determine the cooling load for different building areas, were studied. ANN has been applied to predict building energy. Different ANN structures has been developed and used to predict building energy such as back propagation, auto associative neural network, and general regression neural network. For example, in [8], BP neural network was used to predict electricity, natural gas, water, and steam consumption for a university campus center based on weather, building occupancy, and activity. In [9], BP neural network was used to model the energy demand in China based on 16 variables which describe the characteristics of buildings. Energy consumption for HVAC (heating, ventilation, and air conditioning) system was optimized using ANN. The independent variables considered are weather and building occupancy while the dependent variables were mixed air temperature, chilled water temperature, duct static pressure and chilled water flow rate [10]. In [2], recurrent neural networks were used to predict the energy consumption of building regardless the immediate past energy consumption. In [11], Bayesian neural network was used to estimate building energy. In [12], auto-associative neural network was applied to predict the non-recorded building energy data based on feedforward network. Clustering analysis has also been used for analyzing building energy. In [13], K-means clustering was used to analyze the effects of occupant behavior on building energy. Various parameters were used to explain the occupant behavior such as climate of the city, house type, house area, and energy sources. In this study, ANN is used to predict the energy efficiency of buildings considering cooling and heating factors. Input factors to the ANN model include: Relative Compactness, Surface Area, Wall Area, Roof Area, Overall Height, Orientation, Glazing Area, and Glazing Area Distribution. Cluster analysis is then used to cluster the buildings based on the input factors.

3. Methodology 3.1 Data Description The dataset used in this research was obtained from the University of California-Irvine repository. The data was created and processed by [14]. The data was obtained by simulating 12 different building shapes in Ecotect software (www.autodesk.com). The dataset comprises 768 samples and 8 features that will be used to predict two real values responses. The eight attributes or features include: Relative Compactness, Surface Area, Wall Area, Roof Area, Overall Height, Orientation, Glazing Area, and Glazing Area Distribution. The two outputs or response variables are heating load and cooling load. The data is described in Table 1.

Variable Type

Input Variables

Output Variables

Table 1: Summary of data attributes Variable Name Number of Possible Values Relative Compactness (X1) 12 Surface Area (X2) 12 Wall Area (X3) 7 Roof Area (X4) 4 Overall Height (X5) 2 Orientation (X6) 4 Glazing Area (X7) 4 Glazing Area Distribution (X8) 6 Heating Load (Y1) 586 Cooling Load (Y2) 636

3.2 Data Mining Model The data mining model combines Artificial Neural Networks (ANN) and K-means clustering method. The model was built in SPSS Modeler® software). A snapshot of the model is shown in Figure 2. The data is pulled from an Excel sheet into a partition node which is used to partition the data into two samples (train and test) or three (train, test, and validation). The partitioned data is then input into the ANN node which generated the ANN model. The data is also clustered into groups using K-means clustering node. The final results can be presented in both table and graph formats. Once the models are built, new data points can be used for testing.

Aqlan, Ahmed, Srihari and Khasawneh

Figure 2: SPSS Modeler stream

4. Results and Analysis The 768 data points were used to build and test the ANN and K-means clustering models. The data was divided into three groups: 70% of the data for training, 15% for testing, and remaining 15% for validation. The training dataset was used to train the ANN model and the testing dataset was used to guard against potential overtraining of the model during training. The validation dataset was used to measure the generalizability of the trained model. Multilayer Perception (MLP) model is selected and the number of hidden layer is determined automatically. The stopping rule used is “Error cannot be further decreased”. The results show that 1 neuron with 4 hidden layers is selected and the accuracy of the model is 96.9% (Figure 3). Figure 4 shows the predictor (input variable) importance which indicates the relative importance of each predictor in estimating the model. Since the values are relative, the sum of the values for all predictors on the display is 1.0. Predictor importance does not relate to model accuracy. It just relates to the importance of each predictor in making a prediction, not whether or not the prediction is accurate. Figures 5-6 show the validation results for both response variables, Y1 and Y2, which indicate that the results of the ANN model using the validation set (15%) are consistent with the observed values of the two response variables.

Figure 3: Accuracy of ANN model

Figure 4: Predictor importance of ANN model

The buildings were then clustered using K-means clustering method. To study the effect of the number of clusters on the clustering quality, different numbers of clusters were considered (see Figure 7). Clustering quality is represented by Silhouette measure which provides a graphical representation of how well each item lies within its cluster [15]. The Silhouette measure scores the cluster quality as poor (0.0-0.2), fair (0.2-0.5), and good (0.5-1.0). Figure 5 shows that when the number of clusters is 2-7, the cluster quality is good and when the number of clusters is 8-15, the cluster quality is fair. Five clusters were selected to represent the energy (cooling and heating) efficiency of buildings. The clusters are labeled as: very high energy efficiency, high energy efficiency, medium energy efficiency, low energy efficiency, and very low energy efficiency. Figure 8 shows the five K-means clusters of buildings, which can be classified based on heating and cooling requirements as follows: 1) very low cooling and heating requirements  Cluster 4; 2) low cooling and heating requirements  Cluster 1; 3) medium cooling and heating requirements  Cluster 3; 4) high cooling and heating requirements  Cluster 5, and very high cooling and heating requirements  Cluster 2. The building distribution in the clusters is: 21.2% (163 buildings) in Cluster 1, 12.4% (95 buildings) in Cluster 2, 18.8% (144 buildings) in Cluster 3, 30.1% (231 buildings) in cluster 4, and 17.6% (135 buildings) in Cluster 5. The models are used to predict

Aqlan, Ahmed, Srihari and Khasawneh

the cooling and heating requirements for 10 new buildings. ANN is used to predict the values of the two response variables, cooling load and heating load. -means clustering model is then used to distribute the 10 buildings among the five clusters.

Figure 5: Predicted by observed scatter plot for Y1

Figure 6: Predicted by observed scatter plot for Y2

The data mining framework presented in this paper represents an assessment methodology for the energy efficiency of buildings. It integrates ANN and K-means clustering method and uses simulation data for 768 diverse residential buildings obtained from literature. The data consists of eight input variables and two response variables. Both ANN and K-means models can predict the energy requirements of building with high accuracies. Results presented in this study showed that the input factors (Relative Compactness, Surface Area, Wall Area, Roof Area, Overall Height, Orientation, Glazing Area, and Glazing Area Distribution) can be used to build accurate prediction models for cooling and heating efficiency of buildings. Results can provide valuable implications to engineering managers for designing and building houses. Engineering managers can focus on the input factors to control and heating and cooling requirements for the houses. As indicated by the NN model in Figure 4, the most important factors are X1, X2, X3, X4, X5, and X7. Among these factors, the most important factor is X5 Overall Height. By knowing this fact, engineers should reduce the overall height to reduce cooling and heating requirements.

Figure 7: Effect of changing number of clusters on cluster quality

Aqlan, Ahmed, Srihari and Khasawneh

Very low

Low

High

Medium

Very High

Figure 8: K-means clusters of buildings

Building

X1

Table 2: Estimating energy efficiency and clustering for 10 buildings X2 X3 X4 X5 X6 X7 X8 Y1 (ANN) Y2 (ANN) Cluster (K-means)

1

0.8

514

300

110

7

2

1

3

24.87

26.87

Cluster 3

2

0.9

514

300

110

5

3

1

4

11.33

13.67

Cluster 4

3

0.9

515

300

115

7

4

2

4

33.55

35.34

Cluster 5

4

0.8

600

260

150

6

5

3

5

33.08

34.91

Cluster 5

5

0.8

550

290

140

5

2

1

3

22.67

24.79

Cluster 3

6

0.8

510

300

160

7

3

1

2

31.17

33.05

Cluster 5

7

0.9

510

290

110

7

4

2

3

31.72

33.56

Cluster 5

8

0.9

510

280

110

6

5

3

4

31.02

32.89

Cluster 5

9

0.8

550

294

160

6

2

2

5

31.95

33.80

Cluster 5

10

0.8

550

294

130

5

3

2

1

25.11

27.10

Cluster 3

Further K-means clustering analysis was performed considering the input parameters only. Since different input parameters are measured in different scales (Table 2), the error function will be dominated by the variables in large scale [16]. To remove the effect of different scales, normalization is required before training (see Figure 9). The centroids of the five clusters are shown in Table 3.

Figure 9: Normalizing input data for cluster analysis in RapidMiner

Aqlan, Ahmed, Srihari and Khasawneh

The clustering of buildings based on normalized input data is shown in Figure 10. The percentages of buildings in the five clusters are: 22.8%, 27.4%, 11.7%, 19.5%, and 18.6%, respectively. These percentages are close to the percentages in Figure 8 which are based on the non-normalized output data. Table 3: Centroids of K-means clusters based on normalized input data Attribute Cluster 1 Cluster 2 Cluster 3 Cluster 4 Cluster 5 X1

0.157

0.157

0.644

0.644

0.644

X2

0.792

0.792

0.278

0.278

0.278

X3

0.357

0.357

0.500

0.500

0.500

X4

1.000

1.000

0.204

0.204

0.204

X5

0.000

0.000

1.000

1.000

1.000

X6

0.167

0.833

0.533

0.147

0.847

X7

0.586

0.586

0.233

0.685

0.703

X8

0.563

0.563

0.240

0.656

0.667

Figure 10: K-means clustering of building based on normalized input data

6. Conclusions In this study, ANN and K-means cluttering approaches were used to evaluate energy efficiency of buildings. It was found that both models can predict the energy consumption of buildings with a high degree of accuracy. It was also found that the most important factors affecting heating and cooling energy of buildings are: Overall Height, Surface Area, Relative Compactness, Wall Area, Roof Area, and Glazing area, respectively. These results totally agree with the results presented in [14], in which classification and regression tree (CART) method was used. The results present an essential insight into the factors that can be controlled to minimize the energy consumption of buildings. However, other factors (such as building material) can be investigated in the future. Furthermore, environmental factors such as outside temperature, wind, rain, etc., can also be studied. Building other models and comparing them with the current model can also be investigated in the future. To evaluate building performance, a decision support system based on the current work can also be developed.

Aqlan, Ahmed, Srihari and Khasawneh

References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11.

12. 13.

14. 15. 16.

Pérez-Lombard, L., Ortiz, J., and Pout, C., 2008, “A Review on Buildings Energy Consumption Information,” Energy and Buildings, 40(3), 394–398. Kreider, J., 1991, “Artificial Neural Networks Demonstration for Automated Generation of Energy Use Predictors for Commercial Buildings,” Ashrae Transactions, 97(1), 775–779. Fausett, L., 1994, “Fundamentals of Neural Networks: Architectures, Algorithms, and Applications,” Prentice-Hall. Jain, A., 2010, “Data Clustering: 50 Years Beyond K-Means,” Pattern Recognition Letters, 31(8), 651–666. Teknomo, K., 2006, “K-Means Clustering Tutorial,” Medicine, 100(4), 3. Wan, K., Li, D., Liu, D., and Lam, J., 2011, “Future Trends Of Building Heating and Cooling Loads and Energy Consumption in Different Climates,” Building and Environment, 46(1), 223–234. Schiavon, S., Lee, K., Bauman, F., and Webster, T., 2010, “Influence of Raised Floor on Zone Design Cooling Load in Commercial Buildings,” Energy and Buildings, 42(8), 1182–1191. Anstett, M. and Kreider, J., 1993, “Application of Neural Networking Models to Predict Energy Use,” Transactions-American Society of Heating Refrigerating and Air Conditioning Engineers, 99, 505–505. Yu, W., Li, B., Lei, Y., and Liu, M., 2011, “Analysis of a Residential Building Energy Consumption Demand Model,” Energies, 4(3), 475–487. Curtiss, P., Brandemuehl, M., and Kreider, J., 1994, “Energy Management in Central HVAC Plants Using Neural Networks,” ASHRAE Transactions, 100(1), 476–493, 1994. Chonan, Y., Nishida, K., and Matsumoto, T., 1996, “Great Energy Predictor Shootout II–A Bayesian Nonlinear Regression with Multiple Hyperparameters,” ASHRAE Transactions-American Society of Heating Refrigerating Airconditioning Engineering, 102(2), 405–411. Jang, K., 1997, “Measuring Retrofit Energy Savings Using Autoassociative Neural Networks,” Fuel and Energy Abstracts, 38, 276–276. Yu, Z., Fung, B., F., Haghighat, H., and Morofsky, E., 2011, “A Systematic Procedure to Study the Influence of Occupant Behavior on Building Energy Consumption,” Energy and Buildings, 43(6), 1409– 1417. Tsanas, A., and Xifara, A., 2012, “Accurate Quantitative Estimation of Energy Performance of Residential Buildings Using Statistical Machine Learning Tools,” Energy and Buildings, 49, 560–567. Rousseeuw, P., 1987, “Silhouettes: A Graphical Aid to the Interpretation and Validation of Cluster Analysis,” Journal of Computational and Applied Mathematics, 20, 53–65. Zheng, B., Yoon, S.W., and Lam, S.S., 2014, “Breast Cancer Diagnosis Based on Feature Extraction Using a Hybrid of K-Means and Support Vector Machine Algorithms,” Expert Systems with Applications, 41(4), 1476-1482.