Comparing Methods: Using Multilevel Modelling and ...

Comparing Methods: Using Multilevel Modelling and Artificial Neural Networks in the Prediction of House Prices based on property, location and neighbourhood characteristics Yingyu Feng1 and Kelvyn Jones2 School of Geographical Sciences, University of Bristol

Abstract Two advanced modelling approaches, Multi-Level Models and Artificial Neural Networks are employed to model house prices. These approaches and the standard Hedonic Price Model are compared in terms of predictive accuracy, capability to capture location information, and their explanatory power. These models are applied to house prices in the Greater Bristol area, 2001-2013 using secondary data from the Land Registry, the Population Census and Neighbourhood Statistics so that these models could be applied nationally. The results indicate that MLM offers good predictive accuracy with high explanatory power, especially if neighbourhood effects are explored at multiple spatial scales. KEYWORDS: House Prices, Multilevel Modelling, Artificial Neural Networks, Predictive Accuracy

Draft not to be quoted without permission

1 [email protected] 2 [email protected]

1

Introduction

“Location, location, location” is frequently used by real estate agents when marketing residential properties. Properties in good locations generally command a premium. Indeed, numerous studies have demonstrated the importance of location and the surrounding neighbourhoods of properties in price determination, whether through urban economic theory (Mills, 1972; Muth, 1969; Meyer, Kain, and Wohl, 1965; Alonso, 1964; Wingo, 1961), spatial econometrics (Osland 2010), multilevel studies (Orford 2002; Jones & Bullen 1993), or geographically weighted regression (Lu et al., 2014; Bitter et al., 2006; Fotheringham et al., 1998). However, some neighbourhood attributes may be unobservable, and it remains an open question of how best to capture locational information in house price modelling. Here we are concerned about how model choices affect the predictive accuracy – the degree to which it is possible to make accurate predictions of price based on the location of properties with sparse information on the properties. The standard approach to estimating residential house price uses the Hedonic Pricing Model (HPM) (Malpezzi, 2001; Adair et al., 1996; Powe et al., 1995; Li & Brown, 1980). HPM describes house price as a function of the physical characteristics of the houses and the characteristics of the surrounding neighbourhoods. However, this approach ignores the hierarchical structure of the housing market that houses are located in neighbourhoods, resulting in incorrectly estimated standard errors for the parameters (Raudenbush and Bryk, 2002). This spatially invariant regression approach implies that there are no contextual effects of the neighbourhoods on house prices, which effectively denies geography (Foster, 1991). Moreover, the functional form of HPM also needs to be pre-specified, but there is little theoretical or empirical guidance on the correct functional form. Relationships can be non-linear and involve complex interactions. If these relationships are not fully and properly specified in the HPM model, the estimated coefficients will be unreliable and biased (Peterson & Flanagan, 2009). We propose to evaluate two alternative modelling approaches to the standard HPM. Multilevel models (Goldstein, 1999) are designed to analyse data with a hierarchical structure – in this case properties nested within neighbourhoods. In particular it not only allows the correct estimation of the effect of observed neighbourhood characteristics, but also reveals the unobserved ‘desirability’ of neighbourhoods where properties are located, that is it models unobservable or latent variables as well as measured one. MLM regards houses (micro-level units) as nested in neighbourhoods (macro-level units) and relaxes the assumption of independent observations for individual properties. It can properly account for the contextual effect of neighbourhood and models the ‘within-neighbourhood’ as well as the ‘between-neighbourhood’ relations simultaneously within a single overall equation. Artificial Neural Network (ANN) is the second proposed approach. It is capable of handling potential non-linear price functions and interaction effects between predictor variables. In previous work, ANN has been found to be very effective in revealing non-linear relationships and to be more robust to ‘noise’ than HPM (Peterson & Flanagan 2009). The principal objective of this paper is to present these two advanced quantitative approaches and compare their performance in terms of model fit, predictive accuracy and explanatory power. Previous empirical studies using MLM or ANN tend to focus on the comparison of each approach against HPM. Indeed, there is no published work to date comparing ANN with MLM. This is the case conceptually and in terms of practical adequacy. In particular, this paper explores a range of realistic scenarios as to how the effects of locations can be captured and compares the performance of all three approaches under each scenario. The study area is Greater Bristol, and the price data is drawn from the Land Registry of England and Wales, consisting of 65,302 historical sales during 2001-2013. The use of a much larger dataset than previous publications is another important contribution of this study. The rest of the paper is organized as follows. Section 2 provides a schematic specification of each modelling approach and a brief summary of their use in real estate price modelling. Section 3 considers how the models are operationalised, in terms of study area, available data, scenarios to capture locational information, and performance measures for competing models. The results are presented in section 4 with the conclusions in section 5. 2

Specification of HPM, MLM and ANN and their applications to house price modelling

In this section a brief introduction to each methodology is first presented and then a review of previous

studies applying the approach is provided. Finally, the gap in the existing literatures is identified and the contributions of this study are presented at the end of this section. 2.1

The Hedonic Price Model

The Hedonic Price Model (HPM) was first introduced by Lancaster (1964) and formalised by Rosen (1974). However, Rosen did not provide any guidance as to the exact functional form of the equation. Traditional HPM regresses the house prices on a range of its constituent attributes (e.g. number of bedrooms, type of house, or distance to the city centre and so on) and is usually calibrated using the ordinary least squares (OLS) method. The resultant regression coefficients are estimates of the contribution of each element to the price, known as the ‘hedonic’ prices for each attribute (Rosen, 1974). It has become a popular approach to modelling house price as it is easy to apply and can reveal the comparative size of effects of various property and neighbourhood characteristics on house prices. Its functional form is typically specified as: 𝒚𝒊 = 𝛃𝟎 + ∑𝒎 𝒋=𝟏 𝛃𝒋 𝒙𝒋𝒊 + 𝒆𝒊

( 𝒋 = 𝟏, … 𝒎; 𝒊 = 𝟏, … , 𝒏)

(1)

where 𝑦𝑖 is the observed sale price of house i, 𝑥𝑗𝑖 is jth explanatory variable for house i, β0 is the intercept, β𝑗 is the regression coefficient for 𝑥𝑗𝑖 and 𝑒𝑖 is the random error of house i, which is typically assumed to come from a normal distribution and is summarised in a variance term which assesses the degree of unexplained price variation between houses. This random term is in effect an unobservable variable at the house level- a positive value, suggesting a premium is being paid for a desirable house given its observed characteristics; while a negative value suggests a discounted house. The total number of observations is signified by 𝑛 while 𝑚 is the number of explanatory variables. Generally a strictly additive linear form is assumed, but often it is hoped to capture non-linear multiplicative differences by modelling the logarithm of the response. Even when logarithm transformation is used, or polynomial terms of predictors are included, such models are still intrinsically linear (Greene, 2011 p165). There are numerous studies using HPM to model house price (e.g. Kong et al. 2007; Anselin & Le Gallo 2006; Adair et al. 1996; Li & Brown 1980 amongst others) and a number of reviews have also been conducted including Freeman(2011), Chau et al (2004), Sirmans et al. (2005), and Malpezzi (2002). However, this approach does not recognize that property and neighbourhood characteristics are in fact measured at different scales. Moreover, all the unexplained variation is assumed to be between individual houses. The number of houses is not distinguished from the number of neighbourhoods (there are usually considerably fewer of the latter than the former), the standard errors of the measured neighbourhood variables will have mis-estimated precision and will be too low. There is a considerably increased chance of committing a type 1 error – finding significant relations that are really not different from chance ones. Multilevel models are designed to analyse such hierarchies and do not suffer these problems. The HPM’s parametric nature is also questioned and semi- or non-parametric models have been proposed, including ANN as one of these non-parametric approaches (Bin, 2004; Páez & Scott, 2004; Kauko et al., 2002; Mason & Quigley, 1996; McGreal, Adair, McBurney, & Patterson, 1998; Rossini, 1997). We now examine the MLM and ANN in more detail. 2.2 2.2.1

Multilevel Model Model specification

MLM combines the micro-level equation, describing the within-neighbourhood between-house relationship for individual properties and the macro-level equation, the between-neighbourhood relationship, into one model. The micro-equation with one house-level explanatory variable 𝑥1𝑖𝑗 can be expressed as: 𝒚𝒊𝒋 = 𝜷𝟎𝒋 + 𝜷𝟏𝒋 𝒙𝟏𝒊𝒋 + 𝐞𝒊𝒋

(2)

Where 𝑦𝑖𝑗 and 𝑥1𝑖𝑗 represent the house price and house-type for level-1 house 𝑖 in level-2 neighbourhood 𝑗, respectively. The random residuals e𝑖𝑗 represent the between-house price deviation withinneighbourhood 𝑗, which are assumed to be mutually independent and follow a normal distribution with a mean of 0 and a constant variance 𝜎𝑒2 , that is 𝑒𝑖𝑗 ~ 𝑁(0, 𝜎𝑒2 ). Unlike the standard HPM there are a set

of intercepts and ‘slopes’3 which are index by the subscript 𝑗. Consequently, if 𝑥1𝑖𝑗 is a binary variable with 1 being detached house and 0 being non-detached property, 𝛽0𝑗 is the mean price of a non-detached property in a neighbourhood and 𝛽1𝑗 is the mean differential for detached properties in a neighbourhood. These intercepts and slopes are specified as the responses in two macro-models: Intercept: 𝜷𝟎𝒋 = 𝜷𝟎 + 𝒖𝟎𝒋

(3)

𝜷𝟏𝒋 = 𝜷𝟏 + 𝒖𝟏𝒋

(4)

Slope:

where 𝛽0 and 𝛽1are the mean price of non-detached houses and the mean price differentials for detached houses averaged across all neighbourhoods, respectively. The terms 𝑢0𝑗 and 𝑢1𝑗 represent the unexplained price differentials of neighbourhood j in term of intercept and slope after taking into account house types. Substituting the macro-models into micro-model and re-arranging the order results in the combined equation: 𝒚𝒊𝒋 = 𝜷𝟎 + 𝜷𝟏 𝒙𝒊𝒋 + (𝒖𝟎𝒋 + 𝒖𝟏𝒋 𝒙𝒊𝒋 + 𝐞𝒊𝒋 )

(5)

The first part of the combined equation is called the fixed part representing the means and the second part (within the bracket) is called the random part. It is assumed that these two neighbourhood-level residuals 𝑢0𝑗 and 𝑢1𝑗 have a mean 0 and a covariance 𝜎𝑢01 . The distributional assumptions are denoted as 𝑢0𝑗 𝜎2 [𝑢 ] ~𝑁(0, [ 𝑢0 1𝑗 𝜎

𝑢01

2 𝜎𝑢1

])

2 so that 𝜎𝑢0 , the variance of the random intercepts summarises the extent to which the non-detached 2 house price differential varies from place while 𝜎𝑢1 , the variance of the random ‘slopes’, summarises the extent to which the differentials for detached houses differ between neighbourhoods. The covariance assesses the extent to which a neighbourhood that is expensive for non-detached properties is differentially expensive for detached properties. These random effects are unmeasured or latent variables representing the differential ‘desirability’ of a neighbourhood for non-detached and detached property – the premium or discount people are willing to pay to live in that location. This allows the unobserved neighbourhood characteristics to play a role ‘behind the scenes’ in the effects of the observed variables on house prices (Snijders & Bosker, 1999). We can estimate these latent effects in the multilevel model and use them in our predictions.

The MLM can readily decompose the unexplained variance to micro house-level and macro neighbourhood-level. In the above model, e𝑖𝑗 is a differential unexplained effect for an individual property and represents whether it is relatively expensive or cheap given its type and neighbourhood. These unexplained differences are summarised in the variance 𝜎e 2 which now captures the withinneighbourhood between house variations. In the random intercepts and slopes model, the unexplained price variations between neighbourhoods is a quadratic function of the level-1variable 𝑥1𝑖𝑗 , represented 2 by 𝜎𝑢20 + 2𝜎𝑢01 𝑥1𝑖𝑗 + 𝜎𝑢21 𝑥1𝑖𝑗 . The Variance Partition Coefficient (𝑉𝑃𝐶) can be used to assess the proportion of the total unexplained variance that is accounted for by the neighbourhoods, given the property characteristics 𝑥1𝑖𝑗 , calculated as

2 𝜎𝑢20 + 2𝜎𝑢01 𝑥1𝑖𝑗 +𝜎𝑢21 𝑥1𝑖𝑗 2 2 2 𝜎𝑢0 + 2𝜎𝑢01 𝑥1𝑖𝑗 +𝜎𝑢1 𝑥1𝑖𝑗 +𝜎𝑒2

. It can also be interpreted as the

similarities of the prices of two individual properties of the same type randomly drawn from the same neighbourhood, which is related to “Intraclass Correlation Coefficient (ICC)” (Snijders & Bosker, 1999). It is also possible to include measured neighbourhood characteristics in the model. The micro level model remains the same, but the neighbourhood intercept 𝛽0𝑗 and ‘slope’ 𝛽1𝑗 of the macro-models are now given by: Intercept: 𝜷𝟎𝒋 = 𝜷𝟎 + 𝜷𝟐 𝒁𝟏𝒋 + 𝒖𝟎𝒋

3

Given the categorical nature of the level 1 predictor these are really differentials rather than slopes but the latter is widely used.

(6)

Slope:

𝜷𝟏𝒋 = 𝜷𝟏 + 𝜷𝟑 𝒁𝟏𝒋 + 𝒖𝟏𝒋

(7)

where 𝑍1𝑗 is an observed level-2 variable for neighbourhood 𝑗, for example the crime rate. Substituting the macro-models into micro-model and re-arranging the order results in the combined equation: 𝒚𝒊𝒋 = 𝜷𝟎 + 𝜷𝟏 𝒙𝟏𝒊𝒋 + 𝜷𝟐 𝒁𝟏𝒋 + 𝜷𝟑 𝒁𝟏𝒋 𝒙𝟏𝒊𝒋 + (𝒖𝟎𝒋 + 𝒖𝟏𝒋 𝒙𝟏𝒊𝒋 + 𝐞𝒊𝒋 )

(8)

where 𝛽0 and 𝛽1 is the mean price of non-detached houses and mean differential premium for detached house in a neighbourhood where crime rate is zero, respectively. The 𝛽2 and 𝛽3 terms represent the price differentials for non-detached and detached property with a unit increase in crime rate, respectively. The fixed part therefore has two additional terms: a contextual effect of crime (𝛽2 ) and a cross-level interaction effect 𝛽3 between the level-1 variable 𝑥1𝑖𝑗 and level-2 variable 𝑍1𝑗 . Although the random part of the combined equation does not change from the earlier equation, 𝑢0𝑗 and 𝑢1𝑗 now have a different meaning – they are the unexplained differences between neighbourhoods taking account of neighbourhood composition in terms of house type and now additionally, its crime rate. We would anticipate the associated variances becoming smaller as an important neighbourhood variable is included. The model can readily be extended by including more house predictors in the micro model and more contextual variables in the macro models. The local estimates of residuals 𝑢0𝑗 and 𝑢1𝑗 are precision-weighted and potentially “shrunken” towards the overall mean relationship for all neighbourhoods (Jones & Bullen, 1994). The degree of the shrinkage depends on the number of observations in a neighbourhood. If the number is small, the estimates are “pulled” towards the grand mean price of the all the houses across all neighbourhoods, so that the house prices in a small neighbourhood can “borrow the strength” from what is going on generally. If the number of sales in a neighbourhood is large, there would be little shrinkage. The MLM estimates of the neighbourhood effects are reliable and can be used to improve predictions of prices by including these robust estimates of the un-modelled effects. Moreover, the MLM specification allows proper estimation of the standard errors of parameters for the neighbourhood and property characteristics as the structure allows for the dependency between the observations within the same neighbourhood (which is measured by the VPC). This standard 2-level model can be easily extended to more than two levels, or used for non-hierarchical data where two types of higher-level units overlap with each other. For example, a house may belong to one higher-level units (e.g. a neighbourhood) but also belong to another type of higher-level unit (e.g. a school district). This is called a cross-classified model where the boundaries of the neighbourhoods and school districts are not nested, but overlap with each other. 2.2.2

Review of previous work applying MLM

There is only limited work employing MLM in house prices research, while most applications are in education (Harris, Johnston, & Burgess, 2007; Goldstein, 1999; Haurin & Brasington, 1996; Aitkin & Longford, 1986); public health (Gee 2008; Diez-roux 2000) and epidemiology (Merlo et al., 2006; Congdon, 2003; Bryk & Raudenbush, 1992). Jones (1991) is the first paper to employ a multilevel approach to house price research. He applied a 3-level model to a small sample of 918 house sales in Southampton over almost a decade where houses are nested in time periods and then in districts. He demonstrated that the “shrunken” estimators in MLM make efficient use of information between districts and represent a considerable improvement over single-level fixed-effect models estimated by OLS, namely HPM. This 3-level design allows the relationships between house prices and the predictors to vary in both time and geographical space. In his subsequent joint work with Bullen (Jones & Bullen 1994; 1993), they compared the fixed- and random-coefficient models for London house prices and demonstrated that MLM implicitly models autocorrelation within spatial data. This new approach opened up a new arena for house price research. Thus Orford (2000) and Orford (2002) applied the MLM approach to samples of Cardiff house sales in 1995 and illustrated how this approach allowed various predictors to enter into the pricing model at different spatial scales. In his 2002 work, he specified a four-level model, which individual properties are viewed as nested in streets, then in turn nested in housing condition survey areas (HCS) and finally nested in the communities, or wards, the administrative units by the local council. Other works that examined the effects of house and

various neighbourhood characteristics on house prices under the multilevel framework include the study of the effect of median travel time to work (Brown & Uyar 2004), transportation access (Habib, and Miller 2008), public facilities for leisure and sports (Lee, 2010), accessibility (Shin et al. 2011), air quality and urban noise (Chasco & Gallo 2012), locational neighbourhood variables (Brunauer et al. 2013), and school characteristics (Uyar & Brown 2007; Goodman & Thibodeau 2003; Goodman & Thibodeau 1998). A key requirement of MLM approach is that the levels have to be defined a priori. However, the definition or specification of the higher-level units has never reached a consensus. They can be either existing geographical units used for other purpose (for example, postcodes, districts, census areas, administrative areas and school districts) or derived housing submarkets whose boundaries are delineated based on certain criteria. Using MLM as a market delineation tool to segment housing market into relatively homogenous submarkets was featured in Goodman and Thibodeau's (1998) paper. They then compared the predictive accuracy of the MLM specified using ‘derived’ submarkets (90 submarkets) against the model using ‘imposed’ submarkets constructed from zip code (55 zip code submarkets) or the census area (82 census submarkets) using 28,000 single-family sales during 19951997 (Goodman & Thibodeau 2003) and found the former provides more accurate predictions. Leishman et al. (2013) also compared two different definition of submarkets in calibrating MLM models. One is the spatial submarkets defined by Real Estate Institute of Western Australia, and the other defined by postcodes, which is finer grained than the former. They analysed 60,699 house sales in Perth in Australia and found that the one using postcodes achieving the most accurate predictions. Most studies have analysed strictly hierarchical structures in which the lower level unit is nested in one and only one higher-level unit. However, Uyar & Brown (2007) specified a cross-classified model to analyse the effect of neighbourhood affluence and school achievement scores on house prices, where houses are viewed as nested in neighbourhoods (defined as census block groups) and school zones at the same time. They found that both the neighbourhood and school zones account for significant variation in house prices. MLM has also been used to handle spatiotemporal heterogeneity in house prices. For example, in Habib & Miller's (2008) work, individual houses were treated as nested within neighbourhoods, and then nested in years. MLM specified in this way effectively models the spatiotemporal variations in house prices and its explanatory power is enhanced. Leishman (2009) also used MLM method to identify temporal trend of housing market of Glasgow in two non-consecutive years (2002 and 2006) and suggested that MLM was a useful approach for both segmenting submarkets and recognizing temporal changes in the submarket delimitation. Combinations of MLM with other statistical models also emerging. For example, a hybrid model, multilevel structured additive regression (STAR) model was used in the study by Brunauer et al.(2013) to analyse the effects of structural and locational factors on house prices. Another application is the use of a multilevel spatial autoregressive model (Baltagi, Fingleton, & Pirotte, 2014) to explain annual house price variation for 2000–2007 across 353 local authority districts in England. The discrete hierarchical structure models the spatial heterogeneity and implicitly allows autocorrelations within the higher-level units, while the spatial autoregressive specification allows spatial spill-over effects of prices across the districts that diminish as distance increases. 2.3

Artificial Neural Networks

In this section, ANN is introduced as another alternative approach to modelling house prices. Its conceptual background is firstly introduced and a survey of past studies applying ANN is then provided. 2.3.1

ANN Model

ANN is a form of artificial intelligence that consists of a number of interconnected processing elements, or neurons, that mimic the functions of biological neurons to process information in parallel (Maureen Caudill, 1988). One of the most popular ANNs is Multi-Layer Perceptron (MLP) as it is capable of mapping the non-linear relationships within the data. MLP typically contains an input layer, an output as the last layer, and one or more hidden layer(s) between the input and output layer. Each layer consists of a number of ‘neurons’, which are interconnected to the neurons in the immediate adjacent layers by a set of weights, representing the strength of connection. Back-propagation (BP) training algorithm

(Rumelhart et al., 1986) is commonly used to train the ANNs, which calculate the errors between the predicted output and the target output (actual house price) and back-propagate the error to adjust the connection weights between the neurons in adjacent layers with the aim to find the optimal connections between the neurons that best map the relationships between inputs (e.g. various attributes of individual houses and neighbourhoods) and output (e.g. house prices). Once the relationship is established, the networks can be used to predict house prices. Figure 1 shows the structure of MLP with a single hidden layer. We propose a feed-forward ANN in our study, where neurons are only connected forwards and there are no connections back as feedforward is best suited (as argued by Callan ,1999) for modelling relationships between a set of inputs and one or more output variable(s).

Figure 1 Multi-Layer Perceptron with a single hidden layer In building the neural network, the original total dataset is generally subdivided into 3 subsets: training set, validation set and test set (Ripley, 1996 p.354). The training set is a set of observations that are used to initially calibrate or train the network to find the most appropriate strength of connection between the neurons so as to produce an output as close as possible to the target output. The validation set is a set of observations used to avoid the risk of the network being over fitted. Once the network architecture is fully trained, its performance can be assessed using the test set. As the test set is a hold-out sample and has never been used to train or tune the model, this ex post hold-out sample prediction can be used to evaluate the generalisation capability of the networks from existing data to new data. There are many advantages of using ANNs to estimate house prices. It is equation free and does not require a priori assumption on the relationships between the inputs and outputs and can be utilised as an exploratory tool to detect the complex relations within house price data. In addition, ANNs have a high degree of automation and the predictions are generally noise resistant (Openshaw & Openshaw, 1997) 2.3.2

Review of previous works on ANN application

The application of ANN in real estate started in 1990s but views about its predictive accuracy compared with other modelling techniques did not reach consensus. Almost all the studies compared ANN with multiple regression calibrated with OLS estimation, essentially the standard HPM model, with few exceptions (McCluskey et al., 2012; McCluskey et al, 20134). Some found that ANN is superior to standard HPM in terms of the predictive accuracy (Peterson & Flanagan, 2009; Limsombunchai et al., 2004; Nguyen & Cripps, 2001; Cechin 2000; Mccluskey & Anand 1999; Do and Grudnitski, 1992), but others found that this was not necessarily so (McGreal et al., 1998; Worzala, Lenk, & Silva, 1995; Rossini, 1997). For example, Do and Grudnitski’s (1992) found that the number of the predicted values within 5% of the actual sales price using ANN was almost twice as much as using multiple regression. Peterson and Flanagan (2009) pointed out that regression analysis suffered from serious non-linearity problems and demonstrated that ANN produced more accurate hold-out sample prediction and extrapolated better from large datasets, especially when there is a large number of dummy variables representing the characteristics of house and neighbourhood. However, Worzala, et al (1995) found that the predictive accuracy of ANN and HPM were in fact very similar when replicating Do and 4 McCluskey et al (2012) compared ANN with a non-linear regression model and with two other approaches (simultaneous autoregressive model and geographically weighted regression (GWR)) in McCluskey et al. (2013) and found that non-linear regression has higher predictive accuracy and GWR outperform ANN in terms of prediction accuracy, consistency, transparency and model explanatory capability.

Grudnitski’s (1992) study by using a similar dataset that had the same house characteristics and price range with the same neural network architecture. They also applied ANN to a more homogenous housing market in only one Zip code and did not find that ANN predicted any better than HPM. They cautioned that ANNs tended to produce inconsistent results and this view is endorsed by Lenk et al. (1997) and McGreal et al. (1998). It has been argued that a larger training set should enhance predictive capability (Ge 2004; McGreal et al. 1998) since this approach excels at ‘learning’ from the known data to estimate the ‘unknown’. Indeed, this is demonstrated by Peterson & Flanagan's (2009) and Nguyen & Cripps (2001). But, in contrast, Rossini (1997) prefers ANN for smaller datasets as it predicts better and is less influenced by outliers and he prefers HPM for large datasets due to the speed of computation. In short, there is no consensus as to whether ANN is more suitable for larger or smaller datasets. All the studies to date have used less than 6,000 observations, with only two exceptions, Peterson & Flanagan (2009) with 46,467 and Zurada et al (2011) with 16,366 properties. The previous ANN studies are not just limited in the size of the dataset, they are also generally limited by the type of data being modelled. Despite the economic theory and numerous non-ANN empirical studies stressing the importance of location and neighbourhood attributes, previous ANN studies have focused on house characteristics and only a limited number have considered neighbourhood characteristics (Collins & Evans, 1994; Limsombunchai et al., 2004; McGreal et al. 1998; Mimis et al., 2013) and only one study (García et al., 2008) considers locational effects by explicitly including the geographical coordinates in ANN model. However, this study used a very small dataset (591 properties). Overall, ANN is favoured for its capability to model non-linearity within noisy data, but it has also been criticised for its black-box nature and inability to interpret the effects of various factors on house prices ( McGreal et al. 1998; Lenk et al. 1997). The capability of detecting the underlying patterns and relations from the data seems to rely heavily on the quality and characteristics of the data. The results also depend on the software and the structure of the network (Worzala et al., 1995). The training time required for the network sometimes can also be very long (Vellido et al. 1999). However, with the advance in computer technology, many of the drawbacks could be alleviated. ANN could be a cost-effective method with comparatively little computation time (Zurada et al. 2011) and remedies for interpreting the importance of the affecting variables have now been suggested (Paliwal & Kumar 2009; Olden & Jackson 2002). Some software (for example, IBM SPSS package version 21) has the inbuilt features to automatically select the optimal number of the hidden neurons and rank the relative importance of inputs. This significantly reduces the searching time for the optimal structure and enhances the explanatory power of ANN. 2.4

Contributing to the existing literature

There is no published work to date comparing ANN with MLM either conceptually or empirically in terms of house price prediction. In addition, there are contradictory views as to whether ANNs extrapolate better for larger datasets. The present study serves as a large-scale empirical study using the largest dataset deployed in an ANN approach to house price predictions. Moreover, since locations are found to be one of the most important factors in residential price determination, this study builds several scenarios depending on how the locations of properties are captured under different modelling frameworks, with the following research questions in mind: 1. What are the options to include the locations of residential properties in house price modelling? 2. How do these options compare within the same modelling framework, and how do different modelling approaches compare with each other when locations are modelled in the same way? 3. Is there still need to specify the local neighbourhoods of the properties when their locations have been included in the models? 4. If so, at what spatial scales and to what degree does neighbourhood matter and is there is still a need to include neighbourhood attributes in house price prediction? 5. Apart from the prediction accuracy, are there any there other criteria should be considered when comparing competing models? The questions will be addressed using routinely available secondary data.

3

Data and scenario comparison

In this section we first describe the setting and the dataset that is available for modelling, then outline the set of scenarios and models that will be compared in each scenario, and finally we will consider the grounds on which the comparison is made. 3.1

Setting and data

In order to have a dataset which contains reasonable representations of both rural and urban housing, this study selects Greater Bristol as the study area as it has a diversified urban and rural mixture and includes affluent suburbs and former social, local-authority owned housing. It also has a mixture of different property types. Greater Bristol includes the City of Bristol and its surroundings, typically known as the County that Used to Be Avon (CUBA)5. The area has a population of approximately 1.08 million people in 20096 and covers around 1,342 km2 of land. The house prices and property attributes are from the Land Registry of England and Wales (http://www.landregistry.gov.uk/), as this is a credible data source and contains a complete historical house sale record since 19957. Each record contains the actual sold price, the date of the sale, the address, the unit postcode, property type (detached houses, semi-detached houses, terraces or flats), duration (leasehold or freehold), and whether the property is newly-built at the time of the sale. Although this Land Registry data is somewhat restricted in terms property attributes, it is geographically rich. In order to capture the effect of location on house prices, the location of each sale is geocoded based on the unit postcode of the property to obtain its Ordnance Survey National Grid references8. In this way, the location can be either represented by its Grid references or by the neighbourhood where the property is located. The distance to city centre and various neighbourhood characteristics can also be obtained based on the geocodes. The definition and explanation of the property attributes are set out in Table 1. Table 1 Definition and explanation of house price data and attributes Variables Definition and explanation Price Sale price stated on the Transfer deed in thousands (£‘000) Yrmth The year and the month when the sale was completed as stated on the Transfer deed, expressed in numerical form Type “Det” for Detached house “Semi” for Semi-Detached house “Terr” for Terraced houses “Flat” for Flats/Maisonettes New "New" for a newly built property "Old" for an established residential building Duration Types of legal interests in land: “Free” for freehold, where the legal interest in land is held by the owner of the land “Lease” for leasehold, where the interest in land or property is held by the tenant who lets the property from the landlord East The Ordnance Survey postcode grid reference: Easting Nth The Ordnance Survey postcode grid reference: Northing Dist Euclidean distance to the city centre, Cabot Circus in Bristol 3.1.1

Neighbourhoods characteristics

It is necessary to define the neighbourhood first before obtaining neighbourhood characteristics. This is

5

The city of Bristol was part of historic county Avon, which was abolished in 1996 and split into four new unitary authorities: Bath and North East Somerset, City of Bristol, North Somerset and South Gloucestershire. They are also called CUBA, the "County (or Councils) that Used to Be Avon" after Avon was abolished. 6 West of England Key statistics Autumn 2010, http://www.westofengland.org/media/80749/autumn%202010.pdf 7 Transactions involving a corporate body, company, business, and any house sales that may not have been for full market value are not included in this dataset. 8 National Statistics Postcode Lookup (NSPL) is used for the geocoding process as it contains the Ordnance Survey National Grid references for all the unit postcodes, including those already terminated.

also an essential step in the MLM design which requires the higher-level units to be pre-specified. Although there is no consensus on the definition of local neighbourhood as it depends on the purpose of the research, a general view is that it should be relatively homogeneous in terms of housing stocks and/or the residents characteristics (Watkins 2001; Adair et al. 1996). We therefore decided to use 2001 census Output Areas (OAs) as the finest neighbourhood level in our analysis. The reasons for choosing OAs are threefold. First, 2001 OAs are built from adjacent unit postcodes and are designed to be homogenous in terms of household tenure and property type. Urban or rural mixes and clear physical boundaries such as major roads or waterways are generally avoided (Martin, 2001). OAs therefore represent meaningful local neighbourhoods which are nationally defined. Secondly, OAs are the smallest geographical units for census output, typically containing an average 124 households (ONS, 2012 p6). Using OAs as the lowest level of neighbourhood provides the most detailed information about neighbourhoods and avoids the need for aggregated neighbourhood characteristics from sample surveys as they are directly obtainable from the census. Lastly, as each sale is geocoded and the OA has digital boundaries, the neighbourhood statistics can be easily linked to the house price data9. It is possible that house prices are determined by processes operating at a variety of spatial scales. Consequently, in addition to OAs two further census geographies were used: lower layer super output areas (LSOAs) and middle layer super output areas (MSOAs). LSOAs are generated from groups of adjacent OAs using the same clustering criteria as the OAs to maintain social homogeneity and relative compact shape. Typically four to six OAs are aggregated to form LSOAs (ONS, 2008a), which have an average of 630 households. LSOAs are then grouped together in the same manner to form MSOAs which contain, on average 3011 households. The boundaries of OAs, LSOAs and MSOAs are maintained as stable as possible to allow consistent reporting and comparison of neighbourhood statistics over time10. The usage of these off-the-shelf geography units allows the methodology to be applied nationally and routinely without time consuming and expensive research in the submarket delineation. The linkage between OAs, LSOAs and MSOAs was achieved through GeoConvert (http://geoconvert.mimas.ac.uk/). Oriented by previous studies on the neighbourhood effects on house prices (Munroe, 2007; Bengochea, 2003; Thaler, 1978; Richardson et al., 1974; Ridker & Henning, 1967), a range of neighbourhood characteristics (displayed in Table 2) are abstracted from 2001 census data and the Neighbourhood Statistics website (https://neighbourhood.statistics.gov.uk/). Most variables are measured at the OA level except the 2004 Index of Multiple Deprivation Score and three aspects of deprivation score (measured at LSOA level) and modelled median income (estimated at MSOA level).

9 The spatial joining is achieved in ArcGIS10.2 (ESRI, 2011) by matching OA polygons to the sale points that is inside or on the boundary of the polygon. 10 Only 2.6% of OAs, 2.5% of LSOAs and 2.1 % of MSOAs in England and Wales have been changed in 2011 Census, respectively.

Table 2 Definition of neighbourhood variables Variables IMD IMDbar

IMDenv

IMDcrime

Green Det_area Terr_area Flat_area Room Noheat Unemploy Lnincome SocRent PriRent Occupancy

Young Old Black Noedu Degree 3.1.2

Definition and Explanation 2004 Index of Multiple Deprivation score (IMD)11 2004 deprivation score on “Barriers to Housing and Services”, measuring barriers to housing such as affordability and geographical barriers to key local services. 2004 Deprivation score in the living environment, comprising the ‘indoors’ living environment which measures the quality of housing and the ‘outdoors’ living environment for air quality and road traffic accidents. 2004 Deprivation Crime Domain Score, which measures the rate of recorded crime for four key dimensions of crime: burglary, theft, criminal damage and violence. Green space area percentage of total land use area Proportion of detached house Proportion of terrace house Proportion of flats Average number of rooms per household, used as proxy of average size of properties Proportion of houses that have no central heating Proportion of people aged 16-74 who are not in employment, including retired, students aged over 16 years old and other people Natural log of Experian income at MSOA level in 2004 Proportion of social rented from council or others Proportion of private rented from council or others The Occupancy Rating provides a measure of under-occupancy and overcrowding. It relates the actual number of rooms to the number of rooms ‘required’ by the members of the household Proportion of people aged 17 or under Proportion of people aged 65 or older Proportion of black ethnic Proportion of people have no academic or professional qualifications Proportion of people have at least First degree or Higher degree

Level LSOA LSOA

LSOA

LSOA

OA OA OA OA OA OA OA MSOA OA OA OA

OA OA OA OA OA

The study period

Figure 2 shows the mean price of houses in the Bristol region on a monthly basis for the study period, which lasts from 2001 to 2012 for model calibration and then predictions are made for 201312. The choice of 2001 allows the use of the 2001 census to define neighbourhoods and obtain neighbourhood statistics. In contrast to all previous studies where random samples within the same time period are generally used for ex post prediction, this study uses a different year, 2013, as a hold-out sample. This design allows the examination of house price changes over a relatively long period and provides a stringent and demanding test of the models in a period of substantial market turbulence. During this period, house prices in the UK and Bristol have surged at the beginning and then dropped drastically during the global financial crisis around 2008 and then have increased again in recent years.

11 A higher deprivation score indicates a higher proportion of people living there who are classed as deprived. 12 ANNs require a training set to fit the network and a validation set to avoid over-fitting in the calibration process, therefore a random sample of 90% of the 2001-2012 data were used as training set and the rest 10% are used as validation set.

250 100

150

200

Mean Actual House Prices (£k)

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

Year

Figure 2 Mean monthly actual sale prices for Greater Bristol 3.1.3

Obtaining the sample

There are in total 257,547 sales in the study area from 2001-2013, after removing the incomplete or incorrect records13. This large number of records is computationally demanding and dramatically lengthens the training time of ANN without a potentially much gain in prediction. Therefore, a stratified sampling strategy based on the location of the property is used to ensure a good mixture of different types of area and to have a spatially representative sample. Consequently OAs are used as the stratification unit with up to 20 sales randomly drawn from each. The resultant dataset is 65,302 house sales nested in 3320 OAs, which in turn are nested in 653 LSOAs and 137 MSOAs. This is still the biggest sample used in applying ANN to model house prices and should provide a good empirical test of the suitability of ANN for large datasets. 3.2

Scenarios for comparison

An important aspect of this study is how to incorporate location and neighbourhood into the predictions. Borst (2007) provide a detailed summary of the options.14. Oriented by this review, three scenarios are designed with different representations of space and place. This addresses the first and second research questions, and allows vertical comparison of different scenarios within the same modelling framework and at the same time horizontal comparison between different modelling approaches under the same scenario. Scenario 1: Property characteristics only Three models are specified for each modelling approach under this scenario, named HPM1, MLM1 and ANN1. The dependent variable is the logarithm of house prices expressed in thousands of pounds, as the raw house prices are markedly positively skewed. The predictor variables include house characteristics (the reference property category is a non-new build, freehold, semi-detached property) and the time of the sale (expressed in numerical form by its year and month). The property characteristics are specified as full interactions representing 16 combinations of property type. Time of the sale is also included as a third order polynomial to capture the time trend in house price changes. No locational or measured neighbourhood characteristics are included in these initial models, but the MLM requires the specification of the higher-level units, the nested multiple neighbourhood structure (OAs, LSOAs, and

13

Some records have missing postcode or durations. A single-level logistic regression model has been employed to examine whether the missingness is at random. The result shows that new leasehold properties sold in earlier years have higher probabilities to have missing postcode. But given the new properties represents just under 10% of the sample and the records with missing information (postcodes or unspecified duration) is a very small proportion of the original dataset (0.17%), these incomplete records have been removed. 14 They are grouped into six categories: (1) market segmentation; (2) using the neighbourhood delineation variable; (3) neighbourhood influence variables; (4) accessibility measures; (5) explicit use of location and (6) advanced model specifications

MSOAs) is used in this scenario and all the subsequent scenarios for MLM. Scenario 2: Property characteristics and the grid references In this scenario, the Grid references of property location (Easting and Northing) are included as additional explanatory variables, both as absolute values of the geographical location and as the distance from the city centre. The explicit use of absolute location originated in the work by Casetti (1972) in a hedonic model. The polynomial expansion of the coordinates in the model is also possible, which effectively is a trend surface analysis (Unwin, 1969), Jones and Bullen (1994) extended it to a multilevel approach. The multilevel model specified in this scenario answers the third research question, testing whether neighbourhood still matters when the absolute locations are included in the model. Six models, HPM2(a)/(b), MLM2(a)/(b) and ANN2(a)/(b) are specified under this scenario where situation (a) uses the absolute locations while (b) additionally includes the distance to the city centre.. Scenario 3: Property and neighbourhood characteristics In this scenario the Grid references and distance to city centre are replaced by measured neighbourhood characteristics. To facilitate model calibration and interpretation, all continuous explanatory variables are group-mean-centred so the intercept term represents the reference property category in a typical neighbourhood. Another three models are calibrated, HPM3, MLM3 and ANN3 to answer the fourth research question on the extent to which neighbourhood still matters when neighbourhood and property characteristics are accounted for. Given that there are 20 neighbourhood variables, we have examined the predictors for multicollinearity by calculating variance inflation factors (VIFs) (Belsley et al., 1980). VIF measures how “inflated” the variance of an estimated regression coefficient is because of multicollinearity (Freund and Littell 2000: 98). Generally VIF greater than 10 indicates potentially damaging multicollinearity. In this set of neighbourhood characteristics, all the VIF are below 10 and the majority are below 5, with the VIF for 2004 IMD score (VIF=8.89) and proportion of people have no qualifications (VIF=7.38) being the highest. 3.3

Performance measures

In order to reach a balanced view of a model’s performance, a range of criteria are considered. We now examine each criterion in turn. 𝑅 2 is used to as the goodness-of-fit measure. It is generally used in linear regression to assess the proportion of the total variance accounted for by the model. Here it is generalised to MLM and ANN and is calculated as the square of the correlation between the actual outcomes and the predicted values15. In this study, 𝑅 2 is calculated for both the in-sample data and the hold-out sample as this tests the generalisation capability of each modelling strategy from the modelled data to the unknown new data. However the use of only 𝑅 2 has been argued to be insufficient to make distinctions between competing models (Davies, 1981; Powell, 1980; Willmott, 1981, 1982), consequently regression analyses are also performed to explore further the relationships between the predictions and actual outcomes. A range of error measures are commonly used to measure the errors (or squared errors) between the predicted and actual outcomes (McCluskey et al. ,2013; Cavazos & Hewitson, 2005; Fekete et al., 2004). Willmott & Matsuura (2005) advocate using MAE as the average error performance measure. We therefore use Mean Absolute Error (MAE) as the accuracy measure and mean absolute percentage error (MAPE) as a relative measure of accuracy. MAE and MAPE are defined as 𝟏

̂𝒊 | MAE= 𝒏 ∑𝒏𝒊=𝟏|𝒚𝒊 − 𝒚 𝟏

MAPE = 𝒏 ∑

(9)

𝒏

̂𝒊 𝒚𝒊 −𝒚 | 𝐲 𝐢 𝒊=𝟏

|

(10)

where 𝑦𝑖 is the actual value of the response variable, 𝑦̂ is the predicted value, n is the number of samples and it can be seen that the sign of the differences is ignored. These two measures are provided for both

15

𝑅 2 can also be calculated for each level of the multilevel model and the method can be found in Gelman & Pardoe (2006).

the log price scale and the exponentiated value for easier interpretation. In addition to these quantitative measures of performance, it is also important to consider the face validity of the models, that is the extent to which the explanatory power of property and neighbourhood variables in accounting for house price is revealed. Here we are particularly concerned with which variables are the most influential in determining the predicted prices. 4

Empirical results and discussion

In total, 12 models are specified under the three different scenarios. HPM and MLM are fitted in the software package MLwiN 2.32 (Rasbash et al., 2009) as this software can fit standard regression models and multilevel models and additionally has the functionality that can easily exclude hold-out cases during model calibration and then include them for prediction. The ANN models are calibrated using IBM SPSS package version 21 (IBM Corp. 2012) as this implementation accepts categorical variables without the need for manual transformation16. We have chosen to let the system automatically select the number of hidden neurons for networks with one hidden layer, with the default activation functions between the neurons in neighbouring layers. It also outputs the relative importance of the inputs and handles large datasets. Due to limited space, detailed model results are not included here but available on request. 4.1

Goodness-of-fit

The comparison of goodness-of-fit are presented in Table 3 where the highest 𝑅 2 for each scenario are highlighted in bold. Vertically, the 𝑅 2 of HPM and ANN increase sequentially from scenario 1 to 3 for both in-sample and hold-out samples. This indicates that the locations are better represented by neighbourhood characteristics than grid references for HPM and ANN. The 𝑅 2 of MLM however hardly changes between different scenarios, implying that including the grid references of the location and the neighbourhood attributes have not improved the goodness-of-fit. This is because to some extent the influence of measured locational information has already been incorporated in MLM design, represented by the multiple levels of latent place effects.. The comparison between ANN and HPM in terms of goodness-of-fit is not consistent. In some scenarios, ANN has a higher 𝑅 2, but in others it has a lower or almost the same 𝑅 2. This inconsistency is also found by Worzala et al.(1995) and McGreal et al. (1998). MLM has the best model fit in all scenarios, illustrating the necessity of accounting for the hierarchical structure of housing market. For example, in scenario 2, the major improvement in 𝑅 2 from HPM to MLM shows that even when the absolute location are already included in the model there is still the need to specify the local neighbourhoods. Similarly, in scenario 3, the best model fit of MLM demonstrates the benefit of distinguishing the number of houses from the number of neighbourhoods in MLM. MLM permits characteristics measured at different levels to enter price determination at their appropriate levels by specifying the neighbourhood structure. This answers our third research question. Table 3 Comparisons of Goodness-of-fit Scenario 1

Scenario 2

Scenario 3

HPM1 MLM1 ANN1 HPM2a MLM2a ANN2a HPM2b MLM2b ANN2b HPM3 MLM3 ANN3 2

R (in-sample) R2 (hold-out)

0.39 0.23

0.75 0.75

0.39 0.23

0.43 0.30

0.75 0.75

0.41 0.26

0.43 0.31

0.75 0.75

0.47 0.38

0.68 0.65

0.75 0.74

0.69 0.67

To further explore the relationships between the predictions and actual outcome values, scatter plots and regression analysis are performed for scenario 3 where HPM and ANN have achieved the highest goodness-of-fit. Three scatter plots (Figure 3) are drawn with x being the actual log prices and y being the predictions from HPM3 (Figure 3a), MLM3 (Figure 3b) and ANN3 (Figure 3c). From a visual 16

SPSS recodes categorical variable (such as property characteristics) using one-of-c coding with c being the categories of a variable by default. We also choose to adjusted normalised continuous input variable (such as neighbourhood characteristics) and output variable (log house price) to set the value between -1 and 1 to improve network training.

5

6

ANN3 Prediction

6

MLM3 Prediction

6

4

4

5

5 4

HPM3 Prediction

7

7

7

8

examination, MLM predictions seem to be more tightly fitted to the actual log prices, and further regression analysis of the predictions regressed on the actual log prices supports our finding. In all three plots the scatter around the line is fairly constant suggesting the degree of error is not changing with mean price; there is not for example greater error with more expensive properties.

3

4

5

6

7

8

3

lnP

4

5

6

7

3

8

4

5

(a) HPM3

6

7

8

lnP

lnP

(b) MLM3

(c) ANN3

Figure 3 Scatter plots of predictions and actual log prices in scenario 3 by 3 different models 4.2

Predictive accuracy

The accuracy comparisons are summarized in Table 4, based on the 4141 hold-out samples in 2013. This shows how well these modelling strategies extrapolate to new set of data based on historical house prices, at a time of a turbulent market environment. Table 4 Comparisons of prediction accuracies for hold-out samples Hold-out sample: 4141 cases MAE(lnP)

Scenario 1

Scenario 2

Scenario 3

MLM1

ANN1

HPM2a

0.319

0.178

0.318

0.304

0.178

0.313

0.303

0.178

5.89%

3.29%

5.85%

5.61%

3.29%

5.76%

5.59%

MAE(expo lnP)

80.4

48.6

80.1

77.0

48.6

79.0

MAPE(expo lnP)

30.9%

17.5%

30.0%

29.4%

17.5%

29.8%

MAPE(lnP)

HPM1

MLM2a ANN2a HPM2b MLM2b ANN2b

HPM3

MLM3

ANN3

0.286

0.210

0.178

0.216

5.26%

3.89%

3.30%

4.00%

76.9

3.29% 48.6

73.7

56.3

48.8

56.6

29.4%

17.5%

27.2%

20.7%

17.6%

20.7%

Again, the models are compared vertically and horizontally. Similar conclusions are reached as in the goodness-of-fit comparison. The predictive accuracy has improved in HPM and ANN sequentially from scenario 1 to 3, while the predictive accuracy of MLM hardly changes. Horizontal comparison between different approaches shows marginal differences between HPM and ANN, both achieving a MAPE of 21% in the exponentiated scale in scenario 3. Compared with the results from past studies using ANN (MAPE ranges vastly from 5% to almost 30%); this results sit in the middle, probably due to the limited property information included in the model. MLM remains the best performer in each scenario. The average error is £49k between the predicted and actual house prices, or 18% of the actual prices for MLM in all scenarios. These are impressive results considering the housing market has been very turbulent in the model calibration period 2001-2012. This demonstrates that the extrapolation capability of MLM from modelled data to newly unknown data is reasonably high, even when the house prices have been very volatile. 4.3

Model explanatory capability

The final way of comparing the models is to examine their explanatory capacity, their ability to reveal the nature of the estimated relationships and what still remains unexplained. In this section, we compare the results from ANN and MLM models for scenario 3. ANN is traditionally seen as rather opaque in revealing the relationships between the inputs and the output. However, the IBM SPSS software can produce the importance for each input variable, through sensitivity analysis. It calculates the changes in predicted house price when one input changes while all other inputs remain constant. This is computed from the in-sample data. Table 5 displays the importance and the normalised importance of each input (the latter is derived from the former by dividing the

importance value by the maximum importance value), ranked from the highest to the lowest. It shows that the average room size of a neighbourhood has the greatest effect on house prices, followed by the time of the sale and the percentage of the resident who hold higher degrees. The percentage of terraced house in OAs, the IMD scores in housing barrier and environmental and aspects appear to be the least influential factors to house prices. However, this measure does not indicate the ‘direction’ of the effects and there are no confidence intervals on the estimates17. The exact effects are still very hard to quantify, especially for categorical variables such as the type of property. Table 5 Importance of Input Variables in ANN3 Input Variable

Normalized Importance Importance

Levels

Room 0.149 100.0% OA Yrmth 0.114 76.7% Property Degree 0.104 69.7% OA Type 0.094 62.8% Property IMD 0.079 52.9% LSOA Flat_area 0.057 38.2% OA Unemploy 0.051 34.1% OA Young 0.047 31.6% OA Noedu 0.045 30.1% OA IMDcrime 0.032 21.2% LSOA Det_area 0.030 20.4% OA New 0.030 19.8% Property Lnincome 0.027 18.4% MSOA Duration 0.026 17.7% Property Black 0.019 12.7% OA Occupancy 0.015 10.3% OA SocRent 0.014 9.7% OA PriRent 0.012 8.0% OA Old 0.010 6.8% OA Green 0.010 6.4% OA Noheat 0.009 6.2% OA IMDenv 0.009 6.1% LSOA IMDbar 0.008 5.6% LSOA Terr_area 0.008 5.6% OA The primary outputs for the MLM are regression-like coefficients for the fixed part of the model alongside the standard errors of the estimates. In addition, the random part has variances at each level, which summarise the explained variation at each scale. In order to compare the relative importance of each predictor, we have used a graphical approach. Figure 4 shows the predicted house price (after exponentiation) on a common scale. The predictions are for all sixteen types of property, holding everything else at their mean value (Figure 4a). The time effects (Figure 4b) are for the typical property in a typical neighbourhood. The effects of the neighbourhood characteristics (Figure 4 c-f) show the relationship between house price and each variable, holding all other variables constant at their typical value. In each figure, 95% confidence intervals of the in-sample model are also plotted. We used the customised prediction facility of MLwiN to obtain these graphs through simulation, which for example overcomes the problem of obtaining confidence intervals for the non-reference category (Firth, 2003).

17

Bootstrapping techniques (Efron and Tibshirani, 1993.) can be employed to estimate the confidence intervals where repeated samples with replacement are used to train the network to obtain a large number of predictions for the output and the range of the output can be regarded as the interval of the prediction. However this process is rather time consuming and is not directly obtained from one calibration as in MLM.

The property characteristics (4a) show a very clear set of result. The most expensive properties are detached freehold both new-build and existing properties. For freehold, non-new build properties, semidetached are significantly more expensive than terraces and flat, as shown by non-overlapping confidence bounds. For new-build freehold properties, there is no significant differences between semidetached and terraced houses. For freehold properties, both new and existing flats have the lowest price on average, but new-build flats have quite wide confidence interval reflecting that there are relatively few sales of new freehold flats. The leasehold properties are generally cheaper, but retain the same pattern for existing properties. The new-build leasehold properties are quite different. The dearest are terraces; semi and detached are cheaper but with a very wide confidence interval reflecting the small number of sales. New build leasehold flats are significantly cheaper than terraces, and they have a tight confidence interval due to relative large number of sales. These property characteristics are important and have strong face validity. The same can be said for the time effects, shown in Figure 4 (b), which are the results of fitting a third order polynomial. There was a rapid rise from 2001 to 2007 and then the trend plateaued followed by a small decline post 2010. The confidence intervals around the trend are narrower in the earliest years, reflecting greater sales during that period.

Old (Price £k)

Free

Lease

300

300

240

240

180

180

120

120

New (Price £k)

Semi

Det

Terr

Flat

300

300

240

240

180

180

Semi

Det

Terr

Flat

Semi

Det

Terr

Flat

120

120 Semi

Det

Terr

Flat

Figure 4 (a) Effect size of predictors

Price £k

240

180

120

2001

2004

2007 Year

2010

Figure 4 (b) We now turn to the effects of the measured neighbourhood variables. It can be seen that the most influential variable is the average number of rooms in OA (Figure 4 (d)), which is the same result from ANN. As this increases, the price also increases markedly. As the average room goes from 2.93 to 8.73, the price increases by £200k. Of course, there is no measure of individual property size in the Land Registry data, so undoubtedly this aggregated variable is ‘soaking up’ in part the effects of the missing lower level variable. Other influential variables are those variables representing aggregate “people” characteristics (Figure 4 (f)) so that areas with higher percentage of people with a degree are expensive, as are areas with more old people. Areas with a larger percentage of people lacking education are cheaper, as are areas with more young people and black ethnic. The effects of the latter variable are less pronounced and there is a wide confidence bound for areas with high black ethnicity. Figure 4 (c) shows the effects for social and physical environmental variables. The strongest effect is for 2004 Index of Multiple Deprivation score which shows that as general deprivation increases, house prices are predicted to go down. Modest positive relationships are found for barriers to housing and living and percentage of green space after taking into account 2004 IMD scores, but there is little relationship between price and crime once other variables being taken into account. The effects for aggregate property characteristics are shown in 4 (d). We have already commented on the importance of average property size. The proportion of terrace properties and non-heated properties in a local neighbourhood have relatively little influence on price. The proportion of detached and flat properties show a greater influence, but this is the opposite of the individual characteristics, such that areas with a high proportion of flats are dearer and neighbourhoods with a high proportion of detached are relatively cheap. It must be stressed that this is not untoward as the variables have a different meaning at the different scales and portray the relationships having taken account of other variables. Figure 4 (e) shows that price increases with log income in a place, and perhaps surprisingly with the unemployment (but of course this is after taking account of deprivation which may be more robustly measured at LSOA level). Higher prices are also found when the area has a large proportion of private rented and a higher occupancy rating (under-occupied). The proportion of social rented houses in a neighbourhood does have a negative effect on price, but the effect is not substantial.

IMD04

IMDBar

IMDenv

IMDcrime

Green

300

300

300

300

240

240

240

240

240

180

180

180

180

180

Price £k

300

120

120 20

40

60

120

80

11

22

33

44

120 0

17

34

51

120 0

0.7

1.4

2.1

0

0.3

0.6

0.9

Figure 4 (c) Det_area

Terr_area

Flat_area

Room

Noheat

300

300

300

300

240

240

240

240

240

180

180

180

180

180

Price £k

300

120

120 0

0.3

0.6

0.9

120 0

0.3

0.6

0.9

120 0

0.3

0.6

0.9

120 3.2

4.8

6.4

8.0

0

0.11

0.22

0.33

Figure 4 (d) Unemploy

Lnincome

SocRent

PriRent

Occupancy

300

300

300

300

240

240

240

240

240

180

180

180

180

180

Price £k

300

120 0

0.21

0.42

0.63

120 9.0

120 9.6

10.2

10.8

120 0

0.3

0.6

Figure 4 (e)

0.9

120 0

0.23

0.46

0.69

0

0.16

0.32

0.48

Young

Old

Black

Noedu

Degree

300

300

300

300

240

240

240

240

240

180

180

180

180

180

Price £k

300

120

120 0

0.16

0.32

0.48

120 0

0.23

0.46

0.69

120 0

0.14

0.28

0.42

120 0

0.19

0.38

0.57

0

0.19

0.38

0.57

Figure 4 (f) Table 6 gives the size of the effect for the range from the minimum to the maximum for each neighbourhood variable and the 16 house types in the order of their importance. It can be seen that the average size of the property in OA, property characteristics and the time when it was sold are very important predictors. These are similar to the ANN results, but there are also some major differences for some other predictors such as the percentage of old people and the 2004 IMD crime score.

Table 6 Importance of Predictors in MLM3 and ANN3 Variable Room 16 Types Year Degree Old Flat IMD04 Noedu Black Inact Det Occup Young P_rent LnIncome IMDbar IMDenv Green Noheat S_rent Terr IMDcrime

Min 121 121 87 164 175 178 154 162 150 175 160 184 171 184 174 177 183 184 179 179 183 184

Max 323 311 191 252 246 227 196 202 187 212 194 211 197 208 197 200 195 194 188 188 188 189

Range 202 190 105 88 71 48 42 40 37 37 34 27 26 24 23 22 12 10 9 9 4 4

Level OA Property Property OA OA OA LSOA OA OA OA OA OA OA OA MSOA LSOA LSOA OA OA OA OA LSOA

Rank 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

ANN rank 1 4,12,14 2 3 19 6 5 9 15 7 11 16 8 18 13 22 23 20 21 17 24 10

An important aspect of the MLM analysis is not just to consider the influence of measured variables,

but also to consider how much remains to be explained and at what level. Figure 5 shows a series of models the percentage unexplained in log prices, the values are obtained from the variance estimates after the fixed variables have been included. The total unexplained variance is also partitioned into each level. Thus in the null model which just has a single overall mean, the total unexplained variance is 100%. The majority of this variance is at the macro MSOA level and the least unexplained variance is between OAs. It is the macro geography that is an important aspect of house prices. As the time trend is taken into account, the total unexplained variance reduces to 84% and now the majority of which still remains to be explained lies between properties. The inclusion of the 16 property types brings a further improvement in the model and the unexplained variance reduces to 65%. Now the majority of that lies at MSOA scale (VPC=44%). Finally by including the 20 neighbourhood predictors, there is a substantial fall in the unexplained variance to 34% and the majority of this lies between properties in terms of unmeasured attributes (VPC=78%) while at the neighbourhood scale, it is the macro MSOAs that are still the most important (VPC=11%). This means that some properties and some neighbourhoods are desired over and above the attributes that we are able to measure. This also demonstrates that spatial dependency is implicitly modelled in MLM framework (Jones & Bullen 1994; 1993) as the prices of two properties randomly drawn from the same OA are around 22% correlated, which if not properly modelled would violate the underlying assumption of HPM.

Null

+Time

+Property

+Neighbourhood

Percent Unexplained

100 100

100

100

80

80

80

60

60

60

40

40

40

20

20

20

80

60

40

20

0

0 Total Prop

OA LSOA MSOA

0 Total Prop

OA LSOA MSOA

0 Total Prop

OA LSOA MSOA

Total Prop

OA LSOA MSOA

Figure 5 Percentage unexplained in log prices To help appreciate the nature of the unexplained variation, we can consider the relative size of the random effects, that is how different places vary in the price. In Figure 6, we have plotted the coverage intervals, which represent the 95% limit of data variability and show the price for the typical property in the 2.5% highest and lowest priced areas18. The results are shown for when the MLM model contains only time, when property characteristics are added, and when neighbourhood characteristics are also included. The triangles represent the typical property in a typical place in mid-2006, which is priced at £187,000 in all the models. When only time is included in the fixed part, the total 95% range of the data lies between £90k and £400k, and there are clearly substantial differences between neighbourhoods. The differences are most marked between MSOAs and the 95% coverage lies between £108k and 340k. The least influential spatial scale is LSOA, but even for those while the average property is £187k, the dearest 2.5% attracts £252k compared to the least desirable 2.5% where the typical house costs £145k. When property characteristics are taken into account, these differences reduce somewhat but remain large, with the macro MSOA scale being particularly important. When the same type of house is considered, it matters considerably in which part of the city region it is located. Finally when neighbourhood characteristics are taken into account, the coverage intervals reduce quite strongly, but if the same type of property is considered and has the same neighbourhood characteristics, there is still an unexplained 18

They are derived from overall mean plus and minus 1.96 times the square root of the variance on the log scale, which is then exponentiated, which is why asymmetrical values are found.

premium/discount between neighbourhoods, which is still most marked at the macro MSOA level. The top 2.5% adds a 24% premium to the mean price of the typical property.

Price £k

Time

+Property

+Neighbourhood

360

360

360

270

270

270

180

180

180

90

90 Total

OA

LSOA

MSOA

90 Total

OA

LSOA

MSOA

Total

OA

LSOA

MSOA

Figure 6 Predicted mean prices and the coverage intervals at different levels 5

Conclusions

This paper illustrates the use of MLM and ANN approach to modelling housing prices and compares them with the widely accepted HPM approach in terms of goodness-of-fit, predictive accuracy and explanatory power. The effects of location are investigated under different scenarios, represented by geographical grid references, the specification of neighbourhood, and/or neighbourhood characteristics. Neither ANN nor HPM is capable of including neighbourhood in the model due to the large number of categorical variables required for practical specification, while MLM is able to specify by simply defining them as macro-level units. The various performance criteria also help us to reach a balanced view in the comparison of different competing models. All performance measures show that MLM is superior to ANN and HPM in each scenario, indicating that specification of neighbourhood is helpful in house price predictions, even when the locations or neighbourhood characteristics have been included in the model. Once the appropriate hierarchical structure of housing market has been defined in MLM, location and neighbourhood characteristics will only further explain the price variation between neighbourhoods, but will not further improve the predictive accuracy. The estimations of the coefficients of MLM are more robust and reliable than HPM, which reveals the effects of the measured variables at various spatial scales. In addition, it also indicates how much variance remained to be explained and at what level, providing further insights on the importance of neighbourhoods. This study has also demonstrated that ANN can be applied to large dataset, but its predictive accuracy is not necessarily better than standard HPM. This supports the finding by Lenk et al. (1997) but contrary to many other studies that find ANN is superior to HPM. The MLM which has proved to be very effective in capturing the underlying patterns of house prices and fully exploiting the information in the data in quite a simplistic version of the model. Further studies could extend it in a number ways, such as allowing each neighbourhood to have its own time trend, or permitting the effects of property characteristics to be differential between neighbourhoods (random slopes model); including cross-level interactions between property and neighbourhood characteristics; and alternative definitions of neighbourhood (for example, school catchment areas) in addition to census geography (cross-classified structure). While such developments would not necessarily bring improved explanatory understanding, they are likely to bring improved predictive accuracy. The multilevel approach is capable of more fully exploiting the data on both properties and neighbourhoods to achieve better predictions, to understand what is driving the predictions and pointing to what scale the unexplained elements are to be found.

References Aitkin, M. and N. Longford, 1986. Statistical Modelling in School Effectiveness Studies, with Discussion, Journal of the Royal Statistical Society, Series A,, 149, pp1–43. Adair, a. S., Berry, J.N. & McGreal, W.S., 1996. Hedonic modelling, housing submarkets and residential valuation. Journal of Property Research, 13(1), pp.67–83. Alonso, William, 1964. Location and Land Use: Towards a General Theory of Land Rent. Cambridge: Cambridge University Press Anselin, L. & Le Gallo, J., 2006. Interpolation of Air Quality Measures in Hedonic House Price Models: Spatial Aspects. Spatial Economic Analysis, 1(1), pp.31–52 Baltagi, B.H., Fingleton, B. & Pirotte, A., 2014. Spatial Lag Models With Nested Random Effects : an Instrumental Variable Procedure with an Application to English House Prices. Journal of Urban Economics. Belsley, David. A., Edwin. Kuh, and Roy. E. Welsch. 1980. Regression Diagnostics: Identifying Influential Data and Sources of Collinearity. New York: John Wiley and Sons Bengochea, A. (2003): A hedonic valuation of urban green areas. Landscape and Urban Planning 66 (2003) pp35–41 Bin, O., 2004. A prediction comparison of housing sales prices by parametric versus semi-parametric regressions. Journal of Housing Economics, 13, pp.68–84. Borst, R.A., 2007. Discovering and Applying Location Influence Patterns in the Mass Valuation of Domestic Real Property. Thesis. ,PhD.. University of Ulster. Brown, K.H. & Uyar, B., 2004. A Hierarchical Linear Model Approach for Assessing the Effects of House and Neighborhood Characteristics on Housing Prices. Journal of Real Estate Practice and Education, 7(1), pp.15–24. Brunauer, W., Lang, S. & Umlauf, N., 2013. Modelling house prices using multilevel structured additive regression. Statistical Modelling, 13(2), pp.95–123. Callan, R. 1999. The Essence of Neural Networks. London: Prentice Hall Europe. Casetti, E., 1972, Generating models by the expansion method: Applications to geographical research, Geographical Analysis 4, pp81-91 Cavazos T, Hewitson BC (2005) Performance of NCEP-NCAR reanalysis variables in statistical downscaling of daily precipitation. Clim Res 28. pp95–107 Caudill, M., Neural Network Primer: Part III, AI Expert, (June 1988) 53-59. Cechin, A.L., Souto, A. and Gonzalez, M.A.S. (2000), Real estate value at Porto Alegre city using artificial neural networks, paper presented at the 6th Brazilian Symposium on Neural Networks (SBRN’2000), SBRN, Rio de Janeiro, Brazil

Chasco, C. & Gallo, J. Le, 2012. The Impact of Objective and Subjective Measures of Air Quality and Noise on House Prices:A Multilevel Approach for Downtown Madrid. Economic Geography, 89(2), pp.127–148. Chau, K. W., Yiu, C. Y., Wong, S. K., & Lai, L. W., 2004. Hedonic Price Modelling of Environmental Attributes: A Review of the Literature and a Hong Kong Case Study. In Encyclopaedia of Life Support Systems (EOLSS) Oxford, UK: UNESCO (Vol. 2). Chica-Olmo, J., 2007. Prediction of Housing Location Price by a Multivariate Spatial Method : Cokriging. Journal of Real Estate Research, 29(1), pp.91–114. Collins, A. & Evans, A., 1994. Aircraft Noise and Residential Property Values An Artificial Neural Network Approach. Journal of Transport Economics and Policy, 28(2), pp.175–197. Congdon, P., 2003. Modelling Spatially Varying Impacts of Socioeconomic Predictors on Mortality Outcomes. Journal of Geographical Systems, 5(2), pp.161–184. Diez-roux, A. V, 2000. Multilevel Analysis in Public Heath. Annual Review of Public Health, 21, pp.171–192. Do Q, Grudnitski G. 1992. A neural network approach to residential property appraisal. The Real Estate Appraiser: pp.38-45. Efron B., Tibshirani R. J., 1993. An Introduction to the Bootstrap, Chapman and Hall ESRI 2011. ArcGIS Desktop: Release 10. Redlands, CA: Environmental Systems Research Institute. Fekete BM, Vörösmarty CJ, Roads JO, Willmott CJ, 2004 Uncertainties in precipitation and their impacts on runoff estimates. J Clim 17:294–304 Firth, D., 2003.. Overcoming the reference category problem in the presentation of statistical models. Sociological Methodology 33, pp1–18.. Foster, S. A. 1991. The Expansion Method: Implications for Geographic Research. Professional Geographer 43:131-42 Fotheringham, a S., Charlton, M.E. & Brunsdon, C., 1998. Geographically Weighted Regression: a Natural Evolution of the Expansion Method for Spatial Data Analysis. Environment and Planning A, 30(11), pp.1905–1927. Freund, Rudolf J. and Ramon C. Littell. 2000. SAS System for Regression ,Third edition.. Cary, NC: SAS Institute. Freeman, A.M., 2011. Hedonic Prices, Property Values and Measuring Environmental Benefits: A Survey of the Issues. The Scandinavian Journal of Economics, 81(2), pp.154–173. García, N., Gámez, M. & Alfaro, E., 2008. ANN+GIS: An Automated System for Property Valuation. Neurocomputing, 71(4-6), pp.733–742. Ge, X.J., 2004. Modeling Property Prices Using Neural Network Model for Hong Kong. International Real Estate Review, 7(1), pp.121–138.

Gee, G.C., 2008. A Multilevel Analysis of the Relationship Between Institutional and Individual Racial Discrimination and Health Status. American Journal of Public Health, 98(9 Suppl), pp.S48–56. Goldstein, H., 1999. Multilevel Statistical Model, Arnold. Goodman, A.C. & Thibodeau, T.G., 1998. Housing Market Segmentation. Journal of Housing Economics, 7(2), pp.121–143. Goodman, A.C. & Thibodeau, T.G., 2003. Housing market segmentation and hedonic prediction accuracy. Journal of Housing Economics, 12(3), pp.181–201. Greene, William H. , 2011 Econometric Analysis ,7th Edition. : Prentice Hall Grether, D M and Mieszkowski, P., 1974. Determinants of Real Estate Values. Journal of Urban Economics, 146, pp.127–145. Habib, M.A. & Miller, E.J., 2008. Influence of Transportation Access and Market Dynamics on Property Values: Multilevel Spatiotemporal Models of Housing Price. Transportation Research Record: Journal of the Transportation Research Board, 2076(-1), pp.183–191. Harris, R., Johnston, R. & Burgess, S., 2007. Neighborhoods, Ethnicity and School Choice: Developing a Statistical Framework for Geodemographic Analysis. Population Research and Policy Review, 26(5-6), pp.553–579. Haurin, D.R. & Brasington, D., 1996. School Quality and Real House Prices: Inter- and Intrametropolitan Effects. Journal of Housing Economics, 5(4), pp.351–368. IBM SPSS Statistics for Windows, Version 21.0. IBM Corp. Released 2012. Armonk, NY: IBM Corp. Jackson, J. R., 1979, Intraurban Variation in the Price of Housing, Journal of Urban Economics, 6, pp.464-479 Jones, K., 1991. Specifying and Estimating Multi-Level Models for Geographical Research. Transactions of the Institute of British Geographers, 16(2), pp.148–159. Jones, K. & Bullen, N., 1993. A Multi-level Analysis of the Variations in Domestic Property Prices: Southern England, 1980–87. Urban Studies, 30(8), pp.1409–1426. Jones, K. & Bullen, N., 1994. Contextual Models of Urban House Prices : A Comparison of Fixedand Random-Coefficient Models Developed by Expansion. Economic Geography, 70(3), pp.252–272. Kauko, T.O.M., Hooimeijer, P. & Hakfoort, J., 2002. Capturing Housing Market Segmentation : An Alternative Approach based on Neural Network Modelling. , 17(6), pp.875–894. Kong, F., Yin, H. & Nakagoshi, N., 2007. Using GIS and landscape metrics in the hedonic price modeling of the amenity value of urban green space: A case study in Jinan City, China. Landscape and Urban Planning, 79, pp.240–252. Lancaster, K.J. (1966), “A new approach to consumer theory”, Journal of Political Economy, Vol. 74, pp. 132-57.

Lee, C.C., 2010. The impact of facilities of leisure and sports on housing prices in taiwan: An application of hierarchical linear modeling. Journal of Real Estate Practice and Education 13(2): 159-175. Leishman, C., 2009. Spatial Change and the Structure of Urban Housing Sub-markets. Housing Studies, 24(5), pp.563–585. Leishman C, Costello G, Rowley S, Watkins C, 2013. The Predictive Performance of Multilevel Models of Housing Sub-markets: A Comparative Analysis. Urban Studies, 50(6), pp.1201–1220 Lenk, M.M., Worzala, E.M. & Silva, A., 1997. High-tech Valuation: Should Artificial Neural Networks Bypass The Human Valuer? Journal of Property Valuation and Investment, 15(1), pp.8–26. Li, M.M. & Brown, H.J., 1980. Micro-Neighborhood Externalities, Micro-neighborhood Prices, Hedonic Housing. Land Economics,, 56(2), pp.125–141. Limsombunchai, V., Gan, C. & Lee, M., 2004. House Price prediction: Hedonic Price Model vs. Artificial Neural Network. American Journal of Applied Sciences, 1(3), pp.193–201. Malpezzi, S., 2002. Hedonic Pricing Models: A Selective and Applied Review. Martin, David, 2001, Geography for the 2001 Census in England and Wales, London: Office for National Statistics. Mason, C. & Quigley, J.M., 1996. Non‐parametric Hedonic Housing Prices. Housing Studies, 11(January 2015), pp.373–385. McCluskey, W. J., McCord, M., Davis, P. T., Haran, M., & McIlhatton, D., 2013. Prediction accuracy in mass appraisal: a comparison of modern approaches. Journal of Property Research, 30(4), 239–265. McCluskey, William, Davis, Peadar, Haran, Martin , McCord, Michael and McIlhatton, David, 2012. The Potential of Artificial Neural Networks in Mass Appraisal: The Case Revisited. Journal of Financial Management of Property and Construction, 17 (3). pp. 274-292. Mccluskey, W.J. & Anand, S., 1999. The Application of Intelligent Hybrid Techniques for the Mass Appraisal Of Residential Properties. Journal of Property Investment & Finance, 17(3), pp.218– 239. McGreal S, Adair A, McBurney D, Patterson D. 1998. Neural Networks: the Prediction of Residential Values. Journal of Property Valuation and Investment, 16(1), pp.57–70. Merlo, J., Chaix, B., Ohlsson, H., Beckman, A., Johnell, K., Hjerpe, P.Råstam, L and K. Larsen 2006. A Brief Conceptual Tutorial of Multilevel Analysis In Social Epidemiology: Using Measures Of Clustering In Multilevel Logistic Regression To Investigate Contextual Phenomena. Journal of epidemiology and community health, 60(4), pp.290–297. Meyer, J., Kain, J., and Wohl, M., 1965. The Urban Transportation Problem. Harvard University Press, Cambridge, MA. Mills, E.S., 1972, Studies in the Structure of the Urban Economy. Baltimore, MD. Johns Hopkins University Press.

Mimis, A., Rovolis, A. & Stamou, M., 2013. Property Valuation With Artificial Neural Network: the Case of Athens. Journal of Property Research, 30(2), pp.128–143. Munroe DK, 2007 Exploring the determinants of spatial pattern in residential land markets: amenities and disamenities in Charlotte, NC, USA. Environ Plann B Plann Des 34(2):336–354 Muth, R.F. ,1969. Cities and Housing. Chicago. University of Chicago Press Nguyen, N. & Cripps, A., 2001. Predicting Housing ValueL A Comparison of Multiple Regression Analysis and Artificial Neural Networks. Journal of Real Estate Research, 22(3), pp.313–336. Olden, J.D. & Jackson, D. a, 2002. Illuminating the “Black Box”: a Randomization Approach for Understanding Variable Contributions In Artificial Neural Networks. Ecological Modelling, 154(1-2), pp.135–150. Openshaw, S., and Openshaw, C., 1997, Artificial Intelligence in Geography. Chichester, UK: John Wiley and Sons. Orford, S., 2000. Modelling Spatial Structures in Local Housing Market Dynamics: A Multilevel Perspective. Urban Studies, 37(9), pp.1643–1671. Orford, S., 2002. Valuing locational externalities: a GIS and multilevel modelling approach. Environment and Planning B: Planning and Design, 29(1), pp.105–127. Osland, L., 2010. An Application Relation of Spatial Econometrics Price to Hedonic House Modeling Author. Journal of Real Estate Research, 32(3), pp.289–320. Páez, A. & Scott, D.M., 2004. Spatial Statistics for Urban Analysis: a Review of Techniques with Examples. GeoJournal, 61(1), pp.53–67. Paliwal, M. & Kumar, U. a., 2009. Neural Networks and Statistical Techniques: a Review Of Applications. Expert Systems with Applications, 36(1), pp.2–17. Peterson, S. & Flanagan, A.B., 2009. Neural Network Hedonic Pricing Models in Mass Real Estate Appraisal. Journal of Real Estate Research, 31(2), pp.147–164. Powe, N.A., Garrod, G.D. & Willis, K.G., 1995. Valuation of Urban Amenities Using an Hedonic Price Model. Journal of Property Research, (12), pp.137–147. Richardson, Harry W., Vipond, John, Furbey, Robert A., 1974, Determinants of Urban House Prices, Urban Studies (1974), 11, 189-199. Ridker, R. G. and J. A. Henning, 1967, The Determinant of Residential Property Value with Special Reference to Air Pollution, Review of Economics and Statistics, Vol. 49, pp.147-157. Ripley, B.D. (1996): Pattern Recognition and Neural Networks. Cambridge University Press, Cambridge UK. Rasbash, J., Charlton, C., Browne, W.J., Healy, M. and Cameron, B. 2009. MLwiN Version 2.1. Centre for Multilevel Modelling, University of Bristol. Raudenbush, S.W. and A.S. Bryk, 2002. Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage Publications Inc.

Rosen, S. (1974), “Hedonic prices and implicit markets: product differentiation in pure competition”, Journal of Political Economy, Vol. 82, pp. 34-55. Rossini, P., 1997. Application of Artificial Neural Networks to the Valuation of Residential Property. In Third Annual Pacific-Rim Real Estate Society Conference Palmerston North, New Zealand, 20th-22nd January. Rumelhart, D.E., Hinton, G.E. & Williams, R.J., 1986. Learning Representations By BackPropagating Errors. Nature, 323(6088), pp.533–536. Sirmans, S.G., Macpherson, D.A., and Zietz, E.N., 2005. The Composition of Hedonic Pricing Models, Journal of Real Estate Literature 13(1), pp1-44, Snijders, Tom A B and Bosker, Roel J. 1999. Multi- level Analysis: An Introduction to Basic and Advanced Multilevel Modeling. Thousand Oaks, CA: Sage Shin, W.-J., Saginor, J. & Zandt, S., 2011. Evaluating Subdivision Characteristics on Single-Family Housing Value Using Hierarchical Linear Modeling. Journal of Real Estate Research, 33(3), pp.317–348. Thaler, R. 1978, ‘A Note on the Value of Crime Control: Evidence from the Property Market’, Journal of Urban Economics, 5, 137–45. Unwin D J. 1969. The areal extension of rainfall records. Journal of Hydrology 7: 404–414. Uyar, B. & Brown, K.H., 2007. Neighborhood Affluence, School-Achievement Scores and Housing Journal of Housing Research, 16(2), pp.97–116. Vellido, A., Lisboa, P.J.G. & Vaughan, J., 1999. Neural Networks in Business : A Survey Of Applications (1992 – 1998). Expert System with Application, 17, pp.51–70. Watkins, C. A. (2001). The Definition and Identification of Housing Submarkets, Environment and Planning A 33, 2235–2254. Willmott, C.J., 1982. Some Comments on the Evaluation of Model Performance. Bulletin of the American Meteorological Society, pp.1309–1313. Willmott, C.J. & Matsuura, K., 2005. Advantages of the Mean Absolute Error (MAE) over the Root Mean Square Error (RMSE) in Assessing Average Model Performance. Climate Research, 30, pp.79–82. Wingo, L. ,1961. Transportation and Urban Land. Johns Hopkins Press, Baltimore. Worzala, E., Lenk, M. & Silva, A., 1995. An Exploration of Neural Networks and Its Application to Real Estate Valuation. The Journal of Real Estate Research, (January), pp.185–202. Zurada, J., Levitan, A.S. & Guan, J., 2011. A Comparison of Regression and Artificial Intelligence Methods in a Mass Appraisal Context. Journal of Real Estate Research, 33(3), pp.349–387.