SP Models in Prediction - INFORMS

informs Vol. 45, No. 1, February 2011, pp. 98–108 issn 0041-1655
eissn 1526-5447
11
4501
0098

®

doi 10.1287/trsc.1100.0334 © 2011 INFORMS

On the Use of Mixed RP/SP Models in Prediction: Accounting for Systematic and Random Taste Heterogeneity Elisabetta Cherchi

CRiMM-Dipartimento di Ingegneria del Territorio, Facoltà di Ingegneria-Università di Cagliari, 09123 Cagliari, Italy, [email protected]

Juan de Dios Ortúzar

Department of Transport Engineering and Logistics, Pontificia Universidad Católica de Chile, 7820436 Santiago, Chile, [email protected]

A

basic assumption in mixed revealed preference (RP)/stated preference (SP) estimation is that both data sets represent basically the same phenomenon. Thus, we would expect individuals to show the same tastes regardless of the tool used to elicit their preferences. However, different and significant parameters are often found in each case. Although this is not an issue from an estimation standpoint, understanding why differences appear is crucial in forecasting because the model structure used in that case differs from the estimated one. This problem is compounded if differences between both data affect their ability to reproduce systematic or random taste variations because (i) microeconomic conditions on individual behaviour are more difficult to fulfil, and (ii) an erroneous specification may have a major impact on the predicted results. Problems associated with using joint RP/SP models in forecasting have received scant attention and no studies have examined the case where both types of data show different systematic or random heterogeneity. We review the problem from a theoretical viewpoint and suggest analyses that could aid decision taking in this context. Using real data, we provide evidence on the effects of using different joint RP/SP models in forecasting and highlight the importance of performing these analyses. Key words: mixed RP/SP models; random heterogeneity; policy forecasts History: Received: March 2008; accepted: June 2009. Published online in Articles in Advance December 20, 2010.

1.

Introduction

because significant random parameters with RP data often capture omitted structures rather than real heterogeneity among individuals (Cherchi and Ortúzar 2003). There is little evidence on the estimation of random heterogeneity with mixed RP/SP models and it mainly concerns the case when all random and nonrandom parameters (we refer to random parameters not to error components) only differ in the RP/SP scale (Walker 2001; Bhat and Castelar 2002; Lee, Chon, and Park 2004). In fact, we found only two papers on the estimation of random heterogeneity specific to SP data alone. Bhat and Sardesai (2006) estimated a random parameter specific to SP data because that attribute was unavailable while Brownstone, Bunch, and Train (2000) found an opposite sign for one nonrandom parameter because of poor RP data in a linear utility model but were able to pool the remaining generic attributes. We discuss the use of partial heterogeneity in a mixed RP/SP model with special reference to the case where attributes are available for both the RP and SP data sets, but their estimated parameters are significant and have different signs in both cases. We also

The mixed revealed preference (RP)/stated preference (SP) estimation method is based on the general idea of data enrichment by pooling information from different sources. Although the two types of data are different (and therefore need to be scaled before pooling), they represent basically the same phenomenon. Thus, we would expect individuals to show the same tastes irrespective of the method used to elicit their preferences. However, this is not always the case. For example, Small, Winston, and Yan (2005) and Bhat and Castelar (2002) report that in a multinomial logit model (MNL) estimation context, the value of time derived from the RP data is significantly larger than that derived from the SP data. The same differences might (and actually do) occur when more complex behaviour is accounted for in model estimation, for example, when individuals show random heterogeneity in preferences and response. The now-popular mixed logit (ML) model has gathered sufficient evidence in the literature, but most applications refer only to SP data 98

Cherchi and Ortúzar: On the Use of Mixed RP/SP Models in Prediction

99

Transportation Science 45(1), pp. 98–108, © 2011 INFORMS

discuss why the RP and SP data sets might show different abilities to account for individual heterogeneity. This issue, to the best of our knowledge, has never been discussed in the literature. Specifying an ML model with specific attributes for the RP or SP data sets does not present major problems from the estimation point of view. Understanding why such differences appear and interpreting results correctly, though, is crucial if models are to be used in forecasting. Only RP data should be used in forecasting because they represent the real world; however, when attributes estimated for SP data alone are included in the utility used in forecasting, the resulting model is “hybrid,” i.e., consistent neither with the RP nor with the SP environment. This means that all tests performed on the estimated model, especially those related to satisfying the usual microeconomic conditions on the marginal utilities and those related to model sensitivity, need to be performed and verified also in the forecasting structure because it is usually different. Moreover, because the effect of accounting for individual heterogeneity is to differentiate the impact of policies among the population, misuse of the ML model in forecasting might have severe consequences. The rest of the paper is organised as follows. In §2, we briefly review the basic theory of the ML model and discuss its application to mixed RP/SP data for partial preference homogeneity; in particular, we discuss the problem of using these models in forecasting. In §3, we illustrate the characteristics of the sample used. Next, in §4, we describe the results of estimating mixed RP/SP models with systematic and random heterogeneity in tastes. We also discuss the results of their application in forecasting. Section 5 summarises our main conclusions.

the modeller. Let us write the utility that individual q associates to alternative j in choice situation t as Uqjt = bj xqjt +
qjt zqjt +qjt 
 
 

Mixed Logit Model with RP/SP Data

2.1. Model Formulation The ML model is one of the most powerful models currently available. It allows us to treat unobserved response heterogeneity by assuming that it can be explained through an error term that measures the individual deviation from average population behaviour (Train 2003). The ML utility function is characterised by an error term with at least two components: one for obtaining the logit probability that has the usual extreme value type 1 (EV1) independent and identical distribution (i.i.d.); the second accounts for the different components of unobserved heterogeneity, the distribution of which can be chosen by

qjt

Vqjt

where bj is a vector of taste parameters that might vary over alternatives j but is fixed over the individuals and choice situations, xqjt is a vector of attributes (including level-of-service and socioeconomic variables and any form of interactions between them as well as alternative specific constants),
qjt is a vector of unobserved components of the individual utility that induce heterogeneity in tastes and choices, zqjt is a vector of attributes that might be known (i.e., equal to xqjt or unknown (and thus set equal to one) and qjt are EV1 error terms. The ML probability is the integral over all components of the vector
qjt of standard logit probabilities over a density of parameters f  qjt
, where denote the population parameters of the distribution   eVqjt +qjt  Pqjt = (2)  Vqit +qit f  qjt
d  ie Equation (2) may have different degrees of complexity depending on the structure of the unobserved components. The vector of unobserved components qit can be decomposed to better capture the different aspects of individual random heterogeneity. The most typical components are random coefficients and error components: Uqjt = kj + bjk xqjkt k


+



k




2.

(1)

Vqjt



qjk xqjkt + 

qjt

m

qm yjm +qjt 

(3)



where qjk and qm are individual parameters fixed over choice situations and randomly distributed with zero mean. yjm is an index that equals one if m appears in the utility function of alternative j and zero otherwise. The random coefficients (RC) component involves an error term that shares the vector of attributes with the systematic component of the utility. The “pure” error components (EC) are completely unknown terms that account for response heterogeneities specific to each alternative or group of alternatives (Walker, Ben-Akiva, and Bolduc 2007; Gopinath et al. 2005). As the vector xqjt includes level-of-service attributes, socioeconomic variables, and any interactions among them, Equation (3) allows us to account for systematic and random heterogeneity around the

Cherchi and Ortúzar: On the Use of Mixed RP/SP Models in Prediction

100

Transportation Science 45(1), pp. 98–108, © 2011 INFORMS

mean, nonlinearities in the level-of-service attributes, and different forms of correlation among alternatives. Depending on the specification of the covariance matrix, Equation (3) can also account for correlation among different parameters in the same “state” (i.e., same alternative and choice situation) and autocorrelation of the same parameter over choice situations (Gopinath et al. 2005). Apart from the above components, Greene and Hensher (2007) also include observed heterogeneity around the standard deviation of the random coefficients and error component effects, specifying the standard deviation of the random coefficients as a function of the individual socioeconomic characteristics. Using this popular model, sufficient evidence has been provided in the literature about the existence of several sources of heterogeneity. Although most applications refer only to SP data, the estimation of a logit kernel with multiple data sources does not add major difficulties. In fact, given two sources of data (say, one coming from an RP survey and the other from an SP one), Equation (3) can be specified for each data set as UqjRP = VqjRP + qjRP + qj j + RP qj

2 RP qj ∼ 0 RP 

SP SP SP Uqjt = Vqjt + qjt + qj j + SP qjt

2 SP qjt ∼ 0 SP 

(4)

where the elements (V RP + RP ) and (V SP + SP ) differ because of the data and might also include different attributes or random components. The term qj j allows accounting for the correlation between the choice made by each individual in the real world (RP) and the choices made in the hypothetical (SP) situations (Walker 2001, Bhat and Castelar 2002, Cantillo and Ortúzar 2005). In fact, qj is an i.i.d. standard normal term across alternatives but equal across responses (both RP and SP) of each individual while j is an unknown parameter that allows us to capture the correlation across responses from the same individual. Moreover, it is important to note that this commonality between RP and SP behaviour can be accounted for also in that part of the random heterogeneity associated with known attributes (i.e., in the random coefficients). In fact, although the RP and SP data differ in nature, the individual who evaluates each attribute in the SP context is the same one (with the same known or unknown socioeconomic characteristics) who reveals his current choice in the RP data. Therefore, to account for correlation across preferences for a RP RP RP given individual, we should have RP qjk xqjk = vqjk jk xqjk SP SP SP SP and qjk xqjkt = vqjk jk xqjkt , where as before qjk is a random term associated with the kth random coefficient and is independent across alternatives but equal across the responses of each individual. Finally, RP

and SP are EV1 i.i.d. distributed random terms associated with the RP and SP utilities, respectively, the variances of which (2RP and 2SP ) will be generally different. Let us call qjt any random terms other than the EV1; for example, we would have qjt = qj + qj j in Equation (4). In the ML specification, the variance associated with each data set is the sum of the EV1 term variance (which is inversely related to the MNL scale ) and the variance of all the random components into which  is decomposed: 2 ML = 2 +

2

62



(5)

However, the elements 2 of the matrix are parameters to be estimated (as well as the elements of the vector b). Thus, the scale parameter that allows obtaining the same variance in the RP and SP data sets is still given by !=

SP  RP

(6)

but we note that the scale parameter in an ML model should be larger than in an MNL model (Sillano and Ortúzar 2005). 2.2.

Estimating and Forecasting with the RP/SP Mixed Logit Model To estimate the joint ML model, we need to maximize a likelihood function such as (7), where the integral is over all values of the  parameters: L=

 

RP



RP V RP + RP qj qj

e

j∈IRP

RP V RP + RP qj qj

e

·

SP



SP ·f qjRP
f qjt
d RP d SP 



RP !V SP + SP qjt qjt

e

j∈ISP

RP !V SP + SP qjt qjt

e

(7)

Note that the likelihood function is scaled only by the unknown (inestimable) RP , the actual effect of which is to constrain the SP utility to the same scale parameter as the RP utility. Rescaling is based on the data enrichment paradigm, where it is implicitly assumed that the SP data are pooled to enrich RP information that may be deficient and the “true” parameters are invariant whatever the method used to elicit the individual preferences. However, there are several reasons why significantly different parameters might be found in the two sets of data. For fixed parameters, Louviere, Hensher, and Swait (2000) point to the error implicit in each data set because RP data often suffer from poorer quality or missing information, while SP data may fail to reflect real choices. Cherchi and Ortúzar (2006) add that differences in estimated values between the RP and SP data sets might be implicit in the need for SP data or even in what we are looking for when using

Cherchi and Ortúzar: On the Use of Mixed RP/SP Models in Prediction

101

Transportation Science 45(1), pp. 98–108, © 2011 INFORMS

SP data, especially when there is a need to test structural changes (i.e., departure from the actual market) in prediction and when there are nonlinear utilities in the attributes. Differences in individual behaviour in the two data sets are even more evident when we try to account for the several components of individual heterogeneity, be they systematic or random. To estimate complex behaviour, we need sufficiently exhaustive (and usually a great deal of) information. RP data sets are often small and too scanty to avoid empirical misspecification problems (Chiou and Walker 2007). Conversely, SP data sets are usually at least eight to nine times larger than RP data sets and contain richer information, not only because the trade-offs are controlled by the analyst but also because more than one piece of information is available for estimating individual tastes. In fact, the ability of the ML model to reproduce random heterogeneity increases with the quantity of repeated information (SP choice tasks or panel data) provided by each individual (Rose and Bliemer 2007; Cherchi and Ortúzar 2008). Moreover, because random heterogeneity is due to “something the modeller is omitting,” it is not surprising that RP and SP data, which differ essentially in the error term, might show different random heterogeneity. Interestingly, this rescaling can also be applied in the partial preference homogeneity approach as long as the estimation of at least one parameter can be improved by pooling the associated RP and SP attributes. The problem with this partial data enrichment approach arises in forecasting because only the RP environment should be considered no matter what data (RP or SP) are used for parameter estimation. In fact, when all attributes are generic for both sets, making predictions with a mixed RP/SP model does not create additional problems because the parameters are constrained to be the same in both environments. However, if different RP and SP parameters are estimated, it is crucial (i) to understand why such differences arise and (ii) to verify to what extent the lack of consistency between the estimated model and that used in prediction may affect the forecasting results. In particular, if different RP and SP parameters are estimated for a given attribute and both estimates are significant, Louviere, Hensher, and Swait (2000) suggest conducting tests with both to determine how sensitive the results are in each case and to use the parameter believed to be more reliable. Usually this is the parameter estimated with the SP data, but it is crucial to thoroughly check the data. Differences might occur because of different scenarios implicit in the two data sets or nonlinear effects in some attributes (Cherchi and Ortúzar 2006). In these cases, the choice of whether to use the RP or SP parameter should be based also on the analysis of the policies to

be tested. If a specific SP parameter is estimated, the problem is whether to use it in forecasting (because only RP data can be included) or not. Here, if we believe that the true phenomenon underlying both data sets is the same and that the problem resides only in the data gathered (i.e., missing or poor quality data), then the SP parameter should be used in forecasting (Bhat and Sardesai 2006; Brownstone, Bunch, and Train 2000). However, as discussed, if the differences are because of implicitly different scenarios in the two data sets, then the decision to include the parameters estimated with SP data alone in forecasting depends also upon the policies to be tested. Note that this latter case is always associated with nonlinear effects for some attributes—something very likely to occur. Other than these considerations, the key issue when parameters estimated only with SP data are used in forecasting is that the utility function specified for estimation is different from that used for prediction. This problem is particularly evident when we depart from the simple “linear in the attributes” with fixed parameter specification. This implies that model validation (i.e., satisfaction of microeconomic conditions and sensitivity tests) needs to be performed both on the estimated model (for both the RP and SP environments separately) and on the model to be used in forecasting. In fact, if the best RP/SP estimated model (from a statistical point of view) has the following utility specification: RP 2 UqjRP = bk Xkqj + RP RP qj ∼ 0 RP  qj  SP Uqjt

=

k=1K

k=1K

SP bk Xkqjt

 SP SP SP + SP X1qjt + bSE SEq X1qjt 

SP SP SP + b12 X1qjt X2qjt + SP qjt

(8)

2 SP qjt ∼ 0 SP 

then the microeconomic conditions on the marginal utilities (MU) of the estimated model are different in the two data sets because they are constant in the first case but vary among individuals and alternatives in the SP case: RP ∀ q j t MUXk=1 qj = b1 (9) SP SP SP SP SP MUXk=1 qj = b1 + + bSE SEq + b12 X2qjt  More importantly, they differ in estimation and prediction such that if the parameters estimated as specific for the SP data are moved into the RP domain, the specification used in the forecasting model should be that of Equation (10), and the microeconomic conditions must satisfy Equation (11):   RP RP SP RP RP Uqj = bk Xkqj + SP X1qj + bSE SEq X1qj k=1K

SP RP RP + b12 X1qj X2qj + RP qj  RP SP SP SP RP MUXk=1 qj = b1 + + bSE SEq + b12 X2qj 

(10) (11)

Cherchi and Ortúzar: On the Use of Mixed RP/SP Models in Prediction

102

Transportation Science 45(1), pp. 98–108, © 2011 INFORMS

However, because the RP and SP data differ, conditions (9) and (11) will give different results, and the sensitivity of the model will obviously change depending on whether or not the SP-specific parameters are included in prediction. Although the final decision on the structure of the forecasting model will be based on the analyst’s judgement, experience, and expertise, the above analyses on model consistency should aid in making decisions. It is also important to note that not only do the above microeconomic conditions vary among individuals but they also set the limits of the scenarios that can actually be tested. In fact, Equation (11) must be satisfied for all individuals and for any changes in the level-of-service attributes (X implied in each scenario tested. Finally, we would argue that this analysis is important when we estimate several components of heterogeneity (along with a careful analysis of whether each term really reveals that effect or something else) because there is ample evidence in the literature of estimated utility functions showing counterintuitive microeconomic behaviour and, proverbially, even oaks fall sometimes (Brownstone 2001, p. 115).

3.

Databank

The data used were collected in 1998 and refer to choice among car, bus, and train users for suburban trips in and out of Cagliari, the capital of Sardinia. The method used for building the data bank incorporated three kinds of surveys: a qualitative survey using focus groups for gaining a good understanding of the phenomenon, a revealed preference survey describing current trips, and a stated preference survey to evaluate the introduction of a new rail commuter option. In particular, two focus groups were conducted before designing the RP questionnaire and were crucial for gaining insight into the phenomenon and remaining confident about the estimation results. The RP survey was conducted on a sample of 300 households living in the corridor, randomly extracted from the telephone directory. Interviews comprised two parts: first, a 24-hour self-completion travel diary survey where up to 10 trips could be described in considerable detail and, second, a “general” section containing information about availability of alternative modes as well as socioeconomic information for each household member. Although the diaries were filled in personally by each respondent, the socioeconomic information was gathered by an interviewer partly on the first contact before delivering the diary and partly (mainly income data) when respondents were recalled after completion. Each household was contacted at least three times over a period of approximately one week. This approach

allowed us to check the first information gathered and to contact respondents again to correct or clarify unclear information and fill in any gaps. The third survey, SP data, was conducted on the same individuals who answered the RP questionnaire. A choice experiment between the current bus or car mode and a modified train service was set up. The SP design was customised to make it more realistic. The design included four variables at three levels (trip time, cost, frequency, and comfort) for bus users, and three variables at three levels (trip time, cost, and frequency) and one variable (comfort) at two levels for car users. Two-term interactions between cost, frequency, and travel time were considered, which allowed us to account for any nonadditive effects of those variables in the analysis. A block design was used to reduce the 27 hypothetical situations into groups of only 9 situations presented to different respondents. The order of presentation of the situations inside each block was randomised, and a second check was made to ensure that there were no dominant situations. Data were subject to strict quality screening. In particular, we excluded those observations for which the alternative chosen was objectively compulsory and those that corresponded to people who did not select the mode themselves or did not pay for their trips. The final sample (i.e., mixed RP/SP data set) of 1,396 observations used for model estimation was composed of 45% train users, 32% car users, and 24% bus users. More details can be found in Cherchi and Ortúzar (2002).

4.

Results

Using the data described in the previous section, we first estimated several models using the RP and SP data separately. For each data set, we searched for the best specification using nonlinear in the attributes utility functions (interactions between level of service variables and square terms) and testing for deterministic (i.e., depending on socioeconomic characteristics; see Ortúzar and Willumsen 2001, p. 261) as well as random taste heterogeneity. It is important to note that two dummy variables were used to specify the comfort variable: Comfort1 equals one if the level of comfort was poor and zero otherwise, and Comfort2 equals one if comfort was sufficient and zero otherwise. The “good” level was left as reference. It was also implicitly assumed that “car” had high comfort; thus, the comfort variable was only introduced in public transport (PT) alternatives. Moreover, we used an expenditure rate specification (Jara-Díaz and Farah 1987) where cost is divided by the expenditure rate g (i.e., income available to be spent in leisure time). Thus, in both the RP

Cherchi and Ortúzar: On the Use of Mixed RP/SP Models in Prediction Transportation Science 45(1), pp. 98–108, © 2011 INFORMS

and SP data sets, we also accounted for systematic heterogeneity around travel cost because of income and available time (Cherchi and Ortúzar 2002). Results were compared in order to identify candidate generic attributes in the joint RP/SP estimation. We found that all the fixed parameters (except that for Walking time) estimated separately in both cases only differed in scale; however, significant differences appeared in the case of parameters that were allowed to vary among individuals. Interestingly but not surprisingly, differences appeared for both systematic and random heterogeneity. In particular, when we estimated the two models separately (i.e., using just RP or just SP data), we found more sources of systematic heterogeneity in SP than in RP. In fact, in the SP data set, we found four significant interactions irrespective of the mode chosen (gender and walking time, gender and cost, students and the two comfort variables) and one interaction (specific only for the car alternative) between the variable car/licences (number of cars per household divided by the number of members with driving licences) and travel time by car. By contrast, in the RP data, we found only one source of generic systematic heterogeneity to be significant among alternatives (interaction between gender and walking time) but many (public transport or car) specific interactions. At the same time, in the RP set, we found that significant random heterogeneity was specific to travel time by car, although this did not account properly for taste heterogeneity but rather for differences between modes (Cherchi and Ortúzar 2003). For the SP data, instead, we found that travel time and the two comfort variables showed highly significant random taste heterogeneity while random heterogeneity around the alternative specific constants was not present. In particular, using the SP data, we found significant random heterogeneity in travel time, generic among alternatives, though the mean travel time parameter differed for public transport and car. To the best of our knowledge, differences in systematic and random heterogeneity have never been reported before and clearly show that although both data sets represent on average the same phenomenon, as long as we try to elicit the complexity of individual behaviour, the different nature of the two data sets becomes evident. This is because of the fact that the RP data set is usually much smaller than the SP one, but it also reveals the greater ability of repeated choices to account for individual differences in tastes and preferences. Moreover, this result also suggests that comparing RP and SP estimations, a vital preliminary step for the subsequent joint estimation, might be less enlightening for the systematic and random heterogeneity terms.

103 Now, although our sample is small and thus cannot support complex structures, the data is of high quality and this gives us confidence about the estimated results. At the same time, our SP data imply a different (in some cases, substantially different) scenario from the RP one, which may be the source of the above differences. When we performed a joint RP/SP estimation, we found that the entire systematic and random heterogeneities specific to the RP data lost significance while the effects estimated with the SP data were still highly significant. In Table 1, we can see that our sample shows systematic heterogeneity in several attributes but this can only be detected with the SP data. In particular, we found that students give less importance to the variation of comfort on board (for both levels of comfort) and men perceive a higher travel cost disutility than women. Men also seem to dislike more walking to get to a transit stop or station or to a car park. More interesting is the effect of Car/licences because in the joint RP/SP estimation, we found that when it was present we obtained evidence of systematic variation around travel time by car and around the alternative specific constant. The first effect can only be captured with the SP observations and the second only with the RP data. This result is different from what we found estimating models for the RP data alone because both types of heterogeneity were highly significant in that case (Cherchi and Ortúzar 2003). We first estimated a mixed RP/SP model with only systematic heterogeneity (Model NL in Table 1). Then, we tested different specifications for the random heterogeneity, either generic or specific, between the RP and SP data sets. When we performed the joint RP/SP estimation, we tested again if all coefficients could be random. The results confirmed the results estimated using one set of data at a time. Interestingly, we found that the RP and SP data sets provided the same estimate for the average taste parameters (apart from the different scale) but the heterogeneity in tastes could be elicited only with the SP data. In fact, the best mixed RP/SP model (Model ML2) was obtained with generic travel time parameters for RP and SP but specific for car and PT and by estimating the unobserved heterogeneity for travel time using just the SP data set. The same applies for the comfort variable. Model ML2 includes three random coefficients (travel time and two comfort variables) specific to the SP data set while, in Model ML1, they are estimated as generic for the RP and SP sets. We also tried to estimate a model with these three random coefficients specific to RP and SP data, but it did not converge basically because the only significant random variable in the RP data was travel time by car. The problem was caused by the randomness associated with travel time being generic among alternatives. We

Cherchi and Ortúzar: On the Use of Mixed RP/SP Models in Prediction

104

Transportation Science 45(1), pp. 98–108, © 2011 INFORMS

Table 1

Testing for Systematic and Random Heterogeneity and for Correlation Among SP Situations and Between RP and SP Data NL

ML1

Attributes

Mean

Mean

Travel time (PT ) Travel time (car ) Travel time

−0
0529 −1
9 −0
1274 −2
5

−0
0887 −2
95 −0
1887 −3
65

Cost/g

−0
0287 −2
8 0
5125 3
6 −1
261 −2
5 −4
116 −3
8 −1
946 −3
5 −0
1695 −2
0 7
270 2
6

−0
0379 −4
65 0
5036 5
84 −1
0384 −2
55 −3
7899 −6
25 −1
6008 −5
45 −0
2444 −3
16 11
2313 3
34

0
0011 2
9 −0
0082 −2
4 −0
9914 −2
8 1
9840 2
2 −0
2036 −3
3 −0
0276 −1
5

Frequency Transfer Comfort1 Comfort2 Early/Late (train ) Car/Licences (car-RP ) Travel time∗ Cost/g (SP ) Travel time∗ freq (SP ) K_train K_car Walking time (RP ) Walking time (SP ) Travel time (SP ) Comfort1 (SP )

ML2 St. dev.

Mean

St. dev.

−0
0809 −3
16 −0
1685 −3
47

∗

(Cost/g)∗ Gender b SP Walk. time∗ Gender b SP Travel time∗ Car/Lic (SP ) PT correlation SP scale c L(max ) 2 (C) a

Mean

St. dev.

−0
0928 −3
06 −0
1946 −3
75

0
1319 4
54

0
1408 4
74 −0
0341 −4
72 0
4923 6
20 −0
9789 −2
52 −3
6826 −6
75 −1
5891 −5
59 −0
1589 −2
62 8
9494 3
33

−0
0393 −4
71 0
5250 6
12 −1
0157 −2
52 −3
8396 −6
35 −1
5940 −5
38 −0
2472 −2
94 10
3680 3
35

0
0015 4
20

0
0014 4
17

0
0016 4
38

−0
0095 −3
19 −1
0804 −3
46 2
2944 2
63 −0
2746 −3
83 −0
0173 −0
91

−0
0092 −3
38 −1
0317 −3
48 2
1293 2
68 −0
2003 −3
94 −0
0169 −0
91

−0
0103 −3
61 −1
0991 −3
51 2
4070 2
80 −0
2413 −3
45 −0
0153 −0
81

1
3373 4
63 1
0919 4
74

1
2889 4
62 1
0959 4
80

0
1347 4
54 −1
3882 −4
58 −1
0921 −4
75

Comfort2 (SP ) Comfort1 Student a Comfort2 ∗ Student a

ML3

1
5230 2
7 0
8721 2
1 −0
0491 −2
5

1
6486 2
88 0
7225 1
81 −0
0449 −2
70

1
5829 3
09 0
7388 1
88 −0
0457 −2
74

1
6399 2
90 0
7233 1
81 −0
0456 −2
73

−0
0554 −2
4

−0
0480 −2
63

−0
0483 −2
66

−0
0492 −2
69

−0
1053 −2
7 0
3529 5
22∗∗

−0
0738 −2
63

−0
0767 −2
79

−0
0746 −2
66

0
657 1
96 −713
5147 0
2830

6
3606 3
19 1
05 0
22

5
1144 3
29 0
95 0
38

−678.371 0.3183

5
7547 3
00 1
02 0
21

−676.677 0.3200

Student = 1 if the individual is a student at any level; b gender = 1 if male; c t-test with respect to one.

−676.853 0.3198

Cherchi and Ortúzar: On the Use of Mixed RP/SP Models in Prediction

105

Transportation Science 45(1), pp. 98–108, © 2011 INFORMS

Table 2

Percentage of Individuals in the RP Environment That Does Not Fulfil the Microeconomic Conditions Assuming the scenarios implicit in the SP environment

Marginal utilities of travel time > 0 By train By car By bus

Actual situation ML1 (%) 23 3 10

ML2_a (%) 18 5 10

Frequency by train 5 min ML2_b (%) 37 14 25

ML1 (%) 2 3 10

ML2_a (%) 2 5 10

Cost by train −20%

ML2_b (%)

ML2_b (%)

20 14 25

17 14 25

Notes. ML_a: includes level-of-service interactions and systematic heterogeneity estimated with SP data alone. ML_b: also includes random heterogeneity estimated with SP data alone.

tested other models with random heterogeneity specific between RP and SP and specific for travel time by car and by public transport; we found that we could only account for heterogeneity with the SP data. This effect can be clearly appreciated in comparing Models ML1 and ML2. Although they cannot be compared with the likelihood ratio (LR) test, it is evident that the latter is superior (the overall log-likelihood is greater and all the t-statistics are higher).1 It is also worth noting that when we account for random heterogeneity (as in Models ML1 and ML2), the SP scale parameter is not significantly different from one (as was the case in a nested logit model). This result is not surprising and it is in line with findings by other authors (Bhat and Castelar 2002; Brownstone, Bunch, and Train 2000). Finally, in Models ML1 and ML2, we accounted for correlation across preferences in the SP choice situations, which yielded much better results than in models estimated assuming that the SP choices were independent. Instead, Model ML3 was estimated accounting for correlation across preferences between the RP and SP sets as discussed in §2.2 It is interesting to note that Model ML3 is superior to Model ML1 (which has the same utility specification but does not account for correlation between RP and SP data) while it is equivalent to Model ML2 (almost the same loglikelihood). Thus, randomness in tastes appears to be only because of the SP data and this result does not change if we extend the correlation among individuals across the different data sets. Note also that model ML2 does not change if we account for this correlation between RP and SP because the random coefficients are specific only to SP data. No other significant components of heterogeneity were found in our models. 1 Note that to confirm that randomness in tastes is only because of the SP data, it is sufficient that ML1 be not superior to ML2. 2 Note that though Model ML1 only accounts for correlation among the SP responses, Model ML3 also accounts for the correlation among the RP and SP observations that pertain to the same individual. Accounting for this correlation does not imply the estimation of additional parameters but is only a different way of associating the random draws to each observation when estimating the standard deviation of the random parameters.

In order to use a mixed RP/SP model in forecasting, all information must be moved into the RP environment but the model should also include the systematic and random heterogeneity estimated with the SP data alone. However, as discussed previously, whether to move the attributes estimated as specific to the SP data and which attributes to move depends also on the requirement that the model is reasonable and satisfies the usual microeconomic conditions. Quite different results might be obtained depending on how we specify utility in prediction. Table 2 compares Models ML1 and ML2 (illustrated in Table 1) on the basis of their ability to fulfil the microeconomic conditions in the actual situation (i.e., the estimated model) and in prediction (i.e., under some policy). In particular, we tested two ways of using Model ML2 in forecasting: (i) Model ML2_a includes all the estimated parameters, either RP/SP generic or only SP specific, with the exception of the random heterogeneity estimated using SP data alone; (ii) Model ML2_b includes all the parameters estimated in the joint RP/SP model with no exceptions. Note that when the models are applied to reproduce the actual situation, they all show a considerable percentage of individuals with incorrect marginal utility of travel time; this percentage increases when random heterogeneity (Model ML2_b) is included in forecasting in addition to systematic heterogeneity (Model ML2_a). However, it is also clear that this effect is because of the fact that when we move the parameters estimated only with SP data to the RP domain, we are not accounting for the different scenarios implicit in the SP domain. In fact, when those scenarios are explicitly considered (last four columns of Table 2), the percentage of individuals that do not fulfil the microeconomic conditions diminishes drastically. This result confirms the importance of clearly exploring the differences implicit in the RP and SP data sets and demonstrates how different the results might be if this effect was disregarded. The limits in terms of scenarios that can be predicted is another element to be considered in deciding whether parameters estimated specific for the SP data should be used in forecasting. Table 3 shows the maximum values that each attribute can assume in order to

Cherchi and Ortúzar: On the Use of Mixed RP/SP Models in Prediction

106 Table 3

Transportation Science 45(1), pp. 98–108, © 2011 INFORMS

Maximum Values That the Attributes Can Take to Satisfy the Microeconomic Conditions on Marginal Utility ML1

MU(travel time PT ) < 0

Frequency = 1 Frequency = 6 Frequency = 12

MU(travel time Car ) < 0 Cars/licences = 0
5 Cars/licences = 1
0 MU(cost ) < 0 MU(frequency ) > 0

Female Male

ML2

Cost/g