FACULTEIT ECONOMIE EN BEDRIJFSKUNDE HOVENIERSBERG 24 B-9000 GENT Tel. Fax.
: 32 - (0)9 – 264.34.61 : 32 - (0)9 – 264.35.92
WORKING PAPER Wrapped Input Selection using Multilayer Perceptrons for Repeat-Purchase Modeling in Direct Marketing
Stijn Viaene1, Bart Baesens1, Dirk Van den Poel2, Guido Dedene1 & Jan Vanthienen1 1
K.U.Leuven, Dept. of Applied Economic Sciences, Naamsestraat 69, B-3000 Leuven, Belgium. 2
Ghent University, Dept. of Marketing, Hoveniersberg 24, B-9000 Ghent, Belgium.
June 2001 2001/102
Corresponding author: Stijn Viaene, Dept. of Applied Economic Sciences, Katholieke Universiteit Leuven, Naamsestraat 69, B-3000 Leuven, Belgium. Phone: ++32 (16) 32 68 90, Fax: ++32 (16) 32 67 32, e-mail:
[email protected]. Sponsor : Part of this work was carried out in the context of the KBC Insurance Research Chair that was set up in September 1997 as a pioneering research co-operation between the Leuven Institute for Research on Information Systems (LIRIS) and the KBC bank and insurance group, which is one of the larger super regional bank and insurance groups in the Benelux (Europe) with head office in Belgium.
D/2001/7012/03
Wrapped Input Selection using Multilayer Perceptrons for Repeat-Purchase Modeling in Direct Marketing
Stijn Viaene1, Bart Baesens1, Dirk Van den Poel2, Guido Dedene1 & Jan Vanthienen1 1
K.U.Leuven, Dept. of Applied Economic Sciences, Naamsestraat 69, B-3000 Leuven, Belgium.
2
Ghent University, Dept. of Marketing, Hoveniersberg 24, B-9000 Ghent, Belgium.
Abstract
In this paper, we try to validate existing theory on and develop additional insight into repeat-purchase behavior in a direct marketing setting by means of an illuminating case study. The case involves the detection and qualification of the most relevant RFM (Recency, Frequency and Monetary) variables, using a neural network wrapper as our input pruning method. Results indicate that elimination of redundant and/or irrelevant inputs by means of the discussed input selection method allows us to significantly reduce model complexity without degrading the predictive generalization ability. It is precisely this issue that will enable us to infer some interesting marketing conclusions concerning the relative importance of the RFM predictor categories and their operationalizations. The empirical findings highlight the importance of a combined use of RFM variables in predicting repeat-purchase behavior. However, the study also reveals the dominant role of the frequency category. Results indicate that a model including only frequency variables still yields satisfactory classification accuracy compared to the optimally reduced model.
Keywords: direct marketing, multilayer perceptrons, input selection, response modeling, classification.
2
1
Introduction
It need not be emphasized that customer retention is at least as important as customer acquisition in the current context of competitive markets, not in the least for mail-order companies. Service providers are finding themselves in mature markets in which they have to switch their marketing efforts from acquiring new customers towards retention of existing customers (Grant, 1995; Stone, 1996). Customer relations undoubtedly represent an important opportunity cost: 5% fewer customers may result in profit losses between 25% and 85% (Reichheld, 1990). The objective of this paper is to validate existing theory on and develop additional insight into repeat-purchase behavior in a direct mail setting. The empirical study focuses on the repeat-purchase-incidence. It studies the issue whether or not an existing customer, i.e. a customer who has bought before, purchases a product listed in the direct mail catalogue within a limited time frame. As to the choice of the independent variables, the set-up of the experiment is limited to assessing the predictive importance of several alternative operationalizations of the traditionally discussed (R)ecency, (F)requency and (M)onetary predictor categories. This choice is motivated by the fact that most previous research cites them as being most predictive and because they are internally available at very low cost (Bauer, 1988; Bult, 1995). For modeling repeat-purchase behavior, several techniques have already been proposed and operationalized. To date, most research on this topic uses traditional statistical models. Examples include logit, probit and discriminant analysis. In this paper we use Multilayer Perceptron (MLP) neural networks as our baseline modeling technique. As universal approximators, MLPs have shown to be very promising supervised learning tools for modeling non-linear relationships (Bishop, 1995; Chang, 1996; Lacher, 1995; Mobley, 2000; Piramuthu, 1999; Sharda, 1996). This, especially in situations where one is confronted with a lack of domain knowledge, which in turn prevents any valid argumentation to be made concerning model selection bias on the basis of prior knowledge. The experiment rests upon the application of a step-wise, MLP-based input pruning method to a carefully gathered, real-life mail-order data set. The case study data consists of a detailed sample of 1,200 data points obtained from a Belgian mail-order company. It is shown that by making use of the discussed input selection method, model complexity can be significantly reduced without degrading the predictive generalization ability. It is precisely this issue that will allow us to infer some interesting
3
marketing conclusions concerning the relative importance of the RFM predictor categories. As argued in Van den Poel (1999), this has never been thoroughly investigated, let alone in the context of connectionist modeling. Bass and Wind (1995) cite the confidentiality of the data, resulting from the high business value of information in this area, as one of the major obstacles. This paper is organized as follows. In Section 2, we provide a concise overview of response modeling issues in direct marketing and discuss the data set used in the experiments. Section 3 presents the theoretical underpinnings of the use of MLPs for classification. The basic experimental set-up is outlined in Section 4. In Section 5 we present the outline of the step-wise, MLP-based input pruning wrapper method and its application to the purchase-incidence case at hand. Section 6 offers additional interpretation and a critical contemplation of the results that were obtained.
2
Response modeling in direct marketing
In the first subsection, we briefly elaborate on some response modeling issues typical of direct marketing. Hereby, we position both the nature of the problem statement and the chosen variable categories, in casu the response model and the RFM predictors. In subsection 2.2, we comment on the study data sample used in the experiment reported on in this paper.
2.1
Setting the stage
Cullinan (1977) is generally credited for identifying the three sets of variables most often used in database marketing: (R)ecency, (F)requency and (M)onetary (Bauer, 1988; Kestnbaum, 1992). Since then, the literature has accumulated so many uses of these three variable categories, that there is overwhelming evidence, both from academic studies as from practitioners’ experience, that RFM variables are among the most important predictor categories for modeling mail-order repeat-purchasing. However, when browsing the vast amount of literature, it becomes evident that only very limited attention has been devoted to selecting the right set of operationalizations of RFM variables to include into the model of mail-order repeat-buying. In fact, most studies do not offer a formal justification of their choice
4
of variables, which is therefore often of an ad-hoc nature. Instead, the focus of these articles lies mostly on selecting the appropriate modeling technique. For mail-order response modeling, several alternative problem formulations have been proposed based on the choice of the dependent variable. The first category is purchase-incidence modeling (Bult, 1993). In this problem formulation, the main question is whether a customer will purchase during the next mailing period, i.e. one tries to predict the purchase-incidence within a fixed time interval. Other authors have investigated related problems dealing with both the purchase-incidence and the amount of purchase in a joint model (Levin, 1998; Van der Scheer, 1998). A third alternative perspective for response modeling is to model inter-purchase time through survival analysis or (split-) hazard rate models (Dekimpe, 1997; Van den Poel, 1998) which model whether a purchase takes place together with the duration of time until a purchase occurs. This paper focuses on purchase-incidence modeling in a customer retention context.
More
specifically, we consider the issue whether an existing customer, i.e. a customer who has bought before, purchases a product offered by the mail-order company within a restricted time window. This choice is motivated by the fact that the majority of previous research in the direct marketing literature focuses on the purchase-incidence problem (Nash, 1994; Zahavi, 1997). Furthermore, this is exactly the setting that mail-order companies are typically confronted with. They have to decide whether or not a specific mailorder offering will be sent to a customer during a certain mailing period. More specifically, we will investigate how a wrapped, step-wise MLP-based input pruning method can assist in determining which purchase behavior variables may play a pivotal role in predicting repeatpurchase behavior by mail-order. The adoption of MLPs for modeling purposes is motivated by the fact that they are flexible, non-parametric modeling techniques, allowing to perform any complex function mapping with arbitrarily desired accuracy (Hornik, 1989). Moreover, as a direct consequence, their inherent ability to account for higher-order input interactions and locally predictive inputs makes them an excellent alternative for exploratory input selection purposes.
5
2.2
Data description
From a major European mail-order company, we obtained Belgian data on past purchase behavior at the order-line level, i.e. we know when a customer purchased what quantity of a particular product at what price as part of what order. This allowed us, in close co-operation with domain experts and guided by the literature to derive all the necessary purchase behavior variables for a total sample size of 1,200 customers. For each customer, these variables were measured in the period between July 1st 1993 and June 30th 1997. The goal is to predict whether an existing customer will repurchase in the observation period between July 1st 1997 and December 31st 1997 using the information provided by the purchase behavior variables. This six-month running period for a catalogue is typical of most European generalpurpose mail-order companies. Our four-year purchase history compares favorably to the six-month prediction period (Van den Poel, 1999). Of the 1,200 customers, 37.6% of existing customers actually repurchased during the observation period. We used the customer as unit of analysis (as opposed to the household) because it corresponds to the level at which the results of the analysis are used, i.e. the individual customer will receive an individually addressed mail package. As a form of pre-processing, the few missing values were handled by the unconditional mean imputation procedure which replaces them by the average of the corresponding input over the whole data set (Little, 1992). The Recency, Frequency and Monetary variables have then been modeled as follows. Recency is operationalized as the number of days since the last purchase (Bauer, 1988).
An alternative
operationalization would be the number of consecutive mailings without response (Bult, 1995). As noted by Kestnbaum (Kestnbaum, 1979) predictor variables may have to be transformed to obtain their full predictive performance. Hence, we include the "Log" transformation of the recency variable as an additional input to the MLPs. We motivate the choice of the Log transformation as a means to reduce the skewness of the distribution of the original recency variable. Although in most studies no detailed results are reported, the frequency variable is generally considered to be the most important of the RFM predictors (Nash, 1994).
Frequency is usually
operationalized as the number of purchases made in a certain time period (Bauer, 1988; Bult, 1995). When considering time interval length, inclusion versus exclusion of returned items and order-line versus
6
order level processing, many combinations of these variables are possible. We decide to consider 2 levels for each factor, which results in a 2x2x2 design (i.e. 8 operationalizations) as indicated in Figure 1. In the frequency column, “Fr” refers to frequency. “Year” refers to the frequency during the last 12 months and “Hist” refers to the frequency during the whole customer history. ”NoRet” refers to the fact that returned items are omitted, whereas ”Returns” refers to the fact that returned merchandise are included in the count. ”Orderlines” refers to the fact that the frequency reflects a count of the number of order-lines and ”DiffOrders” refers to the fact that not order-lines but rather the number of different dates (i.e. the purchase occasions) on which orders are placed are counted. Monetary value can either be operationalized as (a) the total accumulated monetary amount of spending by a customer during a certain amount of time (Cullinan, 1977), (b) the highest transaction sale or (c) the average order size (Nash, 1994). In the monetary column of Figure 1, ”Mon” refers to monetary value. ”Year” refers to the monetary value accumulated over the last 12 months whereas ”Hist” refers to the monetary value accumulated over the whole customer history. ”Max” refers to the highest transaction sale over the whole customer history and ”Avg” refers to the average transaction order size over the whole customer history. ”NoRet” refers to the fact that returns are deleted before processing. Most authors do agree on the fact that the monetary value should reflect the total net dollars of past orders, i.e. sales from past orders minus the dollar value of returns and refunds (Bauer, 1988). Therefore we do not consider operationalizations including returns. As suggested by Shepard (1997), we again use the logarithmic transformation of all monetary variables to reduce the skewness in the distribution of values.
Figure 1: RFM operationalizations included in the data set.
3
Multilayer perceptrons for classification
Neural networks (NNs) have shown to be very promising supervised learning tools for modeling complex non-linear relationships (Bishop, 1995). They are designed to deal with both regression and classification tasks.
Since regression is beyond the scope of this paper, the discussion will be limited to the
7
classification problematic. As universal approximators, NNs can significantly improve the predictive accuracy of an inference model compared to statistical techniques that are linear in the model parameters (Hornik, 1989; Bishop, 1995). A NN is typically composed of an input layer, one or more hidden layers and an output layer, each consisting of several neurons. Each neuron processes its inputs and generates one output value that is transmitted to the neurons in the subsequent layer. In a multilayer perceptron (MLP), all neurons and layers are arranged in a feedforward manner. A three-layer MLP then performs the following non-linear function mapping:
y ( m ) = f 2 ( w2 f 1 ( w1 x ( m ) )) ,
(m)
where x ( m ) = ( x1
(m)
, x2
(m)
,..., x d
(1)
) is a d -dimensional input vector corresponding to a specific data
instance m ∈ [1..n] that is labeled by a target variable t ( m ) . w1 and w2 are weight vectors of the hidden and output layer, respectively, and y ( m ) is the MLP-produced output vector associated with the m th data instance.
f 1 and f 2 are termed transfer functions and essentially allow the network to perform
complex non-linear function mappings. For a binary classification problem, one commonly opts for a three-layer MLP with one output unit. It is then convenient to use the logistic function, i.e.
f(z)=
1 1 + exp( − z )
(2)
as the transfer function in the output layer ( f 2 ) , since its output is limited to a value within the range
[0..1] .
This allows the output y ( m ) of the MLP to be interpreted as a bayesian posterior probability
(Bishop, 1995). These probabilities may then be mapped to class labels using a threshold value of e.g. 0.5. The weight vectors w1 and w2 together make up the parameter vector w , which needs to be
{
}
estimated (learned) during a training process. Given a training data set D = x ( m ) , t ( m ) m = [1..n] , the
8
weight vector w of the MLP is randomly initialized and iteratively adjusted so as to minimize an objective function, typically a sum of squared errors
ED
1 = 2
å (t
E D , i.e.
2
n
(m)
−y
(m)
m =1
).
(3)
The backpropagation algorithm performs this minimization by using repeated evaluation of the gradient
E D and the chain rule of derivative calculus. Due to the problems of slow convergence and
relative inefficiency of this algorithm, new and improved optimization methods have been suggested. The reader is referred to Bishop (1995) for further details. In this paper, we will use the LevenbergMarquardt method to minimize the objective function F ( w ) , i.e.
F ( w ) = βE D + αE w ,
whereby typically E w =
1 2
(4)
å wi2
with i running over all elements of the weight vector w . The
i
inclusion of E w in F ( w ) constrains the size of the network weights w and is referred to as regularization. When the weights are kept small, the network response will be smooth so that the network is prevented from fitting the noise in the training data (Bishop, 1995; MacKay, 1992). There is no standard way to determine both parameters α and β . They are often tuned off-line. We adopt the evidence framework of MacKay (1992) to determine both α and β . In this bayesian framework, both α and β are interpreted as model parameters and are optimized on-line during the Levenberg-Marquardt optimization. This approach chooses α and β by maximizing the likelihood function P( D α , β , H ) with H being the functional form of the network. This is done using bayes' theorem which allows to map all prior assumptions (e.g. probability distributions) into posterior knowledge. For more mathematical details on bayesian learning, we refer to the work of MacKay (1992).
9
4
Basic experimental set-up
As a pre-processing stage to the training of the MLP, all 18 inputs are statistically normalized to a mean of zero and a standard deviation of one (Bishop, 1995) according to the following formula:
(m) − xi ( m ) xi ~ xi = , i ∈ 1..d and m ∈ 1..n σx i
[ ]
[ ].
(5)
All MLPs in this study have one hidden layer, influenced by theoretical works, which show that a single hidden layer is sufficient to approximate any complex non-linear function with any desired degree
[
]
of accuracy (Hornik, 1989). After experimentation within the range 1..10 , the number of hidden units
was set to 3. Both hidden and output units use logistic transfer functions. All analyses were conducted using the Neural Network 3.0 toolbox of the MatlabTM 5.2 workbench. Performance is measured by means of the Percentage Correctly Classified instances (PCC) and the Area under the Receiver Operating Curve (AUROC). The PCC represents the classification accuracy using a default threshold value of 0.5 to map the bayesian posterior output probabilities of the MLP into binary class labels. This implicitly assumes equal misclassification costs for false positive and false negative predictions (Provost, 1998). The Receiver Operating Characteristic (ROC) curve, on the other hand, is a 2-dimensional graphical illustration of the sensitivity ('true alarms') on the Y-as versus (1specificity) ('false alarms') on the X-axis for various values of the classification threshold (decision criterion) (Egan, 1975; Swets, 1982; Hanley, 1983). Remember that the sensitivity is the percentage of buyers that are correctly identified by the MLP, whereas the specificity is the percentage of non-buyers that are correctly identified by the MLP. The ROC curve basically illustrates the behavior of a classifier without regard to specific misclassification costs. Classifier A is superior to classifier B if the ROC curve associated with A is situated above that of B. However, both ROC curves may intersect making a comparison less obvious. This may be overcome by calculating the Area under the Receiver Operating Curve (AUROC) which is an overall performance measure that has achieved wide acceptance.
10
In our experiments, we use a 10-fold cross-validation procedure in which the data set consisting of 1,200 data instances is split into 10 mutually exclusive folds of equal size. 10-fold cross-validation is a re-sampling technique in which 10 classifiers, in our case MLPs, are trained each time using only 9 folds of the data (training folds) and the remaining fold (testing fold) for testing. As a result, all observations are used for training and each observation is used exactly once for testing.
PCCtest (PCCtrain) and
AUROCtest (AUROCtrain) are then computed by averaging the PCC and AUROC on the testing (training) folds over all 10 sub-experiments of the 10-fold cross-validation procedure.
5 5.1
Input selection experiment Input selection in a nutshell
Input selection is a commonly adhered technique to reduce model complexity. The goal is to find a reduced co-ordinate system that allows to project a data sample on a more compact representation. The general assumption underlying this operation and justifying it, is that the studied data sample approximately lies within the bounds of this reduced space. As such, models with fewer inputs are capable of improving both human understanding and computational performance. Moreover, elimination of redundant and/or irrelevant inputs may then also improve the predictive power of an algorithm (Bellman, 1961). Finding the optimal input subset to an induction algorithm from among a multitude of available predictors is a highly non-trivial problem. An optimal input subset can only be obtained when the input space is exhaustively searched. When k inputs are present, this would imply the need to evaluate
k 2 − 1 input subsets. Unfortunately, as k grows, this very quickly becomes computationally infeasible (John, 1994). For that reason, a heuristic search procedure through the vast search space is often preferred. Input selection can then either be performed as a pre-processing step and independent of the induction algorithm, or explicitly make use of it. The former approach is termed ‘filter’, the latter ‘wrapper’ (John, 1994). Filter methods operate independently of any learning algorithm. Undesirable inputs are filtered out of the data before induction commences. Filters typically make use of all the
11
available training data when selecting a subset of inputs. Among the well-known filter approaches are Focus (Almuallim, 1991) and Relief (Kira, 1992). Wrapper methods make use of the actual target learning algorithm to evaluate the usefulness of inputs. Typically the input evaluation heuristic that is used is based upon inspection of the trained parameters and/or comparison of predictive performance under different input subset contingencies. Input selection is then often performed in a sequential fashion, e.g. guided by a best-first input selection strategy. The backward selection scheme starts from a full input set and step-wise prunes input variables that are undesirable. The forward selection scheme starts from the empty input set and step-wise adds input variables that are desirable. Hybrids of the above also exist.
5.2
Step-wise pruning using a multilayer perceptron wrapper
In this paper, input selection is implemented using a typical wrapper approach with a best-first search heuristic guiding the backward search procedure towards the optimal input set (John, 1994). Starting with the full set, all inputs are pruned sequentially, i.e. one by one. We use multilayer perceptron neural networks as our baseline induction mechanism. Given an initial data set D with d inputs, we proceed in steps of training and input desirability evaluation. Figure 2 presents the outline of the procedure.
Figure 2: Outline of pruning step k of the backward input selection procedure.
In each step k ∈ [1..d ] of the backward input selection procedure, a 10-fold cross-validation
[
experiment is carried out. The desirability (importance) of each input i ∈ 1.. Fk
] of the input subset
Fk
at the start of step k of the pruning scheme is then assessed in two stages. In a first stage an assessment is made per sub-experiment of the 10-fold cross-validation, giving rise to 10 assessments S i1 , S i 2 ,..., S i10 per input i . More specifically, the assessment per sub-experiment is based on the use of (a) the 9 training data folds of that sub-experiment and (b) the multilayer perceptron trained on the latter data. In a subsequent stage, a global assessment of the desirability of an input i in pruning step k is made by aggregating all 10 assessments S i1 , S i 2 ,..., S i10 .
12
The evaluation of the desirability of the i th input in step k is then operationalized in the following sensitivity index ( S i ):
Si =
10
å S ij , where S ij = E D ( xij , w j ) − E D ( xij , w j ) .
(5)
i =1
E D stands for the sum of squared errors as defined in (3). The argument w j represents the trained
weight vector of the MLP for sub-experiment j . The argument x ij represents the i th input and the argument x ij its average over all training folds of sub-experiment j . The index S ij implements the concept of sensitivity of the MLP trained in sub-experiment j to the presence/absence of input i . The input is perturbed to its mean and the impact on the network output is computed in terms of a difference in E D . This evaluation essentially amounts to a strategy of constant substitution, treating the input to be neglected as missing by substituting its effect to its mean over the whole sample (Moody, 1991; Moody, 1992; Van De Laar, 1999). Notice that no retraining of the MLP is needed while computing these sensitivities. Remember, as a result of the 10-fold cross-validation set-up, each input i is characterized by 10 sensitivity indices, S i1 , S i 2 ,..., S i10 , one for each of the 10 sub-experiments. These S ij are summed per input i over all 10 sub-experiments, giving rise to S i . The input p with the lowest aggregated sensitivity index is then removed from further consideration in the next steps of the backward selection procedure. The concept of sensitivity of the model to the presence/absence of an input, as defined by the above sensitivity measure in (5), does not completely correspond to the concept of causal relevance of an input within the real, but unknown functional relationship. Interaction and correlation effects among inputs tend to obscure a rightful assessment of the causal relevance of an input. However, the step-wise nature of the input selection approach partially counters this. The way interaction effects are accounted for, is illustrated in the following example. Suppose two inputs are interacting in a significant way. Setting either one of these inputs to their mean will, by definition, destroy the interaction resulting in a large value for the aggregated sensitivity index of that input. Correlation effects are coped with according
13
to the following rationale. Consider the situation where two inputs are nearly perfectly correlated and at least one of them has an inherent significant causal contribution within the underlying functional relationship. At first sight, the MLP will seem individually insensitive to either one of these inputs since setting either one of them to their mean causes little information loss. Both inputs will show low aggregated sensitivity and be considered for elimination. Suppose that in step k , one of these inputs, i.e. per definition the one with the lowest aggregated sensitivity, is to be omitted. Due to the re-training of the MLPs in step k + 1 , the aggregated sensitivity of the other input will rise, thus clearly indicating its significance. Figure 3 provides an indication of the mean PCC and mean AUROC, averaged over all 10 sub-
experiments of the 10-fold cross-validation, for each step of the pruning procedure. Notice that the MLPs in step 19 have only the bias term as their input, yielding a mean PCC of approximately 62.4%, which equals to the majority prediction of non-buyers.
Figure 3: Mean training and test performance at each pruning step.
5.3
Determining the pruned input set
After having discussed the mechanics of the input pruning procedure, the question naturally arises where to situate the cut-off in order to determine the pruned input set. In deciding how much inputs to prune, a trade-off between model complexity and model accuracy must be evaluated, also referred to as the bias/variance trade-off (Friedman, 1997; Geman, 1992). Pruning more inputs results in more compact and more efficient models, but at the potential cost of a loss in predictive effectiveness. In the literature, several criteria (heuristics) have been devised to effectively cope with this model selection problem, among which the Network Information Criterion (Murata, 1994) and the Akaike Information Criterion (Akaike, 1974). In this paper, we will determine the cut-off point by means of a series of statistical hypothesis tests. The procedure is fairly straightforward and makes use of the step-wise nature and cross-validation set-up discussed in the previous sections. We start by identifying the top of the mean PCCtrain curve
14
which depicts the percentage correctly classified training instances averaged over all 10 sub-experiments for each step of the pruning procedure. Naive reasoning would then go for the input set at this point as the pruned input set. For the RFM case at hand, this would lay the cut-off at pruning step 1. The resulting 'reduced' input space would then consist of all inputs. However, the cut-off decision would then be purely based on a mean performance criterion evaluated on the training data. In order to take into account the beneficial effect of reduced model complexity, we proceed with a sequence of one-tailed t-tests in the following manner. In subsequent steps, we move along the mean PCCtrain curve, starting at pruning step 1 (i.e. maximum mean PCCtrain) and perform a series of one-tailed t-tests to determine the point at which the mean PCCtrain value decreases significantly (5% significance level) vis-à-vis the starting point i.e. pruning step 1. This procedure allows taking into account the variance of the PCCtrain values over all 10 crossvalidation sub-experiments per pruning step. Using this procedure for the RFM-case at hand, the cut-off is situated at pruning step 13. The pruned input set then consists of 6 inputs.
5.4
Discussion
It has to be clear that the wrapper procedure we presented in the previous subsections, is in fact a generic 'meta-level' input selection scheme in the sense that basically any classification mechanism could be plugged in to operationalize the sensitivity based input assessment.
As mentioned in previous
sections, we opted for the use of multilayer perceptron neural networks as our baseline classification mechanism. This choice is motivated by the fact that MLPs are flexible, non-parametric modeling techniques, allowing to perform any complex function mapping with arbitrarily desired accuracy (Hornik, 1989). They are among the best-suited techniques to account for higher-order input interactions and locally predictive inputs, which makes them an excellent choice for exploratory input selection purposes. Table 1 depicts the results of applying the input selection procedure described above to the RFM
case at hand. Both full and reduced model results are presented in terms of PCC and AUROC.
Table 1: Mean results contrasted for full and reduced model.
15
Observe from Table 1 how the suggested input selection procedure allows to significantly reduce the model complexity (from 18 to 6 inputs) without degrading the generalization behavior in terms of test set performance for both the PCC and AUROC. This clearly illustrates the usefulness of input selection for the RFM case. The convergent trend for both performance criteria provides additional confidence in the presented findings. The order in which the inputs are pruned is given in Table 2.
Table 2: Order of input removal of the step-wise pruning procedure.
Some interesting marketing conclusions can also be inferred. Among the 6 remaining inputs of the reduced model, predictors of all three input categories (Recency, Frequency and Monetary) are encountered. This clearly suggests that the combined use of all three RFM variable categories yields the richest model for repeat-purchase behavior. Notice the presence of the “Log(Recency)” input which clearly confirms that reducing the skewness of the Recency input by means of a logarithmic transformation augments its predictive capability (Kestnbaum, 1979). It must also be remarked that an input set consisting of only 4 inputs, in casu with only frequency variables, (pruning step 15 in Table 2) still yields a mean PCCtest of about 71.5% and a mean AUROCtest of about 75%. This clearly illustrates the importance of the frequency variables in predicting mail-order repeatpurchase behavior. This piece of empirical evidence supports the hypothesis that the frequency variable is to be considered the most important of the RFM predictors (Nash, 1994). Moreover, it shows that including four alternative operationalizations of the frequency variable results in very high predictive performance.
It highlights that not only recent purchase history data (as indicated by the "Year"
specification), but also cumulative information over the past four years (as indicated by the "Hist" specification) is relevant for predicting future repeat-purchasing behavior. A similar conclusion holds for the inclusion or exclusion of returned merchandise (“Returns” vs. “NoRet”) and whether orderline level data or data on order counts should be used (“Orderlines” vs. “DiffOrders”). The possibility of using several operationalizations of the same variable category has been overlooked in most existing research, which in most of the cases only includes one typical operationalization per R, F and M variable.
16
6
Summary and conclusions
In this paper, we studied a wrapped neural network input selection method in a direct marketing setting by means of an illuminating case study. The case involved the detection and qualification of the most relevant RFM (Recency, Frequency and Monetary) inputs.
Results indicate that elimination of
redundant/irrelevant inputs by means of the discussed input selection approach allows to significantly reduce model complexity without degrading generalization ability. It is precisely this element that allows to infer some interesting marketing conclusions concerning the relative importance of the RFM predictor categories. The empirical findings highlight the importance of a combined use of all three variable categories in predicting mail-order repeat-purchase behavior. However, the results also illustrate the dominant role of the frequency variable.
Even a model with only frequency variables still yields
satisfactory classification performance when compared to the optimally reduced model. Moreover, we show that the use of alternative operationalizations for the same variable category (in particular for the frequency category) is a fruitful pursuit in terms of obtaining higher predictive performance.
7
Bibliography
Akaike H. 1974. A New Look at the Statistical Model Identification. IEEE Transactions on Automatic Control AC-19: 716-723. Almuallim H, Dietterich TG. 1991. Learning with many irrelevant features. In Proceedings of the Ninth National Conference on Artificial Intelligence; 547-552. Bass FM, Wind J. 1995. Introduction to the special issue: empirical generalisations in marketing. Marketing Science. 14.(G1-G6). Bauer A. 1988. A direct mail customer purchase model. Journal of Direct Marketing. 2(3): 16-24. Bellman R. 1961. Adaptive Control Processes: A Guided Tour. Princeton University Press. Bishop CM. 1995. Neural networks for pattern recognition. Oxford University Press. Bult JR, Wansbeek TJ. 1995. Optimal selection for direct mail. Marketing Science. 14: 378-394. Bult JR. 1993. Target selection for direct marketing. Phd. Dissertation. Groningen University.
17
Chang LK, Han I, Kwon Y. 1996. Hybrid neural network models for bankruptcy predictions. Decision Support Systems. 18(1): 63-72. Cullinan GJ. 1977. Picking them by their batting averages’ recency-frequency-monetary method of controlling circulation. Manual release 2103. Direct Mail/Marketing Association. N.Y. Dekimpe MG, Degraeve Z. 1997. The attrition of volunteers. European Journal of Operational Research. 98: 37-51. Egan JP. 1975. Signal Detection Theory and ROC analysis, Series in Cognition and Perception, Academic Press, New York. Friedman J. 1997. On Bias, Variance, 0/1 Loss, and the Curse of Dimensionality. Data Mining and Knowledge Discovery 1: 55-77. Geman S, Bienenstock E, Doursat R. 1992. Neural Networks and the bias/variance dilemma. Neural Computation 4: 1-58. Grant AWH, Schlesinger LA. 1995. Realize Your Customers’ Full Profit Potential. Harvard Business Review. Sept-Oct: 59-72. Hanley JA, McNeil BJ. 1983. The meaning and use of the area under a receiver operating characteristic (roc) curve. Radiology 148: 839-843. Hornik K, Stinchcombe M, White H. 1989. Multilayer Feedforward Networks are Universal Approximators. Neural Networks 2: 359-366. John GW, Kohavi R, Pfleger K. 1994. Irrelevant Inputs and the Subset Selection Problem. In Machine Learning: Proceedings of the Eleventh International Conference; 121-129. Kestnbaum RD. 1979. Customer record predictive model for list segmentation, Manual Release 620.2, Direct Marketing Association, New York. Kestnbaum RD. 1992. Quantitative database methods. The Direct Marketing Handbook. Nash EL; 588-597. Kira K, Rendell LA. 1992. The Feature Selection Problem: Traditional Methods and a New Algorithm. In Proceedings of AAAI 1992; 129-134. Lacher RC, Coats PK, Sharma Shanker C, Fant LF. 1995. A neural network for classifying the financial health of a firm. European Journal of Operational Research. 85(1): 53-65. Levin N, Zahavi J. 1998. Continuous predictive modeling: a comparative analysis. Journal of Interactive Marketing. 12(2): 5-22.
18
Little RJA. 1992. Regression with missing x’s: a review. Journal of the American Statistical Association. 87(420): 1227-1237. MacKay DJC. 1992. Bayesian Methods for Adaptive Models, Phd. Dissertation, Computation and Neural Systems, California Institute of Technology, Pasadena, CA. Mobley BA, Schechter E, Moore WE, McKee PA, Eichner JE. 2000. Predictions of coronary artery stenosis by artificial neural network. Artificial Intelligence in Medicine. 18(3): 187-203. Moody J, Utans J. 1992. Principled Architecture Selection for Neural Networks: Application to Corporate Bond Rating Prediction. NIPS4; 683-690. Moody J. 1991. Note on Generalisation, Regularization and Architecture Selection. In Nonlinear Learning Systems: First IEEE-SP Workshop on Neural Networks for Signal Processing; 1-10. Murata N, Yoshizawa S, Amari S. 1994. Network Information Criterion - Determining the Number of Hidden Units for an Artificial Neural Network Model. IEEE Transactions on Neural Networks. 5: 865872. Nash EL. 1994. Direct marketing: strategy, planning, execution. 3rd edition. McGraw-Hill. New York. Piramuthu S. 1999. Financial credit-risk evaluation with neural and neurofuzzy systems. European Journal of Operational Research. 112(2): 310-321. Provost F, Fawcett T, Kohavi R. 1998. The case against accuracy estimation for comparing classifiers. In Proceedings of the Fifteenth International Conference on Machine Learning; 445-553. Reichheld FF, Sasser WE. 1990. Zero Defections: Quality comes to Service. Harvard Business Review. SeptOct: 301-307. Sharda R, Wilson R. 1996. Neural network experiments in business failures prediction: A review of predictive performance issues. International Journal of Computational Intelligence and Organizations. 1(2): 107-117. Shepard D. 1997. A primer on statistical modeling for database marketers. In Proceedings of the 19th NCDM Conference; 83-170. Stone M, Woodcock N. 1996. Relationship Marketing. Kogan Page. Swets JA., Pickett RM. 1982. Evaluation of Diagnostic Systems: Methods from Signal Detection Theory, Academic Press, New York.
19
Van De Laar P, Heskens T, Gielen S. 1999. Partial Retraining: A New Approach to Input Relevance Determination. International Journal of Neural Systems. 9(1): 75-85. Van den Poel D, Leunis J. 1998. Database marketing modeling for financial services using hazard rate models. International Review of Retail, Distribution and Consumer Research. 8 (2): 243-257. Van den Poel D. 1999. Response Modeling for Database Marketing using Binary Classification. Phd. Dissertation. K.U. Leuven. Faculty of Economic and Applied Economic Sciences. Van der Scheer HR. 1998. Quantitative approaches for profit maximization in direct marketing.
Phd.
Dissertation. Groningen University. Zahavi J, Levin N. 1997. Issues and problems in applying neural computing to target marketing. Journal of Direct Marketing. 11(4): 63-75.
Mean Results
Full Model
Reduced Model
PCCtrain
74.03%
73.30%
PCC test
71.33%
71.83%
AUROCtrain
78.14%
77.25%
AUROCtest
73.72%
74.68%
18
6
Inputs
Table 1: Mean results contrasted for full and reduced model.
20
Pruning Step
Input
Pruning Step
Input
1
FrHistNoRetOrderlines
10
FrYearNoRetDiffOrders
2
Recency
11
Log(MonAvgNoRet)
3
FrHistReturnsDiffOrders
12
Log(MonYearNoRet)
4
FrHistReturnsOrderlines
13
MonAvgNoRet
5
Log(MonHistNoRet)
14
Log(Recency)
6
MonYearNoRet
15
FrYearReturnsOrderlines
7
Log(MonMaxNoRet)
16
FrYearReturnsDiffOrders
8
MonMaxNoRet
17
FrYearNoRetOrderlines
9
MonHistNoRet
18
FrHistNoRetDiffOrders
Table 2: Order of input removal of the step-wise pruning procedure.
21
Recency
Frequency
Monetary
-
Recency
-
FrYearNoRetOrderlines
-
MonYearNoRet
-
Log(Recency)
-
FrYearNoRetDiffOrders
-
MonHistNoRet
-
FrYearReturnsOrderlines
-
MonMaxNoRet
-
FrYearReturnsDiffOrders
-
MonAvgNoRet
-
FrHistNoRetOrderlines
-
Log(MonYearNoRet)
-
FrHistNoRetDiffOrders
-
Log(MonHistNoRet)
-
FrHistReturnsOrderlines
-
Log(MonMaxNoRet)
-
FrHistReturnsDiffOrders
-
Log(MonAvgNoRet)
Figure 1: RFM operationalizations included in the data set.
22
PRUNING STEP k ∈ [1..d ]
[
∀ i ∈ 1.
Cross valid
Fk : start
[
input su
]
i ∈ 1.. Fk : index of
i
th
input in
10 − fold cross validation , i.e. 10 sub − experim
S ij : sensitivity of
input i in j
th
sub − experi
Testing fold Training fold
Figure 2: Outline of pruning step k of the backward input selection procedure.
10
Si = å
ì Fk +1 = Fk \ íinput î
F : number of inputs in k
[
]
∀ i ∈ 1.. Fk :
j=
s s = arg min ( i
23
Mean PCCtrain and PCCtest Curves
Mean AUROCtrain and AUROCtest Curves
0.76
0.85
Train 0.74
0.8
Train
Test 0.75
Mean AUROC
Mean PCC
0.72
0.7
0.68
Test 0.7
0.65
0.66
0.6
0.64
0.55
0.62 1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19
0.5 1
2
3
4
5
6
7
Pruning Step
Figure 3: Mean training and test performance at each pruning step.
8
9
10 11 12 13 14 15 16 17 18 19
Pruning Step
FACULTEIT ECONOMIE EN BEDRIJFSKUNDE HOVENIERSBERG 24 9000 GENT
WORKING PAPER SERIES
Tel. Fax.
: 32 - (0)9 – 264.34.61 : 32 - (0)9 – 264.35.92
1
94/01
L. GOUBERT, E. OMEY, The long-term labour demand and the role of productivity in manufacturing in eight OECD-countries, June 1994, 24 p.
94/02
F. HEYLEN, Social, economic and political institutions and taxes on labour, September 1994, 38 p. (published in Weltwirtschaftliches Archiv, 1995).
94/03
P. JOOS, H. OOGHE, Comparison between market determined and accounting determined measures of risk : an empirical assessment for the non-financial firms listed on the Brussels stock exhange, October 1994, 35 p.
94/04
R. VANDER VENNET, Market structure and operational efficiency a determinants of EC bank profitability, September 1994, 25 p. (published in Journal of International Financial Markets, Institutions and Money, 1994).
94/05
S. MANIGART, B. CLARYSSE, K. DEBACKERE, Entry despite the network : exploring the relationship between network structure and entry patterns in emergent organizational populations, December 1994, 39 p.
95/06
G. VAN HUFFEL, P. JOOS, H. OOGHE, Semi-annual earnings announcements and market reaction : some recent findings for a small capital market, February 1995, 23 p. (published in European Accounting Review, 1996).
95/07
H. SAPIENZA, S. MANIGART, W. VERMEIR, A comparison of venture capitalist governance and value-added in the U.S. and Western Europe, February 1995, 31 p. (published in Journal of Business Venturing, 1996).
95/08
F. HEYLEN, L. GOUBERT, E. OMEY, Unemployment in Europe : a problem of relative or aggregate demand shocks ? , March 1995, 16 p. (published in International Labour Review, 1996).
95/09
R. VANDER VENNET, The effect of mergers and acquisitions on the efficiency and profitability of EC credit institutions, April 1995, 35 p. (published in Journal of Banking and Finance, 1996).
95/10
P. VAN KENHOVE, A comparison between the "pick any" method of scaling and the semantic differential, April 1995, 14 p.
95/11
K. SCHOORS, Bad loans in transition economies, May 1995, 42 p.
95/12
P. JOOS, H. OOGHE, Problemen bij het opstellen van classificatiemodellen : een toepassing op commerciële kredietscoring bij financiële instellingen, Juni 1995, 39 p. (gepubliceerd in Tijdschrift voor Economie en Management, 1998).
95/13
I. DE BEELDE, The evolution of industrial accounting thought in Belgium in the first half of the 20th century. A textbook approach, July 1995, 29 p.
95/14
D. SCHOCKAERT, Japanse laagconjunctuur en vastgoedmarktontwikkelingen, Oktober 1995, 24 p. (gepubliceerd in Maandschrift Economie, 1996).
95/15
P. GEMMEL, R. VAN DIERDONCK, The design of a MRP-based hospital service requirements planning system : the impact of different sources of uncertainty, October 1995, October 1995, 23 p.
95/16
J. MATON, The Cape of Good Hope. Employment and income distribution in South Africa, September 1995, October 1995, 59 p.
96/17
D. WAEYTENS, Activity-based information in budgeting : the impact on information asymmetry, budget slack creation and related dysfunctional behaviors - a lab experiment, February 1996, 40 p.
96/18
R. SLAGMULDER, Using management control systems to achieve alignment between strategic investment decisions and strategy, February 1996, 36 p. (published in Management Accounting Research, 1997).
96/19
N. VALCKX, W. DE VIJLDER, Monetary policy and asset prices : a comparison of the Fed's announcement policies 1987-1995, March 1996, 19 p. (published in Bank- en Financiewezen, 1996).
96/20
S. VANDORPE, J. DENYS, E. OMEY, De arbeidsmarktintegratie van afgestudeerden uit TSO en BSO : een longitudinale studie, Mei 1996, 21 p. (gepubliceerd in Economisch en Sociaal Tijdschrift, 1997)
FACULTEIT ECONOMIE EN BEDRIJFSKUNDE HOVENIERSBERG 24 9000 GENT
Tel. Fax.
: 32 - (0)9 – 264.34.61 : 32 - (0)9 – 264.35.92
WORKING PAPER SERIES
2
96/21
N. VALCKX, Business cycle properties of financial indicators in Germany, October 1996, 29 p.
96/22
T. TERMOTE, De arbeidsmarktparticipatie van de vrouw, ontwikkeling van de dienstensector en werkgelegenheid, November 1996, 35 p.
97/23
M. VERHUE, Demand for unemployment insurance : a survey-based analysis, January 1997, 25 p.
97/24
R. VAN HOVE, R. FRAMBACH, P. VAN KENHOVE, The impact of physical attractiveness in advertising on consumer attitude : the role of product involvement, January 1997, 25 p.
97/25
I. DE BEELDE, Creating a profession 'out of nothing'. The case of the Belgian auditing profession, February 1997, 27 p.
97/26
L. GOUBERT, De flexibiliteit van de Belgische relatieve lonen, Maart 1997, 27 p.
97/27
S. MANIGART, K. DE WAELE, M. WRIGHT, K. ROBBIE, Venture capitalist's appraisal of investment projects : an empirical study in four European countries, March 1997, 18 p. (published in Entrepreneurship Theory & Practice, 1997).
97/28
P. DE PELSMACKER, J. VAN DEN BERGH, Advertising content and irritation. A Study of 226 TV commercials, April 1997, 27 p. (published in Journal of International Consumer Marketing, 1998).
97/29
R. VANDER VENNET, Determinants of EU bank takeovers : a logit analysis, April 1997, 23 p. (published as ‘Causes and consequences of EU bank takeovers’, in S. Eijffinger, K. Koedijk, M. Pagano and R. Portes (eds.), The Changing European Financial Landscape, CEPR, 1999).
97/30
R. COOPER, R. SLAGMULDER, Factors influencing the target costing process : lessons from Japanese practice, April 1997, 29 p.
97/31
E. SCHOKKAERT, M. VERHUE, E. OMEY, Individual preferences concerning unemployment compensation : insurance and solidarity, June 1997, 24 p.
97/32
F. HEYLEN, A contribution to the empirical analysis of the effects of fiscal consolidation : explanation of failure in Europe in the 1990s, June 1997, 30 p. (revised version, co-authored by G. Everaert, published in Public Choice, 2000).
97/33
R. FRAMBACH, E. NIJSSEN, Industrial pricing practices and determinants, June 1997, 33 p. (published in D. Thorne Leclair and M. Hartline (eds.), Marketing theory and applications, vol. 8, Proceedings AMA Winter Conference 1997).
97/34
I. DE BEELDE, An exploratory investigation of industry specialization of large audit firms, July 1997, 19 p. (published in International Journal of Accounting, 1997).
97/35
G. EVERAERT, Negative economic growth externalities from crumbling public investment in Europe : evidence based on a cross-section analysis for the OECD-countries, July 1997, 34 p.
97/36
M. VERHUE, E. SCHOKKAERT, E. OMEY, De kloof tussen laag- en hooggeschoolden en de politieke houdbaarheid van de Belgische werkloosheidsverzekering : een empirische analyse, augustus 1997, 30 p. (gepubliceerd in Economisch en Sociaal Tijdschrift, 1999).
97/37
J. CROMBEZ, R. VANDER VENNET, The performance of conditional betas on the Brussels Stock exchange, September 1997, 21 p. (published in Tijdschrift voor Economie en Management, 2000).
97/38
M. DEBRUYNE, R. FRAMBACH, Effective pricing of new industrial products, September 1997, 23 p. (published in D. Grewal and C. Pechmann (eds.), Marketing theory and applications, vol. 9, Proceedings AMA Winter Conference 1998).
97/39
J. ALBRECHT, Environmental policy and the inward investment position of US 'dirty' industries, October 1997, 20 p. (published in Intereconomics, 1998).
FACULTEIT ECONOMIE EN BEDRIJFSKUNDE HOVENIERSBERG 24 9000 GENT
Tel. Fax.
: 32 - (0)9 – 264.34.61 : 32 - (0)9 – 264.35.92
WORKING PAPER SERIES
3
97/40
A. DEHAENE, H. OOGHE, De disciplinering van het management : een literatuuroverzicht, oktober 1997, 28 p. (published in Economisch en Sociaal Tijdschrift, 2000).
97/41
G. PEERSMAN, The monetary transmission mechanism : empirical evidence for EU-countries, November 1997, 25 p.
97/42
S. MANIGART, K. DE WAELE, Choice dividends and contemporaneous earnings announcements in Belgium, November 1997, 25 p. (published in Cahiers Economiques de Bruxelles, 1999).
97/43
H. OOGHE, Financial Management Practices in China, December 1997, 24 p. (published in European Business Review, 1998).
98/44
B. CLARYSSE, R. VAN DIERDONCK, Inside the black box of innovation : strategic differences between SMEs, January 1998, 30 p.
98/45
B. CLARYSSE, K. DEBACKERE, P. TEMIN, Innovative productivity of US biopharmaceutical start-ups : insights from industrial organization and strategic management, January 1998, 27 p. (published in International Journal of Biotechnology, 2000).
98/46
R. VANDER VENNET, Convergence and the growth pattern of OECD bank markets, February 1998, 21 p. (forthcoming as ‘The law of proportionate effect and OECD bank sectors’ in Applied Economics, 2001).
98/47
B. CLARYSSE, U. MULDUR, Regional cohesion in Europe ? The role of EU RTD policy reconsidered, April 1998, 28 p. (published in Research Policy, 2000).
98/48
A. DEHAENE, H. OOGHE, Board composition, corporate performance and dividend policy, April 1998, 22 p.
98/49
P. JOOS, K. VANHOOF, H. OOGHE, N. SIERENS, Credit classification : a comparison of logit models and decision trees, May 1998, 15 p.
98/50
J. ALBRECHT, Environmental regulation, comparative advantage and the Porter hypothesis, May 1998, 35 p. (published in International Journal of Development Planning Literature, 1999)
98/51
S. VANDORPE, I. NICAISE, E. OMEY, ‘Work Sharing Insurance’ : the need for government support, June 1998, 20 p.
98/52
G. D. BRUTON, H. J. SAPIENZA, V. FRIED, S. MANIGART, U.S., European and Asian venture capitalists’ governance : are theories employed in the examination of U.S. entrepreneurship universally applicable ?, June 1998, 31 p.
98/53
S. MANIGART, K. DE WAELE, M. WRIGHT, K. ROBBIE, P. DESBRIERES, H. SAPIENZA, A. BEEKMAN, Determinants of required return in venture capital investments : a five country study, June 1998, 36 p. (forthcoming in Journal of Business Venturing, 2001)
98/54
J. BOUCKAERT, H. DEGRYSE, Price competition between an expert and a non-expert, June 1998, 29p. (published in International Journal of Industrial Organisation, 2000).
98/55
N. SCHILLEWAERT, F. LANGERAK, T. DUHAMEL, Non probability sampling for WWW surveys : a comparison of methods, June 1998, 12 p. (published in Journal of the Market Research Society, 1999).
98/56
F. HEYLEN. Monetaire Unie en arbeidsmarkt : reflecties over loonvorming en macro-economisch beleid, juni 1998, 15 p. (gepubliceerd in M. Eyskens e.a., De euro en de toekomst van het Europese maatschappijmodel, Intersentia, 1999).
98/57
G. EVERAERT, F. HEYLEN, Public capital and productivity growth in Belgium, July 1998, 20 p. (published in Economic Modelling, 2001).
98/58
G. PEERSMAN, F. SMETS, The Taylor rule : a useful monetary policy guide for the ECB ?, September 1998, 28 p. (published in International Finance, 1999).
FACULTEIT ECONOMIE EN BEDRIJFSKUNDE HOVENIERSBERG 24 9000 GENT
Tel. Fax.
: 32 - (0)9 – 264.34.61 : 32 - (0)9 – 264.35.92
WORKING PAPER SERIES
4
98/59
J. ALBRECHT, Environmental consumer subsidies and potential reductions of CO2 emissions, October 1998, 28 p.
98/60
K. SCHOORS, A payment system failure and its consequences for interrepublican trade in the former Soviet Union, December 1998, 31 p.
98/61
M. DE LOOF, Intragroup relations and the determinants of corporate liquid reserves : Belgian evidence, December 1998, 29 p. (published in European Financial Management, 2000).
98/62
P. VAN KENHOVE, W. VAN WATERSCHOOT, K. DE WULF, The impact of task definition on store choice and store-attribute saliences, December 1998, 16 p. (published in Journal of Retailing, 1999).
99/63
P. GEMMEL, F. BOURGONJON, Divergent perceptions of TQM implementation in hospitals, January 1999, 25 p. (forthcoming in Journal of Management in Medicine, 2000)
99/64
K. SCHOORS, The credit squeeze during Russia's early transition. A bank-based view, January 1999, 26 p.
99/65
G. EVERAERT, Shifts in balanced growth and public capital - an empirical analysis for Belgium, March 1999, 24 p.
99/66
M. DE LOOF, M. JEGERS, Trade Credit, Corporate Groups, and the Financing of Belgian Firms, March 1999, 31 p. (published in Journal of Business Finance and Accounting, 1999).
99/67
M. DE LOOF, I. VERSCHUEREN, Are leases and debt substitutes ? Evidence from Belgian firms, March 1999, 11 p. (published in Financial Management, 1999).
99/68
H. OOGHE, A. DEHAENE, De sociale balans in België : voorstel van analysemethode en toepassing op het boekjaar 1996, April 1999, 28 p. (gepubliceerd in Accountancy en Bedrijfskunde Kwartaalschrift, 1999).
99/69
J. BOUCKAERT, Monopolistic competition with a mail order business, May 1999, 9 p. (published in Economics Letters, 2000).
99/70
R. MOENAERT, F. CAELDRIES, A. LIEVENS, E. WOUTERS, Communication flows in international product innovation teams, June 1999, p. (published in Journal of Product Innovation Management, 2000).
99/71
G. EVERAERT, Infrequent large shocks to unemployment. New evidence on alternative persistence perspectives, July 1999, 28 p.
99/72
L. POZZI, Tax discounting and direct crowding-out in Belgium : implications for fiscal policy, August 1999, 21 p.
99/73
I. VERSCHUEREN, M. DE LOOF, Intragroup debt, intragroup guaranties and the capital structure of Belgian firms, August 1999, 26 p.
99/74
A. BOSMANS, P. VAN KENHOVE, P. VLERICK, H. HENDRICKX, Automatic Activation of the Self in a Persuasion Context , September 1999, 19 p. (forthcoming in Advances in Consumer Research, 2000).
99/75
I. DE BEELDE, S. COOREMAN, H. LEYDENS, Expectations of users of financial information with regard to the tasks carried out by auditors , October 1999, 17 p.
99/76
J. CHRISTIAENS, Converging new public management reforms and diverging accounting practices in Belgian local governments, October 1999, 26 p. (forthcoming in Financial Accountability & Management, 2001)
99/77
V. WEETS, Who will be the new auditor ?, October 1999, 22 p.
99/78
M. DEBRUYNE, R. MOENAERT, A. GRIFFIN, S. HART, E.J. HULTINK, H. ROBBEN, The impact of new product launch strategies on competitive reaction in industrial markets, November 1999, 25 p.
99/79
H. OOGHE, H. CLAUS, N. SIERENS, J. CAMERLYNCK, International comparison of failure prediction models from different countries: an empirical analysis, December 1999, 33 p.
FACULTEIT ECONOMIE EN BEDRIJFSKUNDE HOVENIERSBERG 24 9000 GENT
Tel. Fax.
: 32 - (0)9 – 264.34.61 : 32 - (0)9 – 264.35.92
WORKING PAPER SERIES
5
00/80
K. DE WULF, G. ODEKERKEN-SCHRÖDER, The influence of seller relationship orientation and buyer relationship proneness on trust, commitment, and behavioral loyalty in a consumer environment, January 2000, 27 p.
00/81
R. VANDER VENNET, Cost and profit efficiency of financial conglomerates and universal banks in Europe., February 2000, 33 p. (forthcoming in Journal of Money, Credit, and Banking, 2001)
00/82
J. BOUCKAERT, Bargaining in markets with simultaneous and sequential suppliers, April 2000, 23 p. (forthcoming in Journal of Economic Behavior and Organization, 2001)
00/83
N. HOUTHOOFD, A. HEENE, A systems view on what matters to excel, May 2000, 22 p.
00/84
D. VAN DE GAER, E. SCHOKKAERT, M. MARTINEZ, Three meanings of intergenerational mobility, May 2000, 20 p. (forthcoming in Economica, 2001)
00/85
G. DHAENE, E. SCHOKKAERT, C. VAN DE VOORDE, Best affine unbiased response decomposition, May 2000, 9 p.
00/86
D. BUYENS, A. DE VOS, The added value of the HR-department : empirical study and development of an integrated framework, June 2000, 37 p.
00/87
K. CAMPO, E. GIJSBRECHTS, P. NISOL, The impact of stock-outs on whether, how much and what to buy, June 2000, 50 p.
00/88
K. CAMPO, E. GIJSBRECHTS, P. NISOL, Towards understanding consumer response to stock-outs, June 2000, 40 p. (published in Journal of Retailing, 2000)
00/89
K. DE WULF, G. ODEKERKEN-SCHRÖDER, P. SCHUMACHER, Why it takes two to build succesful buyer-seller relationships July 2000, 31 p.
00/90
J. CROMBEZ, R. VANDER VENNET, Exact factor pricing in a European framework, September 2000, 38 p.
00/91
J. CAMERLYNCK, H. OOGHE, Pre-acquisition profile of privately held companies involved in takeovers : an empirical study, October 2000, 34 p.
00/92
K. DENECKER, S. VAN ASSCHE, J. CROMBEZ, R. VANDER VENNET, I. LEMAHIEU, Value-at-risk prediction using context modeling, November 2000, 24 p. (forthcoming in European Physical Journal B, 2001)
00/93
P. VAN KENHOVE, I. VERMEIR, S. VERNIERS, An empirical investigation of the relationships between ethical beliefs, ethical ideology, political preference and need for closure of Dutch-speaking consumers in Belgium, November 2000, 37 p. (forthcoming in Journal of Business Ethics, 2001)
00/94
P. VAN KENHOVE, K. WIJNEN, K. DE WULF, The influence of topic involvement on mail survey response behavior, November 2000, 40 p.
00/95
A. BOSMANS, P. VAN KENHOVE, P. VLERICK, H. HENDRICKX, The effect of mood on self-referencing in a persuasion context, November 2000, 26 p. (forthcoming in Advances in Consumer Research, 2001)
00/96
P. EVERAERT, G. BOËR, W. BRUGGEMAN, The Impact of Target Costing on Cost, Quality and Development Time of New Products: Conflicting Evidence from Lab Experiments, December 2000, 47 p.
00/97
G. EVERAERT, Balanced growth and public capital: An empirical analysis with I(2)-trends in capital stock data, December 2000, 29 p.
00/98
G. EVERAERT, F. HEYLEN, Public capital and labour market performance in Belgium, December 2000, 45 p.
00/99
G. DHAENE, O. SCAILLET, Reversed Score and Likelihood Ratio Tests, December 2000, 16 p.
FACULTEIT ECONOMIE EN BEDRIJFSKUNDE HOVENIERSBERG 24 9000 GENT
WORKING PAPER SERIES
Tel. Fax.
: 32 - (0)9 – 264.34.61 : 32 - (0)9 – 264.35.92
6
01/100 A. DE VOS, D. BUYENS, Managing the psychological contract of graduate recruits: a challenge for human resource management, January 2001, 35 p. 01/101 J. CHRISTIAENS, Financial Accounting Reform in Flemish Universities: An Empirical Study of the implementation, February 2001, 22 p.
