Computation of maximum score type estimators by mixed integer programming Kostas Florios1 and Spyros Skouras2 1
Laboratory of Industrial and Energy Economics, School of Chemical Engineering, National Technical University of Athens, Zographou Campus, 15780, Greece. E-mail:
[email protected] 2 Department of International and European Economic Studies, Athens University of Economics and Business, 76 Patision Street, 10434, Athens, Greece. E-mail:
[email protected]
Abstract: In this paper we show that optimization of a family of “maximum score type” estimators can be effectively reformulated as a Mixed Integer Programming (MIP) problem. The maximum score type family includes Manski’s classic maximum score estimator, but also more recently proposed estimators that optimize profit, utility and absolute deviation objective functions, all of which are similarly irregular in that they are discontinuous in model parameters. The MIP formulation of this family of estimators allows them to be computed exactly using standard commercial software packages such as GAMS. Applying the MIP algorithm to Horowitz’s (1993) transport choice model and data we find it compares very favorably to currently used alternatives. Furthermore, our exact estimates lead to a very different interpretation of this data showing that, at least in the context considered, economic interpretation is sensitive to choice of computation procedure. Finally, using simulated data to explore the domain of applicability of the MIP approach, we find it will be most effective in modeling data with 2-5 parameters and 250-1000 observations. Keywords: Maximum score, binary choice models, Mixed Integer Programming, disjunctive constraints JEL classification: C13; C14; C25; C44; R41
1. Introduction Various limited dependent variable models have been studied in the past in the context of semiparametric estimation. Such models are, for example, the binary choice (BC), the ordered response (OR) and the censored regression (CR) model. Some semiparametric estimators proposed in order to suit the aforementioned models are the maximum score/ profit/ utility (MS/P/U) (or max-score-type estimators in our analysis), the sum of absolute deviations (SAD) and the mode regression (MR) estimates, respectively. The computation of these estimators is difficult mainly due to the discontinuous non linear objective function that has to be optimized in every case. Furthermore, in practice, large samples may be necessary in order for these estimates to converge. The usual approach followed so far in these estimation problems is the smoothing of the discontinuous objective function. A new approach is proposed for these calculations based on Mixed Integer Programming (MIP). Computational results are presented for the maximum score estimator of the binary choice model. Theoretical results extend the framework to the maximum profit/ utility estimators of the binary choice model as well as to the ordered response and censored regression models and their corresponding estimators, SAD and MR, respectively.
The methodological framework which is proposed in this paper for the computation of max-score-type estimators is based on the Mixed Integer Programming optimization technique and specifically on disjunctive constraints (DC) modeling using binary variables. Instead of smoothing the problematic step function that is contained in the definition of the max-score-type estimators a different strategy is followed. A binary decision variable is introduced for every observation in the sample indicating whether the specific observation is successfully modeled by the regression equation. A careful model construction is pursued using, firstly, tight big M coefficients for the disjunctive constraints and, secondly, an efficient normalization approach that allows even tighter big M coefficients to be computed in general. Also, special care is taken so that the usual scaling that sets a parameter’s value equal to unity is imposed on a statistically significant regressor. Experimental evidence is provided for the binary choice model and the maximum score estimator of Manski. Our proposed approach is tested, firstly, on a previously published dataset which is called ‘Horowitz transport mode choice dataset’ and achieves to provide the exact parameter values, which are identical to those provided by another existing exact algorithm. Both algorithms resulted in the same score value of 90.8% which is the analytical maximum. The gain of our approach, in this case, is more efficient computation of more or less the same parameter values. A second detailed scalability and Monte Carlo analysis based on the same dataset of Horowitz, suggests that our method is at least as fast and accurate as the existing exact algorithm of Pinkse in a greater range of sample size and number of parameters values. Indeed, in most cases of this second analysis, our method is significantly faster, especially as the number of parameters increases. Our approach, nevertheless, seems to work well only when a good fit of the data generating process by the binary choice model is possible. For significantly lower maximum score values, e.g. 65%, a third distinct Monte Carlo analysis, with random data this time, suggested that our approach is not very practical when the maximum score value is rather low. Nevertheless, our model is an interesting equivalent to the existing exact algorithm of Pinkse for maximum score computations, that proved more practical in a real world application sample. Finally, one has to notice that our approach is exact in the sense that it provides the analytical estimate of the parameters, and not just a heuristic estimate like, for instance, the MSCORE algorithm of Manksi and Thompson. This fact, makes our approach of value, since only the exact estimates of parameters provide reliable estimation results, a fact that has limited practical applications of this class of estimators so far. As it can be conjectured from the experimental analysis, the analytical estimates are sometimes very different from the heuristic estimates and this is demonstrated in the Horowitz transport mode choice dataset. The paper is organized as follows. Section 2 presents the methodological framework of semiparametric estimation using Mixed Integer Programming models. Section 3 reviews several empirical applications of the Maximum Score Estimator of Manksi. Section 4 reports on computational experience obtained from a random data Monte Carlo Analysis. Section 5 presents results on a well known dataset which is called Horowitz transport mode choice dataset. Section 6 reports more results on this particular data generating process by means of a Monte Carlo Analysis. The paper ends with some concluding remarks in Section 7.
2
2. Methodological Framework The main argument of this paper is that several semiparametric estimators can be computed by solving a corresponding mixed integer programming (MIP) program. The motivation behind the use of MIP as an optimization approach is the guideline stated in [3]. According to this guideline, when a discontinuous nonlinear programming (DNLP) optimization problem is at hand, the researcher is strongly discouraged to optimize it in its current form and the use of binary variables is recommended to model non-smooth functions. Non-smooth functions, like the sign() function, play a key role in the formulation of the semiparametric estimators we are interested in. Consequently, one has to construct a proper MIP model which is equivalent to the initial DNLP optimization problem that computes the estimator. A formal mathematical definition of linear mixed integer programming (MIP) with 0 – 1 variables is the following [10] min cT x + d T y s.t. Ax + By ≤ b x ≥ 0, x ∈ X ⊆ R n y ∈ {0,1}q Where x is a vector of n continuous variables, y is a vector of q 0 – 1 variables, c,d are (n × 1) and (q × 1) vectors of parameters, A,B are matrices of appropriate dimension, b is a right hand side vector of p inequalities
Another formal mathematical definition of linear mixed-integer programming problems is [26] max{cx + hy : Ax + Gy ≤ b, x ∈ Z +n , y ∈ R+p }, Where Z +n is the set of nonnegative integral n-dimensional vectors, R+p is the set of nonnegative real p-dimensional vectors, and x = ( x1 ,..., xn ) and y = ( y1 ,..., y p ) are the variables or unknowns. An instance of the problem is specified by the data (c, h, A, G , b) , with c an n-vector, h a p-vector, A a m × n matrix, G a m × p matrix and b an m-vector. This problem is called mixed because of the presence of both integer and continuous (real) variables. In many models, the integer variables are used to represent logical relationships and therefore are constrained to equal 0 or 1. Thus we obtain the 0-1 MIP in which x ∈ Z +n is replaced by x ∈ {0,1}n Often, semiparametric estimators have a linear part and a discontinuous non linear part. A general strategy is to keep the linear part for the linear programming part of the mixed integer program and substitute the undesirable discontinuous function by at least as many binary variables as the sample size of the estimation. So, instead of a single discontinuous function in the formulation of the estimator we have several binary variables and a well behaved linear part. If the MIP model is constructed carefully it can be consistent and tractable. Furthermore, a key modeling question in our paper is disjunctive constraints modeling [26, 18, 31]. Modeling of disjunctions in an MIP model can be made in two
3
alternative ways [26, 31]: either with the big M formulation or the convex hull formulation. Disjunctive constraints are inequalities of the form ∨ [ Ai x ≤ bi ] i∈D
Where one inequality suffices to hold The big M formulation is Ai x ≤ bi + M i (1 − yi ) i ∈ D
∑y i∈D
i
=1
The difficulty is parameter Mi determination. Mi must be sufficiently large to render inequality redundant, but too large a value yields poor linear programming relaxation. The Convex-hull formulation is x = ∑ zi i∈D
Ai zi ≤ bi yi
∑y i∈D
i
i∈D
=1
0 ≤ zi ≤ Uyi
i∈D
yi = 0, 1 In our case, of the maximum score estimator, a generalized k out of m constraints disjunction has been employed which reduces to the above simpler equations in the special case when k=1. For the full k of m equations the following analysis holds [26]. A typical disjunctive set of constraints states that a point must satisfy at least k of m sets of linear constraints. The case of k=1 and m=2 is shown in Figure 1, where the feasible region is shaded. Suppose P i = { y ∈ R+p : Ai y ≤ bi , y ≤ d } for i = 1,..., m . Note that there is a vector ω such that, for all i, Ai y ≤ bi + ω is satisfied for any y, 0 ≤ y ≤ d . Hence there is a y contained in at least k of the sets P i if and only if the set Ai y ≤ bi + ω (1 − xi ), for i=1,...,m m
∑x
i
≥k
i=1
y≤d x ∈ {0,1}m , y ∈ R+p is nonempty. This follows since xi = 1 yields the constraint Ai y ≤ bi while xi = 0 yields the redundant constraints Ai y ≤ bi + ω .
4
Figure 1. Disjunctive set of constraints that states that a point must satisfy at least k of m sets of linear constraints. The case k=1, m=2 is shown, where the feasible region is shaded and non convex
A heuristic rule exists according to which the following trade-off exists. Big-M offers fewer variables but a weaker relaxation while convex hull offers a tighter relaxation at the cost of more variables [18, 31]. Our choice was the big M relaxation for k out of m disjunctive constraints modeling because it appeared simpler to implement, appeared earlier in the literature and is easier to understand than the convex hull relaxation. It is interesting to realize that mixed integer programming reformulations of apparently nonlinear programming problems are not a completely novel idea. Dantzig in two early papers [5, 6] showed how a variety of nonlinear and nonconvex optimization problems could be formulated as mixed-integer programs [26]. The key of mixed integer programming success in general is successful modeling. As the classical textbook of the field states [26]: ‘In integer programming, formulating a “good” model is of crucial importance to solving the model’. The subject of good model formulation is a major topic of the best textbooks [26, 32] and is closely related to the algorithms themselves [26]. Very good books that help construct a good MIP model are [26, 32]. • Key questions and answers about the proposed framework Why linear mixed integer programming? Because it is a proof technique that finds the global optimum of a problem, if applicable to the problem. Why branch and cut as the solution algorithm of linear MIP? Because it is the state of the art algorithm for linear mixed integer programming. Furthermore, we had access to a commercial high performance solver implementing branch and cut. • More specific algorithmic key questions Why k of m disjunctive constraints modeling? Because it is a natural enough representation of the maximum score principle and is well documented in the literature. It seemed better than an arbitrary piecewise linear approximation approach for the indicator function. Why big M relaxations (e.g. for k of m disjunctive constraints modeling)? Because it is a natural enough representation of the disjunctions. Also, tight big M coefficients can be effectively computed. Finally, this approach is better documented than convex hull relaxations in the literature, not to mention much simpler.
5
• Time complexity of several algorithms for the maximum score estimator Algorithm 1: Exact Computation of Pinkse [28] Algorithm 2: Great Circle Search of Manski [21, 28, 23] Algorithm 3: Naïve grid search Algorithm 4: Combinatorial algorithm MS-MIP (proposed) Algorithm 5: Combinatorial algorithm based on Algorithm 1 (future research) Algorithm 1 is a global optimization exact algorithm reported for solving instances of T=100 and p=4 in 1993 [28]. Two factors are given in [28] for the T and p complexity, namely (1 + ( K − 1) /( N + 2 − K )) for the change from (N, K-1) to (N+1, K-1) (e.g. N=T and K-1=p) and (( N + 1) / K − 1) for the change from (N, K-1) to (N, K). A complexity of O(C (T , p) ⋅ ( p 3 + T )) , where C (T , p ) = Combinations (T , p ) can be calculated. Algorithm 2 is a local optimization heuristic algorithm if p > 1 and a global optimization exact algorithm only if p = 1 . In the global case, which is the onedimensional search, a complexity of O(TlogT) is present. In the local case, complexity is rather low but only a local optimum is located. Algorithm 3 is not a global optimization algorithm in the strict sense. It has a complexity of O(T p ) , and despite this fact, it does not find the global optimum. Algorithm 4 is a global optimization exact algorithm (proposed in this paper) and seems to have a O(2T ) complexity. This is overcome by the branch and bound algorithm which is the second phase of the branch and cut algorithm. In practical instances (see Horowitz [14]) a much lower complexity is observed. Algorithm 5 is a global optimization exact algorithm (future research). It could be, in a sense, a hybrid of Algorithms 1 and 4. In the following we will show how five semiparametric estimators that are hard to compute due to the difficult discontinuous objective function which has to be optimized can be reformulated as MIP programs: these are the maximum score estimator of Manski [21], the maximum profit estimator of Skouras [27], the maximum utility estimator of Elliot and Lieli [9] (all three for the binary choice model), the sum of absolute deviations estimator of the ordered response model [28] and the mode regression estimator of the censored regression model [28].
2.1 Maximum score estimator of Manski The classical maximum score estimator of Manski is computed by the maximization of T
max
β 0 , β1 ,..., β p
z MSE = ∑ (2 yt − 1) ⋅ sgn( β 0 + β1 z1t + ... + β p z pt )
(MS)
t =1
Where yt is the binary endogenous variable and zkt , k = 1,..., p are the continuous exogenous variables of the binary choice model yt = 1( β 0 + β1 z1t + ... + β p z pt + ut ) (BC) Where ut is the unobservable variable. The reformulation of the (MS) estimator as an MIP problem follows the idea of disjunctive constraints modeling in [26]. The proper MIP is
6
max
T
∑x
t
(1)
(MS-MIP)
t=1
st p
(1 − 2 yt ) β 0 + ∑ β k ζ kt ≤ 0 + ωt (1 − xt ), t = 1,..., T
(2)
k =1
β ∈ B ⊂ R p , β 0 ∈ {−1,1}, x ∈ X = {0,1}T p
ωt = 1 + ∑ abs( zkt ) ⋅ sup(abs( β k )), t = 1,..., T k =1
⎧ − zkt , if yt = 1⎫ ⎬ , k = 1,..., p, t = 1,..., T ⎩ zkt , if yt = 0 ⎭ d k = sup(abs ( β k )), k = 1,..., p The raw data are yt and zkt as in the initial (MS) formulation. Auxiliary parameters of the MIP model are ωt and ζ kt . The decision variables are β , β 0 and x . The objective function to be maximized is the sum of all xt variables. The proposition that supports this reformulation can be found in [26], pp.12. Briefly, the logic behind the model is that the t point sign coincidence condition (e.g. constraint) between 2 yt − 1 and β 0 + β1 z1t + ... + β p z pt is relaxed by using T binary
ζ kt = ⎨
variables xt . These, when true impose the disjunctive constraint to the model but when false make the same constraint redundant. So, the count of disjunctive constraints that are imposed (i.e. sign coincidences) is the objective function and is by definition the max score of Manski. Observe that in the sole vectorized constraint of the model, which is also called disjunctive constraint, the sign of the coefficient for β 0 , (1 − 2 yt ) , is the negative of (2 yt − 1) in the definition of the maximum score estimator. This statement in addition to the auxiliary parameter ζ kt definition helps in removing the undesirable indicator function from the MIP model. Finally, ωt is the big M constant of the disjunctive constraint and its value has been calculated by taking the triangular inequality for the left hand side of constraint (2). It can be noted that the domain for β k is [-dk, dk], k = 1,..., p where dk is typically chosen between 10 and 50. Typically, in model MS-MIP, raw data zkt is transformed according to z RAW − μk zkt = kt , k = 1,..., p, t = 1,..., T
σk
so that tight ωt , t = 1,..., T , can be effectively computed by choosing dk only equal to 10 (see Example 3). If this so called ‘mi sigma normalization’ is not done, a naïve approach will use dk equal to 50 (see Example 1). For more information on modeling disjunctive constraints through the use of binary variables one may read the classical book of Wolsey and Nemhauser [26], pp.12.
7
2.2 Maximum profit estimator of Skouras First of all, the maximum profit estimator of Skouras [27] is defined for the binary choice model (BC). It is computed by the maximization of T
max
β 0 , β1 ,..., β p
z MPE = ∑ wt (2 yt − 1) ⋅ sgn( β 0 + β1 z1t + ... + β p z pt )
(MP)
t =1
Where yt is the binary endogenous variable and zkt , k = 1,..., p are the continuous exogenous variables of the binary choice model yt = 1( Rt + ut ) (BC) Rt = β 0 + β1 z1t + ... + β p z pt wt = abs ( Rt ) Where ut is the unobservable variable. The MP-MIP reformulation follows simply from MS-MIP max
T
∑w x
(1′ )
t t
(MP-MIP)
t=1
st p
(1 − 2 yt ) β 0 + ∑ β k ζ kt ≤ 0 + ωt (1 − xt ), t = 1,..., T
(2′)
k =1
β ∈ B ⊂ R p , β 0 ∈ {−1,1}, x ∈ X = {0,1}T p
ωt = 1 + ∑ abs( zkt ) ⋅ sup(abs( β k )), t = 1,..., T k =1
⎧ − zkt , if yt = 1⎫ ⎬ , k = 1,..., p, t = 1,..., T ⎩ zkt , if yt = 0 ⎭ d k = sup(abs( β k )), k = 1,..., p The only addition to MS-MIP is the presence of a weight coefficient wt , t = 1,..., T , which is data of the DGP, either on the inner part of the sum in MP equation or in the objective function of the MP-MIP model. The body of constraints for models MS-MIP and MP-MIP is the same.
ζ kt = ⎨
2.3 Maximum utility estimator of Elliot & Lieli Likewise, the maximum utility estimator of Elliot & Lieli [9] is defined for the binary choice model (BC). It is computed by the maximization of max
β 0 , β1 ,..., β p
T
z MUE = ∑ wt (2 yt − 1) ⋅ sgn( β 0 + h( Z t , β ))
(MU)
t =1
Where yt is the binary endogenous variable and zkt , k = 1,..., p are the continuous exogenous variables of the binary choice model yt = 1( Rt + ut ) (BC) Rt = β 0 + h( Zt , β ) wt = f ( Rt ) Where ut is the unobservable variable.
8
Iff f ( Rt ) = abs ( Rt ) AND h( Z t , β ) = zt′β ↔ MU ≡ MP The MU-MIP reformulation follows from MS-MIP (or MP-MIP) as well T
max
∑w x
t t
(1′′ )
(MU-MIP)
t=1
st (1 − 2 yt ) β 0 + (1 − 2 yt )h( Z t , β ) ≤ 0 + ωt (1 − xt ), t = 1,..., T
(2′′)
β ∈ B ⊂ R , β 0 ∈ {−1,1}, x ∈ X = {0,1} ωt = 1 + sup(abs(h( Z t , β ))), t = 1,..., T d k = sup(abs ( β k )), k = 1,..., p p
T
Now there are two additions to MS-MIP: First, is the presence of a weight coefficient wt , t = 1,..., T , which need not always be abs( Rt ) , as in maximum profit, but also, for instance, {abs ( Rt )}n , n = 2, 3 . Second, the function h need not be linear in β , and it could be, for instance, quadratic, convex or non-convex in β . The body of constraints for model MU-MIP differs, as the disjunctive constraint is based on the inequality of the h funtion, whose parametric form is known. A suitable, as tight as possible, ωt , t = 1,..., T , needs to be computed taking advantage of the form of h and the compactness of B . It is important to underline here, that model MU-MIP is actually the only model of this paper, which is, in fact, a Mixed Integer Non Linear Programming (MINLP ) model iff h is not linear in β . Of course, iff h is a quadratic in β then model MU-MIP is actually a Mixed Integer Quadratic Programming model (MIQP) and so on. The interested reader is referred to the book of Floudas [10] for general MINLP model formulations and solution methods.
2.4 Sum of absolute deviations estimator of the ordered response model The sum of absolute deviations estimator of the ordered response model [28] is computed by the minimization of L −1 1 N bSAD := arg min ∑ yi* − ∑ wl I ( xi′b > Al ) (SAD) N i =1 b∈R K l =0 Where the underlying ordered response model is briefly as follows yi* := l , A l-1 < yi ≤ Al , i=1,...,N, l=1,...,L. (OR) With yi defined by yi = xi′β + ui , i=1,...N The Ai’s are thresholds chosen beforehand to match the specific problem of the econometrician [28]. The wl’s are weight parameters of the model, often equal to unity. The minimization for the SAD estimator of the OR model can be formulated as a Mixed Integer Program. However, this formulation is not as elegant as in the maximum score estimator. This is justified, since the sum of absolute deviations estimator is more sophisticated than the maximum score estimator
9
Two issues arise in the modeling effort of the SAD estimator. • Use of disjunctive constraints as in maximum score to model logical or conditions • Use of goal programming to remove the absolute value operator from the MIP model For L=2 the sum of absolute deviations estimator of the ordered response model is simply the maximum score estimator of the binary choice model choosing A0=Infinity, A1=0 and A2=+Infinity. For L=3 the sum of absolute deviations estimator of the ordered response model has the following specific form assuming wl is equal to one for all l=0,…,L-1: 3−1 1 N * min ( abs y I ( xi′b − Al )) = − ∑ ∑ i b∈B Ν i=1 l =0
1 N ∑ abs(1+I (yi − A1 ) + I ( yi − A2 ) − I (xi′b − A0 ) − I ( xi′b − A1 ) − I ( xi′b − A2 ))= b∈B Ν i=1 1 N = min ∑ abs( I (yi − A1 ) + I ( yi − A2 ) − I ( xi′b − A1 ) − I ( xi′b − A2 )) b∈B N i=1 Let us now define a binary variable vli such that = min
vli : b satisfies inequalities xi′b − Al ≥ 0, l=1,2 if possible, k i times, whereas if not possible, to approach k i from both sides as close as possible, where k i is defined later on.
For L=3 we define for every i and l=1,2 in total 2N binary variables v1i = I ( xi′b − Al ) , v2i = I ( xi′b − A2 ) and the relevant parameters ki=I(yi-A1)+I(yi-A2), where I(x) is one if x is positive and is zero if x is negative or zero, thus is an ordinary indicator function. Observe that v1i and v2i belong in {0, 1} but ki belongs in {0, 1, 2}. Then it suffices to
minimize for every i=1, N, every absolute value abs (v1i + v2i − ki ) , because SAD is a sum of positive quantities (e.g. sum of all absolute values abs(v1+v2-k) in i’s ). For L>3 (more general case) we define vli = 1 iff xi′b − Al ≥ 0 iff -xi′b + Al ≤ 0 and vli = 0 iff xi′b − Al < 0 iff -xi′b + Al > 0 for all l=1,…,L-1 (case l=0 is redundant). The optimization problem of point i becomes For L=3 min di+ + di− (SAD-i-MIP) st v1i + v2i − di+ + di− = ki , i = 1,..., N p
−∑ xij b j + Al ≤ 0 + ωli (1 − vli ), i = 1,..., N , l = 1, 2 j =1
v1i , v2i ∈ {0,1}, di+ , di− ∈ R+ ki ∈ Z + ⊂ R+ a parameter ki = I ( yi − A1 ) + I ( yi − A2 ) A1, A2 are a priori parameters set by the econometrician. (SAD-i-MIP) sub problem follows a goal programming logic. The key though is the fact that v1i and v2i are binary variables, and ki is a natural parameter. So di+ and d i− implicitly are integers too, though that need not be declared in the model. Briefly, as
10
one can read i.e. in [25] the di+ deviation is the (undesirable) surplus and the d i− deviation is the (undesirable too) slack of the sum v1i + v2i compared to the goal ki . Because we model an absolute value behaviour we include both positive real variables di+ and d i− in the objective function of the goal programming method [25]. For L ≥ 3 the complete mixed integer programming model is as follows min
N
∑d
i +
+ d −i
(3)
(SAD-MIP)
i=1
st L −1
(∑ vli ) − d +i + d −i = k i , i = 1,..., N
(4)
l =1 p
−∑ x ij b j + Al ≤ 0 + ωli (1 − vli ), i = 1,..., N , l = 1,..., L − 1
(5)
j =1
b ∈ B ⊆ R p , v ∈{0,1}L −1 × {0,1}N , d + ∈ D+ ⊆ R+N , d − ∈ D− ⊆ R+N (implicitely d + ∈ D+ ⊆ Ζ +N and d − ∈ D− ⊆ Z +N ) Parameters are Al , l=1,…, L-1 set by the analyst (thresholds of the OR model) xij , j=1,…,p, i=1,…,N exogenous variables of the model L −1 ⎧ 1, x ≥ 0 ⎫ k i , k i = ∑ I ( yi − Al ), where I(x)= ⎨ ⎬ , i=1,…,N l =1 ⎩ 0, x 0 Proposition: A suitable big M coefficient ω g for g ≤ ω g , ∀x ∈ (−ε , ε ) is sup(abs ( g i ) = ε i +
Proof:
ωg : ωg − ε =
p ε i − x0 ( x − ε i ), ε i = ∑ abs ( xij ) ⋅ sup(abs (b j )) □. 2ε i j =1
ε − x0 ε − x0 ( x − ε ) or ωg = ε + (x − ε ) 2ε 2ε
But ε is taken from triangular inequality on x = xi′b, p
ε i = ∑ abs ( xij ) ⋅ sup(abs(b j )) . j =1
So, sup(abs ( gi )) = sup( gi ) = ωg = ε i +
ε i − x0 ( x − ε i ), with ε i computed from above□. 2ε i
31
