A polynomial case of cardinality constrained ... - Optimization Online

A polynomial case of cardinality constrained quadratic optimization problem∗ Jianjun Gao

†

Duan Li

‡

August 27, 2010

Abstract We investigate in this paper a fixed parameter polynomial algorithm for the cardinality constrained quadratic optimization problem, which is NP-hard in general. More specifically, we prove that, given a problem of size n, the number of decision variables, and s, the cardinality, if, for some 0 < k ≤ n, the n − k largest eigenvalues of the coefficient matrix of the problem are identical, we can construct a solution algorithm with computa
tional complexity of O n2k , which is independent of the cardinality s. Our main idea is to decompose the primary problem into several convex subproblems, while the total number of the subproblems is determined by the cell enumeration algorithm for hyperplane arrangement in Rk space.

Keywords: Cardinality constrained quadratic optimization, cell enumeration, nonconvex optimization, fixed parameter polynomial algorithm

1

Introduction

We consider in this paper the following cardinality constrained quadratic optimization problem (CCQO), 1 (P) : min f (x) = x′ Qx + q ′ x x ) (2 n X Subject to: x ∈ ∆(s) , x ∈ Rn | δ(xi ) ≤ s < T ,

(1)

t=1

where Q ∈ Rn×n is positive definite, q ∈ Rn \ {0}, and the indicator function δ(·) : R → {0, 1} is defined such that δ(a) = 1 if a is non-zero and δ(a) = 0 otherwise. ∗

This work was supported by Research Grants Council of Hong Kong, under grants 414207 and 414808. Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong, Shatin, Hong Kong ([email protected]) ‡ Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong, Shatin, Hong Kong.([email protected]). †

1

Polynomially solvable case of CCQP

2

This class of quadratic optimization problems with a cardinality constraint arises naturally from various applicationsThe exact and approximate solution approaches for cardinality constrained optimization problems are studied in the literatures, e.g., [2], [3], [9], [12]. To our best knowledge, only a few results on polynomially solvable cases of CCQO problem are reported in the literature. In the context of subset selection problem, Das and Kempe [5] proposed an approximate algorithm for the case where the covariance possesses a constant bandwidth and developed an exact algorithm for the case where the covariance graph is of a tree structure. Recently, Donoho and Candes [4][6] showed that, under some conditions, using l1 norm to replace the cardinality constraint in the sparse signal reconstruction problem yields the exact solution with an overwhelming probability. We focus in this paper on a class of CCQO problems with a special structure. More specifically, we consider situations where the n−k largest eigenvalues of matrix Q are identical, 0 ≤ k ≤ n, which we term as a matrix Q with a k-degree freedom (see Definition 2.1). We prove that, for fixed k, this class of CCQO problems is polynomially solvable. Motivated by the geometrical characteristics of CCQO problem, we decompose the problem (P) into several convex quadratic programming subproblems and the number of these sub-problems is determined by a cell enumeration algorithm for the hyperplane arrangement in Rk space [1]. From the complexity point of view, if k is fixed, the solution scheme is a polynomial-time algorithm. To certain extent, our result in this paper is similar to a polynomially solvable case in binary quadratic program, where the rank of coefficient matrix is fixed (see [8]). This paper is organized as following. After the introduction in this section, we develop the solution scheme for CCQO problem with a k-degree freedom coefficient matrix in Section 2. As the derived solution scheme depends heavily on a distance function between the cardinality feasible set and an affine space, we develop a scheme for identifying such a distance function in Section 3 using cell enumeration of hyperplane arrangement in discrete geometry. After presenting an illustrative example in Section 4, we conclude the paper in Section 5. Throughout the paper, we use v(·) to denote the optimal value of problem (·), S ≻ 0 a positive definite matrix, Sn++ the set of positive definite matrices, diag{a} ∈ Rn×n the diagonal matrix with a ∈ Rn being its diagonal, 0 the vector with all elements being 0 and k · k the l2 norm. Furthermore, we denote the ellipsoid and the ball in Rn , respectively, by  (2) E(P, p, ρ) , y ∈ Rn | (y − p)′ P (y − p) ≤ ρ , P ≻ 0, ρ ≥ 0,  2 n 2 2 B(p, r ) , y ∈ R | ky − pk ≤ r . (3)

2 2.1

Solution scheme of problem (P) Preliminary

Problem (P) has been proved to be, in general, NP-hard (see the proof in [11]). Here we give an alternative proof for the NP-hardness of problem (P), which appears to be much simpler than the one in [11]. Let us construct the following problem, n o (G) : minn fˆ := M kx − 1k2 + kAxk2 | x ∈ ∆(s) , x∈R

3

Polynomially solvable case of CCQP

where M > 0 is a large number, 1 is the vector with all elements being 1, and A ∈ Rl×n with l ≤ n. Note that any instance of problem (G) is polynomially reducible to an instance of problem (P). That is to say, solving problem (G) is no more difficult than solving problem (P). Since x ∈ ∆(s), minimizing the first term of (G) enforces xi to take either 0 or 1 for i = 1, · · · , n. More specifically, at least n − s of xi ’s are zero. Thus, the optimal value of problem G is lower bounded, i.e., v(G) ≥ M (n − s). Answering the question “whether equality v(G) = M (n − s) holds or not ” turns out P to find the integer (binary) solution of linear systems Ax = 0 such that x ∈ {0, 1}n and i=1 xi ≤ s, which is a known NP-complete decision problem [7]. Our conclusion for the NP-hardness of problem (P) follows the simple reduction method [7]. Geometrically, the objective contour of (P) is an ellipsoid in Rn space, E(Q, h, ρ) ,{x ∈ Rn | f (x) ≤ τ } ={x ∈ Rn | (x − h)′ Q(x − h) ≤ ρ}, where h , − Q−1 q, 1 C , − q ′ Q−1 q, 2 ρ , 2τ − 2C.

(4) (5) (6)

Clearly, we must have τ ≥ C. Minimizing f (x) under constraint (1) is now equivalent to finding the minimum ellipsoid that touches the set ∆(s), or equivalently, we can reformulate problem (P) as follows, 1 ρ + C, 2 Subject to: x ∈ E(Q, h, ρ), (P1 ) :

min x,ρ

(7) (8)

x ∈ ∆(s). In the following, we choose to deal with problem formulation (P1 ), instead of problem formulation (P). Figure 1 illustrates a case where n = 2 and s = 1 and the feasible set ∆(s) consists of both x-axis and y-axis in R2 plane. It is clear from the figure that the optimal contour is the minimum ellipsoid x-axis. Note that the number of feasible subspaces Psthatj touches the j n! could be as large as j=1 Cn , where Cn = j!(n−j)! . Let the spectral decomposition of matrix Q be Q = Γ′ ΛQ Γ, where matrix Γ is unitary and Q Q ΛQ , diag{λQ 1 , λ2 , · · · , λn }.

(9)

Without loss of generality, we assume the eigenvalues of matrix Q to be arranged in an ascending order, Q Q 0 < λQ 1 ≤ λ2 ≤ · · · ≤ λn .

Definition 2.1. Matrix H ∈ Sn+ is said to be of k-degree freedom, if there exists k such that H H H H H H 0 ≤ k ≤ n and 0 ≤ λH 1 ≤ λ2 · · · λk < λk+1 = λk+2 = · · · = λn , where λi is the i-th smallest eigenvalue of H.

4

Polynomially solvable case of CCQP

4 3.5 3 2.5 2 1.5 1 0.5 0 −0.5 −1 −1.5 −1

0

1

2

3

4

5

6

7

8

9

10

Figure 1: The optimal contour and ∆(s) when n = 2 and s = 1 In plain words, a positive definite matrix is of k-degree freedom, if the n − k largest eigenvalues are identical. If k = 0, the 0-degree freedom matrix is a diagonal matrix with identical eigenvalues. If k = n, all the eigenvalues of the matrix have their full freedom. It is very interesting to investigate the cases when 0 < k < n, as we will demonstrate in the following that any CCQO problems can be always approximated by such a matrix of k-degree freedom. For problem (P), we can chose a k and construct an auxiliary problem , 1 min fˆ(x) = (x − h)′ A(x − h) + C, x 2 Subject to: x ∈ ∆(s), (A) :

where A ∈ Sn+ is a matrix of k-degree freedom specified by A = Γ′ Λk Γ with Q Q Q Q
Λk = diag λQ 1 , λ2 , · · · , λk , λk , · · · , λk .

Lemma 2.1. If Q ≻ 0, the following relationships hold,

Q v(A) ≤ v(P) and v(P) − v(A) ≤ (λQ n − λk )Φ,

where Φ is a parameter dependent on problem (P). Proof. Since Q  A, we have fˆ(x) ≤ f (x) for all x ∈ Rn , which implies that v(A) ≤ v(P). Let x ˆ be the optimal solution of problem (A), thus giving rise to 1 v(P) − v(A) ≤ f (ˆ x) − fˆ(ˆ x) = (ˆ x − h)′ (Q − A)(ˆ x − h) 2 1 ≤ (λQ − λQ x − hk2 . k )kˆ 2 n

(10)

As the following holds true for any x ∈ ∆(x) in problem (A), 2 λQ x − hk2 + C ≤ fˆ(ˆ x) ≤ λQ 1 kˆ k kx − hk + C,

∀x ∈ ∆(x).

(11)

5

Polynomially solvable case of CCQP

2 We can minimize the upper bound λQ ¯ = (¯ x1 , · · · , x ¯n ) ∈ ∆(s) with k kx − hk in (11) by taking x ¯i = 0 for i 6∈ I, where I is the index set consisting of the first s largest x ¯i = hi for i ∈ I and x elements of |hi |. Then, inequality (11) becomes

kˆ x − hk2 ≤

X λQ k

λQ 1 i6∈I

h2i .

(12)

Combining (10) and (12) yields the conclusion in the lemma. Lemma 2.1 suggests that increasing the order of k may reduce the gap between the auxiliary and the primary problems. However, the computational burden of solving auxiliary problem (A) may increase for a larger k at the same time.

2.2

A decomposition approach

In this section, we develop an efficient solution scheme for problem (P) with Q being of a k-degree freedom. Our main idea for solving such a class of problems is to decompose (P) into several convex quadratic subproblems. Before we state our main results, we introduce some results on the decomposition of ellipsoid E(Q, h, ρ) when Q is of a k-degree freedom. Theorem 2.1. Let E(Λ, 0, γ) be an ellipsoid with γ > 0, where Λ = diag{λi }|ni=1 is of k-degree freedom with λi being the i-th smallest eigenvalue, 1 < k < n. (i) For any α ∈ Rn such that (

) k 2 u2 X λ λ i k+1 i α ∈ Ek , (u1 , · · · , un )′ | ≤ γ, and uj = 0, j = k + 1, · · · , n , (λk+1 − λi )2 i=1

the following holds, r 2 (α) ,

γ λk+1

−

k X i=1

λi α2i ≥ 0. λk+1 − λi

(13)

(ii) The ellipsoid E(Λ, 0, γ) is the union of the balls expressed as follows, [ E(Λ, 0, γ) = B(α, r 2 (α)). α∈Ek

Proof. (i) Since Λ is of k-degree freedom, we have λk+1 > λi > 0 for i = 1, · · · , k, which further implies λk+1 /(λk+1 − λi ) > 1, for i = 1, · · · , k. Then, for any α ∈ Ek , the following inequality holds, k k X X λi λ2k+1 α2i λi λk+1 α2i ≤ ≤ γ. (λk+1 − λi ) (λk+1 − λi )2 i=1

i=1

Dividing both sides by λk+1 gives rise to the result in (i).

6

Polynomially solvable case of CCQP (ii) For any y ∗ ∈ E(Λ, 0, γ), we have k X

n X

λi (yi∗ )2 + λk+1

i=1

(yj∗ )2 ≤ γ.

(14)

j=k+1

Let α∗ = (α∗1 , α∗2 , · · · , α∗n )′ be defined such that α∗i = yi∗ (λk+1 − λi )/λk+1 for i = 1, · · · , k and α∗j = 0 for j = k + 1, · · · , n. Then, we have n k k X X X λi λ2k+1 (α∗i )2 ∗ 2 = (yi ) λi ≤ γ − λk+1 (yi∗ )2 ≤ γ, (λk+1 − λi )2 i=1

i=1

i=k+1

which implies that α∗ ∈ Ek . Define 2

γ

∗

r (α ) =

λk+1

k X λi (α∗i )2 − . λk+1 − λi i=1

From the result in (i), we have r 2 (α∗ ) ≥ 0. We can further conclude that y ∗ ∈ B(α∗ , r 2 (α∗ )) by checking the following inequality, ky ∗ − α∗ k22 − r 2 (α∗ ) =

k X

(yi∗ − yi∗

i=1

n X λk+1 − λi 2 ) + (yj∗ )2 λk+1 j=k+1

k X

γ (yi∗ )2 λi (λk+1 − λi ) −( − ), 2 λk+1 λ k+1 i=1 k n ∗ 2 X X (yi ) λi γ = + (yj∗ )2 − ( ) ≤ 0, λk+1 λk+1 i=1 j=k+1 where the last inequality is implied by (14). Thus, we conclude that, for any y ∗ ∈ ∗ ∈ E such that y ∗ ∈ B(α∗ , r 2 (α∗ )), which further implies E(Λ, 0, γ), there k S exists α ∗ E(Λ, 0, γ) ⊆ α∗ ∈Ek B(α , r 2 (α∗ )). S On the other hand, for any y¯ ∈ α∈Ek B(α, r 2 (α)), there exists α ¯ ∈ Ek such that y¯ ∈ B(¯ α, r 2 (¯ α)) with r 2 (¯ α) =

γ λk+1

−

k X i=1

α ¯2i λi ≥ 0. λk+1 − λi

As, for i = 1, · · · , k, λk+1 − λi α ¯ i λk+1 2 (¯ yi − ) ≥ 0, λk+1 λk+1 − λi we have y¯i2 λi α ¯2i λi ≤ (¯ yi − α ¯ i )2 + , i = 1, · · · , k, λk+1 λk+1 − λi

(15)

7

Polynomially solvable case of CCQP

3

2

1

0

−1

−2

−3 −4

−3

−2

−1

0

1

2

3

4

Figure 2: Decomposition of ellipsoid in R2 which further gives rise to k n k n X X X X  α ¯ 2i λi  λi y¯i2 + y¯j2 ≤ (¯ yi − α ¯ i )2 + + y¯j2 . λk+1 λk+1 − λi i=1

i=1

j=k+1

(16)

j=k+1

Since y¯ ∈ B(¯ α, r 2 (¯ α)), we have k X 

(¯ yi − α ¯ i )2 ] +

i=1

n X

y¯j2 ≤

j=k+1

γ λk+1

−

k X i=1

α ¯ 2i λi . λk+1 − λi

(17)

Combining inequalities (16) and (17) yields k X i=1

λi y¯i2 +

n X

λk+1 y¯j2 ≤ γ,

j=k+1

which implies y¯ ∈ E(Λ, 0, γ). We finally conclude that

S

α∈Ek

B(α, r 2 ) ⊆ E(Λ, 0, γ).

Theorem 2.1 actually provides a parameterized representation of the ellipsoid E(Λ, 0, γ) by infinite number of balls, when Λ is of k-degree freedom. Figure 2 illustrates this decomposition scheme for an ellipsoid in R2 with 1-degree freedom. It is obvious that the center of the balls is along the longer radius of the ellipsoid and the union of infinite such balls is nothing but the ellipsoid itself. We now proceed to extend the decomposition scheme in Theorem 2.1 to the ellipsoid E(Q, h, ρ) in constraint (8), when Q is of k-degree freedom. For any ρ ≥ 0, we first decompose ellipsoid E(ΛQ , 0, ρ) as follows, [ E(ΛQ , 0, ρ) = B(α, r 2 (α)), α∈EkQ

8

Polynomially solvable case of CCQP where EkQ

,

(

n

u∈R |

ιi ,

κi u2i

)

≤ ρ, uj = 0, j = k + 1, · · · , n ,

i=1

r 2 (α) , (ρ − κi ,

k X

k X

(ιi α2i ))/λQ k+1 ,

(18)

(19)

i=1 Q Q λi (λk+1 )2 , for i = 1, · · · , k, Q 2 (λQ k+1 − λi ) Q (λQ i λk+1 ) , for i = 1, · · · , k. Q λQ k+1 − λi

(20) (21)

As the shape and size of ellipsoid E(ΛQ , 0, ρ) are coordinate independent, the affine transformation x = Γ′ y + h maps y ∈ E(Λ, 0, ρ) to x ∈ E(Q, h, ρ). Thus, the constraint (8) in problem (P) can be expressed as follows according to Theorem 2.1, [ B(Γ′ α + h, r 2 (α)), x ∈ E(Q, h, ρ) = α∈EkQ

where EkQ and r 2 (α) are defined by (18) and (19), respectively. Furthermore, problem (P1 ) can be reformulated as, 1 ρ + C, α,ρ 2 k X [  ′ 2 x | λk+1 kx − Γ α − hk2 + ιi α2i ≤ ρ , Subject to: x ∈

(P2 ) : min

(22)

i=1

α∈EkQ

x ∈ ∆(s).

(23)

The formulation of problem (P2 ) is still hard to solve, as the constraint in (22) involves an infinite number of balls. As we pointed out before, minimizing f (x) under constraint (1) is equivalent to finding the minimum ellipsoid that touches the set ∆(s). We further recognize here that, among infinite balls which together make up the minimum ellipsoid, one specific ball offers the tangent point to achieve this task. Please refer to Figure 1 again, in which the red ball serves exactly this purpose. Based on this argument, we now construct the following problem to identify this particular ball in order to solve problem (P2 ), albeit indirectly, ˆ : (P)

min ρ,β

1 ρ + C, 2

Subject to: λk+1 dis(β) +

k X

ιi βi2 ≤ ρ,

(24)

i=1

k X i=1

κi βi2 ≤ ρ,

(25)

9

Polynomially solvable case of CCQP

where decision variables are β ∈ Rk , ρ ∈ R and the distance function dis(·) : Rk → R+ defined as  (26) dis(β) , minn kx − Hβ − hk22 | x ∈ ∆(s) , x∈R

with H ∈ Rn×k being formed by taking the first k columns of Γ′ . Let Hi ∈ R1×k be the i-th row of matrix H and hi be the i-th element of h. Define b(β) = (b1 (β), b2 (β), · · · , bn (β))′ , where bi (β) , |Hi β + hi |, for i = 1, · · · , n.

(27)

Definition 2.1. We define an index set I(β) ⊂ {1, 2, · · · , n} as the set that includes indices ¯ of the first n − s smallest elements in b(β) and the complementary set of I(β) as I(β) = {1, · · · , n} \ I(β). Note that for any fixed β, I(β) may not be unique. When we have multiple candidates of bi (β) to be chosen as the (n − s)th smallest element in b(β) or the last element in I(β), we can take an arbitrary choice and this does not affect our discussion. ˆ ρˆ) solves problem (P), ˆ then solution-triple (ˆ Theorem 2.2. If solution-pair (β, α, ρˆ, x ˆ) solves ′ ′ ′ ′ ˆ problem (P2 ), where α ˆ = ((β) , 0n−k ) , x ˆ = (ˆ x1 , · · · , x ˆn ) with x ˆi =



ˆ ¯ β), Hi βˆ + hi i ∈ I( ˆ 0 i ∈ I(β),

(28)

ˆ and I( ˆ are defined in Definition 2.1. ¯ β) and index sets I(β) Proof. Substituting (ˆ α, ρˆ, x ˆ) into (22) and (23) confirms the feasibility of (ˆ α, ρˆ, x ˆ) in problem ˆ Now we assume that solution-triple (¯ (P2 ), thus giving rise to v(P2 ) ≤ v(P). α, ρ¯, x ¯) solves 1 1 ˆ problem (P2 ) with v(P2 ) = 2 ρ¯ + C < v(P) = 2 ρˆ + C. Since (¯ α, ρ¯, x ¯) is the solution of problem ˆ α, ρ¯) satisfies constraint (25) in problem (P). On the other hand, note that the following (P2 ), (¯ inequality always holds, λk+1 (min kx − H β¯ − hk22 ) + x

k X

ιi β¯i2 ≤ λk+1 k¯ x − Γ′ α ¯ − hk22 +

i=1

k X

ιi α ¯ 2i ≤ ρ¯,

(29)

i=1

where α ¯ and β¯ satisfy α ¯ = (β¯′ , 0′n−k )′ . Inequality (29) implies that (¯ α, ρ¯) satisfies constraint (24). From our assumption that 12 ρ¯ + C < 12 ρˆ + C, we find a better solution (¯ α, ρ¯, x ¯) for ˆ ˆ problem (P), which contradicts the optimality of solution (ˆ α, ρˆ) for problem (P). Note that once optimal βˆ is fixed, the optimal x ˆ can be found by minimizing (26), i.e., we have ˆ = min dis(β)

x∈∆(s)

n X

(xi − Hi βˆ − hi )2 .

(30)

i=1

Since x ∈ ∆(s), we choose x ˆi = Hi βˆ + hi , for i ∈ I¯ and xi = 0 for i ∈ I which minimizes dis(β) in (30).

10

Polynomially solvable case of CCQP

ˆ and (P2 ). While we will focus Theorem 2.2 reveals an equivalence between problems (P) ˆ on problem (P) in the following, the key issue we are facing is how to handle function dis(β) defined in (26). Clearly, function dis(β) measures the distance between the affine space, {y ∈ Rn | y = Hβ + h} and the set ∆(s). We will carry out detailed discussion on how to identify function dis(β) in Section 3.

3

Distance Evaluation

Given h ∈ Rn and H ∈ Rn×k with rank(H) = k, the distance function, dis(·), is defined in (26). In this section, we focus on distance function dis(β), which plays a key role in solving ˆ In particular, we show that dis(β) is a piece-wise continuous convex quadratic problem (P). function and all the coefficients can be found explicitly by implementing our proposed algorithm. Geometrically, dis(β) identifies the minimum distance between an affine space and the feasible set ∆(s),  dis(β) = minn ky − xk2 | y ∈ Yk (H, h), x ∈ ∆(s) , x∈R

 where Yk (H, h) = y ∈ Rn | y = Hβ + h, β ∈ Rk . Although the number of feasible subspaces in ∆(s) is of a combinatorial nature, the function dis(β) can be still characterized efficiently. We now borrow some concepts from the discrete geometry [1] by considering the following hyperplane arrangements generated by the following hyperlanes in Rk , p1i,j , {β ∈ Rk | (Hi + Hj )β + (hi + hj ) = 0},

(31)

p2i,j

(32)

k

, {β ∈ R | (Hi − Hj )β + (hi − hj ) = 0},

for (i, j) ∈ I , {(i, j)|i = 1, · · · , n − 1, j = i + 1, · · · , n}. Note that the total number of such hyperplanes is n(n − 1). A cell E of the hyperplane arrangements corresponding to p1i,j and p2i,j is a k-dimensional polyhedral set formed by the half spaces induced by hyperlanes p1i,j and p2i,j , (i, j) ∈ I. We characterize the positive and negative half spaces of p1i,j and p2i,j , respectively, by 1 wi,j 2 wi,j

= =

 

+ if (Hi + Hj )β + (hi + hj ) ≥ 0 , for (i, j) ∈ I, − if (Hi + Hj )β + (hi + hj ) < 0

(33)

+ if (Hi − Hj )β + (hi − hj ) ≥ 0 , for (i, j) ∈ I. − if (Hi − Hj )β + (hi − hj ) < 0

(34)

Thus, any cell can be characterized by an (n × n) up-triangular sign matrix w,   0 w1,2 w1,3 · · · w1,n  0 w2,3 · · · w2,n     ··· ··· ···  sign(E) = w =  ,  wn−1,n  0

(35)

11

Polynomially solvable case of CCQP where wi,j is specified by

1 2 ◦ wi,j ) wi,j = (wi,j

(36)

and the operator “◦” is defined such that (+ ◦ +) = +, (+ ◦ −) = −, (− ◦ +) = − and (− ◦ −) = +. Lemma 3.1. In each cell E, induced by hyperplane arrangements p1i,j and p2i,j , (i, j) ∈ I, the order of functions {bi (β)}ni=1 is invariant within the cell, i.e., for a permutation of index set {1, 2, · · · , n}, {i1 , i2 , · · · , in }, the following holds true when β varies within cell E, bi1 (β) ≤ bi2 (β) · · · ≤ bin (β). Proof. Clearly, the order of {bi (β)}ni=1 is determined by comparing whether bi (β) − bj (β) ≥ 0 or not, for all pair (i, j) ∈ I. Since bi (β) ≥ 0 for all i, checking whether bi (β) − bj (β) ≥ 0 or not is equivalent to checking whether the difference bi (β)2 − bj (β)2 is nonnegative or not. Note that, for any (i, j) ∈ I, we have

(bi (β))2 − (bj (β))2 = (Hi + Hj )β + (hi + hj ) (Hi − Hj )β + (hi − hj ) , (37) which further implies 2

2

(bi (β)) − (bj (β))



1 = +, w 2 = +) or (w 1 = −, w 2 = −), ≥ 0 if (wi,j i,j i,j i,j 1 = −, w 2 = +) or (w 1 = +, w 2 = −). ≤ 0 if (wi,j i,j i,j i,j

As any point β in cell E possesses the same sign vector sign(E), thus the order of {bi (β)}|ni=1 is invariant within each cell E. When k = 0, the affine space Yk (H, h) degenerates to a singleton h, i.e., Y0 = {y ∈ Rn | y = h}, and {b}|ni=1 = {|hi |}|ni=1 . As the index set I(β), in such a case, includes the indices corresponding to the first n − s smallest elements of {|h|i }|ni=1 , the distance function dis(β) becomes a constant which can be explicitly expressed. Lemma 3.2. When k = 0, the projection of h on ∆(s) is x∗ = {x∗i }|ni=1 = arg minx∈∆(s) kh − xk22 with  ¯ hi i ∈ I(β) x∗i = 0 i ∈ I(β) and dis(β) =

P

2 i∈I(β) (hi ) .

Note that the projection point may not be unique. We continue to prove that, when k ≥ 1, the distance function is a piece-wise quadratic function. Since each cell of a hyperplane arrangement is a polyhedra, we use a unified expression Ψt β ≤ ηt , where Ψt ∈ Rm×k and ηt ∈ Rm , for cell t. While m is always bounded as m ≤ n(n − 1), it is bounded from above more tightly by the number of hyperplanes which are active for the concerned cell. We will describe in details our algorithm in Section 3.1 and Section 3.2 to search for all cells.

12

Polynomially solvable case of CCQP

Theorem 3.1. The distance function dis(β) is a piece-wise continuous quadratic function, with the following quadratic form with respect to β for each cell indexed by t ∈ {1, · · · , N }, i.e., dis(β) = β ′ Dt β + dt β + ct , ∀β satisfying Ψt β ≤ ηt ,

(38)

where Dt ∈ Sk++ , dt ∈ Rk and
ct ∈ R. Furthermore, the total number of cells, N , is bounded from above by O (n2 − n)k . ˜ applying Lemma 3.2 gives rise to Proof. For any fixed β, ˜ = min kH β˜ + h − xk2 = min dis(β) 2 x∈∆(s)

=

X

x∈∆(s)

˜ = β D β˜ + dβ˜ + c, bi (β) 2

˜′

n X (Hi′ β˜ + hi − xi )2 i=1

(39)

˜ i∈I(β)

where D=

X

Hi′ Hi , d = 2

˜ i∈I(β)

X

hi Hi , c =

˜ i∈I(β)

X

h2i .

(40)

˜ i∈I(β)

¯ While the sets I(β) and I(β) are completely determined by the order of {bi (β)}ni=1 , the order n of functions {bi (β)}i=1 is invariant within each cell E induced by hyperplane arrangements in (31) and (32) according to Lemma 3.1. It has been known that the upper bound on the number
of cells of the hyperplane arrangement generated by (31) and (32) is in order of O (n2 − n)k (see [1]). Now we prove the continuity of function dis(β). Since dis(β) is a continuous function in the interior of each cell E, we only have to check the boundary between cells. Without loss of generality, we consider two neighboring cells, E1 and E2 , which are separated by hyperplane p1i∗ ,j ∗ . There exist two different cases: i) If E1 and E2 define a same index set I(β), then dis(β) is the same for both E1 and E2 , which implies the continuity of dis(β). ii) Assume that cells of E1 and E2 define two different index sets I1 (β) and I2 (β). Since hyperplane p1i∗ ,j ∗ separates these two cells, we know that bi∗ (β) and bj ∗ (β) change order from the proof of Lemma 3.1. More specifically, the index sets I1 (β) and I2 (β) are different in two indices, i∗ and j ∗ , I1 (β) = {i∗ } ∪ (I1 (β) ∩ I2 (β)), I2 (β) = {j ∗ } ∪ (I1 (β) ∩ I2 (β)). Thus, we have dis(β) =

X

bi (β)2 + b2i∗ (β), β in cell E1 ,

i∈I1 (β)∩I2 (β)

dis(β) =

X

i∈I1 (β)∩I2 (β)

bi (β)2 + b2j ∗ (β), β in cell E2

13

Polynomially solvable case of CCQP

Clearly, on the hyperplane p1i∗ ,j ∗ , which is the boundary between cells E1 and E2 , we have b2i∗ (β) = b2j ∗ (β) from the definition (31) and relationship (37), which further implies that dis(β) is continuous on the boundary p1i∗ ,j ∗ . From Theorem 3.1, we know that the distance function dis(β) is a piece-wise quadratic function defined on the cells of hyperplane arrangements, which takes the form in (38) and ˆ is not the total number of the pieces, N , is bounded by O((n2 − n)k ). Although problem (P) convex, it can still be solved by evaluating N subproblems Pˆt , t = 1, · · · , N , separately, as follows, (Pˆt ) : Subject to:

min

1 ρ + C, 2

′ λQ k+1 (β Dt β

+ dt β + ct ) +

(41) k X

ιi βi2 ≤ ρ,

i=1

k X

κi βi2 ≤ ρ,

(42)

i=1

Ψt β ≤ ηt . Once we solve all these subproblems, we can identify the optimal solution (β ∗ , ρ∗ ) by solving a particular sub-problem (Pˆt∗ ), where t∗ = arg min v(Pˆt ). t=1,··· ,N

Note that when β ∗ is fixed, we simply use formulation (28) to identify the optimal solution of problem (P) with optimal objective v(P) = 12 ρ∗ + C. Remark 3.1. From Corollary 3.1 in Section 3, when k = 1, the total number of the quadratic pieces of dis(β) has a bound, N ≤ n(n − 1). Furthermore, the sub-problem (Pˆt ) can be simplified to the following problem, (Pˆt ) :

1 min τ = ρ + C, β,ρ 2

2 2 Subject to: λQ 2 (Dt β + dt β + ct ) + ι1 β ≤ ρ, 2

κ1 β ≤ ρ,

(43) (44)

It ≤ β ≤ It+1 . Compared with the general case with k > 1, a more efficient algorithm is devised in Section 3.1 to identify the distance function dis(β) for k = 1. Remark 3.2. Initial bounds of β are critical for identifying the distance function dis(β), as they affect significantly the speed of the search procedure described in Section 3. Generally speaking, when Q ≻ 0, such bounds on Pkβ can be obtained from Pk the following observation. In ˆ problem (P), the constraint in (42), i=1 κi βi ≤ ρ, implies i=1 κi βi ≤ 2v(P) + 2C, where v(P) is an upper bound of problem (P). In other words, we are only interested in identifying

Polynomially solvable case of CCQP

14

the distance function dis(β) on some bounded domain of β. From an algorithmic point of view, we prefer box-type of bound on β. Thus, we may assume β is confined in a box , n o β ∈ [ω l , ω u ] , β ∈ Rk | ωil ≤ βi ≤ ωiu , i = 1, · · · , k ,

where ω l = {ωil }|ni=1 and ω y = {ωib }|ni=1 . An upper bound v(P) of problem (P) can be easily found by some heuristics, e.g., from the objective value of the incumbent (the best feasible solution obtained). Although Theorem 3.1 shows that the function dis(β) has at most O((n2 − n)k ) pieces, in real application, the total number of the pieces is far less than this upper bound, especially, when we add box bound on β (see Remark 3.2). In the following subsections, we focus on developing an algorithm to identify the function dis(β) in a bounded domain of β, i.e., to identify the coefficients, {Dt , dt , ct }, t = 1, · · · , N , for β ∈ [wl , wu ]. We separate our discussion for cases of k = 1 and k > 1.

3.1

Identification of dis(β) for k = 1

When k = 1, both H and h are vectors in Rk . Corollary 3.1. When k = 1, the distance function dis(β) consists of at most N ≤ n(n − 1) pieces of quadratic functions. Proof. The proof of the corollary follows Theorem 3.1. However, when k = 1, cells of the hyperplane arrangement degenerate to intervals on a real line. An upper bound of the total number of cells, N , can be calculated. Clearly, the set I(β) changes only when some bi (β) ¯ intersects with bj (β), with i ∈ I(β) and j ∈ I(β). That is to say, N ≤ Sn , where Sn is the total number of intersection points between the functions bi (β) and bj (β) for i 6= j, and i, j = 1, · · · , n. The number Sn can be computed in a recursive way. When n = 2, it holds that S2 = 2, and, when n > 3, the recursion Sn = Sn−1 + 2(n − 1) holds. Solving such a recursion yields Sn = (n − 1)n. Thus, theoretically, we can divide the interval [−∞, ∞] into at most N ≤ n(n − 1) consecutive intervals [Ij , Ij+1 ], for j = 1, · · · , N . From Corollary 3.1, we know that N ≤ n(n − 1), where notation “O” is dropped. However, it is still expensive and unnecessary to compute all N quadratic functions directly. To identify function dis(β) with β ∈ [ω l , ω u ], we partition the interval [ω l , ω u ] into several sub-intervals, where functions {bi (β)}|ni=1 are linear in each of there sub-intervals. Note that such a partition always exists. Since bi (β) is a constant when Hi = 0, we assume that Hi 6= 0 for all i = 1, · · · , n. Function bi (β) achieves its minimum point at −hi /Hi with bi (β) = 0, for i = 1, · · · , n. If −hi /Hi 6∈ [ω l , ω u ], then bi (β) is linear in [ω l , ω u ]. Without loss of generality, we assume that points {−hi /Hi }|ni=1 are arranged in an ascending order and are all in interval [ω l , ω u ], ωl < −

h1 h2 hn ≤− ≤ ··· ≤ − < ωu. H1 H2 Hn

15

Polynomially solvable case of CCQP

h1 hn Clearly, in each interval of [ω l , − H ], · · · , [− H , ω u ], function bi (β) is linear with respect to β. n 1 (see Figure 3). Now we concentrate on identifying dis(β) in each sub-interval within β ∈ [β l , β u ]. Our main scheme is to sequentially check the intersection point between bi (β), i ∈ I(β) and bj (β), ¯ j ∈ I(β) from β l to β u . Once an intersection point is identified, the index set I(β) is modified accordingly. More specifically, we use the following Table, T, to store the data.

T I1

I¯1 T1,1

I¯2 T1,2

··· ···

I2 .. .

T2,1 .. .

T2,2 .. .

···

In−s

Tn−s,1

···

··· ···

I¯s T1,s .. . .. . Tn−s,s

In table T , the first column and the first row are corresponding to the index sets Iβ and I¯β , ¯ respectively, i.e., Ii ∈ I(β), i = 1, · · · , n − s and I¯i ∈ I(β) for i = 1, · · · , s. Element T(i, j) in the table stores the intersection point of bIi (β) and bIj (β). Since any two linear functions intersect at most once, each time we only need to modify one column and one row, which leads to a linear time operation O(n). We present formally such an algorithm in Algorithm 1 and use the following Example 3.1 to illustrate the procedure. Algorithm 1 Procedure for identifying dis(β) when k = 1 Input: Interval [ω l , ω u ], H and h Output: All individual pieces of function dis(β) ¯ l ). (1) Let β l ← ω l , β u ← ω u , flag ← 1. Sort {bi (β l )}|ni=1 and construct I(β l ) and I(β l ¯ l ). (2) Initialize Table T by filling first column and first row with index set I(β ) and I(β For i = 1, · · · , n − s and j = 1, · · · , s, set ( hI −hI hI −hI i j if β l ≤ HIi −HIj ≤ β u , HIj −HIi Ti,j = (45) j i +∞ otherwise, while flag = 1 do Construct I(β l ) and identify dis(β) as given in (39) and (40). Output dis(β) with [β l , β u ]. if Ti,j 6= ∞ for all i = 1, · · · , n − s, j = 1, · · · , s, then Find the minimum value Ti∗ ,j ∗ in Table T . β l ← β u , β u ← Ti∗ ,j ∗ and Ti∗ ,j ∗ ← ∞. Exchange index Ij ∗ and Ii∗ and update j ∗ -th column and i∗ -th row by using (45). if β u = ω u , then flag ← 0 end if else flag ← 0 end if end while

16

Polynomially solvable case of CCQP Example 3.1. We consider an example with n = 6, k = 1, s = 3 and
′ H = −0.5 0.5 1 −0.75 −2 −4 ,
′ h = 1 2.5 4 3.4 3.5 5.2 .

We want to identify function dis(β) with β ∈ [−1, 1.3]. For this example, functions {bi (β)}6i=1 are specified as follows (also see Figure 3), b1 (β) = | − 0.5β + 1|, b2 (β) = |0.5β + 2.5|, b3 (β) = |β + 4|, b4 (β) = | − 0.75β + 3.4|, b5 (β) = | − 2β + 3.5|, b6 (β) = | − 4β + 5.2|. Interval [−5, 5] can be decomposed as follows, [−5, 5] = ∪[−5, −4] ∪ [−4, 1.3] ∪ [1.3, 1.75] ∪ [1.75, 2] ∪ [2, 4.53] ∪ [4.53, 5], such that in each of the sub-intervals, all bi (β), i = 1, · · · , 6, are linear. We demonstrate our algorithm, in particular, for the interval of [−1, 1.3] in which b1 (β) = −0.5β + 1, b2 (β) = 0.5β + 2.5, b3 (β) = β + 4, b4 (β) = −0.75β + 3.4, b5 (β) = −2β + 3.5, b6 (β) = −4β + 5.2. The functions bi (β) in [−6, 6] 8

b6 (β)

7

b4 (β) b3 (β)

6

b1 (β)

5 4

b5 (β)

3 2

b2 (β)

1 0 −6

−4

−2

0

2

4

6

β The functions bi (β) in [−1, 1.3] 9 8

gi (β) b6 (β)

7

b5 (β)

6

b3 (β)

b4 (β)

5 4 3

b2 (β)

2

b1 (β)

1 0 −2

−1.5

−1

−0.5

0

0.5

1

1.5

2

β

Figure 3: Functions bi (β) in Example 3.1 We now use Algorithm 1 to identify function dis(β) in the interval of [−1, 1.3].

Polynomially solvable case of CCQP

17

• In the first step, we compute bi (−1) for i = 1, · · · , 6, initialize the index sets as I(−1) = ¯ {1, 2, 3} and I(−1) = {4, 5, 6} according to the values of {bi (−1)}|6i=1 , and construct Table 1, where the element in the position corresponding to bi ∈ I and bj ∈ I¯ represents their intersection. We find the minimum value −0.342 in Table 1, which indicates that Table 1: Table of Step 1 in Example 3.1 6 4 5 1 1.200 +∞ +∞ 2 0.600 0.720 0.400 3 0.240 −0.343 −0.167 b4 (β) and b3 (β) intersecting at β = −0.342. We can thus conclude that, in the subinterval of β ∈ [−1, −0.342], the index set I(β) = {1, 2, 3} remains unchanged. We further identify the first piece of function dis(β) for β ∈ [−1, −0.342] as dis(β) = 1.5β 2 + 9.5β + 23.25, β ∈ [−1, −0.342], where the coefficients are computed according to (39) with index set I([−1, −0.342]) = {1, 2, 3}. Since −0.342 is the intersection point of b3 (β) and b4 (β), we exchange the positions of b3 by b4 in Table 1, which leads to Table 2. Table 2: Table of Step 2 in Example 3.1 6 3 5 1 1.200 +∞ +∞ 2 0.600 +∞ 0.400 4 0.554 +∞ 0.008 • The minimum value in Table 2 is 0.08 and we derive the second piece of dis(β) as dis(β) = 1.062β 2 − 36β + 18.51, β ∈ [−0.342, 0.080],

(46)

where all coefficients are computed according to index set I([−0.342, 0.08]) = {1, 2, 4}. Updating row 3 and column 3 in Table 2 yields Table 3. Table 3: Table of Step 3 in Example 3.1 6 3 4 1 1.200 +∞ +∞ 2 0.600 +∞ 0.720 5 0.850 +∞ +∞ • The minimum value of Table 3 is 0.60 and we derive the third piece of function dis(β) as dis(β) = 4.5β 2 − 12.5β + 19.5, β ∈ [0.08, 0.60],

(47)

18

Polynomially solvable case of CCQP 22

20

18

16

1.06β 2 − 3.6β + 18.5 2

1.5β + 9.5β + 23

14

4.5β 2 − 12.5β + 19.5 12

dis(β)

10

20β 2 − 56.6β + 40.3

8

6

4

2

0.08

−0.3429 0 −1

−0.5

0

0.6 0.5

1

β

Figure 4: Functions {bi (β)}|6i=1 where all coefficients are computed according to index set I([0.08, 0.6]) = {1, 2, 5}. Updating row 2 and column 1 in Table 3 yields Table 4. Table 4: Table of Step 4 in Example 3.1 2 3 4 1 +∞ +∞ +∞ 6 +∞ +∞ +∞ 5 +∞ +∞ +∞ • The minimum value of Table 4 is 1.3 and we get the fourth piece of function dis(β) as dis(β) = 20.25β 2 − 56.6β + 40.29, β ∈ [0.60, 1.3].

(48)

As all elements in Table 4 are ∞, we complete the characterization of the distance function of dis(β) in [−1, 1.3] (See Figure 4).

3.2

Identification of dis(β) for k > 1

We know from Theorem 3.1 that the distance function dis(β) can be constructed according to the cells of hyperplane arrangement. Enumerating the cells of the hyperplane arrangement has been investigated in the literature. For example, the authors in [1] and [10] proposed a cell enumeration method by reverse searching method. Such a method consumes O((n2 − n)k Clp ) time to enumerate all the cells, where Clp is the time for a linear programming. Note that the cells are searched in the whole Rk space when adopting the reverse searching method. In our study, we are only interested in enumerating all the cells within a bounded region β ∈ [ω l , ω u ], which is also ready to be solved by the algorithm in [10].

19

Polynomially solvable case of CCQP

As we have illustrated in Lemma 3.1, the order of all {bi (β)}|ni=1 is fixed in each cell induced by hyperplanes p1i,j and p2i,j , for i, j ∈ I. Such cells are characterized by the sign matrix sign(E) in (35). Now we specify a mapping from a sign matrix to an order of functions {bi (β)}|ni=1 . Given a sign matrix sign(E) in (35), we first construct matrix Ω(E) by letting Ωii (E) = 0, copying the upper-triangle of sign(E) into its upper-triangle and inserting the opposite of the upper-triangle of sign(E) into its lower-triangle. From (37), (36) and (33), we can conclude that bi (β) is the (t + 1)-th smallest element among {bi (β)}ni=1 for β in cell E if there are totally t of “+” in i-th row of Ω(E), i.e., we have (+, −, +, · · · , −) ←→ bi (β) is (t + 1)-th smallest elemetnt. {z } | There are t “+”s.

Example 3.2. We illustrate how to identify function dis(β) with k > 1 by considering the following example of identifying function dis(β) with k = 2, n = 4, s = 2 and parameters set as follows,     4 2 −2  5   1   , h =  −6  . H=  1  −1  −1  2 − 0.5 2 Assume that β = (β1 , β2 )′ is confined in the box of −1 ≤ β1 ≤ 4 and −1.5 ≤ β2 ≤ 1. According to (31) and (32), we introduce the following hyperplanes, p11,2 p11,3 p11,4 p12,3 p12,4 p13,4

: : : : : :

1 (β) = −β + β + 4 = 0, z1,2 1 2 1 (β) = 3β + 3β − 1 = 0, z1,3 1 2 1 (β) = 2β + 2.5β − 4 = 0, z1,4 1 2 1 (β) = 4β + 2β − 5 = 0, z2,3 1 2 1 (β) = 3β + 1.5β − 8 = 0, z2,4 1 2 1 (β) = −β − 0.5β − 3 = 0, z3,4 1 2

p21,2 p21,3 p21,4 p22,3 p22,4 p23,4

: : : : : :

2 (β) = 9β + 3β − 8 = 0, z1,2 1 2 2 (β) = 5β + β − 3 = 0, z1,3 1 2 2 (β) = 6β + 1.5β = 0, z1,4 1 2 2 (β) = 6β − 7 = 0, z2,3 1 2 (β) = 7β + 0.5β − 4 = 0, z2,4 1 2 2 (β) = 3β − 1.5β + 1 = 0. z3,4 1 2

It can be verified that the region specified by −1 ≤ β1 ≤ 4 and −1.5 ≤ β2 ≤ 1 is on one side of the following hyperplane, 2 2 2 1 2 z1,3 (β) > 0, z1,4 (β) > 0, z2,4 (β) > 0, z3,4 (β) < 0, z3,4 (β) > 0. 2 = +, ω 2 = +, ω 2 = +, ω 1 = −, Thus, for any β in this region, it holds true that ω1,3 1,4 2,4 3,4 2 ω3,4 = +. We can enumerate the cells of the arrangements generated by these hyperplanes in the box region, β1 ∈ [1, 4] and β2 ∈ [−1.5, 1]. Implementing the algorithm of cell enumeration [10] generates 15 cells. While the sign vectors of these 15 cells are listed in Table 5, all the hyperplanes and their arrangement are illustrated in Figure 5, in which the arrow indicates the positive side of the hyperplane. We can then figure out the order of the {bi (β)}4i=1 in each cell. For example, let us consider cell2 in Table 5 with   (− ◦ +) (+ ◦ +) (+ ◦ +)  (+ ◦ +) (+ ◦ +)  . sign(cell2 ) =   (− ◦ +) 

Polynomially solvable case of CCQP

20

Then we construct matrix Ω(cell2 ) from the sign matrix sign(cell2 ) as   0 − + +  + 0 + +   Ω(cell2 ) =   − − 0 − . − − + 0

which yields an order of bi (β), i.e., b3 (β) < b4 (β) < b1 (β) < b2 (β). Once the order of bi (β) is achieved, the distance function can be expressed by applying (39) and (40), dis(β) = 5β12 + 1.25β22 − 8β1 β2 + 6β1 + 5. All the other pieces of the distance function can be derived in a similar fashion. Table 5: No 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

4

The cells of hyperplanes in Example 3.2 (w1,2 , w1,3 , w1,4 ), (w2,3 , w2,4 ), (w3,4 ) (++, ++, ++), (++, ++), (−+) (−+, ++, ++), (++, ++), (−+) (−+, ++, −+), (++, ++), (−+) (−+, ++, −+), (++, +−), (−+) (++, ++, −+), (++, +−), (−+) (++, ++, −+), (−+, +−), (−+) (++, +−, −+), (−+, +−), (−+) (+−, +−, −+), (−+, +−), (−+) (+−, +−, −+), (−−, +−), (−+) (+−, −−, −+), (−−, +−), (−+) (+−, −−, −+), (−−, ++), (−+) (+−, −−, −+), (−+, ++), (−+) (+−, −−, −+), (+−, ++), (−+) (+−, −−, ++), (+−, ++), (−+) (+−, −−, ++), (++, ++), (−+)

Illustrative Example

We demonstrate in this section a complete implementation of our solution scheme developed in this paper via an illustrative example.

21

Polynomially solvable case of CCQP 1 2 z23 (β) 1 z24 (β) 1 z14 (β)

0.5

cell1

cell15

β2

0

−0.5

cell2

1 z12 (β)

cell5 ((−+)(++)(++), (++)(++), (−+))

1 z13 (β)

−1 1 z23 (β)

cell4 cell3

2 z12 (β)

−1.5 1

1.5

2

2.5

3

3.5

4

β1

Figure 5: The cells of hyperplane arrangement in Example 3.2 Example 4.1. Let us consider an example of problem (P) with n = 6, s = 2 and 

   Q =    q=

27.171 −5.738 2.479 −2.768 4.931 1.725 −5.738 18.358 5.030 −5.615 10.004 3.500 2.479 5.030 27.827 2.426 −4.322 −1.512 −2.768 −5.615 2.426 27.292 4.825 1.688 4.931 10.004 −4.322 4.825 21.404 −3.007 1.725 3.500 −1.512 1.688 −3.007 28.948 37.745 −26.329 −80.284 34.905 7.296 −51.002



   ,   


′

.

It can be verified that Q is of a k = 1- freedom with one eigenvalue of λ1 = 1 and five eigenvalues of λ2 = 30. For this simple example, adoption of an enumeration method identifies its optimal solution of x∗ = (0, 0, 2.989, 0, 0, 1.918)′ and the corresponding optimal value of v(P) = −168.91. We compute H and h defined (26) as H = ( 0.312, 0.634, − 0.274, 0.306, − 0.544, − 0.190 )′ , h = ( − 11.369, − 19.636, 11.539, − 11.057, 17.384, 7.867)′ . Using the parameters defined in Section 2.2, we have κ1 = 1.0701, ι1 = 1.0344 and C = −749.43, which are defined in (20), (21) and (5), respectively. Since k = 1, we identify the distance function dis(β) as follows in the interval [0, 37] by using the method discussed in

22

Polynomially solvable case of CCQP Section 3.1,  9.063β 2 − 695.19β + 13396.41,     15.71β 2 − 1073.53β + 18467.86,      18.86β 2 − 1252.13β + 20969.04, 24.83β 2 − 1606.95β + 26156.91, dis(β) =   26.66β 2 − 1730.09β + 28178.20,      15.71β 2 − 1073.53β + 18467.85,   9.06β 2 − 695.19β + 13396.41,

I(β) = {2, 5}, I(β) = {2, 3}, I(β) = {3, 5}, I(β) = {3, 1}, I(β) = {3, 6}, I(β) = {2, 3}, I(β) = {2, 5},

β β β β β β β

∈ [0, 21.59], ∈ [21.59, 25.26], ∈ [25.26, 25.91], ∈ [25.91, 28.75], ∈ [28.75, 33.37], ∈ [33.37, 35.34], ∈ [35.34, 36.60].

(49)

Note that function dis(β) consists of total N = 7 pieces of convex quadratic functions. We can now explicitly write out the left hand of constraint (43) for each of sub-problem (Pˆt ), t = 1, · · · , 7,  g1 (β) = 10.10β 2 − 695.19β + 13396.41 β ∈ [0, 21.59],    2  g2 (β) = 16.74β − 1073.53β + 18467.86 β ∈ [21.59, 25.26],      g3 (β) = 19.89β 2 − 1252.13β + 20969.04 β ∈ [25.26, 25.91], λ2 dis(β1 ) + ι1 β12 = g4 (β) = 25.86β 2 − 1606.95β + 26156.91 β ∈ [25.91, 28.75],   g5 (β) = 27.70β 2 − 1730.09β + 28178.20 β ∈ [28.75, 33.37],      g (β) = 16.74β 2 − 1073.53β + 18467.85 β ∈ [33.37, 35.34],   6 g7 (β) = 10.10β 2 − 695.19β + 13396.41 β ∈ [35.34, 36.60].

On the other hand, constraint (44) keeps the same form, g¯(β) = 1.07β 2 ≤ ρ. The sub-problem (Pˆt ), t = 1, · · · , 7, becomes (Pˆt ) : min β,ρ

n1 2

o ρ + C | gt (β) ≤ ρ, g¯(β) ≤ ρ .

We plot all the constraints in Figure 6. Solving all these sub-problems, we find that the optimal value is achieved in interval [25.91, 28.75] with index set I(β) = {3, 1}, the optimal ρ∗ equal to 1161.053 and the optimal value v(P) = 12 ρ∗ + C = −168.91. From Theorem 2.2, we further get optimal solution x∗3 = 2.989, x∗6 = 1.918, x∗1 = 0, x∗2 = 0, x∗4 = 0, x∗5 = 0.

5

Conclusions

We have investigated a class of CCQO problems with their coefficient matrices being of a k-degree freedom. We have demonstrated that we can decompose such a CCQO problem into several quadratic convex subproblems, where the total number of the subproblems is determined by the number of the cells generated by some Rk dimensional hyperplanes within a bounded region. It is interesting to note that such a number is bounded by O(n2k ). In particular, for the case with k = 1, we have developed an efficient method to identify all the subproblems. For cases with k ≥ 2, by modifying the reverse searching method proposed in [1], we have also developed an efficient algorithm to identify the subproblems. Once subproblems are known, we can solve each subproblem individually and find the solution of the original CCQO problem.

23

Polynomially solvable case of CCQP

4

ρ × 10 −3 3.5

g2 (β)

3

g3 (β)

2.5

g4 (β)

2

g6 (β)

g7 (β)

g5 (β) 1.5

1

ρ∗ = 1.161 0.5

25.26

21.59 0 20

22

24

25.91 26

33.37

28.75 28

30

β ∗ = 31.23

32

35.35 34

36

β

Figure 6: Functions {gi (β)}|6i=1 and g¯(β) of Example 4.1

References [1] D. Avis and K. Fukuda. Reverse search for enumeration. Discrete Applied Mathematics, 65:21–46, 1996. [2] D. Bertsimas and R. Shioda. Algorithm for cardinality-constrained quadratic optimization. Computational Optimization and Applications, 2007. [3] D. Bienstock. A computational study of a family of mixed-integer quadratic programming problems. Mathematical Programming, 74:121–140, 1996. [4] E. J. Candes, J. Romberg, and T. Tao. Robust uncertainty principles: exact signal reconstruction from highly incomplete freguency information. IEEE Transcation on Information Theory, 52(2):489–509, 2006. [5] A. Das and D. Kempe. Algorithms for subset selection in linear regression. In Proceedings of the 40th annual ACM symposium on Theory of computing, pages 45–54, 2008. [6] D. L. Donoho. Compressed sensing. IEEE Transaction Information Theory, 52(4):1289, 2006. [7] M. R. Garey and D. S. Johnson. Computer and Intractability, A Guide to The Thoery of NP-Completeness. W. H. Freeman Co, Francisco, 1979. [8] T. M. Liebling E. Steiner K. Allemand, K. Fukuda. A polynomial case of unconstrained zero-one quadratic optimization. Mathematical Programming, 91:49–52, 2001. [9] D.X. Shawa, S. Liub, and L. Kopmanb. Lagrangian relaxation procedure for cardinalityconstrained portfolio optimization. Optimization Methods and Software, 23:411–420, 2008.

Polynomially solvable case of CCQP

24

[10] N. Sleumer. Output-sensitive cell enumeration in hyperplane arrangements. Nordic journal of computing, 6:137–161, 1999. [11] W.J. Welch. Algorithm complexity: three np-hard problems in computational statistics. Journal of Statistical Computation and Simulation, 15:17–25, 1982. [12] J. Xie, S. He, and S. Zhang. Randomized portfolio selection with constraints. Pacific Journal of Optimization, 4:89–112, 2008.