Gradient Dynamic Approach to the Tensor Complementarity Problem

Gradient Dynamic Approach to the Tensor Complementarity Problem

arXiv:1808.03021v2 [math.NA] 10 Aug 2018

Xuezhong Wang∗

Maolin Che†

Liqun Qi‡

Yimin Wei§

August 13, 2018

Abstract Nonlinear gradient dynamic approach for solving the tensor complementarity problem (TCP) is presented. Theoretical analysis shows that each of the defined dynamical system models ensures the convergence performance. The computer simulation results further substantiate that the considered dynamical system can solve the tensor complementarity problem (TCP).

Key words: Tensor complementarity problem; dynamical system; activation function; convergence. AMS subject classifications: 15A18, 15A69, 65F15, 65F10, 90C11

∗

E-mail: [email protected]. School of Mathematics and Statistics, Hexi University, Zhangye, 734000, P. R. of China. This author is supported by the National Natural Science Foundation of China under grant 11771099. † E-mail: [email protected] and [email protected]. School of Economic Mathematics, Southwest University of Finance and Economics, Chengdu, 611130, P. R. of China. This author is supported by the Fundamental Research Funds for the Central Universities under grant JBK 1801058. ‡ E-mail: [email protected]. Department of Applied Mathematics, the Hong Kong Polytechnic University, Hong Kong. L. Qi is supported by the Hong Kong Research Grant Council (Grant No. PolyU 15302114, 15300715, 15301716 and 15300717) § Corresponding author (Y. Wei). E-mail: [email protected] and [email protected]. School of Mathematical Sciences and Shanghai Key Laboratory of Contemporary Applied Mathematics, Fudan University, Shanghai, 200433, P. R. of China. This author is supported by the National Natural Science Foundation of China under grant 11771099.

1

1

Introduction

Let R be the real field. A tensor can be regarded as a high-order generalization of a matrix, which takes a form A = (ai1 i2 ...im ),

ai1 i2 ...im ∈ R,

1 ≤ i1 , i2 , . . . , im ≤ n.

Such a multi-array A is said to be an m-order n-dimensional real tensor. We denote the set of all m-order n-dimensional real tensor by R[m,n] . Let x ∈ Rn , the n dimensional vector Axm−1 is defined as [26]: (Ax

m−1

)i =

n X

aii2 ,...,im xi2 . . . xim , i = 1, 2, . . . , n,

(1.1)

i2 ,...,im

where xi denotes the ith component of x. For any q ∈ Rn , we consider the tensor complementarity problem, a special class of nonlinear complementarity problems, denoted by TCP(A, q): finding x ∈ Rn such that x ≥ 0,

Axm−1 + q ≥ 0,

x> (Axm−1 + q) = 0.

This is a generalization of the linear complementarity problem. So far many researchers have paid attention to this topic [1, 4, 8, 10, 15, 19, 25, 29, 30, 33] because of its applications such as DNA micro-arrays, communication and n-person non-cooperative game [19, 25]. In [29], Song and Qi showed that TCP(A, q) has a solution if and only if A is nonnegative with its diagonal entries being positive. Song and Qi [30] discussed the solution of TCP(A, q), when A is strictly semi-positive. Che, Qi and Wei [4] discussed the existence and uniqueness of solution of TCP(A, q) with some special tensors. Luo, Qi and Xiu [25] obtained the sparsest solutions to TCP(A, q) with a Z-tensor. Song and Yu [31] obtained global upper bounds of the solution of the TCP(A, q) with a strictly semi-positive tensor. Gowda, Luo, Qi and Xiu [15] studied the various equivalent conditions for the existence of solution to TCP(A, q) with a Z-tensor. Ding, Luo and Qi [8] showed the properties of TCP(A, q) with a P -tensor. Bai, Huang and Wang [1] considered the global uniqueness and solvability for TCP(A, q) with a strong P -tensor. Wang, Huang and Bai [32] gave the solvability of TCP(A, q) with exceptionally regular tensors. Numerical algorithms for solving tensor complementarity problems have been proposed recently. Xie, Li and Xu [33] presented numerical methods for finding the least solution to the TCP(A, q) with a Z-tensor. Liu, Li and Vong [24] proposed the modulus equation for TCP(A, q) and based on this equation, they developed the corresponding nonsmooth Newton’s method for solving TCP(A, q). Huang and Qi [19] proposed a smoothing type algorithm. Han [17] introduced a Kojima-Megiddo-Mizuno type continuation method for solving TCP(A, q). Du et al. [10] showed that the tensor absolute equation is equivalent to a generalized tensor complementarity problem and proposed an inexact Levenberg-Marquardt method for solving the tensor absolute equation. Also, Du and Zhang [9] gave a mixed integer programming model to solve the TCP(A, q). 2

The introduction of dynamic models in optimization started in 1980s [5, 18]. Since then, significant research results have been achieved for various optimization problems, such as linear programming [35], quadratic programming [3], linear complementarity problems [22], and nonlinear programming [28]. The essence of dynamic approach for optimization is to establish an energy function (nonnegative). The dynamic system is normally in the form of first-order ordinary differential equations. It is expected that for an initial state, the dynamic system will approach its static state (or equilibrium point) which corresponds the solution of the underlying optimization problem. An important requirement is that the energy function decreases monotonically as the dynamic system approaches an equilibrium point. The gradient dynamical system (GDS) has now been regarded as a powerful alternative for online computation [36], linear complementarity problems [22] and nonlinear complementarity problems [23], in view of its high speed processing nature and its convenience of hardware implementation in practical applications [13, 27]. To effectively solve the linear complementarity problem and nonlinear complementarity problems, linear gradient dynamical system (LGDS) is thus obtained [22, 23]. Note that the LGDS with application to online linear (nonlinear) complementarity problems solving have been investigated by the previous work [22, 23]. The existence and the convergence of the trajectory of the dynamical system are addressed in detail in [22, 23]. In addition, Liao, Qi and Qi [23] also explore the stability properties, such as the stability in the sense of Lyapunov, the asymptotic stability and the exponential stability, for the dynamical system model. However, to the best of our knowledge, there exists few research results on solving the TCP(A, q) via the nonlinear gradient dynamical system (NGDS). Motivated by this reason, we thus design, propose and investigate different NGDS for solving TCP(A, q) by defining error-monitoring functions. It is theoretically proved that defined NGDS converge to the theoretical solution. Through illustrative computer-simulation examples, the efficacy and the superiority of the proposed dynamical system model for online computation of the TCP(A, q) with Axm−1 + q is a P -function is well-verified. The main contributions of the paper are listed as follows. (1) One type of NGDS for solving the TCP(A, q) with Axm−1 + q for any q ∈ Rn is a P -function are presented; (2) Theoretical analysis shows the convergence of the presented gradient neural networks to the theoretical solution of the TCP(A, q) with Axm−1 + q for any q ∈ Rn is a P -function; (3) Computer simulation results via illustrative examples are presented, compared and discussed. Generated numerical results comparatively substantiate that the NGDS with nonlinear activation function are much more efficient in solving the TCP(A, q) with Axm−1 + q for any q ∈ Rn is a P -function, as compared to the NGDS with linear activation function proposed in this paper. This paper is organized as follows. In Section 2, we recall some preliminary definitions and results. Dynamical system models with different nonlinear activation functions for online solution of the TCP(A, q) with Axm−1 + q for any q ∈ Rn is a P -function are presented in Section 3. Convergence properties of the presented dynamical system models will be discussed in Section 4. Illustrative numerical examples are presented in Section 5.

3

2

Preliminaries

Han [16] gave a method to partially symmetrize tensor A = (ai1 i2 ...im ) ∈ R[m,n] with respect to the indices i2 . . . im , which will be used in sequel. In detail, the partially symmetrized tensor Ab = (b ai1 i2 ...im ) as follows X 1 b ai1 i2 ...im = a , (m − 1)! π i1 π(i2 ...im ) where the sum is over all the permutations π(i2 . . . im ). For any A ∈ R[m,n] , we can get a partially b m−1 , by an averaging procedure. symmetrized tensor Ab ∈ R[m,n] such that Axm−1 = Ax

2.1

Function tensors and matrices

We first recall the definitions of P -matrix, P -tensor and P -function as follows. Lemma 2.1. ([2]) Let A ∈ Rn×n , then A is called a P -matrix if all the principal minors of A are positive. Throughout the paper, we assume that F : Rn → Rn is a continuously differentiable function. Definition 2.1. ([11]) A function F : K ⊆ Rn → Rn is called a P -function on K if for all x, y ∈ Rn with x 6= y, it holds that max(xi − yi )[Fi (x) − Fi (y)] > 0. i

(2.1)

Lemma 2.2. ([11]) F : Ω ⊃ K ⊆ Rn → Rn be continuously differentiable on the open set Ω containing the set K. F is a P -function on K if and only if Jacobian matrix F 0 (x) is a P -matrix for all x ∈ K. Definition 2.2. ([1, 24]) Let A ∈ R[m,n] , then A is called (i) a P -tensor, if for each nonzero x ∈ Rn there exists index i such that xi (Axm−1 )i > 0;

(2.2)

(ii) a strong P -tensor, if and only if Axm−1 is a P -function; (iii) a strong strictly semi-positive tensor if Axm−1 + q is a P -function in Rn+ for any q ∈ Rn , where Rn+ denotes the set of all the nonnegative vectors. From the definitions of the P -tensor, the strong P -tensor and the strong strictly semi-positive tensor, it is easy to see that every strong P -tensor must be a P -tensor, a strong strictly semipositive tensor is a P -tensor. In additional, we can obtain if for any q ∈ Rn , Axm−1 + q is a P -function, then tensor A is a P -tensor, furthermore, we have Jacobian matrix (Axm−1 )0 is a P -matrix. Bai, Huang, and Wang [1] proved that a TCP possesses the global uniqueness and solvability property if the tensor is a strong P -tensor. Liu et al. [24] showed that a TCP possesses the global uniqueness and solvability property if the tensor is strong strictly semi-positive tensor. We summarize their results in the following theorem. 4

Lemma 2.3. ([1, 24]) Let A ∈ R[m,n] , (i) if A is a P -tensor, then, for any q ∈ Rn , the solution set of T CP (A, q) is nonempty and compact; (ii) if A is a strong P -tensor, then T CP (A, q) has the global uniqueness and solvability property; (iii) if A is a strong strictly semi-positive tensor, then T CP (A, q) has the global uniqueness and solvability property.

2.2

The Fischer-Burmeister NCP function

Below, we introduce the classical nonlinear complementarity problem (NCP). It will be shown that the tensor complementarity problem TCP(A, q) is a special kind of nonlinear complementarity problem. Definition 2.3. ([11]) Given a mapping F : Rn → Rn , the nonlinear complementarity problem, denoted by NCP(F), is to find a vector x ∈ Rn satisfying x ≥ 0,

F (x) ≥ 0,

x> F (x) = 0.

(2.3)

Note that if F (x) = Axm−1 + q, then NCP reduces to TCP(A, q). Many solution methods developed for NCP or related problems are based on reformulating them as a system of equations using so-called NCP-functions. Here, a function φ : R2 → R is called an NCP-function if φ(a, b) = 0 ⇔ ab = 0,

a ≥ 0,

b ≥ 0.

(2.4)

Here, we use the following Fischer-Burmeister NCP-function [14], p φ(a, b) = a2 + b2 − a − b, which are widely used in nonlinear complementarity problems. Lemma 2.4. The square of φ(a, b) is continuously differentiable; φ(a, b) is twice continuously differentiable everywhere except at the origin; but it is strongly semismooth at the origin. By a simple calculation procedure, we have  (  √ a √ b − 1, − 1 , a2 + b2 6= 0, a2 +b2 a2 +b2 ∂φ(a, b) = (α − 1, β − 1) with α2 + β 2 ≤ 1, a2 + b2 = 0. From Lemma 2.4, we have ∂φ(a, b) is a singleton except at the region.

5

2.3

Activation function

The matrix-valued activation function F(E), E = (eij ), is defined as (f (eij )), i, j = 1, 2, . . . , n, where f (·) is a scalar-valued monotonically-increasing odd function. The following real-valued linear and nonlinear odd and monotonically increasing functions f (·) are widely used. Linear function flin (x) = x; Bipolar-sigmoid function fbs (x, q) =

1 + exp(−q) 1 − exp(−qx) , q > 2; 1 − exp(−q) 1 + exp(−qx)

Power-sigmoid function ( fps (x, p, q) =

xp , 1+exp(−q) 1−exp(−qx) 1−exp(−q) 1+exp(−qx) ,

if |x| ≥ 1 , q ≥ 2, p ≥ 3; otherwise

Smooth power-sigmoid function 1 1 + exp(−q) 1 − exp(−qx) fsps (x, p, q) = xp + , p ≥ 3, q > 2. 2 1 − exp(−q) 1 + exp(−qx) In general, any monotonically increasing odd activation function f (·) can be used for the construction of the dynamical system. As it was shown in [21, 37], the convergence rate can be thoroughly improved by an appropriate activation function. So far, the influence of various nonlinear activation functions was investigated for different dynamical system models. We investigate this scenario on several dynamical system models which are introduced in this paper.

3

Nonlinear dynamical system methods

In this section, the error monitoring function is designed for deriving a gradient dynamical system (GDS). Specifically, by defining different EFs, different GDSs can be obtained for online solution of the TCP(A, q). We construct nonlinear gradient dynamical system models, called NGDS, and consider their convergence. Since TCP(A, q) can be equivalently reformulated as finding a solution of the following equation:   φ(x1 , (Axm−1 + q)1 )  φ(x2 , (Axm−1 + q)2 )    Φ(x) =  (3.1)  = 0. ..   . φ(xn , (Axm−1 + q)n )

We note that Φ(x) is locally Lipschitz continuous everywhere, so that Clarke’s [6] generalized Jacobian ∂Φ(x) is well defined at any point. 6

Thus, we can define the error monitoring function as 1 ε(t) = ε(x(t)) = kΦ(x(t))k22 . 2 The function ε(x(t)) is continuously differentiable [12], which follows from the semi-smoothness of Φ(x). In order to force ε(t) to converge to zero, the negative of the gradient (i.e., -∂(ε(t))/∂x) is used as the descent direction, which leads to the so-called GDS design formula in the form of a first-order differential equation: dx ∂(ε(t)) = −γ = −γV > Φ(x), dt ∂x

(3.2)

where V ∈ ∂Φ(x) and γ is a positive scaling constant. Note that γ corresponds to the reciprocal of a capacitance parameter, of which the value should be set as large as the hardware would permit, or appropriately large for modeling and experimental purposes. The dynamic equation (3.2) will be simply termed the LGDS model. Following the principle of nonlinear activation in the LGDS model defined in Section 2.3, the conventional LGDS model (3.2) can be improved into the following nonlinear GDS model by exploiting a nonlinear activation function array F(·): dx = −γV > F(Φ(x)), dt

(3.3)

where F(·) denotes a matrix-valued activation function. The dynamic equation (3.3) will be simply termed the NGDS model. We give the precise definition of V which is necessary for the implementation of our model. Lemma 3.1. ([7]) Any V with the following structure is an element of ∂Φ(x) b m−2 , V = Da (x) + (m − 1)Db (x)Ax where Da (x) =diag(a1 (x), . . . , an (x)), Db (x) =diag(b1 (x), . . . , bn (x)) are diagonal matrices whose ith diagonal elements are given by ai (x) = q

xi

(Axm−1 + q)i bi (x) = q −1 x2i + (Axm−1 + q)2i

− 1,

x2i + (Axm−1 + q)2i

if (xi , (Axm−1 + q)i ) 6= (0, 0), and by ai (x) = αi − 1,

bi (x) = βi − 1

for every (αi , βi ) ∈ R2 such that αi2 + βi2 ≤ 1 if (xi , (Axm−1 + q)i ) = (0, 0).

7

4

Stability analysis

In this section, we address the stability issues on the dynamical system (3.3) to the solution of the TCP(A, q). For given x∗ ∈ Rn with x∗ is a solution of the TCP(A, q), define δ = min kx∗ − uk2 for any u ∈ Rn . Thus, for any 0 < δb ≤ δ, we define a neighbourhood of x∗ as b := {x | kx − x∗ k2 ≤ δ}, b B(x∗ ; δ)

(4.1)

where x ∈ Rn . Now we recall some stability results from [34] on the following differential equation: dx(t) = f (x(t)), dt

x(t0 ) ∈ Rn .

(4.2)

The following classical results on the existence and uniqueness of the solution to (4.2) hold. Definition 4.1. ([34]) Let x(t) be a solution of (4.2). An isolated equilibrium point x∗ is Lyapunov stable if for any x(t0 ) and any scalar  > 0 there exists a δˆ > 0 so that if x(t0 ) ∈ b then kx(t) − x∗ k2 <  for t ≥ t0 . B(x∗ , δ) Definition 4.2. ([34]) An isolated equilibrium point x∗ is said to be asymptotic stable if in addition to being a Lyapunov stable it has the property that x(t) → x∗ as t → +∞, if x(t0 ) ∈ ˆ B(x∗ , δ). Then we focus on a particular case where the equilibrium point is isolated. Let S denote the solution set of the TCP(A, q) and x ∈ S implies Φ(x) = 0. Togather with F(·) is odd and monotonically increasing function, we have F(Φ(x)) = 0, consequently, dx dt = 0. Hence, we have the following result. Theorem 4.1. Every solution to the TCP(A, q) is an equilibrium point of the dynamical system (3.3). Conversely; if x ∈ Rn is an equilibrium of (3.3) and for any q ∈ Rn , Axm−1 + q is a P -function, then x ∈ S. Proof. We only need to address the second part of the Theorem. Since Axm−1 + q is a continuously differentiable P -function, it’s Jacobian matrix is a P -matrix for all x ∈ Rn . Analogy to the proof of [7, Corollary 4.4], we have a stationary point of (3.3) is a solution to the TCP(A, q). We complete our proof. Note that if A is a strong P -tensor (strong strictly semi-positive tensor), then T CP (A, q) has the global uniqueness and solvability property. Together with a strong P -tensor (strong strictly semi-positive tensor) is a P -tensor, we obtain the following results. Theorem 4.2. Let A be a strong P -tensor (strong strictly semi-positive tensor) and x be a solution of the TCP(A, q), then x is an unique equilibrium point of the dynamical system (3.3).

8

Proof. Since A is a strong P -tensor (strong strictly semi-positive tensor), then T CP (A, q) has the global uniqueness and solvability property. That is if x is a solution of the TCP(A, q), then x is a unique solution of the TCP(A, q). From the Theorem 4.1, we get x is an unique equilibrium point of the neural network (3.3). The proof is thus completed. We have the following result of the convergence of the NGDS model. Theorem 4.3. Given δb > 0 and x∗ be a solution of the TCP(A, q), if nonzero vector x(0) ∈ b and Ax(t)m−1 + q is a P -function for any q ∈ Rn , then the state x(t) of the NGDS B(x∗ ; δ) b converges to the solution x∗ ∈ Rn of model (3.3), starting from the initial state x(0) ∈ B(x∗ ; δ), the TCP(A, q). Proof. We construct the following Lyapunov function: 1 L(t) = ε(x(t)) = kΦ(x(t))k22 ≥ 0, 2

(4.3)

with its time derivative being dL(t) dt


d = 21 dt Tr Φ(x(t))> Φ(x(t))  = Φ(x(t))> dΦ(x(t))) dt
dx > = Φ(x(t)) V dt
= −γ Φ(x(t))> V V > F(Φ(x(t)))
= −γTr V V > Φ(x(t))> F(Φ(x(t))) .

(4.4)

Since V is a P -matrix, which follows from Axm−1 + q is a P -function for any q ∈ Rn , then, V V > is a symmetric positive definite matrix, and then, we have

λmin Tr Φ(x(t))> F(Φ(x(t))) ≤ Tr V V > Φ(x(t))> F(Φ(x(t)))
≤ λmax Tr Φ(x(t))> F(Φ(x(t))) , where λmin and λmax denote the smallest eigenvalue and the largest eigenvalue of matrix V V > , respectively. Tr(A) is the trace of the matrix A. Since the scalar-valued function f (·) is an odd and monotonically increasing function, it immediately follows f (−x) = −f (x) and   > 0, if x > 0, = 0, if x = 0, f (x)  < 0, if x < 0, which implies  xf (x) It follows that

> 0, if x 6= 0, = 0, if x = 0.

  > 0, if Φ(x(t)) 6= 0, Tr Φ(x(t)) F(Φ(x(t))) = 0, if Φ(x(t)) = 0. 

>

9

Due to the fact that the design parameter satisfies α > 0, in view of (4.4), it follows that  dL1 (t) < 0, if Φ(x(t)) 6= 0, = 0, if Φ(x(t)) = 0. dt By the Lyapunov theory, Φ(x(t)) can converge to zero; or, equivalently speaking, state x(t) of NGDS model (3.3) is asymptotic stable at one of solutions x∗ with Φ(x∗ ) = 0 starting from b The proof is thus complete. an initial state x(0) ∈ B(x∗ ; δ). We summarize the convergence result of the NGDS method (3.3) when A is strong P -tensor (strong strictly semi-positive tensor) in the following theorem. Theorem 4.4. Suppose that A ∈ R[m,n] is a strong P -tensor (strong strictly semi-positive tensor) and q ∈ Rn . The state x(t) of the NGDS model (3.3), starting from an arbitrary initial state x(0) ∈ Rn , converges to the unique solution x∗ ∈ Rn of the TCP(A, q). Proof. Since A is a strong P -tensor (strong strictly semi-positive tensor), from Theorem 4.2, we know T CP (A, q), for given q ∈ Rn , has the global uniqueness and solvability property. Together with the proof of Theorem 4.3, we can show that the state x(t) of the NGDS model (3.3), starting from an arbitrary initial state x(0) ∈ Rn , converges to the unique solution x∗ ∈ Rn of the TCP(A, q). We complete our proof.

5

Numerical examples

In this section, some computer-simulation examples are demonstrated to verify the efficacy and the superiority of the proposed neural network models. We apply the NGDS to the TCP(A, q). All computations are carried out in Matlab Version 2014a, which has a unit roundoff 2−53 ≈ 1.1 × 10−16 , on a laptop with Intel Core(TM) i5-4200M CPU (2.50GHz) and 7.89GB RAM. Example 5.1. Consider the tensor A ∈ R[4,2] , from [1, 24], defined by: a1111 = 1, a1222 = −1, a1122 = 1, a2222 = 1, a2111 = −1, a2211 = 1, and ai1 i2 i3 i4 = 0 otherwise. This tensor is a P -tensor, but not a strong P -tensor [1]. However, it is a strong strictly semi-positive tensor [24]. In our tests, we take different vectors q ∈ R2 and initial vector as x0 = rand(2, 1). Trajectories of state variables corresponding to the LGDS with γ = 10 are shown in Figure 1 (a), (c), (e) and Figure 2 (a) and (c), respectively. Residual errors Res = kΦ(x)k2 ,

10

(5.1)

(a) γ = 10, q = (−5, −3)> .

(b) γ = 1 × 106 , q = (−5, −3)> .

(c) γ = 10, q = (−5, 3)> .

(d) γ = 1 × 106 , q = (−5, 3)> .

(e) γ = 10, q = (5, 3)> .

(f) γ = 1 × 106 , q = (5, 3)> .

Figure 1: Trajectories of state variables and residual errors of the NGDSs for Example 5.1. derived by employing the NGDS with γ = 1 × 106 shown in Figures 1 (b), (d), (f) and Figure 2 (b) and (d), respectively, where blue stars indicate that f (·) is the linear function, red stars correspond to the bipolar-sigmoid function fbs (x, 5), pink stars denote that power-sigmoid function fps (x, 3, 5) and green stars are generated using fsps (x, 3, 7) as the smooth power-sigmoid function. 11

(a) γ = 10, q = (2, −3)> .

(b) γ = 1 × 106 , q = (2, −3)> .

(c) γ = 10, q = (0, −5)> .

(d) γ = 1 × 106 , q = (0, −5)> .

Figure 2: Trajectories of state variables and residual errors of the NGDSs for Example 5.1. Example 5.2. Let A ∈ R[5,3] be defined by akkkkk = k for k = 1, 2, 3 and ai1 i2 i3 i4 i5 = 0 otherwise. This tensor is both a strong P -tensor [1] and a strong strictly semi-positive [17]. For different vectors q ∈ R3 , we take initial vector as x0 = rand(3, 1). Trajectories of state variables corresponding to the LGDS with different γ are shown in Figure 3 (a), (c), (e) and Figure 4 (a), (c) and (e), respectively. Residual errors (5.1) derived by employing the NGDS with γ = 1 × 106 shown in Figures 3 (b), (d), (f) and Figure 4 (b), (d) and (f), respectively.

12

(a) γ = 1000, q = (1, 2, 3)> .

(b) γ = 1 × 106 , q = (1, 2, 3)> .

(c) γ = 1000, q = (1, −2, 3)> .

(d) γ = 1 × 106 , q = (1, −2, 3)> .

(e) γ = 100, q = (−3, −2, −3)> .

(f) γ = 1 × 106 , q = (−3, −2, −3)> .

Figure 3: Trajectories of state variables and residual errors of the NGDSs for Example 5.2. 13

(a) γ = 100, q = (3, 3, 3)> .

(b) γ = 1 × 106 , q = (3, 3, 3)> .

(c) γ = 100, q = (−3, −1, −2)> .

(d) γ = 1 × 106 , q = (−3, −1, −2)> .

(e) γ = 100, q = (−1, −1, −2)> .

(f) γ = 1 × 106 , q = (−1, −1, −2)> .

Figure 4: Trajectories of state variables and residual errors of the NGDSs for Example 5.2. 14

Where blue stars indicate that f (·) is the linear function, red stars correspond to the bipolarsigmoid function fbs (x, 7), pink stars denote that power-sigmoid function fps (x, 5, 7) and green stars are generated using fsps (x, 5, 9) as the smooth power-sigmoid function, respectively. Example 5.3. Let A ∈ R[4,2] be defined by a1111 = 1, a1112 = −2, a1122 = 1, a2222 = 1, and all other ai1 i2 i3 i4 = 0. This tensor is a P -tensor [1]. Taking q = (0, −1)> , it is easy to see that x∗ = (0, 1)> and x∗ = (1, 1)> are the solutions to the TCP(A, q). We choose initial vector as x0 = (0.1, 0.5)> for x∗ = (0, 1)> and x0 = (1.5, 1.1)> for x∗ = (1, 1)> . Trajectories of state variables corresponding to the LGDS with γ = 100 and γ = 10000 are shown in Figure 5 (a) and Figure 6 (a), respectively. Residual errors (5.1) derived by employing the NGDS with γ = 1 × 106 shown in Figures 5 (b) and Figures 6 (b), where blue stars indicate that f (·) is the linear function, red stars correspond to the bipolar-sigmoid function fbs (x, 7), pink stars denote that power-sigmoid function fps (x, 5, 9) and green stars are generated using fsps (x, 7, 11) as the smooth power-sigmoid function, respectively.

(a) γ = 100, x∗ = (0, 1)> .

(b) γ = 1 × 106 , x∗ = (0, 1)> .

Figure 5: Trajectories of state variables and residual errors of the NGDSs for Example 5.3. From the computer simulation results derived in the illustrative examples, the following conclusions can be highlighted: 1. The NGDS models with different types of activation functions presented in section 3, exactly and efficiently solve the TCP(A, q) with P -tensors, strong P -tensor and strong strictly semi-positive tensor.

15

(a) γ = 10000, x∗ = (1, 1)> .

(b) γ = 1 × 106 , x∗ = (1, 1)> .

Figure 6: Trajectories of state variables and residual errors of the NGDSs for Example 5.3. 2. NGDS models could achieve different performances if different activation function arrays are used. In general, the convergence performance of nonlinear activation functions is superior to that of the linear activation function.

References [1] X. Bai, Z. Huang, and Y. Wang, Global uniqueness and solvability for tensor complementarity problems, Journal of Optimization Theory and Applications, 170 (2016), pp. 72–84. [2] A. Berman and R. J. Plemmons, Nonnegative Matrices in the Mathematical Sciences. SIAM, Philadelphia, PA, 1994. [3] A. Bouzerdoum and T. R. Pattison, Neural network for quadratic optimization with bound constraints, IEEE Transactions on Neural Networks, 4 (1993), pp. 293–304. [4] M. Che, L. Qi, and Y. Wei, Positive-definite tensors to nonlinear complementarity problems, Journal of Optimization Theory and Applications, 168 (2016), pp. 475–487. [5] L. O. Chua and G. Lin, Nonlinear programming without computation, IEEE Transactions on Circuits and Systems, 31 (1984), pp. 182–188. [6] F. H. Clarke, Optimization and Nonsmooth Analysis, SIAM, Philadelphia, PA, 1990. [7] T. De Luca, F. Facchinei, and C. Kanzow, A semismooth equation approach to the solution of nonlinear complementarity problems, Mathematical Programming, 75 (1996), pp. 407–439. 16

[8] W. Ding, Z. Luo, and L. Qi, P-tensors, P0 -tensors, and their applications, Linear Algebra and its Applications, 555 (2018), pp. 336–354. [9] S. Du and L. Zhang, A mixed integer programming approach to the tensor complementarity problem, arXiv:1804.00406v1, (2018). [10] S. Du, L. Zhang, C. Chen, and L. Qi, Tensor absolute value equations, Science China Mathematics, (2018,https://doi.org/10.1007/s11425-017-9238-6). [11] F. Facchinei and J.-S. Pang, Finite-dimensional Variational Inequalities and Complementarity Problems, Springer Science & Business Media, 2007. [12] F. Facchinei and J. Soares, A new merit function for nonlinear complementarity problems and a related algorithm, SIAM Journal on Optimization, 7 (1997), pp. 225–247. [13] D. Feng, S. Yang, and T. Chen, Gradient-based identification methods for hammerstein nonlinear armax models, Nonlinear Dynamics, 45 (2006), pp. 31–43. [14] A. Fischer, A special newton-type optimization method, Optimization, 24 (1992), pp. 269– 284. [15] M. S. Gowda, Z. Luo, L. Qi, and N. Xiu, Z-tensors and complementarity problems, arXiv:1510.07933, (2015). [16] L. Han, A homotopy method for solving multilinear systems with M-tensors, Applied Mathematics Letters, 69 (2017), pp. 49 – 54. [17] L. Han, A continuation method for tensor complementarity problems, arXiv:1803.01433v1, (2018). [18] J. J. Hopfield and D. W. Tank, Neural computation of decisions in optimization problems, Biological Cybernetics, 52 (1985), pp. 141–152. [19] Z. H. Huang and L. Qi, Formulating an n-person noncooperative game as a tensor complementarity problem, Computational Optimization and Applications, 66 (2017), pp. 557– 576. [20] D. L. Kleinman and M. Athans, The design of suboptimal linear time-varying systems, IEEE Transactions on Automatic Control, 13 (2003), pp. 150–159. [21] S. Li, S. Chen, and B. Liu, Accelerating a recurrent neural network to finite-time convergence for solving time-varying sylvester equation by using a sign-bi-power activation function, Neural Processing Letters, 37 (2013), pp. 189–205. [22] L. Liao and H. Qi, A neural network for the linear complementarity problem, Mathematical and Computer Modelling, 29 (1999), pp. 9–18.

17

[23] L. Liao, H. Qi, and L. Qi, Solving nonlinear complementarity problems with neural networks: a reformulation method approach, Journal of Computational and Applied Mathematics, 131 (2001), pp. 343–359. [24] D. Liu, W. Li, and S. W. Vong, Tensor complementarity problems: the gus-property and an algorithm, Linear and Multilinear Algebra, 66 (2018), pp. 1726–1749. [25] Z. Luo, L. Qi, and N. Xiu, The sparsest solutions to z-tensor complementarity problems, Optimization Letters, 11 (2017), pp. 471–482. [26] L. Qi, Eigenvalues of a real supersymmetric tensor, J. Symbolic Comput., 40 (2005), pp. 1302–1324. [27] S. Ramezani, Nonlinear vibration analysis of micro-plates based on strain gradient elasticity theory, Nonlinear Dynamics, 73 (2013), pp. 1399–1421. [28] A. Rodriguezvazquez, R. Dominguezcastro, A. Rueda, J. L. Huertas, and E. Sanchezsinencio, Nonlinear switched capacitor ’neural’ networks for optimization problems, IEEE Transactions on Circuits and Systems, 37 (1990), pp. 384–398. [29] Y. Song and L. Qi, Properties of some classes of structured tensors, Journal of Optimization Theory and Applications, 165 (2015), pp. 854–873. [30]

, Tensor complementarity problem and semi-positive tensors, Journal of Optimization Theory and Applications, 169 (2016), pp. 1069–1078.

[31] Y. Song and G. Yu, Properties of solution set of tensor complementarity problem, Journal of Optimization Theory and Applications, 170 (2016), pp. 85–96. [32] Y. Wang, Z. H. Huang, and X. L. Bai, Exceptionally regular tensors and tensor complementarity problems, Optimization Methods and Software, 31 (2016), pp. 815–828. [33] S. L. Xie, D. H. Li, and H. R. Xu, An iterative method for finding the least solution to the tensor complementarity problem, Journal of Optimization Theory and Applications, 175 (2017), pp. 119–136. [34] J. Zabczyk, Mathematical Control Theory: an Introduction, Springer Science & Business Media, 2009. [35] S. H. Zak, V. Upatising, and S. Hui, Solving linear programming problems with neural networks: a comparative study, IEEE Transactions on Neural Networks, 6 (1995), pp. 94– 104. [36] Y. Zhang, A set of nonlinear equations and inequalities arising in robotics and its online solution via a primal neural network, Neurocomputing, 70 (2006), pp. 513–524.

18

[37] Y. Zhang, Y. Shi, K. Chen, and C. Wang, Global exponential convergence and stability of gradient-based neural network for online matrix inversion, Applied Mathematics and Computation, 215 (2009), pp. 1301–1306.

19