Lyapunov Functions to Caputo Fractional Neural Networks with ... - MDPI

axioms Article

Lyapunov Functions to Caputo Fractional Neural Networks with Time-Varying Delays Ravi Agarwal 1,2,† , Snezhana Hristova 3, * 1 2 3 4

* †

ID

and Donal O’Regan 4

Department of Mathematics, Texas A&M University-Kingsville, Kingsville, TX 78363, USA; [email protected] Florida Institute of Technology, Melbourne, FL 32901, USA Department of Applied Mathematics, University of Plovdiv Paisii Hilendarski, 4000 Plovdiv, Bulgaria School of Mathematics, Statistics and Applied Mathematics, National University of Ireland, H91 CF50 Galway, Ireland; [email protected] Correspondence: [email protected] Distinguished University Professor of Mathematics.

Received: 29 March 2018; Accepted: 5 May 2018; Published: 9 May 2018







Abstract: One of the main properties of solutions of nonlinear Caputo fractional neural networks is stability and often the direct Lyapunov method is used to study stability properties (usually these Lyapunov functions do not depend on the time variable). In connection with the Lyapunov fractional method we present a brief overview of the most popular fractional order derivatives of Lyapunov functions among Caputo fractional delay differential equations. These derivatives are applied to various types of neural networks with variable coefficients and time-varying delays. We show that quadratic Lyapunov functions and their Caputo fractional derivatives are not applicable in some cases when one studies stability properties. Some sufficient conditions for stability of equilibrium of nonlinear Caputo fractional neural networks with time dependent transmission delays, time varying self-regulating parameters of all units and time varying functions of the connection between two neurons in the network are obtained. The cases of time varying Lipschitz coefficients as well as nonLipschitz activation functions are studied. We illustrate our theory on particular nonlinear Caputo fractional neural networks. Keywords: nonlinear Caputo fractional neural networks; delays; Lyapunov functions; stability; fractional derivative of Lyapunov functions

1. Introduction Neural networks have a wide range of applications in pattern recognition, associative memory, combinatorial optimization, etc. Delays are incorporated into the model equations of the networks because of the finite speeds of the switching and transmission of signals in a network and this leads to time delays in a working network. It was observed both experimentally and numerically in [1] that time delay could induce instability, causing sustained oscillations which may be harmful to a system. Additionally, when one considers memory and hereditary properties of various materials and processes ([2]), it is natural to consider fractional derivatives in the model. Neural networks in biology, coupled lasers, wireless communication and power-grid networks in physics and engineering ([3–6]) are modeled by fractional order differential equations. In controlling nonlinear systems, the Lyapunov second method provides a way to analyze the stability of the system without explicitly solving the differential equations. Stability results concerning Axioms 2018, 7, 30; doi:10.3390/axioms7020030

www.mdpi.com/journal/axioms

Axioms 2018, 7, 30

2 of 18

integer-order neural networks can be found in [7–9] and recently Lyapunov stability theory for fractional order systems was discussed (see [10,11]). Fractional order Lyapunov stability theory was applied for various types of fractional neural networks using quadratic Lyapunov functions (see [2,12–14]) and stability analysis of fractional-order delay neural networks can be found in [15–19], for example. Finite-time stability of fractional-order neural networks with constant transmission delay and constant self-regulating parameters is studied in [20,21] and for distributed delays and constant self-regulating parameters see [22]. The main goal of this paper is to study stability properties of Caputo fractional order neural networks with delays. We study the general case of time varying transmission delays as well as time varying self-regulating parameters of all units and time varying functions of the connection between two neurons in the network. Note that usually in the literature the activation functions in the models are Lipschitz with constant Lipschitz coefficients. In this paper, the cases of time varying Lipschitz coefficients as well as nonLipschitz activation functions are studied. The investigation is based on the fractional Lyapunov method. It is worth pointing out that the Lyapunov functional method can not be easily generalized to fractional order systems. It is mainly connected with the derivative taken in the given fractional system. Taking these factors into consideration, we present a brief overview of the existing literature on the various definitions of fractional order derivatives of Lyapunov functions among Caputo fractional differential equations with variable delays and we compare them in several examples, discussing their advantages and disadvantages. Using the appropriate fractional derivative of Lyapunov functions, some stability sufficient criteria are provided and illustrated with examples. 2. Lyapunov Functions and Their Derivatives among the Delay Fractional Differential Equation We first consider the derivative of Lyapunov functions among the delay fractional differential equation. In this paper we consider Caputo fractional derivatives with order q ∈ (0, 1) defined by (see, for example [23]) C q t 0 Dt m ( t )

1 = Γ (1 − q )

Zt

(t − s)−q m0 (s)ds,

t ≥ t0 ,

t0

where Γ(.) denotes the Gamma function. Consider the initial value problem (IVP) for the nonlinear Caputo fractional delay differential equations (FrDDE) C q t 0 Dt x ( t )

= f (t, xt ) for t ∈ [t0 , t0 + T ),

n

n

x (t0 + Θ) = ψ0 (Θ), Θ ∈ [−r, 0],

n

(1) n

where x ∈ R , f ∈ C [[t0 , t0 + T ) × R , R ], t0 ∈ R+ is the initial time, ψ0 ∈ C ([−r, 0], R ) is the initial function, T ≤ ∞ and for any t ∈ [t0 , t0 + T ) the notation xt (Θ) = x (t + Θ), Θ ∈ [−r, 0] is used. Let x (t), t ∈ [t0 − r, t0 + T ), be a solution of the IVP for the FrDDE (1) and let V (t, x ) be a Lyapunov function, i.e., V (t, x ) : [t0 − r, t0 + T ) × ∆ → R+ is continuous and locally Lipschitzian with n respect to its second argument, where ∆ ⊂ R , 0 ∈ ∆. In the literature there are three types of derivatives of Lyapunov functions among solutions of fractional differential equations used to study stability properties: -

first type–Let x (t) be a solution of IVP for FrDDE (1). Then the Caputo fractional derivative of the Lyapunov function V (t, x (t)) is defined by C q t0 Dt V ( t, x ( t ))

1 = Γ (1 − q )

Zt t0

(t − s )−q

 d V (s, x (s)) ds, ds

t ∈ ( t0 , t0 + T ).

(2)

Axioms 2018, 7, 30

-

3 of 18

This type of derivative is applicable for continuously differentiable Lyapunov functions. It is used mainly for quadratic Lyapunov functions to study several stability properties of fractional differential equations (see, for example [10]). second type–this type of derivative of V (t, x ) among FrDDE (1) was introduced by I. Stamova, G. Stamov (Definition 1.12 [24]): D(+1) V (t, φ(0)) = lim sup h →0

i 1h q V ( t, φ ( 0 )) − V ( t − h, x − h f ( t, φ )) , t ∈ ( t0 , t0 + T ) 0 hq

(3)

n

where φ ∈ C ([−r, 0], R ) and the notation φ0 (Θ) = φ(0 + Θ), Θ ∈ [−r, 0] is used. Note that the operator defined by (3) has no memory (the memory is typical for fractional derivatives) (see Case 2.1 of Example 1 and Case 2.2 of Example 2). Remark 1. In general D(+1) V (t, x (t)) 6=

C q 0 Dt V ( t, x ( t ))

where x (t) is a solution of (1).

Now, let us recall the remark in [25] concerning definition (3) where V (t − h, x − hq f (t, x )) is defined by [ q

V (t − h, x − h f (t, x )) =

t − t0 h ]

∑

(−1)r+1 q Cr V (t − rh, x − hq f (t, x ))

r =1

where q Cr =

q(q−1)...(q−r +1) . r!

Following this notation the fractional derivative of the Lyapunov function is defined by

D(+1) V (t, φ(0); t0 ) 1h = lim sup q V (t, φ(0)) − h →0 h

[

t − t0 h ]

∑

i (−1)r+1 q Cr V (t − rh, φ(0) − hq f (t, φ0 )) .

(4)

r =1

The derivative (4) has memory and it depends on the initial time t0 . It is closer to both the Grunwald-Letnikov fractional derivative and the Riemann-Liouville fractional derivative, than to the Caputo fractional derivative of a function. We will call the derivative (4) the Dini fractional derivative of the Lyapunov function. The Dini fractional derivative is applicable for continuous Lyapunov functions. Remark 2. In the general case D(+1) V (t, φ(0)) 6= D(+1) V (t, φ(0)). -

n

third type - for any φ ∈ C ([−r, 0], R ) the derivative of the Lyapunov function V (t, x ) among IVP n for FrDDE (1) with initial point t0 and initial function ψ0 ∈ C ([−r, 0], R ) is defined by: q c (1) D+ V ( t, φ (0); t0 , ψ0 ) t−t [ h0]

−

∑

(−1)

r +1

q Cr



= lim sup

q

V (t − rh, φ(0) − h f (t, φ0 )) − V (t0 , ψ0 (0))

r =1

for t ∈ (t0 , t0 + T ), or its equivalent

h →0+

 1 V (t, φ(0)) − V (t0 , ψ0 (0)) hq 

(5)

Axioms 2018, 7, 30

4 of 18

q c (1) D+ V ( t, φ (0); t0 , ψ0 ) t−t   [ h0] 1 = lim sup q V (t, φ(0)) + ∑ (−1)r q Cr V (t − rh, φ(0) − hq f (t, φ0 )) h →0+ h r =1

−

V (t0 , ψ0 (0)) , ( t − t0 ) q Γ (1 − q )

(6)

for t ∈ (t0 , t0 + T ).

The derivative (6) depends significantly on both the fractional order q and the initial data (t0 , ψ0 ) of IVP for FrDDE (1) and this type of derivative is close to the idea of the Caputo fractional derivative of a function. We call the derivative given by (5) or its equivalent (6) the Caputo fractional Dini derivative. This type of derivative is applicable for continuous Lyapunov functions (see, for example [26–28]). Remark 3. The equality q c (1) D+ V ( t, φ (0); t0 , ψ0 )

= D(+1) V (t, φ(0); t0 ) −

V (t0 , ψ0 (0)) ( t − t0 ) q Γ (1 − q )

n

(7) n

holds for any t ∈ (t0 , t0 + T ), φ0 ∈ C ([−r, 0], R ) and for any initial data (t0 , ψ0 ) ∈ R+ × C ([−r, 0], R ) : q c (1) D+ V ( t, φ (0); t0 , ψ0 )

= D(+1) V (t, φ(0); t0 ), if V (t0 , ψ0 (0)) = 0

(8)

q c (1) D+ V ( t, φ (0); t0 , ψ0 )

< D(+1) V (t, φ(0); t0 ), if V (t0 , ψ0 (0)) > 0.

(9)

Remark 4. Note that in the case of delays in the differential equations, the derivative of the Lyapunov function n is considered for functions φ ∈ C ([−r, 0], R ) and points t such that V (t, φ(0)) = sups∈[−r,0] V (t + s, φ(s)). This condition is called the Razumikhin condition. We will illustrate the application of the above given derivatives on the quadratic Lyapunov function (used in stability of neural networks [2,12–14]). Example 1. Let n = 1. Consider the quadratic Lyapunov function i.e., V (t, x ) = x2 . Case 1. First type of derivative. Let x (t) be a solution of the IVP for FrDDE (1). According to [29] we get C q t0 Dt V ( t, x ( t ))

=

2 Γ (1 − q )

Zt

q

(t − s)−q x (s) x 0 (s)ds ≤ 2x (t) Ct0 Dt x (t) = 2x (t) f (t, xt ).

(10)

t0

Case 2. Second type of derivative. n Case 2.1: Let the function φ ∈ C ([−r, 0], R ). From Formula (3) we obtain D(+1) V (t, φ(0)) = lim sup h →0

2  2 i 1 h q φ ( 0 ) − φ ( 0 ) − h f ( t, φ ) = 2φ(0) f (t, φ0 ). 0 hq

(11)

It is seen that the derivative D(+1) V (t, φ(0)) does not depend on any initial data of the fractional equation. n

Case 2.2: Dini fractional derivative. Let the function φ ∈ C ([−r, 0], R ). From Formula (4) we obtain

Axioms 2018, 7, 30

5 of 18

D(+1) V (t, φ(0); t0 ) 2 1 h = lim sup q φ(0) − h →0 h 2 1 h = lim sup q φ(0) + h →0 h  2 1 = φ(0) lim sup q h h →0

[

[

t − t0 h ]

∑

 2 i (−1)r+1 q Cr φ(0) − hq f (t, φ0 )

r =1

[

t − t0 h ]

∑

 2 i (−1)r q Cr φ(0) − hq f (t, φ0 )

r =1

t − t0 h ]

∑

(12)

(−1)r q Cr

r =0



q 2

+ lim sup − 2φ(0) f (t, φ0 ) + h f (t, φ0 ) h →0

[

t − t0 h ]

∑

(−1)r q Cr

i

r =1


2

= 2φ(0) f (t, φ0 ) +

φ (0) ( t − t0 ) q Γ (1 − q )

The Dini fractional derivative depends on both the fractional order q and initial time. Case 3. Third type of derivative. Caputo fractional Dini derivative . According to Remark 4 and Case 2.2 the inequality
2
2 ψ0 (0) φ (0) = 2φ(0) f (t, φ0 ) + − ( t − t0 ) q Γ (1 − q ) ( t − t0 ) q Γ (1 − q )

q c (1) D+ V ( t, φ (0); 0, ψ0 )

(13)

holds. The Caputo fractional Dini derivative depends on both the fractional order q and initial data (t0 , ψ0 ). From the literature we note that one of the sufficient conditions for stability is connected with the sign of the derivative of the Lyapunov function of the equation. Example 2. Consider the IVP for the scalar linear FrDDE C q 0 Dt x ( t )

= g(t) x (t) + 0.1x (t − π ) for t > 0,

where q ∈ (0, 1), g(t) = −0.5

RL q 0 D



sin2 (t)+0.1

sin2 (t)+0.1

x (s) = 1 for s ∈ [−π, 0],

(14)



− 0.1, t > 0.

Case 1. Consider the quadratic Lyapunov function [12–14]) i.e., V (t, x ) = x2 . Let x (t) be a solution of IVP for FrDDE (14) and t > 0 be such that V (t, x (t)) = x2 (t) = sups∈[−π,t] x2 (s) = sups∈[−π,t] V (s, x (s)). Then according to [29] we get C q 0 Dt V ( t, x ( t )) q

q

≤ 2x (t) 0C Dt x (t) = 2( x (t))2 g(t) + 0.2x (t) x (t − π ) ≤ 2x2 (t)(0.1 + g(t)).

The sign of 0C Dt V (t, x (t)) is changeable (the graph of the function g(t) + 0.1 for various values of q is given on Figure 1). Case 2. Consider the function V (t, x ) = (sin2 (t) + 0.1) x2 . Case 2.1: Caputo fractional derivative. Let x (t) be a solution of IVP for FrDDE (14). From Equation (2) we obtain

Axioms 2018, 7, 30

6 of 18

C q 0 Dt V ( t, x ( t ))

1 = Γ (1 − q )

Zt

  (t − s)−q 2(sin2 (s) + 0.1) x (s) x 0 (s) + sin(2s) x2 (s) ds.

0

The fractional derivative of this function V is difficult to obtain so it is difficult to discuss its sign. Case 2.2: Use Formula (3) and for any function φ ∈ C ([−π, 0], R) and any point t > 0 such that V (t, φ(0)) = (sin2 (t) + 0.1)φ2 (0) = sups∈[−π,0] (sin2 (t + s) + 0.1)φ2 (s) = sups∈[−π,0] V (t + s, φ(s)) we obtain D(+14) V (t, φ(0))
1h = lim sup q sin2 (t) − sin2 (t − h) φ2 (0) h →0 h

−2hq (sin2 (t − h) + 0.1)φ(0) φ(0) g(t) + 0.1φ(−1)
2 i +h2q (sin2 (t − h) + 0.1) φ(0) g(t) + 0.1φ(−π )




= 2φ(0)(sin2 (t) + 0.1)(φ(0) g(t) + 0.1φ(−π )) ≤ 2φ2 (0)(sin2 (t) + 0.1) g(t) + 0.2φ(0)φ(−π )(sin2 (t) + 0.1) ≤ 2φ2 (0)(sin2 (t) + 0.1) g(t) + 0.1φ2 (0)(sin2 (t) + 0.1) + 0.1φ2 (−π )(sin2 (t − π ) + 0.1) ≤ 2φ2 (0)(sin2 (t) + 0.1)( g(t) + 0.1) i.e., the sign of the derivative D(+14) V (t, φ) is changeable. Case 2.3: Dini fractional derivative. For any function φ ∈ C ([−π, 0], R) and t > 0 such that V (t, φ(0)) = (sin2 (t) + 0.1)φ2 (0) = sups∈[−π,0] (sin2 (t + s) + 0.1)φ2 (s) = sups∈[−π,0] V (t + s, φ(s)) we apply Formula (4) and obtain

D(+14) V (t, φ(0); 0) 1h = lim sup q (sin2 (t) + 0.1)φ2 (0) h →0 h [ ht ]


2 i − ∑ (−1)r+1 q Cr (sin2 (t − rh) + 0.1) φ(0) − hq φ(0) g(t) + 0.1φ(−π ) r =1

1 = φ (0) lim sup q h h →0 2

[ ht ]

∑ (−1)r (sin2 (t − rh) + 0.1)

r =1

[ ht ]

−2 lim sup ∑ (−1)r q Cr (sin2 (t − rh) + 0.1) φ2 (0) g(t) + 0.1φ(0)φ(−π ) h →0

r =1

+ lim sup h h →0




q

[ ht ]

∑ (−1)r+1 q Cr (sin2 (t − rh) + 0.1)

φ(0) g(t) + 0.1φ(−π )


2

r =1

  ≤ 2φ (0)( g(t) + 0.1)(sin2 (t) + 0.1) + φ2 (0) 0RL D q sin2 (t) + 0.1 = 0. 2

Case 2.4: Caputo fractional Dini derivative . According to Remark 4 and Case 2.3 the inequality q c (14) D+ V ( t, φ (0); 0, ψ0 )

= D(+14) V (t, φ(0); 0) −

0.1ψ02 (0) 0.1ψ02 (0) = − ≤0 t q Γ (1 − q ) t q Γ (1 − q )

holds. Therefore, for (14) both the Dini fractional derivative and the Caputo fractional Dini derivative seem to be more applicable than the Caputo fractional derivative of Lyapunov function.

Axioms 2018, 7, 30

7 of 18

x 2

1

q=0.3 t 2

4

6

8

q=0.8

10

q=0.5 -1

-2

Figure 1. Example 2. Graph of the function g(t) + 0.1 for various values of q.

Remark 5. The above example notes that the quadratic function for studying stability properties of neural network might not be successful (especially when the right hand side depends directly on the time variable). Formula (3) is not appropriate for applications to fractional equations. The most general derivatives for non-homogenous fractional differential equations are Dini fractional derivatives and Caputo fractional Dini derivatives. 3. Remarks on Some Applications of Derivatives of Lyapunov Functions Some authors use the derivative defined by (3) to study stability properties of delay fractional differential equations ([24,30]) and delayed reaction-diffusion cellular neural networks of fractional order ([31]). The proofs are based on the following result: Lemma 1. ([24]) Assume that: 1. 2. 3.

The function g : [t0 , ∞) × R+ → R is continuous in [t0 , ∞) × R+ and is nondecreasing in u for each t ≥ t0 . q The maximal solution u+ (t) of the IVP for scalar Caputo fractional differential equation ct0 Dt u(t) = g(t, u(t)), t > t0 , u(t0 ) = 1 exists on [t0 , ∞). The Lipschitz function V (t, x ) is such that for t ≥ t0 , φ ∈ C ([−r, 0], ∆) the inequality D(+1) V (t, φ(0)) ≤ g(t; V (t, φ(0))) n

is valid whenever V (t + Θ, φ(Θ)) ≤ V (t, φ(0)) for −r ≤ Θ ≤ 0 where ∆ ⊂ R , 0 ∈ ∆. Then max−r≤Θ≤0 V (t0 + Θ) ≤ u0 implies V (t, x (t; t0 , φ0 )) ≤ u+ (t; t0 , u0 ), t ∈ [t0 , ∞), where x (t; t0 , φ0 ) is the solution of (1) with initial data (t0 , ψ0 ) ∈ R+ × C [[−r, 0], ∆]. The claim of Lemma 1 is not true. We will give a counterexample. h(t)

Example 3. Consider the IVP for FrDDE (1) with n = 1, t0 = 0, r = 0, g(t, u) = h(t), f (t, xt ) = 0.5 x(t) , t2−q F [{1},{3/2−q/2,2−q/2},−(t2 /4)]

p q n z ψ0 (t) ≡ 1, and h(t) = − where p Fq [{ a}, {b, c}, z] = ∑∞ n=0 (b)n (c)n n! with Γ(1−q)(1−q)(2−q) ( a)n = a( a + 1)( a + 2) . . . ( a + n − 1) is the general hypergeometric function or Aomoto-Gelfand hypergeometric function [32]. We denote its solution by x (t). Consider the Lyapunov function V (t, x ) = x2 . According to Example 1, Case 2.1, Equation (11) we get D(+1) V (t, φ(0)) = 2φ(0) f (t, φ0 ) = h(t). The function u(t) = cos(t) is a solution of the IVP for the comparison scalar fractional differential equation

c q 0 Dt u ( t )

= h(t), t > 0,

( a)

u (0) = 1

n

Axioms 2018, 7, 30

8 of 18

Then V (0, x (0)) = 1 but the inequality 0 ≤ V (t, x (t)) = ( x (t))2 ≤ cos(t) = u(t) does not hold for t ≥ 0. The application of the derivative D(+1) V (t, φ(0)) defined by (3) is not appropriate in studying stability properties of neural networks of fractional order (as presented in [31]). We will consider fractional neural networks with delays and using Lyapunov functions and their Caputo fractional derivative, Dini fractional derivative or Caputo fractional Dini derivative (and without applying Lemma 1) we study stability. 4. Stability for Caputo Fractional Order Neural Networks with Time Varying Delays 4.1. System Description Consider the general model of fractional order neural networks with time varying delay (FODNN) C q 0 Dt x i ( t )

n

n

j =1

j =1

= −ci (t) xi (t) + ∑ aij (t) f j ( x j (t)) + ∑ bij (t) g j ( x j (t − τ (t))) + Ii (t)

for t > 0,

(15)

i = 1, 2, . . . n,

where q ∈ (0, 1), n represents the number of units in the network, xi (t) is the pseudostate variable n of the i-th unit of master system, x (t) = [ x1 (t), x2 (t), . . . , xn (t)] T ∈ R , ci (t) > 0, i = 1, 2, . . . , n, is the self-regulating parameters of the i-th unit, , aij (t), bij (t), i, j = 1, 2, . . . , n, correspond to the connection of the i-th neuron to the j-th neuron at times t and t − τ (t) respectively, f j ( x j (t)) and g j ( x j (t − τ (t))) denote the activation functions of the neurons at time t and t − τ (t), respectively, f ( x ) = [ f 1 ( x1 ), f 2 ( x2 ), . . . , f n ( xn )] T , g( x ) = [ g1 ( x1 ), g2 ( x2 ), . . . , gn ( xn )] T and I = [ I1 , I2 , . . . , In ] T is an external bias vector, τ (t) ∈ C (R+ , [0, r ]) is the transmission delay. The initial conditions of the given system (15) are listed as xi (t) = ψi (t) for t ∈ [−r, 0],

(16)

where ψi ∈ C (−r, 0], R), i = 1, 2, . . . , n, with the norm ||ψ|| = supt∈[−r,0] ∑in=1 |ψi (t)|. Remark 6. The problem of existence and uniqueness of equilibrium states of fractional-order neural networks was investigated by several authors; for sufficient conditions which guarantee the existence and uniqueness of solutions for FODNN with constant delays we refer the reader to [15]. n

Definition 1. A vector x ∗ ∈ R , x ∗ = ( x1∗ , x2∗ , . . . , xn∗ ) is an equilibrium point of Caputo FODNN (15), if the equalities 0 = −ci (t) xi∗ + ∑nj=1 aij (t) f j ( x ∗j ) + ∑nj=1 bij (t) g j ( x ∗j ) + Ii (t) hold for all t > 0, i = 1, 2, . . . , n. We will discuss the equilibrium points on some FODNN with various activation functions. It will be useful for stability analysis. Example 4. Consider the system of FODNN (15) with n = 3, ci (t) ≡ ci , with the activation functions f j (s) = g j (s) = 0.5 tanh(s), and aij (t) ≡ aij , bij (t) ≡ bij and Ii (t) ≡ Ii . Then the point x ∗ = (ξ 1 , ξ 2 , ξ 3 ) is an equilibrium point iff 0 = 2ci ξ i + ∑3j=1 ai,j tanh(ξ i ) + ∑3j=1 bi,j tanh(ξ i ) + Ii , i = 1, 2, 3.

Axioms 2018, 7, 30

9 of 18

Example 5. Let n = 1, c is a constant and consider the scalar FODNN C q 0 Dt x ( t )

= −cx (t) + a(t) f ( x (t)) + b(t) g( x (t − r )) + I for t > 0.

(17)

Case 1. Let I = 0.5πc and the activation function be the cosine function f (u) = g(u) = cos(u) (see [33]). The point x ∗ = 0.5π is an equilibrium point of FODNN(17) because −c0.5π + a(t) cos(0.5π ) + a(t) cos(0.5π ) + 0.5πc = 0 for all t > 0. √ Case 2. Let I = 0.5c and the activation function be the Probit function f (u) = g(u) = 2 erf−1 (2u − 1) Ru 2 (see Figure 2) where erf(u) = √2π 0 e−t dt is the error function. Then erf−1 (0) = 0. The point x ∗ = 0.5 is an equilibrium point of FODNN(17) because −c0.5 + a(t) f (0.5) + b(t) g(0.5) + 0.5c = 0 for all t > 0.   Case 3. Let I = 0.5c and the activation function be the Logit function f (u) = g(u) = log

u 1− u

(see Figure 3). Then f (0.5) = 0. The point x ∗ = 0.5 is an equilibrium point of Caputo FODNN (17) because −c0.5 + a(t) f (0.5) + b(t) g(0.5) + 0.5c = 0 for all t > 0. x 3 2 1

-0.4

-0.2

0.2

0.4

0.6

0.8

1.0

t

-1 -2 -3 -4

Figure 2. Example 5. Graph of the function f (t) =

√

2 erf−1 (2t − 1).

x

4

2

0.2

0.4

0.6

0.8

1.0

t

-2

-4

-6

Figure 3. Example 5. Graph of the function f (u) = log



u 1− u



.

Assumption 1. Let the Caputo FODNN (15) have an equilibrium point x ∗ . If Assumption 1 is satisfied then we can shift the equilibrium point x ∗ of system (15) to the origin. The transformation y(t) = x (t) − x ∗ is used to put system (15) in the following form:

Axioms 2018, 7, 30

10 of 18

C q 0 Dt y i ( t )

n

n

j =1

j =1

= −ci (t)yi (t) + ∑ aij (t) Fj (y j (t)) + ∑ bij (t) Gj (y j (t − τ (t)))

(18)

for t > 0, i = 1, 2, . . . n, where Fj (u) = f j (u + x ∗j ) − f j ( x ∗j ), Gj (u) = g j (u + x ∗j ) − g j ( x ∗j ), u ∈ R, j = 1, 2, . . . , n. We will give the Dini fractional derivative of a Lyapunov function depending directly on the time t among the FODNN (18) which will be used in the proofs of some of the main results. Example 6. Consider the IVP for FODNN (18). Consider the function V (t, x ) = p(t) ∑in=1 xi2 where p ∈ C ([−r, ∞), R+ : α ≤ p(t) ≤ β for t ≥ −r n where α, β > 0 are constants. Let φ ∈ C ([−r, 0], R ) and t > 0 be such that V (t, φ(0)) = p(t) ∑in=1 φi2 (0) = n 2 sups∈[−r,0] p(t + s) ∑i=1 φi (s) = sups∈[−r,0] V (t + s, φ(s)). Then from Formula (4) we obtain for the Dini fractional derivative:

D(+18) V (t, φ(0); 0) n

=

∑ φi2 (0) lim sup h →0

i =1

1 hq

[ ht ]

∑ (−1)r p(t − rh)

r =1

[ ht ] 2 (−1)r q Cr p(t − rh) +2 ci (t)φi (0) lim sup h →0 r =1 i =1 n

∑

∑

−2



n

n

∑ φi (0) ∑ aij (t) Fj (φj (0))

i =1

j =1

n

n




[ ht ]

lim sup ∑ (−1)r q Cr p(t − rh) h →0

∑ φi (0) ∑ bij (t)Gj (φj (−τ (t))

−2

i =1

j =1




r =1

[ ht ]

lim sup ∑ (−1)r q Cr p(t − rh) h →0

[ ht ]

r =1

n

n

i =1

j =1

∑ (−1)r+1 q Cr p(t − rh) ∑ − ci (t)φi (0) + ∑ aij (t) Fj (φj (0))

+ lim sup hq h →0

r =1

n

+ ∑ bij (t) Gj (φj (−τ (t))


2

j =1



n

≤ 2p(t) − ∑

ci (t)φi2 (0) +

i =1

n

+

∑ φi2 (0)


RL 0

n

n

i =1

j =1

∑ φi (0) ∑



aij (t) Fj (φj (0)) + bij (t) Gj (φj (−τ (t))



D q p ( t ).

i =1

4.2. Some Results for Caputo Fractional Differential Equations We will give some results for fractional derivatives of Lyapunov functions among Caputo fractional differential equations which will be used for our main results concerning the stability of FODNN. n×n

n

Lemma 2. ([29]). Let P ∈ R be constant, symmetric and positive definite matrix and X (t) : R+ → R be a function with the  Caputo fractional  derivative existing. Then

1 C q 2 0 Dt

q

x T (t) Px (t) ≤ x T (t) P 0C Dt x (t), t ≥ 0.

Axioms 2018, 7, 30

11 of 18

Lemma 3. (Theorem 3.1 [16]). Let w1 , w2 ∈ C (R, R+ ) be nondecreasing functions, wi (s) > 0, i = 1, 2 for s > 0, wi (0) = 0, i = 1, 2, w2 (s) us strictly increasing and there exists a continuously differentiable Lyapunov n function V (t, x ) ∈ R+ × R → R+ such that (i) (ii)

n

w1 (|| x ||) ≤ V (t, x ) ≤ w2 (|| x ||), t ≥ 0, x ∈ R , for any t > 0 such that supΘ∈[−r,t] V (Θ, x (Θ)) = V (t, x (t)) the inequality C β 0 Dt V ( t, x ( t ))

≤0

holds where x (t) is a solution of (1) with t0 = 0. Then the equilibrium point of (1) is uniformly stable. For the Caputo fractional Dini derivative we have: Lemma 4. (Theorem 6 [34]). Assume x = 0 be an equilibrium point for (1) with T = ∞ and there exists a Lyapunov function V (t, x ) : V (t, 0) = 0, t ≥ 0, such that (i)

α1 (|| x ||) ≤ V (t, x ) ≤ α2 (|| x ||) for t ≥ t0 , x ∈ ∆,

(ii)

where αi ∈ C ([0, ∞), [0, ∞)), i = 1, 2 are strictly increasing and αi (0) = 0. for any initial data (t0 , ψ0 ) ∈ R+ × C ([−r, 0], ∆) and any function φ ∈ C ([−r, 0], ∆) such that if for a point t ≥ t0 we have V (t + Θ, φ(Θ)) ≤ V (t, φ(0)) for Θ ∈ [−r, 0) then the inequality q c (1) D+ V ( t, φ (0); t0 , ψ0 )

≤0

holds. Then the equilibrium point of (1) is uniformly stable. 4.3. Stability Analysis We will study stability properties of several different types of FODNN (1) using different types of Lyapunov functions and their fractional derivatives given in Section 2. 4.3.1. Lipschitz Activation Functions and Quadratic Lyapunov Functions The stability analysis of the fractional-order neural networks with constant delays and Lipschitz active functions was studied in [35] and the argument was based on topological degree theory, nonsmooth analysis and a nonlinear measure method. We will apply the Lyapunov method to derive some sufficient conditions for stability in the case of variable delays. We assume the following: Assumption 2. The neuron activation functions are Lipschitz, i.e., there exist positive numbers Li , Hi , i = 1, 2, . . . , n such that | f i (u) − f i (v)| ≤ Li |u − v| and | gi (u) − gi (v)| ≤ Hi |u − v|, i = 1, 2, . . . , n for u, v ∈ R. Assumption 3. There exists positive numbers Mi,j , Ci,j , i, j = 1, 2, . . . , n such that | ai,j (t)| ≤ Mi,j , |bi,j (t)| ≤ Ci,j for t > 0. Assumption 4. The inequality 2 min ci > i =1,n

holds.

n

n

i =1

j =1

Mij L j + max Cij Hj ) + max( ∑ ( Mij L j + Cij Hj ) ∑ (max j j i

(19)

Axioms 2018, 7, 30

12 of 18

Remark 7. If Assumption 2 is satisfied then the functions F, G in FODNN (18) satisfy | Fj (u)| ≤ L j |u|, | Gj (u)| ≤ Hj |u|, j = 1, 2, . . . , n for any u ∈ R. The case of multiple time constant delays and the functions of the connection of the i-th neuron to the j-th neuron at time t in FODNN (15) are constants studied using the quadratic Lyapunov function in [14]. We will study the case of variable bounded functions of connection. Theorem 1. Let c j (t) ≡ c j > 0, j = 1, 2, . . . , n and Assumptions 1–4 are satisfied. Then the equilibrium point x ∗ of Caputo FODNN (15) is uniformly stable. Proof. Consider the quadratic functions V (t, x ) = x T x. Let the point t > 0 be such that supΘ∈[−r,t] V (Θ, x (Θ)) = supΘ∈[−r,t] ∑nj=1 x2j (Θ) = ∑nj=1 x2j (t) = V (t, x (t)) where x (t) is a solution of the FODNN (18). Then since τ (t) ∈ [0, r ], t ≥ 0, we have ∑nj=1 ( x j (t))2 ≥ ∑nj=1 ( x j (t − τ (t)))2 , i = 1, 2, . . . , n. Applying Lemma 2 we get C q 0 Dt V ( t, x ( t ))

=

C q T 0 Dt x ( t ) x ( t )

n

≤ 2x T (t) 0C Dt x (t) = 2 ∑ xi (t) 0C Dt xi (t) q

q

i =1

n

n

≤ −2 ∑ ci xi2 (t) + 2 ∑ i =1

n

∑ |aij (t)|| Fj (x j (t))|xi (t)|

i =1 j =1

n

+2∑

n

∑ |bij (t)||Gj (x j (t − τ (t)))|xi (t)|

i =1 j =1

n

i =1,n n

+2∑

n

n

∑ xi2 (t) + ∑ ∑ Mij L j (x2j (t) + xi2 (t))

≤ −2 min ci

i =1

i =1 j =1

n

∑ Cij Hj |x j (t − τ (t))||xi (t)|

(20)

i =1 j =1

n

≤ −2 min ci i =1,n n

+∑

n

n

n

n

∑ xi2 (t) + ∑ ∑ Mij L j x2j (t) + ∑ ∑ Mij L j xi2 (t)

i =1

i =1 j =1

i =1 j =1

n

n

n

∑ Cij Hj x2j (t − τ (t)) + ∑ ∑ Cij Hj xi2 (t)

i =1 j =1

i =1 j =1

n

≤ −2 min ci i =1,n

∑ xi2 (t) +

i =1

n

n



∑ (max Mij L j + max Cij Hj )

i =1

j

j



n

∑ x2j (t)

j =1

n

+ ∑ xi2 (t) max( ∑ ( Mij L j + Cij Hj ) ≤ 0.


i =1

i

j =1

Inequality (20) and Lemma 3 prove the claim. Example 7. Consider the system of FODNN (15) with n = 3, ci (t) ≡ ci , with the activation functions f j (s) = g j (s) = 0.5 tanh(s), the delay τ (t) ≡ 1 and | aij (t)| ≤ Mij , |bij (t)| ≤ Cij , i, j = 1, 2, 3, t ≥ 0 where M = { Mij }, C = {Cij } are given by 

 −0.1 0.5 0.3   M = −0.2 0.3 0.2  0.4 −0.2 −0.1



0.1  C =  0.3 −0.2

 −0.1 −0.2  0.2 −0.1 0.5 0.3

(21)

Axioms 2018, 7, 30

13 of 18

The point x ∗ = (0, 0, 0) is an equilibrium point of Caputo FODNN (15) if Ii (t) ≡ 0, i = 1, 2, 3 (see Example 4). Then Li = Hi = 0.5 and ∑3i=1 (max j Mij L j + max j Cij Hj ) = (0.25 + 0.1) + (0.15 + 0.15) + (0.2 + 0.25) = 1.1, maxi (∑3j=1 ( Mij L j + Cij Hj ) = 0.85. Therefore, if ci > uniformly stable.

1.95 2

= 0.925, i = 1, 2, 3 then according to Theorem 1 the zero equilibrium is

4.3.2. Non-Lipschitz Activation Functions and Quadratic Lyapunov Functions There are many types of activation functions which are not Lipschitz (see Example 2, Cases 2 and 3). In this case we assume: Assumption 5. There exists a function ξ ∈ C (R+ , R) such that for any solution x (t) of the FODNN (18) and any point t > 0 : ∑nj=1 x2j (t) = sups∈[−r,0] ∑nj=1 x2j (s) and i = 1, 2, . . . , n the inequality xi ( t )



n

∑ ai,j (t)( f j (x j (t) + x∗j ) − f j (x∗j )) +

j =1

≤

n

∑ bi,j (t)( gj (x j (t − τ (t)) + x∗j ) − gj (x∗j ))

j =1

 (22)

ξ (t) xi2 (t)

holds. Assumption 6. There exists a function η ∈ C (R+ , (0, ∞)) such that ci (t) ≥ η (t), i = 1, 2, . . . , n, t ≥ 0.

(23)

Theorem 2. Let Assumptions 1, 5 and 6 be satisfied with ξ (t) ≤ η (t), t ≥ 0. Then the equilibrium point x ∗ of Caputo FODNN (15) is uniformly stable. Proof. Consider the quadratic functions V (t, x ) = x T x . Let x (t) be a solution of the FODNN (18). Let t > 0 be such that supΘ∈[−r,t] V (Θ, x (Θ)) = supΘ∈[−r,t] ∑nj=1 x2j (Θ) = ∑nj=1 x2j (t) = V (t, x (t)). Then applying Lemma 2, we get for the Caputo fractional derivative C q 0 Dt V ( t, x ( t )) n 

n

n

j =1

j =1

≤ −2 ∑ ci (t) xi2 (t) − xi (t) ∑ ai,j (t) Fj ( x j (t)) − xi (t) ∑ bi,j (t) Gj ( x j (t − τ (t))) i =1

 (24)

  n ≤ −2 η (t) − ξ (t) ∑ xi2 (t). i =1

From inequality (24) and Lemma 3 the claim follows. Example 8. Let n = 1 and consider the scalar nonlinear FODNN C q 0 Dt x ( t )

= −cx (t) + a(t) f ( x (t)) + b(t) f ( x (t − r )) + I for t > 0,

(25)

where q ∈ (0, 1), r = const > 0, I = 0.5c, c > 0, the functions a ∈ C ([0, ∞), (∞, 0]), b ∈ C ([0, ∞), [0, ∞)) : a(t) + b(t) ≤ µ(t) ≤ 0 and the activation function f (t) is the Probit function or the Logit function (see Example 6). Then the Equation (25) has an equilibrium point x ∗ = 0.5 (see Example 5). Neither of the activation functions are Lipschitz and Theorem 1 cannot be applied. Let t > 0 be such that supΘ∈[−r,t] | x (Θ)| = | x (t)|. Consider the following possible cases:

Axioms 2018, 7, 30

-

14 of 18

Let x (t) < 0 and x (t − r ) < −0.5 < 0. Since r > 0 we obtain x (t − r ) ≥ x (t) and from the monotonicity property of the function f we get f ( x (t − r ) + 0.5) ≥ f ( x (t) + 0.5) and b(t) f ( x (t − r ) + 0.5) x (t) ≤ b(t) f ( x (t) + 0.5) x (t). Then applying x f ( x + 0.5) ≥ x2 ≥ 0, x ∈ R we get a(t) f ( xu(t) + 0.5) x (t) + b(t) f ( x (t − r ) + 0.5) x (t)   ≤ a(t) + b(t) f ( x (t) + 0.5) x (t) ≤ µ(t) x2 (t).

-

Let x (t) < 0 and x (t − r ) > −0.5. Then f ( x (t − r ) + 0.5) ≥ 0 and applying x f ( x + 0.5) ≥ x2 ≥ 0, x ∈ R we obtain a(t) f ( x (t) + 0.5) x (t) + b(t) f ( x (t − r ) + 0.5) x (t)

≤ a(t) f ( x (t) + 0.5) x (t) ≤ a(t) x2 (t) ≤ µ(t) x2 (t). -

Let x (t) > 0. Then | x (t − r )| ≤ x (t) and from monotonicity property of the function f it follows f ( x (t − r ) + 0.5) ≤ f (| x (t − r )| + 0.5) ≤ f ( x (t) + 0.5). Therefore, a(t) f ( x (t) + 0.5) x (t) + b(t) f ( x (t − r ) + 0.5) x (t)   ≤ a(t) + b(t) f ( x (t) + 0.5) x (t) ≤ µ(t) x2 (t).

Therefore, the Assumption 5 is satisfied. Then, according to Theorem 2 the equilibrium point x ∗ = 0.5 of FODNN (25) is uniformly stable for all c > 0. 4.3.3. Non-Lipschitz Activation Functions and Time-Depended Lyapunov Functions In the case the function η (t) in Assumption 6 not being large enough, we introduce Assumption 7. There exists a continuous positive function p(t) ∈ C ([0, ∞), R+ ) such that 0 < α ≤ p(t) ≤ β q and the fractional derivative 0RL Dt p(t) exists for t > 0. In this case, Assumption 5 could be weakened to n

Assumption 8. There exists a function ξ ∈ C (R+ , R) such that for any function φ ∈ C ([−r, 0], R ) and t > 0 be such that p(t) ∑in=1 φi2 (0) = sups∈[−r,0] p(t + s) ∑in=1 φi2 (s) the inequality n

n

∑ φi (0) ∑

i =1



aij (t)( f j (φj (0) + x ∗j ) − f j ( x ∗j )) + bij (t)(( g j (φj (−τ (0) + x ∗j ) − g j ( x ∗j ))



j =1

n

≤ ξ (t) ∑

(26)

φi2 (0)

i =1

holds. Theorem 3. Let the Assumptions 1, 6–8 be satisfied and q

− p(t)η (t) + p(t)ξ (t) + 0.5 0RL Dt p(t) ≤ 0, t > 0. Then the equilibrium point x ∗ of Caputo FODNN (15) is uniformly stable. Proof. In this case the quadratic function V (t, x ) = ∑nj=1 xi2 , x = ( x1 , x2 , . . . , xn ) does not work.

(27)

Axioms 2018, 7, 30

15 of 18

Consider the function V (t, x ) = p(t) ∑in=1 xi2 , x = ( x1 , x2 , . . . , xn ) where the function p(t) is defined in Assumption 7. Then, according to Assumption 7 condition (i) of Lemma 4 is satisfied with n α1 (u) = αu and α2 (u) = βu. Let φ ∈ C ([−r, 0], R ) and t > 0 be such that n

n

V (t, φ(0)) = p(t) ∑ φi2 (0) = sup p(t + s) ∑ φi2 (s) = sup V (t + s, φ(s)). s∈[−r,0]

i =1

s∈[−r,0]

i =1

Then from Formula (4) and Example 6 we obtain for the Dini fractional derivative:

D(+18) V (t, φ(0); 0)   n n n  ≤ 2p(t) − ∑ ci (t)φi2 (0) + ∑ φi (0) ∑ aij (t) Fj (φj (0)) + bij (t) Gj (φj (−τ (0)) i =1

n

+

∑ φi2 (0)

i =1


RL 0

j =1

(28)

D q p(t)

i =1

≤



− 2p(t)η (t) + 2p(t)ξ (t) +

RL q 0 Dt p ( t )



n

∑ φi2 (0) ≤ 0.

i =1

According to Remark 3 and inequality (28) we get the inequality q c (18) D+ V ( t, φ (0); 0, ψ0 )

< D(+18) V (t, φ(0); 0) ≤ 0,

i.e., condition (ii) of Lemma 4 is satisfied. Applying Lemma 4 we prove the claim. Example 9. Let n = 1 and consider the scalar FODNN (17) with c(t) = I (t) ≡ 0, a, b ∈ C (R+ , R+ ) : a(t) + b(t) =

0.45Eq (−tq ) , the Eq (−tq )+0.1 eu −e−u , τ (t) ≡ r eu +e−u

0.55 tq Γ(1−q)( Eq (−tq )+0.1)

+

0.005 , Eq (−tq )+0.1

activation functions are the Continuous

> 0 and the equilibrium point x ∗ = 0. Tan-Sigmoid Function f (u) = g(u) = tanh(u) = Theorem 1 is not applicable since the coefficient before x is not a constant. Let x (t) be a solution of FODNN (17). Then using the inequality | f ( x )| ≤ | x |, x ∈ R we get x (t) a(t)( f ( x (t)) − f (0)) ≤ a(t) x2 (t). In addition, let t > 0 be such that ( x (t))2 = sups∈[−r,0] ( x (s))2 then | f ( x (t − r ))| ≤ | x (t − r )| ≤ | x (t)| and x (t)b(t)( f ( x (t − r )) − f (0)) ≤ b(t)| x (t)( f ( x (t − r ))| ≤ b(t)( x (t))2 i.e., Assumption 5 is satisfied with ξ (t) ≡ a(t) + b(t). However the inequality ξ (t) = a(t) + b(t) = 0.45Eq (−tq ) Eq (−tq )+0.1

≤ η (t) = c(t) = tq Γ(1−q)(0.55 + Eq (−0.005 is not satisfied (see Figure 4 for q = 0.8). Eq (−tq )+0.1) tq )+0.1 Therefore, Theorem 2 cannot be applied. Consider the function p(t) = ( Eq (−tq ) + 0.1). Then we get the inequality q

− p(t)η (t) + p(t)ξ (t) + 0.5 0RL Dt p(t)   0.005 0.55 + = −( Eq (−tq ) + 0.1) q t Γ(1 − q)( Eq (−tq ) + 0.1) Eq (−tq ) + 0.1 q   0.45Eq (−t ) RL q q + ( Eq (−tq ) + 0.1) + 0.5 D E (− t ) + 0.1 q 0 t Eq (−tq ) + 0.1   0.55 1.1 =− q − 0.005 + 0.45Eq (−tq ) + 0.5 − Eq (−tq ) + q t Γ (1 − q ) t Γ (1 − q ) 0.55 0.55 =− q − 0.005 + 0.45Eq (−tq ) − 0.5Eq (−tq ) + q t Γ (1 − q ) t Γ (1 − q )   q q = −0.005 − 0.05Eq (−t ) = −0.05 Eq (−t ) + 0.1 ≤ −0.005.

(29)

Axioms 2018, 7, 30

16 of 18

According to Theorem 3 the equilibrium point x ∗ = 0 of Caputo FODNN (17) is uniformly stable. x

0.40

0.35

xHtL

0.30

hHtL 0.25

0.20

0.15 t 0

2

4

6

8

10

Figure 4. Example 9. Graph of the functions ξ (t) and η (t) for q = 0.8.

Therefore, in the case that the activation functions are not Lipschitz and the coefficients ci are not constants in FODNN, we can use Lyapunov function depending directly on the time variable; its Caputo fractional Dini derivative is applicable to study the stability. 5. Conclusions Several sufficient conditions are given for stability of the equilibrium point of Caputo fractional order neural networks with delays of order 0 < q < 1. We study the general case when -

the transmission delays are time variables; the self-regulating parameters of all units are variable in time; the functions of the connection between two neurons in the network are variable in time bounded functions; the activation function is nonLipschitz (usually in the literature only the case of Lipschitz activation functions is studied).

The investigation is based on applying the fractional Lyapunov approach and various types of Lyapunov functions are discussed (not only the quadratic one as is usually considered in the literature). The presence of the fractional derivative in the model requires an appropriate defined and used derivative of Lyapunov functions among the fractional model. A brief overview of the existing literature and the main types of derivatives among Caputo fractional equations are presented. These types of derivatives are used as an apparatus to study stability properties of the considered Caputo fractional order neural networks for different types of activation functions. Many examples with different types of activation functions such as tanh function, Logit function, continuous Tan-Sigmoid function, are provided to illustrate the practical application of the obtained sufficient conditions. Author Contributions: The authors have made the same contribution. All authors read and approved the final manuscript. Acknowledgments: Research was partially supported by Fund MU17-FMI-007, University of Plovdiv “Paisii Hilendarski”. Conflicts of Interest: The authors declare no conflict of interest.

References 1.

Marcus, C.; Westervelt, R. Stability of analog neural networks with delay. Phys. Rev. A 1989, 39, 347–359. [CrossRef]

Axioms 2018, 7, 30

2.

3. 4. 5. 6. 7.

8. 9. 10. 11. 12. 13. 14.

15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25.

17 of 18

Chen, D.; Zhang, R.; Liu, X.; Ma, X. Fractional order Lyapunov stability theorem and its applications in synchronization of complex dynamical networks. Commun. Nonlinear Sci. Numer. Simulat. 2014, 19, 4105–4121. [CrossRef] Kaslik, E.; Sivasundaram, S. Nonlinear Dynamics and Chaos in Fractional Order Neural Networks. Neural Netw. 2012, 32, 245–256. [CrossRef] [PubMed] Newman, M.E.J. Communities, modules and large-scale structure in networks. Nat. Phys. 2012, 8, 25–31. [CrossRef] Stigler, J.; Ziegler, F.; Gieseke, A.; Gebhardt, J.C.M.; Rief, M. The complex folding network of single calmodulin molecules. Science 2011, 334, 512–516. [CrossRef] [PubMed] Wanduku, D.; Ladde, G.S. Global properties of a two-scale network stochastic delayed human epidemic dynamic model. Nonlinear Anal.-Real World Appl. 2012, 13, 794–816. [CrossRef] Huang, Y.; Zhang, H.; Wang, Z. Dynamical stability analysis of multiple equilibrium points in time-varying delayed recurrent neural networks with discontinuous activation functions. Neurocomputing 2012, 91, 21–28. [CrossRef] Li, C.; Feng, G. Delay-interval-dependent stability of recurrent neural networks with time-varying delay. Neurocomputing 2009, 72, 1179–1183. [CrossRef] Wu, H.; Wang, L.; Wang, Y.; Niu, P.; Fang, B. Global Mittag-Leffler projective synchronization for fractional-order neural networks: An LMI-based approach. Adv. Differ. Eq. 2016, 2016, 132. [CrossRef] Li, Y.; Chen, Y.; Podlubny, I. Stability of fractional-order nonlinear dynamic systems: Lyapunov direct method and generalized Mittag-Leffler stability. Comput. Math. Appl. 2010, 59, 1810–1821. [CrossRef] Li, Y.; Chen, Y.; Podlubny, I. Mittag-Leffler stability of fractional order nonlinear dynamic systems. Automatica 2009, 45, 1965–1969. [CrossRef] Li, G.; Liu, H. Stability Analysis and Synchronization for a Class of Fractional-Order Neural Networks. Entropy 2006, 18. [CrossRef] Liu, H.; Li, S.; Wang, H.; Huo, Y.; Luo, J. Adaptive Synchronization for a Class of Uncertain Fractional-Order Neural Networks. Entropy 2015, 17, 7185–7200. [CrossRef] Zhang, W.; Wu, R.; Cao, J.; Alsaedi, A.; Hayat, T. Synchronization of a class of fractional-order neural networks with multiple time delays by comparison principles. Nonlinear Anal. Modell. Control 2017, 22, 636–645. [CrossRef] Chen, L.; Chai, Y.; Wu, R.; Ma, T.; Zhai, H. Dynamic analysis of a class of fractional-order neural networks with delay. Neurocomputing 2013, 111, 190–194. [CrossRef] Chen, B.; Chen, J. Razumikhin-type stability theorems for functional fractional-order differential systems and applications. Appl. Math. Comput. 2015, 254, 63–69. [CrossRef] Wang, H.; Yu, Y.; Wen, G. Stability analysis of fractional-order Hopfield neural networks with time delays. Neural Netw. 2014, 55, 98–109. [CrossRef] [PubMed] Zhang, W.; Cao, J.; Chen, D.; Alsaadi, F.E. Synchronization in fractional-order complex-valued delayed neural networks. Entropy 2018, 20, 16p. [CrossRef] Bao, H.; Park, J.H.; Cao, J. Synchronization of fractional-order complex-valued neural networks with time delay. Neural Netw. 2016, 81, 16–28. [CrossRef] [PubMed] Wu, R.-C.; Hei, X.-D.; Chen, L.-P. Finite-time stability of fractional-order neural networks with delay. Commun. Theor. Phys. 2013, 60, 189–193. [CrossRef] Yang, X.; Song, Q.; Liu, Y.; Zhao, Z. Finite-time stability analysis of fractional-order neural networks with delay. Neurocomputing 2015, 152, 19–26. [CrossRef] Alofi, A.; Cao, J.; Elaiw, A.; Al-Mazrooei, A. Delay-Dependent Stability Criterion of Caputo Fractional Neural Networks with Distributed Delay. Discr. Dynam. Nat. Soc. 2014, 2014, 529358. [CrossRef] Podlubny, I. Fractional Differential Equations; Academic Press: San Diego, CA, USA, 1999. Stamova, I.; Stamov, G. Functional and Impulsive Differential Equations of Fractional Order: Qualitative Analysis and Applications; CRC Press: Boca Raton, FL, USA, 2016. Devi, J.V.; Rae, F.A.M.; Drici, Z. Variational Lyapunov method for fractional differential equations. Comput. Math. Appl. 2012, 64, 2982–2989. [CrossRef]

Axioms 2018, 7, 30

26. 27. 28. 29.

30. 31. 32. 33. 34. 35.

18 of 18

Agarwal, R.; Hristova, S.; O’Regan, D. A survey of Lyapunov functions, stability and impulsive Caputo fractional differential equations. Fract. Calc. Appl. Anal. 2016, 19, 290–318. [CrossRef] Agarwal, R.; Hristova, S.; O’Regan, D. Lyapunov functions and strict stability of Caputo fractional differential equations. Adv. Differ. Eq. 2015, 2015, 346. [CrossRef] Agarwal, R.; O’Regan, D.; Hristova, S.; Cicek, M. Practical stability with respect to initial time difference for Caputo fractional differential equations. Commun. Nonlinear Sci. Numer. Simulat. 2017, 42, 106–120. [CrossRef] Duarte-Mermoud, M.A.; Aguila-Camacho, N.; Gallegos, J.A.; Castro-Linares, R. Using general quadratic Lyapunov functions to prove Lyapunov uniform stability for fractional order systems. Commun. Nonlinear Sci. Numer. Simulat. 2015, 22, 650–659. [CrossRef] Stamova, I. On the Lyapunov theory for functional differential equations of fractional order. Proc. Am. Math. Soc. 2016, 144, 1581–1593. [CrossRef] Stamova, I.; Simeonov, S. Delayed Reaction-Diffusion Cellular Neural Networks of Fractional Order: Mittag-Leffler Stability and Synchronization. J. Comput. Nonlinear Dyn. 2017, 13, 011015. [CrossRef] Gelfand, I.M. General theory of hypergeometric functions. Doklady Akademii Nauk SSSR 1986, 288, 14–18. (English translation in collected papers, volume III) Rahimi, A.; Recht, B. Weighted sums of random kitchen sinks: Replacing minimization with randomization in learning. Adv. Neural Inf. Process. Syst. 2008, 21, 1313–1320. Agarwal, R.; Hristova, S.; O’Regan, D. Lyapunov functions and stability of Caputo fractional differential equations with delays. Unpublished work, 2018. Li, R.; Cao, J.; Alsaedi, A.; Alsaadi, F. Stability analysis of fractional-order delayed neural networks. Nonlinear Anal. Model. Control 2017, 22, 505–520. [CrossRef] c 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access

article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).