Keywords: Cellular Neural Networks; complete stability; robustness; complex dynamics. 1. Introduction. Under the hypothesis that the neuron interconnec-.
Letters International Journal of Bifurcation and Chaos, Vol. 12, No. 6 (2002) 1357–1362 c World Scientific Publishing Company
COMPLEX DYNAMICS IN NEARLY SYMMETRIC THREE-CELL CELLULAR NEURAL NETWORKS M. DI MARCO and M. FORTI Dipartimento di Ingegneria dell’Informazione, Universit` a di Siena, v. Roma 56 – 53100 Siena, Italy A. TESI Dipartimento di Sistemi e Informatica, Universit` a di Firenze, v. S. Marta 3 – 50139 Firenze, Italy Received June 7, 2001; Revised June 29, 2001 The paper introduces a class of third-order nonsymmetric Cellular Neural Networks (CNNs), and shows through computer simulations that they undergo a cascade of period doubling bifurcations which leads to the birth of a large-size complex attractor. A major point is that these bifurcations and complex dynamics happen in a small neighborhood of a particular CNN with a symmetric interconnection matrix. Keywords: Cellular Neural Networks; complete stability; robustness; complex dynamics.
1. Introduction Under the hypothesis that the neuron interconnection matrix is symmetric, the standard Cellular Neural Networks (CNNs) [Chua & Yang, 1988] are Completely Stable (CS), i.e. each trajectory converges towards some equilibrium point. In a recent paper [Di Marco et al., 2000a], the question of robustness of Complete Stability (CS) with respect to small perturbations of the nominal symmetric interconnection matrix of a standard CNN has been addressed. This issue is of fundamental importance, among other reasons, since it is not possible to realize perfectly symmetric neuron interconnections in the electronic (e.g. VLSI) neural network implementation [Vidyasagar, 1993; Wang & Michel, 1994]. The main conclusion reached in [Di Marco et al., 2000a] is that in the general case CS is not robust. More specifically, the analysis of a three-cell competitive standard CNN has shown the existence of stable limit cycles which attract almost all trajectories, even for parameter sets
arbitrarily close to some nominal symmetric interconnection matrix. The cycles may be generated by local Hopf-like bifurcations at the equilibrium point, or by global heteroclinic bifurcations. A peculiar feature is that in the ideal case of piecewise-linear neuron activations, the size of the limit cycles is large and does not tend to zero as the perturbation vanishes. This paper further investigates the above fundamental problem, by studying the dynamics of a class of three-cell standard CNNs with nonsymmetric mixed cooperative–competitive interconnections, which are described by a one-parameter family of differential equations. It is shown through computer simulations that the CNNs undergo a cascade of period doubling bifurcations which leads to the birth of a large-size complex attractor. The important fact is that these bifurcations and complex dynamics are located very close to a particular CNN with a symmetric neuron interconnection matrix.
1357
1358 M. Di Marco et al.
2. Complex Dynamics in Three-Cell CNNs Let us consider the three-cell standard CNN
0
x3
1 0 0 g(x1 ) x˙ 1 x1 x˙ 2 = − x2 + 0 1 0 g(x2 ) x˙ 3 x3 g(x3 ) 0 0 1
−0.810 0.720 0.533 g(x1 ) + ε −1.203 µ 5.333 g(x2 ) g(x3 ) 2.732 −5.333 µ
−0.5
1.5 −1
1 0.5 0
1.5
1
3.7874 + ε 3.7088 −0.5496
−0.5
0.5
0
−0.5
−1
−1
x1
x2
ε = 10−3 ; µ = 0.12
(1) Fig. 1.
Two limit cycles for the three-cell CNN (1).
where xi are the neuron state variables, g(xi ) are the neuron outputs, and ε, µ are real parameters. Function g is the piecewise-linear neuron activation −1
g(xi ) =
xi 1
xi ≤ −1 −1 < xi < 1 . xi ≥ 1
0.2
(2)
0
−0.2
1 0 0 −0.810 0.720 0.533 1 0 + ε −1.203 µ 5.333 . 0 0 1 2.732 −5.333 µ
0
We have found it convenient to split the matrix in two parts in view of the subsequent analysis. Note that the interconnection matrix is symmetric if ε = 0, while it is nonsymmetric for any ε 6= 0. The neural network has both inhibitory and excitatory neuron interconnections, i.e. it is a mixed cooperative–competitive CNN. The constant vector in (1) represents the total input bias. In this section, we suppose that ε is constant, and consider µ as a bifurcation parameter. In the next section, we study what happens when ε tends to 0, for fixed values of µ. We have set ε = 10−3 and analyzed the dynamics of (1) as a function of parameter µ, by means of computer simulations.1 For µ = 0.12 the CNN displays two stable limit cycles, as shown in Fig. 1.
x3
−0.4
The neuron interconnection matrix is given by
−0.6
−0.8 1.5 −1
1 0.5
1.5
0 1
−0.5
0.5
0
−0.5
−1
−1
x1
x2
ε = 10−3 ; µ = 0.3 Fig. 2.
Cycle of period 2.
In what follows we consider the bifurcations of the larger limit cycle, which happen as µ increases from 0.12 to 0.85. These are reported in Figs. 2–7. It is seen that the limit cycle undergoes a cascade of period doubling bifurcations which leads to the birth of a complicated attractor (µ = 0.68). For larger values of µ, periodic windows that separate regions with complex dynamics are also observed. Two of them, which correspond to cycles of periods 3 and 5, are reported in Figs. 6 and 7, respectively.
The results presented are obtained using MATLAB routine ODE23S, with absolute tolerance 10−8 and relative tolerance 10−6 .
1
Complex Dynamics in Nearly Symmetric Three-Cell CNNs 1359
0
0
−0.2
−0.2
−0.4
−0.4
x3
0.2
x3
0.2
−0.6
−0.6
−0.8
−0.8 1.5
−1
1.5 −1
1
1
0.5 1.5
0.5
0 1
1.5
−0.5
0.5
0
−0.5
−1
−1
0 1
−0.5
0.5
0
x1
x2
−1
−1
x1
x2
ε = 10−3 ; µ = 0.68
ε = 10−3 ; µ = 0.45 Fig. 3.
−0.5
Fig. 5.
Cycle of period 4.
0.2
0
0
−0.2
−0.2
−0.4
−0.4
x3
x3
0.2
Complex attractor.
−0.6
−0.6
−0.8
−0.8 1.5 −1
1
1.5 −1
1 0.5
0.5 1.5
0 1
1.5
−0.5
0.5
0
−0.5
−1
−1
0 1
−0.5
0.5
0
ε = 10−3 ; µ = 0.46 Cycle of period 8.
It is of interest to note that the cycles and complex attractors have a size comparable with the saturation level of the neuron nonlinearities. Moreover, they develop almost completely within the CNN linear region {x ∈ R3 : |x1 |, |x2 |, |x3 | ≤ 1}, i.e. they only slightly enter partial saturation regions of (1) with one saturated variable.2 The shape of the complex attractors has similarities with the R¨ossler-type attractors already observed in Chua’s circuit [Chua et al., 1986]. One basic difference,
−1
−1
x1
x2
x2
Fig. 4.
−0.5
x1
ε = 10−3 ; µ = 0.8 Fig. 6.
Cycle of period 3.
however, is that Chua’s circuit is characterized by only one nonlinearity, while the attractor of Fig. 5 involves the saturation of the three nonlinear functions g(xi ) in (1). Finally, we note that the CNN (1) has been designed through an extensive random search in the parameter space. Several other complex attractors, with a shape different from that of Fig. 5, have been observed for CNNs with interconnection matrices close to symmetry.
Let us consider the hypercube {x ∈ R3 : |x1 |, |x2 |, |x3 | ≤ 1 + 20ε}. It can easily be verified that for µ ≤ 1 the vector field defining (1) points towards the interior of H, on the boundary of H. This implies that the attractor is necessarily confined within H. Note that for ε ≤ 10−3 , H is only slightly larger than the CNN linear region.
2
1360 M. Di Marco et al.
0
0
−0.2
−0.2
−0.4
−0.4
x3
0.2
x3
0.2
−0.6
−0.6
−0.8
−0.8 1.5
−1
1.5 −1
1
1
0.5
0.5
0
1.5
1
1.5
−0.5
0.5
0
−0.5
−1
−1
0 1
0.5
x2
−0.5
−1
−1
x1
x2
ε = 0.3 · 10−3 ; µ = 0.3
ε = 10−3 ; µ = 0.85 Fig. 7.
−0.5 0
x1
Fig. 8. Cycle of period 2 for the three-cell CNN (1) obtained for ε = 0.3 · 10−3 and µ = 0.3 (cf. Fig. 2).
Cycle of period 5.
3. Discussion The results obtained in the previous section have the following significant interpretation. Let us consider the nominal third-order CNN
0
1 0 0 g(x1 ) x˙ 1 x1 x ˙ = − x + 0 1 0 g(x 2 2 2 ) , (3) x˙ 3 x3 g(x3 ) 0 0 1
−0.2
−0.4
x3
0.2
−0.6
−0.8
which is obtained from (1) when ε = 0. The CNN (3) has a symmetric interconnection matrix which coincides with the 3 × 3 identity matrix, while the total input vector is 0. Clearly, we can see (1) as a perturbation of the nominal symmetric CNN (3). Indeed, the nonsymmetric perturbations ∆A of the interconnection matrix, and ∆I of the total input vector, are given by
−0.810 0.720 5.333 µ 5.333 ; ∆A = ε −1.203 2.732 −5.333 µ
(4)
3.7874 ∆I = ε 3.7088 . −0.5496 Note that for ε = 10−3 , and all values of µ relative to the dynamical scenario presented in Sec. 2, we have maxi,j=1,2,3 |∆Aij | = 5.333 × 10−3 and 3
1.5 −1
1 0.5
1.5
0 1
0.5
−0.5 0
−0.5
−1
−1
x1
x2
ε = 10−7 ; µ = 0.3 Fig. 9.
Cycle of period 2 obtained for ε = 10−7 and µ = 0.3.
maxi=1,2,3 |∆Ii | = 3.7874×10−3 . Thus, we can conclude that all the complex dynamics displayed by (1) in Figs. 1–7 turn out to be located close to the nominal symmetric CNN (3).3 This conclusion is further strengthened by the next simulations, which show that the dynamical scenario in Sec. 2 can be pushed even more close to the nominal symmetric CNN (3). To this end, let us repeat the previous analysis for smaller values of ε. In Figs. 8 and 9 we have reported the simulations obtained for µ = 0.3 and ε = 0.3 · 10−3 ,
It is worth to recall that complex attractors have already been observed for CNNs. However, they concern cases where the interconnection matrix is far from the symmetry condition [Zou & Nossek, 1993; Zou et al., 1993].
Complex Dynamics in Nearly Symmetric Three-Cell CNNs 1361
ε = 10−7 , respectively. It is seen that the cycle of period 2 in Fig. 2, which corresponded to the same µ and ε = 10−3 , is found almost unchanged even when more close to symmetry. Note, for example, that when ε = 10−7 the maximum deviations ∆Aij and ∆Ii are on the order of 10−6 . A similar result is obtained for the other period doubling bifurcations and the complex attractors presented in the previous section. As an example, Figs. 10 and 11 display the complex attractors of (1) obtained for µ = 0.68 and ε = 0.3 · 10−3 , ε = 10−7 . By comparing Fig. 5, which corresponded to ε = 10−3 and the
x3
0
same value of µ, it turns out that the shape of the attractor of Fig. 5 is actually preserved even when approaching closer to the nominal symmetric CNN (3). We remark that as ε decreases, the trajectories run across the attractors more slowly, but the shape of the attractors remains almost unchanged. In particular, the size of the cycles and the complex attractors does not vanish as ε tends to 0. The results previously obtained are of obvious theoretical interest, since it was quite unexpected a priori to find such dynamical complexities so close to symmetric CNNs, which are characterized by a simple convergent behavior of the trajectories [Chua & Yang, 1988; Forti & Tesi, 2001]. Moreover, these results are relevant also in view of the CNN realization. In fact, due to tolerances or parasitic couplings, in any electronic implementation it is impossible to obtain interconnections which are perfectly symmetric [Vidyasagar, 1993; Wang & Michel, 1994]. This fundamental point is further discussed in the following remarks.
−0.5
Remarks 1.5 −1
1 0.5
1.5
0 1
0.5
−0.5 0
−0.5
−1
−1
x1
x2
ε = 0.3 · 10−3 ; µ = 0.68 Fig. 10. Complex attractor for the three-cell CNN (1) obtained for ε = 0.3 · 10−3 and µ = 0.68 (cf. Fig. 5).
x3
0
−0.5
1.5 −1
1 0.5
1.5
0 1
0.5
−0.5 0
−0.5
−1
−1
x1
x2
ε = 10−7 ; µ = 0.68 Fig. 11. Complex attractor obtained for ε = 10−7 and µ = 0.68.
(1) In the perturbed third-order CNN (1), there are nonzero interconnections also for neurons that are not interconnected in the nominal model (3). In fact ∆A in (4) is a full matrix, while the nominal symmetric interconnection matrix A coincides with the identity matrix. Actually, this can be justified by noting that it is realistic to assume that in the electronic implementation there is the birth of small parasitic couplings between any pair of neurons [Seiler et al., 1993]. (2) It would be also of interest to analyze for general CNNs the ideal limiting case where the errors in the implementation of the interconnections are due to tolerances, but not to parasitic couplings. This means that ∆A has the same structure of A, i.e. ∆Aij = 0 for Aij = 0. Under this assumption, it can be expected that the sparsity of A, which is due to the local interconnecting structure postulated by the CNN paradigm, will permit sometimes to rule out the presence of complex dynamics near the nominal symmetric case. Some partial results confirming this conjecture are already available, at least in relation to Hopf-bifurcations close to symmetry [Di Marco et al., 2000b], see also the fundamental studies [Thiran, 1997; Setti et al., 1998]. This topic, however, deserves further investigation.
1362 M. Di Marco et al.
4. Conclusion This study has revealed the presence of unexpectedly rich dynamics for small perturbations of a three-cell standard CNN with a nominal symmetric interconnection matrix. Beyond the theoretical interest, this conclusion is also relevant in view of the practical electronic realization of CNNs, since it is not possible to implement exactly symmetric neuron interconnections, due to tolerances or parasitic couplings.
References Chua, L. O., Komuro, M. & Matsumoto, T. [1986] “The double scroll family,” IEEE Trans. Circuits Syst. 33, 1073–1018. Chua, L. O. & Yang, L. [1988] “Cellular neural networks: Theory,” IEEE Trans. Circuits Syst. 35, 1257–1272. Di Marco, M., Forti, M. & Tesi, A. [2000a] “Bifurcations and oscillatory behavior in a class of competitive cellular neural networks,” Int. J. Bifurcation and Chaos 10, 1267–1293. Di Marco, M., Forti, M. & Tesi, A. [2000b] “On robustness of complete stability for a class of cellular neural networks,” Proc. ISCAS 2000, IEEE Int. Symp. Circuits and Systems, Geneve, CH. pp. IV–141–144.
Forti, M. & Tesi, A. [2001] “A new method to analyze complete stability of PWL cellular neural networks,” Int. J. Bifurcation and Chaos 11, 655–676. Seiler, G., Schuler, A. J. & Nossek, J. A. [1993] “Design of robust cellular neural networks,” IEEE Trans. Circuits Syst. I 40, 358–364. Setti, G., Thiran, P. & Serpico, C. [1998] “An approach to information propagation in 1-D cellular neural networks-Part II: Global propagation,” IEEE Trans. Circuits Syst. I 45, 790–811. Thiran, P. [1997] Dynamics and Self-Organization of Locally Coupled Neural Networks (Presses Polytechniques et Universitaires Romandes, Lausanne, Switzerland). Vidyasagar, M. [1993] “Location and stability of the high-gain equilibria of nonlinear neural networks,” IEEE Trans. Neural Networks 4, 660–672. Wang, K. & Michel, A. N. [1994] “Robustness and perturbation analysis of a class of nonlinear systems with applications to neural networks,” IEEE Trans. Circuits Syst. I 41, 24–32. Zou, F., Kat´erle, A. & Nossek, J. A. [1993] “Homoclinic and heteroclinic orbits of the three-cell cellular neural networks,” IEEE Trans. Circuits Syst. I 40, 843–848. Zou, F. & Nossek, J. A. [1993] “Bifurcation and chaos in cellular neural networks,” IEEE Trans. Circuits Syst. I 40, 166–173.
