Information Processing in 1-D Systems with Chaos - IEEE Xplore

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: FUNDAMENTAL THEORY AND APPLICATIONS, VOL. 44, NO. 1, JANUARY 1997

21

Information Processing in 1-D Systems with Chaos Yuri V. Andreyev, Alexander S. Dmitriev, and Sergey O. Starkov Abstract— Mathematical models are proposed, based on onedimensional piecewise-linear maps, in which complex dynamics, bifurcation phenomena and chaos are used for information processing.

I. INTRODUCTION

T

HE ROLE THAT DYNAMIC chaos plays in processing information by human and animal brains is extensively investigated in the last decades. The very existence of chaotic regimes in the brain is considered doubtless, and the efforts of the researchers are concentrated now on the study of those special functions of brain, for which chaos is either necessary, or has some advantages compared to simple dynamics. The functional role of cortical chaos appearing due to thalamo-cortical interaction is discussed in [1] and [2]. Chaos is considered a possible mechanism of self-referential logic, and as a machine for a short-term memory based on that logic. Freeman observed chaotic activity in the learning process in the rabbit’s olfactory system [3]–[7]. He has found, that the rabbit remembers a known smell by coding it in spatially coherent and temporally almost periodic activity of the olfactory potential. In the case when the animal feels a new smell the coding mechanism does not work and the activity of the olfactory bulb becomes a low-dimensional chaos, as if it were a filter of “novelty,” forming the state “I don’t know.” Based on the analysis of human electro-encephalograms a hypothesis was suggested [8]–[10], that the functional role of chaos is determined by the property of chaotic dynamics to increase the resonance capacity of the brain, giving a chance for extremely rich responses to an external stimulus. Among other hypotheses about the functional role of chaos we want to note the following ones: a nonlinear pattern classifier [5], [7], a catalyst of learning [3], a stimulus interpreter [12], a memory searcher [13], etc. A more thorough list of possible roles of cortical chaos in information processing can be found in [14], along with a rich reference base for the studies of the 1970’s until the 1990’s. Thus, experimental investigations of electric activity of the brain and its certain neural subsystems, simulation of various neural networks and qualitative analysis of information processes in the brain allowed to suggest and to prove, to some extent, several hypotheses about the role of chaos in the brain activity. The use of different approaches, models, and methods in the study of the functional role of chaos leads to an idea of Manuscript received June 2, 1995. This work was supported in part by the Russian Foundation for Fundamental Investigations Grant 93-012-730 and by INTAS Grant INTAS-94-2899. This paper was recommended by Associate Editor M. Ogorzalek. The authors are with the Academy of Sciences, Institute of Radio Engineering and Electronics, Moscow 103 907, Russia. Publisher Item Identifier S 1057-7122(97)00816-7.

the existence of general principles of information processing in chaotic systems, independent of the concrete nature and realization of the systems. This allows to hope to investigate main relations of information processing using simple models. Here the problem of the proper choice of the dynamical system arises, which must be convenient, i.e., be simple enough and allow thorough description, and at the same time exhibit complex and chaotic behavior. The approach that we follow in this paper implies that information processing in a dynamical system is associated with a notion of an attractor in the system phase space carrying information. Information processing, e.g., recognition, is associated with structural transformations of the attractors (bifurcations) and essential change in the system’s behavior. The first step of information processing, the storing, is coupled with the synthesis of a nonlinear dynamical system with the phase space of a special structure, i.e., with attractors corresponding to stored information. This approach is used, for example, in neural networks where for a given set of images a neural network is synthesized (trained) such that these images correspond to equilibrium states of this dynamical system. Here the most simple type of attractors, a stable point in the system phase space, is used as the carrier of information. Efforts are also known (e.g., [14]–[16]) of using more complicated attractors, such as cycles (periodic orbits) and strange attractors, for carrying information in neural networks. But the enormous complexity of cooperative motion of the neurons in conventional neural networks makes direct synthesis (calculation) of these networks very difficult or even practically impossible. Instead, to design such a dynamical system, one has to use time-consuming procedures of training, which obscures investigations of the general principles of information processing. We tried to derive some simple mathematical models allowing easy and thorough description and exhibiting complex and even chaotic behavior. Issuing from the above concept of the existence of general principles of information processing independent of the concrete dynamical system, we proposed to use a class of discrete-time one-dimensional systems, namely, piecewise-linear maps of a segment (an interval) into itself The efforts were concentrated on the synthesis of dynamical systems with prescribed cycles in the system phase space. As a result, a method of storing information using stable limit cycles of 1-D maps as information carriers was proposed [17], [18]. (The term “stable limit cycle,” that we use further, means a discrete finite-period cycle which is a limit for any phase trajectory starting from any point within a certain vicinity of the cycle. Similar considerations can be applied to the term “unstable limit cycle”.)

1057–7122/97$10.00  1997 IEEE

22

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: FUNDAMENTAL THEORY AND APPLICATIONS, VOL. 44, NO. 1, JANUARY 1997

The use of more complicated attractors, i.e., cycles rather than equilibrium points, offers new capabilities of information processing, for example, associative memory [17], [18]. Further investigations have shown that this method of storing information can be applied in practice to storing pictures, texts, signals, etc., [19]–[22]. The method was extended also to storing information in 2-D and multidimensional maps and to storing multidimensional information sequences (cycles of vectors) [23]. In this paper we propose a further development of the original method, using chaotic systems and unstable limit cycles for storing information, and discuss opportunities of information processing in such systems. In Section II we briefly describe the original method [17], [18] of storing information in 1-D maps as stable and unstable cycles and realization of associative memory. In Section III we demonstrate chaotic dynamics of 1-D map with information stored as unstable cycles, discuss the ways of retrieving information from the map, and apply the method of direct map function control to image recognition. In Section IV an adaptive model of a memory is introduced, in which the map becomes a device driven by an input information stream, and the whole system transforms the input stream into an output, either regular or chaotic, depending on the presence of stored information blocks in the input stream. “Long-term” and “short-term” memories are shown in the adaptive model. Finally, we summarize information processing functions realized in the proposed dynamical systems, and draw some conclusions on the role of chaos in the discussed models. II. STORING INFORMATION

IN

1-D MAPS

In [17] and [18] we proposed to store information as stable cycles of the maps of a segment into itself. Let us recall the main notions and terms that we use on an example of storing two information blocks, finite 1-D strings “ ” and “ ”. For simplicity, we use a subset of the Latin characters as the alphabet. The length of the alphabet Our aim is to design the function of a 1-D map such that in the phase space of this dynamical system stable cycles exist, and each information block of length stored in the map is unambiguously related to a -period cycle The symbols of the strings are encoded by the amplitude of the mapping variable We will store the words in a 1-D map using second-level storing, which means that each point of the cycles is determined by a pair of successive symbols. We divide the phase space of the dynamical system (the unit interval into subintervals of the first level (each with the length and relate them to the elements of the alphabet. Then we repeat the procedure and divide each of the subintervals of the first level into subintervals of the second level (with the length and also relate them to the alphabet elements, as shown in Fig. 1. Now we design two cycles unambiguously related to the stored information blocks. Three cycle points for the word are related with the block fragments (pairs) , , and (the information block is mentally closed

Fig. 1. Storing two information blocks “babe” and “add” in a 1-D map. Cycle points are designated with squares and diamonds, respectively. Storage level q 2; s = 0:5: (a) The map function. (b) Information carrying cycles.

=

in a loop). The cycle point corresponding to the fragment is the center of the second-level subinterval located within the first-level subinterval corresponding to the symbol Other cycle points are created similarly. The cycle points corresponding to block are determined by the pairs , , , Having created the cycles in 1-D phase space, we construct a dynamical system possessing the phase space of such a structure. In the plane we plot the pairs of successive points for all the cycles. These points form the ”skeleton” of the map function Through these points, we then draw short straightline segments (called information regions), all with the same fixed slope We will control the stability of the cycles by changing the slope of these segments (turning them around the central point which coincides with the cycle point lying at this segment). As is known, the stability of a cycle is determined by its eigenvalue . In the case of a 1-D map the is equal to eigenvalue for the cycle (1) If the cycle is stable, otherHere, wise it is unstable. To complete the synthesis of the piecewisewe connect the information regions linear map function and the unit interval endpoints in series with straightline

ANDREYEV et al.: INFORMATION PROCESSING

segments, which we will further call noninformative segments. The plot of the map with the information cycles is shown in Fig. 1. Iterates of the designed map produce the output information stream: an occurrence of the system variable in a first-level subinterval is treated as “generation” of the corresponding alphabet element. Mathematically, it is where is the order number of this element in the alphabet, and denotes the integer part of a number. Thus, the motion of the phase trajectory over a cycle in the system phase space is accompanied by continuous reproduction of the corresponding information block. Storing information as stable limit cycles allows easy associative access to the stored information. If an equilibrium point is used as an information carrier, all information or its most part is necessary to access the point and to retrieve information. If an image is stored as a stable cycle, as in our case, each point of the cycle is related to only a part of the image, and only a piece of the original information is necessary to get a point near the cycle and to retrieve the whole image by iterating the dynamical system. Thus, associative access to the stored information becomes possible, yet by expense of the iteration time. Indeed, if we take an excerpt of an information block with the length equal to or greater than the storage level then we can apply the same procedure as in the creating the cycle points and get a point lying at the corresponding cycle: we take the first-level subinterval related to then the second-level subinterval within it corresponding to and so on times. At last we get a point within the th level subinterval, and because of the map design procedure we occur at an information region and this point is a point of the cycle related to this information block. Nothing else is necessary now but to iterate the map until we return to the same point, and the whole stored image is recovered then. Actually, we can begin from any point within this information region, the output information stream will be the same. Note that this is direct access to information, because the offered excerpt is not compared to all the images, instead, an initial point lying at the required cycle is directly calculated, so the access to information is very fast. A development of the method is discussed in [19] and [20], where it is shown that the amount of information that can be stored and retrieved in such systems may be essentially increased by encoding of initial information blocks based on compression (elimination of redundancy). In the case of software realization the designed dynamical systems allow storage of several megabytes of information (texts, signals, pictures) in ordinary personal computers while retaining the property of very fast associative access [21], [22]. III. UNSTABLE CYCLES AND RETRIEVAL OF INFORMATION FROM THE MAP Storing information as periodic motions in 1-D maps is an easy and efficient method of associative memory organization. However, an analysis of the papers devoted to investigation of information processing in natural brains indicates of essentially complicated behavior in natural neural networks, (e.g., see the review in [14]). In particular, periodic motion often indicates

23

of some “degenerate” states in the brain, the “usual” state being chaotic [3], [7], [9]. Besides, the existence of stable limit cycles in the phase space of a 1-D map leads to competition of the cycles, which means that iterates from arbitrary initial points can result in convergence to any system cycle regardless of the images presented for the recognition, because the attraction basins of the cycles have fractal structure [17], [21]. Moreover, “parasitic” stable cycles not related to the stored images can appear sometimes in such systems. Of course, all this is a certain drawback of the model with stable cycles discussed above, and we propose to overcome it by storing information as unstable cycles. The proposed systems offer new capabilities of information processing. In the above described procedure of storing information, the cycles may be easily made unstable. All that is necessary is to change the slope of the information regions in the piecewisethen the eigenvalues for all the linear map function: if and the cycles become unstable. cycles are No stable information cycles exist in the map then. If there is a “parasitic” stable cycle, this means that it passes through a noninformative segment with the slope less that 1 in magnitude, which makes its eigenvalue also less than one. Such cycles may be excluded by a local correction of the map function, while the structure of the unstable cycles with the stored information will remain undisturbed. Because of the absence of stable limit cycles in the map, the phase trajectory wanders chaotically with mapping iterates over the phase space visiting its most part. A number of methods can be used to retrieve information from the map, here we suggest direct control of the map function (slope switching) to make the desired cycle stable while retaining others unstable, and apply this method to image recognition [19]. According to the map design procedure, the map phase space contains a skeleton of unstable periodic orbits coupled with the stored information blocks and passing through information regions of the map function. It also contains some unstable periodic orbits coupled with noninformative segments of the map; they will be ignored in the further discussion. We want to derive a regular procedure of the map function deformation, such that if a presented image coincides with one of the stored images (or is close enough to it), the corresponding cycle becomes stable and attracting. The phase trajectory converges to this stable periodic orbit then, and the stored image is reproduced by the system, which can be treated as recognition. Otherwise, no stable cycles appear, and the motion in the dynamical system remains chaotic. Thus, the character of the motion in the modified dynamical system, regular or chaotic, indicates of the result of recognition. Let information blocks be stored in the map at th level, and an image be presented for recognition in the form of a string with symbols. The question is, does this image correspond to any of the stored information blocks, or not? The procedure is as follows. We look through fragments of this string, each symbols long, and change the slopes of those information regions of the map function that correspond to the fragments, so that they become less than one in magnitude, as in Fig. 2. If the presented image coincides with

24

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: FUNDAMENTAL THEORY AND APPLICATIONS, VOL. 44, NO. 1, JANUARY 1997

(a)

Fig. 2. Switching the slope of an information region.

an image stored in the system, all information regions of the map coupled with this image become switched, and the cycle eigenvalue becomes less than one. Hence, the cycle becomes stable. If we iterate the modified map now beginning from an arbitrary initial point, we find that in some time the phase trajectory converges to this single stable limit cycle, and the system’s behavior becomes regular. If the presented image has nothing in common with the stored images, the map function is not distorted, and the motion in the dynamical system remains chaotic. This procedure is not as complicated as it may seem. To represent a piecewise-linear map function we need to remember only the coordinates of the information region centers which the cycles pass through where is the number of the information regions in the map, We also have to each with its own value of the slope remember the storage level The cycle points are sorted by abscissa and kept in ascending order. When making a corresponds decision on whether a fragment to an information region, we do not compare it to all the -long substrings of all the information blocks, but take the central point of the related subinterval of th level, and search for it in the set of the cycle points using dichotomy. So, for comparisons each fragment of the presented image should be made. It is important, that only one attracting cycle appears in the system phase space. Therefore, if the global chaotic attractor in the original map had invariant measure (distribution of the system phase trajectory values over the phase space) intersecting information regions of the map, then in the modified map the trajectory will inevitably “fall” in some time onto an information region of the stable cycle and converge to this cycle. Pictorially, this can be described as an appearance of a “hole” in the chaotic set, through which the trajectory “leaks” out from the chaotic to the regular regime. This is a case of the loss of stability of a chaotic set and transition from stable to metastable (transient) chaos. The appearance of this transition (crisis of chaotic attractor) allows to realize recognition of information. Indeed, the above considerations imply that the changes to the map function are small, so that the system invariant measure remains roughly the same. The studies have shown that this condition is satisfied by storage at levels higher that the first, because the size of information regions becomes small compared to the unit interval, and the switching of the slope of an information segment of the map function have but a weak effect on the slopes of neighbor segments (see Fig. 2).

(b)

(c) Fig. 3. Chaotic dynamics of the map with information stored as unstable cycles. Three blocks 123, 14568, and 97583 are stored at the second level s 1:5. (a) The map function. The cycle for 97583 is shown. (b) Invariant measure of the map (distribution of the phase trajectory over the phase space). (c) Time series generated by the map.

=

The stable limit cycle appearing because of the crisis is unique, so the recognition process is practically independent of the initial conditions for the phase trajectory. The choice of an initial point determines only the duration of the transient process from the metastable chaotic set to the stable limit cycle, i.e., the rate of recognition. The dynamic properties of the system with information stored as unstable cycles are illustrated on example with three information blocks 97583, 14 568, and 123 stored at the second level (Fig. 3). The alphabet here consists of 10 symbols, the digits 0 to 9. The slope of information regions of the map is Note, that the phase trajectory of the initial map with unstable cycles visits all information regions of the map (Fig. 3 (b)). This property is assumed to remain, if the disturbances to the map caused by the change of the slopes are small. If we present now an image stored in the system, the corresponding stable cycle will appear in the system phase space, and the trajectory will converge to it in some time. But the described method can also be applied to the cases when the information on the image is incomplete, or partially incorrect or distorted, or only some parts of an image are available. Indeed, the cycle can become stable even if some of the

ANDREYEV et al.: INFORMATION PROCESSING

25

Assume that an image is presented then, and the slopes of information regions of the corresponding cycle are switched to and remain unchanged. The condition for the cycle stability is (2) After simple transformations we find the condition for the necessary number of switched regions providing the stable cycle

(a)

(b)

(c) Fig. 4. Image recognition in the map from Fig. 3. Convergence of the phase trajectory to the corresponding cycle after presentation of an image. The 1.5 to s 0:5: related information region slopes are switched from s (a) String 123 is presented. Cycle 123 is stabilized. (b) String 13568 is presented. Cycle 14568 is stabilized. (c) String 94683 is presented. Cycle 97583 is stabilized.

=

=

Relation (2) allows to estimate the admissible number of errors in a block, or helps to calculate the slope by which the related cycle will become stable for a prescribed error level. For instance, the eigenvalues of the cycles from Fig. 4(a)–(c) are and , respectively. A corollary from (2): for the cycle is absolutely stable The method of direct map function control that was designed to retrieve information from the map is based on the knowledge of the concrete map construction, but some general methods, such as cycle stabilization after OGY procedure [24], or chaotic synchronization [25], [26], also seem applicable for this purpose, however this question needs further investigations. Of course, the dynamic phenomena in the 1-D map with information stored as unstable cycles are much more rich. For instance, with a gradual change of the crucial parameter from the values to the system undergoes a set of bifurcations, local chaotic attractors (cycles of chaotic intervals) appear in the place of the stable cycles and then collapse, giving rise to global chaos. Detailed analysis of these phenomena is beyond the scope of this paper (see [27]). IV. ADAPTIVE MODEL

information regions which it passes through remain unchanged (with the slope greater than one), because its eigenvalue (1) is the product of the corresponding slopes. To illustrate this opportunity, the strings 1231, 135 681, and 946 839 were used, i.e., a stored information block without errors in the first case, with one error in the second case, and two errors in the third (all blocks closed in a loop). In all three cases a stable limit cycle appeared, related with the corresponding information block, and after a transient process, caused by the wandering of the phase trajectory over the metastable set, the trajectory “fell” onto the corresponding stable limit cycle (Fig. 4). The slopes here were switched from 1.5 to 0.5, and in all three cases iterates have begun from the 0.6. The difference in the time of same initial point iterates for the string 1231 and 2 iterates convergence, for 946839, is casual and is determined by the initial point. We can easily estimate the percentage of erroneous symbols in the presented strings for which the recognition is still successful using expression (1). Assume, that an information block with the length is stored in a 1-D map as an unstable cycle with the slopes of its information regions

AND

RECOGNITION

The above discussed possibility of retrieving information stored as unstable cycles of a 1-D map is an intermediate step in creation of a model of an “ON-LINE” system processing a continuos external information stream and capable of selection (“recalling”) of information images stored in the system. If there are excerpts of “known” information objects in the input information stream, the system “recalls” them and reproduces them thoroughly (because of the associative property) in the output information stream; if there are no “known” objects in the input stream, the system must return to the initial state (restore chaotic behavior). These different system states can be related to the “short-term” memory (”inspiration”) and “longterm” memory (“storage”) inherent of natural brain systems. Realization of these properties is possible with an adaptive model that we introduce here. It is a generalization of the above model, it represents a 1-D map with information stored as unstable cycles, and the form of the map function is controlled by an external signal (Fig. 5). The external signal here is an endless sequence of symbols fed to the system input, e.g., a random sequence, or a sequence containing successive repetitions of some of the stored information blocks.

26

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: FUNDAMENTAL THEORY AND APPLICATIONS, VOL. 44, NO. 1, JANUARY 1997

others are turned upward. If the input sequence consists of successive repetitions of a stored image, the related information regions eventually become modified (their slopes set less than one in magnitude), and a stable cycle appears in the system. When the repetitions end, the slopes return to the initial state, and the stable cycle disappears. Thus, the system permanently adapts itself to the input signal. Let us designate the upper boundary for the information region slopes as and the lower boundary as Then the relaxation to for th information region at th time step (no signal for this region) is described with equation

where determines the rate of relaxation. The convergence of the information region slope to the state is described with a similar equation

where the coefficient determines the rate of convergence. In numerical experiments we used the values and which provided fast convergence to and slow return to General equation describing the dynamics of is then Fig. 5. Block diagram of the adaptive recognition model.

The elements of the external signal all belong to the same alphabet as the stored information blocks. They are fed to the system input synchronously, one per iterate. The input signal influences not the system variable but directly controls the system function by changing the slopes of the information regions. The slopes in this model are not switched, but oscillate between two boundaries. Now, when an entire image at the input is not known immediately as a whole, the map function is modified permanently at each step according to the piece of information available at the moment. If the input contains pieces of information related to some information regions, their slopes are rapidly decreased with iterates, if not, they slowly return to the initial value. The system operation cycle now consists of two main stages. First, we modify the system function (each segment) according to the symbol present at the input at this moment. At the second stage, we iterate the modified map. If the storage was made at th level, the input symbols are accumulated to form a fragment of the length and, consequently, a point in the unit interval, the central point of the th level subinterval related to this fragment. If for th time moment the signal at the input corresponds to an information region , i.e., last symbols of the input stream form a fragment related to the information region the slope of this region is decreased according to some rule and may become less than one in magnitude. Besides, backward relaxation is introduced: if at a moment the input signal does not correspond to this information region then the slope begins return to the initial state, i.e., to the value of the slope in the absence of the external signal. Thus, at each time moment only one information region in the map function can be turned downward (decreased), all the

(3) where is equal to one, if for th step the external signal corresponds to information region otherwise it is zero. It is taken into account in (3), that at each time moment each information region is turned only in one direction. As follows from (3), the convergence to in the presence of the corresponding external signal may take place only if the condition

is satisfied, where is the length of the corresponding cycle. The simplest case corresponds to one-step convergence. The larger the magnitudes of and the slower the processes and return to of convergence to We will show now how the notions of “long-term memory” and “short-term memory”, widely used in the study of the principles of memory functioning in living systems (e.g., [11]), are applicable to the behavior of the adaptive system. Let us begin with the long-term memory. After information blocks are stored, they are present in the system all the time, and the carriers of information are the unstable cycles. Hence such a system may be interpreted as a long-term memory. Information is present in the system, but an external stimulus is necessary to retrieve it. The external stimulus is a signal containing information blocks stored in the system, precise or with some errors. In general, the system does not respond to other information, remaining in chaotic state. If the external signal with a stored information is fed to the system input, a stable limit cycle appears in the place of one of the unstable cycles. When the external stimulation is ceased, this cycle remains stable until the slopes of the corresponding information regions return to

ANDREYEV et al.: INFORMATION PROCESSING

27

Fig. 6. Time series of an adaptive system with the blocks 123, 14568, and 97583 stored at the second level for the external signal representing the block 123.

initial values, i.e., while the condition for the cycle stability is fulfilled. The dynamics of the system is determined by initial conditions. Consider two typical cases. The First Case: We begin iterations just after the cease of the external stimulation by the signal containing a stored image. A point at the related cycle is taken for the initial conditions. In this case the trajectory occurs directly at the stable limit cycle, and the system “recalls” (recognizes) the stored information. The external signal being absent then, the slopes of information regions gradually return to the initial state The phase trajectory remains in the vicinity of the cycle for some time (this time is composed of the time of the cycle stability loss and the time of “running away” from the unstable cycle), and then leaves it and begins wandering over the phase space, leading to chaotic behavior of the system. Thus, for some time the system “remembers” information actively, and then “forgets” it. The stored information again becomes passive, i.e., transferred to the long-term state. The active phase, a period of time when the system phase trajectory is in the vicinity of the limit cycle, may be considered as “short-term memory”. The Second Case: The initial point is arbitrary. Iterations are begun simultaneously with the external signal. A stable limit cycle appears in the system in some time, but this does not automatically lead to falling of the system trajectory onto this cycle. The trajectory may still wander over the phase space. If the invariant measure of the autonomous system (without external stimulation) has such a property, that the trajectory hits from time to time upon a map information region, belonging to the vicinity of this stable cycle, then a “hole” appears in the system’s global chaotic attractor, and it becomes metastable. If the limit cycle remains stable for sufficiently long time, the trajectory will eventually fall onto it. The above discussion is illustrated on example of three information blocks 123, 14568, 97583, stored at the second level, as in Section III. An external signal, consisting of successive repetitions of the information block 123 (100 repetitions) is given to the system input (Fig. 6). Immediately, iterations are begun with initial conditions The input signal is

Fig. 7. “ON-LINE” recognition of the input stream representing repetitions of the stored information blocks.

present for 300 iterates, then it is stopped. Approximately in 22 iterations from the start the phase trajectory falls onto the appeared stable cycle. It remains in the vicinity of the cycle for 318 iterates, while the external signal is active plus the time for the limit cycle to lose stability after the cease of the external signal and the time for the trajectory to “run away” from the unstable cycle. The information region slopes oscillate here between and It should be noted, that sometimes in numerical experiments the system trajectory converges so close to the stable cycle, that precisely coincides with it at last; when the cycle becomes unstable the trajectory remains at the cycle and never leaves it. This is a computer effect associated with finite accuracy of calculations. An external random noise was added to the right-hand side of the system equation in order to avoid this effect: the noise amplitude was 0.01 of the information region length, i.e., An example of an “ON-LINE” system is given in Fig. 7, where the dynamical system with the 1-D map from Fig. 6 demonstrates switching between three stored information blocks. The input signal, the output information stream and the map trajectory are presented. The input signal is pieces of the oscillations representing the stored images, 200 points each. As is seen from the figure, the phase trajectory follows (though with a time delay) the input images, so the dynamical system successfully recognizes the images in the input signal. A certain delay in switching between the cycles in the output stream is associated with the discussed effect of the short-time memory.

28

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: FUNDAMENTAL THEORY AND APPLICATIONS, VOL. 44, NO. 1, JANUARY 1997

V. CONCLUSION Let us discuss now the role of dynamical chaos in the above described chaotic models. We can distinguish the following important properties that appear with chaos. A competition of the stable information cycles caused by the strong dependence on the initial conditions disappears, because a single attracting cycle exists in recognition process. The chaos can be treated here as a reservoir containing “useful” trajectories (along with many other ones). The main role of the global chaos in these systems seems to be the global mixing, providing guaranteed though random access to all stored images: independently of the initial point the system phase trajectory will sooner or later occur in the vicinity of a required cycle (in the properly designed map). Along with the original models proposed in [17] and [18], the models studied in this paper proved to be a useful tool in studying information processing in chaotic systems. They implement a wide range of information processing functions, such as storing and retrieval, associative memory, memory scanning based on intermittency [20], image recognition based on storing by unstable cycles and direct map function control, “novelty filter,” “long-term,” and “short-term” memories, etc. The discussed models, very simple and allowing complete description, possess considerable information capacity, and may be of practical interest from the viewpoint of information processing technologies using chaos.

[16] B. Baird and F. Eeckman, “A normal form projection algorithm for associative memory,” in Associative Neural Memories: Theory and Implementation, M. H. Hassoun, Ed. New York: Oxford Univ. Press, 1992. [17] A. S. Dmitriev, “Storing and recognition information in one-dimensional dynamical systems,” Radiotek. Elektron., vol. 36, no. 1, pp. 101–108, 1991 (in Russian). [18] A. S. Dmitriev, A. I. Panas, and S. O. Starkov, “Storing and recognition information based on stable cycles of one-dimensional maps.” Phys. Lett. A, vol. 155, no. 8/9, pp. 494–499, 1991. [19] Yu. V. Andreyev, Yu. L. Belsky, and A. S. Dmitriev, “Information processing in nonlinear systems with dynamic chaos,” in Proc. Int. Seminar on Nonlinear Circuits Syst., Moscow, 1992, vol. 1, pp. 51–60. [20] Yu. V. Andreyev, A. S. Dmitriev, L. O. Chua, and C. W. Wu, “Associative and random access memory using one-dimensional maps,” in Int. J. Bifurc. Chaos, vol. 2, no. 3, pp. 483–504, 1992. [21] A. S. Dmitriev, “Chaos and information processing in dynamical systems.” Radiotek. Elektron., vol. 38, no. 1, pp. 1–24, 1993 (in Russian). [22] A. S. Dmitriev, D. A. Kuminov, V. V. Pavlov, and A. I. Panas, Storing and Processing Texts In 1-D Dynamical Systems, Preprint no. 3 (585), Inst. Radioeng. Electron. RAS, Moscow, 1993 (in Russian). [23] Yu. V. Andreyev, Yu. L. Belsky, and A. S. Dmitriev, “Storing and recognition of information using stable cycles of 2-D and multi-dimensional maps,” Radiotek. Elektron., vol. 39, no. 1, pp. 114–123, 1994 (in Russian). [24] E. Ott, C. Grebogi, and J. A. Yorke, “Controlling chaos,” Phys. Rev. Lett., vol. 57, pp. 1196-1199, 1990. [25] V. S. Afraimovich, N. I. Verichev, and M. I. Rabinovich, “Chaotic synchronization of oscillations in dissipative systems,” Izv. VUZov. Radiofiz., 1986, vol. 29, no. 9, p. 1050. [26] L. M. Pecora and T. L. Carrol, “Synchronization in chaotic systems,” Phys. Rev. Lett., vol. 64, pp. 821–824, 1990. [27] Yu. V. Andreyev, “Attractors and bifurcation phenomena in 1-D dynamical systems with stored information,” Prikladnaya nelineinaya dinamika, vol. 3, no. 4/5, 1995 (in Russian) to appear.

REFERENCES [1] J. S. Nicolis, “Should a reliable information processor be chaotic?” Kybernets, vol. 11, pp. 269–274, 1982. [2] J. S. Nicolis and I. Tsuda, “Chaotic dynamics of information processing—The ‘magic number seven plus-minus two’ revisited,” Bull. Math. Biol., vol. 47, pp. 343–365, 1985. [3] C. A. Skarda and W. J. Freeman, “How brains make chaos in order to make sense of the world.” Behavioral Brain Sci., vol. 10, pp. 161–165, 1987. [4] W. J. Freeman, “Simulation of chaotic EEG patterns with a dynamic model of the olfactory system,” Biolog. Cybern., vol. 56, pp. 139–150, 1987. [5] W. J. Freeman, Y. Yao, and B. Burke, “Central pattern generating and recognizing in olfactory bulb,” Neural Networks, vol. 1, pp. 277–278, 1988. [6] J. Eisenberg, W. J. Freeman, and B. Burke, “Hardware architecture of a neural network model simulating pattern recognition by the olfactory bulb,” Neural Networks, vol. 2, pp. 315–325, 1989. [7] Y. Yao and W. J. Freeman, “Model of biological pattern recognition with spatially chaotic dynamics,” Neural Networks, vol. 3, no. 2, pp. 153–170, 1990. [8] A. Babloyantz, “Evidence of chaotic dynamics of brain activity during the sleep cycle,” in Dimension and Entropies in Chaotic Systems, G. Mayer-Kress, Ed. Berlin: Springer-Verlag, 1986, pp. 252–259. [9] A. Babloyantz and A. Destexhe, “Low-dimensional chaos in an instance of epilepsy,” in Proc. Nat. Acad. Sci. USA, vol. 83, pp. 3513–3517, 1986. [10] A. Destexhe, J. A. Sepulchre, and A. Babloyantz, “A comparative study of the experimental quantification of deterministic chaos.” Phys. Lett. A, vol. 132, pp. 101–106, 1988. [11] R. J. Klatzky, Human Memory. Structures and Processes. San Francisco, CA: W. H. Freeman, 1975. [12] I. Tsuda, “A hermeneutic process of the brain,” Progress of Theoretical Physics, Supplement, vol. 79, pp. 241–259, 1984. [13] I. Tsuda, E. Koerner, and H. Shimizu, “Memory dynamics in asynchronous neural networks,” Progress of Theoretical Physics, vol. 78, pp. 51–71, 1987. [14] I. Tsuda, “Dynamic link of memory—chaotic memory map in nonequilibrium neural networks,” Neural Networks, vol. 5, pp. 313–326, 1992. [15] , “Can stochastic renewal of maps be a model for cerebral cortex?” Physica D, vol. 75, pp. 165–178, 1994.

Yuri V. Andreyev was born in Ufa, U.S.S.R., in 1960. He graduated from the Moscow Institute of Physics and Technology in 1983. He received the Ph.D. degree from the Institute of Radio Engineering and Electronics (IRE), Moscow, in 1993. Since 1983 he has been with the IRE of the Russian Academy of Sciences, Moscow, working in the field of microwave solid-state devices and chaotic dynamics; and, since 1991 he has been working the field of information processing based on chaotic dynamics.

Alexander S. Dmitriev was born in Kuibyshev, U.S.S.R., in 1948. He graduated from the Moscow Institute of Physics and Technology in 1971. He received the Ph.D. degree in 1974 from the same Institute. In 1988 he received the D.Sc. degree from the Institute of Radioengineering and Electronics, U.S.S.R. Academy of Sciences, Moscow. He is presently the chief of the Nonlinear Dynamics Group of the Institute of Radioengineering and Electronics. His research interests include nonlinear dynamics, bifurcation phenomena, neural networks, and chaos.

Sergey O. Starkov was born in Moscow, U.S.S.R., in 1956. He graduated from the Moscow Institute of Physics and Technology, in 1979, and received the Ph.D. degree in 1986 from the same Institute. He is currently a senior researcher with the Institute of Radio Engineering and Electronics, Russian Academy of Science. His research interests include dynamical chaos phenomena and their applications in radio engineering and information processing.