On neural network topology design for nonlinear control Jens Haecker and Stephan Rudolph* Institute for Statics and Dynamics of Aerospace Structures Universität Stuttgart, Pfaffenwaldring 27, D-70569 Stuttgart, Germany Proceedings SPIE Aerosense 2001 Conference On Applications and Science of Computational Intelligence IV, Orlando, Florida, April 16-20th, 2001 ABSTRACT Neural networks, especially in nonlinear system identi£cation and control applications, are typically considered to be blackboxes which are dif£cult to analyze and understand mathematically. Due to this reason, an indepth mathematical analysis offering insight into the different neural network transformation layers based on a theoretical transformation scheme is desired, but up to now neither available nor known. In previous works it has been shown how proven engineering methods such as dimensional analysis and the Laplace transform may be used to construct a neural network controller topology for timeinvariant systems. Using the knowledge of neural correspondencies of these two classical methods, the internal nodes of the network could also be successfully interpreted after training. As a further extension to these works, the paper describes the latest results of a theoretical interpretation framework describing the neural network transformation sequences in nonlinear system identi£cation and control. This can be achieved by incorporation of the method of exact input-output linearization in the above mentioned two transformation sequences of dimensional analysis and the Laplace transformation. Based on these three theoretical considerations neural network topologies may be designed in special situations by a pure translation in the sense of a structural compilation of the known classical solutions into their correspondent neural topology. Based on known exemplary results, the paper synthesizes the proposed approach into the visionary goals of a structural compiler for neural networks. This structural compiler for neural networks is intended to automatically convert classical control formulations into their equivalent neural network structure based on the principles of equivalence between formula and operator, and operator and structure which are discussed in detail in this work. Keywords: modular neural networks, engineering principles, dimensional analysis, Laplace transform, exact input-output linearization
1. INTRODUCTION Arti£cial neural networks (ANNs) are an alternative computational paradigm that calculates function approximations in a system of interconnected simple computational cells.1 Neural networks of the feed-forward multilayer perceptron type that are in the scope of this work can be trained to represent virtually any function using data samples since they are universal function approximators.2 Because of the simplicity of a single neuron, the complexity of the function represented by the whole system evolves from the interconnections between the cells. Typical applications of ANN in engineering are pattern recognition, databased modelling where processes are not well known or understood, and/or optimization problems, when a near sub-optimal solution is acceptable. Neural networks are less suitable for numerical calculations, when high precision is required, or modeling, when there already is a good physical model available, nor optimization, when the global optimum is required. Although performance strongly depends on the topology of ANNs and the choice of network activation functions, no reliable methods for determining those aspects are known up to date. So engineers currently rely more or less on trial-and-error methods to £nd a suitable topology and a training method. Moreover, learned information after training in the neural network is typically distributed over the parameters of the whole network which limits the possibility of interpreting and explaining the results. On the other hand, in engineering a number of principles and design rules to simplify and solve certain (standard) problems are known. These are, among others the Fourier transform for frequency dependent phenomena, the Laplace transform to transform linear differential equations into algebraic equations, or even an input-output linearization3 scheme for certain nonlinear * Correspondence: Email:
[email protected]; WWW: http://www.isd.uni-stuttgart.de/, Phone: +49 711 685 3799, Fax: +49 711 685 3706
control problems. For some problems system speci£c knowledge and principles are known such as the necessary condition of dimensional homogeneity which holds for any physically correct functional relation. It is claimed that this a priori knowledge has to be included in the network topology design process to make neural networks more applicable and transparent. The main interest in neural networks is currently concentrated on the use in nonlinear control problems. Many engineering solutions are tailored to suit linear problems. Generally linear systems pose therefore no unsurmountable problems. In the last years neural networks have mainly been used to model nonlinear systems in control. ANNs can £nd simple suboptimal solutions to control problems and can be applied to systems where classical approaches based on system linearization do not work. Yet, ANNs lack methods for determining control stability or the possibility to interpret the results analytically. Various approaches and applications of neural control exist in the literature.4,5 However from a scienti£c viewpoint it is still necessary to break the black-box structure of most network models. To include knowledge and interpretability into the neural network it is tried to identify neural correspondencies for engineering principles and to translate them into neural network modules or prede£ned sub-structures of the network. The goal is to de£ne a descriptor system that analyzes a problem and puts together the known parts of the network from prede£ned components. Some principles can be translated into primary data processing layers that can be attached to a core neural network with carefully devised degrees of freedom left in the network weights that are still to be learned. Thus, a simple translation of classical control theory solutions into a neural controller could be envisioned and constructed that results in a controller that performs at least as good as the classical or conventional approach. Further on, this controller could be made adaptive to adjust to a more complex real system using on-line training algorithms while being in service. The expected advantages are a transparent design scheme, an improved interpretation, and last not least more con£dence in the system in operation.
2. NETWORK TOPOLOGY Arti£cial neural networks of various kinds are known. Even though this examination is restricted to feed-forward multilayer perceptrons that have a comparatively simple structure, the freedom in the topology design allows various different architectures. Neural networks can be build up having any number of layers and neurons and virtually any mathematical operator as activation function in the neurons. The classical approach to use simple functions such as sigmoidal or hyperbolic functions in large networks has several disadvantages. For large networks training is slow and not reliable. Additionally, local minima of the optimization procedures interfere with the desire to £nd the global optimum, which is a major drawback of any multi parameter optimization. Another problem is the neural network’s currently limited ability to interpret and explain the training results, since the information learned is distributed over the whole network.
2.1. Thoughts on neural operators For neural operators there are all kinds of mathematical functions possible. Classically sigmoidal functions are used. They model the response functions found in biological neurons that jump from being inactive to an activated state when a certain threshold of excitation from the precedent neurons is exceeded. The sigmoidal function is similar to a step function, but is differentiable which is a property that is necessary for learning procedures like the backpropagation algorithm. Even though it is possible to model complex functions with nets of sigmoidal neurons, the trade-off is that the network structure will be large and complex. As stated, many problems with neural networks emerge from their size. However, this principle can be used in the opposite sense as a scheme to structure the neural network design process as explained in the following. The equivalence of a mathematical operator in its symbolic representation as a mathematical formula, and a network structure computing the corresponding mathematical expression are straightforward. Looking at the exponential function ex in Figure 1 for instance, it can be observed that three different forms of representations are possible. The operator as one compact element (bottom left), its power series expansion as a formula (top) and a network consisting of operators and parameters (bottom right) are mathematically equivalent expressions. The operators in the shown network structure could be expanded the same way in sub-nets which encode the operators xi in an even simpler way. It is easy to see the correspondency looking at the mathematical operator on the left side and the symbolic representation as mathematical formula, or looking at the network structure on the right and the symbolic representation. From these equivalencies it is clear that a mathematical operator can be coded in a network using very simple operators in many different but equivalent forms. Usually these equivalences are used to expand complex operators into networks of simpler operators. However, large networks have also disadvantages. The number of local minima of the optimization procedures, e.g. during the training, depends on the number of degrees of freedom, that is the number of used neurons and internal connections (weights). For this reason the demand for something one could call operator compactness is desirable. Instead of complex networks with simple functions one could prefer to build up simpler networks with more complex operators instead. This would reduce the number of parameters in the net and would further improve the training and the physical interpretation of the nets later on.
Figure 1. Equivalence of a mathematical operator (bottom left), a symbolic representation as a mathematical formula (top), and a network structure (bottom right) at the example of the exponential function ex .
Another property that could be exploited later is that all physically correct formulas belong to the so-called class of dimensionally homogeneous functions. If it is attempted to encode physical equations into a neural network, the condition of dimensional homogeneity should therefore always be observed.
2.2. Thoughts on network topology design using knowledge In the following the principles stated are visualized with a very simple physical system. In order to achieve this, a beam with a single vertical load P at the unsupported tip is considered. The de¤ection u of the tip due to bending of the beam can be analytically derived and expressed with the simple formula 6 u=
1 P l3 3 EI
,
(1)
where l is the length of the beam, E is Young’s modulus , and I is the moment of inertia of the beam about the axis of bending. It is a nonlinear time invariant static problem. Supposed the goal is to train a neural network to predict the de¤ection of the beam given the measurements of the system’s parameter vector, that is the static mapping ui = fN N (Pi , li , Ei , Ii ). Therefore a certain number m of training vectors (Pi , li , Ei , Ii ) and the system’s response (ui ) are used to train a network for representing the formula given in equation (1). This can be achieved very easily. However, the interpretation of the learned data from the network is not trivial. One ends up with parameters of the network connections collected in weight matrices Wij . Supposed a neural net was analyzed to extract the formula represented by the net, dif£culties may occur as can be seen in the example network in Figure 1 (bottom right). The more speci£c and compact the used operators are, the better is the possible physical understanding and interpretation of the network structure after training. In the following several steps to gradually incorporate knowledge into the topology of the network will be shown by making use of the principles described above. As can be seen easily, this will improve the interpretation and even the approximation result. This is achieved through the reduction of the degrees of freedom leaving only those parameters for the network free to learn that are really unknown. This will be shown in the following example. The neural network to start with is of a classical conservative structure. The used neurons calculate the sum of their weighted inputs and have sigmoidal activation functions s(x). The example uses four input neurons for the input vector (Pi , li , Ei , Ii ) and one for the output of (ui ) as shown in Figure 2a. The number of neurons in the intermediate layer is arbitrary. However, with a higher number of neurons dif£culties with the training of the network tend to increase. A closer examination of physical formulas reveals that most of them are of a certain structure. They tend to be of a product structure with exponents, e.g. p0 = ν0 pν11 × pν22 × . . . × pνmm (or sums thereof). This can be implemented directly in the neural network topology by simply using logarithmic functions in the £rst layer and exponential functions in the second layer. A net of that structure is shown in Figure 2b. However, by mathematical analysis it is found that in that structure all neurons in the second layer are calculating redundant functions and could be replaced by only one neuron as shown in Figure 3. The weight matrices Wij are replaced by parameters
u = fN N (P, l, I, E)
u=
P
ν0i P ν1i lν2i E ν3i I ν4i
i
Figure 2. Neural network to predict the de¤ection of a beam. Typical black-box net (a) and structured network (b) with speci£c neuron functions that determine a de£ned goal function.
that are directly interpretable as exponents of the function variables. The two steps considered in the Figures 2 and 3 can be automated by means of genetic algorithms that vary the neuron functions and the network topology as well as pruning techniques that remove neurons with small weighted connections as shown in previous works.7
u = w0 P w1 lw2 E w3 I w4
u = 31 P 1 l3 E −1 I −1
Figure 3. Network topology compaction by removing redundant neurons. From the trained network the exact function is obtained. After training the neural network shown in Figure 3b is obtained. It calculates the exact function for the bending of the beam. Possible small deviations from the exact parameters, which are the exponents, could be corrected by introducing a constraint that allows only integer numbers (or fractions thereof). Most exponents in physical formulas are in this respect elements out of the interval [−4, 4]. An even harder constraint can be formulated using the relations between physical units that are subject of the Buckingham or Pi-theorem.8 Only dimensionally homogeneous functions can be physically correct. That allows only certain combinations of physical quantities x that, according to equation (2), consist of a number {x} and a unit representation [x]. Generally, for any physical quantity x holds: x = {x}[x] (2) For this reason, any relationship applies to the numerical values of x as well as to its dimensional representation which is expressed in Buckingham’s theorem. By applying the Buckingham theorem to the relevant variables x1 . . . xn of the function, a set of dimensionless variables π1 . . . πm is found that form a dimensionless function equivalent to the original one. The dependencies between the variables are given by this theorem and can be used as known weight connections between neurons. So only the remaining unknown parameters have to be adapted by a training process. Figure 4b shows the new network. Obviously a core neural network is enclosed by a £rst and a last layer of neurons that are just a forward transformation layer from the physical quantities to dimensionless variables and a backward transformation, which are called π-transform and the π −1 -transform respectively. 6
u = w0 P w1 lw2 E w3 I w4
u = lν0 π1ν1 π2ν2 = lν0 u = 31 lπ1−1 π2−1
³
El2 P
´ ν 1 ¡ ¢ν 2 I l4
Figure 4. Transforming the network into a dimension homogeneous network by exploiting the relations between physical units.
2.3. Thoughts related to network interpretation and generalization It is clear from the example that a systematic incorporation of knowledge into the network topology results in a more structured and less complex network topology. The degrees of freedom, i.e. the free parameters or connection weights in the neural net are reduced to a smaller number. Thus, the training algorithms, i.e. the optimization procedures that usually suffer from the curse of dimensionality, are more ef£cient due to the reduction of free parameters. Moreover, in this easy example the represented equation can be extracted directly from the trained neural network. Yet, the technique of dimensional analysis offers another advantage which lies in the generalization of the function represented by the network in Figure 4b. Generalization is the ability of a neural network to process correctly unknown data which are not contained in the training samples. This means that the network can reasonably interpolate between given samples as well as extrapolate out of the sample range. The state-of-the-art in testing the network performance and its generalizing capability typically is to calculate a sum-squared error for a set of training data samples and generalization test data sets presented to the neural network using a crossvalidation technique. Dimensionally homogeneous neural networks generalize correctly all data that are physically similar to known data points. The usage of the Pi-theorem to construct data not given in the training data explicitely as alternative validation data has been proposed.7 A neural network trained with this data will represent not only known data samples but the underlying dimensionless functional relationship. This is an equivalent but more general representation of the functional relationship of the physical quantities based on the theoretical concept of similarity functions.6
3. MODULAR DESIGN OF NEURAL NETWORKS In the last section it was shown how knowledge can be incorporated in a neural network for the beam de¤ection example. It was a very easy static mapping problem so a small neural network was suf£cient to model the functional dependencies describing the system. More complex problems require bigger neural networks which will be more dif£cult to understand. Moreover, the training ef£ciency in neural network learning strongly depends on the network size as well. Hence, it is proposed to divide neural networks up into smaller subnetworks or modules that perform local computation. It is known that in the human brain, which serves as the neural network role model for its arti£cial counterparts, local regions of physically close neurons perform speci£c computational tasks. Examples are Brocca’s or Wernike’s region that are specialized on the recognition of spoken language.9 Naturally, researchers are still far from being able to model complex structures comparable to the human brain, but it is argued that complex behavior requires bringing together several different kinds of knowledge and processing paradigms, which seems not possible without structure, i.e. computational modularity. The modular approach permits to apply neural concepts to large scale networks. Even complex architectures could be greatly simpli£ed by identifying separate distinct subtasks in the problem and embedding them into a neural network structure. This way each task could be trained off-line and integrated later in a global architecture. Even hybrid models could be realized. Why should one wish to train a neural network for a task that is already completely described or modelled? This is an important question often ignored in the neural network community. It is seldom the case that from a process to be modelled nothing is known at all. Modular neural networks would allow the assembly of different simple plant models or to introduce a priori knowledge into the architecture by combining differently structured modules.
The solution strategy when working with modular neural networks implies three steps according to Ronco.10 First, the decomposition of a task into meaningful subtasks has to be mentioned. It is desirable that each module performs an explicit, interpretable and relevant function according to the mechanical and physical properties of the system. Second, the organization of the global modular architecture is important. Proposed architectures consist of several local system models and a gating system that switches between the models. Third and £nally, the integration of the inter-module communication is necessary. Signi£cant learning improvement is expected compared to a single big network. In small networks learning is faster and the learning results are easier to interpret. Moreover, modularity clari£es the overall presentation of the system because the activity of modules can be associated to certain operating regions in the plant. If those modules are linear in order to consequently connect the idea to control applications, the construction of good linear controllers with conventional techniques is possible, as well as a local analysis of their properties, such as stability. Another network can be used to trigger between adequate models and controllers, and the whole scheme can easily be coded into one global neural network as well.11 A schematic example of such a gated modular neural network (GMNN) used as an identi£ed model of a dynamic system is shown in Figure 5.
Figure 5. Neural network with a modular structure for the identi£cation of a plant. The neural network models NN2 and NN3 are £tted for different plant operating regions shown in the plot on the right. NN1 is a gating network responsible for detecting the appropriate system representation according to the input vector x and switching between the models. NN4 can be used to either process the switching signal or even calculate combinations of the two model network outputs.10
4. NEURAL CONTROL DESIGN ASPECTS Control theory offers powerful tools from linear algebra to be used for system analysis and control as long as the system behaves linearly. Assumptions of system linearity have been made for this reason to develop a control theory on a solid mathematical basis. Control design from system linearization is a widely applied technique in industry. However, in reality most systems are nonlinear. It is the ability of neural networks to model nonlinear systems which is the feature most readily exploited in the synthesis of nonlinear controllers. Neural control techniques have successfully been applied to problems in robotics and other highly nonlinear systems. 5 A growing number of different neural control schemes exist that are £tted only for certain problems. However, the usage of neural networks in nonlinear control does not make sense per se. There are still many open research topics, such as the characterization of theoretical properties such as stability, controllability and observability or even the system identi£ability. It is not intended to give a survey on neural control methods here, since many of the basic principles are shown in the reports by Hunt 12 or Narendra.13 The idea of a neural network structural compiler originates from the (re-)use of existing control theory applications, which intends the construction of mathematical controllers designed after classical theories and their representation in the form of neural networks. As indicated in Figure 1, the calculations done in the neural network will be identical. Therefore it is claimed that it will be possible to design neural controllers at least as good as the classical ones.14 However, by providing the network with additional degrees of freedom and applying training algorithms common in neural network computation, even an improved adaption could be achieved. Adaptivity is an important feature because the real world environment of the controller can be expected to be different from the simpli£ed linearized model used for the controller design. Analogous approaches to the idea promoted here already exist in techniques summarized under the term of intelligent control15 which represents an attempt to bring together arti£cial intelligence techniques and control theory. Controllers are put together from prede£nes components in a structured design approach with a knowledge-based expert system as integration tool. This is realized for instance in the neuro-fuzzy control scheme.15,16 A structure is provided by the fuzzy logic approach which builds up control laws from linguistic rules. Then the scheme is implemented in a neural network. The structure is determined after a simple algorithm from modules. Finally, the learning ability of the neural network is used to adapt the controller to the speci£c control situation by learning the controller’s parameter values.
4.1. Neural correspondencies to classical control theory Classical control theory utilizes a number of engineering principles that could be or are partly already applied to neural control. Preliminary studies done have concentrated on neural correspondences of engineering principles used in control and how these principles could be coded into a neural network scheme. 14,17,18 It is found that some realizations are clearly straightforward whereas others require more sophisticated procedures which can still be improved. In the following some of these principles will be explained in brief. Naturally, this list is far from being complete and should only indicate that neural correspondencies to classical engineering principles exist due to the equivalence between a neural topology and a mathematical formulation. These engineering principles are: • Dimensional analysis.14 The power of the dimensional analysis has already been demonstrated for the beam de¤ection example and is visualized in Figure 4. • The Laplace transform can be used to transform linear differential equations into algebraic equations and though is helpful for the analysis of dynamic systems. It is found that this transformation scheme can be transferred to a neural topology.17,14 However, the Laplace transform is only applicable to linear systems. • The input-output linearization3 scheme has already been applied to neural control. 19,18 The basic idea is to identify a feedback which linearizes the otherwise nonlinear behavior of the system. This way a system can be constructed which can be controlled like a linear system by using the standard classical approaches. The consequent combination of the above three principles results in a network with prede£ned layers that do some sort of data pre- and post-processing for a core network that can still be regarded as a black-box and which is the only part to be learned. Figure 6 shows exemplary a schematic diagram of this neural structure. It can be described as a network with a butter¤y topology which indicates that the information processed by the neural network is smaller than the original given data due to an intelligent selection of a data transformation sequence. The modular neural network shown in Figure 6 consists of the Pi-transform layers (Π and its inverse Π−1 ), the Laplace layers (L and its inverse L−1 ), and the input-output linearization layers (I/O) that encapsulate a core neural network. It should be noted that this structure is not a standard feed-forward network, because the Pi-transform uses shortcuts from the £rst to the last layer and the linearization requires feedback connections. This structure is found to be suitable for the identi£cation of a dynamic system, which is a basis for many neural control applications. 14
Figure 6. Modular neural representation of a nonlinear dynamic system. The butter¤y diagram: a core network encapsulated by transformation layers. From the inside: core neural network as black-box, input-output linearization layer (I/O), Laplace transform layer, and Pi-transform layer. Some thoughts should be given to common concepts in control such as the PID controller and the method of gain scheduling.20 This method provides a linear controller for several linearized states or operational points of a system. It is straightforward to implement gain scheduling in neural networks because the feedback gain coef£cient matrices usually are represented as look-up-tables and could as well be stored in neural networks.12,21 Even complex designs such as LQR or LQG controllers22,20 can be implemented most readily into a neural control scheme.
4.2. Neural Structure compiler concept This work aims at promoting the vision of a structural compiler for neural network topologies. Its function will be to assemble neural controllers from prede£ned modules based on a mathematical description as well as to disassemble the modules for system analysis. The modules are constructed from various engineering principles and classical control strategies by means of the identi£cation of neural correspondencies. Some examples of neural correspondencies to engineering principles have been shown in the previous section. For the example of system identi£cation using neural networks, studies have already been performed which investigate modular neural network topologies consisting of layers that perform forward- and backward-transformation around a core network with free parameters as shown in Figure 6. The underlying principles and their justi£cation have been discussed in detail.
Figure 7. Vision of a structure compiler for neural network topologies as an assembler and disassembler of neural modules that are correspondencies to engineering principles.
Based on the principle of neural correspondencies such a future neural network structure compiler will assemble a control scheme in an neural network topology which in the beginning will have the very same behavior as the corresponding conventional controller. By implementing additional free parameters it should be possible to use learning or optimization algorithms and the adaption capability of the neural networks to adapt the controller to the real world system to be controlled. To achieve this, a neural network structural compiler will be of help to translate and assemble the identi£ed principles and to link the world of control theory closer to the world of neural networks in an interpretable way. Advantages in training, generalization, interpretation, and adaption are expected due to the higher transparency of the module functionalities. Moreover, when the structure of the correspondencies is understood, the disassembling capability of the structural compiler can be used to analyze a controller adapted to a real world control situation. Due to an improved transparency of the network design and learning behavior, a better insight into the nature of nonlinear system dynamics and disturbances might be achievable as well.
5. SUMMARY AND CONCLUSIONS Based on the initial view of neural networks as black-boxes it has been shown how the neural network topology can be made more transparent and interpretable by the stepwise addition of engineering knowledge in form of engineering principles. This advantage has been shown in a step-by-step procedure for the example of a neural network which encodes a static nonlinear functional relationship based on the engineering principle of dimensional homogeneity. As a consequence of this result, the creation of a future structural compiler for neural networks is envisioned. As the main building blocks of this future structural compiler for neural networks the neural correspondencies of several classical engineering principles such as dimensional homogeneity, the Laplace transform and the exact input-output linearization have been identi£ed as useful candidate principles. Starting with a neural network which represents an exact neural correspondent to a conventionally designed controller, the additional free parameters in the network could be used for an improved adaptation to nonlinear effects not included in the original modelling. This property gives rise to the hope that such a neural controller design would be capable of outperforming its original. By means of neural correspondencies and their modular composition technique, the current lack of understanding of important questions in neural control theory concerning stability and control behavior may be advantageously investigated as well. Finally, in contrary to the current black-box approach, the internal functionality of the network (i.e. the weights and the activation functions in the neurons) can be much better analyzed and interpreted after training.
REFERENCES 1. A. Cichocki and R. Unbehauen, Neural Networks for Optimization and Signal Processing, John Whiley & Sons Ltd., New York, NY, 1993. 2. K. Hornik, M. Stinchcomb, and H. White, “Multilayer feedforward networks are universal function approximators,” IEEE Transactions Neural Networks 2(5), pp. 359–366, 1989.
3. F. Allgöwer, Näherungsweise Ein-/Ausgangs-Linearisierung nichtlinearer Systeme. PhD thesis, University of Stuttgart, Stuttgart, Germany, 1996. 4. G. Irwin, K. Warwick, and K. Hunt, eds., Neural Network Applications in Control, Control Engineering Series 53, The Institution of Electrical Engineers, Stevenage, UK, 1995. 5. T. W. Miller, R. S. Sutton, and P. J. Werbos, eds., Neural Networks for Control, MIT Press, Cambridge, MA, 1995. 6. S. Rudolph, “On topology, size and generalization of non-linear feed-forward neural networks,” Neurocomputing 16, pp. 1–22, 1997. 7. S. Rudolph, “On a genetic algorithm for the selection of optimally generalizing neural network topologies,” in Proceedings of ACEDC, PEDC, (University of Plymouth, UK), 1996. 8. T. Szirtes, Applied Dimensional Analysis and Modeling, McGraw-Hill, New York, NY, 1997. 9. E. Sanchez-Sinencio and C. Lau, eds., Arti£cial Neural Networks, IEEE Press, New York, NY, 1992. 10. E. Ronco and P. J. Gawthrop, “Neural networks for modelling and control,” Techncal Report CSC-97008, Center for Systems and Control, Glasgow, UK, November 1997. 11. E. Ronco, P. J. Gawthrop, and D. J. Hill, “Gated modular neural networks for control oriented modelling,” Technical Report EE-98009, Laboratory for Dynamic Systems and Control, Sydney, Australia, 1998. 12. K. J. Hunt, R. Sbarbaro, R. Zibikowski, and P. J. Gawthrop, “Neural networks for control systems - a survey,” Automatica 28(6), pp. 1083–1112, 1992. 13. K. S. Narendra and K. Parthasarathy, “Identi£cation and control for dynamic systems using neural networks,” IEEE Trans. on Neural Networks 1, pp. 4–27, March 1990. 14. G. Schneider, D. Korte, and S. Rudolph, “Neural network correspondencies of engineering principles,” in Proceedings SPIE Conference On Applications and Science of Computational Intelligence III, (Orlando, FL), April 2000. 15. D. A. White and D. A. Sofge, eds., Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, Van Nostrand Reinhold, New York, NY, 1992. 16. K. M. Passino and S. Yurkovich, Fuzzy Control, Addison-Wesley, Menlo Park, CA, 1998. 17. D. Korte, “Approximation der Laplace Transformation mit Neuronalen Netzen,” Master’s Thesis, Institute for Flight Mechanics and Control, University of Stuttgart, Stuttgart, Germany, 1999. 18. W. Napolskich, “Implementation und Vergleich verschiederner Regelungskonzepte anhand der Regelung eines Ausspulvorganges eines seilgefesselten Rückkehrkörpers,” Master’s Thesis, Institute for Mechanics A, University of Stuttgart, Stuttgart, Germany, 2000. 19. M. Nørgaard, Neural networks for modelling and control of dynamic systems: a practitioner’s handbook, Advanced textbooks in control and signal processing, Springer, London, UK, 2000. 20. H. Unbehauen, Regelungstechnik I, II, III, Vieweg & Sohn, Braunschweig, Germany, 9. ed., 1997. 21. H. Gläßel, “Neuronale Regelung orbitaler Seilstrukturen am Beispiel eines Ausspulvorganges während einer Seilgestützten Rückkehrmission,” Thesis, Institut for Space Systems, University of Stuttgart, Stuttgart, Germany, 1999. 22. K. Ogata, Modern Control Engineering, Prentice-Hall Inc, Eaglewood Cliffs, N.J., 1970.
