common in a standard self-tuning controller. The paper explains the algorithm for a general case, and then a specific application on a nonlinear plant.
Neural Comput & Applic (1999)8:143–150 1999 Springer-Verlag London Limited
An Application of a Neural Self-Tuning Controller to an Overhead Crane J.A. Me´ndez, L. Acosta, L. Moreno, S. Torres and G.N. Marichal Department of Applied Physics, University of La Laguna, La Laguna, Tenerife, Spain
A neural network-based self-tuning controller is presented. The scheme of the controller is based on using a multilayer perceptron, or a set of them, as a self-tuner for a controller. The method proposed has the advantage that it is not necessary to use a combined structure of identification and decision, common in a standard self-tuning controller. The paper explains the algorithm for a general case, and then a specific application on a nonlinear plant is presented. The plant is an overhead crane which involves an interesting control problem related to the oscillations of the load mass. The method proposed is tested by simulation in different conditions. A comparison was made with a conventional controller to evaluate the efficiency of the algorithm. Keywords: backpropagation; tuning controller
neurocontrol;
self-
1. Introduction The application of intelligent techniques to control systems has been a matter of wide study in recent years. These methods are used to solve complex problems that, in many cases, do not have an analytical solution. Neural Networks (NN), due to their ability to learn, have become a powerful tool in the development of control systems. In fact nowadays, a new branch in control theory has arisen: neurocontrol. This discipline studies the design of control systems aided by NN. Although in the process industry simple conventional controllers such as the PID have largely been Correspondence and offprint requests to: J. A. Me´ndez, Department of Applied Physics, University of La Laguna, 38271 La Laguna, Tenerife, Spain. E-mail: jamp얀cyc.dfis.ull.es
extended, and show good performance for many tasks, when the plant or the process under control is complex or has high non-linearities, the control performance degrades notably. In this case, different solutions are employed to improve the effectiveness of conventional controllers. Specifically, adaptive control systems are frequently used for this kind of plant. This paper deals with the design of adaptive control systems aided by NNs. There are two main ways in which to use a NN in a control system [1]. The most direct way is to use the NN directly as a control block replicating the inverse dynamics of the plant. One of the main weaknesses of this configuration is the robustness problem. Due to the problems of predicting the behaviour of the network, it is difficult to study the system’s stability. An alternative way in which to use NNs in control is to use them as an adaptive control system. The neurocontroller proposed here is a self-tuning system consisting of a conventional controller combined with a NN to calculate the coefficients of the controller on-line. These parameters are adjusted using a pre-specified adaptation law, such as a linear quadratic function. The aim of this scheme is to reduce the training time of the controller in order to make the real-time application of this algorithm possible. In addition, the use of a conventional controller instead of an inverse NN controller increases the system’s robustness. An application of this controller was implemented on a nonlinear plant. The plant, very typical in the process industry, is an overhead crane. Its task is to transport containers between two points in a plane. Due to its nonlinear properties, the load mass suffers from oscillations along the trajectory, which are particularly undesirable upon the arrival. One well known method of dealing with the
144
J.A. Mendez et al.
cancellation of oscillations is posicast control [2]. It consists of an open-loop technique which guarantees no transient overshoot and no oscillations. Nevertheless, the application of this method to a nonlinear plant is not easy. Some approaches proposed recently to solve the anti-swing control problem are open-loop schemes [3], optimal control [4], pole placement control [5] or linearisation techniques [6]. All these control schemes have limited performance under certain conditions, especially when unmodelled dynamics affect the plant. Although adaptive control systems have been developed to solve this problem, they demonstrate practical problems in the convergence of the algorithm. This work is divided into two main parts. The first is dedicated to the design of the self-tuner controller; the second is an application of the algorithm to a nonlinear system, an overhead crane. Some simulations were carried out to verify the efficiency of the self-tuner, and then a real-time implementation on a scale prototype was performed.
2. Review of Neurocontroller Structures There are three main configurations for a neuromorphic controller [7,8]. The basic one is called direct inverse control (Fig. 1a). In this scheme, the NN learns the inverse dynamics of the plant, i.e. if the plant can be represented by a function F that maps u into y, the NN is trained to model the function F−1 that maps y into u. Thus, the input to the network with this scheme will be the reference signal. The training signal is built from the error between the output of the net and the command applied to the plant in the training stage. The controllers in Fig. 1(b) and Fig. 1(c) are adaptive controllers. In these configurations, learning is performed on-line with a training signal which evaluates the error of the system. Figure 1(c) includes another net whose function is to emulate the plant behaviour, taking the error between the output of the net and the output of the system as a training signal. Although good performance can be achieved after enough learning, these schemes require a long training time until good performance is achieved. Even in this case, it is not possible to ensure robustness for this kind of controller. In this work, an alternative scheme combining a conventional controller with a NN-based system to adaptively tune the coefficients of the controller is proposed.
Fig. 1. Control configurations. (a) Direct inverse controller, (b) direct adaptive controller, (c) indirect adaptive controller. In all three configurations u and y inputs are converted into vectors by applying the backward-shift operator (q−1) to the input.
3. An Alternative Self-Tuning Controller The aim of this work is to propose an effective method for adjusting the parameters in a controller adaptively. To do that, a NN is used to provide the parameters for the controller by using information on the system’s state. The classical structure of a self-tuning controller has two main elements [9]: a feedback controller; and a self-tuner consisting of a parameter estimator and a control design algorithm (Fig. 2). In this scheme, a parametric model for the plant is assumed. The parameter estimator calculates an estimate of the parameter vector of this model. This information is passed to the control design algorithm, which generates the new parameters for the feedback control law. An implicit self-tuner is also possible: in this case, the estimator directly provides the parameters to the controller. The self-tuner proposed in this work differs from this structure. There is no assumption about a para-
An Application of a Neural Self-Tuning Controller to an Overhead Crane
145
Then, it is necessary to calculate the term ⭸J(k)/⭸wl,j,i for each weight of the network. In the controller proposed here, the outputs of the network, qj, are the controller parameters, and do not appear in the cost function explicitly. So the sensitivity function in the last layer becomes ⭸J(k) ⭸J(k) ⭸qj = ⭸wL,j,i ⭸qj ⭸wL,j,i
(9)
The first factor of this expression is ⭸J(k) ⭸J(k) ⭸y(k) ⭸u(k) ⭸J(k) ⭸u(k) = + ⭸qj ⭸y(k) ⭸u(k) ⭸qj ⭸u(k) ⭸qj Using Eq. (7), this expression becomes Fig. 2. Explicit self-tuning controller.
metric model for the plant, and the NN does the self-tuner task. It acts as an implicit self-tuner, directly providing the parameters to the controller. Actually, there is no estimation and control design stage. The NN at each instant simply looks for the optimal parameters to minimise a cost-function using information on the system state. The basic structure of this controller for a SingleInput Single-Output (SISO) plant is shown in Fig. 3. In this scheme, the inputs to the NN are the error and the error derivative. The outputs are the parameters for the controller. The NN is trained on-line to minimise a quadratic cost function of the form
␥ J(k) = (yref(k) − y(k))2 + u2(k) 2 2
(7)
where and ␥ are costing parameters on the error and on the command, respectively. The learning of the network is done following the backpropagation algorithm. As is well known, this algorithm is based on a gradient descent technique. The process for updating the weights is done according to ⭸J(k) wl,j,i(k + 1) = wl,j,i(k) − ⭸wl,j,i
Fig. 3. Neuromorphic self-tuning controller.
(8)
冉
冊
⭸J(k) ⭸y(k) ⭸u(k) = ␥u(k) − e(k) ⭸qj ⭸u(k) ⭸qj
(10)
where e(k) = yref(k) − y(k) is the output error. The factor ⭸y(k)/⭸u(k) is obtained immediately if the model of the plant is known. Avoiding any assumption about the plant model, this term can be obtained by difference approximation: ⭸y(k) y(k) − y(k − 1) ⬇ ⭸u(k) u(k) − u(k − 1)
(11)
Obviously, this expression presents numerical problems in its evaluation when u(k) = u(k − 1). If this situation occurs, then the NN stops training until u(k) and u(k − 1) have different values. On the other hand, if the system is affected by noise perturbations, then it is necessary to apply a low-pass filter to the variable y prior to the calculation of the partial derivative. With these two special precautions, the approximation (11) can be used with guarantees. The factor ⭸u(k)/⭸qj in Eq. (10) is calculated from the expression for the controller. If the controller is changed, it is only necessary to change the sensitivity function, and the rest of the algorithm remains the same. One of the main points in the design of any control scheme is concerned with the stability properties of the overall system. In neuro-controllers based on the structures shown in Fig. 1, the analysis of the system stability seems arduous. This is mainly because of the difficulties in modelling the NN dynamics. On the other hand, in the self-tuning method proposed here, the classical feedback structure is maintained, and the NNs do not take part directly in the calculation of the commands. Thus, for a given plant it is possible to study the stability of the overall system by using any known techniques, such as the functional analysis pioneered by Popov, or the Liapunov method. Furthermore, the
146
J.A. Mendez et al.
4. Application to an Overhead Crane 4.1. Plant Modelling
Fig. 4. Self-tuning PI of a nonminimum-phase plant by means of neural networks.
results of the analysis can be incorporated into the NN by means of the criterion function J. In such a way, specific regions of the parameter space can be penalised if they drive the system into an unstable response. As a test of the stability properties of this algorithm, some simulations were carried out with non-minimum-phase and open-loop unstable plants. The structure of the self-tuning is similar to that shown in Fig. 3 using a PI controller. In Figs 4 and 5, the results obtained can be seen. Note that, especially when the plant is in a non-minimumphase, a good tuning of the parameters is critical to ensure stability. As can be observed, in both cases the response of the closed-loop system is stable, and achieves the reference signal.
As an application of the controller, a system with non-linear properties was chosen. The plant is an overhead crane, a simplification model of which is shown in Fig. 6. The task of the crane is to bring containers with different masses from an initial point to a final point. There are four degrees of freedom: the position of the crab (x); the linear velocity (x·); · the rope angle (); and its angular velocity (). It is assumed that the crab has a mass M, the length of the rope is l and the load mass is m. A simplification is done by assuming that the rope has a negligible mass. The input of the plant, u, represents the horizontal force applied to the crab. The application of a non-zero force on the plant produces undesirable oscillations in the load mass. The control objective is to make the crab arrive at the final position with · and equal to zero. Defining the state variables · x1 = x, x2 = x·, x3 = and x4 = , the dynamics of the plant can be described by the following non-linear second-order equations:
冎
ml(x¨3cos x3 − x24sin x3) + (M + m)x¨1 = u x¨1cos x3 + lx¨3 = −gsin x3
(12) The first equation describes the horizontal motion, and the second the vertical motion. 4.2. Controller Design The goal of the controller is to move the cart from an initial point to a final one, ensuring a small swing angle along the trajectory (smooth motion), and especially at the final point, where the objective
Fig. 5. Self-tuning PI of an open-loop unstable plant by means of neural networks.
Fig. 6. Schematic view of the plant.
An Application of a Neural Self-Tuning Controller to an Overhead Crane
is to achieve a swing angle and a swing velocity angle equal to zero. In terms of the state variables, the control objective is to take the crane from the initial state x1 = x1i, x2 = 0, x3 = 0, x4 = 0 to the final state defined by x1 = x1 ref, x2 = 0, x3 = 0 and x4 = 0. For this plant a simple controller, such as a PID, is not able to accomplish the control objective. For this task, the control scheme shown in Fig. 7 is proposed. As can be seen, an LQR scheme is applied on the x2 and x3 state variables, and on the x1 error variable. Thus, the command applied is u(k) = −[f1 f2 f3] × [e1(k) x2(k) x3(k)]T
147
find the optimal values for f1, f2 and f3. The computation of the gradient is done according to Eq. (9). Now, the terms ⭸u/⭸qj are ⭸u(k) ⭸u(k) ⭸u(k) = e1(k); = x2(k); = x3(k) ⭸f1 ⭸f2 ⭸f3 Starting from the instant k, the algorithm can be summarised as follows: Step 1 Step 2
(13)
being e1(k) = x1 ref(k) − x1(k) Thus, there are three parameters to tune: the gains f1, f2 and f3 corresponding to e1, x2 and x3, respectively. To improve the convergence of the networks, the tuning scheme proposed was split into two selftuning subsystems. One NN (NN1) was used to tune the parameters related to the crab variables, f1 and f2, and another NN (NN2) was used to provide the parameter related to the load mass variable, f3. The inputs to the net NN1 are the x1 error and the x1 derivative, and for NN2 the x3 error and the x3 derivative. The cost function proposed is J(k) = p1(k)(x1 ref(k) − x1(k))2 + p2(k)x22(k) + p3(k)x23(k) + p4(k)x24(k)
冉 冉
Step 4 (14)
where pi(k) are time-dependent functions that weigh each variable in the cost function. Depending on the position of the crab, the pi(k) variables will take the appropriate value. The training of the network is done on-line. At each stage of the process, the NNs are trained to
Fig. 7. Structure of the controller for the crane system.
READ x1(k), x2(k) x3(k) and x4(k) (Networks Training) NN1 training: ⭸u(k) ← e(k) ⭸f1 ⭸u(k) ← x2(k) ⭸f2 Backpropagation ⭸u(k) ⭸u(k) . , ; e(k),e(k); pi(k)兩i=1,%,4 ; ⭸f1 ⭸f2 UPDATE f1 and f2 NN2 training: ⭸u(k) ← x3(k) ⭸f3 Backpropagation ⭸u(k) ; x3(k),x4(k); pi(k)兩i=1,%,4 ; ⭸f3 UPDATE f3 COMPUTE command u(k) = −(f1 f2 f3)(x1(k) x2(k) x3(k))T APPLY u(k) and WAIT until t = (k + 1)T k ← k + 1 GOTO Step 1.
Step 5 Step 6 Step 7
冊
冊
5. Results The two neural networks used in the implementation of the algorithm have three layers and three nodes per layer in the hidden layers, with a sigmoid activation function. For the output layer, the activation function is chosen as the identity function. The initial weights are chosen to be random and small. The initial values for the parameters f1, f2 and f3 are also small to avoid brusque initial changes in the state. For the cost function, only one change was considered for the weights pi(k). The initial values must predominantly weigh the variables x1 and x2. Conversely, the final values for pi(k) must in the main weigh the x3 and x4 variables. The sample period chosen for simulation was T = 0.03 s. Figure 8 shows the evolution of the x1 and x3 variables for one of these experiences. It can be seen that x1 evolves from the initial point to the final value x1 ref, arriving smoothly and with no
148
J.A. Mendez et al.
reach a stationary state. The command applied to the plant in this experience is shown in Fig. 10. To evaluate the efficiency of this neuromorphic self-tuning, a comparison with another method was performed. The method studied is based on an explicit identification process [10]. This technique consists of considering the following linear approximation of the plant in each point of the trajectory: x(k + 1) = F(k)x(k) + G(k)u(k); k = 0,1,%
Fig. 8. Evolution of the x1 and x3 variable for a neuromorphic self-tuning (solid line) and a standard self-tuning (dashed line).
overshooting. Simultaneously, the variable x3 starts oscillating until the crane is near to the final position. At this point, the mass oscillation decreases and tends towards the equilibrium point x3 = 0, x4 = 0. The evolution of the controller parameters can be seen in Figs 9(a), (b) and (c). The coefficients change adaptively along the trajectory, and then
(15)
where F(k) is 4 × 4 and G(k) is 4 × 1. The identification process, based on a recursive least squares algorithm, produces an estimation for the matrices ˆ (k). The control law applied each instant Fˆ (k) and G is u = −Kx, where the gain vector K must be chosen to minimise the following criterion function: 1 J(k) = [xT(k + 1)Qx(k + 1) + uT(k)Ru(k)] 2 Thus, the vector K is found to be ˆ T(k)QG ˆ (k)QFˆ(k) ˆ (k)]−1G K = −[R + G
(16)
The response obtained with this scheme, after a fine tuning of the cost matrixes Q and R, appears in Fig. 8 (dashed lines). As can be seen, similar performance is produced with both approaches. However, slightly better damping for x3 and a smoother approach to the x1 reference is achieved with the neuro self-tuning. Furthermore, the performance of the conventional scheme depends largely upon the cost matrices chosen in the optimisation process. An improper choice of these coefficients drives the system to an unstable behaviour. This problem does not appear when using neural networks. To verify the robustness of both approaches in the presence of noise perturbations, a measurement
Fig. 9. Evolution of f1, f2 and f3 for an experience with M = 10 Kg and m = 5 Kg.
Fig. 10. Evolution of the command for the experience shown in Fig. 8.
An Application of a Neural Self-Tuning Controller to an Overhead Crane
noise was added to the x3 variable. The noise considered has a gaussian distribution with mean 0.0 and variance 10e-6. The evolution of the x3 variable under these conditions is shown in Fig. 11. There now exists a clear difference between both methods. In the neuro self-tuning method, the perturbations produce initial oscillations with higher amplitudes, but the damping characteristic is excellent, ensuring zero amplitude oscillations at the final stage. On the contrary, the explicit self-tuning method presents a very badly damped oscillation for the x3 variable that reduces the efficiency of this method. One point to be considered in order to implement a real-time application of this strategy is related to the learning phase of the networks until they get convergence. In this training stage, the system can show undesirable behaviour which could cause unsatisfactory performance of the control system. One of the easiest ways to solve this problem is to take the network to a region of the weight space that is close to the point of convergence. This can be accomplished by means of previous simulation training. To check the real-time applicability of the method proposed here, an analysis of the computational time required for the algorithm was carried out. It consisted of measuring the total computation time in each sample stage, accomplished by performing simulations on different machines. The results for different platforms are displayed in Table 1. It is interesting to note that, for low series processors (PC-AT and PC-80386), the method is not applicable, since the time taken for calculus exceeds the fixed sample period (T = 0.03 s). However, for a PC486 and higher, it is possible to ensure the applicability of this method from a computational point of view. The real-time implementation of this controller
Fig. 11. Comparison of the neuromorphic self-tuning (solid line) and a standard self-tuning (dashed line) assuming a measurement noise in x3.
149
Table 1. Computational time per stage for different processors. Computer
Time (s)
PC-AT 16 MHz. PC-386 25 MHz. PC-486 50 MHz. PC-486 66 MHz. Pentium 100 MHz. Pentium 200 MHz.
0.185 0.079 0.016 0.013 0.009 0.002
Fig. 12. General view of the crane prototype.
was performed on a scale prototype of an overhead crane. Figure 12 shows a general view of this plant. It consists basically of a structure with an arm on which a cart powered by two DC motors can move horizontally transporting a container. The input to this system is the voltage applied to the motors that can be in the range 0–9 V. The maximum angular speed is about 60 rpm. The parameters of this prototype are M = 0.5 Kg, m = 0.5 Kg and l = 0.5 m. Several experiments were carried out on the prototype using the neurocontroller proposed. In Fig. 13
Fig. 13. Real time implementation on the crane prototype. Evolution of the x1 and x3 variables.
150
J.A. Mendez et al.
the results obtained for one such experiment are presented. As can be observed, the control objective is obtained satisfactorily. The cart attains the desired position, arriving smoothly and without oscillations in the container.
relative short period of time. To ensure the practical applicability of the algorithm, a computational time analysis demonstrated that the implementation is possible with a standard processor. Thus, the realtime results obtained on a scale prototype of a crane also gave satisfactory performance.
6. Conclusions
References
An adaptive controller based on neural networks has been proposed in this paper. The structure of this controller consists of a combination of a conventional controller and a neural network subsystem to adjust the parameters of the controller. The neural network used is a multilayer perceptron trained with a backpropagation algorithm. The role of the NN is to avoid an explicit identification and a decision algorithm in the self-tuning controller. The use of a NN in the control structure presents two main advantages in this sense. First, there is no linearisation process for the plant, so it is supposed that there is no loss of accuracy in the control. Secondly, there is no model assumption in the algorithm, thus avoiding possible errors due to unmodelled dynamics. The main drawback is the behaviour of the NN in the first stage of learning. In this sense, there could be improvements by proposing an adequate set of initial weights in order to avoid an undesirable motion at the beginning. The performance of this controller was evaluated using a typical nonlinear plant in industry: an overhead crane. The simulation results showed that the NN, trained by means of a backpropagation algorithm, performed a correct on-line learning, achieving convergence in a
1. Miller III W, Sutton R, Werbos P. (eds). Neural Networks for Control. 1990. MIT Press, Cambridge, MA 2. Smith OJM. Feedback Control Systems. 1958. McGraw-Hill, NY 3. Strip DR. Swing-free transport of suspend objects: a general treatment. IEEE Trans Robotics and Automation 1988; 5: 234–236 4. Hara K, Yamamoto T, Kobayashi A, Okamoto M. Jib crane control to suppress load swing. International Journal of Systems Science 1989; 20: 715–731 5. Gustaffson T. On the design and implementation of a rotary crane controller. European Journal of Control 1996; 2(2): 166–175 6. Cheng C, Chen C. Controller design for an overhead crane system with uncertainty. Control Engineering Practice 1996; 4(5): 645–653 7. Fukuda T, Shibata T. Theory and applications of neural networks for industrial control systems. IEEE Trans Industrial Electronics 1992; 39(6): 472–489 8. Tanomaru J, Omatu S. Process control by on-line trained neural networks. IEEE Trans Industrial Electronics 1992; 39(6): 511–521 9. Clarke D. Introduction to self-tuning controllers. In Harris CJ, Billings SA (eds), Self-tuning and Adaptive Control: Theory and Applications. 1985. Peter Peregrinus, London 10. Lee CSG, Chung MJ. An adaptive control strategy for mechanical manipulators. IEEE Trans Automatic Control 1984; 29(9): 837–840
