Integrated Batch-to-Batch Control and within Batch

A Neural Network Based Strategy for the Integrated Batch-to-Batch Control and within Batch Control of Batch Processes Jie Zhang School of Chemical Engineering and Advanced Materials University of Newcastle, Newcastle upon Tyne NE1 7RU, U. K. E-mail: [email protected]

Abstract An integrated batch-to-batch control and within batch re-optimisation control strategy for batch processes using neural network models is presented in this paper. In order to overcome the difficulties in developing detailed mechanistic models, neural network models are developed from process operation data. Due to model-plant mismatches and unknown disturbances, the optimal control policy calculated based on the neural network model may not be optimal when applied to the actual process. Utilising the repetitive nature of batch processes, neural network model based iterative learning control is used to improve the process performance from batch to batch. However, batch-to-batch control can only improve the performance of the future batches but cannot improve the performance for the current batch. Within batch re-optimisation should be used to overcome the detrimental effect of disturbances on the current batch. In the proposed integrated control scheme, the effect of unknown disturbance is estimated using a neural network based inverse model using mid-batch process measurements. The estimated effect of unknown disturbance is then used to re-optimise the control actions for the remaining period of the batch operation. The proposed technique is successfully applied to a simulated batch polymerisation process.

Keywords: Batch processes, batch-to-batch control, neural networks, optimal control, polymerisation.

1. Introduction

Batch processes are suitable for the responsive manufacturing of high value added products, such as special polymers, special chemicals, and pharmaceuticals. A typical feature of batch processes is that the same process unit is used in the manufacturing of different products and each product is of relatively small amount and high added value (Bonvin, 1998; Ruppen et al., 1995). In responsive agile manufacturing, production objective changes dynamically with custom demands. Optimal control can be used to improve the profit of batch process manufacturing (Thomas and Kiparissides, 1984). To implement optimal control, accurate process models are usually required. Early works on batch process optimal control concentrate on first principle models (Park and Ramirez, 1988; Thomas and Kiparissides, 1984). A difficulty in the optimal control of batch processes, such as batch polymerisation reactors, is that first principle models of batch processes are usually very complicated

1

and difficult to obtain and implement for on-line control. Data based empirical modelling can be a very useful alternative in this case. Neural networks have been shown to be capable of approximating any continuous nonlinear functions (Cybenko, 1989; Girosi and Poggio, 1990; Park and Sandberg, 1991) and have been applied to nonlinear process modelling (Bhat and McAvoy, 1990; Bulsary, 1995; Narendra and Parthasarathy, 1990; Su et al., 1992; Tian et al., 2001).

Optimal control of batch processes based on neural network models have been reported recently (Tian et al., 2001; 2002). In order to achieve good control performance, the neural network models developed need to have good generalisation capability. The generalisation capability of a neural network model is usually determined by the training data and training methods. To build accurate neural network models, ideally a large amount of data covering different process operating conditions should be used in network training. In batch processes, data for building neural network models are usually not abundant due to limited batch runs in the manufacturing of a particular product. Thus the neural network model developed is only an approximation of the modelled batch process and model plant mismatches are unavoidable (Zhang, 2004a). Furthermore, unknown disturbances, such as reactive impurities and raw material variations, usually exist in batch processes. Due to the model plant mismatches and the presence of unknown disturbances, the optimal control policy calculated on the model may not be optimal when applied to the actual process. Several methods have been proposed to address this issue. Tian et al. (2004) and Xiong and Zhang (2005a) propose using on-line re-optimisation to overcome the detrimental effects of model-plant mismatches and unknown disturbance within a batch. Zhang (2004a) presents a reliable batch process optimisation method using bootstrap aggregated neural network models by incorporating model prediction confidence bounds in the optimisation objective function. Wide model prediction confidence bounds are penalised so that the model predictions under the calculated control policy are reliable and, hence, lead to reliable optimal control.

The repetitive nature of batch process operations allows information from previous batch runs be used to improve the operation of the next batch. Iterative learning control (ILC) can be used to implement batch-to-batch control (Gao et al., 2001; Lee et al., 2000; Xiong and Zhang, 2003; 2004; 2005b; Zhang, 2003). The basic idea of ILC is to update the control trajectory for a new batch run using the information from previous batch runs so that the output trajectory converges asymptotically to the desired reference trajectory. Refinement of control signals based on ILC can significantly enhance the performance of tracking control systems. Campbell et al. (2002) present a brief survey of batch-tobatch control algorithms based on linear models for batch processes. Amann et al. (1996) propose an optimal iterative learning algorithm based on optimization principle by combining the Riccati feedback control with the typical ILC feed-forward control. The scheme has the advantage of automatic determination of step-size and hence guarantees exponential convergence. Gao et al. (2001)

2

extend the optimal ILC algorithm for applications to general batch processes with uncertain disturbances where exact initial resetting is not available. Lee and co-workers (Chae et al., 2000; Chin et al., 2000; Lee et al., 1999; Lee et al., 2000) propose the Quadratic criterion-based ILC approach for tracking control for temperature of batch processes based on a linear time-varying (LTV) tracking error transition model. Xiong and Zhang (2003; 2004) use ILC for the tracking control of product quality in batch processes based on an LTV perturbation model and the convergence of tracking error is guaranteed. To address the problem of model-plant mismatches, the model prediction errors in the previous batch run are added directly to the model predictions for the current batch run. Since ILC is well developed for linear models, most of the ILC based batch-to-batch control schemes are based on linear models. Nonlinear model based ILC schemes for batch processes have also recently been proposed (Xiong and Zhang, 2005b; Zhang, 2003). Xiong and Zhang (2005b) present a recurrent neural network based ILC scheme for batch processes where filtered recurrent neural network prediction errors from previous batches are added to the model predictions for the current batch and optimisation is performed based on the updated predictions. Zhang (2003) presents a neural network model based ILC scheme where the neural network model is linearised around the current batch. Based on this linearised model, the control policy for the next batch is modified using ILC to minimise the control errors at the end of the next batch. The procedure is repeated from batch to batch.

One limitation of batch-to-batch control is that it can only improve the operations of the future batches. When a disturbance occurs in the current batch, batch-to-batch control can only reduce the detrimental effect of the disturbance from the next batch and not on the current batch. To improve the operation of the current batch, within batch re-optimisation control should be utilised. This paper presents an integrated control approach to batch process control by combining batch-to-batch control and within batch re-optimisation control. In this approach, the effect of disturbance is estimated using mid-batch process measurements and control actions for the remaining batch period are re-optimised.

The paper is organised as follows. Section 2 presents neural network modelling of batch processes. A neural network model based integrated control scheme combining batch-to-batch control and within batch re-optimisation control is presented in Section 3. Section 4 presents an application to a simulated batch polymerisation reactor. The last section concludes this paper.

2. Neural Network Modelling of Batch Processes

In general, a neural network based dynamic model for a batch process is typically of the following form (Tian et al., 2001; Xiong and Zhang, 2005b): y(t) = f[y(t-1), y(t-2), …, y(t-n), u(t-1), u(t-2), …, u(t-m)]

3

(1)

where y(t) is the product quality variable at time t, u(t) is the control action at time t, m and n are the time lags. However, in many batch processes only a very limited number of samples of product quality variables are taken during a batch. In such cases, it would be difficult to build a neural network model given by Eq(1) though spline based interpolation can be used to obtain extra estimates of product quality measurements (Tian et al., 2001). In order to overcome this difficulty, the following neural network model is used to model product quality variables at several fixed time points of a batch, including the batch ending point.

Y = f(X0, U)

(2)

where Y = [y(t1) y(t2) ... y(tf)]T is a matrix of product quality variables (or a vector of product quality variable) at times t1 to tf, with tf being the batch end time, X0 is the batch process initial condition, U = [u1 u2 … uN]T is a vector of control actions, and f() is a nonlinear function represented using a neural network with the network weights trained using process operational data. The model given by Eq(2) is basically a static model and can be represented by feed forward type of neural networks, such as multilayer feed forward network or radial basis function networks.

Unknown disturbances, such as reactive impurities and raw material variations, usually exist in batch processes (Zhang et al., 1999). These disturbances can usually be represented as variations in the effective initial condition X0. An inverse model can be developed to estimate the effect of unknown disturbances from the process measurements during the initial stage of the batch process (Zhang et al., 1999). Here a neural network is used to build such an inverse model.

Let YM be a vector of process measurements taken during the time interval [0, M] (i.e. the early stage of a batch) and UM be a vector of control actions during the time interval [0, M], then the effective initial process condition, X0e, can be estimated as X0e = g(UM, YM)

(3)

where the nonlinear function g( ) is represented by a neural network that can be trained using historical process operational data.

3. Integrated Batch-to-Batch Control and within Batch Re-optimisation Control

Based on the neural network model Eq(2), the optimal control policy U can be obtained by solving the following optimisation problem

4

min J[Y(tf)]

(4)

U s.t. product quality and process constraints

where J[Y(tf)] is an objective function in terms of the final product quality. The optimisation problem is usually solved using a nonlinear programming method such as the sequential quadratic programming method (Støren and Hertzberg, 1995). However, model plant mismatch is unavoidable and the optimal control policy calculated based on a neural network model may not be optimal when applied to the actual process (Zhang, 2004a). In order to limit the deterioration of control performance due to model plant mismatches and unknown disturbances, a neural network model based batch-to-batch optimal control strategy using ILC was previously developed by the author (Zhang, 2003; 2004b). This control strategy utilises the information of the current and previous batch runs to enhance the operation of the next batch.

The first order Taylor series expansion of Eq(2) around a nominal control profile can be expressed as

yˆ (t f )  f 0 

f f f u1  u 2    u N u1 u 2 u N

(5)

For the kth batch, the actual product quality can be written as the model prediction plus an error term

y k (t f )  yˆ k (t f )  ek

(6)

where yk(tf) and yˆ k (t f ) are the actual and predicted product quality values at the end of a batch, and ek is the model prediction error. The model prediction error ek is typically due to model plant mismatches and unknown disturbances. It is reasonable to assume that, without updating the model and without drastic changes of the batch control profiles, model plant mismatches would remain the same in two consecutive batches. Disturbances in batch processes are typically associated with the variations in raw materials and it is reasonable to assume that the same disturbance presents in a consecutive number of batches. Thus it is reasonable to assume that the model prediction errors of two consecutive batches are approximately the same.

The prediction for the (k+1)th batch can be approximated using the first order Taylor series expansion based on the kth batch

5

yˆ k 1 (t f )  yˆ k (t f ) 

f u1

(u1k 1  u1k )    Uk

f u N

(u Nk 1  u Nk )

(7)

Uk

 yˆ k (t f )  G T (U k )U k 1

 f where Uk+1 = [u1k+1 u2k+1 … uNk+1]T and G T (U k )    u1

Uk

f u 2

Uk

f  u N

T

  . k  U 

After completing the kth batch run, the model prediction error ek is calculated. As stated earlier, it is reasonable to assume that the model prediction error remains the same for the next batch, i.e. ek+1 = ek. Thus the modified prediction of the product quality at the end of the (k+1)th batch can be given as

~ y k 1 (t f )  yˆ k 1 (t f )  ek 1  yˆ k (t f )  G T (U k )U k 1  ek

(8)

Optimal control of the (k+1)th batch can be represented as

min J ~ y k 1 (t f )  y d k 1

U

Set

2 Q

 U k 1

2

(9)

R

J  0 , the optimal control updating can be calculated as U k 1

U k 1  [G(U k )QGT  R]1 G(U k )Q[ y d  yˆ k (t f )  ek ]

(10)

Uk+1 = Uk + Uk+1

(11)

For a neural network model, the gradient of model output with respect to the control Uk, G(Uk), can be calculated analytically. If single hidden layer feed forward neural networks are used, hidden layer neurons use the sigmoidal activation function, and output layer neurons use the linear activation function, then the element on the ith row and jth column of G(Uk), Gij, can be calculated as

Gij 

nh nh yi y Ok  i  W2k ,i Ok (1  Ok )W1 j ,k u j k 1 Ok u j k 1

(12)

where nh is the number of hidden neurons, Ok is the output of the kth hidden neuron, W2k,i is the output layer weight from the kth hidden neuron to the ith output layer neuron, and W1j,k is the output layer weight from the jth input to the kth hidden neuron.

6

However, batch-to-batch control can only improve the performance of future batch runs and cannot improve the performance of the current batch run. Furthermore, batch-to-batch control is not applicable when the disturbance only presents in a single batch not a consecutive number of batches. When an unknown disturbance presents in the current batch run, its effects could be sensed from some mid-batch process measurements. Therefore, using these mid-batch process measurements, it would be possible to infer the effects of unknown disturbances on the initial batch condition. Corrective control actions for the remaining period of batch process operation can therefore be taken so that any detrimental effects of the unknown disturbance on the final product quality can be minimised.

An integrated batch-to-batch control and within batch re-optimisation control strategy for batch processes is proposed here. During the operation of a particular batch, some mid-batch process measurements are taken. Based on these measurements, the effect of disturbances can be estimated using the inverse model Eq(3). The estimated effective initial process condition, X0e, is then compared with the nominal initial process condition, X0. If significant difference exists between the two, then unknown disturbance presents in this batch and the control actions for the remaining batch period, UR, needs to be re-optimised. The modified batch-to-batch prediction taking into account of the estimated disturbances can be expressed as follows.

yˆ k 1 (t f )  yˆ k (t f )  G T (U k )U k 1 

f X 0

( X 0ke1  X 0ke )

(13)

X 0ke

where X0ek is the estimated effective batch initial condition for the kth batch. Control actions for the remaining batch period can then be obtained through solving the following optimisation problem.

min J  yˆ k (t f )  GRT U Rk 1  e~k  y d k 1

U R

2 Q

 U Rk 1

2 R'

(14)

where

f e~k  ek  GMT U Mk 1  X 0

( X 0ke1  X 0ke )

(15)

X 0ke

In Eq(14) and Eq(15), ∆URk+1 is a vector of control actions for the remaining batch period, GR and R’ are, respectively, these parts of the matrices G(Uk) and R corresponding to the remaining batch period, ∆UMk+1 is a vector of control actions for the batch period [0, M], and GM is that part of G(Uk) corresponding to the batch period [0, M].

7

The optimal control action from Eq(14) can be obtained as:

U Rk 1  (GR Q' GRT  R' ) 1 GR Q'[ y d  yˆ (t f )  e~k ]

(16)

URk+1 = URk + URk+1

(17)

Although the approximation accuracy of Eq(7) can affect the performance and convergence of the proposed control strategy, the updating of model prediction using the prediction error at the current batch in Eq(8) eliminates the prediction error from batch to batch. As will be shown in the results of the next section, the approximation error drops quickly with the batch numbers and the control error normally reaches a stable level after just a few batches.

4. Application to a Simulated Batch Polymerisation Reactor 4.1 A batch polymerisation reactor

The batch polymerisation reactor studied in this paper is a simulated process based on a pilot scale polymerisation reactor developed in the Department of Chemical Engineering, Aristotle University of Thessaloniki, Greece. The batch polymerisation reactor is shown in Figure 1. The free-radical solution polymerisation of MMA is considered in this paper. The solvent used is water and the initiator used is benzoyl peroxide. The jacketed reactor is provided with a stirrer for thorough mixing of the reactants. By circulating water at appropriate temperature through the reactor jacket, heating and cooling of the reaction mixture is achieved. The reactor temperature is controlled by a cascade control system consisting of a primary PID and two secondary PID controllers. The reactor temperature is fed back to the primary controller whose output is taken as the set-point of the two secondary controllers. The manipulated variables for the two secondary controllers are hot and cold water flow rates. The hot and cold water streams are mixed before entering the reactor jacket and provide heating or cooling for the reactor. The jacket outlet temperature is fed back to the two secondary controllers.

A general description of the reactions during the free radical solution polymerisation of MMA initiated by benzoyl peroxide is as follows: Initiator decomposition d I   2 R0

k

Initiation

R0  M i  R1 k

Propagation kp

Rx  M  Rx 1 8

Transfer to monomer m Rx  M   Px  R1

k

Transfer to solvent

Rx  S s  Px  R1 k

Termination by disproportionation td Rx  Ry   Px  Py

k

Termination by combination tc Rx  Ry   Px  y

k

In the polymerisation process, initiator I is decomposed into an initiator radical R0. The initiator radical R0 reacts with monomer M and a radical R1 of length 1 is generated. Monomer M is added onto the end of the radical Rx of length x, forming a new radical Rx+1 of length x+1. The chains of radical Rx is transferred to monomer M and solvent S, forming dead polymers Px and radicals R1 of length 1. Termination by disproportionation generates polymers Px and Py, while termination by combination generates polymers Px+y. A detailed mathematical model covering reaction kinetics and heat and mass balances was developed and validated using the real reactor operation data (Achilias and Kiparissides, 1992; Penlidis et al., 1992). Based on this model, a rigorous simulation programme was developed and used to generate polymerisation data under different batch operating conditions. These data were used to build and validate neural network based inferential estimation models.

4.2 Modelling of the batch polymerisation reactor using neural network

Since the number average and weight average molecular weights are usually difficult to measure, it is typical that only a small number of samples of the molecular weights are collected during reactor operation. Thus, each batch only provides a limited data set for building a neural network model. When build neural network models for a batch polymerisation reactor, these practical considerations should be taken into account.

In this study, we consider the following modelling and control scheme. The maximum batch time for this reactor is about 180 minutes. Samples of the monomer conversion and the number average and weight average molecular weights are collected from 60 minutes at a 20 minute interval. Thus during a batch up to 7 samples of molecular weights are collected. The control variables considered here are the initial reactor temperature setpoint for the time interval [0 min, 40 min] and the reactor temperature

9

setpoints at the following time intervals: [40 min, 60 min], [60 min, 80 min], [80 min, 100 min], [100 min, 120 min], [120 min, 140 min], [140 min, 160 min], and [160 min, 180 min]. These reactor temperature setpoints form a control trajectory for the reactor.

A neural network model for predicting polymer quality variables at time tN is then of the following form:

Y(tN) = f[I0, U(tN)]

(18)

where Y(tN) = [X(tN) Mn(tN) Mw(tN)]T U(tN) = [Tsp0 Tsp1 Tsp2 … TspN]T In the above equations, Tsp0 to TspN are the trajectory of reactor temperature setpoints in the time interval [0, tN], X(tN), Mn(tN), and Mw(tN) are the monomer conversion, the number average molecular weight, and the weight average molecular weight at time tN respectively. The time tN in the above model takes one of the following values: 60, 80, 100, 120, 140, 160, and 180 min. When this model is used in optimal control, each of these values is considered as a possible batch ending time and the best result is selected based on the optimisation results. In order to “simulate” the building of neural network models in an industrial environment, 50 batches were simulated with controls generated from Monte-Carlo simulation. The sampled data were corrupted with typical measurement noises. For each possible batch ending time, a neural network model is developed. The neural network contains 10 hidden neurons and the network weights were initialised as random numbers in the range (-0.1, 0.1). The networks were trained using the LevenbergMarquardt optimisation algorithm (Marquardt, 1963) with regularisation. The objective to include a regularisation term is to improve the generalisation capability of the networks. Due to the different magnitudes of the model input and output data, the data for neural network training have to be scaled first. In this study, Mn is scaled down by a factor of 105, Mw is scaled down by a factor of 106, I0 is scaled down by a factor of 3, and the reactor temperature setpoint is scaled through (Tsp - 338)/20. A further 10 batches were simulated to generate a set of unseen data to validate the developed neural network models.

Figures 2 to 4 show the neural network model prediction performance on the unseen batches at 100 min, 120 min, and 140 min respectively. It can be seen that the neural network model predictions are quite good on most of the unseen batches. However, they are not accurate on some batches. This indicates that, although neural network models can model the batch polymerisation process, model-

10

plant mismatches do exist. Hence, it is essential to apply batch-to-batch ILC to overcome the detrimental effects of model-plant mismatches.

A neural network based inverse model for estimating the effective initial initiator mass is also developed (Zhang et al., 1999). This inverse model is of the following form:

I 0  f (Tsp0 , X 15 , X 20 , X 25 , X 30 )

(19)

where I0 is the estimated effective initial initiator mass, X15, X20, X25, and X30 are the monomer conversion measurements at time 15, 20, 25, and 30 minutes from the start of a batch, and f() is a nonlinear function represented by a neural network.

A stacked neural network, containing 30 single hidden layer feedforward neural networks, was used to represent Eq(18). The reason to use stacked neural networks is that they possess better generalization capability than single neural networks (Zhang et al., 1997). To train neural network models for the initial initiator concentration estimation, training data covering various initial batch conditions should be generated. In this study, 40 different batches of polymerization are simulated using initial conditions obtained from Monte-Carlo simulation. In this reactor, the nominal value for the initial initiator mass is 2.5g. In the Monte-Carlo simulation, the initial initiator masses are in the range [0.5g, 2.5g]. A further 10 batches were simulated and the resulting data serve as unseen testing data. Data for building neural network models were re-sampled through bootstrap re-sampling with replacement (Efron, 1982) to form 30 different data sets. For each re-sampled data set, 60% of the data were randomly selected as training data and the remaining severs as testing data. A neural network model is then developed for each re-sampled data set. Each networks were trained using the LevenbergMarquardt optimization algorithm together with an “early stopping” mechanism. Network weights were initialized as random numbers uniformly distributed in the range (-0.1, 0.1). The number of hidden neurons is determined by considering a number of networks with hidden neurons from 5 to 25 and selecting the one giving the least errors on the testing data. The individual networks were then combined together through principal component regression (Zhang et al., 1997). Figure 5 shows the actual and estimated amount of reactive impurities on the unseen testing data. It can be seen that the estimation performance is quite satisfactory.

4.3 Integrated batch-to-batch iterative optimal control and within batch re-optimisation control In this study, the desired final product quality is selected as yd = [2×105 5×105 1]T, which represents that the desired Mn, Mw, and X are 2×105 g/mol, 5×105 g/mol, and 1 respectively. The weighting

11

matrix Q is selected as Q = [2×10-5 8×10-6 4]T, and the weighting matrix R is selected as R = [0.1 0.1 0.1 0.1]T. The following constraint is also considered: reactor temperature is bounded between 320 K and 360 K.

Based on the neural network model, optimal control policy was first calculated off-line. Since the neural network model can predict polymer quality variables at 60, 80, 100, 120, 140, 160, and 180 minutes, each of these time instances is considered as a possible batch ending time during optimisation and the one leads to the best performance is selected. In this study, a batch ending time of 100 minutes gives the best performance. When this optimal control policy is applied to the process (i.e. mechanistic model simulation), the final product quality variables are: Mn(tf) = 1.3898×105 g/mol, Mw(tf) = 5.1655×105 g/mol, and X(tf) = 0.8924. It can be seen that there exist significant differences between the actual performance and desired performance. This is due to model plant mismatches of the neural network model.

With the implementation of the proposed batch-to-batch optimal control method, it can be seen from Figure 6 that the control performance is significantly improved in the 2nd batch and further improved in the 3rd batch. After the 3rd batch, the control performance is still improving from batch to batch, but the magnitude of improving is not very significant indicating the batch-to-batch optimal control strategy has nearly converged. The final product quality variables at the 10th batch are: Mn(tf) = 1.8735×105 g/mol, Mw(tf) = 4.9134×105 g/mol, and X(tf) = 0.8432. This demonstrates that the proposed batch-to-batch optimisation technique can effectively overcome the problem of model plant mismatches. Figure 7 shows the final product quality variable variations at different batches while Figure 8 shows the optimal control policies during iterative optimisation.

To test the performance of the proposed control strategy under unknown disturbances, the amount of initial initiator was reduced from its nominal value of 2.5 g to 1.9 g from the 11 th batch forward to simulate the presence of 0.6 g of reactive impurities. Reactive impurities commonly exist in industrial polymerisation processes (Kiparissides, 1996; Penlidis et al., 1988; Zhang et al., 1999). Reactive impurities in polymerisation processes are typically traces of inhibitors or oxygen. The effect of reactive impurities can usually be represented by a step decrease in the initial initiator concentration (Penlidis et al., 1988; Zhang et al., 1999).

Due to the presence of this unknown amount of reactive impurities, the optimal control profile at the 10th batch is no longer optimal and the sum of squared control error increased sharply at the 11 th batch as shown in Figure 6. The final product quality variables at the 11th batch are: Mn(tf) = 2.4316×105 g/mol, Mw(tf) = 8.7177×105 g/mol, and X(tf) = 0.8462, which are much worse than those at the 10th batch. With the implementation of the batch-to-batch optimal control strategy, the control performance

12

is significantly improved in the 12th and 13th batches. At the 13th batch, the batch-to-batch optimal control almost converged with the final product quality variables as Mn(tf) = 1.9959×105 g/mol, Mw(tf) = 6.0527×105 g/mol, and X(tf) = 0.8842. It can be seen from Figures 6 and 7 that batch-to-batch control can only improve process operation from the 12th batch onward. It cannot improve the operation of the 11th batch where the disturbance first occurred. Using the proposed integrated batch-to-batch control and within batch re-optimisation control, the performance of the 11th batch can also be improved as demonstrated in Figure 9. Comparing Figures 6 and 9, it can be seen that the sum of squared control errors at the 11th batch under the integrated control strategy is much lower than that under the batch-to-batch control. Figure 10 shows the control policies at different batches. Comparing Figure 10 with Figure 8, it can be seen the control policies from the batch-to-batch control scheme and integrated control scheme for the 11th to the 20th batches are different. Figure 11 shows the final product quality variables under the integrated batch-to-batch control and within batch re-optimisation control. The final product quality variables at the 11th batch under the integrated control scheme are: Mn(tf) = 2.8422×105 g/mol, Mw(tf) = 8.2861×105 g/mol, and X(tf) = 0.8898, which are overall better than those under batch-to-batch control only.

5. Conclusions

An integrated batch-to-batch iterative optimal control and within batch re-optimisation control strategy based neural network models is proposed in this paper. In order to avoid the difficulty in developing detailed mechanistic models, neural networks are used to model batch processes from process operational data. Although the developed neural network models can be used to calculate optimal control policies, the obtained “optimal” control policies may not be optimal when applied to the actual processes due to model plant mismatches and the presence of unknown disturbances. The repetitive nature of batch processes allows the information of previous batch runs being used to update the control profile of the current batch. Linearization of the neural network model is used in calculating the optimal control profile updating. However, batch-to-batch control can only improve future control performance and cannot improve the performance for the current batch. In order to improve the performance of the current batch, within batch re-optimisation has to be used. The effect of disturbance on the batch process is estimated using mid-batch process measurements and control actions for the remaining batch period are re-optimised taking into account the estimated effect of disturbance. The proposed integrated control scheme combines the advantages of both batch-to-batch control and within batch re-optimisation control. Application of the proposed integrated control scheme to a simulated batch polymerisation process demonstrates that the proposed method can

13

effectively overcome the problems of model plant mismatches and unknown disturbances. The proposed technique could also be applied to molecular weight distribution control in polymerisation processes.

Acknowledgement The work was supported by the UK EPSRC through the Grant GR/N13319 – “Nonlinear Optimising Control in Agile Batch Manufacturing” and GR/R10875 – “Intelligent Computer Integrated Batch Manufacturing”. The author thanks Prof. C. Kiparissides of Aristotle University of Thessaloniki, Greece, for providing the polymerisation simulation program.

References Achilias, D., C., Kiparissides, “Development of a general mathematical framework for modelling diffusion-controlled free-radical polymerisation reactions”, Macromolecules 1992, 25, 3739-3750. Amann, N., D. H. Owens, E. Rogers, “Iterative learning control for discrete-time system with exponential rate of convergence”, IEE Proc. D -Control Theory Applications 1996, 143, 217-224. Bhat, N. V. and T. J. McAvoy, “Use of neural nets for dynamical modelling and control of chemical process systems”, Computers and Chemical Engineering, 1990, 14, 573-583. Bonvin, D. “Optimal operation of batch reactors: a personal view”, Journal of Process Control, 1998, 8, 355-368. Bulsari, A. B., (Ed), Computer-Aided Chemical Engineering, Vol.6, Neural Networks for Chemical Engineers, Elsevier: Amsterdam, 1995. Campbell, W. J., S. K. Firth, A. J. Toprac, T. F. Edgar, “A comparison of run-to-run control algorithms”, In Proceedings of the 2002 American Control Conference; IEEE: New Jerse, 2002; 21502155. Chae, C. D., S. I. Chin, S. K. Lee, H. Rho, H. Rhee, J. H. Lee, “Integrated quality tracking control of a batch PMMA reactor using QBMPC technique”, Comput. Chem. Eng. 2000, 24, 953-958. Chin, I. S., K. S. Lee, J. H. Lee, “A technique for integrated quality control, profile control, and constraint handling for batch processes”, Ind. Eng. Chem. Res. 2000, 39, 693-705.

14

Cybenko, G., “Approximation by superposition of a sigmoidal function”, Math. Control Signal Systems, 1989, 2, 303-314. Efron, B., The Jackknife, the Bootstrap and Other Resampling Plans, Society for Industrial and Applied Mathematics, Philadelphia, 1982. Gao, F., Y. Yang, C. Shao, “Robust iterative learning control with applications to injection molding process”, Chemical Engineering Science, 2001, 56, 7025-7034. Girosi, F. and T. Poggio, “Networks and the best approximation property”, Biological Cybernetics, 1990, 63, 169-179. Kiparissides, C., “Polymerisation reactor modelling: A review of recent developments and future directions”, Chemical Engineering Science, 1996, 51, 1637-1659. Lee, K. S., I. S. Chin, H. J. Lee, J. H. Lee, “Model predictive control technique combined with iterative learning control for batch processes”, AIChE J. 1999, 45, 2175-2187. Lee, J. H., K. S. Lee, W. C. Kim, “Model-based iterative learning control with a quadratic criterion for time-varying linear systems”, Automatica 2000, 36, 641-657. Marquardt, D., “An algorithm for Least Squares Estimation of Nonlinear Parameters”, SIAM J. Appl. Math., 1963, 11, 431-441. Narendra, K. S. and K. Parthasarathy, “Identification and control of dynamical systems using neural networks”, IEEE Transactions on Neural Networks, 1990, 1, 4-27. Park, S. and W. F. Ramirez, “Optimal production of secreted protein in fed-batch reactors”, AIChE Journal, 1988, 34, 1550-1558. Park, J. and I. W. Sandberg, “Universal approximation using radial basis function networks”, Neural Computation, 1991, 3, 246-257. Penlidis, A, J. F. MacGregor, A. E. Hamielec, “Effect of impurities on emulsion polymerisation: case I Kinetics”, Journal of Applied Polymer Sciences, 1988, 35, 2023-2038. Penlidis, A., S. R. Ponnuswamy, C. Kiparissides, K. F. O’Driscoll, “Polymer reaction engineering: modelling considerations for control studies”, Chemical Engineering Journal 1992, 50, 95-107.

15

Ruppen, D., C. Benthack, and D. Bonvin, “Optimization of batch reactror operation under parametric uncertainty – computational aspects”, Journal of Process Control, 1995, 5, 235-240. Støren, S. and T. Hertzberg, “The sequential linear quadratic programming algorithm for solving dynamic optimization problems – a review”, Computers & Chemical Engineering, 1995, 19, s495s500. Su, H. T., T. J. McAvoy, and P. Werbos, “Long-term prediction of chemical process using recurrent neural networks: a parallel training approach”, Ind. Eng. Chem. Res., 1992, 31, 1338-1352. Thomas, I. M. and C. Kiparissides, “Computation of the near-optimal temperature and initiator policies for a batch polymerisation reactor”, Canadian Journal of Chemical Engineering, 1984, 62, 284-291. Tian, Y., J. Zhang, A. J. Morris, “Modeling and optimal control of a batch polymerization reactor using a hybrid stacked recurrent neural network model”, Ind. Eng. Chem. Res., 2001, 40, 4525-4535. Tian, Y., J. Zhang, and A. J. Morris, “Optimal Control of a Fed-Batch Bioreactor based upon an Augmented Recurrent Neural Network Models”, Neurocomputing, 2002, 48, 919-936. Tian, Y., J. Zhang, and A. J. Morris, “Dynamic on-line re-optimisation control of a batch MMA polymerisation reactor using hybrid neural network models”, Chemical Engineering and Technology, 2004, 27, 1030-1038. Xiong, Z. and J. Zhang, “Product quality trajectory tracking in batch processes using iterative learning control based on time-varying perturbation models”, Ind. Eng. Chem. Res., 2003, 42, 6802-6814. Xiong, Z. and J. Zhang, “Batch-to-batch optimal control of non-linear batch processes based on incrementally updated models”, IEE Proceedings - Control Theory and Applications, 2004, 151, 158165. Xiong Z. and J. Zhang, “Neural network model based on-line re-optimisation control of fed-batch processes using a modified iterative dynamic programming algorithm”, Chemical Engineering and Processing, 2005a, 44, 477-484. Xiong Z. and J. Zhang, “Batch-to-Batch Iterative Optimisation Control based on Recurrent Neural Network Models”, Journal of Process Control, 2005b, 15, 11-21. Zhang, J., E.B. Martin, A.J. Morris, & C. Kiparissides, “Inferential estimation of polymer quality using stacked neural networks”, Computers & Chemical Engineering, 1997, 21, s1025-s1030. 16

Zhang, J, A. J. Morris, E. B. Martin, and C. Kiparissides, “Estimation of impurity and fouling in batch polymerisation reactors through the application of neural networks”, Computers and Chemical Engineering, 1999, 23, 301-314. Zhang, J, “Neural network model based batch-to-batch optimal control”, Proceedings of The 2003 IEEE International Symposium on Intelligent Control, Houston, Texas, U.S.A., 5 - 8 October, 2003, 352-357. Zhang, J, “A reliable neural network model based optimal control strategy for a batch polymerisation reactor”, Ind. Eng. Chem. Res., 2004a, 43, 1030-1038. Zhang, J, “Integrated batch-to-batch control and within batch control of batch processes using neural network models”, Proceedings of The 2004 IEEE International Symposium on Intelligent Control, Taipei, Taiwan, 2 - 4 September, 2004b, 114-119.

17

Tsp TT TC

TC

TT

TC

CW HW

Figure 1. A batch polymerisation reactor

Figure 2. Actual and neural network model predicted product quality variables at 100 min on the unseen batches

18

Figure 3. Actual and neural network model predicted product quality variables at 120 min on the unseen batches

Figure 4. Actual and neural network model predicted product quality variables at 140 min on the unseen batches

19

Figure 5. Impurities estimation on unseen testing batches

Figure 6. Sum of squared control errors under batch-to-batch control

20

Figure 7. Product quality variables under batch-to-batch control

Figure 8. Control policies under batch-to-batch control 21

Figure 9. Sum of squared control errors under the integrated batch-to-batch control and within batch re-optimisation control

Figure 10. Control policies under the integrated batch-to-batch control and within batch reoptimisation control

22

Figure 11. Product quality variables under the integrated batch-to-batch control and within batch reoptimisation control

23