Application of End-Point Control and Trajectory Tracking to Batch ...

11th International Symposium on Computer Applications in Biotechnology Leuven, Belgium, July 7-9, 2010

Application of End-Point Control and Trajectory Tracking to Batch Processes H. Lin*. B. Lennox **. O. Marjanovic***. * School of Electrical and Electronic Engineering, The University of Manchester, UK (e-mail: [email protected]). ** School of Electrical and Electronic Engineering, The University of Manchester, UK (e-mail: [email protected]). *** School of Electrical and Electronic Engineering, The University of Manchester, UK (e-mail: [email protected]). Abstract: Batch processing has grown in importance in recent years as the interest in producing high value-added materials has increased. To help ensure stringent product quality metrics are met, batch processes have become increasingly instrumented. Advanced instruments, such as NIR, Raman and other spectral devices are now common place in the pharmaceutical industry. In this paper it is shown how measurements from spectral devices can be used within a feedback control framework to improve product quality. Two specific control systems are applied; one is used to ensure that the final spectral measurement from the end-point of the batch meets target specifications, while the other regulates features within the spectral measurement throughout the batch. The relative merits of each of the proposed control systems are demonstrated through their application to a simulated chemical batch reactor. The case study shows that the end-point controller is able to track set point changes but cannot maintain product quality in the presence of a disturbance. In contrast the trajectory tracking controller is able to reject disturbances but cannot be implemented to track unseen set-point changes. Keywords: Batch Process, End-Point Control, Near Infrared Spectroscopy, Model Predictive Control 

1. INTRODUCTION Batch process control is becoming increasingly important in the manufacturing industry and particularly in the production of low-volume, high-value added products. In the past few decades, batch process control techniques have been actively researched in various fields, including fine and specialty chemicals (Faggian et al., 2009), food (Alonso et al., 1993), pharmaceuticals (Fujiwara et al, 2005), biochemicals (Sousa and Almeida, 2001) and polymers (Tomasz and Wayne, 2009). Batch processing introduces many difficulties not encountered in continuous production: the processing of raw materials into the end product takes place over a finite duration and there are rarely steady states conditions; process dynamics are typically time-varying and non-linear; quality measurements are often only available at the end of the batch (Lennox et al, 2001). Quality control of batch processes is usually implemented by regulating several process variables, such as temperature and pH, with the hope that keeping these variables fixed will ensure consistent end-point product quality. Unfortunately, variation in raw material properties means that this style of operation may not produce consistent product. Advanced control methods have been shown to improve product consistency. Kravaris et al. (1989), for example, used globally linearizing control (GLC) for trajectory tracking. Clarke-Pringle and MacGregor (1997) used a nonlinear adaptive controller incorporating an extended Kalman filter 978-3-902661-70-8/10/$20.00 © 2010 IFAC

to provide temperature control in a batch reactor. Wang et al. (1995) applied adaptive control, together with an extended Kalman filter, to a simulated batch styrene polymerization reactor to track a specified output in the presence of model uncertainty. Aziz et al. (2000) applied generic model control (GMC) coupled with a neural network, which estimated heatrelease to track an optimal temperature profiles. In addition to regulating process variable measurements, there have been several studies in to optimizing process variable trajectories in batch processes. In these applications, the trajectories of process variables, such as temperature, were optimized at various decision points during the batch in an attempt to maintain final product quality. Crowley and Choi (1998), for example, controlled the polymer weight and chain length distribution by computing an optimal discrete sequence of reactor temperature set-points using sequential quadratic programming. Ruppen et al. (1998) successfully applied an on-line optimization strategy to the acetoacetylation of pyrrole with diketene in a laboratory-scale reactor. However, Lin et al. (2009) demonstrated that even in a simple chemical batch reactor, maintaining important process variables, such as reactor temperature at their set-points does not guarantee that the final product will meet its specifications. Changes in reaction rates, for example, caused by differing raw material properties can alter the reaction pathways and affect final product quality. Lin et al. (2009) went on to show that Near Infra Red (NIR) spectral measurements could be used to provide an indication of

407

10.3182/20100707-3-BE-2012.0051

CAB 2010 Leuven, Belgium, July 7-9, 2010

product quality during a batch and that this information could be incorporated into a feedback controller to provide accurate control of end-point product quality despite changes in reaction rates. To address the issue of the absence of quality measurements on a typical batch process, NIR instruments are used with growing frequency in batch processing. In some situations they can be used to provide an inference of a particular product quality. However, there are many processes where it may not be possible to directly relate the spectral measurements to a specific product quality measurement. In such cases, it is anticipated that by maintaining the spectral measurements, across the full range of frequencies, or wavenumbers, consistent, then the final product quality measurements will also maintain their consistency. Controlling the full spectrum measured by an instrument, such as an NIR device, is far from trivial as there are typically several hundred wavenumbers associated with a spectral measurement and with only one or two manipulated variables in a typical batch process, the control problem becomes intractable. To resolve this problem, Lin et al. (2009) applied Principal Component Analysis (PCA) to the spectral measurement and then regulated the primary score through the batch. Whilst this led to good quality control, there may be significant information lost in the discarded scores and product consistency may not be guaranteed. In this paper the trajectories of the process variables are manipulated to ensure that the end-point NIR measurement, across all wavenumbers is controlled. End-point control methods have been applied in several studies. Yabuki et al. (2002), for example, used simple empirical regression models to predict the final product properties. If the prediction then fell outside of a no-control region, a midcourse correction was made to bring the end product quality back to the target using online and offline measurements available up to the midpoint of the process. Flores-Cerrillo and MacGregor (2004) proposed a method to control the complete quality variable trajectory by controlling the process in the reduced space of a latent variable model rather than in the real space of the manipulated variables. At various decision points during the batch, the quality of the end product was predicted using online and offline process variables measurements and corrective action was made if necessary to guarantee the quality of the end product. This end-point controller was successfully applied to regulate a simulated batch process. In this paper an end-point controller using NIR measurements is applied to a simulated batch process and its performance compared with the score based controller developed by Lin et al. (2009). The paper is organised as follows. Section 2 provides some preliminaries of the mathematical techniques applied. In Section 3, the simulation used in this study is introduced . Section 4 describes the control methodologies used in this study. Results and discussions are provided in Section 4 and finally, conclusions are made in Section 5.

2. PRELIMINARIES In this section, the general concepts of model predictive control (MPC), principal component analysis (PCA), and partial least squares (PLS) are briefly introduced to facilitate the understanding of the algorithms used in this paper. 2.1 Model Predictive Control (MPC) MPC (Maciejowski, 2002) refers to a class of control algorithms that utilise an explicit process model to predict the future response of a plant. At each sampling instant, the MPC algorithm attempts to optimise future process behaviour by computing a sequence of adjustments that should be made to the manipulated variables. The first input in the optimal sequence is then implemented, and the entire calculation is repeated at the next sampling instant. The key component of the MPC controller is a prediction model used to forecast future process behaviour. In this paper the following ARX structure (auto regressive with exogenous inputs) is chosen for the prediction model: ny

nu

i1

j 1

y(k)  ai y(k  i)  bj u(k  j)  e(k)

(1)

where y (k ) and u (k ) are the controlled and manipulated variable, respectively, at a sampling instant k . The model error is represented by e(k ) . The order of the ARX model is determined by the values of n y and nu . The cost function employed by MPC to select the appropriate control action is given in Equation (2): p

m

J  yr (k i / k)  yˆ(k i / k) uk  j 1/ k i1

2

2

(2)

j1

Where J is the cost function to be minimized, p and m are the prediction and control horizons, respectively. yr and yˆ are the reference (set-point) values and estimated future output values, respectively,  and  are the weighting parameters for the controlled and manipulated variables, respectively. Finally, u is the change in manipulated variable (incremental control move) that is to be computed by the MPC algorithm. The target of the cost function in Equation (2) is to force the future output to track the reference trajectory over the specified prediction window p , while taking into account the balance between the energy in the error and incremental control moves. 2.2 Principal Component Analysis (PCA) The primary objective of Principal Component Analysis is to capture the majority of variation present in data using a minimal number of composite variables, named principal

408

CAB 2010 Leuven, Belgium, July 7-9, 2010

components (PCs) (Berrar, 2003). This dimensionality reduction is performed by exploiting the inter-dependence between measured process variables, such as the individual wavenumbers represented in a NIR spectra. For the analysis of spectroscopic data, which is required in this study, the power of PCA lies in its ability to condense the correlated information from hundreds of wavenumbers into a small number of mutually orthogonal principal components (PCs). PCA performs the following matrix decomposition:

X  TP T  E

(3)

where X represents measured process data organised in n rows and m columns. PCA decomposes this data matrix into the product of two matrices T and P , as shown in Equation (3). T and P matrices contain, as columns, the so-called PCA scores and PCA loadings, respectively. The E matrix represents the information contained within the matrix X that is not represented in the first nc principal components. Normally, each column of the data matrix X corresponds to a particular process variable, while each row is related to a specific sampling instant in time. In the context of NIR spectra, the columns of X represent specific spectral channels or wavenumbers while the rows contain data related to the whole NIR spectrum measured at a particular instance in time. Due to the fact that the columns of the loadings matrix P are orthogonal, the expression for the calculation of scores is given as:

T  XP

(4)

2.3 Partial Least Squares (PLS) Partial least squares (PLS) (Martens and Næs, 1989) is a method to find a linear regression model by projecting the predicted variable matrix Y and the observed variable matrix X to a latent variable space. The general model for PLS is shown in Equations (5-6).

X  TP T  E Y  TQ T  F

(5) (6)

3. CASE STUDY 3.1 Case Study Description Before describing the control systems applied in this work, it is advantageous to describe the case study that has been used. The specific system that is investigated is a benchmark simulation of a chemical batch reactor (Cott and Macchietto, 1989). The reactions in the reactor are shown below. k1

k2

A  B C ; A  C  D

Where A and B are raw materials, C is the desired product, D is a waste product and k1 and k 2 are the reaction rates. The control objective is to maximize the production of C . The reaction takes place in a reactor which can be heated or cooled by varying the amount of steam or cold water that flows through a jacket surrounding it. NIR measurements are simulated in this study by specifying a random reference spectrum for each component A, B, C and D. The NIR measurement at each sampling instant is then simply a linear combination, based on concentrations, of these individual spectra. 4. CONTROL METHODOLOGIES 4.1 Score-Based MPC Control (Sc-MPC) As described by Lin et al. (2009), the structure of Scorebased MPC (Sc-MPC), for regulating the temperature in a reactor, is shown in Fig. 1. Sc-MPC incorporates a standard cascade controller (TCC) with an additional outer control loop, which is MPC. The structure of the TCC system is shown in Fig. 2. In TCC, the master loop controls the reactor temperature by adjusting the temperature set-point of the jacket. The slave control loop then maintains the jacket temperature by adjusting the steam or coolant flow in to the jacket. For the MPC outer control loop, the manipulated variable is the reactor temperature set point, while the controlled variable is the first score generated from a PCA model of the NIR spectrum. The reference NIR score trajectory is the value of the first PCA score that is obtained from a nominal or golden batch.

where T is the latent variables, P and Q are the loading matrices for X and Y respectively, and E and F are residual matrices. There have been several algorithms proposed for calculating the PLS model parameters. One commonly used method, and that applied in this work, is Non-linear iterative partial least squares (NI-PALS) algorithm (Wold, 1975). A particular challenge when using PCA and PLS is to identify an appropriate number of scores or latent variables. In this work cross validation is used.

(7)

Fig. 1. Basic Structure of the Sc-MPC Control

409

CAB 2010 Leuven, Belgium, July 7-9, 2010

Training data for RLS was obtained using the TCC system structure, as shown in Fig. 2. To excite the process dynamics so that an accurate model of the dynamics could be generated, the reference temperature trajectory was perturbed for three batches by adding a PRBS signal of amplitude 0.1 OC with switching time of 60 seconds. Fig. 2. Structure of Temperature Cascade Control (TCC) System

ARX based prediction models were then identified, using cross validation, with n y  2 and nu  80 .

4.2 End-Point Control

The output signal considered during the model identification is the deviation of the controlled variable from its nominal trajectory. This controlled variable is the first PCA score. A PCA model was developed using NIR spectra collected from a single nominal batch. The first PCA score captured 93.8% of the variation present in the NIR spectra. The loadings vector associated with the first PCA score was then used in real-time to compute the score value from the measured NIR spectra according to Equation (4).

The end-point control method uses the control strategy proposed by Flores-Cerrillo and MacGregor (2004). The basic idea of this strategy is to control the end-product quality by adjusting the trajectories of the manipulated variables. This method requires an empirical model which can use the manipulated variable trajectories to provide an estimate of batch end-point quality. As this process operates in batch mode, it is suggested that this model be identified using multi-way partial least squares (PLS). At certain decision points during the batch, the final product quality is predicted and if the square prediction error between the product quality and desired product quality is large, then control action is taken. Detailed algorithms can be found in Flores-Cerrillo and MacGregor (2004). The structure of end-point control method is shown in Fig. 3. The key difference to Sc-MPC is that end-point control augments the TCC system with an end point controller while Sc-MPC augments the TCC system with a conventional MPC controller. The manipulated variable of the end-point controller is the reactor temperature set-point while the controlled variables are the intensities of NIR spectra over all wavenumbers. The end-point control system adjusts the reactor temperature set point to minimize the difference between the estimated end-point NIR spectrum and the reference spectrum (which is obtained from the reference or golden batch. The controller can either act at every sampling instant and re-compute the trajectories of the manipulated variables, or as suggested by Flores-Cerrillo and MacGregor (2004) a number of decision points are identified during the batch and only at these instances does the controller operate.

5.2 End-Point Control The model for end-point control is identified using PLS. Data was collected from 30 batches, each of which had a PRBS signal applied to the output of the controller. Twenty batches of data were used to identify the PLS model. Cross validation suggested five latent variables be used in this model. The remaining ten batches of data were used to test the accuracy of the PLS model. During this study it was apparent that many more batches of data were required for the model used with the end-point controller. When using multi-way PLS it is necessary to infer future values of the process measurements at each sampling instant during the batch. In this work Single Component Projection (SCP) (Nelson, 1996) was used. 5.3 Comparison of Sc-MPC and End-Point Control The capabilities of Sc-MPC and end-point control were compared in their abilities to control the NIR spectrum at the end-point of the batch during an unmeasured disturbance and also a set-point change. 5.3.1 Test 1: Disturbance Rejection The disturbance considered here is a change in reaction rate

k11 . k11 was decreased by 1.5% at the very start of a batch.

Fig. 3. Basic Structure of End Point Control 5. RESULTS AND DISCUSSIONS 5.1 Score-Based MPC Control (Sc-MPC) The model for Sc-MPC was identified using the Recursive Least Squares (RLS) algorithm (Davis and Vinter, 1985).

The resulting NIR spectra at the end-point, obtained when using the end-point controller and Sc-MPC are compared in Fig. 4. The NIR spectrum obtained by Sc-MPC is similar to the nominal NIR spectra. However, the NIR spectrum obtained using the end-point controller is significantly different from the nominal NIR spectra, illustrating that this controller is unable to reject the disturbance. This result might be expected as there is no real feedback in the endpoint controller to reflect the effect of the disturbance. This could be addressed by using other process variables, such as

410

CAB 2010 Leuven, Belgium, July 7-9, 2010

jacket temperature in the PLS model and this is the subject of on-going work. 0.4 0.3

NIR very well, demonstrating the effectiveness of the endpoint control method for tracking set point changes. Fig. 7 shows that the controller has had to manipulated the reactor temperature set point, albeit slightly, to reach the new operating condition.

Amplitudes

0.2 0.1

0.3

0

0.2

-0.1

0.1

-0.2 0 NIR-endpoint NIR-Sc Nominal NIR

-0.3 -0.4

0

50

100

150 Wavenumbers

200

250

-0.1 300 -0.2

Fig. 4. Comparison of NIR Spectra at Batch End-point

Reference NIR Actual NIR Nominal NIR

-0.3

Fig. 5 shows the reactor temperature for each of the control systems. This figure confirms that the end-point controller makes little change to the reactor temperature trajectory. However, the Sc-MPC increases the temperature to reduce the impact of the disturbance.

-0.4

0

50

100

150

200

250

300

Fig. 6. Comparison of Reference NIR (Set Point), Actual NIR and Nominal NIR 90

90 80 80 70

Amplitudes

Amplitudes

70

60

50

60

50

40 40

20

30

Trsp-endpoint Trsp-Sc Nominal Trsp

30

0

100

200

300

400 Samplings

500

600

700

Trsp Nominal Trsp 20 800

0

100

200

300

400 Samplings

500

600

700

800

Fig. 7. Comparison of Reactor Temperature Set Point

Fig. 5. Comparison of Reactor Temperature Set-Point 5.3.2 Test 2: Set Point Change The performance of the end-point control method in tracking set-point changes is demonstrated in this test. The set-point for the NIR spectrum at the end point of the batch is changed, as shown in Fig. 6. The black line is the nominal NIR spectrum and the blue line is the new set point for the NIR spectrum. A single decision point is used in this test and that point occurs at sampling time 600. Each batch lasted for 750 sampling points. Care needs to be taken with earlier decision points as there is only limited information available from the batch and therefore poor performance may result. Fig. 6 shows the results obtained using the end-point controller. This figure compares the nominal NIR (obtained using the original set-point), the updated reference NIR (the new setpoint) and the NIR achieved using the end-point controller. This figure shows that the actual NIR matches the reference

To apply Sc-MPC to track a set point change, the situation becomes highly complicated. To track a change in the NIR end-point setpoint, a new trajectory for the first score must be determined. If an example trajectory exists, as was the case in the first test, then this new trajectory can be applied. However, if a trajectory is unavailable then it is difficult to compute. The trajectory of the first score is a linear combination of the trajectories of NIR spectra at different wavenumbers. Therefore, finding the trajectories for the NIR spectra at every wavenumber is required which may present difficulties. This subject is the topic of further research.

6. CONCLUSIONS This paper described the application of two different methods for incorporating NIR spectral measurements in a feedback control strategy for batch processes. The first method

411

CAB 2010 Leuven, Belgium, July 7-9, 2010

condensed the information in the NIR measurement using PCA and then regulated a single score. The alternative method controlled the final end-point spectrum only. This latter approach allowed the full spectrum to be regulated, rather than rely on the information contained in a PCA score. Detailed comparison of these methods showed that the endpoint method could track set-points very well. However, it was unable to cope with unmeasured disturbances. This limitation could be addressed by selecting other process variables in the model and this is the subject of on-going research. In contrast it was not possible to employ the score controller for set-point tracking but it was able to reject unmeasured disturbances very well. REFERENCES Alonso, A.A., Perez-Martin, R.I., Shukla, N.V., and Deshpande, P.B. (1993). On-line quality control of nonlinear batch systems: application to the thermal processing of canned foods. Journal of Food Engineering, 19(3), 275-289. Aziz, N., Hussain, M.A., and Mujtaba, I.M. (2000). Performance of different types of controllers in tracking optimal temperature profiles in batch reactors. Computers & Chemical Engineering, 24(2-7), 1069-1075. Berrar, D.P., Dubitzky, W., and Granzow, M. (2003). Singular value decomposition and principal component analysis. A Practical Approach to Microarray Data Analysis, Kluwer: Norwell, MA. Clarke-Pringle, T., and MacGregor, J.F. (1997). Nonlinear adaptive temperature control of multi-product, semibatch polymerization reactors. Computers & Chemical Engineering, 21(12), 1395-1409. Cott, B.J., and Macchietto, S. (1989). Temperature Control of Exothermic Batch Reactors Using Generic Model Control. Industrial & Engineering Chemistry Research, 28(8), 1177-1184. Crowley, T.J., and Choi, K.Y. (1998). Experimental studies on optimal molecular weight distribution control in a batch-free radical polymerization process. Chemical Engineering Science, 53(15), 2769-2790. Davis, M.H.A. and Vinter, R.B. (1985). Stochastic Modelling and Control, Springer. Faggian, A., Franco, P., Doplicher, F., Bezzo, F., and Barolo, M. (2009). Multivariate statistical real-time monitoring of an industrial fed-batch process for the production of specialty chemicals. Chemical Engineering Research and Design, 87(3), 325-334. Flores-Cerrillo, J. and MacGregor, J.F. (2004). Control of batch product quality by trajectory manipulation using latent variable models. Journal of Process Control, 14(5), 539-553. Fujiwara, M., Nagy, Z.K., Chew, J.W., and Braatz, R.D. (2005). First-principles and direct design approaches for the control of pharmaceutical crystallization. Journal of Process Control, 15(5), 493-504. Kravaris, C., Wright, R.A., and Carrier, J.F. (1989). Nonlinear controllers for trajectory tracking in batch processes. Computers & Chemical Engineering, 13(1-2), 73-82.

Lennox, B, Hiden, H.G., Montague, G.A., Kornfeld, G. and Goulding, P.R., (2001), ‘Process monitoring of an industrial fed-batch fermentation’, Biotechnology and Bioengineering, 74 (2), 125-135 Lin, H., Marjanovic, O., Lennox, B., and Shamekh, A. (2009). Application of Near-infrared Spectroscopy in Batch Process Control. International Symposium on Advanced Control of Chemical Processes, Istanbul, Turkey Maciejowski, J. (2002). Predictive Control With Constraints, Addison Wesley Longman. Martens, H. and Næs, T. (1989). Multivariate calibration. Wiley. Nelson, P.R.C., Taylor, P.A., and MacGregor, J.F. (1996). Missing data methods in PCA and PLS: Score calculations with incomplete observations. Chemometrics and Intelligent Laboratory Systems, 35(1), 45-65. Ruppen, D., Bonvin, D., and Rippin, D.W.T. (1998). Implementation of adaptive optimal operation for a semibatch reaction system. Computers & Chemical Engineering, 22(1-2), 185-199. Sousa, R., Almeida, P.I.F. (2001). Design of a fuzzy system for the control of a biochemical reactor in fed-batch culture. Process Biochemistry, 37(5), 461-469. Tomasz, K., Wayne, F.R. (2009). Predictive control and verification of conversion kinetics and polymer molecular weight in semi-batch free radical homopolymer reactions. European Polymer Journal, 45(8), 2288-2303. Wang, Z.L., Pla, F., and Corriou, J.P. (1995). Nonlinear adaptive control of batch styrene polymerization. Chemical Engineering Science, 50(13), 2081-2091. Wold, H. (1975). Path models with latent variables: The NIPALS approach. Academic Press. Yabuki, Y., Nagasawa, T., and MacGregor, J.F. (2002). Industrial experiences with product quality control in semi-batch processes. Computers & Chemical Engineering, 26(2), 205-212.

412