6438
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 61, NO. 11, NOVEMBER 2014
A New Method of Dynamic Latent-Variable Modeling for Process Monitoring Gang Li, S. Joe Qin, Fellow, IEEE, and Donghua Zhou, Senior Member, IEEE
Abstract—Dynamic principal component analysis (DPCA) is widely used in the monitoring of dynamic multivariate processes. In traditional DPCA where a time window is used, the dynamic relations among process variables are implicit and difficult to interpret in terms of variables. To extract explicit latent variables that are dynamically correlated, a dynamic latent-variable model is proposed in this paper. The new structure can improve the modeling and the interpretation of dynamic processes and enhance the performance of monitoring. Fault detection strategies are developed, and contribution analysis is available for the proposed model. The case study on the Tennessee Eastman Process is used to illustrate the effectiveness of the proposed methods. Index Terms—Contribution plots, dynamic latent-variable (DLV) model, dynamic principal component analysis (DPCA), process monitoring and fault diagnosis, subspace identification method.
I. I NTRODUCTION
A
S the complexity of industrial processes grows quickly, the process monitoring problem has become a popular topic in the area of process control [1], [2]. Among different process models, multivariate statistical process monitoring provides a data-driven framework for monitoring the industrial process without accurate physical models, which is convenient for implementation. Principal component analysis (PCA) and partial least squares (PLS) are two of the most widely used models in statistical process monitoring [3]–[6]. Static PCA models are suitable for manufacturing processes where the measured variables are fairly independent over time. However, applying static PCA to dynamically correlated data tends to have unnecessarily high missed detection rates. To deal with autocorrelated measurements, time-series models can be used to generate the residual for monitoring. However, this approach Manuscript received August 31, 2013; revised November 21, 2013; accepted December 25, 2013. Date of publication January 21, 2014; date of current version June 6, 2014. This work was supported in part by the National Basic Research Program (973 Program) under Grant (2010CB731800), in part by the National Natural Science Foundation of China under Grant 61074084, Grant 61074085, Grant 61210012, Grant 61290324, Grant 61273173, Grant 61304107, and Grant 61020106003, in part by the Beijing Key Discipline Development Program under Grant XK100080537, and in part by the Beijing Natural Science Foundation under Grant 4122029. G. Li is with the Department of Chemical Engineering and Materials Science, University of Southern California, Los Angeles, CA 90089 USA (e-mail:
[email protected]). S. J. Qin is with the Department of Chemical Engineering and Materials Science and the Department of Electrical Engineering, University of Southern California, Los Angeles, CA 90089 USA (e-mail:
[email protected]). D. Zhou is with the Department of Automation, Tsinghua University, Beijing 100084, China (e-mail:
[email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TIE.2014.2301761
is often applicable to the process monitoring where there is little cross correlation among variables [7]. For multivariate time series with highly cross-correlated variables, dynamic factor models are preferred for appropriate process monitoring and diagnosis. In the last two decades, dynamic multivariate projections were developed for this objective. Ku et al. proposed a lagged versions of PCA to process multivariate variables with dynamic property [8], which is called the dynamic PCA (DPCA) model. In the DPCA model, a singular value decomposition (SVD) is conducted on an augmented data matrix, which includes lagged measurements within a time window. However, there are limitations for this technique, one of which is that autocorrelations are mixed with cross correlations, making the number of model parameters overly large. The mixing also leads to implicit dynamic relations, which are difficult to interpret. There are also some ideas based on subspace modeling. Negiz and Cinar used a canonical variate state-space model to describe dynamic processes, which is equivalent to a vector autoregressive (AR) moving-average time-series model [9]. They used a stochastic realization based on canonical variate analysis (CVA) to handle a large number of variables that are autocorrelated, cross-correlated, and collinear. The constructed state variables are linear combinations of the past measurements, which can explain future variabilities in the data. As PLS is able to model the relationship between two data sets, the subspace model identified by the PLS algorithm can also be used for statistical performance monitoring [10]. The comparison between CVA and PLS shows that CVA can provide more rapid detection for the faults. However, due to the sensitivity of small eigenvalues of the covariance matrix, PLS is often able to obtain more numerically stable results than the CVA method. The subspace identification method (SIM) identifies a statespace model for the process [11]. PCA can be used to develop a consistent state-space model using a parity space under error-in-variable situations [12]. Li and Qin investigated the relationship between DPCA and SIM under state-space descriptions of dynamic processes [13]. With the presence of process and measurement noise, they proposed a consistent DPCA algorithm, namely, indirect DPCA (IDPCA) with consistency conditions. Recently, Ding et al. combined SIM and model-based fault detection technologies to propose observerbased fault detection scheme [14]. Using the linear dynamic state-space description of processes, the residual generator of the parity space method after identifying a subspace model is equivalent to IDPCA modeling for normal data. Another dynamic probabilistic model termed the linear Gaussian statespace model was proposed for dynamic process monitoring
0278-0046 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
LI et al.: NEW METHOD OF DLV MODELING FOR PROCESS MONITORING
by Wen et al. [15], which uses the expectation–maximization algorithm for the structure selection and the Kalman filter for the innovation estimation. However, these subspace-based methods all ignore cross correlations among variables, which are useful for process monitoring. It is desirable to extract both dynamic autocorrelations and cross correlations among variables to build a compact or parsimonious dynamic factor model for process monitoring. In this paper, a new dynamic latent-variable (DLV) model is proposed to extract autocorrelations and cross correlations explicitly, which gives the dynamic model clear insights. The remainder of this paper is organized as follows. In Section II, existing DPCA and subspace-based methods are reviewed briefly. Then, a new DLV model is proposed in Section III. Fault detection and diagnosis schemes are developed based on the proposed model in Section IV. In Section V, a case study on the Tennessee Eastman Process (TEP) is used to illustrate the effectiveness of the proposed methods. Conclusions are given in Section VI. II. DYNAMIC PCA AND S TATE -S PACE M ODELING Suppose a sample vector xk ∈ Rm consists of m sensor measurements at sampling time k. With the effect of dynamic processes and closed-loop control, measurement samples at different k are not independent, which can be both autocorrelated and cross-correlated. A. Direct DPCA Modeling To capture the dynamic relations inside the variables, DPCA performs a PCA procedure on the following augmented data matrix with k = 0 [8]: ⎤T ⎡ x x ··· x k+q
k+q+1
⎢ xk+q−1 Zk = ⎢ .. ⎣ . xk+1
xk+q .. . xk+2
··· .. . ···
k+n+q−1
xk+n+q−2 ⎥ ⎥ .. ⎦ . xk+n
(1)
wz
wzT ZT0 Z0 wz
s.t. wz = 1.
(2)
The following SVD can be performed, and the largest A singular directions are selected as the dynamic principal components 1 T Z Z0 = U1 D1 V1T . n 0
(3)
Ku et al. suggested a method to determine the order of dynamic process [8]. If measurement noise terms have identical variances for all variables, the direct DPCA method is unbiased. Let ˜ D = U1 (:, A + 1 : mq). Denote zk = PD = U1 (:, 1 : A), P [xTk , . . . , xTk−q+1 ]T ; then, the dynamic principal components and the residual are as follows: tk = PTD zk ˜ DP ˜ T zk . rk = P D
B. IDPCA Modeling If the measurement noise does not have identical variances, the DPCA model is biased. Li and Qin suggested to use an IDPCA algorithm based on instrument variables to eliminate the effect of noise [13]. In their work, a SIM-based IDPCA is ˜ I . Their method performs used for extracting the party vector P SVD as follows: 1 T Z Z0 = U2 D2 V2T n q
(6)
˜ I = U2 (:, A + 1 : mq). Li and Qin and PI = U2 (:, 1 : A), P used the Akaike information criterion (AIC) for choosing the proper lagged number q [13]. The number of dynamic principal factors A is determined as the number of diagonal elements of D2 that are zero or nearest to zero. C. State-Space Modeling In addition to the given IDPCA modeling, some other techniques are used to build a dynamic relationship between a future data set Zq and a past data set Z0 . Negiz and Çinar used the CVA technique to build a state-space model for dynamic process monitoring [9]. It provided the principal directions of a linear dynamic system through the canonical variables, which are combination of the past measurements that are highly correlated to the future measurements. The objective of CVA is to search for correlation in the future and past data sets cTz ZTq Z0 wz
max w, c
s.t. Zq cz = Z0 wz = 1.
(7)
The procedure is to perform the following SVD:
ZTq Zq
−1/2
−1/2 ZTq Z0 ZT0 Z0 = U3 D3 V3T
(8)
and the canonical variate states are obtained by
where q is the lagged number of the data matrix, and n is the number of samples. The objective of DPCA is to find the solution of the following optimization: max
6439
(4) (5)
−1/2
tk = PTc R0
zk
(9)
where Pc = V3 (:, 1 : A) includes singular vectors with respect to the A largest singular value, and R0 = (1/n)ZT0 Z0 means the covariance matrix of zk . The order A can be determined by the AIC [16]. PLS is conceptually similar to CVA in building a relationship between future and past data sets [10], which has the following objective: max w,c
cTz ZTq Z0 wz
s.t. cz = wz = 1
(10)
and the PLS scores are obtained by tk = (WT P)−1 WT zk
(11)
where W and P are the weighting and loading matrices from the PLS algorithm [17].
6440
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 61, NO. 11, NOVEMBER 2014
Taking derivatives with respect to w and β leads to
III. DLV M ODELING The aforementioned dynamic process models extract only the dynamic relations among variables and use them for residual generation. However, there are two problems. On one hand, there exist static and dynamic correlations, which should be both extracted. The existing dynamic modeling methods do not distinguish dynamic relations from static relations. On the other hand, the augmentation of many lagged variables in an extended vector does not retain the structure of the original variable space. To retain the variable space structure in a dynamic model, a structured DPCA algorithm is proposed to extract DLVs from the original data space in this section. Furthermore, a DLV model is used to characterize the dynamics as well as static cross correlations in process variables.
Let Xk = [xq−k , xq−k+1 , . . . , xq−k+n−1 ]T (k = 0, . . . , q − 1) represent the process data with k time lags, where q is the maximum of the lagged number, and n is the number of samples used for training the model. The proposed algorithm maximizes the following objective: max wT β0 XT0 + · · · βq−1 XTq−1 w,β
(β0 X0 + · · · βq−1 Xq−1 ) w s.t. w = 1, β = 1
(12)
where β = [β0 , . . . , βq−1 ]T . βk is the weight coefficient for Xk w. This objective represents the chief dynamic variation from process variable space, by searching the weight vector w and coefficient vector β. This dynamic extension does not increase the dimension of w and characterizes dynamic correlations in a compact latent structure. Notice that Z0 = [X0 , X1 , . . . , Xq−1 ]; the given objective can be rewritten as β ⊗ w)T ZT0 Z0 (β ⊗ w)
w,β
s.t. w = 1, T
β = 1
(13)
T T
J = (β ⊗ w) ⊗ w) + λw (1 − w w) + λβ (1 − β T β). T
Sz ≡ ZT0 Z0 ∈ Rmq×mq Sw ≡ (Iq ⊗ w)T Sz (Iq ⊗ w) ∈ Rq×q Sβ ≡ (β ⊗ Im )T Sz (β ⊗ Im ) ∈ Rm×m .
ZT0 Z0 (β T
(14)
From the properties of Kronecker products, we have the following: β ⊗ w = (Iq ⊗ w)β = (β ⊗ Im )w.
(15)
(17)
(18)
The given necessary conditions can be summarized as Sw β = λβ β Sβ w = λw w.
(19) T
According to (13), by multiplying β and w to both sides in (19), the following result can be obtained when the objective reaches the maxima at optimal solution β opt and wopt : Jmax = λw = λβ .
(20)
However, the optimal solution cannot be calculated directly based on (19) because two eigenvalue decompositions are coupled with each other. To determine the optimal solution, an iterative algorithm is proposed to search w and β by setting an initial value of w. Then, the dynamic latent score vector is calculated as t = Xw.
(21)
After that, the loading vector p can be obtained by p = XT t/tT t
(22)
which generates the residual by deflating the data with the extracted dynamic latent scores, i.e., E = X − tpT .
where β ⊗ w = [β0 w , β1 w , . . . , βq−1 w ] is the Kronecker product. Remark 1: The given objective is reduced to static PCA when q = 1, and the result is the same as the static PCA algorithm. If the number of variables m = 1, the objective is to find the maximum variance of the linear combination of lagged variables. Comparing optimization in (2) and (13), it can be found that the objects are the same except for the restrictions. The optimization (2) has no restriction for the form of wz , whereas optimization (13) uses a predetermined structure of wz = β ⊗ w, which also obeys the norm restriction in (2). To maximize the structured DPCA objective, Lagrange multipliers are used as follows: T
(16)
Define
T
A. Structured Dynamic PCA Algorithm
max
∂J = 2 (Iq ⊗ w)T ZT0 Z0 (Iq ⊗ w)β − λβ β = 0 ∂β
∂J = 2 (β ⊗ Im )T ZT0 Z0 (β ⊗ Im )w − λw w = 0. ∂w
(23)
To sum up, the structured DPCA procedure is listed as follows. Algorithm 1 (Structured Dynamic PCA algorithm): Center the raw data of X to zero mean and scale them to unit variance, and set i = 1, i.e., Xi = X. 1) Set wi = [1, 0, . . . , 0]T , calculate Sw , and solve the following eigenvalue decomposition, and let βi be the eigenvector corresponding to the largest eigenvalue Sw β i = λβ β i .
(24)
2) Calculate Sβ and solve the following problem, and let wi be the eigenvector corresponding to the largest eigenvalue S β wi = λ w wi .
(25)
Return to Step 1 until wi converges. 3) ti = Xi wi 4) pi = XiT ti /tTi ti i+1 i+1 5) Xi+1 = Xi − ti pi T , Zi+1 = [Xi+1 0 0 , X1 , . . . , Xq−1 ], i+1T i+1 Z0 Sz = Z0 Return to Step 1 until i > A.
LI et al.: NEW METHOD OF DLV MODELING FOR PROCESS MONITORING
The convergence of the structured DPCA algorithm is given in the Appendix. As the given algorithm resembles the PLS algorithm in the procedure, the structure on X space provided by the structured DPCA is also similar to PLS. Denote W = [w1 , w2 , . . . , wA ], P = [p1 , p2 , . . . , pA ], T = [t1 , t2 , . . . , tA ], and R = W(PT W)−1 ; then T = XR.
6441
TABLE I M EANING OF D IFFERENT S UBSPACES
(26)
Given a sample of process vector xk , the score and residual are calculated as tk = R T x k (27) ek = (I − PRT )xk .
In addition, for a sample vector xk , the decomposition can be described as xk = Ptk + Ps ts, k + er, k (31) tk = pj=1 αj tk−j + vk .
B. Dynamic Modeling of the Latent Scores
We refer to (31) as the DLV model. The DLV models focus dynamic variation of process in a low-dimensional latent space, which is beneficial for process monitoring. The DLV model captures the dynamic variation first and then extracts dominant static variations. As a result, the DLV model characterizes dynamic and static relationships in the process data explicitly, rather than implicitly as in traditional DPCA. Note that autocorrelations are first extracted before cross correlations are extracted. The geometric interpretation of DLV models is analyzed as follows. Lemma 1: The DLV model induces an oblique decomposition of the X-space as follows:
As the latent variables contain most dynamics in the data, it is necessary to build a dynamic model to characterize the autocorrelation inside the latent variables. Assuming processes under the normal operation are stationary, it is reasonable to describe tk with a vector AR model as follows: tk =
p
αj tk−j + vk = ΘT ϕk + vk
(28)
j=1
where vk is assumed to be an i.i.d. random process with zero mean and constant variance, representing the independent driving source of the normal variation; Θ = [α1 , . . . , αp ]T ; ϕk = [tTk−1 , . . . , tTk−p ]T ; and p is the model order. The parameters of the given model can be estimated by a multivariable least squares algorithm directly as follows [18]: ⎛ ⎞−1 ⎛ ⎞ p+n p+n
ˆ =⎝ Θ (29) ϕi ϕTi ⎠ ⎝ ϕi tTi ⎠ . i=p+1
i=p+1
ˆ+x ˆ s + er x =x ˆ = PRT x ∈ SD x ˆ s = Ps PTs (I − PRT )x ∈ Ss x er = I − Ps PTs (I − PRT )x ∈ Sr .
(32)
Lemma 1 is easily proven and is omitted due to page limitations. A similar proof procedure can be found in [19]. The meaning of different subspaces are summarized in Table I.
C. Further Modeling After Extracting Dynamics After the extraction of dynamic factors, ek has very little autocovariance, which is desirable for further extraction of static cross correlations. Denoting E = [e1 , . . . , en ]T , E represents the time-independent variations in the process data after extracting the dynamic latent factors. This leaves E with mostly static cross correlations as the dynamic correlations have been extracted. Thus, it is necessary to decompose E further by the static PCA. The whole procedure of DLV modeling can be summarized as follows. 1) Determine the lagged number q and the number of dynamic factors A. Use the structured DPCA algorithm to extract DLVs, which is represented by (27). 2) Determine the model order p, and use the multivariable LS algorithm to build the inner model of DLVs. 3) Determine the static principal components number As . Perform PCA on residual E as E = Ts Ps + Er , where Ts contain As components. The ultimate space decomposition of X space can be represented as X = TPT + Ts Ps + Er .
(30)
D. Determination of Model Parameters In the DLV model, there are some parameters to be decided. First of all, the lagged number q in (12) should be determined before structured DPCA can be performed. The method used here was first suggested by Ku [8]. Considering that there are r0 static relations in m original variables X0 , there must be 2r0 static relations and r1 dynamic relations in 2m augmented data [X0 , X1 ]; the remainder can be done in the same manner. If, for (q + 1)m augmented data, there are no new relations, i.e., rq = 0, then it can be seen that q is enough for covering all dynamic correlations. If there is no noise, both static and dynamic relations can be easily determined as the right null space of Z0 . However, there is always noise in practice. Thus, the relations can be identified by the smallest singular values of Z0 , which are near zero. Similarly, when a dynamic factor is extracted, the number of dynamic relations will be reduced. After A dynamic factors are extracted, the dynamic relations should be completely removed, . which reflects that there are only static relations in ZA+1 0 When all dynamic factors are extracted, a vector AR model is
6442
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 61, NO. 11, NOVEMBER 2014
TABLE II FAULT D ETECTION I NDICES
TABLE III I TERATION E RRORS AND O PTIMIZATION VALUE FOR E ACH S TEP
adopted to describe the dynamics. Moreover, the model order p is determined by the AIC [18]. Finally, the static PCA model is built on the residual after the structured DPCA. The number of components in the static PCA model is determined by the cumulative percentage of variance (CPV) [20]. IV. FAULT D ETECTION AND D IAGNOSIS BASED ON DLV Conventional PCA-based fault detection uses squared prediction error and Hotelling’s T 2 control charts for process monitoring. For the DLV-model-based process monitoring, scores and residuals are calculated first as follows: tk = R T x k p
v k = tk − αj tk−j j=1 T T ts, k = P )xk s (I − PR T er, k = I − Ps Ps (I −
PRT )xk
(33)
where tk and vk represent the score and residual in the dynamic process variations, and ts, k and er, k represent the score and residual in static process variations. Note that tk is not a time-independent series, which is not proper for monitoring. Therefore, three indexes are constructed to monitor the process with the DLV model, which are listed in Table II. There are explicit meanings of these statistics. Tv2 monitors the innovation process of dynamic factors, which is statistically independent of historical dynamic variation. On the other hand, Ts2 reflects the variation of static principal components, whereas Qr measures the noise level and modeling error, including unmodeled dynamics. As Ts2 and Qr have a quadratic form of xk , which can lead to contribution analysis directly. Motivated by [21], reconstruction based contribution (RBC) for variable j and sample xk is defined as T 2 ξj Φxk (34) RBCj, k = ξjT Φξj where ξj is the jth column of the identity matrix Im , and T T 2 Φ = (I − PRT )T (Ps Λ−1 s Ps )(I − PR ) for Ts and Φ = T T T T (I − PR ) (I − Ps Ps )(I − PR ) for Qr . V. C ASE S TUDY ON THE TEP Here, the effectiveness of the proposed method for process monitoring and fault diagnosis is investigated by a case study
on the TEP. The TEP was created to provide a benchmark for evaluating process control and monitoring approaches, including PCA, PLS, and Fisher discriminant analysis (FDA) [22]. Yin et al. compared basic data-driven fault diagnosis techniques based on the benchmark of TE process, including PCA, DPCA, FDA, PLS, total projection to latent structures (T-PLS), modified PLS (MPLS), independent component analysis (ICA), and the subspace-aided approach [23]. The TEP contains 12 manipulated variables and 41 measured variables. In this paper, 22 process measurements and 11 manipulated variables, i.e., XMEAS(1-22) and XMV(1-11), are chosen as X. Other measured variables are concentrations of different components, which are measured with a delay and can be seen as quality variables. There are 15 known faults in TEP, which includes seven step faults, five random faults, and three sticking and slow change faults, as listed in the Table IV. First, 480 normal samples are centered to zero mean and scaled to unit variance. Then, these normal samples are used to determine the parameters and to build the data model. Parallel analysis is used to determine the model parameters q. When the lagged number is 0, there are 20 relations which are all static. When the lagged number increases to 1, the number of new relations increase to 25, where five new dynamic relations are found in Xg , and the other 20 relations are still static as the same as in the previous case. When the lagged number grows to 2, the new relations no longer increase and stabilize around 25, which means there are only five added dynamic relations and the lagged number of 1 is enough for describing the dynamics of the TEP. Then, the structured DPCA algorithm is built with q = 2. Then, parallel analysis is used again for choosing the proper A. After six dynamic factors are extracted from the original data, the relations do not increase for the augmented matrix. Thus, A = 6 is enough for describing the dynamic for TEP. Table III lists the iteration error and the optimization result of Jk for each step in iterations 1–6. From the table, it can be seen that the iteration process converges fast to the optimal value, and iteration steps are not more than 3 in this case. The optimal Jmax for each iteration is always below the upper bound λmax (Sz ), which is consistent with the analysis in the Appendix. The results show that the computational cost of structured DPCA algorithm is considerably acceptable even
LI et al.: NEW METHOD OF DLV MODELING FOR PROCESS MONITORING
Fig. 1.
6443
Autocorrelation coefficients of DLVs (t). Fig. 3. Autocorrelation coefficients of static variables (ts ).
Fig. 2.
Autocorrelation coefficients of innovation variables (v).
for industrial implementation. Following that, the order of the vector AR model is selected as p = 4 using AIC, and As = 16 is decided for static PCA according to CPV. After modeling, the correlation plots are plotted to check the availability of the proposed DLV model. Figs. 1–3 depict the autocorrelations of the first six components of dynamic factors (t), innovations(v), and static factors(ts ), respectively. The results reflect that there are evident autocorrelations and cross correlations in the dynamic factors, and in the innovation part v, the autocorrelations and cross correlations are both removed. Meanwhile, there is no autocorrelation in static component ts , whereas its cross correlation is extracted by a PCA model. To monitor the process with the DLV model, three statistics indexes are developed, as described in Section IV. Based on the test data of the normal case (IDV 0) and faulty data with respect to known faults (IDV 1–15), the fault detection rates (FDR) and false alarm rates (FAR) with DLV model and four existing methods are compared in Table IV. In the PCA model, the number of principal components is selected as 19
according to co-percent variance test, including about 95% variation information. In the DPCA model, q = 2 and A = 35 are selected, which contains 95% variation. In the SIM-based method, q = 2, and A = 16, as suggested in [23]. In the CVA method, q = 2, and A = 29, as suggested in [16]. For multiple indexes in a method, if any statistic exceeds its control limit, the fault is detected. From the FDRs for all faults (except IDV 3, 9, 15), it can be seen that the DLV-based method can detect faults more stably than DPCA and outperforms PCA and SIMbased methods in most cases. Although CVA-based method has a higher FDR, its FAR is also high, which is caused by the inversion of the small values of R0 in (8). However, the DLV model is more convenient for the interpretation of the abnormal situations in the original space. Take IDV(10) as an example. Fig. 4 shows the detection result using DLV-based indexes. When this fault occurs, a random change is added in the temperature of C feed, which is compensated by feedback control. Fig. 5 plots the RBCj to Qr at the 400th sample, where the horizontal axis represents the variable index, and the vertical axis is the contribution. The variables corresponding to high contributions are significant candidates responsible for the fault. It is clear to find that variables 18 and 19 have the highest contributions, which are the measurements of the stripper temperature and the stripper steam flow, respectively. Further, it can be concluded that variables 18 and 19 are the most responsible for the fault. To confirm the fact, we plot the RBC for all samples after the fault is detected in Fig. 6, where the horizontal axis represents the sample index, and the vertical axis is the variable index. The RBC for each sample and each variable is drawn by a color tape, where the darker color represents the higher RBC. From the plot, it is observed that variables 18 and 19 are indeed the most significant candidates for this fault, which is consistent with the conclusions in [15] and [24]. However, traditional dynamic methods such as DPCA, SIM, and CVA do not reveal a structure in the lagged variables, which makes the diagnosis analysis difficult to implement. Moreover, the decomposition structure can be used for further fault estimation and reconstruction.
6444
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 61, NO. 11, NOVEMBER 2014
TABLE IV FAR FOR IDV (0) AND FDR FOR IDV (1–15) W ITH D IFFERENT M ETHODS
Fig. 4. Fault detection result with Td2 , Ts2 , and Qr . Fig. 6. RBC to Qr during the whole process. Variables with high contribution are dark in color.
namic autocorrelations from static cross correlations of process variables. For DLVs, a vector AR model is adopted to construct an innovation-like residual. In addition, a PCA decomposition is applied to extract the static variations from the remaining part. Several statistics are developed based on the DLV model. Td2 and Ts2 are used to monitor the dynamic and static variations, respectively, and Qr is used for detecting the whole residual of the process. The DLV model has a clear interpretation in variable space, which can be used for diagnosis with contribution analysis directly. The case study on the TEP shows clear advantages of the proposed modeling and related methods. A PPENDIX C ONVERGENCE OF THE S TRUCTURED DPCA A LGORITHM Fig. 5. RBC to Qr at 400th sample.
VI. C ONCLUSION In this paper, a new DLV model has been proposed for datadriven modeling of dynamic processes. A structured DPCA algorithm has been also proposed to extract DLVs, which cover the most of dynamic variations. This procedure separates dy-
Denote Jk = J(wk , β k ) as the optimization objective during the k iteration of Steps 1 and 2 in the structured DPCA algorithm. First of all, observing the optimization objective in (2), it can be concluded that a solution is also a feasi of (13) m 2 2 β ( ble solution of (2) as (β ⊗ w)2 = q−1 j=0 j i=1 wi ) = 1. According to (2) 0 < Jk ≤ max wzT Sz wz = λmax (Sz ) wz
(k = 1, 2, . . .) (35)
LI et al.: NEW METHOD OF DLV MODELING FOR PROCESS MONITORING
where λmax (Sz ) denotes the maximum of the eigenvalue of Sz . The given relation reveals that Jk is a bounded sequence. Further, the following relationship holds with Algorithm 1: J(wk , β k ) = (β k ⊗ wk )T Sz (β k ⊗ wk ) ≤ max(β ⊗ wk )T Sz (β ⊗ wk ) β
= (β k+1 ⊗ wk )T Sz (β k+1 ⊗ wk ) ≤ max(β k+1 ⊗ w)T Sz (β k+1 ⊗ w) w
= (β k+1 ⊗ wk+1 )T Sz (β k+1 ⊗ wk+1 ) = J(wk+1 , β k+1 )
(36)
6445
[20] S. Valle, W. Li, and S. Qin, “Selection of the number of principal components: The variance of the reconstruction error criterion with a comparison to other methods?” Ind. Eng. Chem. Res., vol. 38, no. 11, pp. 4389–4401, 1999. [21] C. Alcala and S. Qin, “Reconstruction-based Contribution for Process Monitoring,” Automatica, vol. 45, no. 7, pp. 1593–1600, Jul. 2009. [22] L. H. Chiang, E. Russell, and R. D. Braatz, Fault Detection and Diagnosis in Industrial Systems. London, U.K.: Springer-Verlag, 2001. [23] S. Yin, S. X. Ding, A. Haghani, H. Hao, and P. Zhang, “A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark tennessee eastman process,” J. Process Control, vol. 22, no. 9, pp. 1567–1581, Oct. 2012. [24] G. Li, C. F. Alcala, S. J. Qin, and D. Zhou, “Generalized reconstructionbased contributions for output-relevant fault diagnosis with application to the tennessee eastman process,” IEEE Trans. Control Syst. Technol., vol. 19, no. 5, pp. 1114–1127, Sep. 2011.
which means Jk is a monotone increasing sequence. Therefore, Jk must be convergent and, hence, the algorithm.
Gang Li received the Bachelor’s degree in mechanics and the M.Sc. and Ph.D. degrees in control from Tsinghua University, Beijing, China, in 2004, 2008, and 2011, respectively. He is currently a Postdoctoral Scholar with the Department of Chemical Engineering and Materials Science, University of Southern California, Los Angeles, CA, USA. His research interests include dynamic and nonlinear modeling with multivariate statistical techniques, data-driven fault detection and diagnosis, and fault identification with contribution
R EFERENCES [1] S. Bersimis, S. Psarakis, and J. Panaretos, “Multivariate statistical process control charts: An overview,” Quality Rel. Eng. Int., vol. 23, no. 5, pp. 517–543, 2007. [2] S. Yin, H. Luo, and S. Ding, “Real-time implementation of fault-tolerant control systems with performance optimization,” IEEE Trans. Ind. Electron., vol. 61, no. 5, pp. 2402–2411, May 2014. [3] S. J. Qin, “Statistical process monitoring: Basics and beyond,” J. Chemometrics, vol. 17, no. 8/9, pp. 480–502, 2003. [4] R. Muradore and P. Fiorini, “A pls-based statistical approach for fault detection and isolation of robotic manipulators,” IEEE Trans. Ind. Electron., vol. 59, no. 8, pp. 3167–3175, Aug. 2012. [5] V. Choqueuse, M. E. H. Benbouzid, Y. Amirat, and S. Turri, “Diagnosis of three-phase electrical machines using multidimensional demodulation techniques,” IEEE Trans. Ind. Electron., vol. 59, no. 4, pp. 2014–2023, Apr. 2012. [6] X. Li and X. Yao, “Multi-scale statistical process monitoring in machining,” IEEE Trans. Ind. Electron., vol. 52, no. 3, pp. 924–927, Jun. 2005. [7] A. Negiz, E. Lagergren, and A. Cinar, “Statistical quality control of multivariable continuous processes,” in Proc. Amer. Control Conf., 1994, vol. 2, pp. 1289–1293. [8] W. Ku, R. Storer, and C. Georgakis, “Disturbance detection and isolation by dynamic principal component analysis,” Chemometrics Intell. Lab. Syst., vol. 30, no. 1, pp. 179–196, Nov. 1995. [9] A. Negiz and A. Çlinar, “Statistical monitoring of multivariable dynamic processes with state-space models,” AIChE J., vol. 43, no. 8, pp. 2002– 2020, Aug. 1997. [10] A. Simoglou, E. Martin, and A. Morris, “Statistical performance monitoring of dynamic multivariate processes using state space modelling,” Comput. Chem. Eng., vol. 26, no. 6, pp. 909–920, Jun. 2002. [11] S. Qin, “An overview of subspace identification,” Comput. Chem. Eng., vol. 30, no. 10–12, pp. 1502–1513, Sep. 2006. [12] J. Wang and S. Qin, “A new subspace identification approach based on principal component analysis,” J. Process Control, vol. 12, no. 8, pp. 841– 855, Dec. 2002. [13] W. Li and S. Qin, “Consistent dynamic PCA based on errors-in-variables subspace identification,” J. Process Control, vol. 11, no. 6, pp. 661–678, Dec. 2001. [14] S. Ding, P. Zhang, A. Naik, E. Ding, and B. Huang, “Subspace method aided data-driven design of fault detection and isolation systems,” J. Process Control, vol. 19, no. 9, pp. 1496–1510, Oct. 2009. [15] Q. Wen, Z. Ge, and Z. Song, “Data-based linear Gaussian state-space model for dynamic process monitoring,” AIChE J., vol. 58, no. 12, pp. 3763–3776, Dec. 2012. [16] E. Russell, L. Chiang, and R. Braatz, “Fault detection in industrial processes using canonical variate analysis and dynamic principal component analysis,” Chemometrics Intell. Lab. Syst., vol. 51, no. 1, pp. 81–93, 2000. [17] A. Hóskuldsson, “PLS regression methods,” J. Chemometrics, vol. 2, no. 3, pp. 211–228, 1988. [18] S. De Waele and P. Broersen, “Order selection for vector autoregressive models,” IEEE Trans. Signal Process., vol. 51, no. 2, pp. 427–433, Feb. 2003. [19] D. Zhou, G. Li, and S. J. Qin, “Total projection to latent structures for process monitoring,” AIChE J., vol. 56, no. 1, pp. 168–178, Jan. 2010.
analysis.
S. Joe Qin (F’01) received the B.S. and M.S. degrees in automatic control from Tsinghua University, Beijing, China, in 1984 and 1987, respectively, and the Ph.D. degree in chemical engineering from the University of Maryland, College Park, MD, USA, in 1992. He is currently the Fluor Professor with the Department of Chemical Engineering and Materials Science and with the Department of Electrical Engineering, University of Southern California, Los Angeles, CA, USA. His research interests include statistical process monitoring, fault diagnosis, model predictive control, system identification, building energy optimization, and control performance monitoring. Dr. Qin is a Fellow of the International Federation of Automatic Control. He currently serves as an Associate Editor for the Journal of Process Control and the IEEE Control Systems Magazine, and a Member of the Editorial Board of the Journal of Chemometrics. He received the National Science Foundation (NSF) CAREER Award, the Northrop Grumman Best Teaching Award, the DuPont Young Professor Award, the Halliburton/Brown and Root Young Faculty Excellence Award, the NSF–China Outstanding Young Investigator Award, and the Changjiang Scholarship from Tsinghua University.
Donghua Zhou (SM’02) received the B.Eng., M.Sc., and Ph.D. degrees from Shanghai Jiaotong University, Shanghai, China, in 1985, 1988, and 1990, respectively. From 1995 to 1996, he was an Alexander von Humboldt Research Fellow with the University of Duisburg, Essen, Germany, and from July 2001 to January 2002, he was a Visiting Scholar with Yale University, New Haven, CT, USA. He is currently a Professor and the Head of the Department of Automation, Tsinghua University, Beijing, China. He is the author of over 120 papers published in peer-reviewed international journals and five monographs in the areas of fault diagnosis and fault-tolerant control, reliability prediction, and predictive maintenance. Dr. Zhou is a member of the Technical Committee on Fault Diagnosis and Safety of Technical Processes of the International Federation of Automatic Control (IFAC) and a Council Member and the Associate Chair of the Chinese Association of Automation. He was the National Organizing Committee Chair of the Sixth IFAC Symposium on SAFEPROCESS in 2006. He also serves as an Associate Editor of the Journal of Process Control.
