Regularization Methods for Inferential Sensing in Nuclear Power ...

Regularization Methods for Inferential Sensing in Nuclear Power Plants by J. Wesley Hines, Andrei V. Gribok, Ibrahim Attieh, and Robert E Uhrig Nuclear Engineering Department The University of Tennessee Knoxville, Tennessee 37996

Abstract Inferential sensing is the use of information related to a plant parameter to infer its actual value. The most common method of inferential sensing uses a mathematical model to infer a parameter value from correlated sensor values. Collinearity in the predictor variables leads to an ill-posed problem that causes inconsistent results when data based models such as linear regression and neural networks are used. This paper presents several linear and non-linear inferential sensing methods including linear regression and neural networks. Both of these methods can be modified from their original form to solve ill-posed problems and produce more consistent results. We will compare these techniques using data from Florida Power Corporation's Crystal River Nuclear Power Plant to predict the drift in a feedwater flow sensor. According to a report entitled "Feedwater Flow Measurement in U.S. Nuclear Power Generation Stations" that was commissioned by the Electric Power Research Institute, venturi meter fouling is "the single most frequent cause" for derating in Pressurized Water Reactors. This paper presents several viable solutions to this problem.

1 Introduction The safe and economical operation of Nuclear Power Plants requires knowledge of the state of the plant. This knowledge is obtained by measuring critical plant parameters with sensors and their instrument chains. The correct operation of the sensor systems must be validated to assure the safe, efficient operation of nuclear power plants. Traditional approaches to sensor validation at nuclear power plants involve the use of redundant sensors coupled with periodic instrument calibration. Many periodic sensor calibration techniques require the process shut down, the instrument taken out of service, and the instrument loaded and calibrated. This method can lead to equipment damage, incorrect calibrations due to adjustments made under non-service conditions, increased radiation exposure to maintenance personnel, and possibly increased downtime. Since few of the sensors are actually out of calibration, the end result is that many instruments are unnecessarily maintained. While correct adjustment is vital to maintaining proper plant operation, an alternative condition based technique is desirable. When implementing condition based calibration methods, the instruments are calibrated only when they are determined to be out of calibration. On-line, real-time sensor calibration monitoring will allow nuclear utilities to reduce the maintenance efforts necessary to assure the instruments are in calibration and increase the reliability of the components. The EPRI/Utility On-Line Monitoring Working Group estimates an industry wide cost savings of $40M to $290M over the next 20 years depending on the values applied to indirect benefits.

Inferential sensing is the prediction of a plant variable through the use of correlated plant variables. Most on-line calibration monitoring systems produce an inferred value and compare it to the sensor value to determine the sensor status. The system can be used to monitor sensors for drift or other failures making periodic instrument calibrations unnecessary. There are many methods used for inferential sensing including many types of regression, neural networks, fuzzy logic, and several statistical techniques. Several of these techniques produce believable results but can be adversely affected by collinearity of the predictor variables. This paper investigates how collinearity adversely affects inferential sensing techniques by making the results inconsistent and unrepeatable; and presents regularization as a potential solution. Sensor fault detection is a subset of fault detection and isolation that includes the detection of faults in other components. Several surveys on FDI technologies have been published including those by Basseville 1988, Gertler 1988, Isermann 1984, Patton 1991, and Willsky 1976. Manyl techniques have been studies for use in sensor fault detection in Nuclear Power Plants including expert systems (Qualls, Uhrig and Upadhyaya 1988, Tsoukalas 1992), model based techniques (Glockler 1991, Grini 1989, Hardy1992, Holbert 1990, Kittamura 1980 and others), state estimation techniques (Black 1998, Gross 1997, Singer 1997), artificial neural networks (Hines 1997b, Kavaklioglu and Upadhyaya 1994, Uhrig 1998, Upadhyaya and Eryurek 1992, and others), fuzzy logic (Hines 1997a, Holbert 1995), and hybrid combinations of these techniques (Ikonomopoulos 1992). All of these techniques can be divided into two basic categories: physical model based techniques and data driven models. We will define physical model based techniques as those that use mathematical models developed from first principals while data driven techniques are constructed using data collected from the process. This paper will investigate four linear and several non-linear data driven methodologies for inferential sensing: 1.

Linear Regression (LR)

2.

Ridge Regression (RR)

3.

Truncated Singular Value Decomposition (TSVD)

4.

Partial Least Squares (PLS)

5.

Neural Networks (NN)

These methodologies will be compared on the basis of prediction performance including their ability to model non-linearities, their ease of design, and their ability to handle collinear predictor variables while producing consistent results. The example used to make these comparisons will be that of inferential measurement of nuclear power plant feedwater flow. The data set is extremely ill-conditioned and will illustrate the necessity for the application of regularization techniques. Most inferential sensing problems are not this extreme and will be easier to solve, although regularization techniques should still be used.

1.1 Feedwater Flow Measurement In the United States, a nuclear power plant's operating limit is tied to its thermal power production. The steam generators' feedwater flow rate is an input to the calculation of thermal power and therefore must be accurately known. The majority of pressurized water reactors (PWRs) utilize venturi meters to measure feedwater flow because of their ruggedness and precision. However, these meters are susceptible to fouling due corrosion products that are present in the feedwater. The fouling increases the measured pressure drop across the meters, which results in an over estimation of the flow rate. Consequently, the reactors' thermal power is also overestimated (Nuclear News, 1993). To stay within regulatory limits, reactor operators are forced to derate their plants. According to a report entitled "Feedwater Flow Measurement in U.S. Nuclear Power Generation Stations" that was commissioned by EPRI (1992); venturi meter fouling is "the single most frequent cause" for derating in PWRs. The amount of derating, according to the report, varied from insignificant to 3% of full power. On average, the derating was between 1% and 2% of full power. It is estimated that derating an 800 MWe unit by 2% will cost the utility approximately $20,000 per day given a cost of electricity of $0.05/kWh.

Despite the susceptibility of the venturi meter to fouling, it is still the most common flow measurement instrument used in nuclear reactors. To overcome the loss of generating capacity, some utilities have developed a fouling coefficient or a correction factor to offset the degradation in the measurements' accuracy. The drawback of this method is that the flow measurement could be "corrected" when in fact there is no venturi fouling resulting in the reactor operating above its thermal power limits. The best solution to this problem would be to have an inferential sensing system that can accurately predict feedwater flow. This paper presents several methodologies to perform that function.

1.2 Prediction Problems Caused by Collinear Data Traditionally, the on-line prediction of instrument performance is based on the use of redundant sensors. The use of redundant sensors and other highly correlated measurements as predictor variables causes a potential problem in data based prediction models. This problem occurs due to the collinearity in the predictor variables. Variables are collinear if the data vectors representing them lie on the same line (i.e., subspace of dimension one). More generally, k variables are collinear if the vectors that represent them lie in a subspace of dimension less than k; that is, if one of the vectors is a linear combination of the others. In practice, such “exact collinearity” rarely occurs due to process and measurement noise. A broader notion of collinearity is therefore needed to deal with the problem as it affects statistical estimation. More loosely, two variables are collinear if they lie almost on the same line; that is, if the angle between them is small. In the event that one of the variables is not constant, this is equivalent to saying that they have a high degree of correlation between them. To show the degrading effects of collinearity on prediction, consider two redundant sensors (x1 and x2) that are operating in a noisy environment and being used to infer the value of a third redundant sensor using the common linear regression model of equation 1.

x3 + ε 3 = ( x1 + ε 1 )w1 + ( x2 + ε 2 )w 2

(1)

Since x1 and x2 are redundant sensors, their outputs are the same except for their respective noise term ε i, and therefore, are almost perfectly collinear. Singular Value Decomposition (SVD) (Jolliffe 1996) can be used to determine if the k=2 variables lie in a subspace of dimension less than two. This method performs a linear transformation of the data into a new coordinate system so that a maximum amount of the variation of the data is along one principal axis and the remaining variation is along the second axis. A mathematical simulation of this redundant sensor case resulted in singular values of 7.0871 and 0.0059. The square of the singular values is proportional to the amount of variance in the data. In this case over 99.9% of the variation of the original signals could be represented with a single signal. In other words, the 2 variables lie in a single dimensional subspace. This shows the high degree of collinearity in this illconditioned problem (the condition number is over 1,000). Solving for this prediction model using linear regression yields coefficients of: w1 = -388.4 and w2 = 389.4. These huge weights result in a solution that has a much larger variance than that of the predictor variables. This results in noisy and inconsistent predictions. Using regularization, a technique presented in the next section, the following solution: w1 = 0.47 and w2

= 0.47 was generated. The optimal solution to the problem would be w1 = 0.5 and w2 = 0.5, which is an average of the two redundant predictors. Since the regularization solution is slightly biases towards a small norm resulting in weights that are slightly smaller than the optimal solution. This is the cost of regularization; the benefit of regularization is a stable inferred value with a smaller variance than that of the predictors.

2 Methodology The development of a data based inferential sensing system consists of collecting training and testing data, preprocessing the data to remove outliers, and scaling the data to allow the use of statistical signal evaluation techniques. Once the data is collected and preprocessed, the inferential model is developed and

tested. The final system is shown in Figure 1 where X is a vector of predictor signals, Xs is the preprocessed and scaled signal, and Y is the inferred signal.

X

Xs Data Pre-processing

Model

Y

Fig. 1 Inferential Sensing System

Several inferential models can be used for inferential sensing including linear techniques such as linear regression, principal component regression, ridge regression, and partial least squares; and non-linear techniques such as non-linear regression, non-linear partial least squares, artificial neural networks, and fuzzy inference systems. There are two primary categories of methods used to deal with the problem of collinearity. The first category transforms the predictors to a new orthogonal space thus removing the collinearity, while the second category, called direct regularization methods, deals with making the ill-conditioned problem a well-conditioned problem. The following matrix shows which techniques will be used with each inferential sensing method. We will investigate each of these methodologies in the following section. Method Linear Regression Partial Least Squares Neural Network

Predictor Transformations X X

Direct Regularization X X

2.1 Data Preprocessing Prior to any statistical evaluation of the data, a number of preprocessing techniques should be applied to the raw data to ensure consistent results. The most common preprocessing techniques are filtering and scaling. 2.1.1 Outliers It is a well-known fact that least squares models are very sensitive to outliers. Just one outlier can significantly distort the estimation computed with a least squares method. In fact, a noise spike occurring at the same sample time, in two otherwise uncorrelated signals can result in a correlation coefficient of 0.95. To reduce measurement noise we suggest using a median filter because it has well known outlier rejection and fast digital implementation properties. 2.1.2 Data Scaling Data scaling is one of the mostly controversial issues addressed in the regularization of ill-posed problems. Scaling can change the singular value spectrum of the data matrix and change the precision of the regularized solution. Two types of data scaling are commonly used: the first one is the well known Zscore scaling which provides signals with zero mean and unit variance and the second one we term range scaling which linearly scales the data’s range to between zero and one. In the following examples we will use range scaling and also mean center the data.

2.2 Regularization The inferential measurement of feedwater flow sensor drift is based on the inference of actual feedwater flow rate. Actual flow is predicted through its relationship to other correlated plants parameters. The problem with using these parameters as predictors is that they are not only highly correlated with feedwater flow, but they are also correlated with each other. If this degree of correlation is quite high, the data matrix becomes ill-conditioned and the problem of drift detection

becomes ill-posed in the sense of Hadamard (1923). Hadamard defined a well-posed problem as a problem which satisfies the three following conditions: 1.

The solution for the problem exists.

2.

The solution is unique.

3.

The solution is stable or smooth under small perturbations of the data; i.e. small perturbations in the data should produce small perturbations in the solution.

If any of these conditions are not met, the problem is termed ill-posed and special considerations must be taken to ensure a reliable solution. To understand the essence of ill-posed problems for inferential sensing, let us consider the linear least squares model whose objective is to find a linear combination of predictor variables that accurately models the response variable. 2

min Xw − y 2 ,

X ∈ R mxn , m ≥ n

(2)

Where X is a data matrix containing m samples of n predictor variables related to feedwater flow rate, y is a vector of measured values of feedwater flow and w is a solution of regression coefficients. A very valuable tool in the analysis of ill-posed problems is singular value decomposition (SVD) (Golub, 1996). The SVD of data matrix X can be written as n

X=UΣ VT=

∑

uiσ i viT

i=1

(3)

where u and v are called left and right eigenvectors of X and σ i are the singular values of the matrix X. In terms of the SVD, the solution for the least squares problem (2) can be written as: n

wLS=

∑

i= 1

u iT y vi σi

(4)

Here we assume that matrix X has a full rank of n. Equation (4) gives insight into the essence of illconditioning. The division by small singular values results in amplification of high-frequency oscillations of the right singular vectors of the data matrix X. To deal with ill-conditioned problems, several methods have been developed which essentially damp or filter out these high frequency oscillations. These methods are called regularization methods because they regularize or smooth potentially unstable least squares solutions. The most simple regularization method is the truncated SVD (TSVD) method. This method truncates the sum in equation 2 at some value k