ARTICLE IN PRESS
JID: NEUCOM
[m5G;March 13, 2018;3:14]
Neurocomputing 0 0 0 (2018) 1–9
Contents lists available at ScienceDirect
Neurocomputing journal homepage: www.elsevier.com/locate/neucom
Machinery health indicator construction based on convolutional neural networks considering trend burr Liang Guo b, Yaguo Lei a,∗, Naipeng Li b, Tao Yan b, Ningbo Li b a b
Key Laboratory of Education Ministry for Modern Design and Rotor-Bearing System, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China Shaanxi Key Laboratory of Mechanical Product Quality Assurance and Diagnostics, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China
a r t i c l e
i n f o
Article history: Received 22 September 2017 Revised 2 January 2018 Accepted 27 February 2018 Available online xxx Communicated by Hongli Dong Keywords: Machinery health indicator Convolutional neural network Outlier region correction Deep learning Trend burr
a b s t r a c t In the study of data-driven prognostic methods of machinery, much attention has been paid to constructing health indicators (HIs). Most of the existing HIs, however, are manually constructed for a specific degradation process and need the prior knowledge of experts. Additionally, for the existing HIs, there are usually some outlier regions deviating to an expected degradation trend and reducing the performance of HIs. We refer to this phenomenon as trend burr. To deal with these problems, this paper proposes a convolutional neural network based HI construction method considering trend burr. The proposed method first learns features through convolution and pooling operations, and then these learned features are constructed into a HI through a nonlinear mapping operation. Furthermore, an outlier region correction technique is applied to detect and remove outlier regions existing in the HIs. Unlike traditional methods in which HIs are manually constructed, the proposed method aims to automatically construct HIs. Moreover, the outlier region correction technique enables the constructed HIs to be more effective. The effectiveness of the proposed method is verified using a bearing dataset. Through comparing with commonly used HI construction methods, it is demonstrated that the proposed method achieves better results in terms of trendability, monotonicity and scale similarity. © 2018 Elsevier B.V. All rights reserved.
1. Introduction Thanks to the development of advanced sensing and computing techniques, massive condition monitoring data are available now, which raises the research attention on data-driven prognostic methods of machinery. In the study of those methods, lots of effort has been taken to construct an effective health indicator (HI) [1–3]. HI construction attempts to identify and quantify a history and ongoing degradation process by extracting feature information from the acquired data [4–6]. Therefore, the quality of the constructed HI largely influences the efficacy of data-driven prognostic methods. From this perspective, it is critical to construct an effective HI for machinery prognostics. Generally, HIs are categorized into physical HIs (PHIs) and fused HIs (FHIs). PHIs extract physical significance related information from the acquired data using statistics or signal processing algorithms [3,7,8]. In contrast to PHIs, FHIs are usually constructed with multiple PHIs or multiple sensor information using data fusion algorithms [9]. In recent years, lots of FHIs have been constructed to assess performance degradation of machinery. Qiu et al.
∗
Corresponding author. E-mail address:
[email protected] (Y. Lei).
[10] used a self-organizing map (SOM) method to fuse the extracted features into a FHI to monitor the conditions of bearings. Kumar et al. [11] developed and compared various FHIs using different algorithms, namely singular value decomposition, average value of the cumulative feature and Mahalanobis distance to assess the conditions of rolling element bearings. Yu et al. [12,13] established several FHIs and applied them in health monitoring of bearings. Those HIs could be referred to as manual HIs because they are constructed using some handcrafted feature extraction and data fusion algorithms. Even though the aforementioned manual HIs achieve promising results, they still face some problems. Firstly, to construct a manual HI, it is necessary to understand the characteristic of the acquired data. However, this knowledge is difficult to obtain, and even if it is available, the HI construction still takes a great deal of effort. Secondly, a manual HI is generally constructed for a specific degradation process, therefore, it may not be generalized well to others. With these challenges, a feature learning based method is required to construct HIs automatically. Deep learning, as a powerful feature learning tool, uses multiple layers of nonlinear processing units to learn features [14]. The basic idea of deep learning is to capture the intrinsic information hidden in the input data [15]. Recently, deep learning based methods have achieved remarkable success in a wide field of application,
https://doi.org/10.1016/j.neucom.2018.02.083 0925-2312/© 2018 Elsevier B.V. All rights reserved.
Please cite this article as: L. Guo et al., Machinery health indicator construction based on convolutional neural networks considering trend burr, Neurocomputing (2018), https://doi.org/10.1016/j.neucom.2018.02.083
ARTICLE IN PRESS
JID: NEUCOM 2
[m5G;March 13, 2018;3:14]
L. Guo et al. / Neurocomputing 000 (2018) 1–9
as CHI. As a case study, a bearing dataset is used to demonstrate the effectiveness of the proposed method. In the case study, some commonly used HI construction methods are also calculated to compare with CHI. The results show that CHI provides the best results in terms of trendability, monotonicity and scale similarity. The main contributions of this paper are summarized as follows:
Fig. 1. Outlier regions in a HI.
including speech recognition, image recognition, and so on [16,17]. In the field of machinery fault diagnosis, Lei et al. [18,19], Thirukovalluru et al. [20] and Ding and He [21] have demonstrated that features learned by supervised or unsupervised deep learning models outperform traditional handcrafted features in the bearing and gearbox fault diagnosis. Convolutional neural networks (CNNs), one of the most successful deep learning models, are designed to take advantage of the spatial structure of input data. This is achieved using local connections and some forms of pooling, and those operations are able to generate translation invariant features [22,23]. Therefore, CNNs have shown their effectiveness in vibration signal processing. Abdeljaber et al. [24] established a structural damage detection system using a one-dimension CNN. Ince et al. [25] developed a motor condition monitoring and early fault detection system using a CNN. Sun et al. [26] proposed a feed-forward convolutional pooling architecture to diagnose motor faults. Although the CNN based methods have achieved excellent results in condition monitoring and fault diagnosis of machinery, there are few research reports on their application in HI construction. Additionally, another problem of the existing HIs is that there are some outlier regions which affect the performance of HIs. As shown in Fig. 1, the amplitude of a HI tends to increase over time, while the outlier regions, i.e., region O1 and O2 , are far away from the increase trend. Those outlier regions usually affect the performance of the HI, especially trendability and monotonicity, and may eventually reduce the accuracy of prognostic estimation. We refer to this phenomenon as trend burr. In order to address this problem, the outlier regions in HIs should be corrected. The first step of outlier region correction is to detect outliers. Lots of outlier detection techniques have been developed, such as the techniques based on machine learning, information theory, and statistics [27,28]. Statistics based techniques apply a statistical inference test to determine whether an observed point belongs to the normal trend, and are often proved to be effective. In detail, for certain problems in statistics, an event is considered to be practically impossible if it lies in the distribution region of a random variable at a distance from its mean of more than three times the standard deviation. Let X be a normally distributed random variable, then a sample in X can differ from its mean μ by a quantity exceeding 3σ on the average in not more three times in a thousand trials. Therefore, in statistics, the so-called 3σ rule is a simple and widely used heuristic for outlier detection. Currently, the 3σ rule has been successfully employed in various outlier detection tasks [29,30]. In order to automatically construct HIs and reduce trend burr in HIs, this paper proposed a CNN based HI construction method considering trend burr. The proposed method consists of two stages. In the first stage, several convolution and pooling operations are stacked to learn features, and then these learned features are mapped into a HI through a nonlinear mapping operation. In the second stage, the performance of the HI is further improved by detecting and removing outlier regions. The CNN based HI is named
(1) A new HI construction method is proposed. Different from the traditional HI construction methods based on handcrafted feature extraction and data fusion algorithms, the proposed method automatically constructs a HI based on CNN. Consequently, the method is preferable to reduce the need of the expert prior knowledge and labor resources. Moreover, an outlier region correction technique is utilized to remove the outlier regions in the constructed HI to reduce trend burr. (2) A metric to assess the scale similarity of HIs is proposed. If the HIs of different samples in a dataset have the similar range scales, the failure threshold for prognostics could be decided easily. Therefore, a new HI assessment metric, namely scale similarity, is proposed to quantitatively evaluate the similarity of range scales of HIs in the same dataset. The rest of the paper is organized as follows. The detailed procedure of the proposed method is presented in Section 2. In Section 3, a dataset collected from the accelerated degradation experiment on rolling element bearings is used to evaluate the performance of the proposed method. Finally, conclusions are drawn in Section 4. 2. The proposed method This section details the proposed method, which includes two stages: HI construction and outlier region correction. The illustration and flowchart of the method are shown in Fig. 2. 2.1. Health indicator construction In the first stage of the proposed method, a CNN model is established to automatically construct a HI. As depicted in Fig. 2, the established model contains eight layers: one input layer, two convolutional layers C1 and C2, two pooling layers P1 and P2, and three fully connected (FC) layers F1, F2 and F3. Firstly, the raw vibration signals in the input layer are fed into the first convolutional layer C1. Then, features are learned from the raw vibration signals through two convolutional and pooling layers. Finally, in the FC layers, the learned features are constructed into a HI. The input layer is composed of machinery vibration signals, and is connected to the convolutional layer C1. The convolutional layer is the core building block of the established model. It includes a set of learnable filters, which are also known as kernels. The length of a kernel is generally smaller than the length of a data volume. Therefore, in each convolution operation only a sub-region of the data volume is connected with the kernel. Let In, m denote the nth data volume at layer m, and Nm be the data number of such volume. The nth data volume In, m is j : j +L
segmented as In,m ker , where Lker is the length of the sub-region of the data volume as well as the length of the kernel, and j denotes the jth point in the data volume. Then the convolution operation can be defined as j zk,m = +1
Nm
j: j+Lker
In,m
∗ wk,n,m + bn,m ,
(1)
n=1
where ∗ is a one-dimension convolution operation, wk, n, m is referred to as the kernel connecting the nth group at the mth layer to the kth group at the m + 1th layer and bn, m is the bias paj rameter. zk,m+1 is an intermediate result of the convolutional layer.
Please cite this article as: L. Guo et al., Machinery health indicator construction based on convolutional neural networks considering trend burr, Neurocomputing (2018), https://doi.org/10.1016/j.neucom.2018.02.083
ARTICLE IN PRESS
JID: NEUCOM
[m5G;March 13, 2018;3:14]
L. Guo et al. / Neurocomputing 000 (2018) 1–9
3
Fig. 2. The flowchart of the proposed method.
j
Then, zk,m+1 is transferred into the output of a convolutional layer through an activation function j j yk,m = f (zk,m ), +1 +1
(2)
j yk,m+1
where is the output of the convolutional layer, and f: → is an activation function. In this paper, tanh is chosen as the nonlinear activation function of the convolutional layers as it is a scaled activation function [31]. The convolutional layer is followed by a pooling layer. In a pooling layer, the input data is first partitioned into a set of sub-region. Then, the optimal value for each sub-region is calculated through a pooling function. In the proposed method, the max pooling function is utilized, which returns the maximum value within a certain sub-region as follows j pn,m = max{
j∗ L :( j+1 )∗ Lp yn,mp
},
(3)
where Lp is the pooling length that controls the length of the subj
region in the pooling operation, and pn,m is the output of the jth point at the nth group from the mth layer. The convolution and pooling operations enable the established model to learn shift invariant features from raw vibration signals to construct a HI. The output of the two convolutional and pooling layers is learned features. Those learned features are further mapped into a HI through a nonlinear mapping operation. The applied nonlinear mapping operation consists of three FC layers F1, F2 and F3. F1 is flatten from the second pooling layer P2. Next, F2 is connected with F1, and the output of F2 can be calculated as
Yf2 = f (Wf2Yf1 + bf2 ),
(4)
where Yf1 is the output of layer F1, Wf2 denotes the weights between layer F1 and layer F2, bf2 is the corresponding bias, and f: → is the activation function. In order to keep the value of the constructed HI ranging from 0 to 1, the last layer of the model is activated with logistics regression
Yθ =
1 1 + e−(Wf3Yf2 +bf3 )
(5)
where Yf2 is the output of layer F2, Wf3 and bf3 are the weights and bias between layer F2 and layer F3, respectively. Yθ is the output of the CNN model, which is also the constructed HI. After the CNN model is built up, it then should be trained using training samples. Suppose that the training samples are tr {(xi , yi )}Ni=1 , where Ntr is the total number of training samples, xi
is the input signal, and yi is the corresponding label indicating the degradation degree. In the proposed method, output value yθ (xi ) is expected to be as close as possible to its corresponding label yi by minimizing the square of the Euclidian distance between output value yθ (xi ) and the actual label yk corresponding to input xi ,
Jθ =
Ntr 1 yθ (xi ) − yi 2 . 2
(6)
i=1
The structure parameters of the established model are optimized by minimizing Eq. (6) using training samples. After the training step, the raw vibration signals of testing samples are feed into the trained model to construct the HI through Eqs. (1)–(5). 2.2. Outlier region correction Once a HI is constructed, the outlier regions in the HI are corrected to reduce trend burr. This technique consists of two steps: outlier region detection and outlier region removal. We assume that the constructed HI increases with a certain tendency, which indicates that the difference of the constructed HI remains a certain value. Therefore, a 3σ rule based method is presented to detect outlier regions in the constructed HI. Then those outlier regions are removed by connecting starting point with end point of outlier regions. Algorithm 1 describes the detailed procedure of the proposed outlier region correction technique.
Algorithm 1 Outlier region correction. Technique: Outlier region correction Input: HI Hc , length K of a HI Hc , length L of the positive outliers and negative outliers in a outlier region. Initialization: number of the positive outliers Np , number of the negative outliers Nn , location set of the positive outliers Lp , location set of the negative outliers Ln , location of outliers lxt . Output: CHI Hi . 1: Compute the difference dHc of Hc through Eq. (7) 2: Compute the mean u and the standard deviation σ 3: for k := 0 to K do 4: if dHkc ≥ μ + 3σ then, Np = Np + 1,Lp = Lp ∪ lxt 5: else if dHkc < μ − 3σ then, Nn = Nn + 1,Ln = Ln ∪ lxt 6: end if 7: end for 8: if Np > L and Nn > L then 9: Remove the outlier regions in HI through (9), and output CHI 10: end if
Please cite this article as: L. Guo et al., Machinery health indicator construction based on convolutional neural networks considering trend burr, Neurocomputing (2018), https://doi.org/10.1016/j.neucom.2018.02.083
ARTICLE IN PRESS
JID: NEUCOM 4
[m5G;March 13, 2018;3:14]
L. Guo et al. / Neurocomputing 000 (2018) 1–9
At the step of outlier region detection, the difference of a HI is calculated as follows
dHkc =
hck+1 − hck
k
,
(7)
hck
dHkc
where is the HI at time tk , and is the corresponding difference at time tk . For the difference of a CHI, it is expected that its distance from the mean is within three times the standard deviation. Therefore, the outlier detection threshold is defined as
Threshold =
μ + 3σ , upper threshold , μ − 3σ , lower threshold
(8)
where μ is the mean value of the dHc , and σ is the standard deviation value of the dHc . Then, within the constructed HI, the point whose difference exceeds the upper threshold is named as a positive outlier, and the point whose difference is below the lower threshold is named as a negative outlier. The data region where both the number of positive outliers and the number of negative ones are larger than L is referred to as the outlier region, where L is a parameter to decide the outlier region. At the outlier region removal step, among the outlier region, the starting outlier at time ts is denoted as htcs and the ending outlier at time te is referred to as htce . In order to remove the detected outlier region, the starting point htcs directly connects to htcc . Let htcc be one point at time tc in the outlier region of a HI. Then the corresponding CHI at time tc can be formally described as
hti c = htcs +
htce − htcs te − ts
(tc − ts ),
(9)
where hti c is the CHI at time tc . 2.3. Assessment metric
(1) Trendability: It is designed to evaluate the correlation between the degradation trend of a HI and the operation time [32],
K k=1 (hk − H¯ )(tk − T¯ )
K
k=1
2
(hk − H¯ )
K
k=1
2
,
(10)
(tk − T¯ )
where H¯ = (1/K ) Kk=1 hk , T¯ = (1/K ) Kk=1 tk , K is the length of a HI H, and hk is the HI value at time tk . (2) Monotonicity: It is designed to assess the property of monotonic increase or decrease trend of a HI. This metric can be described through the following formula [7],
No. of dH > 0 No. of dH < 0 , Mon(H ) = − K −1 K −1
(11)
where dH is the difference of a HI H, and K is the length of the HI H. (3) Scale similarity: Different from the trendability and monotonicity which reflect the properties of a single HI, the proposed scale similarity measures the similarity among all HIs in the training set. Concretely, in the data-driven prognostic methods, the prognostic estimation is obtained when the HI exceeds a pre-defined failure threshold. Therefore, it is important to define a suitable failure threshold for accurate prognostics. Actually, the failure threshold is highly related to the properties of HIs. If the HIs in the training set have the same range scales, the failure threshold could be set easily. On the contrary, if some HIs in the training set range in a specific scale such as between 0 and 10 while the others range between 0 and 5, it
j Ntr hmax − hmax + hmin − h j 1 min Scales (H ) = 1− , Ntr len(H ) + len(H j ) j=1
(12) where Ntr is the number of samples in the training set. hmax and hmin are the maximum and minimum values of the selected HI. j j hmax and hmin are the maximum and minimum values of the jth HI in the training set. len(H) and len(Hj ) are the amplitude lengths of the selected HI and the jth HI in the training set, respectively. It can be found that the value of a scale similarity ranges from 0 to 1. If the range scale of the selected HI totally overlaps with range scales of HIs in the training set as shown in Fig. 3(a), the Scales(H) equals 1. On the contrary, if the range scales of HIs in the training set are absolutely non-overlapped as indicated in Fig. 3(b), the Scales(H) equals 0. Additionally, if two HIs partially overlap, the Scales(H) is a value between 0 and 1. For instance, as presented in Fig. 3(c), let HI1 be the selected HI, and HI2 be the 1th HI in the training set. HI1 and HI2 have a same minimum j value hmin = hmin = 0. Whereas, they have different maximum values, i.e., hmax equals 0.6, and h1max equals 1. The lengths of HI1 and HI2 are len(H ) = 0.6 and len(H 1 ) = 1, respectively. Therefore, the scale similarity is Scales (H ) = 0.75. The above three metrics assess three different properties of a HI. Each of those metrics, however, is not able to comprehensively evaluate the suitability of a HI. To handle this problem, a hybrid metric which combines the above three metrics is developed.
HM =
In order to quantitatively assess the performance of HIs, two existing metrics, i.e., trendability and monotonicity, and one proposed metric, i.e., scale similarity, are utilized.
T red (H, T ) =
would be hard to set the failure threshold. Consequently, in order to assess the scale similarity of HIs, a new metric is proposed. It is calculated as follows
T red + Mon + Scales . 3
(13)
3. Experiment and discussion 3.1. Data description As one of the most important components, rolling element bearings are widely used in various machines. Therefore, a rolling element bearing dataset is used as an example to validate the proposed method. The data were acquired from PRONOSTIA [33], which is a platform developed to conduct accelerated degradation experiments of bearings. In the experiments, seventeen bearings were tested, and we name those bearings as bearing 1 to bearing 17, respectively. As we know, accelerated degradation tests on those seventeen bearings are conducted with three different working conditions. However, those working conditions are not considered in our experiments. In other words, although the seventeen bearings are operated in three different working conditions, no working condition information is added into our method. To validate the proposed method, the data collected from sixteen bearings are chosen to construct the training set, then the data collected from the remaining one are employed as the testing set. For example, if the data from bearing 2 to bearing 17 are chosen as the training set, then the data from bearing 1 are employed as the testing set. We define the ith training sample as i {(xik , yik )}Kk=0 , where xik ∈ Nin is the input signal of the ith training sample at time tk , Nin equals 2560, Ki is the sequence length of the ith sample and yik is the corresponding label, which indicates the degradation percentage of the ith bearing at time tk . For example, supposing that the life of the ith bearing is 22,0 0 0 s, the degradation percentage equals 0.7 when the bearing is operated for 15,400 s.
Please cite this article as: L. Guo et al., Machinery health indicator construction based on convolutional neural networks considering trend burr, Neurocomputing (2018), https://doi.org/10.1016/j.neucom.2018.02.083
JID: NEUCOM
ARTICLE IN PRESS
[m5G;March 13, 2018;3:14]
L. Guo et al. / Neurocomputing 000 (2018) 1–9
5
Fig. 3. HIs with different range scales.
3.2. Health indicator construction
Fig. 4. Results using various kernel lengths of the proposed method.
There are several parameters in the established CNN model to be determined, i.e., the kernel length and the pooling length. So the selection of those parameters is primarily investigated. In the procedure of parameter selection, a sample is randomly selected from the training set to validate the trained model. First of all, the kernel length is encoded as a decision variable to conduct parameter selection experiments. Each experiment repeats 20 times. Fig. 4 shows the boxplot of the results. It can be seen from the figure that a better performance is obtained at the length of 25. Then, the selection of the pooling length is investigated. The results displayed in Fig. 5 show that a better performance is obtained when the pooling length equals 8. In fact, a large pooling length may omit some important information in the features, while a small pooling size may limit the property of the translation invariance of the CNN. Therefore, the lengths of the kernel and pooling are set to 25 and 8, respectively. After selecting those structural parameters, the sizes of eight layers are confirmed. Using those parameters, the HIs are constructed. Fig. 6(a) and (b) is HIs of bearing 6 and bearing 7, respectively. It is observed that the values of those HIs generally increase as operation time grows until they are about to reach 1. It means that the constructed HIs may be able to roughly characterize the degradation process of these bearings. 3.3. Outlier region correction
Fig. 5. Results using various pooling lengths of the proposed method.
For the constructed HIs in Fig. 6(a) and (b), it is noted that although they are generally monotonous, the trend burr phenomenon can be found in those HIs. Therefore, in this section, the outlier region correction technique is used to reduce trend burr. The selection of the outlier region length L is first investigated. As displayed in Fig. 7, the length of 5 indicates a better perfor-
Fig. 6. The HIs of bearing 6 and bearing 7: (a) the HI of bearing 6 and (b) the HI of bearing 7.
Please cite this article as: L. Guo et al., Machinery health indicator construction based on convolutional neural networks considering trend burr, Neurocomputing (2018), https://doi.org/10.1016/j.neucom.2018.02.083
JID: NEUCOM 6
ARTICLE IN PRESS
[m5G;March 13, 2018;3:14]
L. Guo et al. / Neurocomputing 000 (2018) 1–9
Fig. 7. Results using various lengths of the outlier region.
regions in the constructed HI as shown in Fig. 8. Following the procedure of the proposed method, the difference of the HI is firstly calculated, which is depicted in Fig. 8(b). Then the 3σ threshold is estimated through Eq. (8), and the upper threshold 0.0055 and the lower threshold −0.0054 are set to detect the outlier regions. At last, those detected outlier regions are removed through Eq. (9), and the results are plotted in Fig. 8(a). From the figure, it is observed that the proposed method is able to detect and correct the outlier regions in the HI. Furthermore, in order to verify the effectiveness of the outlier region correction technique, a comparison experiment of seventeen bearings is conducted, where the hybrid metrics of HIs and CHIs are calculated and displayed in Fig. 9. The result fits our hypothesis that the proposed outlier region correction technique enables CHIs to be more effective than HIs in the term of hybrid metric. 3.4. Comparison
Fig. 8. The CHI and the difference of HI of bearing 6: (a) CHI and (b) difference of HI.
mance. Actually, if the length of the outlier region L is too small, the detected region may not be a correct outlier region. On the contrary, if it is too large, some potential outlier regions may be ignored. Therefore, 5 is chosen as the length of the outlier region. Based on the selected parameters, all the constructed HIs are processed through the proposed outlier region correction technique. Taking bearing 6 for example, there are two obvious outlier
To show the effectiveness of the proposed method, five HI construction methods are employed for comparison. The detail information about those methods can be found in Table 1. In method 1, a deep learning model stacked autoencoder (SAE) with eight layers is utilized to construct HI, in which the input is raw vibration signals, and the constructed HI is called as SAE-HI. In method 2, 14 specifically designed features are extracted, including one timedomain related-similarity feature, five frequency-domain relatedsimilarity features in different frequency bands and eight wavelet package energy ration features [9]. Based on those handcrafted features, the HI is constructed by a FC neural network, and the constructed HI is named as FC-HI1. In method 3, 14 commonly used features are extracted, including mean, root mean square, kurtosis, variance, crest factor, wave factor and eight energy ratios of wavelet package transform [29]. Then those 14 commonly used features are input into a FC neural network to construct FCHI2. It should be noted that the FC neural network applied in the method 2 and method 3 is also the model that is used to conduct the nonlinear mapping operation in the proposed method. In method 4, 14 aforementioned specifically designed features are input into an unsupervised learning model SOM to construct SOMHI1. In method 5, 14 aforementioned commonly used features are input into SOM to construct SOM-HI2. For various comparison purposes, those above-mentioned methods, i.e., SAE, FC neural networks and SOM, can be classified into three types. The first type is designed to compare CNN with SAE for feature learning of vibration signals. In those two methods, the raw vibration signals are directly input to construct a HI. The second one is designed to compare learned features with handcrafted features for HI construction. Therefore, the FC neural network based method is ap-
Fig. 9. The hybrid metrics of HIs and CHIs of the seventeen bearings.
Please cite this article as: L. Guo et al., Machinery health indicator construction based on convolutional neural networks considering trend burr, Neurocomputing (2018), https://doi.org/10.1016/j.neucom.2018.02.083
ARTICLE IN PRESS
JID: NEUCOM
[m5G;March 13, 2018;3:14]
L. Guo et al. / Neurocomputing 000 (2018) 1–9
7
Table 1 Comparison of different HI construction methods. Methods
Description
HI
Trendability
Monotonicity
Scale Similarity
1 2 3 4 5 6
SAE Special features + FC Common features + FC Special features + SOM Common features + SOM Proposed method
SAE-HI FC-HI1 FC-HI2 SOM-HI1 SOM-HI2 CHI
0.125 0.803 0.793 0.802 0.560 0.897
0.020 0.264 0.229 0.152 0.113 0.406
0.679 0.817 0.801 0.715 0.594 0.904
Fig. 10. The constructed HIs from bearing 1 to bearing 7: (a) CHI, (b) SAE-HI, (c) FC-HI1, (d) FC-HI2, (e) SOM-HI1 and (f) SOM-HI2.
plied because the difference between the proposed method and the FC neural network based method is just the applied features. The third one is designed to compare the proposed method with a state-of-the-art method, i.e., SOM based HI construction method, as it has been widely used in the field of prognostics. For directly comparing those constructed HIs, the CHI, SAE-HI, FC-HI1, FC-HI2, SOM-HI1 and SOM-HI2 of the first seven bearings from the experiment are plotted in Fig. 10, and three quantitative assessment metrics, i.e., trendability, monotonicity and scale similarity, are calculated and summarized in Table 1. In those constructed HIs, the CHI and SAE-HI are both automatically constructed from raw vibration signals by deep learning models, whereas they obtain different results. The CHI constructed by the proposed method is displayed in Fig. 10(a). It is observed that the CHI shows an obvious degradation trend over operation time, and the range scales of the seven CHIs are close to each other. However, as shown in Fig. 10(b), the SAE-HI constructed by method 1 does not reveal the degradation trend. The possible reason for
that result is that the acquired vibration signals possess the property of shift variant, and the SAE applied in method 1 is not able to process shift variant signals. In contrast, the proposed method has the advantage of learning shift invariant features, so it is able to directly process raw vibration signals and construct an effective HI. Different from the CHI which is an automatically constructed HI, the FC-HI1 and FC-HI2 are manually constructed ones. As shown in Fig. 10(c) and (d), the FC-HI1 and FC-HI2 present the degradation trend over time. However, the values of the seven HIs are different under the failure condition. Therefore, it is usually hard to decide a failure threshold for bearing prognostics using the FC neural network based HIs. Moreover, as displayed in Table 1, the CHI obtains a higher value than the FC neural network based HIs in terms of trendability, monotonicity and scale similarity. More precisely, the trendability, monotonicity and scale similarity of FC-HI1, FC-HI2 and CHI are (0.803, 0.264, 0.817), (0.793, 0.229, 0.801) and (0.897, 0.406, 0.904), respectively. Actually, the only
Please cite this article as: L. Guo et al., Machinery health indicator construction based on convolutional neural networks considering trend burr, Neurocomputing (2018), https://doi.org/10.1016/j.neucom.2018.02.083
JID: NEUCOM 8
ARTICLE IN PRESS
[m5G;March 13, 2018;3:14]
L. Guo et al. / Neurocomputing 000 (2018) 1–9
difference between the method 2, method 3 and the proposed method is that the features applied in method 2 and method 3 are handcrafted, while the features applied in the proposed method are automatically learned from the raw vibration signals. It is illustrated that the features automatically learned through the proposed method outperform the handcrafted features for constructing HIs of bearings. From Fig. 10 and Table 1, it can also be found that HIs constructed with various handcrafted features appear to perform differently. Specifically, Fig. 10 displays that the FC-HI1 and SOM-HI1 exhibit the better trend over operation time than the FC-HI2 and SOM-HI2, respectively. Moreover, the quantitative comparison results in Table 1 show that the FC-HI1 and SOM-HI1 obtain better results than the FC-HI2 and SOM-HI2 in terms of trendability, monotonicity and scale similarity, respectively. In fact, the construction procedures of the FC-HI1 and SOM-HI1 are the same as ones of the FC-HI2 and SOM-HI2 except for the application of the extracted features. For the FC-HI1 and SOM-HI1, the specifically designed features range within a certain scale from 0 to 1, and appear to be of trendability. On the contrary, for the FCHI2 and SOM-HI2, the commonly used features range in different scales, and are not specifically designed to indicate the degradation processes of machines. The specifically designed features are in a certain range scale, therefore, the normalization processing is not required before those features are input into the data fusion algorithm. Additionally, the trendability of features is the basic requirement for the HI construction. Those reasons probably explain why the specifically designed features are better than the commonly used features in this HI construction task. Those results and analyses indicate that the quality of the extracted features influences the effectiveness of manual HIs. Therefore, in order to construct an effective manual HI, prior knowledge is needed to design suitable feature extraction algorithms. Meanwhile, in the proposed method, features are automatically learned from the raw vibration signals, which enables the proposed method to construct HI with no need prior knowledge to extract features. From this point, it can be obtained that the proposed method releases us from the tough work of designing effective feature extraction algorithms. 4. Conclusions The manual HI construction methods need prior knowledge to design feature extraction and data fusion algorithms. In order to solve the problem, this paper proposed a feature learning based method to automatically learn features and construct a HI. Additionally, facing the phenomenon of trend burr existing in the constructed HI, an outlier region correction technique was presented to improve the performance of the HI. A bearing dataset was used to demonstrate the effectiveness of the proposed method. Three conclusions can be drawn in this paper: (1) the proposed method automatically constructed HIs with little prior knowledge to extract features; (2) the constructed CHI was better than manual HIs in terms of trendability, monotonicity and scale similarity. Especially, the scale similarity of the CHI almost equaled 1, which indicated the advantage of the CHI in deciding a failure threshold; (3) the outlier region correction technique was able to detect and remove the outlier regions, which improved the trendability, monotonicity and scale similarity of the constructed HIs. Acknowledgments This research was supported by National Natural Science Foundation of China (U1709208 and 61673311), and National Program for Support of Top-notch Young Professionals.
References [1] Y. Lei, Intelligent Fault Diagnosis and Remaining Useful Life Prediction of Rotating Machinery, Butterworth-Heinemann, 2016. [2] F. Jia, Y. Lei, L. Guo, J. Lin, S. Xing, A neural network constructed by deep learning technique and its application to intelligent fault diagnosis of machines, Neurocomputing 272 (2017) 619–628. [3] D. Wang, K.L. Tsui, Theoretical investigation of the upper and lower bounds of a generalized dimensionless bearing health indicator, Mech. Syst. Signal Process. 98 (2018) 890–901. [4] F. Yang, M.S. Habibullah, T. Zhang, Z. Xu, P. Lim, S. Nadarajan, Health index-based prognostics for remaining useful life predictions in electrical machines, IEEE Trans. Ind. Electron. 63 (4) (2016) 2633–2644. [5] N. Li, Y. Lei, L. Guo, T. Yan, J. Lin, Remaining useful life prediction based on a general expression of stochastic process models, IEEE Trans. Ind. Electron. 64 (7) (2017) 5709–5718. [6] Y. Lei, N. Li, L. Guo, N. Li, T. Yan, J. Lin, Machinery health prognostics: a systematic review from data acquisition to RUL prediction, Mech. Syst. Signal Process. 104 (2018) 799–834. [7] K. Javed, R. Gouriveau, N. Zerhouni, P. Nectoux, Enabling health monitoring approach based on vibration data for accurate prognostics, IEEE Trans. Ind. Electron. 62 (1) (2015) 647–656. [8] H. Hanachi, J. Liu, A. Banerjee, Y. Chen, A. Koul, A physics-based modeling approach for performance monitoring in gas turbine engines, IEEE Trans. Reliab. 64 (1) (2015) 197–205. [9] L. Guo, N. Li, F. Jia, Y. Lei, J. Lin, A recurrent neural network based health indicator for remaining useful life prediction of bearings, Neurocomputing 240 (2017) 98–109. [10] H. Qiu, J. Lee, J. Lin, G. Yu, Robust performance degradation assessment methods for enhanced rolling element bearing prognostics, Adv. Eng. Inf. 17 (3) (2003) 127–140. [11] H. Kumar, S.P. Pai, N. Sriram, G. Vijay, Rolling element bearing fault diagnostics: development of health index, Proc. Inst. Mech. Eng. Part C: J. Mech. Eng. Sci. 231 (21) (2017) 3923–3939 0954406216656214. [12] J. Yu, Health condition monitoring of machines based on hidden Markov model and contribution analysis, IEEE Trans. Instrum. Measure. 61 (8) (2012) 2200–2211. [13] J. Yu, Bearing performance degradation assessment using locality preserving projections and Gaussian mixture models, IEEE Trans. Instrum. Measure. 61 (8) (2012) 2200–2211. [14] G.E. Hinton, R.R. Salakhutdinov, Reducing the dimensionality of data with neural networks, Science 313 (5786) (2006) 504–507. [15] L. Deng, D. Yu, Deep learning: methods and applications, Found. Trends Signal Process. 7 (3–4) (2014) 197–387. [16] W. Liu, Z. Wang, X. Liu, N. Zeng, Y. Liu, FE. Alsaadi, A survey of deep neural network architectures and their applications, Neurocomputing 234 (2017) 11–26. [17] A. Alzu’bi, A. Amira, N. Ramzan, Content-based image retrieval with compact deep convolutional features, Neurocomputing 249 (2017) 95–105. [18] F. Jia, Y. Lei, J. Lin, X. Zhou, N. Lu, Deep neural networks: a promising tool for fault characteristic mining and intelligent diagnosis of rotating machinery with massive data, Mech. Syst. Signal Process. 72 (2016) 303–315. [19] Y. Lei, F. Jia, J. Lin, S. Xing, S.X. Ding, An intelligent fault diagnosis method using unsupervised feature learning towards mechanical big data, IEEE Trans. Ind. Electron. 63 (5) (2016) 3137–3147. [20] R. Thirukovalluru, S. Dixit, R.K. Sevakula, N.K. Verma, A. Salour, Generating feature sets for fault diagnosis using denoising stacked auto-encoder, 2016 IEEE International Conference on Prognostics and Health Management (ICPHM), IEEE, 2016. [21] X. Ding, Q. He, Energy-fluctuated multiscale feature learning with deep convnet for intelligent spindle bearing fault diagnosis, IEEE Trans. Instrum. Measure. 66 (8) (2017) 1635–1926. [22] A. Krizhevsky, I. Sutskever, G.E. Hinton, Imagenet classification with deep convolutional neural networks, Adv. Neural Inf. Process. Syst (2012) 1097–1105. [23] O. Janssens, V. Slavkovikj, B. Vervisch, K. Stockman, M. Loccufier, S. Verstockt, R. Van de Walle, S. Van Hoecke, Convolutional neural network based fault detection for rotating machinery, J. Sound Vib. 377 (2016) 331–345. [24] O. Abdeljaber, O. Avci, S. Kiranyaz, M. Gabbouj, D.J. Inman, Real-time vibration-based structural damage detection using one-dimensional convolutional neural networks, J. Sound Vib. 388 (2017) 154–170. [25] T. Ince, S. Kiranyaz, L. Eren, M. Askar, M. Gabbouj, Real-time motor fault detection by 1-d convolutional neural networks, IEEE Trans. Ind. Electron. 63 (11) (2016) 7067–7075. [26] W. Sun, R. Zhao, R. Yan, S. Shao, X. Chen, Convolutional discriminative feature learning for induction motor fault diagnosis, IEEE Trans. Ind. Inf. 13 (3) (2017) 1350–1359. [27] V. Chandola, A. Banerjee, V. Kumar, Outlier detection: a survey, ACM Comput. Surv. (2007). [28] M. Gupta, J. Gao, C.C. Aggarwal, J. Han, Outlier detection for temporal data: a survey, IEEE Trans. Knowledge Data Eng. 26 (9) (2014) 2250–2267. [29] N. Li, Y. Lei, J. Lin, S.X. Ding, An improved exponential model for predicting remaining useful life of rolling element bearings, IEEE Trans. Ind. Electron. 62 (12) (2015) 7762–7773. [30] Y. Wang, Y. Peng, Y. Zi, X. Jin, K.-L. Tsui, A two-stage data-driven-based prognostic approach for bearing degradation problem, IEEE Trans. Ind. Inf. 12 (3) (2016) 924–932.
Please cite this article as: L. Guo et al., Machinery health indicator construction based on convolutional neural networks considering trend burr, Neurocomputing (2018), https://doi.org/10.1016/j.neucom.2018.02.083
JID: NEUCOM
ARTICLE IN PRESS
[m5G;March 13, 2018;3:14]
L. Guo et al. / Neurocomputing 000 (2018) 1–9 [31] A. Radford, L. Metz, S. Chintala, Unsupervised representation learning with deep convolutional generative adversarial networks, preprint arXiv:1511.06434 (2015). [32] B. Zhang, L. Zhang, J. Xu, Degradation feature selection for remaining useful life prediction of rolling element bearings, Qual. Reliab. Eng. Int. 32 (2) (2016) 547–554. [33] P. Nectoux, R. Gouriveau, K. Medjaher, E. Ramasso, B. Chebel-Morello, N. Zerhouni, C. Varnier, PRONOSTIA: An experimental platform for bearings accelerated degradation tests, IEEE International Conference on Prognostics and Health Management, PHM’12, 2012 IEEE Catalog Number: CPF12PHM-CDR. Liang Guo received the B.S. and Ph.D. degrees in mechanical engineering from Southwest Jiaotong University, Chengdu, PR China, in 2011 and 2016, respectively. He is currently working as a Postdoctoral researcher at the State Key Laboratory for Manufacturing System Engineering, Xi’an Jiaotong University, Xi’an, PR China. His current research interests include machinery condition monitoring, intelligent fault diagnostics and remaining useful life prediction.
Yaguo Lei received the B.S. and Ph.D. degrees in mechanical engineering from Xi’an Jiaotong University, Xi’an, PR China, in 2002 and 2007, respectively. He is currently a Full Professor of mechanical engineering at Xi’an Jiaotong University. Prior to joining Xi’an Jiaotong University in 2010, he was a Postdoctoral Research Fellow with the University of Alberta, Edmonton, AB, Canada. He was also an Alexander von Humboldt Fellow with the University of Duisburg-Essen, Duisburg, Germany. His research interests focus on machinery condition monitoring and fault diagnosis, mechanical signal processing, intelligent fault diagnostics, and remaining useful life prediction. Dr. Lei is a member of the editorial boards of more than ten journals, including Mechanical System and Signal Processing and Neural Computing & Applications. He is also a member of ASME and a member of IEEE. He has pioneered many signal processing techniques, intelligent diagnosis methods, and remaining useful life prediction models for machinery.
9 Naipeng Li is currently working toward the Ph.D. degree in mechanical engineering at the State Key Laboratory for Manufacturing System Engineering, Xi’an Jiaotong University, PR China. He received the B.S. degree in mechanical engineering from Shandong Agricultural University, PR China, in 2012. His research interests are machinery condition monitoring, intelligent fault diagnostics and remaining useful life prediction of rotating machinery.
Tao Yan is currently working toward the Ph.D. degree in mechanical engineering at the State Key Laboratory for Manufacturing System Engineering, Xi’an Jiaotong University, PR China. He received the B.S. degree in mechanical engineering from Central South University, PR China, in 2016. His research interests are machinery condition monitoring, intelligent fault diagnostics and remaining useful life prediction of rotating machinery.
Ningbo Li is currently working toward the Ph.D. degree in mechanical engineering at the State Key Laboratory for Manufacturing System Engineering, Xi’an Jiaotong University, PR China. He received the B.S. degree in mechanical engineering from Sichuan University, PR China, in 2015. His current research interests include machinery condition monitoring, remaining useful life prediction and condition-based maintenance of machinery.
Please cite this article as: L. Guo et al., Machinery health indicator construction based on convolutional neural networks considering trend burr, Neurocomputing (2018), https://doi.org/10.1016/j.neucom.2018.02.083
