Support Vector Machine Classification of Ultrasonic Shaft Inspection Data Using Discrete Wavelet Transform Kyungmi Lee∗and Vladimir Estivill-Castro† School of Computing and Information Technology Griffith University Nathan, QLD 4111, Australia Abstract While many non-destructive ultrasonic signal test scenarios involve very shallow surfaces, signals for testing shafts are long and the new problem of mode-converted reflection emerges. They are echoes that do not correspond to cracks in the material, neither to characteristics of the shaft. Also, the length of the signals demands the application of efficient feature extraction mechanism to reduce the dimension of the pattern vectors and make classifier training feasible. The previous study by authors [9] established experimentally that Discrete Wavelet Transform (DWT) provided faster and more reliable feature extraction for Artificial Neural Networks (ANN) in these long signals in shafts. As an extended work, a new comparative experiment involving Support Vector Machines (SVM) approach instead of ANN models are made so that more confidence can be placed in the comparison result especially when there are only a limited number of training examples. The classification result from employing two different feature extraction schemes is also analyzed to investigate whether there exists any special relationship between feature types and class types. This investigation set the baseline for the future work of the authors, which is building a more advanced hybrid classifier model using multiple feature extraction schemes.
Keywords: Feature selection and classification, Support Vector Machines Classification, Signal Processing Applications in Engineering, Discrete Wavelet Transforms
1
in shafts and pins are usually preceded by fatigue cracking and this has been detected mainly through non-destructive testing (NDT) techniques using an ultrasonic A-scan method. However, A-scan signals taken from shafts are more complicated to interpret, compared to Ascans from other machine parts like plate or pipe surfaces which produce shorter and simpler signals. Since A-scans for a shaft are taken from the end of a lengthy shaft, an ultrasonic pulse can travel many paths echoing off the machined surfaces. Moreover, most shafts are often more complex than a plain solid cylinder. Shafts typically have multiple diameter sections or more complex geometry like splines, lubrication or bolt holes. Thus, the many reflections from these machined features render it difficult to correctly attribute signals to defects. Also, the critical faint echoes from a cracked surface may lay somewhere among the multiple secondary echoes. These multiple secondary echoes result from multiple instances of mode conversion between a shear wave and a compression wave as the ultrasonic wave travels the length of the shaft in various angles from the source depending on the diameter of the shaft. Figure 1 presents the principle of generating mode-converted echoes by showing some example lines of wave propagation of ultrasonic pulse on a lengthy shaft. These modeconverted echoes causes a misleading indication of the position of features.
INTRODUCTION
The early detection of conditions that lead to inservice failures of rotating of shafts or fixed pins is necessarily required because they are susceptible to failures without apparent warning. Failures ∗
[email protected] †
[email protected]
Figure 1: Ultrasonic pulse generating modeconverted echoes in shaft testing.
Conventional NDT techniques, which are based on the heuristic experience-based pattern identification methods, bring about costly, lengthy and erratic analysis and thus lead to inconsistencies in results. To solve this, various modern signal processing techniques and artificial intelligence tools have been adopted [8][19] and these approaches are integrated as automatic ultrasonic signal classification (AUSC) systems. In an AUSC system, ultrasonic flaw signals acquired in a form of digitized data are preprocessed and informative features are extracted using various digital signal processing techniques. The set of selected features becomes the basis of decision-making for classification. Like most other pattern recognition and classification matters, a main interest to AUSC research communities is to extract the better set of features with which classification might be performed more efficiently. As the relationship between ultrasonic signal characteristics and flaw classes is not straightforward, various digital signal processing techniques are used for the determination of features from the raw signal. Among various ultrasonic feature extraction schemes which have been previously proposed, Fast Fourier Transform (FFT) has been known as a useful scheme for extracting frequency-domain signal features [2][11]. More recent studies on the ultrasonic flaw classification employ DWT as a part of their feature extraction scheme as DWT provides effective signal compression as well as time-frequency presentation [12][17]. In order to seek for the better set of features for AUSC, many researchers have made a comparison between these two feature extraction schemes (FFT and DWT). More specifically, they employed two different preprocessing techniques using coefficients of FFT and coefficients of DWT in order to extract feature sets, and compared the classification performance using each feature set. As most research involving AUSC has focused on classifying short signals taken from plate or pipe surfaces and distinguishing any flaw they have, most comparison studies between FFT and DWT also have been done for these shallow signals and most results showed a superiority of DWT to FFT [14][15][20]. Our previous study [9] investigated whether DWT outperforms FFT at extracting features in ultrasonic signals from shafts which were complexity than shallow signals. The study also established experimentally that DWT had better potential as feature extraction scheme for training ANNs with arbitrary training data and using the ANN models for in-field ultrasonic shaft signal classification. However, a problem issued is that the superior-
ity of DWT was validated by the comparison of its classification performance with FFT’s only through ANN classifiers. ANNs have been popularly employed as a classifier for AUSC due to its ability of making non-linear decision boundaries. However, considering many problems inherent in the ANN learning method such as generalization control, overfitting and parameter tuning, we need to be careful confirming DWT’s predominance. In order to assure the potential of DWT as an efficient feature extraction scheme for ultrasonic shaft signal classification, we made a new comparative experiment involving SVM approach instead of ANN models. SVM has gained the reputation for the generalization control capability thus avoidance of overfitting, and thus, more confidence can be placed in the comparison result using SVM rather than using ANN, especially when there are only a limited number of training examples. The aim of this article is to report our investigation into whether DWT can outperform FFT at extracting fetures in ultrasonic signals from shafts through SVM modelling so that the validation of the superiority of DWT becomes more reliable. The classification result of both scheme is also analyzed to investigate whether different feature extraction scheme affects the classification performance in different class. The analysis and investigation result obtained forms the basis of the future work of the authors, which is building a more advanced hybrid AUSC system using both feature schemes. The rest of paper is organized as follows. Section 2 explains the overall procedure of our experiment and details the steps involved in the experiment. That is, it describes the procedure of preprocessing and constructing a database for our experiment, and applying SVM learning model. Section 3 analyzes the experimental result of both schemes and compares and discusses their performance, followed by conclusions in Section 4.
2 2.1
EXPERIMENT Overall Process
The experimental setting consists of three main processes. Firstly, signal segments of interest are selected from the whole ultrasonic A-scan signals. In order to apply a consistent way of signal segmentation which is necessary for suppressing timevariance problem of DWT, we used a systematical echo capturing method with zero-padding, which were proposed in previous works [10]. We also recorded whether the signal was originated from
a crack or not and whether it was a primary or secondary (mode-converted) echo. We used SHAFTEST TM 1 as a tool to capture these A-scans and build up an initial database of required echoes. And then, two different feature extraction techniques (FFT and DWT) were applied to extract informative features from the time-domain signals. Once an initial database had been built up, this database was mapped to feature domains based on two different feature extraction techniques, generating two different sets of training data. These two sets of training data were then used for training two SVM models and their classification performance was validated through cross-validation. Figure 2 summarizes the whole procedure of our experiment as a flow chart. A more detailed description is presented in following subsections.
2.2
Preprocessing and Feature Extraction
We collected data from eight different shafts ranging between 100mm to 1300mm long. Five of these shafts contained cracks and the rest were clean shafts. Mode-converted echoes were captured from these shafts, except for one 100mm calibration block which did not produce mode-converted echoes because of its geometry.2 A-scan signals were acquired from seven shafts by using 2 MHz frequency. For the selection of signal region of interest, our previous research paper [10] has proposed a new methodology for gating a signal section to single out an echo, which plays an effective role in overcoming the time-variance problems of DWT. In this way, a border of echoonly area for every signal echo is decided by applying a consistent systematic rule, and the affection of the background noise is minimized by excluding the neighboring background grass area through zero-padding. Through this process, we produced a set of time-domain signal vectors of 768 samples long and they were downsampled into 256 samples long to reduce the dimensions in the SVM classifier. For every pattern vector, whether the echo was caused by CRack(CR), Mode-Conversion(MC) or Backwall-Reflection(BR), was recorded. Among these three causes of echoes, BR is considered to be easily distinguished compared to other factors 1 It
is a trade mark of CCI Pope. this size shaft (75mm diameter × 100mm length), there is no geometric possibility for the shear waves to reflect and convert back to compression waves before they are simply absorbed, because the shear waves are generated at 33◦ 2 For
CR or MC [2]. Practically, one of the issues of concern in ultrasonic shaft inspection is that the signal echoes from CR can be confused by fainted echoes from MC and vice versa. Therefore, in our experiments, different feature extraction schemes are compared by the classification performance in classifying the received signal echoes just as those from CR or those from MC. For this purpose, each 50 CR signals and 69 MC signals were chosen randomly from the sample pool for the experiment. Thus, in total a data set with 119 time-domain signals became the basis for further processing.
2.3
Feature Extraction
The next step was converting the time-domain data into different domain feature set which would later become the input to the SVM classifiers. For our purpose, time-domain signal data, which was gained through preprocessing explained in Section 2.2, was converted into the frequency domain using FFT and also into the time-frequency domain using DWT. Although FFT is used as a main idea for the conversion from time-domain to frequency-domain, various FFT-based features can be extracted and they have been used for various ultrasonic signal classification [1][16][18][21]. More recently, instead of using a set of extracted features from frequency domains, a whole signal section derived from the ultrasonic scans employing a minimum of preprocessing using only Fast Fourier Transform (FFT), has been used directly as inputs into the classifier [2][11]. By the way, the claims in previous studies from similar scenarios on the superiority of DWT coefficients to FFT coefficients as features for the ultrasonic signal classification is subject to debate. The reason is that most previous comparison studies used only the magnitude components of the FFT-transformed signals and their phase components were naturally excluded through the process of using FFT coefficients as feature vectors for a classifier. To overcome this problem, a new FFTbased feature extraction scheme was proposed in our previous research paper [9]. The new proposed method is to extract phase components as well as magnitude components of each FFT coefficients and form a new set of FFT feature vectors which effectively represent both components of FFT coefficients. The study also showed that employing the newly proposed FFT feature extraction scheme enhanced the performance of ultrasonic shaft signal classification compared to using the original FFT-
Figure 2: Flow chart of overall procedure of our experiment. based scheme. Therefore, in order to make a more fair comparison between FFT and DWT as feature extraction schemes for SVM classification of ultrasonic shaft signals, we also applied the newly proposed method to the construction of the FFT-based feature vectors in this experiment. For the comparison in this experiment, the preprocessed time-domain data are once more downsampled into 128 points and then transformed by FFT. The sequence of FFT of a signal is a sequence of complex numbers which consists of magnitude and phase of the transformed sequence. As the last half of the sequence of FFT corresponds to complex conjugates of the components in the first half of the sequence, FFT of 128 points long signal yields 64 magnitude components and 64 phase components. The concatenated sequences of magnitude components and phase components become 128 dimensional pattern vectors for SVM classification. On the other hand, the originally preprocessed 256 points long time-domain signal patterns are also converted by DWT. The idea of DWT starts from decomposing discrete time signals into their coarse and detail information by passing them through a high-pass and a low-pass filter respectively. This decomposition is repeated several times
only for low-pass filtered signals, and the output from high-pass filtering in each decomposition level, presented as DWT coefficients, are concatenated starting with the coarsest coefficients. We compressed the 256 points long DWT coefficients into 128 samples by discarding the last 128 coefficients as these were supposed not to contain much information but mainly noise, and these 128 long DWT coefficients were stored as DWT feature sets. For our experiment, we applied Daubechies wavelets [4] for filtering.
2.4
SVM Classification
As mentioned before, this paper aims at investigating whether DWT shows superiority to FFT as a feature extraction scheme for the ultrasonic shaft test signals using SVM classification approach. Our previous study [9] reported that DWT outperformed FFT at extracting features in ultrasonic signals from shafts by comparing their classification performance using ANN classifier models. However, it is important to have enough training patterns for effective ANN classifier learning. For a more reasonable comparison between FFT and DWT, therefore, it is required to employ a more efficient classifier in order to avoid the danger of over-
fitting inherent in high dimensional feature spaces and we chose SVM classifier for our experiment in this paper for this reason. SVM has recently gained wide acceptance as powerful supervised learning algorithm based on statistical learning theory and they have shown remarkably robust performance in several pattern recognition applications [6][7][13]. The most important property SVM offers is the generalization control capability, the avoidance of overfitting, which is very important concerns in defect detection systems like AUCS for shaft test [3]. For our experiment, we employed Radial Basis Function (RBF) as kernel for constructing SVM classifiers considering RBF provides nonlinear mapping, requires comparatively small numbers of hyperparameters and has less numerical difficulties. Two parameters; penalty parameter C and kernel parameter γ are required for RBF kernel. Among several previously suggested methodology for searching the best parameters in SVM classification model, we used, for our classification problem, a grid searching algorithm [5] where pairs of C and γ are tried and the one with the best 10-foldcross-validation accuracy is picked. Following this method, we set 16.0 on C and 0.03125 for γ for DWT-based patterns while setting 4.0 on C and 0.03125 for γ for FFT-based patterns. The input samples were shuffled and randomly divided up into 5 sets. In turn, 4 of these were used to train the SVM classifier, and the remaining set was used to validate the constructed SVM classifier. This was repeated with all 5 possible combinations and furthermore, the process was repeated 10 times to get an average of the training ability of the SVM models. This 5 fold cross-validation test was carried out on two SVM models, which are trained by two feature sets: FFT and DWT. The performance of the two different schemes can be compared by the result of each cross-validation test.
3
RESULTS AND ANALYSIS
The results of using two feature extraction mechanisms for ultrasonic shaft signal classification using SVMs are compared through 5-fold cross validation tests (totally 50 tests for each SVMs). For the comparison, we recorded the classification accuracy for each of 5-fold cross validation tests and also analyzed how many instances of each class (CR and MC) were classified correctly. The classification results in percentage are summarized in Table 1 by
presenting the average result values from ten different tests for each run of 5-fold cross validation tests. The visual comparison between the classification performance of using two different feature extraction schemes is offered by Figure 3 which shows histograms presenting relative values of the results shown in Table 1.
Table 1: Classification percentage of two feature extraction schems: FFT vs DWT.
Figure 3: Relative comparison between feature extraction schemes: FFT vs DWT and between different classes: CR vs MC. The following points are noteworthy. • The SVM classification performance using DWT is superior to FFT’s in every different test run, which matches the result of the previous comparison applying ANN models. It implies that DWT is more reliable feature extraction scheme, that is it can be employed for constructing a better classifier for in-field ultrasonic shaft signal test. • As shown in Table 1 and Figure 3, DWT shows better performance in classifying CR than MO while FFT shows a reverse performance. This result provides a potential as a useful priory knowledge when constructing a hybrid AUSC system for testing shafts using a combined feature extraction scheme.
• It has been assumed that extra feature selection process is required for avoiding the wellknown time-variance problem of DWT coefficients. However, our experimental result reassures that DWT coefficients can be efficiently used as feature vectors without much preprocessing.
4
CONCLUSION
In this paper we have demonstrated there are benefits to be gained by using DWT as feature extraction scheme in ultrasonic shaft signal classification. Especially by employing SVM approach as a learning model, the superiority of DWT is validated more reliably. The experiment result also shows different aspects in classification ability of two different feature schems: FFT and DWT. This implies the possibility of constructing a more advanced AUSC system for testing shafts using multiple feature extraction schemes.
References [1] L. M. Brown and R. DeNale. Classification of Ultrasonic Defect Signatures Using An Artificial Neural Network. Review of Progress in Quantitative Nondestructive Evaluation, 10A:705–712, 1991. [2] G. Cotterill and J. Perceval. A New Approach to Ultrasonic Testing of Shafts. In Proceedings of the 10th Asia-Pacific Conference on NonDestructive Testing (APCNDT), 2001. [3] Nello Cristianini and John Shawe-Taylor. An Introduction to Support Vector Machines and other kernel-based learning methods. Cambridge University Press, 2000. [4] I. Daubechies. Orthogonal bases of compactly supported wavelets. Commun. Pure Appl. Math., 41:909–996, 1988. [5] Chih-Wei Hsu, Chih-Chung Chang, and ChihJen Lin. A practical guide to support vector classification. [6] D. K. Iakovidis, D. E. Maroulis, S. A. Karkanis, and I. N. Flaounas. Color Texture Recognition in Video Sequences using Wavelet Covariance Features and Support Vector Machines. In Proceedings of the 29th Euromicro Conference, pages 199–205, 2003.
[7] D. A. Karras, S. A. Karkanis, D. Iakovidis, D.E. Maroulis, and B.G. Mertzios. Support Vector Machines for Improved Defect Detection in Manufacturing Using Novel Multidimensional Wavelet Feature Extraction Involving Vector Quantization and PCA Techniques. In Proceedings of NIMIA Advanced Study Institute on Neural Networks for Instrumentation, Measurement and Related Industrial Applications, pages 139–144, 2001. [8] G. Katragadda, S. Nair, and G. P. Singh. Neuro-Fuzzy Systems in Ultrasonic Weld Evaluation. Review of Progress in Quantitative Nondestructive Evaluation, 16:765–772, 1997. [9] K. Lee and V. Estivill-Castro. Classification of Ultrasonic Shaft Inspection Data Using Discrete Wavelet Transform. In Proceedings of the IASTED international conferences on Artificial Intelligence and appliction, pages 673–678. ACTA Press, 2003. [10] K. Lee and V. Estivill-Castro. Feature Extraction Techniques for Ultrasonic Shaft Signal Classification. In Proceedings of the International Conference on Hybrid Intelligent Systems, pages 479–488. IOS Press, 2003. [11] F. W. Margrave, K. Rigas, D. A. Bradley, and P. Barrocliffe. The use of neural networks in ultrasonic flaw detection. Measurement, 25:143– 154, 1999. [12] M. S. Obaidat, M. A. Suhail, and B. Sadoun. An intelligent simulation methdology to characterize defects in materials. Information Sciences, 137:33–41, 2001. [13] E. Osuna, R. Freund, and F. Girosi. Training support vector machines:an application to face detection:an application to face detection. In Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR 97), pages 130–136, 1997. [14] R. Polikar, L. Udpa, S. S. Udpa, and T. Taylor. Frequency Invariant Classification of Ultrasonic Weld Inspection Signals. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 45(3):614–625, May 1998. [15] D. Redouane, K. Mohamed, and B. Amar. Flaw Detection in Ultrasonics Using Wavelets Transform and Split Spectrum. In Proceedings of the 15th World Conference on NonDestructive Testing, 2000.
[16] J. B. Santos and F. Perdig˜ ao. Automatic defects classification - a contribution. NDT & E International, 34:313–318, 2001. [17] G. Simone, F. C. Morabito, R. Polikar, P. Ramuhalli, L. Udpa, and S. Udpa. Feature extraction techniques for ultrasonic signal classification. In Proceedings of the 10th Int. Symposium on Applied Electromagnetics and Mechanics (ISEM 2001), 2001. [18] S. J. Song, H. J. Kim, and H. Cho. Development of an intelligent system for ultrasonic flaw classification in weldments. Neuclear Engineering and Design, 212:307–320, 2002. [19] S. J. Song, H. J. Kim, and H. Lee. A systematic approach to ultrasonic pattern recognition for real-time intelligent flaw classification in weldments. Review of Progress in Quantitative Nondestructive Evaluation, 18:865–872, 1999. [20] J. Spanner, L. Udpa, R. Polikar, and P. Ramuhalli. Neural networks for ultrasonic detection of intergranular stress corrosion cracking. The e-Journal of Nondestructive Testing And Ultrasonics, 5(7), July 2000. [21] M. Thavasimuthu, C. Rajagopalan, P. Kalyanasundaram, and B. Raj. Improving the evaluation sensitivity of an ultrasonic pulse echo technique using a neural network classifier. NDT & E International, 29(3):175–179, 1996.
