Application of support vector machine for pattern classification of ...

STRUCTURAL CONTROL AND HEALTH MONITORING

Struct. Control Health Monit. 2015; 22:903–918 Published online 1 December 2014 in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/stc.1724

Application of support vector machine for pattern classification of active thermometry-based pipeline scour monitoring Xuefeng Zhao1,*,†, Weijie Li1, Lei Zhou2, Gangbing Song3, Qin Ba1, Siu Chun Michael Ho3 and Jinping Ou1,4 1

3

School of Civil Engineering, Dalian University of Technology, Dalian, 116024, China 2 Engineering Company, Offshore Oil Engineering Co., Ltd Tianjin, 300451, China Department of Mechanical Engineering, University of Houston, Houston, TX 77204-4006, USA 4 School of Civil Engineering, Harbin Institute of Technology, Harbin, 150090, China

SUMMARY Pipeline scour monitoring is becoming one of the key requirements in oil and gas industry. To implement scour monitoring for offshore pipeline, a monitoring system that based on active thermometry is proposed. Our previous investigations have shown that the system has provided many advantages over traditional scour monitoring methods. In this paper, a novel scour automatic detection scheme based on nonlinear curve fitting and support vector machine (SVM) is proposed to realize automatic diagnosis of pipeline scour. On account of the varied heat transfer patterns of a line heat source in sediment and water scenarios, the experimental temperature profiles are nonlinearly fitted to their theoretical models. Features extracted by nonlinear curve fitting can dramatically reduce the dimensions of the data. Subsequently, the extracted features are inputted into SVM classifier to judge where the pipeline is exposed to water or buried in the sediment. In order to evaluate the performance of SVM, SVM with different kernel functions are compared with the back-propagation neural networks, which is the most popular neural network for pattern recognition and classification. Results show that the SVM model with radial basis function kernel outperformed other classification models. Finally, aiming to obtain the optimal heating time of the system, the optimal SVM model is employed to recognize datasets with different heating time. Copyright © 2014 John Wiley & Sons, Ltd. Received 18 May 2013; Revised 22 September 2014; Accepted 4 November 2014 KEY WORDS:

scour monitoring; offshore pipeline; active thermometry; support vector machine; pattern classification

1. INTRODUCTION Nowadays, offshore pipelines play an important role in the development of oil and gas industry all over the world. To ensure safer performance of offshore pipelines, pipelines would often be embedded in a trench. Over a rough seabed or on a seabed subjected to local turbulent flow, pipeline scour is inevitable. With the development of pipeline scour, the pipeline is firstly exposed to direct sea flow and subsequently free-spanned over an appreciable distance, which consequently suffer from fatigue damage because of similar value between natural frequency of free span and vortex shedding frequency [1]. Free-spanned pipelines even undergo threats posed by third-party activities, such as impacts of anchors, nets and hulls of cargo, fishing, and drilling rigs [2]. Failure of pipeline can potentially cost millions of dollars and lead to serious environmental damage caused by the leaking products, even the loss of human lives. Hence, catastrophic outcomes of pipeline breakdown due to scour are calling for better and more efficient methods to monitor scour condition and predict the failure of offshore pipeline.

*Correspondence to: Xuefeng Zhao, School of Civil Engineering, Dalian University of Technology, Dalian, 116024, China. † E-mail: [email protected]

Copyright © 2014 John Wiley & Sons, Ltd.

904

X. ZHAO ET AL.

The traditional methods [3–6] for offshore pipelines scour monitoring are vibration-based methods that mainly focus on the indirectly metering the free-spanning vibration, which are incapable of detecting scour conditions if the vortex-induced vibration is weak. These methods encounter construction problem because sensors need to be embedded into the pipeline to measure vibration signals. To overcome drawbacks of conventional scour monitoring methods, we proposed a novel active thermometrybased nondestructive testing method for scour monitoring of offshore pipelines. Working on the principle of transient line source method, active thermometry was found to be very effective in measuring thermal properties of materials, such as thermal conductivity, thermal resistivity, specific heat, and soil water content [7–9]. By using active thermometry, the system was composed of thermal cables, data acquisition unit (DAU), and data processing unit. Thermal cable was further broken into a heating belt, distributed optical fiber temperature sensors and packaging elements. Our first study [10] proved the feasibility of the proposed scour monitoring method, whose temperature reading is based on distributed Brillouin optic sensing techniques. The second study [11] developed a DS18B20 temperature sensing based scour monitoring system for near-shore environments and proposed a three-index estimator to identify ambient mediums along pipeline. Subsequent numerical and experimental investigation of the heat transfer behavior of the system showed good agreement between numerical simulation and experiment results [12]. More recently, a DS18B20 sensor network for near-shore scour monitoring and its detection algorithm based on K-means clustering was investigated [13]. Our previous studies verified that active thermometry-based pipeline scour monitoring system was direct in scour measurement, convenient in installment, and exhibited high precision. However, an overall automatic detection scheme has not yet been developed to identify the different ambient environments by which the pipeline may be surrounded. Ambient environment for offshore pipelines is a typical binary condition, either buried in sediment or exposed to sea water. As the heat transfer behaviors of a line heat source varies in sediment and water media, temperature profiles with time show two different patterns. Previously, we determined the scour condition of pipeline by manual inspection. In practice, the substantial data volume requires an efficient automatic detection algorithm, which is capable of classifying the binary ambient medium along pipeline based on the various temperature profiles. The support vector machine (SVM), a variant of artificial neural network (ANN) algorithm, which was introduced by Vapnik [14], has emerged as an effective tool for solving classification and regression problems. SVM provides a practical solution to the automatic detection of offshore pipeline scour monitoring system. Machine learning approaches such as SVM and ANN have been increasingly applied in many domains including health monitoring of pipelines. Belsito et al. developed a system for monitoring leak size and location using ANN [15]. The system successfully detected leaks as small as 1% of flow rate in pipelines and correctly localized leaks with a probability of success greater than 50% for small leaks. Chen et al. formulated the negative pressure wave detection approach and employed SVM to monitor the presence of leaks [16]. Results showed that the system provided better performance that outperformed the Wavelet-based method. Qu et al. presented SVM-based pipeline leakage detection and pre-warning system [17]. The SVM was used to identify vibration signals generated because of leakage. Etemad-Shahidi et al. studied the usage of M5 model tree to predict wave-induced scour depth under submarine pipelines [18]. The approach exhibited high performance for both live bed and clear water conditions, thus reducing error statistics. Lee et al. proposed a novel classification technique, the Euclidean-SVM, to classify data from pipelines, which were monitored using long range ultrasonic transducers [19]. Independent of the choice of kernel function and parameters selection, the Euclidean-SVM was suitable to be used in continuous monitoring of pipeline data. However, very few researchers have investigated the suitability of SVM for offshore pipeline scour monitoring. Inspired by aforementioned approaches, this paper implements SVM for investigation of pipeline scour with the aim to achieve automatic detection of scour along pipeline. The determination of where the pipeline is exposed to water or buried in sediment is a key outcome of the scour monitoring system. In this system, the temperature profiles with time along pipeline are measured by distributed optical fiber temperature sensors. Subsequently, features of temperature profiles can be extracted by performing nonlinear curve fitting. The SVM will be trained with datasets from both sediment scenario and water scenario. During the training phase, different combination of Copyright © 2014 John Wiley & Sons, Ltd.

Struct. Control Health Monit. 2015; 22: 903–918 DOI: 10.1002/stc

SVM FOR PIPELINE SCOUR MONITORING

905

kernel functions and penalty parameter C will result in various classification accuracies. The SVM model with the best classification performance is employed as classifier in the system to recognize scour conditions along the pipeline. The performance of SVM is compared against back-propagation neural networks (BPNNs), which is the most popular in all of the neural network applications and has advantages of yielding high classification accuracy. In final, the selected optimal SVM model will be applied to implement system optimization.

2. SUPPORT VECTOR MACHINE Support vector machine is a machine learning algorithm developed to efficiently train linear learning machines in kernel-induced feature spaces by adopting the generalization theory of Vapnik [14]. SVM learning is based on minimizing the structural risk against minimizing the empirical risk [20]. The capability of the method to generalization to unseen data is achieved by trading off between minimizing the training error and minimizing the Vapnik–Chervonenkis dimension [21]. Given two data points of classes, SVMs strive to find an optimal hyperplane. Figure 1 shows different hyperplanes separating two classes, but the one illustrated in Figure 2 gives the maximum margin, which is called the optimal hyperplane [22]. Consider a training dataset containing N number of samples represented by {xi,yi}(i = 1, …, N), where x ∈ Rk is a k-dimensional space and yi ∈ {1, + 1} is a class label. As can be seen from Figure 2, the optimal hyperplane is defined as w xi + b = 0, where x is a point lying on the hyperplane, parameter

Figure 1. Hyperplanes for linearly separable data.

Figure 2. Support vectors and optimal hyperplane for the binary case of linearly separable datasets. Copyright © 2014 John Wiley & Sons, Ltd.


906

X. ZHAO ET AL.

w the adjustable weight vector, and b the bias. For the linearly separable case, a separating hyperplane can be defined for two classes as wxi þ b ≥ þ1 for yi ¼ þ1

(1)

wxi þ b ≤ 1 for yi ¼ 1

(2)

The aforementioned two equations can be combined into a single inequality: yi ðwxi þ bÞ 1 ≥ 0 ∀ i ¼ 1; …; k

(3)

The training data points on these two hyperplanes, defined by the functions w xi + b = ± 1, are the support vectors. The classes are said to be linearly separable if a hyperplane is described in Equation (3). The margin between these planes is equal to 2/∥w ∥. As the distance between hyperplane and the closest point is 1/∥w ∥, the optimal separating hyperplane can be acquired by minimizing the norm of ∥ w ∥ under the constraints of Equation (3). Finding the optimal hyperplane becomes the following optimization problem: 1 min ∥w∥2 2

(4)

subject to yi(w xi + b) 1 ≥ 0. We can solve this quadratic optimization problem by finding the saddle point of the Lagrange function as follows: 1 min Lp ðw; b; αÞ ¼ ∥w∥2 ∑Ni¼1 αi ½yi ðwxi þ bÞ 1 2

(5)

where αi are the Lagrange multipliers, hence αi ≥ 0. By differentiating with respect to w and b, the following equations are obtained: ∂Lp ¼ 0; w ¼ ∑Ni¼1 αi yi xi ∂w

(6)

∂Lp ¼ 0; ∑Ni¼1 αi yi ¼ 0 (7) ∂b By substituting Equations (6) and (7) into Equation (5), the optimization problem becomes the following expression: 1 max Lp ðw; b; αÞ ¼ ∑Ni¼1 αi ∑Ni1 ∑Nj αi αj yi yj xTi xj 2

(8)

under the constrains ∑NI¼1 αi yi ¼ 0 and αi ≥ 0. The Karush–Kuhn–Tucker optimality conditions are introduced in solving the optimal value of w and b. w ¼ ∑Ni¼1 αi yi xi

(9)

b ¼ yi wxi

(10)

Once w and b are acquired, the linear decision function is given by f ðxÞ ¼ sign ∑Ni¼1 ∑Nj¼1 αi yi xi ; ; xj þ b

(11)

For a nonlinearly separable case, datasets cannot be classified into two classes with a linear function in input space, as shown in Figure 3. The objective is to find a hyperplane that creates minimum errors; therefore, a slack variables ξ i and penalty variable C are introduced [23] as follows: XN 1 min ∥w∥2 þ C i¼1 ξ i 2 subject to yi(w xi + b) + ξ i 1 ≥ 0 and ξ i ≥ 0. Copyright © 2014 John Wiley & Sons, Ltd.

(12)


907


Figure 3. Map the nonlinear datasets to the high-dimensional space with a kernel function.

Table I. Formulation of commonly used kernel functions. K(xi, xj)

Kernel Linear Polynomial Radial basis function Sigmoid

T xi xTj γxi xj

d þ r ;γ > 0 2 exp(γ||x T i xj|| ), γ > 0 tanh γxi xj þ r

Because it is not able to determine the hyperplane by linear equations, the datasets are projected into a high-dimensional feature space by nonlinear mapping functions ø, as shown in Figure 3. Again, Lagrange technique is applied to Equation (12), and we have a similar dual quadratic optimization, 1 QðαÞ ¼ ∑Ni¼1 αi ∑Ni¼1 ∑Nj¼1 αi αj yi yj øT ðxi Þø xj 2

(13)

under the constraints ∑Ni¼1 αi yi ¼ 0 and 0 ≤ αi ≤ C. K(xi, xj) = øT(xi)ø(xj) is called the inner-product kernel function. Thus, Equation (13) becomes 1 (14) QðαÞ ¼ ∑Ni¼1 αi ∑Ni¼1 ∑Nj¼1 αi αj yi yj K xi ; xj 2 Followed by steps described in linear case, we obtain decision function of the following form: (15) f ðxÞ ¼ sign ∑Ni¼1 ∑Nj¼1 αi yi K xi ; xj þ b There are four commonly used kernel functions in SVM, namely, linear, polynomial, radial basis function (RBF), and sigmoid [24]. Their formulas and parameters are summarized in Table 1. A commonly used method for kernel selection is the cross-validation method [25].

Figure 4. Schematic diagram of the scour monitoring system. Copyright © 2014 John Wiley & Sons, Ltd.


908

X. ZHAO ET AL.

Figure 5. Thermal cable placement (left) and the running water environment (right).

3. METHODOLOGY 3.1. System overview The scour monitoring system consists of thermal cables, Brillouin optical time-domain analysis analyzer, which acted as the DAU, and computer, which acted as the data processing unit, as depicted in Figure 4. Among them, the thermal cable was further broken into a heating belt, distributed optical fiber temperature sensors and packaging elements. In parallel with the pipeline, one of the thermal cables was placed on the upper surface of the pipeline to detect exposure, whereas the other one was positioned in the lower surface to monitor free span. Experiments were conducted on a scaled pipeline model. The 18-m-long pipeline, which had a diameter of 100 mm and a thickness of 2.5 mm, was placed in a 48-m-long indoor experimental flume. The flume was 1 m wide and 1.5 m high, as shown in Figure 5. Water was continuously added to the flume to simulate a running water environment. The pipeline was buried in sediment initially, and then, it was artificially exposed and free-spanned to test the system, as illustrated in Figure 6. Experiments were performed with different exposure and freespanning lengths, namely, 2 m, 4 m, 6 m, and multiple cases with 2.5 m and 6 m. During the experiments, the heating belts were connected to a power supply to generate heat for 3 h; at the same time, temperatures along the pipeline were measured by the distributed optical fiber temperature sensors and acquired by the Brillouin optical time-domain analysis analyzer with a sampling interval of 0.41 m. After heating for 3 h, the power was disconnected to allow a cool down of 2 h. Lastly, temperature signals were processed by a computer to identify the scour condition and location of the pipeline. More details about the system and experiments were illustrated in our previous work [12].

Figure 6. Pipeline upper surface exposure experiments (left) and free-spanning experiments. Copyright © 2014 John Wiley & Sons, Ltd.



909

Figure 7. Temperature profiles of different distance with time in (a) 2 m, (b) 4 m, (c) 6 m, and (d) 2.5 + 6 m freespanning experiment.

Figure 7 illustrates the temperature profiles of different distance varied with time under experiments of different free-spanning lengths. There is a ditch of roughly 2 m in Figure 7(a), representing the temperature profiles in water scenario, which is corresponding to free-spanning section of pipeline. Sections of increasing temperature in heating stage are corresponding to sediment scenario. Figure 7(b) and (c) is corresponding to 4 m and 6 m free-spanning experiments, respectively. The temperatures of multiple free-spanning texts are shown in Figure 7(d), where it shows two ditches. As can be seen, an obvious temperature gap can be found between sediment scenario and water scenario. Temperatures in sediment scenario were higher, while those in water scenario, temperatures were lower because of the various heat transfer patterns of line heat source in sediment and water scenarios. Typically, heat transfer of a line heat source placed in sediment scenario was conduction but convection in water scenario. These two distinct heat transfer behaviors contributed to different temperature histories. As the heating belt started generating heat, temperatures in both sediment and water rose rapidly. Throughout the heating period, temperatures in sediment kept increasing asymptotically. However, parts of the pipeline exposed to water quickly reached a plateau and presented no obvious changes. These differences in temperature profiles make the sediment and water scenarios distinguishable. Previously, the scour condition of the pipeline was determined by manual inspection. In practice, substantial data require an efficient automatic detection algorithm, which is capable of classifying the binary ambient medium along the pipeline because of the various temperature profiles. In this study, SVM was employed as the classification algorithm to facilitate the automatic detection of pipeline scour. In the first place, the practical problem should be translated into the mathematical model, which can be solved by SVM. This is the model selection process, including feature extraction, selection of SVM, and selection of SVM kernel functions and parameters. 3.2. Feature extraction by nonlinear curve fitting method Theoretically, heat transfer behaviors for a line heat source (heating belt) in solids and liquids are different. For sections buried in sediment, heat transfer is by way of conduction. According to ‘transient heat method’ [26], during heating period, the excess temperature ΔT as a function of time t at a radial distance r from the line source is given by q 4α lnt þ ln 2 γ (16) ΔT ¼ 4πλ r where ΔT = T T0; T0 is the initial temperature; γ is the Euler’s constant (γ = 0.5772); q is the heat input Copyright © 2014 John Wiley & Sons, Ltd.


910

X. ZHAO ET AL.

per unit length of the line source during heating; α is the thermal diffusivity of the solid (α = λ/ρc); λ, ρ, and c are the thermal conductivity, the density, and the specific heat of the solid, respectively; and r is the radial distance from the line source. When the heat source stops at time t1, the relation becomes q t ln ΔT ¼ (17) 4πλ t t1 For sections exposed to water, heat transfer occurs by means of convection. For t ≤ t1, the solution is [27], q t (18) 1 exp ΔT ¼ hA τc where τ c = ρcV/hA is the time constant; h is the convective heat transfer coefficient; ρ and c are the density and the specific heat, respectively; and A and V are the convective area and volume per unit length, respectively. For t > t1, the solution is t t1 (19) ΔT ¼ ðT ðt 1 Þ T 0 Þexp τc As expressed in Equation (16), temperatures in sediment scenario had a logarithmic relationship with heating time. However, as indicated in Equation (18), temperatures in the water scenario exponentially reached a plateau (q/hA) and remained stable with time. Conceptually, fitting the temperature histories in a specific scenario to its corresponding mathematical model can yield corresponding thermal coefficients and better fit. For example, using sediment temperature histories to fit the corresponding mathematical model [Equation (16)] creates better fit and less fitting error. However, using the same data to fit the disparate mathematical model [Equation (18)] generally produces worse fit and larger fitness error. With this in mind, the fitting parameters can be selected as input parameters for SVM to classify the sediment and water scenarios. Because the system needs several more hours to settle, when conducting nonlinear curve fitting, we ignore the cool down temperature for convenience. 3.3. Comparative algorithm Currently, the most common neural network model is a multilayer feed-forward neural network based on error back-propagation learning, for instance, BPNN. BPNN has attracted many researchers and has emerged as the most popular tool for pattern recognition and classification. The network model consists of an input layer, an output layer, and one or more hidden layers between them. The adjacent layers achieve full connectivity between neurons, but there is no connection between neurons in the same layer. The algorithm uses the gradient descent method to minimize the total error of the output computed by the network. The network functions as follows: An input pattern is given to the input layer of the network during the training phase. Based on the given input pattern, the network will calculate the output in the output layer. Then, the output is compared with the desired output pattern. The goal of the backpropagation learning rule is to define a method of adjusting the weights of the networks. Ultimately, the network will produce the output that matches the desired output pattern given any input pattern in the training set. For neuron j, the input Ij and output Oj are determined by X Ij ¼ w O (20) i ij i Oj ¼ f I j þ θ j

(21)

where wij is the weight of the connection from the ith neuron in the previous layer to the neuron j, f (Ij + θj) is activation function of the neurons, Oj is the output of neuron j, and θj is the biases input to Copyright © 2014 John Wiley & Sons, Ltd.



911

the neuron. In this paper, we use a tangent-sigmoid activation function defined by the following equation: f ðx Þ ¼

2 1 ð1 þ expð2xÞÞ

(22)

There are also two commonly used error functions, namely, the mean square error function and the mean absolute error (MAE) function. In this paper, we used the MAE function, which is defined as qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 XX ðT nl Onl Þ2 (23) E¼ n l 2n where n is the number of training patterns and l is the number of output nodes. Ol and Tl are the output value and the target value, respectively. The MAE is used to evaluate the learning effects, the training will keep up until the MAE falls below some threshold or tolerance level. The gradient descent method searches for the global optimum of the network weights, and partial derivatives ∂E/∂w are calculated for each weight in the network. And the weights are adjusted using the following expression: wðm þ 1Þ ¼ wðmÞ η∂E ðmÞ=∂wðmÞ (24) where m is the number of epochs and η is the learning rate.

4. RESULTS AND DISCUSSION 4.1. Data preparation Features were extracted using the aforementioned nonlinear curve fitting method. We ignore the cool down temperatures. According to Equations (16) and (18), in order to obtain one or two coefficients using

Figure 8. Examples of curve fitting data in multiple free-spanning experiments. Copyright © 2014 John Wiley & Sons, Ltd.


912

X. ZHAO ET AL.

Table II. Curve fitting results. Fit sediment model Location of data (m) 22.79 24.42 27.27 29.30 32.56 35.00

Fit water model

A

B

SSE

R2

RMSE

C

D

SSE

R2

RMSE

0.953 7.15 1.43 0.9618 6.933 8.727

1.176 5.641 3.321 1.751 5.866 5.889

15.77 10.89 23.15 23.45 17.53 11.67

0.5471 0.9899 0.6494 0.4292 0.9829 0.9927

0.7505 0.6236 0.9093 0.9151 0.7913 0.6456

7.182 25.84 7.763 6.393 24.36 29.95

649.5 2910 908.7 673.2 3433 3268

13.06 27.75 22.96 15.48 27.32 26.33

0.6248 0.9744 0.6522 0.6231 0.9734 0.9836

0.6831 0.9955 0.9056 0.7437 0.9878 0.9697

SSE, sum of squares for error; RMSE, root mean square error.

the curve fitting method, other thermal coefficients in the equations need to be determined. However, it is very difficult to accurately measure all the thermal coefficients of ambient medium along the pipeline because of the complex hydrodynamic and geomorphic conditions. The measurement of thermal parameters is beyond the scope of this study. To facilitate the nonlinear curve fitting, the aforementioned two equations [Equations (16) and (18)] are generalized into empirical equations, as illustrated by 8 < ΔT ¼ Að lnt þ BÞ for sediment t : ΔT ¼ C 1 exp for water D

(25)

where A, B, C, and D are empirical coefficients. Temperature histories were fitted to both sediment model and water model. The nonlinear curve fitting was based on the least squares method. Temperatures in specific locations were considered as belonging to a specific scenario whose model yielded better goodness of fit. In goodness of fit, SSE is sum of squares for error, R2 is coefficient of determination, and RMSE is root mean square error. The fit with larger R2 and smaller SSE and RMSE represents better fit. Figure 8 shows the examples of curve fitting of sediment data and water data in different location as indicated in the graph. Figure 8(a), (c), and (d) is in water scenario, while Figure 8(b), (e), and (f) is in sediment scenario. Scatter dots are temperatures with time acquired from DAU, which were fitted to sediment theoretical model and water theoretical model, respectively. Without relying on resulting goodness of fit, it is hard to tell which model fits better. Table 2 lists the fitting coefficients and goodness of fit. The data are taken from multiple free-spanning experiments. As can be seen, sediment data fitted the sediment model better, while water data fitted the water model better. Subsequently, empirical coefficients were classified into two groups, where larger A, C, and D and smaller B tend to belong to sediment group and vice versa for water group. Based on their inherent differences in fitting coefficients, empirical coefficients A, B, C, and D were selected to form feature vectors, which were inputted into the SVM for classification. Each temperature curve for specific sampling points was processed using the aforementioned nonlinear curve fitting method to obtain the feature vector. The database consisted of 368 samples, in which 261 were sediment scenario samples and the remaining 107 were water scenario samples. In order to establish the SVM classification model, the database has been divided into training and testing sets. The training set was used to construct the optimal model. Because the database is naturally unbalanced, we select randomly 50% samples from each class for training and the remaining for testing to avoid biased simulation. Therefore, 140 out of 368 samples were selected as training set, in which 70 samples are randomly selected from scenario and the remaining half are from water scenario. And the testing set, independent of the training set, was used to assess the classifier’s performance. In this study, the remaining 228 samples were considered as testing set. Also, 20 random trials were conducted to justify the classification accuracies. In SVM, data normalization is very important before building the classification model. Data normalization can prevent coefficients in greater numeric ranges from dominating those in smaller numeric ranges. In addition, it also avoids numerical difficulties during the calculation. Because most kernel values rely on the inner products of feature vectors, large coefficient values might induce numerical Copyright © 2014 John Wiley & Sons, Ltd.


913


Table III. Optimal parameters for support vector machine with four different kernels. Kernel type

C

γ

r

d

Linear Polynomial RBF Sigmoid

10 10 10 10

— 1 10 0.1

— 0 — 0

— 3 — —

RBF, radial basis function.

problems. Thus, each input coefficient was linearly scaled to the range [0, 1] for both training and testing sets. The normalization equation is as follows: x minðxÞ (26) x’ ¼ maxðxÞ minðxÞ 4.2. Configuration of support vector machine classification models In this study, we have implemented the tested SVM classifier with four common kernel functions for SVM, namely, linear kernel, polynomial kernel, RBF kernel, and sigmoid kernel. Because the number of samples was limited, it was important to obtain the best generalization performance and reduce the over-fitting problem. The penalty parameter C and the parameters of the selected kernel function had a great influence on the performance of the SVM model. In order to find the optimal combination of parameter C and parameter of kernel functions, a two-step grid search method using cross-validation, the commonly used searching method, was adopted in this study. Fivefold cross-validation was used to determine the optimal parameters. Basically, different combinations of parameter C and kernel functions were tried, and the one with the best cross-validation accuracy was picked. For the kernel parameters r and d of polynomial and sigmoid functions, we used standard values [22]. The objective of grid search using cross-validation is to find the best pair of (C, γ). In the first step, a broad grid search was performed using the following sets of values: C = {0.001, 0.01, 0.1, 1, 10, 100, 1000} and γ = {0.001, 0.01, 0.1, 1, 10, 100, 1000}. After identifying a better region, an optimal pair (C0, γ0) was picked. In the second step, an intensive grid search was conducted around (C0, γ0), where C = {0.25 C0, 0.5 C0, 0.75 C0, C0, 2.5 C0, 5 C0, 7.5 C0} and γ = {0.25 γ0, 0.5 γ0, 0.75 γ0, γ0, 2.5 γ0, 5 γ0, 7.5 γ0}. The optimal parameter pair was picked in this step. For linear kernel function, grid search was only used for selecting parameter C. Optimal parameters for SVM with four different kernel functions in this study are shown in Table 3. We classify the ambient scenarios such that +1 represents the sediment scenario and 1 denotes the water scenario. For SVM calculation, we applied the LIBSVM software, available at http://www.csie.ntu.edu.tw/~cjlin/libsvm/, to construct the SVM classification models. For example, for polynomial kernel, grid search was only used for selecting parameter C. As shown in Figure 9(a), C = 10 gave the highest accuracy of 98.6% during coarse search and thus being picked for fine search. During fine search, C = 5, 7.5, and 10 gave the same accuracy, and finally, C = 10 was chosen as the optimal parameter. For RBF and sigmoid kernel, we have to find the best pair of (C, γ). (a) 100

(b)

Coarse Search

95

Accuracy (%)

Accuracy (%)

95 90 85 80

90 85 80 75

75 70

Fine Search

100

0.001 0.01 0.1

1

10

Parameter C

100 1000

70

2.5

5

7.5

10

25

50

75

Parameter C

Figure 9. Polynomial kernel: (a) coarse search and (b) fine search. Copyright © 2014 John Wiley & Sons, Ltd.


914

X. ZHAO ET AL.

Figure 10. Sigmoid kernel: (a) coarse search and (b) fine search.

Table IV. Parameter list for back-propagation neural network model. Item no. 1 2 3 4 5 6 7 8 9 10 11 12 13

Name of parameters

Value of parameters

Neurons of input layer The amounts of hidden layer Neurons of hidden layer Neurons of output layer Transfer function of hidden layer Transfer function of output layer Goal of MAE Epochs Minimum descent of gradient Initial learning rate Initial weight vectors Initial bias term The training function

4 1 5 2 Tangent-sigmoid Tangent-sigmoid 0.05 5000 6 10 0.01 0 0 SCGBP

MAE, mean absolute error; SCGBP, scaled conjugate gradient back-propagation.

Take sigmoid kernel for instance, as shown in Figure 10(a), the best pair (10, 0.1), which gave 98.6% successful rate, was picked in coarse search. In the fine search phase, most of the pairs gave the same accuracy, and the pair (10, 0.1) was chosen. 4.3. Configuration of back-propagation neural network classification model The BPNN selected for a comparative algorithm in this paper is a three-layer network, consisting of an input layer, an output layer, and one hidden layer. The structure of the BPNN network is set to include an input layer with four neurons according to the number of attributes being used for classification and an output layer with two neurons according to the number of classes of ambient scenario. As for the hidden layer, selecting the appropriate number of hidden neurons is very important for training the BPNN. If the number of hidden neurons is few, the network will not be able to cope with complex problems, and if the number is many, it will increase the training time dramatically. In order to determine the best number of hidden neurons, we employed the trial-and-error method. Thus, the BPNN is trained with varying sizes of hidden neurons, from 2 to 13 neurons, and the best one is selected as the final configuration. pffiffiffiffiffiffiffiffiffiffiffiffi

The pffiffiffiffiffiffiffiffiffiffiffiffi scope of hidden neurons is determined by the following formula, n þ m þ 1; n þ m þ 10 , where n denotes the number of input neurons and m the number of output neurons. After trial and error, we select 5 as the number of hidden neurons. The training function designated as scaled conjugate gradient back-propagation is used for the development of BPNN model. All essential BPNN parameters in this study are summarized in Table 4. The same datasets used in SVM models development are used to establish BPNN classification model. The target output for the network was chosen to be 1 for the correct class and 0 for the other class. In Copyright © 2014 John Wiley & Sons, Ltd.


915


Table V. Performance comparison between support vector machine with four different kernels and backpropagation neural network. Classifier

Kernel type

Training accuracy (%)

Testing accuracy (%)

SVM

Linear Polynomial RBF Sigmoid

99.7 99.6 99.9 99.4 98.3

98.7 98.8 98.9 98.8 97.8

BPNN

SVM, support vector machine; BPNN, back-propagation neural network; RBF, radial basis function.

particular, output ‘1 0’ indicates the sediment scenario, and ‘0 1’ represents the water scenario. Because the testing results for the BPNN are not unique, the experiments were replicated 10 times in order to obtain the average training and testing accuracy. The training, testing, and performance evaluation of BPNN has been carried out using the Neural Network toolbox in MATLAB.

4.4. Classification performance of support vector machine with four different kernels and back-propagation neural network We evaluated the classification performance of SVM with four different types of kernel function and compared it with that of BPNN in Table 5. The accuracies of SVM models are average accuracies from 20 different random trials. The table shows both training and testing accuracy that were obtained under different methods. As can be seen, all models had a relatively good classification success rate, and the SVM models were slightly better than BPNN model. Of all four SVM models, the one with RBF has achieved the best classification performance, providing 99.9% success rate on training set and 98.9% on testing set. Additionally, the BPNN can also give relatively good results, with 98.3% on training accuracy and 97.8% on testing accuracy. The performance of BPNN has shown that it is less sensitive to selection of parameters. However, as can be seen from Table 5, as long as the selection of the appropriate parameters and kernel is guaranteed, the SVM with RBF kernel exhibits better performance. High training accuracy often leads to over-fitting problem in many cases. In our case, we had adopted several techniques to avoid over-fitting. We extracted features using nonlinear curve fitting, thus dimensions of data were greatly reduced into four. As can be seen from Table 2, these four features tend to be similar within group and diverse between groups, which make the data more discernable. Also, in the training phase, fivefold cross-validation was employed to avoid over-fitting. Results of 20 different trails showed that these models have good generalization performance. For certain combination of training and testing sets, the results of SVM models with repeated experiments are identical; however, those of BPNN model are varied. This inconsistency can be attributed to the differences in calculation principles employed to develop the SVM and BPNN models. BPNN applies iterations and error back-propagation methods to acquire an approximate optimal solution, while SVM uses a structural risk minimization technique to achieve a unique, optimal solution. Therefore, SVM models are more stable in terms of solution stability. On the other hand, comparing the results of Table 3 with those of Table 4, it shows that SVM models require four decision parameters at most, (b)

2

Status of pipeline

Status of pipeline

(a)

1 0 -1 -2 20

25

30

Distance (m)

35

40

2 1 0 -1 -2

90

95

100

105

Distance (m)

Figure 11. (a) Classification results for the lower sensor; (b) classification results for the upper sensor. Copyright © 2014 John Wiley & Sons, Ltd.


916

X. ZHAO ET AL.

Classification accuracy (%)

100 95 90 85 80 75

0

0.5

1

1.5

2

2.5

3

3.5

Heating time (h)

Figure 12. Classification accuracy with different heating time.

while BPNN model needs 13 decision parameters. Accordingly, the SVM models present more conveniences in model selection because they demand fewer control parameters. Based on the aforementioned results and discussion, SVM models show good effectiveness in our case. Moreover, among the four types of SVM models, the one with RBF kernel performs better. Thus, the optimal SVM model with RBF kernel was selected as the classifier for the scour monitoring system. The testing of this classifier is shown in the remaining of this subsection and in the next subsection. To illustrate the decision made by the selected classifier, Figure 11 shows the classification results for the two free span experiments, which includes a minor free span with a length of 2.5 m and a major free span with a length of 6 m. Figure 11(a) displays the classification results of distributed optical temperature sensor positioned on the lower surface of pipeline, and Figure 11(b) exhibits the one with sensor placed on the upper surface of pipeline. The dot sign (at +1: indicating sediment scenario and at 1: indicating water scenario) shows the status of pipeline for each cases. There were 46 sampling points for each sensor. Of all 92 sampling points, only two of them were misclassified, showing an accuracy of 97.8% in this case. As for the scour length calculation in this study, the detected free-spanning length is equal to the amounts of successive water scenario sampling points multiplied by the sampling interval (0.41 m in this application [12]). For example, for the case shown in Figure 11(a), there were two series of successive points detected to be in water scenario, 8 points in minor free span and 16 points in major free span. In succession, the calculated free-spanning length were 3.28 m for minor free span case and 6.56 m for the major free span case. Examining the artificially induced free-spanning lengths (2.5 m and 6 m, respectively) and the detected lengths, we found that there exist some errors, with error rates of 31.20% for minor free span and 9.33% for major free span. A longer free span length reduces the error rate. Considering that offshore pipelines are over hundreds of kilometers, these errors can be neglected. The detected free-spanning lengths can be used for calculating the remaining fatigue life of the offshore pipeline. The pipeline operator can make a decision on protecting the pipeline to avoid any scour-induced pipeline failures or third-party interferences.

4.5. System optimization In our experiments, the heating time for the thermal cables was 3 h. After 3 h of heating, the sediment and water scenarios were quite discernable, as shown in Figure 7. Although longer heating time gives more discernable results and higher accuracy, it also costs more energy. In view of the long distance nature of offshore pipelines, a small extension of heating time can significantly add up the energy consumption and operation cost. In order to optimize the system and save energy, we need to seek the optimal heating time with considerable detection precision and shorter heating time. To implement this goal, the previous 92 samples used in Section 4.4 were adopted. Feature extraction using nonlinear curve fitting was executed for the selected dataset with different heating time. The interval was 0.5 h. The data were then inputted into the optimal SVM classifier with RBF kernel to test the classification accuracy. Copyright © 2014 John Wiley & Sons, Ltd.



917

Figure 12 shows the classification accuracy with different heating time. The classification accuracy increased rapidly during 0.5–1 h and showed slight increase afterwards. As can be seen, the performance of the classifier reached a steady state of 97.8% when the heating time was 2 h. The classification precision showed no obvious improvement as the heating time increased. Heating of 2 h can serve as a good reference for the laboratory investigation. When it comes to real world applications, a corresponding investigation should be performed before determining the optimal heating time of the system.

5. CONCLUSIONS In this paper, a novel scour automatic detection scheme based on nonlinear curve fitting and SVM has been introduced to determine scouring conditions along pipelines for the active thermometry-based scour monitoring system. For this purpose, temperature profiles from free-spanning experiments have been used for feature extraction to reduce the dimensions of raw datasets. The feature extraction is based on nonlinear curve fitting. In particular, on account of the varied heat transfer patterns of a line heat source in sediment and water scenarios, the experimental temperature profiles are nonlinearly fitted to their theoretical models. The resulting fitting coefficients are used to form feature vectors, which are then inputted into SVM classifier to detect scouring along a pipeline. Support vector machine models with different kernel functions (linear, polynomial, RBF, and sigmoid) are compared with BPNN. Both SVM and BPNN models showed high classification accuracies. SVM models have shown slightly better performance than BPNN in the present study. Moreover, among the four types of SVM models, the one with RBF kernel outperformed other models, showing a classification accuracy of 99.9% for training sets and 98.9% for testing sets. The SVM model with RBF kernel was chosen as the classifier for the scour monitoring system. When it comes to system optimization, that is to say, finding the optimal heating time that ensuring high classification accuracy and saving energy, the selected SVM model is employed to recognize datasets with different heating time. Results show that the optimal heating time is 2 h, which is capable of providing an accuracy of 97.8%. ACKNOWLEDGEMENTS

This research was financially supported by National Basic Research Program of China (2011CB013702), National Natural Science Foundation of China (grant no. 51479031), Key Projects in the National Science & Technology Pillar Program during the Twelfth Five-Year Plan Period (2011BAK02B02), ‘863 programs’ National High Technology Research and Development Program (2008AA092701-6), and the Science Fund for Creative Research Groups from the National Science Foundation of China under grant no. 51221961.

REFERENCES 1. Yeganeh BA, Ghaheri A, Valipour R. Analysis of offshore pipeline allowable free span length. International Journal of Civil Engineerng 2007; 5(1):84–91. 2. National Research Council. Improving the safety of marine pipelines. The National Academies Press: Washington, DC, USA, 1994. 3. Jin W, Shao J, Zhang E. Basic strategy of health monitoring on submarine pipeline by distributed optical fiber sensor. Proceedings of the 22nd International Conference on Offshore Mechanics and Arctic Engineering, Cancun, Mexico, 8–13 June 2001. 4. Feng X, Zhou J, Li X, Hu JS. Structural condition identification for free spanning submarine pipelines. Proceedings of the Eighteenth International Offshore and Polar Engineering Conference, Vancouver, BC, Canada, 6–11 July 2008; ISBN: 978-1-880653-70-8. 5. Yan G, Peng X, Hao H. Localization of free-spanning damage using mode shape curvature. Journal of Physics: Conference Series 2011; 305(1). doi:10.1088/1742-6596/305/1/012017. 6. Bao C, Hao H, Li Z. Vibration-based structural health monitoring of offshore pipelines: numerical and experimental study. Structural Control and Health Monitoring 2013; 20(5):769–788. 7. Bristow KL. Measurement of thermal properties and water content of unsaturated sandy soil using dual-probe heat-pulse probes. Agricultural and Forest Meteorology 1998; 89(2):75–84. 8. Cote A, Carrier B, Leduc J, Noel P, Beauchemin R, Soares M, Garneau C, Gervais R. Water leakage detection using optical fiber at the Peribonka Dam. Seventh International Symposium on Field Measurements in Geomechanics, Boston, MA, USA, 24–27 September 2007; ISBN: 978-0-7844-0940-4. Copyright © 2014 John Wiley & Sons, Ltd.


918

X. ZHAO ET AL.

9. Sayde C, Gregory C, Gil-Rodriguez M, Tufillaro N, Tyler S, van de Giesen N, English M, Cuenca R, Selker JS. Feasibility of soil moisture monitoring with heated fiber optics. WATER RESOURCES RESEARCH 2010. doi:10.1029/ 2009WR007846. 10. Zhao XF, Li L, Ba Q, Ou JP. Scour monitoring system of subsea pipeline using distributed Brillouin optical sensors based on active thermometry. Optics & Laser Technology 2012; 44:2125–2129. 11. Zhao XF, Ba Q, Li L, Gong P, Ou JP. A three-index estimator based on active thermometry and a novel monitoring system of scour under submarine pipelines. Sensors and Actuators A: Physical 2012; 183:115–122. 12. Zhao X, Li W, Song G, Zhu Z, Du J. Scour monitoring system for subsea pipeline based on active thermometry: numerical and experimental studies. Sensors 2013; 13(2):1490–1509. 13. Zhao X, Li W, Zhou L, Song G, Ba Q, Ou J. Active thermometry based DS18B20 Temperature sensor network for offshore pipeline scour monitoring using K-means clustering algorithm. International Journal of Distributed Sensor Networks 2013. doi:10.1155/2013/852090. 14. Vapnik VN. The nature of Statistical Learning Theory. Springer-Verlag: New York, 1995. 15. Belsito S, Lombardi P, Andreussi P, Banerjee S. Leak detection in liquefied gas pipelines by artificial neural networks. AIChE Journal 1998; 44(12):2675–2688. 16. Chen H, Ye H, Chen LV, Su H. Application of support vector machine learning to leak detection and location in pipelines. Proceedings of the 21st IEEE, Instrumentation and Measurement Technology Conference 2004; 3:2273–2277. 17. Qu Z, Feng H, Zeng Z, Zhuge J, Jin S. A SVM-based pipeline leakage detection and pre-warning system. Measurement 2010; 43(4):513–519. 18. Etemad-Shahidi A, Yasa R, Kazeminezhad MH. Prediction of wave-induced scour depth under submarine pipelines using machine learning approach. Applied Ocean Research 2011; 33(1):54–59. 19. Lee LH, Rajkumar R, Lo LH, Wan CH, Isa D. Oil and gas pipeline failure prediction system using long range ultrasonic transducers and Euclidean-support vector machines classification approach. Expert Systems with Applications 2013; 40:1925–1934. 20. Cristianini C, Shawe-Taylor J. An Introduction to Support Vector Machines. Cambridge University Press: Cambridge, United Kingdom, 2000. 21. Burges CJC. A tutorial on support vector machines for pattern recognition. Proceedings of International Conference on Data Mining and Knowledge Discovery 1998; 2(2):121–167. 22. Pal M, Mather PM. Support vector classifiers for land cover classification. arXiv preprint arXiv:0802.2138, 2008. 23. Haykin S. Neural Networks: A Comprehensive Foundation. Prentice-Hall: NJ, 1999. 24. Cristianini N, Taylor JS. An Introduction to Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press: Cambridge, United Kingdom, 2000. 25. Shin HJ, Eom DH, Kim SS. One-class support vector machines—an application in machine fault detection and classification. Computers & Industrial Engineering 2005; 48(2):395–408. 26. de Vries DA, Peck AJ. On the cylindrical probe method of measuring thermal conductivity with special reference to soils. I. Extension of theory and discussion of probe characteristics. Australian Journal of Physics 1958; 11:255–271. 27. Incropera FP, Dewitt DP. Fundamentals of Heat and Mass Transfer. John Wiley & Sons: Hoboken, NJ, United States, 2011.

Copyright © 2014 John Wiley & Sons, Ltd.
