fuzzy logic coupled with peridynamics for image processing to detect

4th International Symposium on Railway Systems Engineering (ISERSE’18), October 10-12, 2018, Karabuk, Turkey

FUZZY LOGIC COUPLED WITH PERIDYNAMICS FOR IMAGE PROCESSING TO DETECT CRACKS IN RAILS Cihan Mizrak1, Erdogan Madenci2, Yusuf Yurekli3 and Fatih Pehlivan4

1 Karabük

University, Engineering Faculty, Karabük Turkey,e-mail: [email protected], 2 The University of Arizona, Tucson, AZ, USA e-mail: [email protected], 3TCDD, Karabük, Turkey e-mail: [email protected],4 Karabük University, Engineering Faculty, Karabük, Turkey,e-mail: [email protected]

Abstract Railways require high investment costs and need continuous and controlled monitoring for defects on the surface and inside the rail material. The existing nondestructive methods for crack detection are commonly based image processing. This study provides a comparison of the existing methods, and presents a new method to detect cracks also based on image processing. The present approach utilizes the fuzzy logic coupled with the peridynamic differential operator. Its robustness is demonstrated by detecting cracks in an image of a rail with cracks.

Keywords: rail, crack, image, process, peridynamic

1. Introduction A non-destructive material inspection is a method for detecting flaws without physically damaging the material. Non-destructive examinations can be performed by different methods such as visual inspection, radiographic inspection, ultrasonic inspection, magnetic particle inspection, liquid penetrant inspection and vortex flow inspection. Non-destructive inspection can also be performed accurately by employing high precision cameras with the advancement of image processing technologies. In the context of rail inspection, such precision cameras can be mounted on rail vehicles without any contact with the rails [1-4]. Limin et al. [5] presented a method using image processing techniques to detect defects on the rail surface. Trinh et al. [6] developed a new method using multiple cameras to eliminate the railway components that hamper image processing. Ying et al. [7] introduced an automatic rail track maintenance system for rail component detection by using the Sobel operator and the Hough transformation. Quingyong et al. [8] developed an image processing based a control system to detect defects on the rail surface in real time. Recently, Yaman et al. [9] demonstrated that image processing based non-contact inspection can detect failures on the rail surfaces. They placed two cameras at different angles on both sides of the rail to obtain images. The employed Canny edge detection and Hough transform to process these images and detect potential surface crack on the rails. Zhang et al. [10] also presented an automatic crack detection and classification methodology to detect cracks in rails through their digital images. They specifically employed an average softening filter, threshold and morphological opening techniques along with shape-based image processing methods, and are able to detect only grouped defects in rails. Similarly, Liu et al. [11] estimates the regions experiencing material loss with dynamic thresholding and morphological filling techniques in conjunction with an averaging filter of images. As extension of this study, Zhang et al. [12] presented a method to identify the defect through a known image of the rail. Unlike the previous investigations, this study employs the fuzzy logic concept along with the peridynamic differential operator to process images to detect cracks in rails.

419

Mızrak, C., Madenci, E., Yürekli, Y. and Pehlivan, F.

2. Existing Crack Detection Techniques The capillary cracks that form on the surface of the rail mushroom at the first stage are determined by different edge detection algorithms. These algorithms are based on the numerical derivative of the image with the help of a matrix called mask defined within a specific area. The first-order derivative of the image determines the locations where the contrast is maximum, such as edge transitions, while its the second-order derivative determines the transitions with zero values at the edges as shown in Fig. 1. The most commonly used edge detection algorithms are Sobel, Laplacian and Canny operators [13]. Before these algorithms are applied, the image is converted to gray scale and noise is eliminated through the use of the Gaussian mask [14]. In the Sobel operator, the derivative of the first order is applied with separate masks along the x- and y-directions [15]. In the Laplacian operator, a single mask is used for determining the second order derivatives. Canny operator is the customized version of the Sobel operator. At the last stage, the edges are made thinner and clearer by non-maximum pressing [14].

Fig. 1. PD differentiation of the image.

3. Peridynamic Differential Operator Peridynamic theory introduced by Silling [16] and Silling et al. [17] provides the nonlocal representation of a scalar field f = f (x) at point x by accounting for the effect of its interactions with the other points, x , in the domain of interaction, as shown in Fig. 2. Each point has its own family members in the domain of interaction (family), and occupies an infinitesimally small entity such as volume, area or a distance. The points x and x only interact with the other points in their own

420

Mızrak, C., Madenci, E., Yürekli, Y. and Pehlivan, F.

families, H x and H x , respectively. Neither point x nor x is necessarily symmetrically located in their interaction domains. The initial relative position, ξ , between the material points x and x can be expressed as ξ = x - x . This ability permits each point to have its own unique family with an arbitrary position. Therefore, the size and shape of each family can be different, and they significantly influence the degree of nonlocality. The degree of the interaction between the material points in each family is specified by a nondimensional weight function, w( ξ ) which can vary from point to point. The interactions become more local with decreasing family size. Thus, the family size and shape are important parameters. The Peridynamic Differential Operator (PDDO) introduced by Madenci et al. [18-20]. The PDDO employs the concept of PD interactions, and it is based on the orthogonality property of the PD functions. It restores the nonlocal interactions at a point by considering its association with the other points within an arbitrary domain of interaction. The PD differentiation recovers the local differentiation as this interaction domain approaches zero. It converts the local form of differentiation to its nonlocal PD form. It is simply a bridge between differentiation and integration. Therefore, the PDDO enables numerical differentiation through integration. The PDDO enables the computational solution of complex differential equations and evaluation of derivatives of smooth or scattered data in the presence of jump discontinuities or singularities.

Fig. 2. Interaction of peridynamic points, x and x with arbitrary family size and shape.

In an M - dimensional space, the Taylor Series Expansion (TSE) of a scalar field f (x) = f (x + ξ) with many variables can be expressed as

f (x + ξ) =

N

N − n1

N − n1 − nN −1

   

n1 = 0 n2 = 0

nN = 0

 n1 + n2 ++ nN f ( x ) 1 1n1  2n2   MnN n1 n2 n1 !n2 !L nN ! x1 x2  xMnN

(1)

+ R ( N , x)

where ξ = x − x with R( N , x) representing the remainder. Assuming the contribution of the remainder is negligibly small and invoking the property of the orthogonal function, g Np1 p2  pN (ξ ) , result in the PD nonlocal expression for the partial derivatives of any order as

421

Mızrak, C., Madenci, E., Yürekli, Y. and Pehlivan, F.

 p1 + p2 ++ pN f (x) =  f (x + ξ ) g Np1 p2  pN (ξ)dx1dx2 L dxM pN p1 p2 x1 x2  xM Hx

(2)

in which pi denotes the order of differentiation with respect to variable xi with i = 1,...., M . The PD functions g Np1 p2  pN (ξ ) possess the orthogonality property of

1 1n1  2n2   MnM g Np1 p2  pN (ξ)dx1dx2 L dxM =  n1 p1  n2 p2 L  nN −1 pN −1  nN pN n1 !n2 !L nN ! Hx

(3)

in which ni = 0, ..., N . They can be constructed as

g Np1 p2  pN (ξ ) =

N



N − q1

 

q1 = 0 q2 = 0

N − q1 − qN −1



qN = 0

aqp11qp22LL qpNN wq1q2 L qN ( ξ ) 1q1  2q2   MqN

(4)

where wq1q2 L qN ( ξ ) are the weight functions associated with each term 1q1  2q2   MqN in the polynomial expansion. Depending on the nature of the nonlocality, the weight function representing the degree of interaction may be the same or different for each term in the TSE. The unknown coefficients, aqp11qp22LL qpMM , can be determined from the solution of

N

N − q1

N − q1 − qN −1

   

q1 = 0 q2 = 0

qN = 0

A( n1n2 L nN )( q1q2 L qN ) aqp11qp22LL qpNN = bnp11np2 2LLnNpN

(5)

in which qi = 0, ..., N . The coefficient (shape) matrix is constructed as

A( n1n2 L nN )( q1q2 L qN ) =

w

q1q2 L qN

(ξ) 

n1 + q1 1

 2n

2 + q2

  MnN + qN dx1dx2 L dxM

(6a)

Hx

and

bnp11np2 2LLnNpN = n1 !n2 !L nM !  n1 p1  n2 p2 L  nN pN .

(6b)

422

Mızrak, C., Madenci, E., Yürekli, Y. and Pehlivan, F.

The PDDO recovers the local differentiation as the size of family, H x decreases or the number of terms in the functions g Np1 p2  pN (ξ ) increases. It requires the computation of the coefficients, aqp11qp22LL qpNN , and the condition number of the coefficient (shape) matrix, An1n2 L nN q1q2 L qN , may become poor for higher order derivatives. Therefore, when computing higher order derivatives, the family size needs to be adjusted accordingly. If it is too small then round-off errors dominate, and if it is too large then the results deviate from local values. The coefficients of the PD functions can be determined without any difficulty. Although it is not a limitation, the weight functions, wq1q2 q3 ( ξ ) in Eq. (6a) can be replaced with wn ( ξ ) for simplification based on the order of differentiation. A Matlab code presented in a recent book by Madenci et al. [20] (2018) elucidates the PDDO for the N -th order derivative of a function with M dimensions.

4. Fuzzy Logic The concept of fuzzy logic introduced by Zadeh [21- 22] can be described through the block diagram shown in Fig. 3. The essence of fuzzy logic lies is that it does not give a certain result. The fuzzy logic application can return different output values for the same specified input values.

Fig. 3. Fuzzy Logic system block diagram

The membership values of the fuzzy set elements are in the range of 0-1. A fuzzy set membership function defined as B in the U universal set, is characterized by

 B ( x)

[21- 22] as shown in Fig. 4.

A membership function can be defined as a triangle, trapezoid or sigmoid to determine the membership value of its elements.

423

Mızrak, C., Madenci, E., Yürekli, Y. and Pehlivan, F.

Fig. 4. Sample of Fuzzy membership function.

The rule base contains a series of fuzzy rules that define the relation between the input and output. In general there are three general forms of fuzzy rules for any verbal variable. Fuzzy rules is a set of conditional statements such as assignment, conditional and unconditional sentences [23]. The inference unit performs fuzzy inference with fuzzy rules. There exit many different structures for fuzzy inference. The most commonly used method is the Mamdani inference method. In this method, depending on the membership degree for each entry value, the portion above the membership value of the fuzzy set is cut. The output value is usually obtained by applying the weighted average method to these fuzzy sets [24]. Fuzzy inference results in a fuzzy set. This result needs to be converted to an analog value. This process is called rinse. The rinse unit is a non-fuzzy information from fuzzy information from the decision-making unit and provides real values. Various methods are used in the rinsing process.

5. Crack detection based on fuzzy logic and PDDO The lower and upper limits of the first and second order peridynamic derivatives of the image with respect to spatial coordinates provide the input parameters Ix, Iy, Ixx, Iyy and Ixy, respectively, to the fuzzy logic system. The three triangular type membership functions are assigned to each input parameter, the upper and lower limit values are determined according to the maximum and minimum points of the image. While the positions of the cracks remain the same in all derivative values, the positions of other noises such as shadows and reflections are different. Therefore, the fuzzy logic rule base is established by way of common intersections of five different derivatives, and 13 rules are added to the system for clarification/refinement as explained in Table 1. The input parameter fuzzification is performed for each pixel value of the image, subsequently the defuzzification by the rule base for the output image.

Table 1. Fuzzy logic rule base

No

Rules

1

If (Ix is L) and (Iy is L) and (Ixy is L) and (Ixx is L) and (Iyy is L) then (Iout is L)

2

If (Ix is M) and (Iy is M) and (Ixy is M) and (Ixx is M) and (Iyy is M) then (Iout is M)

3

If (Ix is H) and (Iy is H) and (Ixy is H) and (Ixx is H) and (Iyy is H) then (Iout is H)

424

Mızrak, C., Madenci, E., Yürekli, Y. and Pehlivan, F.

4

If (Ix is L) or (Iy is H) or (Ixy is H) or (Ixx is H) or (Iyy is H) then (Iout is M)

5

If (Ix is M) or (Iy is H) or (Ixy is H) or (Ixx is H) or (Iyy is H) then (Iout is M)

6

If (Ix is L) or (Iy is M) or (Ixy is H) or (Ixx is H) or (Iyy is H) then (Iout is M)

7

If (Ix is L) or (Iy is L) or (Ixy is H) or (Ixx is H) or (Iyy is H) then (Iout is M)

8

If (Ix is L) or (Iy is L) or (Ixy is M) or (Ixx is H) or (Iyy is H) then (Iout is M)

9

If (Ix is L) or (Iy is L) or (Ixy is L) or (Ixx is H) or (Iyy is H) then (Iout is M)

10

If (Ix is L) or (Iy is L) or (Ixy is L) or (Ixx is M) or (Iyy is H) then (Iout is M)

11

If (Ix is L) or (Iy is L) or (Ixy is L) or (Ixx is L) or (Iyy is H) then (Iout is L)

12

If (Ix is L) or (Iy is L) or (Ixy is L) or (Ixx is L) or (Iyy is M) then (Iout is M)

13

If (Ix is L) or (Iy is L) or (Ixy is L) or (Ixx is L) or (Iyy is L) then (Iout is M)

6. Demonstration The gray scale image shown in Fig. 5 is considered to demonstrate the capability of the present approach. The degree of membership is constructed for each derivative as shown in Fig. 6. The degree of membership for the output parameter is shown in Figure 7.

Fig. 5. Image of a defective rail in gray scale

425

Mızrak, C., Madenci, E., Yürekli, Y. and Pehlivan, F.

(a)

(b)

(c)

(d)

(e) Fig. 6. Degree of membership for input parameters based on the PDDO: (a) 1st order derivative along the x direction, (b), 1st order derivative along the y direction, (c) 2nd order derivative along the x direction (d), 2nd order derivative along the y direction, and (e), 2nd order cross derivative

426

Mızrak, C., Madenci, E., Yürekli, Y. and Pehlivan, F.

Fig. 7. Degree of membership for the output parameter.

The output image shown in Fig. 8 clearly captures the cracks with significant distinction. Also, the threshold is applied to obtain the values 0 and 1 of the image. As shown in Fig. 9, the cracks are more distinctive and the noise is considerably reduced. Figures 10-12 present the results obtained by using the Canny, Sobel and Laplacian operators. It is evident that the present PDFuzzy logic approach is more robust than the commonly existing methods.

Fig. 8. PD-fuzzy logic output image.

427

Mızrak, C., Madenci, E., Yürekli, Y. and Pehlivan, F.

Fig. 9. PD-fuzzy logic output image after applying the threshold

Fig. 10. Canny operator with non-maximal suppression

428

Mızrak, C., Madenci, E., Yürekli, Y. and Pehlivan, F.

Fig. 11. Sobel operator with non-maximal suppression

Fig. 12. Laplacian operator with non-maximal suppression

7. Remarks This study provides a comparison of the existing methods, and presents a new method to detect cracks also based on image processing. The new method utilizes the fuzzy logic coupled with the peridynamic differential operator. Its robustness is demonstrated by detecting cracks in an image of a rail with cracks. The cracks are more distinctive and with considerable noise reduction. The present PD-Fuzzy logic approach is more robust than the commonly existing methods.

Acknowledgments

This work is supported by The Scientific and Technological Research Council of Turkey (TUBITAK) in the scope of 2219 Postdoctoral Research Scholarship Program.

429

Mızrak, C., Madenci, E., Yürekli, Y. and Pehlivan, F.

References

[1] Singh, M., Singh, S., Jaiswal, J., Hempshall, J., Autonomous Rail Track Inspection using Vision Based System, Proceedings of the 2006 IEEE International Conference on Computational lntelligence for Homeland Security and Personal Safety, 56-69, 2006. [2] Karaduman, G., Karakose, M., Akin, E., Experimental fuzzy diagnosis algorithm based on image processing for rail profile measurement, 15th International Symposium MECHATRONIKA, 1-6, 2012. [3] Limin, C., Yin, L., Kaimin, W., Inspection of rail surface defect based on machine vision system, 2nd International Conference on Information Science and Engineering (ICISE), 3793 - 3796, 2010. [4] Trinh, H., Haas, N., Pankanti, S., Multisensor evidence integration and optimization in rail inspection, 21st International Conference on Pattern Recognition (ICPR), 886 – 889, 2012. [5] Limin, C., Yin, L., Kaimin, W., Inspection of rail surface defect based on machine vision system, 2nd International Conference on Information Science and Engineering (ICISE), 3793 - 3796, 2010. [6] Trinh, H., Haas, N., Pankanti, S., Multisensor evidence integration and optimization in rail inspection, 21st International Conference on Pattern Recognition (ICPR), 886 – 889, 2012. [7] Ying, L., Trinh, T., Haas, N., Otto, C., Pankanti, S., Rail Component Detection, Optimization, and Assessment for Automatic Rail Track Inspection, IEEE Transactions on Intelligent Transportation Systems,15, 760 – 770, 2014. [8] Qingyong, L., Shengwei, R., A Real-Time Visual Inspection System for Discrete Surface Defects of Rail Heads, IEEE Transactions on Instrumentation and Measurement, 61, 21892199, 2012. [9] Yaman, O., Karaköse, M., Akın, E., Aydın, İ., Image Processing Based Fault Detection Approach for Rail Surface, 23nd Signal Processing and Communications Applications Conference (SIU), Malatya, 1118-1121, 2015. [10] Zhang, W., Zhang, Z., Qi, D., Liu, Y., Automatic Crack Detection and Classification Method for Subway Tunnel Safety Monitoring, Sensors, 14, 19307-19328, 2014. [11] Liu, Z., Wang, W., Zhang, X., Jia, W., Inspection of Rail Surface Defects Based on Image Processing, 2nd InternationalAsia Conference on Informatics in Control, Automationand Robotics, 472-475, 2010. [12] Zhendong, H., Yaonan, W., Feng, Y., Fie L, Surface defect detection for high speed rails using inverse P_M diffusion model, Sensor, 31(1) 86-97, 2016. [13] Ağırbaşlı, Ş., Bulanık Mantık Yaklaşımını kullanarak Endokardiyalin Kenar Çıkarımı, Yüksek Lisans Tezi, Gazi Üniversitesi, Fen Bilimleri Enstitüsü, 2013. [14] Korkusuz, N., O., Otomatik Hedef Tanıma, Yüksek Lisans Tezi, Yıldız Teknik Üniversitesi, Fen Bilimleri Enstitüsü, 2008. [15] Geçmez, A., İki Boyutlu Sayısal Filtreler, Yüksek Lisans Tezi, Fırat Üniversitesi, Fen Bilimleri Enstitüsü, 2014. [16] Silling, S.A., 2000. Reformulation of elasticity theory for discontinuities and long-range forces. Journal of the Mechanics and Physics of Solids, Vol.48No.1, pp.175-209. [17] Silling, S.A., Epton, M., Weckner, O., Xu, J. and Askari, E., 2007. Peridynamic states and constitutive modeling. Journal of Elasticity, Vol.88No.2, pp.151-184.

430

Mızrak, C., Madenci, E., Yürekli, Y. and Pehlivan, F.

[18] Madenci, E., Barut, A., Futch, M., Peridynamic differential operator and its applications,” Comput. Methods Appl. Mech. Engrg., 304, 408–451, 2016. [19] Madenci, E., Dorduncu, M., Barut, A., Futch, M., Numerical solution of linear and nonlinear partial differential equations using the peridynamic differential operator, Numerical Methods for Partial Differential Equations, 33, 1726–1753, 2017. [20] Madenci, E., Barut, A. and Dorduncu, M., 2018. Peridynamic differential operator for numerical analysis. New York: Springer (in-press) [21] Zadeh, L.A., Fuzzy algorithms, Information and Control, 12(2), 94-102, 1968. [22] Zadeh, L.A., Fuzzy sets, Information and Control, 8(3), 338-353, 1965. [23] Sağıroglu, Ş., Beşdok, E. and Erler, M., Mühendislikte yapay zeka uygulamaları-I, Ufuk Yayıncılık, Kayseri, 1-5, 2003. [24] Elmas, Ç., Yapay zeka uygulamaları, Seçkin Yayıncılık, İstanbul, 2007.

431