Development of Image Processing for Crack Detection on Concrete

applied sciences Article

Development of Image Processing for Crack Detection on Concrete Structures through Terrestrial Laser Scanning Associated with the Octree Structure Soojin Cho 1 , Seunghee Park 2 , Gichun Cha 3 and Taekeun Oh 4, * 1 2 3 4

*

Assistant Professor, Dept. of Civil Engineering, University of Seoul, Seoul 02504, Korea; [email protected] Associate Professor, School of Civil, Architectural Engineering and Landscape Architecture, Sungkyunkwan University, Gyeonggi 440-746, Korea; [email protected] Graduate Student, Dept. of Convergence Engineering for Future City, Sungkyunkwan University, Gyeonggi 440-746, Korea; [email protected] Associate Professor, Dept. of Safety Engineering, Incheon National University, Incheon 406-772, Korea Correspondence: [email protected]

Received: 18 October 2018; Accepted: 21 November 2018; Published: 23 November 2018







Featured Application: This work proposes practical image processing to identify main cracks on concrete surfaces by TLS associated with the octree structure. Abstract: Terrestrial laser scanning (TLS) provides a rapid remote sensing technique to model 3D objects but can also be used to assess the surface condition of structures. In this study, an effective image processing technique is proposed for crack detection on images extracted from the octree structure of TLS data. To efficiently utilize TLS for the surface condition assessment of large structures, a process was constructed to compress the original scanned data based on the octree structure. The point cloud data obtained by TLS was converted into voxel data, and further converted into an octree data structure, which significantly reduced the data size but minimized the loss of resolution to detect cracks on the surface. The compressed data was then used to detect cracks on the surface using a combination of image processing algorithms. The crack detection procedure involved the following main steps: (1) classification of an image into three categories (i.e., background, structural joints and sediments, and surface) using K-means clustering according to color similarity, (2) deletion of non-crack parts on the surface using improved subtraction combined with median filtering and K-means clustering results, (3) detection of major crack objects on the surface based on Otsu’s binarization method, and (4) highlighting crack objects by morphological operations. The proposed technique was validated on a spillway wall of a concrete dam structure in South Korea. The scanned data was compressed up to 50% of the original scanned data, while showing good performance in detecting cracks with various shapes. Keywords: terrestrial laser scanning; 3D scan data; octree data structure; crack detection; image processing; Otsu’s method; K-means clustering

1. Introduction 1.1. Background and Purpose of the Study Terrestrial laser scanning (TLS) is a system that can acquire the three-dimensional (3D) position information of an object remotely with a laser. Terrestrial laser scanning has been increasingly used in the surveys of buildings, bridges, and collapsed rock slopes because it allows easy and rapid surveying, even in the lack of access due to its contactless data collection. Terrestrial laser scanning is mainly Appl. Sci. 2018, 8, 2373; doi:10.3390/app8122373

www.mdpi.com/journal/applsci

Appl. Sci. 2018, 8, 2373

2 of 19

developed for the purpose of providing a rapid and accurate remote sensing for 3D objects modeling but can be also used to monitor the structural condition. Moreover, TLS shows an equivalent capacity in damage detection and deformation (or volumetric change) evaluation to conventional photographic methods [1]. For the detection of damages such as crack, spalling, and abrasion, Anil et al. [2] showed the feasibility of TLS by presenting that the measurable width, depth, and orientation of cracks according to the sampling interval of the laser scanner, and the range of the laser beam. Giri and Kharkovsky [3] proved that a laser displacement sensor can measure a cracks’ width up to 0.7 mm on a concrete specimen. Laefer et al. [4] configured the fundamental mathematics to determine the detectable minimum crack width with a terrestrial laser scanner in unit-based masonry by a parametric study using orthogonal offset, interval scan angle, crack orientation, and crack depth. Li et al. [5] proposed a crack detection method based on a second derivative image processing of thermal images from laser heated spots, comparing with the dye penetrant inspection. Kim et al. [6] presented a detection technique that can simultaneously localize and quantify spalling defects on concrete surfaces using TLS. Valenca et al. [7] developed an automatic method based on image processing and laser scanning in assessing cracks in bridges. The captured images were orthorectified by geometric information surveyed by TLS, and then an image processing was performed for crack characterization. Xu et al. [8] suggested an automatic crack detection method through TLS for concrete beam surfaces using a few simple imaging processes techniques such as conversion to gray scale, median filtering, and thresholding. For the deformation or deflection of structures, Tsakiri et al. [9] took advantage of planes fitted to point clouds extracted from the TLS data when gauging the displacement for monitoring of structural deformation. The plane model is suitable for small areas; thus the section is divided into grid cells. As for monitoring of tunnels, Van Gosliga et al. [10] configured the tunnel model with a cylinder to detect deformations in a bored tunnel. Chang et al. [11] analyzed the deformation of a structure surface by statistical regression such as a polynomial function. Koch [12] proposed the lofting method by estimation of control points and the method showed good accuracy in the 3D non-uniform rational b-splines (NURBS) surface. Jafaru et al. [13] presented a 3D point cloud change analysis approach for tracking small movements over time through a combination of a direct point-wise distance metrics in conjunction with statistical sampling. Khaloo and Lattanzi [14] suggested an algorithm to identify structural anomalies in 3D point clouds of infrastructure systems by linear and non-linear transformations from RGB to non-RGB spaces. Above these, many researchers have carried out various studies on the damage detection and deformation quantification on various structures by TLS or other equivalent methods, such as photography [15–20]. Although these studies have evaluated TLS for the purpose of structural health monitoring, the data capacity, resolution, and accuracy of crack recognition are still pointed out as drawbacks of TLS [21,22]. More specifically for damage detection algorithms, the data from photogrammetry, laser scanners [23], and microwave [24] have been utilized, and the associated imaging processing methods such as edge detection techniques [25], histogram matching, image filtering and change detection [26], and automatic thresholding [27] have been proposed by many researchers. Also, in conjunction with image processing, a logical classification has been applied to find and identify various types of defects for the reduction of inspection time and efforts. Sinha et al. [28] used classification of neuro-fuzzy networks and Khanfar et al. [24] applied the fuzzy logic technique to find concrete structural defects. Moon and Kim [29] used neural networks to detect cracks from images, and Kawamura et al. [30] used genetic algorithms to extract crack patterns from digital images. In many papers, the classification methods show a diversity of algorithms, which means that the operational environment is different from place to place. For example, there are many stains, sediments, joints, efflorescence, and micro-cracks on the surface of structures, and it is not easy to distinguish them from the major structural cracks.

Appl. Sci. 2018, 8, 2373 x FOR PEER REVIEW

3 of 19

1.2. Scope and Method of Research 1.2. Scope and Method of Research In this study, an effective technique is proposed to detect main cracks on a concrete structure by In this study, an effectivealgorithms technique is proposed to cracksfrom on athe concrete applying imaging processing appropriate to detect imagesmain extracted octreestructure structureby of applying imaging processing algorithms appropriate to images extracted from the octree structure of TLS data. The proposed technique aims to overcome the pre-mentioned drawbacks while fully taking TLS data. The proposed technique aims to overcome pre-mentioned drawbacks fully taking advantage of the contactless measurement of TLS. Tothe respond to the drawbacks of while the data capacity, advantage of the contactless measurement of TLS. To respond to the drawbacks of the data capacity, resolution, and accuracy of crack recognition, a data compression scheme is proposed based on octree resolution, and which accuracy crack recognition, a data compression scheme is proposed based on octree data structure, hasofbeen applied to various research fields such as computer graphics, robotics, data structure, hasto been to various research fieldsof such as computer robotics, etc. [31–36] andwhich proved be aapplied reasonable approach in terms memory usage graphics, and search speed etc. [31–36] and proved to be a reasonable approach in terms of memory usage and search speed [37,38]. [37,38]. Moreover, an improved algorithm using a 3D significant eigenvector-based shape descriptor Moreover, improved using a 3Dimage significant eigenvector-based shape descriptor allows allows the an octree methodalgorithm to provide a better recognition [39]. the octree method to method provide aisbetter image recognition [39]. The proposed summarized by the following work procedures. First, a shape The proposed method is summarized by the following procedures. First, athe shape information information model that can maintain the shape change inwork structure throughout laser scan data model that can maintain the shape change in structure throughout the laser scan data was constructed. was constructed. The shape information model uses an octree data structure through point-cloud The shape and information model uses anefficiently octree data structure through point-cloud processing and processing voxelization, and it can downsample and visualize a large-scale scan data. voxelization, and it can efficiently downsample and visualize a large-scale scan data. To respond To respond to the diversity of operation environments in detecting cracks, a combination of image to the diversity of operation in detecting cracks, a combination of image processing processing algorithms, such environments as K-means clustering [40], subtraction with median filtering [41], algorithms, such as K-means clustering [40], subtraction with median filtering [41], binarization [42], binarization [42], and morphological operations were proposed. The crack detection procedure and morphological operations were proposed. The crack detection procedure involved the following involved the following steps: (1) segmentation for preprocessing, (2) classification of an image into 3 steps: (1) (i.e., segmentation forstructural preprocessing, (2) classification of surface) an image intoK-means 3 categories (i.e., categories background, joints, and sediments, and using clustering background, structural joints, and sediments, and surface) using K-means clustering according to according to the color similarity, (3) conversion to gray scale and image enhancement using median the color and similarity, (3) conversion to gray scale andofimage enhancement median filtering and filtering improved subtraction, (4) deletion non-crack parts onusing the surface by K-means improved subtraction, (4) deletion of non-crack parts on the surface by K-means clustering results, clustering results, (5) detection of major crack objects on the surface based on the Otsu’s binarization (5) detection crack objects the surface based on theoperations. Otsu’s binarization method, and (6) method, and of (6)major highlighting crackon objects by morphological The overall workflow is highlighting crack objects by morphological operations. The overall workflow is presented in Figure 1. presented in Figure 1.

Figure 1. The workflow of crack detection by terrestrial laser scanning (TLS) measurement. measurement.

Then, the proposed technique is validated on a spillway wall of a concrete dam structure in South Korea. The test wall is larger than 100 m2 and has many cracks with various shapes. The wall

Appl. Sci. 2018, 8, 2373

4 of 19

Then, the proposed technique is validated on a spillway wall of a concrete dam structure in South Korea. The test wall is larger than 100 m2 and has many cracks with various shapes. The wall also has many structural joints and contaminated by serious sediments, which are expected to significantly impede the detection of cracks on the wall surface. The wall is scanned by a Leica ScanStation C5 model at 32 m distance. Along with the original scanned data, the compressed data up to 30%, 50%, and 80% are prepared by the proposed octree data structure. Then, the proposed crack detection procedure is applied to original and compressed images, and the results from the images are compared with the visual inspection result to validate the performance of the proposed technique. 2. TLS Measurement and Data Compression 2.1. 3D Laser Scanning Terrestrial laser scanning is a technology that helps users easily collect data and generates a 3D virtual model of the physical asset using point clouds as a set of points in 3D space. Laser scanning refers to deflection of the laser beam due to movement of the reflector (mirror), reflection of the laser beam on the object surface, and reception of the reflected laser beam. The laser scanning method includes a triangulation method, a phase difference method, and a time-of-flight method [43]. Laser scanning consists of on-site installation, positioning scan, surface scan, and post-processing of point group coordinate data. The hardware required to operate the laser scanner in the field includes a scanner, a laptop computer, a power supply, and a conveying and a mounting device. When the hardware installation is finished, the scan angle is adjusted to set the scan range on a target object [44]. Since the entire model is achieved by merging the point group coordinate values obtained by each scan, a reference point (reflection target) should be provided to the target point so that at least four reference points are overlapped for each adjacent scanning point. The reference point installed at this time may be absolute coordinates, or relative coordinates around the installation position of the laser scanner. Point coordinate values are accumulated in database (DB) form and can be restored as a 3D image using a post-processing program or numerical analysis of the degree of deformation compared with the original design drawing. 2.2. Voxelization and Octree Data Structure Voxelization conceptualizes geospatial spaces and express them as volume elements (voxels) using voxel data structures. A voxel is a rectangular parallelepiped block defined in terms of its length (l), width (w), and height (h) and voxel position is indexed by column (i), row (j) and layer (k). The laser scanning point (x, y, z) is transformed to the position of the voxel (i, j, k) [45]. Octree is a data representation method that creates a bounding box for a given shape, and stores it as a visualization object if the shape is contained in the hexahedron equally divided in the x, y, and z coordinates [46]. The octree divides the bisected space in each direction of the x, y, and z axes into eight subspaces and expresses them in a finer area. The eight sub-spaces can be stored in a tree structure as an element called a node, and the depth level of the octree is determined by the number of divided bounding boxes [47]. In this study, the 3D object model (Voxel model) is first generated to easily remove empty nodes by checking empty cells in the sub and left nodes. Thus, it is efficient to implement segmentation and octree in the spatial properties of 3D objects rather than in irregularly distributed point clouds. Octree has the largest number of leaf nodes corresponding to NL = 8D , and this is equivalent to a 3D grid with a resolution of 2D × 2D × 2D . Figure 2 shows the number of nodes according to the octree depth level. Here, octree is used to improve processing speed and to save space, and the depth first (DF)-representation proposed by Tamminen [48] is the most popular way to encode octree. This method sequentially stores the information of the nodes encountered in a depth-first search of the octree, and each node is displayed by black, white, and gray, as shown in Figure 2. A black node indicates that the area is full. A white node means it is completely empty, and a gray node means

Appl. Sci. 2018, 8, 2373

5 of 19

Appl. Sci. 2018, 8, x FOR PEER REVIEW

5 of 19

the node is partially black. The gray nodes here indicate that further segmentation may be required. The marked as black or white no longer split. An can octree cantobea set toof a level to be hexeshexes marked as black or white no longer split. An octree be set level depthoftodepth be divided divided can be repeatedly bythe setting thethe size of the bounding box. or can beorrepeatedly divided divided by setting size of bounding box.

Figure 2. 2. The The depth depth first first (DF)-representation (DF)-representation of the octree data structure. Figure

3. Image Processing 3. Image Processing The point cloud data obtained by the TLS is compressed by the octree data structure, and it can The point cloud data obtained by the TLS is compressed by the octree data structure, and it can be converted to an image. To detect cracks from the converted image, a series of image processing be converted to an image. To detect cracks from the converted image, a series of image processing techniques are used. Image processing for crack detection on concrete surfaces involves the following techniques are used. Image processing for crack detection on concrete surfaces involves the following seven steps: (1) K-means clustering, (2) conversion into the gray-level image, (3) median filtering and seven steps: (1) K-means clustering, (2) conversion into the gray-level image, (3) median filtering and improved subtraction, (4) removal of unnecessary parts using K-means clustering, (5) binarization, improved subtraction, (4) removal of unnecessary parts using K-means clustering, (5) binarization, (6) morphological closing, and (7) blob removal using area and eccentricity. These seven steps are (6) morphological closing, and (7) blob removal using area and eccentricity. These seven steps are tabulated in Table 1. tabulated in Table 1.

Step

Step 1

1 2 2 3 3 4

4

5

5 6 6 7 7

Table 1. Seven image processing steps for the crack detection. Table 1. Seven image processing steps for the crack detection. Operation

Operation K-means Clustering: Divide original image into three images according to color Convert original image into gray-level image similarity Subtract original gray-scale image to remove background Convertmedian-filtered original imageimage into from gray-level image Remove parts according to structural joints, sediments, usingtosegment Subtractimage median-filtered image from original gray-scaleetc. image removeinformation background obtained by k-means clustering Remove image parts according to structural joints, sediments, etc. using segment Binarization Otsu’sby method (threshold = 0.2 × Ostu’s threshold) informationusing obtained k-means clustering Binarization using Otsu’s method (threshold = pixels 0.2 × Ostu’s threshold) Morphological closing to connect slightly-separated Morphological closing to connect slightly-separated pixels Remove blobs (i.e., pixel groups) with the controlled area and eccentricity Remove blobs (i.e., pixel groups) with the controlled area and eccentricity K-means Clustering: Divide original image into three images according to color similarity

3.1. K-Means Clustering 3.1. K-Means Clustering K-means clustering is one of the most famous unsupervised learning methods [40]. K-means K-means is unlabeled one of thedata most famous unsupervised learningbetween methods [40]. K-means clustering aimsclustering to partition into K clusters using the distance each pair of data, clustering aims to partition unlabeled data into K clusters using the distance between each of and the cluster to be included is determined to minimize the mean distance in each cluster. pair Given data, and the cluster to be included is determined to minimize the mean distance in each cluster. a set of data x = { x1 , x2 , · · · , xk } where each data has the dimension, K sets S = {S1 , S2 , · · · , Sk } are x within-cluster  {x1 , x2 , , xvariance Given a set of data the determined to minimize as each data has the dimension, K sets k } where

S  {S1 ,S2 ,

,Sk } are determinedK to minimize the within-cluster variance as K argmin ∑ S

∑ kx − µi k2 = argmin ∑K |Si |Var(Si )

K1 x ∈S i= i

arg min  x  i S

i 1 xSi

2

S

i =1

S

i 1

 arg min  Si Var ( Si )

(1) (1)

Appl. Sci. 2018, 8, 2373 Appl. Sci. 2018, 8, x FOR PEER REVIEW

6 of 19 6 of 19

where , |S,i |,| Var (i.e., centroid), count, andcount, variance data in Sof i ) denote i , respectively. Si | ,(SVar (Si ) mean whereµi denotevalue mean value (i.e., centroid), andofvariance data in Si , i Based on the law of total variance [49], the minimizing the within-cluster variance equals to maximizing respectively. Based on the law of total variance [49], the minimizing the within-cluster variance the out-of-cluster variance. equals to maximizing the out-of-cluster variance. The basic algorithm of K-means clustering starts from selecting K points as the initial centroids. The basic algorithm of K-means clustering starts from selecting K points as the initial centroids. Then the K clusters are initially formed by assigning all points to the closest centroid and the centroid Then the K clusters are initially formed by assigning all points to the closest centroid and the centroid of each cluster is computed. Then reforming of K clusters and recomputing the centroid of each of each cluster is computed. Then reforming of K clusters and recomputing the centroid of each reformed cluster is repeated until the centroids do not change. reformed cluster is repeated until the centroids do not change. In the image processing, the data corresponds to the color vector in each pixel, and the Euclidean In the image processing, the data corresponds to the color vector in each pixel, and the Euclidean distance of the color vector represents the dissimilarity of the colors of the pixels. By applying K-means distance of the color vector represents the dissimilarity of the colors of the pixels. By applying Kclustering, the image can be segmented into K pieces of pixels according to their color similarity means clustering, the image can be segmented into K pieces of pixels according to their color corresponding to the concrete surface conditions. Figure 3 shows an example of image segmentation similarity corresponding to the concrete surface conditions. Figure 3 shows an example of image by K-means clustering. The original image in Figure 3a is segmented into three differently colored segmentation by K-means clustering. The original image in Figure 3a is segmented into three images in Figure 3b. The blue segment (BS) illustrates the background, and the green segment (GS) differently colored images in Figure 3b. The blue segment (BS) illustrates the background, and the illustrates the concrete surface. Notably, the red segment (RS) highlights the sediments and structural green segment (GS) illustrates the concrete surface. Notably, the red segment (RS) highlights the joints that are generally dark-colored in the original image, even compared to the cracks on the concrete sediments and structural joints that are generally dark-colored in the original image, even compared surface. Since the sediments and structural joints are the biggest impediments in identifying cracks to the cracks on the concrete surface. Since the sediments and structural joints are the biggest from the images, the third segments obtained at this step will be used in Step 4 to tweeze out the impediments in identifying cracks from the images, the third segments obtained at this step will be impediment for successful image processing. used in Step 4 to tweeze out the impediment for successful image processing.

(a)

(b)

Figure 3. 3. Image Image segmentation segmentation by by K-means K-means clustering clustering(K (K== 3). 3). (a) (a) Original image; (b) (b) Image Image segmented segmented Figure Original image; by K-means clustering (K = 3). by K-means clustering (K =

3.2. 3.2.Conversion Conversioninto intothe theGray-Scale Gray-ScaleImage Image As gray-level image As aa next next step, step, the the conversion conversion of of the the true true color color image image (RGB) (RGB) into into aa gray-level image by by removing removing hue and saturation information while maintaining luminance is done. In this process, the hue and saturation information while maintaining luminance is done. In this process, the amount amount of of 8 )3 = 16,777,216 color) is converted into image data decreases. Specifically, the RGB color of 24-bit ((2 8 3 image data decreases. Specifically, the RGB color of 24-bit ((2 ) = 16,777,216 color) is converted into 8 8-bit 256 level). level). 8-bit gray-level gray-level (2 (28 = = 256 3.3. Median Filtering and Improved Subtraction 3.3. Median Filtering and Improved Subtraction The obtained gray-level image is processed with improved subtraction processing. Subtraction The obtained gray-level image is processed with improved subtraction processing. Subtraction processing typically eliminates the slight variation such as shadings or irregularly noticeable processing typically eliminates the slight variation such as shadings or irregularly noticeable components. Thus, it is for elimination of the noise as well as the illuminated components from components. Thus, it is for elimination of the noise as well as the illuminated components from cracks. cracks. Figure 4 illustrates how the subtraction finds cracks against the other objects including shading. Figure 4 illustrates how the subtraction finds cracks against the other objects including shading. Illuminance of the original gray-level visualizes objects in the image, and the median filtering removes Illuminance of the original gray-level visualizes objects in the image, and the median filtering out sharp illuminance changes that represent small noises or cracks and remains smoothed background removes out sharp illuminance changes that represent small noises or cracks and remains smoothed image representing concrete surface, large objects, and shading. Then, the subtraction of the original background image representing concrete surface, large objects, and shading. Then, the subtraction of the original image from the filtered image removes out the background image while remaining

Appl. Sci. 2018, 8, 2373

7 of 19

Appl. Sci. 2018, 8, x FOR PEER REVIEW

7 of 19

image from the filtered image removes out the background image while remaining objects with sharp objects with Since sharpthe illuminance. Since the is visualized by profile very low profile with illuminance. crack is visualized bycrack very low illuminance withilluminance shape change, the crack shape can change, the crackfrom object be highlighted the subtracted image,image if combined with object be highlighted thecan subtracted image, iffrom combined with subsequent processing to subsequent imagenoises. processing to remove out small noises. remove out small

Figure 4. Crack visualization visualization by by subtraction subtraction processing processing [41]. Figure 4. Crack [41].

The The subtracted image is is The median median filtering filtering converts converts the the gray-level gray-level image image to to aa smoothed smoothed one. one. The subtracted image obtained by the Equation (3). obtained by the Equation (3). ( ) medianxxj ∈j RRj I ((xxjj))−  I ( xii )   median   Is (Ixsi ()x= max (2) (2)  i )  max  00  





where is the the intensity intensity of the pixel xi, Rii is is the the neighborhood neighborhood of that pixel, and I(xjj) is the the intensity intensity where IIss (xii) is of the pixels in Ri . When When the the result of the subtraction subtraction is a negative negative number, the result is represented with 0. In addition, the subtraction result from the Equation (2) was used to get an enhanced image by a manually decided threshold value T. T. The The thresholding thresholding operation operation is is processed processed by by the the Equation Equation(4). (4). ( ) 2  I ( x, y) if I ( x, y)  T  ) max max 2 × Iss ( x, y) if Is (s x, y) > T  (3 I ISI (ISx,( xy,)y= (3)  IIss((xx,, y) if IIs (s (x,x,yy) )≤TT  where IIIS IS(x,y) where (x,y) and and IIss(x,y) (x,y) are are the the results results of of the the improved improved and and the the original original subtraction subtraction method, method, respectively.The Thethreshold threshold value is determined manually. this process, the illuminance respectively. value T is Tdetermined manually. By this By process, the illuminance difference differencecracks between noises is larger and itthe improves the noisefor reduction the next step. between andcracks noises and is larger and it improves noise reduction the nextfor step.

3.4. Removal of Unnecessary Parts Using K-Means Clustering In Section 3.1, RS data have been separated using the K-means clustering to remove out the parts corresponding to the structural joints and sediments. However, However, the gradient at the boundary of RS data may of of thethe adjacent GSGS data. Since objects withwith a linear and thin may still stillbe beincluded includedatatthe theboundary boundary adjacent data. Since objects a linear and shape are to betoidentified during the image processing, the the gradient remaining at the boundary of thin shape are be identified during the image processing, gradient remaining at the boundary the GSGS data can bebe misrecognized asascracks. of the data can misrecognized cracks.InInthis thisstudy, study,the theGS GSdata dataisisdilated dilatedby by55 pixels pixels and and the overlap of removed from thethe GSGS data, resulting in erosion of the of the the dilated dilatedRS RSdata dataand andthe theGS GSdata datais is removed from data, resulting in erosion of GS at the interface withwith RS data as shown in Figure 5. This process cancan significantly minimize the the data GS data at the interface RS data as shown in Figure 5. This process significantly minimize false recognition of cracks. the false recognition of cracks. 3.5. Binarization Binarization is the simple and intuitive method in image segmentation. Object pixels can be simply extracted from background by image binarization. This step converts an input gray-level image to a binary image where the values of points are expressed by 0 (black) or 1 (white). The pixel

threshold T by minimizing the within variance of two groups separated by the thresholding operator. The method enables automatic thresholding, but it requires assuming a bimodal distribution of graylevel images. Considering the cracks are a small portion of the image and have high intensity against the background (after removing the dilated RS data) in the subtracted image, the threshold determined by Otsu’s method can be modified to minimize identification of the other objects against8 of 19 Appl. Sci. 2018, 8, 2373 cracks. This in this study, 0.2 times T (=0.2 T) is used to binarize crack objects.

Figure 5. Dilatedred redsegment segment (RS) (RS) and segment (GS). Figure 5. Dilated anderoded erodedgreen green segment (GS).

3.5. Binarization 3.6. Morphological Operation: Closing and Blob Removal Morphological were pursued to analyze binarysegmentation. images based on the form Binarization is the operations simple and intuitive method in image Object pixelsand can be structure of the image. Morphological operations included labeling, dilation, erosion, closing,image simply extracted from background by image binarization. This step converts an input gray-level opening, etc. Among various operations, closing aims toor fill1 the holes The between to a binary image where the values of pointsmorphological are expressed by 0 (black) (white). pixelthe whose objects in the binarized image (i.e., connected pixel groups), and in this study it was used to connect intensity is greater than a threshold T is assigned into 1, and the pixel whose intensity is equal to or crack objects by filling the small breaks between the objects and build a large crack object. The smaller than T is assigned into 0. maximum filling size as set as 2 pixels not to distort the original crack objects. There are various binarization methods according to the method of determining the threshold After the morphological closing, blobs that had small areas and small eccentricity (ratio between T, and Otsu’s method used binarization method. The out. OtsuInmethod selects the major and minor[42] axesisofthe an most object)widely as shown in Figure 6 were finally removed this study, a threshold T by minimizing the within variance of two groups separated by the thresholding operator. the objects with areas smaller than 100 pixels or eccentricity smaller than 0.3 in the original image The method enables were removed out.automatic thresholding, but it requires assuming a bimodal distribution of gray-level images. Considering the cracks are a small portion of the image and have high intensity against the background (after removing the dilated RS data) in the subtracted image, the threshold determined by Otsu’s method can be modified to minimize identification of the other objects against cracks. This in this study, 0.2 times T (=0.2 T) is used to binarize crack objects.

3.6. Morphological Operation: Closing and Blob Removal Morphological operations were pursued to analyze binary images based on the form and structure of the image. Morphological operations included labeling, dilation, erosion, closing, opening, etc. Among various operations, morphological closing aims to fill the holes between the objects in the binarized image (i.e., connected pixel groups), and in this study it was used to connect crack objects by filling the small breaks between the objects and build a large crack object. The maximum filling size as Figure 6.crack Majorobjects. and minor axes of objects. set as 2 pixels not to distort the original After the morphological closing, blobs that had small areas and small eccentricity (ratio between the major and minor axes of an object) as shown in Figure 6 were finally removed out. In this study, the objects with areas smaller than 100 pixels or eccentricity smaller than 0.3 in the original image were removed out.

crack objects by filling the small breaks between the objects and build a large crack object. The maximum filling size as set as 2 pixels not to distort the original crack objects. After the morphological closing, blobs that had small areas and small eccentricity (ratio between the major and minor axes of an object) as shown in Figure 6 were finally removed out. In this study, the2018, objects with areas smaller than 100 pixels or eccentricity smaller than 0.3 in the original image9 of 19 Appl. Sci. 8, 2373 were removed out.

Figure 6. 6.Major axesofofobjects. objects. Figure Majorand and minor minor axes

Figure 7 shows an example showing how the seven image processing steps in Table 1 affected the original image to the end of crack identification. The figure shows that the cross-shape cracks on the original image are highlighted against the structural joints, sediments, and noises by proceeding the steps.

Figure 7. Example of the proposed image processing.

Appl. Sci. 2018, 8, 2373

10 of 19

4. Construction of Octree-Based Shape Information Model 4.1. Test-Bed and Equipment Laser scanning was performed on the spillway left wall of a concrete dam structure in South Korea. Figure 8 shows the panoramic picture of the TLS measurement setup, and the scan was performed on the second wall surrounded by red lines. The left and right heights of the wall were 11.741 m and 10.118 respectively, and the width was 10.00 m. Figure 9 and Table 2 show the enlarged view10 ofofthe Appl. Sci.m, 2018, 8, x FOR PEER REVIEW 19 Appl. Sci.structure 2018, 8, x FOR REVIEW 10 of 19 target andPEER the information of main crack pattern map through visual inspection. It should be noted that the cracks do notthat represent the do stains, sediments,the efflorescence, and microcracks on inspection. It should be noted the cracks not represent stains, sediments, efflorescence, inspection. It should be noted that the cracks do not represent the stains, sediments, efflorescence, the surface. and microcracks on the surface. and microcracks on the surface.

Figure 8. The panoramic view of TLS measurement set-up. Figure 8. The panoramic view of TLS measurement set-up.

(a) (a)

(b) (b)

Figure 9. The enlargement of the target structure and crack pattern map by visual inspection. (a) Figure 9. The The enlargement enlargementofofthe the target structure crack pattern by visual inspection. (a) Figure 9. target structure andand crack pattern map map by visual inspection. (a) Target Target structure; (b) Crack pattern map. Target structure; (b)pattern Crack pattern structure; (b) Crack map. map. Table 2. The width and approximate length of cracks. Table 2. The width and approximate length of cracks.

No. No. 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9

Width (mm) Width (mm) 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2

Length (mm) Length (mm) 1070 1070 1810 1810 1840 1840 650 650 960 960 1350 1350 640 640 450 450 1480

No. No. 11 11 12 12 13 13 14 14 15 15 16 16 17 17 18 18 19

Width (mm) Width (mm) 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.2

Length (mm) Length (mm) 770 770 1560 1560 670 670 810 810 1330 1330 890 890 620 620 850 850 1500

Appl. Sci. 2018, 8, 2373

11 of 19

Table 2. The width and approximate length of cracks. No.

Width (mm)

Length (mm)

No.

1 2 3 4 5 6 7 8 9 10

0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.1

1070 1810 1840 650 960 1350 640 450 1480 740

11 12 13 14 15 16 17 18 19

Width (mm) Length (mm) 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.2

770 1560 670 810 1330 890 620 850 1500

In this study, we used the Leica ScanStation C5 model as specified in Table 3 to scan the test-bed. The minimum and maximum measurement distance was 31.907 m and 31.918 m, respectively. The left and right angles were 11◦ and 11.5◦ and the up and down angles were 90◦ and −45◦ as presented in Appl. Sci. 2018, 8, x FOR PEER REVIEW 11 of 19 Table 4. The scan mode was set to high-resolution mode, the scanning time was about 28 min and 12 s, and theinscan (PTS format) sizeset was MB. The 3D scan data had x,time y, and coordinates at each Tabledata 4. The scan mode was to 19.56 high-resolution mode, the scanning waszabout 28 min and point. 12 Linux Ubuntu 12.04.2 LTS and C++ were used to configure the octree shape information model s, and the scan data (PTS format) size was 19.56 MB. The 3D scan data had x, y, and z coordinates and development language, respectively. at each point.programming Linux Ubuntu 12.04.2 LTS and C++ were used to configure the octree shape information model and development programming language, respectively. Table 3. Specifications of the terrestrial laser scanner. Table 3. Specifications of the terrestrial laser scanner. Model Leica ScanStation C5

Model Leica ScanStation C5 11 of 19 Measurement distance 300 m Measurement distance 300 m in Table 4. The scan mode was set to high-resolution mode,From the scanning min and 0–50time m: was 4.5 about mm 28 (FWHH-based), Spot size 12 s, and the scan data (PTS format) size was 19.56 MB. The 3D scan data had x,4.5 y, and z coordinates From 0–50 m: mm (FWHH-based), 7 mm (Gaussian-based) Spot size at each point. Linux Ubuntu 12.04.2 LTS and C++ were used to configure the octree shape information 7 mm (Gaussian-based) model and development programming language, respectively. Range accuracy 35 m at 300 m Range accuracy 35 m at 300 m Table 3. Specifications of the terrestrial laser scanner. Precision 2 mm Precision Model 2 mm Leica ScanStation C5 Speed 50,000 point/s Speed 50,000 point/s Measurement distance 300 m ◦ (max) From 0–50 m: 4.5 mm (FWHH-based), Horizontal 360 Spot size Horizontal 360° (max) Range 7 mm (Gaussian-based) ◦ (max) Range Vertical 270 Range accuracy 35 m at 300 m Vertical 270° (max) LaserClass ClassPrecision 3R2 mm (IEC Laser 3R (IEC60825-1) 60825-1) Speed 50,000 point/s Horizontal 360° (max) Memory 80 GB Memory Range 80 GB Appl. Sci. 2018, 8, x FOR PEER REVIEW

Laser Class valuesMemory of terrestrial

Vertical 270° (max) 3R (IEC 60825-1) scanner.80 GB

Table 4. Setting laser scanner. Table 4. Setting values of terrestrial laser

Table 4. Setting values of terrestrial laser scanner. Scan Parameter Setting Value Scanning View Scan Parameter Setting Value Scanning View Scan Parameter Setting Value Scanning View Left −11° ◦ Left −11° Left −11 Right 11.5° 11.5° Right Top Right 11.5◦90° Top 90° Bottom −45° ◦ Top Bottom −45° 90

Bottom

Resolution Mode

Resolution Mode

Resolution Mode

−45◦

Highest Resolution

Highest Resolution

Highest Resolution 4.2. Construction and Visualization of an Octree-Based Shape Information Model

4.2. Construction and Visualization of an Octree-Based Shape Information Model Figure 10 shows the target image obtained from the 3D laser scan data with 1,367,274 points extracted from the target spillway wall. The octree-based shape information model was constructed

Figure 10 shows the target obtained from thefirst3D laser scan 1,367,274 points accordingimage to the flow chart shown in Figure 11. The step was to convert the data scan datawith from pixels to voxels (in 3D pixels). The second step was to convert the voxel model to the octree data structure, 4.2. from Construction and Visualization of an Octree-Based Shape Information Model extracted the target spillway wall. The octree-based shape information model was constructed and the last step was the creation of an octree-based shape information model. according to the flow chartthe shown inimage Figureobtained 11. Thefrom first step was to convert scan data from pixels Figure 10 shows target the 3D laser scan datathe with 1,367,274 points to voxels (in 3D pixels). The second step was to convert the voxel model to the octree data structure, extracted from the target spillway wall. The octree-based shape information model was constructed and theaccording last steptowas the creation of an information model. the flow chart shown in octree-based Figure 11. The shape first step was to convert the scan data from pixels to voxels (in 3D pixels). The second step was to convert the voxel model to the octree data structure, and the last step was the creation of an octree-based shape information model.

Appl. Sci. 2018, 8, x 2373 FOR PEER REVIEW Appl. Sci. 2018, 8, x FOR PEER REVIEW

12 of 19 12 of 19

Figure10. 10.The Thescan scan image image extracted extracted from Figure fromthe theTLS TLSdata. data.

Figure 11. Process of building the octree information model.

Figure Figure 11. 11. Process Process of of building building the the octree octree information information model. model. The first step (i.e., the creation of a voxelization model) involved converting the point-cloud data to The the voxel type.(i.e., In this the point cloud was read on the basis of each X, Y, and direction. The first step thestep, creation of aa voxelization model) involved converting theZpoint-cloud data creation voxelization model) read data was reconstructed as a voxel model and was used to generate the root node in the octree to the voxel type. In this step, the point cloud was read on the basis of each X, Y, and Z direction. The the voxel type. In this step, the point cloud was read on the basis of each X, Y, and Z direction. data structure. The voxel model stores the x, y, and z coordinate values and depth, height, and width readread datadata waswas reconstructed as aasvoxel model and was used The reconstructed a voxel model and was usedtotogenerate generatethe theroot rootnode nodein in the the octree octree values of the 3D were as input parameters in building the octree information data structure. Theobject voxelmodel. modelThese stores the used x, y, and z coordinate values and depth, height, and width x, y, model. The algorithm consists of a numerical codeinput for generating anin octree data the structure via a voxel values of the 3D object model. These These were were used used as as input parameters parameters in building building the octree octree information information model, and it determines the octree resolution by setting a res value. After creating the octree, the model. The algorithm consists of a numerical code for generating an octree octree data structure via a voxel inner nodes of the octree were iteratively divided until they reached the res value. After the model, and and ititdetermines determinesthe theoctree octree resolution by setting a res value. After creating the octree, the resolution by setting a res value. After creating the octree, the inner segmentation was completed, the center point (x, y, z) of the 3D object and the depth, height, and inner nodes of thewere octree were iteratively divided until they reached the res the value. After the nodes of values the octree iteratively until reached theof res segmentation width were sequentially readdivided and stored inthey the inner nodes thevalue. octree.After If all the point values segmentation was completed, the(x, center point (x, y,object z) of and the 3D object and the depth, height, and was completed, the center point y, z) of the 3D the depth, height, and width values were stored, the empty areas were removed and the octree file was finally created. width values were sequentially read and stored in the inner nodes of the octree. If all the point values were sequentially and stored in the ofwas the used octree. If all thethe point values were stored, Octovis [50],read a viewer program withinner a 3D nodes tool kit, to identify shape information of were stored, the empty areas were removed and the octree file was finally created. thethe empty areas were removed and the octree file was finally created. generated octree. The number of points on the target wall was originally 1,367,274, and the file

Octovis [50], a viewer program with a 3D tool kit, was used to identify the shape information of the generated octree. The number of points on the target wall was originally 1,367,274, and the file

Appl. Sci. 2018, 8, 2373

13 of 19

Octovis [50], a viewer program with a 3D tool kit, was used to identify the shape information of the generated octree. The number of points on the target wall was originally 1,367,274, and the Appl. Sci. 2018, 8, x FOR PEER REVIEW 13 of 19 file size was 19.56 MB and it was reduced to about 30%, 50%, and 80% as presented in the Table 5 to size confirm could be reduced detectedtoeven at 30%, a small number of points. Also, in thethe extracted scan was that 19.56cracks MB and it was about 50%, and 80% as presented Table 5 to images according to the compression ratio is shown in Figure 12. confirm that cracks could be detected even at a small number of points. Also, the extracted scan images according to the compression ratio is shown in Figure 12.

Table 5. The number of compressed points and data size by octree structure.

Table 5. The number of compressed points and data size by octree structure. Point Data Data Size (MB) Octree Level Div. Compression (%) (%) Points Data Points Octree Level Point Compression Data Size (MB)

Div. OriginalOriginal 1,367,274 1,367,274 934,061 Case1 Case1 934,061 Case2 678,319 Case2 678,319 Case3 269,942 Case3 269,942

11 99 10 10 11 11

(a)

0 0 31.631.6 50.350.3 80.2

80.2

(b)

19.56 13.36 9.7 3.86

19.56 13.36 9.7 3.86

(c)

Figure TheTLS TLSscan scanimages imagesby by point point compression compression rates. 2; 2; (c)(c) Case 3. 3. Figure 12.12.The rates.(a) (a)Case Case1;1;(b) (b)Case Case Case

4.3.4.3. Image Processing Image Processingfor forCrack CrackDetection Detection The validity theproposed proposedalgorithm algorithm was was evaluated from. The original The validity ofofthe evaluated for forthe theextracted extractedimages images from. The original snapshot resolution was 1513 × 810 pixels, but 3026 × 1620 pixels (×2 resolution) was applied for snapshot resolution was 1513 × 810 pixels, but 3026 × 1620 pixels (×2 resolution) was applied precise crack detection and it was composed of RGB color. First, the original and compressed images for precise crack detection and it was composed of RGB color. First, the original and compressed were were processed using using the procedure described in the in Table Based on theon compression rate, the images processed the procedure described the 1. Table 1. Based the compression rate, accuracy of crack detection through the proposed imaging process was evaluated as shown in Figure the accuracy of crack detection through the proposed imaging process was evaluated as shown in 13. In13.case 3, the werewere hardly recognized duedue to the poor resolution, so the result waswas notnot Figure In case 3, cracks the cracks hardly recognized to the poor resolution, so the result presented. As a result, most major cracks were identified well even in the Case 2 with compression presented. As a result, most major cracks were identified well even in the Case 2 with compression rate of 50%, and some minor cracks due to drying shrinkage of concrete were recognized as well. The rate of 50%, and some minor cracks due to drying shrinkage of concrete were recognized as well. The optimum values of the parameters for each case are shown in Table 6 and will be applied generally optimum values of the parameters for each case are shown in Table 6 and will be applied generally within the range in the table for other cases.

within the range in the table for other cases.

Table 6. The parameter optimization of three cases. Table 6. The parameter optimization of three cases.

Factors Factors

K-means clustering K-means clustering

Median filter size Median filterlevel size Segmentation Control of area, Segmentation level Eccentricity Control of area, Eccentricity

Original 300 × 550,Original 600 × 100, A 300 × 550, 600 × 500 × 200 A 100, 500 × 200 B 20 C B 23 20 DC D

23 >100, >0.3 >100, >0.3

Case 1 Case 300 × 550, 6001 × 100, 300500 × 550, 600 × × 200 100, 500 × 16 200 16 17

Case 2 300 ×Case 550, 2600 × 100, 300 ×500 550,×600 200× 100, 500 14 × 200 1415

>100,17>0.3

15 >0.3 >100,

>100, >0.3

>100, >0.3

Appl. Sci. 2018, 8, 2373

14 of 19

Appl. Sci. 2018, 8, x FOR PEER REVIEW

14 of 19

(a)

(b)

(c)

(d)

Figure image resultsby bythe the proposed proposed method (a) (a) Crack mapmap by visual Figure 13. 13. TheThe image results methodfor forthree threecases. cases. Crack by visual inspection; (b) Original image; (c) Case 1; (d) Case 2. inspection; (b) Original image; (c) Case 1; (d) Case 2.

For quantitative comparison of each case, true positive (TP), false negative (FN), and number of For quantitative comparison of each case, true positive (TP), false negative (FN), and number of false positive (FP) were analyzed and calculated in Figure 14 and Table 7. The quantitative analysis false positive (FP) were analyzed and calculated in Figure 14 and Table 7. The quantitative analysis presents the evaluation of accuracy and error in comparison with the crack pattern map detected by presents the evaluation and error in comparison the crack pattern map detected visual inspection. Forof theaccuracy quantification, two preconditions arewith assumed: (1) A crack is regarded as a by visual inspection. For the quantification, two preconditions are assumed: (1) A crack is regarded as noise and removed if its length was less than 450 mm (the smallest crack length) in the crack pattern a noise and if its length was less 450 mm smallest crackthan length) inone theby crack pattern map. (2)removed A crack was considered to be TP than if its length was(the in line with more 50% of visual map.inspection. (2) A crack considered to be TP ifthat its length was inofline with more than of one by visual Aswas a result, it was confirmed the accuracy crack detection in the50% original image was bestAs among all theit cases terms of TP, FN, and number of FP, anddetection that the accuracy decreasedimage inspection. a result, was in confirmed that the accuracy crack in the original according to the compression rate. However, it should be noted that the accuracy in even 2 was best among all the cases in terms of TP, FN, and number of FP, and that the accuracycase decreased showed 84% in TP. In the original image, all the cracks were identified, but were not fully satisfied according to the compression rate. However, it should be noted that the accuracy in even case 2 with crack lengths by visual inspection. In cases 1 and 2, the identified crack lengths were often showed 84% in TP. In the original image, all the cracks were identified, but were not fully satisfied with exceeded with ones by visual inspection, which can be considered to be overestimated by resolution crack lengths by visual inspection. In cases 1 and 2, the identified crack lengths were often exceeded reduction. with onesInby visual inspection, which be each considered be overestimated by resolution comparison of the number of can FP for case, thetoerrors increased sharply according toreduction. the In comparison the to number of FP for each case, errors according compression rateofdue erroneous recognition of thethe joints andincreased sedimentssharply as cracks. Thus, itto the compression erroneous recognition the joints sediments Thus, it concludes concludesrate thatdue thetohigher the resolution, theofhigher the and probability that as thecracks. K-means clustering recognizes the crackthe group. that accurately the higher the resolution, higher the probability that the K-means clustering accurately

recognizes the crack group. Table 7. Quantitative comparison of the three cases.

No. Original No. Case 1 Case 2 Original Case 1 Case 2

Table Quantitative comparison three cases. True7.Positive False Negativeof theNumber of False Positive 100% (19/19) 0% (0/19) 0 False True Number of False 95%Positive (18/19) 5% (1/19) 10 Positive Negative 84% (16/19) 16% (3/19) 15 100% (19/19) 0% (0/19) 0 95% (18/19) 5% (1/19) 10 84% (16/19) 16% (3/19) 15

Appl. Sci. 2018, 8, 2373

15 of 19

Appl. Sci. 2018, 8, x FOR PEER REVIEW

15 of 19

Appl. Sci. 2018, 8, x FOR PEER REVIEW

15 of 19

(a)

(b)

(a)

(b)

(c)

(d)

Figure 14. 14. The The cracks cracks identified identified by by the the proposed proposed method method for for three three cases. cases. (a) Crack map map by by visual visual Figure (a) Crack (c) (d) image; (c) (c) Case Case 1; 1; (d) (d) Case Case 2. 2. inspection; (b) Original image; Figure 14. The cracks identified by the proposed method for three cases. (a) Crack map by visual

For inspection; comparative evaluation, Talab’s original image image as as well. well. (b) Original image; (c) Case 1; (d) Case 2. [51] was applied to the original evaluation, Talab’s method Talab’s Talab’s method method was was developed developed to to detect detect cracks cracks in photographic photographic images, and it has the following For comparative evaluation, Talab’s method [51] was applied to the original image as well. summarized steps. 1. 1. 2. 3. 3. 4. 4.

Talab’s method was developed to detect cracks in photographic images, and it has the following summarized steps. Convert image Convert RGB RGB image to to grayscale. grayscale.

Use the edge detector to UseConvert the Sobel Sobel edge detector to find the edges with masks and get the value of the edge image. 1. RGB image to grayscale. Use Otsu’s thresholding value obtain the image. 2.UseUse the Sobel edge detector find the edges with masks and get the value of the edge image. Otsu’s thresholding valuetoto to obtain the binary binary image. Find the connected areas in the binary image with the controlled area andand putput them to the 3. Use Otsu’s thresholding value to obtain the binary image. Find the connected areas in the binary image with the controlled area them to 4. Find the connected areas in the binary image with the controlled area and put them to the background. the background. background.

First, Talab’s method for a clear crack case with some stains was applied and compared with the First, Talab’s method forfor a clear case withsome somestains stains was applied and compared with the method a clearcrack crack caseresults with was applied and compared with detected the proposedFirst, one,Talab’s as presented in Figure 15. The in Figure 15 show that Talab’s method proposed one,one, as presented ininFigure resultsininFigure Figure show Talab’s method detected proposed as presented Figure15. 15. The The results 1515 show thatthat Talab’s method detected some undesirable areas such as stains, making it difficult to distinguish them from the major and somesome undesirable areas such as stains, making it difficult to distinguish them from the major undesirable areas such as stains, making it difficult to distinguish them from the major and and minor cracks on the concrete surface. minor cracks on the concrete surface. minor cracks on the concrete surface.

(a)

(a)

(b)

(b)

(c)

(c)

Figure 15. The comparison of the proposed method with Talab’smethod methodfor foraaclear clearcrack crackcase. case. (a) (a) Clear Figure 15. The comparison of the proposed method with Talab’s Figure 15.crack The case; comparison ofmethod; the proposed method with Talab’s method for a clear crack case. (a) (b)method; Talab’s (c) Proposed method. crackClear case; (b) Talab’s (c) Proposed method. Clear crack case; (b) Talab’s method; (c) Proposed method.

Appl. Sci. 2018, 8, 2373

16 of 19

Appl. Sci. 2018, 8, x FOR PEER REVIEW

16 of 19

Moreover, the filtering by the classification in the proposed method was more effective in our Moreover, the stains, filteringsediments, by the classification proposed method was the more effectivemethod in our case which had more and joints, in as the shown in Figure 16. Thus, proposed case which had more stains, sediments, andhelp joints, shown in Figure 16. Thus, the proposed method outperforms in detecting cracks with the of as properly selected image processing algorithms. outperforms in detecting cracks with the help of properly selected image processing algorithms. The The K-means clustering significantly helped to remove many parts related to structural joints and K-means clustering significantly helped to remove many parts related to structural joints and sediments while keeping crack objects. Based on the other image processing algorithms, such as sediments while keeping crack objects. Based on thethe other image processing algorithms, such as improved subtraction and morphological operations, crack objects were comparably identified improved subtraction and morphological operations, the crack objects were comparably identified while minimizing the noises. Though the quantitative accuracy was not achieved due to the difficulty in while minimizing the noises. Though the quantitative accuracy was not achieved due to the difficulty assessing the target structure in detail, the visual comparison of Figure 16b,c clarifies the effectiveness in assessing the target structure in detail, the visual comparison of Figure 16b,c clarifies the of the proposed method in detecting the cracks even by the scanning 30 m distant from the test effectiveness of the proposed method in detecting the cracks even by the scanning 30 m distant from structure. After detecting the cracks using the proposed method, quantification of cracks can be done the test structure. After detecting the cracks using the proposed method, quantification of cracks can by employing existing quantification algorithms mostly based on combination of morphological be done by employing existing quantification algorithms mostly based on combination of operations, such as median filtering, morphological opening and closing, skeletonization, edge morphological operations, such as median filtering, morphological opening and closing, detection, etc. [52–55]. Though the lengths and widths of cracks are not quantified in this study, skeletonization, edge detection, etc. [52–55]. Though the lengths and widths of cracks are not the crack widths obtained by visual inspection in Figure 16a are within 0.2 mm. According to the quantified in this study, the crack widths obtained by visual inspection in Figure 16a are within 0.2 Korean structural inspection guide, a crack width less than 0.1 mm leads to the grade of the inspected mm. According to the Korean structural inspection guide, a crack width less than 0.1 mm leads to the structure as “a”, and width between 0.1 mm and 0.3 mm leads to “b” [56]. Thus, the proposed method grade of the inspected structure as “a”, and width between 0.1 mm and 0.3 mm leads to “b” [56]. can be successfully used as a prospective alternative to the conventional visual inspection of large Thus, the proposed method can be successfully used as a prospective alternative to the conventional structures with minimal labor and logistics time. visual inspection of large structures with minimal labor and logistics time.

(a)

(b)

(c)

Figure Thecomparison comparisonofofthe theproposed proposedmethod methodwith withTalab’s Talab’smethod methodand andthe thecrack crack maps maps by visual Figure 16.16. The inspection. (a) Crack pattern map; (b) Talab’s method; (c) Proposed method. inspection. (a) Crack pattern map; (b) Talab’s method; (c) Proposed method.

5. 5. Conclusions Conclusions This study proposed improved This study proposeda aTLS-based TLS-basedcrack crackdetection detectiontechnique techniqueon onconcrete concrete surfaces surfaces using using improved image processing associated with data compression. For the data compression, an octree-based image processing associated with data compression. For the data compression, an octree-based shape shape information model was constructed laser scanning data. The octree shape information information model was constructed usingusing laser scanning data. The octree shape information model model first transforms thedata scanfrom datapixels fromtopixels toinvoxels in 3Dand pixels, and the voxel model is first transforms the scan voxels 3D pixels, the voxel model is converted converted into an octree data structure to generate an octree-based shape information model. For crack into an octree data structure to generate an octree-based shape information model. For crack detection detection using TLS data, a combination image processing algorithms is proposed to minimize using the TLS the data, a combination of imageofprocessing algorithms is proposed to minimize the false the false recognition diverse environments. operation environments. The proposed can recognition of cracksofincracks diverseinoperation The proposed algorithm canalgorithm automatically automatically identify the existence of cracks against stains, sediment, and structural joints with identify the existence of cracks against stains, sediment, and structural joints with the help of K-means the help of K-means clustering. Furthermore, concrete crack pattern is visualized through other clustering. Furthermore, the concrete crack the pattern is visualized through other image processing image processing algorithms as improved subtraction withOstu’s median filtering, method, algorithms such as improvedsuch subtraction with median filtering, method, andOstu’s morphological operations. and morphological operations. validation testwas wascarried carriedout outon onaaspillway spillwaywall wallin inaaconcrete concretedam dam using using aa Leica Leica ScanStation AA validation test C10 model located distancefrom fromthe thewall, wall,and andthe theresults resultsare aresummarized summarizedas asfollows: follows: C10 model located 3030mmdistance (1) The original scan data 19.56 MB was compressed using the octree data structure up to the 31.6%, (1) The original scan data 19.56 MB was compressed using the octree data structure up to the 31.6%, 50.3%, and 80.2% corresponding to the data size of 13.36 MB, 9.7 MB, and 3.86 MB, respectively. 50.3%, and 80.2% corresponding to the data size of 13.36 MB, 9.7 MB, and 3.86 MB, respectively. (2) Most major cracks as well as some minor cracks due to drying shrinkage of concrete could be identified successfully even in the 50% compressed image of the proposed method.

Appl. Sci. 2018, 8, 2373

(2) (3) (4)

17 of 19

Most major cracks as well as some minor cracks due to drying shrinkage of concrete could be identified successfully even in the 50% compressed image of the proposed method. The identified cracks by the proposed method had good agreement with the cracks obtained by visual inspection. The proposed method could minimize the false recognition of cracks on the structural joints and sediments with the help of K-means clustering, compared to Talab’s method.

For further research, the proposed algorithm needs to be modified and improved with the better accuracy and generality for various cases and other optimization methods can be applied to determine the parameters required in the image processing. The imaging processing associated with the TLS measurement is expected to have an important role in the visualization of cracks and the acquisition of the exact crack information. Author Contributions: S.C., S.P. and T.O. conceived and designed the experiments; G.C. and T.O. performed the experiments; S.C. and G.C. analyzed the data; S.P. contributed device/analysis tools; S.C. and T.O. wrote the paper. Funding: This research was funded by the Incheon National University. Acknowledgments: This work was supported by the Incheon National University Research Grant in 2015. Conflicts of Interest: The authors declare no conflict of interest

References 1. 2.

3. 4. 5. 6. 7.

8. 9. 10.

11.

12. 13.

Olsen, M.J.; Kuester, F.; Chang, B.J.; Hutchinson, T.C. Terrestrial laser scanning-based structural damage assessment. J. Comput. Civ. Eng. 2009, 24, 264–272. [CrossRef] Anil, E.B.; Akinci, B.; Garrett, J.H.; Kurc, O. Characterization of laser scanners for detecting cracks for post-earthquake damage inspection. In Proceedings of the International Symposium on Automation and Robotics in Construction and Mining (ISARC), Montreal, QC, Canada, 11–15 August 2013. Giri, P.; Kharkovsky, S. Detection of surface crack in concrete using measurement technique with laser displacement sensor. IEEE Trans. Instrum. Meas. 2016, 65, 1951–1953. [CrossRef] Laefer, D.F.; Truong-Hong, L.; Carr, H.; Singh, M. Crack detection limits in unit based masonry with terrestrial laser scanning. NDT & E Int. 2014, 62, 66–76. Li, T.; Almond, D.P.; Rees, D.A.S. Crack imaging by scanning pulsed laser spot thermography. NDT & E Int. 2011, 44, 216–225. Kim, M.K.; Sohn, H.; Chang, C.C. Localization and quantification of concrete spalling defects using terrestrial laser scanning. J. Comput. Civ. Eng. 2014, 29, 04014086. [CrossRef] Valença, J.; Puente, I.; Júlio, E.; González-Jorge, H.; Arias-Sánchez, P. Assessment of cracks on concrete bridges using image processing supported by laser scanning survey. Constr. Build. Mater. 2017, 146, 668–678. [CrossRef] Xu, X.; Yang, H.; Neumann, I. Concrete crack measurement and analysis based on terrestrial laser scanning technology. Sens. Transducers 2015, 186, 168–172. Tsakiri, M.; Lichti, D.; Pfeifer, N. Terrestrial Laser Scanning for Deformation Monitoring. In Proceedings of the 3rd IAG/12th FIG Symposium, Baden, Austria, 22–24 May 2006; pp. 22–24. Van Gosliga, R.; Lindenbergh, R.; Pfeifer, N. Deformation analysis of a bored tunnel by means of terrestrial laser scanning. In Proceedings of the International Archives of the Photogrammetry, Dresden, Germany, 25–27 September 2006; pp. 25–27. Chang, Y.; LAI, S.; Peng, S.; Chang, K. Non-Contact Bridge Deflection Measurement: Application of Laser Technology. In Proceedings of the Integrating Generations FIG Working Week, Stockholm, Sweden, 14–19 June 2008; pp. 14–19. Koch, K. Three-dimensional NURBS surface estimated by lofting method. J. Adv. Manuf. Technol. 2010, 49, 1059–1068. [CrossRef] Jafari, B.; Khaloo, A.; Lattanzi, D. Deformation tracking in 3D point clouds via statistical sampling of direct cloud-to-cloud distances. J. Nondestruct. Eval. 2017, 36, 65. [CrossRef]

Appl. Sci. 2018, 8, 2373

14. 15. 16. 17. 18.

19. 20.

21. 22.

23. 24.

25. 26. 27. 28. 29. 30. 31.

32. 33. 34. 35. 36. 37.

18 of 19

Khaloo, A.; Lattanzi, D. Automatic Detection of Structural Deficiencies Using 4D Hue-Assisted Analysis of Color Point Clouds. Conf. Proc. Soc. Exp. Mech. 2019, 2, 197–205. Dai, K.; Li, A.; Zhang, H.; Chen, S.E.; Pan, Y. Surface damage quantification of the post-earthquake building based on terrestrial laser scan data. Struct. Control Health 2018, 25, e2210. [CrossRef] Dawood, T.; Zhu, Z.; Zayed, T. Computer vision–based model for moisture marks detection and recognition in subway networks. J. Comput. Civ. Eng. 2017, 32, 04017079. [CrossRef] Law, D.W.; Silcock, D.; Holden, L. Terrestrial laser scanner assessment of deteriorating concrete structures. Struct. Control Health 2018, 25, e2156. [CrossRef] Song, M.; Yousefianmoghadam, S.; Mohammadi, M.E.; Moaveni, B.; Stavridis, A.; Wood, R.L. An application of finite element model updating for damage assessment of a two-story reinforced concrete building and comparison with lidar. Struct. Health Monit. 2018, 17, 1129–1150. [CrossRef] Lõhmus, H.; Ellmann, A.; Märdla, S.; Idnurm, S. Terrestrial laser scanning for the monitoring of bridge load tests–two case studies. Surv. Rev. 2018, 50, 270–284. [CrossRef] Baeza, F.J.; Ivorra, S.; Bru, D.; Varona, F.B. Structural health monitoring systems for smart heritage and infrastructures in Spain. In Mechatronics for Cultural Heritage and Civil Engineering; Springer: New York, NY, USA, 2018; pp. 271–294. Laefer, D.F.; Gannon, J.; Deely, E. Reliability of crack detection methods for baseline condition assessments. J. Infrastruct. Syst. 2010, 16, 129–137. [CrossRef] Koch, C.; Georgieva, K.; Kasireddy, V.; Akinci, B.; Fieguth, P. A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure. Adv. Eng. Inform. 2015, 29, 196–210. [CrossRef] Chae, M.; Abraham, D. Neuro-fuzzy approaches for sanitary sewer pipeline condition assessment. J. Comput. Civ. Eng. 2001, 15, 4. [CrossRef] Khanfar, A.; Abu-Khousa, M.; Qaddoumi, N. Microwave near-field nondestructive detection and characterization of disbonds in concrete structures using fuzzy logic techniques. Compos. Struct. 2003, 62, 335–339. [CrossRef] Adbel-Qader, I.; Abudayyeh, O.; Kelly, M. Analysis of edge-detection techniques for crack identification in bridges. J. Comput. Civ. Eng. 2003, 17, 255. [CrossRef] Guo, W.; Soibelman, L.; Garrett, J., Jr. Automated defect detection for sewer pipeline inspection and condition assessment. Autom. Constr. 2009, 18, 587–596. [CrossRef] Ng, H. Automatic thresholding for defect detection. Pattern Recogn. Lett. 2006, 27, 1644–1649. [CrossRef] Sinha, S.; Fieguth, P. Neuro-fuzzy network for the classification of buried pipe defects. Autom. Constr. 2006, 15, 73–83. [CrossRef] Moon, H.; Kim, J. Intelligent crack detecting algorithm on the concrete crack image using neural network. In Proceedings of the 28th ISARC, Seoul, Korea, 29 June–2 July 2011; pp. 1461–1467. Kawamura, K.; Miyamoto, A.; Nakamura, H.; Sato, R. Proposal of a crack pattern extraction method from digital images using an interactive genetic algorithm. JSCE 2003, 742, 115–131. Goswami, P.; Zhang, Y.; Pajarola, R.; Gobbetti, E. High Quality Interactive Rendering of Massive Point Models using Multi-way kd-Trees. In Proceedings of the 18th Pacific Conference on Computer Graphics and Applications 2010, Hangzhou, China, 25–27 September 2010; pp. 93–100. Saxena, M.; Finnigan, P.M.; Graichen, C.M.; Hathaway, A.F.; Parthasarathy, V.N. Octree-based automatic mesh generation for non-manifold domains. Eng. Comput. 1995, 11, 1–14. [CrossRef] Woo, H.; Kang, E.; Wang, S.; Lee, K.H. A new segmentation method for point cloud data. Int. J. Mach Tools Manuf. 2002, 42, 167–178. [CrossRef] Wang, M.; Tseng, Y.H. Lidar data segmentation and classification based on octree structure. In Proceedings of the XXth ISPRS Congress, Istanbul, Turkey, 12–23 July 2004. Schnabel, R.; Wahl, R.; Klein, R. Efficient RANSAC for point-cloud shape detection. Comput. Gr. Forum 2007, 26, 214–226. [CrossRef] Cho, H.; Cho, W.; Park, J.; Song, N. 3D building modeling using aerial LiDAR data. Korean J. Remote Sens. 2008, 24, 141–152. Marechal, L. Advances in octree-based all hexahedral mesh generation: Handling sharp features. In Proceedings of the 18th International Meshing Roundtable, Salt Lake City, UT, USA, 25–28 October 2009; pp. 65–84.

Appl. Sci. 2018, 8, 2373

38. 39. 40. 41. 42. 43. 44. 45.

46. 47. 48. 49. 50. 51. 52. 53. 54. 55. 56.

19 of 19

Han, S.; Lee, S.; Kim, S.P.; Kim, C.; Heo, J.; Lee, H.A. Comparison of 3D R-tree and octree to index large point clouds from a 3D terrestrial laser scanner. Korean J. Geomat. 2011, 29, 531–537. [CrossRef] Wang, J.; Lindenbergh, R.; Menenti, M. SigVox—A 3D feature matching algorithm for automatic street object recognition in mobile laser scanning point clouds. ISPRS J Photogramm. 2017, 128, 111–129. [CrossRef] Richard, O.D.; Peter, E.H. Pattern Classification and Scene Analysis; John Wiley and Sons: New York, NY, USA, 1973. Fujita, Y.; Hamamoto, Y. A robust automatic crack detection method from noisy concrete surfaces. Mach. Vis. Appl. 2010, 22, 1–10. [CrossRef] Otsu, N. A threshold selection method from gray-level histograms. IEEE. Trans. Syst. Man Cybern. C 1979, 9, 62–66. [CrossRef] Fukuchi, T.; Shiina, T. Industrial Applications of Laser Remote Sensing; Bentham Science Publishers: Oak Par, IL, USA, 2012. Hocken, R.J.; Pereira, P.H. Coordinate Measuring Machines and Systems; CRC Press: Boca Raton, FL, USA, 2012. Wu, B.; Yu, B.; Yue, W.; Shu, S.; Tan, W.; Hu, C.; Huang, Y.; Wu, J.; Liu, H. A Voxel-Based Method for Automated Identification and Morphological Parameters Estimation of Individual Street Trees from Mobile Laser Scanning Data. Remote. Sens 2013, 5, 584–611. [CrossRef] Kim, Y.J.; Kim, J.J. Compression of Normal Vectors using Octree Encoding. Soc. CAD/CAM Eng. 2007, 12, 109–117. Ryu, J.H. Efficient Octree Encoding for Real Time Transmission of 3D Geometry Data Through Internet. Soc. CAD/CAM Eng. 2002, 7, 262–268. Tamminen, M. Efficient Octree Conversion by Connectivity Labeling. In Proceedings of the 11th Annual Conference on Computer Graphics and Interactive Techniques, Minneapolis, MN, USA, 23–27 July 1984. Weiss, N.A. A Course in Probability; Addison–Wesley: Singapore, 2005; pp. 385–386. Hornung, A.; Wurm, K.M.; Bennewitz, M.; Stachniss, C.; Burgard, W. OctoMap: An efficient probabilistic 3D mapping framework based on octrees. Auton. Robots 2013, 34, 189–206. [CrossRef] Ahmed, M.T.; Huanga, Z.; Fan, X.; Liu, H.M. Detection crack in image using Otsu method and multiple filtering in image processing techniques. Optik 2015, 127, 1–4. [CrossRef] Liu, C.; Tang, C.S.; Shi, B.; Suo, W.B. Automatic quantification of crack patterns by image processing. Comput. Geosci. 2013, 57, 77–80. [CrossRef] Adhikari, R.S.; Moselhi, O.; Bagchi, A. Image-based retrieval of concrete crack properties for bridge inspection. Autom. Constr. 2014, 39, 180–194. [CrossRef] Liu, Y.; Cho, S.; Spencer, B.F., Jr.; Fan, J. Automated assessment of cracks on concrete surfaces using adaptive digital image processing. Smart Struct. Syst. 2014, 14, 719–741. [CrossRef] Liu, Y.F.; Cho, S.; Spencer, B.F., Jr.; Fan, J.S. Concrete crack assessment using digital image processing and 3D scene reconstruction. J. Comput. Civ. Eng. 2014, 30, 04014124. [CrossRef] Korea Ministry of Land, Infrastructure, and Transport (MLIT) and Korea. Guideline for Regular and Detailed Inspection; Korea Infrastructure Safety and Technology Corporation (KISTeC): Jinju, Korea, 2017; pp. 6–23. © 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).