PAVEMENT PATHOLOGIES CLASSIFICATION USING GRAPH-BASED FEATURES. Kelwin Fernandes, Lucian Ciobanu. INESC TEC. Porto, Portugal.
PAVEMENT PATHOLOGIES CLASSIFICATION USING GRAPH-BASED FEATURES Kelwin Fernandes, Lucian Ciobanu INESC TEC Porto, Portugal ABSTRACT Pavement cracks involve important information to measure road quality. Crack classification is a challenging problem given the diversity of possible cracks, therefore, it is needed to retrieve good features in order to facilitate the learning of predictive models with as few samples as possible. In this paper, we propose a graph-based set of features to efficiently describe cracks. These features proved to have high degree of expressiveness and robustness when used for crack classification. We show that the proposed features succeed in the assessment of 525 images with different kinds of cracks. We proved the robustness of the approach applying different levels of noise to the images and evaluating the classification accuracy.
Alligator cracking
Index Terms— Crack classification, Crack segmentation, Graph-Based features, Minimum Spanning Trees, Support Vector Machines
Longitudinal cracking Fig. 1. Images with cracks of the same class
1. INTRODUCTION In conjunction with other assets like traffic signs and road marks, pavement pathologies involve important information to measure road quality [1][2]. Moreover, in order to prioritize areas and to estimate costs, it is needed to perform an assessment of the road that includes not only the presence of pathologies but also the kind of crack and the degree of degradation. Given the huge extension of the road network of a country, it is time consuming and expensive task to evaluate this conditions manually [1]. Therefore, it is needed to provide an automatic method capable to trustfully detect and classify this kind of deteriorations. In addition to the common problems related to image processing, this task has some domain-specific problems [3]. Some difficulties related to the background information are: the diversity of materials involved in roads, patches, preprocessed deteriorations and the presence of other elements like the trace of tires, trash, drains, among others. Also, cracks have a high intra-class variability (Fig. 1). This paper aims to solve this last issue by building a model that extracts highlevel features from a graph representation of the pathologies, in order to efficiently classify them with a few training samples. These features are intended to represent the topology
978-1-4799-5751-4/14/$31.00 ©2014 IEEE
793
of the crack instead of the specific spatial distribution of a given sample. Despite our focus is on crack classification, we also briefly describe the segmentation method used, for completeness. The rest of the paper is presented as follows. Section 2 overviews previous work on detection and classification of cracks. The proposed system is exposed in the section 3 and its evaluation in the section 4. Finally, some conclusions and future work are presented in section 5. 2. PREVIOUS WORK In the literature, several approaches have been driven for segmenting [1][2][4][5][6] and classifying [1][3][6][7] cracks in pavement images. Most of the work on cracks processing has focused on the segmentation task, encompassing different techniques from thresholding methods [1][4][5], unsupervised [3], supervised learning techniques [6] and filtering [2]. The algorithms that use thresholding based on an analysis of the histogram [4][5] assume that pixels belonging to crack and non-crack areas can be separated using global statistics.
ICIP 2014
Although these methods are efficient, they fail when used in diverse environments, with changing environment conditions [8]. Moussa and Hussain used thresholding for identifying reasonably good crack and background candidates and then used a graph-based approach to detect the connections between these points of interest [1]. Aiming to reach a more robust solution, Younes et. al [3] and Oliveira and Correia [6] propose learning based methods. Younes et. al. presented a clustering based algorithm for crack segmentation [3]. Although this method overcomes the problem of fixing a threshold, it fails when the crack intensity is non-homogeneous or when the rate between crack and non-crack areas is highly unbalanced. In other approach, Oliveira and Correia [6] introduced a supervised approach using region local statistics (mean and standard deviation) as features. Zou et. al. [2] explored a filtering-based method using local intensity-differences between a pixel and its neighbours. Then, tensor voting is applied to reduce false positives. The graph reconstruction is made using a Minimum Spanning Tree algorithm over a graph that models the possible connections between candidates. Despite this reconstruction can be suitable for crack detection, it prunes valuable information of the crack topology that might be useful to classify certain kind of pathologies. In the area of crack classification, most work is based on the selection of low-level features distant from human perception. Younes et. al [3] use Principal Component Analysis over the segmented images and K-Nearest Neighbors. Despite they report high accuracy (80% of correctly classified images), they have a small test set of 40 images distributed among the 8 classes of their catalogue. Different approaches try to classify cracks using features based on classic statistics that can successfully represent longitudinal and transversal cracks but that are not informative enough to represent more complex cracks. Two such solutions were proposed by Moussa and Hussain [1] and by Oliveira and Correia [6]. Former uses features related to the distance between local peaks by row and column. For training, they use a Support Vector Machine with a RBF kernel [1]. The latter includes as features the mean and standard deviation by row and column [6]. Li et. al [7] propose a set of features with a higher level of information than the previously mentioned works. They use as features relations between area and direction of the sections generated by a Delaunay Triangulation of the binarized crack images. Although this approach reports results with high precision (98%), it uses a complex model to predict the class of the crack.
Round drain
Rectangular drain
Road-mark
Trace of tires
Fig. 2. Common non-crack elements found in pavement images in the context of crack classification. 3.1. Segmentation Road networks are complex environments where non-crack elements can be easily found (Fig. 2). A frequently unconsidered step in crack detection is the removal of these elements. The proposed system uses Hough transform to remove circular and rectangular drains and a filter based on local intensities differences and convex hull to remove areas with road-marks. Then, we apply the initial steps of the algorithm presented by Zou et. al. [2]. As a preprocessing step for the detection of crack pixels, the illumination compensation proposed by them is performed to overcome problems derived by the presence of shadows and uneven illumination. Then, crack pixel candidates are selected using filtering based on the histogram of local intensity-difference. At this point, we change the segmentation pipeline. Zou et. al. connect crack candidates using Minimum Spanning Trees over a crack probability map generated by tensor voting. This linking algorithm removes loops from the original crack, hiding valuable connectivity information that might discriminate some kind of pathologies. Instead of that, we smooth the candidates using a big Gaussian kernel and enhance local maxima, where most of the crack pixels are located. Then, cracks are segmented flooding the image through the local maxima. The result of these steps can be appreciated in Fig. 3.
3. SYSTEM DESCRIPTION As many object recognition systems, the proposed one consists of three stages: segmentation, feature extraction and classification. This section is organized to cover these steps
794
3.2. Feature Extraction The novelty of the proposed system relies on the extraction of high-level features. As shown in the literature review, several
ICIP 2014
Input image
Table 1. Distribution of the deterioration classes Type of Pathology Rate (%) Longitudinal cracking 15.23 Transversal cracking 12.00 Mixed cracking 36.19 Alligator cracking 14.47 Healthy pavement 23.42
Illumination compensation
Candidates to crack pixels
Enhancement of local maxima
Fig. 3. Crack segmentation steps
Input image
Segmented image
Output graph
Fig. 4. Graph induced by a crack image
approaches use graph-based methods in the segmentation task [1][2] but this information is generally unseen when studying the classification of cracks. We strongly believe that cracks can be described by their connectivity as it has been validated in this paper. The use of expressive features aims to diminish the number of training samples and the complexity of the model used to predict the new samples. We create an undirected planar graph G = (V, E) with nodes corresponding to branching and terminal points in the skeleton of the pathology. For this purpose, we use the thinning algorithm proposed by Zhang and Suen [9]. This algorithm preserves pixel connectivity and end points. The resulting skeleton has also unitary thickness, therefore it is computationally inexpensive to retrieve the nodes from the output graph. Edges are defined by the direct connection of the interest nodes. A direct connection is defined as a path through the skeleton that does not includes any other vertex. Edges are built using a Breadth First Search algorithm that allows the detection of them in linear time in increasing order. Small trivial components are removed given that they represent small noisy and unlinked candidates. An example of the obtained graph is shown in Fig. 4. The set of considered features is:
795
1. Number of nodes. 2. Number of connected components. 3. Global density [10]. 4. Output degree by node. 5. Densities of each component. 6. Summation of the edge’s lengths for each component. 7. Verticality of each component. 8. Difference between the total weight of the original graph and its Minimum Spanning Tree (MST). The features that consist of more than one value (4-7) are synthesised using basic statistics: minimum, maximum, mean and standard deviation of the values. The last feature has an important role on the detection of alligator cracks. This kind of crack is humanly identifiable by a high presence of loops in the structure of the crack. MST computation would have a computational cost of O(|E| · log |E|) but given that we retrieved edges in ascending order, the initial edge sorting of Kruskal’s algorithm [11] for finding MST can be skipped. Moreover, using an implementation with a Disjoint-Set data structure with union by rank and path compression, this feature can be computed in linear time - O(|E|) - for feasible cases [12]. The worst case of the algorithm is when every pixel in the image is a crack pixel, in which case the whole set of features can be computed in linear time. 3.3. Classification Each feature is normalized and fed to a Support Vector Machine (SVM) [13] with linear kernel. This model is used to prove the expressiveness of these features but better results can be achieved using more complex models. 4. EXPERIMENTS AND RESULTS To evaluate the performance of the proposed system, we are using a set of 206 images provided by Zou et. al. [2] plus 319 images acquired by us from diverse roads. Our dataset consists of images with five types of pathologies: longitudinal cracks, transversal cracks, mixed cracks, alligator cracks and healthy pavement. Every image have a size of 800 × 600. The images incorporated by us are public under request. The proposed solution was implemented in C++ and tested with Ubuntu 12.04 equipped with 3.00GHz CPU and 3.7GiB of RAM. Table 1 shows the dataset distribution. Some images have more than one assigned class. We trained independent Support Vector Machines for every class. For evaluation purposes, we are using 5-fold cross
ICIP 2014
Table 2. Crack classification performance on 525 images Type of Pathology Precision Recall F-measure Longitudinal 0.92 0.80 0.85 Transversal 0.88 0.82 0.85 Mixed 0.72 0.77 0.74 Alligator 0.86 0.84 0.85 Healthy 1.00 1.00 1.00 Total 0.86 0.85 0.85
Input image Fig. 5. Distribution of the cracks. Left: MST vs. Mean degree for alligator cracking. Right: MST vs. Mean angle for transversal cracking. validation. Results of these experiments are shown in table 2. The system is able to build the graph of each image and to extract the features in 34 milliseconds. We also tested the robustness of the set of features removing random pixels of the skeleton. We applied different degrees of degradations from 0% to 100% of crack pixels removed. Every degree of noise is tested several times (10) with different random cuts. Fig. 6 shows the variation of the classification performance for each class. We can see that with 40% of pixels removed from the final skeleton the F-measure is still over 0.75 for every evaluated class. It is important to note that every removed pixel directly disconnects two nodes in the derived graph because our skeleton has unitary thickness. This experiment highlights the robustness of the proposed set of features, which even with a simple learning model is capable to retain enough information to discriminate between these classes. Fig. 5 shows some class distributions related to the MST feature. 5. CONCLUSIONS In this paper we evaluated a novel approach for classifying cracks from pavement images. Our method is based on the extraction of high-level features and can be used by any crack segmentation algorithm. Our set of features is based on graph properties that succinctly describe the topology of the crack. In the literature, cracks are described using information related to their position and spatial distribution in the image. Using this approach, we can describe cracks based on their connectivity which is nearest to the human perception to determinate the kind of pathology.
796
Fig. 6. F-measure evaluation training with noisy data. Noise is introduced removing points in the segmented skeleton. We acquired 319 images (plus 206 provided by Zou et. al.) with different kinds of cracks to evaluate the proposed method. We show that our methodology successfully classifies cracks with a simple predictive model. The feature extraction procedure is also fast so that it can be applied to environments with real-time constraints. We plan to conduct work on the translation of this model to video sequences. Additionally, we aim to validate our approach with more types of pathologies. We strongly believe that graph-based features can successfully describe other pathologies like potholes and reparations. This technique may also be applied to describe other road assets like road-marks. 6. ACKNOWLEDGMENTS This work is financed by the ERDF - European Regional Development Fund through the COMPETE Programme (operational programme for competitiveness) and by National Funds through the FCT - Fundac¸a˜ o para a Ciˆencia e a Tecnologia (Portuguese Foundation for Science and Technology) within project FCOMP-01-0124-FEDER-037281 . 7. REFERENCES [1] Ghada Moussa and H Hussain, “A new technique for automatic detection and parameters estimation of pavement crack,” in Proceedings of the 4th International Multi-Conference on Engineering and Technological Innovation, 2011. [2] Qin Zou, Yu Cao, Qingquan Li, Qingzhou Mao, and Song Wang, “Cracktree: Automatic crack detection
ICIP 2014
from pavement images,” Pattern Recognition Letters, vol. 33, no. 3, pp. 227–238, 2012. [3] Guellouma Younes, Cherroun Hadda, N Attia, and Z Djelloul, “Supervised learning and automatic recognition of asphalt pavement deteriorations,” in Computer Systems and Applications, 2009. AICCSA 2009. IEEE/ACS International Conference on. IEEE, 2009, pp. 205–210.
[13] Corinna Cortes and Vladimir Vapnik, “Support-vector networks,” Machine learning, vol. 20, no. 3, pp. 273– 297, 1995.
[4] Hyoungkwan Kim, Hamid Soleymani, Seung Heon Han, and Hana Nam, “Evaluation of asphalt pavement crack sealing performance using image processing technique,” in International Symposium on Automation and Robotics in Construction (ISARC2006), 2006, pp. 341– 345. [5] Miguel Gavil´an, David Balcones, Oscar Marcos, David F Llorca, Miguel A Sotelo, Ignacio Parra, Manuel Oca˜na, Pedro Aliseda, Pedro Yarza, and Alejandro Am´ırola, “Adaptive road crack detection system by pavement classification,” Sensors, vol. 11, no. 10, pp. 9628–9657, 2011. [6] Henrique Oliveira and Paulo Lobato Correia, “Supervised crack detection and classification in images of road pavement flexible surfaces,” Recent Advances in Signal Processing, pp. 159–184, 2009. [7] Li Qingquan, Zou Qin, and Liu Xianglong, “Pavement crack classification via spatial distribution features,” EURASIP Journal on Advances in Signal Processing, vol. 2011, 2011. [8] Sylvie Chambon and Jean-Marc Moliard, “Automatic road pavement assessment with image processing: Review and comparison,” International Journal of Geophysics, vol. 2011, 2011. [9] TY Zhang and Ching Y. Suen, “A fast parallel algorithm for thinning digital patterns,” Communications of the ACM, vol. 27, no. 3, pp. 236–239, 1984. [10] Thomas F Coleman and Jorge J Mor´e, “Estimation of sparse Jacobian matrices and graph coloring blems,” SIAM journal on Numerical Analysis, vol. 20, no. 1, pp. 187–209, 1983. [11] Joseph B Kruskal, “On the shortest spanning subtree of a graph and the traveling salesman problem,” Proceedings of the American Mathematical society, vol. 7, no. 1, pp. 48–50, 1956. [12] Michael Fredman and Michael Saks, “The cell probe complexity of dynamic data structures,” in Proceedings of the twenty-first annual ACM symposium on Theory of computing. ACM, 1989, pp. 345–354.
797
ICIP 2014
