Moving Object Segmentation Using Various Features from Aerial Images

RESEARCH ARTICLE

Adv. Sci. Lett. 24, 961–965, 2018

Copyright © 2018 American Scientific Publishers All rights reserved Printed in the United States of America

Advanced Science Letters Vol. 24, 961–965, 2018

Moving Object Segmentation Using Various Features from Aerial Images: A Review A.F.M. Saifuddin Saif1, Zainal Rasyid Mahayuddin2, 1

Faculty of Science and Information Technology American International University - Bangladesh 2 Faculty of Information Science and Technology, 2 University Kebangsaan Malaysia, Selangor, Malaysia 1

Extraction of moving object is elusively unsolved issue in computer vision and pattern recognition research field. Differentiate of the achieved from two consecutive frames into no crossed frame such no sequential frame is similar is referred as extraction of moving object. Three perspective needs to be addressed i.e. types of features used during detection, segmentation methods and overall frameworks. This research illustrates critical review for segmentation of moving object which is depicted based on four aspects, i.e. types of features, specified methods, frameworks, datasets and parameters to be validated. In addition, this research states the justification to develop new segmentation method based on the demonstrated critical and comprehensive review. The overall reviews performed in this research, have been comprehensively investigated in the previous research which is expected to significantly contribute in three research field, i.e. computer vision and pattern recognition, artificial intelligence and expert systems and image processing. Keywords: Segmentation, Features, Aerial Images, Computer Vision, Pattern Recognition, Artificial Intelligence, Image Processing.

1. INTRODUCTION

Blur aerial images cause huge challenges to segment moving object using aerial images. In addition, thin edges and challenges of noise reduction make the task more difficult for the same purpose. Canny, Sobel and Prewitt segmentation methods do not have the ability to hold sharp edges and functionalities to increase or decrease number of edges during segmentation for optimum detection performance. A comprehensive review for the existing segmentation for moving object detection is demonstrated like segmentation based on various features, existing methods, frameworks, datasets for validation etc. Later, a justification is illustrated to necessitate new segmentation method based on the critical review on the existing research. * Email Address: [email protected] Separation of frames achieved from frame differential approach into a set of non matching unique matched area such that any two frequent ones are not similar is referred to as segmentation of moving object1-4. Detection task 1

Adv. Sci. Lett. Vol. 24, No. 2, 2018

more crucial due to blur aerial images and altitudes issues during detection5-7. Three types of features are available like color, corner and edge feature. Besides, three methods were previously implemented like optical flow, background subtraction, frame difference where issues were raised like static object detection using moving and static camera, blob size and computation time etc. A recent progress of moving object segmentation using various features, methods, datasets and validation is presented for moving object detection. Initially, a comprehensive review is presented for the usage for various features. Later, illustration based on previous segmentation methods is performed in the next section. After that, comprehensive review based on existing frameworks for moving object detection is shown in the subsequent section following by validation based on various datasets. Finally, justification for the need of new segmentation method is illustrated after validation section. 2. VARIOUS FEATURES

Detection performance in terms with moving object detection in computer vision research field mainly depends on appropriate selection of features which is 1936-6612/2011/4/400/008

doi:10.1166/asl.2018.10667

RESEARCH ARTICLE

Adv. Sci. Lett. 4, 400–407, 2016 considered as key factor for optimum detection performance to evaluate performance with detection rate and computation time. Besides, collection of aerial image from various altitudes levels affects motion pixel selection process. Three types of features previously used for moving object detection, i.e. corner 8-11, color12-14 and edge1-3 feature shown in figure 1.

Color Features

Corner Edge

Fig.1. Available for moving object detection in the existing research 2.1 Color Feature

Hsu-Yung et al. (2012) extended classification based on pixels by keeping relation using color features in neighboring pixels in a specific state in image. However, they were not able to differentiate adjacent color feature but non similar several object in the similar scene. Zezhong et al. (2012) recognized candidate key points of object pixels where road vector map was used like a training datasets for detection of moving object. However, they mostly depend on structural shape of the object. Long et al. (2012) implemented color feature specially for complicated background for urban environment. Regarding constraint of considering gray scale input images, their research output cannot be considered as reliable solution13. 2.2 Corner Feature Kembhavi et al. (2011) extracted neighboring pixel using corner features where two selection methodology is used to decrease execution duration or selection of features. Gleason et al. (2011) implemented corner features to come across challenges of the respective research consistently dealing with 3D image orientation although they ignored object background during detection. Wang et al. (2011) used corner feature by considering context-aware saliency detection algorithm in corporate with surrounded environment to extract points in order to attract concentration in computer vision. For shape resolution and variant of appearance of objects are the only means of optimum detection performance in their research. 2.3 Edge Feature

Sheng (2011) considered edge features by adopting a unique feature segmentation framework using shadow

with the combination of rotational invariant shape matching of edge features by implementing shape context descriptor segmented from object edge. However, sheng could not identify object for clocked shadows due to dependency on lightening condition. Oreifej et al. (2010) experimented edge feature for low quality aerial images and various pose across the set which is due to changes in the object location and articulation. Their research did not provide enough evidence of optimum detection performance due to poor performance in high frame rate videos since their proposed method obeys the assumption that moving object location in the subsequent frame will be close to its location in the active frame. Bhuvaneswari and Abdul Rauf (2009) clustered single points achieved from motion estimation although their research did not support expected results for the complication of shortening environment, real time change of background and inconspicuous features of objects. Mofaddel and Abd-Elhafiez (2011) considered motion compensation and analysis using edge features. However, they dealt with a lot of parameters during their overall detection process. 3. VARIOUS METHODS

Categorization for various methods for moving object detection can be depicted as follows, 1. Optical Flow 2. Background Subtraction 3. Frame Difference The following issues are the most concerned topics for all of these methods, o Compatible to detect only moving object but not static object. o Depend on blob size, Increase in size of blob reduce detection rate. o Depend on number of motion block. If number of motion block increase then detection rate decrease. 3.1 Optical Flow

Subsequent change of moving object’s location or deformation between adjacent frames is considered as the definition for optical flow which demonstrates the connection between the change of image gray and two dimensional velocity field and illustrates the direction and speed of moving pixels. Although this method provides high detection accuracy with high computational complexity13, cannot obtain accurate outline of moving object in a given scene. Meuel et al. (2013) used global motion estimation and global motion compensation wherein a cluster filter eliminates error in the optical flow by assuming smooth vector field as the global motion model. Wang et al. (2012) computed optical flow among every adjacent frames first to get the motion information for each pixel where they defined motion model named 2

RESEARCH ARTICLE “pixel motion process” which means the motion changes (optical flow value changes) of a particular pixel over time and transfer Gaussian mixture model framework used for modelling background in the stationary scene to model the background model. However, they could not detect object for various direction of moving object. 3.2 Background Subtraction

Basic idea of background subtraction is to classify the pixels as background or foreground by thresholding the difference between background image and current image23-27. Number of challenges has been raised recently to improve background subtraction method like, 1. Background subtraction must be robust against changes in illumination. 2. Background subtraction should avoid detection of non-stationary back ground object such as moving leaves, rain, snow and shadows cast by moving object. 3. Internal background model should react quickly to changes in background such as starting and stopping of vehicles. Steps of processing for moving objects are video processing, frame display, background subtraction and detection. Dhananjaya et al. (2015) introduced background subtraction algorithm augmented with morphological processing. At first, numbers of frames are extracted to perform histogram analysis on the frames to extract background and then each different frame from the background is subtracted. Liang et al. (2013) used background subtraction to generate the candidate for Temporal Context (TC) based on the candidate that have been classified as positive by Histogram of Oriented Gradient (HOG) with Multiple Kernel Learning (MKL). 3.3 Frame Difference

Registration of two consecutive frames followed by the operation of finding different frame is the definition of difference method. This method has the ability to remove complex background where each frame is updated and movement or motion in the frame is checked and gain the ability to ignore background motion and identify foreground object. Bhuvaneswari and Abdul Rauf (2009) used frame difference method for data association between people detection in different frames is highly challenging and ambiguous which needs to be discriminative enough in order for data association across long periods of partial and full occlusions. Classifier is one of most effective factor in computer vision research field especially for complexity. Classifier used by Ga̧szczak et al. (2011) was not able to separate the features of rigid and nonrigid object. 3

Adv. Sci. Lett. Vol. 24, No. 2, 2018

Adv. Sci. Lett. 24, 961–965, 2018 4. FRAMEWORKS Effective framework is an essential issue due to reduce computation cost and increment of high detection rate which is considered as the main liability to increase or decrease detection rate, false alarm rate and computation time. Ibrahim et al. (2010) used MODAT framework which stands for Moving Object Detection and Tracking Framework and gathered useful information from aerial images by mapping visited areas to detection moving object from the captured images. However, they ignored dynamic background and fixed motion estimation for overall detection process. . COCOA framework used by Ibrahim et al. (2010) causes a combination of feature based method named RANSAC3. COCOA framework uses gradient based method to solve homography by implementing frame difference and background subtraction approach together for motion detection. Framework used by Sheng (2011) depicted shadow detection using invariant shape matching of corner features. However, their method depends mainly on lighting condition where shadow based segmentation method is able to identify moving object on clocked shadows. Another framework implemented by Pingting et al. (2012) used clustering single points obtained from motion estimation where noise and proper camera motion estimation were not considered for detection and cannot be adapted with unfixed motion changes and unfixed moving object detection. Ga̧szczak et al. (2011) was unable to differentiate different object in the same scene whereas Qian and Medioni (2009) combined color, shape and edge cues under a particle filter framework to provide robust tracking result and involves an adoption scheme to select most effective cues in different conditions. However, computation time as well as complexity depends on the number of steps used in frame works. Table 1 shows the steps that were used in the previous research in frameworks. Table. 1. Existing research for the usage of steps in frameworks Literature

Framework

Ibrahim et al. 3 (2010) Pollard Antone 9 (2012)

and

Wang et al. 18 (2011) Cheraghi and Sheikh 10 (2012) Pingting et al. 11 (2012)

Input, Frame difference (Motion analysis, Classification, Final detection) Raw input video, frame difference(motion analysis), final detection Input image, preprocessing, segmentation ,final detection Input image, feature extraction, frame difference, final detection Input image, feature extraction, classification, final

1936-6612/2011/4/400/008

Detection Rate

Detection Speed(fps)

False positive

80%

92ms per frame

18%

50%

x

x

3.97fps

x

x 85% 90%

x

x

x

x

doi:10.1166/asl.2018.10667

RESEARCH ARTICLE

Adv. Sci. Lett. 4, 400–407, 2016 detection Moranduzzo and Melgani 12 (2012) Long et al. 13 (2012)

Kembhavi et 16 al. (2011) Sheng 2 (2011) Zezhong et al. 15 (2012) Hsu-Yung et 8 al. (2012) Bhuvaneswari and Abdul Rauf 19 (2009) Gleason et al. 17 (2011) Huang et al. 21 (2010)

Input image, feature extraction, Classification, final detection Input image, feature extraction, Classification, final detection Input image, feature extraction, Classification, final detection Input image, feature extraction, Classification, final detection Input image, feature extraction, Classification, final detection Video sequence, Preprocessing, Feature extraction, post processing, final detection. Input image, Feature extraction, Classification, detection Input image, feature extraction, Classification, detection Input video image, frame difference, segmentation, detection

65% x

x

x 6 min computation time for Adaboost classification method 95 windows /second

96.6%

x

94.80%

x

x x

x

3.4% x

5.2 fps

3%

x

24.2fps

x

85%

x

x

x

x

x

92.31%

5. DATASETS AND VALIDATION Five kinds of existing datasets can be found in the previous research i.e. random UAV video sequence, PVD dataset9 , Standard libraries of UAV aerial images (UAV datasets, Public aerial datasets)18, DAPRA VIVID10 and Aerial images from Google Earth13 . Although almost all of this dataset are not noise free images, by the use of proper robust handling approach, noise can be handled. The main purpose of moving object extraction is the correct detection of object in every image. Performance measurements are demonstrated using six metrics, i.e. i) TP -True Positive ii) FP - False Positive iii) FN - False Negative iv) DR - Detection Rate v) FAR - False Alarm Rate vi) CT - Computation Time3,8,19,32 7. JUSTIFICATION FOR THE NEED OF NEW SEGMENTATION METHOD

Three kinds of methods are described in section 3, however each of the methods have disadvantages, i.e. optical flow methods require higher computation time in lieu with higher requirement of the hardware, background subtraction method requires quickly establishment of the background so as to free from surrounding interferences, frame difference method can obtain motion of single moving object but cannot extract overall shape of the object. Based on the indiscriminate review, this research found that by using only segmentation based approach can overcome the shortcomings of using frame difference based method only. Image Segmentation is used to determine the candidates of moving objects in each frame. Image segmentation extracts a more complete shape of the objects and reduces computation cost for moving object detection using aerial images2,33 .But it does not have the ability to distinguish moving regions from the static background. To attain complete object using frame difference is not possible because frame difference method only detects pixel with motion26. However, hybridization like combine frame difference and segmentation together8,17,18,32 can be a good hope to achieve optimum detection performance shown in figure 2 and figure 3 although Huang et al. (2010) and Oreifej et al. (2010) could not attain trusted and expected performance due assumption of pure plane where execution times need to be low. Due to the weakness of applying segmentation and frame difference separately, a new segmentation methods need to develop to achieve higher detection results like reduced computation time with higher accuracy.

95% 90% 85% 80% 75% 70%

92.31%

Detection rate 85.00%

80.00%

Frame Difference Segmentation Frame difference (Hsu-Yung et al. (Gleason et al. and Segmentation (2012)) (2011)) (Breckon (2009))

Fig.2. Detection rate using frame difference and segmentation together and separately in state of art.

4

RESEARCH ARTICLE 7.00 6.50 6.00 5.50 5.00 4.50 4.00 3.50 3.00 2.50 2.00

Computation time 5.2 fps

Adv. Sci. Lett. 24, 961–965, 2018

6.25fps

3.97 fps

Frame Difference (Hsu-Yung et al. (2012))

Segmentation Frame (Wang et al. Difference and (2011)) Segmentation (Breckon (2009))

Fig.3. Computation time using frame difference and segmentation together and separately in state of art. 8. CONCLUSION The main purpose of this research is to illustrate segmentation of moving object using aerial images based on features, various methods, datasets with validation metrics and finally, the need of a new segmentation method. At first, segmentation using various features like colour, corner and edge feature is demonstrated to depict the disadvantage of each feature. After that, various methods like optical flow, background subtraction and frame difference methods are discussed with disadvantages. Next, various frameworks are mentioned with the gaps for optimum detection performance and least computation time. Next, datasets with validation matrics are presented to validate segmentation method which have been using by the previous researchers. Finally, the need of a new segmentation method is justified for optimum detection performance. Moving object segmentation is one of most crucial challenges in computer vision research especially for aerial type of images. Review presented in this research is expected to enhance the performance of the current research for segmentation of moving object in computer vision research field.

ACKNOWLEDGEMENT This research is supported by Ministry of Higher Education Malaysia research grant scheme of FRGS/2/2013/ICT01/UKM/02/4.

REFERENCES [1] Y. Yingqian, L. Fuqiang, W. Ping, L. Pingting, L. Xiaofeng, Vehicle detection methods from an unmanned aerial vehicle platform. IEEE International Conference on Vehicular Electronics and Safety (ICVES), (2012),pp. 411-415. 5

Adv. Sci. Lett. Vol. 24, No. 2, 2018

[2] W. Sheng, Vehicle Detection on Aerial Images by Extracting Corner Features for Rotational Invariant Shape Matching. IEEE 11th International Conference on Computer and Information Technology (CIT), (2011), pp. 171-175. [3] A. W. N. Ibrahim, C. Pang Wee, G. L. G. Seet, W. S. M. Lau, W. Czajewski, 2010, Moving Objects Detection and Tracking Framework for UAV-based Surveillance. Fourth Pacific-Rim Symposium on Image and Video Technology (PSIVT), (2010), pp. 456-461. [4] Z. Yihua, L. Yongli, Z. Jianbiao,A video semantic object extraction method based on motion feature and visual attention. IEEE International Conference on Intelligent Computing and Intelligent Systems (ICIS), (2010), vol. 2, pp. 845-849. [5] L. Coulter et al., Automated detection of people and vehicles in natural environments using high temporal resolution airborne remote sensing. Proceedings of the ASPRS Annual Conference. (2012), pp. 78-90. [6] P. Jinhui, L. Shipeng, Z. Ya-Qin, Automatic extraction of moving objects using multiple features and multiple frames. IEEE International Symposium on Circuits and Systems,(2000), vol. 1, pp. 36-39 vol.1. [7] A. F. M. S. Saif, A. S. Prabuwono, Z. R. Mahayuddin, T. Mantoro, Vision-based human face recognition using extended principal component analysis. International Journal of Mobile Computing and Multimedia Communications 5, 82 (2013). [8] C. Hsu-Yung, W. Chih-Chia, C. Yi-Ying, Vehicle Detection in Aerial Surveillance Using Dynamic Bayesian Networks. IEEE Transactions on Image Processing, 2152 (2012). [9] T. Pollard, M. Antone, Detecting and tracking all moving objects in wide-area aerial video. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2012. (2012), pp. 15-22. [10] S. A. Cheraghi, U. U. Sheikh, Moving object detection using image registration for a moving camera platform. IEEE International Conference on Control System, Computing and Engineering (ICCSCE), (2012), pp. 355-359. [11] L. Pingting, L. Fuqiang, L. Xiaofeng, Y. Yingqian, Stationary vehicle detection in aerial surveillance with a UAV. 8th International Conference on Information Science and Digital Content Technology (ICIDT),. (2012), vol. 3, pp. 567-570. [12] T. Moranduzzo, F. Melgani, A SIFT-SVM method for detecting cars in UAV images. IEEE International in Geoscience and Remote Sensing Symposium (IGARSS), (2012), pp. 6868-6871. [13] C. Long, J. Zhiguo, Y. Junli, M. Yibing, A coarse-to-fine approach for vehicles detection from aerial images.International Conference on Computer Vision in Remote Sensing (CVRS),. (2012), pp. 221-225. [14] Y. Yingqian, L. Fuqiang, W. Ping, L. Pingting, L. Xiaofeng, Vehicle detection methods from an unmanned aerial vehicle platform. IEEE International Conference on Vehicular Electronics and Safety (ICVES), (2012), pp. 411-415. [15] Z. Zezhong, W. Xiaoting, Z. Guoqing, J. Ling, Vehicle detection based on morphology from highway aerial images. IEEE International Conference on Geoscience and Remote Sensing Symposium (IGARSS), (2012), pp. 5997-6000. [16] A. Kembhavi, D. Harwood, L. S. Davis, Vehicle Detection Using Partial Least Squares. Pattern Analysis and Machine Intelligence, IEEE Transactions on 33, 1250 (2011). [17] J. Gleason, A. V. Nefian, X. Bouyssounousse, T. Fong, G. Bebis, Vehicle detection from aerial imagery . IEEE International Conference on Robotics and Automation (ICRA), (IEEE, 2011), pp. 2065-2070. [18] L. Wang, H. Zhao, S. Guo, Y. Mai, S. Liu, The adaptive compensation algorithm for small UAV image stabilization. IEEE International on Geoscience and Remote Sensing Symposium (IGARSS), (IEEE, 2012), pp. 4391-4394. [19] K. Bhuvaneswari, H. Abdul Rauf, Edgelet based human detection and tracking by combined segmentation and soft decision. International Conference on Communication and Energy Conservation in Control, Automation (2009), pp. 1-6. 1936-6612/2011/4/400/008

doi:10.1166/asl.2018.10667

RESEARCH ARTICLE

Adv. Sci. Lett. 4, 400–407, 2016 [20] M. A. Mofaddel, W. M. Abd-Elhafiez, Fast and accurate approaches for image and moving object segmentation International Conference on Computer Engineering & Systems (ICCES), (2011), pp. 252-259. [21] C.-H. Huang, Y.-T. Wu, J.-H. Kao, M.-Y. Shih, C.-C. Chou, A Hybrid Moving Object Detection Method for Aerial Images in Advances in Multimedia Information Processing - PCM 2010, G. Qiu et al., Eds. (Springer Berlin Heidelberg, 2010), vol. 6297, pp. 357-368. [22] H. Meuel, M. Munderloh, M. Reso, J. Ostermann, Optical flow cluster filtering for ROI coding. Picture Coding Symposium (PCS), 2013. (2013), pp. 129-132. [23] P. Jie, C. Wenjie, P. Wenhui, A new moving objects detection method based on improved SURF algorithm. 25th Chinese on Control and Decision Conference (CCDC), (2013), pp. 901-906. [24] M. K. Fard, M. Yazdi, M. MasnadiShirazi, A block matching based method for moving object detection in active camera. 5th Conference on Information and Knowledge Technology (IKT), (2013), pp. 443-446. [25] B. N. Subudhi, S. Ghosh, A. Ghosh, Moving object detection using Gaussian background model and Wronskian framework.International Conference on Advances in Computing, Communications and Informatics (ICACCI), (2013), pp. 17751780. [26] M. Kamaraj, Balakrishnan, An improved motion detection and tracking of active blob for video surveillance. Fourth International Conference on Computing, Communications and Networking Technologies (ICCCNT), (2013), pp. 1-7. [27] C. Bo-Hao, H. Shih-Chia, Accurate Detection of Moving Objects in Traffic Video Streams over Limited Bandwidth Networks,

[28] [29] [30] [31] [32] [33]

IEEE International Symposium on Multimedia (ISM), (2013), pp. 69-75. B. Dhananjaya, B. R. Murthy, P. Thimmaiah, Detection of Objects in Aerial Videos for Object Extraction and Tracking for UAV Applications. International Journal of Computer Applications 112, (2015). J. Liang et al., A method to improve accuracy of parameter identification for single image blurred by uniform linear motion. Beijing Ligong Daxue Xuebao/Transaction of Beijing Institute of Technology 32, 818 (2012). A. Gaszczak, T. P. Breckon, J. Han, Real-time people and vehicle detection from UAV imagery. IS&T/SPIE Electronic Imaging. (International Society for Optics and Photonics, 2011), pp. 78780B-78780B-13. Q. Yu, G. Medioni, Motion pattern interpretation and detection for tracking moving vehicles in airborne video, IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009. (IEEE, 2009), pp. 2671-2678. T. P. a. B. Breckon, S.E. and Eichner, M.L. and Wahren, K., Autonomous Real-time Vehicle Detection from a Medium-Level UAV. 24th International Conference on Unmanned Air Vehicle Systems. (2009). M. Teutsch, W. Kruger, Spatio-temporal fusion of object segmentation approaches for moving distant targets. 15th International Conference on Information Fusion (FUSION), (2012), pp. 1988-1995.

.

6