5th International Symposium on Computational and Business Intelligence
Determining Direction of Moving Object Using Object Tracking for Smart Wheelchair Controller
Fitri Utaminingrum, Ali Fauzi, Dahnial Syauqy, Randy Cahya W, Anggi Gustiningsih Hapsani Computer Vision Research Group, Computer Science Brawijaya University Malang, Indonesia e-mail:
[email protected],
[email protected],
[email protected],
[email protected],
[email protected] people on normal people requires some vigilance to ensure the safety of disabled people. On the other side, the vigilance cause a burden to normal people while doing daily activity, especially on outdoor. They cannot fully do their activity, while at the same time they must aware to the disabled people. The wheelchair need an improved controller to solve the problem and ease the activity of both normal and disabled people. The contribution of this paper is the new controller of smart wheelchair by using object tracking which track and follow the movement of target object. Object tracking is continuously locate the moving object in the video which recorded by camera. Object tracking have been implemented in many application to help the human daily activity. There are two kind of object tracking, single and multiple object tracking. Single object tracking is object tracking that only track one object in every frame of video, the example is [6]. Multiple object tracking is object tracking that track more than 2 objects in each frame of video. Some researches that implement the multiple object tracking is [7] and [8]. Baheti etc [7] compare the accuracy result of combination SIFT and RANSAC with MEANSHIFT and KLT. The combination SIFT and RANSAC have a robust tracking result which solved the occlusion problem, but have complex calculation that considerably takes time to process. Xia and Ludwig [8] use PSO to detect the density. We proposed single object tracking using CAMSHIFT algorithm. CAMSHIFT is method in object tracking which have simple calculation, short processing and good accuracy. Based on the advantages of CAMSHIFT, object tracking process on this paper will be realtime and rapid. The rest of the paper is organized as follows: in section 2 presents the explanation about proposed method. Experiment result and discussion is given in Section 3 and the conclusion in Section 4.
Abstract—People with disabilities who cannot move their whole body need other people to control the smart wheelchair or track the moving of object interest, in this case people. In this paper, we have proposed new movement controller of smart wheelchair using object tracking for disabled people who cannot move their whole body. The proposed method for determining direction of moving object using object tracking has been evaluated using invariant video. Our result study have success rate of multiple object detection is 82.01%, tracking object interest is 90.00%, and determining the moving object directions is 79.63%. Keywords- object tracking; smart wheelchair; disabled person; camshift; hog
I.
INTRODUCTION
People with disabilities have limitations to move themselves. They must depend on normal people to support their mobility. The common accommodation that available for person with disabilities is wheelchair. There are some kinds of wheelchair, manual and powered wheelchair. The powered wheelchair is controlled by user (most commonly using joystick) and having some batteries or electric motors to stir the wheelchair. Along with the increase of technology, the controller of wheelchair have been developed and adjusted to physical condition of the user. The physical condition of each people with disabilities is different. There are people who cannot move their hand and leg but still can move their head. The wheelchair that controlled by head movement have been conducted in [1] and [2]. Rivera [2] using K-means algorithm to cluster and recognize the head pose. The people with disabilities can control their wheelchair via mobile phone that have been built in [3]. The voice recognition also have been developed to improve the controller of smart wheelchair by Simpson [4][5]. The previous research and technology about recent improvement for smart wheelchair controller most commonly use the part of user’s body that are still in normal condition. Based on facts, there are people with disabilities who cannot move their whole body. The disabled people who cannot move their whole body need others to control the smart wheelchair along the time. The dependency of disabled
978-1-5386-1772-4/17/$31.00 ©2017 IEEE
II.
PROPOSED METHOD
In this study, the method consists of three main steps: multiple object detection, tracking the object of interest location, and determine the movement direction of the object of interest. The overview of the proposed method is illustrated in Fig. 1. The input data of system is video. Only the first frame of video data will be processed to multiple object detection. The object that have been selected by user
6
Start
Video, object interest
Determine the moving direction of object interest
Multiple Object Detection
Direction
Tracking the object interest location
End
Figure 1. Overview of Proposed Method
will be targeted and its location being an initial position in tracking process for the next frame.
Figure 3. Person as target object
A. Multiple Object Detection The target object in this paper is people. The aim of multiple object detection process is to detect people in the first frame and choose the one people who will be tracked in the next frames of the video. The feature was used to detect the blob or object of interest is full body of human. The full body image of target object will be the input data in tracking process. The result of multiple object detection process is shown in Fig. 2. Before the HOG person detector implemented in this study, the image need to be preprocessed to remove noises. One of many goals in developing noise removal method is preserving important detail of the texture [15]. The important texture detail is the most important feature in object detection. We implemented HOG person detector algorithm that proposed by Dalal [10]. The HOG person detector algorithm is one of the most robust and popular person detector to date. The HOG person detector use SVM approach as classification method. The SVM algorithm classify the object into “person” or “not a person”. The HOG person detector uses detection window that scan on over image to get the descriptor value. The size of detection window is 8x8. Each detection window calculate gradient vector value and put the values into 9-bin histogram. The histogram range is 0-180 degree, which mean there are 20 degrees per bin. The value of histogram is calculated from magnitude of gradient vector.
The challenge of robust feature is invariant in illumination. The solution to get robust feature is normalize the histogram that have been obtained. Dalal and Triggs [10] used block normalization rather than normalize the histogram. The block that used consisted of 2 cells by 2 cells. The descriptors that have been calculated will be trained and classified in SVM classifier. All objects that were detected as people were marked by bounding box, see Fig.2. The user set one of the objects as target object that will be tracked along time in the next step. B. Tracking the Object of Interest The aim of tracking the object of interest process is to locate the people in each frame of video. The target object initial location have been set in multiple object detection. The system will track the movement of this target object and ignore the other people. The image of human as object of interest in this study is shown in Figore 3. We implement CAMSHIFT algorithm to track the object of interest for each frame. CAMSHIFT is based principle of the MEANSHIFT. The advantages for using CAMSHIFT than MEAN SHIFT is that CAMSHIFT works well in dynamic probability distribution whether the MEAN SHIFT only works well in static. The color space that used to track the interest object is hue. We have try the another color space of image like grayscale, CbCr and multiplication result of grayscale and hue channel, but the best result was obtained from hue channel. The Figure. 4 show the result of tracking interest object in Figure.3. The CAMSHIFT algorithm have been implemented as such : 1. The location of target object that obtained in multiple object tracking process is set as initial location of search window 2. The color probability distribution is calculated 3. MEAN SHIFT is performed until suitable convergence. The zeroth moment and centroid coordinate are computed. 4. The search window for the next frame of video centered around the centroid of object in current frame and the size is scaled by a function of the zeroth movement. 5. Repeat to step 2
Figure 2. Multiple object detection
7
Figure 4. Peoples that have been tracked
Figure 5. Our application
C. Determine the Object Movement Direction The output of tracking the object of interest process is bounding box area of object that convergen with object interest in the next subsequent frame. The centroid will be calculated based on this bounding box area. The change of centroid coordinate D(x,y), horizontally (based on xcoordinate) between one frame and the next subsequent frame will also be calculated to know about moving direction. Direction
left , ® ¯right ,
if D( x, y ) 15 if D( x, y ) ! 15
Table 1 shows the result of the accuracy by using our proposed method to detect and to track the object of interest. For single object detection and tracking, the proposed method is showing promise. However, the result of the multiple object detection was not good enough because the algorithm failed to detect some people who wear wide clothes like dress so that the feature of human can not be detected. The illustration about people who wear a big clothes like dress and can not be detected as object is illustrated on Fig. 6. There are some false tracking result we get while running the tracking algorithm. In some result, we can see that the other person or pedestrian was also detected by bounding box. This issue make our result not too high. The solution of this problem is to find the right color space of image that can be used as feature while tracking the object of interest. Fig. 7 show the false tracking of pedestrian. We also tried to use invariant channel as feature in object tracking. Table 2 show the comparison result between invariant channel regarding object tracking. The best result gotten from hue channel with value is 90%. In the determine of movement object direction, the simple calculation give good result direction. The direction for this research is limited to right and left only. We cannot measure the depth of object based on 2D image. Because of that, we cannot determine the forward and backward of object movement. This matter can be solved if we use the stereo image. By using the stereo image, we can calculate the depth of object that can be important value to determine the forward and backward object.
(1)
The changing in x coordinate of the centroid will be used to predict the direction of object’s movement. We use threshold = 15 pixels when the object of interest is moving to the right, and threshold = -15 pixels when the object of interest is moving to the left. If the change of D(x,y), is more than 15 pixels, the object of interest is moving to right. If D(x,y) is less than -15, the object of interests is moving to left. The determination of direction by using Equation 1. III.
RESULT AND DISCUSSION
Fig. 5 show the application that have been implemented. The proposed method for determining direction of moving object using object tracking has been evaluated using variant image. The multiple object detection evaluated by using 50 selected frames of two videos. The object tracking was evaluated by using 350 sequence frames. The determine the object moving directions was evaluated by using 2 videos that consist of 350 and 489 frames. We implement the proposed method in computer with Pentium Dual Core processor and 2 Gigabyte of RAM. The accuracy of the proposed systems is calculated using Equation 2. Where T is the total true results that are identified by system, and n is total frame. The results of the experiment then reported and compared with other methods. accuracy
T u 100% n
IV.
CONCLUSION
The disabled who can not move their whole body need other peoples to control the smart wheelchair and track the moving of object (people) interest. In this paper, we have proposed controller of smart wheelchair using object tracking for disabled who cannot move their whole body. The method consists of three steps, they are multiple object detection, tracking the object interest location, and determine
(2)
8
TABLE I. Multiple Object Detection
RESULT OF EXPERIMENT Viola-John (%)
Proposed Method (%)
Video 1 (20 frames)
45.00
82.50
Video 2 (30 frames)
24.92
81.52
Average
34.96
TABLE II.
propose method to determine the direction to onward and backward movement by using stereo images. REFERENCES [1]
82.01
RESULT OF COLOR SPACE IMAGE EXPERIMENT ON TRACKING
[2]
Accuracy (%) Hue Channel
90
Gray Channel
30
Incorporate Hue and Gray
79
[3]
[4] TABLE III.
RESULT OF DETERMINING THE DIRECTION
Determine the object moving direction
Proposed Method (%)
Video 1
84.85
Video 2
74.41
Average
79.63
[5]
[6]
[7]
[8]
[9] [10]
[11] Figure 6. Bad result of detection process : Some people did not detected (left), other people that also tracked by algorithm (right)
[12]
the moving direction of object interest. The multiple object detection was evaluated by using 50 selected frames of two video. The object tracking evaluated by using 350 sub sequence frames from video. The determine the object moving directions by using two videos that consist of 350 and 489 frames. Our study has good enough results with success rate of multiple object detection is 82.01%, tracking object interest is 90.00% and determine the movement object direction is 79.63%. The result of the multiple object detection was not good enough because there are some persons which can not be detected by our method. This research focus to determine the right and left directions. In future research, we will
[13]
[14]
[15]
9
F. Abedan Kondori, S. Yousefi, L. Liu And H. Li, Head Operated Electric Wheelchair, Southwest Symposium On Image Analysis And Interpretation, San Diego, Ca, 2014, Pp. 53-56, 2014. L. A. Rivera, G. N. Desouza And L. D. Franklin, Control Of A Wheelchair Using An Adaptive K-Means Clustering Of Head Poses, IEEE Symposium On Computational Intelligence In Rehabilitation And Assistive Technologies (Cirat), Singapore, 2013, Pp. 24-31, 2013. R. Achkar, G. A. Haidar, H. Dourgham, D. Semaan And H. Araji, Mobile Controlled Wheelchair, IEEE European Modelling Symposium (Ems), Madrid, Pp. 429-434, 2015. R. C. Simpson And S. P. Levine, Voice Control Of A Powered Wheelchair, In IEEE Transactions On Neural Systems And Rehabilitation Engineering, Vol. 10, No. 2, Pp. 122-125, June 2002. R. C. Simpson And S. P. Levine, Adaptive Shared Control Of A Smart Wheelchair Operated By Voice Control, Intelligent Robots And Systems, 1997. Iros '97., Proceedings Of The 1997 IEEE/RSJ International Conference On, Grenoble, Pp. 622-626 Vol.2, 1997. Hyunhak Shin, Chuljin Cho And Hanseok Ko, Single Object Tracking Based On Active And Passive Detection Information In Distributed Heterogeneous Sensor Network, 2016 13th Ieee International Conference On Advanced Video And Signal Based Surveillance (Avss), Colorado Springs, Co, USA, Pp. 444-449, 2016. B. Baheti, U. Baid And S. Talbar, An Approach To Automatic Object Tracking System By Combination Of Sift And Ransac With Mean Shift And Klt, 2016 Conference On Advances In Signal Processing (Casp), Pune, India, Pp. 254-259, 2016. G. Xia And S. A. Ludwig, Object-Tracking Based On Particle Filter Using Particle Swarm Optimization With Density Estimation, 2016 IEEE Congress On Evolutionary Computation (Cec), Vancouver, Bc, Canada, Pp. 4151-4158, 2016. P. Soille. Morphological Image Analysis: Principles And Applications, Vol. 2 (Springer- Verlag, New York, 2004). N. Dalal And B. Triggs, Histograms Of Oriented Gradients For Human Detection, 2005 IEEE Computer Society Conference On Computer Vision And Pattern Recognition (Cvpr'05), San Diego, Ca, Usa, Pp. 886-893 Vol. 1, 2005. P. Viola and M. Jones, Rapid object detection using a boosted cascade of simple features, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, pp. I-511-I-518 vol.1, 2001. Kruppa H., Castrillon-Santana M., and B. Schiele. Fast and Robust Face Finding via Local Context. Proceedings of the Joint IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pp. 157–164, 2003. T. Tanaka, K. Shoji, F. Toyama, and J. Miyamichi, Layout analysis of tree-structured sceneframes in scene images, IJCAI pp. 2885–2890 , 2007. F. Utaminingrum, K. Uchimura and G. Koutaki, High density impulse noise removal based on linear mean-median filter, in Proceedings of Frontiers of Computer Vision, (FCV), 2013 19th Korea-Japan Joint Workshop on Computer Vision, pp. 11–17. F. Utaminingrum., K. Uchimura, G. Koutaki, Mixed Gaussian And Impulse Noise Removal Based On Kernel Observation and Edge Direction . International Journal of Innovative Computing, Information and Control, 5 vol. 11, pp. 1509-1523, 2015.
