A Fast Algorithm for Detecting, Labeling and Tracking

2011 3rd International Conference on Signal Acquisition and Processing (ICSAP 2011)

A Fast Algorithm for Detecting, Labeling and Tracking Volleyball Players in Sport Videos

Hananeh Salehifar

Mohammad Mahdi Dehshibi

Faculty of Electrical, Computer, and IT Islamic Azad University, Qazvin Branch, Qazvin, Iran [email protected]

Faculty of Electrical, Computer, and IT Islamic Azad University, Qazvin Branch, Qazvin, Iran [email protected]

Azam Bastanfard Dep. of Computer, Faculty of Engineering Islamic Azad University, Karaj Branch, Karaj, Iran [email protected] Abstract—this paper aims at successful tracking of volleyball athletes during competition using only a single camera. Due to the wide range of possible motions and non-rigid shape changes, the tracking task becomes quite complex. We propose a novel method based on adaptive background subtraction by two concurrent frames. The proposed method detects and labels with high precision. It is a reliable tracking in cases of facing conditions such as multiple player occlusions. Keywords-human motion;tracking; volleyball athletes

I.

ensured a reliable background subtraction system, which are not very practical approach. This paper proposes a new solution for detecting, labeling, and tracking of players in Volleyball videos. This algorithm performs adaptive background subtraction on two concurrent frames in the sequence. This paper is organized as follows: Labeling and Tracking algorithms are described in section II. Experimental results and implementation are reported in section III. Finally, this paper is concluded in section IV.

INTRODUCTION

In recent years, new trends have aroused for computer vision and image processing. These new trends are more practical. One the remarkable application is analyzing the players’ activity in different sports. Locating, labeling, and tracking players have broad application, especially for a team analyzer, and in the broadcast sports videos industry. There are several issues which make this kind of video analyzing a challenging task. Players’ occlusion, similar player’s appearance, various numbers of players, unexpected camera motion, noise, and video blur are the samples of this difficulty. Many algorithms have been proposed to deal with the multiple target-tracking (MTT) problem in which Particle Filter [1], [2], Joint Probabilistic Data Association Filter (JPDAF) [3], Multiple Hypothesis Tracking (MHT) [4], MCMC data association [5], [6] and track linking [7], [8], [9] are used. Several researchers also study the specific problem of labeling and tracking of players in sports video [2][10][11]. In [10], a clustering based trajectory matching method is proposed to track players in a soccer video. In this work, labeling of individuals is achieved through a supervised classification method. Comaniciu et al [11] build a track graph, and take the tracking problem as an inference problem in a Bayesian network. In both works, a multicamera system is used to get a fixed, high-resolution and wide-field view of a soccer game. These pre-defined settings

978-1-4244-9386-9/11/$26.00

C

2011 IEEE

II.

TRACKING ALGORITHM

In most cases, different movie, which should be analyzed, have different number of frames; thus, the proposed method first determines the number of frames related to the input film and extracts frames as still images. As the proposed method contains an iterative procedure, in order to save memory space and speed up the algorithm, just one frame loads into memory during each iteration. “Table I” shows the functionality of the proposed method: TABLE I.

ADAPTIVE BACKGROUND SUBTRACTION PROCEDURE.

Adaptive Background Subtraction Algorithm 1. Initialize step /* x and y are the pixels’ position */ B(x, y) = First Frame; 2. Subtract Next Frame From B D(x, y) = (B(x, y) +1) –(B(x, y)); 3. Filtering step /* α is threshold */ I x, y 4.

Assign new frame:

0, D x, y 1, D x, y

α α

B(x, y) = B(x, y) +1; While there is any frame GOTO Step 1;

In order to enhance highly connected "blobs" in the image, dilation operator is applied to the filtered image. Then, each blob is ANDed with the subtracted background image to isolate the connected components. The objective of this

V2-398

2011 3rd International Conference on Signal Acquisition and Processing (ICSAP 2011)

step is allowing the algorithm to draw the smallest possible rectangle around the moving object. In this algorithm, the

blobs that their size is smaller than 100 pixels assume as noise and do not consider by the object’s rectangle.

Figure 1. the proposed tracking algorithm scheme.

Finally, cross correlation between the normalized blob and a list of known blobs from previous fames is calculated to find an adaptation. If any matching can be found then the color corresponds to the known patch is used as the rectangle color. The known patch is then updated with the new image by cutting out the middle of the blob. If there are too many failures to track an individual object, i.e. if the known image list gets too large, the algorithm will fall back into initial step. However, in this step yellow color rectangles are utilized. In order to detect each blob once, a clever method should be defined. To achieve this, every blob is removed from the binary image before the next iteration of the blob detection. In this manner, blobs are never detected more than once and as a result, only one yellow rectangle is drawn around each moving object. In order to find out boundary of the moving objects and to draw rectangles around them, a sole algorithm is developed. The aim of the proposed algorithm is calculating the bounds of the connected components which identified as a single entity. For each frame, bounds for multiple boxes are stored and then drawn on the original frame using a predefined color. After applying the whole procedure to the last frame of video, the output labeled video is generated and saved with “avi” format. Overall procedure of the proposed method is illustrated in “Fig. 1”. The structure of the proposed method is such designed that any changes in motion across the sequence of videos can be detected. Therefore, if there is no moving in the sequence of the frames, we will lose the still object. “Fig. 2” shows since the player in the right side of frame has no movement, the algorithm cannot detect and track it.

(a)

(b) Figure 2. Changing in image detection process for (a) original image, (b) the difference image.

III.

IMPLEMENTATION

In order to implement the proposed algorithm (ABS), two kinds of data related to the Volleyball sport, where the cameras are fixed, are used. The main reason that we chose fixed camera traced back to the nature of the proposed

V2-399

2011 3rd International Conference on Signal Acquisition and Processing (ICSAP 2011)

method; becaause the methood should be apply on diff fference of two conseecutive framees. As it is evident e in “F Fig. 3”, utilizing ABS players arre detected and a are labeeled in f Conseequently, trackking players in two consecutive frames. concurrent fraames in the seqquence is achiieved.

thhreshold value of 15. Althouugh this value eliminates a llot of ouutliers caused by noise in the image fraames while at a the saame time dep picting the m moving objectss which we were innterested in, it was the ooptimum valuue. As conduucted exxperiments aree shown, thosse values that were less thaan 15 remove both deetected noisy pixels as welll as other mooving v obbjects in the scene. On thhe other handd, threshold values higher than 15 cause renderring real movving objects inn the sccene hardly disstinguishable. The last issuue that we faace was determ mining a thresshold vaalue for blob detection d proccess. In our bllob detection loop, blobs with less than 100 pixeels are discardded and yellow w box m. We use thiis threshold vvalue arre not drawn around them beecause experim ments show tthat this valuee can successsfully exxclude much of o the noisy piixel in the video frames thatt was ennhanced by applying dilationn process. IV.

Figure 3. playeers and ball are suuccessfully trackeed by using the prroposed system, colorr rectangle show tthe location wherre the object detected.

There are two major isssues should be b considered during implementatioon, which wee describe theem in detail in this section. Initiaally we designn our program m to load all of the image framess at the start oof the program m. This worked fine during the eaarly developm ment of our alggorithm, takin ng only twenty second ds or so to loaad all of the images. i Howeever, as we further developed oour code to perform Ad daptive S (A ABS), loadingg all of the im mages at Background Subtraction the start of thhe program cconsumes too much memo ory. To solve this prooblem, we maade the decisioon to revise how h we loaded the im mage frames. To T achieve thiss, loading fram mes left on the fly as the adaptive background b suubtraction wass being performed. In this mannner we savedd memory annd the proposed methhod is executeed significantlly faster. Another issue is chooosing a coorrect and acccurate p threshold for the adaptive background subtraction process. mplate was subtracted s froom the When the baackground tem current imagee frame, we haad to find a vaalue for the thrreshold that would elliminate mostt of the outlieers. We settleed on a

C CONCLUSION

s out the pproblem of traacking and labbeling This paper sorts off volleyball athhletes during competition using u only a single s caamera. Due to the wide rangge of possiblee motions and nonriggid shape chhanges, the tracking taskk becomes quite coomplex. We propose p a noovel method based b on adaaptive baackground subbtraction on two concurreent frames inn the seequence. usion is a com mmon With the beest of our knowledge, occlu prroblem encounntered in trackking multiple people. p This might m bee due to compplete or partiall overlap of a person by annother peerson. Anotherr contribution to this work is to overcom me the prroblem of partial occluusion. Adapptive Backgrround Suubtraction (A ABS) can deetect players with separrating coonnected com mponent in thhe most fram mes. The m missed deetection by thhe proposed system s referrred some com mplex sittuations due to t full occludeed objects witth each other since thhe proposed system s deals only on the partial occlu usion. Annother reason for the misseed detection by y proposed sy ystem is the image ressolution. In this work, the syystem deals w with a low resolution video sceness to increase the accuracyy and mount of occlu usion exxecution time of the system;; hence this am is natural and allso it is less thhan the existing g methods. As we disccussed earlierr, when the objects o movee, the H wh hen an object stops alggorithm workks very well. However, m moving, the bo ox around it ggoes away. This T is because the differencing is adaptive a and nnon-moving obbjects will beccome thhe backgroundd image. Annother benefitt of the propposed m method is diagnnosis ball in thhe game. The ball always m moves inn the consecuttive frames annd it can be detected d very well I conclusion,, this fast and efficient algorrithm uttilizing ABS. In whhich we calleed Adaptive B Background Subtraction S (A ABS), caan detect playyers and ball iin analyzing a Volleyball game. g “F Fig. 4” demonnstrates the abbility of the prroposed methood in traacking and labbeling the athhletes and otheer moving objjects, i.ee., ball.

V2-400

2011 3rd International Conference on Signal Acquisition and Processing (ICSAP 2011)

Frame 10

Fram me 36

Frame 89

Fram me 91

Figuure 4. this figuree demonstrates thhe ability of the prroposed to track people p in complex situation

[8]]

REFERENCES [1]

[2]

[3]

[4]

[5]

[6]

[7]

J. Sullivan and S. Carlssonn, “Tracking andd Labeling of Innteracting MultipleTaargets,” In Proc.. European Connf. on Computerr Vision (ECCV), 20 006. Thomas Maauthner1, Christinna Koch2,“Visuaal Tracking of Atthletes in Beach Vollleyball Using a Single Camera”,, International Joournal of Computer Science S in Sport,22008. M. Isard annd J. MacCormicck, “BraMBLe: A Bayesian Multiple-Blob Tracker,” In I Proc. IEEE Innt’l Conf. on Computer Vision (ICCV), 2001. K. Okuma, A. Taleghani, N. N de Freitas, J. J. J Little, and D. G. G Lowe, “A Boosted d Particle Filter: Multitarget Dettection and Trackking,” In Proc. Europpean Conf. on Coomputer Vision (E ECCV), 2004 D. Reid, “An “ Algorithm For Tracking Multiple M Targetss,” IEEE Transactionn on Automatic Control, vol. 24, 2 no. 6, pp. 843–854, 8 December 1979. R and S. Sastry, S “Markov Chain Monte Caarlo Data S. Oh, S. Russell, Associationn for Multiple-Taarget Tracking,” University of Caalifornia, Berkeley, Technical T Report UCB//ERL M05/19, June 2005. Q. Yu, G. Medioni, “Map-E Enhanced Detecttion and Trackingg from a D Associationn,” IEEE Moving Pllatform with Loccal and Global Data Workshop on o Motion and Video V Computing (WMVC'07), 20007.

A.G.A.Pereraa,C.Srinivas,A.Hooogs,G.Brooksby,WenshengHu,“M MultiObject Trackiing Through Sim multaneous Long Occlusions and SplitMerge Condittions,”IEEE Int’ll Conf. on Compu uter Vision and P Pattern Recognition (CVPR), ( 2006. [9]] C. Stauffer, “Estimating “ Tracck Sources and Sinks”, In Proc. IEEE Workshop on Event Mining inn Video, 2003. [100] P. Nillius, J. Sullivan and S S. Carlsson, “Mu ulti-Target Trackking – Linking Identtities using Bayessian Network Infference,” In Proc.. IEEE Int’l Conf. on Computer Vission and Pattern Recognition (CVPR), 2006. [11] D. Comaniciuu, P. Meer, “ M Mean shift: a ro obust approach ttoward feature space analysis”, IEEE E Transactions on n Pattern Analyssis and Machine Intellligence 24 (5) (22002) 603–619.

V2-401