IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 7, NO. 5, OCTOBER 2005
853
Fast Object Tracking Using Adaptive Block Matching Karthik Hariharakrishnan and Dan Schonfeld, Senior Member, IEEE
Abstract—We propose a fast object tracking algorithm that predicts the object contour using motion vector information. The segmentation step common in region-based tracking methods is avoided, except for the initialization of the object. Tracking is achieved by predicting the object boundary using block motion vectors followed by updating the contour using occlusions/disocclusion detection. An adaptive block-based approach has been used for estimating motion between frames. An efficient modulation scheme is used to control the gap between frames used for motion estimation. The algorithm for detecting disocclusion proceeds in two steps. First, uncovered regions are estimated from the displaced frame difference. These uncovered regions are classified into actual disocclusions and false alarms by observing the motion characteristics of uncovered regions. Occlusion and disocclusion are considered as dual events and this relationship is explained in detail. The algorithm for detecting occlusion is developed by modifying the disocclusion detection algorithm in accordance with the duality principle. The overall tracking algorithm is computationally superior to existing region-based methods for object tracking. The immediate applications of the proposed tracking algorithm are video compression using MPEG-4 and content retrieval based on standards like H.264. Preliminary simulation results demonstrate the performance of the proposed algorithm. Index Terms—Adaptive motion estimation, K-means clustering, segmentation, visual tracking.
I. INTRODUCTION
V
ISUAL tracking has been an area of intensive research in the field of computer vision. With the advent of emerging multimedia standards like MPEG-4 it has become even more essential to develop a system that performs visual tracking in a robust as well as computationally efficient manner. Applications of object tracking range from video compression, video retrieval, interactive video, scene composition, etc. A variety of techniques have been employed for segmenting a semantically meaningful object out of a video scene. The most common approaches that have been proposed fall into the following categories. Region based tracking, active contours (a.k.a snakes) and mesh-based tracking. In the first approach (region-based tracking), the video object is initially defined by the user/object recognition algorithm. Video sequences are then segmented using a classical tool like Manuscript received October 29, 2002; revised March 14, 2004. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Chalapathy Neti. K. Hariharakrishnan was with the Multimedia Communications Laboratory, Department of Electrical and Computer Engineering (M/C 154), University of Illinois at Chicago, Chicago, IL 60607-7053 USA. He is now with the Multimedia Group, Motorola, Inc., Bangalore, India (e-mail:
[email protected]). D. Schonfeld is with the Multimedia Communications Laboratory, Department of Electrical and Computer Engineering (M/C 154), University of Illinois at Chicago, Chicago, IL 60607-7053 USA (e-mail:
[email protected]; http://www.ece.uic.edu/~ds). Digital Object Identifier 10.1109/TMM.2005.854437
the watershed transformation. Correspondence between the segmented regions in consecutive frames is established and this enables tracking of the video object in subsequent frames [3], [7], [13]. Contour-based approaches usually do not make use of the spatial and motion information of the entire object and rely only on the information closer to the boundary of the video object [2], [5], [11], [12], [16]. Snakes [10] was proposed as a method for tracking the boundary of the video objects by using a parametric planar curve (active contours). Mesh based approaches [1], [9], [15], [18] define an initial set of nodes on the boundary, as well as the interior of the object using gradient and motion information. These set of nodes are then joined using a triangulation procedure like the Delaunay triangulation to produce a content-based mesh. These set of nodes are tracked forward by sampling the node motion vectors from the estimated optical flow [8]. A variation of region-based tracking, referred to as motionbased tracking, is introduced in [17]. Motion-based clustering on the optical flow has been employed for generating the regions that are coherent in motion. Although motion provides a powerful description of the visual scene, motion-based criteria in isolation are insufficient for object tracking. One of the main problems confronted by tracking algorithms is partial occlusion. Several approaches have been proposed for occlusion detection in video sequences [1], [5], [14]. Occlusion detection methods proposed thus far have focused exclusively on the progressive hiding of a portion of the object by an occluding body [1], [5], [14]. In the literature, a complementary effort to address the task of disocclusion detection has been limited. The computational complexity of object tracking systems is due to the costly segmentation, optical flow or motion estimation operations. Practical real-time systems must, therefore, avoid costly repetition of any of these operations. The algorithm outlined in this paper aims to predict the object boundary for a long duration without the need for user interaction. Section II gives detailed information on the proposed approach. Section III discusses the algorithm used for identifying occlusion/disocclusions in video sequences. Section IV includes experimental results to demonstrate the proposed method. Conclusions and further research have been included in the last section. II. BASIC TRACKING ALGORITHM A. General Approach The proposed algorithm can be categorized as a region-based tracking algorithm. The occlusion and disocclusion techniques that have been developed can be employed in other region-based
1520-9210/$20.00 © 2005 IEEE
854
IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 7, NO. 5, OCTOBER 2005
Fig. 2. Object initialization: (a) segmentation map and (b) regions in the segmentation map with reasonable motion.
Fig. 1. Tracking algorithm.
techniques to improve the tracking accuracy. The overall algorithm has been outlined in Fig. 1. All the steps in Fig. 1 are explained in the following sections. In the following sections, re-segmentation refers to segmentation of the frame using the algorithm mentioned below followed by user interaction to re-initialize the object partition. B. Object Mask Initialization For initializing the tracking algorithm, we use a segmentation algorithm followed by an association operator. The association operator gives information about the regions that belong to the object. The segmentation proposed by [6] was used as it gives better results when compared to standard watershed segmentation. The algorithm for initializing the object is given below. 1) Segment the initial frame using four-band multi-valued segmentation [6]. The initial segmentation map is denoted by . that be2) The aim of this step is to locate regions from long to the object. To locate these regions, we compute motion vectors of all the regions in the segmentation map . All the regions that have reasonable motion are labeled as object regions in a mask. 3) The previous step might include regions that belong to the background. Hence, a post-processing operation is performed to remove small regions. The area opening operator has been employed for morphological post-processing. The holes in the mask are then filled to give the final mask. If the obtained mask is too erroneous, manual initialization is performed. The result of applying the above steps on the bream sequence is shown in Fig. 2. The above-mentioned method works well for static camera scenes. The object can also be drawn using a graphical interface for manual initialization. Automatic initialization can also be done if the class of objects to be tracked is known. For example, skin color can be used to initialize a face tracker. The result of the tracking algorithm depends to a great
Fig. 3. Estimated seed blocks for bream sequence: (a) frame 75 and (b) seed motion blocks in frame 78 (black—uncertain blocks, white—seed motion blocks of object, gray—seed motion blocks of background).
extent on the initialization. A perfect initial contour is the best input that can be given to a tracking algorithm. Tracking results with automatic and manual initialization are given in Section IV. C. Motion Estimation Motion estimation is a fundamental measurement to object tracking and hence an accurate estimation of motion is one of the most important steps. The method presented in this paper calculates the block sizes depending on the location of the block. Varying the block sizes near the boundary of the object is equivalent to mesh-based motion estimation. Hence, motion estimation is more accurate than traditional block-matching techniques. 1) Block Classification: Let I(x,y,k) denote the kth frame of a video sequence with (x,y) denoting x and y coordinates of a pixel. Seed motion blocks are estimated for every frame. A seed motion block is defined as one that lies either entirely within the object or background. The algorithm starts with an initial block size of 16 16 pixels and backward motion is estimated (Every ) is matched with the corresponding block from I(x,y, block in I(x,y,k). Exhaustive search has been employed for computing motion vectors on a search window of 32 32. Motion estimation has been performed in the Y,U,V space. Blocks that lie on the boundary are labeled as uncertain blocks and are processed by the next stage of the estimation step. Fig. 3 shows the seed blocks that have been computed for the bream sequence. The uncertain blocks are subdivided into smaller blocks (8 8) and new seed blocks are estimated. The search range is also reduced to make sure that misclassifications do not occur. 4 This procedure is continued until a fixed block size (4 pixels) is reached.
HARIHARAKRISHNAN AND SCHONFELD: FAST OBJECT TRACKING USING ADAPTIVE BLOCK MATCHING
D. Modulation Scheme In many video sequences, the motion between consecutive frames is comparatively less. In the proposed approach, frames. A modulatracking is performed once every three tion scheme is provided that calculates object motion between and slows down motion estimation if the frames & estimated motion is relatively high. This modulation scheme enables motion estimation to be applied infrequently if the motion in the video sequence is low. Average motion between the frames is found by fitting an affine motion model to the initial seed blocks estimated in the initial part (Fig. 3) of the motion estimation algorithm. The affine model is defined by
where , in the matrix are the model paramin eters. The transformation displaces a point the reference frame to a new location in the previous frame. A least squares based algorithm is used to extract the motion model parameters. The translational component of the affine model reflects the amount of motion undergone by the object. If the above norm is greater of the translational component is greater than a , motion estimation is repeated over frames that are threshold . This corrects for substantial tracking errors, adjacent if the algorithm were applied once in three . If the size of the object is small compared to the size of the frame, there might not be any object seed blocks present. In this case, motion vectors computed from smaller blocks (e.g., 4 4 blocks) are used to find the affine model. E. Object Mask Generation The partition of previous frame that corresponds to the object is denoted by and the goal of this component is to generate , given the motion the current partition of object support vectors. Let denote the motion vector that has been computed for the region (block) . Adrepresent the translated version of region ditionally let by . In our case h represents the computed motion vector for each block. Every block in the current frame is motion compensated to find out the portion of the block that lies within . This . The gives us the object support for the current frame object mask needs to be modified suitably to take care of occlusions and disocclusions. Occlusions and disocclusions are handled using the method detailed in the next section. III. OCCLUSIONS AND DISOCCLUSIONS The most common method to deal with occlusion/disocclusion consists of finding the motion compensated frame from the computed motion vector field [1]. The compensated frame is to give subtracted from the original frame and thresholded the outlier pixels in the current frame. If forward motion is estiand frame k is reconstructed using frame mated
855
the outlier pixels detected correspond to regions that will . For backward motion, frame be covered in frame is reconstructed using frame k. The outlier pixels detected in this case correspond to new (uncovered) regions that have appeared . Ideally, the new regions should correspond to in frame disocclusions and the covered regions to occlusions. But there are many cases under which some of the uncovered regions do not correspond to disocclusions as explained below. The aim of the disocclusion algorithm is to detect uncovered regions that are actual disocclusions. The following paragraph explains the algorithm for disocclusion detection. The duality principle can be employed to derive the algorithm for occlusion detection. The regions which cannot be motion compensated accurately are declared as covered or uncovered regions. In many instances, existing regions cannot be motion compensated accurately due to nonrigid structure or illumination changes. In such cases, some covered and uncovered regions detected do not correspond to occlusions and disocclusions. It is clear that a further classification is required to find actual disocclusions from uncovered regions. This can be achieved by using a motion-based criterion. A. Duality Principle Occlusion and disocclusion are viewed as dual events. For detecting disocclusion (occlusion), the current frame is motion compensated using the previous (next) frame to give uncovered (covered) regions. In the case of disocclusion (occlusion), the uncovered (covered) regions that belong to the object exhibit motion characteristics that are similar to (different from) the object. The algorithm that detects disocclusion (occlusion) looks for this motion similarity (dissimilarity). Based on this duality, an algorithm that performs disocclusion detection can be formulated and suitably modified to detect occlusions. B. Disocclusion Detection 1) Uncovered Regions: To enforce the disocclusion detection step, regions that will be uncovered in future frames need to be estimated. The object contour has already been predicted using the motion vectors calculated . New regions of . To estithe object might appear in the current frame is motion commate these regions, the current frame pensated using frame . find the uncovered regions Some postprocessing operations are applied on the mask to remove noise. 2) Region Classification: As already noted, uncovered regions do not correspond to actual disocclusions. We have used the color based criterion to estimate the uncovered regions. In the second stage of classification, motion is used as a criterion. The following is true for uncovered regions that belong to the object. • •
The uncovered region belonging to the object will exhibit motion characteristics similar to the object. Check the motion consistency of the uncovered regions with the rest of the object.
Average motion vector for the uncovered regions is estimated. The similarity test is then employed to classify the uncovered regions as actual disocclusions and false alarms.
856
IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 7, NO. 5, OCTOBER 2005
Fig. 4. Extracted foreman object.
Fig. 5. Extracted carphone object.
a) Motion vector clustering: Removing the covered reforms a new mask . The motion gions from vectors in are clustered using a K-means algorithm [4] described below. The covered regions are removed because the motion vectors are inaccurate in those regions, and hence might lead to errors during clustering. Having used a block-based approach for motion estimation, we have only one motion vector for a block. The vectors to be clustered are these block motion vectors. Clustering needs to . Hence, the be performed only on pixels in the mask number of pixels that lie in the mask for each block is calculated to give a weighted sample set for clustering
where represents the motion vector of the ith macrodenotes the number of pixels in the ith macro-block block, , and is the total number of macrothat lie inside blocks which are partially or completely inside . A clustering algorithm that chooses the number of clusters adaptively is used. b) Similarity test: The following similarity test is perdenote the formed for all the uncovered regions: Let uncovered region’s forward motion vector, and represent the centroid of the motion cluster of the object. For every calculate
Only if (where ), the region is treated as a disocclusion and included in . The updated object mask is given by . The uncovered regions that do not satisfy the above condition are false alarms. The algorithm for occlusion detection can be derived in a straightforward manner and hence is skipped here. IV. SIMULATIONS The tracking has been tested on some common MPEG test sequences and real video sequences. The proposed method for object tracking can be considered as a variation of the region based tracking techniques that have been reported in literature. The
approach predicts the object contour mostly using motion vectors and this has wide implications when a tracking algorithm needs to be applied for compressed video data. The computation time required for tracking of one frame has been compared with two other region based tracking approaches. The software written for all the algorithms have not been optimized to give better performance. A. Video Sequences In the foreman sequence, the object motion is not uniform and hence the tracking is slowed down when significant motion has been observed. Fig. 4 shows the extracted video objects obtained at various instances. The method generates object masks almost accurately as the segmentation-based approaches and at the same time it is not computationally very expensive. A comparison with other region-based methods is provided in the later section for reference. Fig. 5 shows the extracted object masks for the carphone sequence. B. Modulation Detection Scheme The modulation scheme described in Section II-D is used to skip frames when the motion of the object is relatively less. The object outlines for the skipped frames can be interpolated. Fig. 6 shows the number of frames skipped versus frame number for the foreman sequence and the bream sequence. In the bream sequence, the motion of the object is very less until the 110th frame. The modulation scheme detects high motion and slows down motion estimation. Slowing down the tracking process enables accurate tracking for a longer period. The Foreman sequence has high motion at various instances, as shown in the figure. C. Occlusion and Disocclusion Detection Fig. 7 shows the effect of the disocclusion detection mechanism detailed in the previous section. The tail of the fish reappears and this region is not similar in color to the body and hence tracking algorithms that consider color similarity for region association would fail. The disocclusion is detected in our method and associated with the corresponding object. The algorithm has also been applied to real life videos. The following
HARIHARAKRISHNAN AND SCHONFELD: FAST OBJECT TRACKING USING ADAPTIVE BLOCK MATCHING
Fig. 6.
857
Number of frames skipped versus frame number for the foreman and bream sequence.
Fig. 7. Disocclusion (tail of the fish) detected and associated with object.
Fig. 8. Occlusion and disocclusion detection for the Shirley sequence.
Fig. 9.
Tracking the person with occlusion/disocclusion detection.
figure (Fig. 8) illustrates the performance of the occlusion detection algorithm. Another person obstructs the tracked object partially and then moves away. Block matching algorithms rely on a translational model and are usually not suited for nonrigid objects. However, when tracking nonrigid objects, the occlusion and disocclusion detection algorithms include/discard pixels close to the boundary of the object. This handles nonrigid objects to some extent. D. Algorithm Comparisons The following sequence shows the performance comparison of the proposed algorithm with two region-based methods for
object extraction. A table of the computation times for the different methods is also given. The following methods are used for comparison. 1) Object Extraction using Partition Lattice Operators [6], [7]. 2) Region-based video coding using Mathematical Morphology [12], [13]. Fig. 9 shows the output of the algorithm with occlusion/disocclusion detection using both forward and backward motion. Figs. 10 and 11 show the results of the algorithms used for comparison. For this example the tracked object is better than the region-based methods. Avoiding forward motion speeds up the
858
IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 7, NO. 5, OCTOBER 2005
Fig. 10. Tracking the person with partition lattice operators.
Fig. 11.
Tracking the person with partition projection.
TABLE I COMPARISON OF COMPUTATION TIMES
Fig. 12.
algorithm but affects the quality of extraction. The Adaptive K-Means clustering, followed by the occlusion disocclusion classification are necessary for the successful tracking of the hand. The tracking algorithm designed was primarily for use in object-based video coding, where pixel accurate tracking of objects is not very critical. Thus the tracked object boundary is not accurate compared to the region-based methods. The following table (Table I) shows the computation times involved for the various implementations. All the algorithms were implemented in MATLAB. Hence the absolute timings do not make much sense. We also have a C implementation of the algorithm and it runs from 4–10 frames/s depending on the size of the object to be tracked. The speed can be increased by a factor of two by eliminating forward motion and using only backward motion. We are investigating this issue further. E. Results for Automatic and Manual Initialization In the following video sequence (Figs. 12 and 13), we compare the results for tracking a video for automatic and manual initializations. Automatic initialization works well for a static background. For backgrounds that are changing, a sophisticated method must be used for initialization. In Fig. 12 and 13, we show the tracking results for the hand sequence. For automatic initialization, we have also used a skin color model in conjunction with the algorithm mentioned in Section II. We actually
Tracking the hand with manual initialization.
Fig. 13. Tracking the hand with automatic initialization (both the hands are tracked).
would like to track only the drawing hand. Automatic initialization enables us to track both the hands. In this case, since we have a strong skin color model the initialization is almost perfect. In many other videos, using the trivial initialization algorithm mentioned in Section II yields inaccurate results. V. DISCUSSION In this paper, we have proposed a simple tracking algorithm that avoids segmentation except for initialization of the object partition during the initial frame. Object tracking using block motion vectors have seldom been exploited. Such an approach can be implemented using parallel processors and hence lends itself to a real-time implementation. The goal was to develop an algorithm that extracts video objects whose accuracy is close to the region-based approaches reported in the literature and at the same time perform tracking in a computationally efficient manner. Occlusion and disocclusion are viewed as reverse problems. An efficient algorithm for detecting occlusions is proposed and modified in accordance with the duality principle to
HARIHARAKRISHNAN AND SCHONFELD: FAST OBJECT TRACKING USING ADAPTIVE BLOCK MATCHING
develop a disocclusion algorithm. As the object mask is modified to take care of occlusions/disocclusions, the object can be tracked accurately for a longer time without requiring re-initialization/re-segmentation. The tracking algorithm proposed inherently can be extended in a straightforward manner to extract multiple objects. The tracking algorithm, however, performs poorly for objects that are relatively small and relevant changes are currently being investigated. The tracking obtained using the method can be used as a predicted position for methods that employ snakes to track contours. The prediction can be based on the computed affine model and the innovations process can be used to correct for errors in the prediction. This approach is similar to the Kalman filter based approaches. This approach is also being considered for future research. Motion estimation using the sum of absolute differences does not deal with gaussian noise and hence the motion vectors tend to be erroneous causing the tracked result to deteriorate. This is another aspect that requires study. REFERENCES [1] Y. Altunbasak and A. M. Tekalp, “Occlusion-adaptive, content-based mesh design and forward tracking,” IEEE Trans. Image Process., vol. 6, no. 9, pp. 1270–1280, Sep. 1997. [2] A. A. Amini, T. E. Weymouth, and R. C. Jain, “Using dynamic programming for solving variational problems in vision,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 12, no. 9, pp. 855–867, Sep. 1990. [3] M. J. Black and A. Jepson, “Eigentracking: robust matching and tracking of articulated objects using a view-based representation,” Int. J. Comput. Vis., vol. 26, no. 1, pp. 63–84, 1998. [4] B. Everitt, Cluster Analysis, 3rd ed. London, U.K.: Hodder, 1993. [5] Y. Fu, T. Erdem, and A. M. Tekalp, “Tracking visible boundary of objects using occlusion adaptive motion snake,” IEEE Trans. Image Process., vol. 9, no. 12, pp. 2051–2060, Dec. 2000. [6] D. Gatica-Perez, C. Gu, and M. T. Sun, “Semantic video object extraction using four-band watershed and partition lattice operators,” IEEE Trans. Circuits Syst. Video Technol., vol. 11, pp. 603–618, May 2001. [7] , “Multiview extensive partition operators for semantic video object extraction,” IEEE Trans. Circuits Syst. Video Technol., vol. 11, no. 7, pp. 788–801, Jul. 2001. [8] B. K. P. Horn and B. G. Schunck, “Determining optical flow,” Artif. Intell., vol. 17, pp. 185–203, 1981. [9] C.-L. Huang and C.-Y. Hsu, “A new motion compensation method for image sequence coding using hierarchical grid interpolation,” IEEE Trans. Circuits Syst. Video Technol., vol. 4, p. 4251, Feb. 1994. [10] M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: active contour models,” Int. J. Comput. Vis., vol. 1, pp. 321–331, 1987. [11] N. Peterfreund, “The velocity snake,” Proc. IEEE Nonrigid Articulated Motion, pp. 70–79, 1997. [12] P. Salembier and M. Pardas, “Hierarchical morphological segmentation for image sequence coding,” IEEE Trans. Image Process., vol. 3, no. 5, pp. 639–651, Sep. 1994. [13] P. Salembier, L. Torres, F. Meyer, and C. Gu, “Region-based video coding using mathematical morphology,” Proc. IEEE, vol. 83, no. 6, pp. 843–857, Jun. 1995.
859
[14] D. Schonfeld and D. Lelescu, “VORTEX: video retrieval and tracking from compressed multimedia databases multiple object tracking from MPEG-2 bitstream,” J. Vis. Commun. Image Represent. (Special Issue on Multimedia Database Management), vol. 11, pp. 154–182, 2000. [15] A. M. Tekalp, P. Van Beek, C. Toklu, and B. Gunsel, “Two-dimensional mesh-based visual object representation for interactive synthetic. Natural digital video,” Proc. IEEE, vol. 86, no. 5, pp. 1029–1051, Jun. 1998. [16] D. Terzopoulos and R. Szeliski, “Tracking with Kalman snakes,” in Active Vision, A. Blake and A. Yuille, Eds. Cambridge, MA: MIT Press, 1992, pp. 3–20. [17] J. Y. A. Wang and E. H. Adelson, “Representing moving images with layers,” IEEE Trans. Image Process., vol. 3, no. 5, pp. 625–638, Sep. 1994. [18] Y. Wang and O. Lee, “Active meshA feature seeking and tracking image sequence representation scheme,” IEEE Trans. Image Process., vol. 3, pp. 610–624, Sep. 1994.
Karthik Hariharakrishnan was born in TamilNadu, India, in 1979. He received the B.E. degree in electronics and instrumentation from the Birla Institute of Technology and Science, Pilani, India, and the M.S. degree in electrical and computer engineering from the University of Illinois at Chicago in 2003. In July 2004, he joined the DSP and Multimedia Group of Motorola, India. His current research interests are in multimedia compression and retrieval and signal, image, and video processing.
Dan Schonfeld (SM’05) was born in Westchester, PA, in 1964. He received the B.S. degree in electrical engineering and computer science from the University of California, Berkeley, and the M.S. and Ph.D. degrees in electrical and computer engineering from the Johns Hopkins University, Baltimore, MD, in 1986, 1988, and 1990, respectively. In August 1990, he joined the Department of Electrical Engineering and Computer Science, University of Illinois, Chicago, where he is currently an Associate Professor in the Departments of Electrical and Computer Engineering, Computer Science, and Bioengineering, and Co-Director of the Multimedia Communications Laboratory (MCL) and member of the Signal and Image Research Laboratory (SIRL). He has authored over 60 technical papers in various journals and conferences. He has served as a Consultant and Technical Standards Committee Member in the areas of multimedia compression, storage, retrieval, communications, and networks. He has previously served as President of Multimedia Systems Corporation and provided consulting and technical services to various corporations including AOL Time Warner, Chicago Merchantile Exchange, Dell Computer Corp., Getco Corp., EarthLink, Fish & Richardson, IBM, Jones Day, Latham & Watkins, Mirror Image Internet, Motorola, Multimedia Systems Corp., nCUBE, NeoMagic, Nixon & Vanderhye, PrairieComm, Teledyne Systems, Touchtunes Music, Xcelera, and 24/7 Media. His current research interests are in multimedia communication networks, multimedia compression, storage, and retrieval, signal, image, and video processing, image analysis and computer vision, and pattern recognition and medical imaging. Dr. Schonfeld served as an Associate Editor for the IEEE TRANSACTIONS ON IMAGE PROCESSING and the IEEE TRANSACTIONS ON SIGNAL PROCESSING. He was a member of the organizing committees of the IEEE International Conference on Image Processing and the IEEE Workshop on Nonlinear Signal and Image Processing. He was the plenary speaker at the INPT/ASME International Conference on Communications, Signals, and Systems.
