A New Auto-Focus Method based on Focal Window Searching and Tracking Approach for Digital Camera Tsung-Han Tsai
Chung-Yuan Lin
Department of electrical engineering National central university, Taiwan, R.O.C
[email protected]
Department of electrical engineering National central university, Taiwan, R.O.C.
[email protected]
Abstract—This paper presents a new method for auto-focus digital camera system. Our approach differs from other common auto-focus approaches, since a focal window will be searched and tracked before applying auto-focus algorithm to move the lens toward the proper position. The advantages are increasing the focus reliability and flexibility than common auto-focus approaches. Our approach consists of two major stages, motion detection and focal window searching and tracking. Motion detection firstly detects the region-of-interest (ROI) in image, then focal window searching and tracking stage selects a proper focal window among detected ROIs, and tracks this window through the consecutive image. The performance is tested by a high motion and a low motion sequences. Focus values derived from the selected focal window are shown to demonstrate that our algorithm can achieve auto-focus in both high motion and low motion conditions.
I. INTRODUCTION In the past years, digital camera has been playing a more important role in the consumer electronics market. During the recent years, the market of digital camera has been obtaining large growth per year in the world. The low cost, high definition and compact digital camera will be the focus in consumer market. Auto-focus (AF) is a basic function in mega-pixel digital camera. Conventional camera have used pan focus or manual focusing techniques in which users would adjust the location of lens with their own hand for producing a focused image. The pan focus technique allows for all scenes, background and foreground, to be focused instantly for a snap-shot. But these skills are limiting the image quality since it is hard to focus on the object precisely. There have several auto-focus techniques reported in [I][3]. The most basic method is to calculate the focus value and derive the hest-focused lens position by climbing search method. Because auto-focus algorithm must be real-time, the traditional auto-focus algorithm will have some problems due to computation expanding with pixel number increasing. An obvious problem is that convergence speed of auto-focus is slowed down due to more computation. Additionally, the probability of defocus may increase because of the tradeoff
between the computation and the reliability. Sub-sampling method seems to be usually employed to reduce the computation especially in high definition camera. However, in the sub-sampling method, some detailed information is lost and noise floor increases. As a result, the lens deviates from the best-focused position. To handle this problem, subwindow method [2] was proposed which can keep more detailed information. However, when sub-window becomes small, the focus reliability decreases due to losing information outside the windows. To solve above problems, we develop a focal window searching and tracking approach to improve the reliability. Our proposed approach employs motion detection in finding the region-of-interest (ROI) in image, if more than one ROIs are detected, focal window searching algorithm automatically decides which ROI should be focused based on their position related to camera. After the ROI has been detected, a dynamic window is constructed and tracked through the consecutive image. A dynamic window technique will has higher focus reliability than constant window. Besides, our proposed focal window searching and tracking algorithm can combine with various auto-focus algorithms to construct high reliability auto-focus digital camera. This paper is organized as follow. First, the auto-focus digital camera system is described in Section II. In, Section III, our proposed focal window searching and tracking is described in detail. The experiment results will be given in Section IV. Conclusion is in Section V. II. AUTO-FOCUS CAMERA SYSTEM Basically, auto-focus camera module consists of autofocus lens, motor, image sensor, auto-focus lens driver, and image signal processor. Fig. 1 shows our proposed configuration for auto-focus operation in digital camera. Optical signals passing through AF lens are reaching image sensor, which transforms optical signal to electrical. Motion detection detects ROIs by analyzing the consecutive image converted by A/D, then focal window searching and tracking module derives a window to be focused from the detected
ROIs and tracks the window through the consecutive image. Focus values, frequency components, are generated by focus lens driver and AF algorithm finds the state of the focused image in image signal processor. Auto-focus algorithm decides the proper lens position on the basis of the focus values and controls lens driver to move AF lens at relative position.
intensity distributions of in-focus and out-focus image are also shown in Fig. 2., more smooth character of the outfocus image can be seen by comparing the 3-D intensity distribution. To overcome this problem, a higher order statistical (HOS) [6] test is employed.
Figure 1. The proposed auto-focus camera system with focal window searching and trackig functionality.
Figure 2. 3-D intensity distribution figure with different focus degree.
Because the auto-focusing technique is an essential function in camera, various auto-focusing methods have been developed for a long time. Climbing search algorithm, which has been developed for fast searching algorithm can be adopted as auto-focus algorithm. Climbing search algorithm has been developed for fast searching, which is described as MCS (mountain climb servo) in [1] or HCS (hill-climbing search) in [2]. Climbing search algorithm is split to two different searching stages in order to obtain fast convergence speed. Generally, in the first searching stage, a large step size is used for lens moving, which is always a constant. When a mountain peak is found, it enters into the second searching stage, and the smallest step size is used for lens moving toward the best focus position. Here the first stage searching can be defined as out-focus searching, as well as the second stage searching can be defined as in-focus searching. III. SEARCHING AND TRACKING FOCAL REGION The main goal of this section is to search and track the focal window until the AF lens moving to the best focal position. The inputted signal is the consecutive image converted by A/D. Two main stages, motion detection and focal window searching and tracking, of our proposed algorithm are described in the follow. A. Motion detection There were many motion detection methods proposed [4],[5]. These algorithms can work well on detecting moving region from video sequence. However, considering the outfocus image shown in Fig. 2., a general motion detection almost cannot detect moving region well since the interior region of the object becoming extremely smooth. Both 3-D
The HOS is a statistical method based on a sample estimate of the block I that is an average value of a region. i.e. Ik =
1 M ´N
M
N
å å d [m , n ], m =1 n =1
k
k = 1, 2 ,3,...
where, dK[m,n] is difference image in the k-th consecutive image at location (m,n), and M ´ N is the dimension of block size. The statistical value H, which presents motion features is tested by HOS, i.e.
H(I1, I 2 , I 3 , I 4 ) = I 4 + I3 - 3I1 (I 2 - I12 ) - 3I 22 - I13 + 2I14 then, the statistical value H is transformed to a binary image as the motion detection resulted image T, the formula is described as follow ìT ( x, y ) = 1 í îT ( x , y ) = 0
if H ( I 1 , I 2 , I 3 , I 4 ) if H ( I 1 , I 2 , I 3 , I 4 )
> Threshold £ Threshold
where 1 denotes as moving block and 0 denotes as stationary block. In our scheme, threshold is defined within 5~8, and block size is set as M=8, N=8. However, the motion detection resulted image T may still has some error detection, due to the factors such as moving background, serious thermal noise of image-capturing device and the change of light source. Therefore, we propose motion regularization for motion detection resulted image refinement. Our proposed motion regularity for moving and stationary block refinement uses a regularity window shown in Fig. 3. Different coefficient is set to each position to obtain the distance relationship within the window. Motion regularization firstly considers all stationary blocks as central block using the regularity window, if the neighbor block
belongs to moving block, then the coefficient is accumulative. After examining all neighbor blocks, the stationary block is changed to moving block, if the accumulative value exceeds 6. Then, all moving blocks is considering as central block using the same window, if the neighbor block belongs to stationary block, then the coefficient is accumulative. After examining all neighbor blocks, the moving block is changed to stationary block if the accumulative value exceeds 8. All blocks of the motion detection resulted image will become more reliability after motion regularity process.
1
2
1
2
0
2
1
2
1
Figure 3. The regularity window with coefficients using in Motion regularization.
B. Focal window searching and tracking We apply the connected component analysis to label motion regularity resulted image before applying focal window searching and tracking. Each connected block will be assign an united number n as the ROI’s label. We then determine a rectangular window for each ROI. Notice that the rectangular window must completely mark the ROI. A central point Pn is also be determined for representation n-th ROI. The objective of focal window searching and tracking stage is to generate a window to be focused and track this window until the lens moving to the best focal position. Two problems need to be considered here. One is to choose a ROI from ROIs to generate a focal window. The other is to track this window through consecutive image. Our key idea is that ROI locating more closely to the camera will has higher priority to be chosen, then, the window of the chosen ROI will be applied as focal window and track this focal window by comparing the Pn of each consecutive image. Fig. 3. shows a 3-D camera relationship [7], where used the coordinate system (O, i, J, K). O coincides with the lens, vectors i and J form a basis for a vector plane parallel to the image plane Π’ and have a distance f from O. A line passing through the image center C’ is called optical canter. Let P denote a scene point with coordinate (x, y, z) and P’ denotes its image with coordinates (x’, y’, z’). There exits a scaleλso that (x’ ,y’, z’) = λ(x, y, z). Since P’ lies in the image plane, we have z’ = f. Based on the reason, three points P, O and P’ are collinear, thus. Thus we have OP’ = λOP, where x’ = f * x/z and y’ = f * y/z. Due to the collinear relationship of P, O and P’, we can determine distances of each object to the camera by denoting a point CPC, which implies the mapped camera position in
image plane Π’. The CPC locates at the bottom boundary of the image across the line CP. We then choose n-th ROI as the focal object and assign its window as the focal window by following equation
n = arg min( Px - PCP ) xÎm
where m represents the amount of ROI. A gradient operator that extracts the high frequency components out of image as focus value is applied to the selected focal window. Then the auto-focus algorithm can be applied to achieve auto-focus by using focus values to decide the proper lens position. In this paper, we adopt the gradient v operator, which denotes x = {-1,0,1} for x direction and
v y = {-1,0,1}T for y direction as the focus value generator. J
CP P’
f K * O
C’ Π’
i
* P
Figure 4. Scene plane is projected on image plane via camera.
IV. EXPERIMENTAL RESULTS We construct many shot of video sequences with different focus degree under the same view as our test sequence. Fig 5. shows the motion detection results of high motion sequences with different focus degree, and Fig 6. shows the motion detection results of low motion sequences with different focus degree. Our proposed motion detection algorithm can work well even applying to the most blur outfocus sequence under both high motion and low motion conditions. Our proposed focal window searching and tracking algorithm is also applies to both high motion and low motion sequences. The focal window searching only works under multi-ROI situation. Fig. 7 shows that the focal window searching results of the high motion sequence, where white rectangular windows represent the selected window by Focal window searching and tracking algorithm. Finally, focus values derived from focal window by gradient operator are shown in Fig 8, where four different focus degree sequences are shown in upper row and its respective focus values are shown in bottom row. One can observe that average focal value largely increases in the first two blur sequences, and average focal value slowly increase in the last two clear sequences. This phenomenon demonstrates that we can achieve auto-focus by applying the focal value for auto-focus algorithm.
Figure 5. Motion detection results of high motion sequences.
Figure 6. Motion detection results of low motion sequences.
Figure 8. Focal value hsitogram with respective image. Focal values are derived by gradient operator from the focal window
algorithm can achieve auto-focus in both high motion and low motion conditions.
REFERENCES [1] Figure 7. Focal window searching results under multi-ROI situation.
V. CONCLUSION This paper presents a new approach for auto-focus digital camera system. This approach consists of two major stages, motion detection and focal window searching and tracking. Motion detection firstly detects ROIs by HOS test, then, motion regularity is applied for motion detection result refinement. In the second stage, the focal window searching and tracking algorithm is applied to selection of a proper focal window among detected ROIs, and tracks this window through the consecutive image. The experimental results show that our proposed algorithm can detect ROI in both infocus and out-focus image, and tracks the selected window through consecutive image well. Final, focus value within the selected window is also shown to demonstrate that our
[2]
[3]
[4] [5]
[6]
[7]
Kazushige Ooi etc. “An advanced autofocus system for video camera usine ouasi condition reasonid.” IEEE Trans. on Consumer Electronics, vol.36, no.3, pp526-530, Aug. 1990. K. S. Chai, J. S. Lee, S. 1. KO, “New autofocus technique using the freauencv selective weighted median filter for video cameras”. IEEE Trans. on Consumer Electronics, vol.45, no.3, pp.820-827, Aug. 1999 J. H.’Lee, K. S. Kim, B. D. Nam, “Implementation of a passive automatic focusing algorithm for digital Still camera”, IEEE Trans. on Consumer Electronics, vol.41, no.3, pp.449-454, Aug. 1995. T. Aach and A. Kaup, “Statistical model-based change detection in moving video,” Signal Processing., vol. 31, pp. 165–180, 1993. A. Smolic´, T. Sikora, and J.-R. Ohm, “Long-term global motion estimation and its application for sprite coding, content description and segmentation,” IEEE Trans. Circuits Syst. Video Technol., vol. 9, pp. 1227–1242, Dec. 1999. T.-H. Tsai and C.-Y. Lin, “Video Segmentation using Multiscale Feature Extraction and Fusion,” IEEE International Symposium on Circuits and Systems, May 2005. David A. Forsyth, Jean Ponce, “Computer Vision: A Modern Approach,” Prentice Hall 2002
