Stabilization. Yan Liu1, Mouyan Zou1,2 .... Authorized licensed use limited to: yan liu. .... [2] Liu Yan, Zou Mou-Yan, Video Stabilization Technique for Spatially.
P-1-5
Distortion Identification Technique in Video Stabilization Yan Liu1, Mouyan Zou1,2 Graduate University, Chinese Academy of Sciences, Beijing, China; 2 Institute of Electronics, Chinese Academy of Sciences, Beijing, China 1
Abstract — For non-stationary displacement sequence of the real-world image sequence, this paper presents a distortion identification technique based on Hilbert Huang Transform (HHT) to identify the distortion model and distortion frequency of the displacement sequence. Experiment results show that this technique can identify the distortion model and distortion frequency of the displacement sequence accurately and quickly. Based on these identification results, we can realize the video stabilization effectively. Index Terms— Image sequence distortion; Video Stabilization; Distortion model identification; Distortion frequency identification; Hilbert Huang Transform. 1.
INTRODUCTION
In the video stabilization systems, it is necessary to identify the distortion model and distortion frequency when we obtained the displacement series. Once the identification of distortion model and distortion frequency is realized, the digital filter for filtering high-frequency jitter and smoothing low-frequency translation can be designed. Then motion smoothing and motion compensation will be completed based on the motion filtering techniques mentioned in [1] [2]. However, the estimated displacement sequence is usually nonstationary because of unstable random camera motion. For those non-stationary sequences, Fourier analysis method can’t reflect the dynamic property of the distortion. Based on the estimated non-stationary displacement series, this paper realizes the identification of the distortion model and distortion frequency of the real-world image sequence based on Hilbert-Huang Transform (HHT) technique. Experimental results show that the proposed technique can identify the distortion model and distortion frequency of the displacement sequence accurately and quickly. On the basis of identified model and identified frequency, we can effectively realize the stabilization of the distorted video sequence by use of motion filtering techniques mentioned in [1] [2]. 2.
IDENTIFICATION TECHNIQUE BASED ON HHT METHOD
In order to make up the deficiency of the Fourier analysis in analyzing non-stationary signal, Huang et al. [3] proposed Empirical Mode Decomposition (EMD) method for adaptively decomposing signal. Based on EMD, they gave the HilbertHung Transform (HHT) method as a new time-frequency analysis method. The HHT measures the concept of instantaneous frequency of each of the signal components
from the EMD and presents the result as a time-frequency analysis in a Hilbert spectrum plot [3] [4]. We can analyze the dynamic property of the distortion frequency based on their instantaneous frequency and amplitude. Considering the validity and rationality of the HHT method used for analyzing non-stationary signal, we present a technique based on it to identify the distortion model and distortion frequency and discuss little about HHT method itself. As we known, EMD method always extracts the primary information firstly, i.e. several initial Intrinsic Mode Functions (IMFs) include the most obvious information of the original signal. The main object of analysis and identification of the dynamic property of object is to identify the distortion model and distortion frequency, which is a process to find the main frequency components of the displacement sequence. Therefore, according to the contribution of every frequency component in the whole spectrum, we can identify the main distortion characteristics of displacement sequences by use of Hilbert boundary spectral analysis method [3]. The frequency of energy concentrated reflects the main distortion characteristics of the signal. If the energy mainly exists in the high frequency, it means there are serious jitter distortions in the sequence; accordingly, there are slow low frequency translations in the sequence if the energy mainly locates in the low frequency. Thus, the simple identification of distortion model can be realized. In order to realize video stabilization, we need design a frequency filter based on the identification results of distortion frequency. Boundary spectrum with several peak values will bring some estimation error if we simply use the instantaneous frequency with maximum amplitude as cutoff frequency of the filter. It is well-known that the distortion frequency identification aims to find primary frequency component of the displacement sequence. We can identify the distortion frequency by analyzing the occurrence frequency of frequency components of every IMF. Based on HHT method, this paper proposes a simple distortion frequency identification method, which reduces the estimation error and realizes the identification correctly and easily. Assuming we have estimated a set of IMF components ck ,( k = 1,…, n) of the displacement sequence and its
corresponding Hilbert spectrum H k (ω , t ) , ( k = 1,… , n ) using
EMD method and Hilbert spectral analysis method separately. Then, statisticing the instantaneous frequency components ωk ( t ) of every IMF and its occurrence frequency. Next, choosing the frequency components f k ,( k = 1,…, n ) of every
978-1-4244-2559-4/09/$25.00 ©2009 IEEE Authorized licensed use limited to: yan liu. Downloaded on April 06,2010 at 22:33:53 EDT from IEEE Xplore. Restrictions apply.
Firstly, we use the simulation results in [1] as an example to explain the efficiency of HHT method used in the identification of distortion model and distortion frequency. In this experiment, the frequency of jitter distortion and shift distortion in x and y directions are initialized as follows: fjx = fjy = 6 Hz , fshx = fshy = 2 Hz .
Assuming we have estimated jitter displacement sequence and shift displacement sequence. Next, we use EMD method to discompose four displacement sequences to obtain their corresponding IMFs. Then, the Hilbert boundary spectrums of these four sequences are shown in Fig.1. Fig.1(a) and Fig.1(b) show that the energy mainly locates in high frequency, which explains that the estimated sequence contains high frequency jitter, i.e. it is accord with the characteristics of the jitter displacement field model. Similarly, in Fig.1(c) and Fig.1(d), we can see that the estimated sequence contains low frequency translation and is accord with the characteristics of the zeroorder displacement field model. Thus, we realized the identification of distortion model. It can be seen that the identification results are consistent with the simulation settings. 2
1.8
1.8
1.6
0.8
3.5
1 0.8 0.6
0.6
2
Amplitude
Amplitude
Amplitude
Amplitude
1
1.5
3 2.5 2 1.5
1
0.4
0.4
1 0.5
0.2
0.2 0
1
2
3
4 5 Frequency/ Hz
6
7
8
0
0
1
(a)
2
3
4 5 Frequency/ Hz
6
7
8
0 0
0.5 1
(b)
2
3
4 5 Frequency / Hz
6
7
8
(c)
0
0
1
2
3
4 5 Frequency / Hz
6
7
8
(d)
Fig.1. Hilbert boundary spectrums of the jitter displacement and translation displacement sequences (a) Jitter displacement sequence in horizontal direction (b) Jitter displacement sequence in vertical direction (c) Translation displacement sequence in horizontal direction (d) Translation displacement sequence in vertical direction Table 1 Identification results of the dynamic displacement field model Experimental sequence
2 1 0 -1 -2 -3 -4
Before stabilization After stabilization
2.5 2 1.5 1 0.5 0 -0.5 -1 -1.5
0
5
10
15
20
25 Frames
30
35
40
45
50
-2
0
5
10
15
(c)
20
25 Frames
30
35
40
45
50
(d)
Fig.2. Stabilization results of the real-world sequence (a) Original image sequence (b) Stabilized image sequence (c) Displacement sequence in horizontal direction (d) Displacement sequence in vertical direction
5
2.5
1.2
1.2
(b) 3
Before stabilization After stabilization
3
4.5
3
1.4
1.4
0
3.5
4
1.6
(a) 4
Displacement sequence in vertical direction
3. EXPERIMENTAL RESULTS
technique mentioned above. Compared to identification results by Hilbert boundary spectrum, we can see that the identification results using the proposed method are closer to the simulation setting frequency (see Table1). Next, we use the proposed identification method to identify the distortion frequency of real-world video sequence of the car (see Fig.2 (a)) and use it as the cutoff frequency of the motion filter. A few seconds of the stabilization results in AVI format are available in the following link: http://www.liveshare.com/files/329259/Identification__results.rar.html. From the displacement curve of the displacement sequence before and after stabilization (see Fig.2(c) and Fig.2(d)), we can see that high frequency jitter in vertical direction is preferably suppressed. In the video results, it can be seen clearly that jitter distortion caused by unstable camera motion is greatly eliminated.
Displacement sequence in horizontal direction
IMF, which has the biggest occurrence frequency. Finally, accumulating every frequency component and averaging the n sum to obtain the estimated distortion frequency fˆ = 1 f . ∑ k n k =1 Based on the identified distortion model and estimated distortion frequency, video stabilization will be realized using some corresponding stabilization methods presented in [1] and [2].
Identification results by Hilbert boundary spectrum (Hz)
Identification results by proposed method (Hz)
x
y
x
y
Jitter displacement sequence
6.4941
6.5981
6.1961
6.3605
Translation displacement sequence
1.8301
1.6499
2.1336
1.8135
After statisticing instantaneous frequency components of every IMF and their occurrence frequency, we can obtain the estimated distortion frequency by use of the identification
4. CONCLUSION Considering the non-stationary of the real-world image sequence, this paper proposed an identification technique based on HHT method to identify the distortion model and distortion frequency. Experimental results validate the efficiency of this technique that used in distortion identification and show that it can identify the distortion model and distortion frequency of the real-world sequence correctly and quickly. Based on the identification results, we can efficiently stabilize the distorted video sequence. REFERENCES [1] Yan Liu, Mouyan Zou, et al, Dynamic displacement field model used as a new camera motion model in video stabilization, Proc.of the ICCE, Las Vegas, Jun. 2007, pp.1-2. [2] Liu Yan, Zou Mou-Yan, Video Stabilization Technique for Spatially Variable Distorted Image Sequence Based on Dynamic Displacement Field Model, Journal of the Graduate School of the Chinese Academy of Sciences, 2008, 25(1), pp. 117-122. (in Chinese) [3] Huang.N.E, et al, The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis, J.Proc.R.Soc.Lond.A, 1998, 454, pp.903-995. [4] Anna Linderhed, Variable Sampling of the Empirical Mode Decomposition of Two-Dimensional Signals, International Journal of Wavelets, Multiresolution and Information Processing (IJWMIP) , 2005, 3(3), pp. 435-452.
Authorized licensed use limited to: yan liu. Downloaded on April 06,2010 at 22:33:53 EDT from IEEE Xplore. Restrictions apply.
