Arial www.huawei.com. Copyright©2010 Huawei Technologies Co., Ltd. All Rights Reserved. Inter-layer prediction mode based on base layer sharpness filter ...
Security Level:
Inter-layer prediction mode based on base layer sharpness filter www.huawei.com
Copyright©2010 Huawei Technologies Co., Ltd. All Rights Reserved.
Abstract •
Proposed contribution describes method of predicting higher resolution layer images from lower resolution layer images when scalable mode is used. Described algorithm based on sharpness filter applied to upsampling low resolution frame.
•
The main reason to use sharpening is to increase the amount of high frequency details in picture which was lost by downscaling.
•
The simulation results show that it achieves 1.7% and 1.0% BD rate savings on average for AI-2x and AI-1.5x, respectively, compared with anchors. The Class A test sequences show 2,9% BD rate saving. Encoding times are 113,3% and 111,2%, and decoding times are 117,7% and 116,0%.
Copyright©2010 Huawei Technologies Co., Ltd. All Rights Reserved.
2/n
Scalable video encoder flow chart Spatial enhancement layer
Temporal scalable coding
Prediction
SNR scalable coding Base layer coding
sharpening filter Multiplex downsampling filter
upsampling filter Spatial base layer
Temporal scalable coding
SNR scalable coding Prediction
Copyright©2010 Huawei Technologies Co., Ltd. All Rights Reserved.
Base layer coding
3/n
Sharpness filter flow chart Base Layer frame
Base Layer upscaled frame
ENCODER (RDO) / DECODER
BL prediction
Edge map obtained using Prewitt filter with gradient vector magnitude estimated using simplified formula: K*(|dx|+|dy|)-||dx|-|dy|| with clipping
Blurred edge map using filter (1/16,1/4,3/8,1/4,1/16)
BL Sharp prediction
Base Layer Sharpened frame
Copyright©2010 Huawei Technologies Co., Ltd. All Rights Reserved.
Warping pixels to the edges using bilinear interpolation and edge map as warping direction parameters
4/n
Sharpening process At the first step, the edge map is obtained by using Prewitt filter: 1 0 −1 𝑑𝑥 = 1 0 −1 ∗ 𝐼𝑚𝑔 1 0 −1 1 1 1 𝑑𝑦 = 0 0 0 ∗ 𝐼𝑚𝑔 −1 −1 −1 For each pixel the simplified gradient vector magnitude are estimated (slide 6 (top)): 𝑑 = 𝑐𝑙𝑖𝑝( 𝐾 ∗ 𝑑𝑥 + 𝑑𝑦 − 𝑑𝑥 − 𝑑𝑦 , 0, 𝑇𝑟 ) At the second step the blurring filter is applied to the edge map using filter each rows and each columns to smooth vector gradient map (slide 6 (bottom)).
1
1 3 1
, , , ,
1
16 4 8 4 16
to
At the last step the sharpening of upscaled frame if performed by warping source pixels to the edges with bilinear interpolation. The blurred edge map is used as parameters of warping: 𝑤𝑥 = 𝑆ℎ𝐷 ∗ 𝑑 𝑥 − 1, 𝑦 − 𝑑 𝑥 + 1, 𝑦 𝑤𝑦 = 𝑆ℎ𝐷 ∗ 𝑑 𝑥, 𝑦 − 1 − 𝑑 𝑥, 𝑦 + 1 Sharpened frame is obtained by using bilinear interpolation with displacement vector 𝑤𝑥, 𝑤𝑦 . 𝑆ℎ𝐼𝑚𝑔 𝑥, 𝑦 = 𝐼𝑚𝑔 𝑥 + 𝑤𝑥, 𝑦 + 𝑤𝑦
Copyright©2010 Huawei Technologies Co., Ltd. All Rights Reserved.
5/n
Contours before and after blurring
Copyright©2010 Huawei Technologies Co., Ltd. All Rights Reserved.
6/n
Warping vectors used for sharpening
Copyright©2010 Huawei Technologies Co., Ltd. All Rights Reserved.
7/n
Syntax elements sps_strong_intra_smoothing_enable_flag
u(1)
sps_sharpening_depth
ue(v)
sps_sharpening_depth_chroma
ue(v)
sps_sharpening_trunc
ue(v)
sps_sharpening_p0
ue(v)
vui_parameters_present_flag
u(1)
The non-zero value of sps_sharpening_depth signals the usage of sharpness filter. All tuning parameters can be replaced by several presets with adaptation of choosing or can be constants and only one flag of usage sharpness filter will be needed.
base_mode_flag
u(1)
If ( base_mode_flag ) sharpness_mode_flag
Copyright©2010 Huawei Technologies Co., Ltd. All Rights Reserved.
u(1)
8/n
Results Table 1. Experimental results on AI configuration
Class A Class B Overall (Test vs Ref) Overall (Test vs single layer) Overall (Ref vs single layer) EL only (Test vs Ref) Enc Time[%] Dec Time[%] BL Match
Y -2,9% -1,1% -1,7% 10,4% 12,3% -2,7%
AI HEVC 2x U -2,1% -0,6% -1,0% 12,0% 13,2% -1,9% 113,3% 117,7% Matched
V -2,3% -0,8% -1,3%
Y -1,0% -1,0%
11,4% 12,8% -2,2%
9,2% 10,3% -1,2%
AI HEVC 1.5x U -0,3% -0,3% 9,7% 10,1% -0,4% 111,2% 116,0% Matched
V -0,7% -0,7% 8,7% 9,5% -0,9%
Table 2. Experimental results on AI configuration in REF_IDX_FRAMEWORK AI HEVC 2x
AI HEVC 1.5x
Y
U
V
Y
U
V
Class A
-2.9%
-2.1%
-2.3%
Class B
-1.1%
-0.6%
-0.9%
-1.0%
-0.3%
-0.7%
Overall (Test vs Ref)
-1.6%
-1.0%
-1.3%
-1.0%
-0.3%
-0.7%
Overall (Test vs single layer)
10.8%
13.5%
12.7%
9.4%
9.4%
8.5%
Overall (Ref vs single layer)
12.6%
14.7%
14.2%
10.5%
9.8%
9.3%
EL only (Test vs Ref)
-2.7%
-2.0%
-2.3%
-1.2%
-0.4%
-0.9%
Enc Time[%]
114.8%
110.7%
Dec Time[%]
133.2%
124.5%
BL Match
Matched
Matched
Copyright©2010 Huawei Technologies Co., Ltd. All Rights Reserved.
9/n
Additional results Table 3. Experimental results on AI configuration with second approach of color sharpening AI HEVC 2x
AI HEVC 1.5x
Y
U
V
Y
U
V
Class A
-2,9%
-2,9%
-3,4%
Class B
-1,0%
-1,2%
-1,3%
-0,8%
-1,1%
-1,2%
Overall (Test vs Ref)
-1,6%
-1,7%
-1,9%
-0,8%
-1,1%
-1,2%
Overall (Test vs single layer)
10,5%
11,2%
10,7%
9,4%
8,8%
8,2%
Overall (Ref vs single layer)
12,3%
13,2%
12,8%
10,3%
10,1%
9,5%
EL only (Test vs Ref)
-2,5%
-2,7%
-2,9%
-0,7%
-1,2%
-1,3%
Enc Time[%]
113,3%
111,2%
BL Match
Matched
Matched
Copyright©2010 Huawei Technologies Co., Ltd. All Rights Reserved.
10 / n
Coding gain at lower bitrates (higher QP)
Copyright©2010 Huawei Technologies Co., Ltd. All Rights Reserved.
11 / n
Subjective comparison Left pictures without sharpening
Right pictures with sharpening
Copyright©2010 Huawei Technologies Co., Ltd. All Rights Reserved.
12 / n
Further work •
Tune sharpness filter parameters
•
Try different sharpness filters
•
Use the adaptation on sequence level, on picture level, on CU level
•
Reduce complexity
Copyright©2010 Huawei Technologies Co., Ltd. All Rights Reserved.
13 / n
Thank You www.huawei.com
