Low Complexity Adaptive Intra-Refresh Rate for Real ...

/RZ&RPSOH[LW\$GDSWLYH,QWUD5HIUHVK5DWHIRU 5HDO7LPH:LUHOHVV9LGHR7UDQVPLVVLRQ $QJHOR5GHOD&UX]

5KDQGOH\'&DMRWH

(OHFWURQLFV(QJLQHHULQJ'HSDUWPHQW )DFXOW\RI(QJLQHHULQJ 8QLYHUVLW\RI6DQWR7RPDV0DQLOD3KLOLSSLQHV DUGHODFUX]#PQOXVWHGXSK

(OHFWULFDODQG(OHFWURQLFV(QJLQHHULQJ,QVWLWXWH 8QLYHUVLW\RIWKH3KLOLSSLQHV'LOLPDQ&DPSXV 4XH]RQ&LW\3KLOLSSLQHV UKDQGOH\#HHHXSGHGXSK  

Abstract— Real-time wireless video transmission systems must consider both error resiliency and low complexity. However, most error resilient features of recent video coding standards tend to increase computational complexity of the encoder. In this paper, we propose a low complexity error resilient joint source-channel adaptive intra-refresh rate scheme where the optimum number of intra-coded macroblocks is determined at frame level based on the minimum estimated endto-end distortion. In this work, we propose source and transmission distortion models whose parameters are independent on sequence type which allows real-time video encoding. The source distortion model is based on residual information and quantization step using linear least square method. The residual information is estimated using the meanabsolute difference (MAD) prediction model based on the linear relationship between intra-refresh rate and MAD. The transmission distortion model is based on recursive model using reliable feedback channel. The proposed models are used to implement a joint source-channel video coding scheme using standard H.264/AVC encoder. Accurate estimate of the actual distortion at various refresh rates are achieved and able to estimate the distortion before encoding the frame. The proposed scheme is compared with random and periodic intra refresh schemes under wireless fading channel. Improvements in PSNR quality are measured which verifies the effectiveness of the proposed scheme especially in time varying channel conditions. Keywords—intra-refresh, distortion, error-control.

,

source

distortion,

transmission

,1752'8&7,21

0RVW RI HUURU UHVLOLHQW IHDWXUHV RI UHFHQW YLGHR FRGLQJ VWDQGDUGV WHQG WR LQFUHDVH WKH FRPSXWDWLRQDO FRPSOH[LW\ RI YLGHRHQFRGHU>@+RZHYHUWKHGHVLJQRIUHDOWLPHZLUHOHVV YLGHR WUDQVPLVVLRQ UHTXLUHV ERWK HUURU UREXVWQHVV DQG ORZ FRPSOH[LW\ (UURU FRQWURO WRROV DUH RQO\ XVHIXO GXULQJ XQUHOLDEOH FKDQQHO FRQGLWLRQ DQG EHFRPH XQQHFHVVDU\ GXULQJ JRRG FKDQQHO FRQGLWLRQ 7KXV LQ WLPH YDU\LQJ FKDQQHO LW LV LPSRUWDQW WR DGMXVW WKH HUURU FRQWURO SDUDPHWHUV EDVHG RQ WKH HVWLPDWHG FKDQQHO FRQGLWLRQ $PRQJ WKH YDULRXV HUURU FRQWURO VFKHPHV RI YLGHR HQFRGHU >@ LQWUDUHIUHVK ,5  VFKHPHV DUH VWLOO DWWUDFWLYH EHFDXVH RI LWV ORZ FRPSOH[LW\ LPSOHPHQWDWLRQV DQG LWV HIILFLHQF\ LQ UHGXFLQJ WUDQVPLVVLRQ GLVWRUWLRQ ,Q WKLV ZRUN ZH SURSRVH D ORZ FRPSOH[ DGDSWLYH LQWUDUHIUHVK UDWH VFKHPH IRU ZLUHOHVV YLGHR WUDQVPLVVLRQ ZKLFK RSWLPL]HV WKH UHTXLUHGQXPEHURILQWUDFRGHGPDFUREORFNV0%V IRUDJLYHQ IUDPH EDVHG RQ WKH HVWLPDWHG FKDQQHO FRQGLWLRQ ,Q OLQH ZLWK

‹$36,3$

WKLV ZH HPSOR\ WKH VRXUFH GLVWRUWLRQ PRGHO DQG LQWUDUHIUHVK UDWH SUHGLFWLRQ PRGHO LQ >@ DQG SURSRVHG D UHFXUVLYH WUDQVPLVVLRQ GLVWRUWLRQ PRGHO EDVHG RQ D UHOLDEOH IHHGEDFN FKDQQHO 7KH HVWLPDWHG HQGWRHQG GLVWRUWLRQ LV XVHG WR VHOHFW WKHEHVWLQWUDUHIUHVKUDWHIRUDJLYHQIUDPH ,Q VHFWLRQ ,, WKH TXDQWL]DWLRQ GLVWRUWLRQ PRGHO EDVHG RQ UHVLGXDO LQIRUPDWLRQ TXDQWL]DWLRQ SDUDPHWHU DQG LQWUDUHIUHVK UDWH LV SUHVHQWHG DQG GLVFXVVHG ,Q VHFWLRQ ,,, WKH SURSRVHG UHFXUVLYHWUDQVPLVVLRQGLVWRUWLRQPRGHOLVSUHVHQWHG,QVHFWLRQ ,9WKHSURSRVHGIUDPHZRUNDQGIUDPHOD\HUUDWHFRQWUROZLOO EHGLVFXVVHG)LQDOO\VHFWLRQ9GLVFXVVHVWKHH[SHULPHQWDOVHW XSDQGUHVXOWVRIXVLQJWKHSURSRVHGIUDPHZRUNXQGHUZLUHOHVV IDGLQJ FKDQQHO 2EMHFWLYH TXDOLW\ XVLQJ 3615 PHWULF LV PHDVXUHG DQG FRPSDUHG ZLWK LQWUDUHIUHVK VFKHPHV RI -0>@  ,,

6285&(',67257,21$1$/@ ,W LV ZRUWK QRWLFLQJ WKDW QRQH RI WKH H[LVWLQJ VRXUFH GLVWRUWLRQ PRGHOV H[SORLW WKH UHVLGXDO LQIRUPDWLRQ WR HVWLPDWH TXDQWL]DWLRQ GLVWRUWLRQ ,Q +$9& WKH UHVLGXDO LQIRUPDWLRQ PHDVXUHV WKH GLIIHUHQFH LQ PHDQDEVROXWHGLIIHUHQFH 0$'  EHWZHHQ WKHRULJLQDOIUDPHDQGPRWLRQFRPSHQVDWHGSUHGLFWHGIUDPH,W GLFWDWHVWKHUHTXLUHGTXDQWL]DWLRQSDUDPHWHU43 IRUHQFRGLQJ WKHIUDPHZKLFKJUHDWO\DIIHFWVWKHVRXUFHGLVWRUWLRQ1RWHWKDW KLJKHU0$'UHVXOWVWRKLJKHU43ZKLFKUHVXOWVWRDQLQFUHDVH LQ VRXUFH GLVWRUWLRQ 7KXV WKHUH H[LVW D VWURQJ UHODWLRQVKLS DPRQJWKHVRXUFHGLVWRUWLRQUHVLGXDOLQIRUPDWLRQLQ0$'DQG 43 5HVXOWV LQ >@ VKRZ WKDW D OLQHDU UHODWLRQVKLS EHWZHHQ VRXUFHGLVWRUWLRQDQGWKHSURGXFWRI0$'DQG430$'[43  H[LVWDVVKRZQLQ)LJXUH %\XVLQJOLQHDUOHDVWVTXDUHPHWKRGZHFDQREWDLQDOLQHDU PRGHO WKDW HVWLPDWHV VRXUFH GLVWRUWLRQ DV IXQFWLRQV RI 0$' DQG43DVVKRZQLQ 7KHPRGHOSDUDPHWHUV αDQG αDUH GHWHUPLQHG DQG XSGDWHG IRU D JLYHQ ZLQGRZ OHQJWK L XVLQJ LQIRUPDWLRQRISUHYLRXVO\HQFRGHGIUDPHVDVVKRZQLQ DQG  xUHSUHVHQWVSUHYLRXV MADxQPYDOXHVRI ZLQGRZ OHQJWK

$36,3$

2.5

x 10

5

HVWLPDWHGEDVHGRQWKHUHVLGXDOLQIRUPDWLRQ0$'DQGWH[WXUH ELWVWKXVLWLVLQWXLWLYHWKDWWKHDFWXDO0$'RIDJLYHQIUDPH VKRXOGDOVRLQFUHDVH5HVXOWVLQ>@VKRZWKDW0$'LVOLQHDUO\ LQFUHDVLQJZLWKLQWUDUHIUHVKUDWHDVGHSLFWHGLQ)LJXUH7KXV ZHFDQHVWLPDWH0$'RIFXUUHQWIUDPHIRUDQ\JLYHQUHIUHVK UDWHDVVKRZQLQ  ‫ܦܣܯ‬௡ ሺȞ௡ିଵ ሻLVWKHSUHGLFWHG0$'IURP SUHYLRXVIUDPHZKLOH ‫ܦܣܯ‬௡ ሺͳሻLVWKHFRPSXWHG0$'ZLWKDOO 0%V LQWUDFRGHG 7KH SURSRVHG PRGHOV LQ   DQG   DUH FRPELQHG WR HVWLPDWH WKH VRXUFH GLVWRUWLRQ DV D IXQFWLRQ RI LQWUDUHIUHVKUDWH 20

32kbps 48kbps 64kbps 96kbps 128kbps

15

MAD

L ZKLOH y UHSUHVHQWV DFWXDO VRXUFH GLVWRUWLRQ RI SUHYLRXVO\ HQFRGHGIUDPHVRIZLQGRZOHQJWKL  ‫ܦ‬ொ ሺ݊ሻ ൌ ߙଵ ‫ܦܣܯ‬௡ ܳܲ௡ ൅ ߙଶ ሺͳሻ  ‫ ܮ‬σ ‫ ݕݔ‬െ σ ‫ ݔ‬σ ‫ݕ‬ ሺʹሻ Dଵ ൌ ‫ ܮ‬σ ‫ ݔ‬ଶ െ ሺσ ‫ݔ‬ሻଶ  σ ‫ ݕ‬σ ‫ ݔ‬ଶ െ σ ‫ ݔ‬σ ‫ݕݔ‬ ሺ͵ሻ Dଶ ൌ ‫ ܮ‬σ ‫ ݔ‬ଶ െ ሺσ ‫ݔ‬ሻଶ  1RWHWKDW43DQG0$'DUHFRPSXWHGZLWKRXWHQFRGLQJWKH IUDPHZKLFKDOORZVXVWRHVWLPDWHWKHVRXUFHGLVWRUWLRQEHIRUH HQFRGLQJWKHFXUUHQWIUDPH)LJXUHVKRZVDVDPSOHDFFXUDF\ RI WKH SURSRVHG PRGHO XVLQJ PRWKHUGDXJKWHU VHTXHQFH DW NESV 

10

Source Distortion (SSE)

2

1.5

5 0

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Refresh Rate 1

0.5

0 10

20

30

40

50

60

70

80

90

MADxQP

 

0.1

)LJXUH5HODWLRQVKLSEHWZHHQ6RXUFH'LVWRUWLRQDQG0$'[43 IRUPRWKHUGDXJKWHU6HTXHQFHDWNESV

,,,

5

2.5



 )LJXUH5HODWLRQVKLSEHWZHHQ0$'DQG5HIUHVK5DWHIRU  FRDVWJXDUG6HTXHQFHDW)UDPH   ‫ܦܣܯ‬௡ ሺͳሻ െ ‫ܦܣܯ‬௡ ሺȞ௡ିଵ ሻ ሺȞ െ Ȟ௡ିଵ ሻ ‫ܦܣܯ‬௡ ሺȞሻ ൌ ͳ െ Ȟ௡ିଵ ൅ ‫ܦܣܯ‬௡ ሺȞ௡ିଵ ሻሺͶሻ

x 10

Actual Estimate



75$160,66,21',67257,21$1$/@ DV GHSLFWHG LQ   DVVXPHV WKH DYDLODELOLW\ RI WKH PRGHO SDUDPHWHUV α DQG βEHIRUH HQFRGLQJ ZKLFK PDNHV WKH PRGHO 1.5 VHTXHQFH GHSHQGHQW ,W LV DOVR DVVXPHG WKDW WKH PRGHO SDUDPHWHUV IRU HDFK W\SH RI YLGHR VHTXHQFH DUH FRQVWDQW 1 7KHVH OLPLWDWLRQV GR QRW DOORZ WKH WUDQVPLWWHU WR HQFRGH WKH VHTXHQFHRIUHDOWLPHYLGHRLQIRUPDWLRQEHFDXVHLWUHTXLUHVa priori LQIRUPDWLRQ DERXW WKH WHVW VHTXHQFH WKDW ZLOO EH 0.5 HQFRGHG ,Q WKLV ZRUN WKH WUDQVPLVVLRQ VWDWXV RI SUHYLRXVO\ WUDQVPLWWHGYLGHRSDFNHWVLVWUDQVPLWWHGIURPWKHUHFHLYHUVLGH 0 0 50 100 150 Frame Index XVLQJDUHOLDEOHIHHGEDFNFKDQQHO7KXVWKHDFWXDOHQGWRHQG  GLVWRUWLRQRISUHYLRXVO\HQFRGHGIUDPHVDUHHDVLO\FRPSXWHGDW )LJXUH(VWLPDWHG4XDQWL]DWLRQ'LVWRUWLRQIRUPRWKHU  GDXJKWHU6HTXHQFHDWNESV WKH HQFRGHU VLGH 7KLV SUHYLRXV LQIRUPDWLRQ DOORZV XV WR  FRPSXWH WKH PRGHO SDUDPHWHUV UHFXUVLYHO\ DV VKRZQ LQ   ZKHUHpLVWKHSDFNHWORVVSUREDELOLW\DQGFDLVGLIIHUHQFHRI A. Intra-Refresh Rate WZRFRQVHFXWLYHIUDPHVLQ66( ,QWKLVZRUNZHDVVXPHWKDW *LYHQ D FRQVWDQW ELWUDWH LQFUHDVLQJ WKH QXPEHU RI LQWUD WKH VRXUFH GLVWRUWLRQ DQ DQG WUDQVPLVVLRQ GLVWRUWLRQ DC DUH FRGHG 0%V WHQGV WR LQFUHDVH VRXUFH GLVWRUWLRQ EHFDXVH RI XQFRUUHODWHGDVGHSLFWHGLQ DQLVHVWLPDWHGXVLQJ ZKLOH KLJKHU 43 UHTXLUHG WR HQFRGH WKH IUDPH 1RWH WKDW 43 LV

DC LV HVWLPDWHG XVLQJ   1RWH WKDW WKH DFWXDO GLVWRUWLRQ RI SUHYLRXVO\ WUDQVPLWWHG IUDPHV DUH DYDLODEOH DW WKH HQFRGHU VLGH )LJXUH  VKRZV WKH YDOLGLW\ DQG DFFXUDF\ RI WKH WUDQVPLVVLRQGLVWRUWLRQPRGHO  ෡ொ ൅ ‫ܦ‬ ෡஼ ሺͷሻ ෡௡ ൌ ‫ܦ‬ ‫ܦ‬ ௡ ௡  ෡஼ ൌ ‫ܦ‬஼ ሾ‫ ݌‬൅ ሺͳ െ ‫݌‬ሻߚሺͳ െ Ȟሻሿ ൅ ‫ܦܨߙ݌‬௡ ሺ͸ሻ ‫ܦ‬ ௡ ௡ିଵ



‫ܦܨ‬௡ ൌ ‫ܧ‬ሾሺ݂௡ െ ݂௡ିଵ ሻଶ ሿǡሺ͹ሻ



 ሺͳ െ ‫݌‬ሻሺͳ െ Ȟ௡ିଵ ሻ‫ܦ‬஼ ௡ିଶ ߚ ቂ ቃൌቈ ሺͳ െ ‫݌‬ሻሺͳ െ Ȟ௡ିଶ ሻ‫ܦ‬஼ ௡ିଷ ߙ

‫ܦܨ݌‬௡ିଵ ቉ ‫ܦܨ݌‬௡ିଶ

ିଵ



‫ܦ‬஼ െ ‫ܦ݌‬஼ ௡ିଶ ൤ ௡ିଵ ൨ሺͺሻ ‫ܦ‬஼ ௡ିଶ െ ‫ܦ݌‬஼ ௡ିଷ

9

6

2.5

x 10

End-to-End Distortion (SSE)

2

Actual Estimate

1.5

1

0.5

0 0

50

100

150

Frame Index

)LJ(VWLPDWHG(QGWR(QG'LVWRUWLRQIRUPRWKHUGDXJKWHU  6HTXHQFHDW615 G%





,9

/2:&203/(;,7@ % .DW] 6 *UHHQEHUJ 1 @ 7 6WRFNKDPPHU 00 +DQQXNVHOD 7 :LHJDQG +$9& LQ :LUHOHVV(QYLURQPHQWVIEEE Trans. on Circuit and Systems for Video Technology9RO1RSS >@ $5GHOD&UX]5'&DMRWH-RLQW6RXUFH&KDQQHO5DWH$OORFDWLRQIRU :LUHOHVV 9LGHR 7UDQVPLVVLRQ %DVHG RQ 'LVWRUWLRQ (VWLPDWLRQ International Conference on Information Science and Applications (ICISA)6HRXO6RXWK.RUHDSS0D\ >@ -RLQW 9LGHR 7HDP -97 5HIHUHQFH 6RIWZDUH (QFRGHU YHUVLRQ  KWWSZZZKKLGHVHXKULQJWPO >@ = +H - &DL &: &KHQ -RLQW 6RXUFH &KDQQHO 5DWH'LVWRUWLRQ $QDO\VLV IRU $GDSWLYH 0RGH 6HOHFWLRQ DQG 5DWH &RQWURO LQ :LUHOHVV 9LGHR &RGLQJ IEEE Trans. on Circuits and Sytems for Video TechnologyYRO1R-XQH >@ =&KHQ':X5DWH'LVWRUWLRQ2SWLPL]HG&URVV/D\HU5DWH&RQWURO LQ :LUHOHVV 9LGHR &RPPXQLFDWLRQ IEEE Trans. on Circuit and Systems for Video TechnologyYRO1R0DUFK >@ :1 /LH =: *DR 7/ /LX 3& -XL -RLQW 6RXUFH&KDQQHO 9LGHR &RGLQJ %DVHG RQ WKH 2SWLPL]DWLRQ RI (QGWR(QG 'LVWRUWLRQV Springer-Verlag Berlin Heidelberg36,97/1&6SS