This effort describes a novel shadow restoration algorithm based on atmospheric effects and characteristics of sun light for aerial images. Firstly, shadow regions ...
Proceedings of 2010 IEEE 17th International Conference on Image Processing
September 26-29, 2010, Hong Kong
$129(/6+$'2:5(6725$7,21$/*25,7+0%$6('21 $70263+(5,&())(&76)25$(5,$/,0$*(6 dD÷ODU$\WHNLQDQG$$\GÕQ$ODWDQ (OHFWULFDODQG(OHFWURQLFV(QJLQHHULQJ'HSDUWPHQW0(78 %DOJDW$QNDUD7h5.ø@ 7KH ODWWHU RFFXUV ZKHQ UDGLDWLRQ LQWHUDFWV ZLWK SDUWLFOHV LQ WKH DWPRVSKHUH ZLWK GLDPHWHU PXFK ODUJHU WKDQ WKH ZDYHOHQJWK RI WKH UDGLDWLRQ ZKHUHDV WKH IRUPHU 5D\OHLJK VFDWWHULQJ RFFXUV LQ WKH FDVHV WKDW WKHGLDPHWHURIWKHSDUWLFOHWKDWLVLQWHUDFWHGZLWKLVPXFKVPDOOHU WKDQ WKH ZDYHOHQJWK RI WKH OLJKW 7KHUHIRUH SDUWLFOH VFDWWHULQJ DIIHFWVVKRUWHUZDYHOHQJWKV7KHLQWHQVLW\,RIOLJKWVFDWWHUHGE\D VLQJOH VPDOO SDUWLFOH IURP D EHDP RI OLJKW RI ZDYHOHQJWK ߣ DQG LQWHQVLW\,LVJLYHQE\WKHIROORZLQJUHODWLRQ>@ ଶ ݀ ͳ ܿ ݏଶ ߠ ʹߨ ସ ݊ଶ െ ͳ ൬ ൰ ቆ ቇ ൬ ൰ ܫൌ ܫ ߣ ʹ ݊ଶ ʹ ʹܴଶ ZKHUHܴLVWKHGLVWDQFHWRWKHSDUWLFOHߠLVWKHVFDWWHULQJDQJOH݊ LVWKHUHIUDFWLYHLQGH[RIWKHSDUWLFOHDQG݀LVWKHGLDPHWHURIWKH SDUWLFOH &RQVLGHULQJ WKH IDFW WKDW D EHDP RI VXQOLJKW FRQWDLQLQJ FRPSRQHQWVRIGLIIHUHQWZDYHOHQJWKVLQWHUDFWVZLWKDSDUWLFOHLQWKH DWPRVSKHUH WKH RQO\ YDU\LQJ SDUDPHWHU LQ WKH DERYH HTXDWLRQ LV
ICIP 2010
WKH ZDYHOHQJWKߣ +HQFH RQH VKRXOG H[SHFW WKDW VKRUWHU ZDYHOHQJWKV LH EOXH FRPSRQHQW RI OLJKW ZLOO EH VFDWWHUHG PXFK PRUHWKDQRWKHUV6LQFHWKHPDLQLOOXPLQDWLRQVRXUFHRIWKHVKDGRZ UHJLRQV LV WKH VFDWWHUHG OLJKW IURP WKH DWPRVSKHUH LW LV H[SHFWHG WKDWWKHLOOXPLQDWLRQLQWHQVLW\RIEOXHOLJKWLQVKDGRZUHJLRQVWREH ODUJHU WKDQ RWKHUV 7KLV LGHD LV ZLGHO\ XVHG IRU GHWHFWLQJ WKH FDQGLGDWHVKDGRZUHJLRQVDVLQ >@$VWKHEOXHOLJKWLVWKH PRVW VFDWWHUHGFRPSRQHQWRIGLUHFWVXQOLJKWVLPLODUO\UHGOLJKWVKRXOG EH WKH OHVV VFDWWHUHG FRPSRQHQW 7KHUHIRUH WKH SL[HOV ZKRVH % EOXH FRPSRQHQWODUJHUWKDQ5 UHG FRPSRQHQWLQDQ5*%LPDJH DUH DFFHSWHG DV FDQGLGDWH VKDGRZ UHJLRQV 2EYLRXVO\ D VZLPPLQJSRRO RU DQ\ EOXHFRORUHG REMHFW LV DOVR H[SHFWHG WR EH PLVFODVVLILHG DV D VKDGRZ UHJLRQ ,Q RUGHU WR JHW ULG RI VXFK UHJLRQV D WKUHVKROGLQJ DOJRULWKP FRXOG EH DSSOLHG VLQFH WKH LOOXPLQDWLRQLQWHQVLW\RIVKDGRZ UHJLRQVDUHH[SHFWHGWREH PXFK VPDOOHUWKDQWKHVHPLVFODVVLILHGUHJLRQV>@ +HQFHDSL[HOLVFODVVLILHGDVDVKDGRZSL[HO LILWV%FRPSRQHQWLV ODUJHUWKDQ5DQG*FRPSRQHQWVDQGLILWVLOOXPLQDWLRQLQWHQVLW\LV ORZHUWKDQDWKUHVKROGYDOXHZKLFKLVGHWHUPLQHGPDQXDOO\ 6KDGRZ5HVWRUDWLRQ :KHQUHVWRULQJVKDGRZVRQHVKRXOGILUVWFRQVLGHUWKDWHYHU\SRLQW LQDVKDGRZUHJLRQLVQRWXQLIRUPO\LOOXPLQDWHG$FWXDOO\VKDGRZV DUHVHSDUDWHGLQWRWZRGLIIHUHQWUHJLRQV XPEUDDQGSHQXPEUD>@ 8PEUDLVWKHUHJLRQLQZKLFKWKHLOOXPLQDWLRQVRXUFHLVFRPSOHWHO\ RFFOXGHG ZKHUHDV SHQXPEUD UHJLRQ LV WKH ERXQGDU\ UHJLRQ LQ ZKLFKRQO\DSRUWLRQRIOLJKWVRXUFHLVREVFXUHGE\WKHRFFOXGLQJ ERG\7KHSHQXPEUDUHJLRQVFDQEHWKRXJKWRIDVPRRWKWUDQVLWLRQ EHWZHHQXPEUDDQGWKHQRQVKDGRZUHJLRQV7KLVWUDQVLWLRQFDQEH PRGHOHGZLWKDVLJPRLGIXQFWLRQDVLQ>@GXHWRWKHVKDSHRIWKH VXQ LWVHOI ,Q RWKHU ZRUGV VLQFH WKH PDLQ OLJKW VRXUFH LQ DHULDO LPDJHVLHWKHVXQFDQEHPRGHOHGDVDVSKHUHWKHSRUWLRQRIWKH VXQWKDWLVREVHUYHGE\WKHSHQXPEUDUHJLRQILUVWGHFUHDVHVVORZO\ WKHQ DURXQG WKH FHQWHU RI WKH VSKHUH LW GHFUHDVHV TXLFNO\ WKHQ DJDLQ GHFUHDVHV JUDGXDOO\ ZKLOH LOOXPLQDWLQJ WKH XPEUD UHJLRQV IURPWKHQRQVKDGRZUHJLRQV VHH)LJXUH
)LJXUH 5HJLRQRIWKHVXQYLVLEOH LQZKLWH DVJRLQJGHHSHULQ SHQXPEUDUHJLRQ )URPOHIWWRULJKW 5HVXOWLQJ IURP RWKHU VFDWWHULQJ HIIHFWV DQG GLIIUDFWLRQ RI OLJKWDWWKHHGJHVRIREMHFWVWKLV6W\SHFXUYH VHH)LJXUH FDQEH GLVWRUWHG WR VRPH H[WHQW 7KH SL[HOV KDYLQJ PD[LPXP DQG WKH PLQLPXPLQWHQVLW\LQHDFKWUDQVLWLRQDUHLQWXLWLYHO\WKHSL[HOVWKDW DUH PLQLPDOO\ DIIHFWHG E\ WKHVH RWKHU HIIHFWV 7KHUHIRUH DIWHU GHWHUPLQLQJWKHVHSL[HOVRQHFDQPRGHOHDFKWUDQVLWLRQDVIROORZV ͳ ሺܫ௫ െ ܫ ሻ ܫௗௗ ሺ݊ሻ ൌ ܫ ଵଶିି ିቀ ቁ ିଵ ͳ݁ ZKHUHQLVWKHORFDWLRQRISL[HOܫௗௗ ሺ݊ሻLVWKHPRGHOWKDWZH FRQVLGHU WKH LQWHQVLWLHV LQ WKH WUDQVLWLRQ KDYH ܫ DQG ܫ௫ DUH UHVSHFWLYHO\ WKH PLQLPXP DQG WKH PD[LPXP LQWHQVLW\ YDOXHV LQ WKDWWUDQVLWLRQYHFWRUDQG/LVWKHOHQJWKRIWKHWUDQVLWLRQYHFWRULQ
402
SL[HOV 7KH DERYH UHODWLRQ VLPSO\ VKLIWV E\ ±/ DQG VFDOHV E\ / DFDQRQLFDOVLJPRLGIXQFWLRQ )RU WKH VKDGRZ FRPSHQVDWLRQ VWHS LQ RUGHU WR UHVWRUH SHQXPEUD UHJLRQV WKH RULJLQDO LQWHQVLW\ YDOXHV QDPHO\ ܫ ሺ݊ሻ LQ HYHU\ WUDQVLWLRQ IRU HDFK FRORU EDQG LV PXOWLSOLHG ZLWK DQ LQYHUVH VLJPRLGIXQFWLRQLQRUGHUWRREWDLQWKHUHVWRUHGLQWHQVLWLHVDV ܫ௫ ൰ ܫሺ݊ሻ ܫ௦௧ௗ ሺ݊ሻ ൌ ൬ ܫௗௗ ሺ݊ሻ $IWHUUHVWRULQJSHQXPEUDUHJLRQVWKHXPEUDUHJLRQVDUHDOVR FRPSHQVDWHG$OWKRXJKWKHIXQGDPHQWDOLOOXPLQDWLRQVRXUFHRIDQ XPEUDUHJLRQLVWKHVFDWWHUHGOLJKWIURPWKHDWPRVSKHUHQRWHYHU\ SDUW RI WKH UHJLRQ LV LOOXPLQDWHG XQLIRUPO\ GXH WR VFDWWHULQJ RI OLJKW IURP DQRWKHU REMHFWV DQG GLIIUDFWLRQ RI OLJKW LQ WKH HGJHV RI WKH REMHFWV 6LQFH LW LV D GLIILFXOW WDVN WR PRGHO WKLV QRQXQLIRUP LOOXPLQDWLRQRQHFRXOGRYHUVHJPHQW ODUJHQXPEHURIVPDOOVL]HG UHJLRQV WKHXPEUDUHJLRQVE\XVLQJPHDQVKLIWVHJPHQWDWLRQ>@ (DFK VPDOO VHJPHQW FDQ EH DVVXPHG DV D XQLIRUPO\ LOOXPLQDWHG DQG FRQVWDQW DOEHGR UHJLRQ $IWHU ILQGLQJ WKHVH VHJPHQWV WKH XQNQRZQ UDWLR EHWZHHQ WKH LQWHQVLW\ RI WKLV VHJPHQW DQG WKH QHLJKERULQJQRQVKDGRZUHJLRQLVGHWHUPLQHGIRUHDFKFRORUEDQG 1H[W WKH SL[HO LQWHQVLWLHV LQ WKH RULJLQDO LPDJH IRU HDFK VHJPHQW DUHPXOWLSOLHGZLWKWKHVHUDWLRVIRUHDFKFRORUEDQGIRUUHVWRUDWLRQ &RQVLGHULQJ WKH XPEUD UHJLRQV DV D ILUVW VWHS WKH XPEUD UHJLRQV DUH RYHU VHJPHQWHG XVLQJ PHDQVKLIW VHJPHQWDWLRQ >@ 7KHQ IRU HDFK FRQQHFWHG XPEUD UHJLRQ ZH VWDUW IURP RXWHU VHJPHQWV DQG UHVWRUH WKHLU LOOXPLQDWLRQ ZLWK PXOWLSO\LQJ ZLWK WKH HVWLPDWHGQRQVKDGRZWRVKDGRZUDWLR7KLVVWHSLVUHSHDWHGXQWLO QRVHJPHQWLVOHIWLQDVKDGRZDUHD (;3(5,0(17$/5(68/76 7KHSURSRVHGDOJRULWKPLVDSSOLHGWRVRPHVDWHOOLWHLPDJHVWKDWDUH FROOHFWHG IURP *RRJOH (DUWK 6RPH RWKHU LPDJHV IURP VLPLODU UHVHDUFKHIIRUWVDUHDOVRXWLOL]HGGXULQJVLPXODWLRQV
)LJXUH 6KDGRZUHVWRUDWLRQUHVXOWIRUDW\SLFDOVDWHOOLWHLPDJH WRS 2ULJLQDO,PDJH ERWWRP 5HVWRUHG,PDJH
, Q
Q
D
PRGHOOHG
Q
,
,Q )LJXUH DV D W\SLFDO H[DPSOH WKH JURXQG VKDGRZ RI WKH PRQXPHQW LV UHPRYHG DQG FRPSHQVDWHG ZLWK VRPH VXFFHVV 7KH SURSRVHGDOJRULWKPLVDOVRDSSOLHGIRUQRQDHULDOLPDJHVDV ZHOO ,Q)LJXUHWKHDOJRULWKPSHUIRUPVEHWWHU ZLWKPLQLPXPQXPEHU RI GLIIHUHQW REMHFWV DQG WH[WXUH LQIRUPDWLRQ ZLWKLQ WKH VKDGRZ UHJLRQVVLQFHWKHUHVWRUHGDUHDKDVDOPRVWFRQVWDQWVXUIDFHDOEHGR KHQFHWKHGLVWRUWLRQVGXHWRIURPYDULDWLRQVLQWKHWH[WXUHDUHOHVV WKDQ RWKHU LPDJHV ,Q )LJXUH IRU D QRQDHULDO LPDJH GHWHFWHG VKDGRZV RI WKLV LPDJH DV SDLQWHG LQ EOXH FRORU WKH LPDJH DIWHU SHQXPEUDUHVWRUDWLRQWKHLPDJHDIWHUXPEUDUHVWRUDWLRQDQGRXWSXW RIWKHVKDGRZUHVWRUDWLRQIRUWKHDOJRULWKPLQ> @DUHDOOSUHVHQWHG 7KH SURSRVHG PHWKRGV JLYHV D PXFK EHWWHU UHVWRUDWLRQ UHVXOW FRPSDUHGWR> @ ,Q )LJXUH WKH SHQXPEUD OLQH WUDQVLWLRQ LQWHQVLWLHV UHG FRPSRQHQW RQO\ WKH PRGHOHG WUDQVLWLRQ LQWHQVLWLHV UHVWRUHG OLQH LQWHQVLWLHV ZLWK WKH DOJRULWKP SURSRVHG DQG VDPH OLQH LQWHQVLWLHV ZLWKWKHPHWKRGSURSRVHGLQ> @DUHLOOXVWUDWHG 7KHSURSRVHGPHWKRGSURSRVHGLVDOVRFRPSDUHGZLWKDQRWKHU VKDGRZUHPRYDO PHWKRG>@WKDWGHSHQGVRQDSULRULLQIRUPDWLRQ VXFK DV LOOXPLQDWLRQ YHFWRU DQG \LHOGHG VDWLVIDFWRU\ UHVXOWV DV LOOXVWUDWHGLQ)LJXUH7KHWH[WXUHYDULDWLRQVXQGHUWKHVKDGRZVRI VRPHEXLOGLQJVDUHREWDLQHGEHWWHUWKDQWKHDOJRULWKPLQ>@
Q
E
UHVWRUHG
Q
,
D E
F
F G
Q
Q
UHVWRUHG
,
H )LJXUH D 2ULJLQDOLPDJH E 'HWHFWHGVKDGRZV F 3HQXPEUDUHJLRQVUHVWRUHG G 8PEUDUHJLRQVUHVWRUHG H 5HPRYDOUHVXOWZLWKWKHDOJRULWKPLQ> @
Q
G )LJXUH ,QWHQVLW\ UHGFRPSRQHQW SURILOHIRUWKH KRUL]RQWDOOLQHVKRZQ LQUHG LQ)LJXUHD D $SHQXPEUD WUDQVLWLRQOLQHZLWK/ ,PD[ ,PLQ E PRGHOHG WUDQVLWLRQ F UHVWRUHGLQWHQVLW\SURILOHIRUWKHOLQH G UHVXOW IRUWKHDOJRULWKPLQ> @
403
D
E
F )LJXUH 6KDGRZUHVWRUDWLRQUHVXOWIRUDVDWHOOLWHLPDJH D 2ULJLQDO,PDJH E 5HVWRUHGLPDJHE\WKHPHWKRGSURSRVHG F 5HVWRUHGLPDJHZLWKWKHPHWKRGSURSRVHGLQ>@ &21&/86,21 0RGHOLQJ DWPRVSKHULF HIIHFWV DQG WKH WUDQVLWLRQ IURP WKH QRQ VKDGRZ UHJLRQV DQG WKH XPEUD UHJLRQV IRU VKDGRZ GHWHFWLRQ DQG
404
UHVWRUDWLRQ SUREOHPV \LHOGV SURPLVLQJ UHVXOWV %DVHG RQ WKH REVHUYHGH[SHULPHQWDOUHVXOWVWKHDGYDQWDJHVRIWKLVPHWKRGFRXOG EHVXPPDUL]HGDVIROORZV 7KHSURSRVHGPHWKRG • GRHV QRW UHTXLUH DQ\ SULRU NQRZOHGJH VXFK DV WKH LOOXPLQDWLRQYHFWRURUDGLJLWDOVXUIDFHPRGHO • KDV D WKHRUHWLFDO IRXQGDWLRQ EDVHG RQ SK\VLFDO SKHQRPHQD UDWKHUWKDQVRPHDGKRFDSSURDFKHV • FDQEHXVHGLQGLIIHUHQWLPDJHVRWKHUWKDQDHULDORQHV • LV LQGHSHQGHQW RI DQ\ GHVLJQ SDUDPHWHU H[FHSW IRU WKH WKUHVKROGLQJVWHSGXULQJVKDGRZGHWHFWLRQSDUW 5()(5(1&(6 >@36DUDEDQGL)@ 0 5 $VKDULI DQG + (WHPDGQLD ³6KDGRZ ,GHQWLILFDWLRQ E\ ,PDJH &RPSRQHQWV 6HSDUDWLRQ 8VLQJ +RPRPRUSKLF 6\VWHP´ %XOOHWLQRIWKH)DFXOW\RI(QJLQHHULQJ8QLYHUVLW\RI5\XN\XVYRO -DSDQSS >@ + (WHPDGQLD DQG 0 5 $VKDULI ³$XWRPDWLF ,PDJH 6KDGRZ ,GHQWLILFDWLRQ XVLQJ /3) LQ +RPRPRUSKLF 3URFHVVLQJ 6\VWHP´ 3URF 9,,WK 'LJLWDO ,PDJH &RPSXWLQJ 7HFKQLTXHV DQG $SSOLFDWLRQV6\GQH\ >@-06FDQODQ'0&KDEULHVDQG5:&KULVWLDQVHQ³$ 6KDGRZ'HWHFWLRQ$QG5HVWRUDWLRQ$OJRULWKP)RU',PDJHV´ $FRXVWLFV6SHHFKDQG6LJQDO3URFHVVLQJ9RO86$SS >@
