ASYMMETRIC GRADIENT-BASED IMAGE ALIGNMENT

$6@ REH\ WKH JHQHUDOVFKHPHRILWHUDWLYH*DXVV1HZWRQDSSURDFKHV  6WHS 'HILQHLQLWLDOL]DWLRQSDUDPHWHUV—DQG—UHI  6WHS ([SUHVVWKHPRWLRQSDUDPHWHUVZUWDQLQFUHPHQWDO FRUUHFWLRQ SDUDPHWHU YHFWRU į— ȝ , ȝ ,  ȝ N  Gȝ  DQG

ȝ7

ȝ7  ȝUHI  Gȝ  &RPSXWH į— E\ XVLQJ RQH VWHS RI

JUDGLHQWEDVHGRSWLPL]DWLRQRI(

ICASSP 2009



 - W - - W H ȝ N ȝUHI  

įȝ

 

ZKHUH - LV WKH -DFRELDQ ZUW į— RI WKH SL[HOZLVH HUURU YHFWRUHIRUPHGE\VWDFNLQJDOOHLYHUWLFDOO\  6WHS   8VH WKH JHQHULF XSGDWH UXOH LQWURGXFHG LQ >@ WR REWDLQWKHHVWLPDWHGSDUDPHWHUV

ȝ N 



m ȝ,

 ȝ N  įȝ $

ȝ7  ȝUHI

 įȝ  $

ȝUHI 

 

  DQG FRPSXWLQJ LWV GHULYDWLYH \LHOGV WKH IROORZLQJ -DFRELDQ w,Iȝ N $ įȝ[ L     - )& [ L - ,)& [ L wįȝ įȝ  7KH ,QYHUVH &RPSRVLWLRQDO ,&  DSSURDFK >@ LV UHODWHG >@ WR ȝ, ȝ N DQG ȝ7 ȝUHI $GP  WKDW\LHOGV 

-

L



- 7,& [ L



w7Iȝ UHI $ įȝ[L wįȝ



 

įȝ 

V\PPHWULFDOFRPELQDWLRQRIWKHJUDGLHQWVRQ,DQG7

- 6& [L





 )& ,&   - , [L  - 7 [L

 

7KH (IILFLHQW 6HFRQGRUGHU 0LQLPL]DWLRQ (60  DOJRULWKP >@ LV EDVHG RQ D /LH JURXS SDUDPHWHUL]DWLRQ įȝ LV SDUDPHWHUL]HG DURXQG įȝ   E\ D /LH DOJHEUD YHFWRU įYOLH 7KHOLQNEHWZHHQWKHWZRYHFWRUVLVJLYHQE\WKHH[SRQHQWLDO PDS³H[S´UHODWLRQVKLS>@    įȝ H[SįYOLH  7KLV DSSURDFK FRUUHVSRQGV WR D 6\PPHWULF &RPSRVLWLRQDO ([SRQHQWLDOPDS DSSURDFK 6&(  >@ ZLWK ȝ, ȝ N $ H[SGYOLH  DQG ȝ7 ȝUHI $ H[SGYOLH  ZKLFK \LHOGVWKHIROORZLQJ-DFRELDQ 

- 6&([L

- 6&[L

w H[SGYOLH wGYOLH GY

OLH



 





982

ȝUHI $ D Gȝ  

 

ZKHUH Į> @ LV D WXQLQJ SDUDPHWHU FRUUHVSRQGLQJ WR DQ DV\PPHWULF FRQVWUDLQW LPSRVHG RQ WKH ELGLUHFWLRQDO SDUDPHWHUYHFWRU—,—7   7KH -DFRELDQ PDWUL[ LV REWDLQHG E\ SOXJJLQJ HTXDWLRQ   DQG LQWR DQGFRPSXWLQJLWVGHULYDWLYH  - D$&  [L



  D - ,)&  [L  D - 7,&  [L 

 

7KH VSHFLDO FDVH Į  FRUUHVSRQGV WR WKH -DFRELDQ RI WKH 6\PPHWULF&RPSRVLWLRQDODSSURDFKHVSUHVHQWHGEHIRUH7KH WZRH[WUHPHĮYDOXHVFRUUHVSRQGWRWKH)& PHWKRG>@IRU Į DQGWKH,& PHWKRG>@IRUĮ   8VLQJ LQVWHDG D /LH JURXS SDUDPHWHUL]DWLRQ ZH FDQ DOVR GHILQH WKH $V\PPHWULF &RPSRVLWLRQDO ([SRQHQWLDO PDS $&( DSSURDFKEDVHGRQ ȝ, ȝ N $ H[S  D GYOLH      ȝ7 ȝUHI $ H[S D GYOLH    ZKLFKOHDGVWRWKHIROORZLQJ-DFRELDQ - D$&([L

- D$& [L

w H[SGYOLH wGYOLH GY

    7KH$&(DSSURDFKLVWKHUHIRUHDJHQHUDOL]DWLRQRIWKH(60 DOJRULWKP>@ZKLFKFDQEHREWDLQHGIRUĮ 7KHVSHFLDOV FDVHV Į  DQG Į  FDQ EH DGGHG WR WKH WD[RQRP\ DV )RUZDUG&RPSRVLWLRQDO([SRQHQWLDOPDS)&( DQG,QYHUVH &RPSRVLWLRQDO([SRQHQWLDOPDS,&(   7DEOHVKRZVDVXPPDU\RIKRZWRLQWHJUDWHWKHSURSRVHG $&  DQG $&(  PHWKRGV ZLWKLQ WKH WD[RQRP\ VFKHPH RI >@ZLWKLWVUHODWLRQVKLSWRSXEOLVKHGZRUNV  7DEOH  6\QRSWLF SUHVHQWDWLRQ RI WKH SURSRVHG DSSURDFK FRPSDUHGWRH[LVWLQJDOJRULWKPV *HQHULF -DFRELDQ 'LUHFW /LHJURXS FDWHJRU\ SDUDP 3DUDP HT  HT  )RUZDUGV )&  )&(  - )& - $&  >@>@ >@ ,&  ,&(  ,QYHUVH - ,& - $&  >@ >@>@ 6&  6&(  6\PPHWULF - 6& - $&   >@ >@ >@>@ Į $&  $&(  - Į$&  $V\PPHWULF  

7KH 6\PPHWULF &RPSRVLWLRQDO 6&  DSSURDFK >@ XVHV ȝ, ȝ N $ Gȝ  DQG ȝ7 ȝUHI $ Gȝ   ZKLFK \LHOGV D 

ȝ7



6WHS 5HSHDWVWHSV DQG XQWLOFRQYHUJHQFH  &DWHJRULHVRIDSSURDFK  *UDGLHQW EDVHG DSSURDFKHV KDYH EHHQ VKRZQ LQ >@ WR EH HTXLYDOHQWWRWKHILUVWRUGHULQWKHHVWLPDWHGį—ZLWKLQHDFK RI WKH IROORZLQJ FDWHJRULHV )RUZDUGV ,QYHUVH DQG 6\PPHWULF :H ZLOO WKHUHIRUH FRQVLGHU RQO\ RQH DSSURDFK ZLWKLQHDFKFDWHJRU\  7KH )RUZDUGV &RPSRVLWLRQDO )&  DSSURDFK >@ XVHV ȝ, ȝ N $ Gȝ DQG ȝ7 ȝUHI  3OXJJLQJ WKHVH H[SUHVVLRQV LQWR

,& [

$6@ 

7KH6&( DSSURDFKKDVDODUJHUUHJLRQRIFRQYHUJHQFH WKDQ ERWK RI WKHP E\ WDNLQJ DGYDQWDJH RI JUDGLHQW LQIRUPDWLRQIURPERWK,DQG7DVZDVGLVFXVVHGLQ>@7KH QHZ $&  DOJRULWKP KDV HYHQ EHWWHU UHVXOWV E\ WDNLQJ LQWR DFFRXQW ERWK LPDJHV EXW ZHLJKWLQJ DSSURSULDWHO\ WKH JUDGLHQWVDVVRFLDWHGWRWKHQRLV\LPDJH,7KHUHVXOWVRQWKH FRQYHUJHQFH UDWH )LJ   OHDG WR DQ DQDORJ DQDO\VLV WKH $& FRQYHUJHVIDVWHU 

 )LJ  $YHUDJH IUHTXHQF\ RI FRQYHUJHQFH ZLWK FRUUXSWHG LPDJH,ZLWKı, DQGĮ  

 )LJ  &RQYHUJHQFH UDWH ZLWK FRUUXSWHG LPDJH , ZLWK ı, DQGĮ DQGı  

,QIOXHQFHRIWKHĮYDOXH  6LQFHWKH)RUZDUGV,QYHUVHDQG6\PPHWULFDSSURDFKHVFDQ EHFRQVLGHUHGDVVSHFLDOFDVHVRIWKH$V\PPHWULFDSSURDFK ZH QRZVWXG\ WKHLQIOXHQFHRI WKHFKRLFHRIWKHĮYDOXHRQ WKHSHUIRUPDQFHV 7KH UHVXOWV RI ILJXUH  DUH REWDLQHG LQ WKH VDPH FRQILJXUDWLRQDVLQILJXUHVDQGEXWUHVWULFWLQJWRDIL[HG 3RLQW6LJPDı DQGYDU\LQJWKHĮSDUDPHWHUIURPWR )RUWKHUDWHRIFRQYHUJHQFHı )LJULJKW DOOSRLQWVRQ WKH VDPHYHUWLFDOUHSUHVHQWWKH506SRLQW HUURU IRUDIL[HG Į DQG IRU WKH VXFFHVVLYH LWHUDWLRQV RQO\ WKH ILUVW ILYH LWHUDWLRQVDUHVKRZQ  7KH SHUIRUPDQFHV RI WKH )&  ,&  DQG 6&(  DSSURDFKHVGLVSOD\HGLQ ILJXUHV DQGFDQEHUHWULHYHGLQ ILJXUHIRUĮ DQGUHVSHFWLYHO\7KHRSWLPDOĮYDOXH LQ VXFK D FRQILJXUDWLRQ FRUUHVSRQGV WR Į§ ZKLFK FRXOG QRWEHREWDLQHGXVLQJSUHYLRXVDSSURDFKHV  LW LW LW«

 )LJ/HIW$YHUDJHIUHTXHQF\RIFRQYHUJHQFH5LJKW5DWH RIFRQYHUJHQFHILUVWLWHUDWLRQV ZLWKQRLVHLQLPDJH,  :KHQWKHUHLVQRQRLVHLQWKHLPDJHVı, ı7 VHHILJ OHIW  WKH RSWLPDO YDOXH LV Į  ZKLFK SXWV LQ SHUVSHFWLYH WKHUHVXOWVVKRZQLQ>@DQG>@WKDWZKHQWKHWZRLPDJHVDUH LGHQWLFDO XS WR PRWLRQ FRPSHQVDWLRQ XVLQJ D 6\PPHWULFDO DSSURDFK\LHOGVPXFKLPSURYHGSHUIRUPDQFH :KHQ QRLVH LV LQFUHDVHG LQ RQH RI WKH LPDJHV WKH RSWLPDO Į YDOXH LV VKLIWHG DZD\ VHH )LJ  QDWXUDOO\ ZHLJKWLQJ GRZQ WKH FRUUXSWHG JUDGLHQWV 7KH RSWLPDO Į YDOXHLQGHHGFRPHVIURPDWUDGHRIIEHWZHHQWKHXVHRIERWK LPDJH JUDGLHQWV DQG WKH TXDOLW\ RI WKRVH JUDGLHQWV )URP D SUDFWLFDO SRLQW RI YLHZ WKH ,&  DSSURDFK Į   XVHV RQO\ RQH -DFRELDQ ZKLFK FDQ EH FRPSXWHG RIIOLQH 7KH XVH RI ERWKJUDGLHQWVUHTXLUHVDGGLWLRQDOFRPSXWDWLRQVSHULWHUDWLRQ ZKLFK DUH FRPSHQVDWHG E\ D IHZHU QXPEHU RI LWHUDWLRQV )LJ DQGDEHWWHUUREXVWQHVV)LJOHIW)LJ 



)LJ  &RQYHUJHQFH UDWH IRU WKH ILUVW  LWHUDWLRQV /HIW ZLWKRXWLPDJHQRLVH5LJKWZLWKQRLVHLQLPDJH7ı7 

984

6KLIWLQJRIWKH RSWLPDOYDOXH

$PRXQW RIQRLVH

 )LJ  $YHUDJH IUHTXHQF\ RI FRQYHUJHQFH ZUW SDUDPHWHU ĮIRULQFUHDVLQJDPRXQWRIQRLVHı,LQLPDJH,ı7    &21&/86,21  ,Q WKLV SDSHU ZH KDYH SUHVHQWHG D QHZ DSSURDFK IRU SDUDPHWULFLPDJHDOLJQPHQWWKH$V\PPHWULF&RPSRVLWLRQDO DSSURDFKZKLFKKDVEHHQVKRZQWRJHQHUDOL]HH[LVWLQJVWDWH RIWKH DUW JUDGLHQWEDVHG LPDJH DOLJQPHQW DSSURDFKHV %HQFKPDUN H[SHULPHQWV KDYH KLJKOLJKWHG VLWXDWLRQV ZKHUH WKH QHZ PHWKRG RXWSHUIRUPV WKH H[LVWLQJ RQHV HVSHFLDOO\ ZKHQWKHWHPSODWHRUWKHLPDJHLVFRUUXSWHGE\QRLVH$WWKLV VWDJHWKHPHWKRGUHTXLUHVWKHSDUDPHWHUĮWREHGHILQHG,WV DGDSWLYHHVWLPDWLRQLVWKHVXEMHFWRIIXWXUHZRUN  5()(5(1&(6  >@% /XFDV DQG 7 .DQDGH ³$Q LWHUDWLYH LPDJH UHJLVWUDWLRQ WHFKQLTXHZLWKDQDSSOLFDWLRQWRVWHUHRYLVLRQ´,Q3URFHHGLQJVRI WKH ,QWHUQDWLRQDO -RLQW &RQIHUHQFH RQ $UWLILFLDO ,QWHOOLJHQFH SS   >@6 %DNHU DQG , 0DWWKHZV ³/XFDV.DQDGH  @5 0pJUHW -% $XWKHVVHUUH DQG @6 %HQKLPDQH DQG ( 0DOLV ³+RPRJUDSK\EDVHG ' 9LVXDO 7UDFNLQJ DQG 6HUYRLQJ´ 7KH ,QWHUQDWLRQDO -RXUQDO RI 5RERWLFV 5HVHDUFK9ROSS  >@@+@( %D\UR&RUURFKDQR - 2UWHJyQ$JXLODU ³/LH DOJHEUD DSSURDFK IRU WUDFNLQJ DQG'PRWLRQ HVWLPDWLRQ XVLQJ PRQRFXODU YLVLRQ´ ,PDJH DQG 9LVLRQ &RPSXWLQJ SS  -XQH