A Survey of Coded Image and Video Quality Assessment

$ 6XUYH\RI&RGHG,PDJHDQG9LGHR4XDOLW\$VVHVVPHQW $PDO3XQFKLKHZD'RQDOG*%DLOH\DQG50+RGJVRQ ,QVWLWXWHRI,QIRUPDWLRQ6FLHQFHV 7HFKQRORJ\ 0DVVH\8QLYHUVLW\3DOPHUVWRQ1RUWK1HZ=HDODQG *$3XQFKLKHZD#PDVVH\DFQ]'*%DLOH\#PDVVH\DFQ]50+RGJVRQ#PDVVH\DFQ]

$EVWUDFW $Q LPSRUWDQW DVSHFW RI FRGLQJ LV TXDOLW\ DVVHVVPHQW 7KLV SDSHU UHYLHZV D UDQJH RI FRGHG LPDJH DQG YLGHR TXDOLW\ DVVHVVPHQW WHFKQLTXHV WKHLU PHULWV GHPHULWV DQG WKHLU UHODWLRQVKLSV 2EMHFWLYH DQG VXEMHFWLYH DVVHVVPHQWV DUH WKH WZR PDMRU FODVVHV RI LPDJH DQG YLGHR TXDOLW\ DVVHVVPHQW 6LQFH SLFWXUH TXDOLW\ LV YHU\ VXEMHFWLYH PDLQO\ GXH WR WKH YHU\ FRPSOH[ LQWHUDFWLRQV ZLWKLQ WKH +XPDQ 9LVXDO 6\VWHP +96  WKH SULPDU\ WHFKQLTXHVIRUPHDVXULQJSLFWXUHERWKLPDJHDQGYLGHR TXDOLW\DUHVXEMHFWLYHPHWKRGVVXFKDV,785HF 2IWHQREMHFWLYHPHDVXUHVRILPDJHTXDOLW\DUHSRRUO\FRUUHODWHGZLWKSHUFHLYHGSLFWXUHTXDOLW\7KLVPDNHVWKH GHYHORSPHQW RI YDULRXV REMHFWLYH PHDVXUHV KDUGHU +RZHYHU D IHZ REMHFWLYH TXDOLW\ PHWULFV KDYH EHHQ GHYHORSHG WKDW KDYH D KLJK FRUUHODWLRQ ZLWK VXEMHFWLYH PHDVXUHV 5HVHDUFKHUV LQ WKH ILHOG VWLOO IDFH REVWDFOHV ZKHQLWFRPHVWRHYDOXDWHWKHTXDOLW\RIWKHLULPDJHDQGYLGHRFRGLQJZRUNDVQRTXDOLW\PHWULFLVXQLYHUVDOO\ DFFHSWHG 7KH 9LGHR 4XDOLW\ ([SHUW *URXS LV ZRUNLQJ WRZDUGV VXFK PHWULFV ZLWK WKH SDUWLFLSDWLRQ RI PDQ\ ,QWHUQDWLRQDO7HOHFRPPXQLFDWLRQ8QLRQ,78 VWXG\JURXSV

.H\ZRUGVLPDJHYLGHRTXDOLW\DUWHIDFWVVXEMHFWLYHREMHFWLYHFRGLQJ+9694(*LPSDLUPHQWVVXUYH\ 

OLPLWDWLRQDWDQ\VWDJHRIWKHFRPPXQLFDWLRQSURFHVV >@

,QWURGXFWLRQ

7KH SDVW \HDUV KDYH VHHQ D JURZLQJ LQWHUHVW LQ HOHFWURQLF LPDJLQJ DQG PXOWLPHGLD DSSOLFDWLRQV &RQVXPHU SURGXFWV DUH DSSHDULQJ RQ WKH PDUNHW WKDW RIIHU D QHZ DQG EURDG UDQJH RI LPDJLQJ DSSOLFDWLRQV WR WKH SXEOLF 03(* LV LQ FRPPRQ XVH 'LJLWDO SKRWRJUDSK\GLJLWDO79SKRWRJUDSKLFTXDOLW\SULQWHUV DQG '9'V DUH VXFK H[DPSOHV 7KLV QHZ WHFKQRORJ\ KDV EHHQ PDGH SRVVLEOH E\ UHFHQW DGYDQFHV LQ LPDJH FDSWXUH LPDJH SURFHVVLQJ DQG FRGLQJ +RZHYHU D GLIILFXOW\IDFHGE\UHVHDUFKHUVDQGSURGXFWGHYHORSHUV LV WKDW QR VDWLVIDFWRU\ DQG UHDGLO\ DFFHSWHG TXDOLW\ PHWULFVDYDLODEOHIRULPDJHDQGYLGHRTXDOLW\ )RU WKH SXUSRVH RI FRPPXQLFDWLRQ ZH RIWHQ QHHG WR DFTXLUH SURFHVV DQG GHOLYHU YLVXDOO\SUHVHQWHG GDWD LQFOXGLQJLPDJHVDQGYLGHR$EORFNGLDJUDPIRUWKLV SURFHVVLVVKRZQLQ)LJXUH'LJLWDOLPDJHDQGYLGHR V\VWHPVKDYHPDQ\DGYDQWDJHVRYHUH[LVWLQJDQDORJXH V\VWHPV 'LJLWDO LPDJH DQG YLGHR SURFHVVLQJ DQG WUDQVPLVVLRQ WHFKQLTXHV FDQ EH HIILFLHQW DQG HFRQRPLFDO LQ UHVSHFW WR VLJQDO SURFHVVLQJ VWRUDJH DQGIUHTXHQF\VSHFWUXPXVDJH :KHQ YLVXDO LQIRUPDWLRQ LV FDSWXUHG SURFHVVHG DQG GHOLYHUHG WR WKH ILQDO UHFLSLHQW WKH GLVSOD\HG SLFWXUH PD\GLIIHUIURPWKHRULJLQDO$UWHIDFWVDUHDQ\YLVLEOH GLIIHUHQFHV WKDW DUH D GLUHFW UHVXOW RI VRPH WHFKQLFDO 'LJLWDO ,PDJHRU 9LGHR

6LJQDO 3URFHVVLQJ &RGLQJ 

,Q D PXOWLPHGLD HQYLURQPHQW DQ LPDJH FDQ EH GHVFULEHG DV D WZRGLPHQVLRQDO UHSUHVHQWDWLRQ RI D VFHQH RU RWKHU YLVXDO GDWD 9LGHR FDQ VLPSO\ EH WUHDWHG DV D VHTXHQFH RI LPDJHV EXW RIWHQ FRQWDLQV DGGLWLRQDO LQIRUPDWLRQ VXFK DV WLPLQJ RU V\QFKURQL]DWLRQVLJQDOV 7KHTXDOLW\RIWUDGLWLRQDODQDORJXHYLGHRV\VWHPVZDV HYDOXDWHGXVLQJDWHVWSDWWHUQRUDWHVWVLJQDO)RUVKRUW WLPHSHULRGV WKHV\VWHPV DUHDVVXPHGWREHDVOLQHDU DQGWLPHLQYDULDQW/7, 7KXVDVWDWLFWHVWSDWWHUQRU WHVW VLJQDO VXFK DV D PXOWLEXUVW FDQ EH XVHG WR PHDVXUH D V\VWHP DWWULEXWH VXFK DV WKH IUHTXHQF\ UHVSRQVH ZLWK FRQILGHQFH WKDW WKH V\VWHP ZRXOG UHVSRQGVLPLODUO\IRURWKHUYLGHRPDWHULDOHJYLGHR ZLWK PRWLRQ  $ JUHDW GHDO RI UHVHDUFK KDV EHHQ SHUIRUPHG WR UHODWH WKH WUDGLWLRQDO DQDORJXH YLGHR QRQOLQHDU TXDOLW\ PHDVXUHV HJ GLIIHUHQWLDO JDLQ GLIIHUHQWLDO SKDVH VKRUW WLPH ZDYHIRUP GLVWRUWLRQ HWF WRSHUFHLYHGFKDQJHVLQYLGHRTXDOLW\>@7KHVH WUDGLWLRQDO PHDVXUHV RI YLGHR TXDOLW\ DUH QRW DSSOLFDEOH LQ GLJLWDO YLGHR V\VWHPV ZKHUH QHZ IRUPV RI SURFHVVLQJ DUH DSSOLHG IRU FRPSUHVVLRQ VWRUDJH DQG WUDQVPLVVLRQ 'LJLWDO YLGHR V\VWHPV W\SLFDOO\ DGDSWDQGFKDQJHWKHLUEHKDYLRXUGHSHQGLQJXSRQWKH LQSXW VFHQH SURSHUWLHV VXFK DV VSDWLDO DQG WHPSRUDO

'HOLYHU\

6LJQDO 3URFHVVLQJ 'HFRGLQJ

'LVSOD\

)LJXUH'LJLWDO,PDJH9LGHRSURFHVVLQJIRUFRPPXQLFDWLRQ

326

Image and Vision Computing NZ

UHGXQGDQFLHV WR RSWLPLVH WKH FRPSUHVVLRQ 0RVW LPDJH GDWD FRQWDLQV VLJQLILFDQW UHGXQGDQF\ LQ WKH IRUP RI KLJK FRUUHODWLRQ RI DGMDFHQW SL[HO YDOXHV VSDWLDO UHGXQGDQFLHV  DQG EHWZHHQ DGMDFHQW IUDPHV RI YLGHR WHPSRUDO UHGXQGDQFLHV  0RUH UHFHQWO\ GLJLWDOYLGHRV\VWHPVKDYHEHHQGHYHORSHGWKDWDGMXVW WKHLU EHKDYLRXU EDVHG RQ WKH DYDLODEOH EDQGZLGWK 7KHUHIRUHWRFKDUDFWHULVHDYLGHRRULPDJHV\VWHPLW LV LPSRUWDQW WR WHVW IRU D ZLGH UDQJH RI SRWHQWLDO DUWHIDFWV IRU D JLYHQ V\VWHP E\ XVLQJ DQ DSSURSULDWH VHWRIWHVWVLJQDOV7KLVSDSHUVXUYH\VFXUUHQWVWDWXVRI SLFWXUHTXDOLW\DVVHVVPHQWRIFRGHGGLJLWDOLPDJHVDQG YLGHR



GHIHFWV ZLWK YDU\LQJ VWUHQJWKV RI EORFNLQHVV ORVVRI GHWDLO ULQJLQJ DQG PRVTXLWR QRLVH IHDWXUHV DIWHU 03(*FRPSUHVVLRQ>@

%ORFNLQHVV

%OXU

3LFWXUH4XDOLW\DQG$UWHIDFWV

&RGLQJ GLJLWDO LPDJHV WR H[SORLW WKH UHGXQGDQF\ SUHVHQW LQHYLWDEO\ UHVXOWV LQ D ORVV RI TXDOLW\ LQ WKH UHVXOWDQWLPDJHRUYLGHR7KLVLVEHFDXVHLQUHPRYLQJ UHGXQGDQWLQIRUPDWLRQRIWHQWKHVRPHRIWKHHVVHQWLDO LQIRUPDWLRQ LV GLVWRUWHG RU GHVWUR\HG UHVXOWLQJ LQ DUWHIDFWV LPSDLUPHQWV RU GHIHFWV )UHTXHQWO\ WKH WHUPV LPSDLUPHQWV DUWHIDFWV DQG GHIHFWV DUH XVHG V\QRQ\PRXVO\ +RZHYHU LPSDLUPHQWV JHQHUDOO\ UHIHU WR D VXEMHFWLYH PHDVXUH RI WKH GHJUDGDWLRQ RI GDWD $UWHIDFWV DUH YLVLEOH GLIIHUHQFHV RU IHDWXUHV WKDW PD\ EH DGGHG WR RU UHPRYHG IURP DQ LPDJH RU YLGHR UHVXOWLQJ IURP WHFKQLFDO OLPLWDWLRQ XVXDOO\ RI WKH FRGLQJ SURFHVV 'HIHFWVPD\ EH LQWURGXFHGDQ\ZKHUH ZLWKLQ WKH FRPPXQLFDWLRQ FKDLQ DQG DUH WKHUHIRUH PRUH JHQHUDO WKDQ DUWHIDFWV 7KHVH LQIOXHQFH WKH KXPDQYLVXDOV\VWHPLQVXFKDZD\WKDWWKHSHUFHLYHG LPDJHVGLIIHUIURPWKDWLQWHQGHG'LJLWDOYLGHRFRGLQJ LQWURGXFHV IXQGDPHQWDOO\ GLIIHUHQW W\SHV RI LPSDLUPHQWV WKDQ WKRVH FUHDWHG E\ WUDGLWLRQDO DQDORJXHWHFKQLTXHV $UWHIDFWV FUHDWH XQGHVLUDEOH YLVXDO HIIHFWV RQ WKH SLFWXUH7KHUHDUHQHZIRUPVRIDUWHIDFWVDVDUHVXOWRI WKHPLJUDWLRQWRWKHGLJLWDOGRPDLQ7KHFRPSUHVVLRQ DOJRULWKPVXVHGWKHSLFWXUHFRQWHQWDQGWKHRULJLQRI WKH VRXUFH PDWHULDO GHWHUPLQH WKH FRGLQJ DUWHIDFWV 7KH JUHDWHVW WHFKQLFDO OLPLWDWLRQ LV WKH DYDLODEOH EDQGZLGWK ZKLFK DIIHFWV WKH FRPSUHVVLRQ UDWLR DQG GDWD UDWH ,Q JHQHUDO DUWHIDFWV ZLOO EHFRPH PRUH YLVLEOH DV WKH FRPSUHVVLRQ UDWLR LV LQFUHDVHG &RPSUHVVLRQUHODWHG DUWHIDFWV VXFK DV EORFNLQHVV DQG FRORXU EOHHGLQJ LQ WKH GLJLWDO GRPDLQ DV GHSLFWHG LQ ILJXUH DUHFRPPRQDWORZELWUDWHV %ORFNLQJ DUWHIDFWV DUH WKH UHVXOW RI WKH LQGHSHQGHQW SURFHVVLQJ RI HDFK EORFN LQ EORFNEDVHG VLJQDO SURFHVVLQJ %OXU DQG ULQJLQJ DUH DUWHIDFWV UHVXOW IURP WUXQFDWLRQ RU TXDQWLVDWLRQ RI FRHIILFLHQWV LQ WKH IUHTXHQF\ GRPDLQ 5LQJLQJ DUWHIDFWV W\SLFDOO\ DSSHDU DV VKDUS RVFLOODWLRQV DORQJ WKH HGJHV RI DQ REMHFW DJDLQVW D UHODWLYHO\ XQLIRUP EDFNJURXQG $Q\ PRWLRQ RI WKH REMHFW LQ D YLGHR UHVXOWV LQ WKHVH RVFLOODWLRQV IOLFNHULQJJLYLQJPRVTXLWRQRLVH>@ 2IWHQ GHIHFWV PD\ EH GHFRPSRVHG LQWR IHDWXUHV RU VSHFLDOFKDUDFWHULVWLF)RUH[DPSOHLPDJHVPD\UHSRUW

Palmerston North, November 2003

5LQJLQJ

)LJXUH%ORFNLQHVVULQJLQJDQGEOXUDUWHIDFWV )RU GLJLWDO YLGHR V\VWHPV WKH VSDWLDO DQG WHPSRUDO LQIRUPDWLRQFRQWHQWRIWKHVRXUFHSOD\VDFUXFLDOUROH LQ GHWHUPLQLQJ WKH DPRXQW RI FRPSUHVVLRQ WKDW LV SRVVLEOH DQG KHQFH WKH SRWHQWLDO VHYHULW\ RI WKH FRPSUHVVLRQDUWHIDFWV7KHRSHUDWLQJFKDUDFWHULVWLFVRI WKHGLJLWDOWUDQVPLVVLRQFKDQQHOPD\DOVREHG\QDPLF LQ UHODWLRQ WR ELW UDWH GURSSHG SDFNHWV DQG WKLV ZLOO FDXVH WKH UHFHLYHG YLGHR TXDOLW\ WR YDU\ HYHQ ZKHQ WKH LQIRUPDWLRQ FRQWHQW RI WKH LQSXW YLGHR VWUHDP LV IL[HG:KHQGDWDLVWUDQVPLWWHGVRPHRIWKHGDWDPD\ EHORVWGLVWRUWHGRUUHIOHFWLRQVPD\UHVXOWLQPXOWLSOH GDWD 1RLVH DQG GLVWRUWLRQV FDQ UHVXOWLQPLVVLQJGDWD EORFNVRUGDWDSDFNHWVLQGLJLWDOV\VWHPVDVGHSLFWHG LQILJXUH 

)LJXUH7KHHIIHFWVRIPLVVLQJEORFNVRUGDWD SDFNHWV ,PDJH DQG YLGHR GHOLYHU\ RQ PRELOH V\VWHPV LV EHFRPLQJLQFUHDVLQJO\FRPPRQ,QORZELWUDWHYLGHR WHOHSKRQ\ VHUYLFHV IRU PRELOH WKLUG JHQHUDWLRQ V\VWHPV FKDQQHO HUURUV UHVXOW LQ KLJKO\ DQQR\LQJ GHIHFWV>@)RUWKHVHVHUYLFHVLWLVLPSRUWDQWWRDVVHVV WKHSLFWXUHDVWKDWDIIHFWVWKH4XDOLW\RI6HUYLFH

327

7KHFRQYHQWLRQDOV\QFKURQRXVPRGHOIRUGLJLWDOYLGHR LQ ZKLFK YLGHR LV UHFRQVWUXFWHG V\QFKURQRXVO\ DW WKH GHFRGHU RQ D IUDPHE\IUDPH EDVLV DVVXPHV WKDW WUDQVSRUW LV GHOD\MLWWHUIUHH +RZHYHU LQ PRGHUQ LQWHJUDWHG VHUYLFH SDFNHW QHWZRUNV VXFK DV WKH ,QWHUQHWQHWZRUNGHOD\MLWWHUYDULHVRYHUDZLGHUDQJH >@ 7KH IROORZLQJ WDEOH  VXPPDULVHVWKHPHQWLRQHG LPSDLUPHQWV 7DEOH6XPPDU\RIFRPPRQLPSDLUPHQWVIRXQGLQ GLJLWDOLPDJHDQGYLGHRV\VWHPV>@ ,PSDLUPHQW %ORFNLQHVV 

'HVFULSWLRQ 'LVWRUWLRQ RI WKH LPDJH FKDUDFWHUL]HG E\ WKH DSSHDUDQFH RI DQXQGHUO\LQJEORFNVWUXFWXUH %OXU *OREDO GLVWRUWLRQ RYHU WKH HQWLUH LPDJH FKDUDFWHUL]HG E\ UHGXFHG VKDUSQHVV RI HGJHV DQG VSDWLDO GHWDLO 5LQJLQJ $SSHDUVDVHFKRHVRIWKHKDUGHGJHV  LQ WKH SLFWXUH RU D ULSSOLQJ RI DQ HGJH &RORXU 2FFXUV DW WKH HGJHV EHWZHHQ DUHDV EOHHGLQJ ZLWKKLJKFRORXUFRQWUDVW %ODFN VTXDUH 'XH WR ORVW SDFNHWV RI GDWD LQ EORFNV WUDQVPLVVLRQ *KRVWLQJ 'HOD\HG YHUVLRQ RI WKH LPDJH DSSHDU RQ WKH VFUHHQ WR WKH OHIW RU WR WKH ULJKW GHSHQGLQJ RQ SUH RU SRVWJKRVWV 03(* LV D QHZ VWDQGDUG IRU FRGLQJ PXOWLPHGLD LQIRUPDWLRQ ZKLFK XVHV VKDSH FRGLQJ LQ SODFH RI EORFN FRGLQJ IRU REMHFWV LQ LPDJHV 2QH SXUSRVH RI VKDSH FRGLQJ LV WR HOLPLQDWH WKH DUWHIDFWV LQWURGXFHG E\ VWURQJ HGJHV ZLWK LPDJHV 1R OLWHUDWXUH FRXOG EH IRXQG DERXW SRWHQWLDO QHZ IRUPV RI DUWHIDFWV LQWURGXFHGE\VKDSHFRGLQJ



&ODVVHVRI3LFWXUH4XDOLW\ $VVHVVPHQW

7KHUH DUH WZR FODVVHV RI SLFWXUH TXDOLW\ DVVHVVPHQW PHWKRGVVXEMHFWLYHDQGREMHFWLYH

 6XEMHFWLYH7HVW0HWKRGV 6XEMHFWLYHWHVWPHWKRGVUHTXLUHKXPDQYLHZHUVWRUDWH WKHTXDOLW\RUGLIIHUHQFHLQTXDOLW\EHWZHHQWZRLPDJH RU YLGHR FOLSV ,Q PRVW WHVWLQJ VFHQDULRV WKHVH WZR FOLSVGLIIHULQWKDWRQHZLOOEHWKHVRXUFHDQGWKHRWKHU ZLOO EH SURFHVVHG LQ VRPH PDQQHU 6XEMHFWLYH DVVHVVPHQW FDQ EH D FRVWO\ DQG WLPHFRQVXPLQJ SURFHVV EXW RQH WKDW FDQ \LHOG UHSHDWDEOH UHVXOWV IRU DQ\ JLYHQ HYDOXDWLRQ HYHQ ZKHQ QRQH[SHUW YLHZHUV DUH XVHG >@ 7KLV W\SH RI DVVHVVPHQW LV SDUWLFXODUO\ QHFHVVDU\ LQ FULWLFDO VLWXDWLRQV VXFK DV ILQDO SURGXFW ZKLFK SURFHVVHV LPDJH RU YLGHR  HYDOXDWLRQ DQG VWDQGDUGL]DWLRQ SURFHVVHV ZKHUH TXDOLW\ PXVW EH DVVXUHG

328

+XPDQ REVHUYHUV DUH JHQHUDOO\ DVNHG WR UDWH YLGHR TXDOLW\ LQ WHUPV RI DQQR\DQFH ZKHUH DQQR\DQFH LV D PHDVXUH RI KRZ ³EDG´ WKH REVHUYHU WKLQNV WKH LPSDLUPHQW LV 7KH DQQR\DQFH YDOXH FRUUHODWHV ZLWK WKHVWUHQJWKRIWKHLPSDLUPHQW7DEOH  7DEOH)LYHSRLQWVFDOHRI,785HF VXEMHFWLYHDVVHVVPHQWRILPDJHV>@ 4XDOLW\  ([FHOOHQW  *RRG  )DLU  3RRU  %DG

,PSDLUPHQW  ,PSHUFHSWLEOH  3HUFHSWLEOHQRWDQQR\LQJ  6OLJKWO\DQQR\LQJ  $QQR\LQJ  9HU\DQQR\LQJ

7KH 0HDQ 2EVHUYHU¶V 6FRUH 026  LV D VXEMHFWLYH HUURU PHDVXUH DQG LV FDOFXODWHG E\ DYHUDJLQJ WKH DQQR\DQFH OHYHO IRU DOO REVHUYHUV 'RXEOH VWLPXOXV FRQWLQXRXV TXDOLW\ VFDOH '6&46  LV D PHWKRG LQ ZKLFKVRXUFHDQGSURFHVVHGYLGHRFOLSVDUHSUHVHQWHG LQSDLUVWRREVHUYHUV7KHYLGHRRULPDJHSUHVHQWDWLRQ VHTXHQFHLVUDQGRPLVHG9LHZHUVJUDGHWKHTXDOLW\RI HDFK FOLS WKHQ WKH GDWD LV SURFHVVHG LQ SDLUV 8QWLO YHU\ UHFHQW WLPHV VXEMHFWLYH PHDVXUHV VXFK DV 026 DQG '6&46 RIIHUHG WKH PRVW UHOLDEOH TXDOLW\ PHDVXUHV 6RPH LVVXHV WKDW DULVH ZLWK VXEMHFWLYH DVVHVVPHQW LQFOXGHWKHFRVWDQGWKHIDFWWKDWWKHVHPHWKRGVFDQQRW EH XVHG WR PRQLWRU YLGHR LPDJH TXDOLW\ LQ UHDO WLPH 7KLV SURFHVV UHTXLUHV VSHFLDO HTXLSPHQW DQG PDQ\ SHRSOH7UDGLWLRQDODQDORJXHREMHFWLYHPHDVXUHPHQWV ZKLOHVWLOOQHFHVVDU\DUHQRWDGHTXDWHWRPHDVXUHWKH TXDOLW\ RI V\VWHPV XVLQJ GLJLWDO FRPSUHVVLRQ 7KXV WKHUH LV D QHHG WR GHYHORS QHZ REMHFWLYH PHWKRGV LQFRUSRUDWLQJ WKH FKDUDFWHULVWLFV RI WKH KXPDQ YLVXDO V\VWHPLQFOXGLQJSHUFHSWLRQSURFHVV>@

 2EMHFWLYH7HVW0HWKRGV 2EMHFWLYHWHVWPHWKRGVGRQRWXVHKXPDQVXEMHFWVEXW UDWKHU DQDO\WLFDOO\ HYDOXDWH WKH YLGHR VLJQDO 7UDGLWLRQDODQDORJXHPHWKRGVFDQDFFXUDWHO\PHDVXUH DQG DVVHVV DQDORJXH LPSDLUPHQWV WR D YLGHR VLJQDO +RZHYHU ZLWK WKH LQWURGXFWLRQ DQG GHYHORSPHQW RI GLJLWDO WHFKQRORJLHV YLVXDOO\ QRWLFHDEOH DUWHIDFWV DUH EHLQJ PDQLIHVWHG LQ ZD\V WKDW DUH GLIIHUHQW IURP DQDORJXHDUWHIDFWV7KLVFKDQJHKDVOHGWRWKHQHHGIRU QHZ REMHFWLYH WHVW PHWKRGV 7KH QHZ REMHFWLYH PHWKRGV DQDO\VH WKH YLGHR VLJQDO LQWKHLPDJHVSDFH XWLOL]LQJ NQRZOHGJH RI WKH KXPDQ YLVXDO V\VWHP 7KHVHPHWKRGVLPSOHPHQWDQDOJRULWKPWKDWPHDVXUHV LPDJHTXDOLW\XVXDOO\EDVHGRQWKHFRPSDULVRQRIWKH VRXUFH DQG WKH SURFHVVHG VHTXHQFHV 7KHVH DOJRULWKPV UHIHUUHG WR DV PRGHOV GHVLJQ WR LQFRUSRUDWH WKH IXQFWLRQLQJ RI WKH KXPDQ YLVXDO V\VWHP ZKLOH WU\LQJ WR V\VWHPDWLFDOO\ PHDVXUH WKH SHUFHSWLEOH GHJUDGDWLRQ RFFXUULQJ LQ WKH YLGHR LPDJHU\ ,Q VRPH VLWXDWLRQV WKHVH REMHFWLYH PHWKRGV PD\ VXFFHVVIXOO\ UHSODFH WKH XVH RI VXEMHFWLYH DVVHVVPHQW

Image and Vision Computing NZ

'XHWROLPLWDWLRQVLQWKHH[LVWLQJTXDOLW\PHWULFVLWLV LPSRUWDQW WR GHYHORS TXDOLW\ PHWULFV WKDW DUH WLHG WR WKH SURSHUWLHV RI KXPDQ YLVXDO V\VWHP ,PSDLUPHQW PHWULFV KDYH EHHQ GHYHORSHG EDVHG RQ KXPDQ YLVXDO V\VWHP PRGHOV /DPEUHFKW HW DO SUHVHQWHG D VSDWLR WHPSRUDO YLVLRQ PRGHO DQG SDUDPHWHULVHG E\ SV\FKRSK\VLFDOH[SHULPHQWV>@ 6HYHUDOREMHFWLYHTXDOLW\PHWULFVKDYHEHHQGHYHORSHG IRU DEVROXWH PHDVXUHPHQW RI SLFWXUH TXDOLW\ DQG IRU XVH LQ DUWHIDFW PLWLJDWLRQ DOJRULWKPV $WWHPSWV ZHUH PDGH WR TXDQWLI\ WKH TXDOLW\ ZLWKRXW UHIHUHQFH WR WKH RULJLQDO VRXUFH )RU H[DPSOH PDQ\ UHVHDUFKHUV KDYH SURSRVHGDOJRULWKPVWRPHDVXUHEORFNLQHVV> @'LIIHUHQWFRGLQJ VFKHPHVLQWURGXFHGLIIHUHQW DUWHIDFWV PDNLQJ LW GLIILFXOW WR GHVLJQ D XQLYHUVDO REMHFWLYHTXDOLW\PRGHO7DQDQG*KDQEDULSURSRVHGD PXOWLPHWULFPRGHOFRPSULVLQJRIDSHUFHSWXDOPRGHO DQG D EORFNLQHVV GHWHFWRU 7KLV DSSURDFK FRPELQHV D SLFWXUH TXDOLW\ PRGHO IRU HDFK NLQG RI NQRZQ GLVWRUWLRQ DFFRUGLQJ WR WKH SHUFHSWXDO LPSDFW RI HDFK W\SHRILPSDLUPHQW>@



2EMHFWLYH4XDOLW\0HWULFV

7KH WHUP ³TXDOLW\ PHWULF´LVJHQHUDOO\DSSOLHGWRDQ\ SK\VLFDO DQG SV\FKRSK\VLFDO PHDVXUH RI LPDJH TXDOLW\ 2EMHFWLYH PHDVXUHV RI LPSDLUPHQW DUH FDOOHG ILGHOLW\ PHDVXUHV DQG DUH EDVHG RQ WKH GLIIHUHQFH EHWZHHQ UHIHUHQFH DQG WHVW YLGHRV %\ GHILQLWLRQ ILGHOLW\ PHDVXUHV UHTXLUH LQIRUPDWLRQ DERXW WKH UHIHUHQFH YLGHR ZKLFK FDQ EH YHU\ UHVWULFWLYH 7KH ILGHOLW\ PHDVXUHV DUH FDWHJRUL]HG DFFRUGLQJ WR WKH DPRXQW RI UHIHUHQFH LQIRUPDWLRQ WKH\ UHTXLUH )XOO UHIHUHQFH ILGHOLW\ PHDVXUHV UHTXLUH WKH HQWLUH UHIHUHQFHYLGHRDQGPD\QRWEHYHU\SUDFWLFDOIRUUHDO WLPH WUDQVPLVVLRQ DSSOLFDWLRQV 5HGXFHGUHIHUHQFH ILGHOLW\ XVHV D UHGXFHG VL]H YHUVLRQ RI WKH UHIHUHQFH ZKLOH QRQUHIHUHQFH ILGHOLW\ PHDVXUHV XVH QR UHIHUHQFHDWDOO7KHGLIIHUHQFHEHWZHHQWKHUHIHUHQFH DQGWKHGHFRGHGLPDJHSURYLGHVDQREMHFWLYHPHDVXUH

 0HDVXUHPHQWRI(UURU)XOO UHIHUHQFH &RPPRQ PHDVXUHV RI WKH HUURUV LQ VLJQDO SURFHVVLQJ DUH PHDQ VTXDUH HUURU 06(  DQG WKH SHDN VLJQDO WR QRLVHUDWLR3615 7KHUHDUHRWKHUPHDVXUHVVXFKDV WRWDO DEVROXWH HUURU 7$(  URRW PHDQ VTXDUH HUURU 506 DQGVLJQDOWRQRLVHUDWLR615  ,IWKHUHIHUHQFHVLJQDOLV[ DQGWKH\ LVGHFRGHGVLJQDO RUWKHDSSUR[LPDWLRQRI[ HDFKKDYLQJ1YDOXHVWKHQ WKHVHPHWULFVDUHGHILQHGDV 1

7$( = ∑ _ \ − [ _  L = 1

06(

 \ − [ ∑ L =

1

 





Palmerston North, November 2003

 

506 06(  3615 615 =

PD[

[



06(

∑[

  

 



∑ [ − \





 

)UHTXHQWO\WKH615RU3615PHDVXUHVDUHH[SUHVVHG LQGHFLEHOV+RZHYHUQRQHRIWKHVHPHDVXUHVLQGLFDWH KRZ WKH LPDJH ZRXOG DSSHDU WR WKH KXPDQ H\H )RU H[DPSOHGLWKHULQJLPSURYHVWKHDSSHDUDQFHRIFHUWDLQ LPDJHVEXWGHJUDGHVWKHREMHFWLYHPHDVXUHV :LWK GLJLWDO YLGHR D IXOO UHIHUHQFH LV RIWHQ QRW DYDLODEOH DW WKH UHFHLYHG HQG WR HQDEOH WKH DERYH TXDOLW\ DVVHVVPHQW PHDVXUHV WR EH XVHG +RZHYHU GLJLWDO WHFKQLTXHV HQDEOH VRPH GDWD WR EH HPEHGGHG ZLWKWKHPDLQVWUHDPDQGWKLVFDQEHXVHGIRUDSDUWLDO TXDOLW\ DVVHVVPHQW 7ZR VXFK H[DPSOHV DUH SURSRVHG E\ $W]RUL HW DO DQG &DUOL HW DOZKHUHHPEHGGHGGDWD ZLWKLQ WKH FRGHG LPDJH DFWV DV D UHIHUHQFH VLJQDO IRU ERWK HUURU GHWHFWLRQ DQG FRQFHDOPHQW WKH SURFHVV RI KLGLQJHUURU>@

 'RXEOHHQGHG4XDOLW\0HDVXUH $ GXDOHQGHG TXDOLW\ PHDVXUH FRPSDUHV WKH LQSXW VLJQDO ZLWK WKH RXWSXW VLJQDO 7R WHVW D FRPSUHVVLRQ V\VWHP WKH XQFRPSUHVVHG VRXUFH LV FRPSDUHG ZLWK WKHFRUUHVSRQGLQJFRPSUHVVHGDQGWKHQGHFRPSUHVVHG VLJQDODVVKRZQLQ)LJXUH7KLVDOORZVWKHHQFRGLQJ V\VWHP WR EH WHVWHG LQ D ODERUDWRU\ HQYLURQPHQW +RZHYHUVWUHDPVXVHGIRUWKHWHVWVKRXOGEHGHVLJQHG RUFKRVHQWRKLJKOLJKWWKHGHWDLOVRISRWHQWLDODUWHIDFWV 'XDOHQGHG PHDVXUHPHQW DOVR SUHYHQWV WKH XVH RI YLGHR SURFHVVLQJ WHFKQLTXHV OLNH VWDQGDUGV FRQYHUVLRQ DVSHFW UDWLR FRQYHUVLRQ GLJLWDO YLGHR HIIHFWV IDGHV EHWZHHQ WKH VRXUFH DQG PHDVXUHPHQW )LQDOO\ WKH VRXUFH PDWHULDO UHTXLUHPHQW PDNHV DQ\ GXDOHQGHGDOJRULWKPLPSUDFWLFDOIRUPRVWIRUGD\WR GD\TXDOLW\PHDVXUHRSHUDWLRQVHVSHFLDOO\RQPXOWLSOH FKDQQHOV &RGHU

6RXUFH

'HFRGHU

0HDVXUHPHQW

'HOD\

)LJXUH'RXEOHHQGHGSLFWXUHTXDOLW\ PHDVXUHPHQWV



7KH FRPSHQVDWLQJ GHOD\ VKRZQ PXVW SUHFLVHO\ DOLJQ WKHWZR LPDJHVWREHFRPSDUHG7KLVFDQEHGRQHE\ LQFRUSRUDWLQJ UXJJHG HDVLO\ LGHQWLILDEOH UHJLRQV LQWR WKH VRXUFH LPDJHV RU VHTXHQFHV IURP ZKLFK VSDWLDO DQG WHPSRUDO DOLJQPHQW LQIRUPDWLRQ FDQ EH UHWULHYHG HYHQ ZKHQ WKH VHTXHQFH KDV XQGHUJRQH VHYHUH GHJUDGDWLRQ 'RXEOHHQGHG PHDVXUHPHQW LV QRW ZHOO

329

VXLWHG WR PRQLWRULQJ DSSOLFDWLRQV ZKHUH WKH VRXUFH LV QHLWKHUDYDLODEOHQRUFRQWUROODEOHEXWLWLVXVHIXODVD ODERUDWRU\WRROZKHUHWKRVHUHVWULFWLRQVSRVHOHVVRID SUREOHP >@ +HQFH LW FDQEHXVHGIRUWHVWLQJFRGHF FRGHUGHFRGHU GHVLJQLQFRPSUHVVLRQWHFKQRORJLHV

 6LQJOHHQGHG4XDOLW\0HDVXUH ,Q PRVW SUDFWLFDO WUDQVPLVVLRQ DSSOLFDWLRQV LQWHUQHW EURDGFDVW FDEOH HWF D UHIHUHQFH VLJQDO LV QRW DYDLODEOH XQOHVV VWRUHG LQ WKH V\VWHP $ VLQJOHHQGHG PHDVXUH RQO\ XVHV WKH RXWSXW VLJQDO WR FDOFXODWH WKH TXDOLW\PHDVXUH)LJXUH ,WLVWKHUHIRUHHFRQRPLFDO DQG SUDFWLFDO DV LW GRHVQ W UHTXLUH WKH LQSXW VLJQDO 7KLV IDFW DOORZV PHDVXUHPHQW WR EH PDGH RI DQ\ FRGHG GDWD VWUHDP IURP DQ\ SRLQW LQ WKH FRPPXQLFDWLRQFKDLQ+RZHYHULWPD\UHTXLUHFHUWDLQ FRPSUHVVLRQ SDUDPHWHUV SUHVHQW LQ WKHRXWSXWVWUHDP 7KH UHVHDUFKHUV ZKR KDYH GHYHORSHG VXFK PHDVXUHV FODLP WKDW WKH\ DUH DFFXUDWH PHDVXUHV .QHH > @ KDVGHYHORSHGDSLFWXUHDSSUDLVDOUDWLQJ3$5 ZKLFK LV D VLQJOHHQGHG SLFWXUH TXDOLW\ PHDVXUH IRU 03(*    0HDVXUHPHQW &RGHU 'HFRGHU    6RXUFH &RGLQJLQIRUPDWLRQ    )LJXUH6LQJOHHQGHGSLFWXUHTXDOLW\PHDVXUHPHQWV

 &RPPHUFLDO6\VWHPV 7KHUH DUH RQO\ D IHZSURGXFWVGHYHORSHGE\ LQGXVWU\ WR PHDVXUH LPDJH DQG YLGHR TXDOLW\ 0RVW RI WKHVH KDYHEHHQRSWLPLVHGIRUWKHPHDVXUHPHQWRI03(* GDWD VWUHDPV 5RKGH DQG 6FKZDU] KDYH GHYHORSHG D GLJLWDO YLGHR TXDOLW\ DQDO\VHU DQG PRQLWRU ZKLFK IXQFWLRQVLQUHDOWLPHDVDQRUHIHUHQFHYLGHRTXDOLW\ DQDO\VLV V\VWHP >@ 6QHOO  :LOFR[ KDYH VHYHUDO SURGXFWV WKDW XVH WKH SLFWXUH DSSUDLVDO UDWLQJ 3$5  PHDVXUH >@ 7HNWURQL[ KDYH GHYHORSHG 03(*VFRSH WHVW V\VWHP ZKLFK FRQFHQWUDWHV PRUH RQ DQDO\VLQJ WUDQVPLVVLRQ LPSDLUPHQWV LQ 03(* WUDQVSRUW VWUHDPV>@



9LGHR4XDOLW\([SHUWV*URXS94(* 

$ JURZLQJ FRQFHUQ RI YLGHR UHVHDUFKHUV DQG EURDGFDVWHUVDOLNHLVWKHDVVXUDQFHDQGPDLQWHQDQFHRI DQDFFHSWDEOHVHUYLFHTXDOLW\OHYHOIRUWKHGLVWULEXWLRQ RI YLGHR 7KH 9LGHR 4XDOLW\ ([SHUWV *URXS94(*  LV D JURXS RI H[SHUWV IURP YDULRXV EDFNJURXQGV DQG DIILOLDWLRQV LQFOXGLQJ SDUWLFLSDQWV IURP VHYHUDO LQWHUQDWLRQDOO\ UHFRJQL]HG RUJDQL]DWLRQV ZRUNLQJ LQ WKH ILHOG RI YLGHR TXDOLW\ DVVHVVPHQW 7KH JURXS ZDV IRUPHG LQ  :LWK WKH VKLIW IURP DQDORJXH WR GLJLWDO V\VWHPV QHZ REMHFWLYH PHWKRGV DUH QHHGHG WR DVVHVV WKH UHODWHG DUWHIDFWV 7KH PDMRULW\ RI 94(* SDUWLFLSDQWV DUH DFWLYH LQ WKH ,QWHUQDWLRQDO

330

7HOHFRPPXQLFDWLRQ 8QLRQ ,78  DQG 94(* FRPELQHVWKHH[SHUWLVHDQGUHVRXUFHVIRXQGLQVHYHUDO ,78 6WXG\ *URXSV WR ZRUN WRZDUGV D FRPPRQ JRDO >@ 7KH ORQJVWDQGLQJ EHQFKPDUN XVHG IRU DVVHVVPHQW RI YLGHR LPDJHV KDV EHHQ VXEMHFWLYH DVVHVVPHQW )RU PRUHWKDQ\HDUVUHVHDUFKHUVZRUOGZLGHKDYHXVHG WKHPHWKRGVRXWOLQHGLQ,7855HFRPPHQGDWLRQ  >@ WR HYDOXDWH YLGHR TXDOLW\ LQ WHOHYLVLRQ VHUYLFHV 0RUH UHFHQWO\ WKH ,787 KDV GHYHORSHG 5HFRPPHQGDWLRQ 3 WR VWDQGDUGL]H PHWKRGV IRU PXOWLPHGLDTXDOLW\DVVHVVPHQW>@$VQRWHGEHFDXVH VXEMHFWLYH DVVHVVPHQW UHTXLUHV WKH XVH RI KXPDQ REVHUYHUVWRUDWHYLGHRVHTXHQFHVXVXDOO\VKRUWFOLSV LW LV LPSUDFWLFDO DQG LPSRVVLEOH WR XVH WKHVH PHWKRGV IRU WKH LQVHUYLFH FRQWLQXRXV HYDOXDWLRQ RI LPDJH TXDOLW\ +HQFH REMHFWLYH PHWKRGV WKDW HPXODWH WKH KXPDQ YLVXDO V\VWHP DUH UHTXLUHG 7KH ILUVW JRDO RI 94(* LV WR SURYLGH WKH EURDGFDVW WHOHFRPPXQLFDWLRQV DQG UHVHDUFK ZRUOGV ZLWK REMHFWLYHPHWKRGVIRUYLGHRLPDJHTXDOLW\HYDOXDWLRQ 6HYHUDO PRGHOV RI REMHFWLYH TXDOLW\ PHWULF DUH EHLQJ GHYHORSHG

 'LVFXVVLRQDQG&RQFOXVLRQV ,Q WKLV UHYLHZ SDSHU ZH KDYH SUHVHQWHG DQ RYHUYLHZ RI LPDJH DQG YLGHR TXDOLW\ DVVHVVPHQW WHFKQLTXHV DQG WKHLU PHULWV DQG GHPHULWV 4XDOLW\ DVVHVVPHQW LV FODVVLILHG LQWR WZR FODVVHV EDVHG RQ WKH GHJUHH RI KXPDQ LQYROYHPHQW 6XEMHFWLYH PHDVXUHV DUH EDVHG RQ+96DQGWKHUHIRUHDFFXUDWHO\UHIOHFWWKHSHUFHLYHG TXDOLW\ >@ 2EMHFWLYH PHDVXUHV QHHG WR EH VWURQJO\ FRUUHODWHG WR WKH +96 DQG DUH RIWHQ WDUJHWHG WR LGHQWLI\LQJ DQG PHDVXULQJ VSHFLILF GLJLWDO FRGLQJ DUWHIDFWV 7KH VLPSOH REMHFWLYH PHDVXUHV RI LPDJH TXDOLW\ DUH SRRUO\ FRUUHODWHG ZLWK SHUFHLYHG SLFWXUH TXDOLW\ +RZHYHU PRUH UHFHQWO\ GHYHORSHG REMHFWLYH TXDOLW\ PHWULFV VXFK DV SLFWXUH DSSUDLVDO UDWLQJ KDYH EHWWHU FRUUHODWLRQ ZLWK VXEMHFWLYH PHDVXUHV >@ $OO REMHFWLYH TXDOLW\ PRGHOV GHYHORSHG KDYH WKHLU RZQ VWUHQJWKV DQG ZHDNQHVVHV 7KH FRPSOH[LWLHV RI WKH KXPDQ YLVXDO V\VWHP LQFOXGLQJ SHUFHSWLRQ PDNH WKH JRDO RI GHVLJQLQJ VLQJOH REMHFWLYH PHDVXUH WKDW FRUUHVSRQGV WR ZKDW LV JDXJHG E\ KXPDQ REVHUYHUV HOXVLYH 2WKHU IDFWRUV ZKLFK FRPSOLFDWH VXEMHFWLYH PHDVXUHPHQWV DUH SHUVRQDO SUHIHUHQFHV DQG FXOWXUDO IDFWRUV $VWKHYROXPHRIGDWDDQGSURFHVVLQJSRZHULQFUHDVH ZKLOHDWWKHVDPHWLPHWKHXVDEOHVSHFWUXPSHUXVHULV GLPLQLVKLQJ LW LV LPSRUWDQW WKDW PRGHUQ GLJLWDO FRPPXQLFDWLRQV\VWHPVKDQGOHWKHDYDLODEOHVSHFWUXP HIILFLHQWO\ :LWK LQFUHDVLQJ SURFHVVLQJ SRZHU DYDLODEOH DW WHUPLQDO HTXLSPHQW LW LV SRVVLEOH WR LPSURYH WKH TXDOLW\ RI LPDJHV DQG YLGHR WR EH GLVSOD\HG E\ LPSURYLQJ WKH SHUIRUPDQFH RI FRGHFV 7KLVLVFRPSOLFDWHGE\HDFKQHZJHQHUDWLRQRIFRGHF LQWURGXFLQJGLIIHUHQWDUWHIDFWVWKDWZHUHQ¶WDSSDUHQWLQ

Image and Vision Computing NZ

SUHYLRXV JHQHUDWLRQV +RZHYHU WKHUH DUH QR WHVW VLJQDOVRUZRUOGVWDQGDUGIRUDQREMHFWLYHPHDVXUHRI SLFWXUH TXDOLW\ 7KHUH LV D QHHG IRU DQ HDV\ WR XVH SRUWIROLR RI WHVW VLJQDOV WR EH XVHG ZLWK D FRGHF WR DVVHVVLWVSHUIRUPDQFH 5HVHDUFKHUV LQ WKH ILHOG VWLOO IDFH REVWDFOHV ZKHQ LW FRPHVWRHYDOXDWHWKHTXDOLW\RIWKHLULPDJHDQGYLGHR FRGLQJ ZRUN DV QR TXDOLW\ PHWULF LV XQLYHUVDOO\ DFFHSWHG7KH9LGHR4XDOLW\([SHUW*URXSLVZRUNLQJ WRZDUGV VXFK PHWULFV ZLWK WKH SDUWLFLSDWLRQ RI PDQ\ ,QWHUQDWLRQDO 7HOHFRPPXQLFDWLRQ 8QLRQ ,78  VWXG\ JURXSV

 )XWXUHZRUN 7KHDXWKRUVDUHGHYHORSLQJDVHWRIWHVWVLJQDOVWREH XVHG LQ WKH DVVHVVPHQW RI SLFWXUH TXDOLW\ RI VSHFLILF FRGHFV 7KHVH WHVW VLJQDOV ZLOO EH GHVLJQHG WR GHPRQVWUDWHWKHSUHVHQFHRIDUWHIDFWVDQGWRDOORZWKDW H[WHQW WR EH PHDVXUHG 7KH WHVW VLJQDOV DUH WR EH GHYHORSHG WR EH DQDORJRXV WR FRORXU EDUV DQG PXOWL EXUVW VLJQDOV XVHG LQ DQDORJXH YLGHR 7KHVH WHVW VLJQDOV FRXOG ZRUN ZLWK DYDLODEOH REMHFWLYH TXDOLW\ PHWULFV RU DQ\ REMHFWLYH TXDOLW\ PHWULF WKDW WKH DXWKRUV PD\ GHYHORS LQ WKH SURFHVV 7KH QHZ WHVW VLJQDOVDUHYHU\OLNHO\WREHFRGHFRUVWDQGDUGVSHFLILF DV WKH\ ZLOO EH GHVLJQHG ZLWK D NQRZOHGJH RI WKH VSHFLILF PHFKDQLVPV DQG ZHDNQHVVHV LQKHUHQW LQ WKH FRGHFRUVWDQGDUG

 5HIHUHQFHV >@ 3XQFKLKHZD $ DQG %DLOH\ '* $UWHIDFWV LQ ,PDJH DQG 9LGHR 6\VWHPV &ODVVLILFDWLRQ DQG 0LWLJDWLRQ 3URFHHGLQJV RI ,PDJH DQG 9LVLRQ &RPSXWLQJ 1HZ =HDODQG  SS   >@ )DULDV&4GH7KHVLVGLVVHUWDWLRQSURSRVDORI 0\OHQH & 4 GH )DULDV 0D\   ³+RPH SDJH´ KWWSZZZHQJUXFVEHGXaP\OHQH YLVLWHGRQ >@ ,78 5HFRPPHQGDWLRQ 3 3ULQFLSOHV RI D 5HIHUHQFH,PSDLUPHQWV\VWHPIRU9LGHR  >@ )HQLPRUH & /LEHUW - DQG 5RLWPDQ 3 0RVTXLWRQRLVHLQ03(*FRPSUHVVHGYLGHRWHVW SDWWHUQV DQG PHWULFV 3URFHHGLQJV RI 63,( +XPDQ 9LVLRQ DQG (OHFWURQLF ,PDJH 9 6DQ -RVH&DOLIRUQLD  >@ 'XEXF & %RXGUHDX ' DQG 3DWHQDXGH ) 7KH 'HVLJQ DQG 6LPXODWHG 3HUIRUPDQFH RI D 0RELOH 9LGHR7HOHSKRQ\$SSOLFDWLRQIRU6DWHOOLWH7KLUG *HQHUDWLRQ :LUHOHVV 6\VWHP ,((( 7UDQVDFWLRQ RQ0XOWLPHGLD   >@ &KDQJ @:ROI 6 DQG 3LQVRQ 0 + ,Q6HUYLFH 3HUIRUPDQFH 0HWULFV IRU 03(* 9LGHR 6\VWHPV 3URFHHGLQJV RI PDGH WR PHDVXUH  ± 0HDVXUHPHQW 7HFKQLTXHV RI WKH 'LJLWDO $JH 7HFKQLFDO6HPLQDU6ZLW]HUODQG  >@/DPEUHFKW & - 9DQ 'HQ % 9HUVFKHXUH 2 8UEDLQ-DQG7DVVLQ)4XDOLW\$VVHVVPHQWRI ,PDJH )HDWXUHV LQ 9LGHR &RGLQJ ,((( ,QWHUQDWLRQDO &RQIHUHQFH RQ ,PDJH 3URFHVVLQJ 6DQWD%DUEDUD,  >@7DQ . 7 DQG *KDQEDUL 0 $ 0XOWL0HWULF 2EMHFWLYH 3LFWXUH4XDOLW\ 0HDVXUHPHQW 0RGHO IRU 03(* 9LGHR ,((( 7UDQVDFWLRQ RQ &LUFXLWV DQG6\VWHPV   >@$W]RUL / 1DWDOH )*% DQG 'H 3HUUD &  $ 6SDWLR7HPSRUDO &RQFHDOPHQW 7HFKQLTXH 8VLQJ %RXQGDU\ 0DWFKLQJ $OJRULWKP DQG 0HVK%DVHG :DUSLQJ %0$0%:  ,((( 7UDQVDFWLRQ RQ 0XOWLPHGLD   >@&DUOL 0 %DLOH\ ' )DULDV 0 DQG 0LWUD 6 . (UURU &RQWURO DQG &RQFHDOPHQW IRU 9LGHR 7UDQVPLVVLRQ XVLQJ 'DWD KLGLQJ ,QWHUQDWLRQDO V\PSRVLXP RQ ZLUHOHVV SHUVRQDO PXOWLPHGLD FRPPXQLFDWLRQV +RQROXOX +DZDLL SS    >@.QHH07KH3LFWXUH$SSUDLVDO5DWLQJ3$5  D VLQJOHHQGHG SLFWXUH TXDOLW\ PHDVXUH IRU 03(* SUHVHQWHG DW ,QWHUQDWLRQDO %URDGFDVW &RQYHQWLRQLQ$PVWHUGDP  >@5KRGH DQG 6FKRZ] +RPH 3DJH KWWSZZZUVGGHYLVLWHGRQ >@7HNWURQL[ PHDVXUHPHQW SDJH KWWSZZZWHNWURQL[FRP0HDVXUHPHQW YLVLWHG RQ >@9LGHR 4XDOLW\ ([SHUW *URXS +RPH 3DJH KWWSZZZYTHJRUJYLVLWHGRQ

331