In Quest of Image Semantics

,Q4XHVWRI,PDJH6HPDQWLFV $UH:H/RRNLQJIRU,W8QGHUWKH5LJKW/DPSSRVW" Emanuel Diamant VIDIA-mant, P.O. Box 933, 55100 Kiriat Ono, Israel HPDQ O#  Q H W  L O

Abstract.,QWKHODVW\HDUVZHZLWQHVVDGUDPDWLFJURZWKRIUHVHDUFKIRFXVHGRQ VHPDQWLF LPDJH XQGHUVWDQGLQJ ,QGHHG ZLWKRXW XQGHUVWDQGLQJ LPDJH FRQWHQW VXFFHVVIXO DFFRPSOLVKPHQW RI DQ\ LPDJHSURFHVVLQJ WDVN LV VLPSO\ LQFUHGLEOH 8SWRWKHUHFHQWWLPHVWKHXOWLPDWHQHHGIRUVXFKXQGHUVWDQGLQJKDVEHHQPHW E\ WKH NQRZOHGJH WKDW D GRPDLQ H[SHUW RU D YLVLRQ V\VWHP VXSHUYLVRU KDYH FRQWULEXWHG WR HYHU\ LPDJHSURFHVVLQJ DSSOLFDWLRQ 7KH DGYHQW RI WKH ,QWHUQHW KDVGUDVWLFDOO\FKDQJHGWKLVVLWXDWLRQ,QWHUQHWVRXUFHVRIYLVXDOLQIRUPDWLRQDUH GLIIXVHGDQGGLVSHUVHGRYHUWKHZKROH:HEVRWKHGXW\RILQIRUPDWLRQFRQWHQW GLVFRYHU\ DQG HYDOXDWLRQ PXVW EH UHOHJDWHG QRZ WR DQ LPDJH XQGHUVWDQGLQJ DJHQW D PDFKLQH RU D FRPSXWHU SURJUDP  FDSDEOH WR SHUIRUP LPDJH FRQWHQW DVVHVVPHQW DW D UHPRWH LPDJH ORFDWLRQ 'HYHORSPHQW RI &RQWHQW %DVHG ,PDJH 5HWULHYDO &%,5  WHFKQLTXHV ZDV D ULJKW PRYH LQ D ULJKW GLUHFWLRQ ODXQFKHG DERXW WHQ \HDUV DJR 8QIRUWXQDWHO\ YHU\ OLWWOH SURJUHVV KDV EHHQ PDGH VLQFH WKHQ7KHUHDVRQIRUWKLVFDQEHVHHQLQDUDQNRIORQJODVWLQJPLVFRQFHSWLRQV WKDW&%,5GHVLJQHUVDUHFRQWLQXLQJWRDGKHUHWR,KRSHP\DUJXPHQWVZLOOKHOS WKHPWRFKDQJHWKHLUPLQGV

1 Introduction 7KHUH LV D ZLGHVSUHDG DVVXPSWLRQ WKDW ZH RZH WKH FXUUHQW XSULVH RI WKH LQWHUHVW LQ LPDJHVHPDQWLFVWRWKHXQSUHFHGHQWHGVXFFHVVRIWKH,QWHUQHW DQG:RUOG :LGH :HE GHSOR\PHQW ZKLFK PDGH KXJH DPRXQWV RI LQIRUPDWLRQ LQFOXGLQJ ORWV RI YLVXDO LQIRUPDWLRQ  HDVLO\ DFFHVVLEOH DQG DIIRUGDEOH RYHU WKH HQWLUH ZRUOG &HUWDLQO\ WKLV DVVHUWLRQ LV QRW FRUUHFW 8QGHUVWDQGLQJ LPDJH LQIRUPDWLRQ FRQWHQW ZDV DOZD\V FRQVLGHUHG D FUXFLDOO\ LPSRUWDQW SUHUHTXLVLWH IRU DQ\ LPDJHSURFHVVLQJ WDVN $Q\ VXFFHVVIXODFFRPSOLVKPHQWRIVXFKDWDVNFDQQRWEHHYHQLPDJLQHGZLWKRXWKLJK OHYHO NQRZOHGJHPHGLDWLRQRULQRWKHUZRUGVZLWKRXWXQGHUVWDQGLQJLPDJHVHPDQWLFVDQG VSHFLILFPHDQLQJRILWVFRQVWLWXWLQJSDUWV 7KH ³SODFH´ RI LPDJH XQGHUVWDQGLQJ LQ WKH ZKROH IUDPH RI LPDJH LQIRUPDWLRQ SURFHVVLQJZDVVHWXSPRUHWKDQ\HDUVDJRE\WKHFODVVLFDOZRUNVRI7UHLVPDQ>@ 0DUU>@DQG%LHGHUPDQ>@7UHLVPDQ¶VERWWRPXSWRSGRZQSURFHVVLQJSDUDGLJPLV UHJDUGHGDVWKHPRVWUHFRJQL]HGLQFDUQDWLRQRIWKHLGHD$FFRUGLQJWRLWSURFHVVLQJRI LPDJH LQIRUPDWLRQ FRQWHQW LV DVVXPHG WR EH DQ LQWHUDFWLRQ DQG DQ LQWHUSOD\ RI WZR LQYHUVHO\ GLUHFWHG LPDJH SURFHVVLQJ VWUHDPV 2QH LV DQ XQVXSHUYLVHG ERWWRP XS GLUHFWHG SURFHVV RI LQLWLDO LPDJH LQIRUPDWLRQ SLHFHV GLVFRYHU\ DQG ORFDOL]DWLRQ WKH

VRFDOOHGORZOHYHOSURFHVVLQJVWUHDP 7KHRWKHULVDVXSHUYLVHGWRSGRZQGLUHFWHG SURFHVVZKLFKFRQYH\VWKHUXOHVDQGWKHNQRZOHGJHQHHGHGWRJXLGHWKHOLQNLQJDQG ELQGLQJ RI WKH GLVMRLQW ORZOHYHO LQIRUPDWLRQ SLHFHV LQWR SHUFHSWXDOO\ PHDQLQJIXO LPDJH REMHFWV 7KDW LV WKH KLJKOHYHO SURFHVVLQJ VWUHDP DVVRFLDWHG ZLWK LPDJH XQGHUVWDQGLQJDQGFRJQLWLYHLPDJHSHUFHSWLRQ :KLOH WKH LGHD RI ORZOHYHO SURFHVVLQJ IURP WKH YHU\ EHJLQQLQJ ZDV REYLRXV DQG LQWXLWLYHO\ DSSHDOLQJ WKHUHIRUH HYHQ WRGD\ WKH PDLQVWUHDP RI LPDJH SURFHVVLQJ LV RFFXSLHG PDLQO\ ZLWK ORZOHYHO SL[HORULHQWHG FRPSXWDWLRQV  WKH HVVHQFH RI KLJKOHYHOSURFHVVLQJZDVDOZD\VREVFXUHP\VWHULRXVDQGXQGHILQHG7KHSDUDGLJP VDLGQRWKLQJDERXWWKHURRWVRIKLJKOHYHONQRZOHGJHRULJLQDWLRQRUDERXWWKHZD\LW KDV WR EH LQFRUSRUDWHG LQWR WKH LQWURGXFWRU\ ORZOHYHO SURFHVVLQJ 8QWLO QRZ KRZHYHU WKH SUREOHP ZDV XVXDOO\ E\SDVVHG E\ FDSLWDOL]LQJ RQ WKH H[SHUW GRPDLQ NQRZOHGJHDGDSWHGWRHDFKDQGHYHU\DSSOLFDWLRQFDVH,WLVQRWVXUSULVLQJWKHUHIRUH WKDWWKHZKROHUHDOPRILPDJHSURFHVVLQJKDVEHHQDQGFRQWLQXHVWREH IUDJPHQWHG DQGVHJPHQWHGDFFRUGLQJWRKLJKOHYHONQRZOHGJHFRPSHWHQFHRIWKHGRPDLQH[SHUWV )RU WKLV UHDVRQ ZH KDYH WRGD\ PHGLFDO LPDJLQJ DHURVSDFH LPDJLQJ LQIUDUHG ELRORJLF XQGHUZDWHU JHRSK\VLFV UHPRWH VHQVLQJ PLFURVFRS\ UDGDU ELRPHGLFDO ;UD\DQGVRRQ³LPDJLQJV´ 7KH DGYHQW RI WKH ,QWHUQHW ZLWK KXJH YROXPHV RI LQIRUPDWLRQ LQFOXGLQJ YDULRXV IRUPV RI YLVXDO LQIRUPDWLRQ  VFDWWHUHG RYHU WKH ZHE UDLVHG DQ XUJHQW GHPDQG IRU PRUH JHQHUDO PHDQV RI LPDJH VHPDQWLFV UHFRYHU\ FDSDEOH WR KDQGOH YLVXDO LQIRUPDWLRQ LQ D KXPDQOLNH LQWHOOLJHQW PDQQHU DQG DW UHPRWH LPDJH ORFDWLRQV +RZHYHU GHSULYHG RI DQ\ UHDVRQDEOH VRXUFHV RI WKH QHHGHG KLJKOHYHO NQRZOHGJH DQG WUDSSHG E\ WKH WHQHWV RI ERWWRPXSWRSGRZQ LPDJH SURFHVVLQJ SDUDGLJP  FRPSXWHUYLVLRQGHVLJQHUVDUHIRUFHGWRSURFHHGRQO\LQRQHSRVVLEOHGLUHFWLRQ±WRWU\ WRGHULYHWKHKLJKOHYHONQRZOHGJHIURPWKHDYDLODEOHORZOHYHOLQIRUPDWLRQSLHFHV 6RPHWKHRUHWLFDOZRUNVLQELRORJLFDODQGFRPSXWHUYLVLRQKDYHEHHQGRQHLQRUGHU WR VXSSRUW DQG WR MXVWLI\ WKLV HQWHUSULVH 7ZR DSSURDFKHV DUH SUHYDOHQW FKDRWLF DWWUDFWRUVDSSURDFK>@>@>@>@>@ DQGVDOLHQF\DWWHQWLRQPDSDSSURDFK>@ >@>@>@ %RWKSUHVXPHD6KDQQRQOLNHVHQVHRILQIRUPDWLRQZKLFKLVLQGHHG QDWXUDO IRU D ORZOHYHO ERWWRPXS LPDJH SURFHVVLQJ DUUDQJHPHQW  %RWK DUH FRPSXWDWLRQDOO\ H[SHQVLYH %RWK GHILQLWHO\ YLRODWH WKH EDVLF DVVXPSWLRQ DERXW WKH SULQFLSDO UROH RI KLJKOHYHO NQRZOHGJH LQ ERWWRPXS ORZOHYHO SURFHVVLQJ %XW WKH SUHVVXUHIURP,QWHUQHWSURYLGHUVDQGXVHUVLVVRKLJKWKHODFNRIRWKHUSRVVLELOLWLHVWR UHDFK WKH QHHGHG LQIRUPDWLRQ LV VR GHVSHUDWH WKDW QR RWKHU RSWLRQV DUH OHIW WR &%,5 GHVLJQHUV 7KHUHIRUH DOO FRQWHPSRUDU\ &%,5 GHYHORSPHQWV DUH FRQWLQXLQJ WR PRYH IRUZDUG ZLWKRXW ERWKHULQJ WKHPVHOYHV DERXW WKH ODFN RI FRPPRQ VHQVH RU D ILUP WKHRUHWLFDOMXVWLILFDWLRQIRUWKHLUNHHQDQGLQH[KDXVWLEOHHIIRUWV

7KHKDVWHLVGHYLO¶VDFFRPPRGDWLRQ $VLWZDVDOUHDG\PHQWLRQHGDERYHWKHXUJHQWQHHGWRKDQGOHLPDJHVRQWKH:HELQD KXPDQOLNHLQWHOOLJHQWPDQQHUZKLOHWKHGXWLHVRIWKLVPDQDJHPHQWDUHGHOHJDWHGWR DQDXWRQRPRXVO\SHUIRUPLQJUHPRWHDJHQWKDVLQVSLUHGDZDYHRIH[WHQVLYHUHVHDUFK DQGGHYHORSPHQWDFWLYLWLHVNQRZQDV&%,5GHYHORSPHQWV7KHZDYHLVODVWLQJDERXWD

GHFDGH ORQJ WLPH SHULRG $ SDOSDEOH XSULVH LQ WKH DFWLYLWLHV KDV KDSSHQHG VLQFH WKH ZRUN JRW WKH DXVSLFH RI WKH (8 ,67 3URJUDPPH $ VHULHV RI H[FHOOHQW IROORZ XS UHSRUWV GHSLFWLQJ WKH VWDWHRIWKHDUW DQG WKH SURJUHVV PDGH DW GLIIHUHQW SURJUDP VWDJHV KDYH EHHQ UHJXODUO\ SXEOLVKHG GXULQJ WKLV WLPH >@ >@ >@ >@ >@ >@ >@ >@ >@ 7KHUH LV QR QHHG WKXV WR UHSHDW DQG WR VFUXWLQL]H DJDLQ WKHLU DFFRXQWV0XFKPRUHLQWHUHVWLQJZRXOGEHWRWDNHWKHEXONRIWKLVPDWWHUDQGWRXVHLW LQ RUGHU WR DQDO\]H DQG WR XQGHUVWDQG WKH UHDVRQV IRU D VXVSLFLRXVO\ ORQJ GHOD\ LQ DFKLHYLQJWKHHQWHUSULVHILQDOJRDOV $UHSHDWLQJPRWLIWKDWFURZQVDOPRVWHYHU\RQHRIWKHDERYHPHQWLRQHGUHSRUWVLV 'HVSLWHWKHHIIRUWVDQGKDUGZRUNLQYHVWHGLQ&%,5GHYHORSPHQW³PRUHZRUNQHHGV WREHGRQHLQRUGHUWREHDEOHWRDXWRPDWLFDOO\H[WUDFWREMHFWLYHVHPDQWLFVIURP:HE SDJHVZKLFKZRXOGDOORZWKHUHWULHYDORILPDJHVRQWKHEDVLVRIKLJKOHYHOFRQFHSWV´ >@SDJH  ,W VHHPV WKDW WKH SKUDVH LQ WKH TXRWDWLRQ PDUNV DOUHDG\ FRQWDLQV WKH FOXH IRU WKH DERYH UDLVHG TXHVWLRQ :H PHDQ D VSHFLILF SDUW RI WKH SKUDVH ³WR DXWRPDWLFDOO\ H[WUDFW REMHFWLYH VHPDQWLFV´ :H ZLOO LJQRUH IRU D ZKLOH WKH GRXEWIXO GHILQLWLRQ ³REMHFWLYH VHPDQWLFV´ XVXDOO\ VHPDQWLFV LV FRQVLGHUHG WR EH VXEMHFWLYH  :H ZLOO FRQFHQWUDWHRQWKHH[SUHVVLRQ³VHPDQWLFVH[WUDFWLRQ´ZKLFKH[SOLFLWO\DQGLPSOLFLWO\ SUHVHQW LQ HYHU\RQH RI WKH UHSRUWV MXVW PHQWLRQHG DERYH DV ZHOO DV LQ PDQ\ RWKHU SXEOLVKHG GRFXPHQWV UHOHYDQW WR WKH VXEMHFW 7KLV H[SUHVVLRQ UHOLDEO\ UHSUHVHQWV WKH JHQHUDOZD\RIWKLQNLQJGRPLQDWLQJWRGD\ WKHILHOG RI FRQWHQWEDVHG LPDJH UHWULHYDO UHVHDUFKDQGGHYHORSPHQW$FFRUGLQJWRLWFRQWHQWEDVHGLPDJHUHWULHYDOSKLORVRSK\ XQFRQGLWLRQDOO\ DVVXPHV WKDW LPDJH VHPDQWLFV FDQ EH H[WUDFWHG IURP D JLYHQ LPDJH LQRQHZD\RUDQRWKHU GHVSLWHDOONQRZQDQGZHOOUHFRJQL]HGGLIILFXOWLHV>@ /RJLFDOO\ZKDWIROORZVIURPWKLVLVVHPDQWLFVFDQEHH[WUDFWHGIURPDQLPDJHRQO\ LI LW EHORQJV WR LW WKDW LV LI LW LV DQ LPDJH SURSHUW\ UHVLGHQW SUHVHQW QDWLYH WR DQ LPDJH  %XWWKLVLVQRWWUXH7KLVFRQWUDGLFWVDOOWKHRULHVHJ+HUPDQQYRQ+HOPKROW]>@ DQGHYHQFRPPRQVHQVHDVVHUWVZKLFKFODLPWKDWLPDJHVJHWWKHLUVHPDQWLFPHDQLQJ DVDUHVXOWRIDQDFWRILPDJHLQWHUSUHWDWLRQ:KRFDUULHVRXWWKLVLQWHUSUHWDWLRQ"7KH DQVZHUFDQEHRQO\RQHWKHKXPDQREVHUYHUZKRZDWFKHVWKHLPDJH1RERG\HOVHFDQ GR WKLV LQVWHDG :KDW IROORZV IURP WKLV LPPHGLDWHO\ LV WKDW VHPDQWLFV LV DQ H[FHSWLRQDOO\ KXPDQ¶V SURSHUW\ ,W LV KXPDQ¶V GXW\ DQG XQGHQLDEOH SULYLOHJH WR SURYLGHWKHLPDJHZLWKLWVVHPDQWLFV 7KLVFKDQJHVWKHVLWXDWLRQUDGLFDOO\ &HUWDLQO\,DPQRWWKHILUVWZKRSDLGDWWHQWLRQWRWKLVSHFXOLDULW\±WRWKHEHVWRIP\ NQRZOHGJHLWZDV6DQWLQLHWDO>@ZKRFRLQHGWKHQRWLRQLPDJHVDUHHQGRZHGZLWK WKHLU VHPDQWLFV 5HFHQWO\ +XGHORW HW DO >@ KDYH UHLQYHQWHG DQG UHLWHUDWHG WKLV GHILQLWLRQ +RZHYHU WKH UHVW RI WKH UHVHDUFK FRPPXQLW\ FRQWLQXHV WR DGKHUH WR WKH IDXOW\DVVXPSWLRQWKDWVHPDQWLFVLVDQLQKHUHQWSURSHUW\RIDQLPDJHDQGWKXVFDQEH H[WUDFWHGIURPLWRURWKHUZLVHWUHDWHGDVDEXLOWLQSURSHUW\RILPDJHGDWD &KDQJLQJWKHPLQGVDQGDFFHSWLQJDGLIIHUHQWSRLQWRIYLHZZLOOGHILQLWHO\VKDNHXS WKH &%,5 GHVLJQ SKLORVRSK\ )URP QRZ RQ LW VKRXOG EH WLHG DQG URRWHG LQ WKH QHXURELRORJLFDO DQG QHXURSK\VLRORJLFDO VWXGLHV RI KXPDQ EUDLQ QRW LQ SXUH PDWKHPDWLFDO DQG VWDWLVWLFDO DQDO\VHV :H ZLOO FRQWLQXH WR VFUXWLQL]H WKH SRVVLEOH LPSOLFDWLRQVRIWKLVVKDNHXSEXWEHIRUHWKDWZHZRXOGOLNHWRH[DPLQHWKHUHVWRIWKH EDVLFDVVXPSWLRQXQGHUSLQQLQJWKH&%,5GHVLJQVWRGD\

:KDWZHKDYHLQPLQGLVWKHFODVVLFDOIRUPXODWKDWWKHKLJKOHYHOLPDJHVHPDQWLFV KDVWREHGHULYHGIURPWKHDYDLODEOHORZOHYHOLQIRUPDWLRQIHDWXUHV$JODQFHRQDOLVW RIVRPHUHSUHVHQWDWLYHUHIHUHQFHVZLOOFRQILUPWKDW,GRQRWH[DJJHUDWHWKHXELTXLW\RI WKLV DWWLWXGH LQ WKH FRQWHPSRUDU\ &%,5 GHVLJQV >@ >@ >@ >@ 3OHDVH UHPHPEHU WKDW WKHVH DUH H[DPSOHV WKDW HPSKDVL]H RQO\ D VSHFLILF ORZOHYHO±WR± KLJKOHYHO SURFHVVLQJ SDVVDJH 7KH EXON RI WKH HYLGHQFH LV LQ WKH UHJXODU UHSRUWV DOUHDG\PHQWLRQHGDERYH  $ KLGGHQ ORJLFDO LQFRQVLVWHQF\ FDQ EH UHYHDOHG DOVR LQ WKLV DVVXPSWLRQ LQ WKHLU HYHU\GD\OLYHVSHRSOHSHUFHSWWKHREVHUYDEOHREMHFWVDOZD\VDVDZKROH6HPDQWLFVLV JUDQWHGWRDQREMHFWDOZD\VLQLWVHQWLUHW\QRWWRLWVORZOHYHOFRPSRQHQWVRUIHDWXUHV )RU D ORQJ WLPH HYLGHQFH DFFXPXODWHG IURP SV\FKRSK\VLFDO VWXGLHV KDV VKRZQ WKDW SHRSOH FDQ DQDO\]H FRPSOH[ YLVXDO VFHQHV LQ H[WUHPHO\ VKRUW WLPH LQWHUYDOV ORZHU WKDQ PV $W VXFK VKRUW WLPHV RQO\ LPSUHFLVH DQG FRDUVHVFDOH UHSUHVHQWDWLRQV FRXOG EH WDNHQ LQWR DFFRXQW DQ\ DQDO\VLV RI ILQHVFDOH ORZOHYHO LPDJH IHDWXUHV LV VLPSO\LPSRVVLEOH>@>@ ,W ZLOO EH LQWHUHVWLQJ WR UHFDOO WKDW DQ ³RSSRVLWLRQ´ WR WKH ORZOHYHO ERWWRPXS SURFHVVLQJZKLFKKDVDOZD\VLQVSLUHGWKH&%,5GHVLJQV KDVFRPHLQWREHLQJDOPRVW VLPXOWDQHRXVO\ ZLWK WKH LQDXJXUDWLRQ RI WKH ERWWRPXS SURFHVVLQJ LGHD 1DYRQ¶V ³IRUHVW EHIRUH WKH WUHHV´ K\SRWKHVLV >@ DQG &KHQ¶V ³JOREDOWRORFDO WRSRORJLFDO PRGHO´ RI SHUFHSWXDO RUJDQL]DWLRQ >@ SXEOLVKHG UHVSHFWLYHO\ LQ  DQG  KDYHGHFODUHGDOPRVW\HDUVDJRWKDW³JOREDOWRSRORJLFDOSHUFHSWLRQLVSULRUWRWKH SHUFHSWLRQRIRWKHUIHDWXUDOSURSHUWLHV´>@7KHSUREOHPLVWKDWDOOWKHVHLPSRUWDQW LQVLJKWV KDYH KDSSHQHG LQ ELRORJLFDO YLVLRQ UHVHDUFK ZKLOH FRPSXWHU YLVLRQ GHYHORSHUVDUHLQGLIIHUHQWWRVXFKWULQNHWV 1RZ,KDYHVRPHGLIILFXOWLHVWRH[SODLQZK\WKHFRQFHSWRIWKH6HPDQWLF*DSVR ZLGHVSUHDGDQGSRSXODUDPRQJ&%,5GHYHORSHUVLVWKHQH[WIDXOW\LWHPLQP\OLVWRI &%,5GHVLJQPLVFRQFHSWLRQV,QWKHOLJKWRIZKDWZDVDOUHDG\VDLGDERYHLWVHHPVWR PHWKDWWKHVHPDQWLFJDSVLPSO\GRHVQRWH[LVW'HILQLWHO\,ZRXOGEHLQWURXEOHLI, ZRXOGKDYHWRSRLQWRXWZKHUHLVWKHVHPDQWLFJDSEHWZHHQWKHPRRQRI$ODEDPDDQG WKHIDPRXV%UHFKW:HLOO¶VVRQJDERXWLW6R,ZLOOOHDYHWKLVWKHPHXQGHYHORSHGDQG ZLOOVKLIWWRWKHQH[WVHFWLRQRIRXUGLVFXVVLRQ

3 The time is rape to change your minds 'DYLG 0DUU ZDV WKH ILUVW ZKR KDV DSSURDFKHG YLVLRQ DV DQ LQIRUPDWLRQ SURFHVVLQJ FRPSXWDWLRQDOSURFHVV$OWKRXJKLQWKHODWHVLWVHHPHGWRKLPWKDWYLVLRQV\VWHP FRPSXWDWLRQVDUHHQJDJHGRQO\ZLWKORZOHYHOSURFHVVLQJ,Q>@KHZULWHV³,QWKH HDUO\VWHSVRIWKHDQDO\VLVRIDQLPDJHWKHUHSUHVHQWDWLRQVXVHGGHSHQGPRUHRQZKDW LWLVSRVVLEOHWRFRPSXWHIURPDQLPDJHWKDQRQZKDWLVXOWLPDWHO\GHVLUDEOH´ ,DSSUHFLDWH0DUU¶VLQVLJKWVZKLFKKDYHUXOHGFRPSXWHUYLVLRQGHYHORSPHQWDOPRVW IRU  \HDUV EXW , ZRXOG OLNH , GDUH  WR SURSRVH D GLIIHUHQW YLHZ RQ WKH VXEMHFW &RQVLGHULQJ YLVXDO LQIRUPDWLRQ SURFHVVLQJ , ZRXOG OLNH WR SURSRVH WKH IROORZLQJ GHILQLWLRQRIWKHQRWLRQ³YLVXDOLPDJHLQIRUPDWLRQ´,PDJHLQIRUPDWLRQLVFRPSULVHG OLNHDOORWKHUIRUPVRILQIRUPDWLRQ RIWZRSDUWV7KHILUVWLVWKHPHDQLQJOHVVSK\VLFDO REMHFWLYH LQIRUPDWLRQZKLFKLVDGHVFULSWLRQRIYLVLEOHGDWDVWUXFWXUHVGLVFHUQDEOHLQ

DQLPDJH7KHVHFRQG±LVDKXPDQSURGXFHGLQWHUSUHWDWLRQRIWKLVLQIRUPDWLRQ7KDW PHDQVDKLJKOHYHOGHVFULSWLRQRIWKHPHDQLQJRIWKHSK\VLFDOLQIRUPDWLRQSUHVHQWLQ DQLPDJHZKLFKHVVHQWLDOO\LVWKHLPDJHVHPDQWLFV , FDPH DFURVV WKH SUREOHPV RI YLVXDO LQIRUPDWLRQ WUHDWPHQW DERXW ILYH \HDUV DJR 6RPHUHVXOWVRIP\LQYHVWLJDWLRQDUHDOUHDG\SXEOLVKHG>@>@6RPHUHVXOWV,DP WU\LQJ WR SXVK QRZ 7DNLQJ WKH SURSRVHG GHILQLWLRQ DV D VWDUW SRLQW , ZRXOG OLNH WR HPEDUNRQDPRUHUHDVRQDEOH&%,5GHVLJQDSSURDFK 3.1 Physical information representation +RZ ELRORJLFDO YLVLRQ V\VWHPV DFTXLUH DQG SURFHVV WKHLU LQSXW LQIRUPDWLRQ , KDYH OHDUQHGTXLWHODWHZKHQ,KDGDOUHDG\GHYHORSHGP\RZQDSSURDFKWRWKHHQWDQJOHG DQG LQWULJXLQJ SUREOHP RI LPDJH LQIRUPDWLRQ FRQWHQW GLVFRYHU\ DQG HYDOXDWLRQ , ORRNHG IRU VRPH MXVWLILFDWLRQV IRU P\ UHVXOWV DQG ZDV SOHDVHG WR ILQG RXW WKDW VXFK MXVWLILFDWLRQVDUHLQGHHGH[LVWHQW,WZDVUHDOO\VXUSULVLQJEHFDXVHP\VROXWLRQZDVIDU DZD\ IURP ELRORJ\ ,Q P\ UHVHDUFK , ZDV LQVSLUHG E\ WKH LQVLJKWV RI 6RORPRQRII¶V ,QIHUHQFH 7KHRU\ &KDLWLQ¶V $OJRULWKPLF ,QIRUPDWLRQ 7KHRU\ DQG .ROPRJRURY¶V &RPSOH[LW\ 7KHRU\ >@ &DSLWDOL]LQJ RQ WKHP , KDYH GHYHORSHG P\ RZQ ZD\ WR KDQGOH LPDJH LQIRUPDWLRQ FRQWHQW LQLWLDOO\ UHSUHVHQWHG DV UDZ LPDJH GDWD ,Q VLPSOH ZRUGVP\DSSURDFKORRNVDVIROORZV .ROPRJRURYKDVFRPELQHGWKHWKHRU\RIFRPSXWDWLRQDQGDFRPELQDWRULDODSSURDFK WR LQIRUPDWLRQ WKHRU\ LQ D SURSRVDO IRU DQ REMHFWLYH DQG DEVROXWH GHILQLWLRQ RI WKH LQIRUPDWLRQ FRQWDLQHG LQ DQ LQGLYLGXDO ILQLWH REMHFW &RQWUDU\ WR WKH 6KDQQRQ¶V GHILQLWLRQ ZKLFK JLYHV DQ LQWHJUDWHG DYHUDJHG RYHU WKH ZKROH VLJQDO HQVHPEOH PHDVXUH RI WKH FRQWDLQHG LQIRUPDWLRQ  )ROORZLQJ .ROPRJRURY¶V LQVLJKWV , KDYH SURSRVHGWRGHILQHWKHLQIRUPDWLRQFRQWHQWRIDQLPDJHDVDVHWRIGHVFULSWLRQVRIWKH YLVLEOH LPDJH GDWD VWUXFWXUHV 7KH .ROPRJRURY¶V WKHRU\ SUHVFULEHV WKDW VXFK GHVFULSWLRQVPXVWEHFUHDWHGLQDKLHUDUFKLFDODQGUHFXUVLYHPDQQHU7KDWLVVWDUWLQJ ZLWKDPRVWJHQHUDOL]HGDQGVLPSOLILHGGHVFULSWLRQRILPDJHVWUXFWXUHWKHSURFHVVKDV WRSURFHHGLQDWRSGRZQPDQQHUWRWKHORZHUGHVFULSWLRQOHYHOVZKHUHPRUHDQGPRUH ILQHLQIRUPDWLRQGHWDLOVFDQEHIXUWKHUHODERUDWHG $SUDFWLFDODOJRULWKPLPSOHPHQWLQJWKLVLGHDKDVEHHQGHYLVHGDQGLWVVFKHPDLV GHSLFWHG LQ WKH )LJXUH  $V LW IROORZV IURP WKH VFKHPD WR JHW WKH LQWURGXFWRU\ JHQHUDOL]DWLRQWKHLQSXWLPDJHLVLQLWLDOO\VTXHH]HGWRDVPDOOVL]HRIDSSUR[LPDWHO\  SL[HOV 7KH UXOHV RI WKLV VKULQNLQJ RSHUDWLRQ DUH YHU\ VLPSOH DQG IDVW IRXU QRQRYHUODSSLQJQHLJKERXUSL[HOVLQDQLPDJHDWOHYHO/DUHDYHUDJHGDQGWKHUHVXOWLV DVVLJQHG WR D SL[HO LQ D KLJKHU / OHYHO LPDJH 7KHQ DW WKH WRS RI WKH VKULQNLQJ S\UDPLG WKH LPDJH LV VHJPHQWHG DQG HDFK VHJPHQWHG UHJLRQ LV ODEHOHG 6LQFH WKH LPDJHVL]HDWWKHWRSLVVLJQLILFDQWO\UHGXFHGDQGVLQFHLQWKHFRXUVHRIWKHERWWRPXS LPDJH VTXHH]LQJ D VHYHUH GDWD DYHUDJLQJ LV DWWDLQHG WKH LPDJH VHJPHQWDWLRQFODVVLILFDWLRQ SURFHGXUH GRHV QRW GHPDQG VSHFLDO FRPSXWDWLRQDO UHVRXUFHV$Q\ZHOONQRZQVHJPHQWDWLRQPHWKRGRORJ\ZLOOVXIILFH:HXVHRXURZQ SURSULHWDU\ WHFKQLTXH WKDW LV EDVHG RQ D ORZOHYHO ORFDO  LQIRUPDWLRQ FRQWHQW HYDOXDWLRQDQGFRQVHTXHQWPHUJLQJRI]RQHERUGHUV>@EXWWKLVLVQRWREOLJDWRU\ )URPWKLVSRLQWRQWKHWRSGRZQSURFHVVLQJSDWKLVFRPPHQFHG$WHDFKOHYHOWKH WZR SUHYLRXVO\ GHILQHG PDSV DYHUDJH UHJLRQ LQWHQVLW\ PDS DQG WKH DVVRFLDWHG ODEHO PDS DUHH[SDQGHGWRWKHVL]HRIDQLPDJHDWWKHQHDUHVWORZHUOHYHO6LQFHWKHUHJLRQV

DWGLIIHUHQWKLHUDUFKLFDOOHYHOVGRQRWH[KLELWVLJQLILFDQWFKDQJHVLQWKHLUFKDUDFWHULVWLF LQWHQVLW\ WKH PDMRULW\ RI QHZO\ DVVLJQHG SL[HOV DUH GHWHUPLQHG LQ D VXIILFLHQWO\ FRUUHFW PDQQHU 2QO\ SL[HOV DW UHJLRQ ERUGHUV DQG VHHGV RI QHZO\ HPHUJLQJ UHJLRQV PD\ VLJQLILFDQWO\ GHYLDWH IURP WKH DVVLJQHG YDOXHV 7DNLQJ WKH FRUUHVSRQGLQJ FXUUHQWOHYHOLPDJHDVDUHIHUHQFHWKHOHIWVLGHXQSURFHVVHGUDZLPDJH WKHVHSL[HOV FDQ EH HDVLO\ GHWHFWHG DQG VXEMHFWHG WR D UHILQHPHQW F\FOH ,Q VXFK D PDQQHU WKH SURFHVV LV VXEVHTXHQWO\ UHSHDWHG DW DOO GHVFHQGLQJ OHYHOV XQWLO WKH VHJPHQWDWLRQFODVVLILFDWLRQRIWKHRULJLQDOLQSXWLPDJHLVVXFFHVVIXOO\DFFRPSOLVKHG

%RWWRPXSSDWK

7RSGRZQSDWK

/DVWWRS OHYHO WRFRPSUVG LPDJH

WRFRPSUHVVHG LPDJH

6HJPHQWDWLRQ &ODVVLILFDWLRQ

/HYHOQ

2EMHFWVKDSHV /DEHOHGREMHFWV

2EMHFWOLVW

7RSOHYHOREMHFWGHVFULSWRUV

WRH[SDQGHG REMHFWPDSV

/HYHOQREMHFWV

 WRFRPSUHVVHG LPDJH

2ULJLQDOLPDJH

/HYHO

/HYHO

WRH[SDQGHG REMHFWPDSV

WRH[SDQGHG REMHFWPDSV

/HYOREM

/

)LJ7KH6FKHPDRIWKHDSSURDFK

$W HYHU\ SURFHVVLQJ OHYHO HYHU\ LPDJH VXEUHJLRQ MXVW UHFRYHUHG RU DQ LQKHULWHG RQH LVUHJLVWHUHGLQWKHREMHFWV¶DSSHDUDQFHOLVWZKLFKLVWKHWKLUGFRQVWLWXWLQJSDUWRI WKH SURSRVHG VFKHPD 7KH UHJLVWHUHG REMHFWUHJLRQ SDUDPHWHUV DUH WKH DYDLODEOH VLPSOLILHG REMHFW¶V DWWULEXWHV VXFK DV VL]H FHQWHURIPDVV SRVLWLRQ DYHUDJH REMHFW LQWHQVLW\DQGKLHUDUFKLFDODQGWRSRORJLFDOUHODWLRQVKLSZLWKLQDQGEHWZHHQWKHREMHFWV ³VXESDUW RI«´ ³DW WKH OHIW RI«´ HWF  7KH\ DUH VSDUVH JHQHUDO DQG \HW VSHFLILF HQRXJKWRFDSWXUHWKHREMHFW¶VFKDUDFWHULVWLFIHDWXUHVLQDYDULHW\RIGHVFULSWLYHIRUPV 7KLV ZD\ D JHQHUDOL]HG DQG D VLPSOLILHG GHVFULSWLRQ RI UHSUHVHQWDWLYH LPDJH VWUXFWXUHVLVFRQVHUYHGDQGFDUHIXOO\UHFUHDWHGDWHDFKSURFHVVLQJOHYHO7KXVDYHU\ HIIHFWLYH DQG RSWLPDO LQ WKH VHQVH RI WKH QXPEHU RI UHFRYHUHG VXEUHJLRQ  LPDJH VHJPHQWDWLRQPHWKRGRORJ\LVGHYLVHGDQGGHYHORSHG :KDWLVDEVROXWHO\FOHDULVWKDWLPDJHLQIRUPDWLRQUHYHDOHGLQVXFKDPDQQHUUHIHUV RQO\WRWKHREMHFWLYHYLVLEOHLPDJHGDWDVWUXFWXUHVGLVFHUQDEOHLQWKHLPDJH7KDWLV ZK\ , FDOO LW ³SK\VLFDO LQIRUPDWLRQ´  ,W LV DEVROXWHO\ LQGHSHQGHQW DQG GLVVRFLDWHG IURP DQ\ KLJKOHYHO VHPDQWLF NQRZOHGJH DERXW WKH LPDJH 7KDW RSHQV D XQLTXH

RSSRUWXQLW\ WR UHFRQVLGHU WKH ZD\ LQ ZKLFK ZH KDYH SUHYLRXVO\ DSSURDFKHG LPDJH VHPDQWLFVDQGWRWU\WRWUHDWLWLQDSURSHUVHSDUDWHGDQGLQGHSHQGHQWPDQQHU 3.2 Semantic knowledge base representation $V ZH KDYH DOUHDG\ GHILQHG HDUOLHU LPDJH SK\VLFDO LQIRUPDWLRQ LV LQWHUSUHWHG E\ D KXPDQ REVHUYHU LQ DFFRUGDQFH ZLWK KLV SUHYLRXV NQRZOHGJH DERXW WKH WKLQJV LQ WKH VXUURXQGLQJ ZRUOG 7KLV NQRZOHGJH DFFXPXODWHG LQ KXPDQ¶V KHDG GXULQJ KLV HQWLUH OLIH VSDQ LV XVXDOO\ FDOOHG KXPDQ¶V ZRUOG RQWRORJ\ 9DULRXV DVSHFWV RI RQWRORJ\ GHVLJQXVHDQGDPHOLRUDWLRQDUHQRZDKRWWRSLFRIWKHRQJRLQJUHVHDUFKLQWKHILHOG RI NQRZOHGJH UHSUHVHQWDWLRQ DQG PDQDJHPHQW &ODVVLFDO SXEOLFDWLRQV UHODWHG WR WKH VXEMHFWDUHZHOONQRZQ>@>@>@&%,5GHVLJQHUVDUHDOVRGHHSO\LQYROYHGLQ DWWHPSWV WR LQFRUSRUDWH RQWRORJ\EDVHG DSSURDFKHV LQWR &%,5 DUFKLWHFWXUHV >@ >@>@>@>@>@>@>@>@7KDWLVRQO\DSDUWLDOOLVWWKHUHDORXWFRPH LV PXFK PRUH H[WHQVLYH 7KHUHIRUH DJDLQ , ZLOO QRW UHYLHZ KHUH WKH DYDLODEOH SXEOLFDWLRQV 6LQFH WKH EDVLF SKLORVRSK\ RI WKH H[LVWLQJ DSSURDFKHV DQG P\ DUH VR GLIIHUHQW,ZRXOGOLNHWRFRQFHQWUDWHRQWKHFRQVHTXHQFHVZKLFKIROORZIURPVXFKD GLIIHUHQFH )LUVW RI DOO P\ DLP LV WR UHVROYH WKH SUREOHPV RI D FRPSXWHUEDVHG UHPRWH DJHQW SHUIRUPLQJ DXWRQRPRXVO\ RQ WKH :HE DQG H[SHFWHG WR LQWHUSUHW WR XQGHUVWDQG SUHYLRXVO\XQVHHQLPDJHV)RUWKLVUHDVRQWKHDJHQWPXVWEHSURYLGHGZLWKDUHSOLFD RIKXPDQRQWRORJ\ZKLFKZLOOIDFLOLWDWHWKHVHDJHQWDWWHPSWVWRHQGRZLPDJHVRQWKH :HEZLWKWKHLUVHPDQWLFV +RZHYHUEHLQJSODFHGLQVXFKJHQHUDODQGXQFRQVWUDLQHG FRQGLWLRQVHYHQDKXPDQZLOOEHXQDEOHWRIXOILOOVXFKGXWLHV(YHQKXPDQVDUHRIWHQ OLPLWHG LQ WKHLU NQRZOHGJH DQG SHUIRUP ZHOO RQO\ LQ NQRZQ DQG IDPLOLDU VXUURXQGLQJV:K\WKDWVKRXOGEHGLIIHUHQWIRUDQDUWLILFLDOFUHDWXUH"2XUDJHQWWKXV KDV WR EH SURYLGHG DW OHDVW DW WKH EHJLQQLQJ  ZLWK D YHU\ UHVWULFWHG VHW RI SDUWLDO RQWRORJLHV ZKLFK ZLOO FRYHU RQO\ D OLPLWHG QXPEHU RI HQFRXQWHUHG VLWXDWLRQV 7KH ZLGHVSUHDG WUHQG WR FRYHU DQ H[WHQGHG VSHFWUXP RI LPDJH FDWHJRULHV ORRNV WR PH DEVROXWHO\XQUHDOLVWLF (YHQWKHUHVWULFWHGNQRZOHGJHZKLFKLVUHTXLUHGIRUDJHQW¶VIXQFWLRQLQJLVLQLWLDOO\ QRW LQ LWV GLVSRVDO 7KLV NQRZOHGJH PXVW EH DSSURSULDWHO\ SLFNHG XS SURFHVVHG HQFRGHGDUUDQJHGDQGILQDOO\LQRQHZD\RUDQRWKHU³VKDUHG´DVRQWRORJ\SHRSOHFDOO WKLV EHWZHHQWKHDJHQWDQGLWVVXSHUYLVRUV7KHDJHQWLWVHOILVXQDEOHWRFDUU\RXWVXFK WDVNV7KHUHIRUHLWLVWKHGHVLJQHU¶VGXW\DQGWKHGHVLJQHU¶VRUV\VWHPVXSHUYLVRURU DJHQW¶V FRSDUWQHU  UHVSRQVLELOLW\ WR PDNH DOO WKHVH WDVNV WR EH DFFRPSOLVKHG 7KDW FHUWDLQO\ FRQWUDGLFWV WKH JHQHUDO WUHQG RI RWKHU RQWRORJ\ V\VWHPV GHVLJQV ZKLFK SUHVXPHWKDWV\VWHPVPXVWEHDSWWROHDUQDQGWRDGDSWWKHPVHOYHVLQGHSHQGHQWO\WR WKH FKDQJLQJ HQYLURQPHQW HJ QHZ LPDJHV EHORQJLQJ WR GLIIHUHQW NQRZOHGJH GRPDLQV 2EYLRXVO\,GHQ\WKHQHHGIRU VXFK TXDOLWLHV , WKLQN WKDW WKH ZLGHVSUHDG UHTXLUHPHQW IRU PDFKLQH OHDUQLQJ FDSDELOLWLHV LV VWURQJO\ H[DJJHUDWHG $IWHU DOO KXPDQV DUH DFTXLULQJ WKHLU NQRZOHGJH QRW LQ D PDFKLQH OHDUQLQJ IDVKLRQ EXW LQ D GHFODUDWLYHDQG LQVWUXFWLYH PDQQHU 7KH NQRZOHGJH LV WUDQVIHUUHG IURP D SDUHQW WR D FKLOGIURPDWHDFKHUWRDVWXGHQWIURPDQLQVWUXFWRUWRDWUDLQHHRQO\LQDGHFODUDWLYH VXSHUYLVHGPDQQHU5HFHQWO\LWKDVEHHQGLVFRYHUHGWKDWWKDWLVQRWDQH[FHSWLRQDOO\ KXPDQWUDLWDQLPDOVDOVRWUDQVIHUNQRZOHGJHEHWZHHQWKHPLQDWHDFKHUSXSLOIDVKLRQ >@6RZK\LWVKRXOGEHGLIIHUHQWLQRXUFDVH"$VDJRRGWHDFKHUZKLFKSURYLGHV

KLV VWXGHQWV ZLWK GHOLEHUDWHO\ SUHSDUHG UHIHUHQFH PDWHULDO DJHQW¶V VXSHUYLVRU DSSURSULDWLQJWKHDJHQWIRUDQHZWDVNPXVWSURYLGHLWZLWKDQHZVXLWDEOHRQWRORJ\ ,Q WKLV UHJDUG D TXHVWLRQ RI D VXLWDEOH OHDUQLQJ ODQJXDJH RU PRUH VSHFLILFDOO\ D TXHVWLRQ RI D ODQJXDJH WKDW ZRXOG EHVW ILW WKH SXUSRVHV RI RQWRORJ\ FUHDWLRQ FRPHV LQWR FRQVLGHUDWLRQ 7RGD\ RQWRORJ\ GHVLJQHUV SUHYDOHQWO\ DGKHUH WR WKH VRFDOOHG 'HFODUDWLYH /DQJXDJH D VRUW RI D IRUPDO ODQJXDJH VXLWDEOH IRU PDFKLQH LQIRUPDWLRQ H[FKDQJH DQG PDFKLQH UHDVRQLQJ , DP QRW VXUH WKDW WKDW LV JRRG FKRLFH IRU XV +XPDQVLQWKHLUSUDFWLFHRILQWHUFRPPXQLFDWLRQDQGNQRZOHGJHH[FKDQJHLQFOXGLQJ YDULRXV OHYHOV RI HGXFDWLRQ DQG OHDUQLQJ  PDQDJH SUHWW\ ZHOO ZLWK WKHLU QDWXUDO ODQJXDJH2IFRXUVHWKLVODQJXDJHLVQRWVRZHOOIRUPDOL]HGLWLVIX]]\DQGLPSUHFLVH EXWXVXDOO\ZHGRQ¶WIHHODQ\LQFRQYHQLHQFHZLWKWKLV7KHQDWXUDOODQJXDJHH[WUHPHO\ ZHOOVHUYHVRXUQHHGVLQFRPPXQLFDWLRQVNQRZOHGJHVKDULQJDQGH[FKDQJHOHDUQLQJ UHDVRQLQJ DQG SUREOHP VROYLQJ $QWKURSRPRUSKRXV JRDOV RI RXU DJHQW WKDW KDV WR HQGRZLPDJHVZLWKWKHLUVHPDQWLFV DVVXPHWKDWLWVNQRZOHGJHEDVHVXSSRUWLQJDQG HQDEOLQJVXFKDFWLYLW\ VKRXOGDOVREHDKXPDQOLNHPDFKLQHU\7KDWLVWKHRQWRORJ\ GHVFULSWLRQVWREHXVHGPXVWEHLPSOHPHQWHGLQDQDWXUDOODQJXDJH $IWHU DOO VHPDQWLFV LV DQ HQFRGHG DQG PHPRUL]HG H[SHULHQFH JDLQHG LQ FRXUVH RI KXPDQLQWHUDFWLRQZLWKWKHVXUURXQGLQJZRUOG6HPDQWLFVXQGHUVWDQGLQJLVDUHFDOOWR WKLVPHPRU\LQDWWHPSWWRILQGDVXLWDEOHVLPLODUVLWXDWLRQWRFDSLWDOL]HRQLWLQIXUWKHU UHDVRQLQJDQGDFWLRQSODQQLQJ,IZHDJUHHWKDWWKHPHPRUL]HGGHVFULSWLRQVFRXOGEH LPSOHPHQWHGLQZRUGVRIDQDWXUDOODQJXDJHWKDQRQWRORJLFDOGHVFULSWLRQVWKDWZHDUH VSHDNLQJDERXWFDQEHVHHQDVDVHWRIQDUUDWLYHFKURQLFVWRWDNHDFFRXQWIRUDWLPH GLPHQVLRQ XVXDOO\ LPSUXGHQWO\ ORVW ZKHQ ZH DUH GHDOLQJ ZLWK LPDJHV  7KH HTXLYDOHQWRIWKHKXPDQ¶V/RQJ7HUP0HPRU\ZKHUHWKHRQWRORJ\GHVFULSWLRQVDUH WR EH PHPRUL]HG FDQ EH VHHQ WKHQ DV D ODUJH ILOH V\VWHP NQRZQ WR XV DQG IDPLOLDU IURP PHPRU\ RUJDQL]DWLRQ LQ FRQWHPSRUDU\ FRPSXWHUV 0D\ EH DQ DQDORJXH ZLWK 2EMHFWRULHQWHG 3URJUDPPLQJ (QYLURQPHQW ZLOO EH HYHQ PRUH DSSURSULDWH ,Q DQ\ FDVHWKLVZD\RIRQWRORJ\GHVFULSWLRQFUHDWLRQDQGKDQGOLQJORRNVPRUHQDWXUDOWKDQ WKHFXUUHQWO\XVHGDSSURDFKHV +RZDQLPDJHFDQJHWLWVVHPDQWLFV $V ZH MXVW GHILQHG DERYH VHPDQWLFV LV DQ HQFRGHG DQG PHPRUL]HG GHVFULSWLRQ RI KXPDQ¶V H[SHULHQFH JDLQHG DV D UHVXOW RI KXPDQ LQWHUDFWLRQ ZLWK WKH VXUURXQGLQJ ZRUOG ,W LV QRW NQRZQ IRU VXUH EXW LW LV TXLWH SRVVLEOH WKDW KXPDQ¶V RQWRORJ\ GHVFULSWLRQVDUHUHDOL]HGDVQDUUDWLYHFRPSRVLWLRQVRIQDWXUDOODQJXDJHZRUGV,QWKH SUHYLRXVVXEVHFWLRQZHKDYHGUDZQDSDUDOOHOEHWZHHQDOLQJXLVWLFGHVFULSWLRQDQGDQ RQWRORJLFDO GHVFULSWLRQ FRPPRQO\ XVHG WRGD\ LQ NQRZOHGJH UHSUHVHQWDWLRQ DQG PDQDJHPHQW,ZRXOGOLNHWRFRQWLQXHDQGWRH[WHQGWKLVDQDORJ\/LQJXLVWLFODEHOVIRU LPDJHREMHFWVFDQEHVHHQDVHTXLYDOHQWV WR FRQFHSW FODVVHV DQG REMHFW VXESDUWV DV HTXLYDOHQWV WR FRQFHSW VXEFODVVHV ,Q WKH UHJXODU RQWRORJ\ GHVFULSWLRQV FODVVHV DV ZHOODVVXEFODVVHVDUHXVXDOO\DXJPHQWHGZLWKDWWULEXWHVZKLFKGHSLFWVRPHSK\VLFDO SURSHUWLHVRIWKHFRQFHSWXDOL]HGHQWLWLHVUHSUHVHQWHGE\DJLYHQFODVVRUDVXEFODVV  1RZ ZH PXVW WR UHFDOO KRZ D UDZ LPDJH LVLQLWLDOO\ SUHSURFHVVHG LQ RXU SURSRVHG DSSURDFKVHHVXEVHFWLRQ 7KHSK\VLFDOLPDJHLQIRUPDWLRQLVUHSUHVHQWHGILQDOO\ DV D KLHUDUFKLFDO GHVFULSWLRQ OLVW RI YLVXDOO\ GLVWLQJXLVKDEOH LPDJH VXEUHJLRQV 2QH FDQ HDVLO\ VHH D VWULNLQJ VLPLODULW\ EHWZHHQ WKH KLHUDUFKLFDO GHVFULSWLRQ RI LPDJH

VXEUHJLRQVDQGWKHKLHUDUFK\RIFODVVVXEFODVVDWWULEXWHVLQDVFHQHUHODWHGRQWRORJ\ ,W LV QRZ WKH GHVLJQHU¶V GXW\ DQG UHVSRQVLELOLW\ WR HVWDEOLVK WKH VLPLODULW\ PHDVXUHV DQG UXOHV OLQNLQJ EHWZHHQ HTXLYDOHQW LPDJH REMHFWV DQG WKHLU ODEHOV 2EYLRXVO\ DOO WKLV PXVW EH GRQH PDQXDOO\ DV D SDUW RI WKH H[SOLFLW GHFODUDWLYH NQRZOHGJH WUDQVIHU OHDUQLQJ PRGHDQQRXQFHGDVWKHSULQFLSDOPRGXVRSHUDQGLRIRXUV\VWHP ,Q VXFK D ZD\ DQ DJHQW FDQ QRZ ODEHO WKH LPDJH REMHFWV 7KDW LV LQLWLDOO\ HQGRZ WKHPZLWKWKHLUVHPDQWLFV7KLVLVRQO\WKHILUVWVWHSLQDQLPDJHVHPDQWLFVUHFRYHU\ SURFHGXUH +DYLQJ LPDJH REMHFWV ODEHOHG IXUWKHU PRUH FRPSOH[ VHPDQWLF GHVFULSWLRQVFDQEHGHULYHGQRZFOLPELQJRQWKHOLQJXLVWLFRQWRORJ\ODGGHUDQGWKXV SDZLQJDZD\WRDPRUHDGYDQFHGWH[WXDOLPDJHDQQRWDWLRQ 3HUIRUPLQJ WKLV SDUWLDO JRDO LW PXVW EH UHPHPEHUHG WKDW WRSRORJLFDO LQWHUUHODWLRQV EHWZHHQ LPDJH SDUWV REMHFWV  DQG WKHLU VXESDUWV VXEREMHFWV  LQIRUPDWLRQ DERXW FRDUVHILQH SDUWV FRQILJXUDWLRQV LV FULWLFDOO\ LPSRUWDQW IRU WKH DFFRPSOLVKPHQW RI LPDJH VHPDQWLFV HQGRZPHQW >@ >@ >@ ,Q WKLV UHJDUG LW PXVW EH HVSHFLDOO\ HPSKDVL]HG WKDW WKH SURSRVHG SK\VLFDO LQIRUPDWLRQ UHSUHVHQWDWLRQ VFKHPH VXEVHFWLRQ SURYLGHVXQSUHFHGHQWHGRSSRUWXQLWLHVIRUH[SORLWLQJVXFKWRSRORJLFDO DWWULEXWHV LQ FRXUVH RI LPDJH REMHFWV LGHQWLILFDWLRQ UHFRJQLWLRQ  DQG VXEVHTXHQW ODEHOLQJ$OOWKHVHWRSRORJLFDOUHODWLRQVFDQEHHDVLO\GHSLFWHGDQGPDQXDOO\OLQNHGWR WKHDJHQW¶VVFHQHRQWRORJ\

4 Some concluding remarks ,Q WKLV SDSHU , KDYH SUHVHQWHG DQ XQFRPPRQ LPDJH XQGHUVWDQGLQJ DQG LPDJH VHPDQWLFVUHFRYHU\DSSURDFKZKLFKWRGD\LVDYH[HGWRSLFLQPDQ\LPDJHKDQGOLQJ UHODWHGUHVHDUFKDQGGHYHORSPHQWHQWHUSULVHV ,SRVLWWKDWWKHWUDGLWLRQDOERWWRPXSWRSGRZQDSSURDFKGRHVQRWKROGPRUH7KDW SK\VLFDO LPDJH LQIRUPDWLRQ FDQ EH H[WUDFWHG IURP DQ LPDJH LQ DQ XQVXSHUYLVHG WRSGRZQ PDQQHU LQGHSHQGHQW IURP DQ\ KLJKOHYHO NQRZOHGJH DERXW LPDJH FRQWHQW :KDWIROORZVIURPWKLVLVWKDWKLJKOHYHOFRQFHSWVDUHQRWLQYROYHGLQORZOHYHOLPDJH SURFHVVLQJ DQG DQ\ VHPDQWLF JDS EHWZHHQ WKHP GRHV QRW H[LVW 6HPDQWLFV LV QRW D SURSHUW\RIDQLPDJHLWLVDSURSHUW\RIDKXPDQREVHUYHUWKDWZDWFKHVWKHLPDJH7KH REVHUYHU SURFHHGV LPDJH VHPDQWLFV LQ DFFRUGDQFH ZLWK KLV NQRZOHGJH DERXW WKH WKLQJVLQWKHRXWHUZRUOG7KLVNQRZOHGJHLVXVXDOO\FDOOHGWKHRQWRORJ\RIWKHZRUOG $ YLVXDO URERW RU DQ DJHQW FDQ RSHUDWH OLNH D KXPDQ HQGRZLQJ LPDJHV ZLWK VHPDQWLFV 7R HQDEOH VXFK DFWLYLW\ LW PXVW EH IXUQLVKHG ZLWK D UHSOLFD RI D KXPDQ RQWRORJ\7KH ODWWHU KDV QRW EH HQWLUHO\ IXOO DV LQ KXPDQV  EXWLW PXVW EH PRGXODU FRQFLVHDQGVSHFLILFHQRXJKDERXWXVHU¶VYLHZRQWKHSDUWLFXODUWDVNLQKDQG6XFKD UHSOLFD FDQ QRW EH DFKLHYHG LQ D XVXDO PDFKLQHOHDUQLQJ IDVKLRQ EXW LW PXVW EH VWHSE\VWHS GHYHORSHG DQG GHOLYHUHG WR WKH DJHQW¶V GLVSRVDO E\ WKH DJHQW¶V KXPDQ VXSHUYLVRU 6XFK SHFXOLDULWLHV VRPHWLPHV EOXU WKH FRPSUHKHQVLRQ RI WKH WKLQJV SURSRVHG LQ WKLV DSSURDFK , KRSH , ZDV OXFN\ WKLV WLPH WR PDNH P\ SRLQW FOHDU HQRXJK

5HIHUHQFHV  $ 7UHLVPDQ DQG * *HODGH ³$ IHDWXUHLQWHJUDWLRQ WKHRU\ RI DWWHQWLRQ´ &RJQLWLYH 3V\FKRORJ\YROSS-DQ  ' 0DUU ³9LVLRQ $ &RPSXWDWLRQDO ,QYHVWLJDWLRQ LQWR WKH +XPDQ 5HSUHVHQWDWLRQ DQG 3URFHVVLQJRI9LVXDO,QIRUPDWLRQ´)UHHPDQ6DQ)UDQFLVFR  ,%LHGHUPDQ³5HFRJQLWLRQE\&RPSRQHQWV$7KHRU\RI+XPDQ,PDJH8QGHUVWDQGLQJ´ 3V\FKRORJLFDO5HYLHZYROQRSS  0 6 %DUWOHWW DQG 7 - 6HMQRZVNL ³/HDUQLQJ YLHZSRLQW LQYDULDQW IDFH UHSUHVHQWDWLRQV IURPYLVXDOH[SHULHQFHLQDQDWWUDFWRUQHWZRUN´1HWZRUN&RPSXWDWLRQLQ1HXUDO6\VWHPV YROQRSS