Web Prefetching Performance Metrics: A Survey - UPV

:HE3UHIHWFKLQJ3HUIRUPDQFH0HWULFV$6XUYH\ -RVHS'RPqQHFK-RVp$*LO-XOLR6DKXTXLOORDQG$QD3RQW 'HSDUWPHQWRI&RPSXWHU(QJLQHHULQJ',6&$ 8QLYHUVLWDW3ROLWqFQLFDGH9DOqQFLD6SDLQ MRGRGH#GRFWRUXSYHV^MDJLOMVDKXTXLDSRQW`#GLVFDXSYHV

$EVWUDFW :HE SUHIHWFKLQJ WHFKQLTXHV KDYH EHHQ SRLQWHG RXW WR EH HVSHFLDOO\ LPSRUWDQW WR UHGXFH SHUFHLYHG ZHE ODWHQFLHV DQG FRQVHTXHQWO\ DQ LPSRUWDQW DPRXQWRI ZRUN FDQ EH IRXQG LQ WKHRSHQ OLWHUDWXUH%XWLQJHQHUDOLWLVQRWSRVVLEOHWRGRDIDLUFRPSDULVRQDPRQJWKHSURSRVHGSUHIHWFKLQJ WHFKQLTXHVGXHWRWKUHHPDLQUHDVRQVL WKHXQGHUO\LQJEDVHOLQHV\VWHPZKHUHSUHIHWFKLQJLVDSSOLHG GLIIHUVZLGHO\DPRQJWKHVWXGLHVLL WKHZRUNORDGXVHGLQWKHSUHVHQWHGH[SHULPHQWVLVQRWWKHVDPH LLL GLIIHUHQWSHUIRUPDQFHNH\PHWULFVDUHXVHGWRHYDOXDWHWKHLUEHQHILWV 7KLV SDSHU IRFXVHV RQ WKH WKLUG UHDVRQ 2XU PDLQ FRQFHUQ LV WR LGHQWLI\ ZKLFK DUH WKH PHDQLQJIXO LQGH[HV ZKHQ VWXG\LQJ WKH SHUIRUPDQFH RI GLIIHUHQW SUHIHWFKLQJ WHFKQLTXHV )RU WKLV SXUSRVH ZH SURSRVHDWD[RQRP\EDVHGRQWKUHHFDWHJRULHVZKLFKSHUPLWVXVWRLGHQWLI\DQDORJLHVDQGGLIIHUHQFHV DPRQJ WKH LQGH[HV FRPPRQO\ XVHG ,Q RUGHU WR FKHFN LQ D PRUH IRUPDO ZD\ WKH UHODWLRQ EHWZHHQ WKHPZHUXQH[SHULPHQWVDQGHVWLPDWHVWDWLVWLFDOO\WKHFRUUHODWLRQDPRQJDUHSUHVHQWDWLYHVXEVHWRI WKRVH PHWULFV 7KH VWDWLVWLFDO UHVXOWV KHOS XV WR VXJJHVW ZKLFK LQGH[HV VKRXOG EH VHOHFWHG ZKHQ SHUIRUPLQJHYDOXDWLRQVWXGLHVGHSHQGLQJRQWKHGLIIHUHQWHOHPHQWVLQWKHFRQVLGHUHGZHEDUFKLWHFWXUH 7KHFKRLFHRIWKHDSSURSULDWHNH\PHWULFLVRISDUDPRXQWLPSRUWDQFHIRUDFRUUHFWDQGUHSUHVHQWDWLYH VWXG\ $V RXU H[SHULPHQWDO UHVXOWV VKRZ GHSHQGLQJ RQ WKH PHWULF XVHG WR FKHFN WKH V\VWHP SHUIRUPDQFHUHVXOWVFDQQRWRQO\ZLGHO\YDU\EXWDOVRUHDFKRSSRVLWHFRQFOXVLRQV

,QWURGXFWLRQ 7KH LQWHUQDWLRQDO DQG JOREDO QDWXUH RI ,QWHUQHW PDNHV LW DUGXRXV WR LQFUHDVH WKH V\VWHP SHUIRUPDQFH ZRUNLQJ LQ WKH QHWZRUN KDUGZDUH DQG LWV LQWHUFRQQHFWLRQ HOHPHQWV EHFDXVH RI WKH H[LVWLQJ JDS RI WLPH EHWZHHQWHFKQRORJLFDODGYDQFHVLQLQIUDVWUXFWXUHDQGLWVSUDFWLFDOXVH$VDFRQVHTXHQFHUHVHDUFKHIIRUWV KDYHEHHQFRQFHQWUDWHGRQWKHZHEDUFKLWHFWXUHDQGRUJDQL]DWLRQXVLQJDQGH[SRUWLQJWHFKQLTXHVDOUHDG\ OHDUQHGDQGZLGHO\XVHGLQFRPSXWHUDUFKLWHFWXUHWRLPSURYHWKHSHUIRUPDQFH 0RVWRIWKRVHWHFKQLTXHVWDNHDGYDQWDJHRIWKHORFDOLW\SURSHUWLHVLQKHUHQWWRWKHZHEREMHFWDFFHVVHV ,Q WKH :HE WKH ORFDOLW\ KDV WKUHH GLIIHUHQW FKDUDFWHULVWLFV WHPSRUDO VSDWLDO DQG JHRJUDSKLFDO ZKLFK SHUPLWV WR LPSOHPHQW HIILFLHQWO\ FDFKLQJ SUHIHWFKLQJ DQG UHSOLFDWLRQ WHFKQLTXHV LQ RUGHU WR LQFUHDVH SHUIRUPDQFH$VDUHVXOWPDQ\HIIRUWVFRPPHUFLDODQGUHVHDUFK DSSO\LQJWKRVHWHFKQLTXHVLQWKHZHE DUFKLWHFWXUH KDYH EHHQ FDUULHG RXW WR LPSURYH SHUIRUPDQFH ,Q WKLV SDSHU ZH IRFXV RQ SUHIHWFKLQJ WHFKQLTXHVDOWKRXJKVRPHRIWKHFRQFOXVLRQVSUHVHQWHGLQWKLVSDSHUFDQDOVREHDGDSWHGWRLQFOXGHDQ\ ZHEWHFKQLTXHDLPHGWRUHGXFHWKHILQDOXVHUSHUFHLYHGODWHQF\ 0DQ\ UHVHDUFK VWXGLHV FRQFHQWUDWH RQ WKH SURSRVDOV RI QHZ SUHIHWFKLQJ WHFKQLTXHV 3HUIRUPDQFH FRPSDULVRQVWXGLHVDPRQJWKHPFDQQRWEHIDLUO\GRQHEHFDXVHWKHSURSRVHGDSSURDFKHVDUHDSSOLHGDQG

WHVWHGLQGLIIHUHQWEDVHOLQHV\VWHPVXVLQJDOVRGLIIHUHQWZRUNORDGVDQGFRQGLWLRQV,QDGGLWLRQWKHVWXGLHV SUHVHQWGLIIHUHQWSHUIRUPDQFHNH\PHWULFVWRHYDOXDWHWKHLUEHQHILWV ,Q RUGHU WR GR IDLU SHUIRUPDQFH FRPSDULVRQ VWXGLHV ZH QHHG WR WDFNOH WKH PHQWLRQHG GUDZEDFNV 7R GHDO ZLWK WKH ILUVW RQH ZH SURSRVHG LQ D SUHYLRXV ZRUN >@ D JHQHUDO H[SHULPHQWDO IUDPHZRUN ZKLFK SHUPLWVWKHLPSOHPHQWDWLRQRISUHIHWFKLQJWHFKQLTXHVLQDIOH[LEOHDQGHDV\ZD\XQGHUWKHVDPHSODWIRUP DQGUHDOZRUNORDGVWKHUHIRUHWKHVDPHH[SHULPHQWDOFRQGLWLRQVDUHFRQVLGHUHG ,QWKLVSDSHUZHDGGUHVVRXUZRUNWRWDFNOHZLWKWKHVHFRQGGUDZEDFN7RWKLVHQGZHDQDO\]HDODUJH VXEVHW RI NH\ PHWULFV DQG SURSRVH D WD[RQRP\ EDVHG RQ WKUHH PDLQ FDWHJRULHV ZKLFK SHUPLWV XV WR LGHQWLI\ DQDORJLHV DQG GLIIHUHQFHV DPRQJ WKH PHWULFV FRPPRQO\ XVHG DQG FKHFN H[SHULPHQWDOO\ WKHLU UHODWLRQ $V D FRQVHTXHQFH ZH DOVR LQWURGXFH QHZ E\WH EDVHG LQGH[HV DV FRPSOHPHQWDU\ PHWULFV 7KLV SDSHUH[WHQGVWKHZRUNSUHVHQWHGLQ>@LQVHYHUDOZD\V:HLGHQWLI\QHZDQDO\WLFDOUHODWLRQVDPRQJWKH LQGH[HV DQG D QHZ PHWULF LV SURSRVHG WR HVWDEOLVK WKH DQDO\WLFDO UHODWLRQ :H DOVR LQFOXGH PRUH H[SHULPHQWVFRQVLGHULQJDQRWKHUSUHGLFWLRQDOJRULWKP)LQDOO\UHIHUHQFHVKDYHEHHQDFFRUGLQJO\XSGDWHG DQGGLVFXVVHG 7KH UHPDLQGHU RI WKLV SDSHU LV RUJDQL]HG DV IROORZV 6HFWLRQ GHVFULEHV D JHQHULF ZHE SUHIHWFKLQJ V\VWHPLQRUGHUWRVHWWKHEDVLFJORVVDU\RIWHUPVWKDWZLOOEHXVHGLQWKHUHPDLQLQJVHFWLRQV6HFWLRQ SUHVHQWV WKH SURSRVHG WD[RQRP\ IRU SUHIHWFKLQJ SHUIRUPDQFH PHWULFV 6HFWLRQ GHVFULEHV WKH H[SHULPHQWDOHQYLURQPHQWXVHGWRUXQWKHH[SHULPHQWV6HFWLRQGLVFXVVHVWKHUHODWLRQEHWZHHQLQGH[HV IURP WZR GLIIHUHQW DSSURDFKHV DQDO\WLFDO DQ H[SHULPHQWDO DQG ILQDOO\ 6HFWLRQ SUHVHQWV VRPH FRQFOXGLQJUHPDUNV

*HQHULF3UHIHWFK$UFKLWHFWXUHDQG%DVLF*ORVVDU\ :HDVVXPHDJHQHULFZHEDUFKLWHFWXUHFRPSRVHGE\WKUHHPDLQHOHPHQWVFOLHQWVVHUYHUVDQGSUR[LHV 1RWHWKDWSUR[LHVDFWERWKDVDFOLHQWIRUWKHVHUYHUDQGDVDVHUYHUIRUWKHFOLHQW ,WLVLPSRUWDQWWRUHPDUNWKHGLIIHUHQFHEHWZHHQWKHXVHUDQGWKHFOLHQW7KHXVHULVWKHSHUVRQLQIURQW RI D FRPSXWHU RU D VLPLODU GHYLFH GHPDQGLQJ LQIRUPDWLRQ ZKHUHDV WKH FOLHQW LV WKH VRIWZDUH LH WKH EURZVHU ZLWKZKLFKWKHXVHULQWHUDFWVWKDWUHTXHVWVWKHGHPDQGHGLQIRUPDWLRQWRWKHDSSURSULDWHVHUYHU 7KHPDLQDLPRISUHIHWFKLQJWHFKQLTXHVLVWRUHGXFHWKHDYHUDJHODWHQF\SHUFHLYHGE\WKHXVHU6HYHUDO SUHIHWFKLQJ UHODWHG WHFKQLTXHV KDYH EHHQ SURSRVHG IRFXVLQJ RQ GLIIHUHQW LGHDV WR H[SORLW WKH VSDWLDO ORFDOLW\RIZHEREMHFWVXVLQJSUHIHWFKLQJIRULQVWDQFHVRPHUHVHDUFKVWXGLHVSURSRVHFOLHQWVWRGRZQORDG REMHFWV SULRU WR EH UHTXHVWHG E\ WKH XVHU >@ >@ >@ >@ RWKHU VWXGLHV SURSRVH WR SUHSURFHVV G\QDPLFFRQWHQW>@VRPHRWKHUVFRQFHQWUDWHRQKRZWRPDNHSUHFRQQHFWLRQVWRVHUYHU>@DQGVRRQ $OOSUHIHWFKLQJUHODWHGWHFKQLTXHVVWDUWSUHGLFWLQJRUWU\LQJWRJXHVVWKHQH[WREMHFWVWRZKLFKWKHFOLHQW ZLOO DFFHVV 7KLV SDUW RI WKH SUHIHWFK LV XVXDOO\ UHIHUUHG DV WKH SUHGLFWLRQ HQJLQH 7KHQ WKH SUHGLFWLRQ UHVXOWV DUH VXEPLWWHG WR WKH SUHIHWFKLQJ HQJLQH ZKLFK GHFLGHV ZKHWKHU WR SUHIHWFK RU QRW VXFK UHVXOWV GHSHQGLQJRQRWKHUSDUDPHWHUVHJDYDLODEOHEDQGZLGWKRUVHUYHUORDG,QWKLVZD\SUHIHWFKHGREMHFWV DUHDVXEVHWRIWKHSUHGLFWHGREMHFWV1RWLFHWKDWERWKWKHSUHGLFWLRQHQJLQHDQGWKHSUHIHWFKLQJHQJLQHFDQ EHIRXQGLQWKHVDPHHOHPHQWFOLHQWSUR[\RUVHUYHU :HGHILQHEHORZVRPHEDVLFYDULDEOHVWKDWZLOOEH XVHGLQ6HFWLRQ

− 3UHGLFWLRQVDPRXQWRIREMHFWVSUHGLFWHGE\WKHSUHGLFWLRQHQJLQH − 3UHIHWFKVDPRXQWRIREMHFWVSUHIHWFKHGE\WKHSUHIHWFKLQJHQJLQH − *RRG3UHGLFWLRQVDPRXQWRIREMHFWVSUHGLFWHGWKDWDUHVXEVHTXHQWO\GHPDQGHGE\WKHXVHU − %DG3UHGLFWLRQVWKRVHSUHGLFWLRQVWKDWGRQRWUHVXOWLQJRRGSUHGLFWLRQV − 3UHIHWFK+LWVDPRXQWRISUHIHWFKHGREMHFWVWKDWDUHVXEVHTXHQWO\GHPDQGHGE\WKHXVHU − 2EMHFWV1RW8VHGDPRXQWRISUHIHWFKHGREMHFWVQHYHUGHPDQGHGE\WKHXVHU − 8VHU5HTXHVWVDPRXQWRIREMHFWVWKHXVHUGHPDQGV )LJXUHVKRZVJUDSKLFDOO\WKHUHODWLRQVEHWZHHQWKHYDULDEOHVGHILQHGDERYH$UHSUHVHQWVDQREMHFW UHTXHVWHG E\ WKH XVHU EXW QHLWKHU SUHGLFWHG QRU SUHIHWFKHG % UHSUHVHQWV D *RRG3UHGLFWLRQ WKDW KDV QRW EHHQSUHIHWFKHGZKLOH&UHSUHVHQWVD3UHIHWFK+LWLHD*RRG3UHGLFWLRQWKDWKDVEHHQSUHIHWFKHG'LVDQ 2EMHFW1RW8VHGZKLFKLVDOVRD%DG3UHGLFWLRQ)LQDOO\(UHSUHVHQWVD%DG3UHGLFWLRQ $QDORJRXVO\ ZHFDQGHILQH E\WHUHODWHGYDULDEOHV3UHGLFWLRQV%3UHIHWFKV%DQGVRRQ E\UHSODFLQJ WKHREMHFWVZLWKWKHFRUUHVSRQGLQJVL]HLQE\WHVLQWKHLUGHILQLWLRQ

3UHGLFWLRQV

8VHU5HTXHVWV $

(

% &

'

3UHIHWFKV

)LJ6HWVUHSUHVHQWLQJWKHUHODWLRQEHWZHHQ8VHU5HTXHVWV3UHGLFWLRQVDQG3UHIHWFKV

:HE3HUIRUPDQFH,QGH[HV7D[RQRP\ 7KLVVHFWLRQVXUYH\VWKHZHESHUIRUPDQFHLQGH[HVDSSHDUHGLQWKHRSHQOLWHUDWXUHIRFXVLQJRQSUHIHWFK DVSHFWV7RWKHEHWWHUXQGHUVWDQGLQJRIWKH PHDQLQJRIWKRVHLQGH[HV ZHFODVVLI\WKHP LQWRWKUHH PDLQ FDWHJRULHVVHH)LJXUH DWWHQGLQJWRWKHV\VWHPIHDWXUHWKH\HYDOXDWH − &DWHJRU\SUHGLFWLRQUHODWHGLQGH[HV − &DWHJRU\UHVRXUFHXVDJHLQGH[HV − &DWHJRU\HQGWRHQGSHUFHLYHGODWHQFLHVLQGH[HV 7KH ILUVW FDWHJRU\ LV WKH PDLQ RQH ZKHQ FRPSDULQJ SUHGLFWLRQ DOJRULWKPV SHUIRUPDQFH DQG LQFOXGHV WKRVHLQGH[HVZKLFKTXDQWLI\ERWKWKHHIILFLHQF\DQGWKHHIILFDF\RIWKHDOJRULWKPHJSUHFLVLRQ 7KH VHFRQG FDWHJRU\ TXDQWLILHV WKH DGGLWLRQDO FRVW WKDW SUHIHWFKLQJ LQFXUV HJ WUDIILF LQFUHDVH RU SURFHVVRU WLPH 7KLVFRVWPD\EHFRPHUHDOO\KLJK)RUWKDWUHDVRQLWPXVWEHWDNHQLQWRDFFRXQWZKHQFRPSDULQJ SUHIHWFKLQJ WHFKQLTXHV 7KHUHIRUH WKRVH LQGH[HV FDQ EH VHHQ DV FRPSOHPHQWDU\ PHDVXUHV )LQDOO\ WKH WKLUGFDWHJRU\VXPPDUL]HVWKHSHUIRUPDQFHDFKLHYHGE\WKHV\VWHPIURPWKHXVHU¶VSRLQWRIYLHZ1RWLFH WKDWSUHIHWFKLQJWHFKQLTXHVPXVWWDNHFDUHRIWKHFRVWLQFUHDVHEHFDXVHWKH\FDQQHJDWLYHO\LPSDFWRQWKH RYHUDOOV\VWHPSHUIRUPDQFHWUDIILFLQFUHDVHXVHUSHUFHLYHGODWHQFLHV 7KHUHIRUHWKHWKUHHFDWHJRULHVDUH

FORVHO\UHODWHGVLQFHLQRUGHUWRDFKLHYHD JRRGRYHUDOOSHUIRUPDQFH&DWHJRU\ SUHIHWFKLQJ V\VWHPV PXVWWUDGHRIIWKHDJJUHVVLYHQHVVRIWKHDOJRULWKP&DWHJRU\ DQGWKHFRVWLQFUHDVHGXHWRSUHIHWFKLQJ &DWHJRU\ 'LIIHUHQW GHILQLWLRQV IRU WKH VDPH LQGH[ FDQ EH IRXQG LQ WKH OLWHUDWXUH HJ SUHFLVLRQ DQG WKLV IDFW LQFUHDVHVWKHKHWHURJHQHLW\RIWKHUHVHDUFKHIIRUWV,QRUGHUWRPDNHWKLVVXUYH\PRUHUHDGDEOHZHRQO\ LQFOXGHWKHGHILQLWLRQWKDWZHFRQVLGHUPRUHSUHFLVHDQGDSSURSULDWHIRUHYDOXDWLRQSXUSRVHV,QWKHFDVHV ZKHUH VHYHUDO QDPHV PDWFK WKH VDPH GHILQLWLRQ ZH VHOHFW WKH PRVW DSSURSULDWH LQGH[ QDPH IURP RXU SRLQW RI YLHZ 7KH JRDO RI WKLV VHFWLRQ LV QRW RQO\ WR KHOS WKH XQGHUVWDQGLQJ RI WKH LQGH[HV EXW DOVR WR GLVFXVV WKHLU XVHIXOQHVV GLVWLQJXLVKLQJ WKRVH XVHG IRU FRPSDULVRQ SXUSRVHV LQ DQ\ SUHIHWFKLQJ V\VWHP IURPWKRVHDSSOLFDEOHWRDSDUWLFXODUSUHIHWFKLQJWHFKQLTXHLHVSHFLILF 6SHFLILFLQGH[HVDUHRQO\IRXQG LQWKH&DWHJRU\3UHGLFWLRQ DVVKRZQLQ)LJXUH 3UHIHWFK0HWULFV 5HVRXUFH 8VDJH

3UHGLFWLRQ *HQHULF

6SHFLILF

1HWZRUN /HYHO

&RPSXWLQJ /HYHO

/DWHQF\

)LJ3UHIHWFKLQJPHWULFVWD[RQRP\

3UHGLFWLRQ5HODWHG,QGH[HV 7KLVJURXSLQFOXGHVWKRVHLQGH[HVDLPHGDWTXDQWLI\LQJWKHSHUIRUPDQFHWKDWWKHSUHGLFWLRQDOJRULWKP SURYLGHV 3UHGLFWLRQ SHUIRUPDQFH FDQ EH PHDVXUHG DW GLIIHUHQW PRPHQWV RU LQ GLIIHUHQW HOHPHQWV RI WKH DUFKLWHFWXUHIRULQVWDQFHZKHQZKHUH WKHDOJRULWKPPDNHVWKHSUHGLFWLRQDQGZKHQZKHUH SUHIHWFKLQJ LVDSSOLHGLQWKHUHDOV\VWHP7KXVHDFKLQGH[LQWKLVFDWHJRU\KDVDGXDOLQGH[HJZHFDQUHIHUWRWKH SUHFLVLRQRIWKHDOJRULWKPDQGWRWKHSUHFLVLRQRIWKHSUHIHWFK1RWLFHWKDWWKRVHLQGH[HVPHDVXUHGZKHQ WKH SUHGLFWLRQ OLVW LV JLYHQ GR QRW WDNH LQWR DFFRXQW V\VWHP ODWHQFLHV EHFDXVH WKH SUHGLFWLRQ DOJRULWKP ZRUNVLQGHSHQGHQWO\RIWKHXQGHUO\LQJQHWZRUNDQGWKHXVHU¶VUHVWULFWLRQV

*HQHULF3UHGLFWLRQ,QGH[HV 3UHFLVLRQ 3F 3UHFLVLRQ PHDVXUHV WKH UDWLR RI JRRG SUHGLFWLRQV WR WKH QXPEHU RI SUHGLFWLRQV >@ >@ >@ >@ >@ VHH (TXDWLRQ 3UHFLVLRQ GHILQHG LQ WKLV ZD\ MXVW HYDOXDWHV WKH DOJRULWKP ZLWKRXW FRQVLGHULQJSK\VLFDOV\VWHPUHVWULFWLRQVHJFDFKHQHWZRUNRUWLPHUHVWULFWLRQVWKHUHIRUHLWFDQEHVHHQ DVDWKHRUHWLFDOLQGH[ 6RPHUHVHDUFKVWXGLHVUHIHUWRWKLVLQGH[DVDFFXUDF\>@>@>@>@>@>@>@RUKLWUDWLR >@ZKLOHRWKHUVXVHDSUREDELOLVWLFQRWDWLRQHJ3UKLW_PDWFK LQVRPHPRGHOVEDVHGRQ0DUNRYFKDLQV OLNH>@

7KHUHDUHRWKHUUHVHDUFKZRUNVWKDWPHDVXUHWKHLPSDFWRQSHUIRUPDQFHRIWKHSUHFLVLRQ>@>@,Q WKHVH FDVHV WKH QXPEHU RI SUHIHWFKHG REMHFWV DQG SUHIHWFK KLWV DUH XVHG LQVWHDG RI WKH QXPEHU RI SUHGLFWLRQVDQGJRRGSUHGLFWLRQVUHVSHFWLYHO\VHH(TXDWLRQ

3F =

*RRG3UHGLFWLRQV 3UHGLFWLRQV

HT

3UHIHWFK+LWV 3UHIHWFKV

HT

3F =

,EUDKLPDQG;X>@HYDOXDWHWKHSUHIHWFKLQJSHUIRUPDQFHWKURXJKWKHZDVWHUDWLRZKLFKLVGHILQHGDV WKH SHUFHQWDJH RI XQGHVLUHG GRFXPHQWV WKDW DUH SUHIHWFKHG $V RQH FDQ REVHUYH WKLV LQGH[ LV WKH FRPSOHPHQWDU\RIWKHSUHFLVLRQ 5HFDOO5F 5HFDOOPHDVXUHVWKHSHUFHQWDJHRIUHTXHVWHGREMHFWVWKDWZHUHSUHYLRXVO\SUHIHWFKHG>@>@ >@ 7KH UHFDOO TXDQWLILHV WKH ZHLJKW RI WKH SUHGLFWHG VHH (TXDWLRQ RU SUHIHWFKHG REMHFWV VHH (TXDWLRQ RYHUWKHDPRXQWRIREMHFWVUHTXHVWHGE\WKHXVHU1RWLFHWKDWWKLVLQGH[RQO\GHDOVZLWKWKH QXPEHURIJRRGSUHGLFWLRQVPDGHEXWQRWZLWKWKHWRWDODPRXQWRISUHGLFWLRQV 6RPHUHVHDUFKZRUNVUHIHUWRWKLVPHWULFDVXVHIXOQHVV>@>@>@KLWUDWLR>@>@RUDFFXUDF\ >@>@3UHGLFWDELOLW\KDVEHHQHPSOR\HGLQ>@WRUHIHUWRWKHXSSHUOLPLWRIWKHUHFDOO

5F =

*RRG3UHGLFWLRQV 8VHU5HTXHVWV

HT

3UHIHWFK+LWV 8VHU5HTXHVWV

HT

5F =

$SSOLFDELOLW\ %RQLQR HWDO>@>@GHILQH DSSOLFDELOLW\DVWKHUDWLRRI WKH QXPEHURISUHGLFWLRQVWRWKH QXPEHURIUHTXHVWV,Q>@DXWKRUVLQFRUUHFWO\VWDWHWKDWWKLVLQGH[LVDV\QRQ\PRXVRIUHFDOO1RWLFHWKDW WKLVLQGH[FDQEHREWDLQHGIURPWKHSUHYLRXVRQHVLHGLYLGLQJWKHUHFDOOE\WKHSUHFLVLRQ WKHUHIRUHQR DGGLWLRQDOLQIRUPDWLRQLVJLYHQ7KLVLVWKHPDLQUHDVRQZK\QRRWKHUZRUNXVHVWKLVLQGH[

$SSOLFDELOLW\ =

3UHGLFWLRQV 8VHU5HTXHVWV

HT

3UHFLVLRQDORQHRUWRJHWKHUZLWKUHFDOOKDVEHHQWKHPRVWZLGHO\XVHGLQGH[WRHYDOXDWHWKHJRRGQHVV RI SUHGLFWLRQ DOJRULWKPV 7KH WLPH WDNHQ EHWZHHQ WKH FOLHQW UHTXHVW PDNLQJ DQG WKH UHVSRQVH UHFHLYLQJ FRQVLVWVRIIRXUPDLQFRPSRQHQWVLHFRQQHFWLRQHVWDEOLVKPHQWWLPHUHTXHVWWUDQVIHUHQFHWLPHUHTXHVW SURFHVVLQJWLPHDQGUHVSRQVHWUDQVIHUHQFHWLPH%RWKSUHFLVLRQDQGUHFDOODUHFORVHO\UHODWHGWRWKHWKUHH ILUVW FRPSRQHQWV 7KHUHIRUH DXWKRUV FRQVLGHU WKDW D FRPSOHWH FRPSDULVRQ VWXG\ DERXW SUHGLFWLRQ DOJRULWKPV VKRXOG DOVR LQFOXGH E\WH UHODWHG LQGH[HV WR TXDQWLI\ WKH ODVW FRPSRQHQW ,Q WKLV VHQVH DQDORJRXVO\ WR WKH ZHE SUR[\ FDFKLQJ LQGH[HV HJ WKH E\WH KLW UDWLR >@ ZH SURSRVH WKH XVH RI E\WH

SUHFLVLRQDQGE\WHUHFDOODVLQGH[HVWRHVWLPDWHWKHLPSDFWRISUHIHWFKLQJRQWKHWLPHWKDWWKHXVHUZDVWHV ZKHQZDLWLQJIRUWKHE\WHVRIWKHUHTXHVWHGREMHFWV %\WH3UHFLVLRQ3F% %\WHSUHFLVLRQPHDVXUHVWKHSHUFHQWDJHRISUHGLFWHGRUSUHIHWFKHG E\WHVWKDWDUH VXEVHTXHQWO\UHTXHVWHG,WFDQEHFDOFXODWHGE\UHSODFLQJWKHQXPEHURISUHGLFWHGREMHFWVZLWKWKHLUVL]H LQE\WHVLQ(TXDWLRQ 5HPDUNWKDWOLNHSUHFLVLRQGRHVE\WHSUHFLVLRQTXDQWLILHVKRZJRRGWKHSUHGLFWLRQVDUHEXWPHDVXUHG LQE\WHVLQVWHDGRILQREMHFWV$QHDUOLHUDSSURDFKWRWKLVLQGH[LVWKH0LVVUDWHUDWLR>@GHVFULEHGEHORZ %RXUDVHWDO>@DOVRPHQWLRQHGWKDWGLIIHUHQWUHVXOWVFRXOGEHREWDLQHGZKHQXVLQJWKHQXPEHURIE\WHV LQVWHDGRIWKHQXPEHURIREMHFWV

3F% =

*RRG3UHGLFWLRQV % 3UHGLFWLRQV %

HT

%\WH5HFDOO5F% %\WHUHFDOOPHDVXUHVWKHSHUFHQWDJHRIGHPDQGHGE\WHVWKDWZHUHSUHYLRXVO\SUHGLFWHG RU SUHIHWFKHG $V PHQWLRQHG DERYH WKLV LQGH[ TXDQWLILHV KRZ PDQ\ DFFXUDWH SUHGLFWLRQV DUH PDGH PHDVXUHGLQWUDQVIHUUHGE\WHV 7KLVLQGH[EHFRPHVPRUHKHOSIXOWKDQWKHSUHYLRXVO\PHQWLRQHGUHFDOOZKHQWKHWUDQVPLVVLRQWLPHLV DQLPSRUWDQWFRPSRQHQWRIWKHRYHUDOOXVHUSHUFHLYHGODWHQF\

5F% =

*RRG3UHGLFWLRQV % 8VHU5HTXHVWV %

HT

6SHFLILF3UHGLFWLRQ,QGH[HV 5HTXHVW VDYLQJV 5HTXHVWVDYLQJV PHDVXUHVWKH QXPEHURI WLPHVWKDWD XVHUUHTXHVWKLWV LQWKHEURZVHU FDFKHRUWKHUHTXHVWHGREMHFWLVEHLQJSUHIHWFKHGLQSHUFHQWDJHRIWKHWRWDOQXPEHURIXVHUUHTXHVWV>@ )XUWKHUPRUH WKH UHTXHVW VDYLQJV FDQ EH EURNHQ GRZQ LQWR WKUHH JURXSV GHSHQGLQJ RQ LI WKH\ ZHUH SUHYLRXVO\SUHIHWFKHGKDYHEHHQSDUWLDOO\SUHIHWFKHGRUFDFKHGDVQRUPDOREMHFWV 1RWLFH WKDW ZKHQ SUHIHWFKLQJ LV XVHULQLWLDWHG WKH QXPEHU RI UHTXHVWV LQFUHDVHV WKHUHIRUH WKLV LQGH[ PDNHVVHQVHZKHQSUHIHWFKLQJLVSUR[\RUVHUYHULQLWLDWHG)DQHWDOXVHWKLVLQGH[LQDSUHIHWFKV\VWHP ZKHUHWKHSUR[\SXVKHVREMHFWVWRWKHFOLHQW1HYHUWKHOHVVWKLVLQGH[FRXOGEHDOVRXVHGZKHQSUHIHWFKLQJ SXVKHVREMHFWVIURPWKHVHUYHUWRWKHSUR[\RUIURPWKHVHUYHUWRWKHFOLHQWZLWKQRLQWHUPHGLDWHSUR[LHV 0LVVUDWHUDWLR%HVWDYURVGHILQHVWKHPLVVUDWHUDWLR>@DVWKHUDWLREHWZHHQWKHE\WHPLVVUDWHZKHQWKH SUHIHWFKLVHPSOR\HGWRWKHE\WHPLVVUDWHZKHQWKHSUHIHWFKLVQRWHPSOR\HGZKHUHWKHE\WHPLVVUDWHIRU DJLYHQFOLHQWLVWKHUDWLRRIE\WHVQRWIRXQGLQWKHFOLHQW¶VFDFKHWRWKHWRWDOQXPEHURIE\WHVDFFHVVHGE\ WKDWFOLHQW $VRQHFDQVHHWKLVLQGH[TXDQWLILHVLQZKLFKSHUFHQWDJHWKHPLVVUDWHLQWKHFOLHQWFDFKHGURSVGXHWR WKH SUHIHWFKLQJ V\VWHP 7KLV LV D VSHFLILF LQGH[ VLQFH LW LV RQO\ DSSOLFDEOH WR WKRVH V\VWHPV VWRULQJ WKH SUHIHWFKHGREMHFWVLQWKHFOLHQW¶VFDFKH

3UREDELOLW\ RI PDNLQJ D SUHGLFWLRQ 3LWNRZ HW DO >@ TXDQWLI\ WKH SUREDELOLW\ WKDW WKH ODVW DFFHVVHV PDWFK WKH SDWWHUQ SUHGLFWLRQ LQ VXFK FDVH WKH SUHIHWFK V\VWHP FRPSXWHV WKH SUHGLFWLRQ RXWFRPHV 7KLV LQGH[FDQEHDSSOLHGIRUH[DPSOHWRWKRVHV\VWHPVEDVHGLQ0DUNRYPRGHOVEXWFDQQRWEHDSSOLHGWRD ODUJHVXEVHWRISUHIHWFKLQJV\VWHPVHJWKHWRSDSSURDFKHV>@VRLWLVFODVVLILHGDVVSHFLILF

5HVRXUFHXVDJH 7KHEHQHILWVRISUHIHWFKLQJDUHDFKLHYHGDWWKHH[SHQVHRIXVLQJDGGLWLRQDOUHVRXUFHV7KLVRYHUKHDGDV PHQWLRQHGDERYHPXVWEHTXDQWLILHGEHFDXVHWKH\FDQQHJDWLYHO\LPSDFWRQSHUIRUPDQFH $OWKRXJK VRPH SUHGLFWLRQ DOJRULWKPV PD\ UHTXLUH KXJH PHPRU\ RU SURFHVVRU WLPH HJ KLJK RUGHU 0DUNRYPRGHOV LWLVQRWWKHFXUUHQWJHQHUDOWUHQGZKHUHWKHPDLQSUHIHWFKLQJERWWOHQHFNLVWKHQHWZRUN WUDIILF7KHUHIRUH ZHGLYLGHLQGH[HV LQWKLVFDWHJRU\ LQWRWZRVXEJURXSVQHWZRUNOHYHO DQGFRPSXWLQJ OHYHO

1HWZRUNOHYHO 7UDIILFLQFUHDVHΔ7U% 7UDIILFLQFUHDVHTXDQWLILHVWKHLQFUHDVHRIWUDIILFLQE\WHV GXHWRXQVXFFHVVIXOO\ SUHIHWFKHG GRFXPHQWV >@ VHH (TXDWLRQ ,W LV DOVR FDOOHG ZDVWHG EDQGZLGWK >@ H[WUD E\WHV >@ QHWZRUNWUDIILF>@>@DQGEDQGZLGWKUDWLR>@ :KHQ XVLQJ WKH SUHIHWFK QHWZRUN WUDIILF XVXDOO\ LQFUHDVHV GXH WR WZR VLGH HIIHFWV RI WKH SUHIHWFK REMHFWVQRWXVHGDQGRYHUKHDG2EMHFWVQRWXVHGZDVWHQHWZRUNEDQGZLGWKEHFDXVHWKHVHREMHFWVDUHQHYHU UHTXHVWHG E\ WKH XVHU 2Q WKH RWKHU KDQG WKH QHWZRUN WUDIILF LQFUHDVHV GXH WR WKH SUHIHWFK UHODWHG LQIRUPDWLRQLQWHUFKDQJHFDOOHG1HWZRUNRYHUKHDGE\>@ 6HYHUDOUHVHDUFKVWXGLHVIDLOWRWDNHLQWRDFFRXQWWKDWRYHUKHDG>@>@>@WKHUHIRUHWKHLUUHVXOWV FDQQRWHVWLPDWHSUHIHWFKLQJFRVWVDFFXUDWHO\

Δ7U% =

2EMHFWV1RW8VHG % + 1HWZRUN2YHUKHDG % + 8VHU5HTXHVWV % 8VHU5HTXHVWV %

HT

([WUD FRQWURO GDWD SHU E\WH ([WUD FRQWURO GDWD SHU E\WH TXDQWLILHV WKH H[WUD E\WHV WUDQVIHUUHG WKDW DUH UHODWHG WR WKH SUHIHWFK LQIRUPDWLRQ DYHUDJHG SHU HDFK E\WH UHTXHVWHG E\ WKH XVHU ,W LV WKH 1HWZRUN RYHUKHDG UHIHUUHG LQ >@ EXW TXDQWLILHG SHU UHTXHVWHG E\WHV DQG ZLOO EH XVHG EHORZ ZKHQ UHODWLQJ 7UDIILFLQFUHDVHWRWKHSUHGLFWLRQLQGH[HV

([WUD&RQWURO'DWD% =

1HWZRUN2YHUKHDG % 8VHU5HTXHVWV %

HT

2EMHFW WUDIILF LQFUHDVH Δ7URE 2EMHFW WUDIILF LQFUHDVH TXDQWLILHV LQ ZKLFK SHUFHQWDJH WKH QXPEHU RI GRFXPHQWVWKDWFOLHQWVJHWZKHQXVLQJSUHIHWFKLQJLQFUHDVHV1DQRSRXORVHWDO>@UHIHUWRWKLVLQGH[DV QHWZRUNWUDIILFDQG5DELQRYLFK>@DVH[WUDUHTXHVWV

$V(TXDWLRQVKRZVWKLVLQGH[HVWLPDWHVWKHUDWLRRIWKHDPRXQWRISUHIHWFKHGREMHFWVQHYHUXVHG ZLWK UHVSHFW WR WKH WRWDO XVHU¶V UHTXHVWV ,W LV DQDORJRXV WR WKH WUDIILF LQFUHDVH EXW LW PHDVXUHV WKH RYHUKHDGLQQXPEHURIREMHFWV

Δ7URE =

2EMHFWV1RW8VHG + 8VHU5HTXHVWV 8VHU5HTXHVWV

HT

&RPSXWLQJOHYHO 6HUYHU ORDG UDWLR 6HUYHU ORDG UDWLR LV GHILQHG DV WKH UDWLR EHWZHHQ WKH QXPEHU RI UHTXHVWV IRU VHUYLFH ZKHQVSHFXODWLRQLVHPSOR\HGWRWKHQXPEHURIUHTXHVWVIRUVHUYLFHZKHQVSHFXODWLRQLVQRWHPSOR\HG>@ 6SDFHDQG7LPHRYHUKHDG,QDGGLWLRQWRWKHVHUYHUORDGVRPHUHVHDUFKZRUNVGLVFXVVKRZWKHRYHUKHDG LPSDFWVRQSHUIRUPDQFH)RULQVWDQFH'XFKDPS>@GLVFXVVHVWKHPHPRU\DQGSURFHVVRUWLPHWKDWWKH SUHIHWFKZRXOGQHHG 3UHGLFWLRQWLPH7KLVLQGH[TXDQWLILHVWKHWLPHWKDWWKHSUHGLFWRUWDNHVWRPDNHDSUHGLFWLRQ,WLVXVHGLQ >@WRFRPSDUHGLIIHUHQWSUHGLFWLQJDOJRULWKPV

/DWHQF\UHODWHGLQGH[HV ,QGH[HV EHORQJLQJ WR WKLV FDWHJRU\ LQFOXGH WKRVH DLPHG DW TXDQWLI\LQJ WKH HQGWRHQG ODWHQFLHV HJ XVHU RU SUR[\ UHODWHG ODWHQFLHV 7KH PDLQ GUDZEDFN RI WKHVH LQGH[HV LV WKDW WKH\ LQFOXGH VHYHUDO WLPH FRPSRQHQWVVRPHRIWKHPGLIILFXOWWRTXDQWLI\0DQ\WLPHVUHVHDUFKHUVGRQRWGHWDLOZKLFKFRPSRQHQWV WKH\ PHDVXUH DOWKRXJK WKH\ XVH D W\SLFDO LQGH[ QDPH LH ODWHQF\ 6HYHUDO QDPHV KDYH EHHQ XVHG LQVWHDGIRULQVWDQFHDFFHVVWLPH>@VHUYLFHWLPH>@DQGUHVSRQVLYHQHVV>@>@7KLVVLWXDWLRQLVQRW WKHEHVWIRUWKHUHVHDUFKFRPPXQLW\GXHWRWKHIDFWWKDWWKHGLIIHUHQWSURSRVDOVFDQQRWEHIDLUO\FRPSDUHG DPRQJWKHP 7KURXJK WKH GLIIHUHQW UHVHDUFK ZRUNV ODWHQFLHV DUH PHDVXUHG ERWK SHU SDJH DQG SHU REMHFW 7KH /DWHQF\SHUSDJH/S LVFDOFXODWHGE\FRPSDULQJWKHWLPHEHWZHHQEURZVHU¶VLQLWLDWLRQRIDQ+70/SDJH *(7 DQG EURZVHU¶V UHFHSWLRQ RI WKH ODVW E\WH RI WKH ODVW HPEHGGHG LPDJH RU REMHFW IRU WKDW SDJH >@ $QDORJRXVO\WKH/DWHQF\SHUREMHFW/RE FDQEHGHILQHGDVWKHHODSVHGWLPHVLQFHWKHEURZVHUUHTXHVWV DQ REMHFW XQWLO LW UHFHLYHV WKH ODVW E\WH RI WKDW REMHFW ,Q RUGHU WR LOOXVWUDWH WKH EHQHILWV RI SUHIHWFKLQJ UHVHDUFKHUVFDOFXODWHWKHUDWLRRIWKHODWHQF\WKDWSUHIHWFKLQJDFKLHYHVHLWKHUSHUSDJH>@>@>@RU SHUREMHFW>@ WRWKHODWHQF\ZLWKQRSUHIHWFKLQJ 8QIRUWXQDWHO\ VRPH SURSRVDOV WKDW XVH ODWHQF\ ZKHQ PHDVXULQJ SHUIRUPDQFH GR QRW VSHFLI\ ZKLFK ODWHQF\WKH\DUHUHIHUULQJWRHJ>@>@>@7KLVIDFWFDQEHPLVOHDGLQJEHFDXVHERWKLQGH[HVGRQRW SHUIRUP LQ WKH VDPH ZD\ ,Q RUGHU WR LOOXVWUDWH WKLV SRLQW ZH SUHVHQW WKH IROORZLQJ H[DPSOH D XVHU UHTXHVWVDQ+70/REMHFWHPEHGGLQJWZRLPDJHV,0*DQG,0* $V)LJXUHVKRZVWKHWUDQVIHUHQFH RIWKH+70/ILOHVWDUWVDWWDQGHQGVDWW$WWLPHVWDQGWWKHEURZVHUUHDGVDQGSURFHVVHVWKH,0* WDJVRIWKHHPEHGGHGLPDJHV7KHQLWVWDUWVWKHWUDQVIHUHQFHVZKLFKHQGDWWDQGWUHVSHFWLYHO\,QWKLV

FDVHWKHFXPXODWLYH/RELVWKHVXPRIWKHWLPHWDNHQE\WKHWKUHHWUDQVIHUHQFHV/RE WWWWWW ZKHUHWKH/DWHQF\SHUSDJH/S LVWW,ILWLVDVVXPHGWKDW,0*ZDVSUHYLRXVO\SUHIHWFKHGQRZDLWLQJ WLPHIRUVXFKREMHFWZLOOEHSORWWHGLHWZLOOPDWFKWLQVXFKDZD\WKDWLWZLOOUHGXFH/REEXWQRW/S ZKLFKZLOOUHPDLQWKHVDPHYDOXH 7RREVHUYHWKLVIHDWXUHLQDUHDOHQYLURQPHQWIRUDJLYHQFOLHQWLVQHFHVVDU\WKDWWKHREMHFWUHWULHYLQJ WLPHV DUH LQGHSHQGHQW 1HYHUWKHOHVV DOWKRXJK WLPHV DUH QRW LQGHSHQGHQW ERWK LQGH[HV KDYH GLIIHUHQW YDOXHVDVH[SHULPHQWDOUHVXOWVVKRZ +70/ ,0* ,0*

W

W

W

7LPH

W

W

W

)LJ([DPSOHZKHUHODWHQF\PHWULFVEHKDYHGLIIHUHQWZKHQWKHSUHIHWFKKLWVRQRQHRIWKHLPDJHV

)XUWKHUPRUHWKH/DWHQF\SHUREMHFWPHDVXUHGZKHQDQREMHFWLVGRZQORDGHGE\WKHZHEEURZVHUKDV WZRPDLQFRPSRQHQWVL WKHTXHXLQJWLPHVLQFHWKHEURZVHUKDVDOLPLWHGQXPEHURIFRQQHFWLRQVDQGLL WKH UHTXHVW FRPSOHWLRQ WLPH LQFOXGLQJ WKH WLPH RI FRQQHFWLRQ HVWDEOLVKPHQW LI QHHGHG DQG WKH REMHFW WUDQVIHUHQFH7KHILUVWFRPSRQHQWLVRIWHQLJQRUHG>@DQGRWKHUUHVHDUFKVWXGLHV>@>@GRQRWVSHFLI\ ZKHWKHU WKH PHDVXUHG ODWHQF\ LQFOXGHV WKH TXHXLQJ WLPH 7KH ILUVW FRPSRQHQW VKRXOG QRW EH LJQRUHG EHFDXVHSUHIHWFKKLWVGRQRWFRPSHWHIRUDIUHHFRQQHFWLRQVRWKHTXHXLQJWLPHRIWKHUHPDLQLQJREMHFWV GHFUHDVHVDQGFRQVHTXHQWO\WKHLUODWHQF\ 1RWH WKDW DOO WKH VWDWHG UHODWLRQV EHWZHHQ ERWK ODWHQF\ UHODWHG LQGH[HV DUH YDOLG IRU DQ\ ZHE ODWHQF\ UHGXFWLRQ WHFKQLTXH DQG QRW RQO\ IRU SUHIHWFKLQJ ,Q WKLV VHQVH WKH GHILQLWLRQV RI WKHVH LQGH[HV FDQ EH DGDSWHGWRLQFOXGHDQ\ ZHEWHFKQLTXHDLPHGDWUHGXFLQJWKHILQDOXVHU¶VSHUFHLYHGODWHQF\7KHODWHQF\ SHUSDJHRUSHUREMHFW UDWLRFDQEHGHILQHGLQDZLGHUZD\DVWKHUDWLREHWZHHQWKHODWHQF\SHUSDJHRU SHUREMHFW ZKHQDODWHQF\UHGXFWLRQWHFKQLTXHLVXVHGWRWKHODWHQF\SHUSDJHRUSHUREMHFW ZKHQWKDW WHFKQLTXHLVQRWXVHG

6XPPDU\V\QRQ\PRXVDQGH[SHULPHQWDOLQGH[FDWHJRU\XVHG 7KURXJK WKH SURSRVHG WD[RQRP\ ZH KDYH GLVFXVVHG WKH ODUJH YDULHW\ RI LQGH[ QDPHV V\QRQ\PRXV XVHG WR UHIHU WR WKH VDPH PHWULF 7KLV KHWHURJHQHLW\ FDQ EH REVHUYHG WKURXJK ZHE SHUIRUPDQFH VWXGLHV DSSHDUHGLQWKHOLWHUDWXUHDIDFWWKDWDGGVH[WUDGLIILFXOW\WRREWDLQDJHQHUDOYLHZRIWKHVXEMHFW7DEOH RIIHUVDVFKHPHRIWKHFXUUHQWVLWXDWLRQ,QDGGLWLRQ ZHDOVRKDYH IRXQGWKDWWKHVDPHLQGH[ QDPH KDV EHHQXVHGZKHQPHDVXULQJGLIIHUHQWYDULDEOHVHJDFFXUDF\DSSHDUVERWKIRUSUHFLVLRQDQGIRUUHFDOO $V RQH FDQ REVHUYH WKH ZLGHVW KHWHURJHQHLW\ LV SUHVHQW LQ WKH ILUVW FDWHJRU\ GXH WR WKH ZLGH UDQJH RI SUHGLFWLRQ DOJRULWKPV DSSHDUHG LQ WKH OLWHUDWXUH DQG GXH WR LWV LPSRUWDQFH LQ WKH SUHIHWFKLQJ V\VWHPV ZKLFKDUHWKHPDLQIRFXVRIWKLVZRUN

7DEOHFODVVLILHVDUHSUHVHQWDWLYHVXEVHWRIUHVHDUFKZRUNVWKDWFDQEHIRXQGLQWKHRSHQOLWHUDWXUHDQG VKRZV WKH FDWHJRULHV RI LQGH[HV WKDW DUH XVHG LQ VXFK ZRUNV $V GLVFXVVHG DERYH D FRPSOHWH UHVHDUFK ZRUNVKRXOGLQFOXGHLQGH[HVEHORQJLQJWRWKHWKUHHFDWHJRULHV+RZHYHUZHRQO\KDYHREVHUYHGWKLVIDFW LQ RI WKH H[SORUHG ZRUNV MXVW IRXU RI WKH WZHQW\ILYH ZRUNV URZ 1RWLFH WKDW WKH ILUVW FROXPQ VKRZV WKDW RI WKH UHVHDUFK ZRUNV KDYH DW OHDVW RQH RI WKH LQGH[HV EHORQJLQJ WR WKH SUHGLFWLRQ FDWHJRU\ 0RUHRYHU RI WKH VWXGLHV RQO\ PHDVXUH SUHGLFWLRQ PHWULFV WR HYDOXDWH WKH V\VWHP SHUIRUPDQFH7KHVHFRQGFROXPQ VWDWHVWKDWRIWKHSDSHUVGRQRWPHDVXUHUHVRXUFHXVDJH PHWULFV )LQDOO\ WKH WKLUG FROXPQ VKRZV WKDW RI WKH UHVHDUFK ZRUNV GR QRW TXDQWLI\ WKH HQG SHUFHLYHG ODWHQFLHV 7DEOH5HODWLRQEHWZHHQWKHVHOHFWHGLQGH[QDPHLQWKLVSDSHUDQGWKRVHDSSHDUHGLQWKHOLWHUDWXUH

&DWHJRU\

6HOHFWHGQDPH

3UHFLVLRQ 3UHGLFWLRQ

5HFDOO

$SSOLFDELOLW\ 7UDIILFLQFUHDVH 5HVRXUFH XVDJH

2EMHFW LQFUHDVH

1DPH

5HIHUHQFHV

3UHFLVLRQ

>@>@>@>@>@>@

$FFXUDF\

>@>@>@>@>@>@>@

3UKLW_PDWFK

>@

+LWUDWLR

>@

:DVWHUDWLR

>@

5HFDOO

>@ >@ >@

8VHIXOQHVV

>@ >@ >@

+LW5DWLR

>@>@

3UHGLFWDELOLW\

>@

$FFXUDF\

>@ >@

$SSOLFDELOLW\

>@>@

7UDIILFLQFUHDVH

>@ >@ >@

:DVWHG%DQGZLGWK >@ %DQGZLGWKUDWLR

>@

([WUDE\WHV

>@

'DWDWUDQVIHU

>@

1HWZRUN7UDIILF

>@ >@

WUDIILF 1HWZRUN7UDIILF

3UHGLFWLRQWLPH /DWHQF\

/LWHUDWXUH

>@

([WUDUHTXHVWV

>@

3UHGLFWLRQWLPH

>@

/DWHQF\SHUSDJH /DWHQF\ 5HVSRQVLYHQHVV /DWHQF\SHUREMHFW /DWHQF\

>@ >@ >@ >@ >@ >@ >@ >@

$FFHVVWLPH

>@

6HUYLFHWLPHUDWLR

>@

7DEOH5HODWLRQEHWZHHQWKHUHVHDUFKVWXGLHVDQGWKHLQGH[HVXVHGJURXSHGE\FDWHJRU\

&DWHJRU\

5HIHUHQFHV

3UHGLFWLRQ 5HVRXUFH /DWHQF\

>@>@>@>@

;

;

;

;

;

;

;

;

>@>@>@>@

;

;

>@>@>@>@>@>@>@>@>@>@>@

;

>@>@ >@ >@>@>@

727$/

([SHULPHQWDOHQYLURQPHQW ,Q WKLV VHFWLRQ ZH ILUVW GHVFULEH WKH H[SHULPHQWDO IUDPHZRUN XVHG IRU WKH SHUIRUPDQFH HYDOXDWLRQ VWXG\6HFRQGO\ ZHEULHIO\SUHVHQWWKHSUHIHWFKLQJDOJRULWKPVFRQVLGHUHGLQWKHHYDOXDWLRQDQGILQDOO\ ZHSUHVHQWWKHPDLQFKDUDFWHULVWLFVRIWKHZRUNORDGHPSOR\HG

6\VWHP$UFKLWHFWXUH ,Q>@ZHSURSRVHGDQGGHVFULEHGLQGHWDLODQH[SHULPHQWDOIUDPHZRUNIRUWHVWLQJZHESUHIHWFKLQJ WHFKQLTXHV ,Q WKLV VHFWLRQ ZH VXPPDUL]H WKH PDLQ IHDWXUHV RI VXFK HQYLURQPHQW DQG KRZ LW KDV EHHQ H[WHQGHGWRFDUU\RXWWKHH[SHULPHQWVSUHVHQWHGLQWKLVSDSHU 7KHDUFKLWHFWXUHLVFRPSRVHGE\WKUHHPDLQSDUWVDVVKRZQLQ)LJXUH WKHEDFNHQGVHUYHUDQG VXUURJDWH WKHIURQWHQGFOLHQW DQGRSWLRQDOO\WKHSUR[\ZKLFKLVQRWXVHGLQWKLVSDSHU7KHIUDPHZRUN LPSOHPHQWDWLRQFRPELQHVERWKUHDODQGVLPXODWHGSDUWVLQRUGHUWRSURYLGHIOH[LELOLW\DQGDFFXUDF\ Set of clients

Proxy . . .

. . .

Set of clients

Surrogate

Web Server

Proxy

)LJ$UFKLWHFWXUHRIWKHVLPXODWLRQHQYLURQPHQW

7KHEDFNHQGSDUWLQFOXGHVWKHZHEVHUYHUDQGWKHVXUURJDWHVHUYHU7KHIUDPHZRUNHPXODWHVDUHDO VXUURJDWHZKLFKLVXVHGWRDFFHVVWRDUHDOZHEVHUYHU$OWKRXJKWKHPDLQJRDORIVXUURJDWHVLVWRDFWDVD FDFKHIRUWKHPRVWSRSXODUVHUYHUUHVSRQVHVZHXVHLWDVDSUHGLFWRUHQJLQH 7KHIURQWHQGRUFOLHQWSDUWUHSUHVHQWVWKH XVHUV¶EHKDYLRUEURZVLQJWKH ZHE ZLWKDSUHIHWFKLQJ HQDEOHGEURZVHUOLNH0R]LOOD>@ 7RPRGHOWKHVHWRIXVHUVWKDWDFFHVVFRQFXUUHQWO\WRDJLYHQVHUYHU WKHVLPXODWRUFDQEHIHGE\XVLQJHLWKHUUHDORUV\QWKHWLFDOO\JHQHUDWHGWUDFHV7KHVLPXODWRUFROOHFWVEDVLF

LQIRUPDWLRQIRUHDFKUHTXHVWSHUIRUPHGWRWKH ZHEVHUYHUDQGWKHQZULWHVLWWRDORJILOH%\DQDO\]LQJ WKLVORJDWSRVWVLPXODWLRQWLPHDOOSHUIRUPDQFHPHWULFVFDQEHFDOFXODWHG 7KHH[WHQVLRQRIWKH VLPXODWRUSUHVHQWHGLQ WKLVSDSHUFRQFHQWUDWHVRQWKHFOLHQWSDUWLQRUGHUWR SURYLGH WKH DGGLWLRQDO LQIRUPDWLRQ QHHGHG WR HYDOXDWH WKH XVHIXOQHVV RI WKH SURSRVHG LQGH[HV ,Q WKLV FRQWH[WWKHH[WHQVLRQLQFOXGHVWKUHHPDLQIHDWXUHV •

7KH FRQFHSW RI SDJH LV LQWURGXFHG ,Q WKLV ZD\ VWDWLVWLFV SHU SDJH FDQ DOVR EH SURYLGHG LH WKH VLPXODWRU HVWLPDWHV ERWK WKH SHUFHLYHG ODWHQF\ IRU GRZQORDGLQJ HDFK LQGLYLGXDO REMHFW DQG WKH SHUFHLYHGODWHQF\IRUGRZQORDGLQJWKHZKROHSDJH

•

1HZLQGH[HVDUHFDOFXODWHG7KHSURSRVHGLQGH[HVDUHQRZLPSOHPHQWHGDQGFDOFXODWHGWRFKHFNLWV XVHIXOQHVV

•

8VHUVKDYHDOLPLWHGVHVVLRQWLPH$IWHUWKLVWLPHFOLHQWFDFKHLVHPSWLHGUHSUHVHQWLQJWKHH[SLUDWLRQ WLPHRIWKHFDFKHGREMHFWV

3UHIHWFKLQJ$OJRULWKPV ,QRUGHUWRFKHFNWKHEHKDYLRURIWKHLQGH[HVWKHH[SHULPHQWVZHUHUXQXVLQJWZRGLIIHUHQWWUDFHVDQG WZRGLIIHUHQWSUHIHWFKLQJDOJRULWKPVWKH'HSHQGHQF\*UDSK'* EDVHGRQWKHDOJRULWKPSURSRVHGE\ 3DGPDQDEKDQ DQG 0RJXO >@ DQG WKH 3UHGLFWLRQ E\ 3DUWLDO 0DWFK 330 EDVHG RQ WKH DOJRULWKP SURSRVHGE\3DOSDQDVDQG0HQGHO]RQ>@ 7KH'*SUHGLFWLRQDOJRULWKPFRQVWUXFWVDGHSHQGHQF\JUDSKWKDWGHSLFWVWKHSDWWHUQRIDFFHVVHVWRWKH VHUYHUILOHV7KHJUDSKKDVDQRGHIRUHYHU\ILOHWKDWKDVHYHUEHHQDFFHVVHG7KHUHLVDQDUFIURPQRGH$ WR%LIDQGRQO\LIDWVRPHSRLQWLQWLPHDFOLHQWDFFHVVHG%ZLWKLQ ZDFFHVVHVDIWHU $ ZKHUHZLVWKH ORRNDKHDGZLQGRZVL]H7KHZHLJKWRQWKHDUFLVWKHUDWLRRIWKHQXPEHURIDFFHVVHVWR%ZLWKLQDZLQGRZ DIWHU$WRWKHQXPEHURIDFFHVVHVWR$LWVHOI7KHSUHIHWFKLQJDJJUHVVLYHQHVVLVFRQWUROOHGE\DWKUHVKROG SDUDPHWHUDSSOLHGWRWKHDUFVZHLJKW 7KH330SUHGLFWLRQDOJRULWKPXVHV0DUNRYPRGHOVRIPRUGHUVWRVWRUHSUHYLRXVFRQWH[WV3UHGLFWLRQV DUH REWDLQHG IURP FRPSDULQJ WKH FXUUHQW FRQWH[W ZLWK HDFK 0DUNRY PRGHO /LNH '* WKH SUHIHWFKLQJ DOJRULWKPFRQWUROVWKHDJJUHVVLYHQHVVWKURXJKDXVHUGHILQHGWKUHVKROG

:RUNORDG'HVFULSWLRQ 7KH EHKDYLRU SDWWHUQRI WKH XVHUV ZDV WDNHQ IURP ORJV FROOHFWHG GXULQJ HLJKW GD\V EHWZHHQ 0D\ WK DQG0D\WKE\D6TXLGSUR[\RIWKH3RO\WHFKQLF8QLYHUVLW\RI9DOHQFLD)URPWKLVORJDFFHVVHV WRWZRRIWKHPRVWSRSXODUVHUYHUVLQWKLVHQYLURQPHQWZHUHWDNHQDQGUHSURGXFHGLQWKHVLPXODWLRQXVLQJ WKHILUVWZHHNWRWUDLQWKHSUHIHWFKLQJDOJRULWKPDQGWKHIROORZLQJGD\WRREWDLQWKHSHUIRUPDQFHLQGH[HV 7KH ILUVW WUDFH $ FRQWDLQV DFFHVVHV WR D QHZV ZHE VHUYHU ZKHUHDV WKH VHFRQG RQH % LQFOXGHV WKH DFFHVVHV WR D VWXGHQW LQIRUPDWLRQ ZHE VHUYHU 7KH PDLQ FKDUDFWHULVWLFV RI WKH WUDFH FDQ EH IRXQG LQ 7DEOH

7DEOH7UDFHVFKDUDFWHULVWLFV

7UDFH

$

%

7UDLQLQJDFFHVVHV

([SHULPHQWDFFHVVHV

+70/VDFFHVVHV

8VHUV %\WHVWUDQVIHUUHG.%

5HODWLRQEHWZHHQLQGH[HV 7KLVVHFWLRQKDVWKUHHPDLQJRDOV7KHILUVWRQHLVWRLOOXVWUDWHWKHXVHIXOQHVVRIWKHLQGH[HVLQWURGXFHG LQWKLVSDSHULHE\WHSUHFLVLRQE\WHUHFDOODQGH[WUDFRQWUROGDWDSHUE\WH DVZHOODVWKHZD\LQZKLFK WKH\ GLIIHU IURP WKH DQDORJRXV FODVVLFDO RQHV 7KH VHFRQG RQH LV WR VWXG\ DQG VKRZ KRZ LQGH[HV DUH UHODWHGDPRQJWKHP)LQDOO\WKHWKLUGJRDOLVWRLGHQWLI\WKHPRVWPHDQLQJIXOLQGH[HVZKHQHYDOXDWLQJWKH RYHUDOOV\VWHPSHUIRUPDQFH:HXVHERWKHPSLULFDODQGDQDO\WLFDODSSURDFKHVWRVKRZKRZLQGH[HVDUH LQWHUUHODWHG:KHQDQDO\WLFDOUHODWLRQVZHUHIRXQGUHVXOWVIRUWKHHPSLULFDODSSURDFKDUHQRWVKRZHG

(PSLULFDOWHVWV 7KHUHODWLRQVDPRQJLQGH[HVDUHFKHFNHGE\FRPSDULQJWKHPE\SDLUVWKDWFRXOGEHSRWHQWLDOO\UHODWHG WKURXJK GLIIHUHQW SUHIHWFKLQJ VFHQDULRV 3DLUV ZHUH VHOHFWHG IURP WKH PRVW ZLGHO\ XVHG LQGH[HV DV GLVFXVVHG LQ 6HFWLRQ SUHFLVLRQ E\WH SUHFLVLRQ UHFDOO E\WH UHFDOO ODWHQF\ SHU SDJH DQG ODWHQF\ SHU REMHFW ,QRUGHUWRGHWHFWWKHSRVVLEOHUHODWLRQVDIRXUJUDSKLFILJXUHLVSORWWHGIRUHDFKSDLURILQGH[HVDVD UHVXOWRIFRPELQLQJWKHWZRGHVFULEHGDOJRULWKPVXVLQJWKHWZRPHQWLRQHGWUDFHV7KHQLQWKRVHJUDSKLFV ZKHUH VRPH VLJQ RI OLQHDU UHODWLRQ DSSHDUV ZH TXDQWLI\ LW WKURXJK WKH OLQHDU FRUUHODWLRQ FRHIILFLHQW ,Q RUGHUWRREVHUYHKRZWKHLQGH[HVEHKDYHLQDZLGHUDQJHRIVLWXDWLRQVWKHH[SHULPHQWVZHUHUXQUDQJLQJ WKH WKUHVKROG YDOXH RI WKH SUHIHWFKLQJ DOJRULWKPV IURP WR ZLWK D LQFUHPHQW 3RLQWV LQ WKH JUDSKLFVUHIHUWRWKHSHUIRUPDQFHLQGH[HVPHDVXUHGIRUHYHU\FOLHQWLQHDFKH[SHULPHQW5HPDUNWKDWWKH PHDVXUHGYDOXHVRIWKHSUHGLFWLRQLQGH[HVUHIHUWRWKHSUHIHWFKHGREMHFWVLQVWHDGRIWKHSUHGLFWHGREMHFWV VLQFHWKH\DUHFORVHUWRWKHV\VWHPSHUIRUPDQFH )LJXUH SORWV WKH /DWHQF\ SHU REMHFW UDWLR WR WKH /DWHQF\ SHU SDJH UDWLR ERWK UHIHUULQJ WR GLIIHUHQW DOWHUQDWLYHV DERXW KRZ ODWHQF\ FDQ EH PHDVXUHG DV GLVFXVVHG LQ 6HFWLRQ 7KH SRLQWV UHIHU WR WKH SHUIRUPDQFHLQGH[PHDVXUHGIRUHYHU\FOLHQWLQHDFKH[SHULPHQW

Trace: A, Algorithm: dg

Trace: A, Algorithm: ppm 1.1

Latency per page ratio


1.1

1.05

1

0.95

0.9

1.05

1

0.95

0.9 0.9

0.95

1 1.05 Latency per object ratio

1.1

0.9

0.95

D

1.1

E

Trace: B, Algorithm: dg

Trace: B, Algorithm: ppm 1.1


1.1


1 1.05 Latency per object ratio

1.05

1

0.95

0.9

1.05

1

0.95

0.9 0.9

0.95

1

1.05

1.1

Latency per object ratio

0.9

F

0.95

1

1.05

1.1


G

)LJ/DWHQF\SHUREMHFWUDWLRDVDIXQFWLRQRIODWHQF\SHUSDJHUDWLR

1RWLFHWKDWGHSHQGLQJRQWKHZD\WKHODWHQF\LVPHDVXUHGLWLVSRVVLEOHWRUHDFKQRWRQO\GLIIHUHQWEXW DOVRRSSRVLWHFRQFOXVLRQV6XSSRVHWKDWZHWDNHDSRLQWLQWKHXSSHUOHIWVLGHTXDUWHUDQGZHFRQVLGHUWKH /DWHQF\ SHU REMHFW LQ VXFK FDVH WKH SUHIHWFKLQJ V\VWHP RXWSHUIRUPV WKH QRQSUHIHWFKLQJ V\VWHP +RZHYHU LI ZH FRQVLGHUHG WKH /DWHQF\ SHU SDJH ZH ZRXOG FRQFOXGH WKH RSSRVLWH 7KH FRUUHODWLRQ FRHIILFLHQWEHWZHHQERWKODWHQF\UDWLRVFRUUHVSRQGLQJWRWKHSRLQWVSORWWHGLQ)LJXUHUDQJHVIURPWR LHWKHLQGH[HVSUHVHQWDFHUWDLQOLQHDUFRUUHODWLRQEXWLWLVIDUIURPEHLQJVWURQJVRWKDW/DWHQF\ SHUREMHFWDQG/DWHQF\SHUSDJHUDWLRVFDQQRWEHGLUHFWO\FRPSDUDEOH $VDFRQVHTXHQFHZHVXJJHVWWKDWVWXGLHVVKRXOGGLIIHUHQWLDWHWKHXVHRIERWKODWHQF\UDWLRVEHFDXVH WKH\DUHDLPHGDWH[SORULQJWKHSHUIRUPDQFH IURPGLIIHUHQWSRLQWVRI YLHZ7KH/DWHQF\ SHUSDJHUDWLR HYDOXDWHV WKH V\VWHP SHUIRUPDQFH IURP WKH XVHU¶V SRLQW RI YLHZ VLQFH LW PHDVXUHV WKH ODWHQF\ DV SHUFHLYHGE\WKHXVHU ZKLOHWKHODWHQF\SHUREMHFWUDWLRPHDVXUHVWKHSHUIRUPDQFHIURPWKHSRLQWRIYLHZ RIWKH+7737KHUHIRUHLWVKRXOGEHXVHGZKHQWKHPHDQLQJRIDSDJHLVQRWVRFOHDUIRULQVWDQFHZKHQ ZHDUHPHDVXULQJWKHODWHQF\LQDSUR[\VHUYHU


Trace: A, Algorithm: ppm 2

Linear model Latency per page ratio


2

1.5

1

0.5

Linear model

1.5

1

0.5

0

0 0

0.2

0.4 0.6 Precision

0.8

1

0

0.2

0.4 0.6 Precision

D

Linear model

1.5

1

0.5

Linear model

1.5

1

0.5

0

0 0

0.2

0.4

0.6

0.8

1

Byte-Precision

0

0.2

0.4

0.6

0.8

1

Byte-Precision

F

G

)LJ5HODWLRQEHWZHHQSUHFLVLRQDQGODWHQF\UHODWHGLQGH[HV

)LJXUHJUDSKLFVDDQGE SUHVHQWDQDOPRVWKRUL]RQWDOFXUYHVKRZLQJQRDSSDUHQWUHODWLRQEHWZHHQ SUHFLVLRQ DQG ODWHQF\ UHODWHG LQGH[HV 7KH FRUUHODWLRQ FRHIILFLHQWV VKRZHG LQ WDEOH TXDQWLI\ WKLV QHJOLJLEOHOLQHDUFRUUHODWLRQZKLFKFRQILUPVWKHYLVXDODSSUHFLDWLRQ*UDSKLFVF DQGG VKRZWKDWWKHUH LV D ZHDN UHODWLRQ EHWZHHQ WKH E\WH SUHFLVLRQ LQGH[ DQG WKH ODWHQF\ UHODWHG LQGH[HV 7KH UHPDLQGHU FRPELQDWLRQVRIWUDFHVDQGDOJRULWKPVDUHQRWVKRZQEHFDXVHWKH\DUHYHU\VLPLODU )LJXUHVKRZV WKHUHODWLRQEHWZHHQWKH UHFDOODQGWKHODWHQF\SHUREMHFWUDWLR$VRQHFDQREVHUYH GHSHQGLQJRQWKHZRUNORDGDQGWKHSUHGLFWLRQDOJRULWKPFRQVLGHUHGDOLQHDUUHODWLRQEHWZHHQWKHUHFDOO LQGH[DQGWKHODWHQF\SHUREMHFWUDWLRPLJKWH[LVW6RPHH[WUDHOHPHQWVQRWJDWKHUHGE\WKHUHFDOOOLNHWKH TXHXLQJ WLPH DQG WKH REMHFW VL]H KHWHURJHQHLW\ MXVWLI\ ZK\ WKH UHODWLRQ LV QRW VWURQJHU )LJXUH VKRZV 7DEOH/LQHDUFRUUHODWLRQFRHIILFLHQWEHWZHHQSUHGLFWLRQDQGODWHQF\LQGH[HV

3UHFLVLRQ 3F

/DWHQF\ 7UDFH?$OJ '* /S /RE

Trace: B, Algorithm: ppm 2



1

E

Trace: B, Algorithm: dg 2

0.8

5HFDOO

3FE 330

'*

5F 330

'*

5FE 330

'*

330

$

%

$

%



Linear model Latency per object ratio


2

1.5

1

0.5

0

Linear model

1.5

1

0.5

0 0

0.2

0.4

0.6

0.8

1

Recall

0

0.2

0.4

D

1


Linear model Latency per object ratio


0.8

E


0.6 Recall

1.5

1

0.5

0

Linear model

1.5

1

0.5

0 0

0.2

0.4

0.6

0.8

1

Recall

0

0.2

0.4

0.6

0.8

1

Recall

F

G

)LJ5HFDOODVDIXQFWLRQRIODWHQF\SHUREMHFWUDWLR Trace: A, Algorithm: dg


Linear model



2

1.5

1

0.5

0

Linear model

1.5

1

0.5

0 0

0.2

0.4

0.6

0.8

1

Recall

0

0.2

D

1

E 2

Linear model



0.8

Trace: B, Algorithm: ppm

1.5

1

0.5

0

Linear model

1.5

1

0.5

0 0

0.2

0.4

0.6

0.8

Recall

F )LJ5HFDOODVDIXQFWLRQRIODWHQF\SHUSDJHUDWLR

0.6 Recall


0.4

1

0

0.2

0.4

0.6 Recall

G

0.8

1

WKDWDFHUWDLQFRUUHODWLRQH[LVWVEHWZHHQWKHUHFDOODQGWKHODWHQF\SHUSDJHUDWLREXWLWLVIDUIURPEHLQJ VWURQJDVWKHOLQHDUFRUUHODWLRQFRHIILFLHQWVKRZVLQ7DEOH7KHODWHQF\SHUSDJHUDWLRLVQRWFRPSOHWHO\ H[SODLQHG E\ WKH UHFDOO LQGH[ VLQFH LW LQYROYHV PRUH HOHPHQWV WKDW DIIHFW LW OLNH WKH VLPXOWDQHRXV WUDQVIHUHQFH RI REMHFWV DV H[SODLQHG LQ 6HFWLRQ 7KHVH UHVXOWV FRUURERUDWH WKH GLIIHUHQFHV EHWZHHQ ERWKODWHQFLHVREVHUYHGLQ)LJXUH )LJXUHV DQG VKRZ WKH UHODWLRQ EHWZHHQ WKH E\WH UHFDOO DQG ERWK ODWHQF\ LQGH[HV 7KH IRUPHU SUHVHQWVDYHU\ZHDNFRUUHODWLRQRIWKHE\WHUHFDOOWRWKHODWHQF\SHUREMHFWUDWLRKHQFHWKHIDFWWKDWWKH E\WHUHFDOOGRHVQRWH[SODLQWKHODWHQF\SHUREMHFWUDWLR+RZHYHU)LJXUHDQGLWVDVVRFLDWHGFRUUHODWLRQ FRHIILFLHQWVKRZWKDWWKHE\WHUHFDOOLVVWURQJHUFRUUHODWHGWRWKHODWHQF\SHUSDJHUDWLRWKDQWKHUHFDOOVR WKLVLQGH[H[SODLQVEHWWHUWKHXVHUSHUFHLYHGODWHQF\0RUHGHWDLOVFDQEHIRXQGLQ>@ Trace: A, Algorithm: dg


Linear model



2

1.5

1

0.5

0

Linear model

1.5

1

0.5

0 0

0.2

0.4 0.6 Byte-Recall

0.8

1

0

0.2

D

1


Linear model



0.8

E


0.4 0.6 Byte-Recall

1.5

1

0.5

0

Linear model

1.5

1

0.5

0 0

0.2

0.4 0.6 Byte-Recall

0.8

1

F

0

0.2

0.4 0.6 Byte-Recall

0.8

1

G

)LJ%\WH5HFDOODVDIXQFWLRQRIODWHQF\SHUREMHFWUDWLR

$QDO\WLFDO5HODWLRQV ,Q RUGHU WR LGHQWLI\ DQDO\WLFDO UHODWLRQV DPRQJ LQGH[HV ZH XVH WKH LQGH[ GHILQLWLRQV SUHVHQWHG LQ 6HFWLRQ:HIRXQGWKDWWKHQHWZRUNOHYHOLQGH[HVFDQEHREWDLQHGIURPWKHSUHGLFWLRQUHODWHGLQGH[HV (TXDWLRQ VKRZV WKDW WKH REMHFW WUDIILF LQFUHDVH GHSHQGV RQ WKH UHFDOO 5F DQG WKH SUHFLVLRQ 3F DFKLHYHG E\ WKH SUHIHWFKLQJ V\VWHP 7KH HTXDWLRQ LV FRQVLVWHQW ZLWK WKH PHDQLQJ RI ERWK LQGH[HV D GHFUHDVHLQWKHSUHFLVLRQPHDQVDKLJKHUREMHFWWUDIILFLQFUHDVHZKHUHDVWKHKLJKHUWKHUHFDOOLVWKHORZHU WKHREMHFWWUDIILFLQFUHDVHLV



Linear model Latency per page ratio


2

1.5

1

0.5

0

Linear model

1.5

1

0.5

0 0

0.2

0.4

0.6

0.8

1

Byte-Recall

0

0.2

0.4

D

1

Linear model

1.5

1

0.5

0

Linear model

1.5

1

0.5

0 0

0.2

0.4 0.6 Byte-Recall

0.8

1

0

0.2

F

0.4 0.6 Byte-Recall

0.8

1

G

)LJ%\WH5HFDOODVDIXQFWLRQRIODWHQF\SHUSDJHUDWLR

2EMHFWV1RW8VHG + 8VHU5HTXHVWV Δ7URE = = 8VHU5HTXHVWV

3F ⋅ 3 5F = − 5F + 5F HT 3F ⋅ 3 3F 5F

3 − 3F ⋅ 3 +

(TXDWLRQVKRZVWKHUHODWLRQRIWKHWUDIILFLQFUHDVHLQGH[ZLWKWKRVHPHWULFVUHODWHGWRWKHSUHGLFWLRQ LHWKHE\WHUHFDOO5F% WKHE\WHSUHFLVLRQ3F% DQGWKH H[WUDFRQWUROGDWDSHUE\WH$VWKLV LQGH[LV UHODWHGWRWKHQXPEHURIE\WHVWUDQVIHUUHGWKHPDLQGLIIHUHQFHZLWKWKHSUHYLRXVRQHOLHVLQWKHIDFWWKDWLW GHSHQGVRQWKHE\WHUHFDOODQGE\WHSUHFLVLRQLQVWHDGRIRQWKHUHFDOODQGWKHSUHFLVLRQ1RWLFHWKDWDQHZ WHUP DSSHDUV ZLWK UHVSHFW WR WKH (TXDWLRQ WKDW UHIHUV WR WKH H[WUD FRQWURO GDWD SHU E\WH VR RQH FDQ XQGHUVWDQGWKLVWHUPDVWKHLQFUHDVHLQQHWZRUNWUDIILFGXHWRWKHWUDQVIHUHQFHRIGDWDXVHGWRFRQWUROWKH SUHIHWFKLQJHQJLQHHJSUHIHWFKKLQWV SHUHDFKE\WHUHTXHVWHGE\WKHXVHU

Δ7U% =

=




0.8

E


0.6

Byte-Recall

2EMHFWV1RW8VHG % + 1HWZRUN2YHUKHDG % + 8VHU5HTXHVWV% = 8VHU5HTXHVWV% 3% − 3F% ⋅ 3% + 12% + 3F% ⋅ 3% 5F%

3F% ⋅ 3% 5F%

5F = − 5F% + % + ([WUD&RQWURO'DWD% 3F%

HT

&RQFOXVLRQV $ODUJHYDULHW\RINH\PHWULFVKDYHDSSHDUHGLQWKHRSHQOLWHUDWXUHLQRUGHUWRHYDOXDWHWKHSHUIRUPDQFH RI ZHE SUHIHWFKLQJ V\VWHPV 7KLV SDSHU KDV DQDO\]HG WKLV ZLGH KHWHURJHQHLW\ WU\LQJ WR L FODULI\ ERWK LQGH[GHILQLWLRQVDQGKRZWKH\DUHLQWHUUHODWHGDQGLL KHOSZKHQGHFLGLQJZKLFKPHWULFRULQGH[VKRXOG EHVHOHFWHGWRHYDOXDWHWKHV\VWHPSHUIRUPDQFH :H KDYH SURSRVHG D WD[RQRP\ FODVVLI\LQJ WKH LQGH[HV UHODWHG WR SUHIHWFK LQ WKUHH PDLQ FDWHJRULHV DFFRUGLQJWRWKHSDUWRIWKHV\VWHPWKH\SXUVXHWRHYDOXDWHSUHGLFWLRQUHVRXUFHXVDJHDQGODWHQF\7KH JRDORIWKLVWD[RQRP\LVQRWRQO\WRFRQWULEXWHWRWKHXQGHUVWDQGLQJRIWKHGHILQLWLRQRIWKHLQGH[HV,WLV DOVRDLPHGDWDQDO\]LQJDQDORJLHVDQGGLIIHUHQFHVEHWZHHQWKHPLQRUGHUWRLGHQWLI\ZKLFKDUHWKHPRVW XVHIXOPHWULFVZKHQFDUU\LQJRXWSHUIRUPDQFHVWXGLHV :HKDYHSURYLGHGHDFKPHWULFRULQGH[ZLWKDGHILQLWLRQWKHRQHZHFRQVLGHUHGPRUHSUHFLVH IURP WKH ODUJH YDULHW\ DSSHDUHG LQ WKH OLWHUDWXUH 7KHQ ORRNLQJ IRU WKHVH GHILQLWLRQV ZH REVHUYHG FHUWDLQ DQDORJLHV:HXVHGERWKHPSLULFDODQGDQDO\WLFDODSSURDFKHVWRVKRZ KRZ LQGH[HVDUHLQWHUUHODWHG:H IRXQGDQDO\WLFDOUHODWLRQVEHWZHHQQHWZRUNOHYHOLQGH[HVDQGSUHGLFWLRQUHODWHGLQGH[HV7RFKHFNRWKHU SRVVLEOHUHODWLRQVDPRQJLQGH[HV ZHUDQH[SHULPHQWVDQGFDOFXODWHGWKHVWDWLVWLFDOFRUUHODWLRQDPRQJD UHSUHVHQWDWLYHVHWRIWKHP)URPWKHVHUHVXOWVZHH[WUDFWWKHPDLQFRQFOXVLRQVWKDWIROORZ − 3HUIRUPDQFH VWXGLHV VKRXOG LQFOXGH DW OHDVW ODWHQF\ UHODWHG PHWULFV 'HSHQGLQJ RQ WKH JRDO RI WKH SHUIRUPHGVWXG\ODWHQF\SHUSDJHRUSHUREMHFWLVSUHIHUUHG)RULQVWDQFHLIWKHJRDOLVWRDQDO\]HWKH XVHU¶VSRLQWRIYLHZWKHODWHQF\SHUSDJHPXVWEHLQFOXGHG+RZHYHUZKHQHYDOXDWLQJIURPWKHSRLQW RI YLHZ RI D SUR[\ VHUYHU WKH ODWHQF\ SHU SDJH PDNHV QR VHQVH GXH WR WKH ODFN RI SDJH FRQFHSW &RQVHTXHQWO\WKHODWHQF\SHUREMHFWFDQEHWKHPRVWXVHIXOPHWULF − /DWHQFLHV FDQ QRW EH XVHG DV RQO\ PHWULFV WR FKHFN SHUIRUPDQFH 6WXGLHV PXVW DQDO\]H KRZ WKH ODWHQFLHV UHGXFWLRQ KDV EHHQ DFKLHYHG IRU D JLYHQ SURSRVDO ,Q WKLV VHQVH UHVRXUFH XVDJH LQGH[HV VKRXOGEHWDNHQLQWRDFFRXQW7UDIILFLQFUHDVHLVWKHRQHWKDWSURYLGHVPRUHLQIRUPDWLRQWKHUHIRUHZH VXJJHVWWKDWSHUIRUPDQFHVWXGLHVVKRXOGLQFOXGHDWOHDVWWKLVLQGH[ − 2XU GLVFXVVLRQ KDV VKRZQ WKDW LV QRW UHFRPPHQGDEOH WR SHUIRUP VWXGLHV DERXW WKH EHKDYLRU RI SUHIHWFKLQJ WHFKQLTXHV MXVW IRFXVLQJ RQ WKH DOJRULWKP SRLQW RI YLHZ 1HYHUWKHOHVV LI VRPH VWXGLHV IRFXVRQWKLVSDUWWKH\VKRXOGLQFOXGHUHFDOODQGE\WHUHFDOODVSHUIRUPDQFHPHWULFVEHFDXVHWKH\DUH WKHPRVWFRUUHODWHGWRWKHODWHQF\SHUREMHFWDQGODWHQF\SHUSDJHUHVSHFWLYHO\

$FNQRZOHGJHPHQWV 7KLV ZRUN KDV EHHQ SDUWLDOO\ VXSSRUWHG E\ 6SDQLVK 0LQLVWU\ RI (GXFDWLRQ DQG 6FLHQFH DQG WKH (XURSHDQ,QYHVWPHQW)XQGIRU5HJLRQDO'HYHORSPHQW)('(5 XQGHUJUDQW76,&

5HIHUHQFHV >@

$OEUHFKW'=XNHUPDQ,DQG1LFKROVRQ$3UH6HQGLQJGRFXPHQWVRQWKH::: $FRPSDUDWLYHVWXG\

3URFHHGLQJVRIWKHWK,QWHUQDWLRQDO-RLQW&RQIHUHQFHRQ$UWLILFLDO,QWHOOLJHQFH6WRFNKROP6ZHGHQ >@

%HVWDYURV$8VLQJVSHFXODWLRQWRUHGXFHVHUYHUORDGDQGVHUYLFHWLPHRQWKH:::3URFHHGLQJVRIWKHWK

$&0,QWHUQDWLRQDO&RQIHUHQFHRQ,QIRUPDWLRQDQG.QRZOHGJH0DQDJHPHQW%DOWLPRUH86$ >@

%RQLQR'&RUQR)DQG6TXLOOHUR*$UHDOWLPHHYROXWLRQDU\DOJRULWKPIRUZHESUHGLFWLRQ3URFHHGLQJVRI

WKH,QWHUQDWLRQDO&RQIHUHQFHRQ:HE,QWHOOLJHQFH+DOLID[&DQDGD >@

%RQLQR ' &RUQR ) DQG 6TXLOOHUR * '\QDPLF 3UHGLFWLRQ RI :HE 5HTXHVWV 3URFHHGLQJV RI WKH ,(((

&RQJUHVVRQ(YROXWLRQDU\&RPSXWDWLRQ&DQEHUUD$XVWUDOLD >@

%RXUDV&.RQLGDULV$DQG.RVWRXODV'3UHGLFWLYHSUHIHWFKLQJRQWKHZHEDQGLWVSRWHQWLDOLPSDFWLQWKH

ZLGH DUHD :RUOG :LGH :HE ,QWHUQHW DQG :HE ,QIRUPDWLRQ 6\VWHPV .OXZHU $FDGHPLF 3XEOLVKHUV 7KH 1HWKHUODQGV >@

&RKHQ ( DQG .DSODQ +3UHIHWFKLQJ WKH PHDQV IRU GRFXPHQW WUDQVIHU $ QHZ DSSURDFK IRU UHGXFLQJ ZHE

ODWHQF\3URFHHGLQJVRIWKH,(((,1)2&207HO$YLY,VUDHO >@

&RKHQ ( .ULVKQDPXUWK\ % DQG 5H[IRUG - (IILFLHQW DOJRULWKPV IRU SUHGLFWLQJ UHTXHVWV WR ZHE VHUYHUV

3URFHHGLQJVRIWKH&RQIHUHQFHRQ&RPSXWHUDQG&RPPXQLFDWLRQV1HZ@

&XQKD&5DQG-DFFRXG&)%'HWHUPLQLQJ:::XVHU¶VQH[WDFFHVVDQGLWVDSSOLFDWLRQVWRSUHIHWFKLQJ

3URFHHGLQJVRIWKHQG,QWHUQDWLRQDO6\PSRVLXPRQ&RPSXWHUVDQG&RPPXQLFDWLRQ$OH[DQGULD(J\SW >@

&KHUNDVRYD / DQG &LDUGR * &KDUDFWHUL]LQJ WHPSRUDO ORFDOLW\ DQG LWV LPSDFW RQ ZHE VHUYHU SHUIRUPDQFH

3URFHHGLQJV RI WKH WK ,QWHUQDWLRQDO &RQIHUHQFH RQ &RPSXWHU &RPPXQLFDWLRQ DQG 1HWZRUNV /DV 9HJDV 86$ >@ 'DYLVRQ%'/HDUQLQJ:HE5HTXHVW3DWWHUQV:HE'\QDPLFV$GDSWLQJWRFKDQJHLQFRQWHQWVL]HWRSRORJ\ DQGXVH6SULQJHU >@ 'DYLVRQ % ' 3UHGLFWLQJ ZHE DFWLRQV IURP +70/ FRQWHQW 3URFHHGLQJV RI WKH WK $&0 &RQIHUHQFH RQ +\SHUWH[WDQG+\SHUPHGLD&ROOHJH3DUN86$ >@ 'RPqQHFK-3RQW$6DKXTXLOOR-DQG*LO-$$QH[SHULPHQWDOIUDPHZRUNIRUWHVWLQJZHESUHIHWFKLQJ WHFKQLTXHV3URFHHGLQJVRIWKHWK(XURPLFUR&RQIHUHQFH5HQQHV)UDQFH >@ 'RPqQHFK-6DKXTXLOOR-*LO-$DQG3RQW$$ERXWWKHKHWHURJHQHLW\RIZHESUHIHWFKLQJSHUIRUPDQFH NH\ PHWULFV 3URFHHGLQJV RI WKH ,),3 ,QWHUQDWLRQDO &RQIHUHQFH RQ ,QWHOOLJHQFH LQ &RPPXQLFDWLRQ 6\VWHPV %DQJNRN7KDLODQG >@ 'RPqQHFK - 6DKXTXLOOR - 3RQW $ DQG *LO - $ 'HVLJQ NH\V WR DGDSW ZHE SUHIHWFKLQJ DOJRULWKPV WR HQYLURQPHQWFRQGLWLRQV3URFHHGLQJVRIWKHVW,QWHUQDWLRQDO&RQIHUHQFHRQ&RPPXQLFDWLRQ6\VWHP6RIWZDUHDQG 0LGGOHZDUH1HZ'HOKL,QGLD >@ 'RQJVKDQ ; DQG -XQ\L 6 $ QHZ PDUNRY PRGHO IRU ZHE DFFHVV SUHGLFWLRQ &RPSXWLQJ ,Q 6FLHQFH DQG (QJLQHHULQJ9RO,VV >@ 'XFKDPS '3UHIHWFKLQJ K\SHUOLQNV3URFHHGLQJVRI WKHQG 86(1,; 6\PSRVLXP RQ ,QWHUQHW 7HFKQRORJLHV DQG6\VWHPV%RXOGHU86$ >@ )DQ/&DR3DQG-DFREVRQ4:HESUHIHWFKLQJEHWZHHQORZEDQGZLGWKFOLHQWVDQGSUR[LHVSRWHQWLDODQG SHUIRUPDQFH 3URFHHGLQJV RI WKH $&0 6,*0(75,&6 &RQIHUHQFH RQ 0HDVXUHPHQW DQG 0RGHOLQJ RI &RPSXWHU 6\VWHPV$WODQWD86$ >@ )LVKHU ' DQG 6DNVHQD * /LQN SUHIHWFKLQJ LQ 0R]LOOD $ VHUYHUGULYHQ DSSURDFK 3URFHHGLQJV RI WKH WK ,QWHUQDWLRQDO:RUNVKRSRQ:HE&RQWHQW&DFKLQJDQG'LVWULEXWLRQ1HZ@ ,EUDKLP7DQG;X&=1HXUDOQHWVEDVHGSUHGLFWLYHSUHIHWFKLQJWRWROHUDWH:::ODWHQF\3URFHHGLQJVRI WKHWK,(((,QWHUQDWLRQDO&RQIHUHQFHRQ'LVWULEXWHG&RPSXWLQJ6\VWHPV7DLSHL7DLZDQ

>@ .KDQ-,DQG7DR4([SORLWLQJZHEVSDFHRUJDQL]DWLRQIRUDFFHOHUDWLQJZHESUHIHWFKLQJ3URFHHGLQJVRIWKH ,QWHUQDWLRQDO&RQIHUHQFHRQ:HE,QWHOOLJHQFH+DOLID[&DQDGD >@ .RNNX 5 @ 3LWNRZ - DQG 3LUROOL 3 0LQLQJ ORQJHVW UHSHDWLQJ VXEVHTXHQFHV WR SUHGLFW :RUOG :LGH :HE VXUILQJ 3URFHHGLQJVRIWKHQG86(1,;6\PSRVLXPRQ,QWHUQHW7HFKQRORJLHVDQG6\VWHPV%RXOGHU86$ >@ 5DELQRYLFK0DQG6SDWVFKHFN2:HEFDFKLQJDQGUHSOLFDWLRQ$GGLVRQ:HVOH\%RVWRQ86$ >@ 6FKHFKWHU6.ULVKQDQ0DQG6PLWK0'8VLQJSDWKSURILOHVWRSUHGLFW+773UHTXHVWV3URFHHGLQJVRI WKHWK,QWHUQDWLRQDO:RUOG:LGH:HE&RQIHUHQFH%ULVEDQH$XVWUDOLD >@ 7DR 4 ,PSDFW RI :HEVSDFH 2UJDQL]DWLRQ DQG 8VHU ,QWHUDFWLRQ %HKDYLRU RQ D 3UHIHWFKLQJ 3UR[\ 3K ' 7KHVLV.HQW6WDWH8QLYHUVLW\86$ >@ =XNHUPDQ,$OEUHFKW'DQG1LFKROVRQ$3UHGLFWLQJXVHUV UHTXHVWVRQWKH:::3URFHHGLQJVRIWKHWK ,QWHUQDWLRQDO&RQIHUHQFHRQ8VHU0RGHOLQJ%DQII&DQDGD