systems or applications considered by the authors e.g. (Daniel et al., 2008; Ganapathi & ... relate to long term observations of many workstations and servers.
&GRGPFCDKNKV['XCNWCVKQP$CUGFQP5[UVGO/QPKVQTKPI
:
&GRGPFCDKNKV['XCNWCVKQP$CUGF QP5[UVGO/QPKVQTKPI
-DQXV]6RVQRZVNLDQG0DUFLQ.UyO
,QVWLWXWHRI&RPSXWHU6FLHQFH:DUVDZ8QLYHUVLW\RI7HFKQRORJ\ 3RODQG
+PVTQFWEVKQP
5HFHQWO\ GHSHQGDELOLW\ PDLQWDLQDELOLW\ DQG SHUIRUPDQFH DUH EHFRPLQJ FKDOOHQJLQJ LVVXHV IRUV\VWHPGHVLJQHUVDQGXVHUV7KLVUHVXOWVIURPWKHLQFUHDVLQJFRPSOH[LW\RIKDUGZDUHDQG VRIWZDUH ,Q FRQVHTXHQFH WKHVH LVVXHV WULJJHUHG YDULRXV PHDVXUHPHQWEDVHG VWXGLHV LQ WKH OLWHUDWXUHLQSDUWLFXODUWKH\UHODWHWRWKHGHWHFWLRQRUSUHGLFWLRQRIFULWLFDOVLWXDWLRQV0RVW SXEOLVKHG UHVXOWV DUH IRFXVHG RQ UHVWULFWHG DQG IUDJPHQWHG SUREOHPV HQFRXQWHUHG LQ WKH V\VWHPV RU DSSOLFDWLRQV FRQVLGHUHG E\ WKH DXWKRUV HJ 'DQLHO HW DO *DQDSDWKL 3DWWHUVRQ +RIIPDQQHWDO /LHWDO 0DNDQMXHWDO ,QFRQWHPSRUDU\ FRPSXWHU V\VWHPV YDULRXV PRQLWRULQJ PHFKDQLVPV DUH SURYLGHG XVXDOO\ WKH\ UHODWH WR HYHQW DQG SHUIRUPDQFH PRQLWRULQJ -RKQ (FNKRXW 6LPDFKH .DDQLFKH 6WHDUOH\ =KDQJ PLQ@
)LJ7KHQXPEHURIHYHQWVZLWKLQWKHVSHFLILHGWLPHZLQGRZ /RRNLQJIRUWKHVRXUFHVRIUHVWDUWVZHFDQDQDO\VHWKHGLVWULEXWLRQRIHYHQWVUHJLVWHUHGLQWKH VSHFLILHG ZLQGRZ EHIRUH WKH HYHQW VHTXHQFH UHODWHG WR UHERRW 7KLV LV LOOXVWUDWHG LQ ILJ 7KH [D[LV VSHFLILHV GLIIHUHQW HYHQWV ,Q SDUWLFXODU [ UHODWHV WR HYHQW 6UY WKHGLVNLVDOPRVWIXOO\RXPD\QHHGWRGHOHWHVRPHILOHV [ HYHQWZWLPH WKH :LQGRZV7LPH6HUYLFHZDVQRWDEOHWRILQGD'RPDLQ&RQWUROOHUDWLPHDQGGDWHXSGDWHZDV QRW SRVVLEOH [ ² HYHQW DXWRPDWLF SURJUDP XSGDWHV [ ² HYHQW DSSOLFDWLRQ HUURU HWF 6XFKJUDSKIDFLOLWDWHVWRLGHQWLI\WKHPRVWIUHTXHQWVRXUFHVRIUHVWDUWV&RPSOHWHGLVWULEXWLRQ
&GRGPFCDKNKV['XCNWCVKQP$CUGFQP5[UVGO/QPKVQTKPI
RI DOO UHJLVWHUHG HYHQWV LQ D GHFUHDVLQJ RUGHU RI RFFXUUHQFH LV DOVR XVHIXO WR LGHQWLI\ RWKHU SUREOHPV7\SLFDOO\ RIUHJLVWHUHGHYHQWVUHODWHGWRRQO\GLIIHUHQWHYHQWW\SHV IURPWKH WRWDOOLVWRIRYHUSRVVLEOHHYHQWV 7R ILQG VRXUFHV RI VRPH HYHQW $ LW LV XVHIXO WR FKHFN HYHQWV EHIRUH WKLV HYHQW ZLWKLQ D VSHFLILHGWLPHZLQGRZ')RUWKLVSXUSRVH/RJ0RQSURYLGHVWKHFDSDELOLW\RIILQGLQJVXFK VWDWLVWLFV IRU D VSHFLILHG ZLQGRZ ' )LJ VKRZV WKH QXPEHU RI UHJLVWHUHG HYHQWV IRU VHUYHUVLQIXQFWLRQRIWKHWLPHZLQGRZZLGWK'EHIRUHUHVWDUWV:HFDQREVHUYHVRPHNLQG RI VDWXUDWLRQ IRU ' PLQXWHV GHSHQGLQJ XSRQ WKH V\VWHP %DVLQJ RQ WKH UHJLVWHUHG HYHQWVZHFDQLGHQWLI\UHVWDUWV)RUWKHFRQVLGHUHGVHUYHUV ILJ ZHKDYHLGHQWLILHG UHVWDUWV RQDYHUDJHHYHQWVSHUUHVWDUWLQWKHZLQGRZ 7KHGHYHORSHGV\VWHP/RJ0RQSURYLGHVYDULRXVGDWDH[SORUDWLRQFDSDELOLWLHV,QSDUWLFXODULW FDQLGHQWLI\UHDVRQVRIUHVWDUWVDQGGLUW\UHVWDUWV,QWKHFDVHRIUHVWDUWVZHKDYHGHILQHGVRPH UHJXODU H[SUHVVLRQV GHVFULELQJ HYHQWV PRVW SUREDEO\ UHODWHG WR VSHFLILHG VLWXDWLRQV HJ SURJUDP XSGDWH UHVWDUWV 7DE VKRZV UHVWDUW VWDWLVWLFV IRU ODERUDWRULHV // HDFK FRPSULVLQJ ZRUNVWDWLRQV DQG VHUYHUV 66 ,W JLYHV WKH SHUFHQW RI UHVWDUWV FDXVHG E\ SURJUDPXSGDWHVDSSOLFDWLRQHUURUVDQGKDUGZDUHHUURUV,QVRPHFDVHVWKHUHVWDUWVRXUFHLV DPELJXRXV UHODWHG WR PRUH WKDQ RQH VRXUFH PRVWO\ D SURJUDP XSGDWH DQG VRPH RWKHU VRXUFH 4XLWH VLJQLILFDQW SHUFHQWDJH RI GHWHFWHG UHVWDUWV XQNQRZQ FDXVH GLG QRW FRPSULVH DGGLWLRQDO HYHQWV IDFLOLWDWLQJ WKHLU LGHQWLILFDWLRQ 7KH\ UHODWH WR SRZHU GRZQV DQG UHVWDUWV LQLWLDWHGE\WKHXVHULQUHVSRQVHWRVRPHPHVVDJHVDSSHDULQJRQWKHVFUHHQVRPHRIWKHPFDQ EHLGHQWLILHGIURPWKHDSSOLFDWLRQORJ7KHWDEOHFRPSULVHVWKHUHVWDUWIUHTXHQF\ 5) H[SUHVVHG LQWKHQXPEHURIUHVWDUWVSHUGD\ SHUVLQJOHFRPSXWHU 5HODWLYHO\ORZYDOXHVRI5)IRUWZR VHUYHUV 6DQG6 UHVXOWIURPWKHVWDEOHSURILOHRIWKHLUXVDJH / / / / 6 6 6 5) XSGDWHV DSSOLF KDUGZDUH DPELJ XQNQRZQ 7DEOH5HVWDUWVWDWLVWLFVIRUZRUNVWDWLRQVDQGVHUYHUV 6SHFLDODWWHQWLRQLVQHHGHGWRGLUW\UHVWDUWV'LUW\UHVWDUWVPRVWO\UHODWHWRVXFKHYHQWVDV VDYH GXPS 3UHVVLQJ 5(6(7 EXWWRQ DOVR FDXVHV GLUW\ UHVWDUW $W WKH V\VWHP OHYHO SRZHUGRZQLVWUHDWHGLQWKHVDPHZD\DVIDVWVZLWFKLQJRIIWKHFRPSXWHU,QORJVZLWKSRZHU GRZQFORVLQJHYHQWZDVPLVVLQJ$QDO\VLQJHYHQWVLQWKHZLQGRZEHIRUHWKHGLUW\UHVWDUW ZHREVHUYHGVPDOOQXPEHURIHYHQWV7KLVUHODWHVPRVWO\WRWKHVLWXDWLRQZLWKQRSRVVLELOLW\RI UHFRUGLQJ WKH HYHQW GXH WR WKH UHVWDUW SUREOHP ,Q D SHULRG RI GD\V ZH KDYH LGHQWLILHG W\SLFDOO\GLUW\UHVWDUWVSHUFRPSXWHU+RZHYHUIRUDIHZFRPSXWHUVWKLVZDVLQWKHUDQJH RI ROG FRPSXWHUV ,Q WKH FDVH RI VHUYHUV SUDFWLFDOO\ WKH GLUW\ UHVWDUWV ZHUH FDXVHG E\ KDUGZDUHHUURUV,QWKHFDVHRIZRUNVWDWLRQVGLUW\UHVWDUWVZHUHFDXVHGPRVWO\E\DSSOLFDWLRQ HUURUVZKLFKFDXVHGV\VWHPKDQJXSVRURSHUDWLRQDOLQVWDELOLW\ GXHWRGHYHORSLQJDQGWHVWLQJ YDULRXVSRJURPPRGXOHV 0RUHRYHUPDQ\VWXGHQWSURMHFWVFRPSULVHGFULWLFDOHUURUVOHDGLQJ WRUHVWDUWV
%QORWVCVKQPCN+PVGNNKIGPEGCPF/QFGTP*GWTKUVKEU
1XPEHURIFDVHV
7RWDOHYHQWV
7LPHLQWHUYDOV KRXUV
)LJ'LVWULEXWLRQRIQRQDFWLYLW\SHULRGVLQDZRUNVWDWLRQ )LJVKRZVGLVWULEXWLRQRIWLPHSHULRGVEHWZHHQHYHQW$DQG%ZKLFKFRUUHVSRQGWRFORVLQJ DQGVWDUWLQJWKHV\VWHP7KH[D[LVRIWKHSORWKDVWKHJUDQXODULW\RIVLQJOHKRXUV7KHILUVWEDU FDVHV UHODWHV WR VKRUW WLPH LQWHUYDOV OHVV WKDQ KRXU DQG LW FRUUHVSRQGV WR VKRUW RSHUDWLRQEUHDNVUHODWHGWRUHVWDUWV7KHQH[WJURXSRIKLJKHUEDUVUHODWHVWRWKHSHULRGVRI KRXUV FRUUHVSRQGLQJ WR VZLWFKLQJ RII WKH FRPSXWHU IRU WKH HYHQLQJ DQG QLJKW SHULRGV 6XEVHTXHQW JURXSV RIEDUVUHODWHWRORQJHU QRQDFWLYLW\ SHULRGVHJZHHNHQGVKROLGD\V HWF 3RZHU VXSSO\ SUREOHPV FDQ EH GLUHFWO\ LGHQWLILHG E\ FKHFNLQJ WKH WLPH EHWZHHQ HYHQWV 836 QRWLILFDWLRQRISRZHUGRZQDQGVXSSO\GHOLYHUHGIURPWKHEDWWHULHV DQG 836 SRZHU UHFRYHU\ RU 836 SRZHU VZLWFKHG RII LQ WKH V\VWHP JHQHUDWHG E\ 836 SRZHU VXSSO\ )RU DQ LOOXVWUDWLRQ ZH JLYH VRPH SRZHU VWDWLVWLFV UHODWHG WR VHUYHUV 6$ DQG 6% ,Q SDUWLFXODUZHVSHFLI\WKHQXPEHURISRZHUGRZQHYHQWVSHUPRQWKIRU VXEVHTXHQWPRQWKV 2FWREHU-XQH 6$ WRWDO HYHQWV DQG6% WRWDOHYHQWV )RUDOOSRZHUGRZQHYHQWVWKHVHUYHUVZHUHVXSSOLHGIURP836EDWWHULHVGXHWRVKRUWSRZHU RXWDJHV SRZHU RXWDJHV ZHUH WROHUDWHG E\ 836 3RZHU RXWDJHV ORQJHU WKDQ PLQXWHV DQG VHFUHVXOWLQ6$DQG6%VHUYHUVZLWFKLQJRIIUHVSHFWLYHO\7KHGLVWULEXWLRQRIWKHGXUDWLRQ RISRZHURXWDJHVZDVDVIROORZV IRUVHUYHU6$ HYHQWV PLQXWHPLQXWHVd HYHQWV PLQXWHVDQGHYHQWZLWKPLQXWHVGXUDWLRQ IRUVHUYHU6% HYHQWV VVd HYHQWV V V d HYHQWV V V d HYHQWV V DQG V d HYHQWV d V &RPSXWHUV ZLWKRXW836FUDVKHGDQGQHHGHGUHVWDUWLQJ6RPHFUDVKHVDSSHDUHGVLPXOWDQHRXVO\LQVHYHUDO FRPSXWHUV FRPPRQSRZHUIDLOXUH ,QFDVHVWKHSRZHUGRZQVVLJQDOOHGE\836VRI6$DQG 6% VHUYHUV ZHUH FRUUHODWHG FDXVHG E\ WKH VDPH SRZHU QHWZRUN RXWDJH KRZHYHU WKH DWWULEXWHG WLPHVWDPSV GLIIHUHG E\ DERXW PLQXWHV GXH WR WKH ODFN RI WKH FORFN V\QFKURQLVDWLRQLQERWKVHUYHUV VXFKDQRPDOLHVQHHGLGHQWLILFDWLRQ $Q LPSRUWDQW LVVXH LV WR HYDOXDWH WKH EHKDYLRXU RI YDULRXV XVHG SURJUDPV ZLWKLQ ORQJHU SHULRGV DQG FRUUHODWH LW ZLWK V\VWHP XSJUDGHV UHFRQILJXUDWLRQ ORDG QXPEHU RI XVHUV SURFHVVRU VWUHVV )RU DQ LOOXVWUDWLRQ WDEOH VKRZV WKH IUHTXHQF\ GLVWULEXWLRQ RI UHJLVWHUHG SURJUDP HUURUV SHU GD\ IRU WZR ZRUNVWDWLRQ :6 DQG :6 ZLWKLQ WKH RQH\HDU SHULRG :6LVOHVVUHOLDEOHGXHWRKLJKHUQXPEHURIXVHGSURJUDPV
&GRGPFCDKNKV['XCNWCVKQP$CUGFQP5[UVGO/QPKVQTKPI
&RPSXWHU 1XPEHURIHUURUVSHUGD\ :6 R :6 7DEOH'LVWULEXWLRQRISURJUDPHUURUVIRUZRUNVWDWLRQ:6DQG:6 (YHQWVUHODWHGGLUHFWO\WRKDUGZDUHIDXOWVDSSHDUEHIRUHUHVWDUWV7KHPRVWIUHTXHQWUHODWHWR PHPRU\PHGLDQHWZRUNFDUGVSULQWHUVDQGRWKHU,2GHYLFHV)RUH[DPSOHHYHQWZLWK GHVFULSWLRQ WKDW WKH V\VWHP FRXOG QRW ZULWH RU UHDG GDWD IURP D VSHFLILHG ILOH GHYLFH HWF 2WKHUH[DPSOHVDUH IDXOW\EORFNRI&'520WLPHRXWVLWXDWLRQ,3DGGUHVVFRQIOLFWIDLOXUH WRORDGVSHFLILHGGULYHUVDSSOLFDWLRQHUURUVHWF,WLVZRUWKQRWLQJWKDWPDQ\HYHQWVGRQRW FRPSULVHGHVFULSWLRQVRQWKHRWKHUKDQGVRPHGHVFULSWLRQVDUHDPELJXRXV0DQ\IDXOWVFDQ EHLGHQWLILHGIURPVHTXHQFHVRIHYHQWV)RUWKLVSXUSRVHVRPHNQRZOHGJHGDWDEDVHFDQEH V\VWHPDWLFDOO\ GHYHORSHG WDNLQJ LQWR DFFRXQW WKH JDLQHG H[SHULHQFH IURP WKH V\VWHP H[SORLWDWLRQDQGPDLQWHQDQFH
'ZRNQTKPIFCVCKPGXGPVNQIU $QDO\VLQJORJVLVWKHEDVLVIRUDXWRPDWLFV\VWHPPDQDJHPHQWDQGKHOSIXOLQDVVXULQJKLJK GHSHQGDELOLW\7KHUHJLVWHUHGUHSRUWVPD\EHUHODWHGWRGLIIHUHQWIRUPDWVWKHWH[WPHVVDJHV DUH XVXDOO\ UHODWLYHO\ VKRUW FRQWDLQ D IUHH IRUPDW GHVFULSWLRQ RI HYHQWV XVLQJ D ODUJH VL]H YRFDEXODU\ DQG TXLWH RIWHQ DUH DPELJXRXV +HQFH GDWD PLQLQJ LV QRW WULYLDO DQG QHHGV PDQ\SUHOLPLQDU\VWXGLHVWRLGHQWLI\VSHFLILFLW\DQGDEQRUPDOEHKDYLRXURIWKHPRQLWRUHG V\VWHPV,QWKLVSURFHVVVRPHFDWHJRULVDWLRQRIWH[WPHVVDJHVLQWRDVHWRIFRPPRQFODVVHV RYHU YDULRXV V\VWHP FRPSRQHQWV LV UHTXLUHG 0RUHRYHU DQ LPSRUWDQW LVVXH LV WR YLVXDOLVH YDULRXVVWDWLVWLFVWHPSRUDOGHSHQGHQFLHVHWF 6LPSOHGDWDPLQLQJFDQEHWDUJHWHGDWGLVFRYHULQJIUHTXHQWDQGVRPHVSHFLILFZHOOGHILQHG SDWWHUQV /LP HW DO 0DNDQMX HW DO 3HQJ HW DO 5D]DYL .RQWRJLDQQLV 9DDUDQGL ,QPRUHDGYDQFHGDQDO\VLVRIORJUHSRUWVZHVKRXOGWDNHLQWRDFFRXQW QRWRQO\WKHLQGLYLGXDOPHVVDJHVEXWDOVRWKHLUWHPSRUDOGHSHQGHQFLHVZKLFKFDQSURYLGH VXSSOHPHQWDU\ FRQWH[W LQIRUPDWLRQ )RU H[DPSOH D PDVVDJH RQ VWDUWLQJ D SURJUDP XSGDWH PD\EHIROORZHGE\VRPHHUURUVGXHWRLQFRQVLVWHQF\LQWKHV\VWHPFRQILJXUDWLRQ+DYLQJ WUDQVIRUPHG PHVVDJHV LQ VRPH FRQFLVH FDWHJRULVHG IRUP VLPSOLILHV IXUWKHU GDWD SURFHVVLQJ DQG ILQGLQJ FKDUDFWHULVWLF SDWWHUQV 8QIRUWXQDWHO\ GLIIHUHQW V\VWHPV XVH GLIIHUHQW ORJ IRUPDWV HWF +HQFH GDWD FROOHFWLRQ DQG DQDO\VLV KDV WR EH WXQHG WR WKHVH V\VWHPV 7KLV LV VXIILFLHQW IRU LQGLYLGXDO V\VWHP PRQLWRULQJ ,Q SUDFWLFH ZH DUH DOVR LQWHUHVWHG LQ JHQHUDO SURSHUWLHV RI PDQ\ V\VWHPV VR VRPH VSHFLILFDWLRQ RI VLPLODULWLHV KDV WR EH GHILQHG WR LGHQWLI\FRPPRQFKDUDFWHULVWLFVHWF 7KHORJSUHSURFHVVLQJPD\LQYROYHYLVXDOLVDWLRQRIHYHQWW\SHVRUFDWHJRULHVLQUHODWLRQWR WKHLUDSSHDUDQFH WLPHVWDPSV )URPVXFKSORWLWLVHDV\WRLGHQWLI\VRPHJHQHUDOSURSHUWLHV HJ WKH IDFW WKDW HYHQW $ XVXDOO\ KDSSHQV DIWHU HYHQW % WKH WLPH GLVWDQFH EHWZHHQ VXFK HYHQWV LWFDQEHGHWHUPLQLVWLFRUUDQGRP $QHYHQWPD\DSSHDUZLWKVRPHSHULRGLFLW\ HJ DQWLYLUXV XSGDWHV V\VWHP KHDUWEHDW RU UDQGRPO\ 6RPH HYHQWV PD\ IRUP D ORRS LQ D FLUFXODUSDWWHUQRUDQHYHQWFKDLQ HJUHODWHGWRDSUREOHPSURJUHVVLQSUHGLFWDEOHZD\ $Q HYHQWPD\DSSHDUVLPXOWDQHRXVO\ZLWKRWKHUHYHQWV9DULRXVWHPSRUDOUHODWLRQVKLSVFDQEH UHSUHVHQWHGE\DSSURSULDWHJUDSKV 3HQJHWDO /RRNLQJIRUWHPSRUDOGHSHQGHQFLHV ZH DQDO\VH WKH GLVWULEXWLRQ RI WLPH GLVWDQFHEHWZHHQ HYHQWV RUFRPSDUH WKH XQFRQGLWLRQDO
%QORWVCVKQPCN+PVGNNKIGPEGCPF/QFGTP*GWTKUVKEU
SUREDELOLW\RIWKHZDLWLQJWLPHIRUVRPHHYHQWZLWKDFRQGLWLRQDOSUREDELOLW\LQUHOHYHQFHWR VRPHRWKHUHYHQW9DULRXVHYHQWSDWWHUQVPD\VLJQDOV\VWHPSUREOHPVRUFRQILUPLWVKHDOWK HJ KHDUWEHDWV VXFFHVVIXO SURJUDP XSGDWHV 7KHLU LQWHUSUHWDWLRQ FDQ EH VLPSOLILHG E\ FRUUHODWLQJWKHPZLWKSHUIRUPDQFHSURSHUWLHV VHFWLRQ $Q LPSRUWDQW LVVXH LQ WUDFLQJ HYHQW ORJV LV DSSURSULDWH V\VWHP FRQILJXUDWLRQ ,I ZH DUH LQWHUHVWHG LQ PRQLWRULQJ V\VWHP DFWLYLW\ LW LV LPSRUWDQW WR DFWLYDWH ZULWLQJ LQWR ORJV VXSSOHPHQWDU\ LQIRUPDWLRQ RQ XVHU ORJLQV ILOH RSHQLQJV SURFHVVRU XVDJH HWF ,W LV DOVR LPSRUWDQW WR DVVXUH FRPSOHWHQHVV RI FROOHFWHG GDWD 6RPH GDQJHU DULVHV LI ORJV DUH UHDG SHULRGLFDOO\ VR RYHUZULWLQJ PD\ KDSSHQ +HQFH LW LV UHDVRQDEOH WR FROOHFW WKLV GDWD V\VWHPDWLFDOO\ DQG VWRULQJ LW LQ D VHSDUDWH VHUYHU &RPSOHWH LQIRUPDWLRQ RQ DOO FRPSXWHUV VLPSOLILHVILQGLQJYDULRXVFRUUHODWLRQV
2GTHQTOCPEG/QPKVQTKPI 2GTHQTOCPEGQDLGEVUCPFXCTKCDNGU ,Q PRVW FRPSXWHU V\VWHPV YDULRXV GDWD RQ SHUIRUPDQFH FDQ EH FROOHFWHG LQ DSSURSULDWH FRXQWHUV HJSURYLGHGE\:LQGRZV/LQX[ DQGDFFRUGLQJWRVRPHVDPSOLQJSROLF\ HJLQ PLQXWH SHULRGV -RKQ (FNKRXW 5HLQGHUV 7KHVH FRXQWHUV DUH FRUUHODWHG ZLWK SHUIRUPDQFH REMHFWV VXFK DV SURFHVVRU SK\VLFDO PHPRU\ FDFKH SK\VLFDO RU ORJLFDO GLVFV QHWZRUN LQWHUIDFHV VHUYHU RI VHUYLFH SURJUDPV HJ ZHE VHUYLFHV ,2 GHYLFHV HWF )RU HDFK REMHFW PDQ\ FRXQWHUV YDULDEOHV DUH GHILQHG FKDUDFWHULVLQJ LWV RSHUDWLRQDO VWDWH XVDJHDFWLYLWLHVDEQRUPDOEHKDYLRXUSHUIRUPDQFHSURSHUWLHVHWF6SHFLDOFRXQWHUVUHODWHG WR GHYHORSHG DSSOLFDWLRQV FDQ DOVR EH DGGHG 7KHVH FRXQWHUV SURYLGH GDWD XVHIXO IRU HYDOXDWLQJ V\VWHP GHSHQGDELOLW\ SUHGLFWLQJ WKUHDWV WR XQGHUWDNH DSSURSULDWH FRUUHFWLYH DFWLRQVHWF7KHOLVWRIFRXQWHUVZKLFKFDQEHFRQILJXUHGLVYHU\ORQJ)RUDQLOOXVWUDWLRQZH GHVFULEHVRPHUHSUHVHQWDWLYHFRXQWHUVUHODWHGWR:LQGRZVV\VWHPV 3URFHVVRU7LPHLVWKHSHUFHQWDJHRIHODSVHGWLPHWKDWWKHSURFHVVRUVSHQGVWRH[HFXWHDQRQ LGOH WKUHDG 7KLV FRXQWHU LV WKH SULPDU\ LQGLFDWRU RI SURFHVVRU DFWLYLW\ DQG GLVSOD\V WKH DYHUDJH SHUFHQWDJH RI EXV\ WLPH REVHUYHG GXULQJ WKH VDPSOH LQWHUYDO 8VHU 7LPH DQG 3ULYLOHJHG7LPHUHODWHWRWKHSHUFHQWDJHRIHODSVHGWLPHWKHSURFHVVRUVSHQGVLQWKHXVHUDQG LQ SULYLOHJHGPRGH UHVSHFWLYHO\ 3URFHVVRU 4XHXH /HQJWK LV WKH QXPEHU RI UHDG\ WKUHDGV LQ WKH SURFHVVRU TXHXH 3URFHVVHV LV WKH QXPEHU RI SURFHVVHV DW WKH WLPH RI GDWD FROOHFWLRQ 6LPLODUO\DUHFRXQWHGWKUHDGVHYHQWVVHPDSKRUHVHWF&RQWH[W6ZLWFKHVVHFLVWKHFRPELQHG UDWH DW ZKLFK DOO SURFHVVRUV RQ WKH FRPSXWHU DUH VZLWFKHGIURP RQH WKUHDG WR DQRWKHU HJ ZKHQ D UXQQLQJ WKUHDG YROXQWDULO\ UHOLQTXLVKHV WKH SURFHVVRU LV SUHHPSWHG E\ D KLJKHU SULRULW\ UHDG\ WKUHDG RU VZLWFKHV EHWZHHQ XVHUDQG SULYLOHJHG NHUQHO PRGH WR XVH DQ H[HFXWLYHRUVXEV\VWHPVHUYLFH ,QWHUUXSWVVHF LV WKH DYHUDJH UDWH LQ LQFLGHQWV SHU VHFRQG DW ZKLFK WKH SURFHVVRU UHFHLYHG DQG VHUYLFHG KDUGZDUH LQWHUUXSWV ,W GRHV QRW LQFOXGH GHIHUUHG SURFHGXUH FDOOV '3&V ZKLFK DUH FRXQWHG VHSDUDWHO\ 7KLV YDOXH LV DQ LQGLUHFW LQGLFDWRU RI WKH DFWLYLW\ RI GHYLFHV WKDWJHQHUDWHLQWHUUXSWVVXFKDVWKHV\VWHPFORFNWKHPRXVHGLVNGULYHUVQHWZRUNLQWHUIDFH FDUGV DQG RWKHU SHULSKHUDO GHYLFHV 7KHVH GHYLFHV QRUPDOO\ LQWHUUXSW WKH SURFHVVRU ZKHQ WKH\ KDYH FRPSOHWHG D WDVN RU UHTXLUH DWWHQWLRQ 7KH V\VWHP FORFN W\SLFDOO\ LQWHUUXSWV WKH SURFHVVRU HYHU\ PLOOLVHFRQGV FUHDWLQJ D EDFNJURXQG RI LQWHUUXSW DFWLYLW\ 7KLV FRXQWHU GLVSOD\VWKHGLIIHUHQFHEHWZHHQWKHYDOXHVREVHUYHGLQWKHODVWWZRVDPSOHV,QWHUUXSW7LPH
&GRGPFCDKNKV['XCNWCVKQP$CUGFQP5[UVGO/QPKVQTKPI
LVWKHWLPHWKHSURFHVVRUVSHQGVUHFHLYLQJDQGVHUYLFLQJKDUGZDUHLQWHUUXSWVGXULQJVDPSOH LQWHUYDOV 6\VWHP8S7LPHLVWKHHODSVHGWLPH LQVHFRQGV WKDWWKHFRPSXWHUKDVEHHQUXQQLQJVLQFHLW ZDVODVWVWDUWHGWLOOWKHFXUUHQWWLPH&7LPHLVWKHSHUFHQWDJHRIWLPHWKHSURFHVVRUVSHQGV LQ WKH & ORZSRZHU LGOH VWDWH HQDEOHV WKH SURFHVVRU WR PDLQWDLQ LWV HQWLUH FRQWH[W DQG TXLFNO\UHWXUQWRWKHUXQQLQJVWDWH VLPLODUWLPHVDUHPHDVXUHGIRU& DORZHUSRZHUDQG KLJKHUH[LWODWHQF\VWDWHWKDQ&LWPDLQWDLQVWKHFRQWH[WRIV\VWHPFDFKH DQG& DORZHU SRZHU DQG KLJKHU H[LW ODWHQF\ VWDWH WKDQ & LV XQDEOH WR PDLQWDLQ WKH FRKHUHQF\ RI LWV FDFKHV VWDWHV7KHUHDUHDOVRFRXQWHUVUHODWHGWRWUDQVLWLRQVWRWKHVHVWDWHV $YDLODEOH%\WHVLVWKHDPRXQWRISK\VLFDOPHPRU\LQE\WHVDYDLODEOHWRSURFHVVHVUXQQLQJRQ WKHFRPSXWHU,WLVFDOFXODWHGE\DGGLQJWKHDPRXQWRIVSDFHRQWKH=HURHG)UHHDQG6WDQGE\ PHPRU\ OLVWV )UHH PHPRU\ LV UHDG\ IRU XVH =HURHG PHPRU\ FRQVLVWV RI SDJHV RI PHPRU\ ILOOHG ZLWK ]HURV WR SUHYHQW VXEVHTXHQW SURFHVVHV IURP VHHLQJ GDWD XVHG E\ D SUHYLRXV SURFHVV 6WDQGE\PHPRU\LVPHPRU\WKDWKDVEHHQUHPRYHGIURPDSURFHVVZRUNLQJVHW LWV SK\VLFDOPHPRU\ RQURXWHWRGLVNEXWLVVWLOODYDLODEOHWREHUHFDOOHG7KLVFRXQWHUGLVSOD\V WKHODVWREVHUYHGYDOXH )UHH6SDFHLVWKHSHUFHQWDJHRIWRWDOXVDEOHVSDFHRQWKHVHOHFWHGORJLFDOGLVNGULYHWKDWZDV IUHH $YJ 'LVN %\WHV5HDG LV WKH DYHUDJH QXPEHU RI E\WHV WUDQVIHUUHG IURP WKH GLVN GXULQJ UHDGRSHUDWLRQVVLPLODUFRXQWHURQZULWHRSHUDWLRQVLVDYDLODEOHDOVR 3DJH )DXOWVVHF LV WKH DYHUDJH QXPEHU RI SDJHV IDXOWHG SHU VHFRQG D UHIHUHQFHG SDJH LQ YLUWXDO PHPRU\ LV QRW DYDLODEOH LQ WKH ZRUNLQJ DUHD +DUG IDXOWV UHTXLUH GLVN DFFHVV DQG VRIW IDXOWV FRYHU IDXOWHG SDJHV IRXQG HOVHZKHUH LQ SK\VLFDO PHPRU\ 0RVW SURFHVVRUV FDQ KDQGOHODUJHQXPEHUVRIVRIWIDXOWVZLWKRXWVLJQLILFDQWFRQVHTXHQFHV+RZHYHUKDUGIDXOWV ZKLFKUHTXLUHGLVNDFFHVVFDQFDXVHVLJQLILFDQWGHOD\V6LPLODUO\&DFKH)DXOWVVHFLVWKHUDWH DWZKLFKIDXOWVRFFXUZKHQDSDJHVRXJKWLQWKHILOHV\VWHPFDFKHLVQRWIRXQGDQGPXVWEH UHWULHYHGIURPHOVHZKHUHLQPHPRU\RUGLVN 3DJH5HDGVVHFLVWKHUDWHDWZKLFKWKHGLVNZDVUHDG WKHQXPEHURIUHDGRSHUDWLRQVZLWKRXW UHJDUGWRWKHQXPEHURISDJHVUHWULHYHGLQHDFKRSHUDWLRQ WRUHVROYHKDUGSDJHIDXOWV3DJHV 2XWSXWVHFLVWKHUDWHDWZKLFKSDJHVDUHZULWWHQWRGLVNWRIUHHXSVSDFHLQSK\VLFDOPHPRU\ $ KLJK UDWH RI 3DJHV 2XWSXW PLJKW LQGLFDWH D PHPRU\ VKRUWDJH 3RRO 3DJHG )DLOXUHV LV WKH QXPEHU RI WLPHV DOORFDWLRQV IURP SDJHG SRRO KDYH IDLOHG ,W LQGLFDWHV WKDW WKH FRPSXWHU V SK\VLFDOPHPRU\RUSDJLQJILOHDUHWRRVPDOO )LOH5HDG2SHUDWLRQVVHFLVWKHFRPELQHGUDWHRIILOHV\VWHPUHDGUHTXHVWVWRDOOGHYLFHVRQWKH FRPSXWHULQFOXGLQJUHTXHVWVWRUHDGIURPWKHILOHV\VWHPFDFKH7KLVFRXQWHUGLVSOD\VWKH GLIIHUHQFHEHWZHHQWKHYDOXHVREVHUYHGLQWKHODVWWZRVDPSOHVGLYLGHGE\WKHGXUDWLRQRI WKHVDPSOHLQWHUYDO)LOH&RQWURO2SHUDWLRQVVHFLVWKHFRPELQHGUDWHRIILOHV\VWHPRSHUDWLRQV WKDW DUH QHLWKHU UHDGV QRU ZULWHV VXFK DV ILOH V\VWHP FRQWURO UHTXHVWV DQG UHTXHVWV IRU LQIRUPDWLRQDERXWGHYLFHFKDUDFWHULVWLFVRUVWDWXV6SOLW,26HFUHSRUWVWKHUDWHDWZKLFK,2V WRWKHGLVNZHUHVSOLWLQWRPXOWLSOH,2V,WPD\UHVXOWIURPUHTXHVWLQJGDWDRIDVL]HWKDWLV WRRODUJHWRILWLQWRDVLQJOH,2RUWKDWWKHGLVNLVIUDJPHQWHG 6HUYHUSHUIRUPDQFHFRXQWHUVJLYH WKHQXPEHURIE\WHVWKHVHUYHUKDVUHFHLYHG RUVHQW IURP WKHQHWZRUN LQGLFDWHVWKHVHUYHUORDG WKHQXPEHURIVHVVLRQVWKDWKDYHEHHQFORVHGGXHWR XQH[SHFWHG HUURU FRQGLWLRQVRU VHVVLRQV WKDW KDYH UHDFKHG WKH DXWRGLVFRQQHFW WLPHRXWDQG KDYH EHHQ GLVFRQQHFWHG QRUPDOO\ IDLOHG ORJRQ DWWHPSWV WR WKH VHUYHU SDVVZRUG JXHVVLQJ SURJUDPVDUHEHLQJXVHGWRFUDFNWKHVHFXULW\ WKHQXPEHURIVHVVLRQVWKDWKDYHEHHQIRUFHG WR ORJRII GXH WR ORJRQ WLPH FRQVWUDLQWV WKH QXPEHU RI VHVVLRQV WKDW KDYH WHUPLQDWHG
%QORWVCVKQPCN+PVGNNKIGPEGCPF/QFGTP*GWTKUVKEU
QRUPDOO\ WKLVDOORZVWRILQGSHUFHQWDJHRIWKHVHVVLRQVWLPHRXWVRUHUURUV 2WKHUFRXQWHUV SURYLGH VRPH VWDWLVWLFV RQ ILOH RSHUDWLRQV VXFK DV WKH QXPEHU RI WLPHV DFFHVVHV WR ILOHV RSHQHGVXFFHVVIXOO\ZHUHGHQLHG LPSURSHUDFFHVVDXWKRULVDWLRQHWF WKHQXPEHURIIDLOHG ILOH RSHQV DWWHPSWLQJ WR DFFHVV ILOHV QRW SURSHUO\ SURWHFWHG WKH QXPEHU RI ILOHV FXUUHQWO\ RSHQHG LQ WKH VHUYHU WKH QXPEHU RI VHDUFKHV IRU ILOHV FXUUHQWO\ DFWLYH WKH QXPEHU RI VHVVLRQVFXUUHQWO\DFWLYHLQWKHVHUYHU LQGLFDWHVFXUUHQWVHUYHUDFWLYLW\ 7KHUH DUH PDQ\ FRXQWHUV FKDUDFWHULVLQJ QHWZRUN WUDIILF RU 7&3,3 SURWRFRO DFWLYLW\ +HUH DUHJLYHQVRPHH[DPSOHV%\WHV5HFHLYHGVHFLVWKHUDWHDWZKLFKE\WHVDUHUHFHLYHGRYHUHDFK QHWZRUN DGDSWHU LQFOXGLQJ IUDPLQJ FKDUDFWHUV &XUUHQW %DQGZLGWK LV DQ HVWLPDWH RI WKH FXUUHQWEDQGZLGWKRIWKHQHWZRUNLQWHUIDFHLQELWVSHUVHFRQG3DFNHWV5HFHLYHG(UURUVLVWKH QXPEHU RI LQERXQG SDFNHWV WKDW FRQWDLQHG HUURUV SUHYHQWLQJ GHOLYHU\ WR D KLJKHUOD\HU SURWRFRO 3DFNHWV 5HFHLYHG 'LVFDUGHG LV WKH QXPEHU RI LQERXQG SDFNHWV WKDW ZHUH GLVFDUGHG HYHQ WKRXJK QR HUURUV KDG EHHQ GHWHFWHG HJ WR IUHH XS EXIIHU VSDFH 3DFNHWV 5HFHLYHG 8QNQRZQ LV WKH QXPEHU RI SDFNHWV UHFHLYHG WKURXJK WKH LQWHUIDFH WKDW ZHUH GLVFDUGHG EHFDXVHRIDQXQNQRZQRUXQVXSSRUWHGSURWRFRO2XWSXW4XHXH/HQJWKLVWKHOHQJWKRIWKH RXWSXWSDFNHWTXHXHLIWKLVLVORQJHUWKDQWZRWKHUHDUHGHOD\VDQGWKHERWWOHQHFNVKRXOGEH IRXQG DQG HOLPLQDWHG &RQQHFWLRQ )DLOXUHV LV WKH QXPEHU RI WLPHV 7&3 FRQQHFWLRQV KDYH PDGHDGLUHFWWUDQVLWLRQWRWKH&/26('VWDWHIURPWKH6
