Classifier Systems In Combat: Two Sided Learning of ... - CiteSeerX

&ODVVLILHU6\VWHPV,Q&RPEDW7ZR6LGHG/HDUQLQJRI 0DQHXYHUV)RU$GYDQFHG)LJKWHU$LUFUDIW 5(6PLWK %$'LNH‚5.0HKUD°%5DYLFKDQGUDQ†DQG$(O)DOODK

Abstract

7KLVSDSHUUHSRUWVWKHFRQWLQXLQJUHVXOWVRIDSURMHFWZKHUHD JHQHWLFVEDVHGPDFKLQHOHDUQLQJV\VWHPDFTXLUHV UXOHVIRUQRYHOILJKWHUFRPEDWPDQHXYHUVWKURXJKVLPXODWLRQ,QWKLVSURMHFWDJHQHWLFVEDVHGPDFKLQHOHDUQLQJ V\VWHPZDVLPSOHPHQWHGWRJHQHUDWHKLJKDQJOHRIDWWDFNDLUFRPEDWWDFWLFVIRUDGYDQFHGILJKWHUDLUFUDIW7KLV V\VWHPZKLFKZDVEDVHGRQDOHDUQLQJFODVVLILHUV\VWHPDSSURDFKHPSOR\HGDGLJLWDOVLPXODWLRQPRGHORIRQH YHUVXVRQHDLUFRPEDWDQGDJHQHWLFDOJRULWKPWRGHYHORSHIIHFWLYHWDFWLFVIRUWKH;H[SHULPHQWDOILJKWHU DLUFUDIW 3UHYLRXV HIIRUWV ZLWK WKLVV \VWHP VKRZHG WKDWW KH UHVXOWLQJ PDQHXYHUV DOORZHG WKH ; WR VXFFHVVIXOO\ H[SORLW LWV SRVWVWDOO FDSDELOLWLHV DJDLQVW D FRQYHQWLRQDO ILJKWHU RSSRQHQW 7KLV GHPRQVWUDWHG WKH DELOLW\RIWKHJHQHWLFOHDUQLQJV\VWHPWRGLVFRYHUQRYHOWDFWLFVLQDG\QDPLFDLUFRPEDWHQYLURQPHQW7KHUHVXOWV JDLQHGIDYRUDEOHHYDOXDWLRQIURPILJKWHUDLUFUDIWWHVWSLORWV+RZHYHUWKHVHSLORWVQRWHGWKDWWKHVWDWLFVWUDWHJ\ HPSOR\HG E\ WKH ; V RSSRQHQW ZDV D OLPLWDWLRQ ,Q UHVSRQVH WR WKHVHF RPPHQWV WKLV SDSHUU HSRUWV QHZ UHVXOWV ZLWK WZRVLGHG OHDUQLQJ ZKHUH ERWK DLUFUDIW LQ D RQHYHUVXVRQHF RPEDW VFHQDULR XVH JHQHWLFVEDVHG PDFKLQHOHDUQLQJWRDGDSWWKHLUVWUDWHJLHV7KHH[SHULPHQWVVXFFHVVIXOO\GHPRQVWUDWHERWKDLUFUDIW GHYHORSLQJ REMHFWLYHO\LQWHUHVWLQJVWUDWHJLHV+RZHYHUWKHUHVXOWVDOVRSRLQWRXWWKHFRPSOH[LW\RIHYDOXDWLQJUHVXOWVIURP PXWXDOO\DGDSWLYHSOD\HUVGXHWRWKHUHGTXHHQHIIHFW7KHVHFRPSOH[LWLHVDQGIXWXUHGLUHFWLRQVRIWKHSURMHFW DUHGLVFXVVHGLQWKHSDSHU©VFRQFOXVLRQV

,QWURGXFWLRQ

&XUUHQWO\ ILJKWHU DLUFUDIW DUH EHLQJG HYHORSHG ZLWKQ HZ KLJKO\ DJLOH IOLJKW FKDUDFWHULVWLFV 2QHF DQ NQRZDJUHDWGHDODERXWWKHDHURG\QDPLFFKDUDFWHULVWLFVRIDQDLUFUDIWIRUVLPXODWLRQZLWKRXWDSK\VLFDO SURWRW\SH+RZHYHUWKLVGRHVQRWPHDQWKDWRQHQHFHVVDULO\NQRZVKRZWKHVHFKDUDFWHULVWLFVFDQEHXVHG LQFRPEDWPDQHXYHUV6HYHUDODSSURDFKHVWRGLVFRYHULQJPDQHXYHUVDUHSRVVLEOH

)OLJKWW HVWV RU PDQLQWKHORRS VLPXODWLRQV SURYLGH WKH PRVW UHDOLVWLFH QYLURQPHQWV IRU FRQWUROOHG WHVWLQJRIILJKWHUHIIHFWLYHQHVV+RZHYHUPDQQHGWHVWLQJUHTXLUHVWULDODQGHUURUWDFWLFVGHYHORSPHQWE\ KLJKO\WUDLQHGSLORWVDQGWKHFRVW RI FRPSUHKHQVLYH WDFWLFDO DQDO\VLV LV SURKLELWLYH 0RUHRYHU PDQLQ WKHORRSVLPXODWLRQRUIOLJKWWHVWLQJFDQH[SORUHDWPRVWDVPDOOVXEVHWRIPHDQLQJIXODLUFRPEDWWDFWLFV DQGRQO\DWJUHDWFRVWDQGHIIRUW7KHUHIRUHWRIXOO\DGGUHVVWKHIDFWRUVWKDWLQIOXHQFHPRGHUQDLUFRPEDW WDFWLFV RQH PXVW FRQGXFW HYDOXDWLRQV ZLWKRXWL QYROYLQJ H[KDXVWLYHH [SHULPHQWDWLRQ LQ PDQQHG VLPXODWLRQVRUIOLJKWWHVWUDQJHV $GHTXDWHH[SORUDWLRQRISRVVLELOLWLHVIRUDLUFRPEDWWDFWLFVUHTXLUHVVRPHIRUPRIDXWRPDWHGVHDUFKDQG RSWLPL]DWLRQSURFHVV2IIOLQHPHWKRGVDUHQHHGHGWRGHILQHDQGHYDOXDWHRSWLPDOWDFWLFVHIILFLHQWO\DQG V\VWHPDWLFDOO\,I WKH EHVW RU QHDUEHVW PDQHXYHUV DUH QRWL GHQWLILHG WKH IXOO SRWHQWLDO DQG YDOXH RI WKH QH[WJHQHUDWLRQDJLOHDLUFUDIWPD\QRWEHUHDOL]HG

2IIOLQH PHWKRGV IRU WUDMHFWRU\ RSWLPL]DWLRQK DYH EHHQ HPSOR\HG LQ WDFWLFV UHVHDUFK IRU D QXPEHU RI \HDUV 7KH VWDQGDUG DSSURDFKHV XVXDOO\ LQYROYH RU VLGHG RSWLPDO FRQWURO DOJRULWKPV WR VROYH D GLIIHUHQWLDO JDPH SUREOHP :KLOH SUHFLVH DQG ZHOOXQGHUVWRRG WKHVHF DOFXOXVEDVHG PHWKRGV KDYH VLJQLILFDQW VKRUWFRPLQJV ZKLFK IDOOL QWR WKUHHF DWHJRULHV )LUVW WKH PHWKRGV DUHH VVHQWLDOO\ ORFDOL Q QDWXUH VLQFH WKH\ HLWKHU LPSOLFLWO\ RU H[SOLFLWO\ HPSOR\ GHULYDWLYHV RU JUDGLHQW VHDUFKHV WR ORFDWH WKH VROXWLRQ6HFRQGFRQYHQWLRQDOPHWKRGVKDYHODUJHDQDO\WLFDODQGFRPSXWDWLRQDOFRVWV7KLUGDQGPRVW LPSRUWDQWO\FRQYHQWLRQDOWUDMHFWRU\RSWLPL]DWLRQPHWKRGVJURVVO\RYHUVLPSOLI\PDQ\RSHUDWLRQDOIDFWRUV SUHVHQWLQDLUFRPEDW

7KH,QWHOOLJHQW&RPSXWLQJ6\VWHPV&HQWUH7KH8QLYHUVLW\RIWKH:HVWRI(QJODQG&ROGKDUERXU/DQH

)UHQFKD\%ULVWRO%64@XVHVDPDFKLQHOHDUQLQJV\VWHPWRWDNHWKHSODFHRIWKH WHVW SLORWL Q VLPXODWLRQ 7KURXJK UHSHDWHG LQWHUDFWLRQ ZLWK VLPXODWHG RSSRQHQWV WKH PDFKLQH OHDUQLQJ V\VWHPOHDUQVQRYHOFRPEDWPDQHXYHUVWKURXJKH[SHULHQFH

7KLVDSSOLFDWLRQRIJHQHWLFVEDVHGPDFKLQHOHDUQLQJWRDLUFRPEDWWDFWLFVRSWLPL]DWLRQGLIIHUVIURPPDQ\ RWKHU PDFKLQH OHDUQLQJ DSSOLFDWLRQV VLQFH RQOLQHF RQWUROL V QRWW KH SULPDU\ JRDO 5DWKHU VLPSOH GLVFRYHU\ RI LQWHUHVWLQJQ HZ LGHDV LV D ZRUWKZKLOH JRDOL Q LWVHOI ,I QRYHO QHZ PDQHXYHUV FDQE H GLVFRYHUHGLQVLPXODWHGFRPEDWWKHVHPDQHXYHUVFDQEHXVHGDVIHHGEDFNWRDQDO\VWVDLUFUDIWGHVLJQHUV FXVWRPHUVDQGSLORWV 7KLVSDSHUSUHVHQWVDFRQWLQXDWLRQRI SUHYLRXV ZRUN ZLWK WKH JHQHWLFVEDVHG PDFKLQH OHDUQLQJ V\VWHP DSSOLHGWRDLUFRPEDW,QRXUSUHYLRXVZRUNRQH*$FRQWUROOHGDLUFUDIWDGDSWHGWRIDFHDQRWKHUDLUFUDIW WKDW HPSOR\HG D UHDFWLYH EXW VWDWLF FRPEDW VWUDWHJ\ $OWKRXJK WKH UHVXOWV ZHUH MXGJHG ZRUWKZKLOH E\ DFWXDO ILJKWHU SLORWV WKH\ VXJJHVWHG WKDW ERWKS OD\HUVV KRXOGE H DOORZHG WR DGDSWW KHLU VWUDWHJLHV 3UHOLPLQDU\H[SHULPHQWVZKHUHERWKSOD\HUVDGDSW WZRVLGHGOHDUQLQJ  DUHSUHVHQWHGLQWKLVSDSHU ,QWKHUHPDLQGHURIWKHSDSHUZHPRWLYDWHPHWKRGVXVHGLQWKHSURMHFWDQGUHYLHZSDVWUHVXOWV>@:H WKHQG LVFXVV WZRVLGHG OHDUQLQJ DQGG HVFULEH WKUHH WZRVLGHG OHDUQLQJ DOJRULWKPV 5HVXOWV IURP WKH DOJRULWKPVDUHUHYLHZHG5HVXOWVIURPVLPXODWLRQVZLWKRXWOHDUQLQJZLWK RQHVLGHG OHDUQLQJ DQG ZLWK WZRVLGHGOHDUQLQJDUHFRPSDUHG7KHFRPSOLFDWLRQVLQYROYHGLQLQWHUSUHWLQJWZRVLGHGOHDUQLQJUHVXOWV DUHQRWHGDQGGLVFXVVHG)LQDOO\IXWXUHGLUHFWLRQVRIWKLVSURMHFWDUHSUHVHQWHG

)LJKWHU$LUFUDIW0DQHXYHULQJ

7RXQGHUVWDQGPRWLYDWLRQVIRUWKHZRUNSUHVHQWHGKHUHDQGWKHUHVXOWVSUHVHQWHGRQHPXVWJUDVSDIHZ EDVLFDVSHFWVRIILJKWHUDLUFUDIWPDQHXYHUV7KLVVHFWLRQSURYLGHVDQRYHUYLHZ)RUPRUHFRPSUHKHQVLYH LQIRUPDWLRQVHH>@

$QLPSRUWDQWDVSHFWRIPDQ\PRGHUQILJKWHUDLUFUDIWLVWKHXVHRISRVWVWDOOWHFKQRORJ\367 367UHIHUV WR V\VWHPVV XFK DV WKUXVW YHFWRULQJ DQG DGYDQFHG IOLJKW FRQWUROV ZKLFK HQDEOH WKH SLORWW R IO\ DW H[WUHPHO\KLJK DQJOHVRIDWWDFN ZHOOEH\RQGWKHQRUPDOVWDOOOLPLWVRIFRQYHQWLRQDODLUFUDIW $QJOHRI DWWDFNUHIHUVWRWKHDQJOHEHWZHHQWKHDLUFUDIW VYHORFLW\YHFWRUDQGWKHDLUFUDIW VQRVHDVPHDVXUHGLQWKH JHRPHWULF SODQHF RQWDLQLQJ WKH DLUFUDIW V QRVH DQG YHUWLFDOW DLO 8VLQJ 367 IOLJKW PRGHV SLORWV KDYH GHYHORSHGDQHQWLUHO\QHZFODVVRIFRPEDWPDQHXYHUV 367PDQHXYHUVLQFOXGH x 7KH+HUEVW0DQHXYHU ZKLFKLVWKHPRVWZHOONQRZQRI367PDQHXYHUV,QD+HUEVWPDQHXYHU WKHDLUFUDIWTXLFNO\UHYHUVHVGLUHFWLRQWKURXJKDFRPELQDWLRQRIKLJKDQJOHRIDWWDFNDQGUROOLQJ7KH +HUEVWPDQHXYHULVQDPHGDIWHU:ROIJDQJ+HUEVWRQHRIWKHRULJLQDOGHYHORSHUVRI367 x 7KH+HOLFRSWHU*XQ$WWDFN ZKHUHWKH367DLUFUDIWSRLQWVLWVQRVHDWDFLUFOLQJDGYHUVDU\ZKLOH VWD\LQJLQWKHFHQWHURIWKHDGYHUVDU\ VWXUQLQJFLUFOH x 7KH&REUD0DQHXYHU ZKLFKZDVILUVWGHPRQVWUDWHGE\WKH5XVVLDQSLORW3RXJDFKHY x 7KH367+DPPHUKHDG7XUQ ZKHUHWKHDLUFUDIWUHYHUVHVGLUHFWLRQE\SHUIRUPLQJDPDQHXYHU UHVHPEOLQJDEDFNIOLS

2QHW\SLFDODWWULEXWHRI367WDFWLFVLVRXWRISODQHPDQHXYHULQJZKHUHWKHDWWDFNLQJDLUFUDIWIOLHVLQD GLIIHUHQW PDQHXYHULQJS ODQH WKDQ WKH WDUJHW 7KH PDQHXYHULQJS ODQH LV GHWHUPLQHGE \ DQ DLUFUDIW V YHORFLW\YHFWRUDQGDHURG\QDPLFOLIWYHFWRU$IUHTXHQWFKDUDFWHULVWLFRI367PDQHXYHUVLVDFRQWLQXDOO\ FKDQJLQJPDQHXYHUSODQH 367PDQHXYHUVDUHSULPDULO\XVHIXOIRUDLUWRDLUJXQDWWDFNVEHFDXVHWKHJXQLVIL[HGLQDOLJQPHQWZLWK WKHDLUFUDIW VQRVHUHTXLULQJWKHDLUFUDIWWREHSRLQWHGGLUHFWO\DWWKHWDUJHW $LUWRDLUPLVVLOHDWWDFNVGR QRW UHTXLUH D VLPLODU GHJUHH RI QRVH SRLQWLQJE HFDXVH PLVVLOHV WXUQ DIWHU ODXQFK LQ WKH GLUHFWLRQ RI WKH WDUJHW0LVVLOHVDUHWKHSUHIHUUHGPRGHRIDWWDFNIRUDLUFRPEDW +RZHYHUDLUWRDLUJXQDWWDFNVPD\EH UHTXLUHGLQWDFWLFDOVLWXDWLRQVLIWKHWDUJHWLVLQVLGHWKHPLQLPXPPLVVLOHUDQJHRULIWKHWDUJHWHPSOR\V FRXQWHUPHDVXUHVZKLFKGHIHDWPLVVLOHV

2QHFULWLFDOWDFWLFDO PHDVXUH LQ DLU FRPEDWL V WDUJHW DVSHFW DQJOH ZKLFK LV WKH DQJOH EHWZHHQ D WDUJHW DLUFUDIW VQRVHDQGWKHOLQHRIVLJKWWRWKHDWWDFNLQJDLUFUDIW,IWKHWDUJHWDVSHFWDQJOHLVWKHQWKHWDUJHW DLUFUDIWLVSRLQWHGGLUHFWO\DWWKHDWWDFNHU ,I WKH WDUJHW DVSHFW DQJOH LV  WKHQ WKH DWWDFNHU LV RQ WKH WDUJHW V WDLO 7DFWLFDO DGYDQWDJH LQ FORVHLQ DLU FRPEDW FDQE H PHDVXUHG DV WKH GLIIHUHQFH EHWZHHQ WKH WDUJHW DVSHFW DQJOH DQG RZQVKLS DVSHFW DQJOH VLQFH WKH PRVW GHVLUDEOHF RQGLWLRQ IRU WKH DWWDFNHU LV WR SRLQWGLUHFWO\DWWKHWDUJHWZKLOHWKHWDUJHWLVSRLQWHGGLUHFWO\DZD\IURPWKHDWWDFNHU 'XH WR WKHLU FRPSOH[ G\QDPLFV 367 PDQHXYHUV DUH GLIILFXOWW R FKDUDFWHUL]H ZLWKRXWW KH XVH RI ' YLVXDOL]DWLRQ WRROV 7KH SULPDU\ YLVXDOL]DWLRQ WRRO XVHG LQ WKLVV WXG\ ZDV $JLOH9X D ' JUDSKLFV

DQLPDWLRQS DFNDJH GHYHORSHGE \ WKH 86 1DY\ IRU DQDO\]LQJ DLUFUDIW IOLJKWW UDMHFWRULHV $JLOH9X SURYLGHVWKHXVHUZLWKWKHDELOLW\WRH[DPLQHPDQHXYHUVIURPDYDULHW\RIYLHZSRLQWVDQGGHWHUPLQHWKH GHJUHHRIRXWRISODQHPDQHXYHULQJ ,QWKHH[SHULPHQWVSUHVHQWHGKHUHWZRDLUFUDIWDUHHPSOR\HG2QHLVWKH;H[SHULPHQWDOILJKWHUSODQH ZKLFKK DVV LJQLILFDQW 367 FKDUDFWHULVWLFV 7KH RWKHU LV DQ ) ZKLFKK DV PRUH VWDQGDUG ILJKWHU FKDUDFWHULVWLFV

*HQHWLFV%DVHG0DFKLQH/HDUQLQJ

$OWKRXJK *$V DUH PRVW RIWHQX VHG LQ RSWLPL]DWLRQ D JURZLQJ DUHD RI *$ DSSOLFDWLRQ LV LQ PDFKLQH OHDUQLQJ2SWLPL]DWLRQFDQEHGLVWLQJXLVKHGIURPOHDUQLQJLQVHYHUDOLPSRUWDQWUHVSHFWV,QRSWLPL]DWLRQ D V\VWHP OLNH WKH *$ LV JLYHQ D IL[HG VHW RI SDUDPHWHUV WR WXQH DQG LWV RQO\ IHHGEDFN LV WKH XWLOLW\ RI YDULRXVSDUDPHWHUVHWWLQJV,QDPDFKLQHOHDUQLQJSUREOHPDV\VWHPPXVWLQWHUDFWZLWKDQHQYLURQPHQW WKDW SURYLGHV XWLOLW\UHODWHG IHHGEDFN DQG IHHGEDFN WKDWL QGLFDWHVV RPH WLPHYDU\LQJ VWDWH RI WKH HQYLURQPHQW,QJHQHUDOWKHV\VWHPPXVWILQGVHYHUDOKLJKXWLOLW\VHWVRISDUDPHWHUVWKDWFRUUHVSRQGWR YDULRXVV WDWHIHHGEDFN VLJQDOV 2QH DSSURDFK WR PDFKLQH OHDUQLQJS UREOHPV LV WR FRQYHUWW KHP WR RSWLPL]DWLRQSUREOHPVE\DVVLJQLQJDIXOOVHWRISDUDPHWHUVIRUHYHU\SRVVLEOHHQYLURQPHQWDOVWDWHDQG VLPSO\ RSWLPL]LQJ ,QJ HQHUDOW KLV DSSURDFK LV LPSUDFWLFDO ,QVWHDG RQH ZRXOG OLNH IRU WKH OHDUQLQJ V\VWHP WR JHQHUDOL]H ZLWK VHWV RI SDUDPHWHUV WKDW SHUIRUP ZHOO DFURVV D UDQJH RI HQYLURQPHQWDO VWDWHV %HFDXVH WKH DSSURSULDWH GHJUHH RI JHQHUDOL]DWLRQ LV LWVHOI DQXQNQ RZQ WKHH IIHFWLYH QXPEHU RI SDUDPHWHUV QHHGHG LV XQNQRZQ DV ZHOO 0RUHRYHU LW PLJKW EH QHFHVVDU\ WRR EWDLQ WKHVH SDUDPHWHUV JUDGXDOO\LQFKXQNVWKDWEXLOGWRZDUGVDILQDOJHQHUDOL]HGVROXWLRQ7KXVWKHSUREOHPLVGLIILFXOWWR FDVWDVRQHRIW\SLFDORSWLPL]DWLRQ 7KLVZRUNFRQVLGHUV*$VLQDPDFKLQHOHDUQLQJFRQWH[W$WKRURXJKUHYLHZRIJHQHWLFVEDVHGPDFKLQH OHDUQLQJFDQQRWEHLQFOXGHGKHUHIRUWKHVDNHDEUHYLW\+RZHYHUDFRQFLVHUHYLHZRI/&6VWKHSULPDU\ JHQHWLFVEDVHGPDFKLQHOHDUQLQJWHFKQLTXHLVSURYLGHGLQWKHIROORZLQJVHFWLRQ

/HDUQLQJ&ODVVLILHU6\VWHPV

&RQVLGHUWKHIROORZLQJPHWKRGIRUUHSUHVHQWLQJDVWDWHDFWLRQSDLU LQ D UHLQIRUFHPHQWO HDUQLQJS UREOHP >@ HQFRGH DQ HQYLURQPHQWDO VWDWH LQE LQDU\ DQG FRXSOH LW WR DQ DFWLRQ WKDW FDQE H WDNHQ LQ WKH HQYLURQPHQWZKLFKLVDOVRHQFRGHGLQELQDU\,QRWKHUZRUGVWKHVWULQJ 

UHSUHVHQWVRQHRIVWDWHVDQGRQHRIHLJKWDFWLRQV7KLVVWULQJFDQDOVREHVHHQDVDUXOHWKDWVD\V¦,)LQ VWDWH     7+(1 WDNH DFWLRQ § ,Q D /&6 VXFK D UXOH LV FDOOHG D FODVVLILHU 2QHF DQ HDVLO\ DVVRFLDWHDFRVWYDOXHRURWKHUSHUIRUPDQFHPHDVXUHVZLWKDQ\JLYHQFODVVLILHU 1RZFRQVLGHUJHQHUDOL]LQJRYHUDFWLRQVE\LQWURGXFLQJD¦GRQ WFDUH§FKDUDFWHU LQWRWKHVWDWHSRUWLRQ RIDFODVVLILHU,QRWKHUZRUGVWKHVWULQJ 

LVDUXOHWKDWVD\V¦,)LQVWDWH25VWDWH25VWDWH25VWDWH7+(1WDNH DFWLRQ § 7KH LQWURGXFWLRQ RI WKLV JHQHUDOLW\ DOORZV DQ /&6 WR UHSUHVHQW FOXVWHUV RI VWDWHV DQG DVVRFLDWHGDFWLRQV+RZHYHURQHFDQNHHSRQO\DOLPLWHGOLVWRIWKHVHFOXVWHUWHPSODWHVZKLFKDUHDOVR FDOOHG UXOHV RU FODVVLILHUV  7KH VHDUFKS UREOHP LQYROYHG LV GHFLGLQJ ZKLFK UXOHV WR NHHS DQG WKXV GHVLJQDQHIIHFWLYHUXOHGLFWDWHGPDQHXYHUVWUDWHJ\7KLVLVDVHDUFKSUREOHPWKDWQDWXUDOO\OHQGVLWVHOIWR WKH JHQHWLF DOJRULWKP *$  7KH *$ LV XVHG LQ D /&6 WR ORFDWH D JRRG VHW RI JHQHUDOL]DWLRQV FOXVWHU WHPSODWHVRUJHQHUDOL]HGUXOHV 1RWHWKDWWKLVSUREOHPUHTXLUHVDGLYHUVHVHWRIFRRSHUDWLYHFODVVLILHUVWR GLFWDWHDVWUDWHJ\7KXVWKH/&6DSSOLHGWRWKLVSUREOHPPXVW FRHYROYH DQDSSURSULDWHVHWRIFODVVLILHUV >@%\XVLQJWKH*$WRVHDUFKIRUVXFKVWULQJVRQHFDQVHDUFKIRUZD\VRIFOXVWHULQJVWDWHVWRJHWKHUVXFK WKDWWKH\FDQEHDVVLJQHGMRLQWSHUIRUPDQFHVWDWLVWLFV>@>@

Black Box Environment Message

Detectors

Action

Effectors

Reward

CA

External Messages Message Space Internal Messages CR

Rule Set

GA

)LJXUH6WUXFWXUHRID/HDUQLQJ&ODVVLILHU6\VWHP

)LJXUHVKRZVDGLDJUDPRIWKHVWUXFWXUHRIDQ/&6,QDVWLPXOXVUHVSRQVH/&6OLNHWKHV\VWHPXVHG LQWKLVSDSHU DWHDFKWLPHVWHSDPHVVDJHLVSRVWHGE\WKHV\VWHP VGHWHFWRUVUHSUHVHQWLQJWKHVWDWHRI WKHH QYLURQPHQW &ODVVLILHUV DUH PDWFKHG DJDLQVWW KLV H[WHUQDO PHVVDJH )URP DPRQJVWW KH PDWFKLQJ FODVVLILHUV VRPHF RQIOLFW UHVROXWLRQ &5  PHWKRG LV XVHG WR VHOHFW D VLQJOHF ODVVLILHU WR DFW 7KDW FODVVLILHU©V DFWLRQ LVV XEPLWWHG WR WKHH QYLURQPHQW WKURXJK WKH V\VWHP©V HIIHFWRUV ,I UHZDUG SHUIRUPDQFH IHHGEDFN  LV UHFHLYHG IURP WKHH QYLURQPHQW LW LV GLVWULEXWHGE \ D FUHGLW DOORFDWLRQ &$ V\VWHP6RPH/&6VDOVRDOORZIRULQWHUQDOPHVVDJHVWKDWUHSUHVHQWFRPPXQLFDWLRQEHWZHHQFODVVLILHUV RQVXFFHVVLYHWLPHVWHSV+RZHYHUWKLVSRVVLELOLW\LVQRWFRQVLGHUHGKHUH 7KH*$LVDSSOLHGSHULRGLFDOO\DIWHUDVSHFLILHGQXPEHU RI RSHUDWLRQDO F\FOHV 7\SLFDOO\ D KLJKILWQHVV SRUWLRQ RI WKHF ODVVLILHU OLVW VHUYHV DV WKH *$ SRSXODWLRQ 1HZO\ SURGXFHG FODVVLILHUV UHSODFH D ORZ ILWQHVVSRUWLRQRIWKHOLVW

*LYHQWKHRXWOLQHRIWKHEDVLFOHDUQLQJV\VWHPVWUXFWXUHRQHPXVWFRQVLGHUZK\DQ /&6 LV DSSURSULDWH IRUGLVFRYHULQJUXOHVWKDWGLFWDWHFRPEDWPDQHXYHUORJLF7KHPDMRUPRWLYDWLRQIRUWKLVDSSURDFKLVWKH H[WHQVLYH WKHRUHWLFDO DQG HPSLULFDO HYLGHQFH WKDW *$V KDYH UREXVW DGDSWLYH DELOLW\ LQ LOOFRQGLWLRQHG VHDUFK VSDFHV >@>@ 0RUHRYHU VXFK V\VWHPV DOORZ IRU GLVFRYHU\ RI FRDGDSWHG UXOHV WKDW GLFWDWH FRPSOH[ PDQHXYHUV )LQDOO\ DOOW KHVH DELOLWLHV RQO\ GHSHQG RQG LUHFW SHUIRUPDQFH RQO\ IHHGEDFN PDNLQJWKHV\VWHPLGHDOIRUOHDUQLQJIURPH[SHULHQFHLQVLPXODWLRQ

3DVW(IIRUWV

7KH V\VWHP XVHG LQ RXU SDVW HIIRUWV >@ DQG H[WHQGHGK HUH LQWHUDFWV ZLWK WKH $LUWR$LU 6\VWHP 3HUIRUPDQFH(YDOXDWLRQ0RGHO$$63(0 ILJKWHUFRPEDWVLPXODWRU $$63(0LV D 86 *RYHUQPHQW FRPSXWHU VLPXODWLRQ RI DLUWRDLU FRPEDW DQG LV RQH RI WKH VWDQGDUG PRGHOV IRU WKLV WRSLF ,W FRQWDLQV GHWDLOHG UHSUHVHQWDWLRQV RI DLUFUDIW PLVVLOHV VHQVRUV DQG RWKHU V\VWHPV DV ZHOO DV WDFWLFV ORJLF IRU ZLWKLQYLVXDOUDQJHDQGEH\RQGYLVXDOUDQJHFRPEDW ,QWKHVHRQHYHUVXVRQHVLPXODWLRQVDJHQHWLFVEDVHGPDFKLQHOHDUQLQJFRQWUROOHG DLUFUDIW UXQV DJDLQVW DQ RSSRQHQW DLUFUDIW ,Q RXU HDUO\ ZRUN WKH RSSRQHQW DLUFUDIW IOHZ XVLQJ VWDWLF VWDQGDUG $$63(0 FRPEDWORJLF,QWKHZRUNSUHVHQWHGK HUH WKH V\VWHP DOORZV ERWK DLUFUDIWW R OHDUQ YLD WKH *$ SURFHVV ZKDWZHZLOO FDOOW ZRVLGHG OHDUQLQJ 7KH ZRUN WR GDWHF RQVLVWV RI RQHYHUVXVRQH VLPXODWLRQ GLYLGHG LQWR  6HFRQG LQWHUYDOV ZLWK VHFRQGV RI VLPXODWLRQS HU HQJDJHPHQW 7KH VLPXODWLRQV FRQVLGHU FORVHLQFRPEDWJXQVRQO\FRPEDW 7KHILQDOHQJDJHPHQWVFRUHLVEDVHGRQILULQJRSSRUWXQLWLHVIRUERWK DLUFUDIW,QRWKHUZRUGVHDFKDLUFUDIW©VPHDVXUHRIHIIHFWLYHQHVVFRQVLGHUVLWVRSSRUWXQLWLHVIRUDNLOODQG LWVRSSRUWXQLWLHVWREHNLOOHGRYHUWKHVHFRQGHQJDJHPHQW

7KH IROORZLQJ VHFWLRQV RYHUYLHZ WKHF RPPRQ HOHPHQWV VLPXODWLRQ HQYLURQPHQW PHDVXUHV RI HIIHFWLYHQHVVWHVWPDWUL[DQGFODVVLILHUV\QWD[ RIWKHSDVWDQGFXUUHQWSURMHFWV7KHQWKHUHVXOWVRIRXU SDVWHIIRUWVDUHUHYLHZHGZKLFKPRWLYDWHVWKHFXUUHQWZRUNSUHVHQWHGKHUH

6LPXODWLRQ(QYLURQPHQWDQG*HQHWLF/HDUQLQJ6\VWHP$UFKLWHFWXUH

6LQFH WKH VWDQGDUG $$63(0DLUFUDIW PRGHO GRHV QRW UHSUHVHQW SRVWVWDOO IOLJKW FKDUDFWHULVWLFV %RHLQJ GHYHORSHG $$63(0 PRGLILFDWLRQV ZKLFK DGGJ UHDWHU ILGHOLW\ DQG IOH[LELOLW\ IRU 367 DLUFUDIW

DSSOLFDWLRQV7KLVPRGLILHG$$63(0HQYLURQPHQWZDVFRXSOHGWRDQ/&6EDVHGOHDUQLQJV\VWHPLQRXU H[SHULPHQWV Environment Simulation AASPEM

Engagement MOE

Credit Allocation

Rule Activation

Environment Interface

History

Rule Triggering and Activation

Rule Population

Fitness

Values

Genetic Algorithm

)LJXUH*HQHWLF/HDUQLQJ6\VWHP$UFKLWHFWXUH

)LJXUHLOOXVWUDWHVWKLVDUFKLWHFWXUH7KHV\VWHPLVFRPSRVHGRIHOHPHQWV 

$VHWXSSURFHGXUHZKLFKUDQGRPO\LQLWLDOL]HVWKHSRSXODWLRQIRUWKHILUVWJHQHUDWLRQ



$Q (QYLURQPHQW ,QWHUIDFH ZKLFKG HWHUPLQHV WKHF ODVVLILHUV WR DFWLYDWH ZLWKLQ WKHH QJDJHPHQW VLPXODWLRQ'XULQJWKLVHQJDJHPHQWPDWFKLQJUXOHVZLWKKLJKHVWILWQHVVDUHVHOHFWHGWRDFWDWHDFK HQJDJHPHQWWLPHVWHSE\WKHFRQIOLFWUHVROXWLRQPHFKDQLVP



  

$Q (QYLURQPHQW 6LPXODWLRQ $$63(0  ZKLFK H[HUFLVHV WKH UXOH SRSXODWLRQ DQGS URGXFHV DQ HQJDJHPHQWPHDVXUHRIPHULWIRUWKHUXOHSRSXODWLRQ

$&UHGLW$OORFDWLRQDOJRULWKPZKLFKFRQYHUWVWKHHQJDJHPHQWPHDVXUHRIPHULWLQWRILWQHVVYDOXHV IRUWKH*$VHHEHORZIRUGHWDLOV  $*$ZKLFKSURGXFHVVXEVHTXHQW,)7+(1FODVVLILHUUXOH SRSXODWLRQV $JHQHWLFVEDVHGPDFKLQHOHDUQLQJH[HFXWLYHZKLFKFRQWUROVWKHSURFHVV

0HDVXUHVRI(IIHFWLYHQHVV

2XUHIIRUWVKDYHLQFOXGHGDQHYDOXDWLRQRIGLIIHUHQWPHDVXUHVRIHIIHFWLYHQHVVZLWKLQWKHJHQHWLFVEDVHG PDFKLQHOHDUQLQJV\VWHPWRGHWHUPLQHWKHUHODWLYHVHQVLWLYLW\RIWKHSURFHVV,QLWLDOFDQGLGDWHPHDVXUHV LQFOXGHGH[FKDQJHUDWLRWLPHRQDGYDQWDJHWLPHWRILUVWNLOODQGRWKHUUHOHYDQWYDULDEOHV

.15 .10 f .05 .00 0

30 60 90 Target Aspect Angle

)LJXUH0HDVXUHRIHIIHFWLYHQHVVZKLFKLVDOVRXVHGDVDQLQSXWWRWKH*$ ILWQHVVIXQFWLRQ

7KHPHDVXUHRIHIIHFWLYHQHVVXOWLPDWHO\VHOHFWHGWRIHHGEDFNLQWRWKH*$ILWQHVVIXQFWLRQZDVEDVHGRQ IRXUVWHSV,QWKHILUVWVWHSFODVVLILHUVDFWLYDWHGZHUHGLUHFWO\DVVLJQHGWKHVFRUH 'IVKRZQLQ)LJXUH 7KLV YDOXH LV FDOFXODWHGE DVHG RQ WKH ODVW FRPSOHWHG HQJDJHPHQW 7KH VFRUH ZDV EDVHG RQ DYHUDJH DQJXODU DGYDQWDJH RSSRQHQWW DUJHW DVSHFW DQJOH PLQXV RZQVKLS WDUJHW DVSHFW DQJOH  RYHU WKH HQJDJHPHQW 7R HQFRXUDJH PDQHXYHUV WKDW PLJKW HQDEOH JXQ ILULQJ RSSRUWXQLWLHV DQ DGGLWLRQDO VFRUH ZDVDGGHGZKHQWKHWDUJHWZDVZLWKLQG HJUHHVRIWKHDLUFUDIW VQRVH&UHGLWDVVLJQPHQWZDVHSRFKDO LQVXULQJWKDWHDFKDFWLYDWHGFODVVLILHUZDVHTXDOO\UHZDUGHGRQWKHEDVLVRIWKHILQDOHQJDJHPHQWVFRUH ,QWKHVHFRQGVWHSQRQDFWLYDWHGFODVVLILHUVZHUHDVVLJQHGWKHDYHUDJHILWQHVVYDOXHRIWKHLUSDUHQWVIURP

WKHSUHYLRXVJHQHUDWLRQ7KLVLQKHULWDQFHSURFHVV>@LQVXUHGWKDWHOHPHQWVRIKLJKSHUIRUPLQJFODVVLILHUV ZHUH SUHVHUYHG DFURVV JHQHUDWLRQV HYHQ LI WKH\ ZHUH QRW DOZD\V DFWLYDWHG ,Q WKH WKLUG VWHS D WD[ ZDV DSSOLHG WR QRQILULQJ FODVVLILHUV WR GLVFRXUDJH WKH SUROLIHUDWLRQ RI SDUDVLWHF ODVVLILHUV ZKLFK FRQWDLQ HOHPHQWVRIKLJKSHUIRUPDQFHFODVVLILHUVEXWKDYHLQVXIILFLHQWPDWHULDOIRUDFWLYDWLRQ,QWKHILQDOVWHSDOO QRQILULQJ FODVVLILHUV WKDW ZHUH LGHQWLFDOW R D ILULQJ FODVVLILHU ZHUH UHDVVLJQHG WKH ILULQJ FODVVLILHU V ILWQHVV7KLVVWHSUHGXFHGWKHFODVVLILHUILWQHVVQRLVHOHYHO7KLVDOORFDWLRQRIFUHGLWSURFHVVDFFRPSOLVKHG WKHGXDO REMHFWLYHV RI UHLQIRUFLQJ WKH DFWLYDWHG FODVVLILHUV ZKLOH UHWDLQLQJS UHYLRXVO\ HIIHFWLYHF ODVVLILHU HOHPHQWVWKURXJKWKHLQKHULWDQFHPHFKDQLVP

7KH6WDUWLQJ&RQGLWLRQ7HVW0DWUL[

$WZRWLHUDSSURDFKZDVHPSOR\HGIRUGHILQLQJUXQFRQGLWLRQVDQGOHDUQLQJV\VWHPSDUDPHWHUV)LUVWD EDVHOLQHPDWUL[RILQLWLDOSRVLWLRQVUHODWLYHJHRPHWULHVDQGHQHUJ\VWDWHVZDV LGHQWLILHG LQ FRQMXQFWLRQ ZLWK 1$6$ UHTXLUHPHQWV 7KH SULPDU\ VRXUFH GRFXPHQW IRU WKLVV WHS ZDV WKH ; 3URMHFW 3LQEDOO ,, 7DFWLFDO8WLOLW\6XPPDU\ZKLFKFRQWDLQHGUHVXOWV IURPP DQQHG VLPXODWLRQ HQJDJHPHQWV FRQGXFWHG LQ  DW 2WWREUXQQ *HUPDQ\ ,QLWLDO ILQGLQJV IURP WKH ; 7DFWLFDO 8WLOLW\ )OLJKW 7HVW FRQGXFWHG DW 'U\GHQ)OLJKW5HVHDUFK&HQWHU ZHUHDOVRXVHGWRFRPSDUHZLWKUHVXOWVIURPWKLVSURMHFW7KHEDVHOLQH VWDUWLQJFRQGLWLRQWHVWPDWUL[DVVKRZQLQ)LJXUHZDVEDVHGRQ;PDQQHGVLPXODWLRQDQGIOLJKW WHVW FRQGLWLRQV DQG ZDV WDLORUHG WR WKH ;S HUIRUPDQFHH QYHORSH IOLJKWW HVW UDQJHF RQVWUDLQWV DQG RWKHUWHVWLQJFRQVLGHUDWLRQV 1RWH WKDW HDFK RI WKHVH UHSUHVHQWV D VHSDUDWH LQLWLDO FRQGLWLRQ IRU D ILJKWHU FRPEDW VLPXODWLRQ (DFK OHDUQLQJUXQFRQVLVWVRIUHSHDWHGHQJDJHPHQWVDOOVWDUWLQJIURPWKHVDPHRQHRIWKHVHFRQGLWLRQV,QLWLDO FRQGLWLRQVIRUHDFKOHDUQLQJUXQZLOOEHLQGLFDWHGE\DWKUHHRUIRXUOHWWHUDFURQ\P

'HIHQVLYH'()

2IIHQVLYH2))

+LJK6SHHG +HDGRQ3DVV +6+3

IW

IW

IW

NFDVNIW

NFDVNIW

6ORZ6SHHG /LQH$EUHDVW 66/$

+LJK6SHHG /LQH$EUHDVW +6/$

IW

NFDVNIW

IW

NFDVNIW

NFDVNIW

;

)

)LJXUH%DVHOLQHWHVWPDWUL[RILQLWLDOFRQGLWLRQVIRUWKHFRPEDWVLPXODWLRQV 7ULDQJOHVUHSUHVHQWILJKWHUSODQHV

7KHILUVW IRXU VWDUW FRQGLWLRQV 'HIHQVLYH '()  2IIHQVLYH 2))   6ORZ6SHHG /LQH $EUHDVW 66/$  DQG+LJK6SHHG/LQH$EUHDVW+6/$ ZHUHGHULYHGGLUHFWO\IURPWKH3LQEDOO,,SURMHFW7KHILIWKVWDUW FRQGLWLRQ +LJK 6SHHG +HDG2Q 3DVV +6+3  ZDV DGGHG WR WKH PDWUL[ WR SURYLGH DQ DGGLWLRQDO JHRPHWU\ZKLFKZRXOGQRWH[FOXVLYHO\UHVXOWLQDFORVHWXUQLQJILJKW7KHRSSRQHQWDLUFUDIWZDVDQ)$ 7KHEDVHOLQHPDWUL[IRUPHGDVHWRI FRUHF RQGLWLRQV WR JHQHUDWH ; WDFWLFV UHVXOWV IRU D EDODQFHG FURVVVHFWLRQRIWDFWLFDOO\UHOHYDQWFRQGLWLRQV7KHWHVWFRQGLWLRQVVSHFLILHGWKHLQLWLDO JHRPHWULHV ; DQGRSSRQHQWVSHHGVDOWLWXGHVDQGUDQJHV

&ODVVLILHU6\QWD[

7KHF RPEDWO RJLF IRU WKH JHQHWLF OHDUQLQJ V\VWHP DLUFUDIWL V VWRUHG LQ WKH IRUP RI ,)7+(1 UXOHV FODVVLILHUV  7KH ,) VHFWLRQ RI WKH UXOH LV D FRQGLWLRQ WKDWL V PDWFKHG DJDLQVWW KHF XUUHQW FRPEDW VWDWH

VWDWHLVHYDOXDWHGHYHU\WKRIDVHFRQG 7KHFRQGLWLRQVDUHGHVLJQHGWRPDWFKDELQDU\HQFRGLQJRI WKHIHDWXUHVRIWKHVWDWHRIWKHFRPEDWVLWXDWLRQVKRZQLQ7DEOH &RQGLWLRQV1XPEHURI%LWV (QFRGLQJ 2ZQVKLS$VSHFWGHJUHHV 7KUHHELWV

%LQVUHSUHVHQWHGLQELQDU\     /HVVWKDQ WR WR WR    

(QFRGLQJ 5DQJHIHHW 7ZR%LWV

 /HVVWKDQ 

2SSRQHQW$VSHFWGHJUHHV 7KUHHELWV

/HVVWKDQ 

$LUVSHHGNQRWV 7ZR%LWV

/HVVWKDQ 

'HOWD$LUVSHHGNQRWV 7ZR%LWV

/HVVWKDQ N

$OWLWXGHIHHW 7ZR%LWV

/HVVWKDQ 

'HOWD$OWLWXGHIHHW 7ZR%LWV

&OLPE$QJOHGHJUHHV 7ZR%LWV 2SSRQHQW &OLPE GHJUHHV 7ZR%LWV 7RWDOELWV

/HVVWKDQ 

$QJOH

/HVVWKDQ  /HVVWKDQ 

WR 

WR 

N WR N N WRN

N WR N NWR N

  WR   WR   WR

   WR

  WR   WR  WR 

  WR 

WR 

 *UHDWHU WKDQ  *UHDWHU WKDQ  *UHDWHU WKDQ  *UHDWHU WKDQ N *UHDWHU WKDQN

  WR   WR 

  WR   WR 

  WR   WR 

  WR   WR 

*UHDWHU WKDQ *UHDWHU WKDQ

7DEOH(QFRGLQJRIILJKWHUHQYLURQPHQWVWDWHLQWRELWV

,QWKLVOLVWWKHQXPEHURIELQVUHIHUVWRWKHUHJLRQVWKDWWKHUXOHFRQVLGHUVDVGLVWLQFWVWDWHVHQFRGHGLQ ELQDU\ )RULQVWDQFHUDQJHLVUHSUHVHQWHGWRWKHUXOHVDVKDYLQJG LVWLQFWYDOXHVKLJKPHGLXP KLJK PHGLXP ORZ DQG ORZ  7KHVH YDOXHV DUH GLVWLQFWL QWHUYDOV RI DOO SRVVLEOH UDQJH YDOXHV 7R DOORZ IRU JHQHUDOL]HGUXOHVWKHELQVDUHUHSUHVHQWHGDVVWULQJVRIELWVZLWKWKHQXPEHUVRI ELWV GHVFULEHG DERYH 7KH,)FRQGLWLRQRIWKHUXOHVWKHPVHOYHVDUHIURPWKHDOSKDEHW^`ZKHUHWKHFKDUDFWHUPDWFKHV HLWKHURULQWKHFRPEDWVWDWH 7KH7+(1VHFWLRQRIWKHUXOHVWKHDFWLRQV LQFOXGHVWKHHOHPHQWVVKRZQLQ7DEOH &RPPDQGV$FWLRQV (QFRGLQJ 5HODWLYH%DQN$QJOHGHJUHHV 7KUHH%LWV $2$GHJUHHV 7KUHH%LWV (QFRGLQJ 6SHHGNQRWV 7ZR%LWV 7RWDOELWV

$FWLRQ9DOXHVVXJJHVWHGWR$$63(0           

 



 



 



 



 



 



 



7DEOHELWHQFRGLQJRIWKHDFWLRQVVXJJHVWHGE\WKHJHQHWLFOHDUQLQJV\VWHP V UXOHV

:KHQDUXOHLVPDWFKHGDQGVHOHFWHGWRDFWWKHDFWLRQLVVXEPLWWHGWR$$63(0DVDGHVLUHGYDOXHIRU

WKHVH WKUHH YDULDEOHV ,WL V LPSRUWDQWW R QRWH WKDW $$63(0GRHV QRW GHOLYHU WKHVHH [DFW YDOXHV IRU WKH DLUFUDIW,QVWHDGLWUHWXUQVDQ DHURG\QDPLFDOO\ FRUUHFW UHVSRQVHF RPPDQGLQJ WKH DLUFUDIWW RZDUGV WKHVH GHVLUHGYDOXHV:HEHOLHYHWKLVRQHRIWKHUHDVRQVIRUWKHVXFFHVVRIRXUJHQHWLFVEDVHGPDFKLQHOHDUQLQJ DSSURDFK

2QHVLGHG/HDUQLQJ5HVXOWV

7KHUHVXOWVRIRXUSUHYLRXVRQHVLGHGOHDUQLQJ ZRUN>@ZHUHH[DPLQHGLQYDULRXVDQDO\VLVDQGUHSOD\ IRUPDWVE\SURMHFWVWDIIGXULQJWKHHIIRUW(DFKWHVWFDVHZDVUHYLHZHGWRHYDOXDWHWKHHIIHFWLYHQHVVRIWKH JHQHWLFVEDVHGPDFKLQHOHDUQLQJSURFHVV 7KHEHVWREVHUYHG;WDFWLFVIURPWKHJHQHWLFOHDUQLQJV\VWHPZHUHUHYLHZHGE\1$6$DQGFRQWUDFWRU SLORWVWRSURYLGHDQDVVHVVPHQWRIWDFWLFDOXWLOLW\DQGRSHUDWLRQDOUHDOLVP x

x

x

;7DFWLFV5HDOLVPDQG8WLOLW\ 3LORWVJHQHUDOO\DJUHHGZLWK WKHF KRLFH DQG VHTXHQFH RI ; WDFWLFVIURPWKHJHQHWLFOHDUQLQJV\VWHPV0DQ\RIWKHPDQHXYHUVFORVHO\UHVHPEOHGWKRVHHPSOR\HG LQWKH0XOWL6\VWHP,QWHJUDWHG&RQWUROV0X6,& 6WXG\>@ZKLFKLQYROYHGDPDQQHGVLPXODWLRQRI DPRGLILHG)DLUFUDIWZLWKSLWFK\DZWKUXVWYHFWRULQJDQG;IOLJKWWHVWLQJ7KHXVHRIDKLJK DOSKD RXWRISODQHF RQLQJ PDQHXYHU IROORZHGE \ D IODW GHVFHQGLQJ WXUQ UHSOLFDWHG WKH -WXUQ DQG KHOLFRSWHUJXQDWWDFNWDFWLFVXVHGLQWKH0X6,&DQG3LQEDOOSURMHFWV7KHREMHFWLYHRIPDLQWDLQLQJD ILULQJSRVLWLRQLQVLGHWKHRSSRQHQW VWXUQUDGLXVZDVXVXDOO\DFKLHYHGLQWKHJHQHWLFOHDUQLQJV\VWHP UXQV $LUFUDIW &RQILJXUDWLRQV $ FRQFHUQ H[SUHVVHGE \ SLORW UHYLHZHUV ZDV WKH ZHDSRQ V\VWHP FRQILJXUDWLRQRIERWKWKH;DQGRSSRQHQWDLUFUDIW)XWXUHFRPEDWVFHQDULRVRIWHQSURMHFWWKHXVH RI KLJK RIIERUHVLJKW PLVVLOHV DQGK HOPHWPRXQWHG VLJKWV LQ FORVHLQ FRPEDW 7KH LPSDFW RI WKHVH V\VWHPVRQ367DLUFUDIWHPSOR\PHQWDQGWDFWLFVLVD VWURQJO\ GHEDWHG LVVXH LQ WKH RSHUDWLRQDO DQG HQJLQHHULQJ FRPPXQLW\ LQYROYHG LQ ILJKWHU DLUFUDIWW HFKQRORJLHV :KLOH RXU SUHYLRXVV WXG\ >@ IRFXVHG RQ H[SORULQJ WKHH IIHFWLYHQHVV RI 367 WDFWLFV LQ D FRQYHQWLRQDO ZHDSRQ HQYLURQPHQW DGYDQFHG PLVVLOHV DQG VHQVRU V\VWHPV DUH D ORJLFDO H[WHQVLRQ RI WKHF XUUHQW VWDWH RI DLU FRPEDW WHFKQRORJ\7KLVZLOOEHDVXEMHFWRIIXWXUHSURMHFWHIIRUWV

6XLWDELOLW\R I 2SSRQHQW 7DFWLFV 7KH LQDELOLW\ RI WKH $$63(0 RSSRQHQWW DFWLFV WR DGDSWW R WKH JHQHWLFOHDUQLQJV\VWHPPDQHXYHUVUHFHLYHGWKHVWURQJHVWFULWLFLVPIURPWKHSLORWUHYLHZHUV$FKLHI REMHFWLRQ WR WKH RSSRQHQW DLUFUDIWW DFWLFV UHODWHG WR WKH VWDWLF QDWXUH RI WKH SURJUDPPHG $$63(0 ORJLF7KHJHQHWLFOHDUQLQJV\VWHPZDVDOORZHGWRPRGLI\WKHHQWLUHFRQWHQWRILWVWDFWLFVVHWZKLOH WKHRSSRQHQWDLUFUDIWZDVOLPLWHGWRWKHWDFWLFVVSHFLILHGLQWKH$$63(0LQSXWILOHDWWKHVWDUWRIWKH UXQ 7KLV UHVWULFWLRQK DG DWO HDVWW ZR SRVVLEOHF RQVHTXHQFHV )LUVW WKH JHQHWLFVEDVHG PDFKLQH OHDUQLQJS URFHVV PD\ KDYH EHHQ DUWLILFLDOO\ FRQVWUDLQHGGX H WR WKH QDUURZ VFRSH RI LWV WDFWLFDO HQYLURQPHQW6HFRQG WKHUHVXOWLQJWDFWLFVPD\KDYHEHHQRSWLPDORQO\DJDLQVWWKHUHODWLYHO\VPDOO VHWRIRSSRQHQWPDQHXYHUV2QHSRVVLEOHDSSURDFKIRUH[SDQGLQJWKHRSSRQHQWWDFWLFVHWZRXOGEH WRVLPXOWDQHRXVO\SURFHVVERWKWKH;DQGRSSRQHQWDLUFUDIWPDQHXYHUVXVLQJWKHJHQHWLFOHDUQLQJ V\VWHP7KDWDSSURDFKLVH[SORUHGLQWKLVSDSHU

7ZRVLGHG/HDUQLQJ

7KH SULPDU\ REMHFWLYH RI WKH ZRUNS UHVHQWHGK HUH ZDV WR GHPRQVWUDWH WKH IHDVLELOLW\ RI D WZRVLGHG OHDUQLQJ DOJRULWKP ZLWKLQ D RQHYHUVXVRQH DLU HQJDJHPHQW VFHQDULR ,WL V LPSRUWDQWW R QRWH WKDW ZKHQ ERWKSOD\HUVDGDSWQHZFRPSOH[LWLHVDUHLQWURGXFHGLQWRWKHV\VWHP VG\QDPLFVDQGWKHLQWHUSUHWDWLRQRI UHVXOWV7KHVHFRPSOH[LWLHVDUHEHVWVHHQIURPWKHSHUVSHFWLYHRIH[WDQWOLWHUDWXUHRQWZRSOD\HUJDPHV

,Q PDQ\ ZD\V WKH RQHYHUVXVRQH ILJKWHU FRPEDW VFHQDULR LV OLNH D FRQWLQXRXV YHUVLRQ RI WKH LWHUDWHG SULVRQHUV GLOHPPD ,3' >@>@,QWKH,3'WKHJDPHWDNHVSODFHLQGLVWLQFWURXQGVLQ ZKLFK HDFK SOD\HU FDQ WDNH RQH RI WZR DFWLRQV FRRSHUDWH RU GHIHFW ,I ERWKS OD\HUV FRRSHUDWH ERWKS OD\HUV JHW D PRGHUDWHUHZDUG,IRQHSOD\HUGHIHFWVZKLOHWKHRWKHUFRRSHUDWHVWKHIRUPHUJHWVDODUJHUHZDUGDQGWKH ODWWHUJHWVDODUJHSXQLVKPHQWQHJDWLYHUHZDUG ,IERWKGHIHFWERWKJHWDQHYHQODUJHUSXQLVKPHQW 7KLVLVVLPLODUWRWKHVLWXDWLRQLQWKHRQHYHUVXVRQHILJKWHUFRPEDWVFHQDULR(DFKDLUFUDIWKDVDFKRLFH RI DWWDFNLQJ WKH RWKHU DQDORJRXV WR GHIHFWLQJ  RU HYDGLQJ WKH RWKHU DQDORJRXV WR FRRSHUDWLQJ  7KLV DQDORJ\ LV XVHIXO VLQFH ,3' KDV EHHQ VWXGLHG D JUHDW GHDO ERWK ZLWK VWDWLF DQG DGDSWLYH SOD\HUV +RZHYHULWLVLPSRUWDQWWRQRWHWKDWWKHFXUUHQWZRUNPD\EHRQHRIWKHILUVWH[SORUDWLRQVRIDUHDOZRUOG FRQWLQXRXVWLPHDQDORJRI,3'ZLWKWZRDGDSWLYHSOD\HUV

7KH ,3' OLWHUDWXUH IRU FRDGDSWLYH SOD\HUVV KRZV XVV HYHUDOL QWHUHVWLQJE HKDYLRUV (DFK RI WKHVH LV DQ DUWLIDFWRIWKH UHGTXHHQHIIHFW VRFDOOHGEHFDXVHWKHUHGTXHHQLQ$OLFHLQ:RQGHUODQGVWDWHVWKDWLQKHU

ZRUOG \RX PXVW NHHS UXQQLQJ MXVWW R VWDQG VWLOO >@ ,Q DQ DQDORJRXV ZD\ WKH SHUIRUPDQFH RI HDFK SOD\HU LQ WKH WZRVLGHG OHDUQLQJS UREOHP LV UHODWLYH WR WKDW RI LWV RSSRQHQW ,Q RWKHU ZRUGV ZKHQ RQH SOD\HU DGDSWV DQG WKH RWKHU XVHV D VWDWLF VWUDWHJ\ DV LQ RXU SUHYLRXV ZRUN  WKH SHUIRUPDQFH RI WKH DGDSWLYHSOD\HULVDEVROXWHZLWKUHVSHFWW R LWV RSSRQHQW +RZHYHU ZKHQE RWKS OD\HUV DUH DGDSWLYH WKH SHUIRUPDQFHFHDVHVWRKDYHDQDEVROXWHPHDQLQJ,QVWHDGLWVPHDQLQJLVRQO\UHODWLYHWRWKHVWDWHRILWV FXUUHQWRSSRQHQW7KLVLVDQLPSRUWDQWHIIHFWWKDWPXVWEHFRQVLGHUHGLQWKHLQWHUSUHWDWLRQRIRXUFXUUHQW UHVXOWV $OVR EHFDXVH RI WKH UHGTX HHQ HIIHFW WKH G\QDPLF V\VWHP FUHDWHGE \ WZR SOD\HUV KDVV HYHUDO SRVVLEOH DWWUDFWRUV7KHVHLQFOXGH x x x x

)L[HG 3RLQWV :KHUHH DFKS OD\HU DGDSWV D VWDWLF VWUDWHJ\ DV LV ZLOOLQJ WR FRSH ZLWK D SRVVLEO\ LQIHULRU VWHDG\VWDWH 3HULRGLF%HKDYLRU :KHUHH DFKS OD\HU F\FOHV WKURXJK D UDQJH RI VWUDWHJLHV VRPHWLPHV ORVLQJ DQG VRPHWLPHVZLQQLQJ

&KDRWLF %HKDYLRU :KHUHH DFKS OD\HU YLVLWV D UDQGRP VHULHV RI VWUDWHJLHV ZLWKQ R HYHQWXDO HVFDODWLRQRIWDFWLFV $UPV5DFHV :KHUHHDFKSOD\HUFRQWLQXRXVO\UDPSVXSWKHVRSKLVWLFDWLRQRILWVVWUDWHJLHV

7KH ODWWHU LV FOHDUO\ WKH EHKDYLRU ZH ZDQW RXU VLPXODWLRQV WR HQFRXUDJH 2XU FXUUHQW UHVXOWV KDYH TXDOLWDWLYHO\ VKRZQSURPLVHLQWKLVDUHDLHZHKDYHVHHQDQHVFDODWLRQRIVWUDWHJLHVEHWZHHQWKHWZR DLUFUDIW 

([SHULPHQW6FHQDULRV

7KHPDLQREMHFWLYHRIWKHH[SHULPHQWVSUHVHQWHGKHUHZDVWRH[SORUHDQGGLVFRYHUQHZPDQHXYHUVZKHQ ERWKVLGHVDUHDOORZHGWROHDUQDGDSWLYHVWUDWHJLHVZKDWZHWHUPWZRVLGHGOHDUQLQJ7KUHHYHUVLRQVRI WKH WZRVLGHG OHDUQLQJ DOJRULWKP ZHUH GHYHORSHG ,Q HDFK FDVH D UXQ FRQVLVWV RI  FRPEDW HQJDJHPHQWV 





$OWHUQDWH IUHH]H OHDUQLQJ $/7  ,Q HDFK UXQ RQH VLGH LV OHDUQLQJ ZKLOH WKH RWKHU LV IUR]HQ QRW DOWHULQJLWVUXOHEDVH 7KHIUR]HQSOD\HUXVHVWKHSRSXODWLRQUXOHV REWDLQHGIURPWKHSUHYLRXVUXQ LQ ZKLFK LW ZDV OHDUQLQJ 7KH OHDUQLQJS OD\HU VWDUWV HDFKQ HZ UXQ ZLWK D UDQGRP VHW RI UXOHV ,Q RWKHUZRUGVUXOHVDUHOHDUQHG¦IURPVFUDWFK§DJDLQVWWKHUXOHVOHDUQHGLQWKHODVWF\FOHE\WKHRWKHU SOD\HU

$OWHUQDWH IUHH]H OHDUQLQJ ZLWK PHPRU\ 0(0  7KLV OHDUQLQJ VFKHPHF DQE H YLHZHG DV DQ H[WHQGHGYHUVLRQRIWKH$/7OHDUQLQJ$WWKHHQGRIHDFKUXQWKHUHVXOWVRIWKHHQJDJHPHQWVDUH VFDQQHG WRR EWDLQ WKH KLJKHVW PHDVXUH RI HIIHFWLYHQHVV 7KH UXOHV IURP WKH KLJKHVW VFRULQJ HQJDJHPHQWDUHXVHGIRUWKHIUR]HQVWUDWHJ\LQWKHQH[WUXQ)XUWKHUPRUHWKHVHUXOHVDUHPHPRUL]HG DQG DUH DGGHG WR WKH SRSXODWLRQ LQ WKH XSFRPLQJ OHDUQLQJ VHTXHQFH UXQV 7KXV WKH V\VWHP KDV PHPRU\RILWVSUHYLRXVO\OHDUQHGEHKDYLRU 3DUDOOHOOHDUQLQJ3$5  *HQHWLFOHDUQLQJJRHVRQFRQWLQXRXVO\IRUERWKSOD\HUV

(DFK RI WKH WKUHH WZRVLGHG DOJRULWKPV ZHUH WHVWHG DQG HYDOXDWHG RQ DOO ILYH 3LQEDOO FDVHV ,Q DOO H[SHULPHQWVDSRSXODWLRQRIPDQHXYHUVZDVJHQHUDWHGLQDQLWHUDWLYHSURFHVVEHJLQQLQJZLWKDQLQLWLDO SRSXODWLRQRIUDQGRPFODVVLILHUV7KHPDQHXYHUSRSXODWLRQZDVVXFFHVVLYHO\LPSURYHGXVLQJ$$63(0 DLUFRPEDWHQJDJHPHQWVWRHYDOXDWHWKHPDQHXYHU1RWHWKDWLQHDFKFDVHWKH%OXHSOD\HULVDQ; VKRZQ DV EODFN LQ WKH JUD\VFDOH ILJXUHV  DQG WKH 5HG SOD\HU LV DQ ) VKRZQ DV JUD\ LQ WKH JUD\VFDOHILJXUHV 

*HQHWLF$OJRULWKP3DUDPHWHUV

,QHDFKVLPXODWLRQWKH*$HPSOR\HGWRXUQDPHQWVHOHFWLRQZLWKDWRXUQDPHQWVL]HRI7KHWRXUQDPHQW VHOHFWLRQSURFHGXUHZRUNVDVIROORZV  (LJKWUXOHVIURPWKHSRSXODWLRQDUHVHOHFWHGDWUDQGRP

 )URPWKHVHHLJKWWKHUXOHZLWKWKHKLJKHVWILWQHVVLVVHOHFWHGWRSDUWLFLSDWHLQDPDWLQJSRRO

 7KHSURFHVVLVUHSHDWHGXQWLOWKHPDWLQJSRROLVRIWKHVL]HUHTXLUHGWRJHQHUDWHWKHGHVLUHGQXPEHURI LQGLYLGXDOV

7KHSRSXODWLRQVZHUHRYHUODSSLQJZLWKDJHQHUDWLRQJDSRI7KDWLVWKHPDWLQJSRROZDVRIKDOIWKH VL]HRIWKHUXOHSRSXODWLRQ$SRSXODWLRQVL]HRIZDVHPSOR\HG7KHKLJKVHOHFWLYHSUHVVXUHRIWKLV

VFKHPH VHHPHG WRRY HUFRPH WKH QRLV\ QDWXUH RI WKH WZRVLGHG OHDUQLQJS UREOHP ZLWKRXW SUHPDWXUH FRQYHUJHQFH

0DWLQJSDLUVZHUHVHOHFWHGDWUDQGRPIURPWKHPDWLQJSRRODQGHDFKSDLUZDVVXEMHFWHGWRVLQJOHSRLQW FURVVRYHUZLWKDFURVVRYHUSUREDELOLW\RIWRJHQHUDWHWZRFKLOGUHQUXOHV0XWDWLRQZDVDSSOLHGWR WKHVHQHZUXOHVZLWKDSUREDELOLW\RISHUELWSRVLWLRQ:KHQDFRQGLWLRQELWLVVHOHFWHGIRUPXWDWLRQ LWLVVHWWRRQHRIWKHWKUHHSRVVLEOHFKDUDFWHUYDOXHVLVVHOHFWHGRU ZLWKHTXDOSUREDELOLW\1RWH WKDWWKLV DFWXDOO\\ LHOGV D HIIHFWLYH PXWDWLRQS UREDELOLW\ RI    &KLOGUHQ UXOHV UHSODFHG UDQGRPO\VHOHFWHGUXOHVLQWKHSRSXODWLRQ

0HDVXUHVIRU7HVWVDQG(YDOXDWLRQ

2QH UHDVRQDEOH PHDVXUH RI WKH UHVXOWV LVV LPSO\ WKH PHDVXUH RI HIIHFWLYHQHVV WKDW ZDVV HOHFWHG IRU WKH *$ ILWQHVV IXQFWLRQ 7KLV PHDVXUH ZDV EDVHG RQ DYHUDJH DQJXODU DGYDQWDJH RSSRQHQWW DUJHW DVSHFW DQJOHPLQXVRZQVKLSWDUJHWDVSHFWDQJHO SOXVWKHDGGLWLRQDOVFRUHQRWHGSUHYLRXVO\ )RU D EHWWHU IRFXV LQ DQDO\]LQJ WKH UHVXOWV ZHH [WUDFWHG WKH ¦EHVW§ UXQ IURP HDFKS URFHVVLQJ F\FOH LQ WHUPVRIWKHILWQHVVIXQFWLRQ7KHVHUHVXOWVDUHH[DPLQHGLQWKHIROORZLQJVHFWLRQV7KHH[WUDFWLRQRIWKH EHVWUXQZDVPDGHIRUERWKVLGHV%OXH%HVWDQG5HG%HVW +RZHYHUDVQRWHGDERYHWKHVHEHVWUXQVDUH RQO\¦EHVW§LQDUHODWLYHVHQVHJLYHQWKHUHGTXHHQHIIHFW:HZLOOGLVFXVV WKLV DVSHFW RI WKH UHVXOWV LQ PRUHGHWDLOLQWKHSDSHU VFRQFOXVLRQV

7DUJHW$VSHFW$QJOH9HUVXV7LPH+LVWRULHV HSLA (Blue)

SSLA (Blue)

200

200

150

150

100

100

50

50

0

0

0

10

20

30

B2BH (Blue)

100 50

0

HSLA (Red)

10

20

30

0

0

SSLA (Red)

150

100

50

50

0

0

10

20

30

OFF (Blue)

200

200

150

150

100

100

50

50

0

0

B2BH (Red)

150

100

DEF (Blue)

150

10

20

30

0

0

DEF (Red)

10

20

30

OFF (Red)

200

200

200

150

150

150

100

100

100

50

50

50

0

0

0

)LJXUH$QJXODUDGYDQWDJHYHUVXVWLPHIRUDOOILYHEDVHOLQHFDVHV5HG) VROLGOLQHDQG%OXH; GRWWHGOLQH7KHILUVWURZVKRZVWKHEHVWHQJDJHPHQW UHVXOWVIRU%OXHDQGWKHVHFRQGURZVKRZVWKHEHVWHQJDJHPHQWUHVXOWVIRU5HG 0

10

20

30

0

10

20

30

0

10

20

30

0

10

20

30

0

10

20

)LJXUHFRQWDLQVWLPHKLVWRU\SORWVRIWDUJHWDVSHFWDQJOHVIRUWKHEHVWUHVXOWRIHDFKFDVHLQWKHEDVHOLQH PDWUL[ IRU WKH $/7 OHDUQLQJ V\VWHP 7KH JUDSK LOOXVWUDWHV WKH WLPHV RI JUHDWHVW DQG OHDVW DGYDQWDJH RI HDFK VLGH RYHU LWV RSSRQHQW GXULQJ WKHF RXUVH RI WKHH QWLUHH QJDJHPHQW 7KH ILUVW URZ VKRZV WKH %HVW %OXH; UHVXOWVLQGLFDWLQJWKDWWKH;KDVWKHDGYDQWDJHGXULQJWKHHQWLUHHQJDJHPHQWDQGIRUDOO ILYHF DVHV 7KLV FOHDUO\ GHPRQVWUDWHV WKH VXFFHVV RI WKLV OHDUQLQJ V\VWHP 7KH '() FDVH ZDUUDQWV D VSHFLDODWWHQWLRQDVWKH;LVVWDUWLQJIURPDGLVDGYDQWDJHGSRVLWLRQ1RWHWKDWLQOHVVWKDQ VHFRQGV WKH;JDLQVDQDGYDQWDJHWKDWODVWWKURXJKRXWWKHUHVWRIWKHHQJDJHPHQW,QRXURQHVLGHGUHVXOWV>@ WKLV UHJDLQLQJ RI WKH DGYDQWDJH ZDV QRW DFKLHYHG 7KLVV KRZV WKH EHQHILWV RI WZRVLGHG OHDUQLQJ LQ GLVFRYHULQJQHZ PDQHXYHUV WKDWW KH RQHVLGHG OHDUQLQJ ZDV XQDEOH WR GLVFRYHU 7KH WDFWLFDO PDQHXYHU IRUWKLVFDVHLVGLVFXVVHGLQDODWHUVHFWLRQ

,QWKH+6/$+6+3DQG'() FDVHV WKH UHGS OD\HU PDQDJHV WR PDLQWDLQ DQ DGYDQWDJH RYHU WKHH QWLUH HQJDJHPHQW7KH2))EHVW5HGFDVHLVDVLPLODUFDVHWR'()%HVW%OXHDQGUHTXLUHVDVSHFLDODWWHQWLRQ +HUHWKHUHGSOD\HULVVWDUWLQJIURPDGLVDGYDQWDJHGSRVLWLRQEXWZDVDEOHWRJDLQDQDGYDQWDJHLQDERXW VHFRQGV1RWHKRZHYHUWKDWLWWRRNORQJHUIRUWKHUHGSOD\HUWRJDLQDQDGYDQWDJHZKHQFRPSDUHGWR WKH%OXH%HVW'()UHVXOW7KLVLVPRVWOLNHO\GXHWRWKHGLIIHUHQFHLQWKHDLUFUDIWSHUIRUPDQFHVLQFHWKH UHGSOD\HU) LVOHVVDJLOHWKDQWKHEOXHSOD\HU;  7KHDVSHFWDQJOHJUDSKDOVRLQGLFDWHVWKDWWKH66/$5HG LVDVSHFLDOFDVHDVWKHUHGSOD\HUORVHVWKH DGYDQWDJHDIWHUURXJKO\VHFRQGV7KLVLVGXHWRDJLOHPDQHXYHUSHUIRUPHGE\WKHEOXHSOD\HU; 

30

:HEHOLHYHWKHGLIIHUHQFHLQDLUFUDIWSHUIRUPDQFHDOVRSOD\VDPDMRUIDFWRULQWKLVUHVXOW

$*,/(983ORWV

7KH$*,/(98YLVXDOL]DWLRQSURJUDPZDVXVHGIRULQLWLDODVVHVVPHQWV RI WKH WDFWLFV ,Q WKH IROORZLQJ VHFWLRQVZHH[DPLQHOHDUQHGWDFWLFVLQ$*,/(98VWLOOSORWV1RWHWKDWLQWKHVHSORWVDLUFUDIWVL]HVDUH H[DJJHUDWHGIRUFODULW\$OVRRQHLPDJHRIHDFKDLUFUDIWLVGLVSOD\HGHYHU\VHFRQGVHYHQWKRXJKWKH VLPXODWLRQDOORZVIRUDFRQWURODFWLRQHYHU\ WK RIDVHFRQG

)LJXUH'()%OXH$/77RS9LHZ

)LJXUHLOOXVWUDWHVWKH'()%OXHUHVXOW7KHJHQHWLFDOO\OHDUQHGUXOHVFRPPDQG WKH ; WR H[HFXWH D KLJK VSHHG HYDVLRQ WR D KLJKHU DOWLWXGH IROORZHGE \ D VKDUS WXUQ WKDWO HG WR D SXUVXLW IURP D KLJKHU DOWLWXGH,WWRRNOHVVWKDQ VHFRQGVIRUWKH;WRDFKLHYHWKHSXUVXLQJSRVLWLRQ,WPDLQWDLQHGDQRVH SRLQWLQJDWWLWXGHWRZDUGWKHWDUJHWWKURXJKRXWWKHUHVWRIWKHHQJDJHPHQW7KLVLVD+HUEVWW\SHPDQHXYHU ZLWK WKH ; FOLPELQJ DQG URWDWLQJ DERXWW KH YHORFLW\ YHFWRU VXFFHVVIXOO\ UHYHUVLQJ 7KHH [DPSOH LOOXVWUDWHV WKH DELOLW\ RI WKLV WZRVLGHG OHDUQLQJ VFKHPH WR GLVFRYHU LQQRYDWLYH PDQHXYHUV LQG LIIHUHQW WDFWLFDOVLWXDWLRQVE\H[SORLWLQJWKHPDQHXYHULQJSRWHQWLDORIWKH;

)LJXUH2))5HG$/77RS9LHZ

$VLPLODUUHVXOWZDVGLVFRYHUHGE\WKH)LQWKH2))%HVW5HGUHVXOWVKRZQLQ)LJXUH+HUHWKH) LVDOVRH[HFXWLQJD+HUEVWW\SHPDQHXYHUEXWWKHLQIHULRULW\RIWKLVWRWKRVHRIWKH;WKDWZHKDYH REVHUYHGLQRWKHUH[SHULPHQWVLVDSSDUHQW7KLVLV RI FRXUVH GXH WR WKH GLIIHUHQFH LQ WKH DJLOLW\ RI WKH WZRDLUFUDIW

&RPSDULVRQZLWK1R/HDUQLQJDQG2QH6LGHG/HDUQLQJ

,QWKLVVHFWLRQZHFRPSDUHWKHUHVXOWVREWDLQHGLQWKHWZRVLGHG$/7VFKHPHZLWKWKRVHRIQROHDUQLQJ

DQGRQHVLGHGOHDUQLQJ:HFKRVHWKH66/$VWDUWLQJFRQGLWLRQIRUWKLVFRPSDULVRQ

)LJXUH66/$1R/HDUQLQJ7RS9LHZ

)LJXUH  VKRZV WKH $*,/(98 SORW IRU VLPXODWHG HQJDJHPHQW ZKHUH ERWK DLUFUDIW IROORZHG VWDQGDUG $$63(0 FRPEDW PDQHXYHU ORJLF %OXH JDLQV D WRWDO SRVLWLYH DGYDQWDJH XVLQJ D KHOLFRSWHU JXQ PDQHXYHUZLWKDPHDVXUHRIHIIHFWLYHQHVVRI7KLVJDLQLVD UHVXOW RI WKH GLIIHUHQFH LQS HUIRUPDQFH EHWZHHQWKHWZRDLUFUDIW

)LJXUH66/$%OXH2QHVLGHG /HDUQLQJ7RS9LHZ

)LJXUHVKRZVWKH$*,/(98HQJDJHPHQWSORWREWDLQHGZKHQWKHEOXHSOD\HULVOHDUQLQJZKLOHWKHUHG SOD\HUIROORZVWKHVWDQGDUG$$63(0ORJLF7KHEOXHSOD\HUDJDLQZLQVEXWWKLVWLPHZLWKDPHDVXUHRI HIIHFWLYHQHVV RI  7KH LPSURYHG HIIHFWLYHQHVV LV D UHVXOW RI WKH OHDUQLQJS URFHVV 7KH SORW VKRZV WKH EOXHSOD\HUH[HFXWLQJDKLJKDOSKDFRQLQJPDQHXYHUIROORZHGE\DUHILQHGKHOLFRSWHUJXQ DWWDFN LQVLGH WKHUDGLXVRIWKH)RSSRQHQWZKLOHPDLQWDLQLQJDQRVHSRLQWLQJDWWLWXGHWRZDUGWKHWDUJHW

)LJXUH66/$%HVW%OXH$/7

)LJXUH  VKRZV WKHH QJDJHPHQW IRU WKH WZRVLGHG $/7 %HVW %OXH UHVXOW 7KH EOXH WDFWLFV DUH DV H[SHFWHGEHWWHUWKDQWKRVHGLVFRYHUHGLQWKHRQHVLGHGFDVH7KHSORWVKRZVWKDWDWWKHEHJLQQLQJRIWKH HQJDJHPHQWWKHEOXHSOD\HUSHUIRUPVDFRQLQJPDQHXYHUWKDWLVVLPLODUWRWKHRQHVLGHGFDVH$IWHUZDUGV LW SHUIRUPV D URWDWLRQ DERXWW KH YHORFLW\ YHFWRU IROORZHGE \ D GHVFHQGLQJ DOWLWXGH WKDW SODFHG LW LQ D SXUVXLW SRVLWLRQ 7KLV DOORZHG WKH EOXH SOD\HU WR PDLQWDLQ D QRVHSRLQWLQJ DWWLWXGH WRZDUG WKH ) WKURXJKRXWW KH UHPDLQGHU RI WKHH QJDJHPHQW 7KH SORW DOVR GHPRQVWUDWHV XQVXFFHVVIXO HYDVLRQ PDQHXYHUVSHUIRUPHGE\WKHUHGS OD\HU $OWKRXJKXQ VXFFHVVIXO WKHVH PDQHXYHUV LOOXVWUDWH OHDUQLQJ RQ WKHSDUWRIWKHUHGSOD\HUVLQFHWKH\LQYROYHVKDUSWXUQVDURXQGLWVYHORFLW\YHFWRU

)LJXUH66/$5HG$/77RS9LHZ

7KH LQWHUSOD\ RI WKH UHG DQGE OXH SOD\HUV DGYDQFHV EH\RQG VWDQGDUG FRPEDW PDQHXYHU ORJLF DUH LOOXVWUDWHGE\WKLVUHVXOWDQGE\WKH66/$%HVW5HGUHVXOWVKRZQLQ)LJXUH  +HUH WKH ) IDLOV WR KDYHDQDGYDQWDJHIRUPRUHWKDQWKHODVWKDOIRIWKHHQJDJHPHQWGXH WR DJLOH PDQHXYHUV SHUIRUPHGE \ WKH ; 7KH ; H[HFXWHV D QRVH GLYH PDQHXYHU IROORZHGE \ +HUEVWW\SH PDQHXYHU DOORZLQJ LW WR JDLQDQDGYDQWDJH $OWKRXJK5HGGRHVQRWJDLQDGYDQWDJHLWLVLQWHUHVWLQJWRQRWHLWVHYDVLYHEHKDYLRU LQUHDFWLRQWREOXH©VPDQHXYHU1RWHWKDWWKHUHVXOWVKRZQLQ)LJXUHRFFXUVLQDQHDUOLHUHQJDJHPHQW WKDWWKHUHVXOWLQ)LJXUH,WLVOLNHO\WKDWOHDUQLQJIURPWKHUHGSOD\HU VHYDVLRQ)LJXUH UHVXOWHGLQ WKH LPSURYHGE HKDYLRU VHHQ LQ WKH %HVW %OXH UHVXOW )LJXUH   7KLV LOOXVWUDWHV WKH DGYDQWDJH RI WZR VLGHGOHDUQLQJ

&RPSDULVRQRIWKH7KUHH/HDUQLQJ6\VWHPV$/73$5DQG0(0

7DEOHVKRZVWKHPHDVXUHRIHIIHFWLYHQHVVIRUWKHWKUHHOHDUQLQJV\VWHPV$/73$50(06RPHRI WKHILJXUHVDUHUHODWLYHO\HDV\WRLQWHUSUHWZKLOHRWKHUVUHTXLUHIXUWKHUH[DPLQDWLRQ

$/7

+6/$ 66/$ +6+3 '() 2))

%OXH     

5HG     

3$5

%OXH     

5HG     

0(0

%OXH     

5HG     

7DEOH0HDVXUHRIHIIHFWLYHQHVVYDOXHVIRUDOOWKUHHOHDUQLQJV\VWHPV$/7 3$5$1'0(0

&RQVLGHU WKH PHDVXUH RI HIIHFWLYHQHVV ILJXUHV IRU WKH 3$5 FDVH %HDU LQ PLQG WKDWW KH QDWXUH RI WKLV VFKHPH SODFHV WKH WZR DLUFUDIW RQ HTXDO IRRWLQJ ZLWK UHJDUG WR OHDUQLQJ 7KLV LV UHIOHFWHG LQ WKH WDEOH ZLWKERWKVLGHVVKRZLQJDERXWWKHVDPHSHUIRUPDQFH5HFDOOWKDW2))%OXHLVVLPLODUWR'()5HG 7KHJUHDWHVWDGYDQWDJHIRU HLWKHU SOD\HU RFFXUV LQ WKH 2)) VWDUWLQJS RVLWLRQ ZKLOH WKH OHDVW DGYDQWDJH RFFXUVLQWKH'()VWDUWLQJSRVLWLRQDVH[SHFWHG

$OWKRXJKWKHPHDVXUHRIHIIHFWLYHQHVVLVDKHOSIXOILJXUHRIPHULWLQWKHLQWHUSUHWDWLRQRIWKHUHVXOWVLWLV LPSRUWDQWW R UHFDOOW KDW ZKHQE RWKS OD\HUV DGDSW WKH UHGTX HHQ HIIHFW UHQGHUV WKH PHDVXUH RI HIIHFWLYHQHVVDORQHLQVXIILFLHQWIRUDFRPSUHKHQVLYHHYDOXDWLRQRIWKHUHVXOWV ,QSDUWLFXODUIRUWKH$/7DQG0(0WKHWDEOHLQGLFDWHVWKDWWKHPHDVXUHRIHIIHFWLYHQHVVE\LWVHOILVQRW VXIILFLHQWIRUWKHHYDOXDWLRQGXHWRWKHUHGTXHHQHIIHFW2WKHUPHWKRGVRIHYDOXDWLRQDUHDVXEMHFWRIRXU FXUUHQWSURMHFWHIIRUWV7KHVHPHWKRGVLQFOXGH x

TXDOLWDWLYHHYDOXDWLRQWKURXJKPDQHXYHUYLVXDOL]DWLRQ

x

FRPSDULVRQRISOD\HUVWRDOLEUDU\RISDVWRSSRQHQWVDQG

x x

UHSHDWHGFRPSDULVRQRISOD\HUVWRDVWDWLF¦H[SHUW§SOD\HU GHYHORSPHQW RI RWKHU REMHFWLYH PHDVXUHV RI PDQHXYHUV DQG WKHLU FRPSOH[LW\ IRU H[WHUQDO H[DPLQDWLRQE\DQDO\VWV

&RQFOXVLRQV

7KH SULPDU\ REMHFWLYH RI WKH SURMHFW SKDVH SUHVHQWHGK HUH ZDV WR GHPRQVWUDWH WKH IHDVLELOLW\ RI D WZR VLGHG OHDUQLQJ ZLWKLQ D RQHYHUVXVRQH DLU HQJDJHPHQW VFHQDULR /HDUQLQJ ERWK RQH VLGHG DQG WZR VLGHGKDVEHHQDQGLVRIJUHDWLQWHUHVWW R D QXPEHU RI FRPPXQLWLHV +RZHYHU LW LV DOVR UHFRJQL]HG DV EHLQJDYHU\FKDOOHQJLQJSUREOHPWKXVPRVWRIWKHOHDUQLQJZRUNKDVEHHQZLWKLQWKHFRQILQHVRI ¦WR\ SUREOHPV§ :H EHOLHYH WKDW RXU ZRUN LV RQH RI WKH ILUVW H[SORUDWLRQV RI D UHDO ZRUOG FRQWLQXRXV WLPH LPSOHPHQWDWLRQRIWZRVLGHGOHDUQLQJ2XUUHVXOWVVKRZWKDWERWKDLUFUDIWDUHDEOHWROHDUQDQGGLVFRYHU REMHFWLYHO\ LQWHUHVWLQJ PDQHXYHUV $V H[SHFWHG WKH UHVXOWVV KRZ PRUH DGYDQFHG PDQHXYHUV WKDQ ZH KDYHREVHUYHGLQVLPXODWLRQVZLWKRXWOHDUQLQJRULQVLPXODWLRQVZLWKRQHVLGHGOHDUQLQJ0RUHRYHUWKH SDWWHUQRIHYROXWLRQDVLQ)LJXUHVDQG LQGLFDWHVWKHVRUWRIHYROXWLRQDU\LQWHUSOD\IURPWZRVLGHG OHDUQLQJWKDWVKRXOG\LHOGPRUHLQWHUHVWLQJWDFWLFV

)XWXUH:RUN

*LYHQWKHFRPSOH[LW\RIHYDOXDWLQJUHVXOWVLQWZRVLGHGOHDUQLQJDQGSLORWFRPPHQWVIURPRXUSUHYLRXV HIIRUWVZHDUHFXUUHQWO\SXUVXLQJVHYHUDOQHZGLUHFWLRQVIRUWKLVSURMHFW7KHVHLQFOXGH x x

,QWHUDFWLRQZLWKDLUFRPEDWDQDO\VWVIRUUHTXLUHPHQWVVSHFLILFDWLRQVDQGYDOLGDWLRQ ,QFUHDVLQJWKHVFRSHRIWKHSUREOHPGRPDLQLQFOXGLQJ

L ¦6RIW§SHUIRUPDQFHEDVHG VWRSSLQJFULWHULRQIRUWKHOHDUQLQJV\VWHP LL ([SDQGLQJWKHPHDVXUHVRIHIIHFWLYHQHVV LLL $GGLWLRQDOW\SHVRIDLUFUDIW

LY ([WHQVLRQ DQG DSSOLFDWLRQ RI WZRVLGHG OHDUQLQJ WR Y DQGE H\RQG YLVXDO UDQJH %95  DLU HQJDJHPHQWV

x

i i i i

,QFUHDVHVFRSHDQGDQDO\VLVRIOHDUQLQJDOJRULWKPVLQFOXGLQJ LPSURYHGH[DPLQDWLRQRIFRHYROXWLRQDU\HIIHFWV>@

TXDOLWDWLYHHYDOXDWLRQWKURXJKPDQHXYHUYLVXDOL]DWLRQ

UHSHDWHGFRPSDULVRQRISOD\HUVWRDVWDWLF¦H[SHUW§SOD\HU

FRPSDULVRQ RI SOD\HUV WR D OLEUDU\ RI SDVW RSSRQHQWV GHYHORSPHQW RI REMHFWLYH PHDVXUHV RI PDQHXYHUVDQGWKHLUFRPSOH[LW\IRUH[WHUQDOH[DPLQDWLRQE\DQDO\VWV

7KH RYHUDOO UHVXOWV RI WKH SURMHFW DUH SURPLVLQJ IRU DFTXLULQJ LQWHUHVWLQJ LQIRUPDWLRQ DERXW DLU FRPEDW PDQHXYHULQJ 0RUHRYHU WKH\ SRLQWW R D EURDGHU DUHD RI DSSOLFDWLRQ ZKHUH JHQHWLFVEDVHG PDFKLQH OHDUQLQJ LV XVHG WR GLVFRYHU QRYHO VWUDWHJLHV LQ VLPXODWLRQ 6XFKQ RYHO VWUDWHJLHV FDQ WKHQE H XVHG WR SURYLGHIHHGEDFNWRDQDO\VWVGHVLJQHUVFXVWRPHUVDQGRSHUDWRUV

$FNQRZOHGJPHQWV

7KH DXWKRUV JUDWHIXOO\ DFNQRZOHGJH WKDWW KLV ZRUN LVV SRQVRUHGE \ $LU )RUFH )& 7KH DXWKRUVDOVRJUDWHIXOO\DFNQRZOHGJHWKHVXSSRUWSURYLGHGE\1$6$IRUWKHHDUO\SKDVHVRIWKLVSURMHFW XQGHUJUDQW1$6

5HIHUHQFHV

>@5(6PLWKDQG% $ 'LNH /HDUQLQJQ RYHO ILJKWHU FRPEDW PDQHXYHUU XOHV YLD JHQHWLF DOJRULWKPV ,QWHUQDWLRQDO-RXUQDORI([SHUW6\VWHPV   >@5/6KDZ)LJKWHU&RPEDW7DFWLFVDQG0DQHXYHULQJ8QLWHG6WDWHV1DYDO,QVWLWXWH3UHVV  >@566XWWRQDQG$*%DUWR5HLQIRUFHPHQW/HDUQLQJ$Q,QWURGXFWLRQ0,73UHVV 

>@6::LOVRQ=&6$]HURWKOHYHOFODVVLILHUV\VWHP(YROXWLRQDU\&RPSXWDWLRQ  

>@ - + +ROODQG . - +RO\RDN 5 ( 1LVEHWW DQG 3 5 7KDJDUG ,QGXFWLRQ 3URFHVVHV RI LQIHUHQFH OHDUQLQJDQGGLVFRYHU\0,73UHVV&DPEULGJH0$  >@-++ROODQG$GDSWDWLRQLQ1DWXUDODQG$UWLILFLDO6\VWHPV0,73UHVV 

>@ ' ( *ROGEHUJ *HQHWLF $OJRULWKPV LQ 6HDUFK 2SWLPL]DWLRQ DQG 0DFKLQH /HDUQLQJ $GGLVRQ :HVOH\  >@5(6PLWK%$'LNHDQG6$6WHJPDQQ,QKHULWDQFHLQ*HQHWLF$OJRULWKPVLQ3URFHHGLQJVRI WKH$&06\PSRVLXPRQ$SSOLHG&RPSXWLQJ$&03UHVV 

>@ 3 0 'RDQH & + *D\ DQG - $ )OLJJ 0XOWLV\VWHP LQWHJUDWHG FRQWURO 0X6,&  SURJUDP ILQDO UHSRUW7HFKQLFDOUHSRUW:ULJKW/DERUDWRULHV:ULJKW3DWWHUVRQ$)%2+ >@5$[HOURG7KH(YROXWLRQRI&RRSHUDWLRQ%DVLF%RRNV1HZ@5'/XFHDQG+5DLIID*DPHVDQG'HFLVLRQV'RYHU3XEOLFDWLRQV 

>@')ORULDQRDQG61ROIL6*RGVDYHWKHUHGTXHHQ&RPSHWLWLRQLQFRHYROXWLRQDU\URERWLFVLQ 3URFHHGLQJVRIWKH6HFRQG,QWHUQDWLRQDO&RQIHUHQFHRQ*HQHWLF3URJUDPPLQJ  >@&'5RVLQDQG5.%HOHZ1HZPHWKRGVLQFRPSHWLWLYHFRHYROXWLRQ(YROXWLRQDU\&RPSXWDWLRQ