evolutionary approach for training recurrent neural networks. (RNNs). ... optimization of network structure involves stochastic and performance-driven adaptation ...
0XOWLREMHFWLYH(YROXWLRQDU\5HFXUUHQW1HXUDO1HWZRUNVIRU 6\VWHP,GHQWLILFDWLRQ -+$QJ& . *RK( - 7HRK DQG$$0DPXQ
$EVWUDFW²7KLV SDSHU SURSRVHV D QHZ PXOWLREMHFWLYH HYROXWLRQDU\ DSSURDFKIRU WUDLQLQJ UHFXUUHQW QHXUDO QHWZRUNV 511V 7KH DOJRULWKP XVHV IHDWXUHV RI D YDULDEOH OHQJWK UHSUHVHQWDWLRQ DOORZLQJ HDV\ DGDSWDWLRQ RI QHXUDO QHWZRUNV VWUXFWXUHV DQG D PLFUR JHQHWLF DOJRULWKP *$ ZLWK DQ DGDSWLYHORFDOVHDUFKLQWHQVLW\VFKHPHIRUORFDOILQHWXQLQJ,Q DGGLWLRQDVWUXFWXUDOPXWDWLRQ 60 RSHUDWRUIRUHYROYLQJWKH DSSURSULDWH QXPEHU RI QHXURQV IRU 511V LV XVHG 6LPXODWLRQ UHVXOWV GHPRQVWUDWHG WKHHIIHFWLYHQHVV RI SURSRVHG PHWKRGIRU V\VWHPLGHQWLILFDWLRQWDVNV
7
, ,1752'8&7,21
+( VLPXOWDQHRXV HYROXWLRQ RI ERWK QHXUDO QHWZRUN DUFKLWHFWXUHDQGZHLJKWVZLOOLQHYLWDEO\UHVXOWLQDODUJH LQFUHDVHLQWKHVL]HRIWKHVHDUFKVSDFH:KLOHHYROXWLRQDU\ DOJRULWKPV ($V DUH FDSDEOH RI H[SORULQJ DQG LGHQWLI\LQJ SURPLVLQJ UHJLRQV RI WKH VHDUFK VSDFH WKH\ UHTXLUH D UHODWLYHO\ORQJHUWLPHWRORFDWHWKHORFDORSWLPD:LWKWKLVLQ PLQG PDQ\ UHVHDUFKHUV KDYH VRXJKW WR FRPSOHPHQW WKH JOREDOH[SORUDWLRQDELOLW\RI($VE\LQFRUSRUDWLQJGHGLFDWHG OHDUQLQJ RU ORFDO VHDUFK /6 KHXULVWLFV 6XSSRUW IURP H[SHULPHQWDO VWXGLHV KDV VKRZQ WKDW ($/6 K\EULGV RU K\EULG($VDUHFDSDEOHRIPRUHHIILFLHQWVHDUFKFDSDELOLWLHV > @>@ :KLOH PDQ\ UHVHDUFKHUV KDYH DFNQRZOHGJHG WKH LPSRUWDQFH LQ FRQVLGHULQJ ERWK DUFKLWHFWXUH DQG FRQQHFWLRQ ZHLJKWV FRQFXUUHQWO\ >@ >@ >@ >@ WKH PXOWL REMHFWLYH 02 QDWXUH RI WKH SUREOHP LV UDUHO\ FRQVLGHUHG XQWLO UHFHQWO\ >@ >@ >@ >@ >@ >@ >@ $EEDVV >@ DSSOLHGD02DSSURDFKLQWKHRSWLPL]DWLRQRI$11VZKHUH WKHSURSRVHGPHPHWLF3DUHWR$11V 03$11V LVEDVHGRQ DGLIIHUHQWLDOHYROXWLRQ*LXVWROLVLDQG6LPHRQH>@FRQVLGHUV WKH DGGLWLRQDO REMHFWLYH RI PRGHO LQSXW GLPHQVLRQ *DUFLD 3HGUDMDV HW DO >@ LQYHVWLJDWHG WKH LQIOXHQFH RI GLIIHUHQW REMHFWLYHV DQG UHVXOWV LQGLFDWHG WKH DGYDQWDJHV RI WKH 02 DSSURDFKRYHUVLQJOHREMHFWLYH 62 DSSURDFK 7KH XVH RI DUWLILFLDO QHXUDO QHWZRUNV $11V LQ FRQWURO KDVDOVREHHQUHFHLYLQJLQFUHDVLQJDWWHQWLRQGXHWRLWVDELOLW\ WR OHDUQ DQG KDQGOH QRQOLQHDULWLHV DQG XQFHUWDLQWLHV SUHVHQW LQ WKH V\VWHP > @ >@ ,Q SDUWLFXODU UHFXUUHQW QHXUDO QHWZRUNV 511V WKURXJK LWV XQFRQVWUDLQHG V\QDSWLF FRQQHFWLYLW\ DQG UHVXOWLQJ VWDWHGHSHQGHQW QRQOLQHDU G\QDPLFV RIIHU D JUHDWHU OHYHO RI FRPSXWDWLRQDO DELOLW\ DV FRPSDUHG WR UHJXODU IHHGIRUZDUG QHXUDO QHWZRUNV )11V -L +XD %ULDQ $QJ &KL .HRQJ *RK (X -LQ 7HRK DQG $EGXOODK $O 0DPXQ DUH ZLWK WKH 'HSDUWPHQW RI (OHFWULFDO DQG &RPSXWHU (QJLQHHULQJ 1DWLRQDO8QLYHUVLW\RI6LQJDSRUH(QJLQHHULQJ'ULYH6LQJDSRUH HPDLO ^EULDQDQJ FNJRK HXMLQWHRK HOHDDP`#QXVHGXVJ
DUFKLWHFWXUHV +RZHYHU D PDMRU GUDZEDFN RI WKH KLJKHU FRPSOH[LW\ LV WKDW JUDGLHQWEDVHG EDFNSURSDJDWLRQ OHDUQLQJ DOJRULWKPV DQG UHDOWLPH UHFXUUHQW OHDUQLQJ >@ IRU 511V DUH PRUH OLNHO\ WR EH WUDSSHG LQ ORFDO RSWLPD UHVXOWLQJ LQ VXERSWLPDO VROXWLRQV 7KXV WKH FRQFHSW RI LQFRUSRUDWLQJ HYROXWLRQDU\ HOHPHQWV LQWR QHXUDO QHWZRUNV KDV OHDG WR WKH HPHUJHQFH RI HYROXWLRQDU\ DUWLILFLDO QHXUDO QHWZRUNV ($11V LQUHFHQW\HDUV'ULYHQE\DQHYROXWLRQDU\ VHDUFK SURFHVV WKDW SURYLGHV D JOREDO SHUVSHFWLYH WR WKH DGDSWDWLRQ SURFHVV ($11V KDYH GHPRQVWUDWHG WR EH OHVV VHQVLWLYH WR LQLWLDO FKRLFHV RI ZHLJKWV DQG DUH DOVR FDSDEOH RI GHDOLQJ ZLWK WKH SUREOHP RI SUHPDWXUH FRQYHUJHQFH WKDW LV XVXDOO\ DVVRFLDWHGZLWKFRQYHQWLRQDOJUDGLHQWEDVHGDSSURDFKHV>@ :LWK WKHVH LQ PLQG WKLV SDSHU LV FRQFHUQHG ZLWK WKH GHYHORSPHQW RI D PXOWLREMHFWLYH HYROXWLRQDU\ UHFXUUHQW QHXUDO QHWZRUN 02(511 GHVLJQ 6LPLODU WR >@ >@ WKH SURSRVHG 02(511 XWLOL]HV WKH FRQFHSWV RI 3DUHWR RSWLPDOLW\ ZKLFK DFFRXQWV IRU WKH LQKHUHQW WUDGHRIIV LQ FDSDFLW\ DQG FRPSOH[LW\ 0RVW ZRUN FRQVLGHULQJ RSWLPL]DWLRQ RI QHWZRUN VWUXFWXUH LQYROYHV VWRFKDVWLF DQG SHUIRUPDQFHGULYHQ DGDSWDWLRQ RI QHWZRUN DUFKLWHFWXUH ZKLFKZLOOLQHYLWDEO\UHVXOWLQODUJHUQHWZRUNFRPSOH[LWLHV 7ZR SUREOHP VSHFLILF RSHUDWRUV QDPHO\ WKH YDULDEOH OHQJWKUHSUHVHQWDWLRQDQGWKHPLFURJHQHWLFDOJRULWKP *$ DUHSURSRVHGWRKDQGOHWKHLVVXHVRIDUFKLWHFWXUDODGDSWDWLRQ DQG ORFDO ILQHWXQLQJ 7KH YDULDEOH OHQJWK UHSUHVHQWDWLRQ HQFRGHV WKH HQWLUH 511V GHVLJQ ZLWK HDFK QHXURQ DQG LWV DVVRFLDWHG ZHLJKWV DV WKH EDVLF EXLOGLQJ EORFN DQG LV GHVLJQHGWRDOORZIRUHDV\DGDSWDWLRQRI511VDUFKLWHFWXUH 7KH *$ LV DSSOLHG IRU WKH H[SORLWDWLRQ RI FRQQHFWLRQ ZHLJKWV DQG LQFRUSRUDWHV WKH SVHXGRLQYHUVH WR ILQG WKH RSWLPDORXWSXWZHLJKWV %DFNJURXQG LQIRUPDWLRQ LV SURYLGHG LQ 6HFWLRQ ZKLOH WKH 0RRUH3HQURVH JHQHUDOL]HG LQYHUVH SVHXGRLQYHUVH IRU ORFDO VHDUFK LV JLYHQ LQ 6HFWLRQ 7KH GHWDLOV RI WKH SURSRVHGIHDWXUHVLQWKH02(511DUHGHVFULEHGLQ6HFWLRQ ([SHULPHQWDO VWXGLHV XSRQ V\VWHP LGHQWLILFDWLRQ SUREOHP DUH FRQGXFWHG LQ 6HFWLRQ &RQFOXVLRQV DUH VXEVHTXHQWO\ GUDZQLQ6HFWLRQ ,, %$&.*5281',1)250$7,21 $ 0XOWLREMHFWLYH(YROXWLRQDU\2SWLPL]DWLRQ ,Q FRQWUDVW WR 62 RSWLPL]DWLRQ WKH VROXWLRQ WR 02 RSWLPL]DWLRQSUREOHPH[LVWVLQWKHIRUPRIDOWHUQDWHWUDGHRIIV NQRZQ DV 3DUHWR RSWLPDO VHW 7KH GLIIHUHQW GRPLQDQFH UHODWLRQVKLS LV LOOXVWUDWHG LQ )LJXUH ZKHUH WKH VROXWLRQV GHQRWHG E\ EODFN FLUFOHV IRUP WKH RSWLPDO 3DUHWR IURQW DQG
1586 c 1-4244-1340-0/07/$25.00 2007 IEEE
GRPLQDWHG WKH VROXWLRQV UHSUHVHQWHG E\ ZKLWH FLUFOHV 1RWH WKDW HDFK REMHFWLYH FRPSRQHQW RI DQ\ QRQGRPLQDWHG VROXWLRQ LQWKH 3DUHWR RSWLPDO VHW FDQ RQO\ EH LPSURYHG E\ GHJUDGLQJ DW OHDVW RQH RI LWV RWKHU REMHFWLYH FRPSRQHQWV >@ 7UDGLWLRQDO RSHUDWLRQDO UHVHDUFK DSSURDFKHV WR 02 RSWLPL]DWLRQ W\SLFDOO\ HQWDLOV WKH WUDQVIRUPDWLRQ RI WKH RULJLQDOSUREOHP LQWR D 62 SUREOHP DQG HPSOR\V SRLQWE\ SRLQW DOJRULWKPV VXFK DV EUDQFKDQGERXQG WR LWHUDWLYHO\ REWDLQ D EHWWHU VROXWLRQ 6XFK DSSURDFKHV KDYH VHYHUDO OLPLWDWLRQVLQFOXGLQJWKHUHTXLUHPHQWRIWKH02SUREOHPWR EHZHOOEHKDYHGLHGLIIHUHQWLDELOLW\VHQVLWLYLW\WRWKHVKDSH RI WKH 3DUHWRIURQW DQG WKH JHQHUDWLRQ RI RQO\ RQH VROXWLRQ IRUHDFKUXQ2QWKHRWKHUKDQGPHWDKHXULVWLFDODSSURDFKHV WKDWDUHLQVSLUHGE\ELRORJLFDOSKHQRPHQDKDVEHHQJDLQLQJ LQFUHDVLQJDFFHSWDQFHDVDPXFKPRUHIOH[LEOHDQGHIIHFWLYH DOWHUQDWLYH WR FRPSOH[ RSWLPL]DWLRQ SUREOHPV LQ WKH UHFHQW \HDUV $PRQJ WKHVH PHWDKHXULVWLFV PXOWLREMHFWLYH HYROXWLRQDU\DOJRULWKP 02($ LVRQHRIWKHPRUHSRSXODU VWRFKDVWLF VHDUFK PHWKRGRORJLHV WR VROYH 02 SUREOHPV (PXODWLQJWKH'DUZLQLDQ:DOODFHSULQFLSOHRI³VXUYLYDORI WKHILWWHVW´LQQDWXUDOVHOHFWLRQDQGDGDSWDWLRQ02($VKDYH WKH GLVWLQFW DGYDQWDJH RI EHLQJ DEOH WR VDPSOH PXOWLSOH VROXWLRQV VLPXOWDQHRXVO\ 6XFK D IHDWXUH SURYLGHV WKH 02($ZLWKDJOREDOSHUVSHFWLYHRIWKH02SUREOHPDVZHOO DVWKHFDSDELOLW\WRILQGDVHWRI3DUHWRRSWLPDOVROXWLRQVLQD VLQJOHUXQ I
WKURXJK WKLV UHFXUUHQF\ 7KH UHFXUUHQW QDWXUH RI WKH VLJQDO IORZ LQ 511V KRZHYHU LQFUHDVHV WKH FRPSOH[LW\ RI WKH UHVXOWLQJQHWZRUNDQGDVDUHVXOWFRPSOLFDWLQJWKHPRGHOLQJ DQG VXEVHTXHQW WUDLQLQJ RI WKH IUHH SDUDPHWHUV V\QDSWLF ZHLJKWV RIWKHQHWZRUN )RU 511V WKHUH DUH ERWK VWUXFWXUDO DV ZHOO DV WHPSRUDO VWDWH GHSHQGHQFLHV 6WUXFWXUDO GHSHQGHQFLHV DULVH IURP WKH V\QDSWLF ZHLJKWV WKDW DUH SUHVHQW EHWZHHQ QHXURQV WKLV LV FRPPRQ LQ IHHGIRUZDUG DUFKLWHFWXUHV ZKHUHDV WHPSRUDO GHSHQGHQFLHV DUH D FRQVHTXHQW RI WKH IDFW WKDW WKH SUHVHQW VWDWH RI D QHXURQ GHSHQGV RQ WKH VWDWH RI D QHXURQ D IHZ WLPHVWHSV VD\NWLPHVWHSV DJR7KLVEHLQJVDLGIRUNVWHS VWDWHGHSHQGHQFLHVWKHFXUUHQWVWDWHGHSHQGVRQWKHVWDWHVRI WKH QHXURQV IURP « N WLPHVWHSV DJR )RU HDFK WLPH VWHS GHSHQGHQF\ D PDWUL[ :N GHILQHV WKH VHW RI V\QDSWLF ZHLJKWV OHDGLQJ IURPWKH VWDWHV RI WKHQHXURQV NWLPHVWHSV DJR WR WKH SUHVHQW VWDWH 7KXV IRU D V\VWHP ZLWK NVWHS GHSHQGHQF\ZHFDQKDYHXSWRNZHLJKWPDWULFHV: $VVXPLQJ DQ DIILQH IRUP WKH VWDWH XSGDWH RI D W\SLFDO 511V LV PDWKHPDWLFDOO\ GHVFULEHG E\ [ W = I :[ W − + E IRU D V\QFKURQRXV SDUDOOHO XSGDWH ZKHUHDOOQHXURQVLQWKHV\VWHPDUHXSGDWHGVLPXOWDQHRXVO\ DQGE\ [L W = I :L [ W − + EL IRUDQDV\QFKURQRXV VHULDO XSGDWHZKHUHRQO\WKHLWKQHXURQWKDWLVVHOHFWHGLVXSGDWHG ]−N
] − \ K W
2SWLPDO3DUHWR)URQW
[ W 1RQGRPLQDWHGVROXWLRQV
[ W
\ R W
,QIHDVLEOH5HJLRQ
[L W I
)LJ,OOXVWUDWLRQRIWKHRSWLPDO3DUHWR IURQWDQGWKHUHODWLRQVKLSEHWZHHQ GRPLQDWHGDQGQRQGRPLQDWHGVROXWLRQV
% 5HFXUUHQW1HXUDO1HWZRUNV$UFKLWHFWXUH 7KH XVH RI 511V IRU V\VWHP LGHQWLILFDWLRQ WDVN KDV WKH IROORZLQJDGYDQWDJHV • 511 GRHV QRW UHTXLUH DQ\ D SULRUL LQWHUQDO XQGHUVWDQGLQJRIWKHSODQWRUDQ\OLQHDULW\DVVXPSWLRQV • 511 DSSUR[LPDWHV SODQW G\QDPLFV WKURXJK DGMXVWPHQW RI WKH QXPEHU RI KLGGHQ QRGHV DQG WKH YDOXHV RI ZHLJKWV 7KH QXPEHU RI DGMXVWDEOH IUHH SDUDPHWHUV RI WKH QHWZRUN ZKLFK LQ WKLV FDVH DUH WKH KLGGHQ QRGHV GHWHUPLQHV WKH FRPSOH[LW\ RI WKH V\VWHP 7KXV VHOHFWLQJ D PRGHO VWUXFWXUH LV HTXLYDOHQW WR VLPSO\ FKDQJLQJWKHQXPEHURIKLGGHQQRGHV7KHUHDOVWDWHVRI WKHSODQWDUHQRWQHFHVVDU\ 7KH 511V VWUXFWXUH DGRSWHG LQ WKLV SDSHU LV VKRZQ LQ )LJXUH7KLVSHUPLWVVWDWHGHSHQGHQFLHVDOORZLQJSUHYLRXV VWDWHV WR LQIOXHQFH WKH DFWLYDWLRQ RI VXEVHTXHQW DFWLYDWLRQV
\ KM W ]−N
] −
)LJ5HFXUUHQWQHXUDOQHWZRUNVWUXFWXUH
,,, 0225(3(1526(*(1(5$/,=(' 36(8'2,19(56( 7R UHVROYH D OLQHDU V\VWHP RI WKH JHQHUDO IRUP +β = 7 LV VWUDLJKWIRUZDUG LI WKH PDWUL[ + LV VTXDUH DQG QRQVLQJXODU +RZHYHUXQGHUPDQ\SUDFWLFDOFLUFXPVWDQFHVWKHPDWUL[ + LVXVXDOO\VLQJXODUDQGOLNHO\WR EHUHFWDQJXODU7KH0RRUH 3HQURVH JHQHUDOL]HG LQYHUVH VLPSOLILHV WKH WUHDWPHQW E\ SURYLGLQJWKHVROXWLRQWRWKHOLQHDUV\VWHPLQDOHDVWVTXDUHV VHQVH 7KH SVHXGRLQYHUVH RI + LV GHILQHG GLIIHUHQWO\ GHSHQGLQJRQWKHUDQNDQGGLPHQVLRQDOLW\RI+ ,QPRVWSUDFWLFDOSUREOHPWKHV\VWHPLVRYHUGHWHUPLQHG DQG KHQFH ZRXOG ZDQW ILQG WKH OHDVWVTXDUH HUURU RI
2007 IEEE Congress on Evolutionary Computation (CEC 2007)
1587
__ +β − 7 __ LQ WKH SUHVHQFH RI WKH LQFRQVLVWHQFLHV LQWURGXFHG 7KXV β LV REWDLQHG IURP __ +β − 7 __ 7KH SVHXGRLQYHUVHFDQEHVKRZQWREHWKHPLQLPXPQRUPOHDVW VTXDUHV VROXWLRQ RI WKH V\VWHP LH WKH SVHXGRLQYHUVH RI β ZKLFK LV β + PLQLPL]HV __ +β − 7 __ )RU IXUWKHU GHWDLOV RQ WKHSVHXGRLQYHUVHUHDGHUVDUHGLUHFWHGWR>@ ,9 +@DVLPSOHHOLWLVW($WHQGVWRFRQYHUJHWRZDUGV DVLQJOHVROXWLRQDQGRIWHQORVHVVROXWLRQVGXHWRWKHHIIHFWV
2007 IEEE Congress on Evolutionary Computation (CEC 2007)
RIVHOHFWLRQVSUHVVXUHVHOHFWLRQQRLVHGULIWLQJDQGRSHUDWRU GLVUXSWLRQ 7KH XVH RI QLFKH VKDULQJ ZDV SURSRVHG E\ *ROGEHUJ DQG 5LFKDUGVRQ >@ WR SUHYHQW JHQHWLF GULIW DQG WR SURPRWH WKH VDPSOLQJ RI WKH ZKROH 3DUHWR IURQW E\ WKH SRSXODWLRQ 7KH EDVLF LGHD LV WKDW LQGLYLGXDOV LQ D SDUWLFXODU QLFKH VKRXOG VKDUH WKH DYDLODEOH UHVRXUFHV DQG WKH ILWQHVV YDOXH RI DQ LQGLYLGXDO LV GHJUDGHG DV WKH QXPEHU RI LQGLYLGXDOV LQ LWV QHLJKERXUKRRG LQFUHDVHV 1LFKLQJ FDQ EH SHUIRUPHG LQ HLWKHUWKHGHFLVLRQYDULDEOHGRPDLQRUWKHREMHFWLYHGRPDLQ 7KH FKRLFH EHWZHHQQLFKLQJ LQ GHFLVLRQ YDULDEOH GRPDLQ RU REMHFWLYH GRPDLQ GHSHQGV RQ ZKDW LV GHVLUHG IRU WKH XQGHUO\LQJ SUREOHP ,Q JHQHUDO QLFKH VKDULQJ LV DFKLHYHG XVLQJ D VKDULQJ IXQFWLRQ /HW G EH WKH (XFOLGHDQ GLVWDQFH EHWZHHQ[DQG\7KHQHLJKERUKRRGVL]HLVGHILQHGLQWHUPV RIGDQGVSHFLILHGE\WKHVRFDOOHGQLFKHUDGLXV σ VKDUH 7KH VKDULQJIXQFWLRQLVGHILQHGDVIROORZV α VK G = ® − G σ VKDUH LIG σ VKDUH
¯RWKHUZLVH
$QGWKHQLFKH FRXQW IXQFWLRQ LV GHILQHG ZLWKWKHKHOS RI VKDULQJIXQFWLRQ VK GLVW [ \ QF [ =
¦\
σ VKDUH LV D NH\ SDUDPHWHU WKDW DIIHFWV 02($ V HIIHFWLYHQHVV,PSURSHUVHWWLQJRIWKHYDOXHZLOOUHVXOWLQEDG GLVWULEXWLRQRIWKHSRSXODWLRQ,QSUDFWLFHWKHQLFKHUDGLXVLV GLIILFXOW WR HVWLPDWH EHFDXVH WKHUH LV QR D SULRUL NQRZOHGJH DERXW WKH VKDSH RI WKH 3DUHWR IURQW IRU PDQ\ SUREOHPV )RQVHFDDQG)OHPLQJ>@JDYHVRPHERXQGLQJJXLGHOLQHVRI DSSURSULDWH σ VKDUH YDOXHVIRUPXOWLREMHFWLYHSUREOHPVZKHQ WKH QXPEHU RI LQGLYLGXDOV LQ WKH SRSXODWLRQ DQG WKH PLQLPXPPD[LPXPYDOXHVLQHDFKREMHFWLYHGLPHQVLRQDUH JLYHQ,QWKLVSDSHUWKHREMHFWLYHVSDFHLVQRUPDOL]HGRQOLQH EDVHGRQWKHHYROYHGWUDGHRIIVDQGWKHVKDULQJGLVWDQFHLVVHW WR DUFKLYHBVL]H 7KH QLFKH FRXQW ZLOO EH XVHG LQ WKH VHOHFWLRQDQGDUFKLYHXSGDWLQJ
UHSUHVHQWDWLRQ LPSOLHV WKDW LW IXOILOV VRPH FULWHULD VXFK DV HDVH RI LPSOHPHQWDWLRQ RU H[SORLWDWLRQ RI WKH SUREOHP VWUXFWXUH )RU VLPSOLFLW\ WKH FKURPRVRPH LV RIWHQ UHSUHVHQWHG DV D IL[HGVWUXFWXUH ,Q ($11V D K\EULG VWUXFWXUH EHWZHHQ ELQDU\ DQG UHDOQXPEHU UHSUHVHQWDWLRQ LV FRPPRQO\ XVHG IRU WKH VLPXOWDQHRXV RSWLPL]DWLRQ RI ZHLJKWVDQGDUFKLWHFWXUH+RZHYHUVXFKDUHSUHVHQWDWLRQLV QRW VXLWDEOH IRU 511V GHVLJQ SUREOHP ZKHUH IOH[LELOLW\ LV HVVHQWLDO IRU WKH VLPXOWDQHRXV HYROXWLRQ RI DUFKLWHFWXUH DQG FRQQHFWLRQZHLJKWV ,Q WKLV SDSHU D YDULDEOHOHQJWK FKURPRVRPH UHSUHVHQWDWLRQ LV DGRSWHG WR UHSUHVHQW WKH 511V WRSRORJ\ LQFOXGLQJWKHQXPEHURIQHXURQVLQWKHKLGGHQOD\HUVDQGWKH FRQQHFWLRQ ZHLJKWV OLQNLQJ WKH LQSXW KLGGHQ DQG RXWSXW OD\HU DV LOOXVWUDWHG LQ )LJXUH D )LJXUH E LV WKH LQVWDQWLDWLRQ RI WKH UHSUHVHQWDWLRQ LQ )LJXUH D (DFK QHXURQLVFRXSOHGZLWKLWVDVVRFLDWHGZHLJKWVWKXVDOORZLQJ HDV\ PDQLSXODWLRQ E\ VHDUFK RSHUDWRUV IRU WKH DGGLWLRQ RU GHOHWLRQ RI QHXURQV 7KH FKURPRVRPH PD\ FRQVLVW RI GLIIHUHQWQXPEHURIQHXURQVZKLFKUHIOHFWVWKHFRPSOH[LW\RI WKH 511V EXW WKH QXPEHU RI FRQQHFWLRQV LV IL[HG E\ WKH QXPEHURILQSXWDWWULEXWHV6XFKDUHSUHVHQWDWLRQLVHIILFLHQW DQG IDFLOLWDWHV WKH GHVLJQ RI SUREOHPVSHFLILF JHQHWLF RSHUDWRUV 9DULDEOHOHQJWK&KURPRVRPH 9DULDEOHQXPEHURI+LGGHQ1HXURQV 1
1
1
2XWSXW
@ EDVHG RQQLFKH FRXQW LV XVHG WR HOLPLQDWH WKH PRVW FURZGHG DUFKLYH
1590
PHPEHU 9 (;3(5,0(17$/678',(6 $ ([SHULPHQWDO6HWXS 02(511 LV LPSOHPHQWHG XVLQJ WKH 0$7/$% WHFKQLFDO FRPSXWLQJ SODWIRUP DQG FRUUHVSRQGLQJ VLPXODWLRQV DUH SHUIRUPHGRQDQ,QWHO3HQWLXP*+]FRPSXWHU7KLUW\ LQGHSHQGHQW UXQV IRU HDFK VHWWLQJ RQ WKH V\VWHP LGHQWLILFDWLRQSUREOHPDUHSHUIRUPHG7KHSDUDPHWHUVHWWLQJV XVHGLQWKHVLPXODWLRQDUHWDEXODWHGLQ7DEOH+LGGHQOD\HU QHXURQV XVH D K\SHUEROLF WDQJHQW QRQOLQHDULW\ ZKLOH WKH RXWSXWQRGHVXVHDOLQHDURXWSXWDFWLYDWLRQIXQFWLRQ 7KHQRQOLQHDUG\QDPLFV\VWHPPRGHOLVWDNHQIURP>@ >@7KHV\VWHPWR EHLGHQWLILHGLVEDVHGRQWKHGLIIHUHQFH HTXDWLRQ \ S N + =
\ S N \ S N − \ S N − X N − > \ S N − − @ + X N + \ S N − + \ S N −
7KHWUDLQLQJVDPSOHVIRULQSXW X N FRQWDLQVLQVWDQFHV IURP DQ LQGHSHQGHQW DQG LGHQWLFDOO\ GLVWULEXWHG XQLIRUP VHTXHQFH LQ WKH UDQJH >@ DQG LQVWDQFHV IURP D VLQXVRLGDO VLJQDO VLQ πN )RU FKHFNLQJ SKDVH WKH LQSXW X N FRPSULVHV LQVWDQFHV IURP WKH IROORZLQJ VLJQDO VLQ πN ° °° X N = ®− ° VLQ πN + VLQ πN ° ¯°+ VLQ πN
N < ≤ N < ≤ N < ≤ N <
7$%/( 3$5$0(7(56(77,1*6)2502(511 3RSXODWLRQV
3RSXODWLRQVL]H $UFKLYHVL]H
&KURPRVRPHW\SH 6HOHFWLRQ 0XWDWLRQPHWKRG 8QLIRUP0XWDWLRQ 6WUXFWXUDO0XWDWLRQ 0XWDWLRQVL]H 1LFKHUDGLXV
9DULDEOHOHQJWKUHDOQXPEHUUHSUHVHQWDWLRQ %LQDU\WRXUQDPHQWVHOHFWLRQ 1RUPDOO\GLVWULEXWHGPXWDWLRQ $GDSWLYH '\QDPLF
% ([SHULPHQWDO5HVXOWV 7KH DOJRULWKP SURSRVHG LQ WKLV SDSHU PDQDJHG WR REWDLQ D ORZ URRW PHDQ VTXDUH 506 HUURU RI ZKLFK WUDFNV WKH V\VWHP ZHOO &RPSDUHG WR WUDLQLQJ QHWZRUNV ZLWK PXWDWLRQEXWZLWKRXWORFDOVHDUFKDQGZLWKORFDOVHDUFKEXW ZLWKRXW PXWDWLRQ WKH\ REWDLQ 506 HUURU RI DQG UHVSHFWLYHO\ +HQFHLWGHPRQVWUDWHVWKHLPSRUWDQFH RI KDYLQJ WKHVH WZR RSHUDWRUV IRU WUDLQLQJ 511V :LWKRXW PXWDWLRQ RSHUDWRU WKH DOJRULWKP LV QRW DEOH WR H[SORUH D ODUJHUVHDUFKVSDFHDQGWKXVKLJKHUSUREDELOLW\RIWUDSSLQJLQ
2007 IEEE Congress on Evolutionary Computation (CEC 2007)
ORFDO RSWLPD 7KH ORFDO VHDUFK RSHUDWRU DOVR PDQLIHVW LQ EHLQJDEOHWRWUDLQWKHZHLJKWVE\GRLQJORFDOILQHWXQLQJ 1 0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 -1
0
100
200
300
400
500
600
700
800
900
1000
)LJD02511ZLWKPXWDWLRQDQGQRORFDOVHDUFK
1 0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 -1
0
100
200
300
400
500
600
700
800
900
1000
)LJE02511ZLWKORFDOVHDUFKDQGQRPXWDWLRQ 1 0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 -1
0
100
200
300
400
500
600
700
800
)LJF02511ZLWKORFDOVHDUFKDQGPXWDWLRQ
900
1000
9, &21&/86,216 $ QHZ DSSURDFK RI K\EULGL]LQJ 511V DQG($V LQ DPXOWL REMHFWLYHSHUVSHFWLYHLVSURSRVHGLQWKLVSDSHU7KHQXPEHU RI KLGGHQ QHXURQV LV HYROYHG XVLQJ 60 ZKLOH D YDULDEOH FKURPRVRPH LV XVHG WR PRGHO WKH QHWZRUN LQ WKH JHQRW\SH VSDFH ,Q DGGLWLRQ WKH QHWZRUN SHUIRUPDQFH LV JUHDWO\ HQKDQFHG XVLQJORFDO VHDUFK RSHUDWRU 6LPXODWLRQUHVXOWV RQ ZHOO NQRZQ EHQFKPDUNLQJ SUREOHPV GHPRQVWUDWHG WKDW WKH DOJRULWKPLVHIIHFWLYHIRUV\VWHPLGHQWLILFDWLRQWDVN )RU IXWXUH ZRUN LW ZRXOG EH FKDOOHQJLQJ WR LPSURYH WKH QHWZRUNIRUDSSOLFDWLRQLQQRLV\DQGXQFHUWDLQHQYLURQPHQWV 5()(5(1&(6 >@ +$$EEDVV³6SHHGLQJXSEDFNSURSDJDWLRQXVLQJPXOWLREMHFWLYH HYROXWLRQDU\DOJRULWKPV´1HXUDO&RPSXWDWLRQYROSS±
>@ +$$EEDVV$PHPHWLFSDUHWRHYROXWLRQDU\DSSURDFKWRDUWLILFLDO QHXUDOQHWZRUNV,Q06WXPSWQHU'&RUEHWWDQG0%URRNVHGLWRUV 3URFHHGLQJVRIWKHWK$XVWUDOLDQ-RLQW&RQIHUHQFHRQ$UWLILFLDO ,QWHOOLJHQFH $, %HUOLQ6SULQJHU9HUODJSS >@ -()LHOGVHQGDQG66LQJK³3DUHWRHYROXWLRQDU\QHXUDOQHWZRUNV´ ,(((7UDQVDFWLRQVRQ1HXUDO1HWZRUNV9RO SS >@ '%)RJHO(&:DVVRQDQG(0%RXJKWRQ³(YROYLQJQHXUDO QHWZRUNVIRUGHWHFWLQJEUHDVWFDQFHU´&DQFHU/HWWHUVYRO QR SS ± >@ &0)RQVHFDDQG3-)OHPLQJ³*HQHWLFDOJRULWKPIRU PXOWLREMHFWLYHRSWLPL]DWLRQIRUPXODWLRQGLVFXVVLRQDQG JHQHUDOL]DWLRQ´LQ3URFHHGLQJRIWKH)LIWK,QWHUQDWLRQDO&RQIHUHQFH RQ*HQHWLF$OJRULWKPVSS >@ 1*DUFLD3HGUDMDV&+HUYDV0DUWLQH]'2UWL] %R\HU³&RRSHUDWLYHFRHYROXWLRQRIDUWLILFLDOQHXUDOQHWZRUNV HQVHPEOHVIRUSDWWHUQFODVVLILFDWLRQ´,(((7UDQVDFWLRQVRQ (YROXWLRQDU\&RPSXWDWLRQYRO QRSS >@ 2*LXVWROLVLDQG96LPHRQH³2SWLPDOGHVLJQRIDUWLILFLDOQHXUDO QHWZRUNVE\DPXOWLREMHFWLYHVWUDWHJ\ JURXQGZDWHUOHYHO SUHGLFWLRQV´+\GURORJLFDO6FLHQFHV-RXUQDOYROQR >@ '(*ROGEHUJDQG-5LFKDUGVRQ³*HQHWLFDOJRULWKPVZLWKVKDULQJ IRUPXOWLPRGDOIXQFWLRQRSWLPL]DWLRQ´LQ3URFHHGLQJVRIWKH6HFRQG ,QWHUQDWLRQDO&RQIHUHQFHRQ*HQHWLF$OJRULWKPVSS > @ *+HUUPDQQ66*HDQG**XR³3UDFWLFDOLPSOHPHQWDWLRQRID QHXUDOQHWZRUNFRQWUROOHULQDKDUGGLVNGULYH´,(((7UDQVDFWLRQVRQ 6\VWHPV7HFKQRORJ\YROQRSS >@ +,VKLEXFKLDQG70XUDWD³$PXOWLREMHFWLYHJHQHWLFORFDOVHDUFK DOJRULWKPDQGLWVDSSOLFDWLRQWRIORZVKRSVFKHGXOLQJ´,((( 7UDQVDFWLRQRQ6\VWHPV0DQDQG&\EHUQHWLFV3DUW&YROQR SS ± >@ $-DV]NLHZLF]³2QWKHSHUIRUPDQFHRIPXOWLSOHREMHFWLYHJHQHWLF ORFDOVHDUFKRQWKHNQDSVDFNSUREOHPDFRPSDUDWLYH H[SHULPHQW´,(((7UDQVDFWLRQRQ(YROXWLRQDU\&RPSXWDWLRQYRO QRSS >@ &)-XDQJDQG&7/LQ³$UHFXUUHQWVHOIRUJDQL]LQJQHXUDOIX]]\ LQIHUHQFHQHWZRUN´,(((7UDQVDFWLRQRQ1HXUDO1HWZRUNVYRO QRSS >@ 6$.DU]UOLV6(3DSDGDNLV-%7KHRFKDULVDQG93HWULGLV ³0LFURJHQHWLFDOJRULWKPVDVJHQHUDOL]HGKLOOFOLPELQJRSHUDWRUVIRU *$RSWLPL]DWLRQ´,(((7UDQVDFWLRQVRQ(YROXWLRQDU\&RPSXWDWLRQ YROQRSS >@ ().KRU.&7DQ7+/HHDQG&.*RK³$VWXG\RQ GLVWULEXWLRQSUHVHUYDWLRQPHFKDQLVPLQHYROXWLRQDU\PXOWLREMHFWLYH RSWLPL]DWLRQ´$UWLILFLDO,QWHOOLJHQFH5HYLHZYROQRSS >@ 2.XOMDFD16ZDP\)//HZLVDQG&0.ZDQ³'HVLJQDQG ,PSOHPHQWDWLRQRI,QGXVWULDO1HXUDO1HWZRUN&RQWUROOHU8VLQJ %DFNVWHSSLQJ´,(((7UDQVDFWLRQVRQ,QGXVWULDO(OHFWURQLFVYRO QRSS >@ 6:0DKIRXG1LFKLQJPHWKRGVIRUJHQHWLFDOJRULWKPV3K'WKHVLV 8QLYHUVLW\RI,OOLQRXVDW8UEDQD&KDPSDLJQ >@ 90DQLH]]R³*HQHWLFHYROXWLRQRIWKHWRSRORJ\DQGZHLJKW GLVWULEXWLRQRIQHXUDOQHWZRUNV´,(((7UDQVDFWLRQVRQ1HXUDO 1HWZRUNVYROQRSS ± >@3$0DVWRURFRVWDWVDQG-%7KHRFKDULV³$UHFXUUHQWIX]]\QHXUDO PRGHOIRUG\QDPLFV\VWHPLGHQWLILFDWLRQ´,(((7UDQVDFWLRQVRQ 6\VWHPV0DQDQG&\EHUQHWLFV3DUW% &\EHUQHWLFVYROQR > @ 30HU]DQG%)UHLVOHEHQ³$FRPSDULVRQRIPHPHWLFDOJRULWKPV 7DEXVHDUFKDQGDQWFRORQLHVIRUWKHTXDGUDWLFDVVLJQPHQWSUREOHP´ LQ3URFHHGLQJVRIWKH &RQJUHVVRQ(YROXWLRQDU\&RPSXWDWLRQ YROSS >@ @65RWK$*HSSHUWKDQG&,JHO0XOWLREMHFWLYHQHXUDOQHWZRUN RSWLPL]DWLRQIRUYLVXDOREMHFWGHWHFWLRQ,Q@ '6HUUH0DWULFHV 7KHRU\DQG$SSOLFDWLRQV6SULQJHU9HUODJ1HZ @ 16ULQLYDVDQG.'HE³0XOWLREMHFWLYHRSWLPL]DWLRQXVLQJQRQ GRPLQDWHGVRUWLQJLQJHQHWLFDOJRULWKPV´(YROXWLRQDU\&RPSXWDWLRQ YROQRSS >@6:LHJDQG&,JHODQG8+DQGPDQQ³(YROXWLRQDU\PXOWLREMHFWLYH RSWLPL]DWLRQRIQHXUDOQHWZRUNVIRUIDFHGHWHFWLRQ´,QWHUQDWLRQDO -RXUQDORI&RPSXWDWLRQDO,QWHOOLJHQFHDQG$SSOLFDWLRQVYRO SS >@ 5-:LOOLDPVDQG'=LSVHU³([SHULPHQWDODQDO\VLVRIWKHUHDOWLPH UHFXUUHQWOHDUQLQJDOJRULWKP´&RQQHFWLRQ6FLHQFHYROQRSS >@ ;@ 4=KDRDQG7+LJXFKL³(YROXWLRQDU\OHDUQLQJRIQHDUHVWQHLJKERU 0/3´,(((7UDQVDFWLRQVRQ1HXUDO1HWZRUNVYROSS±
1592
2007 IEEE Congress on Evolutionary Computation (CEC 2007)
