Learning Pattern Classification Tasks with

Download PDF
0 downloads 0 Views 265KB Size Report
Comment
significantly fewer training instances of one class compared to another class. ..... into account both the sensitivity, (the accuracy on the positive examples) and the ... be a vector consisting of all free network parameters, including weights and ...
.GCTPKPI2CVVGTP%NCUUKſECVKQP6CUMUYKVJ+ODCNCPEGF&CVC5GVU



 :

 .GCTPKPI2CVVGTP%NCUUKHKECVKQP6CUMUYKVJ +ODCNCPEGF&CVC5GVU 

*LDQJ+RDQJ1JX\HQ$EGHVVHODP%RX]HUGRXPDQG6RQ/DP3KXQJ



8QLYHUVLW\RI:ROORQJRQJ $XVWUDOLD

+PVTQFWEVKQP 7KLVFKDSWHULVFRQFHUQHGZLWKWKHFODVVLPEDODQFHSUREOHPZKLFKKDVEHHQUHFRJQL]HGDVD FUXFLDOSUREOHPLQPDFKLQHOHDUQLQJDQGGDWDPLQLQJ7KHSUREOHPRFFXUVZKHQWKHUHDUH VLJQLILFDQWO\IHZHUWUDLQLQJLQVWDQFHVRIRQHFODVVFRPSDUHGWRDQRWKHUFODVV0RVWPDFKLQH OHDUQLQJDOJRULWKPVZRUNZHOOZLWKEDODQFHGGDWDVHWVVLQFHWKH\DLPWRRSWLPL]HWKHRYHUDOO FODVVLILFDWLRQ DFFXUDF\ RU D UHODWHG PHDVXUH )RU LPEDODQFHG GDWD VHWV WKH GHFLVLRQ ERXQGDU\HVWDEOLVKHGE\VWDQGDUGPDFKLQHOHDUQLQJDOJRULWKPVWHQGVWREHELDVHGWRZDUGV WKHPDMRULW\FODVVWKHUHIRUHWKHPLQRULW\FODVVLQVWDQFHVDUHPRUHOLNHO\WREHPLVFODVVLILHG 7KHUH DUH PDQ\ SUREOHPV WKDW DULVH IURP OHDUQLQJ ZLWK LPEDODQFHG GDWD VHWV 7KH ILUVW SUREOHP FRQFHUQV PHDVXUHV RI SHUIRUPDQFH (YDOXDWLRQ PHWULFV DUH NQRZQ WR SOD\ D YLWDO UROHLQPDFKLQHOHDUQLQJ7KH\DUHXVHGWRJXLGHWKHOHDUQLQJDOJRULWKPWRZDUGVWKHGHVLUHG VROXWLRQ 7KHUHIRUH LI WKH HYDOXDWLRQ PHWULF GRHV QRW WDNH WKH PLQRULW\ FODVV LQWR FRQVLGHUDWLRQWKHOHDUQLQJDOJRULWKPZLOOQRWEHDEOHWRFRSHZLWKFODVVLPEDODQFHYHU\ZHOO :LWK VWDQGDUG HYDOXDWLRQ PHWULFV VXFK DV WKH RYHUDOO FODVVLILFDWLRQ DFFXUDF\ WKH PLQRULW\ FODVVKDVOHVVLPSDFWFRPSDUHGWRWKHPDMRULW\FODVV7KHVHFRQGSUREOHPLVUHODWHGWRODFNRI GDWD ,Q DQ LPEDODQFHG WUDLQLQJ VHW D FODVV PD\ KDYH YHU\ IHZ VDPSOHV $V D UHVXOW LW LV GLIILFXOWWRFRQVWUXFWDFFXUDWHGHFLVLRQERXQGDULHVEHWZHHQFODVVHV)RUDFODVVFRQVLVWLQJRI PXOWLSOHFOXVWHUVVRPHFOXVWHUVPD\FRQWDLQDVPDOOQXPEHURIVDPSOHVFRPSDUHGWRRWKHU FOXVWHUV WKHUHIRUH WKH ODFN RI GDWD FDQ RFFXU ZLWKLQ WKH FODVV LWVHOI 7KH WKLUG SUREOHP LQ OHDUQLQJ IURP LPEDODQFHG GDWD LV QRLVH 1RLV\ GDWD KDYH D VHULRXV LPSDFW RQ PLQRULW\ FODVVHVWKDQRQPDMRULW\FODVVHV)XUWKHUPRUHVWDQGDUGPDFKLQHOHDUQLQJDOJRULWKPVWHQGWR WUHDWVDPSOHVIURPDPLQRULW\FODVVDVQRLVH ,QWKLVFKDSWHUZHUHYLHZWKHH[LVWLQJDSSURDFKHVIRUVROYLQJWKHFODVVLPEDODQFHSUREOHP DQGGLVFXVVWKHYDULRXVPHWULFVXVHGWRHYDOXDWHWKHSHUIRUPDQFHRIFODVVLILHUV)XUWKHUPRUH ZH LQWURGXFH D QHZ DSSURDFK WR GHDOLQJ ZLWK WKH FODVV LPEDODQFH SUREOHP E\ FRPELQLQJ ERWKXQVXSHUYLVHGDQGVXSHUYLVHGOHDUQLQJ7KHUHVWRIWKHFKDSWHULVRUJDQL]HGDVIROORZV 6HFWLRQGHVFULEHVWKHSUREOHPVFDXVHGE\FODVVLPEDODQFH6HFWLRQUHYLHZVFXUUHQWVWDWH RIWKHDUW WHFKQLTXHV IRU WDFNOLQJ WKHVH SUREOHPV 6HFWLRQ  GHVFULEHV H[LVWLQJ FODVVLILFDWLRQ SHUIRUPDQFH PHDVXUHV IRU LPEDODQFHG GDWD 6HFWLRQ  GHVFULEHV RXU SURSRVHG OHDUQLQJ DSSURDFK WR KDQGOH WKH FODVV LPEDODQFH SUREOHP 6HFWLRQ  SUHVHQWV H[SHULPHQWDO UHVXOWV DQG6HFWLRQJLYHVFRQFOXGLQJUHPDUNV



2CVVGTP4GEQIPKVKQP

%NCUU+ODCNCPEG2TQDNGOU &ODVV LPEDODQFH RFFXUV ZKHQ WKHUH DUH VLJQLILFDQWO\ IHZHU WUDLQLQJ LQVWDQFHV RI RQH FODVV FRPSDUHGWRRWKHUFODVVHV,QVRPHDSSOLFDWLRQVFODVVLPEDODQFHLVDQLQWULQVLFSURSHUW\)RU H[DPSOH LQ FUHGLW FDUG XVDJH GDWD WKHUH DUH YHU\ IHZ FDVHV RI IUDXG WUDQVDFWLRQV DV FRPSDUHGWRWKHQXPEHURIQRUPDOWUDQVDFWLRQV+RZHYHULPEDODQFHGGDWDFDQDOVRRFFXU LQDUHDVWKDWGRQRWKDYHDQLQKHUHQWLPEDODQFHSUREOHP,QVWHDGWKHLPEDODQFHLVPDLQO\ FDXVHGE\OLPLWDWLRQVLQFROOHFWLQJGDWDVXFKDVFRVWSULYDF\DQGWKHODUJHHIIRUWUHTXLUHGWR REWDLQ D UHSUHVHQWDWLYH GDWD VHW &ODVV LPEDODQFH SUHVHQWV VHYHUDO GLIILFXOWLHV LQ OHDUQLQJ LQFOXGLQJ LPEDODQFHG LQ FODVV GLVWULEXWLRQ ODFN RI GDWD DQG FRQFHSW FRPSOH[LW\  7KHVH IDFWRUVDUHH[SODLQHGLQPRUHGHWDLOLQWKHIROORZLQJVXEVHFWLRQV



+ODCNCPEGKPENCUUFKUVTKDWVKQP 7KHFODVVLPEDODQFHSUREOHPVFDQDULVHHLWKHUIURPEHWZHHQFODVVHV LQWHUFODVV RUZLWKLQD VLQJOHFODVV LQWUDFODVV :HILUVWGLVFXVVLVVXHVUHODWHGZLWKLQWHUFODVVLPEDODQFHZKHUHWKH QXPEHU RI H[DPSOHV RI RQH FODVV LV PXFK ODUJHU WKDQ WKH QXPEHU RI H[DPSOHV RI DQRWKHU FODVVQDPHO\WKHPLQRULW\FODVV7KHGHJUHHRILPEDODQFHFDQEHUHSUHVHQWHGE\WKHUDWLRRI VDPSOHVL]HRIWKHPLQRULW\FODVVWRWKDWRIWKHPDMRULW\FODVV0RVWFODVVLILFDWLRQWHFKQLTXHV VXFK DV GHFLVLRQ WUHH GLVFULPLQDQW DQDO\VLV DQG QHXUDO QHWZRUNV DVVXPH WKDW WKH WUDLQLQJ VDPSOHV DUH HYHQO\ GLVWULEXWHG DPRQJVW GLIIHUHQW FODVVHV +RZHYHU LQ UHDOZRUOG DSSOLFDWLRQVWKHUDWLRRIPLQRULW\WRPDMRULW\VDPSOHVFDQEHDVORZDVWRWRRU WR &KDZODHWDO +HQFHWKHVWDQGDUGFODVVLILHUVDUHDIIHFWHGE\WKHSUHYDOHQW FODVVHVDQGWHQGWRLJQRUHRUWUHDWWKHVPDOOFODVVHVDVQRLVH:HLVVDQG3URYRVWLQYHVWLJDWHG WKHUHODWLRQVKLSEHWZHHQWKHLPEDODQFHUDWLRRIWUDLQLQJVDPSOHVLQHDFKFODVVDQGFODVVLILHU SHUIRUPDQFHVLQWHUPVRIRYHUDOODFFXUDF\DQGDUHDXQGHUWKH52&FXUYH $8&  :HLVVDQG 3URYRVW   7KH\ XVHG D GHFLVLRQWUHH FODVVLILHU DQG WHVWHG LW RQ D QXPEHU RI GDWD VHWV IURP WKH 8&, 5HSRVLWRU\ $VXQFLRQ DQG 1HZPDQ   7KHLU H[SHULPHQWDO UHVXOWV LQGLFDWHG WKDW WKH UDWLR RI VDPSOHV LQ HDFK FODVV  GHSHQGV RQ WKH HYDOXDWLRQ PHWULFV XVHG :KHQ WKH SHUIRUPDQFH LV PHDVXUHG XVLQJ FODVVLILFDWLRQ DFFXUDF\ WKH EHVW UDWLR LV QHDU WKH QDWXUDOUDWLRRQWKHRWKHUKDQGZKHQWKH$8&PHDVXUHLVXVHGWKHEHVWUDWLRLVQHDUWKH EDODQFHG UDWLR 9LVD DQG 5DOHVFX DOVR UHSRUWHG VLPLODU UHVXOWV XVLQJ IX]]\ FODVVLILHUV 9LVD DQG5DOHVFX +RZHYHUZHVKRXOGQRWHWKDWWKHLPEDODQFHUDWLREHWZHHQFODVVHVLVQRW WKH RQO\ IDFWRU WKDW UHGXFHV FODVVLILFDWLRQ SHUIRUPDQFH RWKHU IDFWRUV VXFK DV WUDLQLQJ VL]H DQGFRQFHSWFRPSOH[LW\DOVRDIIHFWSHUIRUPDQFH  ,Q WDVNV WKDW LQYROYH OHDUQLQJ D FRQFHSW RU GHWHFWLQJ DQ HYHQW GDWD LPEDODQFH FDQ DSSHDU ZLWKLQ D VLQJOH FODVV 7KH ZLWKLQFODVV LPEDODQFH SUREOHP RFFXUV ZKHQ D FODVV FRQVLVWV RI VHYHUDOVXEFOXVWHUVRUVXEFRQFHSWVDQGWKHVHVXEFOXVWHUVGRQRWKDYHWKHVDPHQXPEHURI VDPSOHV -DSNRZLF]    7KH ZLWKLQFODVV DQG EHWZHHQFODVV LPEDODQFHV WRJHWKHU DUH NQRZQDVWKHSUREOHPRIVPDOOGLVMXQFWV +ROWHHWDO LQZKLFKFODVVLILHUVDUHELDVHG WRZDUGV  UHFRJQL]LQJ ODUJH GLVMXQFWV FRUUHFWO\ EXW RYHUILWWLQJ DQG PLVFODVVLI\LQJ VDPSOHV UHSUHVHQWHG E\ VPDOO GLVMXFWV  ,Q PRVW FODVVLILFDWLRQ WDVNV WKH SUHVHQFH RI ZLWKLQFODVV LPEDODQFH LV LPSOLFLW ,W LV NQRZQ WR KDYH QHJDWLYH HIIHFWV RQ WKH SHUIRUPDQFH RI VWDQGDUG FODVVLILHUV DQG LQFUHDVHV WKH FRPSOH[LW\ RI FRQFHSW OHDUQLQJ