AbstractâA brain-computer interface (BCI) based on motor ... computer screen for 3.5 seconds. During .... Average time course features over all the trials of each.
Proceedings of the 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference Shanghai, China, September 1-4, 2005
&RPPRQ6SDWLDO3DWWHUQ0HWKRGIRU&KDQQHO6HOHFWLRQLQ 0RWRU,PDJHU\%DVHG%UDLQFRPSXWHU,QWHUIDFH
@ $Q DGDSWLYH DXWRUHJUHVVLYH $$5 PRGHO ZDVXVHGIRUIHDWXUHH[WUDFWLRQ+5DPRVHUHWDOGHVLJQHG RSWLPDO VSDWLDO ILOWHUV E\ WKH PHWKRG RI FRPPRQ VSDWLDO SDWWHUQ&63 IRUILOWHULQJVLQJOHWULDO((*GXULQJLPDJLQHG KDQG PRYHPHQWV >@ % %ODQNHUW] HW DO VLJQLILFDQWO\ ERRVWHG 0,EDVHG %&, SHUIRUPDQFHV WKURXJK WKH FRPELQDWLRQ RI FODVVLILHUV WRZDUGV GLIIHUHQW SK\VLRORJLFDO SKHQRPHQDLH(5'DQG/53>@ 6R IDU 0,EDVHG %&, LV DQ LQGHSHQGHQW V\VWHP ZLWK KLJKHU FODVVLILFDWLRQ DFFXUDF\ )RU H[DPSOH WKH DYHUDJH DFFXUDF\ RI FODVVLI\LQJ LPDJLQHG OHIW DQG ULJKW KDQG PRYHPHQWVZDVDERYH>@>@+RZHYHU0,EDVHG%&, LV VWLOO QRW XVHG IRU UHDOOLIH DSSOLFDWLRQ 2QH LPSRUWDQW IDFWRULVWKDWPRVWDYDLODEOHDOJRULWKPVIRFXVRQDQDO\]LQJ PXOWLFKDQQHO ((* ZKLFK QHHGV UHODWLYHO\ H[SHQVLYH HTXLSPHQW LQFRQYHQLHQW UHFRUGLQJ SUHSDUDWLRQ DQG FRPSOLFDWHG FDOFXODWLRQ 7R GHVLJQ D SUDFWLFDO 0,EDVHG %&,RQHZD\LVWRVHOHFWIHZHUFKDQQHOVIRUDSSOLFDWLRQ ,QWKLVSDSHUZHSUHVHQWDPHWKRGIRUFKDQQHOVHOHFWLRQ LQ 0,EDVHG %&, 2SWLPDO FKDQQHOV ZHUH VHOHFWHG E\ VHDUFKLQJ WKH PD[LPXPV RI VSDWLDO SDWWHUQ YHFWRUV GHULYHG IURP &63 PHWKRG $QG WRZDUGV WKH IHZFKDQQHO ((* FODVVLILFDWLRQ D VLPSOLILHG DOJRULWKP ILW IRU RQOLQH DSSOLFDWLRQZDVSURSRVHGDQGYDOLGDWHG ,,0(7+2'2/2*@ 7KH PDLQ LGHDLVWRXVHDOLQHDUWUDQVIRUPWRSURMHFWWKHPXOWLFKDQQHO ((* GDWD LQWR ORZGLPHQVLRQDO VSDWLDO VXEVSDFH ZLWK D SURMHFWLRQPDWUL[RIZKLFKHDFKURZFRQVLVWVRIZHLJKWVIRU FKDQQHOV7KLVWUDQVIRUPDWLRQFDQPD[LPL]HWKHYDULDQFHRI WZRFODVV VLJQDO PDWULFHV &63 PHWKRG LV EDVHG RQ WKH VLPXOWDQHRXVGLDJRQDOL]DWLRQRI WKH FRYDULDQFH PDWULFHV RI ERWKFODVVHV 'HWDLOV RI WKH DOJRULWKP DUH GHVFULEHG DV IROORZV ZLWK WKHH[DPSOHRIFODVVLI\LQJVLQJOHWULDO((*GXULQJKDQGDQG IRRW PRYHPHQWV ;+ DQG ;) GHQRWH WKH SUHSURFHVVHG ((* PDWULFHV XQGHU WZR FRQGLWLRQV KDQG DQG IRRW ZLWK GLPHQVLRQV1h7ZKHUH1LVWKHQXPEHURIFKDQQHOVDQG7 LV WKH QXPEHU RI VDPSOHV SHU FKDQQHO 7KH QRUPDOL]HG VSDWLDOFRYDULDQFHRIWKH((*FDQEHUHSUHVHQWHGDV
5+
; + ; +7 5) WUDFH ; + ; +7
; ) ; )7 WUDFH ; ) ; )7
;7LVWKHWUDQVSRVHRI;DQGWUDFH$ FRPSXWHVWKHVXPRI WKH GLDJRQDO HOHPHQWV RI $ 7KH DYHUDJHG QRUPDOL]HG BBB BBB FRYDULDQFH 5 + DQG 5 ) DUH FDOFXODWHG E\ DYHUDJLQJ RYHU DOO WKH WULDOV RI HDFK JURXS 7KH FRPSRVLWH VSDWLDO FRYDULDQFHFDQEHIDFWRUL]HGDV
5
BBB
BBB
5+ 5 )
7
8 68
ZKHUH8LVWKHPDWUL[RIHLJHQYHFWRUVDQGȈLVWKHGLDJRQDO PDWUL[RIHLJHQYDOXHV7KHZKLWHQLQJWUDQVIRUPDWLRQPDWUL[
3
6 8
7
WUDQVIRUPVWKHDYHUDJHFRYDULDQFHPDWULFHVDV
6+
BBB
3 5+ 3 7 6 )
BBB
3 5) 3 7
6+ DQG 6) VKDUH FRPPRQ HLJHQYHFWRUV DQG WKH VXP RI FRUUHVSRQGLQJHLJHQYDOXHVIRUWKHWZRPDWULFHVZLOODOZD\V EHRQH
6+
86 +8 7 6 )
86 )8 7 6 + 6 )
,
7KH HLJHQYHFWRUV ZLWK WKH ODUJHVW HLJHQYDOXHV IRU 6+ KDYH WKH VPDOOHVW HLJHQYDOXHV IRU 6) DQG YLFH YHUVD 7KH WUDQVIRUPDWLRQ RI ZKLWHQHG ((* RQWR WKH HLJHQYHFWRUV FRUUHVSRQGLQJ WR WKH ODUJHVW HLJHQYDOXHV LQ Ȉ+ DQG Ȉ) LV RSWLPDOIRUVHSDUDWLQJYDULDQFHLQWZRVLJQDOPDWULFHV7KH SURMHFWLRQPDWUL[:LVGHQRWHGDV :
8 7 3
:LWK WKH SURMHFWLRQ PDWUL[ : WKH RULJLQDO ((* FDQ EH WUDQVIRUPHGLQWRXQFRUUHODWHGFRPSRQHQWV = :; = FDQ EH VHHQ DV ((* VRXUFH FRPSRQHQWV LQFOXGLQJ FRPPRQ DQG VSHFLILF FRPSRQHQWV RI GLIIHUHQW WDVNV 7KH RULJLQDO((*;FDQEHUHFRQVWUXFWHGE\
;
: =
ZKHUH : LV WKH LQYHUVH PDWUL[RI : 7KHFROXPQV RI : DUHVSDWLDOSDWWHUQVZKLFKFDQEHFRQVLGHUHGDV((*VRXUFH GLVWULEXWLRQ YHFWRUV 7KH ILUVW DQG ODVW FROXPQV RI : DUH WKH PRVW LPSRUWDQW VSDWLDO SDWWHUQV WKDW H[SODLQ WKH ODUJHVW YDULDQFHRIRQHWDVNDQGWKHVPDOOHVWYDULDQFHRIWKHRWKHU '&KDQQHOVHOHFWLRQ 7 1 /DO HW DO XVHG WKH IHDWXUH VHOHFWLRQ DOJRULWKPV EDVHG RQ WKH WUDLQLQJ RI VXSSRUW YHFWRU PDFKLQHV IRU WKH SXUSRVH RI VHOHFWLQJ FKDQQHOV IRU FODVVLI\LQJ ((* GXULQJ PHQWDO WDVNV >@ 7KHLU VWXGLHV VKRZHG WKDW WKH QXPEHU RI FKDQQHOVFRXOGEHUHGXFHGVLJQLILFDQWO\ZLWKRXWGHFUHDVLQJ WKH FODVVLILFDWLRQ UDWH DQG DQ DSSURSULDWH QXPEHU RI FKDQQHOV FRXOG KDYH QHDUO\ WKH VDPH DFFXUDF\ DV WKDW DFTXLUHG E\ DOO FKDQQHOV %\ UHGXFLQJ WKH QXPEHU RI FKDQQHOVWKHUHFRUGLQJSUHSDUDWLRQFDQEHPRUHFRQYHQLHQW DVZHOODVWKHFDOFXODWLRQWLPHFDQEHHQRUPRXVO\GHFUHDVHG 6SDWLDOSDWWHUQVGHULYHGIURP&63PHWKRGFDQEHVHHQ DV ((* VRXUFH GLVWULEXWLRQ YHFWRUV :H DVVXPH WKDW RQO\ WKHILUVWDQGODVWVSDWLDOSDWWHUQVDUHUHODWHGZLWKWKHVSHFLILF VRXUFHV RI WKH WZR WDVNV 7KHQ WKH WZR FKDQQHOV FRUUHVSRQGLQJWRWKHPD[LPDOFRHIILFLHQWVRIVSDWLDOSDWWHUQ YHFWRUV PD\ EH WKH FKDQQHOV PRVW FRUUHODWHG ZLWK WKH WDVN VSHFLILF VRXUFHV :H DSSOLHG WKLV LGHD WR VHDUFK IRU WZR SDLUV RI RSWLPDO FKDQQHOV FRUUHVSRQGLQJ WR &63 DQDO\VHV RQ(5'DQG53UHVSHFWLYHO\7KHUHVXOWLQJFKDQQHOVDJUHHG ZHOO ZLWK WKH H[SHFWHG XQGHUO\LQJ FRUWLFDO DFWLYLW\ GLVWULEXWLRQVGXULQJWKH0,WDVNV )LJD GLVSOD\V WKH WZR PRVW LPSRUWDQW VSDWLDO SDWWHUQV EDVHG RQ (5' RI RQH VXEMHFW 7KH (5' RI LPDJLQHG ULJKW KDQG PRYHPHQWV DSSHDUV GRPLQDQWO\ RYHU OHIW KHPLVSKHUH WKH IRRW DUHD ORFDOL]HV RQ WKH FHQWUDO DUHD EHWZHHQ ERWK KHPLVSKHUHV 7KHUHIRUH ULJKW KDQG LPDJHU\ FDXVHVDUHODWLYHO\LQFUHDVHG((*SRZHURYHUWKHIRRWDUHD
5393
DQG IRRW LPDJHU\ KDV DQ LQFUHDVHG ((* YDULDQFH RYHU WKH ULJKWKDQGDUHD7KDWLVWRVD\WKHVSDWLDOSDWWHUQRIRQHWDVN HVVHQWLDOO\ SUHVHQWV WKH (5' GLVWULEXWLRQ RI WKH RWKHU WDVN )LJE VKRZVWKHVSDWLDOSDWWHUQVGHULYHGIURP53ZKLFK DUHREYLRXVO\GLIIHUHQWIURPWKRVHRI(5'7RVRPHH[WHQW LWVXJJHVWVWKDW(5'DQG53DUHLQGHSHQGHQWSK\VLRORJLFDO SKHQRPHQDZLWKGLIIHUHQWVSDWLDOGLVWULEXWLRQV 7KH VSDWLDO SDWWHUQ RI KDQG LPDJHU\ LV JLYHQ DV 63+ DQG 63) IRU IRRW 7KH RSWLPDO FKDQQHOV FDQ EH GHWHUPLQHG WKURXJK VHDUFKLQJ WKH PD[LPXPV RI WKH DEVROXWH YDOXH RI 63+DQG63) &++ ILQG_63+_
0D[_63+_
&+) ILQG_63)_
0D[_63)_
XQGHUWZRFRQGLWLRQV2Q&KDQG(5'VKRZVPXFKORZHU DPSOLWXGH %XW RQ &3] IRRW (5' LV VOLJKWO\ ORZHU )LJ SUHVHQWVDYHUDJH53VRQ&]DQG)&]$PSOLWXGHRIIRRW53 LVORZHUWKDQKDQG53RQERWKFKDQQHOV,WPD\VXJJHVWWKDW 53 RI IRRW LPDJHU\ LV PRUH VLJQLILFDQW WKDQ WKDW RI KDQG LPDJHU\ 7KHDPSOLWXGHGLIIHUHQFHVRIWLPHFRXUVH(5'DQG53 EHWZHHQ KDQG DQG IRRW PRYHPHQWV HQVXUH WKH VXEVWDQWLDO GLVFULPLQDELOLW\RIWKHVHWZRVWDWHV)HDWXUHVFRUUHVSRQGLQJ WR(5'DQG53FDQEHGHILQHGDV>@>@ I(5' PHDQ_[&(5'_ I(5' PHDQ_[&3](5'_ I53 PHDQ[&]53 I53 PHDQ[)&]53
ZKHUHILQG ILQGVLQGLFHVRIHOHPHQWV)RUWKLVVXEMHFWWKH RSWLPDOFKDQQHOVIRU(5'DUH&DQG&3]DQG&]DQG)&] IRU536HH)LJ 7KHVHIRXUFKDQQHOVDUHXVHGIRUIHDWXUH H[WUDFWLRQDQGFODVVLILFDWLRQSURFHGXUHV
ZKHUH[GHQRWHVSUHSURFHVVHG((*VLJQDORIRQHFKDQQHO DQGPHDQ[ FDOFXODWHVWKHPHDQYDOXHRI[
)LJ $YHUDJH WLPH FRXUVH (5'V RQ & DQG &3] ³´ GHQRWHV IRRW LPDJHU\DQG³²´GHQRWHVKDQGLPDJHU\
)LJ6SDWLDO PDSSLQJVRIWKHWZRVSDWLDOSDWWHUQVFRUUHVSRQGLQJWRULJKW KDQG DQG IRRW LPDJHU\ WDVNV EDVHG RQ D (5' DQG E 53 UHVSHFWLYHO\ 7KHWKLFNGRW³g´LQGLFDWHVWKHVHOHFWHGRSWLPDOFKDQQHO
()HDWXUHH[WUDFWLRQ $YHUDJHWLPHFRXUVHIHDWXUHVRYHUDOOWKHWULDOVRIHDFK JURXS FDQ FRQWULEXWH WR FRQVLGHULQJ WKH IHDWXUH H[WUDFWLRQ PHWKRG 5HDGLQHVV SRWHQWLDO LV RQH FRPSRQHQW RI PRYHPHQWUHODWHG SRWHQWLDOV 053V DQG LW FDQ EH H[WUDFWHG E\ DYHUDJLQJ WHFKQLTXHV +RZHYHU (5' LV QRW SKDVHGORFNHGDQGWKXVZLOOEHHOLPLQDWHGE\DYHUDJLQJ>@ )RUWXQDWHO\ LW FDQ EH GHWHFWHG E\ SRZHU DQDO\VLV 7LPH FRXUVH (5' FDQ EH SUHVHQWHG DV SRZHU VDPSOHV REWDLQHG WKURXJK FDOFXODWLQJ WKH DEVROXWH YDOXH RI WKH DPSOLWXGH VDPSOHV $V VKRZQ LQ )LJ DYHUDJH WLPH FRXUVH (5'V RQ WKH RSWLPDO FKDQQHOV & DQG &3] GLVSOD\ GLIIHUHQW SDWWHUQV
)LJ$YHUDJHWLPHFRXUVH53VRQ&]DQG)&]³´GHQRWHVIRRWLPDJHU\ DQG³²´GHQRWHVKDQGLPDJHU\
)&ODVVLILFDWLRQDQGFURVVYDOLGDWLRQ $ ZHOO NQRZQ OLQHDU FODVVLILFDWLRQ PHWKRG )LVKHU 'LVFULPLQDQW )' LV XVHG DV WKH FODVVLILHU 7KH IHDWXUH YHFWRUV S(5' >I(5' I(5'@7 DQG S53 >I53 I53@7 DUH GHILQHG IRU WUDLQLQJ RI WZR FODVVLILHUV 7KH RXWSXWV DUH FDOFXODWHGDV D(5' :(5' 7S(5'E(5'D53 :53 7S53E53 ZKHUH:DQGEDUHWKHZHLJKWVDQGRIIVHWGHWHUPLQHGE\WKH WUDLQLQJGDWD D(5' DQGD53 GHQRWH WKH FODVVLILFDWLRQ UHVXOWV EDVHG RQ (5' DQG 53 UHVSHFWLYHO\ 7KH ILQDO GHFLVLRQ LV PDGHE\VXPPDUL]LQJWKHWZRZHLJKWHGRXWSXWVDV
5394
D VJQZ(5'D(5'Z53D53
7KHZHLJKWVZ(5'DQGZ53DUHGHILQHGDV (5'
Z
DFF
(5'
P
53
Z
53
P
DFF
ZKHUH DFF LV WKH FODVVLILFDWLRQ DFFXUDF\ REWDLQHG IURP WKH WUDLQLQJ VWDJH DQG P LV D FRQWURO SDUDPHWHU XVHG WR HPSKDVL]HWKHIHDWXUHZLWKKLJKHUDFFXUDF\ $ hIROG FURVVYDOLGDWLRQ LV XVHG WR HVWLPDWH WKH FODVVLILFDWLRQDFFXUDF\ ,,,5(68/76 7KH FODVVLILFDWLRQ DFFXUDF\ UDWHV RI WZR VXEMHFWV DUH OLVWHG LQ 7DEOH ZKLFK FRQVLVWV RI WKUHH SDUWV )LUVW ³ FKQ´UHIHUVWRRQO\PDNLQJXVHRIWKHIRXURSWLPDOFKDQQHOV WR GR FODVVLILFDWLRQ 7KH FRPELQDWLRQ RI (5' DQG 53 UHVXOWHG LQ D KLJK DFFXUDF\ IRU WKH WZR VXEMHFWV DQG 6HFRQG ³FKQ´ GHQRWHV DSSO\LQJ WKH RSWLPDO FKDQQHOV LQFOXGLQJ DGGLWLYH FKDQQHOV FRUUHVSRQGLQJ WR WKH VHFRQG PD[LPXPV RI VSDWLDO SDWWHUQ YHFWRUV IRU FODVVLILFDWLRQ 7KH DGGLWLYH FKDQQHOV FRQWULEXWHG WR LQFUHDVH WKH FODVVLILFDWLRQ DFFXUDF\ RI VRPH SHUFHQWEXWGHFUHDVLQJWKHFRQYHQLHQFHRIV\VWHPRSHUDWLRQ DW WKH VDPH WLPH )RU WKHVHWZR VXEMHFWVFKDQQHOUHVXOWV KDYH DOUHDG\ EHHQ YHU\ FORVH WR WKRVH RI PXFK PRUH FKDQQHOV )LQDOO\ ³VWRV´ PHDQV FODVVLILFDWLRQ XVLQJ VXEMHFWWRVXEMHFW FKDQQHO FRQILJXUDWLRQ )RU VXEMHFW 6 WKHDFFXUDF\GHFUHDVHGIURPWRGLVSOD\LQJ WKHDGYDQWDJHRIVXEMHFWVSHFLILFFKDQQHOVHOHFWLRQ 7$%/(,
PHWKRG 7KH IHDWXUH H[WUDFWLRQ DQG FODVVLILFDWLRQ PHWKRG EDVHGRQOLQHDUGLVFULPLQDQWDQDO\VLVPD\EHDSSOLFDEOHIRU RQOLQH DQDO\VLV ZLWK DQ HDV\ FDOFXODWLRQ 7KH SUHVHQW PHWKRGPD\KHOSWRFRQVWUXFWDSUDFWLFDO0,EDVHG%&,,Q DGGLWLRQFRQWLQXRXVFODVVLILHURXWSXWVFRXOGEHHDVLO\XVHG DV WKH IHHGEDFN WR UHDOL]H DQ DV\QFKURQRXV %&, ZKLFK DOORZVWKHXVHUWRRSHUDWHWKHV\VWHPDWDQ\PRPHQW>@ $&.12:/('*0(17
7KLV SURMHFW LV VXSSRUWHG E\ 1DWLRQDO 1DWXUDO 6FLHQFH )RXQGDWLRQ RI &KLQD DQG WKH .H\ 3URMHFW RI &KLQHVH0LQLVWU\RI(GXFDWLRQ12 $QGZHDUH JUDWHIXO WR .ODXV5REHUW 0OOHU %HQMDPLQ %ODQNHUW] DQG *DEULHO&XULRIRUSURYLGLQJWKHLUGDWD 5()(5(1&(6
&/$66,),&$7,21$&&85$&@ %%ODQNHUW].50OOHU*&XULR709DXJKDQ*6FKDON - 5 :ROSDZ $ 6FKORHJO & 1HXSHU * 3IXUWVFKHOOHU 7 +LQWHUEHUJHU 0 6FKURHGHU DQG 1 %LUEDXPHU ³7KH %&, &RPSHWLWLRQ SURJUHVV DQG SHUVSHFWLYHV LQ GHWHFWLRQ DQG GLVFULPLQDWLRQRI((*VLQJOHWULDOV´,(((7UDQV%LRPHG(QJ YROQRSS >@ * 3IXUWVFKHOOHU DQG ) + /RSHV GD 6LOYD ³(YHQWUHODWHG ((*0(* V\QFKURQL]DWLRQ DQG GHV\QFKURQL]DWLRQ EDVLF SULQFLSOHV´&OLQ1HXURSK\VLROYROSS >@ .% (%|FNHU &+ 0%UXQLD DQG3-0&OXLWPDQV ³$ VSDWLRWHPSRUDO GLSROH PRGHO RI WKH UHDGLQHVV SRWHQWLDO LQ KXPDQV ,, )RRW PRYHPHQW´ (OHFWURHQFHSKDORJU &OLQ 1HXURSK\VLROYROSS >@ * 3IXUWVFKHOOHU & 1HXSHU $ 6FKORJO DQG . /XJJHU ³6HSDUDELOLW\ RI ((* VLJQDOV UHFRUGHG GXULQJ ULJKW DQG OHIW PRWRULPDJHU\XVLQJDGDSWLYHDXWRUHJUHVVLYHSDUDPHWHUV´,((( 7UDQV5HKDE(QJYROQRSS >@ + 5DPRVHU - 0 *HUNLQJ DQG * 3IXUWVFKHOOHU ³2SWLPDO VSDWLDO ILOWHULQJ RI VLQJOH WULDO ((* GXULQJ LPDJLQHG KDQG PRYHPHQW´,(((7UDQV5HKDE(QJYROQRSS >@ * 'RUQKHJH % %ODQNHUW] * &XULR DQG . 5 0OOHU ³%RRVWLQJ ELW UDWHV LQ QRQLQYDVLYH ((* VLQJOHWULDO FODVVLILFDWLRQV E\ IHDWXUH FRPELQDWLRQ DQG PXOWLFODVV SDUDGLJPV´ ,((( 7UDQV %LRPHG (QJ YRO QR SS >@ 71/DO06FKURGHU7+LQWHUEHUJHU-:HVWRQ0%RJGDQ 1 %LUEDXPHU DQG % 6FKRONRSI ³6XSSRUW YHFWRU FKDQQHO VHOHFWLRQ LQ %&,´ ,((( 7UDQV %LRPHG (QJ YRO QR SS >@ % %ODQNHUW] * &XULR DQG . 5 0OOHU ³&ODVVLI\LQJ VLQJOH WULDO ((* WRZDUGV EUDLQ FRPSXWHU LQWHUIDFLQJ´ ,Q 7 * 'LHWWULFK 6 %HFNHU DQG = *KDKUDPDQL HGV $GYDQFHV LQ 1HXUDO,QI3URF6\VWHPV1,36 YROSS >@