Human-machine Interaction Based on Voice

Available online at www.sciencedirect.com

AASRI Procedia 3 (2012) 583 – 588



$$65,&RQIHUHQFHRQ0RGHOLQJ,GHQWLILFDWLRQDQG&RQWURO

+XPDQPDFKLQH,QWHUDFWLRQ%DVHGRQ9RLFH -LDQL/XD+XD4LDQD $LSLQJ;LDRD0LDRZHQ6KLE D

6FKRRORI7HFKQRORJ\%HLMLQJ)RUHVWU\8QLYHUVLW\%HLMLQJ&KLQD E 6FKRRORI6RIWZDUH(QJLQHHULQJ %HLMLQJ8QLYHUVLW\RI3RVWVDQG7HOHFRPPXQLFDWLRQV%HLMLQJ&KLQD 

$EVWUDFW 7KHKXPDQPDFKLQHV\VWHPKROGVWKHLPSRUWDQWSRVLWLRQLQHYHU\ILHOGRIPDQDQGPDFKLQH,WLVDOVRDNH\WHFKQRORJ\IRU PDQWRRSHUDWHRQHPDFKLQH9RLFHEDVHGLGHQWLILFDWLRQLVRQHRISRSXODUZD\WRFRQWDFWWKHXVHUDQGGHYLFH,WKDVEHHQ XVHG ZLGHO\ LQ PDQ\ DUHDV ZLWK WKH GHYHORSPHQW RI KDUGZDUH WHFKQRORJLHV DQG WKH LPSURYHPHQW RI PDWKHPDWLFDO DOJRULWKPV7KHDUWLFOHKHUHHODERUDWHVWKHWKHRU\DQGDOJRULWKPVZKLFKXVHGLQWKHV\VWHPDWWKHVDPHWLPHVKRZVWKH UHVXOWVRIH[SHULPHQW

 ‹3XEOLVKHGE\(OVHYLHU%96HOHFWLRQDQGRUSHHUUHYLHZXQGHUUHVSRQVLELOLW\RI$PHULFDQ$SSOLHG © 2012 The Authors. Published by Elsevier B.V. Selection and/or peer review under responsibility of American Applied Science Research Institute 6FLHQFH5HVHDUFK,QVWLWXWH  .H\ZRUGV0DQPDFKLQHLQWHUDFWLRQ9RLFHUHFRJQLWLRQ3UHWUHDWPHQW)HDWXUHH[WUDFWLRQ$OJRULWKPV

,QWURGXFWLRQ 7KHGHILQLWLRQVRIPDQPDFKLQHLQWHUDFWLRQDUHH[WHQVLYH6LPSO\VWDWHGLWLVDYHU\LPSRUWDQWWHFKQRORJ\ IRUPDQWRRSHUDWHRQHPDFKLQH:LWKDOONLQGVRIGHYLFHVZHFDQVHHWKDWWKHYDULHW\RILQWHUDFWLRQVPXVWEH ULFK ,Q  %LOO *DWHV WKH &(2 RI WKH 0LFURVRIW &RUSRUDWLRQ SXWV IRUZDUG WKH FRQFHSW RI ³QDWXUDO XVHU LQWHUIDFH´DQGSUHGLFWVWKHPRGHRIKXPDQFRPSXWHULQWHUDFWLRQZLOOEHFKDQJHGDORWLQWKHQH[WIHZ\HDUV +HWROGWKHUHSRUWHUVWKHNH\ERDUGDQGWKHPRXVHZLOOUHSODFHGE\WRXFKVFUHHQYRLFHUHFRJQLWLRQPDFKLQH YLVLRQ1RZDGD\V PRUHVLPSOHDQG PRUHFRQYHQLHQWEXW YHU\HIIHFWLYHFRQWURO PHWKRGEHFRPHVDWUHQG $  

&RUUHVSRQGLQJDXWKRU+XD4LDQ7HO (PDLODGGUHVVTLDQKXD#EMIXHGXFQ



2212-6716 © 2012 The Authors. Published by Elsevier B.V. Selection and/or peer review under responsibility of American Applied Science Research Institute doi:10.1016/j.aasri.2012.11.092

584

Jia-ni Lu et al. / AASRI Procedia 3 (2012) 583 – 588

VHULHVRIDUWLILFLDOLQWHOOLJHQFH PHWKRGVLQFOXGLQJYRLFHUHFRJQLWLRQDQG PDFKLQH YLVLRQ KDVEHHQDSSOLHGWR UHDOLW\7KLVSDSHUUHSRUWVDVWXG\RQWKHNH\WHFKQRORJLHVRIYRLFHUHFRJQLWLRQ $OJRULWKPDQG5HDOL]DWLRQ 6SHHFKUHFRJQLWLRQXVHVYRLFHVLJQDOVDVLQSXWLQVWHDGRIWUDGLWLRQDONH\ERDUGPRXVHDQGWRXFKVFUHHQ,WLV PRUHFRQYHQLHQWDQG IDVWHU,WLVDOVRDNLQGRIWHFKQRORJ\ PDNHVWKH PDFKLQHWRDGDSWWRSHRSOH:LWKWKH GHYHORSPHQWRIKDUGZDUHWHFKQRORJLHVDQGWKHLPSURYHPHQWRIPDWKHPDWLFDODOJRULWKPVLWEHFRPHVPRUHDQG PRUHFRQVXPPDWH 7KHVSHHFKUHFRJQLWLRQLVDNLQGRISDWWHUQUHFRJQLWLRQHVVHQWLDOO\)HDWXUHH[WUDFWLRQSDWWHUQPDWFKLQJ DQGUHIHUHQFHOLEUDU\DUHWKHEDVLFXQLWVRILW7KHSURFHVVRIWKHV\VWHPLVVKRZQDVILJXUH>@

 )LJ3URFHVVRIWKHYRLFHUHFRJQLWLRQ

3UHWUHDWPHQW %HIRUH UHFRJQLWLRQ ZH PXVW GR VRPH SUHWUHDWPHQWV IRU YRLFH VLJQDO ILUVWO\ ZKLFK LQFOXGH SUHHPSKDVLV IUDPHVZLQGRZHGDQGYRLFHDFWLYLW\GHWHFWLRQ(DFKVWHSKDVDJUHDWLPSDFWRQWKHSHUIRUPDQFHRIWKHHQWLUH V\VWHP )XQFWLRQV RI SUHHPSKDVLV DUH LPSURYH WKH KLJK IUHTXHQF\ SDUW RI VLJQDO %HFDXVH RI WKH HQHUJ\ RI WKH VSHHFK VLJQDO DWWHQXDWHV ZKHQ WKH IUHTXHQF\ LQFUHDVHV ZH FDQ XVH SUHHPSKDVLV WR FRPSHQVDWH WKH KLJK IUHTXHQF\ SDUW RI VLJQDO ZKLFK GHSUHVVHG E\ YRFDO RUJDQV 7KH IRUPXOD QDPHG ³ILUVW RUGHU ]HURHV GLJLWDO ILOWHU´>@LVXVHGPRVWIUHTXHQWO\

+  ] =  − E] −  E = >@   7KH³E´LVFDOOHGWKHSUHHPSKDVLVFRHIILFLHQW)LJXUHVKRZVWKHRVFLOORJUDPRI³WHOODMRNH´EHIRUHDQG DIWHU3UHHPSKDVLV

 )LJ7KHRVFLOORJUDPRI³WHOODMRNH´EHIRUHDQGDIWHU3UHHPSKDVLV



585

Jia-ni Lu et al. / AASRI Procedia 3 (2012) 583 – 588

,QIDFWZHGLYLGHWKHYRLFHVLJQDOVLQWRVRPHWLPHIUDJPHQWVDERXWaPV FDOOHG³IUDPH´ZKHQZH GHDOZLWKLW7KHIUDPHLVWKHVPDOOHVWXQLWRIVLJQDOSURFHVVLQJ%HWZHHQWKHIUDPHDQGWKHDGMDFHQWIUDPH WKHUHLVDFHUWDLQRYHUODSJHQHUDOO\RYHUODSSHGE\RU7KHQRQRYHUODSSLQJSDUWLVFDOOHGWKHIUDPH VKLIW,QJHQHUDOIUDPHOHQJWKWDNHVPVZKHQIUDPHVKLIWWDNHVPV$QGWKHQZHVKRXOGZLQGRZHGWKH VLJQDOLQRUGHUWRPDNHWKHPDLQOREHREYLRXVO\DQGPLQRUOREHOHVVREYLRXVO\7KHUHDVRQIRUGRLQJWKLVLVWR HOLPLQDWH WKH GLVFRQWLQXLW\ RI VSHHFK IUDPH DW WKH OHIW DQG ULJKW +DPPLQJ ZLQGRZ LV WKH PRVW FRPPRQO\ XVHGZLQGRZIXQFWLRQ>@  

­  −  FRV πQ :  Q = ® 1 − ¯ 

 < Q < 1 −   Q = RWKHU

)LJXUHVKRZVWKHRVFLOORJUDPRI³WHOODMRNH´PVDEVWUDFWLRQEHIRUHDQGDIWHUDGGLQJDKDWFK

 )LJ7KHRVFLOORJUDPRI³WHOODMRNH´PVDEVWUDFWLRQEHIRUHDQGDIWHUDGGLQJDKDWFK

9RLFHDFWLYLW\GHWHFWLRQLVDFULWLFDOVWHSLQWKHSUHWUHDWPHQWRIWKHVLJQDORQO\LIWKHGHYLFHGHWHUPLQHV WKH HQGSRLQW RI WKH VLJQDO FRUUHFWO\ FDQ LW GHDO ZLWK WKH YRLFH E\ UXOH DQG OLQH :H FKRRVH GXDO WKUHVKROG PHWKRGIRUYRLFHDFWLYLW\GHWHFWLRQ7KHUHDUHGLIIHUHQFHVEHWZHHQWZRWHUPVVKRUWWHUPHQHUJ\DQDO\VLVDQG VKRUWWHUP]HURUDWHDOJRULWKPDQDO\VLV)RUWKHQIUDPHVLJQDO [Q P WKHIRUPXODRIVKRUWWHUPHQHUJ\LVDV IROORZ



(Q =

∞



¦> [P ZQ − P @

 

P=−∞

$QGVKRUWWHUP]HURUDWHDOJRULWKPLVORRNWKLV

=Q =

∞

¦ _ VJQ> [P @ − VJQ> [P −  @ _ ZQ − P  

P= −∞

6JQPHDQVDVLJQXPH[SUHVVHGDVIXQFWLRQ







586

Jia-ni Lu et al. / AASRI Procedia 3 (2012) 583 – 588

­  :  Q = ® ¯ −

[  Q ≥  [  Q < 

 

%\XVLQJWKHGXDOWKUHVKROGPHWKRGIRUYRLFHDFWLYLW\GHWHFWLRQZHVKRXOGVHWWZRWKUHVKROGVIRUVKRUW WHUPHQHUJ\DQDO\VLVDQGVKRUWWHUP]HURUDWHDOJRULWKPDQDO\VLVRQHKLJKDQGRQHORZIRUHDFK7KHORZRQH LVVHQVLWLYHIRUVLJQDOFKDQJHVVRLWLVHDV\WREHRYHUVWHSSHGEXWWKHKLJKRQHRQO\FDQEHRYHUVWHSSHGE\WKH VLJQDOZKLFKUHDFKHVWRVRPHLQWHQVLW\7KHPHWKRGLVGLYLGHGLQWRIRXUVWDJHVPXWHWUDQVLWLRQYRLFHDQGHQG 'XULQJWKHPXWHLIRQHRIWKHHQHUJ\RUWKH]HURUDWHRYHUWKHORZWKUHVKROGWKHIODJZLOOEHFKDQJHG7KLV LQGLFDWHVWKHSURFHVVHQWHUVWKHWUDQVLWLRQEXWQRZZHFDQQRWEHVXUHZKHWKHUWKHUHUHDOO\LVDYRLFHVLJQDORU QRWWKHUHIRUHLIWKHQXPHULFDOYDOXHGRZQWREHORZWKHORZWKUHVKROGWKHFXUUHQWVWDWHJRHVEDFNWRWKHPXWH %XWLIDQ\RQHRIWKHSDUDPHWHURYHUVWHSVWKHKLJKWKUHVKROGZHFDQGHWHUPLQHWKDWLWJHWLQWRWKHYRLFHSDUW $WWKLVSDUWLIWRZSDUDPHWHUVGRZQWREHORZWKHWKUHVKROGDWWKHVDPHWLPHWKHWLPHLVVKRUWHUWKDQWKHWLPH WKUHVKROGZHEHOLHYHWKDWWKLVVSHHFKVLJQDOLVQRLVH,IQRWZHODEHOWKHHQGSRLQWVDQGUHWXUQ )HDWXUHH[WUDFWLRQ :H FKRRVH WKH 0HO )UHTXHQF\ &HSVWUXP &RHILFLHQWV0)&&  WR GR WKH IHDWXUH H[WUDFWLRQ IURP PDQ\ NLQGVRIPHWKRGV7KHWKHRU\LVEDVHGRQOLQHDUSUHGLFWLYHFRGLQJ/3& DFFRUGLQJWRSKRQDWLRQPHFKDQLVP DQGSV\FKRORJ\PRGHORIDXGLWRU\V\VWHP0HOIUHTXHQF\FDQEHIRUPXODWHGDVIROORZ

I PHO =  ORJ +

I   

7KH0)&&FRQLVLWVE\IRXUVWHSV • )DVW)RXULHU7UDQVIRUP))7 7KHFKDUDFWHULVWLFVRIWKHYRLFHVLJQDOUHIOHFWHGE\WKHHQHUJ\GLVWULEXWLRQLQ WKH IUHTXHQF\ GRPDLQ $IWHU HDFK IUDPH RI VSHHFK VLJQDO LV PXOWLSOLHG E\ D +DPPLQJ ZLQGRZ LW DOVR QHHGVWRSDVVWKH))7WRJHWWKHHQHUJ\GLVWULEXWLRQLQWKHIUHTXHQF\GRPDLQ • 7ULDQJXODUEDQGSDVVILOWHU/HW VDVVXPHWKHQXPEHURIRYHUODSSLQJWULDQJXODUEDQGSDVVILOWHULV07KHQ ZHFDQILJXUHRXWWKH.WKRXWSXWRIWKHILOWHULV[N 'LDJUDPPDWLFVNHWFKRIWULDQJOHEDQGSDVVILOWHULV VKRZQDV)LJ

 )LJ'LDJUDPPDWLFVNHWFKRIWULDQJOHEDQGSDVVILOWHU

• ,QYHUVHGLVFUHWHFRVLQHWUDQVIRUP)LUVWO\WDNHWKHORJRIWKHRXWSXWRIHDFK ILOWHUWRJHWSRZHUVSHFWUXP WKHQXVHWKHLQYHUVHGLVFUHWHFRVLQHWUDQVIRUPLQRUGHUWRJHWVHYHUDO0)&&FRHIILFLHQWVILQDOO\WDNHDERXW aFRHIILFLHQWV



Jia-ni Lu et al. / AASRI Procedia 3 (2012) 583 – 588

• 7KH0)&&IHDWXUHVZLOOEHUHJDUGDVVWDWLFFKDUDFWHULVWLFVDQGWKHQGRWKHILUVWRUGHUDQGVHFRQGRUGHU GLIIHUHQWLDORQWKHPWRVXPPDUL]HWKHG\QDPLFFKDUDFWHULVWLFV 5HFRJQLWLRQ$OJRULWKPV :H WDNH WKH +LGGHQ 0DUNRY 0RGHOV +00  WR DFKLHYH D QRQVSHFLILF UHFRJQLWLRQ V\VWHP +00 LV D SUREDELOLW\ PRGHO IRU WKH VWDWLVWLFDO SURSHUWLHV RI WKH UDQGRP SURFHVV HYROYHG E\ WKH 0DUNRY FKDLQ $ FRPSOHWH+00QHHGVWREHJLYHQVRPHSDUDPHWHUVDUHDVIROORZVVWDWHQXPEHU1WKHQXPEHURIREVHUYDWLRQ V\PEROV0DQGWKHDJJUHJDWLRQDVZHOODVWKHSUREDELOLW\GLVWULEXWLRQ$%SZKHUH$LVWKHVWDWHWUDQVLWLRQ SUREDELOLW\PDWUL[%LVWKHREVHUYDWLRQV\PEROSUREDELOLW\PDWUL[SLVWKHLQLWLDOVWDWHSUREDELOLWLHV7\SLFDOO\ WKHIROORZLQJIRUPXODVDUHKHUHWRUHSUHVHQWDSDUWLFXODUPRGHO

O =  S $ %   7KH+00FDQEHWKRXJKWRIDVWKHJHQHUDWRURIRQHREVHUYDWLRQVHTXHQFHREVHUYDWLRQVHTXHQFH4 ZKHUH 7LVWKHOHQJWK4H[SUHVVLRQVDUHDVIROORZ

4 = 4  4  4 !47   )RU+00PRGHOWKHUHDUHWKUHHEDVLFTXHVWLRQVQHHGWREHDGGUHVVHG • &DOFXODWHWKHRXWSXWSUREDELOLW\*LYHQWKHREVHUYHGVHTXHQFH 4 DQGWKHPRGHO O KRZWRFDOFXODWHWKH RXWSXWSUREDELOLW\ S4 _ O TXLFNO\DQGHIILFLHQWO\ • )LQGWKHEHVWVWDWHVHTXHQFH1RZ 4 DQG O DUHNQRZQKRZWRILQGRXWWKHEHVWVWDWHVHTXHQFH • 0RGHOWUDLQLQJ+RZWRDGMXVWWKHPRGHO O VXFKWKDWWKHFRQGLWLRQDOSUREDELOLW\ S4 _ O LVWKHPD[LPXP &RQVLGHURQHSUREOHPRIDQRQVSHFLILFDQG1ZRUGLGHQWLILFDWLRQ)RUHDFK ZRUGLQWKH ZRUGWDEOH ZH QHHGWRGHVLJQDQ;VWDWH+007KHVSHHFKVLJQDORIRQHJLYHQZRUGDVDQREVHUYDWLRQVHTXHQFH+HUHLWLV DVVXPHG WKDW WKH FRGHERRNV ZLWK WKH 0 FRGHERRNV YHFWRU HQFRGLQJ IRU VSHHFK VSHFWUXP YHFWRUV HDFK REVHUYDWLRQLVDQLQGH[DQGWKHLQGH[FRUUHVSRQGLQJWRWKHFRGHERRNYHFWRUVLVFORVHVWWRWKHVSHFWUXPRIWKH VSHHFK VLJQDO YHFWRU LQ D ZD\ 7KH YRLFH UHFRJQLWLRQ V\VWHP LV WR GUDZ UHFRJQLWLRQ UHVXOWV GHSHQG RQ WKLV LQGH[ 7KHH[SHULPHQWDOUHVXOWV ,Q RUGHU WR LPSURYH WKH FUHGLELOLW\ RI WKH UHVXOWV RI H[SHULPHQWV ZH WHVWHG WKH V\VWHP LQ WRZ GLIIHUHQW HQYLURQPHQW VOLJKWO\ QRLV\ DQG QRLV\ WKH SHUVRQQHO LQYROYHG LQ WKH WHVWLQJ ZHUH VWULFW VFUHHQLQJ ,Q ERWK HQYLURQPHQWVWKHUHZHUHDGXOWVPDOHVIHPDOHV DQGFKLOGUHQPDOHIHPDOH WRRNWKHWHVW HYHU\RQHIRUZRUGV7KHWHVWUHVXOWVVKRZQLQ7DEOH 7DEOH7KHH[SHULPHQWDOUHVXOWV





'HWHFWDEOHUDWH

)DOVHGHWHFWDEOH UDWH

6OLJKWO\QRLV\





1RLV\





$GXOWV





587

588

Jia-ni Lu et al. / AASRI Procedia 3 (2012) 583 – 588

&KLOGUHQ





)URP WKH H[SHULPHQWDO UHVXOWV ZH FDQ NQRZ WKDW WKH HQYLURQPHQW KDV D YHU\ LPSRUWDQW LPSDFW RQ WKH UHFRJQLWLRQUHVXOWV,QDGGLWLRQWKHDFFXUDF\RIWKHSURQXQFLDWLRQDOVRDIIHFWWKHUHFRJQLWLRQUDWH7KHVOLJKWO\ QRLV\HQYLURQPHQWKDVDKLJKHUGHWHFWDEOHUDWH7KHSURQXQFLDWLRQRIDGXOWVLVPRUHDFFXUDWHDQGPRUHVWDEOH WKDQFKLOGUHQVRWKHLUUHFRJQLWLRQUDWHLVDOVRKLJKHUWKDQWKHP 6XPPDU\ 7KLV SDSHU GHVFULEHV D YRLFHEDVHG LQWHUDFWLYH V\VWHP DQG WKH PDLQ SDUW RI WKH VSHHFK UHFRJQLWLRQ LQFOXGLQJ SUHWUHDWPHQW IHDWXUH H[WUDFWLRQ DQG UHFRJQLWLRQ DOJRULWKPV DUH H[SODLQHG LQ GHWDLO DQG WHVWHG E\ VRPHH[SHULPHQWV6SHHFKUHFRJQLWLRQV\VWHPVKDYHEHFRPHLQFUHDVLQJO\FRPPRQWRRXUOLIHLWLVERXQGWR EULQJXVDQHZLQWHUDFWLYHH[SHULHQFH $FNQRZOHGJHPHQWV 7KLVVXEMHFWLVIXQGHGE\VWDWHIRUHVWU\DGPLQLVWUDWLRQLWHP 7KLVIXOOQDPHRIWKLVIRXQGDWLRQLWHPLVVWDWHIRUHVWU\DGPLQLVWUDWLRQLQWURGXFHGDGYDQFHGIRUHVWU\VFLHQFH DQGWHFKQRORJ\SURMHFW&ULWLFDOWHFKQRORJ\ LQWURGXFWLRQRI)ORWDWLRQ&ROXPQUHF\FOLQJILEHUIURPGHLQNLQJ SXOS1XPEHU 5HIHUHQFHV >@=KDR/L7KHEDVLVRIURERWLFV0%HLMLQJ&KLQD0DFKLQH3UHVV >@%DL6XQ[LDQ5HVHDUFKRI&KLQHVHLVRODWHGZRUGYRLFHUHFRJQLWLRQ'&KHQJGX6RXWKZHVW-LDRWRQJ 8QLYHUVLW\ >@6KHQ+RQJ\X5HVHDUFKDQG,PSOHPHQWDWLRQRIWKH'63EDVHGYRLFHV\VWHP'-LDQJVX-LDQJQDQ XQLYHUVLW\ >@/X4LQJTL%DL