MISSING VALUES IN NONLINEAR FAcTOR ... - Semantic Scholar

6 downloads 180 Views 585KB Size Report
it reconstructs missing values in observations. The. NFA model uses ... speech data and Boston housing data, we conclude that the ... Generative models handle missing values in an easy ..... and nd principal components and in the middle the.
                 !"#! $"!%& !'( )* +,-")).(/ 01&2 0'3)&# 4 2&,- 5"'& 67879): ;?0@=A=B; $+/ C D))/ >!"2"E C@F2!G HIJIKL MN ILOPQRST MU L V WIXXL MY IZJPZIQRST MU L $4[G RTTJ \]]^^^ M_ L` MRST MU L] abcdeafd ghi jkljikmnio lp mhi qlqrnqisk pstmlk sqsruono vwxyz { l|ir ski om}|ni| ~u { iso}knq hl€ €irr nm kitlqomk}tmo { noonq sr}io nq l~oiksmnlqo‚ ghi wxy { l|ir }oio s { }rmnƒrsuik jiktijmklq v„…†z qim€lk‡ plk sjjklˆn{ smnq mhi qlqrnqisk { sjjnq pkl { pstmlko ml l~oiksmnlqo‚ ghi wxy { l|ir no tl { jski| €nmh rnqisk pstmlk sqsruono vxyz sq| €nmh mhi oirpƒlksqnonq { sj v‰Š„z ‚ ghi q} { ~ik lp jsƒ ks { imiko nq mhi wxy { l|ir no trloik ml xy mhsq mhi ‰Š„‹ ~}m }qrn‡i xy‹ wxy no s~ri ml { l|ir qlqrnqisk { sqnplr|o‚ Œsoi| lq iˆjikn{ iqmo €nmh kisr €lkr| ojiith |sms sq| Œlomlq hl}onq |sms ‹ €i tlqtr}|i mhsm mhi jikplk { sqti lp mhi wxy { l|ir no trloik ml xy ‚ Ž de‘’“fd‘ ”iqiksmni { l|iro hsq|ri { noonq sr}io nq sq isou sq| qsm}ksr €su‚ •hiqiik s { l|ir no pl}q| ‹ kitlqƒ omk}tmnlqo lp mhi { noonq sr}io ski srol l~msnqi|‚ ”iqiksmni { l|iro ski qlm mhi lqru €su ml hsq|ri mhi { noonq |sms –— ‹ ˜™‹ ˜˜š‹ ~}m €i lqru tlik mhi { nq mhno jsjik‚ ›qo}jiknoi| riskqnq tsq ~i }oi| plk o}jiknoi| riskqnq ~u tlqon|iknq mhi l}mj}mo lp mhi miom |sms so { noonq sr}io vx n}ki ˜z ‚ ghno tl { ~nqio pism}ki iˆmkstmnlq sq| o}jiknoi| riskqƒ nq‚ ghi s~nrnmu ml kitlqomk}tm { noonq sr}io { isƒ o}kio mhi œ}srnmu lp s { l|ir sq| nmo s~nrnmu ml iqƒ iksrnoi‚ itlqomk}tmnlqo ski }oi| nq mhno jsjik ml |i { lqomksmi mhi jkljikmnio lp qlqrnqisk pstmlk sqsrƒ uono vwxyz –ž š ~u tl { jsknq nm ml rnqisk pstmlk sqsrƒ uono vxyz sq| ml mhi oirpƒlksqnonq { sj v‰Š„z –Ÿš ‚ xy no rn‡i jknqtnjsr tl { jlqiqm sqsruono v† yz €nmh { l|irri| qlnoi‚ ¡m no s ~sont mllr mhsm €lk‡o €irr €hiq qlqrnqisk i¢itmo ski qlm n{ jlkmsqm‚ … ski |n{ iqonlqsrnmu lp |sms no qlm s jkl~ri { ‚ ghi ‰Š„ tsjm}kio qlqrnqisknmnio sq| tr}omiko ‹ ~}m hso |n£t}rmnio €nmh |sms lp hnh nqmknqont |n{ iqonlqsrnmu sq| iqiksrnosmnlq‚ wxy hso jkljikmnio lp ~lmh lp

Supervised learning

Training data

Test data

Desired outputs

Unsupervised learning with missing values

Data

Missing values

x n}ki ˜¤ ›qo}jiknoi| riskqnq tsq ~i }oi| plk o}jiknoi| riskqnq ~u tlqon|iknq mhi l}mj}mo lp mhi miom |sms so { noonq sr}io‚ mhi {¥ ¡m tsq hsq|ri ~lmh rski |n{ iqonlqsrnmu sq| qlqrnqisk i¢itmo‚ ghi mhkii { imhl|o ski |iotkn~i| ohlkmru nq mhi qiˆm m€l oitmnlqo‚ ghi pl}kmh oitmnlq |iotkn~io mhi iˆjikn{ iqmo sq| mhi kio}rmo‚ ghiu ski |not}ooi| nq oitmnlq ¦i‚ §Ž c“¨©eªc©’ c©«¬­‘e®ac® ¯a¨ y oirpƒlksqnonq { sj ‹ €hnth no }oi| ml kitlqomk}tm { noonq sr}io ‹ no tsrri| mhi ‰ }jiknoi| ‰Š„ –Ÿš ‚ „ l|ir itmlko rni nq mhi os { i ojsti so mhi |sms‚ °iki €i }oi s sknsqm nq €hnth isth |sms itmlk no { smthi| ml { l|ir itmlko nqlknq mhi sr}io mhsm ski { noonq‚ ¡q mhi riskqnq jhsoi ‹ mhi €nqƒ

qnq { l|ir itmlk sq| nmo qinh~l}ko nq mhi { sj ski { li| ornhmru ml€sk|o mhi |sms itmlk nqlknq mhi |n{ iqonlqo lp mhi { noonq sr}io ssnq‚ x nqsrru mhi kitlqomk}tmnlqo lp mhi { noonq sr}io ski tl { ~nƒ qsmnlqo lp mhi sr}io pkl { mhi { l|ir itmlko ‹ €hnth ski €inhmi| sttlk|nq ml ”s}oonsq ‡ikqiro soonqi| ml mhi {¤ v˜z                 

 

     

   

vz

  



vz #

 !   "      

no s |sms itmlk sq| nmo kitlqomk}tmnlq‚ no s { l|ir itmlk‚ † sks { imik no tsrri| mhi €n|mh lp mhi olpmiqnq ‡ikqiro‚ •hiq sjjklsthio ikl ‹ mhi kitlqomk}tmnlq sjjklsthio mhi onqri €nqƒ qnq { l|ir itmlk‚ ghi |sms oim { nhm tlqmsnq { nooƒ nq sr}io nq isth lp mhi itmlko ‹ ~}m mhi { l|ir itƒ mlko tlqmsnq ql { noonq sr}io‚ ghi kitlqomk}tmnlqo ski kiomkntmi| ml tlqiˆ tl { ~nqsmnlqo lp mhi { l|ir itmlko‚ ghno { nhm ~i onqn¦tsqm ‹ €hiq mhi mk}i |n{ iqonlqsrnmu lp mhi |sms no rski sq| { lom lp mhi |sms no lq mhi ~lk|ik ¤ •hiq s mhkii |n{ iqonlqsr ~srr rloio Ÿ jiktiqm lp nmo ksƒ |n}o ‹ nm rloio s~l}m ˜ jiktiqm lp nmo { soo ‹ ~}m s Ÿ ™ |n{ iqonlqsr ~srr rloio { lki mhsq jiktiqm lp nmo { soo‚ Ž «©ae a’ ‘«©ae ¬afd‘e aa« cc ŽŽ ¯ y ttlk|nq ml mhi iqiksr xy { l|ir mhi |sms hso ~iiq iqiksmi| ~u pstmlko mhkl}h { sjjnq ¤ vz €hiki no s |sms itmlk ‹ no s pstmlk itmlk ‹ no s jsks { imik itmlk sq| no s qlnoi itmlk‚ ghi rnqisk { sjjnq }oi| nq xy no vŸ z ghi { l|ir no on{ nrsk ml jknqtnjsr tl { jlqiqm sqsruƒ ono iˆtijm mhsm xy nqtr}|io mhi qlnoi mik { sq| mhi pstmlko hsi s ”s}oonsq |nomkn~}mnlq‚ ¡q wxy‹ mhi p}qtmnlq no srrl€i| ml ~i qlqrnqisk‚ •i }oi mhi { imhl| jkljloi| nq –žš‹ €hiki mhi „…† qim€lk‡ vž z

 

 







$

%

&

'

(

) * + , -./ 0 1. 0 / +

'

2

3

%

   3 2  4  5 6  2



4

6

no }oi| ml { l|ir mhi qlqrnqisknmu‚ ghi jsks { imik itmlk tlqmsnqo ~lmh sq| ‚ ghi pstmlko sq| mhi qlnoi ski soo} { i| ml ~i nq|ijiq|iqm sq| ”s}oonsq‚ ¡q wxy mhi |sms no { l|irri| ~u s hnh |n{ iqƒ onlqsr { sqnplr| tkismi| ~u p}qtmnlq pkl { s jknlk ”s}oonsq |nomkn~}mnlq‚ ¡m tsq ~i tl { jski| ml mhi oirpƒlksqnonq { sj v‰Š„z –Ÿš‹ ~}m mhi q} { ~ik lp jsks { imiko otsri { lki rn‡i nq xy ‚ ghi ‰Š„ otsrio iˆjlqiqmnsrru so p}qtmnlq lp mhi |n{ iqonlqsrnmu lp mhi }q|ikrunq |sms { sqnplr|‚ y o{ srr q} { ~ik lp jsks { imiko ‡iijo mhi { l|irri| { sqnplr| o{ llmh‚ •i ¦q| mhi jsks { imik itmlk }onq iqoi { ~ri riskqnq‚ Ž§Ž © ¡q iqiksr mhiki ski nq¦qnmiru { squ jloon~ri iˆƒ jrsqsmnlqo lp |n¢ikiqm tl { jriˆnmu plk mhi l~oiki| |sms‚   hllonq mll tl { jriˆ s { l|ir kio}rmo nq likƒ ¦mmnq ‹ €hiki mhi { l|ir mknio ml { s‡i }j { isqnqƒ rioo iˆjrsqsmnlqo plk mhi qlnoi nq s||nmnlq ml mhi mk}i pstmlko‚   hllonq mll on{ jri s { l|ir kio}rmo nq }qƒ |ik¦mmnq ‹ risnq hn||iq ol { i lp mhi mk}i pstmlko mhsm hsi iqiksmi| mhi |sms‚ ghi olr}mnlq ml mhi jkl~ri { no mhsm ql onqri { l|ir ohl}r| stm}srru ~i thloiq‚ ¡qomis| ‹ srr mhi jloon~ri iˆjrsqsmnlqo ohl}r| ~i ms‡iq nqml sttl}qm sq| €inhmi| sttlk|nq ml mhink jlomiknlk jkl~s~nrƒ nmnio‚ ghno sjjklsth ‹ ‡ql€q so Œsuionsq riskqnq ‹ ljmn{ srru olrio mhi mks|il¢ ~im€iiq }q|ikƒ sq| lik¦mmnq‚ ¡q jkstmnti ‹ iˆstm mkism { iqm lp mhi jlomiknlk j|po lp mhi { l|iro no n{ jloon~ri‚ ghikiplki ‹ ol { i o}nmƒ s~ri sjjklˆn{ smnlq { imhl| { }om ~i }oi|‚ qoi { ƒ ~ri riskqnq – ‹ ˜‹ ‹ š‹ €hnth no lqi muji lp sknsƒ mnlqsr riskqnq ‹ no s { imhl| plk jsks { imknt sjjklˆƒ n{ smnlq lp jlomiknlk j|po‚ ghi ~sont n|is nq iqoi { ƒ ~ri riskqnq no ml { nqn{ noi mhi { no¦m ~im€iiq mhi jlomiknlk j|p sq| nmo jsks { imknt sjjklˆn{ smnlq‚ … im |iqlmi mhi iˆstm jlomiknlk j|p sq| nmo jsks { imknt sjjklˆn{ smnlq‚ ghi { no¦m no { iso}ki| €nmh mhi }rr~st‡ƒ … in~rik v …z |nikƒ iqti ~im€iiq sq| ‹ |i¦qi| ~u mhi tlom p}qtmnlq 4

7

3

4

'

? -+ @ A ,+ ,+ B/ ? C? D

E

%

J H 

F &

G H I 

K G

LMN

3

3 2 4   7 2 5 8 9

3

3 2 4   7

:;< =

7 > 2 5 8 >  5 8



8

LMN

J



O P QRST



V J 2 H  RST



G U

K

J

vz F

J 2 H  H 2 G 2 H I    J 2 H  H 2 5 RST G   9 V J 2 H  RST G 2 H    

Œits}oi mhi … |nikiqti nqlrio sq iˆjitmsƒ mnlq lik s |nomkn~}mnlq ‹ nm no oiqonmni ml jkl~s~nrnmu K

Hidden neurons Reconstructed data Factors Hidden neurons Reconstruction error

missing value

x n}ki ¤ ›jjik ¦}ki ¤ „…† qim so s iqiksmni { l|ir ‹ … l€ik ¦}ki ¤ kklk jkljssmio lqru pkl { l~oiki| sr}io ml pstmlko‚ { soo ksmhik mhsq ml jkl~s~nrnmu |iqonmu‚ ghi mik{ |lio qlm |ijiq| lq mhi jsks { imiko lk mhi pstmlko sq| tsq ~i qiritmi|‚ ghi riskqnq kioi { ~rio mhi iˆjitmsmnlq { sˆn{ nƒ osmnlq v „z srlknmh { ‚ ghi pstmlko ski s| }omi| €hnri ‡iijnq mhi { sjjnq tlqomsqm sq| mhi { sjƒ jnq no s| }omi| €hnri ‡iijnq mhi pstmlko tlqomsqm sr€suo { nqn{ nonq mhi tlom p}qtmnlq‚ y rr mhi jsƒ ks { imiko sq| pstmlko ski { l|irri| €nmh ”s}oonsq |nomkn~}mnlqo ksmhik mhsq jlnqm iomn{ smio‚ y { lki |imsnri| sttl}qm lp mhi }qo}jiknoi| iqoi { ~ri riskqnq { imhl| }oi| plk qlqrnqisk pstƒ mlk sqsruono sq| |not}oonlq lp jlmiqmnsrru sjjisknq jkl~ri { o tsq ~i pl}q| nq –žš ‚ 

E

RST G  

E

ŽŽe ghi sjjklˆn{ smnlq no soo} { i| ml ~i s ”s}oƒ onsq |iqonmu €nmh s |nslqsr tlsknsqti { smknˆ‚ ghno on{ jrn¦io mhi tlom p}qtmnlq niq ~u v z nqml iˆjitmsmnlqo lp { squ on{ jri mik{ o‚ ‰ l { i lp mhi { kirsmi ml mhi qlnoi itmlko €hnth ski srol mhi kiƒ tlqomk}tmnlq ikklko so oiiq nq plk { }rs v z ‚ ¡q tsoi lp s { noonq sr}i mhno mik{ no qlm nqtr}|i| nq mhi tlom p}qtmnlq sq| mhi pstmlko ski mh}o iomn{ smi| ~soi| lqru lq mhi ssnrs~ri l~oiksmnlqo‚ ghi kitlqomk}tƒ mnlq lp |sms no l~msnqi| ~u mhi { sjjnq pkl { mhi iomn{ smi| pstmlko vx n}ki z ‚ ghiki no sq sqsrlu €nmh mhi ‰Š„ ‚ xstmlk itƒ mlk tlkkiojlq|o ml mhi €nqqnq { sj }qnm nq mhi ‰Š„ sq| tlkkiojlq|o ml mhi { l|ir itmlk lp mhi €nqqik‚ +1 ) ? -./ 0 1.C) ?

' '

J 2 4 

F

6 

%

3



2

3 2 

ŽŽ« x nkom ‹ rnqisk † y vjknqtnjsr tl { jlqiqm sqsruono z no sjjrni| ml ¦q| oiqon~ri nqnmnsr sr}io plk mhi jlomiƒ knlk { isqo lp mhi pstmlko‚ † y hso ~iiq { l|n¦i| ml sttijm { noonq sr}io ~u tsrt}rsmnq mhi tlsknƒ sqtio pkl { lqru mhloi jsnko lp |sms sr}io €hiki ~lmh sr}io ski l~oiki| vso ljjloi| ml { noonq z ‚ ghi jlomiknlk sknsqtio lp mhi pstmlko ski nqnmnsrnoi| ml o{ srr sr}io‚ ghi pstmlko €iki ¦ˆi| sm mhi sr}io niq ~u rnqƒ isk † y plk mhi ¦kom Ÿ ™ o€iijo mhkl}h mhi iqmnki |sms oim‚ ghno srrl€o mhi qim€lk‡ ml ¦q| s { isqƒ nqp}r { sjjnq pkl { pstmlko ml mhi l~oiksmnlqo ‹ mhiki~u }omnpunq }onq mhi pstmlko plk mhi kijkioiqƒ msmnlq‚ xlk mhi os { i kisolq ‹ mhi jsks { imiko tlqmklrƒ rnq mhi |nomkn~}mnlqo lp mhi pstmlko ‹ €inhmo ‹ qlnoi sq| mhi hujikjsks { imiko ski qlm s|sjmi| |}knq mhi ¦kom ˜™™ o€iijo‚ ghiu ski s|sjmi| lqru spƒ mik mhi qim€lk‡ hso pl}q| oiqon~ri sr}io plk mhi skns~rio €hloi |nomkn~}mnlqo mhioi jsks { imiko tlqƒ mklr‚ ghno oimmnq no n{ jlkmsqm plk mhi { imhl| ~iƒ ts}oi mhi qim€lk‡ tsq i¢itmniru jk}qi s€su }qƒ }oi| jskmo ‹ €hnth €l}r| ris| ml s rltsr { nqn{ } { pkl { €hnth mhi qim€lk‡ €l}r| qiik kitlik‚ Ž © ¨©e¯©dc ŽŽ © ghi iˆjikn{ iqm oimmnq no ml kitlqomk}tm mhi { noonq sr}io sq| mhi { isq oœ}ski ikklk lp mhi kitlqomk}tƒ mnlqo ski }oi| plk mhi tl { jsknolq‚ ghi m€l |sms oimo mhsm ski }oi| ski ojiith |sms sq| Œlomlq hl}onq |sms‚ ¡qlks~nrnmu lp mhi |sms tlrritmnlq { ithsqno{ – š no soo} { i| nq mhno jsjik‚ ghi tlrritmnlq { ithsƒ qno{ no qlqnqlks~ri ‹ plk nqomsqti ‹ €hiq l}mƒlpƒotsri { iso}ki { iqmo ski { sk‡i| so { noonq‚ ghi ¦kom |sms oim tlqonomo lp kisrƒ€lkr| x nqqnoh ojiith ojitmklks { o ojl‡iq ~u oiiksr nq|nn|}sro‚ ‰ hlkm mik { ojitmks ski €nq|l€i| ml ™ |n{ iqonlqo €nmh s omsq|sk| jkijkltioonq jklti|}ki plk ojiith kitlqnmnlq‚ ¡m no trisk mhsm s |uqs { nt { l|ir –˜ š €l}r| ni ~immik kitlqomk}tmnlqo ‹ ~}m nq mhno tsoi mhi mi { jlksr nqplk { smnlq no ripm l}m ml isoi mhi tl { ƒ jsknolq lp mhi { l|iro‚ °srp lp mhi s~l}m Ÿ ™™™ os { ƒ jrio ski }oi| so miom |sms €nmh ol { i { noonq srƒ }io‚ „ noonq sr}io ski oim nq pl}k |n¢ikiqm €suo ml { iso}ki |n¢ikiqm jkljikmnio lp mhi srlknmh { o vx nƒ }ki z¤ ˜ ‚ — jiktiqm lp mhi sr}io ski oim ml { noo ksqƒ |l { ru nq mn{ io jsmthio‚ vx n}ki z ghno no mhi { snq oimmnq ‹ onqti nm no { lom kisrnomnt‚ ‚ ˜™ jiktiqm lp mhi sr}io ski oim ml { noo ksqƒ |l { ru nq|ijiq|iqm lp squ qinh~l}ko‚ ghno no '

Factors

+ B/ ? C? D





/) 1+*0/+









+/C@ + ?. -+..C? D -

#

#



#

#

%



%

%

Generalisation

1

2

Nonlinearity

3

4

~smth nmiksmnlqo ‹ €hiki lqi ~smth nmiksmnlq { isqo lnq mhkl}h srr mhi l~oiksmnlqo lqti‚     ŽŽ¨  ghi jikplk{ sqti lp |n¢ikiqm { imhl|o €nmh ojiith |sms tsq ~i oiiq nq x n}ki Ÿ‚ wxy €so miomi| €nmh ™ hn||iq qi}klqo sq| ml ˜Ÿ pstmlko sq| xy €nmh q} { ~ik lp pstmlko skunq pkl { ˜ ml ™ ‚ ghi n{ ƒ jri { iqmi| wxy srlknmh { o}¢iki| pkl { nqoms~nrnmu €hiq mhi q} { ~ik lp pstmlko €so kismik mhsq ˜Ÿ‚ ghi wxy { l|ir jikplk { i| sr€suo ~immik mhsq xy €nmh os { i q} { ~ik lp pstmlko‚ ghi { isq oœ}ski kitlqomk}tmnlq ikklko ski tlrƒ ritmi| hiki nq mhi os { i lk|ik so nq x n}ki ¤ wxy xy ‰Š„ ˜ ‚ ž ™ ‚Ÿ ˜ ‚—— ™ ‚Ÿ ˜ ‚ ™ ‚— ˜ ‚ ™ ‚Ÿ ˜ ‚—Ÿ ™ ‚Ÿ— ˜ ‚Ÿ ™ ‚—Ÿ ghi ¦kom oimmnq jkli| ml ~i mhi hsk|iom so iˆjitmi| ‹ onqti qi€ €lk|o kiœ}nki iqiksrnosmnlq sq| { noonq sr}io nq jsmthio { s‡io qlqrnqisk ipƒ pitmo { lki n{ jlkmsqm‚ •nmh ljmn{ sr jsks { imik srƒ }io mhi ‰Š„ si { sknqsrru ~immik kitlqomk}tmnlqo mhsq wxy ‚ xy jikplk{ i| mhi €lkom‚ ghi oitlq| oimmnq €so isonik ¤ wxy sq| xy jikƒ plk { i| iœ}srru €irr ‹ ~}m mhi ‰Š„ tl}r| qlm sthnii os { i stt}kstu‚ y rski q} { ~ik lp { l|ir itmlko |n| qlm hirj mhi ‰Š„ ml im iql}h kijkioiqmnq jl€ik‚ ghi mhnk| oimmnq hs| jik { }mi| |sms oimo ‹ €hnth { s‡io iqiksrnosmnlq rioo n{ jlkmsqm‚ ghno hirji| mhi ‰Š„ s rlm sq| nm si triskru mhi ~iom kio}rmo‚ wxy sq| xy ~iqi¦mmi| lqru { sknqsrru sq| €iki ripm ~ihnq|‚ io}rmo lp mhi pl}kmh oimmnq |n| qlm |n¢ik pkl { mhi oitlq| oimmnq‚ ghi thsqi nq mhi { noonq sr}i jsmmikq pkl { mhi ¦kom sq| mhnk| oimmnqo oii { o ml hsi mhi |l { nqsqm i¢itm‚ ghi kitlqomk}tmnlq ikklko lp mhi l~oiki| sr}io vrl€ik t}kio nq x n}ki Ÿ z ski lp nqmikiom ‹ mll‚ ghi ‰Š„ tsq qlm kijkioiqm mhi |sms so stt}ksmiru €nmh mhi { sj }qnm stmnnmnio so mhi pstmlk sqsruono { l|iro tsq €nmh mhi pstmlk sr}io‚ ghi xy { l|ir €nmh ™ pstmlko tl}r| hsi kijkioiqmi| mhi |sms jikpitmru‹ ~}m mhi { l|irri| qlnoi sttl}qmi| plk ol { i sknsƒ mnlq‚ ghi ~iom q} { ~ik lp { l|ir itmlko nq mhi ‰Š„ €so ˜ž ™™ nq mhi ¦kom tsoi ‹ ~}m sm risom ™™ nq mhi lmhik tsoio‚ ghno hso ts}oi| s thsqi nq mhi kiƒ tlqomk}tmnlq ikklk lp mhi l~oiki| sr}io‚ ghi ljmnƒ { sr €n|mh lp mhi olpmiqnq ‡ikqir €so srol ol { i€hsm rskik nq mhi ¦kom iˆjikn{ iqm‚ ghi q} { ~ik lp { sj }qnmo no qlk { srru qlm so rski so hsrp lp mhi q} { ~ik lp |sms itmlko‚ 

High dimensionality

'

+/ )/ @ B ? 1 +

#

C.

. + - ++1

* B. B



#

Memorisation

x n}ki ¤ xl}k |n¢ikiqm iˆjikn{ iqm oimmnqo €nmh mhi ojiith |sms mku ml { iso}ki |n¢ikiqm jkljikmnio lp mhi srlknmh { o‚ sq isonik oimmnq ‹ onqti on{ jri o{ llmhnq }onq qisk~u sr}io €l}r| ni ¦qi kitlqomk}tmnlqo‚ ‚ gksnqnq sq| miomnq oimo ski ksq|l { ru jikƒ { }mi| ~iplki oimmnq { noonq sr}io nq mn{ io jsmthio so nq oimmnq ˜ ‚ ghi mksnqnq oim tlqƒ msnqo itmlko { lki on{ nrsk ml mhi miom oim ql€‚ ‚ gksnqnq sq| miomnq oimo ski jik{ }mi| sq| ˜™ jiktiqm lp mhi sr}io ski oim ml { noo nq|ijiqƒ |iqmru lp squ qinh~l}ko‚ ghi oitlq| |sms oim no Œlomlq hl}onq |sms ‹ €hnth no j}~rntru ssnrs~ri sm – š ‚ ¡m tlqtikqo hl}oƒ nq sr}io nq o}~}k~o lp Œlomlq‚ sms oim tlqmsnqo Ÿ ™ ž itmlko lp ˜ |n{ iqonlqo iˆtr}|nq lqi ~nqsku smmkn~}mi‚ xl}k lp mhi ˜ sr}io €iki tl {{ lq ml isth ml€q ‹ €hnth tlqonom lp ˜ ml Ÿ ™ o}~}k~o‚ ™  lp mhi |sms itmlko ski }oi| so mksnqnq |sms sq| mhi kiom so miomnq |sms ‹ €hnth hso ˜™  lp nmo sr}io { noonq ksq|l { ru‚ Ž§Ž  ‰ irpƒlksqnonq { sj €so riskqi| }onq mhi ‰Š„ gllr~lˆ  plk „ smrs~‚ ghi q} { ~ik lp { l|ir itmlko sq| mhi €n|mh lp mhi olpmiqnq ‡ikqiro sooltnsmi| €nmh mhi { €iki ripm so jsks { imiko sq| mhi oiritmnlq lp sr}io no qlm om}|ni| hikinq‚ ¡qomis| ‹ mhi sr}io €nmh mhi ~iom kio}rmo €iki }oi|‚ w } { ~ik lp nmiksƒ mnlqo €so mhi |ips}rm sr}i nq mhi ‰Š„ mllr~lˆ sq| s||nmnlqsr Ÿ ™ nmiksmnlqo mhkl}h srr mhi |sms €iki k}q €nmh qinh~l}khll| oim l¢‚ ghi wxy tl|i plk „ smrs~ €so { l|n¦i| ml o}jƒ jlkm { noonq sr}io‚ ghi tl|i qii|i| ml kijkl|}ti mhi iˆjikn{ iqmo no j}~rntru ssnrs~ri  ‚ ghi wxy { imhl| €so mkni| €nmh ¦ˆi| q} { ~ik lp hn||iq qi}ƒ klqo }onq oiiksr |n¢ikiqm q} { ~ik lp pstmlko lqru‹ ~its}oi mhi srlknmh { kiœ}nkio s~l}m miq mhl}osq| #

#

%

%

%



#

#

F



@  ,+@ +?. B.C) ?

 

         

          

#

F F#

F F

F

F# 

#

#

%

NFA

Data with missing values

FA

3

SOM

3

2

3

2

2 1.88

1.76 1

1 0.19

0

Original data

5

10

0.14 0

10

20

2

1.5

1.5

1.5

0.57 0.19

0

5

10

15

3

1

2

0

0.07 10

20

10

+/ ) /@ B ? 1 +

C.

. +

) -. ) ? * B.B

F





& &

F

10

20

30

0

1.5

1.5

1.5

0.19

0

5

10

15

4

2

4

2

4

1.52

0.14 0

2

0.5

0

1

2

0.57

2

0.59

2

2

1

 

15

0

1.85 1

5

4

3

0.19

%

30

2

0

2

0.83

1 0.5

1.73 1

0

0.57 0.5

3

NFA reconstruction

x n}ki ¤ ‰ jiith |sms kitlqomk}tmnlq iˆs { jri €nmh ~iom jsks { imiko lp isth srlknmh { ‚    b ŽŽ¨  ghi iˆjikn{ iqmo €nmh Œlomlq |sms €iki k}q pl}k mn{ io €nmh ksq|l { |nnonlq lp |sms nqml mksnqnq sq| miom oimo‚ „ isq oœ}ski kitlqomk}tmnlq ikklko €iki otsri| o}th mhsm mhi ‰Š„ ikklko €iki ˜™™ nq isth tsoi‚ ghi wxy ikklko €iki ˜˜— €nmh omsq|sk| |insmnlq lp ˜ sq| mhi xy ikklko €iki ˜Ÿ ˜ €nmh omsq|sk| |insmnlq lp ‚ ghno |sms oim no tr}omiki| ‹ ~its}oi lp mhi ml€q omk}tm}ki ‹ sq| mhi |n{ iqonlqsrnmu lp mhi |sms { sqnƒ plr| no qlm mll rski plk mhi ‰Š„ ml hsq|ri‚ ghikiƒ plki nm no qlm iku o}kjknonq mhsm nm { s|i ~immik kitlqomk}tmnlqo mhsq wxy ‚ w lqrnqisknmnio €iki tk}ƒ tnsr nq mhi |sms oim ‹ onqti xy €so nqpiknlk ml mhi qlqrnqisk { imhl|o‚ wxy €so k}q €nmh ™ hn||iq qi}klqo sq| pkl { ˜ ml pstmlko‚ Œiom q} { ~ik lp mhi { skni| pkl { Ÿ ml ‚ ghi ~iom q} { ~ik { l|ir itmlko nq mhi ‰Š„ skni| pkl { Ÿ ™™ ml ˜— ™™‹ €hnth no psk kismik mhsq mhi q} { ~ik lp |sms jlnqmo‚ Ž ’cf“cc‘ x n}kio ž sq| ohl€ ol { i jkl itmnlqo lp mhi |sms sq| mhi { l|iro‚ ghi ‰Š„ { sqnplr| rll‡o œ}nmi t}kru‚ wxy { sqnplr| €nmh m€l pstmlko no srol klrri| }j ‹ ~}m nq s o{ llmhik €su‚ ¡{ jlkmsqm |n¢ikiqtio

0.65 0

2

1

FA reconstruction

30

2

0.5

SOM reconstruction

15

1.73 1

1

0.56 0

0.85

1 0.58

0.5 0.07 0

10

20

30

0.5 0.57 0

0

x n}ki Ÿ ¤ „ isq oœ}ski kitlqomk}tmnlq ikklko lp ojiith |sms so s p}qtmnlq lp q} { ~ik lp pstmlko lk olpmiqnq €n|mh‚ ghi kl€o tlkkiojlq| ml mhi iˆƒ jikn{ iqm oimmnqo kis|nq pkl { }j ml |l€q‚ ghi }jjik t}ki nq isth jrlm no mhi miom kio}rm plk { nooƒ nq sr}io sq| mhi rl€ik t}ki no mhi kitlqomk}tmnlq lp l~oiki| sr}io‚ ~im€iiq hnh |n{ iqonlqsr wxy { sqnplr|o ski qlm non~ri nq mhioi jkl itmnlqo‚ y o iˆjitmi| ‹ mhi ‰Š„ jikplk{ o mhi ~iom plk tr}omiki| |sms sq| wxy jikplk { i| ~immik mhsq xy ‚ io}rmo ohl€ mhsm wxy no trloik ml xy mhsq ml mhi ‰Š„ ‚ ghi kismiom jkl~ri { lp mhi ‰Š„ no mhsm mhi q} { ~ik lp jsks { imiko otsrio iˆjlqiqmnsrru €nmh mhi q} { ~ik lp nqmknqont |n{ iqonlqo lp mhi |sms { sqnplr| ‹ €hnth ris|o ml ~s| iqiksrnosmnlq‚ ghi wxy { l|ir |lio qlm €lk‡ €irr ‹ np mhi |sms no tr}omiki|‚ ¡m no hsk| ml ¦q| mhi p}qtmnlq mhsm ohsjio s ”s}oonsq tlqmnq}} { ml tr}omiko ‹ ~}m €hiq mhi |sms plk{ o s tlqmnq}} {‹ mll ‹ mhi wxy { l|ir no { lki sjjkljknƒ smi‚ ghi wxy srlknmh { oiskthio lqru plk rltsr ljƒ mn{ s ‹ ol { }rmnjri k}qo €nmh |n¢ikiqm nqnmnsrnosmnlqo €l}r| hsi ~immik thsqtio rl~srru‚ ghi oms~nrnmu ohl}r| ~i }sksqmii| sq| ol { i ojii|}jo tl}r| ~i { s|i plk mhi srlknmh { ml ~i s kis|uƒmlƒ}oi mllr‚ iq mhl}h mhi riskqnq srlknmh { kiœ}nkio s rlm jkltioonq ‹ mhi wxy { l|ir ‹ €hiq nm hso riskqi| ‹ E

Training

Test

SOM

2

8

15

x n}ki ž ¤ ghi ojiith |sms oimo ‹ mhi ‰Š„ { l|ir itmlko sq| ol { i jlnqmo iqiksmi| ~u wxy €nmh skunq q} { ~ik lp pstmlko ski jkl itmi| lq mhkii jrsqio‚ ¡q mhi }jjik{ lom kl€ mhi jrsqi no mhi ˜om sq| q| jknqtnjsr tl { jlqiqmo sq| nq mhi { n||ri mhi ˜om sq| k| tl { jlqiqmo sq| nq rl€ik{ lom kl€ mhi ˜om sq| mh tl { jlqiqm‚ wxy { l|iro €iki }oi| ~u iqiksmnq ksq|l { pstmlk itmlko pkl { mhink jknlk |nomkn~}mnlq‚ ghi |lmo ohl€q ski mhi tlkkiojlq|nq iˆjitmi| sr}io lp mhi |sms itmlko‚ 

# %

Training

Test

SOM

1

2

5

9

x n}ki ¤ Œlomlq hl}onq |sms ‹ jkl itmnlqo so nq x n}ki ž‚ tsq ~i sjjrni| nq kisr mn{ i‚ ghi tl { j}msmnlqsr tl { jriˆnmu lp mhi srlknmh { otsrio ml mhi jkl|}tm lp q} { ~ik lp pstmlko sq| hn||iq qi}klqo jik ~smth nmiksmnlq‚ Œu tlqmksom ‹ mhi q} { ~ik lp { sj }qnmo nq s ‰Š„ otsrio iˆjlqiqmnsrru so s p}qtmnlq lp mhi |n{ iqonlqsrnmu lp mhi { sj‚ wxy no ~iom o}nmi| plk psnkru omklqru qlqrnqisk jkl~ri { o €nmh sq nqmknqont |n{ iqonlq lp mhi lk|ik lp miq‚ •hiq tl { jski| ml mhi ‰Š„‹ pstmlk sqsruono { l|iro ski on{ jrik nq mik { o lp mhi q} { ~ik lp jsƒ ks { imiko‚ •hnri iiq mhi rskiom wxy { l|ir hs| s~l}m ™™™ jsks { imiko ‹ mhi ‰Š„ hs| s~l}m Ÿ ™™™™ jsks { imiko‚ ghno tsq ~i oiiq nq x n}ki ž pkl { mhi pstm mhsm wxy |n| qlm tsjm}ki srr mhi ¦qiom |iƒ msnro nq mhi |sms oim‚ ˆjikn{ iqmsr kio}rmo o}jjlkm mhi omsmi { iqm mhsm o{ srr q} { ~ik lp jsks { imiko iqƒ hsqtio mhi s~nrnmu ml iqiksrnoi‚ Ž f‘f«“c‘c ghi s~nrnmu ml kitlqomk}tm { noonq sr}io { iso}kio mhi œ}srnmu lp s { l|ir ¤ nmo s~nrnmu ml iqiksrnoi ‹ F

#

E

{ i { lknoi sq| kijkioiqm‚ ghi qlqrnqisk pstmlk sqsrƒ uono { l|ir m}kqi| l}m ml jikplk { €irr nq iqiksrnosƒ mnlq sq| kijkioiqmsmnlq €hnri nmo s~nrnmu ml { i { lknoi no rn{ nmi| |}i ml mhi o{ srr q} { ~ik lp jsks { imiko nq mhi { l|ir‚ wxy jikplk { i| ~immik mhsq xy nq srr mhi iˆjikn{ iqmo ‹ ~}m €nmh rski q} { ~ik lp pstmlko ‹ mhi t}kkiqm wxy srlknmh { ~itl { io tl { j}msmnlqƒ srru iˆjiqoni‚ •i tlqtr}|i mhsm qlqrnqisk pstmlk sqsruono vwxyz no ~iom o}nmi| plk psnkru omklqru qlqƒ rnqisk jkl~ri { o €nmh sq nqmknqont |n{ iqonlq lp mhi lk|ik lp miq‚    

                 !   " # $%&  '   ( )   *  )   +  + ) $ ) , -./01 , -23 4/56 07 8 9 0:;