Supervised Learning for Neural Networks: A Tutorial ... - LCN - EPFL


 

     
!#"$&% '(*) + , -
%* ."/'0 1%32 4 6 , 57, 9 8;:

< >= ?A@3BDCFEHGJI K!LMEHNPOHQRLME

US TWVYX[ZP\[]_^a`bdc]UX[^9efg;hjik`YZkl[m[ZP]_^9nkoqprmMst^Roqc#uwvxcH]_^yprprbzik^ycH]|{}~_]_^ys€~az‚ƒ^yl[bz]_^yl…„}†‡pz}c[^aˆ Zkc[l‰Š^aol‹ok`U^y~UVam!Œ\[`U^yprbrsŽbrc[ZP`U}[


  

              "!  #%$ & '(*) + , -."! $ /
0 + 1 + 2354 765 %8 9;:=A@CBD=? £  Ú uÚ  7 S j€GIS=U\S…¬HsJ€Q\GISg`mcIw{rSVUe]DhHs`IPIc^Q€isHK`IS=JeL=]Y`aZYS=U\WYHs`IW]D`mQ\]`^SVcIU_]Y`ŽŸˆOD`I@b :·HKJeOÈh]YU\wODi QTG^U\SVJ_GI]YisbŽPCODU\ODwxS=QTS=UV&u A/S=Q¢cIJ¢bIS=‰C`^S»QTG^SnQ\]DQNODiMHs`IPIc^Q¢ODJ‘@™ .  4 7 S    - 8;:<kuEG^S J\HswP^iKSVJŠQ3LNGI]YHsLVS‹]DhdO`I]Y`^iKHK`^StODU_HsQkf’jF]YcIisb rS‹OQ\GIU\S=J\GI]DiKb7hcI`IL=Q\HK]Y`„B˜ Ä=#4EDGF

kË `‚QTG^S{PIU\S=LVS=bIHK`IWJ\cIIJ_SVLpQTHK]D`‡z„QkjF]%QkfRPrSVJ¿]~hyPCODUTO~wSpQTSVU¢GCOtZYS{rSVS=`·HK`aQTU_]Rb^cILVS=b‡z}ZqHKŒ~usz jFSVHKWDGmQ\J   OD`Ib O·QTG^U\SVJ_GI]Yisb :ku %S‹j€HsiKi¼`^]Aj"J\GI]?j QTGIO~Q3QTGIS—QTGIU_SVJ_GI]YiKb LtOD`rS QTU_StO~Q\SVb†ODJ _cIJ_Qe]Y`^SnSÐoqQTUTOÈjFSVHKWDGmQVuyE;]3bI]xJ\]3jƒS¿j€HsiKi„JŠQNODUŠQ€hU\]Yw  Ú u Ú  - .  13 6 4    A@4E8

aDG:*4EA7 ;@='2

EGISdJ_HKwxPIiKSPrSVU_LVS=P^QTU_]Y`%LtO~`%J\]Yi[ZYS¢PIU\]Y^iKSVwxJj€GIHKLNG%ODU\SeiKHs`IStODU_isf…J\SVPIODUTOD^iKSe•;IcRQ¼GI]?j LtO~`ŽjƒS»‰C`IbŽQ\GIS…Pr]YJ\H[QTHs]Y`—]DhQTGISn]YP^Q\HKwODi/GafqP}S=U\PIiKOD`IS EGIS…HKbISVO]DhQ\GIS…PrSVU_LVS=P^QTU_]Y` ODisWY]YU\H[QTGIw&HKJ¿Q\GCO~Q¿jFSxQ\SVJ_Q»]Y`IS3bCO QNO—P}]YHs`aQ  dO~hœQTS=U¿QTGIS3]DQ\GISVU=uEXSVJŠQTHK`^WŽO—bCO~QTO Pr]YHK`aQMwxStOD`IJ/Q\GCO~Q/jFSFO~PIPIisf  ‡O~Q/QTG^SˆHs`IPIc^QMiKOtfYSVUM]DhqQTG^SˆPrSVU_LVS=P^QTU_]Y`…O~`Ib»L=]YwxPCODU\S QTG^SÊ`^SVcIU_]Y`CODid]Dc^QTPIcRQ   xj€HsQ\G QTGISbIS=J\HsU\b*]Yc^Q\PIc^QΚ  ÐuË hnQTGIS·]Dc^QTPIcRQ—]Dh QTG^SÎP}S=U\LVS=P^QTU_]Y`7HKJ{L=]YU\U_SVLpQtzyjƒSÎbI]D`‡ÅÂQÈQTODlDS—O~`mf ODL=Q\HK]Y`„uÊË h€Q\GISÎ]Yc^Q\PIc^Q3HKJ…HK`IL=]YU\U_SVL=Q   .  š  pz„Q\GIS…jFSVHsWYGaQdZYS=L=QT]DU HsJdLNGCOD`IWDSVb 8  ‚u¿EGIS{P^U\]qLVSVJ_J ]DhXjƒS=HKWYGaQ¢ODbCO~P^QNO~Q\HK]Y`—HsJLtODisiKSVbÎisStODU_`IHK`^WIuEGISgiKSVODU\`^HK`IWÈU_cIiKS¿HsJ  Ú u ÚYÚ   . š   8       j€GIS=U\ S D ExHsJJ\]YwxSgJ\wODiKi‡PIODUTODwxS=Q\SVU=uˆË QeHsJec^J\S=hc^iMQ\]Hs`aQTU\]qbIcIL=SnOÈZ~ODU_H|OD^iKS   . š   8   puEGISgiKSVODU\`^HK`IWÈU_cIiKSdQ\GISV`‹}S=LV]YwxSVJ  Ú u Ú IÁ   .     j€GIHsLNG†wODlDS=J¼QTGIS¿WDSV]YwxS=Q\U\HKLVODi„JŠQTU_cIL=Q\cIU\Sg]DhMQTG^S¿iKStO~U\`IHs`IW{PIU_]RL=SVJ\J€wx]YU\S¢QTUTO~`IJ\PCO~U\SV`aQVu Ë hyQTGIS3O~L=QTcIODi.]YcRQTPIc^Q   ¢HsJ‘LV]DU\U\S=L=QVz   eHsJdŒVS=U\]—OD`Ib‚Q\GIS{jFSVHKWDGmQ‘ZYSVLpQT]YU¿HsJd`I]DQ LNGCOD`^WYSVb‡@u A/S=Q…cIJ…L=]Y`IJ\HsbISVU»O†bIO~QNO‹P}]YHs`aQ  ‘j€H[QTGQTODU\WDS=QnZDO~iKcIS3š  . Ú h]YU j€GIHsLNGÌQTGIS‹PrSVU\L=SVP^Q\U\]Y`ÌWYHsZDSVJ3QTGIS—hÇODiKJ_S†]Dc^QTPIcRCQ   . E^uÌËk` QTG^HKJxLtODJ_S   {HKJ Pr]YJ\H[QTHsZDS%OD`^b O†LNGCO~`IWYS  (]~hQTGISxjƒS=HKWYGaQ…ZDSVLpQT]YU     Ž]Dh  nHsJ{ODPIP^iKHKS=b‡u 








 


































































 





























































o o

x

x

x o

x x o

∆w

w

o

o

x

Ú u ^˜ XSVU_LVSVPRQTU\]D` ODisWY]YU\H[QTGIw‹udË hFO%Pr]YHK`aQ¿HsJdwHsJ\L=i|ODJ_J\Hs‰ISVb‚isHKlDS»QTGISÈL=U\]YJ_$J ZYS=L=Q\]YU d]Y`·QTG^SxisS=hœQŠ•kGCOD`^b·J\HKb^SÈ]DhFQ\GIS3J\S=PCODUTO QTHK`^W‹isHK`ISÇbCODJ_GISVb-pz/QTGISÈjFSVHsWYGaQnZDSVLpQT]YU HKJ LNGCOD`^WYSVb%Hs`b^HKU\S=L=Q\HK]Y`3]Dh uEGIHsJFU_]DQNO~Q\SVJyQTGIS€J_SVPIU\O~QTHs`IW…isHK`ISHs`xQ\GIS€bISVJ_HKU\S=b%bIHsU\SVLpQTHs]Y`‡u EGIHsJ¿U\]DQTO~QTS=J‘QTGIS…GmfqPrSVU_PIi|OD`^S3ODU\]Yc^`Ib‚QTGISÈ]DU\HKWDHK`‡u ¢]DQ\S{QTGCO~Q¢QTGIS3O~iKWY]YU_HsQ\GIw½ODJdH[QdHKJ PIU_SVJ\S=`aQTSVb G^SVU\SxJ\c^PIP}]DJ\SVJnQTGCO Q»Q\GISÈQTGIU_SVJ\G^]YiKbHKJnODIJ_]YU\rSVbHK`ÊOD` O~bIbIHsQ\HK]Y`IODi;jFSVHsWYGaQ LV]DwPr]Y`IS=`mQVu %’Sej€HKisirJ_SVSdHs`Q\GIS¢h]YisiK]?j€HK`^WgQ\GCO~Qƒ`IStODU_isfODisiC]Dh/QTGIS¢iKSVODU\`IHs`IWnU\cIisSVJƒGIO?ZDS O3J_QTU_cIL=Q\cIU\SnOD`IODiK]YWD]YcIJ¼QT@]  Ú u Ú IÁ pu      7G4












8

E GISÎQTG^SV]YUŠf ]DheJ\HswPIisS%PrSVU\L=SVP^Q\U\]Y`^JxGIODb7}S=SV`bIS=ZDSViK]DP}S=b Hs` QTGIS%‰IhœQ\HKS=JÈbIcIU\Hs`IW‚QTG^S ‰CU_J_QIHsW…Z~ODWYcIS¢]DhM`IS=cIUTODi}`IS=QkjF]YU_lqJ=uFEG^SdJ\HswPIisS¢PrSVU_LVS=P^QTU_]Y`ÎGCODJ¼]D`ISd]YaZqHK]Yc^JƒbIHsJTODbR• Z~OD`aQNODWDSD˜H[QFLVOD`]D`IisfÈJ_]YisZDS¢isHK`ISVODU\i[f{J_SVPCODU\ODIisS¢PIU_]YIisSVwxJVu º aZqHK]YcIJ_isf{Q\GISVU_SdODU_S¢wOD`af PIU_]YIiKS=wJxj€GIHKLNG ODU\S†`I]DQ%isHK`ISVODU\i[f J\S=PCODU\ODIiKS~u#EGIS‚P^U\]DQ\]DQkfqP}S†]DhnO’`I]Y`q•kJ\S=PCODU\ODIiKS PIU_]YIiKS=w"HKJ¼Q\GIS  º • PIU\]Y^iKSVw‹u %*GCO~QgLtO~`ÊjFS3bI]Îj€HsQTG·Q\GI]YJ\S3PIU_]YIisSVwj€G^HKLNGODU\S3`I]DQgisHK`ISVODU\i[f‹J\SVPIODUTOD^iKS º `^S HKb^StO‚HKJ»QTGCO~Q…jƒSÎwOtf’rS—ODIisSQT]‚‰I`IbO‚PIU\S=PIU\]qLVS=J\J\Hs`IWJ\LNGIS=wSÎj€GIHsLNG7U\SVL=]Rb^SVJ{QTG^S PIU_]YIiKS=w J\]‹QTGCO~QgQ\GISU_SVwODHK`IHs`IWÎPIU_]YIiKS=w HKJgiKHs`IStO~U\isfŽJ\S=PCODUTO~IiKS~u%EGISHsbIStO—HKJgHsiKiKc^JŠ• QTU\O~QTS=b·HK` ".HKWIu Ú u Ú ÁRueEGIS{Hs`IPIc^Q¢ZYS=L=QT]DU  eHsJ¢P^U\Sp• PIU\]qLVS=J\J_SVbHK`·O3‰CU\JŠQ¿i|OtfDSVUdO~`Ib fqHKS=iKbIJyOdJ_S=Qy]Dh©hStO QTcIU_SVJj€HsQ\GÈLV]q(S 'xLVHsSV`aQTJ  S    ‘£t£t£ !    pu.EG^SVJ\SeODU_S¼QTGIS=` QTU_StO~Q\SVb†ODJ€HK`IP^c^QQT]xO3J\HKwxPIisSdP}S=U\L=SVP^Q\U\]YB`   F. G  4 4 7 S       Au '&












































o

d

o

x

o o

x

o

x

x o

o

x x

x

x x

Ú u ÚYÚ ˜ƒEGIS{]YP^Q\HKwODi‡PrSVU_LVSVPRQTU\]D`‡u‘EGIS»QTGIHsLNl‹isHK`ISgPIU_]?ZRHsbISVJ¢QTGIS»J\SVPIODUTO~Q\HK]Y`‹j€HsQ\G QTG^SnwO oRHKw…cIwwODU\WDHK`‡u;EGIS»bIODJ\GIS=b†isHK`IS¿WDHsZYS=JeO3J_cIR• ]YP^QTHswO~i„J\S=PCODU\O~QTHs]Y`‡u  `^h]YUŠQTcI`IO~QTS=isfYzDGI]?jƒSpZYSVU=zqQ\GISVU_S€SpoRHKJŠQTJ`I]¿cI`^HsZYS=U\J\ODiCPIU\S=PIU\]qLVS=J\J_HK`IWgwxS=QTG^]Rb{j€GIHsLNG HKJ  DHs`Q\GIS¢Hs`IPIc^QFJ\PIODLVS‘OD`IbjF]YcIisbQTU\OD`IJŠh]YU\w OD`afxbIH 'L=cIisQˆPIU_]YIisSVw HK`O»isHK`ISVODU\i[f J\S=PCODU\ODIiKS€]Y`^SDuËk`ÈhÇODLpQtzq؆Hs`IJ\laf3OD`^b ;ODP}S=U_QyWaOtZDS€Q\GIS€Spo^ODwxPIiKS]~hrQTODJ\l»j€GISVU_Se‰CWDcIU\S=J GCOtZYSQT]‹}SxLViKODJ\J_Hs‰CS=b’HK`aQ\]ŽLV]D`I`ISVLpQTS=b ]Y`IS=J ÇL=]Y`IJ_HKJ_Q\HK`IW‹]Dh€O—J\Hs`IWYisSxisHK`IS ¿OD`IbQ\GI]YJ_S j€GIHsLNGODU_S€`I]DQyLV]D`I`ISVLpQTS=b‡uFvRHK`IL=SQTGIHsJHKJyO¿WYiK]DCODi mcIS=J_QTHs]Y`3j€GIHKLNGxLtOD`x`I]DQyrS€bISVL=HKbIS=b iK]qLtO~iKisfDzY`I]»iK]qLtODiIP^U\SVP^U\]qLVSVJ_J\Hs`IW{LtO~`J\HswP^iKHshœfnQTG^S¢PIU_]YIisSVw•XQTGISeU\S=wO~HK`IHs`IWdQNODJ_lÈQTGCO~Q GCODJÈQ\]rS—J\]Yi[ZYSVbHK` Q\GIS—LVi|O~J\J\H[‰CLtO QTHK]D` J_QTS=PÌHsJ3ODisjƒOtfRJx]DheQTGIS  º QkfqP}S‹OD`^b LVOD` QTG^SVU\Sph]YU\Sg`I]~Q€}S¿J_]YisZDSVb—af—OÈJ\HKwxPIisS‘PrSVU\L=SVP^Q\U\]Y`„uˆEG^Snr]q]Yl]Dh.؆Hs`IJ\laf—O~`Ib .ODPrSVUŠQ À ÃGCO~Jd}S=SV`‚ZDSVU_f‚HK` c^SV`aQTHKODi.O~`IbŽQTG^SVHKU¢LVU_HsQTH mcIS…]DhyQ\GIS{PrSVU\L=SVP^Q\U\]Y`·IODJ\HsLtODisisf—lmHKisiKSVb QTG^S€‰ISViKbx]Dh}`ISVc^UTODiC`ISpQkjƒ]DU\lmJˆh]DUyOnis]Y`IW¿PrSVU\Hs]Rb„u.EX]RbCOtfÈH[QHKJ]~hœQTSV`xJ_QTO~QTS=bxQ\GCO~QyJ_HKwxPIiKS PrSVU\L=SVP^Q\U\]Y`^J‘ODU_Sg]Dh.G^HKJ_Q\]YU\HsLtODi}HK`aQTS=U\S=J_Qe]Y`Ii[fYu EGIHKJHsJVzYGI]?jFS=ZDSVUVzR`I]DQyL=]YwP^iKS=Q\SVi[fnQ\U\cIS~uy؆]YU_S€U\SVL=SV`aQTi[f{QTG^SHKbISVOd]Dh„P^U\SVP^U\]qLVSVJ_J\Hs`IW h]YisiK]?jƒS=b’af’O†LViKODJ\J_Hs‰CLVO~QTHs]Y`’afJ\HswPIisS3P}S=U\LVS=P^QTU_]Y`IBJ œQTGIHsJ»HsbIStO†HK`hÇO~L=Q»`IS=ZDSVUÈLV]Dw3• PIisS=QTS=isfbIHKS=b ˆGIODJLV]YwxS‘IODLNlHs`QTGIS¿L=]Y`aQTSÐoRQ€]~Fh + vRc^PIP}]DU_Q”ySVLpQT]YU؎ODLNGIHs`ISVJ=Å ¡vq”¿2Ø pu U\S=PIU\]qLVS=J\J\Hs`IWxmf%LV]Y`aZY]DiKc^Q\HK]Y`%j€H[QTG‹O{J\SpQ¢]~h;l~SVU_`ISVisJewODPIJ¼Q\GIS¿PIU_]YIiKS=wHs`mQ\]3O{GIHKWDG ÇPr]DQ\SV`aQTHKODiKi[fHK`^‰C`^HsQTS nbIHswS=`IJ\Hs]Y`CODiyhStO QTcIU_S%J\PCO~LVSDuÊËk` QTG^ShStO~Q\cIU\SÎJ_PCODL=S%QTGIS%J_SVPCOA• UTO QTHK]D`‹j€H[QTGŽwO oRHKw…cIwwODU\WDHK`—HKJeLNGI]YJ_SV`‡u€EGIS»LVi|O~J\J\H[‰CLtO QTHK]D`†O~PIPIU\]YODLNGŽHsJeQ\GISVU_S=h]YU_S Spo^ODLpQTi[f%QTGISg]Y`^Snc^J\SVb†HK`ÎQTG^S +¥]YP^Q\HKwODi„PrSVU_LVS=P^QTU_]Y`‡Å[u  J\Hs`IWnOnJ_wO~U_QˆwO QTGIS=wO~Q\HKLVODiRh]YU_w{cIiKO~QTHs]Y`{]Dh}QTGIS€PIU_SVPIU_]RL=SVJ\J_HK`IW»J_Q\SVP‡zqHsQˆHKJPr]YJ_J\H[• IisSdQT]xOtZY]YHsb—Q\GISgSpoRPIisHKLVH[QLtO~iKLVc^i|O~Q\HK]Y`%]~h;Q\GISdhStO~Q\cIU\S=J€mf—L=]Y`aZY]Yisc^QTHs]Y`‡uEG^S»PIU_]YIisSVw ]Dh/‰C`^bIHK`IWnQTGIS‘]YP^QTHswO~iCJ\SVPIODUTO~Q\HK]Y`LtOD`%rS¢h]DU\wODcIi|O QTSVbxbIHsU\SVLpQTi[fxODJO mcCO~bIUTO~Q\HKL¢PIU\]~• WYU\ODwwxHK`^W{PIU\]DIiKS=w†Fu "C]YUb^S=QNO~HKiKJJ_SVS~zCSDuÂWIuszXÀ ÚYÚ ÃÄu 


   



'&









ϕ

1

w1 xout wn ϕ

n

adaptive weights

fixed weights

linearly separable problem

smart preprocessing

Ú u Ú ÁR˜—EGIS†HsbIStO]Dh¢QTGIS—PrSVU\L=SVP^Q\U\]Y`„u̐&PIU\S=PIU\]qLVS=J\J\Hs`IW’JŠQTS=PÌHKJ{h]DiKiK]?jFSVbafO J\HswP^iKSdPrSVU_LVSVPRQTU\]D`‡u ®3¯ ¶·Ô=²¹VÔVÓ xÖ¿³ 5Ö¿³y¸eÖ Ž²³´±·× ؆cIi[QTHsi|OtfYS=UP}S=U\LVS=P^QTU_]Y`IJ‘ODU\Sg]YU_WaOD`IHsŒVSVb‹HK`—iKO?fDSVU_J¢]Dh.`ISVc^U\]Y`IJ¢OD`Ib‹HKwxPIisSVwxSV`aQ€OÈhSVSVbq• h]YUŠj¼ODU_b PIU\]qLVS=J\J_HK`IW·LNGCO~HK`‡u‚EG^SÎi|OtfYS=U\J{rS=QkjFSVS=`7Q\GIS%Hs`IPIc^QÈ`^]RbIS=JÈOD`IbQTGIS%]Dc^QTPIcRQ `IS=cIU\]Y`^J»O~U\SÈLtO~iKiKS=b‚GIHsbIbISV`iKO?fDSVU_JVuÈ؋cIisQ\HKi|OtfDSVU XSVU\L=SVP^Q\U\]Y`^J…ODU_S{wx]YU\S SpoRHKIisS…QTGIOD` J\HswP^iKS€P}S=U\L=SVP^Q\U\]Y`IJ=uEGIS‘wx]YJŠQƒHswPr]YU_QTOD`aQyiKSVODU\`IHs`IWgU\cIisSeh]DUˆ`^S=QkjF]YU\l3QTU\ODHK`IHs`IWgHKJQTG^S CODLNlmPIU_]YPCODWYO~QTHs]Y`‹ODiKWY]DU\HsQ\GIw‹u 











$

 

'





]=#0 

(*) -).(

/

b#A@4

'D9032

Ëk`ŽQ\GIS»P^U\SVL=SVbIHs`IWJ_SVLpQTHK]D`†jFS…GCOtZYS»J\S=SV`ŽQ\GCO~QeQTGISnJ\HswPIisSgP}S=U\LVS=P^QTU_]Y`‚LVOD`Ž]Y`^isfÎJ_]YisZDS iKHs`IStO~U\isf‹J_SVPCODU\ODIisS{PIU\]DIiKS=wJ=unEG^SÈPIU\]DQ\]DQkfqP}S…]DhFOD`·Spo^ODwxPIiKS»j€GIHKLNGHKJ¢`I]DQgisHK`ISVODU\i[f J\S=PCODU\ODIiKS†HKJ3QTGIS  º PIU_]YIisSVw‹uÍË Q%HKJxJ_Q\UTODHsWYGaQ\h]YUŠj¼O~U\b Q\]’L=]Y`IJ_Q\U\cILpQ—OÊQkjF]~• i|OtfYS=U PrSVU\L=SVP^Q\U\]Y`j€HsQTGQkjF]Ž`IS=cIU\]Y`^J…HK`ÊO—GIHKb^bISV`i|OtfYS=Unj€GIHsLNG’J_]YisZDSVJgQTGIS  º • PIU\]Y^iKSVw‹¾ ".HsWIu Ú u Ú ^uEGISΉCU\JŠQ3`ISVc^U\]Y`HK` Q\GISÎGIHKb^bISV`7i|OtfDSVUÈcIJ_SVJO‚iKHK`^StODU{J_SVPCO~UTO~Q\HK]Y` j€GIHsLNG J\P^iKHsQ\J‘]DÆÊQTG^S º " " º " "HK`IPIcRQtudEGIS…J\SVL=]Y`Ib`IS=cIU\]Y`·J_SVPCO~UTO~Q\SVJdQTG^S º º Hs`IPIc^Q hU\]DwQTGISgU_SVJ_QVuyË hr]DQTG—`IS=cIU\]Y`^J¢Hs`—Q\GISgGIHKb^bISV`‹i|OtfYS=U€ODU\SgHs`CODL=Q\HsZDSDz^Q\GISV`‹QTGISgHK`^PIc^Q€HKJ SVH[QTGIS=U º º " "Ê]DU º " " º nIu %*GCO~QFU\S=wODHs`IJh]YUQTGIS¢`ISVc^U\]Y`HK`3QTGIS¢]Yc^QTP^c^QƒiKOtfYSVUˆHKJ `I]?jOiKHs`IStO~U\isf%J\SVPIODUTOD^iKS…P^U\]YIisSVw‹u€EGISÈODr]?ZYSÈODU_WYcIwxSV`aQTJ¢J\GI]?jÉQTGIO~Q¿Oxw{cIisQ\HKiKO?fDSVU PrSVU\L=SVP^Q\U\]Y`7HsJ…wx]YU\S SpoRHK^iKS%O~`Ib LtOD`„zHK` PIU_HK`IL=HKPIisSDz;J\]Yi[ZYS%PIU_]YIisSVwxJ»QTGIO~Q3OŽJ_HKwxPIiKS PrSVU\L=SVP^Q\U\]Y`—LVOD`I`I]DQ¼J\]Yi[ZYSDuEG^S¿LV]Y`IJŠQTU_cIL=Q\HK]Y`%b^]RS=J`I]DQtzRGI]?jƒSpZYS=UVzIQTS=iKircIJ¼GI]?j QT]…‰I`Ib QTG^S]YPRQTHKwODiXjƒS=HKWYGaQTJnHK`ÊO—WYSV`^SVUTO~i¡zMP}]~QTSV`aQ\H|ODisisf†w{cILNGÊw]YU_S3LV]YwxPIiKHsLtO~Q\SVb·J_HsQTcIO~QTHs]Y`‡u EX]WYSpQ€jƒS=HKWYGaQTJeODcRQT]YwO~QTHsLtODisisfDzmjFSn`ISVS=bŽO3iKSVODU\`IHs`IW{U_cIiKS~u '&

'&






 "

 "



"

"

XOR problem is solvable

first layer x

second layer

#2 On/On

Off/Off o

o Neuron #1 active

o Off/Off

x

x XOR

o #1 On/On

Neuron #2 active

Ú u Ú ^˜.vR]YiKcRQTHK]D`{]Dh©QTGIS  º P^U\]YIisSVw af3OgÁA•kiKOtfYSVUPrSVU_LVS=P^QTU_]Y`@ iKOtfYSVU]~h}Hs`IPIc^Q `I]qbISVJ puyEGIS=U\S»ODU\S¿QkjF]`IS=cIU\]D`IJ¢Hs`ÎQTGISgGIHsbIbISV`—iKO?fDSVU=u 


'&






(*) -),+

Q

^




b#A@4Eb#QR:EQI;>7G4

8

D9:*4EA7 ;>='2

E GIS
¼ODLNlmPIU\]DPCODWaO~Q\HK]Y`ODisWY]YU\H[QTGIwxJdHsJgQ\GISxWYUTODb^HKSV`aQ»bISVJ_LVS=`mQ{U\cIisSO~PIPIiKHsSVb·QT]ŽO‹w…cIi[• QTHsi|OtfYS=U3P}S=U\LVS=P^QTU_]Y`‡u &LVisS=ZDSVUxcIJ\S†]Dh‘Q\GIS—LNGCODHK` U\cIisS‹ODiKis]Aj€JÈc^JxQ\]’HKwxPIisSVwxSV`aQ{QTG^S ODisWY]YU\H[QTGIw UTO QTGIS=US('xLVHsSV`aQTi[fYu %’S{c^J\S{QTG^S{`I]DQTO~QTHs]Y`ŽHs`mQ\U\]qbIcIL=SVb·HK`2".HKWIu Ú u ÁOD`Ib ©uC Ú u IÐu¿‘JdHs`ŽQTGIS…J\SVLpQTHs]Y` ]Y`‹WYUTO~bIHKS=`mQ€b^SVJ\L=SV`aQtz©QTG^S»O~HKw HKJ¼QT]xwxHK`IHswHsŒVS€QTGIS mcCODbIU\O~QTHsL¿SVU_U\]YU  S . ÁÚ 6 6 š    8       Ú u Á E   j€GIS=U\SdQTG^S‘J\c^w ]?ZYS=UŸ.U_cI`IJ¼]?ZYS=UODisir`^SVcIU_]Y`IJ¼Hs`QTGIS¢]Yc^Q\PIc^Q¼iKOtfYSVU=uËk`Î]YU\b^SVUFQ\]ÈODPIP^isf QTG^SnWDUTODbIHsSV`aQ€bIS=J\LVS=`aQdU\c^iKS  Ú uÁ Ú      . 8  žmž   jFS†`IS=SVb7QT]LtO~iKLVc^i|O~Q\SQTGIS—bIS=U\H[ZDO QTHsZDS—ž Dža   u EGIS—J_cIP}S=U\J_LVU\HsP^Q U_S=hS=U\JÈQT]·QTG^S i|OtfDSVUVu "C]YUyQTG^S‘JTO~lDS‘]~h/J\HswPIisHKS L=HsQkfYzYjFS‘Hs`aQTU\]qbIcIL=S¢Q\GIS¢Q\]DQNODi©Hs`IPIc^QˆQT]»`ISVcIU_]Y`%Ÿ;HK`xi|OtfYS=U ODJș    .       gj€GIS=U\S    .   pBu %’ScIJ\SQ\GISLNGIODHK` U_cIiKS OD`Ib—j€U\H[QTS ž . 6 ž žq™    £  Ú u ÁYIÁ  žq™   ža   ža   A/S=QecIJ€bISp‰C`IS    F. 8‘ž ~žR™  uEGIHKJfqHKS=iKbIJ€OÈiKSVODU\`^HK`IWÈU_cIiKSg]DhXQTGIS¿h]YU_w  Ú uÁ      . 6      S j€GIHsLNG‹jFS»U_SVL=]YWY`IHsŒVSD¾CLph–u qJ=u  Ú u Ú ÁRz Ú u Ú puEGIS …hÇODLpQT]YU_J€ODU\S   4 . š    8       4  Ú uÁ  













 

























 





 

















L




























 































 
























P
































δ

(1)

δ

(2)

(1)

wjk x (0) k

x (1) j

Ú u Ú I˜
¼ODLNlmPIU_]YPCODWaO QTHK]D`‹]~h;Q\GIS …ZDO~iKcIS=JVu h]YU¼Q\GISg]Yc^QTP^c^Qei|OtfYS=U . ¬ÊOD`^b  S .  S   6        Ú u Á I h]YUˆODisiC]DQTGIS=Uyi|OtfYS=U\J Ú ¬ 8 Ú u.EGIS¢‰C`IODiCS mcCO~Q\HK]Y`  Ú u Á WYO?ZDS‘U_HKJ\SQ\]nQ\GIS¢`IODwS ÅÂCODLNlmPIU\]DPCODWaO~Q\HK]Y`Ì]Dh¿SVU_U\]YU_JVÅyh]YU3QTGISŽODiKWY]DU\HsQ\GIw‹u EGIS t• hÇO~L=QT]DU\JODU\S†J\SVS=`*ODJis]RLVODi SVU_U\]YU_JVudEX]%LVODiKL=cIi|O~Q\SnQ\GIS{S=U\U\]DUdO~Q‘i|OtfYS=U 8 Ú z©jƒSÈ`^SVSVb‚Q\GIS…S=U\U_]YU¿O~Q¢i|OtfYS=U j€GIHsLNG HKJeQTGIS=U\Sph]YU\S»PIU\]YPIODWaO~Q\SVb‚CO~LNlmjƒODU_bIJVu ZDSV`·QTGI]DcIWYGŽ]DcIU‘bIHsJ\L=cIJ\J_HK]Y`‚GIODJ¢h]qLVcIJ_J\S=bÊ]D` `ISpQkjƒ]YU_lmJ%j€HsQTGÌOJ_HK`IWYisS—GIHKbIb^SV`ÌiKOtfYSVU=z¼QTGIS‹JTO~wS†ODiKWD]YU\H[QTGIw LtOD`ÍO~iKJ\]ÊrSŽcIJ_SVbÌh]YU `ISpQkjƒ]YU_lmJ¢j€HsQ\G—J\S=ZDSVU\ODi;G^HKbIbIS=`—i|OtfYS=U\JVu  WYU\ODbIHsSV`aQnbISVJ_LVSV`aQxODisWY]YU_HsQTG^wJdL=]Y`aZYS=U\WYS=J…QT]†iK]qLtODiwxHK`IHswORuË Q»HKJg`I]DQ»WYcCO~UTOD`R• QTS=SVb‚Q\GCO~Q¿H[Q¢‰C`^bIJ¿OWYis]YCODiMwxHK`IHsw{cIw‹uË Q¿HKJeQTGIS=U\Sph]YU\S»`ISVL=SVJ\J\ODU_f‚HK`·ODP^PIiKHsLtO~Q\HK]Y`IJQ\] U\S=J_QTODU_QnQTGISODisWY]YU\H[QTGIw&J\SpZYS=UTODiyQTHswS=Jdj€HsQTGÊbIH[ƄS=U\SV`aQnHK`^HsQTHKODi.L=]Y`IbIH[QTHs]Y`IJVuxEGISx]Y`IisHK`IS ZYS=U\J_HK]Y`‚]~hQTGIS»ODisWY]YU\H[QTGIw j€GIHKLNGŽHK`aQTU_]RbIc^LVSVJ¢OL=SVUŠQNODHs`†bIS=WYU\S=S…]DhJ_QT]qLNGCODJŠQTHsLVHsQkf—G^SViKP^J QT]…SVJ\LVODPrS‘hU\]Yw J_wO~iKi©iK]qLtODiCwxHK`^HKwO^zYIc^QFQTGIS=U\SdwOtf3}S¢bISVS=PÎiK]qLtODi©wxHK`IHswOdHs`j€GIHsLNG QTG^S»O~iKWY]YU_HsQ\GIw HsJ¼QTU\ODPIPrSVb‡u       -   V'2

;

03A2

B

Ëk`·PIU\ODL=Q\HKLtO~i;O~PIPIiKHsLtO~Q\HK]Y`^JeQ\GIS…PIc^U\S…h]YU_w½]DhˆOWYU\ODbIHsSV`aQdbIS=J\L=SV`aQnODiKWD]YU\H[QTGIw"HsJ‘U\ODU\S=isf cIJ_SVb‡ueEGISVU_SÈODU_S»J_S=ZYS=UTODiXQTU_HKLNlmJ‘O~`Ib‚wxS=QTG^]RbIJ€QT]J\PrSVS=b‚cIP‚L=]Y`aZYSVU_WYSV`^LVSDu º `IS…JŠQNOD`R• bCODU_b‹Z~ODU_H|O~Q\HK]Y`]DhXWYUTODb^HKSV`aQ€bIS=J\L=SV`aQdHsJƒQTGISnODb^bIHsQ\HK]Y`%]~hOÈwx]YwS=`aQTcIw QTSVU_w†u¼QeSVODLNG HsQ\SVU\O~QTHs]Y`‡zFQ\GIS†jFSVHsWYGaQLNGIOD`IWYS‚l~SVS=PIJÎOisHsQ_QTiKSÎ^HsQx]DhdQ\GISŽbIHsU\S=L=QTHs]Y`]DhdQTG^SŽPIU_S=ZqHK]Yc^J jFSVHKWDGmQ¢LNGCOD`^WYSDuFEGqc^J¢QTG^SgjFSVHsWYGaQTJe}S=GCOtZYS…ODJ€HshXQTGISpf—GCODbŽJ_]YwxSnHs`ISVUŠQTHKO{]YU + wx]YwxSV`R• QTc^w†Å[u AMSpQcIJODJ_J\cIwxS QTGCO Q‚QTGIS7LNGCO~`IWYS ]Dh3jFSVHKWDGmQ   HK`#Q\GIS i|O~J_Q‚Q\HKwxS’J_Q\SVPÉj¼O~J     L





 

Z

 



G"
















#$



 

pu

 9
"F









&



*

$

"





(*)

),+

TU4

8'D9798#03QRA



b

;>7.257




QI;>7G4

8

†Ø ]YU_SÎWYSV`^SVUTO~iKisfjFSÎwOtfÊj€HKJ_G QT]·]YP^Q\HKwxHKŒ=SxQ\GIS%LVS=`aQTSVU_JxB OD`^b QTGISÎPCO~UTODwxS=Q\SVU\J…]Dh QTG^SCO~J\HKJdhc^`IL=Q\HK]Y`IJgafÊO—J\cIPrSVUŠZRHsJ\S=b’iKSVODU\`^HK`IWÎPIU_]RL=SVbIc^U\SDuC"C]YiKis]?j€HK`IWQ\GISxJTODwxS3ODUŠ• WYcIwxSV`aQ\J3ODJ…h]YU…Q\GISÎbIS=U\HsZ~O~Q\HK]Y`]Dh€QTGISCODLNlmPIU\]DPCODWaO~Q\HK]Y`7O~iKWY]YU_HsQ\GIw†zMjFSÎLtOD`ODPIP^isf WYU\ODbIHKS=`aQgbISVJ_LVSV`aQgQT]‹B O~`Ib·QT]%QTG^S{ZDO~U\H|O~`ILVS unEGIS3bIS=U\HsZ~O~Q\HK]Y`Ž]DhFQ\GIS{h]YU_w{cIi|O~J‘HKJ



J_Q\UTODHsWYGaQ\h]YUŠj¼O~U\b‡zˆIc^QÈQ\SVbIHs]YcIJVu ¢`I]DQ\GISVU3Pr]YJ\J_HKIHsiKHsQkfÊ]Dh¢PCODU\ODwxS=QTS=UÈ]YP^Q\HKwxHKŒtO QTHK]D`’HKJ WYH[ZYSV`Êaf‚QTG^S .oRPrSVL=QTO~QTHs]Y`؎O oRHKwxHKŒVO~QTHs]Y`·ODisWY]YU_HsQTG^w†ugËk`ÊiK]?jb^HKwxSV`IJ_HK]Y`IJ‘HsQgHKJdODiKJ_] Pr]YJ\J_HKIisS†QT]’cIJ\S~zeHs`IJ_Q\StODbÍ]~h¿UTODb^H|ODielDS=U\`IS=iKJVz ODcIJ_J\H|O~`IJj€H[QTGÍODU_IHsQ\UTODUŠfLV]?Z~ODU\HKOD`ILVS wO~QTU_H[o}u          dS=‰I`IS¿O{J\SpQ]DhXODr]Yc^QÁ E{bCO~QTO…Pr]YHK`aQTJ j€GIHsLNG†fY]wOtfÎjƒOD`aQ¢Q\]b^HKJ_Q\U\HsIc^QTSgHs`mQ\]xQkjF]L=iKcIJŠQTS=U\JVuvRS=QeQTGISg`mcIw{rSVUe]Dh.lDSVU_`ISVisJ¢Q\] QkjF]Iu % O QTLNGG^]AjÌQTGIS€Qkjƒ]»lDS=U\`IS=iKJƒwx]?ZYS€QT]gQTGISeLViscIJ_Q\SVU_JƒOD`^b%ODb _c^J_QyQTGIS=U\S¢LV]?Z~ODU\HKOD`ILVS wO~QTU_H[oJ\]7O~JQT] ]DP^QTHswODisisf wx]qbISViQTG^S‚bCO~QTO bIHKJŠQTU\HsIc^Q\HK]Y`‡u %*GCO~Q%GCODPIPrSV`IJÎH[hdQTG^S `mcIw{rSVU€]Dh;lDS=U\`IS=iKJebI]RS=J€`I]DQewO QTLNGÎQTGISg`mcIw{rSVU€]Dh;LViscIJ_Q\SVU_J 

c





%





 


















>$

H





(*)



)('

?7GB

V#B

B>7G4

8

EGIS—hUTODwxS=jF]YU_l ]Dh‘Q\GIS—UTODb^H|ODi¼IODJ\HsJ3hc^`IL=Q\HK]Y``IS=QkjF]YU_l wODl~SVJ3QTGIS‹LV]D`I`ISVLpQTHs]Y` rSp• QkjFSVSV`·`^SVcIU\ODi;`^S=QkjF]YU\lmJgOD`Ib†LViKODJ\J_HKLtO~i;O~PIPIU\]toRHKwO~Q\HK]Y`ÎQTS=LNGI`IH qc^SVJdHs`†JŠQNO~Q\HKJ_Q\HKL=J¢wx]YU\S QTU\OD`IJ_PCODU\S=`aQtu*Ëk`ÌhÇODLpQtz¼wxS=Q\GI]Rb^Jxj€GIHsLNGÌcIJ_S‚OÊJ\cIPrSVU_P}]YJ_HsQ\HK]Y`]Dh¿J_S=ZYS=UTODi O~cIJ\J_H|OD`IJ ODU_S%LV]Ywxwx]Y`Iisfc^J\SVb7Hs` bISV`^J\HsQkf’SVJŠQTHKwO~Q\HK]Y`‡u†Ëk` J_QTO~QTHsJ_QTHsLVJnQTGIS=J\SÎwxS=QTG^]RbIJ{ODU\SODiKJ_] LtO~iKiKS=bwxH[oqQ\cIU\S3wx]RbIS=iKJg]YU=z;wx]YU\SxJ\PrSVL=Hs‰CLVODiKi[fYzXO†wxH[oqQ\cIU\S3]Dh ODcIJ_J\HKOD`IJVuEGISwODHK` bIH[ƄS=U\SV`^LVSÈHKJeQTGCO QdUTO~bIH|ODiXCODJ_HKJ‘hc^`IL=Q\HK]Y`‚`^S=QkjF]YU\l†cIJ\cCO~iKisf†ODJ_J\cIwxS…Q\GCO~Q¿J_S=ZYS=UTODi.]Yc^Q_• PIc^Q…cI`IH[QTJgLtOD`’J\GCODU_SxQ\GISJ\ODwxSxIODJ\HsJnhcI`^L=QTHs]Y`IJ   Ðz Ú ¬.uÎEGISJ_QTOD`IbCODU_b ODPIP^U\]aODLNGHK`7bIS=`IJ\H[Qkf7SVJŠQTHKwO~Q\HK]Y`’jF]YcIisb7}SÎQT]cIJ_S‹h]DUÈStODLNGLViKODJ\JxOD`7HK`^bISVPrSV`Ib^SV`aQ J\SpQn]DhˆCODJ\HsJdhcI`ILpQTHK]D`IJVu{vRSVSÈQTG^SÈ}]q]Yl‹]Dh
FHKJ\G^]YP‚h]YUgOD`·Hs`R•kbIS=P^QTGbIHsJ\L=cIJ\J_HK]Y`]DhyQTG^S U\S=i|O~Q\HK]Y`ÊÀ DÃÄu ‘`I]DQ\GISVU€O~b^Z~OD`aQNODWYS¿]~h;U\ODbIHKODi„CO~J\HKJ¼hc^`IL=Q\HK]Y`IJHsJƒQTGIO~QQTGISpf—ODisiK]?j*w]DU\SdSVODJ\HsisfÎO~` HK`aQ\SVU\P^U\S=QTO~QTHs]Y`·]Dhj€GCO Q¿QTGIS{`^S=QkjF]YU\l‚HsJdbI]YHs`IWIugEGIHsJdWY]qSVJdHs`‚Qkjƒ]—bIHsU\SVLpQTHs]Y`IJVu ";HKU\JŠQtz SpoRPrSVUŠQ—lq`^]Aj€isSVbIWDS·LtOD`*rS‚StODJ_HKi[f HswP^iKSVwxSV`aQ\SVbÍOD`^bÍcIJ\S=b*h]DUÎOD` Hs`IHsQ\H|ODisHKŒtO QTHK]D` ]Dh PCODU\ODwxS=QTS=U\J=u€vRSVL=]Y`Ib‡z}O~hœQ\SVUQTU\ODHK`IHs`IW{QTG^S»wODPIPIHs`IWÈQTGCO Q¢Q\GIS»`^S=QkjF]YU\l—GCODJ€h]YcI`^b†LVOD` rSOD`IODisfqŒVS=b‡uÎËk`’wOD`afLtO~J\SVJ»HsQnHKJgPr]YJ\J_HKIisSÈQT]ŽSÐoqQTUTO~L=QnQTGISxlm`I]?j€iKSVb^WYSHs`ÊQ\SVU\wxJn]Dh U\c^iKSVJ=u3EGISU_SViKO~QTHs]Y`‚rS=QkjFSVSV`’`ISVcIU\ODi`ISpQkjƒ]YU_lmJ{OD`IbÊSpoRPrSVUŠQ»lm`I]?j€isSVbIWYSxHKJ¿SÐoRPIiK]YH[QTS=b HK`ÎQ\GISgLV]Y`aQTSÐoqQ‘]DhFÅ ¢SVc^U\]~• "©c^ŒVŒ=fa• ¢S=QkjF]YU_lqJ=Åsu ®3¯ ‚Ö ³´ ·ÔVÖ¿Ò ´ Öd³ ·²;²¹t± -Ó{±·µ ÙÖ¿±‚Ö¿³Ó{ÔV¹ ¢Ó ²¹V´x± 

c

c













"







(*) -).(

)



,"



4R;@7,6RQI;>794E8



"



&%



 

SÈGCOtZYS3J\S=SV`HK`‚Q\GISÈPIU\S=LVS=bIHK`IWÎJ_SVLpQTHK]D`IJdQTGCO QnO%J\HKwxPIisS»PrSVU_LVSVPRQTU\]D`LVOD`·]Y`Ii[fŽJ_]YisZDS iKHs`IStO~U\isf’J\S=PCODUTO~IiKS—PIU_]YIiKS=wJ{j€G^SVU\SVODJ%Ow{cIi[QTHsi|OtfYS=U{P}S=U\L=SVP^Q\U\]Y` HKJ3w]DU\SÎPr]AjFSVUŠhcIi¡u EGIS mcISVJŠQTHK]D`%ODU_HKJ\S=Jj€GIS=Q\GISVUFHsQyHKJˆODi[j¼OtfqJybISVJ_HKU\ODIiKSQ\]»GCOtZDS‘O SpoRHsIiKS€`ISpQkjƒ]YU_l©uˆEG^S OD`IJŠjƒS=UÈHKJ{`^SVWaO~Q\HsZDSDu·Ëk` J_]YwSLtODJ_SVJÈHsQ…HKJ»}SpQ\QTS=U…QT]ŽGCOtZYSÎ`ISpQkjƒ]YU_lj€GIHsLNG7bI]RS=J{`I]DQ PrSVU_h]DU\w PrSVUŠhSVL=Q\isfŽ]Y`·QTGIS3bCO~QTOÎJ\S=QVuÈE;]‹cI`Ib^SVU\JŠQNOD`^bÊj€GafYzXiKSpQgcIJgU\S=LtODisi;QTG^S3QTODJ\lŽ]Dh J\c^P}S=U_ZqHKJ_SVbÎiKSVODU\`^HK`IWIu;EGISdbCO QNO»J_S=QHsJFc^J\SVb%Q\]»Q\UTODHs`%O»`^S=QkjF]YU\lxQT]…PrSVUŠh]YU\w OnbISVJ_HKU\S=b %



@ x

o

o

x

o o

o

x

o o o

x

o

o

o

o

x

o

x

x

x

x

x

x o

x

x x

o

x

x

o

o

9

x

o

x

o

o

x

o o

o

x

o o o

x

o

o

x

o

x

x

o

o

x

x

x

x o

x

o

o

x

x x

o o

x

x

Ú u Ú ^˜gEGISJ_SVPCO~UTO~Q\HK]Y`]DhƒO‹bIO~QNO—J\SpQ…OI¿afÊO—J\HswPIisS{PrSVU\L=SVP^Q\U\]Y`’OD`IbÊdafÊO w{cIi[QTHsi|OtfYS=UˆPrSVU_LVS=P^QTU_]Y` ÇJ_LNGISVwO~QTHsL ÐuEGIS‘w{c^isQTHsi|OtfYS=UyP}S=U\LVS=P^QTU_]Y`ÎPrSVU_h]DU\wxJFPrSVUŠhSVL=Q\isf ]Y`xQTGIS€bCO~QTOnJ_S=QVu %*GCO~QˆwO Q\QTS=U\J=zqG^]AjFS=ZDSVU=zIHK`ODPIPIisHKLVO~QTHs]Y`IJHsJyQ\GIS€PIU\S=bIHKLpQTHs]Y`xh]DUˆ`^S=j bCO~QTOnPr]YHs`mQ\Jj€GISVU_SeQ\GISeOD`IJŠjƒS=UFHsJˆ`^]DQylm`I]?j€`‡zRisHKl~S¼QTGISeP}]DHK`aQyHK`Ib^HKLtO QTSVbÈafÈQ\GISeODU\U_]Ajgu HK`^PIc^Q_• ]Yc^Q\PIc^QXwO~PIPIHK`^WIzASDuÂWIusz h]YUXODc^Q\]YwO QTHKLL=i|ODJ_J\H[‰CLtO~Q\HK]Y`„uXEG^SF‰C`CO~iq`^S=QkjF]YU\l¿j€HKisiYrS cIJ_SVb—]Y`  ·bCO QNO»j€GIHsLNG—ODU\S¢`I]DQƒPCODU_Qƒ]Dh/QTG^S‘QTU\ODHK`IHs`IWnbCO~QNO»CODJ\S~uyEGISdPrSVU_h]DU\wOD`ILVS ]Dh€QTG^SÎ`IS=QkjF]YU\l’GCODJ{QTG^SVU\Sph]YU\S%Q\]·}SwSVODJ\cIU_SVb7]Y`7`ISpjbCO QNOŽj€GIHsLNGGIODJÈ`I]DQÈrSVS=` cIJ_SVbŽb^cIU\Hs`IWÈQTUTO~HK`IHs`IWIu EGISCODJ_HKL¼HsbIStOdHsJHKisiKcIJŠQTUTO QTSVb»HK` ".HsWIu Ú u Ú ^u.Ëk`xO XQ\GIS¼bCO QNOdJ\SpQˆGCODJrSVS=`J_SVPCODU\O~QTS=b af‚OxJ\HswPIisS¿P}S=U\L=SVP^Q\U\]Y`‡u‘vRHs`ILVSgQTGIS»bCO~QTOJ\SpQdHsJ¢`I]~Q‘isHK`ISVODU\i[f%J\S=PCODU\ODIiKS»OxhSpjÉSVU_U\]YU_J U\S=wO~HK`‡udEG^SÈw]YU_S SÐoRHKIisS…`ISpQkjƒ]YU_l‚HK`Ž ¢HKJ¢LtODPIODIiKS…]DhˆOPrSVU_hS=L=QgJ_SVPCODU\O~QTHs]Y`‡u A/SpQ cIJg`I]?jJ\SVS3GI]?jQTGISÈ`ISpQkjƒ]DU\l·PrSVU_h]DU\wxJd]Y``IS=jbCO~QTO^z‡h]YUdSÐo^ODwP^iKS{QTG^SÈbCO~QNO%Pr]YHs`mQ HK`^bIHKLVO~QTS=b·mf‚OD`ODU_U\]?j HK`‚QTG^S{HK`IP^c^Q¿J\PIODLVS~u…EGISÈ`ISpQkjƒ]DU\l‚Hs`‚ ejF]YcIisbO%P^U\SVb^HKL=Q¿O LViKODJ\J_Hs‰CLVO~QTHs]Y` ODJ3O‚L=U\]YJ_JVzQTG^SÎ]Y`IS%Hs`7O …ODJÈO‚Œ=SVU_]IuEG^S‹P^U\]YIisSVwÙHKJnQTGCO~QxO‚P^U\HK]DU\H jFSŽbI]Y`„ÅÂQxlm`I]?j"j€GIHsLNGÌ]Y`IS—]Dh¢QTGIS—QkjF]’LViKODJ\J_Hs‰CLVO~QTHs]Y`IJÈHsJxL=]YU\U_SVLpQtu "©]DUxSÐo^ODwP^iKS—HsQ wO?f3rS¢Q\GCO~QˆQTGIS¢LVU_]YJ\Jˆj€GIHKLNG%LVODcIJ_SVbQ\GIS¢Q\]Y`IWYcIS¢HK`3QTGIS¢J\S=PCODUTO QTHK]D`r]YcI`IbCO~U_f3HKJyQTG^S 


P





P

U\S=J\cIi[QTJ¼]~hXO…`I]YHsJ_fwSVODJ\cIU_SVwxSV`aQO~`Ib%QTGCO QƒQTG^SdiKHslDSVisHKGI]q]qbx]~hMŒVSVU_]YJ¼HsJƒw{cILNGÎG^HKWYGIS=UƒHs` QTGIO~QdU_SVWYHs]Y`†Q\GCOD`ŽQ\GIS…]Y`^S»h]YUeLVU_]YJ\J_SVJVu‘EGISV`‚Q\GIS…PIU_SVbIHsL=Q\HK]Y`†]DhQTG^S{J\HswPIisSg`IS=QkjF]YU\l jF]YcIiKbÎHs`hÇODLpQ¼rSdrS=Q\Q\SVUˆQTGCOD`%Q\GIS‘]D`ISd]Dh/Q\GIS‘wx]YU\S SpoRHsIiKS¢`IS=QkjF]YU\l©uyË Q¼h]YisiK]?j€JyQTGCO~Q HK`PCODUŠQTHsLVcIiKODU{h]YU3`I]YHsJ_f7bCO~QTO^zFjFS‹GCOtZYS†Q\]L=]Y`aQTU_]YiQTGIS SpoRHsIHKisHsQkf ]~hd]YcIUx`ISpQkjƒ]YU_l©u º QTGIS=U_j€HsJ\S»Q\GIS…`^S=QkjF]YU\l—wO?f—isStODU_`†Q\GI$S +¥`I]YHsJ\S~ÅrHs`IJ_Q\StODb†]DhQ\GIS…cI`^bISVU_isfqHK`IWxJ_Q\U\cILpQTcIU_S ]Dh€QTG^SÎHK`IPIcRQ_•k]Dc^QTPIcRQÈwO~PIPIHK`^WI5u A/StODU_`IHK`IW‚Q\GISÎJ\PrSVL=Hs‰CLÎbIO~QNO‚J\ODwP^iKS%Hs`ILViscIbIHK`^W†QTG^S `I]YHsJ\SgHKJlm`I]?j€`ŽODJQTGISgPIU_]YIiKS=w]Dh;]?ZYS=U_‰IQ_QTHK`^WIu EGIS{JTO~wS»PIU\]Y^iKSVw½HsJ‘ODisJ\]%lm`I]?j€`·Hs`†hcI`ILpQTHs]Y`·ODPIP^U\]toRHKwO~QTHs]Y`‡u A/S=Qdc^JdJ\cIP^P}]YJ_S jFSGCOtZDS O bCO~QTOCO~J\Sj€GIHKLNG LV]Y`aQTODHK`IJ—`I]DHKJ_f HK`IPIcRQ_•k]Dc^QTPIcRQ†PIODHKU_J    _š   j€HsQ\G  _š Ë u¢EGIS»QNODJ_l‹]~hhcI`IL=Q\HK]Y`ŽODPIPIU_]to^HswO QTHK]D`†HsJeQ\]‰C`Ib‚Q\GIS…c^`IbISVU_isfqHK`^W J_Q\U\cILpQTcIU_S]~h»Q\GISbCO~QNO bIHsJ_Q\U\HK^c^QTHs]Y`‡u ‰IQ—af O Pr]Yisfq`I]YwxH|O~iej€H[QTG ODJ—wO~`mf hU\S=S PCODU\ODwxS=QTS=U\J¿O~J‘bCO~QTOPr]YHK`aQTJ‘ODiKis]?j€J¢]Y`^S…QT]%Hs`aQTSVU_P}]Di|O~Q\SgQ\GIS{bCO QNOPrSVUŠhSVL=Q\isfDz„Ic^QdwOtf LtO~cIJ\S·O~U\IH[QTUTO~U\HKi[f i|O~U\WYS—]YJ_LVHKisi|O~Q\HK]Y`^JVuÍEGIS‚]Dc^QTPIcRQ%]Dh¿Q\GIS†hcI`^L=QTHs]Y`Ìh]YUx`ISpjHs`IPIc^Q Pr]YHK`aQTJ  HsJ…QTG^SV`7wSVOD`IHs`IWYiKS=J\J=u WY]q]qb ‰IQÈJ\G^]YcIiKb7L=]Y`aQNODHs` hS=j hU\SVS—PCO~UTODwxS=Q\SVU\J j€GIHsLNG‹h]YU_LVS=JeQ\GIS¿hcI`IL=Q\HK]Y`ÎQ\]OtZDSVUTO~WYS…OtjƒOtfÎPCODUŠQ¢]~h;Q\GISg`I]YHKJ_SDu Ëk`’`IS=cIUTODiy`ISpQkjƒ]DU\lmJVz.Q\GIS3jƒS=HKWYGaQTJ»    {PIiKO?fŽQTGISxU\]YisS3]DhƒQ\GIS‰IQnPCO~UTODwxS=Q\SVU\J=u %S w{cIJŠQ¿QTGIS=U\Sph]YU\SÈL=]Y`aQTU\]Di.QTGISÈSpƄS=L=Q\HsZYSÈ`mcIw…}S=U¿]DhyjƒS=HKWYGaQTJ=uÈ؆]DU\SÈ]YU¿isSVJ\J‘J_fqJ_Q\SVwO~QTHsL wxS=QTG^]RbIJ]~h}Q\GIS€LV]Y`aQTU_]YiC]Dhrwx]RbIS=i SÐo^HsIHKisHsQkf»ODU_Sec^J\cCODisisf»J\cIwxwODU\HsŒVSVbÈc^`IbISVUQ\GISQTHsQ\iKS ]Dh€U_SVWYcIiKODU\HsŒtO~Q\HK]Y`PIU_]RL=SVbIc^U\SVJ=uÊvR]YwxS]~hQTGIS=J\S%wxS=Q\GI]Rb^J{ODU_SbIHsJ\LVc^J\J\S=b7HK`Q\GIS%`ISÐoqQ J\c^IJ\S=L=QTHs]Y`IJ=u            ]gCODLNl…Q\]gQ\GIS O~bIH|ODi
¼ODJ_HKJ "CcI`ILp• QTHs]Y` ¢SpQkjƒ]DU\l—OD`Ib—JŠQTcIb^f%GI]?j Q\GISg]YJ\L=HKiKiKO~QTHs]Y`IJˆHK`%Q\GIS¿U\S=PIU\S=J\SV`aQTO~QTHs]Y`†Hs`ILVU_StODJ_S¿HshMQTG^S `mcIw{rSVUF]Dh‡‰IQƒPCODUTO~w…QTS=U\JFODPIPIU_]aODLNGIS=JƒQTGIS¢`mcIw{rSVUˆ]Dh/bCO~QTO»Pr]YHK`aQ\JVu dS=‰I`ISdOnbCO~QNOnJ\SpQ j€GIHsLNG bI]qSVJ…`^]DQ{L=]AZDSVU…Q\GISxHK`IPIcRQ{J\PIODLVSS=ZYS=`IisfDzh]YUgSpo^ODwxPIiKS~z;fD]Yc LV]DcIiKbLNGI]R]DJ\SQ\] GCOtZYSÈwx]YU_S…Pr]YHK`aQTJeQT]?jƒODU\bIJ¢QTGIS{r]YU_bISVU_J‘QTGCO~`‚HK`ŽQ\GIS{wxHKbIb^iKSg]DhQTG^S{HK`aQTS=U_Z~ODiÄu %*GCO Q GCODP^P}S=`IJ 



















&














 

 



Z





Y





c

&

"

X$





(*) -),+

A@QX798'7.8#:5QX8#F

6RQXD979F'QI;>7G4

8

B

0Z;

*GCO~Q\S=ZDSVU€QTGISgJ\PrSVL=Hs‰CLgU_SVWYcIiKODU\HsŒtO~Q\HK]Y`wSpQTGI]qb‡z^jFSnODisjƒOtfqJeJ_PIiKH[Q¼QTG^SnbIO~QNOÈJ\SpQ ’HK`aQ\] QkjF]ÈPCODU_Q\JVz S OD`Ib uË h‡Q\GISdbCO~QTO»J_S=QL=]Y`aQNODHs`IJ¼O»Q\]DQNO~ir]~h *. Ú EIEIEnbCO~QNOnP}]DHK`aQTJ=zqjFS wO?fÈh]DUFSÐoIO~wPIisS¢J_SVisSVL=QFJ_Q\]RLNGIODJ_Q\HKLtO~iKisf S . EIEgP}]DHK`aQTJyh]YUybCO~QTO»J\SpQ S OD`^bcIJ_S¢QTG^S U\S=wO~HK`IHs`IW 8 S Pr]YHK`aQ\J¼h]YU u º `^isfbCO~QNO·J_S=Q S HsJÈcIJ\S=b bIc^U\HK`^WÊisStODU_`IHK`^WIuË QxHsJ{LtODisiKS=b QTGISÎQTU\ODHK`^HK`IW‚J_S=QVu ¼Q StO~LNG‚J_Q\SVPŽ]~h;Q\GIS»isStODU_`IHK`^WxP^U\]qLVSVb^cIU\S~z©jFSgQNODl~S…O3PCO~Q_QTSVU_` S z}O~`Ib†LVODiKL=cIi|O QTSdQTG^S jFSVHKWDGmQ¢cIPrbCO~Q\SDuy¢hœQ\SVU S cIPrbCO~QTS=J ]Y`IS  pzRjƒSnHK`aQTS=U\U_cIP^Q€iKSVODU\`^HK`IWxOD`Ib—QTS=J_Q€QTG^S PrSVU_h]DU\wOD`ILVS]~hrQ\GISLVc^U\U\S=`aQF`^S=QkjF]YU\lÈ]Y`ÈQ\GIS 8 S JTO~wPIisSVJ]~h„bCO QNOdJ\SpQ uEGIS€J\SpQ PIi|OtfqJ‘QTG^S{U\]YisS…]D&h +¥`IS=jgńbCO~QNOxj€GIHKLNG·QTG^S{`IS=QkjF]YU_l‚GCODJd`^]DQ¿J\S=SV`bIcIU_HK`IWQTUTO~HK`IHs`IWIu Ë Q¢HKJ€QTGIS=U\S=h]DU\SnLtODisiKSVb—Z~ODiKHsbCO~Q\HK]Y`ÎJ_S=Qt u %S{O~iKJ\]3LVODiKL=cIi|O~Q\SnO~hœQTS=U¢SVODLNG‚SVPr]qLNG†QTG^S…SVU_U\]YU ]Y`ÎQTGIS¿QTU\ODHK`^HK`IWÈJ_S=Qtu
ƒ]~QTG—SVU_U\]YU_J¢O~U\S¿PIis]DQ\Q\SVb‹ODJeO{hcI`IL=Q\HK]Y`—]Dh;iKSVODU\`^HK`IW{Q\HKwxSDz  wxStODJ_cIU\S=b†afÎQTG^Sn`mcIw…}S=Ue]~h.S=P}]qLNGIJVu dcIU\Hs`IW¿QTGIS¢HK`IH[QTH|O~i^PIGCO~J\Sd]Dh„iKSVODU\`IHs`IWn}]DQ\GQ\UTODHs`IHK`IWnOD`^bZ~ODiKHsbCO~Q\HK]Y`ÈS=U\U\]DUFcIJ_cCODisisf %























?J
















$




@   











E E val E learn training time

Ú u Ú R˜ º ZDSVUŠ‰IQ\Q\HK`IWIu dcIU_HK`IWÈQTU\ODHK`^HK`IW{j€H[QTG‹WYUTODb^HKSV`aQ€bIS=J\L=SV`aQdU_cIiKS=JVzCQTGISgisStODU_`IHK`IW SVU_U\]YU  bIS=LVU\SVODJ\S=J,ÇJ_]YiKHsb’iKHs`IS Ðu—EGISZ~ODisHKbCO~Q\HK]Y`S=U\U\]DU  bCODJ\G^SVbÐzGI]?jƒSpZYSVU=z wO?f%HK`IL=U\StO~J\SnODWaODHs`—O~hœQTS=UJ\]YwxS¿QTHswS~u bIS=LVU\SVODJ\S~u5EGIHsJwxStO~`IJQ\GCO~QxQTGIS‹`IS=QkjF]YU_l isStODU_`IJxQ\GISŽJŠQTU_cIL=Q\cIU\S‚]~h‘QTG^SŽbCO~QTO’ODJHsQ J\G^]YcIiKb„uÍË hdQTGIS†`IS=QkjF]YU_lHsJxQ\]R] SÐo^HsIiKS~zƒQTG^SV` Q\GIS†Z~ODisHKbCO QTHK]D` SVU\U_]YU  J_QTODU_Q\JQ\] HK`^LVU\SVODJ\S¢ODWaO~HK`xO~hœQTS=UˆOnL=SVU_QTODHK`x`mcIw{rSVUˆ]Dh‡S=P}]qLNGIJyj€GIHKisSQTGIS€QTU\ODHK`IHs`IW¿SVU_U\]YUyL=]Y`aQTHK`mcIS=J QT]·b^SVLVU_StODJ_SDuEGIHKJ{wxStO~`IJ…QTGIO~Q{QTG^S%`IS=QkjF]YU_l `I]?jisStODU_`IJ…Q\GIS%J\PrSVL=Hs‰CLÎSÐo^ODwP^iKSVJ…]Dh QTG^BS Ç`I]DHKJ_f €bCO~QNOJ\S=QVu¿EGISÈWYS=`ISVU\ODiKHsŒtO~Q\HK]Y`—QT]%`^S=j bCO~QTO^z„GI]?jFS=ZDSVUVz/WDS=QTJ¢jƒ]YU_J\S~unEG^S `ISpQkjƒ]YU_l—J_QNO~U_QTJQ\B] +¥]?ZYS=U_‰IQVÅ©Q\GISgbCO~QTO^¾IL=h–u ".HsWIu Ú u Ú qu               0

;>V#A@0

º ZYS=U_‰IQ_QTHK`^W…]qLVLVc^U\JˆHshrQTGISeS=Æ}SVLpQTHsZDS¢`mcIw…}S=UF]DhrhU\S=S¢PIODUTODwxS=Q\SVU_JˆHs`xQ\GISe`IS=QkjF]YU\l3HsJQT]q] i|O~U\WYS~uËk``IS=cIUTO~ir`^S=QkjF]YU\lmJVzRQTGIS¢`qc^w{}S=UF]~h/PCODU\ODwSpQTS=UˆHs`ILVU_StODJ_SVJˆj€HsQTGxQTGIS¢`mcIw{rSVUˆ]Dh `IS=cIU\]Y`^JdHK`‹QTGIS»GIHKbIb^SV`‚iKO?fDSVU=u¢J\HKwxPIisSnP^U\]qLVSVb^cIU\SnQT]%L=]Y`aQTU\]DiXQ\GIS SÐoRHKIHsiKHsQkf%]DhQTG^S `ISpQkjƒ]YU_l…HsJXQ\GISVU_S=h]YU_SFQT]¢QTSVJŠQ.Z~ODU_HK]YcIJX`ISpQkjƒ]YU_l{ODU_LNGIHsQ\SVLpQTcIU_SVJ;j€H[QTG{b^HsÆ}SVU\S=`aQ`mcIw{rSVU;]Dh `IS=cIU\]Y`^JVuË h©QTGISFZDO~iKHKbIO~QTHs]Y`»S=U\U_]YUHK`^LVU\SVODJ\S=JLV]Y`^J\HKJŠQTS=`mQ\isfnQT]?jƒODU\bIJQ\GISSV`^bx]~h©iKStO~U\`IHs`IWIz QTG^SV`·QTGIS…`IS=QkjF]YU\lŽHKJ¢Q\]R] SpoRHK^iKSDugËk`‚Q\GIHKJdLVODJ\S~zMOJ\wODisiKSVUe`ISpQkjƒ]YU_lŽj€HsQ\G·O%U\S=bIcIL=SVb `mcIw{rSVU3]Dh¢`ISVc^U\]Y`IJxJ\G^]YcIiKb7rSÎQTU\HsSVb‡u º ` QTG^S‹]~QTGIS=UxGIOD`Ib‡zˆHsh€QTG^S‹`^S=QkjF]YU\l HsJ{QT]q] J\wODisi¡z^Q\GISV`‚r]DQ\G†QTG^S»isStODU_`IHK`IWOD`Ib‹Z~ODiKHsbCO~QTHs]Y`ÎSVU_U\]YU¢ODU_S»cI`^`ISVL=SVJ\J\ODU\Hsisf‹i|O~U\WYSgO~QeQTG^S SV`^b ]DheisStODU_`IHK`IW^u·EGIS—]YP^QTHswO~iFJ_HKŒVS]Dh€QTGISÎ`ISpQkjƒ]DU\l HKJ{ODLNGIHKSpZYS=b H[hZDO~iKHKbIO~QTHs]Y`’O~`Ib iKSVODU\`^HK`IWÈS=U\U\]DU¢O~U\S¿iK]?j5O~hœQTS=U€LV]Y`aZYS=U\WYS=`ILVS»]DhXQTGISgiKSVODU\`^HK`IWÈPIU_]RL=SVbIc^U\SDu



(*) -)

/10C:




V'DGQRA7

8

 

QI;>7G4

0CQRAD



B

;>4Eb'b'798#:

ƒGCOD`^WYHK`IWQTGIS{`^S=QkjF]YU\l†ODU\LNG^HsQTS=L=Q\cIU\S{U_SVPrStO~Q\SVbIi[fŽHsJdO3QTSVb^HK]YcIJ‘PIU\]qLVS=bIcIU\S~ug؆]YU_SV]?ZYS=U HsQ…HKJ{U\O~QTG^SVU{L=]aODU\J_S%HK`QTGIS%J_SV`IJ_SÎQTGCO~Q{StODLNGODbIb^HsQTHs]Y`CODiy`IS=cIU\]D` wSVOD`IJ{J_S=ZYS=UTODi`^S=j PCODU\ODwxS=QTS=U\J¢   ?ueEGmcIJ‘HsQeHKJe]DhœQTS=`Ž`I]~QdLV]DwPIisS=Q\SVisf%LViKSVODU€j€GCO~Q‘Q\GISn]YP^QTHswO~i/`IS=QkjF]YU\l J\HsŒVSgU\SVODiKi[fHsJVu ÍJ\HKwxPIisSVU.jƒOtfÈ]Dh SpoRHK^HKiKH[QkfnLV]Y`aQTU_]YiCHKJWDHsZYS=`3mfÈOgwxS=Q\GI]qbxLVODiKisSVbÈSVODU\i[f{J_Q\]YPIPIHs`IWIu EX]Êc^J\SÎQTGIHsJ{wxS=QTG^]Rb‡zjFSÎQNODl~S—O·`IS=QkjF]YU\lj€GIHKLNG7HKJ…J\iKHsWYGaQTi[fÊQ\]R]·iKODU\WDS%J\]·QTGIO~Q{QTG^S Z~ODiKHsbCO~Q\HK]Y`SVU_U\]YUnJ_QTODU_Q\J»Q\]‚HK`IL=U\SVODJ\S=J{O~hœQ\SVU…J_]YwxSxQ\HKwxS3š u—EGIS}S=J_Q{J_S=QÈ]~h¼jƒS=HKWYGaQ\J HKJ¿Q\GISVU_S=h]YU_SxQ\GIS]D`ISO Qnš •‘OD`IbÊ`I]DQgQTGISx]Y`ISO~QgQTG^SS=`Ib’]DhƒiKSVODU\`IHs`IWIu dcIU_HK`IWÎQTG^S iKSVODU\`^HK`IW{P^U\]qLVSVJ_JjƒS¿QTG^SVU\Sph]YU\S¿l~SVS=P†QTG^S¿J\S=Q€]~hXjFSVHsWYGaQTJeO~Q¼š HK`—O{J_SVPCO~UTO~Q\S¿wS=w]YUŠfYu EGIS=J\S¼O~U\SQTGISyjFSVHKWDGmQ\J/j€GIHKLNGnjƒSyj€HKisiYLNGI]q]YJ\Sh]YU/QTG^Sy‰I`CODimODPIP^iKHKLVO~QTHs]Y`¿]DhqQTGISˆ`ISpQkjƒ]YU_l©u dcIU\Hs`IW€iKStO~U\`IHs`IWeQ\GISFZ~ODisHKbCO QTHK]D`¿SVU\U_]YUXwOtf cILpQTcCO QTSƒJ_iKHsWYGaQTisf‘]?ZYSVU.J\S=ZDSVU\ODiRS=P}]qLNGIJ=u  ‰IU\J_Q.wHs`IHKw…cIw#]Dh  wOtfn}Sˆh]YiKis]?jƒS=b»af…O¢J\S=LV]Y`^b‡zabISVS=P}S=UwxHK`IHsw{cIw5w…cILNG{i|O QTSVU HK`ÊQTGISxiKSVODU\`^HK`IW—PIU_]RL=SVJ\J=u— L=]YU\U_SVL=Q…cIJ\S]Dh¼SVODU\i[fJ_Q\]YPIPIHs`IW‹Q\GISVU_S=h]YU_SU_S mcIHKU_SVJnQTGCO~Q iKSVODU\`^HK`IW…OD`IbZ~ODisHKbCO~Q\HK]Y`3S=U\U\]DU\J¼O~U\S¢jƒO~Q\LNGISVb—]?ZYS=UO…iK]D`IWnQ\HKwxSDFu A/SVODU\`IHs`IW…J_GI]YcIisb%`I]DQ rSnJŠQT]YP^P}S=b‹Q\GIS¿‰CU\JŠQ¢Q\HKwxS‘QTGCO QQTGIS¿Z~ODiKHsbCO~Q\HK]Y`%S=U\U_]YUHK`IL=U\SVODJ\S=JVu ˆO~U\isfJ_Q\]YPIPIHs`IW3bI]RS=J€`I]DQh]YU_wODisisfxU\S=bIcIL=SnQ\GISg`qc^w{}S=U€]Dh;hU_SVSgPCODU\ODwxS=QTS=U\J=uˆEG^S U\SVODJ\]D` HsQnjƒ]YU_lmJ{HKJgQTGCO QtzO~Q»QTGIS}S=WYHK`^`IHK`IW^zXQ\GIS%`ISpQkjƒ]YU_lb^]RS=J{`I]DQnfYSpQÈcIJ\S%ODiKiQTG^S SÐo^HsIHKisHsQkfÈP^U\]?ZqHKbIS=bÎmf3Q\GISdi|O~U\WYS€`mcIw{rSVU¼]~h/PCODU\ODwSpQTS=U\JVu.¼Q¼QTGIS¢}S=WYHK`^`IHK`IW^zqJ_S=ZDSVUTO~i `IS=cIU\]Y`^J…HK`QTG^SGIHsbIbIS=`iKO?fDSVUniKSVODU\`ODPIPIU_]to^HswO QTSVi[fŽQ\GISJ\ODwxSxjFSVHsWYGaQnZDSVLpQT]YU=u º `^isf O~hœQ\SVUƒJ_]YwxS‘QTHswS€QTGISpfJ_P}S=LVHKODiKHsŒVS€QT]{bISpQTS=L=Q€bIH[ƄS=U\SV`aQFhStO~Q\cIU\S=JVu
FfStODU_isfxJ_Q\]YPIPIHs`IW»jFS QTG^SVU\Sph]YU\SnLV]D`mQ\U\]Yi}QTGIS    ¼`mcIw{rSVU€]DhXhU\S=SgPCODUTO~wSpQTSVU_J€cIJ\S=b†afÎQTG^Sn`^S=QkjF]YU\l©u 

Y$

$





*









(*) -)

/10C:

V'DGQRA7






QI;>7G4

8

 

b

0W8#QXD ;



;

03A2

SVWYcIiKODU\HsŒtO~Q\HK]Y`Îaf—PrSV`CO~isQkfÎQTS=U\wxJ€HKJ€PIU\]DCODIi[f%QTGISnwx]YJ_QeJ_fqJŠQTSVwO~Q\HKL¿j¼Otf—QT]xLV]D`mQ\U\]Yi QTG^S SÐo^HsIHKisHsQkf]Dh.Q\GIS…`ISpQkjƒ]DU\l©u€EGIS…HsbIStO3HKJ€QTGISnU\S=PIi|O~LVSgQTGIS»J\HKwxPIisS qcIODbIUTO QTHKLnSVU_U\]YU hcI`ILpQTHs]Y`ÎQTGCO~Q€jFSgGCOtZYSncIJ\S=bŽStO~U\iKHsSVU¼af‹OÈ`ISpj#SVU_U\]YU€L=U\HsQ\SVU_H|O . ÁÚ 6 š 8     
 PrSV`CO~isQkf £  Ú u IE  EGISdSÐoqQTUTO…Q\SVU_w œj€GIHKLNGÎHsJƒJ_P}S=LVHs‰ISVbÎrSViK]?j HKJFHK`aQTS=`IbISVb%Q\]{PrSV`CODisHKŒ=S¢J_]YiKc^Q\HK]Y`^Jˆj€GIHsLNG ODU_SQT]q] SpoRHKIisSDu—EGISSVU\U_]YU»hc^`IL=Q\HK]Y`  Ú u IE ¿bISVPrSV`^bIJ{]Y`ÊQTGISPCODUTO~wSpQTSVU
OD`Ib O~J rS=h]YU_Sn]D`—Q\GIS¿Z~ODiKcIS¿]~h.QTG^SgjFSVHsWYGaQTJ€   u J¢]?jÍLtO~`xjFS¢LNGIODUTODLpQTS=U\HKŒ=S¢J_]YiKcRQTHK]D`IJj€GIHKLNGODU\S€QT]q] SpoRHsIiKS ‹Ëk`hc^`IL=Q\HK]Y`xODPIP^U\]tom• HKwO~Q\HK]Y`—j€HsQ\G  \š Ë jƒS…LV]YcIisb‚WYH[ZYSÈOPrSV`IODisQkf—QT]J\]Yisc^QTHs]Y`IJ¢j€H[QTG‚G^HKWYGŽZ~ODiscISVJ ]DheLVc^U_Z~O~QTc^U\SDu·WY]R]qb LNGI]DHKLVS]DhQTGISP}S=`CODisQkfÊQTS=U\w jƒ]DcIiKb Q\GISVU_S=h]YU_S%}SQTGIS%wxStO~` mcCODbIU\O~QTHsLÈLVcIUŠZ~O~QTcIU_S MÀž  – ÄIž   à Iž  j€GISVU_SxQ\GISÈHK`aQ\SVWYU\ODiU\c^`IJ¿]?ZYSVU¿Q\GIS3PCODU_Q ]DhXQTGISgHs`IPIc^QeJ\PIODLVS¿QTGIO~Q€jƒSnODU_SnHs`aQTSVU_SVJŠQTSVb‹HK`„u Ëk` LtODJ_S%]Dh`^SVcIU\ODiF`^S=QkjF]YU\lmJ{j€HsQ\G GIHsWYGR•kb^HKwxSV`IJ_HK]Y`CO~iHK`IP^c^Q…bIO~QNO^z;QTGISLVcIUŠZDO QTcIU_S wxStODJ_cIU\S€HKJ.`I]DQyJ\]gSVODJ_f{Q\]gGCOD`IbIisSDu
ƒc^QjFSewOtf _cIJŠQˆODJjƒS=iKi^QTODlDSeOdbIH[ƄS=U\S=`mQˆPrSV`CO~isQkf &





 





























 



&

 











H

E E val E learn 10

−2

(1/λ∗)

10

3

(1/λ)

Ú u Ú ^˜ A/SVODU\`IHs`IW‹OD`Ib·Z~ODiKHsbCO~Q\HK]Y`‹SVU_U\]YUgODJgO%hcI`^L=QTHs]Y`·]DhˆQTGISÈU_SVWYc^i|ODU_HKŒtO QTHK]D`ŽPCO • UTO~wSpQTSVU
u QTS=U\w‹uÉP}S=`CODisQkfQTSVU_wj€GIHsLNG†HsJStODJŠf%QT]xcIJ\SgHsJ PrSV`IODisQkCf . ÁÚ  6     £ Ú u Ú EGISnJ\cIw JŠQNODUŠQTJeO~QeŸ . Ú ) . Ú . Ú OD`Ib‹U\c^`IJ¢]?ZDSVU¢ODiKirjƒS=HKWYGaQTJeHK`‹ODisi‡i|OtfDSVU\J=uFEG^S J\c^wwO~QTHs]Y` bI]qSVJ3`I]DQ3Hs`ILViscIbIS%Q\GISÎQTGIU_SVJ\G^]YiKbI@J œj€GIHKLNG7jƒ]DcIiKbLV]YU_U\S=J\Pr]Y`Ib7QT]QTSVU_wJ j€HsQ\G†Hs`IbIHsLVSVJ   Ðu EGIS¼wx]DQTH[ZDO QTHK]D`»rSVGIHs`IBb  Ú u Ú XHsJ;QTGIO~QOdJ_]YiKc^Q\HK]Y`nj€HsQ\G{iK]DQ\J.]Dh©L=cIU_Z~O~Q\cIU\SƒLtOD`È]Y`^isf rSxL=]Y`IJŠQTU\c^L=QTS=b’j€HsQ\GiKODU\WDS…jƒS=HKWYGaQnZ~ODiKcIS=JVu cI`IHsJ\GIHs`IW†iKODU\WDS{jƒS=HKWYGaQTJ¿j€HsiKi.Q\GISVU_S=h]YU_S LV]D`mQ\U\]YiYQ\GISˆS=Æ}SVL=Q\HsZDS SpoRHKIHsiKH[Qkf‘]DhRQTGISy`ISpQkjƒ]DU\l©u º `nQ\GISy]DQTG^SVUMGCOD`Ib„z Q\GIU\S=J\GI]YisbnZ~ODiscISVJ ]Y`Ii[fJ\GIH[hœQgQTGISIH|ODJnOD`^b’J\G^]YcIiKbÊQTGIS=U\Sph]YU\S}S3QTU_StO~Q\SVb’b^HsÆ}SVU\S=`aQTisfDBu AMODU\WYSÈQ\GIU\S=J\GI]Yisb Z~ODiKc^SVJ¿ODU_S»Q\GISVU_S=h]YU_S{`I]DQ‘P}S=`CODiKHsŒVS=b‡udË QdLVOD`·rS…J\G^]Aj€`·Q\GCO~Q‘h]DU‘iKHs`IStO~U¢]YPRQTHKwxHKŒVO~QTHs]Y` PIU_]YIiKS=wJ œQTGCO QHKJ=zYQTGISS=U\U\]DUHKJ mcCODb^UTO~Q\HKLHK`{Q\GISjƒS=HKWYGaQ\J O¿P}S=`CODi[Qkf…QTS=U\w  Ú u Ú .O~`Ib StO~U\isf%J_QT]DPIPIHK`^WO~U\S¿S mcIH[Z~ODiKS=`mQVu E;]wHs`IHKwxHKŒ=S  Ú u IEI%jFSÊwOtf*cIJ_SWYU\ODbIHsSV`aQ‹bISVJ_LVSV`aQVz   . 8 ž Dža  u EGISÈbIS=U\H[ZDO QTHsZDS»]~hQTGIS…‰IU\J_Q‘QTSVU_wHs;`  Ú u E eWYH[ZYS=J‘QTGIS»J_QTOD`IbC ODU_b‚
QT4 S=U\w½QTGCO~Q‘GCODJdrSV
S=4 ` bIHsJ\LVc^J\J\S=b PIU_S=ZqHK]Yc^J\isf HK` QTGIS‹LV]D`mQ\SpoqQ]DhdCO~LNlqP^U\]YPCO~WaO~QTHs]Y`‡u7EGIS‹bISVU_HsZ~O~Q\HsZYS—]DheQTG^S PrSV`CODi[Qkf%QTS=U\w  Ú u Ú ƒHKJ 
uEGIS»`^S=j#isStODU_`IHK`^WÈU\cIisSnHsJ¼QTGIS=U\Sph]YU\S  Ú u aIÁ     4
4 .  JŠQNOD`^bCODU\bRQTSVU_w  8 
 
4 j€GIHsLNG†HsJ¼QTGISg]YisbÎiKStO~U\`IHs`IWÈU\cIisS¿PIiKc^J€O 9     ‘Q\SVU_w†uEGISgPrSV`CO~isQkfQ\SVU_w  Ú u Ú  HKJ¼Q\GISVU_S=h]YU_S»LVODiKisSVbÎU_SVWYcIiKODU\HsŒtO~Q\HK]Y`mfjƒS=HKWYGaQebISVLVOtfYu EGIS¢PrSV`IODisQkfÈQ\SVU\w  Ú u Ú ;h]qLVcIJ_SVJƒ]D`xQ\GIS¢wxStO~` mcCODb^UTO~Q\HKLjƒS=HKWYGaQVu AMSpQƒc^JFbIS=`I]DQ\S QTG^S3Q\]DQNO~i`mcIw{rSVUg]DhˆjƒS=HKWYGaQTJnaf u3 P}S=`CODi[Qkf‚QTS=U\w  Ú u Ú ¢bI]qSVJg`I]DQnbIHsJ_Q\HK`IWYc^HKJ\G rS=QkjFSVSV`*OJ\H[QTcCO QTHK]D`j€GIS=U\S†ODiKi¼jFSVHsWYGaQTJxGCOtZYS‹QTGIS‹JTODwxS‹iK]?jZ~ODiscIS—]Dh Ú    O~`Ib OD`I]~QTGIS=U»]Y`^Sxj€GIS=U\SO‹J_HK`IWYisSÈjƒS=HKWYGaQnGCODJ»O‹Z~ODiscIS3]Dh Ú OD`Ib’ODiKi.Q\GISx]DQTGIS=U\J»ODU\SxŒVS=U\]Iu Ëk`hÇO~L=Qtz;HK`QTGISxi|O Q\QTS=UgLtODJ_SODisijƒS=HKWYGaQ\J»j€GIHsLNG ODU_SŒ=SVU\]†LV]Yc^iKbÊ}SxU\S=w]?ZYS=b’j€HsQ\GI]Yc^Q LNGCOD`^WYHK`IW¿Q\GISU\S=J\cIi[QtuË QyHsJ.QTGIS=U\Sph]YU\SeODU_WYcISVbÈQ\GCO~QQTG^S`qc^w{}S=U]Dh}PCODUTO~wSpQTSVU_JcIJ_SVbaf 





















 L 




Y




#

*

 











 




 








 






















QTG^S¢`ISpQkjƒ]DU\l3HK`ÈQTG^S¢J_SVLV]D`IbLVODJ\SeHKJyJ\wODisiKSVU;QTGCOD`xQTGIS€]Y`^S¢Hs`3Q\GIS€‰CU\JŠQFLVODJ\S~uEGISVU_S=h]YU_S QTG^S¿P}S=`CODisQkf3h]YU¼O…GI]Ywx]YWYS=`ISV]Yc^JƒjƒS=HKWYGaQ€bIHsJ_Q\U\HK^c^QTHs]Y`J_GI]YcIisb%}S‘i|ODU_WYSVUˆQTGCOD`%Q\GISd]Y`^S h]YU€O3bIHKJŠQTU_HKIc^Q\HK]Y`j€HsQTG‹J\SpZYS=UTODiMŒVS=U\]~•¡jƒS=HKWYGaQ\JVuy QTSVU_wj€GIHsLNG†bI]qSVJQ\GIHKJHsJ PrSV`CODi[QkCf . 6  ¨         Ú u  ODJeLtOD`‹rS»LNGIS=LNlDS=b·mf—J_]YwxSnSÐo^ODwP^iKSVJ=uƒEGISnPCODU\ODwxS=QTS=Ue¨dHKJeOÈP}]YJ_HsQ\HsZYSgL=]Y`IJ_QTOD`aQtuF PrSV`CODi[QkfQ\SVU_w ]DhƒQTGISh]DU\w  Ú u gHKJ»LtODisiKSVb 9