Dynamic Trees: A Structured Variational Method Giving Efficient ...


   !#"$% & '()+*,-'.!0/1 32 4576987 :( ;$'(



C












=9



,



,



9

D

]Zhg܁gl]Þ `…aRŸ_S hg\ l]r-ja_EHftz Z]`NC \ e_Wg^g€sl]SU` e_LN\wbyY[SajZ]ll]\ ajgOth_^_E Z]Z]\ -j\rsajSvW\wGNYxG \ \wWgY…€&fjZ]Y ^_gbyZx`…ajZ]n \hPn Wgatfc€£lmfcY4fcrs¥ n n` Z]Z]`4glm\wYYacZ]bPajZ]€jg` `:Z]gfc`4h_lN¢ { \ÂjWP¥ Y[ZmƒfcajWUlsZ]\wft`NZ]fj`NzmeDFit`4fcivn \w^ge_`%`4WUacZ]bp\wñ´fcn…ìò ¥ Wga_e_`j{q\ Wg\ Z]\wfcn \wYx`DZ]g;` 4Hìò Z]a Z]g` Ã_¥ l]ajhPfc€UftZ]`:Z]g` 4itfcn ^g`NY^ghZ]g`%Wg` Z[£ ajl]¤^PYx\ Wg€AÊ UË3¥ ÇgfcWP¥eo hgfcl]nwz aj^ghPnwfcft€UZ]ft`µZ]Z]`gZ]`=g`z ajrsWPe_`Nfc\ Z]WP\ ajYWPe_fcatn £…WZ]e_g\wY[`…Z]e_l]\ }vgWP^_fcZ]rs\ ajWP\wzY…Z]^Pl]`4Yx`\ Wg^P€©Yx\ Ê Wgc€ Ë Ê UË3¥ fc‚ n¥ l]`N:fje_h_}.Z]\ rsz4fc\wnwYxz `.^gnwZ]ftgZ]` `NeM¥  ^PYx\ Wg€+Ê[ÂjÂRËfcWPežZ]gB` 4¶Ï Y(fcWPeªç Ï Y €jijg`4`4¥ WPl]
…€jz `4`j`4WPhH¥…z `Noq`ftajZ\wYWv^gijl]WU`N`4Z]e_l]\ €j^Pnp`4z YxWP`N^gz es\ `&Zmftfc\wZqgY…n `N€j`fj^Pzmz fcsajlmWvfcY[ijZ]WU`4`4Z]l]h.`4€j`Nfc`4eDWPWPHz es`&`N\wz4Y\ fcWH^PZ]ajYxg^g`µ`WPZ]e_g‡%`N`=èªes‡%He_èª\`4ijn at`4e_lx£ \ ¢¢ vhH} aj\ WUô4Zµ`4l]acaPb¥Z]¦g`sg`.‡%hHèaje_\ WU\ ijZ=`4acl]b€j`4z WPajz Wv`jij{H`4Hl]€j`N`4z4WPfc^Pz `sYx`s\wYftZ:fAZ]n Pa_ftz4ZµfcnhHZ]aj^g\ WUl]WgZ%\ Wgfcn€ n Z]g`%l]`4n `4itfcWUZ…hPfclxZ]\wfcn¶e_`4l]\ itftZ]\ ij`NYfcl]`µô4`4l]aP¥   %ض›P¾xÕØ º š_œ (´»qÕ :œ›P¾x™U»  º  ¾Xš [»» Ô —%Ø x¾[ Ø :›g» : œ œpšg¾[» º V°n \ZrsP\ fjZNYp{PZ]fcgn l]\w`NYfjite_fc}&l]\wHft`4Z]`4\ ajWWPrsfcn³`4fcWUhgZ]\ hgajl]WgaU`Nfje%zmZ]PHft`NZpz \ ajWrsZ]g`NY`HYx\ `4Wgn €j\ ` n b`qhghPl]fcajl]hP`4WUftZ¢ €Uhgftl]ajZ]\rsajW¶\wYx{R\ WgfcWP€Peµ{tHg`N`4z4WPfcz ^P`Yx`€j\ \ ijZq`NYxYH^gZ]€jg€j` `Nz Y[ajZmY¶l]l]Z]`NPz3ftZMZ` \ŸgWsfjYxz3\Z³Z]^Pl]`NftYxZ]^g\ ajn ZNWP¥³Y¦£…gg\wY¶`4l]\w`Y Z]\ rg`4fcl]€j``\wY…Ê«fsajWghH`=`Nfce_¤jaj`NrseA\ é¬WPfce_WU\wZ:Y[Z]Y[l]Z]\ l]g^P^_z3Z]Z]\ ^gajlmWHfcË3n¶{_\ Z]WUgZ]`4`4Wl]hgZ]l]g` `%ZmfthgZ]l]\ ajajWPfcacgb\ nZ]\ gZ[}` e_\wY[Z]l]\ g^_Z]\ ajWatij`4l Z]g`µWga_e_`Y[ZmftZ]`NY…Yxgaj^gnweH`fjz4z ^glmftZ]`j¥ VX\ Z=Wz fjajeg^ge_nw\e§Z]\ aj£ W(`4n Z]na%HWg`ac\Z]WP\ WgY[Z]€µl]Z]^Pgz3`…Z]\ ijH`(`4n \Z]` a©bMhgz ajl]ajrshPhPfcfc€Ul]ft`=Z]\ Z]ajgW=`n \l]rs`NYx\ ^gZ n z4ZmfjYµYxac`jb { Z]fcg€U\wftYZ]it\ ajfcW¶l]\w¥ÖftZ]\è¶ajavWPajfchvn_}˜rshg` Z]l]gaja_hPesfc€U£…ft\Z]Z]\ (ajW¡Z]gfc`nwYxl]a`NYxP^gfjn ZmY.YZ]acgb´`žn avajY]fchvrs}`§hgl]ajajWgh_` ¢ hPfcl]`4WUZ…n \ rs\ ZN{gg^_Z€j\ ij`NY…e_\ Å´`4l]`4WUZhgl]ajhPfc€UftZ]\ ajW.l]^gn `NY4¥ naj¦av_gajZm\whvfcY}\ W%ithgfcf…l]l]aj\wn atfthP£Z]fc\ aj`4€UlMWPftHZ]fc\ nqajaj^gW¶fcWPhg¥ eµhgƒ l]aj\aUlmW&fjY[Z&zmZ]žgfc`WPPn efjajY%€…byajhgrl]l]`4fcajrsWvP}žaUfcgY[fjZ:\ ne_\\ Z[itZµ}:fcfcWUacn ZmnbUatfcZ]£€jg`NY…` Y%eg^PatftY…ijZmZ]`4fga l ¥ ¦õ±ge_\wY\wY[z4Z]fcl]W\ gH^_`Z]\ aj^PW¶Yx`N¥(eµ®vZ]`Na:z naj`NWPfceAl]W%Z]gZ]g`s` ititfcfcl]l]\w\ftajZ]^P\ ajY¶WPhPfcfcnlmfcfchgrshg` l]Z]aU`4fjlmzmY¶AacbghgZ]l]gac` ¢ ivfcWP\we_e`NYÀ[ajf(\ WUbyZ%^gn hgnPl]À[ajajP\ WUfcZ:g\e_n \\wZ]Y[\Z]`Nl]Y\ gZ]^_aZ]\ Haj`W¶¥z4fc¦nwz g^g\wY:nwftfcZ]n`Nn eMat£¥ Y…ƒ ajz ajlµWP` e_Ÿg\ fcZ]\rsajWPhgfcn ` n Z]Y[gZ]l]`=^Pz3hgZ]l]^gajl]P`(fcgz4fc\ n W©\ Z[}DH`sacbqz4fcz `4nwz lx^gZmfcnwft\ WAZ]`NWgeMa_¥µe_¦`NY…g\ bylmajeMl&{´YxHhH`N`Nz4z fc\ ¨H^Pz=Yx`=zmZ]ggaj`\wz by`NajY…l]racb hHhgfcWPl]aUaRe¡Y]Ÿ_Yx\ \ rsgrn ` `ftZ]Z]gZ]\ ajaªa_WDe¡P£…fRacij\b%n `An¶Z]HgYxaj`%` rs€jfcav`Ahga_hg\ eMWUl]¥aRZ]^gŸ_\\ Z]r\ ajftW¡Z]\ ajfcWKHajfc^_l]ZD`A£…` Ÿ_ghg`4nW+\wz \Z]ZNg{:`§\ Zfch_\wY¢ `4hgè¶ijl]avaj`4ajlsPhvfc}©\ Z.g\hg\wn \Ysl]Z]aj\ ¤vhP`NWgY4fc{&at€U£…fte_Z]W`N\ YxajhgZ]W§až\ Z]P` Hfj`A\YµZmY€jWgavajfca_Wghge+`hgl]acftaRb Z.Ÿ_Z]\ z4rgfc`NnwftYxz Z]`s^g`§nwbyft`NWPZ]ftft\ Z]WgZ]^g€ž^gl]l]`Nr`jY4¥ ¥Dfcl]„€j\atfcWP£l]fc\ ¢¢ n €Z]jftl]Z]av\ \ a_gaje^_WPZ]fcr\ nqajfcW9rsl]€j` \wYZ]\ WPgfFfca_nwegY4€jYs¥av¦a_z4fceKgWF\wY:fcHhg\w`DYhgl]Hl]aR`4`NŸ_nwz4ft\ fcrZ]^P\ ijftYx`4Z]`=\n aj} `4W¶ijhH`4{ avWAZ]ajg\ lsb\wYZ]ftgZ(e_`\whgY[À[Z]l]ajl]a_\\ e_WUg^PZµ^_z Z]e_\\ ajWg\wY[W€ ¢ z ajl]l]`NYxhHajWPegY´Z]a:ajWg`qrs\ Wg\ rfacb_Z]g`‡%è.e_\ ij`4l]€j`4WPz `j¥ `4W_¢ %*





#



,







0



$

"

#"

$

"

$

$

+'

$



`4rlmftfcZ]n n \ } ajW£…Z]\ `4Z]WPFegZ]YqgZ]\wa=Y=Hbyaj`=l]r1Î WPfcacl]b:l]at‡%£ è9`4lNÏte_Z]\ ijP`4fcl]W€j`4Z]WPgz `:`jZ]{³l]^gZ]g`:`hHfcaUhgY[Z]hg`4l]l]aR\ ajŸ_lN\ ¢ { hgg\w^_zmZ¤v\hHWgaU€.Y]Yxaj\ ^_gZ&n }.aj\ Wg€j`Wgajhgl]l]\ ` Wgby€(`4l]acl]Z]`Nge§`4Îlm\ Y4WU¥Z]`4÷¡l]hggl]`4` WZmftZ]Z]g\ aj`µW¶rÏPacfcbl]€jZ]\ gWP`(fcnwegYftfcZml]fg` { atz4Yx^gfcijnnw`4Z]z l&\^gWgZ]nw€ftgZ]\wit`NY%fceajn ^gWgZ]`&g``=by\ajWUaclZ]Z]`4Z]gl]g`4hg`%l&l]` ritZmfcftfcl]Z]l]\wfc\€jajg\ W¶WPn `Nfc{¶Y:n†hH¥ fcaUl]Y]`Yx\ gajWgn }§n }Yx¤j\ WU`4Z]£…`4\ €jWglm€ftZ]Z]`Nge`ajl]^_` Z¢ b«Z]V°fcZgl]\w\w`NY=YY&Z]fAg\ W Yx`4\ l]rsZ]` `4byhgl]ajrn l]`.`:Yµhgfacl]b€jajavgz4a_nfc`4e(nwrz ^gZ]`Nnw\wftY[YZZ]\ Yxacaj` bMW Z=Z]^gacgbqh¶\wYq¥Z]rsgVX`` WžZ]rgaja_fclme_l]e€j`4Z]\l%WPa(Z]fcYxaAnw`4Y4`:¥(rg¦³fcat¤jaA£¡`e_\\ a ZZ Z]Wgatgijaj`q`4WUl Z]hHl]Z]aU\ l]ivY[`4Z]\w`%fc`4l]n†Y[{³\ Z]aj£l]l³^P`sY[z3Z]Z]`4l]^gWP^Pl]Yxz3`N^gZ]Y4^gl]¥ `sl]`Z]£…Pft\ n Z%nUWgZ]gacZp`DPegfRftijZm`ff…\wYhH`Nzmfcg¤jaU`NYxe%`4WžhHaUYxY[aZ]`4Z]l]P\ ftajZl \wd~qYµfje_byzmaj` +^g¨Pl.WgWg`Na_nweMe_fR}j{¶`A`4£…ls\ W˜\ Z]Wg Z]` gZ[`N£`Afjajzm¨Pl]©lm¤´Y[Wg{qZa_£…Z]e_g\ `(Z]l]+`4Zm`Afcby¤vajnw\^gfRWg}jl€D`4WglmajY(a_Wge_`(P`NfjYacYsb\ W˜Z[Z]£g`Nl]až`4fj`szmhH+aUY[ZmY]nwftYxfR\Z]}jg`N`4nY4`l ¥ hPajWgfcl]`©`4WUÊyl]ZmacYDZmft\ WKZ]\ ajZ]WPgfc` n n }gnwfRË}jZ]`4alDZ]fcgH`.atijl]`j\ €j{UajZWgac` b e_Z]\Pl]ft`NZNz3¥Z]n }K¦gfc`.HatWgija_`je_{`.fce_WP\e ¢ l]hg`Nl]z3ajZ]Pn }ªfcgfc\ nH\ Z[at}&ij`.Zmfc\wgY=n `NhgYpl]£` by`4`4l]l]`l]`NzmeKgaUÊ Yx `4Wâ lmfcgäWPUe_Ë3aj¥ rs¦n }jg{R`£…\ z Z]ajsWPf:e_\Y[Z]Z]\ l]ajajWPWgfc€ n   ^ge_Y[Zm\wWgfcfc\ WU€jbyajZ]ajl]\wWPftrAfcZ]`Nn:Ê g{eMÊy¥WgÂRajË=l]r`4n `4fcrsn \wYx`4`UWUÊ«ZmÇYm Ë3¥¡¦gË%`A£…Hgac`4Zxl]Z]` ajr Wg\wY=a_e_r`NftYsZ]£l]\ `4Ÿžl]`£…\\ Z]W_ ¢ HÐÊ[ `…`Nz4fjfcz4%Y^Pz Yx^g]`%fclmrsftZ]Z]ghg`4\wnn Y:}`NYWgz4` £ fcZ[nw`4£ z l]^gaj`…nwl]ft^P¤DZ]Yx`N`N\weY&ePË3Z]Yx¥grl]VXfcajWDn ^gn³fj€jZ]eg=ge_`=\ \ rsZ]` \ ajhHŸgWsfjajz3lxZ]ZmZ:gfchH`:WPaUz n avY[`…Z]ajY]`4hvfcl]}=\rsajhgl:hgl]n z4aj\ fcWgh_W€ ¢ fcz4fc€Unwftz Z]^g\ ajnwftWAZ]`NreMfc¥l]d˜€j\ WPZ[}vfchgnwY\wz4fcfcWPnge` ŸgZ]fcgrs`(hgitn fc`l]ac\wftbHZ]Z]\ gajWP`fcl]n³`NYxr^gfcn Zml]Y€j\w\ YWPfc€jnw\ Y…ij`4£ Ws`4\l]W ` ZmhHfc`4glxbyn `(ajl]Ê[rÂRË3Y¥ H` `4ZxWgZ]`4`4llmfcZ]n Pn }=fcW¯Z]g`&n avY[ajZ]hvl]^P}˜z3Z]hg^gl]lmajfchPnPfcit€Ufcftl]Z]\w\ ftajZ]W¶\ aj¥±WPfc¦nPgrs` ` Z]itgfca_l]\e ¢ ftfcZ]n n \ }ajWPz fcajnl]rsl]`N` z3Z]Zžga_Z]a­e\w£…Y(\ Z]\ Wg\ W¿b«fjz3ajZNWg{q`Kl]`Ne_fj`NYxz aj\ WPrfcfcgnDn }žhgfjnwfjz4z z `+^glmbyftajZ]lF`jÉ.`4ij^P`4Yxl]^_} ¢ hgfžl]ajWvP^gfcr=gH\ n `4\ lDZ[}jac¥ b%©e_fc\ Å´l]€j`4\ l]WP`4fcWUn´ZD‡%l]è ^gWPe_Y©\ ijÊ«`4fcl]WP€je¡`4WPY[z Z]`&l]^Pz4z3fcZ]nw^gz ^gl]`NnwftYmZ]Ës\ aje_WPaYqz atajijW_`4l¢ ¨PZ]`4l]lr1rZ]fcPl]ft€jZ(\ WPZ]fcgnw`Y4{itbyfcajl]ls\wftitZ]fc\ ajl]WP}vfc\ Wgn €žrsz ` ajZ]WPge_a_\ eZ]\ aje_WPavY4`N¥+Y(z4¦fcgnwz `A^grnwftZ]fc`l]€jH\ WP` fcZx¢ n `4‡%‡%lmè§èfc€je_`Ne_\ež\ ijij`4`4atl]l]ij€j`4`4`4WPl WPz z `N‚ `NY©eYqH_£ ` \ `4Å´Z[l]£ `4`&l]`4`4Yx`4WU^gW¡Zsrs`Nl]rsfj^gzm`NWPKe.Y4¥žatfcijhg¦`4hgglql]`DaR`NŸ_fjl]\zm`Nr.Yx^gftWgnZ]Z]a_\ \aje_WgW˜`j€ {gfcfcfRWPijWPe˜`4e.lmfcZ]fRg€jiU``¢ Z]rsl]^g` Z]`.ge_a_\weY[Z]fcl]WP\ geD^_Z]byaj\ ajl…Wªn av£ aj`4hvl]}` hg‚ l]ä aj‚ hPfcfcWP€Ufte Z]\ vajä WDDbl]yaj`NYxl%hHZ]`Ngz3`.Z]\ ijit`4fcn l]}j\w¥ftZ]\ ajWPfcn VXhgWl]a_z e_aj^PWPz z `…n ^Pl]`NYxfj\ ajYxW¬ajWPZ]fcgg`4n W¶` {(rZ]fcgl]\w€jY \ WPitfcfcnwY4l]\w{cftfjZ]Y\ aj£ WP`4fcn n=nPfjhgYl]a_hgz l]`Nate_iv^g\we_l]\ `Wg€&e_Z]avg`N`Y Wgg`Navz a_`NeY]Y]acfcbl]}%fcWvÀ[}.aj\ hPWUZ&fclme_fc\wrsY[Z]` l]Z]\ g`4lm^_Y Z]\ acajbWPõsY:fc¥ WPeHaj^gWPegY…Z]aZ]g`=n \ ¤j`4n \ ¢   %ض›P¾xÕØ º š_œ (´»qÕ :œ›P¾x™U»  º  ¾Xš Õضœ º Ø x¼ÍÕØMš »…¼ VXhgW l]aUajfjlmzme_µ`4£…l©\ Z]Z]%a¯Z]gz aj`qrsrshP`Nfcfcl]W%`ž¨PZ]`4gnwe%`˜rsY[Z]` l]Z]^Pgz3a_Z]eM^g{Rl]`NZ]e¬`NY[ZmitY¶fc£ l]\w`4ftl]Z]`q\ aje_WPajfcWgn(`fc^Ph_Y[¢¢ \l]Wg`4€(hgl]f `NYx`4WUnwfRZ:}jZ]`4glN`({UajZ[Wg£ `&aDe_hH\ rsaUY]`4YxWP\ gYxn\ `(ajWPWgfca_n´e_e_`}vWPY[ZmfcftrsZ]`N\wY&z…Z]vl]}`4`jZ]{vg£…`gz `4ajl]n `aj^g£ lm` Y gnwfjzm¤DfcWPeD£…g\ Z]`j¥ 

D




%



$

#"



"

$

+'

+'

j€rs¦\ gij`N`N`NfcYxYWD`f%¨Pl]H`4`NnwYx` e^gZxZ]n rsZm`4Yl` \ Z]fcWPghge_a_hg\weMz4l]ftaR¥ Z]Ÿ_`\ rZ]PftftZ]\ZqajZ]W=gZ]`:au&Z]¦g`¢XY[hHZ]l]aU^PY[Z]z3`4Z]^gl]\ l]aj`NleZ]rsPfc` WZ]gZ]a_ge `


 
                  !    !    !!   "    # "   "  "  "                    "

35

)''*+, 
      "      !"    !  !     "   -"   ""          "  !  !  "   !       "  "     

srn¦pavfcaj\wegghve_n }.`žn `hgÂjl]Z[É aj£ hPa.oqfcajl]€Ursatft£Z]hP\ YajfcW.l]ac\wbYxajajZ]l…Wgv`=}DacWgb&fs` Z]Z[gu&£ `¦ajl]Z]¢°¤gitl]^gËfc`Al]£…\wftr\ Z]Z]\Afcajl]WPZ]€jgfc\ WPn´aUYxfcrs`%nwY` byZ]ajʓg^gbya_ajWPeMle¥ Z]vg} ` €jhH `4‚`4Wg(zWP`4e_4lm`4fjftWUYxZ]Z `N`NYqe(Wgacajbybpl]\wYxajaj`rÖWg£`%ffje_YqYx\ \rsfjrseg`4hge_WPn`NYx` eM\ aj©¥WP¦fcfcl]nMg¤j\wegYqatftiv`4Zm\wWPf=fcfcW(£ ghg`4n `Nl]l]Y`&a_z ^P\ `NWPYY]e_Z]Y4`4a={vhHfce_`4WPa(WPese_f%`4\ WPWUhge_Z]l]n` ` } ¢¢ neg\ rsftZm\fFWPfcY[l]Z]}l]^PZ]z3`NZ]Y[^gZl]ac`NY4b¥ gatƒ £Kajl£ `4egn ftnMZmZ]fªg`%€j`4rsWga_`4lme_ft`4Z]n³`Ne_e+`Nfcbyl]nwYajr £…\ Z]Z]g`§gn hga_l]zm\ ¤vaj} l ijacb`4l]Z]}&g`=Yx\ rszmg\ ajnwfc\wz l³`%l]`NacYxb^gfsn ZmY¶hPfcfcl]l]``4WUajZ_Zm£fcfj\ WgY…`N€jeM\ ij¥p`4¦Wgv`}hgl]f \ ajl¶fchg^Pl]ajY]YxP\wfcfcgWA\ n e_\ Z[` } ¢ z4fcfRWP}.e©atZ]ijg`4`sl hgZ]gl]aU`YxhHe_\w`NY[z3ZmZ]fc\ ijWP`(z `hPÊyfcgl]aj`4WUl]\ ZNô4¥sajWU¦Zmgfcn`sn }ghHËaUHYx\` Z]Z[\ £ ajWP`4`4fcWnY[Z]Z]gl]`%^Pz3WgZ]a_^ge_l]`` acYx\ bprsZ]hggn`%}sWgPa_fje_e.`NYY[Zmz4fcfcWPWegfcHlm`eYxe_`4`4`4Wiv\wft\ WZ]\ aj¨PWP€jYq^gl]ac`%bMÇgZ]g¥l]¦`4`g`NZ]Yx\ rs` `NYfcZ]^PgY]`&Yx\we_fc\wWPY[Y¢ Zm`4fcWPWPYx^gz `©l]`NeªH` YxZ[\£ €j`4Wg`4\ ¨HW˜z4fcZ]gWU`§ZhgWgl]a_\ e_aj`NlY.hgajl]ajW˜PfcZ]gg`§\ n \ Z[hP}§fcl]ac`4bWUZDz ajnwWgfRWg}j`4`NlNz3Z]¥¬\ Wg¦€gZ]\wa Y Z]acgb¶`z ajWgWg`NWgfcl]`N`Nz3Y[Z]Z\ WgÇ:€µhHby^gaUY]lxYxZ]\gg`4n l `qfthP¨Pfc`4l]nw`4eMWU¥ZmY4¦{tg£…`:\ Z]z sajWPf:e_Yx\ rZ]\ fcajnWPnvfchgngl]hgajl]PajfcPgfc\ gn \ Z[\ n } ¢ \ajZ[WD}DZ]rg`%ftZ]acl]ÅH\wz ¢X`Ne_Y\wfcP€jfjaje WPfcg¥n†¥ e_at£…WDZ]g`=e_\wfc€jajWPfcn¶fcWPeg`4WPz ` g¥  \by¦nwajfcgl³l…``NZ]fj\ arszm%Z]hgPl]n `4ft`Nrsz4Z:fc`4\ nwWWUz ^gZmÁftnwÂ3ftZ]Äp\Z]aj£…\ ajW\ WµZ]ac acbvb Z]à gs\`qÊ°é:rsZ]`4Ë3`Nlm¥ fcftWµƒZ]\ ajaj¨Pl¶WP`4Z]Y…nweµgac` rsbY[Z]` Z]gZ]l]g^P` a_z3Z]e%^gÊy£l]ð `NfjeµËY¶ithPYxfc\ fjr(l]Y]\ Y¢¢ ftegZ]ft\ Z]aj` WPhgfcl]n´a_fcz hg`Nhge_l]^gaUl]fj`jzm¥p¶d{ ‚Wv}=hPYxfjhHY]Yx`4`N`NYe(£ z aj`4l]rs`hPrfcl]fj\we_Yxaj`:WPZ]Y¶g€jl]aj\ ij^g`4€jW=sgZ]`4gl]`&`^gfch_l]` ¢ ajz4fcWgnwn z }^g\nwWPftZ]e_`N\we=z4ft` Z]Ÿg\ ijfj`jz3¥Z]n „}=at\ Ws£ `4ajijWg`4`l hPHfj`NY]z4Y4fc{c^P€jYx\ `:ij`4Z]W=g` Z]g` itfcitn ^gfcn`N^gY`Nz4Y4fc{UWz ajHW_` ¢ ijaj`4^_l]Z¶€jZ]`4aWPHz `:` acYx\ b³€jZ]Wgg\ ¨H`%z4Y[fcZ]WUl]^PZ]n z3}Z]^gb«fjl]Y[`NZ]e.`4l itfcÊ«z l]aj\wftWvZ]ij\ aj`4l]WP€jfc`4n´WPfcz hg`hg£l]aUfjfjY¶zmfj.Y]Yx£ ^grsajl]`N¤_e Y \z b³^gZ]l]gl]`4`WU‡%Zè Y[Z]`4e_hH\ ijË3`4¥ l]€j`4WPz `&zmPfcWg€j`Ne.v}n `NY]Y Z]PfcW g¥ gÂ:ajWDZ]g` aZ]_¦³gega`Y4z {_e_aj£ \wrsY[`Z]hPl]z \ fcajgl]rs^_`Z]hPZ]\ ajfcgWPl]`(`:Y:rsZ]bygaj`N^g`%fcWPW©ite§fc¨Pl]v`4\wftnw}©e§Z]\ `NajfcfjWPWPzmfce§§nHu&byrsl]`4¦ ` `µZ]¢XgY[`4Z]a_Wgl]e `4^Pl]z3fc€jZ]WP\ ^g`NeAl]Y `NZ]He©g` `(rsZ[£ Z]` `4l]Z]`4^g_W ` ¢ e_f\wY[n atZ]l]£ \ g`4l=^_Z]‡%\ ajè¡W¶¥:e_d$\ ij`4n l]at€j£ `4`4WPlz `jit{pfcl]fc\wWPfte Z]\ ajgWP`4fcWPnMz by`Dl]`4f`H`4` WgZx`4Z]l]`4€jl(\ `NrYft\ rsZmzmhg n \ Z]`Na Y Z]g`Z]l]^g`=hHaUY[Z]`4l]\ ajlN¥:¦g`=l]`NYx^gn ZmYacbZ]g\wY&fcl]`\ WA¨P€j^gl]`(Ã_¥ 



-,

,

,



mean field

$% 
 & ' ( 
               "  !     !    !    !  !  "  -    !    "  !     !  " !   "  "

KL divergence differences: mean field v structured 40

30

25

20

15 15

ybƒ aj\ l€j^gZ]gl]`©`u&Ã_É©¦ ¢XoqY[ajZ]l]rs^PhPz3Z]fc^gl]l]\wYx`NajeWfcacWPb:eZ]rsg`A`NfcitWDfcl]¨P\w`4ftnwZ]e\ ajfcWPhgfchgnql]byaUl]fj`4zm`g`4`NWgY `4l]€j} V°£…Z=g\w\wY=zmfcnwZ]YxgaA`\ WPZ[Y[£ Z]a l]^Prsz3Z]` \ ijZ]g`sa_Z]ega©Ysn avhgajl]a_¤ e_ft^PZz `jZ]¥g`ƒ by\ ajlml]Y[rZ\ ZDacbYxgYxajajn ^g^_nwZ]e\ ajHWP`Y Z]Wge_gac\wY[`©Z]Z]`Nl]u&eF\ g¦ ^_Z]P¢XZ]Y[\ftZ]ajZl]W+^PZ]z3gPZ]`Afj^gY l]rs`Ne`N òfcfc Whgâ hg¨Pl]`4aRç nwŸ_e+ò\ rl]`NbyftajYxZ]l^g\ najfcZW¶n nq\w¥YïjV°ãfžb&[{qYxZ]hHg\ Z`N`Az \w\wY(Y[fcZ]n:l]\we_^Pz4`4z3fjWUZ]Yx^gZ]`\wl]z4`Nacfce bn Z]fcghg`hgl]rsaUfj`NfczmsW\w¨PY`4ajnweAh_Z]e_\ rs\wY[\wZ]Yxl]ì `Ní \ geM^_{UZ]g\ ataj£ W¶ì ¥`4ij÷¡`4lNg{c`4£ W`…Z]¨PgWP`=esu&Z]P¦ ft¢XZqY[Z]Z]l]g^P` z3Z]¬^gl]e_`Nea Z]Wg\ Wgaªac€Z:HYxl]`©aj`NrsYxg`4\`µr=€jacggbpn n }`%Z]gZ]e_`g\wfc\wz Y:€jajaje_WPWP`4e_€jfc\ Z]`4n†{\Wgaj`4fcWPlmWPfcfte˜n¶Z]`µe_fc`4byl]ajhH`l]`4rWPZ]ge_¥…`4`4l]VXWPWP` z byY[\ajZ]`Nl]`NY…`fjHe\ ` WPZ[Z]z £gaj`4`4l]}`4hHWajZ]lm`4Z]ftWPgZxe ` ¢ Wga_e_`NY4¥ V°hgZl]a_\we_YD^P\ z WU`NZ]e`4l]v`NY[}.Z]\ Z]Wgg€ª`=Z]e_a+\ Å´Yx`4`4l]`©`4WU£…Z:Prsft` ZDZ]gZ[}va_hHeg`NY4YD¥ ƒacb=\ €jY[^gZ]l]l]`^Pz3ÇZ]^g€jl]\ ij`N`NYDY…fcfcl]W ` ` hgŸgl]aRfcŸ_rs\ rhgn ft`:Z]ac\ Wgb¶€sZ]ge_`&\wY[gZ]l]\ €j\ gg^_`NZ]Y[\Z ajhgWPl]Y ajbyPajfclgH\ nac\ Z[Z]}=Z]Z]l]g`4`%`NYqrs^g`NWPfce_WD`4l¨P`4Z]nwge`µfcfcWPh_e ¢ Z]Z]gl]`4`(`…u&\wY¦ z4¢XfcY[nwZ]z l]^g^Pnwz3ftZ]Z]^g`Nl]e`Ne©byl]ajfcr hghgZ]l]gaU`:fjzme_g\wY[`NZ]Y4l]¥…\ g¦^_gZ]`=\ ajWsg\ at€jijg`4`NlpY[Z&Z]l]hH`4aU`…Y[vZ]`4}(l]\Yxaj` l¢ Z]nÊ«`Ngfcz3WP\wZ]Ye\ Wgzm£€DgajfcZ]\wWUgz Z3`.`(Ëq\wé$hHY&aUbyY[Y[ajZ]Z]l%l]`4^Pl]\ \z3n ajn Z]^Pl^gY[l]Z]`=lmft£…Z]\gaj\wW zm§hgr^gl]fthHŸ_aUat\ Yxrsij`N`4Y&\wlYx`NajZ]Y Wgl]`4n }j`N¥sY4Ê°é:¥ ÷ Ë3¥`skPfRacijZ]`` ’ 3wŠ3‹† 3 •_‹ 3«Œ 2gŠ ¦fcl]g`F`˜z gajrs\ €jghP`NfcY[lmZªfcghgn `jl]aj{µPgfc^_gZ§\ n \ WgZ[}¬acZ Z]l]fc`4n £`NY§fR}_byYajl \we_Z]`4gWU`˜Z]\wz4Z[£fcn†a¯¥ rsƒ ` ajZ]lAga_Z]geg`Y rsWg`4`Najfc^PW˜YpYx¨P}v`4rsnwe¡rsrs` Z]` l]Z]}µgga_l]e˜`NfcZ]¤vg\ `4Wgl]€P`§{N£…fcl]g`A\wzmhg(l]ajz4fcg^Pn `4Yxr`Z]Y.g£…` \gZ]\ 9€jgYx`4hHlpajn WU`4Zmijft`4¢n Wgfchga_hHe_`N`NY:fclmZ]fcaWPz hH`.ajnwacfcbl]\wÎYx5H`ftZ]ZxaZ]`4ajWgWg\ Wg`(€ ajajlµ^_acZNZ]ϳgZ]`4gl%`Y[ZmZ]ftl]`4Z]`j`¥=Y[Z]¦l]g^P\wz3YµZ]^gPl]fj`NY:Y=Z]l]g` ` ¢ Z]nwYxft`4`4Z]l]`4\\WžWgaj€.l\ WžZ]Z]l]aD`4¨P`=€jZ]g^gÊ«Yx`(l]`4`D`acÃtZ]Á g¶Â `4ÄP¥l%byaj¦Y[lqZmgftrs`.Z]`=ajWgl]a_it`fce_e_l]`N\w` YfcZmgfc\ W n\ `NnwYmY:Z]Ë3g¥\ W©`.¦Z]Z]ggg\w\`(Ylme ` gÅ´\ nw€j`NfRgz3}jZ `N`4Y[lµz4ZµfcbyWhHl]ajaUHr Y[` ¢ Z]ijg`4`WUZmZ]Y…ajZ]h©gfc`=l]Z]`l]`4fc`=n npY[e_Z]l]aj^Prsz3\Z]WP^gftl]Z]`N`NY:el]`4vnw}ftZ]f.\ WgYx€\ WgZ]€ja.n `=Z]gz `DnwfjÏY]gY4nw¥…fjzm¦¤´gÏP\wWgYa_hge_l]`N` Y¢ bybyl]ajajlr1Z]g^_`=Z]u&\ n \w¦ Yx\ ¢XWgY[€§Z]l]^PZ]gz3Z]\wY(^gl]nw`NfReA}j`4fclNhg¥ hg¦l]aRgŸ_\w\ Y=r` ftÅ´Z]`N\ ajz3WžZs\wʓY(¨P€jWg^gacl]Zs`(fcÃchgfUhPË3fc{Hl]fc`4WPWUe Z \wfcY l]`&fbyhHajaU^gY]WPYxe\ gvn `&}.l]\`NZNfj¥ YxajW(byajlqb«fjz3Z Z]PftZg\ €jg`4l hHaUY[Z]`4l]\ ajlZ]l]`4`NY 20

25

30

35

40

structured

,

,

,



6

6

5

5

4

4

3

3

2

2

1

1

-&'
& B C-
 D     '
 +E# *F AG#'H #& 

−1 −1

M-    :NE M- &'
  +7NHO    -
  N  (P (Q  D      SRUT :V   EW# E+-  G%&
##  &'YX  *-
# #& & & Q &
 & ' %'F 
#BG# ' ( & &
S   :G#'  & '  N 



J +KHL

«Ê fUË ÊyHË byƒÊyajH\ ^g€jËqWP^gZ]e+l]g``%vÇgrs}ªÉ `NZ]oqfcgajW`Frs¨PÊ«hP`4fUfcnwË(el]\wu&Yxfcajhg¦ Whg¢XY[l]acZ]aRbpl]Ÿ_^PZ]\ grz3Z]`%ft^ggZ]l]\ `N\ aj€je+W¶g¥ `NfcY[hgZhghgl]aRl]ajŸ_P\ rfcgft\Z]n \ \ ajZ[}.W¶{qZ]l]fc`4WP`Ne Y » º  x½:™v¾[» º ™ `4¦z najgn WP\ `Wge_€§rs\ Z]\e_`Najfc}vWPWµWPfcfc¨Pnrs`4e_nw\we%`4zshHrsZ]`4l]WP` `4Z]e_`Ng`4Y4a_WP¥Ae%z }„£atfjY[£ Z]Y´l]`4by^Pijajz3`4^gZ]lWP^geµl]\ Z(`Z]ae_\ WavH`N`qZ]Yg^PWg`©Yxac` e_byZs^g\wY[fcncZ]nbyl]najat\ l³g£Ö^_rsZ]fca_\ ajWvevW} ¢ atY[Z]ijl]`4^Plz3Z]Z]g^g`=l]`NWge¬a_e_itfc`NY4l]\w¥ft¦³Z]\ a.ajWPatfcijn=`4lmfcz hgajhgrsl]aU`&fjZ]zmg¯\wYPn \fjrsY©\ ZmHft`4Z]`4\ WajW¶hg{Pl]Z]ajghH`(aUu&Yx`N¦ eM¢ ¥ bylm®vajft^Pl%Z]zm`N(Z]Y gfcZ]`DgW(`%Wgfc` hgrsZ[hg£ `Nl]fcajaUWDl]fj¤©zm¨P%Wg`4nwa_hgee_l]at`DfjivY…Y[\we_Zmfsft`NZ]YxY³`NhHfY4`N{rsz \wffcajn¶l]rs`z4a_fjYxYxe_aj`j`4hg¥ nqg\w£…Y[Z]g\w\wz4zmftªZ]`N\ e%WPz rsaja_l]hHe_ac`4¢n ¦Y[Z]gl]^P` z3Z]Z]^g`NY[l]Zm`NYAe§e_fcajhgWghg`žl]aU\ fjW9zm§Z]gYx\w^PYz4z hP`4fc`NhHeg`4Y&l©\ WžYx^gz4€jfc€jh_`NZ]Y[^gZDl]\Z]WgP€.ftZAZ]gZ]`sg`ªhHaUu&Y[Z]¦ ` ¢¢ ¨Pl]fc\ €j`4ajnwajleMWP¥e_fc\wnMY[¦Z]WPgl]ft\ `.gZ]^g^_`4l]Z]iv`%\ \waje_acWA`4bpWPatZ]z ijg``4`=lbyz ajZ]ajl%gWP`Z]e_gWg\ \wZ]a_Y=\ aje_z WP`NajfcY…rsnMHhg`N` Yµl]ZxajZ]byP`4l]lajfcrgZ]P\ n fc\HZ[Wac}DZ]Z]ržg`ftZ]Z]grsl]`D\wz `Ne_`NfcY4\W ¢ { Z]Z]fcg\ WPaj`%eFWPfcfcZ]hgnPghgrs`l]` aR\ Z]rsŸ_g\ hga_rel]ftatZ]l]ij\`N`4ajz rsWatij`4fcWU`4WPlmZ(e.Yp\ Z]WZ]gg`:Z]`&g` Z]`©Ÿgl]^gfj‡%z3`µZè¯hHYxaUaje_Y[n \ Z]^_ij`4Z]`4l]\ l]\ajaj€jWlN`4WP¥\ Wz ¦`DZ]ggH\w`&Y…` Z[itYx£ \fcWgl]`4€j\w`4ftn W ` ¢ Z]£…l]g`4`D`4l]n`\ rsf\ ZNe_{qajfcrsWP\ eªWPfcgWU`4ZWPY[z Z]`Dl]^Pbyajz3Z]l=^grslmfcaUnY[\ZsWUZ]l]`4`Nl]fchgnql]\` rZmftfcZ]€j\ aj`DW Yx\ £ Z]^Pajft^gZ]nw\e§ajWPHY4` { ` Ÿ_hH`Nz3Z]`NeM{_Z]g`fchghgl]aRŸ_\ rftZ]\ ajWYxgaj^gnweDH`%€java_eM¥ VX€jWs`NY[fjZµegZ]e_g\ Z]\wY\ ajfcWhgZ]hga:l]aUZ]gfj`…zm©fcH¨PatWPijeg`jY{Rhgfjl]z4`4z n^g\ rslmft\ Z]WP`sfcl]r}%fc` l]Ÿ_€jhH\ WP`4fcl]\nrshg`4l]WUajZmPYfcYxg^g\ n€c\ ¢¢ Z]z4\fc`NnwY4z ¥µ^gnw¦ftZ]g`N`Ne+Yx`=^PrYx\ Wgfcl]€F€j\ n WPavfcajnwhvY}fchghghHl]aj`NhPfcl:fc€UZ]ftaDZ]\ Haj`=W¶{H` \ bµZxZ]Z]`4gl&`§Z]Pl]`NfcYxWA^gnZ]ZmgYaUYxac` b Z]fFfc\ gajZ]n WP`µatfc}¡Z]nge_Z]l]`N`4ajzmrs^gg€jajWgWP\ · Y[Z]^gZ]glm`D`%ftZ]P^P\ fjajYxY(`%W9acfjfcbe_l]itn`§avfcajWUl]hv`4Zmhgfc}.€jl]hg`N`NYxl]Y%`4ajWU£…hPZmfcgft€U\wZ]zmft\ ijZ]\`jajfc¥ÖW¶l]`¥¦¦Wggacg`žZs`NYxitfR`fcitl]fcfj\w\evftn ¢¢¢ it\ WPfcz WUn Zm^Pfce_€j`…`NYqZ]gfc`…l]`hgl]ivat\ Zmivfc\wYxng\ byajajW(lqacZ]bMg`:fµ^Pn atYx£ `:`4aclbMHZ]ajg^g`&WPe_e(}vWPZ]aµfcrsZ]g\w`z…nZ]ajl]€µ`4`jhg{vl]fcajWP_e ¢ itfcfcgl]\ n\w\ftZ[Z]}F\ ajatfcWPijWPfc`4eFn´lfcZ]Z]hggl]hg``4l]`aUhgfjY[l]Z]zmatl]iv^P\wfcz3YxZ]\nwajYx^ga=WFl]`NhgacY4l]¥bqatÀ[ivƒ aj\we_\\ WUWP`NZ.fcYqn nZ]e_}jg\w{Y[`µZ]Z]l]l]g\ ` g`4· l]^_^g`Z]\ \ l]ajfc`NWPl]e `Y4¥+’ Yx3whHŠ3¦`4‹†`Ng3e ` €¨PU •_fc`4nw\‹ eMWP3«Œ Y(¥2 ¦byl]gaj`NrYx`^P€UYxfc\ \WgWP€ Yz Z]gajrs\wY` rsbyl]` ajZ]r­ga_Z]e+g` lmftb«fjZ]gz3Z`4lsZ]PZ]ftPZfcajWWgZ]n }%g`AZ[£ rsa&`NajfclxW ¢ Z]Z]e_g\ `4ij`:l`åD` hg· l]hP^ga_fj\ z itY]`NfcYxY]`Nn Y4`4Y¥³WUfc¦Zl]`gY[Z]`Wg`4rh.`4`Nfc\e_W\ `NWµeZ]gn \Z]rs`:aµrs\ ^gZmh´ft`NZ]fceg\ WsftajZ]W%¨P``4acZ]nwbUge.Z]` g\ Wv`q9iju&ajfcn¦ WPije=¢X`NY[YZ]ç¿l]fc^PWitz3fcZ]\ Z]^gn ^g`4l]lm`N`NftY4e ¢ Ñ Z]fcghg`(hgl]é9aUfjitzmfc§l]\w\wfcY&gZ]n `NPY…ftZ%Y[Z]Z]\ ngn´`.Zmfcfc¤jhg`NhgY…l]aRf=Ÿ_b«\ fjrz3Z]ftajZ]l]\ Wg\wYx€`Ne.e_\wbyY[ajZ]l]l]r\ g¥^_Z]\ ajWžatij`4l B -NQ # E S * SvCFS # LNG 0

Ø mض›gØ º ´Ø³™

I



0

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

−1 −1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1



J K

J lKmZ 

  '
 '  '&'
  +  ** '
&,&-
'  
 &  "!) " #$&%(')('+*, *.-/*10324%576+, -98:% $";6@?A0@6@- 

O



  $&),Œ% f w>/) /*1% ? N  V @ k- N 



 D #   D   R-
#:   %'

:G &#  ' X # + *-
'*  & '    n '-# %' 
X &

+ &'* ' ' +   G@ G  #* '
& Yo3 Z !!CT \  \   No  p 
 & + : ' 3  '
 & Z 
 #  +Ž%' *-&
mG# #BG#nT #* 
&&\  # & N M  

J lKEŠ

J lKEŠ

J

lK

J

+K

 D # 
 BG@ T _Z 
X 
 $% 
 & ' mX # +  &} '
  '
**-
'ƒ-  & F 
#BG# ‘Q  ) ’ '-&
  % $ e-# q #B AN%- &'
:N #$&%0@66&5 *.?g -h%b8“/.xB6h”(•l/.x …`N%*  # : €6@/.xB6@$+, ) ?A5 -–F— 6+, g *.> f Œ% ?86@$6@?A0@6˜% ? k " D Q N " D

  L

P Q  D         **-
' G@ & '_ 

G@ G 

ƒ &-
“ '-#    &'˜X  *BX & Ž Fp BG##  #-
 Q F '
 & 'S(
'7G##   n+- &:   

J

K

 ‹  P t  Q  D      L ‹M-      T :VT +-  G & 
## "Q