SPT: Hierarchical Agent Control: A Framework For Defining Agent Behavior
!#"$%'&(*)+,.-&/)0%123))546% Marc S. Atkin
Gary W. King
David L. Westbrook
University of Massachusetts 140 Governor’s Lane Amherst, MA 01003
University of Massachusetts 140 Governor’s Lane Amherst, MA 01003
University of Massachusetts 140 Governor’s Lane Amherst, MA 01003
[email protected]
[email protected]
[email protected]
7980:=':?@?+A83=!AB@/CEDGF:$H5IKJMLH5I.?+LCEDN?+A80=!I+:%A%I+O0?+:QP;RDGJNS=!TU@ F/:$H3:?@/CEI+L5LC!VW=!IYXZL/?[T.\ :$A$=]XZ^=!H3F_@/H`@F:$H5IBa TYb :$8@BcW=!L/?Bde;RDGJ T.O0\3\ L?.I+T@/A%I+=!LH@/b0TfI.?@/A%I+=!LHgM?+:$T.L/O0?+A$:Kh@H@F:$hi:$H5IBgjT.:$H0T.L/? =!H5I+:$F?@I+=!LHgk@HlK=!Tnm9:$C!CoT.O3=]I+:BlpI+LqA$LH5I.?+L/C'C!=!H3FC'@?+F:*HO0h*b :%?+T LXr@F:$H5I+Tn=!Hsl^H@hi=!At:%HcW=]?+LH0hi:$H5I+T$dnuvIn?+:$C!=!:$TRL/H[I+8?+:$:t83=!:%?.w @x?+A80=!:$T$yz@A%I+=!LHg{T.:$H3T.L?Bgo@/H3lsA$L/H5I+:%|WIBdU7980:@/A%I+=!L/H 83=!:%?@x?+A85^ A%LH5I.?+LC!TRI+83:*@/F/:$H5IBa T}b :%8@BcW='L?BdEu2I}=!T}L?+F@/H0=!~$:BlA%LH5I+:%|WIM80=!:%?@?+A85^i=!T9@}83=!:%?@x?+A85^iLXF/L@C'T$duH@/l3l=!I+=!L/HI+L I+80:$=]?G\0?+=!h@?.^qFL@/C!T$g hiL/TfIG@/A%I+=!LH0Tn@x?+:}T.=!h*O3C]I@H3:$L/O3T.C]^I.?.^=!H3F I+Lt@A80=':%c:n@}T.:%I9L/X=!hi\3C!=!A$=]Ir@/T.T.O0hi\0I+=!LH0T$do7983:%T.:R@/T.T.O0hi\0I+=!LH0T @x?+:Nh@l0::%|\3C!=!A$=]II+8?+LO0F8I+80:9A$L/HWI+:|Io80=!:%?@?+A85^z@Hl\?+Lxc='l: I+80:NA$LH5I+:%|WIjO3H3l0:%?omN83=!A8zI+83:G@/F/:$H5IML/\ :%?@I+:%T$djJT.O3\0\ L/?.I+TR@H5^Yl0L/h@/=!HY='HKmN80='A8[b :$8@Bc5w =!L?+T9AB@/Hqb :N?+:Bl0O0A$:Bl*
[email protected]@C!CT.:%I9LX\?+=!hi=!I+=!c:N:%:%A%I+L/?+T9T.O0A8 @TKN3_@/H3l0+f33dD9II+80='TI+=!hi:[l0:%3H3=!H3F@F:$H5I b :%8@BcW='L?*?+:B 5O0=!?+:%Tz¡=!T.\¢\?+LF/?@hihi=!H3FKT.VW=!C'C!T$£9mM:U@x?+:hiLxcW=!H3F I+LBm9@?l0TGhiL/?+:nF/?@/\083=!AB@C\0?+L/F/?@hihi='H0F*C'@/H0FO3@/F:%T$d
ABSTRACT
1.
INTRODUCTION
N¤ ¥,¦ §¨xª© ¬«¯®t°3±k²$§¨x¯²Be¤N³,°3¦e´ °3¥µ¬¶®t²$¤n·B¸ «tW¨¹°0¨x¹²ºt ²$W°3¢»*W°¼½° °
µ§soW¨ ¾q 0¿À °3k³,¢§kµk¥,²B°µk´Á{°3¦ p²% ±k¨x ²$¼²/¸«Â¹ 3±jW¨x°3²%¢ µ° °3±½ºik¥¼½° ´¥ZÃMW¨xµ²[²Â±o< ¾²$W¨x¨°3¥µ¸Ä«9W¨x¨x°¥µ ¥µÁo§0µ¼z 0¿ µ²E$±jW´Å°3µk´¾v ¨¢zk°3¨x¨¥,W¨»o²$W¨xÀ ¨x°¥µ¢°³$ K°ÃM¼²$t§µk¥,²E¿¥B¥,{¥³¥,²Â¸NÆDz$W°Èºi¥µk}ºiµ¢¥,² ¼°0±k²B§¨xz°³³9¥,²$* ±k±o µµ²É'*Á{°0¦ ¸ G:$F@x?l0C!:$T.TKL/XtI+83:I+83@InAB@/HYb :tA83@?@/AI+:%?+=!~$:BlYbW^p@/F/:$H5I+TRhiLxc=!H0F@/HlY@\0w \0C]^0=!H0FqXZL/?+A$:dtD>o?+:$\0?+:%T.:$H5I+T}@/F/:$H5I+T}@/b0TfI.?@/A%I+C]^5g@/T%+b3C!L/b3T$g mN80=!A8p83@Bc:}@zT.h@/C!CT.:%IGL/X{\085^0T.=!AB@CXZ:B@I+O?+:$T$g =!H3A%C'O3l0=!H3F*h@/T.T$g c:$C!L5A$=]I^5g XZ?+=!A%I+=!L/Hg?@l='O0T$g@I.I@/AVpTfI.?+:$H0F/I+8g@/HlpT.LqLHd &MC!L/b3T A$@/H_:$=]I+80:%?b :KhiLWl0:$C!:BlQ@/T83@xcW=!H3F @sO3H0=]X,L/?+hÈh@T.Tql0:$H0T.=!I2^5g L?rm9:>AB@/HqO3T.:n@t\3@?.I+=!A$C!:%wvT.\0?+=!H3FhiLWl0:$CI+LE?+:$\?+:$T.:$H5IN@?+b0=]I.?@?.^ T.83@/\ :$TmN83=!A8QAB@Hl:%X,L?+h @/H3l ?+:Bl0=!TfI.?+=!b3OI+:qI+80:$=]?ih@/T.TPT.:$: @C!T.LD>\3\ :%Hl0=]|qDEd Sd D¹b0C'L/b='T9@/Hq@/b0TfI.?@/A%IrO0H3=]IB£W=]I9A$L/O3C'lb :n@H@x?+h^5gW@}T.L/C#l=!:%?BgL/? @}\ L/C'=]I+=!AB@C:$H5I+=]I^5d{oc:%?.^b3C!L/bi8@/
[email protected]@/C!C3T.:IMLX\?+=!hi=!I+=!c:N@A%w I+=!L/H3T*=]I*AB@/H\ :%?.XZL/?+hpgj( 'Z) '+*('!G3 g{03 g@/Hl - , .0/W 1 ,d
DGC'CGL/I+80:%?qb :$8@BcW=!L/?+Tq@?+:Yb0O3=!C]IqX#?+LhÈI+83:%T.: \?+=!hi=!I+=!c:%T$djoXZ?+L/h @*b3=!C!C!='@?lpb@C!CT.=!hzw O0C'@I+L/?=!H5I+L @[l0=!cW=!T.=!LHC!:$c:$CNhi='C!=]I@?.^:%H3F@F:$hi:%HWIzT.='hO3C'@I+L? T.=!hi\0C!^Ub5^pA8@/H0F=!H3FiI+80:tO0Hl:%?+C]^0=!H0Fi\385^T.='A%T$dn7980=!T>m9L/O3C'lK=!H0w cLC!c:EA83@/H0F=!H3Fi80Lxmh@/T.TG=!Tn@x:$A%I+:BlKb5^UA%LC!C!='T.=!L/H3T$gmN8@xI>I+80: X#?+=!A%I+=!LHK=!TrX,L/?n@*b0C'L/bphiLxcW='H0FLxc:?GI+:?.?@/=!Hg3@/H3lT.LiLHd D>oHWO0hzw b :?EL/Xrb0C'L/b3TE=!HsI+80:zT.=!h*O3C'@xI+L/?BdD>oYmr@Ttl0:$T.=!FH0:Bl[X#?+Lh I+80: L/O0I+T.:%IMI+Ltb :>@/b0C!:>I+LET.=!h*O3C'@xI+:>C'@?+F/:nHWO3hb :%?+TGPLHiI+83:GL/?l:%?9LX 8WO0Hl?+:$l0TrL/?NI+80LO3T+@HlTSL/X{b0C'L/b3T$d
3.
HAC: HIERARCHICAL AGENT CONTROL
DGH='hi\ L?.I@/H5IU@/T.\ :$A%IpL/X*@/H5^@/F/:$H5IY@?+A83=]I+:$AI+O0?+:[='TU83LBmª=]I h@H@F:$TRI+80:5LBm LX9=!H0XZL/?+h@I+=!L/Hg@/HlUXZO0I+O?+:t\0C'@/H3TNX,L?nI+80:@x?+A80=]I+:$A%w I+O?+:d7980:K@/\0\ :$Hl=!|¢I+L[I+83:p\3@/\ :%?C!=!TfI+TiI+83:LH3C!=!H3:KhiLxcW=!:$T$g mN80=!A8p80=!F83C!=!F85IGV:%^XZ:B@I+O?+:$TNL/X{I+83:n;RDGJ_@/\0\3C!=!AB@I+=!LHd
Follow Attack Move-to-Point
Move Move ...
...
Apply-Force
¤N¥,¦ §¨x76 nÆq¼²B¥, µkG¾v ¨ °¥,¨x°0¨/¼Âj»¼/ µ²%¨ ³¥µ¾v ¨ °0À ²B¥, µ ¥p±°B$´`´ ºiµ98n B$°3¦°0¨xűk°3B$´`§±M¸« ³, º$²r³,W¿ ³0°0¨t°3¦30µ²r5ÃM¼²% ¨»²B*¢¥´k´k³,}³,°W W¨G¼ µÀ B¥$²B< 3¾U ¨x¼ ±{³,:;8} W²A$L/H3TfI.?@/=!H5I+TRL/HYI+80:tl0:$T.=!FH[L/XM@/Aw I+='L/H3T$9d UL/TfIG@/H5^iA$L/HWI.?+L/CT.A80:$hi:>A%LO3C'lb :n=!hi\3C!:$hi:%HWI+:$l ='HQ;RDGJ>dj3L?i:%|0@/hi\0C!:gmM:U83@xc:UO3T.:$lQ;nD>J I+L@?+A83=]I+:$AI+O0?+:d
5. LESSONS LEARNED
`
;nDGJ>aTHE T[A$L?+:¢HAC T.='hO3C'@APPLICATION I+L? @Hl\0C'@/H3H0:%?Ym9:%?+:Ql:$c:$C!L\ :$l
O3T.=!H0F D }WuRJML/hihiLH ¡=!T.\ 4 £{I+80:zF/?@\383=!AB@CMO0T.:%?t='H5I+:%?.X,@/A$:iO0T.:$T ¡=!T.\i:%|WI+:$H3T.=!LH0TT.\ :%A$=]ArI+L p@/A%='H5I+L/T.8UJML/hihiLH*¡=!T.\KP KJ9¡{Sd 7980:nTf^TfI+:$h ?+O3H0TMO3H3l0:%? p@/AB} ZWd|zL/? Wd |i@/Hlz?+:B 5O0=!?+:%Tr@t L?ib :%I.I+:%%? p@/A$=!H5I+LT.8mN=]I+8@xIiC!:B@/TfI U:$F@/b5^WI+:$TiL/X>>D sd p@\Kl0=!T.\0C#@$^TN@/H3lcW=!T.O@C'=!~B@xI+=!LH3Tr?+:B 5O0=!?+:Rl0=!T.\3C'@$^T9mN=!I+8I+83LOw T+@HlT*L/?hi=!C!C'=!L/H3TtL/XGA$LC!L?+Ti@I*@U?+:$T.LC!OI+='L/HL/X>@IC!:B@/Tf2I N | Zd;RDGJT.O0\3\ L/?.I+TG@I+:|INb3@/T.:BlH3:%I2m9L/?+VT.L5AV:%IG=!H5I+:%?.X@A$: I+83@IMAB@/Hb :GO3T.:Blib L/I+8iXZL/?rH0:%ImML/?+V*\0C'@B^i@/HlzI+LEA$L/HWI.?+L/C L/I+80:%? T.=!hO3C'@I+L?+TNI+83@IGT.O3\0\ L/?.IrI+80:RT+@/hi:}=!H5I+:%?.X@A$:d
4.
X
W
X
N%[ /&1"%E)1. j1"B])+ B."B,97980:R=!T.T.O3:}L/X?+:$T.LO?+A$: h@/H3@/F/:$hi:$H5Ir 5O3=!AVWC!^b :%AB@/hi:>\@?@hiLO0HWIM=!HI+83:nl0:$T.=!F/H L/Xq=!H5I+:$C!C!='F/:$H5I¢@A%I+=!LH3T$d¬WLhi: ?+:$T.L/O0?+A$:%T@I+=!hi:gT.Lhi:E@x?+:}A$LH0T.O3hi:Blg0@/Hl T.Lhi:E:$hi:?+F:}LH0C]^p=!HUI+83:E\?+L5A$:$T.TnLXo\ :%?.X,L?+hi=!H3F@/HY@A%w I+=!LHdEuH3=]I+='@/C!C]^YLxc:%?+C!L5LVW=!H3FI+83:t=!hi\ L/?.I@H3A$:L/Xj?+:$T.LO?+A$: h@/H3@/F/:$hi:$H5IMm9@/TN\0?+L/b@/b0C]^LH3:nL/XkI+80:nh@/=!HqC!:$T.T.LH0TNmM: C!:B@?+H0:Bldp=!H0A$:iLO?80=!:%?@?+A85^b0TfI.?@/A%INT.:$H0T.L/?+TGb :$AB@hi: @Kc:%?.^@/Hl =!hi\ L/T.:$TrH3LtA$LH0TfI.?@/=!H5I+TrLHI+83:RJ9 I+='hi:GO3T.:$lqb5^@Hp@/AI+='L/Hd 0O0I+O0?+:i:%H3F=!H0:$TmN=!C!C9b :L\ :%?@xI+=!H3FK=!H@U?+:B@/C]w I+=!hi:gjl0:%A$:$H5I.?@/C]w =!~$:$lq:$HWcW=]?+LH0hi:$H5IBg5@/H3lzmN='C!C H0:$:BlzI+Ltl0:B@CmN=!I+8mN='l0:$C]^ic/@?.^=!H3F I+=!hi:NT.AB@C':%T$gX#?+Lh½hi=!A%?+LT.:$A%LHlTI+Ltl3@$^T$do7980:>A$O?.?+:$H5I+C]^zO3T.:$lg A%:$H5I.?@/C!=!~$:BlK@/AI+='L/HK 5O3:$O0:nmN=!C!Cb :R?+:$\0C#@A$:BlUb5^@*hiL?+:}F:$H0:%?@/C hi:%A83@/H3=!T.hÇI+8@IRT.='hi\0C]^pX,L?.mr@?lTE:$c:$H5I+TnI+LI+83:*@/\3\?+L\?+=#@xI+: @A%I+=!LHgmN83:I+83:%?{=]Io=!To@>C!L5AB@C3@A%I+=!LHt?+O3H3H0=!H3FGLHEI+83:MT+@hi:M\0=':%A$: LXo8@x?lmr@x?+:}L/?>@/Hp@/A%I+=!LH?+O3H3H0=!H3F?+:%hiL/I+:$C]^5d
^U
97 80=!TnSUMMARY \@\ :%?>8@/TG=!H5I.?+LWl0AND O3A%:BlU;RRELATED DGJ¹@/T>@l0L/h@/=!WORK H0wvF:%H3:%?@C@F:$H5I l:$T.=!FHUI+L5L/Cd97980:$T.:E@?+:}I+83:n='T.T.O0:$TNm9:}b :$C!=!:$c:};nD>J_@/l3lW?+:$T.T.:$T mM:$C!Cy D>F/:$H5IjA$LH5I.?+LCg@A%I+=!LHi:%|:$A$OI+=!LHg\3C'@/H0H3=!H3FgW@/H3l*T.:$H3T.=!H0F @?+:E@C!C\@x?.IGL/XkI+80:RT+@/hi:RX#?@/hi:%mML/?+V d G:$T.LO?+A$:$TG@?+:E:%|\3C!=!A$=]I+C]^qhiLWl0:$C!:Bld D>AI+='L/H3TN@x?+:RH3LINhiLH3L/C!=!I+80=!AR:$H5I+=]I+=':%T9I+8@IG@C]mr@$^TN?+O0HqI+L A$L/hi\3C!:%I+=!LHdD>A%I+=!L/H3TiT.:$H3lÅhi:$T.T+@F:$Tq@/b L/O0IiI+80:$=]?TfI@w I+O3T$gA%Lhi\3C!:%I+=!L/HÅP:$=]I+80:%?RT.O0A$A$:$T.TfXZO3CoL/?}O0H3T.O0A$A$:$T.TfXZO3CZSgL/? \0?+L/b3C!:$hiT$dp7983:%^sAB@/H¢b :@Hl LXZI+:$H@x?+:q=!H5I+:%?.?+O0\0I+:Bl[L/? ?+:$T.A83:BlO3C!:Bld ;RDGJ ='Tz@[hiLWlO3C'@?Tf^TfI+:$hp£9T.O0\ :%?+c:$H0=':%H3A$:q:$H@b3C!:$TzO0T b3O0=!C#l?+:%wvO3T+@b3C!:E@/A%I+=!L/HKhiLWl0O3C!:$T$d ¡@xI+:$H5IGFL@C!Tn@C'C!LBm
O3H0XZL/?+:$T.:%:$HK:$c:$H5I+T9I+Lb :}:%|\3C!L=]I+:Bld ;RDGJ =!TL/?+F@H3=!~$:Bl¹@x?+LO0HlÅI@T.VTiI+83@Iq8@Bc:pI+L b :Y@A%w A$L/hi\3C!=!T.83:Bld G:$T.LO?+A$:$T*I+80:$Hb :$A$Lhi:I+83:@/F/:$H5I+T*I+8@xI =!hi\3C!:$hi:$H5IN@A%I+=!LH0T$d
6.
7983:pnnDn0\0C'@/H3H0:%?z=!H5I+:$F?@I+:$T@KHWO3hb :%?*LXnH0:%mÄ@Hl¢LC'l '= l0:B@TiI+L¢l0:B@CGmN=]I+8_A%LH5I+=!HO0LO0T@/HlÅ@lc:%?+T+@?+='@CRl0L/h@/=!H3T=!H ?+:B@C!w I+=!hi:douvI9b3O0=!C#lT9O3\ L/HqI+80:>:%TfI@/b3C!=!T.83:$lH3LI+=!LHqL/X{@EA$L/HWI.?+L/C 83=!:%?@x?+A85^5gO3T.:Bl*=!Hzh@H5^*@F:$H5I@?+A83=]I+:$AI+O0?+:$T@/H3l*83=!:%?@x?+A80='A$@/C I@/T.VUH3:%I2m9L/?+V\3C'@/H0H3:%?+TP:d F0d!g /g 4,SdR7980:='l:B@iL/Xo?+:B@/T.L/H3=!H3F O3T.=!H0F[\?+L5A$:BlO0?@/CMVH0LBmNC':$l0F:8@Tz@/C!T.L[b :$:$HO3T.:$l ='HQ@KHWO3hzw b :%?L/XL/I+80:%?Tf^0TfI+:%hiT$g5=!H3A$C!Ol=!H3FEoR2 YK g0jnWw2¡=]I+: Y g3 W 10. K g5B ,B .0' D gI+80:rl3@xI@n@H@/C]^T.=!ToTf^0TfI+:%h` ' 4 gW@Hl =!HC#@H3F/O@/F/:$TMXZL/?9?+:B@A%I+=!c:GA$LH5I.?+LC3T.O3A8@TM>Dn vg 5 Y 4 @/H3l 30 Y D dY7980: Å@x?+A80=]I+:$A%I+O0?+:q@C'T.L[@xI.I+:$hi\0I+TRI+L h@/H3@/F/:Rh*O0C!I+=!\0C':NI@/T.VWTr=!HUA%Lhi\3C!:%|gWO0H3A$:%?.I@=!HU:$HWcW=]?+LH3hi:%HWI+T$g \3C'@A$=!H3Fi\@x?.I+=!A$O3C'@?n:$hi\383@/T.=!TNLHUI+80:}\0?+L/b3C!:$h L/X{?+:$T.L/C!c=!H0F?+:w T.LO?+A$:$TGA$LBH =!A%I+T Y Dvd D>C]I+80LO3F/8¢h@/H5^ Tf^TfI+:$hiTE?+:B@T.LHQ@/b L/O0IzhO3C]I+=!\3C!:A$LH0A$O0?.?+:%HWI FL@/C!T$gEn>DR½=!TqO3H0=# 5O0:Y@/hiLH0F \@x?.I+='@/CR80=':?@?+A83=!AB@/Cn\3C'@/Hw H3:?+TG=!HUI+83@IG=]IG\3C'@/A$:%T>hO3A8LX{I+83:}b3O?l0:$HL/Xk?+:%T.LC!cW='H0FzI+83:$T.: FL@/C!T*L/HDR0¹lL5:$TRH0L/IR?+:B 5O0=!?+:EI+83:l0:%T.='F/H3:%?ELX9@A%I+=!LH0TiP,I@/T.VWTSrI+L @/H5I+=!A$=!\@xI+:n:$c:%?.^z\ L/T.T.='b0C!:n:$c:%HWI=!H5I+:%?@/A%I+=!L/HdjoC#@H3TjI+8@I?+:B@/AI I+L}O3HX,L?+:$T.:$:$Hi:$c:$H5I+T{AB@/Hib :rV:%\0IjA$LH0A$:$\I+O@/C!C]^T.:$\@x?@I+:NXZ?+Lh I+83L/T.:RI+8@xIG@?+:E=!hi\3C!:$hi:$H5I+=!H3FtC!LH3F/:%?NI+:%?+hÄFL@/C!T$d ¡=!V:Eong;RDGJ_@/C!C!LBmNT>XZL/?NI+83:}T.\ :$A%=!3AB@I+=!L/HpL/Xob0C'L5AVW=!H3Fi@Hl H3L/H0wvb3C!L5AVW='H0F¢A80=!C#lW?+:$H PA83=!C'l¹@/A%I+=!LH0TI+83@I?+O3H¹=!H¹T.:B 5O3:%H3A$: mN=]I+8[I+80:$=]?}\@x?+:$H5I+TRL/?R='HY\3@?@/C!C!:$CZSg{@HlKC!=!V:*C'@I+:?}c:%?+T.=!LH0T}L/X >Dn Z4 g3T.O0A$A$:$T.TG@HliX@/=!C!O0?+:t@x?+:nI.?+:B@I+:$lC!=!V:}@/H5^LI+83:%?Nhi:$Tfw T+@/F/:g>@/H3l_l0L H0L/I=!hi\3C!=!A$=]I+C!^Ål:%I+:%?+hi=!H3:pI+80: LBm¯L/XEA$L/HWI.?+L/C b :%I2m9:$:$Hp@/A%I+=!LH0T$d ;RDGJ`@Hl¢n>DR O0T.:zI+83:iT+@/hi:?+:$\0?+:$T.:%HWI@xI+=!LH X,L?z@A%I+=!LH0T @I@C'C>C!:$c:$C!TL/X}I+80:p83=!:%?@x?+A85^5gN@Hl@C!T.L X,L/?\3C'@H3Tq@/H3lÅT.:$Hw T.L/?+T$dnJML/HWI.?@TfInI+80='TNmN=]I+8KI+83:Eh@ $fL/?+=]I^UL/XjA$O?.?+:$H5In@F:$H5InA$LHw I.?+LC>@?+A83=]I+:$AI+O0?+:$T$gr:/d Fd
00 B 1 1 KD}@HlQnD> 4 grmN80='A8 l0=!TfI+=!H3F/O3=!T.8qb :%ImM:$:$H\0?+L5A$:$l0O0?@CC'LBmrwvC!:$c:$9C .T.VW='C!C!T.*L5? +b :%8@Bc5w =!L/?+T.@/H3lz83=!F80:%?9C!:$c:%CTf^h*b LC!=!Ar?+:B@/T.L/H3=!H3F;d 3R=]:%?+:$H5IMTf^TfI+:$hiT @?+:*L/XZI+:%HsO0T.:BlYI+L=!hi\3C!:$hi:$H5I>:B@/A8sC!:$c:$CrP 00 B 1 1tA$Lhb3=!H3:$T u{Fw }@/HliongXZL/?r:|3@hi\3C!:Sd;RDGJÅlL5:$TMH0L/IA$LH0A$:$\0I+O3@/C!C]^ l0=]:%?+:%HWI+='@xI+:pb :%ImM:$:$Hl0=!T.A%?+:%I+:Y@A%I+=!LH3Tq@HlA$LH5I+=!HWO3LO0Ti\0?+Lw A$:$T.T.:%T$goH3L/?lL5:$T=]IC!='hi=]IEI+83:I+83:iC'@/H0FO@F:qO3T.:Bl[I+LKl0:$T.A?+='b : I+83:%hpdkDGC!I+80LO0F8Em9:9\?+LcW='l0:h@/A%?+L/T{@/H3lEXZO3H0A%I+=!LH3TkI+LGTfI.?+:B@hzw C!=!H3:EI+83:tb :$8@BcW=!L/?nmr?+=]I+=!H3Fq\0?+L5A%:$T.T$g@/C!CkI+83:t\ LBm9:%?nL/XjI+83:t¡=!T.\ \0?+L/F/?@hihi='H0F*C'@/H0FO3@/F:EAB@Hpb :RO0T.:BlU=!Hp@HW^@A%I+=!LHKL/?G\0C#@Hd
X
]
]
U
7983=!Ti?+:$T.:B@x?+A8¹=!TT.O3\0\ L/?.I+:BlQb5^R3>D>>D }0D>
O0Hl0:?A$LHw I.?@/AI HWO3hb :%?+T %w xwfJw Z gp Nxw xw %w N4Z Wgp@Hl xw BXw w N d7983: d djnLxc:?+H3hi:$H5I='Tz@/O0I+80L/?+=!~$:Bl I+L ?+:$\?+LWl0O3A%:r@/H3l*l0=!TfI.?+=!b3OI+:M?+:$\?+='H5I+TkXZL/?jFLxc:%?+H0hi:$H5I@/C5\0O0?+\ LT.:%T H3LImN=]I+83TfI@Hl='H0Fq@/H5^pA$L/\5^?+=!F/8WI>H3LI@I+=!LHY80:%?+:$LHdG7983:tcW=':mNT @/H3l¢A$LH0A$C!O3T.=!LH0TiA$LH5I@/=!H0:Bl¢83:%?+:$=!HÅ@?+:I+83L/T.:UL/XGI+83:@/OI+83L?+T @/H3ltT.80LO0C#l}H0L/I{b :='H5I+:%?+\?+:%I+:BlE@/T{H0:$A$:$T.T+@x?+=!C!^E?+:$\?+:$T.:$H5I+=!H3FNI+83: L/ÊzA$='@/C\ LC!=!A$=!:$T9L?r:$HlL/?+T.:$hi:%HWI+Tj:$=]I+83:?9:%|\0?+:%T.T.:BliL/?r=!hi\3C!=!:Blg L/XUI+83: 3n:%X,:$H0T.:¹Dnlc@H3A$:$l G:$T.:B@x?+A8o?+L $.:%A%I+TDGF:$H0A%^ /DG=!? 3L?+A$J: p@I+:%?+=!:$CkJMLhih@HlL?NI+83: td d3nLc:%?+H3hi:$H5IBd
Acknowledgments
W
]
U
]X
W
U
PU
Dnl0l0=]I+=!LH3@/C @O0I+80L/?+T$y ?+:$H5I ;n:$:%?+=!H0F@ P nH0=!c:%?+T.=]I^ L/X Sg D>H0w p@/T.T+@A8WO0T.:%I.I+T$g :$h@/=!Cy l?+:m ;R@H3H3L/H P nH3=!c:?+T.=!I2^ L/X p@/T.T+@A8WO0T.:%I.I+T$g :$h@/=!Cy SgN@/H3l¢{@/O0CMEdoJML80:$H¹P nH0=!c:%?+T.=]I^sL/X p@/T.T+@A8WO0T.:%I.I+T$g:$h@=!C y Sd
7. ADDITIONAL &AUTHORS
U -$ 49 9
-
U 9 9
U
-
9 9
sdd3DrI+VW=!H@/HlU{d0EdJML/83:$Hd >T.=!H3FzT.=!h*O0C'@I+=!LHK@Hl A%?+=]I+=!AB@/C\ L=!H5I+TrI+Lil0:%3H3:RTfI@xI+:$TN=!HpA$LH5I+=!HWO3L/O3TrT.:B@x?+A8 T.\@A$:$T$d3uH91 B"+.-U N5 E#W444 U %1 " ,(*&])UU $%1. 3"%0g d Jsdd3DrI+VW=!HBg 3*d0¡od0B /X Z&( EK) O - (> & %1 r)/(i%+g\3@/F:%JT Wg Wd Jsdd3DrI+VW=!HBg 3*d0¡od0I+80:RA$L/HWI.?+L/CLX \0?+L/b3C!:$hÄT.L/C'cW=!H3FiTf^TfI+:$hiT$d 1f) 0)"BU # 05 K " %(* ) M) 3- +%[1 $U "$ r0." )? Z+&05 K r]) - U " "+-/&0 Q 9) O- ,F1 0g P Sy I g nLxc:%h*b :%? d Ekd3Ed3JML80:$H(g sd¡od >?+:$:%Hb :?+F0g 3*(d sd0;R@x?.IBg@HlDtd3Nd ;nLBmM:d7?+=#@Cb5^3?+:y >Hl:%?+TfI@/H3l0=!H3FtI+83:}l:$T.=!FH ?+:B 5O3=]?+:$hi:%HWI+TX,L/?n@/F/:$H5I+TN='HUA$Lhi\0C':|:%HcW=]?+LH0hi:$H5I+T$d- ) ) Q %g 0P Sy NZg3@/C!;C Z d@C'T.L7:$A80H3=!AB@C G:$\ L/?.IBgJrnu R 3R:$\IBg nH3=!c:?+T.=!I2^qL/;X p@T.T+@/A8O0T.:%I.I+T$d DzdJMO0?.?+=!:E@/H3lUDtd37k@xI+:d >w2oC'@/Hy7980:}L\ :$H\3C'@/H0H3=!H3F @?+A83=]I+:$AI+O0?+:(d G1%U " ) [ $, Y5 "%g KWy Z g B/d }Ed 3d3k=]?+b5^5d3DGHU=!HWc:$TfI+=!F@xI+=!LHp=!H5I+L*?+:B@A%I+=!c:}\3C'@/H0H3=!H3F*='H A$L/hi\3C!:%|l0L/h@/=!H3T$d0uH r1F B"+.- U N E#0 " # })U # K ) $%1. 3"5 G1%U j" ) [ $, Y5 "%g\3@/F:% T NB g :$@I.I+C!:g3 @/T.80='H0F/I+L/Hg Z Wd ZD}Ed 3d3k=]?+b5^5d37@/T.VqH3:%I2m9L/?+VWTrX,L?GA$LH5I.?+LC!C!=!H3FiA$L/H5I+='HWO0LO3T \0?+L5A%:$T.T.:$T$d u2H 91 B"++K- U E#0 " ." - [ %X1 )U # K ) $%1. 3"5 G1%U j" ) [ $, Y5 " r]) - U " %(*+g \3@/F:%T 0g 0d D}Ed 3d3k=]?+b5^5Id ULWlO3C'@?+=]I2^U=!T.T.O3:%T>=!H?+:$@/A%I+=!c:E\3C'@H3H3=!H0F0d0uH 91 B"++K- U J }#0 ,1f- [ %X1 )U # ) %%1. " K G1U " ) [ $Z 5 " r]) -U N " %(*+g \3@/F/:$T KZ Z 5g Wd Y 45sd30?+:$:Bl(d p@H@/F/=!H3Fih*O0C]I+='\0C!:nI@/T.VWTN=!HpA$Lhi\0C!:%|g l^H3@/hi=!An:$HWcW=]?+LH0hi:$H5I+T$d0u2H 91 B"+.- U J }#0 ; % + #
})U # 3)9 $%1. 3" G1%U " ) [ $Z 5 "%g\3@/F/:$T N ! N 5?g p@l='T.L/Hg0¹u0g ZWd YK 5sd3{dn:$L?+F:%D>uo?+:%T.T$0g U:$H0C'L{@x?+V g J9DEd YD5sd3{dn:$L?+F:%d3;>:$:%?+=!H3F@i@/H3lUkd3Ed3JML80:$Hd3D>HO3Hl:%?+C]^='H0F*hiLWl:$C X,L/?>l:%X,:B@xInhi:%A83@/H3=!T.hiT$dWu2H91B "++-KU J } #0)5 444 2U3 %1 " , (&])U# $ %1. 3 "%g0 Wd Y ]E td0¡:%c=!H0T.LHd3DF/:$H3:%?@C\?+LF?@/hihi=!H3FC#@H3F/O@/F/:}X,L/? O3H3=]3:Bl\0C#@H3H0='H0Fz@HlUA$L/H5I.?+LCd0G 1U j" ) [3 $, 5 3 "%g
0P wF Sy W W ]5g; ]Wd Y D *3 d(U A R3 :%?+hiL/I.IBd07?@H3TfXZL/?+h@I+=!L/H@/C\3C'@/H0H3=!H3F*L/Xk?+L/b L/I b :$8@BcW=!L/?Bd37:$A83H3=!AB@CG:$\ L?.I+M* Dn¡^U Jr3 G , XBg 9* @C': U>H3=!c:%?+T.=]I2^5g n:%m ;R@Bc:%Hg3J97ngIR3 :%Ad? W d Y z d¡odI ^:%?+T$d D \0?+L5A$:BlO0?@CVWH3LBmNC!:BlF:E@/\0\0?+L@/A8UI+L I@/T.V5wvC':%c:$CA$LH5I.?+LCd0uHr1FB "+.-U N J } #0 , 1[ %1X )U# K ) K $ %1f " G 1U " , ) [3 $, 5 " N]) -U N " %(g\@/F/:$T ]4ZI ]5g d YZDt ¡jd\ :$A%I+L?G@/H3l(3 d0;n:$H3l0C!:%?Bd37983:}O3T.:nL/XoT.O3\ :?+c:$H3=!:$H0A$: ='Hpl^H@hi='AwvmML/?+C'l\3C'@/H0H3=!H3Fd3u2H(z d0;R@/hihiL/Hlg5:Bl0=]I+L?Bg r1FB "+.-U N J 0 " +" 3 - [3 %1[3 )U# 3 )9 $ %1. 3 "
G 1%U j" ) [ $, Y5 " r]) - U " %(*+g\3@/F:%T ]KZI W g X0d Y DE td3WIBd3D>h@H5IBd- .- [ - U )UUW r]) - U 1B )" # - Y 1)# 1 ))7 3 ) XZ d3o8-` 3 I+83:%T.='T$g&UnH0=!c:%?+T.=]I^ L/X;p @T.T+@/A8O0T.:%.I.I+T$gDGhi83:%?+TfIBg( WdD>C!T.Li@Bc/@/=!C'@/b0C':t@T I+:$A80H3=!AB@C?+:$\ L?.InJ!p M0JMu2w xw WW d D d¡od 5I+:$:$C!: ?Bd; (*( / , / ) N &)5 %dIR3 =!F/=!I@C o?+:$T.T$g3T.:$A$L/HlU:Bl=]I+='L/Hg d [ *3 drd¹=!C'VW=!H3T$d091f)"BU "+)1N]) -U N5 2 -Q U #0 o ])+X, ")( 3r]) -U N-r)1f)-K (*dIU L/?+F@/HE @/OX,h@H3Hg ZKZ d *3 drd¹=!C'VW=!H3T$g4z d3¡jdI ^5:%?+T$g43 d-3 d¡Lxmr?@H3A$:/g @/H3lU¡odkd :$T.C!:%^5d0oC#@H3H0='H0F@/H3lq?+:B@A%I+=!H3Fi=!HUO0H3A$:%?.I@=!HK@/H3l l^H@hi='A>:$HWcW=!?+L/H3hi:$H5I+T$d$" &W1X )? + 0%1X, (i 3 )k)3 1.$U "+)0 g Px Sy BK Wg ]5d 0
¤N¥,¦ §¨x }«Y®t²B¤ B¥@?+:}I+83:}T+@hi: 80:$=!F85IBdjn\0\ LT.=!H3FGC'@/Hlt@/H3l@=]?oO3H3=]I+T@I.I+:$hi\II+LnA$@/\0I+O?+:M:B@A8 LI+83:%?BaT@/FT$d7983:^AB@/HQ:%|\3C!L=]IL?b :U83=!H3l0:%?+:Bl¢b5^ I+:%?.?@=!H XZ:B@I+O?+:$TMT.O0A8z@/TM80=!C'C!T$g5?+=!c:%?+T$g5@/HltI+:%?.?@/=!HiI^\ :$ToT.O3A8i@TX,L?+:$TfI @HlETfmr@hi\d{7980:9lW^0H3@/hi=!A$TL/X:%H3F@F:$hi:%HWI+Tk@?+:9=!H 3O3:$H0A$:Bl}b5^ hiLWl:$C!TzL/XR\3Tf^A80LC!LF/=!AB@/CNX@A%I+L/?+T ÅX,:B@x?BgNhiL/?@C':/gMX@xI+=!FO3:p@/Hl I+80:}C!='V:d n\0\ LT.=!H3F O3H0=]I+TsAB@H b :ÅA$L/H5I.?+LC!C!:Bl½b5^8O0h@/H0TsL? b5^I+80: n>DR[\0C'@/H3H0:%?Bg5@Hl@nA$LhihiL/HA%LH03FO?@I+=!LHz=!T{X,L?j8O0h@/H0T I+LU\3C'@$^mN=]I+83L/O0I>A$LH0TfI@/H5InT.O0\ :%?+cW=!T.='L/HP:dF0d!gLH0: A$@/H_l=]?+:$A%Iq@AI+='L/H3TN@Bc/@/=!C'@/b3C!:}I+LzO0H3=]I+Tr=!H3A$C!Ol:y ULxc:nI+Li@*C!L5AB@xI+=!LHg L5A%A$O3\5^¢P l:%X,:$H3l S>@C!L5AB@I+=!L/Hg?+:%I@/=!Hs%@ @/F0gb0C'L5AV gX,LC!C!LBm @/Hl @T.T.O3hi:g XZLC!C!LBm@HlKT.O3\0\ L/?.IBg XZL/?.mr@x?l[\3@/T.T+@F:zLXMC!=!H3:gl=!?+:%A%I @xI.I@/AV g3=!Hl=]?+:$A%I>@I.I@AV[P:dF0d!gb5^q@x?.I+='C!C!:%?.^Sd J9@\0I+O?+:nI+83:}kC'@Fz\0?+LxcW='l0:$T9C'@HlU@Hl@/=]?NA$Lhb@xIrO3H0=!I+TrLXI+80: XZLC!C!LBmN='H0FzI^\ :$T$yj7k@H3V g U:$A8@H3=!~$:BlquH0X,@/H5I.?.^5g¡=!F/8WINu2H0X,@/H5I.?.^5g J9@Bc/@/C]?.^5gWDN?.I+='C!C!:%?.^5gW@HliDGc='@xI+='L/Hdo7980:NT.h@/C!C!:$TfI9O0H3=]I+T=!HqJ9@\0w I+O?+:I+80:pkC'@/FH3=]I+T 83@Bc:iC!='hi=]I+:Bl T.:%H3T.L/?+T$gj@Hl C!=!hi=]I+:BlF@hi:n@xIN@/H5^zI+=!hi:>@/HliI+83:%H?+:I+O0?+HiI+L*@E\0?+:%c=!L/O3T.C]^T+@Bc:Bl TfI@I+:/dK3O?.I+83:%?+hiL?+:gI+80:%?+:@?+:iT.:$c:%?@/Chi:$HWOLHiI+83:>T.:$A$L/HlA$C!='AV d 3L?*:%|0@hi\3C!:gk=]XNI+83:i\0C'@B^:%?tA$C!='AVWTL/H¢L\ :$HsI+:%?.?@/=!Hg{I+80:iO3H0=!I mN=!C!C9hiLxc:zI+LK=]IB£=]XNI+83:i\0C'@B^:%?*A%C'=!AVWTL/H@/H¢:$H0:$h^[O3H0=]IBgkI+83: O3H0=]ImN=!C!C @I.I@AV=]IB£W@/H3lz=!XI+80:r\3C'@$^5:%?A$C!='AVWTL/H@RXZ?+=!:$HlC]^zO3H0=]IBg I+83:O3H0=!ImN=!C'CG\0?+LxcW='l0:q=]IimN=!I+8QT.O3\0\ L/?.IBduvXnI+83:q\3C'@$^5:%?z83LC'lT l0LBmNH[I+83:*A%LH5I.?+LCoV:%^KmN80=!C':iA%C'=!AVW='H0F0gkJ9@/\I+O0?+:*I+80:zkC'@/FUmN=!C!C l0=!T.\0C#@$^Y@ihi:$HWOUmN=]I+8[@/C!C{LXI+80:@Bc/@/=!C'@/b0C':*@/A%I+=!L/H3TzP k=!FO?+: Sd 7983=!TA$LHWc:$H5I+=!LH@C!C'LBmNTUXZL/?Ub L/I+8¹T.=!hi\3C!:[@/H3l_T.L\380=!TfI+='A$@I+:Bl \3C'@$^zO0T.=!H3F}I+83:NT+@/hi:GF:%H3:%?@C=!HWI+:?.X@/A%:d¡@/TfI+C]^5g5hO3C]I+=!\3C!:NO3H0=!I+T AB@Htb :oF/='c:$Ht@NA%Lhih@/H3lnT.=!h*O0C!I@H3:$L/O3T.C]^Rb5^nT.:$C!:$A%I+=!H3FNI+83:{3?+TfI O3H0=]I9@Hl*I+83:%Hi83LC'l='H0FEl0LBmNHiI+83:NT.83=]X#I9V:%^tmN83=!C!:GT.:$C!:$A%I+=!H3FEI+83: L/I+80:%?+T$d
A.1.2 Agent Control
7983=!T A$C!=!\ =!C'C!O0TfI.?@I+:$T LO?
\3@?.I+=!A$C!:%wvT.\0?+=!H0FÄhiLWl0:$CYX,L?
H0LHw 3O H0=]X,L/?+h
b3C!Lb0T$do3L/O0?kXZ:B@I+O?+:$TkL/X0I+83:jhiLWl0:$CW@x?+:9l:$hiLH0TfI.?@I+:Bld 7983:t3?+TfIRXZ:B@I+O?+:z=!TRI+83:*@/b3=!C!=]I^KI+LA83@/H3F/:z@b3C!Lba T}T.8@/\ :*bW^ ?+:$A$L/H3H0:$A%I+=!H3FT.\0?+=!H3F/T$di7983:*b3C!Lb0TRTfI@?.IEL/O0IE=!H[I+80:zl:$C]I@qX,L?.w h@I+=!L/Hg3@/HlqI+83:$H $.O3TfIGb5^@/C]I+:%?+=!H3FiI+80:RA$LHF/O0?@I+=!L/HKL/X{I+83:%=!? T.\0?+=!H0FT$g5I+83:^tXZL/?+h½H3:%mÅT.8@\ :$T$d nLI+=!A$:rI+8@IoI+80:rL/?+=!F=!H@C \@x?.w I+=!A$C!:$T{hiLxc:oI+LGI+80:MA$C!LT.:%TfIT.\ LI{=!HtI+80:MH0:%m X,L?+h@I+=!LH*@/H3l}I+80:$H I+83:Gh@/T.T=!T9l0=!TfI.?+=!b3OI+:Bli:$c:$H0C]^i@/hiL/H3Ft@/C!CI+80:>\3@?.I+=!A$C!:$T9=!HiI+83: b3C!L/bd 7983:iT.:%A$LH3l[X,:$@I+O0?+:=!T}I+83:z::$A%I}I+:%?.?@='H?+:$T.LO?+A$:}AB@/HUb : O0T.:BlpI+Lq@/A83=!:$c:tT.:$c:%?@CkI@/T.VWT$dn3L/?n='H0TfI@/H0A$:g=]XjI2m9%L @FTR@x?+: A%C'L/T.:I+L/F:%I+80:%?Bg LH0:EO3H3=]IGhi=!F85IG\0?+LI+:$A%I>b L/I+8d>>?Bg@lc@H3A$=!H3F I+LBm9@?l0TM@/HiL/\3\ LH0:$H5IBa 9T @F}hi=!F85I@/C!T.LtXZL/?+A$:NI+83:NL/\3\ LH0:$H5II+L ?+:I.?+:B@IBg0I+8O0T9?+:$C!=!:$cW='H0FzT.Lhi:}\?+:$T.T.O0?+:}L/HUL/H3:a TGLxmN2H @/F/T$d n>DR[=!To@G\@?.I+='@C080=':?@?+A83=!AB@/C0\3C'@/H0H3:%?Bg/:%|WI+:$H3l0:Bl}I+Ln83@/HlC!: hO3C]I+=!\3C!:nT.=!hO3C]I@/H0:$LO0T9FL@/C!T$drn>DR¢F:$H0:%?@I+:$TN@HU@\3\?+Lx|=]w h@xI+:>\0C'@/Hq 5O3=!AVWC]^5g3C'C!=!H3Ft=!Hql0:%I@='C!Tr@T9=]Ir=!TM:%|:$A%O0I+:Bldo{@?.I+='@/C 80=!:%?@?+A83=!AB@Cr\3C'@/H0H3:%?+TEI^\3=!AB@C!C!^ l0:$A%=#l:b :%ImM:$:$H¢\3C'@H3T*b3@/T.:Bl L/H83:$O?+='TfI+=!ANA%?+=]I+:%?+='@dMn>DRs=!H3TfI+:B@/li\ :%?.X,L?+hiTM@E 5O@C'=]I@xI+='c: T.=!hO3C'@I+=!LHsLHs:B@/A8