The HuGS Platform: A Toolkit for Interactive Optimization

MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com

The HuGS Platform: A Toolkit for Interactive Optimization

Gunnar W. Klau, Neal Lesh, Joe Marks, Michael Mitzenmacher, Guy T. Schafer

TR2002-08

December 2002

Abstract In this paper we develop a generalized approach to visualizing andrapid development of humanguided search systems. ´ AVI02

This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. c Mitsubishi Electric Research Laboratories, Inc., 2002 Copyright 201 Broadway, Cambridge, Massachusetts 02139

MERLCoverPageSide2

Submitted November 2001.

The HuGS Platform: A Toolkit for Interactive Optimization

       "!#%$'&  (  ) +*,- . )-+/0%123 4%5 -+/  #5 $5 6 , *HFJI%ST
87 - 9 :& D A6NJ:/ D" (:(G- > D$: >: a->.O.%QA*O D 4&% ):N8 D A6NJ:/ D-O+ (:1OR9+r4O%$:9 : )A& ?/ A / D\ \ (::$? 6  w$? r9C QV 48/&x?5>pu$% $%(:-4O/&‚4:$%-  8DM%( 6?4d 4 k--O3/0 1C5

  - .?4x4O%(::N:%%*B .E 6 )h\C  +?*B 4‚ QV DcO:Ql-Q 4O:.i E $%R& D$? ‚9CRE ?N%(:?4_5 f ‚€9 ( $E1X  - 6: &QX 4 4O(:DB& $%% G* 4}&$%-.i QX'q D\%% +?4X9+X6 (:%5M!0& :g% B&6 (C:q\Di \C :  (G}& 3 J-Ql9CD" E4'&8$%- . QX>& D>\( $%?4_5 !0& X\O:QV D‚) ([:Lhe-((:([ DRQV D +t - %. L\C  9(: /&  ->cO$%%?4O: )}& 34:.E $%L( :QXE D: # 5[pr? 6& > D

XQX: :QX:g? D: h\9(:%Q 9+a& ?J: )}& L) N (n9C8VQX: :QX:g% &  J- QR9C - k- ((:(:?4w  - %.%5u!0& a%$% 4 6w) (: lQX:  QX:g%& 4:.E $%E ?N%(:?4_¤O9 -[  D(/q %Oq9C0 e-((:("QX L  - %.& tVE 6%(n(:%%5R!0& l Dl :QX $% .E : +V $? D\ $:b‡(::QXE D: %5Å!0& t\O9 (:%Q‰   -  $E& (:(:% ): )O*_& ?/0%?*C9C%$? D- L3: JN(:N>9CD&t- 9( $i : t D 4V-8QX:  QX:g? 6: #5 !0& & %0¹³ ·b¸'¿kà D\ \( :$? 6:  # q:QX\ (:?4: L +&o\O %: Oi e ( 4: )x\9(:%QA*o- : )d& &+ 4J\& 9:$i~&J4J\ &:( :$VQXJ4( : +J4- $?4z9+‡^3:(:(}‘ ŽD¡„5 f z&:aQXJ4O%(Y*>&d \ -Oa:h …'i  - % $%lD QX: h D$% 43%\O%% +: )A \D%: #5X;o $E&u DQX: 

$|4u:( 9C%(:?4w DR%&l&+4O \ &9 :$XDR&+4O \ & (::$5rÆY i 4'R& RQXJ4%(Y*n&a: \-$% - ( 4w QX\(u9C t9 :  6u.: ) %\O%% +: )3/& '& M? $E&V $|4X: l&>  -% $>:o&+ 4J\&  i 9 :$XDR&+4O \ & (::$5 Èz!0& X  -% $X > QX: A $|4ORQR- .L9C \ ( D$%?4y ‡ ‚):N% y)% QX…x& D}&A 6A x6N( \Oi \ : )d QX: d D$% 4R 4ut& DR 46 $% +X QX: t $% 4Ol: u&   -  $%q 6q 46 D$%% +?5"ZM/00$%  4'o&0)QXLD&  /[  ib4:QX% :  D(!0& )N D(/0Dib4:QX% :  (O( D:$%0&0\ 9 (:%Q :"Œ3ni & 6E4{‘ ?¡„5yj-l|E QX/[ 1w? D:(:y D\ \ (::%l‚ D D: }- $E&

3 D )-( DL)  4O8 8QXDl$% QX\ (:ctQXJ4O%(:3 [ +E D$:  9Cb/0%% V 4D. $%% +0 QX: L D$% 4%5M;o% l& >9C%.M1J ?/ X& %-O:.i :$ (:)D:&QX4OX D D\ \C? 6$% \ D9 (:*C: t) % E D(Y*C o(:J: ) \ 9 (:%QX}9 ?4… v  -  $%%l 3 a&O- 4J?4… QX: x $% 4O ‘ ' ¡„5[, c QX\ (:3:>& ?/ h: t€
2. lAPPLICATIONS & :"%$: #*N/[>9O:† R4AND $:9CTERMINOLOGY k-O0 D\ \ (::$? 6  M _-O

f . O.%QA*G/& :$E&x/:(:([X Lc QX\(:%t4O%$:9C}& e- $i :  (:dD[ -8.O.%Q : t& L%QV : 4O3DM&:3\ D\C?5pr

D(:l: +J4- $ D9 .E D$: M& D0 (:(:?/Å L- eDQœ4O%$:\:

Dn& %L D 4Ve--O8 \\ (::$? D: %5 !0&h²n¹³D»E»¿eà 3 \\ (::$? D: h  ) E D\ &(| %+ -\O9 (:%Q ‘ TD¡#

/&:$'&8& n) N D(+ G DE D )o J4O%G D#qQX: :QX:g%$% .E : +%* L& 60$%E D:

9RQR- .89CX\O+$%%?4u u$%E D rQV $E& : %89C'e VD& %5 ,>(  6 %(+*_  %N'}. 9‚QV %h %?4AX- t xNAQV 6i $E& : 5oj-n|E QX/[ 1L$? D }? :(R9C0cJ% 4O?483& D 4(:0-$'&  D 6  %5 ÈujM$%-OR& \9(:%Q :3Œ"i~& DE4d 4d: 4%?4 N% l$% %"/& )    ”I6¥3 Dq'cOQX%(l4O X$%- (n3 (:N : a\E $' $5M€  QXD3  V& 3&:.   ac DQX\ (:3 " -> D\ \ (::$? 6  t:>&?/ A: t€»'³  ¶J·Y¿„³ Ã8& ) :N% r\O9( QA5V,œn %: u (:-: G*BeDc QX\ (:*#: h(:+$? Di

l  4\Oe $%%1J ?/ a9J & 8-%5q,4 4O::  (:(+* k QX. \\ (::$? D: %*C>:\C  9(: }$% D}¿eÃ%Ä?¸¾ »¿ *  ¸[(:-: /& :$'&aJ  ( D8QX8D&a\O9( QA5u€ D}c DQX\ (:*o t^3%(::Nw (:-:

QV %wcO$%%%4…&h4O .E D $%h$% .E : +?*0  ‚n %: ‡ (:-:

QV %V D4O%$:9CL l\O\C>\ 6 € DX? $E&y \ \(: $% D: v/0h& ?Nh4O%:) ?4y ‚XD\C  9(: ´X³DÀ ¸'»E*M RE .e QV 6: l …(:-: %5…,>\ \ (O: )d tQX6N  X(:-: d\OJ4- $%3 X '/” ( -O: #5€ D3c QX\(:*: t&  ‹ 9 &  \#*nPM: )O*o 4‚^3%(::Nx \\ (::$? D: %*n }\C  9(: QX6Nd:hy./0 \ /[‡ 4D D$%% +t \CE D: %*8 J4O%%*3Dt$%-.i QX'%*[%\C%$' %(+5{€ DX& A^3( :N'‡ D\ \( :$? 6: %*D&  O\C%> oQX6N%>: $%(:- 4 44: )V %QX6J )}$%-.QX>kQ & $%-% +q -5
€ $% 4O%*N D 43& "%(:%QX% +#  ‹9& \R D & M\CE 6: %5o;o $E&QXDM:"4' %4 D" (: )QX[-9   "&R%(:%QX J> o\O9( QA5 f t) E D(„*G .E :)&+k i /q 6E4xA4O l& R\ 9 (:%QXL 4x ( -O: 8k L& X D\ \( :$? 6i : %5MZ?/0%N'?*O&8 68 k h%N'E (G\: > ? D ( D9 (:3eD 4' : )}& L o\C:9 (:RQXD%> 4h/&:$'&d%(:%QX% +q& '

(?5on 60D\O:QX g% D:

.O.%Q :$E& +: )&?/”XQXJ4(#& LQX6N%%50,/0R4:$%- 9C%(:?/8*_& %R4%$ : 3$? D t& ?NR X ) $% +3:QX\ D$ t&  9C%& 6J: 3 0 -L: +E $' l\O:QX g% D: x..QA5R]O:QX:(| 6(+* 4 ) : )A a)+J4x 0E D .eDQV D: L r (:-: & L :) : $? +M:QX\ $'" lk- (:(} D-QV 6:$*+& -:.:$q? D$E&X (:)  i & QX D/[%(:(Y5 3. VISUAL CONTROL OF SEARCH ALGORITHMS f t& %$: ‚/0}4O%$:9CX V)% E ("QX%$E& D :Q  & D3 (i

(:?/t- h…J:- D( (Å D DE Dx%(:%QX% +aDX w(:-O  ”:

 E4O8)-  4O} a$QX\ -O3% D$E&x QX\O6N&:3(:-: #5 ;o $E&x%(:%QX J$? x9Cl D:) %4u &:)&#*GQX?4O:- QA*CD8(:6/Š´}³º *¿  ¿e· D5n=>-)& (X\C% 1J: )*+&% D$E&h D( )  & Q :[e+$%-?4V


!0&qQX6N%B: & qn %: l \ \(: $% D: X 60QX [$%QX\( :$? 6?4 & D l& [QXD%n C&q & '[ D\ \( :$? 6: [ 4l D9CN 4L&  $% \C8 n& :\ D\C?5

D(: )}& ) &i~QX9:( bX%(:%QX% +q D 4a:0k 9  44% Vk QŠ (i : )3 +8(:?/qi~QX9 (:L( QX% +%5 f 4' %4a D D(:' )5 pr4%QX .E 60& q$% $%%\O[D_QX9 (:/&V :QX\ (:cJi

DQX\ (:5x]J- \ \C X& V\ 9 (:%Q $% +E : R%% w%(:%QX% +R 4 &t\C:9 (:d(:-O  Xr& V\O9 (:%Q 6d (:(\C :9 (:d i 4O: ) h L&%d%(:%QX% +%5s€ -O& a -QXt& DV&‚ (

D(:( ?/[?4xQX6N8:V./q D\‚b/0 46 D$%% +8%(:%QX% +%5,./q D\xD b/08%(:%QX J%*N _$% -*+:M4 ?4l M (: )L (R&q/[ ( QX% +%5x]O- \\C*B& V$%- J ( -O: …:} e ( (:?/%*[ 4 /[8& ?N :) ?4h%(:%QX% +S(:6/ÅQX 9 :(::bdÆY& ?/ h: ?4È'*O%(i QX% +>¥l
4 lQX%4:- Q QX9 :(:‚ÆY& ?/ a a+%(:(:6/È'* 4& .Dn& 3( QX% +q& ?N3& ) &AQX9:( bxÆY&?/ A: h) % CÈ'¢ ,Å? D$E&d (:)  & QŠ/0 - ( 4h9C8 D(:( ?/[?4h}./q D\d +\ D:>D

4D. $%% +%(:%QX JM (:V< Dq(:? D. & Dq& :)&QX9 (:X 4

%& >& 63(:6/ÉQX9:( b+5"!0& 8? 6$'&d D(:) &Q $- ( 4V&O- 'c\(: 3& \ D$%8 n (:-: & D> 63? $E& D9 (:89+ l:% D"- $E&h./0 \ %* : $%(:-4O: )¢

Œ & 6[: )( QX% +0S3L(:6/zQX9 :(:R%% + (:(4i J  4&\O9( Q : +t/[t%\ 6E D- 9 \O9( QX%*n9C &wD /&:$'&r D}QR- $E&‚QV D(:( '8& ‚& R :):  D(M 5},(:*#/& :(: QX%4:- Q}i~QX9:( bd%(:%QX% +L$? w$E& D )\C: G*n& %%( Di :NLDE4$?  D9CL$E& D )%4G* 5 )O5:*_:::QX\C  9(:LXQX6N ( QX% +  h9CeDX%(:%QX% +R¥J5 f xe $?*#e & :8:QX\ (:}'c DQ}i \(:*N& ' 6 (hI?F\C:9 (:>(:-: 0)  % l& QX 9 :(::b ' )%5np{&-oQX 9 :(:::%%*& q 6  8¥ H 'NH8\C:9 (:(i -O: %5q!0&O-%* &%8QX9 (::%?4-$%8& 8? 6$'&t\ $%L9+a e $DD '+FDHO5[!0& ::QX\ (:3c QX\(:34QX .E 6%0& 34Ê_i  J'Ê_%$ "? D$'&h  4 4%4R&-O8 & 68 D(:? D. 8 /

%:)&J9C > h&  -L 6k& 8QX6N:>\C'e QX%4G5

=>?4-$%: )d& h( QX% +Rx&O%A4OÊ_% +}O\C%} (:(:6/

 6-E D( J:- (::g? 6  V +e $%q& Do\6J 4%"& q-M 3) ? 6 4% (0D$% +(Y5tpra4O%$:9CX/[t9 :$V/q %O& D& VQX i 9 :(::%8 4a& LJ:- D(/:(:([9C 4 X$- (Rd D$'&:%NV>&V O-Ql9CLD>\C:9 (:QXD%L R-|i $  J(‚( D) 5t!0&X- L$? D w :) wQX 9 :(: La& X$%-.i QX'n & 0\O9( Q”k+$%- n& [$%QX\-n% D$E&l l D? D /& :$E&x&d& : 1t& 6l& }QX .L\C % + ("eDLe-&L\Oi QX:g? D: #5 P[  4'&a&O%h%)   V 3$%- . QX}& ?/

: v€4O%$:9C9C( ?/8*J-[$? D V (:QV J- D( (RQXJ4kR&  $%-O% +M (:-: G5 7 J4kO: )8&0$%- Jo ( -O: X$? D }9Cq\ 6i :$%- ( 6(:-e-(N/& % L& o? D$E&l%QX#>9C[$? D- ) &J D /0( (Y5s€ ac QX\ (:*-\ \C d&t- a D:$%%V& Da D { Q}i \C E D +$- .QX':> D A& 8$%-O% +q-O5q;"N% h&-)& & l$%- . QXL:3Y DL|Q & l$%-% +8 -*G& }-8/q D +

/[ ib4:QX% :  D(C)  4V$? 9C( $?4h D 4}& >QX9:( bX < (:(

QX: } $|4O: & D%) : A$? D A9C8?5 4.puoOVERVIEW

?/u4%$:9C"& "OF e-( (NEUSER

D ) [ - ACTIONS 'n $'  D>\%$% QX\ -O?4 (:-: G5 pu 6/z4%$' 9C>? D$'&XO\C B $'  #5o!0&-[$? D VQV i - D(:(:AQXJ4ka& L$%- J> ( -O: ‚9Ja\Ck QX: ) +h "&  \C:9( dQX6N%4O ?4vk V& t$%-O% +a \ \(: $% D: { ‡&  $%-O% +L(:-: G*n- $E&u D8%\C:  )d u \CE D: u: r&  ‹ 9 &  \u D\ \ (::$? 6  #5 f dQV +tD0 -L D\ \ (::$? 6  %*< : ) (: - '[ D$: X }& m f $? l JN 1Nq%N'E ( QX6N%5 f & q^3'i (::Nh \ \(: $% D: G* eD'c DQX\ (:*O& 3->$? t( $>Ql- (:\ (: $%-.QX> 4%QX6N>& %Q D(:(G/&t l: )(:L9 - h\O%%5 '[$? D a D(:R: JN1 8$% QX\ -O[% D$E&Vk  89C'[ ( -Oi : G50!0&8? D$E&d (:)D& Q .E Dk Q &3$%-O% + (:-:

4tcO\ (: %& \ $l M(:-O  3& D3$% ‚9C? D$'&?4d9J

\\ (O: )3QX6N%n/& $E&V D> D( (:?/0%4}):N% }&qQX9 :(:l :) Oi QX% +M q4O%$:9C?4 9C6N5"!0&-0$? D : JN 1N>  D:bX  4Ê_ J% D$E&d (:)D:&QX%5[P[-% +(+*/[8\6J 43.%\C%.i 4$%% +0 D 4l)D%?4J}cO& D- .:Nq? 6$'& D( )  & QXM 4X 3\C\Oi - ( 6l&%-:.:$V (:)D& QA*n$? D(:( %4rE 9 -r? D$E&{‘I6FJ*[I?SD¡„5AK[ & cO& D- .:N (:)D& QX[.0% (:- 63 D( (C(:%) (GQX6N%%*N&%  (:( $% Ql9 : D: 0 E 9-V (i )D& Q \C? D?4O(dQV 1N%>& 9C%.3\C :9 (:}QX6NLkQ &  $%-O% + (:-: G*/& :$E&tQV ?VO:%( 4t l/0DR ( -O: #5q,” 6i bxDQX%$E& :QX8 DX-?4xh\O%N% +8E D9 -x|Q :QXQX?4i

D(:}9 $E1+E $E1J )L 4l8\kq D(:' )L%(:%QX% +o& DM& ?N

D9C%% w D(:'?4r%$%% +(+5Aj-L:  (0'c\C'  $%X& 9C%%

& 6"E 9 -R? D$E&} -\Ck QX"cO& D- .:N0? 6$'&l9 -"o%%QX - 'e- (_}\ODJ 4Ql- (:\ (:3% D$E&t (:)D& QXl& 3-%5 ,>|[&-0& 6>: JN 1N?4X L? D$E&h D(:) &QA*+& l$?

QX  n D 4X 4O\O[ 3'[D  (:-: R& 6L/0V\$%QX\-?4‚9Jd&X? D$E&… (:)D& QX \O: l& 8: +E D$:N3\O:QX g% D: A  #5 5.prXCREATING

?/œ4:$%- L/&A 6NEW  L - APPLICATION ?4‚A\J4O- $%V D w +E D$i

:NV \:QX:g? D: … D\ \( :$? 6: u eDX A /œ\O9( Q -: )t-O Dkb/q D8\( DeDQA*+/&a%QX\& D M X/& 6q:[  - ?4XD#& J:- D(: g% D: d$% QX\C  J?5 !0&[QV ?.D:bL & [$J4[: L& %0 \ \(: $% D: n n& D?4L9+

D(:(+ J& %QA*? 4&J-#$% - ( 48 (:>9C"- ?489+3 q /u D\ \( :$? 6: #5 m :$l$J4L:3- ?4hXQV D: JE D: A& $%-% +/0D1J )a (:-i : #*N&qQX9 :(::%%*O 4R& q& .D+5"!0& 0 (: f \-'£j-O\ -O?* : $%(:-4: ) ?J: ) 4}(:N 4O: )L #\ 9 (:%QXo 4V (:-: %*O 4 (: )): )‚-L9C%& 6J: ?*n DV (:t\Ck QX?4‚9+x) % ' $X$J45 €-& 'QX *J D(:(C-0 \\ (::$? D: 0- & > QX>:QX\ (:%QX% +E 6i : } &q? D$E&X D( )  & QXo D 4&m f ˆ "k o: JN1J: ) 4 QX  : )d&%QA5‚j-Ol:QX\ (:%QX JE 6: uD>&XE 9 -w% D$E&

D(:) &Qœe- $: 09+XQXJ4|: )L&QX 9 :(:V D ) QX% +%5 !0&J-%*n& V:R d 44:  D(09 -OE4% r u& 4O%N%(:\C'lD>

/É \\ (::$? D: t: ADE4ql9CL 9 (:3}-3E 9-h? 6$'&G5 j#$% -*J& 4%%(:\C[ n L '/ \\ (::$? D: aQl-.q4'  /& D V\ 9 (:%Q :L 4A/& D V(:-O  u:eD8& 6L \\ (::$? Di : #5  ;o $E&l\O9 (:%Q : .E $q %?4O/ (:(o9C} D\ \ (::?4d&l$%-O% +8(:-: x& 63& R.O.i %Q QV D: JE D: %5a]O$% 4G*B&}J:- (::g? 6  w$%QX\C % +8QR- . & ?N} u-\_4 DRe- $: ‚/& :$'&G*B/&% u$? D( (:?4_* u D\ \ $E&r:QX:( DLAZ-m8]t/0 L- %4‚A$% DV x: i 'E $:N) E D\ &ib4JE ?/: ).O.%QA*/& :$E&d (:(:6/- >X\C. $ .E D: Jq X&   D((:-: #* [/[%(:(# D[\CeDQœQV Oi - (GQX6N%q D 4Rk%%g8) E \&h%(:%QX Jq: & %$%-O% +\C i : őI ?¡~5uj& 'R%% D$E&v& RcO\ (: ?4w D(  6 hQX&Oi J4OheD‚4:J 4: )w& d/0D1{9C/[%% Å&J- QV D 4{$% QX\ -' : ‚$%+\C'E D:Nl \:QX:g? D: xD84 ) #5 f t& \ $i~&O-O(: $E& %4- (:: )u \ \(: $% D: s‘ D¡„*eDhc DQX\ (:*0&A$%QX\-4Oi $L 4d% (:N%L$% O†:$8: +J4-$%?4d9Jh& l-?ˆ 8 i QX JA a$E& %4- (:5 f ‚: +E $:Nl$% .E : +ib9 ?4x4OE %/qi : )a D\ \( :$? 6: 5 )5:*o‘I?ŽO*nI '* ¸'»¸¾ ¹½ ,9C³?½¿~¸'· D*CS¥OƄTÈ'¢ ŽFJ7I JŽNF O*"I?T Ž 'O5