Highly Configurable Operating Systems for Ultrascale Systems Arthur B. Maccabe and Patrick G. Bridges
Department of Computer Science, MSC01-1130 1 University of New Mexico Albuquerque, NM 87131-0001
Ron Brightwell and Rolf Riesen
Sandia National Laboratories PO Box 5800; MS 1110 Albuquerque, NM 87185-1110
Trammell Hudson
Operating Systems Research, Inc. 1729 Wells Drive NE Albuquerque, NM 87112
rbbrigh,
[email protected] [email protected] maccabe,
[email protected] ABSTRACT
! "#%$&'()$* ' +',- +
. /&0*+.. +-.!/* /1 #2%3 4 $5#670!)3985)!: !+*",; !!? .7 # 1.5 ( 9 4 $* !. 1@
1. n7 ")INTRODUCTION +* >M3oKp!qHW_0!+r85!) j!:s7 !(p6
7n 0#6. 1M,:tu ! +>1/;8_(#%$&' 1> k(0%>4 . ',9$* K0! ! * !4Y$& D %V *.0 +^?>?4 .@ ]_?>?./v,wM m!.0'oKp!qHW"xD%J/;oKpqHW"y;) ! 1 O/z #GoKp!qHW-l\!*/O) ! #M$ X>(,H#68?./z ^ * 1?%/S
0!6$)!D*0!#* ! 5_0=L#7!+* '0=* 1/F _* !> ,wD9).%' -,zI !. 1@ WQ }"0#0=%/S8_0! 2 Z0!0!&^36$& .: .6* S,#0$) S00& SF>"06$)) !+7?.9)%k "0= L" %D,v30!0%* E # $* !. 1%@57 ! 0!0!*,w1 !'* !().0!. 1* U .0=**
, a9bBc d?e*fBghdi%bBchg /0=* ? ,w ! #Q>'#67 jF=:.0=*%'*+* D (M$*6 Q>, 8@1) !+0!0!*0):. 4Y $/! 1.K # *0= +(?>?,C ) #)X>V% ^F= ?.%V63F ) .; ! .S5u0=L#u !,!!;8? +*:.|/0*+.. +3.!|/! 0!0% N@ ]_:?D,S!D00= -D+& %(D,w)%85?j/u # + 58_- M,RQ8?.M # +.,| %UF1> *0= + ?>?.7 (2 /O #(0! 170! ). #6>( M4 F= ? 8_=/# # * (9 * ) !@ 2. MOTIVATION 2.1 Current ! U !%jand ?>?Future ./v,C System m!.0!Demands !$* !Mo7pqHW
!O #7!vt;63p9 !)%/6%$&z85)!>K$> +?M4 $&J #M !! 4Y$!? j/oKpqHWux-%O/5:*!* O!?F %..> $& >90#6=0!1 ) !+2! _3* ! !u,O / %k85T}- 9F=M,K0!1G{Y*~@`WQ k ?%/
2.2 Problems 4Y0! !0=*7*0=with 6) !+9Current ?>?._ !Approaches 5PO m0!6$)!K
85)9 !+*9,H$ @9]_!M $K #( K1)6% * N? - #F"*0!?)%(00%6) -85(4 0F!):,w"$ %* V E 1 4Y X8_ !+!@2]_:+* !4 Q>^ ! !,C >E."3M *?3,50= ,C M,C w 00!)%6* zS !;!H*0= +D?>!? F=% _,! 6$&%!5,S ! !6>$ @ WX ` \ )"6.0!. .% !('0= ,w !E!J 4 */J8H9.0#(!20=,w 2,;7 !¡j #}¢¡7oKp yGF= !!J* 3oKp!qHW#xDK#8?.@Ol^v !_qH +&6%/!v0!! ! % $* * 9,!!_v !? Y/&z!H0=)) 0= !+2?>?j/ #PN !mMH!D+ 4X0! !0=7*0=4 !+^?>?j@\]V&'j.0#6) TM,|'^PO) mk 0=F*/v8_#%$&0=%}qH0!R 1£¤9$* * Z,?./) ! #) !+j)V,5* 1+* !* "..>G)%>&* ! !K1)%(]?* o7pqHW.x-%\!1 j 04 0=G #4Y..>U0!+..) !+Z.!7* ` #!$)! *.0 V !!/:85!)^!^! G0),C.( !00=($*64 X>T,2%U..>k0!+..) !+^ ? /D !T %*7 # 0!@_]_-#-%*M!"!$**0. ,.m%4X.v0!0%* !#6z9F!) !;juWO 370= =j {|_0!%!~:,w >M )55%0#F;,O?>?.v85 )6+*H !9F= J,010= z !!*@oD0!0%* !6$*; F= G!$**0=%,C7!"0),C.-#D m "F=* ! #4 6:,H!* S0!+..) !+.!|@:]_!M)?F! !% .0!) . 1* ",_S> 2 0) !+7) !+* +*H)v H ! m.0%6B@o7$* j0!*+.. +'.! )/O !}K! F#oDzp0#v.!|/H5+&) ! +- 00=N85!)
!K0#6 *.0 +3*.2 X>1@ tv$& Z85 Z' m9,7E0= L#'0!*+.. +(. ! MjvW/S00!)* !2% Z#%$&.85)!$)* !2 ^!
!9F= 9 ZX>0=,K?>?$ 9 >V%I1 j #Z%
:#%$&H$> +-%I1 . 1,CvH $ . 1z 385 ! >' N@_A- m!.0*/3qH*.. (qH.0=* Do5! 4 9?:) (!9$&*0!. 1-,_juW-0!0% /OF K I !"> #.2F!6>}$ KF=M%$)F!)22 !4 $) #?M4Y$&-0=* 16?6%/76V F= *. +3 'm!0= %M0#6_,N!D? #6M0#6O00!)4 (!$**0. H $ . 1%@ ]_!5 +D.,J?.
0! 1"*.M,} +*M,wM'85R. +.,-00%6) @ 70= +-?>?.N,Cv%0#F!)X>4Y) 3?>?.N,C 204 $)!V ?)% !+^.!9{B!%)%`(04X% .6~5 #j !%j0!6$)!"* !>. N $) /O%85) !+ .(*0= 6 +^?>?È0!.%* !@70= +V?>!? . ,CH%0Q>4Y 1%2?>!? ./& M ?%/1+* )>. !00= 2 j.KÉ mF" +2./= G-.#6) !+!/# # 2 ?'0!6$)!G!* D $) ! #! !+ ]5q_H6W?Ê 1 4
! Q8H +. #> #.)D0!1 -%* O@ -o 2.3.0.2 00!)%6* Changing G6$*^$*6Application >) !+T !Demands. ,CE.))6}0= !+ ?>?È$ 0= ! +E* k' !@qH > ?4 . +G $) .% k#%$&})+j.0# M 004 *
0= ,C !*@Go-3G * m!.0*/* !)! 3,w ! J 158-,C:M+ #N.F="!$& % (0!0%
!% !+97 0!-,S9 X8_'0& % Ë WQ..!))$* >^ !) !+G) 1 !0!'{|*@ +!@Vs77WXÌÊ+4
#~_,C5%4B.:00%6) Ë qH& +j,_2 0")+ #)3 #j8T{|*@ +@)/H
m0!)20=*R~2% 9
1> 1U}& !|/: F1>`. . + +* 3# #! + 6$& !%*J@ Ë tSm #! !+"!5& 8500%6) !4Y0=LD # K,w-0= ,C 4Y %N+* Ë A !+5_ 8V0!1J5# #!S%: ! 8})+ #C60#& {|*@ +!@_Í6Î1Ï*Ð&Ñ. js7DWQÌK~
8?j/_++$&>.0 ,w!KF#%9 .*F$&% 70 !@?4 j¦ "%%$1>8_+{|@ +@$* ~K01",CÒ#%$) !+ 0= ,CÇ!"R6 >4Y) j $) 7..* j G*0= + ?>?.@ zA 2.3.0.4 #)>1/- Changing )! '($Environmental X>,"#6%TServices. $ . 1-4 $?.F= ) !+.! $&*0=%J/! #' .0. 16) #%I1 !v# +* !+!? !00=%@ oDu? !?:* !2 m!.0*/#!9PO ?"L#2?>?Ö 6vK 1> F= +M!$& )0=%'M 0):7Avp' j !%"?>?.@;PN !?4 "%I1 K.0= L2. +*4X0 +)>1 D,C*¥"*0=4 !+j?>?Ô{||@ *@/Sz~/S V ?:2+* ! ; 4 8_) !+} !6> } !00=9KAvp^F M
!
0!6$)! 9 !GF= "0= ,w . E%)FQ>1@p).)>1/N!4 0=* 1 +Z $%I1 E . .k! . (*0= + ?>?Å?D 9 X8_MI1 *@SAz) >1/& -$ D,R .0!) . 1%M_ 4Y$&) '+*1Q8H+*1?.×& 3_/&!_*0= +K?>?9 ?z0!6$R_ 8j 1%63 !?%j#6%( $) 700!)* ! -?>?Ê !!@ 3. ]_ZCHALLENGES V,C* }0>T,w G( É# E^+*
,u0= +?>?.5,wK %3*.0! ! +M?>?._04 0%6) } !%/N?>?Ç +*9./N S.! ) {|F="?>?M4Y$&& #: !! 4Y$&& ~/& #K;!4 +* j,S#6% $5&KL#:?>?.@ Programming Model
Shared Services
Usage Model
72.3.0.3 0= +-?Changing >?[? ! Hardware !u% 20Architecture. 1zF z5#68"D m0=_ 6$$&H ,C% "M00!)* ^"j M !E,w%6 !"0!4 .;*0= 6 +-?>!? 0= ,C !*@z]_H FQ>",# 1 0= !+7?>?.ND!-70 1S5+* L% 1S.0=%). 9#8? !00=;,C50#64 F%G0!1 4Y !4Y..>^{BvW(~D?>?.2 B@M70= + ?>?.2,w' !U .9 !?MF=É mF !* !+*U 0=,wÒ% + #E* 1% k }!4 !5 #&D J $&7 K,'F= Q8H j1%z..>' #
Q8_=@;p #>1/00!)%6* MF= >1* M-.) !< !%M3 0!4 0=;-0!+.. !+:.!|@vo- 00!)* !$**0=%M !?4 +E(*.0= 12 !'%>}%I1 ?>?Ö$ M F!)D: 7,v>! .7F!6@ l\!)})+X8_+j*0= 6) !+U?>?.'#%$&EF= `!685 \ 0!0=3!$**0. 3,7%)F00!)* !/;!)904 0!*\0) \ ! F ! TG00%6) `!$*4 *0=%@ )$*
1>E,w%6 !Q>0%)>V1)6% 85Z.
0= !+3?>?.-{|*@ +!@/sD7WQÌ1&~/1 5_6H%?j,v! 00!)%6* 5#6-!2 - !7K,C% @ oD!$ !%\0!+.. !+k.j?$&}U0!6$)!^ +!4 $*HF? * V,_2* "0!6$)!%GF1>(M.0 4 !+E?>?j@jnK F !+E *.0 * !") ^.2,7F? * ! 2 # 30=F!)Q>V V% V%! !MM.* ! 1 ,v J5 !%%'.$&0j 0!0%* O@vl\:+*4Y) $& F? * .J +* !L% 1.F= L!/_00%6) T$&*04 7,C%I1 ! 1> (F1>0#D".0. 6* -,u F? * 2,w9'_0#69,D!' !#M'). %|@HA!,O-!D)
0!0%* .>2F=!System ? Æ +Usage 5.!!Models ! L z!?.O K85:0#O,!u?>?Å{|*@ +@)/M4 0! !< !!~v5 H%J/1F u9 0_ ! H>3F=5 + J 1M09,D!?>?Ö9'.).×_ #^. 4 %T?>?. T85T( !GF= +Z !%TF1> 2 0K ! - !$&:).0 _* j0= ,C 785! ! >': !5 %!%J@
3.3 o5!Architectures !_.! )30! 19#) !+*M9X8_G$&.!
!!5$& 256$& )!?>?.v) $&|@uoD M !$R #!M4
0 . .%>E m!)F!9'85R$*6)Q>},- !u,w%64 ! /1 # +!9 !0!)501/ !00=;,C?./85S!0= #* !7!:0$* >'. 1* !% ,| ; 2850)3 ",?j/&,C3D+*20= ,w _ 4 8_=/ #!2%( $K ! #!20#) zL#9?>? 855* GqH +*5M0! 5 ! -R 1L#%6) N@ 7 !3+&;3($&0Z,C. 8_E,C2F !))) !+j*0= + " ! 1).u?>?.Nv6H3D!H0= L;%I1 !4 . 1v0! MF>9 00!)%6* N/1!5?>!? é !+D.|/ .?>?ã6! !*/S #E.#6%V $@G7 !204 0!*UMEF !))kE * * T, a2bBc de*fBg hdi%b|chg 1 #6v 0!0=S_ !_ ? ! * 9,J7+*1Q8H+*1u*04 6) !+M?>?Ê,wD90=L#:5,v !.? @
4.1 o__:.Micro-Services 2 j/6%'0!0% 85= %9. 4Y $u,w
+) !+'!90)6>(* :. .>1/=01%/*M4 9 ! %6) N/ #'LK?>?j@zl^7% )+* 7$& N.0 4 . 1 ,Cv%2,#!H. 4X$ @u7 ;..>K4 1% . $) -.+v0=,wÅ.0_ * 1+* ;1% N× !! K.+*170(01>%0#+2,R.-6F>(1%64 * !u 3_*+%=S0#1 ; ! ;%>20!6$)! 0!! $&"0#+*:0). 1%@5oê017 +* . 1D4 $9>j.0!> E' !+*M0!1 785 $&:0!1 -6$)F!*/= # 5.+*15 ) !7%*'!%! !) !+!@ ]_"F=_!0= ! !S #2 !*.0F) v85!) 3 . 4Y $@oD( [m.0*/"k*.2 !)* \. 4 $75 .H* 1+* !* 55 01>%>}* 1+* !* {w K! ) !+.,-j..> ! 0!~98H* )T0= k \V..>k1%* \$ #650!6$)!_!)!0=D,u -00!) !+!@v]_! K85) F=j0= ! !2F=Q8_ U. 4Y $) M #Z?>? !+* .! )@zAN m!.0!*/1 * '*0,u9F#:*.2 %6* ' $) *@ Az >1/8HD85J %'çQ+) !èH$ v.)4Y $HV,7s-Z ^' m2,5! Az m"0!6? _%ì6B@;D1*z85F=H %!%K- v0 4 !)5 $è. 4Y $) @ ]_Z0= L#Z+* M$& >. 4Y $^85M! 0= #*
D !_,z700!)* O@uoD ')..!)D!$& >$ 8_ )2.,C>:;* 1F!1:,wSu+*v0!1v-# 2+* #; ! K,C"20!1:: (!M mD).3 0!1 ")"% %G,w3 m !* O@'oæ* +(+ #;!4 $& >j$38_ )().0!> j:$ 1K ,w6*
:&Sz ,w6* 3%$*))F;D !z. 4Y $ #6!? .S 3?>? ,RQ8_( F!?>?./u) ! #) !+}.)!! 8?.@
5.1 *?M?Configurable #!Z*0= Operating +E?>?.2 !Systems k.PN !mV) ! #
.. 15,v * !L+* * 5% jF=: .*! .6$&u F!?>!? .z "!$_$& ,R*!;& |@zäD%8}!1 : "+ >V m G *0= +2?>!? æ,w ! / _D!%! !)?>?j@5WX G*!!* N/: !L#+ !6) E6$)F!2
1>3 F!?>?.v .u< Q8_9?9 #9!_L#5?>?4 ¥7* 4Y+ %E j).%J×J 13 Q8_) !+.? VL#?>?."% ZF='*!!%}3.6$&%O/zF !"! F!4 ?>?.7 !7+* ! >GF=2 *.0=*( #( !L+* !E6: 9 !ZL .+ )6X>1@TWX TPN !mJ/u,wm.0*/uj ]5q_;W?ÅMyH 1U?V%
F='*0! #>Z ) !%Z
D& !|/F !H.5L# ! 4Y+ !%.) ,w* F= ! 85^0!1*:85HF=M !%}% !3%>}F=M %Gj !?4 *.7?>!? * L#+* * O@ D0= +5?>?.J#%$&u)%8_3.vL 4Y+ %D* 4 L#+ !6) N@SqH*.0=* ! 14YF#D*0= +?.O :zp !u%6Y/ 7Az !m:pë:7 B/6qH"B/ #']_ 1>#:p( B/* '7 # #268U& !zKF=?.:(F= %^ Z$&*0! +( 8& @(sD &* ! 0!0=*B/!68H$& %/ !* -,ND?>?.H6$*D 6% * G !?*. +'?>?r,w ! * Q>G6K"L# !2+ )X>
>Ej*9,w !)H*$* +*M,5 8é8_ $* !4
. Oz*0!.v,wz;!J 1 +H., ?>?.@ & !- #ZF!6>Z*0= 6 +V?>?.3 kPOU B/ tSm4Y& !V B/3 #`zFF!U 6B/K,CG m!.0*/768Ò0= 4 6 +E?>!? Ö 1) 2GF= ?*.k6M.0 4B.*/ F=14B)./z 4B).vF1>:# +* !+-!; $* Sz)F>:#6 06$)" $*/N* !+*G:.0=*F!)X>}"$& }. &4X+ 9!-?>?.u F=%.F=6$&@up !9É m4 F!)Q>j+* ! >G*. KK'0*/J%8H$& %×=!20= !+ ?>?.->j6$*" !2.3?>?¥%): #j !0!4Y%D .0!) . 1-M+*$& j$"# (M.* !*:*0= +.?>?4 j/ +-) "!)+ z$* %!@WXS7% 3 !S 95* ,u *?4Y F!?>?r*0!.%6) (*0!0= !) @V !) !+. L 4Y+ %:? ! ! :-0! $&u *?4Y F!?>?[0!4 .%6* 00= H* !+*1*-!+* !%m!0!) >.,C *.0=* !+M?>?,w ! #Q>1@
8í0= +k #U ! .j?>?.. 0Z $&*$ + ! #?r !7,C700!)* 4X$* * K +. 1%@1WQ ú_de%chh bwüøg3e?û7ïCù!h ïCù!ó #" ôþ%a $egbwîa¥e' ü &($!hd?ï|bwüø.ôþgïXha9gDú_dbwü#c b $=)hg /!0#+ **/JqH*00=K 1 xD%/=q # 0 + * % J / p
1 ? 4 j !/=A */J%
*
*ì@ 'DÊ,0!1*J?5 L)D?>?.@SWQ ' ? 9* !D00!*N/! :,S!:?>?. *- jM j 1K*0= 6 +M?>?j@ o3@pF= #@#]_!"uF!FK.0=* ! 14XF% *0= 6) !+M?>?j@WQ ú?\åkp]_x7tuoK(p2B/6;ð4Y& !!B/ * 1 >1/q_o3/!s:po3/=
*
*
@ #Uq?.9.)>1 4 6"ä2@-ä 6: )+/=}@!ä72 !N/ @#PO)&*/p=@=p; *: F= +!/! # F%J/:85ê% *.0=* ! 1G!L# !+U ^012)>1 %@ !@!lV* %@]_:0= ,C K, 4X* 4YF#%?>?.@ p.))6j00!*#%$&(F= \ %,CjF! )! +Z?F! WQ ú? !: * D* : ! +5$& 14YF#%D *.0=** : : ! Q8H
X8_*% !@!WQ ó-dcùbwïXhc ï|îdzô=î $$edïJûed 01/! !;-.-+* ! =0= +3?>?é? ú?Ç ( ! 1)." 4 ?>!? .@ 5 111A8düg&c ï|bBeü!g3eüú-d*w)h N>ü $* !. 1:,w90;*.0! ! +@yH%^ ^9 !2, B bwgï|dbBõ îïXh .ôþgïXha9g /z!{B&~ *61%/S%
*
@ 0.) >(m!0=). /8_9 * ) !3#D2! #D, %6"2@ä7 !) !* Pu@Pu@z N@]_3ð4Y& !|uoD
! 13 },w !M !%)?>?."% "F=*!% 7,wD.0!) . 1) !+2 ! Q8H0!1*@ F1>M+* 4Y0! !0=*:0= !+.?>!? ,v!+*!4Y$& )5,S0= 4 5 1911A8d?üg&c ï|bBeüg3eüGô=eQûDï C;d, h 1j 1K #,C !4 pJ@n:85* O/#x"@!D>1/x3@!yH)./#yD@#j*B/ K#68_K0!R6,w./!0*+.. +3.!/ +*K.4 S @!].) !* O/# #j}@!x-*F= @!WQ.0$ +" )K #(00!)%6* (%I1 . 1@-]' DD0!*F!j/ % )F!)X>,S0#61?F5F1>*!! +30#68_ ! H8 Z0! T+* ,CVU,C. 8_`E E. 4 2 :0= !+M?>?j@WQ ú N@ ! !%!Z,C% / %k.0. 4 WQ 8Nùbw*d ñ bCüî ð v îgïXhd%g 5ügï|bwï|îïXIh ueüû%hdhü#ch /=7 *F= 6* ',7%I1 %k,C% /D U F* !? *
**@
"n9@*F=+* 7 Pu@Pu@z N@#j +M0! m05 :p !5*0= +M?>?j@WX ú_de%chhbwüøg e?ûKïCùKh J&Lü 0/N2ô 14365 7éô!þ,a $egbwîa¥e' ü &($!hdï|bCüø'ô!þgïXha9g B hgb)øü}> ü M5,a $=)hahü!ïYï|bBe0 ü N&_ô B 5PO /!0#+:66#*/ %
*@ @SD ëM@#qH ! O@#o0#6*!+,C- ! !?F * ,S9 !).!)@ uea2a9îüb|cï|bBeüg:e?ûKïCùh ó!#" /&{B&~%6%*
/N*@ %63AH@z |/n2@!ë:F> N/! jpJ@S N@]_7%", . +9 !0= .0 !?j@ %ó 89T!8 U h w ñ &õed?ïYedbYhV g 8Jhcùü!bBc*# Weîdü=* /#*{B&~ */ %
@
2p! #R3K6) #PNF=6@ ó7#ô #56X5h /N
*
*@ 10N *%8 ü &&õ)h3ô!þgïXha2g /N%
*@ 6"x"@!$ xD */!ëM@!z@#yH N/!x3@!A!% N/ }@!äK>!! O/ #'n9@!o3@ë"%@o[,C. 8_.,wD0!1* .0=** jäD !@!WX ú_de%ch*h bwüøg2e?ûKïCùh bQ!ïCù!ó #" ú_dbwü#c b $=)hgKe?û B bwgï|dbBõ îïXh cue,a $#îï|bwüd ø ueü%ûhdhü=ch /0#+ *%
/
*
*@ "x"@!$ xD */!ëM@!z@#yH N/#}@ä7%>! N/ o3@å;>!F! !J/ #'n9@o3@ë"%@yH )! !+.*!0!$& ?>?.< !+Mtv 2F*@ ô=e?ûDï CHdhKú_d&c ï|b|ch3Lü 1 ð $!hdbYhü#ch /J{B
&~
*61
&ì
/v%
*
@ 62pJ@x"@!lT%6%/o"@#y-@=j%F=/#x"@!x- O/n9@lê@!$
n7%/# #]7@=}@=p1 0O@!usKjo3oD '*0= 6) !+ ?>?,CD$&>'0#6?>?.@WQ újS/O%
*
!@ 63t