ApacheCon2009: Practical HBase

3UDFWLFDO+%DVH -RQDWKDQ*UD\DQG0LFKDHO6WDFN

$SDFKH&RQ

8V -RQDWKDQ*UD\ +%DVH&RPPLWWHU &RIRXQGHUDQG&72RI6WUHDP\FRP MJUD\#VWUHDP\FRP 0LFKDHO6WDFN +%DVH&RPPLWWHU+DGRRS30& :RUNVIRU0LFURVRIW6HDUFK VWDFN#DSDFKHRUJPLVWDFN#PLFURVRIWFRP

7KLV7DON    

4XLFNRYHUYLHZ $QVZHUVWRFRPPRQXVHUTXHVWLRQV $ORRNDWDIHZSURGXFWLRQ+%DVHLQVWDOOV 1H[W+%DVH5HOHDVH

4XL]%LJ7DEOHLV" $$SLHFHRIIXUQLWXUHIURPWKH,UZLQ$OOHQ79VKRZ/DQGRI WKH*LDQWV "  RU   %$*RRJOH7HFKQRORJ\GHVFULEHGLQDSDSHU%LJWDEOH$ 'LVWULEXWHG6WRUDJH6\VWHPIRU6WUXFWXUHG'DWDE\&KDQJHW DO"

%LJ7DEOH5HFDS7KH%DVLFV 'LVWULEXWHGFROXPQRULHQWDWHGVWRUH %XLOWRQWRSRIWKH*RRJOH)LOH6\VWHP 0XOWLGLPHQVLRQDOVRUWHG0DS 7DEOH!5RZ!&ROXPQ)DPLO\!&ROXPQ4XDOLILHU!9HUVLRQ!9DOXH

6HPL 6WUXFWXUHGGDWD 1RQUHODWLRQDOQRW64/ 1RWDGURSLQUHSODFHPHQWIRU! $OODERXWVFDOLQJ  1HDUOLQHDUDV\RXDGGFRPPRGLW\ER[HV %7NQRZQWRVFDOHWR!PDFKLQHV

%LJ7DEOH5HFDS'DWD0RGHO 7DEOHRI5RZVDQG&ROXPQ)DPLOLHV &ROXPQ)DPLOLHVGHILQHGXSIURQW 1HZFROXPQVFDQEHDGGHGWRDIDPLO\RQWKHIO\ :LWKRXWVFKHPDPRG &HOOVDUHYHUVLRQHGUGGLPHQVLRQ 6WURQJO\FRQVLVWHQW 5RZPRGLILFDWLRQVDUHDWRPLF 5RZVVWRUHGLQE\WH䇲㼘㼑㼤㼛㼓㼞㼍㼜㼔㼕㼏㻌㼟㼛㼞㼠㼑㼐㻌㼛㼞㼐㼑㼞㼫 7DEOHG\QDPLFDOO\VSOLWLQWRWDEOHWVRU³UHJLRQV´ $XWRPDWLFDOO\VSOLWDVJURZV (DFKUHJLRQFRQWDLQVFRQWLJXRXVYDOXHV>VWDUW.H\HQG.H\  5HJLRQVDUHZKDWDUHVSUHDGDERXWWKHFOXVWHU

4XL]+%DVHLV" $&KHPLFDOWHUPIRUWKH %DVHIRUPRI+HURLQ" RU    %$Q$SDFKH2SHQ6RXUFH15'%06LQVSLUHGE\*RRJOH V %LJ7DEOHSDSHU"

+%DVH3URMHFW 3URMHFW*RDO%LOOLRQVRIURZV;PLOOLRQVRIFROXPQVRQ FOXVWHUVRIFRPPRGLW\+DUGZDUH /HYHUDJHV+')6DVRSSRVHGWR*)6 3DUWRI+DGRRSVXLWH 8VHZKHQQHHGUDQGRPUHDOWLPHUHDGZULWHDFFHVV

WR   

%LJ'DWD

)HDWXUHV &RQYHQLHQWEDVHFODVVHVIRUEDFNLQJ+DGRRS0DS5HGXFH MREVZLWK+%DVHWDEOHV 4XHU\SUHGLFDWHSXVKGRZQYLDVHUYHUVLGHVFDQDQGJHW ILOWHUV 2SWLPL]DWLRQVIRUUHDOWLPHTXHULHV 7KULIWJDWHZD\ $5(67IXO:HEVHUYLFHJDWHZD\WKDWVXSSRUWV;0/ 3URWREXIDQGELQDU\GDWDHQFRGLQJRSWLRQV &DVFDGLQJVRXUFHDQGVLQNPRGXOHV ([WHQVLEOHMUXE\EDVHG-,5% VKHOO 0HWULFVYLD+DGRRSPHWULFVVXEV\VWHP )LOHV*DQJOLDRUYLD-0;

+%DVH 0DMRUSHUIRUPDQFHUHOHDVH 5DQGRPDFFHVVRQSDUZLWK5'%06  5HPRYHGVLQJOHSRLQWRIIDLOXUH 0XOWLPDVWHUZDXWRPDWLFIDLORYHU=RR.HHSHU  0DQ\PRUHLPSURYHPHQWV 5ROOLQJUHVWDUWVIRUPLQRUXSJUDGHV 1HZVLPSOLILHG$3, +)LOH.H\9DOXHEORFNFDFKHHWF

+%DVH&OXVWHU

:K\+%DVH" ([LVWLQJVROXWLRQVDUHFOXQN\ 5'%06GLVWULEXWLRQDQGVKDUGLQJQRWIXQDQGH[SHQVLYH /RVHPRVWIXQFWLRQDOLW\DORQJWKHZD\  'LVWULEXWLRQDQGSDUWLWLRQLQJDUHEXLOWLQ 'HVLJQHGWRVFDOHRQFRPPRGLW\KDUGZDUH  +%DVHLVDOODERXWVFDOLQJ %LOOLRQVRIURZV7HUDE\WHVRIGDWD :HOO\HDKH[FHSW

&RPPRQ4XHVWLRQV 5HYLHZRIWKLVPRQWK VPDLOLQJOLVW 2UGHULQJFOXVWHUZKDWGR,JHWDQGKRZVKRXOG,OD\LWRXW" +RZGR,FRQQHFW+%DVHWR0DS5HGXFH"([DPSOHV" +RZGR,ORDGP\17%LQWR+%DVHTXLFNO\" +RZGR,FRQQHFWWRWKH+%DVH&OXVWHU" :KDWDUHDOOWKHVH=RR.HHSHUPHVVDJHVDERXW" +RZGR,PDNHWKLQJVIDVWHU" +RZGR,EDFNXSP\+%DVHFOXVWHU" +RZGR,PRGHOP\5'%06VFKHPDLQ+%DVH" &DQ,PLJUDWHP\ROGHU+%DVHLQVWDOO" :KDWDUHILOWHUVDQGKRZGRWKH\ZRUN"  $QVZHULQJWKHVHLVZKDWWKLVSUHVHQWDWLRQLVDERXW

4XLFN\$QVZHUV ,WVZRQ WFRQQHFW,WVIDLOLQJZHLUG,WZRQ WVWDUW 5HDG *HWWLQJ6WDUWHG

3DVVZGOHVV66+ LQJDURXQGFOXVWHUDQGWRVHOI )LOHGHVFULSWRUVOLPLWXOLPLWQ! ;FHLYHUVLQ+')6GIVGDWDQRGHPD[[FLHYHUV >VLF@ 'DWDQRGHORJ[FHLYHU&RXQWH[FHHGVWKHOLPLWRI FRQFXUUHQW[FLHYHUV $WOHDVW+')6SDWFKZKHQUXQQLQJ05 6LQFHFRPPLWWHGLQDQG+DGRRS :LWKRXWLW1ROLYHQRGHVFRQWDLQFXUUHQWEORFN  :DQWODWHVW+%DVHEXWRXU+DGRRSLVRU 3DWFKLQ+%$6( %UDQFKWRUXQRQ+DGRRS

+RZVKRXOG,OD\RXWP\FOXVWHU" 0RVWFOXVWHUVFRORFDWH56DQG'1 8VXDOO\D77DQG05WDVNVDVZHOO 7XQHIRU,2'1 5$056 &3877 EDODQFH  *HWHQRXJKKDUGZDUHIRU\RXUDSSOLFDWLRQ 1RVZDSSLQJQRVWDUYDWLRQQRIXOOGLVNXWLOL]DWLRQ  +%DVHUXQVRQPRGHVWKDUGZDUHIRUGHY FRUHV*%VLQJOHGLVN  %XW+%DVHFDQWDNHDGYDQWDJHRIJRRGVWXII FRUHV*%66'V

:KDWNLQGRI+:VKRXOG,XVH" 5HJLRQ6HUYHUVORYHPHPRU\VRGR'1V *%SRVVLEOHEXW*%UHFRPPHQGHG  (QRXJKFRUHVIRU\RXUDSSOLFDWLRQ ,I\RXZLOOUXQVLJQLILFDQW05FRQVLGHUFRUHV  3HUIRUPDQFHDOVRWLHGWR,2 5DQGRPDFFHVVVHHNVVWUHDPLQJVFDQVFRPSDFWLRQV 7KHPRUHGLVNVWKHEHWWHU-%2'5$,'  +%DVHUHTXLUHVDFOXVWHU 7\SLFDOSHUIRUPDQFHFRPHVZLWKQRGHV  ,QJHQHUDOMXVWVD\QRWR90V

&RPPRQ&OXVWHU&RQILJXUDWLRQV 6DQGER[IRU$3,IDPLOLDULW\DQGXQLWWHVWLQJ 3VHXGRGLVWULEXWHGPRGHRQ\RXUORFDOPDFKLQH 127IRUSHUIRUPDQFHWHVWLQJ  'HYHORSPHQWDQGSURWRW\SLQJFOXVWHU QRGHVZFRUH*%[*%'156>77@  6PDOOSURGXFWLRQFOXVWHUZR05 QRGHVZFRUH*%[*%'156  /DUJHUSURGXFWLRQFOXVWHUZ05 QRGHVZFRUH*%[7%'15677

+RZGR,PDNHWKLQJVIDVWHU" &RQILJXUDWLRQRSWLRQV  0D[LPXPUHJLRQVL]H KEDVHKUHJLRQPD[ILOHVL]H  0GHIDXOW ,QFUHDVHLI\RXKDYHODUJHNH\VRUDKXJHGDWDVHW  0HP6WRUHPXOWLSOLHU KEDVHKUHJLRQPHPVWRUHEORFNPXOWLSOLHU  GHIDXOW 5HJLRQ6HUYHUEORFNVXSGDWHVRQFHLWUHDFKHVOLPLW LQFUHDVHLI\RXUKDUGZDUHFDQKDQGOHLW

0RUH3HUIRUPDQFH7XQLQJ *DUEDJH&ROOHFWLRQ 8VH&06GRQRWXVHSDUDOOHOZLWKFRUHV ;;8VH&RQF0DUN6ZHHS*& /RZHU&06WKUHVKROG ;;&06,QLWLDWLQJ2FFXSDQF\)UDFWLRQ  ([SOLFLWO\VL]H3DU1HZ ;;1HZ6L]H P;;0D[1HZ6L]H P  /=2&RPSUHVVLRQ 3HUIRUPDQFHJDLQVDFURVVWKHERDUG 1RWLQFOXGHGRXWRIWKHER[IRUOLFHQVLQJUHDVRQV %XWDFOHDUZLQVRZRUWKWKHHIIRUW *=LVLQFOXGHGVORZHUWKDQ/=2EXWIDVWHUWKDQQR FRPSUHVVLRQDWDOO

3HUIRUPDQFH7ULFNV 'LVDEOHEORFNFDFKLQJIRUVFDQV 6FDQVHW&DFKH%ORFNVIDOVH 3UHYHQWVEORFNVIURPHQWHULQJWKH/58  'LVDEOH:$/IRUSXWV 3XWVHW:ULWH7R:$/IDOVH 3UHYHQWVHGLWVIURPJRLQJWRZULWHDKHDGORJ :$51,1*3UHYHQWVGDWDUHFRYHU\XQGHUIDLOXUH

+RZGR,EDFNXS+%DVH" 0DS5HGXFHMREGRQHDWWKHFOLHQWOHYHO RUJDSDFKHKDGRRSKEDVHPDSUHGXFH([SRUW RUJDSDFKHKDGRRSKEDVHPDSUHGXFH,PSRUW $VRI &DQEHVORZDQGUHVRXUFHLQWHQVLYHEXWVDIHU  +')6OHYHOILOHFRS\ &RS\WDEOHGLUHFWRULHVWRORFDOGLVNRUXVHGLVWFS $OVRQHHGWRJUDEFDWDORJWDEOHV )DVWEXWGRHVQRWLQFOXGHGDWDLQ0HP6WRUHV 6RPDQXDOO\IOXVKWDEOHVILUVW  )XWXUH&RS\DEOHVQDSVKRWV

:KDWDUHILOWHUVDQGKRZGRWKH\ZRUN" &XUUHQWO\ILOWHUVPXVWEHVWDWHOHVV 1RW\HWDEOHWRULGHVWDWHDFURVVUHJLRQV 5XQLQDURZFRQWH[WVHUYHUVLGH 0RVWO\XVHIXOIRUUHJH[OLNHRSHUDWLRQVLQODUJHVFDQV 5HJH[RQURZTXDOLILHUYDOXH 3UHIL[PDWFKLQJRQURZNH\V  ([DPSOH6FDQDOOXVHUVZKRDUHIHPDOH  6FDQV QHZ6FDQ  VVHW)LOWHUQHZ6LQJOH&ROXPQ9DOXH)LOWHUIDPLO\ %\WHVWR%\WHVJHQGHU &RPSDUH2S(48$/ %\WHVWR%\WHVIHPDOH  

)LOWHUVRIWKH)XWXUH 6XSSRUWIRUVWDWHIXOILOWHUV +%$6( '\QDPLFORDGLQJRIQHZILOWHUV +%$6( 6LPSOHTXHU\ODQJXDJH +%$6( )DEULFDWHGUHVXOWILOWHUV +%$6( 3DUDOOHOVFDQQHUVILOWHUV +%$6(

&DQ,UXQ+%DVHRQ(&"