Scalability is a Ney concern for SMP based architecture in the current context. NUMA based architecture design seems to be a promising hope addressing this ...
Available online at www.sciencedirect.com
ScienceDirect Procedia Computer Science 78 (2016) 276 – 283
,QWHUQDWLRQDO&RQIHUHQFHRQ,QIRUPDWLRQ6HFXULW\ 3ULYDF\ ,&,63 'HFHPEHU 1DJSXU,1',$
$1RYHO$OJRULWKPLF$SSURDFKIRUDQ$XWRPDWLF'DWD3ODFHPHQW IRU180$%DVHG'HVLJQ -LWHQGUD0DGDUNDU9*RSL&KDQG396DL.LUDQ5HGG\+DUVK$URUD SCSE VIT University vellore,Tamilnadu, 632014, India
$EVWUDFW
6FDODELOLW\ LV D NH\ FRQFHUQ IRU 603 EDVHG DUFKLWHFWXUH LQ WKH FXUUHQW FRQWH[W 180$ EDVHG DUFKLWHFWXUH GHVLJQ VHHPVWREHDSURPLVLQJKRSHDGGUHVVLQJWKLVNH\FRQFHUQ$WWKHVDPHWLPH&&180$EDVHGGHVLJQDUFKLWHFWXUH GHPDQGV D GHHSHU XQGHUVWDQGLQJ DQG RSHQ YLVWDV IRU NH\ DUHDV RI LPSURYHPHQW 2XU SURSRVHG UHVHDUFK WULHV WR LQYHVWLJDWH HYROYH DQG DQDO\]H RQH RI WKH NH\ GHVLJQ LVVXHV IRU 180$ PDFKLQH DQG SURSRVHV DQ LQQRYDWLYH VROXWLRQWRDGGUHVVWKLVNH\GHVLJQLVVXHXQGHULQYHVWLJDWLRQLQWKHFXUUHQWSKDVHRIRXUZRUN ©7KH$XWKRUV3XEOLVKHGE\(OVHYLHU%9 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). 3HHUUHYLHZXQGHUUHVSRQVLELOLW\RIRUJDQL]LQJFRPPLWWHHRIWKH,&,63 Peer-review under responsibility of organizing committee of the ICISP2015 Keywords 180$ 60 &&180$
,1752'8&7,21 :LWKWKHQHHGIRUPXOWLSURFHVVLQJGHVLJQVKDUHGPHPRU\DUFKLWHFWXUHSURYLGHVSURFHVVRUVWRVKDUHDFRPPRQ PHPRU\8QLIRUPPHPRU\DFFHVV 80$ LVRQHVXFKDUFKLWHFWXUHZKHUHWKHPHPRU\LVXQLIRUPO\DFFHVVHGE\DOO WKH SURFHVVRUV ,I WKH GDWD VL]H LV LQFUHDVHG WKH SURFHVVLQJ VSHHG QHHG WR EH LQFUHDVHG ZKHUH LQ WXUQ QXPEHU RI SURFHVVRUV VKRXOG EH PRUH 6LQFH 80$ VKDUHV D FRPPRQ EXV WKH DOORFDWLRQ RI EDQGZLGWK WR WKH SURFHVVRUV LV D ERWWOHQHFNZKLFKUHVXOWVLQDVFDODELOLW\SUREOHP 7R RYHUFRPH WKLV SUREOHP 1RQ8QLIRUP 0HPRU\ $FFHVV 180$ EDVHG GHVLJQ KDV EHHQ SURSRVHG ,Q 180$ WKH PHPRU\ LV SK\VLFDOO\ GLVWULEXWHG DPRQJ 180$ QRGHV ZKHUH D FRPPRQ JOREDO DGGUHVV VSDFH LV PDLQWDLQHG7KHVH QRGHV DUH FRQQHFWHG ZLWK DQ LQWHUFRQQHFW 0HPRU\ DFFHVV LV IDVW DQG WKHUH LV QR LVVXH RI EDQGZLGWK,Q80$WKHODWHQFLHVDUHXQLIRUPZKHUHDVLQ180$LWGLIIHUVZLWKWKHGLVWDQFHRI180$QRGHV,Q 180$HDFKQRGHLVDVVRFLDWHGZLWKDORFDOPHPRU\,IWKHPHPRU\DFFHVVHVKDSSHQIURPWKLVORFDOQRGHWKHQ WKHUHLVQRLVVXHRIEDQGZLGWK$VWKHUHFDQEHDQDFFHVVWRJOREDODGGUHVVVSDFHDWVRPHWLPHWKHUHLVDQHHGWR DFFHVVGDWDIURPWKHUHPRWHQRGHV7KLVFUHDWHVDODWHQF\GLIIHUHQFHEHWZHHQORFDODQGUHPRWHPHPRU\DFFHVVHVDQG KHQFHUHPRWHDFFHVVVKRXOGEHPLQLPL]HG(YHQWKRXJK180$RYHUFRPHVWKHVFDODELOLW\LVVXHVRPHIDFWRUVQHHG WR EH RSWLPL]HG 6XFK NH\ GHVLJQ IDFWRUV LQFOXGH 'DWD SODFHPHQW 3URFHVVRU $IILQLW\ /RDG %DODQFLQJ &DFKH &RKHUHQFH 7KUHDG 6FKHGXOLQJ ,Q WKLV SDSHU ZH SODQ WR GLVFXVV GLIIHUHQW VWUDWHJLHV WR HQVXUH RSWLPDO GDWD SODFHPHQWLQ180$EDVHGV\VWHPXVLQJDXWRPDWLFSDJHPLJUDWLRQDQGUHSOLFDWLRQVFKHPH
1877-0509 © 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of organizing committee of the ICISP2015 doi:10.1016/j.procs.2016.02.044
Jitendra Madarkar et al. / Procedia Computer Science 78 (2016) 276 – 283
12'( &38 &38 &38 &38 &DFKH &DFKH &DFKH &DFKH 0HPRU\ 0HPRU\ 0HPRU\ 0HPRU\ ,175&211(&7 )LJ 180$ 1RQ8QLIRUP0HPRU\$UFKLWHFWXUH ,Q180$GHVLJQIRUHIIHFWLYHGDWDSODFHPHQWLVWKHPDMRUFRQFHUQ,PSURSHUGDWDORFDOLW\OHDGVWRLQFUHDVHLQ UHPRWHDFFHVV)RUDQHIILFLHQW'DWDSODFHPHQWSURSHUPHPRU\PDQDJHPHQWDORQJZLWKSURFHVVVFKHGXOLQJKDVWREH GRQH.H\UHTXLUHPHQWVIRUGHVLJQRIRSWLPDOGDWDSODFHPHQW180$HQJLQHUHTXLUH D 3ODFHUGDWDVKDOOKDYHWREHDYDLODEOHORFDOO\WRD180$QRGH E 3ODFHUVKRXOGDYRLGWKHRYHUORDGWUDQVPLVVLRQWKURXJKLQWHUFRQQHFW F 5HPRWHDFFHVVKDVWREHPLQLPL]HG G ,I D SDUWLFXODU QRGH KDV LQVXIILFLHQW PHPRU\ PHPRU\ UHFODLP PHFKDQLVP QHHGV WR EH EXLOW LQ WKH 'DWD SODFHU H ,Q FDVH RI PHPRU\ UHFODLP IDLOXUH WKHUH VKRXOG EH DQ DOWHUQDWH PHFKDQLVP ZLWK SODFHU WR DOORFDWH QHLJKERXULQJ180$QRGHV SUHIHUUHGQRGHV PHPRU\ 12'(
0HPRU\DIILQLW\LVRQHVXFKGDWDSODFHPHQWSROLF\ZKHUHLQZHFDQDFKLHYHWKHDERYHPHQWLRQHGIDFWRUV0HPRU\ DIILQLW\LQYROYHVGDWDDOORFDWLRQGDWDPLJUDWLRQGDWDUHSOLFDWLRQZKLFKLVHIILFLHQWO\EHLQJGHVLJQHGSURSRVHGLQWKLV SDSHU 7KH UHVW RI WKH SDSHU LV SODQQHG DV IROORZV ,Q VHFWLRQ ZH SURYLGHG WKH YDULRXV ZD\V RI SURYLGLQJ PHPRU\DIILQLW\LQ180$DUFKLWHFWXUHVHFWLRQJLYHVDGHWDLOHGH[SODQDWLRQRIYDULRXVPHPRU\SROLFLHVIROORZHG XSZLWKRXUDOJRULWKPLQVHFWLRQ6HFWLRQVXPPDUL]HVRXUGHVLJQDORQJZLWKGLVFXVVLRQDQGVHFWLRQSURYLGHV WKH&RQFOXVLRQZLWKUHOHYDQWIXWXUHZRUNLQIXWXUH /,7(5$785(6859(@7KHPDLQDGYDQWDJHRIWKLVNHUQHOV\VWHPFDOOLVWKDWPHPRU\DOORFDWLRQFDQEHFRQWUROOHGLQDEHWWHUZD\>@ 8VDJHRI/LE180$LVDFRPSOH[WDVNVLQFHLWLQYROYHVELWPDVNVSRLQWHUVPHPRU\SDJHV$OVRWRXVHWKLV /LE180$ RQH PXVW PDQXDOO\ HQWHU WKH V\VWHP FDOOV QXPDFWO VR WKDW WKH DSSOLFDWLRQ FXVWRPV WR WKH PHPRU\ SROLFLHV ZKLFK EHFRPHV D FRPSOLFDWHG ZRUN IRU WKH SURJUDPPHUV 6R WKHUH PXVW EH D PHFKDQLVP E\ ZKLFK WKH DSSOLFDWLRQ PXVW DXWRPDWLFDOO\ DGDSW WR WKH DUFKLWHFWXUH 2QH VXFK SURSRVHG PHFKDQLVP LV GLVFXVVHG LQ WKH ODWHU VHFWLRQV 2.2 Open MP 2SHQ03LVDODQJXDJHZKLFKVXSSRUWVHIILFLHQWSDUDOOHOLVP7KHDSSOLFDWLRQPXVWEHFRGHGXVLQJ2SHQ03
277
278
Jitendra Madarkar et al. / Procedia Computer Science 78 (2016) 276 – 283
SULPLWLYHV LQ RUGHU WR JHW PXOWLWKUHDGLQJ 7R SODFH WKH GDWD LQ EHWWHU ZD\ LQ 180$ PDFKLQHV FHUWDLQ H[WHQVLRQV ZHUH DGGHG WR WKH 2SHQ03 DQG LW UHTXLUHV DQ H[SOLFLW VXSSRUW IURP FRPSLOHUV %XW DOO FRPSLOHUV GRQ¶W JLYH WKH SURYLVLRQIRU2SHQ03DQGGLUHFWLYHVDUHDSSOLHGLQDVWDWLFPDQQHU ,QDPHFKDQLVPWRJXDUDQWHH PHPRU\ DIILQLW\RQ180$ PDFKLQHXVLQJ2SHQ03ZDVSUHVHQWHGE\WKH DXWKRU 7KH PDLQ LGHD LQ WKH SDSHU ZDV WRPDNH UHODWLRQEHWZHHQ WKUHDGV DQGGDWD $OVR WKH ZRUN SURYLGHV VRPH VXJJHVWLRQVRIKRZWRH[WHQG2SHQ03LQ180$PDFKLQHV7KHLUUHVXOWVSURYHGWKDW2SHQ03FDQSHUIRUPZHOORQ WLJKWO\FRXSOHG 180$ PDFKLQHV 7KHLUZRUN ZDVQRW H[WHQGHG WR DXWRPDWLF GDWD SODFHPHQW 2XUSURSRVHGZRUN WULHVWRSURYLGHWKLVPHFKDQLVPXVLQJDXWRPDWLFPHPRU\DIILQLW\LQUXQWLPH ,Q DXWKRU SUHVHQWHG DQ HIILFLHQW PHPRU\ DOORFDWLRQ LQ 2SHQ03 XVLQJ FHUWDLQ VHW RI 2SHQ03 GLUHFWLYHV 7KHVH GLUHFWLYHV JXLGH GHYHORSHUV WR DOORFDWH GDWD HIILFLHQWO\ RQ WKH 180$ DUFKLWHFWXUH $OO WKHVH GLUHFWLYHV DUH OLPLWHG WR WKH )2575$1 ODQJXDJH 8VLQJ WKHVH GLUHFWLYHV HIILFLHQW GDWD GLVWULEXWLRQ DQG SDJH ORFDOLW\ FDQ EH DFKLHYHG %XW WKHVH GLUHFWLYHV KDYH WR EH LQFOXGHG IRU DQ DSSOLFDWLRQ LQ DQ H[SOLFLW PDQQHU ZKLFK UHTXLUHV KDUGZDUH VSHFLILFDWLRQV LQ SULRU 7KHUH PXVW EH D EHWWHU DSSURDFK LQGHSHQGHQW RI SULRU NQRZOHGJH RQ KDUGZDUH ZKLFKPXVWSURYLGHHIILFLHQWGDWDSODFHPHQW %$6,6:25. Kernel’s Memory affinity policies 0HPRU\DIILQLW\LVHQVXULQJWKDWSURFHVVLQJXQLWDOZD\VKDVWKHLUGDWDFORVHWRWKHP6RWRDFKLHYHXWPRVW SHUIRUPDQFHRQ180$PDFKLQHWKHQXPEHURIUHPRWHDFFHVVGXULQJWKHH[HFXWLRQKDVWRPLQLPL]H3URFHVVRUDQG GDWDQHHGWRVFKHGXOHVRDVWKHGLVWDQFHEHWZHHQWKHPDUHWREHFORVHG7RPLQLPL]HWKHGLVWDQFHDQGDFFRUGLQJO\ LQFUHDVH WKH SHUIRUPDQFH RI 180$ DUFKLWHFWXUH ZH QHHG VRPH PHFKDQLVP DQG WRRO LQ RUGHU WR VROYH PHPRU\ DOORFDWLRQUHSOLFDWLRQDQGPLJUDWLRQWRHQVXUHPHPRU\DIILQLW\LQ180$V\VWHP+RZHYHUQRQHRIWKHVHVROXWLRQV SURYLGHSRUWDELOLW\DVPHPRU\DIILQLW\FRQWURO7RDFKLHYHWKLVPHPRU\DIILQLW\LVHQVXUHGE\DSSO\LQJDPHPRU\ SROLF\IRUDQHQWLUHSURFHVV)LUVWWRXFKDOJRULWKPZDVSURSRVHGLQ/LQX[NHUQHOZKLFKZDVWKHGHIDXOWSROLF\ WRPDQDJHPHPRU\DIILQLW\,WSODFHVWKHSDJHZLWKUHVSHFWWRWKUHDGRQWKH180$QRGHWKDWDFFHVVLWILUVW%\WKLV SROLF\GDWDLVDOORFDWHGE\WKUHDGRUPDVWHUWKUHDGWRLWVORFDOPHPRU\ZKLFKUHGXFHVUHPRWHDFFHVVHV+RZHYHUILUVW WRXFK SROLF\ DSSO\ RQ DSSOLFDWLRQ WKDW DFFHVV V\PPHWULF GDWD UHJXODU ,I WKUHDG GRHV QRW DFFHVV VLPLODU GDWD LW OHDGVWRKLJKQXPEHURIUHPRWHDFFHVV 7RUHGXFHUHPRWHDFFHVVWKHSDJHVKRXOGEHPLJUDWHGEHWZHHQ180$QRGHVWRPDNHVXUHWKDWGDWDLVFORVHUWRWKH SURFHVVRUWKDWDFFHVVLW%HIRUHSDJHPLJUDWLRQWKHIROORZLQJVHTXHQFHRIVWHSKDVWREHIROORZHG D 7KHYLFWLPSDJHZKLFKLVWREHPLJUDWHGVKRXOGEHUHPRYHGIURPWKH/HDVW5HFHQWO\XVHGOLVWV E $OOWKHUHIHUHQFHVRI37( SDJHWDEOHHQWULHV WRWKHROGSDJHDUHIUHHGDQGWKHSDJHLVSXWLQWRVOHHSWLOOWKH PLJUDWLRQRIWKHSDJHLVRYHU F ,IVXSSRVHDQ\NHUQHOUHIHUHQFHVDUHPDGHWRWKHSDJHWKHQWKHSDJHPLJUDWLRQLVDERUWHGDQGVKRXOGEH IXUWKHUSURFHVVHGDIWHUWKHNHUQHOXVDJHLVRYHU G 1HZNHUQHOUHIHUHQFHVDUHWREHPDGHIRUWKHQHZSDJHDIWHUPLJUDWLRQ 1H[WWRXFKSROLF\ZDVLQWURGXFHGWRSURYLGHDXWRPDWLFSDJHPLJUDWLRQ7KLVSROLF\LVDSSOLFDEOHIRUUHJXODU DQG LUUHJXODU GDWD 1H[W WRXFK SROLF\ SURYLGH DXWRPDWLF G\QDPLF SDJH PLJUDWLRQ XVLQJ ³FRS\ RQ ZULWH ³ &2: 8VLQJ&2:SDJHPLJUDWLRQRFFXUVRQO\LWLVHVVHQWLDOO\QHHGHG *HQHUDOO\SDJHWDEOHHQWU\ 37( FRQWDLQSURWHFWLRQELW ZULWHDFFHVVDQGUHDGDFFHVVELW WRFKHFNWKHSDJH GHWDLO&2:LVLPSOHPHQWHGLQ/,18;WRPRGLI\WKHZULWHDFFHVVELWRIWKHSDJHIURPWKHSDJHWDEOHHQWU\ZKLFK LQGLUHFWO\JHQHUDWHVDSDJHIDXOWRQDZULWHDFFHVV 7KHILJXUHH[SODLQVWKHIORZRISURFHVVRFFXUUHGLQWKHQH[WWRXFKSROLF\,QLWLDOO\E\XVLQJILUVWWRXFK SROLF\WKUHDGDVVRFLDWHVELQGVSDJHZLWKUHVSHFWLYH180$QRGH,QQH[WWRXFKSROLF\WKHSDJHLVPDUNHGE\IODJ ELW DQG LQGLFDWHV WKDW LW ZLOO EH XVHG LQ QHDU IXWXUH ,Q RUGHU WR PDUN WKH SDJH XVLQJ IODJ ELW LQ /,18; LW LV DFFRPSOLVKHGXVLQJPDGYLVH V\VWHPFDOO
Jitendra Madarkar et al. / Procedia Computer Science 78 (2016) 276 – 283
)LJ 1H[WWRXFKSROLF\LPSOHPHQWDWLRQ :KHQHYHUZHZDQWWRVHWWKHQH[WWRXFKIODJZHKDYHWRPRGLI\WKH37(SURWHFWLRQELW'XHWRWKHORDG EDODQFLQJWKHUHLVQHHGWRPLJUDWHWKHWKUHDGVIURPRQH180$QRGHWRDQRWKHULUUHVSHFWLYHRIWKHGDWDORFDWLRQRI WKHWKUHDG7KLVOHDGVWRSDJHIDXOWZKHQHYHUWRXFKRFFXUVIRUWKHSDJH:KHQWKLVKDSSHQSDJHIDXOWKDQGOHUWDNH FDUHRIWKHIDXOWVE\FKHFNLQJWKHQH[WWRXFKIODJRIWKHSDJH,WDOVRVHHVWKDWDFWXDOSDJHIDXOWLVRFFXUUHGEHFDXVH RI1H[W7RXFKSROLF\RUDUHDOSDJH)DXOW,IWKHIDXOWLVGXHWRWKH1H[WWRXFKWKHQSDJHZLOOEHPLJUDWHG$IWHUWKH SDJHPLJUDWLRQWKH1H[WWRXFKIODJLVUHPRYHGIURPWKHFRUUHVSRQGLQJEXIIHUDQGHQDEOHVLWVRULJLQDOSURWHFWLRQELWV LQWKH37(6DPHSURFHGXUHKDSSHQVLI1H[WWRXFKIDXOWKDSSHQV ,Q JHQHUDO VRPH SDJHV DUHSURWHFWHG 7KLV FDQ EHGRQHE\ XVLQJ PSURWHFW V\VWHP FDOO ZKLFKSUHYHQWV DSSOLFDWLRQ WR DFFHVV WKDW SDJH ,I WKDW SDJH LV WR EH DFFHVVHG LW OHDGV WR VHJPHQWDWLRQ IDXOW ZKLFK LV KDQGOHG E\ FXVWRPVLJQDOKDQGOHU7KLVFXVWRPKDQGOHUUHPRYHVWKHSURWHFWLRQWHPSRUDULO\WKHQPLJUDWHVWKHUHVXOWDQWEXIIHUV DQG UHVWRUHV LWV SURWHFWLRQ EDFN 7KLV PHFKDQLVP LV QRW IOH[LEOH WR LPSOHPHQW DV ZH DUH FDOOLQJ WKH PSURWHFW V\VWHPFDOO WZLFHWRKDQGOHWKHVHJPHQWDWLRQIDXOWUHVXOWLQJLQFKDQJLQJWKH7/%VIUHTXHQWO\ 7KH QH[W WRXFK SROLF\ SURGXFHV D EHWWHU SHUIRUPDQFH ZKHQ FRPSDUHG ZLWK QXPDBPLJUDWHB SDJHV OLE180$¶V$3, EHFDXVH QXPDBPLJUDWHBSDJHV 6KLIWV HQWLUH SURFHVV DGGUHVV VSDFHRQ WR QHZ 180$ QRGHZKLFKLVQRWQHFHVVDU\DVGLVFXVVHGLQWKHSUHYLRXVVHFWLRQV 7KH SURSRVHG GHVLJQ DGGUHVVHG DOO WKH DERYH PHQWLRQHG SROLFLHV DORQJ ZLWK UHSOLFDWLRQ LQ QH[W WRXFK SROLF\ 'HVLJQDQGDUFKLWHFWXUHIRUSURSRVHGV\VWHP 2XUSURSRVHGV\VWHPLVWU\LQJWRSURYLGHDUHSOLFDWLRQPHFKDQLVPIRUWKHDIRUHPHQWLRQHGQH[WWRXFKSROLF\ ZLWK VRPH WKUHVKROG FRQVWUDLQW 2XU DOJRULWKP WULHV WR JLYH DQ HIIHFWLYH PHFKDQLVP IRU 0HPRU\ DIILQLW\ ZKLFK WDUJHWVLQUHGXFLQJWKHUHPRWHDFFHVVHV7KHSURSRVHGPHFKDQLVPDOVRWULHVWRLQFRUSRUDWHDOOWKH/LE180$VXSSRUW PLJUDWLRQUHSOLFDWLRQSUHIHUUHGDQGLQWHUOHDYHG automatically 'HVLJQ0LJUDWRU %HKDYLRU(QJLQH 25 'HVLJQ ,QIHUHQFH(QJLQH0HPRU\ $OORFDWRU 'HVLJQ5HSOLFDWRU
279
280
Jitendra Madarkar et al. / Procedia Computer Science 78 (2016) 276 – 283
)LJ 'HVLJQDQG$UFKLWHFWXUHRIWKHSURSRVHGV\VWHP 0RGXOH 'HVLJQ$OORFDWRU 'HVLJQDOORFDWRUDOORFDWHVPHPRU\WRDQRGHE\GHIDXOWSROLF\ )LUVWWRXFK $FFRUGLQJWRILUVWWRXFKSROLF\ PHPRU\ZLOOEHDOORFDWHGWRWKHWKUHDGWKDWILUVWWRXFKHVLW,WVHWVWKHEHKDYLRURIWKHVKDUHGSDJH7KHQLWVHWVWKH QH[WWRXFKIODJLQGLFDWLQJWKDWWKHSDJHZLOOEHDFFHVVHGLQWKHIXWXUH%HIRUHGRLQJKLVZHPXVWFKHFNZKHWKHUWKH PHPRU\LVDYDLODEOHIRUPLJUDWLQJRUUHSOLFDWLQJWKHSDJHVWRWKHGHVWLQDWLRQQRGHVRUQRW,ISDJHIDXOWYDOXHLVOHVV WKDQ WKH WKUHVKROG YDOXH DQG PHPRU\ LV DYDLODEOH PLJUDWH WKH SDJH ,Q FDVH LI WKH PHPRU\ LV QRW DYDLODEOH WKHQ PHPRU\UHFODLPKDVWREHGRQHE\XVLQJPDGYLVHBIUHH 0RGXOH %HKDYLRUHQJLQH %HKDYLRUHQJLQHGHFLGHVZKHWKHUSDJHFDQGLGDWHLVILWWR0LJUDWHRU5HSOLFDWHEDVHGRQWKHWKUHVKROGYDOXH 7KHWKUHVKROGYDOXHLVWREHFKHFNHGZLWKSDJHIDXOWFRXQWHU,WDOVRSURYLGHVDDFFHVVPHFKDQLVPIRUSDJHV HJ 0RUHZULWHDFFHVVLPSOLFLWO\GLVDEOHVWKHSDJHIURPUHSOLFDWLRQ 0RGXOH 'HVLJQ0LJUDWRURU5HSOLFDWRU 7KLVIRUPVWKHFRUHHQDEOHUIRU0LJUDWLRQRU5HSOLFDWRU%HIRUHWKHPLJUDWLRQRUUHSOLFDWLRQKDSSHQVDJDLQ ZH KDYH WR FKHFN IRU VXIILFLHQW PHPRU\ DYDLODELOLW\ ,I WKH VXIILFLHQW PHPRU\ LV QRW SUHVHQW PHPRU\ UHFODLP PDGYLVHBIUHH KDVWREHKDSSHQHG,IWKHSDJHIDXOWFRXQWLVJUHDWHUWKDQWKUHVKROGIUHTXHQF\GDWDUHSOLFDWLRQKDV WREHGRQH3DJHIDXOWFRXQWHUVKRXOGEHLQFUHPHQWHGRQO\RQWKHQH[WWRXFK:KHUHDVLIWKHSDJHIDXOWFRXQWLVOHVV WKDQ WKUHVKROG IUHTXHQF\ GDWD PLJUDWLRQ KDV WR EH GRQH )LJXUH GHSLFWV WKH RYHUDOO SURSRVHG PHWKRGRORJ\ IORZ GLVFXVVHGDERYH 7R LQFOXGH UHSOLFDWLRQ PHFKDQLVP ZH LQWURGXFHG VRPH YDULDEOHV DV WKUHVKROG UHVHW 567 DQG ZULWH IUHTXHQF\WKUHVKROG7KHWKUHVKROGIUHTXHQF\FKHFNVWKHQRRIDFFHVVRIWKHSDJHIRUDFHUWDLQSHULRGDFURVVWKHQ 180$ QRGHV 7KH UHVHW IODJ UHVHWV WKH SDJH IDXOW FRXQWHU YDOXH DIWHU D SDUWLFXODU FORFN F\FOH WLPH 7KH ZULWH IUHTXHQF\WKUHVKROGILQGVKRZPDQ\ZULWHDFFHVVKDSSHQHGWRDSDJH,ILWLVKLJKWKHQFKHFNWKHZULWHIUHTXHQF\ WKUHVKROG LI WKH QXPEHU RI ZULWH DFFHVV LV JUHDWHU WKDQ ZULWH IUHTXHQF\ WKUHVKROG GR QRWKLQJ (OVH FKHFN IRU WKH PHPRU\DJDLQDQGUHSOLFDWHWKHSDJH
)LJ 1H[WWRXFKSROLF\ZLWKUHSOLFDWLRQ
Jitendra Madarkar et al. / Procedia Computer Science 78 (2016) 276 – 283
$OJRULWKP ,QLWBFRQILJXUDWLRQBFXPBDOORFDWRUB180$ Step 1: Allocate a page in NUMA node as follows: Step1.1.a Choose default memory policy( ) Step 1.1.b Initialize the Memory Access parameters (like page_table_entry,protectionbits, pid, bitmask, writeaccess bit, vma, main memory etc…) Step1.2 set the behavior of the shared space using madv_behavior() Step 1.3 set the next touch flag using madv_willneed() Step1.4 Change the page table entry protection bits i.e pte_modify(page table entry, get_protection(0)). Step1.5 Initialize threshold and reset value 6KDUHGEHKDYLRUSURFHVV Step 2 Check the recent access to Page i.e if (touch==true)do the following steps { /*checking whether the page fault occurred from the next touch or not*/ 2.1.Check the pages for its local or remote access using Page_found=check_page(pid,bitmask,pageaddress) 2.2. If page is being from local node i.e if (Page_found==1) Do nothing and abort () /* next touch policy not needed*/ Else /* page fault handler mechanism and increment the page fault count */ 2.2. Generate a page fault using intpage_fault_count=page_fault_handler(addr,writeacess,pagetableentry ,vma ,mm) 2.2.1Increment the page fault using page_fault_count++; /*until reset values doesn’t cross the periodical time value, if crosses reset the page fault values*/ 6WHS5HSOLFDWLRQRU0LJUDWLRQ 3.1. Check the Page faults limit& compare the same with threshold limit set using. If (page_fault_count>= threshold value) { 3.2.Replicate the page using page_replicate(node mask_all,vma,page table entry,pd,mm) } else /* (if page_fault_count< threshold value) { 3.1 Migrate the page using next touch policy as page_migrate(mm,pte,ptl,mm,vma) /*provide migration*/ } Algorithm for Main design block design for data placer() Automatic_data_placer_for_NUMA() { Step a) Configure the NUMA nodes &allocate the pages; Init_configuration_cum_allocator_NUMA ( );/* step 1 above*/ Step b)Organize the NUMA shared process behavior; Perform step2 above; Step c) Based upon the threshold frequency value Either Migrate or Replicate; /* step 3* above */ `
281
282
Jitendra Madarkar et al. / Procedia Computer Science 78 (2016) 276 – 283
7KHDOJRULWKPGHVFULEHVWKHQH[WWRXFKSROLF\ZLWKUHSOLFDWLRQ,QLWLDOO\HYHU\DSSOLFDWLRQREH\VWKHGHIDXOW PHPRU\ SROLF\ DV ILUVW WRXFK 0DGYLVH V\VWHP FDOO JLYHV D SULRU NQRZOHGJH WR NHUQHO RI KRZ DQ DSSOLFDWLRQ H[SHFWVWRXVHWKHPHPRU\7KHIROORZLQJWDEOHGHVFULEHVWKHGLIIHUHQWIXQFWLRQDOLWLHVRIPDGYLVH V\VWHPFDOO%\ XVLQJPDGYBZLOOQHHG V\VWHPFDOOIRUZKLFKZHFKDQJHWKHSDJHWDEOHHQWU\ELWV UHDGZULWHDFFHVVIODJ LQGLFDWHVWKDWSDJHZLOOEHXVHGLQWKHIXWXUH1RZZKHQHYHUIDXOWRFFXUVNHUQHOFRQILUPVWKDWWKLVIDXOWLVE\QH[W System call
Description
madvise_normal
Default kernel way of accessing the addresses.
madvise_sequential
Informs kernel that application will access a listed range addresses in a successive way.
madvise_random
Tells kernel that page references are in a random manner.
madvise_willneed
The specified address range will be referenced in future.
madvise_dontneed
The specified address range will not be referenced in future.
madvise_free
Intimating kernel that the specified range of addresses are no longer important(these addresses are freed when the memory pressure is high)
WRXFKSROLF\DQGGHFLGHVWRWDNHGHFLVLRQRIPLJUDWLRQRUUHSOLFDWLRQ 7DEOH 'LIIHUHQWIXQFWLRQDOLWLHVRIPDGYLVH &RPSDUDWLYHDQDO\VLVDQG'LVFXVVLRQV ,QWKLVSURSRVHGGHVLJQZHKDYHLQYHVWLJDWHGDQGDGGUHVVHGWKHNH\FRQFHUQVRUOLPLWDWLRQVLPSRVHGE\WKH H[LVWLQJZRUNV2XU$OJRULWKPLFGHVLJQSURSRVHGVHHPVWREHRXWSHUIRUPLQJZLWKUHVSHFWWR/LE180$VSHFLILFDOO\ &RQVLGHUDQDSSOLFDWLRQZKLFKLVJHQHUDWLQJWKUHDGVZKHUHWKHVHWKUHDGVDUHUDQGRPO\GLVWULEXWHGLQ SDUDOOHO DPRQJ WKH 180$ QRGHV ,Q /LE180$ XVLQJ PELQG ZH FDQ DOORFDWH PHPRU\ WR DQ\ QRGH FSXELQG DOORZV SURFHVV WR H[HFXWH RQ D VHW RI &38V IRU D VSHFLILHG QRGH /LE180$ DOVR SURYLGHV SDJH PLJUDWLRQ XVLQJ QXPDBPLJUDWHBSDJHV ZKHUH D ODUJH EXIIHUV DUH WR EH PRYHG ZKLFK OHDGV WR LQFUHDVH LQ ODWHQFLHV XQQHFHVVDU\ SDJHV DUH DOVR WUDQVIHUUHG >@)RU WKH /LE180$ VXSSRUW WKH SURJUDPPHU VKRXOG DOWHU WKH DSSOLFDWLRQ FRGH PDQXDOO\ZKLFKLVDFRPSOH[WDVN> @ %\ ILUVW WRXFK SROLF\ DGRSWHG E\ XV WKH WKUHDG ZKLFK WRXFKHV WKH SDJH ILUVW ZLOO EH DOORFDWHG WR WKH FRUUHVSRQGLQJ QRGH WR PDNH WKH DFFHVV ORFDO ,I RWKHU WKUHDGV IURP GLIIHUHQW QRGHV QHHG WR DFFHVV WKH VDPH SDJH UHPRWH YHU\ IUHTXHQWO\ LQ D VKRUW SHULRG WKHQ PLJUDWLRQ EHFRPHV D WHGLRXV MRE ZKHUH LQ 7/% YDOXHV DUH WR EH IOXVKHG IUHTXHQWO\ 6R LQVWHDG RI PLJUDWLRQ ZH UHSOLFDWH WKH SDJH EDVHG RQ WKH SDJH IDXOW FRXQW LI SDJH IDXOW LV JUHDWHUWKDQWKHWKUHVKROGFRXQW LQRXUSURSRVHGV\VWHPDGGUHVVLQJWKHNH\FRQFHUQ 5HSOLFDWLRQPD\DOVRFDXVHPHPRU\FRQJHVWLRQ'XSOLFDWHFRSLHVLQGHHGUHVXOWLQUHGXQGDQF\RIGDWD6R LQRUGHUWRPDNHWKHPHPRU\DYDLODEOHZHXVHPDGYLVH PDGYBIUHH UHVHWVLQDSHULRGLFDOLQWHUYDODQGWKHUHSOLFDWHG SDJH LV UHPRYHG EDVHG RQ WKH QHFHVVLW\ )RU HYHU\ PLJUDWLRQ RU UHSOLFDWLRQ WKH VWDWXV RI WKH SDJH LV FKHFNHG LQ DGYDQFH +HQFH ZH HQVXUHG WKDW RXU SURSRVHG V\VWHP SURYLGHV D EHWWHU DOWHUQDWLYH WR WKH H[LVWLQJ ZRUNV ZLWK DXWRPDWLFSURFHVVLQJEXLOWLQ
Jitendra Madarkar et al. / Procedia Computer Science 78 (2016) 276 – 283
&RQFOXVLRQDQGIXWXUHZRUN 6FDODELOLW\ EHLQJ D PDMRU FRQFHUQ LQ 80$ EDVHG V\VWHP 180$ DGGUHVVHG WKLV NH\ FRQFHUQ YHU\ HIIHFWLYHO\:LWKWKHDGYHQWRIPDQ\FRUHHQYLURQPHQWVZKHUHPHPRU\LVGLVWULEXWHGDPRQJWKHGLIIHUHQWFRUHVLWLV FKDOOHQJLQJ WR GHVLJQ D WKUHDG VFKHGXOHU DORQJ ZLWK SURSHU GDWD GLVWULEXWLRQ DFURVV GLIIHUHQW QRGHV LQ DQ HIILFLHQW PDQQHU :H KDYH WKRURXJKO\ LQYHVWLJDWHG H[LVWLQJ VROXWLRQ ZLWK UHVSHFW WR GDWD SODFHPHQW DQG H[SORUHG WKHLU OLPLWDWLRQ LQ FXUUHQW FRQWH[W ODFN RI G\QDPLF DGDSWLRQ WR UXQ WLPH GLUHFWLYHV LQ 2SHQ03 DQG ODFN RI DXWRPDWLF SODFHULQWKHFXUUHQW/LE180$$3,V :HKDYHSURSRVHGLQYHVWLJDWHGDQGHYROYHGDQHIILFLHQWGHVLJQIRU180$EDVHGPDFKLQHIRUDXWRPDWLF GDWDSODFHPHQWLQVFDODEOHIDVKLRQWKHUHE\DGGUHVVLQJWKHNH\FRQFHUQ G\QDPLFPLJUDWLRQDQGUHSOLFDWLRQ ,QQH[WSKDVHRIRXUZRUNZHSODQWRYDOLGDWHRXUSURSRVHGIUDPHZRUNIRU180$GDWDSODFHPHQWHQJLQH DQG SHUIRUPDQFH DQDO\]H WKH VDPH ZLWK UHVSHFW WR H[LVWLQJ DIRUHPHQWLRQHG ZRUNV 6XEVHTXHQWO\ ZH SODQ WR LQYHVWLJDWHWKHRWKHUNH\DUHDVRILPSURYHPHQWLGHQWLILHGHUVWZKLOHLQ180$EDVHGGHVLJQ 5HIHUHQFHV ,&&RPSLOHU7KUHDGDIILQLW\LQWHUIDFH KWWS VRIWZDUHLQWHOFRPHQXVLQWHOFRPSLOHUV *&&RPSLOHU7KUHDGDIILQLW\LQWHUIDFH KWWS JFFJQXRUJRQOLQHGRFVOLEJRPS(QYLURQPHQW9DULDEOHVKWPO $QGL.OHHQ686(/DEV$Q180$$3,IRU/LQX[ $.OHHQ$180$$3,IRU/LQX[7HFK5HS1RYHOO$SULO 2QOLQH$YDLODEOH KWWS ZKLWHSDSHUV]GQHWFRXNSKWP '61LNRORSRXORV($UWLDJD($\JXDGpDQG-/DEDUWD([SORLWLQJ0HPRU\$IILQLW\LQ2SHQ037KURXJK6FKHGXOH5HXVH6,*$5&+ &RPSXWHU$UFKLWHFWXUH1HZVYRO QRSS ± -%LUFVDN3&UDLJ5&URZHOO=&YHWDQRYLF-+DUULV&$1HOVRQDQG&'2IIQHU([WHQGLQJ2SHQ03IRU180$0DFKLQHVLQ6& ¶ 3URFHHGLQJVRIWKH$&0,(((&RQIHUHQFHRQ6XSHUFRPSXWLQJ'DOODV7H[DV86$ + 5LFKDUGVRQ +LJK 3HUIRUPDQFH )RUWUDQ KLVWRU\ RYHUYLHZ DQGFXUUHQW GHYHORSPHQWV 7HFK 5HS 70& KWWS KSIIULFHHGXSXEOLFDWLRQVLQGH[KWPO 6 %HQNQHU DQG 7 %UDQGHV (IILFLHQW 3DUDOOHO 3URJUDPPLQJ RQ 6FDOH6KDUHG 0HPRU\ 6\VWHPV ZLWK +LJK 3HUIRUPDQFH )RUWUDQ &RQFXUUHQF\ 3UDFWLFHDQG([SHULHQFHYROSS ± &KULVWLDQH3RXVD5LEHLUR-HDQ)UDQoRLV0pKDXW0$L 0HPRU\$IILQLW\,QWHUIDFHLQULD YHUVLRQ-XQ &KDUOHV.RHOEHO'DYLG/RYHPDQ5REHUW6FKUHLEHU*X\6WHHOHDQG0DU\=RVHOThe HighPerformance Fortran Handbook )UDQoRLV%URTXHGLV1DWKDOLH)XUPHQWR%ULFH*RJOLQ5D\PRQG1DP\VW3LHUUH$QGUp:DFUHQLHU'\QDPLF7DVNDQG'DWD3ODFHPHQWRYHU 180$$UFKLWHFWXUHV DQ2SHQ035XQWLPH3HUVSHFWLYHSXEOLVKHGLQ,QWHUQDWLRQDO:RUNVKRS 70X-7DR06FKXO]DQG6$0F.HH,QWHUDFWLYH/RFDOLW\2SWLPL]DWLRQRQ180$$UFKLWHFWXUHVLQ6RIW9LV¶ 3URFHHGLQJVRIWKH $&06\PSRVLXPRQ6RIWZDUH9LVXDOL]DWLRQ1HZ
