Data Grid: Supporting Data-Intensive applications ... - Semantic Scholar

6 downloads 269314 Views 98KB Size Report
Data Grid: Supporting Data-Intensive applications in Wide-Area Networks. Xiao Qin and Hong Jiang. Department of Computer Science and Engineering.
DataGrid:SupportingData-Intensiveapplications inWide-AreaNetworks

XiaoQinand

HongJiang

DepartmentofComputerScience andEngineering University oNebraska-Lincoln, f Lincoln, NE68588-0 {xqin, jiang}@cse.unl.edu

TechnicalReport No. TR-03-05-01 May2003 tyoNebraska-Lincoln f Lincoln, NE 68588-0115

115

Abstract Adatagridisacollectionofgeographicallydispersedstorag network.Thegoalof

eresourcesoverwidearea

datagridsystemistoprovidealargevirtualstorageframewor

kwith

unlimitedpowerthroughcollaborationamongindividuals,institutions,andresourc study, we first review naumber of data grid services such as,

to just name few, a metadata

dataaccessservice,andperformancemeasurementservice.Wet techniquesforboostingtheperformanceodata f grid,andthesek

service,

heninvestigatevarious eytechniqueshavefalleninto

threecamps,namely,datareplication,scheduling,anddatamovement.Si becomes one othe f prime concerns in grid computing, we review two int indatagrid.Finally andimportantly,wehaveidentifiedfiveinte outsome potentialsolutionsto

es.Inthis

ncesecurityissue eresting issues of security restingopenissues,andpointed

the open problems.

1

1. Introduction A grid icasollection ogeographically f dispersed computing resource

s, providing large a virtual

computingsystemtousers.Therearefourcommonlyusedkindsofres

ourcesingrids:

computation,storage,communications,andsoftware.In whatfollows, four t

ypesof resourcesare

briefly introduced. • Computation. Computingresourcesuppliedbytheprocessorsotfhemachinesisthe commonresourceingrids.Thisresourceihs eterogeneousin ature processorswith variety a ospeeds, f architectures,andsoftwarepl

most

since , iistcombination a of atforms.Ajobsubmittedbyan

endusercanutilizecomputationresourcesbyeitherrunningonasin

gleavailablemachineor

beingbrokendownintonumber a ofsub-jobstobexecutedinparallelonm

ultiplemachinesin

the grid. • Storage. Storage iviewed s athe s second most commonly used resource igan presents an integrated view of storage icoined s a“Data s grid”

rid. The grid that or “Data Grid”. Memory, hard disk,

andotherpermanentstoragemediaarereferredtoasstorager

esources.Inthisstudy,weare

particularly interestedinsecondarystorageinthegrid,since

secondstoragei1000 s timesslower

than the memory attached tprocessor. ao Throughout this report, we

only address the issues of data

gridwithrespecttosecondary storage. Many Networked file systems, which exhibit security and relia applied tdoata grid. These file systems include: Network File S

bility features, have been widely ystem (NFS) [27], Distributed File

System (DFS), Andrew File System (AFS) [18], General Para

llel File System (GPFS), and Parallel

VirtualFile System(PVFS) [15][43]. • Communication. The data grid becomesincreasingly moreattractivewith the rapi communication capacity, since the high performance data grid re movingdataandjobsamongthesitesinthegrid.Toenhancethereliabi 2

dgrowth in

sts on the available bandwidth for lityofthegrid,itis

desirabletoapplyredundantcommunicationpaths,whichics apableoaf ll burdenimposedbyexcessivedatatraffic.Amonitorsystemasso

eviatingthenetwork ciatedwithagridmanagement

canbdeevelopedtoidentify the communicationbottlenecksin grid. a • Software.Incasewheresoftwareisnotinstalledoneverymachineinthe scheduler, inaddition tboalancing the grid load, can assign jobs requirin

grid,thegrid tghe specific software to

machinesthatisable tomeetthe jobs’ needs. Theamountofscientificdatageneratedbysimulationsorcollec experimentsigs enerallylarge,andsuchdatatendstobegeog

tedfromlarge-scale raphicallystoredoverwide-area

networksforthesakeolfarge-scalecollaborations.Thenotionofc

omputationalgridhasbeen

proposedforseveralyears,mainlyfocusingoneffectiveusageofg

lobalcomputationaland

networkresourcesinwide a areanetwork/Internetenvironment.Howeve computationalgrid,wheretheeffectiveusageofstorageresourc

r,theperformanceoaf esisignored,willdecrease

substantiallyivast f majorityoapplications f runningintheg

ridaredata-intensive.Toovercome

thisproblem,varioustechniqueshavebeendevelopedandincorporatedinto soa infrastructure.Inthisreport,wewilltakeacloselookatthese

calleddatagrid techniquespresentedinthe

literature,andpointoutsome openissuesindata grid research. Therestofthereportisorganizedafsollows.Inthesectiontha

follows, t threemajor datagrid

servicesarereviewed.InSection3techniques , forimprovingtheperf described idnetail. Security issue idnata grid ibriefly s re

ormanceodf atagridare viewed iS n ection 4Section . identifies 5

several open issues, and presents the potential and possible solutions t Section6concludesthereportbysummarizingthemainopenproblemsand

tohe open problems. Finally, ourpreliminary

solutions.

2. DataGrid Services Theobjectiveodafatagridsystemistwo-fold.First,tointegr

ateheterogeneousdataarchives 3

storedilarge na numberofgeographically distributedsitesi

nto single a virtualdatamanagement

system.Second,toprovidediverseservicestofittheneedsohighf

performancedistributedand

data-intensivecomputing.Inthissection,wefocusonthreeessential

services:metadataservice,

data accessservice,andperformance measurement. 2.1MetadataServicesfor Data GridSystems TheMetadataodafatagridsystemisthemanagementinformat

ionregardingtothedatagrid.

The metadata includes file instances, the contents of file ins

tances, and variety a ostorage f systems

in the data grid. The goal of metadata service ito sfacilita

te aenfficient means of naming, creating

andretrievingthemetadataofdatagrid[12].Therearetwotypes

ofmetadata:application

metadataandfabricmetadata.Whiletheformerdefinesthelogic

alstructureotfheinformation

representedbydatafiles(e.g.XML[9]),thelattercorres

pondstotheinformationrelatedtothe

characteristicsof the data grid. Sincethedatagridconsistsoalarge f numberosftorageresourc

esgeographicallydispersed

overalarge-scaledistributedenvironment,anessentialrequirement capability obeing f scalable, meaning that the performance

ofthedatagridisthe

can bseustained acatertain level while

supportingalargenumberodf ataentities.Wahleat l.haveproposed

aso-calledLightweight

DirectoryAccessProtocol(LDAP)[40],whichachievesahighsca

labilitybyapplyinga

hierarchicalnamingstructurealongwithrichdatamodels.The operateattribute-basedsearchingandwritingofdataelements developed distributed a directory service for general grid metada

LDAPprotocolisdesignedto [40].Fitzgeraldetal.have ta, which itnurn can baepplied to

representthe metadata odata f grid. Barueal. tproposed metadata a catalog(MCAT)for satorage

resourcebroker [3].MCAT

onlydistinguishedlogicalnamespacefromphysicalnamespace, specificationolafogicalcollectionhierarchy,whichachieved 4

butalsoprovidedthe ha ighscalability.Inaddition,the

not

performance wasimproved inMCAT by introducing containersto aggrega

te smallfiles.

Chervenakeal. trecently developed prototype a Metadata Service (MC the metadata iclassified s into two categories: logical fi Alogicalfilenameinthisprototype

S) for data grids, where

le metadata and physical file metadata [13].

representsauniqueidentifierfordatacontent,whereasa

physicalfilenamedenotesanidentifierforpa hysicalcontent

onstoragesystem.UnlikeMCAT

mentionedearlier,theservicesinMCSarededicatedtodescribing

files,andthephysicalfile

propertiesarenotcoveredbyMCS.WhileMCATutilizespropriet a

arydataexchangeformatto

communicatewiththestorageresourcebrokerserver,MCSemploys

XMLinterface[9]to

communicate withthe clients. 2.2DataAccessServicesfor DatagridSystems Datastoredandretrievedbyadatagridsystemmayreside,

bydesign,indifferentsiteson

different storage devices. Moreover, it is an essential requirem

ent that end users be not necessarily

aware of the physical locations of accessed data. Consequently,

the data grid system has to provide

galobalview of data for applicationsrunning otnhe grid[12]. Data access services, defined bsytorage API and command lines

are , required tboceompatible

withtraditionalfilesystems.Indoingso,applicationsthatare

notoriginallydesignedforgrid

environmentcanrunonagridsystemwithoutcomplicatedmodifications. systems couple with desire a teonrich the interface odata f a

Inaddition,grid

ccess service teoxploit full benefits of

the gridsystems. Beforeproceedingtodiscussingthedataaccessservicesi

nthegridenvironment,webriefly

review twowell-knowndistributed file systems.The transfer s

ize oeach f data accessisrestricted

inNFS[27]duetothefindingthatstorageaccessperformancein

thewideareanetworkgreatly

dependsontransfersize.Whileaccessingdatathatmightbe

merely portion a ofafile,AFS[18]

transfersandcopiesthe entire file,causing unnecessary I/O tr

affic. 5

Globus, serving aan sinfrastructure for grid computing, provides acces

to sremote files through

x-gass, ftp oHTTP r protocols [17]. White eal. thave proposed the Le

gion I/O model that provides

raemoteaccesscapability [41].Legionachieves55-65% offtp’s

write bandwidth and 70-85%of

ftp’sreadbandwidthformasstransfers.TheperformanceoL f egi overhead ithe f transfer sizes are less than 1Mbyte. To allow e

onsuffersduetotheprotocol nd users to exploit context space and

manipulate context structure,Legion also offers some command line ut those iUnix an File System. For example, legion_cp copies files

ilities, which are similar to within context space obetween r a

nativefilesystemandcontextspace.Furthermore,Legionallows

applicationstoaccess

BasicFileObjectsby meansof raemote I/O interface. PattenandHawickproposed Distributed a ActiveResourceArchitect

ure(DARC)thatenables

thedevelopmentofportable,extensible,andadaptivestorageservice

os ptimizedforendusers’

requirements[23].ToevaluatetheeffectivenessoDARC, f daist

ributedfilesystemusing DARC

wasdeveloped[23].Thefilesystem,calledStrata[23],hasaloc hierarchy.InStrata, saitewilling toffer satorageresour In case where client a is going to

ation-transparentdirectory cehastousethemechanisminDARC.

access raemote data service, its site calls for laoca

of the specified data resource, which itnurn communicates with

instantiation l

remote storage services. The data

accessservicesimplementedinStrataincludelookup,create,mkdir

r,ead,write,andreaddir

operations. 2.3Performance Measurement inDataGrid Asmentionedearlier,m a onitorcanbde evelopedtoidentifycommunicat

ionbottlenecksina

grid. In general, performance a monitor plays an important role in t

he grid environment to measure

theperformanceodiverse f resources,suchastoragesystem,n

etwork,andcomputers,therefore

identifying performance problems in different resources. With the can analyze the monitoring data from different point of views. Since 6

monitoring service in place, one multiple resources may affect

oneanotheracrosstimeandspace,performanceproblems(e.g.low

throughputorhighlatency)

mightbeimposedbcombination ay oresources. f Thus,itisimportant

andnon-trivaltodiagnose

performanceproblemsinaccordancewiththeperformancemeasureme

ntsprovidedbythe

monitor. The objective othe f monitoring service ito sobserve performance

bottlenecks, which are likely

tooccurinanycomponentsodf atagridsystems.Previousstudyhas

shownthatmonitoringat

application level is an effective approach for both performance anal

ysis and application debugging

[8][38].Forexample,Snodgrassincorporatedrelationaldatabasesi

ntoamonitoringsystemto

monitor complexsystems[28]. GMA(GridMonitoringArchitecture)proposedbyaworkinggroupdesc

ribesthekey

componentsoagrid f monitoringsystemalongwithsomeessentialint modelusedintheNetworkWeatherService[33]wasrecentlyext

eractions[37].Thedata endedbyLeeeal. ttomeetthe

needs of monitoring data archives in grid environments [20]. More specifi raelational monitoring data archive that was designed teoffici

cally, Lee et al. proposed ently handle high volume streams of

monitoringdata[20].Ahigh-energyphysicsgridprojectreveals gridcanboeptimizedbymeansoanalyzing f thedetailedinstrum

thatthe

datatransfersindata a

entationdatamaintainedby the

monitoring service associated withthe data grid[45].

3. High Performance DataGrid Inthissection,wereviewtheissuesodata f replication,scheduli

ng anddatamovement,which

mustbe addressedinorder toachieve highperformance and scalability

in data a grid.

3.1DataReplication Datareplicationios neothe f keytechniquestoachievebetter

accesstimesandreliabilityin

datagridsystems.Alargebodyofworkhasbeenfoundondatarepli applications[10][31][34].Ingeneral,datareplicationcanbeutili 7

cationfordistributed zedtoimprovedatalocality,

increasedependability,reduceaccessdelay,andenhancescalabili

tyfordistributedsystemsin

wideareacomputingenvironments.Theapproachesoriginallyproposed arealsoapplicabletodatagridsystems,sincedatagridscompr data sites. The existing techniques for

fordistributedsystems iselarge a numberofdistributed

data replication idnata grids can bdivided e into four camps,

namely,(1) Architecture and management for data replication, (2) D

ata replication placement, (3)

Data replicationselection,and (4) Data consistency. • Architecture oadf atareplicationservice Before discussing data grid replication architecture, it is nec

essary troeview the following three

featuresodf atagrid.First,dataids istributedandreplicatedto

geographicallydistributedsites,

implying that data consistency has to bm e aintained

under wide-area environments where network

latencies are generally long [22]. Second, data dnoot change och r

ange infrequently. For example,

data im n any scientific collaboration systems arenot published to

the community until it hasbeen

prepared[14].

Althoughalargenumberofdata-intensiveapplicationssharethisfeat

ure,the

feature does not necessarily apply teovery data-intensive applic

ation ignrid environments. Third,

areplicationserviceigs enerallyimplementedontopoadf atabas

emanagementsystemordata

store,therebyallowingvariousstoragetechniquesappliedtodiffer

entsites.Inotherwords,data

gridstrivestobuilduniform a datareplicationservicethatis

applicabletobroad a rangeodata f

stores.Therefore,datareplicationisimplementedasamiddlewa

recomponentbetweengrid

applicationsandthe physicaldata storage. Ageneralhigh-levelarchitecturewithfourlayersfordata

replicationserviceinadatagrid

systemisdepictedinFig. [112].We describe eachlayer asf

ollows:

(1)Filetransferlayerconsistsofiletransferprotocols

suchas:FTP,HTTP,andGridFTP[2].

Note thatGridFTPian sextendedversionoFTP. f (2) The responsibility othe f Replica catalogues layer is to

map logical file names to physical files. 8

For example, Replica Catalog iG n lobus Data Grid tools [2] is capa copiesof laogicalfile bfyacilitating a

ble of keeping track ophysical f

map fromlogicalfile namesto physicallocations.

4.Replica Selection 3.Replica Manager Replica 2. Catalogue File T1.ransfer FigureArchitecture 1. ofadata ReplicationServiceinadatagridsystem (3) The replica manager layer’s functionalities include create,

move, modify and delete replicas in

thedatagrid.Thereplicacatalogfunctions,suchacsreatingandd

eletinglogicalfilenames,can

both bienvoked bgyridapplications and operations issued ireplica an mana

gement layer. Once a

fileics opiedfromonesitetoanotherbythereplicamangerl

ayerandregisteredinthereplica

catalogue,itcanbreetrieved bayny applicationrunning oanrbitrary

site itnhe grid.

(4)Replicaselectionlayerisresponsibleforchoosingthemost

appropriatereplicafrom

copiesgeographicallydispersedacrossthegrid.Theselectionsorf

those

eplicationpoliciesdependon

readandwriteaccesstodata,whichcanbcelassifiedintothet

wocategories.(1)Read-only data,

(2) writable data withwell-definedfile ownership,

and (3) writable data with varying writers.

• DataReplicationManagement We first take look a areplication t policies designed for read

-only data. A pilot project called the

GridDataManagementPilot(GDMP)hasbeenlaunchedtodevelopaprotot managementwithin data a gridenvironment[26].IntheGDMPsoftware, database files has been used, and high-level a replica catalog

ypeforreplication saecurereplicationof interface iimplemented s buysing the

native objectivity federation catalogue. Consequently, the GDMP sof Objectivity file. However, replicate files of more flexible

tware is restricted troeplicate data types can bseupported bryeplacing

9

the objectivity file catalogue with other catalogues, such aGlobus s

Replica Catalogue [17].

The replication policy inGDMP iasynchronous s with subscription a m

odel. Before describing

thereplicationpolicyimplementedinGDMP,someimportantdefinitions follows. A producer is saite where one omore r files are written, and a going tolocally replicatethese files.The withtheir relatedinformation,whereasthe have not been updated athe t consumer

areintroducedas consumer is the

site that is

exportcatalogue is laistof allnewly created filesalong importcatalogue is list a ofnewly publishedfilesthat site. It is the producer’s responsibility tdoetermine when to

write new filesintothe exportcatalogue andtherefore making the

mavailable for itsconsumers.

Thereplicationcanbeimplementedatdifferentlevelsogf ranular

ity,suchasfileleveland

objectlevel[30].Realizingthatfilereplicationinadataa

nalysisapplicationispotentially

inefficient,Stockingereat l.hasimplementedobjectreplicat

ioninadatagridprototypeusing

GlobusData Gridtools[30]. Foster et al. recently developed general a framework, which defi

nes parameterized a set of basic

mechanismtocreateavarietyofreplicalocationservices[

14].Byconfiguringthesystem

parametersdynamically,oneisabletotunesystemperforman

cebymeansocf onsideringthe

tradeoff among reliability andcommunicationoverhead, and storage update/

accesscosts.

• DataReplicationPlacement In recent a study [24],severalreplicationselectionstrategie beenproposedandevaluated.Comparedtostaticreplication,dynamicr addsanddeletesreplicasinaccordancewithchangingworkload,ther

fsormanaginglargedatasethave eplicationautomatically ebysustainingahigh

performance evenwhendata accesspatternchanges[24]. • DataReplicationSelection Once data replicas are generated and stored amultiple t site

according s tvariety ao oreplication f

placement strategies discussed above, the performance odata f grid 10

can bneoticeably improved by

choosing optimizedreplica location for eachdata access. Replica selection ihasigh level service that helps grid applica

tions to choose replica a based on

systemperformanceanddataaccessfeatures.Vazhkudaietal

.[39]havedesignedand

implementedareplicaselectionservicethatusesinformationwi

threspecttoreplicalocationas

wellasapplication’sspecific requirementstodetermine repl a

ica among allreplica alternatives.

• DataConsistencyinDataGridSystems Thedatareplicationtechniquesindatagridsystemsreviewedabove

mainlyaimatread-only

files,therebyavoidingdataconsistencyproblems.Ingeneral,l

ocalconsistencyateachsiteis

guaranteedbydatabasemanagementsystems(DBMS),whereasgl

obalconsistencyindatagrid

systemsneedstobemaintainedbyaso-calledgridconsistencys

ervice.Todesignapractical

consistency service,one hastotake intoaccountthe following two dynam (1)File and replica catalogues are updated once filesare cre

ic factors[22]: ated, deleted omoved. r There are two

typesorfeplicacatalogues:centralreplicacataloguea

nddistributedcatalogue.Asforacentral

replicacatalogue, daatabasemanagementsystemcanbaepplie information,aswellasupdatethefileinformation.Incasewhere distributed, the replica catalogue itself needsto bseynchronize

dtostorelogicalandphysicalfile thereplicacatalogueis d. For example, creating new a file

meansitsfile name hastobuepdatedianllcataloguereplica

s. Thissynchronization ireferred s to

asmeta-datasynchronizationsincereplicacatalogueim s et

a-informationusedtomanageactual

data. The consistency model proposed bD y üllmann eal. tassumes that

meta-data synchronization

ishandledbytheunderlyingreplicacatalogueservice,andthisass

umptionnarrowsthefocus

downtothe consistency issue oactual f data [22]. (2)Updateofilecontents.Whenreplicateddataisupdatedatonesit propagatedtoandbecome visible taollother replicaswithin speci a

e,itschangesshouldbe fic mount a of time.

Someworkhasbeendoneondataconsistencyindistributedenvironments 11

[32][42].For

example,Sunetal.proposedanovelconsistencymodelwiththreeproper

ties,namely,

convergence,causality-preservation,andintention-preservation[32].Them consistencymodelisreal-timecooperativeeditingsystemsthat

aintargetofthis

allowmulti-user,physically

dispersedpeople,toview andedit sahareddocumentatthe same time

over networks.

Toaddresstheissueofgriddataconsistencymaintenance,Düllmann consistency service [22], whichisupported s beyxisting datagrid s catalogues [12] mentioned earlier (See Fig.1). This service hel

etal.proposeda ervicesthatinclude the replica ps grid applications in updating data

andpropagating the updateddata tother sitesthathave subscribed ttohe

consistency service.

Wenowsummarizefivepossibleconsistencylevelsdiscussedin[22 toleratespossiblyinconsistentcopiesirseferredtoacsonsist

].(1)Thelevelthat encylevel

unabletoguaranteethatafilereplicacopiedfromonesiteto

–1.Atthislevel,oneis

anotheritshelatestversion.(2)

Consistent level g0uarantees that the newly copied replica is

snapshot a of the original file at some

point. (3) At consistency level 1, each replica has been created whe

nw o rite transactions issue on

itsoriginalfile,therebymakingotherclientsusethereplica

withoutanyinternalconsistency

problem.Consistencylevel1istheminimalconsistencyofferedb systemusingtheconceptoftransactions,(4)Sincereplicaandit

yadatabasemanagement os riginaldataarenotpartofa

commondatabasesystem,itisnot saurprisewhentwocopiesaredi

vergent.Atconsistency level

2,itisassumedthateachsitehasanindependentdatastoreandi

us nawareotfhedatastoreof

replicatedsites.Thus,thereplicasofafilearestoredandma

nagedbythelocaldatabase

managementsystems.(5)Consistencylevel3isviewedasthes

trictestlevel,wherecomplex

locking and recovery procedures are involved. In grid a environment, this le

vel of consistency can

beimplementedifalldataaccessoperationsshareacommoninte

rface.Inotherwords,a

distributeddatabasemanagementsystemcanbedirectlyappliedto

thegridenvironmentto

guarantee the dataconsistency athis t strictlevel. However, it

isnotfeasible for mostapplications 12

inthegrid,sinceitincurssevereperformancepenaltieswhendat

avolumeaccessedbythe

applicationgrowsuptoseveralPetabytes. Tomaintainthebasicconsistencylevel,areplicationoperationmus database read transaction [22]. Thus, read a lock ofile an

bt emanipulatedasa

has to bobtained e before site a iready s to

produceanewreplicaofthefile.Thesitereleasestherea

dlockwhenthereplicahasbeen

successfullyduplicated.Likewise,theconsistencyservicerests operationsare issued

onwritelockswhentheupdate

on set a of replicas.

Alldata replicationtechniquesdiscussedabove are summarized iT na

ble I.

TableI.SummaryoData f ReplicationTechniques Researchers Samar andStockinger, [26]

Granularity File

DateType ReadOnly

Consistency No.Producer decides when topublish new files No

Vazhkudai,Tuecke,et File ReadOnly al.[39] Stokinger etal.[30] File/Object ReadOnly No.Con

Chervenak,Deelman, File ReadOnly andFoster [14] Ranganathan and File ReadOnly Foster [24] Lamehamedietal.[19] File ReadOnly Mullmann,etal.[22] File ReadandWrite Sun eal. t[40] String ReadandWrite

sumer decides when to replicateobjects Relaxedconsistency for metadata No No

Yes Yes

Technique Subscription model,partial replication model Replication selection Object replication

Replication location service Replication placement

Replication placement Consi stency service ThreeNovelproperties

3.2SchedulinginDatagridSystems Duetothelargenumberouf sers,datasize,anddistanceinvolved,sch challengingissuetomakedatagridscalableaswellasef

eduling becomesa ficientforlargenumberofjobs.

Therefore,itisinteresting todiscussthe relatedwork regardi

ng to scheduling idnata grid.

Previousresearchhasshownthatschedulingiasfundamentaltechniquef performanceforgridapplications[6][29][1][34].However,thesea

orachievinggood pproachesdonottakeinto

accountthe distributeddata constraints. Schedulingschemesdesignedfordata-intensivejobshavegenerallybe 13

enclassifiedinto

two

majorcategories,namely,movingdatatojobandmovingjobtodata.Ingr

idenvironments,both

approachesneedtoconsiderdatalocationwhendeterminingjobplacem

ent.Toreducethe

frequencyorfemotedataaccessandtheamountoftransferreddata

, datareplicationcanbean

optimizationtechnique awe s discussed inSection3.1. Thain eal. tdeveloped system a where execution sites band together

into I/O communities [35].

EachI/OcommunitycomprisesseveralCPUsandstoragedevicest

hatprovidestoringand

retrievingdataservicesbothforlocalandremotejobs.Forexampl

e,adistributedfilesystem

sharedamongagroupofmembersisviewedasanI/Ocommunity.To constraintsonstoragedeviceswithinI/Ocommunities,Thain

enablejobstodeclare etal.extendedClassAdsthatare

currentlyusedinCondorsystem[21]todescribejobs’requirementsi singleClassAdialist s of(attribute,value)pairs,whereval

nadistributedsystem.A uecanbeitheratomicocomplex r

expressions.Figure 2illustratesthree differentkindsof ClassA

[35]. d

Type = “job” Type = “storage” ype = “machine” TargetType TargetType = “machine” Name = “machine”= “job” Name = “raven” Cmd= “sim.exe” HasCMSData = true Owner = “thain” CMSDataPath = “/cmsdata = “slinux” Requirements= rements= (Owner = “thain”) && Onwer = “thain”) NearestStorage.HasCMSData NearestStorage = (Name = “turkey”) && (Type = “storage Figure2ExamplesofjobClassAd,Machine



”)

ClassAd,andStorageClassAd

CondormatchesajobClassAdsinputwithbothmachines’andstorages’C storedinthesystem.Once match a ifsound,thejobwillbeexecut

lassAdsthatare edby themachinelinkedwith

the job’sspecific storage devices. Tominimizeremotedataaccessoverhead,Basneyeal. t haveinvent

edaso-calledexecution

domainframeworktodefineanaffinitybetweenCPUanddataresource

sinthegrid[4].By

applyingtheframeworktotheCondorsystem,data-intensivejobscan

bescheduledtorunon

CPUsthathave accesstorequireddata. 14

To gain aenfficient execution oparameter f sweep applications on

the grid, Casanova et al. have

studiedfouradaptiveschedulingalgorithmsthatconsiderdistributeddat

astorage[11].The

proposed algorithms are heuristic,yet effective iennvironments whe

re some computing nodes are

bestforsometasksbutnotforothers.Thekeyideabehindtheseschedul

ingalgorithmsisto

judiciouslyplacefilesformaximumreuse.Theseschedulingalg

orithmshavebeenimplemented

in user-level a gridmiddleware,whichcentrally handlestask sche

duling.

Comparedwithcentralizedschedulingframework, daistributedscheduli be more scalable and efficient for the grid storage that provides

ngapproachseemsto

grid I/O service for hauge number

of data-intensive jobs. RanganathanandFosterhaveproposedadistributedschedulingframeworkw consistsoaf nexternalscheduler,alocalscheduler,andadatase schedulerofsaiteirsesponsiblefordispatchingthesubmittedjobs

hereeachsite scheduler t [25].Theexternal toanappropriatesite,which

caneitherbeonewiththeleastloadortheonewhererequireddata functionality othe f local scheduler is to decide the order in whic

havebeenstaged.The jhobs are executed athe t site. The

datasetscheduler makesdecisionson whenandwhere to replicate data

.

3.3DataMovement Incomputational a grid,datamovementmechanismplaysanimportant systemperformance.Thisids rivenbytworequirementsimposedby datagriditself.First,applicationsneedtoaccessdatathat

roleinachievinghigh bothgridapplicationsand isnotlocatedatthesitewherethe

computationips erformed.Second,replicationservicesareresponsible

formanagingreplicasby

meansof copying andmoving data throughnetworks. Tofulfillthe above tworequirements, Bester etal.have proposed a service calledGlobal Access to Secondary Storage (GASS) [7]. data movement strategies that are geared for the common I/O pa

data movementand access The service not only incorporates tterns of grid applications, but also

15

supportsprogrammermanagementodf atamovement.Theperformanceeval

uationshowsthat

GASSian sefficientapproach todata movementingridenvironments. Thainetal.havedevelopedadatamovementsystemcalledKanga reliabilityaswellasthroughputofgridapplicationsbyhidingne

roo,whichimproves tworkstoragedevicesbehind

memoryanddiskbuffer[36].Morespecifically,theapproachrelie

sonKangaroo’sabilityto

overlap CPU and I/O processing intervals by using background process

es to move data and handle

errors.

4. SecurityIssue Withtheusageodifferent f machinesiGrid, na theissueosec f

urity,whichhasbeenprovided

bymostexistingdatagrids[12][26][41],becomesoneoftheprime traditionalsecuritytechniques,suchasencryptionandaccesscontrol applied tgrid. ao For example, uaser has to baeuthenticated and author

concerns.Inprinciple, c, anbeconservatively ized before contacting any

remote site. Duetothespacelimitation,inwhatfollows,weonly reviewtwoint

erestingissuesinsecurity

addressedinGDMP[26].SamarandStockingerhavestudiedthemainsec sensitivity odata f andunauthorized use onetwork f

urityissuesof

bandwidth to transfer hugefiles. Specifically,

thesecuritymoduleoG f DMPserver,basedontheGlobusSecurity functionstoacquirecredentials,initiatecontextestablishmenton requestsontheserverside,encryptinganddecryptingmessagesandc Consequently, the security module protects saite from any undesire

Service(GSS)API,offers clientsideandacceptcontext lientauthorization. fdile transfers and rejects any

unauthorizedrequests.

5. Open Issues Although variety a otechniques f have been proposed taochieve high performa therearestillsomeopenissuestobaeddressed.Thissectionlis

nce idnata grids, tstheopenissuesrelatedtohigh

16

performance data grid. 5.1ApplicationReplication Althoughsomeeffortshavebeenmadetoreplicateactualdataac little attention has been paid troeplicating frequently used gri

cessedbygridapplications, adpplications across the grid. Once a

schedulingschemedecidestomoveanapplicationtowardsitsdata,a

replicationselectionpolicy

forapplicationscanbedevelopedtochoosethemostappropriateapplicati

oncopyinthegrid.

There are three reasonsthatmotivate uto sdistinguishapplication

code fromactualdata:

(1)Foremost,anexecutablecodeoan fapplicationaone t sitemightnot

beable troun

site.Onecanstraightforwardlytacklethisproblembyusinga

onanother

java-programmingenvironment,

whichbecomesincreasinglypopular[44].However,alargenumberofexi applications have been developed iF nORTRAN, C or other programming lang problem,onecanhavethesourcecodemovedtodestinationsitesandobtainthe

stingscientific uages. To solve this executablecode

by compiling the source code athe t destinationsites. (2)Theapplicationreplicationtechniquesprovideefficientsupporttoap architecture.Forexample,awell-designedpartitionalgorithm effectively divide data among the processors where the processing

arallel-shared-nothing canbeusedtodynamicallyand application have been staged to

achieve maximumparallelism. (3)Itisreasonably assumedthatdatacanbsetoredaany t sit

e,providedthatithasregisteredaas

member of daatagrid.Unfortunately,thisassumption inot s practical sitesmightnotbe able tofulfillapplications’ hardware or/and sof

for applicationssince some tware requirements.

Tofacilitateaparallelcomputingenvironmentincomputational directionscanbestudyinganapplicationreplicaservice,whichincl managementandapplicationselection.Theapplication duplicate,remove,andupdatecopiesofapplicationinstances,aswella 17

grids,oneofthefuture udesapplicationreplica replicamanagementisdesignedto stomaintainthe

consistencyamongthereplicas.Theapplicationselectionservice

isdevisedtochoosean

optimizedlocationwhere anapplicationreplica hasbeen stored troun t

he job.

5.2ConsistencyMaintenance InSection3.1,wehavediscussedtherequirementsocf onsistencymaint scientificdataandmetadata.Webelievethattheproposedapplicat

ionreplicationservicesmake

theconsistencymaintenanceevenmorecomplex.Thisisbecause,in scalability,theservicesarerequiredtoboefferedidis na

enanceimposedby

ordertoobtainhigh tributedfashion.Inaddition, garoupof

programmers are allowed tcoollaborate with one another to develop in grid a environment,andgridapplicationsare likely tobpeeriodical We realize that, from the summary odata f replication techniques consistencymodelsindatagridworkwellforreadonly data,butli

some large-scale applications ly upgraded. given iT n able Imost , existing ttleattentionhasbeenpaidto

the consistency issuesfor data thatneedsto bw e rittenmultiple

times.In data a grid environment,

manydatasetsaremultiple-writteninnature.Whenagroupofprogr

ammerscollaboratively

develop large-scale a scientificapplicationsonthegrid,somepr

ogrammersmightbeworking on

thesamesourcecode,implying thatthecodenotonly needstobe

writtenmultipletimesbutalso

likelytobewrittenconcurrentlybyseveralprogrammers.The

refore,iw t illbeinterestingto

developarelaxedconsistencymodeltoeffectivelyhandlethecon

sistencyproblemsin

multiple-writtendata discussedabove. 5.3AsynchronizedDataMovement Theimpactofdatamovementontheperformanceoflocalsiteis especiallytrueithe f volumeotransferred f dataihs ugeodat r

enormous,andthisis m a ovementsoccurfrequently.To

alleviate such burden a resulting fromdata movements, paotential of moving data withoutsacrificing the performance oapplications f runni protocol can bdeesigned tm o ove asynchronized data if the loads of sour 18

solutionito sstudy new a way ng olnocalsites. Thus, a ce site and destination site

arebelowacertainthreshold.Comparedwiththeexistingdatamov

ementapproachesthattreat

synchronizedandasynchronizedequally,thepotentialsolutionisexpec

tedtoimprovethe

performance odata f gridwhenthe I/O andnetworkloadsateach site

are bursting inature.

5.4PrefecthingSynchronizedData We notice thatdata prefetching schemesfor synchronized data have

caughtlittle attention. We

believethatdataprefetchingtechnique,whichcomplementsthenewdata

movementapproach

presentedinSection5.3,canoptimize the performance odata f grid in

casewhererequired data is

movedtowardstothe remote site where itsjobresides. By using ClassAd language to explicitly define applications’ r

equired files, static a prefectching

schemecanmakethesefilesavailablebeforeapplicationsneedt

oaccessthem.Asoneotfhe

possiblefuturedirections,anewprefetchingalgorithm,whichis predicating files, will be developed teostablish correlations am

capableofdynamically ong files based ostatistic an model.

This new dynamic prefetching algorithm actively provides grid appli

cations with files by shipping

requiredfilesinadvance before applicationsare locally loading the

files.

5.5DataReplication AscanbseeenfromTableIthe , ideaoreplicating f frequently

accesseddataonmultiplesites

hasbeeneagerlypursued.Thedrawbackofthemostexistingreplication replication granularity ifile, s w a hole big file might be repli

techniquesitshat,as cated even siafmall portion othe f file

isfrequentlyaccessed.Thisproblemisanalogoustofalsesharing system.To

indistributedsharedmemory

provideapotentialsolutiontotacklethisproblem,onecaninvestigat

model,wherethereplicationgranularitycanbedynamicallyadjus

esanewdata tedaccordingtothebenefit

gainedfromreplicasandthe overhead ogenerating f replications. Interestingly,feweffortshavebeenmadetodevelopsomecachinga diskcachereplacementpolicies.Therefore,itwillbeinterest

lgorithmswithefficient ingtostudy caching a algorithm,in

19

which files cached olnocal disks are viewed atemporary s replic

as. The main difference between a

cached file and regular a replica is that the file in disk ca

che might be removed when the disk cache

isunable toaccommodate newly arrivedfiles.

6. Conclusions Wehavereviewedanumberodf atagridservicesandvarioustechni performance odata f grid. Additionally, the issue of security idnat

quesforimprovingthe grid a has been briefly discussed.

Five openissuesandthe possible future directionsare summarized as

below.

(1)While significant a amountofdatagridresearchhasbeendonei

ndatareplication,theideaof

replicatingfrequentlyusedgridapplicationsacrossthegridhasr

eceivedlittleattention.One

possiblefutureworkitsodevelop new a applicationreplicaservice,w

hichcanbbeuiltontopof

data replicationservice. (2)Iitsnoticedthatmostconsistencymodelsintheexisting

datagridshavenotaddressed

consistencyissuesformultiple-writtendata.Asoneotfhefuture

directions,onecandevelopa

relaxedconsistencymodeltoeffectivelyhandletheconsistency

problemsforanumberof

commonfile accesspatternsincluding multiple-written pattern. (3)Iw t illbeinterestingtostudyanewwayofmovingdataa

atime t whenbothsourceand

destinationsitesarelightlyloaded,therebyachievingbetterpe

rformanceindatagridwithout

sacrificing the performance oapplications f running on localsites. (4)Afourthinterestingfuturedirectionistodesign

anewprefetchingalgorithmthatcan

dynamically predicate filesthatare likely tboreetrieved. (5)Lastbutnotleast,oneotfhepossiblefuturedirectionsitsos

tudyafile-cachingalgorithm,

where the replication granularity onew af data model can bdy e

namically tuned ianccordance with

the benefitand the overheadogenerating f replicas.

20

Acknowledgement WewouldliketothankDr.ByravRamamurthy,Dr.SteveGoddard,Dr.Da

vidSwanson,and

Dr.XiaoChengZengfor their valuableandinsightfulcomments

andsuggestions. Thiswork was

supportedbyanNSFgrant(EPS-0091900)andaNebraskaUniversityFoundat

iongrant

(26-0511-0019).

References [1]D.Abramson, J.Giddy, I.Foster, and L.Kotler, “High Performance Pa Killer Application for the Global Grid,”

rametric Modeling with Nimrod /G:

Proc. of the International Parallel and Distributed Processing

Symposium May 2000. [2]BAllcock, . J.Bester,J.Bresnahan,A.Chervenak,IFoster, . e

t.al.,“EfficientDataTransportand

Replica Management for High-Performance Data-Intensive Comput onMassStorageSystems

ing,” Proc. 18th IEEE Symposium

2001. ,

[3]CB . aru,RM . oore,AR . ajasekar,andM.Wan,“TheSDSC

StorageResourceBroker,”

Procof

CASCON Conference1998. [4]J.Basney,ML . ivny,andP.Mazzanti,“UtilizingWidelyDistr Efficiently withExecution Domains”,

ibutedComputationalResources

ComputerPhysicsCommunications,

[5]J.Basney,R.Raman,andM.Livny,“High-throughputMonteCarlo,” Conferenceon ParallelProcessing forScientificComputing

Volume140,2001. Proc.ot heNinthSIAM

March , 1999.

[6]F.Berman,RW . olski,SF . igueira,JS . chopf,andG.Shao,“Applicati

on-LevelSchedulingon

Distributed HeterogeneousNetworks,”Proc.ofSupercomputing 1996. [7]J.Bester,I.Foster,C.Kesselman,J.Tedesco,andSTuecke, . “

GASS:ADataMovementandAccess

ServiceforWideAreaComputingSystems,”Proc.of

SixthWorkshoponI/OinParalleland

Distributed Systems May , 51999. , . [8]WBethel, . B.Tierney,J.Lee,D.Gunter,andSLau, . "UsingHigh-

SpeedWANsandNetworkData

CachestoEnableRemoteandDistributedVisualization,"

Proc.oftheIEEESupercomputing

Conference 2002. [9]TB . ray,JP . aoli,andC.Sperberg-McQueen,TheExtensibleMar

kupLanguage(XML)1.0W#C

Recommendation,World WideWebConsortium, Feb.1998. [10]LBreslau, . P.Cao,L,Fan,G.Phillips,andSShenker, . "WebCac

hingandZipf-likeDistributions:

EvidenceandImplications,"ProceedingsofIEEE Infocom,1999. [11]HCasanova, . G.Obertelli,F.Berman,andR.Wolski,“TheAppLeS User-LevelMiddlewarefor theGrid,” [12]AChervenak, .

ParameterSweepTemplate:

Proc. Supercomputing 2000.

I.Foster,C.Kesselman,C.Salisbury,andS.Tuecke,“TheDataG

ArchitecturefortheDistributedManagementandAnalysisof NetworkStorageSymposium, 2000.

21

rid:Towardsan

LargeScientificDatasets,”

Proc.of

[13]A.Chervenak,E.Deelman,C.Kesselman,

L.Pearlman,andGSingh, . “AMetadataCatalogService

forDataIntensiveApplications”,GriPhyNTechnicalReport,Informa

tionScienceInstitute,August,

2002. [14]A.Chervenak,E. Deelman,I. Foster, etal.,“Giggle:A Frame Location Services,”

work for Constructing ScalableReplica

Proc.IEEESupercompuing Conference

2002 ,

[15]AChing, . A.Choudhary,W.Liao,R.Ross,andW.Gropp,"NoncontiguousI/

OthroughPVFS,"

Proceedingsof2002IEEEInternationalConferenceoC n lusterComputing [16] S. Fitzgerald, I. Foster, C. Kessenlman, G. Laszewski, W. S

mith, and STuecke, . “A Directory Service

forConfiguringHigh-performanceDistributedComputations,” PerformanceDistributed Computing

September, , 2002. Proc.6thIEEESymposiumonHigh

pp.365-375, , 1997.

[17]I.FosterandCKesselman, . “Globus:AMetacomputinginfrastruc

turetoolkit,”InternationalJournal

ofSupercomputer Applications, Vol.11, No.2,pp.115-128, 1997. [18] J. Howard, M. Kazar, S. Menees, D. Nichols, M. Satyanarayanan, R. andPerformanceiDistributed a n FileSystem,”

Sidebotham, and MWest, . “Scale ACMTransactionsoComputer f Systems

Vol6, , No.1,

pp.51-81, 1988. [19] H.Lamehamedi,B.Szymanski,Z.shentu,andE.Deelman,“DataReplicat

ionStrategiesinGrid

Environments,” Proc.Othe f FifthInternationalConferenceonAlgorithmsandA

rchitecturesfor

ParallelProcessing (ICA3PP'02), 2002. [20]J.Lee,D.Gunter,M.Stoufer,andBTiemey, . "MonitoringDataArc

hivesforGridEnvironments,"

Proc.Supercomputing 2002. [21]MLitzkow, . M.Livny,

andM.Mutka,“Condor-AHunteroIfdleWorkstations,”Proc.ofthe8th

InternationalConferenceonDistributedComputing Systems, pp.104-111,1988. [22] D. Mullmann, W. Hosckek, J. Jaen-Martinez, and BSegal, . “M Consistency in DataGrid,”

odels for Replica Synchronizationand

Proc. IEEESymp. on HighPerformanceoD n istr.Computing

[23]C.J.Pattenand K.A.Hawick,“FlexibleHigh-performanceAcce ProcoHigh f PerformanceDistributed Computing

ssto DistributedStorageResources,” 2000. ,

[24]K.RanganathanandIFoster, . “IdentifyingDynamicReplica DataGrid,”

2001. ,

tionStrategiesforaHigh-Performance

Proc.InternationalGridComputing Workshop

Denver, , November 2001.

[25]K.RanganathanandIFoster,“DecouplingComputationandDataSc

hedulinginDistributed

Data-IntensiveApplications,” Proc.ofthe11thIEEESymposiumonHighPerformanceDistributed Computing, 2002. [26] A. Samar, H. Stockinger, "Grid Data Management Pilot (GDM

P): A Tool for Wide Area Replication,"

Proc.ofInternationalConferenceoA n pplied Informatics

Feb. , 2001.

[27]R.Sandberg,D.Goldberg,S.Kleiman,D.Walsh,andBLyon, . “Desig SunNetwork Filesystem,”

Proc. ofUSENIX Conference

[28]RS . nodgrass,"ARelationalApproachtoMonitoringComplexSy

nandImplementationothe f Berkeley, , CA,1985. stem,"ACMTransactionson

Computer Systems,Vol.6,No.2, pp.157-196, 1988. [29]SpringandRWolski, . “Applicationlevelscheduling:Geneseque

22

ncelibrarycomparison,”Proccof

ACMInternationalConferenceoS n upercomputing 1998. [30]H.

Stockinger,AS . amar,BA . llock,IF . oster,KH . oltman,andB.Tierney

Replication in DataGrids,”

“, FileandOjbect

Proc. ofIEEESymp. on High PerformanceDistributed Computing

[31]HeinzStockinger,“DistributedDatabaseManagementSyst

2001. ,

emsandtheDataGrid,”

SymposiumonMassStorageSystemsand9thNASAGoddardConferenceonM

18thIEEE

assStorageSystems

andTechnologie s,San Diego,April17-20, 2001. [32] C. Sun, X. Jia, Y. Zhang, Y. Yang, and DChen, . “Achieving convergence

causality-preservation, , and

intention-preservation in real-time cooperative editing systems,” Computer-Human Interaction,

ACM Transactions on

Vol.5, No.1, March, 1998, pp.63-108.

[33]MSwany . andRWolski, . "RepresentingDynamicPerformance

InformationinGridEnvironments

withNetworkWeatherService,"Proc.ofthe2ndIEEEInter

nationalSymposiumonCluster

Computing and theGrid, May 2002. [34]RTewari, . M.Dahlin,H.Vin,andJKay, . "DesignConsideration

fsorDistributedCachingonthe

Internet," Proc.ofIEEEInternationalConferenceoD n istributed ComputingSystems [35]DThain, . J.Bent,A.Arpaci-Dusseau,R.Arpaci-Dusseau,andMLiv . Creating Communitiesfor Grid I/O,”

1999. , ny,“Gatheringathe t Well:

Proc. Supercomputing 2001. ,

[36] D. Thain, J. Basney, S. Son, and MLivny, . “The Kangaroo Appr

oach tD o ata Movement on the Grid,”

Proc. ofthe10th IEEESymposium on High PerformanceDistributed Computing,2001. [37]B.Tierney,R.Aydt,D. Gunter, W. Smith, V. Taylor, R.Wolski,and

M.Swany,"A GridMonitoring

ServiceArhitecture,"GlobalGrid Forum WhilePaper. [38]B.Tierney,W.Johnston,B.Crowley,G.Hoo,C.Brooks,andD.Gunter

,"TheNetLogger

MethodologyforHIghPerformanceDistributedSystemsPerformance

Analysis,"Proc.ofIEEEHigh

PerformanceDistributed Computing,July,1998. [39]SVazhkudai, . S.Tuecke,andI.Foster,“ReplicaSelectioninthe

GlobalDataGrid,”

InternationalWorkshop oD n ata Modelsand Databaseson ClustersandtheGrid [40]MW . ahl,TH . owes,andS.Kille,LightweightDirectory

Proc.ofthe

2001. ,

AccessProtocol,RFC2251,Internet

Engineering Task Force, 1997. [41]B.White,A.S.Grimshaw,andANguyen-Tuong, . “Grid-BasedFileAc Proc.ofHigh PerformanceDistributed Computing

cess:TheLegionI/OModel,”

2000. ,

[42]HYu . andAVahdat, . "DesignandEvaluationoaC f onit-basedContinuou Replicated Services. "

C s onsistencyModelfor

ACMTransactionsonComputerSystems(TOCS)

August , 2002.

[43]Y.Zhu,H.Jiang,X.Qin,D.Feng,andDSwanson, . "ImprovedReadPer CostEfficient,Fault-TolerantParallelVirtualFileSys

formanceinCEFT-PVFS: tem,"inproceedingof

Workshopon ParallelI/O in ClusterComputing and ComputationalGrids [44]J.Al-Jaroodi,N.Mohamed,H.Jiang,andDSwanson, . "AnAgent-Based Java oH n eterogeneous Clusters",

IEEE/ACMCCGRID May , 2003,Japan InfrastructureforParallel

In Proc. of 4th IEEE International Conference oC n luster Computing

(CLUSTER2002),Chicago,Illinois, September 2002. [45]GriPhyN Project:http://www.griphyn.org

23