Bryan Lawrence on behalf of the NDG,. BADC and BODC. Ray Cramer, Marta
Gutierrez, Kerstin Kleese, Siva Kondapalli, Sue Latham,. Roy Lowry, Kevin O'
Neill ...
NERC DataGrid: Googling for Secure Data
Bryan Lawrence on behalf of the NDG, BADC and BODC. Ray Cramer, Marta Gutierrez, Kerstin Kleese, Siva Kondapalli, Sue Latham, Roy Lowry, Kevin O’Neill, Ag Stephens, Andrew Woolf
British Atmospheric Data Centre http://badc.nerc.ac.uk
Outline • • • •
NDG Aims and Metadata Taxonomy Demonstration of NDG in action NDG Authorisation – the security bit! Status
British Atmospheric Data Centre http://badc.nerc.ac.uk
Timelines & Bottom Line • 2002: E-science arrives at NERC: – Legacy Systems with millions of files and terabytes of data and existing access and authorisation systems that cannot easily be replaced. – Complex existing DISCOVERY metadata systems. – Discovery (where it exists) based on Z39.50 – Utilisation based on file retrieval.
• 2004: NERC DataGrid ready to move forward – New metadata systems describe data as well as datasets. – OAI based harvesting supports scalable FAST data discovery. – Requirements capture for new authorisation systems complete, and coding underway for implementation. – New communities involved, and international discovery very close to operational reality.
• 2005: – Utilisation based on metadata, on demand server side behaviours, grid-based back end parallelisation etc British Atmospheric Data Centre http://badc.nerc.ac.uk
Complexity + Volume + Remote Access = Grid Challenge British Atmospheric Data Centre
Simulations
British Oceanographic Data Centre Assimilation British Atmospheric Data Centre http://badc.nerc.ac.uk
http://ndg.nerc.ac.uk
NDG Metadata Taxonomy
British Atmospheric Data Centre http://badc.nerc.ac.uk
NDG Metadata Architecture
Service based model: • clear separation between discovery and use • discovery service standards compliant and interoperable British Atmospheric Data Centre http://badc.nerc.ac.uk
(D) - Discovery Open Archives Initiative – Digital Library Protocol for harvesting metadata. OAI
OAI
NDG Supports Multiple Discovery Services – “build your own”
Multiple Protocol Support will be built into the “NDG Vanilla Discovery Service” Existing Metadata
Intermediate Schema Document(s) (XML)
XSLT Ingest Transformation
NDG Discovery Service Element
British Atmospheric Data Centre http://badc.nerc.ac.uk
XSLT Processor
Directory Interchange Format
XSLT Processor
Dublin Core
XSLT Processor
GEO Profile (Z39.50)
ISO 19115?
Catalogue Interoperabiltiy Protocol ?
Internet Link
tape robot
Online Data
XML database BADC NDG Wrapper
Online Data
Online Data
XML database
XML database
BODC NDG Wrapper
Group NDG Wrapper
Wider Internet NERC Grid Software Agent
Grid User
ESG (&other) Applications
Satellite
Research Group Data Sources
Wider Internet
NDG Web Portal
Internet User
Internet Link
XML database
British Atmospheric Data Centre http://badc.nerc.ac.uk
Supercomputer
Discovery
British Atmospheric Data Centre http://badc.nerc.ac.uk
Choose to go to A service or B service.
Can order responses by title or data centre (or default random)
Flexible Information Return
British Atmospheric Data Centre http://badc.nerc.ac.uk
Look at DIFs in either HTML or XML
Current Interface
British Atmospheric Data Centre http://badc.nerc.ac.uk
British Atmospheric Data Centre http://badc.nerc.ac.uk
British Atmospheric Data Centre http://badc.nerc.ac.uk
British Atmospheric Data Centre http://badc.nerc.ac.uk
British Atmospheric Data Centre http://badc.nerc.ac.uk
British Atmospheric Data Centre http://badc.nerc.ac.uk
British Atmospheric Data Centre http://badc.nerc.ac.uk
British Atmospheric Data Centre http://badc.nerc.ac.uk
British Atmospheric Data Centre http://badc.nerc.ac.uk
British Atmospheric Data Centre http://badc.nerc.ac.uk
British Atmospheric Data Centre http://badc.nerc.ac.uk
British Atmospheric Data Centre http://badc.nerc.ac.uk
British Atmospheric Data Centre http://badc.nerc.ac.uk
British Atmospheric Data Centre http://badc.nerc.ac.uk
Background activity being parallelised with GODIVA/CCLRC e-science collaboration (spectral -> gridpoint + CDMS + visualisation tools)
Download either plot or the data that went into the plot. British Atmospheric Data Centre http://badc.nerc.ac.uk
British Atmospheric Data Centre http://badc.nerc.ac.uk
International Dimension
British Atmospheric Data Centre http://badc.nerc.ac.uk
Southampton Oceanography Centre
British Atmospheric Data Centre http://badc.nerc.ac.uk
Authorisation • Role-based access:
Signed “conditions of use” form exists for this dataset
badc.nerc.ac.uk ukmo-obs researcher ukmo-obs nerc
• Key concept: Only hosts that trust each other share data, even within a larger virtual organisation: e.g. at BADC: ndg.bodc.nerc.ac.uk nerc ashoe nerc bodc British Atmospheric Data Centre http://badc.nerc.ac.uk
NDG Security
Certificate based, pass encrypted credentials between user and gatekeeper.
British Atmospheric Data Centre http://badc.nerc.ac.uk
Where are we? • Migration to web services underway for some components, new A services in design phase, implementation details not yet obvious (e.g. GT4 etc). • Major effort on defining feature types for observation types so we can build an OGC/ISO compatible data extractor for observations and numerical data. • Security Infrastructure Development – Collaboration with CCLRC e-science, ECOGrid
• Ongoing work on metadata definition and population: – Oceanographic data – Atmospheric Chemistry data • Major issues with (un)controlled vocabularies
– Numerical Modelling data • DIF numerical definition (moving to ISO), BADC and UK Community • Katherine Bouton’s work at NCAS/CGAM (“B” MODEL METADATA)
– Remote Sensing Data • Collaboration with NEODC and PML
• Ongoing work on databases and interfaces, DIF to ISO and “B” British Atmospheric Data Centre http://badc.nerc.ac.uk
