Progress of Parallel Validation Tools for Fusion Simulations as Applied to Synthetic Diagnostic Efforts http://www.txcorp.com
S. Vadlamani, S. Kruger, S. Shasharina, M. Durant, C. Holland, J. Candy
[email protected]
Interoperability requires a common data format • Possible data formats • ASCII (E.g., geqdsk) • Advantages: No library dependency, can be relatively selfdescribing • Disadvantages: Doesn’t work with viz tools. Each reader has to be hand-written.
• Home-grown binary format • Similar to ASCII but takes less space. Agreement on data locations is required
• Self-describing binary – three worth considering • netCDF3, netCDF4 • HDF5 • For interoperability efforts, agreement on their use is widespread Agreement on names/standards is required
Differences among selfdescribing format • netCDF3 – Fairly simple build, widely used, relatively simple interfaces – Now deprecated, although maintained as part of netCDF4 – F90/C bindings are good. Python bindings are poor
• netCDF4 – Can use either netCDF3 or HDF5 as back ends – Can be as flexible as HDF5, but has better interfaces – Still new enough that build issues are problematic
• HDF5 – – – –
Very flexible: hierarchial, rich attributes, … good parallel I/O Complex API although has improved Excellent python support through tables
One Cannot Write One Viz Reader For All HDF5 data • HDF5 consists of groups (like directories), datasets (like files – end leaves) and attributes (for small data and metadata) • One can organize data in many ways using HDF5 constructs • One could use any kinds of names and no metadata at all • How one can understand what is what? • • • •
What is supposed to be visualized? What order is used (row-major etc)? Where is the mesh of the data? What are the components of the mesh?
VizSchema Is An Attempt to Standardize HDF5 for Viz
• Based on experience working with
• VORPAL (uniform and unstructured meshes and particles) • NIMROD (structured meshes) • UEDGE (structured, multi-domain meshes) • TXFLUID (unstructured meshes) • FACETS (combination of the above)
• VizSchema • Data Model (agreement about data organization and metadata) • C++ reader (independent of Viz tool) of HDF5 data into in memory viz objects • VisIt plugin
• Funded by FSML grant (DOE SBIR, FES), FACETS grant (DOE SciDAC) and Tech-X Corporation
VizSchema: Minimalistic but Enough for Viz • Viz entities: • • • •
Variables (live on external mesh) Variables with meshes (spatial info is mixed in) Meshes Derived variables
• Metadata • Identifying the entities and specifying their kinds (if any) • Providing information needed for minimal viz
• Principles • Metadata is minimal (and results of many friendly discussions) • Metadata is in attributes starting with “vs” • Groups and datasets names are not regulated
Variables Need to State Type, Centering (and Ordering) Dataset "phi" { Att vsType = "variable” // Register as something // to “look at” and look for vsMesh Att vsMesh = "mycartgrid”// This should exist //somewhere in the file Att vsCentering = "zonal”// Nodal is default }
Everything Is 1D Array – Which Order? • Start with fastest varying (3D): compMinorC compMinorF compMajorC compMajorF
= = = =
(i0, (ic, (ic, (i2,
i1, i2, i0, i1,
i2, i1, i1, i0,
ic) i0) i2)// Same as compMinorF for 1D ic)// Same as compMinorC for 1D
• CompMinorC in 2D: Bx(0,0),Bx(1,0),Bx(2,0),Bx(0,1),Bx(1,1),Bx(2,1), By(0,0),By(1,0),By(2,0),By(0,1),By(1,1),By(2,1)
• One needs to use VsOrder attribute for datasets in variables, variables with meshes and meshes to specify this (compMinorC is default)
Meshes Specify Kinds And Things to Build Itself • Structured grid has just a list of points • Need vsOrder (if not default) • Rectilinear mesh has list of numbers in each direction • Uniform mesh has number of points in each direction, max and min in each direction • Unstructured mesh needs points and elements (polygons, quads etc) • The list of meshes will grow • Work with users to provide alternative ways to describes same kinds of meshes
To use VizSchema • APIs exist but were developed independently rather than being derived from single code base • C++ (part of txBase: Sides, Miah, Hakim, …) • F90 (vsHdf5: Kruger, Cary, Vadlamani) • Python (vsPy: Cary, Kruger, …) • VisIT (vsHdf5 plugin: Durant, Shasharina, Cary, …) • Advantage: • Developed to feel natural and easy to use in each language • Disadvantage: • Features are not uniform across APIs but converging • Methods may not be uniform (not too onerous because one rarely finds a person who wants to know both APIs) • Convergence is occuring
Proposal for standards is to extend attributes • Allow codes to dump data into HDF5/netcdf into their native format • Minimize dependencies to I/O libraries themselves if desired. • Can use lightweight vsLibraries for their language to have easier API to use • For HDF5, python with tables can be used to add attributes later (adding attributes is fast, takes up almost no space)
VizSchema is integrated into GYRO • GYRO trunk builds a separate library libvshdf5.a which • holds fortran overloaded wrappers to HDF5 calls, • complies to VizSchema standards. • Overloading of “dump” to hdf5 routines makes it easier for the user. • Same “dump_h5” name for scalars as well as 4D arrays • HDF5 attributes (sans data) passed as derived type to “dump_h5”
GYRO data stored in course and fine resolution hdf5 files. • gyro_profile.h5 contains simulation parameters and other shot specific time invariant quantities: • Geometric factors • gyro.h5 contains coarse grid and integrate data • fluxes, spectra,modal • gyrofine.h5 • interpolation • toroidal planes • gyro3D.h5 • real quantities
Electron density of in linear phase of a case used by C. Holland for synthetic diagnostic studies.
VisIT allows for ease of merging data
• BES channel locations obtained from experimentalists for particular shot
Future Work • Integration of VizSchema compliant data dump methods in the CU-GEM code. • Construct python-based workflow for synthetic diagnostic analysis • Perform verification studies with GYRO and GEM using synthetic diagnostic derived quantities.
Information • VizSchema information and download: https://ice.txcorp.com/trac/vizschema/wiki/ WikiStart • VizSchema is a standard VisIt plugin.
