A Case Study of Verifying and Validating an Astrophysical Simulation ...

The Center for Astrophysical Thermonuclear Flashes

A Case Study of Verifying and Validating an Astrophysical Simulation Code Alan Calder October 16, 2007 B. Fryxell, T. Plewa, R. Rosner, J. Dursi, G. Weirs,T. Dupont, H. Robey, J. Kane, B. Remington, P. Drake, G. Dimonte, M. Zingale, A. Siegel, A. Cacares, K. Riley, N. Vladimirova, P. Ricker, F. Timmes, K. Olson, and H. Tufo An Advanced Simulation and Computing (ASC) Academic Strategic Alliances Program (ASAP) Center at The University of Chicago

Outline

Introduction V&V methodology Verification test: isentropic vortex advection Test suite Validation test: Laser-driven shock propagating through a threelayer decreasing density target. Validation test: Rayleigh-Taylor instability. Summary, conclusions, and spear catching (a.k.a. constructive criticism)

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago

The FLASH Code

Shortly: Relativistic accretion onto NS

Flame-vortex interactions Compressed turbulence

Type Ia Supernova

The FLASH code Gravitational collapse/Jeans instability 1. Parallel, adaptive-mesh simulation code Wave breaking on white dwarfs 2. Designed for compressible reactive flows 3. Ideal, Resistive, and Hall MHD (Cartesian coords) 4. Has a modern CS-influenced architecture 5. Can solve a broad range of (astro)physics problems Intracluster interactions Laser-driven shock instabilities Nova outbursts on white dwarfs Rayleigh-Taylor instability 6. Portable- runs on many massively-parallel systems 6. Scales and performs well- Gordon Bell prize 7. Is available on the web: http://flash.uchicago.edu 8. Flash 3 beta soon to be released. Helium burning on neutron stars Magnetic Rayleigh-Taylor

Orzag/Tang MHD vortex

Cellular detonation The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago

Richtmyer-Meshkov instability

Verification and Validation

Verification: “solving the equations right” Validation: “solving the right equations” The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago

See AIAA Guide

Our V&V Methodology Choose V&V tests/problems for particular code modules e.g. hydrodynamics. Test integrated code also if possible. Verification test problems Unit tests Investigate convergence of error with resolution Investigate error in secondary modules e.g. EOS Regularly re-verify with nightly/weekly automated tests

Validation problems Quantify measurements in experiment and simulation Quantify error and uncertainty in experiment and simulation Resolution study (verification-type tests) Note: astrophysics is largely prediction The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago

Verification test: isentropic vortex


Verification Test: Isentropic Vortex

Demonstrates expected 2nd order convergence of error The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago

Verifiability: The New Test Suite Flash test suite automatically runs set of verification tests Essential for identifying bugs unintentionally introduced New test suite will be released with FLASH3 so users can monitor their own research problems and performance test parameters easily modified through GUI handles unit tests, compilation tests, comparison tests automatically uploads test suite data to benchmarks database

Green light indicates all runs were successful

Platform Date of run

Floating statistics box gives immediate overview of results Red light indicates 1 or more tests failed The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago

Test suite The steps Setup

Build

Execute

Test Result

Compilation tests build all existing applications and touch all parts of the code in limited time. Unit tests comprehensively test individual unit in isolation. Comparison tests run applications and verify the output against approved benchmarks.


Info File Manager Expandable graphic allows user to easily modify all problems in test suite

“test.info” file holds all necessary information for running a FLASH problem

Control Panel for modification of “test.info” file tree Editing of “test.info” file done right in browser


Test Suite Performance History History of 2D and 3D Sedov Problems

Details of 2D History

The new test suite adds the ability to upload selected test log files to the database to track daily code performance.


Motivation For Choice of Validation Problems Problem must test non-trivial, nonlinear behavior Validation, not Verification Problem should relate to the astrophysics of interest

Problem must have a well-documented laboratory counterpart Collaboration with National Labs (LANL, LLNL, Sandia) Collaborations with other groups

Problem must be intrinsically interesting Non-trivial problems are hard Fundamental aspect of research

Likely candidates involve fluid instabilities


Three-layer Shock Imprint Experiment

10 beams @ 420 J/beam 1 ns pulse 7 x 1014 W/cm2

Performed at the Rochester Omega laser facility Strong shock driven through a planar, copper-plastic-foam threelayer target. Design inspired by core-collapse supernovae Rayleigh-Taylor and Richtmyer-Meshkov instabilities Full details in Kane et al. 2001, Robey et al. 2001 The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago

Three-layer Target Simulation

Initial Conditions



Movie The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago


39.9 ns (early)

66.0 ns (late)

Radiographs from the experiment The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago


Simulated radiographs at same times



Length of Cu spike



Convergence results: percent difference from most resolved The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago


Comparison to Experiment The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago





Shortcomings: Incomplete Physics

Simulations used a gamma-law EOS, P = (γ –1)ρε, with choice of gamma to match experimental result Periodic boundary conditions on sides- no shock tube in the simulations Radiation deposition mechanism not included in the simulations Experimental diagnostics do not allow us to determine the correct amount of small scale structure. Radiograph is a 2-d shadow of a 3-d object


Summary of Results Verification tests: Pass! Test suite critical!!! Validation test: Three-layer targets- Good agreement with experiment Incomplete physics Additional work won’t improve astrophysical simulations Good collaboration with experimentalists is essential! Access to experimental results and error/uncertainty assessment Benefits theorists and experimentalists Increased confidence in results We are learning how to establish the limits of validity of Flash We are establishing a methodology for systematic comparisons between experiments and simulations


Rayleigh-Taylor Instability in a Type Ia Supernova


Multi-mode Rayleigh-Taylor “α-Group” Consortium Organized by G. Dimonte (Oct. 1998) Purpose – to determine if the t2 scaling law holds for the growth of the RT mixing layer, and if so, to determine the value of α simulation - experiment comparisons inter-simulation comparisons hb,s = αb,s gAt2, where A = (ρ2 - ρ1)/ (ρ2 + ρ1) Definition of standard problem set (D. Youngs) Dimonte et al. Phys. Fluids 16 1668 (2004)


Rayleigh-Taylor Instabilities Density schematic:

Multi-mode velocity perturbation:

Denser fluid g

Lighter fluid

“Toughest @#%$^* simulation problem one can imagine” -Leo Kadanoff

2.5-5 % sound speed with highest magnitude near the interface


Aside: Mesh Adaptivity and R-T Instability

AMR allows an increased range of scales in a simulation by adding resolution where it is needed. RTI increases the area of the flame, thereby boosting the burning rate.

Calder, et al.


Multi-mode Rayleigh-Taylor: 3-d Simulation

Horizontally Averaged Density

Modes 32-64 perturbed


Multi-mode Rayleigh-Taylor: Inverse Cascade Bubbles of the lighter fluid in the denser fluid

t = 7.00 sec

t = 14.75 sec


Multi-mode Rayleigh-Taylor

Rendering of Mixing Zone

Density (g/cm3) at t = 14.75 sec The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago

Multi-mode R-T Experimental LIF Image

It looks similar to the simulation….. The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago

Multi-mode R-T Simulated LIF Image

It looks similar to the experiment….. The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago

Multi-mode Rayleigh-Taylor FLASH Simulation

Are we adequately resolved?


Multi-mode Rayleigh-Taylor FLASH Simulation

αspike = 0.026 αbubble = 0.021


Multi-mode Rayleigh-Taylor Experiment

αspike = 0.058 αbubble = 0.052


Rat “Pill”: Single-mode 3-d Rayleigh-Taylor

4

8

16

32

64

128

λ (grid points) The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago

256

Density (g/cc) t = 3.1 sec

Single-mode 3-d Rayleigh-Taylor

4

8

16

32

64

128

λ (grid points) The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago

256

Abundance t = 3.1 sec

Bibliography Flash Code: Fryxell, et al., ApJS, 131, 273, 2000 Calder, et al., Proc. SC2000 Antypas, et al. accepted to PDPTA ‘06 Validation: Calder, et al., ApJS 143, 201, 2002 Calder, et al. CiSE 6, 10, 2004 Weirs, et al. Ap&SS 298, 341, 2005 Weirs, Plewa, Dupont et al. submitted soon! Hearn, et al. in progress Related field: www.hedla.org


A Case Study of Verifying and Validating an Astrophysical Simulation ...

A Case Study of Verifying and Validating an Astrophysical Simulation ...

Suggest Documents