Development of verification and validation of CFD predicted results in fire engineering will be reviewed in this ... Organization for Standardization ISO have.
International Journal on Architectural Science, Volume 5, Number 3, p.58-67, 2004
‘VERIFICATION AND VALIDATION’ COMPUTATIONAL FLUID DYNAMICS
IN
MODELING
FIRE
BY
W.K. Mok and W.K. Chow Department of Building Services Engineering, The Hong Kong Polytechnic University, Hong Kong, China (Received 1 September2004; Accepted 16 September 2004)
ABSTRACT Computational fluid dynamics (CFD) is an important tool in fire safety engineering. CFD is applied in many projects while the global trend is moving towards the implementation of engineering performance-based fire codes. However, questions are always raised on the model credibility. Verification and validation of the CFD predicted results in fire engineering are not yet carried out with systematical full-scale burning tests. Development of verification and validation of CFD predicted results in fire engineering will be reviewed in this paper. Sources of error commonly encountered in modeling fire will be outlined. Three international standards on the assessment of predictive capability of fire models are reviewed and compared. Based on the study, eight methods for verifying and validating CFD predicted results are suggested on: • • • • • • • •
Checking the completeness of technical documentation. Checking the scientific ground by independent experts. Source code checking by independent experts. Analytical solution comparison. Benchmark fire code comparison. Grid size and time step refinement exercise. Monitoring the residual error of governing equations. Validation with experimental results.
1.
INTRODUCTION
The application of computational fluid dynamics (CFD) is an important fundamental tool in solving fire engineering problems, especially when physical experiments are difficult to carry out. Quantitative information on fire spreading would be predicted [1,2]. Worldwide movement towards the implementation of performance-based fire codes has led to more popular use of CFD in the construction industry. The results are useful to design appropriate fire safety provisions for the possible fire scenarios [3]. However, a common question within the community of fire engineering is on how good are the CFD predicted results [1,2]. A detailed analysis quantifying the modeling and numerical uncertainties in those fire simulations is therefore required. The credibility, quality, uncertainties and limitations of the simulated results are definitely one of the most crucial research areas for fire protection engineers, CFD modelers, government officials and even facility managers who might not be good at mathematics.
Within the last decade, the quality or credibility of CFD fire simulations has drawn an increasing attention from the society of fire engineering. However, there is little agreement on procedures for assessing the credibility of fire simulation codes [4]. For example, the American Society for Testing and Materials ASTM and International Organization for Standardization ISO have established their own standard on the governing of the credibility of CFD models. A guide was also published by the American Institute of Aeronautics and Astronautics for the verification and validation of CFD simulations [4]. Different standards or polices are written up for determination of credibility of CFD codes. At the moment, there is no universal agreement within the community of fire engineering.
2.
SOURCES OF ERROR CFD SIMULATION
WITHIN
Most CFD fire codes generally work in the following sequence: a scientific model of reality, a mathematical model of the scientific model, a discrete numerical model of the continuous
58
International Journal on Architectural Science
mathematical model, an application-program model of the numerical model, and a computational model of the application model. Errors are piling up as each model in this list is an approximation to its predecessor. In addition, all computers do their operations with limited precision, and the number of significant digits degenerates as operations are performed. As a result, errors and uncertainties are accumulated. A graphical representation of the processes of modelling was constructed by a technical committee of the Society for Computer Simulation (SCS) chaired by Schlesinger [5] in 1979, as shown in Fig. 1. CFD essentially can be regarded as a composition of two types of models: a conceptual model and a computerized model. The conceptual model is derived from the fundamental theories of fluid dynamics by analyzing and observing the physical system. They are presented in the form of mathematical equations with initial and boundary conditions. In fire simulation, the conceptual model is dominated by the system of partial differential equations (PDEs) on the air flow variables derived from conservation of mass, momentum, energy, turbulent parameters and combustion products (if any). The computerized model is the computational version of the conceptual model using numerical techniques and discrete mathematics in order to facilitate a solution to the modelled real problem.
The accuracy of the simulated results can be affected by the uncertainties in the CFD simulation. Different uncertainties can be generated in either the conceptual modelling or during the computational design phase, which should be measured or bound for the error. The credibility of CFD simulated results for design purposes is strongly dependent on whether the uncertainties are identified and quantified, irrespective of their sources. Reduction of uncertainties would mean better representation of the real problem and thus increasing the confidence on the use of CFD simulation codes. CFD simulation can be described by two models: a conceptual model and a computerized model; likewise, the sources of uncertainties can also be classified into two types. These are uncertainties arising due to the modelling of physics such as a misunderstanding of the phenomena leading to falsifying assumptions, and uncertainties due to the computational design such as from the approximations and simplifications about the parameters that govern the fluid dynamics. A detailed analysis quantifying the modelling and numerical uncertainties in the simulation is therefore required. Verification and validation (V&V) are believed to be the primary means for building and quantifying the credibility of a CFD simulation [6].
3.
VERIFICATION AND VALIDATION
Although verification and validation are the common interest of different organizations and societies, different meanings are used in the various technical disciplines. Operations research (OR) community is likely to be the first group of organization to discuss the term “verification and validation” [7]. As the simulations in OR field cover a wide range of interests including world politic models and nations conflict models, there are studies on the accuracy of simulated results. The definitions of V&V by SCS [5] are: Fig. 1: Phases of Modeling and Simulation and the Role of V&V (from Schlesinger et al. 1979) CFD model can be taken as a simulation tool. The results of such simulations might not be reality. However, CFD code can be regarded as a reliable or acceptable method if the numerical solution produced by it can accurately represent the reality within a given accuracy under the limit of its applicability. In other words, the accuracy and the limit of applicability of CFD code are vital data to be determined.
59
•
Verification: Substantiation that a computerized model represents a conceptual model within specified limits of accuracy.
•
Validation: Substantiation that a computerized model within its domain of applicability possesses a satisfactory range of accuracy consistent with the intended application of the model.
In late 1970s, the application of computercontrolled systems was increasing and penetrating into the commercial sectors and public life.
International Journal on Architectural Science
General definitions of V&V provided by the Institute of Electrical and Electronics Engineers (IEEE) [8,9] are: •
•
Verification: The process of evaluating the products of a software development phase to provide assurance that they meet the requirements defined for them by the previous phase.
The first international standard on evaluating the predictive capability of deterministic fire model was published by the American Society for Testing and Materials (ASTM) [14] in 1997. The definitions of V&V designated for fire modeling are introduced as: •
Verification: The process of determining the correctness of the solution of a system of governing equations in a model. With this definition, verification does not imply the solution of the correct set of governing equations, only that the given set of equations is solved correctly.
•
Validation: The process of determining the correctness of the assumptions and governing equations implemented in a model when applied to the entire class of problems addressed by the model.
Validation: The process of testing a computer program and evaluating the results to ensure compliance with specific requirements.
Both the American Nuclear Society (ANS) and the International Organization for Standardization (ISO) adopt the same definitions by IEEE in the Guidelines for the Verification and Validation of Scientific and Engineering Computer Programs for the Nuclear Industry [10] and ISO 9000-3: Part 3: Guidelines for the Application of ISO 9001 to the Development, Supply and Maintenance of Software [11] respectively. The Defense Modeling and Simulation Organization (DMSO) of the U.S. Department of Defense (DoD) [12,13] is one of the most active institutes working on the field of V&V. Clear and concise definitions are provided: •
Verification: The process of determining that a model implementation and its associated data accurately represents the developer’s conceptual description and specifications.
•
Validation: The process of determining the degree to which a model and its associated data are an accurate representation of the real world from the perspective of the intended uses of the model.
The American Institute of Aeronautics and Astronautics (AIAA) is another U.S. organization that plays an active role on V&V development. Definition by DoD was adopted with some modification on the accuracy of the numerical solution, which is essential in the arena of CFD [4]: •
Verification: The process of determining that a model implementation accurately represents the developer’s conceptual description of the model and the solution to the model.
•
Validation: The process of determining the degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model.
In 1999, ISO published a standard on the assessment of totally deterministic CFD fire modeling [15]. Based on the work by ASTM, modification was mainly made on the definition of verification, whereas the definition of validation was kept the same as by ASTM. •
Verification: The process of checking a mathematical fire model for correct physical representation and mathematical accuracy for a specific application or range of applications. The process involves checking the theoretical basis, the appropriateness of the assumptions used in the model, that the model contains no unacceptable mathematical errors and that the model has been shown, by comparison with experimental data, to provide predictions of the course of events in similar fire situations with a known accuracy.
•
Validation: The process of determining the correctness of the assumptions and governing equations implemented in a model when applied to the entire class of problems addressed by the model.
Amongst most of the standards and guidelines discussed above, AIAA, ASTM and ISO are developing the processes of V&V for the credibility of simulation models based on the aims, which match with the field of fire CFD modeling the most, especially for ASTM and ISO. There are some important implications in the definitions of V&V to be summarized in general. Verification does not assert whether the solutions from a code are physically valid or not, but only determines the level of accuracy achieved by a code through
60
International Journal on Architectural Science
purely numerical exercises. Validation is the process of estimating the level of accuracy of a CFD simulation by comparison with experimental data and provides evidence for how accurately the computational simulates reality. Currently, the implementation of verification and validation (V&V) for credibility quantification in CFD fire modeling is in the preliminary stage. V&V is still not solidly planted in the work of CFD modelers. Some available CFD fire codes do not go through cautious tests of V&V, their work can only be regarded as just a demonstration that some special fire scenarios are probably simulated. This situation persists in the arena of CFD fire modeling, especially for the sector of CFD commercial codes [16]. One of the possible reasons is that there is no commonly accepted V&V methodology available, although at least three different standards on the credibility of CFD have been published by ASTM, AIAA and ISO.
4.
The theoretical basis of the fire models is recommended to be reviewed by one or more independent experts. The focuses should be on the appropriateness of scientific assumptions and approximations, mathematical approaches, and the application of empirical constants and default values being used in CFD code. •
The computational fire model should be verified so that the system of governing equations is solved correctly in accordance with the user manual. This verification process can be performed by analytical tests, code checking and numerical tests. Besides, special attention should be paid to stiff problems, which is one of the numerical problems arising from the implementation of substantially different time scales of different physical processes, such as combustion, conduction and so on, in the same CFD fire code.
STANDARD GUIDE FOR EVALUATING THE PREDICTIVE CAPABILITY OF DETERMINISTIC FIRE MODELS
Analytical test is a powerful method to verify the correctness of a numerical code by comparing the solution of the CFD fire code to a situation which an analytical solution is known.
The standard guide developed by ASTM describes the process for evaluating the predictive capability of fire models and intends to provide invaluable guidance for fire model developers, fire model users, approving officials and educators.
Code checking is a rather time-consuming process, in which an independent CFD modeler goes through the fire code line by line to check for irregularities and inconsistencies.
The predictive capability of fire models is recommended to be assessed on four areas [14]: •
Defining the model and scenarios for which the evaluation is to be conducted One of the major sources of error comes from the unintentional misuse of fire model, hence, sufficient and careful documentation will be necessary. It should not only cover the description of scientific basis of the fire models and accuracy of computational procedures, but also thorough description of the scenarios of interest in the evaluation to facilitate appropriate application of the fire models. From the Standard Guide for Documenting Computer Software for Fire Models E 1472 – 92 (Reapproved 1998) [17], a complete guidance is given for writers of technical document, user manuals and installation, maintenance and programming manual.
•
61
Verifying the appropriateness of the theoretical basis and assumptions used in the model
Verifying the mathematical and numerical robustness of the model
Numerical test is focusing on the numerical error of CFD fire code. In fire modeling, the phenomenon is usually expressed in the form of higher-order differential equations. Such set of equations is normally solved by numerical techniques for finding approximate solutions. Hence, by measuring the magnitude of the residuals from the solution of the governing equations, a vital indicator of numerical convergence is achieved. And it is the most common method employed for model verification purpose. •
Quantifying the uncertainty and accuracy of the model results in predicting the course of events in similar fire scenarios Uncertainty of a fire model comes from different sources of error, which can be simplified as model uncertainty and experimental uncertainty. Model uncertainty refers to the error of the model inputs, which
International Journal on Architectural Science
may be based on experimental measurements, empirical correlations or estimation by engineering judgment. A detailed description has been provided by ASTM in the Standard Guidance for Obtaining Data for Deterministic Fire Models E 1591 – 00 [18]. Sensitivity analysis is recommended for judging this type of uncertainty. A sensitivity analysis of a fire model is a study of how changes in model parameters affect the results generated by the model. Model predictions may be sensitive to different sources of error, such as the uncertainties in input data, the sophistication of the modeling to relevant physics and chemistry, the accuracy of numerical treatments and so on. Sensitivity analysis is a method to assess the extent that the uncertainty in model inputs contributes to the uncertainty in the results of interest from the fire model. Experimental uncertainty is seldom considered by CFD modelers and is always regarded to be the job of experimentalists. Actually, it is part of the validation process and takes an important role in the assessment of credibility of a CFD fire model. Experimental data is the approximation or estimation of on-site measurement. The accuracy of experimental result depends on the quality of measurement and should be quantified in association with the experimental results. After ensuring the correct model inputs, appropriate selection of scenarios and correct mathematical methods, CFD fire model evaluation can be made by comparison with one or more of the following tests or data: standard tests, full-scale test conducted specifically for the chosen evaluation, previously published full-scale tests data, documented fire experiences and proven benchmark models. However, the standard guide [14] concludes that there is no universal quantification method which can represent the comparison with the tests for most of the fire models.
5.
GUIDE FOR THE VERIFICATION AND VALIDATION OF COMPUTATIONAL FLUID DYNAMICS SIMULATIONS
This guide is established by AIAA and intended for assessing the credibility of not only the fire modeling but also a wide variety of computational fluid dynamics. The assessing process is very
much emphasized to work in two different stages, which are verification and validation. Verification is the evaluation stage to determine that a CFD model accurately represents the conceptual model. The errors in the computational model and its solution are identified and kept within an acceptable level. According to the Guide for the Verification and Validation of Computational Fluid Dynamics Simulations [4], there are four major sources of error in CFD modeling: insufficient spatial discretization convergence, insufficient temporal discretization convergence, lack of iterative convergence and computer programming. For the first two types of errors, insufficient spatial discretization convergence and insufficient temporal discretization convergence, systematic refinement of grid size and time step are useful tools to estimate the associated errors. Theoretically, the discretization error should asymptotically approach to zero, except the computer round-off error, if the grid size and time step are reduced to a minimum. And the constant relationship between the order of accuracy and the reduction of grid and time step within the asymptotic region can be a proof for both grid and time-step convergence in the numerical scheme of CFD codes. Besides, grid and time-step refinement exercise are important methods to locate boundary condition discretization errors and programming errors. However, grid and time-step reduction are also time-consuming and demand much computer resources. Iterative convergence is a common problem for CFD simulations which iteration is required for solving the set of governing equations. Fire modeling is one of the typical examples. It is quite common to set a relative tolerance, say the preset value. If the difference between the solutions of successive iteration steps at each point is lesser than the preset value, the numerical scheme is defined to be iteratively convergent. In the recommended verification test, a sensitivity test can be set up between the magnitude of the convergence criteria and the solution of the CFD model to find out the most consistent value for controlling the convergence performance of the iterative numerical scheme. Residual error is another useful control method for iterative convergence. By calculating the difference between the left and right sides of the governing equations, residual error can be found for each iteration at each grid point. The summation of residual error of all grid points throughout the whole computational domain at the beginning is taken as the reference. By comparing
62
International Journal on Architectural Science
the summation of residual error, the convergence performance can be determined. Consistency check is recommended to be used in verification test. However, cautions are required when applying it to unsteady flows, such as air flow induced by fire. Finally, the most accurate and reliable method to quantify the error in the computational solution of CFD is the comparison of the highly accurate solution, which can be classified as analytical solutions and benchmark numerical solutions. However, for the case of fire modeling, analytical solution is not easy to be achieved due to the highly complex nature in the governing partial differential equations. Benchmark PDE solutions are considered to be a possible way for fire modeling. This method can produce very accurate numerical solutions to some special and relatively simple cases. By applying CFD model to the same tailormade situation, the simulated result and the calculated data can be compared to verify the credibility of CFD code. Validation is the final phase of the credibility process of CFD models and is interpreted as the stage to determine the degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model. Within the guide [4], the real world can be regarded as a complete engineering system. A direct comparison of the complete system to the simulated result of CFD model is so complicated and not reliable as the data taken during experiments are very limited and usually associated with a fairly high degree of uncertainty. This problem can be minimized by dividing the complete engineering system into three progressively simpler phases: subsystem cases, benchmark cases and unit problems. Subsystem cases are the first level of decomposition of the complete engineering system. They are characterized by fewer flow phenomena and simpler geometric feature. For this kind of situations, large experimental uncertainty is expected. Besides, limited initial conditions and boundary conditions can be measured. Benchmark cases are the further decomposition of the complete engineering system from the subsystem cases. Usually, not more than two separate features of the flow physics and two flow features are found in benchmark cases. Due to the simpler conditions, most of the initial and boundary conditions can be measured with relatively moderate uncertainty.
63
Unit problems are the simplest form of the complete engineering system. They possess few special features and can be monitored by special tailor-made hardware with high precision. Simple geometries and one dominant flow feature are the main characteristics which enable accurate measurement of all vital boundary and initial conditions. At this fundamental stage, verification process, such as grid and time-step refinement exercise, iterative convergence and residual error check, can be performed with relatively less computational resources. The reliability of experimental data is discussed. The uncertainty of the measured data is heavily influenced by the complexity of the flow condition. In order to enhance the reliability of validation, the complete engineering system can be broken down into a number of unit problems by means of a systematic approach.
6.
ASSESSMENT AND VERIFICATION OF MATHEMATICAL FIRE MODELS
This standard targets the deterministic fire models and provides guidance on the assessment of the accuracy of predicted results. The coverage is divided into five areas: documentation of CFD fire model, methodology of error checking and testing, assessment of numerical accuracy and stability, uncertainty of experimental measurements and the use of sensitivity analysis. At the beginning of the standard, quite a lengthy discussion is made on the requirement of documentation for the CFD fire model. The ASTM standard [14] is taken as the primary source of information in this part, the contents of technical documents and user manual are clearly and concisely listed. This move represents that both institutes share the same point of view that a well-organized documentation is not only the first but important information for the assessment of the accuracy of CFD fire model. In respect of the general methods for errors checking and testing, the common sources of error are proposed and summarized as the following [15]: •
The use of inappropriate algorithms or wrong physics to describe the fire processes and sub-processes that are being modelled;
•
The use of incorrect or unsubstantiated constants or default values;
•
The omission of (sub)-processes in describing the development of a fire (this is essentially that the model oversimplifies the phenomena which it is attempting to represent);
International Journal on Architectural Science
•
The use of inappropriate numerical algorithms to solve the equation set(s) that result from the application of algorithms to describe the (sub)-processes;
•
Errors in the computer code.
Independent experts are recommended to review the theoretical basis of the fire model for the appropriate application of fire sciences theory and completeness of all technical documentation. If the open fire code is available, it is always useful to conduct line by line code checking for irregularities and inconsistencies. For some special cases having a known mathematical solution, invaluable information on the accuracy of CFD fire model can be quantified by carrying out comparison between the simulated result and the analytical solutions. Unfortunately, cases with analytical solutions are seldom available, especially for the field of fire engineering. One of the suggested alternatives is benchmark fire code. If a fire code is well recognized and validated, the simulated result can be used for comparison purpose. Empirical verification means the direct comparison between the simulated result and the measured data in an experiment. It seems to be the most confident way to work out the credibility of a CFD fire model. However, this verification process will easily be a trap to mislead the CFD modelers and users, when the uncertainties in the measurements are not known. Numerical accuracy is another major concern for CFD fire modelling. General numerical methods are introduced with explanation of common sources of error, such as stiff problem, round-off error and so on. Since the numerical convergence is caused by both the original mathematical model and the method of discretization, no universal method is concluded for checking the consistency and stability in every case in this guide. Several possible methods are proposed. The rate of convergence can be an indicator of numerical convergence. It can be done by plotting the error against different time step. If direct proportion relationship is achieved, the numerical method can be said to be consistent. Besides, examination of the sensitivity of the solution to grid refinement is also proposed. Residual error for each of the governing equation solved is suggested to be employed as a check for numerical convergence. This calculation can be elaborated to the overall mass and energy balances for the whole computational domain to form additional monitoring indices.
Measurement uncertainty of data is not only an issue of experimenters but also the CFD fire modelers and users. Experimental data taken for validation purpose is subject to the uncertainty of measurement. Most of the content in this part is taken from Taylor and Kuyatt [19]. A simple description on sensitivity analysis is made in this section. A well-structured sensitivity analysis can provide useful information on the acceptable range and sensitivity of values for each input variable, the dominant variables and deterministic parameters, and the identification of the level of care to be taken in selecting inputs and running the model. Sensitivity analysis can also be applied to experiment, although large resource input is expected.
7.
DISCUSSION
Three international standards on the assessment of credibility of fire models are reviewed and discussed. Although not all of the standards are aimed at CFD fire modelling, essential guidance can be drawn and learnt. By analyzing the suggestions proposed in the standards, some fundamental but essential rules can be extracted to form the eight basic means to determine the credibility of CFD fire model. •
Technical Documentation Checking
As mentioned in both standards of ASTM and ISO, technical documentation is the most important tool for CFD model users. Misuse of CFD fire package, due to improper input and invalid scenario, is one of the major sources of error. This error can simply be erased by a well-structured user manual or technical documentation. ASTM and ISO have produced excellent guides on the preparation of technical documentation of CFD model [14,15,17]. •
Scientific Ground Checking
Scientific ground checking is quite heavily dependent on the quality of technical documentation, especially for commercial CFD fire package. As the source code is not available, the theories are hindered. The corresponding published papers from well-recognized journals or conferences will help the work a lot. •
Code Checking
Code checking is an exhausting job but can be very helpful. Quite a lot of disastrous errors come from the typing mistakes and program loops other than embedded numerical errors and deviated scientific theories. This area is also the weakest part of CFD
64
International Journal on Architectural Science
fire modelers. Within the last few decades, the size of CFD fire code is increasing dramatically due to additional sophisticated modelling to real fire situations such as turbulence, combustion, radiation, fire spreading and so on. A normal CFD fire code may be made up of more than ten thousand lines. As the complexity is blowing up, any single change to the code could potentially cause havoc in large fragments of the simulation leading to errors. To prevent this, quality management system is required for development of CFD fire code. Good reference can be made to ISO 9000-3: Part 3: Guidelines for the Application of ISO 9001 to the Development, Supply and Maintenance of Software [11]. •
Analytical Solution Comparison
Analytical solution comparison is the simplest and most reliable method to verify the accuracy of CFD fire code and is recommended by all three standards of AIAA, ASTM and ISO. Exact solutions are closed-form analytic solutions to the set of PDE equations governing the fluid dynamics problems. However, in fire engineering, full threedimensional Navier-Stokes equations are inherently non-linear, thus make it very difficult to find a special case that an exact solution can satisfy the governing equations and boundary conditions [20]. In view of this shortcoming, other substitutes should be developed. Method of Manufactured Solutions (MMS) is a technique for developing a special type of analytical solution to be used for testing numerical algorithms, which was proposed and developed by Steinberg and Roache [21]. MMS was successfully employed to verify the computer codes on complex fluid flow problems [22,23]. Excellent guideline was presented in the technical report of Sandia National Laboratories [20]. The MMS is a general procedure for generating or manufacturing analytical solutions to the governing equations that characterize the CFD problem and can be used for accuracy verification of codes. The core idea behind the method is to manufacture a solution to the governing equations, where a specific form of the solution function is assumed to satisfy the PDE of interest. By inserting the function into the PDE, all the derivatives are analytically derived. The equation is rearranged so that all remaining terms in excess of the terms in the original PDE are grouped into the source term. This source term is then considered to be simply added to the original PDE so that the assumed solution function satisfies the new PDE exactly. Then an exact solution is “manufactured”.
65
•
Benchmark CFD Code Comparison
Benchmark CFD code comparison is an alternative to exact solution. The computed result is recommended to compare with the simulated solution by benchmark fire code to quantify the credibility. The major disadvantage of this method lies in the fact that there is insufficient discussion made, even in the standards of AIAA and ISO, to give a full description on the selection criteria of benchmark CFD code. •
Grid Size and Time Step Refinement
Grid size and time step refinement exercises are the most widely accepted methods for verification purpose and proposed by all three standards by AIAA, ASTM and ISO. These methods are comparatively simple as only very minor modification in CFD code is necessary. The major drawback for this type of verification method is the huge computational resources demand. •
Residual Errors Checking
Application of residual errors to monitor the numerical convergence of fire code is a basic, typical and well-developed method in CFD modelling. And it is not surprising to find it in the standards of AIAA, ASTM and ISO. This fundamental technique is recommended to extend not only to all governing equations but also the mass and energy balance for the whole computational domain. •
Validation with Experimental Data
Last but not least, code validation can be conducted by means of comparison with the experimental result. Standards by ASTM and ISO remind the CFD modelers and users on the uncertainty of experimental measurement. Detailed description can be found in the Standard Guide for Obtaining Data for Deterministic Fire Models E1591-00 by ASTM [18]. Standard by AIAA establishes a practicable way to tackle the problem. The complex real fire situation is recommended to be subdivided into several simpler levels, so called the subsystem cases, benchmark cases and the simplest one unit problems. For unit problem, it is able to offer accurate experimental measurement for code validation purpose. Experimentalists in the field of fire engineering can design a number of unit problem cases with only few special fire features. By passing through a sufficient number of validation tests of different unit problem cases, a CFD fire code can be certified to be validated for a specific real fire scenario. However, special attention should be
International Journal on Architectural Science 5.
S. Schlesinger, R.E. Crosbie, R.E. Gagne, G.S. Innis, C.S. Lalwani, J. Loch, R.J. Sylvester, R.D. Wright, N. Kheir and D. Bartos, “Terminology for model credibility”, Simulation, Vol. 32, No. 3, pp. 103-104 (1979).
6.
W.L. Oberkampf, T.G. Trucano and C. Hirsch, “Verification, validation, and predictive capability in computational engineering and physics”, Foundations for Verification and Validation in the 21st Century Workshop, Johns Hopkins University/ Applied Physics Laboratory, Maryland, USA (2002).
7.
W.L. Oberkampf and T.G. Trucano, Verification and validation in computational fluid dynamics, SAND2002-0529, Sandia National Laboratories, New Mexico, USA (2002).
8.
Institute of Electrical and Electronics Engineers, IEEE standard dictionary of electrical and electronics terms, ANSI/IEEE Std 100 – 1984, USA (1984).
9.
Institute of Electrical and Electronics Engineers, IEEE standard glossary of software engineering terminology, IEEE Std 610.12 – 1990, USA (1991).
10.
American Nuclear Society, Guidelines for the verification and validation of scientific and engineering computer programs for the nuclear industry, ANSI/ANS – 10.4 – 1984, USA (1984).
11.
International Organization for Standardization, ISO 9000-3: Quality management and quality assurance standards – Part 3: Guidelines for the application of ISO 9001 to the development, supply and maintenance of software, International Organization for Standardization, Geneva, Switzerland (1991).
12.
K.T. Yang, “Role of fire field models as a design tool for performance-based fire-code implementation”, International Journal on Engineering Performance-Based Fire Codes, Vol. 1, No. 1, pp. 11-17 (1999).
U.S. Department of Defense, DoD instruction 5000.59: modeling and simulation (M&S) management, Defense Modeling and Simulation Office, Office of the Director of Defense Research and Engineering, USA (1994).
13.
W.K. Chow, “Review on fire models and their application to design based on fire engineering approach”, Keynote Lecture – International Conference on Building Fire Safety, 20-21 November 2003, Gardens Point Campus, Queensland University of Technology, Brisbane, Queensland, Australia; appeared in the proceedings, pp. 12-24 (2003).
U.S. Department of Defense, DoD instruction 5000.61: modeling and simulation (M&S) verification, validation, and accreditation (VV&A), Defense Modeling and Simulation Office, Office of the Director of Defense Research and Engineering, USA (2003).
14.
American Society for Testing and Materials, Standard guide for evaluating the predictive capability of deterministic fire models, E 1355 – 97, American Society for Testing and Materials, USA (1997).
15.
International Organization for Standardization, Fire safety engineering – Part 3: Assessment and verification of mathematical fire models, BS ISO TR 13387-3, International Organization for Standardization, Geneva, Switzerland (1999).
16.
P.J. Roache, Verification and validation in computational science and engineering, Hermosa, New Mexico, USA (1998).
17.
American Society for Testing and Materials, Standard guide for documenting computer software
paid to the uncertainty of measured data and the complexity of fire experiments.
8.
CONCLUSION
The development of verification and validation (V&V) methods in the community of engineering has been reviewed in this paper. Three major international standards on the assessment of the credibility of CFD fire code are presented and compared. Eight essential means for the purpose of CFD fire code verification and validation are extracted and summarized. Although eight essential means for V&V are proposed, it does not mean that all methods have to be applied in order that the credibility of a CFD fire code can be determined. Satisfactory results obtained from all eight essential means do not stand for completion of V&V, but it can be claimed that there are strong evidences of satisfactory V&V checking. On the other hand, failure to satisfy any one of the proposed methods reveals some kind of deficiencies embedding in the CFD fire code. In respect of validation, an important concept is given in the standard by AIAA [4]. Since a complex fire scenario can be disintegrated into a number of different unit problem cases, it is worth for the experimentalists in the field of fire engineering to build up a database of fire experiments of different unit problem cases.
REFERENCES 1.
2.
3.
PD 7974 Application of fire safety engineering principles to the design of buildings – Part 0 Guide to design framework and fire safety engineering procedures, British Standards Institute, UK (2002).
4.
American Institute of Aeronautics and Astronautics, Guide for the verification and validation of computational fluid dynamics simulations, AIAA G-077-1998, American Institute of Aeronautics and Astronautics, Reston, USA (1998).
66
International Journal on Architectural Science for fire models, E 1472 – 92 (Reapproved 1998), American Society for Testing and Materials, USA (1998). 18.
American Society for Testing and Materials, Standard guide for obtaining data for deterministic fire models, E 1591 – 00, American Society for Testing and Materials, USA (2000).
19.
B.N. Taylor and C.E. Kuyatt, Guidelines for evaluating and expressing the uncertainty of NIST measurement results, NIST Technical Note 1297, NIST, Gaithersburg, MD, USA (1994).
20.
K. Salari and P. Knupp, Code verification by the method of manufactured solutions, SAND20001444, Sandia National Laboratories, New Mexico, USA (2000).
21.
S. Steinberg and P.J. Roache, “Symbolic manipulation and computational fluid dynamics”, AIAA Journal, Vol. 22, pp. 1390-1394 (1984).
22.
D.P. Aeschliman, W.L. Oberkampf and F.G. Blottner, “A proposed methodology for computational fluid dynamics code verification, calibration, and validation”, 16th International Congress on Instrumentation in Aerospace Simulation Facilities (ICIASF), Wright-Patterson AFB, OH 45433 (1995).
23.
T.M. Shih, “A procedure to debug computer programs”, International Journal of Numerical Methods in Engineering, Vol. 21, pp. 1027-1037 (1985).
67
