ABSTRACT. A careful evaluation of the multizone nodal air flow simulation code COMIS was performed. The purpose of the evaluation of a computer code is to ...
EVALUATION OF THE MULTIZONE AIR FLOW SIMULATION CODE COMIS Claude-Alain Roulet Swiss Federal Institute of Technology
LESO-PB EPFL CH1015 Lausanne
Jean-Marie Fürbringer Romano Borchiellini NIST
Dipartimento di Energetica
Building Environment Division, IAQ group Building 226, Room A313
Cso Duca degli Abruzzi 26
Gaithersburg MD 20899
I -10100 Torino
ABSTRACT A careful evaluation of the multizone nodal air flow simulation code COMIS was performed. The purpose of the evaluation of a computer code is to assess its limits of validity (i.e. to verify its validity within some limits) and to improve its use ability. Evaluation is an alternative to validation since validation can never actually be achieved. The steps of this evaluation are analytical evaluation, comparison with experimental data and user sensitivity. The methods used are described and the result of the evaluation is presented. The conclusion addresses the need for user-friendly tools and guidelines for the analysis of simulation output for professionals.
KEYWORDS Ventilation, Computer model, Evaluation, Sensitivity analysis,
INTRODUCTION This contribution summarises the evaluation work made on COMIS within the International Energy Agency Energy - Conservation in Buildings and Community Systems - Annex 23
Politecnico di Torino
research program. [Fürbringer, Roulet, Borchiellini, 1995]. COMIS (Conjunction of Multizone Infiltration Specialists) is a multizone air flow and contaminant model which was started in 1989 during a one-year international workshop. The program, consisting of up-to-date models and numerical methods, as well as integrating original works of the group, is aimed at allowing the user to simulate air flow and pollutant distributions in a multizone structure. COMIS is a nodal model based on pressure boundary conditions. Basically, the program includes the following elements: cracks, duct systems, fans, volumes, stratification layers, vertical large openings, sources and sinks of pollutants and pressure coefficients of facades. It solves a static system of equations using a Newton-Raphson algorithm [Feustel and Raynor-Hoosen, 1990]. Validation is a word that is somewhat abused since a model can never be validated, but rather can only be not yet invalidated. The use of simulation in practice requires confidence in the results and this is only possible by a comprehensive evaluation and generalised sensitivity analysis. For this assessment of the simulation results, several tools have been developed, tested and improved. The whole methodology of 'validation' has been reviewed, re-analysed and adapted to this field [Fürbringer et al. 1995 and 1996].
Roomvent '96, 5th Conference on air distribution in rooms, Yokohama, Japan, July 17-19, 1996
1
Several tasks are required for this purpose: code check, sensitivity analysis, inter-model comparison, experimental comparisons and user tests. Code check was performed first by their authors for each module, and secondly by running the code for several benchmarks (analytical evaluation). Sensitivity analysis is an unavoidable step before inter - model or experimental comparison. Such a study provides the variations of the outputs related to variations (or uncertainties) in the inputs. New in this research programme was the use of the theory of experimental planning to perform the sensitivity analysis. This allowed us to obtain a large amount of information with a minimum of work. Inter-model comparison was performed with 14 other models, on 8 different types of problems. A large experimental validation was accomplished, using comparisons with many experiments in several countries. For each comparison, a sensitivity study was performed. This showed that the sensitivity itself depends on the case studied. Therefore, in order to know the confidence interval of the results of a simulation, a sensitivity analysis should be performed in each case. A module was hence added to the user-friendly version of the code to allow for such studies. For the user test, the code was distributed together with a problem to several users, asking them to solve the problem. Resulting input and output files were collected together with the comments of the users. Comparisons of the results showed misunderstandings of the user guide, which was then greatly improved.
RESULTS & DISCUSSION When a computer program is used to solve the mathematical model of a physical phenomena, many different tests can be performed to evaluate its behaviour, e.g. the program results may be compared with an analytical solution, the solution of another program or measured values. In the framework of the Annex 23 evaluation task, the comparison of the COMIS results with analytical solution or with the solution obtained using specialised mathematical software packages, has been called analytical evaluation and the relative comparison of different models has been called inter-model comparison.
Analytical evaluation This task aims to check as well the modelling of the physical effects as the respective algorithms in the code and the proper functioning of the program with respect to input data processing, error handling etc. For this purpose, the code is run with simple test cases, which results are known by a mean totally different from the tested code. A data base of test cases has been established, ranging from simple cases for testing specific physical models or routines in the code to complex problems combining different physical effects and topics. In Table 1 an attempt is made to roughly summarise the available test cases by defining some topics related to the individual elements of the modelling and the calculation steps implemented in COMIS and to classify the cases accordingly. The Table is not complete and shows only the most important topics. These benchmarks allowed to debug the code, the detect and solve convergence problems, and to define conditions for some limit
Table 1 - Input and calculation topics covered by the available test cases Topic related to input Meteo
Topic related to calculation Wind Stack Flow pressure pressure Crack Window HVAC
Pollutant transport
Schedules
Meteorological data Building orientation One zone Several zones Zone layers Crack Window HVAC Pollutants, Sources, Sink, Filters Schedules Occupants
Roomvent '96, 5th Conference on air distribution in rooms, Yokohama, Japan, July 17-19, 1996
2
cases.
the models used for the comparison for each topic tested.
Inter-model comparison COMIS results have been compared with those of 14 different models. A brief presentation of each of these models can be found in Furbringer et al. 1995
These comparisons show that there is a good agreement between COMIS results and the results of the other models; that is COMIS is able to predict the air flow behaviour as well or as bad as other models.
Each comparison was focused on a defined topic: comparison of the results using the same sets of input data, large openings, a simple wind and stack actuated infiltration problem, mass flow equation, sensitivity to uncertainty in input data, and smoke control. Table 2 summarises
The results obtained in “Comparison of the results using the same sets of input data”, “Simple infiltration problem” and “Mass flow equations”, show that no differences are found among the results of the models if the same data are correctly applied to each model.
Table 2 - Summary of the models used in the inter-model comparison models
Result com- Large open- Simple parison) ings case
Mass flow equation
Sensitivity to uncertainty in input data
Smoke propagation
AIDA AIRNET ASCOS BREEZE BREVENT CBSAIR CONTAM93/94 ESP LBL model MZAP NORMA PASSPORT TURBUL VENCON The prediction of the flow for a large vertical opening, in the single-side natural ventilation case, has been performed using six different air flow models and the agreement among the results obtained from these six models is very good; the correlation coefficient are greater than 0.95 except for the results obtained using NORMA. Two special investigations were also performed: the first was devoted to sensitivity to uncertainty in input data and it was shown that increasing the complexity of the input data corresponded to an increased uncertainty range of the results; the second was devoted to understand the smoke control in a building and the obtained results show that COMIS can be used for this purpose as well as ASCOS, a computer program developed especially for the smoke control simulation.
EMPIRICAL EVALUATION Principles The experimental comparison is the key aspect of the model evaluation. Following the principle of experimental science, the confrontation with experimental data should certify that COMIS works well. It shows indeed that it works coherently with some measured data. The experimental data is not free of errors and their confidence intervals should be considered. The result of a simulation or a measurement is not a single number or a line in a graph but an uncertainty range. Moreover, in the case of significant disagreement between experimental data and corresponding simulated data, different possibilities must be investigated before deciding which one, of the numerical model or the measurement, is wrong. For many reasons, data coming from both calculation and measurement may contain errors, and the com-
Roomvent '96, 5th Conference on air distribution in rooms, Yokohama, Japan, July 17-19, 1996
3
parison shall take account of these uncertainties. The results will be considered as being in good agreement when their confidence intervals overlap, and in disagreement when their confidence intervals do not overlap. To make things clear, it can briefly be said that the purpose of a comparison between measured data and simulation is to find possible discrepancies, but not to prove any agreement. Two images of reality are compared (Figure 1): an experimental model and a numerical one, and the question is: "when do they differ and, in this occurrence, why?"
Modelling errors
Reality
Measurement errors
Sensitivity and error analysis The aim of the sensitivity analysis is to examines the change of amplitude of some results provided by the code in response of changes in input data. The interference effects (e.g. the effect of combined variation of two or more variables) are also studied. This step also gives the possible error on the result when realistic errors are assumed for the input data. Figure 2 shows the principle of such an analysis, for a code having two input variables and one output. When the output is not proportional to input, the sensitivity of the output to changes in input depends on the values of input variables themselves. Therefore, the sensitivity analysis should be carried at several locations in the space of input variables.
Model used for measurements
Model used in computer code
Internal errors
External errors
1 Computer code
Data for input
2
Measurement results
y 4
Output data
Comparison
Output data
Figure 1: Experimental validation is comparing the results of two models of the reality.
Data specifications To be used for an experimental validation, the data should fulfil the following specifications which are usually only met in data sets obtained intentionally for validation: • compatibility: the data shall be measured on a building or a case which can be modelled with COMIS, • completeness: all the data necessary to run the code for the specified case and to compare results should be provided, • known accuracy: all the data shall be provided with their correct confidence intervals, • good accuracy: the confidence intervals should be as small as possible, according to the state of the art, • synchronism: all variable parameters should be measured at the same time.
x
3
Figure 2: Principle of the sensitivity and error analysis: variations in output (vertical axis) is assessed for various changes in input (x and y axes). It was early seen from preliminary analysis that the most sensitive input parameters change with the case studied and with the values of input parameters themselves. This results from the non linearity of the model. A general error analysis was not possible, and it was therefore performed for each case used in experimental comparison. Since COMIS requires a large amount of input data (often 50 values or more), such analysis requires a careful planning, using the most powerful tools. New in this study is the use of the theory of experimental planning to perform such analysis [Box et al. 1978, Law et al. 1982, Fürbringer, 1992 and 1994]. This tools allows to determine the sets of input data with which the code is successively run, to obtain the required knowledge with a minimum amount of runs. Fractional factorial plans were used to obtain the sensitivity factors or error multipliers (including combined effects), while Monte-Carlo technique provides the global
Roomvent '96, 5th Conference on air distribution in rooms, Yokohama, Japan, July 17-19, 1996
4
uncertainty on the result in less than 100 runs, whatever the number of input variables is.
Experimental comparisons Nine buildings have been investigated in the framework of Annex 23. The study is fully presented in the final report [Fürbringer et al, 1995], and extensively in Fürbringer et al,
[1996]. Only a short presentation of interesting points of the nine cases can be given below, illustrating the use of sensitivity analysis. Table 3 indicates the features of COMIS used in the test cases. It can be seen that, despite the large number of experiments, some COMIS features were not checked.
Table 3: Features of COMVEN used in the reported test cases Individual models
Case study: Optibat Solar Family house (J) house (J) (F)
LESO (CH)
Passys (B)
Namur flat (B)
Passys (GR)
Large Openings (GR)
Italgas (I)
Air flow components: Cracks Fans Straight ducts Duct fittings Flow controllers Large vertical openings Test data components Zone layer Pollutants Schedules: Links Large vertical openings Fans Zone temperatures Zone humidity Pollutant source or sinks Building orientation, terrain and wind profile data Pressure coefficients Weather data
OPTIBAT is an experimental one-floor flat, comprising six zones. It is built in a large experimental hall at CETHIL laboratory of INSA near Lyon (France). The external environment is controlled: it is not a field case. The main interest of this is to by-pass the much decried problem of pressure coefficients. Calculations and measurements were performed for various climatic conditions. In most cases, there are significant differences between calculated and measured air flow rates, even for total air flow rates in zones. There could be several reasons for this: • Bugs in COMVEN should not be an explanation, since inter-model comparisons show good agreement with other programs, which cannot likely contain the same bugs. • The network model does not correspond to reality. This is possible, since two users have obtained slightly different results.
• Confidence intervals on measurements are underestimated: This is possible too, since two different measurement techniques sometimes provided differing results. In the Japanese SOLAR HOUSE, the air exchanges between 3 zones on the same floor are accurately investigated. The air tightness is well known. The main interest of this case resides in the simplicity of the structure. For most of the cases, the confidence intervals overlap, but there are also some significant differences between the simulated and measured data which can not be explained by the accepted inaccuracy. The Japanese FAMILY HOUSE has nine zones distributed over two floors. This case is representative of an important part of the Japanese building stock. The presence of two floors is of great interest for observing the interaction
Roomvent '96, 5th Conference on air distribution in rooms, Yokohama, Japan, July 17-19, 1996
5
between wind and stack effect. As it was the case for OPTIBAT, it can be observed that some flows are large in the measured data while they are close to zero in the simulated data and inversely, which means that the measured network again does not corresponds to the simulated one. The LESO Building is a three-storey administrative building. It houses a building physics laboratory and is a good representative of a small office building. Its thermal, as well as its ventilation characteristics, have been investigated for many years [Dorer, Fürbringer et al, 1992]. This building is especially well instrumented. The structure of the building is however quite complicated. For the measurements, the 19 rooms were grouped in 11 zones. In the comparison, measured and simulated data for main flows overlap most of the time. The sensitivity analysis has been especially focused on the problem of the pressure coefficients. The sensitivity to the pressure coefficient uncertainty depends on wind direction and speed. This study shows once more the complexity of the air flow pattern behaviour and the necessity of having user friendly tools to perform on line sensitivity analysis and parametric study when simulating. It also shows that the pressure coefficient still is a critical parameter. The Belgian and the Greek PASSYS cells have been investigated by the BBRI and the University of Athens, respectively. The influence of the wind on a large opening was investigated. The experiment is sufficiently simple to be well controlled. The sensitivity analysis has been performed very comprehensively including both Monte-Carlo technique and factorial design for the Belgian case. It was shown that, while air flow through large openings are quite well predicted by COMIS in absence of wind, there are significant discrepancies when the wind velocity is higher than 0,2 m/s. This means that the wind speed effect is not sufficiently taken into account by the large opening model integrated within COMIS. The NAMUR FLAT was used for the evaluation of contaminant spreading and also air exchange through large openings. The flat has seven rooms and is located on the ground floor of a nine-storey building. The sensitivity analysis was also been performed very comprehensively, with both Monte-Carlo and factorial design techniques. The analysis of the input uncertainty is also exemplary and shows the efficiency of the sophisticated tools developed within this annex.
The agreement between measurement and simulation is good for all rooms, except for the injection room. This difference is explained by possible differences between real temperatures and those input in simulations. The so called ‘large opening experiment in Greece’ case has allowed for an evaluation of the influence of a large opening geometry. The ITALGAS Building, investigated by the Politecnico di Torino, is a one level family house. Built by a gas company for the investigation of gas heaters, the building is well instrumented. It is a sufficiently simple case to be studied with accuracy, but also sufficiently complex to be representative of real buildings. A comprehensive data set was obtained with this facility. The analysis of the results on this building has shown the strong influence of the stack effect in the gas furnace flue on the room ventilation. Therefore, special attention should be given to the choice of the single loss coefficient representing the flue butterfly valves as many different values can be found in literature [Borchiellini and Fürbringer 1996].
User test The objectives of the user tests performed within Annex 23 were: 1. to assess the difficulties experienced by users when using COMIS, 2. to improve the specification of data sets and the input routines of network models, 3. to determine the errors made by users in interpreting network input data. Two tests were proposed and performed by users from the nine participating countries [Roulet et al, 1996]. Benchmark test The first test represents a simple benchmark analysis in which the network and all input data are provided. No interpretation of building leakage and weather data is necessary. Two rounds were performed with the simple benchmark. The first round showed significant differences between results, which might be caused by errors in introducing input data as well as by differences between various versions of COMIS. The numerous comments from users were used to improve both the code and the User Guide. In order to clearly separate the effects of COMIS versions and of users, the second round
Roomvent '96, 5th Conference on air distribution in rooms, Yokohama, Japan, July 17-19, 1996
6
was performed exclusively with COMIS 1.2, which was version 1.1 corrected for bugs detected by the first run, and which took account of some comments.
participants. Therefore, and also because input errors and misinterpretation of the user guide, large differences were observed between simulation results (Figure 3).
In this case, apart two exceptions, results are pretty close to each other. A careful analysis of input files shown that the main reason for differences are input errors and options taken by participants.
An elementary sensitivity study has shown that the meteorological reference height, the building orientation and pressure coefficients have the largest influence. Whenever one door between extract duct and the facades is closed, the other internal leaks do not have a large influence on global air change. If there is a short circuit between extraction and the facades, no solution can be found.
Real case test The second test case is an open test requiring interpretation of the data by the user. It is an apartment situated at the centre of the fifth floor in a nine floors building located in mainland Europe. Ventilation is provided by natural stack effect and make up air enters through natural leakages. Provided data are those an engineer can usually obtain from an architect at the design phase (drawings, meteorological data, etc.). In particular, pressure coefficient were not given. Large differences appeared among the eight participants to this test. In particular, there were as many ways of modelling the flat as Wind 0, Temp. 0
Discrepancy between the results of participants resulted from different ways of modelling the flat, differences in input data, and input errors or omissions. Since comparisons of files presenting strong differences because of unclear definitions are not easy, input files were corrected for input errors or omissions and made similar to the reference file for the following variables: reference heights, building orientation, wind direction and wind exponent. Despite these corrections, large differences between results still remain. Wind 0 Temp. 20
Wind 10 Temp. 0
350
Air flow rate [kg/h]
300 250 200 150 100 50 0 User:
A
B
C
D
E
F
G
H
Figure 3: Air flow rates as calculated for the real case test by 8 different users for three different meteorological conditions.
CONCLUSIONS This work could be the basis of an exacting treatment of uncertainty in simulation which is an absolute requirement for a confident use of simulation in practice. The extensive evaluation procedure has shown that: 1. COMIS, which includes several features not included in other models, predicts air
flows in single- and multizone buildings like other simulation codes. 2. Comparisons to experimental data have shown that, as far as proper input is given, good agreement can be found between simulated and measured main air flows, that is air flows from or to the external environment, or the air change rate in a given room.
Roomvent '96, 5th Conference on air distribution in rooms, Yokohama, Japan, July 17-19, 1996
7
3. Discrepancies are found in inter-zonal air flows, which are often more sensitive to small variations in input. 4. Largest differences cannot be attributed to the computer code itself, but to the difficulty to model a given building with a network of conductances connected to pressure coefficients, which properly represent the real building. 5. This is directly related to differences observed in the user round robin tests. 6. The accuracy of today's computer codes in predicting air flow rates in buildings depends more on the accuracy of input data than on the quality of the code itself. Therefore, the user should be informed on the confidence intervals of the output resulting form uncertainties on the input. 7. This can be obtained by adding to the codes a Sensitivity Analysis Module (SAM), which should automatically perform the sensitivity analysis for the case studied by the user: Put a SAM in your program! [Fürbringer, 1996]. Several tools for such a task were developed within this work. An up-to-date methodology with a robust background and efficient tools was developed as well for the evaluation procedure as for such SAM's. There is a mandatory necessity for tools and precise rules for air flow and contaminant simulation analysis. The user has the largest influence on the results of simulation. He therefore needs a guide for the analysis, teaching him what to look for in the output. Till now, such simulation models were used principally by their authors. Now things are changing, since this type of models are spread out as planning tools among the building physics professional. To make this work useful for professionals, simulation computer programmes should also include error analysis tools. It is a challenge for developers to distribute products which can not be misused too easily.
ACKNOWLEDGEMENTS The authors will cheerfully thank all participants to IEA-ECB&CS Annex 23 research programme, who contributed to this study by improving and debugging COMIS, by performing extensive measurements and analyses, and by carefully reporting all this work.
REFERENCES Box, G. E. P., Hunter, W. G. and Hunter J. S (1978): Statistics for Experimenters, an Introduction to Design, Data Analysis and Model Building. John Wiley, New York, 1978. Borchiellini, R. and Fürbringer, J.-M. (1996): An evaluation exercise of a multizone air flow model. Submitted to Energy and Buildings, 1996 Dorer V, Fürbringer J.-M. Huck F., Roulet C.A. (1992): Evaluation of COMERL with the LESO data set, final report BEW, EMPA, Dübendorf, CH, 1992. Feustel H. A. (1996) Summary of Annex 23. Submitted to Energy and Buildings, 1996 Feustel, H.-E. and Raynor-Hoosen, A. (Editors) (1990): Fundamental of the Multizone Air Flow Model COMIS. AIVC Technical note 29, Air Infiltration and Ventilation Centre, Coventry. Fürbringer J.M. (1992): Evaluation Procedure Using Sensitivity Analysis of Model and Measurement. International Symposium on air flow in multizone structures, Budapest. Fürbringer J.-M. (1994): Sensibilité de modèles et de mesures en aéraulique du bâtiment à l’aide de plan d’expériences, PhD thesis No 1217, EPFL, 1015 Lausanne Switzerland. Fürbringer J.-M., Roulet C.-A., Borchiellini R. (Editors) (1995): Evaluation of COMIS, final report IEA.ECB&CS Annex 23 Multizone Air flow Modelling, LESO-PB, EPFL, 1015 Lausanne Switzerland. Fürbringer J.-M., Roulet C.-A., Borchiellini R.: (1996): An overview on the evaluation activities of IEA ECB&CS Annex 23. Submitted to Energy and Buildings Fürbringer J.-M., Roulet C.-A. (1996): Confidence in simulation results: Put a SAM in your model. Submitted to Energy and Buildings. Law A.M., Kelton W.D. (1982): Simulation modelling and analysis, McGraw-Hill, New York. Roulet C.-A. and Cretton P. (1996): The Influence of the User on the Results of Multizone Air Flow Simulations with COMIS. Submitted to Energy and Buildings
Roomvent '96, 5th Conference on air distribution in rooms, Yokohama, Japan, July 17-19, 1996
8
