Biomass and Bioenergy 115 (2018) 27–34
Contents lists available at ScienceDirect
Biomass and Bioenergy journal homepage: www.elsevier.com/locate/biombioe
Research paper
Dealing with small sets of laboratory test replicates for Improved Cooking Stoves (ICSs): Insights for a robust statistical analysis of results
T
op C
Francesco Lombardi∗, Fabio Riva, Emanuela Colombo Politecnico di Milano, Department of Energy, Via Lambruschini 4, Milan, Italy
Improved Cooking Stoves (ICSs) represent the most commonly promoted solution to alleviate the burden associated with the use of traditional biomass in a short-term perspective. However, criticism is raising about the methodologies used for assessing their performance, with a particular focus on laboratory-based testing protocols. One of the key weaknesses of current protocols consists in the inaccurate and biased approach adopted for reporting and statistically analysing test results, which can lead to misleading conclusions about the actual improvements ensured by ICSs. This study proposes a robust procedure to deal with the statistical analysis of small sample sizes, and subsequently verify it through its application to an experimental comparison – based on the Water Boiling Test – between three models of stove. The results show that the current practice based on 3 or 5 replicates often produces biases in the analyses, as at least 13 replicates might be needed to achieve reliable results. Moreover, the study shows how the t-test is in most cases improperly applied, while the proposed procedure allows to deal both with normally and of non-normally distributed data sets in a robust way. In one case, the apparent improvement of an ICS model as compared to the three-stone fire, is refuted by the application of our procedure.
m de ca
1. Introduction
A B S T R A C T
ra fo
Keywords: Cookstove Statistic Test Protocol Performance ICS
y
A R T I C LE I N FO
e us
ly on
∗
ic
Biomass-fuelled Improved Cooking Stoves (ICSs) are commonly promoted as a potential interim solution to the lack of access to clean cooking facilities in developing countries, notwithstanding increasing scientific evidences about the limited real-life benefits they are able to bring as compared to the laboratory-based performance [1–6]. In this framework, an accurate assessment of the performance of ICSs represents a critical issue, which is widely debated in the literature with a particular focus on laboratory-based testing protocols, that represent the most widespread methodology for performance assessment [7–16]. Lombardi et al. [17] identified the main issues related to the existing testing protocols, and notably to the most common, viz. the Water Boiling Test (WBT), including: (i) the lack of real-life relevance [1,2,7,17–20], (ii) the low repeatability [10,17,21], and (iii) the inaccuracy of methodologies for the statistical analysis of results [9,10,17]. The present study focuses on the latter major issue, which is particularly relevant since biases in statistical inferences can lead to the promotion of non-significantly improved stoves, regardless of the testing protocol employed. Wang et al. [9] and Riva et al. [10] stress this concept, and highlight two major shortcomings that are common to all the statistical approaches of current testing protocols. The first is
related to the minimum number of test replicates – typically just 3 – prescribed to evaluate a stove performance and to perform statistical inferences [17]. As a matter of fact, performing a larger set of test replicates allows to achieve a more reliable value of standard deviation, i.e. a value that is representative of the statistical population [9,10], avoiding potential biases in statistical inferences. However, there is a trade-off between the reliability of the standard deviation and the number of test replicates – and thus the time and effort – required. To this end, Wang et al. [9] discourage relying on a three-replicates standard deviation and finally suggest performing at least 5 replicates as a “rule of thumb” to obtain sufficiently reliable results, though such threshold value has never been counter-checked by any other study. The second shortcoming regards the unjustified assumption of the data set being normally distributed. Indeed, the normality condition is an essential formal prerequisite for the application of the t-test, which is nonetheless proposed as a method to perform statistical inferences in a large part of the ICSs testing literature regardless of a prior analysis of data distribution [10]. This practice may easily lead to biased inferences, since performance parameters of biomass stoves are likely to experience deviations from normality when a sufficiently large sample size is considered [10,22]. To our knowledge, there are no studies in the literature that discuss how to deal with non-normally distributed data
Corresponding author.Via Lambruschini 4, 21056 Milan, Italy. E-mail addresses:
[email protected] (F. Lombardi),
[email protected] (F. Riva),
[email protected] (E. Colombo).
https://doi.org/10.1016/j.biombioe.2018.04.004 Received 2 November 2017; Received in revised form 20 March 2018; Accepted 12 April 2018 0961-9534/ © 2018 Elsevier Ltd. All rights reserved.
Biomass and Bioenergy 115 (2018) 27–34
F. Lombardi et al.
sets in the framework of ICSs performance assessment. In order to overcome the abovementioned issues, the goal of the present study is to provide a rigorous and practical procedure – verified via its application to a set of experimental tests on two commercial ICSs models and a three-stone fire – to perform a robust statistical analysis of the results of laboratory tests on cooking stoves, consisting of 3 main phases: (i) a practical guide to identify a minimum reliable number of replicates in an experimental campaign, (ii) an accurate and statistically sound method for analysing the statistical distribution and computing the uncertainty, and (iii) a scheme for comparing indicators of performances between two stoves. The order chosen to present the phases of our procedure reflects the chronological order of their practical fulfilment, as shown in Section 4.
2.2. Analysis of the statistical distribution and the uncertainty Given a number of n test replicates that is sufficient to obtain a reliable value of standard deviation, the results shall always be averaged and reported as in Equation (1):
X ± Ue (c.l.%)
(1)
Where:
op C
− X is the average value of the [Xi …Xn ] observations of the selected indicator of performance (e.g. Thermal efficiency η , Specific consumption SC , Time to boil); − Ue is the expanded uncertainty of the indicator X , for a selected confidence level (c.l.%), usually 90% or 95%.
2. Procedure for statistical analysis In order to compute Ue , two-steps shall be followed:
The proposed procedure builds and improves upon our report titled “Guidelines for reporting and analysing laboratory test results for biomass cooking stoves” [23].
y
1 Verify the normality of the data set [Xi …Xn ] by means of a ShapiroWilk test, which is the most powerful normality test in the conditions of interest [24,25]; 2a If the normality hypothesis is not rejected, Ue shall be calculated based on a t-student distribution [26]; 2b If the normality hypothesis is rejected, provide the uncertainty based on the Chebyshev's inequality – i.e. the most conservative interval. In this case, Ue = 1 Sn [27], with α equal to 0.10 or 0.05 α based on the desired confidence level, and Sn the standard deviation of the data sample.
ra fo
2.1. Identifying a minimum reliable number of replicates
m de ca
Current testing protocols set the minimum number of test replicates required to have a reliable value of standard deviation at 3, while Wang et al. [9] state that at least 5 replicates should be performed, based on their empirical observations with the WBT. Conversely, we propose a practical guide based on an iterative procedure that allows to check, after each additional test replicate performed, if a sufficient level of reliability for the standard deviation has been achieved or not, regardless of the testing protocol employed. A few simple steps shall be followed:
Relying on Chebyshev's inequality to compute the expanded uncertainty will lead to safer though larger confidence intervals. The final result is considered acceptable if the computed value of Ue is smaller than X .
1 Perform at least n = 5 test replicates; 2 For each performance indicator of interest a Compute the standard deviation; b Calculate the percentage change of the standard deviation (Sn ), between the n -th and (n − 1)-th replicates ( Δ%1), between the (n − 1)-th and (n − 2 )-th replicates ( Δ%2 ), and between the Sn − S (n − 1) ; (n − 2 )-th and (n − 3)-th replicates ( Δ%3 ), as Δ% = S
2.3. Comparing the performance of two different stove models
ic
In order to compare the indicators of performance between two stove models and to assess the relative improvements, the two respective confidence intervals for a selected parameter shall be compared. If they do not numerically overlap, it is always possible to provide statistically significant conclusions (e.g. one stove performs better than the other). Conversely, if there is overlapping between the two, the tester shall perform a statistical test to draw significant conclusions. Again, two-steps shall be followed for each performance indicator that needs to be compared:
(n − 1)
e us
c If Δ%1, Δ%2 and Δ%3 are both less than 10% in absolute terms, n is the minimum number of replicates required. Otherwise, add one more test replicate and iterate back from 2. The rationale behind this practical guide is to check whether the variation of the standard deviation after the addition of a new test replicate is negligible (i.e. less than 10%). Furthermore, since a low variation of the standard deviation can be fortuitously detected just to be refuted once a further replicate is added, our procedure requires that this condition holds true for at least three consecutive replicates ( Δ%1, Δ%2 and Δ%3 ). If this condition is consistently respected, it is reasonable to assume that the value of standard deviation is approximately representative of the statistical population. Nonetheless, the testers shall consider the possibility that significant outliers – due to systematic errors – may arise in the middle of a consistent trend of low percentage change of the standard deviation. In this case, the outliers should be carefully evaluated and eventually discarded. To this regard, the proposed analysis of the percentage change of the standard deviation might be also seen as a support towards an intuitive identification of systemic errors. Typically, different performance indicators will require a slightly different minimum reliable number of replicates to meet the criterion. Accordingly, the overall minimum number of replicates will correspond to the value for which such criterion is met for all the indicators that the tester needs to measure.
ly on
1 Verify the normality of the data sets [Xi …Xn ]Stove1, [Xi …Xn ]Stove2 related to the selected performance indicator for each stove, by means of a Shapiro-Wilk test [23,24]; 2a If the normality hypothesis is not rejected for each of the two data sets, the selected indicator shall be compared by means of a t-test assuming unequal variances [26]; 2b If the normality hypothesis is rejected for at least one of the data sets, the tester shall compare the data sets of the selected indicator by means of a non-parametrical test, i.e. the Mann-Whitney rank sum test [28]. The outlined procedure allows to provide the uncertainty and to perform statistical inferences both in case of normally and non-normally distributed data sets, preventing biases such as the use of t-tests for cases in which they are not rigorously valid. It is also worth noting that this procedure keeps valid even when comparing two samples of different sizes, as both the unequal variances t-test and the MannWhitney rank sum test can be performed with unequal sample sizes [29]. 28
Biomass and Bioenergy 115 (2018) 27–34
F. Lombardi et al.
op C
Fig. 1. Schematic of the WBT procedure. Source: Lombardi et al. [17].
3. Experimental procedure and apparatus
y
while a moisture meter (precision ± 0.1%) was used to measure the moisture mass fraction of fuelwood. Fuelwood consisted of small pieces of commercial Picea abies, with the following specifics (as provided by the retailer):
ra fo
The experimental campaign was carried out at the ICS-Lab of Politecnico di Milano – Department of Energy, in the period January–March 2016, by specifically trained testers. The testing protocol employed was the WBT v.4.2.3. The protocol consists of three phases performed in sequence, namely Cold-Start High-Power, Hot-Start High-Power and Simmering Low-Power phases, which are summarised in Fig. 1. Further details about the testing concept and procedure can be found in the official protocol [30] or in Lombardi et al. [17]. For simplicity, the present analysis is restricted to stoves energy performance only. Three energy performance indicators are considered, namely High-Power Thermal Efficiency, Low-Power Specific Consumption and Time to boil; the former two are those officially defined within the ISOIWA guidelines [30,31], whereas Time to boil has been selected since it is one of the two parameters evaluated in the study by Wang et al. [9], and could thus serve as a means of comparison. Further details about performance metrics are provided in Appendix A. The tests were conducted using stainless steel pots (diameter 19.5 cm, height 11.5 cm), filled with a volume of 2.5 L of water. We selected for testing two commercial stove models by Envirofit – the Econofire and the M-5000 – and a three-stone fire (TSF) (Fig. 2). In order to ensure that the TSF configuration remained unaltered in each test repetition, the stones position and orientation was marked on the floor of the lab. We performed additional test replicates until the criteria defined in sub-section 2.1 were met for all the selected indicators and stove model. A Portable Emissions Monitoring System (PEMS) [32] was used to capture all the flue gases and to vent them outside the lab, and to connect a K-type thermocouple for reading water temperature. A digital scale (precision ± 1 g) was used to weigh the fuel and the pots,
moisture mass fraction (at point of sale): 10–12%; cross-section diameter: 15 mm; length: 220 mm; origin: Bosnia and Herzegovina; LHVdry: 19.250 kJ/kg.
m de ca
− − − − −
The fuel was transported in controlled conditions to the ICS-Lab and stored indoor. Small pieces of commercial wood-based firelighters were used as kindling material and lighted by means of a fire starter. The pot was placed on the stove as soon as the flames from the kindling material were able to light the wood, following the protocol prescriptions. 4. Results and discussion
ic
Following the first step of our procedure, we iteratively investigate if the number of replicates performed is sufficient to meet the reliability criteria outlined in sub-section 2.1. Fig. 3 shows how the fractional value of the standard deviation, for each indicator and stove model, changes in percentage when adding one more test replicate to the sample size, starting from the conventional 3. The results clearly highlight how the variation trends tend to converge as the sample size increases, stabilising at low values of percentage variation (consistently below 10%) only in a range between 9 and 13 test replicates, depending on the indicator and the stove model. In particular, the reliability criteria are satisfied after 11 test replicates for the Efficiency and the Low-
e us
ly on
Fig. 2. The stoves in use during the experimental campaign. a) Envirofit Econofire; b) Envirofit M-5000; c) Three-stone fire. 29
Biomass and Bioenergy 115 (2018) 27–34
F. Lombardi et al.
y
op C ic
m de ca
ra fo e us ly on Fig. 3. Percentage variation trend of the standard deviation for each additional test replicate, for the different performance indicators and stoves considered. Dotted lines represent the 10% reliability range.
30
Biomass and Bioenergy 115 (2018) 27–34
F. Lombardi et al.
Power Thermal Efficiency and Low-Power Specific Consumption than the TSF. Conversely, it is not possible to draw any other statistically significant conclusion for the other indicators from the mere observation of the results in Fig. 4, and statistical tests are required to provide additional information. As remarked in the third step of our procedure (sub-section 2.3), the commonly employed t-tests also require that the compared data sets are normally distributed, whereas in our case this is not true when comparing TSF and Econofire or M-5000 in terms of Thermal Efficiency. Therefore, we apply different statistical tests – the t-test and the MannWhitney rank-sum test – for each comparison, following the approach that we detailed. The tests are one-tailed in this case, since the aim is to assess if there is a significant improvement in performance. The results of statistical tests summarised in Table 3 and Table 4, allow us to conclude that the Econofire is significantly improved, as compared to the TSF, for all the three indicators considered, with a confidence level always equal to or higher than 95%, whereas the M5000 does not show any significant improvement. Particularly relevant is the conclusion related to the Time to boil for the M-5000, which is not significantly improved as compared to the TSF value. In fact, this result is in contrast with what would have been concluded if merely looking at the average value of the parameter without adopting a proper method of statistical analysis. Fig. 5 compares the average results that have been obtained based
Table 1 Shapiro-Wilk test p-values for different performance indicators and stove models. Values in italic are those for which the normality hypothesis is rejected.
TSF Econofire M-5000
High-Power Thermal Efficiency
Low-Power Specific Consumption
Time to boil
0.4173 0.0112 0.0003
0.1218 0.7442 0.7290
0.7513 0.6395 0.1375
y
op C
Power Specific Consumption, while 13 tests are required for the Time to boil, leading to the identification of an overall minimum number of 13 replicates. The conventional 3 or 5 replicates would not have been enough meet the required criteria. We underline, however, that in performing this first step of the analysis we discarded the results of one test (related to the Envirofit Econofire, as reported in the Supplementary material) that clearly emerged as an outlier due to the likely presence of a systemic error; in fact, the test – counterintuitively and in discordance with all the others – resulted in the longest Time to boil associated with the highest High-Power Thermal Efficiency. Proceeding with the second step of our procedure, data sets for each indicator and for each stove model are thus analysed in terms of statistical distribution by means of the Shapiro-Wilk test. The test rejects the normality hypothesis (p-value < 0.1) for the datasets related to the High-Power Thermal Efficiency of the TSF and the Econofire (values in italic font in Table 1). Accordingly, we compute the expanded uncertainty for non-normally distributed data sets relying on the more conservative Chebyshev's inequality, which results as expected in larger confidence intervals. Table 2 reports the average results (out of 13 test replicates) and the expanded uncertainty for all the cases considered. Unexpectedly, the M-5000 results seem to be worse than those of the TSF in terms of efficiency and specific consumption. Nevertheless, most of the confidence intervals defined in this way do overlap (as more easily seen in Fig. 4), preventing the tester from drawing statistically significant conclusions. Indeed, it is only possible to conclude, with a 90% confidence level, that the Econofire has a significantly better High-
ra fo
Table 3 Statistical inferences based on one-tailed tests to assess the improvement in the performance of the Econofire as compared to the TSF, for selected indicators.
m de ca
Econofire Vs TSF
High-Power Thermal Efficiency
Low-Power Specific Consumption
Time to boil
Statistical test
Mann-Whitney rank-sum test 9.62E-05 Yes***
t-test
t-test
1.19E-06 Yes***
0.0445 Yes**
p-value Significant improvement?
ic
TSF Econofire M-5000
e us
Table 2 Test results based on 15 replicates for different performance indicators and stove models. Results are reported as the average value ± the expanded uncertainty for a selected confidence level. High-Power Thermal Efficiency (%)
Low-Power Specific Consumption (MJ min−1 L−1)
Time to boil (min)
18.1 ± 0.5 (c.l. 90%) 25.2 ± 3.1 (c.l. 90%) 17.3 ± 2.7 (c.l. 90%)
0.085 ± 0.006 (c.l. 90%) 0.059 ± 0.004 (c.l. 90%) 0.092 ± 0.004 (c.l. 90%)
28.9 ± 1.5 (c.l. 90%) 26.8 ± 1.4 (c.l. 90%) 27.8 ± 2.9 (c.l. 90%)
ly on Fig. 4. Average results based on 15 replicates for different performance indicators and stove models. Bars represent the expanded uncertainty for a 90% confidence level. 31
Biomass and Bioenergy 115 (2018) 27–34
F. Lombardi et al.
hazardous can be to rely on a 3-replicate standard deviation to perform statistical inferences. What is more, our results also question the “ruleof-thumb” proposed by Wang et al. [9], i.e. a threshold value of 5 replicates. Indeed, our campaign would suggest that at least 13 test replicates are needed in order to be able to perform statistical inferences and to compute confidence intervals with an acceptable level of reliability for all the considered indicators. Still, such threshold number should not be intended as an absolute rule, but rather as a demonstration that small sample sizes may be strongly biased in terms of standard deviation, leading to possible errors in performing statistical inferences. The threshold identified in the present study would need to be confirmed by other independent studies in similar conditions and to be assessed also for indicators of emission performance, which might be subject to higher variability [9]. We also highlight that it is not possible to assess to which extent the large number of replicates needed to compute a reliable value of standard deviation is due to the intrinsic variability of the biomass combustion phenomenon rather than to the variability related to the WBT procedure [15]. It could be possible that different testing protocols with an improved repeatability may require a lower threshold number than the one we identified.
Table 4 Statistical inferences based on one-tailed tests to assess the improvement in the performance of the M-5000 as compared to the TSF, for selected indicators. M-5000 Vs TSF
High-Power Thermal Efficiency
Low-Power Specific Consumption
Time to boil
Statistical test
Mann-Whitney rank-sum test 0.9964 No
t-test
t-test
0.9613 No
0.2873 No
p-value Significant improvement?
y
op C
on 15 test replicates with those that could have been obtained adopting the widespread “3-replicates-only” practice. For all stove models, the difference is modest for the High-Power Thermal Efficiency indicator, whereas it is more relevant for the Low-Power Specific Consumption and the Time to boil. In one case (viz. the Low-Power Specific Consumption of the Econofire) the 3-replicate result leads to a clear overestimation of the performance of the stove, which might be classified as a borderline Tier 1 instead of Tier 0. Moreover, in accordance with Wang et al. [9], regardless of the relevance of the difference in the average values, performing only 3 test replicates does not allow providing reliable values of standard deviation and confidence intervals. In some cases, confidence intervals encompass more than one Tier of performance – for the two indicators for which Tiers are applicable – underlying the high uncertainty attached to the results that cannot be captured relying on a 3-replicate data set. Table 5 provides further insights about this issue; as a matter of fact, in all cases, the value of the standard deviation estimated based on 3 or even 5 replicates is significantly biased as compared to the value related to the 15-replicate sample size. The findings produced throughout the application of our procedure prove with enhanced evidence how
ra fo
5. Conclusions
ic
m de ca
The statistical approach we propose allows performing a rigorous statistical analysis of the results obtained via lab tests on ICSs, both in case of normally and non-normally distributed data sets. Such analysis is critically needed for justifying or refuting, based on statistically significant conclusions, the actual improvement ensured, in a controlled laboratory setting, by ICSs. Indeed, in one case the results of our robust analysis bring us to refute the improvement that is apparently ensured by an ICS model (Envirofit M-5000) if merely relying on the average results. Moreover, our investigation proves how the reliance on only
e us ly on
Fig. 5. Comparison between 15-replicate and 3-replicate based average results, for different performance indicators and stove models. Bars represent the expanded uncertainty for a 90% confidence level. Dotted lines represent ISO-IWA Tiers of Performance, when applicable to the indicator. Table 5 –Average results and standard deviation for different performance indicators and stove model, based on 3, 5 and 15 replicates. # Replicates
3 5 13
Average stdev Average stdev Average stdev
Three-stone fire
Econofire
M-5000 Wood
High-Power Thermal Efficiency (%)
Low-Power Specific Consumption (MJ min−1 L−1)
Time to boil (min)
High-Power Thermal Efficiency (%)
Low-Power Specific Consumption (MJ min−1 L−1)
Time to boil (min)
High-Power Thermal Efficiency (%)
Low-Power Specific Consumption (MJ min−1 L−1)
Time to boil (min)
18.2% 0.4% 18.0% 0.4% 18.1% 1.1%
0.080 0.006 0.083 0.009 0.085 0.012
29.5 3.4 30.3 2.9 28.9 3.1
24.6% 0.1% 25.6% 1.5% 25.2% 1.0%
0.051 0.007 0.054 0.007 0.059 0.008
25.4 2.3 25.9 1.8 26.8 2.9
17.0% 0.3% 17.1% 0.5% 17.3% 0.9%
0.090 0.006 0.090 0.005 0.092 0.008
27.3 5.0 28.4 3.8 27.8 5.9
32
Biomass and Bioenergy 115 (2018) 27–34
F. Lombardi et al.
three test replicates can be hazardous due to the likely unreliability of the standard deviation estimated with such a small sample size. Indeed, based on the procedure that we defined to identify the minimum reliable number of test replicates, our experimental campaign required 13 replicates before obtaining a consistent and reliable value of standard deviation for all the selected performance indicators. Such findings provide further evidence to the critical need for more accurate, robust and reliable approaches for the statistical analysis of the results from laboratory tests on ICSs. We hope that such findings will represent a helpful guideline for all the testing centres, as well as a valuable contribution to the work of the stakeholders involved in the ISO process for the definition of new standard protocols, so that the
misinterpretations and the biases that characterise the statistical approach of current testing protocols may be finally overcome. Nonetheless, it is important to remark that other major issues of performance assessment remain to be addressed and that the success of clean cooking programmes do not only depend on ICSs performance but also on flexibility and adaptability to the users' needs and on the engagement and empowerment of local communities. Acknowledgements The authors gratefully acknowledge the support of Daniele Greco and Francesco Acerbi in performing the experimental campaign.
op C
Appendix B. Supplementary data
Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.biombioe.2018.04.004. Acronyms
Three-stone fire Improved Cooking Stove Water Boiling Test
y
TSF ICS WBT
ra fo
Nomenclature
ic
m de ca
α Significance level (−) Δ% Percentage variation of the standard deviation (−) Density of water (m3 kg−1) ρ c. l. Confidence level (%) Specific heat capacity of water (kg−1K−1) cp, w fd Equivalent dry fuel consumed (kg) Specific enthalpy of evaporation of water (kJ kg−1) hlv LHVwood, dry Lower Heating Value of the fuelwood (dry wood) (MJ kg−1) m w, i Mass of water in the pot at the beginning of a test phase (kg) meva Total mass of water evaporated during a test phase (kg) n Sample size Standard deviation for a sample of n data Sn Tb Local boiling temperature (°C) Temperature of water in the pot at the beginning of a test phase (°C) Ti tboil Time to boil (min) Start time for a test phase (min) ti tf Ending time for a test phase (min) tsimmering Duration of the simmering phase (min) Ue Expanded uncertainty i-th element of a data set Xi
e us
Appendix A
ly on
Performance metrics High-Power Thermal Efficiency (ηISO − IWA) and Low-Power Specific Consumption (SCISO − IWA) have been defined as per the ISO-IWA guidelines [30,31], while Time to boil is based on the WBT 4.2.3 definition [30]. High-Power Thermal Efficiency is thus defined as
ηISO − IWA =
ηcold + ηhot 2
(%)
Where ηcold and ηhot are the Thermal efficiencies related to the Cold-Start High-Power and Hot-Start High-Power phases of the WBT. The mathematical formulation is the same for both phases:
η=
m w, i cp, w (Tb − Ti ) + meva hlv fd ⋅LHVwood, dry
(%)
Where: − − − − −
m w, i is the mass of water in the pot at the beginning of a test phase (kg) ; cp, w is the specific heat capacity of water (kJ kg−1K−1) ; Tb is the local boiling temperature (°C); Ti is the temperature of water in the pot at the beginning of a test phase (°C) meva is the total mass of water evaporated during a test phase (kg) ; 33
Biomass and Bioenergy 115 (2018) 27–34
F. Lombardi et al.
− hlv is the specific enthalpy of evaporation of water (kJ kg−1) ; − fd is the equivalent dry fuel consumed (kg) ; − LHVwood, dry is the Lower Heating Value of the fuelwood on a dry basis (kJ kg−1) ; Low-Power Specific Consumption is defined as:
fd ⋅LHVwood, dry
SC =
m w, f ⋅ρ /1000⋅tsimmering
(MJ min−1 l−1)
Where: − ρ is the density of water (m3 kg−1) ; − tsimmering is the duration of the simmering phase (%) ;
op C
Time to boil is instead defined as the difference between start and finish times for a test phase:
tboil = t f − ti (min) Where:
y
− t f is the ending time for a test phase (min) ; − ti is the start time for a test phase (min) .
ra fo
References
[15]
[1] T.W. Aung, G. Jain, K. Sethuraman, J. Baumgartner, C. Reynolds, A.P. Grieshop, J.D. Marshall, M. Brauer, Health and Climate-relevant Pollutant Concentrations from a Carbon-finance Approved Cookstove Intervention in Rural India, (2016), http://dx.doi.org/10.1021/acs.est.5b06208. [2] R. Wathore, K. Mortimer, A.P. Grieshop, In-use Emissions and Estimated Impacts of Traditional, Natural- and Forced-draft Cookstoves in Rural Malawi, (2017), http:// dx.doi.org/10.1021/acs.est.6b05557. [3] S. Sambandam, K. Balakrishnan, S. Ghosh, A. Sadasivam, S. Madhav, R. Ramasamy, M. Samanta, K. Mukhopadhyay, H. Rehman, V. Ramanathan, Can currently available advanced combustion biomass cook-stoves provide health relevant exposure Reductions ? Results from initial assessment of select commercial models in India, EcoHealth (2015) 25–41, http://dx.doi.org/10.1007/s10393-014-0976-1. [4] R. Quansah, S. Semple, C.A. Ochieng, S. Juvekar, F. Ato, I. Luginaah, J. Emina, Effectiveness of interventions to reduce household air pollution and/or improve health in homes using solid fuel in low-and-middle income countries : a systematic review and meta-analysis, Environ. Int. 103 (2017) 73–90, http://dx.doi.org/10. 1016/j.envint.2017.03.010. [5] K. Mortimer, C.B. Ndamala, A.W. Naunje, J. Malava, C. Katundu, W. Weston, D. Havens, D. Pope, N.G. Bruce, M. Nyirenda, D. Wang, A. Crampin, J. Grigg, J. Balmes, S.B. Gordon, A cleaner burning biomass-fuelled cookstove intervention to prevent pneumonia in children under 5 years old in rural Malawi ( the Cooking and Pneumonia Study ), a cluster randomised controlled trial 6736 (2016) 1–9, http:// dx.doi.org/10.1016/S0140-6736(16)32507-7. [6] D. Pope, N. Bruce, M. Dherani, K. Jagoe, E. Rehfuess, Real-life effectiveness of “ improved ” stoves and clean fuels in reducing PM 2. 5 and CO : systematic review and meta-analysis, Environ. Int. 101 (2017) 7–18, http://dx.doi.org/10.1016/j. envint.2017.01.012. [7] M. Johnson, R. Edwards, V. Berrueta, O. Masera, New approaches to performance testing of improved cookstoves, Environ. Sci. Technol. 44 (2010) 368–374, http:// dx.doi.org/10.1021/es9013294. [8] F. Lombardi, Performance Evaluation of Improved Cooking Stoves (ICSs): A Critical Review and a Theoretical and Experimental Study for Better Testing, Politecnico di Milano, 2016. [9] Y. Wang, M.D. Sohn, Y. Wang, K.M. Lask, T.W. Kirchstetter, A.J. Gadgil, How many replicate tests are needed to test cookstove performance and emissions? — Three is not always adequate, Energy Sustain. Dev. 20 (2014) 21–29, http://dx.doi.org/10. 1016/j.esd.2014.02.002. [10] F. Riva, F. Lombardi, C. Pavarini, E. Colombo, Fuzzy interval propagation of uncertainties in experimental analysis for improved and traditional three – stone fire cookstoves, Sustain, Energy Technol. Assessments 18 (2016) 59–68, http://dx.doi. org/10.1016/j.seta.2016.09.007. [11] Z. Zhang, Y. Zhang, Y. Zhou, R. Ahmad, Systematic and conceptual errors in standards and protocols for thermal performance of biomass stoves, Renew, Sustain. Energy Rev (2016) 1–12, http://dx.doi.org/10.1016/j.rser.2016.10.037. [12] P. Raman, N.K. Ram, J. Murali, Improved test method for evaluation of bio-mass cook-stoves, Inside Energy 71 (2014) 479–495, http://dx.doi.org/10.1016/j. energy.2014.04.101. [13] M. Johnson, R. Edwards, A. Ghilardi, V. Berrueta, O. Masera, Why current assessment methods may lead to significant underestimation of GHG reductions of improved stoves, Boil. Point 54 (2007) 11–14 http://db.sparknet.info/docs/BP54JohnsonEtAl-4.pdf. [14] P. Arora, S. Jain, A review of chronological development in cookstove assessment
[16]
[18]
[19]
[20]
[21]
[23]
[24]
[26] [27]
[28]
[29] [30] [31]
[32]
34
ly on
[25]
e us
[22]
ic
m de ca
[17]
methods: challenges and way forward, Renew, Sustain. Energy Rev. 55 (2016) 203–220, http://dx.doi.org/10.1016/j.rser.2015.10.142. C. L'Orange, M. DeFoort, B. Willson, Influence of testing parameters on biomass stove performance and development of an improved testing protocol, Energy Sustain. Dev. 16 (2012) 3–12, http://dx.doi.org/10.1016/j.esd.2011.10.008. S. Bhattacharya, D. Albina, A. Myint Khaing, Effects of selected parameters on performance and emission of biomass-fired cookstoves, Biomass Bioenergy 23 (2002) 387–395, http://dx.doi.org/10.1016/S0961-9534(02)00062-4. F. Lombardi, F. Riva, G. Bonamini, J. Barbieri, Laboratory protocols for testing of Improved Cooking Stoves ( ICSs ): a review of state-of-the-art and further developments, Biomass Bioenergy 98 (2017) 321–335, http://dx.doi.org/10.1016/j. biombioe.2017.02.005. C.A. Roden, T.C. Bond, S. Conway, A. Benjamin, O. Pinel, N. Maccarty, D. Still, Laboratory and field investigations of particulate and carbon monoxide emissions from traditional and improved cookstoves, Atmos. Environ. 43 (2009) 1170–1181, http://dx.doi.org/10.1016/j.atmosenv.2008.05.041. M. Johnson, R. Edwards, C. Alatorre, O. Masera, In-field greenhouse gas emissions from cookstoves in rural Mexican households 42 (2008) 1206–1222, http://dx.doi. org/10.1016/j.atmosenv.2007.10.034. P. Medina, V. Berrueta, M. Martínez, V. Ruiz, I. Ruiz-mercado, O.R. Masera, Closing the gap between lab and fi eld cookstove tests : benefits of multi-pot and sequencing cooking tasks through controlled burning cycles, Energy Sustain. Dev. 41 (2017) 106–111, http://dx.doi.org/10.1016/j.esd.2017.08.009. M. DeFoort, C. L'Orange, C. Kreutzer, Stove Manufacturers Emissions & Performance Test Protocol (EPTP), (2009) http://cleancookstoves.org/binary-data/ DOCUMENT/file/000/000/75-1.pdf. C. V Preble, O.L. Hadley, A.J. Gadgil, T.W. Kirchstetter, Emissions and Climaterelevant Optical Properties of Pollutants Emitted from a Three-stone Fire and the Berkeley-Darfur Stove Tested under Laboratory Conditions, (2014). F. Lombardi, F. Riva, E. Colombo, Guidelines for reporting and analysing laboratory test results for biomass cooking stoves, Politecnico di Milano (2017), http://dx.doi. org/10.13140/RG.2.2.23861.06884/1. B.W. Yap, Power comparisons of shapiro-wilk, Kolmogorov-Smirnov, lilliefors and anderson-darling tests, J. Stat. Model. Anal 2 (2011) 21–33. S.S. Shapiro, M.B. Wilk, H.J. Chen, A comparative study of various tests for normality, J. Am. Stat. Assoc. (1968) 1343–1372 PLEASE https://doi.org/10.1080/ 01621459.1968.10480932. NASA, Measurement Uncertainty Analysis Principles and Methods, (2010). C. Baudrit, D. Dubois, Practical representations of incomplete probabilistic knowledge, Comput. Stat. Data Anal. 51 (2006) 86–108, http://dx.doi.org/10.1016/j. csda.2006.02.009. H.B. Mann, D.R. Whitney, On a test of whether one of two random variables is stochastically larger than the other, Ann. Math. Stat. 18 (1947) 50–60 http://www. jstor.org/stable/2236101. W. Navidi, Statistics for Engineers and Scientists, McGraw-Hill, 2015. The Water Boiling Test 4.2.3, (2014) https://cleancookstoves.org/binary-data/ DOCUMENT/file/000/000/399-1.pdf , Accessed date: 11 July 2016. ISO, International Workshop Agreement IWA 11 Guidelines for Evaluating Cookstove Performance, (2012) https://www.iso.org/obp/ui/# iso:std:iso:iwa:11:ed-1:v1:en. Aprovecho Research Center, Aprovecho Research Center - Portable Emissions Monitoring System (PEMS), (n.d.). http://aprovecho.org/portfolio-item/portableemissions-monitoring-system/.
