Quality assurance for cookstoves testing centers: calculation of expanded uncertainty for WBT

Transcription

1 Technical Paper Quality assurance for cookstoves testing centers: calculation of expanded uncertainty for WBT Resumen: MarceloGorritty, GabrielaTrujillo IIDEPROQ, Stoves Testing Center, Bolivia. Muchas decisiones importantes se basan en los resultados de un laboratorio de ensayo, por eso es importante tener indicadores de la calidad de dichos resultados, es decir el nivel de validez y de confianza que poseen; la incertidumbre de las mediciones es un indicador de este nivel, para esto los resultados de un laboratorio de ensayo deberían ir acompañados de la incertidumbre asociada a las mediciones. Debido a la importancia de la información generada por el Centro de Pruebas de Cocinas (CPC) en cuanto a la Prueba de Hervor de Agua (Water Boiling Test, WBT), la calidad de los datos difundidos debe contar con protocolos de control interno de resultados, de modo que los valores y la incertidumbre asociados a los reportes del WBT de las cocinas testeadas sean evaluados constantemente. El presente trabajo, desarrolla un estudio y evaluación de la incertidumbre aplicando simulación probabilística asociada a la prueba WBT en la evaluación de estufas mejoradas. El caso de estudio utilizado muestra datos provenientes del Centro de Pruebas de Cocinas Mejoradas de Bolivia. Palabras clave: resultados, calidad, confianza, incertidumbre, Centro de Pruebas de Cocinas, Prueba de Hervor de Agua, control, simulación. Abstract: Many important decisions are based on the results from a laboratory test, thus, it is important to have indicators of the quality of these results, in other words, the level of validity and confidence they yield; the uncertainty of measurement is one such indicator. The results from a laboratory test should be accompanied by an indicator of uncertainty associated with the measurements. Due to the importance of the information generated from the Bolivian Stoves Testing Center (Centro de Pruebas de Cocinas, CPC) regarding the Water Boiling Test (WBT), the quality of the disseminated data must follow a protocol for internal control of the results so that the values and the uncertainty associated with the reports of the WBT will be constantly evaluated. This paper undertakes an analysis and an evaluation for the uncertainty of measurements using a probabilistic simulation associated with the WBT for the evaluation of improved stoves. The case study used in this paper uses sample data from the Bolivian Stoves Testing Center with the Malena stoves model. Keywords: quality, trust, uncertainty, Clean Cookstove Test Center, Water Boiling Test, control, simulation.

2 2. Theoretical Backgroud The assessment of measurement uncertainty in testing laboratories offers a number of important advantages: Indicators for measuring uncertainty in a quantifiable manner can be powerful tools in various areas, such as risk management, as well as for adding credibility of the test results. The expression of measurement uncertainty can offer a directly competitive advantage that can add value and meaning to the results. Knowledge of the quantitative effects of specific magnitudes in a test result increases the reliability of the test procedure. In this manner, we could take corrective actions with greater efficiency; making this more effective in relation to their costs. The assessment of measurement uncertainty is a starting point to optimize test procedures through a better understanding of the process. Customers, such as organizations who operate product certification, need information about the uncertainty associated with the results to assess compliance with specifications. Calibration costs can be reduced with this type of evaluation, if it can show that some quantities of influence do not make a significant contribution to the uncertainty. [1] There are two types of approaches for calculating the uncertainty: The intra-laboratory approach (own laboratory), which is subdivided into: o o Calculating uncertainty based on a mathematical model, that is, an equation giving the quantitative relationship between the measured amount and all dependent variables. Calculating uncertainty using validation data about the procedure in the laboratory. The inter-laboratory approach (collaboration studies) is divided into: o Using performance data of the method in a collaborative trial (for example: according to ISO 5725). [2] o Using data from proficiency testing (inter-laboratory). [3] In the present study the calculation of the uncertainty of measurement is based on a mathematical model following the intra-laboratory approach noted above.. The Guide for the Expression of Uncertainty in Measurement (GUM) [4] provides a set of procedures aimed at such a calculation. The GUM uses deterministic methods of classical statistics (frequentist) for evaluation of type A uncertainty, and it carries out the evaluation of type B uncertainties, as well as uncertainties combining the Bayesian method with analytical solutions. For the Water Boiling Test (WBT) [5], the application of the law of propagation of uncertainty poses a problem, because the calculation of partial derivatives is tedious, difficult and too complex. An alternative to this problem is the use of

3 numerical simulation, such as the Monte Carlo method, which provides a well-known and valid alternative. The Monte Carlo simulation method encompasses a collection of techniques for obtaining solutions to mathematical or physical problems through repeated random testing. In practice, random testing results are replaced by calculations using random numbers. This process can be summarized as follows: after generating a population of random input variables following a Gaussian distribution, a normal population of the measurand is generated and its uncertainty is calculated. [6] Specific programs have been developed for the application of this method, which allows us to graphically evaluate the influence of the variation of important magnitudes on the final results. These software tools allow you to define the input variables as realistic ranges of values, calculate all possible outcomes, and records them for further analysis and reporting. 3. Methodology Due to the uncertainty inherent in the input data for the calculations of the WBT, a Monte Carlo analysis on the set of equations used in the protocol to know the likelihood of energy performance of clean stoves will be applied. With this approach, kitchen tests can be simulated in order to analyze their sensitivity to the change of values in certain operating ranges. During the simulations, ranges of data variation for actual testing conditions, especially for the variables with greater uncertainty, such as humidity, ambient conditions, heat of combustion and mass and temperature measurements, are applied. The following sub-sections summarize the tasks needed to obtain an estimate of the uncertainty associated with a measurement result. The steps involved are: [3] 3.1. Specify the measurand This step is to analyze all the equations used in the WBT protocol. For this, the document and the spreadsheet of the WBT will be used as the analysis basis. All equations described therein will be reviewed and compared with other versions to verify recent changes Identify sources of uncertainty To accomplish this, we make a list of possible sources of uncertainty. It includes the sources that contribute to the uncertainty of the parameters in the relationship specified in the previous step. But it can include other sources and it should include sources that result from chemical hypothesis Quantifying the uncertainty components This is done to measure or estimate the size of the components of the uncertainty associated with each identified potential source of uncertainty. It is often possible to estimate or determine a single contribution of the uncertainty associated with a number of separate sources. It is also important to consider whether the available data sufficiently covers all sources of uncertainty. It is also important to carefully plan additional experiments and studies to ensure that all sources of uncertainty are properly taken into account.

4 Probability distribution There are distribution functions for continuous random variables. In the assessment of uncertainty, the following are considered: Table1: Probability distributions for uncertainty assessment. Distribution function Used when: Uncertainty Uniform (rectangular) A certificate or other specifications have limits without giving information on confidence. Estimation in form as a maximum range without knowledge about the distribution function has to be done. (1) 3 Triangular The available information on x is less restricted than in the case of a uniform distribution. Values that are close to x are more probable than the limits. Estimation is in the form of a maximum rank described by a symmetric distribution. (2) 6 Normal An estimate of repeated observations of a process with changes/random errors is made. Uncertainty has been determined as a standard deviation, a relative standard deviation or a coefficient of variance % without specification of the distribution. Uncertainty for a defined level of confidence (95% or otherwise), without knowing the function of the distribution. (3) % 100 (4) 2 (5) For a 95% 3 (6) For a 99.7% Source: Quality Assurance Partners (QAP), Diploma in Management Systems for Laboratories ISO / IEC 17025, Section 6: "Estimation of uncertainty in measurement", 2013.

5 It is considered that all the input variables have a normal probability distribution, for the explanation giving above, and the uncertainty is given by the standard deviation of a set of historical data. In order to properly fulfill the above, very different values to those found in the range of historical data must be removed, so the test of Grubbs [7] is applied, which involves comparing the calculated value of with the value tabulated. (7) Where: = calculated value for the Grubbs test. = value of the input variable. = average value for the input variable. = standard deviation for the input variable. When then it concludes that there are atypical outliers which should be removed and the mean and standard deviation should be re-calculated for performing the Grubbs test thereafter. Note that, in the event of an atypical outlier, the entire test is eliminated. 4. Calculate the combined uncertainty The data obtained in the previous step will consist of a number of quantified contributions of the total uncertainty; it can be associated with individual sources or combined effects of different sources. The contributions must be combined and expressed as standard deviations according to appropriate rules to give a combined standard uncertainty. To complete this step you must enter the data into the software, including the equations that generate the results for the WBT. The key for using the software is to define certain input cells in the spreadsheet as assumptions, and output cells as forecasts. Once you have defined the cells, the software uses Monte Carlo simulations to model the complexity of a real scenario. For each trial of a simulation, the software repeats the following three steps: 1. For each cell of assumptions, it generates a random number according to the range defined by you and then places it in the calculation sheet. 2. Procedures to recalculate the worksheet calculation. 3. Each of the cells of forecasts generates a value. This value is added to the graph in the forecast graphics. This is an iterative process that continues until: The simulation reaches a detention criterion given by the person who is performing the simulation. Or until you stop the simulation manually.

6 The input variables are defined as cases with their respective uncertainty and results as forecasts. An assumption for the data cell selecting a probability distribution describing the uncertainty of the data is defined. The following figure shows the chosen distribution and values for the variable: Fuel Humidity. Figure1. Normal distribution for the variable: Humidity fuel. Source: Authors calculation. Having defined the cells with the model s assumptions, we can define the forecast cells. The forecast cells contain formulas that refer to one or more cells of assumptions. Next, the simulation is performed with 100,000 iterations. This number of iterations is used because the probability distribution function is generated with greater precision and the GUM [4] recommends it because you often expect a range of coverage of 95% of the amount of output so two or more decimal digits are correct in the result. The software runs the simulation for the situation contained in the workbook and it displays a forecast chart while it is calculating the results. The forecast chart reflects the combined uncertainty of the assumption cells on the output results of the model. Graphs for forecasts were produced for all output variables of the WBT; we show the most important ones:

7 Figure2. Forecast: Thermal Efficiency Cold Start. Figure3. Forecast: Thermal Efficiency Hot Start.

8 Figure4. Forecast: Thermal Efficiency Simmer. Figure5. Forecast: Energy Benchmark 5L.

9 Figure6. Forecast: Fuel Benchmark 5L. Figure7. Forecast: Thermal Efficiency High Calorific Power.

10 Figure8. Forecast: Thermal Efficiency High Calorific Power. We now proceed to create the simulation report. It shows that all forecast graphs are normally distributed, so that the combined uncertainty is the value of the standard deviation. Only the graphs for the thermal efficiency of the three stages have a log-normal distribution, for which the standard deviation is also the combined uncertainty. 5. Report of the uncertainty The final step is to multiply the combined standard uncertainty by the coverage factor (k) chosen to obtain an expanded uncertainty. The expanded uncertainty is required to provide a range, which may cover a large fraction of the distribution of values that could reasonably be attributed to the measurand. For most purposes it is preferred that is equivalent to 2. However, this value may be insufficient when the combined uncertainty is based on statistical observations with relatively few degrees of freedom (less than 6). Note that we work with 22 tests; the value of the coverage factor 2 is assumed.

11 4. Results and discussion The determination of the expanded uncertainty is shown in table 2. Table2: Expanded uncertainty for the Water Boiling Test. RESULT SYMBOL UNIT Average Uncertainty Combined Uncertainty Expanded % COV % Uncertainty COLD START Water vaporized from all pots w cv g , % 44.48% Specific fuel consumption SC c g/liter 100,985 20, % 40.53% Temp-corr time to boil Pot # 1 Dt T c min 22,239 4,620 9, % 41.55% Temp-corr sp consumption SC T c g/liter ,559 41, % 40.50% Temp-corr sp energy consumption SE T C kj/liter % 40.50% Thermal Efficiency h c % 20,148 4,397 8, % 43.65% Equivalent dry fuel consumed f cd g % 36.17% Effective mass of water boiled w cr g % 17.63% Firepower FP c watts % 61.67% Burning rate r cb g/min 40,396 12,456 24, % 61.67% HOT START Water vaporized from all pots w hv g , % 43.44% Specific fuel consumption SC h g/liter 77,122 15,851 31, % 41.11% Temp-corr time to boil Pot # 1 Dt T h min 16,431 2,678 5, % 32.59% Temp-corr sp consumption SC T h g/liter ,405 32, % 41.68% Temp-corr sp energy consumption SE T H kj/liter % 41.68% Thermal Efficiency h h % 25,566 5,860 11, % 45.85% Equivalent dry fuel consumed f hd g % 37.94% Effective mass of water boiled w hr g % 15.28% Firepower FP h watts % 52.25% Burning rate r hb g/min 42,299 11,051 22, % 52.25% SIMMER Water remaining at end - all Pots w sr g % 5.61% Water vaporized w sv g , % 37.81% Specific fuel consumption SC s g/liter 136,076 32, % 47.72% Specific energy consumption SE S kj/liter % 47.72% Thermal Efficiency h s % 12,368 4,831 9, % 78.12% Equivalent dry fuel consumed f sd g 584, % 47.34% Firepower FP s watts % 47.34% Burning rate r sb g/min 13,000 3,077 6, % 47.34% Turn down ratio TDR - 3,344 1,600 3, % 95.67% Energy Benchmark 5L WBT BE kj % 29.81% Fuel Benchmark 5L WBT BF g % 29.81% Thermal efficiency high calorific power % 0,057 0,014 0, % 47.72% Specific fuel consumption low calorific power MJ/min 22,857 3,849 7, % 33.68%

12 We can see from the results that the overall uncertainty of the test is about 50%. Considering a control parameter called Coefficient of Variation (COV [%]) in the spreadsheet of the WBT, it is worth noting that the expanded uncertainty must have a value less than 25% of the result of the measurand. For that, the calculated uncertainty can be considered overestimated, which can be attributed to the important variation of environmental conditions and frequent changes of the operator. Likewise, two more sheets were included in the worksheet of the WBT, which automatically performs the calculation of the uncertainty of an individual test in order to report results including the uncertainty. 5. Conclusions A base of historical data has been developed with values from the Bolivian Stoves Testing Center for Water Boiling Test based on the Malena stoves model. An analysis of the formulas of international protocol Water Boiling Test (WBT) was undertaken based on the concepts of mass and energy balance, leading to the conclusion that, the mathematical model for energy efficiency of Stage 3 (simmer phase) has an error in the accompanying balance of mass used. This formula has been corrected and we continued with the following procedures. We identified the sources of uncertainty from different protocol variables, which we classify as follows: Input Parameters Humidity Temperature Mass Time Parameters of influence Staff X X Accommodation and environmental conditions Methods of testing and calibration X X X Equipments X X X X Measurement traceability X X X Sampling X X Handling of test items X X An estimate of repeated observations of the test with alterations/random errors was realized, determining the function of the input variables as a normal distribution and calculating its uncertainty as the standard deviation of the set. For this, atypical values in the tests were excluded. We developed the calculation system for WBT based on Monte Carlo Simulations. All input variables for the WBT model were supposed to have a normal distribution of occurrence for its historical values. Expecting that the model output variables maintain this trend, however, it has been found that the forecast of the thermal efficiency fits a log-normal distribution.

13 Because of the probability function distribution of input variables such as output variables, we conclude that the standard deviation is equivalent to the combined uncertainty. Considering a control parameter called Coefficient of Variation (COV [%]) in the spreadsheet of the WBT, it is worth noting that the expanded uncertainty must have a value less than 25% of the result of the measurand. We can see from the results that the overall uncertainty of the test is about 50%. This uncertainty can be considered overestimated, which can be attributed to the important variation of environmental conditions and frequent changes of the operator. Clean Cookstoves Test Centers in other countries can apply the same procedure with their own data for knowing the levels of dispersion and the uncertainty of the WBT for different stoves models and different environmental conditions. 6. Bibliography [1] IBMETRO - DTA, Estimation of measurement uncertainty in Testing Laboratories, [2] Sylvain Gregoire, Repeatability reproducibility with a standard method ISO , [3] Quality Assurance Partners (QAP), Diploma in Management Systems for Laboratories ISO / IEC 17025, Section 6: "Estimation of uncertainty of measurement", [4] Guide to the Expression of Uncertainty in Measurement (GUM), [5] US Environmental Protection Agency (EPA), The Partnerchip for Clean Indoor Air (PCIA), The Global Alliance for Clean Cookstoves (GACC), International Organization for Standardisation (ISO), International Workshop Agreement (IWA), The Water Boiling Test - Version 4.2.2, [6] Universitas, Volume 3, Number 1, Editorial Universitaria, [7] Quality Assurance Partners (QAP), Diploma in Management Systems for Laboratories ISO / IEC 17025, Section 4: "Internal control of the quality of laboratory results", 2013.