Field intercomparison tests: The empirical assessment of uncertainty

Transcription

1 Field intercomparison tests: The empirical assessment of uncertainty Application to different streamgauging techniques Pour mieux affirmer ses missions, le Cemagref devient Irstea First meeting of the Management Committee (MC) for the CHy Project on the Assessment of the Performance of Flow Measurements and Techniques for the Period Geneva, 2 to 5 December 2013 Jérôme Le Coz (IAHS) Irstea Hydrology-Hydraulics Lyon, France

2 Techniques for assessing measurement uncertainty The propagation of uncertainties (GUM, JCGM 2008) the whole measurement process must be modelled by a data reduction equation (DRE) the uncertainties are propagated either analytically (GUM) or through numerical simulations (MCMC) in case the DRE is complex this reference method makes it possible to: use available uncertainty informations (specs, calibration, tests...) compute uncertainty budgets and improve the process estimate the uncertainty of a measurement The empirical assessment of uncertainty (ISO 21748, ISO 5725) the whole measurement process may not be modelled the uncertainty of the measurement method is inferred from the variability of results reproduced under repeatability and reproducibility conditions often used method (chemistry...), standard-based and consistent with GUM example : analysis of interlaboratory tests 2

3 Techniques for assessing measurement uncertainty Some troubles which affect uncertainty analysis in hydrometry... The propagation of uncertainties There is no international reference for large discharges in rivers propagation of errors in elementary quantities is required some important error sources cannot be put into equations complex measurement environments, operator skills... elementary measurements (width, depth, velocity, angles, temperature...) are not all, or not always, calibrated against measurement standards some uncertainty components must be assessed by expert knowledge the assessment of covariance terms is quite problematic... 3

4 Techniques for assessing measurement uncertainty Some troubles which affect uncertainty analysis in hydrometry... The empirical assessment of uncertainty (interlaboratory) measurement standard for discharge in rivers are generally not available the intrinsec method bias is difficult to assess by the interlaboratory test all error sources may not contribute to the observed variability, depending on the field test procedure the resulting uncertainties may be overestimated when the field test procedure is of limited quality, or when some assumptions on errors are not accepted based on some consensus 4

5 Objectives of hydrometric interlaboratory tests One of the following objectives must be chosen: assessment of the uncertainty related to the discharge measurement method under given conditions (ISO 13528, ISO 5725) proficiency testing: verification of the individual performance of instruments and laboratories in applying a given gauging method (ISO 13528) Secondary objectives share and homogenize practices and methods discover technical products from users and vendors provide field training and briefings progress towards methods acknowledged by metrologists participate in local, national, international networks of hydrometers 5

6 Some examples of recent comparison tests involving several French hydrometric organizations Intercomparison / Organised by Instruments Deployment Reference discharge Participants Attending vendors Vézère 2009 Groupe Doppler 37 ADCP tethered station, tracer dilution 21 teams (~70 people) none Châteauneuf 2009 CNR 8 ADCP vessel scintillometry in conduits 5 teams none Génissiat 2010 Groupe Doppler 26 ADCP vessel transit-time in conduits 13 teams (~50 people) RDI, Sontek Gentille 2011 Groupe Doppler 34 ADCP 12 fluorometers tethered immersed turbine ratings, currentmeters 30 teams (~100 people) RDI, Sontek, Ott Aix 2011 Irstea 11 currentmeters wading rod ADCP, tracer dilution 8 teams (~25 people) none station 4 teams (33 people) Industron Ott Vic-le-Comte currentmeters hydrometric DREAL RA vans 6

7 Example: tethered ADCP regatta (Vézère 2009) Arrangement of 12 parallel measurement transects 3 series for 36 ADCP Discharge kept constant over half-days by the upstream dam 7

8 Example: tethered ADCP regatta (Vézère 2009) 8

9 Example: currentmeter tests (Aix 2011) The Toulourenc river at Veaux The Ouvèze river at Entrechaux Mechanical currentmeters (Ott C2) Electromagnetic currentmeters (Marsh Mac Birney FLO-MATE, Ott Nautilus) Acoustic currentmeters (Ott ADC, Sontek Flowtracker) 9

10 Example: currentmeter tests (Aix 2011) Toulourenc at Veaux Team-averaged results according to different discharge computation procedures Uncertainty derived from interlaboratory analysis : ±16% (at 95% level of confidence) likely overestimated value due to test procedure defaults (non constant discharge) 10

11 Example: currentmeter tests (Vic-le-Comte 2011 and 2013) The Loire river downstream of Villerest dam Mechanical currentmeters from 12 hydrometric vans Discharge was maintained constant by the dam Two replicates for each team 11

12 Example: other streamgauging techniques 8 fluorometers used for tracer 11 surface velocity radars (Génissiat 2012) The Rhône river downstream of Génissiat dam dilution Discharge was maintained constant by the dam Tested parameters : surface velocity, tilt angle Tested factors of influence : light, concentration, water type... 12

13 Example: vessel-mounted ADCP regatta (Génissiat 2011) Contrasted measurement conditions: downstream of dam (GE site: poor) at Pyrimont bridge (PY site: good) 13

14 Example: vessel-mounted ADCP regatta (Génissiat 2011) 3 stable discharge time slots for each of the 2 days 6 field comparison tests (6 discharge levels) Vessels at rest during lunch break 14

15 Example: vessel-mounted ADCP regatta (Génissiat 2011) Nominal discharge for each of the 6 turbines is 107 m³/s. The 6 dam conduits are now equipped with transittime flow measuring system. Uncertainty analysis showed that the expanded uncertainty in the mean measured discharge is likely to be less than ± 2-3%. Transit-time acoustic system for monitoring discharge in a conduit of Génissiat dam (Ø 6 m) 15

16 Organising field comparison tests in hydrometry Security / Logistics Vézère 2009 : synchronisation of ADCP transects with a pennant Risk assement, risk prevention plan, especially downstream of dams Application of security rules Permanent communication between leaders, participants, dam managers List of participants and contacts (cell phone numbers) Logistics : accomodation, food/water, toilets, boats, computers, power, instruments... Planning : briefing, application of the test procedure, discussions and product demonstrations, debriefing and feeback 16

17 Organising field comparison tests in hydrometry Requirements for interlaboratory uncertainty analysis Constant discharge (monitor water level and velocity fluctuations) Homogeneous site: close cross-sections, with similar conditions Stable procedure : do not change team positions, configurations, instruments (minimum 4), etc. Repetition of simultaneous measurements, same number for each instrument (minimum 2) Clear distinction between fixed and variable parameters and conditions Bonus to improve the uncertainty analysis Independent discharge measurement (reference discharge) with assessed uncertainty makes it possible to assess the bias of the gauging method Hydraulic modelling of the intercomparison site gives information on the test conditions (stable stage and velocity) 17

18 Analysis of interlaboratory tests Consider Qi, the instantaneous discharge measured by team #i during a stable discharge time slot: Qi = Qtrue + Bm + Bi + ε with : Qmean Qtrue true discharge value (unknown) B bias associated with the measurement method m Q average of all discharge values (Ntot measurements mean for each of the ptot instruments over the test) B bias related to team #i i ε random error 18

19 Analysis of interlaboratory tests Qi = Qmean + Bi + εi Interlaboratory standard deviation Bi SL Bi + εi Reproducibility standard deviation 0 SR Repeatability standard deviation εi Sr S R = S r +S L 0 19

20 Analysis of interlaboratory tests Gauging method bias Bm u(bm ) How to estimate the uncertainty um related to the gauging method bias Bm? Bm Numerical estimation: Sensitivity tests on parameters and assumptions: for instance, the uncertainty related to extrapolated discharges for the ADCP Génissiat 2010 test was estimated to be 2.5% on average Numerical simulation or additional measurements of the real flow Propagation of uncertainties (reference method, GUM...) Comparison to reference discharge value Qref with assessed uncertainty uref (example : transit-time system in Génissiat dam conduits, uref ~2%) B m =Q mean Q ref 2 2 sr sl u ( B m) = + +u2ref N tot p tot p tot with Ntot and ptot the total number of transects and instruments over the test Note. Many comparison tests are necessary to assess Bm, u(bm) and their possible dependencies on major influence factors. 20

21 Analysis of interlaboratory tests ISO standards provide procedures for computing estimates sr and sl from the comparison test results, and for assessing the expanded uncertainty U at 95% level of confidence: U (Q 1,1 )=2s R= sr +s L +u m with : Q1,1 discharge measurement from 1 transect for 1 ADCP sr repeatability standard deviation (estimate) sl interlaboratory standard deviation (estimate) sr reproducibility standard deviation (estimate) um standard uncertainty related to the gauging method bias 21

22 Analysis of interlaboratory tests In case the discharge measurement, QN,p, is actually the average of N repeated gaugings for each of p instruments: U (Q N,p )= 2 2 sr Np 2 + sl p 2 +u m Example: a gauging conducted with an ADCP is usually the mean of N = 4 to 6 successive transects. In particular cases, several (p) ADCP may be used simultaneously and their results may be averaged altogether. 22

23 Application to ADCP field comparison tests ADCP regatta downstream of Génissiat dam (2010) site PY: sr = 2.4%, sl = 2.2% (from interlaboratory analysis) um = 1.25% (from sensitivity analysis) 23

24 Application to ADCP field comparison tests ADCP regatta downstream of Génissiat dam (2010) site GE: sr = 4.5%, sl = 3.9% (from interlaboratory analysis) um = 1.25% (from sensitivity analysis) 24

25 Comparison of uncertainty results from interlaboratory tests and propagation methods 16 currentmeters on wading rods Cernon at St-Georges-de-Luzençon, a small karstic river Uncertainty propagation results (constant Q) Interlaboratory test results ISO748 and Q+ (Le Coz et al. 2012) 25

26 Comparison of uncertainty results from interlaboratory tests and propagation methods 16 currentmeters on wading rods Durzon at Nant, a small karstic river (constant Q) Interlaboratory test results Uncertainty propagation results ISO748 and Q+ (Le Coz et al. 2012) 26

27 Conclusions In addition to the GUM reference method for propagating uncertainty, field comparison tests are an efficient and sounded method for empirically assessing the uncertainty related to a gauging method in given conditions. The empirical method is especially suited for ADCP regattas, or tracer dilution comparison tests, for which a large number of repeated measurements can be conducted with several instruments in a limited period of time, under stable hydraulic conditions. It is also applicable to current-meters. The results of the Génissiat 2010 ADCP regatta brought reliable assessments of the discharge uncertainty depending on the number of averaged transects and instruments. They confirmed the decisive influence of the environmental error sources (site-specific errors). 27

28 Documents ISO standards In preparation by French NHS: a national guide a spreadsheet GUM (JCGM 2008). Evaluation of measurement data Guide to the expression of uncertainty in measurement. JCGM member organizations (BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP and OIML), 120 p. ISO :1994 Accuracy (trueness and precision) of measurement methods and results Part 2: Basic method for the determination of repeatability and reproducibility of a standard measurement method ISO :1998 Accuracy (trueness and precision) of measurement methods and results Part 5: Alternative methods for the determination of the precision of a standard measurement method ISO 21748:2010 Guidance for the use of repeatability, reproducibility and trueness estimates in measurement uncertainty estimation ISO 13528:2005 Statistical methods for use in proficiency testing by interlaboratory comparisons Publications on ADCP applications of the interlaboratory method Intercomparison of ADCPs on the Rhône downstream of Génissiat dam (2010/10/12-15), K. Pobanz, J. Le Coz, G. Pierrefeu, technical report, 59 p. Stage-discharge hysteresis evidenced by multi-adcp measurements, J. Le Coz, K. Pobanz, J.-B. Faure, G. Pierrefeu, B. Blanquart, Y. Choquette, RiverFlow2012, 5-7 September 2012, San José, Costa Rica 28