Expert judgement and uncertainty quantification for climate change

Transcription

1 Expert judgement and uncertainty quantification for climate change 1. Probabilistic inversion as applied to atmospheric dispersion Atmospheric dispersion is mathematically simple, yet involves many features of more complex environmental models. It has been extensively studied in connection with radionuclide dispersion after an accident with nuclear power plants (for references see 1,2 ) and represents a poster child for uncertainty quantification (UQ) with probabilistic inversion (PI) for parametrized models. Consider a simple Gaussian plume dispersion model for concentration C(x,y,z) in point (x,y,z) of a neutrally buoyant contaminant released at height H in a constant wind field with wind speed u in the x direction: Q exp(-y 2 /(2σ 2 y(x))) exp[-(z-h) 2 /(2σ 2 z(x)) + exp[-(z+h) 2 /(2σ 2 z(x)) + mixing layer terms C(x,y,z) = [g/m 3 ] (1) uσ y (x) (2π) σ z (x) (2π) Q = release rate [g/s] σ y (x) = A y x By ; σ z (x) = A z x Bz [m] (2) The term in (z+h) captures reflection from the ground. Similar terms in mixing layer height L capture reflections from the mixing layer, to and fro. Without the parameters H and L, (1) emerges as a solution of the Navier Stokes laws of motion under ideal (anisotropic Fickian) conditions. The standard deviations σ y (x), σ z (x) are given by σ i (x) 2 = 2K i x/u where the eddy diffusivities K i [m 2 /s] are constant, i = y,z. Since each standard deviation grows with the square root of downwind distance x, B y = B z = ½ and the dimensions on both sides of (1) agree. These ideal conditions are known not to apply. The windfield is not constant, there is an inversion layer which reflects the spreading plume back to Earth and ground reflects the plume skyward. There is surface roughness, vertical wind profile, plume meander, Coriolis forces, plume depletion and so on. Fixes are added to (1) in an attempt to account for these features while keeping the data appetite manageable. One such fix involves categorizing the stability of the atmosphere and expressing the diffusion coefficients as functions of downwind distance σ y (x) = A y x By ; σ z (x) = A z x Bz (2) where the parameters A y, B y,a z, B z depend on atmospheric stability. B y, and, B z thereby differ from their theoretical value of ½. Stability is classified by any of a variety of schemes, commonly depending on time of day, insolation, cloud cover and release height. Adding such fixes is sometimes called a "parameterization". (2) is not derived from physical laws. In the best case NATURE CLIMATE CHANGE 1

2 these coefficients are based on regression of experimental results. The meaning of these coefficients is dependent on all the other fixes that have been applied, and no one actually believes these models. They are used because they are judged fit to purpose given data and computing constraints. Independent experts are reluctant to quantify their uncertainty on the diffusion coefficients A y, B y, A z, B z directly, for a variety of reasons: (i) these coefficients' meaning depends on the whole set of parametric fixes (ii) the experts may not subscribe to this model cum fixes, and (iii) the uncertainty on these coefficients is strongly dependent both on each other (if A is large, then B must be small) and on other parameters. To get independent expert input, experts were asked to quantify their uncertainty on measurable quantities such as cross wind dispersion at various downwind distances after a release under stipulated conditions. Focusing the elicitation on measurable quantities has two advantages; first it renders the uncertainty quantification model independent and second, in enables comparison of expert uncertainties with actual measured values. The expert distributions are combined using weights derived from their performance based on comparisons with measured values. Supplementary Table 1 compares the statistical accuracy and informativeness i scores of the participating experts with the scores from the performance weighted and equal weighted combinations of experts, and also with the modelers' uncertainty distributions ("MUSEMET"). Dispersion Statistical accuracy (P-value) Relative Information # Calibration variables Expert Expert Expert Expert Expert Expert Expert Expert Expert Expert Expert Equal weighted Performance weighted MUSEMET Supplementary Table 1 Expert and combination performance scores for dispersion panel (from Cooke 1994) 4. 2 NATURE CLIMATE CHANGE

3 After combining the experts' distributions, something like Figure 1 (main text) for σ y (x) emerges. The dotted curves represent experts' combined uncertainty for σ y at four downwind distances. The five solid lines represent the function σ y (x) for five choices of (A y, B y ). Probabilistic inversion (PI) draws a large sample of values (A y, B y ) from a diffuse distribution which covers the support of the dotted curves. Weights are then assigned to each solid curve such that when the initial distribution of (A y, B y ) is re-sampled using these weights, the dotted distributions are optimally recovered. If the PI problem is feasible, an optimal set of weights, in the sense of minimal departure from the starting distribution, is quickly found. This distribution typically introduces complex dependencies between A y and B y, and is given numerically. Summary of Results Modelers have placed distributions on the coefficients in (2) in an attempt to capture uncertainty in the use of model (1) with (2). Figure S1 illustrates the differences between the modelers' and the independent experts' uncertainties. Eleven experts quantified their uncertainties on the ratio of the centerline concentration to the release rate (a proxy for σ y σ z at four downwind distances 4 ). The ratio of the 95th to the 5th percentiles is depicted, indicating the spread in the uncertainty distribution. Whereas the uncertainty of the experts, and of the equal weighted (eqdm) and performance weighted (dm) combinations of experts' distributions grow considerably at downwind distances of 10km and 30km, this is scarcely the case for the modeler's distribution ("musemet"). Indeed, the modelers' uncertainty is generally smaller than that of any of the experts. Figure S1: Ratio of 95/5 percentiles for 11 experts, optimized performance based combinations (dm), equal weight combinations (eqdm) and the modelers' in house uncertainty assessment, for the centerline concentration ratio in stability class B at downwind distances 1937m, 4250m, 10km, 30km (from Cooke 1994) 4. NATURE CLIMATE CHANGE 3

4 If a probabilistic inverse exists, it is generally not unique and we seek a preferred inverse. If no inverse exists, the problem is infeasible, and we seek a distribution which minimizes some appropriate error function. The mathematics of PI are embedded in algorithms for iteratively finding the weights for weighted re-sampling the original distribution. The principal tool, the iterative proportional fitting algorithm (IPF), was introduced by 5 and re discovered by 6 and many others. In case of feasibility, reference 7 showed that IPF converges to the solution which is the minimally informative distribution with respect to the starting distribution satisfying the constraints. In case of infeasibility, little is known about the behavior of IPF. However, variations of this algorithm will always converge to a distribution which minimizes an appropriate error functional 8. For more information on probabilistic inversion see references 3, 9, and 10. It may happen that no distribution over model parameters adequately captures the distributions over the target variables and this is an important element in model criticism. 2. Spreadsheet Example for Ice Sheets PI (Supplementary Table 2) BA13 11 performed a structured expert elicitation to assess the uncertainty in sea level rise in 2100, and found 5%, 50% and 95% values which differ significantly from the values emerging from the simplified Process Based Model in L Here, we "invert the PBM model " at the BA13 11 distributions, to obtain distributions over D(EAIS), D(WAIS), SMB(EAIS) and SMB(WAIS) such that when pushed through the PBM, the resulting 5%, 50% and 95% values for SLR(EAIS) and SLR(WAIS) agree with those in BA The operation of inverting a model at a set of joint distributions is termed "probabilistic inversion" (PI), and the primary tool is the iterative proportional fitting algorithm (IPF). A formal definition of a probabilistic inversion is as follows: Let X R M, and let functions H i : X R, i = 1 n be given. Suppose distributions F i are assigned to H i, i = 1 n. We seek a distribution over X such that the functions H i take the distributions F i under this distribution. Such a distribution is a probabilistic inverse of H 1 H n at F 1 F n. We may assign a set C i of distributions to H i, for example the set of distributions satisfying some quantile constraints. A probabilistic inverse at C 1,..C n is defined analoguously. For more detail, see Kurowicka and Cooke (2006) 3. The Ice sheet PI Example (Supplementary Table 2) gives detailed calculations of the probabilistic inversion using IPF applied to Antarctic Sea Level rise described in the paper. Given an initial sample from a prior distribution over D(EAIS), D(WAIS), SMB(EAIS) and SMB(WAIS), IPF iteratively computes weights for re-weighting the original sample. When the original sample is re-sampled using the IPF weights, the resulting distribution complies with the BA13 11 percentile values. All calculations are spreadsheet functions so the reader can follow all details. In the real application, 100,000 samples were drawn from the prior distribution for discharge, surface mass balance and sea level rise for the East and West Antarctica Ice Sheets. In this toy calculation 4 NATURE CLIMATE CHANGE

5 only 30 samples are used for illustrative purposes. Abstract formulations of the IPF and similar algorithms can be found in the literature 3. i. Experts assessed their 5, 50 and 95th percentiles of their subjective uncertainty for all variables, including calibration variables. Statistical accuracy is measured as the p-value at which the hypothesis that the expert's probability statements were accurate would be falsely rejected, assuming the calibration variables are independently drawn from a distribution complying with the expert's quantile assessments. Information is measured as Shannon relative information with respect to a background measure determined by all experts' assessed quantiles. References 1. Radiation Protection Dosimetry Expert Judgement and Accident consequence Uncertainty Analysis, (2000) special issue L.H.J. Goossens and G.N. Kelly (eds) Vol 90, No,3, 2. Harper FT, Hora SC, Young ML, Miller LA, Lui CH, McKay MD, Helton JC, Goossens, LHJ, Cooke RM Pasler-Sauer J, Kraan B, Jones JA. (1995) Probabilistic accident consequence uncertainty study: Dispersion and deposition uncertainty assessment. Prepared for U.S. Nuclear Regulatory Commission and Commission of European Communities NUREG/CR-6244, EUR EN, SAND , Washington/USA, and Brussels-Luxembourg, November 1994, published January Volume I: Main report, Volume II: Appendices A and B, Volume III: Appendices C, D, E, F, G, H; 3. Kurowicka, D., and Cooke, Roger M. (2006) Uncertainty Analysis and High Dimensional Dependence Modeling Wiley, New York. ISBN-13: , ISBN-10: Cooke, R.M. (1994) Uncertainty in dispersion and deposition in accident consequence modeling assessed with performance-based expert judgment, Reliability Engineering & System Safety, Volume 45, Issues 1 2, 1994, Pages 35-46, ISSN , ( 5. Kruithof, J. (1937). Telefoonverkeersrekening. De Ingenieur, 52(8):15-25, Deming, E.W. and Stephan, F.D.(1940). On a least squares adjustment of a sampled frequency table when the expected marginal totals are known. Annals of Mathematical Statistics, 11(4): , Csiszar I.(1975) I-divergence geometry of probability distributions and minimization problems. The Annals of Probability, 3(1):146(158, Matus, F. (2007) On iterated averages of I-projections, Statistiek und Informatik, Universit at Bielefeld, Bielefeld, Germany, matus@utia.cas.cz. 9. Kraan, B.C.P and Bedford. T.J (2005) Probabilistic inversion of expert judgements in the quantification of model uncertainty. Management Science, 51(6): , Du C., Kurowicka D., Cooke R.M.(2006) Techniques for generic probabilistic inversion, Comp. Stat. & Data Analysis (50) 2006, Bamber, J. L. & Aspinall, W. P. An expert judgement assessment of future sea level rise from the ice sheets. Nature Climate Change 3, (2013). NATURE CLIMATE CHANGE 5

6 12. Little, C. M., M. Oppenheimer, and N. M. Urban (2013b), Upper bounds on twenty-firstcentury Antarctic ice loss assessed using a probabilistic framework, Nature Climate Change, 3, , doi: /nclimate TU Delft TU Delft TU Delft TU Delft (/ TU Delft TU Delft TU Delft TU Delft TU Delft TU Delft TNO TNO TNO NO TNO TNO Virtual experts TUD OP.DM TNO OP.DM" TUD + TNO OP.DM TUD EQ.DM TNO EQ.DM" TUD + TNO EQ.DM MUSEMET b NATURE CLIMATE CHANGE