Multi-modelling, multi-scenarios, hypothesis rejection and model diagnostics challenges and bottlenecks for uncertainty analysis frameworks Jim Freer

Transcription

1 Multi-modelling, multi-scenarios, hypothesis rejection and model diagnostics challenges and bottlenecks for uncertainty analysis frameworks Jim Freer Modelling World PREDICTIONS Monitored World Real World Model evaluation Errors Natural Variability Model Development Failures Limitations Perceptual Understanding Thanks to: Hannah Cloke, Yi He, Fredrik Wetterhall, Florian Pappenberger, Phil Younger, Tobias Krueger, Keith Beven and others. AGU, San Francisco, Dec 2009 Jim Freer slide 1

2 Model Development, Prediction and Evaluation a learning framework? Perceptual Understanding (P processes) Strategy Learning Feed back into Model Development and Monitoring Understanding Evaluate Prediction Uncertainties Monitor Environment incl. V(R, Q, P, S) Develop Multiple Feasible Models Run Multiple Feasible Model Simulations Evaluate Model Predictions Retain good Simulations Evaluate Observations (limitations/ uncertainties) INPUT Observational Scenarios OUTPUT Observational Ranges Reject poor Simulations AGU, San Francisco, Dec 2009 Jim Freer slide 2

3 Different observations have clearly different error structures... WT 1 WT 2 WT 2 WT 1 AGU, San Francisco, Dec 2009 Jim Freer slide 3

4 Not even all discharge should be treated as having equal quality! Catchment 1 Catchment 2 Catchment 3 Catchment 4 AGU, San Francisco, Dec 2009 Jim Freer slide 4

5 Timestep based Performance Measures that include data uncertainties Uncertain Observation Example limits of acceptability Performance Measure? Over Prediction Under Prediction AGU, San Francisco, Dec 2009 Jim Freer slide 5

6 Example of ensemble multi-model prediction under climate change... End-to-End modelling and cascading uncertainties FREE AGU, San Francisco, Dec 2009 Jim Freer slide 6

7 A complex cascaded modelling problem under climate change 2... HBV LISFLOOD-RR AGU, San Francisco, Dec 2009 Jim Freer slide 7

8 Ensemble of RCM members to account of the uncertainties in input distributions to 2100 GCM output ERA40 Bias-corrected RCM Bias-corrected RCM Bias-corrected Bias-corrected RCM RCM RCM output Hydrological model 1 Bias-corrected Bias-corrected RCM RCM Statistical downscaling Hydrological model 2 16 RCMs 25 x 25 km (ENSEMBLE) C4I-RCA3 UCLM-PROMES ICTP-REGCM3 INM-RCA3 ETH-CLM KNMI-RACMO2 HC-HADRM3 CNRM-RM4 DMI-HIRHAM5 OURANOS MPI-REMO METNO-HIRHAM GKSS-CLM SMHI-RCA CHMI-ALADIN 11 Ensemble members HadRM3 (UKCP09) 10 RCMs 50 x 50 km (PRUDENCE) GKSS-CLM ETH-CHRM DMI-HIRHAM5 HC-HADRM3 MPI-REMO UCLM-PROMES MRI-AGCM SMHI-RCAO ICTP-REGCM3 KNMI-RACMO2 AGU, San Francisco, Dec 2009 Jim Freer slide 8

9 UKCP09: 12 climate model ensembles of changes in summer rainfall Looking at single projections from different climate models. Used HadCM3 and had 280 variants in model structure and parameter combinations and hence is known as a perturbed physics ensemble AGU, San Francisco, Dec 2009 Jim Freer slide 9

10 Are GCM/RCM outputs reliable to assess the impacts of climate?... AGU, San Francisco, Dec 2009 Jim Freer slide 10

11 Simulations of LISFLOOD-RR using the climate change ensemble members... AGU, San Francisco, Dec 2009 Jim Freer slide 11

12 Concluding remarks... The use of multiple sources of data to evaluate models but diverse data (WT, sediment, NEE, etc.) has to be treated differently as their information content (error characteristics) and ability to represent physical quantities are different. If we want to regionalise or classify we know no two places are the same - Orwellian 4 catchments good 2 catchments bad Is more better if the different information content in data between places is not accounted for? A limits of acceptability approach forces the characterisation of the uncertainty in data to construct performance measures, this allows for improved model assessment and timestep based evaluation of multiple model structures. If we try to quantify the risk of climate change impacts we must assess the credibility of the ensemble members as part of the modelling cascade Current RCM s must be used with extreme caution to assess hydrological impacts We would like to work with other groups on these methods and challenges... AGU, San Francisco, Dec 2009 Jim Freer slide 12

13 Some recent papers associated with these methods Liu, Y. L., J. Freer, K. Beven, and P. Matgen (2009), Towards a limits of acceptability approach to the calibration of hydrological models: Extending observation error, Journal of Hydrology, 367(1-2), Dean, S., J. E. Freer, K. J. Beven, A. Wade, and D. Butterfield (2009), Uncertainty assessment of a process-based integrated catchment model of phosphorus, Stochastic Environmental Research and Risk Assessment, He, Y., F. Wetterhall, H. L. Cloke, F. Pappenberger, M. Wilson, J. Freer, and G. McGregor (2009), Tracking the uncertainty in flood alerts driven by grand ensemble weather predictions, Meteorological Applications, 16(1), Krueger, T., J. N. Quinton, J. Freer, C. J. A. Macleod, G. S. Bilotta, R. E. Brazier, P. Butler, and P. M. Haygarth (2009), Uncertainties in data and models to describe event dynamics of agricultural sediment and phosphorus transfer, Journal of Environmental Quality, 38(3), Younger, P. M., J. E. Freer, and K. J. Beven (2009), Detecting the effects of spatial variability of rainfall on hydrological modelling within an uncertainty analysis framework, Hydrological Processes, 23(14), Due out shortly... Di Baldassarre, G., G. Schumann, P. Bates, J. Freer, and K. J. Beven (In press), Floodplain mapping: A critical discussion of deterministic and probabilistic approaches, Hydrological Sciences Journal-Journal Des Sciences Hydrologiques. McMillan, H., J. Freer, F. Pappenberger, T. Krueger, and M. Clark (In press), Impacts of uncertain flow data on rainfall-runoff model calibration and discharge predictions, Hydrological Processes. Mitchell, S. M., K. J. Beven, and J. E. Freer (In press), Multiple sources of predictive uncertainty in modelled estimates of net ecosystem CO2 exchange, Ecological Modelling. This research is primarily funded by NERC and Defra with additional funds via international collaborations AGU, San Francisco, Dec 2009 Jim Freer slide 13

14 Paper updates McMillan, H., J. Freer, F. Pappenberger, T. Krueger, and M. Clark (2010), Impacts of uncertain river flow data on rainfall-runoff model calibration and discharge predictions, Hydrological Processes, 24(10), Mitchell, S., K. Beven, and J. Freer (2009), Multiple sources of predictive uncertainty in modeled estimates of net ecosystem CO2 exchange, Ecological Modelling, 220(23), AGU, San Francisco, Dec 2009 Jim Freer slide 14