Data, Models and Model Calibration

Transcription

1 Global Conference on G WADI more than a decade enhancing water and sustainable development for arid regions, Beijing, China, October 25 27, 2016 Data, Models and Model Calibration Qingyun Duan GCESS/Beijing Normal University October 26, 2016

2 A Plethora of Hydrological Models with Different Aims and Complexities F Interception P Precipitation Surface Runoff t UPPER ZONE PERCOLATION EVAPOTRANSPIRATION TENSION FREE INTERFLOW INFILTRATION SURFACE RUNOFF DIRECT RUNOFF Streamflow Discharge q LOWER ZONE PRIMARY FREE TENSION TENSION SUPPLE- MENTAL FREE API Model Q R Q B t Sacramento Model BASEFLOW SUBSURFACE OUTFLOW RESERVED RESERVED VIC Model Mike SHE Model

3 How to Improve the Performance of Weather/Climate Models? Enhance the Model Physics Better models Higher space/time resolution Better numerical schemes Enhance the Representations of External Forcings and Boundary Conditions Better Observational Systems Better Data Assimilation Methods Enhance the Estimation of Model Parameters Quantifying parameter uncertainties Tuning model parameters so simulations match observations

4 Hydrologic Modeling: 3 Elements! DATA MODEL Model Calibration

5 The Three Horsemen of Model Improvement The Modeler The Model Calibrator The Data Assimilator

6 Illustrating Model Calibration Forcing Inputs Real World Observed Outputs Y t MODEL ( ) Computed Outputs + - t Prior Info Optimization Procedure Calibration: constraining the model simulations to be consistent with observations

7 Challenges in Automatic Calibration of Large Complex Geophysical Models High dimensionality of the uncertain parameters (10 s 100 s) High dimensionality of the model outputs (can be millions) Difficult to prescribe parameter uncertainties (the priors) Models may be expensive to evaluate (many CPU hours) Complex models show highly nonlinear (may be discontinuous) inputoutput relationships Data scarcity for the full system (difficult to calibrate) Models are often created by data far from operating conditions extrapolation may be needed Unknown unknowns can greatly complicate the UQ process.

8 A Model Calibration Methodology For Large Complex Geophysical Models Preparation Parameter Screening Surrogate Modeling UQ Analyses

9 Uncertainty Quantification Python Laboratory (UQ PyL) pyl.com

10 What is UQ PyL? A new, general purpose, cross platform UQ framework with a GUI Made of several components that perform various functions, including Design of Experiments Statistical Analysis Sensitivity Analysis Surrogate Modeling Parameter Optimization; Suitable for parametric uncertainty analysis of any computer simulation models

11 The Front page of UQ PyL

12 The Interactive Page of UQ PyL Variable Explorer Editor Command Windows

13 ANALYSIS AND OPTIMIZATION OF PARAMETRIC UNCERTAINTY OF WRF MODEL

14 The Weather Research and Forecasting Model Weather and Research Forecast (WRF) is a widely used regional weather and climate modeling system. The model includes seven major physical processes: Microphysics Cumulus Cloud Surface Layer Land Surface Planetary Boundary Layer Longwave Radiation Shortwave Radiation 2 level nested grids Level 1: 27km, grids Level 2: 9km, 87x55 grids

15 WRF Model Parameters To Be Examined number scheme name Default range description 1 xka [ ] The parameter for heat/moisture exchange coefficient Surface layer 2 (module_sf_sfclay.f) The coefficient for coverting wind speed to roughness CZO [ ] length over water 3 pd 0 [-1 1] The coefficient related to downdraft mass flux rate 4 pe 0 [-1 1] The coefficient related to entrainment mass flux rate 5 Cumulus ph 150 [50 350] Starting height of downdraft above USL 6 (module_cu_kfeta.f) TIMEC 2700 [ ] Compute convective time scale for convection the maximum turbulent kinetic energy (TKE) value 7 TKEMAX 5 [3 12] between the level of free convection (LFC)and lifting condensation level (LCL) 8 ice_stokes_fac [ ] Scaling factor applied to ice fall velocity 9 n0r [ ] Intercept parameter rain Microphysics 10 dimax [ ] The limited maximum value for the cloud-ice diameter (module_mp_wsm6.f) 11 peaut 0.55 [ ] Collection efficiency from cloud to rain auto conversion 12 short wave radiation cssca [ ] Scattering tuning parameter in clear sky 13 (module_ra_sw.f) Beta_p 0.4 [ ] Aerosol scattering tuning parameter 14 Longwave (module_ra_rrtm.f) Secang 1.66 [ ] Diffusivity angle 15 hksati 0 [-1 1] hydraulic conductivity at saturation 16 Land surface porsl 0 [-1 1] fraction of soil that is voids 17 (module_sf_noahlsm.f) phi0 0 [-1 1] minimum soil suction 18 bsw 0 [-1 1] Clapp and hornbereger "b" parameter 19 Brcr_sbrob 0.3 [ ] Critical Richardson number for boundary layer of water 20 Brcr_sb 0.25 [ ] Critical Richardson number for boundary layer of land Planetary Boundary Profile shape exponent for calculating the momentum 21 Layer pfac 2 [1 3] diffusivity coefficient (module_bl_ysu.f) Coefficient for prandtl number at the top of the surface 22 bfac 6.8 [ ] laer Countergradient proportional coefficient of nonlocal 23 sm 15.9 [12 20] flux of momentum moh 2002

16 Forecasted Events Jun Jul Aug

17 The Experimental Setup: Model Setup 2 Level nested grids: Level 1: 27 km, with 60x48 grids Level 2: 9 km, with 87x55 grids Nine 5 day forecasts during Jun Aug from NCEP reanalysis data used to initiate the forecasts 23 WRF model parameters examined for study their sensitivity with respect to precipitation forecast Sensitivity method used: Morris One At a Time (MOAT) Optimization method used: Adaptive Surrogate Modeling based Optimization (ASMO) Computational cost 4.5 CPUs for one 5 day forecast Nine 5 day forecasts require 180 CPUs

18 Sensitivity Results For Precipitation Sensitive parameters for precipitation: P3, P4, P5, P12, P16, P21

19 Sensitivity Results For Air Temperature Sensitive parameters for temperature:p3,p5,p12,p16,p20,p21

20 Summary of Parameter Sensitivities to Different Model Outputs [Jiping Quan et.al. 2015, Submitted RQJMet]

21 Automatic Optimization of WRF Model: The Experiment Setup Adaptive Surrogate Modeling based Optimization (ASMO) method is used to optimize the eight most sensitive parameters found by global sensitivity analysis: Parameter optimized: P3 P4 P5 P8 P10 P12 P16 P21 GP surrogate model is created with 100 initial samples generated using LPtau design Adaptive search is then conducted to update the GP surrogate model (i.e., by adding more samples points based on existing response surface) Performance measures Used: Mean Absolute Error (MAE): Thread Score (TS) Bias Score SAL(Structure, Amplitude, Location) Three Optimization Runs: Optimize P only Optimize SAT only Optimize both P and SAT

22 The Optimization Results

23 Improvement in Performance Measure MAE

24 Improvement in Performance Measure Based on Lead times

25 Improvement in Performance Measure TS TS NA NA NB NC Obs\Fcst Yes No Yes NA NC No NB ND Category Threshold:mm Light Rain (0.1,10] Moderate Rain (10, 25] Heavy Rain (25,50] Storm (50,100] Severe Storm (100,250]

26 Improvement in Performance Measure Other Scores (Bias & SAL)

27 The Validation Events Calibration events: Dashed lines Validation events: Solid lines

28 Improvement in Validation Events

29 Improvement in Validation Events

30 Summary and Discussion of WRF Parametric Uncertainty and Optimization Research Considerable parametric uncertainties exist in NWP models UQ PyL contains tools for screening and optimizing important parameters The most sensitive parameters identified for precipitation and surface air temperature are: P3, P4, P5, P8, P12, P16, P18, and P21 Optimization experiments with the eight most sensitive parameters for 8 calibrated events has improved the model performance by 14 18% Other performance measures for calibrated events confirmed the improvement Validation using 15 independent storm data shows an improved model performance by 18 21%

31 Questions?