Automatic Optimization of WRF Model Parameters

Transcription

1 Automatic Optimization of WRF Model Parameters Qingyun Duan BNU Hydrology Group July 12, 2014

2 Effect of Post-processing on CRPSS in Huaihe Basin (B3) CRPSS of Raw Ensemble Forecasts CRPSS of Post-process Ensemble Forecasts CMA ECMWF JMA NCEP UKMO Period Forecast days Day 1 Day 2 Day 3 Day days 1 3 days 1 4 days 5 6 days 7 9 days 5 9 days 1 9 days

3 Modeling The Weather/Climate System

4 How to Improve the Performance of Weather/Climate Models? Enhance the Model Physics Better models(e.g., AMIPs, CMIPs, WRF, etc.) Higher space/time resolution Better numerical schemes Enhance the Representations of External Forcings and Boundary Conditions Better Observations and Assimilation of Observations A NEW APPROACH - Enhance the Estimation of Model Parameters Tuning model parameters so simulations match observations

5 The Three Horsemen of Model Improvement The Modeler The Model Calibrator The Data Assimilator

6 Calibration of the Single Column Climate Model (SCCM) (Bastidas et al., 2003) (a) Simulations before calibration (b) Simulations after calibration LDRD 2004.Duanl, et al

7 Challenges in Optimization of Large Complex Dynamical System Models Difficult to prescribe parameter uncertainties (the priors) High-dimensionality of the uncertain parameters (10 s -100 s) High-dimensionality of the model outputs (can be millions) Models may be expensive to evaluate (many CPU-hours) Complex models show highly nonlinear (may be discontinuous) inputoutput relationships Data scarcity for the full system (difficult to calibrate) Models are often created by data far from operating conditions extrapolation may be needed Model-specific uncertainties are difficult/expensive to quantify Unknown unknowns can greatly complicate the UQ process.

8 A UQ Methodology For Large Scale Dynamical System Models Problem specification (model, variables) Expert judgment diligence Characterize parameter/model uncertainties Derive credible ranges Shapes and forms Parameter Screening: stage I For nparams >> 100 Single effect analysis Parameter Screening: stage II Response surface analysis For nparams ~ 100 e.g. use MOAT/GP/MARS (multi-algorithmic) For expensive models ~10 (use MARS,ANN,SVM,GP) Calibration Quantify uncertainty, Sensitivity, reliability Design optimization/ exploration

9 Step 1: PARAMETER SCREENING

10 Parameter Importance based on Sobol Variance Decomposition*. * 2000 Samples are used [Jianduo Li et.al. 2013, HESS]

11 Step 2: SURROGATE MODELING

12 What Is A Surrogate Model? The real world A system stimulus / response The dynamic simulation model Abstraction of the real world Based on physical processes, high computational complexity The surrogate model A model of the model Response surface, meta-model, emulator Based on statistical theory, low computational complexity

13 Step 3: ADAPTIVE SURROGATE MODELING BASED OPTIMIZATION

14 Objective function How Does ASMO Work? True response surface Input: 1.Parameters to be optimized 2.Objective function Design of Experiment Model Simulation Add sampling points Build surrogate model Assessing model Parameter value Adaptive sampling Surrogate based optimization [Chen Wang et.al. 2013, EMS]

15 Case Study: ANALYSIS AND OPTIMIZATION OF PARAMETRIC UNCERTAINTY OF WRF MODEL

16 Analysis of Parametric Uncertainty of WRF Model Weather and Research Forecast (WRF) is a widely used regional weather and climate modeling system. The model includes seven major physical processes: Microphysics Cumulus Cloud Surface Layer Land-Surface Planetary Boundary Layer Longwave Radiation Shortwave Radiation

17 Definition of the Problem Many processes and many choices: Microphysics Long-wave radiation Short-wave radiation Surface layer Land surface PBL Cumulus There are = (combinations) Problems: What physical processes are most sensitive to the meteorological variables of interest? Which combinations of physical parameterization schemes are the best? Given a combination of physical parameterization scheme, how can we find the most sensitive parameters and their optimal values?

18 The Study Domain 北京 2-level nested grids Level 1: 27km, grids Level 2: 9km, 87x55 grids

19 2008 Forecasted Events Jun Jul Aug

20 WRF Model Parameters To Be Examined number scheme name Default range description 1 xka [ ] The parameter for heat/moisture exchange coefficient Surface layer 2 (module_sf_sfclay.f) The coefficient for coverting wind speed to roughness CZO [ ] length over water 3 pd 0 [-1 1] The coefficient related to downdraft mass flux rate 4 pe 0 [-1 1] The coefficient related to entrainment mass flux rate 5 6 Cumulus (module_cu_kfeta.f) ph TIMEC [50 350] [ ] Starting height of downdraft above USL Compute convective time scale for convection the maximum turbulent kinetic energy (TKE) value 7 TKEMAX 5 [3 12] between the level of free convection (LFC)and lifting condensation level (LCL) 8 ice_stokes_fac [ ] Scaling factor applied to ice fall velocity 9 n0r [ ] Intercept parameter rain Microphysics 10 dimax [ ] The limited maximum value for the cloud-ice diameter (module_mp_wsm6.f) 11 peaut 0.55 [ ] Collection efficiency from cloud to rain auto conversion 12 short wave radiation cssca [ ] Scattering tuning parameter in clear sky 13 (module_ra_sw.f) Beta_p 0.4 [ ] Aerosol scattering tuning parameter 14 Longwave (module_ra_rrtm.f) Secang 1.66 [ ] Diffusivity angle 15 hksati 0 [-1 1] hydraulic conductivity at saturation 16 Land surface porsl 0 [-1 1] fraction of soil that is voids 17 (module_sf_noahlsm.f) phi0 0 [-1 1] minimum soil suction 18 bsw 0 [-1 1] Clapp and hornbereger "b" parameter 19 Brcr_sbrob 0.3 [ ] Critical Richardson number for boundary layer of water 20 Brcr_sb 0.25 [ ] Critical Richardson number for boundary layer of land Planetary Boundary Profile shape exponent for calculating the momentum 21 Layer pfac 2 [1 3] diffusivity coefficient (module_bl_ysu.f) Coefficient for prandtl number at the top of the surface 22 bfac 6.8 [ ] laer Countergradient proportional coefficient of nonlocal flux of momentum moh 23 sm 15.9 [12 20] 2002

21 The Experimental Setup (1): Model Setup 2-Level nested grids Level 1: 27 km, with 60x48 grids Level 2: 9 km, with 87x55 grids Nine 5-day forecasts during Jun-Aug from st day as spin-up, last 4 day results analyzed NCEP reanalysis data used to initiate the forecasts 23 WRF model parameters examined for study their sensitivity with respect to precipitation forecast Computational cost 4.5 CPUs for one 5-day forecast Nine 5-day forecasts require 180 CPUs

22 The Experimental Setup (2) Validation Datasets Variable name Table 1 Ground observation data products Precipitation, Temperature, Wind speed, Wind direction, Humidity, Pressure, Downward shortwave radiation, upward shortwave radiation Variable name Cloud Fraction Product name Horizontal resolution Temporal resolution time range source 0.05 o 3 hours BNU Zheng Group (1/16) o 1hour CMA Table 2 Other observation data products MOD06_L2-Level 2 Cloud Product; MYD06_L2-Level 2 Cloud Product; Horizontal resolution 5km 5km Temporal resolution Time-varying time range MOD: MYD: Source gov/data/search.html Total Precipitable Water Aqua AIRS Level 2 Standard Physical Retrieval (AIRS+AMSU) (AIRX2RET.006) 50km 50km Time-varying Boundary Layer Height MERRA Chem 2D IAU Diagnostics, Fluxes and Meteorology, Time Average 3-hourly (MAT3FXCHM.5.2.0) hours Upward long wave radiation at top of the atmosphere FY-2D 卫星 9210 格式日平均射出长波辐射产品 day ortalsite/data/satellite.aspx

23 The Experimental Setup (3) Analysis Method Sensitivity Analysis method used: Morris One-At-a-Time (MOAT) Objective function used: MAE 1 n n i 1 Sim i Obs i sim i and obs i are the forecasted and observed daily precipitation at i th grid Number of parameter samples used: 240 Total CPU hours: 240 x 180 = 43,000 hours

24 MOAT Results Precipitation based on lead times: Sensitive parameters for precipitation: P3, P4, P5, P12, P16, P21

25 MOAT Results Precipitation based on storm types: Sensitive parameters for precipitation: P3, P4, P5, P12, P16, P21

26 MOAT Normalized Results - Precipitation: All parameters normalized to [0 1] range, with purple red indicating sensitive, cyan indicating insensitive. Sensitive parameters found: P3 P4 P5 P12 P16 P21 Cumulus: P3 P4 P5; Shortwave radiation: P12; Land surface: P16; Planatary BL: P21;

27 Summary of Parameter Sensitivities parameterization scheme cloud variable Planetary boundary Precipitable water layer height Upward long wave radiation at top of the atmosphere surface-layer cumulus P3,P4,P5,P7 P3 P3,P4,P5 P3,P4,P5 microphysics P8,P10 P8,P10 P8,P10 short wave radiation P12 P12 P12 P12 long wave radiation P14 land surface P16,P18 P16 P16 P16 boundary layer P21 P20,P21,P22 P21,P22 P21 Sensitive parameters for precipitation: P3, P4, P5, P12, P16, P21 Sensitive parameters for temperature: P3, P5, P12, P16, P20,P21 Sensitive Parameters for humidity: Sensitive Parameters for Wind: P3, P12, P16, P18, P20, P21 P3, P5, P12, P16, P20, P21

28 Optimization Experiment Setup Adaptive Surrogate Modeling based Optimization (ASMO) method is used to optimize the six most sensitive parameters found by global sensitivity analysis: Parameter optimized: P3 P4 P5 P8 P10 P12 P16 P21 GP surrogate model is created with 100 initial samples generated using LPtau design Adaptive search is then conducted to update the GP surrogate model (i.e., by adding more samples points based on existing response surface) Objective Function Used Root Mean Square Error (RMSE): RMSE 1 n n Sim i Obs i i 1 2

29 MAE The Optimization Result 7.20 The Optimization Convergence Process Function Evaluations

30 Initial Optimization Results: RMSE Values Rainfall (mm) (Default parameters) (Optimized parameters)

31 Improvement in Performance Skill Mean Daily Precipitation Daily Mean Absolute Error

32 The Validation Events Black box: Calibration Red box: Validation

33 Improvement in Validation Events Cumulative Precipitation of All Validation Events Improvement in MAE

34 Summary and Discussion of WRF Parametric Uncertainty Research 240 model runs are used to identify the most important parameters in WRF that exert great influence on precipitation forecasting skill in Beijing area The most sensitive parameters identified are: P3, P4, P5, P12, P16, and P21 Initial optimization experiment with the six most sensitive parameters has improved the model performance by >17% after 137 model runs Validation using independent storm data shows an improved model performance by ~10%

35 Special Acknowledgement Drs. Charles Tong and Yunwei Sun of Lawrence Livermore National Laboratory Natural Science Foundation of China (Grant #:No ) Chinese Ministry of Science and Technology 973 Research Program (Grant #: No. 2010CB428402) References: Charles Tong, (2005). PSUADE Users Manual, Rep., LLNL-SM Li J, Duan QY, Gong W, Ye A, et al., Assessing parameter importance of the Common Land Model based on qualitative and quantitative sensitivity analysis. Hydrol. Earth Syst. Sci. 17, Gan Y, et al., A comprehensive evaluation of various sensitivity analysis methods: A case study with a hydrological model. Environmental Modelling & Software 51, Wang, C. et al., Adaptive surrogate modeling based optimization approach, EMS