Model Calibration and Sensitivity Analysis

Transcription

1 Model Calibration and Sensitivity Analysis

2 Calibration Criteria 1. Mean Error (ME) n 1 ME = cal obs n i= 1 ( ) 2. Mean Absolute Error (MAE) n 1 MAE = cal obs n i= 1 3. Root Mean Squared (RMS) RMS n = i n i= 1 i i 1 2 ( cal obsi) 4. Correlation Coefficient (ϒ) γ = n n i i 12 ( cali cal)( obsi obs) i= 1 2 ( cali cal) ( obsi obs) i= 1 i= 1 n 2 A Framework for Model Applications 1

3 Presentation of Calibration Results qualitative comparison of calculated and observed contour maps tabulated results with summary statistics scatter diagrams comparison of observed and calculated breakthrough curves spatial distribution of residual errors comparison of observed and calculated mass distributions A Framework for Model Applications 2

4 Observed concentrations superimposed on calculated contours A Framework for Model Applications 3

5 A Framework for Model Applications 4

6 65 Predicted Head (m-amsl) Observed Head (m-amsl) A Framework for Model Applications 5

7 Comparison of calculated and observed discharge from 4 pumping wells A Framework for Model Applications 6

8 Scatter Diagram Comparison of breakthrough curves A Framework for Model Applications 7

9 NATS Site, Columbus, MS 300 MLS Boundary lnk (cm/s) y (m) 100 Abandoned Meander y (m) Source Trench Groundwater Flow x (m) 0 Source Trench x (m) K (cm/s) 62 Source Trench z (m-amsl) y (m) (Julian et al., 2001) A Framework for Model Applications 8

10 50 Observed Predicted (a) y (m) 20 MLS Source Trench (b) y (m) x (m) x (m) Bromide (ppm) Observed and Calculated Bromide Concentrations A Framework for Model Applications 9

11 Calculated Concentration (ppm) Snapshot 3 Snapshot Observed Concentration (ppm) Scatter diagram of observed versus calculated bromide peak concentrations. Snapshots 3 and 4 correspond to observation times of 152 and 278 days after source emplacement. A Framework for Model Applications 10

12 Methods for Model Calibration 1 by trial-and-error procedures a) select one or more calibration criteria b) adjust one input parameter at a time c) compare values of calibration criteria 2 by automated procedures a) parameter values b) parameter structures c) available codes 3 points to ponder a) perform transient calibration, if possible at all b) calibrate flow rates, if possible c) much can be gained from calibration against transport data A Framework for Model Applications 11

13 Model Calibration as an Optimization Problem Minimize objective function ( ) S = ω obs cal i i i subject to one or more specified constraints 2 Example of simple objective function A Framework for Model Applications 12

14 Example of more complex objective function with multiple local optima A Framework for Model Applications 13

15 Illustrative Example 6000 No-flow boundary Y Axis (m) Specified-flow boundary (Q=0.25 m /day) Observation well Zone 1 Zone 3 P2 P3 Zone 2 P1 Pumping well No-flow boundary Constant head boundary (h=100 m) X Axis (m) Figure 4-1. Configuration of the two-dimensional test problem. The solid dots and open circles indicate the locations of pumping and monitoring wells where hydraulic heads and/or concentrations are used to estimate hydraulic conductivity in the three zones. A Framework for Model Applications 14

16 Zone 1 Zone 2 True head distribution Y Axis (m) 2000 Zone X Axis (m) Concentration (ppm) True concentration data at 3 wells Time (days) P1 P2 P3 A Framework for Model Applications 15

17 Objective Functions for Parameter Estimation Head data only NH i= 1 ( ˆ ) 2 Minimize S = α h h i i i Both head and concentration Data NH NC ( ˆ ) β ( ˆ ) 2 2 Minimize S = α h h + C C i i i j j j i= 1 j= 1 K 1 (m/d) K 2 K 3 Obj. Func. # Generations True H only H & C A Framework for Model Applications 16

18 Sensitivity Analysis sensitivity coefficient measure of the effect of change in one factor on another factor y$ i y$ i X ik, = a a k k normalized with respect to a k, y$ i y$ i X ik, = a a a a k k k k dimensionless, y$ i y$ i y$ i y$ X ik, = a a a a k k k i k A Framework for Model Applications 17

19 Procedure for Sensitivity Analysis base case: calibrated model change a parameter by a certain percentage from the base case ( a/a) run the model again calculate the change in model response ( y) which could be any variable of interest, such as head, flow rate, concentration at a receptor, etc. calculate sensitivity coefficient and evaluate the results A Framework for Model Applications 18

20 Examples of Sensitivity Analysis A Framework for Model Applications 19

21 A Framework for Model Applications 20

22 A Framework for Model Applications 21

23 N (a) BTEX (d) Ferrous iron m 300 ft (b) Oxygen (e) Sulfate (c) Nitrate (f) Methane Observed plumes of BTEX and electron acceptors at the Hill Air Force Base (Lu et al., 1999) Results of sensitivity analyses for total BTEX mass and BTEX plume front (Lu et al., 1999) Sensitivity Coefficient Total BTEX mass K Recharge Long. Disp. Retard. Factor Degradation rate constants DO Nitrate Ferrous iron Sulfate methane Front of BTEX plume a a Sensitivity analysis performed on 0.05 mg/l contour line of BTEX plume A Framework for Model Applications 22

24 Prediction and Uncertainty simulation of future conditions evaluation/assessment vs. prediction sources of uncertainty conceptual related to the mathematical model geological harder to quantify stress and other conventional parameters targets of most uncertainty analysis studies A Framework for Model Applications 23

25 Methods for Uncertainty Analysis sensitivity analysis simple, straightforward, but cannot examine correlation between parameters first-order error analysis in the simplest form N i= 1 [ ] = [ ][ ] Var y Var x y x approximation i i 2 Monte Carlo simulation most general, but computationally intensive A Framework for Model Applications 24

26 A Framework for Model Applications 25

27 A Framework for Model Applications 26

28 Procedure for Monte Carlo Analysis A Framework for Model Applications 27

29 Example of Monte Carlo Analysis Woldt et al. (1992) A Framework for Model Applications 28

30 Only K as random variable Only Initial Plume as random variable Both K and Initial Plume as random variables A Framework for Model Applications 29