Bayesian Approaches for Model Evaluation and Uncertainty Estimation

Transcription

1 Bayesian Approaches for Model Evaluation and Uncertainty Estimation Tsuyoshi TADA National Defense Academy, Japan APHW 2006 Bangkok 1

2 Introduction In wellgauged basins Parameters can be obtained by calibrations. In ungauged or poorgauged basins Parameters have to be estimated according to Physical knowledge (conductivity, roughness, remotesensing, etc ) Logical or theoretical knowledge (upscaling, downscaling, etc ) Stochastic or statistical knowledge (range, mean value, covariance, probability, etc ) Combination of these method 2

3 Introduction Bayesian approaches for parameter estimation and uncertainty analysis have been proposed. Integration of prior knowledge and observation Improvement of prediction performance in poorgauged basin Purpose of this study Which model should we select? Which model can utilize prior knowledge effectively? New evaluation methodology or criterion is needed. 3

4 Model Structure (lumped conceptual model) P E P E 1BFI P E A11 S k, p C BFI S surf H1 B1 A12 Z12 A2 Z11 S = kq p 1K surf H2 Z2 S base B2 A3 Storage function (simplified) 1K base H3 B3 Z3 2 parameters (0 initial conditions) AWBM (Australian Water Balance Model) H4 A4 6 parameters (2 initial conditions) Sugawara s Tank model 16 parameters (4 initial conditions) 4

5 Bayesian methodology Based on the Generalised Likelihood Uncertainty Estimation (Beven and Binley,1992) No single optimum parameter set exists (equifinality) Montecarlo simulation (multiple parameter sets are considerd) Bayesian inference L( y θ) π ( θ) P( θ y) = C π (θ) : Prior distribution Probability density of parameter set θ (prior knowledge) L (y θ) : Likelihood distribution Likelihood of observation data y with given parameter θ P (θ y) : Posterior distribution Probability density of parameter θ with given observation y Details are described later 5

6 Scheme of this study Observation Target basin s observation data y Prior knowledge Training basins optimum parameter sets θ k opt Optimimum parameter set θ opt (y) Likelihood distribution L (y θ) P ( θ y) L = ( y θ) π ( θ) C Prior distribution π (θ) Posterior distribution P (θ y) Target basins Prediction Target basin s Prediction Target basin s Prediction 6

7 Study Area Target Basins (ungauged or poorgauged) Training Basins (wellgauged) km 2 chatchment area Coverd with forest over 5080% Contain no dam In the same climate (temperate zone) Little effect from snow Japane Iwai, 555km 2 Ohnogi, 304km 2 Bansho, 278km 2 7

8 Data Set Target Basins for calibration and likelihood distribution for validation Training Basins for prior distribution Long term prediction in poorgauged basin is assumed Calibration period is 1 year (1992). Prediction and validation period is 10 years ( ). The target region is wellgauged in whole. 8

9 Training Basins Optimum Parameter sets 42 optimum parameter sets are calculated 14 basins x 3 years SCEUA and PSO (Particle Swarm Optimization) algorithm Storage function Tank model AWBM 9

10 Generating Prior Distribution Prior distribution π (θ) is generated by optimum parameter sets of training basins. Storage function Tank model p B2 Z3 log(k ) Multivariate normal distribution The 1st order approximation (easy to handle) Equifinality can be reflected ( ) N ( ) B1 Z2 1 1 T ( ) ( ) p 2 10 π θ = θ, = exp θ θ θ θ ( 2π )

11 Generating Likelihood Distribution Likelihood distribution L (y θ) is generated by observation data y of target basins. Error model (Gaussian error model) The residual is normally distributed. y i = M i (θ) + N (0, σ 2 ) Error variance σ is proportional to the data variance. Likelihood function Respecting the rules of Bayesian inference When the additional observation becomes available, likelihood distribution can be updated. L yt + T θ = L yt θ L yt θ ( ) ( ) ( ) n 2 2 { ( )} { ( )} 1 yi M y M i θ θ L( y θ) = T T E( θ y) n i i i= 1 exp exp α exp α, 2 n i= 1 2πσ 2σ 2 ( yi y ) i= 1 11

12 Generating Posterior Distribution Posterior distribution is generated from π (θ) and L (y θ). Sampling 10,000 montecarlo samples θ by MCMC Calculating prior distribution π (θ) Calculating simulated streamflow of all samples Evaluating likelihood distribution L (y θ) Using observation data y in the calibration period Evaluating the posterior distribution P (θ y) P (θ y) = L (y θ) π (θ) / C P L π = x θ θ θ 12

13 Prediction from Posterior distribution Optimum parameter set Montecarlo samples Posterior distribution P(θ y) q P weighted predictions θ 2 θ 1 t q Optimum prediction Likelihood weighted ensemble mean Confidence interval Empirical Probability density t 13

14 Result (Storage function) E = 0.61 E = 0.65 E = 0.19 shrink adapt Optimum parameter set θ opt (y 92 ) Posterior distribution P(θ y 92 ) Prior distribution π (θ) In most periods, π (θ) shows the best fitting. However, only few peaks are overestimated eminently. 14

15 Result (AWBM) E = 0.70 E = 0.69 E = 0.68 Optimum parameter set θ opt (y 92 ) Posterior distribution P(θ y 92 ) Prior distribution π (θ) Mathematically optimum parameter set is not qualitatively optimum. 15

16 Result (Tank model) E = 0.64 E = 0.68 E = 0.73 Optimum parameter set θ opt (y 92 ) Posterior distribution P(θ y 92 ) Prior distribution π (θ) Insufficient calibration data is damaging for prediction. 16

17 Result Nashsutcliffe efficiency of Calibration(1992) and Prediction ( ) Storage function AWBM Tank model Optimum parameter set Posterior distribution Prior distribution Calibration with 1 year data Calibration with sufficient data AWBM is best for prediction for conventional prediction. Tank model is best for prediction with prior knowledge. Tank model is NOT best for conventional prediction Too complex to predict with insufficient observation data 17

18 BIC (Bayesian information criterion) Marginal distribution largest is the best model ( ) ( ) π ( ) p y = L y θ θ dθ BIC (laplace approximation) smallest is the best model { L( ) π ( ) d } L( ˆ ) k ( n) BIC = 2log y θ θ θ 2log y θ + log SIC (Schwarz s information criterion; Gaussian error model) RSE SIC = nlog + klog n n ( ) n: number of data k: number of parameter RSE: root square error 18

19 Result (SIC) Storage function AWBM Tank model Optimum parameter set Posterior distribution Prior distribution Calibration accuracy SIC SIC may represent the model performance with prior knowledge. Storage function is misevaluated. Further research is required 19

20 Summary Prior knowledge is represented by the probability density (prior distribution) of optimum parameter sets in training basins, and is approximated by multivariate normal distribution. Posterior distribution is calculated with the prior distribution and observation data based on bayesian inference. Prediction of streamflow and its uncertainty is calculated from the prior and the posterior distribution. Prediction performance of these models are evaluated. Prediction performance with prior knowledge can be evaluated by SIC (BIC). 20

21 Thank you for your attention 21