Bayesian model mergings for multivariate extremes Application to regional predetermination of oods with incomplete data

Transcription

1 Bayesian model mergings for multivariate extremes Application to regional predetermination of oods with incomplete data Anne Sabourin PhD defense University of Lyon 1 Committee: Anthony Davison, Johan Segers (Rapporteurs) Anne-Laure Fougères (Director), Philippe Naveau (Co-director), Clémentine Prieur, Stéphane Robin, Éric Sauquet. September 24 th,

2 Multivariate extreme values Risk management: Largest events, largest losses Hydrology: `ood predetermination'. Return levels (extreme quantiles) Return periods (1 / probability of occurrence ) digs, dams, land use plans. Simultaneous occurrence of rare events can be catastrophic Multivariate extremes: Probability of jointly extreme events? 2

3 Censored Multivariate extremes: oods in the `Gardons' joint work with Benjamin Renard Daily streamow data at 4 neighbouring sites : St Jean du Gard, Mialet, Anduze, Alès. Joint distributions of extremes? probability of simultaneous oods. Recent, `clean' series very short Historical data from archives, depending on `perception thresholds' for oods (Earliest: 1604). censored data Gard river Neppel et al. (2010) How to use all dierent kinds of data? 3

4 Multivariate extremes for regional analysis in hydrology Many sites, many parameters for marginal distributions, short observation period. `Regional analysis': replace time with space. Assume some parameters constant over the region and use extreme data from all sites. Independence between extremes at neighbouring sites? Dependence structure? Idea: use multivariate extreme value models 4

5 Outline Multivariate extremes and model uncertainty Bayesian model averaging (`Mélange de modèles') Dirichlet mixture model (`Modèle de mélange'): a re-parametrization Historical, censored data in the Dirichlet model 5

6 Multivariate extremes Random vectors Y = (Y 1,..., Y d, ) ; Y j 0 Margins: Y j F j, 1 j d (Generalized Pareto above large thresholds) Standardization ( unit Fréchet margins) X j = 1/ log [F j (Y j )] ; P(X j x) = e 1/x, 1 j d Joint behaviour of extremes: distribution of X above large thresholds? P(X A X A 0 )? (A A 0, 0 / A 0 ), A 0 `far from the origin'. X2 X A u 2 A 0 : Extremal region u 1 X1 6

7 Polar decomposition and angular measure Polar coordinates: R = d j=1 X j (L 1 norm) ; W = X R. W simplex S d = {w : w j 0, j w j = 1}. Angular probability measure: H(B) = P(W B) (B S d ). X 2nd component w B x S d W R B st component 7

8 Fundamental Result de Haan, Resnick, 70's, 80's Radial homogeneity (regular variation) P(R > r, W B R r 0 ) r 0 r 0 r H(B) (r = c r 0, c > 1) Above large radial thresholds, R is independent from W H (+ margins) entirely determines the joint distribution X1 X2 x w x X1 X2 x w x One condition only for genuine H: moments constraint w dh(w) = ( 1 d,..., 1 d ). Center of mass at the center of the simplex. Few constraints: non parametric family! 8

9 Estimating the angular measure: non parametric problem Non parametric estimation (empirical likelihood, Einmahl et al., 2001, Einmahl, Segers, 2009, Guillotte et al, 2011.) No explicit expression for asymptotic variance, Bayesian inference with d = 2 only. Restriction to parametric family: Gumbel, logistic, pairwise Beta... Coles & Tawn, 91, Cooley et al., 2010, Ballani & Schlather, 2011 : How to take into account model uncertainty? 9

10 Outline Multivariate extremes and model uncertainty Bayesian model averaging (`Mélange de modèles') Dirichlet mixture model (`Modèle de mélange'): a re-parametrization Historical, censored data in the Dirichlet model 10

11 BMA: Averaging estimates from dierent models Disjoint union of several parametric models Sabourin, Naveau, Fougères, 2013 (Extremes) Package R: `BMAmevt', available on CRAN repositories

12 Outline Multivariate extremes and model uncertainty Bayesian model averaging (`Mélange de modèles') Dirichlet mixture model (`Modèle de mélange'): a re-parametrization Historical, censored data in the Dirichlet model 12

13 Dirichlet distribution w S d, diri(w µ, ν) = Γ(ν) d i=1 Γ(νµ i) d i=1 w νµ i 1 i. µ S d : location parameter (point on the simplex): `center'; ν > 0 : concentration parameter. w w3 w1 13

14 Dirichlet distribution w S d, diri(w µ, ν) = Γ(ν) d i=1 Γ(νµ i) d i=1 w νµ i 1 i. µ S d : location parameter (point on the simplex): `center'; ν > 0 : concentration parameter. w w3 w1 13

15 Dirichlet mixture model Boldi, Davison, 2007 µ = µ,1:k, ν = ν 1:k, p = p 1:k, ψ = (µ, p, ν), k h ψ (w) = p m diri(w µ, m, ν m ) m=1 Moments constraint on (µ, p): k p m µ.,m = ( 1 d,..., 1 d ). m=1 Weakly dense family (k N) in the space of admissible angular measures 14

16 Bayesian inference and censored data Two issues : (i) parameters constraints (ii) censorship (i) Bayesian framework: MCMC methods to sample the posterior distribution. Constraints Sampling issues for d > 2. Re-parametrization: No more constraint, tting is manageable for d = 5: Sabourin, Naveau, 2013 (ii) Censoring: data points but segments or boxes in R d. Intervals overlapping threshold: extreme or not? Likelihood: density dr dh(w) integrated over boxes. r 2 Sabourin, under review ; Sabourin, Renard, in preparation 15

17 Bayesian inference and censored data Two issues : (i) parameters constraints (ii) censorship (i) Bayesian framework: MCMC methods to sample the posterior distribution. Constraints Sampling issues for d > 2. Re-parametrization: No more constraint, tting is manageable for d = 5: Sabourin, Naveau, 2013 (ii) Censoring: data points but segments or boxes in R d. Intervals overlapping threshold: extreme or not? Likelihood: density dr dh(w) integrated over boxes. r 2 Sabourin, under review ; Sabourin, Renard, in preparation 15

18 Re-parametrization: intermediate variables (γ 1,..., γ k 1), partial barycenters ex: k = 4 γ 0 γ m : Barycenter of kernels 'following µ.,m : µ.,m+1,..., µ.,k. γ m = ( ) 1 p j p j µ.,j j>m j>m 16

19 γ 1 on a line segment: eccentricity parameter e 1 (0, 1). ex: k = 4 I1 γ0 γ1 µ1 Draw (µ,1 S d, e 1 (0, 1)) γ 1 dened by γ 0 γ 1 γ 0 I 1 = e 1 ; p 1 = γ 0 γ 1 µ,1 γ 1. 17

20 γ 2 on a line segment: eccentricity parameter e 2 (0, 1). ex: k = 4 I1 µ 2 γ1 γ2 I2 γ0 µ 1 Draw (µ,2, e 2 ) γ 2 : γ 1 γ 2 γ 1 I 2 = e 2 p 2 18

21 Last density kernel = last center µ,k. ex: k = 4 I3 µ4 I1 µ 2 γ1 γ2 I2 γ0 µ 3 µ 1 Draw (µ,3, e 3 ) γ 3 p 3, µ.,4 = γ 3. p 4 19

22 Summary I3 µ4 I1 µ 2 γ1 γ2 I2 γ0 µ 3 µ 1 Given (µ.,1:k 1, e 1:k 1 ), One obtains (µ.,1:k, p 1:k ). The density h may thus be parametrized by θ = (µ.,1:k 1, e 1:k 1, ν 1:k ). 20

23 Bayesian model Unconstrained parameter space : union of product spaces (`rectangles') } Θ = Θ k ; Θ k = {(S d ) k 1 [0, 1) k 1 (0, ] k 1 k=1 Inference: Gibbs + Reversible-jumps. Restriction (numerical convenience) : k 15, ν < ν max, etc... `Reasonable' prior `at' and rotation invariant. Balanced weight and uniformly scattered centers. 21

24 MCMC sampling: Metropolis-within-Gibbs, reversible jumps. Three transition types for the Markov chain: Classical (Gibbs): one µ.,m, e m or a ν m is modied. Trans-dimensional (Green, 1995): One component (µ.,k, e k, ν k+1 ) is added or deleted. `Shue': Indices permutation of the original mixture: Re-allocating mass from old components to new ones. 22

25 Results: model's and algorithm's consistency Ergodicity: The generated MC is φ-irréducible, aperiodic and admits π n (= posterior W 1:n ) as invariant distribution. Consequence: g C b (Θ), 1 T T g(θ t ) E πn (g). Key point: π n is invariant under the `shue' moves. Posterior consistency for π n under `weak conditions' 2, π-a.s., U weakly open containing θ 0, t=1 π n (U) n 1. Consequence: E πn (g) g(θ 0). n Key: The Euclidian topology is ner than the Kullback topology in this model. 2 If the prior grants some mass to every Euclidian neighbourhood of Θ and if θ 0 is in the Kullback-Leibler closure of Θ 23

26 Convergence checking (simulated data, d = 5, k = 4) θ summarized by a scalar quantity (integrating the DM density against a test function) Original algorithm standard tests: Re-parametrized version Stationarity (Heidelberger & Welch, 83) variance ratio (inter/intra chains, Gelman, 92) 24

27 Estimating H in dimension 3, simulated data MCMC: T 2 = ; T 1 = w w3 w1 25

28 Dimension 5, simulated data Angular measure density for one pair (T 2 = , T 1 = ) X2/(X2+X5) 0 1 X2/(X2+X5) Gelman ratio: Original version: 2.18 ; Re-parametrized: Credibility sets (posterior quantiles): wider. 26

29 Bayesian inference and censored data Two issues : (i) parameters constraints (ii) censorship (i) Bayesian framework: MCMC methods to sample the posterior distribution. Constraints Sampling issues for d > 2. Re-parametrization: No more constraint, tting is manageable for d = 5: Sabourin, Naveau, 2013 (ii) Censoring: data points but segments or boxes in R d. Intervals overlapping threshold: extreme or not? Likelihood: density dr dh(w) integrated over boxes. r 2 Sabourin, under review ; Sabourin, Renard, in preparation 27

30 Bayesian inference and censored data Two issues : (i) parameters constraints (ii) censorship (i) Bayesian framework: MCMC methods to sample the posterior distribution. Constraints Sampling issues for d > 2. Re-parametrization: No more constraint, tting is manageable for d = 5: Sabourin, Naveau, 2013 (ii) Censoring: data points but segments or boxes in R d. Intervals overlapping threshold: extreme or not? Likelihood: density dr dh(w) integrated over boxes. r 2 Sabourin, under review ; Sabourin, Renard, in preparation 27

31 Outline Multivariate extremes and model uncertainty Bayesian model averaging (`Mélange de modèles') Dirichlet mixture model (`Modèle de mélange'): a re-parametrization Historical, censored data in the Dirichlet model 28

32 Censored data: univariate and pairwise plots Univariate time series: St Jean Mialet Débit (m3/s) u Année Anduze Débit (m3/s) u Année Ales Débit (m3/s) u Année Débit (m3/s) u Année 29

33 Censored data: univariate and pairwise plots Bivariate plots: Streamflow at Mialet (m3/s) Streamflow at Anduze (m3/s) Streamflow at Ales (m3/s) Streamflow at StJean (m3/s) Streamflow at StJean (m3/s) Streamflow at StJean (m3/s) 29

34 Using censored data: wishes and reality Take into account as many data as possible Censored likelihood, integration problems Information transfer from well gauged to poorly gauged sites using the dependence structure Estimate together marginal parameters + dependence 30

35 Data overlapping threshold and Poisson model How to include the rectangles overlapping threshold in the likelihood? {( t n, X t n ), 1 t n } Poisson Process (Leb λ) on [0, 1] A u,n λ: ` exponent measure', with Dirichlet Mixture angular component dλ dr dw (r, w) = d r 2 h(w). Overlapping events appear in Poisson likelihood as [ { P N ( t 2 n t 1 n ) 1 } ] n A i = 0 = exp [ (t 2 t 1 )λ(a i )] 31

36 `Censored' likelihood: model density integrated over boxes Ledford & Tawn, 1996: partially extreme data censored at threshold, GEV models Explicit expression for censored likelihood. Here: idem + natural censoring Poisson model No closed form expression for integrated likelihood. Two terms without closed form: Censored regions A i overlapping threshold: exp { (t 2 t 1 )λ(a i )} Classical censoring above threshold censored region dλ dx. 32

37 Data augmentation One more Gibbs step, no more numerical integration. Objective: sample [θ Obs] likelihood (censored obs) Additional variables (replace missing data component): Z Full conditionals [Z i Z j j, θ, Obs], [θ Z, Obs],... explicit (Thanks Dirichlet): Gibbs sampling. Sample [z, θ Obs] + (augmented distribution) on Θ Z. 33

38 Censored regions above threshold Censored region dλ dx dx j1:j r : Generate missing components under univariate conditional distributions Z j 1:r [X missing X obs, θ] x2 Augmentation data Z j = [X censored X observed, θ] Censored interval u 2 /n Extremal region u 1 /n x1 Dirichlet Explicit univariate conditionals Exact sampling of censored data on censored interval 34

39 Censored regions overlapping threshold augmentation Poisson process N i on E i A i. e (t 2,i t1,i )λ(a i ) + Functional ϕ(n i ) X2 Augmentation Z i = PP(τ.λ) on E i φ(#{points in Ai}) U' 1 E i A i u 2 /n Censored region u 1 /n U' 2 X1 [z, θ Obs] }{{}... [N i ]ϕ(n i ) density terms, prior, augmented missing components 35

40 Simulated data (Dirichlet, d = 4, k = 3 components), same censoring as real data Pairwise plot and angular measure density (true/ posterior predictive) S h S X3/( X3 + X4 ) 36

41 Simulated data (Dirichlet, d = 4, k = 3 components), same censoring as real data Marginal quantile curves: better in joint model. S3 discharge : dependent independent true return period (years, log scale) 36

42 Angular predictive density for Gardons data h St Jean/(St Jean+Mialet ) h St Jean/(St Jean+Anduze ) h St Jean/(St Jean+Ales ) h Mialet/(Mialet+Anduze ) h Mialet/(Mialet+Ales ) h Anduze/(Anduze+Ales ) 37

43 Conditional exceedance probability P( Mialet > y St Jean > 300 ) P( Anduze > y St Jean > 300 ) P( Ales > y St Jean > 300 ) Condit. exceed. proba empirical probability of an excess empirical error posterior predictive posterior quantiles Condit. exceed. proba Condit. exceed. proba threshold (y) threshold (y) threshold (y) P( Anduze > y Mialet > 320 ) P( Ales > y Mialet > 320 ) P( Ales > y Anduze > 520 ) Condit. exceed. proba Condit. exceed. proba Condit. exceed. proba threshold (y) threshold (y) threshold (y) 38

44 Conclusion Building Bayesian multivariate models for excesses: Dirichlet mixture family: `non' parametric, Bayesian inference possible up to re-parametrization Censoring data augmenting (Dirichlet conditioning properies) Two packages R: DiriXtremes, MCMC algorithm for Dirichlet mixtures, DiriCens, implementation with censored data. High dimensional sample space (GCM grid, spatial elds)? Impose reasonable structure (sparse) on Dirichlet parameters Dirichlet Process? Challenges : Discrete random measure continuous framework 39

45 Bibliographie I M.-O. Boldi and A. C. Davison. A mixture model for multivariate extremes. JRSS: Series B (Statistical Methodology), 69(2):217229, Coles, SG and Tawn, JA Modeling extreme multivariate events JR Statist. Soc. B, 53:377392, 1991 Gómez, G., Calle, M. L., and Oller, R. Frequentist and bayesian approaches for interval-censored data. Statistical Papers, 45(2):139173, Hosking, J.R.M. and Wallis, J.R.. Regional frequency analysis: an approach based on L-moments Cambridge University Press, Ledford, A. and Tawn, J. (1996). Statistics for near independence in multivariate extreme values. Biometrika, 83(1): Neppel, L., Renard, B., Lang, M., Ayral, P., Coeur, D., Gaume, E., Jacob, N., Payrastre, O., Pobanz, K., and Vinet, F. (2010). Flood frequency analysis using historical data: accounting for random and systematic errors. Hydrological Sciences JournalJournal des Sciences Hydrologiques, 55(2): Resnick, S. (1987). Extreme values, regular variation, and point processes, volume 4 of Applied Probability. A Series of the Applied Probability Trust. Springer-Verlag, New York. 40

46 Bibliographie II Sabourin, A., Naveau, P. and Fougères, A-L. (2013) Bayesian model averaging for multivariate extremes. Extremes, 16(3) Sabourin, A., Naveau, P. (2013) Bayesian Dirichlet mixture model for multivariate extremes: a re-parametrization. Computation. Stat and Data Analysis Schnedler, W. (2005). Likelihood estimation for censored random vectors. Econometric Reviews, 24(2): Tanner, M. and Wong, W. (1987). The calculation of posterior distributions by data augmentation. Journal of the American Statistical Association, 82(398): Van Dyk, D. and Meng, X. (2001). The art of data augmentation. Journal of Computational and Graphical Statistics, 10(1):

47 Outline Multivariate extremes and model uncertainty Bayesian model averaging (`Mélange de modèles') Dirichlet mixture model (`Modèle de mélange'): a re-parametrization Historical, censored data in the Dirichlet model 42

48 Bayesian Model Averaging: reducing model uncertainty. Parametric framework: arbitrary restriction, dierent models can yield dierent estimates. First option: Fight! Choose one model (Information criterions: BIC/ AIC /AICC) 43

49 Bayesian Model Averaging: reducing model uncertainty. Parametric framework: arbitrary restriction, dierent models can yield dierent estimates. First option: Fight! Choose one model (Information criterions: BIC/ AIC /AICC) H's family M 2 M 1 43

50 Bayesian Model Averaging: reducing model uncertainty. Parametric framework: arbitrary restriction, dierent models can yield dierent estimates. BMA = averaging predictions based on posterior model weights H's family M 2 Models' average M 1 Already widely studied and used in several contexts ( weather forecast... ). Hoeting et al. (99), Madigan & Raftery (94), Raftery et al. (05) 43

51 BMA: principle J statistical models M (1),... M (J), with parametrization Θ j, 1 j J and priors π j dened on Θ j 44

52 BMA: principle J statistical models M (1),... M (J), with parametrization Θ j, 1 j J and priors π j dened on Θ j BMA model = disjoint union: Θ = J 1 Θ j, with prior p on index set {1,..., J}: p(m j ) = `prior marginal model weight' for M j prior on Θ: π( J 1 B j) = J 1 p(m j) π j (B j ) (B j Θ j ) 44

53 BMA: principle J statistical models M (1),... M (J), with parametrization Θ j, 1 j J and priors π j dened on Θ j BMA model = disjoint union: Θ = J 1 Θ j, with prior p on index set {1,..., J}: p(m j ) = `prior marginal model weight' for M j prior on Θ: π( J 1 B j) = J 1 p(m j) π j (B j ) (B j Θ j ) posterior (conditioning on data X ) = weighted average J π( B j X ) = 1 posterior inm j J {}}{ p(m j X ) π j (B j X ) }{{} 1 posterior marginal model weight Key: posterior weights. (Laplace approx or standard MC?) p(m j X ) p(m j ) Likelihood(X θ) dπ j (θ) Θ j 44

54 BMA for multivariate extremes Sabourin, Naveau, Fougères (2013) Background: univariate EVD's of dierent types Stephenson & Tawn (04) or multivariate, asymptotically dependent/ independent EVD's Apputhurai & Stephenson (10) Our approach: averaging angular measure models, with angular data W. F 1,..., F J max-stable distributions j p jf j not max-stable. H 1,... H J angular measures (moments constraint) j p j H j is a valid angular measure! (linearity) 45

55 BMA for multivariate extremes Sabourin, Naveau, Fougères (2013) Background: univariate EVD's of dierent types Stephenson & Tawn (04) or multivariate, asymptotically dependent/ independent EVD's Apputhurai & Stephenson (10) Our approach: averaging angular measure models, with angular data W. F 1,..., F J max-stable distributions j p jf j not max-stable. H 1,... H J angular measures (moments constraint) j p j H j is a valid angular measure! (linearity) 45

56 Implementing and scoring BMA for Multivariate extremes Does the BMA framework perform signicantly better than selecting models based on AIC? Yes, in terms of logarithmic score for the predictive density (Kullback-Leibler divergence to the truth) Madigan & Raftery (94) In average over the union model, w.r.t prior! Simulation study : Evaluation via proper scoring rules (Logarithmic + probability of failure regions) 2 models of same dimension: Pairwise-Beta / Nested asymmetric logistic 100 data sets simulated from another model Results: The BMA framework performs consistently (for all scores), slightly (1/20 to 1/100), better than model selection. 46

57 Implementing and scoring BMA for Multivariate extremes Does the BMA framework perform signicantly better than selecting models based on AIC? Yes, in terms of logarithmic score for the predictive density (Kullback-Leibler divergence to the truth) Madigan & Raftery (94) In average over the union model, w.r.t prior! Simulation study : Evaluation via proper scoring rules (Logarithmic + probability of failure regions) 2 models of same dimension: Pairwise-Beta / Nested asymmetric logistic 100 data sets simulated from another model Results: The BMA framework performs consistently (for all scores), slightly (1/20 to 1/100), better than model selection. 46

58 Implementing and scoring BMA for Multivariate extremes Does the BMA framework perform signicantly better than selecting models based on AIC? Yes, in terms of logarithmic score for the predictive density (Kullback-Leibler divergence to the truth) Madigan & Raftery (94) In average over the union model, w.r.t prior! Simulation study : Evaluation via proper scoring rules (Logarithmic + probability of failure regions) 2 models of same dimension: Pairwise-Beta / Nested asymmetric logistic 100 data sets simulated from another model Results: The BMA framework performs consistently (for all scores), slightly (1/20 to 1/100), better than model selection. 46

59 Implementing and scoring BMA for Multivariate extremes Does the BMA framework perform signicantly better than selecting models based on AIC? Yes, in terms of logarithmic score for the predictive density (Kullback-Leibler divergence to the truth) Madigan & Raftery (94) In average over the union model, w.r.t prior! Simulation study : Evaluation via proper scoring rules (Logarithmic + probability of failure regions) 2 models of same dimension: Pairwise-Beta / Nested asymmetric logistic 100 data sets simulated from another model Results: The BMA framework performs consistently (for all scores), slightly (1/20 to 1/100), better than model selection. 46

60 Discussion BMA vs selection: Moderate gain for large sample size : Posterior concentration on `asymptotic carrier regions' = points (parameters) of minimal KL divergence from truth BMA : simple if several models have already been tted (`only' compute posterior weights) Way out: Mixture models for increased dimension of the parameter space. (product) 47