Simulation of a Gaussian random vector: A propagative approach to the Gibbs sampler

Transcription

1 Simulation of a Gaussian random vector: A propagative approach to the Gibbs sampler C. Lantuéjoul, N. Desassis and F. Fouedjio MinesParisTech Kriging and Gaussian processes for computer experiments CHORUS Workshop,

2 Outline Presentation of the problem reminder and notation running Gibbs sampler on large Gaussian random vectors Propagative version of the Gibbs sampler two remarks basic algorithm possible extensions Applications simulation of a nonstationary Gaussian random field conditional simulation of a Cox process 2

3 Presentation of the problem 3

4 Reminder and notation Let Y S = (Y s, s S) be a standardized Gaussian random vector with covariance matrix C. (Simple) kriging: If s S and A S. The kriging of Y s on Y A is a linear estimator Ys A = α A λ αy α. The weights λ α are chosen so as to minimize the estimation variance V ar{y s Ys A }. They are solutions of the linear system β A λ βc α,β = C α,s α A The estimation variance is σ 2A s Conditional distribution: = V ar{y s Y A s } = 1 α A λ αc α,s Y s Y A = y A is normally distributed. More precisely, we have Y s Y A = y A N ( ) ys A, σs 2A 4

5 Simulation of a Gaussian random vector using the Gibbs sampler Problem: Simulate the Gaussian vector Y S iteratively. Algorithm: (i) reset y c S; (ii) put y n S = ys; c (iii) select s U(S) and generate ys n N ( ) ys S\s, σs 2S\s ; (iv) put y c S = y n S and goto (ii). Problem: When S is large, the kriging matrices cannot be inverted. 5

6 Simulation of a Gaussian random vector using the Gibbs sampler; approximate algorithm To illustrate the approach, the components of S are depicted as the nodes of a square grid, which makes it possible to consider kriging neighbourhoods as balls B s,r. r Algorithm: (i) reset y c S; (ii) put y n S = y c S; (iii) select s U(S) and generate ys n N ( y B s,r\s s (iv) put y c S = y n S and goto (ii). ), σ 2B s,r\s s ; 6

7 Example (10000 components), spherical covariance (range 10) Neighbourhood radius of

8 Example (10000 components), spherical covariance (range 10) Neighbourhood radius of

9 Extrema Scans 9 Extrema

10 Gibbs sampler for Gaussian random vectors A propagative version 10

11 Two useful remarks Remark 1: Let Y S N(0, C). The kriging estimate ys S\s and its estimation variance σs 2S\s can be expressed using the inverse C 1 of matrix C: Remark 2: y S\s s = t s C 1 st C 1 ss y t σ 2S\s s = 1 C 1 ss Galli and Gao (2001): If X = C 1 Y, then X N(0, C 1 ). Consequence: As (C 1 ) 1 = C, the Gibbs sampler can be run exactly on X. Idea: Apply the exact Gibbs sampler to X and transport the results to Y using the relation Y = CX. 11

12 Running the Gibbs sampler on X Remark: X S\s s = t s C st C ss X t = t s C st X t = X s Y s V ar{x s X S\s s } = 1 C ss = 1 Algorithm: (i) reset x c S; (ii) put x n S = x c S; (iii) select p U(S) and generate x n p N(x c p y c p,1); (iv) put x c S = x n S and goto (ii). Definition: The point p is called a pivot. 12

13 Transporting the Gibbs sampler to Y If x n p N(x c p y c p, 1), then x n p = x c p y c p + u with u N. Then y n s = t C st x n t = t p C st x c t + C sp x n p = y c s C sp x c p + C sp (x c p y c p + u) = y c s + C sp (u y c p) Taking s = p, we obtain y n p = u, which finally gives y n s = y c s + C sp (y n p y c p) s S Algorithm: (i) reset y c S; (ii) select p U(S) and generate y n p N; (iii) put y n s = y c s + C sp (y n p y c p) for each s S; (iv) put y c S = y n S and goto (ii). 13

14 Example of a vector with components Spherical covariance function with range

15 Variograms of simulations gamma(h) Spherical covariance (10) h Simulation variograms obtained after 1, 2, 3, 5, 7, 10, 15 and 20 scans. In black, the variogram model. 15

16 Generalizations Generating the pivot value y n p can be generated as a function of y c p, e.g. y n p N(ry c p, 1 r 2 ). It is sometimes convenient to draw y n close to y c. Blocking strategy y n s = y c s + C sp (y n p y c p) y n s C sp y n p = y c s C sp y c p Thus, kriging residuals are preserved by the propagative approach of the Gibbs sampler. This remark remains valid when the pivot p is replaced by a family P of pivots. (i) put y c = 0; (ii) select P at random in S and generate y n P N(0, C PP ); (iii) put ys n = ys c + ys n,p (iv) put y c = y n, and goto (ii). y c,p s for each s S; 16

17 Application Simulation of a non-stationary Gaussian random field 17

18 Simulation of a nonstationary Gaussian random field Let Z be a nonstationary Gaussian random field. It can be written as m + σy, where Y is a (non necessarily stationary), standardized Gaussian random field. Problem: Generate Z in the discrete domain S. Notation: Let C be the covariance matrix of Y S. Algorithm: (i) generate y S N(0, C); (ii) put z s = m s + σ s y s for each s S. 18

19 Example: point source model C s,t = exp ( λ s t ) exp ( µ e ν s e ν t ) m = 0 and σ = 1. λ = 0.05, µ = 5 and ν = Simulation field

20 Application Conditional simulation of a Cox process 20

21 Motivation A piece of land is covered with trees. It is partitioned into congruent plots. The number of trees is known on a number of plots. We want to predict the number of trees on each plot This can be done by averaging a set of conditional simulations. The spatial distribution of trees is modelled by a Cox process. 21

22 Cox process Definition: A Cox process is a Poisson point process with random intensity function. Remark: The random intensity function is also called potential. 22

23 Properties of the Cox process Let N S = (N s, s S) be the number of trees on all plots, and let Z S be their potential. Mean value: Covariance between plots: Cov{N s, N t } = E{N s } = E{Z s } { Cov{Zs, Z t } if s t V ar{z s } + E{Z s } if s = t Conditional distribution: Given the potential, the numbers of trees on differents plots are conditionally independent. P {N S = n S Z S = z S } = exp( z s ) zn s s n s! s S 23

24 Cox-lognormal process The potential is lognormally distributed, i.e. Z S = exp(µ + σy S ), where Y S is a standardized Gaussian random vector (covariance matrix C). µ = and σ = 0.7. C s,t = exp( s t /15). Simulation field unit plots. 24

25 Conditional simulation of a Cox-lognormal process Problem: Generate N S given N A = n A for some A S. Idea: At each iteration, candidate values are proposed for y S using the propagative version of the Gibbs sampler. They are accepted or rejected using Metropolis-Hastings criterion. Algorithm: (i) put y c S = 0; (ii) generate p U(S) and y n p N; (iii) put z c A = e µ+σyc A, z n A = z c A e σc Ap (yn p yc p ), and generate u U; (iv) if u > p(n A z n A)/p(n A z c A), then goto (ii); (v) put y c S = y c S + C Sp (y n p y c p); (vi) goto (ii). 25

26 20 conditioning data points Reality (TL), conditioning data points (TR), and two conditional simulations (BL and BR) 26

27 100 conditioning data points Reality (TL), conditioning data points (TR), and two conditional simulations (BL and BR) 27

28 500 conditioning data points Reality (TL), conditioning data points (TR), and two conditional simulations (BL and BR) 28

29 Conclusions An iterative algorithm, derived from the Gibbs sampler, has been proposed for simulating Gaussian random vectors. At each iteration, a component is selected and randomly assigned a new value. This value is then linearly propagated to the other components. Because it does not require the inversion of the covariance matrix, this algorithm does not incur the dimensionality problem of the standard Gibbs sampler. This can be used to simulate random processes related to Gaussian random fields even if they are nonstationary or subject to soft constraints. 29

30 References Desassis N. et Lantuéjoul C. (2011) - Simulation d un vecteur gaussien: une approche propagative de l chantillonneur de Gibbs. 43 emes Journées de Statistique, Tunis (Tunisie). Lantuéjoul C. and Dessassis N. (2012) - Simulation of a Gaussian random vector: A propagative version of the Gibbs sampler. 9 th Int. Geostatistics Congress, Oslo (Norway). Fouedjio F. (2014) - Développement de modèles géostatistiques globaux non-stationnaires. Thèse MinesParisTech (à soutenir). 30

31 Rate of convergence Let Y (n) the random vector generated at the n th scan. Systematic scan If Y (0) N(0, C (0) ), then Y (n) N(0, C (n) ). Besides, we have C (n) C = B n( C (0) C )t B n where B = (Id L) 1 U. The matrices Id, L and U are respectively the identity matrix, the lower and upper parts of C C = Id + L + U If follows that the rate of convergence is governed by the square of the spectral radius of B. 31