Spatial Statistics with Image Analysis. Lecture L08. Computer exercise 3. Lecture 8. Johan Lindström. November 25, 2016

Transcription

1 C3 Repetition Creating Q Spectral Non-grid Spatial Statistics with Image Analysis Lecture 8 Johan Lindström November 25, 216 Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 1/39 Lecture L8 C3 Repetition Creating Q Spectral Non-grid Computer exercise 3 Repetition Bayesian hierarchical modelling Parameter estimation Creating Q CAR1 models Matérn? Spectral Representation Example Fields Stochastic Partial Differential Equation The finite element method Boundary effects Non-gridded data Triangulate Basis functions Solution Observations Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 2/39 Computer exercise 3 C3 Repetition Creating Q Spectral Non-grid Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 3/39

2 Computer exercise 3 C3 Repetition Creating Q Spectral Non-grid Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 4/39 C3 Repetition Creating Q Spectral Non-grid BHM Estimation Bayesian hierarchical modelling using GMRF Data model, p y z, θ: Describing how observations arise assuming a known latent field z. Latent model, p z θ: Describing how the latent field behaves. z = Ax + Bβ x N, Q θ Parameters, p θ: Describing our, sometimes vauge, prior knowledge of the parameters. Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 5/39 Inference C3 Repetition Creating Q Spectral Non-grid BHM Estimation Given a Bayesian hierarchical model we are interested in θ = arg max pθ y arg max p y x, θ p x θ d x. θ θ We note that conditional distributions provide the equality p y x, θ p x θ = p y, x θ = p x y, θ p y θ This gives p θ y p y x, θ p x θ p x y, θ p θ for any x. py θ Allowing us to avoid explicitly computing the integral. Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 6/39

3 C3 Repetition Creating Q Spectral Non-grid BHM Estimation Parameter estimation Gaussian Observations For the case with Gaussian observations of a GMRF y x, θ N à x, Q ε x θ N, Q, We have: Q 1/2 Q ε 1/2 py θ exp 1 [ μ Qx y 1/2 x y 2 Qμ x y + y Ãμ x y Qε y x y ] Ãμ. and E x y, θ = μ x y = Q x y à Q ε y, V x y, θ = Q x y = Q + à Q ε Ã. Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 7/39 C3 Repetition Creating Q Spectral Non-grid BHM Estimation Parameter estimation Non-Gaussian Observations For the case with Non-Gaussian observations of a GMRF p y x = n p y i x i i=1 x θ N, Q, we use the same trick for the conditional distributions as before p θ y p y x, θ p x θ p x y, θ p θ for any x. py θ However for non-gaussian data p x y, θ does not have a close form solution. Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 8/39 C3 Repetition Creating Q Spectral Non-grid BHM Estimation Parameter estimation Taylor Expansion A Taylor expansion of the log-posterior around x à x log p x y, θ x à f H f 1 2 x Q à H f à x + const. gives an approximate Gaussian: x y, θ N μ x y, Q x y with μ x y = Q x y à f à H f à x Q x y = Q à H à f Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 9/39

4 C3 Repetition Creating Q Spectral Non-grid BHM Estimation Parameter estimation Non-Gaussian Observations If the Gaussian approximation is performed at the mode of p x y, θ then the expectation of the Gaussian approximation coincides with the mode arg max p x y, θ = x = E x x x y, θ and the approximate posterior simplifies to p y x, θ p x θ p θ y p θ Qx y 1/2 Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 1/39 CAR1 models C3 Repetition Creating Q Spectral Non-grid CAR1 models Matérn? In order to use GMRFs insead of full covariance models, we need to construct useful, sparse, Q-matrices. Conditional autoregressive models CAR A mean zero CAR1 models is defined by x i {x j : j N i } N 1 1 K 2 x j, + N i τk 2 + N j N i i Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 11/39 CAR1 models C3 Repetition Creating Q Spectral Non-grid CAR1 models Matérn? For observations on a regular grid the resulting local q-pattern with pixel x i marked in red is q = τ 4 + K 2 The q-pattern can be divided into a component containing the parameter, K 2, and a 2 nd order finite difference operator q = τ K I Δ G Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 12/39

5 How to create Q? C3 Repetition Creating Q Spectral Non-grid CAR1 models Matérn? The Matérn covariance family The covariance between two points at distance h is r M h = σ 2 Γν 2 ν K h ν K ν K h Fields with Matérn covariances are solutions to a Stochastic Partial Differential Equation SPDE, K 2 Δ α/2 xs = 1 τ Ws. Here Ws is white noise and Δ = i 2 2 s i. Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 13/39 Spectral Density For a stochastic process in time, xt with stationary covariance function, rt the spectral density and covariance function form a Fourier transform pair fω = 1 2π rt = rte iωt dt = Fr fωe iωt dω = F f Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 14/39 Linear Filters Applying a linear filter with impulse response, hu, to a Gaussian process results in a transformed Gaussian process yt = ht uxu du = huxt u du, with spectral density f y ω = Hω 2 f x ω where Hω is the transfer function Hω = hue iωu du Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 15/39

6 Linear Filters Proofs The covariance for the filtered process yt = ht uxu du is assuming that all functions are nice r y τ = C yt + τ, yt = = C huxt + τ u du, = = hvxt v dv hu hv C xt + τ u, xt v du dv hu hv r x τ + v u du dv Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 16/39 Linear Filters Proofs The resulting spectral density for yt is f y ω = 1 2π = 1 2π r y τe iωτ dτ with the substitution τ = τ + v u we have = 1 2π = huhvr x τ + v ue iωτ du dv dτ huhvr x τ e iωτ v+u du dv dτ hue iωu du hve iωv dv 1 2π = HωHωf x ω = Hω 2 f x ω r x τ e iωτ dτ Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 17/39 Example Exponential smoothing Assume an exponential smoothing filter in continuous time { βe αu, u hu =, u < with transfer function Hω = e iωu hu du = βe α+iωu hu du = β α + iω Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 18/39

7 Example Exponential smoothing The filtered smoothed process yt = ht uxu du = has spectral density f y ω = Hω 2 f x ω = β α + iω If xt is white noise with t 2 βe αt u xu du. f x ω = β 2 α 2 + ω 2 f xω. r x t = δ t f x ω = 1 2π then yt is a time continuous version of an AR1-process the Ornstein-Uhlenbeck process with f y ω = β 2 2πα 2 + ω 2 r y t = β 2 2α e α t Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 19/39 Spectral Density For a stochastic field, xs, in R d with stationary covariance function, rh the spectral density is fω = 1 rhe iω h dh, 2π d R d rh = fωe iω h dω R d Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 2/39 Spectral Density Isotropy For an isotropic stationary covariance function in R 2 a change to polar coordinates gives the spectral density as fω = 1 2π 2 = 1 2π rh 2π rh h J ωh dh, e iωh cos θ dθ h dh where ω = ω, h = h, and J is a Bessel function of the first kind. Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 21/39

8 Spectral Density Isotropy For a stochastic field, xs, in R d with isotropic stationary covariance function, rh the spectral density is fω = 1 2π d/2 rh h d J d 2/2ωh ωh d 2/2 dh, rh = 2π d/2 fω ω d J d 2/2ωh ωh d 2/2 where J k is a Bessel function of the first kind. dω, Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 22/39 Spectral Density Matérn The Matérn covariance family The covariance between two points at distance h is r M h = σ 2 Γν 2 ν K h ν K ν K h has spectral density in R 2 f M ω = σ 2 2πΓν 2 ν = νσ2 K 2ν π 1 K 2 + ω 2 ν+1 Kh ν K ν Kh h J wh dh Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 23/39 Matérn covariances from an SPDE If we consider the SPDE as a linear filter we can write xs = K 2 Δ α/2 1 τ Ws. with transfer function 1 Hω = F τ K 2 Δ α/2 since 1 τ 1 K 2 + ω 2 α/2 F f = iωff F f = ω 2 Ff Proportional to since we re ignoring 2π-constants in the Fourier-transform Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 24/39

9 Matérn covariances from an SPDE If the SPDE K 2 Δ α/2 xs = 1 τ Ws. is driven by spatial white noise, Ws, with f W ω 1 and r W h = δ h then the resulting spectral density for xs f x ω = Hω 2 f W ω 1 τ 1 K 2 + ω 2 α is Matérn. Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 25/39 Matérn covariances cont. In R d a field with Matérn covariance in given as the solution to the SPDE K 2 Δ α/2 xs = 1 τ Ws, where we have: Δ = is the Laplacian. s 2 x s 2 y Wu is spatial white noise. α = ν + d/2. Parameter link: σ 2 = 1 Γν τ ΓαK 2ν 4π d/2 Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 26/39 The finite element method The following is a highly simplified solution sketch: For α = 2 and gridded data we discretize the field xs to a vector X of values on the the grid points. And replace the differential operator K 2 Δ, with a finite difference matrix K 2 I + G. The discretized SPDE becomes a SAR1: K 2 I + G X D = 1 τ ε where ε is the discretized white noise, Wu. Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 27/39

10 Lattice on R 2, Grid size h, τ = 1 Order α = 1 ν =, or CAR1: K 2 h } {{ } } {{ } C Δ G Order α = 2 ν = 1, or SAR1: 1 K 4 h K h C Δ G Δ 2 G 2 =GC G Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 28/39 Boundary effects Three options for handeling the boundary effects exist. Dirichlet boundary condition: xu = on the boundary. Neumann boundary condition: x u = perpendicular to the boundary. Torus: By folding the image onto a torus we completely avoid edges, elimating the need for boundary conditions. However it introduces strange dependencies... Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 29/39 Boundary effects The three options correspond to different edge corrections q Dirichlet = 4 + K 2 q Neumann = 3 + K 2 q Torus = 4 + K 2 Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 3/39

11 Boundary effects cont. Dirichlet Neumann Torus Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 31/39 Non-gridded data Basic idea Construct a discrete approximation of the continuous field using basis functions, {ψ k }, and weights, {x k }, xs = k ψ k sx k Find the distribution of x k by solving κ 2 Δ α/2 xs = Ws Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 32/39 Non-gridded data Triangulate Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 33/39

12 What s a good basis? Here we use Piecewise linear basis, i.e. a set of pyramids. Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 34/39 Scalar product We define the scalar product between two functions as fs, gs = fsgs ds R d Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 35/39 Solving the SPDE A stochastic weak solution to the SPDE is given by weights, {x k }, such that [ ψ i s, κ 2 Δ ] α/2 D xs = [ ψ i s, Ws ] i=1,...,n i=1,...,n Replacing xs with k ψ ksx k gives [ ψ i s, κ 2 Δ ] α/2 ψk s x k k i=1,...,n D = [ ψ i s, Ws ] i=1,...,n For α = 2 we have κ 2 [ ψ i s, ψ k s ] + [ ψ i s, Δψ k s ] x = D [ ψ k s, Ws ] C G N,C Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 36/39

13 Solution to the SPDE The equality κ 2 [ ψ i s, ψ k s ] + [ ψ i s, Δψ k s ] x = D [ ψ k s, Ws ] C G N,C can be written in matrix form as κ 2 C + G x N, C where elements in G and C are given by G ij = ψ i s, Δψ j s and C ij = ψ i s, ψ i s To obtain a diagonal C-matrix and sparse C we use the finte element approximation: C ii ψ i s, 1 = ψ i s ds and C ij if i j Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 37/39 Solution to the SPDE A weak solution to the SPDE κ 2 Δ xs = Ws. is given by xs = k ψ k sw k where κ 2 C + G w N, C The precision of the weights, w, is V w = Q 2 = κ 2 C + G C κ 2 C + G Q 1 =κ 2 C + G Q 2 = κ 2 C + G C κ 2 C + G Q α = κ 2 C + G C Q α 2 C κ 2 C + G, α = 3, 4, 5,... Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 38/39 Observations The A-matrix The field is created as a weighted sum of basis functions. xs = N ψ k s x k, k=1 The locations of the basis functions do not need to match observation locations. Observations ys = xs + ε = k ψ k sx k + ε We introduce a sparse matrix A i = [ ψ 1s i ψ Ns i ] linking the field to the observation. Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 39/39