Spatial Statistics with Image Analysis. Lecture L02. Computer exercise 0 Daily Temperature. Lecture 2. Johan Lindström.

Transcription

1 C Stochastic fields Covariance Spatial Statistics with Image Analysis Lecture 2 Johan Lindström November 4, 26 Lecture L2 Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L /2 C Stochastic fields Covariance Computer exercise Stochastic fields Properties of Stochastic fields Stationarity A Model for Spatial Data Multivariate Normal Distribution Covariance functions Variogram The Matérn covariance function Estimation? Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 2/2 C Stochastic fields Covariance Computer exercise Daily Temperature 2 Lund Daily Temperature Jan3 Jan4 Jan y(t) = β + β 2 sin(2πt/36) + β 3 cos(2πt/36) + η(t) = X(t)β + η(t) Residuals Jan3 Jan4 Jan η(t) = αη(t ) + ν(t), ν(t) N ( ), σ 2 ν. Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 3/2

2 C Stochastic fields Covariance Computer exercise Prediction Assuming that we have observed temperature up to time t and know β a prediction of the temperature at t + τ is: E(y(t + τ) Y :t, β) = E(X(t + τ)β + η(t + τ) Y :t, β) with prediction variance: = X(t + τ)β + α τ η(t), V(y(t + τ) Y :t, β) = V(X(t + τ)β + α τ η(t) + = σ 2 α 2τ η α 2. τ α τ i ν(t + i) Y :t, β) i= Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 4/2 C Stochastic fields Covariance Computer exercise Prediction 2 Jan3 Jan4 Jan 3/28 4/4 Image Reconstruction Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L /2 C Stochastic fields Covariance Spatial Interpolation Given observations at some locations (pixels), y(s i ), i =... n we want to make statements about the value at unobserved location(s), y(s ). The typical model consists of a stochastic field Y(s) = μ(s) + η(s) + ε(s), observed at locations s i, i =,..., n. Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 6/2

3 Spatial Stochastic Process / Stochastic Field / Random Field A stochastic field Y(s), s D, is a random function defined on some index set D. In image analysis and related applications, typically D N 2 or D R 2. The collection of all finite-dimensional distributions determines the field distribution: p(y(s ), Y(s 2 ),..., Y(s n )), s,..., s n D, n finite. Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 7/2 Properties of Stochastic fields For a stochastic field Y(s), the expectation function μ Y (s) = E(Y(s)) collects the point-wise expectations of the field. For the covariance between different locations, we write ( ) r(s, s 2 ) = C Y(s ), Y(s 2 ) ( [Y(s = E ) μ Y (s ) ][ Y(s 2 ) μ Y (s 2 ) ]) r(s, s 2 ) is called the covariance function. Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 8/2 2 nd order (weak) stationarity A field is said to be 2 nd order stationary if the expectation and covariance are unchanged under translation. μ Y (s) =μ Y (s + h) = const. r(s, s 2 ) =r(s + h, s 2 + h) r(s, s + h) = r(, h) = r(h) A stationary field is sometimes said to be homogeneous. If, in addition, the covariance depends only on the distance between points and not the direction, r(h) = r( h ), the field is said to be isotropic. Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 9/2

4 Strong stationarity A field is said to be (strongly/strictly) stationary if all the finite-dimensional densities are unchanged under translation, i.e. for any h. p(y(s ), Y(s 2 ),..., Y(s n )) = p(y(s + h), Y(s 2 + h),..., Y(s n + h)), If V(Y(s)) < : stong stationarity = weak stationarity. For a Gaussian field: stong stationarity = weak stationarity. Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L /2 Example: The AR() process In time an autoregressive (AR) model of order one is given by: ( y(t) = αy(t ) + ν(t), ν(t) N, σ 2), with independent innovations ν(t) and α < This is a stationary process with E(y(t)) = r(τ) = C ( y(t), y(t + τ) ) = V ( y(t) ) = r() = σ2 α 2 σ2 α 2 α τ Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L /2 Example: Estimating the covariance function Given observations of a mean-zero stationary process an estimate of the covariance function can be obtained as r(τ) = y(t)y(t + τ) E(y(t)y(t + τ)) n τ t Correlation 2 2 Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 2/2

5 A Model for Spatial Data For modelling a stochastic process is often divided into y(s) = μ(s) + η(s) + ε(s), where η(s) is assumed to be stationary with one of the above covariance functions. ε(s) is called the nugget and represents small scale variability and measurement noise. The resulting covariance function for Y(s) is: r y (h) = r η (h) + I( h = )σ 2 ε = { σ 2 + σ 2 ε, h =, r η (h), h >. Discrete representations Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 3/2 If the true field is defined on (a subset of) R 2, some form of discretisation is needed for practical computations. For a given set of points, {s,..., s n }, the full field is represented by the random variables Y(s i ), i =,..., n. Almost all numerical calculations are performed for this discretely indexed field. Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 4/2 The Multivariate Normal distribution The Gaussian (Normal) distribution will be used extensively. Y N (μ, Σ) The expectation is μ, with μ i = E(Y(s i )) = E(Y i ). The covariance matrix is Σ: ( [Y ][ ] ) Σ i,j = C(Y i, Y j ), Σ = C(Y, Y) = E μ Y μ The density is given by ( p(y) = (2π) N/2 exp ) Σ /2 2 (Y μ) Σ (Y μ) where Y and μ are column vectors of length N, and Σ is a N N-matrix. Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L /2

6 Covariance functions We often need to estimate the N N covariance matrix Σ based on one sample (N observations) of the field. This implies estimating N(N + )/2 unknowns from N observations... The requirements (positive definite) on Σ implies restrictions on the estimation. The covariance matrix is often assumed to come from a parametric family of covariance functions. Reducing the problem to estimation of the covariance parameters. The resulting estimation problem does not have a closed form solution, leading to numerical optimisation. Matérn: Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 6/2 σ 2 r(h) = Γ(ν) 2 ν (K h )ν K ν (K h ), Exponential: Matérn with ν = /2 Gaussian: Matérn with ν r(h) = σ 2 exp( K h ) r(h) = σ 2 exp( 2 h 2 /ρ 2 ) Cauchy: r(h) = σ 2 ( + ( h /ρ) 2) κ Spherical: r(h) = σ 2 {. ( h /ρ) +. ( h /ρ) 3, h ρ, h > ρ Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 7/2 Examples of Covariance functions Spherical Gaussian Exponential Matern, k= Matern, k=3 Cauchy, k= Cauchy, k= distance Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 8/2

7 Semi-variogram Semi-variogram For a stationary, isotropic field the semi-variogram is defined as γ( h ) = ) (Y(s 2 V + h) Y(s) = r() r( h ) The linke between variogram and covariance function is: γ(h) = σ 2 + σ 2 ε r η ( h ) σ 2 + σ 2 ε, as h r(h) = r η (h) + I( h = )σ 2 ε σ 2 + σ 2 ε, as h with nugget σ 2 ε, partial sill σ 2, and sill σ 2 + σ 2 ε. Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 9/2 Covariance function nugget partial sill range distance Semi Variogram partial sill sill nugget range distance Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 2/2 Examples of Semi-variograms Spherical Gaussian Exponential Matern, k= Matern, k=3 Cauchy, k= Cauchy, k= distance Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 2/2

8 Matérn covariance r M (h) = σ 2 Γ(ν) 2 ν (K h )ν K ν (K h ), h R d, where K ν is a modified Bessel function of the second kind. Parameters, θ, of the covariance are: variance (σ 2 ), scale (K > ), shape (ν > ). A measure of the range is given by ρ = 8ν/K. Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 22/2 nu=. nu= nu=2. nu= Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 23/2 nu=. nu= nu=2. nu= Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 24/2

9 Prediction and Estimation? If θ = {μ, Σ} are known, how can we predict Y at unobserved locations? If μ is unknown? If Σ in unknown? Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L 2/2