Karhunen-Loeve Expansion and Optimal Low-Rank Model for Spatial Processes
|
|
- Margery Gaines
- 6 years ago
- Views:
Transcription
1 TTU, October 26, 2012 p. 1/3 Karhunen-Loeve Expansion and Optimal Low-Rank Model for Spatial Processes Hao Zhang Department of Statistics Department of Forestry and Natural Resources Purdue University
2 TTU, October 26, 2012 p. 2/3 Outline Motivations of studying KL expansion in spatial statistics Numerical algorithms Some examples and results Applications in multivariate spatial processes
3 Spatial data Y (s i ),s i D R 2 or R 3. Correlated Massive Satellite data Remotely sensed data Censor network Multi-source X X1 TTU, October 26, 2012 p. 3/3
4 TTU, October 26, 2012 p. 4/3 Spatial process Review the spatial data as a realization of an underlying spatial process: data=mean+error, mean function=deterministic, linear function of some covariates (i.e., lat and long). error process=2nd order process, often assumed to be stationary. Spatial correlation attributable to unobserved latent variables; accounting for the spatial variation unaccounted for by the mean. I assume the mean is a constant (known or unknown).
5 TTU, October 26, 2012 p. 5/3 Covariogram and Kriging Covariogram: C(s 1,s 2 ) = Cov(Y (s 1 ),Y (s 2 )). Stationary if C(s 1,s 2 ) = C(0,s 2 s 1 ). Isotropic if C(s 1,s 2 ) = C( s 1 s 2 ). Kriging=Best Linear Prediction: Given Y (s 1 ),,Y (s n ), n Ŷ (s 0 ) = E(Y (s 0 )) + α i (Y (s i ) EY (s i )) i=1 = E(Y (s 0 )) + Cov(Y (s),y )[V ar(y )] 1 (Y EY ).
6 TTU, October 26, 2012 p. 6/3 Large covariance matrix n is hundreds of thousands or millions. Data are correlated The inverse of V n n is needed in the full MLE or Bayesian inferences, and kriging, rending the traditional methods impractical: L n (θ) = n 2 log(2π) 1 2 log V n(θ) 1 2 (Y n Xβ) Vn 1 (θ)(y n Xβ), The inverse matrix is needed for prediction.
7 TTU, October 26, 2012 p. 7/3 Large storage and memory For an n n matrix, the double precision storage is 8n 2 bites. When n = 10 4, the storage is 800MB! When n = 10 6, the storage is 8000GB! Computation for V 1 n n is O(n 3 ) unless V n n has a special structure. Therefore, for massive spatial data, traditional geostatistical methods may not be applicable and innovative methods are need.
8 TTU, October 26, 2012 p. 8/3 Low rank models Y (s) = µ + a(s) Z + ǫ(s), (1) where a(s) = (a 1 (s),,a m (s)) is a function of s that may depend on some parameters θ, Z = (Z(u 1 ),,Z(u m )) is a vector of latent variables with u i s pre-chosen, and ǫ(s) is a white noise with variance τ 2 and is independent of Z. Gaussian process convolution: a j (s) = exp( s u j 2 /φ)/ 2π, Z(u i ) s are i.i.d. Fixed rank kriging: a j (s) = b( s u j 2 /φ) for some known basis function b and Z = (Z(u 1 ),,Z(u m )) is a realization of a latent process at locations u j, j = 1,,m. Predictive process: a(s) = V ar(z) 1 Cov(Z,Z(s)) where Z(s) is a Gaussian stationary process with mean 0 and some parametric covariogram.
9 TTU, October 26, 2012 p. 9/3 Covariance matrix of low rank Observing Y = (Y (s i ),i = 1,,n) from the low rank model, write Y = µ1 + AZ + ǫ. The covariance matrix of Y is V = AV z A + τ 2 I n, where A is n m and V z = V ar(z). Woodbury Formula: V 1 = τ 2 (I n A(τ 2 V 1 z + A A) 1 A ). Memory requirement is 8(n + nm + m 2 ) and linear in n. Sylvester s determinant theorem: V = τ 2(n m) V z τ 2 V 1 z + A A.
10 TTU, October 26, 2012 p. 10/3 Optimal low rank models Given a random field Y (s),s D, consider the best low rank model, i.e., to minimize D E ( Y (s) ) 2 m a k (s)z k ds, k=1 over all functions a k (s) and random variables Z k. Solution: a k (s) = λ k f k (s) where λ k and f k (s), k = 1,,m are the first m largest eigenvalues and the corresponding eigenfunctions of the Karhunen-Loeve expansion of the random field.
11 TTU, October 26, 2012 p. 11/3 K-L expansion Y (s) = λk f k (s)z k,s D k=1 where λ 1 λ 2 0, Z k are white noises of unit variance and f j (s)f k (s)ds = δ jk. D Hence C(s,x) = λ k f k (s)f k (x) k=1 and D C(s,x)f k (x)dx = λ k f k (s).
12 TTU, October 26, 2012 p. 12/3 Truncated KL D E D { E Y (s) { } 2 m λk f k (s)z k ds k=1 Y (s) } 2 m a k (s)z k ds for any functions a k (s) and zero-mean random variables Z k where D is bounded. k=1
13 TTU, October 26, 2012 p. 13/3 Computation of eigenpairs Given a covariance function C(s,x),x,s D R d, find λ > 0 and function f(x) such that Approximation: D C(s,x)f(x)dx = λf(s). Choose a basis {φ i,i = 1,,m} L 2 (D). Approximate an eigenfunction by f(s) = m i=1 d iφ i (s), so that D C(s,x)f(x)dx λf(s) is orthogonal to each basis function φ i (x).
14 TTU, October 26, 2012 p. 14/3 Galerkin approximation Finding λ and d: D ( m d i i=1 D C(s, x)φ i (s)dx λ ) m d i φ i (s) φ j (s)ds = 0, for j = 1,, m, i=1 Gd = λbd, (2) where G is m m whose (i,j) element is D D φ i(s)c(s,x)φ j (x)dsdx, B is m m whose (i,j) element is D φ i(s)φ j (s)ds, d = (d 1,,d m ). Solve the generalized eigenvalue problem (2) to get λ and d. The m 2 elements of G are each 4-dimensional integrals. Ghanem and Spanos, 1991, Schwab and Todor, 2005, Phoon, et al. 2002, Huang, et al., 2001
15 TTU, October 26, 2012 p. 15/3 Gaussian Quadrature Method Gaussian Legendre quadrature: 1 1 g(x)dx m r i g(x i ). i=1 The approximation is exact for polynomials of order (2m 1) or less. The nodes x i are the roots of Legendre polynomial of m order and the weights r i also calculated from the Legendre polynomial. Extends to high dimension: g(y 1,y 2 )dy 1 dy 2 m i=1 m r i r j g(x i,x j ) j=1
16 TTU, October 26, 2012 p. 16/3 Example of Nodes Nodes, m=
17 TTU, October 26, 2012 p. 17/3 Gaussian Quadrature Method for KL Given C(s,x), s,x [ 1,1] d (d = 2), find λ and f(x) such [ 1,1] d C(s,x)f(x)dx = λf(x). Let u i, i = 1,...,m be the m roots of Legendre polynomial and u i, i = 1,...,m 2 denotes the pairs (u j,u k ),j,k = 1,...,m. Find f = (f(u 1 ),...,f(u m 2)) by solving m 2 j=1 where w j = u k u l if u j = (u k,u l ). w j C(u i,u j )f(u j ) = λf(u i )
18 TTU, October 26, 2012 p. 18/3 Gaussian Quadrature Method for KL Write V = (C(u i,u j )),W = diag(w 1,...,w m 2). Then V Wf = λf A generalized eigenvalue problem. Hence λ is an eigenvalue of W 1/2 V W 1/2 and W 1/2 f is the corresponding eigenvector.
19 TTU, October 26, 2012 p. 19/3 Polynomial Interpolation Having obtained f, we now have f(u i ). Now approximate the eigenfunction by a polynomial through the Lagrange interpolation.
20 TTU, October 26, 2012 p. 20/3 Example 1 covariance True Function 5%quantile median 95%quantile covariance True Function 5%quantile median 95%quantile covariance True Function 5%quantile median 95%quantile distance (a) distance (b) distance (c) covariance True Function 5%quantile median 95%quantile covariance True Function 5%quantile median 95%quantile covariance True Function 5%quantile median 95%quantile distance (d) distance (e) Top: Gaussian quadrature; Bottom: Galerkin projection. distance (f)
21 TTU, October 26, 2012 p. 21/3 Example 2 covariance True Function 5%quantile median 95%quantile covariance True Function 5%quantile median 95%quantile covariance True Function 5%quantile median 95%quantile distance (a) distance (b) distance (c) covariance True Function 5%quantile median 95%quantile covariance True Function 5%quantile median 95%quantile covariance True Function 5%quantile median 95%quantile distance (d) distance (e) Top: Gaussian quadrature; Bottom: Galerkin projection. distance (f)
22 TTU, October 26, 2012 p. 22/3 Simulation study Objective: To compare the predictive performance of the different low-rank approximations True model: Gaussian process Y (s),s [0,1] 2 with mean 0 and exponential covariance function exp( h/0.3). We use three low-rank models to approximate the true model: KL by Galerkin projection (GPKL), KL by Gaussian quadrature (GQKL) and the predictive process (Banerjee, et al., 2008). Simulate from the true model on a 81*81 lattice, but use each of the low rank model to estimate model parameters.
23 TTU, October 26, 2012 p. 23/3 Predictive Performance MSE: (1/n) (Y (s i ) Ŷ (s i )) 2. Logarithmic score (LogS): Let Z i = (Y (s i ) Ŷ (s i))/σ i (s i ). 1 n n i=1 Continuous ranked probability score: ( 1 2 log(2πˆσ i(s i ) 2 ) Z2 i ) crps(f,x) = (F(y) 1{y x}) 2 dy. CRPS = 1 n Gneiting, et al., 2006, n crps(f i,y (s i )). i=1
24 (a) MSE scores(b) CRPS scores(c) LogS scores using 150 eigen TTU, October terms 26, 2012 p. 24/3 150 eigen terms MSE GQKL PP GPKL CRPS GQKL PP GPKL (a) (b) LogS GQKL PP GPKL (c)
25 (a) MSE scores(b) CRPS scores(c) LogS scores using 300 eigen TTU, October terms 26, 2012 p. 25/3 300 eigen terms MSE GQKL PP GPKL CRPS GQKL PP GPKL (a) (b) LogS GQKL PP GPKL (c)
26 (a) MSE scores(b) CRPS scores(c) LogS scores using 300 eigen TTU, October terms 26, 2012 p. 26/3 600 eigen terms MSE GQKL PP GPKL CRPS GQKL PP GPKL (a) (b) LogS GQKL PP GPKL (c)
27 TTU, October 26, 2012 p. 27/3 Multivariate covariogram Two processes Y i (s), i = 1,2. Marginal or direct covariogram: C ii (s,x) = Cov(Y i (s),y i (x)). Cross covariogram: C ij (s,x) = Cov(Y i (s),y j (x)), i j. The matrix valued function C(s,x) = (C ij (, )) must satisfy: For any s 1,,s n, i j α i C(s i,s j )α j 0 for any vectors α i R p. It is not trivial to specify a cross covariogram.
28 TTU, October 26, 2012 p. 28/3 Some existing models Proportional model: C ij (s,x) = σ ij ρ(s,xx). Linear model of coregionalization: Sum of proportional models. Convolution models: Gaspari and Cohn (1999), Gelfand et al. (2004) Asymmetric models: Apanasovich and Genton (2009), Li and Zhang (2010) Multivariate Matern model: Gneiting et al. (2010)
29 TTU, October 26, 2012 p. 29/3 Limitation of existing models Estimation of parameters becomes more complex; constrained maximization. Do not allow arbitrary marginal covariogram models. There are cases when the multivariate covariogram model does not improve over the univariate model in predictive performance.
30 TTU, October 26, 2012 p. 30/3 Valid cross covariogram for given marginals The problem: Given two marginal covariograms C ii (s,x), s,x R d, i = 1,2,, find the class of all valid cross covariograms. A theoretical solution: Apply the Karhunen-Loeve expansion to the marginals to write C ii (s, x) = k=1 λ ik f ik (s)f ik (x), s, x D where λ k is a decreasing sequence of positive numbers and D f ik(s) 2 ds = 1 and D is a compact subset of R d. The class of all possible valid cross covariograms is C ij (s, x) = r kl λ ik λ jl f ik (s)f jl (x) k=1 l=1 where r kl [ 1,1] are such that the matrix (r kl ) n k,l=1 cross covariograms are given by such an array {r kl }. is non-negative definite for any n. Hence all
31 TTU, October 26, 2012 p. 31/3 Non-parametric cross covariogram Consider a bivariate process. Write the KL expansion for each process Y i (s) = λik f ik (s)z ik,s D, k=1 where Z ik,k = 1,2, are a white noise (i = 1,2). In a special case, when Z 1j and Z 2k are either uncorrelated or perfectly correlated, we get the non-parametric cross covariogram C 12 (s,x) = j,k:cov(z 1j,Z 2k )=1 λ1j λ 2k f 1j (s)f 2k (x).
32 TTU, October 26, 2012 p. 32/3 Two practical steps For the non-parametric cross covariogram to work in practice, we need An efficient algorithm to computer the eigenpairs (λ j,f j (s)) for any given covariogram. Find the pairs (j,k) such that Cov(Z 1j,Z 2k ) = 1.
33 TTU, October 26, 2012 p. 33/3 Inferences Estimate parameters in each marginal model using only marginal data; Since cross covariogram involves no new parameters, we only need to identify the perfectly correlated pairs of Z 1j and Z 2k. This is equivalent to specifying the correlation matrix Corr(Z 1, Z 2 ) for Z i = (Z i1,, Z im ) which has at most m 1s with the other elements being 0. Start with Corr(Z 1, Z 2 ) = 0. Replace one 0 element with 1 to optimize some objective function (i.e., likelihood, prediction score) Fixing the non-zero elements, replace one more 0 element with 1 to optimize the objective function. Iterate. The non-parameter cross covariogram leads to a much simplified estimation procedure.
34 TTU, October 26, 2012 p. 34/3 An example Wang and Zhang (2012): Let W i (s), s [0,1] 2 be two latent processes with mean 0 and have a bivariate exponential covariogram, i.e., Cov(W i (s + h),w j (s)) = σ ij exp( h /φ ij ). We observe the following process Y i (s) = W i (s) + τ i ǫ i (s) at the 1225 equally spaced locations {(i/35, j/35), i, j = 1,, 35}. The model parameters for the covariogram are chosen to be: φ 11 = 0.2,φ 22 = 0.3,φ 12 = 0.2,σ 12 = 5.4, σ 11 = σ 22 = 9 and τ 2 1 = τ2 2 = 1.
35 TTU, October 26, 2012 p. 35/3 Model fitting Fit two models: The true model and the truncated KL expansion with m=600 terms. For the true model, we estimate the parameters by maximizing the joint likelihood. For the truncated KL model, we estimate the marginal parameters by maximizing the marginal likelihood and identify the perfectly correlated pairs by maximizing the joint likelihood.
36 TTU, October 26, 2012 p. 36/3 Predictive performance Predictive scores: RMSE and logarithmic score: MSE (i) = 1 n LogS (i) = 1 n n ( Yi (s k ) Ŷi(s k ) ) 2 k=1 n j=1 ( 1 2 log(2πˆσ2 ij) + 1 ) 2 z2 ij,i = 1,2 where Ŷ i (s k ) denotes the cokriging prediction of Y i (s k ) using data at all locations but s k ; σ ik the corresponding cokriging standard deviation and z ik = (Y i (s k ) Ŷ i (s k ))/σ ik. Among 100 simulations, the non-parametric cross covariogram model beats the fitted bivariate parametric model in each simulation according to the two predictive scores.
Geostatistical Modeling for Large Data Sets: Low-rank methods
Geostatistical Modeling for Large Data Sets: Low-rank methods Whitney Huang, Kelly-Ann Dixon Hamil, and Zizhuang Wu Department of Statistics Purdue University February 22, 2016 Outline Motivation Low-rank
More informationAsymptotic Multivariate Kriging Using Estimated Parameters with Bayesian Prediction Methods for Non-linear Predictands
Asymptotic Multivariate Kriging Using Estimated Parameters with Bayesian Prediction Methods for Non-linear Predictands Elizabeth C. Mannshardt-Shamseldin Advisor: Richard L. Smith Duke University Department
More informationHierarchical Modelling for Univariate Spatial Data
Hierarchical Modelling for Univariate Spatial Data Sudipto Banerjee 1 and Andrew O. Finley 2 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department
More informationA full scale, non stationary approach for the kriging of large spatio(-temporal) datasets
A full scale, non stationary approach for the kriging of large spatio(-temporal) datasets Thomas Romary, Nicolas Desassis & Francky Fouedjio Mines ParisTech Centre de Géosciences, Equipe Géostatistique
More informationSpatial statistics, addition to Part I. Parameter estimation and kriging for Gaussian random fields
Spatial statistics, addition to Part I. Parameter estimation and kriging for Gaussian random fields 1 Introduction Jo Eidsvik Department of Mathematical Sciences, NTNU, Norway. (joeid@math.ntnu.no) February
More informationStatistics for analyzing and modeling precipitation isotope ratios in IsoMAP
Statistics for analyzing and modeling precipitation isotope ratios in IsoMAP The IsoMAP uses the multiple linear regression and geostatistical methods to analyze isotope data Suppose the response variable
More informationIntroduction to Spatial Data and Models
Introduction to Spatial Data and Models Sudipto Banerjee 1 and Andrew O. Finley 2 1 Department of Forestry & Department of Geography, Michigan State University, Lansing Michigan, U.S.A. 2 Biostatistics,
More informationStochastic Spectral Approaches to Bayesian Inference
Stochastic Spectral Approaches to Bayesian Inference Prof. Nathan L. Gibson Department of Mathematics Applied Mathematics and Computation Seminar March 4, 2011 Prof. Gibson (OSU) Spectral Approaches to
More informationChapter 4 - Fundamentals of spatial processes Lecture notes
TK4150 - Intro 1 Chapter 4 - Fundamentals of spatial processes Lecture notes Odd Kolbjørnsen and Geir Storvik January 30, 2017 STK4150 - Intro 2 Spatial processes Typically correlation between nearby sites
More informationGauge Plots. Gauge Plots JAPANESE BEETLE DATA MAXIMUM LIKELIHOOD FOR SPATIALLY CORRELATED DISCRETE DATA JAPANESE BEETLE DATA
JAPANESE BEETLE DATA 6 MAXIMUM LIKELIHOOD FOR SPATIALLY CORRELATED DISCRETE DATA Gauge Plots TuscaroraLisa Central Madsen Fairways, 996 January 9, 7 Grubs Adult Activity Grub Counts 6 8 Organic Matter
More informationIntroduction to Spatial Data and Models
Introduction to Spatial Data and Models Sudipto Banerjee 1 and Andrew O. Finley 2 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department of Forestry
More informationHierarchical Modelling for Univariate Spatial Data
Spatial omain Hierarchical Modelling for Univariate Spatial ata Sudipto Banerjee 1 and Andrew O. Finley 2 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A.
More informationBayesian Transformed Gaussian Random Field: A Review
Bayesian Transformed Gaussian Random Field: A Review Benjamin Kedem Department of Mathematics & ISR University of Maryland College Park, MD (Victor De Oliveira, David Bindel, Boris and Sandra Kozintsev)
More informationKarhunen-Loève Approximation of Random Fields Using Hierarchical Matrix Techniques
Institut für Numerische Mathematik und Optimierung Karhunen-Loève Approximation of Random Fields Using Hierarchical Matrix Techniques Oliver Ernst Computational Methods with Applications Harrachov, CR,
More informationMLES & Multivariate Normal Theory
Merlise Clyde September 6, 2016 Outline Expectations of Quadratic Forms Distribution Linear Transformations Distribution of estimates under normality Properties of MLE s Recap Ŷ = ˆµ is an unbiased estimate
More informationModels for spatial data (cont d) Types of spatial data. Types of spatial data (cont d) Hierarchical models for spatial data
Hierarchical models for spatial data Based on the book by Banerjee, Carlin and Gelfand Hierarchical Modeling and Analysis for Spatial Data, 2004. We focus on Chapters 1, 2 and 5. Geo-referenced data arise
More informationComparing Non-informative Priors for Estimation and Prediction in Spatial Models
Environmentrics 00, 1 12 DOI: 10.1002/env.XXXX Comparing Non-informative Priors for Estimation and Prediction in Spatial Models Regina Wu a and Cari G. Kaufman a Summary: Fitting a Bayesian model to spatial
More informationStatistics & Data Sciences: First Year Prelim Exam May 2018
Statistics & Data Sciences: First Year Prelim Exam May 2018 Instructions: 1. Do not turn this page until instructed to do so. 2. Start each new question on a new sheet of paper. 3. This is a closed book
More informationA Note on the comparison of Nearest Neighbor Gaussian Process (NNGP) based models
A Note on the comparison of Nearest Neighbor Gaussian Process (NNGP) based models arxiv:1811.03735v1 [math.st] 9 Nov 2018 Lu Zhang UCLA Department of Biostatistics Lu.Zhang@ucla.edu Sudipto Banerjee UCLA
More informationOverview of Spatial Statistics with Applications to fmri
with Applications to fmri School of Mathematics & Statistics Newcastle University April 8 th, 2016 Outline Why spatial statistics? Basic results Nonstationary models Inference for large data sets An example
More informationExtreme Value Analysis and Spatial Extremes
Extreme Value Analysis and Department of Statistics Purdue University 11/07/2013 Outline Motivation 1 Motivation 2 Extreme Value Theorem and 3 Bayesian Hierarchical Models Copula Models Max-stable Models
More informationLinear Methods for Prediction
Chapter 5 Linear Methods for Prediction 5.1 Introduction We now revisit the classification problem and focus on linear methods. Since our prediction Ĝ(x) will always take values in the discrete set G we
More informationCross-covariance Functions for Tangent Vector Fields on the Sphere
Cross-covariance Functions for Tangent Vector Fields on the Sphere Minjie Fan 1 Tomoko Matsuo 2 1 Department of Statistics University of California, Davis 2 Cooperative Institute for Research in Environmental
More informationHierarchical Modeling for Univariate Spatial Data
Hierarchical Modeling for Univariate Spatial Data Geography 890, Hierarchical Bayesian Models for Environmental Spatial Data Analysis February 15, 2011 1 Spatial Domain 2 Geography 890 Spatial Domain This
More informationSolving the steady state diffusion equation with uncertainty Final Presentation
Solving the steady state diffusion equation with uncertainty Final Presentation Virginia Forstall vhfors@gmail.com Advisor: Howard Elman elman@cs.umd.edu Department of Computer Science May 6, 2012 Problem
More informationGaussian predictive process models for large spatial data sets.
Gaussian predictive process models for large spatial data sets. Sudipto Banerjee, Alan E. Gelfand, Andrew O. Finley, and Huiyan Sang Presenters: Halley Brantley and Chris Krut September 28, 2015 Overview
More informationStat 5101 Lecture Notes
Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random
More informationNon-gaussian spatiotemporal modeling
Dec, 2008 1/ 37 Non-gaussian spatiotemporal modeling Thais C O da Fonseca Joint work with Prof Mark F J Steel Department of Statistics University of Warwick Dec, 2008 Dec, 2008 2/ 37 1 Introduction Motivation
More informationPolynomial Chaos and Karhunen-Loeve Expansion
Polynomial Chaos and Karhunen-Loeve Expansion 1) Random Variables Consider a system that is modeled by R = M(x, t, X) where X is a random variable. We are interested in determining the probability of the
More informationarxiv: v4 [stat.me] 14 Sep 2015
Does non-stationary spatial data always require non-stationary random fields? Geir-Arne Fuglstad 1, Daniel Simpson 1, Finn Lindgren 2, and Håvard Rue 1 1 Department of Mathematical Sciences, NTNU, Norway
More informationHandbook of Spatial Statistics Chapter 2: Continuous Parameter Stochastic Process Theory by Gneiting and Guttorp
Handbook of Spatial Statistics Chapter 2: Continuous Parameter Stochastic Process Theory by Gneiting and Guttorp Marcela Alfaro Córdoba August 25, 2016 NCSU Department of Statistics Continuous Parameter
More informationPh.D. Qualifying Exam Friday Saturday, January 6 7, 2017
Ph.D. Qualifying Exam Friday Saturday, January 6 7, 2017 Put your solution to each problem on a separate sheet of paper. Problem 1. (5106) Let X 1, X 2,, X n be a sequence of i.i.d. observations from a
More informationPositive Definite Functions on Spheres
Positive Definite Functions on Spheres Tilmann Gneiting Institute for Applied Mathematics Heidelberg University, Germany t.gneiting@uni-heidelberg.de www.math.uni-heidelberg.de/spatial/tilmann/ International
More informationStatement: With my signature I confirm that the solutions are the product of my own work. Name: Signature:.
MATHEMATICAL STATISTICS Homework assignment Instructions Please turn in the homework with this cover page. You do not need to edit the solutions. Just make sure the handwriting is legible. You may discuss
More informationAnalysing geoadditive regression data: a mixed model approach
Analysing geoadditive regression data: a mixed model approach Institut für Statistik, Ludwig-Maximilians-Universität München Joint work with Ludwig Fahrmeir & Stefan Lang 25.11.2005 Spatio-temporal regression
More informationIntroduction to Geostatistics
Introduction to Geostatistics Abhi Datta 1, Sudipto Banerjee 2 and Andrew O. Finley 3 July 31, 2017 1 Department of Biostatistics, Bloomberg School of Public Health, Johns Hopkins University, Baltimore,
More informationPart 6: Multivariate Normal and Linear Models
Part 6: Multivariate Normal and Linear Models 1 Multiple measurements Up until now all of our statistical models have been univariate models models for a single measurement on each member of a sample of
More informationIdentifiability problems in some non-gaussian spatial random fields
Chilean Journal of Statistics Vol. 3, No. 2, September 2012, 171 179 Probabilistic and Inferential Aspects of Skew-Symmetric Models Special Issue: IV International Workshop in honour of Adelchi Azzalini
More informationThe problem is to infer on the underlying probability distribution that gives rise to the data S.
Basic Problem of Statistical Inference Assume that we have a set of observations S = { x 1, x 2,..., x N }, xj R n. The problem is to infer on the underlying probability distribution that gives rise to
More informationNonstationary cross-covariance models for multivariate processes on a globe
Nonstationary cross-covariance models for multivariate processes on a globe Mikyoung Jun 1 April 15, 2011 Abstract: In geophysical and environmental problems, it is common to have multiple variables of
More informationA Complete Spatial Downscaler
A Complete Spatial Downscaler Yen-Ning Huang, Brian J Reich, Montserrat Fuentes 1 Sankar Arumugam 2 1 Department of Statistics, NC State University 2 Department of Civil, Construction, and Environmental
More informationRestricted Maximum Likelihood in Linear Regression and Linear Mixed-Effects Model
Restricted Maximum Likelihood in Linear Regression and Linear Mixed-Effects Model Xiuming Zhang zhangxiuming@u.nus.edu A*STAR-NUS Clinical Imaging Research Center October, 015 Summary This report derives
More informationNonparametric Estimation of the Dependence Function for a Multivariate Extreme Value Distribution
Nonparametric Estimation of the Dependence Function for a Multivariate Extreme Value Distribution p. /2 Nonparametric Estimation of the Dependence Function for a Multivariate Extreme Value Distribution
More informationNonstationary spatial process modeling Part II Paul D. Sampson --- Catherine Calder Univ of Washington --- Ohio State University
Nonstationary spatial process modeling Part II Paul D. Sampson --- Catherine Calder Univ of Washington --- Ohio State University this presentation derived from that presented at the Pan-American Advanced
More informationNon-stationary Cross-Covariance Models for Multivariate Processes on a Globe
Scandinavian Journal of Statistics, Vol. 38: 726 747, 2011 doi: 10.1111/j.1467-9469.2011.00751.x Published by Blackwell Publishing Ltd. Non-stationary Cross-Covariance Models for Multivariate Processes
More informationGaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012
Gaussian Processes Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 01 Pictorial view of embedding distribution Transform the entire distribution to expected features Feature space Feature
More informationLecture 25: Review. Statistics 104. April 23, Colin Rundel
Lecture 25: Review Statistics 104 Colin Rundel April 23, 2012 Joint CDF F (x, y) = P [X x, Y y] = P [(X, Y ) lies south-west of the point (x, y)] Y (x,y) X Statistics 104 (Colin Rundel) Lecture 25 April
More informationESTIMATING THE MEAN LEVEL OF FINE PARTICULATE MATTER: AN APPLICATION OF SPATIAL STATISTICS
ESTIMATING THE MEAN LEVEL OF FINE PARTICULATE MATTER: AN APPLICATION OF SPATIAL STATISTICS Richard L. Smith Department of Statistics and Operations Research University of North Carolina Chapel Hill, N.C.,
More informationCovariance function estimation in Gaussian process regression
Covariance function estimation in Gaussian process regression François Bachoc Department of Statistics and Operations Research, University of Vienna WU Research Seminar - May 2015 François Bachoc Gaussian
More informationOptimization. The value x is called a maximizer of f and is written argmax X f. g(λx + (1 λ)y) < λg(x) + (1 λ)g(y) 0 < λ < 1; x, y X.
Optimization Background: Problem: given a function f(x) defined on X, find x such that f(x ) f(x) for all x X. The value x is called a maximizer of f and is written argmax X f. In general, argmax X f may
More informationSolving the Stochastic Steady-State Diffusion Problem Using Multigrid
Solving the Stochastic Steady-State Diffusion Problem Using Multigrid Tengfei Su Applied Mathematics and Scientific Computing Advisor: Howard Elman Department of Computer Science Sept. 29, 2015 Tengfei
More informationPoint-Referenced Data Models
Point-Referenced Data Models Jamie Monogan University of Georgia Spring 2013 Jamie Monogan (UGA) Point-Referenced Data Models Spring 2013 1 / 19 Objectives By the end of these meetings, participants should
More informationCS281 Section 4: Factor Analysis and PCA
CS81 Section 4: Factor Analysis and PCA Scott Linderman At this point we have seen a variety of machine learning models, with a particular emphasis on models for supervised learning. In particular, we
More informationBasics of Point-Referenced Data Models
Basics of Point-Referenced Data Models Basic tool is a spatial process, {Y (s), s D}, where D R r Chapter 2: Basics of Point-Referenced Data Models p. 1/45 Basics of Point-Referenced Data Models Basic
More informationMachine Learning 2nd Edition
INTRODUCTION TO Lecture Slides for Machine Learning 2nd Edition ETHEM ALPAYDIN, modified by Leonardo Bobadilla and some parts from http://www.cs.tau.ac.il/~apartzin/machinelearning/ The MIT Press, 2010
More informationSTA 294: Stochastic Processes & Bayesian Nonparametrics
MARKOV CHAINS AND CONVERGENCE CONCEPTS Markov chains are among the simplest stochastic processes, just one step beyond iid sequences of random variables. Traditionally they ve been used in modelling a
More informationBAYESIAN KRIGING AND BAYESIAN NETWORK DESIGN
BAYESIAN KRIGING AND BAYESIAN NETWORK DESIGN Richard L. Smith Department of Statistics and Operations Research University of North Carolina Chapel Hill, N.C., U.S.A. J. Stuart Hunter Lecture TIES 2004
More informationNearest Neighbor Gaussian Processes for Large Spatial Data
Nearest Neighbor Gaussian Processes for Large Spatial Data Abhi Datta 1, Sudipto Banerjee 2 and Andrew O. Finley 3 July 31, 2017 1 Department of Biostatistics, Bloomberg School of Public Health, Johns
More informationComputation fundamentals of discrete GMRF representations of continuous domain spatial models
Computation fundamentals of discrete GMRF representations of continuous domain spatial models Finn Lindgren September 23 2015 v0.2.2 Abstract The fundamental formulas and algorithms for Bayesian spatial
More informationOn prediction and density estimation Peter McCullagh University of Chicago December 2004
On prediction and density estimation Peter McCullagh University of Chicago December 2004 Summary Having observed the initial segment of a random sequence, subsequent values may be predicted by calculating
More informationChapter 5 Prediction of Random Variables
Chapter 5 Prediction of Random Variables C R Henderson 1984 - Guelph We have discussed estimation of β, regarded as fixed Now we shall consider a rather different problem, prediction of random variables,
More informationWrapped Gaussian processes: a short review and some new results
Wrapped Gaussian processes: a short review and some new results Giovanna Jona Lasinio 1, Gianluca Mastrantonio 2 and Alan Gelfand 3 1-Università Sapienza di Roma 2- Università RomaTRE 3- Duke University
More informationGaussian Process Regression
Gaussian Process Regression 4F1 Pattern Recognition, 21 Carl Edward Rasmussen Department of Engineering, University of Cambridge November 11th - 16th, 21 Rasmussen (Engineering, Cambridge) Gaussian Process
More informationTAMS39 Lecture 10 Principal Component Analysis Factor Analysis
TAMS39 Lecture 10 Principal Component Analysis Factor Analysis Martin Singull Department of Mathematics Mathematical Statistics Linköping University, Sweden Content - Lecture Principal component analysis
More informationSimulating with uncertainty : the rough surface scattering problem
Simulating with uncertainty : the rough surface scattering problem Uday Khankhoje Assistant Professor, Electrical Engineering Indian Institute of Technology Madras Uday Khankhoje (EE, IITM) Simulating
More informationEstimation of Copula Models with Discrete Margins (via Bayesian Data Augmentation) Michael S. Smith
Estimation of Copula Models with Discrete Margins (via Bayesian Data Augmentation) Michael S. Smith Melbourne Business School, University of Melbourne (Joint with Mohamad Khaled, University of Queensland)
More informationGAUSSIAN PROCESS REGRESSION
GAUSSIAN PROCESS REGRESSION CSE 515T Spring 2015 1. BACKGROUND The kernel trick again... The Kernel Trick Consider again the linear regression model: y(x) = φ(x) w + ε, with prior p(w) = N (w; 0, Σ). The
More informationStudent-t Process as Alternative to Gaussian Processes Discussion
Student-t Process as Alternative to Gaussian Processes Discussion A. Shah, A. G. Wilson and Z. Gharamani Discussion by: R. Henao Duke University June 20, 2014 Contributions The paper is concerned about
More informationA Statistical Look at Spectral Graph Analysis. Deep Mukhopadhyay
A Statistical Look at Spectral Graph Analysis Deep Mukhopadhyay Department of Statistics, Temple University Office: Speakman 335 deep@temple.edu http://sites.temple.edu/deepstat/ Graph Signal Processing
More informationStatistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach
Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach Jae-Kwang Kim Department of Statistics, Iowa State University Outline 1 Introduction 2 Observed likelihood 3 Mean Score
More informationNonparametric Bayesian Methods
Nonparametric Bayesian Methods Debdeep Pati Florida State University October 2, 2014 Large spatial datasets (Problem of big n) Large observational and computer-generated datasets: Often have spatial and
More informationMatrices and Multivariate Statistics - II
Matrices and Multivariate Statistics - II Richard Mott November 2011 Multivariate Random Variables Consider a set of dependent random variables z = (z 1,..., z n ) E(z i ) = µ i cov(z i, z j ) = σ ij =
More informationModelling Dropouts by Conditional Distribution, a Copula-Based Approach
The 8th Tartu Conference on MULTIVARIATE STATISTICS, The 6th Conference on MULTIVARIATE DISTRIBUTIONS with Fixed Marginals Modelling Dropouts by Conditional Distribution, a Copula-Based Approach Ene Käärik
More informationSpring 2012 Math 541A Exam 1. X i, S 2 = 1 n. n 1. X i I(X i < c), T n =
Spring 2012 Math 541A Exam 1 1. (a) Let Z i be independent N(0, 1), i = 1, 2,, n. Are Z = 1 n n Z i and S 2 Z = 1 n 1 n (Z i Z) 2 independent? Prove your claim. (b) Let X 1, X 2,, X n be independent identically
More informationLECTURE 16 GAUSS QUADRATURE In general for Newton-Cotes (equispaced interpolation points/ data points/ integration points/ nodes).
CE 025 - Lecture 6 LECTURE 6 GAUSS QUADRATURE In general for ewton-cotes (equispaced interpolation points/ data points/ integration points/ nodes). x E x S fx dx hw' o f o + w' f + + w' f + E 84 f 0 f
More informationA Stochastic Collocation based. for Data Assimilation
A Stochastic Collocation based Kalman Filter (SCKF) for Data Assimilation Lingzao Zeng and Dongxiao Zhang University of Southern California August 11, 2009 Los Angeles Outline Introduction SCKF Algorithm
More informationChapter 4 - Fundamentals of spatial processes Lecture notes
Chapter 4 - Fundamentals of spatial processes Lecture notes Geir Storvik January 21, 2013 STK4150 - Intro 2 Spatial processes Typically correlation between nearby sites Mostly positive correlation Negative
More informationLinear Methods for Prediction
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this
More informationFast Algorithm for Computing Karhunen-Loève Expansion
Fast Algorithm for Computing Karhunen-Loève Expansion Qian-Yong Chen Abstract Karhunen Loève expansion has been proven to be an efficient way to approximate second-order random processes. But numerical
More informationMonitoring Wafer Geometric Quality using Additive Gaussian Process
Monitoring Wafer Geometric Quality using Additive Gaussian Process Linmiao Zhang 1 Kaibo Wang 2 Nan Chen 1 1 Department of Industrial and Systems Engineering, National University of Singapore 2 Department
More informationFast Direct Methods for Gaussian Processes
Fast Direct Methods for Gaussian Processes Mike O Neil Departments of Mathematics New York University oneil@cims.nyu.edu December 12, 2015 1 Collaborators This is joint work with: Siva Ambikasaran Dan
More informationRegression diagnostics
Regression diagnostics Kerby Shedden Department of Statistics, University of Michigan November 5, 018 1 / 6 Motivation When working with a linear model with design matrix X, the conventional linear model
More informationMetropolis-Hastings Algorithm
Strength of the Gibbs sampler Metropolis-Hastings Algorithm Easy algorithm to think about. Exploits the factorization properties of the joint probability distribution. No difficult choices to be made to
More informationMultiple Random Variables
Multiple Random Variables This Version: July 30, 2015 Multiple Random Variables 2 Now we consider models with more than one r.v. These are called multivariate models For instance: height and weight An
More information10708 Graphical Models: Homework 2
10708 Graphical Models: Homework 2 Due Monday, March 18, beginning of class Feburary 27, 2013 Instructions: There are five questions (one for extra credit) on this assignment. There is a problem involves
More informationLecture 1: Center for Uncertainty Quantification. Alexander Litvinenko. Computation of Karhunen-Loeve Expansion:
tifica Lecture 1: Computation of Karhunen-Loeve Expansion: Alexander Litvinenko http://sri-uq.kaust.edu.sa/ Stochastic PDEs We consider div(κ(x, ω) u) = f (x, ω) in G, u = 0 on G, with stochastic coefficients
More informationMultivariate spatial modeling
Multivariate spatial modeling Point-referenced spatial data often come as multivariate measurements at each location Chapter 7: Multivariate Spatial Modeling p. 1/21 Multivariate spatial modeling Point-referenced
More informationMIT Spring 2015
Regression Analysis MIT 18.472 Dr. Kempthorne Spring 2015 1 Outline Regression Analysis 1 Regression Analysis 2 Multiple Linear Regression: Setup Data Set n cases i = 1, 2,..., n 1 Response (dependent)
More informationCopula modeling for discrete data
Copula modeling for discrete data Christian Genest & Johanna G. Nešlehová in collaboration with Bruno Rémillard McGill University and HEC Montréal ROBUST, September 11, 2016 Main question Suppose (X 1,
More informationPrinciple Components Analysis (PCA) Relationship Between a Linear Combination of Variables and Axes Rotation for PCA
Principle Components Analysis (PCA) Relationship Between a Linear Combination of Variables and Axes Rotation for PCA Principle Components Analysis: Uses one group of variables (we will call this X) In
More informationNumerical methods for the discretization of random fields by means of the Karhunen Loève expansion
Numerical methods for the discretization of random fields by means of the Karhunen Loève expansion Wolfgang Betz, Iason Papaioannou, Daniel Straub Engineering Risk Analysis Group, Technische Universität
More informationModel Selection for Geostatistical Models
Model Selection for Geostatistical Models Richard A. Davis Colorado State University http://www.stat.colostate.edu/~rdavis/lectures Joint work with: Jennifer A. Hoeting, Colorado State University Andrew
More informationPracticum : Spatial Regression
: Alexandra M. Schmidt Instituto de Matemática UFRJ - www.dme.ufrj.br/ alex 2014 Búzios, RJ, www.dme.ufrj.br Exploratory (Spatial) Data Analysis 1. Non-spatial summaries Numerical summaries: Mean, median,
More informationspbayes: An R Package for Univariate and Multivariate Hierarchical Point-referenced Spatial Models
spbayes: An R Package for Univariate and Multivariate Hierarchical Point-referenced Spatial Models Andrew O. Finley 1, Sudipto Banerjee 2, and Bradley P. Carlin 2 1 Michigan State University, Departments
More informationAn Introduction to Spatial Statistics. Chunfeng Huang Department of Statistics, Indiana University
An Introduction to Spatial Statistics Chunfeng Huang Department of Statistics, Indiana University Microwave Sounding Unit (MSU) Anomalies (Monthly): 1979-2006. Iron Ore (Cressie, 1986) Raw percent data
More informationModeling Real Estate Data using Quantile Regression
Modeling Real Estate Data using Semiparametric Quantile Regression Department of Statistics University of Innsbruck September 9th, 2011 Overview 1 Application: 2 3 4 Hedonic regression data for house prices
More informationThe Expectation Maximization or EM algorithm
The Expectation Maximization or EM algorithm Carl Edward Rasmussen November 15th, 2017 Carl Edward Rasmussen The EM algorithm November 15th, 2017 1 / 11 Contents notation, objective the lower bound functional,
More informationMultivariate Random Variable
Multivariate Random Variable Author: Author: Andrés Hincapié and Linyi Cao This Version: August 7, 2016 Multivariate Random Variable 3 Now we consider models with more than one r.v. These are called multivariate
More informationMultivariate Analysis and Likelihood Inference
Multivariate Analysis and Likelihood Inference Outline 1 Joint Distribution of Random Variables 2 Principal Component Analysis (PCA) 3 Multivariate Normal Distribution 4 Likelihood Inference Joint density
More informationJoint Factor Analysis for Speaker Verification
Joint Factor Analysis for Speaker Verification Mengke HU ASPITRG Group, ECE Department Drexel University mengke.hu@gmail.com October 12, 2012 1/37 Outline 1 Speaker Verification Baseline System Session
More information