Spatial Statistics with Image Analysis. Outline. A Statistical Approach. Johan Lindström 1. Lund October 6, 2016
|
|
- Sharleen Hall
- 6 years ago
- Views:
Transcription
1 Spatial Statistics Spatial Examples More Spatial Statistics with Image Analysis Johan Lindström 1 1 Mathematical Statistics Centre for Mathematical Sciences Lund University Lund October 6, 2016 Johan Lindström - johanl@maths.lth.se Spatial Statistics 1/31 Spatial Statistics Spatial Examples More Outline Spatial Statistics with Image Analysis Bayesian statistics Hierarchical modelling Estimation Procedures Spatial Statistics Stochastic Fields Gaussian Markov Random Fields Image Reconstruction Examples Environmental Data Corrupted Pixels NDVI Learn more A Statistical Approach Johan Lindström - johanl@maths.lth.se Spatial Statistics 2/31 Spatial Statistics Spatial Examples More Bayesian statistics Hierarchical Estimation All measurements contain random measurement errors/variation. Most natural phenomena have natural random variation. Often the uncertainty of an estimate is, at least, as important as the estimate itself. We need to describe and model random variation and uncertainties! Johan Lindström - johanl@maths.lth.se Spatial Statistics 3/31
2 Bayesian modelling Spatial Statistics Spatial Examples More Bayesian statistics Hierarchical Estimation We assume that there is some unknown truth, that we would like to find out about. This reality can be measured, usually with measurement variation, and often only partially. Bayesian modelling A Bayesian model consists of A prior, a priori, model for reality, x, given by the probability density π(x). A conditional model for data, y, given reality, with density p(y x). The prior can be expanded into several layers creating a Bayesian hierarchical model. Johan Lindström - johanl@maths.lth.se Spatial Statistics 4/31 Bayes Formula Spatial Statistics Spatial Examples More Bayesian statistics Hierarchical Estimation How should the prior and data model be combined to make statements about the reality x, given observations of y? Bayes Formula p(x y) = p(y x)π(x) p(y) p(y x)π(x) = x Ω p(y x )π(x ) dx p(x y) is called the posterior, or a posteriori, distribution. Often, only the proportionality relation p(x y) p(x, y) = p(y x)π(x) is needed, when seen as a function of x. Johan Lindström - johanl@maths.lth.se Spatial Statistics 5/31 Hierarchical Models Spatial Statistics Spatial Examples More Bayesian statistics Hierarchical Estimation We often have some prior knowledge of the reality. Given knowledge of the true reality, what can we say about images and other data? Construct a model for observations given that we know the truth. Given data, what can we say about the unknown reality? This is the inverse problem. Johan Lindström - johanl@maths.lth.se Spatial Statistics 6/31
3 Spatial Statistics Spatial Examples More Bayesian statistics Hierarchical Estimation Bayesian hierarchical modelling (BHM) A hierarchical model is constructed by systematically considering components/features of the data, and how/why these features arise. Bayesian hierarchical modelling A Bayesian hierarchical model typically consists of (at least) Data model, p(y x): Describing how observations arise assuming known latent variables x. Latent model, p(x θ): Describing how the latent variables (reality) behaves, assuming known parameters. Parameters, π(θ): Describing our, sometimes vauge, prior knowledge of the parameters. Estimation Procedures Johan Lindström - johanl@maths.lth.se Spatial Statistics 7/31 Spatial Statistics Spatial Examples More Bayesian statistics Hierarchical Estimation Maximum A Posteriori (MAP): Maximise the posterior distribution p(x y) with respect to x. Standard optimisation methods Specialised procedures, using the model structure Simulation: Simulate samples from the posterior distribution p(x y). Estimate statistical properties from these samples. The samples can be seen as representative possible realities, given the available data. Markov chain Monte Carlo (MCMC) Gibbs sampling Image Reconstruction Johan Lindström - johanl@maths.lth.se Spatial Statistics 8/31 Spatial Interpolation Given observations at some locations (pixels), y(u i ), i = 1... n we want to make statements about the value at unobserved location(s), x(u 0 ). The typical model consists of a latent Gaussian field x N (μ, Σ), observed at locations u i, i = 1,..., n, with additive Gaussian noise (nugget or small scale variability) ( ) y i = x(u i ) + ε i ε i N 0, σ 2 ε. Johan Lindström - johanl@maths.lth.se Spatial Statistics 9/31
4 Stochastic Fields To perform the reconstruction (interpolation) we need a model for the spatial dependence between locations (pixels). 1. Assume a latent Gaussian field x N (μ, Σ). 2. Assume a regresion model for μ = Bβ. 3. Assume a parametric (stationary) model for the dependence (covariance) Σ i,j = C(x(u i ), x(u j )) = r(u i, u j ; θ) = r( ui u j ; θ). r(u i, u j ; θ) is called the covariance function. Johan Lindström - johanl@maths.lth.se Spatial Statistics 10/31 Matérn covariances functions One of the most common families of covariance functions is named after Bertil Matérn, who worked for Statens Skogsforskningsinstitut (Forest Research Institute of Sweden). Variance σ 2 > 0, scale parameter κ > 0 and shape parameter ν > 0 r M (h) = σ 2 Γ(ν) 2 ν 1 (κ h )ν K ν (κ h ), h R d, A measure of the range is given by ρ = 8ν/κ. Johan Lindström - johanl@maths.lth.se Spatial Statistics 11/ nu= nu= nu= nu= Johan Lindström - johanl@maths.lth.se Spatial Statistics 12/31
5 nu=0.5 nu= nu=2.0 nu= A local model Johan Lindström - johanl@maths.lth.se Spatial Statistics 13/31 Instead of specifying the covariance function we could consider the local behaviour of pixels. A popular model is the conditional autoregressive, CAR(1) model. x ij = 1 ( ) xi 1,j 4 + κ 2 + x i+1,j + x i,j 1 + x i,j+1 + ε, ε N (0, 1τ ) 2. This corresponds to a model for x where ( x N 0, Q 1), where Q is called the precision matrix Matérn covariances Johan Lindström - johanl@maths.lth.se Spatial Statistics 14/31 The Matérn covariance family The covariance between two points at distance h is r M (h) = σ 2 Γ(ν) 2 ν 1 (κ h )ν K ν (κ h ) Fields with Matérn covariances are solutions to a Stochastic Partial Differential Equation (SPDE) (Whittle, 1954), ( κ 2 Δ) α/2 x(u) = W(u). Johan Lindström - johanl@maths.lth.se Spatial Statistics 15/31
6 Lattice on R 2 Order α = 1 (ν = 0): κ 2 1 } {{ } (C) } 1 {{ } Δ (G) Order α = 2 (ν = 1): 1 1 κ κ }{{}}{{} 1 (C) Δ (G) }{{} Δ 2 (G 2 =GC 1 G) Spatial models for data Johan Lindström - johanl@maths.lth.se Spatial Statistics 16/31 GMRF representations of SPDEs can be constructed for to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 Δ)(τ x(u)) = W(u), u R d Spatial models for data Johan Lindström - johanl@maths.lth.se Spatial Statistics 17/31 GMRF representations of SPDEs can be constructed for to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 Δ)(τ x(u)) = W(u), u Ω Johan Lindström - johanl@maths.lth.se Spatial Statistics 17/31
7 Spatial models for data GMRF representations of SPDEs can be constructed for to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 e iπθ Δ)(τ x(u)) = W(u), u Ω Spatial models for data Johan Lindström - johanl@maths.lth.se Spatial Statistics 17/31 GMRF representations of SPDEs can be constructed for to oscillating, anisotropic, non-stationary, non-separable spatio-temporal, and multivariate fields on manifolds. (κ 2 u + m u M u )(τ u x(u)) = W(u), u Ω Johan Lindström - johanl@maths.lth.se Spatial Statistics 17/31 Image Reconstruction II Model with observations, y, and latent field, x, ( ) ( y x N Ax, σ 2 I x N μ, Q 1). and Q = κ 2 C + G or Q = κ 4 C + 2κ 2 G + GC 1 G. Interpolation using a GMRF E (x y) = μ + 1 σ 2 Q 1 x y A (y Aμ) V (x y) = Q 1 x y = (Q + 1 σ 2 A A ) 1 Johan Lindström - johanl@maths.lth.se Spatial Statistics 18/31
8 Spatial Statistics Spatial Examples More Fields GMRF Reconstruction Image Reconstruction Johan Lindstro m - johanl@maths.lth.se Spatial Statistics Spatial Examples More Johan Lindstro m - johanl@maths.lth.se Spatial Statistics Spatial Examples More Spatial Statistics 19/31 Environmental Corrupted Pixels NDVI Spatial Statistics 20/31 Environmental Corrupted Pixels NDVI Global Temperature Data January 2003 Johan Lindstro m - johanl@maths.lth.se July 2003 Spatial Statistics 21/31
9 Spatial Statistics Spatial Examples More Environmental Corrupted Pixels NDVI Global Temperature Reconstruction Global mean: 15 C. Johan Lindström - johanl@maths.lth.se Spatial Statistics 22/31 Spatial Statistics Spatial Examples More Environmental Corrupted Pixels NDVI Satellite Data Vegetation January 1999 July 1999 Johan Lindström - johanl@maths.lth.se Spatial Statistics 23/31 Spatial Statistics Spatial Examples More Environmental Corrupted Pixels NDVI Satellite Data Trend in Vegetation K 2 Estimate Independent estimates K 2 Estimate Correlated estimates Johan Lindström - johanl@maths.lth.se Spatial Statistics 24/31
10 Spatial Statistics Spatial Examples More Environmental Corrupted Pixels NDVI Image Reconstruction Corrupted Pixels Typically we don t know which pixels that are bad. A better model is then Assume an underlying image, x. Assume an indicator image for bad pixels, z. Given the indicator we either observe the correct pixel value from x or noise. Use Bayes formula to compute the distribution for the unknown image (and indicator) given observations and parameters. Johan Lindström - johanl@maths.lth.se Spatial Statistics 25/31 Spatial Statistics Spatial Examples More Environmental Corrupted Pixels NDVI Image Reconstruction Corrupted pixels image reconstruction bad pixels bad pixels estimate Johan Lindström - johanl@maths.lth.se Spatial Statistics 26/31 Spatial Statistics Spatial Examples More Environmental Corrupted Pixels NDVI Normalized difference vegetation index (NDVI) NDVI = R NIR R RED R NIR + R RED R RED is the amount of reflected red light ( μm) R NIR is the amount of reflected near-infrared light ( μm) Johan Lindström - johanl@maths.lth.se Spatial Statistics 27/31
11 Spatial Statistics Spatial Examples More Environmental Corrupted Pixels NDVI Smoothed version of the NDVI Data Smooth the data to fill in missing values and remove noise due to cloud cover, etc Important ecological questions: Plant phenology (start and end of season) Plant productivity (integral) Johan Lindström - johanl@maths.lth.se Spatial Statistics 28/31 Spatial Statistics Spatial Examples More Environmental Corrupted Pixels NDVI Smoothing of Satellite Based Vegetation Measurements NDVI May 1993 Smooth Johan Lindström - johanl@maths.lth.se Spatial Statistics 29/31 Spatial Statistics Spatial Examples More Environmental Corrupted Pixels NDVI Smoothing of Satellite Based Vegetation Measurements 240 Obs Gaussian Gaussian 1D 220 Nig 1D Johan Lindström - johanl@maths.lth.se Spatial Statistics 30/31
12 Learn more! Spatial Statistics Spatial Examples More What? Spatial statistics with image analysis, FMSN20 When? HT2-2016, October December Where? Information and Matlab files will be available at Who? Lecturer: Johan Lindström MH:319 Johan Lindström - johanl@maths.lth.se Spatial Statistics 31/31
Spatial Statistics with Image Analysis. Lecture L08. Computer exercise 3. Lecture 8. Johan Lindström. November 25, 2016
C3 Repetition Creating Q Spectral Non-grid Spatial Statistics with Image Analysis Lecture 8 Johan Lindström November 25, 216 Johan Lindström - johanl@maths.lth.se FMSN2/MASM25L8 1/39 Lecture L8 C3 Repetition
More informationSpatial Statistics with Image Analysis. Lecture L02. Computer exercise 0 Daily Temperature. Lecture 2. Johan Lindström.
C Stochastic fields Covariance Spatial Statistics with Image Analysis Lecture 2 Johan Lindström November 4, 26 Lecture L2 Johan Lindström - johanl@maths.lth.se FMSN2/MASM2 L /2 C Stochastic fields Covariance
More informationLatent Gaussian Processes and Stochastic Partial Differential Equations
Latent Gaussian Processes and Stochastic Partial Differential Equations Johan Lindström 1 1 Centre for Mathematical Sciences Lund University Lund 2016-02-04 Johan Lindström - johanl@maths.lth.se Gaussian
More informationFast algorithms that utilise the sparsity of Q exist (c-package ( Q 1. GMRFlib) Seattle February 3, 2009
GMRF:s Results References Some Theory for and Applications of Gaussian Markov Random Fields Johan Lindström 1 David Bolin 1 Finn Lindgren 1 Håvard Rue 2 1 Centre for Mathematical Sciences Lund University
More informationSpatial Statistics with Image Analysis. Lecture L11. Home assignment 3. Lecture 11. Johan Lindström. December 5, 2016.
HA3 MRF:s Simulation Estimation Spatial Statistics with Image Analysis Lecture 11 Johan Lindström December 5, 2016 Lecture L11 Johan Lindström - johanl@maths.lth.se FMSN20/MASM25 L11 1/22 HA3 MRF:s Simulation
More informationData are collected along transect lines, with dense data along. Spatial modelling using GMRFs. 200m. Today? Spatial modelling using GMRFs
Centre for Mathematical Sciences Lund University Engineering geology Lund University Results A non-stationary extension The model Estimation Gaussian Markov random fields Basics Approximating Mate rn covariances
More informationMultivariate Gaussian Random Fields with SPDEs
Multivariate Gaussian Random Fields with SPDEs Xiangping Hu Daniel Simpson, Finn Lindgren and Håvard Rue Department of Mathematics, University of Oslo PASI, 214 Outline The Matérn covariance function and
More informationA short introduction to INLA and R-INLA
A short introduction to INLA and R-INLA Integrated Nested Laplace Approximation Thomas Opitz, BioSP, INRA Avignon Workshop: Theory and practice of INLA and SPDE November 7, 2018 2/21 Plan for this talk
More informationSpatial smoothing over complex domain
Spatial smoothing over complex domain Alessandro Ottavi NTNU Department of Mathematical Science August 16-20 2010 Outline Spatial smoothing Complex domain The SPDE approch Manifold Some tests Spatial smoothing
More informationThe Bayesian approach to inverse problems
The Bayesian approach to inverse problems Youssef Marzouk Department of Aeronautics and Astronautics Center for Computational Engineering Massachusetts Institute of Technology ymarz@mit.edu, http://uqgroup.mit.edu
More informationDeblurring Jupiter (sampling in GLIP faster than regularized inversion) Colin Fox Richard A. Norton, J.
Deblurring Jupiter (sampling in GLIP faster than regularized inversion) Colin Fox fox@physics.otago.ac.nz Richard A. Norton, J. Andrés Christen Topics... Backstory (?) Sampling in linear-gaussian hierarchical
More informationPractical Bayesian Optimization of Machine Learning. Learning Algorithms
Practical Bayesian Optimization of Machine Learning Algorithms CS 294 University of California, Berkeley Tuesday, April 20, 2016 Motivation Machine Learning Algorithms (MLA s) have hyperparameters that
More informationIntroduction to Bayesian methods in inverse problems
Introduction to Bayesian methods in inverse problems Ville Kolehmainen 1 1 Department of Applied Physics, University of Eastern Finland, Kuopio, Finland March 4 2013 Manchester, UK. Contents Introduction
More informationSummary STK 4150/9150
STK4150 - Intro 1 Summary STK 4150/9150 Odd Kolbjørnsen May 22 2017 Scope You are expected to know and be able to use basic concepts introduced in the book. You knowledge is expected to be larger than
More informationBayesian Inference and MCMC
Bayesian Inference and MCMC Aryan Arbabi Partly based on MCMC slides from CSC412 Fall 2018 1 / 18 Bayesian Inference - Motivation Consider we have a data set D = {x 1,..., x n }. E.g each x i can be the
More informationR-INLA. Sam Clifford Su-Yun Kang Jeff Hsieh. 30 August Bayesian Research and Analysis Group 1 / 14
1 / 14 R-INLA Sam Clifford Su-Yun Kang Jeff Hsieh Bayesian Research and Analysis Group 30 August 2012 What is R-INLA? R package for Bayesian computation Integrated Nested Laplace Approximation MCMC free
More informationAn EM algorithm for Gaussian Markov Random Fields
An EM algorithm for Gaussian Markov Random Fields Will Penny, Wellcome Department of Imaging Neuroscience, University College, London WC1N 3BG. wpenny@fil.ion.ucl.ac.uk October 28, 2002 Abstract Lavine
More informationMCMC Sampling for Bayesian Inference using L1-type Priors
MÜNSTER MCMC Sampling for Bayesian Inference using L1-type Priors (what I do whenever the ill-posedness of EEG/MEG is just not frustrating enough!) AG Imaging Seminar Felix Lucka 26.06.2012 , MÜNSTER Sampling
More informationLecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing
More informationLecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond Department of Biomedical Engineering and Computational Science Aalto University January 26, 2012 Contents 1 Batch and Recursive Estimation
More informationBayesian spatial hierarchical modeling for temperature extremes
Bayesian spatial hierarchical modeling for temperature extremes Indriati Bisono Dr. Andrew Robinson Dr. Aloke Phatak Mathematics and Statistics Department The University of Melbourne Maths, Informatics
More informationDensity Estimation. Seungjin Choi
Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More informationHierarchical Modeling for Univariate Spatial Data
Hierarchical Modeling for Univariate Spatial Data Geography 890, Hierarchical Bayesian Models for Environmental Spatial Data Analysis February 15, 2011 1 Spatial Domain 2 Geography 890 Spatial Domain This
More informationLecture 13 Fundamentals of Bayesian Inference
Lecture 13 Fundamentals of Bayesian Inference Dennis Sun Stats 253 August 11, 2014 Outline of Lecture 1 Bayesian Models 2 Modeling Correlations Using Bayes 3 The Universal Algorithm 4 BUGS 5 Wrapping Up
More informationRiemann Manifold Methods in Bayesian Statistics
Ricardo Ehlers ehlers@icmc.usp.br Applied Maths and Stats University of São Paulo, Brazil Working Group in Statistical Learning University College Dublin September 2015 Bayesian inference is based on Bayes
More informationSTAT 518 Intro Student Presentation
STAT 518 Intro Student Presentation Wen Wei Loh April 11, 2013 Title of paper Radford M. Neal [1999] Bayesian Statistics, 6: 475-501, 1999 What the paper is about Regression and Classification Flexible
More informationIntroduction to Markov Chain Monte Carlo & Gibbs Sampling
Introduction to Markov Chain Monte Carlo & Gibbs Sampling Prof. Nicholas Zabaras Sibley School of Mechanical and Aerospace Engineering 101 Frank H. T. Rhodes Hall Ithaca, NY 14853-3801 Email: zabaras@cornell.edu
More informationBayesian Methods and Uncertainty Quantification for Nonlinear Inverse Problems
Bayesian Methods and Uncertainty Quantification for Nonlinear Inverse Problems John Bardsley, University of Montana Collaborators: H. Haario, J. Kaipio, M. Laine, Y. Marzouk, A. Seppänen, A. Solonen, Z.
More informationSwitching Regime Estimation
Switching Regime Estimation Series de Tiempo BIrkbeck March 2013 Martin Sola (FE) Markov Switching models 01/13 1 / 52 The economy (the time series) often behaves very different in periods such as booms
More informationBAYESIAN METHODS FOR VARIABLE SELECTION WITH APPLICATIONS TO HIGH-DIMENSIONAL DATA
BAYESIAN METHODS FOR VARIABLE SELECTION WITH APPLICATIONS TO HIGH-DIMENSIONAL DATA Intro: Course Outline and Brief Intro to Marina Vannucci Rice University, USA PASI-CIMAT 04/28-30/2010 Marina Vannucci
More informationNon-Parametric Bayes
Non-Parametric Bayes Mark Schmidt UBC Machine Learning Reading Group January 2016 Current Hot Topics in Machine Learning Bayesian learning includes: Gaussian processes. Approximate inference. Bayesian
More informationAn example of Bayesian reasoning Consider the one-dimensional deconvolution problem with various degrees of prior information.
An example of Bayesian reasoning Consider the one-dimensional deconvolution problem with various degrees of prior information. Model: where g(t) = a(t s)f(s)ds + e(t), a(t) t = (rapidly). The problem,
More informationExercises Tutorial at ICASSP 2016 Learning Nonlinear Dynamical Models Using Particle Filters
Exercises Tutorial at ICASSP 216 Learning Nonlinear Dynamical Models Using Particle Filters Andreas Svensson, Johan Dahlin and Thomas B. Schön March 18, 216 Good luck! 1 [Bootstrap particle filter for
More informationDRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING. Inversion basics. Erkki Kyrölä Finnish Meteorological Institute
Inversion basics y = Kx + ε x ˆ = (K T K) 1 K T y Erkki Kyrölä Finnish Meteorological Institute Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 1 Contents 1. Introduction: Measurements, models, inversion
More informationNonstationary spatial process modeling Part II Paul D. Sampson --- Catherine Calder Univ of Washington --- Ohio State University
Nonstationary spatial process modeling Part II Paul D. Sampson --- Catherine Calder Univ of Washington --- Ohio State University this presentation derived from that presented at the Pan-American Advanced
More informationBayesian Methods for Machine Learning
Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),
More informationGaussian Process Regression
Gaussian Process Regression 4F1 Pattern Recognition, 21 Carl Edward Rasmussen Department of Engineering, University of Cambridge November 11th - 16th, 21 Rasmussen (Engineering, Cambridge) Gaussian Process
More informationSequential Monte Carlo Samplers for Applications in High Dimensions
Sequential Monte Carlo Samplers for Applications in High Dimensions Alexandros Beskos National University of Singapore KAUST, 26th February 2014 Joint work with: Dan Crisan, Ajay Jasra, Nik Kantas, Alex
More informationDynamic System Identification using HDMR-Bayesian Technique
Dynamic System Identification using HDMR-Bayesian Technique *Shereena O A 1) and Dr. B N Rao 2) 1), 2) Department of Civil Engineering, IIT Madras, Chennai 600036, Tamil Nadu, India 1) ce14d020@smail.iitm.ac.in
More informationA Search and Jump Algorithm for Markov Chain Monte Carlo Sampling. Christopher Jennison. Adriana Ibrahim. Seminar at University of Kuwait
A Search and Jump Algorithm for Markov Chain Monte Carlo Sampling Christopher Jennison Department of Mathematical Sciences, University of Bath, UK http://people.bath.ac.uk/mascj Adriana Ibrahim Institute
More informationHastings-within-Gibbs Algorithm: Introduction and Application on Hierarchical Model
UNIVERSITY OF TEXAS AT SAN ANTONIO Hastings-within-Gibbs Algorithm: Introduction and Application on Hierarchical Model Liang Jing April 2010 1 1 ABSTRACT In this paper, common MCMC algorithms are introduced
More informationRobust MCMC Sampling with Non-Gaussian and Hierarchical Priors
Division of Engineering & Applied Science Robust MCMC Sampling with Non-Gaussian and Hierarchical Priors IPAM, UCLA, November 14, 2017 Matt Dunlop Victor Chen (Caltech) Omiros Papaspiliopoulos (ICREA,
More informationLinear Dynamical Systems (Kalman filter)
Linear Dynamical Systems (Kalman filter) (a) Overview of HMMs (b) From HMMs to Linear Dynamical Systems (LDS) 1 Markov Chains with Discrete Random Variables x 1 x 2 x 3 x T Let s assume we have discrete
More informationThe Metropolis-Hastings Algorithm. June 8, 2012
The Metropolis-Hastings Algorithm June 8, 22 The Plan. Understand what a simulated distribution is 2. Understand why the Metropolis-Hastings algorithm works 3. Learn how to apply the Metropolis-Hastings
More informationClimate Change: the Uncertainty of Certainty
Climate Change: the Uncertainty of Certainty Reinhard Furrer, UZH JSS, Geneva Oct. 30, 2009 Collaboration with: Stephan Sain - NCAR Reto Knutti - ETHZ Claudia Tebaldi - Climate Central Ryan Ford, Doug
More informationDefault Priors and Effcient Posterior Computation in Bayesian
Default Priors and Effcient Posterior Computation in Bayesian Factor Analysis January 16, 2010 Presented by Eric Wang, Duke University Background and Motivation A Brief Review of Parameter Expansion Literature
More informationAges of stellar populations from color-magnitude diagrams. Paul Baines. September 30, 2008
Ages of stellar populations from color-magnitude diagrams Paul Baines Department of Statistics Harvard University September 30, 2008 Context & Example Welcome! Today we will look at using hierarchical
More informationInference and estimation in probabilistic time series models
1 Inference and estimation in probabilistic time series models David Barber, A Taylan Cemgil and Silvia Chiappa 11 Time series The term time series refers to data that can be represented as a sequence
More informationModels for models. Douglas Nychka Geophysical Statistics Project National Center for Atmospheric Research
Models for models Douglas Nychka Geophysical Statistics Project National Center for Atmospheric Research Outline Statistical models and tools Spatial fields (Wavelets) Climate regimes (Regression and clustering)
More informationSpatio-temporal precipitation modeling based on time-varying regressions
Spatio-temporal precipitation modeling based on time-varying regressions Oleg Makhnin Department of Mathematics New Mexico Tech Socorro, NM 87801 January 19, 2007 1 Abstract: A time-varying regression
More informationParameter Estimation. William H. Jefferys University of Texas at Austin Parameter Estimation 7/26/05 1
Parameter Estimation William H. Jefferys University of Texas at Austin bill@bayesrules.net Parameter Estimation 7/26/05 1 Elements of Inference Inference problems contain two indispensable elements: Data
More informationHierarchical Modeling for Spatial Data
Bayesian Spatial Modelling Spatial model specifications: P(y X, θ). Prior specifications: P(θ). Posterior inference of model parameters: P(θ y). Predictions at new locations: P(y 0 y). Model comparisons.
More informationChapter 4 - Fundamentals of spatial processes Lecture notes
Chapter 4 - Fundamentals of spatial processes Lecture notes Geir Storvik January 21, 2013 STK4150 - Intro 2 Spatial processes Typically correlation between nearby sites Mostly positive correlation Negative
More informationChapter 4 - Fundamentals of spatial processes Lecture notes
TK4150 - Intro 1 Chapter 4 - Fundamentals of spatial processes Lecture notes Odd Kolbjørnsen and Geir Storvik January 30, 2017 STK4150 - Intro 2 Spatial processes Typically correlation between nearby sites
More informationKernel adaptive Sequential Monte Carlo
Kernel adaptive Sequential Monte Carlo Ingmar Schuster (Paris Dauphine) Heiko Strathmann (University College London) Brooks Paige (Oxford) Dino Sejdinovic (Oxford) December 7, 2015 1 / 36 Section 1 Outline
More informationHierarchical Modelling for Univariate Spatial Data
Hierarchical Modelling for Univariate Spatial Data Sudipto Banerjee 1 and Andrew O. Finley 2 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department
More informationProbabilistic Machine Learning
Probabilistic Machine Learning Bayesian Nets, MCMC, and more Marek Petrik 4/18/2017 Based on: P. Murphy, K. (2012). Machine Learning: A Probabilistic Perspective. Chapter 10. Conditional Independence Independent
More informationOverlapping block proposals for latent Gaussian Markov random fields
NORGES TEKNISK-NATURVITENSKAPELIGE UNIVERSITET Overlapping block proposals for latent Gaussian Markov random fields by Ingelin Steinsland and Håvard Rue PREPRINT STATISTICS NO. 8/3 NORWEGIAN UNIVERSITY
More informationRECONSTRUCTION OF NDVI DATA WITH NORMAL VARIANCE-MEAN MIXTURE NOISE USING STOCHASTIC GRADIENT METHODS
RECONSTRUCTION OF NDVI DATA WITH NORMAL VARIANCE-MEAN MIXTURE NOISE USING STOCHASTIC GRADIENT METHODS DAVID KRAFT Master s thesis 217:E1 Faculty of Engineering Centre for Mathematical Sciences Mathematical
More informationIntroduction to Machine Learning
Introduction to Machine Learning Brown University CSCI 1950-F, Spring 2012 Prof. Erik Sudderth Lecture 25: Markov Chain Monte Carlo (MCMC) Course Review and Advanced Topics Many figures courtesy Kevin
More informationLecture: Gaussian Process Regression. STAT 6474 Instructor: Hongxiao Zhu
Lecture: Gaussian Process Regression STAT 6474 Instructor: Hongxiao Zhu Motivation Reference: Marc Deisenroth s tutorial on Robot Learning. 2 Fast Learning for Autonomous Robots with Gaussian Processes
More informationIMAGE MODELLING AND ESTIMATION FINN LINDGREN A STATISTICAL APPROACH CENTRUM SCIENTIARUM MATHEMATICARUM. Third edition, , Chapter 1 5
IMAGE MODELLING AND ESTIMATION A STATISTICAL APPROACH FINN LINDGREN Third edition, 2006-10-20, Chapter 1 5 Centre for Mathematical Sciences Mathematical Statistics CENTRUM SCIENTIARUM MATHEMATICARUM 1
More informationGaussian with mean ( µ ) and standard deviation ( σ)
Slide from Pieter Abbeel Gaussian with mean ( µ ) and standard deviation ( σ) 10/6/16 CSE-571: Robotics X ~ N( µ, σ ) Y ~ N( aµ + b, a σ ) Y = ax + b + + + + 1 1 1 1 1 1 1 1 1 1, ~ ) ( ) ( ), ( ~ ), (
More informationCPSC 540: Machine Learning
CPSC 540: Machine Learning MCMC and Non-Parametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is
More informationStatistics & Data Sciences: First Year Prelim Exam May 2018
Statistics & Data Sciences: First Year Prelim Exam May 2018 Instructions: 1. Do not turn this page until instructed to do so. 2. Start each new question on a new sheet of paper. 3. This is a closed book
More informationCirculant embedding of approximate covariances for inference from Gaussian data on large lattices
Circulant embedding of approximate covariances for inference from Gaussian data on large lattices Joseph Guinness and Montserrat Fuentes Abstract Recently proposed computationally efficient Markov chain
More informationLecture 6: Gaussian Mixture Models (GMM)
Helsinki Institute for Information Technology Lecture 6: Gaussian Mixture Models (GMM) Pedram Daee 3.11.2015 Outline Gaussian Mixture Models (GMM) Models Model families and parameters Parameter learning
More informationMarkov Chain Monte Carlo methods
Markov Chain Monte Carlo methods By Oleg Makhnin 1 Introduction a b c M = d e f g h i 0 f(x)dx 1.1 Motivation 1.1.1 Just here Supresses numbering 1.1.2 After this 1.2 Literature 2 Method 2.1 New math As
More informationHierarchical Modelling for Univariate Spatial Data
Spatial omain Hierarchical Modelling for Univariate Spatial ata Sudipto Banerjee 1 and Andrew O. Finley 2 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A.
More informationProbabilistic Graphical Models Lecture 20: Gaussian Processes
Probabilistic Graphical Models Lecture 20: Gaussian Processes Andrew Gordon Wilson www.cs.cmu.edu/~andrewgw Carnegie Mellon University March 30, 2015 1 / 53 What is Machine Learning? Machine learning algorithms
More informationBayesian parameter estimation in predictive engineering
Bayesian parameter estimation in predictive engineering Damon McDougall Institute for Computational Engineering and Sciences, UT Austin 14th August 2014 1/27 Motivation Understand physical phenomena Observations
More informationPrinciples of Bayesian Inference
Principles of Bayesian Inference Sudipto Banerjee University of Minnesota July 20th, 2008 1 Bayesian Principles Classical statistics: model parameters are fixed and unknown. A Bayesian thinks of parameters
More informationComputer Vision Group Prof. Daniel Cremers. 14. Sampling Methods
Prof. Daniel Cremers 14. Sampling Methods Sampling Methods Sampling Methods are widely used in Computer Science as an approximation of a deterministic algorithm to represent uncertainty without a parametric
More informationMarkov chain Monte Carlo methods in atmospheric remote sensing
1 / 45 Markov chain Monte Carlo methods in atmospheric remote sensing Johanna Tamminen johanna.tamminen@fmi.fi ESA Summer School on Earth System Monitoring and Modeling July 3 Aug 11, 212, Frascati July,
More informationSTA414/2104 Statistical Methods for Machine Learning II
STA414/2104 Statistical Methods for Machine Learning II Murat A. Erdogdu & David Duvenaud Department of Computer Science Department of Statistical Sciences Lecture 3 Slide credits: Russ Salakhutdinov Announcements
More informationMonte Carlo simulation inspired by computational optimization. Colin Fox Al Parker, John Bardsley MCQMC Feb 2012, Sydney
Monte Carlo simulation inspired by computational optimization Colin Fox fox@physics.otago.ac.nz Al Parker, John Bardsley MCQMC Feb 2012, Sydney Sampling from π(x) Consider : x is high-dimensional (10 4
More informationLearning of dynamical systems
Learning of dynamical systems Particle filters and Markov chain methods Thomas B. Schön and Fredrik Lindsten c Draft date August 23, 2017 2 Contents 1 Introduction 3 1.1 A few words for readers of the
More informationIntroduction to Systems Analysis and Decision Making Prepared by: Jakub Tomczak
Introduction to Systems Analysis and Decision Making Prepared by: Jakub Tomczak 1 Introduction. Random variables During the course we are interested in reasoning about considered phenomenon. In other words,
More informationModels for spatial data (cont d) Types of spatial data. Types of spatial data (cont d) Hierarchical models for spatial data
Hierarchical models for spatial data Based on the book by Banerjee, Carlin and Gelfand Hierarchical Modeling and Analysis for Spatial Data, 2004. We focus on Chapters 1, 2 and 5. Geo-referenced data arise
More informationSTA 4273H: Sta-s-cal Machine Learning
STA 4273H: Sta-s-cal Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 2 In our
More informationMinimum Message Length Analysis of the Behrens Fisher Problem
Analysis of the Behrens Fisher Problem Enes Makalic and Daniel F Schmidt Centre for MEGA Epidemiology The University of Melbourne Solomonoff 85th Memorial Conference, 2011 Outline Introduction 1 Introduction
More informationCSci 8980: Advanced Topics in Graphical Models Gaussian Processes
CSci 8980: Advanced Topics in Graphical Models Gaussian Processes Instructor: Arindam Banerjee November 15, 2007 Gaussian Processes Outline Gaussian Processes Outline Parametric Bayesian Regression Gaussian
More informationNonparametric Bayesian Methods (Gaussian Processes)
[70240413 Statistical Machine Learning, Spring, 2015] Nonparametric Bayesian Methods (Gaussian Processes) Jun Zhu dcszj@mail.tsinghua.edu.cn http://bigml.cs.tsinghua.edu.cn/~jun State Key Lab of Intelligent
More informationNonlinear Model Reduction for Uncertainty Quantification in Large-Scale Inverse Problems
Nonlinear Model Reduction for Uncertainty Quantification in Large-Scale Inverse Problems Krzysztof Fidkowski, David Galbally*, Karen Willcox* (*MIT) Computational Aerospace Sciences Seminar Aerospace Engineering
More informationMiscellaneous. Regarding reading materials. Again, ask questions (if you have) and ask them earlier
Miscellaneous Regarding reading materials Reading materials will be provided as needed If no assigned reading, it means I think the material from class is sufficient Should be enough for you to do your
More informationGaussian processes. Chuong B. Do (updated by Honglak Lee) November 22, 2008
Gaussian processes Chuong B Do (updated by Honglak Lee) November 22, 2008 Many of the classical machine learning algorithms that we talked about during the first half of this course fit the following pattern:
More informationComputer Vision Group Prof. Daniel Cremers. 11. Sampling Methods
Prof. Daniel Cremers 11. Sampling Methods Sampling Methods Sampling Methods are widely used in Computer Science as an approximation of a deterministic algorithm to represent uncertainty without a parametric
More informationExtreme Value Analysis and Spatial Extremes
Extreme Value Analysis and Department of Statistics Purdue University 11/07/2013 Outline Motivation 1 Motivation 2 Extreme Value Theorem and 3 Bayesian Hierarchical Models Copula Models Max-stable Models
More informationVariational Methods in Bayesian Deconvolution
PHYSTAT, SLAC, Stanford, California, September 8-, Variational Methods in Bayesian Deconvolution K. Zarb Adami Cavendish Laboratory, University of Cambridge, UK This paper gives an introduction to the
More informationMarkov Chain Monte Carlo, Numerical Integration
Markov Chain Monte Carlo, Numerical Integration (See Statistics) Trevor Gallen Fall 2015 1 / 1 Agenda Numerical Integration: MCMC methods Estimating Markov Chains Estimating latent variables 2 / 1 Numerical
More informationNew Insights into History Matching via Sequential Monte Carlo
New Insights into History Matching via Sequential Monte Carlo Associate Professor Chris Drovandi School of Mathematical Sciences ARC Centre of Excellence for Mathematical and Statistical Frontiers (ACEMS)
More informationState Space Representation of Gaussian Processes
State Space Representation of Gaussian Processes Simo Särkkä Department of Biomedical Engineering and Computational Science (BECS) Aalto University, Espoo, Finland June 12th, 2013 Simo Särkkä (Aalto University)
More informationIntegrated Non-Factorized Variational Inference
Integrated Non-Factorized Variational Inference Shaobo Han, Xuejun Liao and Lawrence Carin Duke University February 27, 2014 S. Han et al. Integrated Non-Factorized Variational Inference February 27, 2014
More informationMarginal density. If the unknown is of the form x = (x 1, x 2 ) in which the target of investigation is x 1, a marginal posterior density
Marginal density If the unknown is of the form x = x 1, x 2 ) in which the target of investigation is x 1, a marginal posterior density πx 1 y) = πx 1, x 2 y)dx 2 = πx 2 )πx 1 y, x 2 )dx 2 needs to be
More informationChapter 3: Maximum-Likelihood & Bayesian Parameter Estimation (part 1)
HW 1 due today Parameter Estimation Biometrics CSE 190 Lecture 7 Today s lecture was on the blackboard. These slides are an alternative presentation of the material. CSE190, Winter10 CSE190, Winter10 Chapter
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate
More informationBayesian Regression Linear and Logistic Regression
When we want more than point estimates Bayesian Regression Linear and Logistic Regression Nicole Beckage Ordinary Least Squares Regression and Lasso Regression return only point estimates But what if we
More informationLecture 7 and 8: Markov Chain Monte Carlo
Lecture 7 and 8: Markov Chain Monte Carlo 4F13: Machine Learning Zoubin Ghahramani and Carl Edward Rasmussen Department of Engineering University of Cambridge http://mlg.eng.cam.ac.uk/teaching/4f13/ Ghahramani
More informationDisease mapping with Gaussian processes
EUROHEIS2 Kuopio, Finland 17-18 August 2010 Aki Vehtari (former Helsinki University of Technology) Department of Biomedical Engineering and Computational Science (BECS) Acknowledgments Researchers - Jarno
More informationEfficient adaptive covariate modelling for extremes
Efficient adaptive covariate modelling for extremes Slides at www.lancs.ac.uk/ jonathan Matthew Jones, David Randell, Emma Ross, Elena Zanini, Philip Jonathan Copyright of Shell December 218 1 / 23 Structural
More information