Bayesian Registration of Functions with a Gaussian Process Prior
|
|
- Magnus Sims
- 6 years ago
- Views:
Transcription
1 Bayesian Registration of Functions with a Gaussian Process Prior
2 RELATED PROBLEMS Functions Curves Images Surfaces
3 FUNCTIONAL DATA Electrocardiogram Signals (Heart Disease) Gait Pressure Measurements (Parkinson s) Gene Expression Data Respiration Strain During 4D-CT Scan Signature Acceleration Curves (Fraud Detection) Hand-grip Strength (Arthritis) Spike Trains Growth Curves
4 ALIGNMENT PROBLEM DESCRIPTION
5 MAIN GOALS The function alignment problem has been referred to by many names including registration and warping. Note: I will use these terms interchangeably throughout this talk. Main Goals: 1. Function alignment: principled approach to separate the amplitude and phase variabilities of functional datasets for subsequent statistical analysis. 2. Alignment uncertainty: framework that allows computation of confidence values associated with alignment estimates. 3. Multiple plausible alignments: approach that provides a recipe for identifying multiple good registrations. 4. Statistical analysis of warping functions: tools for computing statistics on the space of warping functions including the mean, covariance, PCA, cluster analysis, etc.
6 RELATED WORK Bayesian alignment of functions and curves: B-splines: Telesca and Inoue, 2008 (JASA) Multiresolution approach or warplets : Claeskens et al., 2010 (JRSSB) SRVF with Dirichlet prior: Cheng et al., 2016 (Bayesian Analysis) Bayesian deformable models: Allassonniere et al., 2010 (Bernoulli) Computational anatomy: Ma et al., 2008 (NeuroImage); Tang et al., 2013 (PLoS ONE) Other works: Srivastava and Jermyn, 2009 (PAMI); Zhang and Fletcher, 2016 (Algorithmic Advances in Riemannian Geometry and Applications); Wassermann et al., 2014 (WBIR); Simpson et al., 2015 (Medical Image Analysis); etc.
7 PAIRWISE ALIGNMENT PROBLEM Observation space : set of absolutely continuous functions on [0,1]. Parameter space of warping functions: acts on the function space by composition:. PQRST complex f 1 and f 2 γ f 1, f 2 γ
8 PAIRWISE ALIGNMENT PROBLEM Most common optimization approach (Dynamic Time Warping): Known issues: (1) lack of symmetry, (2) pinching effect: (one can pinch the entire function f 2 to get arbitrarily close to f 1 in norm), (3) not a proper metric on the quotient space : cannot be used for statistical analysis. Another approach based on the norm: Adding a regularization penalty on γ eliminates the pinching effect, but the other drawbacks remain. The choice of the regularization parameter λ is not obvious in general cases.
9 SOLUTION BASED ON SRSF Square-root slope function (SRSF):. Inverse-mapping is well defined and easy to compute. Space of SRSFs:. Action of warping group on SRSF space:. Important property:. Equivalence class of SRSF under warping:. Distance can be efficiently computed using Dynamic Programming (DP): This approach overcomes all previously mentioned issues. Separate issue: this is a purely optimization-based approach, which does not allow the user to easily assess alignment uncertainty or to discover multiple plausible alignments.
10 OUTLINE OF PROPOSED APPROACH Bayesian Model + Riemannian Geometry + Markov Chain Monte Carlo (1) Bayesian Alignment Model: Allows comprehensive exploration of the variable space. Provides credible intervals of warping function estimates. Allows discovering multiple registration solutions through multimodal posteriors. (2) Riemannian Geometry of Warping Group: Enables efficient Riemannian computation on the space of warping functions: tools for computing statistics and implementation of a k-means clustering algorithm to discover multiple modes in the posterior. Allows a geometric prior distribution on the space of warpings. (3) Markov Chain Monte Carlo: Allows sampling from the posterior distribution. We use a specialized preconditioned Crank Nicolson algorithm for functional parameters.
11 RIEMANNIAN GEOMETRY OF WARPINGS Riemannian metric on (, Fisher-Rao metric): This results in a non-trivial Riemannian geometry of. Simplification: Definition: Define the mapping. Then, given an element, define a new representation using the square-root of its derivative as. We refer to ψ as the square-root density (SRD). Main Result: simplifies to the positive orthant of the Hilbert sphere and the Fisher-Rao metric simplifies to the metric. The Riemannian geometry of the Hilbert sphere is well known providing efficient tools for statistical analysis on the warping group.
12 GEOMETRIC TOOLS Identity mapping, i.e. no warping:. denotes the inner product. Tangent space:. Geodesic distance:. Exponential and inverse-exponential maps:
13 STATISTICS OF WARPING FUNCTIONS Given: a sample of warping functions and their corresponding SRDs. Karcher Mean:. Geometric Median (Fletcher et al., 2009):. Both quantities found using a standard gradient descent algorithm. K-means clustering: utilize the Karcher mean and Fisher-Rao distance on the warping group.
14 PAIRWISE ALIGNMENT MODEL Function discretization:. Pairwise alignment model: Prior truncated to Ψ. C g : pre-specified covariance operator. a and b: pre-specified constants. Sampling from posterior via Metropolis within Gibbs: Update sequentially from full conditionals of g (Metropolis step using pcn) and (direct draw).
15 Z-MIXTURE pcn MCMC ALGORITHM We use the Z-mixture preconditioned Crank Nicolson (pcn) to update the function g (Cotter et al., 2013): 1. Propose. 2. Accept g with probability. (prior) via Karhunen-Loeve expansion: 1. Assume the eigenpairs of C g are (λ i2, b i (t); i 1). We use a Fourier basis. 2. Sample and let.
16 Z-MIXTURE pcn MCMC ALGORITHM β is a tuning parameter: 1. Large (small) values imply large (small) jumps in the parameter space. 2. We use a mixture to improve MCMC convergence. Advantages of mixture pcn MCMC algorithm: 1. The acceptance ratio is independent of the dimension of the discretized g. 2. The tuning parameter β is random; flexible to control size of jump.
17 MULTIPLE ALIGNMENT MODEL Multiple alignment model: Sampling from posterior by updating sequentially: g 1,,g C, q* (Metropolis steps using pcn) and (direct draw).
18 EXAMPLES: PAIRWISE SIMULATIONS Data: Three simulation settings: (1) randomly generate γ using the first pair of Fourier basis (fourier1), (2) randomly generate γ using the first 20 pairs of Fourier basis (fourier20), (3) generate γ by smoothing a random step function (random) Each simulation setting is repeated ten times. Evaluation: Fisher-Rao distance between posterior mean and true warping.
19 EXAMPLES: PAIRWISE SIMULATIONS
20 EXAMPLES: MULTIPLE ALIGNMENT Data: right knee flexion functions for 12 individuals
21 EXAMPLES: MULTIPLE ALIGNMENT Original Data Aligned Data Cross-sectional Averages Estimated Warps Simulated Gait Data: Pelvis Right Roll Spike Trains Pinch Force
22 SUMMARY AND FUTURE WORK Contributions: 1. We proposed a novel Bayesian model for pairwise and multiple alignment of functional data. 2. We use the Riemannian geometry of the space of warping functions for: defining a Gaussian prior; efficient sampling from the posterior distribution; computation of statistics of warping functions. 3. The proposed model is very good at exploring the entire variable space. In some cases, it performs better than the optimization-based DP algorithm. Future Work: 1. Theoretical properties of the model. 2. Sensitivity analysis. 3. Alternative priors and posterior sampling strategies for warping functions. 4. Extension to higher-dimensional curves, images and surfaces.
23 CBMS CONFERENCE CBMS Conference: Elastic Functional and Shape Data Analysis (EFSDA) Where: The Ohio State University, Columbus, OH When: July 16-20, 2018 Primary Lecturer: Prof. Anuj Srivastava Organizers: Sebastian Kurtek (Statistics, OSU), Facundo Memoli (Mathematics, OSU), Yusu Wang (Computer Science and Engineering, OSU), Tingting Zhang (Statistics, University of Virginia), Hongtu Zhu (Biostatistics, MD Anderson Cancer Center)
24 THANK YOU! QUESTIONS?
Alignment and Analysis of Proteomics Data using Square Root Slope Function Framework
Alignment and Analysis of Proteomics Data using Square Root Slope Function Framework J. Derek Tucker 1 1 Department of Statistics Florida State University Tallahassee, FL 32306 CTW: Statistics of Warpings
More informationThe Square Root Velocity Framework for Curves in a Homogeneous Space
The Square Root Velocity Framework for Curves in a Homogeneous Space Zhe Su 1 Eric Klassen 1 Martin Bauer 1 1 Florida State University October 8, 2017 Zhe Su (FSU) SRVF for Curves in a Homogeneous Space
More informationRegistration of Functional Data Using Fisher-Rao Metric
Registration of Functional Data Using Fisher-Rao Metric arxiv:3.387v [math.st] 6 May A. Srivastava, W. Wu, S. Kurtek, E. Klassen, and J. S. Marron Dept. of Statistics & Dept. of Mathematics, Florida State
More informationCPSC 540: Machine Learning
CPSC 540: Machine Learning MCMC and Non-Parametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is
More informationGenerative Models and Stochastic Algorithms for Population Average Estimation and Image Analysis
Generative Models and Stochastic Algorithms for Population Average Estimation and Image Analysis Stéphanie Allassonnière CIS, JHU July, 15th 28 Context : Computational Anatomy Context and motivations :
More informationChris Fraley and Daniel Percival. August 22, 2008, revised May 14, 2010
Model-Averaged l 1 Regularization using Markov Chain Monte Carlo Model Composition Technical Report No. 541 Department of Statistics, University of Washington Chris Fraley and Daniel Percival August 22,
More informationSupervised Dimension Reduction:
Supervised Dimension Reduction: A Tale of Two Manifolds S. Mukherjee, K. Mao, F. Liang, Q. Wu, M. Maggioni, D-X. Zhou Department of Statistical Science Institute for Genome Sciences & Policy Department
More informationIntroduction to Machine Learning
Introduction to Machine Learning Brown University CSCI 1950-F, Spring 2012 Prof. Erik Sudderth Lecture 25: Markov Chain Monte Carlo (MCMC) Course Review and Advanced Topics Many figures courtesy Kevin
More informationPattern Recognition and Machine Learning
Christopher M. Bishop Pattern Recognition and Machine Learning ÖSpri inger Contents Preface Mathematical notation Contents vii xi xiii 1 Introduction 1 1.1 Example: Polynomial Curve Fitting 4 1.2 Probability
More informationBayesian Nonparametric Regression for Diabetes Deaths
Bayesian Nonparametric Regression for Diabetes Deaths Brian M. Hartman PhD Student, 2010 Texas A&M University College Station, TX, USA David B. Dahl Assistant Professor Texas A&M University College Station,
More information(5) Multi-parameter models - Gibbs sampling. ST440/540: Applied Bayesian Analysis
Summarizing a posterior Given the data and prior the posterior is determined Summarizing the posterior gives parameter estimates, intervals, and hypothesis tests Most of these computations are integrals
More informationContents. Part I: Fundamentals of Bayesian Inference 1
Contents Preface xiii Part I: Fundamentals of Bayesian Inference 1 1 Probability and inference 3 1.1 The three steps of Bayesian data analysis 3 1.2 General notation for statistical inference 4 1.3 Bayesian
More informationA Bayesian Nonparametric Model for Predicting Disease Status Using Longitudinal Profiles
A Bayesian Nonparametric Model for Predicting Disease Status Using Longitudinal Profiles Jeremy Gaskins Department of Bioinformatics & Biostatistics University of Louisville Joint work with Claudio Fuentes
More informationBayesian Methods for Machine Learning
Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),
More informationNumerical Analysis for Statisticians
Kenneth Lange Numerical Analysis for Statisticians Springer Contents Preface v 1 Recurrence Relations 1 1.1 Introduction 1 1.2 Binomial CoefRcients 1 1.3 Number of Partitions of a Set 2 1.4 Horner's Method
More informationMonte Carlo in Bayesian Statistics
Monte Carlo in Bayesian Statistics Matthew Thomas SAMBa - University of Bath m.l.thomas@bath.ac.uk December 4, 2014 Matthew Thomas (SAMBa) Monte Carlo in Bayesian Statistics December 4, 2014 1 / 16 Overview
More informationFLORIDA STATE UNIVERSITY COLLEGE OF ARTS AND SCIENCES FUNCTIONAL COMPONENT ANALYSIS AND REGRESSION USING ELASTIC METHODS J.
FLORIDA STATE UNIVERSITY COLLEGE OF ARTS AND SCIENCES FUNCTIONAL COMPONENT ANALYSIS AND REGRESSION USING ELASTIC METHODS By J. DEREK TUCKER A Dissertation submitted to the Department of Statistics in partial
More informationBayesian Inference of Multiple Gaussian Graphical Models
Bayesian Inference of Multiple Gaussian Graphical Models Christine Peterson,, Francesco Stingo, and Marina Vannucci February 18, 2014 Abstract In this paper, we propose a Bayesian approach to inference
More informationLecture 16: Mixtures of Generalized Linear Models
Lecture 16: Mixtures of Generalized Linear Models October 26, 2006 Setting Outline Often, a single GLM may be insufficiently flexible to characterize the data Setting Often, a single GLM may be insufficiently
More informationNonparametric Bayesian Methods (Gaussian Processes)
[70240413 Statistical Machine Learning, Spring, 2015] Nonparametric Bayesian Methods (Gaussian Processes) Jun Zhu dcszj@mail.tsinghua.edu.cn http://bigml.cs.tsinghua.edu.cn/~jun State Key Lab of Intelligent
More informationReducing The Computational Cost of Bayesian Indoor Positioning Systems
Reducing The Computational Cost of Bayesian Indoor Positioning Systems Konstantinos Kleisouris, Richard P. Martin Computer Science Department Rutgers University WINLAB Research Review May 15 th, 2006 Motivation
More informationA Bayesian Approach to Phylogenetics
A Bayesian Approach to Phylogenetics Niklas Wahlberg Based largely on slides by Paul Lewis (www.eeb.uconn.edu) An Introduction to Bayesian Phylogenetics Bayesian inference in general Markov chain Monte
More informationDurham Research Online
Durham Research Online Deposited in DRO: 5 April 26 Version of attached le: Accepted Version Peer-review status of attached le: Peer-reviewed Citation for published item: Srivastava, A. and Jermyn, I.H.
More informationInformation geometry for bivariate distribution control
Information geometry for bivariate distribution control C.T.J.Dodson + Hong Wang Mathematics + Control Systems Centre, University of Manchester Institute of Science and Technology Optimal control of stochastic
More informationMixed effect model for the spatiotemporal analysis of longitudinal manifold value data
Mixed effect model for the spatiotemporal analysis of longitudinal manifold value data Stéphanie Allassonnière with J.B. Schiratti, O. Colliot and S. Durrleman Université Paris Descartes & Ecole Polytechnique
More informationBayesian Principal Geodesic Analysis in Diffeomorphic Image Registration
Bayesian Principal Geodesic Analysis in Diffeomorphic Image Registration Miaomiao Zhang and P. Thomas Fletcher School of Computing, University of Utah, Salt Lake City, USA Abstract. Computing a concise
More informationDevelopment of Stochastic Artificial Neural Networks for Hydrological Prediction
Development of Stochastic Artificial Neural Networks for Hydrological Prediction G. B. Kingston, M. F. Lambert and H. R. Maier Centre for Applied Modelling in Water Engineering, School of Civil and Environmental
More informationCSC 2541: Bayesian Methods for Machine Learning
CSC 2541: Bayesian Methods for Machine Learning Radford M. Neal, University of Toronto, 2011 Lecture 3 More Markov Chain Monte Carlo Methods The Metropolis algorithm isn t the only way to do MCMC. We ll
More informationNONPARAMETRIC BAYESIAN INFERENCE ON PLANAR SHAPES
NONPARAMETRIC BAYESIAN INFERENCE ON PLANAR SHAPES Author: Abhishek Bhattacharya Coauthor: David Dunson Department of Statistical Science, Duke University 7 th Workshop on Bayesian Nonparametrics Collegio
More informationManifold Monte Carlo Methods
Manifold Monte Carlo Methods Mark Girolami Department of Statistical Science University College London Joint work with Ben Calderhead Research Section Ordinary Meeting The Royal Statistical Society October
More informationParameter estimation and forecasting. Cristiano Porciani AIfA, Uni-Bonn
Parameter estimation and forecasting Cristiano Porciani AIfA, Uni-Bonn Questions? C. Porciani Estimation & forecasting 2 Temperature fluctuations Variance at multipole l (angle ~180o/l) C. Porciani Estimation
More informationSmall-variance Asymptotics for Dirichlet Process Mixtures of SVMs
Small-variance Asymptotics for Dirichlet Process Mixtures of SVMs Yining Wang Jun Zhu Tsinghua University July, 2014 Y. Wang and J. Zhu (Tsinghua University) Max-Margin DP-means July, 2014 1 / 25 Outline
More informationEfficient Bayesian Multivariate Surface Regression
Efficient Bayesian Multivariate Surface Regression Feng Li (joint with Mattias Villani) Department of Statistics, Stockholm University October, 211 Outline of the talk 1 Flexible regression models 2 The
More informationBayesian model selection in graphs by using BDgraph package
Bayesian model selection in graphs by using BDgraph package A. Mohammadi and E. Wit March 26, 2013 MOTIVATION Flow cytometry data with 11 proteins from Sachs et al. (2005) RESULT FOR CELL SIGNALING DATA
More informationForward Problems and their Inverse Solutions
Forward Problems and their Inverse Solutions Sarah Zedler 1,2 1 King Abdullah University of Science and Technology 2 University of Texas at Austin February, 2013 Outline 1 Forward Problem Example Weather
More informationComputer Vision Group Prof. Daniel Cremers. 11. Sampling Methods
Prof. Daniel Cremers 11. Sampling Methods Sampling Methods Sampling Methods are widely used in Computer Science as an approximation of a deterministic algorithm to represent uncertainty without a parametric
More information27 : Distributed Monte Carlo Markov Chain. 1 Recap of MCMC and Naive Parallel Gibbs Sampling
10-708: Probabilistic Graphical Models 10-708, Spring 2014 27 : Distributed Monte Carlo Markov Chain Lecturer: Eric P. Xing Scribes: Pengtao Xie, Khoa Luu In this scribe, we are going to review the Parallel
More informationGaussian Mixture Model
Case Study : Document Retrieval MAP EM, Latent Dirichlet Allocation, Gibbs Sampling Machine Learning/Statistics for Big Data CSE599C/STAT59, University of Washington Emily Fox 0 Emily Fox February 5 th,
More informationProbabilistic Graphical Models Lecture 17: Markov chain Monte Carlo
Probabilistic Graphical Models Lecture 17: Markov chain Monte Carlo Andrew Gordon Wilson www.cs.cmu.edu/~andrewgw Carnegie Mellon University March 18, 2015 1 / 45 Resources and Attribution Image credits,
More informationAlgorithms for Variational Learning of Mixture of Gaussians
Algorithms for Variational Learning of Mixture of Gaussians Instructors: Tapani Raiko and Antti Honkela Bayes Group Adaptive Informatics Research Center 28.08.2008 Variational Bayesian Inference Mixture
More informationParameter Estimation. William H. Jefferys University of Texas at Austin Parameter Estimation 7/26/05 1
Parameter Estimation William H. Jefferys University of Texas at Austin bill@bayesrules.net Parameter Estimation 7/26/05 1 Elements of Inference Inference problems contain two indispensable elements: Data
More informationPart IV: Monte Carlo and nonparametric Bayes
Part IV: Monte Carlo and nonparametric Bayes Outline Monte Carlo methods Nonparametric Bayesian models Outline Monte Carlo methods Nonparametric Bayesian models The Monte Carlo principle The expectation
More informationAn Algorithm for Bayesian Variable Selection in High-dimensional Generalized Linear Models
Proceedings 59th ISI World Statistics Congress, 25-30 August 2013, Hong Kong (Session CPS023) p.3938 An Algorithm for Bayesian Variable Selection in High-dimensional Generalized Linear Models Vitara Pungpapong
More informationOutline. Binomial, Multinomial, Normal, Beta, Dirichlet. Posterior mean, MAP, credible interval, posterior distribution
Outline A short review on Bayesian analysis. Binomial, Multinomial, Normal, Beta, Dirichlet Posterior mean, MAP, credible interval, posterior distribution Gibbs sampling Revisit the Gaussian mixture model
More informationMarkov Chain Monte Carlo Lecture 6
Sequential parallel tempering With the development of science and technology, we more and more need to deal with high dimensional systems. For example, we need to align a group of protein or DNA sequences
More informationMonte Carlo methods for sampling-based Stochastic Optimization
Monte Carlo methods for sampling-based Stochastic Optimization Gersende FORT LTCI CNRS & Telecom ParisTech Paris, France Joint works with B. Jourdain, T. Lelièvre, G. Stoltz from ENPC and E. Kuhn from
More informationDavid B. Dahl. Department of Statistics, and Department of Biostatistics & Medical Informatics University of Wisconsin Madison
AN IMPROVED MERGE-SPLIT SAMPLER FOR CONJUGATE DIRICHLET PROCESS MIXTURE MODELS David B. Dahl dbdahl@stat.wisc.edu Department of Statistics, and Department of Biostatistics & Medical Informatics University
More informationECE 5984: Introduction to Machine Learning
ECE 5984: Introduction to Machine Learning Topics: (Finish) Expectation Maximization Principal Component Analysis (PCA) Readings: Barber 15.1-15.4 Dhruv Batra Virginia Tech Administrativia Poster Presentation:
More informationDesigning Kernel Functions Using the Karhunen-Loève Expansion
July 7, 2004. Designing Kernel Functions Using the Karhunen-Loève Expansion 2 1 Fraunhofer FIRST, Germany Tokyo Institute of Technology, Japan 1,2 2 Masashi Sugiyama and Hidemitsu Ogawa Learning with Kernels
More informationMarkov chain Monte Carlo methods in atmospheric remote sensing
1 / 45 Markov chain Monte Carlo methods in atmospheric remote sensing Johanna Tamminen johanna.tamminen@fmi.fi ESA Summer School on Earth System Monitoring and Modeling July 3 Aug 11, 212, Frascati July,
More informationLecture Note 1: Background
ECE5463: Introduction to Robotics Lecture Note 1: Background Prof. Wei Zhang Department of Electrical and Computer Engineering Ohio State University Columbus, Ohio, USA Spring 2018 Lecture 1 (ECE5463 Sp18)
More informationBayesian inference. Fredrik Ronquist and Peter Beerli. October 3, 2007
Bayesian inference Fredrik Ronquist and Peter Beerli October 3, 2007 1 Introduction The last few decades has seen a growing interest in Bayesian inference, an alternative approach to statistical inference.
More informationComputer Vision Group Prof. Daniel Cremers. 14. Sampling Methods
Prof. Daniel Cremers 14. Sampling Methods Sampling Methods Sampling Methods are widely used in Computer Science as an approximation of a deterministic algorithm to represent uncertainty without a parametric
More informationStein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm Qiang Liu and Dilin Wang NIPS 2016 Discussion by Yunchen Pu March 17, 2017 March 17, 2017 1 / 8 Introduction Let x R d
More informationMarkov Chain Monte Carlo Algorithms for Gaussian Processes
Markov Chain Monte Carlo Algorithms for Gaussian Processes Michalis K. Titsias, Neil Lawrence and Magnus Rattray School of Computer Science University of Manchester June 8 Outline Gaussian Processes Sampling
More informationComputer Practical: Metropolis-Hastings-based MCMC
Computer Practical: Metropolis-Hastings-based MCMC Andrea Arnold and Franz Hamilton North Carolina State University July 30, 2016 A. Arnold / F. Hamilton (NCSU) MH-based MCMC July 30, 2016 1 / 19 Markov
More informationThe Bayesian Choice. Christian P. Robert. From Decision-Theoretic Foundations to Computational Implementation. Second Edition.
Christian P. Robert The Bayesian Choice From Decision-Theoretic Foundations to Computational Implementation Second Edition With 23 Illustrations ^Springer" Contents Preface to the Second Edition Preface
More informationDimension-Independent likelihood-informed (DILI) MCMC
Dimension-Independent likelihood-informed (DILI) MCMC Tiangang Cui, Kody Law 2, Youssef Marzouk Massachusetts Institute of Technology 2 Oak Ridge National Laboratory 2 August 25 TC, KL, YM DILI MCMC USC
More informationCS242: Probabilistic Graphical Models Lecture 7B: Markov Chain Monte Carlo & Gibbs Sampling
CS242: Probabilistic Graphical Models Lecture 7B: Markov Chain Monte Carlo & Gibbs Sampling Professor Erik Sudderth Brown University Computer Science October 27, 2016 Some figures and materials courtesy
More informationMarkov Chain Monte Carlo (MCMC)
Markov Chain Monte Carlo (MCMC Dependent Sampling Suppose we wish to sample from a density π, and we can evaluate π as a function but have no means to directly generate a sample. Rejection sampling can
More informationBayesian Analysis. Bayesian Analysis: Bayesian methods concern one s belief about θ. [Current Belief (Posterior)] (Prior Belief) x (Data) Outline
Bayesian Analysis DuBois Bowman, Ph.D. Gordana Derado, M. S. Shuo Chen, M. S. Department of Biostatistics and Bioinformatics Center for Biomedical Imaging Statistics Emory University Outline I. Introduction
More informationDistributed Estimation, Information Loss and Exponential Families. Qiang Liu Department of Computer Science Dartmouth College
Distributed Estimation, Information Loss and Exponential Families Qiang Liu Department of Computer Science Dartmouth College Statistical Learning / Estimation Learning generative models from data Topic
More informationMachine Learning Linear Classification. Prof. Matteo Matteucci
Machine Learning Linear Classification Prof. Matteo Matteucci Recall from the first lecture 2 X R p Regression Y R Continuous Output X R p Y {Ω 0, Ω 1,, Ω K } Classification Discrete Output X R p Y (X)
More informationStat 516, Homework 1
Stat 516, Homework 1 Due date: October 7 1. Consider an urn with n distinct balls numbered 1,..., n. We sample balls from the urn with replacement. Let N be the number of draws until we encounter a ball
More informationNon-convex optimization. Issam Laradji
Non-convex optimization Issam Laradji Strongly Convex Objective function f(x) x Strongly Convex Objective function Assumptions Gradient Lipschitz continuous f(x) Strongly convex x Strongly Convex Objective
More informationStatistical techniques for data analysis in Cosmology
Statistical techniques for data analysis in Cosmology arxiv:0712.3028; arxiv:0911.3105 Numerical recipes (the bible ) Licia Verde ICREA & ICC UB-IEEC http://icc.ub.edu/~liciaverde outline Lecture 1: Introduction
More informationRonald Christensen. University of New Mexico. Albuquerque, New Mexico. Wesley Johnson. University of California, Irvine. Irvine, California
Texts in Statistical Science Bayesian Ideas and Data Analysis An Introduction for Scientists and Statisticians Ronald Christensen University of New Mexico Albuquerque, New Mexico Wesley Johnson University
More informationElastic Handling of Predictor Phase in Functional Regression Models
Elastic Handling of Predictor Phase in Functional Regression Models Kyungmin Ahn Department of Statistics Florida State University Tallahassee, FL k.ahn@stat.fsu.edu J. Derek Tucker Sandia National Laboratories
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 7 Approximate
More informationBayesian Estimation of Input Output Tables for Russia
Bayesian Estimation of Input Output Tables for Russia Oleg Lugovoy (EDF, RANE) Andrey Polbin (RANE) Vladimir Potashnikov (RANE) WIOD Conference April 24, 2012 Groningen Outline Motivation Objectives Bayesian
More informationPhysics 403. Segev BenZvi. Numerical Methods, Maximum Likelihood, and Least Squares. Department of Physics and Astronomy University of Rochester
Physics 403 Numerical Methods, Maximum Likelihood, and Least Squares Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Review of Last Class Quadratic Approximation
More informationDisk Diffusion Breakpoint Determination Using a Bayesian Nonparametric Variation of the Errors-in-Variables Model
1 / 23 Disk Diffusion Breakpoint Determination Using a Bayesian Nonparametric Variation of the Errors-in-Variables Model Glen DePalma gdepalma@purdue.edu Bruce A. Craig bacraig@purdue.edu Eastern North
More informationPATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA Contents in latter part Linear Dynamical Systems What is different from HMM? Kalman filter Its strength and limitation Particle Filter
More informationDefault Priors and Effcient Posterior Computation in Bayesian
Default Priors and Effcient Posterior Computation in Bayesian Factor Analysis January 16, 2010 Presented by Eric Wang, Duke University Background and Motivation A Brief Review of Parameter Expansion Literature
More informationIntroduction to Bayesian methods in inverse problems
Introduction to Bayesian methods in inverse problems Ville Kolehmainen 1 1 Department of Applied Physics, University of Eastern Finland, Kuopio, Finland March 4 2013 Manchester, UK. Contents Introduction
More informationNon-Parametric Bayes
Non-Parametric Bayes Mark Schmidt UBC Machine Learning Reading Group January 2016 Current Hot Topics in Machine Learning Bayesian learning includes: Gaussian processes. Approximate inference. Bayesian
More informationCSC 2541: Bayesian Methods for Machine Learning
CSC 2541: Bayesian Methods for Machine Learning Radford M. Neal, University of Toronto, 2011 Lecture 4 Problem: Density Estimation We have observed data, y 1,..., y n, drawn independently from some unknown
More informationConsistent Downscaling of Seismic Inversions to Cornerpoint Flow Models SPE
Consistent Downscaling of Seismic Inversions to Cornerpoint Flow Models SPE 103268 Subhash Kalla LSU Christopher D. White LSU James S. Gunning CSIRO Michael E. Glinsky BHP-Billiton Contents Method overview
More informationAdvanced Statistical Computing
Advanced Statistical Computing Fall 206 Steve Qin Outline Collapsing, predictive updating Sequential Monte Carlo 2 Collapsing and grouping Want to sample from = Regular Gibbs sampler: Sample t+ from π
More informationBlind Equalization via Particle Filtering
Blind Equalization via Particle Filtering Yuki Yoshida, Kazunori Hayashi, Hideaki Sakai Department of System Science, Graduate School of Informatics, Kyoto University Historical Remarks A sequential Monte
More informationSpatial Normalized Gamma Process
Spatial Normalized Gamma Process Vinayak Rao Yee Whye Teh Presented at NIPS 2009 Discussion and Slides by Eric Wang June 23, 2010 Outline Introduction Motivation The Gamma Process Spatial Normalized Gamma
More informationKernel adaptive Sequential Monte Carlo
Kernel adaptive Sequential Monte Carlo Ingmar Schuster (Paris Dauphine) Heiko Strathmann (University College London) Brooks Paige (Oxford) Dino Sejdinovic (Oxford) December 7, 2015 1 / 36 Section 1 Outline
More informationPrinciples of Bayesian Inference
Principles of Bayesian Inference Sudipto Banerjee University of Minnesota July 20th, 2008 1 Bayesian Principles Classical statistics: model parameters are fixed and unknown. A Bayesian thinks of parameters
More informationLecture 16 Deep Neural Generative Models
Lecture 16 Deep Neural Generative Models CMSC 35246: Deep Learning Shubhendu Trivedi & Risi Kondor University of Chicago May 22, 2017 Approach so far: We have considered simple models and then constructed
More informationConsistent high-dimensional Bayesian variable selection via penalized credible regions
Consistent high-dimensional Bayesian variable selection via penalized credible regions Howard Bondell bondell@stat.ncsu.edu Joint work with Brian Reich Howard Bondell p. 1 Outline High-Dimensional Variable
More informationComputer Vision Group Prof. Daniel Cremers. 10a. Markov Chain Monte Carlo
Group Prof. Daniel Cremers 10a. Markov Chain Monte Carlo Markov Chain Monte Carlo In high-dimensional spaces, rejection sampling and importance sampling are very inefficient An alternative is Markov Chain
More informationDeep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, Spis treści
Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, 2017 Spis treści Website Acknowledgments Notation xiii xv xix 1 Introduction 1 1.1 Who Should Read This Book?
More informationPatterns of Scalable Bayesian Inference Background (Session 1)
Patterns of Scalable Bayesian Inference Background (Session 1) Jerónimo Arenas-García Universidad Carlos III de Madrid jeronimo.arenas@gmail.com June 14, 2017 1 / 15 Motivation. Bayesian Learning principles
More informationBagging During Markov Chain Monte Carlo for Smoother Predictions
Bagging During Markov Chain Monte Carlo for Smoother Predictions Herbert K. H. Lee University of California, Santa Cruz Abstract: Making good predictions from noisy data is a challenging problem. Methods
More informationA Process over all Stationary Covariance Kernels
A Process over all Stationary Covariance Kernels Andrew Gordon Wilson June 9, 0 Abstract I define a process over all stationary covariance kernels. I show how one might be able to perform inference that
More informationRiemannian Stein Variational Gradient Descent for Bayesian Inference
Riemannian Stein Variational Gradient Descent for Bayesian Inference Chang Liu, Jun Zhu 1 Dept. of Comp. Sci. & Tech., TNList Lab; Center for Bio-Inspired Computing Research State Key Lab for Intell. Tech.
More informationGentle Introduction to Infinite Gaussian Mixture Modeling
Gentle Introduction to Infinite Gaussian Mixture Modeling with an application in neuroscience By Frank Wood Rasmussen, NIPS 1999 Neuroscience Application: Spike Sorting Important in neuroscience and for
More informationMCMC Sampling for Bayesian Inference using L1-type Priors
MÜNSTER MCMC Sampling for Bayesian Inference using L1-type Priors (what I do whenever the ill-posedness of EEG/MEG is just not frustrating enough!) AG Imaging Seminar Felix Lucka 26.06.2012 , MÜNSTER Sampling
More informationAfternoon Meeting on Bayesian Computation 2018 University of Reading
Gabriele Abbati 1, Alessra Tosi 2, Seth Flaxman 3, Michael A Osborne 1 1 University of Oxford, 2 Mind Foundry Ltd, 3 Imperial College London Afternoon Meeting on Bayesian Computation 2018 University of
More informationMetamorphic Geodesic Regression
Metamorphic Geodesic Regression Yi Hong, Sarang Joshi 3, Mar Sanchez 4, Martin Styner, and Marc Niethammer,2 UNC-Chapel Hill/ 2 BRIC, 3 University of Utah, 4 Emory University Abstract. We propose a metamorphic
More informationCPSC 540: Machine Learning
CPSC 540: Machine Learning More Approximate Inference Mark Schmidt University of British Columbia Winter 2018 Last Time: Approximate Inference We ve been discussing graphical models for density estimation,
More informationBayesian model selection: methodology, computation and applications
Bayesian model selection: methodology, computation and applications David Nott Department of Statistics and Applied Probability National University of Singapore Statistical Genomics Summer School Program
More informationCan we do statistical inference in a non-asymptotic way? 1
Can we do statistical inference in a non-asymptotic way? 1 Guang Cheng 2 Statistics@Purdue www.science.purdue.edu/bigdata/ ONR Review Meeting@Duke Oct 11, 2017 1 Acknowledge NSF, ONR and Simons Foundation.
More informationProbability for Statistics and Machine Learning
~Springer Anirban DasGupta Probability for Statistics and Machine Learning Fundamentals and Advanced Topics Contents Suggested Courses with Diffe~ent Themes........................... xix 1 Review of Univariate
More informationCSC 446 Notes: Lecture 13
CSC 446 Notes: Lecture 3 The Problem We have already studied how to calculate the probability of a variable or variables using the message passing method. However, there are some times when the structure
More information