Mixed effect model for the spatiotemporal analysis of longitudinal manifold value data

Size: px
Start display at page:

Download "Mixed effect model for the spatiotemporal analysis of longitudinal manifold value data"

Transcription

1 Mixed effect model for the spatiotemporal analysis of longitudinal manifold value data Stéphanie Allassonnière with J.B. Schiratti, O. Colliot and S. Durrleman Université Paris Descartes & Ecole Polytechnique

2 Computational Anatomy Represent and analyse geometrical elements upon which deformations can act Describe the observed objects as geometrical variations of one or several representative elements Quantify this variability inside a population Deformable template model from Grenander How does the deformation act? What is a representative element? How to quantify the geometrical variability? 2

3 Computational Anatomy Important issues in atlas estimation: Register any new data in the «coordinates» of the reference shape: Transport the available information from the representative element «Registration» penalised as a function of its «normality» Quantify anatomical structure variability in different sub-groups Targetted applications: Pathology effects Classification of new patients Early diagnostic 3

4 Computational Anatomy One solution: Quantify the distance between observations using deformations Provide a statistical model to approximate the generation of the observed population from the atlas Propose a statistical learning algorithm Optimise the numerical estimation 4

5 Bayesian Mixed Effect model First model : One observation per subject Image or shape (viewed as currents) Deformations either linearized or diffeomorphic Homogeneous or heterogeneous populations (mixture models) 5

6 Bayesian Mixed Effect model 6

7 Bayesian Mixed Effect model 7

8 Bayesian Mixed Effect model First model : One observation per subject Image or shape (viewed as currents) Deformations either linearized or diffeomorphic Homogeneous or heterogeneous populations (mixture models) Limitations One observation per subject Corresponding acquistion time 8

9 Longitudinal Data Analysis Longitudinal model : Several observation per subject Image, shape, etc Atlas = representative trajectory and population variability 9

10 Longitudinal Data Analysis Time (age) subject #1 subject #2 subject #3 How to learn representative trajectories of data changes from longitudinal data? Linear vector-valued mixed-effects data models Temporal marker of progression (e.g. time since drug injection, seeding, birth, etc..) Regression (e.g. compare measurements at same time-point) [Laird&Ware 82, Diggle et al., Fitzmaurice et al.] No temporal marker of progression (e.g. in aging, neurodegenerative diseases, etc..) Learning spatiotemporal distribution of trajectories - Find temporal correspondences - Compare data at corresponding stages of progression Needs to disentangle manifold-valued differences in: data - (normalized Measurements data, positive - Dynamics matrices, of shapes, measurement etc;;) changes 10

11 Spatiotemporal Statistical Model Statistical model inclinding: a representative trajectory of data changes spatiotemporal variations in: measurement values pace of measurement changes Orthogonality condition ensures identifiability (unique space/ time decomposition) Random effects: Acceleration factor Time-shift Time is not a covariate but a random variable Space-shift Fixed effects: and 11 [Schiratti et al. IPMI 15, NIPS 15]

12 Spatiotemporal Statistical Model Submanifold value observations Parallel curve Representative trajectory Linear time reparametrization Hidden random variables: Acceleration factor Time shift Space shift Parameters: Mean trajectory parametrization and prior parameter 12

13 Spatiotemporal Statistical Model Comparison with previous work: Interest: Parallel transport keep invariant the structure of the distribution, but updated it in time 13

14 Spatiotemporal Statistical Model The straight line model y t 0 time t 0 time x x Time at which measurement of the i th subject reaches Measurement of the i th subject at time t 0 Schiratti et al. (2015) Laird & Ware (1982) 14

15 Spatiotemporal Statistical Model The logistic curve model Geodesic are logistic curves It is not equivalent to a linear model on the logit of the observations (i.e. the Riemannian log at p 0 = 0.5), since p 0 is estimated If we fix p 0 = 0.5 in our model! end up with our previous linear case (different from Laird&Ware) 15

16 Spatiotemporal Statistical Model The propagation model Geodesics are logistic curves in each coordinate Parametric family of geodesics seen as a model of propagation of an effect The parallel curve in the direction of the space-shift v i writes à The parallel changes the relative timing of the effect onset across coordinates 16

17 Parameter Estimation Maximum Likelihood: EM: Distribution from the curved exponential family 17

18 Parameter Estimation: stochastic algorithm SA-EM: replaces integration by one simulation of the hidden variable: sample from, and a stochastic approximation of the sufficient statistics Maximization step (unchanged) MCMC-SAEM: replaces sampling by a single Markov Chain step For each coordinate p (Gibbs sampler) sample Set with probability otherwise [Delyon, Lavielle, 18 Moulines. 99] [Allassonnière et

19 Parameter Estimation: stochastic algorithm Theoretical properties of the sampler: Under mild conditions: Drift property Small set Geometric ergodicity uniformly on any compact set of the parameters Theoretical properties of the estimation algorithm: a.s. convergence towards the MAP estimator Normal asymptotic behaviour: speed Normal asymptotoc behaviour with optimal speed with averaging sequences 19

20 Neuropsychological tests ADAS-Gog from ADNI 248 subjects who converted from MCI to AD 6 time-points per subjects on average (min 3, max 11) Data points with propagation logistic model Model of Alzheimer s disease progression The average trajectory of data changes 20 [Schiratti et al. IPMI 15,

21 Model of Alzheimer s disease progression Distinguish fast vs. slow progressers Time-shift Distinguish early vs. late onset individuals [Schiratti et al. IPMI 15,

22 Model of Alzheimer s disease progression Decomposition vector A 1 Decomposition vector A 2 Variability in the relative timing and ordering of the events [Schiratti et al. IPMI 15,

23 Model of Alzheimer s disease progression [Schiratti et al. IPMI 15,

24 Model of Alzheimer s disease progression [Schiratti et al. IPMI 15,

25 Model of Alzheimer s disease progression 25

26 Model of diffusion tensors Geodesic in the Riemannian manifold of positive definite matrices Parallel transport the tensors Reprearamtrize in time Sample this curse 26

27 Model of diffusion tensors Synthetic data not generated from the model but imitating a non smooth evolution 100 subjects 5 time points in average First eigenvalue Second eigenvalue Third eigenvalue

28 Model of diffusion tensors Fitting the model to a new patient First eigenvalue Second eigenvalue Third eigenvalue

29 Comparison AD vs Controls 29

30 Comparison MCI vs Controls 30

31 Computational comparisons Comparison of: MCMC-SAEM STAN MONOLIX Number of iterations: MCMC-SAEM: (6s / iterations) STAN: (25min / iterations) MONOLIX: (3,5 min / iterations) 31

32 Computational comparisons Comparison of: MCMC-SAEM STAN MONOLIX True values MCMC-SAEM STAN Monolix 32

33 Conclusion Generic statistical model to learn spatiotemporal distribution of trajectories on manifolds: Calibrated on longitudinal data sets using MCMC-SAEM Automatically finds temporal correspondences among similar events that may happen at different age/time Estimates the variability of the data at the corresponding events It allows us to position disease progression within the life and history of the patient Future work: Derive instances of the model for more complex manifoldvalued data (e.g. spatially distributed data, shape data, etc..) 33

34 Thank you! 34

Generative Models and Stochastic Algorithms for Population Average Estimation and Image Analysis

Generative Models and Stochastic Algorithms for Population Average Estimation and Image Analysis Generative Models and Stochastic Algorithms for Population Average Estimation and Image Analysis Stéphanie Allassonnière CIS, JHU July, 15th 28 Context : Computational Anatomy Context and motivations :

More information

Multivariate Statistical Analysis of Deformation Momenta Relating Anatomical Shape to Neuropsychological Measures

Multivariate Statistical Analysis of Deformation Momenta Relating Anatomical Shape to Neuropsychological Measures Multivariate Statistical Analysis of Deformation Momenta Relating Anatomical Shape to Neuropsychological Measures Nikhil Singh, Tom Fletcher, Sam Preston, Linh Ha, J. Stephen Marron, Michael Wiener, and

More information

arxiv: v1 [cs.cv] 23 Nov 2017

arxiv: v1 [cs.cv] 23 Nov 2017 Parallel transport in shape analysis: a scalable numerical scheme arxiv:1711.08725v1 [cs.cv] 23 Nov 2017 Maxime Louis 12, Alexandre Bône 12, Benjamin Charlier 23, Stanley Durrleman 12, and the Alzheimer

More information

Analysis of Longitudinal Shape Variability via Subject Specific Growth Modeling

Analysis of Longitudinal Shape Variability via Subject Specific Growth Modeling Analysis of Longitudinal Shape Variability via Subject Specific Growth Modeling James Fishbaugh 1, Marcel Prastawa 1, Stanley Durrleman 2, Joseph Piven 3 for the IBIS Network, and Guido Gerig 1 1 Scientific

More information

Perturbed Proximal Gradient Algorithm

Perturbed Proximal Gradient Algorithm Perturbed Proximal Gradient Algorithm Gersende FORT LTCI, CNRS, Telecom ParisTech Université Paris-Saclay, 75013, Paris, France Large-scale inverse problems and optimization Applications to image processing

More information

Pattern Recognition and Machine Learning

Pattern Recognition and Machine Learning Christopher M. Bishop Pattern Recognition and Machine Learning ÖSpri inger Contents Preface Mathematical notation Contents vii xi xiii 1 Introduction 1 1.1 Example: Polynomial Curve Fitting 4 1.2 Probability

More information

Multivariate models of inter-subject anatomical variability

Multivariate models of inter-subject anatomical variability Multivariate models of inter-subject anatomical variability Wellcome Trust Centre for Neuroimaging, UCL Institute of Neurology, 12 Queen Square, London WC1N 3BG, UK. Prediction Binary Classification Curse

More information

Introduction to Machine Learning CMU-10701

Introduction to Machine Learning CMU-10701 Introduction to Machine Learning CMU-10701 Markov Chain Monte Carlo Methods Barnabás Póczos & Aarti Singh Contents Markov Chain Monte Carlo Methods Goal & Motivation Sampling Rejection Importance Markov

More information

Introduction to Machine Learning

Introduction to Machine Learning Introduction to Machine Learning Brown University CSCI 1950-F, Spring 2012 Prof. Erik Sudderth Lecture 25: Markov Chain Monte Carlo (MCMC) Course Review and Advanced Topics Many figures courtesy Kevin

More information

CPSC 540: Machine Learning

CPSC 540: Machine Learning CPSC 540: Machine Learning MCMC and Non-Parametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is

More information

Bayesian Nonparametrics

Bayesian Nonparametrics Bayesian Nonparametrics Peter Orbanz Columbia University PARAMETERS AND PATTERNS Parameters P(X θ) = Probability[data pattern] 3 2 1 0 1 2 3 5 0 5 Inference idea data = underlying pattern + independent

More information

Pattern Recognition and Machine Learning. Bishop Chapter 11: Sampling Methods

Pattern Recognition and Machine Learning. Bishop Chapter 11: Sampling Methods Pattern Recognition and Machine Learning Chapter 11: Sampling Methods Elise Arnaud Jakob Verbeek May 22, 2008 Outline of the chapter 11.1 Basic Sampling Algorithms 11.2 Markov Chain Monte Carlo 11.3 Gibbs

More information

Stéphanie Allassonniere, Estelle Kuhn. To cite this version: HAL Id: hal https://hal.archives-ouvertes.fr/hal v4

Stéphanie Allassonniere, Estelle Kuhn. To cite this version: HAL Id: hal https://hal.archives-ouvertes.fr/hal v4 Convergent Stochastic Expectation Maximization algorithm with efficient sampling in high dimension. Application to deformable template model estimation Stéphanie Allassonniere, Estelle Kuhn To cite this

More information

Learning distributions of shape trajectories from longitudinal datasets: a hierarchical model on a manifold of diffeomorphisms

Learning distributions of shape trajectories from longitudinal datasets: a hierarchical model on a manifold of diffeomorphisms Learning distributions of shape trajectories from longitudinal datasets: a hierarchical model on a manifold of diffeomorphisms Alexandre Bône Olivier Colliot Stanley Durrleman The Alzheimer s Disease Neuroimaging

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate

More information

Fast Template-based Shape Analysis using Diffeomorphic Iterative Centroid

Fast Template-based Shape Analysis using Diffeomorphic Iterative Centroid Fast Template-based Shape Analysis using Diffeomorphic Iterative Centroid Claire Cury, Joan Alexis Glaunès, Marie Chupin, Olivier Colliot To cite this version: Claire Cury, Joan Alexis Glaunès, Marie Chupin,

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 7 Approximate

More information

Geodesic Refinement Using James-Stein Estimators

Geodesic Refinement Using James-Stein Estimators Geodesic Refinement Using James-Stein Estimators Greg M. Fleishman 1,2, P. Thomas Fletcher 3, Boris A. Gutman 2, Gautam Prasad 2, Yingnian Wu 4, and Paul M. Thompson 2 1 UC Los Angeles, Department of Bioengineering

More information

Diffeomorphic Shape Trajectories for Improved Longitudinal Segmentation and Statistics

Diffeomorphic Shape Trajectories for Improved Longitudinal Segmentation and Statistics Diffeomorphic Shape Trajectories for Improved Longitudinal Segmentation and Statistics Prasanna Muralidharan 1, James Fishbaugh 1, Hans J. Johnson 2, Stanley Durrleman 3, Jane S. Paulsen 2, Guido Gerig

More information

Missing Data Issues in the Studies of Neurodegenerative Disorders: the Methodology

Missing Data Issues in the Studies of Neurodegenerative Disorders: the Methodology Missing Data Issues in the Studies of Neurodegenerative Disorders: the Methodology Sheng Luo, PhD Associate Professor Department of Biostatistics & Bioinformatics Duke University Medical Center sheng.luo@duke.edu

More information

STA 4273H: Sta-s-cal Machine Learning

STA 4273H: Sta-s-cal Machine Learning STA 4273H: Sta-s-cal Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 2 In our

More information

dans les modèles à vraisemblance non explicite par des algorithmes gradient-proximaux perturbés

dans les modèles à vraisemblance non explicite par des algorithmes gradient-proximaux perturbés Inférence pénalisée dans les modèles à vraisemblance non explicite par des algorithmes gradient-proximaux perturbés Gersende Fort Institut de Mathématiques de Toulouse, CNRS and Univ. Paul Sabatier Toulouse,

More information

Stat 5101 Lecture Notes

Stat 5101 Lecture Notes Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random

More information

VCMC: Variational Consensus Monte Carlo

VCMC: Variational Consensus Monte Carlo VCMC: Variational Consensus Monte Carlo Maxim Rabinovich, Elaine Angelino, Michael I. Jordan Berkeley Vision and Learning Center September 22, 2015 probabilistic models! sky fog bridge water grass object

More information

Extension of the SAEM algorithm for nonlinear mixed models with 2 levels of random effects Panhard Xavière

Extension of the SAEM algorithm for nonlinear mixed models with 2 levels of random effects Panhard Xavière Extension of the SAEM algorithm for nonlinear mixed models with 2 levels of random effects Panhard Xavière * Modèles et mé thodes de l'évaluation thérapeutique des maladies chroniques INSERM : U738, Universit

More information

Math 1553, Introduction to Linear Algebra

Math 1553, Introduction to Linear Algebra Learning goals articulate what students are expected to be able to do in a course that can be measured. This course has course-level learning goals that pertain to the entire course, and section-level

More information

Bayesian Registration of Functions with a Gaussian Process Prior

Bayesian Registration of Functions with a Gaussian Process Prior Bayesian Registration of Functions with a Gaussian Process Prior RELATED PROBLEMS Functions Curves Images Surfaces FUNCTIONAL DATA Electrocardiogram Signals (Heart Disease) Gait Pressure Measurements (Parkinson

More information

Anatomical Regularization on Statistical Manifolds for the Classification of Patients with Alzheimer s Disease

Anatomical Regularization on Statistical Manifolds for the Classification of Patients with Alzheimer s Disease Anatomical Regularization on Statistical Manifolds for the Classification of Patients with Alzheimer s Disease Rémi Cuingnet 1,2, Joan Alexis Glaunès 1,3, Marie Chupin 1, Habib Benali 2, and Olivier Colliot

More information

Disk Diffusion Breakpoint Determination Using a Bayesian Nonparametric Variation of the Errors-in-Variables Model

Disk Diffusion Breakpoint Determination Using a Bayesian Nonparametric Variation of the Errors-in-Variables Model 1 / 23 Disk Diffusion Breakpoint Determination Using a Bayesian Nonparametric Variation of the Errors-in-Variables Model Glen DePalma gdepalma@purdue.edu Bruce A. Craig bacraig@purdue.edu Eastern North

More information

Convex Optimization CMU-10725

Convex Optimization CMU-10725 Convex Optimization CMU-10725 Simulated Annealing Barnabás Póczos & Ryan Tibshirani Andrey Markov Markov Chains 2 Markov Chains Markov chain: Homogen Markov chain: 3 Markov Chains Assume that the state

More information

Fast Geodesic Regression for Population-Based Image Analysis

Fast Geodesic Regression for Population-Based Image Analysis Fast Geodesic Regression for Population-Based Image Analysis Yi Hong 1, Polina Golland 2, and Miaomiao Zhang 2 1 Computer Science Department, University of Georgia 2 Computer Science and Artificial Intelligence

More information

Identification of Gaussian Process State-Space Models with Particle Stochastic Approximation EM

Identification of Gaussian Process State-Space Models with Particle Stochastic Approximation EM Identification of Gaussian Process State-Space Models with Particle Stochastic Approximation EM Roger Frigola Fredrik Lindsten Thomas B. Schön, Carl E. Rasmussen Dept. of Engineering, University of Cambridge,

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 3 Linear

More information

An efficient stochastic approximation EM algorithm using conditional particle filters

An efficient stochastic approximation EM algorithm using conditional particle filters An efficient stochastic approximation EM algorithm using conditional particle filters Fredrik Lindsten Linköping University Post Print N.B.: When citing this work, cite the original article. Original Publication:

More information

Differential Geometry, Lie Groups, and Symmetric Spaces

Differential Geometry, Lie Groups, and Symmetric Spaces Differential Geometry, Lie Groups, and Symmetric Spaces Sigurdur Helgason Graduate Studies in Mathematics Volume 34 nsffvjl American Mathematical Society l Providence, Rhode Island PREFACE PREFACE TO THE

More information

Statistical approach for dictionary learning

Statistical approach for dictionary learning Statistical approach for dictionary learning Tieyong ZENG Joint work with Alain Trouvé Page 1 Introduction Redundant dictionary Coding, denoising, compression. Existing algorithms to generate dictionary

More information

Identification of Gaussian Process State-Space Models with Particle Stochastic Approximation EM

Identification of Gaussian Process State-Space Models with Particle Stochastic Approximation EM Preprints of the 9th World Congress The International Federation of Automatic Control Identification of Gaussian Process State-Space Models with Particle Stochastic Approximation EM Roger Frigola Fredrik

More information

Mixture modelling of recurrent event times with long-term survivors: Analysis of Hutterite birth intervals. John W. Mac McDonald & Alessandro Rosina

Mixture modelling of recurrent event times with long-term survivors: Analysis of Hutterite birth intervals. John W. Mac McDonald & Alessandro Rosina Mixture modelling of recurrent event times with long-term survivors: Analysis of Hutterite birth intervals John W. Mac McDonald & Alessandro Rosina Quantitative Methods in the Social Sciences Seminar -

More information

Computer Intensive Methods in Mathematical Statistics

Computer Intensive Methods in Mathematical Statistics Computer Intensive Methods in Mathematical Statistics Department of mathematics johawes@kth.se Lecture 16 Advanced topics in computational statistics 18 May 2017 Computer Intensive Methods (1) Plan of

More information

Multilevel Statistical Models: 3 rd edition, 2003 Contents

Multilevel Statistical Models: 3 rd edition, 2003 Contents Multilevel Statistical Models: 3 rd edition, 2003 Contents Preface Acknowledgements Notation Two and three level models. A general classification notation and diagram Glossary Chapter 1 An introduction

More information

Latent Variable Models for Binary Data. Suppose that for a given vector of explanatory variables x, the latent

Latent Variable Models for Binary Data. Suppose that for a given vector of explanatory variables x, the latent Latent Variable Models for Binary Data Suppose that for a given vector of explanatory variables x, the latent variable, U, has a continuous cumulative distribution function F (u; x) and that the binary

More information

HMM part 1. Dr Philip Jackson

HMM part 1. Dr Philip Jackson Centre for Vision Speech & Signal Processing University of Surrey, Guildford GU2 7XH. HMM part 1 Dr Philip Jackson Probability fundamentals Markov models State topology diagrams Hidden Markov models -

More information

(5) Multi-parameter models - Gibbs sampling. ST440/540: Applied Bayesian Analysis

(5) Multi-parameter models - Gibbs sampling. ST440/540: Applied Bayesian Analysis Summarizing a posterior Given the data and prior the posterior is determined Summarizing the posterior gives parameter estimates, intervals, and hypothesis tests Most of these computations are integrals

More information

Online but Accurate Inference for Latent Variable Models with Local Gibbs Sampling

Online but Accurate Inference for Latent Variable Models with Local Gibbs Sampling Online but Accurate Inference for Latent Variable Models with Local Gibbs Sampling Christophe Dupuy INRIA - Technicolor christophe.dupuy@inria.fr Francis Bach INRIA - ENS francis.bach@inria.fr Abstract

More information

Beyond stochastic gradient descent for large-scale machine learning

Beyond stochastic gradient descent for large-scale machine learning Beyond stochastic gradient descent for large-scale machine learning Francis Bach INRIA - Ecole Normale Supérieure, Paris, France Joint work with Eric Moulines - October 2014 Big data revolution? A new

More information

Parameter Estimation in the Spatio-Temporal Mixed Effects Model Analysis of Massive Spatio-Temporal Data Sets

Parameter Estimation in the Spatio-Temporal Mixed Effects Model Analysis of Massive Spatio-Temporal Data Sets Parameter Estimation in the Spatio-Temporal Mixed Effects Model Analysis of Massive Spatio-Temporal Data Sets Matthias Katzfuß Advisor: Dr. Noel Cressie Department of Statistics The Ohio State University

More information

Non-convex optimization. Issam Laradji

Non-convex optimization. Issam Laradji Non-convex optimization Issam Laradji Strongly Convex Objective function f(x) x Strongly Convex Objective function Assumptions Gradient Lipschitz continuous f(x) Strongly convex x Strongly Convex Objective

More information

Bayesian Methods for Machine Learning

Bayesian Methods for Machine Learning Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),

More information

Weak convergence of Markov chain Monte Carlo II

Weak convergence of Markov chain Monte Carlo II Weak convergence of Markov chain Monte Carlo II KAMATANI, Kengo Mar 2011 at Le Mans Background Markov chain Monte Carlo (MCMC) method is widely used in Statistical Science. It is easy to use, but difficult

More information

Morphometrics with SPM12

Morphometrics with SPM12 Morphometrics with SPM12 John Ashburner Wellcome Trust Centre for Neuroimaging, 12 Queen Square, London, UK. What kind of differences are we looking for? Usually, we try to localise regions of difference.

More information

Introduction to Graphical Models

Introduction to Graphical Models Introduction to Graphical Models The 15 th Winter School of Statistical Physics POSCO International Center & POSTECH, Pohang 2018. 1. 9 (Tue.) Yung-Kyun Noh GENERALIZATION FOR PREDICTION 2 Probabilistic

More information

The Particle Filter. PD Dr. Rudolph Triebel Computer Vision Group. Machine Learning for Computer Vision

The Particle Filter. PD Dr. Rudolph Triebel Computer Vision Group. Machine Learning for Computer Vision The Particle Filter Non-parametric implementation of Bayes filter Represents the belief (posterior) random state samples. by a set of This representation is approximate. Can represent distributions that

More information

Markov Chains. Arnoldo Frigessi Bernd Heidergott November 4, 2015

Markov Chains. Arnoldo Frigessi Bernd Heidergott November 4, 2015 Markov Chains Arnoldo Frigessi Bernd Heidergott November 4, 2015 1 Introduction Markov chains are stochastic models which play an important role in many applications in areas as diverse as biology, finance,

More information

L11: Pattern recognition principles

L11: Pattern recognition principles L11: Pattern recognition principles Bayesian decision theory Statistical classifiers Dimensionality reduction Clustering This lecture is partly based on [Huang, Acero and Hon, 2001, ch. 4] Introduction

More information

Empirical Risk Minimization

Empirical Risk Minimization Empirical Risk Minimization Fabrice Rossi SAMM Université Paris 1 Panthéon Sorbonne 2018 Outline Introduction PAC learning ERM in practice 2 General setting Data X the input space and Y the output space

More information

Learning the Linear Dynamical System with ASOS ( Approximated Second-Order Statistics )

Learning the Linear Dynamical System with ASOS ( Approximated Second-Order Statistics ) Learning the Linear Dynamical System with ASOS ( Approximated Second-Order Statistics ) James Martens University of Toronto June 24, 2010 Computer Science UNIVERSITY OF TORONTO James Martens (U of T) Learning

More information

Gentle Introduction to Infinite Gaussian Mixture Modeling

Gentle Introduction to Infinite Gaussian Mixture Modeling Gentle Introduction to Infinite Gaussian Mixture Modeling with an application in neuroscience By Frank Wood Rasmussen, NIPS 1999 Neuroscience Application: Spike Sorting Important in neuroscience and for

More information

Sparse Stochastic Inference for Latent Dirichlet Allocation

Sparse Stochastic Inference for Latent Dirichlet Allocation Sparse Stochastic Inference for Latent Dirichlet Allocation David Mimno 1, Matthew D. Hoffman 2, David M. Blei 1 1 Dept. of Computer Science, Princeton U. 2 Dept. of Statistics, Columbia U. Presentation

More information

CSC 2541: Bayesian Methods for Machine Learning

CSC 2541: Bayesian Methods for Machine Learning CSC 2541: Bayesian Methods for Machine Learning Radford M. Neal, University of Toronto, 2011 Lecture 3 More Markov Chain Monte Carlo Methods The Metropolis algorithm isn t the only way to do MCMC. We ll

More information

Using the package hypergsplines: some examples.

Using the package hypergsplines: some examples. Using the package hypergsplines: some examples. Daniel Sabanés Bové 21st November 2013 This short vignette shall introduce into the usage of the package hypergsplines. For more information on the methodology,

More information

Probabilistic Time Series Classification

Probabilistic Time Series Classification Probabilistic Time Series Classification Y. Cem Sübakan Boğaziçi University 25.06.2013 Y. Cem Sübakan (Boğaziçi University) M.Sc. Thesis Defense 25.06.2013 1 / 54 Problem Statement The goal is to assign

More information

Bayesian Networks Inference with Probabilistic Graphical Models

Bayesian Networks Inference with Probabilistic Graphical Models 4190.408 2016-Spring Bayesian Networks Inference with Probabilistic Graphical Models Byoung-Tak Zhang intelligence Lab Seoul National University 4190.408 Artificial (2016-Spring) 1 Machine Learning? Learning

More information

Symmetric Image Normalization in the Diffeomorphic Space

Symmetric Image Normalization in the Diffeomorphic Space Symmetric Image Normalization in the Diffeomorphic Space Brian Avants, Charles Epstein, James Gee Penn Image Computing & Science Lab Departments of Radiology and Mathematics University of Pennsylvania

More information

Bayesian Networks: Construction, Inference, Learning and Causal Interpretation. Volker Tresp Summer 2016

Bayesian Networks: Construction, Inference, Learning and Causal Interpretation. Volker Tresp Summer 2016 Bayesian Networks: Construction, Inference, Learning and Causal Interpretation Volker Tresp Summer 2016 1 Introduction So far we were mostly concerned with supervised learning: we predicted one or several

More information

Heeyoul (Henry) Choi. Dept. of Computer Science Texas A&M University

Heeyoul (Henry) Choi. Dept. of Computer Science Texas A&M University Heeyoul (Henry) Choi Dept. of Computer Science Texas A&M University hchoi@cs.tamu.edu Introduction Speaker Adaptation Eigenvoice Comparison with others MAP, MLLR, EMAP, RMP, CAT, RSW Experiments Future

More information

CONVERGENCE OF A STOCHASTIC APPROXIMATION VERSION OF THE EM ALGORITHM

CONVERGENCE OF A STOCHASTIC APPROXIMATION VERSION OF THE EM ALGORITHM The Annals of Statistics 1999, Vol. 27, No. 1, 94 128 CONVERGENCE OF A STOCHASTIC APPROXIMATION VERSION OF THE EM ALGORITHM By Bernard Delyon, Marc Lavielle and Eric Moulines IRISA/INRIA, Université Paris

More information

Stochastic Dynamic Causal Modelling for resting-state fmri

Stochastic Dynamic Causal Modelling for resting-state fmri Stochastic Dynamic Causal Modelling for resting-state fmri Ged Ridgway, FIL, Wellcome Trust Centre for Neuroimaging, UCL Institute of Neurology, London Overview Connectivity in the brain Introduction to

More information

Algorithms for Variational Learning of Mixture of Gaussians

Algorithms for Variational Learning of Mixture of Gaussians Algorithms for Variational Learning of Mixture of Gaussians Instructors: Tapani Raiko and Antti Honkela Bayes Group Adaptive Informatics Research Center 28.08.2008 Variational Bayesian Inference Mixture

More information

T xi (M) n i. x i b 1xi. x j. h j. x p j b 2x i

T xi (M) n i. x i b 1xi. x j. h j. x p j b 2x i Gaussian Random Fields for Statistical Characterization of Brain Submanifolds. Sarang Joshi Department of Radiation Oncology and Biomedical Enginereing University of North Carolina at Chapel Hill Sarang

More information

The Lady Tasting Tea. How to deal with multiple testing. Need to explore many models. More Predictive Modeling

The Lady Tasting Tea. How to deal with multiple testing. Need to explore many models. More Predictive Modeling The Lady Tasting Tea More Predictive Modeling R. A. Fisher & the Lady B. Muriel Bristol claimed she prefers tea added to milk rather than milk added to tea Fisher was skeptical that she could distinguish

More information

Nonparametric Bayesian Methods (Gaussian Processes)

Nonparametric Bayesian Methods (Gaussian Processes) [70240413 Statistical Machine Learning, Spring, 2015] Nonparametric Bayesian Methods (Gaussian Processes) Jun Zhu dcszj@mail.tsinghua.edu.cn http://bigml.cs.tsinghua.edu.cn/~jun State Key Lab of Intelligent

More information

1. Geometry of the unit tangent bundle

1. Geometry of the unit tangent bundle 1 1. Geometry of the unit tangent bundle The main reference for this section is [8]. In the following, we consider (M, g) an n-dimensional smooth manifold endowed with a Riemannian metric g. 1.1. Notations

More information

Variational inference

Variational inference Simon Leglaive Télécom ParisTech, CNRS LTCI, Université Paris Saclay November 18, 2016, Télécom ParisTech, Paris, France. Outline Introduction Probabilistic model Problem Log-likelihood decomposition EM

More information

Bayesian Principal Geodesic Analysis in Diffeomorphic Image Registration

Bayesian Principal Geodesic Analysis in Diffeomorphic Image Registration Bayesian Principal Geodesic Analysis in Diffeomorphic Image Registration Miaomiao Zhang and P. Thomas Fletcher School of Computing, University of Utah, Salt Lake City, USA Abstract. Computing a concise

More information

16 : Markov Chain Monte Carlo (MCMC)

16 : Markov Chain Monte Carlo (MCMC) 10-708: Probabilistic Graphical Models 10-708, Spring 2014 16 : Markov Chain Monte Carlo MCMC Lecturer: Matthew Gormley Scribes: Yining Wang, Renato Negrinho 1 Sampling from low-dimensional distributions

More information

STA414/2104 Statistical Methods for Machine Learning II

STA414/2104 Statistical Methods for Machine Learning II STA414/2104 Statistical Methods for Machine Learning II Murat A. Erdogdu & David Duvenaud Department of Computer Science Department of Statistical Sciences Lecture 3 Slide credits: Russ Salakhutdinov Announcements

More information

Basic Principles of Unsupervised and Unsupervised

Basic Principles of Unsupervised and Unsupervised Basic Principles of Unsupervised and Unsupervised Learning Toward Deep Learning Shun ichi Amari (RIKEN Brain Science Institute) collaborators: R. Karakida, M. Okada (U. Tokyo) Deep Learning Self Organization

More information

Bias-Variance Trade-Off in Hierarchical Probabilistic Models Using Higher-Order Feature Interactions

Bias-Variance Trade-Off in Hierarchical Probabilistic Models Using Higher-Order Feature Interactions - Trade-Off in Hierarchical Probabilistic Models Using Higher-Order Feature Interactions Simon Luo The University of Sydney Data61, CSIRO simon.luo@data61.csiro.au Mahito Sugiyama National Institute of

More information

Stochastic approximation EM algorithm in nonlinear mixed effects model for viral load decrease during anti-hiv treatment

Stochastic approximation EM algorithm in nonlinear mixed effects model for viral load decrease during anti-hiv treatment Stochastic approximation EM algorithm in nonlinear mixed effects model for viral load decrease during anti-hiv treatment Adeline Samson 1, Marc Lavielle and France Mentré 1 1 INSERM E0357, Department of

More information

Diffusion/Inference geometries of data features, situational awareness and visualization. Ronald R Coifman Mathematics Yale University

Diffusion/Inference geometries of data features, situational awareness and visualization. Ronald R Coifman Mathematics Yale University Diffusion/Inference geometries of data features, situational awareness and visualization Ronald R Coifman Mathematics Yale University Digital data is generally converted to point clouds in high dimensional

More information

Gaussian Mixture Model

Gaussian Mixture Model Case Study : Document Retrieval MAP EM, Latent Dirichlet Allocation, Gibbs Sampling Machine Learning/Statistics for Big Data CSE599C/STAT59, University of Washington Emily Fox 0 Emily Fox February 5 th,

More information

Bayesian Inference in GLMs. Frequentists typically base inferences on MLEs, asymptotic confidence

Bayesian Inference in GLMs. Frequentists typically base inferences on MLEs, asymptotic confidence Bayesian Inference in GLMs Frequentists typically base inferences on MLEs, asymptotic confidence limits, and log-likelihood ratio tests Bayesians base inferences on the posterior distribution of the unknowns

More information

Computational Brain Anatomy

Computational Brain Anatomy Computational Brain Anatomy John Ashburner Wellcome Trust Centre for Neuroimaging, 12 Queen Square, London, UK. Overview Voxel-Based Morphometry Morphometry in general Volumetrics VBM preprocessing followed

More information

Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, Spis treści

Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, Spis treści Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, 2017 Spis treści Website Acknowledgments Notation xiii xv xix 1 Introduction 1 1.1 Who Should Read This Book?

More information

Computer Vision Group Prof. Daniel Cremers. 11. Sampling Methods

Computer Vision Group Prof. Daniel Cremers. 11. Sampling Methods Prof. Daniel Cremers 11. Sampling Methods Sampling Methods Sampling Methods are widely used in Computer Science as an approximation of a deterministic algorithm to represent uncertainty without a parametric

More information

Hmms with variable dimension structures and extensions

Hmms with variable dimension structures and extensions Hmm days/enst/january 21, 2002 1 Hmms with variable dimension structures and extensions Christian P. Robert Université Paris Dauphine www.ceremade.dauphine.fr/ xian Hmm days/enst/january 21, 2002 2 1 Estimating

More information

ECE 275B Homework #2 Due Thursday MIDTERM is Scheduled for Tuesday, February 21, 2012

ECE 275B Homework #2 Due Thursday MIDTERM is Scheduled for Tuesday, February 21, 2012 Reading ECE 275B Homework #2 Due Thursday 2-16-12 MIDTERM is Scheduled for Tuesday, February 21, 2012 Read and understand the Newton-Raphson and Method of Scores MLE procedures given in Kay, Example 7.11,

More information

STA 414/2104: Machine Learning

STA 414/2104: Machine Learning STA 414/2104: Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistics! rsalakhu@cs.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 9 Sequential Data So far

More information

SRB measures for non-uniformly hyperbolic systems

SRB measures for non-uniformly hyperbolic systems SRB measures for non-uniformly hyperbolic systems Vaughn Climenhaga University of Maryland October 21, 2010 Joint work with Dmitry Dolgopyat and Yakov Pesin 1 and classical results Definition of SRB measure

More information

Introduction to Machine Learning CMU-10701

Introduction to Machine Learning CMU-10701 Introduction to Machine Learning CMU-10701 Markov Chain Monte Carlo Methods Barnabás Póczos Contents Markov Chain Monte Carlo Methods Sampling Rejection Importance Hastings-Metropolis Gibbs Markov Chains

More information

Markov chain Monte-Carlo to estimate speciation and extinction rates: making use of the forest hidden behind the (phylogenetic) tree

Markov chain Monte-Carlo to estimate speciation and extinction rates: making use of the forest hidden behind the (phylogenetic) tree Markov chain Monte-Carlo to estimate speciation and extinction rates: making use of the forest hidden behind the (phylogenetic) tree Nicolas Salamin Department of Ecology and Evolution University of Lausanne

More information

On Bayesian Computation

On Bayesian Computation On Bayesian Computation Michael I. Jordan with Elaine Angelino, Maxim Rabinovich, Martin Wainwright and Yun Yang Previous Work: Information Constraints on Inference Minimize the minimax risk under constraints

More information

Prerequisite: STATS 7 or STATS 8 or AP90 or (STATS 120A and STATS 120B and STATS 120C). AP90 with a minimum score of 3

Prerequisite: STATS 7 or STATS 8 or AP90 or (STATS 120A and STATS 120B and STATS 120C). AP90 with a minimum score of 3 University of California, Irvine 2017-2018 1 Statistics (STATS) Courses STATS 5. Seminar in Data Science. 1 Unit. An introduction to the field of Data Science; intended for entering freshman and transfers.

More information

Population Games and Evolutionary Dynamics

Population Games and Evolutionary Dynamics Population Games and Evolutionary Dynamics William H. Sandholm The MIT Press Cambridge, Massachusetts London, England in Brief Series Foreword Preface xvii xix 1 Introduction 1 1 Population Games 2 Population

More information

Stat 516, Homework 1

Stat 516, Homework 1 Stat 516, Homework 1 Due date: October 7 1. Consider an urn with n distinct balls numbered 1,..., n. We sample balls from the urn with replacement. Let N be the number of draws until we encounter a ball

More information

Morphometrics with SPM12

Morphometrics with SPM12 Morphometrics with SPM12 John Ashburner Wellcome Trust Centre for Neuroimaging, 12 Queen Square, London, UK. What kind of differences are we looking for? Usually, we try to localise regions of difference.

More information

Slice Sampling with Adaptive Multivariate Steps: The Shrinking-Rank Method

Slice Sampling with Adaptive Multivariate Steps: The Shrinking-Rank Method Slice Sampling with Adaptive Multivariate Steps: The Shrinking-Rank Method Madeleine B. Thompson Radford M. Neal Abstract The shrinking rank method is a variation of slice sampling that is efficient at

More information

STA414/2104. Lecture 11: Gaussian Processes. Department of Statistics

STA414/2104. Lecture 11: Gaussian Processes. Department of Statistics STA414/2104 Lecture 11: Gaussian Processes Department of Statistics www.utstat.utoronto.ca Delivered by Mark Ebden with thanks to Russ Salakhutdinov Outline Gaussian Processes Exam review Course evaluations

More information

Monte Carlo in Bayesian Statistics

Monte Carlo in Bayesian Statistics Monte Carlo in Bayesian Statistics Matthew Thomas SAMBa - University of Bath m.l.thomas@bath.ac.uk December 4, 2014 Matthew Thomas (SAMBa) Monte Carlo in Bayesian Statistics December 4, 2014 1 / 16 Overview

More information

Mathematical Formulation of Our Example

Mathematical Formulation of Our Example Mathematical Formulation of Our Example We define two binary random variables: open and, where is light on or light off. Our question is: What is? Computer Vision 1 Combining Evidence Suppose our robot

More information