Wavelet-Based Nonparametric Modeling of Hierarchical Functions in Colon Carcinogenesis

Size: px
Start display at page:

Download "Wavelet-Based Nonparametric Modeling of Hierarchical Functions in Colon Carcinogenesis"

Transcription

1 Wavelet-Based Nonparametric Modeling of Hierarchical Functions in Colon Carcinogenesis Jeffrey S. Morris University of Texas, MD Anderson Cancer Center Joint wor with Marina Vannucci, Philip J. Brown, and Raymond J. Carroll 4-Aug-05 Jeffrey S. Morris 1

2 Outline Introduction Colon Carcinogenesis Studies Hierarchical Functional Data Wavelets and Wavelet Regression Overview of Method Choice of Smoothing Parameters Application Conclusions 4-Aug-05 Jeffrey S. Morris

3 Colon Carcinogenesis Studies Colon cancer asymptomatic until advanced Limited treatment options for advanced disease Preventive approaches crucial Better understand carcinogenic mechanisms Identify & understand ris factors Dietary factors ey in colon cancer ris 4-Aug-05 Jeffrey S. Morris 3

4 Colon Carcinogenesis Studies Carcinogen-induced colon cancer in rats model used to study biological mechanisms in vivo Rats given treatments of interest (diet) over time Exposed to carcinogen Animals euthanized, colon removed and examined. IHC staining used to quantify responses Slides treated with chemical Staining intensity indicates level of response. 4-Aug-05 Jeffrey S. Morris 4

5 Architecture of Colon Lumen crypts Stem Cells: Mother cells near bottom Depth in crypt ~ age of cells Relative Cell Position (Depth): t (0,1) 4-Aug-05 Jeffrey S. Morris 5

6 Colon Carcinogenesis Studies Data structure: Multiple treatment groups (diets) Multiple rats for each treatment Multiple crypts sampled for each rat Response quantified over fixed grid of t along side of each selected crypt Hierarchical functional data Multi-level random effects model with functional observations 4-Aug-05 Jeffrey S. Morris 6

7 Hierarchical Functional Data -level HF model: Y g g abc abc ab = g ( t) + e ( t) = ( t) = abc g g a ab ( t) ( t) + ξ abc + η ab, abc ( t) ( t) where eabc ~ MVN( 0, σ e I ), η abc ( ) and ξ ab ( ): mean 0 with covariance matrices and Aug-05 Jeffrey S. Morris 7 Σ ( t, t ) Σ ( t, t ).

8 Hierarchical Functional Data Parametric model (linear): g(t) = X(t) β at each level of hierarchy: e.g. line or parabola at each level. i.e. g abc (t) = X(t) β abc, Cov(β abc β ab ) = Σ β1 g ab (t) = X(t) β ab, Cov(β ab β a ) = Σ β g a (t) = X(t) β a Can be fit using standard mixed models software. Efficient and easy IF you can find a parametric form for the g( ) functions. 4-Aug-05 Jeffrey S. Morris 8

9 Hierarchical Functional Data DNA Adduct Level (Damage) Fish,T 6,R 1,C Fish,T 6,R 1,C 6 Fish,T 6,R 1,C 11 Fish,T 6,R 1,C 15 repair repair repair repair x x x x Corn,T 3,R,C 1 Corn,T 3,R,C 6 Corn,T 3,R,C 9 Corn,T 3,R,C 11 repair repair repair repair x x x x Corn,T 1,R 1,C 4 Corn,T 1,R 1,C 9 Corn,T 1,R 1,C 14 Corn,T 1,R 1,C 16 repair repair repair repair x x x x 4-Aug-05 Jeffrey S. Morris 9

10 Hierarchical Functional Data MGMT Level (DNA Repair Enzyme) Fish, T 3, R 1, C 6 Fish, T 9, R 3, C 8 Corn, T 9, R 3, C 19 MGMT MGMT MGMT Fish, T 3, R 1, C 3 t Fish, T 9, R 3, C 19 t Corn, T 9, R 3, C 0 t MGMT MGMT MGMT t t t 4-Aug-05 Jeffrey S. Morris 10

11 Goals Develop method to analyze H.F.D. Est. population-level functions g a (t) (nonparametrically) Perform Bayesian inference Uncover relative variability at each hierarchical level. Wor for hierarchical models with 1 or levels Adust for possibly unequal sample sizes Model spie-lie data at lowest hierarchical level Our method first to do all these Apply to colon carcinogenesis data set Use of wavelets fundamental to our approach 4-Aug-05 Jeffrey S. Morris 11

12 Wavelets & Wavelet Regression Wavelets: families of orthonormal basis functions. Wavelet series for f (t): Wavelet basis functions: ψ, Mother wavelet: ψ (t), f ( t) = d ψ / ( t) = ψ(, I t ), ( t) Wavelet coefficients: d, = f ( t) ψ, ( t) dt 4-Aug-05 Jeffrey S. Morris 1

13 Wavelets & Wavelet Regression Wavelet coefficients calculated using Discrete Wavelet Transform (DWT) Efficient -- O(n) n data points y n wavelet coefficients d Indexed by: resolution level (scale), wavelet number (location). Can be written: d = W y (W orthogonal) Inverse Transform (IDWT): y= W t d 4-Aug-05 Jeffrey S. Morris 13

14 Wavelets & Wavelet Regression Key Properties: Compact support -- model local features in time (spies) Whitening property -- less correlation in wavelet space, allowing parsimonious modeling Sparsity -- most wavelet coefficients 0 for given function Can be used to perform Nonparametric Regression (Donoho & Johnstone, 1994, many others) 4-Aug-05 Jeffrey S. Morris 14

15 Wavelets & Wavelet Regression Data space model: y = f(t) + e t = equally spaced grid, length n= J, on (0,1) e ~ MVN(0,σ I) In wavelet space: d = Wy = θ + e d = empirical wavelet coefficients θ = true wavelet coefficients By orthogonality, e ~ MVN(0,σ I) 4-Aug-05 Jeffrey S. Morris 15

16 Wavelets & Wavelet Regression Wavelet Regression procedure: (1) Transform to wavelet space () Threshold/shrin coefficients 0 (3) Transform bac to data space (IDWT) Thresholding/shrinage filters out noise, performs regularization. Shrinage can be done by placing a certain prior structure on the θ. 4-Aug-05 Jeffrey S. Morris 16

17 Method: Overview 1. Convert data y abc to wavelet space d abc - Involves 1 DWT for each crypt. Fit hierarchical model in wavelet space to obtain Posterior distribution of true wavelet coefficients θ a corresponding to g a (t) Variance component estimates to assess relative variability 3. Use IDWT to obtain posterior distributions of g a (t) for estimation and inference 4-Aug-05 Jeffrey S. Morris 17

18 Method: Model in Wavelet Space d abc = { d, } = W y abc abc d, abc ~ iid N ( θ, abc, σ e ) θ, abc ~ iid N ( θ, ab, σ 1, ) θ, ab ~ iid N ( θ, a, σ, ) 1, σ e assumed nown, in practice estimated by Var( d abc ) IG priors on σ and σ 1,, when unnown 4-Aug-05 Jeffrey S. Morris 18

19 Method; Shrinage Prior, Prior on θ thresholding/shrinage a Mixture of point mass at 0 and Normal,, θ ~ N (0, γ τ a a, γ a ~ Bernoulli( p ) ) p and τ : smoothing parameters 4-Aug-05 Jeffrey S. Morris 19

20 Method: Estimating g a (t) If VC nown closed form posterior ( θ ˆ Θ, a, NS h(.), a σ d, σ µ = = 1,, σ ˆ Θ, a, NS Var = MLE of θ a, ( ) Θ ˆ,, a, NS Normal( µ, σ ( ), τ, p, Θˆ a, NS assuming no shrinage 4-Aug-05 Jeffrey S. Morris 0 h h ( ), τ, p, Θˆ a, NS = shrinage function Using inverse wavelet transform, we get entire posterior distribution of g a (t). ) ~ )

21 Method: Estimating θ, a VC Unnown -- no closed form posterior Metropolis-within-Gibbs Sampler used on marginalized model. Sampling scheme: (1) Sample from Let and {, Θ, = 1, K, K } 4-Aug-05 Jeffrey S. Morris 1 Ω Θ () Sample from Θ a = { σ = 1, a, σ, ( ), Θa d, Ω for each, ( Ω d, ) for each a }

22 Method: Estimating Variability Relative variability at different hierarchical levels can help guide design of future experiments Assessed using the trace: { } { } { tr Σ, tr Σ, and tr σ I} Orthogonality Compute trace in wavelet space V 0 V 1 V = = = tr tr tr { } σ I = nσ e { Σ1} = { Σ} = K σ K σ e 1,, 4-Aug-05 Jeffrey S. Morris 1 Relative variability at level i: V /( V V ) i V e

23 Choice of Smoothing Parameters Shrinage Function: Pr Shrinage BF a BF + 1 p 1 Z T 1+ T exp 1 p T (,, γ = 1 ) a d =, BF = Bayes Factor ( ) { BF = Bayes Factor Z h( Z, T, p ) Prior Odds ˆ, / Var( ˆ = θ θ, a, NS a, NS ) T = Pr γ a a T Nonlinear Lielihood Ratio {, = 1 d, } 1 Shrinage Linear, a, NS 4-Aug-05 Jeffrey S. Morris 3 T / Var( ˆ = τ θ )

24 Choice of Smoothing Parameters p : Expected proportion of non-negligible wavelet coefficients at resolution p Belief that many coeffs. are essentially noise. Results in more smoothing of features at scale Generally -- expect p to decrease in Higher frequency behavior mostly noise 4-Aug-05 Jeffrey S. Morris 4

25 Choice of Smoothing Parameters τ : Variation of nonzero wavelet coeffs. ( ), = τ / Var ˆ a, NS T Θ Must tae care in choosing Too small too much linear shrinage Too large Lindley s paradox Too much nonlinear shrinage for some Z T (10,100) seem to wor quite well. 4-Aug-05 Jeffrey S. Morris 5

26 Choice of Smoothing Parameters (a) Shrinage functions for p= 0.5 (b) Shrinage functions for p= 0.1 h(z,p,t) T= T= 5 T= 0 T= 100 T= 1000 h(z,p,t) T= T= 5 T= 0 T= 100 T= Z Z 4-Aug-05 Jeffrey S. Morris 6

27 Example: Data MGMT expression (DNA repair enzyme) diets (fish/corn) x 5 times (0h,3h,6h,9h,1h) 3 rats per diet x time combination ~ 5 crypts per rat, function sampled at 56= 8 equally spaced t Daubechies-8 wavelet chosen p={1,.0,.07,.03,.015,.01,0.005}, T = 0 MCMC 5000 iterations; Burn-in= Aug-05 Jeffrey S. Morris 7

28 Example: Results Estimates & 90% posterior bounds by diet/time Fish Oil MGMT level Fish Oil 0 hr Fish Oil 3 hr Fish Oil 6 hr Fish Oil 9 hr Fish Oil 1 hr 0 h 3 h 6 h 9 h 1 h MGMT level MGMT level MGMT level MGMT level Relative Cell Position Relative Cell Position Relative Cell Position Relative Cell Position Relative Cell Position Corn Oil 0 hr Corn Oil 3 hr Corn Oil 6 hr Corn Oil 9 hr Corn Oil 1 hr Corn Oil MGMT level MGMT level MGMT level MGMT level MGMT level Relative Cell Position Relative Cell Position Relative Cell Position Relative Cell Position Relative Cell Position 4-Aug-05 Jeffrey S. Morris 8

29 Conclusion: Biological Results More MGMT at top of crypts Diet difference at 1 h near luminal surface (t ~ 1) Fish oil-fed rats -- MGMT from 9 h to 1 h Corn oil-fed rats -- MGMT from 9h to 1 h Fish MGMT > Corn MGMT Crypt-level dominates variability 90.0% at crypt level, 9.6% at rat level, 0.4% residual Need lots of crypts!! MGMT operates on cell-by-cell basis 4-Aug-05 Jeffrey S. Morris 9

30 Conclusion: Summary Method to fit hierarchical functional data nonparametrically estimate mean functions w/ bounds evaluate relative variability at hierarchical levels perform Bayesian inference can handle 1 or hierarchical levels & unbalanced designs appropriate when functions at base level irregular Fitting done in wavelet space Whitening property allows parsimonious modeling of Σ i Can represent spatially heterogeneous functions well Motivated by colon carc., but applicable elsewhere Possible extensions of method 4-Aug-05 Jeffrey S. Morris 30

Wavelet-based functional mixed models

Wavelet-based functional mixed models J. R. Statist. Soc. B (26) 68, Part 2, pp. 79 99 Wavelet-based functional mixed models Jeffrey S. Morris University of Texas MD Anderson Cancer Center, Houston, USA and Raymond J. Carroll Texas A&M University,

More information

Bayesian Wavelet-Based Functional Mixed Models

Bayesian Wavelet-Based Functional Mixed Models Bayesian Wavelet-Based Functional Mixed Models Jeffrey S. Morris U M.D. Anderson Cancer Center Raymond J. Carroll exas A&M University Functional Data Analysis Functional Data: Ideal units of observation:

More information

BAYESIAN METHODS FOR VARIABLE SELECTION WITH APPLICATIONS TO HIGH-DIMENSIONAL DATA

BAYESIAN METHODS FOR VARIABLE SELECTION WITH APPLICATIONS TO HIGH-DIMENSIONAL DATA BAYESIAN METHODS FOR VARIABLE SELECTION WITH APPLICATIONS TO HIGH-DIMENSIONAL DATA Part 3: Functional Data & Wavelets Marina Vannucci Rice University, USA PASI-CIMAT 4/28-3/2 Marina Vannucci (Rice University,

More information

Functional Latent Feature Models. With Single-Index Interaction

Functional Latent Feature Models. With Single-Index Interaction Generalized With Single-Index Interaction Department of Statistics Center for Statistical Bioinformatics Institute for Applied Mathematics and Computational Science Texas A&M University Naisyin Wang and

More information

Part 6: Multivariate Normal and Linear Models

Part 6: Multivariate Normal and Linear Models Part 6: Multivariate Normal and Linear Models 1 Multiple measurements Up until now all of our statistical models have been univariate models models for a single measurement on each member of a sample of

More information

Bayesian Analysis of Mass Spectrometry Data Using Wavelet-Based Functional Mixed Models

Bayesian Analysis of Mass Spectrometry Data Using Wavelet-Based Functional Mixed Models Bayesian Analysis of Mass Spectrometry Data Using Wavelet-Based Functional Mixed Models Jeffrey S. Morris UT MD Anderson Cancer Center joint work with Philip J. Brown, Kevin R. Coombes, Keith A. Baggerly

More information

Or How to select variables Using Bayesian LASSO

Or How to select variables Using Bayesian LASSO Or How to select variables Using Bayesian LASSO x 1 x 2 x 3 x 4 Or How to select variables Using Bayesian LASSO x 1 x 2 x 3 x 4 Or How to select variables Using Bayesian LASSO On Bayesian Variable Selection

More information

Default Priors and Effcient Posterior Computation in Bayesian

Default Priors and Effcient Posterior Computation in Bayesian Default Priors and Effcient Posterior Computation in Bayesian Factor Analysis January 16, 2010 Presented by Eric Wang, Duke University Background and Motivation A Brief Review of Parameter Expansion Literature

More information

Bayesian non-parametric model to longitudinally predict churn

Bayesian non-parametric model to longitudinally predict churn Bayesian non-parametric model to longitudinally predict churn Bruno Scarpa Università di Padova Conference of European Statistics Stakeholders Methodologists, Producers and Users of European Statistics

More information

Making rating curves - the Bayesian approach

Making rating curves - the Bayesian approach Making rating curves - the Bayesian approach Rating curves what is wanted? A best estimate of the relationship between stage and discharge at a given place in a river. The relationship should be on the

More information

Part 8: GLMs and Hierarchical LMs and GLMs

Part 8: GLMs and Hierarchical LMs and GLMs Part 8: GLMs and Hierarchical LMs and GLMs 1 Example: Song sparrow reproductive success Arcese et al., (1992) provide data on a sample from a population of 52 female song sparrows studied over the course

More information

Multivariate Bayes Wavelet Shrinkage and Applications

Multivariate Bayes Wavelet Shrinkage and Applications Journal of Applied Statistics Vol. 32, No. 5, 529 542, July 2005 Multivariate Bayes Wavelet Shrinkage and Applications GABRIEL HUERTA Department of Mathematics and Statistics, University of New Mexico

More information

Multivariate Survival Analysis

Multivariate Survival Analysis Multivariate Survival Analysis Previously we have assumed that either (X i, δ i ) or (X i, δ i, Z i ), i = 1,..., n, are i.i.d.. This may not always be the case. Multivariate survival data can arise in

More information

Analysing geoadditive regression data: a mixed model approach

Analysing geoadditive regression data: a mixed model approach Analysing geoadditive regression data: a mixed model approach Institut für Statistik, Ludwig-Maximilians-Universität München Joint work with Ludwig Fahrmeir & Stefan Lang 25.11.2005 Spatio-temporal regression

More information

Bayesian Methods for Machine Learning

Bayesian Methods for Machine Learning Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),

More information

MIXED EFFECTS MODELS FOR TIME SERIES

MIXED EFFECTS MODELS FOR TIME SERIES Outline MIXED EFFECTS MODELS FOR TIME SERIES Cristina Gorrostieta Hakmook Kang Hernando Ombao Brown University Biostatistics Section February 16, 2011 Outline OUTLINE OF TALK 1 SCIENTIFIC MOTIVATION 2

More information

CPSC 540: Machine Learning

CPSC 540: Machine Learning CPSC 540: Machine Learning MCMC and Non-Parametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is

More information

Bayesian linear regression

Bayesian linear regression Bayesian linear regression Linear regression is the basis of most statistical modeling. The model is Y i = X T i β + ε i, where Y i is the continuous response X i = (X i1,..., X ip ) T is the corresponding

More information

Density Estimation. Seungjin Choi

Density Estimation. Seungjin Choi Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/

More information

ECE276A: Sensing & Estimation in Robotics Lecture 10: Gaussian Mixture and Particle Filtering

ECE276A: Sensing & Estimation in Robotics Lecture 10: Gaussian Mixture and Particle Filtering ECE276A: Sensing & Estimation in Robotics Lecture 10: Gaussian Mixture and Particle Filtering Lecturer: Nikolay Atanasov: natanasov@ucsd.edu Teaching Assistants: Siwei Guo: s9guo@eng.ucsd.edu Anwesan Pal:

More information

Introduction Wavelet shrinage methods have been very successful in nonparametric regression. But so far most of the wavelet regression methods have be

Introduction Wavelet shrinage methods have been very successful in nonparametric regression. But so far most of the wavelet regression methods have be Wavelet Estimation For Samples With Random Uniform Design T. Tony Cai Department of Statistics, Purdue University Lawrence D. Brown Department of Statistics, University of Pennsylvania Abstract We show

More information

Principles of Bayesian Inference

Principles of Bayesian Inference Principles of Bayesian Inference Sudipto Banerjee University of Minnesota July 20th, 2008 1 Bayesian Principles Classical statistics: model parameters are fixed and unknown. A Bayesian thinks of parameters

More information

BAYESIAN METHODS FOR VARIABLE SELECTION WITH APPLICATIONS TO HIGH-DIMENSIONAL DATA

BAYESIAN METHODS FOR VARIABLE SELECTION WITH APPLICATIONS TO HIGH-DIMENSIONAL DATA BAYESIAN METHODS FOR VARIABLE SELECTION WITH APPLICATIONS TO HIGH-DIMENSIONAL DATA Intro: Course Outline and Brief Intro to Marina Vannucci Rice University, USA PASI-CIMAT 04/28-30/2010 Marina Vannucci

More information

Approximate Bayesian Computation

Approximate Bayesian Computation Approximate Bayesian Computation Michael Gutmann https://sites.google.com/site/michaelgutmann University of Helsinki and Aalto University 1st December 2015 Content Two parts: 1. The basics of approximate

More information

Bayesian Semiparametric GARCH Models

Bayesian Semiparametric GARCH Models Bayesian Semiparametric GARCH Models Xibin (Bill) Zhang and Maxwell L. King Department of Econometrics and Business Statistics Faculty of Business and Economics xibin.zhang@monash.edu Quantitative Methods

More information

Bayesian Semiparametric GARCH Models

Bayesian Semiparametric GARCH Models Bayesian Semiparametric GARCH Models Xibin (Bill) Zhang and Maxwell L. King Department of Econometrics and Business Statistics Faculty of Business and Economics xibin.zhang@monash.edu Quantitative Methods

More information

Advanced Machine Learning

Advanced Machine Learning Advanced Machine Learning Nonparametric Bayesian Models --Learning/Reasoning in Open Possible Worlds Eric Xing Lecture 7, August 4, 2009 Reading: Eric Xing Eric Xing @ CMU, 2006-2009 Clustering Eric Xing

More information

Aspects of Feature Selection in Mass Spec Proteomic Functional Data

Aspects of Feature Selection in Mass Spec Proteomic Functional Data Aspects of Feature Selection in Mass Spec Proteomic Functional Data Phil Brown University of Kent Newton Institute 11th December 2006 Based on work with Jeff Morris, and others at MD Anderson, Houston

More information

Bayesian Inference. Chapter 4: Regression and Hierarchical Models

Bayesian Inference. Chapter 4: Regression and Hierarchical Models Bayesian Inference Chapter 4: Regression and Hierarchical Models Conchi Ausín and Mike Wiper Department of Statistics Universidad Carlos III de Madrid Advanced Statistics and Data Mining Summer School

More information

MCMC Sampling for Bayesian Inference using L1-type Priors

MCMC Sampling for Bayesian Inference using L1-type Priors MÜNSTER MCMC Sampling for Bayesian Inference using L1-type Priors (what I do whenever the ill-posedness of EEG/MEG is just not frustrating enough!) AG Imaging Seminar Felix Lucka 26.06.2012 , MÜNSTER Sampling

More information

10-701/15-781, Machine Learning: Homework 4

10-701/15-781, Machine Learning: Homework 4 10-701/15-781, Machine Learning: Homewor 4 Aarti Singh Carnegie Mellon University ˆ The assignment is due at 10:30 am beginning of class on Mon, Nov 15, 2010. ˆ Separate you answers into five parts, one

More information

Modelling trends in the ocean wave climate for dimensioning of ships

Modelling trends in the ocean wave climate for dimensioning of ships Modelling trends in the ocean wave climate for dimensioning of ships STK1100 lecture, University of Oslo Erik Vanem Motivation and background 2 Ocean waves and maritime safety Ships and other marine structures

More information

Bayesian Nonparametric Rasch Modeling: Methods and Software

Bayesian Nonparametric Rasch Modeling: Methods and Software Bayesian Nonparametric Rasch Modeling: Methods and Software George Karabatsos University of Illinois-Chicago Keynote talk Friday May 2, 2014 (9:15-10am) Ohio River Valley Objective Measurement Seminar

More information

Gaussian kernel GARCH models

Gaussian kernel GARCH models Gaussian kernel GARCH models Xibin (Bill) Zhang and Maxwell L. King Department of Econometrics and Business Statistics Faculty of Business and Economics 7 June 2013 Motivation A regression model is often

More information

Review: Probabilistic Matrix Factorization. Probabilistic Matrix Factorization (PMF)

Review: Probabilistic Matrix Factorization. Probabilistic Matrix Factorization (PMF) Case Study 4: Collaborative Filtering Review: Probabilistic Matrix Factorization Machine Learning for Big Data CSE547/STAT548, University of Washington Emily Fox February 2 th, 214 Emily Fox 214 1 Probabilistic

More information

Bayesian Additive Regression Tree (BART) with application to controlled trail data analysis

Bayesian Additive Regression Tree (BART) with application to controlled trail data analysis Bayesian Additive Regression Tree (BART) with application to controlled trail data analysis Weilan Yang wyang@stat.wisc.edu May. 2015 1 / 20 Background CATE i = E(Y i (Z 1 ) Y i (Z 0 ) X i ) 2 / 20 Background

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 3 Linear

More information

ABC random forest for parameter estimation. Jean-Michel Marin

ABC random forest for parameter estimation. Jean-Michel Marin ABC random forest for parameter estimation Jean-Michel Marin Université de Montpellier Institut Montpelliérain Alexander Grothendieck (IMAG) Institut de Biologie Computationnelle (IBC) Labex Numev! joint

More information

Supplemental materials to Reduced Rank Mixed Effects Models for Spatially Correlated Hierarchical Functional Data

Supplemental materials to Reduced Rank Mixed Effects Models for Spatially Correlated Hierarchical Functional Data Supplemental materials to Reduced Rank Mixed Effects Models for Spatially Correlated Hierarchical Functional Data Lan Zhou, Jianhua Z. Huang, Josue G. Martinez, Arnab Maity, Veerabhadran Baladandayuthapani

More information

Bayesian data analysis in practice: Three simple examples

Bayesian data analysis in practice: Three simple examples Bayesian data analysis in practice: Three simple examples Martin P. Tingley Introduction These notes cover three examples I presented at Climatea on 5 October 0. Matlab code is available by request to

More information

Bayesian Regression Linear and Logistic Regression

Bayesian Regression Linear and Logistic Regression When we want more than point estimates Bayesian Regression Linear and Logistic Regression Nicole Beckage Ordinary Least Squares Regression and Lasso Regression return only point estimates But what if we

More information

Supplement to A Hierarchical Approach for Fitting Curves to Response Time Measurements

Supplement to A Hierarchical Approach for Fitting Curves to Response Time Measurements Supplement to A Hierarchical Approach for Fitting Curves to Response Time Measurements Jeffrey N. Rouder Francis Tuerlinckx Paul L. Speckman Jun Lu & Pablo Gomez May 4 008 1 The Weibull regression model

More information

Modeling Real Estate Data using Quantile Regression

Modeling Real Estate Data using Quantile Regression Modeling Real Estate Data using Semiparametric Quantile Regression Department of Statistics University of Innsbruck September 9th, 2011 Overview 1 Application: 2 3 4 Hedonic regression data for house prices

More information

Bayesian Inference. Chapter 4: Regression and Hierarchical Models

Bayesian Inference. Chapter 4: Regression and Hierarchical Models Bayesian Inference Chapter 4: Regression and Hierarchical Models Conchi Ausín and Mike Wiper Department of Statistics Universidad Carlos III de Madrid Master in Business Administration and Quantitative

More information

g-priors for Linear Regression

g-priors for Linear Regression Stat60: Bayesian Modeling and Inference Lecture Date: March 15, 010 g-priors for Linear Regression Lecturer: Michael I. Jordan Scribe: Andrew H. Chan 1 Linear regression and g-priors In the last lecture,

More information

Model Selection in Bayesian Survival Analysis for a Multi-country Cluster Randomized Trial

Model Selection in Bayesian Survival Analysis for a Multi-country Cluster Randomized Trial Model Selection in Bayesian Survival Analysis for a Multi-country Cluster Randomized Trial Jin Kyung Park International Vaccine Institute Min Woo Chae Seoul National University R. Leon Ochiai International

More information

Dynamic Factor Models and Factor Augmented Vector Autoregressions. Lawrence J. Christiano

Dynamic Factor Models and Factor Augmented Vector Autoregressions. Lawrence J. Christiano Dynamic Factor Models and Factor Augmented Vector Autoregressions Lawrence J Christiano Dynamic Factor Models and Factor Augmented Vector Autoregressions Problem: the time series dimension of data is relatively

More information

Stat 451 Lecture Notes Markov Chain Monte Carlo. Ryan Martin UIC

Stat 451 Lecture Notes Markov Chain Monte Carlo. Ryan Martin UIC Stat 451 Lecture Notes 07 12 Markov Chain Monte Carlo Ryan Martin UIC www.math.uic.edu/~rgmartin 1 Based on Chapters 8 9 in Givens & Hoeting, Chapters 25 27 in Lange 2 Updated: April 4, 2016 1 / 42 Outline

More information

Monte Carlo in Bayesian Statistics

Monte Carlo in Bayesian Statistics Monte Carlo in Bayesian Statistics Matthew Thomas SAMBa - University of Bath m.l.thomas@bath.ac.uk December 4, 2014 Matthew Thomas (SAMBa) Monte Carlo in Bayesian Statistics December 4, 2014 1 / 16 Overview

More information

Bayesian Prediction of Code Output. ASA Albuquerque Chapter Short Course October 2014

Bayesian Prediction of Code Output. ASA Albuquerque Chapter Short Course October 2014 Bayesian Prediction of Code Output ASA Albuquerque Chapter Short Course October 2014 Abstract This presentation summarizes Bayesian prediction methodology for the Gaussian process (GP) surrogate representation

More information

F denotes cumulative density. denotes probability density function; (.)

F denotes cumulative density. denotes probability density function; (.) BAYESIAN ANALYSIS: FOREWORDS Notation. System means the real thing and a model is an assumed mathematical form for the system.. he probability model class M contains the set of the all admissible models

More information

Gaussian processes for spatial modelling in environmental health: parameterizing for flexibility vs. computational efficiency

Gaussian processes for spatial modelling in environmental health: parameterizing for flexibility vs. computational efficiency Gaussian processes for spatial modelling in environmental health: parameterizing for flexibility vs. computational efficiency Chris Paciorek March 11, 2005 Department of Biostatistics Harvard School of

More information

Statistics & Data Sciences: First Year Prelim Exam May 2018

Statistics & Data Sciences: First Year Prelim Exam May 2018 Statistics & Data Sciences: First Year Prelim Exam May 2018 Instructions: 1. Do not turn this page until instructed to do so. 2. Start each new question on a new sheet of paper. 3. This is a closed book

More information

Super-Resolution. Shai Avidan Tel-Aviv University

Super-Resolution. Shai Avidan Tel-Aviv University Super-Resolution Shai Avidan Tel-Aviv University Slide Credits (partial list) Ric Szelisi Steve Seitz Alyosha Efros Yacov Hel-Or Yossi Rubner Mii Elad Marc Levoy Bill Freeman Fredo Durand Sylvain Paris

More information

1 EM algorithm: updating the mixing proportions {π k } ik are the posterior probabilities at the qth iteration of EM.

1 EM algorithm: updating the mixing proportions {π k } ik are the posterior probabilities at the qth iteration of EM. Université du Sud Toulon - Var Master Informatique Probabilistic Learning and Data Analysis TD: Model-based clustering by Faicel CHAMROUKHI Solution The aim of this practical wor is to show how the Classification

More information

Part 7: Hierarchical Modeling

Part 7: Hierarchical Modeling Part 7: Hierarchical Modeling!1 Nested data It is common for data to be nested: i.e., observations on subjects are organized by a hierarchy Such data are often called hierarchical or multilevel For example,

More information

CTDL-Positive Stable Frailty Model

CTDL-Positive Stable Frailty Model CTDL-Positive Stable Frailty Model M. Blagojevic 1, G. MacKenzie 2 1 Department of Mathematics, Keele University, Staffordshire ST5 5BG,UK and 2 Centre of Biostatistics, University of Limerick, Ireland

More information

Construction of an Informative Hierarchical Prior Distribution: Application to Electricity Load Forecasting

Construction of an Informative Hierarchical Prior Distribution: Application to Electricity Load Forecasting Construction of an Informative Hierarchical Prior Distribution: Application to Electricity Load Forecasting Anne Philippe Laboratoire de Mathématiques Jean Leray Université de Nantes Workshop EDF-INRIA,

More information

Hypothesis Testing. Part I. James J. Heckman University of Chicago. Econ 312 This draft, April 20, 2006

Hypothesis Testing. Part I. James J. Heckman University of Chicago. Econ 312 This draft, April 20, 2006 Hypothesis Testing Part I James J. Heckman University of Chicago Econ 312 This draft, April 20, 2006 1 1 A Brief Review of Hypothesis Testing and Its Uses values and pure significance tests (R.A. Fisher)

More information

STA 216, GLM, Lecture 16. October 29, 2007

STA 216, GLM, Lecture 16. October 29, 2007 STA 216, GLM, Lecture 16 October 29, 2007 Efficient Posterior Computation in Factor Models Underlying Normal Models Generalized Latent Trait Models Formulation Genetic Epidemiology Illustration Structural

More information

Bayes Model Selection with Path Sampling: Factor Models

Bayes Model Selection with Path Sampling: Factor Models with Path Sampling: Factor Models Ritabrata Dutta and Jayanta K Ghosh Purdue University 07/02/11 Factor Models in Applications Factor Models in Applications Factor Models Factor Models and Factor analysis

More information

Bayesian Hierarchical Spatially Correlated Functional Data Analysis with Application to Colon Carcinogenesis. Abstract

Bayesian Hierarchical Spatially Correlated Functional Data Analysis with Application to Colon Carcinogenesis. Abstract Bayesian Hierarchical Spatially Correlated Functional Data Analysis with Application to Colon Carcinogenesis Veerabhadran Baladandayuthapani 1, Bani K. Mallick 2, Mee Young Hong 3 4, Joanne R. Lupton 4,

More information

STAT 518 Intro Student Presentation

STAT 518 Intro Student Presentation STAT 518 Intro Student Presentation Wen Wei Loh April 11, 2013 Title of paper Radford M. Neal [1999] Bayesian Statistics, 6: 475-501, 1999 What the paper is about Regression and Classification Flexible

More information

Bayesian spatial hierarchical modeling for temperature extremes

Bayesian spatial hierarchical modeling for temperature extremes Bayesian spatial hierarchical modeling for temperature extremes Indriati Bisono Dr. Andrew Robinson Dr. Aloke Phatak Mathematics and Statistics Department The University of Melbourne Maths, Informatics

More information

(5) Multi-parameter models - Gibbs sampling. ST440/540: Applied Bayesian Analysis

(5) Multi-parameter models - Gibbs sampling. ST440/540: Applied Bayesian Analysis Summarizing a posterior Given the data and prior the posterior is determined Summarizing the posterior gives parameter estimates, intervals, and hypothesis tests Most of these computations are integrals

More information

Stochastic Spectral Approaches to Bayesian Inference

Stochastic Spectral Approaches to Bayesian Inference Stochastic Spectral Approaches to Bayesian Inference Prof. Nathan L. Gibson Department of Mathematics Applied Mathematics and Computation Seminar March 4, 2011 Prof. Gibson (OSU) Spectral Approaches to

More information

November 2002 STA Random Effects Selection in Linear Mixed Models

November 2002 STA Random Effects Selection in Linear Mixed Models November 2002 STA216 1 Random Effects Selection in Linear Mixed Models November 2002 STA216 2 Introduction It is common practice in many applications to collect multiple measurements on a subject. Linear

More information

Efficient Bayesian Multivariate Surface Regression

Efficient Bayesian Multivariate Surface Regression Efficient Bayesian Multivariate Surface Regression Feng Li (joint with Mattias Villani) Department of Statistics, Stockholm University October, 211 Outline of the talk 1 Flexible regression models 2 The

More information

Wavelet Shrinkage for Nonequispaced Samples

Wavelet Shrinkage for Nonequispaced Samples University of Pennsylvania ScholarlyCommons Statistics Papers Wharton Faculty Research 1998 Wavelet Shrinkage for Nonequispaced Samples T. Tony Cai University of Pennsylvania Lawrence D. Brown University

More information

Ronald Christensen. University of New Mexico. Albuquerque, New Mexico. Wesley Johnson. University of California, Irvine. Irvine, California

Ronald Christensen. University of New Mexico. Albuquerque, New Mexico. Wesley Johnson. University of California, Irvine. Irvine, California Texts in Statistical Science Bayesian Ideas and Data Analysis An Introduction for Scientists and Statisticians Ronald Christensen University of New Mexico Albuquerque, New Mexico Wesley Johnson University

More information

CSC321 Lecture 18: Learning Probabilistic Models

CSC321 Lecture 18: Learning Probabilistic Models CSC321 Lecture 18: Learning Probabilistic Models Roger Grosse Roger Grosse CSC321 Lecture 18: Learning Probabilistic Models 1 / 25 Overview So far in this course: mainly supervised learning Language modeling

More information

Fast Likelihood-Free Inference via Bayesian Optimization

Fast Likelihood-Free Inference via Bayesian Optimization Fast Likelihood-Free Inference via Bayesian Optimization Michael Gutmann https://sites.google.com/site/michaelgutmann University of Helsinki Aalto University Helsinki Institute for Information Technology

More information

Stat 5101 Lecture Notes

Stat 5101 Lecture Notes Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random

More information

Bayesian Inference for DSGE Models. Lawrence J. Christiano

Bayesian Inference for DSGE Models. Lawrence J. Christiano Bayesian Inference for DSGE Models Lawrence J. Christiano Outline State space-observer form. convenient for model estimation and many other things. Bayesian inference Bayes rule. Monte Carlo integation.

More information

Chapter 4 - Fundamentals of spatial processes Lecture notes

Chapter 4 - Fundamentals of spatial processes Lecture notes Chapter 4 - Fundamentals of spatial processes Lecture notes Geir Storvik January 21, 2013 STK4150 - Intro 2 Spatial processes Typically correlation between nearby sites Mostly positive correlation Negative

More information

Linear Mixed Models. One-way layout REML. Likelihood. Another perspective. Relationship to classical ideas. Drawbacks.

Linear Mixed Models. One-way layout REML. Likelihood. Another perspective. Relationship to classical ideas. Drawbacks. Linear Mixed Models One-way layout Y = Xβ + Zb + ɛ where X and Z are specified design matrices, β is a vector of fixed effect coefficients, b and ɛ are random, mean zero, Gaussian if needed. Usually think

More information

Bayesian Hierarchical Models

Bayesian Hierarchical Models Bayesian Hierarchical Models Gavin Shaddick, Millie Green, Matthew Thomas University of Bath 6 th - 9 th December 2016 1/ 34 APPLICATIONS OF BAYESIAN HIERARCHICAL MODELS 2/ 34 OUTLINE Spatial epidemiology

More information

Approximate Bayesian Computation and Particle Filters

Approximate Bayesian Computation and Particle Filters Approximate Bayesian Computation and Particle Filters Dennis Prangle Reading University 5th February 2014 Introduction Talk is mostly a literature review A few comments on my own ongoing research See Jasra

More information

Bayesian Learning. HT2015: SC4 Statistical Data Mining and Machine Learning. Maximum Likelihood Principle. The Bayesian Learning Framework

Bayesian Learning. HT2015: SC4 Statistical Data Mining and Machine Learning. Maximum Likelihood Principle. The Bayesian Learning Framework HT5: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford http://www.stats.ox.ac.uk/~sejdinov/sdmml.html Maximum Likelihood Principle A generative model for

More information

Labor-Supply Shifts and Economic Fluctuations. Technical Appendix

Labor-Supply Shifts and Economic Fluctuations. Technical Appendix Labor-Supply Shifts and Economic Fluctuations Technical Appendix Yongsung Chang Department of Economics University of Pennsylvania Frank Schorfheide Department of Economics University of Pennsylvania January

More information

An Introduction to Wavelets and some Applications

An Introduction to Wavelets and some Applications An Introduction to Wavelets and some Applications Milan, May 2003 Anestis Antoniadis Laboratoire IMAG-LMC University Joseph Fourier Grenoble, France An Introduction to Wavelets and some Applications p.1/54

More information

Bayesian Sparse Linear Regression with Unknown Symmetric Error

Bayesian Sparse Linear Regression with Unknown Symmetric Error Bayesian Sparse Linear Regression with Unknown Symmetric Error Minwoo Chae 1 Joint work with Lizhen Lin 2 David B. Dunson 3 1 Department of Mathematics, The University of Texas at Austin 2 Department of

More information

Web Appendix for Hierarchical Adaptive Regression Kernels for Regression with Functional Predictors by D. B. Woodard, C. Crainiceanu, and D.

Web Appendix for Hierarchical Adaptive Regression Kernels for Regression with Functional Predictors by D. B. Woodard, C. Crainiceanu, and D. Web Appendix for Hierarchical Adaptive Regression Kernels for Regression with Functional Predictors by D. B. Woodard, C. Crainiceanu, and D. Ruppert A. EMPIRICAL ESTIMATE OF THE KERNEL MIXTURE Here we

More information

Metropolis-Hastings Algorithm

Metropolis-Hastings Algorithm Strength of the Gibbs sampler Metropolis-Hastings Algorithm Easy algorithm to think about. Exploits the factorization properties of the joint probability distribution. No difficult choices to be made to

More information

Bayesian Methods for Wavelet Series in Single-Index Models

Bayesian Methods for Wavelet Series in Single-Index Models Bayesian Methods for Wavelet Series in Single-Index Models Chun Gun PARK, Marina VANNUCCI, and Jeffrey D. HART Single-index models have found applications in econometrics and biometrics, where multidimensional

More information

Integrated Non-Factorized Variational Inference

Integrated Non-Factorized Variational Inference Integrated Non-Factorized Variational Inference Shaobo Han, Xuejun Liao and Lawrence Carin Duke University February 27, 2014 S. Han et al. Integrated Non-Factorized Variational Inference February 27, 2014

More information

Some Curiosities Arising in Objective Bayesian Analysis

Some Curiosities Arising in Objective Bayesian Analysis . Some Curiosities Arising in Objective Bayesian Analysis Jim Berger Duke University Statistical and Applied Mathematical Institute Yale University May 15, 2009 1 Three vignettes related to John s work

More information

Bayesian Linear Regression

Bayesian Linear Regression Bayesian Linear Regression Sudipto Banerjee 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. September 15, 2010 1 Linear regression models: a Bayesian perspective

More information

Katsuhiro Sugita Faculty of Law and Letters, University of the Ryukyus. Abstract

Katsuhiro Sugita Faculty of Law and Letters, University of the Ryukyus. Abstract Bayesian analysis of a vector autoregressive model with multiple structural breaks Katsuhiro Sugita Faculty of Law and Letters, University of the Ryukyus Abstract This paper develops a Bayesian approach

More information

Introduction to Probabilistic Machine Learning

Introduction to Probabilistic Machine Learning Introduction to Probabilistic Machine Learning Piyush Rai Dept. of CSE, IIT Kanpur (Mini-course 1) Nov 03, 2015 Piyush Rai (IIT Kanpur) Introduction to Probabilistic Machine Learning 1 Machine Learning

More information

Resolving the White Noise Paradox in the Regularisation of Inverse Problems

Resolving the White Noise Paradox in the Regularisation of Inverse Problems 1 / 32 Resolving the White Noise Paradox in the Regularisation of Inverse Problems Hanne Kekkonen joint work with Matti Lassas and Samuli Siltanen Department of Mathematics and Statistics University of

More information

Parametric Modelling of Over-dispersed Count Data. Part III / MMath (Applied Statistics) 1

Parametric Modelling of Over-dispersed Count Data. Part III / MMath (Applied Statistics) 1 Parametric Modelling of Over-dispersed Count Data Part III / MMath (Applied Statistics) 1 Introduction Poisson regression is the de facto approach for handling count data What happens then when Poisson

More information

ST 740: Linear Models and Multivariate Normal Inference

ST 740: Linear Models and Multivariate Normal Inference ST 740: Linear Models and Multivariate Normal Inference Alyson Wilson Department of Statistics North Carolina State University November 4, 2013 A. Wilson (NCSU STAT) Linear Models November 4, 2013 1 /

More information

... Econometric Methods for the Analysis of Dynamic General Equilibrium Models

... Econometric Methods for the Analysis of Dynamic General Equilibrium Models ... Econometric Methods for the Analysis of Dynamic General Equilibrium Models 1 Overview Multiple Equation Methods State space-observer form Three Examples of Versatility of state space-observer form:

More information

Bayesian inference for multivariate skew-normal and skew-t distributions

Bayesian inference for multivariate skew-normal and skew-t distributions Bayesian inference for multivariate skew-normal and skew-t distributions Brunero Liseo Sapienza Università di Roma Banff, May 2013 Outline Joint research with Antonio Parisi (Roma Tor Vergata) 1. Inferential

More information

Bayesian Model Comparison:

Bayesian Model Comparison: Bayesian Model Comparison: Modeling Petrobrás log-returns Hedibert Freitas Lopes February 2014 Log price: y t = log p t Time span: 12/29/2000-12/31/2013 (n = 3268 days) LOG PRICE 1 2 3 4 0 500 1000 1500

More information

DIC, AIC, BIC, PPL, MSPE Residuals Predictive residuals

DIC, AIC, BIC, PPL, MSPE Residuals Predictive residuals DIC, AIC, BIC, PPL, MSPE Residuals Predictive residuals Overall Measures of GOF Deviance: this measures the overall likelihood of the model given a parameter vector D( θ) = 2 log L( θ) This can be averaged

More information

Log Gaussian Cox Processes. Chi Group Meeting February 23, 2016

Log Gaussian Cox Processes. Chi Group Meeting February 23, 2016 Log Gaussian Cox Processes Chi Group Meeting February 23, 2016 Outline Typical motivating application Introduction to LGCP model Brief overview of inference Applications in my work just getting started

More information

1. Introduction. Hang Qian 1 Iowa State University

1. Introduction. Hang Qian 1 Iowa State University Users Guide to the VARDAS Package Hang Qian 1 Iowa State University 1. Introduction The Vector Autoregression (VAR) model is widely used in macroeconomics. However, macroeconomic data are not always observed

More information

7. Estimation and hypothesis testing. Objective. Recommended reading

7. Estimation and hypothesis testing. Objective. Recommended reading 7. Estimation and hypothesis testing Objective In this chapter, we show how the election of estimators can be represented as a decision problem. Secondly, we consider the problem of hypothesis testing

More information