Bayesian Nonparametric Autoregressive Models via Latent Variable Representation
|
|
- Candace Holt
- 5 years ago
- Views:
Transcription
1 Bayesian Nonparametric Autoregressive Models via Latent Variable Representation Maria De Iorio Yale-NUS College Dept of Statistical Science, University College London Collaborators: Lifeng Ye (UCL, London, UK) Alessandra Guglielmi (Polimi, Milano, Italy) Stefano Favaro (Universitá degli Studi di Torino, Italy) 29th August 2018 Bayesian Computations for High-Dimensional Statistical Models
2 Motivation T=1 T=2 T=3
3 Want flexible model for time-evolving distribution data driven clustering allow for covariates prediction feasible posterior inference framework for automatic information sharing across time possible easily generalizable tools for efficient sharing of information across data dimension (e.g. space)
4 Nonparametric Mixture At time t, data y 1t,..., y nt iid y it ft (y) = K(y θ)g t ( dθ) where G t is the mixing distribution at time t. Assign flexible prior to G, e.g. DP, PT, Pytman-Yor,... Bayesian Nonparametric Mixture Models
5 Dirichlet Process (DP) Probability model on distributions G DP(α, G 0 ), with measure G 0 = E(G) and precision parameter α. G is a.s. discrete
6 Sethuraman s stick breaking representation G = w h δ θh h=1 ξ h Beta(1, α) w h = h 1 ξ h (1 ξ i ), scaled Beta distribution i=1 θ h iid G0, h = 1, 2,... where δ(x) denotes a point mass at x, ψ h are weights of point masses at locations θ h.
7 Dirichlet Process Mixtures (DPM) In many data analysis applications the discreteness is inappropriate. To remove discreteness: convolution with a continuous kernel f (y) = p(y θ)dg(θ) G DP(α, G 0 )
8 Dirichlet Process Mixtures (DPM)... or with latent variables θ i G DP(α, G 0 ) θ i G f (y) = p(y θ i ) Nice feature: Mixture is discrete with probability one, and with small α, there can be high probabilities of a finite mixture. Often p(y θ) = N(β, σ 2 ) f (y) = h=1 w hn(β h, σ 2 )
9 Comment Under G : p(θ i = θ i ) > 0 Observations share the same θ belong to the same cluster DPM induces a random partition of the observations {1,..., n} Note: in the previous model the clustering of observations depends only on the the distribution of y.
10 Models for collection of distributions Observations are associated to different temporal coordinates ind y t f t (y) = K(y θ)g t ( dθ) Recent research focuses on developing models for a collection of random distributions {G t ; t T } Interest: T discrete set. Goal: induce dependence. Reason: properties of the distributions f t are thought to be similar in some way; e.g. similar means, similar tail behaviour, distance between them small.
11 Dependent Dirichlet Process If G DP(G 0, α), then using the constructive definition of the DP (Sethuraman 1994) G t = w tk δ θtk where (θ tk ) k=1 are iid from some G 0(θ) and (w tk ) is a stick breaking process w tk = z tk (1 z tj ) k=1 j<k z tj Beta(1, α t ) Dependence has been introduced mostly in regression context, conditioning on level of covariates x.
12 Dependence Structure Introduce dependence through the base distributions G 0t of conditionally independent nonparametric priors G t Simple but restrictive dependence only in the atoms of the G t efficient computations but not very flexible approach dependence structure in the weights complex and inefficient computational algorithms, limiting the applicability of the models alternative is to assume G t = π t G + (1 πt )G t
13 Dependence through Weights flexible strategy random prob measures share the same atoms under certain conditions we can approximate any density with any atoms varying the weights can provide prob measures very close (similar weights) or far apart
14 Temporal Dependence Literature: Griffin and Steel (2006), Caron, Davy and Doucet (2007), Rodriguez and ter Horst (2008), Rodriguez and Dunson (2009), Griffin and Steel (2009), Mena and Walker (2009), Nieto-Barajas et al. (2012), Di Lucca et al. (2013), Bassetti et al. (2014), Xiao et al. (2015), Gutirrez, Mena and Ruggiero (2016)... There exist many related constructions for dependent distributions defined through Poisson-Dirichlet Process (e.g. Leisen and Lijoi (2011) and Zhu and Leisen (2015)) or correlated normalized completely random measures (e.g. Griffin et al. 2013, Lijoi et al. 2014) Related approach: Covariate Dependent Random Partion Models Idea: dynamic DP extension can be developed by introducing temporal dependence in the weights through a transformation of the Beta random variables and specifying common atoms across G t.
15 Related Approaches G t = w tk δ θk, k=1 k 1 w tk = ξ tk (1 ξ ti ), ξ tk Beta(1, α) i=1 BAR Stick Breaking Process (Taddy 2010): defines evolution equation for w t : ξ t = (u t v t )ξ t 1 + (1 u t ) Beta(1, α) u t Beta(α, 1 ρ), v t Beta(ρ, 1 ρ) 0 < ρ < 1, u t v t DP marginal, corr(ξ t, ξ t k ) = [ρα/(1 + α ρ)] k > 0 Prior simulations show that number of clusters and number of singletons at time t = 1 is different from other times. Very different clustering can corresponds to relatively similar predictive distributions.
16 Latent Gaussian time-series (DeYoreo and Kottas 2018): ξ t = { exp ζ2 + ηt 2 } Beta(α, 1) 2α ζ N(0, 1) η t η t 1, φ N(φη t 1, 1 φ 2 ), φ < 1 η t AR(1) process. k 1 w t1 = 1 ξ t1, w tk = (1 ξ tk ) α 1 implies a 0.5 lower bound on corr(ξ t, ξ t k ). As a result, same weights (For example, the first weight at time 1 and first weight at time 2) always have correlation above 0.5. η (AR component) is squared so that the correlation between different times is positive. These problems can be overcome by assuming time-varying locations. i=1 ξ ti
17 Latent Autoregressive Process Want: Autoregressive model for a collection of random function G t. Goal: to obtain flexible time-dependent clustering. We borrows ideas from copula literature, in particular from Marginal Beta regression (Guolo & Varin 2014) Recall Probability Integral Transformation: If ɛ N(0, 1) and F is the cdf of a Beta distribution then Y = F 1 (Φ(ɛ); a, b) is marginally distributed as a Beta with parameters (a, b).
18 AR process: ARDP ɛ 1 N(0, 1) ɛ t = ψɛ t 1 + η t, t > 1 η t iid N(0, 1 ψ 2 ) Stick-breaking construction {ɛ tk } iid AR(1), k = 1,..., ξ tk = Ft 1 (Φ(ɛ tk ); a t, b t ) w tk = ξ tk (1 ξ tl ) l<k θ k iid G0 G t = w tk δ θk k=1
19 Comments Easy to generate {G t, t = 1,...}. The marginal distribution of ɛ t is N(0,1) and therefore the marginal distribution of ξ t is Beta with desired parameters. If a t = 1, b t = α, then marginally each G t is DP(α, G 0 ). The {ξ t } inherit the same Markov structure (AR(1)) of the {ɛ t } process, and therefore also the {G t, t = 1, 2,...} is AR(1). Easy to derive the k step predictive densities G t+k as it is easy to derive the weights.
20 We can derive: Evolution of the weights through time The conditional distribution of ξ t ξ t 1 : L(ξ t ξ t 1 ) d = 1 (1 Φ(Z)) 1/α Z N ( ψφ 1 (1 (1 ξ t 1 ) α ), 1 ψ 2). The conditional law of ξ t, given ɛ t 1 : L {ξ t ɛ t 1 } = L { Ft 1 } (Φ(ɛ t ); a, b) ɛ t 1 ɛ t ɛ t 1 N(ψɛ t 1, 1 ψ 2 ) ( ) Φ 1 (1 (1 x) α ) ψɛ t 1 P(ξ t x ɛ t 1 ) = Φ 1 ψ 2
21 L(ξ 2 ξ 1 ) psi=0 psi=0.9 psi= psi=0 psi=0.9 psi= ξ 1 = 0.5 ξ 1 = 0.9 α = 1. Ability of accommodate a variety of distributional shapes over the unit interval.
22 Posterior Inference MCMC based on truncation of DP to L component. Blocked Gibbs sampler (Ishwaran and James 2001) is extended to include a Particle MCMC update (Andrieu et al. 2010) for the weights of the DP. Let s t and w t be the allocation and the weight vector at time t, respectively. We need to sample from p(w 1,..., w T s 1..., s T, ψ) Standard algorithm such as the FFBS are not applicable since 1-step predictive distributions, p ψ (w t w t 1 ), cannot be derived analytically. Employ PMCMC, using the prior to generate samples ɛ t. We exploit SMC approximations: L ψ (w t w t 1 ) = L ψ (ɛ t ɛ t 1 ) p ψ (w 1,..., w T s 1..., s T ) = R ωt r δ ɛ 1:T r=1
23 Simulations T = 4, n = 100, independent time points, y it N. T = 4, n = 100. At t = 1 data generated from 2 clusters of equal size. For t = 2, 3, 4, individuals remain in the same cluster and the distribution of cluster will remain same over time. T = 4, n = 100. At t = 1 data generated from 2 clusters of equal size. For t = 2, 3, 4, individuals remain in the same cluster with probability 50% as t = i 1, and with probability 50%, people will switch to the other cluster. Note that the distribution of each cluster will change over time.
24 Disease Mapping Disease incidence or mortality data are typically available as rates or counts for specified regions, collected over time. Data: breast cancer incidence data of 100 MSA (Metropolitan Statistical Area) from United States Cancer Statistics: Incidence, WONDER Online Database. We use the data from year 2004 to year Population data of year 2000 and year 2010 is obtained from U.S. Census Bureau, Population Division. The population data of the remaining years are estimated by linear regression. Primary goal of the analysis: identification of spatial and spatio-temporal patterns of disease (disease mapping) spatial smoothing and temporal prediction (forecasting) of disease risk.
25 Space-Time Clustering Y it = breast cancer incidence counts (number of cases), in region MSA i at time t N it = the number of individuals at risk R it = disease rate Model: Y kt N it, R it Poisson(N it R it ), i = 1,..., 100; t = 1, ln(r it ) = µ it + φ i µ it G t G t {G t, t 1} ARDP(1) G 0 = N(0, 10) φ C, τ 2, ρ N (0, τ 2 [ρc + (1 ρ)i n ] 1)
26 Spatial Component φ C, τ 2, ρ N (0, τ 2 [ρc + (1 ρ)i n ] 1) variance parameter τ 2 controls the amount of variation between the random effect. the weight parameter ρ controls the strength of the spatial correlation between the random effects the elements of C are equal to n k, if j = k c jk = 1, if j k 0, otherwise. where j k denotes area (j, k) are neighbours and n k denotes the number of neighbours of area (k). Prior specification: τ 2 U(0, 10), ρ discrete uniform(0, 0.05, 0.10, 0.15,..., 0.95)
27 Posterior Inference on Clustering
28 Co-Clustering Probability
29 Posterior Inference on Correlations
30 Dose-escalation Study Data: wbc over time for n = 52 patients receiving high doses of cancer chemotherapy CTX: anticancer agent, known to lower a person s wbc GM-CSF: drugs given to mitigate some of the side-effects of chemotherapy WBC profiles over time: initial baseline sudden decline when chemotherapy starts slow S-shaped recovery back to baseline after end of treatment interest in understanding the effect of dose on wbc in order to protect patients against severe toxicity
31 Model log(y it ) µ it, τ it N(µ it, (λτ it ) 1 ) µ it = m it + x it β t m it, τ it G t G t {G t, t 1} ARDP(1) G 0 NormalGamma(µ 0, λ, α, γ)
32 Posterior Inference on ψ
33 Posterior Inference on Clustering
34 Posterior Density Estimation
35 Conclusions Dependent process for time-evolving distributions based on the DP could be generalised to GDP Introduce temporal dependence through latent stochastic process Normal variables. Advantage more general process on ɛ, e.g ARMA flexible can accommodate different correlation pattern dependent clustering borrowing of information across time-periods general applicability allows for prediction of G t+1 posterior computations through PMCMC
Nonparametric Bayesian modeling for dynamic ordinal regression relationships
Nonparametric Bayesian modeling for dynamic ordinal regression relationships Athanasios Kottas Department of Applied Mathematics and Statistics, University of California, Santa Cruz Joint work with Maria
More informationBayesian Statistics. Debdeep Pati Florida State University. April 3, 2017
Bayesian Statistics Debdeep Pati Florida State University April 3, 2017 Finite mixture model The finite mixture of normals can be equivalently expressed as y i N(µ Si ; τ 1 S i ), S i k π h δ h h=1 δ h
More informationImage segmentation combining Markov Random Fields and Dirichlet Processes
Image segmentation combining Markov Random Fields and Dirichlet Processes Jessica SODJO IMS, Groupe Signal Image, Talence Encadrants : A. Giremus, J.-F. Giovannelli, F. Caron, N. Dobigeon Jessica SODJO
More informationHybrid Dirichlet processes for functional data
Hybrid Dirichlet processes for functional data Sonia Petrone Università Bocconi, Milano Joint work with Michele Guindani - U.T. MD Anderson Cancer Center, Houston and Alan Gelfand - Duke University, USA
More informationFlexible Regression Modeling using Bayesian Nonparametric Mixtures
Flexible Regression Modeling using Bayesian Nonparametric Mixtures Athanasios Kottas Department of Applied Mathematics and Statistics University of California, Santa Cruz Department of Statistics Brigham
More informationSTAT Advanced Bayesian Inference
1 / 32 STAT 625 - Advanced Bayesian Inference Meng Li Department of Statistics Jan 23, 218 The Dirichlet distribution 2 / 32 θ Dirichlet(a 1,...,a k ) with density p(θ 1,θ 2,...,θ k ) = k j=1 Γ(a j) Γ(
More informationOrder-q stochastic processes. Bayesian nonparametric applications
Order-q dependent stochastic processes in Bayesian nonparametric applications Department of Statistics, ITAM, Mexico BNP 2015, Raleigh, NC, USA 25 June, 2015 Contents Order-1 process Application in survival
More informationCompound Random Measures
Compound Random Measures Jim Griffin (joint work with Fabrizio Leisen) University of Kent Introduction: Two clinical studies 3 CALGB8881 3 CALGB916 2 2 β 1 1 β 1 1 1 5 5 β 1 5 5 β Infinite mixture models
More informationNon-Parametric Bayes
Non-Parametric Bayes Mark Schmidt UBC Machine Learning Reading Group January 2016 Current Hot Topics in Machine Learning Bayesian learning includes: Gaussian processes. Approximate inference. Bayesian
More informationBayesian non-parametric model to longitudinally predict churn
Bayesian non-parametric model to longitudinally predict churn Bruno Scarpa Università di Padova Conference of European Statistics Stakeholders Methodologists, Producers and Users of European Statistics
More informationModeling for Dynamic Ordinal Regression Relationships: An. Application to Estimating Maturity of Rockfish in California
Modeling for Dynamic Ordinal Regression Relationships: An Application to Estimating Maturity of Rockfish in California Maria DeYoreo and Athanasios Kottas Abstract We develop a Bayesian nonparametric framework
More informationSlice Sampling Mixture Models
Slice Sampling Mixture Models Maria Kalli, Jim E. Griffin & Stephen G. Walker Centre for Health Services Studies, University of Kent Institute of Mathematics, Statistics & Actuarial Science, University
More informationBayesian Nonparametric Regression through Mixture Models
Bayesian Nonparametric Regression through Mixture Models Sara Wade Bocconi University Advisor: Sonia Petrone October 7, 2013 Outline 1 Introduction 2 Enriched Dirichlet Process 3 EDP Mixtures for Regression
More informationA Fully Nonparametric Modeling Approach to. BNP Binary Regression
A Fully Nonparametric Modeling Approach to Binary Regression Maria Department of Applied Mathematics and Statistics University of California, Santa Cruz SBIES, April 27-28, 2012 Outline 1 2 3 Simulation
More informationParticle Learning for General Mixtures
Particle Learning for General Mixtures Hedibert Freitas Lopes 1 Booth School of Business University of Chicago Dipartimento di Scienze delle Decisioni Università Bocconi, Milano 1 Joint work with Nicholas
More informationOn the Support of MacEachern s Dependent Dirichlet Processes and Extensions
Bayesian Analysis (2012) 7, Number 2, pp. 277 310 On the Support of MacEachern s Dependent Dirichlet Processes and Extensions Andrés F. Barrientos, Alejandro Jara and Fernando A. Quintana Abstract. We
More informationFoundations of Nonparametric Bayesian Methods
1 / 27 Foundations of Nonparametric Bayesian Methods Part II: Models on the Simplex Peter Orbanz http://mlg.eng.cam.ac.uk/porbanz/npb-tutorial.html 2 / 27 Tutorial Overview Part I: Basics Part II: Models
More informationarxiv: v3 [stat.me] 3 May 2016
A Bayesian Nonparametric Markovian Model for Nonstationary Time Series arxiv:1601.04331v3 [stat.me] 3 May 2016 Maria DeYoreo and Athanasios Kottas Abstract Stationary time series models built from parametric
More informationNormalized kernel-weighted random measures
Normalized kernel-weighted random measures Jim Griffin University of Kent 1 August 27 Outline 1 Introduction 2 Ornstein-Uhlenbeck DP 3 Generalisations Bayesian Density Regression We observe data (x 1,
More informationDirichlet Processes: Tutorial and Practical Course
Dirichlet Processes: Tutorial and Practical Course (updated) Yee Whye Teh Gatsby Computational Neuroscience Unit University College London August 2007 / MLSS Yee Whye Teh (Gatsby) DP August 2007 / MLSS
More informationBayesian Point Process Modeling for Extreme Value Analysis, with an Application to Systemic Risk Assessment in Correlated Financial Markets
Bayesian Point Process Modeling for Extreme Value Analysis, with an Application to Systemic Risk Assessment in Correlated Financial Markets Athanasios Kottas Department of Applied Mathematics and Statistics,
More informationAn adaptive truncation method for inference in Bayesian nonparametric models
An adaptive truncation method for inference in Bayesian nonparametric models arxiv:1308.045v [stat.co] 1 May 014 J.E. Griffin School of Mathematics, Statistics and Actuarial Science, University of Kent,
More informationConstruction of Dependent Dirichlet Processes based on Poisson Processes
1 / 31 Construction of Dependent Dirichlet Processes based on Poisson Processes Dahua Lin Eric Grimson John Fisher CSAIL MIT NIPS 2010 Outstanding Student Paper Award Presented by Shouyuan Chen Outline
More informationA Bayesian Nonparametric Approach to Monotone Missing Data in Longitudinal Studies with Informative Missingness
A Bayesian Nonparametric Approach to Monotone Missing Data in Longitudinal Studies with Informative Missingness A. Linero and M. Daniels UF, UT-Austin SRC 2014, Galveston, TX 1 Background 2 Working model
More informationBayesian Nonparametrics: Dirichlet Process
Bayesian Nonparametrics: Dirichlet Process Yee Whye Teh Gatsby Computational Neuroscience Unit, UCL http://www.gatsby.ucl.ac.uk/~ywteh/teaching/npbayes2012 Dirichlet Process Cornerstone of modern Bayesian
More informationA Nonparametric Model for Stationary Time Series
A Nonparametric Model for Stationary Time Series Isadora Antoniano-Villalobos Bocconi University, Milan, Italy. isadora.antoniano@unibocconi.it Stephen G. Walker University of Texas at Austin, USA. s.g.walker@math.utexas.edu
More informationNonparametric Bayes Uncertainty Quantification
Nonparametric Bayes Uncertainty Quantification David Dunson Department of Statistical Science, Duke University Funded from NIH R01-ES017240, R01-ES017436 & ONR Review of Bayes Intro to Nonparametric Bayes
More informationLecture 3a: Dirichlet processes
Lecture 3a: Dirichlet processes Cédric Archambeau Centre for Computational Statistics and Machine Learning Department of Computer Science University College London c.archambeau@cs.ucl.ac.uk Advanced Topics
More informationCSCI 5822 Probabilistic Model of Human and Machine Learning. Mike Mozer University of Colorado
CSCI 5822 Probabilistic Model of Human and Machine Learning Mike Mozer University of Colorado Topics Language modeling Hierarchical processes Pitman-Yor processes Based on work of Teh (2006), A hierarchical
More informationBayesian semiparametric inference for the accelerated failure time model using hierarchical mixture modeling with N-IG priors
Bayesian semiparametric inference for the accelerated failure time model using hierarchical mixture modeling with N-IG priors Raffaele Argiento 1, Alessandra Guglielmi 2, Antonio Pievatolo 1, Fabrizio
More informationNonparametric Bayesian models through probit stick-breaking processes
Bayesian Analysis (2011) 6, Number 1, pp. 145 178 Nonparametric Bayesian models through probit stick-breaking processes Abel Rodríguez and David B. Dunson Abstract. We describe a novel class of Bayesian
More informationSpatio-Temporal Models for Areal Data
Spatio-Temporal Models for Areal Data Juan C. Vivar (jcvivar@dme.ufrj.br) and Marco A. R. Ferreira (marco@im.ufrj.br) Departamento de Métodos Estatísticos - IM Universidade Federal do Rio de Janeiro (UFRJ)
More informationBayesian nonparametrics
Bayesian nonparametrics 1 Some preliminaries 1.1 de Finetti s theorem We will start our discussion with this foundational theorem. We will assume throughout all variables are defined on the probability
More informationA comparative review of variable selection techniques for covariate dependent Dirichlet process mixture models
A comparative review of variable selection techniques for covariate dependent Dirichlet process mixture models William Barcella 1, Maria De Iorio 1 and Gianluca Baio 1 1 Department of Statistical Science,
More informationAnalysing geoadditive regression data: a mixed model approach
Analysing geoadditive regression data: a mixed model approach Institut für Statistik, Ludwig-Maximilians-Universität München Joint work with Ludwig Fahrmeir & Stefan Lang 25.11.2005 Spatio-temporal regression
More informationCPSC 540: Machine Learning
CPSC 540: Machine Learning MCMC and Non-Parametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is
More informationTruncation error of a superposed gamma process in a decreasing order representation
Truncation error of a superposed gamma process in a decreasing order representation Julyan Arbel Inria Grenoble, Université Grenoble Alpes julyan.arbel@inria.fr Igor Prünster Bocconi University, Milan
More informationA Simple Proof of the Stick-Breaking Construction of the Dirichlet Process
A Simple Proof of the Stick-Breaking Construction of the Dirichlet Process John Paisley Department of Computer Science Princeton University, Princeton, NJ jpaisley@princeton.edu Abstract We give a simple
More informationBayesian Nonparametric Inference Methods for Mean Residual Life Functions
Bayesian Nonparametric Inference Methods for Mean Residual Life Functions Valerie Poynor Department of Applied Mathematics and Statistics, University of California, Santa Cruz April 28, 212 1/3 Outline
More informationOutline. Binomial, Multinomial, Normal, Beta, Dirichlet. Posterior mean, MAP, credible interval, posterior distribution
Outline A short review on Bayesian analysis. Binomial, Multinomial, Normal, Beta, Dirichlet Posterior mean, MAP, credible interval, posterior distribution Gibbs sampling Revisit the Gaussian mixture model
More informationLecture 16-17: Bayesian Nonparametrics I. STAT 6474 Instructor: Hongxiao Zhu
Lecture 16-17: Bayesian Nonparametrics I STAT 6474 Instructor: Hongxiao Zhu Plan for today Why Bayesian Nonparametrics? Dirichlet Distribution and Dirichlet Processes. 2 Parameter and Patterns Reference:
More informationIntroduction to Probabilistic Machine Learning
Introduction to Probabilistic Machine Learning Piyush Rai Dept. of CSE, IIT Kanpur (Mini-course 1) Nov 03, 2015 Piyush Rai (IIT Kanpur) Introduction to Probabilistic Machine Learning 1 Machine Learning
More informationModelling geoadditive survival data
Modelling geoadditive survival data Thomas Kneib & Ludwig Fahrmeir Department of Statistics, Ludwig-Maximilians-University Munich 1. Leukemia survival data 2. Structured hazard regression 3. Mixed model
More informationSlice sampling σ stable Poisson Kingman mixture models
ISSN 2279-9362 Slice sampling σ stable Poisson Kingman mixture models Stefano Favaro S.G. Walker No. 324 December 2013 www.carloalberto.org/research/working-papers 2013 by Stefano Favaro and S.G. Walker.
More informationBayesian nonparametric Poisson process modeling with applications
Bayesian nonparametric Poisson process modeling with applications Athanasios Kottas Department of Applied Mathematics and Statistics University of California, Santa Cruz Neyman Seminar Department of Statistics
More informationSequential Monte Carlo Methods for Bayesian Computation
Sequential Monte Carlo Methods for Bayesian Computation A. Doucet Kyoto Sept. 2012 A. Doucet (MLSS Sept. 2012) Sept. 2012 1 / 136 Motivating Example 1: Generic Bayesian Model Let X be a vector parameter
More informationSpatial modeling for risk assessment of extreme values from environmental time series: A Bayesian nonparametric approach
Spatial modeling for risk assessment of extreme values from environmental time series: A Bayesian nonparametric approach Athanasios Kottas, Ziwei Wang and Abel Rodríguez Department of Applied Mathematics
More informationSmall-variance Asymptotics for Dirichlet Process Mixtures of SVMs
Small-variance Asymptotics for Dirichlet Process Mixtures of SVMs Yining Wang Jun Zhu Tsinghua University July, 2014 Y. Wang and J. Zhu (Tsinghua University) Max-Margin DP-means July, 2014 1 / 25 Outline
More informationChapter 3. Dirichlet Process
Chapter 3 Dirichlet Process 3.1. The Dirichlet Process Prior 3.1.1. Definition The Dirichlet process (DP) is arguably the most popular BNP model for random probability measures (RPM), and plays a central
More informationSTA 216, GLM, Lecture 16. October 29, 2007
STA 216, GLM, Lecture 16 October 29, 2007 Efficient Posterior Computation in Factor Models Underlying Normal Models Generalized Latent Trait Models Formulation Genetic Epidemiology Illustration Structural
More informationBayes methods for categorical data. April 25, 2017
Bayes methods for categorical data April 25, 2017 Motivation for joint probability models Increasing interest in high-dimensional data in broad applications Focus may be on prediction, variable selection,
More informationModeling conditional distributions with mixture models: Applications in finance and financial decision-making
Modeling conditional distributions with mixture models: Applications in finance and financial decision-making John Geweke University of Iowa, USA Journal of Applied Econometrics Invited Lecture Università
More informationHierarchical Bayesian Languge Model Based on Pitman-Yor Processes. Yee Whye Teh
Hierarchical Bayesian Languge Model Based on Pitman-Yor Processes Yee Whye Teh Probabilistic model of language n-gram model Utility i-1 P(word i word i-n+1 ) Typically, trigram model (n=3) e.g., speech,
More informationBayesian Inference on Joint Mixture Models for Survival-Longitudinal Data with Multiple Features. Yangxin Huang
Bayesian Inference on Joint Mixture Models for Survival-Longitudinal Data with Multiple Features Yangxin Huang Department of Epidemiology and Biostatistics, COPH, USF, Tampa, FL yhuang@health.usf.edu January
More informationA marginal sampler for σ-stable Poisson-Kingman mixture models
A marginal sampler for σ-stable Poisson-Kingman mixture models joint work with Yee Whye Teh and Stefano Favaro María Lomelí Gatsby Unit, University College London Talk at the BNP 10 Raleigh, North Carolina
More informationModelling and computation using NCoRM mixtures for. density regression
Modelling and computation using NCoRM mixtures for density regression arxiv:168.874v3 [stat.me] 31 Aug 217 Jim E. Griffin and Fabrizio Leisen University of Kent Abstract Normalized compound random measures
More informationBayesian Methods for Machine Learning
Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),
More informationGibbs Sampling in Endogenous Variables Models
Gibbs Sampling in Endogenous Variables Models Econ 690 Purdue University Outline 1 Motivation 2 Identification Issues 3 Posterior Simulation #1 4 Posterior Simulation #2 Motivation In this lecture we take
More informationBayesian semiparametric modeling for stochastic precedence, with applications in epidemiology and survival analysis
Bayesian semiparametric modeling for stochastic precedence, with applications in epidemiology and survival analysis ATHANASIOS KOTTAS Department of Applied Mathematics and Statistics, University of California,
More informationHierarchical models. Dr. Jarad Niemi. August 31, Iowa State University. Jarad Niemi (Iowa State) Hierarchical models August 31, / 31
Hierarchical models Dr. Jarad Niemi Iowa State University August 31, 2017 Jarad Niemi (Iowa State) Hierarchical models August 31, 2017 1 / 31 Normal hierarchical model Let Y ig N(θ g, σ 2 ) for i = 1,...,
More informationMotivation Scale Mixutres of Normals Finite Gaussian Mixtures Skew-Normal Models. Mixture Models. Econ 690. Purdue University
Econ 690 Purdue University In virtually all of the previous lectures, our models have made use of normality assumptions. From a computational point of view, the reason for this assumption is clear: combined
More informationBayesian Hidden Markov Models and Extensions
Bayesian Hidden Markov Models and Extensions Zoubin Ghahramani Department of Engineering University of Cambridge joint work with Matt Beal, Jurgen van Gael, Yunus Saatci, Tom Stepleton, Yee Whye Teh Modeling
More informationBAYESIAN NONPARAMETRIC MODELLING WITH THE DIRICHLET PROCESS REGRESSION SMOOTHER
Statistica Sinica 20 (2010), 1507-1527 BAYESIAN NONPARAMETRIC MODELLING WITH THE DIRICHLET PROCESS REGRESSION SMOOTHER J. E. Griffin and M. F. J. Steel University of Kent and University of Warwick Abstract:
More informationMixture Modeling for Marked Poisson Processes
Mixture Modeling for Marked Poisson Processes Matthew A. Taddy taddy@chicagobooth.edu The University of Chicago Booth School of Business 5807 South Woodlawn Ave, Chicago, IL 60637, USA Athanasios Kottas
More informationA Review of Pseudo-Marginal Markov Chain Monte Carlo
A Review of Pseudo-Marginal Markov Chain Monte Carlo Discussed by: Yizhe Zhang October 21, 2016 Outline 1 Overview 2 Paper review 3 experiment 4 conclusion Motivation & overview Notation: θ denotes the
More informationDensity Estimation. Seungjin Choi
Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More informationVariational Bayesian Dirichlet-Multinomial Allocation for Exponential Family Mixtures
17th Europ. Conf. on Machine Learning, Berlin, Germany, 2006. Variational Bayesian Dirichlet-Multinomial Allocation for Exponential Family Mixtures Shipeng Yu 1,2, Kai Yu 2, Volker Tresp 2, and Hans-Peter
More informationDisease mapping with Gaussian processes
EUROHEIS2 Kuopio, Finland 17-18 August 2010 Aki Vehtari (former Helsinki University of Technology) Department of Biomedical Engineering and Computational Science (BECS) Acknowledgments Researchers - Jarno
More informationNonparametric Bayes tensor factorizations for big data
Nonparametric Bayes tensor factorizations for big data David Dunson Department of Statistical Science, Duke University Funded from NIH R01-ES017240, R01-ES017436 & DARPA N66001-09-C-2082 Motivation Conditional
More informationBayesian semiparametric analysis of short- and long- term hazard ratios with covariates
Bayesian semiparametric analysis of short- and long- term hazard ratios with covariates Department of Statistics, ITAM, Mexico 9th Conference on Bayesian nonparametrics Amsterdam, June 10-14, 2013 Contents
More informationBayesian Nonparametrics: some contributions to construction and properties of prior distributions
Bayesian Nonparametrics: some contributions to construction and properties of prior distributions Annalisa Cerquetti Collegio Nuovo, University of Pavia, Italy Interview Day, CETL Lectureship in Statistics,
More informationA Bayesian Nonparametric Approach to Causal Inference for Semi-competing risks
A Bayesian Nonparametric Approach to Causal Inference for Semi-competing risks Y. Xu, D. Scharfstein, P. Mueller, M. Daniels Johns Hopkins, Johns Hopkins, UT-Austin, UF JSM 2018, Vancouver 1 What are semi-competing
More informationColouring and breaking sticks, pairwise coincidence losses, and clustering expression profiles
Colouring and breaking sticks, pairwise coincidence losses, and clustering expression profiles Peter Green and John Lau University of Bristol P.J.Green@bristol.ac.uk Isaac Newton Institute, 11 December
More informationModeling conditional distributions with mixture models: Theory and Inference
Modeling conditional distributions with mixture models: Theory and Inference John Geweke University of Iowa, USA Journal of Applied Econometrics Invited Lecture Università di Venezia Italia June 2, 2005
More informationBayesian spatial quantile regression
Brian J. Reich and Montserrat Fuentes North Carolina State University and David B. Dunson Duke University E-mail:reich@stat.ncsu.edu Tropospheric ozone Tropospheric ozone has been linked with several adverse
More informationOn Simulations form the Two-Parameter. Poisson-Dirichlet Process and the Normalized. Inverse-Gaussian Process
On Simulations form the Two-Parameter arxiv:1209.5359v1 [stat.co] 24 Sep 2012 Poisson-Dirichlet Process and the Normalized Inverse-Gaussian Process Luai Al Labadi and Mahmoud Zarepour May 8, 2018 ABSTRACT
More informationMarkov Chain Monte Carlo methods
Markov Chain Monte Carlo methods Tomas McKelvey and Lennart Svensson Signal Processing Group Department of Signals and Systems Chalmers University of Technology, Sweden November 26, 2012 Today s learning
More informationA Nonparametric Model-based Approach to Inference for Quantile Regression
A Nonparametric Model-based Approach to Inference for Quantile Regression Matthew Taddy and Athanasios Kottas Department of Applied Mathematics and Statistics, University of California, Santa Cruz, CA
More informationDependent mixture models: clustering and borrowing information
ISSN 2279-9362 Dependent mixture models: clustering and borrowing information Antonio Lijoi Bernardo Nipoti Igor Pruenster No. 32 June 213 www.carloalberto.org/research/working-papers 213 by Antonio Lijoi,
More informationBayesian Nonparametric Spatio-Temporal Models for Disease Incidence Data
Bayesian Nonparametric Spatio-Temporal Models for Disease Incidence Data Athanasios Kottas 1, Jason A. Duan 2, Alan E. Gelfand 3 University of California, Santa Cruz 1 Yale School of Management 2 Duke
More informationOn the posterior structure of NRMI
On the posterior structure of NRMI Igor Prünster University of Turin, Collegio Carlo Alberto and ICER Joint work with L.F. James and A. Lijoi Isaac Newton Institute, BNR Programme, 8th August 2007 Outline
More informationAn ANOVA Model for Dependent Random Measures
JASA jasa v.2003/10/20 Prn:15/01/2004; 14:10 F:jasatm02341r2.tex; (Lina) p. 1 An ANOVA Model for Dependent Random Measures Maria DE IORIO, Peter MÜLLER, GaryL.ROSNER, and Steven N. MACEACHERN We consider
More informationarxiv: v1 [math.st] 28 Feb 2015
Are Gibbs type priors the most natural generalization of the Dirichlet process? P. De Blasi 1, S. Favaro 1, A. Lijoi 2, R.H. Mena 3, I. Prünster 1 and M. Ruggiero 1 1 Università degli Studi di Torino and
More informationCollapsed Variational Dirichlet Process Mixture Models
Collapsed Variational Dirichlet Process Mixture Models Kenichi Kurihara Dept. of Computer Science Tokyo Institute of Technology, Japan kurihara@mi.cs.titech.ac.jp Max Welling Dept. of Computer Science
More informationChapter 2. Data Analysis
Chapter 2 Data Analysis 2.1. Density Estimation and Survival Analysis The most straightforward application of BNP priors for statistical inference is in density estimation problems. Consider the generic
More informationBayesian Estimation of DSGE Models 1 Chapter 3: A Crash Course in Bayesian Inference
1 The views expressed in this paper are those of the authors and do not necessarily reflect the views of the Federal Reserve Board of Governors or the Federal Reserve System. Bayesian Estimation of DSGE
More informationBayesian covariate models in extreme value analysis
Bayesian covariate models in extreme value analysis David Randell, Philip Jonathan, Kathryn Turnbull, Mathew Jones EVA 2015 Ann Arbor Copyright 2015 Shell Global Solutions (UK) EVA 2015 Ann Arbor June
More informationSemiparametric Bayesian Inference for. Multilevel Repeated Measurement Data
Semiparametric Bayesian Inference for Multilevel Repeated Measurement Data Peter Müller, Fernando A. Quintana and Gary L. Rosner Department of Biostatistics & Applied Mathematics, The University of Texas,
More informationProbabilistic Graphical Models
School of Computer Science Probabilistic Graphical Models Infinite Feature Models: The Indian Buffet Process Eric Xing Lecture 21, April 2, 214 Acknowledgement: slides first drafted by Sinead Williamson
More informationLatent Variable Models for Binary Data. Suppose that for a given vector of explanatory variables x, the latent
Latent Variable Models for Binary Data Suppose that for a given vector of explanatory variables x, the latent variable, U, has a continuous cumulative distribution function F (u; x) and that the binary
More informationBayesian Hierarchical Models
Bayesian Hierarchical Models Gavin Shaddick, Millie Green, Matthew Thomas University of Bath 6 th - 9 th December 2016 1/ 34 APPLICATIONS OF BAYESIAN HIERARCHICAL MODELS 2/ 34 OUTLINE Spatial epidemiology
More informationDefault priors for density estimation with mixture models
Bayesian Analysis ) 5, Number, pp. 45 64 Default priors for density estimation with mixture models J.E. Griffin Abstract. The infinite mixture of normals model has become a popular method for density estimation
More informationMarkov Chain Monte Carlo (MCMC)
Markov Chain Monte Carlo (MCMC Dependent Sampling Suppose we wish to sample from a density π, and we can evaluate π as a function but have no means to directly generate a sample. Rejection sampling can
More informationA Bayesian Nonparametric Model for Predicting Disease Status Using Longitudinal Profiles
A Bayesian Nonparametric Model for Predicting Disease Status Using Longitudinal Profiles Jeremy Gaskins Department of Bioinformatics & Biostatistics University of Louisville Joint work with Claudio Fuentes
More informationPenalized Loss functions for Bayesian Model Choice
Penalized Loss functions for Bayesian Model Choice Martyn International Agency for Research on Cancer Lyon, France 13 November 2009 The pure approach For a Bayesian purist, all uncertainty is represented
More informationMultivariate spatial modeling
Multivariate spatial modeling Point-referenced spatial data often come as multivariate measurements at each location Chapter 7: Multivariate Spatial Modeling p. 1/21 Multivariate spatial modeling Point-referenced
More informationCMPS 242: Project Report
CMPS 242: Project Report RadhaKrishna Vuppala Univ. of California, Santa Cruz vrk@soe.ucsc.edu Abstract The classification procedures impose certain models on the data and when the assumption match the
More informationBayesian Nonparametric Modelling with the Dirichlet Process Regression Smoother
Bayesian Nonparametric Modelling with the Dirichlet Process Regression Smoother J.E. Griffin and M. F. J. Steel University of Kent and University of Warwick Abstract In this paper we discuss implementing
More informationVCMC: Variational Consensus Monte Carlo
VCMC: Variational Consensus Monte Carlo Maxim Rabinovich, Elaine Angelino, Michael I. Jordan Berkeley Vision and Learning Center September 22, 2015 probabilistic models! sky fog bridge water grass object
More informationGaussian Multiscale Spatio-temporal Models for Areal Data
Gaussian Multiscale Spatio-temporal Models for Areal Data (University of Missouri) Scott H. Holan (University of Missouri) Adelmo I. Bertolde (UFES) Outline Motivation Multiscale factorization The multiscale
More information