Bayesian nonparametric latent feature models
|
|
- Christal Poppy Baldwin
- 5 years ago
- Views:
Transcription
1 Bayesian nonparametric latent feature models François Caron UBC October 2, 2007 / MLRG François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 1 / 29
2 Overview 1 Introduction 2 Dirichlet Process Finite mixture model Equivalent classes Dirichlet Process model Chinese Restaurant Stick-breaking construction 3 Nonparametric latent feature models Finite feature model Equivalent classes Nonparametric latent feature model Indian buffet Stick-breaking construction 4 Applications François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 2 / 29
3 Overview 1 Introduction 2 Dirichlet Process Finite mixture model Equivalent classes Dirichlet Process model Chinese Restaurant Stick-breaking construction 3 Nonparametric latent feature models Finite feature model Equivalent classes Nonparametric latent feature model Indian buffet Stick-breaking construction 4 Applications François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 3 / 29
4 Finite mixture model n objects and K clusters Each object k = 1,..., n can belong to only one cluster j {1,..., K} Represented by an allocation variable c k {1,..., K} François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 4 / 29
5 Finite mixture model Bayesian approach: Dirichlet-multinomial model π 1:K D( α K,..., α K ) and for k = 1,..., n, K : Dirichlet Process c k π 1:K François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 5 / 29
6 Finite latent feature model n objects and K features Each object k can have a finite number of latent features Represented by a binary vector c k {0, 1} K François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 6 / 29
7 Finite latent feature model Bayesian approach: Prior distribution over a binary matrix of size n K K : Nonparametric latent feature model François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 7 / 29
8 Overview 1 Introduction 2 Dirichlet Process Finite mixture model Equivalent classes Dirichlet Process model Chinese Restaurant Stick-breaking construction 3 Nonparametric latent feature models Finite feature model Equivalent classes Nonparametric latent feature model Indian buffet Stick-breaking construction 4 Applications François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 8 / 29
9 Finite mixture model Hierarchical model For j = 1,..., K, For k = 1,..., n, π 1:K D( α K,..., α K ) U j G 0 c k π 1:K z k c k, U 1:K f ( U ck ) François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 9 / 29
10 Finite mixture model Prior distribution Pr(π 1:K, U 1:K, c 1:n ) = Pr(π 1:K, c 1:n ) Pr(U 1:K ) Can integrate out analytically π 1:K (Dirichlet-multinomial distribution) Pr(c 1:n ) = Pr(π 1:K, c 1:n )dπ 1:K = Γ(α) K j=1 Γ(n j + α K ) Γ(α + n)γ( α K )K François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 10 / 29
11 Distribution over partitions Let Π n = {A 1,..., A n(πn)} be a random partition of {1,..., n} and P n the set of partitions of {1,..., n} Ex: Π 5 = {{1, 2, 5}, {3}, {4}} Several different values of c 1:n may induce the same partition Ex: c 1:5 = (1, 1, 2, 1, 3) and c 1:5 = (2, 2, 3, 2, 1) both induce the same partition Π 5 = {{1, 2, 4}, {3}, {5}} Let Π n (c 1:n ) be the partition of {1,..., n} induced by the equivalence relationship i j c i = c j François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 11 / 29
12 Distribution over partitions We have Pr(Π n (c 1:n )) = K! (K n(π n ))! Pr(c 1:n) for all Π n P K where P K = {Π n P n n(π n ) K} and thus for the Dirichlet-multinomial distribution Pr(Π n ) = K! Γ(α) K j=1 Γ(n j + α K ) (K n(π n ))! Γ(α + n)γ( α K )K François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 12 / 29
13 Dirichlet Process Limit when K of the finite model Pr(Π n ) = K! Γ(α) K j=1 Γ(n j + α K ) (K n(π n ))! Γ(α + n)γ( α K )K is given by Pr(Π n ) = αn(πn) n(π n) j=1 Γ(n j ) (α + i 1) n i=1 François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 13 / 29
14 Dirichlet Process Connexion to the usual formulation for k = 1,..., n G DP(α, G 0 ) θ k G G Let Π n (θ 1...., θ n ) be the partition of {1,..., n} induced by the equivalence relationship i j θ i = θ j and U 1,..., U n(πn) be the different values taken by θ 1,..., θ n When integrating out the unknown distribution G n(π n) Pr(θ 1,..., θ n ) = Pr(Π n (θ 1...., θ n )) G 0 (U j ) j=1 François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 14 / 29
15 Chinese Restaurant Let Π k be the partition obtained by removing item k from Π n Conditional association probability of item k is Pr(Π n ) Pr(Π k ) Item k is associated to an existing cluster j = 1,..., n(π k ) with probability n j, k n 1 + α and to a new cluster with probability α n 1 + α François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 15 / 29
16 Stick-breaking representation Let Π 1, Π 2,... be the successive partitions obtained with sequential CRP updates Ex: Π 1 = {{1}}, Π 2 = {{1}, {2}}, Π 3 = {{1, 3}, {2}}, Π 4 = {{1, 3}, {2}, {4}},... Let n j,t the size of cluster j in Π t, where the clusters are numbered in order of appearance Let lim t n j,t t where = π j. We have j 1 π j = β j (1 β i ) i=1 β j B(1, α) Stick-breaking construction or residual allocation model François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 16 / 29
17 Overview 1 Introduction 2 Dirichlet Process Finite mixture model Equivalent classes Dirichlet Process model Chinese Restaurant Stick-breaking construction 3 Nonparametric latent feature models Finite feature model Equivalent classes Nonparametric latent feature model Indian buffet Stick-breaking construction 4 Applications François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 17 / 29
18 Finite feature model Assume the following model For each feature j = 1,..., K, π j B( α K, 1) For each object k and each feature j c k,j Ber(π j ) François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 18 / 29
19 Finite feature model Integrating out π 1:K Pr(c 1:n ) = K j=1 α K Γ(m j + α K )Γ(n m j + 1) Γ(n α K ) Invariant to permutation of objects/features François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 19 / 29
20 Equivalent classes Distribution over binary matrix is invariant with respect to permutations of columns Left-ordered form Pr(ξ(c 1:n )) = K! 2 n 1 h=0 K h! K j=1 α K Γ(m j + α K )Γ(n m j + 1) Γ(n α K ) François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 20 / 29
21 Nonparametric latent feature model Limit when K of the finite model is given by Pr(ξ(c 1:n )) = Pr(ξ(c 1:n )) = K! 2 n 1 h=0 K h! K + 2 n 1 K j=1 h=1 K h! exp( αh n) α K Γ(m j + α K )Γ(n m j + 1) Γ(n α K ) K + j=1 (n m j )!(m j 1)! n! François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 21 / 29
22 Indian Buffet Buffet with infinite number of dishes First customer samples Poisson(α) dishes k eme customer takes each dish with probability m j /k and tries Poisson(α/k) new dishes Caution: Not in left-ordered form! François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 22 / 29
23 Indian Buffet François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 23 / 29
24 Properties Total number of different features K + follows Poisson(α( n k=1 1 k )) Number of features possessed by each object follows Poisson(α) Total number of entries in the matrix follows Poisson(nα) François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 24 / 29
25 Stick-breaking construction Let π (1) > π (2) >... > π (K) be a decreasing ordering of π 1:K where π k B( α K, 1) Limit when K : stick-breaking contruction β (k) B(α, 1) and π (k) = π (k 1) β (k) DP: stick lengths sum to one and are not decreasing (only in average) IBP: stick lengths does not sum to one and are decreasing François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 25 / 29
26 Overview 1 Introduction 2 Dirichlet Process Finite mixture model Equivalent classes Dirichlet Process model Chinese Restaurant Stick-breaking construction 3 Nonparametric latent feature models Finite feature model Equivalent classes Nonparametric latent feature model Indian buffet Stick-breaking construction 4 Applications François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 26 / 29
27 Applications Choice behaviour Protein interaction screens Structure of causal graphs François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 27 / 29
28 Bibliography Random partitions and Dirichlet Processes J. Pitman. Exchangeable and partially exchangeable random partitions Probability Theory and Related Fields, J. Pitman. Some developments of the Blackwell-MacQueen urn scheme In Statistics, probability and game theory; paper in honour of David Blackwell, Indian buffet Z. Ghahramani, T.L. Griffiths and P. Sollich. Bayesian nonparametric latent feature models Proc. Valencia / ISBA 8th World Meeting on Bayesian Statistics, T.L. Griffiths and Z. Ghahramani. Infinite latent feature models and the Indian Buffet Process Technical Report, University College, London, Y.W. Teh, D. Gorur and Z. Ghahramani. Stick-breaking construction for the Indian Buffet Process AISTATS, François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 28 / 29
29 Bibliography Applications of Indian buffet D. Gorur, F. Jakel and C. Rasmussen. A choice model with infinitely many latent features. ICML W. Chu, Z. Ghahramani, R. Krause and D.L. Wild. Identifying protein complexes in high-throughput protein interactions screens using an infinite latent feature model. Biocomputing F. Wood, T.L. Griffiths and Z. Ghahramani. A nonparametric Bayesian method for inferring hidden causes. UAI Still hungry? M. McGlohon. Fried Chicken Bucket Processes SIGBOVIK, Pittsburgh, François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 29 / 29
Haupthseminar: Machine Learning. Chinese Restaurant Process, Indian Buffet Process
Haupthseminar: Machine Learning Chinese Restaurant Process, Indian Buffet Process Agenda Motivation Chinese Restaurant Process- CRP Dirichlet Process Interlude on CRP Infinite and CRP mixture model Estimation
More informationBayesian Nonparametric Models on Decomposable Graphs
Bayesian Nonparametric Models on Decomposable Graphs François Caron INRIA Bordeaux Sud Ouest Institut de Mathématiques de Bordeaux University of Bordeaux, France francois.caron@inria.fr Arnaud Doucet Departments
More informationBayesian nonparametric latent feature models
Bayesian nonparametric latent feature models Indian Buffet process, beta process, and related models François Caron Department of Statistics, Oxford Applied Bayesian Statistics Summer School Como, Italy
More informationInfinite latent feature models and the Indian Buffet Process
p.1 Infinite latent feature models and the Indian Buffet Process Tom Griffiths Cognitive and Linguistic Sciences Brown University Joint work with Zoubin Ghahramani p.2 Beyond latent classes Unsupervised
More informationLecture 3a: Dirichlet processes
Lecture 3a: Dirichlet processes Cédric Archambeau Centre for Computational Statistics and Machine Learning Department of Computer Science University College London c.archambeau@cs.ucl.ac.uk Advanced Topics
More informationNon-Parametric Bayes
Non-Parametric Bayes Mark Schmidt UBC Machine Learning Reading Group January 2016 Current Hot Topics in Machine Learning Bayesian learning includes: Gaussian processes. Approximate inference. Bayesian
More informationA Brief Overview of Nonparametric Bayesian Models
A Brief Overview of Nonparametric Bayesian Models Eurandom Zoubin Ghahramani Department of Engineering University of Cambridge, UK zoubin@eng.cam.ac.uk http://learning.eng.cam.ac.uk/zoubin Also at Machine
More informationThe Indian Buffet Process: An Introduction and Review
Journal of Machine Learning Research 12 (2011) 1185-1224 Submitted 3/10; Revised 3/11; Published 4/11 The Indian Buffet Process: An Introduction and Review Thomas L. Griffiths Department of Psychology
More information19 : Bayesian Nonparametrics: The Indian Buffet Process. 1 Latent Variable Models and the Indian Buffet Process
10-708: Probabilistic Graphical Models, Spring 2015 19 : Bayesian Nonparametrics: The Indian Buffet Process Lecturer: Avinava Dubey Scribes: Rishav Das, Adam Brodie, and Hemank Lamba 1 Latent Variable
More informationDirichlet Processes and other non-parametric Bayesian models
Dirichlet Processes and other non-parametric Bayesian models Zoubin Ghahramani http://learning.eng.cam.ac.uk/zoubin/ zoubin@cs.cmu.edu Statistical Machine Learning CMU 10-702 / 36-702 Spring 2008 Model
More informationFactor Analysis and Indian Buffet Process
Factor Analysis and Indian Buffet Process Lecture 4 Peixian Chen pchenac@cse.ust.hk Department of Computer Science and Engineering Hong Kong University of Science and Technology March 25, 2013 Peixian
More informationProbabilistic Graphical Models
School of Computer Science Probabilistic Graphical Models Infinite Feature Models: The Indian Buffet Process Eric Xing Lecture 21, April 2, 214 Acknowledgement: slides first drafted by Sinead Williamson
More informationBayesian Nonparametrics: Dirichlet Process
Bayesian Nonparametrics: Dirichlet Process Yee Whye Teh Gatsby Computational Neuroscience Unit, UCL http://www.gatsby.ucl.ac.uk/~ywteh/teaching/npbayes2012 Dirichlet Process Cornerstone of modern Bayesian
More informationNon-parametric Clustering with Dirichlet Processes
Non-parametric Clustering with Dirichlet Processes Timothy Burns SUNY at Buffalo Mar. 31 2009 T. Burns (SUNY at Buffalo) Non-parametric Clustering with Dirichlet Processes Mar. 31 2009 1 / 24 Introduction
More informationBayesian Nonparametrics
Bayesian Nonparametrics Lorenzo Rosasco 9.520 Class 18 April 11, 2011 About this class Goal To give an overview of some of the basic concepts in Bayesian Nonparametrics. In particular, to discuss Dirichelet
More informationDirichlet Processes: Tutorial and Practical Course
Dirichlet Processes: Tutorial and Practical Course (updated) Yee Whye Teh Gatsby Computational Neuroscience Unit University College London August 2007 / MLSS Yee Whye Teh (Gatsby) DP August 2007 / MLSS
More informationNon-parametric Bayesian Methods
Non-parametric Bayesian Methods Uncertainty in Artificial Intelligence Tutorial July 25 Zoubin Ghahramani Gatsby Computational Neuroscience Unit University College London, UK Center for Automated Learning
More informationBayesian Nonparametrics for Speech and Signal Processing
Bayesian Nonparametrics for Speech and Signal Processing Michael I. Jordan University of California, Berkeley June 28, 2011 Acknowledgments: Emily Fox, Erik Sudderth, Yee Whye Teh, and Romain Thibaux Computer
More informationInfinite Latent Feature Models and the Indian Buffet Process
Infinite Latent Feature Models and the Indian Buffet Process Thomas L. Griffiths Cognitive and Linguistic Sciences Brown University, Providence RI 292 tom griffiths@brown.edu Zoubin Ghahramani Gatsby Computational
More informationNonparametric Bayesian Models for Sparse Matrices and Covariances
Nonparametric Bayesian Models for Sparse Matrices and Covariances Zoubin Ghahramani Department of Engineering University of Cambridge, UK zoubin@eng.cam.ac.uk http://learning.eng.cam.ac.uk/zoubin/ Bayes
More informationBayesian non parametric approaches: an introduction
Introduction Latent class models Latent feature models Conclusion & Perspectives Bayesian non parametric approaches: an introduction Pierre CHAINAIS Bordeaux - nov. 2012 Trajectory 1 Bayesian non parametric
More informationBayesian nonparametric models for bipartite graphs
Bayesian nonparametric models for bipartite graphs François Caron Department of Statistics, Oxford Statistics Colloquium, Harvard University November 11, 2013 F. Caron 1 / 27 Bipartite networks Readers/Customers
More informationAn Infinite Product of Sparse Chinese Restaurant Processes
An Infinite Product of Sparse Chinese Restaurant Processes Yarin Gal Tomoharu Iwata Zoubin Ghahramani yg279@cam.ac.uk CRP quick recap The Chinese restaurant process (CRP) Distribution over partitions of
More informationMAD-Bayes: MAP-based Asymptotic Derivations from Bayes
MAD-Bayes: MAP-based Asymptotic Derivations from Bayes Tamara Broderick Brian Kulis Michael I. Jordan Cat Clusters Mouse clusters Dog 1 Cat Clusters Dog Mouse Lizard Sheep Picture 1 Picture 2 Picture 3
More informationNonparametric Probabilistic Modelling
Nonparametric Probabilistic Modelling Zoubin Ghahramani Department of Engineering University of Cambridge, UK zoubin@eng.cam.ac.uk http://learning.eng.cam.ac.uk/zoubin/ Signal processing and inference
More informationHierarchical Models, Nested Models and Completely Random Measures
See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/238729763 Hierarchical Models, Nested Models and Completely Random Measures Article March 2012
More informationPriors for Random Count Matrices with Random or Fixed Row Sums
Priors for Random Count Matrices with Random or Fixed Row Sums Mingyuan Zhou Joint work with Oscar Madrid and James Scott IROM Department, McCombs School of Business Department of Statistics and Data Sciences
More informationLecture 16-17: Bayesian Nonparametrics I. STAT 6474 Instructor: Hongxiao Zhu
Lecture 16-17: Bayesian Nonparametrics I STAT 6474 Instructor: Hongxiao Zhu Plan for today Why Bayesian Nonparametrics? Dirichlet Distribution and Dirichlet Processes. 2 Parameter and Patterns Reference:
More informationApplied Nonparametric Bayes
Applied Nonparametric Bayes Michael I. Jordan Department of Electrical Engineering and Computer Science Department of Statistics University of California, Berkeley http://www.cs.berkeley.edu/ jordan Acknowledgments:
More informationBayesian Hidden Markov Models and Extensions
Bayesian Hidden Markov Models and Extensions Zoubin Ghahramani Department of Engineering University of Cambridge joint work with Matt Beal, Jurgen van Gael, Yunus Saatci, Tom Stepleton, Yee Whye Teh Modeling
More informationDirichlet Process. Yee Whye Teh, University College London
Dirichlet Process Yee Whye Teh, University College London Related keywords: Bayesian nonparametrics, stochastic processes, clustering, infinite mixture model, Blackwell-MacQueen urn scheme, Chinese restaurant
More informationAn Introduction to Bayesian Nonparametric Modelling
An Introduction to Bayesian Nonparametric Modelling Yee Whye Teh Gatsby Computational Neuroscience Unit University College London October, 2009 / Toronto Outline Some Examples of Parametric Models Bayesian
More informationBayesian Nonparametrics: Models Based on the Dirichlet Process
Bayesian Nonparametrics: Models Based on the Dirichlet Process Alessandro Panella Department of Computer Science University of Illinois at Chicago Machine Learning Seminar Series February 18, 2013 Alessandro
More informationBayesian Nonparametric Models
Bayesian Nonparametric Models David M. Blei Columbia University December 15, 2015 Introduction We have been looking at models that posit latent structure in high dimensional data. We use the posterior
More informationFeature Allocations, Probability Functions, and Paintboxes
Feature Allocations, Probability Functions, and Paintboxes Tamara Broderick, Jim Pitman, Michael I. Jordan Abstract The problem of inferring a clustering of a data set has been the subject of much research
More informationSTAT Advanced Bayesian Inference
1 / 32 STAT 625 - Advanced Bayesian Inference Meng Li Department of Statistics Jan 23, 218 The Dirichlet distribution 2 / 32 θ Dirichlet(a 1,...,a k ) with density p(θ 1,θ 2,...,θ k ) = k j=1 Γ(a j) Γ(
More informationCS281B / Stat 241B : Statistical Learning Theory Lecture: #22 on 19 Apr Dirichlet Process I
X i Ν CS281B / Stat 241B : Statistical Learning Theory Lecture: #22 on 19 Apr 2004 Dirichlet Process I Lecturer: Prof. Michael Jordan Scribe: Daniel Schonberg dschonbe@eecs.berkeley.edu 22.1 Dirichlet
More informationCollapsed Variational Dirichlet Process Mixture Models
Collapsed Variational Dirichlet Process Mixture Models Kenichi Kurihara Dept. of Computer Science Tokyo Institute of Technology, Japan kurihara@mi.cs.titech.ac.jp Max Welling Dept. of Computer Science
More informationA Stick-Breaking Construction of the Beta Process
John Paisley 1 jwp4@ee.duke.edu Aimee Zaas 2 aimee.zaas@duke.edu Christopher W. Woods 2 woods004@mc.duke.edu Geoffrey S. Ginsburg 2 ginsb005@duke.edu Lawrence Carin 1 lcarin@ee.duke.edu 1 Department of
More informationBayesian nonparametrics
Bayesian nonparametrics 1 Some preliminaries 1.1 de Finetti s theorem We will start our discussion with this foundational theorem. We will assume throughout all variables are defined on the probability
More informationStochastic Processes, Kernel Regression, Infinite Mixture Models
Stochastic Processes, Kernel Regression, Infinite Mixture Models Gabriel Huang (TA for Simon Lacoste-Julien) IFT 6269 : Probabilistic Graphical Models - Fall 2018 Stochastic Process = Random Function 2
More informationBayesian Nonparametrics
Bayesian Nonparametrics Peter Orbanz Columbia University PARAMETERS AND PATTERNS Parameters P(X θ) = Probability[data pattern] 3 2 1 0 1 2 3 5 0 5 Inference idea data = underlying pattern + independent
More informationSpatial Bayesian Nonparametrics for Natural Image Segmentation
Spatial Bayesian Nonparametrics for Natural Image Segmentation Erik Sudderth Brown University Joint work with Michael Jordan University of California Soumya Ghosh Brown University Parsing Visual Scenes
More informationImage segmentation combining Markov Random Fields and Dirichlet Processes
Image segmentation combining Markov Random Fields and Dirichlet Processes Jessica SODJO IMS, Groupe Signal Image, Talence Encadrants : A. Giremus, J.-F. Giovannelli, F. Caron, N. Dobigeon Jessica SODJO
More informationSharing Clusters Among Related Groups: Hierarchical Dirichlet Processes
Sharing Clusters Among Related Groups: Hierarchical Dirichlet Processes Yee Whye Teh (1), Michael I. Jordan (1,2), Matthew J. Beal (3) and David M. Blei (1) (1) Computer Science Div., (2) Dept. of Statistics
More informationOutline. Binomial, Multinomial, Normal, Beta, Dirichlet. Posterior mean, MAP, credible interval, posterior distribution
Outline A short review on Bayesian analysis. Binomial, Multinomial, Normal, Beta, Dirichlet Posterior mean, MAP, credible interval, posterior distribution Gibbs sampling Revisit the Gaussian mixture model
More informationThe Infinite Factorial Hidden Markov Model
The Infinite Factorial Hidden Markov Model Jurgen Van Gael Department of Engineering University of Cambridge, UK jv279@cam.ac.uk Yee Whye Teh Gatsby Unit University College London, UK ywteh@gatsby.ucl.ac.uk
More informationarxiv: v1 [stat.ml] 20 Nov 2012
A survey of non-exchangeable priors for Bayesian nonparametric models arxiv:1211.4798v1 [stat.ml] 20 Nov 2012 Nicholas J. Foti 1 and Sinead Williamson 2 1 Department of Computer Science, Dartmouth College
More informationHierarchical Dirichlet Processes
Hierarchical Dirichlet Processes Yee Whye Teh, Michael I. Jordan, Matthew J. Beal and David M. Blei Computer Science Div., Dept. of Statistics Dept. of Computer Science University of California at Berkeley
More informationNonparametric Mixed Membership Models
5 Nonparametric Mixed Membership Models Daniel Heinz Department of Mathematics and Statistics, Loyola University of Maryland, Baltimore, MD 21210, USA CONTENTS 5.1 Introduction................................................................................
More informationBayesian Nonparametric Learning of Complex Dynamical Phenomena
Duke University Department of Statistical Science Bayesian Nonparametric Learning of Complex Dynamical Phenomena Emily Fox Joint work with Erik Sudderth (Brown University), Michael Jordan (UC Berkeley),
More informationSparsity? A Bayesian view
Sparsity? A Bayesian view Zoubin Ghahramani Department of Engineering University of Cambridge SPARS Conference Cambridge, July 2015 Sparsity Many people are interested in sparsity. Why? Sparsity Many people
More informationBayesian nonparametric models for bipartite graphs
Bayesian nonparametric models for bipartite graphs François Caron INRIA IMB - University of Bordeaux Talence, France Francois.Caron@inria.fr Abstract We develop a novel Bayesian nonparametric model for
More informationarxiv: v2 [stat.ml] 4 Aug 2011
A Tutorial on Bayesian Nonparametric Models Samuel J. Gershman 1 and David M. Blei 2 1 Department of Psychology and Neuroscience Institute, Princeton University 2 Department of Computer Science, Princeton
More informationProperties of Bayesian nonparametric models and priors over trees
Properties of Bayesian nonparametric models and priors over trees David A. Knowles Computer Science Department Stanford University July 24, 2013 Introduction Theory: what characteristics might we want?
More informationDistance dependent Chinese restaurant processes
David M. Blei Department of Computer Science, Princeton University 35 Olden St., Princeton, NJ 08540 Peter Frazier Department of Operations Research and Information Engineering, Cornell University 232
More informationChapter 8 PROBABILISTIC MODELS FOR TEXT MINING. Yizhou Sun Department of Computer Science University of Illinois at Urbana-Champaign
Chapter 8 PROBABILISTIC MODELS FOR TEXT MINING Yizhou Sun Department of Computer Science University of Illinois at Urbana-Champaign sun22@illinois.edu Hongbo Deng Department of Computer Science University
More informationNonparametric Bayesian Methods: Models, Algorithms, and Applications (Day 5)
Nonparametric Bayesian Methods: Models, Algorithms, and Applications (Day 5) Tamara Broderick ITT Career Development Assistant Professor Electrical Engineering & Computer Science MIT Bayes Foundations
More informationNonparmeteric Bayes & Gaussian Processes. Baback Moghaddam Machine Learning Group
Nonparmeteric Bayes & Gaussian Processes Baback Moghaddam baback@jpl.nasa.gov Machine Learning Group Outline Bayesian Inference Hierarchical Models Model Selection Parametric vs. Nonparametric Gaussian
More informationDependent hierarchical processes for multi armed bandits
Dependent hierarchical processes for multi armed bandits Federico Camerlenghi University of Bologna, BIDSA & Collegio Carlo Alberto First Italian meeting on Probability and Mathematical Statistics, Torino
More informationDirichlet Enhanced Latent Semantic Analysis
Dirichlet Enhanced Latent Semantic Analysis Kai Yu Siemens Corporate Technology D-81730 Munich, Germany Kai.Yu@siemens.com Shipeng Yu Institute for Computer Science University of Munich D-80538 Munich,
More informationIPSJ SIG Technical Report Vol.2014-MPS-100 No /9/25 1,a) 1 1 SNS / / / / / / Time Series Topic Model Considering Dependence to Multiple Topics S
1,a) 1 1 SNS /// / // Time Series Topic Model Considering Dependence to Multiple Topics Sasaki Kentaro 1,a) Yoshikawa Tomohiro 1 Furuhashi Takeshi 1 Abstract: This pater proposes a topic model that considers
More informationApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models
Applied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR 2012 Erik Sudderth Brown University Work by E. Fox, E. Sudderth, M. Jordan, & A. Willsky AOAS 2011: A Sticky HDP-HMM with
More informationColouring and breaking sticks, pairwise coincidence losses, and clustering expression profiles
Colouring and breaking sticks, pairwise coincidence losses, and clustering expression profiles Peter Green and John Lau University of Bristol P.J.Green@bristol.ac.uk Isaac Newton Institute, 11 December
More informationCSCI 5822 Probabilistic Model of Human and Machine Learning. Mike Mozer University of Colorado
CSCI 5822 Probabilistic Model of Human and Machine Learning Mike Mozer University of Colorado Topics Language modeling Hierarchical processes Pitman-Yor processes Based on work of Teh (2006), A hierarchical
More informationBayesian Statistics. Debdeep Pati Florida State University. April 3, 2017
Bayesian Statistics Debdeep Pati Florida State University April 3, 2017 Finite mixture model The finite mixture of normals can be equivalently expressed as y i N(µ Si ; τ 1 S i ), S i k π h δ h h=1 δ h
More informationTree-Based Inference for Dirichlet Process Mixtures
Yang Xu Machine Learning Department School of Computer Science Carnegie Mellon University Pittsburgh, USA Katherine A. Heller Department of Engineering University of Cambridge Cambridge, UK Zoubin Ghahramani
More informationBayesian non-parametric model to longitudinally predict churn
Bayesian non-parametric model to longitudinally predict churn Bruno Scarpa Università di Padova Conference of European Statistics Stakeholders Methodologists, Producers and Users of European Statistics
More informationAn Alternative Prior Process for Nonparametric Bayesian Clustering
University of Pennsylvania ScholarlyCommons Statistics Papers Wharton Faculty Research 2010 An Alternative Prior Process for Nonparametric Bayesian Clustering Hanna M. Wallach University of Massachusetts
More informationAdvanced Machine Learning
Advanced Machine Learning Nonparametric Bayesian Models --Learning/Reasoning in Open Possible Worlds Eric Xing Lecture 7, August 4, 2009 Reading: Eric Xing Eric Xing @ CMU, 2006-2009 Clustering Eric Xing
More informationConstruction of Dependent Dirichlet Processes based on Poisson Processes
1 / 31 Construction of Dependent Dirichlet Processes based on Poisson Processes Dahua Lin Eric Grimson John Fisher CSAIL MIT NIPS 2010 Outstanding Student Paper Award Presented by Shouyuan Chen Outline
More informationBayesian Nonparametrics: some contributions to construction and properties of prior distributions
Bayesian Nonparametrics: some contributions to construction and properties of prior distributions Annalisa Cerquetti Collegio Nuovo, University of Pavia, Italy Interview Day, CETL Lectureship in Statistics,
More informationProbabilistic modeling of NLP
Structured Bayesian Nonparametric Models with Variational Inference ACL Tutorial Prague, Czech Republic June 24, 2007 Percy Liang and Dan Klein Probabilistic modeling of NLP Document clustering Topic modeling
More informationGentle Introduction to Infinite Gaussian Mixture Modeling
Gentle Introduction to Infinite Gaussian Mixture Modeling with an application in neuroscience By Frank Wood Rasmussen, NIPS 1999 Neuroscience Application: Spike Sorting Important in neuroscience and for
More informationarxiv: v1 [stat.ml] 8 Jan 2012
A Split-Merge MCMC Algorithm for the Hierarchical Dirichlet Process Chong Wang David M. Blei arxiv:1201.1657v1 [stat.ml] 8 Jan 2012 Received: date / Accepted: date Abstract The hierarchical Dirichlet process
More informationPart IV: Monte Carlo and nonparametric Bayes
Part IV: Monte Carlo and nonparametric Bayes Outline Monte Carlo methods Nonparametric Bayesian models Outline Monte Carlo methods Nonparametric Bayesian models The Monte Carlo principle The expectation
More informationarxiv: v2 [stat.ml] 10 Sep 2012
Distance Dependent Infinite Latent Feature Models arxiv:1110.5454v2 [stat.ml] 10 Sep 2012 Samuel J. Gershman 1, Peter I. Frazier 2 and David M. Blei 3 1 Department of Psychology and Princeton Neuroscience
More informationBeta-Negative Binomial Process and Exchangeable Random Partitions for Mixed-Membership Modeling
Beta-Negative Binomial Process and Exchangeable Random Partitions for Mixed-Membership Modeling Mingyuan Zhou IROM Department, McCombs School of Business The University of Texas at Austin, Austin, TX 77,
More informationVariational Bayesian Dirichlet-Multinomial Allocation for Exponential Family Mixtures
17th Europ. Conf. on Machine Learning, Berlin, Germany, 2006. Variational Bayesian Dirichlet-Multinomial Allocation for Exponential Family Mixtures Shipeng Yu 1,2, Kai Yu 2, Volker Tresp 2, and Hans-Peter
More information28 : Approximate Inference - Distributed MCMC
10-708: Probabilistic Graphical Models, Spring 2015 28 : Approximate Inference - Distributed MCMC Lecturer: Avinava Dubey Scribes: Hakim Sidahmed, Aman Gupta 1 Introduction For many interesting problems,
More informationA Simple Proof of the Stick-Breaking Construction of the Dirichlet Process
A Simple Proof of the Stick-Breaking Construction of the Dirichlet Process John Paisley Department of Computer Science Princeton University, Princeton, NJ jpaisley@princeton.edu Abstract We give a simple
More informationVariational Inference for the Nested Chinese Restaurant Process
Variational Inference for the Nested Chinese Restaurant Process Chong Wang Computer Science Department Princeton University chongw@cs.princeton.edu David M. Blei Computer Science Department Princeton University
More informationParallel Markov Chain Monte Carlo for Pitman-Yor Mixture Models
Parallel Markov Chain Monte Carlo for Pitman-Yor Mixture Models Avinava Dubey School of Computer Science Carnegie Mellon University Pittsburgh, PA 523 Sinead A. Williamson McCombs School of Business University
More informationFoundations of Nonparametric Bayesian Methods
1 / 27 Foundations of Nonparametric Bayesian Methods Part II: Models on the Simplex Peter Orbanz http://mlg.eng.cam.ac.uk/porbanz/npb-tutorial.html 2 / 27 Tutorial Overview Part I: Basics Part II: Models
More informationCSci 8980: Advanced Topics in Graphical Models Analysis of Genetic Variation
CSci 8980: Advanced Topics in Graphical Models Analysis of Genetic Variation Instructor: Arindam Banerjee November 26, 2007 Genetic Polymorphism Single nucleotide polymorphism (SNP) Genetic Polymorphism
More informationEvolutionary Clustering by Hierarchical Dirichlet Process with Hidden Markov State
Evolutionary Clustering by Hierarchical Dirichlet Process with Hidden Markov State Tianbing Xu 1 Zhongfei (Mark) Zhang 1 1 Dept. of Computer Science State Univ. of New York at Binghamton Binghamton, NY
More informationHierarchical Bayesian Nonparametrics
Hierarchical Bayesian Nonparametrics Micha Elsner April 11, 2013 2 For next time We ll tackle a paper: Green, de Marneffe, Bauer and Manning: Multiword Expression Identification with Tree Substitution
More informationSpectral Chinese Restaurant Processes: Nonparametric Clustering Based on Similarities
Spectral Chinese Restaurant Processes: Nonparametric Clustering Based on Similarities Richard Socher Andrew Maas Christopher D. Manning Computer Science Department, Stanford University, Stanford, CA 94305
More informationHierarchical Bayesian Languge Model Based on Pitman-Yor Processes. Yee Whye Teh
Hierarchical Bayesian Languge Model Based on Pitman-Yor Processes Yee Whye Teh Probabilistic model of language n-gram model Utility i-1 P(word i word i-n+1 ) Typically, trigram model (n=3) e.g., speech,
More informationBayesian Mixtures of Bernoulli Distributions
Bayesian Mixtures of Bernoulli Distributions Laurens van der Maaten Department of Computer Science and Engineering University of California, San Diego Introduction The mixture of Bernoulli distributions
More informationHierarchical Bayesian Nonparametric Models of Language and Text
Hierarchical Bayesian Nonparametric Models of Language and Text Gatsby Computational Neuroscience Unit, UCL Joint work with Frank Wood *, Jan Gasthaus *, Cedric Archambeau, Lancelot James SIGIR Workshop
More information27 : Distributed Monte Carlo Markov Chain. 1 Recap of MCMC and Naive Parallel Gibbs Sampling
10-708: Probabilistic Graphical Models 10-708, Spring 2014 27 : Distributed Monte Carlo Markov Chain Lecturer: Eric P. Xing Scribes: Pengtao Xie, Khoa Luu In this scribe, we are going to review the Parallel
More informationBayesian Methods for Machine Learning
Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),
More informationarxiv: v1 [cs.lg] 16 Sep 2014
Journal of Machine Learning Research (2000) -48 Submitted 4/00; Published 0/00 Collapsed Variational Bayes Inference of Infinite Relational Model arxiv:409.4757v [cs.lg] 6 Sep 204 Katsuhiko Ishiguro NTT
More informationTruncation-free Stochastic Variational Inference for Bayesian Nonparametric Models
Truncation-free Stochastic Variational Inference for Bayesian Nonparametric Models Chong Wang Machine Learning Department Carnegie Mellon University chongw@cs.cmu.edu David M. Blei Computer Science Department
More informationTruncation error of a superposed gamma process in a decreasing order representation
Truncation error of a superposed gamma process in a decreasing order representation B julyan.arbel@inria.fr Í www.julyanarbel.com Inria, Mistis, Grenoble, France Joint work with Igor Pru nster (Bocconi
More informationHierarchical Dirichlet Processes with Random Effects
Hierarchical Dirichlet Processes with Random Effects Seyoung Kim Department of Computer Science University of California, Irvine Irvine, CA 92697-34 sykim@ics.uci.edu Padhraic Smyth Department of Computer
More informationCS6220: DATA MINING TECHNIQUES
CS6220: DATA MINING TECHNIQUES Matrix Data: Clustering: Part 2 Instructor: Yizhou Sun yzsun@ccs.neu.edu October 19, 2014 Methods to Learn Matrix Data Set Data Sequence Data Time Series Graph & Network
More informationThe Infinite Latent Events Model
The Infinite Latent Events Model David Wingate, Noah D. Goodman, Daniel M. Roy and Joshua B. Tenenbaum Brain and Cognitive Sciences Massachusetts Institute of Technology Cambridge, MA 02139 Abstract We
More informationTWO MODELS INVOLVING BAYESIAN NONPARAMETRIC TECHNIQUES
TWO MODELS INVOLVING BAYESIAN NONPARAMETRIC TECHNIQUES By SUBHAJIT SENGUPTA A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE
More information