Image segmentation combining Markov Random Fields and Dirichlet Processes
|
|
- MargaretMargaret Page
- 5 years ago
- Views:
Transcription
1 Image segmentation combining Markov Random Fields and Dirichlet Processes Jessica SODJO IMS, Groupe Signal Image, Talence Encadrants : A. Giremus, J.-F. Giovannelli, F. Caron, N. Dobigeon Jessica SODJO ANR meeting 1 / 28
2 Plan 1 Introduction 2 Segmentation using DP models Mixed MRF / DP model Inference : Swendsen-Wang algorithm 3 Hierarchical segmentation with shared classes Principle HDP theory 4 Conclusion and perspective Jessica SODJO ANR meeting 2 / 28
3 Introduction Segmentation partition of an image in K homogeneous regions called classes label the pixels : pixel i z i {1,..., K } Bayesian approach prior on the distribution of the pixels all the pixels in a class have the same distribution characterized by a parameter vector U k Markov Random Fields (MRF) : exploit the similarity of pixels in the same neighbourhood Constraint : K must be fixed a priori Idea : use the BNP models to directly estimate K Jessica SODJO ANR meeting 3 / 28
4 Segmentation using DP models Plan 1 Introduction 2 Segmentation using DP models Mixed MRF / DP model Inference : Swendsen-Wang algorithm 3 Hierarchical segmentation with shared classes Principle HDP theory 4 Conclusion and perspective Jessica SODJO ANR meeting 4 / 28
5 FIGURE: Example of partition Jessica SODJO ANR meeting 5 / 28 ANR meeting Segmentation using DP models Notations N is the number of pixels Y is the observed image Z = {z 1,..., z N } Π = {A 1,..., A K } is a partition and m = {m 1,..., m K } with m k = A k A 1 A 3 A 2 AK m 1 = 1 m 2 = 5 m 3 = 6 m K = 4
6 Segmentation using DP models Mixed MRF / DP model Markov Random Fields (MRF) Description of the image by a neighbouring system Considered pixel Neighbours 4-neighbours 8-neighbours FIGURE: Examples of neighbouring system A clique c is either a singleton either a set of pixels in the same neighbourhood Jessica SODJO ANR meeting 6 / 28
7 Segmentation using DP models Mixed MRF / DP model Markov Random Fields Let θ i {U 1,..., U K } be the parameter vector associated to the i-th pixel MRF p(θ i θ i ) = p(θ i θ V(i) ) where V(i) is the set of neighbours of pixel i Hammersley-Clifford theorem Gibbs field ( p(θ) = 1 exp ( Φ(θ)) = 1 exp Z Φ Z Φ c Φ c (θ c ) ) (1) with Φ c (θ c ) the local potential and Φ(θ) the global one Limitation : K is assumed to be known Jessica SODJO ANR meeting 7 / 28
8 Segmentation using DP models Mixed MRF / DP model Potts model The Potts model is a special MRF defined by : M(Π) exp β ij 1 zi =z j (2) i j where i j means that the pixels i and j are neighbours β ij > 0 if i and j are neighbours and β ij = 0 otherwise Jessica SODJO ANR meeting 8 / 28
9 Segmentation using DP models Mixed MRF / DP model The DP model τ k γ, H Beta(1, γ) τ where Beta(.) is the Beta distribution k 1 k = τ k (1 τ l ) (3) Let us write τ Stick(γ), τ = {τ 1, τ 2,...} and k=1 τ k = 1 with G γ, H DP(γ, H) G = l=1 τ k δ Uk (4) k=1 U k H iid H (5) The distribution of the observations is f, defined as : y i θ i f (. θ i ) and θ i G G (6) Jessica SODJO ANR meeting 9 / 28
10 Segmentation using DP models Mixed MRF / DP model The DP model The Chinese Restaurant Process says, θ i θ i K i k=1 m i k N 1 + γ δ U k + γ N 1 + γ H m i k is the size of cluster k if we remove pixel i from the partition K i is the number of clusters in the image with the i-th pixel removed U k is the parameter vector associated to the k-th cluster Limitation : the spatial interactions are not taken into account Jessica SODJO ANR meeting 10 / 28
11 Segmentation using DP models Mixed MRF / DP model Principle of the segmentation using DP models Define a distribution on the partitions using : a model that allows that pixels in the same neighbourhood are likely to be in the same cluster (MRF) DP model to deduce automatically the number of clusters (and if needed their parameters) Jessica SODJO ANR meeting 11 / 28
12 Segmentation using DP models Mixed MRF / DP model Prior distribution mixing DP and MRF p(θ) 1 exp( Φ i (θ i )) Z G i }{{} Ψ(θ) DP model where C 2 means c 2 and. is the size. Φ i (.) is defined as : Φ i (θ i ) = log G(θ i ) and Z G = Ψ(θ) = N G(θ i ) i=1 1 Z M exp( c C 2 Φ c (θ c )) }{{} M(θ) MRF model N i=1 exp( log G(θ i ))dθ 1... dθ N P. Orbanz & J. M. Buhmann Nonparametric Bayesian image segmentation, International Journal of Computer Vision, 2007 Jessica SODJO ANR meeting 12 / 28
13 Segmentation using DP models Mixed MRF / DP model Prior distribution mixing DP and MRF We can deduce : P(θ i θ i ) K k=1 M(θ i θ i )m i k δ U k + γ Z Φ H (7) Probability of assignment to a new cluster : q i0 f (y i θ)h(θ)dθ (8) Ω θ Probability of assignment to an existing cluster : Parameter update : q ik m i k exp( Φ(U k θ i ))f (y i U k ) (9) U k G 0 (U k ) i i A k f (y i U k ) (10) Jessica SODJO ANR meeting 13 / 28
14 Segmentation using DP models Inference : Swendsen-Wang algorithm Swendsen-Wang algorithm : principle * Estimation based on the joint posterior p(θ, Z Y ) * Intractable Markov Chain Monte Carlo (MCMC) Problem : very slow convergence Goal : Sample faster the partition of the image Introduction of a new set of latent variables r such that : p(π, r) = p(π)p(r Π) p(r Π) = p(r ij Π) 1<i<j<N p(r ij = 1 Π) = 1 exp(β ij δ ij 1 zi =z j ) The marginal posterior p(θ, Z Y ) is unchanged The links define the "so-called" spin-clusters Jessica SODJO ANR meeting 14 / 28
15 Segmentation using DP models Inference : Swendsen-Wang algorithm Swendsen-Wang algorithm : principle Update the labels of the spin-clusters This operation update simultaneously the labels of all the pixels in a spin-cluster FIGURE: Example of label update for spin-clusters Jessica SODJO ANR meeting 15 / 28
16 Segmentation using DP models Inference : Swendsen-Wang algorithm Swendsen-Wang algorithm : principle r ij Ber(1 exp(β ij δ ij 1 zi =z j )) with Ber(.) is the Bernouilli distribution Let S = {S 1,..., S p } be the set of spin-clusters. While removing the spin-cluster S l, Π l = {A l 1,..., A l K l } is the partition obtained while removing all pixels in spin-cluster S l m l k = A l k Jessica SODJO ANR meeting 16 / 28
17 Segmentation using DP models Inference : Swendsen-Wang algorithm Swendsen-Wang algorithm : principle For l = 1 : p * The probability to assign pixels in spin-cluster S l to cluster k is : q lk Ψ(m l 1,..., m l k {(i,j) i S l,r ij =0} + S l,..., m l K l )p(y Sl exp(β ij (1 δ ij )1 zi =z j ) y A l ) k * The probability to assign pixels in spin-cluster S l to a new cluster is : with p(y Ak ) = q l0 = Ψ(m l 1,..., m l K l, S l )p(y Sl ) i A k f (y i U k )H(U k )du k Jessica SODJO ANR meeting 17 / 28
18 Hierarchical segmentation with shared classes Plan 1 Introduction 2 Segmentation using DP models Mixed MRF / DP model Inference : Swendsen-Wang algorithm 3 Hierarchical segmentation with shared classes Principle HDP theory 4 Conclusion and perspective Jessica SODJO ANR meeting 18 / 28
19 Hierarchical segmentation with shared classes Principle Proposed idea Different levels of classification can be considered Coarse categories : urban, sub-urban, forest, etc. Sub-classes shared between the categories : trees, roads, buildings Taking into account the fact that the classes are shared between different categories can help estimating their parameters and thereby improve the segmentation Jessica SODJO ANR meeting 19 / 28
20 Hierarchical segmentation with shared classes HDP theory Solution : Hierarchical DP Let J be the number of categories G 0 γ, H DP(γ, H) G j α 0, G 0 DP(α 0, G 0 ) for j = 1,..., J α 0 R + G 0 is a discrete distribution Discreteness of G 0 clusters shared among categories Jessica SODJO ANR meeting 20 / 28
21 Hierarchical segmentation with shared classes HDP theory G 0 = τ k δ Uk (11) k=1 where τ γ Stick(γ), τ = {τ 1, τ 2,...} and U k H H G j = π jk δ Uk (12) k=1 with π j α 0, τ DP(α 0, τ ) and π j = {π j1, π j2,...} ϕ ji G j G j (13) So, samples of the processes G 0 and G j can be seen as infinite countable mixtures of Dirac measures with respective coefficients τ and π j. Jessica SODJO ANR meeting 21 / 28
22 Hierarchical segmentation with shared classes HDP theory Principle - Chinese Restaurant Franchise NOTATIONS J restaurants Same menu for all restaurants - U 1, U 2,... T j is the number of tables in restaurant j θ jt is the t-th table of restaurant j ϕ ji is the i-th client in restaurant j n jt is the number of clients at a table t η jk is the number of tables in restaurant j which have chosen dish U k and η k = k η jk Jessica SODJO ANR meeting 22 / 28
23 Hierarchical segmentation with shared classes HDP theory Principle - Chinese Restaurant Franchise Menu U 1 U 2 U 3... ϕ 11 ϕ 13 ϕ 12 ϕ 14 θ 11 = U 1 ϕ 21 ϕ 23 ϕ 22 ϕ 25 θ 21 = U 2 ϕ 31 ϕ 32 ϕ 33 θ 31 = U 1 θ 12 = U 2 θ 22 = U 1 θ 32 = U 2 Restaurant 1 θ 13 = U 2... Restaurant 2... Restaurant 3 θ 33 = U 3... Jessica SODJO ANR meeting 23 / 28
24 Hierarchical segmentation with shared classes HDP theory Principle - Chinese Restaurant Franchise ϕ Exemple : Restaurant θ 11 = U 1 n11 = 2 Menu U 1 U 2 η1 = 3 η 2 = 4 ϕ 12 ϕ 14 θ 12 = U 2 θ 13 = U 2 n 12 = 1 n 13 = 1 α 0 ϕ 15 U 3 η 3 = 1 U 4 γ θ 14 Jessica SODJO ANR meeting 24 / 28
25 Hierarchical segmentation with shared classes HDP theory Principle - Chinese Restaurant Franchise ϕ Exemple : Restaurant θ 11 = U 1 n11 = 2 Menu U 1 U 2 η1 = 3 η 2 = 4 ϕ 12 ϕ 14 θ 12 = U 2 θ 13 = U 2 n 12 = 1 n 13 = 1 α 0 ϕ 15 U 3 η 3 = 1 U 4 γ θ 14 Jessica SODJO ANR meeting 24 / 28
26 Hierarchical segmentation with shared classes HDP theory Principle - Chinese Restaurant Franchise ϕ Exemple : Restaurant θ 11 = U 1 n11 = 2 Menu U 1 U 2 η1 = 3 η 2 = 4 ϕ 12 ϕ 14 θ 12 = U 2 θ 13 = U 2 n 12 = 1 n 13 = 1 α 0 ϕ 15 U 3 η 3 = 1 U 4 γ θ 14 Jessica SODJO ANR meeting 24 / 28
27 Hierarchical segmentation with shared classes HDP theory Principle - Chinese Restaurant Franchise ϕ Exemple : Restaurant θ 11 = U 1 n11 = 2 Menu U 1 U 2 η1 = 3 η 2 = 4 ϕ 12 ϕ 14 θ 12 = U 2 θ 13 = U 2 n 12 = 1 n 13 = 1 α 0 ϕ 15 U 3 η 3 = 1 U 4 γ θ 14 Jessica SODJO ANR meeting 24 / 28
28 Hierarchical segmentation with shared classes HDP theory Chinese Restaurant Franchise ϕ ji ϕ j1,..., ϕ ji 1, α 0, G 0 θ jt θ j1,..., θ 21,..., θ jt 1, γ, H T j t=1 K k=1 n jt i 1 + α 0 δ θjt + α 0 i 1 + α 0 G 0 (14) η k k η k + γ δ γ U k + k η k + γ H (15) Y. W. Teh, M. I. Jordan, M. J. Beal & D. M. Blei Hierarchical Dirichlet Processes, JASA, 2006 Jessica SODJO ANR meeting 25 / 28
29 Conclusion and perspective Plan 1 Introduction 2 Segmentation using DP models Mixed MRF / DP model Inference : Swendsen-Wang algorithm 3 Hierarchical segmentation with shared classes Principle HDP theory 4 Conclusion and perspective Jessica SODJO ANR meeting 26 / 28
30 Conclusion and perspective Conclusion Spatial constraints : Potts model Flexibility : DP model Rapidity : Swendsen-Wang algorithm Sharing : HDP Perspective Efficient sampling algorithm Jessica SODJO ANR meeting 27 / 28
31 Thank Thank you for your attention Jessica SODJO ANR meeting 28 / 28
Dependent hierarchical processes for multi armed bandits
Dependent hierarchical processes for multi armed bandits Federico Camerlenghi University of Bologna, BIDSA & Collegio Carlo Alberto First Italian meeting on Probability and Mathematical Statistics, Torino
More informationSpatial Bayesian Nonparametrics for Natural Image Segmentation
Spatial Bayesian Nonparametrics for Natural Image Segmentation Erik Sudderth Brown University Joint work with Michael Jordan University of California Soumya Ghosh Brown University Parsing Visual Scenes
More informationNon-parametric Clustering with Dirichlet Processes
Non-parametric Clustering with Dirichlet Processes Timothy Burns SUNY at Buffalo Mar. 31 2009 T. Burns (SUNY at Buffalo) Non-parametric Clustering with Dirichlet Processes Mar. 31 2009 1 / 24 Introduction
More informationLecture 3a: Dirichlet processes
Lecture 3a: Dirichlet processes Cédric Archambeau Centre for Computational Statistics and Machine Learning Department of Computer Science University College London c.archambeau@cs.ucl.ac.uk Advanced Topics
More informationBayesian Nonparametrics for Speech and Signal Processing
Bayesian Nonparametrics for Speech and Signal Processing Michael I. Jordan University of California, Berkeley June 28, 2011 Acknowledgments: Emily Fox, Erik Sudderth, Yee Whye Teh, and Romain Thibaux Computer
More informationBayesian Nonparametrics: Models Based on the Dirichlet Process
Bayesian Nonparametrics: Models Based on the Dirichlet Process Alessandro Panella Department of Computer Science University of Illinois at Chicago Machine Learning Seminar Series February 18, 2013 Alessandro
More informationNon-Parametric Bayes
Non-Parametric Bayes Mark Schmidt UBC Machine Learning Reading Group January 2016 Current Hot Topics in Machine Learning Bayesian learning includes: Gaussian processes. Approximate inference. Bayesian
More informationSharing Clusters Among Related Groups: Hierarchical Dirichlet Processes
Sharing Clusters Among Related Groups: Hierarchical Dirichlet Processes Yee Whye Teh (1), Michael I. Jordan (1,2), Matthew J. Beal (3) and David M. Blei (1) (1) Computer Science Div., (2) Dept. of Statistics
More informationDirichlet Processes: Tutorial and Practical Course
Dirichlet Processes: Tutorial and Practical Course (updated) Yee Whye Teh Gatsby Computational Neuroscience Unit University College London August 2007 / MLSS Yee Whye Teh (Gatsby) DP August 2007 / MLSS
More informationHierarchical Dirichlet Processes with Random Effects
Hierarchical Dirichlet Processes with Random Effects Seyoung Kim Department of Computer Science University of California, Irvine Irvine, CA 92697-34 sykim@ics.uci.edu Padhraic Smyth Department of Computer
More informationBayesian Nonparametric Models
Bayesian Nonparametric Models David M. Blei Columbia University December 15, 2015 Introduction We have been looking at models that posit latent structure in high dimensional data. We use the posterior
More informationBayesian Nonparametric Models on Decomposable Graphs
Bayesian Nonparametric Models on Decomposable Graphs François Caron INRIA Bordeaux Sud Ouest Institut de Mathématiques de Bordeaux University of Bordeaux, France francois.caron@inria.fr Arnaud Doucet Departments
More informationApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models
Applied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR 2012 Erik Sudderth Brown University Work by E. Fox, E. Sudderth, M. Jordan, & A. Willsky AOAS 2011: A Sticky HDP-HMM with
More informationHierarchical Dirichlet Processes
Hierarchical Dirichlet Processes Yee Whye Teh, Michael I. Jordan, Matthew J. Beal and David M. Blei Computer Science Div., Dept. of Statistics Dept. of Computer Science University of California at Berkeley
More informationNonparametric Mixed Membership Models
5 Nonparametric Mixed Membership Models Daniel Heinz Department of Mathematics and Statistics, Loyola University of Maryland, Baltimore, MD 21210, USA CONTENTS 5.1 Introduction................................................................................
More informationBayesian non parametric approaches: an introduction
Introduction Latent class models Latent feature models Conclusion & Perspectives Bayesian non parametric approaches: an introduction Pierre CHAINAIS Bordeaux - nov. 2012 Trajectory 1 Bayesian non parametric
More informationVariational Bayesian Dirichlet-Multinomial Allocation for Exponential Family Mixtures
17th Europ. Conf. on Machine Learning, Berlin, Germany, 2006. Variational Bayesian Dirichlet-Multinomial Allocation for Exponential Family Mixtures Shipeng Yu 1,2, Kai Yu 2, Volker Tresp 2, and Hans-Peter
More informationDirichlet Enhanced Latent Semantic Analysis
Dirichlet Enhanced Latent Semantic Analysis Kai Yu Siemens Corporate Technology D-81730 Munich, Germany Kai.Yu@siemens.com Shipeng Yu Institute for Computer Science University of Munich D-80538 Munich,
More informationSTAT Advanced Bayesian Inference
1 / 32 STAT 625 - Advanced Bayesian Inference Meng Li Department of Statistics Jan 23, 218 The Dirichlet distribution 2 / 32 θ Dirichlet(a 1,...,a k ) with density p(θ 1,θ 2,...,θ k ) = k j=1 Γ(a j) Γ(
More informationBayesian Hidden Markov Models and Extensions
Bayesian Hidden Markov Models and Extensions Zoubin Ghahramani Department of Engineering University of Cambridge joint work with Matt Beal, Jurgen van Gael, Yunus Saatci, Tom Stepleton, Yee Whye Teh Modeling
More informationBayesian Image Segmentation Using MRF s Combined with Hierarchical Prior Models
Bayesian Image Segmentation Using MRF s Combined with Hierarchical Prior Models Kohta Aoki 1 and Hiroshi Nagahashi 2 1 Interdisciplinary Graduate School of Science and Engineering, Tokyo Institute of Technology
More informationBayesian non-parametric model to longitudinally predict churn
Bayesian non-parametric model to longitudinally predict churn Bruno Scarpa Università di Padova Conference of European Statistics Stakeholders Methodologists, Producers and Users of European Statistics
More informationBayesian nonparametrics
Bayesian nonparametrics 1 Some preliminaries 1.1 de Finetti s theorem We will start our discussion with this foundational theorem. We will assume throughout all variables are defined on the probability
More informationUnsupervised Learning
Unsupervised Learning Bayesian Model Comparison Zoubin Ghahramani zoubin@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc in Intelligent Systems, Dept Computer Science University College
More informationAdvanced Machine Learning
Advanced Machine Learning Nonparametric Bayesian Models --Learning/Reasoning in Open Possible Worlds Eric Xing Lecture 7, August 4, 2009 Reading: Eric Xing Eric Xing @ CMU, 2006-2009 Clustering Eric Xing
More informationA Brief Overview of Nonparametric Bayesian Models
A Brief Overview of Nonparametric Bayesian Models Eurandom Zoubin Ghahramani Department of Engineering University of Cambridge, UK zoubin@eng.cam.ac.uk http://learning.eng.cam.ac.uk/zoubin Also at Machine
More informationCollapsed Variational Dirichlet Process Mixture Models
Collapsed Variational Dirichlet Process Mixture Models Kenichi Kurihara Dept. of Computer Science Tokyo Institute of Technology, Japan kurihara@mi.cs.titech.ac.jp Max Welling Dept. of Computer Science
More information13: Variational inference II
10-708: Probabilistic Graphical Models, Spring 2015 13: Variational inference II Lecturer: Eric P. Xing Scribes: Ronghuo Zheng, Zhiting Hu, Yuntian Deng 1 Introduction We started to talk about variational
More informationBayesian Nonparametrics: Dirichlet Process
Bayesian Nonparametrics: Dirichlet Process Yee Whye Teh Gatsby Computational Neuroscience Unit, UCL http://www.gatsby.ucl.ac.uk/~ywteh/teaching/npbayes2012 Dirichlet Process Cornerstone of modern Bayesian
More informationLecture 13 : Variational Inference: Mean Field Approximation
10-708: Probabilistic Graphical Models 10-708, Spring 2017 Lecture 13 : Variational Inference: Mean Field Approximation Lecturer: Willie Neiswanger Scribes: Xupeng Tong, Minxing Liu 1 Problem Setup 1.1
More informationLecture 16-17: Bayesian Nonparametrics I. STAT 6474 Instructor: Hongxiao Zhu
Lecture 16-17: Bayesian Nonparametrics I STAT 6474 Instructor: Hongxiao Zhu Plan for today Why Bayesian Nonparametrics? Dirichlet Distribution and Dirichlet Processes. 2 Parameter and Patterns Reference:
More informationChapter 8 PROBABILISTIC MODELS FOR TEXT MINING. Yizhou Sun Department of Computer Science University of Illinois at Urbana-Champaign
Chapter 8 PROBABILISTIC MODELS FOR TEXT MINING Yizhou Sun Department of Computer Science University of Illinois at Urbana-Champaign sun22@illinois.edu Hongbo Deng Department of Computer Science University
More informationConstruction of Dependent Dirichlet Processes based on Poisson Processes
1 / 31 Construction of Dependent Dirichlet Processes based on Poisson Processes Dahua Lin Eric Grimson John Fisher CSAIL MIT NIPS 2010 Outstanding Student Paper Award Presented by Shouyuan Chen Outline
More informationApplied Nonparametric Bayes
Applied Nonparametric Bayes Michael I. Jordan Department of Electrical Engineering and Computer Science Department of Statistics University of California, Berkeley http://www.cs.berkeley.edu/ jordan Acknowledgments:
More informationCSci 8980: Advanced Topics in Graphical Models Analysis of Genetic Variation
CSci 8980: Advanced Topics in Graphical Models Analysis of Genetic Variation Instructor: Arindam Banerjee November 26, 2007 Genetic Polymorphism Single nucleotide polymorphism (SNP) Genetic Polymorphism
More informationChapter 4 Dynamic Bayesian Networks Fall Jin Gu, Michael Zhang
Chapter 4 Dynamic Bayesian Networks 2016 Fall Jin Gu, Michael Zhang Reviews: BN Representation Basic steps for BN representations Define variables Define the preliminary relations between variables Check
More informationCPSC 540: Machine Learning
CPSC 540: Machine Learning MCMC and Non-Parametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is
More informationBayesian Nonparametrics
Bayesian Nonparametrics Lorenzo Rosasco 9.520 Class 18 April 11, 2011 About this class Goal To give an overview of some of the basic concepts in Bayesian Nonparametrics. In particular, to discuss Dirichelet
More informationInfinite latent feature models and the Indian Buffet Process
p.1 Infinite latent feature models and the Indian Buffet Process Tom Griffiths Cognitive and Linguistic Sciences Brown University Joint work with Zoubin Ghahramani p.2 Beyond latent classes Unsupervised
More informationarxiv: v1 [stat.ml] 8 Jan 2012
A Split-Merge MCMC Algorithm for the Hierarchical Dirichlet Process Chong Wang David M. Blei arxiv:1201.1657v1 [stat.ml] 8 Jan 2012 Received: date / Accepted: date Abstract The hierarchical Dirichlet process
More informationSpatial Normalized Gamma Process
Spatial Normalized Gamma Process Vinayak Rao Yee Whye Teh Presented at NIPS 2009 Discussion and Slides by Eric Wang June 23, 2010 Outline Introduction Motivation The Gamma Process Spatial Normalized Gamma
More informationA = {(x, u) : 0 u f(x)},
Draw x uniformly from the region {x : f(x) u }. Markov Chain Monte Carlo Lecture 5 Slice sampler: Suppose that one is interested in sampling from a density f(x), x X. Recall that sampling x f(x) is equivalent
More informationMAD-Bayes: MAP-based Asymptotic Derivations from Bayes
MAD-Bayes: MAP-based Asymptotic Derivations from Bayes Tamara Broderick Brian Kulis Michael I. Jordan Cat Clusters Mouse clusters Dog 1 Cat Clusters Dog Mouse Lizard Sheep Picture 1 Picture 2 Picture 3
More informationDirichlet Processes and other non-parametric Bayesian models
Dirichlet Processes and other non-parametric Bayesian models Zoubin Ghahramani http://learning.eng.cam.ac.uk/zoubin/ zoubin@cs.cmu.edu Statistical Machine Learning CMU 10-702 / 36-702 Spring 2008 Model
More informationBayesian nonparametric latent feature models
Bayesian nonparametric latent feature models Indian Buffet process, beta process, and related models François Caron Department of Statistics, Oxford Applied Bayesian Statistics Summer School Como, Italy
More informationarxiv: v2 [stat.ml] 4 Aug 2011
A Tutorial on Bayesian Nonparametric Models Samuel J. Gershman 1 and David M. Blei 2 1 Department of Psychology and Neuroscience Institute, Princeton University 2 Department of Computer Science, Princeton
More informationPriors for Random Count Matrices with Random or Fixed Row Sums
Priors for Random Count Matrices with Random or Fixed Row Sums Mingyuan Zhou Joint work with Oscar Madrid and James Scott IROM Department, McCombs School of Business Department of Statistics and Data Sciences
More informationHierarchical Bayesian Nonparametric Models with Applications
Hierarchical Bayesian Nonparametric Models with Applications Yee Whye Teh Gatsby Computational Neuroscience Unit University College London 17 Queen Square London WC1N 3AR, United Kingdom Michael I. Jordan
More informationShared Segmentation of Natural Scenes. Dependent Pitman-Yor Processes
Shared Segmentation of Natural Scenes using Dependent Pitman-Yor Processes Erik Sudderth & Michael Jordan University of California, Berkeley Parsing Visual Scenes sky skyscraper sky dome buildings trees
More informationBayesian Nonparametric Autoregressive Models via Latent Variable Representation
Bayesian Nonparametric Autoregressive Models via Latent Variable Representation Maria De Iorio Yale-NUS College Dept of Statistical Science, University College London Collaborators: Lifeng Ye (UCL, London,
More informationBayesian nonparametric latent feature models
Bayesian nonparametric latent feature models François Caron UBC October 2, 2007 / MLRG François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 1 / 29 Overview 1 Introduction
More informationDensity Estimation. Seungjin Choi
Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More informationOutline. Binomial, Multinomial, Normal, Beta, Dirichlet. Posterior mean, MAP, credible interval, posterior distribution
Outline A short review on Bayesian analysis. Binomial, Multinomial, Normal, Beta, Dirichlet Posterior mean, MAP, credible interval, posterior distribution Gibbs sampling Revisit the Gaussian mixture model
More information27 : Distributed Monte Carlo Markov Chain. 1 Recap of MCMC and Naive Parallel Gibbs Sampling
10-708: Probabilistic Graphical Models 10-708, Spring 2014 27 : Distributed Monte Carlo Markov Chain Lecturer: Eric P. Xing Scribes: Pengtao Xie, Khoa Luu In this scribe, we are going to review the Parallel
More informationBayesian Structure Modeling. SPFLODD December 1, 2011
Bayesian Structure Modeling SPFLODD December 1, 2011 Outline Defining Bayesian Parametric Bayesian models Latent Dirichlet allocabon (Blei et al., 2003) Bayesian HMM (Goldwater and Griffiths, 2007) A limle
More informationBayesian Learning. HT2015: SC4 Statistical Data Mining and Machine Learning. Maximum Likelihood Principle. The Bayesian Learning Framework
HT5: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford http://www.stats.ox.ac.uk/~sejdinov/sdmml.html Maximum Likelihood Principle A generative model for
More informationProbabilistic Graphical Networks: Definitions and Basic Results
This document gives a cursory overview of Probabilistic Graphical Networks. The material has been gleaned from different sources. I make no claim to original authorship of this material. Bayesian Graphical
More informationTrajectory Analysis and Semantic Region Modeling Using A Nonparametric Bayesian Model Xiaogang Wang,, Keng Teck Ma,, Gee-Wah Ng,, and Eric Grimson
Computer Science and Artificial Intelligence Laboratory Technical Report MIT-CSAIL-TR-2008-015 June 24, 2008 Trajectory Analysis and Semantic Region Modeling Using A Nonparametric Bayesian Model Xiaogang
More informationCSCI 5822 Probabilistic Model of Human and Machine Learning. Mike Mozer University of Colorado
CSCI 5822 Probabilistic Model of Human and Machine Learning Mike Mozer University of Colorado Topics Language modeling Hierarchical processes Pitman-Yor processes Based on work of Teh (2006), A hierarchical
More informationSummary STK 4150/9150
STK4150 - Intro 1 Summary STK 4150/9150 Odd Kolbjørnsen May 22 2017 Scope You are expected to know and be able to use basic concepts introduced in the book. You knowledge is expected to be larger than
More informationVariational Scoring of Graphical Model Structures
Variational Scoring of Graphical Model Structures Matthew J. Beal Work with Zoubin Ghahramani & Carl Rasmussen, Toronto. 15th September 2003 Overview Bayesian model selection Approximations using Variational
More informationPart 3: Applications of Dirichlet processes. 3.2 First examples: applying DPs to density estimation and cluster
CS547Q Statistical Modeling with Stochastic Processes Winter 2011 Part 3: Applications of Dirichlet processes Lecturer: Alexandre Bouchard-Côté Scribe(s): Seagle Liu, Chao Xiong Disclaimer: These notes
More informationLatent Dirichlet Alloca/on
Latent Dirichlet Alloca/on Blei, Ng and Jordan ( 2002 ) Presented by Deepak Santhanam What is Latent Dirichlet Alloca/on? Genera/ve Model for collec/ons of discrete data Data generated by parameters which
More informationParallel Markov Chain Monte Carlo for Pitman-Yor Mixture Models
Parallel Markov Chain Monte Carlo for Pitman-Yor Mixture Models Avinava Dubey School of Computer Science Carnegie Mellon University Pittsburgh, PA 523 Sinead A. Williamson McCombs School of Business University
More informationBayesian Nonparametrics
Bayesian Nonparametrics Peter Orbanz Columbia University PARAMETERS AND PATTERNS Parameters P(X θ) = Probability[data pattern] 3 2 1 0 1 2 3 5 0 5 Inference idea data = underlying pattern + independent
More informationMarkov Chains and MCMC
Markov Chains and MCMC Markov chains Let S = {1, 2,..., N} be a finite set consisting of N states. A Markov chain Y 0, Y 1, Y 2,... is a sequence of random variables, with Y t S for all points in time
More informationBayesian Methods for Machine Learning
Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),
More informationVariational Hidden Conditional Random Fields with Coupled Dirichlet Process Mixtures
Variational Hidden Conditional Random Fields with Coupled Dirichlet Process Mixtures Konstantinos Bousmalis 1, Stefanos Zafeiriou 1, Louis Philippe Morency 2, Maja Pantic 1, and Zoubin Ghahramani 3 1 Imperial
More informationBayesian Classification and Regression Trees
Bayesian Classification and Regression Trees James Cussens York Centre for Complex Systems Analysis & Dept of Computer Science University of York, UK 1 Outline Problems for Lessons from Bayesian phylogeny
More informationSTAT 518 Intro Student Presentation
STAT 518 Intro Student Presentation Wen Wei Loh April 11, 2013 Title of paper Radford M. Neal [1999] Bayesian Statistics, 6: 475-501, 1999 What the paper is about Regression and Classification Flexible
More informationDistance dependent Chinese restaurant processes
David M. Blei Department of Computer Science, Princeton University 35 Olden St., Princeton, NJ 08540 Peter Frazier Department of Operations Research and Information Engineering, Cornell University 232
More informationClustering bi-partite networks using collapsed latent block models
Clustering bi-partite networks using collapsed latent block models Jason Wyse, Nial Friel & Pierre Latouche Insight at UCD Laboratoire SAMM, Université Paris 1 Mail: jason.wyse@ucd.ie Insight Latent Space
More informationMarkov and Gibbs Random Fields
Markov and Gibbs Random Fields Bruno Galerne bruno.galerne@parisdescartes.fr MAP5, Université Paris Descartes Master MVA Cours Méthodes stochastiques pour l analyse d images Lundi 6 mars 2017 Outline The
More informationTopic Modelling and Latent Dirichlet Allocation
Topic Modelling and Latent Dirichlet Allocation Stephen Clark (with thanks to Mark Gales for some of the slides) Lent 2013 Machine Learning for Language Processing: Lecture 7 MPhil in Advanced Computer
More informationStatistical Approaches to Learning and Discovery
Statistical Approaches to Learning and Discovery Bayesian Model Selection Zoubin Ghahramani & Teddy Seidenfeld zoubin@cs.cmu.edu & teddy@stat.cmu.edu CALD / CS / Statistics / Philosophy Carnegie Mellon
More informationPrinciples of Bayesian Inference
Principles of Bayesian Inference Sudipto Banerjee and Andrew O. Finley 2 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department of Forestry & Department
More informationPrinciples of Bayesian Inference
Principles of Bayesian Inference Sudipto Banerjee 1 and Andrew O. Finley 2 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department of Forestry & Department
More informationBayesian Nonparametric Learning of Complex Dynamical Phenomena
Duke University Department of Statistical Science Bayesian Nonparametric Learning of Complex Dynamical Phenomena Emily Fox Joint work with Erik Sudderth (Brown University), Michael Jordan (UC Berkeley),
More informationHierarchical Models, Nested Models and Completely Random Measures
See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/238729763 Hierarchical Models, Nested Models and Completely Random Measures Article March 2012
More informationMarkov Chain Monte Carlo (MCMC)
Markov Chain Monte Carlo (MCMC Dependent Sampling Suppose we wish to sample from a density π, and we can evaluate π as a function but have no means to directly generate a sample. Rejection sampling can
More informationPrinciples of Bayesian Inference
Principles of Bayesian Inference Sudipto Banerjee University of Minnesota July 20th, 2008 1 Bayesian Principles Classical statistics: model parameters are fixed and unknown. A Bayesian thinks of parameters
More informationInfinite-State Markov-switching for Dynamic. Volatility Models : Web Appendix
Infinite-State Markov-switching for Dynamic Volatility Models : Web Appendix Arnaud Dufays 1 Centre de Recherche en Economie et Statistique March 19, 2014 1 Comparison of the two MS-GARCH approximations
More informationProbabilistic Graphical Models
Probabilistic Graphical Models Lecture 9: Variational Inference Relaxations Volkan Cevher, Matthias Seeger Ecole Polytechnique Fédérale de Lausanne 24/10/2011 (EPFL) Graphical Models 24/10/2011 1 / 15
More informationMarkov chain Monte Carlo
1 / 26 Markov chain Monte Carlo Timothy Hanson 1 and Alejandro Jara 2 1 Division of Biostatistics, University of Minnesota, USA 2 Department of Statistics, Universidad de Concepción, Chile IAP-Workshop
More informationUnsupervised Activity Perception by Hierarchical Bayesian Models
Unsupervised Activity Perception by Hierarchical Bayesian Models Xiaogang Wang Xiaoxu Ma Eric Grimson Computer Science and Artificial Intelligence Lab Massachusetts Tnstitute of Technology, Cambridge,
More informationIntroduction to Probabilistic Machine Learning
Introduction to Probabilistic Machine Learning Piyush Rai Dept. of CSE, IIT Kanpur (Mini-course 1) Nov 03, 2015 Piyush Rai (IIT Kanpur) Introduction to Probabilistic Machine Learning 1 Machine Learning
More informationComputational statistics
Computational statistics Markov Chain Monte Carlo methods Thierry Denœux March 2017 Thierry Denœux Computational statistics March 2017 1 / 71 Contents of this chapter When a target density f can be evaluated
More informationMachine Learning Summer School, Austin, TX January 08, 2015
Parametric Department of Information, Risk, and Operations Management Department of Statistics and Data Sciences The University of Texas at Austin Machine Learning Summer School, Austin, TX January 08,
More informationCMPS 242: Project Report
CMPS 242: Project Report RadhaKrishna Vuppala Univ. of California, Santa Cruz vrk@soe.ucsc.edu Abstract The classification procedures impose certain models on the data and when the assumption match the
More informationRecent Advances in Bayesian Inference Techniques
Recent Advances in Bayesian Inference Techniques Christopher M. Bishop Microsoft Research, Cambridge, U.K. research.microsoft.com/~cmbishop SIAM Conference on Data Mining, April 2004 Abstract Bayesian
More informationMarkov Random Fields and Bayesian Image Analysis. Wei Liu Advisor: Tom Fletcher
Markov Random Fields and Bayesian Image Analysis Wei Liu Advisor: Tom Fletcher 1 Markov Random Field: Application Overview Awate and Whitaker 2006 2 Markov Random Field: Application Overview 3 Markov Random
More informationModelling and Tracking of Dynamic Spectra Using a Non-Parametric Bayesian Method
Proceedings of Acoustics 213 - Victor Harbor 17 2 November 213, Victor Harbor, Australia Modelling and Tracking of Dynamic Spectra Using a Non-Parametric Bayesian Method ABSTRACT Dragana Carevic Maritime
More informationCSC 412 (Lecture 4): Undirected Graphical Models
CSC 412 (Lecture 4): Undirected Graphical Models Raquel Urtasun University of Toronto Feb 2, 2016 R Urtasun (UofT) CSC 412 Feb 2, 2016 1 / 37 Today Undirected Graphical Models: Semantics of the graph:
More informationLatent Dirichlet Bayesian Co-Clustering
Latent Dirichlet Bayesian Co-Clustering Pu Wang 1, Carlotta Domeniconi 1, and athryn Blackmond Laskey 1 Department of Computer Science Department of Systems Engineering and Operations Research George Mason
More informationSmall-variance Asymptotics for Dirichlet Process Mixtures of SVMs
Small-variance Asymptotics for Dirichlet Process Mixtures of SVMs Yining Wang Jun Zhu Tsinghua University July, 2014 Y. Wang and J. Zhu (Tsinghua University) Max-Margin DP-means July, 2014 1 / 25 Outline
More informationA Bayesian Nonparametric Model for Predicting Disease Status Using Longitudinal Profiles
A Bayesian Nonparametric Model for Predicting Disease Status Using Longitudinal Profiles Jeremy Gaskins Department of Bioinformatics & Biostatistics University of Louisville Joint work with Claudio Fuentes
More informationProbabilistic modeling of NLP
Structured Bayesian Nonparametric Models with Variational Inference ACL Tutorial Prague, Czech Republic June 24, 2007 Percy Liang and Dan Klein Probabilistic modeling of NLP Document clustering Topic modeling
More informationScaling Neighbourhood Methods
Quick Recap Scaling Neighbourhood Methods Collaborative Filtering m = #items n = #users Complexity : m * m * n Comparative Scale of Signals ~50 M users ~25 M items Explicit Ratings ~ O(1M) (1 per billion)
More informationSequence labeling. Taking collective a set of interrelated instances x 1,, x T and jointly labeling them
HMM, MEMM and CRF 40-957 Special opics in Artificial Intelligence: Probabilistic Graphical Models Sharif University of echnology Soleymani Spring 2014 Sequence labeling aking collective a set of interrelated
More informationSimple approximate MAP inference for Dirichlet processes mixtures
Vol. 0 (2015) 1 8 Simple approximate MAP inference for Dirichlet processes mixtures Yordan P. Raykov Aston University e-mail: yordan.raykov@gmail.com Alexis Boukouvalas University of Manchester e-mail:
More information