A marginal sampler for σ-stable Poisson-Kingman mixture models
|
|
- Jeffrey Marshall
- 5 years ago
- Views:
Transcription
1 A marginal sampler for σ-stable Poisson-Kingman mixture models joint work with Yee Whye Teh and Stefano Favaro María Lomelí Gatsby Unit, University College London Talk at the BNP 10 Raleigh, North Carolina June 22, 2015
2 Table of Contents I Latent variable models Infinite mixture models Poisson-Kingman processes Marginal and conditional samplers Conclusions
3 Latent variable models The world is complex so we need compositional models. These are built from blocks of simple models stacking them together. We would like to expand the grammar of models that we can do inference with. [Picture borrowed from Zoubin Ghahramani s Unsupervised Learning notes]
4 Mixture models Let {Y i } n i=1 be our data. A mixture model is an example of a latent variable model which has a single discrete latent variable per observation X i Discrete(π) Y i X i f ( θ xi ). Under the discrete distribution P(X i = m) = π m, π m 0, k i=1 π m = 1 and P(Y i dy i ) = = k m=1 k m=1 P(Y i dy i X i )P(X i = m) π m F (dy i θ m ).
5 Mixture model For density estimation and/or clustering tasks, we could do the following: 1. Choose the number of mixture components k. 2. Learn model parameters {θ m } k m=1. We could repeat these steps for a bunch of k, say k {1, 2, 3}, and then carry out some model selection procedure to select the best model.
6 Infinite mixture models An infinite mixture model is a mixture model with potentially infinitely many mixture components. It can be formulated in the following hierarchical specification: P Random probability measure (RPM) X i P P Y i X i F Xi. Which RPMs can we choose?
7 Dirichlet process (DP) P KP NRM σ P K(h T ) Gibbs-Type σ σ (0,1) CC Gamma Tilted σ=0 Lamperti Tilted Q σ (F τ ) τ=0, θ> σ MDP ML θ>0 σ<0 NGG(σ, τ) PY(θ, σ) σ=0.5 τ=0 σ 0 θ=0 NIG(τ) NS(σ) DP(θ) MFSD σ 0 [Ferguson, 1973]
8 Pitman-Yor process (PY) P KP NRM σ P K(h T ) Gibbs-Type σ σ (0,1) CC Gamma Tilted σ=0 Lamperti Tilted Q σ (F τ ) τ=0, θ> σ MDP ML θ>0 σ<0 NGG(σ, τ) PY(θ, σ) σ=0.5 τ=0 σ 0 θ=0 NIG(τ) NS(σ) DP(θ) MFSD σ 0 [Perman et al., 1992, Pitman and Yor, 1997]
9 Pitman-Yor process (PY) P KP NRM σ P K(h T ) Gibbs-Type σ σ (0,1) CC Gamma Tilted σ=0 Lamperti Tilted Q σ (F τ ) τ=0, θ> σ MDP ML θ>0 σ<0 NGG(τ, σ) PY(θ, σ) σ=0.5 τ=0 σ 0 θ=0 NIG(τ) NS(σ) DP(θ) MFSD σ 0 [Perman et al., 1992, Pitman and Yor, 1997]
10 Normalized Stable process (NS) P KP NRM σ P K(h T ) Gibbs-Type σ σ (0,1) CC Gamma Tilted σ=0 Lamperti Tilted Q σ (F τ ) τ=0, θ> σ MDP ML θ>0 σ<0 NGG(τ, σ) PY(θ, σ) σ=0.5 τ=0 σ 0 θ=0 NIG(τ) NS(σ) DP(θ) MFSD σ 0 [Kingman, 1975]
11 Normalized Stable process (NS) P KP NRM σ P K(h T ) Gibbs-Type σ σ (0,1) CC Gamma Tilted σ=0 Lamperti Tilted Q σ (F τ ) τ=0, θ> σ MDP ML θ>0 σ<0 NGG(σ, τ) PY(θ, σ) σ=0.5 τ=0 σ 0 θ=0 NIG(τ) NS(σ) DP(θ) MFSD σ 0 [Kingman, 1975]
12 Normalized Inverse Gaussian process (NIG) P KP NRM σ P K(h T ) Gibbs-Type σ σ (0,1) CC Gamma Tilted σ=0 Lamperti Tilted Q σ (F τ ) τ=0, θ> σ MDP ML θ>0 σ<0 NGG(σ, τ) PY(θ, σ) σ=0.5 τ=0 σ 0 θ=0 NIG(τ) NS(σ) DP(θ) MFSD σ 0 [Lijoi et al., 2005]
13 Normalized Random Measures (NRM) P KP NRM σ P K(h T ) Gibbs-Type σ σ (0,1) CC Gamma Tilted σ=0 Lamperti Tilted Q σ (F τ ) τ=0, θ> σ MDP ML θ>0 σ<0 NGG(τ, σ) PY(θ, σ) σ=0.5 τ=0 σ 0 θ=0 NIG(τ) NS(σ) DP(θ) MFSD σ 0 [Regazzini et al., 2003, James et al., 2009]
14 Normalized Generalized Gamma (NGG) P KP NRM σ P K(h T ) Gibbs-Type σ σ (0,1) CC Gamma Tilted σ=0 Lamperti Tilted Q σ (F τ ) τ=0, θ> σ MDP ML θ>0 σ<0 NGG(τ, σ) PY(θ, σ) σ=0.5 τ=0 σ 0 θ=0 NIG(τ) NS(σ) DP(θ) MFSD σ 0 [Brix, 1999]
15 Q class P KP NRM σ P K(h T ) Gibbs-Type σ σ (0,1) CC Gamma Tilted σ=0 Lamperti Tilted Q σ (F τ ) τ=0, θ> σ MDP ML F τ =δ τ F τ =F τ σ,θ σ<0 NGG(τ, σ) PY(θ, σ) σ=0.5 τ=0 σ 0 θ=0 NIG(τ) NS(σ) DP(θ) MFSD σ 0 [James, 2013] and our paper [Favaro et al., 2014]
16 σ-stable Poisson-Kingman processes (σ-pk) PKP NRM σ-pk(h T ) Gibbs-Type σ σ (0,1) Lamperti Tilted CC h 2,T Gamma Tilted σ=0 h 1,T Q σ (F τ ) τ=0, θ> σ h 3,T MDP σ<0 ML F τ =δ τ F τ =F τ σ,θ NGG(τ, σ) PY(θ, σ) σ=0.5 τ=0 σ 0 θ=0 NIG(τ) NS(σ) DP(θ) MFSD σ 0 [Pitman, 2003]
17 σ-stable Poisson-Kingman processes (σ-pk) PKP NRM σ-pk(h T ) Gibbs-Type σ σ (0,1) Lamperti Tilted CC h 2,T Gamma Tilted σ=0 h 1,T Q σ (F τ ) τ=0, θ> σ h 3,T MDP σ<0 ML F τ =δ τ F τ =F τ σ,θ NGG(τ, σ) PY(θ, σ) σ=0.5 τ=0 σ 0 θ=0 NIG(τ) NS(σ) DP(θ) MFSD σ 0 [Pitman, 2003]
18 Poisson-Kingman processes (PKP) PKP NRM σ-pk(h T ) Gibbs-Type σ σ (0,1) Lamperti Tilted CC h 2,T Gamma Tilted σ=0 h 1,T Q σ (F τ ) τ=0, θ> σ h 3,T MDP σ<0 ML F τ =δ τ F τ =F τ σ,θ NGG(τ, σ) PY(θ, σ) σ=0.5 τ=0 σ 0 θ=0 NIG(τ) NS(σ) DP(θ) MFSD σ 0 [Pitman, 2003]
19 σ-stable Poisson-Kingman processes PKP NRM σ-pk(h T ) Gibbs-Type σ σ (0,1) Lamperti Tilted CC h 2,T Gamma Tilted σ=0 h 1,T Q σ (F τ ) τ=0, θ> σ h 3,T MDP σ<0 ML F τ =δ τ F τ =F τ σ,θ NGG(τ, σ) PY(θ, σ) σ=0.5 τ=0 σ 0 θ=0 NIG(τ) NS(σ) DP(θ) MFSD σ 0 [Pitman, 2003]
20 Completely random measures Let X be a Polish space with the Borel σ-algebra B (X). A completely random measure (CRM) µ is a random measure on X such that, for all measurable disjoint sets A, B, µ(a) µ(b). µ is an a.s. discrete measure of the form µ = W i δ Xi. i=1 The distribution of the random weights and locations is characterised by an intensity measure ν. These form a Poisson process over R + X. If it factorizes ν = ρ H 0 then the CRM(ρ, H 0 ) is homogeneous. [Kingman, 1967]
21 Normalization of completely random measures If we take µ and normalize it by its total mass T = µ(x) = i=1 W i, we obtain a normalized random measure P = i=1 W i δ Xi denoted by P NRM(ρ, H 0 ). µ CRM(ρ, H 0 ) P = µ T
22 Poisson-Kingman processes As a generalisation of a CRM, a Poisson Kingman process is given by the following hierarchical specification T γ T (dt) h(dt)f T (dt) G T = t PK(ρ, H 0, δ t ) We call the random probability measure G PK(ρ, H 0, γ) a Poisson-Kingman RPM with Lévy measure ρ, base distribution H 0 and mixing distribution γ. [Pitman, 2003]
23 Poisson-Kingman mixture models We can define a hierarchical model such that T γ h f T G T = t PK(ρ σ, H 0, δ t ) X i G G Y i X i F (X i ) (1) How can we perform inference with this class of infinite mixture models?
24 Ilustration: Size biased sampling PKP T γ V 1 SBS σ (T ) V 1 Represented mass : V 1 Surplus mass: T V 1
25 Ilustration: Size biased sampling PKP X 1 H 0 V 1 X 1 = x 1 Represented mass : V 1 Surplus mass: T V 1
26 Ilustration: Size biased sampling PKP V 2 SBS σ (T V 1 ) V 1 V 2 X 1 = x 1 Represented mass : V 1 + V 2 Surplus mass: T V 1 V 2
27 Ilustration: Size biased sampling PKP X 2 H 0 X 2 = x 2 V 1 V 2 X 1 = x 1 Represented mass : V 1 + V 2 Surplus mass: T V 1 V 2
28 Ilustration: Size biased sampling PKP V 3 SBS σ (T 2 k=1 V k ) X 2 = x 2 V 1 V 2 V 3 X 1 = x 1 Represented mass : V 1 + V 2 + V 3 Surplus mass: T 3 k=1 V k
29 Ilustration: Size biased sampling PKP X 3 H 0 X 2 = x 2 V 1 V 2 V 3 X 3 = x 3 X 1 = x 1 Represented mass : V 1 + V 2 + V 3 Surplus mass: T 3 k=1 V k
30 Ilustration: Size biased sampling PKP X 4 = x 1 X 2 = x 2 V 1 V 2 V 3 X 3 = x 3 X 1 = x 1 Represented mass : V 1 + V 2 + V 3 Surplus mass: T 3 k=1 V k
31 Ilustration: Size biased sampling PKP V 4 SBS σ (T 3 k=1 V k ) X 4 = x 1 X 2 = x 2 V 1 V 2 V 3 V 4 X 3 = x 3 X 1 = x 1 Represented mass : 4 k=1 V k Surplus mass: T 4 k=1 V k
32 Ilustration: Size biased sampling PKP X 4 H 0 X 4 = x 1 X 2 = x 2 X 5 = x 4 V 1 V 2 V 3 V 4 X 3 = x 3 X 1 = x 1 Represented mass : 4 k=1 V k Surplus mass: T 4 k=1 V k
33 Ilustration: Size biased sampling PKP V 5 SBS σ (T 4 k=1 V k ) X 4 = x 1 X 2 = x 2 X 5 = x 4 V 1 V 2 V 3 V 4 V 5 X 3 = x 3 X 1 = x 1 Represented mass : 5 k=1 V k Surplus mass: T 5 k=1 V k
34 Ilustration: Size biased sampling PKP X 5 H 0 X 4 = x 1 X 2 = x 2 X 5 = x 4 X 6 = x 5 V 1 V 2 V 3 V 4 V 5 X 3 = x 3 X 1 = x 1 Represented mass : 5 k=1 V k Surplus mass: T 5 k=1 V k
35 Ilustration: Size biased sampling PKP X 4 = x 1 X 2 = x 2 X 5 = x 4 X 6 = x 5 V 1 V 2 V 3 V 4 V 5 X 3 = x 3 X 7 = x 5 X 1 = x 1 Represented mass : 5 k=1 V k Surplus mass: T 5 k=1 V k
36 Ilustration: Size biased sampling PKP X 4 = x 1 X 2 = x 2 X 5 = x 4 X 6 = x 5 V 1 V 2 V 3 V 4 V 5 X 3 = x 3 X 7 = x 5 X 8 = x 4 X 1 = x 1 Represented mass : 5 k=1 V k Surplus mass: T 5 k=1 V k
37 Joint distribution We can size-bias sample n times and there will be ties. This induces a random partition of size K n = k of n with frequencies n i for i = 1,, k. The corresponding joint distribution is P (Π n = π, {V j dv j } k j=1, {X j dxj }k t) γ(dt) (2) j=1 Can we build an exact MCMC sampler using this construction?
38 1. Marginal samplers. Integrate out the infinite dimensional component, just simulate from the partition induced by it. y 2 y 4 y 6 x 1 y 3 x 2 x 3 x 4 y 7 y 1 y 5 y 8 [MacEachern and Müller, 1998], [Neal, 2000] for DP mixture models,[favaro and Teh, 2013] for NRM mixture models.
39 Marginal sampler s intractabilities After integrating out the size-biased weights, we obtain [f σ (t)] 1 P σ (Π n = π, {X j dx j }k j=1 t) = σ k Γ(n σk) (t σ ) k 0 1 p n σk 1 f σ ((1 p)t) dp [1 σ] ni 1 H 0(dxk ) k i=1 [Kanter, 1975] s σ-stable density s integral representation, [Tanner and Wong, 1987] s data augmentation scheme.
40 σ-stable Poisson-Kingman augmented mixture model Let {Y i } i [n] be n data points, {Xj } are the cluster parameters, the c π joint distribution for P, W = T σ 1 σ, Z, Πn {X j } c π,{y i } i [n] is P σ (Π n = π, P dp, W dw, Z dz) P ({X c dx c } c π, {Y i dy i } i [n] π) = σ k πγ(n σk) w (1 σ)k 2 p n σk 1 (1 p) 1 A(z) 1 σ A(z) exp [ (1 p) σ 1 σ w ] h(w 1 σ σ ) [1 σ] c 1 H 0 (xc ) F (y i xc ). c π i c
41 σ-stable Poisson-Kingman augmented mixture model Let {Y i } i [n] be n data points, {Xj } are the cluster parameters, the c π joint distribution for P, W = T σ 1 σ, Z, Πn {X j } c π,{y i } i [n] is P σ (Π n = π, P dp, W dw, Z dz) P ({X c dx c } c π, {Y i dy i } i [n] π) = Prior Likelihood Posterior
42 σ-stable Poisson-Kingman augmented mixture model h(t) Z T σ P Π H 0 Y i i [n] X c c π Figure: Graphical model for the augmented mixture model
43 Chinese restaurant-like component-wise assigment P σ (l c π i, x, rest) c σ Γ(n σ π ) f (y i x c, {y j } j c ) 1 M σt σ p σ Γ(n σ π +1)) f (y i x new ) l c π i l [M] Algorithm 1 σ-pkmarginalsampler( h T (t), σ, M, H 0, {y i } n i=1 ) for t = 2 T do Update z: Slice sample P (Z dz rest) Update p : Slice sample P (P dp rest) Update t: Slice sample P (T dt rest) Update π, {x c } c π : ReUse(Π n, C, {X c } c Πn, {y i } n i=1, rest) end for Extension of Reuse algorithm from [Favaro and Teh, 2013], M is the number of empty clusters.
44 X 1 X 1 X 1 X 1 X 1 X X 1 1 X 2 X 2 X 2 X 2 X 2 2 X X 2 X 3 X 3 X 3 1 X3 4 1 X3 4 3 X X3 X4 X4 X 4 X 4 X 4 3 X X 5 X 5 X 5 X 5 X 5 X 6 X 6 Figure: An iteration of the component wise cluster assignment, M = 2 empty clusters.
45 Conditional MCMC samplers 2. Conditional samplers. Simulate from the joint posterior without integrating out the weights. These schemes rely on finding a finite representation of the infinite dimensional probability measure. Figure: Slice- conditional sampler [Ishwaran and James, 2001] [Papaspiliopoulos and Roberts, 2008], [Walker, 2007], [Kalli et al., 2011] for DP mixture models, [Favaro and Walker, 2012] for σ-pk mixture models.
46 Marginal sampler s performance assessment Algorithm σ Running time ESS(±std) Pitma-Yor process (θ = 10) Marginal (84.104) ( ) Conditional 0.3 NA NA Marginal (37.077) ( ) Conditional ( ) (41.475) Marginal (52.514) ( ) Conditional 0.7 NA NA Normalized Stable process Marginal ( ) ( ) Conditional 0.3 NA NA Marginal ( ) ( ) Conditional (22.647) ( ) Marginal (85.264) ( ) Conditional 0.7 NA NA Normalized Generalized Gamma process (τ = 1) Marginal (208.98) ( ) Conditional 0.3 NA NA Marginal (93.164) ( ) Conditional ( ) (70.148) Marginal ( ) ( ) Conditional 0.7 NA NA Comparison with our implementation of [Favaro and Walker, 2012] s slice conditional sampler.
47 Multidimensional experiment: olive oil dataset The dataset from [de la Mata-Espinosa et al., 2011] consists of the triacylglyceride profiles of n = 120 oil samples.
48 σ-stable Poisson-Kingman mixture of multivariate Normals 1. PCA on the original dataset (n = 120, d = 4000 dimensional vectors) taking the first 8 principal components which explained 97% of the variance. 2. Prior specification. σ-stable Poisson-Kingman mixture model with parameters (σ, h T (t)) for different h-tilting functions. 3. Likelihood specification Let X be the projected data set, x i R d F (x i φ c ) = Multivariate Normal (x i µ ci, R ci ) H 0 = Multivariate Normal(µ µ 0, R r 0 )Inverse Wishart(R ν, S).
49 Results Examples DP PY NGG Gamma tilted NS Lamperti tilted Average predictive probability e-12 (7.6848e-13) e-12 (7.5549e-13) e-12 (3.4035e-12) e-12(1.5767e-12) e-12(9.7106e-13) e-12 (1.0538e-12) Cluster size = Cluster size = Cluster size = Cluster size = Dendrogram with Matlab linkage routine Cluster size =6
50 Conclusions General purpose MCMC algorithm for a wide class of infinite mixture models. This class is natural in the sense that it is part of Gibbs type priors with σ (0, 1) [De Blasi et al., 2015]. Sequential version of our sampler, uses Sequential Monte Carlo (ongoing work with Pierre Jacob). Benefit from conditional and marginal sampler s perks: exploit the generative model s description (ongoing work).
51 Main References Lomeli, M. and Favaro, S. and Teh, Y.W. A marginal sampler for σ-stable Poisson-Kingman mixture models. Arxiv Favaro, S. and Lomeli, M. and Teh, Y. W. A tractable class of σ-stable Poisson-Kingman processes and an effective marginalized sampler. Statistics and Computing, Many thanks to the Gatsby Charitable Foundation and to the BNP-ISBA travel award for funding.
52 References I Brix, A. (1999). Generalized gamma measures and shot-noise Cox processes. Advances in Applied Probability, 31: De Blasi, P., Favaro, S., Lijoi, A., Mena, R. H., Prüenster, I., and Ruggiero, M. (2015). Are gibbs-type priors the most natural generalization of the Dirichlet process? In IEEE Transactions on Pattern Analysis & Machine Intelligence, volume 37, pages de la Mata-Espinosa, P., Bosque-Sendra, J., Bro, R., and Cuadros-Rodriguez, L. (2011). Olive oil quantification of edible vegetable oil blends using triacylglycerols chromatographic fingerprints and chemometric tools. Talanta, 85: Favaro, S., Lomeli, M., and Teh, Y. W. (2014). A tractable class of σ-stable Poisson-Kingman processes and an effective marginalized sampler. Statistics and Computing, in press.
53 References II Favaro, S. and Teh, Y. W. (2013). MCMC for normalized random measure mixture models. Statistical Science, 28(3): Favaro, S. and Walker, S. G. (2012). Slice sampling sigma-stable Poisson-Kingman mixture models. Journal of Computational and Graphical Statistics (To appear), 1:1. Ferguson, T. S. (1973). A Bayesian analysis of some nonparametric problems. Annals of Statistics, 1(2): Ishwaran, H. and James, L. F. (2001). Gibbs sampling methods for stick breaking priors. Journal of the American Statistical Association, 96: James, L. F. (2013). Stick-Breaking PG(α, ψ)-generalized Gamma processes. ArXiv: James, L. F., Lijoi, A., and Prüenster, I. (2009). Posterior analysis for normalized random measures with independent increments. Scandinavian Journal of Statistics, 36:76 97.
54 References III Kalli, M., Griffiths, J., and Walker, S. (2011). Slice sampling mixture models. Statistics and Computing, 21: Kanter, M. (1975). Stable Densities Under Change of Scale and Total Variation Inequalities. The Annals of Probability, 3: Kingman, J. C. (1967). Completely random measures. Pacific Journal of Mathematics, 21: Kingman, J. F. C. (1975). Random discrete distributions. Journal of the Royal Statistical Society, 37:1 22. Lijoi, A., Mena, R. H., and Prüenster, I. (2005). Hierarchical mixture modelling with normalized inverse-gaussian priors. Journal of the American Statistical Association, 100: MacEachern, S. N. and Müller, P. (1998). Estimating mixtures of Dirichlet process models. Journal of Computational and Graphical Statistics, 7:
55 References IV Neal, R. (2000). Markov chain sampling methods for Dirichlet Process mixture models. Journal of Computational and Graphical Statistics, 1(2):1. Papaspiliopoulos, O. and Roberts, G. (2008). Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models. Biometrika, 95: Perman, M., Pitman, J., and Yor, M. (1992). Size-biased sampling of Poisson point processes and excursions. Probability Theory and Related Fields, 92(1): Pitman, J. (2003). Poisson-Kingman partitions. In Goldstein, D. R., editor, Statistics and Science: a Festschrift for Terry Speed, pages Institute of Mathematical Statistics. Pitman, J. and Yor, M. (1997). The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator. Annals of Probability, 25:
56 References V Regazzini, E., Lijoi, A., and Prüenster, I. (2003). Distributional results for means of random measures with independent increments. Annals of Statistics, 31: Tanner, M. and Wong, W. (1987). The calculation of posterior distributions by data augmentation. Journal of the American Statistical Association, 82: Walker, S. (2007). Sampling the Dirichlet mixture model with slices. Communications in Statistics: Simulation and Computation, 36:45 54.
57 Appendix: Conditions on the CRM s Lévy measure We want the CRM to have infinitely many atoms We want it to have finite total mass 0 0 ρ(w)dw = [1 exp ( w)] ρ(w)dw
58 Algorithm 2 ReUse(Π n, M, {X c } c Πn, {y i } n i=1,rest) Draw {Xj e}m i.i.d. j=1 H 0 for i = 1 n do Let c Π n be such that i c c c {i} if c = then k DiscreteUniform( 1 M ) Xk e X c Π n Π n {c} end if Set c according to Pr[assign i to cluster c rest] if c [M] then Π n Π n {{i}} X {i} X c e Xc e H 0 else c c {i} end if end for
Slice sampling σ stable Poisson Kingman mixture models
ISSN 2279-9362 Slice sampling σ stable Poisson Kingman mixture models Stefano Favaro S.G. Walker No. 324 December 2013 www.carloalberto.org/research/working-papers 2013 by Stefano Favaro and S.G. Walker.
More informationBayesian Nonparametrics: Dirichlet Process
Bayesian Nonparametrics: Dirichlet Process Yee Whye Teh Gatsby Computational Neuroscience Unit, UCL http://www.gatsby.ucl.ac.uk/~ywteh/teaching/npbayes2012 Dirichlet Process Cornerstone of modern Bayesian
More informationTruncation error of a superposed gamma process in a decreasing order representation
Truncation error of a superposed gamma process in a decreasing order representation B julyan.arbel@inria.fr Í www.julyanarbel.com Inria, Mistis, Grenoble, France Joint work with Igor Pru nster (Bocconi
More informationDependent hierarchical processes for multi armed bandits
Dependent hierarchical processes for multi armed bandits Federico Camerlenghi University of Bologna, BIDSA & Collegio Carlo Alberto First Italian meeting on Probability and Mathematical Statistics, Torino
More informationBayesian nonparametric models for bipartite graphs
Bayesian nonparametric models for bipartite graphs François Caron Department of Statistics, Oxford Statistics Colloquium, Harvard University November 11, 2013 F. Caron 1 / 27 Bipartite networks Readers/Customers
More informationBayesian Nonparametrics: some contributions to construction and properties of prior distributions
Bayesian Nonparametrics: some contributions to construction and properties of prior distributions Annalisa Cerquetti Collegio Nuovo, University of Pavia, Italy Interview Day, CETL Lectureship in Statistics,
More informationSlice Sampling Mixture Models
Slice Sampling Mixture Models Maria Kalli, Jim E. Griffin & Stephen G. Walker Centre for Health Services Studies, University of Kent Institute of Mathematics, Statistics & Actuarial Science, University
More informationOn the posterior structure of NRMI
On the posterior structure of NRMI Igor Prünster University of Turin, Collegio Carlo Alberto and ICER Joint work with L.F. James and A. Lijoi Isaac Newton Institute, BNR Programme, 8th August 2007 Outline
More informationDiscussion of On simulation and properties of the stable law by L. Devroye and L. James
Stat Methods Appl (2014) 23:371 377 DOI 10.1007/s10260-014-0269-4 Discussion of On simulation and properties of the stable law by L. Devroye and L. James Antonio Lijoi Igor Prünster Accepted: 16 May 2014
More informationBayesian Nonparametric Models for Ranking Data
Bayesian Nonparametric Models for Ranking Data François Caron 1, Yee Whye Teh 1 and Brendan Murphy 2 1 Dept of Statistics, University of Oxford, UK 2 School of Mathematical Sciences, University College
More informationDependent Random Measures and Prediction
Dependent Random Measures and Prediction Igor Prünster University of Torino & Collegio Carlo Alberto 10th Bayesian Nonparametric Conference Raleigh, June 26, 2015 Joint wor with: Federico Camerlenghi,
More informationBayesian nonparametric latent feature models
Bayesian nonparametric latent feature models Indian Buffet process, beta process, and related models François Caron Department of Statistics, Oxford Applied Bayesian Statistics Summer School Como, Italy
More informationA Brief Overview of Nonparametric Bayesian Models
A Brief Overview of Nonparametric Bayesian Models Eurandom Zoubin Ghahramani Department of Engineering University of Cambridge, UK zoubin@eng.cam.ac.uk http://learning.eng.cam.ac.uk/zoubin Also at Machine
More informationOn some distributional properties of Gibbs-type priors
On some distributional properties of Gibbs-type priors Igor Prünster University of Torino & Collegio Carlo Alberto Bayesian Nonparametrics Workshop ICERM, 21st September 2012 Joint work with: P. De Blasi,
More informationA Nonparametric Model for Stationary Time Series
A Nonparametric Model for Stationary Time Series Isadora Antoniano-Villalobos Bocconi University, Milan, Italy. isadora.antoniano@unibocconi.it Stephen G. Walker University of Texas at Austin, USA. s.g.walker@math.utexas.edu
More informationTruncation error of a superposed gamma process in a decreasing order representation
Truncation error of a superposed gamma process in a decreasing order representation Julyan Arbel Inria Grenoble, Université Grenoble Alpes julyan.arbel@inria.fr Igor Prünster Bocconi University, Milan
More informationBayesian Nonparametric Models
Bayesian Nonparametric Models David M. Blei Columbia University December 15, 2015 Introduction We have been looking at models that posit latent structure in high dimensional data. We use the posterior
More informationPriors for Random Count Matrices with Random or Fixed Row Sums
Priors for Random Count Matrices with Random or Fixed Row Sums Mingyuan Zhou Joint work with Oscar Madrid and James Scott IROM Department, McCombs School of Business Department of Statistics and Data Sciences
More informationCompound Random Measures
Compound Random Measures Jim Griffin (joint work with Fabrizio Leisen) University of Kent Introduction: Two clinical studies 3 CALGB8881 3 CALGB916 2 2 β 1 1 β 1 1 1 5 5 β 1 5 5 β Infinite mixture models
More informationBayesian Nonparametric Regression for Diabetes Deaths
Bayesian Nonparametric Regression for Diabetes Deaths Brian M. Hartman PhD Student, 2010 Texas A&M University College Station, TX, USA David B. Dahl Assistant Professor Texas A&M University College Station,
More informationNon-Parametric Bayes
Non-Parametric Bayes Mark Schmidt UBC Machine Learning Reading Group January 2016 Current Hot Topics in Machine Learning Bayesian learning includes: Gaussian processes. Approximate inference. Bayesian
More informationGraphical Models for Query-driven Analysis of Multimodal Data
Graphical Models for Query-driven Analysis of Multimodal Data John Fisher Sensing, Learning, & Inference Group Computer Science & Artificial Intelligence Laboratory Massachusetts Institute of Technology
More informationNonparametric Bayesian Methods - Lecture I
Nonparametric Bayesian Methods - Lecture I Harry van Zanten Korteweg-de Vries Institute for Mathematics CRiSM Masterclass, April 4-6, 2016 Overview of the lectures I Intro to nonparametric Bayesian statistics
More informationMAD-Bayes: MAP-based Asymptotic Derivations from Bayes
MAD-Bayes: MAP-based Asymptotic Derivations from Bayes Tamara Broderick Brian Kulis Michael I. Jordan Cat Clusters Mouse clusters Dog 1 Cat Clusters Dog Mouse Lizard Sheep Picture 1 Picture 2 Picture 3
More informationLecture 3a: Dirichlet processes
Lecture 3a: Dirichlet processes Cédric Archambeau Centre for Computational Statistics and Machine Learning Department of Computer Science University College London c.archambeau@cs.ucl.ac.uk Advanced Topics
More informationAn adaptive truncation method for inference in Bayesian nonparametric models
An adaptive truncation method for inference in Bayesian nonparametric models arxiv:1308.045v [stat.co] 1 May 014 J.E. Griffin School of Mathematics, Statistics and Actuarial Science, University of Kent,
More informationOutline. Binomial, Multinomial, Normal, Beta, Dirichlet. Posterior mean, MAP, credible interval, posterior distribution
Outline A short review on Bayesian analysis. Binomial, Multinomial, Normal, Beta, Dirichlet Posterior mean, MAP, credible interval, posterior distribution Gibbs sampling Revisit the Gaussian mixture model
More informationVariational Bayesian Dirichlet-Multinomial Allocation for Exponential Family Mixtures
17th Europ. Conf. on Machine Learning, Berlin, Germany, 2006. Variational Bayesian Dirichlet-Multinomial Allocation for Exponential Family Mixtures Shipeng Yu 1,2, Kai Yu 2, Volker Tresp 2, and Hans-Peter
More informationBayesian nonparametrics
Bayesian nonparametrics 1 Some preliminaries 1.1 de Finetti s theorem We will start our discussion with this foundational theorem. We will assume throughout all variables are defined on the probability
More informationCollapsed Variational Dirichlet Process Mixture Models
Collapsed Variational Dirichlet Process Mixture Models Kenichi Kurihara Dept. of Computer Science Tokyo Institute of Technology, Japan kurihara@mi.cs.titech.ac.jp Max Welling Dept. of Computer Science
More informationBayesian non parametric approaches: an introduction
Introduction Latent class models Latent feature models Conclusion & Perspectives Bayesian non parametric approaches: an introduction Pierre CHAINAIS Bordeaux - nov. 2012 Trajectory 1 Bayesian non parametric
More informationUnsupervised Learning
Unsupervised Learning Bayesian Model Comparison Zoubin Ghahramani zoubin@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc in Intelligent Systems, Dept Computer Science University College
More informationBayesian Methods for Machine Learning
Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),
More informationDependent mixture models: clustering and borrowing information
ISSN 2279-9362 Dependent mixture models: clustering and borrowing information Antonio Lijoi Bernardo Nipoti Igor Pruenster No. 32 June 213 www.carloalberto.org/research/working-papers 213 by Antonio Lijoi,
More informationConstruction of Dependent Dirichlet Processes based on Poisson Processes
1 / 31 Construction of Dependent Dirichlet Processes based on Poisson Processes Dahua Lin Eric Grimson John Fisher CSAIL MIT NIPS 2010 Outstanding Student Paper Award Presented by Shouyuan Chen Outline
More informationDirichlet Processes: Tutorial and Practical Course
Dirichlet Processes: Tutorial and Practical Course (updated) Yee Whye Teh Gatsby Computational Neuroscience Unit University College London August 2007 / MLSS Yee Whye Teh (Gatsby) DP August 2007 / MLSS
More informationAsymptotics for posterior hazards
Asymptotics for posterior hazards Pierpaolo De Blasi University of Turin 10th August 2007, BNR Workshop, Isaac Newton Intitute, Cambridge, UK Joint work with Giovanni Peccati (Université Paris VI) and
More informationBayesian Nonparametrics
Bayesian Nonparametrics Lorenzo Rosasco 9.520 Class 18 April 11, 2011 About this class Goal To give an overview of some of the basic concepts in Bayesian Nonparametrics. In particular, to discuss Dirichelet
More informationInfinite Latent Feature Models and the Indian Buffet Process
Infinite Latent Feature Models and the Indian Buffet Process Thomas L. Griffiths Cognitive and Linguistic Sciences Brown University, Providence RI 292 tom griffiths@brown.edu Zoubin Ghahramani Gatsby Computational
More informationHierarchical Dirichlet Processes
Hierarchical Dirichlet Processes Yee Whye Teh, Michael I. Jordan, Matthew J. Beal and David M. Blei Computer Science Div., Dept. of Statistics Dept. of Computer Science University of California at Berkeley
More informationBayesian Nonparametric Autoregressive Models via Latent Variable Representation
Bayesian Nonparametric Autoregressive Models via Latent Variable Representation Maria De Iorio Yale-NUS College Dept of Statistical Science, University College London Collaborators: Lifeng Ye (UCL, London,
More informationIntroduction to Probabilistic Machine Learning
Introduction to Probabilistic Machine Learning Piyush Rai Dept. of CSE, IIT Kanpur (Mini-course 1) Nov 03, 2015 Piyush Rai (IIT Kanpur) Introduction to Probabilistic Machine Learning 1 Machine Learning
More informationNew Dirichlet Mean Identities
Hong Kong University of Science and Technology Isaac Newton Institute, August 10, 2007 Origins CIFARELLI, D. M. and REGAZZINI, E. (1979). Considerazioni generali sull impostazione bayesiana di problemi
More informationImage segmentation combining Markov Random Fields and Dirichlet Processes
Image segmentation combining Markov Random Fields and Dirichlet Processes Jessica SODJO IMS, Groupe Signal Image, Talence Encadrants : A. Giremus, J.-F. Giovannelli, F. Caron, N. Dobigeon Jessica SODJO
More informationBayesian Nonparametric Mixture, Admixture, and Language Models
Bayesian Nonparametric Mixture, Admixture, and Language Models Yee Whye Teh University of Oxford Nov 2015 Overview Bayesian nonparametrics and random probability measures Mixture models and clustering
More informationVariational Principal Components
Variational Principal Components Christopher M. Bishop Microsoft Research 7 J. J. Thomson Avenue, Cambridge, CB3 0FB, U.K. cmbishop@microsoft.com http://research.microsoft.com/ cmbishop In Proceedings
More informationPattern Recognition and Machine Learning. Bishop Chapter 11: Sampling Methods
Pattern Recognition and Machine Learning Chapter 11: Sampling Methods Elise Arnaud Jakob Verbeek May 22, 2008 Outline of the chapter 11.1 Basic Sampling Algorithms 11.2 Markov Chain Monte Carlo 11.3 Gibbs
More informationDistance-Based Probability Distribution for Set Partitions with Applications to Bayesian Nonparametrics
Distance-Based Probability Distribution for Set Partitions with Applications to Bayesian Nonparametrics David B. Dahl August 5, 2008 Abstract Integration of several types of data is a burgeoning field.
More informationNormalized kernel-weighted random measures
Normalized kernel-weighted random measures Jim Griffin University of Kent 1 August 27 Outline 1 Introduction 2 Ornstein-Uhlenbeck DP 3 Generalisations Bayesian Density Regression We observe data (x 1,
More informationDirichlet Process. Yee Whye Teh, University College London
Dirichlet Process Yee Whye Teh, University College London Related keywords: Bayesian nonparametrics, stochastic processes, clustering, infinite mixture model, Blackwell-MacQueen urn scheme, Chinese restaurant
More informationInfinite latent feature models and the Indian Buffet Process
p.1 Infinite latent feature models and the Indian Buffet Process Tom Griffiths Cognitive and Linguistic Sciences Brown University Joint work with Zoubin Ghahramani p.2 Beyond latent classes Unsupervised
More informationSpatial Normalized Gamma Process
Spatial Normalized Gamma Process Vinayak Rao Yee Whye Teh Presented at NIPS 2009 Discussion and Slides by Eric Wang June 23, 2010 Outline Introduction Motivation The Gamma Process Spatial Normalized Gamma
More informationBayesian Statistics. Debdeep Pati Florida State University. April 3, 2017
Bayesian Statistics Debdeep Pati Florida State University April 3, 2017 Finite mixture model The finite mixture of normals can be equivalently expressed as y i N(µ Si ; τ 1 S i ), S i k π h δ h h=1 δ h
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate
More informationBayesian nonparametric model for sparse dynamic networks
Bayesian nonparametric model for sparse dynamic networks Konstantina Palla, François Caron and Yee Whye Teh 23rd of February, 2018 University of Glasgow Konstantina Palla 1 / 38 BIG PICTURE Interested
More informationGibbs Sampling for (Coupled) Infinite Mixture Models in the Stick Breaking Representation
Gibbs Sampling for (Coupled) Infinite Mixture Models in the Stick Breaking Representation Ian Porteous, Alex Ihler, Padhraic Smyth, Max Welling Department of Computer Science UC Irvine, Irvine CA 92697-3425
More informationDefining Predictive Probability Functions for Species Sampling Models
Defining Predictive Probability Functions for Species Sampling Models Jaeyong Lee Department of Statistics, Seoul National University leejyc@gmail.com Fernando A. Quintana Departamento de Estadísica, Pontificia
More informationOn Consistency of Nonparametric Normal Mixtures for Bayesian Density Estimation
On Consistency of Nonparametric Normal Mixtures for Bayesian Density Estimation Antonio LIJOI, Igor PRÜNSTER, andstepheng.walker The past decade has seen a remarkable development in the area of Bayesian
More informationA Review of Pseudo-Marginal Markov Chain Monte Carlo
A Review of Pseudo-Marginal Markov Chain Monte Carlo Discussed by: Yizhe Zhang October 21, 2016 Outline 1 Overview 2 Paper review 3 experiment 4 conclusion Motivation & overview Notation: θ denotes the
More informationThe Indian Buffet Process: An Introduction and Review
Journal of Machine Learning Research 12 (2011) 1185-1224 Submitted 3/10; Revised 3/11; Published 4/11 The Indian Buffet Process: An Introduction and Review Thomas L. Griffiths Department of Psychology
More information27 : Distributed Monte Carlo Markov Chain. 1 Recap of MCMC and Naive Parallel Gibbs Sampling
10-708: Probabilistic Graphical Models 10-708, Spring 2014 27 : Distributed Monte Carlo Markov Chain Lecturer: Eric P. Xing Scribes: Pengtao Xie, Khoa Luu In this scribe, we are going to review the Parallel
More informationOn the Truncation Error of a Superposed Gamma Process
On the Truncation Error of a Superposed Gamma Process Julyan Arbel and Igor Prünster Abstract Completely random measures (CRMs) form a key ingredient of a wealth of stochastic models, in particular in
More informationBayesian Learning. HT2015: SC4 Statistical Data Mining and Machine Learning. Maximum Likelihood Principle. The Bayesian Learning Framework
HT5: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford http://www.stats.ox.ac.uk/~sejdinov/sdmml.html Maximum Likelihood Principle A generative model for
More informationINFINITE MIXTURES OF MULTIVARIATE GAUSSIAN PROCESSES
INFINITE MIXTURES OF MULTIVARIATE GAUSSIAN PROCESSES SHILIANG SUN Department of Computer Science and Technology, East China Normal University 500 Dongchuan Road, Shanghai 20024, China E-MAIL: slsun@cs.ecnu.edu.cn,
More informationSharing Clusters Among Related Groups: Hierarchical Dirichlet Processes
Sharing Clusters Among Related Groups: Hierarchical Dirichlet Processes Yee Whye Teh (1), Michael I. Jordan (1,2), Matthew J. Beal (3) and David M. Blei (1) (1) Computer Science Div., (2) Dept. of Statistics
More informationHaupthseminar: Machine Learning. Chinese Restaurant Process, Indian Buffet Process
Haupthseminar: Machine Learning Chinese Restaurant Process, Indian Buffet Process Agenda Motivation Chinese Restaurant Process- CRP Dirichlet Process Interlude on CRP Infinite and CRP mixture model Estimation
More informationSTAT Advanced Bayesian Inference
1 / 32 STAT 625 - Advanced Bayesian Inference Meng Li Department of Statistics Jan 23, 218 The Dirichlet distribution 2 / 32 θ Dirichlet(a 1,...,a k ) with density p(θ 1,θ 2,...,θ k ) = k j=1 Γ(a j) Γ(
More informationNonparametric Bayesian Methods: Models, Algorithms, and Applications (Day 5)
Nonparametric Bayesian Methods: Models, Algorithms, and Applications (Day 5) Tamara Broderick ITT Career Development Assistant Professor Electrical Engineering & Computer Science MIT Bayes Foundations
More informationBayesian Nonparametrics
Bayesian Nonparametrics Peter Orbanz Columbia University PARAMETERS AND PATTERNS Parameters P(X θ) = Probability[data pattern] 3 2 1 0 1 2 3 5 0 5 Inference idea data = underlying pattern + independent
More informationOn Simulations form the Two-Parameter. Poisson-Dirichlet Process and the Normalized. Inverse-Gaussian Process
On Simulations form the Two-Parameter arxiv:1209.5359v1 [stat.co] 24 Sep 2012 Poisson-Dirichlet Process and the Normalized Inverse-Gaussian Process Luai Al Labadi and Mahmoud Zarepour May 8, 2018 ABSTRACT
More informationExchangeable random partitions and random discrete probability measures: a brief tour guided by the Dirichlet Process
Exchangeable random partitions and random discrete probability measures: a brief tour guided by the Dirichlet Process Notes for Oxford Statistics Grad Lecture Benjamin Bloem-Reddy benjamin.bloem-reddy@stats.ox.ac.uk
More informationarxiv: v1 [stat.ml] 18 Nov 2015
Tree-Guided MCMC Inference for Normalized Random Measure Mixture Models arxiv:5.05650v [stat.ml] 8 Nov 205 Juho Lee and Seungjin Choi Department of Computer Science and Engineering Pohang University of
More informationBayesian Nonparametric Models on Decomposable Graphs
Bayesian Nonparametric Models on Decomposable Graphs François Caron INRIA Bordeaux Sud Ouest Institut de Mathématiques de Bordeaux University of Bordeaux, France francois.caron@inria.fr Arnaud Doucet Departments
More informationLecture 16 Deep Neural Generative Models
Lecture 16 Deep Neural Generative Models CMSC 35246: Deep Learning Shubhendu Trivedi & Risi Kondor University of Chicago May 22, 2017 Approach so far: We have considered simple models and then constructed
More informationBayesian nonparametric latent feature models
Bayesian nonparametric latent feature models François Caron UBC October 2, 2007 / MLRG François Caron (UBC) Bayes. nonparametric latent feature models October 2, 2007 / MLRG 1 / 29 Overview 1 Introduction
More information13: Variational inference II
10-708: Probabilistic Graphical Models, Spring 2015 13: Variational inference II Lecturer: Eric P. Xing Scribes: Ronghuo Zheng, Zhiting Hu, Yuntian Deng 1 Introduction We started to talk about variational
More informationOn the Stick-Breaking Representation for Homogeneous NRMIs
Bayesian Analysis 216 11, Number3,pp.697 724 On the Stick-Breaking Representation for Homogeneous NRMIs S. Favaro,A.Lijoi,C.Nava,B.Nipoti,I.Prünster,andY.W.Teh Abstract. In this paper, we consider homogeneous
More informationBayesian Hidden Markov Models and Extensions
Bayesian Hidden Markov Models and Extensions Zoubin Ghahramani Department of Engineering University of Cambridge joint work with Matt Beal, Jurgen van Gael, Yunus Saatci, Tom Stepleton, Yee Whye Teh Modeling
More informationA Bayesian Nonparametric Model for Predicting Disease Status Using Longitudinal Profiles
A Bayesian Nonparametric Model for Predicting Disease Status Using Longitudinal Profiles Jeremy Gaskins Department of Bioinformatics & Biostatistics University of Louisville Joint work with Claudio Fuentes
More information19 : Bayesian Nonparametrics: The Indian Buffet Process. 1 Latent Variable Models and the Indian Buffet Process
10-708: Probabilistic Graphical Models, Spring 2015 19 : Bayesian Nonparametrics: The Indian Buffet Process Lecturer: Avinava Dubey Scribes: Rishav Das, Adam Brodie, and Hemank Lamba 1 Latent Variable
More informationParallel Markov Chain Monte Carlo for Pitman-Yor Mixture Models
Parallel Markov Chain Monte Carlo for Pitman-Yor Mixture Models Avinava Dubey School of Computer Science Carnegie Mellon University Pittsburgh, PA 523 Sinead A. Williamson McCombs School of Business University
More informationDistance dependent Chinese restaurant processes
David M. Blei Department of Computer Science, Princeton University 35 Olden St., Princeton, NJ 08540 Peter Frazier Department of Operations Research and Information Engineering, Cornell University 232
More informationCMPS 242: Project Report
CMPS 242: Project Report RadhaKrishna Vuppala Univ. of California, Santa Cruz vrk@soe.ucsc.edu Abstract The classification procedures impose certain models on the data and when the assumption match the
More informationCPSC 540: Machine Learning
CPSC 540: Machine Learning MCMC and Non-Parametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is
More informationKernel adaptive Sequential Monte Carlo
Kernel adaptive Sequential Monte Carlo Ingmar Schuster (Paris Dauphine) Heiko Strathmann (University College London) Brooks Paige (Oxford) Dino Sejdinovic (Oxford) December 7, 2015 1 / 36 Section 1 Outline
More informationIntroduction to Markov Chain Monte Carlo & Gibbs Sampling
Introduction to Markov Chain Monte Carlo & Gibbs Sampling Prof. Nicholas Zabaras Sibley School of Mechanical and Aerospace Engineering 101 Frank H. T. Rhodes Hall Ithaca, NY 14853-3801 Email: zabaras@cornell.edu
More informationAges of stellar populations from color-magnitude diagrams. Paul Baines. September 30, 2008
Ages of stellar populations from color-magnitude diagrams Paul Baines Department of Statistics Harvard University September 30, 2008 Context & Example Welcome! Today we will look at using hierarchical
More informationAdvanced Machine Learning
Advanced Machine Learning Nonparametric Bayesian Models --Learning/Reasoning in Open Possible Worlds Eric Xing Lecture 7, August 4, 2009 Reading: Eric Xing Eric Xing @ CMU, 2006-2009 Clustering Eric Xing
More informationAn Alternative Infinite Mixture Of Gaussian Process Experts
An Alternative Infinite Mixture Of Gaussian Process Experts Edward Meeds and Simon Osindero Department of Computer Science University of Toronto Toronto, M5S 3G4 {ewm,osindero}@cs.toronto.edu Abstract
More informationIntroduction to Machine Learning
Introduction to Machine Learning Brown University CSCI 1950-F, Spring 2012 Prof. Erik Sudderth Lecture 25: Markov Chain Monte Carlo (MCMC) Course Review and Advanced Topics Many figures courtesy Kevin
More informationLecture 7 and 8: Markov Chain Monte Carlo
Lecture 7 and 8: Markov Chain Monte Carlo 4F13: Machine Learning Zoubin Ghahramani and Carl Edward Rasmussen Department of Engineering University of Cambridge http://mlg.eng.cam.ac.uk/teaching/4f13/ Ghahramani
More informationBayesian Nonparametrics for Speech and Signal Processing
Bayesian Nonparametrics for Speech and Signal Processing Michael I. Jordan University of California, Berkeley June 28, 2011 Acknowledgments: Emily Fox, Erik Sudderth, Yee Whye Teh, and Romain Thibaux Computer
More informationBayesian Nonparametrics: Models Based on the Dirichlet Process
Bayesian Nonparametrics: Models Based on the Dirichlet Process Alessandro Panella Department of Computer Science University of Illinois at Chicago Machine Learning Seminar Series February 18, 2013 Alessandro
More informationMarkov Chain Monte Carlo (MCMC)
Markov Chain Monte Carlo (MCMC Dependent Sampling Suppose we wish to sample from a density π, and we can evaluate π as a function but have no means to directly generate a sample. Rejection sampling can
More informationDensity Estimation. Seungjin Choi
Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More informationApproximate Inference using MCMC
Approximate Inference using MCMC 9.520 Class 22 Ruslan Salakhutdinov BCS and CSAIL, MIT 1 Plan 1. Introduction/Notation. 2. Examples of successful Bayesian models. 3. Basic Sampling Algorithms. 4. Markov
More informationSpatial Bayesian Nonparametrics for Natural Image Segmentation
Spatial Bayesian Nonparametrics for Natural Image Segmentation Erik Sudderth Brown University Joint work with Michael Jordan University of California Soumya Ghosh Brown University Parsing Visual Scenes
More informationLecture 10. Announcement. Mixture Models II. Topics of This Lecture. This Lecture: Advanced Machine Learning. Recap: GMMs as Latent Variable Models
Advanced Machine Learning Lecture 10 Mixture Models II 30.11.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de/ Announcement Exercise sheet 2 online Sampling Rejection Sampling Importance
More informationDefault priors for density estimation with mixture models
Bayesian Analysis ) 5, Number, pp. 45 64 Default priors for density estimation with mixture models J.E. Griffin Abstract. The infinite mixture of normals model has become a popular method for density estimation
More informationSupplementary Notes: Segment Parameter Labelling in MCMC Change Detection
Supplementary Notes: Segment Parameter Labelling in MCMC Change Detection Alireza Ahrabian 1 arxiv:1901.0545v1 [eess.sp] 16 Jan 019 Abstract This work addresses the problem of segmentation in time series
More information