Celeste: Variational inference for a generative model of astronomical images

Size: px
Start display at page:

Download "Celeste: Variational inference for a generative model of astronomical images"

Transcription

1 Celeste: Variational inference for a generative model of astronomical images Jerey Regier Statistics Department UC Berkeley July 9, 2015 Joint work with Jon McAulie (UCB Statistics), Andrew Miller, Ryan Adams, Matt Homan, Dustin Lang, David Schlegel, and Prabhat

2 An astronomical image An image from the Sloan Digital Sky Survey, showing a galaxy from the constellation Serpens, 100 million light years from Earth, along with several other galaxies and many stars from our own galaxy. 2 / 20

3 Project goals Catalog all galaxies and stars that are visible through the next generation of telescopes. The Large Synoptic Survey Telescope, for example, will house a 3200-megapixel camera producing 8 terabytes of images nightly. Identify promising galaxies for spectrograph targeting. Better understand dark energy and the geometry of the universe. Develop an extensible model and inference procedure, for use by the astronomical community. Future applications might include nding supernovae and detecting killer asteroids. 3 / 20

4 The Celeste graphical model 4 / 20

5 Brightness and colors latent random variable r s models brightness of source s in the reference band the arriving quantity of energy in band r ( rs (as = i) Gamma Υ (i), Φ (i)). latent random vector c s models the B 1 colors of source s color = log ratio of brightness in adjacent bands ( ks (as = i) Categorical cs (ks = d, as = i) MvNormal Ξ (i) 1,..., Ξ(i) D ), ( Ω (i,d), Λ (i,d)). Then, the brightness l sb of each light source s in each band b is a deterministic function of r s and c s. 5 / 20

6 Colors: scientic priors 6 / 20

7 Galaxies: morphology 7 / 20

8 Galaxies: light-density model [ ] cos ϕs sin ϕ R s = s, sin ϕ s cos ϕ s W s = R s [ σ 2 s 0 0 σ 2 s ρ 2 s ] R s, J h si (w ) = η ij φ (w ; µ s, ν ij W s ), i = 0 or 1 j=1 h s (w ) = θ s h s1 (w ) + (1 θ s )h s0 (w ). 8 / 20

9 Idealized sky view Let δ µs denote the Dirac delta functionthe prole of a star. Then brightness for sky position w is S G(w ) = l sb g sas (w ) s=1 where g si (w ) = { δ µs (w ), if i = 0 (star") h s (w ), if i = 1 (galaxy"). 9 / 20

10 Point spread function Credit: Sloan Digital Sky Survey 10 / 20

11 Point spread function PSF mixture of K Gaussians: f nbm (w ) = K ᾱ nbk φ ( w m ; w + ξ ) nbk, τ nbk. k=1 φ is the bivariate normal density. K and the (ᾱ nb, ξ nb, τ nb ) are the parameters of the PSF, determined a priori. 11 / 20

12 Likelihood: idealized sky view + PSF Convolve intermediate sky view w/psf photon arrival rate for pixel m: G nbm = G(w )f (w ) dw S = l sb g sas (w )f (w ) dw. s=1 These normal-normal convolutions are analytic. For stars, f s0 (m) := g s0 (w )f (w ) dw K = ᾱ nbk φ ( m; µ s + ξ ) nbk, τ nbk. k=1 Let θ s1 = θ s and θ s2 = 1 θ s. For galaxies, f s1 (m) := g s1 (w )f (w ) dw = K ᾱ nk 2 θ si k=1 i=1 j=1 J η 1j φ ( m; µ s + ξ ) nbk, τ nbk + ν ij W s. 12 / 20

13 Likelihood Let a = (a s ) S, s=1 r = (r s) S, and s=1 c = (c s) S s=1. Then the expected number of photons landing in pixel m is F nbm (a, r, c) = ι nb [ɛ nb + G nbm ]. For n = 1,..., N, b = 1,..., B, and m = 1,..., M, we model x nbm a, r, c Poisson (F nbm (a, r, c)). 13 / 20

14 Intractable posterior Let Θ = (a, r, c). The posterior on Θ is intractable because of coupling between the sources: p(θ x) = p(x Θ)p(Θ) p(x) and p(x) = = p(x Θ)p(Θ) d Θ N B M p(x nbm Θ)p(Θ) dθ. n=1 b=1 m=1 14 / 20

15 Variational inference Let Q be a family of distributions on latent variables Θ. For q Q, log p(x) E q [log p(x, Θ)] E q [log q(θ)] = log p(x) D KL (q(θ), p(θ x)) =: L(q). Variational inference approximates the exact posterior with a simpler distribution q = arg max q Q L(q). 15 / 20

16 Variational inference: advantages and limitations Advantages: potentially orders of magnitude faster the MCMC no post-processing of samplescan compute statistics of the approximating distribution nearly instantaneously Limitations: biasand few error bounds are known for statistics based on an approximating distribution rather than the true posterior may require modeling changes, to avoid intractable expectations may necessitate solving dicult optimization problems, even if all expectations are tractable 16 / 20

17 Variational optimization...isn't easy L (χ, µ, κ, γ, ζ, β, λ, θ, ρ, σ, ϕ) { N B M = C + n=1 b=1 m=1 x nbm log [ S a {0,1} S s=1 ɛ nb + S s=1 χ a s s (1 χ s) 1 a s { r 1 b r b 1 r s exp {c sb } exp {c sb } j=b j=b r 3 ᾱ nk φ ( ] ) m w; ξ nbk, Σ nbk gsi (w) dw c 1 k 1 k=1 S b r b 1 3 ι nb r s exp {c sb } exp {c sb } ᾱ nk φ ( m w; ξ nbk, Σ ) nbk gsi (w) dw s=1 j=b j=b r k=1 } } dr 1 dc 1 dk 1... dr S dc S dk S { S 2 D KL (q(a s), p(a s)) + χ a s s s=1 i=1 (1 χ s) 1 a s [D KL (q(r s a s = i), p(r s a s = i)) + D KL (q (k s, c s a s = i), p s (k s, c s a s = i))] r S c S k S }. 17 / 20

18 However, A structured mean-eld assumption that factorizes across light sources makes most expectations tractable: q(θ) = S q(θ s ). s=1 The delta method for variational inference approximates the remainining expectations. Existing catalogs provide good initial settings for the variational parameters. Light sources are unlikely to contribute photons to distant pixels. 18 / 20

19 Posterior predictive check on a stamp Left is a 51 pixel 51 pixel sub-region of an astronomical image, captured through the r band lter. Each pixel's value corresponds to the number of photons that hit it. The right panel shows E q [F nbm ], the mean of x nbm with respect to our posterior approximation. 19 / 20

20 Results dataset real synthetic model primary celeste improve celeste improve n position (.00) (.01) 654 missed gals 28 / / (.01) 14 / (.01) 654 missed stars 8 / / (.01) 6 / (.01) 654 brightness (.12) (.08) 654 color u-g (.04) (.05) 582 color g-r (.01) (.02) 654 color r-i (.00) (.01) 654 color i-z (.01) (.02) 654 prole (.02) (.02) 237 eccentricity (.01) (.01) 237 scale (.17) (.04) 237 angle (.80) (.80) / 20

Learning an astronomical catalog of the visible universe through scalable Bayesian inference

Learning an astronomical catalog of the visible universe through scalable Bayesian inference Learning an astronomical catalog of the visible universe through scalable Bayesian inference Jeffrey Regier with the DESI Collaboration Department of Electrical Engineering and Computer Science UC Berkeley

More information

Celeste: Scalable variational inference for a generative model of astronomical images

Celeste: Scalable variational inference for a generative model of astronomical images Celeste: Scalable variational inference for a generative model of astronomical images Jeffrey Regier 1, Brenton Partridge 2, Jon McAuliffe 1, Ryan Adams 2, Matt Hoffman 3, Dustin Lang 4, David Schlegel

More information

Celeste: Variational inference for a generative model of astronomical images

Celeste: Variational inference for a generative model of astronomical images : Variational inference for a generative model of astronomical images Jeffrey Regier, University of California, Berkeley Andrew Miller, Harvard University Jon McAuliffe, University of California, Berkeley

More information

Ages of stellar populations from color-magnitude diagrams. Paul Baines. September 30, 2008

Ages of stellar populations from color-magnitude diagrams. Paul Baines. September 30, 2008 Ages of stellar populations from color-magnitude diagrams Paul Baines Department of Statistics Harvard University September 30, 2008 Context & Example Welcome! Today we will look at using hierarchical

More information

Latent Variable Models

Latent Variable Models Latent Variable Models Stefano Ermon, Aditya Grover Stanford University Lecture 5 Stefano Ermon, Aditya Grover (AI Lab) Deep Generative Models Lecture 5 1 / 31 Recap of last lecture 1 Autoregressive models:

More information

Variational Scoring of Graphical Model Structures

Variational Scoring of Graphical Model Structures Variational Scoring of Graphical Model Structures Matthew J. Beal Work with Zoubin Ghahramani & Carl Rasmussen, Toronto. 15th September 2003 Overview Bayesian model selection Approximations using Variational

More information

GAUSSIAN PROCESS REGRESSION

GAUSSIAN PROCESS REGRESSION GAUSSIAN PROCESS REGRESSION CSE 515T Spring 2015 1. BACKGROUND The kernel trick again... The Kernel Trick Consider again the linear regression model: y(x) = φ(x) w + ε, with prior p(w) = N (w; 0, Σ). The

More information

Hierarchical Bayesian Modeling

Hierarchical Bayesian Modeling Hierarchical Bayesian Modeling Making scientific inferences about a population based on many individuals Angie Wolfgang NSF Postdoctoral Fellow, Penn State Astronomical Populations Once we discover an

More information

Variational Methods in Bayesian Deconvolution

Variational Methods in Bayesian Deconvolution PHYSTAT, SLAC, Stanford, California, September 8-, Variational Methods in Bayesian Deconvolution K. Zarb Adami Cavendish Laboratory, University of Cambridge, UK This paper gives an introduction to the

More information

Robust and accurate inference via a mixture of Gaussian and terrors

Robust and accurate inference via a mixture of Gaussian and terrors Robust and accurate inference via a mixture of Gaussian and terrors Hyungsuk Tak ICTS/SAMSI Time Series Analysis for Synoptic Surveys and Gravitational Wave Astronomy 22 March 2017 Joint work with Justin

More information

Hmms with variable dimension structures and extensions

Hmms with variable dimension structures and extensions Hmm days/enst/january 21, 2002 1 Hmms with variable dimension structures and extensions Christian P. Robert Université Paris Dauphine www.ceremade.dauphine.fr/ xian Hmm days/enst/january 21, 2002 2 1 Estimating

More information

Density Estimation. Seungjin Choi

Density Estimation. Seungjin Choi Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/

More information

Overlapping Astronomical Sources: Utilizing Spectral Information

Overlapping Astronomical Sources: Utilizing Spectral Information Overlapping Astronomical Sources: Utilizing Spectral Information David Jones Advisor: Xiao-Li Meng Collaborators: Vinay Kashyap (CfA) and David van Dyk (Imperial College) CHASC Astrostatistics Group April

More information

Deep Poisson Factorization Machines: a factor analysis model for mapping behaviors in journalist ecosystem

Deep Poisson Factorization Machines: a factor analysis model for mapping behaviors in journalist ecosystem 000 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050

More information

Probabilistic Catalogs and beyond...

Probabilistic Catalogs and beyond... Department of Statistics The University of Auckland https://www.stat.auckland.ac.nz/ brewer/ Probabilistic catalogs So, what is a probabilistic catalog? And what s beyond? Finite Mixture Models Finite

More information

Pattern Recognition and Machine Learning. Bishop Chapter 9: Mixture Models and EM

Pattern Recognition and Machine Learning. Bishop Chapter 9: Mixture Models and EM Pattern Recognition and Machine Learning Chapter 9: Mixture Models and EM Thomas Mensink Jakob Verbeek October 11, 27 Le Menu 9.1 K-means clustering Getting the idea with a simple example 9.2 Mixtures

More information

Riemann Manifold Methods in Bayesian Statistics

Riemann Manifold Methods in Bayesian Statistics Ricardo Ehlers ehlers@icmc.usp.br Applied Maths and Stats University of São Paulo, Brazil Working Group in Statistical Learning University College Dublin September 2015 Bayesian inference is based on Bayes

More information

3% 5% 1% 2% d t = 1,, T d i j J i,d,t J i,d,t+ = J i,d,t j j a i,d,t j i t d γ i > 0 i j t d U i,j,d,t = μ i,j,d + β i,j,d,t γ i + ε i,j,d,t ε i,j,d,t t ε i,,d,t μ β μ μ i,j,d d t d, t j μ i,j,d i j d

More information

Bayesian Learning. HT2015: SC4 Statistical Data Mining and Machine Learning. Maximum Likelihood Principle. The Bayesian Learning Framework

Bayesian Learning. HT2015: SC4 Statistical Data Mining and Machine Learning. Maximum Likelihood Principle. The Bayesian Learning Framework HT5: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford http://www.stats.ox.ac.uk/~sejdinov/sdmml.html Maximum Likelihood Principle A generative model for

More information

Bayesian Machine Learning

Bayesian Machine Learning Bayesian Machine Learning Andrew Gordon Wilson ORIE 6741 Lecture 2: Bayesian Basics https://people.orie.cornell.edu/andrew/orie6741 Cornell University August 25, 2016 1 / 17 Canonical Machine Learning

More information

Algorithms for Variational Learning of Mixture of Gaussians

Algorithms for Variational Learning of Mixture of Gaussians Algorithms for Variational Learning of Mixture of Gaussians Instructors: Tapani Raiko and Antti Honkela Bayes Group Adaptive Informatics Research Center 28.08.2008 Variational Bayesian Inference Mixture

More information

Study Notes on the Latent Dirichlet Allocation

Study Notes on the Latent Dirichlet Allocation Study Notes on the Latent Dirichlet Allocation Xugang Ye 1. Model Framework A word is an element of dictionary {1,,}. A document is represented by a sequence of words: =(,, ), {1,,}. A corpus is a collection

More information

Integrated Non-Factorized Variational Inference

Integrated Non-Factorized Variational Inference Integrated Non-Factorized Variational Inference Shaobo Han, Xuejun Liao and Lawrence Carin Duke University February 27, 2014 S. Han et al. Integrated Non-Factorized Variational Inference February 27, 2014

More information

Binocular Targets: Omega Centauri is also a good object to view in binoculars.

Binocular Targets: Omega Centauri is also a good object to view in binoculars. Eridanus Optics CC April 2006 Targets in Centaurus The following three targets are selected from the Constellation 'Centaurus' to present a naked eye challenge, as well as telescopic challenges. Binoculars

More information

20: Gaussian Processes

20: Gaussian Processes 10-708: Probabilistic Graphical Models 10-708, Spring 2016 20: Gaussian Processes Lecturer: Andrew Gordon Wilson Scribes: Sai Ganesh Bandiatmakuri 1 Discussion about ML Here we discuss an introduction

More information

Tutorial on Gaussian Processes and the Gaussian Process Latent Variable Model

Tutorial on Gaussian Processes and the Gaussian Process Latent Variable Model Tutorial on Gaussian Processes and the Gaussian Process Latent Variable Model (& discussion on the GPLVM tech. report by Prof. N. Lawrence, 06) Andreas Damianou Department of Neuro- and Computer Science,

More information

Cheng Soon Ong & Christian Walder. Canberra February June 2017

Cheng Soon Ong & Christian Walder. Canberra February June 2017 Cheng Soon Ong & Christian Walder Research Group and College of Engineering and Computer Science Canberra February June 2017 (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 679 Part XIX

More information

Astronomical image reduction using the Tractor

Astronomical image reduction using the Tractor the Tractor DECaLS Fin Astronomical image reduction using the Tractor Dustin Lang McWilliams Postdoc Fellow Carnegie Mellon University visiting University of Waterloo UW / 2015-03-31 1 Astronomical image

More information

Contrastive Divergence

Contrastive Divergence Contrastive Divergence Training Products of Experts by Minimizing CD Hinton, 2002 Helmut Puhr Institute for Theoretical Computer Science TU Graz June 9, 2010 Contents 1 Theory 2 Argument 3 Contrastive

More information

. α β γ δ ε ζ η θ ι κ λ μ Aμ ν(x) ξ ο π ρ ς σ τ υ φ χ ψ ω. Α Β Γ Δ Ε Ζ Η Θ Ι Κ Λ Μ Ν Ξ Ο Π Ρ Σ Τ Υ Φ Χ Ψ Ω. Friday April 1 ± ǁ

. α β γ δ ε ζ η θ ι κ λ μ Aμ ν(x) ξ ο π ρ ς σ τ υ φ χ ψ ω. Α Β Γ Δ Ε Ζ Η Θ Ι Κ Λ Μ Ν Ξ Ο Π Ρ Σ Τ Υ Φ Χ Ψ Ω. Friday April 1 ± ǁ . α β γ δ ε ζ η θ ι κ λ μ Aμ ν(x) ξ ο π ρ ς σ τ υ φ χ ψ ω. Α Β Γ Δ Ε Ζ Η Θ Ι Κ Λ Μ Ν Ξ Ο Π Ρ Σ Τ Υ Φ Χ Ψ Ω Friday April 1 ± ǁ 1 Chapter 5. Photons: Covariant Theory 5.1. The classical fields 5.2. Covariant

More information

Variational Principal Components

Variational Principal Components Variational Principal Components Christopher M. Bishop Microsoft Research 7 J. J. Thomson Avenue, Cambridge, CB3 0FB, U.K. cmbishop@microsoft.com http://research.microsoft.com/ cmbishop In Proceedings

More information

Bayesian Image Segmentation Using MRF s Combined with Hierarchical Prior Models

Bayesian Image Segmentation Using MRF s Combined with Hierarchical Prior Models Bayesian Image Segmentation Using MRF s Combined with Hierarchical Prior Models Kohta Aoki 1 and Hiroshi Nagahashi 2 1 Interdisciplinary Graduate School of Science and Engineering, Tokyo Institute of Technology

More information

Chapter 16. Structured Probabilistic Models for Deep Learning

Chapter 16. Structured Probabilistic Models for Deep Learning Peng et al.: Deep Learning and Practice 1 Chapter 16 Structured Probabilistic Models for Deep Learning Peng et al.: Deep Learning and Practice 2 Structured Probabilistic Models way of using graphs to describe

More information

Hierarchical Bayesian Modeling

Hierarchical Bayesian Modeling Hierarchical Bayesian Modeling Making scientific inferences about a population based on many individuals Angie Wolfgang NSF Postdoctoral Fellow, Penn State Astronomical Populations Once we discover an

More information

Parametric Techniques Lecture 3

Parametric Techniques Lecture 3 Parametric Techniques Lecture 3 Jason Corso SUNY at Buffalo 22 January 2009 J. Corso (SUNY at Buffalo) Parametric Techniques Lecture 3 22 January 2009 1 / 39 Introduction In Lecture 2, we learned how to

More information

Variational Bayesian Inference for Parametric and Non-Parametric Regression with Missing Predictor Data

Variational Bayesian Inference for Parametric and Non-Parametric Regression with Missing Predictor Data for Parametric and Non-Parametric Regression with Missing Predictor Data August 23, 2010 Introduction Bayesian inference For parametric regression: long history (e.g. Box and Tiao, 1973; Gelman, Carlin,

More information

Unsupervised Learning

Unsupervised Learning Unsupervised Learning Bayesian Model Comparison Zoubin Ghahramani zoubin@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc in Intelligent Systems, Dept Computer Science University College

More information

Hierarchical Bayesian Modeling with Approximate Bayesian Computation

Hierarchical Bayesian Modeling with Approximate Bayesian Computation Hierarchical Bayesian Modeling with Approximate Bayesian Computation Jessi Cisewski Carnegie Mellon University June 2014 Goal of lecture Bring the ideas of Hierarchical Bayesian Modeling (HBM), Approximate

More information

Image segmentation combining Markov Random Fields and Dirichlet Processes

Image segmentation combining Markov Random Fields and Dirichlet Processes Image segmentation combining Markov Random Fields and Dirichlet Processes Jessica SODJO IMS, Groupe Signal Image, Talence Encadrants : A. Giremus, J.-F. Giovannelli, F. Caron, N. Dobigeon Jessica SODJO

More information

A spatio-temporal model for extreme precipitation simulated by a climate model

A spatio-temporal model for extreme precipitation simulated by a climate model A spatio-temporal model for extreme precipitation simulated by a climate model Jonathan Jalbert Postdoctoral fellow at McGill University, Montréal Anne-Catherine Favre, Claude Bélisle and Jean-François

More information

Survey of JPEG compression history analysis

Survey of JPEG compression history analysis Survey of JPEG compression history analysis Andrew B. Lewis Computer Laboratory Topics in security forensic signal analysis References Neelamani et al.: JPEG compression history estimation for color images,

More information

Lecture 13 : Variational Inference: Mean Field Approximation

Lecture 13 : Variational Inference: Mean Field Approximation 10-708: Probabilistic Graphical Models 10-708, Spring 2017 Lecture 13 : Variational Inference: Mean Field Approximation Lecturer: Willie Neiswanger Scribes: Xupeng Tong, Minxing Liu 1 Problem Setup 1.1

More information

An Example of Telescope Resolution

An Example of Telescope Resolution An Example of Telescope Resolution J. Kielkopf September 23, 2012 1 Principles Light leaves a distant source with the properties of a spherical wave. That is, the phase of the wave is constant on the surface

More information

Dark Matter on Small Scales: Merging and Cosmogony. David W. Hogg New York University CCPP

Dark Matter on Small Scales: Merging and Cosmogony. David W. Hogg New York University CCPP Dark Matter on Small Scales: Merging and Cosmogony David W. Hogg New York University CCPP summary galaxy merger rates suggest growth of ~1 percent per Gyr galaxy evolution is over can we rule out CDM now

More information

Bayesian Hidden Markov Models and Extensions

Bayesian Hidden Markov Models and Extensions Bayesian Hidden Markov Models and Extensions Zoubin Ghahramani Department of Engineering University of Cambridge joint work with Matt Beal, Jurgen van Gael, Yunus Saatci, Tom Stepleton, Yee Whye Teh Modeling

More information

PROBABILISTIC PROGRAMMING: BAYESIAN MODELLING MADE EASY. Arto Klami

PROBABILISTIC PROGRAMMING: BAYESIAN MODELLING MADE EASY. Arto Klami PROBABILISTIC PROGRAMMING: BAYESIAN MODELLING MADE EASY Arto Klami 1 PROBABILISTIC PROGRAMMING Probabilistic programming is to probabilistic modelling as deep learning is to neural networks (Antti Honkela,

More information

A Review of Pseudo-Marginal Markov Chain Monte Carlo

A Review of Pseudo-Marginal Markov Chain Monte Carlo A Review of Pseudo-Marginal Markov Chain Monte Carlo Discussed by: Yizhe Zhang October 21, 2016 Outline 1 Overview 2 Paper review 3 experiment 4 conclusion Motivation & overview Notation: θ denotes the

More information

OPTICAL PHOTOMETRY. Observational Astronomy (2011) 1

OPTICAL PHOTOMETRY. Observational Astronomy (2011) 1 OPTICAL PHOTOMETRY Observational Astronomy (2011) 1 The optical photons coming from an astronomical object (star, galaxy, quasar, etc) can be registered in the pixels of a frame (or image). Using a ground-based

More information

Joint Optimization of Segmentation and Appearance Models

Joint Optimization of Segmentation and Appearance Models Joint Optimization of Segmentation and Appearance Models David Mandle, Sameep Tandon April 29, 2013 David Mandle, Sameep Tandon (Stanford) April 29, 2013 1 / 19 Overview 1 Recap: Image Segmentation 2 Optimization

More information

Data Processing in DES

Data Processing in DES Data Processing in DES Brian Yanny Oct 28, 2016 http://data.darkenergysurvey.org/fnalmisc/talk/detrend.p Basic Signal-to-Noise calculation in astronomy: Assuming a perfect atmosphere (fixed PSF of p arcsec

More information

Variable selection for model-based clustering

Variable selection for model-based clustering Variable selection for model-based clustering Matthieu Marbac (Ensai - Crest) Joint works with: M. Sedki (Univ. Paris-sud) and V. Vandewalle (Univ. Lille 2) The problem Objective: Estimation of a partition

More information

Fast Likelihood-Free Inference via Bayesian Optimization

Fast Likelihood-Free Inference via Bayesian Optimization Fast Likelihood-Free Inference via Bayesian Optimization Michael Gutmann https://sites.google.com/site/michaelgutmann University of Helsinki Aalto University Helsinki Institute for Information Technology

More information

Statistical Tools and Techniques for Solar Astronomers

Statistical Tools and Techniques for Solar Astronomers Statistical Tools and Techniques for Solar Astronomers Alexander W Blocker Nathan Stein SolarStat 2012 Outline Outline 1 Introduction & Objectives 2 Statistical issues with astronomical data 3 Example:

More information

STA 4273H: Sta-s-cal Machine Learning

STA 4273H: Sta-s-cal Machine Learning STA 4273H: Sta-s-cal Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 2 In our

More information

PROBABILISTIC PROGRAMMING: BAYESIAN MODELLING MADE EASY

PROBABILISTIC PROGRAMMING: BAYESIAN MODELLING MADE EASY PROBABILISTIC PROGRAMMING: BAYESIAN MODELLING MADE EASY Arto Klami Adapted from my talk in AIHelsinki seminar Dec 15, 2016 1 MOTIVATING INTRODUCTION Most of the artificial intelligence success stories

More information

Probabilistic Time Series Classification

Probabilistic Time Series Classification Probabilistic Time Series Classification Y. Cem Sübakan Boğaziçi University 25.06.2013 Y. Cem Sübakan (Boğaziçi University) M.Sc. Thesis Defense 25.06.2013 1 / 54 Problem Statement The goal is to assign

More information

Modeling conditional distributions with mixture models: Applications in finance and financial decision-making

Modeling conditional distributions with mixture models: Applications in finance and financial decision-making Modeling conditional distributions with mixture models: Applications in finance and financial decision-making John Geweke University of Iowa, USA Journal of Applied Econometrics Invited Lecture Università

More information

Bayesian Machine Learning

Bayesian Machine Learning Bayesian Machine Learning Andrew Gordon Wilson ORIE 6741 Lecture 4 Occam s Razor, Model Construction, and Directed Graphical Models https://people.orie.cornell.edu/andrew/orie6741 Cornell University September

More information

Probabilistic Graphical Models Lecture 20: Gaussian Processes

Probabilistic Graphical Models Lecture 20: Gaussian Processes Probabilistic Graphical Models Lecture 20: Gaussian Processes Andrew Gordon Wilson www.cs.cmu.edu/~andrewgw Carnegie Mellon University March 30, 2015 1 / 53 What is Machine Learning? Machine learning algorithms

More information

13: Variational inference II

13: Variational inference II 10-708: Probabilistic Graphical Models, Spring 2015 13: Variational inference II Lecturer: Eric P. Xing Scribes: Ronghuo Zheng, Zhiting Hu, Yuntian Deng 1 Introduction We started to talk about variational

More information

Document and Topic Models: plsa and LDA

Document and Topic Models: plsa and LDA Document and Topic Models: plsa and LDA Andrew Levandoski and Jonathan Lobo CS 3750 Advanced Topics in Machine Learning 2 October 2018 Outline Topic Models plsa LSA Model Fitting via EM phits: link analysis

More information

Parametric Techniques

Parametric Techniques Parametric Techniques Jason J. Corso SUNY at Buffalo J. Corso (SUNY at Buffalo) Parametric Techniques 1 / 39 Introduction When covering Bayesian Decision Theory, we assumed the full probabilistic structure

More information

Probablistic Graphical Models, Spring 2007 Homework 4 Due at the beginning of class on 11/26/07

Probablistic Graphical Models, Spring 2007 Homework 4 Due at the beginning of class on 11/26/07 Probablistic Graphical odels, Spring 2007 Homework 4 Due at the beginning of class on 11/26/07 Instructions There are four questions in this homework. The last question involves some programming which

More information

Make Your Own Radio Image Large Public Venue Edition

Make Your Own Radio Image Large Public Venue Edition Make Your Own Radio Image Large Public Venue Edition Adapted from the NRAO s Make Your Own Radio Image Background This Activity has been adapted from the NRAO s Make Your Own Radio Image. Radio telescopes

More information

The Expectation-Maximization Algorithm

The Expectation-Maximization Algorithm 1/29 EM & Latent Variable Models Gaussian Mixture Models EM Theory The Expectation-Maximization Algorithm Mihaela van der Schaar Department of Engineering Science University of Oxford MLE for Latent Variable

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 7 Approximate

More information

Introduction to Systems Analysis and Decision Making Prepared by: Jakub Tomczak

Introduction to Systems Analysis and Decision Making Prepared by: Jakub Tomczak Introduction to Systems Analysis and Decision Making Prepared by: Jakub Tomczak 1 Introduction. Random variables During the course we are interested in reasoning about considered phenomenon. In other words,

More information

Basic math for biology

Basic math for biology Basic math for biology Lei Li Florida State University, Feb 6, 2002 The EM algorithm: setup Parametric models: {P θ }. Data: full data (Y, X); partial data Y. Missing data: X. Likelihood and maximum likelihood

More information

Clustering bi-partite networks using collapsed latent block models

Clustering bi-partite networks using collapsed latent block models Clustering bi-partite networks using collapsed latent block models Jason Wyse, Nial Friel & Pierre Latouche Insight at UCD Laboratoire SAMM, Université Paris 1 Mail: jason.wyse@ucd.ie Insight Latent Space

More information

G8325: Variational Bayes

G8325: Variational Bayes G8325: Variational Bayes Vincent Dorie Columbia University Wednesday, November 2nd, 2011 bridge Variational University Bayes Press 2003. On-screen viewing permitted. Printing not permitted. http://www.c

More information

Introduction to Probabilistic Machine Learning

Introduction to Probabilistic Machine Learning Introduction to Probabilistic Machine Learning Piyush Rai Dept. of CSE, IIT Kanpur (Mini-course 1) Nov 03, 2015 Piyush Rai (IIT Kanpur) Introduction to Probabilistic Machine Learning 1 Machine Learning

More information

Chapter 20. Deep Generative Models

Chapter 20. Deep Generative Models Peng et al.: Deep Learning and Practice 1 Chapter 20 Deep Generative Models Peng et al.: Deep Learning and Practice 2 Generative Models Models that are able to Provide an estimate of the probability distribution

More information

Stats 579 Intermediate Bayesian Modeling. Assignment # 2 Solutions

Stats 579 Intermediate Bayesian Modeling. Assignment # 2 Solutions Stats 579 Intermediate Bayesian Modeling Assignment # 2 Solutions 1. Let w Gy) with y a vector having density fy θ) and G having a differentiable inverse function. Find the density of w in general and

More information

CS Lecture 19. Exponential Families & Expectation Propagation

CS Lecture 19. Exponential Families & Expectation Propagation CS 6347 Lecture 19 Exponential Families & Expectation Propagation Discrete State Spaces We have been focusing on the case of MRFs over discrete state spaces Probability distributions over discrete spaces

More information

PMR Learning as Inference

PMR Learning as Inference Outline PMR Learning as Inference Probabilistic Modelling and Reasoning Amos Storkey Modelling 2 The Exponential Family 3 Bayesian Sets School of Informatics, University of Edinburgh Amos Storkey PMR Learning

More information

Supplementary Material: Weakly Supervised Learning of Heterogeneous Concepts in Videos

Supplementary Material: Weakly Supervised Learning of Heterogeneous Concepts in Videos Supplementary Material: Weakly Supervised Learning of Heterogeneous Concepts in Videos Sohil Shah, Kuldeep Kulkarni, Ariit Biswas, Ankit Gandhi, Om Deshmukh, and Larry S. Davis Expectation Constraints

More information

Posterior Regularization

Posterior Regularization Posterior Regularization 1 Introduction One of the key challenges in probabilistic structured learning, is the intractability of the posterior distribution, for fast inference. There are numerous methods

More information

Annealing Between Distributions by Averaging Moments

Annealing Between Distributions by Averaging Moments Annealing Between Distributions by Averaging Moments Chris J. Maddison Dept. of Comp. Sci. University of Toronto Roger Grosse CSAIL MIT Ruslan Salakhutdinov University of Toronto Partition Functions We

More information

Introduction to Machine Learning

Introduction to Machine Learning Introduction to Machine Learning Brown University CSCI 1950-F, Spring 2012 Prof. Erik Sudderth Lecture 20: Expectation Maximization Algorithm EM for Mixture Models Many figures courtesy Kevin Murphy s

More information

Introduction to Bayesian inference

Introduction to Bayesian inference Introduction to Bayesian inference Thomas Alexander Brouwer University of Cambridge tab43@cam.ac.uk 17 November 2015 Probabilistic models Describe how data was generated using probability distributions

More information

Lecture : Probabilistic Machine Learning

Lecture : Probabilistic Machine Learning Lecture : Probabilistic Machine Learning Riashat Islam Reasoning and Learning Lab McGill University September 11, 2018 ML : Many Methods with Many Links Modelling Views of Machine Learning Machine Learning

More information

CS Lecture 18. Topic Models and LDA

CS Lecture 18. Topic Models and LDA CS 6347 Lecture 18 Topic Models and LDA (some slides by David Blei) Generative vs. Discriminative Models Recall that, in Bayesian networks, there could be many different, but equivalent models of the same

More information

Streaming Variational Bayes

Streaming Variational Bayes Streaming Variational Bayes Tamara Broderick, Nicholas Boyd, Andre Wibisono, Ashia C. Wilson, Michael I. Jordan UC Berkeley Discussion led by Miao Liu September 13, 2013 Introduction The SDA-Bayes Framework

More information

Expectation Propagation for Approximate Bayesian Inference

Expectation Propagation for Approximate Bayesian Inference Expectation Propagation for Approximate Bayesian Inference José Miguel Hernández Lobato Universidad Autónoma de Madrid, Computer Science Department February 5, 2007 1/ 24 Bayesian Inference Inference Given

More information

CS839: Probabilistic Graphical Models. Lecture 7: Learning Fully Observed BNs. Theo Rekatsinas

CS839: Probabilistic Graphical Models. Lecture 7: Learning Fully Observed BNs. Theo Rekatsinas CS839: Probabilistic Graphical Models Lecture 7: Learning Fully Observed BNs Theo Rekatsinas 1 Exponential family: a basic building block For a numeric random variable X p(x ) =h(x)exp T T (x) A( ) = 1

More information

Expectation Maximization

Expectation Maximization Expectation Maximization Aaron C. Courville Université de Montréal Note: Material for the slides is taken directly from a presentation prepared by Christopher M. Bishop Learning in DAGs Two things could

More information

Part 1: Expectation Propagation

Part 1: Expectation Propagation Chalmers Machine Learning Summer School Approximate message passing and biomedicine Part 1: Expectation Propagation Tom Heskes Machine Learning Group, Institute for Computing and Information Sciences Radboud

More information

Bayesian Spatial Model Selection for Detection and Identification in Chemical Plumes Based on Hyperspectral Imagery Data

Bayesian Spatial Model Selection for Detection and Identification in Chemical Plumes Based on Hyperspectral Imagery Data Bayesian Spatial Model Selection for Detection and Identification in Chemical Plumes Based on Hyperspectral Imagery Data - UC, Santa cruz Nicole Mendoza - UC, Santa Cruz Conference on Applied Statistics

More information

Image Processing in Astronomy: Current Practice & Challenges Going Forward

Image Processing in Astronomy: Current Practice & Challenges Going Forward Image Processing in Astronomy: Current Practice & Challenges Going Forward Mario Juric University of Washington With thanks to Andy Connolly, Robert Lupton, Ian Sullivan, David Reiss, and the LSST DM Team

More information

Diversity-Promoting Bayesian Learning of Latent Variable Models

Diversity-Promoting Bayesian Learning of Latent Variable Models Diversity-Promoting Bayesian Learning of Latent Variable Models Pengtao Xie 1, Jun Zhu 1,2 and Eric Xing 1 1 Machine Learning Department, Carnegie Mellon University 2 Department of Computer Science and

More information

Learning Bayesian network : Given structure and completely observed data

Learning Bayesian network : Given structure and completely observed data Learning Bayesian network : Given structure and completely observed data Probabilistic Graphical Models Sharif University of Technology Spring 2017 Soleymani Learning problem Target: true distribution

More information

Probabilistic Reasoning in Deep Learning

Probabilistic Reasoning in Deep Learning Probabilistic Reasoning in Deep Learning Dr Konstantina Palla, PhD palla@stats.ox.ac.uk September 2017 Deep Learning Indaba, Johannesburgh Konstantina Palla 1 / 39 OVERVIEW OF THE TALK Basics of Bayesian

More information

Efficient Likelihood-Free Inference

Efficient Likelihood-Free Inference Efficient Likelihood-Free Inference Michael Gutmann http://homepages.inf.ed.ac.uk/mgutmann Institute for Adaptive and Neural Computation School of Informatics, University of Edinburgh 8th November 2017

More information

Practical Bayesian Optimization of Machine Learning. Learning Algorithms

Practical Bayesian Optimization of Machine Learning. Learning Algorithms Practical Bayesian Optimization of Machine Learning Algorithms CS 294 University of California, Berkeley Tuesday, April 20, 2016 Motivation Machine Learning Algorithms (MLA s) have hyperparameters that

More information

Latent Dirichlet Alloca/on

Latent Dirichlet Alloca/on Latent Dirichlet Alloca/on Blei, Ng and Jordan ( 2002 ) Presented by Deepak Santhanam What is Latent Dirichlet Alloca/on? Genera/ve Model for collec/ons of discrete data Data generated by parameters which

More information

Robust MCMC Sampling with Non-Gaussian and Hierarchical Priors

Robust MCMC Sampling with Non-Gaussian and Hierarchical Priors Division of Engineering & Applied Science Robust MCMC Sampling with Non-Gaussian and Hierarchical Priors IPAM, UCLA, November 14, 2017 Matt Dunlop Victor Chen (Caltech) Omiros Papaspiliopoulos (ICREA,

More information

Real Astronomy from Virtual Observatories

Real Astronomy from Virtual Observatories THE US NATIONAL VIRTUAL OBSERVATORY Real Astronomy from Virtual Observatories Robert Hanisch Space Telescope Science Institute US National Virtual Observatory About this presentation What is a Virtual

More information

Bayesian model selection for computer model validation via mixture model estimation

Bayesian model selection for computer model validation via mixture model estimation Bayesian model selection for computer model validation via mixture model estimation Kaniav Kamary ATER, CNAM Joint work with É. Parent, P. Barbillon, M. Keller and N. Bousquet Outline Computer model validation

More information

Surprise Detection in Multivariate Astronomical Data Kirk Borne George Mason University

Surprise Detection in Multivariate Astronomical Data Kirk Borne George Mason University Surprise Detection in Multivariate Astronomical Data Kirk Borne George Mason University kborne@gmu.edu, http://classweb.gmu.edu/kborne/ Outline What is Surprise Detection? Example Application: The LSST

More information

Cosmology with the Sloan Digital Sky Survey Supernova Search. David Cinabro

Cosmology with the Sloan Digital Sky Survey Supernova Search. David Cinabro Cosmology with the Sloan Digital Sky Survey Supernova Search David Cinabro Cosmology Background The study of the origin and evolution of the Universe. First real effort by Einstein in 1916 Gravity is all

More information