Nonparametric Bayesian Methods - Lecture I

Size: px
Start display at page:

Download "Nonparametric Bayesian Methods - Lecture I"

Transcription

1 Nonparametric Bayesian Methods - Lecture I Harry van Zanten Korteweg-de Vries Institute for Mathematics CRiSM Masterclass, April 4-6, 2016

2 Overview of the lectures I Intro to nonparametric Bayesian statistics II Consistency and contraction rates III Contraction rates for Gaussian process priors IV Rate-adaptive BNP, Challenges,... 2 / 50

3 Overview of Lecture I Bayesian statistics Nonparametric Bayesian statistics Nonparametric priors Dirichlet processes distribution function estimation Gaussian processes nonparametric regression Conditionally Gaussian processes Dirichlet mixtures nonparametric density estimation Some more examples Concluding remarks 3 / 50

4 Bayesian statistics 4 / 50

5 Bayesian vs. frequentist statistics Mathematical statistics: Have data X, possible distributions {P θ : θ Θ}. Want to make inference about θ on the basis of X. Paradigms in mathematical statistics: Classical /frequentist paradigm: There is a true value θ 0 Θ. Assume X P θ0. Bayesian paradigm: Think of data as being generated in steps as follows: Parameter is random: θ Π. Terminology Π: prior. Data given parameter: X θ P θ. Can then consider θ X : posterior distribution. 5 / 50

6 Bayesian vs. frequentist statistics Mathematical statistics: Have data X, possible distributions {P θ : θ Θ}. Want to make inference about θ on the basis of X. Paradigms in mathematical statistics: Classical /frequentist paradigm: There is a true value θ 0 Θ. Assume X P θ0. Bayesian paradigm: Think of data as being generated in steps as follows: Parameter is random: θ Π. Terminology Π: prior. Data given parameter: X θ P θ. Can then consider θ X : posterior distribution. 5 / 50

7 Bayesian vs. frequentist statistics Mathematical statistics: Have data X, possible distributions {P θ : θ Θ}. Want to make inference about θ on the basis of X. Paradigms in mathematical statistics: Classical /frequentist paradigm: There is a true value θ 0 Θ. Assume X P θ0. Bayesian paradigm: Think of data as being generated in steps as follows: Parameter is random: θ Π. Terminology Π: prior. Data given parameter: X θ P θ. Can then consider θ X : posterior distribution. 5 / 50

8 Bayes example - 1 [Bayes, Price (1763)] Suppose we have a coin that has probability p of turning up heads. We do 50 independent tosses and observe 42 heads. What can we say about p? Here we have an observation (the number 42) from a binomial distribution with parameters 50 and p and want to estimate p. Standard frequentist solution: take the estimate 42/50 = / 50

9 Bayes example - 1 [Bayes, Price (1763)] Suppose we have a coin that has probability p of turning up heads. We do 50 independent tosses and observe 42 heads. What can we say about p? Here we have an observation (the number 42) from a binomial distribution with parameters 50 and p and want to estimate p. Standard frequentist solution: take the estimate 42/50 = / 50

10 Bayes Example - 2 Bayesian approach: choose a prior distribution on p, say uniform on [0, 1]. Compute the posterior: beta(43, 9)-distribution (mode is at 42/50 = 0.84) p p prior data posterior 7 / 50

11 Bayes rule Observations X take values in sample space X. Model {P θ : θ Θ}. All P θ dominated: P θ µ, density p θ = dp θ /dµ. Prior distribution Π on the parameter θ. For the Bayesian: θ Π and X θ P θ. Hence, the pair (θ, X ) has density (θ, x) p θ (x) relative to Π µ. Then X has marginal density x p θ (x) Π(dθ), Θ and hence the conditional distribution of θ given X = x, i.e. the posterior, has density relative to the prior Π. θ p θ (x) Θ p θ(x) Π(dθ) 8 / 50

12 Bayes example again Have X Bin(n, θ), θ (0, 1). Likelihood: ( ) n p θ (X ) = θ X (1 θ) n X. X Prior: uniform distribution on (0, 1). By Bayes rule, posterior density proportional to θ θ X (1 θ) n X. Hence, posterior is Beta(X + 1, n X + 1). 9 / 50

13 Bayesian nonparametrics 10 / 50

14 Bayesian nonparametrics Challenges lie in particular in the area of high-dimensional or nonparametric models. Illustration 1: parametric vs. nonparametric regression Y i = f (t i ) + error i t t 11 / 50

15 Bayesian nonparametrics Illustration 2: parametric vs. nonparametric density estimation X 1,..., X n f Frequency Frequency x waiting 12 / 50

16 Bayesian nonparametrics In nonparametric problems, the parameter of interest is typically a function: e.g. a density, regression function, distribution function, hazard rate,..., or some other infinite-dimensional object. Bayesian approach is not at all fundamentally restricted to the parametric case, but: How do we construct priors on infinite-dimensional (function) spaces? How do we compute posteriors, or generate draws? What is the fundamental performance of procedures? 13 / 50

17 Nonparametric priors 14 / 50

18 Nonparametric priors - first remarks Often enough to describe how realizations are generated Possible ways to construct priors on an infinite-dimensional space Θ: Discrete priors: Consider (random) points θ 1, θ 2,..., in Θ and (random) probability weights w 1, w 2,... and define Π = w j δ θj. Stochastic Process approach: If Θ is a function space, use machinery for constructing stochastic processes Random series approach: If Θ is a function space, consider series expansions, put priors on coefficients / 50

19 Nonparametric priors - Dirichlet process 16 / 50

20 Dirichlet process - 1 Step 1: prior on simplex of probability vectors of length k: k 1 = {(y 1,..., y k ) R k : y 1,..., y k 0, y i = 1}. For α = (α 1,..., α k ) (0, ) k, define f α (y 1,..., y k 1 ) = C α k i=0 y α i 1 i 1 (y1,...,y k ) k 1 on R k 1, where y k = 1 y 1 y k 1 and C α is the appropriate normalizing constant. A random vector (Y 1,..., Y k ) in R k is said to have a Dirichlet distribution with parameter α = (α 1,..., α k ) if (Y 1,..., Y k 1 ) has density f α and Y k = 1 Y Y k / 50

21 Dirichlet process - 2 Step 2: definition of DP: Let α be a finite measure on R. A random probability measure P on R is called a Dirichlet Process with parameter α if for every partition A 1,..., A k of R, the vector (P(A 1 ),..., P(A k )) has a Dirichlet distribution with parameter (α(a 1 ),..., α(a k )). Notation: P DP(α). 18 / 50

22 Dirichlet process - 3 Step 3: Prove that DP exists! Theorem. For any finite measure α on R, the Dirichlet process with parameter α exists. Proof. For instance: Use Kolmogorov s consistency theorem to show a process P = (P(A) : A B(R)) with the right fdd s. Prove there exists a version of P such that every realization is a measure. 19 / 50

23 Dirichlet process Ten realizations from Dirichlet process with parameter 25 N(0, 1) 20 / 50

24 Dirichlet process - 5 Draws from the DP are discrete measures on R: Theorem. Let α be a finite measure, define M = α(r) and ᾱ = α/m. If we have independent θ 1, θ 2,... ᾱ and Y 1, Y 2,... Beta(1, M) and V j = Y j 1 j l=1 (1 Y l), then j=1 V jδ θj DP(α). This is the stick-breaking representation. 21 / 50

25 Distribution function estimation The DP is a conjugate prior for full distribution estimation: if P DP(α) and X 1,..., X n P P, then P X 1,..., X n DP(α + n i=1 δ X i ) Simulated data: 500 draws from a N(1, 1)-distribution, prior: Dirichlet process with parameter 25 N(0, 1). Left: 10 draws from the prior. Right: 10 draws from the posterior. 22 / 50

26 Nonparametric priors - Gaussian processes 23 / 50

27 Gaussian process priors - 1 A stochastic process W = (W t : t T ) is called Gaussian if for all n N and t 1,..., t n T, the vector (W t1,..., W tn ) has an n-dimensional Gaussian distribution. Associated functions: mean function: m(t) = EW t, covariance function: r(s, t) = Cov(W s, W t ). The GP is called centered, or zero-mean if m(t) = 0 for all t T. 24 / 50

28 Gaussian process priors - 2 For a 1,..., a n R and t 1,..., t n T, ( ) a i a j r(t i, t j ) = Var ai W ti 0, i j hence r is a positive definite, symmetric function on T T. Theorem. Let T be a set, m : T R a function and r : T T R a positive definite, symmetric function. Then there exists a Gaussian process with mean function m and covariance function r. Proof. Kolmogorov s consistency theorem. 25 / 50

29 Gaussian process priors: examples - 1 Brownian motion: m(t) = 0, r(s, t) = s t Regularity: 1/ / 50

30 Gaussian process priors: examples - 2 Integrated Brownian motion: t 0 W s ds, for W a Brownian motion. m(t) = 0, r(s, t) = s 2 t/2 t 3 / Regularity: 3/ / 50

31 Gaussian process priors: examples - 3 By Fubini and integration by parts, t tn t2 1 W t1 dt 1 dt 2 dt n = 0 0 (n 1)! 0 = 1 n! t 0 t 0 (t s) n 1 W s ds (t s) n dw s. The Riemann-Liouville process with parameter α > 0: W α t = Process has regularity α. t 0 (t s) α 1/2 dw s. 28 / 50

32 Gaussian process priors: examples - 4 Consider a centered Gaussian process W = (W t : t T ), with T R d, such that EW s W t = r(t s), s, t T, for a continuous r : R d R. Such a process is called stationary, or homogenous. By Bochner s theorem: r(t) = e i λ,t µ(dλ), R d for a finite Borel measure µ, called the spectral measure of the process. 29 / 50

33 Gaussian process priors: examples - 5 The squared exponential process: r(s, t) = exp( t s 2 ) Spectral measure: 2 d π d/2 exp( λ 2 /4) dλ Regularity: / 50

34 Gaussian process priors: examples - 6 The Matérn process: µ(dλ) (1 + λ 2 ) (α+d/2) dλ, α > 0. Covariance function: r(s, t) = 21 α Γ(α) t s α K α ( t s ), where K α is the modified Bessel function of the second kind of order α. Regularity: α. For d = 1, α = 1/2, get the Ornstein-Uhlenbeck process. 31 / 50

35 Gaussian process priors: examples - 6 The Matérn process: µ(dλ) (1 + λ 2 ) (α+d/2) dλ, α > 0. Covariance function: r(s, t) = 21 α Γ(α) t s α K α ( t s ), where K α is the modified Bessel function of the second kind of order α. Regularity: α. For d = 1, α = 1/2, get the Ornstein-Uhlenbeck process. 31 / 50

36 Gaussian process regression - 1 Observations: X i = f (t i ) + ε i, t i [0, 1] fixed ε i independent N(0, 1). Prior on f : law of a centered GP with covariance function r. Posterior: this prior is conjugate for this model: (f (t 1 ),..., f (t n )) X 1,..., X n N n ((I + Σ 1 ) 1 X, (I + Σ 1 ) 1 ), where Σ the is matrix with Σ ij = r(t i, t j ). 32 / 50

37 Gaussian process regression - 2 Data: 200 simulated data points. Prior: multiple of integrated Brownian motion Left: 10 draws from the prior. Right: 10 draws from the posterior. 33 / 50

38 Nonparametric priors - Conditionally Gaussian processes 34 / 50

39 CGP s - 1 Observation about GP s: Families of GP s typically depend on auxiliary parameters: hyper parameters. Performance can heavily depend on tuning of parameters. How to choose values of hyper parameters? 35 / 50

40 CGP s - 2 Regression with a squared exponential GP with covariance (x, y) exp( (x y) 2 /l 2 ), for different length scale hyper parameters l. l too small: l correct: y y x x 36 / 50

41 CGP s - 3 Q: How to choose the best values of hyper parameters? A: Let the data decide! Possible approaches: Put a prior on the hyper parameters as well: full Bayes Estimate hyper parameters : empirical Bayes 37 / 50

42 CGP s - 3 Q: How to choose the best values of hyper parameters? A: Let the data decide! Possible approaches: Put a prior on the hyper parameters as well: full Bayes Estimate hyper parameters : empirical Bayes 37 / 50

43 CGP s - 4 Squared exponential GP with gamma length scale: l Γ(a, b) f l GP with cov (x, y) exp( (x y) 2 /l 2 ) Example of a hierarchical prior Prior is only conditionally Gaussian Q: does this solve the bias-variance issue? 38 / 50

44 CGP s - 4 Squared exponential GP with gamma length scale: l Γ(a, b) f l GP with cov (x, y) exp( (x y) 2 /l 2 ) Example of a hierarchical prior Prior is only conditionally Gaussian Q: does this solve the bias-variance issue? 38 / 50

45 Nonparametric priors - Dirichlet mixtures 39 / 50

46 DP mixture priors - 1 Idea: Consider location/scale mixtures of Gaussians of the form p G (x) = ϕ σ (x µ) G(dµ, dσ), where ϕ σ ( µ) G is the N(µ, σ 2 )-density is a probability measure (mixing measure). Construct a prior on densities by making G random. 40 / 50

47 DP mixture priors - 2 Draw g from a Gaussian DP mixture prior: G DP(G 0 ) (G 0 often N IW ) p G p G Another example of a hierarchical prior 41 / 50

48 DP mixture density estimation Data: 272 waiting times between geyser eruptions Prior: DP mixture of normals Posterior mean: Density waiting 42 / 50

49 Some more examples 43 / 50

50 Estimating the drift of a diffusion Observation model: dx t = b(x t ) dt + dw t. Goal: estimate b. Prior: s IG(a, b) J Ps(λ) b s, J s J j=1 j 2 Z j e j e j : Fourier basis, Z j N(0, 1) posterior mean marginal 0.68 cred. posterior mean PPRS marginal 0.68 cred [Van der Meulen, Schauer, vz. (2014)] Figure 14: Comparison of the estimate of drift using the Butane Dihedral Angle data. Red solid: A Fourier prior with = 1.5. Blue dashed: Results of Papaspiliopoulos et al. (2012). The posterior mean with 68% credible bands is pictured. Right: Histogram of the data. 44 / 50

51 Nonparametric estimation of a Poisson intensity Observation model: counts from an inhomogenous Poisson process with periodic intensity λ. Goal: estimate λ. Prior: B-spline expansion with priors on knots and coefficients again with a thinned out dataset. We randomly removed counts, retaining about 1, 000 counts. The same analysis then leads to the posterior plot given in Figure 3. In this case, the uncertainty in the posterior distribution becomes clearly visible x 104 Posterior mean and point wise quantiles (10000 states) Posterior mean Point wise 95% credible intervals 12 Intensity :00 1:00 2:00 3:00 4:00 5:00 6:00 7:00 8:00 9:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 24:00 Time (hours) :00 1:00 2:00 3:00 4:00 5:00 6:00 7:00 8:00 9:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 24:00 Time (hours) Figure 3: Top panel: posterior distribution of the intensity function based on the thinned data. (Blue: posterior mean, red: point-wise 95% credible intervals). Lower panel: posterior distribution of the knot locations (Histogram). [Belitser, Serra, vz. (2015)] We find that the prior that we defined in Section 2.2 is a computationally feasible choice for nonparametric Bayesian intensity smoothing in the context of this kind of periodic 45 / 50

52 Binary prediction on a graph Observation model: P(Y i = 1) = Ψ(f (i)), for f : G R. Prior: Conditionally Gaussian with precision L p, L: graph Laplacian [Hartog, vz. (in prep.)] 46 / 50

53 Concluding remarks 47 / 50

54 Take home from Lecture I Within the Bayesian paradigm it is perfectly possible and natural to deal with nonparametric statistical problems. Many nonparametric priors have been proposed and studied: DP s, GP s, DP mixtures, series expansion,... Numerical techniques have been developed to sample from the corresponding posteriors In a variety of statistical settings, the results can be quite satisfactory. Some (theoretical) questions: So do these procedures do what we expect them to do? Why/why not? Do they have desirable properties like consistency? Can we say something more about performance, e.g. about (optimal) convergence rates? 48 / 50

55 Take home from Lecture I Within the Bayesian paradigm it is perfectly possible and natural to deal with nonparametric statistical problems. Many nonparametric priors have been proposed and studied: DP s, GP s, DP mixtures, series expansion,... Numerical techniques have been developed to sample from the corresponding posteriors In a variety of statistical settings, the results can be quite satisfactory. Some (theoretical) questions: So do these procedures do what we expect them to do? Why/why not? Do they have desirable properties like consistency? Can we say something more about performance, e.g. about (optimal) convergence rates? 48 / 50

56 DP: Some references for Lecture I - 1 Ferguson, T. S. (1973). A Bayesian analysis of some nonparametric problems. Annals of Statistics, 1, DP mixtures: Ferguson, T. S. (1983). Bayesian density estimation by mixtures of normal distributions. In Recent Advances in Statistics, ed. M. Rizvi et al., Escobar, M. and West, M. (1995). Bayesian density estimation and inference using mixtures. Journal of the American Statistical Association, 90, MacEachern, S. N. and Muller, P. (1998) Estimating mixture of Dirichlet Process Models. Journal of Computational and Graphical Statistics, 7 (2), Neal, R. M. (2000). Markov Chain sampling methods for Dirichlet process mixture models. Journal of Computational and Graphical Statistics, 9, / 50

57 Some references for Lecture I - 2 GP priors: Lenk, P. J. (1988). The logistic normal distribution for Bayesian, nonparametric, predictive densities. J. Amer. Statist. Assoc Lenk, P. J. (1991). Towards a practicable Bayesian nonparametric density estimator. Biometrika Rasmussen, C. E. and Williams, C. K. (2006). Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA. General text: Hjort, N.L., et al., eds. Bayesian nonparametrics. Vol. 28. Cambridge University Press, / 50

Bayesian Regularization

Bayesian Regularization Bayesian Regularization Aad van der Vaart Vrije Universiteit Amsterdam International Congress of Mathematicians Hyderabad, August 2010 Contents Introduction Abstract result Gaussian process priors Co-authors

More information

Bayesian Methods for Machine Learning

Bayesian Methods for Machine Learning Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),

More information

Foundations of Nonparametric Bayesian Methods

Foundations of Nonparametric Bayesian Methods 1 / 27 Foundations of Nonparametric Bayesian Methods Part II: Models on the Simplex Peter Orbanz http://mlg.eng.cam.ac.uk/porbanz/npb-tutorial.html 2 / 27 Tutorial Overview Part I: Basics Part II: Models

More information

Non-Parametric Bayes

Non-Parametric Bayes Non-Parametric Bayes Mark Schmidt UBC Machine Learning Reading Group January 2016 Current Hot Topics in Machine Learning Bayesian learning includes: Gaussian processes. Approximate inference. Bayesian

More information

Bayesian Learning. HT2015: SC4 Statistical Data Mining and Machine Learning. Maximum Likelihood Principle. The Bayesian Learning Framework

Bayesian Learning. HT2015: SC4 Statistical Data Mining and Machine Learning. Maximum Likelihood Principle. The Bayesian Learning Framework HT5: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford http://www.stats.ox.ac.uk/~sejdinov/sdmml.html Maximum Likelihood Principle A generative model for

More information

STAT 518 Intro Student Presentation

STAT 518 Intro Student Presentation STAT 518 Intro Student Presentation Wen Wei Loh April 11, 2013 Title of paper Radford M. Neal [1999] Bayesian Statistics, 6: 475-501, 1999 What the paper is about Regression and Classification Flexible

More information

STAT Advanced Bayesian Inference

STAT Advanced Bayesian Inference 1 / 32 STAT 625 - Advanced Bayesian Inference Meng Li Department of Statistics Jan 23, 218 The Dirichlet distribution 2 / 32 θ Dirichlet(a 1,...,a k ) with density p(θ 1,θ 2,...,θ k ) = k j=1 Γ(a j) Γ(

More information

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 2: PROBABILITY DISTRIBUTIONS

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 2: PROBABILITY DISTRIBUTIONS PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 2: PROBABILITY DISTRIBUTIONS Parametric Distributions Basic building blocks: Need to determine given Representation: or? Recall Curve Fitting Binary Variables

More information

A Process over all Stationary Covariance Kernels

A Process over all Stationary Covariance Kernels A Process over all Stationary Covariance Kernels Andrew Gordon Wilson June 9, 0 Abstract I define a process over all stationary covariance kernels. I show how one might be able to perform inference that

More information

Nonparametric Bayesian Methods (Gaussian Processes)

Nonparametric Bayesian Methods (Gaussian Processes) [70240413 Statistical Machine Learning, Spring, 2015] Nonparametric Bayesian Methods (Gaussian Processes) Jun Zhu dcszj@mail.tsinghua.edu.cn http://bigml.cs.tsinghua.edu.cn/~jun State Key Lab of Intelligent

More information

Bayesian Nonparametrics

Bayesian Nonparametrics Bayesian Nonparametrics Lorenzo Rosasco 9.520 Class 18 April 11, 2011 About this class Goal To give an overview of some of the basic concepts in Bayesian Nonparametrics. In particular, to discuss Dirichelet

More information

ICML Scalable Bayesian Inference on Point processes. with Gaussian Processes. Yves-Laurent Kom Samo & Stephen Roberts

ICML Scalable Bayesian Inference on Point processes. with Gaussian Processes. Yves-Laurent Kom Samo & Stephen Roberts ICML 2015 Scalable Nonparametric Bayesian Inference on Point Processes with Gaussian Processes Machine Learning Research Group and Oxford-Man Institute University of Oxford July 8, 2015 Point Processes

More information

Bayesian estimation of the discrepancy with misspecified parametric models

Bayesian estimation of the discrepancy with misspecified parametric models Bayesian estimation of the discrepancy with misspecified parametric models Pierpaolo De Blasi University of Torino & Collegio Carlo Alberto Bayesian Nonparametrics workshop ICERM, 17-21 September 2012

More information

Gaussian Process Regression

Gaussian Process Regression Gaussian Process Regression 4F1 Pattern Recognition, 21 Carl Edward Rasmussen Department of Engineering, University of Cambridge November 11th - 16th, 21 Rasmussen (Engineering, Cambridge) Gaussian Process

More information

Introduction to Probabilistic Machine Learning

Introduction to Probabilistic Machine Learning Introduction to Probabilistic Machine Learning Piyush Rai Dept. of CSE, IIT Kanpur (Mini-course 1) Nov 03, 2015 Piyush Rai (IIT Kanpur) Introduction to Probabilistic Machine Learning 1 Machine Learning

More information

Gaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012

Gaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012 Gaussian Processes Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 01 Pictorial view of embedding distribution Transform the entire distribution to expected features Feature space Feature

More information

Pattern Recognition and Machine Learning. Bishop Chapter 2: Probability Distributions

Pattern Recognition and Machine Learning. Bishop Chapter 2: Probability Distributions Pattern Recognition and Machine Learning Chapter 2: Probability Distributions Cécile Amblard Alex Kläser Jakob Verbeek October 11, 27 Probability Distributions: General Density Estimation: given a finite

More information

Bayesian Nonparametrics

Bayesian Nonparametrics Bayesian Nonparametrics Peter Orbanz Columbia University PARAMETERS AND PATTERNS Parameters P(X θ) = Probability[data pattern] 3 2 1 0 1 2 3 5 0 5 Inference idea data = underlying pattern + independent

More information

Outline. Binomial, Multinomial, Normal, Beta, Dirichlet. Posterior mean, MAP, credible interval, posterior distribution

Outline. Binomial, Multinomial, Normal, Beta, Dirichlet. Posterior mean, MAP, credible interval, posterior distribution Outline A short review on Bayesian analysis. Binomial, Multinomial, Normal, Beta, Dirichlet Posterior mean, MAP, credible interval, posterior distribution Gibbs sampling Revisit the Gaussian mixture model

More information

A Brief Overview of Nonparametric Bayesian Models

A Brief Overview of Nonparametric Bayesian Models A Brief Overview of Nonparametric Bayesian Models Eurandom Zoubin Ghahramani Department of Engineering University of Cambridge, UK zoubin@eng.cam.ac.uk http://learning.eng.cam.ac.uk/zoubin Also at Machine

More information

STA414/2104. Lecture 11: Gaussian Processes. Department of Statistics

STA414/2104. Lecture 11: Gaussian Processes. Department of Statistics STA414/2104 Lecture 11: Gaussian Processes Department of Statistics www.utstat.utoronto.ca Delivered by Mark Ebden with thanks to Russ Salakhutdinov Outline Gaussian Processes Exam review Course evaluations

More information

Hyperparameter estimation in Dirichlet process mixture models

Hyperparameter estimation in Dirichlet process mixture models Hyperparameter estimation in Dirichlet process mixture models By MIKE WEST Institute of Statistics and Decision Sciences Duke University, Durham NC 27706, USA. SUMMARY In Bayesian density estimation and

More information

CPSC 540: Machine Learning

CPSC 540: Machine Learning CPSC 540: Machine Learning MCMC and Non-Parametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is

More information

Review. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda

Review. DS GA 1002 Statistical and Mathematical Models.   Carlos Fernandez-Granda Review DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Probability and statistics Probability: Framework for dealing with

More information

Motivation Scale Mixutres of Normals Finite Gaussian Mixtures Skew-Normal Models. Mixture Models. Econ 690. Purdue University

Motivation Scale Mixutres of Normals Finite Gaussian Mixtures Skew-Normal Models. Mixture Models. Econ 690. Purdue University Econ 690 Purdue University In virtually all of the previous lectures, our models have made use of normality assumptions. From a computational point of view, the reason for this assumption is clear: combined

More information

More Spectral Clustering and an Introduction to Conjugacy

More Spectral Clustering and an Introduction to Conjugacy CS8B/Stat4B: Advanced Topics in Learning & Decision Making More Spectral Clustering and an Introduction to Conjugacy Lecturer: Michael I. Jordan Scribe: Marco Barreno Monday, April 5, 004. Back to spectral

More information

Statistics & Data Sciences: First Year Prelim Exam May 2018

Statistics & Data Sciences: First Year Prelim Exam May 2018 Statistics & Data Sciences: First Year Prelim Exam May 2018 Instructions: 1. Do not turn this page until instructed to do so. 2. Start each new question on a new sheet of paper. 3. This is a closed book

More information

Bayesian RL Seminar. Chris Mansley September 9, 2008

Bayesian RL Seminar. Chris Mansley September 9, 2008 Bayesian RL Seminar Chris Mansley September 9, 2008 Bayes Basic Probability One of the basic principles of probability theory, the chain rule, will allow us to derive most of the background material in

More information

A Nonparametric Model for Stationary Time Series

A Nonparametric Model for Stationary Time Series A Nonparametric Model for Stationary Time Series Isadora Antoniano-Villalobos Bocconi University, Milan, Italy. isadora.antoniano@unibocconi.it Stephen G. Walker University of Texas at Austin, USA. s.g.walker@math.utexas.edu

More information

Variational Bayesian Dirichlet-Multinomial Allocation for Exponential Family Mixtures

Variational Bayesian Dirichlet-Multinomial Allocation for Exponential Family Mixtures 17th Europ. Conf. on Machine Learning, Berlin, Germany, 2006. Variational Bayesian Dirichlet-Multinomial Allocation for Exponential Family Mixtures Shipeng Yu 1,2, Kai Yu 2, Volker Tresp 2, and Hans-Peter

More information

PROBABILITY: LIMIT THEOREMS II, SPRING HOMEWORK PROBLEMS

PROBABILITY: LIMIT THEOREMS II, SPRING HOMEWORK PROBLEMS PROBABILITY: LIMIT THEOREMS II, SPRING 15. HOMEWORK PROBLEMS PROF. YURI BAKHTIN Instructions. You are allowed to work on solutions in groups, but you are required to write up solutions on your own. Please

More information

Learning Bayesian network : Given structure and completely observed data

Learning Bayesian network : Given structure and completely observed data Learning Bayesian network : Given structure and completely observed data Probabilistic Graphical Models Sharif University of Technology Spring 2017 Soleymani Learning problem Target: true distribution

More information

Dirichlet Processes: Tutorial and Practical Course

Dirichlet Processes: Tutorial and Practical Course Dirichlet Processes: Tutorial and Practical Course (updated) Yee Whye Teh Gatsby Computational Neuroscience Unit University College London August 2007 / MLSS Yee Whye Teh (Gatsby) DP August 2007 / MLSS

More information

Nonparametric Bayesian Uncertainty Quantification

Nonparametric Bayesian Uncertainty Quantification Nonparametric Bayesian Uncertainty Quantification Lecture 1: Introduction to Nonparametric Bayes Aad van der Vaart Universiteit Leiden, Netherlands YES, Eindhoven, January 2017 Contents Introduction Recovery

More information

Probability. Machine Learning and Pattern Recognition. Chris Williams. School of Informatics, University of Edinburgh. August 2014

Probability. Machine Learning and Pattern Recognition. Chris Williams. School of Informatics, University of Edinburgh. August 2014 Probability Machine Learning and Pattern Recognition Chris Williams School of Informatics, University of Edinburgh August 2014 (All of the slides in this course have been adapted from previous versions

More information

Lecture 3a: Dirichlet processes

Lecture 3a: Dirichlet processes Lecture 3a: Dirichlet processes Cédric Archambeau Centre for Computational Statistics and Machine Learning Department of Computer Science University College London c.archambeau@cs.ucl.ac.uk Advanced Topics

More information

Foundations of Statistical Inference

Foundations of Statistical Inference Foundations of Statistical Inference Jonathan Marchini Department of Statistics University of Oxford MT 2013 Jonathan Marchini (University of Oxford) BS2a MT 2013 1 / 27 Course arrangements Lectures M.2

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 3 Linear

More information

Bayesian nonparametric estimation of finite population quantities in absence of design information on nonsampled units

Bayesian nonparametric estimation of finite population quantities in absence of design information on nonsampled units Bayesian nonparametric estimation of finite population quantities in absence of design information on nonsampled units Sahar Z Zangeneh Robert W. Keener Roderick J.A. Little Abstract In Probability proportional

More information

Bayesian Nonparametrics: Dirichlet Process

Bayesian Nonparametrics: Dirichlet Process Bayesian Nonparametrics: Dirichlet Process Yee Whye Teh Gatsby Computational Neuroscience Unit, UCL http://www.gatsby.ucl.ac.uk/~ywteh/teaching/npbayes2012 Dirichlet Process Cornerstone of modern Bayesian

More information

Likelihood-free MCMC

Likelihood-free MCMC Bayesian inference for stable distributions with applications in finance Department of Mathematics University of Leicester September 2, 2011 MSc project final presentation Outline 1 2 3 4 Classical Monte

More information

Bayesian Nonparametric Regression for Diabetes Deaths

Bayesian Nonparametric Regression for Diabetes Deaths Bayesian Nonparametric Regression for Diabetes Deaths Brian M. Hartman PhD Student, 2010 Texas A&M University College Station, TX, USA David B. Dahl Assistant Professor Texas A&M University College Station,

More information

Optimality of Poisson Processes Intensity Learning with Gaussian Processes

Optimality of Poisson Processes Intensity Learning with Gaussian Processes Journal of Machine Learning Research 16 (2015) 2909-2919 Submitted 9/14; Revised 3/15; Published 12/15 Optimality of Poisson Processes Intensity Learning with Gaussian Processes Alisa Kirichenko Harry

More information

Bayesian Models in Machine Learning

Bayesian Models in Machine Learning Bayesian Models in Machine Learning Lukáš Burget Escuela de Ciencias Informáticas 2017 Buenos Aires, July 24-29 2017 Frequentist vs. Bayesian Frequentist point of view: Probability is the frequency of

More information

Nonparmeteric Bayes & Gaussian Processes. Baback Moghaddam Machine Learning Group

Nonparmeteric Bayes & Gaussian Processes. Baback Moghaddam Machine Learning Group Nonparmeteric Bayes & Gaussian Processes Baback Moghaddam baback@jpl.nasa.gov Machine Learning Group Outline Bayesian Inference Hierarchical Models Model Selection Parametric vs. Nonparametric Gaussian

More information

PMR Learning as Inference

PMR Learning as Inference Outline PMR Learning as Inference Probabilistic Modelling and Reasoning Amos Storkey Modelling 2 The Exponential Family 3 Bayesian Sets School of Informatics, University of Edinburgh Amos Storkey PMR Learning

More information

Bayesian Nonparametric Models

Bayesian Nonparametric Models Bayesian Nonparametric Models David M. Blei Columbia University December 15, 2015 Introduction We have been looking at models that posit latent structure in high dimensional data. We use the posterior

More information

Introduction. Chapter 1

Introduction. Chapter 1 Chapter 1 Introduction In this book we will be concerned with supervised learning, which is the problem of learning input-output mappings from empirical data (the training dataset). Depending on the characteristics

More information

Spatial Bayesian Nonparametrics for Natural Image Segmentation

Spatial Bayesian Nonparametrics for Natural Image Segmentation Spatial Bayesian Nonparametrics for Natural Image Segmentation Erik Sudderth Brown University Joint work with Michael Jordan University of California Soumya Ghosh Brown University Parsing Visual Scenes

More information

Gaussian processes. Chuong B. Do (updated by Honglak Lee) November 22, 2008

Gaussian processes. Chuong B. Do (updated by Honglak Lee) November 22, 2008 Gaussian processes Chuong B Do (updated by Honglak Lee) November 22, 2008 Many of the classical machine learning algorithms that we talked about during the first half of this course fit the following pattern:

More information

STATS 200: Introduction to Statistical Inference. Lecture 29: Course review

STATS 200: Introduction to Statistical Inference. Lecture 29: Course review STATS 200: Introduction to Statistical Inference Lecture 29: Course review Course review We started in Lecture 1 with a fundamental assumption: Data is a realization of a random process. The goal throughout

More information

Density Estimation. Seungjin Choi

Density Estimation. Seungjin Choi Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/

More information

Bayesian Nonparametrics for Speech and Signal Processing

Bayesian Nonparametrics for Speech and Signal Processing Bayesian Nonparametrics for Speech and Signal Processing Michael I. Jordan University of California, Berkeley June 28, 2011 Acknowledgments: Emily Fox, Erik Sudderth, Yee Whye Teh, and Romain Thibaux Computer

More information

Lecture 16-17: Bayesian Nonparametrics I. STAT 6474 Instructor: Hongxiao Zhu

Lecture 16-17: Bayesian Nonparametrics I. STAT 6474 Instructor: Hongxiao Zhu Lecture 16-17: Bayesian Nonparametrics I STAT 6474 Instructor: Hongxiao Zhu Plan for today Why Bayesian Nonparametrics? Dirichlet Distribution and Dirichlet Processes. 2 Parameter and Patterns Reference:

More information

Lecture 2: Priors and Conjugacy

Lecture 2: Priors and Conjugacy Lecture 2: Priors and Conjugacy Melih Kandemir melih.kandemir@iwr.uni-heidelberg.de May 6, 2014 Some nice courses Fred A. Hamprecht (Heidelberg U.) https://www.youtube.com/watch?v=j66rrnzzkow Michael I.

More information

State Space Representation of Gaussian Processes

State Space Representation of Gaussian Processes State Space Representation of Gaussian Processes Simo Särkkä Department of Biomedical Engineering and Computational Science (BECS) Aalto University, Espoo, Finland June 12th, 2013 Simo Särkkä (Aalto University)

More information

Log Gaussian Cox Processes. Chi Group Meeting February 23, 2016

Log Gaussian Cox Processes. Chi Group Meeting February 23, 2016 Log Gaussian Cox Processes Chi Group Meeting February 23, 2016 Outline Typical motivating application Introduction to LGCP model Brief overview of inference Applications in my work just getting started

More information

Stochastic Processes, Kernel Regression, Infinite Mixture Models

Stochastic Processes, Kernel Regression, Infinite Mixture Models Stochastic Processes, Kernel Regression, Infinite Mixture Models Gabriel Huang (TA for Simon Lacoste-Julien) IFT 6269 : Probabilistic Graphical Models - Fall 2018 Stochastic Process = Random Function 2

More information

A Simple Proof of the Stick-Breaking Construction of the Dirichlet Process

A Simple Proof of the Stick-Breaking Construction of the Dirichlet Process A Simple Proof of the Stick-Breaking Construction of the Dirichlet Process John Paisley Department of Computer Science Princeton University, Princeton, NJ jpaisley@princeton.edu Abstract We give a simple

More information

Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen. Bayesian Learning. Tobias Scheffer, Niels Landwehr

Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen. Bayesian Learning. Tobias Scheffer, Niels Landwehr Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen Bayesian Learning Tobias Scheffer, Niels Landwehr Remember: Normal Distribution Distribution over x. Density function with parameters

More information

MFM Practitioner Module: Quantitative Risk Management. John Dodson. September 23, 2015

MFM Practitioner Module: Quantitative Risk Management. John Dodson. September 23, 2015 MFM Practitioner Module: Quantitative Risk Management September 23, 2015 Mixtures Mixtures Mixtures Definitions For our purposes, A random variable is a quantity whose value is not known to us right now

More information

arxiv: v1 [stat.me] 6 Nov 2013

arxiv: v1 [stat.me] 6 Nov 2013 Electronic Journal of Statistics Vol. 0 (0000) ISSN: 1935-7524 DOI: 10.1214/154957804100000000 A Generalized Savage-Dickey Ratio Ewan Cameron e-mail: dr.ewan.cameron@gmail.com url: astrostatistics.wordpress.com

More information

CSci 8980: Advanced Topics in Graphical Models Gaussian Processes

CSci 8980: Advanced Topics in Graphical Models Gaussian Processes CSci 8980: Advanced Topics in Graphical Models Gaussian Processes Instructor: Arindam Banerjee November 15, 2007 Gaussian Processes Outline Gaussian Processes Outline Parametric Bayesian Regression Gaussian

More information

Announcements. Proposals graded

Announcements. Proposals graded Announcements Proposals graded Kevin Jamieson 2018 1 Bayesian Methods Machine Learning CSE546 Kevin Jamieson University of Washington November 1, 2018 2018 Kevin Jamieson 2 MLE Recap - coin flips Data:

More information

Dirichlet Process. Yee Whye Teh, University College London

Dirichlet Process. Yee Whye Teh, University College London Dirichlet Process Yee Whye Teh, University College London Related keywords: Bayesian nonparametrics, stochastic processes, clustering, infinite mixture model, Blackwell-MacQueen urn scheme, Chinese restaurant

More information

Contents. Part I: Fundamentals of Bayesian Inference 1

Contents. Part I: Fundamentals of Bayesian Inference 1 Contents Preface xiii Part I: Fundamentals of Bayesian Inference 1 1 Probability and inference 3 1.1 The three steps of Bayesian data analysis 3 1.2 General notation for statistical inference 4 1.3 Bayesian

More information

Stat 5101 Lecture Notes

Stat 5101 Lecture Notes Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random

More information

Naïve Bayes classification

Naïve Bayes classification Naïve Bayes classification 1 Probability theory Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. Examples: A person s height, the outcome of a coin toss

More information

Generative Clustering, Topic Modeling, & Bayesian Inference

Generative Clustering, Topic Modeling, & Bayesian Inference Generative Clustering, Topic Modeling, & Bayesian Inference INFO-4604, Applied Machine Learning University of Colorado Boulder December 12-14, 2017 Prof. Michael Paul Unsupervised Naïve Bayes Last week

More information

Bayesian Learning (II)

Bayesian Learning (II) Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen Bayesian Learning (II) Niels Landwehr Overview Probabilities, expected values, variance Basic concepts of Bayesian learning MAP

More information

Introduction: MLE, MAP, Bayesian reasoning (28/8/13)

Introduction: MLE, MAP, Bayesian reasoning (28/8/13) STA561: Probabilistic machine learning Introduction: MLE, MAP, Bayesian reasoning (28/8/13) Lecturer: Barbara Engelhardt Scribes: K. Ulrich, J. Subramanian, N. Raval, J. O Hollaren 1 Classifiers In this

More information

ECE521 Tutorial 11. Topic Review. ECE521 Winter Credits to Alireza Makhzani, Alex Schwing, Rich Zemel and TAs for slides. ECE521 Tutorial 11 / 4

ECE521 Tutorial 11. Topic Review. ECE521 Winter Credits to Alireza Makhzani, Alex Schwing, Rich Zemel and TAs for slides. ECE521 Tutorial 11 / 4 ECE52 Tutorial Topic Review ECE52 Winter 206 Credits to Alireza Makhzani, Alex Schwing, Rich Zemel and TAs for slides ECE52 Tutorial ECE52 Winter 206 Credits to Alireza / 4 Outline K-means, PCA 2 Bayesian

More information

Statistics: Learning models from data

Statistics: Learning models from data DS-GA 1002 Lecture notes 5 October 19, 2015 Statistics: Learning models from data Learning models from data that are assumed to be generated probabilistically from a certain unknown distribution is a crucial

More information

Model Based Clustering of Count Processes Data

Model Based Clustering of Count Processes Data Model Based Clustering of Count Processes Data Tin Lok James Ng, Brendan Murphy Insight Centre for Data Analytics School of Mathematics and Statistics May 15, 2017 Tin Lok James Ng, Brendan Murphy (Insight)

More information

The Bayesian approach to inverse problems

The Bayesian approach to inverse problems The Bayesian approach to inverse problems Youssef Marzouk Department of Aeronautics and Astronautics Center for Computational Engineering Massachusetts Institute of Technology ymarz@mit.edu, http://uqgroup.mit.edu

More information

NPFL108 Bayesian inference. Introduction. Filip Jurčíček. Institute of Formal and Applied Linguistics Charles University in Prague Czech Republic

NPFL108 Bayesian inference. Introduction. Filip Jurčíček. Institute of Formal and Applied Linguistics Charles University in Prague Czech Republic NPFL108 Bayesian inference Introduction Filip Jurčíček Institute of Formal and Applied Linguistics Charles University in Prague Czech Republic Home page: http://ufal.mff.cuni.cz/~jurcicek Version: 21/02/2014

More information

Bayesian linear regression

Bayesian linear regression Bayesian linear regression Linear regression is the basis of most statistical modeling. The model is Y i = X T i β + ε i, where Y i is the continuous response X i = (X i1,..., X ip ) T is the corresponding

More information

Introduction to Bayesian Methods. Introduction to Bayesian Methods p.1/??

Introduction to Bayesian Methods. Introduction to Bayesian Methods p.1/?? to Bayesian Methods Introduction to Bayesian Methods p.1/?? We develop the Bayesian paradigm for parametric inference. To this end, suppose we conduct (or wish to design) a study, in which the parameter

More information

ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING. Non-linear regression techniques Part - II

ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING. Non-linear regression techniques Part - II 1 Non-linear regression techniques Part - II Regression Algorithms in this Course Support Vector Machine Relevance Vector Machine Support vector regression Boosting random projections Relevance vector

More information

Probabilistic modeling. The slides are closely adapted from Subhransu Maji s slides

Probabilistic modeling. The slides are closely adapted from Subhransu Maji s slides Probabilistic modeling The slides are closely adapted from Subhransu Maji s slides Overview So far the models and algorithms you have learned about are relatively disconnected Probabilistic modeling framework

More information

Computer Emulation With Density Estimation

Computer Emulation With Density Estimation Computer Emulation With Density Estimation Jake Coleman, Robert Wolpert May 8, 2017 Jake Coleman, Robert Wolpert Emulation and Density Estimation May 8, 2017 1 / 17 Computer Emulation Motivation Expensive

More information

Bayesian non-parametric model to longitudinally predict churn

Bayesian non-parametric model to longitudinally predict churn Bayesian non-parametric model to longitudinally predict churn Bruno Scarpa Università di Padova Conference of European Statistics Stakeholders Methodologists, Producers and Users of European Statistics

More information

Introduction to Probabilistic Graphical Models

Introduction to Probabilistic Graphical Models Introduction to Probabilistic Graphical Models Sargur Srihari srihari@cedar.buffalo.edu 1 Topics 1. What are probabilistic graphical models (PGMs) 2. Use of PGMs Engineering and AI 3. Directionality in

More information

Review of Probabilities and Basic Statistics

Review of Probabilities and Basic Statistics Alex Smola Barnabas Poczos TA: Ina Fiterau 4 th year PhD student MLD Review of Probabilities and Basic Statistics 10-701 Recitations 1/25/2013 Recitation 1: Statistics Intro 1 Overview Introduction to

More information

Gaussian processes for inference in stochastic differential equations

Gaussian processes for inference in stochastic differential equations Gaussian processes for inference in stochastic differential equations Manfred Opper, AI group, TU Berlin November 6, 2017 Manfred Opper, AI group, TU Berlin (TU Berlin) inference in SDE November 6, 2017

More information

Hierarchical Models & Bayesian Model Selection

Hierarchical Models & Bayesian Model Selection Hierarchical Models & Bayesian Model Selection Geoffrey Roeder Departments of Computer Science and Statistics University of British Columbia Jan. 20, 2016 Contact information Please report any typos or

More information

Introduction to Machine Learning. Lecture 2

Introduction to Machine Learning. Lecture 2 Introduction to Machine Learning Lecturer: Eran Halperin Lecture 2 Fall Semester Scribe: Yishay Mansour Some of the material was not presented in class (and is marked with a side line) and is given for

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate

More information

New Bayesian methods for model comparison

New Bayesian methods for model comparison Back to the future New Bayesian methods for model comparison Murray Aitkin murray.aitkin@unimelb.edu.au Department of Mathematics and Statistics The University of Melbourne Australia Bayesian Model Comparison

More information

Econ 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines

Econ 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines Econ 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines Maximilian Kasy Department of Economics, Harvard University 1 / 37 Agenda 6 equivalent representations of the

More information

STA 4273H: Sta-s-cal Machine Learning

STA 4273H: Sta-s-cal Machine Learning STA 4273H: Sta-s-cal Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 2 In our

More information

PROBABILITY DISTRIBUTIONS. J. Elder CSE 6390/PSYC 6225 Computational Modeling of Visual Perception

PROBABILITY DISTRIBUTIONS. J. Elder CSE 6390/PSYC 6225 Computational Modeling of Visual Perception PROBABILITY DISTRIBUTIONS Credits 2 These slides were sourced and/or modified from: Christopher Bishop, Microsoft UK Parametric Distributions 3 Basic building blocks: Need to determine given Representation:

More information

Normalized kernel-weighted random measures

Normalized kernel-weighted random measures Normalized kernel-weighted random measures Jim Griffin University of Kent 1 August 27 Outline 1 Introduction 2 Ornstein-Uhlenbeck DP 3 Generalisations Bayesian Density Regression We observe data (x 1,

More information

A short introduction to INLA and R-INLA

A short introduction to INLA and R-INLA A short introduction to INLA and R-INLA Integrated Nested Laplace Approximation Thomas Opitz, BioSP, INRA Avignon Workshop: Theory and practice of INLA and SPDE November 7, 2018 2/21 Plan for this talk

More information

Bayesian inference. Rasmus Waagepetersen Department of Mathematics Aalborg University Denmark. April 10, 2017

Bayesian inference. Rasmus Waagepetersen Department of Mathematics Aalborg University Denmark. April 10, 2017 Bayesian inference Rasmus Waagepetersen Department of Mathematics Aalborg University Denmark April 10, 2017 1 / 22 Outline for today A genetic example Bayes theorem Examples Priors Posterior summaries

More information

Lecture 4: Introduction to stochastic processes and stochastic calculus

Lecture 4: Introduction to stochastic processes and stochastic calculus Lecture 4: Introduction to stochastic processes and stochastic calculus Cédric Archambeau Centre for Computational Statistics and Machine Learning Department of Computer Science University College London

More information

INTRODUCTION TO BAYESIAN INFERENCE PART 2 CHRIS BISHOP

INTRODUCTION TO BAYESIAN INFERENCE PART 2 CHRIS BISHOP INTRODUCTION TO BAYESIAN INFERENCE PART 2 CHRIS BISHOP Personal Healthcare Revolution Electronic health records (CFH) Personal genomics (DeCode, Navigenics, 23andMe) X-prize: first $10k human genome technology

More information

Lecture 2: From Linear Regression to Kalman Filter and Beyond

Lecture 2: From Linear Regression to Kalman Filter and Beyond Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing

More information

COMP 551 Applied Machine Learning Lecture 19: Bayesian Inference

COMP 551 Applied Machine Learning Lecture 19: Bayesian Inference COMP 551 Applied Machine Learning Lecture 19: Bayesian Inference Associate Instructor: (herke.vanhoof@mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/comp551 Unless otherwise noted, all material posted

More information

Quantifying the Price of Uncertainty in Bayesian Models

Quantifying the Price of Uncertainty in Bayesian Models Provided by the author(s) and NUI Galway in accordance with publisher policies. Please cite the published version when available. Title Quantifying the Price of Uncertainty in Bayesian Models Author(s)

More information