AN ASTRONOMER S INTRODUCTION TO GAUSSIAN PROCESSES. Dan Foreman-Mackey // github.com/dfm // dfm.io
|
|
|
- Raymond Hawkins
- 6 years ago
- Views:
Transcription
1 AN ASTRONOMER S INTRODUCTION TO GAUSSIAN PROCESSES Dan Foreman-Mackey CCPP@NYU // github.com/dfm // dfm.io
2 cbnd Flickr user lizphung
3 github.com/dfm/gp
4 gaussianprocess.org/gpml Rasmussen & Williams
5 I write code for good & astrophysics.
6 I (probably) do data science.
7 not data science. Photo credit James Silvester silvesterphoto.tumblr.com
8 data science. cb Flickr user Marcin Wichary
9 Data Science PHYSICS DATA p(data physics)
10 I work with Kepler data.
11 I get really passionate about NOISE
12 relative flux time [KBJD] KIC 33000
13 Kepler 3
14 Kepler 3
15 relative flux time [KBJD] Kepler 3
16 HACKS * ALL HACKS and I m righteous!
17 Why not model all the things? with, for example, a Gaussian Process
18 1 The power of correlated noise.
19 y = mx+ b
20 The true covariance of the observations.
21 Let s assume that the noise is independent NX apple [yn f (x n )] log p(y x,, ) = 1 n=1 n + log n Gaussian with known variance
22 Or equivalently log p(y x,, ) = 1 rt C 1 r 1 log det C N log
23 Or equivalently data covariance log p(y x,, ) = 1 rt C 1 r 1 log det C N log
24 Or equivalently data covariance log p(y x,, ) = 1 rt C 1 r 1 log det C N log residual vector r = y 1 f (x 1 ) y f (x ) y n f (x n ) T
25 Or equivalently data covariance log p(y x,, ) = 1 rt C 1 r 1 log det C N log residual vector r = y 1 f (x 1 ) y f (x ) y n f (x n ) T
26 Linear least-squares. apple m b = SA T C 1 y S = A T C 1 A 1 maximum likelihood & in this case only mean of posterior posterior covariance A = 6 4 x 1 1 x 1.. x n C = n y = 6 4 y 1 y. y n 3 7 5
27 Linear least-squares. apple m b = SA T C 1 y S = A T C 1 A 1 maximum likelihood & in this case only mean of posterior posterior covariance assuming uniform priors A = 6 4 x 1 1 x 1.. x n C = n y = 6 4 y 1 y. y n 3 7 5
28 truth
29 4 3 1 posterior constraint? truth
30 4 3 1 posterior constraint? truth
31 But we know the true covariance matrix.
32 log p(y x,, ) = 1 rt C 1 r 1 log det C N log
33 Linear least-squares. apple m b = SA T C 1 y S = A T C 1 A 1 maximum likelihood & in this case only mean of posterior posterior covariance A = 6 4 x 1 1 x 1.. x n C = n y = 6 4 y 1 y. y n 3 7 5
34 Before
35 After
36 the responsible scientist. cbd Flickr user MyFWCmedia
37 So we re finished, right?
38 In The Real World, we never know the noise.
39 Just gotta model it!
40
41 Model it! log p(y x,,, ) = 1 rt K 1 r 1 log det K + C where K ij = i ij + k (x i,x j ) for example [xi x j ] k (x i,x j )=a exp l drop-in replacement for your current log-likelihood function!
42 emceethe arxiv.org/abs/ dan.iel.fm/emcee MCMC Hammer it's hammer time!
43 1 0 b 1.5 ln a ln s m b ln a ln s
44
45
46 Prediction?
47
48
49 take a deep breath. cba Flickr user kpjas
50 The formal Gaussian process.
51 The model. log p(y x,,, ) = 1 [y f (x)] T K (x, ) 1 [y f (x)] 1 log det K (x, ) N log where [K (x, )] ij = i ij + k (x i,x j ) drop-in replacement for your current log-likelihood function!
52 The model. y N(f (x), K (x, )) where [K (x, )] ij = i ij + k (x i,x j ) drop-in replacement for your current log-likelihood function!
53 the data are drawn from one HUGE Gaussian * * the dimension is the number of data points.
54 A generative model y N(f (x), K (x, )) a probability distribution for y values
55 Likelihood samples. k (x i,x j )=exp [xi x j ] ` 3 exponential squared l =0.5 l =1 l = t
56 Likelihood samples. k (x i,x j )=exp [xi x j ] ` 3 exponential squared exponential squared l =0.5 l =1 l = t
57 Likelihood samples. k (x i,x j )= " 1+ p # 3 xi x j ` exp xi x j ` cos xi x j P 3 quasi-periodic l =,P=3 l =3,P=3 l =3,P= t
58 Likelihood samples. k (x i,x j )= " 1+ p # 3 xi x j ` exp xi x j ` cos xi x j P 3 quasi-periodic quasi-periodic l =,P=3 l =3,P=3 l =3,P= t
59 The conditional distribution y N(f (x), K (x, )) apple y y? N apple f (x) f (x? ), apple K, x, x K, x,? y? y N K,?,x K 1, x, x [y f (x)] + f (x? ), K,?,? K,?,x K 1, x, x K, x,? ) just see Rasmussen & Williams (Chapter )
60 What s the catch? Kepler = Big Data (by some definition) Note: I hate myself for this slide too
61 Computational complexity. log p(y x,,, ) = 1 [y f (x)] T K (x, ) 1 [y f (x)] 1 log det K (x, ) N log compute factorization // evaluate log-det // apply inverse naïvely: O(N 3 )
62 import numpy as np from scipy.linalg import cho_factor, cho_solve! def simple_gp_lnlike(x, y, yerr, a, s): r = x[:, None] - x[none, :] C = np.diag(yerr**) + a*np.exp(-0.5*r**/(s*s)) factor, flag = cho_factor(c) logdet = *np.sum(np.log(np.diag(factor))) return -0.5 * (np.dot(y, cho_solve((factor, flag), y)) + logdet + len(x)*np.log(*np.pi))
63 1 log 10 runtime/seconds log 10 N
64 exponential squared quasi-periodic
65 exponential squared quasi-periodic
66 exponential squared quasi-periodic t
67 Aren t kernel matrices Hierarchical Off-Diagonal Low-Rank? no astronomer ever
68 = K (3) K 3 K K 1 K 0 Full rank; Low-rank; Identity matrix; Zero matrix; Ambikasaran, DFM, et al. (arxiv: )
69 github.com/dfm/george
70 import numpy as np from george import GaussianProcess, kernels! def george_lnlike(x, y, yerr, a, s): kernel = a * kernels.rbfkernel(s) gp = GaussianProcess(kernel) gp.compute(x, yerr) return gp.lnlikelihood(y)
71 1 log 10 runtime/seconds log 10 N
72 1 log 10 runtime/seconds log 10 N
73 and short cadence data? one month of data in 4 seconds
74 3 Applications to Kepler data.
75 Parameter Recovery
76 time since transit KIC injection
77 time since transit KIC injection
78 time since transit KIC injection
79 after median-filter figure generated detrending github.com/dfm/tr
80 time since transit KIC injection
81 time since transit KIC injection
82 using Gaussian process figure generated noise model github.com/dfm/tr
![time [days] Ambikasaran,](/docs-images/72/66814509/images/83-0.jpg "DFM, et al. (arxiv:1403.")
83 time [days] Ambikasaran, DFM, et al. (arxiv: )
84 q q t r/r? f? b q q t r/r? b
85 KOI with Bekki Dawson, et al.
86
87
88 Stellar Rotation with Ruth Angus
89 figures from Ruth Angus (Oxford)
90 figures from Ruth Angus (Oxford)
91 4 Conclusions & Summary.
92 correlated noise matters. a Gaussian process provides a drop-in replacement likelihood function if you can compute it
93 Resources gaussianprocess.org/gpml github.com/dfm/ gp george
Probabilistic modeling and Inference in Astronomy
Probabilistic modeling and Inference in Astronomy Dan Foreman-Mackey Sagan Fellow, University of Washington github.com/dfm // @exoplaneteer // dfm.io Dan Foreman-Mackey Sagan Fellow, University of Washington
GWAS V: Gaussian processes
GWAS V: Gaussian processes Dr. Oliver Stegle Christoh Lippert Prof. Dr. Karsten Borgwardt Max-Planck-Institutes Tübingen, Germany Tübingen Summer 2011 Oliver Stegle GWAS V: Gaussian processes Summer 2011
Can you predict the future..?
Can you predict the future..? Gaussian Process Modelling for Forward Prediction Anna Scaife 1 1 Jodrell Bank Centre for Astrophysics University of Manchester @radastrat September 7, 2017 Anna Scaife University
Fast Direct Methods for Gaussian Processes
Fast Direct Methods for Gaussian Processes Mike O Neil Departments of Mathematics New York University oneil@cims.nyu.edu December 12, 2015 1 Collaborators This is joint work with: Siva Ambikasaran Dan
COMS 4721: Machine Learning for Data Science Lecture 10, 2/21/2017
COMS 4721: Machine Learning for Data Science Lecture 10, 2/21/2017 Prof. John Paisley Department of Electrical Engineering & Data Science Institute Columbia University FEATURE EXPANSIONS FEATURE EXPANSIONS
Shallow. Deep. Transits Learning
Shallow Deep Transits Learning Shay Zucker, Raja Giryes Elad Dvash (Tel Aviv University) Yam Peleg (Deep Trading) Red Noise and Transit Detection Deep transits: traditional methods (BLS) work well BLS:
Introduction to Gaussian Processes
Introduction to Gaussian Processes Iain Murray murray@cs.toronto.edu CSC255, Introduction to Machine Learning, Fall 28 Dept. Computer Science, University of Toronto The problem Learn scalar function of
Introduction to Gaussian Processes
Introduction to Gaussian Processes Neil D. Lawrence GPSS 10th June 2013 Book Rasmussen and Williams (2006) Outline The Gaussian Density Covariance from Basis Functions Basis Function Representations Constructing
COMP 551 Applied Machine Learning Lecture 20: Gaussian processes
COMP 55 Applied Machine Learning Lecture 2: Gaussian processes Instructor: Ryan Lowe (ryan.lowe@cs.mcgill.ca) Slides mostly by: (herke.vanhoof@mcgill.ca) Class web page: www.cs.mcgill.ca/~hvanho2/comp55
Nonparametric Bayesian Methods (Gaussian Processes)
[70240413 Statistical Machine Learning, Spring, 2015] Nonparametric Bayesian Methods (Gaussian Processes) Jun Zhu dcszj@mail.tsinghua.edu.cn http://bigml.cs.tsinghua.edu.cn/~jun State Key Lab of Intelligent
Gaussian Processes in Machine Learning
Gaussian Processes in Machine Learning November 17, 2011 CharmGil Hong Agenda Motivation GP : How does it make sense? Prior : Defining a GP More about Mean and Covariance Functions Posterior : Conditioning
GAUSSIAN PROCESS REGRESSION
GAUSSIAN PROCESS REGRESSION CSE 515T Spring 2015 1. BACKGROUND The kernel trick again... The Kernel Trick Consider again the linear regression model: y(x) = φ(x) w + ε, with prior p(w) = N (w; 0, Σ). The
Practical Bayesian Optimization of Machine Learning. Learning Algorithms
Practical Bayesian Optimization of Machine Learning Algorithms CS 294 University of California, Berkeley Tuesday, April 20, 2016 Motivation Machine Learning Algorithms (MLA s) have hyperparameters that
Computer Vision Group Prof. Daniel Cremers. 4. Gaussian Processes - Regression
Group Prof. Daniel Cremers 4. Gaussian Processes - Regression Definition (Rep.) Definition: A Gaussian process is a collection of random variables, any finite number of which have a joint Gaussian distribution.
Introduction to Machine Learning
Introduction to Machine Learning 12. Gaussian Processes Alex Smola Carnegie Mellon University http://alex.smola.org/teaching/cmu2013-10-701 10-701 The Normal Distribution http://www.gaussianprocess.org/gpml/chapters/
Nonparametric Regression With Gaussian Processes
Nonparametric Regression With Gaussian Processes From Chap. 45, Information Theory, Inference and Learning Algorithms, D. J. C. McKay Presented by Micha Elsner Nonparametric Regression With Gaussian Processes
Lecture: Gaussian Process Regression. STAT 6474 Instructor: Hongxiao Zhu
Lecture: Gaussian Process Regression STAT 6474 Instructor: Hongxiao Zhu Motivation Reference: Marc Deisenroth s tutorial on Robot Learning. 2 Fast Learning for Autonomous Robots with Gaussian Processes
Gaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012
Gaussian Processes Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 01 Pictorial view of embedding distribution Transform the entire distribution to expected features Feature space Feature
Model Selection for Gaussian Processes
Institute for Adaptive and Neural Computation School of Informatics,, UK December 26 Outline GP basics Model selection: covariance functions and parameterizations Criteria for model selection Marginal
Statistical Techniques in Robotics (16-831, F12) Lecture#20 (Monday November 12) Gaussian Processes
Statistical Techniques in Robotics (6-83, F) Lecture# (Monday November ) Gaussian Processes Lecturer: Drew Bagnell Scribe: Venkatraman Narayanan Applications of Gaussian Processes (a) Inverse Kinematics
Nonparameteric Regression:
Nonparameteric Regression: Nadaraya-Watson Kernel Regression & Gaussian Process Regression Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro,
Computer Vision Group Prof. Daniel Cremers. 9. Gaussian Processes - Regression
Group Prof. Daniel Cremers 9. Gaussian Processes - Regression Repetition: Regularized Regression Before, we solved for w using the pseudoinverse. But: we can kernelize this problem as well! First step:
Econ 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines
Econ 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines Maximilian Kasy Department of Economics, Harvard University 1 / 37 Agenda 6 equivalent representations of the
Nonparmeteric Bayes & Gaussian Processes. Baback Moghaddam Machine Learning Group
Nonparmeteric Bayes & Gaussian Processes Baback Moghaddam baback@jpl.nasa.gov Machine Learning Group Outline Bayesian Inference Hierarchical Models Model Selection Parametric vs. Nonparametric Gaussian
Talk on Bayesian Optimization
Talk on Bayesian Optimization Jungtaek Kim (jtkim@postech.ac.kr) Machine Learning Group, Department of Computer Science and Engineering, POSTECH, 77-Cheongam-ro, Nam-gu, Pohang-si 37673, Gyungsangbuk-do,
Gaussian Process Regression
Gaussian Process Regression 4F1 Pattern Recognition, 21 Carl Edward Rasmussen Department of Engineering, University of Cambridge November 11th - 16th, 21 Rasmussen (Engineering, Cambridge) Gaussian Process
Ages of stellar populations from color-magnitude diagrams. Paul Baines. September 30, 2008
Ages of stellar populations from color-magnitude diagrams Paul Baines Department of Statistics Harvard University September 30, 2008 Context & Example Welcome! Today we will look at using hierarchical
Gaussian Process Approximations of Stochastic Differential Equations
Gaussian Process Approximations of Stochastic Differential Equations Cédric Archambeau Dan Cawford Manfred Opper John Shawe-Taylor May, 2006 1 Introduction Some of the most complex models routinely run
CSci 8980: Advanced Topics in Graphical Models Gaussian Processes
CSci 8980: Advanced Topics in Graphical Models Gaussian Processes Instructor: Arindam Banerjee November 15, 2007 Gaussian Processes Outline Gaussian Processes Outline Parametric Bayesian Regression Gaussian
Machine Learning. Bayesian Regression & Classification. Marc Toussaint U Stuttgart
Machine Learning Bayesian Regression & Classification learning as inference, Bayesian Kernel Ridge regression & Gaussian Processes, Bayesian Kernel Logistic Regression & GP classification, Bayesian Neural
Logistic Regression Logistic
Case Study 1: Estimating Click Probabilities L2 Regularization for Logistic Regression Machine Learning/Statistics for Big Data CSE599C1/STAT592, University of Washington Carlos Guestrin January 10 th,
Gaussian Processes. 1 What problems can be solved by Gaussian Processes?
Statistical Techniques in Robotics (16-831, F1) Lecture#19 (Wednesday November 16) Gaussian Processes Lecturer: Drew Bagnell Scribe:Yamuna Krishnamurthy 1 1 What problems can be solved by Gaussian Processes?
STA414/2104 Statistical Methods for Machine Learning II
STA414/2104 Statistical Methods for Machine Learning II Murat A. Erdogdu & David Duvenaud Department of Computer Science Department of Statistical Sciences Lecture 3 Slide credits: Russ Salakhutdinov Announcements
CS 7140: Advanced Machine Learning
Instructor CS 714: Advanced Machine Learning Lecture 3: Gaussian Processes (17 Jan, 218) Jan-Willem van de Meent (j.vandemeent@northeastern.edu) Scribes Mo Han (han.m@husky.neu.edu) Guillem Reus Muns (reusmuns.g@husky.neu.edu)
Gaussian Processes for Machine Learning
Gaussian Processes for Machine Learning Carl Edward Rasmussen Max Planck Institute for Biological Cybernetics Tübingen, Germany carl@tuebingen.mpg.de Carlos III, Madrid, May 2006 The actual science of
20: Gaussian Processes
10-708: Probabilistic Graphical Models 10-708, Spring 2016 20: Gaussian Processes Lecturer: Andrew Gordon Wilson Scribes: Sai Ganesh Bandiatmakuri 1 Discussion about ML Here we discuss an introduction
Computer Vision Group Prof. Daniel Cremers. 2. Regression (cont.)
Prof. Daniel Cremers 2. Regression (cont.) Regression with MLE (Rep.) Assume that y is affected by Gaussian noise : t = f(x, w)+ where Thus, we have p(t x, w, )=N (t; f(x, w), 2 ) 2 Maximum A-Posteriori
Introduction to Gaussian Processes
Introduction to Gaussian Processes Iain Murray School of Informatics, University of Edinburgh The problem Learn scalar function of vector values f(x).5.5 f(x) y i.5.2.4.6.8 x f 5 5.5 x x 2.5 We have (possibly
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
Expected precision on planet radii with
Expected precision on planet radii with Adrien Deline 1, Didier Queloz 1,2 1 University of Geneva 2 University of Cambridge 24 th 26 th July 2017 CHEOPS Science Workshop 5 Schloss Seggau, Austria Field
Multiple-step Time Series Forecasting with Sparse Gaussian Processes
Multiple-step Time Series Forecasting with Sparse Gaussian Processes Perry Groot ab Peter Lucas a Paul van den Bosch b a Radboud University, Model-Based Systems Development, Heyendaalseweg 135, 6525 AJ
Neutron inverse kinetics via Gaussian Processes
Neutron inverse kinetics via Gaussian Processes P. Picca Politecnico di Torino, Torino, Italy R. Furfaro University of Arizona, Tucson, Arizona Outline Introduction Review of inverse kinetics techniques
Statistical Techniques in Robotics (16-831, F12) Lecture#21 (Monday November 12) Gaussian Processes
Statistical Techniques in Robotics (16-831, F12) Lecture#21 (Monday November 12) Gaussian Processes Lecturer: Drew Bagnell Scribe: Venkatraman Narayanan 1, M. Koval and P. Parashar 1 Applications of Gaussian
Part 6: Multivariate Normal and Linear Models
Part 6: Multivariate Normal and Linear Models 1 Multiple measurements Up until now all of our statistical models have been univariate models models for a single measurement on each member of a sample of
Measurement Error and Linear Regression of Astronomical Data. Brandon Kelly Penn State Summer School in Astrostatistics, June 2007
Measurement Error and Linear Regression of Astronomical Data Brandon Kelly Penn State Summer School in Astrostatistics, June 2007 Classical Regression Model Collect n data points, denote i th pair as (η
The geometry of Gaussian processes and Bayesian optimization. Contal CMLA, ENS Cachan
The geometry of Gaussian processes and Bayesian optimization. Contal CMLA, ENS Cachan Background: Global Optimization and Gaussian Processes The Geometry of Gaussian Processes and the Chaining Trick Algorithm
arxiv: v2 [astro-ph.ep] 3 Sep 2014
![arxiv: v2 [astro-ph.ep] 3 Sep 2014](/thumbs/86/94158400.jpg "arxiv: v2 [astro-ph.ep] 3 Sep 2014")
Exoplanet population inference and the abundance of Earth analogs from noisy, incomplete catalogs Daniel Foreman-Mackey 1,2, David W. Hogg 2,3,4, Timothy D. Morton 5 ABSTRACT arxiv:1406.3020v2 [astro-ph.ep]
Probabilistic numerics for deep learning
Presenter: Shijia Wang Department of Engineering Science, University of Oxford rning (RLSS) Summer School, Montreal 2017 Outline 1 Introduction Probabilistic Numerics 2 Components Probabilistic modeling
Gaussian processes. Chuong B. Do (updated by Honglak Lee) November 22, 2008
Gaussian processes Chuong B Do (updated by Honglak Lee) November 22, 2008 Many of the classical machine learning algorithms that we talked about during the first half of this course fit the following pattern:
Nearest Neighbor. Machine Learning CSE546 Kevin Jamieson University of Washington. October 26, Kevin Jamieson 2
Nearest Neighbor Machine Learning CSE546 Kevin Jamieson University of Washington October 26, 2017 2017 Kevin Jamieson 2 Some data, Bayes Classifier Training data: True label: +1 True label: -1 Optimal
Click Prediction and Preference Ranking of RSS Feeds
Click Prediction and Preference Ranking of RSS Feeds 1 Introduction December 11, 2009 Steven Wu RSS (Really Simple Syndication) is a family of data formats used to publish frequently updated works. RSS
Kernels for Automatic Pattern Discovery and Extrapolation
Kernels for Automatic Pattern Discovery and Extrapolation Andrew Gordon Wilson agw38@cam.ac.uk mlg.eng.cam.ac.uk/andrew University of Cambridge Joint work with Ryan Adams (Harvard) 1 / 21 Pattern Recognition
Logistic Regression. Jia-Bin Huang. Virginia Tech Spring 2019 ECE-5424G / CS-5824
Logistic Regression Jia-Bin Huang ECE-5424G / CS-5824 Virginia Tech Spring 2019 Administrative Please start HW 1 early! Questions are welcome! Two principles for estimating parameters Maximum Likelihood
Gaussian processes and bayesian optimization Stanisław Jastrzębski. kudkudak.github.io kudkudak
Gaussian processes and bayesian optimization Stanisław Jastrzębski kudkudak.github.io kudkudak Plan Goal: talk about modern hyperparameter optimization algorithms Bayes reminder: equivalent linear regression
An Introduction to Gaussian Processes for Spatial Data (Predictions!)
An Introduction to Gaussian Processes for Spatial Data (Predictions!) Matthew Kupilik College of Engineering Seminar Series Nov 216 Matthew Kupilik UAA GP for Spatial Data Nov 216 1 / 35 Why? When evidence
STA414/2104. Lecture 11: Gaussian Processes. Department of Statistics
STA414/2104 Lecture 11: Gaussian Processes Department of Statistics www.utstat.utoronto.ca Delivered by Mark Ebden with thanks to Russ Salakhutdinov Outline Gaussian Processes Exam review Course evaluations
Advanced Introduction to Machine Learning CMU-10715
Advanced Introduction to Machine Learning CMU-10715 Gaussian Processes Barnabás Póczos http://www.gaussianprocess.org/ 2 Some of these slides in the intro are taken from D. Lizotte, R. Parr, C. Guesterin
Markov Chain Monte Carlo
Department of Statistics The University of Auckland https://www.stat.auckland.ac.nz/~brewer/ Emphasis I will try to emphasise the underlying ideas of the methods. I will not be teaching specific software
EXONEST The Exoplanetary Explorer. Kevin H. Knuth and Ben Placek Department of Physics University at Albany (SUNY) Albany NY
EXONEST The Exoplanetary Explorer Kevin H. Knuth and Ben Placek Department of Physics University at Albany (SUNY) Albany NY Kepler Mission The Kepler mission, launched in 2009, aims to explore the structure
ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING. Non-linear regression techniques Part - II
1 Non-linear regression techniques Part - II Regression Algorithms in this Course Support Vector Machine Relevance Vector Machine Support vector regression Boosting random projections Relevance vector
COS513: FOUNDATIONS OF PROBABILISTIC MODELS LECTURE 9: LINEAR REGRESSION
COS513: FOUNDATIONS OF PROBABILISTIC MODELS LECTURE 9: LINEAR REGRESSION SEAN GERRISH AND CHONG WANG 1. WAYS OF ORGANIZING MODELS In probabilistic modeling, there are several ways of organizing models:
The joint posterior distribution of the unknown parameters and hidden variables, given the
DERIVATIONS OF THE FULLY CONDITIONAL POSTERIOR DENSITIES The joint posterior distribution of the unknown parameters and hidden variables, given the data, is proportional to the product of the joint prior
PILCO: A Model-Based and Data-Efficient Approach to Policy Search
PILCO: A Model-Based and Data-Efficient Approach to Policy Search (M.P. Deisenroth and C.E. Rasmussen) CSC2541 November 4, 2016 PILCO Graphical Model PILCO Probabilistic Inference for Learning COntrol
Parameter estimation and forecasting. Cristiano Porciani AIfA, Uni-Bonn
Parameter estimation and forecasting Cristiano Porciani AIfA, Uni-Bonn Questions? C. Porciani Estimation & forecasting 2 Temperature fluctuations Variance at multipole l (angle ~180o/l) C. Porciani Estimation
Reliability Monitoring Using Log Gaussian Process Regression
COPYRIGHT 013, M. Modarres Reliability Monitoring Using Log Gaussian Process Regression Martin Wayne Mohammad Modarres PSA 013 Center for Risk and Reliability University of Maryland Department of Mechanical
L 0 methods. H.J. Kappen Donders Institute for Neuroscience Radboud University, Nijmegen, the Netherlands. December 5, 2011.
L methods H.J. Kappen Donders Institute for Neuroscience Radboud University, Nijmegen, the Netherlands December 5, 2 Bert Kappen Outline George McCullochs model The Variational Garrote Bert Kappen L methods
Data-driven models of stars
Data-driven models of stars David W. Hogg Center for Cosmology and Particle Physics, New York University Center for Data Science, New York University Max-Planck-Insitut für Astronomie, Heidelberg 2015
Linear Dynamical Systems (Kalman filter)
Linear Dynamical Systems (Kalman filter) (a) Overview of HMMs (b) From HMMs to Linear Dynamical Systems (LDS) 1 Markov Chains with Discrete Random Variables x 1 x 2 x 3 x T Let s assume we have discrete
DEPARTMENT OF COMPUTER SCIENCE AUTUMN SEMESTER MACHINE LEARNING AND ADAPTIVE INTELLIGENCE
Data Provided: None DEPARTMENT OF COMPUTER SCIENCE AUTUMN SEMESTER 204 205 MACHINE LEARNING AND ADAPTIVE INTELLIGENCE hour Please note that the rubric of this paper is made different from many other papers.
Probabilistic Models for Learning Data Representations. Andreas Damianou
Probabilistic Models for Learning Data Representations Andreas Damianou Department of Computer Science, University of Sheffield, UK IBM Research, Nairobi, Kenya, 23/06/2015 Sheffield SITraN Outline Part
Chapter 6. Order Statistics and Quantiles. 6.1 Extreme Order Statistics
Chapter 6 Order Statistics and Quantiles 61 Extreme Order Statistics Suppose we have a finite sample X 1,, X n Conditional on this sample, we define the values X 1),, X n) to be a permutation of X 1,,
Bayesian Support Vector Machines for Feature Ranking and Selection
Bayesian Support Vector Machines for Feature Ranking and Selection written by Chu, Keerthi, Ong, Ghahramani Patrick Pletscher pat@student.ethz.ch ETH Zurich, Switzerland 12th January 2006 Overview 1 Introduction
Inference of cluster distance and geometry from astrometry
Inference of cluster distance and geometry from astrometry Coryn A.L. Bailer-Jones Max Planck Institute for Astronomy, Heidelberg Email: calj@mpia.de 2017-12-01 Abstract Here I outline a method to determine
Bayesian Linear Regression [DRAFT - In Progress]
![Bayesian Linear Regression [DRAFT - In Progress]](/thumbs/92/109828713.jpg "Bayesian Linear Regression [DRAFT - In Progress]")
Bayesian Linear Regression [DRAFT - In Progress] David S. Rosenberg Abstract Here we develop some basics of Bayesian linear regression. Most of the calculations for this document come from the basic theory
Parameter Estimation. William H. Jefferys University of Texas at Austin Parameter Estimation 7/26/05 1
Parameter Estimation William H. Jefferys University of Texas at Austin bill@bayesrules.net Parameter Estimation 7/26/05 1 Elements of Inference Inference problems contain two indispensable elements: Data
Gaussian Processes (10/16/13)
STA561: Probabilistic machine learning Gaussian Processes (10/16/13) Lecturer: Barbara Engelhardt Scribes: Changwei Hu, Di Jin, Mengdi Wang 1 Introduction In supervised learning, we observe some inputs
CPSC 540: Machine Learning
CPSC 540: Machine Learning MCMC and Non-Parametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is
Future Opportunities for Collaborations: Exoplanet Astronomers & Statisticians
Future Opportunities for Collaborations: Exoplanet Astronomers & Statisticians Eric B. Ford Penn State Astronomy & Astrophysics Center for Astrostatistics Center for Exoplanets & Habitable Worlds Institute
Hierarchical Bayesian Modeling of Planet Populations
Hierarchical Bayesian Modeling of Planet Populations You ve found planets in your data (or not)...... now what?! Angie Wolfgang NSF Postdoctoral Fellow, Penn State Why Astrostats? This week: a sample of
CPSC 540: Machine Learning
CPSC 540: Machine Learning Undirected Graphical Models Mark Schmidt University of British Columbia Winter 2016 Admin Assignment 3: 2 late days to hand it in today, Thursday is final day. Assignment 4:
Gaussian processes in python
Gaussian processes in python John Reid 17th March 2009 1 What are Gaussian processes? Often we have an inference problem involving n data, D = {(x i, y i i = 1,..., n, x i X, y i R} where the x i are the
Gaussian process regression for Sensitivity analysis
Gaussian process regression for Sensitivity analysis GPSS Workshop on UQ, Sheffield, September 2016 Nicolas Durrande, Mines St-Étienne, durrande@emse.fr GPSS workshop on UQ GPs for sensitivity analysis
Machine Learning - MT & 5. Basis Expansion, Regularization, Validation
Machine Learning - MT 2016 4 & 5. Basis Expansion, Regularization, Validation Varun Kanade University of Oxford October 19 & 24, 2016 Outline Basis function expansion to capture non-linear relationships
Hierarchical Bayesian Modeling of Planet Populations. Angie Wolfgang NSF Postdoctoral Fellow, Penn State
Hierarchical Bayesian Modeling of Planet Populations Angie Wolfgang NSF Postdoctoral Fellow, Penn State The Big Picture We ve found > 3000 planets, and counting. Earth s place in the Universe... We re
Introduction to Machine Learning
Introduction to Machine Learning 3. Instance Based Learning Alex Smola Carnegie Mellon University http://alex.smola.org/teaching/cmu2013-10-701 10-701 Outline Parzen Windows Kernels, algorithm Model selection
Prediction of Data with help of the Gaussian Process Method
of Data with help of the Gaussian Process Method R. Preuss, U. von Toussaint Max-Planck-Institute for Plasma Physics EURATOM Association 878 Garching, Germany March, Abstract The simulation of plasma-wall
How to build an automatic statistician
How to build an automatic statistician James Robert Lloyd 1, David Duvenaud 1, Roger Grosse 2, Joshua Tenenbaum 2, Zoubin Ghahramani 1 1: Department of Engineering, University of Cambridge, UK 2: Massachusetts
A Brief and Friendly Introduction to Mixed-Effects Models in Linguistics
A Brief and Friendly Introduction to Mixed-Effects Models in Linguistics Cluster-specific parameters ( random effects ) Σb Parameters governing inter-cluster variability b1 b2 bm x11 x1n1 x21 x2n2 xm1
Gaussian with mean ( µ ) and standard deviation ( σ)
Slide from Pieter Abbeel Gaussian with mean ( µ ) and standard deviation ( σ) 10/6/16 CSE-571: Robotics X ~ N( µ, σ ) Y ~ N( aµ + b, a σ ) Y = ax + b + + + + 1 1 1 1 1 1 1 1 1 1, ~ ) ( ) ( ), ( ~ ), (
Learning Gaussian Process Models from Uncertain Data
Learning Gaussian Process Models from Uncertain Data Patrick Dallaire, Camille Besse, and Brahim Chaib-draa DAMAS Laboratory, Computer Science & Software Engineering Department, Laval University, Canada
CSC2541 Lecture 2 Bayesian Occam s Razor and Gaussian Processes
CSC2541 Lecture 2 Bayesian Occam s Razor and Gaussian Processes Roger Grosse Roger Grosse CSC2541 Lecture 2 Bayesian Occam s Razor and Gaussian Processes 1 / 55 Adminis-Trivia Did everyone get my e-mail
Naïve Bayes. Jia-Bin Huang. Virginia Tech Spring 2019 ECE-5424G / CS-5824
Naïve Bayes Jia-Bin Huang ECE-5424G / CS-5824 Virginia Tech Spring 2019 Administrative HW 1 out today. Please start early! Office hours Chen: Wed 4pm-5pm Shih-Yang: Fri 3pm-4pm Location: Whittemore 266
Adaptive Sampling of Clouds with a Fleet of UAVs: Improving Gaussian Process Regression by Including Prior Knowledge
Master s Thesis Presentation Adaptive Sampling of Clouds with a Fleet of UAVs: Improving Gaussian Process Regression by Including Prior Knowledge Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master s Thesis Presentation
Where now? Machine Learning and Bayesian Inference
Machine Learning and Bayesian Inference Dr Sean Holden Computer Laboratory, Room FC6 Telephone etension 67 Email: sbh@clcamacuk wwwclcamacuk/ sbh/ Where now? There are some simple take-home messages from
Probabilistic & Unsupervised Learning
Probabilistic & Unsupervised Learning Gaussian Processes Maneesh Sahani maneesh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc ML/CSML, Dept Computer Science University College London
COMS 4721: Machine Learning for Data Science Lecture 19, 4/6/2017
COMS 4721: Machine Learning for Data Science Lecture 19, 4/6/2017 Prof. John Paisley Department of Electrical Engineering & Data Science Institute Columbia University PRINCIPAL COMPONENT ANALYSIS DIMENSIONALITY
Gaussian Process Optimization with Mutual Information
Gaussian Process Optimization with Mutual Information Emile Contal 1 Vianney Perchet 2 Nicolas Vayatis 1 1 CMLA Ecole Normale Suprieure de Cachan & CNRS, France 2 LPMA Université Paris Diderot & CNRS,
CSE 559A: Computer Vision
CSE 559A: Computer Vision Fall 208: T-R: :30-pm @ Lopata 0 Instructor: Ayan Chakrabarti (ayan@wustl.edu). Course Staff: Zhihao ia, Charlie Wu, Han Liu http://www.cse.wustl.edu/~ayan/courses/cse559a/ Sep