Introduction to Probabilistic Reasoning
|
|
- Julie Hall
- 5 years ago
- Views:
Transcription
1 Introduction to Probabilistic Reasoning inference, forecasting, decision Giulio D Agostini giulio.dagostini@roma.infn.it Dipartimento di Fisica Università di Roma La Sapienza Part 3 Probability is good sense reduced to a calculus (Laplace) G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p.
2 Contents Summary of relevant formulae Propagating uncertainty the straight way Simple case of the Gaussian distribution : when (and why) ML provides a good estimates and why it can never tell something meaningfull about uncertainties the role of conjugate priors, with application to the Bernoulli trials ( binomial problem ): beta pdf. Handling systematics, with the special case of two measurements with Gaussian response of the same detector, affected by a possible systematic error: how the overall uncertainty increases how the two results become correlated Examples with OpenBUGS (see web page) [ JAGS G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 2
3 Important formulae (For continous uncertain variables) product rule: f xy (x,y I) = f x y (x y,i)f y (y I) independence: f xy (x,y I) = f x (x I)f y (y I) marginalizzation: f xy (x,y I)dy = f x (x I) f xy (x,y I)dx = f y (y I) decomposition: f x (x I) = f x y (x y,i)f y (y I)dy f y (y I) = f y x (y x,i)f x (x I)dx weighted average G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 3
4 Important formulae continued Bayes rule (using observation x and true value µ and simplifying the notation) f(µ x,i) = (µ,x I) f(x I) = f(x µ,i)f(µ I) f(x I) = f(x µ,i)f(µ I) f(x µ,i)f(µ I)dµ f(x µ,i)f(µ I) (In the following I will be implicit in most cases) G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 4
5 Propagation of uncertainties What has to do with Bayesian? G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 5
6 Propagation of uncertainties What has to do with Bayesian? Think what most of you have been taught in laboratory courses (typically): Use probabilistic formulae for object that are not random variables (in the frequentistic approach) G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 5
7 Propagation of uncertainties What has to do with Bayesian? Think what most of you have been taught in laboratory courses (typically): Use probabilistic formulae for object that are not random variables (in the frequentistic approach) An insult to logic!! G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 5
8 Propagation of uncertainties What has to do with Bayesian? Think what most of you have been taught in laboratory courses (typically): Use probabilistic formulae for object that are not random variables (in the frequentistic approach) An insult to logic!! In the Bayesian approach it is straightforward because true values are uncertain variables. G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 5
9 Propagation of uncertainties What has to do with Bayesian? Think what most of you have been taught in laboratory courses (typically): Use probabilistic formulae for object that are not random variables (in the frequentistic approach) An insult to logic!! In the Bayesian approach it is straightforward because true values are uncertain variables. For details see 2005 CERN lectures (nr. 4, pp. 7-28) conferencedisplay.py?confid=a04375 G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 5
10 back to our slides a very simple inference in a gaussian model conjugate priors systematics G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 6
11 Simple case of Gaussian errors x N(µσ): f(x µ,i) = f(µ x,i) = IF f(µ I) const f(µ x,i) = E[µ, σ(µ), etc. 2πσ 2σ 2 2πσ 2σ 2 f(µ I) + 2πσ 2σ 2 f(µ I) dµ [ (µ x)2 2πσ 2σ 2. G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 7
12 Simple case of Gaussian errors x N(µσ): f(x µ,i) = f(µ x,i) = IF f(µ I) const f(µ x,i) = E[µ, σ(µ), etc. 2πσ 2σ 2 2πσ 2σ 2 f(µ I) + 2πσ 2σ 2 f(µ I) dµ [ (µ x)2 2πσ 2σ 2. maximum of posterior = maximum of likelihood G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 7
13 Simple case of Gaussian errors x N(µσ): f(x µ,i) = f(µ x,i) = IF f(µ I) const f(µ x,i) = E[µ, σ(µ), etc. 2πσ 2σ 2 2πσ 2σ 2 f(µ I) + 2πσ 2σ 2 f(µ I) dµ frequentistic equivalent of σ(µ)? [ (µ x)2 2πσ 2σ 2. G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 7
14 Simple case of Gaussian errors x N(µσ): f(x µ,i) = f(µ x,i) = IF f(µ I) const f(µ x,i) = E[µ, σ(µ), etc. 2πσ 2σ 2 2πσ 2σ 2 f(µ I) + 2πσ 2σ 2 f(µ I) dµ [ (µ x)2 2πσ 2σ 2. frequentistic equivalent of σ(µ)? NO! G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 7
15 Simple case of Gaussian errors x N(µσ): f(x µ,i) = f(µ x,i) = IF f(µ I) const f(µ x,i) = E[µ, σ(µ), etc. 2πσ 2σ 2 2πσ 2σ 2 f(µ I) + 2πσ 2σ 2 f(µ I) dµ [ (µ x)2 2πσ 2σ 2. frequentistic equivalent of σ(µ)? NO! prescriptions!! G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 7
16 Prior conjugate Since ever there have been computational problems: survive with approximations e.g. Gaussian approximation of the posterior (somehow a reflex of the Central Limit theorem) G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 8
17 Prior conjugate Since ever there have been computational problems: survive with approximations e.g. Gaussian approximation of the posterior (somehow a reflex of the Central Limit theorem) Another famous approximation is to chose a proper shape for the prior: compromize between real beliefs and easy math! G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 8
18 Prior conjugate Since ever there have been computational problems: survive with approximations e.g. Gaussian approximation of the posterior (somehow a reflex of the Central Limit theorem) Another famous approximation is to chose a proper shape for the prior: compromize between real beliefs and easy math! inferring p of binomial black/green board G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 8
19 Beta distribution X Beta(r,s)): f(x Beta(r,s)) = β(r,s) xr ( x) s Denominator is just normalization, i.e. { r, s > 0 0 x. β(r,s) = 0 x r ( x) s dx. beta function, resulting in β(r,s) = Γ(r)Γ(s) Γ(r+s). very flexible distribution file beta_distribution.pdf G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 9
20 Uncertainties due to systematics This is another subjects where the frequentistic approach fails miserably: no consistent theory, just prescriptions. add them linearly 2. add the quadratically 3. do if small, 2 if large;... G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 0
21 Uncertainties due to systematics This is another subjects where the frequentistic approach fails miserably: no consistent theory, just prescriptions. add them linearly 2. add the quadratically 3. do if small, 2 if large;... Straightforward in the bayesian approach: influence quantities from which the final results may depend (calibrations constants, etc.) are characterized by an uncertain value, and hence we can attach to them probabilities, and use probability theory. (This is what, in practice, also physicists wh think to adhere to fquentistic school finally do, as in error propagation.) G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 0
22 . Conditional inference Let s make esplicit one of the pieces of information: f(µ x 0,I) = f(µ x 0,h,I 0 ) f(µ x o ) f(µ x o,h) true value µ o µ f(x µ o,h) f(x µ o ) observed value x o x G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p.
23 . Conditional inference Let s make esplicit one of the pieces of information: f(µ x 0,I) = f(µ x 0,h,I 0 ) then use a probability theory f(µ x 0,I) = f(µ x 0,h,I 0 )f(h I 0 )dh Easy, logical, it does work try it! G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p.
24 2. Joint inference + marginalization We can infer both µ and h from the data (One piece of data, two inferred quantities? So what? they will just be 00% correlated) f(µ,h x 0,I) f(x 0 µ,h,i) f 0 (µ,h I) And then apply marginalization with respect to the so called nuisance parameter f(µ x 0,I) = f(µh x 0,I 0 )f(h I 0 )dh G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 2
25 3. Raw result corrected result As third possibility, we might think at a raw result obtained at a fixed value of h (its nominal, best value) only affected by uncertainties due to random errors (or all others, a part those coming from hour imperfect knpwledge about h f(µr x 0,I) Then think to a corrections due to all possible values of h, consistently with out best knowledge: [g(µ R,h) is not a pdf! µ = g(µ R,h) then use propagation of uncertainties Example: µ = µ R +z, where z is an offset known to be 0±σ z one of the assigned problems G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 3
26 Common systematic correlation As well understood, if measurements have a systematic effect in common, the results will become correlated. As it happens when two quantities depend from a third one: µ = µ R +z µ 2 = µ R2 +z common uncertainty in z affects µ and µ 2 in the same direction: µ and µ 2 will become positively correlated. assigned problem For a more detailed example, using reasoning 2 see file em common_systematics.pdf (sections ) G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 4
27 Conclusions G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 5
28 Conclusions Take the courage to use it! G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 5
29 Conclusions It s easy if you try...! G. D Agostini, Probabilistic Reasoning [3 (Stellenbosch, 23 November 203) c G. D Agostini p. 5
Bayesian Models in Machine Learning
Bayesian Models in Machine Learning Lukáš Burget Escuela de Ciencias Informáticas 2017 Buenos Aires, July 24-29 2017 Frequentist vs. Bayesian Frequentist point of view: Probability is the frequency of
More informationComputational Perception. Bayesian Inference
Computational Perception 15-485/785 January 24, 2008 Bayesian Inference The process of probabilistic inference 1. define model of problem 2. derive posterior distributions and estimators 3. estimate parameters
More informationCSC321 Lecture 18: Learning Probabilistic Models
CSC321 Lecture 18: Learning Probabilistic Models Roger Grosse Roger Grosse CSC321 Lecture 18: Learning Probabilistic Models 1 / 25 Overview So far in this course: mainly supervised learning Language modeling
More informationPreliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com
1 School of Oriental and African Studies September 2015 Department of Economics Preliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com Gujarati D. Basic Econometrics, Appendix
More informationProbability and Estimation. Alan Moses
Probability and Estimation Alan Moses Random variables and probability A random variable is like a variable in algebra (e.g., y=e x ), but where at least part of the variability is taken to be stochastic.
More informationBayesian analysis in nuclear physics
Bayesian analysis in nuclear physics Ken Hanson T-16, Nuclear Physics; Theoretical Division Los Alamos National Laboratory Tutorials presented at LANSCE Los Alamos Neutron Scattering Center July 25 August
More informationBayesian Inference for Normal Mean
Al Nosedal. University of Toronto. November 18, 2015 Likelihood of Single Observation The conditional observation distribution of y µ is Normal with mean µ and variance σ 2, which is known. Its density
More informationMachine Learning. Bayes Basics. Marc Toussaint U Stuttgart. Bayes, probabilities, Bayes theorem & examples
Machine Learning Bayes Basics Bayes, probabilities, Bayes theorem & examples Marc Toussaint U Stuttgart So far: Basic regression & classification methods: Features + Loss + Regularization & CV All kinds
More informationProbability. Machine Learning and Pattern Recognition. Chris Williams. School of Informatics, University of Edinburgh. August 2014
Probability Machine Learning and Pattern Recognition Chris Williams School of Informatics, University of Edinburgh August 2014 (All of the slides in this course have been adapted from previous versions
More informationA Theory of Measurement Uncertainty Based on Conditional Probability
Roma N.079 November 996 arxiv:physics/9606v [physics.data-an] 2 Nov 996 A Theory of Measurement Uncertainty Based on Conditional Probability Giulio D Agostini Dip. di Fisica, Università La Sapienza and
More informationArtificial Intelligence
Artificial Intelligence Probabilities Marc Toussaint University of Stuttgart Winter 2018/19 Motivation: AI systems need to reason about what they know, or not know. Uncertainty may have so many sources:
More informationCOMP 551 Applied Machine Learning Lecture 19: Bayesian Inference
COMP 551 Applied Machine Learning Lecture 19: Bayesian Inference Associate Instructor: (herke.vanhoof@mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/comp551 Unless otherwise noted, all material posted
More informationLecture 5. G. Cowan Lectures on Statistical Data Analysis Lecture 5 page 1
Lecture 5 1 Probability (90 min.) Definition, Bayes theorem, probability densities and their properties, catalogue of pdfs, Monte Carlo 2 Statistical tests (90 min.) general concepts, test statistics,
More informationBayesian Inference. STA 121: Regression Analysis Artin Armagan
Bayesian Inference STA 121: Regression Analysis Artin Armagan Bayes Rule...s! Reverend Thomas Bayes Posterior Prior p(θ y) = p(y θ)p(θ)/p(y) Likelihood - Sampling Distribution Normalizing Constant: p(y
More informationLecture : Probabilistic Machine Learning
Lecture : Probabilistic Machine Learning Riashat Islam Reasoning and Learning Lab McGill University September 11, 2018 ML : Many Methods with Many Links Modelling Views of Machine Learning Machine Learning
More informationStatistical Methods in Particle Physics Lecture 1: Bayesian methods
Statistical Methods in Particle Physics Lecture 1: Bayesian methods SUSSP65 St Andrews 16 29 August 2009 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk www.pp.rhul.ac.uk/~cowan
More informationBayesian RL Seminar. Chris Mansley September 9, 2008
Bayesian RL Seminar Chris Mansley September 9, 2008 Bayes Basic Probability One of the basic principles of probability theory, the chain rule, will allow us to derive most of the background material in
More informationWhy Try Bayesian Methods? (Lecture 5)
Why Try Bayesian Methods? (Lecture 5) Tom Loredo Dept. of Astronomy, Cornell University http://www.astro.cornell.edu/staff/loredo/bayes/ p.1/28 Today s Lecture Problems you avoid Ambiguity in what is random
More informationCS 361: Probability & Statistics
October 17, 2017 CS 361: Probability & Statistics Inference Maximum likelihood: drawbacks A couple of things might trip up max likelihood estimation: 1) Finding the maximum of some functions can be quite
More informationPhysics 509: Propagating Systematic Uncertainties. Scott Oser Lecture #12
Physics 509: Propagating Systematic Uncertainties Scott Oser Lecture #1 1 Additive offset model Suppose we take N measurements from a distribution, and wish to estimate the true mean of the underlying
More informationBasics on Probability. Jingrui He 09/11/2007
Basics on Probability Jingrui He 09/11/2007 Coin Flips You flip a coin Head with probability 0.5 You flip 100 coins How many heads would you expect Coin Flips cont. You flip a coin Head with probability
More informationIntroduction to Systems Analysis and Decision Making Prepared by: Jakub Tomczak
Introduction to Systems Analysis and Decision Making Prepared by: Jakub Tomczak 1 Introduction. Random variables During the course we are interested in reasoning about considered phenomenon. In other words,
More informationData Analysis and Monte Carlo Methods
Lecturer: Allen Caldwell, Max Planck Institute for Physics & TUM Recitation Instructor: Oleksander (Alex) Volynets, MPP & TUM General Information: - Lectures will be held in English, Mondays 16-18:00 -
More informationEstimation of reliability parameters from Experimental data (Parte 2) Prof. Enrico Zio
Estimation of reliability parameters from Experimental data (Parte 2) This lecture Life test (t 1,t 2,...,t n ) Estimate θ of f T t θ For example: λ of f T (t)= λe - λt Classical approach (frequentist
More informationConditional distributions
Conditional distributions Will Monroe July 6, 017 with materials by Mehran Sahami and Chris Piech Independence of discrete random variables Two random variables are independent if knowing the value of
More informationFoundations of Statistical Inference
Foundations of Statistical Inference Julien Berestycki Department of Statistics University of Oxford MT 2016 Julien Berestycki (University of Oxford) SB2a MT 2016 1 / 20 Lecture 6 : Bayesian Inference
More informationProbabilistic Inference and Applications to Frontier Physics Part 2
Probabilistic Inference and Applications to Frontier Physics Part 2 Giulio D Agostini Dipartimento di Fisica Università di Roma La Sapienza G. D Agostini, Probabilistic Inference (MAPSES - Lecce 23-24/11/2011)
More informationStatistics: Learning models from data
DS-GA 1002 Lecture notes 5 October 19, 2015 Statistics: Learning models from data Learning models from data that are assumed to be generated probabilistically from a certain unknown distribution is a crucial
More informationBayesian Data Analysis
An Introduction to Bayesian Data Analysis Lecture 1 D. S. Sivia St. John s College Oxford, England April 29, 2012 Some recommended books........................................................ 2 Outline.......................................................................
More informationPARAMETER ESTIMATION: BAYESIAN APPROACH. These notes summarize the lectures on Bayesian parameter estimation.
PARAMETER ESTIMATION: BAYESIAN APPROACH. These notes summarize the lectures on Bayesian parameter estimation.. Beta Distribution We ll start by learning about the Beta distribution, since we end up using
More informationBayesian Inference: What, and Why?
Winter School on Big Data Challenges to Modern Statistics Geilo Jan, 2014 (A Light Appetizer before Dinner) Bayesian Inference: What, and Why? Elja Arjas UH, THL, UiO Understanding the concepts of randomness
More informationDS-GA 1002 Lecture notes 11 Fall Bayesian statistics
DS-GA 100 Lecture notes 11 Fall 016 Bayesian statistics In the frequentist paradigm we model the data as realizations from a distribution that depends on deterministic parameters. In contrast, in Bayesian
More informationSystematic uncertainties in statistical data analysis for particle physics. DESY Seminar Hamburg, 31 March, 2009
Systematic uncertainties in statistical data analysis for particle physics DESY Seminar Hamburg, 31 March, 2009 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk www.pp.rhul.ac.uk/~cowan
More informationMachine Learning. Probability Basics. Marc Toussaint University of Stuttgart Summer 2014
Machine Learning Probability Basics Basic definitions: Random variables, joint, conditional, marginal distribution, Bayes theorem & examples; Probability distributions: Binomial, Beta, Multinomial, Dirichlet,
More informationMachine Learning 4771
Machine Learning 4771 Instructor: Tony Jebara Topic 11 Maximum Likelihood as Bayesian Inference Maximum A Posteriori Bayesian Gaussian Estimation Why Maximum Likelihood? So far, assumed max (log) likelihood
More informationBayesian inference. Fredrik Ronquist and Peter Beerli. October 3, 2007
Bayesian inference Fredrik Ronquist and Peter Beerli October 3, 2007 1 Introduction The last few decades has seen a growing interest in Bayesian inference, an alternative approach to statistical inference.
More informationLectures on Statistical Data Analysis
Lectures on Statistical Data Analysis London Postgraduate Lectures on Particle Physics; University of London MSci course PH4515 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk
More information2. A Basic Statistical Toolbox
. A Basic Statistical Toolbo Statistics is a mathematical science pertaining to the collection, analysis, interpretation, and presentation of data. Wikipedia definition Mathematical statistics: concerned
More informationA Very Brief Summary of Bayesian Inference, and Examples
A Very Brief Summary of Bayesian Inference, and Examples Trinity Term 009 Prof Gesine Reinert Our starting point are data x = x 1, x,, x n, which we view as realisations of random variables X 1, X,, X
More informationDS-GA 1003: Machine Learning and Computational Statistics Homework 7: Bayesian Modeling
DS-GA 1003: Machine Learning and Computational Statistics Homework 7: Bayesian Modeling Due: Tuesday, May 10, 2016, at 6pm (Submit via NYU Classes) Instructions: Your answers to the questions below, including
More informationParametric Inference Maximum Likelihood Inference Exponential Families Expectation Maximization (EM) Bayesian Inference Statistical Decison Theory
Statistical Inference Parametric Inference Maximum Likelihood Inference Exponential Families Expectation Maximization (EM) Bayesian Inference Statistical Decison Theory IP, José Bioucas Dias, IST, 2007
More informationProbabilistic Inference and Applications to Frontier Physics Part 1
Probabilistic Inference and Applications to Frontier Physics Part 1 Giulio D Agostini Dipartimento di Fisica Università di Roma La Sapienza G. D Agostini, Probabilistic Inference (MAPSES - Lecce 23-24/11/2011)
More informationBayesian Regression (1/31/13)
STA613/CBB540: Statistical methods in computational biology Bayesian Regression (1/31/13) Lecturer: Barbara Engelhardt Scribe: Amanda Lea 1 Bayesian Paradigm Bayesian methods ask: given that I have observed
More informationData Analysis and Uncertainty Part 2: Estimation
Data Analysis and Uncertainty Part 2: Estimation Instructor: Sargur N. University at Buffalo The State University of New York srihari@cedar.buffalo.edu 1 Topics in Estimation 1. Estimation 2. Desirable
More informationExpectation Propagation Algorithm
Expectation Propagation Algorithm 1 Shuang Wang School of Electrical and Computer Engineering University of Oklahoma, Tulsa, OK, 74135 Email: {shuangwang}@ou.edu This note contains three parts. First,
More informationProbability and Information Theory. Sargur N. Srihari
Probability and Information Theory Sargur N. srihari@cedar.buffalo.edu 1 Topics in Probability and Information Theory Overview 1. Why Probability? 2. Random Variables 3. Probability Distributions 4. Marginal
More informationBayesian statistics. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Bayesian statistics DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall15 Carlos Fernandez-Granda Frequentist vs Bayesian statistics In frequentist statistics
More informationRobotics. Lecture 4: Probabilistic Robotics. See course website for up to date information.
Robotics Lecture 4: Probabilistic Robotics See course website http://www.doc.ic.ac.uk/~ajd/robotics/ for up to date information. Andrew Davison Department of Computing Imperial College London Review: Sensors
More informationSome slides from Carlos Guestrin, Luke Zettlemoyer & K Gajos 2
Logistics CSE 446: Point Estimation Winter 2012 PS2 out shortly Dan Weld Some slides from Carlos Guestrin, Luke Zettlemoyer & K Gajos 2 Last Time Random variables, distributions Marginal, joint & conditional
More informationDD Advanced Machine Learning
Modelling Carl Henrik {chek}@csc.kth.se Royal Institute of Technology November 4, 2015 Who do I think you are? Mathematically competent linear algebra multivariate calculus Ok programmers Able to extend
More informationCS 361: Probability & Statistics
March 14, 2018 CS 361: Probability & Statistics Inference The prior From Bayes rule, we know that we can express our function of interest as Likelihood Prior Posterior The right hand side contains the
More informationIntroduction to Machine Learning. Lecture 2
Introduction to Machine Learning Lecturer: Eran Halperin Lecture 2 Fall Semester Scribe: Yishay Mansour Some of the material was not presented in class (and is marked with a side line) and is given for
More informationEstimation Theory. as Θ = (Θ 1,Θ 2,...,Θ m ) T. An estimator
Estimation Theory Estimation theory deals with finding numerical values of interesting parameters from given set of data. We start with formulating a family of models that could describe how the data were
More informationLecture: Gaussian Process Regression. STAT 6474 Instructor: Hongxiao Zhu
Lecture: Gaussian Process Regression STAT 6474 Instructor: Hongxiao Zhu Motivation Reference: Marc Deisenroth s tutorial on Robot Learning. 2 Fast Learning for Autonomous Robots with Gaussian Processes
More informationUniversity of Cambridge Engineering Part IIB Module 3F3: Signal and Pattern Processing Handout 2:. The Multivariate Gaussian & Decision Boundaries
University of Cambridge Engineering Part IIB Module 3F3: Signal and Pattern Processing Handout :. The Multivariate Gaussian & Decision Boundaries..15.1.5 1 8 6 6 8 1 Mark Gales mjfg@eng.cam.ac.uk Lent
More informationOverfitting, Bias / Variance Analysis
Overfitting, Bias / Variance Analysis Professor Ameet Talwalkar Professor Ameet Talwalkar CS260 Machine Learning Algorithms February 8, 207 / 40 Outline Administration 2 Review of last lecture 3 Basic
More informationIntroduction to Statistical Methods for High Energy Physics
Introduction to Statistical Methods for High Energy Physics 2011 CERN Summer Student Lectures Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk www.pp.rhul.ac.uk/~cowan
More informationBayesian Learning (II)
Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen Bayesian Learning (II) Niels Landwehr Overview Probabilities, expected values, variance Basic concepts of Bayesian learning MAP
More information1 A simple example. A short introduction to Bayesian statistics, part I Math 217 Probability and Statistics Prof. D.
probabilities, we ll use Bayes formula. We can easily compute the reverse probabilities A short introduction to Bayesian statistics, part I Math 17 Probability and Statistics Prof. D. Joyce, Fall 014 I
More informationStatistical Methods for Astronomy
Statistical Methods for Astronomy Probability (Lecture 1) Statistics (Lecture 2) Why do we need statistics? Useful Statistics Definitions Error Analysis Probability distributions Error Propagation Binomial
More informationBayesian Regression Linear and Logistic Regression
When we want more than point estimates Bayesian Regression Linear and Logistic Regression Nicole Beckage Ordinary Least Squares Regression and Lasso Regression return only point estimates But what if we
More informationVariational Bayesian Logistic Regression
Variational Bayesian Logistic Regression Sargur N. University at Buffalo, State University of New York USA Topics in Linear Models for Classification Overview 1. Discriminant Functions 2. Probabilistic
More informationPMR Learning as Inference
Outline PMR Learning as Inference Probabilistic Modelling and Reasoning Amos Storkey Modelling 2 The Exponential Family 3 Bayesian Sets School of Informatics, University of Edinburgh Amos Storkey PMR Learning
More informationBayesian Inference for Regression Parameters
Bayesian Inference for Regression Parameters 1 Bayesian inference for simple linear regression parameters follows the usual pattern for all Bayesian analyses: 1. Form a prior distribution over all unknown
More informationIntroduction to Bayesian Learning. Machine Learning Fall 2018
Introduction to Bayesian Learning Machine Learning Fall 2018 1 What we have seen so far What does it mean to learn? Mistake-driven learning Learning by counting (and bounding) number of mistakes PAC learnability
More informationSolution: chapter 2, problem 5, part a:
Learning Chap. 4/8/ 5:38 page Solution: chapter, problem 5, part a: Let y be the observed value of a sampling from a normal distribution with mean µ and standard deviation. We ll reserve µ for the estimator
More informationContents. Decision Making under Uncertainty 1. Meanings of uncertainty. Classical interpretation
Contents Decision Making under Uncertainty 1 elearning resources Prof. Ahti Salo Helsinki University of Technology http://www.dm.hut.fi Meanings of uncertainty Interpretations of probability Biases in
More informationProbability is related to uncertainty and not (only) to the results of repeated experiments
Uncertainty probability Probability is related to uncertainty and not (only) to the results of repeated experiments G. D Agostini, Probabilità e incertezze di misura - Parte 1 p. 40 Uncertainty probability
More informationMultiple regression. CM226: Machine Learning for Bioinformatics. Fall Sriram Sankararaman Acknowledgments: Fei Sha, Ameet Talwalkar
Multiple regression CM226: Machine Learning for Bioinformatics. Fall 2016 Sriram Sankararaman Acknowledgments: Fei Sha, Ameet Talwalkar Multiple regression 1 / 36 Previous two lectures Linear and logistic
More informationBayesian Learning. HT2015: SC4 Statistical Data Mining and Machine Learning. Maximum Likelihood Principle. The Bayesian Learning Framework
HT5: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford http://www.stats.ox.ac.uk/~sejdinov/sdmml.html Maximum Likelihood Principle A generative model for
More informationMachine Learning CMPT 726 Simon Fraser University. Binomial Parameter Estimation
Machine Learning CMPT 726 Simon Fraser University Binomial Parameter Estimation Outline Maximum Likelihood Estimation Smoothed Frequencies, Laplace Correction. Bayesian Approach. Conjugate Prior. Uniform
More informationBayesian Machine Learning
Bayesian Machine Learning Andrew Gordon Wilson ORIE 6741 Lecture 2: Bayesian Basics https://people.orie.cornell.edu/andrew/orie6741 Cornell University August 25, 2016 1 / 17 Canonical Machine Learning
More informationLecture 1: Probability Fundamentals
Lecture 1: Probability Fundamentals IB Paper 7: Probability and Statistics Carl Edward Rasmussen Department of Engineering, University of Cambridge January 22nd, 2008 Rasmussen (CUED) Lecture 1: Probability
More informationBayesian Linear Regression [DRAFT - In Progress]
Bayesian Linear Regression [DRAFT - In Progress] David S. Rosenberg Abstract Here we develop some basics of Bayesian linear regression. Most of the calculations for this document come from the basic theory
More informationLecture 18: Learning probabilistic models
Lecture 8: Learning probabilistic models Roger Grosse Overview In the first half of the course, we introduced backpropagation, a technique we used to train neural nets to minimize a variety of cost functions.
More informationGeneralized Bayesian Inference with Sets of Conjugate Priors for Dealing with Prior-Data Conflict
Generalized Bayesian Inference with Sets of Conjugate Priors for Dealing with Prior-Data Conflict Gero Walter Lund University, 15.12.2015 This document is a step-by-step guide on how sets of priors can
More informationInference for a Population Proportion
Al Nosedal. University of Toronto. November 11, 2015 Statistical inference is drawing conclusions about an entire population based on data in a sample drawn from that population. From both frequentist
More informationReview (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology
Review (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Some slides have been adopted from Prof. H.R. Rabiee s and also Prof. R. Gutierrez-Osuna
More informationMachine Learning using Bayesian Approaches
Machine Learning using Bayesian Approaches Sargur N. Srihari University at Buffalo, State University of New York 1 Outline 1. Progress in ML and PR 2. Fully Bayesian Approach 1. Probability theory Bayes
More informationIntroduction: MLE, MAP, Bayesian reasoning (28/8/13)
STA561: Probabilistic machine learning Introduction: MLE, MAP, Bayesian reasoning (28/8/13) Lecturer: Barbara Engelhardt Scribes: K. Ulrich, J. Subramanian, N. Raval, J. O Hollaren 1 Classifiers In this
More informationPhysics 403. Segev BenZvi. Choosing Priors and the Principle of Maximum Entropy. Department of Physics and Astronomy University of Rochester
Physics 403 Choosing Priors and the Principle of Maximum Entropy Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Review of Last Class Odds Ratio Occam Factors
More informationBayesian Inference. 2 CS295-7 cfl Michael J. Black,
Population Coding Now listen to me closely, young gentlemen. That brain is thinking. Maybe it s thinking about music. Maybe it has a great symphony all thought out or a mathematical formula that would
More informationNPFL108 Bayesian inference. Introduction. Filip Jurčíček. Institute of Formal and Applied Linguistics Charles University in Prague Czech Republic
NPFL108 Bayesian inference Introduction Filip Jurčíček Institute of Formal and Applied Linguistics Charles University in Prague Czech Republic Home page: http://ufal.mff.cuni.cz/~jurcicek Version: 21/02/2014
More informationLecture 4: Probabilistic Learning
DD2431 Autumn, 2015 1 Maximum Likelihood Methods Maximum A Posteriori Methods Bayesian methods 2 Classification vs Clustering Heuristic Example: K-means Expectation Maximization 3 Maximum Likelihood Methods
More informationECE295, Data Assimila0on and Inverse Problems, Spring 2015
ECE295, Data Assimila0on and Inverse Problems, Spring 2015 1 April, Intro; Linear discrete Inverse problems (Aster Ch 1 and 2) Slides 8 April, SVD (Aster ch 2 and 3) Slides 15 April, RegularizaFon (ch
More informationFourier and Stats / Astro Stats and Measurement : Stats Notes
Fourier and Stats / Astro Stats and Measurement : Stats Notes Andy Lawrence, University of Edinburgh Autumn 2013 1 Probabilities, distributions, and errors Laplace once said Probability theory is nothing
More informationPrimer on statistics:
Primer on statistics: MLE, Confidence Intervals, and Hypothesis Testing ryan.reece@gmail.com http://rreece.github.io/ Insight Data Science - AI Fellows Workshop Feb 16, 018 Outline 1. Maximum likelihood
More informationTime Series and Dynamic Models
Time Series and Dynamic Models Section 1 Intro to Bayesian Inference Carlos M. Carvalho The University of Texas at Austin 1 Outline 1 1. Foundations of Bayesian Statistics 2. Bayesian Estimation 3. The
More informationStatistical Methods in Particle Physics
Statistical Methods in Particle Physics Lecture 11 January 7, 2013 Silvia Masciocchi, GSI Darmstadt s.masciocchi@gsi.de Winter Semester 2012 / 13 Outline How to communicate the statistical uncertainty
More informationStat 451 Lecture Notes Numerical Integration
Stat 451 Lecture Notes 03 12 Numerical Integration Ryan Martin UIC www.math.uic.edu/~rgmartin 1 Based on Chapter 5 in Givens & Hoeting, and Chapters 4 & 18 of Lange 2 Updated: February 11, 2016 1 / 29
More informationGrundlagen der Künstlichen Intelligenz
Grundlagen der Künstlichen Intelligenz Uncertainty & Probabilities & Bandits Daniel Hennes 16.11.2017 (WS 2017/18) University Stuttgart - IPVS - Machine Learning & Robotics 1 Today Uncertainty Probability
More informationIntroduction to Machine Learning
Introduction to Machine Learning Introduction to Probabilistic Methods Varun Chandola Computer Science & Engineering State University of New York at Buffalo Buffalo, NY, USA chandola@buffalo.edu Chandola@UB
More informationStatistical Methods in Particle Physics. Lecture 2
Statistical Methods in Particle Physics Lecture 2 October 17, 2011 Silvia Masciocchi, GSI Darmstadt s.masciocchi@gsi.de Winter Semester 2011 / 12 Outline Probability Definition and interpretation Kolmogorov's
More informationStatistics and Data Analysis
Statistics and Data Analysis The Crash Course Physics 226, Fall 2013 "There are three kinds of lies: lies, damned lies, and statistics. Mark Twain, allegedly after Benjamin Disraeli Statistics and Data
More informationBayesian Inference. Chris Mathys Wellcome Trust Centre for Neuroimaging UCL. London SPM Course
Bayesian Inference Chris Mathys Wellcome Trust Centre for Neuroimaging UCL London SPM Course Thanks to Jean Daunizeau and Jérémie Mattout for previous versions of this talk A spectacular piece of information
More informationLearning Bayesian belief networks
Lecture 4 Learning Bayesian belief networks Milos Hauskrecht milos@cs.pitt.edu 5329 Sennott Square Administration Midterm: Monday, March 7, 2003 In class Closed book Material covered by Wednesday, March
More informationHuman-Oriented Robotics. Probability Refresher. Kai Arras Social Robotics Lab, University of Freiburg Winter term 2014/2015
Probability Refresher Kai Arras, University of Freiburg Winter term 2014/2015 Probability Refresher Introduction to Probability Random variables Joint distribution Marginalization Conditional probability
More informationDiscrete Binary Distributions
Discrete Binary Distributions Carl Edward Rasmussen November th, 26 Carl Edward Rasmussen Discrete Binary Distributions November th, 26 / 5 Key concepts Bernoulli: probabilities over binary variables Binomial:
More informationLecture 13: Data Modelling and Distributions. Intelligent Data Analysis and Probabilistic Inference Lecture 13 Slide No 1
Lecture 13: Data Modelling and Distributions Intelligent Data Analysis and Probabilistic Inference Lecture 13 Slide No 1 Why data distributions? It is a well established fact that many naturally occurring
More informationUniversity of Cambridge Engineering Part IIB Module 4F10: Statistical Pattern Processing Handout 2: Multivariate Gaussians
Engineering Part IIB: Module F Statistical Pattern Processing University of Cambridge Engineering Part IIB Module F: Statistical Pattern Processing Handout : Multivariate Gaussians. Generative Model Decision
More information