Treatment and analysis of data Applied statistics Lecture 5: More on Maximum Likelihood
|
|
|
- Cornelia McLaughlin
- 6 years ago
- Views:
Transcription
1 Treatment and analysis of data Applied statistics Lecture 5: More on Maximum Likelihood Topics covered: The multivariate Gaussian Ordinary Least Squares (OLS) Generalized Least Squares (GLS) Linear and Non-Linear Least Squares The approximate covariance of the MLE Two examples of MLE Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 1 The multivariate Gaussian (1) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 2
2 The multivariate Gaussian (2) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 3 The multivariate Gaussian (3) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 4
3 The multivariate Gaussian (4) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 5 The multivariate Gaussian (5) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 6
4 The multivariate Gaussian (6) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 7 The multivariate Gaussian (7) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 8
5 Hierarchy of standard estimation methods (incomplete and not very strict!) no prior normality homoscedasticity linearity single parameter Bayesian estimation Maximum likelihood estimation Generalized least-squares method Ordinary least-squares method Linear regression Sample mean Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 9 Ordinary Least Squares - OLS (1) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 10
6 Ordinary Least Squares (2) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 11 Ordinary Least Squares (3) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 12
7 Ordinary Least Squares (4) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 13 Ordinary Least Squares (5) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 14
8 Ordinary Least Squares (Summary) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 15 Generalized Least Squares - GLS (1) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 16
9 Generalized Least Squares (2) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 17 Generalized Least Squares (3) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 18
10 Generalized Least Squares (Summary) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 19 Generalized Least Squares - An important special case (1) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 20
11 Generalized Least Squares - An important special case (2) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 21 Linear and Non-Linear Least Squares Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 22
12 Non-Linear Least Squares Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 23 Remarks on general minimization routines Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 24
13 Non-Linear Least Squares: Solution by linearization Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 25 OLS Regression of Y versus X (or X versus Y?) Suppose the random variables x, y are expected to have a linear relationship, y = ax + b. If a 0, a mathematically equivalent formulation of the relationship is x = cy + d (where c = 1/a, d = b/a). Given a set of data pairs (x i, y i ), i = 1...n, we may determine the coefficients of the linear model by OLS regression in two different ways: 1. OLS(Y X): ax i + b ~ y i estimates a', b' 2. OLS(X Y): cy i + d ~ x i estimates c', d', a" = 1/c', b" = d'/c' The two results are usually different. In particular, a'c' = r 2 1 (r = sample corr. coeff.). If there is a large scatter among the points ( r is small), then the results are very different. So which should be used (if any)? Good discussion in: Isobe, Feigelson, Akritas & Babu, Linear Regression in Astronomy, I (ApJ 364, 104, 1990) Feigelson & Babu, Linear Regression in Astronomy, II (ApJ 397, 55, 1992). Think about: Is the regression used to predict x or y? Or to find a structural relationship? Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 26
14 Regression of Y versus X (or X versus Y?) Isobe et al., Linear Regression in Astronomy, I ApJ 364, 104 (1990) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 27 Covariance of the MLE Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 28
15 The Cramér-Rao inequality Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 29 Points to note about the CR bound Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 30
16 MLE: Linear regression with errors in both X and Y (1) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 31 MLE: Linear regression with errors in both X and Y (2) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 32
17 MLE: Linear regression with errors in both X and Y (3) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 33 MLE: Linear regression with errors in both X and Y (4) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 34
18 MLE: Linear regression with errors in both X and Y (5) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 35 MLE: Locating a feature in a Poisson process (1) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 36
19 MLE: Locating a feature in a Poisson process (2) τ area = α β (known) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 37 MLE: Locating a feature in a Poisson process (3) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 38
20 MLE: Locating a feature in a Poisson process (4) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 39 MLE: Locating a feature in a Poisson process (5) α f (t τ ) + β f f '( t) ( t) + β /α Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 40
21 MLE: Locating a feature in a Poisson process (6) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 41 MLE: Locating a feature in a Poisson process (7) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 42
22 MLE: Locating a feature in a Poisson process (8) Sept-Oct 2006 Statistics for astronomers (L. Lindegren, Lund Observatory) Lecture 5, p. 43
Treatment and analysis of data Applied statistics Lecture 4: Estimation
Treatment and analysis of data Applied statistics Lecture 4: Estimation Topics covered: Hierarchy of estimation methods Modelling of data The likelihood function The Maximum Likelihood Estimate (MLE) Confidence
Linear regression issues in astronomy. G. Jogesh Babu Center for Astrostatistics
Linear regression issues in astronomy G. Jogesh Babu Center for Astrostatistics Structural regression Seeking the intrinsic relationship between two properties OLS(X Y) (inverse regr) Four symmetrical
Treatment and analysis of data Applied statistics Lecture 6: Bayesian estimation
Treatment and analysis o data Applied statistics Lecture 6: Bayesian estimation Topics covered: Bayes' Theorem again Relation to Likelihood Transormation o pd A trivial example Wiener ilter Malmquist bias
Measurement Error and Linear Regression of Astronomical Data. Brandon Kelly Penn State Summer School in Astrostatistics, June 2007
Measurement Error and Linear Regression of Astronomical Data Brandon Kelly Penn State Summer School in Astrostatistics, June 2007 Classical Regression Model Collect n data points, denote i th pair as (η
Variations. ECE 6540, Lecture 10 Maximum Likelihood Estimation
Variations ECE 6540, Lecture 10 Last Time BLUE (Best Linear Unbiased Estimator) Formulation Advantages Disadvantages 2 The BLUE A simplification Assume the estimator is a linear system For a single parameter
Statistics in Astronomy (II) applications. Shen, Shiyin
Statistics in Astronomy (II) applications Shen, Shiyin Contents I. Linear fitting: scaling relations I.5 correlations between parameters II. Luminosity distribution function Vmax VS maximum likelihood
The 1905 Einstein equation in a general mathematical analysis model of Quasars
DePaul University From the SelectedWorks of Byron E. Bell Spring May 3, 2010 The 1905 Einstein equation in a general mathematical analysis model of Quasars Byron E. Bell Available at: https://works.bepress.com/byron_bell/2/
Copula Regression RAHUL A. PARSA DRAKE UNIVERSITY & STUART A. KLUGMAN SOCIETY OF ACTUARIES CASUALTY ACTUARIAL SOCIETY MAY 18,2011
Copula Regression RAHUL A. PARSA DRAKE UNIVERSITY & STUART A. KLUGMAN SOCIETY OF ACTUARIES CASUALTY ACTUARIAL SOCIETY MAY 18,2011 Outline Ordinary Least Squares (OLS) Regression Generalized Linear Models
Multivariate Bayesian Linear Regression MLAI Lecture 11
Multivariate Bayesian Linear Regression MLAI Lecture 11 Neil D. Lawrence Department of Computer Science Sheffield University 21st October 2012 Outline Univariate Bayesian Linear Regression Multivariate
Lecture 3. G. Cowan. Lecture 3 page 1. Lectures on Statistical Data Analysis
Lecture 3 1 Probability (90 min.) Definition, Bayes theorem, probability densities and their properties, catalogue of pdfs, Monte Carlo 2 Statistical tests (90 min.) general concepts, test statistics,
Machine Learning Basics: Maximum Likelihood Estimation
Machine Learning Basics: Maximum Likelihood Estimation Sargur N. srihari@cedar.buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 Topics 1. Learning
Problem Selected Scores
Statistics Ph.D. Qualifying Exam: Part II November 20, 2010 Student Name: 1. Answer 8 out of 12 problems. Mark the problems you selected in the following table. Problem 1 2 3 4 5 6 7 8 9 10 11 12 Selected
Summer School in Statistics for Astronomers V June 1 - June 6, Regression. Mosuk Chow Statistics Department Penn State University.
Summer School in Statistics for Astronomers V June 1 - June 6, 2009 Regression Mosuk Chow Statistics Department Penn State University. Adapted from notes prepared by RL Karandikar Mean and variance Recall
The Challenges of Heteroscedastic Measurement Error in Astronomical Data
The Challenges of Heteroscedastic Measurement Error in Astronomical Data Eric Feigelson Penn State Measurement Error Workshop TAMU April 2016 Outline 1. Measurement errors are ubiquitous in astronomy,
Fall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.
1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n
Machine Learning for Data Science (CS4786) Lecture 12
Machine Learning for Data Science (CS4786) Lecture 12 Gaussian Mixture Models Course Webpage : http://www.cs.cornell.edu/courses/cs4786/2016fa/ Back to K-means Single link is sensitive to outliners We
Asymptotic standard errors of MLE
Asymptotic standard errors of MLE Suppose, in the previous example of Carbon and Nitrogen in soil data, that we get the parameter estimates For maximum likelihood estimation, we can use Hessian matrix
Bivariate Least Squares Linear Regression: Towards a Unified Analytic Formalism. II. Extreme Structural Models
Applied Mathematical Sciences, Vol. 11, 2017, no. 49, 2393-2446 HIKARI Ltd, www.m-hikari.com https://doi.org/10.12988/ams.2017.78260 Bivariate Least Squares Linear Regression: Towards a Unified Analytic
Ma 3/103: Lecture 24 Linear Regression I: Estimation
Ma 3/103: Lecture 24 Linear Regression I: Estimation March 3, 2017 KC Border Linear Regression I March 3, 2017 1 / 32 Regression analysis Regression analysis Estimate and test E(Y X) = f (X). f is the
Mathematical statistics
October 4 th, 2018 Lecture 12: Information Where are we? Week 1 Week 2 Week 4 Week 7 Week 10 Week 14 Probability reviews Chapter 6: Statistics and Sampling Distributions Chapter 7: Point Estimation Chapter
Censoring and truncation in astronomical surveys. Eric Feigelson (Astronomy & Astrophysics, Penn State)
Censoring and truncation in astronomical surveys Eric Feigelson (Astronomy & Astrophysics, Penn State) Outline 1. Examples of astronomical surveys 2. One survey: Truncation and spatial/luminosity distributions
Multivariate Regression: Part I
Topic 1 Multivariate Regression: Part I ARE/ECN 240 A Graduate Econometrics Professor: Òscar Jordà Outline of this topic Statement of the objective: we want to explain the behavior of one variable as a
Microeconometrics: Clustering. Ethan Kaplan
Microeconometrics: Clustering Ethan Kaplan Gauss Markov ssumptions OLS is minimum variance unbiased (MVUE) if Linear Model: Y i = X i + i E ( i jx i ) = V ( i jx i ) = 2 < cov i ; j = Normally distributed
Statement: With my signature I confirm that the solutions are the product of my own work. Name: Signature:.
MATHEMATICAL STATISTICS Homework assignment Instructions Please turn in the homework with this cover page. You do not need to edit the solutions. Just make sure the handwriting is legible. You may discuss
Statistics Ph.D. Qualifying Exam: Part II November 3, 2001
Statistics Ph.D. Qualifying Exam: Part II November 3, 2001 Student Name: 1. Answer 8 out of 12 problems. Mark the problems you selected in the following table. 1 2 3 4 5 6 7 8 9 10 11 12 2. Write your
PART I INTRODUCTION The meaning of probability Basic definitions for frequentist statistics and Bayesian inference Bayesian inference Combinatorics
Table of Preface page xi PART I INTRODUCTION 1 1 The meaning of probability 3 1.1 Classical definition of probability 3 1.2 Statistical definition of probability 9 1.3 Bayesian understanding of probability
Statistics & Data Sciences: First Year Prelim Exam May 2018
Statistics & Data Sciences: First Year Prelim Exam May 2018 Instructions: 1. Do not turn this page until instructed to do so. 2. Start each new question on a new sheet of paper. 3. This is a closed book
Time Series Analysis. James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY
Time Series Analysis James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY & Contents PREFACE xiii 1 1.1. 1.2. Difference Equations First-Order Difference Equations 1 /?th-order Difference
* Tuesday 17 January :30-16:30 (2 hours) Recored on ESSE3 General introduction to the course.
Name of the course Statistical methods and data analysis Audience The course is intended for students of the first or second year of the Graduate School in Materials Engineering. The aim of the course
arxiv:astro-ph/ v1 1 May 1996
Linear Regression for Astronomical Data with Measurement Errors and Intrinsic Scatter arxiv:astro-ph/9605002v1 1 May 1996 Michael G. Akritas Department of Statistics, The Pennsylvania State University,
Discriminant Analysis and Statistical Pattern Recognition
Discriminant Analysis and Statistical Pattern Recognition GEOFFREY J. McLACHLAN Department of Mathematics The University of Queensland St. Lucia, Queensland, Australia A Wiley-Interscience Publication
TABLE OF CONTENTS CHAPTER 1 COMBINATORIAL PROBABILITY 1
TABLE OF CONTENTS CHAPTER 1 COMBINATORIAL PROBABILITY 1 1.1 The Probability Model...1 1.2 Finite Discrete Models with Equally Likely Outcomes...5 1.2.1 Tree Diagrams...6 1.2.2 The Multiplication Principle...8
Economics 620, Lecture 9: Asymptotics III: Maximum Likelihood Estimation
Economics 620, Lecture 9: Asymptotics III: Maximum Likelihood Estimation Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 9: Asymptotics III(MLE) 1 / 20 Jensen
Regression. Machine Learning and Pattern Recognition. Chris Williams. School of Informatics, University of Edinburgh.
Regression Machine Learning and Pattern Recognition Chris Williams School of Informatics, University of Edinburgh September 24 (All of the slides in this course have been adapted from previous versions
Parameter estimation and forecasting. Cristiano Porciani AIfA, Uni-Bonn
Parameter estimation and forecasting Cristiano Porciani AIfA, Uni-Bonn Questions? C. Porciani Estimation & forecasting 2 Temperature fluctuations Variance at multipole l (angle ~180o/l) C. Porciani Estimation
An Introduction to Bayesian Linear Regression
An Introduction to Bayesian Linear Regression APPM 5720: Bayesian Computation Fall 2018 A SIMPLE LINEAR MODEL Suppose that we observe explanatory variables x 1, x 2,..., x n and dependent variables y 1,
Bayesian Regression Linear and Logistic Regression
When we want more than point estimates Bayesian Regression Linear and Logistic Regression Nicole Beckage Ordinary Least Squares Regression and Lasso Regression return only point estimates But what if we
Ridge regression. Patrick Breheny. February 8. Penalized regression Ridge regression Bayesian interpretation
Patrick Breheny February 8 Patrick Breheny High-Dimensional Data Analysis (BIOS 7600) 1/27 Introduction Basic idea Standardization Large-scale testing is, of course, a big area and we could keep talking
HANDBOOK OF APPLICABLE MATHEMATICS
HANDBOOK OF APPLICABLE MATHEMATICS Chief Editor: Walter Ledermann Volume VI: Statistics PART A Edited by Emlyn Lloyd University of Lancaster A Wiley-Interscience Publication JOHN WILEY & SONS Chichester
Statistícal Methods for Spatial Data Analysis
Texts in Statistícal Science Statistícal Methods for Spatial Data Analysis V- Oliver Schabenberger Carol A. Gotway PCT CHAPMAN & K Contents Preface xv 1 Introduction 1 1.1 The Need for Spatial Analysis
STAT 730 Chapter 4: Estimation
STAT 730 Chapter 4: Estimation Timothy Hanson Department of Statistics, University of South Carolina Stat 730: Multivariate Analysis 1 / 23 The likelihood We have iid data, at least initially. Each datum
Logistic Regression Review Fall 2012 Recitation. September 25, 2012 TA: Selen Uguroglu
Logistic Regression Review 10-601 Fall 2012 Recitation September 25, 2012 TA: Selen Uguroglu!1 Outline Decision Theory Logistic regression Goal Loss function Inference Gradient Descent!2 Training Data
Parametric Models. Dr. Shuang LIANG. School of Software Engineering TongJi University Fall, 2012
Parametric Models Dr. Shuang LIANG School of Software Engineering TongJi University Fall, 2012 Today s Topics Maximum Likelihood Estimation Bayesian Density Estimation Today s Topics Maximum Likelihood
General Regression Model
Scott S. Emerson, M.D., Ph.D. Department of Biostatistics, University of Washington, Seattle, WA 98195, USA January 5, 2015 Abstract Regression analysis can be viewed as an extension of two sample statistical
1. The Multivariate Classical Linear Regression Model
Business School, Brunel University MSc. EC550/5509 Modelling Financial Decisions and Markets/Introduction to Quantitative Methods Prof. Menelaos Karanasos (Room SS69, Tel. 08956584) Lecture Notes 5. The
Chapter 3: Maximum-Likelihood & Bayesian Parameter Estimation (part 1)
HW 1 due today Parameter Estimation Biometrics CSE 190 Lecture 7 Today s lecture was on the blackboard. These slides are an alternative presentation of the material. CSE190, Winter10 CSE190, Winter10 Chapter
Stat 5101 Lecture Notes
Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random
OPTIMAL ESTIMATION of DYNAMIC SYSTEMS
CHAPMAN & HALL/CRC APPLIED MATHEMATICS -. AND NONLINEAR SCIENCE SERIES OPTIMAL ESTIMATION of DYNAMIC SYSTEMS John L Crassidis and John L. Junkins CHAPMAN & HALL/CRC A CRC Press Company Boca Raton London
Linear Regression. Junhui Qian. October 27, 2014
Linear Regression Junhui Qian October 27, 2014 Outline The Model Estimation Ordinary Least Square Method of Moments Maximum Likelihood Estimation Properties of OLS Estimator Unbiasedness Consistency Efficiency
Time Series Analysis. James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY
Time Series Analysis James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY PREFACE xiii 1 Difference Equations 1.1. First-Order Difference Equations 1 1.2. pth-order Difference Equations 7
Econometric Analysis of Cross Section and Panel Data
Econometric Analysis of Cross Section and Panel Data Jeffrey M. Wooldridge / The MIT Press Cambridge, Massachusetts London, England Contents Preface Acknowledgments xvii xxiii I INTRODUCTION AND BACKGROUND
Naive Bayes and Gaussian Bayes Classifier
Naive Bayes and Gaussian Bayes Classifier Elias Tragas tragas@cs.toronto.edu October 3, 2016 Elias Tragas Naive Bayes and Gaussian Bayes Classifier October 3, 2016 1 / 23 Naive Bayes Bayes Rules: Naive
The data can be downloaded as an Excel file under Econ2130 at
1 HG Revised Sept. 018 Supplement to lecture 9 (Tuesda 18 Sept) On the bivariate normal model Eample: daughter s height (Y) vs. mother s height (). Data collected on Econ 130 lectures 010-01. The data
Linear Regression (1/1/17)
STA613/CBB540: Statistical methods in computational biology Linear Regression (1/1/17) Lecturer: Barbara Engelhardt Scribe: Ethan Hada 1. Linear regression 1.1. Linear regression basics. Linear regression
Advanced Introduction to Machine Learning CMU-10715
Advanced Introduction to Machine Learning CMU-10715 Gaussian Processes Barnabás Póczos http://www.gaussianprocess.org/ 2 Some of these slides in the intro are taken from D. Lizotte, R. Parr, C. Guesterin
Suggested Solution for PS #5
Cornell University Department of Economics Econ 62 Spring 28 TA: Jae Ho Yun Suggested Solution for S #5. (Measurement Error, IV) (a) This is a measurement error problem. y i x i + t i + " i t i t i + i
Chapter 8.8.1: A factorization theorem
LECTURE 14 Chapter 8.8.1: A factorization theorem The characterization of a sufficient statistic in terms of the conditional distribution of the data given the statistic can be difficult to work with.
Primer on statistics:
Primer on statistics: MLE, Confidence Intervals, and Hypothesis Testing ryan.reece@gmail.com http://rreece.github.io/ Insight Data Science - AI Fellows Workshop Feb 16, 018 Outline 1. Maximum likelihood
Lecture 32: Asymptotic confidence sets and likelihoods
Lecture 32: Asymptotic confidence sets and likelihoods Asymptotic criterion In some problems, especially in nonparametric problems, it is difficult to find a reasonable confidence set with a given confidence
(a) (3 points) Construct a 95% confidence interval for β 2 in Equation 1.
Problem 1 (21 points) An economist runs the regression y i = β 0 + x 1i β 1 + x 2i β 2 + x 3i β 3 + ε i (1) The results are summarized in the following table: Equation 1. Variable Coefficient Std. Error
Data modelling Parameter estimation
ASTR509-10 Data modelling Parameter estimation Pierre-Simon, Marquis de Laplace 1749-1827 Under Napoleon, Laplace was a member, then chancellor, of the Senate, and received the Legion of Honour in 1805.
Introduction to Bayesian Inference
University of Pennsylvania EABCN Training School May 10, 2016 Bayesian Inference Ingredients of Bayesian Analysis: Likelihood function p(y φ) Prior density p(φ) Marginal data density p(y ) = p(y φ)p(φ)dφ
Overview. DS GA 1002 Probability and Statistics for Data Science. Carlos Fernandez-Granda
Overview DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Probability and statistics Probability: Framework for dealing
GAUSSIAN PROCESS REGRESSION
GAUSSIAN PROCESS REGRESSION CSE 515T Spring 2015 1. BACKGROUND The kernel trick again... The Kernel Trick Consider again the linear regression model: y(x) = φ(x) w + ε, with prior p(w) = N (w; 0, Σ). The
From Practical Data Analysis with JMP, Second Edition. Full book available for purchase here. About This Book... xiii About The Author...
From Practical Data Analysis with JMP, Second Edition. Full book available for purchase here. Contents About This Book... xiii About The Author... xxiii Chapter 1 Getting Started: Data Analysis with JMP...
1. Non-Uniformly Weighted Data [7pts]
![1. Non-Uniformly Weighted Data [7pts]](/thumbs/89/99992891.jpg "1. Non-Uniformly Weighted Data [7pts]")
Homework 1: Linear Regression Writeup due 23:59 on Friday 6 February 2015 You will do this assignment individually and submit your answers as a PDF via the Canvas course website. There is a mathematical
Linear Regression (9/11/13)
STA561: Probabilistic machine learning Linear Regression (9/11/13) Lecturer: Barbara Engelhardt Scribes: Zachary Abzug, Mike Gloudemans, Zhuosheng Gu, Zhao Song 1 Why use linear regression? Figure 1: Scatter
AGEC 621 Lecture 16 David Bessler
AGEC 621 Lecture 16 David Bessler This is a RATS output for the dummy variable problem given in GHJ page 422; the beer expenditure lecture (last time). I do not expect you to know RATS but this will give
Naive Bayes and Gaussian Bayes Classifier
Naive Bayes and Gaussian Bayes Classifier Mengye Ren mren@cs.toronto.edu October 18, 2015 Mengye Ren Naive Bayes and Gaussian Bayes Classifier October 18, 2015 1 / 21 Naive Bayes Bayes Rules: Naive Bayes
Machine Learning (CS 567) Lecture 5
Machine Learning (CS 567) Lecture 5 Time: T-Th 5:00pm - 6:20pm Location: GFS 118 Instructor: Sofus A. Macskassy (macskass@usc.edu) Office: SAL 216 Office hours: by appointment Teaching assistant: Cheol
Introduction to Bayesian Learning. Machine Learning Fall 2018
Introduction to Bayesian Learning Machine Learning Fall 2018 1 What we have seen so far What does it mean to learn? Mistake-driven learning Learning by counting (and bounding) number of mistakes PAC learnability
Naive Bayes and Gaussian Bayes Classifier
Naive Bayes and Gaussian Bayes Classifier Ladislav Rampasek slides by Mengye Ren and others February 22, 2016 Naive Bayes and Gaussian Bayes Classifier February 22, 2016 1 / 21 Naive Bayes Bayes Rule:
ACTEX CAS EXAM 3 STUDY GUIDE FOR MATHEMATICAL STATISTICS
ACTEX CAS EXAM 3 STUDY GUIDE FOR MATHEMATICAL STATISTICS TABLE OF CONTENTS INTRODUCTORY NOTE NOTES AND PROBLEM SETS Section 1 - Point Estimation 1 Problem Set 1 15 Section 2 - Confidence Intervals and
Statement: With my signature I confirm that the solutions are the product of my own work. Name: Signature:.
MATHEMATICAL STATISTICS Take-home final examination February 1 st -February 8 th, 019 Instructions You do not need to edit the solutions Just make sure the handwriting is legible The final solutions should
Chapter 14 Stein-Rule Estimation
Chapter 14 Stein-Rule Estimation The ordinary least squares estimation of regression coefficients in linear regression model provides the estimators having minimum variance in the class of linear and unbiased
The General Linear Model (GLM)
he General Linear Model (GLM) Klaas Enno Stephan ranslational Neuromodeling Unit (NU) Institute for Biomedical Engineering University of Zurich & EH Zurich Wellcome rust Centre for Neuroimaging Institute
Detection & Estimation Lecture 1
Detection & Estimation Lecture 1 Intro, MVUE, CRLB Xiliang Luo General Course Information Textbooks & References Fundamentals of Statistical Signal Processing: Estimation Theory/Detection Theory, Steven
Time Series: Theory and Methods
Peter J. Brockwell Richard A. Davis Time Series: Theory and Methods Second Edition With 124 Illustrations Springer Contents Preface to the Second Edition Preface to the First Edition vn ix CHAPTER 1 Stationary
Heteroskedasticity. Part VII. Heteroskedasticity
Part VII Heteroskedasticity As of Oct 15, 2015 1 Heteroskedasticity Consequences Heteroskedasticity-robust inference Testing for Heteroskedasticity Weighted Least Squares (WLS) Feasible generalized Least
Machine Learning CSE546 Sham Kakade University of Washington. Oct 4, What about continuous variables?
Linear Regression Machine Learning CSE546 Sham Kakade University of Washington Oct 4, 2016 1 What about continuous variables? Billionaire says: If I am measuring a continuous variable, what can you do
Statistical Methods in HYDROLOGY CHARLES T. HAAN. The Iowa State University Press / Ames
Statistical Methods in HYDROLOGY CHARLES T. HAAN The Iowa State University Press / Ames Univariate BASIC Table of Contents PREFACE xiii ACKNOWLEDGEMENTS xv 1 INTRODUCTION 1 2 PROBABILITY AND PROBABILITY
Introduction to Bayesian Methods
Introduction to Bayesian Methods Jessi Cisewski Department of Statistics Yale University Sagan Summer Workshop 2016 Our goal: introduction to Bayesian methods Likelihoods Priors: conjugate priors, non-informative
Multivariate Distributions
Copyright Cosma Rohilla Shalizi; do not distribute without permission updates at http://www.stat.cmu.edu/~cshalizi/adafaepov/ Appendix E Multivariate Distributions E.1 Review of Definitions Let s review
The Bayes classifier
The Bayes classifier Consider where is a random vector in is a random variable (depending on ) Let be a classifier with probability of error/risk given by The Bayes classifier (denoted ) is the optimal
Week 3: Linear Regression
Week 3: Linear Regression Instructor: Sergey Levine Recap In the previous lecture we saw how linear regression can solve the following problem: given a dataset D = {(x, y ),..., (x N, y N )}, learn to
Irr. Statistical Methods in Experimental Physics. 2nd Edition. Frederick James. World Scientific. CERN, Switzerland
Frederick James CERN, Switzerland Statistical Methods in Experimental Physics 2nd Edition r i Irr 1- r ri Ibn World Scientific NEW JERSEY LONDON SINGAPORE BEIJING SHANGHAI HONG KONG TAIPEI CHENNAI CONTENTS
Large Sample Properties of Estimators in the Classical Linear Regression Model
Large Sample Properties of Estimators in the Classical Linear Regression Model 7 October 004 A. Statement of the classical linear regression model The classical linear regression model can be written in
ORDER RESTRICTED STATISTICAL INFERENCE ON LORENZ CURVES OF PARETO DISTRIBUTIONS. Myongsik Oh. 1. Introduction
J. Appl. Math & Computing Vol. 13(2003), No. 1-2, pp. 457-470 ORDER RESTRICTED STATISTICAL INFERENCE ON LORENZ CURVES OF PARETO DISTRIBUTIONS Myongsik Oh Abstract. The comparison of two or more Lorenz
Part IB Statistics. Theorems with proof. Based on lectures by D. Spiegelhalter Notes taken by Dexter Chua. Lent 2015
Part IB Statistics Theorems with proof Based on lectures by D. Spiegelhalter Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly)
Lecture 2: Linear Models. Bruce Walsh lecture notes Seattle SISG -Mixed Model Course version 23 June 2011
Lecture 2: Linear Models Bruce Walsh lecture notes Seattle SISG -Mixed Model Course version 23 June 2011 1 Quick Review of the Major Points The general linear model can be written as y = X! + e y = vector
STA414/2104 Statistical Methods for Machine Learning II
STA414/2104 Statistical Methods for Machine Learning II Murat A. Erdogdu & David Duvenaud Department of Computer Science Department of Statistical Sciences Lecture 3 Slide credits: Russ Salakhutdinov Announcements
Parametric Unsupervised Learning Expectation Maximization (EM) Lecture 20.a
Parametric Unsupervised Learning Expectation Maximization (EM) Lecture 20.a Some slides are due to Christopher Bishop Limitations of K-means Hard assignments of data points to clusters small shift of a
Gibbs Sampling in Linear Models #2
Gibbs Sampling in Linear Models #2 Econ 690 Purdue University Outline 1 Linear Regression Model with a Changepoint Example with Temperature Data 2 The Seemingly Unrelated Regressions Model 3 Gibbs sampling
1 Glivenko-Cantelli type theorems
STA79 Lecture Spring Semester Glivenko-Cantelli type theorems Given i.i.d. observations X,..., X n with unknown distribution function F (t, consider the empirical (sample CDF ˆF n (t = I [Xi t]. n Then
Discriminant analysis and supervised classification
Discriminant analysis and supervised classification Angela Montanari 1 Linear discriminant analysis Linear discriminant analysis (LDA) also known as Fisher s linear discriminant analysis or as Canonical
Bayesian Linear Regression [DRAFT - In Progress]
![Bayesian Linear Regression [DRAFT - In Progress]](/thumbs/92/109828713.jpg "Bayesian Linear Regression [DRAFT - In Progress]")
Bayesian Linear Regression [DRAFT - In Progress] David S. Rosenberg Abstract Here we develop some basics of Bayesian linear regression. Most of the calculations for this document come from the basic theory
Economics 573 Problem Set 5 Fall 2002 Due: 4 October b. The sample mean converges in probability to the population mean.
Economics 573 Problem Set 5 Fall 00 Due: 4 October 00 1. In random sampling from any population with E(X) = and Var(X) =, show (using Chebyshev's inequality) that sample mean converges in probability to..
Experimental Design and Data Analysis for Biologists
Experimental Design and Data Analysis for Biologists Gerry P. Quinn Monash University Michael J. Keough University of Melbourne CAMBRIDGE UNIVERSITY PRESS Contents Preface page xv I I Introduction 1 1.1
Statistical Applications in the Astronomy Literature II Jogesh Babu. Center for Astrostatistics PennState University, USA
Statistical Applications in the Astronomy Literature II Jogesh Babu Center for Astrostatistics PennState University, USA 1 The likelihood ratio test (LRT) and the related F-test Protassov et al. (2002,