Stat 5101 Lecture Notes

Save this PDF as:

Size: px
Start display at page:

Transcription

1 Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001

2 ii Stat 5101 (Geyer) Course Notes

3 Contents 1 Random Variables and Change of Variables Random Variables Variables Functions Random Variables: Informal Intuition Random Variables: Formal Definition Functions of Random Variables Change of Variables General Definition Discrete Random Variables Continuous Random Variables Random Vectors Discrete Random Vectors Continuous Random Vectors The Support of a Random Variable Joint and Marginal Distributions Multivariable Change of Variables The General and Discrete Cases Continuous Random Vectors Expectation Introduction The Law of Large Numbers Basic Properties Axioms for Expectation (Part I) Derived Basic Properties Important Non-Properties Moments First Moments and Means Second Moments and Variances Standard Deviations and Standardization Mixed Moments and Covariances Exchangeable Random Variables Correlation iii

4 iv Stat 5101 (Geyer) Course Notes 2.5 Probability Theory as Linear Algebra The Vector Space L Two Notions of Linear Functions Expectation on Finite Sample Spaces Axioms for Expectation (Part II) General Discrete Probability Models Continuous Probability Models The Trick of Recognizing a Probability Density Probability Zero How to Tell When Expectations Exist L p Spaces Probability is a Special Case of Expectation Independence Two Definitions The Factorization Criterion Independence and Correlation Conditional Probability and Expectation Parametric Families of Distributions Conditional Probability Distributions Axioms for Conditional Expectation Functions of Conditioning Variables The Regression Function Iterated Expectations Joint, Conditional, and Marginal Joint Equals Conditional Times Marginal Normalization Renormalization Renormalization, Part II Bayes Rule Conditional Expectation and Prediction Parametric Families of Distributions Location-Scale Families The Gamma Distribution The Beta Distribution The Poisson Process Spatial Point Processes The Poisson Process One-Dimensional Poisson Processes Multivariate Theory Random Vectors Vectors, Scalars, and Matrices Random Vectors Random Matrices...128

5 CONTENTS v Variance Matrices What is the Variance of a Random Matrix? Covariance Matrices Linear Transformations Characterization of Variance Matrices Degenerate Random Vectors Correlation Matrices The Multivariate Normal Distribution The Density Marginals Partitioned Matrices Conditionals and Independence Bernoulli Random Vectors Categorical Random Variables Moments The Multinomial Distribution Categorical Random Variables Moments Degeneracy Density Marginals and Sort Of Marginals Conditionals Convergence Concepts Univariate Theory Convergence in Distribution The Central Limit Theorem Convergence in Probability The Law of Large Numbers The Continuous Mapping Theorem Slutsky s Theorem Comparison of the LLN and the CLT Applying the CLT to Addition Rules The Cauchy Distribution Sampling Theory Empirical Distributions The Mean of the Empirical Distribution The Variance of the Empirical Distribution Characterization of the Mean Review of Quantiles Quantiles of the Empirical Distribution The Empirical Median Characterization of the Median Samples and Populations Finite Population Sampling

6 vi Stat 5101 (Geyer) Course Notes Repeated Experiments Sampling Distributions of Sample Moments Sample Moments Sampling Distributions Moments Asymptotic Distributions The t Distribution The F Distribution Sampling Distributions Related to the Normal Sampling Distributions of Sample Quantiles Convergence Concepts Continued Multivariate Convergence Concepts Convergence in Probability to a Constant The Law of Large Numbers Convergence in Distribution The Central Limit Theorem Slutsky and Related Theorems The Delta Method The Univariate Delta Method The Multivariate Delta Method Asymptotics for Sample Moments Asymptotics of Independent Sequences Asymptotics of Sample Quantiles Frequentist Statistical Inference Introduction Inference The Sample and the Population Frequentist versus Bayesian Inference Models, Parameters, and Statistics Parametric Models Nonparametric Models Semiparametric Models Interest and Nuisance Parameters Statistics Point Estimation Bias Mean Squared Error Consistency Asymptotic Normality Method of Moments Estimators Relative Efficiency Asymptotic Relative Efficiency (ARE) Interval Estimation Exact Confidence Intervals for Means...252

7 CONTENTS vii Pivotal Quantities Approximate Confidence Intervals for Means Paired Comparisons Independent Samples Confidence Intervals for Variances The Role of Asymptotics Robustness Tests of Significance Interest and Nuisance Parameters Revisited Statistical Hypotheses Tests of Equality-Constrained Null Hypotheses P -values One-Tailed Tests The Duality of Tests and Confidence Intervals Sample Size Calculations Multiple Tests and Confidence Intervals Likelihood Inference Likelihood Maximum Likelihood Sampling Theory Derivatives of the Log Likelihood The Sampling Distribution of the MLE Asymptotic Relative Efficiency Estimating the Variance Tests and Confidence Intervals Multiparameter Models Maximum Likelihood Sampling Theory Likelihood Ratio Tests Change of Parameters Invariance of Likelihood Invariance of the MLE Invariance of Likelihood Ratio Tests Covariance of Fisher Information Bayesian Inference Parametric Models and Conditional Probability Prior and Posterior Distributions Prior Distributions Posterior Distributions The Subjective Bayes Philosophy More on Prior and Posterior Distributions Improper Priors Conjugate Priors The Two-Parameter Normal Distribution...362

8 viii Stat 5101 (Geyer) Course Notes 11.5 Bayesian Point Estimates Highest Posterior Density Regions Bayes Tests Bayesian Asymptotics Regression The Population Regression Function Regression and Conditional Expectation Best Prediction Best Linear Prediction The Sample Regression Function Sampling Theory The Regression Model The Gauss-Markov Theorem The Sampling Distribution of the Estimates Tests and Confidence Intervals for Regression Coefficients The Hat Matrix Polynomial Regression The F -Test for Model Comparison Intervals for the Regression Function The Abstract View of Regression Categorical Predictors (ANOVA) Categorical Predictors and Dummy Variables ANOVA Residual Analysis Leave One Out Quantile-Quantile Plots Model Selection Overfitting Mean Square Error The Bias-Variance Trade-Off Model Selection Criteria All Subsets Regression Bernoulli Regression A Dumb Idea (Identity Link) Logistic Regression (Logit Link) Probit Regression (Probit Link) Generalized Linear Models Parameter Estimation Fisher Information, Tests and Confidence Intervals Poisson Regression Overdispersion A Greek Letters 469

9 CONTENTS ix B Summary of Brand-Name Distributions 471 B.1 Discrete Distributions B.1.1 The Discrete Uniform Distribution B.1.2 The Binomial Distribution B.1.3 The Geometric Distribution, Type II B.1.4 The Poisson Distribution B.1.5 The Bernoulli Distribution B.1.6 The Negative Binomial Distribution, Type I B.1.7 The Negative Binomial Distribution, Type II B.1.8 The Geometric Distribution, Type I B.2 Continuous Distributions B.2.1 The Uniform Distribution B.2.2 The Exponential Distribution B.2.3 The Gamma Distribution B.2.4 The Beta Distribution B.2.5 The Normal Distribution B.2.6 The Chi-Square Distribution B.2.7 The Cauchy Distribution B.2.8 Student s t Distribution B.2.9 Snedecor s F Distribution B.3 Special Functions B.3.1 The Gamma Function B.3.2 The Beta Function B.4 Discrete Multivariate Distributions B.4.1 The Multinomial Distribution B.5 Continuous Multivariate Distributions B.5.1 The Uniform Distribution B.5.2 The Standard Normal Distribution B.5.3 The Multivariate Normal Distribution B.5.4 The Bivariate Normal Distribution C Addition Rules for Distributions 485 D Relations Among Brand Name Distributions 487 D.1 Special Cases D.2 Relations Involving Bernoulli Sequences D.3 Relations Involving Poisson Processes D.4 Normal, Chi-Square, t, and F D.4.1 Definition of Chi-Square D.4.2 Definition of t D.4.3 Definition of F D.4.4 t as a Special Case of F...489

10 x Stat 5101 (Geyer) Course Notes E Eigenvalues and Eigenvectors 491 E.1 Orthogonal and Orthonormal Vectors E.2 Eigenvalues and Eigenvectors E.3 Positive Definite Matrices F Normal Approximations for Distributions 499 F.1 Binomial Distribution F.2 Negative Binomial Distribution F.3 Poisson Distribution F.4 Gamma Distribution F.5 Chi-Square Distribution G Maximization of Functions 501 G.1 Functions of One Variable G.2 Concave Functions of One Variable G.3 Functions of Several Variables G.4 Concave Functions of Several Variables H Projections and Chi-Squares 509 H.1 Orthogonal Projections H.2 Chi-Squares

Statistical Methods in HYDROLOGY CHARLES T. HAAN. The Iowa State University Press / Ames

Statistical Methods in HYDROLOGY CHARLES T. HAAN The Iowa State University Press / Ames Univariate BASIC Table of Contents PREFACE xiii ACKNOWLEDGEMENTS xv 1 INTRODUCTION 1 2 PROBABILITY AND PROBABILITY

PART I INTRODUCTION The meaning of probability Basic definitions for frequentist statistics and Bayesian inference Bayesian inference Combinatorics

Table of Preface page xi PART I INTRODUCTION 1 1 The meaning of probability 3 1.1 Classical definition of probability 3 1.2 Statistical definition of probability 9 1.3 Bayesian understanding of probability

PRINCIPLES OF STATISTICAL INFERENCE

Advanced Series on Statistical Science & Applied Probability PRINCIPLES OF STATISTICAL INFERENCE from a Neo-Fisherian Perspective Luigi Pace Department of Statistics University ofudine, Italy Alessandra

Stat260: Bayesian Modeling and Inference Lecture Date: February 10th, Jeffreys priors. exp 1 ) p 2

Stat260: Bayesian Modeling and Inference Lecture Date: February 10th, 2010 Jeffreys priors Lecturer: Michael I. Jordan Scribe: Timothy Hunter 1 Priors for the multivariate Gaussian Consider a multivariate

Testing Statistical Hypotheses

E.L. Lehmann Joseph P. Romano Testing Statistical Hypotheses Third Edition 4y Springer Preface vii I Small-Sample Theory 1 1 The General Decision Problem 3 1.1 Statistical Inference and Statistical Decisions

Testing Statistical Hypotheses

E.L. Lehmann Joseph P. Romano, 02LEu1 ttd ~Lt~S Testing Statistical Hypotheses Third Edition With 6 Illustrations ~Springer 2 The Probability Background 28 2.1 Probability and Measure 28 2.2 Integration.........

Stat 5101 Notes: Brand Name Distributions

Stat 5101 Notes: Brand Name Distributions Charles J. Geyer February 14, 2003 1 Discrete Uniform Distribution DiscreteUniform(n). Discrete. Rationale Equally likely outcomes. The interval 1, 2,..., n of

Pattern Recognition and Machine Learning. Bishop Chapter 2: Probability Distributions

Pattern Recognition and Machine Learning Chapter 2: Probability Distributions Cécile Amblard Alex Kläser Jakob Verbeek October 11, 27 Probability Distributions: General Density Estimation: given a finite

Numerical Analysis for Statisticians

Kenneth Lange Numerical Analysis for Statisticians Springer Contents Preface v 1 Recurrence Relations 1 1.1 Introduction 1 1.2 Binomial CoefRcients 1 1.3 Number of Partitions of a Set 2 1.4 Horner's Method

From Practical Data Analysis with JMP, Second Edition. Full book available for purchase here. Contents About This Book... xiii About The Author... xxiii Chapter 1 Getting Started: Data Analysis with JMP...

* Tuesday 17 January :30-16:30 (2 hours) Recored on ESSE3 General introduction to the course.

Name of the course Statistical methods and data analysis Audience The course is intended for students of the first or second year of the Graduate School in Materials Engineering. The aim of the course

For Bonnie and Jesse (again)

SECOND EDITION A P P L I E D R E G R E S S I O N A N A L Y S I S a n d G E N E R A L I Z E D L I N E A R M O D E L S For Bonnie and Jesse (again) SECOND EDITION A P P L I E D R E G R E S S I O N A N A

IDL Advanced Math & Stats Module

IDL Advanced Math & Stats Module Regression List of Routines and Functions Multiple Linear Regression IMSL_REGRESSORS IMSL_MULTIREGRESS IMSL_MULTIPREDICT Generates regressors for a general linear model.

Multilevel Statistical Models: 3 rd edition, 2003 Contents

Multilevel Statistical Models: 3 rd edition, 2003 Contents Preface Acknowledgements Notation Two and three level models. A general classification notation and diagram Glossary Chapter 1 An introduction

STATISTICS ( CODE NO. 08 ) PAPER I PART - I

STATISTICS ( CODE NO. 08 ) PAPER I PART - I 1. Descriptive Statistics Types of data - Concepts of a Statistical population and sample from a population ; qualitative and quantitative data ; nominal and

Generalized linear models

Generalized linear models Douglas Bates November 01, 2010 Contents 1 Definition 1 2 Links 2 3 Estimating parameters 5 4 Example 6 5 Model building 8 6 Conclusions 8 7 Summary 9 1 Generalized Linear Models

Hypothesis Testing. 1 Definitions of test statistics. CB: chapter 8; section 10.3

Hypothesis Testing CB: chapter 8; section 0.3 Hypothesis: statement about an unknown population parameter Examples: The average age of males in Sweden is 7. (statement about population mean) The lowest

Linear Models 1. Isfahan University of Technology Fall Semester, 2014

Linear Models 1 Isfahan University of Technology Fall Semester, 2014 References: [1] G. A. F., Seber and A. J. Lee (2003). Linear Regression Analysis (2nd ed.). Hoboken, NJ: Wiley. [2] A. C. Rencher and

Chapter 1 Statistical Inference

Chapter 1 Statistical Inference causal inference To infer causality, you need a randomized experiment (or a huge observational study and lots of outside information). inference to populations Generalizations

Linear Regression Models P8111

Linear Regression Models P8111 Lecture 25 Jeff Goldsmith April 26, 2016 1 of 37 Today s Lecture Logistic regression / GLMs Model framework Interpretation Estimation 2 of 37 Linear regression Course started

Theory and Methods of Statistical Inference. PART I Frequentist theory and methods

PhD School in Statistics cycle XXVI, 2011 Theory and Methods of Statistical Inference PART I Frequentist theory and methods (A. Salvan, N. Sartori, L. Pace) Syllabus Some prerequisites: Empirical distribution

Latent Variable Models for Binary Data. Suppose that for a given vector of explanatory variables x, the latent

Latent Variable Models for Binary Data Suppose that for a given vector of explanatory variables x, the latent variable, U, has a continuous cumulative distribution function F (u; x) and that the binary

Generalized Linear Models (GLZ)

Generalized Linear Models (GLZ) Generalized Linear Models (GLZ) are an extension of the linear modeling process that allows models to be fit to data that follow probability distributions other than the

STATISTIQUE THÉORIQUE ET APPLIQUÉE Tome 1 Statistique descriptive et bases de l inférence statistique Pierre Dagnelie INDEX DES TRADUCTIONS ANGLAISES Bruxelles, De Boeck, 2013, 517 p. ISBN 978-2-8041-7560-3

Theory and Methods of Statistical Inference. PART I Frequentist likelihood methods

PhD School in Statistics XXV cycle, 2010 Theory and Methods of Statistical Inference PART I Frequentist likelihood methods (A. Salvan, N. Sartori, L. Pace) Syllabus Some prerequisites: Empirical distribution

1 Appendix A: Matrix Algebra

Appendix A: Matrix Algebra. Definitions Matrix A =[ ]=[A] Symmetric matrix: = for all and Diagonal matrix: 6=0if = but =0if 6= Scalar matrix: the diagonal matrix of = Identity matrix: the scalar matrix

11. Generalized Linear Models: An Introduction

Sociology 740 John Fox Lecture Notes 11. Generalized Linear Models: An Introduction Copyright 2014 by John Fox Generalized Linear Models: An Introduction 1 1. Introduction I A synthesis due to Nelder and

Subjective and Objective Bayesian Statistics

Subjective and Objective Bayesian Statistics Principles, Models, and Applications Second Edition S. JAMES PRESS with contributions by SIDDHARTHA CHIB MERLISE CLYDE GEORGE WOODWORTH ALAN ZASLAVSKY \WILEY-

INTRODUCTION TO BAYESIAN INFERENCE PART 2 CHRIS BISHOP

INTRODUCTION TO BAYESIAN INFERENCE PART 2 CHRIS BISHOP Personal Healthcare Revolution Electronic health records (CFH) Personal genomics (DeCode, Navigenics, 23andMe) X-prize: first \$10k human genome technology

i=1 h n (ˆθ n ) = 0. (2)

Stat 8112 Lecture Notes Unbiased Estimating Equations Charles J. Geyer April 29, 2012 1 Introduction In this handout we generalize the notion of maximum likelihood estimation to solution of unbiased estimating

Mathematics portion of the Doctor of Engineering Qualifying Examination

Mathematics portion of the Doctor of Engineering Qualifying Examination. The exam will be made up by faculty members of the Department of Mathematics and Computer Science. Dr. Kathy Zhong ( zhongk@udmercy.edu

Multivariate Analysis and Likelihood Inference

Multivariate Analysis and Likelihood Inference Outline 1 Joint Distribution of Random Variables 2 Principal Component Analysis (PCA) 3 Multivariate Normal Distribution 4 Likelihood Inference Joint density

Sigmaplot di Systat Software

Sigmaplot di Systat Software SigmaPlot Has Extensive Statistical Analysis Features SigmaPlot is now bundled with SigmaStat as an easy-to-use package for complete graphing and data analysis. The statistical

Bayesian linear regression

Bayesian linear regression Linear regression is the basis of most statistical modeling. The model is Y i = X T i β + ε i, where Y i is the continuous response X i = (X i1,..., X ip ) T is the corresponding

Theory and Methods of Statistical Inference

PhD School in Statistics cycle XXIX, 2014 Theory and Methods of Statistical Inference Instructors: B. Liseo, L. Pace, A. Salvan (course coordinator), N. Sartori, A. Tancredi, L. Ventura Syllabus Some prerequisites:

Course in Data Science

Course in Data Science About the Course: In this course you will get an introduction to the main tools and ideas which are required for Data Scientist/Business Analyst/Data Analyst. The course gives an

Confidence Intervals for Normal Data Spring 2014

Confidence Intervals for Normal Data 18.05 Spring 2014 Agenda Today Review of critical values and quantiles. Computing z, t, χ 2 confidence intervals for normal data. Conceptual view of confidence intervals.

Test Code: STA/STB (Short Answer Type) 2013 Junior Research Fellowship for Research Course in Statistics

Test Code: STA/STB (Short Answer Type) 2013 Junior Research Fellowship for Research Course in Statistics The candidates for the research course in Statistics will have to take two shortanswer type tests

Linear Regression. Data Model. β, σ 2. Process Model. ,V β. ,s 2. s 1. Parameter Model

Regression: Part II Linear Regression y~n X, 2 X Y Data Model β, σ 2 Process Model Β 0,V β s 1,s 2 Parameter Model Assumptions of Linear Model Homoskedasticity No error in X variables Error in Y variables

Review (Probability & Linear Algebra)

Review (Probability & Linear Algebra) CE-725 : Statistical Pattern Recognition Sharif University of Technology Spring 2013 M. Soleymani Outline Axioms of probability theory Conditional probability, Joint

Applied Regression Modeling

Applied Regression Modeling A Business Approach Iain Pardoe University of Oregon Charles H. Lundquist College of Business Eugene, Oregon WILEY- INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION CONTENTS

Motivation Scale Mixutres of Normals Finite Gaussian Mixtures Skew-Normal Models. Mixture Models. Econ 690. Purdue University

Econ 690 Purdue University In virtually all of the previous lectures, our models have made use of normality assumptions. From a computational point of view, the reason for this assumption is clear: combined

Lessons in Estimation Theory for Signal Processing, Communications, and Control

Lessons in Estimation Theory for Signal Processing, Communications, and Control Jerry M. Mendel Department of Electrical Engineering University of Southern California Los Angeles, California PRENTICE HALL

Karl-Rudolf Koch Introduction to Bayesian Statistics Second Edition

Karl-Rudolf Koch Introduction to Bayesian Statistics Second Edition Karl-Rudolf Koch Introduction to Bayesian Statistics Second, updated and enlarged Edition With 17 Figures Professor Dr.-Ing., Dr.-Ing.

Prentice Hall Stats: Modeling the World 2004 (Bock) Correlated to: National Advanced Placement (AP) Statistics Course Outline (Grades 9-12)

National Advanced Placement (AP) Statistics Course Outline (Grades 9-12) Following is an outline of the major topics covered by the AP Statistics Examination. The ordering here is intended to define the

Model Estimation Example

Ronald H. Heck 1 EDEP 606: Multivariate Methods (S2013) April 7, 2013 Model Estimation Example As we have moved through the course this semester, we have encountered the concept of model estimation. Discussions

Machine Learning Linear Classification. Prof. Matteo Matteucci

Machine Learning Linear Classification Prof. Matteo Matteucci Recall from the first lecture 2 X R p Regression Y R Continuous Output X R p Y {Ω 0, Ω 1,, Ω K } Classification Discrete Output X R p Y (X)

388 Index Differencing test ,232 Distributed lags , 147 arithmetic lag.

INDEX Aggregation... 104 Almon lag... 135-140,149 AR(1) process... 114-130,240,246,324-325,366,370,374 ARCH... 376-379 ARlMA... 365 Asymptotically unbiased... 13,50 Autocorrelation... 113-130, 142-150,324-325,365-369

Statistical techniques for data analysis in Cosmology

Statistical techniques for data analysis in Cosmology arxiv:0712.3028; arxiv:0911.3105 Numerical recipes (the bible ) Licia Verde ICREA & ICC UB-IEEC http://icc.ub.edu/~liciaverde outline Lecture 1: Introduction

Statistics - Lecture One. Outline. Charlotte Wickham 1. Basic ideas about estimation

Statistics - Lecture One Charlotte Wickham wickham@stat.berkeley.edu http://www.stat.berkeley.edu/~wickham/ Outline 1. Basic ideas about estimation 2. Method of Moments 3. Maximum Likelihood 4. Confidence

Statistical Data Analysis 2017/18

Statistical Data Analysis 2017/18 London Postgraduate Lectures on Particle Physics; University of London MSci course PH4515 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk

Statistical Inference Writeup

Statistical Inference Writeup jongman@gmail.com January 19, 2015 This is a personal writeup of Statistical Inference (Casella and Berger, 2nd ed.). The purpose of this note is to keep a log of my impressions

A Very Brief Summary of Bayesian Inference, and Examples

A Very Brief Summary of Bayesian Inference, and Examples Trinity Term 009 Prof Gesine Reinert Our starting point are data x = x 1, x,, x n, which we view as realisations of random variables X 1, X,, X

Module 22: Bayesian Methods Lecture 9 A: Default prior selection

Module 22: Bayesian Methods Lecture 9 A: Default prior selection Peter Hoff Departments of Statistics and Biostatistics University of Washington Outline Jeffreys prior Unit information priors Empirical

STATISTICS-STAT (STAT)

Statistics-STAT (STAT) 1 STATISTICS-STAT (STAT) Courses STAT 158 Introduction to R Programming Credit: 1 (1-0-0) Programming using the R Project for the Statistical Computing. Data objects, for loops,

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 7 Approximate

Advanced Statistics I : Gaussian Linear Model (and beyond)

Advanced Statistics I : Gaussian Linear Model (and beyond) Aurélien Garivier CNRS / Telecom ParisTech Centrale Outline One and Two-Sample Statistics Linear Gaussian Model Model Reduction and model Selection

QUANTITATIVE TECHNIQUES

UNIVERSITY OF CALICUT SCHOOL OF DISTANCE EDUCATION (For B Com. IV Semester & BBA III Semester) COMPLEMENTARY COURSE QUANTITATIVE TECHNIQUES QUESTION BANK 1. The techniques which provide the decision maker

The purpose of this section is to derive the asymptotic distribution of the Pearson chi-square statistic. k (n j np j ) 2. np j.

Chapter 9 Pearson s chi-square test 9. Null hypothesis asymptotics Let X, X 2, be independent from a multinomial(, p) distribution, where p is a k-vector with nonnegative entries that sum to one. That

Generalized Linear Models for Non-Normal Data

Generalized Linear Models for Non-Normal Data Today s Class: 3 parts of a generalized model Models for binary outcomes Complications for generalized multivariate or multilevel models SPLH 861: Lecture

Statistical methods and data analysis

Statistical methods and data analysis Teacher Stefano Siboni Aim The aim of the course is to illustrate the basic mathematical tools for the analysis and modelling of experimental data, particularly concerning

Next is material on matrix rank. Please see the handout

B90.330 / C.005 NOTES for Wednesday 0.APR.7 Suppose that the model is β + ε, but ε does not have the desired variance matrix. Say that ε is normal, but Var(ε) σ W. The form of W is W w 0 0 0 0 0 0 w 0

Bayesian Linear Models

Bayesian Linear Models Sudipto Banerjee 1 and Andrew O. Finley 2 1 Department of Forestry & Department of Geography, Michigan State University, Lansing Michigan, U.S.A. 2 Biostatistics, School of Public

A Course in Applied Econometrics Lecture 14: Control Functions and Related Methods. Jeff Wooldridge IRP Lectures, UW Madison, August 2008

A Course in Applied Econometrics Lecture 14: Control Functions and Related Methods Jeff Wooldridge IRP Lectures, UW Madison, August 2008 1. Linear-in-Parameters Models: IV versus Control Functions 2. Correlated

Engineering Mathematics

Thoroughly Revised and Updated Engineering Mathematics For GATE 2017 and ESE 2017 Prelims Note: ESE Mains Electrical Engineering also covered Publications Publications MADE EASY Publications Corporate

Stochastic Processes. Theory for Applications. Robert G. Gallager CAMBRIDGE UNIVERSITY PRESS

Stochastic Processes Theory for Applications Robert G. Gallager CAMBRIDGE UNIVERSITY PRESS Contents Preface page xv Swgg&sfzoMj ybr zmjfr%cforj owf fmdy xix Acknowledgements xxi 1 Introduction and review

Density Estimation. Seungjin Choi

Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/

Machine Learning 2017

Machine Learning 2017 Volker Roth Department of Mathematics & Computer Science University of Basel 21st March 2017 Volker Roth (University of Basel) Machine Learning 2017 21st March 2017 1 / 41 Section

STA216: Generalized Linear Models. Lecture 1. Review and Introduction

STA216: Generalized Linear Models Lecture 1. Review and Introduction Let y 1,..., y n denote n independent observations on a response Treat y i as a realization of a random variable Y i In the general

Bayesian Linear Models

Bayesian Linear Models Sudipto Banerjee 1 and Andrew O. Finley 2 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department of Forestry & Department

Introduction to Graphical Models

Introduction to Graphical Models The 15 th Winter School of Statistical Physics POSCO International Center & POSTECH, Pohang 2018. 1. 9 (Tue.) Yung-Kyun Noh GENERALIZATION FOR PREDICTION 2 Probabilistic

Noninformative Priors for the Ratio of the Scale Parameters in the Inverted Exponential Distributions

Communications for Statistical Applications and Methods 03, Vol. 0, No. 5, 387 394 DOI: http://dx.doi.org/0.535/csam.03.0.5.387 Noninformative Priors for the Ratio of the Scale Parameters in the Inverted

Maximum Likelihood Estimation; Robust Maximum Likelihood; Missing Data with Maximum Likelihood

Maximum Likelihood Estimation; Robust Maximum Likelihood; Missing Data with Maximum Likelihood PRE 906: Structural Equation Modeling Lecture #3 February 4, 2015 PRE 906, SEM: Estimation Today s Class An

Estimation of Operational Risk Capital Charge under Parameter Uncertainty

Estimation of Operational Risk Capital Charge under Parameter Uncertainty Pavel V. Shevchenko Principal Research Scientist, CSIRO Mathematical and Information Sciences, Sydney, Locked Bag 17, North Ryde,

COMS 4771 Regression. Nakul Verma

COMS 4771 Regression Nakul Verma Last time Support Vector Machines Maximum Margin formulation Constrained Optimization Lagrange Duality Theory Convex Optimization SVM dual and Interpretation How get the

Neutral Bayesian reference models for incidence rates of (rare) clinical events

Neutral Bayesian reference models for incidence rates of (rare) clinical events Jouni Kerman Statistical Methodology, Novartis Pharma AG, Basel BAYES2012, May 10, Aachen Outline Motivation why reference

Linear Regression Models. Based on Chapter 3 of Hastie, Tibshirani and Friedman

Linear Regression Models Based on Chapter 3 of Hastie, ibshirani and Friedman Linear Regression Models Here the X s might be: p f ( X = " + " 0 j= 1 X j Raw predictor variables (continuous or coded-categorical

Bayesian methods in economics and finance

1/26 Bayesian methods in economics and finance Linear regression: Bayesian model selection and sparsity priors Linear Regression 2/26 Linear regression Model for relationship between (several) independent

Hypothesis Testing. Econ 690. Purdue University. Justin L. Tobias (Purdue) Testing 1 / 33

Hypothesis Testing Econ 690 Purdue University Justin L. Tobias (Purdue) Testing 1 / 33 Outline 1 Basic Testing Framework 2 Testing with HPD intervals 3 Example 4 Savage Dickey Density Ratio 5 Bartlett

Fundamentals of Probability Theory and Mathematical Statistics

Fundamentals of Probability Theory and Mathematical Statistics Gerry Del Fiacco Math Center Metropolitan State University St. Paul, Minnesota June 6, 2016 1 Preface This collection of material was researched,

σ(a) = a N (x; 0, 1 2 ) dx. σ(a) = Φ(a) =

Until now we have always worked with likelihoods and prior distributions that were conjugate to each other, allowing the computation of the posterior distribution to be done in closed form. Unfortunately,

Bayesian Inference: Posterior Intervals

Bayesian Inference: Posterior Intervals Simple values like the posterior mean E[θ X] and posterior variance var[θ X] can be useful in learning about θ. Quantiles of π(θ X) (especially the posterior median)

Likelihood and Bayesian Inference for Proportions

Likelihood and Bayesian Inference for Proportions September 9, 2009 Readings Hoff Chapter 3 Likelihood and Bayesian Inferencefor Proportions p.1/21 Giardia In a New Zealand research program on human health

Measuring nuisance parameter effects in Bayesian inference

Measuring nuisance parameter effects in Bayesian inference Alastair Young Imperial College London WHOA-PSI-2017 1 / 31 Acknowledgements: Tom DiCiccio, Cornell University; Daniel Garcia Rasines, Imperial

Chapter 6: Functions of Random Variables

Chapter 6: Functions of Random Variables We are often interested in a function of one or several random variables, U(Y 1,..., Y n ). We will study three methods for determining the distribution of a function

STAT 135 Lab 6 Duality of Hypothesis Testing and Confidence Intervals, GLRT, Pearson χ 2 Tests and Q-Q plots. March 8, 2015

STAT 135 Lab 6 Duality of Hypothesis Testing and Confidence Intervals, GLRT, Pearson χ 2 Tests and Q-Q plots March 8, 2015 The duality between CI and hypothesis testing The duality between CI and hypothesis

Multinomial Data. f(y θ) θ y i. where θ i is the probability that a given trial results in category i, i = 1,..., k. The parameter space is

Multinomial Data The multinomial distribution is a generalization of the binomial for the situation in which each trial results in one and only one of several categories, as opposed to just two, as in

Introduction: MLE, MAP, Bayesian reasoning (28/8/13)

STA561: Probabilistic machine learning Introduction: MLE, MAP, Bayesian reasoning (28/8/13) Lecturer: Barbara Engelhardt Scribes: K. Ulrich, J. Subramanian, N. Raval, J. O Hollaren 1 Classifiers In this

Probability Theory for Machine Learning. Chris Cremer September 2015

Probability Theory for Machine Learning Chris Cremer September 2015 Outline Motivation Probability Definitions and Rules Probability Distributions MLE for Gaussian Parameter Estimation MLE and Least Squares

Glossary for the Triola Statistics Series

Glossary for the Triola Statistics Series Absolute deviation The measure of variation equal to the sum of the deviations of each value from the mean, divided by the number of values Acceptance sampling

Lecture Notes 15 Prediction Chapters 13, 22, 20.4.

Lecture Notes 15 Prediction Chapters 13, 22, 20.4. 1 Introduction Prediction is covered in detail in 36-707, 36-701, 36-715, 10/36-702. Here, we will just give an introduction. We observe training data

A Course in Applied Econometrics Lecture 18: Missing Data. Jeff Wooldridge IRP Lectures, UW Madison, August Linear model with IVs: y i x i u i,

A Course in Applied Econometrics Lecture 18: Missing Data Jeff Wooldridge IRP Lectures, UW Madison, August 2008 1. When Can Missing Data be Ignored? 2. Inverse Probability Weighting 3. Imputation 4. Heckman-Type

Chapter 1. Probability, Random Variables and Expectations. 1.1 Axiomatic Probability

Chapter 1 Probability, Random Variables and Expectations Note: The primary reference for these notes is Mittelhammer (1999. Other treatments of probability theory include Gallant (1997, Casella & Berger

More Spectral Clustering and an Introduction to Conjugacy

CS8B/Stat4B: Advanced Topics in Learning & Decision Making More Spectral Clustering and an Introduction to Conjugacy Lecturer: Michael I. Jordan Scribe: Marco Barreno Monday, April 5, 004. Back to spectral

Overall Objective Priors

Overall Objective Priors Jim Berger, Jose Bernardo and Dongchu Sun Duke University, University of Valencia and University of Missouri Recent advances in statistical inference: theory and case studies University

What s New in Econometrics. Lecture 13

What s New in Econometrics Lecture 13 Weak Instruments and Many Instruments Guido Imbens NBER Summer Institute, 2007 Outline 1. Introduction 2. Motivation 3. Weak Instruments 4. Many Weak) Instruments

Economics 671: Applied Econometrics Department of Economics, Finance and Legal Studies University of Alabama

Problem Set #1 (Random Data Generation) 1. Generate =500random numbers from both the uniform 1 ( [0 1], uniformbetween zero and one) and exponential exp ( ) (set =2and let [0 1]) distributions. Plot the