Stat 5101 Lecture Notes


 Abigayle Collins
 9 months ago
 Views:
Transcription
1 Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001
2 ii Stat 5101 (Geyer) Course Notes
3 Contents 1 Random Variables and Change of Variables Random Variables Variables Functions Random Variables: Informal Intuition Random Variables: Formal Definition Functions of Random Variables Change of Variables General Definition Discrete Random Variables Continuous Random Variables Random Vectors Discrete Random Vectors Continuous Random Vectors The Support of a Random Variable Joint and Marginal Distributions Multivariable Change of Variables The General and Discrete Cases Continuous Random Vectors Expectation Introduction The Law of Large Numbers Basic Properties Axioms for Expectation (Part I) Derived Basic Properties Important NonProperties Moments First Moments and Means Second Moments and Variances Standard Deviations and Standardization Mixed Moments and Covariances Exchangeable Random Variables Correlation iii
4 iv Stat 5101 (Geyer) Course Notes 2.5 Probability Theory as Linear Algebra The Vector Space L Two Notions of Linear Functions Expectation on Finite Sample Spaces Axioms for Expectation (Part II) General Discrete Probability Models Continuous Probability Models The Trick of Recognizing a Probability Density Probability Zero How to Tell When Expectations Exist L p Spaces Probability is a Special Case of Expectation Independence Two Definitions The Factorization Criterion Independence and Correlation Conditional Probability and Expectation Parametric Families of Distributions Conditional Probability Distributions Axioms for Conditional Expectation Functions of Conditioning Variables The Regression Function Iterated Expectations Joint, Conditional, and Marginal Joint Equals Conditional Times Marginal Normalization Renormalization Renormalization, Part II Bayes Rule Conditional Expectation and Prediction Parametric Families of Distributions LocationScale Families The Gamma Distribution The Beta Distribution The Poisson Process Spatial Point Processes The Poisson Process OneDimensional Poisson Processes Multivariate Theory Random Vectors Vectors, Scalars, and Matrices Random Vectors Random Matrices...128
5 CONTENTS v Variance Matrices What is the Variance of a Random Matrix? Covariance Matrices Linear Transformations Characterization of Variance Matrices Degenerate Random Vectors Correlation Matrices The Multivariate Normal Distribution The Density Marginals Partitioned Matrices Conditionals and Independence Bernoulli Random Vectors Categorical Random Variables Moments The Multinomial Distribution Categorical Random Variables Moments Degeneracy Density Marginals and Sort Of Marginals Conditionals Convergence Concepts Univariate Theory Convergence in Distribution The Central Limit Theorem Convergence in Probability The Law of Large Numbers The Continuous Mapping Theorem Slutsky s Theorem Comparison of the LLN and the CLT Applying the CLT to Addition Rules The Cauchy Distribution Sampling Theory Empirical Distributions The Mean of the Empirical Distribution The Variance of the Empirical Distribution Characterization of the Mean Review of Quantiles Quantiles of the Empirical Distribution The Empirical Median Characterization of the Median Samples and Populations Finite Population Sampling
6 vi Stat 5101 (Geyer) Course Notes Repeated Experiments Sampling Distributions of Sample Moments Sample Moments Sampling Distributions Moments Asymptotic Distributions The t Distribution The F Distribution Sampling Distributions Related to the Normal Sampling Distributions of Sample Quantiles Convergence Concepts Continued Multivariate Convergence Concepts Convergence in Probability to a Constant The Law of Large Numbers Convergence in Distribution The Central Limit Theorem Slutsky and Related Theorems The Delta Method The Univariate Delta Method The Multivariate Delta Method Asymptotics for Sample Moments Asymptotics of Independent Sequences Asymptotics of Sample Quantiles Frequentist Statistical Inference Introduction Inference The Sample and the Population Frequentist versus Bayesian Inference Models, Parameters, and Statistics Parametric Models Nonparametric Models Semiparametric Models Interest and Nuisance Parameters Statistics Point Estimation Bias Mean Squared Error Consistency Asymptotic Normality Method of Moments Estimators Relative Efficiency Asymptotic Relative Efficiency (ARE) Interval Estimation Exact Confidence Intervals for Means...252
7 CONTENTS vii Pivotal Quantities Approximate Confidence Intervals for Means Paired Comparisons Independent Samples Confidence Intervals for Variances The Role of Asymptotics Robustness Tests of Significance Interest and Nuisance Parameters Revisited Statistical Hypotheses Tests of EqualityConstrained Null Hypotheses P values OneTailed Tests The Duality of Tests and Confidence Intervals Sample Size Calculations Multiple Tests and Confidence Intervals Likelihood Inference Likelihood Maximum Likelihood Sampling Theory Derivatives of the Log Likelihood The Sampling Distribution of the MLE Asymptotic Relative Efficiency Estimating the Variance Tests and Confidence Intervals Multiparameter Models Maximum Likelihood Sampling Theory Likelihood Ratio Tests Change of Parameters Invariance of Likelihood Invariance of the MLE Invariance of Likelihood Ratio Tests Covariance of Fisher Information Bayesian Inference Parametric Models and Conditional Probability Prior and Posterior Distributions Prior Distributions Posterior Distributions The Subjective Bayes Philosophy More on Prior and Posterior Distributions Improper Priors Conjugate Priors The TwoParameter Normal Distribution...362
8 viii Stat 5101 (Geyer) Course Notes 11.5 Bayesian Point Estimates Highest Posterior Density Regions Bayes Tests Bayesian Asymptotics Regression The Population Regression Function Regression and Conditional Expectation Best Prediction Best Linear Prediction The Sample Regression Function Sampling Theory The Regression Model The GaussMarkov Theorem The Sampling Distribution of the Estimates Tests and Confidence Intervals for Regression Coefficients The Hat Matrix Polynomial Regression The F Test for Model Comparison Intervals for the Regression Function The Abstract View of Regression Categorical Predictors (ANOVA) Categorical Predictors and Dummy Variables ANOVA Residual Analysis Leave One Out QuantileQuantile Plots Model Selection Overfitting Mean Square Error The BiasVariance TradeOff Model Selection Criteria All Subsets Regression Bernoulli Regression A Dumb Idea (Identity Link) Logistic Regression (Logit Link) Probit Regression (Probit Link) Generalized Linear Models Parameter Estimation Fisher Information, Tests and Confidence Intervals Poisson Regression Overdispersion A Greek Letters 469
9 CONTENTS ix B Summary of BrandName Distributions 471 B.1 Discrete Distributions B.1.1 The Discrete Uniform Distribution B.1.2 The Binomial Distribution B.1.3 The Geometric Distribution, Type II B.1.4 The Poisson Distribution B.1.5 The Bernoulli Distribution B.1.6 The Negative Binomial Distribution, Type I B.1.7 The Negative Binomial Distribution, Type II B.1.8 The Geometric Distribution, Type I B.2 Continuous Distributions B.2.1 The Uniform Distribution B.2.2 The Exponential Distribution B.2.3 The Gamma Distribution B.2.4 The Beta Distribution B.2.5 The Normal Distribution B.2.6 The ChiSquare Distribution B.2.7 The Cauchy Distribution B.2.8 Student s t Distribution B.2.9 Snedecor s F Distribution B.3 Special Functions B.3.1 The Gamma Function B.3.2 The Beta Function B.4 Discrete Multivariate Distributions B.4.1 The Multinomial Distribution B.5 Continuous Multivariate Distributions B.5.1 The Uniform Distribution B.5.2 The Standard Normal Distribution B.5.3 The Multivariate Normal Distribution B.5.4 The Bivariate Normal Distribution C Addition Rules for Distributions 485 D Relations Among Brand Name Distributions 487 D.1 Special Cases D.2 Relations Involving Bernoulli Sequences D.3 Relations Involving Poisson Processes D.4 Normal, ChiSquare, t, and F D.4.1 Definition of ChiSquare D.4.2 Definition of t D.4.3 Definition of F D.4.4 t as a Special Case of F...489
10 x Stat 5101 (Geyer) Course Notes E Eigenvalues and Eigenvectors 491 E.1 Orthogonal and Orthonormal Vectors E.2 Eigenvalues and Eigenvectors E.3 Positive Definite Matrices F Normal Approximations for Distributions 499 F.1 Binomial Distribution F.2 Negative Binomial Distribution F.3 Poisson Distribution F.4 Gamma Distribution F.5 ChiSquare Distribution G Maximization of Functions 501 G.1 Functions of One Variable G.2 Concave Functions of One Variable G.3 Functions of Several Variables G.4 Concave Functions of Several Variables H Projections and ChiSquares 509 H.1 Orthogonal Projections H.2 ChiSquares
Statistical Methods in HYDROLOGY CHARLES T. HAAN. The Iowa State University Press / Ames
Statistical Methods in HYDROLOGY CHARLES T. HAAN The Iowa State University Press / Ames Univariate BASIC Table of Contents PREFACE xiii ACKNOWLEDGEMENTS xv 1 INTRODUCTION 1 2 PROBABILITY AND PROBABILITY
More informationPART I INTRODUCTION The meaning of probability Basic definitions for frequentist statistics and Bayesian inference Bayesian inference Combinatorics
Table of Preface page xi PART I INTRODUCTION 1 1 The meaning of probability 3 1.1 Classical definition of probability 3 1.2 Statistical definition of probability 9 1.3 Bayesian understanding of probability
More informationPRINCIPLES OF STATISTICAL INFERENCE
Advanced Series on Statistical Science & Applied Probability PRINCIPLES OF STATISTICAL INFERENCE from a NeoFisherian Perspective Luigi Pace Department of Statistics University ofudine, Italy Alessandra
More informationStat260: Bayesian Modeling and Inference Lecture Date: February 10th, Jeffreys priors. exp 1 ) p 2
Stat260: Bayesian Modeling and Inference Lecture Date: February 10th, 2010 Jeffreys priors Lecturer: Michael I. Jordan Scribe: Timothy Hunter 1 Priors for the multivariate Gaussian Consider a multivariate
More informationTesting Statistical Hypotheses
E.L. Lehmann Joseph P. Romano Testing Statistical Hypotheses Third Edition 4y Springer Preface vii I SmallSample Theory 1 1 The General Decision Problem 3 1.1 Statistical Inference and Statistical Decisions
More informationTesting Statistical Hypotheses
E.L. Lehmann Joseph P. Romano, 02LEu1 ttd ~Lt~S Testing Statistical Hypotheses Third Edition With 6 Illustrations ~Springer 2 The Probability Background 28 2.1 Probability and Measure 28 2.2 Integration.........
More informationStat 5101 Notes: Brand Name Distributions
Stat 5101 Notes: Brand Name Distributions Charles J. Geyer February 14, 2003 1 Discrete Uniform Distribution DiscreteUniform(n). Discrete. Rationale Equally likely outcomes. The interval 1, 2,..., n of
More informationPattern Recognition and Machine Learning. Bishop Chapter 2: Probability Distributions
Pattern Recognition and Machine Learning Chapter 2: Probability Distributions Cécile Amblard Alex Kläser Jakob Verbeek October 11, 27 Probability Distributions: General Density Estimation: given a finite
More informationNumerical Analysis for Statisticians
Kenneth Lange Numerical Analysis for Statisticians Springer Contents Preface v 1 Recurrence Relations 1 1.1 Introduction 1 1.2 Binomial CoefRcients 1 1.3 Number of Partitions of a Set 2 1.4 Horner's Method
More informationFrom Practical Data Analysis with JMP, Second Edition. Full book available for purchase here. About This Book... xiii About The Author...
From Practical Data Analysis with JMP, Second Edition. Full book available for purchase here. Contents About This Book... xiii About The Author... xxiii Chapter 1 Getting Started: Data Analysis with JMP...
More information* Tuesday 17 January :3016:30 (2 hours) Recored on ESSE3 General introduction to the course.
Name of the course Statistical methods and data analysis Audience The course is intended for students of the first or second year of the Graduate School in Materials Engineering. The aim of the course
More informationFor Bonnie and Jesse (again)
SECOND EDITION A P P L I E D R E G R E S S I O N A N A L Y S I S a n d G E N E R A L I Z E D L I N E A R M O D E L S For Bonnie and Jesse (again) SECOND EDITION A P P L I E D R E G R E S S I O N A N A
More informationIDL Advanced Math & Stats Module
IDL Advanced Math & Stats Module Regression List of Routines and Functions Multiple Linear Regression IMSL_REGRESSORS IMSL_MULTIREGRESS IMSL_MULTIPREDICT Generates regressors for a general linear model.
More informationMultilevel Statistical Models: 3 rd edition, 2003 Contents
Multilevel Statistical Models: 3 rd edition, 2003 Contents Preface Acknowledgements Notation Two and three level models. A general classification notation and diagram Glossary Chapter 1 An introduction
More informationSTATISTICS ( CODE NO. 08 ) PAPER I PART  I
STATISTICS ( CODE NO. 08 ) PAPER I PART  I 1. Descriptive Statistics Types of data  Concepts of a Statistical population and sample from a population ; qualitative and quantitative data ; nominal and
More informationGeneralized linear models
Generalized linear models Douglas Bates November 01, 2010 Contents 1 Definition 1 2 Links 2 3 Estimating parameters 5 4 Example 6 5 Model building 8 6 Conclusions 8 7 Summary 9 1 Generalized Linear Models
More informationHypothesis Testing. 1 Definitions of test statistics. CB: chapter 8; section 10.3
Hypothesis Testing CB: chapter 8; section 0.3 Hypothesis: statement about an unknown population parameter Examples: The average age of males in Sweden is 7. (statement about population mean) The lowest
More informationLinear Models 1. Isfahan University of Technology Fall Semester, 2014
Linear Models 1 Isfahan University of Technology Fall Semester, 2014 References: [1] G. A. F., Seber and A. J. Lee (2003). Linear Regression Analysis (2nd ed.). Hoboken, NJ: Wiley. [2] A. C. Rencher and
More informationChapter 1 Statistical Inference
Chapter 1 Statistical Inference causal inference To infer causality, you need a randomized experiment (or a huge observational study and lots of outside information). inference to populations Generalizations
More informationLinear Regression Models P8111
Linear Regression Models P8111 Lecture 25 Jeff Goldsmith April 26, 2016 1 of 37 Today s Lecture Logistic regression / GLMs Model framework Interpretation Estimation 2 of 37 Linear regression Course started
More informationTheory and Methods of Statistical Inference. PART I Frequentist theory and methods
PhD School in Statistics cycle XXVI, 2011 Theory and Methods of Statistical Inference PART I Frequentist theory and methods (A. Salvan, N. Sartori, L. Pace) Syllabus Some prerequisites: Empirical distribution
More informationLatent Variable Models for Binary Data. Suppose that for a given vector of explanatory variables x, the latent
Latent Variable Models for Binary Data Suppose that for a given vector of explanatory variables x, the latent variable, U, has a continuous cumulative distribution function F (u; x) and that the binary
More informationGeneralized Linear Models (GLZ)
Generalized Linear Models (GLZ) Generalized Linear Models (GLZ) are an extension of the linear modeling process that allows models to be fit to data that follow probability distributions other than the
More informationINDEX DES TRADUCTIONS ANGLAISES
STATISTIQUE THÉORIQUE ET APPLIQUÉE Tome 1 Statistique descriptive et bases de l inférence statistique Pierre Dagnelie INDEX DES TRADUCTIONS ANGLAISES Bruxelles, De Boeck, 2013, 517 p. ISBN 9782804175603
More informationTheory and Methods of Statistical Inference. PART I Frequentist likelihood methods
PhD School in Statistics XXV cycle, 2010 Theory and Methods of Statistical Inference PART I Frequentist likelihood methods (A. Salvan, N. Sartori, L. Pace) Syllabus Some prerequisites: Empirical distribution
More information1 Appendix A: Matrix Algebra
Appendix A: Matrix Algebra. Definitions Matrix A =[ ]=[A] Symmetric matrix: = for all and Diagonal matrix: 6=0if = but =0if 6= Scalar matrix: the diagonal matrix of = Identity matrix: the scalar matrix
More information11. Generalized Linear Models: An Introduction
Sociology 740 John Fox Lecture Notes 11. Generalized Linear Models: An Introduction Copyright 2014 by John Fox Generalized Linear Models: An Introduction 1 1. Introduction I A synthesis due to Nelder and
More informationSubjective and Objective Bayesian Statistics
Subjective and Objective Bayesian Statistics Principles, Models, and Applications Second Edition S. JAMES PRESS with contributions by SIDDHARTHA CHIB MERLISE CLYDE GEORGE WOODWORTH ALAN ZASLAVSKY \WILEY
More informationINTRODUCTION TO BAYESIAN INFERENCE PART 2 CHRIS BISHOP
INTRODUCTION TO BAYESIAN INFERENCE PART 2 CHRIS BISHOP Personal Healthcare Revolution Electronic health records (CFH) Personal genomics (DeCode, Navigenics, 23andMe) Xprize: first $10k human genome technology
More informationi=1 h n (ˆθ n ) = 0. (2)
Stat 8112 Lecture Notes Unbiased Estimating Equations Charles J. Geyer April 29, 2012 1 Introduction In this handout we generalize the notion of maximum likelihood estimation to solution of unbiased estimating
More informationMathematics portion of the Doctor of Engineering Qualifying Examination
Mathematics portion of the Doctor of Engineering Qualifying Examination. The exam will be made up by faculty members of the Department of Mathematics and Computer Science. Dr. Kathy Zhong ( zhongk@udmercy.edu
More informationMultivariate Analysis and Likelihood Inference
Multivariate Analysis and Likelihood Inference Outline 1 Joint Distribution of Random Variables 2 Principal Component Analysis (PCA) 3 Multivariate Normal Distribution 4 Likelihood Inference Joint density
More informationSigmaplot di Systat Software
Sigmaplot di Systat Software SigmaPlot Has Extensive Statistical Analysis Features SigmaPlot is now bundled with SigmaStat as an easytouse package for complete graphing and data analysis. The statistical
More informationBayesian linear regression
Bayesian linear regression Linear regression is the basis of most statistical modeling. The model is Y i = X T i β + ε i, where Y i is the continuous response X i = (X i1,..., X ip ) T is the corresponding
More informationTheory and Methods of Statistical Inference
PhD School in Statistics cycle XXIX, 2014 Theory and Methods of Statistical Inference Instructors: B. Liseo, L. Pace, A. Salvan (course coordinator), N. Sartori, A. Tancredi, L. Ventura Syllabus Some prerequisites:
More informationCourse in Data Science
Course in Data Science About the Course: In this course you will get an introduction to the main tools and ideas which are required for Data Scientist/Business Analyst/Data Analyst. The course gives an
More informationConfidence Intervals for Normal Data Spring 2014
Confidence Intervals for Normal Data 18.05 Spring 2014 Agenda Today Review of critical values and quantiles. Computing z, t, χ 2 confidence intervals for normal data. Conceptual view of confidence intervals.
More informationTest Code: STA/STB (Short Answer Type) 2013 Junior Research Fellowship for Research Course in Statistics
Test Code: STA/STB (Short Answer Type) 2013 Junior Research Fellowship for Research Course in Statistics The candidates for the research course in Statistics will have to take two shortanswer type tests
More informationLinear Regression. Data Model. β, σ 2. Process Model. ,V β. ,s 2. s 1. Parameter Model
Regression: Part II Linear Regression y~n X, 2 X Y Data Model β, σ 2 Process Model Β 0,V β s 1,s 2 Parameter Model Assumptions of Linear Model Homoskedasticity No error in X variables Error in Y variables
More informationReview (Probability & Linear Algebra)
Review (Probability & Linear Algebra) CE725 : Statistical Pattern Recognition Sharif University of Technology Spring 2013 M. Soleymani Outline Axioms of probability theory Conditional probability, Joint
More informationApplied Regression Modeling
Applied Regression Modeling A Business Approach Iain Pardoe University of Oregon Charles H. Lundquist College of Business Eugene, Oregon WILEY INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION CONTENTS
More informationMotivation Scale Mixutres of Normals Finite Gaussian Mixtures SkewNormal Models. Mixture Models. Econ 690. Purdue University
Econ 690 Purdue University In virtually all of the previous lectures, our models have made use of normality assumptions. From a computational point of view, the reason for this assumption is clear: combined
More informationLessons in Estimation Theory for Signal Processing, Communications, and Control
Lessons in Estimation Theory for Signal Processing, Communications, and Control Jerry M. Mendel Department of Electrical Engineering University of Southern California Los Angeles, California PRENTICE HALL
More informationKarlRudolf Koch Introduction to Bayesian Statistics Second Edition
KarlRudolf Koch Introduction to Bayesian Statistics Second Edition KarlRudolf Koch Introduction to Bayesian Statistics Second, updated and enlarged Edition With 17 Figures Professor Dr.Ing., Dr.Ing.
More informationPrentice Hall Stats: Modeling the World 2004 (Bock) Correlated to: National Advanced Placement (AP) Statistics Course Outline (Grades 912)
National Advanced Placement (AP) Statistics Course Outline (Grades 912) Following is an outline of the major topics covered by the AP Statistics Examination. The ordering here is intended to define the
More informationModel Estimation Example
Ronald H. Heck 1 EDEP 606: Multivariate Methods (S2013) April 7, 2013 Model Estimation Example As we have moved through the course this semester, we have encountered the concept of model estimation. Discussions
More informationMachine Learning Linear Classification. Prof. Matteo Matteucci
Machine Learning Linear Classification Prof. Matteo Matteucci Recall from the first lecture 2 X R p Regression Y R Continuous Output X R p Y {Ω 0, Ω 1,, Ω K } Classification Discrete Output X R p Y (X)
More information388 Index Differencing test ,232 Distributed lags , 147 arithmetic lag.
INDEX Aggregation... 104 Almon lag... 135140,149 AR(1) process... 114130,240,246,324325,366,370,374 ARCH... 376379 ARlMA... 365 Asymptotically unbiased... 13,50 Autocorrelation... 113130, 142150,324325,365369
More informationStatistical techniques for data analysis in Cosmology
Statistical techniques for data analysis in Cosmology arxiv:0712.3028; arxiv:0911.3105 Numerical recipes (the bible ) Licia Verde ICREA & ICC UBIEEC http://icc.ub.edu/~liciaverde outline Lecture 1: Introduction
More informationStatistics  Lecture One. Outline. Charlotte Wickham 1. Basic ideas about estimation
Statistics  Lecture One Charlotte Wickham wickham@stat.berkeley.edu http://www.stat.berkeley.edu/~wickham/ Outline 1. Basic ideas about estimation 2. Method of Moments 3. Maximum Likelihood 4. Confidence
More informationStatistical Data Analysis 2017/18
Statistical Data Analysis 2017/18 London Postgraduate Lectures on Particle Physics; University of London MSci course PH4515 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk
More informationStatistical Inference Writeup
Statistical Inference Writeup jongman@gmail.com January 19, 2015 This is a personal writeup of Statistical Inference (Casella and Berger, 2nd ed.). The purpose of this note is to keep a log of my impressions
More informationA Very Brief Summary of Bayesian Inference, and Examples
A Very Brief Summary of Bayesian Inference, and Examples Trinity Term 009 Prof Gesine Reinert Our starting point are data x = x 1, x,, x n, which we view as realisations of random variables X 1, X,, X
More informationModule 22: Bayesian Methods Lecture 9 A: Default prior selection
Module 22: Bayesian Methods Lecture 9 A: Default prior selection Peter Hoff Departments of Statistics and Biostatistics University of Washington Outline Jeffreys prior Unit information priors Empirical
More informationSTATISTICSSTAT (STAT)
StatisticsSTAT (STAT) 1 STATISTICSSTAT (STAT) Courses STAT 158 Introduction to R Programming Credit: 1 (100) Programming using the R Project for the Statistical Computing. Data objects, for loops,
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 7 Approximate
More informationAdvanced Statistics I : Gaussian Linear Model (and beyond)
Advanced Statistics I : Gaussian Linear Model (and beyond) Aurélien Garivier CNRS / Telecom ParisTech Centrale Outline One and TwoSample Statistics Linear Gaussian Model Model Reduction and model Selection
More informationQUANTITATIVE TECHNIQUES
UNIVERSITY OF CALICUT SCHOOL OF DISTANCE EDUCATION (For B Com. IV Semester & BBA III Semester) COMPLEMENTARY COURSE QUANTITATIVE TECHNIQUES QUESTION BANK 1. The techniques which provide the decision maker
More informationThe purpose of this section is to derive the asymptotic distribution of the Pearson chisquare statistic. k (n j np j ) 2. np j.
Chapter 9 Pearson s chisquare test 9. Null hypothesis asymptotics Let X, X 2, be independent from a multinomial(, p) distribution, where p is a kvector with nonnegative entries that sum to one. That
More informationGeneralized Linear Models for NonNormal Data
Generalized Linear Models for NonNormal Data Today s Class: 3 parts of a generalized model Models for binary outcomes Complications for generalized multivariate or multilevel models SPLH 861: Lecture
More informationStatistical methods and data analysis
Statistical methods and data analysis Teacher Stefano Siboni Aim The aim of the course is to illustrate the basic mathematical tools for the analysis and modelling of experimental data, particularly concerning
More informationNext is material on matrix rank. Please see the handout
B90.330 / C.005 NOTES for Wednesday 0.APR.7 Suppose that the model is β + ε, but ε does not have the desired variance matrix. Say that ε is normal, but Var(ε) σ W. The form of W is W w 0 0 0 0 0 0 w 0
More informationBayesian Linear Models
Bayesian Linear Models Sudipto Banerjee 1 and Andrew O. Finley 2 1 Department of Forestry & Department of Geography, Michigan State University, Lansing Michigan, U.S.A. 2 Biostatistics, School of Public
More informationLinear Methods for Prediction
This work is licensed under a Creative Commons AttributionNonCommercialShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this
More informationA Course in Applied Econometrics Lecture 14: Control Functions and Related Methods. Jeff Wooldridge IRP Lectures, UW Madison, August 2008
A Course in Applied Econometrics Lecture 14: Control Functions and Related Methods Jeff Wooldridge IRP Lectures, UW Madison, August 2008 1. LinearinParameters Models: IV versus Control Functions 2. Correlated
More informationEngineering Mathematics
Thoroughly Revised and Updated Engineering Mathematics For GATE 2017 and ESE 2017 Prelims Note: ESE Mains Electrical Engineering also covered Publications Publications MADE EASY Publications Corporate
More informationStochastic Processes. Theory for Applications. Robert G. Gallager CAMBRIDGE UNIVERSITY PRESS
Stochastic Processes Theory for Applications Robert G. Gallager CAMBRIDGE UNIVERSITY PRESS Contents Preface page xv Swgg&sfzoMj ybr zmjfr%cforj owf fmdy xix Acknowledgements xxi 1 Introduction and review
More informationDensity Estimation. Seungjin Choi
Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongamro, Namgu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More informationMachine Learning 2017
Machine Learning 2017 Volker Roth Department of Mathematics & Computer Science University of Basel 21st March 2017 Volker Roth (University of Basel) Machine Learning 2017 21st March 2017 1 / 41 Section
More informationSTA216: Generalized Linear Models. Lecture 1. Review and Introduction
STA216: Generalized Linear Models Lecture 1. Review and Introduction Let y 1,..., y n denote n independent observations on a response Treat y i as a realization of a random variable Y i In the general
More informationBayesian Linear Models
Bayesian Linear Models Sudipto Banerjee 1 and Andrew O. Finley 2 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department of Forestry & Department
More informationIntroduction to Graphical Models
Introduction to Graphical Models The 15 th Winter School of Statistical Physics POSCO International Center & POSTECH, Pohang 2018. 1. 9 (Tue.) YungKyun Noh GENERALIZATION FOR PREDICTION 2 Probabilistic
More informationNoninformative Priors for the Ratio of the Scale Parameters in the Inverted Exponential Distributions
Communications for Statistical Applications and Methods 03, Vol. 0, No. 5, 387 394 DOI: http://dx.doi.org/0.535/csam.03.0.5.387 Noninformative Priors for the Ratio of the Scale Parameters in the Inverted
More informationMaximum Likelihood Estimation; Robust Maximum Likelihood; Missing Data with Maximum Likelihood
Maximum Likelihood Estimation; Robust Maximum Likelihood; Missing Data with Maximum Likelihood PRE 906: Structural Equation Modeling Lecture #3 February 4, 2015 PRE 906, SEM: Estimation Today s Class An
More informationEstimation of Operational Risk Capital Charge under Parameter Uncertainty
Estimation of Operational Risk Capital Charge under Parameter Uncertainty Pavel V. Shevchenko Principal Research Scientist, CSIRO Mathematical and Information Sciences, Sydney, Locked Bag 17, North Ryde,
More informationCOMS 4771 Regression. Nakul Verma
COMS 4771 Regression Nakul Verma Last time Support Vector Machines Maximum Margin formulation Constrained Optimization Lagrange Duality Theory Convex Optimization SVM dual and Interpretation How get the
More informationNeutral Bayesian reference models for incidence rates of (rare) clinical events
Neutral Bayesian reference models for incidence rates of (rare) clinical events Jouni Kerman Statistical Methodology, Novartis Pharma AG, Basel BAYES2012, May 10, Aachen Outline Motivation why reference
More informationLinear Regression Models. Based on Chapter 3 of Hastie, Tibshirani and Friedman
Linear Regression Models Based on Chapter 3 of Hastie, ibshirani and Friedman Linear Regression Models Here the X s might be: p f ( X = " + " 0 j= 1 X j Raw predictor variables (continuous or codedcategorical
More informationBayesian methods in economics and finance
1/26 Bayesian methods in economics and finance Linear regression: Bayesian model selection and sparsity priors Linear Regression 2/26 Linear regression Model for relationship between (several) independent
More informationHypothesis Testing. Econ 690. Purdue University. Justin L. Tobias (Purdue) Testing 1 / 33
Hypothesis Testing Econ 690 Purdue University Justin L. Tobias (Purdue) Testing 1 / 33 Outline 1 Basic Testing Framework 2 Testing with HPD intervals 3 Example 4 Savage Dickey Density Ratio 5 Bartlett
More informationFundamentals of Probability Theory and Mathematical Statistics
Fundamentals of Probability Theory and Mathematical Statistics Gerry Del Fiacco Math Center Metropolitan State University St. Paul, Minnesota June 6, 2016 1 Preface This collection of material was researched,
More informationσ(a) = a N (x; 0, 1 2 ) dx. σ(a) = Φ(a) =
Until now we have always worked with likelihoods and prior distributions that were conjugate to each other, allowing the computation of the posterior distribution to be done in closed form. Unfortunately,
More informationBayesian Inference: Posterior Intervals
Bayesian Inference: Posterior Intervals Simple values like the posterior mean E[θ X] and posterior variance var[θ X] can be useful in learning about θ. Quantiles of π(θ X) (especially the posterior median)
More informationLikelihood and Bayesian Inference for Proportions
Likelihood and Bayesian Inference for Proportions September 9, 2009 Readings Hoff Chapter 3 Likelihood and Bayesian Inferencefor Proportions p.1/21 Giardia In a New Zealand research program on human health
More informationMeasuring nuisance parameter effects in Bayesian inference
Measuring nuisance parameter effects in Bayesian inference Alastair Young Imperial College London WHOAPSI2017 1 / 31 Acknowledgements: Tom DiCiccio, Cornell University; Daniel Garcia Rasines, Imperial
More informationChapter 6: Functions of Random Variables
Chapter 6: Functions of Random Variables We are often interested in a function of one or several random variables, U(Y 1,..., Y n ). We will study three methods for determining the distribution of a function
More informationSTAT 135 Lab 6 Duality of Hypothesis Testing and Confidence Intervals, GLRT, Pearson χ 2 Tests and QQ plots. March 8, 2015
STAT 135 Lab 6 Duality of Hypothesis Testing and Confidence Intervals, GLRT, Pearson χ 2 Tests and QQ plots March 8, 2015 The duality between CI and hypothesis testing The duality between CI and hypothesis
More informationMultinomial Data. f(y θ) θ y i. where θ i is the probability that a given trial results in category i, i = 1,..., k. The parameter space is
Multinomial Data The multinomial distribution is a generalization of the binomial for the situation in which each trial results in one and only one of several categories, as opposed to just two, as in
More informationIntroduction: MLE, MAP, Bayesian reasoning (28/8/13)
STA561: Probabilistic machine learning Introduction: MLE, MAP, Bayesian reasoning (28/8/13) Lecturer: Barbara Engelhardt Scribes: K. Ulrich, J. Subramanian, N. Raval, J. O Hollaren 1 Classifiers In this
More informationProbability Theory for Machine Learning. Chris Cremer September 2015
Probability Theory for Machine Learning Chris Cremer September 2015 Outline Motivation Probability Definitions and Rules Probability Distributions MLE for Gaussian Parameter Estimation MLE and Least Squares
More informationGlossary for the Triola Statistics Series
Glossary for the Triola Statistics Series Absolute deviation The measure of variation equal to the sum of the deviations of each value from the mean, divided by the number of values Acceptance sampling
More informationLecture Notes 15 Prediction Chapters 13, 22, 20.4.
Lecture Notes 15 Prediction Chapters 13, 22, 20.4. 1 Introduction Prediction is covered in detail in 36707, 36701, 36715, 10/36702. Here, we will just give an introduction. We observe training data
More informationA Course in Applied Econometrics Lecture 18: Missing Data. Jeff Wooldridge IRP Lectures, UW Madison, August Linear model with IVs: y i x i u i,
A Course in Applied Econometrics Lecture 18: Missing Data Jeff Wooldridge IRP Lectures, UW Madison, August 2008 1. When Can Missing Data be Ignored? 2. Inverse Probability Weighting 3. Imputation 4. HeckmanType
More informationChapter 1. Probability, Random Variables and Expectations. 1.1 Axiomatic Probability
Chapter 1 Probability, Random Variables and Expectations Note: The primary reference for these notes is Mittelhammer (1999. Other treatments of probability theory include Gallant (1997, Casella & Berger
More informationMore Spectral Clustering and an Introduction to Conjugacy
CS8B/Stat4B: Advanced Topics in Learning & Decision Making More Spectral Clustering and an Introduction to Conjugacy Lecturer: Michael I. Jordan Scribe: Marco Barreno Monday, April 5, 004. Back to spectral
More informationOverall Objective Priors
Overall Objective Priors Jim Berger, Jose Bernardo and Dongchu Sun Duke University, University of Valencia and University of Missouri Recent advances in statistical inference: theory and case studies University
More informationWhat s New in Econometrics. Lecture 13
What s New in Econometrics Lecture 13 Weak Instruments and Many Instruments Guido Imbens NBER Summer Institute, 2007 Outline 1. Introduction 2. Motivation 3. Weak Instruments 4. Many Weak) Instruments
More informationEconomics 671: Applied Econometrics Department of Economics, Finance and Legal Studies University of Alabama
Problem Set #1 (Random Data Generation) 1. Generate =500random numbers from both the uniform 1 ( [0 1], uniformbetween zero and one) and exponential exp ( ) (set =2and let [0 1]) distributions. Plot the
More informationGaussian Models (9/9/13)
STA561: Probabilistic machine learning Gaussian Models (9/9/13) Lecturer: Barbara Engelhardt Scribes: Xi He, Jiangwei Pan, Ali Razeen, Animesh Srivastava 1 Multivariate Normal Distribution The multivariate
More informationInverse Sampling for McNemar s Test
International Journal of Statistics and Probability; Vol. 6, No. 1; January 27 ISSN 19277032 EISSN 19277040 Published by Canadian Center of Science and Education Inverse Sampling for McNemar s Test
More information