Simple Linear Regression

Save this PDF as:

Size: px
Start display at page:

Transcription

2 The followig example shows the calculatios as they would be carried out by had, i gruesome detail. eg: To study the effect of ozoe pollutio o soybea yield, data were collected at four ozoe dose levels ad the resultig soybea seed yield moitored. Ozoe dose levels (i ppm)were reported as the average ozoe cocetratio durig the growig seaso. Soybea yield was reported i grams per plat. X Y Ozoe(ppm) Yield (gm/plat) Estimated values for β 0 ad β 1 are ow computed from the data X Y X 2 Y 2 XY Colum sums: x i =.35, y i = 911, x 2 i =.0399, y 2 i = 208, 495, ad x i y i = Meas: x =.0875 ad ȳ = Itermediate terms: = i (x i x) 2 = i x 2 i ( x i) 2 =.0399 (.35)2 4 = SS xy = i (x i x)(y i ȳ) = i x i y i ( x i)( y i ) = (911) 4 = ˆβ 1 = SS xy = , ˆβ 0 = ȳ ˆβ 1 x = ( )(.0875) = (h) the least squares regressio equatio which characterizes the liear relatioship betwee soybea yield ad ozoe dose is ŷ i = x i (i) The error variace, σ 2, is estimated as MSE. (j) Residuals: ˆɛ i = y i ŷ i = y i ( ˆβ 0 + ˆβ 1 x i ) x i y i ŷ i ˆɛ i = y i ŷ i

3 (k) Residual Sum of Squares (I regressio problems, the error sum of squares is also kow as the residual sum of squares). (l) Mea Squared Error: MSE = SSE = ˆɛ 2 i = ( 5.563) 2 + (4.113) 2 + (9.854) 2 + ( 8.404) 2 = SSE ( 2) =

4 x=c(.02,.07,.11,.15) y=c(242,237,231,201) SXX=sum((x-mea(x))^2) SXY=sum((x-mea(x))*(y-mea(y))) SYY=sum((y-mea(y))^2) b1=sxy/sxx b0=mea(y)-b1*mea(x) yp=b0+b1*x resids=y-yp SSE=sum(resids^2) SST=SYY SSR=SST-SSE SS=c(SSR,SSE,SST) =legth(y) df=c(1,-2,-1) MS=SS/df cbid(ss,df,ms) Calculatios by had i R SS df MS [1,] [2,] [3,]

5 Check calculatios usig builti lm, summary ad ANOVA commads i R Call: lm(formula = y ~ x) Coefficiets: (Itercept) x Call: lm(formula = y ~ x) Residuals: Coefficiets: Estimate Std. Error t value Pr(> t ) (Itercept) ** x Sigif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Residual stadard error: o 2 degrees of freedom Multiple R-squared: ,Adjusted R-squared: F-statistic: o 1 ad 2 DF, p-value: Aalysis of Variace Table Respose: y Df Sum Sq Mea Sq F value Pr(>F) x Residuals [1] [1]

7 3. Cofidece Itervals for Regressio Coefficiets (a) Cofidece itervals are costructed usig the stadard errors as follows: ˆβ i ± t α/2, 2 SE ˆβ i (b) I the example, 95% cofidece itervals for β 1 ad β 0 are computed as follows. t α/2, 2 = t.025,2 = For the slope, β 1 : ± 4.303(107.81) ( 757.4, 170.3) For the itercept, β 0 : ± 4.303(10.77) (207.1, 299.8) 95% Cofidece itervals i R upper 2.5th percetile of t-dist with -2 d.f. MSE=SSE/(-2) t=qt(.975,-2) t #upper.025'th percetile of t with -2 df. [1] %cofidece iterval for β 1 SEb1=sqrt(MSE/SXX) #stadard error of beta_1 c(b1-t*seb1,b1+t*seb1) [1]

8 Why does the cofidece iterval have the correct coverage probability? Cosider the example of the iterval for ˆβ 1. We eed the followig facts: (a) β 1 has a ormal distributio with mea β 1 ad ukow variace σ 2 /SXX. A cosequece is that Z = β 1 β 1 σ/ SXX (b) W = ( 2)MSE σ to prove.) 2 N(0, 1) (Easy results to prove.) χ 2 2, a chi-squared distributio with 2 degrees of freedom. (A bit harder (c) β 1 ad SSE are idepedet, implies Z = β 1 β 1 σ/ ( 2)MSE ad are idepedet. (Hard to SXX σ 2 prove. Details ivolve cosiderable matrix algebra, ad are cotaied i appedix C3 of Motgomery et al) (d) Defiitio: If Z is stadard ormal, idepedet of W which is χ 2 ν, the t = have a t distributio with ν degrees of freedom. (e) The see geeral otes o costructig cofidece itervals. Z W/ν is defied to 8

9 4. The correlatio betwee X ad Y is estimated by: r = A alterative expressio is give by or i=1 (y i ȳ)(x i x) i=1 (x i x) 2 i=1 (y i ȳ) 2 r = ˆβ 1 i=1 (x i x) 2 i=1 (y i ȳ) 2 r = ˆβ 1 SSxx SSyy where = i=1 (x i x) 2 ad SS yy = i=1 (y i ȳ) 2 are the sums of squares of the X s ad Y s, respectively. Note that SS yy = SST, the total sum of squares. Note that stadard deviatios of the X s ad the Y s. The correlatio coefficiet lies i the iterval [-1,+1]. SSxx SSyy = s x s y, the ratio of the If the relatioship bewee Y ad X is perfectly liear ad icreasig, the correlatio will be +1. If the relatioship is perfectly liear ad decreasig, the correlatio will be +1. If there is o liear relatioship betwee X ad Y, the correlatio is 0. I the example, r = ˆβ SSxx 1 = =.887 SSyy

10 5. Goodess of fit of the regressio lie is measured by the coefficiet of determiatio, R 2. For simple liear regressio R 2 = r 2. R 2 = SSR SST The Regressio Sum of Squares (SSR) is similar to the Treatmet Sum of Squares i a ANOVA problem. It is give by SSR = SS2 xy. Alterative ways of calculatig the residual sum of squares are to use the additivity relatioship (SSR + SSE = SST), or to use oe of the followig formulas. R 2 = SSR/SST 1 R 2 = (SST SSR)/SST = SSE/SST SSE = (1 R 2 )SST R 2 is the fractio of the total variability i y accouted for by the liear regressio lie, ad rages betwee 0 ad 1. R 2 = 1.00 idicates a perfect liear fit, while R 2 = 0.00 is a complete liear o-fit. I the example: SSR = SS2 xy = ( ) 2 / = SST = SSR + SSE = = R 2 = SSR/SST = Note that R 2 = r 2, the square of the correlatio coefficiet. 78.8% of the variability i Y is accouted for by the regressio model. [1] [1] [1]

11 6. Estimatig the mea of Y (a) The estimated mea of Y whe x = x is ˆµ x = ˆβ 0 + ˆβ 1 x. (b) (c) The stadard error of ˆµ x is ( ˆµ x = ˆβ 0 + ˆβ 1 x N (β 0 + β 1 x 1, σ 2 + (x x) 2 )) SE ˆµx = ( 1 MSE + (x x) 2 ) (d) A cofidece iterval for the mea µ x = β 0 + β 1 x whe x = x is give by ˆµ x ± t α/2, 2 SE ˆµx (e) eg. A 95% cofidece iterval for the mea at x = 0.10 is: Whe x = 0.10, the estimated mea is ˆµ.1 = (0.1) = ( ) SE ˆµ.1 = ( ) = 5.36 t α/2, 2 = t.025,2 = margi of error = 4.303(5.36) = ± (201, ) 95% cofidece iterval for mu at x0=.10 x0=.10 muhat=b0+b1*x0 # estimate of mea at x=x0 muhat SEmu=sqrt(MSE)*sqrt(1/+(x0-mea(x))^2/SXX) #SE of muhat SEmu c(muhat-t*semu, muhat+t*semu) [1] [1] [1]

12 7. Predictig a New Respose Value We are ow iterestig i predictig the value of y at a future value x = x. I makig a predictio iterval for a future observatio o y whe x = x, we eed to icorporate two sources of variatio which accout for the fact that we are replacig the ukow mea by the estimate ˆβ 0 + ˆβ 1 x, ad we are replacig the ukow stadard deviatio σ by the estimate MSE. y ( ˆβ 0 + ˆβ 1 x ) = (y (β 0 + β 1 x )) ( ˆβ 0 + ˆβ 1 x (β 0 + β 1 x )) The first term i brackets o the right had side of this expressio has a N(0, σ 2 ) distributio. From (b) above, the distributio of the secod term is ( 1 N (0, σ 2 + (x x) 2 )) As y represets a future observatio, the distributios of the two terms are idepedet, ad it follows that the distributio of y ( ˆβ 0 + ˆβ 1 x ) is N (0, σ ( (x x) 2 )) (a) The predicted value of y is give by ŷ = ˆβ 0 + ˆβ 1 x (b) The variace of the above distributio is estimated by: ( MSE (x x) 2 ) (c) ad the predictio iterval for y is give by ( ˆβ 0 + ˆβ 1 x ± t α/2, 2 MSE (x x) 2 ) (d) eg. A 95% predictio iterval for y whe x = 0.10 is: For x = 0.10, y = (0.1) = ) SE y = ( ( ) = t α/2, 2 = t.025,2 = margi of error = 4.303(11.69) = ± (173.79, ) SEmu=sqrt(MSE)*sqrt(1+1/+(x0-mea(x))^2/SXX) c(muhat-t*semu, muhat+t*semu) 95% predictio iterval for a ew observatio at x0=.10 12

13 [1]

Linear Regression Models

Liear Regressio Models Dr. Joh Mellor-Crummey Departmet of Computer Sciece Rice Uiversity johmc@cs.rice.edu COMP 528 Lecture 9 15 February 2005 Goals for Today Uderstad how to Use scatter diagrams to ispect

1 Inferential Methods for Correlation and Regression Analysis

1 Iferetial Methods for Correlatio ad Regressio Aalysis I the chapter o Correlatio ad Regressio Aalysis tools for describig bivariate cotiuous data were itroduced. The sample Pearso Correlatio Coefficiet

Simple Regression. Acknowledgement. These slides are based on presentations created and copyrighted by Prof. Daniel Menasce (GMU) CS 700

Simple Regressio CS 7 Ackowledgemet These slides are based o presetatios created ad copyrighted by Prof. Daiel Measce (GMU) Basics Purpose of regressio aalysis: predict the value of a depedet or respose

Properties and Hypothesis Testing

Chapter 3 Properties ad Hypothesis Testig 3.1 Types of data The regressio techiques developed i previous chapters ca be applied to three differet kids of data. 1. Cross-sectioal data. 2. Time series data.

Assessment and Modeling of Forests. FR 4218 Spring Assignment 1 Solutions

Assessmet ad Modelig of Forests FR 48 Sprig Assigmet Solutios. The first part of the questio asked that you calculate the average, stadard deviatio, coefficiet of variatio, ad 9% cofidece iterval of the

SIMPLE LINEAR REGRESSION AND CORRELATION ANALYSIS

SIMPLE LINEAR REGRESSION AND CORRELATION ANALSIS INTRODUCTION There are lot of statistical ivestigatio to kow whether there is a relatioship amog variables Two aalyses: (1) regressio aalysis; () correlatio

3/3/2014. CDS M Phil Econometrics. Types of Relationships. Types of Relationships. Types of Relationships. Vijayamohanan Pillai N.

3/3/04 CDS M Phil Old Least Squares (OLS) Vijayamohaa Pillai N CDS M Phil Vijayamoha CDS M Phil Vijayamoha Types of Relatioships Oly oe idepedet variable, Relatioship betwee ad is Liear relatioships Curviliear

Section 14. Simple linear regression.

Sectio 14 Simple liear regressio. Let us look at the cigarette dataset from [1] (available to dowload from joural s website) ad []. The cigarette dataset cotais measuremets of tar, icotie, weight ad carbo

(all terms are scalars).the minimization is clearer in sum notation:

7 Multiple liear regressio: with predictors) Depedet data set: y i i = 1, oe predictad, predictors x i,k i = 1,, k = 1, ' The forecast equatio is ŷ i = b + Use matrix otatio: k =1 b k x ik Y = y 1 y 1

MOST PEOPLE WOULD RATHER LIVE WITH A PROBLEM THEY CAN'T SOLVE, THAN ACCEPT A SOLUTION THEY CAN'T UNDERSTAND.

XI-1 (1074) MOST PEOPLE WOULD RATHER LIVE WITH A PROBLEM THEY CAN'T SOLVE, THAN ACCEPT A SOLUTION THEY CAN'T UNDERSTAND. R. E. D. WOOLSEY AND H. S. SWANSON XI-2 (1075) STATISTICAL DECISION MAKING Advaced

Statistical Intervals for a Single Sample

3/5/06 Applied Statistics ad Probability for Egieers Sixth Editio Douglas C. Motgomery George C. Ruger Chapter 8 Statistical Itervals for a Sigle Sample 8 CHAPTER OUTLINE 8- Cofidece Iterval o the Mea

Statistics 20: Final Exam Solutions Summer Session 2007

1. 20 poits Testig for Diabetes. Statistics 20: Fial Exam Solutios Summer Sessio 2007 (a) 3 poits Give estimates for the sesitivity of Test I ad of Test II. Solutio: 156 patiets out of total 223 patiets

MA 575, Linear Models : Homework 3

MA 575, Liear Models : Homework 3 Questio 1 RSS( ˆβ 0, ˆβ 1 ) (ŷ i y i ) Problem.7 Questio.7.1 ( ˆβ 0 + ˆβ 1 x i y i ) (ȳ SXY SXY x + SXX SXX x i y i ) ((ȳ y i ) + SXY SXX (x i x)) (ȳ y i ) SXY SXX SY

Regression. Correlation vs. regression. The parameters of linear regression. Regression assumes... Random sample. Y = α + β X.

Regressio Correlatio vs. regressio Predicts Y from X Liear regressio assumes that the relatioship betwee X ad Y ca be described by a lie Regressio assumes... Radom sample Y is ormally distributed with

ECE 901 Lecture 12: Complexity Regularization and the Squared Loss

ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality

Final Examination Solutions 17/6/2010

The Islamic Uiversity of Gaza Faculty of Commerce epartmet of Ecoomics ad Political Scieces A Itroductio to Statistics Course (ECOE 30) Sprig Semester 009-00 Fial Eamiatio Solutios 7/6/00 Name: I: Istructor:

Matrix Representation of Data in Experiment

Matrix Represetatio of Data i Experimet Cosider a very simple model for resposes y ij : y ij i ij, i 1,; j 1,,..., (ote that for simplicity we are assumig the two () groups are of equal sample size ) Y

y ij = µ + α i + ɛ ij,

STAT 4 ANOVA -Cotrasts ad Multiple Comparisos /3/04 Plaed comparisos vs uplaed comparisos Cotrasts Cofidece Itervals Multiple Comparisos: HSD Remark Alterate form of Model I y ij = µ + α i + ɛ ij, a i

Chapter 13, Part A Analysis of Variance and Experimental Design

Slides Prepared by JOHN S. LOUCKS St. Edward s Uiversity Slide 1 Chapter 13, Part A Aalysis of Variace ad Eperimetal Desig Itroductio to Aalysis of Variace Aalysis of Variace: Testig for the Equality of

REVIEW OF SIMPLE LINEAR REGRESSION SIMPLE LINEAR REGRESSION

REVIEW OF SIMPLE LINEAR REGRESSION SIMPLE LINEAR REGRESSION I liear regreio, we coider the frequecy ditributio of oe variable (Y) at each of everal level of a ecod variable (X). Y i kow a the depedet variable.

The variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.

SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample

Confidence Level We want to estimate the true mean of a random variable X economically and with confidence.

Cofidece Iterval 700 Samples Sample Mea 03 Cofidece Level 095 Margi of Error 0037 We wat to estimate the true mea of a radom variable X ecoomically ad with cofidece True Mea μ from the Etire Populatio

II. Descriptive Statistics D. Linear Correlation and Regression. 1. Linear Correlation

II. Descriptive Statistics D. Liear Correlatio ad Regressio I this sectio Liear Correlatio Cause ad Effect Liear Regressio 1. Liear Correlatio Quatifyig Liear Correlatio The Pearso product-momet correlatio

Basis for simulation techniques

Basis for simulatio techiques M. Veeraraghava, March 7, 004 Estimatio is based o a collectio of experimetal outcomes, x, x,, x, where each experimetal outcome is a value of a radom variable. x i. Defiitios

Exam II Review. CEE 3710 November 15, /16/2017. EXAM II Friday, November 17, in class. Open book and open notes.

Exam II Review CEE 3710 November 15, 017 EXAM II Friday, November 17, i class. Ope book ad ope otes. Focus o material covered i Homeworks #5 #8, Note Packets #10 19 1 Exam II Topics **Will emphasize material

Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.

Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio

7-1. Chapter 4. Part I. Sampling Distributions and Confidence Intervals

7-1 Chapter 4 Part I. Samplig Distributios ad Cofidece Itervals 1 7- Sectio 1. Samplig Distributio 7-3 Usig Statistics Statistical Iferece: Predict ad forecast values of populatio parameters... Test hypotheses

Stat 200 -Testing Summary Page 1

Stat 00 -Testig Summary Page 1 Mathematicias are like Frechme; whatever you say to them, they traslate it ito their ow laguage ad forthwith it is somethig etirely differet Goethe 1 Large Sample Cofidece

Chapter 4 - Summarizing Numerical Data

Chapter 4 - Summarizig Numerical Data 15.075 Cythia Rudi Here are some ways we ca summarize data umerically. Sample Mea: i=1 x i x :=. Note: i this class we will work with both the populatio mea µ ad the

Lecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting

Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS

MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak

Probability and statistics: basic terms

Probability ad statistics: basic terms M. Veeraraghava August 203 A radom variable is a rule that assigs a umerical value to each possible outcome of a experimet. Outcomes of a experimet form the sample

Correlation and Covariance

Correlatio ad Covariace Tom Ilveto FREC 9 What is Next? Correlatio ad Regressio Regressio We specify a depedet variable as a liear fuctio of oe or more idepedet variables, based o co-variace Regressio

Chapter 1 (Definitions)

FINAL EXAM REVIEW Chapter 1 (Defiitios) Qualitative: Nomial: Ordial: Quatitative: Ordial: Iterval: Ratio: Observatioal Study: Desiged Experimet: Samplig: Cluster: Stratified: Systematic: Coveiece: Simple

Confidence Intervals for the Population Proportion p

Cofidece Itervals for the Populatio Proportio p The cocept of cofidece itervals for the populatio proportio p is the same as the oe for, the samplig distributio of the mea, x. The structure is idetical:

This is an introductory course in Analysis of Variance and Design of Experiments.

1 Notes for M 384E, Wedesday, Jauary 21, 2009 (Please ote: I will ot pass out hard-copy class otes i future classes. If there are writte class otes, they will be posted o the web by the ight before class

Statistical inference: example 1. Inferential Statistics

Statistical iferece: example 1 Iferetial Statistics POPULATION SAMPLE A clothig store chai regularly buys from a supplier large quatities of a certai piece of clothig. Each item ca be classified either

STA Learning Objectives. Population Proportions. Module 10 Comparing Two Proportions. Upon completing this module, you should be able to:

STA 2023 Module 10 Comparig Two Proportios Learig Objectives Upo completig this module, you should be able to: 1. Perform large-sample ifereces (hypothesis test ad cofidece itervals) to compare two populatio

REGRESSION (Physics 1210 Notes, Partial Modified Appendix A)

REGRESSION (Physics 0 Notes, Partial Modified Appedix A) HOW TO PERFORM A LINEAR REGRESSION Cosider the followig data poits ad their graph (Table I ad Figure ): X Y 0 3 5 3 7 4 9 5 Table : Example Data

The standard deviation of the mean

Physics 6C Fall 20 The stadard deviatio of the mea These otes provide some clarificatio o the distictio betwee the stadard deviatio ad the stadard deviatio of the mea.. The sample mea ad variace Cosider

Goodness-Of-Fit For The Generalized Exponential Distribution. Abstract

Goodess-Of-Fit For The Geeralized Expoetial Distributio By Amal S. Hassa stitute of Statistical Studies & Research Cairo Uiversity Abstract Recetly a ew distributio called geeralized expoetial or expoetiated

LINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity

LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 9 Multicolliearity Dr Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Multicolliearity diagostics A importat questio that

Chapter 1 Simple Linear Regression (part 6: matrix version)

Chapter Simple Liear Regressio (part 6: matrix versio) Overview Simple liear regressio model: respose variable Y, a sigle idepedet variable X Y β 0 + β X + ε Multiple liear regressio model: respose Y,

Introduction to Econometrics (3 rd Updated Edition) Solutions to Odd- Numbered End- of- Chapter Exercises: Chapter 4

Itroductio to Ecoometrics (3 rd Updated Editio) by James H. Stock ad Mark W. Watso Solutios to Odd- Numbered Ed- of- Chapter Exercises: Chapter 4 (This versio August 7, 204) 205 Pearso Educatio, Ic. Stock/Watso

Chapter 6 Sampling Distributions

Chapter 6 Samplig Distributios 1 I most experimets, we have more tha oe measuremet for ay give variable, each measuremet beig associated with oe radomly selected a member of a populatio. Hece we eed to

Tables and Formulas for Sullivan, Fundamentals of Statistics, 2e Pearson Education, Inc.

Table ad Formula for Sulliva, Fudametal of Statitic, e. 008 Pearo Educatio, Ic. CHAPTER Orgaizig ad Summarizig Data Relative frequecy frequecy um of all frequecie Cla midpoit: The um of coecutive lower

Zeros of Polynomials

Math 160 www.timetodare.com 4.5 4.6 Zeros of Polyomials I these sectios we will study polyomials algebraically. Most of our work will be cocered with fidig the solutios of polyomial equatios of ay degree

Summary: CORRELATION & LINEAR REGRESSION. GC. Students are advised to refer to lecture notes for the GC operations to obtain scatter diagram.

Key Cocepts: 1) Sketchig of scatter diagram The scatter diagram of bivariate (i.e. cotaiig two variables) data ca be easily obtaied usig GC. Studets are advised to refer to lecture otes for the GC operatios

STATISTICAL INFERENCE

STATISTICAL INFERENCE POPULATION AND SAMPLE Populatio = all elemets of iterest Characterized by a distributio F with some parameter θ Sample = the data X 1,..., X, selected subset of the populatio = sample

Solutions to Odd Numbered End of Chapter Exercises: Chapter 4

Itroductio to Ecoometrics (3 rd Updated Editio) by James H. Stock ad Mark W. Watso Solutios to Odd Numbered Ed of Chapter Exercises: Chapter 4 (This versio July 2, 24) Stock/Watso - Itroductio to Ecoometrics

Topic 15: Maximum Likelihood Estimation

Topic 5: Maximum Likelihood Estimatio November ad 3, 20 Itroductio The priciple of maximum likelihood is relatively straightforward. As before, we begi with a sample X (X,..., X of radom variables chose

Grant MacEwan University STAT 151 Formula Sheet Final Exam Dr. Karen Buro

Grat MacEwa Uiverity STAT 151 Formula Sheet Fial Exam Dr. Kare Buro Decriptive Statitic Sample Variace: = i=1 (x i x) 1 = Σ i=1x i (Σ i=1 x i) 1 Sample Stadard Deviatio: = Sample Variace = Media: Order

DS 100: Principles and Techniques of Data Science Date: April 13, Discussion #10

DS 00: Priciples ad Techiques of Data Sciece Date: April 3, 208 Name: Hypothesis Testig Discussio #0. Defie these terms below as they relate to hypothesis testig. a) Data Geeratio Model: Solutio: A set

CTL.SC0x Supply Chain Analytics

CTL.SC0x Supply Chai Aalytics Key Cocepts Documet V1.1 This documet cotais the Key Cocepts documets for week 6, lessos 1 ad 2 withi the SC0x course. These are meat to complemet, ot replace, the lesso videos

Y i n. i=1. = 1 [number of successes] number of successes = n

Eco 371 Problem Set # Aswer Sheet 3. I this questio, you are asked to cosider a Beroulli radom variable Y, with a success probability P ry 1 p. You are told that you have draws from this distributio ad

NCSS Statistical Software. Tolerance Intervals

Chapter 585 Itroductio This procedure calculates oe-, ad two-, sided tolerace itervals based o either a distributio-free (oparametric) method or a method based o a ormality assumptio (parametric). A two-sided

3.2 Properties of Division 3.3 Zeros of Polynomials 3.4 Complex and Rational Zeros of Polynomials

Math 60 www.timetodare.com 3. Properties of Divisio 3.3 Zeros of Polyomials 3.4 Complex ad Ratioal Zeros of Polyomials I these sectios we will study polyomials algebraically. Most of our work will be cocered

The Poisson Distribution

MATH 382 The Poisso Distributio Dr. Neal, WKU Oe of the importat distributios i probabilistic modelig is the Poisso Process X t that couts the umber of occurreces over a period of t uits of time. This

Chapter 11 Output Analysis for a Single Model. Banks, Carson, Nelson & Nicol Discrete-Event System Simulation

Chapter Output Aalysis for a Sigle Model Baks, Carso, Nelso & Nicol Discrete-Evet System Simulatio Error Estimatio If {,, } are ot statistically idepedet, the S / is a biased estimator of the true variace.

BHW #13 1/ Cooper. ENGR 323 Probabilistic Analysis Beautiful Homework # 13

BHW # /5 ENGR Probabilistic Aalysis Beautiful Homework # Three differet roads feed ito a particular freeway etrace. Suppose that durig a fixed time period, the umber of cars comig from each road oto the

6.867 Machine learning, lecture 7 (Jaakkola) 1

6.867 Machie learig, lecture 7 (Jaakkola) 1 Lecture topics: Kerel form of liear regressio Kerels, examples, costructio, properties Liear regressio ad kerels Cosider a slightly simpler model where we omit

KLMED8004 Medical statistics. Part I, autumn Estimation. We have previously learned: Population and sample. New questions

We have previously leared: KLMED8004 Medical statistics Part I, autum 00 How kow probability distributios (e.g. biomial distributio, ormal distributio) with kow populatio parameters (mea, variace) ca give

Paired Data and Linear Correlation

Paired Data ad Liear Correlatio Example. A group of calculus studets has take two quizzes. These are their scores: Studet st Quiz Score ( data) d Quiz Score ( data) 7 5 5 0 3 0 3 4 0 5 5 5 5 6 0 8 7 0

Table 12.1: Contingency table. Feature b. 1 N 11 N 12 N 1b 2 N 21 N 22 N 2b. ... a N a1 N a2 N ab

Sectio 12 Tests of idepedece ad homogeeity I this lecture we will cosider a situatio whe our observatios are classified by two differet features ad we would like to test if these features are idepedet

Unit 9 Regression and Correlation

BIOSTATS 540 - Fall 05 Regressio ad Correlatio Page of 44 Uit 9 Regressio ad Correlatio Assume that a statistical model such as a liear model is a good first start oly - Gerald va Belle Is higher blood

Topic 18: Composite Hypotheses

Toc 18: November, 211 Simple hypotheses limit us to a decisio betwee oe of two possible states of ature. This limitatio does ot allow us, uder the procedures of hypothesis testig to address the basic questio:

Topics Machine learning: lecture 2. Review: the learning problem. Hypotheses and estimation. Estimation criterion cont d. Estimation criterion

.87 Machie learig: lecture Tommi S. Jaakkola MIT CSAIL tommi@csail.mit.edu Topics The learig problem hypothesis class, estimatio algorithm loss ad estimatio criterio samplig, empirical ad epected losses

Number of fatalities X Sunday 4 Monday 6 Tuesday 2 Wednesday 0 Thursday 3 Friday 5 Saturday 8 Total 28. Day

LECTURE # 8 Mea Deviatio, Stadard Deviatio ad Variace & Coefficiet of variatio Mea Deviatio Stadard Deviatio ad Variace Coefficiet of variatio First, we will discuss it for the case of raw data, ad the

Binomial Distribution

0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 1 2 3 4 5 6 7 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Overview Example: coi tossed three times Defiitio Formula Recall that a r.v. is discrete if there are either a fiite umber of possible

arxiv: v1 [math.pr] 13 Oct 2011

A tail iequality for quadratic forms of subgaussia radom vectors Daiel Hsu, Sham M. Kakade,, ad Tog Zhag 3 arxiv:0.84v math.pr] 3 Oct 0 Microsoft Research New Eglad Departmet of Statistics, Wharto School,

It should be unbiased, or approximately unbiased. Variance of the variance estimator should be small. That is, the variance estimator is stable.

Chapter 10 Variace Estimatio 10.1 Itroductio Variace estimatio is a importat practical problem i survey samplig. Variace estimates are used i two purposes. Oe is the aalytic purpose such as costructig

STATISTICAL method is one branch of mathematical

40 INTERNATIONAL JOURNAL OF COMPUTING SCIENCE AND APPLIED MATHEMATICS, VOL 3, NO, AUGUST 07 Optimizig Forest Samplig by usig Lagrage Multipliers Suhud Wahyudi, Farida Agustii Widjajati ad Dea Oktaviati

Statistical Fundamentals and Control Charts

Statistical Fudametals ad Cotrol Charts 1. Statistical Process Cotrol Basics Chace causes of variatio uavoidable causes of variatios Assigable causes of variatio large variatios related to machies, materials,

UCLA STAT 110B Applied Statistics for Engineering and the Sciences

UCLA STAT 110B Applied Statistics for Egieerig ad the Scieces Istructor: Ivo Diov, Asst. Prof. I Statistics ad Neurology Teachig Assistats: Bria Ng, UCLA Statistics Uiversity of Califoria, Los Ageles,

Instructor: Judith Canner Spring 2010 CONFIDENCE INTERVALS How do we make inferences about the population parameters?

CONFIDENCE INTERVALS How do we make ifereces about the populatio parameters? The samplig distributio allows us to quatify the variability i sample statistics icludig how they differ from the parameter

On an Application of Bayesian Estimation

O a Applicatio of ayesia Estimatio KIYOHARU TANAKA School of Sciece ad Egieerig, Kiki Uiversity, Kowakae, Higashi-Osaka, JAPAN Email: ktaaka@ifokidaiacjp EVGENIY GRECHNIKOV Departmet of Mathematics, auma

Homework for 4/9 Due 4/16

Name: ID: Homework for 4/9 Due 4/16 1. [ 13-6] It is covetioal wisdom i military squadros that pilots ted to father more girls tha boys. Syder 1961 gathered data for military fighter pilots. The sex of

Analysis of Experimental Data

Aalysis of Experimetal Data 6544597.0479 ± 0.000005 g Quatitative Ucertaity Accuracy vs. Precisio Whe we make a measuremet i the laboratory, we eed to kow how good it is. We wat our measuremets to be both

Asymptotic Results for the Linear Regression Model

Asymptotic Results for the Liear Regressio Model C. Fli November 29, 2000 1. Asymptotic Results uder Classical Assumptios The followig results apply to the liear regressio model y = Xβ + ε, where X is

SALES AND MARKETING Department MATHEMATICS. 2nd Semester. Bivariate statistics LESSONS

SALES AND MARKETING Departmet MATHEMATICS d Semester Bivariate statistics LESSONS Olie documet: http://jff-dut-tc.weebly.com sectio DUT Maths S. IUT de Sait-Etiee Départemet TC J.F.Ferraris Math S StatVar

o <Xln <X2n <... <X n < o (1.1)

Metrika, Volume 28, 1981, page 257-262. 9 Viea. Estimatio Problems for Rectagular Distributios (Or the Taxi Problem Revisited) By J.S. Rao, Sata Barbara I ) Abstract: The problem of estimatig the ukow

Local Polynomial Regression

Local Polyomial Regressio Joh Hughes October 2, 2013 Recall that the oparametric regressio model is Y i f x i ) + ε i, where f is the regressio fuctio ad the ε i are errors such that Eε i 0. The Nadaraya-Watso

Output Analysis and Run-Length Control

IEOR E4703: Mote Carlo Simulatio Columbia Uiversity c 2017 by Marti Haugh Output Aalysis ad Ru-Legth Cotrol I these otes we describe how the Cetral Limit Theorem ca be used to costruct approximate (1 α%

4.1 Sigma Notation and Riemann Sums

0 the itegral. Sigma Notatio ad Riema Sums Oe strategy for calculatig the area of a regio is to cut the regio ito simple shapes, calculate the area of each simple shape, ad the add these smaller areas

Parameter, Statistic and Random Samples

Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,

Chapter 3 Oneway ANOVA

Chapter 3 Oeway ANOVA Page 1. Review of previous tests 3-. What is ANOVA? 3-3 3. Termiology i ANOVA 3-4 4. Uderstadig the F-distributio 3-6 Samplig distributio of the variace Cofidece itervals for the

Probability and Statistics

ICME Refresher Course: robability ad Statistics Staford Uiversity robability ad Statistics Luyag Che September 20, 2016 1 Basic robability Theory 11 robability Spaces A probability space is a triple (Ω,

The picture in figure 1.1 helps us to see that the area represents the distance traveled. Figure 1: Area represents distance travelled

1 Lecture : Area Area ad distace traveled Approximatig area by rectagles Summatio The area uder a parabola 1.1 Area ad distace Suppose we have the followig iformatio about the velocity of a particle, how

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties

General IxJ Contingency Tables

page1 Geeral x Cotigecy Tables We ow geeralize our previous results from the prospective, retrospective ad cross-sectioal studies ad the Poisso samplig case to x cotigecy tables. For such tables, the test

A NOTE ON THE TOTAL LEAST SQUARES FIT TO COPLANAR POINTS

A NOTE ON THE TOTAL LEAST SQUARES FIT TO COPLANAR POINTS STEVEN L. LEE Abstract. The Total Least Squares (TLS) fit to the poits (x,y ), =1,,, miimizes the sum of the squares of the perpedicular distaces

ESTIMATION AND PREDICTION BASED ON K-RECORD VALUES FROM NORMAL DISTRIBUTION

STATISTICA, ao LXXIII,. 4, 013 ESTIMATION AND PREDICTION BASED ON K-RECORD VALUES FROM NORMAL DISTRIBUTION Maoj Chacko Departmet of Statistics, Uiversity of Kerala, Trivadrum- 695581, Kerala, Idia M. Shy

Chapter 6 Principles of Data Reduction

Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a

Median and IQR The median is the value which divides the ordered data values in half.

STA 666 Fall 2007 Web-based Course Notes 4: Describig Distributios Numerically Numerical summaries for quatitative variables media ad iterquartile rage (IQR) 5-umber summary mea ad stadard deviatio Media

Some Basic Probability Concepts. 2.1 Experiments, Outcomes and Random Variables

Some Basic Probability Cocepts 2. Experimets, Outcomes ad Radom Variables A radom variable is a variable whose value is ukow util it is observed. The value of a radom variable results from a experimet;

R. van Zyl 1, A.J. van der Merwe 2. Quintiles International, University of the Free State

Bayesia Cotrol Charts for the Two-parameter Expoetial Distributio if the Locatio Parameter Ca Take o Ay Value Betwee Mius Iity ad Plus Iity R. va Zyl, A.J. va der Merwe 2 Quitiles Iteratioal, ruaavz@gmail.com

Joint Probability Distributions and Random Samples. Jointly Distributed Random Variables. Chapter { }

UCLA STAT A Applied Probability & Statistics for Egieers Istructor: Ivo Diov, Asst. Prof. I Statistics ad Neurology Teachig Assistat: Neda Farziia, UCLA Statistics Uiversity of Califoria, Los Ageles, Sprig

CHAPTER 5. Theory and Solution Using Matrix Techniques

A SERIES OF CLASS NOTES FOR 2005-2006 TO INTRODUCE LINEAR AND NONLINEAR PROBLEMS TO ENGINEERS, SCIENTISTS, AND APPLIED MATHEMATICIANS DE CLASS NOTES 3 A COLLECTION OF HANDOUTS ON SYSTEMS OF ORDINARY DIFFERENTIAL