INDEPENDENCE AND UNCORRELATION

Transcription

1 Julio L Peixoto - Setember 999 INDEPENDENCE AND UNCORRELATION (Some radom commets motivated b class discussio) Statistical Ideedece Radom variables,, are said to be (statisticall) ideedet if kowlede about oe or several of them does ot affect the robabilit distributio of the others The tical model is that of tossi a coi reeatedl The results of oe or several tosses does ot affect the robabilities of results i the other tosses The coi has o memor; it does ot kow what haeed before Assumi their robabilit desit fuctios (df's) exist,,, are ideedet if ad ol if the joit df is idetical to the roduct of the corresodi marial df's (B "idetical" we mea "equal for all values these radom variables ca take") 3 More formall, usi fairl stadard otatio, ad assumi the df's exist,,, are ideedet if ad ol if f,,, (,,, ) ( ) ( ) ( u u u = f u f u f u for all u, u,, u, where u i deotes a value i ca take I the case of discrete radom variables, the desities become robabilities ad it is coveiet to chae the f's to 's, as I'll do i Examles ad below Aother iteresti characterizatio is this oe:,, are ideedet if ad ol if E[ ( ) ( ) ( )] = E[ ( )] E[ ( )] E[ ) ( for all fuctios (!), (!),, (!) (Actuall, it should sa for all measurable fuctios, but I do't wat et ito that te of detail I could't use f's for the fuctios because I had alread used f's for the df's Hece the 's) Ucorrelatio 5 Ucorrelatio is sometimes cofused with ideedece but it is a differet cocet Correlatio is a measure of liear deedece, as we shall see o this course 6 Review of basic cocets: Variace of : ) E [ " E( ) ] of { } = E( ) [ E( ) ] Var ( = ", a measure of the variabilit Covariace betwee ad : Cov (, ) = {[ " E ( )][ " E ( )]} = E ( ) " E ( ) E ( ), a measure of the liear covariatio betwee ad Correlatio betwee ad : r, = Cov Var (, ) ( ) Var( ) )], a measure of the streth of the liear relatioshi betwee ad A correlatio is alwas betwee " ad

2 JLP, Ideedece ad ucorrelatio Pae Relatioshi betwee Ideedece ad Ucorrelatio 7 B commets ad 6 above, ad are ideedet if ad ol if a fuctio of is ucorrelated with a fuctio of For examle, ideedece of ad requires ot ol the ucorrelatio betwee ad but also the ucorrelatio betwee ad, betwee lo( ) ad e, betwee / ad si( ),, betwee a fuctio of ad a fuctio of Clearl, ideedece imlies ucorrelatio but the coverse is ot true Ideedece is a much stroer requiremet 8 However, ucorrelatio ad ideedece are equivalet if we have ormal distributios More recisel, if two variables are joitl ormall distributed (meai that a liear combiatio of the two variables is ormall distributed), the these two variables are ideedet if ad ol if the are ucorrelated This is a excetioal roert of the ormal distributio which caot be exteded to other distributios Examle : Ucorrelated but ot Ideedet 9 Assume we have the four data oits i the followi rah, each with the same robabilit, 05 Obviousl, ad are ot ideedet sice = The joit robabilit distributio of ad is ive o the followi table " " Note that the joit (iside) robabilities are ot the roducts of the marial robabilities, a si of deedece (Read commets ad 3) Aother si of deedece: If we kow that is ", the the robabilit that is is, ie, P ( = = " ) = ; if we kow that is ", the the robabilit that is

3 JLP, Ideedece ad ucorrelatio Pae 3 is 0, ie, P ( = = " ) = 0 Hece, kowlede of affects the robabilit distributio of We have i fact comlete deedece here; if we kow, we kow with comlete certait 3 Calculatio of some exectatios Recall that E [ ( ) ( )] = # ( ) ( ), (, ) all, (, ) (,,!! =,!!, (, where f ) is the joit robabilit distributio fuctio of ad To save sace, I use to deote ) i the followi table Are ad ucorrelated? Cov (, ) = E( E( ) E( ) = 000 " (000)(50) = 0 Yes, the are ucorrelated (If we fit a least-squares lie o the above rah, we et a horizotal lie) Hece, ad are ucorrelated eve if the are ot ideedet 5 Are ad ucorrelated?, = E " E E = 850 " (50)(50) = ( ) ( ) ( ) ( ) 5 Cov No, the are correlated (ie, ot ucorrelated) (I fact, the correlatio betwee ad is sice = ) Sice ad are ot ideedet, there exist fuctios of ad that are correlated (ie, ot ucorrelated) See commet above 6 Are ad ( ) l ucorrelated?, l( ) = E( l E( ) E l( ) = 378 " (50)(93) = ( ) ( ) ( ) Cov No, the are correlated (ie, ot ucorrelated) Read revious commet l ( ) l ( ) " 05 " " " 05 "05 05 "

4 JLP, Ideedece ad ucorrelatio Pae Examle : Ideedet (ad hece Ucorrelated) 7 Assume we have the four data oits i the followi rah, each with the same robabilit, 05-8 The joit robabilit distributio of ad is ive o the followi table " Note that the joit (iside) robabilities are the roducts of the marial robabilities Hece, ad are ideedet (Read commets ad 3) 0 Calculatio of some exectatios Are ad ucorrelated? Cov (, ) = E( E( ) E( ) = 000 " (000)(50) = 0 Yes, the are ucorrelated Are ad ucorrelated? Cov (, ) = E( E( ) E( ) = 50 " (00)(50) = 0 00 Yes, the are ucorrelated l ( ) l ( ) " 05 "05 00 " " 05 "05 05 "

5 JLP, Ideedece ad ucorrelatio Pae 5 3 Are ad ( ) l ucorrelated?, l( ) = E( l E( ) E l( ) = 93 " (00)(93) = ( ) ( ) Cov Yes, the are ucorrelated ( ) Sice ad are ideedet, a fuctio of is ucorrelated with a fuctio of Read commet above Fial Remarks 5 I hoe this has heled to clear the cofusio betwee ideedece ad ucorrelatio 6 Ideedece meas that kowlede of oe variable does ot tell us athi about the robabilit distributio of the others Thik of tossi cois, rolli dice, lai the lotter - all rocesses with o memor, ie, ideedet 7 Correlatio is a measure of the streth of the liear relatioshi Ucorrelatio (ie, zero correlatio, zero covariace) meas that the least-squares lie is horizotal 8 Ideedece imlies ucorrelatio 9 The coverse is ot true, as Examle shows Ucorrelatio does ot iml ideedece 30 However, i the case of ormal variables, ucorrelatio does iml ideedece More recisel, two joitl ormal variables are ideedet if ad ol if the are ucorrelated 3 The followi Ve diaram ma hel ou uderstad what is ossible ad what is ot: joitl ormal ideedet ucorrelated The above diaram defies 8 classes of airs of radom variables However, classes 3, 5, ad 7 are emt

6 JLP, Ideedece ad ucorrelatio Pae 6 It is ot ossible for two radom variables to be ideedet but correlated (Classes 3 ad 5 are emt) It is ot ossible for two radom variables to be joitl ormal, ucorrelated, but ot ideedet (Class 7 is emt) However, it is ossible for two radom variables to be ucorrelated ad ot ideedet - as lo as the are ot joitl ormal (Class is ot emt, as Examle illustrates) Here is the same diaram aai but with the emt classes shaded joitl ormal ideedet ucorrelated Athi that is ot shaded is ossible