Secto Notes Elzabeth Stoe ad Charles Wag Jauar 5, 9 Jot, Margal, ad Codtoal Probablt Useful Rules/Propertes. P ( x) P P ( x; ) or R f (x; ) d. P ( xj ) P (x; ) P ( ) 3. P ( x; ) P ( xj ) P ( ) 4. Baes Rule: P ( xj ) P ( P jx)p (x) P ( ;x) Expectato ad Codtoal Expectato of a Radom Varable. Expectato e to E () P xp(x) R xf (x)dx f dscrete RV f cotuous RV. E (a + b) ae () + b for a,b costat. E ( + ) E () + E ( ) 3. E (h() + g( )) E (h()) + E (g( )) for arbtrar fuctos g ad h. Codtoal Expectato e to E (j ) P Useful Rules/Propertes. fucto of R xp(xj ) f dscrete RV xfj (x)dx f cotuous RV. Take out what s beg codtoed: E ( j) E ( j) 3. Law of terated expectatos: E () E [E (j )]
Proof. E () xf x dx x f ; d dx () x x f j f d dx (3) x xf j f ddx (4) x xf j f dxd (5) x B xf j dxc @ x f d (6) {z } E(j ) E (E (j )) (7) 4. Usefuless of the Law of Iterated Expectatos B Proposto 3 If E ("j) the Cov ("; f()) for a arbtrar fucto g Proof. WTS: E ("f()) E ("f ()) E [E ("f()j)] E 4f()E ("j) 5 {z } Note: So, f we kow E ("j), we ca easl show that ad " are ucorrelated. (Use the dett fucto) 3 3 Covarace, Varace, ad Correlato 3. Covarace e to 4 (Uvarate) Cov (; ) E [( E ()) ( E ( ))] E ( ) E () E ( ) deoted b x h (Vectors): Cov (; ) E ( E ()) ( E ()) T E T E () E () T deoted b. Cov (a + ; ) Cov (; ) for a costat. Cov (a; b ) abcov (; ) 3. Cov ( + ; ) Cov (; ) + Cov (; ) 4. Cov (aw + b; c + d) accov (W; ) + adcov (W; ) + bccov(; ) + bdcov (; ) 5. V ar () Cov(; ) 6. If? ) Cov (; )
3. Varace h e to 5 (Uvarate) V ar () E ( E ()) E E () deoted b x h (Vectors): Cov (; ) E ( E ()) ( E ()) T E T E () E () T deoted b. V ar ( + ) Cov ( + ; + ) V ar () + V ar( ) + Cov(; ). V ar 3. Multvarate Example: Let W :The, V ar () E h( E ()) ( E ()) T h E (W EW ) E [(W EW ) ( E)] 4 E [(W EW ) ( E)] E h( E) W W W 3 5 3.3 Correlato e to 6 Corr (; ) Cov(; ) p V ar()v ar( ) deoted b x. x [ ; ] Proof. WTS: same as showg Cov(; ) p ad V ar()v ar( ) Cov(; ) p V ar()v ar( ) Cov(; ) p, or Cov (; ) p V ar () V ar ( ) V ar()v ar( ) Proof follows from Cauch-Schwartz equalt.. x or a + b (.e. ad are lear trasformatos of each other) 4 (Weak) Law of Large Numbers ad Cetral Lmt Theorem Proposto 7 (LLN, Uvarate) Let f ; :::; g be a sequece of depedetl ad detcall dstrbuted (d) radom varables wth E ( ) ad V ar ( ) : The, P z! P (LLN, Vector) Let f ; :::; g be a sequece of depedetl ad detcall dstrbuted (d) radom vectors wth E ( ) ad V ar ( ) E T E E T The, P! P Proof. Oe wa to show that a radom varable coverges probablt to a costat s to show that Var of the radom varable coverges to ad the bas goes to Use Chebchev Idequalt: P h E ( ) V ar()+(e[ ]) " " " for a arbtrar " E, ad V ar!! so that P "!! Proposto 8 (CLT,Uvarate) Let f ; :::; g be d wth E ( ) ad V ar ( ) :The, p P z! N ; (or p! N ; ) (CLT,Vector) Let f ; :::; g be d wth E ( ) ad V ar ( ) E P T p! N (; ) (or p! N (; ) ) E E T :The, 3
5 Normal ad Bvarate Normal Radom Varables e to 9 bvarate ormal radom varable s a x vector of radom varables, each beg a ormal radom varable. e.g. If N ; ; N ; ;the N (! ; ) where! ad. Lear trasformatos of ormals are ormall dstrbuted: f ~N ;, the for a; b costats, a + b~n a + b; a. How to d the mea / varace of lear trasformatos of bvarate ormals: If N (! ; ), the for x (matrx of costats) ad b x (vector of costats), + b N ( + b; ) 3. Codtoal dstrbuto: If N ;, the j x N B + (x ) ; @ {z C } 4. ddg/subtractg ormal radom varables results a ormal radom varable: Let N ; ; N ; the + N + ; + + 6 Ch-Squared strbuto e to Let ; ; :::; p be d N (; ), the + + ::: + p p Proposto Suppose W k N k ; V. The T (W ) T V (W ) (k). Clearl,. Pcture 3. Wh s t useful regressos? llows us to test hpothess o multple parameters at the same tme (e.g. Suppose ou have a regresso model Icome + Race+ Geder+" for whch ou have estmates ^ ; ^ ; ^ for the populato parameters. Suppose ou wat to test the hpothess that race N geder have o e ect o come,.e. H : ad ) > USE WL STTISTIC Wald Statstc: We saw class that uder the ull hpothess that true b for some hpotheszed, the Wald Statstc p T ^ b V p ^ b whch s dstrbuted (p). Note: p ^ b s ormall dstrbuted wth mea ad varace V uder the ull. 4
Example to show tuto behd wh Wald statstc s ch-sq dstrbuted: Let ~N (; ) ad N (; ) ad ; depedet. Let s wrte ths matrx otato: The, ~N B @ {z} b ; {z } V I C T ( b) T V ( b) T + () sce, stadard ormal ad depedet of each other. other example: Suppose ow ~N ; ad ~N ; ad ; depedet " # Note that here V so that V The T ( b) T V ( b) " + B C @ {z} N(;) + B C @ {z} N(;) # () 5
sce ; stadard ormal ad depedet of each other. What we do the regresso settg s essetall a more geeralzed verso of ths. (o t eed to kow): T ( b) T V ( b) ( b) T C T C ( b) b Cholesk decomposto of V [C ( b)] T [C ( b)] The matrx C essetall re-weghts the coordates such a wa that elemets of the ew vector C ( b) are stadard ormals ad are depedet of each other, ad therefore the er dot product s just a sum of depedet stadard ormal varables, whch s ch-square dstrbuted. 7 Extra Stu (o t Need to kow) 7. Trlog of Theorems (To help us gure out the lmtg dstrbuto for more complcated sequeces of RVs). Slutsk s Theorem: If! ad! P a; B! P b for a, b o-radom costats, the + B! a + b. Cotuous Mappg Theorem: If! P ad g s a cotuous fucto, the g ( )! P g ( ) 3. elta Method (To be covered later) example usg the above: Showg cosstec of the OLS coe cet: Suppose data s geerated accordg to a + bx + " where E ("j) ( ) Ex " ) ) We have a d sample of datapots, geerated accordg to ths process. Questo: What s the probablt lmt of the OLS coe cet ad the lmtg dstrbuto of the OLS coe cet? ^ ( ) ( ) ( + E) + ( ) E + x x! B LLN, P x x! P E (x x P ) B Cotuous mappg, x x!p [E (x x )] B LLN, P x "! P E (x " ) So, b Slutsk s, ^ P + x x P x "!P! x " 6