Chpter 4: Dstrbutos Prerequste: Chpter 4. The Algebr of Expecttos d Vrces I ths secto we wll mke use of the followg symbols: s rdom vrble b s rdom vrble c s costt vector md s costt mtrx, d F m s costt mtrx. Now we defe the expectto of cotuous rdom vrble, such tht E( ) f ( ) d, (4.) where f( ) s the desty of the probblty dstrbuto of. Gve tht f( ) s desty fucto, t must therefore be the cse tht E( ) f ( )d. Ofte ths book, f( ) wll be tke to be orml, but ot lwys. I fct, some stces, wll be dscrete rther th cotuous. I tht cse, J E ( ) Pr( j) j (4.) j where there re J dscrete possble outcomes for. We cll E( ) the expectto opertor. Regrdless s to whether d b re orml, the followg set of theorems pply. Frst, we ote tht the expectto of costt s smply tht costt tself: The expectto of sum s equl to the sum of the expecttos: E(c) c. (4.3) E( + b) E() + E(b). (4.4) The expectto of ler combto comes two flvors; oe for premultplcto d oe for postmultplcto: E(D) DE(). (4.5) E('F) E(')F. (4.6) 36 Chpter 4
You c see from the bove two equtos tht costt mtrx c pss through the expectto opertor, whch ofte smplfes our lgebr gretly. All of these theorems wll be mportt eblg sttstcl ferece d tryg to uderstd the verge of vrous quttes. We ow defe the vrce opertor, V( ), such tht {[ E( )][ E( )] } V ( ) E. (4.7) We could ote here tht f E(), tht s f s me cetered, the vrce of smplfes to E('). Whether s me cetered or ot we lso hve the followg theorems: V( + c) V(). (4.8) Equto (4.8) shows tht the ddto (or subtrcto) of costt vector does ot modfy the vrce of the orgl rdom vector. Tht fct wll prove useful to us qute ofte the chpters to come. But ow t s tme to look t wht s rgubly the most mportt theorem of the book. At lest t s sfe to sy tht t s the most refereced equto the book: V(D) DV()D' (4.9) V('F) F'V()F (4.) Equto (4.9), tht shows tht the vrce of ler combto s qudrtc form bsed o tht ler combto, wll be extremely useful to us, g d g ths book. 4. The Norml Dstrbuto The orml dstrbuto s wdely used both sttstcl resog d modelg mrketg processes. It s so wdely used tht short-hd otto exsts to stte tht the vrble x s ormlly dstrbuted wth me μ d vrce : x ~ N(μ, ). We wll strt out by dscussg the desty fucto of the orml dstrbuto eve though the dstrbuto fucto s somewht more fudmetl (t s, fter ll, clled the orml dstrbuto) d fct the desty s derved from the dstrbuto fucto rther th vce vers. I y cse, the desty gves the probblty tht vrble tkes o prtculr vlue. We plot ths probblty s fucto of the vlue:. Pr(x) x μ x The equto tht sketches out the bell shped curve the fgure s Dstrbutos 37
(x μ) Pr( x) Pr(x x ) exp. (4.) π Most of the cto tkes plce the expoet [d here we remd you tht exp(x) e x ]. I fct, the costt π s eeded solely to mke sure tht the totl probblty uder the curve equls oe, or other words, tht the fucto tegrtes to. You mght lso ote tht the s ot uder the rdcl sg. Altertvely you c clude uder the rdcl. Whe we stdrdze such tht μ d we geerlly reme x to z d the Pr( z) z Pr(z z ) exp φ(z ). (4.) π Note tht φ( ) s very wdely used ottol coveto to refer to the stdrd orml desty fucto. Ths wll show up my plces the chpters to follow. I sttstcl resog, we re ofte terested the probblty tht orml vrble flls betwee two prtculr vlues, sy x d x b. We c pcture ths stuto s below:. Pr[x x x b ] Pr(x) x x b x We c derve the probblty by tegrtg the re uder the curve from x to x b. There s o lytc swer tht s to sy o equto wll llow you to clculte the exct vlue so the oly wy you c do t s by brute force computer progrm tht cretes seres of ty rectgles betwee x d x b. If the bses of these rectgles become suffcetly smll, eve though the top of the fucto s obvously ot flt, we c pproxmte ths probblty to rbtrry precso by ddg up the res of these rectgles. We wrte ths re usg the tegrl symbol s below: (x ) Pr[x x x ] b μ b exp dx. π We c stdrdze, usg the clculus chge-of-vrbles techque, d the move the costt uder the tegrl, ll of whch yelds the sme probblty s bove. Ths s show ext: x x b Pr[ z φ z z b ] (z) dz. z z 38 Chpter 4
We re ow redy to defe the orml dstrbuto fucto, whch mes the probblty tht x s less th or equl to some vlue, lke x b. Ths s pctured below:. Pr(x) Pr[x x b ] x b x Here, to clculte ths probblty, we must tegrte the left tl of the dstrbuto, strtg t - t edg up t x b. Ths wll gve us the probblty tht orml vrte x s less th x b : x b (x μ) Pr[x x b ] exp dx or (4.3) π z b φ ( z) dz b Φ(z ). (4.4) Note the otto Φ(z b ) mples the probblty tht z z b The symbol Φ s uppercse ph whle φ s the lowercse verso of tht Greek letter. It s trdtol to use lower cse letter for fucto, whle the tegrl of tht fucto s sgfed wth the upper cse verso of tht letter. Note lso tht Φ(z) φ(z). z (4.5) A grphcl represetto of Φ(z) s show below:. Φ(z) z The curve pctured bove s ofte clled ogve. Dstrbutos 39
I my cses, for exmple cses hvg to do wth choce probbltes Chpter, we wsh to kow tht probblty tht rdom vrte s greter th : Pr( x 4.3 The Multvrte Norml Dstrbuto ) Φ( μ / ) Φ[ E(x) V(x) ].. (4.6) For purposes of comprso, let us tke the orml dstrbuto s preseted the prevous secto, Pr( x) (x μ) Pr(x x ) exp π / d rewrte t lttle bt. For oe thg,. I tht cse, rewrtg the bove gves us (x μ) Pr( x x ) exp / / (π) ( ) Now lets sy we hve colum vector of p vrbles, x, d tht x follows the multvrte orml dstrbuto wth me vector μ (whch s lso p by ), d vrce mtrx Σ (whch s symmetrc p by p mtrx). I tht cse, the probblty tht the rdom vector x tkes o the set of m vlues tht we wll cll x s gve by. Pr( x x ) (π) p / Σ / exp [( x μ) Σ ( x μ) / ]. (4.7) We would ordrly use short-hd otto for Equto (4.7), syg tht x ~ N(μ, Σ). Mkg some loges, the uvrte expresso ppers the deomtor (of the expoet) whle the multvrte cse we hve Σ - fllg the sme role. You mght lso otce tht the frcto before the expoet, we see the uvrte cse, but Σ / shows up the multvrte cse, the squre root of the determt of the vrce mtrx. I the uvrte cse there s the squre root of π, the multvrte we see the (p/) th root of π. A pcture of the bvrte orml desty fucto ppers below for three dfferet vlues of the correlto ρ. 4 Chpter 4
ρ. ρ.4 ρ.6 4.4 Ch Squre We hve lredy see tht the sclr y, where y ~ N(μ, ), c be coverted to z score, z ~ y μ N (, ) where z. If I squre tht z score I ed up wth ch squre vrte wth oe degree of freedom,. e. z χ. More geerlly, f I hve vector y [ y y L y ] d f y s ormlly dstrbuted wth me vector Dstrbutos 4
μ μ μ L μ d vrce mtrx Σ L L L L L L I L we of course sy tht y ~ N(μ, I). Covertg ech of the y to z scores, tht s z y μ for ll,,,, ; we hve z [ z z L z ]. We c sy tht the vector z ~ N(, I). I tht cse, z z z The Ch Squre desty fucto s pproxmted the followg fgure, usg severl dfferet degrees of freedom to llustrte the shpe. ~ χ. Pr(χ ).5 χ. χ 3.5. χ 7 χ.5. 5 5 5 Wth smll degrees of freedom, the dstrbuto looks lke orml for whch the left tl hs bee folded over the rght. Ths s more or less wht hppes whe we squre somethg - we fold the egtve hlf over the postve. Wth lrger degrees of freedom, the Ch Squre begs to resemble the orml g, d fct, s c be see the grph, the smlrty s lredy qute strkg t degrees of freedom. Ths smlrty s vrtully complete by 3 degrees of freedom. 4 Chpter 4
4.5 Cochr's Theorem For y vector z ~ N(, I) d for y set of mtrces A where A I, the z A z z z (4.8) whch, s we hve just see, s dstrbuted s χ. Further, f the rk (see Secto 3.7) of A s r we c sy tht r d (4.9) z A z χ. (4.) Ech qudrtc form z A z s depedet Ch Squre. The sum of depedet Ch Squre vlues s lso Ch Squre vrble wth degrees of freedom equl to the sum of the compoet's degrees of freedom. Ths llows us to test ested models, such s those foud Chpters 9 d s well s Chpters d 3. I ddto, multple degree of freedom hypothess testg for the ler model s bsed o ths theorem s well. Defg P X(X X) - X d M I - P, the sce ~ r y y y Iy y Py + y My, we hve met the requremets of Cochr's Theorem d we c form F rto usg the two compoets, y Py d y My. I ddto, the compoet y Py c be further prttoed usg the hypothess mtrx A or restrcted models. 4.6 Studet's t-sttstc Lke the orml dstrbuto, the Ch Squre s derved wth kow vlue of. The formul for Ch Squre o degrees of freedom s χ (y μ) [(y y) + (y μ)]. (4.) You wll ote the umertor of the rght hd pece, y hs bee dded d subtrcted. Now we wll squre the umertor of tht rght hd pece whch yelds χ (y y) + y y y μ y + yμ + y yμ + μ. (4.) At ths tme, we c modfy Equto (4.) by dstrbutg the Σ ddto opertor, ccelg some terms, d tkg dvtge of the fct tht Dstrbutos 43
y y. Dog so, we fd tht (y y) (y μ) χ +. (4.3) You mght ote tht t ths pot Equto (4.3) shows the decomposto of degree of freedom Ch Squre to two compoets whch Cochr's Theorem shows us re both themselves dstrbuted s Ch Squre. But the umertor of the summto o the rght hd sde, tht s (y y), s the corrected sum of squres d s such t s equvlet to ( - )s. Rewrtg both compoets slghtly we hve ( )s (y μ) χ + (4.4) whch leves us wth two Ch Squres. The oe o the rght s z-score squred d hs oe degree of freedom. The reder mght recogze t s z score for the rthmetc me, y. The Ch Squre o the left hs - degrees of freedom. At ths pot, to get the ukow vlue to vsh we eed oly crete rto. I fct, to form t-sttstc, we do just tht. I ddto, we dvde by the - degrees of freedom order to mke the t eser to tbulte: (y μ) ( )s t ( ) (4.5) y μ s The more degrees of freedom t dstrbuto hs, the more t resembles the orml. The resemblce s well o ts wy by the tme you rech 3 degrees of freedom. Below you c see grph tht compres the pproxmte desty fuctos for t wth d wth 3 df. 44 Chpter 4
t 3 t The df fucto hs much more weght the tls, s t must be more coservtve. 4.7 The F Dstrbuto Wth the F sttstc, rto s lso formed. However, the cse of the F, we do ot tke the squre root, d the umertor χ s ot restrcted to oe degree of freedom: F r,r χ r r. χ r r Dstrbutos 45