6.3 Stochastic Estimatio ad Cotrol, Fall 004 Lecture 7 Last time: Momets of the Poisso distributio from its geeratig fuctio. Gs () e dg µ e ds dg µ ( s) µ ( s) µ ( s) µ e ds dg X µ ds X s dg dg + ds ds + s µ µ X X µ + µ µ µ X s Eample: Usig telescope to measure itesity of a object Photo flu photoelectro flu. he umber of photoelectros are Poisso distributed. Durig a observatio we cause N photoelectro emissios. N is the measure of the sigal. S N λt N µ λt S λt λt N λt S t λ N For sigal-to-oise ratio of 0, require N 00 photoelectros. All this follows from the property that the variace is equal to the mea. his is a ubouded eperimet, whereas the biomial distributio is for umber of trials. 9/30/004 9:55 AM Page of 0
6.3 Stochastic Estimatio ad Cotrol, Fall 004 3. he Poisso Approimatio to the Biomial Distributio he biomial distributio, lie the Poisso, is that of a radom variable taig oly positive itegral values. Sice it ivolves factorials, the biomial distributio is ot very coveiet for umerical applicatio. We shall show uder what coditios the Poisso epressio serves as a good approimatio to the biomial epressio ad thus may be used for coveiece.! b ( ) p( p)!! ( ) Cosider a large umber of trials,, with small probability of success i each, p, such that the mea of the distributio, p, is of moderate magitude. Defie µ p with large ad p small µ p Recallig: +! ~ π e Stirlig's formula µ µ e lim! µ µ b ( )!( )! + µ e! + + + µ π e µ! + + π ( ) e e µ µ µ e as becomes large relative to! e e µ µ e! he relative error i this approimatio is of order of magitude ( ) Rel. Error ~ µ 9/30/004 9:55 AM Page of 0
6.3 Stochastic Estimatio ad Cotrol, Fall 004 However, for values of much smaller or larger tha µ, the probability becomes small. he Normal Distributio Outlie:. Describe the commo use of the ormal distributio. he practical employmet of the Cetral Limit herorem 3. Relatio to tabulated fuctios Normal distributio fuctio Normal error fuctio Complemetary error fuctio. Describe the commo use of the ormal distributio Normally distributed variables appear repeatedly i physical situatios. Voltage across the plate of a vacuum tube Radar agle tracig oise Atmospheric gust velocity Wave height i the ope sea. he practical employmet of the Cetral Limit herorem X ( i,,..., ) are idepedet radom variables. i Defie the sum of these X i as S S S i i X i X i i Xi he uder the coditio β lim 0 3 β β S i β Xi ( X X ) Xi i i 3 9/30/004 9:55 AM Page 3 of 0
6.3 Stochastic Estimatio ad Cotrol, Fall 004 the limitig distributio of S is the ormal distributio. Note that this is true for ay distributios of the X i. hese are sufficiet coditios uder which the theorem ca be proved. It is ot clear that they are ecessary. Notice that each of the oises metioed earlier deped o the accumulated effect of a great may small causes e.g., voltage across plate: electros travelig from cathode to plate. It is coveiet to wor with the characteristic fuctio sice we are dealig with the sum of idepedet variables. Normal probability desity fuctio: ( m) f( ) e π Normal probability distributio fuctio: ( um) F( ) e du π Where: m m X u m v dv du e π v dv his itegral with the itegrad ormalized is tabulated. It is called the ormal probability fuctio ad symbolized with Φ. 9/30/004 9:55 AM Page 4 of 0
6.3 Stochastic Estimatio ad Cotrol, Fall 004 v Φ ( ) e dv π his is a differet. Note the relatioship betwee this ad the quatity previous defied. We use agai here as this is how Φ is usually writte. Not oly this fuctio but also its first several derivatives which appear i aalytic wor are tabulated. 3. Relatio to tabulated fuctios Eve more geerally available are the closely related fuctios: Error fuctio: u erf ( ) e du π 0 Complemetary error fuctio: u cerf ( ) e du π Φ ( ) erf + t jtm ( m) jt φ() t e e d π t y jt( m+ y) m e e dy, where y π y jtm e (cos( ty) + jsi ( ty) ) e dy π y jtm e cos( t y) e dy π jtm π e e π e Differetiatio of this form will yield correctly the first momets of the distributio. 9/30/004 9:55 AM Page 5 of 0
6.3 Stochastic Estimatio ad Cotrol, Fall 004 Most importat property of ormal variables: ay liear combiatio (weighted sum) of ormal variables, whether idepedet or ot, is aother ormal variable. Note that for zero mea variables f( ) φ() t e e π t Both are Gaussia forms. he Normal Approimatio to the Biomial Distributio he biomial distributio deals with the outcomes of idepedet trials of a eperimet. hus if is large, we should epect the biomial distributio to be well approimated by the ormal distributio. he approimatio is give by the ormal distributio havig the same mea ad variace. hus ( p) pq bp (,, ) e π pq 3 ( p) Relative error is of the order of ( pq) he relative fit is good ear the mea if pq is large ad degeerates i the tails where the probability itself is small. he Normal Approimatio to the Poisso Distributio Also the Poisso distributio depeds o the outcomes of idepedet evets. If there are eough of them, 9/30/004 9:55 AM Page 6 of 0
6.3 Stochastic Estimatio ad Cotrol, Fall 004 P (, µ ) e πµ ( µ ) µ 3 ( µ ) Relative error is of the order of µ he relative fit is subject to the same behavior as the biomial approimatio. Iterpretatio of a cotiuous distributio approimatig a discrete oe: he value of the ormal desity fuctio at ay approimates the value of the discrete distributio for that value of. hi of spreadig the area of each impulse over a uit iterval. he the height of each rectagle is the probability that the correspodig value of will be tae. he ormal curve approimates this step-wise fuctio. Note that i summig the probabilities for values of i some iterval, the approimatig ormal curve should be itegrated over that iterval plus ½ o each ed to get all the probability associated with those values of. N PN ( X N) P ( ) N N N + N N ( µ ) µ P (, µ ) e d πµ N + µ N µ Φ Φ µ µ Multidimesioal Normal Distributio Probability desity fuctio: Τ f ( ) ep ( X) M ( X) ( π ) M 9/30/004 9:55 AM Page 7 of 0
6.3 Stochastic Estimatio ad Cotrol, Fall 004 Assumig zero mea, which is ofte the case: Τ f ( ) ep M ( π ) M For zero mea variables: cotours of costat probability desity are give by: M c Not epressed i pricipal coordiates if the X i are correlated. Need to ow the rudimetary properties of eigevalues ad eigevectors. M is symmetric ad full ra. Mvi λivi, i j vi vj δij 0, i j his probability desity fuctio ca be better visualized i terms of its pricipal coordiates. hese coordiates are defied by the directios of the eigevectors of the covariace matri. he appropriate trasformatio is y V v V M v hus y i is the compoet of i the directio v i. (I terms of the ew variable y, the cotours of costat probability desity are) 9/30/004 9:55 AM Page 8 of 0
6.3 Stochastic Estimatio ad Cotrol, Fall 004 M y V M V y yy y Y V M V Y VMV VM v... v v... λv... λv v λ O λ Y is the covariace matri for the radom variable Y variace of the Y i. λ Y O λ VX, so the λi are the λ y O y c λ y y y yy y + +... + c λ λ λ hese are the pricipal coordiate with itercepts at yi stadard deviatio of y i. c λ ± i with i λ the Note that two radom variables, each havig a ormal distributio sigly, do ot ecessarily have a biormal joit distributio. However, if the radom variables are idepedet ad ormally distributed, their joit distributio is clearly a multidimesioal ormal distributio. 9/30/004 9:55 AM Page 9 of 0
6.3 Stochastic Estimatio ad Cotrol, Fall 004 wo dimesioal case- cotiued: m f X X ij i j m m + m (, ) ep π m ( m m m m m ) m m m µ µ ρ y I terms of these symbols: ρ + f(, ) ep π ρ ( ρ ) Note that if a set of radom variables havig the multidimesioal ormal distributio is ucorrelated, they are idepedet. his is ot true i geeral. 9/30/004 9:55 AM Page 0 of 0