Chapter 4: Elements of Statistics

Transcription

1 Chapter : lemets of tatstcs - Itroducto The amplg Problem Ubased stmators -&3 amplg Theory --The ample Mea ad ace amplg Theorem - amplg Dstrbutos ad Cofdece Itervals tudet s T-Dstrbuto -5 Hypothess Testg -6 Curve Fttg ad Lear Regresso -7 Correlato Betwee Two ets of Data Cocepts ample meas ad sample varace relato to pdf mea ad varace Based estmates of meas ad varaces How close are the sample values to the uderlyg pdf values? Practcal curve fttg, usg a NTC resstor to measure temperature. Notes ad fgures are based o or take from materals the course textbook: Probablstc Methods of gal ad ystem Aalyss (3rd ed.) by George R. Cooper ad Clare D. McGllem; Oxford Press, 999. IBN: B.J. Bazu, prg 05 of 9 C 3800

2 - Itroducto tatstcs Defto: The scece of assemblg, classfyg, tabulatg, ad aalyzg data or facts: Descrptve statstcs the collectg, groupg ad presetg data a way that ca be easly uderstood or assmlated. Iductve statstcs or statstcal ferece use data to draw coclusos about or estmate parameters of the evromet from whch the data came from. Theoretcal Areas: amplg Theory stmato Theory Hypothess testg Curve fttg ad regresso Aalyss of ace selectg samples from a collecto of data that s too large to be examed completely. cocered wth makg estmates or predctos based o the data that are avalable. attempts to decde whch of two or more hypotheses about the data are true. attempt to fd mathematcal expressos that best represet the data. attempt to assess the sgfcace of varatos the data ad the relato of these varaces to the physcal stuatos from whch the data arose. (Moder term ANOVA) Notes ad fgures are based o or take from materals the course textbook: Probablstc Methods of gal ad ystem Aalyss (3rd ed.) by George R. Cooper ad Clare D. McGllem; Oxford Press, 999. IBN: B.J. Bazu, prg 05 of 9 C 3800

3 amplg Theory The ample Mea How may samples are requred to fd a represetatve sample set that provdes cofdece the results? Defect testg, opo polls, fecto rates, etc. Deftos Populato: ample: ample Mea: the collecto of data beg studed N s the sze of the populato a radom sample s the part of the populato selected all members of the populato must be equally lkely to be selected! s the sze of the sample the average of the umercal values that make of the sample Populato: N ample set: x x, x, x, x,, 3 5 x ample Mea x x To geeralze, descrbe the statstcal propertes of arbtrary radom samples rather tha those of ay partcular sample. ample Mea, where are radom varables wth a pdf. Notce that for a pdf, the true mea,, ca be compute whle for a sample data set the above sample mea, s computed. Notes ad fgures are based o or take from materals the course textbook: Probablstc Methods of gal ad ystem Aalyss (3rd ed.) by George R. Cooper ad Clare D. McGllem; Oxford Press, 999. IBN: B.J. Bazu, prg 05 3 of 9 C 3800

4 Notes ad fgures are based o or take from materals the course textbook: Probablstc Methods of gal ad ystem Aalyss (3rd ed.) by George R. Cooper ad Clare D. McGllem; Oxford Press, 999. IBN: B.J. Bazu, prg 05 of 9 C 3800 As may be oted, the sample mea s a combato of radom varables ad, therefore, ca also be cosdered a radom varable. As a result, the hoped for result ca be derved as: If ad whe ths s true, the estmate s sad to be a ubased estmate. Though the sample mea may be ubased, the sample mea may stll ot provde a good estmate. What s the varace of the computato of the sample mea? - ace of the sample mea (the mea tself, ot the value of ) You would expect the sample mea to have some varace about the probablstc or actual mea; therefore, t s also desrable to kow somethg about the fluctuatos aroud the mea. As a result, computato of the varace of the sample mea s desred. For N>> or N fty (or eve a kow pdf), usg the collected samples. For depedet (measuremets are depedet of each other) for for,,

5 Notes ad fgures are based o or take from materals the course textbook: Probablstc Methods of gal ad ystem Aalyss (3rd ed.) by George R. Cooper ad Clare D. McGllem; Oxford Press, 999. IBN: B.J. Bazu, prg 05 5 of 9 C 3800 As a result we ca defe two summato where = ad <>,, where s the true varace of the radom varable,. Therefore, as approaches fty, ths varace the sample mea estmate goes to zero! Thus a larger sample sze leads to a better estmate of the populato mea. Note: ths varace s developed based o samplg wth replacemet. Whe based o samplg wthout replacemet Destructve testg or samplg wthout replacemet a fte populato results aother expresso: N N Note that whe all the samples are tested (N=) the varace ecessarly goes to 0. The varace the mea betwee the populato ad the sample set must be zero as the etre populato has bee measured!

6 xample: How may samples of a ftely log tme waveform would be requred to sure the mea s wth % of the true (probablstc) mea value? For ths relatoshp, let Ifte set, therefore assume that you use the wth replacemet equato : Assume that the true meas s 0 ad that the true varace s 9 so that 0 3. The, A very large sample set sze to estmate the mea wth the desred boud! Notes ad fgures are based o or take from materals the course textbook: Probablstc Methods of gal ad ystem Aalyss (3rd ed.) by George R. Cooper ad Clare D. McGllem; Oxford Press, 999. IBN: B.J. Bazu, prg 05 6 of 9 C 3800

7 Cetral Lmt Theorem stmate Thkg of the characterzato after usg a very large umber of samples Usg the cetral lmt theorem (assume a Gaussa dstrbuto) to estmate the probablty that the mea s wth a prescrbed varace (% from the prevous example): Pr F 0. F 9.9 Assume that the statstcal measuremet desty fucto has become Gaussa cetered aroud 0 wth a % of the mea stadard devato (assumg that 0 ad 0. ). We ca use Gaussa/Normal Tables to determe the probablty Pr Pr Pr Ths mples that, after takg so may measuremet to form a estmate, there s a 68.3% chace the estmate s wth % of the mea or that there s a or 3.7% probablty that the estmate of the populato mea s more tha % away from the true populato mea. ummary, as the umber of sample measured creases, the desty fucto of the estmated mea about the true (probablstc) mea takes o a Gaussa characterstc. Based o the varace of the sample mea computato (related to umber of samples) the probablty that the measuremet mea match the probablstc mea has kow probablty (based o Gaussa statstcs). We wll be dealg wth Gaussa/Normal Dstrbutos as large sum szes wth some radom varable assocato haves ot desty fuctos that are Gaussa Cetral Lmt Theorem. Notes ad fgures are based o or take from materals the course textbook: Probablstc Methods of gal ad ystem Aalyss (3rd ed.) by George R. Cooper ad Clare D. McGllem; Oxford Press, 999. IBN: B.J. Bazu, prg 05 7 of 9 C 3800

8 xample #: A smaller sample sze Populato: 00 trasstors Fd the mea value of the curret ga,. The true populato mea s 0 ad the true populato varace s 5. How large a sample s requred to obta a sample mea that has a stadard devato of % of the true mea? Therefore, we wat A smaller sample sze, sample mea varace ca be computed as N N Determg the umber of samples eeded to meet tolerace A rule-of-thumb s offered to defe large vs. small sample szes, the threshold gve s 30. The ultmate goal s to acheve a ear-gaussa probablty dstrbuto. Notes ad fgures are based o or take from materals the course textbook: Probablstc Methods of gal ad ystem Aalyss (3rd ed.) by George R. Cooper ad Clare D. McGllem; Oxford Press, 999. IBN: B.J. Bazu, prg 05 8 of 9 C 3800

9 -3 amplg Theory The ample ace Whe dealg wth probablty, both the mea ad varace provde valuable formato about the DC ad AC operatg codtos (about what value s expected) ad the varace ( terms of power or squared value) about the operatg pot. Therefore, we are also terested the sample varace as compared to the true data varace. The sample varace of the populato (stdevp) s defed as: ad cotug utl (show the comg pages) where s the true varace of the radom varable. Note: the sample varace s ot equal to the true varace; t s a based estmate! To create a ubased estmator, scale by the basg factor to compute (stdev): ~ x Whe the populato s ot large, the based estmate becomes N N ad removg the bas results ~ N N Notes ad fgures are based o or take from materals the course textbook: Probablstc Methods of gal ad ystem Aalyss (3rd ed.) by George R. Cooper ad Clare D. McGllem; Oxford Press, 999. IBN: B.J. Bazu, prg 05 9 of 9 C 3800

10 Addtoal otes: MATLAB ad M xcel mulato ad statstcal software packages allow for ether based or ubased computatos. I M xcel there are two dstct fuctos stdev ad stdevp. stdev uses (-) - stdevp uses () - I MATLAB, there s a addtoal flag assocate wth the std fucto. std std x var, flag mpled as 0 var, x,, flag specfed as >> help std std tadard devato. For vectors, Y = std() returs the stadard devato. For matrces, Y s a row vector cotag the stadard devato of each colum. For N-D arrays, std operates alog the frst o-sgleto dmeso of. std ormalzes Y by (N-), where N s the sample sze. Ths s the sqrt of a ubased estmator of the varace of the populato from whch s draw, as log as cossts of depedet, detcally dstrbuted samples. Y = std(,) ormalzes by N ad produces the square root of the secod momet of the sample about ts mea. std(,0) s the same as std(). Notes ad fgures are based o or take from materals the course textbook: Probablstc Methods of gal ad ystem Aalyss (3rd ed.) by George R. Cooper ad Clare D. McGllem; Oxford Press, 999. IBN: B.J. Bazu, prg 05 0 of 9 C 3800

11 Notes ad fgures are based o or take from materals the course textbook: Probablstc Methods of gal ad ystem Aalyss (3rd ed.) by George R. Cooper ad Clare D. McGllem; Oxford Press, 999. IBN: B.J. Bazu, prg 05 of 9 C 3800 amplg Theory The ample ace - Proof The sample varace of the populato s defed as Determg the expected value k k k k k k k k k, 3 3

12 Notes ad fgures are based o or take from materals the course textbook: Probablstc Methods of gal ad ystem Aalyss (3rd ed.) by George R. Cooper ad Clare D. McGllem; Oxford Press, 999. IBN: B.J. Bazu, prg 05 of 9 C 3800 Therefore, To create a ubased estmator, scale by the (u-) basg factor to compute: ~ ace of the varace As before, the varace of the varace ca be computed. (Istead of dervg the values, t s gve.) It s defed as where s the fourth cetral momet of the populato ad s defed by Aother proof for extra credt For the ubased varace, the result s ~

13 xample: the radom tme samples problem (frst example) prevously used where the true meas s 0 ad that the true varace s 9. The, ad for = ~ for a Gaussa radom varable, the th cetral momet s 3. Therefore ~ The ace estmate would the be ~ ~ ~ 00 or wth %.7% Whle 900 was selected to provde a mea estmate that was wth %, the varace estmate s ot early as close at.7%. More samples are requred to mprove the varace estmate. ~ 9 Notes ad fgures are based o or take from materals the course textbook: Probablstc Methods of gal ad ystem Aalyss (3rd ed.) by George R. Cooper ad Clare D. McGllem; Oxford Press, 999. IBN: B.J. Bazu, prg 05 3 of 9 C 3800

14 - amplg Dstrbuto ad Cofdece Itervals Now that we have developed sample values, what are they good for What s the probablty that our estmates are wth specfed bouds by measurg samples, ca you prove that what you bult or dd s what was specfed or promsed? To really aswer these questos, t s ecessary to kow the probablty desty fucto assocated wth parameter estmates such as the sample mea ad sample varace. A great deal of effort has bee expeded the study of statstcs to determe these probablty desty fuctos ad may such fuctos are descrbed the lterature. (Iterpretato: the materal s very dffcult, ad, except for those who love math ad statstc, ot ecessary to preset the followg materal whch provdes smplfcatos that are commoly used by egeers). Whe doubt assume Gaussa. The, the ormalzed radom varable becomes (the sample mea wth the mea removed, dvded by the varace of the sample mea) Z f the true populato mea s ot kow, t ca be replaced by the sample varace T ~ Ths dstrbuto s defed as a tudet s t dstrbuto wth - degrees of freedom. Notes ad fgures are based o or take from materals the course textbook: Probablstc Methods of gal ad ystem Aalyss (3rd ed.) by George R. Cooper ad Clare D. McGllem; Oxford Press, 999. IBN: B.J. Bazu, prg 05 of 9 C 3800

15 The tudet s t probablty desty fucto (lettg v=-, the degrees of freedom) s defed as where ft t s the gamma fucto. v v t v v v The gamma fucto ca be computed as k k k k! for ay k for k a teger ad () Note that whe evaluatg the tudet s t-desty fucto, all argumets of the gamma fucto are tegers or a teger plus ½. () Note that: The dstrbuto depeds o ν, but ot μ or σ; the lack of depedece o μ ad σ s what makes the t-dstrbuto mportat both theory ad practce. tudet's dstrbuto arses whe (as early all practcal statstcal work) the populato stadard devato s ukow ad has to be estmated from the data. Textbook problems treatg the stadard devato as f t were kow are of two kds: () those whch the sample sze s so large that oe may treat a data-based estmate of the varace as f t were certa, ad () those that llustrate mathematcal reasog, whch the problem of estmatg the stadard devato s temporarly gored because that s ot the pot that the author or structor s the explag. Note that: The dstrbuto depeds o ν, but ot μ or σ; the lack of depedece o μ ad σ s what makes the t-dstrbuto mportat both theory ad practce. Notes ad fgures are based o or take from materals the course textbook: Probablstc Methods of gal ad ystem Aalyss (3rd ed.) by George R. Cooper ad Clare D. McGllem; Oxford Press, 999. IBN: B.J. Bazu, prg 05 5 of 9 C 3800

16 Comparg the desty fuctos: tudet s t ad Gaussa tudets t ad Gaussa Destes Gaussa T w/ v= T w/ v= T w/ v=8 desty fucto tudet s t Gaussa ee Fg.m ad fucto studets_t.m f ft t x v v t v v v exp x Notes ad fgures are based o or take from materals the course textbook: Probablstc Methods of gal ad ystem Aalyss (3rd ed.) by George R. Cooper ad Clare D. McGllem; Oxford Press, 999. IBN: B.J. Bazu, prg 05 6 of 9 C 3800

17 Cofdece Itervals ad the Gaussa ad tudet s t dstrbutos The sample mea s a pot-estmate (assgs a sgle value). A alteratve to a pot-estmate s a terval-estmate where the parameter beg estmated s declared to le wth a certa terval wth a certa probablty. The terval estmate s the cofdece terval. We ca the defe a q% cofdece terval as the terval whch the estmate wll le wth a probablty of q/00. The lmts of the terval are defed as the cofdece lmts ad q s also defed to be the cofdece level. Thus we are terested k k where k s a costat defed as (otce that t multples the stadard devato) k q 00 f k x dx F k F k Whe the sample sze s suffcet to meet the Cetral Lmt Theorem, a Gaussa ormal dstrbuto ca be used. Gaussa pdf ad PDF Z q z c z c for zc z zc z c q for z c z x, x exp for x F x v x exp dv v Notes ad fgures are based o or take from materals the course textbook: Probablstc Methods of gal ad ystem Aalyss (3rd ed.) by George R. Cooper ad Clare D. McGllem; Oxford Press, 999. IBN: B.J. Bazu, prg 05 7 of 9 C 3800

18 Cofdece Iterval ( %) Two Tal Bouds k or z : z z z 99.99% 0.005% to % % 0.05% to 99.95% % 0.5% to 99.5%.58 95%.5% to 97.5%.96 90% 5% to 95%.6 80% 0% ro 90%.8 50% 5% to 75% To fd the values, () determe the percetage value requred for the boud (e.g. 75% for a 50% -sded terval) () fd that value the Normal table (ut varace). c c c The value of k or zc s ust the row plus colum value that would create the probablty! Z q z c z c for zc z zc z c q for z c z Gaussa q values q= 50.00%, k= f(x) db q= 90.00%, k=.65 q= 95.00%, k=.960 q= 99.00%, k= see Fg 6.m Notes ad fgures are based o or take from materals the course textbook: Probablstc Methods of gal ad ystem Aalyss (3rd ed.) by George R. Cooper ad Clare D. McGllem; Oxford Press, 999. IBN: B.J. Bazu, prg 05 8 of 9 C 3800

19 If the sample sze s ot suffcet, the tudet t-dstrbuto must be used. Remder, as the tudet s t-dstrbuto degrees of freedom crease ( v becomes large), the t-dstrbuto approaches the Gaussa dstrbuto! 0. T-pdf ad Normal pdf, v= 0. T-pdf ad Normal pdf, v= Appedx F provdes tables of t for gve v ad F based o: FT t t x v v v v x v Usg the estmated sample mea ad the varace of the sample mea: tc t ~ t dt F t F t q 00 ft T c T c for c c tc t t t tc q 00 ft t dt FT t c for t c t, rght-tal, -sded Notes ad fgures are based o or take from materals the course textbook: Probablstc Methods of gal ad ystem Aalyss (3rd ed.) by George R. Cooper ad Clare D. McGllem; Oxford Press, 999. IBN: B.J. Bazu, prg 05 9 of 9 C 3800