Probability and Statistics Basic concepts II
|
|
- Lizbeth Osborne
- 5 years ago
- Views:
Transcription
1 Probablty ad Statstcs Basc cocepts II (from a physcst pot of vew) Beot CLEMENT Uversté J. Fourer / LPSC bclemet@lpsc.p3.fr
2 Statstcs PHYSICS parameters θ SAMPLE Fte sze x Observable INFERENCE POPULATION f(x;θ) EXPERIMENT Parametrc estmato : to gve oe value to each parameter Iterval estmato : to derve a terval that probably cotas the true value No-parametrc estmato : to estmate the full pdf of the populato
3 Parametrc estmato From a fte sample {x } -> estmatg a parameter θ Statstc a fucto S f({x }) Ay statstc ca be cosdered as a estmator of θ To be a good estmator t eeds to satsfy : Cosstecy : lmt of the estmator for a fte sample. Bas : dfferece betwee the estmator ad the true value Effcecy : speed of covergece Robustess : sestvty to statstcal fluctuatos A good estmator should at least be cosstet ad asymptotcally ubased 3 Effcet / Ubased / Robust ofte cotradct each other dfferet choces for dfferet applcatos
4 Bas ad cosstecy As the sample s a set of realzato of radom varables (or oe vector varable), so s the estmator : θˆ s a realzato of Θˆ t has a mea, a varace, ad a probablty desty fucto Bas : Mea value of the estmator b( θ) ˆ E[Θ- ˆ θ0 ] μ - θ Θˆ 0 ubased estmator : b( θ) ˆ 0 asymptotcally ubased : b( θ) ˆ 0 + Cosstecy: formally P( θ-θ ˆ > ε) 0, ε + practce, f asymptotcally ubased σ 0 based asymptotcally ubased Θˆ + ubased 4
5 sˆ Emprcal estmator Sample mea s a good estmator of the populato mea -> weak law of large umbers : coverget, ubased Sample varace as a estmator of the populato varace : E[sˆ 1 μˆ x, μμ E[μ] ˆ μ, σμ E[(μ-μ) ˆ ˆ ˆ ] 1 ] (x 1 μ) ˆ σ σ 1 μˆ (x σ μ) σ ubased varace estmator : ( μ μˆ ) 1 σ 1 σˆ -1 σ based, asymptotcally ubased (x μ) ˆ 5 varace of the estmator (covergece) σ ˆ σ 4 σ 1 γ -1 + σ 4
6 Errors o these estmator Ucertaty Estmator stadard devato Use a estmator of stadard devato : (!!! Based ) Mea : σˆ 1 σ μˆ x, σμ Δμˆ ˆ σˆ σˆ Varace : 1 σ 4 σˆ (x μ) ˆ, σ Δσˆ σˆ σˆ -1 Cetral-Lmt theorem -> emprcal estmators of mea ad varace are ormally dstrbuted, for large eough samples 6 ˆ ± Δμˆ ; σˆ ± μ Δσˆ defe 68% cofdece tervals
7 Lkelhood fucto Geerc fucto k(x,θ) x : radom varable(s) θ : parameter(s) fx θ θ 0 (true value) fx x u (oe realzato of the radom varable) Probablty desty fucto f(x;θ) k(x,θ 0 ) f(x;θ) dx1 for Bayesa f(x θ) f(x;θ) Lkelhood fucto L (θ) k(u,θ) L (θ) dθ??? for Bayesa f(θ x) L (θ)/ L (θ)dθ 7 For a sample : depedet realzatos of the same varable X L(θ ) k(x,θ) f(x ; θ)
8 0 0 Estmator varace Start from the geerc k fucto, dfferetate twce, wth respect to θ, the pdf ormalzato codto: 1 k(x, θ)dx k dx θ k dx θ lk k θ lk k θ lk dx E θ dx + lk k θ lk (b + θ)e θ Now dfferetatg the estmator bas : lk dx E θ 0 lk E θ θ + b θ(x)k(x, ˆ θ)dx 1 b k lk + θ(x)k(x,θ)dx ˆ θ dx θk dx (θ-b-θ)k θ θ ˆ ˆ θ θ ˆ lk dx θ 8 Fally, usg Cauchy-Schwartz equalty ( 1+ b' ) b lk + (θ-b-θ) ˆ kdx k dx σ Θ θ θ lk E Cramer-Rao boud θ 1 ˆ
9 Effcecy For ay ubased estmator of θ, the varace caot exceed : 1 σ Θˆ ll E θ The effcecy of a coverget estmator, s gve by ts varace. A effcet estmator reaches the Cramer-Rao boud (at least asymptotcally) : Mmal varace estmator MVE wll ofte be based, asymptotcally ubased 9
10 Maxmum lkelhood 10 For a sample of measuremets, {x } The aalytcal form of the desty s kow It depeds o several ukow parameters θ eg. evet coutg : Follow a Posso dstrbuto, wth a parameter that depeds o the physcs : λ (θ) λ (θ ) x e λ (θ ) L(θ ) x! A estmator of the parameters of θ, are the oes that maxmze of observg the observed result. -> Maxmum of the lkelhood fucto L θ θ θ ˆ rem : system of equatos for several parameters rem : ofte mmze -ll : smplfy expressos 0
11 Propertes of MLE Mostly asymptotc propertes : vald for large sample, ofte assumed ay case for lack of better formato Asymptotcally ubased Asymptotcally effcet (reaches the CR boud) Asymptotcally ormally dstrbuted -> Multormal law, wth covarace gve by geeralzato of CR Boud : ˆ 1 f(θ;θ,σ) π Σ e ˆ (θ-θ) 1 Τ 1 Σ ˆ (θ-θ) ll E θ ll θj Goodess of ft The value of -ll( θˆ ) s Kh- dstrbuted, wth df sample sze umber of parameters Σ 1 j 11 p value + f (x;df)dx -ll ( θˆ ) χ Probablty of gettg a worse agreemet
12 Errors o MLE 1 ˆ Τ (θ-θ) Σ 1 ˆ ˆ 1 (θ-θ) f(θ;θ,σ) e 1 ll Σj E π Σ θ Errors o parameter -> from the covarace matrx ll θj 1 For oe parameter, 68% terval More geerally : ΔlL Cofdece cotour are defed by the equato : Values of β for dfferet umber of parameters θ ad cofdece levels α Δθ σˆ θˆ 1 ll θ 1 1 ll( ˆ θ ) ll( θ ) Σ (θ -θ ˆ )(θj -θˆ ΔlL,j β( θ α j j +,α) wth α oly oe realzato of the estmator -> emprcal mea of 1 value β ) f θ 0 χ O(θ (x; 1 3 (0. 5* θ ) 3 θ ) )dx
13 Least squares 13 Set of measuremets (x, y ) wth ucertates o y Theoretcal law : y f(x,θ) Naïve approach : use regresso w w( θ ) (y f(x,θ)), 0 θ Reweght each term by the error y f(x,θ) K K ( θ ), 0 y θ Maxmum lkelhood : assume each y s ormally dstrbuted wth a mea equal to f(x,θ) ad a varace equal to Δy 1 y f(x,θ) 1 Δ The the lkelhood s : y L(θ ) e πδy L l L K 0 0 Least squares or Kh- ft s θ θ θ the MLE, for Gaussa errors 1 Τ 1 Geerc case wth correlatos: K (θ) (y-f(x,θ) Σ (y-f(x,θ)
14 Example : fttg a le For f(x)ax+b 14 Δy 1 E, Δy y D, Δy x C, Δy x B, Δy x y A E 1.5 Δb, B 1.5 Δa C BE AC DB b, C BE DC AE a ˆ ˆ ˆ ˆ
15 Example : fttg a le dmesoal error cotours o a ad b 15
16 Cofdece terval For a radom varable, a cofdece terval wth cofdece level α, s ay terval [a,b] such as : b Probablty of fdg a P(X [a,b]) f (x)dx α a X realzato sde the terval Geeralzato of the cocept of ucertaty: terval that cotas the true value wth a gve probablty -> slghtly dfferet cocepts For Bayesas : the posteror desty s the probablty desty of the true value. It ca be used to derve terval : P(θ [a,b]) α 16 No such thg for a Frequetst : The terval tself becomes the radom varable [a,b] s a realzato of [A,B] P(A < θ ad B > θ) α Idepedetly of θ
17 Cofdece terval Mea cetered, symetrc terval μ + μ a [μ-a, μ+a] a f(x)dx α Mea cetered, probablty symetrc terval : [a, b], μ b α f(x)dx f(x)dx a μ 17 Hghest Probablty Desty (HDP) b [a, b] f(x)dx α a f(x) > f(y) for x [a,b] ad y [a,b]
18 Cofdece Belt To buld a frequetst terval for a estmator θˆ of θ 1. Make pseudo-expermets for several values of θ ad compute he estmator θˆ for each (MC samplg of the estmator pdf). For each θ, determe A(θ) ad B(θ) such as : θ ˆ < Ξ(θ) for a fracto (1-α)/ of the pseudo-expermets θ ˆ > Ω(θ) for a fracto (1-α)/ of the pseudo-expermets These curves are the cofdece belt, for a CL α Iverse these fuctos. The terval [Ω (θ), ˆ Ξ (θ)] ˆ satsfy: -1 P Ω (θ) ˆ < θ < Ξ (θ) ˆ 1 P Ξ (θ) ˆ < θ -P Ω (θ) ˆ > θ ( ) ( ) ( ) ( ) ( ) 1 P θˆ < Ξ(θ) -P θˆ > Ω(θ) α 18 Cofdece Belt for Posso parameter λ estmated wth the emprcal mea of 3 realzatos (68%CL)
19 Dealg wth systematcs The varace of the estmator oly measure the statstcal ucertaty. Ofte, we wll have to deal wth some parameters whose value s kow wth lmt precso. Systematc ucertates The lkelhood fucto becomes : L( θ, ν ) ν ν 0 ± Δν or ν + Δν 0-ΔΔ + The kow parameters ν are usace parameters 19
20 Bayesa ferece I Bayesa statstcs, usace parameters are dealt wth by assgg them a pror π(ν). Usually a multormal law s used wth mea ν 0 ad covarace matrx estmated from Δν 0 (+correlato, f eeded) f(θ,ν x) f(x θ,ν)π(θ)π(ν) f(x θ,ν)π(θ)π(ν)dθ The fal pror s obtaed by margalzato over the usace parameters dν 0 f(θ x) f(θ,ν x)dν f(x θ,ν)π(θ)π(ν)dν f(x θ,ν)π(θ)π(ν)dθ dν
21 Profle Lkelhood No true frequetst way to add systematc effects. Popular method of the day : proflg Deal wth usace parameters as realzato f radom varables : exted the lkelhood : L( θ, ν ) L' ( θ, ν ) G(ν ) G(v) s the lkelhood of the ew parameters (detcal to pror) For each value of θ, mmze the lkelhood wth respect to usace : profle lkelhood PL(θ). PL(θ) has the same statstcal asymptotcal propertes tha the regular lkelhood 1
22 No parametrc estmato Drectly estmatg the probablty desty fucto Lkelhood rato dscrmat Separatg power of varables Data/MC agreemet Frequecy Table : For a sample {x }, Defe successve tervals (bs) C k [a k,a k+1 [.Cout the umber of evets k C k Hstogram : Graphcal represetato of the frequecy table h(x) f x C k k
23 Hstogram 3 N/Z for stable heavy ucle 1.31, 1.357, 1.39, 1.410, 1.48, 1.446, 1.464, 1.41, 1.438, 1.344, 1.379, 1.413, 1.448, 1.389, 1.366, 1.383, 1.400, 1.416, 1.433, 1.466, 1.500, 1.3, 1.370, 1.387, 1.403, 1.419, 1.451, 1.483, 1.396, 1.48, 1.375, 1.406, 1.41, 1.437, 1.453, 1.468, 1.500, 1.446, 1.363, 1.393, 1.44, 1.439, 1.454, 1.469, 1.484, 1.46, 1.38, 1.411, 1.441, 1.455, 1.470, 1.500, 1.449, 1.400, 1.48, 1.44, 1.457, 1.471, 1.485, 1.514, 1.464, 1.478, 1.416, 1.444, 1.458, 1.47, 1.486, 1.500, 1.465, 1.479, 1.43, 1.459, 1.47, 1.486, 1.513, 1.466, 1.493, 1.41, 1.447, 1.460, 1.473, 1.486, 1.500, 1.56, 1.480, 1.506, 1.435, 1.461, 1.487, 1.500, 1.51, 1.538, 1.493, 1.450, 1.475, 1.500, 1.51, 1.55, 1.550, 1.506, 1.530, 1.487, 1.51, 1.54, 1.536, 1.518, 1.577, 1.554, 1.586, 1.586
24 Hstogram Statstcal descrpto : k are multomal radom varables. wth parameters : μ p k pk P(x Ck ) σ k p (1 p ) μ C k Cov( f X (x)dx, ) p k k k k k p << 1 k k r k r p << 1 For a large sample : lm k k + So fally : μ p k p k f(x) k C k f lm + δ 0 For small classes (wdth δ): X (x)dx h(x) The hstogram s a estmator of the probablty desty 1 δ δf(x c p pk ) lm δ 0 δ k 0 f(x) 4 Each b ca be descrbed by a Posso desty. The 1σ error o k s the : Δ k σˆ μˆ k k k
25 Kerel desty estmators Hstogram s a step fucto -> sometme eed smoother estmator O possble soluto : Kerel Desty Estmator Attrbute to each pot of the sample a kerel fucto k(u) x x u, k(u) k( u), k(u)du 1 w Tragle kerel : Parabolc kerel : Gaussa kerel : k(u) 1- u, for -1< u < 1 3 k(u) (1-u ), for -1< u < 1 k(u) w kerel wdth, smlar to b wdth of the hstogram The a pdf estmator s : K(x) 1 k(u ) 4 1 π 1 e u - k x x w 5 Rem : for multdmesoal pdf : u k x (k) x (k) w (k)
26 Kerel desty estmators If the estmated desty s ormal, the optmal wdth s : 3 w σ (d + ) 1 d+ 4 wth that sample sze ad d the dmeso As for the hstogram bg, o geerc result : try ad see 6
27 Statstcal tests am at: Statstcal Tests Checkg the compatblty of a dataset {x } wth a gve dstrbuto Checkg the compatblty of two datasets {x }, {y } : are they ssued from the same dstrbuto. Comparg dfferet hypothess : backgroud vs sgal+backgroud 7 I every case : buld a statstc that quatfy the agreemet wth the hypothess covert t to a probablty of compatblty/compatblty : p-value
28 Pearso test Test for bed data : use the Posso lmt of the hstogram Sort the sample to k bs C : Compute the probablty of ths class : p C f(x)dx The test statstcs compare, for each b the devato of the observato from the expected mea to the theoretcal stadard devato. χ Data bs ( p ) The χ follow (asymptotcally) a Kh- law wth k-1 degrees of freedom (1 costrat ) p Posso mea Posso varace 8 P-value : probablty of dog worse, p value + f (x;k-1)dx For a good agreemet χ /(k-1) ~ 1, More precsely χ (k 1) ± (k 1) (1σ terval ~ 68%CL) χ χ
29 Kolmogorov-Smrov test Test for ubed data : compare the sample comulatve desty fucto to the tested oe Sample Pdf (ordered sample) 0 x < x0 1 k f s(x) δ(x-) F s(x) xk x < xk+ 1 x > x The the Kolmogorov statstc s the largest devato : D sup F x (x) F(x) The test dstrbuto has bee computed by Kolmogorov: S 1 P(D > β ) ( 1) r 9 [0;β] defe a cofdece terval for D β0.9584/ for 68.3% CL β1.3754/ for 95.4% CL r 1 e r z
30 Example Test compatblty wth a expoetal law : f(x) , 0.036, 0.11, 0.115, 0.133, 0.178, 0.189, 0.38, 0.74, 0.33, 0.364, 0.386, 0.406, 0.409, 0.418, 0.41, 0.43, 0.455, 0.459, 0.496, 0.519, 0.5, 0.534, 0.58, 0.606, 0.64, 0.649, 0.687, 0.689, 0.764, 0.768, 0.774, 0.85, 0.843, 0.91, 0.987, 0.99, 1.003, 1.004, 1.015, 1.034, 1.064, 1.11, 1.159, 1.163, 1.08, 1.53, 1.87, 1.317, 1.30, 1.333, 1.41, 1.41, 1.438, 1.574, 1.719, 1.769, 1.830, 1.853, 1.930,.041,.053,.119,.146,.167,.37,.43,.49,.318,.35,.349,.37,.465,.497,.553,.56,.616,.739,.851, 3.09, 3.37, 3.335, 3.390, 3.447, 3.473, 3.568, 3.67, 3.718, 3.70, 3.814, 3.854, 3.99, 4.038, 4.065, 4.089, 4.177, 4.357, 4.403, 4.514, 4.771, 4.809, 4.87, 5.086, 5.191, 5.98, 5.95, 5.968, 6., 6.556, 6.670, 7.673, 8.071, 8.165, 8.181, 8.383, 8.557, 8.606, 9.03, 10.48, λe λx, λ D p-value σ : [0, ] 30
31 Hypothess testg Two exclusve hypotheses H 0 ad H 1 -whch oe s the most compatble wth data - how compatble s the other oe P(data H0) vs P(data H1) Buld a statstc, defe a terval w - f the observato falls to w : accept H else accept H 0 Sze of the test : how ofte dd you get t rght α L(x H 0 )dx w Power of the test : how ofte do you get t wrog! 1 β L(x H 1 )dx w Neyma-Pearso lemma : optmal statstc for testg hypothess s the Lkelhood rato (x H ) L λ < L (x H ) 0 kα 1
32 CL b ad CL s Two hypothess, for coutg expermet - backgroud oly : expect 10 evets - sgal+ backgroud : expect 15 evets You observe 16 evets CL b CL s+b CL b cofdece the backgroud hypothess (power of the test) Dscovery : 1- CL b < 5.7x10-7 CL s+b cofdece the sgal+backgroud hypothess (sze of the test) Rejecto : CL s+b < 5x10 - Test for sgal (o stadard) CL s CL s+b /CL b 3
Lecture 3. Sampling, sampling distributions, and parameter estimation
Lecture 3 Samplg, samplg dstrbutos, ad parameter estmato Samplg Defto Populato s defed as the collecto of all the possble observatos of terest. The collecto of observatos we take from the populato s called
More informationEconometric Methods. Review of Estimation
Ecoometrc Methods Revew of Estmato Estmatg the populato mea Radom samplg Pot ad terval estmators Lear estmators Ubased estmators Lear Ubased Estmators (LUEs) Effcecy (mmum varace) ad Best Lear Ubased Estmators
More informationLecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model
Lecture 7. Cofdece Itervals ad Hypothess Tests the Smple CLR Model I lecture 6 we troduced the Classcal Lear Regresso (CLR) model that s the radom expermet of whch the data Y,,, K, are the outcomes. The
More informationSimulation Output Analysis
Smulato Output Aalyss Summary Examples Parameter Estmato Sample Mea ad Varace Pot ad Iterval Estmato ermatg ad o-ermatg Smulato Mea Square Errors Example: Sgle Server Queueg System x(t) S 4 S 4 S 3 S 5
More informationSummary of the lecture in Biostatistics
Summary of the lecture Bostatstcs Probablty Desty Fucto For a cotuos radom varable, a probablty desty fucto s a fucto such that: 0 dx a b) b a dx A probablty desty fucto provdes a smple descrpto of the
More informationSpecial Instructions / Useful Data
JAM 6 Set of all real umbers P A..d. B, p Posso Specal Istructos / Useful Data x,, :,,, x x Probablty of a evet A Idepedetly ad detcally dstrbuted Bomal dstrbuto wth parameters ad p Posso dstrbuto wth
More informationCHAPTER VI Statistical Analysis of Experimental Data
Chapter VI Statstcal Aalyss of Expermetal Data CHAPTER VI Statstcal Aalyss of Expermetal Data Measuremets do ot lead to a uque value. Ths s a result of the multtude of errors (maly radom errors) that ca
More informationSTK4011 and STK9011 Autumn 2016
STK4 ad STK9 Autum 6 Pot estmato Covers (most of the followg materal from chapter 7: Secto 7.: pages 3-3 Secto 7..: pages 3-33 Secto 7..: pages 35-3 Secto 7..3: pages 34-35 Secto 7.3.: pages 33-33 Secto
More informationPoint Estimation: definition of estimators
Pot Estmato: defto of estmators Pot estmator: ay fucto W (X,..., X ) of a data sample. The exercse of pot estmato s to use partcular fuctos of the data order to estmate certa ukow populato parameters.
More information22 Nonparametric Methods.
22 oparametrc Methods. I parametrc models oe assumes apror that the dstrbutos have a specfc form wth oe or more ukow parameters ad oe tres to fd the best or atleast reasoably effcet procedures that aswer
More informationTHE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA
THE ROYAL STATISTICAL SOCIETY EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA PAPER II STATISTICAL THEORY & METHODS The Socety provdes these solutos to assst caddates preparg for the examatos future years ad for
More informationUNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Postpoed exam: ECON430 Statstcs Date of exam: Jauary 0, 0 Tme for exam: 09:00 a.m. :00 oo The problem set covers 5 pages Resources allowed: All wrtte ad prted
More informationChapter 5 Properties of a Random Sample
Lecture 6 o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for the prevous lecture Cocepts: t-dstrbuto, F-dstrbuto Theorems: Dstrbutos of sample mea ad sample varace, relatoshp betwee sample mea ad sample
More informationUNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Exam: ECON430 Statstcs Date of exam: Frday, December 8, 07 Grades are gve: Jauary 4, 08 Tme for exam: 0900 am 00 oo The problem set covers 5 pages Resources allowed:
More informationLecture Notes Types of economic variables
Lecture Notes 3 1. Types of ecoomc varables () Cotuous varable takes o a cotuum the sample space, such as all pots o a le or all real umbers Example: GDP, Polluto cocetrato, etc. () Dscrete varables fte
More informationbest estimate (mean) for X uncertainty or error in the measurement (systematic, random or statistical) best
Error Aalyss Preamble Wheever a measuremet s made, the result followg from that measuremet s always subject to ucertaty The ucertaty ca be reduced by makg several measuremets of the same quatty or by mprovg
More informationENGI 4421 Joint Probability Distributions Page Joint Probability Distributions [Navidi sections 2.5 and 2.6; Devore sections
ENGI 441 Jot Probablty Dstrbutos Page 7-01 Jot Probablty Dstrbutos [Navd sectos.5 ad.6; Devore sectos 5.1-5.] The jot probablty mass fucto of two dscrete radom quattes, s, P ad p x y x y The margal probablty
More informationSimple Linear Regression
Statstcal Methods I (EST 75) Page 139 Smple Lear Regresso Smple regresso applcatos are used to ft a model descrbg a lear relatoshp betwee two varables. The aspects of least squares regresso ad correlato
More informationENGI 3423 Simple Linear Regression Page 12-01
ENGI 343 mple Lear Regresso Page - mple Lear Regresso ometmes a expermet s set up where the expermeter has cotrol over the values of oe or more varables X ad measures the resultg values of aother varable
More informationChapter 14 Logistic Regression Models
Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as
More informationSTATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. x, where. = y - ˆ " 1
STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Recall Assumpto E(Y x) η 0 + η x (lear codtoal mea fucto) Data (x, y ), (x 2, y 2 ),, (x, y ) Least squares estmator ˆ E (Y x) ˆ " 0 + ˆ " x, where ˆ
More informationEconometrics. 3) Statistical properties of the OLS estimator
30C0000 Ecoometrcs 3) Statstcal propertes of the OLS estmator Tmo Kuosmae Professor, Ph.D. http://omepre.et/dex.php/tmokuosmae Today s topcs Whch assumptos are eeded for OLS to work? Statstcal propertes
More informationMultivariate Transformation of Variables and Maximum Likelihood Estimation
Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty
More informationParameter, Statistic and Random Samples
Parameter, Statstc ad Radom Samples A parameter s a umber that descrbes the populato. It s a fxed umber, but practce we do ot kow ts value. A statstc s a fucto of the sample data,.e., t s a quatty whose
More informationTESTS BASED ON MAXIMUM LIKELIHOOD
ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal
More informationQualifying Exam Statistical Theory Problem Solutions August 2005
Qualfyg Exam Statstcal Theory Problem Solutos August 5. Let X, X,..., X be d uform U(,),
More informationESS Line Fitting
ESS 5 014 17. Le Fttg A very commo problem data aalyss s lookg for relatoshpetwee dfferet parameters ad fttg les or surfaces to data. The smplest example s fttg a straght le ad we wll dscuss that here
More informationX ε ) = 0, or equivalently, lim
Revew for the prevous lecture Cocepts: order statstcs Theorems: Dstrbutos of order statstcs Examples: How to get the dstrbuto of order statstcs Chapter 5 Propertes of a Radom Sample Secto 55 Covergece
More informationLecture Note to Rice Chapter 8
ECON 430 HG revsed Nov 06 Lecture Note to Rce Chapter 8 Radom matrces Let Y, =,,, m, =,,, be radom varables (r.v. s). The matrx Y Y Y Y Y Y Y Y Y Y = m m m s called a radom matrx ( wth a ot m-dmesoal dstrbuto,
More informationSTATISTICAL INFERENCE
(STATISTICS) STATISTICAL INFERENCE COMPLEMENTARY COURSE B.Sc. MATHEMATICS III SEMESTER ( Admsso) UNIVERSITY OF CALICUT SCHOOL OF DISTANCE EDUCATION CALICUT UNIVERSITY P.O., MALAPPURAM, KERALA, INDIA -
More informationMean is only appropriate for interval or ratio scales, not ordinal or nominal.
Mea Same as ordary average Sum all the data values ad dvde by the sample sze. x = ( x + x +... + x Usg summato otato, we wrte ths as x = x = x = = ) x Mea s oly approprate for terval or rato scales, ot
More informationChapter 8: Statistical Analysis of Simulated Data
Marquette Uversty MSCS600 Chapter 8: Statstcal Aalyss of Smulated Data Dael B. Rowe, Ph.D. Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 08 by Marquette Uversty MSCS600 Ageda 8. The Sample
More informationClass 13,14 June 17, 19, 2015
Class 3,4 Jue 7, 9, 05 Pla for Class3,4:. Samplg dstrbuto of sample mea. The Cetral Lmt Theorem (CLT). Cofdece terval for ukow mea.. Samplg Dstrbuto for Sample mea. Methods used are based o CLT ( Cetral
More informationMaximum Likelihood Estimation
Marquette Uverst Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Coprght 08 b Marquette Uverst Maxmum Lkelhood Estmato We have bee sag that ~
More informationModule 7: Probability and Statistics
Lecture 4: Goodess of ft tests. Itroducto Module 7: Probablty ad Statstcs I the prevous two lectures, the cocepts, steps ad applcatos of Hypotheses testg were dscussed. Hypotheses testg may be used to
More informationTHE ROYAL STATISTICAL SOCIETY 2016 EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA MODULE 2
THE ROYAL STATISTICAL SOCIETY 06 EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA MODULE The Socety s provdg these solutos to assst caddates preparg for the examatos 07. The solutos are teded as learg ads ad should
More information( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model
Chapter 3 Asmptotc Theor ad Stochastc Regressors The ature of eplaator varable s assumed to be o-stochastc or fed repeated samples a regresso aalss Such a assumpto s approprate for those epermets whch
More informationWu-Hausman Test: But if X and ε are independent, βˆ. ECON 324 Page 1
Wu-Hausma Test: Detectg Falure of E( ε X ) Caot drectly test ths assumpto because lack ubased estmator of ε ad the OLS resduals wll be orthogoal to X, by costructo as ca be see from the momet codto X'
More informationFunctions of Random Variables
Fuctos of Radom Varables Chapter Fve Fuctos of Radom Varables 5. Itroducto A geeral egeerg aalyss model s show Fg. 5.. The model output (respose) cotas the performaces of a system or product, such as weght,
More information{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution:
Chapter 4 Exercses Samplg Theory Exercse (Smple radom samplg: Let there be two correlated radom varables X ad A sample of sze s draw from a populato by smple radom samplg wthout replacemet The observed
More informationLecture 1 Review of Fundamental Statistical Concepts
Lecture Revew of Fudametal Statstcal Cocepts Measures of Cetral Tedecy ad Dsperso A word about otato for ths class: Idvduals a populato are desgated, where the dex rages from to N, ad N s the total umber
More informationBayes (Naïve or not) Classifiers: Generative Approach
Logstc regresso Bayes (Naïve or ot) Classfers: Geeratve Approach What do we mea by Geeratve approach: Lear p(y), p(x y) ad the apply bayes rule to compute p(y x) for makg predctos Ths s essetally makg
More informationECONOMETRIC THEORY. MODULE VIII Lecture - 26 Heteroskedasticity
ECONOMETRIC THEORY MODULE VIII Lecture - 6 Heteroskedastcty Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur . Breusch Paga test Ths test ca be appled whe the replcated data
More informationChapter 3 Sampling For Proportions and Percentages
Chapter 3 Samplg For Proportos ad Percetages I may stuatos, the characterstc uder study o whch the observatos are collected are qualtatve ature For example, the resposes of customers may marketg surveys
More informationENGI 4421 Propagation of Error Page 8-01
ENGI 441 Propagato of Error Page 8-01 Propagato of Error [Navd Chapter 3; ot Devore] Ay realstc measuremet procedure cotas error. Ay calculatos based o that measuremet wll therefore also cota a error.
More informationPhD Training in Statistics
PhD Trag Statstcs G.Wlquet. Recall o probablty. Specal dstrbutos 3. Recall o statstcs 4. Samplg dstrbutos 5. Hypothess tests 6. Estmato 7. Maxmum lkelhood 8. Least squares 9. Cofdece levels pathologcal
More informationLECTURE - 4 SIMPLE RANDOM SAMPLING DR. SHALABH DEPARTMENT OF MATHEMATICS AND STATISTICS INDIAN INSTITUTE OF TECHNOLOGY KANPUR
amplg Theory MODULE II LECTURE - 4 IMPLE RADOM AMPLIG DR. HALABH DEPARTMET OF MATHEMATIC AD TATITIC IDIA ITITUTE OF TECHOLOGY KAPUR Estmato of populato mea ad populato varace Oe of the ma objectves after
More informationSTK3100 and STK4100 Autumn 2018
SK3 ad SK4 Autum 8 Geeralzed lear models Part III Covers the followg materal from chaters 4 ad 5: Cofdece tervals by vertg tests Cosder a model wth a sgle arameter β We may obta a ( α% cofdece terval for
More informationLinear Regression with One Regressor
Lear Regresso wth Oe Regressor AIM QA.7. Expla how regresso aalyss ecoometrcs measures the relatoshp betwee depedet ad depedet varables. A regresso aalyss has the goal of measurg how chages oe varable,
More information9 U-STATISTICS. Eh =(m!) 1 Eh(X (1),..., X (m ) ) i.i.d
9 U-STATISTICS Suppose,,..., are P P..d. wth CDF F. Our goal s to estmate the expectato t (P)=Eh(,,..., m ). Note that ths expectato requres more tha oe cotrast to E, E, or Eh( ). Oe example s E or P((,
More informationLecture 02: Bounding tail distributions of a random variable
CSCI-B609: A Theorst s Toolkt, Fall 206 Aug 25 Lecture 02: Boudg tal dstrbutos of a radom varable Lecturer: Yua Zhou Scrbe: Yua Xe & Yua Zhou Let us cosder the ubased co flps aga. I.e. let the outcome
More informationContinuous Distributions
7//3 Cotuous Dstrbutos Radom Varables of the Cotuous Type Desty Curve Percet Desty fucto, f (x) A smooth curve that ft the dstrbuto 3 4 5 6 7 8 9 Test scores Desty Curve Percet Probablty Desty Fucto, f
More informationStatistics MINITAB - Lab 5
Statstcs 10010 MINITAB - Lab 5 PART I: The Correlato Coeffcet Qute ofte statstcs we are preseted wth data that suggests that a lear relatoshp exsts betwee two varables. For example the plot below s of
More informationIntroduction to local (nonparametric) density estimation. methods
Itroducto to local (oparametrc) desty estmato methods A slecture by Yu Lu for ECE 66 Sprg 014 1. Itroducto Ths slecture troduces two local desty estmato methods whch are Parze desty estmato ad k-earest
More informationDetection and Estimation Theory
ESE 54 Detecto ad Estmato heory Joseph A. O Sullva Samuel C. Sachs Professor Electroc Systems ad Sgals Research Laboratory Electrcal ad Systems Egeerg Washgto Uversty Urbauer Hall 34-935-473 (Lyda aswers
More informationSTK3100 and STK4100 Autumn 2017
SK3 ad SK4 Autum 7 Geeralzed lear models Part III Covers the followg materal from chaters 4 ad 5: Sectos 4..5, 4.3.5, 4.3.6, 4.4., 4.4., ad 4.4.3 Sectos 5.., 5.., ad 5.5. Ørulf Borga Deartmet of Mathematcs
More informationOrdinary Least Squares Regression. Simple Regression. Algebra and Assumptions.
Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos
More informationPROPERTIES OF GOOD ESTIMATORS
ESTIMATION INTRODUCTION Estmato s the statstcal process of fdg a appromate value for a populato parameter. A populato parameter s a characterstc of the dstrbuto of a populato such as the populato mea,
More informationUncertainty, Data, and Judgment
Ucertaty, Data, ad Judgmet Sesso 06 Structure of the Course Topc Sesso Probablty -5 Estmato 6-8 Hypothess Testg 9-10 Regresso 11-16 1 Mcrosoft AND Itel (50-50) You vest $,500 MSFT ad $,500 INTC X = Aual
More informationAnalysis of Variance with Weibull Data
Aalyss of Varace wth Webull Data Lahaa Watthaacheewaul Abstract I statstcal data aalyss by aalyss of varace, the usual basc assumptos are that the model s addtve ad the errors are radomly, depedetly, ad
More informationTo use adaptive cluster sampling we must first make some definitions of the sampling universe:
8.3 ADAPTIVE SAMPLING Most of the methods dscussed samplg theory are lmted to samplg desgs hch the selecto of the samples ca be doe before the survey, so that oe of the decsos about samplg deped ay ay
More information1 Solution to Problem 6.40
1 Soluto to Problem 6.40 (a We wll wrte T τ (X 1,...,X where the X s are..d. wth PDF f(x µ, σ 1 ( x µ σ g, σ where the locato parameter µ s ay real umber ad the scale parameter σ s > 0. Lettg Z X µ σ we
More informationChapter -2 Simple Random Sampling
Chapter - Smple Radom Samplg Smple radom samplg (SRS) s a method of selecto of a sample comprsg of umber of samplg uts out of the populato havg umber of samplg uts such that every samplg ut has a equal
More informationBayes Estimator for Exponential Distribution with Extension of Jeffery Prior Information
Malaysa Joural of Mathematcal Sceces (): 97- (9) Bayes Estmator for Expoetal Dstrbuto wth Exteso of Jeffery Pror Iformato Hadeel Salm Al-Kutub ad Noor Akma Ibrahm Isttute for Mathematcal Research, Uverst
More informationTHE ROYAL STATISTICAL SOCIETY 2010 EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA MODULE 2 STATISTICAL INFERENCE
THE ROYAL STATISTICAL SOCIETY 00 EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA MODULE STATISTICAL INFERENCE The Socety provdes these solutos to assst caddates preparg for the examatos future years ad for the
More informationLecture 3 Probability review (cont d)
STATS 00: Itroducto to Statstcal Iferece Autum 06 Lecture 3 Probablty revew (cot d) 3. Jot dstrbutos If radom varables X,..., X k are depedet, the ther dstrbuto may be specfed by specfyg the dvdual dstrbuto
More informationChapter 4 (Part 1): Non-Parametric Classification (Sections ) Pattern Classification 4.3) Announcements
Aoucemets No-Parametrc Desty Estmato Techques HW assged Most of ths lecture was o the blacboard. These sldes cover the same materal as preseted DHS Bometrcs CSE 90-a Lecture 7 CSE90a Fall 06 CSE90a Fall
More informationConstruction and Evaluation of Actuarial Models. Rajapaksha Premarathna
Costructo ad Evaluato of Actuaral Models Raapaksha Premaratha Table of Cotets Modelg Some deftos ad Notatos...4 Case : Polcy Lmtu...4 Case : Wth a Ordary deductble....5 Case 3: Maxmum Covered loss u wth
More informationPart 4b Asymptotic Results for MRR2 using PRESS. Recall that the PRESS statistic is a special type of cross validation procedure (see Allen (1971))
art 4b Asymptotc Results for MRR usg RESS Recall that the RESS statstc s a specal type of cross valdato procedure (see Alle (97)) partcular to the regresso problem ad volves fdg Y $,, the estmate at the
More informationPart I: Background on the Binomial Distribution
Part I: Bacgroud o the Bomal Dstrbuto A radom varable s sad to have a Beroull dstrbuto f t taes o the value wth probablt "p" ad the value wth probablt " - p". The umber of "successes" "" depedet Beroull
More informationCHAPTER 3 POSTERIOR DISTRIBUTIONS
CHAPTER 3 POSTERIOR DISTRIBUTIONS If scece caot measure the degree of probablt volved, so much the worse for scece. The practcal ma wll stck to hs apprecatve methods utl t does, or wll accept the results
More informationProbability and Statistics. What is probability? What is statistics?
robablt ad Statstcs What s robablt? What s statstcs? robablt ad Statstcs robablt Formall defed usg a set of aoms Seeks to determe the lkelhood that a gve evet or observato or measuremet wll or has haeed
More informationEvaluation of uncertainty in measurements
Evaluato of ucertaty measuremets Laboratory of Physcs I Faculty of Physcs Warsaw Uversty of Techology Warszawa, 05 Itroducto The am of the measuremet s to determe the measured value. Thus, the measuremet
More informationSome Applications of the Resampling Methods in Computational Physics
Iteratoal Joural of Mathematcs Treds ad Techoloy Volume 6 February 04 Some Applcatos of the Resampl Methods Computatoal Physcs Sotraq Marko #, Lorec Ekoom * # Physcs Departmet, Uversty of Korca, Albaa,
More informationX X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then
Secto 5 Vectors of Radom Varables Whe workg wth several radom varables,,..., to arrage them vector form x, t s ofte coveet We ca the make use of matrx algebra to help us orgaze ad mapulate large umbers
More informationChapter -2 Simple Random Sampling
Chapter - Smple Radom Samplg Smple radom samplg (SRS) s a method of selecto of a sample comprsg of umber of samplg uts out of the populato havg umber of samplg uts such that every samplg ut has a equal
More informationTHE ROYAL STATISTICAL SOCIETY HIGHER CERTIFICATE
THE ROYAL STATISTICAL SOCIETY 00 EXAMINATIONS SOLUTIONS HIGHER CERTIFICATE PAPER I STATISTICAL THEORY The Socety provdes these solutos to assst caddates preparg for the examatos future years ad for the
More informationChapter 8. Inferences about More Than Two Population Central Values
Chapter 8. Ifereces about More Tha Two Populato Cetral Values Case tudy: Effect of Tmg of the Treatmet of Port-We tas wth Lasers ) To vestgate whether treatmet at a youg age would yeld better results tha
More information: At least two means differ SST
Formula Card for Eam 3 STA33 ANOVA F-Test: Completely Radomzed Desg ( total umber of observatos, k = Number of treatmets,& T = total for treatmet ) Step : Epress the Clam Step : The ypotheses: :... 0 A
More informationIdea is to sample from a different distribution that picks points in important regions of the sample space. Want ( ) ( ) ( ) E f X = f x g x dx
Importace Samplg Used for a umber of purposes: Varace reducto Allows for dffcult dstrbutos to be sampled from. Sestvty aalyss Reusg samples to reduce computatoal burde. Idea s to sample from a dfferet
More informationOverview. Basic concepts of Bayesian learning. Most probable model given data Coin tosses Linear regression Logistic regression
Overvew Basc cocepts of Bayesa learg Most probable model gve data Co tosses Lear regresso Logstc regresso Bayesa predctos Co tosses Lear regresso 30 Recap: regresso problems Iput to learg problem: trag
More informationMultiple Regression. More than 2 variables! Grade on Final. Multiple Regression 11/21/2012. Exam 2 Grades. Exam 2 Re-grades
STAT 101 Dr. Kar Lock Morga 11/20/12 Exam 2 Grades Multple Regresso SECTIONS 9.2, 10.1, 10.2 Multple explaatory varables (10.1) Parttog varablty R 2, ANOVA (9.2) Codtos resdual plot (10.2) Trasformatos
More informationA practical threshold estimation for jump processes
A practcal threshold estmato for jump processes Yasutaka Shmzu (Osaka Uversty, Japa) WORKSHOP o Face ad Related Mathematcal ad Statstcal Issues @ Kyoto, JAPAN, 3 6 Sept., 2008. Itroducto O (Ω, F,P; {F
More informationJAM 2015: General Instructions during Examination
JAM 05 JAM 05: Geeral Istructos durg Examato. Total durato of the JAM 05 examato s 80 mutes.. The clock wll be set at the server. The coutdow tmer at the top rght corer of scree wll dsplay the remag tme
More informationECON 5360 Class Notes GMM
ECON 560 Class Notes GMM Geeralzed Method of Momets (GMM) I beg by outlg the classcal method of momets techque (Fsher, 95) ad the proceed to geeralzed method of momets (Hase, 98).. radtoal Method of Momets
More informationBIOREPS Problem Set #11 The Evolution of DNA Strands
BIOREPS Problem Set #11 The Evoluto of DNA Strads 1 Backgroud I the md 2000s, evolutoary bologsts studyg DNA mutato rates brds ad prmates dscovered somethg surprsg. There were a large umber of mutatos
More informationLikelihood Ratio, Wald, and Lagrange Multiplier (Score) Tests. Soccer Goals in European Premier Leagues
Lkelhood Rato, Wald, ad Lagrage Multpler (Score) Tests Soccer Goals Europea Premer Leagues - 4 Statstcal Testg Prcples Goal: Test a Hpothess cocerg parameter value(s) a larger populato (or ature), based
More informationMedian as a Weighted Arithmetic Mean of All Sample Observations
Meda as a Weghted Arthmetc Mea of All Sample Observatos SK Mshra Dept. of Ecoomcs NEHU, Shllog (Ida). Itroducto: Iumerably may textbooks Statstcs explctly meto that oe of the weakesses (or propertes) of
More informationSTA 105-M BASIC STATISTICS (This is a multiple choice paper.)
DCDM BUSINESS SCHOOL September Mock Eamatos STA 0-M BASIC STATISTICS (Ths s a multple choce paper.) Tme: hours 0 mutes INSTRUCTIONS TO CANDIDATES Do ot ope ths questo paper utl you have bee told to do
More informationThe number of observed cases The number of parameters. ith case of the dichotomous dependent variable. the ith case of the jth parameter
LOGISTIC REGRESSION Notato Model Logstc regresso regresses a dchotomous depedet varable o a set of depedet varables. Several methods are mplemeted for selectg the depedet varables. The followg otato s
More informationKLT Tracker. Alignment. 1. Detect Harris corners in the first frame. 2. For each Harris corner compute motion between consecutive frames
KLT Tracker Tracker. Detect Harrs corers the frst frame 2. For each Harrs corer compute moto betwee cosecutve frames (Algmet). 3. Lk moto vectors successve frames to get a track 4. Itroduce ew Harrs pots
More informationQuantitative analysis requires : sound knowledge of chemistry : possibility of interferences WHY do we need to use STATISTICS in Anal. Chem.?
Ch 4. Statstcs 4.1 Quattatve aalyss requres : soud kowledge of chemstry : possblty of terfereces WHY do we eed to use STATISTICS Aal. Chem.? ucertaty ests. wll we accept ucertaty always? f ot, from how
More informationSimple Linear Regression
Correlato ad Smple Lear Regresso Berl Che Departmet of Computer Scece & Iformato Egeerg Natoal Tawa Normal Uversty Referece:. W. Navd. Statstcs for Egeerg ad Scetsts. Chapter 7 (7.-7.3) & Teachg Materal
More informationModule 7. Lecture 7: Statistical parameter estimation
Lecture 7: Statstcal parameter estmato Parameter Estmato Methods of Parameter Estmato 1) Method of Matchg Pots ) Method of Momets 3) Mamum Lkelhood method Populato Parameter Sample Parameter Ubased estmato
More informationObjectives of Multiple Regression
Obectves of Multple Regresso Establsh the lear equato that best predcts values of a depedet varable Y usg more tha oe eplaator varable from a large set of potetal predctors {,,... k }. Fd that subset of
More informationTHE ROYAL STATISTICAL SOCIETY 2009 EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA MODULAR FORMAT MODULE 2 STATISTICAL INFERENCE
THE ROYAL STATISTICAL SOCIETY 009 EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA MODULAR FORMAT MODULE STATISTICAL INFERENCE The Socety provdes these solutos to assst caddates preparg for the examatos future
More informationChapter 4 Multiple Random Variables
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for Chapter 4-5 Notes: Although all deftos ad theorems troduced our lectures ad ths ote are mportat ad you should be famlar wth, but I put those
More informationChapter 11 The Analysis of Variance
Chapter The Aalyss of Varace. Oe Factor Aalyss of Varace. Radomzed Bloc Desgs (ot for ths course) NIPRL . Oe Factor Aalyss of Varace.. Oe Factor Layouts (/4) Suppose that a expermeter s terested populatos
More informationGENERALIZED METHOD OF MOMENTS CHARACTERISTICS AND ITS APPLICATION ON PANELDATA
Sc.It.(Lahore),26(3),985-990,2014 ISSN 1013-5316; CODEN: SINTE 8 GENERALIZED METHOD OF MOMENTS CHARACTERISTICS AND ITS APPLICATION ON PANELDATA Beradhta H. S. Utam 1, Warsoo 1, Da Kurasar 1, Mustofa Usma
More information6.867 Machine Learning
6.867 Mache Learg Problem set Due Frday, September 9, rectato Please address all questos ad commets about ths problem set to 6.867-staff@a.mt.edu. You do ot eed to use MATLAB for ths problem set though
More information