Kernel density estimator

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Kernel density estimator"

Transcription

1 Jauary, 07 NONPARAMETRIC ERNEL DENSITY ESTIMATION I this lecture, we discuss kerel estimatio of probability desity fuctios PDF Noparametric desity estimatio is oe of the cetral problems i statistics I ecoomics, oparametric desity estimatio plays importat roles i various areas such as, for example, idustrial orgaizatio Guerre et al, 000, empirical fiace Ait-Sahalia, 996, ad etc These otes borrow from the followig sources: Li ad Racie 007, Paga ad Ullah 999, ad Härdle ad Lito 994 erel desity estimator Assumptio a Suppose {X i : i =,, } is a collectio of iid radom variables draw from a distributio with the CDF F ad PDF f b I the eighborhood N x of x, f is bouded ad twice cotiuously differetiable with bouded derivatives Whe discussig f x, we will implicitly assume that f x exists at x The ecoometricia s objective is to estimate f without imposig ay fuctioal form parametric assumptios o the PDF First, cosider estimatio of F Sice a estimator of F ca be costructed as F x = E {X i x}, ˆF x = {X i x} The fuctio ˆF x is called the empirical CDF of X i The WLLN implies that for all x, i= ˆF x p F x As a matter of fact, a stroger results ca be established Gliveko-Gatelli Theorem, see Chapter 9 of va der Vaart 998: sup ˆF x F x as 0 x R Next, by the CLT, / ˆF x F x d N 0, F x F x Furthermore, for ay x, x R, / ˆF x F x ad / ˆF x F x are joitly asymptotically ormal with mea zero ad the covariace F x x F x F x, where x x deotes the miimum betwee x ad x Sice df x F x + h F x h f x = = lim, dx h 0 h from, oe ca cosider the followig estimator for the PDF f: ˆf x = ˆF x + h ˆF x h h = {x h X i x + h }, h i=

2 where h is a small umber ote that we cosider cotiuously distributed radom variables, so that P X i = x h = 0 We write h istead of just h because, typically, it will be a fuctio of the sample size such that lim h = 0 Now, defie the followig kerel fuctio: The, the kerel PDF estimator is give by u = { u } ˆf x = h 3 i= Thus, with the kerel fuctio defied accordig to, the kerel desity estimator is a average umber of observatios i the small eighborhood of x as defied by the smoothig parameter or badwidth also kerel widow The kerel fuctio i is called uiform, because it correspods to the uiform distributio we have that u du = It has a disadvatage of givig equal weights to all observatios iside the h -widow with the ceter at x, regardless of how close they are to the ceter Also, if oe cosiders ˆf x as a fuctio of x, it is rough havig jumps at the poits X i ±h, ad has a zero derivative everywhere else Those problems ca be resolved if oe cosiders alterative kerel fuctios, for example, the quadratic kerel: h u = 5 u { u } 6 The class of estimators 3 with a kerel satisfyig u du = is referred to as Roseblatt-Parze erel Estimator Small sample properties of the kerel desity estimator We will make the followig assumptio cocerig : Assumptio a u du = b u = u c is compactly supported o [, ] ad bouded d u u du 0 The kerel desity estimator is biased: Lemma Uder Assumptios a ad a, E ˆf x f x = u f x + uh f x du Proof E ˆf x = h E i= = h E = h h h u x h f u du Next, usig chage of variable y = u x /h, u = x + yh, ad du = h dy, we obtai E ˆf x = u f x + uh du, ad the result follows sice f x u du = f x by Assumptio a

3 Lemma Uder Assumptios a ad a, the variace of ˆf x is give by V ar ˆf x = h u f x + uh du u f x + uh du Proof Sice the data are iid, V ar ˆf x = V ar h h = E E h h h h By the same chage of variable argumet as i the proof of Lemma, we obtai E u x = f u du h h = h u f x + uh du From Lemma, oe ca expect that the bias icreases with h ; a bigger badwidth implies that more observatios away from x have o-zero weights which cotributes to the bias O the other had, the variace decreases with h, as the estimator averages over more observatios The theorem below establishes more formally the bias-variace trade-off for the kerel estimator Let f s deote the s-order derivative of f: f s x = ds f x dx s Theorem Suppose that h 0 ad h as The, uder Assumptios ad, a E ˆf x f x = c x h + o h, where c x = f x u u du/ b V ar ˆf x = c x / h + O /, where c x = f x u du Proof Sice the first two derivative of f exist by Assumptio b, cosider the followig expasio for f x + uh : f x + uh = f x + f x uh + f x u u h, where x u lies betwee x ad x + uh From Lemma we have E ˆf x f x = u f x uh + f x u u h du = h u f x u u du = c h + h u f x u f x u du 4 The secod equality follows because by Assumptio b, u udu = 0 We will show ext that u f x u f x u du = o, 5 3

4 ad therefore the secod summad i 4 is o h By Assumptio c, we oly eed to cosider u ; by Assumptio b ad sice x u lies betwee x ad x + uh, f x u f f x sup z < z N x Next, sice h 0, Now, by the domiated covergece theorem, lim lim x u = x u f x u f x u du = = 0, u lim f x u f x u du which establishes 5 ad cocludes the proof of part a of the theorem For part b, u f x + uh du = f x u du h h + f x u udu + h u f x u u du = c h + O h, 6 sice u udu = 0 by symmetry Assumptio b, ad u f x u u du = O as i the proof of part a The result of part b follows from 6 ad Lemma Agai, Theorem shows the bias-variace trade-off The optimal choice of badwidth ca be foud by miimizig some fuctio that combies bias ad variace, for example, the mea squared error MSE: MSE ˆf x = E ˆf x f x = V ar ˆf x + E ˆf x f x = c x h = c x h Miimizatio of the leadig term of MSE gives + c x h 4 + O + c x h 4 + o 4c x h 3 = c x h, or + o h 4 h + h 4 7 h = = /5 c x 4c x /5 f x u du /5 f x u u du /5 /5 4

5 Whe the optimal i the MSE sese badwidth is selected, either bias or variace compoets of the MSE domiate each other asymptotically as V ar = Bias = O 4/5 Whe the Itegrated MSE criterio is employed, MSE ˆf x dx, the optimal badwidth becomes u du /5 h = f x /5 dx u u du /5 /5 Let ˆσ deote the sample variace of the data The followig rules of thumb ofte used i practice: h = 364ˆσ u du /5 u u du /5 /5, which is optimal for f x N µ, σ, ad h = 06ˆσ /5, which is optimal for f x N µ, σ ad whe is the stadard ormal desity Cosistecy of the kerel desity estimator Cosistecy of ˆf x follows immediately from Theorem by Chebychev s iequality Corollary Suppose that h 0 ad h as The, uder Assumptios ad, ˆf x p f x Proof By Chebychev s iequality, P ˆf x f x > ε E ˆf x f x ε = c x ε + c x h 4 h ε 0, + o + h 4 h where the secod lie is by 7 A stroger result ca be give, see Newey 994 Suppose that f admits at least m cotiuous derivatives o some iterval [x, x ]; has at least m cotiuous derivatives, is compactly supported ad of order m: u j u du = 0 for all j =,, m ; u m u du 0, ad u du = The sup x [x,x ] ˆf / h x f x = O p + h m log The derivatives of f ca be estimated by the derivative of ˆf, however, with a slower rates of covergece Newey 994 shows that sup x [x,x ] ˆf k h x f k k / x = O p + h m log 5

6 Asymptotic ormality of the kerel desity estimator Write ˆf x = h i= h = v i, where v i = h i= h Note that h ad cosequetly v i deped o The collectio {{v i : i =,, } : N} is called a triagular array I our case, uder Assumptio, v i s are iid The followig CLT is available for idepedet triagular arrays Lehma ad Romao, 005, Corollary, page 47 Lemma 3 Lyapouov CLT Suppose that for each, w,, w are idepedet Assume that Ew i = 0 ad σi = Ew i <, ad defie s = i= σ i Suppose further that for some δ > 0 the followig coditio holds: lim E w i +δ = 0 8 The, i= s +δ w i /s d N 0, i= The coditio 8 is called Lyapouov s coditio Whe the data are ot just idepedet but iid, the Lyapouov s coditio ca be simplified as follows Davidso, 994, Theorem 3 o page 373 Lemma 4 The Lyapouov s coditio is satisfied whe w,, w are iid, σ = Ewi > 0 uiformly i, ad lim E w i +δ / δ/ = 0 for some δ > 0 Proof Sice the data are iid, We have i= s +δ s = σ, ad +δ s +δ = / σ = +δ/ σ +δ E w i +δ = σ δ δ/ E w i +δ i= = σ δ δ/ E w i +δ Therefore, the Lyapouov s coditio is satisfied if δ/ E w i +δ 0, sice σ is uiformly bouded away from zero by the assumptio Assumig that lim σ exists, i the iid case, the result of Lyapouov CLT ca be stated as follows Corollary Suppose that for each, w,, w are iid, Ew i = 0, lim Ewi lim E w i +δ / δ/ = 0 for some δ > 0 The, / w i d N i= 0, lim Ew i > 0 ad fiite, ad 6

7 Next, we prove asymptotic ormality of the kerel desity estimator Theorem Suppose that h ad h / h 0 Assume further that f x > 0 The, uder Assumptios ad, h / ˆf x f x d N 0, f x u du 9 Furthermore, for x x, h / ˆf x f x ad h / ˆf x f x are asymptotically idepedet Proof By Theorem a, Defie The, h / ˆf x f x = h E w i = h / / h / h i= + h / h E h h / ˆf x f x h = = h f x = O h / / h / E h h / f x E h w i + O p h / h 0 i= w i + o p, where the equality i the secod lie is by the assumptio that h / h 0 It is ow left to verify the coditios of Corollary By the defiitio of w i, Ew i = 0 Next, Ewi = E E h h h h As i the proof of Lemma ad by the domiated covergece theorem, E = h u f x + uh du h i= = O h, so that the secod summad i is O h ad asymptotically egligible For the first term i, we ca use the chage of variable argumet agai: E = u x du h h h h = u f x + uh du, f x u du, 3 7

8 where the last result is by the domiated covergece theorem The results i -3 together imply that lim Ew i = f x u du Lastly, we show that E w i +δ / δ/ 0 We will use the c r iequality Davidso, 994, Theorem 98 o page 40 i order to deal with E w i +δ : for r > 0, m r m E X i c r E X i r, i= where c r = whe r, ad c r = m r whe r Now, by the c r iequality, E w i +δ +δ E h +δ/ +δ + h h +δ/ E +δ h By, Further, h +δ/ h +δ/ E +δ h i= E +δ h = = h +δ/ h δ/ = O h δ/ = O h +δ/ u x h +δ f u du u +δ f x + uh du where the equality i the last lie is agai by the domiated covergece theorem Hece, E w i +δ δ/ = O δ/ h This completes the proof of 9 I order to show asymptotic idepedece of ˆf x ad ˆf x, cosider their asymptotic covariace: E = u x u x f u du h h h h h h = u u + x x f x + uh du h Sice the kerel fuctio is compactly supported ad lim x x /h =, lim u + x x = 0, h ad by the domiated covergece theorem, u lim u + x x h Asymptotic idepedece the follows by the Cramer-Wold device, f x + uh du = 0 8

9 From 0, oe ca see that the assumptio h / h 0 is used to make the bias asymptotically egligible Cosequetly, there is uder-smoothig relatively to the MSE-optimal badwidth, ad the bias goes to zero at a faster rate tha the variace Suppose that the badwidth is chose accordig to h = c α The, h / h / α/ α = 5α/, ad for h / h 0 to hold, we eed that 5α < 0 or α > /5 Thus, for asymptotic ormality, the badwidth is o /5, while the MSE-optimal badwidth is h = c /5 A more geeral statemet of the asymptotic ormality result that also icludes the bias result, ie without imposig uder-smoothig is h / ˆf x fx 05h f x u udu d N 0, fx udu 4 The result i 4 holds provided that h ad does ot require that h / h 0 I particular, if oe chooses h = ah /5, the h / ˆf x fx d N a5/ f x u udu, fx udu, ad the kerel desity estimator is asymptotically biased Multivariate kerel desity estimatio ad the curse of dimesioality Suppose ow that {X i : i =,, } is a collectio of iid radom d-vectors draw from a distributio with a joit PDF f x,, x d The uivariate kerel desity estimator ca be exteded to the multivariate case as follows: ˆf x,, x d = d i= j= h Xij x j h = h d i= j= d Xij x j ote h d i the deomiator istead of h Oe ca see that the multivariate kerel desity estimator is a extesio of uivariate kerel smoothig to d dimesios or d variables I the multivariate case, oe ca establish results similar to those of the uivariate case To simplify the otatio, let ad write Also for u = u,, u d R d, let x = x x d R d, fx,, x d = fx d u = d u j, j= h, 9

10 so that ˆf x,, x d = ˆf x = h d i= dx i x/h Note that d udu = u du d u d du d = u du =, where the secod equality follows by Assumptio a Similar results to those show for the uivariate estimator ca be established i the multivariate case Assumptio 3 a Suppose {X i : i =,, } is a collectio of iid radom vectors draw from the distributio with a joit PDF f b I the eighborhood N x of x, f is bouded ad twice cotiuously differetiable with bouded partial derivatives Theorem 3 Suppose that h 0 ad h as The, uder Assumptios ad 3, fx x j a E ˆf x f x = fx + h u d u du j= b V ar ˆf x = fx u h d du + O /h d + / + oh Proof For part a, E ˆf x = h d u x d fudu = h d vfx + h vdv, where we used the chage of variable v = u x/h, u = x + h v, du j du = du du d, dv = dv dv d Next, = h dv j for j =,, d, ad fx + h v = fx + h v fx x + h v fx v x x v, where x v deotes the mea-value satisfyig x v x h v, ie it lies betwee x ad x + h v Sice the kerel fuctio is symmetric aroud zero, v d vdv = 0 By Assumptio 3b ad the same argumets as i the proof of Theorem a, fx v x x fx x x Hece, E ˆf x = fx + h v fx x x v dvdv + oh = fx + d d fx h v i v j d vdv + oh x i x j = fx + h = fx + h d j= i= j= fx x j v v dv d j= v j v j dv j + h fx x j d i= j i + oh, = o fx x i x j v i v i dv i v j v j dv j + oh 0

11 where the equality i the last lie holds due to the symmetry of the kerel fuctio u aroud zero For part b, V ar ˆf x = V ar d h d h [ = h d Ed h = [ h d Ed h = [ h d Ed h = h d Ed h = u x h d d h = h d dufx + h udu + O = h d du fx + h u fx x + h u fx v x x = d h h d fx u du + O h d +, ] h d E d h fx + Oh ] holds by the result i a ] f x + Oh + O fudu + O u du + O where the last lie follows sice ud udu = 0 due to the symmetry of the kerel fuctio aroud zero, ad because d udu = d u du The bias ad variace calculatios imply that ˆf x = fx + O p + h h d, ad therefore the rate of covergece slows dow with the umber of variables d I the oparametric literature, this is referred to as the curse of dimesioality Oe ca derive the MSE-poit-optimal badwidth cosiderig the leadig terms i the bias ad variace expressios implied by Theorem 3: MSE x = 4 h4 u u du Miimizig the MSE with respect to h, we obtai: h 3 u u du d j= Therefore, the MSE optimal badwidth is give by d j= fx x j fx d u du h = d u d u du j= fx x j fx x j + fx d u du h d = d fx d u du h d+ /d+4 /d+4

12 Oe ca see that the rate of the optimal badwidth, /d+4, icreases with the umber of variables d, ie oe should use larger values for the badwidth whe there are more variables Oe ca exted the uivariate CLT to the multivariate case as follows: h d / ˆf x fx d h u d fx u du x d N 0, fx u du j= j To elimiate the asymptotic bias, oe has to choose a uder-smoothig badwidth so that h d / h 0 Oe ca also icorporate differet badwidth values for differet variables: ˆf x,, x d = h h h d i= j= The bias ad variace results i this case take the followig form: d Xij x j E ˆf x = fx + d u fx u du h j + o h + + h d x j= j V ar ˆf x = fx d u du h + O + + h d + h h d h h d h j Aalogously, the CLT statemet ca be modified as h h d / ˆf x fx d u u du j= fx x j h j d N 0, fx u du d

13 Bibliography Ait-Sahalia, Y 996: Testig Cotiuous-Time Models of the Spot Iterest Rate, The Review of Fiacial Studies, 9, Davidso, J 994: Stochastic Limit Theory, New York: Oxford Uiversity Press Guerre, E, I Perrige, ad Q Vuog 000: Optimal Noparametric Estimatio of First-Price Auctios, Ecoometrica, 68, Härdle, W ad O Lito 994: Applied Noparametric Methods, i Hadbook of Ecoometrics, ed by R F Egle ad D L McFadde, Amsterdam: Elsevier, vol 4, chap 38, Lehma, E L ad J P Romao 005: Testig Statistical Hypotheses, New York: Spriger, third ed Li, Q ad J S Racie 007: Noparametric Ecoometrics: Theory ad Practice, Priceto, New Jersey: Priceto Uiversity Press Newey, W 994: erel Estimatio of Partial Meas ad a Geeral Variace Estimator, Ecoometric Theory, 0, Paga, A ad A Ullah 999: Noparametric Ecoometrics, New York: Cambridge Uiversity Press va der Vaart, A W 998: Asymptotic Statistics, Cambridge: Cambridge Uiversity Press 3

LECTURE 2 LEAST SQUARES CROSS-VALIDATION FOR KERNEL DENSITY ESTIMATION

LECTURE 2 LEAST SQUARES CROSS-VALIDATION FOR KERNEL DENSITY ESTIMATION Jauary 3 07 LECTURE LEAST SQUARES CROSS-VALIDATION FOR ERNEL DENSITY ESTIMATION Noparametric kerel estimatio is extremely sesitive to te coice of badwidt as larger values of result i averagig over more

More information

LECTURE 8: ASYMPTOTICS I

LECTURE 8: ASYMPTOTICS I LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

Lecture 19: Convergence

Lecture 19: Convergence Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may

More information

Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.

Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n. Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator

More information

Convergence of random variables. (telegram style notes) P.J.C. Spreij

Convergence of random variables. (telegram style notes) P.J.C. Spreij Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space

More information

Lecture 33: Bootstrap

Lecture 33: Bootstrap Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece

More information

Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.

Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f. Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,

More information

32 estimating the cumulative distribution function

32 estimating the cumulative distribution function 32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio

More information

Study the bias (due to the nite dimensional approximation) and variance of the estimators

Study the bias (due to the nite dimensional approximation) and variance of the estimators 2 Series Methods 2. Geeral Approach A model has parameters (; ) where is ite-dimesioal ad is oparametric. (Sometimes, there is o :) We will focus o regressio. The fuctio is approximated by a series a ite

More information

Sequences and Series of Functions

Sequences and Series of Functions Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges

More information

Distribution of Random Samples & Limit theorems

Distribution of Random Samples & Limit theorems STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to

More information

Lecture 8: Convergence of transformations and law of large numbers

Lecture 8: Convergence of transformations and law of large numbers Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

ECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors

ECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

Lecture 20: Multivariate convergence and the Central Limit Theorem

Lecture 20: Multivariate convergence and the Central Limit Theorem Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece

More information

Kolmogorov-Smirnov type Tests for Local Gaussianity in High-Frequency Data

Kolmogorov-Smirnov type Tests for Local Gaussianity in High-Frequency Data Proceedigs 59th ISI World Statistics Cogress, 5-30 August 013, Hog Kog (Sessio STS046) p.09 Kolmogorov-Smirov type Tests for Local Gaussiaity i High-Frequecy Data George Tauche, Duke Uiversity Viktor Todorov,

More information

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i

More information

Unbiased Estimation. February 7-12, 2008

Unbiased Estimation. February 7-12, 2008 Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom

More information

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1 EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum

More information

Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.

Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1. Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio

More information

1 The Haar functions and the Brownian motion

1 The Haar functions and the Brownian motion 1 The Haar fuctios ad the Browia motio 1.1 The Haar fuctios ad their completeess The Haar fuctios The basic Haar fuctio is 1 if x < 1/2, ψx) = 1 if 1/2 x < 1, otherwise. 1.1) It has mea zero 1 ψx)dx =,

More information

ELEN6887 Lecture 13: Maximum Likelihood Estimation and Complexity Regularization

ELEN6887 Lecture 13: Maximum Likelihood Estimation and Complexity Regularization ELEN6887 Lecture 3: Maximum Likelihood Estimatio ad Complexity Regularizatio R Castro 3/0/00 Review : Maximum Likelihood Estimatio We have iid observatios draw from a ukow distributio Y i iid p θ, i,,

More information

An Introduction to Asymptotic Theory

An Introduction to Asymptotic Theory A Itroductio to Asymptotic Theory Pig Yu School of Ecoomics ad Fiace The Uiversity of Hog Kog Pig Yu (HKU) Asymptotic Theory 1 / 20 Five Weapos i Asymptotic Theory Five Weapos i Asymptotic Theory Pig Yu

More information

Notes 5 : More on the a.s. convergence of sums

Notes 5 : More on the a.s. convergence of sums Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series

More information

ECE 901 Lecture 14: Maximum Likelihood Estimation and Complexity Regularization

ECE 901 Lecture 14: Maximum Likelihood Estimation and Complexity Regularization ECE 90 Lecture 4: Maximum Likelihood Estimatio ad Complexity Regularizatio R Nowak 5/7/009 Review : Maximum Likelihood Estimatio We have iid observatios draw from a ukow distributio Y i iid p θ, i,, where

More information

Integrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number

Integrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios

More information

17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15

17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15 17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig

More information

MA Advanced Econometrics: Properties of Least Squares Estimators

MA Advanced Econometrics: Properties of Least Squares Estimators MA Advaced Ecoometrics: Properties of Least Squares Estimators Karl Whela School of Ecoomics, UCD February 5, 20 Karl Whela UCD Least Squares Estimators February 5, 20 / 5 Part I Least Squares: Some Fiite-Sample

More information

1 Convergence in Probability and the Weak Law of Large Numbers

1 Convergence in Probability and the Weak Law of Large Numbers 36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec

More information

Lecture 27: Optimal Estimators and Functional Delta Method

Lecture 27: Optimal Estimators and Functional Delta Method Stat210B: Theoretical Statistics Lecture Date: April 19, 2007 Lecture 27: Optimal Estimators ad Fuctioal Delta Method Lecturer: Michael I. Jorda Scribe: Guilherme V. Rocha 1 Achievig Optimal Estimators

More information

Lecture 7: Density Estimation: k-nearest Neighbor and Basis Approach

Lecture 7: Density Estimation: k-nearest Neighbor and Basis Approach STAT 425: Itroductio to Noparametric Statistics Witer 28 Lecture 7: Desity Estimatio: k-nearest Neighbor ad Basis Approach Istructor: Ye-Chi Che Referece: Sectio 8.4 of All of Noparametric Statistics.

More information

This section is optional.

This section is optional. 4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore

More information

ECE 901 Lecture 12: Complexity Regularization and the Squared Loss

ECE 901 Lecture 12: Complexity Regularization and the Squared Loss ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality

More information

6.3 Testing Series With Positive Terms

6.3 Testing Series With Positive Terms 6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial

More information

Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].

Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)]. Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x

More information

Stat 421-SP2012 Interval Estimation Section

Stat 421-SP2012 Interval Estimation Section Stat 41-SP01 Iterval Estimatio Sectio 11.1-11. We ow uderstad (Chapter 10) how to fid poit estimators of a ukow parameter. o However, a poit estimate does ot provide ay iformatio about the ucertaity (possible

More information

1 of 7 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 6. Order Statistics Defiitios Suppose agai that we have a basic radom experimet, ad that X is a real-valued radom variable

More information

Lecture 3. Properties of Summary Statistics: Sampling Distribution

Lecture 3. Properties of Summary Statistics: Sampling Distribution Lecture 3 Properties of Summary Statistics: Samplig Distributio Mai Theme How ca we use math to justify that our umerical summaries from the sample are good summaries of the populatio? Lecture Summary

More information

Lecture 15: Density estimation

Lecture 15: Density estimation Lecture 15: Desity estimatio Why do we estimate a desity? Suppose that X 1,...,X are i.i.d. radom variables from F ad that F is ukow but has a Lebesgue p.d.f. f. Estimatio of F ca be doe by estimatig f.

More information

1 Approximating Integrals using Taylor Polynomials

1 Approximating Integrals using Taylor Polynomials Seughee Ye Ma 8: Week 7 Nov Week 7 Summary This week, we will lear how we ca approximate itegrals usig Taylor series ad umerical methods. Topics Page Approximatig Itegrals usig Taylor Polyomials. Defiitios................................................

More information

7.1 Convergence of sequences of random variables

7.1 Convergence of sequences of random variables Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite

More information

Random Variables, Sampling and Estimation

Random Variables, Sampling and Estimation Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig

More information

Chapter 6 Infinite Series

Chapter 6 Infinite Series Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat

More information

Statistical Inference Based on Extremum Estimators

Statistical Inference Based on Extremum Estimators T. Rotheberg Fall, 2007 Statistical Iferece Based o Extremum Estimators Itroductio Suppose 0, the true value of a p-dimesioal parameter, is kow to lie i some subset S R p : Ofte we choose to estimate 0

More information

The variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.

The variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2. SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample

More information

Definition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.

Definition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4. 4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad

More information

CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5

CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5 CS434a/54a: Patter Recogitio Prof. Olga Veksler Lecture 5 Today Itroductio to parameter estimatio Two methods for parameter estimatio Maimum Likelihood Estimatio Bayesia Estimatio Itroducto Bayesia Decisio

More information

Elements of Statistical Methods Lots of Data or Large Samples (Ch 8)

Elements of Statistical Methods Lots of Data or Large Samples (Ch 8) Elemets of Statistical Methods Lots of Data or Large Samples (Ch 8) Fritz Scholz Sprig Quarter 2010 February 26, 2010 x ad X We itroduced the sample mea x as the average of the observed sample values x

More information

Lecture 2: Monte Carlo Simulation

Lecture 2: Monte Carlo Simulation STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?

More information

A RANK STATISTIC FOR NON-PARAMETRIC K-SAMPLE AND CHANGE POINT PROBLEMS

A RANK STATISTIC FOR NON-PARAMETRIC K-SAMPLE AND CHANGE POINT PROBLEMS J. Japa Statist. Soc. Vol. 41 No. 1 2011 67 73 A RANK STATISTIC FOR NON-PARAMETRIC K-SAMPLE AND CHANGE POINT PROBLEMS Yoichi Nishiyama* We cosider k-sample ad chage poit problems for idepedet data i a

More information

FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures

FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING Lectures MODULE 5 STATISTICS II. Mea ad stadard error of sample data. Biomial distributio. Normal distributio 4. Samplig 5. Cofidece itervals

More information

Journal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula

Journal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula Joural of Multivariate Aalysis 102 (2011) 1315 1319 Cotets lists available at ScieceDirect Joural of Multivariate Aalysis joural homepage: www.elsevier.com/locate/jmva Superefficiet estimatio of the margials

More information

STA Object Data Analysis - A List of Projects. January 18, 2018

STA Object Data Analysis - A List of Projects. January 18, 2018 STA 6557 Jauary 8, 208 Object Data Aalysis - A List of Projects. Schoeberg Mea glaucomatous shape chages of the Optic Nerve Head regio i aimal models 2. Aalysis of VW- Kedall ati-mea shapes with a applicatio

More information

STAT Homework 1 - Solutions

STAT Homework 1 - Solutions STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better

More information

Exponential Families and Bayesian Inference

Exponential Families and Bayesian Inference Computer Visio Expoetial Families ad Bayesia Iferece Lecture Expoetial Families A expoetial family of distributios is a d-parameter family f(x; havig the followig form: f(x; = h(xe g(t T (x B(, (. where

More information

Jacob Hays Amit Pillay James DeFelice 4.1, 4.2, 4.3

Jacob Hays Amit Pillay James DeFelice 4.1, 4.2, 4.3 No-Parametric Techiques Jacob Hays Amit Pillay James DeFelice 4.1, 4.2, 4.3 Parametric vs. No-Parametric Parametric Based o Fuctios (e.g Normal Distributio) Uimodal Oly oe peak Ulikely real data cofies

More information

Notes On Median and Quantile Regression. James L. Powell Department of Economics University of California, Berkeley

Notes On Median and Quantile Regression. James L. Powell Department of Economics University of California, Berkeley Notes O Media ad Quatile Regressio James L. Powell Departmet of Ecoomics Uiversity of Califoria, Berkeley Coditioal Media Restrictios ad Least Absolute Deviatios It is well-kow that the expected value

More information

Advanced Stochastic Processes.

Advanced Stochastic Processes. Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.

More information

Notes On Nonparametric Density Estimation. James L. Powell Department of Economics University of California, Berkeley

Notes On Nonparametric Density Estimation. James L. Powell Department of Economics University of California, Berkeley Notes O Noparametric Desity Estimatio James L Powell Departmet of Ecoomics Uiversity of Califoria, Berkeley Uivariate Desity Estimatio via Numerical Derivatives Cosider te problem of estimatig te desity

More information

MAT1026 Calculus II Basic Convergence Tests for Series

MAT1026 Calculus II Basic Convergence Tests for Series MAT026 Calculus II Basic Covergece Tests for Series Egi MERMUT 202.03.08 Dokuz Eylül Uiversity Faculty of Sciece Departmet of Mathematics İzmir/TURKEY Cotets Mootoe Covergece Theorem 2 2 Series of Real

More information

Output Analysis and Run-Length Control

Output Analysis and Run-Length Control IEOR E4703: Mote Carlo Simulatio Columbia Uiversity c 2017 by Marti Haugh Output Aalysis ad Ru-Legth Cotrol I these otes we describe how the Cetral Limit Theorem ca be used to costruct approximate (1 α%

More information

Chapter 2 The Monte Carlo Method

Chapter 2 The Monte Carlo Method Chapter 2 The Mote Carlo Method The Mote Carlo Method stads for a broad class of computatioal algorithms that rely o radom sampligs. It is ofte used i physical ad mathematical problems ad is most useful

More information

1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1

1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1 8. The cetral limit theorems 8.1. The cetral limit theorem for i.i.d. sequeces. ecall that C ( is N -separatig. Theorem 8.1. Let X 1, X,... be i.i.d. radom variables with EX 1 = ad EX 1 = σ (,. Suppose

More information

Lecture Chapter 6: Convergence of Random Sequences

Lecture Chapter 6: Convergence of Random Sequences ECE5: Aalysis of Radom Sigals Fall 6 Lecture Chapter 6: Covergece of Radom Sequeces Dr Salim El Rouayheb Scribe: Abhay Ashutosh Doel, Qibo Zhag, Peiwe Tia, Pegzhe Wag, Lu Liu Radom sequece Defiitio A ifiite

More information

Lecture 14: Maximum Likelihood and Complexity Regularization

Lecture 14: Maximum Likelihood and Complexity Regularization ECE90 Sprig 007 Statistical Learig Theory Istructor: R Nowak Lecture 4: Maximum Likelihood ad Complexity Regularizatio Review : Maximum Likelihood Estimatio We have iid observatios draw from a ukow distributio

More information

Large Sample Theory. Convergence. Central Limit Theorems Asymptotic Distribution Delta Method. Convergence in Probability Convergence in Distribution

Large Sample Theory. Convergence. Central Limit Theorems Asymptotic Distribution Delta Method. Convergence in Probability Convergence in Distribution Large Sample Theory Covergece Covergece i Probability Covergece i Distributio Cetral Limit Theorems Asymptotic Distributio Delta Method Covergece i Probability A sequece of radom scalars {z } = (z 1,z,

More information

Notes 19 : Martingale CLT

Notes 19 : Martingale CLT Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall

More information

Expectation and Variance of a random variable

Expectation and Variance of a random variable Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio

More information

Efficient GMM LECTURE 12 GMM II

Efficient GMM LECTURE 12 GMM II DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet

More information

Lecture 3 The Lebesgue Integral

Lecture 3 The Lebesgue Integral Lecture 3: The Lebesgue Itegral 1 of 14 Course: Theory of Probability I Term: Fall 2013 Istructor: Gorda Zitkovic Lecture 3 The Lebesgue Itegral The costructio of the itegral Uless expressly specified

More information

Lecture 7: Properties of Random Samples

Lecture 7: Properties of Random Samples Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ

More information

Introduction to Extreme Value Theory Laurens de Haan, ISM Japan, Erasmus University Rotterdam, NL University of Lisbon, PT

Introduction to Extreme Value Theory Laurens de Haan, ISM Japan, Erasmus University Rotterdam, NL University of Lisbon, PT Itroductio to Extreme Value Theory Laures de Haa, ISM Japa, 202 Itroductio to Extreme Value Theory Laures de Haa Erasmus Uiversity Rotterdam, NL Uiversity of Lisbo, PT Itroductio to Extreme Value Theory

More information

Frequentist Inference

Frequentist Inference Frequetist Iferece The topics of the ext three sectios are useful applicatios of the Cetral Limit Theorem. Without kowig aythig about the uderlyig distributio of a sequece of radom variables {X i }, for

More information

Economics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator

Economics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters

More information

Math Solutions to homework 6

Math Solutions to homework 6 Math 175 - Solutios to homework 6 Cédric De Groote November 16, 2017 Problem 1 (8.11 i the book): Let K be a compact Hermitia operator o a Hilbert space H ad let the kerel of K be {0}. Show that there

More information

Fall 2013 MTH431/531 Real analysis Section Notes

Fall 2013 MTH431/531 Real analysis Section Notes Fall 013 MTH431/531 Real aalysis Sectio 8.1-8. Notes Yi Su 013.11.1 1. Defiitio of uiform covergece. We look at a sequece of fuctios f (x) ad study the coverget property. Notice we have two parameters

More information

Last Lecture. Wald Test

Last Lecture. Wald Test Last Lecture Biostatistics 602 - Statistical Iferece Lecture 22 Hyu Mi Kag April 9th, 2013 Is the exact distributio of LRT statistic typically easy to obtai? How about its asymptotic distributio? For testig

More information

7.1 Convergence of sequences of random variables

7.1 Convergence of sequences of random variables Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite

More information

Stochastic Simulation

Stochastic Simulation Stochastic Simulatio 1 Itroductio Readig Assigmet: Read Chapter 1 of text. We shall itroduce may of the key issues to be discussed i this course via a couple of model problems. Model Problem 1 (Jackso

More information

Problem Set 4 Due Oct, 12

Problem Set 4 Due Oct, 12 EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios

More information

Estimation of the Mean and the ACVF

Estimation of the Mean and the ACVF Chapter 5 Estimatio of the Mea ad the ACVF A statioary process {X t } is characterized by its mea ad its autocovariace fuctio γ ), ad so by the autocorrelatio fuctio ρ ) I this chapter we preset the estimators

More information

STAT Homework 2 - Solutions

STAT Homework 2 - Solutions STAT-36700 Homework - Solutios Fall 08 September 4, 08 This cotais solutios for Homework. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better isight.

More information

Entropy and Ergodic Theory Lecture 5: Joint typicality and conditional AEP

Entropy and Ergodic Theory Lecture 5: Joint typicality and conditional AEP Etropy ad Ergodic Theory Lecture 5: Joit typicality ad coditioal AEP 1 Notatio: from RVs back to distributios Let (Ω, F, P) be a probability space, ad let X ad Y be A- ad B-valued discrete RVs, respectively.

More information

EFFECTIVE WLLN, SLLN, AND CLT IN STATISTICAL MODELS

EFFECTIVE WLLN, SLLN, AND CLT IN STATISTICAL MODELS EFFECTIVE WLLN, SLLN, AND CLT IN STATISTICAL MODELS Ryszard Zieliński Ist Math Polish Acad Sc POBox 21, 00-956 Warszawa 10, Polad e-mail: rziel@impagovpl ABSTRACT Weak laws of large umbers (W LLN), strog

More information

Rates of Convergence by Moduli of Continuity

Rates of Convergence by Moduli of Continuity Rates of Covergece by Moduli of Cotiuity Joh Duchi: Notes for Statistics 300b March, 017 1 Itroductio I this ote, we give a presetatio showig the importace, ad relatioship betwee, the modulis of cotiuity

More information

4. Partial Sums and the Central Limit Theorem

4. Partial Sums and the Central Limit Theorem 1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems

More information

This exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.

This exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam. Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties

More information

Glivenko-Cantelli Classes

Glivenko-Cantelli Classes CS28B/Stat24B (Sprig 2008 Statistical Learig Theory Lecture: 4 Gliveko-Catelli Classes Lecturer: Peter Bartlett Scribe: Michelle Besi Itroductio This lecture will cover Gliveko-Catelli (GC classes ad itroduce

More information

1.010 Uncertainty in Engineering Fall 2008

1.010 Uncertainty in Engineering Fall 2008 MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval

More information

Math 341 Lecture #31 6.5: Power Series

Math 341 Lecture #31 6.5: Power Series Math 341 Lecture #31 6.5: Power Series We ow tur our attetio to a particular kid of series of fuctios, amely, power series, f(x = a x = a 0 + a 1 x + a 2 x 2 + where a R for all N. I terms of a series

More information

Asymptotic Results for the Linear Regression Model

Asymptotic Results for the Linear Regression Model Asymptotic Results for the Liear Regressio Model C. Fli November 29, 2000 1. Asymptotic Results uder Classical Assumptios The followig results apply to the liear regressio model y = Xβ + ε, where X is

More information

Joint Probability Distributions and Random Samples. Jointly Distributed Random Variables. Chapter { }

Joint Probability Distributions and Random Samples. Jointly Distributed Random Variables. Chapter { } UCLA STAT A Applied Probability & Statistics for Egieers Istructor: Ivo Diov, Asst. Prof. I Statistics ad Neurology Teachig Assistat: Neda Farziia, UCLA Statistics Uiversity of Califoria, Los Ageles, Sprig

More information

Approximations and more PMFs and PDFs

Approximations and more PMFs and PDFs Approximatios ad more PMFs ad PDFs Saad Meimeh 1 Approximatio of biomial with Poisso Cosider the biomial distributio ( b(k,,p = p k (1 p k, k λ: k Assume that is large, ad p is small, but p λ at the limit.

More information

Lecture 18: Sampling distributions

Lecture 18: Sampling distributions Lecture 18: Samplig distributios I may applicatios, the populatio is oe or several ormal distributios (or approximately). We ow study properties of some importat statistics based o a radom sample from

More information

First Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise

First Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >

More information