An Empirical Likelihood Approach To Goodness of Fit Testing

Size: px
Start display at page:

Download "An Empirical Likelihood Approach To Goodness of Fit Testing"

Transcription

1 Submitted to the Beroulli A Empirical Likelihood Approach To Goodess of Fit Testig HANXIANG PENG ad ANTON SCHICK Idiaa Uiversity Purdue Uiversity at Idiaapolis, Departmet of Mathematical Scieces, Idiaapolis, IN 46202, USA hpeg@math.iupui.edu Bighamto Uiversity, Departmet of Mathematical Scieces, Bighamto, NY 3902, USA ato@math.bighamto.edu Motivated by applicatios to goodess of fit testig, the empirical likelihood approach is geeralized to allow for the umber of costraits to grow with the sample size ad for the costraits to use estimated criteria fuctios. The latter is eeded to deal with uisace parameters. The proposed empirical likelihood based goodess of fit tests are asymptotically distributio free. For uivariate observatios, tests for a specified distributio, for a distributio of parametric form, ad for a symmetric distributio are preseted. For bivariate observatios, tests for idepedece are developed. Keywords: ifiitely may costraits; uisace parameter; estimated costrait fuctios; regressio model; testig for a specific distributio; testig for a parametric model; testig for symmetry; testig for idepedece.. Itroductio The empirical likelihood approach was itroduced by Owe (988, 990) to costruct cofidece itervals i a oparametric settig, see also Owe (200). As a likelihood approach possessig oparametric properties, it does ot require us to specify a distributio for the data ad ofte yields more efficiet estimates of the parameters. It allows data to decide the shape of cofidece regios ad is Bartlett correctable (DiCiccio, Hall ad Romao, 99). The approach has bee developed to various situatios, e.g., to geeralized liear models (Kolaczyk, 994), local liear smoother (Che ad Qi, 2000), partially liear models (Shi ad Lau, 2000; Wag ad Jig, 2003), parametric ad semiparametric models i multirespose regressio (Che ad Va Keilegom, 2009), liear regressio with cesored data (Zhou ad Li, 2008), ad plug-i estimates of uisace parameters i estimatig equatios i the cotext of survival aalysis (Qi ad Jig, 200; Wag ad Jig, 200; Li ad Wag, 2003). Algorithms, calibratio ad higher-order precisio of the approach ca be foud i Hall ad La Scala (990), Emerso ad Owe (2009) ad Liu ad Che (200) amog others. It is especially coveiet to icorporate side iformatio expressed through equality costraits. Qi ad Lawless (994) liked

2 2 H. Peg ad A. Schick empirical likelihood with fiitely may estimatig equatios. These estimatig equatios serve as fiitely may equality costraits. I semiparametric settigs, iformatio o the model ca ofte be expressed by meas of ifiitely may costraits which may also deped o parameters of the model. I goodess of fit testig, the ull hypothesis ca typically be expressed by ifiitely may such costraits. This is the case whe testig for a fixed distributio (see Example below), whe testig for a give parametric model (Example 2), whe testig for symmetry about a fixed poit (Example 3), ad whe testig for idepedece (Example 4). Modelig coditioal expectatios ca also be doe by meas of ifiitely may costraits. This has applicatios to heteroscedastic regressio models (Sectio 3) ad to coditioal momet restrictio models treated by Tripathi ad Kitamura (2003) usig a smoothed empirical likelihood approach. Recetly Hjort, McKeague ad Va Keilegom (2009) exteded the scope of the empirical method. I particular, they developed a geeral theory for costraits with uisace parameters ad cosidered the case with ifiitely may costraits. Their results for ifiitely may costraits, however, do ot allow for uisace parameters. I this paper we will fill this gap ad i the process improve o their results. Let us ow discuss some of our results i the followig special case. Let Z,..., Z be idepedet copies of a radom vector Z with distributio Q. Let u, u 2,... be orthoormal elemets of L 2,0 (Q) = {u L 2 (Q) : u dq = 0}. The the radom variables u (Z), u 2 (Z),... have mea zero, variace oe ad are ucorrelated. Now cosider the empirical likelihood based o the first m of these fuctios, { R = sup π j : π P, π j u k (Z j ) = 0, } k =,..., m, where P = {π = (π,..., π ) [0, ] : π + + π = } deotes the closed probability simplex i dimesio. For fixed m, it follows from Owe s work that 2 log R has asymptotically a chi-square distributio with m degrees of freedom. I other words, P ( 2 log R > χ 2 α(m)) α, 0 < α <, (.) where χ 2 β (m) deotes the β-quatile of the chi-square distributio with m degrees of freedom. Hjort et al (2009) have show that (.) holds uder some additioal assumptios eve if m teds to ifiity with by provig the asymptotic ormality result ( 2 log R m)/ 2m = N(0, ). (.2) This result requires higher momet assumptios o the fuctios u, u 2,... ad restrictios o the rate at which m ca ted to ifiity. For example, if the fuctios u, u 2,... are uiformly bouded, the the rate m 3 = o() suffices for (.2). They also state i their

3 A Empirical Likelihood Approach To Goodess of Fit Testig 3 Theorem 4., that if sup k uk q dq is fiite for some q > 2, the m 3+6/(q 2) = o() suffices for (.2). A gap i their argumet was fixed by Peg ad Schick (202). We shall show that larger m are allowed i some cases. I particular, for q = 4, it suffices that m 4 = o() holds (istead of their m 6 = o()) ad if q = 3, the m 6 = o() is eough (istead of their m 9 = o()), see our Theorems 7.2 ad 7.3 below. Our rate m 4 = o() for q = 4 matches the rate give i Theorem 2 of Che, Peg ad Qi (2009). These authors obtai asymptotic ormality for m larger tha i Hjort et al (2009) by imposig additioal structural assumptios. These assumptios, however, are typically ot met i the applicatios we have i mid. Oe of the key poits i our proof is a simple coditio for the covex hull of some vectors x,..., x to have the origi as a iterior poit. Our coditio is that the smallest eige value of i= x ix i exceeds 5 i= x i max j x j. Here x deotes the euclidea orm of a vector x. This sufficiet coditio ties i icely with the other requiremets used to establish the asymptotic behavior of the empirical likelihood ad is typically implied by these. For example, coditios (A) (A3) i Theorem 2. of Hjort et al (2009) already imply their (A0). Thus the coclusio of their theorem is valid uder (A) (A3) oly, see our Theorem 6.. Let us ow look at the case whe the fuctios u, u 2,... are ukow. The we ca work with the empirical likelihood { ˆR = sup π j : π P, where û k is a estimator of u k such that m k= π j û k (Z j ) = 0, } k =,..., m (û k (Z j ) u k (Z j )) 2 = o p (m ). (.3) Now we have the coclusio ( 2 log ˆR m)/ 2m = N(0, ) uder the coditio m ( 2 /2 (û k (Z j ) u k (Z j ))) = op () (.4) k= ad mild additioal coditios such as (i) û k + u k B for some costat B ad all k ad m 3 = o(), or (ii) m k= u 4 k dq = O(m 2 ) ad m 4 = o(). Our results, however, go beyod this simple result. If (.4) is replaced by m 2 ( /2 (û k (Z j ) u k (Z j ) + E[u k (Z)ψ (Z)]ψ(Z j ))) = op () (.5) k= with ψ a measurable fuctio ito R q which is stadardized uder Q i the sese that E[ψ(Z)] = 0 ad E[ψ(Z)ψ (Z)] = I q, the q q idetity matrix, the the coclusio ( 2 log ˆR (m q))/ 2(m q) = N(0, ) holds uder (i) or (ii).

4 4 H. Peg ad A. Schick Our paper is orgaized as follows. I Sectio 2, we give four examples that motivate our research. The emphasis i these examples is o goodess of fit testig. The proposed empirical likelihood based goodess of fit tests are asymptotically distributio free. For uivariate observatios, tests for a specified distributio, for a distributio of parametric form, ad for a symmetric distributio are preseted. For bivariate observatios, tests for idepedece are discussed. Aother example is give i Sectio 3 with a small simulatio study. This example cosiders tests for the regressio parameters i simple liear heteroscedastic regressio. The simulatios compare our ew procedure based o ifiitely may costraits with the classical empirical likelihood procedure ad illustrate improvemets by the ew procedures. I Sectio 4 we itroduce otatio ad recall some results o the spectral orm of matrices. I Sectio 5 we derive a lemma that extracts the essece from the proofs of Owe (200, Chapter ) ad also obtais the aforemetioed sufficiet coditio for a covex hull of vectors to cotai the origi as iterior poit. The results are derived for o-stochastic vectors ad formulated as iequalities. The iequalities are used i Sectio 6 to obtai the behavior of the empirical likelihood with radom vectors whose dimesio may icrease. The results are formulated abstractly ad do ot require idepedece. I Sectio 7 we specialize our results to the case of idepedet observatios with ifiitely may costraits, both kow ad ukow. We also briefly discuss the behavior uder cotiguous alteratives. The details for our examples are give i Sectio Motivatig examples I this sectio, we give examples that motivated the research i this paper. Example. Testig for a fixed distributio. Let X,..., X be idepedet copies of a radom variable X. Suppose we wat to test whether their commo distributio fuctio F equals a kow cotiuous distributio fuctio F 0. Uder the ull hypothesis, we have E[h(X)] = 0 for every h L 2,0 (F 0 ), ad F 0 (X) has a uiform distributio. A orthoormal basis of L 2,0 (F 0 ) is thus give by v F 0, v 2 F 0,... for ay orthoormal basis v, v 2,... of L 2,0 (U), where U is the uiform distributio o (0, ). We shall work with the trigoometric basis φ, φ 2,... defied by φ k (x) = 2 cos(kπx), x [0, ], k =, 2,..., (2.) as these basis fuctios are uiformly bouded by 2. As test statistic we take { } R (F 0 ) = sup π j : π P, π j φ k (F 0 (X j )) = 0, k =,..., m which uses the first m of the trigoometric fuctios. Uder the ull hypothesis we have P ( 2 log R (F 0 ) > χ 2 α(m)) α for every 0 < α < as both m ad ted to ifiity ad m 3 / teds to zero. Thus the test [ 2 log R (F 0 ) > χ 2 α(m)] has asymptotic size α. Here we are still i the framework of Hjort et al (2009) with ifiitely may kow costraits.

5 A Empirical Likelihood Approach To Goodess of Fit Testig 5 Example 2. Testig for a parametric model. Let X,..., X be agai idepedet ad idetically distributed radom variables. But ow suppose we wat to test whether their commo distributio fuctio F belogs to a model F = {F ϑ : ϑ Θ} idexed by a ope subset Θ of R q. Suppose that the distributio fuctios F ϑ have desities f ϑ such that the map ϑ s ϑ = f ϑ is cotiuously differetiable i L 2 with derivative ϑ ṡ ϑ ad the matrix J(ϑ) = 4 ṡ ϑ (x)ṡ ϑ (x) dx is ivertible for each ϑ Θ. I this case we set l ϑ = 2ṡ ϑ /s ϑ. Let ow ˆθ be a estimator of the parameter i the model. We require it to satisfy the stochastic expasio ˆθ = θ + J(θ) lθ (X j ) + o Pθ ( /2 ) (2.2) for each θ Θ, where P θ is the measure for which F = F θ. Such estimators are efficiet i the parametric model. Cadidates are maximum likelihood estimators. As test statistic we take R (Fˆθ), the test statistic from the previous example with F 0 replaced by Fˆθ. Here we are o loger i the framework of Hjort et al (2009) as we ow have ifiitely may ukow costraits. We shall show that uder the ull hypothesis P ( 2 log R (Fˆθ) > χ 2 α(m q)) α for every 0 < α < as both m ad ted to ifiity ad log m 3 / teds to zero. I view of this result the test [ 2 log R (Fˆθ) > χ 2 α(m q)] has asymptotic size α. It is crucial for our result that we have chose a estimator ˆθ satisfyig (2.2). Example 3. Testig for symmetry. Let X,..., X be idepedet copies of a radom variable X with a cotiuous distributio fuctio F. We wat to test whether F is symmetric about zero i the sese that F (t) = F ( t) for all real t. Uder the ull hypothesis of symmetry, the radom variables sig(x) ad X are idepedet, ad sig(x) takes values ad with probability oe half. This is equivalet to E[sig(X)v( X )] = 0 for every v L 2 (H), where H is the distributio fuctio of X. Sice H is cotiuous, a orthoormal system of L 2 (H) is give by φ 0 H, φ H,... where φ 0 = ad φ, φ 2,... are give i (2.). This suggests the test statistic { R = sup π j : π P, } π j sig(x j )φ k (R j ) = 0, k = 0,..., m, where R j = H( X j ) ad H is the empirical distributio fuctio based o X,..., X. We shall show that uder symmetry oe has P ( 2 log R > χ 2 α(m + )) α for every 0 < α < as m ad ted to ifiity ad m 3 / teds to zero. From this we derive that the test [ 2 log R > χ 2 α(m + )] has asymptotic size α. Example 4. Testig for idepedece. Let (X, Y ),..., (X, Y ) be idepedet copies of a bivariate radom vector (X, Y ). We assume that the margial distributio fuctios F ad G are cotiuous. We wat to test whether X ad Y are idepedet. Idepedece is equivalet to E[a(X)b(Y )] = 0 for all a L 2,0 (F ) ad b L 2,0 (G) ad thus equivalet to E[φ k (F (X))φ l (G(Y ))] = 0 for all positive itegers k ad l.

6 6 H. Peg ad A. Schick (a) Assume first that F ad G are kow. This is for example the case i a actuarial settig where X ad Y deote residual lifetimes ad their distributio fuctios are available from life tables. Motivated by the above we take as test statistics { R (F, G) = sup π j : π P, π j φ k (F (X j ))φ l (G(Y j )) = 0, } k, l =,..., r. Uder the ull hypothesis oe has P ( 2 log R (F, G) > χ 2 α(r 2 )) α for every 0 < α < as r ad ted to ifiity ad r 6 / teds to zero. Here we are i the framework of Hjort, McKeague ad Va Keilegom (2009). The above shows that the test [ 2 log R (F, G) > χ 2 α(r 2 )] has asymptotic size α. (b) Now assume that F ad G are ukow. I this case we replace both margial distributio fuctios by their empirical distributio fuctios. The resultig test statistic is R (F, G), where F deotes the empirical distributio based o X,..., X ad G the oe based o Y,..., Y. We shall show that uder the ull hypothesis P ( 2 log R (F, G) > χ 2 α(r 2 )) α for every 0 < α < as r ad ted to ifiity ad r 6 / teds to zero. Thus the test [ 2 log R (F, G) > χ 2 α(r 2 )] has asymptotic size α. Remark 2.. Suppose that (X, Y ) form a simple liear homoscedastic regressio model, Y = β + β 2 X + ε, with X ad ε idepedet. We ca use the test statistic from case (b) to test the hypothesis whether the slope parameter β 2 is zero. Ideed, β 2 = 0 is equivalet to the idepedece of X ad Y. Remark 2.2. The asymptotic distributios of the above tests uder cotiguous alteratives are liked to o-cetral chi-square distributios; see Remark 7.3 for details. As the o-cetrality parameters are bouded, the local asymptotic power alog such a cotiguous alterative coicides with the level. Our tests are asymptotically equivalet to Neyma s smooth tests with icreasig dimesios. I view of the optimality results of Iglot ad Ledwia (996) for those tests uder moderate deviatios, we expect similar results for our tests. Of course, this eeds to be explored more carefully. 3. Aother example ad simulatios Let (X, Y ),..., (X, Y ) be idepedet copies of (X, Y ), where Y = β +β 2 X +ε, with E[ε X] = 0, σ 2 (X) = E[ε 2 X] bouded ad bouded away from zero, ad E[ε 4 ] <. Assume that X has a fiite variace ad a cotiuous distributio fuctio G. We are iterested i testig whether the regressio parameter β = (β, β 2 ) equals some specific value θ. We could proceed as i Owe (99) ad use the test δ 0 = [ 2 log R 0 (θ) > χ 2 α(2)] based o the empirical likelihood { R 0 (θ) = sup π j : π P, ( ) } π j (Y X j θ θ 2 X j ) = 0. j

7 A Empirical Likelihood Approach To Goodess of Fit Testig 7 But this empirical likelihood does ot use all the iformatio of the model. Here we have E[a(X)ε] = 0 for every a L 2 (G). Sice G is cotiuous (but ukow), we work with the empirical likelihood { ˆR (θ) = sup π j : π P, } π j u r (G(X j ))(Y j θ θ 2 X j ) = 0 where u r = (, φ,..., φ r ) ad G is the empirical distributio fuctio based o the covariate observatios X,..., X. It follows from Corollary 7.6 ad Lemma 8. below that P ( 2 log ˆR (β) > χ 2 α( + r)) α if r 4 = o(). The resultig test is δ = [ 2 log ˆR (θ) > χ 2 α(r + )]. Both tests have asymptotic size α. We performed a small simulatio study to compare the procedures. For our simulatio we chose α =.05 ad = 00 ad took θ = (, 2). We modeled the error ε as s(x)η, with s(x) = mi( + X 2, 00) ad η idepedet of X. As distributios for X we chose the expoetial distributio with mea 5 (Ex(5)) ad the t-distributio with three degrees of freedom (t(3)), while for η we chose the stadard ormal distributio (N(0,)) ad the double expoetial distributio with locatio 0 ad scale.5 (L(0,.5)). Table. Simulated powers of the tests δ 0 ad δ. t(3) Ex(5) β β N(0, ) L(0,.5) Table reports simulated powers of the tests δ 0 ad δ (with several choices of r) ad for some values of θ. The reported values are based o 000 repetitios. The colum labeled 0 correspods to Owe s test δ 0, while the colums labeled 2,3,4,5 correspod to our tests δ with r = 2, 3, 4, 5 respectively. Clearly our ew test is more powerful tha the traditioal test. The values i the rows correspodig to the parameter values (.0, 2.0) are the observed sigificace levels of the omial sigificace level.05. Our ew test overall has closer observed sigificace levels tha the traditioal oe except for r = Notatio I this sectio we itroduce some of the otatio we use throughout. We write A for the euclidea orm ad A o for the operator (or spectral) orm of a matrix A which are

8 8 H. Peg ad A. Schick defied by A 2 = trace(a A) = i,j A 2 ij ad A o = sup Au = sup (u A Au) /2. u = u = I other words, the squared euclidea orm A 2 equals the sum of the eige values of A A, while the squared operator orm A 2 o equals the largest eige value of A A. Cosequetly, the iequality A o A holds. Thus we have Ax A o x A x for compatible vectors x. We should also poit out the idetity A o = sup u = v = sup u Av. If A is a oegative defiite symmetric matrix, this simplifies to A o = sup u Au. u = Usig this ad the Cauchy-Schwarz iequality we obtai fg dµ 2 ff dµ gg dµ, (4.) o o o ff dµ f 2 dµ, (4.2) o wheever µ is a measure ad f ad g are measurable fuctios ito R s ad R t such that f 2 dµ ad g 2 dµ are fiite. As a special case we derive the iequality S x+y S x o S y o + 2 S x /2 o S y /2 o ad therefore S x+y S x o ( y i S x /2 o y i 2) /2 (4.3) with S x+y = (x j + y j )(x j + y j ), S x = x j x j, S y = y j yj for vectors x, y,..., x, y of the same dimesio.

9 A Empirical Likelihood Approach To Goodess of Fit Testig 9 5. A maximizatio problem Let x,..., x be m-dimesioal vectors. Set x = max j x j, x = x j, S = x j x j, x (ν) = sup u = (u x j ) ν, ν = 3, 4, ad let λ ad Λ deote the smallest ad largest eige value of the matrix S, λ = if u = u Su ad Λ = sup u Su. u = Usig Lagrage multipliers, Owe (988,200) obtaied the idetity { R = sup π j : π P, } π j x j = 0 = if there exists a ζ i R m such that + ζ x j > 0, j =,...,, ad + ζ x j x j + ζ x j = 0. (5.) He also showed that such a vector ζ exists ad is uique if (i) the origi is a iterior poit of the covex hull of x,..., x ad (ii) the matrix S is ivertible. Let us ow show that the iequality λ > 5x x implies these two coditios. Ideed, the matrix S is the positive defiite ad hece ivertible as its smallest eige value λ is positive. To show (i) we will rely o the followig lemma. Lemma 5.. A radom variable Y with E[Y ] = 0 ad P ( Y K) = for some positive K obeys the iequality P (Y > a) E[Y 2 ] 2Ka 2K 2, 0 a < K. Proof. Fix a i [0, K). By the properties of Y, we obtai 2K 2 P (Y > a) 2KE[Y [Y > a]] 2KE[Y [Y > 0]] 2Ka ad 2KE[Y [Y > 0]] = KE[ Y ] E[Y 2 ]. The origi is a iterior poit of the covex hull of x,..., x if for every uit vector u R m there is at least oe j {,..., } such that u x j > 0. This latter coditio is equivalet to N = if [u x j > 0]. u = For a uit vector u, we have u x x ad thus [u x j > 0] [u (x j x) > x ] = N(u).

10 0 H. Peg ad A. Schick It follows from the triagle iequality that x j x x j + x 2x for j =,...,. Note that x is positive if S is positive defiite. Thus Lemma 5. yields the lower boud N(u)/ (σ 2 (u) 4x x )/(8x 2 ) with σ 2 (u) = (u (x j x)) 2 = u T Su (u x) 2 λ x 2 λ x x. Thus we have N (λ 5 x x )/(8x 2 ). This shows that the iequality λ > 5 x x implies N ad hece the desired coditio (i). Assume ow that the iequality λ > 5x x holds. We proceed as o page 220 of Owe (200). Let u be a uit vector such that ζ = ζ u. The we have the idetity 0 = ad the iequality u x j ( + ζ x j ζ x j ) + ζ x j = u x ζ λ u Su = (u x j ) 2 (u x j ) 2 + ζ x j (u x j ) 2 ( + ζ x ) + ζ x j. Cosequetly, we fid λ ζ ( + ζ x )u x ( + ζ x ) x ad obtai the boud From this oe immediately derives ζ x λ x x. (5.2) ζ x x x λ x x < 4, (5.3) max j + ζ < 4 x j ζ x 3, (5.4) (ζ x j ) 2 = ζ Sζ Λ ζ 2 Λ x 2 (λ x x ) 2. (5.5) The idetity /( + d) + d = d 2 d 3 /( + d) ad (5.4) yield ( r j + ζ x j r j + r j x j ζ ) r j (ζ x j ) r j ζ x j 3 for vectors r,..., r of the same dimesio. Takig r j = S x j, we derive with the help of (5.) ζ S x S x j (ζ x j ) S x j ζ x j 3. 3

11 A Empirical Likelihood Approach To Goodess of Fit Testig Usig x = sup v = v x, the Cauchy-Schwarz iequality, (5.3) ad (5.5) we boud the square of the first summad of the right-had side by (ζ x j ) 4 sup v S v λ ζ 4 x (4) v = ad the square of the secod summad by 6 9λ 2 x2 ζ Sζ Combiig the above we obtai (ζ x j ) 4 Λ 9λ 2 ζ 4 x (4). ζ S x ( 2 2 λ + Λ ) 9λ 2 ζ 4 x (4). (5.6) Usig the iequality 2 log( + t) 2t + t 2 2t 3 /3 t 4 /(2( t ) 4 ) valid for t <, ad the (5.4) we derive 2 log( + ζ x j ) 2ζ x + ζ Sζ 2 (ζ x j ) 3 4 ) ( ζ x j 4. 3 With = ζ S x, we ca write ζ Sζ = ζ x + ζ S ad ζ x = x S x + x, ad obtai the idetity 2ζ x ζ Sζ = xs x S. Usig this ad (5.6) we arrive at the boud 2 log( + ζ x j ) x S x ( 6 ζ 3 x (3) Λ ) λ + 2Λ2 9λ 2 ζ 4 x (4). I view of (5.2) ad Λ λ, this becomes 2 log( + ζ x j ) x S x x 3 x (3) (λ x x ) 3 + Λ2 4 x 4 x (4) λ 2 (λ x x ) 4. (5.7) If we boud x (3) by x Λ ad x (4) by x 2 Λ ad use (5.3) we obtai the boud 2 log( + ζ x j ) x S x Thus we have proved the followig result. (Λ + Λ3 λ 2 ) x x 3 (λ x x ) 3. (5.8) Lemma 5.2. The iequality λ > 5 x x implies that there is a uique ζ i R m satisfyig + ζ x j > 0, j =,...,, ad (5.) to (5.8).

12 2 H. Peg ad A. Schick 6. Applicatios with radom vectors We shall ow discuss implicatios of Lemma 5.2 to the case whe the vectors x j are replaced by radom vectors. We are iterested i the case whe the dimesio of the radom vectors icreases with. Let T,..., T be m -dimesioal radom vectors. With these radom vectors we associate the empirical likelihood { R = sup π j : π P, To study the asymptotic behavior of R we itroduce ad the matrix T = max j T j, T = T j, S = T (ν) T j Tj, } π j T j = 0. = sup u = ad let λ ad Λ deote the smallest ad largest eige values of S, (u T j ) ν, λ = if u = u S u ad Λ = sup u S u. u = We say a sequece W of m m dispersio matrices is regular if the followig coditio holds, 0 < if if u = u W u sup sup u W u <. We impose the followig coditios. (A) m /2 T = o p ( /2 ). (A2) T 2 = O p (m ). (A3) There is a regular sequece of dispersio matrices W such that S W o = o p (m /2 ). (A4) m T (3) = o p ( /2 ) ad m 3/2 T (4) = o p (). u = The first two coditios imply T T = o p (), the third coditio implies that there are positive umbers a < b such that P (a λ Λ b). Thus all three coditios imply that the probability of the evet {λ > 5T T } teds to oe. Cosequetly, by Lemma 5.2, there exists a m -dimesioal radom vector ˆζ which is uiquely determied o this evet by the properties + ˆζ T j > 0, j =,...,, ad T j + ˆζ T j = 0. (6.)

13 A Empirical Likelihood Approach To Goodess of Fit Testig 3 O this evet we have 2 log R = 2 log( + ˆζ T j ). It follows from (A3) that S is ivertible except o a evet whose probability teds to zero. It follows from (A2) ad (A4) that T 3 T (3) = o p (m /2 ) ad T 4 T (4) = o p (m /2 ). Thus, uder (A) (A4), the followig expasio follows from (5.7) 2 log R = T S T + o p (m /2 ). (6.2) From (A3) we ca also derive the rate S W o = o p (m /2 ). Thus, if (A) (A4) hold, the (6.2) holds with S replaced by W, 2 log R = T W T + o p (m /2 ). (6.3) I view of the iequalities T (3) Λ T ad T (4) Λ (T) 2, a sufficiet coditio for (A) ad (A4) is give by m T = o p ( /2 ). (B) I view of the boud (T (3) ) 2 Λ T (4), which is a cosequece of the Cauchy-Schwarz iequality, a sufficiet coditio for (A4) is give by m 2 T (4) = o p (). (B2) We first treat the case whe the dimesio m does ot icrease with. I this case (B) ad (A2) are implied by T = o p ( /2 ) ad T = O p ( /2 ), ad (A3) is implied by the coditio: S = W + o p () for some positive defiite matrix W. Thus we have the followig result. Theorem 6.. Let m = m for all. Suppose T = o p ( /2 ), /2 T = N(0, V ) ad S = W + o p (), for dispersio matrices V ad W, with W positive defiite. The 2 log R coverges i distributio to Z V /2 W V /2 Z, where the m-dimesioal radom vector Z is stadard ormal. For V = W the limitig distributio is a chi-square distributio with m degrees of freedom. If we replace /2 T = N(0, V ) by /2 T = U for some radom variable U, the the coclusio becomes 2 log R coverges i distributio to U W U. This versio of the theorem yields Theorem 2. of Hjort et al (2009) without their (A0). Theorem 6. does ot require the idepedece of the radom vectors T,,..., T,. This is importat whe dealig with estimated costrait fuctios as we shall see below. Suppose the coditio i the theorem hold with V = W. Uder a cotiguous alterative, oe typically has /2 T = N(µ, V ) for some µ differet from zero, but retais the other coditios. I this case, 2 log R has a limitig chi-square distributio with m degrees of freedom ad o-cetrality parameter V /2 µ.

14 4 H. Peg ad A. Schick Let us address some applicatios of Theorem 6.. For this discussio we let Z,..., Z be idepedet copies of a k-dimesioal radom vector Z with distributio Q ad let w be a measurable fuctio from R k ito R m such that E[w(Z)] = w dq = 0 ad W = E[w(Z)w (Z)] = ww dq is positive defiite. Let us first look at the empirical likelihood { } R = sup π j : π P, π j w(z j ) = 0. It follows from Owe that 2 log R has a limitig chi-square distributio with m degrees of freedom. This also follows from Theorem 6. applied with T j = w(z j ). Ideed, the first coditio follows from the iequality P ( max j w(z j) > ɛ /2 ) ɛ 2 E[ w(z) 2 [ w(z) > ɛ /2 ] (6.4) ad the Lebesgue domiated covergece theorem; the cetral limit theorem yields the secod coditio with V = W ; the third coditio w(z j )w (Z j ) = W + o p () (6.5) follows from the weak law of large umbers. This shows that Owe s result is a special case of our result. Now cosider the empirical likelihood { ˆR = sup π j : π P, } π j ŵ(z j ) = 0, where ŵ is a estimator of w based o the observatios Z,..., Z which is cosistet i the followig sese, ŵ(z j ) w(z j ) 2 = o p (). (6.6) The 2 log ˆR has a limitig chi-square distributio with m degrees of freedom if also /2 ŵ(z j ) = /2 w(z j ) + o p () (6.7) holds. To see this, we verify the assumptios of Theorem 6. with T j = ŵ(z j ). The first coditio follows from (6.4), (6.6) ad the iequality T max j w(z j) + ( ŵ(z j ) w(z j ) 2) /2.

15 A Empirical Likelihood Approach To Goodess of Fit Testig 5 The cetral limit theorem, Slutsky s theorem ad (6.7) yield the secod coditio with V = W. The third coditio follows from (6.5), (6.6) ad the iequality (4.3). The requiremet (6.7) is rather strog. Oe ofte oly derives /2 ŵ(z j ) = /2 v(z j ) + o p () (6.8) for some fuctio v satisfyig E[v(Z)] = 0 ad E[ v(z) 2 ] <. Uder (6.6) ad (6.8), 2 log ˆR has limitig distributio as give i Theorem 6. with V the dispersio matrix of v(z). This follows from Theorem 6. whose assumptios are ow verified as above. I situatios whe w(z) = u(z, η) for some q-dimesioal uisace parameter η ad ŵ(z) = u(z, ˆη) for some estimator ˆη of η, oe typically has v(z) = w(z)+dψ(z), where the m q matrix D is the derivative of the map t E[u(Z, η + t)] at t = 0, ad ψ is the ifluece fuctio of ˆη. We ow address the case whe m icreases with the sample size. Theorem 6.2. Let (A) (A4) hold. Suppose that m icreases with to ifiity ad that there are m m dispersio matrices V such that m /trace(v 2 ) = O() ad ( T W T trace(v ))/ 2trace(V 2 ) = N(0, ). (6.9) The we have ( 2 log R trace(v ))/ 2trace(V 2 ) = N(0, ). (6.0) Proof. We have already see that (A) (A4) imply (6.3). It follows from (6.3) ad m /trace(v 2 ) = O() that the differece of the left-had sides of (6.9) ad (6.0) coverge to zero i probability. Thus the desired (6.0) follows from (6.9) ad Slutsky s Theorem. Of special iterest is the case whe V is the m m idetity matrix I m. The trace(v ) = trace(v 2 ) = m ad (6.0) simplifies to (.2). Sufficiet coditios for (6.9) are give by Peg ad Schick (202). 7. Mai results I this sectio we assume that (Z, S ) is a measurable space, that Z,..., Z are idepedet copies of the Z -valued radom variable Z with distributio Q, ad that m is a positive iteger that teds to ifiity with. We let w deote a measurable fuctio from Z to R m such that w dq = 0 ad w 2 dq is fiite. We first study { R = sup π j : π P, } π j w (Z j ) = 0.

16 6 H. Peg ad A. Schick Our goal is to show (.2). To this ed we set w = w (Z j ), W = w (Z j )w (Z j ), W = w w dq ad itroduce the followig coditio. (C) The sequece W is regular. Motivated by the results i Peg ad Schick (202) we call a sequece v of measurable fuctios from Z to R Lideberg if v 2 [ v > ɛ ] dq 0, ɛ > 0. (7.) The followig are easy to check. If the sequeces u ad v are Lideberg, so are the sequeces max{ u, v } ad u + v. If the sequece v is Lideberg ad u v, the the sequece u is also Lideberg. We also eed the followig properties. (L) If v is Lideberg, the oe has the rate max j v (Z j ) = o p ( /2 ). (L2) If v r dq = o( r/2 ) for some r > 2, the v is Lideberg. The first statemet follows from a iequality similar to (6.4), the secod from Remark i Peg ad Schick (202). To show (.2) we apply Theorem 6.2 with T j = w (Z j ). I the presece of (C), the coditios (6.9) ad (A) (A4) of this theorem are implied by ( w W w m )/ 2m = N(0, ), (D0) max j m/2 w (Z j ) = o p ( /2 ), (D) w 2 = O p (m ), (D2) W W o = o p (m /2 ), (D3) m 2 sup u w (Z j ) 4 = o p (). u = (D4) By part (c) of Corollary 3 i Peg ad Schick (202), (D0) follows if the fuctio W /2 w is Lideberg. I the presece of (C), the latter coditio is equivalet to w beig Lideberg. By (L), a sufficiet coditio for (D) is that m /2 w is Lideberg. It follows from (C) that trace(w ) Bm for some costat B. Thus (C) implies E[ w 2 ] = trace(w ) = O(m ) ad hece (D2). I view of (C), a sufficiet coditio for (D3) is that m w is Lideberg. To see this fix ɛ > 0 ad let W, ad W,2 be the matrices obtaied by replacig i the defiitio of W the fuctio w by v = w [ m w ɛ ] ad w v = w [ m w > ɛ ], respectively. The we fid E[ W, E[ W, ] 2 ] E[ v 4 (Z)] ɛ2 m 2 E[ w 2 (Z)] ɛ2 Bm m 2,

17 A Empirical Likelihood Approach To Goodess of Fit Testig 7 ad usig (4.2) P ( W,2 0) P ( max j m w (Z j ) > ɛ ) 0 E[ W,2 ] o E[ w 2 (Z)[ m w (Z) > ɛ ]] = o(m 2 ). The above iequalities show that (C) ad m w is Lideberg imply statemet (D3). The latter coditio also implies (B) ad hece (D) ad (D4), the latter i the presece of (C). Thus we have the followig result. Theorem 7.. Suppose (C) holds ad the sequece m w is Lideberg. The (.2) holds as m teds to ifiity with. From this, simple calculatios ad the property (L2) we immediately derive the followig corollaries. Corollary 7.. Suppose (C) holds ad w m B for some costat B. The (.2) holds if m 3 = o(). Corollary 7.2. Suppose (C) holds ad w r dq = O(m r/2 ) for some r > 2. The (.2) holds if m 3r/(r 2) = o(). These two corollaries give the coclusios i Theorem 4. i Hjort et al (2009) uder slightly weaker coditios i the case of Corollary 7.2. We ow preset some additioal results that allow for larger m if r is small. For example, if r = 4, Corollary 7.2 requires m 6 = o(), while Theorem 7.2 below allows m 4 = o(). For r = 3, Corollary 7.2 requires m 9 = o(), while Theorem 7.3 below allows m 6 = o(). Theorem 7.2. Suppose (C) holds ad w 4 dq = O(m 2 ). The (.2) holds if m 4 = o(). Proof. Usig (L2) ad m 4 = o() we derive that m /2 w is Lideberg. This latter coditio ad (C) imply (D) (D2) as show prior to Theorem 7.. Next we calculate E[ W W 2 ] E[ w 4 (Z)] = O(m 2 ). This yields (D3) i view of W W o W W = O p (m / ) ad m 4 = o(). Fially we have (D4) as the left-had side of (D4) is bouded by m 2 w(z j ) 4 = O p (m 4 ) = o p (). Thus (D0) (D4) hold ad we obtai the desired result from Theorem 6.2. Theorem 7.3. Suppose (C) holds ad w r dq = O(m r/2 ) for some 2 < r < 4. The (.2) holds if m 2r/(r 2) = o().

18 8 H. Peg ad A. Schick Proof. There is a costat B such that w r dq Bm r/2. I view of (L2) ad the properties of m we derive that m /2 w is Lideberg. This coditio ad (C) imply (D0) (D2). It follows from (D), the momet coditio o w, ad the properties of m that m 2 w(z j ) 4 m2 w(z j ) r max w (Z j ) 4 r j = o p (m 2 m r/2 (/m ) (4 r)/2 ) = o p (m r (4 r)/2 ) = o p (). This establishes (D4). Fially (D3) follows as we have W W o = o p (m ). To prove the latter we mimic the argumet prior to Theorem 7. used to verify (D3) if m w is Lideberg. But ow w [m /2 w ] plays the role of v. For the correspodig matrices W ad W 2 we have m 2 E[ W E[ W ( 2 ] m2 ) (4 r)/2bm r/2 m Bmr 0, r/2 P ( W 2 0) P ( max j m/2 w (Z j ) > /2 ) 0, m E[ W,2 ] o r/2 m w r (r 2)/2 dq Bmr 0. r/2 Cosequetly (D0) (D4) hold ad the desired result follows. Now we study { ˆR = sup π j : π P, } π j ŵ (Z j ) = 0, where ŵ is a estimator of w. Let us set Ŵ = ŵ (Z j )ŵ (Z j ). Theorem 7.4. Suppose (C) holds ad assume we have the expasios m max ŵ (Z j ) = o p ( /2 ), (7.2) j Ŵ W o = o p (m /2 ) (7.3) ŵ (Z j ) = v (Z j ) + o p ( /2 ) (7.4)

19 A Empirical Likelihood Approach To Goodess of Fit Testig 9 for some measurable fuctio v from S ito R m such that v dq = 0 ad v is Lideberg. Furthermore assume that the dispersio matrix U = W /2 v v dqw /2 of W /2 v (Z) satisfies U o = O() ad m /trace(u) 2 is bouded. The, as m teds to ifiity with, ( 2 log ˆR trace(u ))/ 2trace(U) 2 is asymptotically stadard ormal. Proof. Set ξ j = W /2 v (Z j ), ad itroduce the averages v = v (Z j ) ad T = ŵ(z j ). It follows from (C) that W /2 o + W /2 o = O(). Usig this ad the Lideberg property of v we derive L (ɛ) = E[ ξ, 2 [ ξ, > ɛ ]] 0, ɛ > 0, (7.5) We have trace(u )/trace(u) 2 U o m /trace(u) 2 = O(). From m /trace(u) 2 = O() we ( coclude trace(u) 2. Thus Theorem 2 i Peg ad Schick (202) yields that v W v trace(u ) )/ 2trace(U) 2 is asymptotically stadard ormal. From this, (C), trace(u ) = O(m ) ad trace(u) 2 U 2 0m we coclude v 2 = O p (m ). With the help of (7.4) ad the assumptio m /trace(u) 2 = O() we the derive T 2 = O p (m ) ad that ( T W T trace(u ) )/ 2trace(U) 2 is asymptotically stadard ormal. Thus i view of (B), coditios (A) (A4) hold with T j = ŵ (Z j ), ad the desired result follows from Theorem 6.2. Let us first metio the special case whe v = w. I this case U equals I m trace(u ) = trace(u) 2 = m. ad Corollary 7.3. Suppose (C), (7.2) ad (7.3) hold, w is Lideberg, ad the followig expasio is valid, ŵ (Z j ) = w (Z j ) + o p ( /2 ). (7.6) The ( 2 log ˆR m )/ 2m is asymptotically stadard ormal. Next we treat v = w A ψ with A ad ψ as i the ext coditio. (C2) There is a measurable fuctio ψ from Z ito R q satisfyig ψ dq = 0 ad ψψ dq = I q such that, with A = w ψ dq, the expasio ŵ (Z j ) = w (Z j ) A ψ(z j ) + o p ( /2 ) ad the covergece trace(a W A ) q hold.

20 20 H. Peg ad A. Schick Corollary 7.4. Suppose (C), (C2), (7.2) ad (7.3) hold, ad w is Lideberg. The ( 2 log ˆR m + q)/ 2(m q) is asymptotically stadard ormal. Remark 7.. Suppose that w is the vector formed by the first m elemets of a orthoormal basis u, u 2,... for L 2,0 (Q). The the ν-th colum of the matrix A is formed by the first m Fourier coefficiets of the ν-th compoet of ψ with respect to this basis. I this case we have the idetity q m ( 2 trace(a W A ) = trace(a A ) = ψ ν u k dq) ν= k= ad obtai uder the assumptios ψ dq = 0 ad ψψ dq = I q the covergece trace(a W A ) ψ 2 dq = q. I our goodess-of-fit examples the followig coditio holds. (C3) There is a costat B such that w B m ad ŵ B m. Uder this coditio, the rate m 3 / 0 implies (7.2), the Lideberg property of m w, ad (D3). Sufficiet coditios for (7.3) ca ow be give directly or by verifyig Ŵ W o = o p (m /2 ). (7.7) I view of the iequality (4.3) a sufficiet coditio for the latter is D = ŵ (Z j ) w (Z j ) 2 = o p (m ). (7.8) Thus we have the followig results. Corollary 7.5. Suppose (C), (C3), m 3 = o(), ad oe of (7.3), (7.7), (7.8) hold. The (i) (7.6) implies that ( 2 log ˆR m )/ 2m is asymptotically stadard ormal, while (ii) (C2) implies that ( 2 log ˆR m +q)/ 2(m q) is asymptotically stadard ormal. Remark 7.2. The coditios i Theorem 7.4 are based o the sufficiet coditio (B) for (A) ad (A4). Workig with (A) ad (B2) istead, we see that (7.2) ca be replaced by the coditios, m /2 max ŵ (Z j ) = O p ( /2 m 2 ) ad ŵ (Z j ) 4 = o p (). j With D as i (7.8), we derive the bouds max ŵ (Z j ) max w (Z j ) + (D ) /2 j j m 2 ŵ (Z j ) 4 8m2 w (Z j ) 4 + 8m 2 D. 2

21 A Empirical Likelihood Approach To Goodess of Fit Testig 2 Here we used that (a + b) 4 8(a 4 + b 4 ) for o-egative a ad b. Assume ow that w 4 dq = O(m 2 ) ad that m 4 / 0. The we have (D) ad (D3) as show i the proof of Theorem 7.2 ad obtai the above two coditios ad (7.3) from (7.8). Corollary 7.6. Suppose (C), (7.8), w 4 dq = O(m 2 ) ad m 4 = o() hold. The (i) (7.6) implies that ( 2 log ˆR m )/ 2m is asymptotically stadard ormal, while (ii) (C2) implies that ( 2 log ˆR m + q)/ 2(m q) is asymptotically stadard ormal. Remark 7.3. Let us ow describe the behavior of 2 log ˆR uder a local alterative. For this we follow Remarks 6 ad 7 i Peg ad Schick (202). As there let h be a measurable fuctio satisfyig h dq = 0 ad h 2 dq < ad let Q,h be a distributio satisfyig /2 ( dq,h dq) (/2)h dq 2 0. (7.9) The the product measures Q,h ad Q are mutually cotiguous. All results i this sectio obtai the expasio 2 log ˆR /2 u (Z j ) 2 = o p (m /2 ) (7.0) for some measurable fuctio u from Z ito R m with the properties u dq = 0, u 2 dq = O(m ), u is Lideberg, ad the matrix U = u u dq satisfies U o = O() ad m /trace(u) 2 = O(). For example, i Theorem 7.4 oe has u = W /2 v. By cotiguity, oe has the expasio (7.0) eve if Z,..., Z are idepedet with distributio Q,h. Uder this distributioal assumptio oe has ( /2 u (Z j ) 2 µ (h) 2 trace(u ) )/ 2trace(U) 2 = N(0, ) with µ (h) = u h dq. Thus, uder the local alterative Q,h oe has ( 2 log ˆR µ (h) 2 trace(u ) )/ 2trace(U 2 ) = N(0, ). If U = I m, this simplifies to ( 2 log ˆR µ (h) 2 m )/ 2m = N(0, ) ad may be iterpreted as 2 log R beig approximately a o-cetral chi-square radom variable with m degrees of freedom ad o-cetrality parameter µ (h). 8. Details for the examples I this sectio we use the results of the previous sectio to provide the details for the examples of Sectios 2. I all examples, the compoets of w are orthoormal ad uiformly bouded, so that (C) ad (C3) hold with W = I m. We begi with a techical lemma.

22 22 H. Peg ad A. Schick Lemma 8.. Let (S, T ),..., (S, T ) be idepedet copies of the bivariate radom vector (S, T ), where T has a cotiuous distributio fuctio H ad E[S T ] = 0 ad σ 2 (T ) = E[S 2 T ] is bouded (by say B) ad bouded away from zero (by say b), Let H deote the empirical distributio fuctio based o T,..., T. Set u r = (, φ,..., φ r ), D j = u r (H(T j )) u r (H(T j )), ad M = E[S 2 u r (H(T ))u r (H(T )]. The we have the followig iequalities b v Mv B, v R +r, v =, (8.) u r (H(T j ))u 2 6π 2 r 2 ( + r) 2 r (H(T j )) I +r 2 a.s., (8.2) E[ S j D j 2 ] [ /2 E ] BE[ D j 2 ] Bπ2 r 3, (8.3) 2] S j D j Moreover, if E[S 4 ] is fiite, the we have the boud E[S 4 u r (H(T )) 4 ) ( + 2r) 2 E[S 4 ]. 2Bπ2 r 3. (8.4) Proof. The last iequality follows from the boud u r 2 + 2r. The iequality (8.) is a easy cosequece of b σ 2 (T ) B. Coditioig o T,..., T shows that the lefthad side of (8.4) is bouded by the left-had side of (8.3) ad yields the first iequality i (8.3). Sice φ k 2πk, we obtai D j 2 2πr 3 (H(T j ) H(T j )) 2. It is easy to check that E[(H(T j ) H(T j )) 2 ] /. This proves (8.3) ad (8.4). Next, we have almost surely, u r (H(T j ))u r (H(T j )) = u r (j/)u r (j/)). For a fuctio h defied o [0, ] with Lipschitz costat L, we have h(j/) 0 h(u) du sup j u j h(j/) h(u) L/. Sice the fuctio φ k φ l is Lipschitz with Lipschitz costat 2π(k + l), we derive the desired boud (8.2). Details for Example 2. Let X,..., X be idepedet copies of a radom variable X that has distributio fuctio F θ ad desity f θ for some θ i the ope subset Θ of R q. Recall we assumed i Example 2 that the map ϑ s ϑ = f ϑ is cotiuously differetiable i L 2 with derivative ϑ ṡ ϑ ad that the iformatio matrix J(ϑ) = 4 ṡ ϑ (x)ṡ ϑ (x) dx is ivertible for each ϑ i Θ. Thus we have ρ(τ) = (s θ+τ (x) s θ (x) τ ṡ θ (x)) 2 dx = o( τ 2 ). (8.5)

23 A Empirical Likelihood Approach To Goodess of Fit Testig 23 Recall also that l θ = 2ṡ θ /s θ deotes the score fuctio. By the properties of the desities, there is a δ > 0 ad a costat K such that f ϑ (x) f ϑ2 (x) dx K ϑ ϑ 2, ϑ θ < δ, ϑ 2 θ < δ. (8.6) As a cosequece we have sup F ϑ (x) F ϑ2 (x) K ϑ ϑ 2, ϑ θ < δ, ϑ 2 θ < δ. (8.7) x R Let m = m ad log()m 3 = o(). It suffices to show ( 2 log R (Fˆθ) m + q )/ 2(m q) = N(0, ). For this, we take w = q F θ ad ŵ = q Fˆθ with q = (φ,..., φ m ) ad verify (7.7) ad (C2) with ψ = J(θ) /2 l θ. The desired result the follows from (ii) of Corollary 7.5. We have W = I m = ŵ ŵ dfˆθ ad obtai ŵ ŵ df θ W 2m fˆθ(x) f θ (x) dx = o p (m /2 ) i view of (8.6) ad (2.2). Thus (7.7) follows if we verify Ŵ W ŵ ŵ 2 df θ + W = op (m ). (8.8) Note that ψ has mea 0 ad idetity dispersio matrix uder F θ ad that A ψ equals D J(θ) l θ, with D = w l θ df θ. Thus (C2) follows from Remark 7., ŵ (X j ) w (X j ) + D (ˆθ θ) = o p ( /2 ), (8.9) the stochastic expasio (2.2), ad the fact that D o is bouded. We are left to verify (8.8) ad (8.9). For this we set U k (t) = V kl (t) = [φ k (F θ+ t(x /2 j )) φ k (F θ (X j ))], [(φ k φ l )(F θ+ t(x /2 j )) (φ k φ l )(F θ (X j ))], ad ote that D = (d,..., d m ) with d k = φ k (F θ ) l θ df θ. The statemets (8.8) ad

24 24 H. Peg ad A. Schick (8.9) follow if we show that, for each fiite C, m m T (C) = sup (V kl (t) E[V kl (t)]) 2 = o p (m ), t C k= l= m T 2 (C) = sup (U k (t) E[U k (t)]) 2 = o p ( ), t C k= m T 3 (C) = sup (E[U k (t)] + /2 d k t) 2 = o( ). t C k= The first two statemets ca be verified usig the expoetial iequality give i Lemma 5.2 i Peg ad Schick (2004). This requires the fact that (log )m 3 / 0. The idetity f θ+τ f θ l θ τf θ = 2s θ (s θ+τ s θ ṡ θ τ)+(s θ+τ s θ ) 2 ad the defiitio of d k yield the formula φ k (F θ (x)))(f θ+τ (x) f θ (x)) dx = d k τ + φ k (F θ (x))(s θ+τ (x) s θ (x)) 2 dx + 2 φ k (F θ (x))s θ (x)(s θ+τ (x) s θ (x) ṡ θ (x)τ) dx. I view of this ad the fact that φ k (F ϑ ) df ϑ = 0 for all ϑ, we have the idetity E[U k (t)] + d k t /2 = (φ k (F θ (x)) φ k (F θ+ t(x)))(f /2 θ+ t(x) f /2 θ (x)) dx φ k (F θ (x))(s θ+ t(x) s /2 θ (x)) 2 dx 2 φ k (F θ (x))s θ (x)(s θ+ t(x) s /2 θ (x) /2 t ṡ θ (x)) dx. Usig (8.6), (8.7) ad the orthoormality of the the fuctios s θ φ k F θ, k =, 2,..., i L 2, T 3 (C) ca be bouded by 6π 2 m 3 K 4 C 4 ( m (s θ+ t(x) s /2 θ (x)) dx) sup ρ( /2 t). t C The desired statemet T 3 (C) = o( ) ow follows from (8.5) ad m 3 = o(). This completes the proof of (7.4). Details for Example 3. Assume that the distributio fuctio of X is symmetric ad cotiuous. The S = sig(x) ad T = X are idepedet, S has mea zero ad variace, ad T has a cotiuous distributio fuctio H. Let R be defied as i Example 3 with as r = r ad r 3 = o(). It suffices to show that ( 2 log R ( + r ))/ 2( + r ) is asymptotically stadard ormal. This follows from Corollary 7.5 if we verify (7.3) ad (7.6). These coditios follow from Lemma 8. applied with S j = sig(x j ) ad T j = X j. Ideed, i view of the properties of r, (7.3) is a cosequece of (8.2) ad (7.6) of (8.4).

25 A Empirical Likelihood Approach To Goodess of Fit Testig 25 Details for Example 4. Assume that X ad Y are idepedet. Part (a) is a immediate cosequece of Corollary 7.. Part (b) follows if we show ( 2 log R (F, G) r 2 )/ 2r is asymptotically stadard ormal. We shall use Corollary 7.5 to coclude this. Here m equals r 2 ad thus satisfies m 3 = o(). We shall ow verify (7.8) ad (7.6). Let us set D klj = φ k (F(X j ))φ l (G(Y j )) φ k (F (X j ))φ l (G(Y j )), Φ kj = φ k (F(X j )) φ k (F (X j )) ad Γ lj = φ l (G(Y j )) φ l (G(Y j )). I view of the iequality D klj 2 Φ kj + 2 Γ lj, we obtai with the help of (8.3) the boud r r E[ D klj 2 ] 8π2 r 4. k= l= From this ad r 6 = o() we coclude (7.8). I view of the idetity D klj = φ k (F (X j ))Γ lj + φ l (G(Y j ))Φ kj + Φ kl Γ jl, (7.6) follows if we verify r r T = ( 2 /2 φ k (F (X j ))Γ lj )) = op (), T 2 = k= l= r r k= l= r r T 3 = k= l= ( 2 /2 Φ kj φ l (G(Y j ))) = op (), ( ) 2 /2 Φ kj Γ lj = op (). Applicatios of (8.4) with S j = φ k (F (X j )) yield the boud E[T ] π 2 r 4 /, ad this proves T = o p (). The proof of T 2 = o p () is similar. To deal with T 3 we set H(k, l) = Φ k,j Γ lj, Φk = Φ kj ad Γ l = Γ lj. Note that R j = F(X j ) is the rak of X j. Give Y,..., Y ad the order statistics X (),..., X (), the sum H(k, l) is a simple liear rak statistic with scores a(j) = φ k (j/) φ k (F (X (j) ) ad coefficiets G lj ad cosequetly has (coditioal) mea Φ k Γl ad (coditioal) variace (Φ kj Φ k ) 2 (Γ li Γ l ) 2 I view of this boud we derive the iequality E[T 3 ] E[ k= Φ 2 kj] r l= E[ Γ 2 lj] + r k= Φ 2 kj E[ Φ 2 k] Γ 2 lj. r l= E[ Γ 2 l ].

26 26 H. Peg ad A. Schick We have E[ Γ 2 k) = E[ Φ 2 l ] = + ( ) 2 φ k (j/) + 2π2 k 2 2. Usig this ad (8.3), we obtai E[T 3 ] = O(r 6 2 ) = o() ad thus T 3 = o p (). Ackowledgemets This work was completed while Ato Schick was visitig the Departmet of Statistics at Texas A&M Uiversity. He wats to thak the members of the departmet for their extraordiary hospitality. Thaks go also to Igrid Va Keilegom for discussios ad for providig a importat referece. Haxiag Peg s research was supported i part by NSF Grat DMS Ato Schick s research was supported i part by NSF Grat DMS Refereces [] Che, S.X., Peg, L. ad Qi, Y.-L. (2009). Effects of data dimesio o empirical likelihood. Biometrika 96, [2] Che, S.X. ad Qi, Y.S. (2000). Empirical likelihood cofidece itervals for local liear smoothers. Biometrika 87, [3] Che, S.X. ad Va Keilegom, I. (2009). A goodess-of-fit test for parametric ad semiparametric models i multirespose regressio. Beroulli, 5, [4] Hall, P. ad La Scala, B. (990). Methodology ad algorithms of empirical likelihood. Iterat. Statist. Review, 58, [5] DiCiccio, T.J., Hall, P. ad Romao, J.P. (99). A. Statist. 9, [6] Emerso, S. ad Owe, A. (2009). Calibratio of the empirical likelihood method for a vector mea. Electro. J. Statist. 3, [7] Hellad, I. (982). Cetral limit theorems for martigales with discrete or cotiuous time. Scad. J. Statist. 9, [8] Hjort, N.L., McKeague, I.W. ad Va Keilegom, I. (2009). Extedig the scope of empirical likelihood. A. Statist. 37, 079. [9] Iglot, T. ad Ledwia, R. (996). Asymptotic optimality of data-drive Neyma s tests for uiformity. A. Statist. 24, [0] Kolaczyk, E.D. (994). Empirical likelihood for geeralized liear models. Statist. Siica 4, [] Li, G. ad Wag, Q.-H. (2003). Empirical likelihood regressio aalysis for right cesored data. Statist. Siica 3, [2] Liu, Y. ad Che, J. (200). Adjusted empirical likelihood with high-order precisio. A. Statist. 38, [3] Neyma, J. (937). Smooth test for goodess of fit. Skad. Aktuarietidskr. 20,

27 A Empirical Likelihood Approach To Goodess of Fit Testig 27 [4] Owe, A. (988). Empirical likelihood ratio cofidece itervals for a sigle fuctioal. Biometrika 75, [5] Owe, A. (990). Empirical likelihood ratio cofidece regios. A. Statist. 8, [6] Owe, A. (99). Empirical likelihood for liear models. A. Statist. 9, [7] Owe, A. (200). Empirical Likelihood. Chapma & Hall/CRC, Lodo. [8] Peg, H. ad Schick, A. (2004). Estimatio of liear fuctioals of bivariate distributios with parametric margials. Statist. Decisios 22, [9] Peg, H. ad Schick, A. (2005). Efficiet estimatio of liear fuctioals of a bivariate distributio with equal, but ukow margials: the least-squares approach. J. Multivariate Aal. 95, [20] Peg, H. ad Schick, A. (202). Asymptotic ormality of quadratic forms with radom vectors of icreasig dimesio. Preprit. [2] Portoy, S. (988). Behavior of likelihood methods for expoetial families whe the umber of parameters teds to ifiity. A. Statist. 6, [22] Qi, G. ad Jig, B.-Y. (200). Empirical likelihood for cesored liear regressio. Scad. J. Statist. 28, [23] Qi, J. ad Lawless, J. (994). Empirical likelihood ad geeral estimatig equatios. A. Statist. 22, [24] Shi, J. ad Lau, T.S. (2000). Empirical likelihood for partially liear models. J. Multivariate Aal. 72, [25] Tripathi, G. ad Kitamura, Y. (2003). Testig coditioal momet restrictios A. Statist. 3, [26] Wag, Q.-H. ad Jig, B.-Y. (200). Empirical likelihood for a class of fuctioals of survival distributio with cesored data. A. Ist. Statist. Math. 53, [27] Wag, Q.-H. ad Jig, B.-Y. (2003). Empirical likelihood for partially liear models. A. Ist. Statist. Meth. 55, [28] Zhou, M. ad Li, G. (2008). Empirical likelihood aalysis of the Buckley-James estimator. J Multivariate Aal. 99,

Empirical likelihood approach to goodness of fit testing

Empirical likelihood approach to goodness of fit testing Beroulli 9(3), 203, 954 98 DOI: 0.350/2-BEJ440 arxiv:307.5982v [math.st] 23 Jul 203 Empirical likelihood approach to goodess of fit testig HANXIANG PENG ad ANTON SCHICK 2 Departmet of Mathematical Scieces,

More information

Convergence of random variables. (telegram style notes) P.J.C. Spreij

Convergence of random variables. (telegram style notes) P.J.C. Spreij Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space

More information

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i

More information

Rank tests and regression rank scores tests in measurement error models

Rank tests and regression rank scores tests in measurement error models Rak tests ad regressio rak scores tests i measuremet error models J. Jurečková ad A.K.Md.E. Saleh Charles Uiversity i Prague ad Carleto Uiversity i Ottawa Abstract The rak ad regressio rak score tests

More information

Lecture 19: Convergence

Lecture 19: Convergence Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may

More information

Asymptotic normality of quadratic forms with random vectors of increasing dimension

Asymptotic normality of quadratic forms with random vectors of increasing dimension Asymptotic ormality of quadratic forms with radom vectors of icreasig dimesio Haxiag Peg ad Ato Schick Abstract. This paper provides sufficiet coditios for the asymptotic ormality of quadratic forms of

More information

7.1 Convergence of sequences of random variables

7.1 Convergence of sequences of random variables Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite

More information

Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.

Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n. Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator

More information

32 estimating the cumulative distribution function

32 estimating the cumulative distribution function 32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio

More information

Sequences and Series of Functions

Sequences and Series of Functions Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges

More information

Journal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula

Journal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula Joural of Multivariate Aalysis 102 (2011) 1315 1319 Cotets lists available at ScieceDirect Joural of Multivariate Aalysis joural homepage: www.elsevier.com/locate/jmva Superefficiet estimatio of the margials

More information

Self-normalized deviation inequalities with application to t-statistic

Self-normalized deviation inequalities with application to t-statistic Self-ormalized deviatio iequalities with applicatio to t-statistic Xiequa Fa Ceter for Applied Mathematics, Tiaji Uiversity, 30007 Tiaji, Chia Abstract Let ξ i i 1 be a sequece of idepedet ad symmetric

More information

Expectation and Variance of a random variable

Expectation and Variance of a random variable Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio

More information

Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.

Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f. Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,

More information

Product measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.

Product measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014. Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the

More information

This exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.

This exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam. Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the

More information

Definition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.

Definition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4. 4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad

More information

Probability and Statistics

Probability and Statistics ICME Refresher Course: robability ad Statistics Staford Uiversity robability ad Statistics Luyag Che September 20, 2016 1 Basic robability Theory 11 robability Spaces A probability space is a triple (Ω,

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak

More information

LECTURE 8: ASYMPTOTICS I

LECTURE 8: ASYMPTOTICS I LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece

More information

4. Partial Sums and the Central Limit Theorem

4. Partial Sums and the Central Limit Theorem 1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

Fall 2013 MTH431/531 Real analysis Section Notes

Fall 2013 MTH431/531 Real analysis Section Notes Fall 013 MTH431/531 Real aalysis Sectio 8.1-8. Notes Yi Su 013.11.1 1. Defiitio of uiform covergece. We look at a sequece of fuctios f (x) ad study the coverget property. Notice we have two parameters

More information

Efficient GMM LECTURE 12 GMM II

Efficient GMM LECTURE 12 GMM II DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet

More information

POWER COMPARISON OF EMPIRICAL LIKELIHOOD RATIO TESTS: SMALL SAMPLE PROPERTIES THROUGH MONTE CARLO STUDIES*

POWER COMPARISON OF EMPIRICAL LIKELIHOOD RATIO TESTS: SMALL SAMPLE PROPERTIES THROUGH MONTE CARLO STUDIES* Kobe Uiversity Ecoomic Review 50(2004) 3 POWER COMPARISON OF EMPIRICAL LIKELIHOOD RATIO TESTS: SMALL SAMPLE PROPERTIES THROUGH MONTE CARLO STUDIES* By HISASHI TANIZAKI There are various kids of oparametric

More information

Empirical Processes: Glivenko Cantelli Theorems

Empirical Processes: Glivenko Cantelli Theorems Empirical Processes: Gliveko Catelli Theorems Mouliath Baerjee Jue 6, 200 Gliveko Catelli classes of fuctios The reader is referred to Chapter.6 of Weller s Torgo otes, Chapter??? of VDVW ad Chapter 8.3

More information

A statistical method to determine sample size to estimate characteristic value of soil parameters

A statistical method to determine sample size to estimate characteristic value of soil parameters A statistical method to determie sample size to estimate characteristic value of soil parameters Y. Hojo, B. Setiawa 2 ad M. Suzuki 3 Abstract Sample size is a importat factor to be cosidered i determiig

More information

Chapter 6 Infinite Series

Chapter 6 Infinite Series Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat

More information

Integrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number

Integrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios

More information

Properties and Hypothesis Testing

Properties and Hypothesis Testing Chapter 3 Properties ad Hypothesis Testig 3.1 Types of data The regressio techiques developed i previous chapters ca be applied to three differet kids of data. 1. Cross-sectioal data. 2. Time series data.

More information

MATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4

MATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4 MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.

More information

Berry-Esseen bounds for self-normalized martingales

Berry-Esseen bounds for self-normalized martingales Berry-Essee bouds for self-ormalized martigales Xiequa Fa a, Qi-Ma Shao b a Ceter for Applied Mathematics, Tiaji Uiversity, Tiaji 30007, Chia b Departmet of Statistics, The Chiese Uiversity of Hog Kog,

More information

Problem Set 4 Due Oct, 12

Problem Set 4 Due Oct, 12 EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

Summary and Discussion on Simultaneous Analysis of Lasso and Dantzig Selector

Summary and Discussion on Simultaneous Analysis of Lasso and Dantzig Selector Summary ad Discussio o Simultaeous Aalysis of Lasso ad Datzig Selector STAT732, Sprig 28 Duzhe Wag May 4, 28 Abstract This is a discussio o the work i Bickel, Ritov ad Tsybakov (29). We begi with a short

More information

Advanced Stochastic Processes.

Advanced Stochastic Processes. Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.

More information

Law of the sum of Bernoulli random variables

Law of the sum of Bernoulli random variables Law of the sum of Beroulli radom variables Nicolas Chevallier Uiversité de Haute Alsace, 4, rue des frères Lumière 68093 Mulhouse icolas.chevallier@uha.fr December 006 Abstract Let be the set of all possible

More information

IMPROVING EFFICIENT MARGINAL ESTIMATORS IN BIVARIATE MODELS WITH PARAMETRIC MARGINALS

IMPROVING EFFICIENT MARGINAL ESTIMATORS IN BIVARIATE MODELS WITH PARAMETRIC MARGINALS IMPROVING EFFICIENT MARGINAL ESTIMATORS IN BIVARIATE MODELS WITH PARAMETRIC MARGINALS HANXIANG PENG AND ANTON SCHICK Abstract. Suppose we have data from a bivariate model with parametric margials. Efficiet

More information

Introduction to Extreme Value Theory Laurens de Haan, ISM Japan, Erasmus University Rotterdam, NL University of Lisbon, PT

Introduction to Extreme Value Theory Laurens de Haan, ISM Japan, Erasmus University Rotterdam, NL University of Lisbon, PT Itroductio to Extreme Value Theory Laures de Haa, ISM Japa, 202 Itroductio to Extreme Value Theory Laures de Haa Erasmus Uiversity Rotterdam, NL Uiversity of Lisbo, PT Itroductio to Extreme Value Theory

More information

Statistics 511 Additional Materials

Statistics 511 Additional Materials Cofidece Itervals o mu Statistics 511 Additioal Materials This topic officially moves us from probability to statistics. We begi to discuss makig ifereces about the populatio. Oe way to differetiate probability

More information

Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].

Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)]. Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x

More information

17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15

17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15 17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig

More information

b i u x i U a i j u x i u x j

b i u x i U a i j u x i u x j M ath 5 2 7 Fall 2 0 0 9 L ecture 1 9 N ov. 1 6, 2 0 0 9 ) S ecod- Order Elliptic Equatios: Weak S olutios 1. Defiitios. I this ad the followig two lectures we will study the boudary value problem Here

More information

A RANK STATISTIC FOR NON-PARAMETRIC K-SAMPLE AND CHANGE POINT PROBLEMS

A RANK STATISTIC FOR NON-PARAMETRIC K-SAMPLE AND CHANGE POINT PROBLEMS J. Japa Statist. Soc. Vol. 41 No. 1 2011 67 73 A RANK STATISTIC FOR NON-PARAMETRIC K-SAMPLE AND CHANGE POINT PROBLEMS Yoichi Nishiyama* We cosider k-sample ad chage poit problems for idepedet data i a

More information

Random Variables, Sampling and Estimation

Random Variables, Sampling and Estimation Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig

More information

Distribution of Random Samples & Limit theorems

Distribution of Random Samples & Limit theorems STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to

More information

Asymptotic distribution of products of sums of independent random variables

Asymptotic distribution of products of sums of independent random variables Proc. Idia Acad. Sci. Math. Sci. Vol. 3, No., May 03, pp. 83 9. c Idia Academy of Scieces Asymptotic distributio of products of sums of idepedet radom variables YANLING WANG, SUXIA YAO ad HONGXIA DU ollege

More information

Goodness-of-Fit Tests and Categorical Data Analysis (Devore Chapter Fourteen)

Goodness-of-Fit Tests and Categorical Data Analysis (Devore Chapter Fourteen) Goodess-of-Fit Tests ad Categorical Data Aalysis (Devore Chapter Fourtee) MATH-252-01: Probability ad Statistics II Sprig 2019 Cotets 1 Chi-Squared Tests with Kow Probabilities 1 1.1 Chi-Squared Testig................

More information

Mathematical Statistics - MS

Mathematical Statistics - MS Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios

More information

( θ. sup θ Θ f X (x θ) = L. sup Pr (Λ (X) < c) = α. x : Λ (x) = sup θ H 0. sup θ Θ f X (x θ) = ) < c. NH : θ 1 = θ 2 against AH : θ 1 θ 2

( θ. sup θ Θ f X (x θ) = L. sup Pr (Λ (X) < c) = α. x : Λ (x) = sup θ H 0. sup θ Θ f X (x θ) = ) < c. NH : θ 1 = θ 2 against AH : θ 1 θ 2 82 CHAPTER 4. MAXIMUM IKEIHOOD ESTIMATION Defiitio: et X be a radom sample with joit p.m/d.f. f X x θ. The geeralised likelihood ratio test g.l.r.t. of the NH : θ H 0 agaist the alterative AH : θ H 1,

More information

Slide Set 13 Linear Model with Endogenous Regressors and the GMM estimator

Slide Set 13 Linear Model with Endogenous Regressors and the GMM estimator Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday

More information

ECE 901 Lecture 12: Complexity Regularization and the Squared Loss

ECE 901 Lecture 12: Complexity Regularization and the Squared Loss ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality

More information

Econ 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara

Econ 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio

More information

Estimation for Complete Data

Estimation for Complete Data Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of

More information

Lecture 7: Properties of Random Samples

Lecture 7: Properties of Random Samples Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ

More information

Kernel density estimator

Kernel density estimator Jauary, 07 NONPARAMETRIC ERNEL DENSITY ESTIMATION I this lecture, we discuss kerel estimatio of probability desity fuctios PDF Noparametric desity estimatio is oe of the cetral problems i statistics I

More information

2.2. Central limit theorem.

2.2. Central limit theorem. 36.. Cetral limit theorem. The most ideal case of the CLT is that the radom variables are iid with fiite variace. Although it is a special case of the more geeral Lideberg-Feller CLT, it is most stadard

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece

More information

ESTIMATING THE ERROR DISTRIBUTION FUNCTION IN NONPARAMETRIC REGRESSION WITH MULTIVARIATE COVARIATES

ESTIMATING THE ERROR DISTRIBUTION FUNCTION IN NONPARAMETRIC REGRESSION WITH MULTIVARIATE COVARIATES ESTIMATING THE ERROR DISTRIBUTION FUNCTION IN NONPARAMETRIC REGRESSION WITH MULTIVARIATE COVARIATES URSULA U. MÜLLER, ANTON SCHICK AND WOLFGANG WEFELMEYER Abstract. We cosider oparametric regressio models

More information

An Introduction to Randomized Algorithms

An Introduction to Randomized Algorithms A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.

More information

Asymptotic Results for the Linear Regression Model

Asymptotic Results for the Linear Regression Model Asymptotic Results for the Liear Regressio Model C. Fli November 29, 2000 1. Asymptotic Results uder Classical Assumptios The followig results apply to the liear regressio model y = Xβ + ε, where X is

More information

Notes 19 : Martingale CLT

Notes 19 : Martingale CLT Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall

More information

ON BARTLETT CORRECTABILITY OF EMPIRICAL LIKELIHOOD IN GENERALIZED POWER DIVERGENCE FAMILY. Lorenzo Camponovo and Taisuke Otsu.

ON BARTLETT CORRECTABILITY OF EMPIRICAL LIKELIHOOD IN GENERALIZED POWER DIVERGENCE FAMILY. Lorenzo Camponovo and Taisuke Otsu. ON BARTLETT CORRECTABILITY OF EMPIRICAL LIKELIHOOD IN GENERALIZED POWER DIVERGENCE FAMILY By Lorezo Campoovo ad Taisuke Otsu October 011 COWLES FOUNDATION DISCUSSION PAPER NO. 185 COWLES FOUNDATION FOR

More information

7.1 Convergence of sequences of random variables

7.1 Convergence of sequences of random variables Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite

More information

f n (x) f m (x) < ɛ/3 for all x A. By continuity of f n and f m we can find δ > 0 such that d(x, x 0 ) < δ implies that

f n (x) f m (x) < ɛ/3 for all x A. By continuity of f n and f m we can find δ > 0 such that d(x, x 0 ) < δ implies that Lecture 15 We have see that a sequece of cotiuous fuctios which is uiformly coverget produces a limit fuctio which is also cotiuous. We shall stregthe this result ow. Theorem 1 Let f : X R or (C) be a

More information

Unbiased Estimation. February 7-12, 2008

Unbiased Estimation. February 7-12, 2008 Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom

More information

ECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002

ECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002 ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom

More information

Introductory statistics

Introductory statistics CM9S: Machie Learig for Bioiformatics Lecture - 03/3/06 Itroductory statistics Lecturer: Sriram Sakararama Scribe: Sriram Sakararama We will provide a overview of statistical iferece focussig o the key

More information

Statistical Inference Based on Extremum Estimators

Statistical Inference Based on Extremum Estimators T. Rotheberg Fall, 2007 Statistical Iferece Based o Extremum Estimators Itroductio Suppose 0, the true value of a p-dimesioal parameter, is kow to lie i some subset S R p : Ofte we choose to estimate 0

More information

LECTURE 14 NOTES. A sequence of α-level tests {ϕ n (x)} is consistent if

LECTURE 14 NOTES. A sequence of α-level tests {ϕ n (x)} is consistent if LECTURE 14 NOTES 1. Asymptotic power of tests. Defiitio 1.1. A sequece of -level tests {ϕ x)} is cosistet if β θ) := E θ [ ϕ x) ] 1 as, for ay θ Θ 1. Just like cosistecy of a sequece of estimators, Defiitio

More information

ECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors

ECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic

More information

Lecture 2: Monte Carlo Simulation

Lecture 2: Monte Carlo Simulation STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?

More information

Probability and statistics: basic terms

Probability and statistics: basic terms Probability ad statistics: basic terms M. Veeraraghava August 203 A radom variable is a rule that assigs a umerical value to each possible outcome of a experimet. Outcomes of a experimet form the sample

More information

Chapter 6 Principles of Data Reduction

Chapter 6 Principles of Data Reduction Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a

More information

1 of 7 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 6. Order Statistics Defiitios Suppose agai that we have a basic radom experimet, ad that X is a real-valued radom variable

More information

Measure and Measurable Functions

Measure and Measurable Functions 3 Measure ad Measurable Fuctios 3.1 Measure o a Arbitrary σ-algebra Recall from Chapter 2 that the set M of all Lebesgue measurable sets has the followig properties: R M, E M implies E c M, E M for N implies

More information

Introduction to Probability. Ariel Yadin

Introduction to Probability. Ariel Yadin Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways

More information

1 Convergence in Probability and the Weak Law of Large Numbers

1 Convergence in Probability and the Weak Law of Large Numbers 36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec

More information

Lecture 2. The Lovász Local Lemma

Lecture 2. The Lovász Local Lemma Staford Uiversity Sprig 208 Math 233A: No-costructive methods i combiatorics Istructor: Ja Vodrák Lecture date: Jauary 0, 208 Origial scribe: Apoorva Khare Lecture 2. The Lovász Local Lemma 2. Itroductio

More information

TAMS24: Notations and Formulas

TAMS24: Notations and Formulas TAMS4: Notatios ad Formulas Basic otatios ad defiitios X: radom variable stokastiska variabel Mea Vätevärde: µ = X = by Xiagfeg Yag kpx k, if X is discrete, xf Xxdx, if X is cotiuous Variace Varias: =

More information

Lecture Notes 15 Hypothesis Testing (Chapter 10)

Lecture Notes 15 Hypothesis Testing (Chapter 10) 1 Itroductio Lecture Notes 15 Hypothesis Testig Chapter 10) Let X 1,..., X p θ x). Suppose we we wat to kow if θ = θ 0 or ot, where θ 0 is a specific value of θ. For example, if we are flippig a coi, we

More information

Statistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.

Statistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample. Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized

More information

Lecture 3 The Lebesgue Integral

Lecture 3 The Lebesgue Integral Lecture 3: The Lebesgue Itegral 1 of 14 Course: Theory of Probability I Term: Fall 2013 Istructor: Gorda Zitkovic Lecture 3 The Lebesgue Itegral The costructio of the itegral Uless expressly specified

More information

Simulation. Two Rule For Inverting A Distribution Function

Simulation. Two Rule For Inverting A Distribution Function Simulatio Two Rule For Ivertig A Distributio Fuctio Rule 1. If F(x) = u is costat o a iterval [x 1, x 2 ), the the uiform value u is mapped oto x 2 through the iversio process. Rule 2. If there is a jump

More information

Notes 5 : More on the a.s. convergence of sums

Notes 5 : More on the a.s. convergence of sums Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series

More information

OFF-DIAGONAL MULTILINEAR INTERPOLATION BETWEEN ADJOINT OPERATORS

OFF-DIAGONAL MULTILINEAR INTERPOLATION BETWEEN ADJOINT OPERATORS OFF-DIAGONAL MULTILINEAR INTERPOLATION BETWEEN ADJOINT OPERATORS LOUKAS GRAFAKOS AND RICHARD G. LYNCH 2 Abstract. We exted a theorem by Grafakos ad Tao [5] o multiliear iterpolatio betwee adjoit operators

More information

Lecture 33: Bootstrap

Lecture 33: Bootstrap Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece

More information

Technical Proofs for Homogeneity Pursuit

Technical Proofs for Homogeneity Pursuit Techical Proofs for Homogeeity Pursuit bstract This is the supplemetal material for the article Homogeeity Pursuit, submitted for publicatio i Joural of the merica Statistical ssociatio. B Proofs B. Proof

More information

Exercise 4.3 Use the Continuity Theorem to prove the Cramér-Wold Theorem, Theorem. (1) φ a X(1).

Exercise 4.3 Use the Continuity Theorem to prove the Cramér-Wold Theorem, Theorem. (1) φ a X(1). Assigmet 7 Exercise 4.3 Use the Cotiuity Theorem to prove the Cramér-Wold Theorem, Theorem 4.12. Hit: a X d a X implies that φ a X (1) φ a X(1). Sketch of solutio: As we poited out i class, the oly tricky

More information

MAT1026 Calculus II Basic Convergence Tests for Series

MAT1026 Calculus II Basic Convergence Tests for Series MAT026 Calculus II Basic Covergece Tests for Series Egi MERMUT 202.03.08 Dokuz Eylül Uiversity Faculty of Sciece Departmet of Mathematics İzmir/TURKEY Cotets Mootoe Covergece Theorem 2 2 Series of Real

More information

Inference about the slope in linear regression: an empirical likelihood approach

Inference about the slope in linear regression: an empirical likelihood approach Aals of the Istitute of Statistical Mathematics mauscript No. (will be iserted by the editor) Iferece about the slope i liear regressio: a empirical likelihood approach Ursula U. Müller Haxiag Peg Ato

More information

Economics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator

Economics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters

More information

Entropy Rates and Asymptotic Equipartition

Entropy Rates and Asymptotic Equipartition Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,

More information

Chapter 6 Sampling Distributions

Chapter 6 Sampling Distributions Chapter 6 Samplig Distributios 1 I most experimets, we have more tha oe measuremet for ay give variable, each measuremet beig associated with oe radomly selected a member of a populatio. Hece we eed to

More information

Linear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d

Linear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y

More information

Random Walks on Discrete and Continuous Circles. by Jeffrey S. Rosenthal School of Mathematics, University of Minnesota, Minneapolis, MN, U.S.A.

Random Walks on Discrete and Continuous Circles. by Jeffrey S. Rosenthal School of Mathematics, University of Minnesota, Minneapolis, MN, U.S.A. Radom Walks o Discrete ad Cotiuous Circles by Jeffrey S. Rosethal School of Mathematics, Uiversity of Miesota, Mieapolis, MN, U.S.A. 55455 (Appeared i Joural of Applied Probability 30 (1993), 780 789.)

More information

Web-based Supplementary Materials for A Modified Partial Likelihood Score Method for Cox Regression with Covariate Error Under the Internal

Web-based Supplementary Materials for A Modified Partial Likelihood Score Method for Cox Regression with Covariate Error Under the Internal Web-based Supplemetary Materials for A Modified Partial Likelihood Score Method for Cox Regressio with Covariate Error Uder the Iteral Validatio Desig by David M. Zucker, Xi Zhou, Xiaomei Liao, Yi Li,

More information

FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures

FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING Lectures MODULE 5 STATISTICS II. Mea ad stadard error of sample data. Biomial distributio. Normal distributio 4. Samplig 5. Cofidece itervals

More information