Empirical Process heory 4.384 ime Series Analysis, Fall 27 Reciaion by Paul Schrimpf Supplemenary o lecures given by Anna Mikusheva Ocober 7, 28 Reciaion 7 Empirical Process heory Le x be a real-valued random k vecor. Consider some R n valued funcion g (x, τ) for τ Θ, where Θ is a subse of some meric space. Remark. In ime series applicaions, generally, Θ = [, ] Le ξ (τ) = = (g (x, τ) Eg (x, τ)) ξ (τ) is a random funcion; i maps each τ Θ o an R n valued random variable. ξ (τ) is called an empirical process. Under very general condiions, sandard argumens show ha ξ (τ) converges poinwise, i.e. τ Θ, ξ (τ ) N(, σ 2 (τ )). Also, sandard argumens imply ha on a finie collecion of poins, (τ,..., τ p ), ξ (τ ). N(, Σ(τ,..., τ p )) () ξ (τ p ) We would like o generalize his sor of resul so ha we alk abou he convergence of ξ (). Example 2. Suppose you wan o es wheher x has cdf F (x). he cdf of x can be esimaed by is empirical cdf, Fˆ (x) = (x x) wo possible saisics for esing wheher Fˆn(x) equals F (x) are he Kolmogorov-Smrinov saisic, sup n(fˆn(x) F (x)) x and he Cramer-von Mises saisic n (Fˆn(x) F (x)) 2 df (x) his fis ino he seup above wih ( ) ξ (τ) = (x τ) F (τ). For independen x, finie dimensional covergence is easy o verify and for any τ, τ 2 we have [ ] ( ( )) ξ (τ ) F (τ )( F (τ )) F (τ ) F (τ 2 ) F (τ )F (τ 2 ) N, ξ (τ 2 ) F (τ ) F (τ 2 ) F (τ )F (τ 2 ) F (τ 2 )( F (τ 2 )) Definiion 3. We define a meric for funcions on Θ as d(b, b 2 ) = sup τ Θ b (τ ) b 2 (τ)
Sufficien Condiions for Sochasic Equiconinuiy 2 Definiion 4. B = bounded funcions on Θ Definiion 5. U(B) = class of uniformly coninuous (wr d()) bounded funcionals from B o R Example 6. Examples of elemens of U(B) include: Evaluaion a a poin: f τ (ξ) = ξ(τ ) Inegraion: f(ξ) = Θ ξ(τ)dτ Definiion 7. convergence in B: ξ ξ iff f U(B) we have Ef(ξ ) Ef(ξ) Remark 8. his definiion of convergence implies poinwise convergence. If ξ ξ, hen by definiion for each τ and k, Eξ (τ ) k Eξ(τ ) k. hen, if he disribuion of ξ(τ ) is compleely deermined by is momens (as i is if, for example, ξ(τ ) is normal or has bounded suppor), i follows ha ξ (τ ) ξ(τ ). Definiion 9. ξ is sochasically equiconinuous if ɛ >, η >, here exiss δ > s.. lim P ( sup ξ (τ ) ξ (τ 2 ) > η) < ɛ τ τ 2 <δ heorem. Funcional Cenral Limi heorem: If. Θ is bounded 2. here exiss a finie-dimensional disribuion convergence of ξ o ξ (as in ()) 3. {ξ } are sochasically equiconinuous hen ξ ξ Remark. Condiion can be removed. Wihou i, condiion 3 mus be srenghened o: ɛ, η > here exiss a pariion of Θ ino finiely many ses, Θ,...Θ k such ha lim sup P (max sup ξ (τ ) ξ (τ 2 ) > η) < ɛ i τ,τ 2 Θ i Proving he heorem involves consrucing a meric on Θ such ha Θ is bounded wih respec o ha meric, so condiion is really a consequence of his sronger version of condiion 3. Remark 2. Condiion 2 can be checked. Condiion 3 is difficul o check, bu los of work has been done o derive simpler sufficien condiions. See Andrews (994 HoE) for some sufficien condiions. Necessary and sufficien condiions for sochasic equiconinuiy are no known. However, very general sufficien condiions are known. Classes of funcions for which he funcional CL holds are called P-Donsker. Sufficien Condiions for Sochasic Equiconinuiy his is largely angenial o wha we will do in class. Definiion 3. A class of funcions, G, is P-Donsker if for every g G, ( ) g(x, ) E[g(x, )] ξ where ξ l (G) In order for a class of funcions o be P-Donsker, sochasic equiconinuiy requires ha he funcion class no be oo complex. One way of measuring he complexiy of a funcion class is by brackeing numbers. An ɛ bracke in L 2, [l, u] is he se of all funcions, f, such ha l f u poinwise wih E[ l u 2 ] /2 < ɛ. he ɛ brackeing number wrien as N [] (ɛ, G) is he minimal number of ɛ brackes needed o cover G. An imporan sufficien condiion for a class o be P-Donsker is he following:
Coninuous Mapping heorem 3 heorem 4. Every class G of measurable funcions wih is P-Donsker. log N [] (ɛ, G)dɛ < Alhough his condiion looks srange and difficul, i can be verified in a number of ineresing siuaions. Example 5. Classes ha are P-Donsker include Disribuion funcions: using brackes of he form [(x < x i ), (x < x i+ )] wih F (x i+ ) F (x i ) < ɛ we can cover G wih C/ɛ 2 brackes, so is finie. log N [] (ɛ, G)dɛ log(c/ɛ 2 )dɛ = log(c) + Parameric Classes: if G = {g θ : θ Θ R k } wih Θ bounded, and a Lipschiz condiion holds: wih E[m(x) 2 ] <. g θ (x) g θ2 (x) m(x) θ θ 2 Smooh funcions from R d R wih uniformly bounded derivaes of order up o α > d/2 Anoher way of characerizing complexiy is hrough uniform covering numbers and uniform enropy inegrals, bu I am no going o say anyhing abou i here. Coninuous Mapping heorem he following heorem is imporan for making he funcional cenral limi heorem useful. heorem 6. Coninuous Mapping heorem: if ξ ξ, hen coninuous funcionals, f, f(ξ ) f(ξ) Example 7. We can use he coninuous mapping heorem o ge he disribuion of he Kolmogorov-Smirnov and Cramer-von Mises saisics. Boh: sup τ ξ(τ) and ξ(τ) 2 df (τ) are coninuous funcionals, so sup n(fˆn(x) F (x)) d sup ξ(x) x and n (Fˆn(x) F (x)) 2 df (x) d ξ(x) 2 df (x) where ξ(x) is a Brownian bridge, i.e. Gaussian wih covariance funcion as above. We can simulae sup x ξ(x) and ξ(x) 2 df (x) o find criical values for hypohesis ess. his heorem definiely holds for iid daa. I migh need o be modified for dependen daa (e.g. he form of he inegral depends on mixing coefficiens), bu I m no cerain. x
Random Walk Asympoics 4 Random Walk Asympoics In lecure 2, we saw ha if y is a random walk and we esimae an AR(), hen 2 (W 2 () ) W (s)dw (s) (ρˆ ) = W (s) 2 ds W 2 (s)ds I is imporan o undersand how we derived hese expressions because, unlike in he saionary case, small changes o he esimaed model can grealy aler he asympoic disribuion. For example, suppose we esimae an AR() wih a consan, so we esimae, y = α + ρy + u Le β = [α ρ] = [ ] and βˆ be he OLS esimae. We know ha: [ ] [ ] [ ] αˆ α = y 2 u (2) ρˆ ρ y y y u o find he asympoic disribuion, we need o examine each of he sums, deermine appropriae scaling facors, and wrie down wha hey converge o. We ve already seen each of hese sums in lecure 2, so I won rewrie he seps here, bu recall ha y σ W ()d 3/2 = 2 2 y σ 2 W 2 (s)ds = u σw () σ y u (W 2 () σ 2 ) 2 [ ] hese resuls sugges scalling βˆ by /2 o arrive a a nondegenerae asympoic disribuion, i.e. [ ] [ ] ([ ] [ ] [ ]) /2 /2 /2 [ ] [ ] /2 αˆ α y u = 2 ρˆ ρ y y y u [ ] ] = 3/2 [ y /2 u 3/2 y 2 2 y y u [ /2 (ˆα α) ] [ σ ] [ ] [ ] W (s)ds W () (ˆρ ρ) W (s)ds W (s) 2 ds 2(W () 2 ) From which, we see ha neiher ˆα nor ˆρ are asympoically normal. Also, ˆα converges a he usual / rae, bu ˆρ converges a rae /. wih Drif Now, le s consider anoher modificaion of he model. Suppose ha y is a random walk wih drif, y = y + α + e. As above, le s assume we esimae by OLS an AR() wih a consan. As above we need o analyze each of he sums in he marices in (2). We canno jus use he resuls from lecure 2 because now he process for y is differen.
wih Drif 5 y : y = (α( ) + y + es ) s< =( α( )) + y + ξ (( )/ ) For α o have a finie limi, we mus normalize i by 2. We know ha 2 y, and 2 ξ (( )/ ) (since 3/2 ξ (( )/ ) = /2 ξ (( )/ ) W (s)ds). here- fore, 2 y lim 2 α( ) = α/2. 2 y : idenical reasoning shows ha we mus normalize by 3 2 and y α 2 /3 e : is unchanged, σw () y e : y e = (( )α + y + e s )e s< 2 = e ( )α + e y + (y 2 e 2 ) he firs erm, e ( )α is O p ( 3/2 ), so we mus normalize by a leas 3/2. e y is O p ( /2 ) 2 2 and y e is O p ( ), so hey vanish. his leaves, 3/2 y e 3/2 e ( )α N(, α 2 /3) Furhermore, joinly we have: [ ] [ ] /2 e α/2 3/2 N(, σ 2 y e α/2 α 2 ) /3 Combining hese resuls, we see ha [ ] [ ] /2 (ˆα α) α/2 3/2 N(, σ 2 (ˆρ ρ) α/2 α 2 /3 hus, we obain asympoic normaliy when we esimae a random walk wih drif. Also, he asympoic variance marix is he same as sandard OLS. However, ˆρ converges a a faser rae han usual. )
MI OpenCourseWare hp://ocw.mi.edu 4.384 ime Series Analysis Fall 23 For informaion abou ciing hese maerials or our erms of Use, visi: hp://ocw.mi.edu/erms.