A uniform central limit theorem for neural network based autoregressive processes with applications to change-point analysis

Size: px

Start display at page:

Download "A uniform central limit theorem for neural network based autoregressive processes with applications to change-point analysis"

Samantha Carter
5 years ago
Views:

1 A uiform cetral limit theorem for eural etwork based autoregressive processes with applicatios to chage-poit aalysis Claudia Kirch Joseph Tadjuidje Kamgaig March 6, 20 Abstract We cosider a autoregressive process with a oliear regressio fuctio that is modeled by a feedforward eural etwork. We derive a uiform cetral limit theorem which is useful i the cotext of chage-poit aalysis. We propose a test for a chage i the autoregressio fuctio which by the uiform cetral limit theorem has asymptotic power oe for a large class of alteratives icludig local alteratives. Keywords: Uiform cetral limit theorem, oparametric regressio, eural etwork, autoregressive process AMS Subject Classificatio 2000: 60F05, 62J02, 62M45 Itroductio Limit theory i geeral ca be described as the heart of probability ad mathematical statistics. Uiform cetral limit theorems for depedet radom variables i particular The work was supported by the DFG graduate college Mathematics ad ractice as well as by the DFG grat KI 443/2-. The positio of the first author was fiaced by the Stifterverbad für die Deutsche Wisseschaft by fuds of the Clausse-Simo-trust. Karlsruhe Istitute of Techology KIT, Istitute for Stochastics, Kaiserstr. 89 D 7633 Karlsruhe, Germay; claudia.kirch@kit.edu Uiversity Kaiserslauter, Departmet of Mathematics, Erwi-Schrödiger-Straße, D Kaiserslauter, Germay; tadjuidj@mathematik.ui-kl.de

2 Itroductio have give a ew dimesio to the asymptotic theory for sums of processes idexed by families of sets by givig rise to may applicatios out of reach of the classical cetral limit theorem. I this paper we cosider a oliear autoregressive time series, where we model the autoregressio fuctio by a feedforward eural etwork. Due to its uiversal approximatio property, a large class of fuctios ca be approximated by a eural etwork to ay degree of accuracy cofer e.g. White [0] or Frake et al. [4]. Therefore, this setup is very geeral ad able to model may real-life time series while at the same time beig mathematical feasible ad computatioally easier to hadle due to its parametric ature. For θ = ν 0,..., ν H, α,..., α H, β,..., β H, α j = α j,..., α jp, fx, θ = ν 0 + H ν h ψ< α h, x > +β h,. h= deotes a oe layer feedforward eural etwork with H hidde euros, <, > is the classical scalar product o R p. I this paper we assume that ψ belogs to the class of sigmoid activatio fuctios that satisfy lim ψx = 0, lim x ψx =, ψx + ψ x =..2 x A popular example is the logistic fuctio ψx = +e x which also fulfills Assumptio C.. The time series model, we have i mid i this paper, is give by X t = f X t, θ 0 + e t,.3 where X t = X t,..., X t p, θ 0 is fixed but ukow, e t idepedet of F t = σ{x u, u t } the σ-algebra geerated by the observatios up to time t. Furthermore, {e t : t } are idepedet idetically distributed radom errors with a positive variace. Stockis et al. [8] use these time series as buildig blocks i a regime-switchig model, so called CHARME-models, i the cotext of fiacial time series. I their model the duratio time i each regime is radom ad drive by a hidde Markov chai, while i classical chage-poit aalysis the duratio time is usually fixed ad determiistic. Motivated by the CHARME time series Kirch ad Tadjuidje-Kamgaig [6] developed chage-poit tests i such a setup. Chage-poit aalysis deals with the questio whether the stochastic structure of the observatios has chaged at some ukow poit i the sample, which is a importat questio i diverse areas such as ecoomy, fiace, geology, physics or quality cotrol. For a detailed discussio we refer to the book by Csörgő ad Horváth [2]. I Sectio 2 we develop a uiform cetral limit theorem ivolvig the above autoregressive time series. Based o these uiform cetral limit theorems we ca elarge i Sectio 3 the class of alteratives for which the chage-poit tests developed i Kirch ad Tadjuidje-Kamgaig [6] have asymptotic power oe allowig i particular for local chages. Fially, i Sectio 4 the proofs ca be foud. 2

3 2 Uiform Cetral Limit Theorem 2 Uiform Cetral Limit Theorem I the followig {X t } is a arbitrary statioary time series ad ot ecessarily of the form.3. The uiform cetral limit theorem for empirical processes traces back to Dudley [3] ad is fouded o otios like VC-subgraphs or coverig umbers. We will make use of this methodology. Deote by F = {fθ, ; θ Θ} the set of all feedforward etworks as defied previously with parameter θ Θ. To obtai the uiform cetral limit theorem we eed the followig assumptios. C.. The set Θ is compact. Furthermore ψ is a sigmoid activatio fuctio as i.2 which is cotiuously differetiable with bouded derivative. The latter assumptio is e.g. fulfilled for the logistic fuctio. C. 2. X t is statioary ad β-mixig with mixig coefficiet β fulfillig for some τ > 2 as k k τ/τ 2 log k 2τ /τ 2 βk 0. Furthermore E X ν < for some ν > 2. This is a classical coditio i oliear time series aalysis ad ca be derived with little effort usig the stability theory for Markov processes, see e.g. Mey ad Tweedie [7]. Ideed, this property is a cosequece of the existece of a statioary solutio that is geometric ergodic. I this situatio the assumptio o the rate is therefore fulfilled sice the time series is β-mixig with a expoetial rate. I a more geeral setup ad for the related CHARME-models this β-mixig property has bee prove by Stockis et al. [8]. For the ext assumptio we first eed to recall the defiitio of coverig umbers. Defiitio 2.. Coverig Numbers The coverig umber N ɛ, F,. is the miimal umber of balls of radius ɛ, i.e. {g : g f < ɛ}, eeded to cover F, where it is ot ecessary that f F. C. 3. Assume that for some 0 < q < 0 log N u, F,. q /2 du <. This uiform etropy coditio is classical i the cotext of weak covergece of empirical processes cf. e.g. Va der Vaart ad Weller [9]. We are ow ready to prove the uiform cetral limit theorem. 3

4 3 Chage-poit tests uder local alteratives Theorem 2.. Assume C. C.3 hold. The, { } w fx t, θ EfX t, θ; θ Θ {Gθ; θ Θ} t= i l Θ, where {Gθ} has a versio with uiformly bouded ad uiformly cotiuous paths with respect to the. 2 -orm. I particular fx t, θ EfX t, θ = O. 2. sup θ Θ t= 3 Chage-poit tests uder local alteratives We observe a time series Z t = X t {t k } + Y t, {t>k } with a possible switch from X t to Y t, at some ukow time poit k = k. The time series X t is ot ecessarily of the form.3 but we assume that it is well approximated by X t = f X t, θ 0 + e t for some θ 0 i the iterior of Θ, Y t, is a arbitrary time series. If X t follows.3, the θ 0 = θ 0. For more details o the derivatio of θ 0 we refer to Kirch ad Tadjuidje-Kamgaig [6]. The ukow parameter k is called the chage-poit ad we are iterested i the testig problem H 0 : k = vs. H : k <. Our testig procedures are based o various fuctioals of the partial sums of estimated residuals k k Ŝ k = ê t = Z t fz t, θ, 3. t=p+ t=p+ where θ is the least-squares estimator of θ 0 uder H 0, i.e. the miimizer of t=p+ Z t fz t, θ 2. Kirch ad Tadjuidje-Kamgaig [6] prove cosistecy ad asymptotic ormality of θ uder the ull hypothesis uder some mild regularity coditios - i additio to some asymptotic results uder alteratives as well as misspecificatio. If the estimator is ot i the iterior of Θ we reject the ull hypothesis as asymptotically this ca oly happe uder the alterative or if the model.3 is ot suitable for the data set at had. Cosequetly, we ca assume w.l.o.g. θ Θ, the iterior of Θ, for asymptotic power cosideratios. Typical test statistics i this cotext are give by p T, = max p<k< k p k Ŝk, T 2, q = max p<k< p q k Ŝk, 3.2 p T 3, G = max p+g<k Ŝ k Ŝk G G 4

5 3 Chage-poit tests uder local alteratives where q is a positive weight fuctios defied o 0, fulfillig certai coditios, e.g. qt = t t γ, 0 γ < /2, ad G with a certai rate. I Kirch ad Tadjuidje-Kamgaig [6] the asymptotics of the above statistics uder the ull hypothesis ad certai regularity coditios are obtaied. Bt T 2, q coverges i distributio to sup 0<t<, where B is a Browia bridge. qt The other two statistics coverge a.s. to ifiity but such that α T β for some α, β coverges to a Gumbel distributio, which suffices to obtai asymptotic critical values. To obtai tests with asymptotic power oe i these cases, it suffices to show that α β T. For T, it holds α, β, log log /2 ad for T 3, G it holds α 3, β 3, log/g /2, where c d c d c for some costat c > 0. I Kirch ad Tadjuidje-Kamgaig [6] it was show that the above tests have asymptotic power oe for certai fixed alteratives, where Y t, = Y t, usig a uiform law of large umbers. Usig the uiform cetral limit theorem of the previous sectio istead allows us to geeralize this result to a larger class of alteratives, icludig local alteratives such as Y t, = X t + d Z t for some d 0 or X t as i.3 ad Y t, = fy t,, θ + e t with θ θ 0. Local alteratives are ofte cosidered i statistics to get a idea about the sesitivity of the test for small differeces. A.. The chage-poit fulfills k = λ for some 0 < λ <. A. 2. {X t } fulfills 2. as well as t= X t EX = O. A. 3. It holds b EfXp, θ θ= b θ EX 2 for b specified below. Uder local alteratives the estimator θ will typically coverge to θ 0 with a rate depedig o the rate of covergece of the local alterative, i.e. the rate with which d 0 resp. θ θ 0 i the examples above. For a geeral eural etwork it is difficult to quatify these rates due to the highly oliear structure of f. I the classical special case of a mea chage i.e. a trivial eural etwork with H = 0 ad X i = µ + e i, Y i, = µ + d + e i, Assumptio A. 3 reduces to the well kow coditio d 0 but b d. The type of mea chage coditio as i A.3 is typical if the test statistics are based o estimated residuals ad already arise i liear regressio models cf. e.g. Hušková ad Koubkova [5]. For some simulatios cocerig detectability of differet types of chages we refer to Kirch ad Tadjuidje-Kamgaig [6]. The ext theorem shows that the tests correspodig to the statistics i 3.2 have asymptotic power oe uder the above coditios. Theorem 3.. Assume that A. A.3 hold with, for a, log log b =, for b, G for c. log/g, 5

6 4 roofs The it holds a log log /2 T,, b T 2, q, c log/g /2 T 3,, if mi η t η qt > 0 for ay η > 0 ad G, log /G 0, G/ 0. 4 roofs We start with some auxiliary lemmas. Lemma 4.. Let fx, θ, be a eural etwork fulfillig C., x = x,..., x p T. The, for ay θ, θ 2 Θ there exists a costat D > 0 ot depedig o θ, θ 2 or x such that fx, θ fx, θ 2 D θ θ 2 2 where 2 is the Euclidia orm. max x i, i=,...,p roof. A applicatio of the mea value theorem with respect to θ yields for ay θ, θ 2 Θ fx, θ fx, θ 2 = fx, ξ T θ θ 2 for some ξ coθ, the covex hull of Θ. Sice Θ is compact, coθ is compact. By this ad the boudedess of the derivative of ψ we fid D > 0 such that fx, ξ T θ θ 2 θ θ 2 2 fx, ξ 2 D θ θ 2 2 max x i. i=,...,p Lemma 4.2. Let fx, θ, be a eural etwork fulfillig C.. The, there exists R > 0, such that for 0 < R it holds for ay probability measure Q R F L N, F,. L 2 Q 2 Q d where F x = D max i=,...,p x i for some costat D > 0, d = Hp ad fx L 2 Q = f 2 x dqx /2. roof. Deote by N [ ] ɛ, F,. the bracketig umber ad by Mɛ, F,. the packig umber, the it holds cf. Va der Vaart ad Weller [9] N, F,. N [ ] 2, F,., 4. N, F,. M, F,

7 4 roofs Lemma 4. ad Theorem 2.7. i Va der Vaart ad Weller [9] yield for ay > 0 N [ ] 2 F L 2 Q, F,. L 2 Q N, F,. 2. Hece by 4. w.l.o.g. F L 2 Q < N, F, L 2 Q N, F, 2 F L 2 Q for which 4.2 implies N, F,. 2 N, F L 2 Q F F,. 2 M, L 2 Q F F,. 2 L 2 Q 3R F d L 2 Q, for ay 0 < R, where F = {fθ, ; θ B0, R} for a suitable R > 0, B0, R is the ball aroud 0 of radius R i R d. The last lie follows from Exercise 6, page 94, of Va der Vaart ad Weller [9]. roof of Theorem 2.. The assertio follows from Lemma 2. of Arcoes ad Yu []. The statemet there is give for the miimal evelope fuctio but the proof shows that it remais true for ay evelope fuctio F. Furthermore it is sufficiet if their coditio 2.0 holds for all 0 < 0 for some 0 > 0. I our case we cosider F as i Lemma 4.. It remais to check the coditios of Lemma 2. of Arcoes ad Yu []. Let p = miτ, ν, the their Coditio 2.3 holds by E X t p < accordig to C.2. Coditio 2.4 holds by C.2, 2.0 by C.3 ad 2. for small by Lemma 4.2. roof of Theorem 3.. Ŝ k = k j=p+ By 3., A. ad A.2 it holds X j fx j, θ k = X i + k EfX p, θ θ= θ b + O sup i=p+ θ Θ = λ EX EfX p, θ θ= θ b + O By A.3 this implies log log /2 T, λ log log k i=p+ fx i, θ EfX p, θ EX EfX p, θ θ= b θ + O log log /2 Sice by A. ad mi η t η qt > c for ay η > 0 it holds qk c for some c > 0, which together with A.3 implies T 2, q λ EX EfX p, θ θ= b θ + O. Similarly Ŝk Ŝk G = G EX EfX p, θ θ= θ b + O G hece by A.3 log/g /2 T 3, G G log/g EX EfX p, θ θ= b θ +O log/g /2.. 7

8 Refereces Refereces [] Arcoes, M. A. ad Yu, B. Cetral limit theorems for empirical ad U-processes of statioary mixig sequeces. Ecoometrica, 7:48 7, 994. [2] Csörgő, M., ad Horváth, L. Limit Theorems i Chage-oit Aalysis. Wiley, Chichester, 997. [3] Dudley, R.M. Cetral limit theorem for empirical processes. A. robab., 6: , 978. [4] Frake, J. ad Mabouba, D. Estimatig market risk with eural etworks. Statist. Decisios, 30:63 82, [5] Hušková, M. ad Koubková, A. Moitorig jump chages i liear models. J. Statist. Res., 39:59 78, [6] Kirch, C. ad Tadjuidje K., J.. Testig for parameter stability i oliear autoregressive models. reprit, 20. [7] Mey, S.. ad Tweedie, R.L.. Markov Chais ad Stochastic Stability. Spriger, Lodo, 993. [8] Stockis, J.-., Frake, J., ad Tadjuidje K., J. O geometric ergodicity of CHARME models. J. Time Ser. Aal., 3:4 52, 200. [9] Va der Vaart, A. W. ad Weller, J. A. Weak covergece ad empirical processes. Spriger, New York, 996. [0] White, H. Coectioist oparametric regressio: Multilayer feedforward etworks ca lear arbitrary mappigs. Neural Networks, 3: ,

A uniform central limit theorem for neural network based autoregressive processes with applications to change-point analysis

A uniform central limit theorem for neural network based autoregressive processes with applications to change-point analysis Claudia Kirch Joseph Tadjuidje Kamgaing March 6, 20 Abstract We consider an