Multivariate Least Weighted Squares (MLWS)

Transcription

1 () Stochastic Modelling in Economics and Finance 2 Supervisor : Prof. RNDr. Jan Ámos Víšek, CSc. Petr Jonáš 23 rd February 2015

2 Contents

3 1 1 Introduction 2 Algorithm 3 Proof of consistency 4 Appendix 5 n consistency (0%)

4 2 Consider the multivariate model Y t = (B 0 ) X t + U t, (1) - Y t R q, X t R p t = 1,..., n, - U t (0, Σ), t = 1,..., n are i.i.d random variables, - Σ is symmetric and positive definite matrix q q. - B 0 R p q contains the coefficients. - without loss of generality assume B 0 = 0,

5 3 Using notation - X = (X 1,..., X n ), Y = (Y 1,..., Y n ), U = (U 1,..., U n ), - y = vec(y ), β 0 = vec(b 0 ), u = vec(u), we can rewrite (1) as follows or u (0, Σ I ). Y = XB 0 + U, (2) y = (I X )β 0 + u. (3)

6 4 Applying Aitken s estimator on (3) we obtain ˆβ = (I (X X ) 1 X )y or equivalently ˆB LS = (X X ) 1 X Y (4) and Σ can be estimated as ˆΣ LS = 1 n (Y X ˆB LS ) (Y X ˆB LS ) n (Y i ˆB LS X i)(y i ˆB LS X i). = 1 n i=1 (5) Replacing n in the denominator by n p, we obtain an unbiased estimate of Σ. For details see Judge et al. (1988) or Lütkepohl (2005).

7 5 The goal is to generalize least weighted squares (LWS) estimator to multivariate case. n ( ) ˆβ n,w i 1 LWS := arg min w r(i) 2 (β), (6) β R n i=1 The Mahalanobis distance is defined as d t (B, Σ) = ((Y t B X t ) Σ 1 (Y t B X t )) 1/2. We can define the multivariate least squares (MLS) estimator by ˆB MLS = {B (B, Σ ) arg min (B,Σ); Σ =1 n i=1 d 2 i (B, Σ)}.

8 6 Assume that not all n points are lying in the same subspace of R p+q. Formally it means that for all α R p and β R q Lemma 1 #{(X i, Y i ) α X i + β Y i = 0} < n. (7) ˆB LS = ˆB MLS, for all data sets satisfying condition (7). Remark If (X i, Y i ) comes as a sample from continuous distribution, condition (7) is satisfied with probability 1.

9 7 Definition 2 Let us define the multivariate least weighted squares () estimator of the matrix of coefficients B as ˆB n,w = arg min (B,Σ) D n ( ) s 1 w d(s) 2 (B, Σ), (8) n s=1 where d (1) (B, Σ) d (2) (B, Σ) d (n) (B, Σ) is the ordered sequence of the residual Mahalanobis distances.

10 We have to enlarge our notation ˆB n,w = arg min (B,Σ) D Let us denote n ( ) π(b, Σ, i) 1 w di 2 (B, Σ), n i=1 F B,Σ (r) = P ( ) (d 1 (B, Σ)) 1/2 < r (9) F (n) B,Σ (r) = 1 n n I (d i (B, Σ) < r) (10) i=1 and realize, that d i (B, Σ) and di 2 (B, Σ) are on the same position in the ordered sequences. 8

11 9 Therefore we can write ˆB n,w = arg min (B,Σ) D n w i=1 ( ) F (n) B,Σ (d i(b, Σ)) di 2 (B, Σ), (11) After some computation we can express the system of normal. NE Y,X,n (B) = n w i=1 ( ) F (n) B,Σ (d i(b, Σ)) X i (Y i B X i ) = 0 (12) In the one dimensional case, the main idea of proving consistency of LWS is to approximate empirical distribution funciton of absolute value of residuals by a continuous distribution function. We try to prove consistency in the simillar way. We shall need following assumptions

12 10 Condition C1 The weight function w : [0, 1] [0, 1] with w(0) = 1 is absolutely continuous and nonincreasing, with derivative w (α) bounded from below by L, (L > 0).

13 11 Condition C2 The sequence {(X i, U i ) } i=1 is a sequence of i.i.d. (p + q) dimensional random vectors distributed accordig to distribution function F X,U (x, u) = F X (x)f U (u). F U (u) is absolutely continuous with density f U (u) bounded by U e. Finally, there is a q > 1 so that E X 1 4q <. Symbol X = (tr(xx )) 1 2 denotes the Frobenius norm.

14 12 Now we want to prove, that all of normal are bounded. Denote ( ) B F B X (r) = P X i X < r (13) and the corresponding empirical distribution function F (n) B X (r) = 1 n n I i=1 i B ( B X i X i B ) < r (14) To prove the boundedness of, the following lemmas will be useful.

15 13 Lemma A1 Under Condition C2 the distribution function F B X (r) is uniformly in r R, uniformly continuous in B, i.e for any δ > 0, there is ξ (0, 1) so that for any B (1) and B (2) such that B (1) B (2) < ξ sup F B (1),Σ (r) F B (2),Σ (r) δ r R sup F (B r R (1) ) X (r) F (B (2) ) X (r) δ

16 14 Lemma A2 Let the Condition C2 hold. For any ɛ > 0 there is a constant K ɛ and n ɛ N so that for all n > n ɛ P([ω Ω : sup u R + (n) sup n F B,Σ (u) F B,Σ(u) < K ɛ ]) > 1 ɛ B R p q

17 15 Lemma A3 Let the Condition C2 hold. For any ɛ > 0 there is a constant K ɛ and n ɛ N so that for all n > n ɛ P([ω Ω : sup u R + (n) sup n F (u) F B R p q B X B X (u) < K ɛ]) > 1 ɛ

18 16 Lemma 2 Let Conditions C1 and C2 be fulfilled. Then for any ɛ > 0 there is θ > 0, > 0 and n ɛ, N such that for any n > n ɛ, ( ) P ω Ω : inf 1 B θ n B NE Y,X,n (B) > > 1 ɛ

19 17 Let Conditions C1 and C2 be fulfilled. Then for any ɛ > 0, δ (0, 1) there is ρ > 0 and n ɛ,δ,ρ N such that for any n > n ɛ,δ,ρ we have P(ω Ω : sup B ρ 1 n n w i=1 ( ) F (n) B,Σ (d i(b, Σ)) B X i (Y i B X i ) ( B E w (F B,Σ (d 1 (B, Σ))) X 1 (Y 1 B X 1 ) ) < ) > 1 ɛ

20 18 Condition C3 There is the only solution of [ ] E w (F B,Σ (d 1 (B, Σ))) X 1 (Y 1 B X 1 ) = 0, (15) namely B 0. (we have assumed B 0 = 0)

21 19 Lemma 4 Let Conditions C1, C2 and C3 be fulfilled. Then any sequence { ˆB n,w }+ n=1 of the of normal NE Y,X,n ( ˆB n,w ) = 0, n = 1, 2,... is weakly consistent.

22 20 I Judge, G., R. C. Hill, W. Griffiths, H. Lütkepohl, and T.-C. Lee (1988). Introduction to the Theory and Practise of Econometrics (2 ed.). John Wiley & Sons. Lütkepohl, H. (2005). New Introduction to Multiple Time Series Analysis. Springer-Verlag.

23 21 Thank You For Your Attention!