Consistent and equivariant estimation in errors-in-variables models with dependent errors

Transcription

1 Consistent and equivariant estimation in errors-in-variables models with dependent errors Michal Pešta Charles University in Prague Department of Probability and Mathematical Statistics ROBUST 2010 February 5

2 Salmo trutta morpha fario; L., 1758

3 Šumava Vltava Novohuťský p. Studený p. Otava Prášilský p. Jezerní p. Mlýnský p. K řemelná Úhlava Otava Plavební p. Studený p. Hrádecký p. Zhůřský p. Pop elný p. Tma vý p. Plaveb n í kanál Hamerský p. Teplá Vltava Volyňka Javoří p. Rok lans ký p. Vydr a Blanice R okytka Slatinn ý p. Fil ipohuťský p. Černoho rský p. P t ačí p. Teplá Vltava Březnický p. Březový p km

4 Vydra (Javoří potok)

5 Pitfalls, problems, and our approach length and weight

6 Pitfalls, problems, and our approach covariate or response

7 Pitfalls, problems, and our approach covariate or response least squares or least absolute distance or... ML

8 Pitfalls, problems, and our approach covariate or response invariant penalization

9 Pitfalls, problems, and our approach covariate or response invariant penalization water depth and changing conditions

10 Pitfalls, problems, and our approach covariate or response invariant penalization data dependence

11 Pitfalls, problems, and our approach covariate or response invariant penalization data dependence measurement units

12 Pitfalls, problems, and our approach covariate or response invariant penalization data dependence equivariant estimate (consistency, asymptotic normality)

13 Pitfalls, problems, and our approach covariate or response invariant penalization data dependence equivariant estimate (consistency, asymptotic normality) unknown quantities

14 Pitfalls, problems, and our approach covariate or response invariant penalization data dependence equivariant estimate (consistency, asymptotic normality) computational feasibility

15 Outline Errors-in-variables estimation EIV Model Equivariant estimate Inference Assumptions of the EIV model Asymptotic properties of the estimate Bootstrapping Moving block bootstrap Conclusions

16 Errors-in-Variables (EIV) Model Y = Z β + ε n 1 n p p 1 n 1 X = n p Z n p + Θ n p

17 Errors-in-Variables (EIV) Model Y = Z β + ε n 1 n p p 1 n 1 X = n p Z + Θ n p n p X and Y... observable random variables

18 Errors-in-Variables (EIV) Model Y = Z β + ε n 1 n p p 1 n 1 X = n p Z + Θ n p n p X and Y... observable random variables Z... unknown constants

19 Errors-in-Variables (EIV) Model Y = Z β + ε n 1 n p p 1 n 1 X = n p Z + Θ n p n p X and Y... observable random variables Z... unknown constants ε and Θ... random errors

20 Errors-in-Variables (EIV) Model Y = Z β + ε n 1 n p p 1 n 1 X = n p Z + Θ n p n p X and Y... observable random variables Z... unknown constants ε and Θ... random errors β... regression parameters (to be estimated)

21 Illustration y [x i θ i, y i ε i ] z i β y i [x i, y i ] z i x i x

22 Outline Errors-in-variables estimation EIV Model Equivariant estimate Inference Assumptions of the EIV model Asymptotic properties of the estimate Bootstrapping Moving block bootstrap Conclusions

23 Estimation inconsistency of the Least Squares estimation

24 Estimation inconsistency of the Least Squares estimation no distributional requirements

25 Estimation inconsistency of the Least Squares estimation no distributional requirements orthogonal direction (errors as small as possible)

26 Estimation inconsistency of the Least Squares estimation no distributional requirements orthogonal direction (errors as small as possible) equivariant estimate with respect to:

27 Estimation inconsistency of the Least Squares estimation no distributional requirements orthogonal direction (errors as small as possible) equivariant estimate with respect to: scale

28 Estimation inconsistency of the Least Squares estimation no distributional requirements orthogonal direction (errors as small as possible) equivariant estimate with respect to: scale, rotation

29 Estimation inconsistency of the Least Squares estimation no distributional requirements orthogonal direction (errors as small as possible) equivariant estimate with respect to: scale, rotation, and coordinate change

30 Construction of the estimate minimize the errors [Θ, ε]

31 Construction of the estimate minimize the errors [Θ, ε] unitary invariant matrix norm

32 Construction of the estimate minimize the errors [Θ, ε] unitary invariant matrix norm matrix norm: UAV = A, unitary U and V (UU = U U = I)

33 Construction of the estimate minimize the errors [Θ, ε] unitary invariant matrix norm matrix norm: UAV = A, unitary U and V (UU = U U = I) for a unitary invariant matrix norm min [Θ, ε] s.t. Y ε = (X Θ)β

34 Class of the UI matrix norms Schatten norms A q = i,j 1/q a q ij ( ) 1/q = σ q i, q 1 i

35 Class of the UI matrix norms Schatten norms A q = i,j 1/q a q ij ( ) 1/q = σ q i, q 1 i nuclear norm (q = 1), Frobenius norm (q = 2)

36 Class of the UI matrix norms Schatten norms A q = i,j 1/q a q ij ( ) 1/q = σ q i, q 1 i nuclear norm (q = 1), Frobenius norm (q = 2) Ky Fan k-norms ( k ) 1/q A (k) q = σ q i, q 1 i=1

37 Class of the UI matrix norms Schatten norms A q = i,j 1/q a q ij ( ) 1/q = σ q i, q 1 i nuclear norm (q = 1), Frobenius norm (q = 2) Ky Fan k-norms ( k ) 1/q A (k) q = σ q i, q 1 i=1 operator norm (k = 1), Schatten norms (k =last)

38 Estimate solution with desired properties for any UI MN ˆβ = (X X λi) 1 X Y

39 Estimate solution with desired properties for any UI MN ˆβ = (X X λi) 1 X Y λ is the (p + 1)-st largest eigenvalue of [X, Y] [X, Y]

40 Outline Errors-in-variables estimation EIV Model Equivariant estimate Inference Assumptions of the EIV model Asymptotic properties of the estimate Bootstrapping Moving block bootstrap Conclusions

41 Weak dependence strong mixing (α-mixing) α(a, B) = sup P(AB) P(A)P(B) A A,B B α(n) = sup α(f1 k, Fk+n) 0, n k N

42 Weak dependence strong mixing (α-mixing) α(a, B) = sup P(AB) P(A)P(B) A A,B B α(n) = sup α(f1 k, Fk+n) 0, n k N uniformly strong mixing (ϕ-mixing) ϕ(a, B) = sup P(B A) P(B) A A,B B ϕ(n) = sup ϕ(f1 k, Fk+n) 0, n k N

43 Weak dependence strong mixing (α-mixing) α(a, B) = sup P(AB) P(A)P(B) A A,B B α(n) = sup α(f1 k, Fk+n) 0, n k N uniformly strong mixing (ϕ-mixing) ϕ(a, B) = sup P(B A) P(B) A A,B B ϕ(n) = sup ϕ(f1 k, Fk+n) 0, n k N uniformly strong mixing strong mixing

44 Assumptions of the EIV model rows [Θ i,, ε i ] are α- or ϕ-mixing

45 Assumptions of the EIV model rows [Θ i,, ε i ] are α- or ϕ-mixing rows [Θ i,, ε i ] with zero mean and non-singular covariance matrix σ 2 I, where σ 2 is unknown (for simplicity)

46 Assumptions of the EIV model rows [Θ i,, ε i ] are α- or ϕ-mixing rows [Θ i,, ε i ] with zero mean and non-singular covariance matrix σ 2 I, where σ 2 is unknown (for simplicity) exists a positive definite matrix := lim n n 1 Z Z

47 Outline Errors-in-variables estimation EIV Model Equivariant estimate Inference Assumptions of the EIV model Asymptotic properties of the estimate Bootstrapping Moving block bootstrap Conclusions

48 Asymptotic properties consistency under uniformly strong mixing β P β, n

49 Asymptotic properties consistency under uniformly strong mixing β P β, n asymptotic normality under stationary strong mixing and finite (4 + δ)-th moment of errors n ( β β ) D N (0, ), n

50 Outline Errors-in-variables estimation EIV Model Equivariant estimate Inference Assumptions of the EIV model Asymptotic properties of the estimate Bootstrapping Moving block bootstrap Conclusions

51 Moving block bootstrap (MBB) asymptotic variance depends on unknown quantities (cannot be estimated)

52 Moving block bootstrap (MBB) asymptotic variance depends on unknown quantities (cannot be estimated) resample blocks of row-pairs [X, Y] with replacement

53 Moving block bootstrap (MBB) asymptotic variance depends on unknown quantities (cannot be estimated) resample blocks of row-pairs [X, Y] with replacement approaching (each other) in distribution almost surely along [X, Y] n( β β) D(a.s.) [X, Y] n( β β) n

54 Moving block bootstrap (MBB) asymptotic variance depends on unknown quantities (cannot be estimated) resample blocks of row-pairs [X, Y] with replacement approaching (each other) in distribution almost surely along [X, Y] n( β β) D(a.s.) [X, Y] n( β β) n ex.: H 0 : β = 2 vs H 1 : β 2

55 Conclusions EIV with weakly dependent errors

56 Conclusions EIV with weakly dependent errors equivariant estimate

57 Conclusions EIV with weakly dependent errors equivariant estimate consistency and asymptotic normality

58 Conclusions EIV with weakly dependent errors equivariant estimate consistency and asymptotic normality MBB correctness

59 Bibliography Merlevède, F. and M. Peligrad (2000) The functional central limit theorem under the strong mixing condition. Annals of Probability, 28(3): Xuejun, W., et al. (2009) Moment inequalities for ϕ-mixing sequences and its applications. Journal of Inequalities and Applications, Volume 2009, Article ID , 12 pages.