ROBUST - September 10-14, 2012

Transcription

1 Charles University in Prague ROBUST - September 10-14, 2012

2 Linear equations We observe couples (y 1, x 1 ), (y 2, x 2 ), (y 3, x 3 ),......, where y t R, x t R d t N. We suppose that members of couples are connected by linear equations y t = x t β 0 + e t t N, where disturbances e t R are unknown and β 0 Θ R d is unknown parameter common for all equations. Our aim is to reasonably approximate β 0 from couples (y t, x t ), t {1, 2,..., T }, where T N.

3 Linear equations Inaccuracy in any equation will be measured using a nonnegative Borel measurable function ρ : R R, ρ 0. As approximation of β 0, we take a parameter β Θ for which the corresponding disturbances are small.

4 Linear equations More precisely, we possess given positive numbers ε T > 0, T N with lim sup ε T = ε, 0 ε < + T + and we find ˆβ T Θ such that 1 T T ρ(y t xt t=1 ˆβ T ) < T + ε T, where { } 1 T T = inf ρ(y t xt β) : β Θ. T t=1

5 Functional setting The problem can be generalized using a function defined for all β Θ and for all µ probability Borel measures on R d+1 by the formula f (β ; µ) = ρ ( x (β 0 β) + e ) µ( dx, de). The integral is always well-defined since ρ is a nonnegative Borel measurable function and µ is a probability Borel measures.

6 Generalized problem Let us consider a sequence µ T, T N of probability Borel measures on R d+1. We find a parameter ˆβ T Θ such that ρ (e ( + x β 0 ˆβ )) T µ T ( dx, de). < T + ε T, where { T = inf ρ ( x (β 0 β) + e ) } µ T ( dx, de) : β Θ.

7 Generalized problem Setting µ T = T t=1 1 T δ (y t,e t) we receive precisely the original problem f (β ; µ T ) = 1 T T ρ ( e t + xt t=1 (β 0 β) ) = 1 T T ρ(y t xt β). t=1

8 LEMMA: For any ν probability Borel measures on R d+1 and any > 0 we have lim κ + inf ν ({ (x, e) : κ x γ }) + e = γ =1 = inf { ν ({ (x, e) R d+1 : x γ 0 }) : γ = 1, γ R d}.

9 PROOF: Let us denote M = inf { ν ({ (x, e) R d+1 : x γ 0 }) : γ = 1, γ R d}. 1) Let γ R d and k N, then { (x, e) : k x γ + e } { (x, e) : x γ 0 }. Therefore, lim κ + inf ν ({ (x, e) : κ x γ }) + e M. γ =1

10 Proof 1 2) Consider a sequence γ k R d, γ k = 1 for all k N such that ν ({ (x, e) : k x γ k + e }) < < inf γ =1 ν ({ (x, e) : k x γ + e }) + 1 k. The sequence belongs to a compact, therefore, it contains a convergent subsequence lim γ k j = ˆγ, ˆγ = 1. j +

11 Proof 2 Hence, { (x, e) : x ˆγ 0 } = + + J=1 j=j + + J=1 j=j { (x, e) : x ˆγ x (ˆγ γ kj ) + 1 } ( + e ) k j { (x, e) : kj x γ kj + e }.

12 Proof 3 Because of σ-additivity of the measure ν, we have M ν ({ (x, e) : x ˆγ 0 }) + { lim (x, e) : kj x } γ kj + e Q.E.D. J + ν j=j lim inf ν ({ (x, e) : k J x γ kj + e }) J + [ lim inf J + inf γ =1 ν ({ (x, e) : κ x γ + e }) + 1 k J = inf γ =1 ν ({ (x, e) : κ x γ + e }). ]

13 Assumptions Assumption 1: Θ R d is a closed set and for each β Θ ρ(e) ν ( dx, de) ρ(e + x (β 0 β)) ν ( dx, de). Assumption 2: There is a function ψ : R + R + which is continuous, nondecreasing and fulfilling: 1. For all t R ρ(t) ψ( t ). 2. For all t > 0 ψ( e + t x )ν ( dx, de) < For all t > 0 ψ( e + t x )µt ( dx, de) < +.

14 Assumptions Assumption 3: ψ D µ T T + ν which means f : R d+1 R bounded continuous f (x, e)µ T ( dx, de) f (x, e)ν ( dx, de), T + t > 0 ψ( e + t x )µ T ( dx, de) ψ( e + t x )ν ( dx, de). T +

15 Assumptions Assumption 4: Denoting H ρ = lim inf inf {ρ(t) : t >, t R}, + M = inf { ν ({ (x, e) R d+1 : x γ 0 }) : γ = 1, γ R d}, we require M > 0 and a balance H ρ M > ρ(e) ν ( dx, de) + lim sup ε T. T +

16 Notation We denote g(β) = ρ(e + x (β 0 β)) ν ( dx, de) β Θ. According to Assumption 1, we know that β 0 is a minimizer of g. The set of all minimizers and the set of all ε-minimizers of g are denoted by Φ g = {β Θ : g(β) = g(β 0 )}, Ψ g; ε = {β Θ : g(β) g(β 0 ) + ε}. We recall that we possess given positive numbers ε T > 0, T N with lim sup ε T = ε, 0 ε < +. T +

17 THEOREM: Let the previous Assumptions 1-4 be fulfilled. Then β 0 Φ g and ˆβ T exists for every T N. The sequence ˆβ T, T N is compact and { } Ls ˆβT, T N Ψ g; ε, ( lim d ˆβT, Ψ g; ε T + ) = 0.

18 PROOF: 1) We know that for each β Θ, T N f (β ; µ T ), f (β ; ν) R.

19 Proof 1 2) Let β, β T Θ such that β T T + β. Let us fix ε > 0. Then there exist, Q and a compact K R d+1 such that β T β 0 for all T N, (ψ( e + x ) Q) ν ( dx, de) < ε, ψ( e + x )>Q µ T (R d+1 \ K) < ε Q for all T N.

20 Proof 2 Now, we are able to derive a convergence. f (β T ; µ T ) = ρ(e + x (β 0 β T ))µ T ( dx, de) = = min{q, ρ ( x (β 0 β) + e ) }µ T ( dx, de) + + min{q, ρ(e + x (β 0 β T ))} min{q, ρ ( x (β 0 β) + e ) }µ T ( dx, de) + ( ρ(e + x (β 0 β T )) Q ) µ T ( dx, de). ρ(e+x (β 0 β T ))>Q

21 1. term The function min{q, ρ} is bounded and continuous. Therefore, min{q, ρ ( x (β 0 β) + e ) }µ T ( dx, de) T + min{q, ρ ( x (β 0 β) + e ) }ν ( dx, de).

22 2. term The second term fulfills min{q, ρ(e + x (β 0 β T ))} min{q, ρ ( x (β 0 β) + e ) }µ T ( dx, de) < < 2ε + min{q, ρ(e + x (β 0 β T ))} min{q, ρ ( x (β 0 β) + e ) }µ T ( d K 2ε + sup min{q, ρ(e + x (β 0 β T ))} min{q, ρ ( x (β 0 β) + e ) } (x,e) K 2ε T + because ρ is continuous, and, hence, uniformly continuous on each compact set.

23 3. term The third term is smaller than ε since ( 0 ρ(e + x (β 0 β T )) Q ) µ T ( dx, de) ρ(e+x (β 0 β T ))>Q ψ( e +T x )>Q (ψ( e + T x ) Q) µ T ( dx, de) = ψ( e + T x )µ T ( dx, de) min{q, ψ( e + T x )}µ T ( dx, de) T + ψ( e +T x )>Q (ψ( e + T x ) Q) ν ( dx, de) < ε.

24 Proof - convergence Thus, we proved lim T + f (β T ; µ T ) = ρ ( x (β 0 β) + e ) ν ( dx, de) = f (β ; ν). It means that f (β ; µ T ) converge to f (β ; ν) uniformally on each compact.

25 Proof - tightness According to Assumptions, there is a number > 0 such that inf ρ(t) M > f (β 0 ; ν) + ε. t > Hence according to Lemma, we are able to find Γ such that inf ρ(t) inf ν ({ (x, e) : Γ x γ + e }) > f (β 0 ; ν) + ε. t > γ =1 Then, we define a compact K = {β Θ : β β 0 Γ}.

26 Proof - tightness For β Θ, β β 0 > Γ we receive following chain of inequalities: f (β ; µ T ) = ρ ( x (β 0 β) + e ) µ T ( dx, de) ρ ( x (β 0 β) + e ) µ T ( dx, de) x (β 0 β)+e > inf t > ρ(t) µ T inf t > ρ(t) µ T inf t > ρ(t) µ T ({ (x, e) : x (β 0 β) + e > }) ({ (x, e) : x (β β 0 ) > + e }) ({ (x, e) : Γ x β β }) 0 β β 0 > + e.

27 Proof - tightness For any ε > 0, properly chosen sequence of γ T, γ T = 1 and its cluster point ˆγ, we have lim inf inf f (β ; µ T ) T + β / K inf ρ(t) lim inf inf { ({ µ T (x, e) : Γ x γ }) } > + e : γ = 1 t > T + inf ρ(t) lim inf µ T t > T + inf ρ(t) lim inf µ T t > T + inf ρ(t) lim inf µ T t > T + ({ (x, e) : Γ x γ T > + e }) ({ (x, e) : Γ x ˆγ > + e + Γ x (γ T ˆγ) }) ({ (x, e) : Γ x ˆγ > (1 + ε) + e }) inf t > ρ(t) ν ({ (x, e) : Γ x ˆγ > (1 + ε) + e }).

28 Proof - tightness Letting ε vanish we have lim inf inf f (β ; µ T ) inf ρ(t) ν ({ (x, e) : Γ x ˆγ + e }) T + β / K t > > f (β 0 ; ν) + ε.

29 Proof - finish We found that it is sufficient to consider β K, only. We know that f (β ; µ T ) converge to f (β ; ν) uniformally on K. That completes the proof. Q.E.D.

30 Chen, X.R.; Wu, Y.H.: Strong consistency of M-estimates in linear model. J. Multivariate Analysis 27,1(1988), Dodge, Y.; Jurečková, J.: Adaptive Regression. Springer-Verlag, New York, Hoffmann-Jørgensen, J.: Probability with a View Towards to Statistics I, II. Chapman and Hall, New York, Huber, P.J.: Robust Statistics. John Wiley & Sons, New York, Jurečková, J.: Asymptotic representation of M-estimators of location. Math. Operat. Stat. Sec. Stat. 11,1(1980), Jurečková, J.: Representation of M-estimators with the second-order asymptotic distribution. Statistics & Decision 3(1985), Jurečková, J.; Sen, P.K.: Robust Statistical Procedures. John Wiley & Sons, Inc., New York, 1996.

31 Kelley, J.L.: General Topology. D. van Nostrand Comp., New York, Knight, K.: Limiting distributions for L 1 -regression estimators under general conditions. Ann. Statist. 26,2(1998), Lachout, P.: Stability of stochastic optimization problem - nonmeasurable case. Kybernetika 44,2(2008), Lachout, P.: Stochastic optimization sensitivity without measurability. In: Proceedings of the 15 th MMEI held in Herĺany, Slovakia (Eds.: K. Cechlárová, M. Halická, V. Borbeĺová, V. Lacko) (2007), Lachout, P.; Liebscher, E.; Vogel, S.: Strong convergence of estimators as ε n -minimizers of optimization problems. Ann. Inst. Statist. Math. 57,2(2005), Lachout, P.; Vogel, S.: On Continuous Convergence and Epi-convergence of Random Functions. Part I: Theory and Relations. Kybernetika 39,1(2003),

32 Leroy, A.M.; Rousseeuw, P.J.: Robust Regression and Outlier Detection. John Wiley & Sons, New York, Robinson, S.M.: Analysis of sample-path optimization. Math. Oper. Res. 21,3(1996), Rockafellar, T.; Wets, R.J.-B.: Variational Analysis. Springer-Verlag, Berlin, Vajda, I.; Janžura, M.: On asymptotically optimal estimates for general observations. Stochastic Processes and their Applications 72,1(1997), van der Vaart, A.W.; Wellner, J.A.: Weak Convergence and Empirical Processes Springer, New York, 1996.