Optimally Bounding the GES(Hampel s Problem)

Transcription

1 Optimally Bounding the GES(Hampel s Problem) Let ˆµ n bea(smooth)locationm-estimatebasedonthemonotonelocation score function ψ and the consistent dispersion estimate ˆσ n. We have shown before that the asymptotic variance of ˆµ n at the central normal model H 0 N ( µ,σ 2) isgivenby AV(ψ,H 0 ) σ 2 E H0 ψ 2 ( y µ σ [ )} EH0 ψ( y µ σ )( )}] y µ 2. () σ Wehavealsoshownbeforethatthegrosserrorsensitivityof ˆµ n atthecentral normalmodeln ( µ,σ 2) isgivenby ψ( ) γ(ψ,h 0 ) σ E H0 ψ( y µ σ )( y µ )}. (2) σ In his Ph.D. dissertation(completed in 967), Hampel posed and solved the following problem. Hampel s Optimality Problem: Consider the class C of all location M- estimates with non-decreasing and bounded score functions ψ. We wish to find the most efficient robust estimate within this class. Suppose that we will measureefficiencybytheasymptoticvariance(atthetargetmodelh 0 )androbustness by the gross error sensitivity. Therefore, we wish to choose the location M-estimatewithinC withthesmallestasymptoticvarianceav(ψ,h 0 )among all estimatesinc withrelatively small gross-errorsensitivityγ(ψ,h 0 ).Hampel soptimalityproblemcanbemathematicallystatedasfollows: findψ C such that and ψ arg min γ(ψ,h 0) K AV(ψ,H 0) (3) Thechangeofvariablesz(y µ)/σin()and(2)gives AV(ψ,H 0 ) σ 2 E H0 ψ 2 ( y µ σ [ )} EH0 ψ( y µ σ )( )}] y µ 2 σ 2 E Φ ψ 2 (z) } [E σ Φ ψ(z)(z)}] 2 (4) ψ( ) ψ( ) γ(ψ,h 0 ) σ E H0 ψ( y µ σ )( )} y µ σ E σ Φ ψ(z)z}. (5)

2 Therefore,exceptforthefixedscalingfactors(σ 2 andσ)theasymptoticvariance and the GES of M-estimates essentially depend on the shape of the distribution H 0 andnotontheparticularvaluesofitslocationanddispersionparameters. Therefore, without loss of generality we can assume that the fixed location and dispersionparametersareµ0andσ(andsoh 0 Φ).Moreover,since (4)and(5)areinvariantundermultiplicationofψbyanon-zeroconstant,we can assume without loss of generality that E Φ ψ(y)y}. (6) Therefore, problem(3) can be re-stated as follows. Minimize (inψ) E Φ ψ 2 (z) } Subject to and (i) E Φ ψ(y)y}. (ii) ψ( ) K For future reference let C K ψ C : (i)and(ii)abovehold}. (7) We have the following result: Theorem LetH 0 N ( µ,σ 2) andconsiderthefamilyofhuber sscorefunctions: y/[2φ(c) ] y c ψ c (y) (8) csign(y)/[2φ(c) ] y >c. Then (a) IfK</[2ϕ(0)],thenC K φ(theemptyset). Inotherwords, γ(ψ,φ) π, forallψ. (9) 2 (b) The median minimizes the gross error sensitivity among all location M- estimates. 2

3 (c) If K > /[2ϕ(0)], then there exists a unique constant c > 0 such that ψ c C K,thatisγ(ψ c,φ)k. Proof: To prove(a) write E Φ ψ(y)y} 2 0 2ψ( ) ψ(y)yϕ(y)dy ψ(y)yϕ(y)dy 0 yϕ(y)dy 2ψ( ) 2ψ( )[ϕ( ) ϕ(0)]2ψ( )ϕ(0) 0 ϕ (y)dy Therefore, 2ψ( )/ 2π. γ(ψ,φ) ψ( ) 2π π E Φ ψ(y)y} 2 2. Part (b)follows directly from(a) and the fact thatin the case of the median ψ(y) sign(y) γ(sign,φ) sign( ) E Φ sign(y)yϕ(y)} sign(y)yϕ(y)dy [ [ 2 [ 2 ] y ϕ(y)dy 0 0 ] yϕ(y)dy ] ϕ (y)dy [ 2ϕ( )+2ϕ(0)] /[2ϕ(0)] π 2. 3

4 Toprove(c)noticethat and therefore γ(ψ c,φ) The function g(c) is continuous, with E Φ ψ c (y)y} c [2Φ(c) ] g(c) and lim g(c) c limg(c) lim c 0 c 0 c [2Φ(c) ] lim c 0 2ϕ(c) π/2. Thisprovestheexistence partof(c). Theuniqueness partof(c)followsfrom the fact g (c) 2Φ(c) 2cϕ(c) [2Φ(c) ] 2 0, forall c 0 (seeproblem3). Theorem2 Given K > /[2ϕ(0)], there exists a unique c > 0 such that γ(ψ c,φ)k (see(8))and AV(ψ,Φ) AV(ψ c,φ), forallψ C K. Proof: Let K > /[2ϕ(0)]. By the previous theorem we know there is a unique c c(k) such γ(ψ c,φ) K. Set β β(c) [2Φ(c) ]. 4

5 Now write J(ψ) E Φ ψ 2 (y) } and I(ψ) [ψ(y) βy] 2 ϕ(y)dy. Forallψ C K, I(ψ) [ψ 2 (y)+β 2 y 2 2βyψ(y)]ϕ(y)dyJ(ψ)+β 2 2β. (0) Therefore, ψ minimizes J(ψ) over C K if and only if ψ minimizes I(ψ)over C K. By symmetry, I(ψ) 2 0 [ψ(y) βy] 2 ϕ(y)dy andsoitsufficestostudythesquareddifference[ψ(y) βy] 2 fory 0. Clearly,sinceψ C K,fory cwehave [ψ(y) βy] 2 0[ψ c (y) βy] 2 for y c. () Ontheotherhand,sinceψ C K,fory>cwehave 0 ψ(y) Kψ c (y), for y>c andso [ψ(y) βy] 2 [K βy] 2 [ψ c (y) βy] 2, for y>c. (2) 5

6 From()and(2) wehave(seefigure) I(ψ) [ψ(y) βy] 2 ϕ(y)dy [ψ c (y) βy] 2 ϕ(y)dyi(ψ c ) (3) Finallythetheoremfollows(i.e. ψ (y)ψ c (y))from(0)and(3). ByTheorem2,foreach K> π/2 there is a unique c > 0 such that ψ c (y) (see Equation (8)) minimizes the asymptotic variance among all ψ functions with γ(ψ c,φ) K. Thespecificvalueofc, c c(k), can be determined from the equation c [2Φ(c) ] K. (4) 6

7 Minimax Variance(Huber s Problem) Consider the symmetrical contamination neighborhood F ɛ H: H(y)( ɛ)φ+ɛg, Gissymmetric}. (5) ThetechnicaladvantageofrestrictingattentiontoF ɛ isthatforanym-estimate ˆµwehave ˆµ(H) 0, forall H F ɛ. Therefore,wedonotneedtoworryabouttheproblemofasymptoticbias when werestrictedtof ɛ. Ontheotherhand,theseriouspracticallimitationsimposed by this restriction are obvious. The original problem considered by Huber was solved assuming that the dispersion parameter σ is known. The solution of Huber s problem when σ is unknown (andmustbe robustly estimated)was obtainedbyliandzamar (99). Huber s goal was to find the location M-estimate score function ψ that minimizes the maximum asymptotic variance over the symmetric contamination family (5). Huber s mathematical approach was very elegant and goes along the following lines. Let AV(ψ) sup AV(ψ,H), F Fɛ bethemaximumasymptoticvarianceoverf ɛ forthelocationm-estimatewith score function ψ. Robust M-estimates are then expected to have a relatively small and stable AV over the contamination neighborhood. Therefore, an estimatewithsmallerav(ψ)shouldbepreferred,fromarobustnesspointofview (other things being equal). Then, we wish to consider the minimization problem inf ψ AV(ψ) inf sup AV(ψ,H). (6) ψ F Fɛ 7

8 Supposethatonecanprove(asHuberdid)thatψ solves(6)ifandonlyifψ solves the dual problem sup inf F Fɛ ψ AV(ψ,H). (7) Then,wemaywishtosolvethe dual optimalityproblem(7)insteadofthe (harder) optimality problem(6). Let ψ H (y) h (y) h(y) bethemaximumlikelihoodscorefunctionunderh andleti(h)bethecorresponding Fisher Information index I(H) E 2 Hψ H (y)} ( ) 2 h (y) h(y)dy. h(y) Itisnotdifficulttoverifythat sup inf F Fɛ ψ AV(ψ,H) sup AV(ψ H,H) F Fɛ sup F F ɛ I(H) inf F F ɛ I(H) Therefore, the original min-max problem reduces to that of finding the least favorable distribution, that is, the distribution with smallest Fisher information in the family (5). Huber did precisely that and found its famous truncatedlinearscorefunctionwhenh 0 isnormal. An Alternative Derivation of Huber s Optimality Result We will see that in the case of contamination neighborhoods Huber s optimality result can be directly obtained using Hampel s result, and vice-versa. 8

9 That is, the two problems are mathematically equivalent. This is not too surprisingifwenoticethatthetwoproblemsyieldthesamefamilyofoptimalscore functions. The elegant approach of solving the min-max variance problem by first findingtheleastfavorabledistributionisnoteasytoapplytomodelsotherthanthe simple pure location model treated by Huber. On the other hand, the approach of attacking the min-max problem directly(brute force approach, so to speak) shows better promise of possible extensions as it has already been successfully applied by Li and Zamar(99) to the location-dispersion model. It is easy to show (using a simplified version of proof of Theorem?? in Section??) thatinthecaseofknowndispersion -takenequaltoone,without loss of generality- the asymptotic distribution of location M-estimates is normal withmean0andvariance AV (ψ,h) E H ψ 2 (x) } [ EH ψ (x) }] 2. (8) We have the following result. Theorem 3 Suppose that the dispersion parameter σ is known(and then taken equal to one without loss of generality). If ψ is non-decreasing then sup AV (ψ,h) H F ɛ ( ɛ) AV (ψ,φ)+ ɛ ( ɛ) 2γ2 (ψ,φ). (9) where γ(ψ, Φ) is the gross-error sensitivity at the Normal model. Therefore the maximum asymptotic variance under the normal ɛ-contamination family F ɛ is alinear combination of thenormal asymptoticvarianceand gross-errorsensitivity,withcoefficients/( ɛ)andɛ/( ɛ) 2,respectively. Proof: Let H ( ɛ)n(0,)+ɛh (20) From(8)and(20)wehave AV (ψ,h) ( ɛ)e Φ ψ 2 (x) } +ɛe H ψ 2 (x) } [ ( ɛ)eφ ψ (x) } +ɛe H ψ (x) ] 2. 9

10 Since ψ is non-decreasing, it is clear that ( ɛ)e Φ ψ 2 (x) } +ɛe H ψ 2 (x) } ( ɛ)e Φ ψ 2 (x) } +ɛψ 2 ( ). (2) Moreover,sinceψ (x) 0forallx,wecanwrite ( ɛ)e Φ ψ (x) } +ɛe H ψ (x) } ( ɛ)e Φ ψ (x) } (22) Using(2)and(22)weobtain AV (ψ,h) ( ɛ)e Φ ψ 2 (x) } +ɛe H ψ 2 (x) } [ ( ɛ)eφ ψ (x) } +ɛe H ψ (x) ] 2 ( ɛ)e Φ ψ 2 (x) } +ɛψ 2 ( ) [ ( ɛ)eφ ψ (x) }] 2 ( ɛ) E Φ ψ 2 (x) } [ EΦ ψ (x) }] 2 + ɛ ( ɛ) 2 ψ 2 ( ) [ EΦ ψ (x) }] 2 ( ɛ) AV (ψ,φ)+ ɛ ( ɛ) 2γ2 (ψ,φ), forallh F ɛ. Therefore, sup AV (ψ,h) H F ɛ ( ɛ) AV (ψ,φ)+ ɛ ( ɛ) 2γ2 (ψ,φ). (23) where γ(ψ, Φ) is the gross-error-sensitivity of the M-estimate at the Normal model. On the other hand, taking H y ( ɛ)n(0,)+ɛuniform (y δ,y+δ; y δ, y+δ) 0

11 wehavethat sup AV (ψ,h) supav (ψ,h y ) lim AV (ψ,h y) H F ɛ y y Therefore,(9) follows from(23) and(24). ( ɛ) AV (ψ,φ)+ ɛ ( ɛ) 2γ2 (ψ,φ). (24) Theorem 3 shows that the maximum asymptotic variance minimized by Huber is the Lagrangian representation of Hampel s optimality problem. Hampel sets an explicit bound on the gross-error-sensitivity while Huber (in a rather fortuitous way) had the gross-error-sensitivity included as a penalty term in the maximum asymptotic variance he minimized. The following result formally shows that Hampel s and Huber s problems are equivalent from a mathematical point of view. However, it is very important to understand that the mathematical equivalence of the two problems doesn t imply their statistical equivalence. More precisely, Hampel s problem has a deep statistical meaning while Huber s problem is just a nice mathematical elaboration. Theorem 4 (Equivalence between Huber s and Hampel s optimal location results) (a)supposethatψ solveshuber soptimalityproblemforsome0<ɛ</2. Then ψ solves Hampel s optimality problemwith gross-error-sensitivity bound equaltokγ(ψ,φ). (b) For each 0 < ɛ < /2 the solution ψ to Huber s min-max variance problemonf ɛ belongstothesetof scorefunctionsψ c givenby(8).thatis, minmaxav(ψ,h)min max AV (ψ c,h) ψ H F ɛ c>0 H F ɛ where the min on the left side is taken over the family of all monotone score functions ψ. Proof: To prove Part(a) notice that, by hypothesis, max H F ɛ AV (ψ,h) max H F ɛ AV(ψ,H), forallψ

12 Let α /( ɛ) β ɛ/( ɛ) 2. (25) ByTheorem3 wehave αav (ψ,φ)+βγ 2 (ψ,φ) αav (ψ,φ)+βγ 2 (ψ,φ), forallψ. Hence, αav (ψ,φ)+βk(c) 2 αav (ψ,φ)+βγ 2 (ψ,φ), forallψ or equivalently Therefore [ β AV (ψ,φ) AV (ψ,φ)+ α γ2 (ψ,φ) β ] α K(c)2, forallψ. AV (ψ,φ) AV (ψ,φ), forallψwithγ(ψ,φ) K and Part(a) follows. Toprovepart(b),let0<ɛ</2befixedandset(asin(25))α/( ɛ) andβɛ/( ɛ) 2. Denote byψ c the solutionto Hampel sproblemwithboundk(c)onthe gross-error-sensitivity. Then AV(ψ c,φ) AV(ψ,Φ) forallγ(ψ,φ) K(c). This implies the weaker statement AV(ψ c,φ) AV(ψ,Φ) forallγ(ψ,φ)k(c). (26) 2

13 Using(26)wecanwrite, αav (ψ c,φ)+βk αav (ψ,φ)+βk(c) 2, forallγ(ψ,φ)k Hence, by Theorem 3 maxav(ψ c,φ) maxav (ψ,h) forallγ(ψ,φ)k(c) (27) H F ɛ H F ɛ Let g(c) αav (ψ c,φ)+βk(c) ɛ AV(ψ K,Φ)+ ɛk ( ɛ) 2 (28) anddenotebyc theminimizerofg(c),thatis, min g(c) c>0 g(c ) (seeproblem2). (29) Finallyby(27),(29)andTheoremwecanwrite maxav(ψ c,φ) maxav(ψ c,φ), forallc>0 H F ɛ H F ɛ max H F ɛ AV(ψ,H), forallγ(ψ,φ)k(c) andforallc>0 max H F ɛ AV(ψ,H), forall ψ. 3

14 Problems Problem Showthatifψ satisfies ψ arg min γ(ψ,h 0) K AV(ψ,Φ) forsomek > 2/π,thenψ alsosatisfies forsomek 2 >. ψ arg min AV(ψ,H 0) K 2 γ(ψ,h 0 ) Problem 2 (a) Show that Equation (4) has a unique solution and write a computerprogramtosolveitforeachk> π/2. (b) Let g(c) αav (ψ c,φ)+βk(c) and recall that α /( ɛ) β ɛ/( ɛ) 2 (see(25)). Find(numerically if necessary) the minimizing value c c (ɛ)argming(c), (30) c>0 forgiven0<ɛ</2. (c) Find lim ɛ 0 c (ɛ) c 0 4

15 and lim ɛ 0.5 c (ɛ) c /2. (d)explainthestatisticalmeaningofc 0 andc /2. Problem 3 Verify that forallc 0. 2Φ(c) 2cϕ(c) [2Φ(c) ] 2 Problem 4 Findasharplowerboundforthegrosserrorsensitivityofdispersion M-estimates at the normal central model. Recall that maxχ( ) b,b} GES(s,Φ)s 0 ( )( )} E H0 χ y m0 x m0 s 0 s(h 0 ) and argue that, without loss of generality we can work with the simplified expression GES(s,Φ) maxχ( ) b,b} E Φ χ. (y)y} Problem 5 Solve Hampel s optimality problem of minimizing the asymptotic varianceatthecentralnormalmodelsubjecttoaboundonthegesfordispersion M-estimates at the normal central model. 0 Problem 6 Supposethat thereexists(x 0,y 0 )suchthat ming(x,y 0 )maxg(x 0,y)g(x 0,y 0 ). (3) x y A point (x 0,y 0 ) with this property is called a saddle point for the function g. Provethatif(x 0,y 0 )isasaddlepointforg then minmaxg(x,y)maxming(x,y)g(x 0,y 0 ). x y y x Problem 7 Find the least favorable distribution H for the family of symmetric distributions F with finite variance. Show that in this case (3) with g(x,y)av (ψ,h), x 0 ψ H and y 0 H, holds. Discuss the convenience and practical relevance issues in relation with F and the strengths and weaknesses of the corresponding minimax estimate. Problem 8 FindtheleastfavorabledistributionH forthefamilyf 2 ofsymmetric and strongly unimodal distributions H with h(µ) k, where µ is the commoncenterofsymmetryforthedistributionsinf 2. Showthatinthiscase (3) also holds and therefore the min-max and max-min problem are equivalent. As in Problem 7 discuss the convenience and practical relevance of the family F 2 andthestrengthsandweaknessesofthecorrespondingminimaxestimate. 5

16 References Huber, P. J.(964). Robust estimation of a location parameter. Ann. Math. Statist. 35, Hampel, F.R.(974). The influence curve and its role in robust estimation. J. Am. Stat. Assoc., 69, Li, B. and Zamar, R.H.(99). Min-max asymptotic variance when scale is unknown. Statistics and Probability Letters,, 39-45,(99). 6