Distribution free tests for polynomial regression based on simplicial depth

Transcription

1 Distribution free tests for poynomia regression based on simpicia depth by Robin Wemann, Peter Harmand, Christine H. Müer University of Kasse, University of Odenburg Juy 30, 007 Abstract A genera approach for deveoping distribution free tests for genera inear modes based on simpicia depth is presented. In most reevant cases, the test statistic is a degenerated U-statistic so that the spectra decomposition of the conditiona expectation of the kerne function is needed to derive the asymptotic distribution. A genera formua for this conditiona expectation is derived. Then it is shown how this genera formua can be specified for poynomia regression. Based on the specified form, the spectra decomposition and thus the asymptotic distribution is derived for poynomia regression of arbitrary degree. An appication on cubic regression demonstrates the appicabiity of the new tests and in particuar their outier robustness. Keywords: Distribution-free tests, simpicia depth, regression depth, poynomia regression, degenerated U-statistic, spectra decomposition, outier robustness. AMS Subject cassification: Primary 6G05, 6G0; secondary 6J05, 6J, 6G0. Introduction Simpicia depth for mutivariate ocation was introduced by Liu (988, 990). It is based on the haf space depth of Tukey (975). Both depth notions ead to a generaization of the median for mutivariate data which is equivariant with respect to affine transformations. Moreover the concept of depth is usefu to generaize ranks.

2 The simpicia depth has the advantage that it is an U-statistic so that in principe the asymptotic distribution is known. However, it is not easy to derive the asymptotic distribution. Arcones et a. (994) derived the asymptotic normaity of the maximum simpicia depth estimator of Liu (988, 990) via the convergence of the whoe U-process. The convergence of the U-process was aso shown by Dümbgen (99). However the asymptotic norma distribution has a covariance matrix which depends on the underying distribution. Hence this resut cannot be used to derive distribution-free tests. Liu (99) and Liu and Singh (993) proposed distribution-free mutivariate rank tests which generaize the Wicoxon s rank sum test for two sampes. Whie the asymptotic normaity is derived for severa depth notions for distributions on IR, it is shown ony for the Mahaanobis depth for distributions on IR k, k >. Hence it is uncear how to generaize the approach of Liu and Singh to other situations. Severa other depth concepts were introduced since Tukey (975). See for exampe the book of Moser (00) and the references in it. Mutivariate depth concepts were transferred to regression by Rousseeuw and Hubert (999), to ogistic regression by Christmann and Rousseeuw (00) and to the Michaeis-Menten mode by Van Aest et a. (00). The depth concept for regression bases on the notion of nonfit introduced by Rousseeuw and Hubert (999). Thereby, a regression parameter θ is caed a nonfit, if there is another parameter θ which provides for a observations z n smaer squared residuas r(z n,θ ). The depth of a regression parameter θ is then given by the minimum number of observations which must be removed so that θ becomes a nonfit. More genera concepts of depth were introduced and discussed by Zuo and Serfing (000a,b) and Mizera (00). Mizera (00) in particuar generaized the regression depth of Rousseuw and Hubert (999) by basing the nonfit on genera quaity functions instead of squared residuas. Using these quaity functions, he introduced the goba depth, the tangent depth and the oca depth and gave a sufficient condition for their equaity. This approach makes it possibe to define the depth of a parameter vaue with respect to given observations in various statistica modes via genera quaity functions. Appropriate quaity functions are in particuar ikeihood functions as studied by Mizera and Müer (004) for a depth notion of ocation and scae and by Müer (005) for a depth notion for generaized inear modes. As for mutivariate ocation, there exist ony few resuts concerning tests based on regression depth and its generaizations. Bai and He (999) derived the asymptotic distribution of the maximum depth estimator for regression so that tests coud be based on this. However, this asymptotic distribution is given impicity so that it is not convenient for inference. Van Aest et a. (00) derived an exact test based on the regression depth of Rousseeuw and Hubert (999) but did it ony for inear regression. Müer (005) derived tests by using the simpicia regression depth which generaizes Liu s (988, 990) simicia depth to regression. A genera simpicia depth for a q dimensiona parameter θ within a sampe z =

3 (z,...,z N ) can be defined by ( ) N d S (θ,z) = q + n <n <...<n q+ N ψ θ (z n,...,z nq+ ), where the symmetrica kerne function ψ θ IL ( q+ n= P Z θ ) is an indicator function which equas one if the depth d(θ, (z n,...,z nq+ )) of θ in z n,...,z nq+ is greater than zero. If the depth d is the regression depth, the genera simpicia depth is caed simpicia regression depth. Müer (005) proposed to base the test statistic directy on the genera simpicia depth d S. For testing a hypothesis of the form H 0 : θ Θ 0, where Θ 0 is an arbitrary subset of the parameter space, the test statistic is defined as T(z,...,z N ) := sup θ Θ0 T θ (z,...,z N ). Thereby T θ (z,...,z N ) is defined as N(dS (θ, (z,...,z N )) µ θ ) T θ (z,...,z N ) :=, (q + ) σ θ if d S (θ,z) is not a degenerated U-statistic, i.e. ψ θ (z ) := E(ψ θ (Z,...,Z q+ ) Z = z ) depends on z, and T θ (z,...,z N ) := N (d S (θ, (z,...,z N )) µ θ ), () if d S (θ,z) is a degenerated U-statistic. The nu hypothesis H 0 is rejected if T(z,...,z N ) is ess than the α-quantie of the asymptotic distribution of T θ (Z,...,Z N ). Thereby the quantities µ θ and σ θ are defined as µ θ = E(ψ θ (Z,...,Z q+ )) and σ θ = Var(ψ θ (Z )). Unfortunatey, the simpicia depth d S is a degenerated U-statistic in the most interesting case, that the true regression function is in the center of the data, which means that the median of the residuas is zero. Whereas nondegenerated U-statistics are asymptoticay norma distributed, simpe asymptotic resuts are not possibe for degenerated U-statistics. In the degenerated case, the asymptotic distributions can be derived by using the second component of the Hoeffding decomposition. We have namey the foowing resut (see e.g. Lee 990, p. 79, 80, 90, Witting and Müer-Funk, p. 650). If the reduced normaized kerne function ψ θ(z,z ) := E(ψ θ (Z,...,Z q+ ) µ θ Z = z,z = z ) () is IL -integrabe, it has a spectra decomposition of the form ψθ(z,z ) = λ ϕ (z ) ϕ (z ), (3) = where the functions ϕ are IL -integrabe, normaized, and orthogona. Then the asymptotic distribution of the simpicia depth is given by ( ) L q + N(d S (θ, (Z,...,Z N )) µ θ ) λ (U ), (4) 3 =

4 where U N(0, ) and U, U,... are independent. In the genera case, it coud happen that the eigenvaues λ depend on the underying parameter θ. However, Müer (005) coud show that this is not the case for poynomia regression in generaized inear modes so that the asymptotic distribution does not depend on the regression parameter. However, Müer (005) was ony abe to find the spectra decompositions for inear and quadratic regression in generaized inear modes. These spectra decompositions were found by soving differentia equations. In this paper we derive the spectra decomposition (3) for poynomia regression of arbitrary degree by a compete new approach. For this approach, we use in Section a rather genera quaity function for defining the depth d used in the simpicia depth d S. This genera quaity function, caed quaity function for extended inear regression, is motivated by the resut of Müer (005) that the tangent depth for quaity functions based on ikeihood functions in generaized inear modes is based on modified residuas g(y n ) x(t n ) θ, where the function g is a transformation of the dependent variabes y n and x is the regression function appied on the expanatory variabes t n. In this paper the quaity function is based on h(z n ) v(z n ) θ, where z n can be z n = (y n,t n ) with h(z n ) = g(y n ) and v(z n ) = x(t n ) but aso other reations are possibe. The simpicia depth is based on the tangent depth for this genera quaity function. However, this simpicia depth attain rather high vaues in subspaces of the parameter space, since it does not provide convex depth contours as a simpicia depth notions do not. This is in particuar a disadvantage in testing if the aim is to reject the nu hypothesis. To avoid this disadvantage, we introduce in Section a harmonized simpicia depth for genera inear modes. This approach eads aso to a method to cacuate the maximum simpicia depth under the nu hypothesis. Whie in Müer (005) ony nu hypotheses coud be rejected for which the nu hypothesis is a point or a ine within the parameter space, we are now abe to treat hypotheses about arbitrary subspaces and poyhedras, as Wemann et a. (007a) showed. In Section 3, we derive a genera formua for the conditiona expectation () for the simpicia depth for extended inear regression modes introduced in Section and we show that the asymptotic distribution can be obtained by cacuating the spectra decomposition of a function K, which ony depends on the probabiity aw of the vector product of regressor variabes. This means in particuar that the asymptotic distribution of the test statistic () does not depend on the unknown regression parameter. The function K is appied to the harmonized form of the simpicia regression depth but the proofs hod aso for the unmodified form. The genera formua for K is specified for poynomia regression of arbitrary degree in Section 4. Based on the specified formua, the spectra decomposition is derived. The spectra decomposition is found by a Fourier series representation of a reated function of IL [, ] which is used to derive the required representation of K. We think that this approach can be used to find the spectra decomposition of other simpicia depth functions. In particuar, Wemann and Müer (007b) derived the asymptotic distribution of 4

5 the simpicia regression depth for different modes of mutipe regression. Section 5 provides an appication on tests in a cubic regression mode. This exampe in particuar shows that the new tests possess high outier robustness. A proofs are given in Section 6. Simpicia depth for extended inear regression We assume that the random vectors Z,...,Z N are independent and identicay distributed throughout the paper. The random vectors Z n have vaues in Z IR p, p. There exist known functions v : Z IR q and h : Z IR so that the random variabes X n = v(z n ) and Y n = h(z n ) satisfy Y n = X T n θ + E n for θ Θ = IR q and random error E n. Random variabes are denoted by capita etters and reaizations by sma etters. The vaue s n (θ) := sign θ (z n ) := sign(y n x T nθ) is caed the sign of the residua of the n-th (transformed) observation. The famiy P = {P (Z,...,Z N ) θ : θ Θ} of probabiity measures with Θ = IR q may be unknown, but for the purpose of deriving tests, we wi assume that the foowing assumptions hod: P θ (S (θ) = X ) a.s., (5) P θ (S (θ) = 0 X ) 0 a.s., and P θ (X,...,X q are ineary dependent) = 0. Whie the ast two conditions of (5) are usuay satisfied for continuous distributions, the first condition can be satisfied by appropriate transformations v and h. In a mode satisfying (5), the foowing quaity function can be used. Definition (Quaity functions for extended inear regression) For z n Z take ϕ zn to be a function with continuous derivatives, which has its maximum and its soe critica point in 0. Let v : Z IR q and h : Z IR be measurabe. Then the function G zn : Θ IR with G zn (θ) := ϕ zn (h(z n ) v(z n ) T θ) is said to be a quaity function for extended inear regression. Mosty, one woud choose the ikeihood functions to be the quaity functions. However, for the resuting transformations, not aways p := P θ (E n > 0 X n ) = is satisfied, so that the true regression function is not in the center of the transformed data. In these cases it 5

6 is more appropriate to choose the transformation h and v, such that p = hods and to define the quaity functions by G zn (θ) := (h(z n ) v(z n ) T θ). Athough quaity functions are needed to define the tangent depth or the goba depth of Mizera (00), the resuting depth functions do not depend on the choice of ϕ zn, so that we may restrict ourseves to the simpest case ϕ zn (x) = x. The foowing exampe demonstrates, how h and v and thus the quaity function can be obtained: For generaized inear modes the observations are given by Z n = (U n,t n ) which have density f (Un,Tn) θ (u n,t n ) = f Un x(tn)=x(tn) θ (u n )f Tn (t n ). Athough these ikeihood functions are quaity functions for extended inear regression with regressors v(z n ) = x(t n ) in most modes for genera inear regression, not aways the assumption p = is satisfied. For exampe, for regression with exponentia distributed dependent observations, that is, f Un x(tn)=x(tn) θ (u n ) = λ n exp( λ n u n ) with λ n = exp ( x(t n ) T θ ), the approach via ikeihood functions eads to Y n = h(z n ) = og(u n ), whereas ony the transformation Y n = h(z n ) = og( Un og() ) eads to p =. Definition (Tangent depth) According to Mizera (00), we define the tangent depth of θ Θ with respect to given observations z,...,z N Z to be d T (θ,z) = min u 0 #{n : ut G zn (θ) 0}, where G z,..., G zn are quaity functions for extended inear regression, z := (z,...,z N ) and G zn (θ) denotes the vector of partia derivatives of G zn in θ. As shown in Mizera (00), this depth notion counts the number of observations that needs to be removed such that there is a better parameter for a remaining observations. It s easy to see, that the tangent depth does not depend on the choice of ϕ zn. Furthermore, for a θ Θ and for given observations z,...,z N Z we have: d T (θ,z) = min u 0 #{n : s n(θ) u T x n 0}. (6) As in Rousseuw and Hubert (999) it can be shown, that the parameter space Θ = IR q is divided up into domains with constant depth by the hyperpanes H n = {θ IR q : s n (θ) = 0},n =,...,N. For given observations et Dom(z) be the set of a those domains. We define d T (G,z) := d T (θ,z) for G Dom(z) and θ G. 6

7 We wi define the simpicia depth to be an U-statistic. If we woud take the tangent depth to be the kerne function of the U-statistic, then the simpicia regression depth attain rather high vaues in subspaces of the parameter space, namey in Border(z) := N n=h n. This is in particuar a disadvantage if the aim is to reject the nu hypothesis. To avoid this disadvantage, we introduce a harmonized depth. Definition 3 (Harmonized depth) The harmonized depth of θ Θ with respect to the observations z,...,z N Z is defined to be ψ θ (z) = min d T (G,z), G Dom(z),θ Ḡ where Ḡ is the cosure of G. Definition 4 (Simpicia depth) The simpicia depth is given by d S (θ,z) = ( ) N ψ θ (z n,...,z nq+ ). q + n <n <...<n q+ N This depth, which transfers the simpicia depth of Liu to regression modes, is aso caed a simpicia depth because it counts the fraction of simpicies that are bounded by q + hyperpanes from H,...,H N and contain θ as an interiour point. Agorithms for cacuating the simpicia depth are based on this view as we and are given in Wemann et a. (007a). The proposed tests are based on the asymptotic distribution of this depth notion. 3 The asymptotic distribution of the simpicia depth The definition of tangent depth shows, that the depth of a parameter is the hafspace depth of 0 with respect to the gradients of the quaity functions. Thereby, the haf space depth of 0 with respect to given vectors r,...,r N IR q is defined as d H (0,r) := min u 0 #{n : ut r n 0}, where r = (r,...,r q+ ) (see Tukey, 975). The next emma is needed to derive the conditiona expectations of the kerne function, which depend ony on q + observations, but it can aso be used to cacuate the simpicia depth of a given parameter. 7

8 Lemma Let r,...,r q+ IR q be in genera position. Then d H (0,r) {0, } and the foowing statements are equivaent: (i) d H (0,r) = 0, (ii) r / IR 0 r IR 0 r q+. Proofs are given in the appendix. The next Proposition shows, that ψ θ(z ) := E(ψ θ (Z,...,Z q+ ) Z = z ) does not depend on z, so that the simpicia depth is a degenerated U-statistic and has asymptoticay the distribution of an infinite inear combination of χ -distributed random variabes (see e.g. Lee 990, p. 79, 80, 90, Witting and Müer-Funk, p. 650). This distribution depends ony on the conditiona expectation ψ θ(z,z ) := E(ψ θ (Z,...,Z q+ ) Z = z,z = z ) E(ψ θ (Z,...,Z q+ )). Proposition Let θ Θ and et z,z Z, such that x,x are ineary independent and s (θ),s (θ) {, }. Then and ψ θ(z,z ) = s (θ)s (θ) q ψ θ(z ) = q ( P θ (x T W x T W < 0) ), where W := X 3... X q+ is the vector product of X 3,...,X q+. With this proposition, we obtain a main resut: We get the asymptotic distribution of the simpicia depth in extended inear regression by cacuating the spectra decomposition of the kerne K, defined by K(x,x ) := P θ (x T W x T W < 0), for x,x IR q. (7) Note that x i W = det(x i,x 3,...,X q+ ) for i =,. The spectra decomposition is a representation K(x,x ) = ( λ j ϕ j (x )ϕ j (x ) in IL P X P ) X, j= ( ) where (ϕ j ) j= is an orthonorma system (ONS) in IL P X and λ,λ,... IR. The functions (ϕ j ) j= are eigenfunctions and the vaues λ,λ,... are the corresponding eigenvaues of the reated integra operator T K, defined by T K : IL (P X ) IL (P X ) with T K f(s) = K(s,t)f(t) dp X (t). 8

9 The system (ψ j ) j=, defined by ψ j (z) := sign θ (z) ϕ j (v(z)) for z Z is an ONS in IL (P Z ) and for ψθ we have the representation ψθ(z,z ) = sign θ(z )sign θ (z ) K(v(z q ),v(z )) = q λ j sign θ (z )ϕ j (v(z )) sign θ (z )ϕ j (v(z )) = Hence, it foows by (4), that j= j= q λ j ψ j (z ) ψ j (z ). N ( d S (θ, (Z,...,Z N )) q ) L = (q + )! ( (q )! qλ U ), (8) where U,U,... are i.i.d. random variabes with U N(0, ). Furthermore, this derivation shows, that the asymptotic distribution does not depend on the underying parameter θ, if the distribution of W does not depend on it. In the next section, this genera resut is appied to poynomia regression. 4 Poynomia regression A specia extended inear regression mode is the poynomia regression mode of degree r = q. In this mode, the unknown parameter is θ = (θ,...,θ q ) IR q, Z IR, Z n = (Y n,t n ), the vector v(z n ) := x(t n ) := (,T n,...,t r n) T is the regressor and h(z n ) = Y n is the dependent variabe. Because of the independence of T,...,T N, the third assumption in (5) is equivaent to P θ (T = t) = 0 for a t IR. In this section, we derive the asymptotic distribution of the simpicia depth by cacuating the spectra decomposition of the kerne K, given in (7). Whie Müer (005) derived it ony for r = and r = in another way, we have now the asymptotic distribution for poynomia regression of arbitrary degree. At first, we give a simpe representation of the kerne K, whis is obtained from (7) via the formua for Vandermonde determinants (see the appendix). Proposition For a t,t IR we have K(x(t ),x(t )) = r ( F T (t ) F T (t ) ) r, where F T is the distribution function of T. 9

10 Müer derived the same formua for the reduced normaized kerne function ψθ (see Proposition in Müer 005). Our proof is based not on ψθ, but on K. This makes the proof much shorter. It remains to derive the spectra decomposition of K, which we obtain in the next proposition via a Fourier series representation of ( z )r in IL [, ]. Proposition 3 The spectra decomposition of ( s t ) r in IL [0, ] is given by ( ) r s t = γ (r) 0 + γ (r) [cos(kπs) cos(kπt) + sin(kπs) sin(kπt)] where for r odd γ (r) = = 0, if is even, k {,...,r} k odd r! r k (r k)! ( π ) k+, if is odd, and for r even γ (r) =, if = 0, (r+) r k {,...,r} k odd r! r k (r k)! ( π ) k+, if is even and > 0, 0, if is odd. Let (ψ j ) j J be the ONS in IL [0, ], given in the proof of Proposition 3, such that (γ (r) j ) j J are the eigenvaues, reated to K(s,t) = ( s t )r. Then the system (ϕ j ) j J, defined by ϕ j := ψ j F T x is an ONS in IL(P X ) and we have the representation K(x,x ) = K(x(x (x )),x(x (x ))) = r ( F T (x (x )) F T (x (x )) ) r = j J ( r γ (r) j )ϕ j (x )ϕ j (x ). Hence, the next Theorem hods: 0

11 Theorem If P(Y n x(t n ) θ 0 T n ) = and T n has a continuous distribution, then the asymptotic distribution of the simpicia ikeihood depth d S (θ, (Z,...,Z N )) for poynomia regression is given by N ( d S (θ, (Z,...,Z N )) r+ ) L for r even and ( N d S (θ, (Z,...,Z N )) ) L λ r+ 0 (U ) + =0 λ + (V + W ) = λ (V + W ) for r odd, where U, V 0,W 0,V,W,... are independent random variabes with standard norma distribution and λ 0 = r +, r+ λ = k {,...,r} k odd (r + )! r k+ (r k)! ( π ) k+ for IN. The cacuation of the test statistic and the critica vaues for any hypothesis of the form H 0 : θ Θ 0 where Θ 0 is a subspace of the parameter space or a poyhedron is described in Wemann et a. (007a). There aso a tabe of the critica vaues is given. The exampe in Section 5 shows the appicabiity of the method for q > 3, whereas in Müer (005) and Wemann et a. (007a) exampes for q 3 were given. In particuar, in Wemann et a. an exampe is cacuated, where the hypotheses is given by a poyhedra. There it is aso shown that the method can be used for tests in two sampe probems. The exampes demonstrate that the tests are outier robust. See aso Wemann (007c). 5 Appication: Test about quadratic function against cubic function The concentration of maondiadehyd (MDA) for 78 women twice after chidbirth (IMDA and IIMDA) at two time points was measured, to find a reation between the eves of IMDA and IIMDA. MDA is a metaboite of ipid peroxides detectabe in pasma. It was measured as an indicator of ipid peroxidation and oxidation stress of women post partum (after chidbirth). The data came from the Cinic of Gynaecoogy, Facuty Hospita with Poicinic, Bratisava-Ružinov (Sovakia). We assume a cubic regression mode (q = 4) and choose t n as IIMDA and y n as IMDA. Normaity of the residuas with respect to the ordinary east square estimation

12 IMDA quadratic function cubic function IMDA deepest quadratic function (q=4) deepest quadratic function (q=3) IIMDA IIMDA Figure : Least squares quadratic and cubic function Figure : Deepest quadratic functions for q = 3 and q = 4. was rejected (p-vaue< 0.00) by the χ goodness-of-fit test (function chisq.gof in S- Pus). Assuming a cubic regression mode with parameter θ = (θ,θ,θ 3,θ 4 ) T IR q, we want to test the hypothesis that the true function is quadratic. Hence, we want to test H 0 : θ Θ 0 with Θ 0 = {θ IR 4 : θ 4 = 0}. In the cubic regression mode with q = 4 and θ 4 = 0, a parameter with maximum simpicia depth is θd = ( , 0.503, , 0) T (Figure ). The maxima simpicia depth is sup θ Θ0 d S (θ, (z,...,z 78 )) = and the test statistic is T(z,...,z 78 ) = 0.364, which is more than the 90%-quantie (see Wemann et a. 007a) and hence we can not reject the nu hypothesis for the significance eve 0%. Thus, we may assume a mode for quadratic regression (q = 3). The deepest quadratic function in the quadratic regression mode is rather simiar to the deepest quadratic function in the cubic regression mode (see Figure ). Note, that aso the hypothesis, that the true function is a inear function, cannot be rejected within the mode for quadratic regression (q = 3), since the test statistic is 0.534, which is aso more than the 90%-quantie of the asymptotic distribution. The east squares estimation within the mode for quadratic regression is θ = ( , ,.0600) T. Within the mode for cubic regression it is θ = ( , , ,.6087) T (Figure ). Athough the residuas are not normay distributed, we tests the nu hypothesis H 0 : θ 4 = 0 against H : θ 4 0 by the F-test (function anova in S-Pus). The F-test provides for the nu hypothesis a p-vaue=0.08, so H 0 : θ 4 = 0 is rejected. This is due to the outiers at the right hand side and without them a quadratic or inear regression function is a good description of the data.

13 6 Proofs For more detais of the proofs see aso Wemann (007c). Proof of Lemma Since r,...,r q are ineary independent, they beong to a hyperpane H with 0 H. There is a γ < 0 and a u IR q, such that H = { v IR q : v T u = γ }. Since r,...,r q don t beong to the haf space { v IR q : v T u 0 }, we have d H (0,r). It remains to show the equivaence. (ii) (i): For any j =,...,q + et H j be the hyperpane that contains the points (r i ) i {,...,q+}\{j}. Step : There is a j {,...,q + } such that 0 and r j are on different sides of H j. Proof: Since r,...,r q+ is a Basis of IR q, there exists γ,...,γ q+ IR such that r = γ r γ q+ r q+. Since r / IR 0 r IR 0 r q+ we may assume that γ > 0. We prove that r and 0 are on different sides of H, if r and 0 are on the same side of H. Hence, we have: (a) r = γ r γ q+ r q+ with γ > 0. (b) There are α > 0 and β 3,...,β q+ IR such that r = r α r + q+ β j (r j r ) IR <0 r + H. j=3 From these equations we obtain two different representations of r : r r = = ( q+ ) α β j r + β 3 r β q+ r q+, j=3 γ r + γ 3 γ r γ q+ γ r q+. Comparing the coefficients eads to 0 < q+ = α β j and γ k = β k for k = 3,...,q +. γ γ j=3 3

14 It foows that γ γ q+ = γ γ β j... γ β q+ q+ ( ) = γ β j = j=3 q+ β j j=3 >. α q+ β j j=3 With (a) we have r = (γ + γ γ q+ )r + γ 3 (r 3 r )... + γ q+ (r q+ r ). Thus there is a λ (0, ) with: λ r r + q+ j=3 IR(r j r ) = H. Hence, r and 0 are on different sides of H. This finishes the proof of Step. Step : Main proof. The vectors r j and 0 are on different sides of this affine hyperpane H j. Let v H j. A vectors r,...,r q+ are in the open haf space IR >0 v + (H j v). The haf space IR q \(IR >0 v+(h j v)) don t contain the vectors r,...,r q+. Thus, d H (0,r) = 0. (i) (ii) The vectors r,...,r q+ beong to an open haf space H with 0 H r,..., r q+ H := IR q \H IR 0 r,...,ir 0 r q+ H IR 0 r IR 0 r q+ H. Because of r H it foows that r / IR 0 r IR 0 r q+. Proof of Proposition Let z,...,z m Z, such that x,...,x m are ineary independent and s n := s n (θ) 0 for n =,...,m, where m q. Let be X := (X m+,...,x q+ ), Z := (z,...,z m,z m+,...,z q+ ), X X gp := v(z), := {(x m+,...,x q+ ) X q+ m : each subset of q vectors from x,...,x q+ is in. indep.}, X ex := {(x m+,...,x q+ ) X gp : s m+,...,s q+ {, } : d H (0, (s x,...,s q+ x q+ )) = }. 4

15 We have to cacuate E(ψ θ (Z,...,Z q+ ) Z = z,...,z m = z m ) = E(ψ θ (Z )). Since P( X X gp ) = and P(ψ θ (Z ) {0, }) =, we have E(ψ θ (Z )) = P(ψ θ (Z ) = and X X gp ). Because of {ψ θ (Z ) = } { X X gp } { X X ex }, it foows that E(ψ θ (Z )) = P(ψ θ (Z ) = and X X ex ). For r,...,r q+ IR q et σ (r,...,r q+ ) : {,...,q + } {, }, such that s x IR 0 σ (r,...,r q+ )()r IR 0 σ (r,...,r q+ )(q + )r q+, if each subset of q vectors from s x,r,...,r q+ is ineary independent. Since x,...,x m are fixed, we can write σ x := σ (x,...,x q+ ) for x = (x m+,...,x q+ ) X gp. Now, we prove that P(ψ θ (Z ) = and X X ex ) = P( n = m +,...,q + : sign θ (Z n ) = σ X(n), X X ex }. Therefore et z m+,...,z q+ Z with x := (x m+,...,x q+ ) X ex. Since x X ex there are s m+,...,s q+ {, } with d H (0, (s x,...,s q+ x q+ )) =. With Lemma it foows that s x IR 0 s x IR 0 s q+ x q+. Hence, the definition of σ x impies that Furthermore, we have s n = σ x (n) for n =,...,m. (9) ψ θ (z) = θ q+ n=h n and d T (θ,z) = sign θ (z n ) 0 for n =,...,q + and d T (θ,z) = sign θ (z n ) 0 for n =,...,q + and d H (0, (s (θ)x,...,s q+ (θ)x q+ )) = Lemma sign θ (z n ) 0 for n =,...,q + and (9) sign θ (z )x IR 0 sign θ (z )x IR 0 sign θ (z q+ )x q+ n =,...,q + : sign θ (z n ) = σ x (n) n = m +,...,q + : sign θ (z n ) = σ x (n). Hence, E(ψ θ (Z )) = P( n = m +,...,q + : sign θ (Z n ) = σ X(n), X X ex ) = P( n = m +,...,q + : sign θ (Z n ) = σ x (n) X = x) dp X( x). X ex 5

16 Since (sign θ (Z m+ ),..., sign θ (Z q+ )) and X are independent, it foows, that E(ψ θ (Z )) = = = = P( n = m +,...,q + : sign θ (Z n ) = σ x (n)) d P X( x) X ex q+ P(sign θ (Z n ) = σ x (n)) d P X( x) X ex n=m+ X ex ( ( ) q+ m d P X( x) ) q+ m P( X X ex). For m = we have X gp X ex and thus ψ θ (z ) = ( ) q+ P( X Xex ) = q. It remains to prove the second equation. Therefore, et m =. Let x 3,...,x q+ X, such that (x 3,...,x q+ ) X gp and et w := x 3... x q+. Then we have (x,...,x q+ ) X ex Def. s 3,...,s q+ {, } : d H (0, (s x,...,s q+ x q+ )) = Prop. s 3,...,s q+ {, } : s x IR <0 s x IR <0 s q+ x q+ α,β > 0, λ IR q,λ 0 : (αs x + βs x,x 3,...,x q+ )λ = 0 α,β > 0 : det(αs x + βs x,x 3,...,x q+ ) = 0 α,β > 0 : (αs x + βs x ) T w = 0 α,β > 0 : αs x T w + βs x T w = 0 sign(s x T w) = sign(s x T w) s s x T wx T w < 0. Note, that the equation P(sU < 0) = sp(u < 0) + s hods for each IR-vaued random variabe U with P(U = 0) = 0 and s {, }. It 6

17 foows that ψθ(z,z ) = E(ψ θ (Z )) E(ψ θ ) ( ) q+ = P( X X ex) q ( ) q = P(s s x T Wx T W < 0) q ( ) q = (s s P(x T Wx T W < 0) + s s ) q = s s (P(x T Wx T W < 0) ). q Proof of Proposition Note, that the equation P ( N U j < 0 ) = ( P(U < 0)) N j= hods for N IN and i.i.d. IR-vaued random variabes U,...,U N with P(U = 0) = 0. Since the occurring determinants are Vandermonde determinants, we have for a t,t IR: K(x(t ),x(t )) = P(x(t ) T (X 3... X q+ ) x(t ) T (X 3... X q+ ) < 0) = P(det (x(t ),x(t 3 ),...,x(t q+ )) det (x(t ),x(t 3 ),...,x(t q+ )) < 0) = P ( (T j t ) (T j T i ) (T j t ) (T j T i ) < 0 ) j 3 3 i<j q+ = P ( q+ (T j t )(T j t ) < 0 ) j=3 j 3 = ( P((T t )(T t ) < 0)) q = ( F T (t ) F T (t ) ) q. 3 i<j q+ Proof of Proposition 3 At first we derive the Fourier series representation of f r where f is given by f : [, ] z f(z) = [ z, ]. 7

18 Since { }, cos( π ), sin( π ); IN is an orthonorma basis of IL [, ] and f r is with f an even function, f r can be represented ony by and the cosine functions, i.e. f r (z) = α (r) 0 + = α (r) cos( π z). Since f r is continuous and piecewise differentiabe, the series is uniformy convergent so that α (r) 0 = f r (z) dz = f r (z) dz 0 and for This impies for r = α (r) = f r (z) cos( π z) dz = 0 α () 0 = 0, f r (z) cos( π z) dz. and for α () = For r =, we obtain 0 ( ) z cos( π z) dz = { 0, if is even, 4, π if is odd. (0) α () 0 = 0 ( ) z dz =, and for α () = For r >, we have 0 ( ) z cos( π z) dz = { 4 π, if is even, 0, if is odd. () α (r) 0 = 0 ( ) r z dz = { 0, if r is odd,, (r+) r if r is even, 8

19 and for, partia integration provides the foowing recursion formua for α (r) α (r) = 0 = π = r π = r π = r π 0 ( ) r z cos( π z) dz ( ) r + r π sin( π z) z 0 ( ) r z sin( π z) dz π π cos( π z) ( z [( ) ( ) r r [ = ( ) +r ] π r α (r ) = r (r ) π [ r π (r )α (r ) r 3 0 ) r r π 0 ( ) ] r r (r ) π ( ) r z sin( π z) dz (r ) π r π α (r ) 0 (r ) π, if + r is odd, ], if + r is even. ( ) r z cos( π z) dz α(r ) Since α () = 0 if is even and α () = 0 if is odd, we obtain α (r) r + is even and, then we have = 0 if r + is odd. If α (r) = k {,...,r} k odd r! r k (r k)! ( π ) k+. This can be seen by induction over r: for r = and odd, it hods according to (0) k {,...,r} k odd r! r k (r k)! ( π ) k+ =! 0! ( π ) = 4 π = α(), and for r = and even, it hods according to () k {,...,r} k odd r! r k (r k)! ( π ) k+ =!! ( π ) = 4 π = α(). 9

20 The induction step is done from r to r +, that is: α (r+) = r + [ ] (r + π )α(r) r = r + π = = + (r + ) r (r + )! r+ 3 (r + )! ( π ) k {,...,r+} k odd k {,...,r} k odd k {,...,r} k odd r! r k (r k)! ( π ) k+ (r + )! r+ (k+) (r + (k + ))! ( π ) k++ (r + )! r+ k (r + k)! ( π ) k+. Hence, we aways have α (r) 0 = γ (r) 0 and α (r) quantities of Theorem 3. = γ (r) for, where γ (r) are the To finish the proof, we transfer the Fourier series representation of f r (z) on [, ] to that of g r (s,t) = f r (s t) on [0, ]. This provides ( ) r s t = f r (s t) = α (r) 0 + = α (r) 0 + = α (r) = α (r) cos( π (s t)) [cos( π s) cos( π t) + sin( π s) sin( π t)] which is the representation given by Theorem 3 using the reation between α (r) The quantities γ (r) are used in Theorem 3 since ony { S =, cos( π ), } sin( π ); IN and γ (r). are normaized functions of IL [0, ]. However, S is not an orthonorma system of IL [0, ]. But, since the quantities γ (r) are zero as soon as r + is odd, ony the systems { } cos( π ), sin( π ); IN and is odd for r odd, {, cos( π ), } sin( π ); IN and is even for r even, are reevant and these are orthonorma systems of IL [0, ]. 0

21 Acknowedgement The data base for the exampe is created by Stanisav Katina. We thank him for permission to incude them. References [] Arcones, M.A., Chen, Z. and Giné, E. (994). Estimators reated to U-processes with appications to mutivariate medians: asymptotic normaity. Ann. Statist., [] Bai, Z.-D. and He, X. (999). Asymptotic distributions of the maxima depth estimators for regression and mutivariate ocation. Ann. Statist. 7, [3] Christmann, A. and Rousseeuw, P.J. (00). Measuring overap in ogistic regression. Computationa Statistics and Data Anaysis 8, [4] Dümbgen, L. (99). Limit theorems for the simpicia depth. Statist. Probab. Lett. 4, 9-8. [5] Lee, A.J. (990). U-Statistics. Theory and Practice. Marce Dekker, New York. [6] Liu, R.Y. (988). On a notion of simpicia depth. Proc. Nat. Acad. Sci. USA 85, [7] Liu, R.Y. (990). On a notion of data depth based on random simpices. Ann. Statist. 8, [8] Liu, R.Y. (99). Data depth and mutivariate rank tests. In L -Statistica Anaysis and Reated Methods, ed. Y. Dodge, North-Hoand, Amsterdam, [9] Liu, R.Y. and Singh, K. (993). A quaity index based on data depth and mutivariate rank tests. J. Amer. Statist. Assoc. 88, [0] Mizera, I. (00). On depth and deep points: A cacuus. Ann. Statist. 30, [] Mizera, I. and Müer, Ch.H. (004). Location-scae depth. Journa of the American Statistica Association 99, With discussion. [] Moser, K. (00). Mutivariate Dispersion, Centra Regions and Depth. The Lift Zonoid Approach. Lecture Notes in Statistics 65, Springer, New York. [3] Müer, Ch.H. (005). Depth estimators and tests based on the ikeihood principe with appication to regression. Journa of Mutivariate Anaysis 95, 53-8.

22 [4] Rousseeuw, P.J. and Hubert, M. (999). Regression depth (with discussion). J. Amer. Statist. Assoc. 94, [5] Tricomi, F.G. (955). Voresungen über Orthogonareihen. Springer, Berin. [6] Tukey, J.W. (975). Mathematics and the picturing of data. In Proc. Internationa Congress of Mathematicians, Vancouver 974,, [7] Van Aest, S., Rousseeuw, P.J., Hubert, M. and Struyf, A. (00). The deepest regression method. J. Mutivariate Ana. 8, [8] Wemann, R., Katina, S. and Müer, Ch.H. (007a). Cacuation of simpicia depth estimators for poynomia regression with appications. Computationa Statistics and Data Anaysis 5, [9] Wemann, R. and Müer, Ch.H. (007b). Tests for mutipe regression based on simpicia depth. Submitted. [0] Wemann, R. (007c). On data depth with appication to regression and tests. Ph.D. thesis. University of Kasse, Germany. [] Witting, H. and Müer-Funk, U. (995). Mathematische Statistik II. Teubner, Stuttgart. [] Zuo, Y. and Serfing, R. (000a). Genera notions of statistica depth function. Ann. Statist. 8, [3] Zuo, Y. and Serfing, R. (000b). Structura properties and convergence resuts for contours of sampe statistica depth functions. Ann. Statist. 8, Department of Mathematics University of Kasse D-3409 Kasse Germany