Minimum Phi-Divergence Estimators and Phi-Divergence Test Statistics in Contingency Tables with Symmetry Structure: An Overview

Transcription

1 Symmetry 010,, ; doi: /sym01108 OPEN ACCESS symmetry ISSN Review Minimum Phi-Divergence Estimators and Phi-Divergence Test Statistics in Contingency Tables with Symmetry Structure: An Overview Leandro Pardo 1, and Nirian Martín 1 Department of Statistics and O.R., Complutense University of Madrid, 8040 Madrid, Spain Department of Statistics, Carlos III University of Madrid, 8903 Getafe Madrid, Spain Author to whom correspondence should be addressed; lpardo@mat.ucm.es. Received: 1 March 010; in revised form: 7 May 010 / Accepted: 10 June 010 / Published: 11 June 010 Abstract: In the last years minimum phi-divergence estimators MϕE and phi-divergence test statistics ϕts have been introduced as a very good alternative to classical likelihood ratio test and maximum likelihood estimator for different statistical problems. The main purpose of this paper is to present an overview of the main results presented until now in contingency tables with symmetry structure on the basis of MϕE and ϕts. Keywords: minimum phi-divergence estimator; phi-divergence statistics; symmetry model Classification: MSC 6B10, 6H15 1. Introduction An interesting problem in a two-way contingency table is to investigate whether there are symmetric patterns in the data: Cell probabilities on one side of the main diagonal are a mirror image of those on the other side. This problem was first discussed by Bowker [1] who gave the maximum likelihood estimator as well as a large sample chi-square type test for the null hypothesis of symmetry. The minimum discrimination information estimator was proposed in [] and the minimum chi-squared estimator in [3]. In [4 7] new families of test statistics, based on ϕ-divergence measures, were introduced. These families contain as a particular case the test statistic given by [1] as well as the likelihood ratio test.

2 Symmetry 010, 1109 Let X and Y denote two ordinal response variables, X and Y having I levels. When we classify subjects on both variables, there are I possible combinations of classifications. The responses X, Y of a subject randomly chosen from some population have a probability distribution. Let p ij = PrX = i, Y = j, with p ij > 0, i, j = 1,..., I. We display this distribution in a rectangular table having I rows for the categories of X and I columns for the categories of Y. Consider a random sample of size n on X, Y and we denote by n ij the observed frequency in the i, jth cell for i, j I I with I I n ij = n. The classical problem of testing for symmetry is given by H 0 : p ij = p ji, i, j I I 1 versus H 1 : p ij p ji, for at least one i, j pair This problem was considered for the first time by Bowker [1] using the Pearson test statistic X = i<j n ij n ji n ij + n ji 3 for which established that X χ k for large n, where k = 1 II 1. In some real problems i.e., medicine, psychology, sociology, etc. the categorical response variables X, Y represent the measure after or before a treatment. In such situations our interest is to determine the treatment effect, i.e., if X Y we assume that X represents the measure after the treatment and Y before the treatment. In the following we understand that X is preferred or indifferent to Y, according to joint likelihood ratio ordering, if and only if iff p ij p ji i j. In this situation the alternative hypothesis is H 1 : p ij p ji, for all i j 4 This problem was first considered by El Barmi and Kochar [8], who presented the likelihood ratio test for the problem of testing H 0 : p ij = p ji against H 1 : p ij p ji, i j and considered the application of it to a real life problem: They tested if the vision of both the eyes, for 7477 women, is the same against the alternative that the right eye has better vision than the left eye. In [5] these results were extended using ϕ-divergence measures. In this paper we present an overview on contingency tables with symmetry structure on the basis of divergence measures. We pay especial attention to the family of ϕ-divergence test statistics for testing H 0 versus H 1, H 0 against H 1 and also for testing H 1 against the alternative H of no restrictions over p ij s, i.e., H 1 : p ij p ji, i j, against H : p ij p ji, i j 5 It is interesting to observe that not only we consider ϕ-divergence test statistics but also we consider minimum ϕ-divergence estimators in order to estimate of the parameters of the model.

3 Symmetry 010, Phi-divergences Measures We consider the set Θ = {θ : θ = p ij ; 1 i, j I, i, j I, I with p ij > 0 and I I,i,j I,I p ij < 1} 6 and we denote pθ = p = p 11,..., p II T, p II = 1 I I,i,j I,I p ij or equivalently by the I I matrix pθ = [p ij ]. The ϕ-divergence between two probability distributions p = [p ij ], q = [q ij ] was introduced independently by [9] and [10]. It is defined as follows: D ϕ p, q = q ij ϕ pij q ij, ϕ Φ 7 where Φ is the class of all convex functions ϕ : [0, R { }, such that ϕ 1 = 0, ϕ 1 > 0; and we define 0ϕ 0/0 = 0 and 0ϕ p/0 = lim u ϕ u /u. For every ϕ Φ that is differentiable at x = 1, the function ψ given by ψ x = ϕ x ϕ 1 x 1 also belongs to Φ. Then we have D ψ p, q = D ϕ p, q, and ψ has the additional property that ψ 1 = 0. Because the two divergence measures are equivalent, we can consider the set Φ to be equivalent to the set Φ Φ {ϕ : ϕ 1 = 0} An important family of ϕ-divergences in statistical problems, is the power divergence family ϕ λ x = x x + λ 1 x ; λ 0, λ 1 8 λ λ + 1 ϕ 0 x = lim λ 0 ϕ λ x = x ln x x + 1 ϕ 1 x = lim λ 1 ϕ λ x = ln x + x 1 which was introduced and studied by [11]. Notice that ϕ λ Φ. In the following we shall denote the power-divergence measures by D ϕλ p, q, λ R. For more details about ϕ-divergence measures see [1]. 3. Hypothesis Testing: H 0 versus H 1 We define B = {a 11,..., a 1I, a,..., a I,..., a I 1I 1, a I 1I T R II : i j a ij < 1, i, j = 1,.., I} the hypothesis 1 can be written as H 0 : θ = gβ, β= p 11,..., p 1I, p,..., p I,..., p I 1I 1, p I 1I T B 9

4 Symmetry 010, 1111 where the function β is defined by β = g ij ; i, j = 1,..., I, i, j I, I with { p ij, i j g ij β =, i, j = 1,..., I 1 p ji, i > j and g Ij β = p ji, j = 1,..., I 1 g ii β = p ii, i = 1,..., I 1 Note that pgβ = g ij β; i, j = 1,..., I T, where g II β = 1 I i,,i,j I,I g ijβ The maximum likelihood estimator MLE of β can be defined as β = arg min β B D KL p, pgβ a.s. where D KL p, pgβ is the Kullback Leibler divergence measure see [13,14] defined by D KL p, pgβ = p ij log p ij g ij β We denote by θ = g θ and by p θ = p 11 θ,..., p II θ T. It is well known that p ij θ = p ij+ p ji, i = 1,..., I, j = 1,..., I. Using the ideas developed in [15], we can consider the minimum ϕ -divergence estimator Mϕ E replacing the Kullback Leibler divergence by a ϕ -divergence measure in the following way θ ϕ = arg min D ϕ p, pgβ; ϕ Φ 10 β B where D ϕ p, pgβ = g ij βϕ pij g ij β We denote θ S,ϕ = g β ϕ and we have see [7,16] ng βϕ β L N 0, I S F β 1 n where I S F θ = Σ θ Σ θ Bθ T BθΣ T θ Bθ 1 BθΣ θ being Σ θ = diagθ θθ T hij θ and Bθ = θ ij. The functions h II 1 ij are given by I 1 h ij θ = p ij p ji, i < j, i = 1,..., I 1, j = 1,..., I It is not difficult to establish that the matrix I S F θ can be written as I S F θ = M T βi F β 1 M β

5 Symmetry 010, 111 where I F β is the Fisher information matrix corresponding to β B. If we consider the family of power divergences we get the minimum power-divergence estimator, θ S,λ of θ, under the hypothesis of symmetry, whose expression is given by For λ = 0 we get θ S,λ ij = p ij + p ji θ S,0 ij 1 p ij + p ji 1, i, j = 1,..., I 11 = p ij + p ji, i, j = 1,..., I hence, we obtain the maximum likelihood estimator for symmetry introduced by [1]. For λ = 1, we obtain as a limit case θ S, 1 p ij p ji 1 ij =, i, j = 1,..., I p ij p ji 1 i.e., the minimum discrimination estimator for symmetry introduced and studied in []. For λ = 1 we get the minimum chi-squared estimator for symmetry introduced in [3], p ij + p 1/ ji θ S,1 ij = p ij + p ji 1/ We denote θ ϕ = g β ϕ and by p θ ϕ = p 11 θ ϕ,..., p II θ ϕ T 1 the Mϕ E of the probability vector that characterizes the symmetry model. Based on p θ ϕ it is possible to define a new family of statistics for testing 1 that contains as a particular case Pearson test statistic as well as likelihood ratio test. This family of statistics is given by T ϕ 1 n θ ϕ n ϕ 1 1 D ϕ 1 p, p θ ϕ = n ϕ 1 1 p ij θ ϕ ϕ 1 p ij p ij θ ϕ We can observe that the family 13 involves two functions ϕ 1 and ϕ, both belonging to Φ. We use the function ϕ to obtain the Mϕ E and ϕ 1 to obtain the family of statistics. If we consider ϕ 1 x = 1 x 1 and ϕ x = x log x x + 1 we get Pearson test statistic whose expression was given in 3 and for ϕ 1 x = ϕ x = x log x x + 1 we get the likelihood ratio test given by G = n ij log In the following theorem the asymptotic distribution of T ϕ 1 n θ ϕ is obtained. i<j 13 n ij n ji + n ij 14

6 Symmetry 010, 1113 Theorem 1 The asymptotic distribution of T ϕ 1 n θ ϕ is chi-squared with m = II 1/ degrees of freedom. Proof. See Chapter 8 in [1]. Thus, for a given significance level α 0, 1, the critical value of T ϕ 1 n θ ϕ may be approximated by χ m,α, the upper 100α% of the chi-square distribution with m degrees of freedom, i.e., reject the hypothesis of symmetry iff T ϕ 1 n θ ϕ χ m,α 15 Now we are going to analyze the power of the test. Let q = q 11,..., q II T be a point at the alternative hypothesis, i.e., there exist at least two indexes i and j for which q ij q ji. We denote by θ ϕ a the point on Θ verifying where Θ 0 is given by and with It is clear that θ ϕ a = arg mind ϕ q, pθ θ Θ 0 Θ 0 = {θ Θ : θ = gβ for some β B} θ ϕ a = f ij q; i, j = 1,..., I, i, j I, I T p θ ϕ a = f ij q; i, j = 1,..., I, T fq f II q = 1 i,j I,I f ij q The notation f ij q indicates that the elements of the vector θ ϕ a power-divergence family ϕ λ x we have depend on q. For instance, for the We also denote and then f ij q = q ij + q 1 ji, i, j = 1,..., I q ij + q 1 ji θ S,ϕ = p S,ϕ ij ; i, j = 1,..., I, i, j I, I T p θ S,ϕ = p S,ϕ ij ; i, j = 1,..., I T f p where f = f ij ; i, j = 1,..., I T. If the alternative q is true we have that p tends to q and p θ S,ϕ to pθ ϕ a in probability. If we define the function Ψ ϕ1 q = D ϕ1 q,fq we have Ψ ϕ1 p = Ψ ϕ1 q + J D ϕ1 q, fq p ij q ij + o p q q ij

7 Symmetry 010, 1114 Then the random variables n Dϕ1 p, f p D ϕ1 q, fq and n J have the same asymptotic distribution. If we define D ϕ1 q,gq q ij p ij q ij l ij = D ϕ 1 q, fq q ij 16 and l = l ij ; i, j = 1,..., I T, we have ndϕ1 p,f p L D ϕ1 q, fq N n 0,lT Σ q l 17 where Σ q = diagq qq T. If we consider the maximum likelihood estimator instead of minimum ϕ-divergence estimator, we get l T Σ q l = q ij m ϕ 1 ij q ij m ϕ 1 ij where m ϕ 1 ij = 1 ϕ q ij + q ji ϕ 1 qij qji q ij + q ji + q ij ϕ qij 1 ϕ qji 1 q ij + q ji q ij + q ji q ij + q ji It is also interesting to observe, if we consider the power divergence measure, that m λ 1 pij pji ij = + λλ + 1 p ij + p ji p ij + p ji + 1 λ λ p ji pij pji λ p ij + p ji p ij + p ji p ij + p ji For λ 0 and λ = 1 we get m 0 ij = log p ij and m 1 ij = p ij 3p ji + p ij p ji p ij + p ji p ij + p ji respectively. Therefore, the corresponding asymptotic variances are given by σ0 p ij = p ij log p ij p ij + p ji p ij log p ij + p ji and σ 1 = p ij 3p ji + p ij p ji p ij p ij + p ji Based on the previous result we can formulate the following theorem. p ij p ij 3p ji + p ij p ji p ij + p ji

8 Symmetry 010, 1115 Theorem The asymptotic power for the test given in 15, at the alternative q, is given by 1 ϕ 1 1 β n,ϕ1,ϕ q = 1 Φ n l T Σ q l n χ m,α nd ϕ1 q, fθ ϕ a where Φ n x is a sequence of distributions functions tending uniformly to the standard normal distribution function Φ x. We consider a contiguous sequence of alternative hypotheses that approaches the null hypothesis H 0 : θ = pgβ, for some unknown β B, at the rate O n 1/. Consider the multinomial probability vector p n,ij = p ij gβ + d ij n 1/, i = 1,..., I, j = 1,..., I where d = d 11,..., d II T is a fixed I 1 vector such that I I d ij = 0, recall that n is the total count parameter of the multinomial distribution and β B. As n, the sequence of multinomial probabilities {p n } n N with p n = p n,ij, i = 1,..., I, j = 1,..., I T, converges to a multinomial probability in H 0 at the rate of O n 1/. Let H 1,n : p n = pgβ + dn 1/, β B 18 In the next theorem we present the asymptotic distribution of the family of test statistics T ϕ 1 n θ ϕ defined in 13, under the contiguous alternative hypotheses given in 18. Theorem 3 Under H 1,n, given in 18, the family of test statistics T ϕ 1 n θ ϕ is asymptotically noncentrality chi-squared distributed with I I 1 / degrees of freedom and noncentrality parameter δ = 1 d ij p ij i<j d ij d ji p ij Proof. See Chapter 8 in [1]. An interesting a simulation study can be seen in [7]. In that study some interesting alternative test statistics appear to the classical Pearson test statistics and likelihood ratio test. 4. Hypothesis Testing: H 0 versus H 1 and H 1 versus H In this section we consider the three hypotheses, H 0, H 1, H given in 1, 4, 5 respectively and some test statistics based on ϕ-divergence D ϕ p, p 0 and D ϕ p, p 1 19 for testing H 0 against H 1 and H 1 against H. In the expression 19, p is the maximum likelihood estimator MLE of p given by p = [ p ij ], where p ij = n ij /n; and p 0 and p 1 denote the MLEs of p under H 0 and H 1 respectively. These MLEs were obtained by [8]. Let θ ij = p ij p ij + p ji, for i > j

9 Symmetry 010, 1116 then H 0 : θ ij = 1/ for i > j and H 1 : θ ij 1/ for i > j, and θ ij = n ij n ij + n ji, θ 0 ij = 1/ and θ 1 ij = max nij, 1 n ij + n ji for i > j It follows that p 0 and p 1 are given by Then we have and p 0 ij D ϕ p, p 1 = = n ij + n ji n D ϕ p, p 0 = i>j and p 1 ij i>j n ij + n ji n = n ij + n ji n n ij + n ji n nij max, 1 n ij + n ji ϕ θ ij + ϕ1 θ ij θij θ 1 ij ϕ θ θ 1 ij ϕ 1 θ ij 1 ij 1 θ ij To solve the problem of testing H 1 against H, [8] consider the likelihood ratio test statistic T 1 = i>j n ij ln θ ij + n ji ln1 θ ij n ij ln θ 1 ij n 1 ji ln1 θ ij This statistic is such that T 1 = nd KL p, p 1 0 where D KL p, p 1 is the Kullback Leibler divergence given by 0 with ϕx = ϕ 0 x defined above. Then the likelihood ratio test statistic is based on the closeness, in terms of the Kullback Leibler divergence measure, between the probability distributions p and p 1. Thus, one could measure the closeness between the two probability distributions using a more general divergence measure if we are able to obtain its asymptotic distribution. One appropriate family of divergence measures for that purpose is the ϕ-divergence measure. As a generalization of the test statistic given in 0 for testing H 1 against H we introduce the family of test statistics T ϕ 1 = n ϕ 1 D ϕ p, p 1 1 To test H 0 against H 1, El Barmi and Kochar [8] consider the likelihood ratio test statistic T 01 = i>j 1 1 n ij ln θ ij + n ji ln1 θ ij n ij ln 1 n ji ln 1 It is clear that T 01 = n D KL p, p 0 D KL p, p 1 As a generalization of this test statistic we consider in this paper the family of test statistics T ϕ 01 = n D ϕ ϕ p, p 0 D ϕ p, p 1 1

10 Symmetry 010, 1117 If ϕ = ϕ 0 then T ϕ 1 = T 1 and T ϕ 01 = T 01, and hence the families of test statistics T ϕ 1 and T ϕ 01 can be considered as generalizations of the test statistics T 1 and T 01 respectively. In order to get the asymptotic distribution of the test statistics given in 1 and, we first define the so-called chi-bar squared distribution with n degrees of freedom, denoted by χ n. Definition 4 Let U = max0, Z where Z N 0, 1, so that the c.d.f. of U is given by { Φu, u > 0 F U u = 0, u < 0 where Φ denotes the standard normal cumulative distribution function. Let V = n k=1 U k, where U 1,..., U n are independent and distributed like U, then V χ n. It is readily shown that EV = 1 n and VarV = 5 4 n This distribution is related to χ distribution. It can be readily shown that PrV > v = n l=0 n 1 l n Prχ l > v by conditioning on L, the number of non-zero U i s. Furthermore, like the χ n distribution, the χ n distribution is stochastically increasing with n. If V χ n and V χ n, where n < n, then V is stochastically smaller than V : PrV > t PrV > t. This follows since V V + W where W χ n n with V and W independent. For more details about the chi-bar squared distribution see [17]. The following theorem present the asymptotic distribution of T01. ϕ Theorem 5 Under H 0, T ϕ 01 L χ K as n where K = II 1/. Proof. See [7]. If we consider the family of power divergences given in 8, we have the power divergence family of test statistics defined as T λ 01 = T ϕ λ 01 which can be used for testing H 0 against H 1. Therefore some important statistics can now be expressed as members of the power divergence family of test statistics T01, λ that is, T01 1 is the Pearson test statistic X01, T 1/ 01 is the Freeman Tukey test statistic F01, T01 is the Neyman-modified test statistic NM01, T01 1 is the modified loglikelihood ratio test statistic NG 01, T01 0 is the loglikelihood ratio test statistic G 01 introduced by [8] and T /3 01 is Cressie Read test statistic see [11]. Theorem 6 Under H 1, T ϕ L 1 χ M as n, where M is the number of elements in the set {i, j : i > j, p ij = p ji } K = 1 II 1 and lim n PrT ϕ 1 t K l=0 K 1 l K Prχ l t

11 Symmetry 010, 1118 If we consider the family of power divergences given in 8, we have the power divergence family of test statistics defined as T λ 1 = T ϕ λ 1 which we can use for testing H 1 against H. Remark 7 In the same way as previously we can obtain the test statistics T1 0 = G 1, T1 1 = NG 1, T1 1 = X1, T 1/ 1 = F1, T1 = NM1 and T /3 1. We will refer here to the example of [18, Section 9.5], where the test proposed by Bowker [1] is applied. The proposed tests in this paper may be used in the situation such that it is hoped that a new formulation of a drug will reduce some side-effects. Example We consider 158 patients who have been treated with the old formulation and records are available of any side-effects. We might now treat each patient with the new formulation and note incidence of side-effects. Table 1 shows a possible outcome for such an experiment. Do the data in Table 1 provide any evidence regarding a less severity of side-effects with the new formulation of the drug? BA The two test statistics given in 1 and are appropriate for this problem. For Table 1. Side-effect levels for old and new formulation. Side-effect levels-new formulation None Slight Severe Total Side-effect None levels-old Slight formulation Severe Total the test statistic T ϕ 01 given in the null hypothesis is that for all off-diagonal counts in the table the associated probabilities are such that all p ij = p ji. The alternative is that p ij p ji for all i j. We have computed the members of the family {T λ 01} given in Remark 7 and the corresponding asymptotic p-values P λ 01 = Pr χ 3 > T λ 01 which are given in the following table: Table. Asymptotic p-values for T λ 01. λ 1 1/ 0 /3 1 T01 λ P01 λ On the other hand, if we consider the usual Pearson test statistic X, we have that the value of this statistic is In this case using the chi-squared distribution with 3 degrees of freedom, the corresponding asymptotic distribution found by Bowker [1], Prχ 3 > X = Then for all

12 Symmetry 010, 1119 the considered statistics there is evidence of a differing incidence rate for side-effects under the two formulations, moreover this difference is towards less severe side effects under the new formulation. Therefore, the two considered tests lead to the same conclusion: There is a strong evidence of a bigger incidence rate for side-effects under the old formulation. The conclusion obtained in [18] is in accordance with our conclusion. Acknowledgements This work was supported by Grants MTM and BSCH-UCM References 1. Bowker, A. A test for symmetry in contingency tables. J. Am. Statist. Assoc. 1948, 43, Ireland, C.T.; Ku, H.H.; Koch, G.G. Symmetry and marginal homogeneity of an r r contingency table. J. Am. Statist. Assoc. 1969, 64, Quade, D.; Salama, I.A. A note on minimum chi-square statistics in contingency tables. Biometrics 1975, 31, Menéndez, M.L.; Pardo, J.A.; Pardo, L. Tests based on ϕ-divergences for bivariate symmetry. Metrika 001, 53, Menéndez, M.L.; Pardo, J.A.; Pardo, L. Tests for bivariate symmetry against ordered alternatives in square contingency tables. Aust. N. Z. J. Stat. 003, 45, Menéndez, M.L.; Pardo, J.A.; Pardo, L. Tests of symmetry in three-dimensional contingency tables based on phi-divergence statistics. J. Appl. Stat. 004, 31, Menéndez, M.L.; Pardo, J.A.; Pardo, L.; Zografos, K. On tests of symmetry, marginal homogeneity and quasi-symmetry in two contingency tables based on minimum ϕ-divergence estimator with constraints. J. Stat. Comput. Sim. 005, 75, El Barmi, H.; Kochar, S.C. Likelihood ratio tests for symmetry against ordered alternatives in a square contingency table. Stat. Probab. Lett. 1995,, Csiszàr, I. Eine Informationstheorestiche Ungleichung und ihre Anwendung auf den Beweis der Ergodizität von Markoffschen Ketten. The Mathematical Institute of Hungarian Academy of Sciences: Budapest, Hungary, 1963; Volume 8, pp Ali, S.M.; Silvey, S.D. A general class of coefficients of divergence of one distribution from another. J. Roy. Stat. Soc. Ser. B Stat. Met. 1966, 8, Cressie, N.; Read, T.R.C. Multinomial goodness-of-fit tests. J. Roy. Stat. Soc. Ser. B Stat. Met. 1984, 46, Pardo, L. Statistical Inference Based on Divergence Measures; Chapman & Hall/CRC: New York, NY, USA, Kullback, S. Information Theory and Statistics; John Wiley: New York, NY, USA, Kullback, S. Marginal homogeneity of multidimensional contingency tables. Ann. Math. Stat. 1971, 4, Morales, D.; Pardo, L.; Vajda, I. Asymptotic divergences of estimates of discrete distributions. J. Stat. Plan. Infer. 1995, 48,

13 Symmetry 010, Pardo, J.A.; Pardo, L.; Zografos, K. Minimum ϕ-divergence estimators with constraints in multinomial populations. J. Stat. Plan. Infer. 00, 104, Robertson, T.; Wrigt, F.T.; Dykstra, R.L. Order Restricted Statistical Inference; Wiley: New York, NY, USA, Sprent, P. Applied Nonparametric Statistical Methods; Chapman & Hall: London, UK, c 010 by the authors; licensee MDPI, Basel, Switzerland. This article is an Open Access article distributed under the terms and conditions of the Creative Commons Attribution license