Expectation of Quadratic Forms in Normal and Nonnormal Variables with Econometric Applications

Transcription

1 Expectation of Quadratic Forms in Normal and Nonnormal Variables with Econometric Applications Yong Bao y Department of Economics Temple University Aman Ullah z Department of Economics University of California, Riverside August 20, 2007 ABSTRACT We derive some new results on the expectation of quadratic forms in normal and nonnormal variables. Using a nonstochastic operator, we show that the expectation of the product of an arbitrary number of quadratic forms in normal variables with nonzero mean follows a recurrence formula. The formula includes the existing result for normal variables with zero mean as a special case. For nonnormal variables, while the existing results are available only for quadratic forms of order up to 3, we derive analytical results for quadratic form of order 4 and half quadratic form of order 3. The results involve the cumulants of the nonnormal distribution up to the eighth order for order 4 quadratic from, and up to the seventh order for order 3 half quadratic from. Setting all the nonnormality parameters equal to zero gives the results for normal vectors. Some econometric applications using our normal and nonnormal results are presented. Keywords: Expectation; Quadratic form; Nonnormality JEL Classi cation: C0; C9 We are grateful to Peter Phillips (the editor) and three anonymous referees for their constructive and detailed feedback that greatly improved the paper. We are also thankful to Jason Abrevaya, Fathali Firoozi, and seminar participants at Purdue University for their comments and suggestions. All remaining errors are our own. y Department of Economics, Temple University, Philadelphia, PA 922, (25) , Fax: (25) , yong.bao@temple.edu. z Corresponding Author: Department of Economics, University of California, Riverside, CA, 9252, (95) , Fax: (95) , aman.ullah@ucr.edu.

2 Introduction In evaluating statistical properties of a large class of econometric estimators and test statistics we often come across the problem of deriving the expectation of the product of an arbitrary number of quadratic forms in random variables. For example, see White (957), Nagar (959), Theil and Nagar (96), Kadane (97), Ullah, Srivastava, and Chandra (983), Dufour (984), Magee (985), Hoque, Magnus, and Pesaran (988), Kiviet and Phillips (993), Smith (993), Lieberman (994a), Srivastava and Maekawa (995), Zivot, Startz, and Nelson (998), and Pesaran and Yamagata (2005); also see the book by Ullah (2004). Econometric examples of the situations where the expectation of the product of quadratic forms can arise are: obtaining the moments of the residual variance; obtaining the moments of the statistics where the expectation of the ratio of quadratic forms is the ratio of the expectations of the quadratic forms, for example, the moments of the Durbin-Watson statistic; and obtaining the moments of a large class of estimators in linear and nonlinear econometric models (see Bao and Ullah, 2007a, 2007b), among others. In view of this econometricians and statisticians have long been interested in deriving E( Q n i= Q i); where Q i = y 0 A i y; A i are nonstochastic symmetric matrices of dimension m m (for asymmetric A i we can always put (A i + A 0 i )=2 in place of A i); and y is an m random vector with mean vector and identity covariance matrix. where y is distributed as normal or nonnormal. We consider the cases When y is normally distributed, the results were developed by various authors, see, for example, Mishra (972), Kumar (973), Srivastava and Tiwari(976), Magnus (978, 979), Don (979), Magnus and Neudecker (979), Mathai and Provost (992), Ghazal (996), and Ullah (2004), where both the derivations and econometric applications were extensively studied. Loosely speaking, many of these works employed the moment generating function approach, while the commutation matrix (Magnus and Neudecker, 979) and a recursive nonstochastic operator (Ullah, 2004) were also used to derive the results. When y is not normally distributed, however, the results are quite limited and are available only for quadratic forms of low order. In some applications, for example, see Section 4.2, the expectation of half quadratic form E(y Q n i= y0 A i y), which is the product of linear function and quadratic forms, is also needed. (When y is a normal vector with zero mean, it is trivially equal to zero.) Under nonnormality, a general recursive procedure does not exist for deriving E( Q n i= y0 A i y) or E(y Q n i= y0 A i y), though for some special nonnormal distributions, including mixtures of normal, we may invoke recursive algorithms (see Section 2.3 of Ullah, 2004). The major purpose of this paper is two-fold. First, we try to derive a recursive algorithm for the expectation of an arbitrary number of products of quadratic forms in the random vector y when it is normally distributed. We are going to utilize a nonstochastic operator proposed by Ullah (2004) to facilitate the derivation. Moreover, we are going to relate this approach to the moment generating function approach and we discuss possible advantage of our recursive approach in terms of computer time. When y has zero mean, the recursive result degenerates to the result given in Ghazal (996). Secondly, we try to derive the expectations of products of quadratic form of order 4 and half quadratic form of order 3 in a general nonnormal random vector y: We express the nonnormal results explicitly as functions of the cumulants of the underlying nonnormal distribution of y: The organization of this paper is as follows. In Section 2, we discuss the normal case and in Section 3

3 we derive the nonnormal results. Section 4 gives econometric applications of the main results. Section 5 concludes. 2 The Normal Case In this section, we focus on the case of normal variables. Before we are going to derive the main results, it would be helpful for us to discuss rst the less known approach of Ullah (2004) and the more popular moment generating function approach. 2. A Nonstochastic Operator In his monograph, Ullah (2004) discussed an approach to deriving the moments of any analytic function involving a normal random vector by using a nonstochastic operator. More formally, he showed that for x N(; ); Eh (x) = h (d) = h (d) and Eh (x) g (x) = h (d) Eg (x) () for the real-valued analytic functions h () and g () ; where d is a nonstochastic operator. This result essentially follows from the fact that the density of x is an exponential function and for analytic functions, di erentiation under the integral sign is allowed. Note that to use the result () correctly, readers must be cautious to the usual caveats, as pointed out by Ullah (2004, p. 2). Most importantly, power of d should be interpreted as a recursive operation. For example, we can easily derive E(x 0 Ax) = d 0 Ad = d 0 A = 0 A+tr(A) (where tr denotes the trace operator) by applying d 0 to Ad = A given the recursive nature of d (alternatively, d 0 A d =tr(add 0 ) = 0 A+tr(A) by applying d to d 0 = 0 ): If one ignores this, d 0 Ad = d 0 A =tr(ad 0 ) =tr(ad 0 ) = 0 A; which is not correct. Given (), a recursive procedure E( Q n i= Q i) = d 0 A d E( Q n Q i) immediately follows for Q i = y 0 A i y; y N(; I). Apparently, we need to expand the operation d 0 A d on E( Q n Q i), which may become demanding when n gets larger. Ullah (2004) gave the expressions for n up to 4 using this operator d: In this paper we take a step further to expand d 0 A d E( Q n Q i) explicitly. As a consequence, a recursive formula is derived for E( Q n i= Q i) for any n. 2.2 The Moment Generating Function Alternatively, it is possible to write down the joint moment generating function of Q i ; i = ; ; n; M Q (t) = E[exp( t i Q i )] = E[exp(y 0 By)]; (2) where t = (t ; ; t n ) 0 and B = P n i= t ia i : Since y is multivariate normal, it can be easily seen M Q (t) = exp 2 0 (I 2B) I ln ji 2Bj : 2 i= For t su ciently close to zero, by expanding (I 2B) and ln ji 2Bj =tr[ln(i 2B)]; one can write ( X M Q (t) = exp 2 ) i 0 B i + 2i 2 i tr(bi ) : (3) i= 2

4 Then using Wilf s (994) notation, E( Q n i= Q i) = [t t 2 t n ]M Q (t): 2 Equivalently n X E( Q i ) = exp 2 ) i 0 B i n 2 i tr(bi ) ; (4) t=0 i= which is accessible via Faà di Bruno s formula (see, e.g., Barndo -Nielsen and Cox, 989). i= Modern symbolic algebra packages such as Maple and Mathematica can readily produce coe cients of this type. Here we do not claim that using the nonstochastic operator has any conceptual advantage over the moment generating function. However, we should point out that the computerized output may not necessarily reveal the recursive nature of the algorithm in calculating E( Q n i= Q i): In terms of numerical programming and applications, this has strong implications. Based on [t t 2 t n ]M Q (t); the computer may quickly produce algebraic expressions in (4), whose number of terms can on the other hand explode also very quickly as n goes up. Then reevaluating these algebraic terms numerically given and A i may be far time consuming, especially when m and n are both large. But if one programs the recursive formula for E( Q n i= Q i) (see (7) in the next subsection) derived using the nonstochastic operator, then computational time can be substantially saved since the in-between results E( Q p i= Q i); p < n can be saved numerically. This will be demonstrated more clearly in the next subsection. Ghazal (996) found a recursive relationship for the case of zero mean normal random vector by following the expansion (4). In this next subsection, we derive results for the more general case of y N(; I) by using Ullah s (2004) nonstochastic operator. 2.3 A Recursive Procedure We rst introduce two lemmas regarding the gradient vector and the Hessian matrix of E( Q n i= Q i) with respect to the mean vector : They are needed when we later try to expand d 0 A d E( Q n Q i): For the starting case when n = ; E(Q ) = 0 A +tr(a ); as shown before. In the following derivations, for multiple summations, the indices are not equal to each other, e.g., in Lemma, j 6= j 2 6= 6= j i in P n j = P n j i= : We also write y = + ": Lemma : The gradient of E( Q n i= Q i) with respect to Q n i= Q i) i= j = j i= 2 i Q Q n A j A ji E Q j Q ji (5) Proof: Change the order of derivative and expectation signs, and use successively the operator (), then the result follows. It should be noted that in (5), the products involved in A j A ji and QQn Q j Q ji are assumed to be one when i < 2 for the former or when i > n for the latter, following the standard convention that an empty product is to be interpreted as one. For )=@ = 2A Q 2 )=@ = 2[A E(Q 2 ) + A 2 E(Q )] + 4(A A 2 + A 2 A ): 3

5 Lemma 2: The Hessian of E( Q n i= Q i) with respect to 2 E( Q n i= Q = 2 Q Q n A j E + 4 j= Q j j= k= E A j yy 0 Q Q n A k : (6) Q j Q k Proof: Using Lemma, direct derivative rules give the result Theorem: The expectation of Q n i= Q i is given by the following recursion X n E( Q i ) = 2 i i= i=0 j =2 j Q2 Q n g jj i E ; (7) Q j Q ji where g jj i = 0 (A A j A ji + A j A A j2 A ji + + A j A ji A )+tr(a A j A ji ): Proof: Using () and replacing y 0 A y with d 0 A d; E( Q i ) = d 0 A d E( Q i ) i= " # = tr A dd 0 E( Q i ) "! Q = tr A d 0 E( Q i ) + A d Q # 0 = A E( Q i ) + tr A 0 E( Q n Q + tr n "!# = 0 A E( Q i ) + tr A E( Q i ) n Q i) +tr n 0 + tr A 2 E( Q n Q = 0 A E( Q i ) + tr(a )E( X n = E(Q )E( Q i ) + 2 i+ +2 i= j =2 Q2 Q n tr(a A j )E j=2 Q j X n = E(Q )E( Q i ) + 2 i+ +2 j=2 i= Q j j =2 Q i ) + 2 Q n A Q + 4 j Q + tr A 2 E( Q n + tr(a E( Q n Q 0 Q2 Q n A A j A ji E Q j Q ji E y 0 A j A A k y Q 2 Q n Q j=2 j Q k k=2 0 Q2 Q n A A j A ji E Q j Q ji j Q2 Q n tr(a A j )E + 4 E j=2 k=2 j<k y 0 (A j A A k + A k A A j ) y Q 2 Q n Q j Q k Q ; 4

6 where the second last equality follows by substituting the results from Lemmas and 2 and noting that E(Q ) = 0 A +tr(a ), and the last equality follows by noting that A j A A k is not symmetric, and we replace it with (A j A A k + A k A A j ) =2; and that the two indices j and k have symmetric roles. Note that in the last equality there is a term E[y 0 (A j A A k + A k A A j ) y Q2Qn Q jq k ]; which is a quadratic form of order n 2: Upon successive substitution, result (7) follows. Note that in (7), as before, an empty product in g jj i and Q2Qn particular, when i = 0; g jj i = 0 A +tr(a ) = E(Q ); we have the following results for n up to 4, Q 2Q n Q j Q ji Q j Q ji is to be interpreted as one. In = Q 2 Q n : Now applying the theorem, E( Q i= Q i) = 0 A +tr(a ); E( Q 2 i= Q i) = E(Q )E(Q 2 ) A A 2 + 2tr(A A 2 ); E( Q 3 i= Q i) = E(Q )E(Q 2 Q 3 )+[4 0 A A 2 +2tr(A A 2 )]E(Q 3 )+[4 0 A A 3 +2tr(A A 3 )]E(Q 2 )+8 0 A A 2 A A A 3 A A 2 A A 3 + 8tr(A A 2 A 3 ); E( Q 4 i= Q i) = E(Q )E(Q 2 Q 3 Q 4 ) + [4 0 A A 2 + 2tr(A A 2 )]E(Q 3 Q 4 ) + [4 0 A A 3 + 2tr(A A 3 )]E(Q 2 Q 4 ) +4[ 0 A A 4 + 2tr(A A 4 )]E(Q 2 Q 3 ) + (8 0 A A 2 A A A 3 A 2 )E(Q 4 ) +(8 0 A A 2 A A A 4 A 2 )E(Q 3 ) + (8 0 A A 3 A A A 4 A 3 )E(Q 2 ) +6 0 A A 2 A 3 A A A 2 A 4 A A A 3 A 2 A A A 3 A 4 A A A 4 A 2 A A A 4 A 3 A A 2 A A 3 A A 3 A A 2 A A 2 A A 4 A A 4 A A 2 A A 3 A A 4 A A 4 A A 3 A 2 +6tr(A A 2 A 3 A 4 ) + 6tr(A A 2 A 4 A 3 ) + 6tr(A A 3 A 2 A 4 ): Upon substitution, we have the results for expectations of quadratic forms up to order 4 as given in Ullah (2004). Note that the above results can alternatively follow from (4). For example, consider n = 3 so that B = t A + t 2 A 2 + t 3 A 3 : Using Faà di Bruno s formula, 3 E( Q i ) = 3 3 u u @t 3 2 } {z t=0 } Part I Part @t 3 ; {z t=0 } Part III where u = P h i 2 i= 2 i 0 B i + 2i i tr(bi ) : Obviously, Part I = 8 0 A A 2 A A A 3 A A 2 A A 3 + 8tr(A A 2 A 3 ); Part II = [4 0 A 2 A 3 + 2tr(A 2 A 3 )]E(Q ) + [4 0 A A 3 + 2tr(A A 3 )]E(Q 2 ) +[4 0 A A 2 + 2tr(A A 2 )]E(Q 3 ); Part III = E(Q )E(Q 2 )E(Q 3 ); so Part I + Part II + Part III gives the same expression for E( Q 3 i= Q i); as derived previously using the recursive formula (7). In terms of numerical calculation, we notice that the main di erence is that Faà di 5

7 Bruno s formula breaks E( Q 3 i= Q i) into functions of single quadratic forms E(Q ); E(Q 2 ); and E(Q 3 ); but the recursive formula (7) breaks E( Q 3 i= Q i) into functions of quadratic forms of lower orders (2 and ). (Interested readers may try with any n and can verify that Faà di Bruno s formula always breaks the result into single quadratic forms, whereas the recursive formula (7) breaks the result into quadratic forms of lower orders.) In terms of computer time, evaluating E(Q )E(Q 2 )E(Q 3 ) + [ 0 A 2 A 3 + 2tr(A 2 A 3 )]E(Q ) (needed using Faà di Bruno s formula) may be more time consuming than evaluating E(Q )E(Q 2 Q 3 ) (needed using (7)). The reason is as follows. To evaluate E(Q )E(Q 2 Q 3 ); E(Q 2 Q 3 ) can be calculated from a subroutine and once it is calculated, only a scalar return value is needed and all matrices (A 2 and A 3 ) involved can be cleared out from computer memory, but in evaluating E(Q )E(Q 2 )E(Q 3 ) + [ 0 A 2 A 3 + 2tr(A 2 A 3 )]E(Q ); all the matrices are not cleared out of computer memory until the calculations are nished. This e ect will be most apparent when both m and n are large. When A A = A 2 = = A n ; the result in Theorem gives the nth moment (about zero) of the quadratic form y 0 Ay; as indicated in the following corollary, which obviously follows from (7). Corollary : The nth moment of y 0 Ay for y N (; I) is given by the following recursion where E (y 0 Ay) 0 = ; g i = n i E[(y 0 Ay) n ] = g i E (y 0 Ay) n i ; (8) i=0 2 i i![tr(a i+ ) + (i + ) 0 A i+ ]: When = 0; the above theorem degenerates to the result of Ghazal (996), as given in Corollary 2. Corollary 2: The expectation of Q n i= Q i when y N (0; I) is given by the following recursion E( Q i ) = E(Q ) E( Q i ) + 2 E(y 0 A j A y y 0 A 2 y y 0 A j i= j=2 y Q k ): (9) k=j+ Proof: Comparing the proof of (7) and (9), we need to show that, when = 0; is in fact equal to U n j=2 W n j=2 Q2 Q n tr(a A j )E + 2 Q j E y 0 A j A y Q 2 Q n Q j j=2 i6=j E y 0 A i A A j y Q2 Q n It is enough to show that the jth summands of W n and U n are equal. This can be proved straightforwardly by induction. Q i Q j : 6

8 3 The Nonnormal Case Now suppose y = (y ; ; y m ) 0 follows a general error distribution with an identity covariance matrix. In general, we can write E( Q i ) = E( n i=q i ) i= = trfe[ n i=(yy 0 A i )]g = trfe[ n i=(yy 0 )]( n i=a i )g = trfe[(y )(y ) 0 ]A g; (0) where denotes and Kronecker product symbol and y = y y y (n terms) and A = n i= A i: Thus, under a general error distribution, the key determinant of the expectation required is the set of product moments of the y i s that appear in the m n m n matrix P = (y )(y ) 0 ; the elements of which are products of the type Q m i= y(i) i, where the nonnegative integers satisfy P m i= (i) = 2n; i.e. they are a composition of 2n with m parts. Numerically, on can always write a computer program to calculate the expectation. However, when A i are of high dimension, it may not be practically possible to store an m n m n matrix P without torturing the computer. So it may be more promising if we can work out some explicit analytical expressions. More importantly, analytical expressions can help us understand explicitly the e ects of nonnormality on the nite sample properties of many econometric estimators, an example of which is provided in the next section. To facilitate our derivation, suppose now that the mean vector = 0 and y i is IID. As it turns out, as n goes up, the work is more demanding. In the literature, results are only available for n up to three, see Chandra (983), Ullah, Srivastava, and Chandra (983), and Ullah (2004). So now we take one step further to derive the analytical result for n = 4: From the analysis in the previous paragraph, when n = 4; the highest power of y i in P is 8. So we assume that under a general error distribution, y i has nite moments m j = E(y j i ) up to the eighth order: m = 0; m 2 = ; m 3 = ; m 4 = 2 + 3; m 5 = ; m 6 = ; m 7 = ; m 8 = ; () where and 2 are the Pearson s measures of skewness and kurtosis of the distribution and these and 3 ; ; 6 can be regarded as measures for deviation from normality. For a normal distribution, the parameters ; ; 6 are all zero. Note that these s can also be expressed as cumulants of y i, e.g., and 2 represent the third and fourth cumulants. For Q m i= y(i) i = y () ym (m) with () + + (m) = 2n = 8; we put Q m i= y(i) i = y i y i8 ; which has nonzero expectation only in the following seven situations:. All the eight indices i ; ; i 8 are equal. 7

9 2. The eight indices consist of two di erent groups, with two equal indices in the rst group and six equal indices in the second group, e.g., i = i 2 ; i 3 = i 4 = = i 8 ; i 6= i 3 : 3. The eight indices consist of two di erent groups, with three equal indices in the rst group and ve equal indices in the second group, e.g., i = i 2 = i 3 ; i 4 = i 5 = = i 8 ; i 6= i 4 : 4. The eight indices consist of two di erent groups, with four equal indices in each group, e.g., i = i 2 = i 3 = i 4 ; i 5 = j 6 = i 7 = i 8 ; i 6= i 5 : 5. The eight indices consist of three di erent groups, with two equal indices in the rst group, two equal indices in the second group, and four equal indices in the third group, e.g., i = i 2 ; i 3 = i 4 ; i 5 = i 6 = i 7 = i 8 ; i 6= i 3 6= i 5 : 6. The eight indices consist of three di erent groups, with two equal indices in the rst group, three equal indices in the second group, and three equal indices in the third group, e.g., i = i 2 ; i 3 = i 4 = i 5 ; i 6 = i 7 = i 8 ; i 6= i 3 6= i 6 : 7. The eight indices consist of four di erent groups, with two equal indices in each group, e.g., i = i 2 ; i 3 = i 4 ; i 5 = i 6 ; i 7 = i 8 ; i 6= i 3 6= i 5 6= i 7 : By some tedious algebra, we can write down the result for n = 4 as follows, where denotes the Hadamard product symbol: E( Q 4 i= Q i) = 2 f P i<j;k<l [tr(a i)tr(a j )tr(a k A l ) (A i A j )tr(a k A l )] +4 P k<l tr(a i)tr(a j (A k A l )) + 8 P j<l tr((i A i)a j A k A l ) + 6 P k<l 0 (I (A A j ))(I (A k A l ))g + 4 f P j<k<l tr(a i)tr(a j A k A l ) + 4 P i<j;k<l tr[a i A j (A k A l )]g + 6 tr(a A 2 A 3 A 4 ) +2 2 f P j<l tr(a i) 0 (I A j )A k (I A l )+2 P j<k 0 (I A i )A j A k (I A l )+2 P j<k<l tr(a i) 0 (A j A k A l ) +4 P i<j 0 (A i A j )A k (I A l ) + 8 P i<l;j<k tr[a i(a j A k )A l ]g + 2 2[ P k<l tr(a A j )tr(a k A l ) +4 P i<l;j<k 0 (I A i )(A j A k )(I A l )+8 0 (A A 2 A 3 A 4 )]+2 3 [ P k<l 0 (I A i )A j (I A k A l ) +4 P j<k<l 0 (I A i )(A j A k A l )] + Q 4 i= tr(a i) + 2 P i<j;k<l tr(a i)tr(a j )tr(a k A l ) +4 P k<l tr(a A j )tr(a k A l ) + 8 P j<k<l tr(a i)tr(a j A k A l ) + 6 P j<l tr(a A j A k A l ) ; where all the summations are over the indices under the summation signs running from to 4, unequal to each other, and sometimes with inequality restrictions. For example, P i<j;k<l indicates summing over the four indices i; j; k; l from to 4; i 6= j 6= k 6= l; i < j; and k < l: Note that sometimes we may also be interested in the expectations of a half quadratic form y Q n i= y0 A i y; which is the product of linear function and quadratic forms. Let E(y Q n i= y0 A i y) have a representative element E(y j Q n i= y0 A i y); j = ; ; m: Then following (0), E(y j n Y i= y 0 A i y) = trfe[y j (y )(y ) 0 ]A g: Analytical results are available for n up to 2 in the literature. Now we focus on the case when n = 3 for Q m the half quadratic form. For y j i= y(i) i Q m y j i= y(i) i = y j y () ym (m) with () + + (m) = 2n = 6; we put = y j y i y i6 ; which has nonzero expectation only in the following four situations: 8

10 . All the seven indices j; i ; ; i 6 are equal. 2. The seven indices consist of two di erent groups, with two equal indices in the rst group and ve equal indices in the second group, e.g., j = i ; i 2 = i 3 = = i 6 ; j 6= i 2 ; or i = i 2 ; j = i 3 = i 4 = = i 6 ; i 6= j: 3. The seven indices consist of two di erent groups, with three equal indices in the rst group and four equal indices in the second group, e.g., j = i = i 2 ; i 3 = i 4 = = i 6 ; j 6= i 3 ; or i = i 2 = i 3 ; j = i 4 = i 5 = i 6 ; i 6= j: 4. The seven indices consist of three di erent groups, with two equal indices in the rst group, two equal indices in the second group, and three equal indices in the third group, e.g., j = i ; i 2 = i 3 ; i 4 = i 5 = i 6 ; j 6= i 2 6= i 4 ; or i = i 2 ; i 3 = i 4 ; j = i 5 = i 6 ; i 6= i 3 6= j: By some tedious algebra, we can write down the result for the half quadratic form when n = 3 : E(y Q 3 i= y0 A i y) = 5 (I A A 2 A 3 ) + 3 Pj<k f4[i A i (A j A k )] + 2A i (I A j A k ) +tr(a i )(I A j A k )g + f4 P j<k [tr(a i)(i (A j A k )) + 2A i (A j A k )] + 8 P i<k [I (A ia j A k )] + P i<j [tr(a i)tr(a j ) (I A k )+2 0 (A i A j )(I A k )] + 2 P [2A i A j (I A k )+tr(a i )A j (I A k )]g + 2 f8(a A 2 A 3 ) + P i<j [4(A i A j )(I A k )+tr(a i A j )(I A k )] + 2 P (I A i )A j (I A k )g; where all the summations are over the indices under the summation signs running from to 3, unequal to each other, and sometimes with inequality restrictions. For example, P i<j i; j; k = ; 2; 3; i 6= j 6= k; and i < j. stands for summation over Note that setting ; ; 6 all equal to zero, then the result for quadratic form of order 4 derived in this section degenerates into the result under normality as given in Ullah (2004), and trivially, the result for half quadratic form of order 3 degenerates to zero. 4 Econometric Applications Now we give some applications of the results for normal and nonnormal variables from the previous sections. We consider the ratio of normal quadratic forms whose expectation is equal to the ratio of expectations. We also study the e ects of nonnormality on the approximate mean squared error (MSE) of the estimator in a simple time series model. 4. Moments of Ratios of Quadratic Forms For an m-variate normal vector y N (0; I) ; consider the ratio of quadratic form R = y 0 By=y 0 Ay: 3 general, the moments of this ratio is not equal to the ratio of moments. However, with some restrictions put on the symmetric matrices A and B; we can have a separation result " y 0 # k By E y 0 = E[(y0 By) k ] Ay E[(y 0 Ay) k ] : (2) The conditions are that A and B are of rank r m; commutative, AB 6= 0; and A is idempotent (see Ghazal, 994). The numerator and denominator in (2) can be easily evaluated using the normal result (9). For A In 9

11 being idempotent, we can verify E[(y 0 Ay) k ] = 2 k (r=2 + k) = (r=2) = r(r + 2) (r + 2(k )): In fact, we can generalize the separation result (2) to the case " Qq # i= E (y0 B i y) ki (y 0 Ay) k = h Qq i E i= (y0 B i y) ki E (y 0 Ay) k ; (3) where k = P q i= k i; and each pair (A, B i ) satis es the same conditions as (A; B) in (2). This can be proved by following Sawa (972) s result, " Qq # i= E (y0 B i y) ki (y 0 Ay) k = Z " # 0 ( t) k+kq =2 jsj k q t ==t q=0 h Qq i where S = I 2tA 2t B 2t q B q : Since E[(y 0 Ay) k ] = 2 k (r=2 + k) = (r=2) and E i= (y0 B i y) ki = h k+kq jj =2 =@t q for = I 2t B 2t q B q ; and also noting that t ==t q=0 Z 0 to show (3), it is su cient to k+kq =2 q ( t ) k ( 2t ) (r=2+k) = t==t q=0 which can be shown to be true by the method of induction. (k) (r=2) 2 k (r=2 + k) ; = ( 2t) (r=2+k) q t==t q=0 The separation result (3) can be used to study the nite sample properties of the maximum likelihood estimator in the spatial autoregressive model y = W y + "; (4) where y is an T vector of observations on the dependent spatial variable, W y is the corresponding spatially lagged dependent variable for weights matrix W; which is assumed to be known a priori, 2 ( ; ; ) is the spatial autoregressive parameter, and " is the vector of IID N(0; 2 ) error terms. To estimate ; the concentrated likelihood function (by concentrating out 2 ) is L() = T P T i= [ln jaj ln(2y0 Dy=n)=2 n=2]; where A = I W; D = I (W +W 0 )+ 2 W 0 W: Since y = A "; then immediately we can write down the score function as well as its higher order derivatives in terms of B i i ln jaj=@ i and (" 0 M ") i (" 0 M 2 ") j =(" 0 ") i+j ; where M i = A 0 (@ i D=@ i )A : For = n B 2 (" 0 M ")=(" 0 2 L()=@ 2 = n B 2 2 (" 0 M 2 ")=(" 0 ") + 2 (" 0 M ") 2 =(" 0 ") 2 3 L()=@ 3 = n B 3 + :5(" 0 M ")(" 0 M 2 ")=(" 0 ") 2 (" 0 M ") 3 =(" 0 ") 3 ; where B = tr(a W ); B 2 = tr[(a W ) 2 ]; B 3 = 2tr[(A W ) 3 ]: As such, Bao and Ullah (2007a) showed that the approximate bias and MSE of the maximum likelihood estimator ^ can be expressed in terms of E[(" 0 M ") i (" 0 M 2 ") j =(" 0 ") i+j ]: They followed the top-order invariant polynomial approach of Smith (989) to work out the expectations. It is obvious that the separation result (3) could have been used by setting A = I and B = M ; M 2 : By using (7), we can easily check that the two methods yield the same results as given in Bao and Ullah (2007a). 0

12 4.2 E ects of Nonnormality on the MSE in AR Model Given the results under nonnormality from Section 3, we can study the e ects of nonnormality on the approximate bias and MSE of the OLS estimator of the autoregressive coe cient in the following AR() model with exogenous regressors y t = y t + x 0 t + " t ; t = ; ; T (5) where 2 ( ; ) ; x t is k xed and bounded so that X 0 X = O (T ), where X = (x ; ; x T ) 0 ; is k ; > 0; and the error term " t is IID and follows some nonnormal distribution with moments (). Denote " = (" ; " 2 ; ; " T ) 0 ; y = (y ; y 2 ; ; y T ) 0 ; M = I X (X 0 X) X 0. Also de ne F to be a T vector with the t-th element being t ; C to be a strictly lower triangular T T matrix with the tt 0 -th lower o -diagonal element being t t0 ; A = MC; r = M (y 0 F + CX) ; where we assume that the rst observation y 0 has been observed. Bao and Ullah (2007b) found that nonnormality a ects the approximate bias, up to O(T ); through the skewness coe cient : Consequently, for nonnormal symmetric distribution (e.g. Student-t), the bias is robust against nonnormality. Setting = 0; the result degenerates into the bias result of Kiviet and Phillips (993) under normality. The approximate MSE under nonnormality, up to O(T 2 ); however, was not available due to the absence of the results given in Section 3. Now we are ready to study the e ects of nonnormality on the MSE. From Bao and Ullah (2007b), we can express the approximate MSE of the OLS estimator ^ of the autoregressive coe cient as M (^) = 6( r 0! 0000 ) (r 0 r ) 2 8[r 0 r( ) r 0 (rr 0! A! A! ! 000 )] (r 0 r ) 3 + 3[(r0 r) 2 ( ) + 2r 0 r( ) ] (r 0 r ) 4 + 3[ r 0 rr 0 (A! A! ! 000 ) + 2(r 0 r) 2 r 0! 0000 ] (r 0 r ) 4 + 6r0 (! ! A! A! 0200 ) (r 0 r ) 4 + o(t 2 ); (6) where ijklm = 2(i+j+k+l+m) E[(" 0 rr 0 ") i (" 0 A") j (" 0 A 0 A") k (" 0 A 0 rr 0 ") l (" 0 A 0 rr 0 A") m ] and! ijklm = 2(i+j+k+l+m)+ E[" (" 0 rr 0 ") i (" 0 A") j (" 0 A 0 A") k (" 0 A 0 rr 0 ") l (" 0 A 0 rr 0 A") m ]. Obviously, expectations of quadratic forms up to order 4 ( ijklm ) and half quadratic form up to 3 (! ijklm ) are needed to evaluate (6). So numerically, it is possible for us to investigate explicitly the e ects of nonnormality on M (^) by using Section 3 (with A and A 0 rr 0 in the quadratic forms symmetrized) and the results in Ullah (2004). For the special case when x t is a vector of ones, i.e., when we have the so-called intercept model, Bao (2007) showed that one can simplify the MSE result (6) analytically to express the result explicitly in terms

13 of model parameters and the nonnormality parameters: " M (^) = 2 + T T ( 2 ) + ( )y0 # 2 + o T 2 : (7) Now only the skewness and kurtosis coe cients matter for the approximate MSE, up to O(T 2 ): The O(T ) MSE, ( 2 )=T is nothing but the asymptotic variance of ^ for the intercept model. Figures -2 plots the true (solid line) and (feasible) approximate MSE, accommodating (short dashed line) and ignoring (dotted and dashed line) the presence of nonnormality, of ^ over 0,000 simulations when the error term " t follows a standardized asymmetric power distribution (APD) of Komunjer (2007). 4 It has a closed-form density function as shown in Komunjer (2007) with two parameters, 2 (0; ); which controls skewness, and > 0; which controls the tail properties. To prevent the signal-to-noise ratio going up as we increase ; we set = : We experiment with 2 = 0:5; ; = 0:0; 0:05; = 0:5; ; T = 50: 5 To calculate the feasible approximate MSE under nonnormality, we put ^; ^ 2 ; ^ ; and ^ 2 into (6). When ignoring nonnormality, we put ^; ^ 2 ; and = 2 = 0 into (7) to calculate the approximate MSE. We use Fisher s (928) k statistics to estimate and 2 ; see Dressel (940) and Stuart and Ord (987) for the expressions of the k statistics in terms of sample moments. As is clear from the two gures, in the presence of nonnormality, (7) approximates the true MSE remarkably well for di erent degrees of nonnormality and magnitudes of the error variance. Ignoring the e ects of nonnormality produces overestimated MSE; accommodating nonnormality, (7) produces very accurate estimate of the true MSE. Lastly, when the degree of nonnormality goes down ( goes from 0.5 to ), the gap between the approximate results accommodating and ignoring nonnormality gets closer, as we can expect. 5 Conclusions We have derived a recursive formula for evaluating the expectation of the product of an arbitrary number of quadratic forms in normal variables and the expectations of quadratic form of order 4 and half quadratic form of order 3 in nonnormal variables. The recursive feature of the result under normality makes it straightforward to program and in terms of computer time, the recursive formula may have advantage over that based on the traditional moment generating function approach when the matrices have high dimension and the order of quadratic forms is large. The nonnormal results involve the cumulants of the nonnormal distribution up to the eighth order for order 4 quadratic from, and up to the seventh order for order 3 half quadratic from. Setting all the nonnormality parameters equal to zero gives the results under normality as a special case. We apply our normal and nonnormal results to two econometric problems. For the spatial autoregressive model, since the score function and its higher order derivatives can be rewritten as ratio of quadratic forms, whose expectation is equal to the ratio of expectations, the normal recursive formula derived in this paper can be applied directly to obtain the nite sample bias and MSE of the maximum likelihood estimator. 2

14 For the time series AR() model with an intercept, our numerical simulations demonstrate that in the presence of nonnormality, the analytical MSE formula approximates the true MSE of the OLS estimator of the autoregressive coe cient remarkably well. Notes For the case of a general covariance matrix ; we can write Q i = ( =2 y) 0 =2 A i =2 ( =2 y): Now =2 y has mean vector =2 and covariance matrix I: For a normal vector y with mean and covariance matrix, this normalization is innocuous for deriving the results for moments of quadratic forms in normal variables. This is also the case for the results on the rst two moments of quadratic forms in nonnormal variables. However such a normalization may invalidate the assumption (see () in Section 3) on moments of the elements of the normalized vector. For tractability and simplicity, in Section 3 we assume that the nonnormal elements are IID, as is usually the case considered for the error vector in a standard regression model 2 We thank a referee for bringing the notation of Wilf (994) to our attention. Here if = ( ; 2 ; ; n ) is a vector of nonnegative integers with jj = P n i= i; the symbol [t t2 2 tn n ] is interpreted as the coe cient of t t2 2 tn n in the expansion of the following function. 3 We note that when y is nonnormally distributed, various attempts have been made to study the approximate moments and distribution of the ratio, see Ali and Sharma (993), Lieberman (994b, 997), and Ullah and Srivastava (994), although the exact properties of the ratio under a general nonnormal density are quite di cult to derive. 4 In the experiment, we set y 0 = : However, the results are not sensitive to the choice of y 0 : For other possible xed or random start-up values, we get similar patterns for the two gures. 5 When = 0:0; as goes from 0.5 to, goes from to.9997, 2 from to ; when = 0:05; as goes from 0.5 to, goes from to.994, 2 from to References Ali, M.M. & S.C. Sharma (993) Robustness of nonnormality of the Durbin-Watson test for autocorrelation. Journal of Econometrics 57, Barndo -Nielsen, O.E. & D.R. Cox (989). Asymptotic Techniques for Use in Statistics. London: Chapman & Hall. Bao, Y. (2007) The approximate moments of the least squares estimator for the stationary autoregressive model under a general error distribution. Econometric Theory 23, Bao, Y. & A. Ullah (2007a) Finite sample properties of maximum likelihood estimator in spatial models. Journal of Econometrics 37, Bao, Y. & A. Ullah (2007b) The second-order bias and mean squared error of estimators in time series models. Journal of Econometrics, 40, Chandra, R. (983) Estimation of Econometric Models when Disturbances are not Necessarily Normal. Ph.D. Dissertation, Department of Statistics, Lucknow University, India. 3

15 Don, F.J.H. (979) The expectations of products of quadratic forms in normal variables. Statistica Neerlandica 33, Dressel, P.L. (940) Statistical semi-invariants and their estimates with particular emphasis on their relation to algebraic invariants. Annals of Mathematical Statistics, Dufour, J.-M. (984) Unbiasedness of predictions from estimated autoregressions when the true order is unknown. Econometrica 52, Fisher, R.A. (928) Moments and product moments of sampling distributions. Proceedings of the London Mathematical Society 30, Ghazal, G.A. (994) Moments of the ratio of two dependent quadratic forms. Statistics and Probability Letters 20, Ghazal, G.A. (996) Recurrence formula for expectations of products of quadratic forms. Statistics and Probability Letters 27, Hoque, A., J.R. Magnus & B. Pesaran (988) The exact multi-period mean-square forecast error for the rst-order autoregressive model. Journal of Econometrics 39, Kadane, J.B. (97) Comparison of k-class estimators when the disturbances are small. Econometrica 39, Kiviet, J.F. & G.D.A. Phillips (993) Alternative bias approximations in regressions with a lagged dependent variable. Econometric Theory 9, Komunjer, I. (2007) Asymmetric power distribution: theory and applications to risk measurement. Journal of Applied Econometrics, forthcoming. Kumar, A. (973) Expectations of product of quadratic forms. Sanhkya (Series B) 35, Lieberman, O. (994a) Saddlepoint approximations for the least squares estimator in rst-order autoregression. Biometrika 8, Lieberman, O. (994b) A Laplace approximation for the moments of a ratio of quadratic forms. Biometrika 8, Lieberman, O. (997) The e ect of nonnormality. Econometric Theory 3, Magee, L. (985) E ciency of iterative estimators in the regression model with AR() disturbances. Journal of Econometrics 29, Magnus, J.R. (978) The moments of products of quadratic forms in normal variables. Statistica Neerlandica 32, Magnus, J.R. (979) The expectation of products of quadratic forms in normal variables: the practice. Statistica Neerlandica 33, Magnus, J.R. & H. Neudecker (979) The commutation matrix: some properties and applications. Annals of Statistics 7, Mathai, A.M. & S.B. Provost (992) Quadratic Forms in Random Variables. New York: Marcel Dekker. Mishra, P.N (972) Recurrence formula for mathematical expectation of products of matrices of structural disturbances. Sanhkya (Series B) 34, Nagar, A.L. (959) The bias and moment matrix of the general k-class estimators of the parameters in simultaneous equations. Econometrica 27,

16 Pesaran, M.H. & T. Yamagata (2005) Testing slope homogeneity in large panels. Working paper, Cambridge University. Sawa, T. (972) Finite sample properties of the k-class estimators. Econometrica 40, Smith, M.D. (989) On the expectation of a ratio of quadratic forms in normal variables. Multivariate Analysis 3, Journal of Smith, M.D. (993) Expectations of ratios of quadratic forms in normal variables: evaluating some top order invariant polynomials. Australian Journal of Statistics 35, Srivastava, V.K. & K. Maekawa (995) E ciency properties of feasible generalized least squares estimators in SURE models under nonnormal disturbances. Journal of Econometrics 66, Srivastava, V.K. & R. Tiwari (976) Evaluation of expectations of products of stochastic matrices. Scandinavian Journal of Statistics 3, Stuart, A. & J.K. Ord (987) Kendall s Advanced Theory of Statistics (Vol., 5th ed.). New York: Oxford University Press. Theil, H. & A.L. Nagar (96) Testing the independence of regression disturbances. Journal of the American Statistical Association 56, Ullah, A. (2004) Finite Sample Econometrics. New York: Oxford University Press. Ullah, A., V.K. Srivastava & R. Chandra (983) Properties of shrinkage estimators when the disturbances are not normal. Journal of Econometrics 2, White, J.S. (957) Approximate moments for the serial correlation coe cients. Annals of Mathematical Statistics 28, Wilf, H.S. (994) Generatingfunctionology. San Diego: Academic Press. Zivot, E, R. Startz & C.R. Nelson (998) Valid con dence intervals and inference in the presence of weak instruments. International Economic Review 39,