On the Relationship Between Concentration and Inertia Hyperellipsoids

Transcription

1 Applied Mathematical Sciences, Vol. 2, 2008, no. 10, On the Relationship Between Concentration and Inertia Hyperellipsoids Brian J. M c Cartin Applied Mathematics, Kettering University 1700 West Third Avenue, Flint, MI USA bmccarti@kettering.edu Abstract We extend the known intricate connections between the concentration ellipsoid of statistics [1, 2] and the inertia ellipsoid of mechanics [3, 4] from three to n dimensions. We then also extend the relationship involving the eccentricities of their principal cross-sections [5] to n dimensions. The shared degeneracies of these hyperellipsoids is noted. Mathematics Subject Classifications: 51N20, 62J05, 70F99 Keywords: concentration hyperellipsoid; correlation hyperellipsoid; hyperellipsoid of residuals; inertia hyperellipsoid; momental hyperellipsoid; hyperellipsoid of gyration; orthogonal regression; least squares approximation 1 Introduction Mathematical statistics and classical mechanics both employ certain hyperellipsoids in their analytical investigations. These hyperellipsoids are distinct in n>3 dimensions while the corresponding two dimensional ellipses essentially coincide. In what follows, we first develop the concentration hyperellipsoid of statistics [1, 2] and the inertia hperellipsoid of mechanics [3, 4]. The known connections between these two hyperellipsoids in three dimensions are then extended to higher dimensions. Next, a relationship concerning the eccentricities of their principal cross-sections [5] is also extended from three to higher dimensions. Finally, it is observed that the previously derived relationships imply that both hyperellipsoids degenerate to hyperspheroids under identical conditions.

2 490 B. J. McCartin 2 Concentration Hyperellipsoid Consider the set of m data points in R n with coordinates {(x (i) 1,...,x (i) n )} m with centroid ( x 1,..., x n ) defined by the mean values x j := 1 m m Furthermore, let us introduce the variances x (i) j ;(j =1,...,n). (1) σj 2 := 1 m m (x (i) j x j ) 2 ;(j =1,...,n), (2) and the covariances p jk := 1 m m (x (i) j x j ) (x (i) k x k); (j =1,...,n; k =1,...,n; j k). (3) Defining the symmetric covariance matrix as σ 2 1 p 12 p 1n p 12 σ 2 C := 2 p 2n....., (4). p 1n p 2n σn 2 observe that v := [a 1 a 2 a n ] T implies that v T C v = E{[a 1 (x 1 x 1 )+a 2 (x 2 x 2 )+ + a n (x n x n )] 2 } 0, (5) where E is the averaging operator E{u} = 1 m m u (i), with equality if and only if the data all lie on the hyperplane of dimension n 1 a 1 (x 1 x 1 )+a 2 (x 2 x 2 )+ + a n (x n x n )=0. (6) Thus, hyperplanarity of the data corresponds to a zero eigenvalue of the positive semidefinite matrix C and the corresponding eigenvector is normal to this hyperplane. Of course, the data may further degenerate to a hyperplane of even lower dimension. Thus, without loss of generality, we will assume that C is positive definite in the ensuing analysis. The concentration hyperellipsoid (also known as a correlation hyperellipsoid) is defined by ( r r) T C 1 ( r r) =n +2, (7) which has the same first and second moments about the centroid as does the data [2, p. 300]. In this sense, it is the hyperellipsoid which is most representative of the data points without any a priori statistical assumptions

3 Concentration and inertia hyperellipsoids 491 concerning their origin. The reciprocal of the concentration hyperellipsoid, obtained by replacement of C 1 by C in the quadratic form of Equation (7), will be referred to as the hyperellipsoid of residuals [1]. Denoting the eigenvalues of C 1 by ν 1 <ν 2 < < ν n, the principal semiaxes of the concentration hyperellipsoid pass through its centroid and lie in the directions of the the corresponding eigenvectors, v j (j =1,...,n), while their lengths are given by 1/ ν j [6, p. 309]. The hyperplanar cross-sections of dimension n p passing through the centroid and perpendicular to p of these semiaxes are the corresponding principal sections of dimension n p of the concentration ellipsoid. Note that these principal sections are hyperellipsoids of dimension n p with principal semiaxes specified by the remaining n p eigenpairs. In particular, for p = n 2, these n(n 1) principal sections are ellipses and will be referred 2 to simply as the principal sections of the concentration hyperellipsoid. For p =1,...,n, the hyperplane of dimension n p which minimizes the mean square distance of the points from the hyperplane is called the orthogonal regression hyperplane of dimension n p and may be thought of as the hyperplane of dimension n p which best fits the data. The following important facts were first established by Pearson [1] in three dimensions and have subsequently been extended to n dimensions to form the basis for so-called orthogonal regression. The sum of the p th smallest eigenvalues of C equals the minimum mean square deviation from the orthogonal regression hyperplane of dimension n p which passes through the centroid and is normal to the corresponding p eigenvectors. These orthogonal regression hyperplanes are nested in that, for p =1,...,n 1, the one of dimension n p contains that of dimension n p 1. 3 Inertia Hyperellipsoid We next focus our attention on orthogonal linear regression which selects (α 1,α 2,...,α n ), the direction angles of a line (i.e., a hyperplane of dimension one), L, passing through the centroid, so as to minimize m n 1 r 2 (α 1,α 2,...,α n ):= { [cos α j (x (i) k x k) cos α k (x (i) j x j )] 2 }, (8) j=1 k=j+1 which is the total square distance of the data from L [7, p. 20]. Alternatively, r 2 (α 1,α 2,...,α n ) may be viewed as the moment of inertia with respect to L of unit masses located at the data points [3, p. 66].

4 492 B. J. McCartin In any event, the mean square deviation is given by n 1 1 m r2 (α 1,α 2,...,α n )= j=1 k=j+1 cos 2 α j σ 2 k 2 cos α j cos α k p jk + cos 2 α k σ 2 j. (9) Now, along L, mark off the two points at a distance l(α 1,α 2,...,α n ):= 1 r from the centroid. As α 2 (α 1,α 2,...,α n) 1,α 2,...,α n vary, these points sweep out a hyperellipsoid centered at the centroid. We see this as follows. Since cos α j =(x j x j )/l, 1 l 2 = r2 (α 1,α 2,...,α n )= j=1 jj (x j x j ) 2 n 1 I +2 l 2 j=1 k=j+1 I jk (x j x j )(x k x k ) l 2, (10) where I jj := m k j σ 2 k and I jk := m p jk are, respectively, the moments of inertia and the products of inertia about the coordinate axes [3, p. 67]. Equation (10) may be rewritten in matrix form as ( r r) T I( r r) =1, (11) where r =(x 1,x 2,...,x n ) T, r =( x 1, x 2,..., x n ) T, and the inertia matrix I is defined as I 11 I 12 I 1n I 12 I 22 I 2n I := (12). I 1n I 2n I nn That this n-dimensional quadric surface is a hyperellipsoid follows from the relation I = m [tr(c)i C], (13) so that the positive definiteness of I follows directly from that of C. This hyperellipsoid with l 2 (α 1,α 2,...,α n ) = 1/(moment of inertia) is called the inertia hyperellipsoid (also known as the momental hyperellipsoid) because of its role in rotational mechanics [3, p. 66]. Replacement of I by I 1 in the quadratic form of Equation (11) yields the reciprocal hyperellipsoid of gyration [4, p. 233]. The minimum value of r 2 (α 1,α 2,...,α n ) is achieved precisely when l 2 (α 1,α 2,...,α n ) assumes its maximum value which will, of course, correspond to the semimajor axis of the inertia hyperellipsoid (i.e., the largest of the principal semiaxes). Therefore, the minimum total square deviation is given by s 2 := r 2 (α 1,α 2,...,α n)=1/(semimajor axis) 2 = λ min, (14)

5 Concentration and inertia hyperellipsoids 493 where (α 1,α 2,...,α n ) are the direction angles of the semimajor axis and λ min is the smallest eigenvalue of I whose corresponding eigenvector points along the semimajor axis of the inertia hyperellipsoid. Alternatively, the orthogonal regression line is parallel to the eigenvector corresponding to the largest eigenvalue of the hyperellipsoid of gyration. In stark contrast to the two-dimensional scenario [5], the inertia hyperellipsoid is not homothetic (i.e., similar ) to the concentration hyperellipsoid for n>2. However, as we now will see, they are indeed closely related. 4 Hyperellipsoidal Relationships Equation (13) indicates that there is an intimate relationship between the inertia and concentration hyperellipsoids, a state of affairs which we now explore. In this investigation, we will denote the eigenvalues of I as λ 1 <λ 2 < <λ n, the eigenvalues of C as μ 1 <μ 2 < <μ n, and the eigenvalues of C 1 as ν 1 <ν 2 < <ν n. The degenerate case of multiple eigenvalues will be subsequently studied. In consequence of Equation (13) we have for i =1,...,n the relations λ i = m (μ 1 + μ μ n μ n i+1 ) with same eigenvector as μ n i+1. (15) Inverting these expressions we arrive at the complementary relations [4, p. 230] μ i = λ 1 + λ λ n (n 1)λ n i+1. (16) m(n 1) Observe that the positive definiteness of C implies the important inertia matrix inequality λ n < (λ 1 + λ λ n 1 )/(n 2) [4, p. 231]. Furthermore, Equations (15) and (16) together imply the desired relationship between the spectra of the inertia and concentration hyperellipsoids: ν i = 1 m(n 1) = with same eigenvector as λ i. μ n i+1 λ 1 + λ λ n (n 1)λ i (17) Thus, the corresponding principal semiaxes of the inertia and concentration hyperellipsoids are parallel. In addition, Equations (15) through (17) together imply the following collection of inequalities: Theorem 1 (Eccentricity Inequalities) For n>2, ν j ν k < λ j λ k (j =1,...,n 1; k = j +1,...,n). (18)

6 494 B. J. McCartin Proof: Since μ i (i =1,...,n) are all positive and n>2, we have Now, since we have μ n j+1 + μ n k+1 < μ i. j<k μ n k+1 <μ n j+1, (μ n j+1 + μ n k+1 ) (μ n j+1 μ n k+1 ) < μ i (μ n j+1 μ n k+1 ). This may be rearranged to produce μ n k+1 ( μ i μ n j+1 μ n k+1 ) <μ n j+1 ( μ i μ n k+1 μ n j+1 ). Adding μ n j+1 μ n k+1 to both sides yields μ n k+1 ( μ i μ n k+1 ) <μ n j+1 ( μ i μ n j+1 ), which implies that μ n n k+1 μ i μ n j+1 < μ n. n j+1 μ i μ n k+1 Finally, this may be recast as ν j < λ j. ν k λ k Note that if n = 2 then all of the above inequalities become equalities. Thus, not only are the principal axes of the inertia and concentration hyperellipsoids parallel, but each principal section of the inertia hyperellipsoid is less eccentric than the corresponding principal section of the concentration hyperellipsoid. This hyperellipsoidal relationship is a natural generalization of the corresponding three dimensional result first presented in [5]. 5 Conclusion We conclude with the straightforward observation that the inertia and concentration hyperellipsoids share degeneracies to hypershperoids, i.e. multiple eigenvalues, μ j 1 <μ j = = μ k <μ k+1, (19) in which case the orthogonal regression planes of dimension n j through n k + 1 will be indeterminate. An auxiliary consequence of the preceding

7 Concentration and inertia hyperellipsoids 495 considerations is that the concentration hyperellipsoid also shares degeneracies with the hyperellipsoid of residuals except that now, due to their reciprocal relationship, the inequalities of Equation (19) become ν n k <ν n k+1 = = ν n j+1 <ν n j+2. (20) The characteristic polynomial of C is called the discriminating polynomial of degree n and multiplicity of its roots is equivalent to degeneracy of these hyperellipsoids to hyperspheroids. Unfortunately, necessary and sufficient conditions for multiple roots of the discriminating polynomial are not readily available for n>3. Acknowledgement. This paper is dedicated to my loving wife, Barbara A. McCartin, for her support of all of my mathematical endeavors. References [1] K. Pearson, On Lines and Planes of Closest Fit to Systems of Points in Space, Phil. Mag. 2 (1901), [2] H. Cramér, Mathematical Methods of Statistics, Princeton University Press, Princeton, NJ, [3] H. Lamb, Higher Mechanics, Second Edition, Cambridge University Press, Cambridge, [4] A. G. Webster, Dynamics of Particles and of Rigid, Elastic, and Fluid Bodies, Third Edition, Stechert, New York, [5] B. J. McCartin, On Concentration and Inertia Ellipsoids, Applied Mathematical Sciences 1(1) (2007), [6] F. R. Gantmacher, Matrix Theory: Volume I, Chelsea, New York, [7] M. G. Kendall, A Course in the Geometry of n Dimensions, Hafner, New York, NY, Received: August 26, 2007