Manabu Sato* and Masaaki Ito**
|
|
- Joanna Powell
- 5 years ago
- Views:
Transcription
1 J. Japan Statist. Soc. Vol. 37 No THEORETICAL JUSTIFICATION OF DECISION RULES FOR THE NUMBER OF FACTORS: PRINCIPAL COMPONENT ANALYSIS AS A SUBSTITUTE FOR FACTOR ANALYSIS IN ONE-FACTOR CASES Manabu Sato* and Masaaki Ito** Applying principal component analysis as a substitute for factor analysis, we often adopt the following greater-than-one rule to decide the number of factors, k: Take the number of eigenvalues of the correlation matrix that is greater than one. Another approach to deciding k is based on the scree graph. In the present paper, the adequacy of these rules for one-factor cases is discussed. On the basis of obtained results, some recommendations for data analysis are given. Our approach to this study is based on the analytical expressions of eigenvalues under some simple but practical cases. In deriving theoretical results, we use a representation of a polynomial in terms of a remainder sequence. This technique is useful for finding the sign of polynomials under inequality constraints, so the idea is also introduced. Key words and phrases: Cubic equation, greater-than-one rule, number of factors, principal component analysis, representation of a polynomial in terms of a remainder sequence, scree test. 1. Introduction Principal component analysis (PCA) is a branch of multivariate statistical analysis concerned with the internal relationships of a set of variables. It is well known that PCA and factor analysis (FA) resemble each other but have slightly different aims (Chap. 7 of Jolliffe (2002), Chap. 14 of Anderson (2003)). However, PCA is very often used for the same purpose as FA. In fact, when PCA is applied, analysts calculate not only principal components but correlations between principal components and original variables (see, for example, Section of Chatfield and Collins (1989)). Using the correlations, which are called factor loadings after FA, they quite often try to derive a latent structure. In this way, PCA is often used as a substitute for FA. Many attempts were made to compare and contrast FA with PCA (see, for example, Bentler and Kano (1990), Schneeweiss and Mathes (1995), Schneeweiss (1997), and Ogasawara (2000)), while Jolliffe (2002, Section 7.3) summarized many results. Deciding the number of factors, k, is an important problem in FA. When Received December 12,2005. Revised June 16,2006. Accepted September 29,2006. *Center for Foundational Arts and Sciences,Faculty of Health Sciences,Hiroshima Prefectural College of Health Sciences,Japan. Current address: Graduate School of Information Sciences, Hiroshima City University,3-4-1 Ozuka-Higashi,Asa-Minami-ku,Hiroshima ,Japan. satomnb@hiroshima-cu.ac.jp **Graduate School of Engineering,Hiroshima University,1-4-1,Kagamiyama,Higashi-Hiroshima ,Japan.
2 176 MANABU SATO AND MASAAKI ITO researchers analyze a correlation matrix, they often adopt the following greaterthan-one rule: The number of factors is taken as the number of eigenvalues of the correlation matrix that is greater than one. Hence we will investigate the adequacy of this rule in the present paper. Another approach to deciding k is based on the scree graph proposed by Cattell (1966). Cattell (1966, p. 249) noticed that this scree invariably began at the k-th eigenvalue when k was the true number of factors. This approach is also widely applied (see, for example, Section of Jolliffe (2002)). Hence we also examine the adequacy of this scree test. Further, we study the properties of the greater-than-one rule and the scree test when the number p of variables increases. Now we will explain an approach, a framework, and assumptions of our investigation. (i) FA model We assume that data follow an FA model, although PCA does not require a structural model. This assumption is natural, because, when researchers want to interpret the factor loadings calculated with PCA, it is assumed implicitly that the FA model holds. Therefore, let s assume a p-dimensional observable vector x follows an FA model: x = µ +Λf + u. Here, µ = (µ 1,...,µ p ) is a mean vector, Λ is a p k (p > k) factor loading matrix of rank k, f =(f 1,...,f k ) is a common factor vector and u =(u 1,...,u p ) is an error term (see, for example, p. 6 of Lawley and Maxwell (1971)). The population variance-covariance matrix Σ of x can be written as Σ = ΛΛ + Ψ under the conditions that E{f } = 0, E{u} = 0, E{fu } = O (a null matrix), and E{uu } = Ψ (a diagonal matrix). Here, the symbol prime ( ) means a transposed vector or matrix. (ii) One-factor case We focus on one-factor cases (k = 1). When we confirm a latent structure in practical data analysis, the complete simple structure, that is, each row of Λ has only one nonzero element, is often assumed. This structure can be reduced to sets of one-factor cases (see, for example, Sato (1990)). Let λ denote a p 1 loading vector, then (1.1) Σ=λλ +Ψ. (iii) Correlation matrix Since it is common to apply a correlation matrix instead of a variancecovariance matrix in view of the property of scale invariance, we consider a correlation matrix. Putting D = diag Σ, and resetting D 1/2 λ λ, D 1/2 ΨD 1/2 Ψ, from (1.1) we can express the population correlation matrix P as (1.2) P = λλ +Ψ. (iv) Range of parameters We give parameters λ =(λ 1,...,λ p ) such that diag(i λλ ) is positive
3 DECISION RULES FOR THE NUMBER OF FACTORS 177 definite, and set parameters Ψ = diag{ψ 1,...,ψ p } = diag(i λλ ) where I is the unit matrix. We assume that 1 >λ 1 λ p > 0 and p 3 throughout this paper. The reasons of these assumptions are as follows: If λ i 1, then the corresponding unique variance ψ i =1 λ 2 i becomes nonpositive. If λ i = 0, then this loading is meaningless. If λ i < 0, by inverting the sign of x i (= µ i + λ i f 1 + u i ) and µ i, we can change the sign of λ i. By reordering λ i, we can assume without loss of generality that λ 1 λ p. In one-factor cases, if and only if p 3 under λ i 0,λ can be determined uniquely up to sign (Theorems 5.1 and 5.6 of Anderson and Rubin (1956)). Section 2 presents the expression and properties of eigenvalues of P defined by (1.2). Section 3 describes applications to principal component analysis. Section 4 examines whether some propositions guaranteed under (1.2) hold for a certain empirical data set which does not have the structure of (1.2) exactly. Section 5 gives some recommendations for data analysis and makes concluding remarks. The appendix gives proofs of some theorems, lemmas and a proposition. To complete these proofs we used a representation of a polynomial in terms of a remainder sequence and here we explain its idea. 2. Expression and properties of eigenvalues of treated matrix At first, inequalities for eigenvalues θ 1 θ p of P defined by (1.2) are introduced. It is difficult to show an explicit expression of θ i in a generic case, although the next proposition gives the upper and lower bounds for θ i. Proposition 1 (Upper and lower bounds for eigenvalues) [Theorem 5.1 of Sato (1992)]. The following inequalities for the eigenvalues θ i of P defined by (1.2) are established. 1+λ λ 2 p 1 θ 1 1+λ λ 2 p > 1 > 1 λ 2 p θ 2 1 λ 2 p 1 θ 3 θ p 1 λ 2 1. The equalities 1+λ λ2 p 1 = θ 1 =1+λ λ2 p hold if and only if λ 1 = = λ p. The equalities 1 λ 2 p+2 i = θ i =1 λ 2 p+1 i hold if and only if λ p+2 i = λ p+1 i (i =2,...,p). We focus on a three-variable case, which is the most fundamental case. We can obtain explicit expressions of eigenvalues by Cardano s or Lagrange s method for solving the associated cubic eigen-polynomial equation. The expressions are, however, unsuitable for investigating the properties of eigenvalues because they include cubic roots of complex numbers. To this end, we express the eigenvalues in terms of cosine functions to obtain their properties. Lemma 1 (Explicit expression of eigenvalues). In the case p = 3, the eigenvalues θ 1 >θ 2 θ 3 of P defined by (1.2) can be expressed as follows: θ 1 = λ 2 1 λ2 2 + λ2 1 λ2 3 + λ2 2 λ2 3 cos φ 3,
4 178 MANABU SATO AND MASAAKI ITO θ 2 =1 2 λ 2 1 λ2 2 + λ2 1 λ2 3 + λ2 2 λ2 3 cos φ + π, 3 3 θ 3 =1 2 λ 2 1 λ2 2 + λ2 1 λ2 3 + λ2 2 λ2 3 cos φ π, λ where cos φ = 2 1 λ2 2 λ2 3 and 0 φ< π (λ 2 1 λ2 2 +λ2 1 λ2 3 +λ2 2 λ2 3 )3/2 2. The equality θ 2 = θ 3 holds if and only if λ 1 = λ 2 = λ 3, or equivalently, φ =0. A proof of this lemma is given in Subsection A.2 of Appendix. When analysts investigate a latent structure of treated data in practice, they do not always examine loadings precisely; they simply check whether they are of large or small magnitude. Thus, we may treat a simple case; the loading vector λ consists of two magnitude levels, say α (large) and β (small). In this case, we can obtain the eigenvalues of P explicitly as the following lemma. Lemma 2 (Eigenvalues for simple case). In the case λ = (α,...,α, }{{} p 1 β,...,β), where 1 > α β > 0 and p = p }{{} 1 + p 2 3, the eigenvalues p 2 θ 1 >θ 2 θ p of P defined by (1.2) are given. θ 1 = ((p 1 1)α 2 +(p 2 1)β 2 + (p 1 1) 2 α 4 +(p 2 1) 2 β 4 +2(p 1 p 2 + p 1 + p 2 1)α 2 β 2 ). Further, θ 2,...,θ p are expressed as follows: (i) If p 1 =1and p 2 2, then θ 2 = = θ p 1 =1 β 2 (multiplicity p 2=p 2 1) and θ p = ((p 2 1)β 2 (p 2 1) 2 β 4 +4p 2 α 2 β 2 ). (ii) If p 1 2 and p 2 =1,then θ 2 = ((p 1 1)α 2 (p 1 1) 2 α 4 +4p 1 α 2 β 2 ) < 1 β 2 and θ 3 = = θ p =1 α 2 (multiplicity p 1 1). (iii) If p 1 2 and p 2 2, then θ 2 = = θ p2 =1 β 2 (multiplicity p 2 1), and θ p2 +1 = ((p 1 1)α 2 +(p 2 1)β 2 (p 1 1) 2 α 4 +(p 2 1) 2 β 4 +2(p 1 p 2 + p 1 + p 2 1)α 2 β 2 ) θ p2 +2 = = θ p =1 α 2 (multiplicity p p 2 1=p 1 1). A proof of this lemma is given in Subsection A.3 of Appendix.
5 DECISION RULES FOR THE NUMBER OF FACTORS Application to principal component analysis as a substitute for factor analysis We know the upper and lower bounds of θ 1 and θ 2 from Proposition 1. Further, owing to Lemma 1, we can investigate the properties of the eigenvalues precisely in a three-variable case. The next theorem shows the behavior of θ 1 on a factor loading. Theorem 1 (Partial derivatives of the largest eigenvalue). In the case p = 3, the largest eigenvalue θ 1 of P defined by (1.2) has the following property: θ 1 > 0 (i =1, 2, 3) λ i except in the case λ 1 = λ 2 = λ 3. A proof of this theorem is given in Subsection A.4 of Appendix. The next theorem shows the behavior of θ 1 θ 2, which is used with the scree test. Theorem 2 (Partial derivatives of difference between the first and second eigenvalues). In the case p = 3,the difference between the first and the second eigenvalues of P defined by (1.2), = θ 1 θ 2, has the following property: > 0 (i =1, 2, 3) λ i except in the case λ 1 = λ 2 = λ 3. A proof of this theorem is given in Subsection A.5 of Appendix. The next proposition shows the behavior of eigenvalues for a two-level loading case. We obtain stronger results for this simple case. Proposition 2 (Behavior of eigenvalues for a simple case). Assume that λ =(α,...,α,β,...,β), where 1 >α β>0 and p = p }{{}}{{} 1 + p 2 3. Letθ 1 and p 1 p 2 θ 2 be the first and the second eigenvalues of P defined by (1.2), respectively. Then, the behavior of θ 1, θ 2 and =θ 1 θ 2 is as follows: (i) θ 1 increases as α, β, p 1 or p 2 increases. (ii) θ 2 β < 0. (iii) increases as α, β, p 1 or p 2 increases. This proposition can easily be proved from Lemma 2. The next proposition shows the behavior of eigenvalues on increasing the number of variables. Proposition 3 (Behavior of eigenvalues when one loading is added). Consider a loading vector λ =(λ 1,...,λ p+1 ) obtained by adding one loading to λ,
6 180 MANABU SATO AND MASAAKI ITO where 1 >λ i > 0(i =1,...,p+1).Let P = λ λ +Ψ, where Ψ = diag{ψ 1,...,ψ p+1 } and ψ i = 1 λ 2 i (i = 1,...,p +1). Take θ 1 θ p+1 and θ 1 θ p to be the eigenvalues of P and those of P defined by (1.2), respectively. Then the inequalities between the eigenvalues of P and P θ j θ j (j =1,...,p) are established. This proposition can be proved directly by using the Sturmian separation theorem (see, for example, Section 1f.2 (vi) of Rao (1973)). Proposition 3 shows the behavior of each eigenvalue when one loading is added. In the next proposition, we deal with the difference between the first and the second eigenvalues for the following simple case: Another factor loading γ is added to the simple loading vector in which all loadings are equal to α. Proposition 4. Consider a loading vector λ =(α,...,α,γ) of (p +1) variables obtained by adding one factor loading γ to λ =(α,...,α) where 1 > α, γ > 0 and p 3. Let P = λ λ +Ψ, where Ψ = diag{ψ 1,...,ψ p+1 }, ψ i =1 α2 (i =1,...,p) and ψ p+1 =1 γ2. Let the eigenvalues of P be θ 1 θ p+1. If γ 1 (3.1) p α, then θ 1 θ 2 θ 1 θ 2 holds. A proof of this proposition is given in Subsection A.6 of Appendix. This proposition suggests the following: As a large enough loading γ is added, the difference between the first and second eigenvalues increases. As p increases, p Table 1. Minimum value of γ such that θ 1 θ 2 θ 1 θ min γ such that θ 1 θ 2 θ1 θ2 2 1 α p α =.6 α =.7 α =.8 α.645α α.661α α.671α α.707α
7 DECISION RULES FOR THE NUMBER OF FACTORS 181 the larger loading is required because the effect of the additional loading is weakened. Table 1 represents the minimum value of γ in Formula (3.1) when p and α are given. 4. Examination of Propositions 3 and 4 for a certain empirical data set In practical data analysis, since a population correlation matrix P is unknown, it is estimated with a sample correlation matrix R. Although P satisfies Formula (1.2), R does not have a decomposition such as (1.2) exactly. Hence, for a certain empirical data set, we examine whether some propositions on the basis of (1.2) shown in the present paper hold, or not. We treat the famous data set introduced by Spearman (1904), in which he originated factor analysis. The data consist of measurements on six variables and their sample size is 33. The number of factors for this set of data is known to be one. Let R (j) (j =3, 4, 5, 6) be the first j variables of the sample correlation matrix (j) and ˆθ i be the i-th eigenvalue of R (j). Table 2 shows the eigenvalues of R (j).we can see that Proposition 3 holds under the condition that the sample correlation matrix is regarded to be the population one. With regard to Proposition 4, we can recognize that the difference between the first and the second eigenvalues enlarges when the number of variables increases. In fact, (ˆθ (4) (4) (3) (3) 1 ˆθ 2 ) (ˆθ 1 ˆθ 2 )=.586 > 0, (ˆθ (5) (5) (4) (4) (6) (6) (5) (5) 1 ˆθ 2 ) (ˆθ 1 ˆθ 2 )=.313 > 0 and (ˆθ 1 ˆθ 2 ) (ˆθ 1 ˆθ 2 )=.397 > 0. Table 2. Eigenvalues of first 3 6 variables of correlation matrix for Spearman s data. Number of Eigenvalues variables Recommendations for data analysis and concluding remarks On the basis of our results, we give the following recommendations for data analysis: (i) It is desirable to increase the number of variables (test items) because the largest eigenvalue increases as the number of variables increases (Proposition 3). As a large enough loading is added, used in the scree test increases (Proposition 4). (ii) It is desirable to select large loadings, because θ 1 and increase as the loading(s) increases (Theorems 1 and 2). After a set of data to be analyzed is collected, of course, it is impossible to follow these recommendations. However, when designing a questionnaire or making up a test battery, it is desirable to carry it out.
8 182 MANABU SATO AND MASAAKI ITO Since the sample size of the data set described in Section 4 is only 33, fluctuation of its sample correlation matrix R is large, so, R does not have a decomposition such as Formula (1.2) exactly. However, some properties based on (1.2) are held for this data set. Therefore, we may expect that many sets of empirical data have such properties. Propositions for new estimators or simulation studies have been carried out for a long time (see, for example, ten Berge and Kiers (1991) and Hoyle and Duvall (2004)). In contrast, our approach in the present paper is on the basis of analytical expressions of eigenvalues under some simple but practical cases. For a three-variable case, each eigenvalue can be expressed explicitly as cosine functions (Lemma 1), and we can explore the properties of them. The results obtained are consistent with experiences of applied users. In order to derive Lemma 1 as well as Theorems 1 and 2, the representation of the concerned polynomials in terms of remainder sequences is efficient. In the present study, we have used MATHEMATICA (Wolfram (1996)), which is a computer algebra system, to perform the extremely tedious algebraic calculations. In multi-factor cases (k 2) in which we cannot reduce to sets of one-factor cases, substitution of PCA for FA is sometimes inadequate (see Section 6 of Sato (1992) and Sato and Ito (2003)). Appendix In this appendix, proofs of the theorems, lemmas and proposition in Sections 2 and 3 are shown. In some of these proofs, we need to find the sign of polynomials in the variables λ 1, λ 2 and λ 3. In the present cases, we can assume without loss of generality that λ i are written in decreasing magnitude, so, λ 1 λ 2 λ 3. In other words, λ i λ j is non-negative provided that i>j. To find the sign of a given polynomial, we use the non-negativity of λ i λ j, in which we rewrite the polynomial in terms of a remainder sequence of polynomial λ i λ j. We shall illustrate the idea of a representation of a polynomial in terms of a remainder sequence in Subsection A.1. This method is generally applicable to find the sign of polynomials under inequality constraints. A.1. Representation of a polynomial in terms of a remainder sequence Let f(x 1,x 2,...,x n ) be a real valued polynomial in the real variables x 1, x 2,...,x n, and denote f(x ). Let p 1 (x ) be a non-negative polynomial associated with a constraint; for example, if a constraint is x 1 x 2, then we adopt a non-negative polynomial p 1 (x )=x 1 x 2. Then, f(x ) can be divided by the polynomial p 1 (x ) to obtain a quotient q 1 (x ) and a remainder r 1 (x ); f(x )=p 1 (x )q 1 (x )+r 1 (x ). This process can be repeated with q 1 (x ) and a non-negative polynomial p 2 (x ) related to another constraint to obtain a quotient q 2 (x ) and a remainder r 2 (x ); q 1 (x )=p 2 (x )q 2 (x )+r 2 (x ).
9 DECISION RULES FOR THE NUMBER OF FACTORS 183 Repeating this process, we obtain the following sequence of polynomials: q 2 (x )=p 3 (x )q 3 (x )+r 3 (x ),. q m 2 (x )=p m 1 (x )q m 1 (x )+r m 1 (x ), q m 1 (x )=p m (x )q m (x )+r m (x ). Using the above sequence of polynomials, f(x ) can be written like synthetic division as follows: f(x )=(( (p m (x )q m (x )+r m (x ))p m 1 (x ) + r m 1 (x ))p m 2 (x ). + r 2 (x ))p 1 (x )+r 1 (x ). We call this expression a representation of a polynomial in terms of a remainder sequence. To show that f(x ) 0, we need only show that r i (x ) 0(i = 1, 2,...,m) and q m (x ) 0. Here we should note that the representation of the given polynomial depends on the choice of non-negative polynomials p i (x ). In the following proofs, we have to handle polynomials in three variables with total degree 12 or 16. Therefore, we can hardly perform our proofs without using computer algebra. Hence, we used the computer algebra system MATHEMATICA to choose p i (x ) such that we can easily prove f(x ) 0 and to obtain the quotients q i (x ) and the remainders r i (x ). A.2. Proof of Lemma 1 The eigenvalues θ s of P are solutions of a polynomial equation det P θi =0. In the three-variable case, using (1.2), this equation is expressed as θ 3 3θ 2 (λ 2 1λ λ 2 1λ λ 2 2λ 2 3 3)θ (2λ 2 1λ 2 2λ 2 3 λ 2 1λ 2 2 λ 2 1λ 2 3 λ 2 2λ )=0. To apply the formula for the solution of a cubic equation (see, for example, p. 10 of Beyer (1987)), we eliminate the θ 2 term. Letting x = θ 1, we have (A.1) x 3 (λ 2 1λ λ 2 1λ λ 2 2λ 2 3)x 2λ 2 1λ 2 2λ 2 3 =0. Using cosine functions, we can express the solutions x 1,x 2,x 3 of equation (A.1) as follows: x 1 = 2 λ (A.2) 2 1 λ2 2 + λ2 1 λ2 3 + λ2 2 λ2 3 cos φ 3 3, x 2 = 2 (A.3) λ 2 1 λ2 2 + λ2 1 λ2 3 + λ2 2 λ2 3 cos φ + π, 3 3 x 3 = 2 (A.4) λ 2 1 λ2 2 + λ2 1 λ2 3 + λ2 2 λ2 3 cos φ π, 3 3
10 184 MANABU SATO AND MASAAKI ITO 3 3λ where cos φ = 2 1 λ2 2 λ2 3 and 0 φ< π (λ 2 1 λ2 2 +λ2 1 λ2 3 +λ2 2 λ2 3 )3/2 2. Next, we show the inequality θ 1 >θ 2. From (A.2), (A.3), θ 1 = x and θ 2 = x 2 +1,wehave θ 1 θ 2 = x 1 x 2 =2 λ 2 1 λ2 2 + λ2 1 λ φ + π λ2 2 λ2 3 cos. 6 Since 0 φ< π 2φ+π 2, then cos 6 > 0. Thus, we have θ 1 >θ 2. We prove the inequality θ 2 θ 3. From (A.3), (A.4), θ 2 = x and θ 3 = x 3 +1,wehave θ 2 θ 3 = x 2 x 3 = 4 λ (A.5) 2 1 λ2 2 + λ2 1 λ2 3 + λ2 2 λ2 3 sin φ 3 3. Since 0 φ< π 2, then sin φ 3 0. Consequently, θ 2 θ 3, and thus we have θ 1 >θ 2 θ 3. Finally, we show that the equality θ 2 = θ 3 holds if and only if λ 1 = λ 2 = λ 3. From (A.5), the equality θ 2 = θ 3 holds if and only if φ = 0, or equivalently, 3 3λ 2 1 λ2 2 λ2 3 (λ 2 1 λ2 2 + λ2 1 λ2 3 + =1. λ2 2 λ2 3 )3/2 Putting f 1 =(λ 2 1λ λ 2 1λ λ 2 2λ 2 3) 3 27λ 4 1λ 4 2λ 4 3, we represent it as a polynomial in terms of a remainder sequence: where f 1 = (((λ 2 1 λ 2 3)q 13 + r 13 )(λ 2 2 λ 2 3)+r 12 )(λ 2 1 λ 2 2)+r 11, q 13 = λ 2 1λ λ 2 1λ 2 2λ λ λ 4 2λ λ 4 3(λ 2 1 λ 2 2), r 13 =(λ λ 2 2λ λ 4 3)(λ λ 2 3)(λ 2 2 λ 2 3), r 12 = λ 6 3(8λ λ 2 3)(λ 2 1 λ 2 3), r 11 = λ 6 2(λ λ 2 3)(λ 2 2 λ 2 3) 2. Thus f 1 0, because we assume λ 1 λ 2 λ 3 > 0 described in Section 1. We see f 1 = 0 if and only if λ 1 = λ 2 = λ 3, or equivalently, φ =0. A.3. Proof of Lemma 2 The eigen-polynomial equation det P θi = 0 yields (θ (1 α 2 )) p 1 1 (θ (1 β 2 )) p 2 1 {θ 2 (2 + (p 1 1)α 2 +(p 2 1)β 2 )θ + {1+(p 1 1)α 2 +(p 2 1)β 2 Therefore, +(1 p 1 p 2 )α 2 β 2 }} =0. θ =1 α 2 (multiplicity p 1 1), 1 β 2 (multiplicity p 2 1) and other solutions can be obtained easily by solving the quadratic equation within the braces.
11 DECISION RULES FOR THE NUMBER OF FACTORS 185 A.4. Proof of Theorem 1 Originally, the range of φ is 0 φ< π 2 from Lemma 1. However, we exclude the endpoint φ = 0, because we treat the derivative with respect to φ. Letting ν = λ 2 1λ λ 2 1λ λ 2 2λ 2 3 and ζ =3 3λ 2 1λ 2 2λ 2 3ν 3/2, we have Since θ 1 =1+ 2 φ ν cos 3 3, φ = cos 1 ζ, θ 1 = 2 { 1 ν ν 1/2 cos φ λ i 3 2 λ i 3 1 ν1/2 3 sin φ 3 = 2 3ν 1/2 } φ λ i } dφ ζ dζ λ i } dφ ζ dζ λ i for i, j, k =1, 2, 3; i j, i k, j k. { 1 2 (λ2 j + λ 2 k)2λ i cos φ ν sin φ 3 = 2 cos φ { 3 (λ 2 3ν 1/2 j + λ 2 k)λ i 1 3 ν tan φ 3 sup tan φ 0<φ<π/2 3 = 1, 3 dφ dζ = 1 = ν 3/2 1 ζ 2 < 0, ζ λ i =3 3λ i λ 2 jλ 2 k(2λ 2 jλ 2 k λ 2 i λ 2 j λ 2 i λ 2 k)ν 5/2, the inequality θ 1 λ i > 0 holds, provided that (A.6) λ 2 j + λ 2 k > λ2 j λ2 k (λ2 i λ2 j + λ2 i λ2 k 2λ2 j λ2 k ). (i) θ 1 λ 1 > 0: Formula (A.6) yields Putting λ λ 2 3 > λ2 2 λ2 3 (λ2 2 (λ2 1 λ2 3 )+λ2 3 (λ2 1 λ2 2 )). f 2 =(λ λ 2 3) 2 (ν 3 27λ 4 1λ 4 2λ 4 3) λ 4 2λ 4 3(λ 2 2(λ 2 1 λ 2 3)+λ 2 3(λ 2 1 λ 2 2)) 2, we represent it as a polynomial in terms of a remainder sequence: f 2 = (((λ 2 2 λ 2 3)q 23 + r 23 )(λ 2 1 λ 2 3)+r 22 )(λ 2 1 λ 2 2)+r 21,
12 186 MANABU SATO AND MASAAKI ITO where q 23 = λ 2 1(λ λ 6 2λ λ 4 2λ λ 2 2λ λ 8 3) +(λ λ 2 2λ λ 4 3)(λ λ 2 3) 2 (λ 2 2 λ 2 3), r 23 =2λ 2 1λ 10 3, r 22 =(λ λ 6 2λ λ 4 2λ λ 2 2λ λ 8 3)(λ λ 2 3)(λ 2 2 λ 2 3) 2, r 21 = λ 6 2(λ λ 4 2λ λ 2 2λ λ 6 3)(λ 2 2 λ 2 3) 2. Thus, f 2 0, because we assume λ 1 λ 2 λ 3 > 0 described in Section 1. From Lemma 1, the case in which f 2 = 0 occurs if and only if λ 1 = λ 2 = λ 3,or equivalently φ = 0. This contradicts 0 <φ< π 2. Consequently, f 2 > 0 holds, and thus the inequality θ 1 λ 1 > 0 is proved. (ii) θ 1 λ 2 > 0: Formula (A.6) yields λ λ 2 3 > λ2 1 λ2 3 (λ2 2 (λ2 1 + λ2 3 ) 2λ2 1 λ2 3 ). > 0 obvi- Case 1. In the case λ 2 2 (λ2 1 + λ2 3 ) 2λ2 1 λ2 3 0, the inequality θ 1 λ 2 ously holds. Case 2. In the case λ 2 2 (λ2 1 + λ2 3 ) 2λ2 1 λ2 3 > 0, we will check the sign of f 3 =(λ λ 2 3) 2 (ν 3 27λ 4 1λ 4 2λ 4 3) λ 4 1λ 4 3(λ 2 2(λ λ 2 3) 2λ 2 1λ 2 3) 2. We represent f 3 as a polynomial in terms of a remainder sequence: f 3 = (((λ 2 2 λ 2 3)q 33 + r 33 )(λ 2 1 λ 2 3)+r 32 )(λ 2 1 λ 2 2)+r 31, where q 33 = λ 6 1λ λ 6 1λ 2 2λ λ 6 1λ λ 4 1λ λ 4 1λ 4 2λ λ 4 1λ λ 2 1λ λ 2 1λ 4 2λ λ 2 1λ λ λ 4 2λ λ λ 2 2λ 2 3(10λ 2 1λ λ 2 1λ λ λ 2 2λ λ 4 3)(λ 2 2 λ 2 3), r 33 = λ 6 3(8λ λ 4 1λ λ 2 1λ λ 6 3), r 32 = λ 2 2(λ λ 2 3)((λ λ 4 2λ λ 2 2λ λ 6 3)(λ 2 2 λ 2 3)+λ 8 3)(λ 2 2 λ 2 3), r 31 = λ 6 2(λ λ 4 2λ λ 2 2λ λ 6 3)(λ 2 2 λ 2 3) 2. Thus, f 3 0, because λ 1 λ 2 λ 3 > 0. From Lemma 1, the case in which f 3 = 0 occurs if and only if λ 1 = λ 2 = λ 3, or equivalently φ = 0. This contradicts 0 <φ< π 2. Consequently, f 3 > 0 holds, and thus the inequality θ 1 λ 2 > 0is proved.
13 DECISION RULES FOR THE NUMBER OF FACTORS 187 (iii) θ 1 λ 3 > 0: Formula (A.6) yields λ λ 2 2 > λ2 1 λ2 2 (λ2 1 (λ2 2 λ2 3 )+λ2 2 (λ2 1 λ2 3 )) 0. Therefore, the inequality θ 1 λ 3 > 0 is proved. A.5. Proof of Theorem 2 From Lemma 1, we see =θ 1 θ 2 = 2 ( ν 1/2 cos φ cos φ + π ) =2ν 1/2 cos ω, 3 where ν = λ 2 1λ λ 2 1λ λ 2 2λ 2 3, cos φ =3 ( 3λ 2 1λ 2 2λ 2 3ν 3/2 0 <φ< π ), 2 ω = φ 3 + π ( ) π 6 6 <ω<π. 3 Then we have { =2λ i ν 1/2 cos ω (λ 2 j + λ 2 λ k) 2 } φ ν tan ω i 3 λ 2. i Since sup tan ω = 3, in order to examine the inequality λ i whether > 0, we check (A.7) λ 2 j + λ 2 k > 2 3 ν φ λ 2 i = 3(λ2 j λ2 k (λ2 j (λ2 i λ2 k )+λ2 k (λ2 i λ2 j ))) for i, j, k =1, 2, 3; i j, i k, j k. (i) λ 1 > 0: Formula (A.7) yields λ λ 2 3 > 3(λ2 2 λ2 3 (λ2 2 (λ2 1 λ2 3 )+λ2 3 (λ2 1 λ2 2 ))) > 0. Therefore, we must check the sign of f 4 =(λ λ 2 3) 2 (ν 3 27λ 4 1λ 4 2λ 4 3) 9(λ 2 2λ 2 3(λ 2 2(λ 2 1 λ 2 3)+λ 2 3(λ 2 1 λ 2 2))) 2. We represent f 4 as a polynomial in terms of a remainder sequence: f 4 = (((λ 2 2 λ 2 3)q 43 + r 43 )(λ 2 1 λ 2 3)+r 42 )(λ 2 1 λ 2 2)+r 41,
14 188 MANABU SATO AND MASAAKI ITO where q 43 =(λ λ 2 3) 2 ((2λ λ 2 2λ λ 4 3)(λ 2 2 λ 2 3)+4λ 6 3), r 43 =(λ λ 2 3) 5 (λ 2 1 λ 2 2), r 42 =(λ λ 2 3)(λ λ 6 2λ λ 4 2λ λ 2 2λ λ 8 3)(λ 2 2 λ 2 3) 2, r 41 = λ 6 2(λ λ 4 2λ λ 2 2λ λ 6 3)(λ 2 2 λ 2 3) 2. Thus, f 4 0, because we assume λ 1 λ 2 λ 3 > 0 described in Section 1. From Lemma 1, the case in which f 4 = 0 occurs if and only if λ 1 = λ 2 = λ 3,or equivalently φ = 0. This contradicts 0 <φ< π 2. Consequently, f 4 > 0 holds, and thus the inequality λ 1 > 0 is proved. (ii) λ 2 > 0: Formula (A.7) yields (A.8) λ λ 2 3 > 3(λ2 1 λ2 3 (λ2 2 (λ2 1 + λ2 3 ) 2λ2 1 λ2 3 )). Case 1. If λ 2 1 λ2 3 (λ2 2 (λ2 1 +λ2 3 ) 2λ2 1 λ2 3 ) 0, inequality (A.8) obviously holds, because the left hand side is positive. Case 2. If λ 2 1 λ2 3 (λ2 2 (λ2 1 + λ2 3 ) 2λ2 1 λ2 3 ) > 0, we check the sign of f 5 =(λ λ 2 3) 2 (ν 3 27λ 4 1λ 4 2λ 4 3) 9(λ 2 1λ 2 3(λ 2 2(λ λ 2 3) 2λ 2 1λ 2 3)) 2. We represent f 5 as a polynomial in terms of a remainder sequence: f 5 = (((λ 2 2 λ 2 3)q 53 + r 53 )(λ 2 1 λ 2 3)+r 52 )(λ 2 1 λ 2 2)+r 51, where q 53 = λ 6 1λ λ 6 1λ 2 2λ λ 6 1λ λ 4 1λ λ 4 1λ 4 2λ λ 4 1λ λ 2 1λ λ 2 1λ λ λ 8 2λ λ 6 2λ λ 4 2λ λ λ 2 1λ 2 2λ 2 3(4λ λ λ 2 3)(λ 2 2 λ 2 3)+31λ 8 3(λ 2 1 λ 2 2), r 53 = λ 6 3(8λ λ 4 1λ λ 2 1λ λ 6 3), r 52 =((λ λ 8 2λ λ 6 2λ λ 4 2λ λ 2 2λ λ 8 3(λ 2 2 λ 2 3))(λ 2 2 λ 2 3) +27λ 12 3 )(λ 2 2 λ 2 3), r 51 = λ 6 2(λ λ 4 2λ λ 2 2λ λ 6 3)(λ 2 2 λ 2 3) 2. Thus, f 5 0, because λ 1 λ 2 λ 3 > 0. From Lemma 1, the case in which f 5 = 0 occurs if and only if λ 1 = λ 2 = λ 3, or equivalently φ = 0. This contradicts 0 <φ< π 2. Consequently, f 5 > 0 holds, and thus the inequality λ 2 > 0is proved.
15 DECISION RULES FOR THE NUMBER OF FACTORS 189 (iii) λ 3 > 0: Formula (A.7) yields λ λ 2 2 > 3(λ2 1 λ2 2 (λ2 1 (λ2 2 λ2 3 )+λ2 2 (λ2 1 λ2 3 ))). We see this inequality holds, because the right hand side is less than or equal to 0. Thus, the inequality λ 3 > 0 is proved. A.6. Proof of Proposition 4 Since θ 1 =1+(p 1)α 2 and θ 2 =1 α 2, then θ 1 θ 2 = pα 2. (i) Case in which γ α: From Case (ii) of Lemma 2, we obtain θ 1 =1+1 2 ((p 1)α2 + (p 1) 2 α 4 +4pα 2 γ 2 ), θ 2 =1+1 2 ((p 1)α2 (p 1) 2 α 4 +4pα 2 γ 2 ), θ 1 θ 2 = α (p 1) 2 α 2 +4pγ 2. Therefore, θ 1 θ 2 >θ 1 θ 2 holds if and only if (p 1) 2 α 2 +4pγ 2 >p 2 α 2. Then we obtain γ> p α. (ii) Case in which γ>α: From Case (i) of Lemma 2, we have θ 1 θ 2 = 1 2 ((p +1)α2 + α (p 1) 2 α 2 +4pγ 2 ). Evaluating γ by α, wehave θ 1 θ 2 > 1 2 ((p +1)α2 + α (p 1) 2 α 2 +4pα 2 )=(p +1)α 2. Hence, we obtain (θ 1 θ 2 ) (θ 1 θ 2 ) >α 2 > 0. Combining Cases (i) and (ii), we obtain the conclusion. Acknowledgements The authors would like to thank Dr. Yasuhiro Ohta of Hiroshima University for his useful advice. They are grateful to anonymous reviewers whose comments and suggestions greatly improved the presentation of the paper. This research was partially supported by the Japan Society for the Promotion of Science, Grantin-Aid for Scientific Research under contract numbers and
16 190 MANABU SATO AND MASAAKI ITO References Anderson, T. W. (2003). An Introduction to Multivariate Statistical Analysis, 3rd ed., John Wiley & Sons. Anderson, T. W. and Rubin, H. (1956). Statistical inference in factor analysis, Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability (ed. J. Neyman), 5, pp Bentler, P. M. and Kano, Y. (1990). On the equivalence of factors and components, Multivariate Behavioral Research, 25, ten Berge, J. M. F. and Kiers, H. A. L. (1991). A numerical approach to the approximate and the exact minimum rank of a covariance matrix, Psychometrika, 56, Beyer, W. H. (1987). CRC Standard Mathematical Tables, 28th ed., CRC Press. Cattell, R. B. (1966). The scree test for the number of factors, Multivariate Behavioral Research, 1, Chatfield, C. and Collins, A. J. (1989). Introduction to Multivariate Analysis, Chapman and Hall. Hoyle, R. H. and Duvall, J. L. (2004). Determining the number of factors in exploratory and confirmatory factor analysis, The Sage Handbook of Quantitative Methodology for the Social Sciences (ed. D. Kaplan), pp , Sage Publications. Jolliffe, I. T. (2002). Principal Component Analysis, 2nd ed., Springer-Verlag. Lawley, D. N. and Maxwell, A. E. (1971). Factor Analysis as a Statistical Method, 2nd ed., Butterworth. Ogasawara, H. (2000). Some relationships between factors and components, Psychometrika, 65, Rao, C. R. (1973). Linear Statistical Inference and its Applications, 2nd ed., John Wiley & Sons. Sato, M. (1990). Some remarks on principal component analysis as a substitute for factor analysis in mono-factor cases, J. Japan Statist. Soc., 20, Sato, M. (1992). A study of an identification problem and substitute use of principal component analysis in factor analysis, Hiroshima Mathematical Journal, 22, Sato, M. and Ito, M. (2003). Some cautionary notes on factor loadings estimated by principal component analysis, New Developments in Psychometrics (eds. H. Yanai et al.), pp , Springer-Verlag. Schneeweiss, H. (1997). Factors and principal components in the near spherical case, Multivariate Behavioral Research, 32, Schneeweiss, H. and Mathes, H. (1995). Factor analysis and principal component analysis, J. Multivariate Anal., 55, Spearman, C. (1904). General Intelligence, objectively determined and measured, American Journal of Psychology, 15, Wolfram, S. (1996). The Mathematica Book, 3rd ed., Wolfram Media/Cambridge University Press.
Chapter 4: Factor Analysis
Chapter 4: Factor Analysis In many studies, we may not be able to measure directly the variables of interest. We can merely collect data on other variables which may be related to the variables of interest.
More information2/26/2017. This is similar to canonical correlation in some ways. PSY 512: Advanced Statistics for Psychological and Behavioral Research 2
PSY 512: Advanced Statistics for Psychological and Behavioral Research 2 What is factor analysis? What are factors? Representing factors Graphs and equations Extracting factors Methods and criteria Interpreting
More informationFACTOR ANALYSIS AS MATRIX DECOMPOSITION 1. INTRODUCTION
FACTOR ANALYSIS AS MATRIX DECOMPOSITION JAN DE LEEUW ABSTRACT. Meet the abstract. This is the abstract. 1. INTRODUCTION Suppose we have n measurements on each of taking m variables. Collect these measurements
More informationImproper Solutions in Exploratory Factor Analysis: Causes and Treatments
Improper Solutions in Exploratory Factor Analysis: Causes and Treatments Yutaka Kano Faculty of Human Sciences, Osaka University Suita, Osaka 565, Japan. email: kano@hus.osaka-u.ac.jp Abstract: There are
More information1 A factor can be considered to be an underlying latent variable: (a) on which people differ. (b) that is explained by unknown variables
1 A factor can be considered to be an underlying latent variable: (a) on which people differ (b) that is explained by unknown variables (c) that cannot be defined (d) that is influenced by observed variables
More informationarxiv: v1 [math.co] 3 Nov 2014
SPARSE MATRICES DESCRIBING ITERATIONS OF INTEGER-VALUED FUNCTIONS BERND C. KELLNER arxiv:1411.0590v1 [math.co] 3 Nov 014 Abstract. We consider iterations of integer-valued functions φ, which have no fixed
More informationRetained-Components Factor Transformation: Factor Loadings and Factor Score Predictors in the Column Space of Retained Components
Journal of Modern Applied Statistical Methods Volume 13 Issue 2 Article 6 11-2014 Retained-Components Factor Transformation: Factor Loadings and Factor Score Predictors in the Column Space of Retained
More informationIntroduction to Factor Analysis
to Factor Analysis Lecture 10 August 2, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Lecture #10-8/3/2011 Slide 1 of 55 Today s Lecture Factor Analysis Today s Lecture Exploratory
More informationConsistent Bivariate Distribution
A Characterization of the Normal Conditional Distributions MATSUNO 79 Therefore, the function ( ) = G( : a/(1 b2)) = N(0, a/(1 b2)) is a solu- tion for the integral equation (10). The constant times of
More informationPrinciple Components Analysis (PCA) Relationship Between a Linear Combination of Variables and Axes Rotation for PCA
Principle Components Analysis (PCA) Relationship Between a Linear Combination of Variables and Axes Rotation for PCA Principle Components Analysis: Uses one group of variables (we will call this X) In
More informationFactor Analysis. Robert L. Wolpert Department of Statistical Science Duke University, Durham, NC, USA
Factor Analysis Robert L. Wolpert Department of Statistical Science Duke University, Durham, NC, USA 1 Factor Models The multivariate regression model Y = XB +U expresses each row Y i R p as a linear combination
More informationEstimation of Unique Variances Using G-inverse Matrix in Factor Analysis
International Mathematical Forum, 3, 2008, no. 14, 671-676 Estimation of Unique Variances Using G-inverse Matrix in Factor Analysis Seval Süzülmüş Osmaniye Korkut Ata University Vocational High School
More informationIntroduction to Factor Analysis
to Factor Analysis Lecture 11 November 2, 2005 Multivariate Analysis Lecture #11-11/2/2005 Slide 1 of 58 Today s Lecture Factor Analysis. Today s Lecture Exploratory factor analysis (EFA). Confirmatory
More informationExploratory Factor Analysis: dimensionality and factor scores. Psychology 588: Covariance structure and factor models
Exploratory Factor Analysis: dimensionality and factor scores Psychology 588: Covariance structure and factor models How many PCs to retain 2 Unlike confirmatory FA, the number of factors to extract is
More informationCausal Inference Using Nonnormality Yutaka Kano and Shohei Shimizu 1
Causal Inference Using Nonnormality Yutaka Kano and Shohei Shimizu 1 Path analysis, often applied to observational data to study causal structures, describes causal relationship between observed variables.
More informationTAMS39 Lecture 10 Principal Component Analysis Factor Analysis
TAMS39 Lecture 10 Principal Component Analysis Factor Analysis Martin Singull Department of Mathematics Mathematical Statistics Linköping University, Sweden Content - Lecture Principal component analysis
More informationChapter 2 Polynomial and Rational Functions
Chapter 2 Polynomial and Rational Functions Overview: 2.2 Polynomial Functions of Higher Degree 2.3 Real Zeros of Polynomial Functions 2.4 Complex Numbers 2.5 The Fundamental Theorem of Algebra 2.6 Rational
More informationFACTOR ANALYSIS AND MULTIDIMENSIONAL SCALING
FACTOR ANALYSIS AND MULTIDIMENSIONAL SCALING Vishwanath Mantha Department for Electrical and Computer Engineering Mississippi State University, Mississippi State, MS 39762 mantha@isip.msstate.edu ABSTRACT
More informationApplied Multivariate Analysis
Department of Mathematics and Statistics, University of Vaasa, Finland Spring 2017 Dimension reduction Exploratory (EFA) Background While the motivation in PCA is to replace the original (correlated) variables
More informationCumulative Review. Name. 13) 2x = -4 13) SHORT ANSWER. Write the word or phrase that best completes each statement or answers the question.
Cumulative Review Name SHORT ANSWER. Write the word or phrase that best completes each statement or answers the question. Evaluate the algebraic expression for the given value or values of the variable(s).
More informationMAXIMUM LIKELIHOOD IN GENERALIZED FIXED SCORE FACTOR ANALYSIS 1. INTRODUCTION
MAXIMUM LIKELIHOOD IN GENERALIZED FIXED SCORE FACTOR ANALYSIS JAN DE LEEUW ABSTRACT. We study the weighted least squares fixed rank approximation problem in which the weight matrices depend on unknown
More informationThe nfactors Package
The nfactors Package July 10, 2006 Type Package Title Non Graphical Solution to the Cattell Scree Test Version 1.0 Date 2006-06-26 Author Gilles Raiche Maintainer Gilles Raiche
More informationRegularized Common Factor Analysis
New Trends in Psychometrics 1 Regularized Common Factor Analysis Sunho Jung 1 and Yoshio Takane 1 (1) Department of Psychology, McGill University, 1205 Dr. Penfield Avenue, Montreal, QC, H3A 1B1, Canada
More informationRV Coefficient and Congruence Coefficient
RV Coefficient and Congruence Coefficient Hervé Abdi 1 1 Overview The congruence coefficient was first introduced by Burt (1948) under the name of unadjusted correlation as a measure of the similarity
More informationSTAT 730 Chapter 9: Factor analysis
STAT 730 Chapter 9: Factor analysis Timothy Hanson Department of Statistics, University of South Carolina Stat 730: Multivariate Data Analysis 1 / 15 Basic idea Factor analysis attempts to explain the
More informationMULTIVARIATE TIME SERIES ANALYSIS AN ADAPTATION OF BOX-JENKINS METHODOLOGY Joseph N Ladalla University of Illinois at Springfield, Springfield, IL
MULTIVARIATE TIME SERIES ANALYSIS AN ADAPTATION OF BOX-JENKINS METHODOLOGY Joseph N Ladalla University of Illinois at Springfield, Springfield, IL KEYWORDS: Multivariate time series, Box-Jenkins ARIMA
More informationMultivariate Statistical Analysis
Multivariate Statistical Analysis Fall 2011 C. L. Williams, Ph.D. Lecture 4 for Applied Multivariate Analysis Outline 1 Eigen values and eigen vectors Characteristic equation Some properties of eigendecompositions
More information22m:033 Notes: 7.1 Diagonalization of Symmetric Matrices
m:33 Notes: 7. Diagonalization of Symmetric Matrices Dennis Roseman University of Iowa Iowa City, IA http://www.math.uiowa.edu/ roseman May 3, Symmetric matrices Definition. A symmetric matrix is a matrix
More informationON A WEIGHTED INTERPOLATION OF FUNCTIONS WITH CIRCULAR MAJORANT
ON A WEIGHTED INTERPOLATION OF FUNCTIONS WITH CIRCULAR MAJORANT Received: 31 July, 2008 Accepted: 06 February, 2009 Communicated by: SIMON J SMITH Department of Mathematics and Statistics La Trobe University,
More informationThe Singular Value Decomposition (SVD) and Principal Component Analysis (PCA)
Chapter 5 The Singular Value Decomposition (SVD) and Principal Component Analysis (PCA) 5.1 Basics of SVD 5.1.1 Review of Key Concepts We review some key definitions and results about matrices that will
More informationFORMAL GROUPS OF CERTAIN Q-CURVES OVER QUADRATIC FIELDS
Sairaiji, F. Osaka J. Math. 39 (00), 3 43 FORMAL GROUPS OF CERTAIN Q-CURVES OVER QUADRATIC FIELDS FUMIO SAIRAIJI (Received March 4, 000) 1. Introduction Let be an elliptic curve over Q. We denote by ˆ
More informationCS281 Section 4: Factor Analysis and PCA
CS81 Section 4: Factor Analysis and PCA Scott Linderman At this point we have seen a variety of machine learning models, with a particular emphasis on models for supervised learning. In particular, we
More informationConcentration Ellipsoids
Concentration Ellipsoids ECE275A Lecture Supplement Fall 2008 Kenneth Kreutz Delgado Electrical and Computer Engineering Jacobs School of Engineering University of California, San Diego VERSION LSECE275CE
More informationFactor Analysis Continued. Psy 524 Ainsworth
Factor Analysis Continued Psy 524 Ainsworth Equations Extraction Principal Axis Factoring Variables Skiers Cost Lift Depth Powder S1 32 64 65 67 S2 61 37 62 65 S3 59 40 45 43 S4 36 62 34 35 S5 62 46 43
More informationTesting Some Covariance Structures under a Growth Curve Model in High Dimension
Department of Mathematics Testing Some Covariance Structures under a Growth Curve Model in High Dimension Muni S. Srivastava and Martin Singull LiTH-MAT-R--2015/03--SE Department of Mathematics Linköping
More informationCSL361 Problem set 4: Basic linear algebra
CSL361 Problem set 4: Basic linear algebra February 21, 2017 [Note:] If the numerical matrix computations turn out to be tedious, you may use the function rref in Matlab. 1 Row-reduced echelon matrices
More informationReview (Probability & Linear Algebra)
Review (Probability & Linear Algebra) CE-725 : Statistical Pattern Recognition Sharif University of Technology Spring 2013 M. Soleymani Outline Axioms of probability theory Conditional probability, Joint
More informationNumerical Analysis: Solving Systems of Linear Equations
Numerical Analysis: Solving Systems of Linear Equations Mirko Navara http://cmpfelkcvutcz/ navara/ Center for Machine Perception, Department of Cybernetics, FEE, CTU Karlovo náměstí, building G, office
More informationA classification of sharp tridiagonal pairs. Tatsuro Ito, Kazumasa Nomura, Paul Terwilliger
Tatsuro Ito Kazumasa Nomura Paul Terwilliger Overview This talk concerns a linear algebraic object called a tridiagonal pair. We will describe its features such as the eigenvalues, dual eigenvalues, shape,
More informationMath 215 HW #9 Solutions
Math 5 HW #9 Solutions. Problem 4.4.. If A is a 5 by 5 matrix with all a ij, then det A. Volumes or the big formula or pivots should give some upper bound on the determinant. Answer: Let v i be the ith
More informationCan Variances of Latent Variables be Scaled in Such a Way That They Correspond to Eigenvalues?
International Journal of Statistics and Probability; Vol. 6, No. 6; November 07 ISSN 97-703 E-ISSN 97-7040 Published by Canadian Center of Science and Education Can Variances of Latent Variables be Scaled
More informationFactor analysis. George Balabanis
Factor analysis George Balabanis Key Concepts and Terms Deviation. A deviation is a value minus its mean: x - mean x Variance is a measure of how spread out a distribution is. It is computed as the average
More informationMATH 425-Spring 2010 HOMEWORK ASSIGNMENTS
MATH 425-Spring 2010 HOMEWORK ASSIGNMENTS Instructor: Shmuel Friedland Department of Mathematics, Statistics and Computer Science email: friedlan@uic.edu Last update April 18, 2010 1 HOMEWORK ASSIGNMENT
More informationLEAST SQUARES METHODS FOR FACTOR ANALYSIS. 1. Introduction
LEAST SQUARES METHODS FOR FACTOR ANALYSIS JAN DE LEEUW AND JIA CHEN Abstract. Meet the abstract. This is the abstract. 1. Introduction Suppose we have n measurements on each of m variables. Collect these
More informationMath 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination
Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column
More informationMore Polynomial Equations Section 6.4
MATH 11009: More Polynomial Equations Section 6.4 Dividend: The number or expression you are dividing into. Divisor: The number or expression you are dividing by. Synthetic division: Synthetic division
More informationMULTIVARIATE ANALYSIS OF VARIANCE UNDER MULTIPLICITY José A. Díaz-García. Comunicación Técnica No I-07-13/ (PE/CIMAT)
MULTIVARIATE ANALYSIS OF VARIANCE UNDER MULTIPLICITY José A. Díaz-García Comunicación Técnica No I-07-13/11-09-2007 (PE/CIMAT) Multivariate analysis of variance under multiplicity José A. Díaz-García Universidad
More informationMathematical foundations - linear algebra
Mathematical foundations - linear algebra Andrea Passerini passerini@disi.unitn.it Machine Learning Vector space Definition (over reals) A set X is called a vector space over IR if addition and scalar
More information(4) Statements that contain the word "including" reference content that must be
111.40. Algebra II, Adopted 2012 (One-Half to One Credit). (a) General requirements. Students shall be awarded one-half to one credit for successful completion of this course. Prerequisite: Algebra I.
More informationSTATISTICAL LEARNING SYSTEMS
STATISTICAL LEARNING SYSTEMS LECTURE 8: UNSUPERVISED LEARNING: FINDING STRUCTURE IN DATA Institute of Computer Science, Polish Academy of Sciences Ph. D. Program 2013/2014 Principal Component Analysis
More informationConceptual Questions for Review
Conceptual Questions for Review Chapter 1 1.1 Which vectors are linear combinations of v = (3, 1) and w = (4, 3)? 1.2 Compare the dot product of v = (3, 1) and w = (4, 3) to the product of their lengths.
More informationConsistency of Test-based Criterion for Selection of Variables in High-dimensional Two Group-Discriminant Analysis
Consistency of Test-based Criterion for Selection of Variables in High-dimensional Two Group-Discriminant Analysis Yasunori Fujikoshi and Tetsuro Sakurai Department of Mathematics, Graduate School of Science,
More informationA Characterization of Distance-Regular Graphs with Diameter Three
Journal of Algebraic Combinatorics 6 (1997), 299 303 c 1997 Kluwer Academic Publishers. Manufactured in The Netherlands. A Characterization of Distance-Regular Graphs with Diameter Three EDWIN R. VAN DAM
More information1. Matrix multiplication and Pauli Matrices: Pauli matrices are the 2 2 matrices. 1 0 i 0. 0 i
Problems in basic linear algebra Science Academies Lecture Workshop at PSGRK College Coimbatore, June 22-24, 2016 Govind S. Krishnaswami, Chennai Mathematical Institute http://www.cmi.ac.in/~govind/teaching,
More information= W z1 + W z2 and W z1 z 2
Math 44 Fall 06 homework page Math 44 Fall 06 Darij Grinberg: homework set 8 due: Wed, 4 Dec 06 [Thanks to Hannah Brand for parts of the solutions] Exercise Recall that we defined the multiplication of
More informationMidterm for Introduction to Numerical Analysis I, AMSC/CMSC 466, on 10/29/2015
Midterm for Introduction to Numerical Analysis I, AMSC/CMSC 466, on 10/29/2015 The test lasts 1 hour and 15 minutes. No documents are allowed. The use of a calculator, cell phone or other equivalent electronic
More informationA note on structured means analysis for a single group. André Beauducel 1. October 3 rd, 2015
Structured means analysis for a single group 1 A note on structured means analysis for a single group André Beauducel 1 October 3 rd, 2015 Abstract The calculation of common factor means in structured
More informationMore Powerful Tests for Homogeneity of Multivariate Normal Mean Vectors under an Order Restriction
Sankhyā : The Indian Journal of Statistics 2007, Volume 69, Part 4, pp. 700-716 c 2007, Indian Statistical Institute More Powerful Tests for Homogeneity of Multivariate Normal Mean Vectors under an Order
More informationRemarks on the Cayley Representation of Orthogonal Matrices and on Perturbing the Diagonal of a Matrix to Make it Invertible
Remarks on the Cayley Representation of Orthogonal Matrices and on Perturbing the Diagonal of a Matrix to Make it Invertible Jean Gallier Department of Computer and Information Science University of Pennsylvania
More informationMultilevel Analysis, with Extensions
May 26, 2010 We start by reviewing the research on multilevel analysis that has been done in psychometrics and educational statistics, roughly since 1985. The canonical reference (at least I hope so) is
More informationReview (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology
Review (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Some slides have been adopted from Prof. H.R. Rabiee s and also Prof. R. Gutierrez-Osuna
More informationSemester Review Packet
MATH 110: College Algebra Instructor: Reyes Semester Review Packet Remarks: This semester we have made a very detailed study of four classes of functions: Polynomial functions Linear Quadratic Higher degree
More information1.1 Limits and Continuity. Precise definition of a limit and limit laws. Squeeze Theorem. Intermediate Value Theorem. Extreme Value Theorem.
STATE EXAM MATHEMATICS Variant A ANSWERS AND SOLUTIONS 1 1.1 Limits and Continuity. Precise definition of a limit and limit laws. Squeeze Theorem. Intermediate Value Theorem. Extreme Value Theorem. Definition
More informationLinear Dimensionality Reduction
Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Introduction 2 Principal Component Analysis 3 Factor Analysis
More informationVectors and Matrices Statistics with Vectors and Matrices
Vectors and Matrices Statistics with Vectors and Matrices Lecture 3 September 7, 005 Analysis Lecture #3-9/7/005 Slide 1 of 55 Today s Lecture Vectors and Matrices (Supplement A - augmented with SAS proc
More informationLECTURE 4 PRINCIPAL COMPONENTS ANALYSIS / EXPLORATORY FACTOR ANALYSIS
LECTURE 4 PRINCIPAL COMPONENTS ANALYSIS / EXPLORATORY FACTOR ANALYSIS NOTES FROM PRE- LECTURE RECORDING ON PCA PCA and EFA have similar goals. They are substantially different in important ways. The goal
More informationAnalysis of the AIC Statistic for Optimal Detection of Small Changes in Dynamic Systems
Analysis of the AIC Statistic for Optimal Detection of Small Changes in Dynamic Systems Jeremy S. Conner and Dale E. Seborg Department of Chemical Engineering University of California, Santa Barbara, CA
More informationK-ANTITHETIC VARIATES IN MONTE CARLO SIMULATION ISSN k-antithetic Variates in Monte Carlo Simulation Abdelaziz Nasroallah, pp.
K-ANTITHETIC VARIATES IN MONTE CARLO SIMULATION ABDELAZIZ NASROALLAH Abstract. Standard Monte Carlo simulation needs prohibitive time to achieve reasonable estimations. for untractable integrals (i.e.
More informationMaths for Signals and Systems Linear Algebra in Engineering
Maths for Signals and Systems Linear Algebra in Engineering Lectures 13 15, Tuesday 8 th and Friday 11 th November 016 DR TANIA STATHAKI READER (ASSOCIATE PROFFESOR) IN SIGNAL PROCESSING IMPERIAL COLLEGE
More informationUnderstanding hard cases in the general class group algorithm
Understanding hard cases in the general class group algorithm Makoto Suwama Supervisor: Dr. Steve Donnelly The University of Sydney February 2014 1 Introduction This report has studied the general class
More informationAn Introduction to Multivariate Statistical Analysis
An Introduction to Multivariate Statistical Analysis Third Edition T. W. ANDERSON Stanford University Department of Statistics Stanford, CA WILEY- INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION Contents
More informationMath 121 Calculus 1 Fall 2009 Outcomes List for Final Exam
Math 121 Calculus 1 Fall 2009 Outcomes List for Final Exam This outcomes list summarizes what skills and knowledge you should have reviewed and/or acquired during this entire quarter in Math 121, and what
More informationMiller Objectives Alignment Math
Miller Objectives Alignment Math 1050 1 College Algebra Course Objectives Spring Semester 2016 1. Use algebraic methods to solve a variety of problems involving exponential, logarithmic, polynomial, and
More informationMath 307 Learning Goals. March 23, 2010
Math 307 Learning Goals March 23, 2010 Course Description The course presents core concepts of linear algebra by focusing on applications in Science and Engineering. Examples of applications from recent
More informationApplied Linear Algebra in Geoscience Using MATLAB
Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in
More information2. The Concept of Convergence: Ultrafilters and Nets
2. The Concept of Convergence: Ultrafilters and Nets NOTE: AS OF 2008, SOME OF THIS STUFF IS A BIT OUT- DATED AND HAS A FEW TYPOS. I WILL REVISE THIS MATE- RIAL SOMETIME. In this lecture we discuss two
More informationWEIGHTED COMPOSITION OPERATORS BETWEEN H AND THE BLOCH SPACE. Sh^uichi Ohno 1. INTRODUCTION
TAIWANESE JOURNAL OF MATHEMATICS Vol. 5, No. 3, pp. 555-563, September 2001 This paper is available online at http://www.math.nthu.edu.tw/tjm/ WEIGHTED COMPOSITION OPERATORS BETWEEN H AND THE BLOCH SPACE
More informationMath 315: Linear Algebra Solutions to Assignment 7
Math 5: Linear Algebra s to Assignment 7 # Find the eigenvalues of the following matrices. (a.) 4 0 0 0 (b.) 0 0 9 5 4. (a.) The characteristic polynomial det(λi A) = (λ )(λ )(λ ), so the eigenvalues are
More informationHigh-dimensional two-sample tests under strongly spiked eigenvalue models
1 High-dimensional two-sample tests under strongly spiked eigenvalue models Makoto Aoshima and Kazuyoshi Yata University of Tsukuba Abstract: We consider a new two-sample test for high-dimensional data
More informationVectors To begin, let us describe an element of the state space as a point with numerical coordinates, that is x 1. x 2. x =
Linear Algebra Review Vectors To begin, let us describe an element of the state space as a point with numerical coordinates, that is x 1 x x = 2. x n Vectors of up to three dimensions are easy to diagram.
More informationComputing a Lower Bound for the Canonical Height on Elliptic Curves over Q
Computing a Lower Bound for the Canonical Height on Elliptic Curves over Q John Cremona 1 and Samir Siksek 2 1 School of Mathematical Sciences, University of Nottingham, University Park, Nottingham NG7
More informationSparse orthogonal factor analysis
Sparse orthogonal factor analysis Kohei Adachi and Nickolay T. Trendafilov Abstract A sparse orthogonal factor analysis procedure is proposed for estimating the optimal solution with sparse loadings. In
More informationMTH 2032 SemesterII
MTH 202 SemesterII 2010-11 Linear Algebra Worked Examples Dr. Tony Yee Department of Mathematics and Information Technology The Hong Kong Institute of Education December 28, 2011 ii Contents Table of Contents
More informationFixed Effects, Invariance, and Spatial Variation in Intergenerational Mobility
American Economic Review: Papers & Proceedings 2016, 106(5): 400 404 http://dx.doi.org/10.1257/aer.p20161082 Fixed Effects, Invariance, and Spatial Variation in Intergenerational Mobility By Gary Chamberlain*
More informationDiscussion of Hypothesis testing by convex optimization
Electronic Journal of Statistics Vol. 9 (2015) 1 6 ISSN: 1935-7524 DOI: 10.1214/15-EJS990 Discussion of Hypothesis testing by convex optimization Fabienne Comte, Céline Duval and Valentine Genon-Catalot
More informationMINIMAL NORMAL AND COMMUTING COMPLETIONS
INTERNATIONAL JOURNAL OF INFORMATION AND SYSTEMS SCIENCES Volume 4, Number 1, Pages 5 59 c 8 Institute for Scientific Computing and Information MINIMAL NORMAL AND COMMUTING COMPLETIONS DAVID P KIMSEY AND
More informationChapter 3: Root Finding. September 26, 2005
Chapter 3: Root Finding September 26, 2005 Outline 1 Root Finding 2 3.1 The Bisection Method 3 3.2 Newton s Method: Derivation and Examples 4 3.3 How To Stop Newton s Method 5 3.4 Application: Division
More informationAn alternating optimization algorithm for two-channel factor analysis with common and uncommon factors
An alternating optimization algorithm for two-channel factor analysis with common and uncommon factors David Ramírez University Carlos III of Madrid, Spain Gregorio Marañón Health Research Institute, Spain
More informationIn Z: x + 3 = 2 3x = 2 x = 1 No solution In Q: 3x = 2 x 2 = 2. x = 2 No solution. In R: x 2 = 2 x = 0 x = ± 2 No solution Z Q.
THE UNIVERSITY OF NEW SOUTH WALES SCHOOL OF MATHEMATICS AND STATISTICS MATH 1141 HIGHER MATHEMATICS 1A ALGEBRA. Section 1: - Complex Numbers. 1. The Number Systems. Let us begin by trying to solve various
More informationPrincipal Component Analysis (PCA) Theory, Practice, and Examples
Principal Component Analysis (PCA) Theory, Practice, and Examples Data Reduction summarization of data with many (p) variables by a smaller set of (k) derived (synthetic, composite) variables. p k n A
More informationResearch Article Minor Prime Factorization for n-d Polynomial Matrices over Arbitrary Coefficient Field
Complexity, Article ID 6235649, 9 pages https://doi.org/10.1155/2018/6235649 Research Article Minor Prime Factorization for n-d Polynomial Matrices over Arbitrary Coefficient Field Jinwang Liu, Dongmei
More informationMaximal perpendicularity in certain Abelian groups
Acta Univ. Sapientiae, Mathematica, 9, 1 (2017) 235 247 DOI: 10.1515/ausm-2017-0016 Maximal perpendicularity in certain Abelian groups Mika Mattila Department of Mathematics, Tampere University of Technology,
More informationClassification of root systems
Classification of root systems September 8, 2017 1 Introduction These notes are an approximate outline of some of the material to be covered on Thursday, April 9; Tuesday, April 14; and Thursday, April
More informationTORIC WEAK FANO VARIETIES ASSOCIATED TO BUILDING SETS
TORIC WEAK FANO VARIETIES ASSOCIATED TO BUILDING SETS YUSUKE SUYAMA Abstract. We give a necessary and sufficient condition for the nonsingular projective toric variety associated to a building set to be
More informationMultivariate Statistics (I) 2. Principal Component Analysis (PCA)
Multivariate Statistics (I) 2. Principal Component Analysis (PCA) 2.1 Comprehension of PCA 2.2 Concepts of PCs 2.3 Algebraic derivation of PCs 2.4 Selection and goodness-of-fit of PCs 2.5 Algebraic derivation
More informationIntroduction to Matrix Algebra
Introduction to Matrix Algebra August 18, 2010 1 Vectors 1.1 Notations A p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the line. When p
More informationLOWER BOUNDS FOR THE MAXIMUM NUMBER OF SOLUTIONS GENERATED BY THE SIMPLEX METHOD
Journal of the Operations Research Society of Japan Vol 54, No 4, December 2011, pp 191 200 c The Operations Research Society of Japan LOWER BOUNDS FOR THE MAXIMUM NUMBER OF SOLUTIONS GENERATED BY THE
More informationDependence. Practitioner Course: Portfolio Optimization. John Dodson. September 10, Dependence. John Dodson. Outline.
Practitioner Course: Portfolio Optimization September 10, 2008 Before we define dependence, it is useful to define Random variables X and Y are independent iff For all x, y. In particular, F (X,Y ) (x,
More informationHessenberg Pairs of Linear Transformations
Hessenberg Pairs of Linear Transformations Ali Godjali November 21, 2008 arxiv:0812.0019v1 [math.ra] 28 Nov 2008 Abstract Let K denote a field and V denote a nonzero finite-dimensional vector space over
More informationPrincipal Component Analysis & Factor Analysis. Psych 818 DeShon
Principal Component Analysis & Factor Analysis Psych 818 DeShon Purpose Both are used to reduce the dimensionality of correlated measurements Can be used in a purely exploratory fashion to investigate
More information