arxiv: v2 [math.na] 27 Dec 2016

Transcription

1 An algorithm for constructing Equiangular vectors Azim rivaz a,, Danial Sadeghi a a Department of Mathematics, Shahid Bahonar University of Kerman, Kerman , IRAN arxiv: v2 [math.na] 27 Dec 2016 Abstract In this paper a generalization of the Gram-Schmidt Algorithm is presented. Actually we provide an algorithm to construct a set of equiangular vectors with the prescribed angle θ (0, π/2) using a set of independent vectors in R n. Then a new type of matrix decomposition is derived. Keywords: Gram-Schmidt Algorithm, Equiangular matrix, Equiangular vectors, Law of cosines, Matrix Decomposition 2010 MSC: 15B99, 15A03, 15A23, 15A30 1. Introduction We know that the Gram-Schmidt (GS) Algorithm has numerous applications in applied mathematics such as the theory of least squares, Projection methods and Eigenvalue problems. This algorithm takes a set of linearly independent vectors {a 1, a 2,..., a k } of a n dimensional space (k n), and then generates a set of orthogonal vectors {q 1, q 2,..., q k } successively step-by-step so that they span the same subspaces [1, 2, 6, 7, 8]. If this algorithm is applied to the column vectors of a nonsingular n n matrix A = [a 1,..., a n ], then the obtained vectors q 1, q 2,..., q n are orthogonal and make the orthogonal matrix Q column-wise. The outcome of GS algorithm is the QR decomposition A = QR, where R is an upper triangular matrix. The purpose of this paper is to generalize the GS algorithm by providing an algorithm that generate linearly independent vectors s 1, s 2,..., s n using a 1, a 2,..., a n, provided that s T i s j is equal to α (0, 1) if i j and 1 if i = j. It means the s i s are equiangular with angle θ that θ = arccos α (0, π/2). In this paper we deal with subspaces of R n with the standard inner product and Euclidean norm. Throughout this paper, x denotes the Euclidean norm of vector x and α = cos θ. Also e = [1,..., 1] T R n is the vector of ones and {e 1, e 2,..., e n } is the standard orthogonal basis of R n. Moreover, All matrices in this paper are real. Proposition 1.1. Suppose that three vectors v 1, v 2, v 3 in R 3 are constructed by linear combination of the vector e and the basis {e 1, e 2, e 3 } as v 1 = [1, x, x] T, v 2 = [x, 1, x] T and v 3 = [x, x, 1] T. Then the vectors s i = v i / v i for i = 1, 2, 3 are equiangular with α = cos θ (0, 1), where x = ( (1 α)(1 + 2α) 1)/(1 2α). Proof. The proof is clear by obtaining s T i s j = (x 2 + 2x)/(2x 2 + 1) = α for i j. A well-ordering representation of an equiangular basis in R n analogous to the basis {e 1, e 2,..., e n } is introduced in the following lemma. Proposition 1.2. Let v i = xe + (1 x)e i for i = 1,..., n. Then the set of {s 1,..., s n } where s i = v i / v i (1 α)(1+(n 1)α) 1 are equiangular vectors with α (0, 1) and x = (1 α)(n 1) 1. Moreover if the vectors s 1,..., s n are 1 embedded into the positive coordinate axes so that n 1 entries of all s i are equal to t (0, n ) and the last one is s = 1 (n 1)t 2. Then s = (1 α)(n 1)(n 2)+n±2(n 1) (1 α)(1+(n 1)α) αn+2(1 α (1 α)(1+(n 1)α)) n, t = n, (1.1) so that the plus sign in the formula of s must be selected and minus sign in t. Corresponding author. arivaz@uk.ac.ir. address: dl.sadeghi@math.uk.ac.ir (Danial Sadeghi) Preprint submitted to Elsevier September 3, 2018

2 Proof. We can check that s T i s 2x + (n 2)x2 j = 1 + (n 1)x 2 = α for i j. For the next part we must have s2 +(n 1)t 2 = 1 and α = 2ts + (n 2)t 2. Then we can verify the Equation (1.1). On the other hand, the scalar t tends to zero as α tends to zero, which justifies the minus sign in Equation (1.1) and vise versa in s. Proposition 1.3. Any set of equiangular vectors {s 1,..., s n } in R n with α ( 1 n 1, 1) is linearly independent. Proof. let x 1 s x n s n = 0, for arbitrary scalars x i. Premultiplying by s T i s gives a system of linear equations Gx = 0, where x = [x 1, x 2,..., x n ] T and G has ones on the main diagonal and cos θ on all off-diagonals. G is a Gram matrix of matrix S = [s 1,..., s n ] [4, 5], and G has no zero eigenvalue, so it is nonsingular and x 0. In this case we have θ (0, arccos( 1 1 n 1 )) and α ( n 1, 1). If the upper bound of α holds, then G = eet which is rank-one and singular. If the lower bound of α holds, then G is singular with the eigenpair (0, e) with algebraic multiplicity one at 0. So the rank of G and S is n 1 and then s 1,..., s n are equiangular but linearly dependent. Actually they are equiangular lines [3]. This paper is organized as follows. In section 2, using the basis {e 1, e 2..., e n }, we construct the so-called standard equiangular basis {s 1, s 2..., s n } with the angle θ. Next we construct the equiangular vectors of n- dimensional case using a set of linearly independent vectors. Then we have A = S.R so that the column vectors of A and S are the input and output vectors of the algorithm and R is an upper triangular matrix. Lemma 1.4. Suppose that s 1, s 2,..., s k are equiangular normalized vectors in R n (k n) with α arccos( 1 n 1, 1). Then k s i = k (1 + (k 1)α). Proof. The proof is clear by obtaining k s i 2 = ( k s i) T ( k s i) = k +k(k 1)α = k(1+(k 1)α). The next problem is finding the relationship between the two angles ϕ and η, where ϕ is the angle of the vectors k s i and s k+1, and η is the angle of the vectors k s i and s j (1 j k < n). An interesting triangular relationship is obtained as follows ( k ) ( s T k+1 s k ) i cos ϕ = s k+1 k s i = kα s T j s i, cos η = k (1 + (k 1)α) s j k s i = 1 + (k 1)α. (1.2) k (1 + (k 1)α) Multiplying the above equalities together implies that cos ϕ cos η = cos θ = α. (1.3) Indeed ϕ, η and θ are the angles of a trihedral angle, in such a way that two faces corresponding to ϕ and η are perpendicular together. Actually (1.3) represents the first law of cosine in the spherical pyramid [9]: Lemma 1.5. Let a, b and c are three linearly independent vectors in R n (n 3) which make a trihedral angle with the angles ϕ, η and θ and also the dihedral angle between two faces opposite to θ is π/2 as illustrated in Figure 1. Then cos θ = cos ϕ cos η. Proof. cos θ = at c a c = (proj ba + (a proj b a)) T (proj b c + (c proj b c)) = (proj ba)(proj b c) = cos ϕ cos η. (1.4) a c a c 2. An algorithm for producing a set of equiangular vectors Suppose that {a 1, a 2,..., a n } is a set of linearly independent vectors in R n. In this section we drive a set of n equiangular vectors with angle θ using the given vectors. First, we normalize a 1 as s 1 = a 1 1 a 1. Next, suppose that vector a 2 makes an acute angle with a 1. Let u 2 = a 2 s 1 s T 1 a 2, b = cot θ u 2 s 1 and v 2 = a 2 (s 1 s T 1 a 2 b). Then v 2 = u 2 + cot θ u 2 s 1 (Fig. 2). Without loss of generality we can write v 2 = q 2 + cot θ s 1, (2.1) where q 2 = u 2 1 u 2 is orthogonal to s 1. So the next normalized equiangular vector spanned by {a 1, a 2 } is s 2 = v 2 / v 2. For the case of obtuse angles between a 1 and a 2 we can use the same method with some slight differences. We now ready to present the main theorem as follows 2

3 Figure 1: Three vectors a, b and c form a pyramid, so that two faces which corresponding to ϕ and η are perpendicular. Theorem 2.1. Suppose that the vectors a 1,..., a n in R n are linearly independent. For an angle θ the equiangular vectors s 1, s 2 which are generated from a 1, a 2 by (2.1) are given. If q k+1 span a 1,..., a k+1 is a normalized orthogonal vector to s i s (i = 1,..., k). Then for 2 k n 1 the (k + 1)th equiangular vector is obtained as s k+1 = v k+1 / v k+1 where v k+1 = q k+1 + ( ) k k s i (sec θ 1)(sec θ+k) k s i. (2.2) Proof. We define a subspace named P 2 span s 1, s 2, a 3 as P 2 = {p p span s 1, s 2, a 3, p T s 1 = p T s 2 }. So each vector p of P 2 has the same angle θ p with s 1 and s 2. Indeed P 2 and the subspace spanned by s 1 and s 2 are perpendicular and their intersection is a line spanned by s 1 + s 2. Since p T s 1 = p T s 2, then p T (s 1 s 2 ) = 0 and s 1 s 2 is the orthogonal complement of P 2 so dim(p 2 ) = 2. Let ā 3 be the projection of a 3 onto P 2, thus it makes the same angle with s 1 and s 2. We can write ā 3 = proj P2 a 3 = a 3 proj s1 s 2 a 3 (2.3) From (1.3) the vector named v 3 which makes angle θ with both s 1, s 2, is in subspace P 2 (v 3 P 2 ). Suppose that ϕ is the angle between v 3 and s 1 + s 2. We have ϕ < θ and from the definition of η in (1.3), η = θ/2. Moreover, cot θ is replaced by cot ϕ as follows cos ϕ = cos θ 2 cos θ cos η = = cos θ 1 + cos θ cos(θ/2), sin ϕ = cos2 (θ/2) cos 2 θ 2 cos θ, then cot ϕ = cos(θ/2) 1 + cos θ 2 cos2 θ. The purpose is to compute v 3 using ā 3. In comparison with the previous step (2.1), ā 3 and s 1 + s 2 can be considered as a 2 and s 1, respectively. Let u 3 = ā 3 (s 1 + s 2 )(s 1 + s 2 ) T ā 3 s 1 + s 2 2. Then from (2.3) u 3 = a 3 proj s1 s 2 a 3 (s 1 + s 2 )(s 1 + s 2 ) T (a 3 proj s1 s 2 a 3 ) s 1 + s 2 2 = a 3 proj s1 s 2 a 3 (s 1 + s 2 )(s 1 + s 2 ) T a 3 s 1 + s 2 2 = a 3 (proj s1 s 2 a 3 + proj s1+s 2 a 3 ). (2.4) Note that u 3 is orthogonal to s 1, s 2 because s 1 s 2 s 1 + s 2. Hence from (2.1) we have v 3 = u 3 + cot ϕ u 3 s 1 + s 2. Without loss of generality we can take s 1 + s 2 v 3 = q 3 + cot ϕ s 1 + s 2 s 1 + s 2, (2.5) where q 3 = u 3 1 u 3 which is orthogonal to s 1, s 2. So the next normalized equiangular vector to s 1, s 2 spanned by {a 1, a 2, a 3 } is s 3 = v 3 / v 3. After k steps the equiangular vectors s 1,..., s k will be produced (k 3). For obtaining s k+1 we begin with defining P k as P k = P ij = { p p span s1,..., s k, a k+1, p (s i s j ) ; 1 i < j k }. (2.6) 3

4 Figure 2: Vector v 2 makes the angle θ with s 1. In this case we compute the vector ā k+1 which is the projection of the vector v k+1 onto the subspace P k and then the unit vector s k+1 which makes the same angle θ with the previous s i s will be obtained using ā k+1. For simplicity, we project a k+1 onto the orthogonal subspace of P k, say W k = {s i s j } then using (2.3) in case of (k + 1)-dimensional, implies a k+1 = proj span a1,...,a k+1 a k+1 = proj Wk P k a k+1 = proj Wk a k+1 + proj Pk a k+1 = proj Wk a k+1 + ā k+1, (2.7) and then ā k+1 = a k+1 proj Wk a k+1. (2.8) It remains to obtain proj Wk a k+1. For this we will find an orthogonal basis B Wk = {B k,i } of W k. We claim that W k has a basis of equiangular vectors as S Wk = {S k,i } = {s 1 s k, s 2 s k,..., s s k }. To check the linearly independence take x i(s i s k ) = 0 so x is i ( x i)s k = 0. Since s 1,..., s k are linearly independent, then x 1 = = x = 0. To prove W k = span S Wk it is enough to show that W k span S Wk. Let w W k, then from the definition of W k w = k x i,j (s i s j ) = i<j k x i,j [(s i s k ) (s j s k )] = y i (s i s k ), (2.9) i<j so w span S Wk. This argument yields that dim(w k ) = k 1 so dim(p k ) = k + 1 (k 1) = 2 and P k is a plane. Now we present a method that takes a set of equiangular vectors such as S Wk and generates an orthogonal basis B Wk = {B k,1,..., B k, }. If k = 2 it is trivial: B W2 = S W2 = {s 1 s 2 }. If k = 3 then S W3 = {S 3,1, S 3,2 } = {s 1 s 3, s 2 s 3 }. Since S 3,1 and S 3,2 are equiangular with the same norm, then we choose B W3 as B 3,1 = S 3,1 S 3,2 = s 1 s 2 and B 3,2 = S 3,1 + S 3,2 = s 1 + s 2 2s 3. So B W3 = {s 1 s 2, s 1 + s 2 2s 3 }. If k = 4 then S W4 = {s 1 s 4, s 2 s 4, s 3 s 4 }. Let B 4,3 = S 4,1 + S 4,2 + S 4,3 = s 1 + s 2 + s 3 3s 4. We define C 4,1 = S 4,1 S 4,3 and C 4,2 = S 4,2 S 4,3. Both C 4,1 and C 4,2 are orthogonal to B 4,3, but C 4,1 C 4,2. Now we choose two linear combinations of them which are orthogonal, similar to the case k = 3. So B 4,1 = C 4,1 C 4,2 = S 4,1 S 4,2 = s 1 s 2 and B 4,2 = C 4,1 + C 4,2 = S 4,1 + S 4,2 2S 4,3 = s 1 + s 2 2s 3 and then B W4 = {s 1 s 2, s 1 + s 2 2s 3, s 1 + s 2 + s 3 3s 4 }. For k > 4, B Wk is obtained recurrently, e.g., if k = 5, then B 5,1 = s 1 s 2, B 5,2 = s 1 + s 2 2s 3, B 5,3 = s 1 + s 2 + s 3 3s 4 and B 5,4 = s 1 + s 2 + s 3 + s 4 4s 5. The general form of B Wk for k = 2,..., n 1 is illustrated as follows B Wk = { s 1 s 2, s 1 + s 2 2s 3,..., } s j (k 1)s k. (2.10) j=1 Since B 2,i = B 3,i =... = B k,i for i = 1,..., k 1, then for simplicity we let B i = B j,i, where B i = s s i is i+1. From (2.8) ā k+1 = a k+1 proj B i a k+1. Moreover using (1.3) we have cos ϕ = cos θ k cos θ 1 + (k 1) cos θ k cos η = cos2 θ and sin ϕ = and then 1 + (k 1)(cos θ) 1 + (k 1)(cos θ) k cos θ cot ϕ = 1 + (k 1) cos θ k cos2 θ. ( Let u k+1 = a k+1 proj k sia k+1 + ) proj B i a k+1. Note that {B 1,..., B, k s i} is a set of orthogonal vectors and span B 1,..., k s i =span s 1,..., s k. Then u k+1 is orthogonal to s 1,..., s k. Now we 4

5 substitute k s i and proj B i a k+1 instead of s 1 + s 2 and proj s1 s 2 a 3 in (2.5), respectively to get v k+1. k We have v k+1 = u k+1 + cot ϕ u k+1 s i k and without loss of generality s i k v k+1 = q k+1 + cot ϕ s i k, (2.11) where q k+1 = u k+1 1 u k+1 is orthogonal to s 1,..., s k. So the (k +1)th normalized equiangular vector spanned by {a 1,..., a k } is s k+1 = v k+1 / v k+1. We can proceed this method to obtain n equiangular vectors in R n. The formula of cot ϕ can be simplified as 1+() k cos θ = k cos θ k cos 2 θ sec 2 θ+() sec θ k = k (sec θ 1)(sec θ+k). s i Algorithm 1. (Equiangular Algorithm). This algorithm produces equiangular vectors s i R n with the angle θ via the linearly independent vectors a k, for k = 1 : n 1: s 1 = a 1 / a 1 ; s = zeros(n, 1) ; r = 0 ; 2: for k = 2 : n do 3: s = s + s ; s = s/ s ; 4: q = v k ( ss T a k + r ) 1 (a k (ss T a k + r) ) ( ; k ) 5: v k = q + 1/ ((sec θ 1)(sec θ + k 1)) s ; 6: s k = v k / v ( k ; r = 0 ; ) 7: B = j=1 s j (k 1)s k ; 8: for i = 1 : k 1 do 9: r = r + B i B T i a k / B i 2 ; 10: end for 11: end for We present an algorithm named Equiangular Algorithm (EA) for this method. We call the outcome matrix of this method S = [s 1,..., s m ] of size n m, Equiangular matrix. We denote the set of all full-rank equiangular matrices of size n m with cos θ = α by EMα n m and also the corresponding nonsingular square ones by EMα n. Orthogonal vectors. The formula (2.11) reduces to the orthogonalization process if α = 0. In this case at the step k+1, we have a k+1 = a k+1 (proj B1 a k proj B a k+1 + proj Bk a k+1 ), k = 1, 2,..., n 1, (2.12) where B k = k s i. It can be said that the normalized orthogonal vector s k+1 in case of α = 0, is constructed using the set of orthogonal vectors {B 1,, B, B k } instead of the previous set of orthonormal vectors {s 1,..., s k } which is unlike the Gram-Schmidt algorithm. Below is an example for outcome of this algorithm using MATLAB. Example 2.2. For the Vandermonde matrix A = gives S 1 = and S 2 =, applying EA with θ 1 = π/3 and θ 2 = π/ , respectively. We can check that the condition number of A is κ(a) = e + 03 but κ(s 1 ) = and κ(s 2 ) = It means that the transformation of the illconditioned matrix A into S 1 and S 2 reduces the condition number. The process of producing equiangular vectors provides a matrix factorization by generating an upper triangular matrix within EA, like QR decomposition. Theorem 2.3. Suppose A R n m has full-rank (m n). For given α (0, 1) there is an equiangular matrix S EMα n m R n m and an upper triangular R R m m so that A = SR. 5

6 Proof. In step k (k < m) of EA which is applied to the column vectors of A = [a 1,..., a m ], we derive a representation of v k followed by (2.11) as v k s k = a k + d is i. So a k = d i s i + v k s k = [s 1,..., s k ] d 1. d v k = S kr k. (2.13) By EA S k can be extended as S = [ S k, S ] m k, where Sm k is a n (m k) equiangular matrix and then a k = [ S k, S ] [ ] [ ] R k Rk m k = S. (2.14) 0 0 This is the kth column vector in both sides of A = SR Example 2.4. For the Minij matrix A = , Theorem 2.3 results A = S 1 R 1 by θ 1 = π/6 where S 1 = and R = and also results A = S 2 R 2 by θ 2 = 3π/8 where S 2 = and R 2 = Corollary 2.5. Any symmetric positive definite (spd) matrix A can be factorized as a Cholesky factorization plus a rank-one matrix. Proof. Suppose A R n n is a real spd matrix, then A can be written as B T B, where B is a nonsingular matrix. SR factorization by Theorem 2.3 implies 1 α A =B T B = R T S T SR = R T R = R T [ (1 α)i n + αee T ] R α 1 =(1 α)r T R + α(r T e)(r T e) T = Cholesky + rank-one. (2.15) It is notable that this factorization is not unique with respect to a fixed angle. Actually, the number of SR factorization of a matrix A R n m of full rank, is 2 n 1. Because by altering the sign of vector q 2 in (2.1) there are two choices for constructing s 2 in step 2. Also by altering the sign of vector q 3 in (2.5) there are two choices for constructing s 3 in step 3. Therefore we have two choices in each step until the nth vector s n is obtained. References References [1] A. Björck, Numerical Methods for Least Squares Problems, SIAM, Philadelphia, PA, [2] A. Björck, Solving linear least squares problems by Gram-Schmidt orthogonalization, BIT 7, 1-21, [3] P. Casazza and J. Kovacevic, Equal-norm tight frames with erasures, Adv. Comp. Math, 18, , [4] R. A. Horn and C. R. Johnson, Matrix Analysis, Cambridge University Press, Cambridge, [5] G. R. G. Lanckriet, N. Cristianini, P. Bartlett, L. E. Ghaoui, M. I. Jordan, Learning the kernel matrix with semidefinite programming, Journal of Machine Learning Research, 5, 2772, p. 29, [6] E. Schmidt, Über die Auflösung linearer Gleichungen mit unendlich vielen Unbekannten, Rend. Circ. Math. Palermo. Ser. 1, 25, 53-77,

7 [7] E. Solivrez, E. Gagliano, Orthonormalization on the plane: a geometric approach, Mex. J. Phys, 31, no. 4, , [8] L. N. Trefethen, and D. Bau, Numerical Linear Algebra III, Philadelphia, PA: SIAM, [9] I. Todhunter, Spherical Trigonometry for the use of college and schools, with numerous examples,, fifth edition, Macmillan and Co., London, 1886; an excellent source on spherical geometry may be found at Nov 12,