Int J Appl Math Comput Sci, 00, Vol 1, No 1, 5 1 A LINEAR PROGRAMMING BASED ANALYSIS OF HE CP-RANK OF COMPLEELY POSIIVE MARICES YINGBO LI, ANON KUMMER ANDREAS FROMMER Department of Electrical and Information Engineering University of Wuppertal, Rainer-Gruenter-Street 1 119 Wuppertal, Germany e-mail: {yingbo, kummert}@uni-wuppertalde Department of Mathematics University of Wuppertal, Gauß Street 0 119 Wuppertal, Germany e-mail: frommer@mathuni-wuppertalde A real matrix A is said to be completely positive (CP) if it can be decomposed as A BB, where the real matrix B has exclusively non-negative entries Let k be the rank of A and Φ k the least possible number of columns of the matrix B, the so-called completely positive rank (cp-rank) of A he present work is devoted to a study of a general upper bound for the cp-rank of an arbitrary completely positive matrix A and its dependence on the ordinary rank k his general upper bound of the cp-rank has been proved to be at most / In a recent pioneering work of Barioli and Berman it was slightly reduced by one, which means that Φ k / 1 holds for k An alternative constructive proof of the same result is given in the present paper based on the properties of the simplex algorithm known from linear programming Our proof illuminates complete positivity from a different point of view Discussions concerning dual cones are not needed here In addition to that, the proof is of constructive nature, ie starting from an arbitrary decomposition A B 1B 1 (B 1 0) a new decomposition A B B (B 0) can be generated in a constructive manner, where the number of column vectors of B does not exceed / 1 his algorithm is based mainly on the well-known techniques stemming from linear programming, where the pivot step of the simplex algorithm plays a key role Keywords: completely positive matrices, cp-rank, linear programming, simplex algorithm, basic feasible solution, pivot process 1 Introduction An n n real symmetric matrix A is called completely positive (belonging to the set CP of completely positive matrices) if an entry-wise non-negative n m matrix B exists with A BB he product A BB can alternatively be written in its rank 1 representation as A m i1 b ib i, in which each b i R n + denotes the i-th non-negative column of the matrix B he minimum number of columns m, for which such a factorization A BB exists, is called the cp-rank of A, and denoted by Φ k in relation to the ordinary rank k of A It is to be pointed out that all vectors throughout this paper are column vectors, unless explicitly defined to be row vectors he non-negative decomposition of completely positive matrices is of interest in various applications including, eg, the study of block designs arising in combinatorial analysis (Hall, 196), problems in statistics, the theory of inequalities, energy conservation (Gray and Wilson, 1980) and models of DNA evolution (Kelly, 199) No definitive test is known yet to determine whether a given real symmetric matrix A is completely positive Many studies on this problem have been performed (Berman and Plemmons, 199; Berman and Shaked- Monderer, 00) One sufficient condition for a nonnegative symmetric matrix to be completely positive given in (Drew et al, 199) says that the comparison matrix of A is an M-matrix A special case of this sufficient condition is that the matrix A is diagonally dominant (Kaykobad, 1988) A qualitative characterization of completely positive matrices using graphs is that the graph of a cp matrix contains no odd cycle of length 5 or more (Berman, 199; Kogan and Berman, 199)
6 he question of a general upper bound for the cprank which works for any completely positive matrix of dimension n n was asked by Hall and Newman (196), with the conclusion that Φ k < n he definition of the cp-rank as above implies that Φ k is obviously greater than or equal to the ordinary rank k of A Hannah and Laffey (198) derived the inequality Φ k n(n + 1)/ (which is independent of k, with k n) Recently, the sharpened upper bound / for Φ k was reduced by Barioli and Berman (00) to / 1 if k In (Drew et al, 199) it was conjectured that Φ k < n / However, this conjecture was proved only for matrices with special graphs, which are bipartite with the two parts as balanced as possible, or for graphs which contain no odd cycle of length 5 or more (Drew and Johnson, 1996), or for all graphs on 5 vertices which are not a complete graph (Berman, 199) In the present paper we give a constructive proof alternative to Barioli and Berman s result that Φ k 1, k he technique of the proof presented here involves the properties of the simplex algorithm known from linear programming and illuminates the problem from a different point of view Above all, discussions concerning dual cones are not required Linear programming is the process of minimizing (or maximizing) a linear real-valued objective function subject to a finite number of linear equality and inequality constraints Linear programming problems are encountered in many branches of technology, science, and economics In this work, the problem of determining an upper bound for the cp-rank of completely positive matrices is reduced to the problem of determining the maximal number of non-zero elements in the basic feasible solutions of a linear programming problem By means of appropriate matrix operations and based on the properties of the simplex algorithm, the desired result will be achieved Furthermore, according to the described constructive procedure, such a decomposition which satisfies the claimed bound constraint can be generated step by step from a known decomposition of an arbitrary size First Reduction Step We start with a known rank 1 representation A m i1 b ib i of a given n n completely positive matrix, where m is an arbitrary large, however finite, integer From now on, we assume that no column b i of B is zero, since otherwise such a column could have been removed from B After the application of the vec-operator (stacking the columns of a matrix to form a column vector) to Y Li et al both the sides, the following expression can be obtained: ( m ) m vec(a) vec b i b i vec(b i b i ) i1 i1 m b i b i, (1) i1 where denotes the Kronecker product Consequently, we may define an n m matrix C b 1 b 1, b b,, b m b m () with c i b i b i, i 1,, m, so that (1) can be represented as vec(a) C 1, 1,, 1 () It is evident that the non-negative linear system of equations vec(a) Cλ with λ i 0 has at least one valid solution, namely the column vector λ 0 1, 1,, 1 In fact, from now on we may look upon Cλ vec(a) with λ i 0 as the linear constraints for an arbitrary linear objective function, which is a linear programming problem in a standard form with all variables of λ non-negatively constrained First of all, it is important to indicate that the feasible polyhedron of this linear programming problem is on the one hand non-empty due to the existence of the feasible solution λ 0 and, on the other hand, bounded due to the non-negativity of C, vec(a) and λ, and the fact that no column in B, and thus no column in C, is zero hus, the feasible region represents a polytope Consequently, the available feasible solutions which satisfy these linear constraints can be expressed as convex combinations of the so-called basic feasible solutions, ie the extreme points (or vertices) of the polytope According to the properties of linear programming problems, in the basic feasible solutions of Cλ vec(a) with λ i 0, only the elements λ i which correspond to a maximal set of linear independent columns of C may be non-zero herefore, all of the basic feasible solutions possess at most rank(c) strictly positive elements Because of the existence of the solution λ 0 1, 1,, 1, it can be concluded that at least one basic feasible solution with at most rank(c) strictly positive elements for Cλ vec(a) with λ i 0 exists Writing rank(c) r, and, if necessary, permuting the columns of B, we may assume that there is a basic feasible solution of the form λ B λ 1, λ,, λ r, 0,, 0 with λ i 0 for 1 i r It is obvious that r r ( vec(a) Cλ B c i λ i λi b i ) ( ) λ i b i, i1 i1 ()
A linear programming based analysis of the CP-rank of completely positive matrices which is equivalent to A r ( )( ) λi b i λi b i (5) i1 Defining B λ 1 b 1, λ b,, λr b r, we have established an alternative non-negative factorization of A, namely A B B λ1 b 1,, λr b r λ1 b 1,, λr b r (6) It should be noted that a basic feasible solution can always be generated constructively by using linear programming techniques (Glashoff and Gustafson, 198; Karloff, 1991) Upper Bounds on the CP-Rank In this section we first clarify that the cp-rank of A, Φ k, is bounded by the rank of the matrix C, which is in turn limited by / again, where k rank(a) he proof of the sharpened upper bound of Φ k, namely / 1, will be provided afterwards Lemma 1 he cp-rank Φ k of A CP is bounded by the rank of the matrix C, ie we have Φ k r rank(c) () Proof Based on the non-negative factorization of A achieved in (6), which can always be realized constructively from an arbitrary non-negative decomposition of A, and the definition of Φ k as the least number of nonzero columns in any decomposition, it is obvious that Φ k r rank(c) Our final goal is to prove that Φ k is bounded by / 1 In order to keep the proof clear and progressive, we will first prove that rank(c) / heorem 1 he ordinary rank of matrix C as defined in () is bounded by rank(c) (8) Proof Since A BB with rank(a) k, it follows immediately that rank(b) k With respect to a geometric interpretation, this means that the dimension of the vector space generated by the b i s is k, ie dim(span(b i 1 i m)) k As a consequence, there must be an isometry available which maps b i R n + onto R k, i 1,, m In matrix terms, this means that we may find an orthonormal matrix V (V V I, where I is the identity matrix), so that V B ( F 0 ), the dimension of F being k m Computationally, V and F can be obtained from the standard QR-decomposition of B, see (Golub and Van Loan, 1989) he following equality results from this isometry: (V V )C (V V ) b 1 b 1,, b m b m (V b 1 ) (V b 1 ),,(V b m ) (V b m ) ( ) ( ) ( ) ( ) f 1 f 1 f,, m f m, (9) 0 0 0 0 where f i V b i is the i-th column vector of matrix F Since there are at most / distinct non-zero row vectors in the matrix (V V )C and V V is orthonormal and thus non-singular, we have r rank(c) rank((v V )C) Corollary 1 In view of () and heorem 1, we can conclude that Φ k is at least bounded by Φ k (10) Let us note that the above proofs are constructive: Given A BB, we have shown how to obtain B in A B B by computing a basic feasible solution of a certain polytope he next step we want to take is proving the sharpened bound Φ k / 1 for k he corresponding proof can be divided into distinct steps If for a given matrix A the associated matrix C has a rank less than /, we can stop, otherwise we have rank(c) / and the reduction procedure must be continued he outline of this algorithm is as follows: After extension of C by a column vector, which is the Kronecker product of an adequately selected column of A with itself, we will show that the rank of the extended matrix C is identical to its original rank hen a pivot step known from simplex techniques will be used, so that a new non-negative decomposition A GG could be created, whereby the column number of G does not exceed that of B Relying on some requirements regarding G, the column number of G can be further reduced by 1 after some straightforward computations By this last step the proof will be completed heorem For k the cp-rank of any completely positive matrix A with its ordinary rank equal to k is
8 bounded by cp-rank (A) Φ k 1 (11) Proof From heorem 1 and Corollary 1 we know that Φ k rank(c) / If rank(c) < /, nothing has to be proven, so that from now on we assume that r rank(c) / Before the actual proof, we would like to point out a useful property of the matrix B (cf (6)) at first From the decomposition A B B in (6) with B λ 1 b 1, λ b,, λr b r, the fact that rank( B) k n and the assumption r /, it can be concluded that at least one of the rows of the non-negative matrix B has more than one non-zero entry Otherwise, if each row of the non-negative matrix B had at most one non-zero element, all columns of B (which does not have zero columns) would be linearly independent, so k r and thus Φ k r k / 1 for k and we have a contradiction to the assumption r / So, for the rest of the proof we may therefore assume that at least one row of B contains at least two non-zero entries Now we extend vec(a) Cλ B in (), which is associated with A a 1, a,, a n and rank(c) r /, to vec(a) C, a j a j λ E, (1) with λ E λ B0, where, in view of the above discussion, the integer j, 1 j n, can be chosen in such a way that the j-th row of non-negative B has at least two non-zero entries he rank of the extended matrix C, a j a j in (1) is unmodified compared with that of the matrix C, ie ) rank (C, a j a j rank(c) r, (1) as we show now he orthonormal matrix V from the proof of heorem 1 will be reused From A BB and V B ( F 0 ) it follows that V A V BB ( F 0 )B ( E 0 ) and, accordingly, V a j ( e j 0 ) with e j as the j-th column of E, whereby the dimension of E is k n, so that (V V ) C, a j a j (V V ) b 1 b 1,, b m b m, a j a j Y Li et al (V b 1 ) (V b 1 ),, (V b m ) (V b m ), (V a j ) (V a j ) ( ) ( ) ( f 1 f 1 f,, m 0 0 0 ) ( f m 0 ( ) ( ) e j e j (1) 0 0 As in the proof of heorem 1, we have rank(c, a j a j ) / Since rank(c) was assumed to be equal to / and for the reason that rank(c) rank(c, a j a j ), we get rank(c, a j a j ) rank(c) / Because all entries of the extended linear constraints system (1) are also non-negative, and there exists a feasible solution λ E λ B0, the feasible region is bounded and non-empty, ie it is again a polytope Consequently, basic feasible solutions of the linear programming problem (1) exist and have at most r / nonnegative variables Furthermore, λ E is a basic solution where the basis is represented exactly by the variables λ 1,, λ r o be more specific, since rank(c, a j a j ) rank(c), any basis of the old linear system is also a basis of the extended system (his would not be true if the rank of the extended system were increased by one when compared with that of the old system) Suppose that the simplex tableau associated with the extended linear system is given for the basis λ 1,, λ r, ie λ m+1, the variable associated with column a j a j of the extended system is a non-basic variable yet Next, the simplex algorithm will be considered, which has been the method of choice used to solve linear programming problems for decades Due to the fact that the basic feasible solutions are nothing else than extreme points of the polytope, and the optimal point of the objective function subject to the polytope is always among these vertices, the simplex algorithm can be regarded as an ordered way of scanning through such vertices he simplex algorithm starts from an arbitrary vertex of the feasible polytope and tries to find a cheaper adjacent vertex, till ideally no neighbouring vertex with a cheaper cost is found he process of changing the basis to move to an adjacent vertex is called the pivot step Basically, a pivot step takes one basic variable out of the basis and, at the same time, it shifts another, originally non-basic, variable into the basis Hence, during a pivot step carried out on the simplex tableau defined above, the last variable of λ E, ),
A linear programming based analysis of the CP-rank of completely positive matrices 9 namely λ m+1, can be deliberately forced to enter the basis, for which one of the formerly basic variables, say λ s, leaves the basis It is worth noticing that through the exchange of the basic variable and the corresponding modification of the basis, the maximal number of non-zero elements in the new basic feasible solution, r /, is unchanged, and all of the elements are still non-negative Denoting by d 1,, d r the new non-negative basic variable values, we have vec(a) b 1 b 1,, b s 1 b s 1, a j a j, b s+1 b s+1,, b r b r d 1,, d s,, d r (15) Following the same procedure as above, a new nonnegative factorization of A of the form A d1 b 1,, d s 1 b s 1, d s a j, ds+1 b s+1,, d r b r d1 b 1,, d s 1 b s 1, d s a j, ds+1 b s+1,, d r b r (16) with G g 1,, g r 1 + αaj g j1,, g j,r 1, ( α (1 d s a jj ) r 1 gjl l1 ) + d s r 1 gjl l1 (1 d sa jj ) r 1 gjl l1 (19) Since d s > 0, the scalar α is also strictly positive, ie α > 0, which shows that G is non-negative Furthermore, α always exists since r 1 l1 g jl 0, which is true since r 1 l1 g jl r l1,l s d lb jl, and we have chosen j in such a manner that at least two values from among b jl, l 1,, r are non-zero he validity of the decomposition in (18) can be simply checked by inserting (19) into (18) and using g1,, g r 1 gj1,, g j,r 1 (1 d s a jj )a j, (0) which follows from a j g 1,, g r 1, ds a j g j1,, g j,r 1, ds a jj, (1) see (1) In view of (18) and (19), the number of columns in G equals / 1 Hence we have proved that cp-rank (A) Φ k r 1 1 () is obtained If at least one of the non-negative numbers d l, l 1,, r, is zero, then cp-rank (A) r 1 / 1, so that the desired result is obtained Otherwise we proceed as follows: After a reordering of the columns of the matrix d 1 b 1,, ds 1 b s 1, ds a j, ds+1 b s+1,, dr b r in such a way that the column d s a j enters the last position, we have the decomposition A GG g 1,, g r 1, g 1,, g r 1, ds a j ds a j (1) with g l g 1l,, g nl 0, l 1,, r 1, where the g l s denote the newly arranged non-negative columns Based on these results, we finally can construct the non-negative decomposition A G G, (18) In order to have all reduction steps in a condensed form without being interrupted by proofs, we summarize them in the form of the sequential algorithm, shown in Fig 1 Illustrative Example In this section the constructive algorithm is demonstrated by applying it to an illustrative example Consider a completely positive matrix A 1, whose rank also equals three here exists a non-negative decomposition A BB with 1 0 0 1 0 1 1 B 0 1 0 1 1 0 0 0 1 0 1 1
0 start, given a cp-matrix A(n n) with rank(a) k, let B(n m) be any known solution to A BB, B 0 if column number of B, m, is less than / we have done, exit otherwise generate the linear system vec(a) Cλ, λ 0 with C defined by C b 1 b 1, b b,, b m b m compute a basic feasible solution λ B programming method by a known linear generate B by means of the non-zero elements of λ B if column number of B, r, is less than / A B B is already the desired factorisation, done and exit otherwise λ E λ B 0 extend C by a j a j, with 1 j n generate the new linear system vec(a) C, a j a j λ E apply one pivot step to bring non-basic variable λ m+1 into the basis, which leads to a new basic feasible solution d generate G by means of the non-zero elements of d if column number of G is less than / A GG and exit otherwise end is already the desired factorisation, done compute G according to (19), herewith we have A G G and the column number of G is at most / 1 Fig 1 Summary of all algorithmic steps he linear system vec(a) Cλ is given by 1 0 0 1 0 1 1 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0 1 1 0 0 0 0 0 1 0 6 0 0 0 0 1 0 6 1 0 0 1 0 1 1 9 λ 1 λ λ λ λ 5 λ 6 λ By means of standard techniques, the basic solution λ B 5/ 5/ 1 / 0 1/ /6 Y Li et al can be generated From (6) we have a new non-negative factorization, namely A B B, where B has only six columns, 5/ 0 0 / 1/ /6 B 0 5/ 0 / 0 1/ 0 0 1 0 1/ 1/ Our next step is to construct a new linear system by extending the matrix C by a a, which leads to 1 0 0 1 0 1 1 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0 1 1 0 0 0 0 0 1 0 6 0 0 0 0 1 0 6 1 0 0 1 0 1 1 9 16 8 8 8 9 8 8 8 1 λ 1 λ λ λ λ 5 λ 6 λ λ 8 he Kronecker product a a is used here, which fulfils the requirement that the third row of B has at least two strictly positive entries It can be verified that the rank of the new extended system matrix is the same as that of C, namely r 6 Starting from the already available basic solution λ E 5/ 5/ 1 / 0 1/ /6 0, we can include λ 8 into the basis by applying only one pivot step, which leads to the new basic solution λ E 0 015 0 16 8 16 0 005 From this a newly arranged non-negative factorization A GG results in 0 0 8/5 0 8/5 /5 G /0 0 8/5 1/5 8/5 9/0 0 /5 0 1/5 0 6/5 Since the matrix G has still six columns, we have to apply (19) in order to reduce the number of columns In our example the scalar α has a positive value of 5/88 1/1, which leads to the final decomposition A G G with 0 015 169 01 15099 G 08 01 169 06 088 0 1 0 658
A linear programming based analysis of the CP-rank of completely positive matrices As has been expected, the non-negative factorisation A eg e has at most / 1 5 columns By G eg e to A the correctness of this factorisation equating G can be easily verified It has thus been demonstrated that given any known decomposition of a cp-matrix A a decomposition with at most / 1 columns can be generated in a constructive manner Obviously, 1