1 date: February 23, 1998 le: papar1. coecient pivoting rule. a particular form of the simplex algorithm.

Transcription

1 1 date: February 23, 1998 le: papar1 KLEE - MINTY EAMPLES FOR (LP) Abstract : The problem of determining the worst case behavior of the simplex algorithm remained an outstanding open problem for more than two decades. In the begining of the 70s, Klee and Minty [9] solved this problem by constructing linear examples on which an exponential number of iterations is required before optimality occurs. In this paper we present the Klee - Minty examples and show how they can be used to show exponential worst case behavior for some well known pivoting rules. 1. Introduction. The problem of determining the worst case behavior of the simplex algorithm remained an outstanding open problem for more than two decades. In the begining of the 70s, Klee and Minty in their cassical paper [9] showed the most commonly used pivoting rule, i.e., Dantzig's [5] largest coecient pivoting rule, performs exponentially bad on some specially constructed linear problems, known today as Klee - Minty examples. Later on, Jeroslow [8] showed similar behavior for the imum improvement pivoting rule. He showed this result by slightly modifying Klee - Minty examples. The Klee - Minty examples have been used by several researchers to show exponential worst case behavior for the great majority of the practical pivoting rules. Avis and Shvatal [1] and independenly, Murty [10],p.439, showed exponential behavior for Bland's [2] least index pivoting rule and Goldfarb and Sit [7] for the steepest edge simplex method [5]. Recently, Roos[13] established exponential behavior for Terlaky's [14] criss - cross method and Paparrizos [11] for a number of pivoting rules some of which use past history. Similar results have been derived by Paparrizos [12] for his dual exterior point algorithm and Dosios and Paparrizos [6] for a new primal dual pivoting rule [3]. In this paper we present the Klee - Minty examples and show some of their properties are used in deriving complexity results of the simplex algorithm. These properties are then used to show exponential behavior for two pivoting rules the least index and the imum coecient pivoting rule. The paper is self contained. Next section describes a particular form of the simplex algorithm. The Klee - Minty examples and their properties are presented in section 3. Section 4 is devoted to complexity results. 2.The simplex algorithm. In describing our results we nd it convenient to use the dictionary form [4] of the simplex algorithm. We will see in the next section this form exhibits some advantages in describing the properties of the Klee - Minty examples. Consider the linear problem in standard form z = c T x s:t: Ax = b (1) x 0 where c x 2 R n b 2 R m A 2 R mn and superscript T denotes transpotition. Without loss of generality wemayassume A is of full row rank, i.e., rank(a) =m (m <n). A basis for problem (1) is a set of indeces B f1 2 ::: ng containing exactly m indices. The element ofb, the components of c and x and the columns of A indexed by B are called basic while the remaining ones are called non basic. Thesetofnonbasic indices is denoted by N N = f1 2 ::: ng B. We also denote by B (N) the submatrix of A containing the columns indexed by B (N). The components of avector x indexed by B (N) are denoted by x B (x N ). With this notation at hand the equality constraints of (1) are written in the form Bx B + Nx N = b: (2) If B is a nonsingular matrix we can set x N =0 and compute x B from (2). Then, we nd x B = B ;1 b. The non singular matrix B is called basic matrix or basis. The solution x N =0and x B = B ;1 b is called basic solution. If, in addition, it is x B = B ;1 b 0, then x B x N is a basic feasible solution. Geometrically, a basic feasible solution of (1) corresponds to a vertex of the polyhedral set of the feasible region.

2 le: papar1 date: February 23, If B is non singular we can express the basic variables x B as a function of the non basic variable x N.Wehave from (2) x B = ;B ;1 Nx N + B ;1 b: (3) Using (3), the objective function of problem (1) is written in the form min hi n;m+1 ;h i N [q] :1 i m h i N [q] < 0 The basic variable x k is called leaving variable. Then, the entering variable x l takes the place of the leaving variable and vice versa, i.e., it is set B[p] N[q] and N[q] B[p]: : z = c T B x B + c T N x N = c T B(;B ;1 Nx N + B ;1 b)+c T N x N = (;c T B B;1 N + c T N)x N + c T B B;1 b: (4) At every iteration the simplex algorithm constructs the system of equations (3) and (4). Let the current feasible basis be B. The corrensponding system of equations is written in the form z = (;c T B B;1 N + c T N )x N + c T B B;1 b x B = ;B ;1 Nx N + B ;1 b (5) We denote the coecients of x N and the constant terms of (5) by H, i.e., c T N ; ct B B;1 N c T B B;1 b ;B ;1 N B ;1 b = H The top row of H, row zero, is devoted to the objective function. Some times we call it cost row. The remaining rows are numbered 1 2 ::: m. The i-th row, 1 i m, corresponds to the basic variable x B[i], where B[i] denotes the i- th element of B. Similarly, the j-th column of H 1 j n ; m, corresponds to non basic variable x N [j]. The last column of H corresponds to the constant terms. We denote the entries of H by h ij. It is well known if h 0j 0, for j = 1 2 ::: n; m, then x B x N is an optimal solution to (1). In this case the algorithm terminates. Otherwise, a nonbasic variable x N [q] = x l such h 0 N [q] > 0ischosen. Variable x l is called entering variable. If the condition h i N [q] 0 for i =1 2 ::: m holds, problem (1) is unbounded and the algorithm stops. Otherwise, the basic variable x B[p] = x k, is determined by the following minimum ratio test x B[p] ;h r N [q] = Thus, a new basis B is contructed and the procedure is repeated. Let H be the tableau corresponding to the new basis B. It is easily seen h ij = ; h pj h pq if i = p = h iq h pq if i 6= p j = q = h ij h pj h pq otherwise (6) 3. Klee - Minty examples. The Klee - Minty examples of order n are the linear problems of the form n " n;j x j s:t: x 1 1 (7) 2 " i;j x j + x i 1 i =2 3 ::: n x j 0 2 ::: n where 0 <" 1=3. In this section we will show the feasible region of (7) is a slightly perturbed cube of dimension n, see Figures 1 and 2. The optimal solution is (0 0 ::: 1) 2 R n x 2 (0 1) (1 1 ; 2") (0 0) (1 0) x 1 Figure 1. Feasible region of Klee - Minty example of order n =2

3 3 date: February 23, 1998 le: papar1 x 3 (0 0 1) (1 0 1 ; 2" ; 2" 2 ) (0 0 0) (0 1 ; 2" 1 ; 2" ; 2" 2 ) (1 1 ; 2" 1 ; 2" 2 ) (0 1 0) (1 0 0) x 2 x 1 (1 1 ; 2" 0) Figure 2. Feasible region of Klee - Minty example of order n =3 and the optimal value is 1. A cube of dimension n has 2 n vertices. In the next section we will describe pivoting rules force the simplex method to pass through all the vertices of the Klee - Minty examples. These pivoting rules require 2 n ; 1 iterations before optimallity is reached and, hence, they are exponential. The standard form of linear problem (1) is n " n;j x j s:t: x 1 + x n+1 =1 (8) 2 " i;j x j + x n+i =1 i =2 3 ::: n x j 0 2 ::: 2n where x n+i 1 i n, is the slack variable corresponding to the i-th inequality constraint of problem (7). We will be intrested in basic solutions of (8) such, for each j =1 2 ::: n either x j or x n+j is basic but not both. Such a basic solution is called distinguished. A tableau corresponding to a distinguished basis is called distinguished tableau. In order to facilitate the presentation we nd it convenient tointroduce the set Q of all zero - one n-sequences (a 1 a 2 ::: a n )such a j = 0 if x j is nonbasic and x n+j is basic = 1 if x n+j is nonbasic and x j is basic: We denote the distinguished basis corresponding to the sequence (0 0 ::: 0) by ^B and the tableau corresponding to ^B by ^H. Itis ^B = fn +1 n+2 ::: 2ng. It is easily veried ^h ij = " n;j if i =0 j n = ;1 if 1 i = j n = 0 if 1 i < j j 6= n +1 = ;2" i;j if i>j (9) = 1 if i n and j = n +1 Tableau ^H is sometimes called initial. A distinguished tableau H corresponding to (a 1 a 2 ::: a n ) 2 Q is constructed by starting from ^H and pivoting only on elements h pp such a p = 1. Using this procedure and relations (6) and (9) we easily conclude h pp = ;1 for p =1 2 ::: n h ij = 0 for 1 i n ; 1 i j n (10) for each distinguished tableau H. Lemma 1. Let B be an arbitrary distinguished basis and H the corresponding tableau. Then h ij + h pj h ip = ;h ij j < p < n i 2 (11) h 0j + h pj h 0j = 0 j <p n i =0: (12) Proof. It suces to show the following induction hypothesis. If the distinguished tableau H satises (11) and (12) and a pivot operation is performed on h rr 1 r n, resulting in tableau H, then H satises (11) and (12) as well. Observe relations (11) and (12) are satised by the initial tableau ^H. So, assume H satises (11) and (12) and a pivot operation is performed on element h rr = ;1. Then, we have from (6) and (10) h ij = h ij + h rj hir i6= r j6= r = ; h ij i6= r j= r (13) = h ij Otherwise:

4 le: papar1 date: February 23, Combining (13) and the induction hypothesis we have h ij = ; h ij if i =0 j r = ; h ij if i>r j r = ; h ij + h rj hir = ; h ii + h rj hir if i>r j = n +1 (14) if i =0 j = n +1 = ; h ij otherwise: There are two cases to be considered, p r and p>r. From relations (14) we have h ij = ; h ij h ip = ; h ip and h pj = h pj for p r and h ij = ; h ij h ip = h ij and h pj = ; h pj for p>r In both cases, h ij + h pj h ip = ; h ij ; h pj h ip = h ij = ;h ij This proves (11). The proof of (12) is similar. Lemma 1 shows pivoting on element h pp of a distinguished tableau H is very easily performed. Just change the signs of the entries h ij such i = p and j p or i>pand j p and set h i n+1 h i n+1 + h ip h p n+1 for i =0ori>p Figure 3 illustrates the entries of H change value when pivoting on h pp. In particular, the entries in areas A and B just change sign. P A B P 0 0 Figure 3. Entries of a distinguished tableau H change value after pivoting on element h pp. Theorem 2. Let H be a distinguished tableau of problem (8) and a =(a 1 a 2 ::: a n ) 0 be the corresponding n-sequence. Then, the following relations hold. For i =1 2 ::: nand j =1 2 ::: nwehave h ij = ;1 i= j (15) = 0 i< j (16) = ;2" i;j i>j and = 2" i;j i>j and For i =0andj =1 2 ::: n we have h 0j = " n;j = ;" n;j n n a k even (17) a k odd (18) a k even (19) a k odd (20) For i =1 2 ::: n and j = n +1wehave h i n+1 = 1 i=1 (21) 2 i n (22) Proof. The proof is by induction. We assume distinguished tableau H satises (15)-(22), and show tableau H computed by pivoting on h pp satises (15)-(22) as well. Observe initial tableau ^H satises (15)-(22). Let a =(a 1 a 2 ::: a n ) be the sequence corresponding to tableau H. Then a j = a j j 6= p a j j= p: Proof of (15)-(18). Relation (15) and (16) have allready been shown. From Lemma 1 we have h ij = h ij and a k = for i p or i>pand j>p.for i>pand j p we have a k = a k ; 2a p Hence, if P a is odd (even), P a k is even (odd). Also, from Lemma 1 we have h ij = ;h ij. Hence, (17) and (18) hold in all cases. a k

5 5 date: February 23, 1998 le: papar1 Proof of (19) and (20). It is easily seen sign(h 0j ) = sign(h mj ) for i n: Now the proof comes from the proof of (17) and (18). Proof of (21) and (22). If i p we have h i n+1 = h i n+1. Hence, (21) and (22) hold trivially from the induction hypothesis. If i > p, then h i n+1 = h i n+1 + h p n+1 h ip +(1; a k (h ik + h pk h ip ) +h ip ; a k h pp h ip ; ; k=p+1 a k h pk )h ip a h ik ; (1 ; a p ) h ip a k hik a k hik : Theorem 3. The feasible region of problem (8) is a slightly perturbed cube. Proof. It suces to show the feasible region has presicely 2 n vertices. Weshow each distinguished tableau, H, is feasible and all the adjacent tableaux are distinguished. From Theorem 2 we have h i n+1 > 1 ; 2"(1 + " + " 2 + ) 2" 1 ; " 0: We show if x N [p] is entering, then x B[p] is leaving variable. Let h ip < 0 (i <p). We show h p n+1 h pp h i n+1 ;h ip or The last relation is equivalently written ;h ip (1 ; a k h pk ) < 1 ; Using Lemma 1 we get ;h ip ; 2 0 < 1+h ip + = 1+ + k=p+1 < 1 ; ; k=p : +(1; a k )h ip We have already shown the last relation holds. 4. Applications Now, we are ready to show exponential behavior for some pivoting rules. Let a 6= b be sequences of Q. Wewritea<bif P for the largest n index j such a j 6= b j it is a k even and P n b k odd. Let now f(a) be the objective value at the vertex corresponding to a 2 Q. It is easily seen f(a) < f(b), if a < b. The immediate succesor of a sequence a 2 Q is the sequence (a 1 a 2 ::: a r 1 ; a p a p+1 ::: a n ) where p is the smallest index such P n j=p a j is even. Given a distinguished tableau H, a nonbasic variable x N [q] is called elligible if h 0 N [p] > 0. A pivoting rule forces the simplex algorithm to pass through all vertices of Klee - Minty examples is the following. For the ease of reference we call it generic pivoting rule. Let a 2 Q be the sequence corresponding to H. The entering variable is x N [p], where p is the smallest index such P n k=p a k is even. From Theorem 2we see h 0 N [p] = " n;p > 0. Hence, x N [p] is elligible, and the generic pivoting rule requires 2 n ; 1 iterations on Klee - Minty examples of order n. Smallest index rule.

6 le: papar1 date: February 23, In the smallest index rule, the entering variable is the eligible variable with the smallest index. We show the smallest index rule, called also Bland's rule, performs exponentially on the slightly modied Klee - Minty examples n " n;j x 2j;1 s:t: x 1 1 (23) 2 " i;j x 2j;1 + x 2 1 i =2 3 ::: n x j 0 2 ::: n We introduce the slack variable x 2i to the i-th constraint of problem (23). Theorem 4. The least index pivoting rule performs exponentially on example (23). Proof. We show the simplex algorithm employing the least index pivoting rule requires 2 n ; 1 iterations when applied to problem (23) and initialized with the basis corresponding to the sequence (0 0 ::: 0) 2 Q. Clearly, all the bases generated by the algorithm are distinguished i.e. for each i either x 2 or x 2i is basic but not both. Let H be the current distinguished tableau corresponding to the sequence a 2 P Q. Let also p be the smallest index such n k=p a k is even. Then, h 0 N [p] > 0 and, hence, x N [p] is eligible. Because of the indexing of the variable in problem (23), N[p] =2p or 2p ; P 1. If q is another n index such h 0 N [q] > 0 ( k=q a k is even), then q > p and, hence, N[q] > N[p]. Hence, the next basis corresponds to the immediate succesor of a 2 Q. This completes the proof. The largest coecient rule. In the largest coecient rule the entering variable x N [p] is chosen so h 0 N [p] = fh 0 N [j] : h 0 N [j] > 0g: This rule solves problem (7) in one iteration when the initial basis is (0 0 ::: 0) 2 Q. We modify problem (7) as follows. We set " =1= x j = y j e 2(j;1) (24) and divide the i-th constraint by " 2() and the objective function by " 2(n;1). Then, problem (7) is written in the equivalent form s:t: 2 n n;j y j (25) i;j y j + y i 2() i =1 2 ::: n y j 0 2 ::: n where = 1 3. Theorem 5. The largest coecient rule performs exponenially on problem (25). Proof. Problem (25) is a scaled version of problem (7). Letx n+i be the slack of constraint i. Then, all the results of the previous section, except those involving the RHS, hold true for problem (25). Because of relation (24), c j 0if and only if y j 0. Hence, every distinguished basis of (25) is feasible. Now, it suces to show the generic and the largest coecient rule coincide when applied to problem (25).However, this statement holds because >1: References [1] Avis, D., and Chvatal, V.: `Notes on Bland's pivoting rule', Mathematical Programming Study 8 (1978), 24{34. [2] Bland, R.G.: `New nite pivoting rules for the simplex method', Math. Oper. Res. 2 (1977), 103{107. [3] Chen, H., Pardalos, P.M., and Saunders, M.A.: `The simplex algorithm with a new primal and dual pivot rule', Oper. Res. Lett. 16 (1994), 121{ 127. [4] Chvatal, V.: Linear Programming, W.H. Freeman and Company, [5] Dantzig, G.B.: Linear Programming and Extentions, Princeton University Press, [6] Dosios, K., and Paparrizos, K.: `Resolution of the problem of degeneracy in a primal and dual simplex algorithm', Operations Research Letters 20 (1996), 45{50. [7] Goldfarb, D., and Sit, W.: `Worst case behavior of the steepest edge simplex method', Discrete Applied Mathematics 1 (1979), 277{285. [8] Jeroslow, R.G.: `The simplex algorithm with the pivot rule of imizing improvement criterion', Discrete Mathematics 4 (1973), 367{377. [9] Klee, V., and Minty, G.J.: How good is the simplex algorithm?, in Inequalities : III ed. O. Shisha, Academic Press, 1972.

7 7 date: February 23, 1998 le: papar1 [10] Murty, K.G.: Linear Programming, John Wiley, [11] Paparrizos, K.: `Pivoting rules directing the simplex method through all feasible vertices of Klee - Minty examples', OPSEARCH 26,No 2 (1989), 77{ 95. [12] Paparrizos, K.: `An exterior point simplex algorithm for (general) linear programming problems', Annals of Operations Research 47 (1993), 497{508. [13] Roos, C.: `An exponential example for Terlaky's pivoting rule for the criss - cross simplex method', Mathematical Programming 46 (1990), 78{94. [14] Terlaky, T.: `A convergent criss - cross method', Math. Oper. Stat. Ser. Optim. 16 (1985), 683{690. Konstantinos Paparrizos University of Macedonia Department of Applied Informatics 156 Egnatia Str Thessaloniki GREECE address: paparriz@macedonia.uom.gr