ECE 307 Techniques for Engineering Decisions

ECE 7 Techniques for Engineering Decisions Introduction to the Simple Algorithm George Gross Department of Electrical and Computer Engineering University of Illinois at Urbana-Champaign ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.

SOLUTION OF SYSTEMS OF LINEAR EQUATIONS We eamine the solution of A = b using Gauss Jordan elimination We first use a simple eample and then generalize to cases of general interest Consider the system of two equations in five unknowns: ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.

SOLUTION OF SYSTEMS OF LINEAR EQUATIONS S + 4 + = ( i) 4 5 = 4 ( ii) 4 5 For this simple eample, the number of unknowns eceeds the number of equations and so the system has multiple solutions; this is the principal reason that the LP solution is nontrivial ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.

SOLUTION OF SYSTEMS OF LINEAR EQUATIONS The Gauss Jordan elimination uses elementary row operations: multiplication of any equation by a nonzero constant addition to any equation of a constant multiple of any other equation We transform S into the set S by multiplying equation (i) by and adding it to equation (ii) S + 4 + = 4 5 + = 4 ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 4 5

DEFINITIONS A basic variable is a variable i that appears with the coefficient in an equation and with the coefficient in all the other equations The variables j that are not basic are called nonbasic variables In the system, appears as a basic variable; S,, 4 and 5 are nonbasic variables Basic variables may be generated through the use of elementary row operations ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 5

DEFINITIONS A pivot operation is the sequence of elementary row operations that reduces a system of linear equations into the form in which a specified variable becomes a basic variable A canonical system is a set of linear equations obtained through pivot operations with the property that the system has the same number of basic variables as the number of equations in the set ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 6

CANONICAL SYSTEM FORM We transform the system into the canonical form of system : S 4 45 = 6 + 4 5 = The basic solution is obtained from a canonical system with all the nonbasic variables set to For the eample, we set = = = and so S S = 6 and = 4 5 ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 7

BASIC FEASIBLE SOLUTION A basic feasible solution is a basic solution in which the values of all the basic variables are nonnegative In the eample of system two variables to be basic S, we may choose any In general for a system of m equations in n n unknowns there are possible combinations m of basic variables ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 8

BASIC FEASIBLE SOLUTION As n increases the number of combinations becomes large even though it is finite For the eample, we have 5 5! = =!! combinations of possible choices ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 9

THE SIMPLEX SOLUTION METHOD We net use a simple eample to construct the simple solution method The simple method is a systematic and efficient way of eamining a subset of the basic feasible solutions of the LP to hone in on an optimal solution We apply the notions introduced in the definitions we introduced above ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.

THE SIMPLEX SOLUTION METHOD The canonical form of the eample allows the determination of a basic feasible solution The corresponding value of the objective is The net step is to improve the basic feasible solution by finding an adjacent basic feasible solution = = = = 8, = 7 4 5 Z = 8 + 7 = ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.

ADJACENT FEASIBLE SOLUTION An adjacent feasible solution is one which differs from the current basic feasible solution in eactly one basic variable Note, we characterize a basic feasible solution by the following traits basic variable nonbasic variable ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. =

ADJACENT FEASIBLE SOLUTION The search for an adjacent basic feasible solution uses the idea of making a nonbasic variable into a basic variable by increasing its value from to the largest positive value without violating any constraints To make the search efficient, we choose the nonbasic variable that can improve the value of Z by the largest amount ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 4

ADJACENT FEASIBLE SOLUTION In the eample, consider the nonbasic variable, we leave = = and eamine the possibility of making into a basic variable The variable enters in both constraints + = 4 8 + = 7 5 ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 5

ADJACENT FEASIBLE SOLUTION The largest value can assume without making either or negative is 4 5 min 7 7 8, = We have the new basic variable with the value =, and the other basic variable is 4 = 7 7 ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 6

ADJACENT FEASIBLE SOLUTION and the three nonbasic variables are set to : = = and = 5 Note that we have an improvement in Z since its value becomes 7 7 8 Z = 5 i = = 6 > We net need to put the system of equations into canonical form: ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 7

SIMPLEX METHODOLOGY EXAMPLE ma Z = 5 + + + 4 5 st.. non - canonical form for + + + = 4 8 (*) + 4 + + 5 = 7 (**) i i =,, 5 ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 8

ADJACENT FEASIBLE SOLUTION multiply equation (**) by and add to equation (*) + 5 + = 7 5 4 multiply equation (**) by + 4 + + = 7 5 ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 9

THE SIMPLEX SOLUTION METHOD We continue this process until the condition of optimality is satisfied: in a maimization problem, a basic feasible solution is optimal if and only if the relative profits of each nonbasic variable is in a minimization problem, a basic feasible solution is optimal if and only if the relative costs of each nonbasic variable is ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.

THE SIMPLEX SOLUTION METHOD The relative profits (costs) are given by the change in Z corresponding to a unit change in a nonbasic variable We use this fact to select the net nonbasic variable to enter the basis ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.

SIMPLEX ALGORITHM FOR MAXIMIZATION Step : start with an initial basic feasible solution with all constraint equations in canonical form Step : check for optimality condition: if the relative profits are < for each nonbasic variable, then the basic feasible solution is optimal and stop; else, go to Step ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.

SIMPLEX ALGORITHM FOR MAXIMIZATION Step : select a nonbasic variable to become the new basic variable; check the limits on the nonbasic variable the limiting constraint determines the basic variable that is being replaced by the selected nonbasic variable Step 4: determine the canonical form for the new set of basic variables through elementary row operations; compute the basic feasible solution, Z and return to Step ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.

THE SIMPLEX TABLEAU We use an efficient way to represent visually the steps in the simple method through a sequence of so-called tableaus We illustrate the tableau for the simple eample for the initial basic feasible solution ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 4

THE SIMPLEX TABLEAU coefficients of the basic variables in Z coefficient of j in Z c B c j basic variables 5 4 5 constraint constants 4 8 5 4 7 ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 5

THE SIMPLEX TABLEAU The optimality check requires the evaluation of For each nonbasic variable j, for our eample, we have T column corresponding = j j B i to in canonical form c c c c c c j 5, i = = =, i = 4, i 4 = = ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 6

THE SIMPLEX TABLEAU We interpret as the change in Z corresponding to a unit increase in j c B c j basic variables 5-4 5 constraint constants - 4 8 5 4 7 c T 4 Z = ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 7

SIMPLEX TABLEAU Note that the optimality test indicates that c = > and = 4> and so the initial basic feasible solution is not optimal c c Since >, we pick as the nonbasic variable to become a basic variable c We eamine the limiting solution for in the two constraint equations: ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 8

THE SIMPLEX TABLEAU equation limiting basic variable 4 5 upper limit on (8/) = 4 (7/) = 7 and so the limiting value is min {4, 7}=4 We replace the basic variable 4 by ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 9

SIMPLEX METHODOLOGY EXAMPLE ma Z = 5 + + + 4 5 st.. canonical form for and 4 5 + + + 4 = 8 (*) + 4 + + 5 = 7 (**) i i =,,5 ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.

THE SIMPLEX TABLEAU For the new basic feasible solution, we put the equations into canonical form multiplying (*) by to produce (* ) subtract (* ) from (**) to produce (** ) The adjacent basic feasible solution is = = 4 = = 4, 5 = + + + 4 = 4 5 + 4 + 5 = (* ) (** ) ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.

THE SIMPLEX TABLEAU c Since >, the basic feasible solution is nonoptimal We eamine how to bring into the basis equation (* ) (** ) limiting basic variable 5 upper limit on 4/(/) = 8 /(5/) = 6/5 ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved.

THE SIMPLEX TABLEAU The variable enters the basis with the value 6 6 min 8, = 5 5 and 5 is replaced as a basic variable by We need to put the equations + + + 4 = 4 (* ) 5 + 4 + 5 = (** ) into canonical form for the basic variables and ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 4

THE SIMPLEX TABLEAU The following elementary row operations are used multiply (** ) by /5 and add to (*) 7 + + = 4 5 5 5 5 5 multiply (** ) by /5 6 6 + + = 5 5 5 5 4 5 and construct the corresponding tableau ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 5

THE SIMPLEX TABLEAU c j 5 constraint c B basic variables 4 5 constants /5 /5 /5 7/5 5 6/5 /5 /5 6/5 c T 6/5 9/5 /5 Z = 8/5 c j implies optimality 6. > 5 ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 6

SIMPLEX TABLEAU EXAMPLE We put this problem into standard form: ma Z st.. = + canonical form + + = 4 + + 4 = 4 + = 5, 4, 5,, are fictitious variables ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 8 5

SIMPLEX TABLEAU EXAMPLE c B c j basic variables 4 5 constraint constants 4 4 4 5 c T Z = column corresponding to c = c c ( T ) j j B j ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 9

SIMPLEX TABLEAU EXAMPLE c T The data in indicates that the highest relative profits correspond to so want to make a basic variable To bring into the basis requires to evaluate 4 min,, = and so replaces 5 with the value We evaluate the basic variable at the adjacent basic feasible solution and convert into canonical form; the new tableau becomes ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 4

SIMPLEX TABLEAU EXAMPLE We reproduce here the calculation of the c T components c c c ( T column corresponding to ) j = i j j B for each nonbasic variable j Note that for each basic variable i by definition c = i ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 4

SIMPLEX TABLEAU EXAMPLE The calculations give c = c = 5 = 5 c c c 4 5 = = by definition since is in the basis by definition since is in the basis by definition since is in the basis = = ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 4 4 indicates possible improvement

SIMPLEX TABLEAU EXAMPLE Clearly, the only choice is to get into the basis and so we need to establish the limiting condition from the three equations by evaluating and so replaces 4, which becomes a nonbasic variable min { 7,, } = We need to rewrite the equations into canonical form for and and construct the new tableau ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 44

SIMPLEX TABLEAU EXAMPLE c B c j basic variables 4 5 constraint constants /5 8/5 6 /5 /5 /5 /5 4 c T c j j optimum Z = 4 ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 45

SIMPLEX TABLEAU EXAMPLE An optimum is at the solution of given by 8 5 4 5 5 4 5 5 4 5 + = + = + + = 4 = 5 = = 6 = = 4 ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 46 5 5 5 6 4 canonical form for, and

LINEAR PROGRAMMING EXAMPLE Consider the following LP ma Z s. t. = + + 4 + 4 The graphical representation corresponds to ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 47

LINEAR PROGRAMMING EXAMPLE The tableau approach leads to C which is an optimal solution with = 4, =, = 6, =, = 4 5 Note that any point along CD has Z = 4 and as such D is another optimal solution corresponding to an adjacent basic feasible solution We may obtain D from C by bringing into the basis the nonbasic variable 5 in Tableau ; note that c = 5 ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 49

LINEAR PROGRAMMING EXAMPLE We may choose 5 as a basic variable without effecting Z since the relative profits are ; we compute the limiting value of 5 The limit is imposed by which, consequently, leaves the basis The corresponding tableau is: ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 5

LINEAR PROGRAMMING EXAMPLE c B c j basic variables 4 5 constraint constants 5/8 /8 5/4 /8 /8 /4 /4 /4 5/ c T c j j Z = 4 ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 5

LINEAR PROGRAMMING EXAMPLE The adjacent feasible solution is given by 5 5 =, =, = 4 =, 5= 4 4 Note that at this basic feasible solution, c j j and so this is also an optimal solution ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 5

ALTERNATE OPTIMAL SOLUTION In general, an alternate optimal solutions is indicated whenever there eists a nonbasic variable with c = j j in an optimal tableau; such a situation corresponds to a non unique optimum for the LP ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 5

MINIMIZATION LP Consider a minimization problem min Z = c st.. In the simple scheme, replace the optimality check by the following : if each coefficient stop; else, select the nonbasic variable with the most negative value in c to become the new basic variable i = ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 54 n A = b i i is c j

MINIMIZATION LP Every minimization LP may be solved as a maimization LP because of equivalence T min Z = c st.. A = b T ma Z = ( c ) st.. A = b with the solutions of Z and Z related by { } = ma{ Z } min Z ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 55

COMPLICATIONS IN THE SIMPLEX METHODOLOGY Two variables j and k are tied in the selection of the nonbasic variables to replace a current basic variable when c = c ; the choice of the new nonbasic variable to enter the basis is arbitrary Two or more constraints may give rise to the same minimum ratio value in selecting the basic variable to be replaced j k We consider the eample of the following tableau ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 56

COMPLICATIONS IN THE SIMPLEX METHODOLOGY c j / constraint c B basic variables 4 5 6 constants 4 c T / Z = candidate for basic variable ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 57

COMPLICATIONS IN THE SIMPLEX METHODOLOGY in selecting the nonbasic variable 4 to enter the basis, we observe that the first two constraints give the same minimum ratio: this means that when 4 is first increased to, both the basic variables and will reduce to zero even though only one of them can be made a nonbasic variable we arbitrarily decide to remove from the basis to get the new basic feasible solution: ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 58

COMPLICATIONS IN THE SIMPLEX METHODOLOGY in the new basic feasible solution =, =, =, =, =, and =, 4 5 6 we treat as a basic variable whose value is, the same as if it were a nonbasic variable ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 6

DEGENERACY A degenerate basic feasible solution is one where one or more basic variables is Degeneracy may lead to a number of complications in the simple approach: an important implication is a minimum ratio of, so that no new nonbasic variable maybe included in the basis and therefore the basis remains unchanged We consider the following eample tableau ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 6

COMPLICATIONS IN THE SIMPLEX METHODOLOGY c B c j basic variables 4 5 / 6 constraint constants 4 / / 5 / / c T / Z = 4 ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 6

DEGENERACY the logical choice being the nonbasic variable 6 to enter the basis; this leads to finding the limiting constraint from two equations = 6 = 6 4 and no constraint in the third equation; thus 5 6 = min 4,, { } ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 6

DEGENERACY Degeneracy may result in the construction of new tableaus without improvement in the objective function value, thereby reducing the efficiency of the computations: theoretically, an infinite loop, the so-called cycling, is possible Whenever ties occur in the minimum ratio rule, an arbitrary decision is made regarding which basic variable is replaced, ignoring the theoretical consequences of degeneracy and cycling ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 64

MINIMUM RATIO RULE COMPLICATIONS The minimum ratio rule may not be able to determine the basic variable to be replaced: this is the case when all equations lead to as the limit Consider the eample and corresponding tableau ma Z = + st.. + = + + 4 = 4, i =,...,4 ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 65 i

MINIMUM RATIO RULE COMPLICATIONS c B c j basic variables 4 constraint constants 4 4 c T Z = The nonbasic variable enters the basis to replacing 4 and the new tableau is ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 66

MINIMUM RATIO RULE COMPLICATIONS c B c j basic variables constraint 4 6 constants 4 c T Z = We select to enter the basis but we are unable to get limiting constraints from the two equations ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 67

MINIMUM RATIO RULE COMPLICATIONS + = 6 = 4 + = 4 = In fact, as increases so do and and Z and therefore, the solution is unbounded The failure of the minimum ratio rule to result in a bound at any simple tableau implies that the problem has an unbounded solution ECE 7 5 9 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved. 68