Linear Programming for Planning Applications

Transcription

1 Communication Network Planning and Performance Learning Resource Linear Programming for Planning Applications Professor Richard Harris School of Electrical and Computer Systems Engineering, RMIT Linear programming formulations are etremely important for specifying the problems associated with planning a communications network.in this Learning Guide we shall study Linear Programming and the methods used to solve them. In particular, we shall try a geometric approach and a formal method known as the Simple Method. Sample eercises are included as part of the Self Assessment Questions located elsewhere as part of the learning package. We shall begin by looking at a simple illustrative eample and then eplore the special terminology associated with this mathematical tool. Finally we shall investigate the many complications associated with formulating and solving Linear Programming problems. (c) Richard Harris, CATT Centre RMIT Page 1

2 Presentation Outline Linear Programming terminology The Simple Algorithm The Simple Tableau A Sample Linear Program using the Simple Tableau Resolution of complications in the Linear Programming formulation and tableau computations. Artificial Variables The Two-phase Simple Method Linear Programming for Communications Applications Slide 2 There is a great deal of special terminology associated with linear programming. We shall introduce the many different terms as we progress through the topic. The Simple Algorithm will be introduced in general terms initially but later on we shall look at a detailed description and solve a simple eample by hand. The method of solving an LP problem is to set up a special tableau a table of values in a matri form that can be manipulated to obtain the optimal solution. The sample problem will show how this tableau method is used. The net section will describe a variety of different complications associated with trying to ensure that a problem is in a form that can be solved by the Simple method, or that may arise during the solution process. The artificial variable method is an approach that overcomes the problems caused by the fact that we cannot find a starting solution to a problem. We shall discuss the role of these special variables and then describe two approaches to using these variables to find a starting point or determine that no feasible solution can eist for the problem. (c) Richard Harris, CATT Centre RMIT Page 2

3 Objectives To develop an understanding of the terminology used in linear programming. To gain practice and proficiency at formulating a linear programming problem and epressing it in standard form. To be able to apply the Simple Algorithm to a simple LP problem using the Simple Tableau. To be able to resolve complications in the formulation and solution of linear programming problems. To be able to apply the notion of artificial variables in the Simple Tableau. Linear Programming for Communications Applications Slide 3 The above objectives should be used as a guide to indicate what are the important points in undertaking this learning material. These principal objectives are repeated from the Topic Learning Guide instructions. (c) Richard Harris, CATT Centre RMIT Page 3

4 Additional References There are many books on linear programming, please look in the RMIT Library in the area of Operations Research and find a suitable tet. Linear Programming for Communications Applications Slide 4 Books by the following authors are classics, but many other books are very suitable. Hillier and Lieberman Taha The RMIT Library has many books on this topic. Choose one and study the sections related to the topics in this Learning Guide. (c) Richard Harris, CATT Centre RMIT Page 4

5 Linear Programming The function of linear programming is is to allocate limited resources among competing activities in an optimal way. Linear Programming for Communications Applications Slide 5 One thing that you may be curious about is the term Linear Programming. We tend to think of Programming as meaning something that we program on a computer. Whilst it is true that many large scale linear programming problems are solved on a computer, the term actually originates from the ideas of planning. The ideas behind linear programming emanated from the Second World War when optimising methods were sought for clearing mines and locating submarines using depth charges etc. These days, most Linear Programming problems are solved using a computer. We shall concentrate on the method attributed to George Dantzig, but there is a new method developed in recent times called the Kamakar Algorithm that solves an LP using interior point methods. Interestingly, the interior point method began when attempts were being made to solve large scale LP problems in telecommunications. (c) Richard Harris, CATT Centre RMIT Page 5

6 Mathematical Formulation of a Linear Program Model The general formulation of the linear programming problem may be stated as follows: We wish to find the values of n decision variables such that we maimise or minimise z = j= 1 c j j Subject to the constraints: n j= 1 aij j = bi 0 n for i = 1,...,m for j =1,...,n j Linear Programming for Communications Applications Slide 6 The general LP is described in the above slide. Note that the function to be optimised is a linear combination of the decision variables hence the name Linear Programming. Note that it is important to show all the constraints for the problem including the fact that the decision variables are non-negative. In our analysis we shall tend to show maimisation problems rather than minimising the function z. It does not matter which way we choose to optimise this function since we can always multiply z by 1 and maimise this if we want to minimise the original function z. The above formulation is the general version of a linear program. We shall see in the net few slides that we normally transform this problem into a special canonical form and this format is the foundation for the solution techniques that we shall employ. (c) Richard Harris, CATT Centre RMIT Page 6

7 Alternative Matri Form of an LP It is possible to generalise this to matri form to give: ma z = c T The vector c is called the price vector or the cost row. Subject to: Where: A = b and 0 1 T = and c = ( c1,..., cm ) n Linear Programming for Communications Applications Slide 7 The above material is included for completeness. Matri formulations of an LP are rather more concise than the verbose methods that spell out the use of summation signs etc. In the above formulation, the price vector or cost vector/row is a row vector and is a column vector of the same number of elements. The matri A is an m n matri and b is an m 1 column vector. (c) Richard Harris, CATT Centre RMIT Page 7

8 Standard Form of an LP - 1 The problem which we have defined on previous slides contained inequalities. In developing solution procedures we shall find it much easier to work with equality constraints. We shall now attempt to reformulate the above problem through the use of slack and surplus variables. Consider a set of constraints of the form Slack variables: n j= 1 a ij j b for i =1,...,K We shall define a new variable to add to the left side of the inequality (1) such that n These new variables are called slack variables. i aijj + n+ i = bi for i =1,...,K j= 1 n+ i 0 for i = 1,...,K (1) We now consider transforming our LP problem into a standard format that has equalities for the constraint rows. All inequalities are removed by the incorporation of slack or surplus variables. The net few slides show how this transformation process is achieved. For the inequalities of the type shown in equation (1) we simply add a new variable. This new variable is known as a slack variable and is positive or zero. It is introduced at no cost and it simply represents the amount of additional resource on the left hand side that would be needed to produce the resource b i shown on the right hand side. (c) Richard Harris, CATT Centre RMIT Page 8

9 Standard Form of an LP - 2 Note that, in introducing these slack variables, we should also introduce matching elements to the price vector so that: c n+i = 0 for i=1,,k The name "slack variables" is derived from the fact that each of these variables represents the difference between the maimum available resource b i and the amount actually used. Surplus variables: Suppose that some of the constraints are of the form: n j= 1 a b for i = K +1,...,L ij j i Then, we add surplus variables to produce n j= 1 a ij j = n+ i b for i i = K +1,..., L For the case where the left hand side eceeds the right hand side, we introduce surplus variables to indicate the amount of resource on the left hand side that eceeds the amount of resource shown on the right hand side. Once again, we introduce these surplus variables at no cost. (c) Richard Harris, CATT Centre RMIT Page 9

10 Standard Form of an LP - 3 Once again, the surplus variables are so named because each one represents the amount by which the minimum requirement has been eceeded. As before, we add zero entries to the price vector c, to account for these new variables. Having introduced these new slack and surplus variables, we can restate the LP problem using only equalities, thus n maz = j= 1 a ij j n j= 1 = b i 0 c j j for i = 1,...,m for j =1,...,n j Linear Programming for Communications Applications Slide 10 The problem is stated as a maimisation, but this is not essential to the discussion. The standard canonical form as it is called consists of equalities for all of the constraints. Note that we do not have to introduce any constraints for the simple inequalities required for the decision variables as we would never finish adding slack or surplus variables to the problem!! Note that it is possible to have special bounds on variables in an LP formulation and these can actually be taken into account by a special process instead of using slack or surplus variables. These constraints look like the following: M 2 (c) Richard Harris, CATT Centre RMIT Page 10

11 LP Terminology 1 A Conve Set is a collection of points, such that for each pair of points, the entire line segment joining the points is also in the collection. Eamples: Conve Not Conve Not Conve Conve Linear Programming for Communications Applications Slide 11 The simple test for conveity is to draw a straight line through the region between two points. This straight line has to always pass through the collection. A set of linear inequalities is generally a conve set. (c) Richard Harris, CATT Centre RMIT Page 11

12 LP Terminology 2 An etreme point of a conve set is a point in a set that does not lie on any line segment that joins two other points in the set. If a set has a finite number of etreme points then this is called a polyhedral conve set. Etreme points Conve Linear Programming for Communications Applications Slide 12 One of the most important aspects of a linear program is that the optimal solution to an LP occurs at an etreme point of the conve polygon formed by the constraint set of the LP. In principle this means that if we had a systematic method for generating all of these etreme points then we would be able to find the optimal solution to the LP simply by enumerating these points. Unfortunately, we quickly discover that the number of such points grows very rapidly and that we are not able to do this in practice without requiring a great deal of computational effort. The Simple Method that we shall consider shortly is a systematic method of generating etreme points in such a way that at each iteration, the cost function value is improved. (c) Richard Harris, CATT Centre RMIT Page 12

13 LP Terminology 3 A feasible solution is a value of = ( 1, 2,, n ) that satisfies all of the constraints, viz: A= b 0 An optimal feasible solution is a feasible solution that optimises the objective function. A basic feasible solution corresponds to an etreme point in the set of feasible solutions. Linear Programming for Communications Applications Slide 13 It should be obvious from previous comments, that the optimal solution to an LP will occur at a basic feasible point. It should be noted that not every intersection point between equations will be feasible points: Not feasible Feasible (c) Richard Harris, CATT Centre RMIT Page 13

14 Properties of Feasible Solutions The collection of feasible solutions to a linear programming problem forms a conve set. If a feasible solution eists, then a basic feasible solution eists. The number of basic feasible solutions for an LP problem is finite. If the objective function has a finite optimal value, then at least one optimal solution is a basic feasible solution. Linear Programming for Communications Applications Slide 14 With respect to the final point, it is possible to have multiple solutions to an LP problem. This situation would usually mean that a whole plane is a solution to the problem. Consequently, we see that there would have to be two basic feasible solutions connected by a plane and the solutions along that plane would also be optimal. Can we prove this? Suppose that there are two feasible solutions that are optimal with cost Z * for the stated LP problem. Then in this case from the feasibility point of view A 1 = b A 2 = b Now consider a linear combination of the two solutions. We show that any point on a straight line joining the two points is feasible. Then we show that it has the same cost function: The cost is given by: A( λ + (1 λ) ) = λa + (1 λ) A = λb+ (1 λ) b * = b T T c= c ( λ + (1 λ) ) 1 2 T = λc + (1 λ) c T 1 2 = λz + (1 λ) Z = Z * * (c) Richard Harris, CATT Centre RMIT Page 14

15 Basis for Linear Programming The basis of a set of vectors is a collection of linearly independent vectors taken from a set of vectors such that every vector in the set is a linear combination of the vectors in the collection. Question: Are the following vectors linearly dependent or independent? Give a reason for your answer = 3; 2 = 4 ; 3 = Linear Programming for Communications Applications Slide 15 The vectors can only be linearly independent if it is not possible to find constants a and b such that a 1 + b 2 = 3. Look at the above vectors and try the following: =? What is the result of the above computation? The above computation gives 3 of course! Thus the vectors are not independent. (c) Richard Harris, CATT Centre RMIT Page 15

16 Equivalence of Systems of Equations The central idea of linear programming is to find the solution of a system of linear equations that maimises or minimises a given objective function. Two systems of linear equations are said to be equivalent if both systems have the same solution set. (ie. A solution to one system is automatically a solution to the other and vice versa.) There are two types of elementary row operations that one can use to obtain equivalent systems: Multiply any equation in the system by a non zero number. Add to any equation, a constant multiple of any other equation in the system. Linear Programming for Communications Applications Slide 16 The elementary row operations are standard procedures in the solving of systems of linear equations so they should not come as a surprise. If you are familiar with the process of solving linear equations using the Gauss-Seidel methods, then you are well-equipped to see how to perform the Simple Method using standard elementary row operations. (c) Richard Harris, CATT Centre RMIT Page 16

17 Canonical Form 1 Consider the following system of linear equations in five unknowns denoted by: = = (S1) Since there are more unknowns than there are equations, this system will have more than one solution. The collection of all possible solutions is called the solution set. By a process of row operations, we can produce the following: = = 2 (S2) Linear Programming for Communications Applications Slide 17 What row operations have been performed on the above set of equations? Try the following: Take the second equation (Row 2) and subtract the first equation (Row 1) from it: This gives: = = Now we add twice the new Row 2 to Row 1: = = This gives us the desired result! = = (c) Richard Harris, CATT Centre RMIT Page 17

18 Canonical Form 2 With the solution 1 6 = 2 2 and 3 0 = Notice that we could have solved using other variables. The system given in S2 is said to be an eample of a canonical system. The variables 1 and 2 are said to be basic variables of the canonical system. Linear Programming for Communications Applications Slide 18 Notice that any two other variables could have been made basic variables by performing suitable row operations. At times the introduction of slack variables into the standard form of an LP problem automatically produces a canonical system. (c) Richard Harris, CATT Centre RMIT Page 18

19 Definitions A variable j is said to be a basic variable in a given equation if it appears with a unit coefficient in that equation and zeros in all other equations. Those variables which are not basic are called nonbasic variables. By applying elementary row operations, a given variable can be made a basic variable. This is called a pivot operation. The solution obtained from a canonical system by setting the nonbasic variables to zero and solving for the basic variables is called a basic solution. A basic feasible solution is a basic solution in which the basic variables are non-negative. Linear Programming for Communications Applications Slide 19 Take note of the above definitions since they are part of the language of Linear Programming! (c) Richard Harris, CATT Centre RMIT Page 19

20 The Simple Method The Simple Method was developed by George Dantzig and is an iterative procedure for solving linear programs epressed in the standard equality form. The Simple Method further requires that the constraint equations be epressed as a canonical system from which a basic feasible solution can readily be obtained. Simple Method -- Outline Start with an initial basic feasible solution in canonical form. Improve the initial solution if possible by finding another basic feasible solution with a better objective function value. Continue to find better basic feasible solutions improving the objective function until this is no longer possible. Linear Programming for Communications Applications Slide 20 The method was first used in a variety of different contets including the solution of military problems. Nowadays, the techniques is widely used for the solution to huge and comple problems in the mining industry, telecommunications, transportation and factories with etensive planning requirements. (c) Richard Harris, CATT Centre RMIT Page 20

21 The Simple Algorithm Without loss of generality, we can consider the maimisation of the objective function. First Iteration: Introduce slack variables as required to put the problem into standard form. Select the slack variables as the initial basis variables. Go to step Choose new entering basic variables by finding the nonbasic variable which, when increased would increase the objective function at the fastest rate. 2. Choose new leaving basis variable by finding that nonbasic variable which first reaches zero as the new entering basis variable is increased. 3. Obtain new basic feasible solution by performing pivot operations which set the new nonbasic variable equal zero. 4. Determine whether this solution is optimal by considering the modified cost row and seeing whether the elements in that row corresponding to the nonbasic variables are all non-negative. If true then STOP, otherwise go back to step 1. If we want to minimise a function, simply multiply the cost function by -1 and then continue with each of the steps outlines above. Alternatively you can interpret the method differently by changing the sign or direction of optimisation. Suit yourself! The method revolves around performing simple pivot steps once the correct entering basic variable has been identified. (c) Richard Harris, CATT Centre RMIT Page 21

22 Sample Problem Consider the following eample problem: Maimise: Z = Subject to: ; 6, 0 Let us begin to apply the Simple algorithm by rewriting the inequality constraints by introducing 3 slack variables 3, 4 and 5 to give: = = = Linear Programming for Communications Applications Slide 22 We start by ensuring that we have an appropriate canonical form to work with. The slack variable approach is useful in this case to help us get the appropriate form and start the Simple method. Now that we have a starting solution with this canonical form, what is the point in the graph that we have started at? Solution: With respect to the original problem we are at the origin (0,0) since the slack variables will have the values 18, 4 and 6 respectively. (c) Richard Harris, CATT Centre RMIT Page 22

23 Geometric Interpretation We begin by selecting the 3 slack variables as the basis, ie. = = = The remaining two variables will be declared to be nonbasic and set to zero. Linear Programming for Communications Applications Slide 23 As we can see from the geometric interpretation, we are located at the origin eactly as predicted on the previous page. Can you guess which way we shall go once the Simple Method is applied? Will it be to the point 1 = 4 or the point 2 = 6? One way to see this from a geometric point of view would be to plot the cost function and see its slope relative to the various constraint lines that are showing on the graph. Try it and see. (c) Richard Harris, CATT Centre RMIT Page 23

24 Simple Eample 1 This means that we have an initial solution given by: = = 0 = 4 = 6 = 18 Z 3 5 = Starting solution Linear Programming for Communications Applications Slide 24 Here is the answer to the question posed and answered algebraically on previous pages. (c) Richard Harris, CATT Centre RMIT Page 24

25 Simple Eample 2 In matri form, the problem can be written as: = Z Where we have chosen to include the objective function in the matri representation. The above representation suggests a tabular form for solving linear programming problems. Linear Programming for Communications Applications Slide 25 Writing the equations in this matri form is a way of indicating how we could establish a tabular method of solution. (c) Richard Harris, CATT Centre RMIT Page 25

26 Tableau Form of an LP Basis Constraint Matri Z RHS Vector In the above tableau we have defined the basis as per step 3 of the algorithm. The net step is to determine whether the solution is optimal. This is done by checking the sign of the elements of the modified cost row which correspond to the nonbasic variables. Notice that the elements in the modified cost row are zeros for basic variables. Linear Programming for Communications Applications Slide 26 Finally we show you the tableau form of an LP. Notice how closely it resembles the way that we wrote the matri form on the previous page. (c) Richard Harris, CATT Centre RMIT Page 26

27 Selecting the Incoming Basic Variable Eamination of the modified cost row from the previous page indicates that the solution is not yet optimal. Thus we shall proceed to step 1 of the algorithm Z Z We choose the nonbasic variable that when increased would increase the objective function at the fastest rate. Which nonbasic variable satisfies this requirement and why? Linear Programming for Communications Applications Slide 27 Note that some authors prefer to have the cost row at the top of the tableau. If you prefer this, it makes no difference to the outcome. We want to increase the objective function at the fastest rate, so the obvious choice is to find a variable that has the steepest gradient. Note that we sometimes call the cost row the modified cost row since it has been modified to take account of the constraints. We suggest that the variable to increase at the fastest rate would be 2 as its modified gradient is -5 in the cost row. Did you guess correctly from the previous pages when we asked which variable would come into the basis first and increase the cost by the greatest amount? What is the biggest value that 2 could take? We know that 2 = 6 is the upper bound. Will the Simple method adjust 2 to this value? Let s see.. If we look at the original cost function, we see that setting 2 = 6 will result in a cost of Z=30. If we had introduced 1 on the other hand, the cost would have been Z=12. (c) Richard Harris, CATT Centre RMIT Page 27

28 Selecting the Outgoing Nonbasic Variable Variable 2 is the new entering basic variable. We must now use step 2 to find out which basic variable should leave the basis. The variable to leave the basis is the one which reaches zero first as the new basic variable increases. Looking at the equations, we see that 4 = 0 gives 2 = 6 5 = 0 gives 2 = 9 Thus the variable 4 reaches zero first as 2 increases. Linear Programming for Communications Applications Slide 28 The variable that increases the cost at the fastest rate is 2 as argued earlier. Here are the equations referred to in the slide. Change 2 to 6 and see what happens to the values of the basic variables listed on the left hand side equations: = = = (c) Richard Harris, CATT Centre RMIT Page 28

29 Updating the Tableau We now perform a pivot operation on the entire matri in the tableau (including the cost row Z.) Entering basic variable Leaving basic variable Z Z Pivot Linear Programming for Communications Applications Slide 29 We have seen that 4 gets to zero first as 2 increases, so it must be the departing basic variable. We perform a pivot operation to move it out of the basis and move the new basic variable ( 2 ) into the basis in that row. We must perform simple row operations to drive the coefficients of 2 to zero in all rows ecept the second row. In that row we want a coefficient of 1 to meet the requirement of a basic variable. (See earlier for a definition of a basic variable if you have forgotten this!!) How do we do this pivot operation? We replace row 3 with the old row 3 minus 3 times row 2. We add 5 times row 2 to the cost row to drive the coefficient down to zero the coefficient of a basic variable in the cost row must be zero. (c) Richard Harris, CATT Centre RMIT Page 29

30 Second Tableau Performing the pivot operation on the tableau produces the following: Z Z Is this optimal? NO! The new entering basic variable is 1. The leaving basic variable is found by considering the minimum: Min{4/1, --, 6/3} = 2 Linear Programming for Communications Applications Slide 30 As outlined on the previous page we end up with the above tableau after performing the pivot steps. We can tell that the solution is not optimal because there is a negative valued coefficient in the cost row. Where will we end up after the net step? Look at the diagram on earlier pages that shows the geometric view. Obviously we would not want to return to the origin (0,0) and we are now located at the point (0,6). We epect to move towards the intersection of the line 2 =6 and the curve = 18. What is this point? oanswer: 1 = 2, 2 = 6 We shall see whether our guess is correct in a moment, but we need to formalise the process of selecting the correct pivot. (c) Richard Harris, CATT Centre RMIT Page 30

31 Selecting the Leaving Basic Variable - 1 Variable 2 is the new entering basic variable. We must now use step 2 to find out which basic variable should leave the basis. = 4 (1) 3 1 = 6 (2) 4 2 = (3) The variable to leave the basis is the one which reaches zero first as the new basic variable increases. Look at the original set of equations: Increasing 2 has no effect in equation (1) In equation (2) we can increase 2 from 0 to 6 before 4 vanishes. In equation (3) we can increase 2 from 0 to 9 and then 5 vanishes. Linear Programming for Communications Applications Slide 31 Look back to the description of the Simple Method. How can we solve the problem of identifying the pivot point? The above slide shows a similar argument to the one given previously in the notes page. (c) Richard Harris, CATT Centre RMIT Page 31

32 Selecting the Leaving Basic Variable 2 Thus variable 4 reaches zero first as 2 increases. We can formalise this procedure in the following way: Using the column j identified as the one that contains the new entering basic variable, determine the row k where the minimum of the ratios defined in the set: b a k kj b i = min ; where aij > 0 and i = 1,..., m aij The The basic basic variable identified by by row row k is is the the departing basic basic variable. Linear Programming for Communications Applications Slide 32 The formal definition is shown in the equation above. (c) Richard Harris, CATT Centre RMIT Page 32

33 Selecting the New Basis The new entering basic variable is 1. The leaving basic variable is found by considering the minimum ratios: Z Ratios / Z /3 Pivot Linear Programming for Communications Applications Slide 33 We simply work out the legal ratios they are shown on the right hand side of the tableau for easy reference. The new pivot point is thus found by moving along the column with -3 in the cost row and taking the ratios as described in the formula on the previous slide. One more point to notice is that the label on the left hand side of Row 2 has been changed from 4 to 2 as a result of the first pivot operation that we did earlier. This is to signify that 2 is in the basis now and has replaced 4. After performing all of the ratios, the smallest ratio is 6/3 = 2 and this occurs on Row 3. We see that the variable identified as 5 on the left hand side will now be replaced by the new incoming basic variable 1. Check the net tableau to see whether we correctly labelled Row 3 with the new incoming basic variable: 1. (c) Richard Harris, CATT Centre RMIT Page 33

34 Pivot Operation We now perform a pivot operation to place 1 into the basis and force 5 to leave the basis: Z Z /3 1/ /3 1/ Solution is optimal! Linear Programming for Communications Applications Slide 34 Can you see that we labelled Row 3 correctly? The pivot process has resulted in a cost row that has only positive or zero coefficients. Check that this implies optimality has been reached. What is the solution and how do we read it from the tableau? Look at the left hand column and the far right hand column. Matching the labels we can read off the solution as o 3 = 2 o 2 = 6 o 1 = 2 o Z* = 36 is the optimal solution cost function value. Our guess about the net point to visit is also correct! It was (2,6) of course. The point is optimal and is a basic feasible solution. (c) Richard Harris, CATT Centre RMIT Page 34

35 Geometric Interpretation 2 First iteration 6 Second iteration is optimal Starting solution Our problem required 2 iterations. Note that there were 3 equations in 5 unknowns which would have meant 10 possible basic solutions. Linear Programming for Communications Applications Slide 35 The comments on the previous page can now be illustrated on the graphical version of the problem as shown in the slide above. We refer to 10 possible basic solutions. Is it true that they are basic feasible solutions? Eercise: o NO! There are only 5 basic feasible solutions. Can you identify them on the graph above? owrite down the 5 basic feasible solutions in terms of the values for the 5 real and slack variables. (c) Richard Harris, CATT Centre RMIT Page 35

36 Simple Algorithm Flow Chart Initial Basic Feasible Solution Compute z j -c j for all vectors not in the basis. Perform pivot operation with (r,k) a All z c 0? j Some z c < 0. All a 0? ij j Optimal bounded solution Unbounded solution Linear Programming for Communications Applications Slide 36 j No min( z c ) = z c r rk j No j j k k No j Yes Yes i = min ; aik > 0 i aik There are some special cases identified in the above flow chart that we have not considered yet. Don t worry, we ll get to them shortly. (c) Richard Harris, CATT Centre RMIT Page 36

37 Unbounded Solutions On the previous page, the flow chart showed a situation where z j -c j < 0 and yet all of the a ij <0. Consider what would have happened in our previous eample if the equations had been = = = If we now attempt to increase 2, what happens? Clearly there is no effect on the first equation and in the second and third equations, neither of the variables will vanish as we increase 2. Thus we can increase the objective without fear of violating the constraints the solutions is unbounded. Linear Programming for Communications Applications Slide 37 Here is the first of the possible unusual outcomes from the Simple method. Follow the discussion shown on the slide. Can you see what happens to the ratio computations when this condition is present? Verify that the rule shown in the flow chart is correct for this situation. (c) Richard Harris, CATT Centre RMIT Page 37

38 Summary of LP Problem Difficulties In our previous discussions we have assumed that we could find a nice canonical form solution and that there were unique entering and leaving basic variables etc. As you might have guessed, not everything is quite as simple as it would appear! 1. Minimise not maimise 2. Ties for the entering or leaving basis variables 3. Degeneracy 4. Multiple solutions 5. Variables may be unconstrained in sign 6. Can t find a canonical form starting solution 7. Redundant and inconsistent equations 8. Non positive RHS vector elements. Linear Programming for Communications Applications Slide 38 We now consider a number of special complications that are encountered whilst performing the Simple Method. Some of these problems can arise in practice and particularly for problems in telecommunications. Others will be rare but you should be aware of them just in case. (c) Richard Harris, CATT Centre RMIT Page 38

39 Minimise not Maimise! This problem can be solved in a number of ways: Change the sign of the objective function Interpret the algorithm differently. Ie. Look for the column with the largest positive nonbasic component to give the new entering basic variable and check for optimality by noting whether all the nonbasic variable entries are negative. Linear Programming for Communications Applications Slide 39 This was mentioned earlier, but it is quite easy to fi. (c) Richard Harris, CATT Centre RMIT Page 39

40 Ties for Basic or Nonbasic Variables 1 Suppose that in the modified row there is a tie, that is, more than one nonbasic variable increases at the same (maimum) rate. Then, in such cases, it is customary to break the tie by choosing the nonbasic variable with the smallest inde j. Suppose that in our previous eample the equations were: = 4 (1) 3 1 = 6 (2) 4 2 = (3) If the new entering basic variable was 2 once again, then we can see that both equations (2) and (3) vanish simultaneously when 2 = 6. Linear Programming for Communications Applications Slide 40 Ties can often lead to other complications as discussed in the following pages. (c) Richard Harris, CATT Centre RMIT Page 40

41 Ties for Basic or Nonbasic Variables 2 As in the previous case, we can arbitrarily choose either 4 or 5. We choose 4, say. Notice that when we perform the pivot operations it is necessary to change the basis. We obtain: = 4 = 0 = 6 This situation is known as a degenerate solution When a basic variable becomes zero, there may be some further complications when it comes to a new iteration. Linear Programming for Communications Applications Slide 41 Degenerate solutions can occur in practical environments. I have seen them in problems related to mobile communication channel allocation problems. (c) Richard Harris, CATT Centre RMIT Page 41

42 Degeneracy - 1 Continuing with the eample, on the previous slide, we have to swap the basis by performing a pivot step. This gives: Z = = = = 0 The new entering basic variable is 1 as the minimum ratio occurs on the last equation. Linear Programming for Communications Applications Slide 42 (c) Richard Harris, CATT Centre RMIT Page 42

43 Degeneracy 2 Pivoting in the usual way gives: Z = = = = This solution is OPTIMAL by the standard criteria. Note that degeneracy is rare in practice. Linear Programming for Communications Applications Slide 43 Whilst it is rare, you should be on the alert for it. Do not assume that it is impossible. (c) Richard Harris, CATT Centre RMIT Page 43

44 Multiple Solutions It is possible for a linear program to have multiple solutions. This situation is illustrated graphically below: Whole plane lies parallel with the objective function. This can also occur in practice but is not very common. Linear Programming for Communications Applications Slide 44 How can we tell that there are multiple solutions to an LP problem? The trick is to look at the cost row. If you see that the coefficient of a nonbasic variable is zero, then this is a signal that the problem will have multiple solutions. The reason can be seen as follows: oif a nonbasic variable has a zero coefficient, we could try to implement the pivot procedure on a suitable candidate in the column where it resides. onote that the pivot operation will not modify the cost row, so that means that the cost will be unchanged, although we have swapped a basic and nonbasic variable during the pivot process. oout earlier discussion showed that if we had two solutions to an LP then the whole plane containing them is also a solution since the points on the plane can be determined as a linear combination of the two points and the will all produce an optimal solution. (c) Richard Harris, CATT Centre RMIT Page 44

45 Unconstrained Variables Sometimes the unknowns in a problem may be able to take on any positive or negative value. Such variables as said to be unconstrained. To resolve this problem and produce an LP in standard form, we introduce two positive valued variables to replace the unconstrained variables as follows: where = y z i i i y, z 0 i i Linear Programming for Communications Applications Slide 45 This is a common trick of the trade. I have not come across this particular situation very often in network planning models, but I guess one day I ll find one. (c) Richard Harris, CATT Centre RMIT Page 45

46 Artificial Variables 1 One of the major requirements of the Simple method that we have described is the ready availability of an initial feasible solution in canonical form. Without such a solution, the Simple Method cannot be started and the tableau cannot be formed. In general, we cannot assume that an identity matri can be found in the matri A. Usually, when the set of constraints contains a or an = sign, an initial identity matri cannot be found. What should we do in such cases? Trial and error. Use artificial variables. Linear Programming for Communications Applications Slide 46 This net section is very important when we cannot set the problem in a standard canonical form. Two approaches are described. The best one seems to be the two phase method for using the artificial variables. A comparison is provided on a later slide. (c) Richard Harris, CATT Centre RMIT Page 46

47 Artificial Variables 2 We shall now concern ourselves with generating an identity matri that (we hope) will lead us to a legitimate solution to the LP problem. Given the original set of constraints A=b, consider a new set: A+ Ia = [ A: I] = b a We have augmented the original variables with a series of m additional variables which we call artificial variables. Corresponding to each variable ai, there is a unit vector ei in the augmented activity matri. Since we now have an identity matri we immediately have a basic feasible solution to the new set of constraints. Note however, that this is not a solution to the original set of constraints. Linear Programming for Communications Applications Slide 47 Please note the final comment. It is important to note that the use of artificial variables is simply a trick to force a situation where a valid starting canonical form will reveal itself once the artificial variables have been driven to zero. It is also an important method for showing that the problem is poorly formulated and has no feasible solutions. (c) Richard Harris, CATT Centre RMIT Page 47

48 Artificial Variables 3 We noted earlier that the solution to the problem after we introduce the artificial variables was not a solution to the original set of constraints. Why then are the artificial variables useful? We begin by noting that: Any solution that satisfies our augmented system that is is also a solution to to our original set of of constraints must have: a a =0 Linear Programming for Communications Applications Slide 48 (c) Richard Harris, CATT Centre RMIT Page 48

49 Artificial Variables 4 It follows from the preceding remarks that we need to work out a method for moving from the initial solution where a =b to a basic feasible solution for the original problem. There are two ways of achieving this end result, viz: The Big M Simple Method The Two Phase Method Linear Programming for Communications Applications Slide 49 (c) Richard Harris, CATT Centre RMIT Page 49

50 The Big M Method In this approach, we reason that we could assign prices in the cost row for the artificial variables which are so unfavourable, that they would be driven out of the basis and set to zero. If Z is to be maimised, then it is usual to assign a very large negative value -M to the artificial variable prices. The actual size of M is usually not specified for hand computations, but in computer programs it is customary to assign a value of at least 1000 times the largest price corresponding to a legitimate variable. Note: It It may not be be necessary to to assign an an artificial variable to to every constraint, eg. If If there is is already a suitable column in in the matri. Linear Programming for Communications Applications Slide 50 The note at the bottom of this slide is important. We need not waste time adding artificial variables to constraints if there is already a candidate for a basic variable present. (c) Richard Harris, CATT Centre RMIT Page 50

51 The Big M Method Eample 1 Consider the following LP problem: Minimise: Subject to: = Z = 1 1 3,, Firstly the problem is converted into the standard form for solution: Minimise: Subject to: = Z = = = 1 1 3,,,, (1) (2) (3) Linear Programming for Communications Applications Slide 51 Note that we have added artificial variables to only the rows that require them. (c) Richard Harris, CATT Centre RMIT Page 51

52 The Big M Method Eample 2 In equation (1) above, the slack variable is OK as a basis variable, but equations (2) and (3) have no obvious basic variables and so we introduce two artificial variables. 6 and 7 0 Finally, we introduce the Big M s into the cost row to force these artificial variables to zero as discussed previously. Thus we would minimise: Z = M + M Linear Programming for Communications Applications Slide 52 (c) Richard Harris, CATT Centre RMIT Page 52

53 Final Remarks for the Big M Method What happens if an artificial variable is still left in the basis after the optimisation, and it is > 0? Such events signal the fact that the original problem was infeasible. In other words, the original problem without the artificial variables does not have a feasible solution. Infeasibility is due to the presence of inconsistent constraints in the problem. Linear Programming for Communications Applications Slide 53 (c) Richard Harris, CATT Centre RMIT Page 53

54 Problems with the Big M Method Once the variables have been removed from the basis, they become an overhead to be retained until the optimal solution has been reached. Round-off error may be present after manipulation of such large numbers. Accuracy problems again related to the presence of large numbers and round-off. Linear Programming for Communications Applications Slide 54 (c) Richard Harris, CATT Centre RMIT Page 54

55 Two Phase Simple Method - 1 An alternative approach involving artificial variables is called the two-phase approach. It tends to be the preferred method in computer applications. Phase I Introduce artificial variables into the problem and temporarily replace the original objective function with a new one, viz: Minimise: Z n = j= 1 ja This objective function is then optimised using the standard Simple Method. (It is already in the correct canonical form.) Linear Programming for Communications Applications Slide 55 (c) Richard Harris, CATT Centre RMIT Page 55

56 Two Phase Simple Method 2 Two cases arise: Phase II The objective function is now minimised and Z=0. This means that we now have a basic feasible solution to the original problem. Proceed to Phase II. The objective function is minimised with Z 0 and some artificial variables are 0. In this case, the original problem was infeasible and we terminate the process. Using the basis established in Phase I, perform the Simple Method (ecluding the artificial variables) using the original objective function. Linear Programming for Communications Applications Slide 56 (c) Richard Harris, CATT Centre RMIT Page 56

57 Comparison of the Big M and Two Phase Methods 1. Both methods have the same basic approach. Namely, introduce artificial variables to get an initial canonical system and then they both attempt to drive them to zero as quickly as possible. 2. The sequence of tableau and basis changes are identical. 3. The number of iterations is the same, 4. The Big M method solves the linear program in one pass, while the two phase method solves it in two stages as two LP s. 5. The drawback with the Big M Method is the presence of large numbers which can create computational problems for computers. Linear Programming for Communications Applications Slide 57 (c) Richard Harris, CATT Centre RMIT Page 57

58 Redundant and Inconsistent Equations 1 To some etent the question of inconsistent equations has already been addressed through the use of the artificial variable approach. Since this method identifies when there is no feasible solution to a problem. When we introduced the artificial variables to the system and optimised, three cases can arise, viz: There are no artificial variables in the basis and the optimal solution (2-phase method) is Z=0. There are one or more artificial variables left in the basis, but they are set equal to zero. There are one or more artificial variables in the basis and they are positive. Linear Programming for Communications Applications Slide 58 (c) Richard Harris, CATT Centre RMIT Page 58

59 Redundant and Inconsistent Equations 2 Case 1: We have a basic feasible problem to the original problem, this is optimal in the case of the Big M Method. Case 2: All artificial variables are zero so the equations are consistent. If for some j, a ij 0 where i corresponds to a column of A containing an artificial variable, then we may pivot and drive it out of the basis until we get a degenerate solution. Thus the constraints are not redundant. The only way that the above procedure wouldn t work occurs when a ij =0 for each of the corresponding columns of A. In this case, it can be shown that the constraints are actually redundant. Linear Programming for Communications Applications Slide 59 (c) Richard Harris, CATT Centre RMIT Page 59

60 Redundant and Inconsistent Equations 3 Case 3: If there are artificial variables which are positive in the solution then we already know that the problem is infeasible and that this could either mean that there are solutions but not feasible ones, or that the constraints are inconsistent. Linear Programming for Communications Applications Slide 60 (c) Richard Harris, CATT Centre RMIT Page 60

61 Non-positive Right Hand Side Vector Elements 1 Finally, we consider the following LP system: Maimise: Subject to: Z = ; 6, 0 (This was our original problem, but the inequality in the first equation has now been reversed.) Suppose that we multiply the inequality by -1, then we obtain: Adding a slack variable to the LHS: = Linear Programming for Communications Applications Slide 61 (c) Richard Harris, CATT Centre RMIT Page 61

62 Non-positive Right Hand Side Vector Elements 2 If we were to introduce this slack variable into the basis now, we would have a non positive RHS vector. The solution procedure here is to introduce an artificial variable and apply the Big M mehod as before. Thus = 18 And maimise: Z = M3 Where the artificial variable is 3. Linear Programming for Communications Applications Slide 62 (c) Richard Harris, CATT Centre RMIT Page 62

63 Concluding Remarks We have now seen the standard form of an LP problem and shown how to overcome the many different modelling scenarios. We now need to consider the application of this theory to the solving of LP problems in the area of telecommunications. Linear Programming for Communications Applications Slide 63 (c) Richard Harris, CATT Centre RMIT Page 63