The Transform and Conquer Algorithm Design Technique

Transcription

1 The Transform and Conquer Algorithm Design Technique In this part of the course we will look at some simple examples of another general technique for designing algorithms, namely transform and conquer Then we will study in depth one very important (for practical uses) such algorithm, the simplex method The idea of transform and conquer is simple: when faced with a problem, figure out how to transform it into a different form or different problem, such that a solution to the transformed problem gives a solution to the original problem An Example: the Element Uniqueness Problem Consider this problem: given n integers a 1, a 2,, a n, determine whether any two of them are the same The obvious algorithm for this problem is to go through all the numbers, in order, and for each a k, see if it is the same as any of a k+1,, a n This algorithm has efficiency in Θ(n 2 ) We can apply the transform and conquer idea: first sort the numbers into, say, increasing order Then easily go through the sorted list and determine whether any two are the same The answer to the question for the sorted list is the same as the answer for the original list This algorithm has efficiency in Θ(n log n), since the pre-sorting takes a number of comparisons in Θ(n log n), and the scan to look for adjacent items that are the same is obviously in Θ(n), which is negligible compared to the work to pre-sort Another Example: Evaluating a Polynomial Here is an apparently simple problem that you have been solving for years: given a polynomial P (x) = a + a 1 x + a 2 x a n x n, compute its value at any desired value for x If a typical calculus student were asked to compute, say P (9) for P (x) = 3+7x 4x 2 +5x 3, they would probably compute 7 times 9, getting 63 and add that to 3, giving 66 Then they might square 9 to get 81, multiply by 4, and subtract that value from 66, and so on This algorithm has number of multiplications in Θ(n), assuming as is typical that each x k is calculated from scratch It is also irritating because it requires computing individual terms and storing them somehow, and then adding them together If we apply the transform and conquer technique (this is all a little contrived it s unlikely that Horner set out to design his algorithm by thinking hmmm, can I transform a given polynomial into an equivalent form that is more efficient to evaluate? ), we might think to write the previous polynomial as P (x) = (((5x 4)x + 7)x + 3 CS 45 Fall 217 Page 56

2 Then computing, say, P (9) can be done with Θ(n) multiplications and no extra storage, by just pressing the keys 5 * 9 = - 4 = * 9 = + 7 = * 9 = + 3 = Yet Another Example: Solving a System of Linear Equations Consider the familiar problem of solving a system of linear equations such as 2x + 3y z = 6 4x + 7y 5z = 3 6x + 11y 4z = 4 One vaguely specified algorithm for solving this problem is to cunningly apply the various operations on the equations: multiply both sides of a single equation by a non-zero number, add a multiple of one equation to another, and rearrange the order of the equations Note that this algorithm is based on the transform and conquer idea we keep transforming the set of equations, maintaining the set of solutions, until they are in a form where the solution is obvious A practical additional transformation is to represent just the numbers in the equations in a matrix, like so: which makes it easier to put into a program or do by hand In this form, we refer to the operations on the equations as row operations Embedded Exercise 37 Working by hand, perform row operations to solve this system of equations In class I will briefly demonstrate how to use a simple application named ManualSimplex to perform row operations that will allow us to interactively transform a given coefficient matrix in various ways in order to perform the simplex method CS 45 Fall 217 Page 57

3 The Linear Programming Problem and the Simplex Method Now we want to study a problem, known as linear programming, and one algorithm that solves this problem, known as the simplex method A lot of real world problems, from computer science, mathematics, various sciences, engineering, business, and economics, can be formulated as linear programming problems In addition to its practical uses, our work in this section will give us an important case study in computational complexity in the last part of the course, when we study computational complexity and P vs NP Software packages that implement the simplex method have been well-developed for half a century, and are still in common use When I taught MTH 325 in the early 198 s, the text we used said that half the computer time being used on the planet was being devoted to performing the simplex method Lists of the most important algorithms typically include the simplex method We have mostly been avoiding numerical algorithms, since they are covered in depth in MTH Technically the simplex method is a numerical algorithm, subject to rounding error issues, but if it were executed with perfectly accurate arithmetic computations, it would behave like a non-numerical algorithm And, it uses simple mathematical ideas in a profoundly powerful way Finally, we will later see how to solve TSP by using the simplex method as a tool The Linear Programming Problem (LP) The linear programming problem, known henceforth as LP, is to find the optimal value of a function, subject to certain constraints on the arguments, where the function and the constraints are linear, namely of the form of a sum of terms consisting of a scalar times a variable, such as 3x 1 + 2x 2 An instance of LP has the standard form max c T x subject to Ax = b x where standard matrix notation is used, n is the number of decision variables, m is the number of equality constraints, x and c are n by 1 matrices, A is m by n, b is m by 1, and for any matrix M, M means that all the components of M are non-negative We also assume that b Later we will see how all sorts of problems that don t have the standard form can be transformed to an equivalent problem that fits this form So, all our theory, and the algorithm we present for solving an LP, will be based on this form CS 45 Fall 217 Page 58

4 First Example of an LP Suppose a farmer has 12 acres of farm land to be planted in either corn or wheat Suppose further that the farmer has 12 units of water to be devoted to these crops, and 18 monetary units with which to buy seed for the crops Each acre of corn requires 8 units of water for the season, and each acre of wheat requires 15 Seed for an acre of corn costs 6 monetary units, while an acre of wheat costs on 3 units per acre If the expected produce from an acre of corn will sell for 2 monetary units and an acre of wheat will produce 3 monetary units, how many acres of corn and wheat should the farmer decide to plant? To solve this word problem, we begin, like with many such problems, by identifying the variables whose values we need to determine In the world of LP, these are known as decision variables So, let x be the number of acres of corn to plant, and y be the number of acres of wheat to plant With these variables, we easily translate the facts of the problem into precise mathematical quantities max 2x + 3y subject to x + y 12 8x + 15y 12 6x + 3y 18 x, y We can graph the boundary lines for the constraints, and some level sets for the objective function (sets on which all points have the same function value) for this problem: (in units of 1) CS 45 Fall 217 Page 59

5 We can solve this instance of LP by finding the feasible region the region consisting of points (x, y) that satisfy all the constraints and then finding the level set which crosses the feasible set and has the largest objective function value We can do this graphically, or by solving pairs of linear equations to find candidates for the winning point CS 45 Fall 217 Page 6

6 Some Theory of LP and the Brute Force Approach Consider an instance of LP, where the coefficient matrix A is m rows by n columns To avoid bogging down in a lot of math, no matter how much fun that might be, we will simply state the crucial theoretical facts that we need to know in order to understand our two approaches to solving LP (brute force and the simplex method) Big Fact: an optimal point can be obtained by selecting m of the n variables to be allowed to be positive, while holding the other n m variables at The m variables that are allowed to be positive are known as basic variables, and the other n m are known, in a stroke of creative nomenclature, as non-basic variables It is possible that there are no points that satisfy both the constraints and the restrictions Such an instance of LP is said to be infeasible The set of points that satisfies both Ax = b and x is known as the feasible set The rest of what we need to know is based on a standard principle of matrix algebra: if we partition a matrix computation in any way that makes sense, then any fact that would be true if the blocks of matrices were numbers is still true Using this principle, let s partition x into m basic variables and n m non-basic variables, and purely for notational convenience pretend that the variables have been rearranged so that [ the basic ] variables come first, followed by the non-basics Then we can partition xb x = We will similarly rearrange the columns of A, denoting the first m columns x N of A by B, and the remaining n m columns by N, [ so ] we have partitioned A = [B N] cb And, we rearrange the components of c so that c = We also want to express the objective function in the same form as the constraints To do this, we introduce a new variable z, and define z = c T x To put this in the same form as the constraints, we need a constant on the right-hand side, so we subtract c T x from both sides, obtaining z c T x = With all this notation, our LP becomes: c N z z 1 x T B x T N c T B c T N x B B N b CS 45 Fall 217 Page 61

7 Note that the top row in this tableau consists of variable names, for convenience, showing us which column corresponds to which variables Similarly, the leftmost column consists of the names, it will turn out, of the current basic variables The column for the z variable never changes (and in fact z doesn t appear in any equation except the objective function one), and is only there to remind us of what the objection function row means, namely 1z c T Bx b c T Nx N = Now we are ready to do some block row operations We first multiply the second row of the partitioned matrix by B 1 (which of course only works if this matrix is non-singular), obtaining the equivalent set of equations z x T B x T N z 1 c T B c T N x B I B 1 N B 1 b (noting that B 1 times the first block, which is a column of all s, gives a column of all s) CS 45 Fall 217 Page 62

8 Then we add the correct multiple, namely c T B, of row 2 to row 1 to zero out row 2 in column 1, obtaining the tableau z x T B x T N z 1 T c T B B 1 N c T N c T B B 1 b x B I B 1 N B 1 b Now, the second row says that Ix B + B 1 Nx n = B 1 b, so if we set x N =, these equations say that x B = B 1 b The first row says that or, using x N =, that z + T x B + (c T BB 1 N c T N)x N = c T BB 1 b, z = c T BB 1 b Thus, the first row of the rightmost column of the tableau gives the value of the objective function corresponding to the current choice of basic variables, and the elements below that in the rightmost column give the values of the basic variables A Brute Force Approach to LP We now know enough to develop an algorithm for LP using a brute force approach We simply try all possible choices of m variables out of n to be basic For each choice, we consider the columns for those variables to be B (though we don t rearrange them we leave them scattered wherever they are in the coefficient matrix), do row operations to transform B into I, and note the results in the last column If any of the values in B 1 b are negative, then that choice of basic variables is infeasible and is discarded For all the choices that give a nonnegative right-hand side, we note the value of the objective function, CS 45 Fall 217 Page 63

9 which is c T B B 1 b, and declare the choice of basic variables that gives the biggest objection function value as the winner The efficiency category for this algorithm is Θ( ( n m) nm), because there are ( n m) choices of basic variables, and each requires row operations to convert the basic columns to columns of the identity, with those row operations doing some work for every component of the coefficient array This efficiency category is pretty horrible! Performing the Brute Force Algorithm with ManualSimplex The in-class tutorial on ManualSimplex should let you perform the brute force algorithm comfortably, but here we can just say that you basically just need to use the arrow keys to move the cursor about and then press p to turn the cursor column into a column of the identity matrix, with the 1 in the cursor row Note that it is possible that some choices of basic variables will not give independent columns, so you won t be able to convert them into an identity matrix For example, with two decision variables, we could have constraint boundary lines that are parallel lines, like x 1 + 3x 2 5 and x 1 + 3x 2 2 Embedded Exercise 38 Solve this LP by the brute force approach; max 3x 1 2x 2 + 4x 3 + 7x 4 subject to x 1 + 2x 2 x 3 + 4x 4 = 8 6x 1 + x 2 + 3x 3 x 4 = 12 x CS 45 Fall 217 Page 64

10 The Simplex Method for Solving LP We can now continue our just the facts development of the simplex method, except in the next part everything we do will be mathematically precise and rigorous (the only fact we aren t proving in this whole discussion is that an optimal point can always be obtained by choosing m basic variables) The simplex method is a little more clever than the brute force method The idea is to let one non-basic variable, say x j, sneak into being basic We simply examine what would happen if we were to let x j increase from its current value of to some positive value, say α If we denote the value in column j of the 1 by n m sized matrix c T BB 1 N c T N (which is known as the reduced costs ) by γ, then the first row says z + γα = c T BB 1 b (because the first m values are, and the remaining n m except for γ are multiplying by non-basic variables that are being held at )), so the new objective function value would be the previous one (c T B B 1 b) minus γα Thus, since α is positive, the objective function will be improved if γ is negative So, if we see a negative number in the objective function row, then we think (though we would be wrong in some cases, as we will see) that we can improve the objective function value by letting the non-basic variable corresponding to a negative reduced cost increase from to some positive value Of course, why not let it increase a whole bunch and really improve? The catch, of course, is that as we let the value of x j increase, the values of some of the x B variables may decrease, and we can t let any of them go negative More precisely, denote column j of the constraints part of the tableau by the vector u, and note that the equations now say or x B + αu = B 1 b, x B = B 1 b αu So, for any u i that is positive, increasing x j to α reduces the value of the ith basic variable by α times that positive amount Since we must maintain the x constraints of the LP, the amount that α can increase is probably restricted Precisely, if we denote B 1 b by r, then for each i such that u i >, we must have r i αu i, CS 45 Fall 217 Page 65

11 or α r i u i If we ever encounter the situation that none of the u i are positive, then the problem is unbounded it is possible to increase α as much as we wish, getting better and better objective function values, without violating any constraints So, in case you missed it, that s the simplex method: given a good set of basic variables, do row operations to transform the corresponding columns of A to an m by m identity matrix, and then examine the reduced costs If any of them are negative, pick one that is negative to enter, find the minimum ratio of right-hand-side elements to positive elements of its column, and increase the non-basic variable by that amount In so doing, the previously basic variable for that row will be forced to, and can now safely be considered non-basic, replaced by the newcomer When there are no negative reduced costs, we have found the optimal point If there are no positive elements in the column, the problem is unbounded Note that ManualSimplex (whose name now finally makes sense!) allows us to manually perform the simplex method (except it does most of the work automatically) Pressing the key m finds the row in the cursor column that has the minimum ratio r i /u i, and moves the cursor to that row Then we can press p to pivot do all the row operations needed to make a 1 at the cursor location and s in all other rows of the cursor column It would be trivial to automate the manual work, which is to find a negative reduced cost, move the cursor to that row, press m, press p, and repeat until an optimal point is found We don t do that in order to force the user to understand the algorithm better Embedded Exercise 39 Here is a typical tableau: z x 7 x 2 x 5 x 4 z x 1 x 2 x 3 x 4 x 5 x 6 x 7 rhs Following the previous discussion, write down what each equation says, assuming that x 3 is allowed to move from up to some positive number α, while the other non-basic variables stay at and the basic variables adjust accordingly Figure out the biggest value of α that will keep all the current basic variables nonnegative Repeat this work assuming that x 6 is allowed to move up to α CS 45 Fall 217 Page 66

12 Transforming a Linear Problem with Inequalities to an LP Note that the farm problem we solved earlier does not actually fit the form required for an instance of LP, because it has inequality constraints We can use a standard technique to convert inequalities into equalities, as follows For any expression E(x) involving the variables x 1, x 2,, x n, and any number r, the inequality E(x) r is equivalent to the equation and inequality E(x) + s = r and s The new variable s introduced here is known as a slack variable because it represents the amount by which E(x) is less than r Similarly, is equivalent to the equation and inequality E(x) r E(x) s = r and s The variable s here is known as a surplus variable Using these techniques, our original farm problem is equivalent to max 2x + 3y subject to x + y + s 1 = 12 8x + 15y + s 2 = 12 6x + 3y + s 3 = 18 x, y, s 1, s 2, s 3 Before converting an inequality to an equality, we should ensure that the right-hand side value is nonnegative if it is negative, we simply multiply both sides of the inequality by 1, which also flips the inequality, before adding in a slack variable or subtracting a surplus variable Minimizing Instead of Maximizing If we want to minimize the objective function instead of maximizing it, we simply maximize ( c) T x, which obviously gives the value of x that minimizes c T x As we do this, we need to remember that the actual objective function value is the opposite of the one we are getting CS 45 Fall 217 Page 67

13 Using Unrestricted Variables Sometimes a problem will involve variables that are unrestricted, meaning they are not restricted to be nonnegative Since keeping all the variables nonnegative is an essential part of the simplex method, this might seem impossible, but a simple idea allows us to incorporate such variables into an LP Suppose we have a variable x 3 that we want to be unrestricted We simply replace that variable by two new variables, typically named x + 3 and x 3 (or x1p and x1m in the program), and everywhere in the objective function and constraints that x 3 occurs, we replace it by x + 3 x 3 These two new variables are just like any other variables However, only one of them can be basic at any given time, because if we pivot in one of these columns, the other column will have 1 in the row where the other has the 1 in an identity column So, either they will both be zero, or one will be zero and the other will be positive If x 3 is basic and positive, say with a value of 7, then x 3 = x + 3 x 3 = 7 = 7 Similarly, if x+ 3 is basic, a positive value for x 3 can be achieved Getting a Good Initial Choice of Basic Columns Our farm example is misleading in the sense that there is a nice so-called slack variable for each constraint, so it is easy to find an initial choice of columns to be basic that is feasible that is, a choice that makes the values of the basic variables nonnegative Consider the general problem of finding an initial choice B of columns to be basic We want two things from these columns First, B must be non-singular, so that we can perform row operations to transform it to the identity matrix Second, B 1 b must be non-negative, because that vector holds the values of the basic variables Note that if we can find a choice of columns with these properties, then we will also have found a feasible point, namely the extreme point B 1 b obtained from B It turns out that the problem of determining whether there are any feasible points is just as hard, in an efficiency analysis sense, as solving any LP The technique we will use to find a good initial choice of basics is known as Phase 1 Then the rest of the simplex algorithm is known as Phase 2 Both phases use the same algorithm, namely moving from extreme point to extreme point, but Phase 1 uses a different objective function than the original one, and then Phase 2 proceeds with the actual objective function Once we have modeled the original problem in the Ax = b form, we first look for columns of A that are already columns of the identity If such a column has its 1 in row i, we check to see if b i is non-negative If it is, then we might as well use that column as basic This is done simply to save time The upcoming technique could be used without this shortcut The idea of Phase 1 is simple: for each row k that does not already have an obvious basic variable, we first multiply the row by 1, if necessary, to make sure that b k is non-negative, and then we make up a new variable, known as an artificial variable, say named a k, and CS 45 Fall 217 Page 68

14 simply add a k to the left hand side of the equation for that row This guarantees that the column for a k is a column of the identity, with a 1 in row k, and a positive right-hand side value, so a k is a good basic variable for row k After doing this as necessary until we have all m columns of the m by m identity matrix identified as basic columns, with all the right-hand side values non-negative, we have a very nice starting point for the simplex method Unfortunately, this problem is not our original problem, so all this appears to make little sense The trick is to use as the objective function the sum of the artificial variables, and to minimize this objective function Using this special Phase 1 objective function, we perform the simplex method until we have to stop, either because the problem is unbounded, or because we find an optimal extreme point Note that a Phase 1 problem can never be unbounded, because the artificial variables are restricted to be nonnegative, so the sum of all the artificial variables, which is the Phase 1 objective function, can t be below So, Phase 1 will always terminate with an optimal extreme point If the original problem has a feasible point, then the Phase 1 problem will have an optimal point with no artificial variables in the set of basic variables, with an objective function value of If the original problem is infeasible, then the Phase 1 problem will terminate with one or more artificial variables still basic If the Phase 1 simplex method produces an optimal value of the objective function that is positive, then we conclude that the original problem is infeasible Otherwise, we make sure that no artificial variables are still basic by perhaps pivoting some more Then, we replace the artificial objective function by the real one, and happily proceed with the simplex method When doing the Phase1-Phase 2 simplex method, be sure before starting either phase that you price out the objective function row It is clear from our theory that the upper-right corner is the current objective function value only if the row operations have been done to make zeros in the objective function row in the basic columns When we add in an artificial variable with a nice identity column, and a 1 in the objective function row, it is easy to forget that we still have to pivot on the 1 in the identity column in order to transform the 1 in the objective function row to a Then, once Phase 1 concludes, we swap in c T in the objective function row, and we again need to remember to pivot on the 1 s in the various basic columns to zero out those same columns in the objective function row Of course, for Phase 2 we either delete the artificial variable columns, or ignore them In particular, if a negative reduced cost appears in an artificial variable column, we must not let the artificial variable become basic With this Phase 1-Phase 2 technique we now have an actual algorithm that can solve any instance of LP CS 45 Fall 217 Page 69

15 Embedded Exercise 4 Apply all the preceding discussion to this LP and solve it by the simplex method: min 3x 1 4x 2 subject to 2x 1 4x 2 = 3 5x 1 + 3x 2 4 x 1 + x 2 5 x 1 CS 45 Fall 217 Page 7

16 Degeneracy and Cycling Note that we only did a taste of the theory of LP and the simplex method We proved, for example, that if x is a strict optimal point, with c T x > c T x for all x F, then x must be an extreme point This issue, of multiple feasible points having the same objective function value, muddles up the clarity of the simplex method When a basic variable ends up having a value of referred to as a degenerate case it is possible to do a pivot step of the simplex method without actually moving in space the set of basic variables changes, but we don t move to a new extreme point and we don t improve the objective function value It is possible in this situation for cycling to occur, where, depending on how the specific implementation of the simplex method chooses which non-basic variable with a negative reduced cost to become basic, and how ties are broken between rows with the same minimum ratio, the simplex method can cycle through the same pivot steps, never actually moving, forever It is easy to make choices, however, that prevent this from happening As a practical matter for us, note that if an artificial variable is basic, but with a value of, to keep things simple we should replace it in the set of basic variables by a non-artificial variable, even though the objective function stays at Efficiency of the Simplex Method and Related Issues The simplex method was invented in the late 194 s George Dantzig (who might have been the inspiration for Matt Damon s character in the movie Good Will Hunting) is generally credited with inventing it Shortly after its invention, researchers showed that it s worst-case efficiency category is exponential in the input size, which is not good, though better than ( ) Θ(n m) for the brute force method Nonetheless, the simplex method was used (and is still used) for tons of purposes in math, engineering, science, business, and economics It has been proved that its average case performance is polynomial in the input size This situation is analogous to the quick sort algorithm, which is quite popular even though its worst-case performance is in Θ(n 2 ), because its average performance is faster, in practical terms, than other guaranteed Θ(n log 2 n) sorting algorithms All the preceding discussion is about the efficiency of the algorithm A different question, which is in the field of computational complexity, is how hard is the problem LP? In the 197 s a group of Russian researchers developed the ellipsoid algorithm, which solves LP and has worst-case efficiency that is polynomial in the input size Before their result, people might have thought that LP was a difficult problem, like all the problem in NP-complete that we will discuss at the end of the semester when we briefly look at computational complexity, but it turns out that LP is only polynomial in difficulty CS 45 Fall 217 Page 71

17 However, the ellipsoid algorithm was not used in practice, because the simplex method was far more efficient on any problem that someone actually wanted to solve (the examples on which the simplex method took exponential time were carefully contrived to give it difficulties, not real world problems) Also, the simplex method was entrenched in the culture that is, many people made their livings using it In the mid 198 s, Narendra Karmarkar, a researcher at Bell Labs in the United States, published a new algorithm for LP, which became known as Karmarkar s algorithm He proved that this algorithm had polynomial worst-case efficiency, which was not a big deal, because the ellipsoid algorithm already showed that this was possible The big deal was that he, rather pompously, claimed that in practice his algorithm was faster than the simplex method Despite some early suspicion (people openly accused him of fine-tuning some critical parameters in his algorithm problem-by-problem to get better performance) and hostility, it was accepted that this algorithm was of practical importance Shortly thereafter, researchers discovered that Karmarkar has independently reinvented an algorithm that other researchers had invented in the 196 s, but had not marketed as aggressively as Karmarkar had pushed his algorithm Currently the algorithm is known as the interior point method Many people still use the simplex method CS 45 Fall 217 Page 72

18 Brief Theory of Linear Programming Since I have it written, and we re not printing the pre-reading materials, there s no cost to giving you this, in case you love mathematical stuff! There s no pressure to look at any of this theory Now we will take a brief look at some mathematical theory for the linear programming problem, intended to provide intuition for the simplex method Basically, we want to be able to view a general LP in the same way we handled the graphical picture in the farm example Definitions Define F = {x Ax = b and x } The set F is known as the feasible set, and we sometimes say x is feasible instead of x F A set S is convex if for every u and v in S, and for every λ [, 1], (1 λ)u + λv S Note that any point on the line passing through u and v has the form u + λ(v u) for some number λ Figure out what values of λ produce a point that lies between u and v Do some algebra to rewrite this quantity with u and v appearing just once Now the definition should make sense draw some 2D sets that are convex and some that are not convex We will refer to the sets of points on the line between two points in space as a segment (short for line segment ) With this terminology, a set is convex if for every pair of points in it, the segment connecting them lies entirely in the set Theorem F is convex Prove this theorem by using algebraic properties of the equations and inequalities that specify the feasible set Theorem Suppose that x is a solution to an instance of LP, and that x is a strict optimum point, meaning that for all x F, if x x then c T x < c T x Then x is an extreme point of F, which is defined to mean that x does not lie in the interior (ie, is not one of the two endpoints) of any segment that is a subset of F Proof: Suppose to the contrary that x is a strict maximum of an instance of LP, and that there are points u and v in F with u v and x in the interior of the segment from u to v So, for some λ (, 1), x = (1 λ)u + λv But then simple matrix algebra, and the fact that x is a strict maximum, shows that which is a contradiction c T x = c T ((1 λ)u + λv) = (1 λ)c T u + λc T v < (1 λ)c T x + λc T x = (1 λ + λ)c T x = c T x, Verify all the details in this proof CS 45 Fall 217 Page 73

19 Theorem Suppose that some x F has at least m+1 positive components Then x is not an extreme point of F, and therefore it is not a strict local maximum of the LP instance Proof: For notational convenience, assume that x has exactly m + 1 positive components, and that they are the first m + 1 The proof would be similar but harder to describe in the general case Denote column k of A by A k By basic matrix algebra, it is easy to see that the system of equations y 1 y A m+1 = has a non-zero solution In other words, there is a vector y such that y 1, y 2,, y m+1 are not all zero, y m+2,, y n are all zero, and Ay = Now, it is easy to see that for all small enough numbers µ, both positive and negative, the points x + µy are feasible Thus, x lies on an interval of positive length that is a subset of F, so x is not an extreme point Fill in the many missing details in this argument Theorem Suppose we pick m columns of A and denote them as basic, and want to solve Ax = b for the corresponding m components of x while setting the other n m components of x to Suppose that these columns row reduce to the identity, making the solution possible, and that x Then the point x is an extreme point of F, and hence might be a strict local maximum of the LP Proof: Again, for notational convenience, assume that the first m columns of A are the ones we are considering basic Partition A = [B N], where B is the non-singular m by m matrix composed of the first m columns of A, and N is the remaining n m columns When we do row operations to row reduce B to the identity, the matrix is transformed to and x = B 1 b [I B 1 N B 1 b ], Now, suppose that x is not extreme, implying that there are points u and v in F such that x = (1 λ)u + λv, for some λ (, 1), with u v It is easy to see that u and v must have their components equal to anywhere that x does, including all those below row m, and then it is easy to see, since the square, non-singular system of equations Bz = b has a unique solution, that we must in fact have u = v = x CS 45 Fall 217 Page 74