INTRO TO MATH PROGRAMMING MA/OR 504, FALL 2015 LECTURE 11

Transcription

1 INTRO TO MATH PROGRAMMING MA/OR 504, FALL 2015 LECTURE 11

2 A COMPLEXITY THEORY CRASH COURSE What is an algorithm? Precise definition requires a model of computation You might have heard some of these (won t define them now): Turing machine: an abstraction of early computers with a tape Random access machine: an abstraction of computers more similar to today s Quantum computer A purely mechanical (finite) Turing machine from 6000 LEGO bricks 2

3 ALGORITHMS, COMPLEXITY Algorithm: reads input, produces output (answer to a question). Early complexity theory: computability If we had a computer, could we compute everything? Curiously, no. E.g., optimization of polynomials over integers(!) Is this a practical concern? Yes, there are fundamental, and very basic-sounding problems that have no algorithmic solutions. One from integer programming: Input is a (multivariate) polynomial that is a sum of squares, so clearly nonnegative. Question: does it become zero anywhere on the integer grid? 3

4 ALGORITHMS, COMPLEXITY Algorithm: reads input, produces output (answer to a question). Early complexity theory: computability If we had a computer, could we compute everything? Curiously, no. E.g., optimization of polynomials over integers(!) But what if I had a quantum computer? Or a DNA computer? All the same. How about a sci-fi supercomputer we can t imagine? Most probably all the same; but we are crossing over to philosophy; see more here: 4

5 ALGORITHMS, COMPLEXITY Algorithm: reads input, produces output (answer to a question). Modern complexity theory: efficiency I have a computer. I figured out how to compute what I need. If I code it up and run it, will it be done before my deadline? Example: here s an LP I want to solve with the simplex method. How long is it going to take? Nobody knows precisely how long will it take, but lower/upper bounds on the order of magnitude would be useful. Obvious: larger computations take longer. Bounds on running times are expressed as functions of input size. 5

6 EFFICIENCY An algorithm is called efficient if the running time can be bounded from above by a polynomial of the input size. The space (memory) needed is also important. Examples: adding two integers; multiplying two integers. What is the size of the input? How many steps do the usual algorithms take? How about computing 2 nn, from the input nn? Why polynomial? 6

7 EFFICIENCY Why polynomial? 1.01 N 2 N 2 microsecs >10 16 years 0.02 msecs >10 29 years

8 CERTIFICATES, NP, CO-NP Verifying that the computed solution is correct maybe easier than finding that solution Example: LP; from the primal-dual optimal solution we can easily check in polynomial time that the solution is indeed optimal. More generally, the existence of solution is typically easier to certify than non-existence. Decision problems: output is 1 bit (YES/NO). A problem is in NP, if YES answers have a polynomial time verifiable certificate. A problem is in co-np, if NO answers have such a certificate. PP NNNN cccccccc, because It does NOT stand for non-polynomial 8

9 NP-HARD, NP-COMPLETE The million-dollar question (literally: Is P=NP? One approach to prove = is to give a polynomial time algorithm for every problem in NP That s a lot of work Fortunately, we have NP-complete problems Def: a problem is NP-hard if the existence of a polynomial time algorithm for that problem implies that there is a polynomial time algorithm for every problem in NP. Def: a problem is NP-complete if it is in NP, and it is NP-hard. Intuitively, it is the hardest problem in NP. Surprisingly, there is such a thing. Example: binary integer programming. (LP with the extra constraint that every variable is either 0 or 1.) 9

10 10 MY GAME IS HARDER THAN YOURS *Complexity is a fun topic to explore further (we won t). More complexity classes include stuff that s much harder than NPcomplete, randomized computation, and more. NP-complete PSPACE-complete

11 11 THE COMPLEXITY OF LINEAR OPTIMIZATION Linear optimization What is the size of a linear program? *What is the size of the optimal solution of a linear program?? Suppose all the coefficients in an LP are rational numbers, and that there is an optimal solution. There is always a rational optimal solution because xx = xx BB, xx NN = (BB 1 bb, 0) Bounding the numerators and denominators in xx BB shows that linear programming is in NNNN cccccccc. The ellipsoid method A polynomial time algorithm for LP Geometric intuition Feasibility vs optimization Generalization to convex programming