CSC 8301 Design & Analysis of Algorithms: Lower Bounds Professor Henry Carter Fall 2016
Recap Iterative improvement algorithms take a feasible solution and iteratively improve it until optimized Simplex method: linear equations Ford-Fulkerson: network flow Bipartite matching: matching cardinality or matching weight 2
How low can you go? Algorithmic power explored so far has been powerful Several implicit limitations have been encountered These limitations established through lower-bound proofs (big-ω) 3
Proving the lower bound Trivial limits Problem reduction Adversarial argument Information-theoretic 4
Trivial Limits A simple lower bound can be established based on the size of the input and output Every input element must be read Every output element must be written 5
Examples Enumerating permutations of n elements Evaluating a polynomial N x N matrix multiplication 6
Tight Bounds The tightness of the bound indicates how close the lower bound is to the actual known lower bound If a lower bound is known along with an algorithm with the same efficiency, the bound is "tight" Examples: Polynomial evaluation N x N matrix multiplication 7
Problem Reduction If we have a lower bound for problem Q, it can be applied to other problems Reducing Q to P implies that the worst-case lower bound for Q also applies to P Since the worst-case Q requires Ω(n) operations no matter how you try to solve it, reducing the problem to P and solving P also faces the same lower bound 8
Common problems for reduction Problem Lower Bound Tightness sorting Ω(n log n) yes Searching a sorted Ω(log n) yes array Element uniqueness Ω(n log n) yes n-digit integer multiplication n x n matrix multiplication Ω(n) Ω(n 2 ) unknown unknown 9
Euclidean Minimum Spanning Tree 10
(Matrix) Multiplication Integer multiplication: x y = (x + y)2 (x y) 2 4 x 2 = x x Matrix multiplication X = apple 0 A A T 0 Y = apple 0 B T B 0 11
Adversarial Arguments An adversarial player generates inputs during execution to incur maximum comparisons The adversary must remain consistent with previous choices Determine how many comparisons the adversary can force Example: merging two sorted arrays 12
Information-Theoretic Arguments Determine the minimum number of questions required to gather the necessary "information" Example: guessing a number between 1 and n Enumerate possible execution paths with decision trees The maximum and average heights of the tree correspond to the worst-case and average number of questions (comparisons) 13
Example tree: Insertion Sort 14
Sorting Lower Bound Lower bound on height: Stirling's formula: n! (2πn)1/2 (n/e) n Average height: 15
Example tree: Searching 16
Example tree: Searching 17
Searching Lower Bound Representation-dependent Ternary decision trees: Not tight Binary decision trees: 18
Practice: finding the median of three elements What is the information-theoretic lower bound using comparison-based algorithms? Draw a decision tree Is the bound illustrated by your decision tree tight? Can we get a better algorithm? 19
Recap Algorithmic power is broad but limited Lower bounds determine whether an algorithm can be improved by more than a constant factor Established through input size, problem reduction, adversarial generation, and information-theoretic decision trees 20
Quantifying Difficulty Is the problem solvable? Is the problem solvable in polynomial time? Foundational questions in computational complexity 21
Decision Problems To simplify analysis, complexity questions are frequently applied to decision problems Examples: Does the program halt? Can the graph be colored with no more than m colors? Many problems in computing can be reduced to a series of decision problems Find the minimum number of colors needed to color a graph 22
Decidability Is the problem solvable? Many decision problems cannot be solved with any algorithm Alan Turing's halting problem is undecidable 23
Halting Problem 24
Tractability Is the problem solvable in polynomial time? Proof: give a (deterministic) polynomial time algorithm Abstractly represented as all problems in the set P Contains many of the problems we've already examined 25
NP Problems Problems with no known polynomial-time solution Specifically, problems solvable using the following nondeterministic algorithm: Randomly generate a solution Deterministically check the correctness of the solution P is a subset of NP How? 26
Examples Hamiltonian circuit Traveling Salesman Knapsack problem Partition problem Bin packing problem Graph-coloring problem Integer linear programming 27
NP-Complete Representative problems in NP Require two characteristics: A belongs to class NP All problems in NP are polynomially reducible to A How do we prove these characteristics? 28
Practice A game of chess can be posed as the following decision problem: given a legal positioning of chess pieces and information about which side is to move, determine whether that side can win. Is this decision problem decidable? 29
Practice A certain problem can be solved using an algorithm whose running time is in O(n log n ). Is the problem tractable or not? Can we be certain? 30
Next Time... Levitin Chapter 11.3-11.4 Remember, you need to read it BEFORE you come to class! Homework: 11.1: 1, 2, 3ab 11.2: 1, 3a 31