Computational Complexity for Algebraists Ross Willard University of Waterloo, Canada January 20, 2009 Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 1 / 23
Outline First hour: Decision problems, Algorithms, and Complexity Classes Second hour: Nondeterminism, Reductions, and Complete problems Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 2 / 23
Three themes: problems, algorithms, efficiency A Decision Problem is A YES/NO question parametrized by one or more inputs. Inputs must: range over an infinite class. be finitistically described What we seek: An algorithm which correctly answers the question for all possible inputs. What we ask: Is there an algorithm? Given an algorithm, how efficient is it? Is there a better (more efficient) algorithm? Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 3 / 23
Directed Graph Reachability problem (PATH) INPUT: A finite directed graph G = (V, E) Two distinguished vertices v start, v end V. QUESTION: Does there exist in G a directed path from v start to v end? Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 4 / 23
An Algorithm for PATH v start = Answer: NO v end = Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 5 / 23
Efficiency of this algorithm How long does this algorithm take? I.e., how many steps as a function of the size of the input graph. I ll give three answers to this. Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 6 / 23
First answer Heuristics Only significant action is changing a vertex s color. Only changes possible are white red red green green blue. So if n = V, then the algorithm requires at most 3n vertex-color changes. I.e., O(n) steps Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 7 / 23
Second answer pseudo-code Simplifying assumptions: V = {0, 1,, n 1} E is encoded by the adjacency matrix M E = [e i,j ] where { 1 if (i, j) E, e i,j = 0 else. Auxiliary variables: i, j will range over {0, 1,, n 1}. For i < n, c i will record the color of vertex i. GreenVar will keep track of whether there are green-colored vertices. Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 8 / 23
Second answer pseudo-code Algorithm: Input n, M E, and start, end {0, 1,, n 1}. For i = 0 to n 1 set c i := white. Set c start = green. Set GreenVar := yes. MAIN LOOP: While GreenVar = yes do: For i = 0 to n 1; for j = 0 to n 1 if e i,j = 1 and c i = green and c j = white then set c j := red. For i = 0 to n 1 If c i = green then set c i := blue Set GreenVar := no For i = 0 to n 1 If c i = red then (set c i := green and set GreenVar := yes) If c end = blue then output YES; else output NO. n + 2 ops n loops n 2 cases O(n 3 ) steps if n = V Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 9 / 23
Third answer machine implementation Again assume V = {0, 1,, n 1}. Assume in addition that: v start = 0 and v end = 1. The adjacency matrix is presented as a binary string of length n 2. Implement the algorithm on a Turing machine. Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 10 / 23
Turing machine Input (ROM): 0 0 1 1 0 1 1 1 0 0 1 0 0 1 0 1 0 1 1 0 0 1 1 0 0 1 R/W Tape 1: a a c e c a a e c a e e a c a e e a c c c e c e c a a e a a c R/W Tape 2: R/W Tape 3: R/W Tape 4: 7 0 3 2 5 3 0 0 1 6 7 2 3 0 1 5 1 5 0 0 0 6 7 2 3 1 3 4 0 1 0 x o x x o o o o x x o x x o A C H X X OO 1 P L E A S E S E N D H E L P A R G G H H R/W Tape 5: Output bit: 0 0 1 1 0 1 1 1 0 0 1 0 0 1 0 1 0 1 1 0 0 1 3 state Tape char 1 st? In 1 2 3 4 5 1 c 1 x E Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 11 / 23
Implementing the algorithm for PATH Input: Tape 1: Tape 2: Tape 3: Tape 4: Tape 5: } {{ } [e i,j ] } {{ } c i }{{} n }{{} i }{{} j = 0 0 1 0 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 1 0 0 # 0 0 1 1 # 0 0 0 0 # 0 0 0 0 # 1 GreenVar # Main loop: For i, j = 0 to n 1 Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 12 / 23
Pseudo-code revisited Point: overhead needed to keep track of i, j, c i, c j. Thus: While GreenVar = yes do: For i = 0 to n 1; for j = 0 to n 1 if e i,j = 1 and c i = green and c j = white then set c j := red. n loops n 2 cases O(n log n) steps SUMMARY: on an input graph G = (V, E) with V = n, our algorithm decides the answer to PATH using: Heuristics Pseudo-code Turing machine O(n) color-change steps O(n 3 ) operations O(n 4 log n) steps (Time) O(n) memory cells (Space) Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 13 / 23
Turing machine complexity Let f : N N be given. Definition A decision problem D (with a specified encoding of its inputs) is: 1 in TIME(f ) if there exists a Turing machine which: Solves D on all inputs, and Runs in at most O(f (N)) steps on inputs of length N. 2 in SPACE(f ) if there exists a Turing machine which: Solves D on all inputs, and uses at most O(f (N)) memory cells (not including the input tape) on inputs of length N. Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 14 / 23
Complexity of PATH Recall that our Turing machine solves PATH on graphs with n vertices in Time: O(n 4 log n) steps Space: O(n) memory cells. Since length N of input = n 2 (when n = V ), this at least proves Theorem PATH TIME(N 2+ɛ ) SPACE( N). (This algorithm is essentially optimal w.r.t. TIME, but not w.r.t. SPACE.) Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 15 / 23
Another problem: Boolean Formula Value (FVAL) INPUT: A boolean formula ϕ in propositional variables x 1,, x k. A sequence c = (c 1,, c k ) {0, 1} k. QUESTION: Is ϕ(c) = 1? Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 16 / 23
An algorithm for FVAL ϕ = ((((x 2 x 4 ) ( (x 3 ))) ((x 1 x 4 ) (x 3 x 2 ))) ( (x 3 (x 1 x 3 )))), c = (1,0,1,1). 0 1 0 1 1 1 1 0 1 1 1 1 x 3 0 1 1 1 1 1 0 1 1 x 2 x 4 x 3 x 1 x 4 x 3 x 2 x 1 x 3 Seems to use TIME(N) and SPACE(N). But space can be re-used. In this example, 3 memory bits suffice. Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 17 / 23
Complexity of FVAL In general, a bottom-up computation, always computing a larger subtree first, can be organized to need only O(log ϕ ) intermediate values. A careful implementation on a Turing machine yields: Theorem (Nancy Lynch, 1977) FVAL TIME(N 2+ɛ ) SPACE(log N). Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 18 / 23
A 3rd problem: Polynomial satisfaction in S 26 (PolSat(S 26 )) INPUT: a word w(x) in variables x 1, x 2, and parameters from S 26. QUESTION: Does the equation w(x) = 1 have a solution in S 26? Naive algorithm: For each assign. x i a i S 26, Calculate w(a). Check if w(a) = 1. Overhead needed to enumerate cases: SPACE(N) 26 k cases, if k variables TIME(N 2+ɛ ) SPACE(log N) constant time, zero space Since k can be as much as N 1 δ, this algorithm in the worst case requires 26 N1 δ O(N 2+ɛ ) = 2 O(N1 δ) steps (though only O(N) space). Hence PolSat(S 26 ) TIME(2 O(N) ) SPACE(N) Question: is there a better algorithim? Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 19 / 23
A fourth problem: Graph 3-Colorability (3COL) INPUT: a finite graph G = (V, E). QUESTION: Is it possible to color the vertices red, green or blue, so that no two adjacent vertices have the same color? Equivalently: does there exist a homomorphism χ :? G K 3 Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 20 / 23
An algorithm for 3COL Naive algorithm: For each function χ : V K 3 : Test if χ works. 3 V = 2 O( N) cases O(N 2+ɛ ) time, O( N) space Theorem The naive algorithm proves: 3COL TIME(2 O( N) ) SPACE( N) Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 21 / 23
Some important complexity classes Definition 1 P = PTIME = k=1 TIME(Nk ) = TIME(N O(1) ). 2 PSPACE = k=1 SPACE(Nk ) = SPACE(N O(1) ). Problems known to be in P are said to be feasible or tractable. Definition 3 EXPTIME = k=1 TIME(2Nk ) = TIME(2 NO(1) ). 4 L = LOGSPACE = SPACE(log(N)). L P PSPACE PATH FVAL 3COL, PolSat(S 26 ) EXPTIME Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 22 / 23
After the break L P PSPACE EXPTIME PATH FVAL 3COL, PolSat(S 26 ) In the second hour I will: Introduce nondeterministic versions of these 4 classes. Discuss problems which are hardest for each class. Ross Willard (Waterloo) Complexity for Algebraists January 20, 2009 23 / 23