CSCI 59 Intro to Computational Complexity Overview of the Course John E. Savage Brown University January 2, 29 John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 / 26
Overview of Theoretical CS 2 Characterization of Computational Problems 3 Serial and Parallel Models of Computation 4 Complexity Classifications 5 Course Outline John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 2 / 26
Goals of the Course Introduce important models Problem statements: functions, languages Serial computational models Parallel computational models Develop methods to classify problems by time and space. E.g. L, NL, P, NP, PSPACE, NPSPACE, etc. Study approximation algorithms for NP-hard problems. Explore resource tradeoff results, such as trading space and time, for problems on real-world computational models. John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 3 / 26
Problem Representation Encodings - representing objects as strings Sets as bit vectors, tables as strings Languages - set of strings over an alphabet Regular {, } ; Context-Free { n n } Functions - mappings from domains to range f : {, } n {, }, f : {, } {, } Hard computational problems Traveling Salesperson Problem, Graph Coloring John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 4 / 26
Serial Models I Circuits Deterministic Finite State Machines (DFM) Nondeterministic FSM (NDFM) Random Access Machine (RAM) Model for the stored program computer John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 5 / 26
Logic Circuits Definition A circuit is a directed acyclic graph in which input vertices are associated with Boolean variables and non-input vertices carry the labels of Boolean functions such as and( ), or ( ), not ( ), and exor ( ). c j+ s j s j g j p j c j+ FA j c j v j u j c j v j u j John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 6 / 26
The DFSM Definition A deterministic finite-state machine (DFSM) is a seven-tuple M = (Σ, Ψ, Q, δ, λ, s, F ) where Σ and Ψ are the input and output alphabets, Q is the set of states of the machine, δ : Σ Q Q is its next-state function, λ : Q Ψ is its output function, s is its initial state, and F Q is a set of final states. A DFSM makes transitions. Before each transition it is in a state q. On input x Σ it makes a transition to a state q = δ(q, x) Q, and produces an output y = λ(q ) Ψ. q/ q/ Start John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 7 / 26
Computation by a DFSM Definition If a sequence of inputs x, x 2,..., x T causes a DFSM M to move from its initial state s to a state q F, the string x x 2 x T is accepted by the M. The language L(M) recognized by M is the set of accepted strings. A DFSM M computes a function f M defined by the mapping from inputs x, x 2,..., x T to outputs y, y 2,..., y T. q/ q/ Start John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 8 / 26
Computation by a DFSM Examples The language L recognized by the DFSM shown below contains all strings over {, } with an odd number of s. q / q / Start John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 9 / 26
Nondeterministic FSMs Definition A nondeterministic finite-state machine (NFSM) is a five-tuple M = (Σ, Q, δ, s, F ) where Σ is the input alphabet, Q is the set of states of the machine, δ : Σ Q 2 Q is its next-state function, s is its initial state, and F Q is a set of final states. The next-state function maps a state and an input to a set of states, thereby providing a set of choices for the next state. q / q / q 3/ Start q 2/ John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 / 26
NFSM Language Recognition Definition A string x, x 2,..., x T is accepted by an NFSM M if when started in state s, there are successor states that result in M arriving at a final state in F. A certificate to acceptance is the set of choices that lead to an accept state. The language recognized by an NFSM is the set of strings that it accepts. q / q / q 3/ Start q 2/ John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 / 26
Equivalence between FSMs and NFSMs Theorem The languages accepted by FSMs and NFSMs are equivalent. Proof. Given NFSM M = (Σ, Ψ, Q, δ, s, F ), create DFSM M whose states are subsets of Q (2 Q ) and start state contains s. If q = {q, q 2,..., q k } is state of M, q i Q, its successor under input a is set of states that reachable from some q i. q accepts if q F φ. Is this NFSM is equivalent to previous FSM? Q = {q }, Q = {q, q 2 }, Q 3 = {q, q 3 }. q/ q/ q3/ Start q2/ John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 2 / 26
Random Access Machine Implements fetch-execute cycle Uses direct and indirect addressing Arithmetic logic unit (ALU) performs addition, single-bit shifts. No multiplication because it introduces parallelism. Words increase at most one bit in length/per cycle CPU Decode ALU reg a reg b prog ctr Random-Access Memory b cmd out wrd in wrd addr m m 2 John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 3 / 26
Serial Models II Deterministic (Nondeterministic) Turing Machines (TMs) One or more tapes whose head(s) are controlled by a DFM (NDFM) The Church-Turing thesis asserts that any function that can be physically realized (i.e. computed ) can be computed by a Turing Machine. Universal TM (UTM) a TM that simulates other TMs Each TM T can be described by a table (string). Used to show that some problems cannot be solved by any TM. 2 m b Control Unit Tape Unit John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 4 / 26
Parallel Models I k-d mesh-based machines Cell (i, i 2,..., i k ) adjacent to (i, i 2,..., i k ) if i r i r = for only one r [... k]. Network-based machines Interconnection of individual FSMs. John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 5 / 26
Parallel Models II Parallel RAM (PRAM) Identical RAMs with individual IDs connected to common memory. In synch they read, compute, and store to memory. Hypercube-based machines Network of 2 k synchronous machines each identified with a binary k-tuple. Neighboring processor IDs differ in one bit. Processors swap data between neighbors, compute and repeat. John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 6 / 26
Languages, Grammars, Machines Languages and Grammars Regular: A u or A vc where A, B are non-terminals (NTs), u, v are terminals. Context-free: A BC or A u Recursively enumerable (r.e.): a b where a has at least one NT. Recursive: r.e. languages whose complements are r.e. Languages and Machines Regular FSMs Context-free Pushdown Automata Recursively enumerable Turing machines Recursive Halting Turing machines John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 7 / 26
Resource Limitations Circuit size and depth Size = number of gates; depth = length of longest path. Time number of computation steps Space no. cells in RAM, log 2 Q in FSM, tapes cells on TM I/O operations number of data movements Chip area for very large integrated circuits (VLSI) Wires have width and occupy area! John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 8 / 26
Relationships Between Models and Tradeoffs Relationships Between Models DFSMs equivalent to NDFSMs, DFMS subset of TMs T -step DFSM (NDFSM) can be simulated by a circuit (with free inputs). T (n)-step DTM (NDTM) on inputs of length n can be simulated by a circuit (all inputs are specified). Tradeoffs Between Resources Space vs time on RAM ST = Ω(n 2 ) for convolution. Area vs time for VLSI AT 2 = Ω(m 4 ) for m m matrix multiplication. John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 9 / 26
Problem Classification by Difficulty Computationally feasible Functions computable (languages recognizable) by some TM. Computationally infeasible Functions not computable (languages not recognizable) by any TM. Functions and languages classified by time and space Focus in theoretical CS primarily on language recognition P, NP, EXPTIME, PSPACE, NPSPACE, L, NL, NC John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 2 / 26
The Role of Reductions What is a reduction? Solve new problem by reducing (transforming) it to the solution of a previously solved problem. Definition Let L Σ and L 2 Σ 2. L is reducible to L 2 (L f L 2 ) if there exists f : Σ Σ 2 such that x L if and only if f (x) L 2. Definition L is recursively enumerable (r.e.) if it is TM-recognizable. John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 2 / 26
The Application of Reductions Assume that L f L 2 If L 2 is r.e. and f is TM-computable, then L is r.e. If L is not r.e. and f is TM-computable, then L 2 is not r.e. If L 2 is in P and f is poly-time, then L is in P. If L is not in P and f is poly-time, then L 2 is not in P. That is, L requires more than polynomial time for recognition. John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 22 / 26
Languages Complete for a Class Definition A language L 2 is complete for a class C if it is the hardest language to recognize in C. Definition A language L 2 is NP-complete if L 2 is in NP and for every other language L in NP there is a poly-time f : Σ Σ 2 such that x is in L if and only if f (x) is in L 2. TSP and SAT are NP-complete. John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 23 / 26
Approximation Algorithms Finding optimal solutions to NP-complete problems appears to be computationally infeasible. Seek poly-time approximation algorithms that find solutions whose performance (e.g. length of a tour for NP) is within a constant factor of optimal. Some problems do not have poly-time approximation algorithms unless P = NP. John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 24 / 26
Tentative Schedule Week Models of Computation Week 2 Turing Machines, P, NP Week 3 NP-completeness, reductions Week 4 EXPTIME, P-Space, NP-Space, L, NL Week 5 Diagonalization Week 6 Circuits Week 7 Formula Size Week 8 Space-Time Tradeoffs Week 9 VLSI Model of Computation Week Parallel Computation, P vs NC Week Probability, Randomized Computation Week 2 IP, PSPACE Week 3 PCP, Hardness of Approximation, PCP-Theorem Week 4 More PCP and Hardness Results. John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 25 / 26
Course Management Six assignments Perhaps a midsemester exam A final exam John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29 26 / 26