A Complexity Theory for Parallel Computation

Size: px
Start display at page:

Download "A Complexity Theory for Parallel Computation"

Transcription

1 A Complexity Theory for Parallel Computation The questions: What is the computing power of reasonable parallel computation models? Is it possible to make technology independent statements? Is it possible to relate parallel computations to sequential computations? What are the limits of parallel computations: are there algorithmic problems with efficient sequential computations which are hard to parallelize? The answers: we relate parallel time and sequential space and define the notion of P-completeness which allows to identify algorithmic problems which are (apparently) hard to parallelize. 1 / 48

2 Parallel Time The question: which algorithmic problems with input size n can be solved by a parallel computer in time T (n)? Observe that we bound the computation time, but not the number of processors. We assume an unlimited supply of processors. Initially only n processors are active. An active processor may activate at most one inactive processor per step. Which parallel computer? Any reasonable parallel computer, maybe the latest computer generation in 2100! Is there a connection between parallel time and sequential space? Intuition: it is possible to simulate a reasonable parallel computer with run time T (n) in sequential space poly(t (n)). Can we simulate a sequential computer with space complexity T (n) by a parallel machine with run time poly(t (n))? A Complexity Theory for Parallel Computation Space Complexity 2 / 48

3 Space Complexity I The architecture of an off-line Turing machine: an input tape which stores the input $w 1 w n $. The dollar symbol is used as end marker, w 1,..., w n belong to the input alphabet Σ. a work tape which is infinite to the left and to the right. The tape alphabet is binary. an output tape which is infinite to the right. Cells store letters from an out alphabet Γ. Modifying the tapes: The input tape has a read-only head which can move to the left or right neighbor neighbor of a cell. Initially the head visits the cell storing the leftmost letter of the input. The work tape has a read/write head with identical movement options. Initially all cells store the blank symbol B. The output tape has a write-only head. A Complexity Theory for Parallel Computation Space Complexity 3 / 48

4 Space Complexity II The computation of an off-line Turing machine M: The Turing machine computes by reading its input, modifying its work tape and printing onto its output tape. If the machine stops with output 1, then we say that the machine accepts the input; the machine rejects the input, if it writes a 0. For Turing machine M and input alphabet Σ, L(M) = {w Σ M accepts w} is the language accepted by M. The space complexity of M: space M (w) is the number of different cells of the work tape which are visited when computing on w. spacem (n) := max w Σ n space M (w) is the worst-case space complexity of M on inputs of length n. A Complexity Theory for Parallel Computation Space Complexity 4 / 48

5 Space Complexity III Let s : N N be a function. Then DSPACE(s) = {L {0, 1} L = L(M) for a Turing machine M with space M (n) = O(s(n)) } is the complexity class of all languages computable in (deterministic) space s. Important space classes: DSPACE(0) is the class of regular languages. DL = DSPACE(log 2 n): deterministic logarithmic space is required to remember an input position. PSPACE = k N DSPACE(nk ) is an extremely powerful complexity class containing P, N P, polynomial time quantum computations... A Complexity Theory for Parallel Computation Space Complexity 5 / 48

6 How Robust Are Space Complexity Classes? Space classes do not change, if we replace Turing machines by reasonable deterministic machine models. What about nondeterministic machines? Define nspace M (w) as the maximal number of cells of the work tape, which are visited during an accepting computation on input w. nspace M (n) := max w Σ n L(M) nspace M (w) is the nondeterministic space of M for input size n. Finally: NSPACE(s) := {L {0, 1} L = L(M) for a nondeterministic Turing }. machine with nspace M (n) = O(s(n)) Theorem of Savitch: NSPACE(s) DSPACE(s 2 ). Important space class: NL = NSPACE(log 2 n), all languages recognizable in nondeterministic logarithmic space. A Complexity Theory for Parallel Computation Space Complexity 6 / 48

7 The Computation Graph of A Nondeterministic TM Let M be a nondeterministic off-line Turing machine with space complexity at most s. Let w be an input of M. A configuration consists of the contents of the work tape, the current state and positions of the input and work tape heads. The computation graph G M (w) of M on input w: The configurations on input w are the nodes of GM (w). An edge (k1, k 2 ) from configuration k 1 to configuration k 2 belongs to G M (w) iff configuration k 2 is a successor of k 1. An additional node yes is added and, for any accepting configuration k, edges (k, yes) are inserted. M accepts input w iff there is a path in G M (w) from the initial configuration k 0 to node yes. If M uses space s(n), then M has at most 2 O(s(n)) O(1) (n + 2) s(n) = n 2 O(s(n)) configurations. A Complexity Theory for Parallel Computation Space Complexity 7 / 48

8 N C and Logarithmic Space - DL NL - and any language in NL can be accepted in time O(log 2 2 n) with a polynomial number of processors. DL NL: obvious. Let L be an arbitrary language in NL. Let L = L(M) for a nondeterministic Turing machine using logarithmic space. G M (w) has N = poly( w ) nodes and can be constructed in logarithmic time with a polynomial number of processors: reserve one processor for any pair of configurations to check, whether the pair corresponds to a transition. M accepts w iff there is a path in GM (w) from the initial configuration k 0 to the unique accepting configuration yes. We need a real fast solution of the transitive closure problem: O(log 2 2 N) time with poly(n) processors will do. A Complexity Theory for Parallel Computation Logarithmic Space 8 / 48

9 The Transitive Closure Problem Revisited Determine the transitive closure of a graph G with N nodes. Use circuits. We have parallelized Warshall s algorithm, but its speed is insufficient. Instead we use boolean matrix multiplication. Let A be the adjacency matrix of G. Set all diagonal entries to one (i.e., insert self loops) and call the new matrix B. B[i, j] = 1 iff there is a path of length one from i to j. And inductively, B r+1 [i, j] = N k=1 Br [i, k] B[k, j]: B r+1 [i, j] = 1 iff there is a path of length r from i to k and an edge from k to j. There is a path from i to j iff B N 1 [i, j] = 1. Compute B, B 2,..., B 2j, B 2j+1,..., B 2t for t = log 2 N. t = log 2 N matrix products. Per matrix product depth O(log 2 N) and size O(N 3 ). All in all: depth O(log 2 2 N) and size O(N3 log 2 N). A Complexity Theory for Parallel Computation Logarithmic Space 9 / 48

10 Uniform Circuit Families We consider {,, }-circuits as a parallel computing model. - Depth corresponds to parallel time. - Is there a connection to sequential space? To apply circuits to inputs of various input length, we have to work with families (S n n N) of circuits S n for input size n. We should require that the description of the family is only moderately complex. We say that the circuit family is uniform if, given input 1 n, its circuit S n can be described by a deterministic off-line Turing machine M with space complexity O(log 2 size(s n )). M has to eventually print all edges of S n onto its output tape as well as all nodes with their assigned operation, resp. with their assigned input bit. A Complexity Theory for Parallel Computation Parallel Time and Sequential Space 10 / 48

11 Parallel Time and Sequential Space I Assume that s(n) = Ω(log 2 n). A nondeterministic off-line Turing machine M with space complexity s(n) can be simulated by a uniform circuit family (S n n N) in time O(s 2 (n)). Input w belongs to L(M) iff there is a path in G M (w) from the initial configuration k 0 to the accepting configuration yes. Let N be the number of nodes of G M (w). Compute the transitive closure of GM (w) in depth O(log 2 2 N). We are required to build a uniform circuit family. Configurations have length O(log 2 n + s(n)) and space O(log 2 (n) + s(n)) is sufficient to output all valid transitions between configurations on input 1 n. We are done, since N = 2 O(s(n)) and log 2 2 N = s2 (n). A Complexity Theory for Parallel Computation Parallel Time and Sequential Space 11 / 48

12 Parallel Time and Sequential Space II A uniform circuit family (S n n N) of depth s(n) can be simulated by a deterministic off-line Turing machine M within space O(s 2 (n)). How does M work? Set n = w. Since S n has depth s(n), the circuit has at most 2 s(n)+1 1 nodes. Evaluate Sn via depth-first search with a stack of height O(s(n)). Since the stack holds nodes from the circuit, all we need is space complexity O(s 2 (n)). Indeed, already space O(s(n)) suffices. Why? We have shown the Theorem of Savitch: A nondeterministic Turing machine with space s(n) can be simulated by a uniform circuit family of depth O(s 2 (n)). The uniform circuit family can be simulated by a deterministic Turing machine with space O(s 2 (n)). A Complexity Theory for Parallel Computation Parallel Time and Sequential Space 12 / 48

13 The Parallel Computation Thesis We have found a quadratic relationship between the depth of circuits and the space complexity of Turing machines. The depth of circuit families and the computing time of PRAMs is also polynomially related. The following conjecture (parallel computation thesis) is therefore natural: parallel time for any reasonable parallel computing model is polynomially related to the space complexity for sequential computations. The thesis is mainly of theoretical interest, since the size of the parallel computing model is not restricted. However, when trying to design a parallel algorithm, first check if low-space algorithms exist. If not, then there may be only little speedup possible. A Complexity Theory for Parallel Computation Parallel Time and Sequential Space 13 / 48

14 The Class N C N C is the class of all problems with parallel algorithms running in time poly(log 2 n) with a polynomial number of processors. N C is a subset of P: each parallel step can be simulated in polynomial time. As a consequence N C is the class of problems with efficient sequential algorithms and drastic parallel speedups. The Class N C 14 / 48

15 Which Algorithmic Problems Belong To N C? Many important problems of linear algebra: the matrix-vector product, the matrix-matrix product, computing the determinant and solving linear systems, and the Fast Fourier Transform. Many dynamic programming problems such as: the transitive closure problem, the all-pairs-shortest path problem, pairwise alignment and the RNA secondary structure prediction. computing connected components and minimum spanning trees. Sorting. All context-free languages (and hence all regular languages). P-Completeness 15 / 48

16 Which Problems Probably Do Not Belong To N C? Which languages in P do not admit extreme speedups? Important observation: there are hardest languages L P: if L has real fast parallel algorithms with polynomially many processors, then all problems in P have fast algorithms with polynomially many processors. Hence, in all likelihood a hardest language has no extreme speedups. The roadmap: Define reductions to compare languages with respect to their potential speedup. Show that many important hardest languages exist. P-Completeness 16 / 48

17 Reductions L 1 is reducible to L 2 (L 1 par L 2 ), iff there is a translation T with w L 1 T (w) L 2. T has to be computable in poly-logarithmic time with a polynomial number of processors. Assume L 1 par L 2 and that L 2 belongs to N C. The reduction should compare languages with respect to achievable speedups. Does L 1 also belong to N C? run the translation T on input w to compute T (w) in poly-logarithmic time with a polynomial number of processors. Since L 2 N C, determine whether T (w) belongs to L 2 in poly-logarithmic time with a polynomial number of processors. Accept w for language L1 iff T (w) L 2. And L 1 also belongs to N C. P-Completeness Parallel Reductions 17 / 48

18 P-complete Problems: The Concept - A language K is P-hard iff L par K for all languages L P. - K is P-complete iff K is P-hard and K belongs to P. What happens, if a P-hard language K belongs to N C? We show that all languages in P have extreme speedups. Let L P be arbitrary. L par K, since K is P-hard. But then L N C, as we have just seen, since K belongs to N C. If some P-hard language belongs to N C, then P = N C and all problems in P have extreme speedups. P-Completeness Parallel Reductions 18 / 48

19 How To Show P-Completeness? - If L 1 par L 2 and L 2 par L 3, then L 1 par L 3. - If K is P-hard and K par L, then L is P-hard. Why is the reduction transitive? If L1 par L 2 holds because of translation T 1 and if L2 par L 3 holds because of translation T 2, then L1 par L 3 holds because of translation T 2 T 1. Assume that K is P-hard and that K par L holds. Let M P be arbitrary. Then M par K, since K is P-hard. Moreover K par L holds. And M par L follows by transitivity. Hence L is P-hard. P-Completeness Parallel Reductions 19 / 48

20 The Circuit Value Problem Let S be a circuit with a single output gate. Determine whether S accepts an input w. I.e., CVP = {(S, w) the circuit S accepts w }. CVP is an easy problem for sequential computation: evaluate a circuit with, say, depth-first search. Hence CVP P. However a parallel evaluation in poly-logarithmic time seems hard for circuits of large depth. Show that CVP is P-complete. P-Completeness The Circuit Value Problem 20 / 48

21 Proof Outline Show L par CVP for an arbitrary language L P. What do we know about L? There is a Turing machine M with a single tape which computes in time t(n) = O(n k ) and w L M accepts w. The goal: simulate M by a circuit family (S n n N) of polynomial size such that Set T (w) = (S w, w). Then M accepts w S w accepts w. w L M accepts w S w accepts w (S w, w) CVP. How to choose S w? P-Completeness The Circuit Value Problem 21 / 48

22 The Two-Dimensional Structure of S w Assume that w is input for M and that w has length n. M runs in time t(n) and hence it visits only the cells with addresses t(n), t(n) + 1,, 1, 0, +1,, t(n). Here we assume that letter wi is written on the cell with address i and that the read/write head of M initially sits on the cell with address 0. We simulate M on input w with a circuit S n which is built like a two-dimensional mesh. The ith row of Sn reflects the configuration of M on input w at time i and its nodes correspond to the tape cells, remember the current state of M and indicate the current position of the read/write head. The circuit has to compute the configuration at time i + 1 from the configuration at time i. P-Completeness The Circuit Value Problem 22 / 48

23 Updating Configurations We work with small subcircuits S i,j which are responsible for cell j of the tape at time i. Subcircuit S i+1,j asks subcircuit S i,j if the head is visiting cell j. If the answer is negative, then S i+1,j assigns the letter remembered by S i,j to cell j. If the answer is positive: S i,j also stores the current state which it transmits together with the current contents of cell j to S i+1,j. S i+1,j determines the new contents of cell j, the new state and the neighbor which will be scanned at time i + 1. All subcircuits S i,j are identical except for the subcircuits S 0,j, since cell j stores w j for 1 j n and the blank symbol otherwise. A binary tree on top of row t(n) has to be added to check whether the final state is accepting. S n can be constructed in logarithmic time with polynomially many processors. P-Completeness The Circuit Value Problem 23 / 48

24 Difficult Variants of CVP In M-CVP we assume that S is a monotone circuit and ask whether S accepts input w. A circuit is monotone, if it does not have negations. M 2 -CVP is defined as M-CVP, except that the circuit is required to have fan-out at most two. In NOR-CVP the circuit is built from NOR-gates only. Remember that NOR(u, v) = (u v). We show that all variants are P-complete. All three problems belong to P. CVP par M-CVP, M-CVP par M 2 -CVP and CVP par NOR-CVP. P-Completeness The Circuit Value Problem 24 / 48

25 M-CVP: Pushing Negations to the Sources I S is a {,, }-circuit. Is there an equivalent circuit S which uses negations only for input bits? Push negations to the inputs using the De Morgan rules (x y) x y and (x y) x y. However there is a problem: The result of a gate may be needed in its negated as well as in its unnegated form. The equivalent circuit S should provide both results. S will at most double in size. Why? P-Completeness The Circuit Value Problem 25 / 48

26 M-CVP: Pushing Negations to the Sources II The circuit S has two gates u pos and u neg for any gate u of S. If u is an input bit x i, then u pos = x i and u neg = x i. u pos keeps the functionality of u, u neg interchanges and. For any edge (u, v) of S, the equivalent circuit S has two edges, namely (u pos, v pos ) and (u neg, v neg ). x 1 x 2 x 3 x 1 x 2 x 3 x 3 x 2 x 1 P-Completeness The Circuit Value Problem 26 / 48

27 M-CVP is P-Complete For any circuit S there is an equivalent circuit S with negations only for inputs. S can be constructed in logarithmic time from S. The size of S at most doubles in comparison to S. Let (S, w) be an input for CVP. Construct the circuit S, remove all negations from S and call the resulting monotone circuit S m. We have to modify the input for Sm, since we removed negations: The input for a not-negated input gate is left unchanged, whereas the input for a negated input gate is flipped. Call the new input w m. Define the translation T (S, w) = (Sm, w m ). Then (S, w) CVP S accepts w S m accepts w m (S m, w m ) M-CVP. We have verified the reduction CVP par M-CVP, since T can be computed in logarithmic time. P-Completeness The Circuit Value Problem 27 / 48

28 M 2 -CVP and NOR-CVP are P-Complete The construction of the reduction M-CVP par M 2 -CVP is left as an exercise. To show: CVP par NOR-CVP. NOR is a basis, since u NOR(u, u) u v NOR( u, v) and u v NOR(NOR(u, v), NOR(u, v)). If (S, w) is an input of CVP: replace all gates by small NOR-circuits to obtain an equivalent NOR-circuit S. The translation (S, w) (S, w) establishes the wanted reduction to NOR-CVP. P-Completeness The Circuit Value Problem 28 / 48

29 Linear Programming Minimize a linear objective function c T x = n i=1 c ix i over the real numbers subject to linear constraints n j=1 A[i, j] x j b i for j = 1,..., m and to non-negativity constraints x 1 0,..., x n 0. In shorthand: min c T x s.t. A x b and x 0, A is the m n constraint matrix with A = (A[i, j]) 1 i m,1 j n, c = (c 1,..., c n ) is the vector of coefficients of the objective function and b = (b 1,..., b m ) is the vector of right hand sides. P-Completeness Linear Programming 29 / 48

30 Linear Programming: What Is Known? Efficient sequential solutions exist, namely the ellipsoid method or interior point methods. May be the most powerful optimization method with efficient algorithms. For instance the bipartite matching problem for the graph G: max e E y e s.t. A G y 1 and y 0. The incidence matrix AG of G has a row for every node u V 0 V 1 and a column for every edge e E: A G [u, e] = 1 iff node u is an endpoint of edge e and A G [u, e] = 0 otherwise. Although fractional solutions are allowed, all vertices of the polytope are integral. We show that the general linear programming problem is P-complete. However fast parallel algorithms exist, if all parameters (A, b and c) are nonnegative. P-Completeness Linear Programming 30 / 48

31 Linear Programming As A Decision Problem The Linear Inequalities problem (LIP): We are given an m n matrix A Z m n and a vector b Z m. The pair (A, b) belongs to LIP iff the system A x b of linear inequalities has a solution. The Linear Programming Problem (LPP) We receive the same input as in LIP, but obtain additionally a vector c Z n and a rational number α. The quadruple (A, b, c, α) belongs to LPP iff there is a vector x that solves the linear system A x b and also satisfies c T x α. LPP is the language version of the Linear Programming Problem: minimize the linear function c T x subject to the linear constraints A x b. P-Completeness Linear Programming 31 / 48

32 Linear Inequalities are P-Complete Show the reduction M-CVP par LIP. Let (S, w) be an input of M-CVP. Assign linear inequalities to any node c of the monotone circuit S. c is a source storing the ith input bit: use the inequalities x c 0, x c 0, if w i = 0 and x c 1, x c 1, if w i = 1. c a b is an AND-gate: use the inequalities x a x c 0, x b x c 0, x c x a x b 1, x c 0. c a b is an OR-gate: now choose x c x a 0, x c x b 0, x a + x b x c 0, x c 1. Show by induction on the depth of a gate c of S: x c = the value of gate c for input w. Finally, if t is the output gate of the circuit: add the inequality x t 1 to the linear system and (S, w) CVP S accepts w the linear system is solvable. P-Completeness Linear Programming 32 / 48

33 Linear Programming is P-Complete Show the reduction LIP par LPP. Choose the translation (A, b) (A, b, 0, 0). The existence of a solution x with A x b and 0 T x 0 is equivalent to the existence of a solution of A x b. Good parallel algorithms exist for positive linear programs: minimize c T x such that A x b and x 0. The constraint matrix A, as well as the right hand side b and the coefficient vector c of the objective function are non-negative. P-Completeness Linear Programming 33 / 48

34 Network Flow An input for the flow problem consists of - a directed graph G = (V, E), - two distinguished nodes, the source s V and the sink t V - and a capacity function c : E R. A function f : E R is a flow, if 0 f (e) c(e) for all edges e E and if flow conservation f (u, v) = f (v, u) u V,(u,v) E u V,(v,u) E holds for any node v V \ {s, t}: the amount of flow entering v equals the amount of flow leaving v. Flow conservation is not required for the source or the sink. A flow f is maximal, if f maximizes f = f (s, u) f (u, s), u V,(s,u) E u V,(u,s) E i.e., if the net flow pumped out of s is maximal. P-Completeness Network Flow 34 / 48

35 The Flow Problem The integrality property holds: if all capacities are integral, then there is a maximal flow which is integral. As a consequence, the bipartite matching problem determine a largest collection of node-disjoint edges for a given bipartite graph B = (V 0 V 1, E) is a special case of the flow problem: Add a source s and a sink t to B. Connect s to all nodes in V0 and all nodes in V 1 to t by edges of capacity one. All edges of B receive capacity one. The Flow Problem FP An input (G, s, t, c) belongs to FP iff all capacities are integral and the maximal flow is odd. P-Completeness Network Flow 35 / 48

36 M 2 -CVP par FP Let (S, w) be an input for M 2 -CVP and let G = (V, E) be the directed graph of the monotone circuit S. Assumptions on S: the output gate tg of S is an OR-gate: if not, then replace S by two copies of S which are fed into an OR-gate. each input node of G has fan-out one: otherwise insert additional copies. All other nodes have fanout at most two. The graph G for FP: add two new nodes s and t to G. The source s is connected to all input gates of G and the output gate t G of S is connected to the sink t. Finally add edges from OR-gates to s as well as edges from AND-gates to t. fanout(z) counts the number of edges of G leaving z. However set fanout(t G ) = 1 to account for the edge (t g, t). P-Completeness Network Flow 36 / 48

37 Defining Capacities - Assume that G has exactly n nodes. - Determine a topological sort nr : V {1,..., n} of G. If z is an input { node for variable x i, then 2 n nr(z) w c(s, z) = i = 1, 0 otherwise. If z is an interior node with immediate predecessors x and y: then c(x, z) = 2 n nr(x) and c(y, z) = 2 n nr(y). If z is an AND-gate: c(z, t) = 2 n nr(x) + 2 n nr(y) fanout(z) 2 n nr(z), If z is an OR-gate: c(z, s) = 2 n nr(x) + 2 n nr(y) fanout(z) 2 n nr(z). Set c(t g, t) = 1. Observe c(tg, s) = 2 n nr(x) + 2 n nr(y) 1. If tg receives the maximal possible flow 2 n nr(x) + 2 n nr(y), then the edge (t G, t) can transport the remaining flow of one to t. All other nodes have links to t of even capacity. P-Completeness Network Flow 37 / 48

38 Capacities Of An AND-Gate fanout(v) 2 n nr(v) t sink 2 n nr(u) + 2 n nr(w) fanout(v) 2 n nr(v) v 2 n nr(u) 2 n nr(w) u w P-Completeness Network Flow 38 / 48

39 The Maximal Flow f Claim: The translation (S, w) (G, s, t, c) establishes a reduction from M 2 -CVP to FP. Define a flow f : If z is an input gate: set f (s, z) = c(s, z). A flow of 2 n nr(z) is pumped into z whenever the input bit of gate z is one and a zero flow otherwise. There is one edge leaving z which has to be filled to its capacity 2 n nr(z). If z is an interior gate which evaluates to one: fill its fan-out(z) many leaving G-edges to capacity and push the excess flow to s respectively to t. If t G fires, then push flow one along edge (t G, t). If z is an interior gate which evaluates to zero: push zero flow across its leaving G-edges and any excess flow to s respectively to t. If t G does not fire, then push flow zero along edge (t G, t). P-Completeness Network Flow 39 / 48

40 The Cut Partition the nodes of G into the sets V 1 = {s} {z G the gate z evaluates to one } V 0 = {t} {z G the gate z evaluates to zero }. all V 1 V 0 edges of G are filled to capacity: G-edges from a node in V1 to a node in V 0 are filled to capacity by definition of f. Edges from AND-gates in V1 to t are also filled to capacity, whereas edges from OR-gates in V1 to s stay in V 1. The edge (t G, t) is also filled to capacity, provided t G fires. All V 0 V 1 edges of G carry flow zero: G-edges from a node in V0 to a node in V 1 carry flow zero. Edges from AND-gates in V0 to t stay within V 0. An OR-gate in V 0 receives zero flow along its two G-edges and hence an OR-gate in V 0 has no flow to distribute. The edge (tg, t) stays inside V 0, if t G V 0, P-Completeness Network Flow 40 / 48

41 Proof of Optimality f = c(x, y). x V 1,y V 0,(x,y) E g x V 1,y V 0,(x,y) E c(x, y) for any flow g, since s V 1 and t V 0. f is a maximal flow. As a consequence: if t G fires, then f is odd, since all edges carry even flow except for the edge (t G, t) which carries flow 1. If t G does not fire, then f is even, since all edges carry even flow. P-Completeness Network Flow 41 / 48

42 FP is P-Complete (S, w) M 2 -CVP S is a monotone circuit with fanout two and S accepts w the maximal flow in (G, s, t, c) is odd (G, s, t, c) FP. The translation (S, w) (G, s, t, c) establishes the reduction M 2 -CVP par FP. Is it necessary to use capacities of exponentional size? Yes. FP with capacities of polynomial size can be solved with a randomized N C-algorithm. P-Completeness Network Flow 42 / 48

43 Lexicographically First Maximal Independent Set Determine a maximal independent set for an undirected graph G = ({1,..., n}, E): I :=. for v = 1 to n do If (v is not connected with a node from I) then I = I {v}. I is a maximal independent set: any node v belongs either to I or is connected with an edge to a node in I. In the lexicographically first maximal independent set problem (LFMIS) we are to determine if a node v belongs to the independent set I computed by the for-loop. LFMIS is trivial sequentially, since we only have to run the for-loop. We show that LFMIS is P-complete and hence parallelizing for-loops is non-trivial. P-Completeness LFMIS 43 / 48

44 The All-Pairs-Longest-Path Problem Solve the all-pairs-longest-path problem for a directed acyclic graph with n nodes in time O(log 2 2 n) with n3 processors. Reduce the all-pairs-longest-path problem to matrix multiplication: Define a non-standard matrix multiplication for n n matrices X Y [u, v] = max X[u, w] + Y [w, v]. w V max replaces addition and addition replaces multiplication. Let A be the adjacency matrix of G and assume that the diagonal of A is zero, i.e., G has no self-loops. Then A k [u, v] = max A[u, w 1 ] + A[w 1, w 2 ] + + A[w k 1, v]. w 1,...,w k 1 Since G is acyclic, no node w i is traversed twice and A n 1 [u, v] is the length of a longest path from u to v. P-Completeness LFMIS 44 / 48

45 Topological Sort For the all-pairs-longest path problem: To compute A n 1 : compute the matrix powers A 2,..., A 2i, A 2i+1,..., A 2t for t = log 2 n. The all-pairs-longest path problem can be solved in time O(log 2 2 n) with n 3 processors. The topological sort of a directed acyclic graph G is a bijection π : V {1,... n} with π(u) < π(v) for any edge (u, v). Let length(v) be the length of a longest path ending in v. Replace each node v by the pair (length(v), v) and sort all pairs. All in all, time O(log 2 2 n) with n3 processors is sufficient to perform a topological sort. P-Completeness LFMIS 45 / 48

46 LFMIS is P-Complete Construct the reduction NOR-CVP par LFMIS. Let (S, w) be an input for NOR-CVP and let G be the directed graph for the NOR-circuit S. Important observation: all gates with value 1 form an independent set. Why? A NOR-gate NOR(u, v) fires iff u = v = 0. The idea: modify the graph G i.e., rename nodes, to obtain a graph G such that the for-loop picks all nodes with value 1. Then ask whether the output gate t of S belongs to I. Done, since (S, w) NOR-CVP t fires t I (G, t) LFMIS, provided the translation (S, w) (G, t) can be computed in poly-logarithmic time. P-Completeness LFMIS 46 / 48

47 The Translation (S, w) (G, t) Determine a topological sort for the graph G of S. To determine the translation: replace the name of a node by its rank within the topological sort. Thus the smallest number is given to a source which may however evaluate to zero. But the node with smallest number will belong to the independent set! Invent a new node 0, connect 0 with all sources which evaluate to 0 and disregard edge direction. Let G be the new (undirected) graph. What does the for-loop do initially: The new node 0 is included into I and automatically all sources with value 0 will be disregarded. Does the reduction work? P-Completeness LFMIS 47 / 48

48 Proof of Correctness The for-loop determines the independent set I = 0 {z z evaluates to one }. We show the claim by induction on the depth of a node. The claim is correct, if z = 0 or if z is a previous source of G. If z = NOR(u, v) and z fires: u = v = 0, since z = 1. By induction hypothesis u I and v I. z is included into I. If z = NOR(u, v) and z does not fire: u = 1 or v = 1 since z = 0. By induction hypothesis u I or v I and hence z will be left out. P-Completeness LFMIS 48 / 48

Lecture 5: The Landscape of Complexity Classes

Lecture 5: The Landscape of Complexity Classes IAS/PCMI Summer Session 2000 Clay Mathematics Undergraduate Program Basic Course on Computational Complexity Lecture 5: The Landscape of Complexity Classes David Mix Barrington and Alexis Maciel July 21,

More information

U.C. Berkeley CS278: Computational Complexity Professor Luca Trevisan 9/6/2004. Notes for Lecture 3

U.C. Berkeley CS278: Computational Complexity Professor Luca Trevisan 9/6/2004. Notes for Lecture 3 U.C. Berkeley CS278: Computational Complexity Handout N3 Professor Luca Trevisan 9/6/2004 Notes for Lecture 3 Revised 10/6/04 1 Space-Bounded Complexity Classes A machine solves a problem using space s(

More information

INAPPROX APPROX PTAS. FPTAS Knapsack P

INAPPROX APPROX PTAS. FPTAS Knapsack P CMPSCI 61: Recall From Last Time Lecture 22 Clique TSP INAPPROX exists P approx alg for no ε < 1 VertexCover MAX SAT APPROX TSP some but not all ε< 1 PTAS all ε < 1 ETSP FPTAS Knapsack P poly in n, 1/ε

More information

Notes on Space-Bounded Complexity

Notes on Space-Bounded Complexity U.C. Berkeley CS172: Automata, Computability and Complexity Handout 6 Professor Luca Trevisan 4/13/2004 Notes on Space-Bounded Complexity These are notes for CS278, Computational Complexity, scribed by

More information

Advanced topic: Space complexity

Advanced topic: Space complexity Advanced topic: Space complexity CSCI 3130 Formal Languages and Automata Theory Siu On CHAN Chinese University of Hong Kong Fall 2016 1/28 Review: time complexity We have looked at how long it takes to

More information

Definition: Alternating time and space Game Semantics: State of machine determines who

Definition: Alternating time and space Game Semantics: State of machine determines who CMPSCI 601: Recall From Last Time Lecture Definition: Alternating time and space Game Semantics: State of machine determines who controls, White wants it to accept, Black wants it to reject. White wins

More information

Definition: Alternating time and space Game Semantics: State of machine determines who

Definition: Alternating time and space Game Semantics: State of machine determines who CMPSCI 601: Recall From Last Time Lecture 3 Definition: Alternating time and space Game Semantics: State of machine determines who controls, White wants it to accept, Black wants it to reject. White wins

More information

CSE200: Computability and complexity Space Complexity

CSE200: Computability and complexity Space Complexity CSE200: Computability and complexity Space Complexity Shachar Lovett January 29, 2018 1 Space complexity We would like to discuss languages that may be determined in sub-linear space. Lets first recall

More information

Notes on Space-Bounded Complexity

Notes on Space-Bounded Complexity U.C. Berkeley CS172: Automata, Computability and Complexity Handout 7 Professor Luca Trevisan April 14, 2015 Notes on Space-Bounded Complexity These are notes for CS278, Computational Complexity, scribed

More information

Computability Theory. CS215, Lecture 6,

Computability Theory. CS215, Lecture 6, Computability Theory CS215, Lecture 6, 2000 1 The Birth of Turing Machines At the end of the 19th century, Gottlob Frege conjectured that mathematics could be built from fundamental logic In 1900 David

More information

Lecture 3. 1 Terminology. 2 Non-Deterministic Space Complexity. Notes on Complexity Theory: Fall 2005 Last updated: September, 2005.

Lecture 3. 1 Terminology. 2 Non-Deterministic Space Complexity. Notes on Complexity Theory: Fall 2005 Last updated: September, 2005. Notes on Complexity Theory: Fall 2005 Last updated: September, 2005 Jonathan Katz Lecture 3 1 Terminology For any complexity class C, we define the class coc as follows: coc def = { L L C }. One class

More information

The space complexity of a standard Turing machine. The space complexity of a nondeterministic Turing machine

The space complexity of a standard Turing machine. The space complexity of a nondeterministic Turing machine 298 8. Space Complexity The space complexity of a standard Turing machine M = (Q,,,, q 0, accept, reject) on input w is space M (w) = max{ uav : q 0 w M u q av, q Q, u, a, v * } The space complexity of

More information

CS6901: review of Theory of Computation and Algorithms

CS6901: review of Theory of Computation and Algorithms CS6901: review of Theory of Computation and Algorithms Any mechanically (automatically) discretely computation of problem solving contains at least three components: - problem description - computational

More information

Boolean circuits. Lecture Definitions

Boolean circuits. Lecture Definitions Lecture 20 Boolean circuits In this lecture we will discuss the Boolean circuit model of computation and its connection to the Turing machine model. Although the Boolean circuit model is fundamentally

More information

CSE355 SUMMER 2018 LECTURES TURING MACHINES AND (UN)DECIDABILITY

CSE355 SUMMER 2018 LECTURES TURING MACHINES AND (UN)DECIDABILITY CSE355 SUMMER 2018 LECTURES TURING MACHINES AND (UN)DECIDABILITY RYAN DOUGHERTY If we want to talk about a program running on a real computer, consider the following: when a program reads an instruction,

More information

Space Complexity. The space complexity of a program is how much memory it uses.

Space Complexity. The space complexity of a program is how much memory it uses. Space Complexity The space complexity of a program is how much memory it uses. Measuring Space When we compute the space used by a TM, we do not count the input (think of input as readonly). We say that

More information

Chapter 1 - Time and Space Complexity. deterministic and non-deterministic Turing machine time and space complexity classes P, NP, PSPACE, NPSPACE

Chapter 1 - Time and Space Complexity. deterministic and non-deterministic Turing machine time and space complexity classes P, NP, PSPACE, NPSPACE Chapter 1 - Time and Space Complexity deterministic and non-deterministic Turing machine time and space complexity classes P, NP, PSPACE, NPSPACE 1 / 41 Deterministic Turing machines Definition 1.1 A (deterministic

More information

Complexity Theory VU , SS The Polynomial Hierarchy. Reinhard Pichler

Complexity Theory VU , SS The Polynomial Hierarchy. Reinhard Pichler Complexity Theory Complexity Theory VU 181.142, SS 2018 6. The Polynomial Hierarchy Reinhard Pichler Institut für Informationssysteme Arbeitsbereich DBAI Technische Universität Wien 15 May, 2018 Reinhard

More information

Outline. Complexity Theory EXACT TSP. The Class DP. Definition. Problem EXACT TSP. Complexity of EXACT TSP. Proposition VU 181.

Outline. Complexity Theory EXACT TSP. The Class DP. Definition. Problem EXACT TSP. Complexity of EXACT TSP. Proposition VU 181. Complexity Theory Complexity Theory Outline Complexity Theory VU 181.142, SS 2018 6. The Polynomial Hierarchy Reinhard Pichler Institut für Informationssysteme Arbeitsbereich DBAI Technische Universität

More information

CSC 5170: Theory of Computational Complexity Lecture 4 The Chinese University of Hong Kong 1 February 2010

CSC 5170: Theory of Computational Complexity Lecture 4 The Chinese University of Hong Kong 1 February 2010 CSC 5170: Theory of Computational Complexity Lecture 4 The Chinese University of Hong Kong 1 February 2010 Computational complexity studies the amount of resources necessary to perform given computations.

More information

Computability and Complexity Theory: An Introduction

Computability and Complexity Theory: An Introduction Computability and Complexity Theory: An Introduction meena@imsc.res.in http://www.imsc.res.in/ meena IMI-IISc, 20 July 2006 p. 1 Understanding Computation Kinds of questions we seek answers to: Is a given

More information

CS3719 Theory of Computation and Algorithms

CS3719 Theory of Computation and Algorithms CS3719 Theory of Computation and Algorithms Any mechanically (automatically) discretely computation of problem solving contains at least three components: - problem description - computational tool - analysis

More information

MTAT Complexity Theory October 13th-14th, Lecture 6

MTAT Complexity Theory October 13th-14th, Lecture 6 MTAT.07.004 Complexity Theory October 13th-14th, 2011 Lecturer: Peeter Laud Lecture 6 Scribe(s): Riivo Talviste 1 Logarithmic memory Turing machines working in logarithmic space become interesting when

More information

PRAM lower bounds. 1 Overview. 2 Definitions. 3 Monotone Circuit Value Problem

PRAM lower bounds. 1 Overview. 2 Definitions. 3 Monotone Circuit Value Problem U.C. Berkeley CS273: Parallel and Distributed Theory PRAM lower bounds. Professor Satish Rao October 16, 2006 Lecturer: Satish Rao Last revised Scribe so far: Satish Rao cribbing from previous years lectures

More information

CSCC63 Worksheet Turing Machines

CSCC63 Worksheet Turing Machines 1 An Example CSCC63 Worksheet Turing Machines Goal. Design a turing machine, M that accepts only strings of the form {w#w w {0, 1} }. Idea. Describe in words how the machine would work. Read first symbol

More information

6.841/18.405J: Advanced Complexity Wednesday, February 12, Lecture Lecture 3

6.841/18.405J: Advanced Complexity Wednesday, February 12, Lecture Lecture 3 6.841/18.405J: Advanced Complexity Wednesday, February 12, 2003 Lecture Lecture 3 Instructor: Madhu Sudan Scribe: Bobby Kleinberg 1 The language MinDNF At the end of the last lecture, we introduced the

More information

Outline. Complexity Theory. Example. Sketch of a log-space TM for palindromes. Log-space computations. Example VU , SS 2018

Outline. Complexity Theory. Example. Sketch of a log-space TM for palindromes. Log-space computations. Example VU , SS 2018 Complexity Theory Complexity Theory Outline Complexity Theory VU 181.142, SS 2018 3. Logarithmic Space Reinhard Pichler Institute of Logic and Computation DBAI Group TU Wien 3. Logarithmic Space 3.1 Computational

More information

Preliminaries and Complexity Theory

Preliminaries and Complexity Theory Preliminaries and Complexity Theory Oleksandr Romanko CAS 746 - Advanced Topics in Combinatorial Optimization McMaster University, January 16, 2006 Introduction Book structure: 2 Part I Linear Algebra

More information

CS151 Complexity Theory. Lecture 1 April 3, 2017

CS151 Complexity Theory. Lecture 1 April 3, 2017 CS151 Complexity Theory Lecture 1 April 3, 2017 Complexity Theory Classify problems according to the computational resources required running time storage space parallelism randomness rounds of interaction,

More information

Notes on Computer Theory Last updated: November, Circuits

Notes on Computer Theory Last updated: November, Circuits Notes on Computer Theory Last updated: November, 2015 Circuits Notes by Jonathan Katz, lightly edited by Dov Gordon. 1 Circuits Boolean circuits offer an alternate model of computation: a non-uniform one

More information

Quantum Computing Lecture 8. Quantum Automata and Complexity

Quantum Computing Lecture 8. Quantum Automata and Complexity Quantum Computing Lecture 8 Quantum Automata and Complexity Maris Ozols Computational models and complexity Shor s algorithm solves, in polynomial time, a problem for which no classical polynomial time

More information

Complete problems for classes in PH, The Polynomial-Time Hierarchy (PH) oracle is like a subroutine, or function in

Complete problems for classes in PH, The Polynomial-Time Hierarchy (PH) oracle is like a subroutine, or function in Oracle Turing Machines Nondeterministic OTM defined in the same way (transition relation, rather than function) oracle is like a subroutine, or function in your favorite PL but each call counts as single

More information

Lecture Notes Each circuit agrees with M on inputs of length equal to its index, i.e. n, x {0, 1} n, C n (x) = M(x).

Lecture Notes Each circuit agrees with M on inputs of length equal to its index, i.e. n, x {0, 1} n, C n (x) = M(x). CS 221: Computational Complexity Prof. Salil Vadhan Lecture Notes 4 February 3, 2010 Scribe: Jonathan Pines 1 Agenda P-/NP- Completeness NP-intermediate problems NP vs. co-np L, NL 2 Recap Last time, we

More information

1 Circuit Complexity. CS 6743 Lecture 15 1 Fall Definitions

1 Circuit Complexity. CS 6743 Lecture 15 1 Fall Definitions CS 6743 Lecture 15 1 Fall 2007 1 Circuit Complexity 1.1 Definitions A Boolean circuit C on n inputs x 1,..., x n is a directed acyclic graph (DAG) with n nodes of in-degree 0 (the inputs x 1,..., x n ),

More information

Complexity (Pre Lecture)

Complexity (Pre Lecture) Complexity (Pre Lecture) Dr. Neil T. Dantam CSCI-561, Colorado School of Mines Fall 2018 Dantam (Mines CSCI-561) Complexity (Pre Lecture) Fall 2018 1 / 70 Why? What can we always compute efficiently? What

More information

6.5.3 An NP-complete domino game

6.5.3 An NP-complete domino game 26 Chapter 6. Complexity Theory 3SAT NP. We know from Theorem 6.5.7 that this is true. A P 3SAT, for every language A NP. Hence, we have to show this for languages A such as kcolor, HC, SOS, NPrim, KS,

More information

Friday Four Square! Today at 4:15PM, Outside Gates

Friday Four Square! Today at 4:15PM, Outside Gates P and NP Friday Four Square! Today at 4:15PM, Outside Gates Recap from Last Time Regular Languages DCFLs CFLs Efficiently Decidable Languages R Undecidable Languages Time Complexity A step of a Turing

More information

SOLUTION: SOLUTION: SOLUTION:

SOLUTION: SOLUTION: SOLUTION: Convert R and S into nondeterministic finite automata N1 and N2. Given a string s, if we know the states N1 and N2 may reach when s[1...i] has been read, we are able to derive the states N1 and N2 may

More information

The Complexity of the Comparator Circuit Value Problem

The Complexity of the Comparator Circuit Value Problem The Complexity of the Comparator Circuit Value Problem Stephen Cook Joint work with Yuval Filmus and Dai Tri Man Lê Department of Computer Science University of Toronto Canada Banff 2013 1 / 28 Outline

More information

Turing Machines Part II

Turing Machines Part II Turing Machines Part II Hello Hello Condensed Slide Slide Readers! Readers! This This lecture lecture is is almost almost entirely entirely animations that that show show how how each each Turing Turing

More information

Turing s Thesis. Fall Costas Busch - RPI!1

Turing s Thesis. Fall Costas Busch - RPI!1 Turing s Thesis Costas Busch - RPI!1 Turing s thesis (1930): Any computation carried out by mechanical means can be performed by a Turing Machine Costas Busch - RPI!2 Algorithm: An algorithm for a problem

More information

DRAFT. Algebraic computation models. Chapter 14

DRAFT. Algebraic computation models. Chapter 14 Chapter 14 Algebraic computation models Somewhat rough We think of numerical algorithms root-finding, gaussian elimination etc. as operating over R or C, even though the underlying representation of the

More information

Complexity. Complexity Theory Lecture 3. Decidability and Complexity. Complexity Classes

Complexity. Complexity Theory Lecture 3. Decidability and Complexity. Complexity Classes Complexity Theory 1 Complexity Theory 2 Complexity Theory Lecture 3 Complexity For any function f : IN IN, we say that a language L is in TIME(f(n)) if there is a machine M = (Q, Σ, s, δ), such that: L

More information

Lecture 1: Course Overview and Turing machine complexity

Lecture 1: Course Overview and Turing machine complexity CSE 531: Computational Complexity I Winter 2016 Lecture 1: Course Overview and Turing machine complexity January 6, 2016 Lecturer: Paul Beame Scribe: Paul Beame 1 Course Outline 1. Basic properties of

More information

CSCE 551 Final Exam, Spring 2004 Answer Key

CSCE 551 Final Exam, Spring 2004 Answer Key CSCE 551 Final Exam, Spring 2004 Answer Key 1. (10 points) Using any method you like (including intuition), give the unique minimal DFA equivalent to the following NFA: 0 1 2 0 5 1 3 4 If your answer is

More information

Lecture 8: Alternatation. 1 Alternate Characterizations of the Polynomial Hierarchy

Lecture 8: Alternatation. 1 Alternate Characterizations of the Polynomial Hierarchy CS 710: Complexity Theory 10/4/2011 Lecture 8: Alternatation Instructor: Dieter van Melkebeek Scribe: Sachin Ravi In this lecture, we continue with our discussion of the polynomial hierarchy complexity

More information

Turing Machines (TM) Deterministic Turing Machine (DTM) Nondeterministic Turing Machine (NDTM)

Turing Machines (TM) Deterministic Turing Machine (DTM) Nondeterministic Turing Machine (NDTM) Turing Machines (TM) Deterministic Turing Machine (DTM) Nondeterministic Turing Machine (NDTM) 1 Deterministic Turing Machine (DTM).. B B 0 1 1 0 0 B B.. Finite Control Two-way, infinite tape, broken into

More information

Lecture 10. Sublinear Time Algorithms (contd) CSC2420 Allan Borodin & Nisarg Shah 1

Lecture 10. Sublinear Time Algorithms (contd) CSC2420 Allan Borodin & Nisarg Shah 1 Lecture 10 Sublinear Time Algorithms (contd) CSC2420 Allan Borodin & Nisarg Shah 1 Recap Sublinear time algorithms Deterministic + exact: binary search Deterministic + inexact: estimating diameter in a

More information

Turing Machines and Time Complexity

Turing Machines and Time Complexity Turing Machines and Time Complexity Turing Machines Turing Machines (Infinitely long) Tape of 1 s and 0 s Turing Machines (Infinitely long) Tape of 1 s and 0 s Able to read and write the tape, and move

More information

Final exam study sheet for CS3719 Turing machines and decidability.

Final exam study sheet for CS3719 Turing machines and decidability. Final exam study sheet for CS3719 Turing machines and decidability. A Turing machine is a finite automaton with an infinite memory (tape). Formally, a Turing machine is a 6-tuple M = (Q, Σ, Γ, δ, q 0,

More information

Turing Machines A Turing Machine is a 7-tuple, (Q, Σ, Γ, δ, q0, qaccept, qreject), where Q, Σ, Γ are all finite

Turing Machines A Turing Machine is a 7-tuple, (Q, Σ, Γ, δ, q0, qaccept, qreject), where Q, Σ, Γ are all finite The Church-Turing Thesis CS60001: Foundations of Computing Science Professor, Dept. of Computer Sc. & Engg., Turing Machines A Turing Machine is a 7-tuple, (Q, Σ, Γ, δ, q 0, q accept, q reject ), where

More information

Finish K-Complexity, Start Time Complexity

Finish K-Complexity, Start Time Complexity 6.045 Finish K-Complexity, Start Time Complexity 1 Kolmogorov Complexity Definition: The shortest description of x, denoted as d(x), is the lexicographically shortest string such that M(w) halts

More information

Busch Complexity Lectures: Turing s Thesis. Costas Busch - LSU 1

Busch Complexity Lectures: Turing s Thesis. Costas Busch - LSU 1 Busch Complexity Lectures: Turing s Thesis Costas Busch - LSU 1 Turing s thesis (1930): Any computation carried out by mechanical means can be performed by a Turing Machine Costas Busch - LSU 2 Algorithm:

More information

where Q is a finite set of states

where Q is a finite set of states Space Complexity So far most of our theoretical investigation on the performances of the various algorithms considered has focused on time. Another important dynamic complexity measure that can be associated

More information

Time Complexity. Definition. Let t : n n be a function. NTIME(t(n)) = {L L is a language decidable by a O(t(n)) deterministic TM}

Time Complexity. Definition. Let t : n n be a function. NTIME(t(n)) = {L L is a language decidable by a O(t(n)) deterministic TM} Time Complexity Definition Let t : n n be a function. TIME(t(n)) = {L L is a language decidable by a O(t(n)) deterministic TM} NTIME(t(n)) = {L L is a language decidable by a O(t(n)) non-deterministic

More information

Space and Nondeterminism

Space and Nondeterminism CS 221 Computational Complexity, Lecture 5 Feb 6, 2018 Space and Nondeterminism Instructor: Madhu Sudan 1 Scribe: Yong Wook Kwon Topic Overview Today we ll talk about space and non-determinism. For some

More information

CSCI 1590 Intro to Computational Complexity

CSCI 1590 Intro to Computational Complexity CSCI 59 Intro to Computational Complexity Overview of the Course John E. Savage Brown University January 2, 29 John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29

More information

The purpose here is to classify computational problems according to their complexity. For that purpose we need first to agree on a computational

The purpose here is to classify computational problems according to their complexity. For that purpose we need first to agree on a computational 1 The purpose here is to classify computational problems according to their complexity. For that purpose we need first to agree on a computational model. We'll remind you what a Turing machine is --- you

More information

Logarithmic space. Evgenij Thorstensen V18. Evgenij Thorstensen Logarithmic space V18 1 / 18

Logarithmic space. Evgenij Thorstensen V18. Evgenij Thorstensen Logarithmic space V18 1 / 18 Logarithmic space Evgenij Thorstensen V18 Evgenij Thorstensen Logarithmic space V18 1 / 18 Journey below Unlike for time, it makes sense to talk about sublinear space. This models computations on input

More information

Complexity Theory of Polynomial-Time Problems

Complexity Theory of Polynomial-Time Problems Complexity Theory of Polynomial-Time Problems Lecture 1: Introduction, Easy Examples Karl Bringmann and Sebastian Krinninger Audience no formal requirements, but: NP-hardness, satisfiability problem, how

More information

Computability and Complexity Theory

Computability and Complexity Theory Discrete Math for Bioinformatics WS 09/10:, by A Bockmayr/K Reinert, January 27, 2010, 18:39 9001 Computability and Complexity Theory Computability and complexity Computability theory What problems can

More information

CS21 Decidability and Tractability

CS21 Decidability and Tractability CS21 Decidability and Tractability Lecture 8 January 24, 2018 Outline Turing Machines and variants multitape TMs nondeterministic TMs Church-Turing Thesis So far several models of computation finite automata

More information

Introduction to Languages and Computation

Introduction to Languages and Computation Introduction to Languages and Computation George Voutsadakis 1 1 Mathematics and Computer Science Lake Superior State University LSSU Math 400 George Voutsadakis (LSSU) Languages and Computation July 2014

More information

Polynomial Time Computation. Topics in Logic and Complexity Handout 2. Nondeterministic Polynomial Time. Succinct Certificates.

Polynomial Time Computation. Topics in Logic and Complexity Handout 2. Nondeterministic Polynomial Time. Succinct Certificates. 1 2 Topics in Logic and Complexity Handout 2 Anuj Dawar MPhil Advanced Computer Science, Lent 2010 Polynomial Time Computation P = TIME(n k ) k=1 The class of languages decidable in polynomial time. The

More information

COMPUTATIONAL COMPLEXITY

COMPUTATIONAL COMPLEXITY ATHEATICS: CONCEPTS, AND FOUNDATIONS Vol. III - Computational Complexity - Osamu Watanabe COPUTATIONAL COPLEXITY Osamu Watanabe Tokyo Institute of Technology, Tokyo, Japan Keywords: {deterministic, randomized,

More information

arxiv: v2 [cs.fl] 29 Nov 2013

arxiv: v2 [cs.fl] 29 Nov 2013 A Survey of Multi-Tape Automata Carlo A. Furia May 2012 arxiv:1205.0178v2 [cs.fl] 29 Nov 2013 Abstract This paper summarizes the fundamental expressiveness, closure, and decidability properties of various

More information

Recap DFA,NFA, DTM. Slides by Prof. Debasis Mitra, FIT.

Recap DFA,NFA, DTM. Slides by Prof. Debasis Mitra, FIT. Recap DFA,NFA, DTM Slides by Prof. Debasis Mitra, FIT. 1 Formal Language Finite set of alphabets Σ: e.g., {0, 1}, {a, b, c}, { {, } } Language L is a subset of strings on Σ, e.g., {00, 110, 01} a finite

More information

1 PSPACE-Completeness

1 PSPACE-Completeness CS 6743 Lecture 14 1 Fall 2007 1 PSPACE-Completeness Recall the NP-complete problem SAT: Is a given Boolean formula φ(x 1,..., x n ) satisfiable? The same question can be stated equivalently as: Is the

More information

A An Overview of Complexity Theory for the Algorithm Designer

A An Overview of Complexity Theory for the Algorithm Designer A An Overview of Complexity Theory for the Algorithm Designer A.1 Certificates and the class NP A decision problem is one whose answer is either yes or no. Two examples are: SAT: Given a Boolean formula

More information

Notes for Lecture 3... x 4

Notes for Lecture 3... x 4 Stanford University CS254: Computational Complexity Notes 3 Luca Trevisan January 18, 2012 Notes for Lecture 3 In this lecture we introduce the computational model of boolean circuits and prove that polynomial

More information

Min/Max-Poly Weighting Schemes and the NL vs UL Problem

Min/Max-Poly Weighting Schemes and the NL vs UL Problem Min/Max-Poly Weighting Schemes and the NL vs UL Problem Anant Dhayal Jayalal Sarma Saurabh Sawlani May 3, 2016 Abstract For a graph G(V, E) ( V = n) and a vertex s V, a weighting scheme (w : E N) is called

More information

Homework Assignment 6 Answers

Homework Assignment 6 Answers Homework Assignment 6 Answers CSCI 2670 Introduction to Theory of Computing, Fall 2016 December 2, 2016 This homework assignment is about Turing machines, decidable languages, Turing recognizable languages,

More information

Turing Machines Part III

Turing Machines Part III Turing Machines Part III Announcements Problem Set 6 due now. Problem Set 7 out, due Monday, March 4. Play around with Turing machines, their powers, and their limits. Some problems require Wednesday's

More information

Theory of Computation. Ch.8 Space Complexity. wherein all branches of its computation halt on all

Theory of Computation. Ch.8 Space Complexity. wherein all branches of its computation halt on all Definition 8.1 Let M be a deterministic Turing machine, DTM, that halts on all inputs. The space complexity of M is the function f : N N, where f(n) is the maximum number of tape cells that M scans on

More information

CSE 105 THEORY OF COMPUTATION

CSE 105 THEORY OF COMPUTATION CSE 105 THEORY OF COMPUTATION Fall 2016 http://cseweb.ucsd.edu/classes/fa16/cse105-abc/ Today's learning goals Sipser Ch 3 Trace the computation of a Turing machine using its transition function and configurations.

More information

CS154, Lecture 15: Cook-Levin Theorem SAT, 3SAT

CS154, Lecture 15: Cook-Levin Theorem SAT, 3SAT CS154, Lecture 15: Cook-Levin Theorem SAT, 3SAT Definition: A language B is NP-complete if: 1. B NP 2. Every A in NP is poly-time reducible to B That is, A P B When this is true, we say B is NP-hard On

More information

Foundations of Query Languages

Foundations of Query Languages Foundations of Query Languages SS 2011 2. 2. Foundations of Query Languages Dr. Fang Wei Lehrstuhl für Datenbanken und Informationssysteme Universität Freiburg SS 2011 Dr. Fang Wei 30. Mai 2011 Seite 1

More information

CSE 2001: Introduction to Theory of Computation Fall Suprakash Datta

CSE 2001: Introduction to Theory of Computation Fall Suprakash Datta CSE 2001: Introduction to Theory of Computation Fall 2012 Suprakash Datta datta@cse.yorku.ca Office: CSEB 3043 Phone: 416-736-2100 ext 77875 Course page: http://www.cs.yorku.ca/course/2001 11/13/2012 CSE

More information

COMP/MATH 300 Topics for Spring 2017 June 5, Review and Regular Languages

COMP/MATH 300 Topics for Spring 2017 June 5, Review and Regular Languages COMP/MATH 300 Topics for Spring 2017 June 5, 2017 Review and Regular Languages Exam I I. Introductory and review information from Chapter 0 II. Problems and Languages A. Computable problems can be expressed

More information

Decidability (What, stuff is unsolvable?)

Decidability (What, stuff is unsolvable?) University of Georgia Fall 2014 Outline Decidability Decidable Problems for Regular Languages Decidable Problems for Context Free Languages The Halting Problem Countable and Uncountable Sets Diagonalization

More information

Circuits. Lecture 11 Uniform Circuit Complexity

Circuits. Lecture 11 Uniform Circuit Complexity Circuits Lecture 11 Uniform Circuit Complexity 1 Recall 2 Recall Non-uniform complexity 2 Recall Non-uniform complexity P/1 Decidable 2 Recall Non-uniform complexity P/1 Decidable NP P/log NP = P 2 Recall

More information

Show that the following problems are NP-complete

Show that the following problems are NP-complete Show that the following problems are NP-complete April 7, 2018 Below is a list of 30 exercises in which you are asked to prove that some problem is NP-complete. The goal is to better understand the theory

More information

The Complexity and Proof Complexity of the Comparator Circuit Value Problem

The Complexity and Proof Complexity of the Comparator Circuit Value Problem The Complexity and Proof Complexity of the Comparator Circuit Value Problem Stephen Cook Joint work with Yuval Filmus, Dai Tri Man Lê, and Yuli Ye Department of Computer Science University of Toronto Canada

More information

Theory of Computation Time Complexity

Theory of Computation Time Complexity Theory of Computation Time Complexity Bow-Yaw Wang Academia Sinica Spring 2012 Bow-Yaw Wang (Academia Sinica) Time Complexity Spring 2012 1 / 59 Time for Deciding a Language Let us consider A = {0 n 1

More information

1 More finite deterministic automata

1 More finite deterministic automata CS 125 Section #6 Finite automata October 18, 2016 1 More finite deterministic automata Exercise. Consider the following game with two players: Repeatedly flip a coin. On heads, player 1 gets a point.

More information

Part I: Definitions and Properties

Part I: Definitions and Properties Turing Machines Part I: Definitions and Properties Finite State Automata Deterministic Automata (DFSA) M = {Q, Σ, δ, q 0, F} -- Σ = Symbols -- Q = States -- q 0 = Initial State -- F = Accepting States

More information

TWO-WAY FINITE AUTOMATA & PEBBLE AUTOMATA. Written by Liat Peterfreund

TWO-WAY FINITE AUTOMATA & PEBBLE AUTOMATA. Written by Liat Peterfreund TWO-WAY FINITE AUTOMATA & PEBBLE AUTOMATA Written by Liat Peterfreund 1 TWO-WAY FINITE AUTOMATA A two way deterministic finite automata (2DFA) is a quintuple M Q,,, q0, F where: Q,, q, F are as before

More information

Computational Complexity Theory. Markus Bläser, Holger Dell, Karteek Sreenivasaiah Universität des Saarlandes Draft June 15, 2015 and forever

Computational Complexity Theory. Markus Bläser, Holger Dell, Karteek Sreenivasaiah Universität des Saarlandes Draft June 15, 2015 and forever Computational Complexity Theory Markus Bläser, Holger Dell, Karteek Sreenivasaiah Universität des Saarlandes Draft June 15, 2015 and forever 2 1 Simple lower bounds and gaps Complexity theory is the science

More information

U.C. Berkeley CS278: Computational Complexity Professor Luca Trevisan August 30, Notes for Lecture 1

U.C. Berkeley CS278: Computational Complexity Professor Luca Trevisan August 30, Notes for Lecture 1 U.C. Berkeley CS278: Computational Complexity Handout N1 Professor Luca Trevisan August 30, 2004 Notes for Lecture 1 This course assumes CS170, or equivalent, as a prerequisite. We will assume that the

More information

Lecture 6: Oracle TMs, Diagonalization Limits, Space Complexity

Lecture 6: Oracle TMs, Diagonalization Limits, Space Complexity CSE 531: Computational Complexity I Winter 2016 Lecture 6: Oracle TMs, Diagonalization Limits, Space Complexity January 22, 2016 Lecturer: Paul Beame Scribe: Paul Beame Diagonalization enabled us to separate

More information

Introduction to Turing Machines. Reading: Chapters 8 & 9

Introduction to Turing Machines. Reading: Chapters 8 & 9 Introduction to Turing Machines Reading: Chapters 8 & 9 1 Turing Machines (TM) Generalize the class of CFLs: Recursively Enumerable Languages Recursive Languages Context-Free Languages Regular Languages

More information

V Honors Theory of Computation

V Honors Theory of Computation V22.0453-001 Honors Theory of Computation Problem Set 3 Solutions Problem 1 Solution: The class of languages recognized by these machines is the exactly the class of regular languages, thus this TM variant

More information

CS4026 Formal Models of Computation

CS4026 Formal Models of Computation CS4026 Formal Models of Computation Turing Machines Turing Machines Abstract but accurate model of computers Proposed by Alan Turing in 1936 There weren t computers back then! Turing s motivation: find

More information

Complexity: Some examples

Complexity: Some examples Algorithms and Architectures III: Distributed Systems H-P Schwefel, Jens M. Pedersen Mm6 Distributed storage and access (jmp) Mm7 Introduction to security aspects (hps) Mm8 Parallel complexity (hps) Mm9

More information

Turing Machines Part II

Turing Machines Part II Turing Machines Part II Problem Set Set Five Five due due in in the the box box up up front front using using a late late day. day. Hello Hello Condensed Slide Slide Readers! Readers! This This lecture

More information

Properties That Characterize

Properties That Characterize JOURNAL OF COMPUTER AND SYSTEM SCIENCES 43, 380404 (1991) Properties That Characterize LOGCFL* H. VENKATESWARAN School of Information and Computer Science, Georgia Institute of Technology, Atlanta, Georgia

More information

Announcements. Problem Set 7 graded; will be returned at end of lecture. Unclaimed problem sets and midterms moved!

Announcements. Problem Set 7 graded; will be returned at end of lecture. Unclaimed problem sets and midterms moved! N P NP Announcements Problem Set 7 graded; will be returned at end of lecture. Unclaimed problem sets and midterms moved! Now in cabinets in the Gates open area near the drop-off box. The Complexity Class

More information

Parallelism and Machine Models

Parallelism and Machine Models Parallelism and Machine Models Andrew D Smith University of New Brunswick, Fredericton Faculty of Computer Science Overview Part 1: The Parallel Computation Thesis Part 2: Parallelism of Arithmetic RAMs

More information

NP, polynomial-time mapping reductions, and NP-completeness

NP, polynomial-time mapping reductions, and NP-completeness NP, polynomial-time mapping reductions, and NP-completeness In the previous lecture we discussed deterministic time complexity, along with the time-hierarchy theorem, and introduced two complexity classes:

More information

Q = Set of states, IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar

Q = Set of states, IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar Turing Machine A Turing machine is an abstract representation of a computing device. It consists of a read/write

More information