Applied Computer Science II Chapter 7: Time Complexity. Prof. Dr. Luc De Raedt. Institut für Informatik Albert-Ludwigs Universität Freiburg Germany

Transcription

1 Applied Computer Science II Chapter 7: Time Complexity Prof. Dr. Luc De Raedt Institut für Informati Albert-Ludwigs Universität Freiburg Germany Overview Measuring complexity The class P The class NP NP-completeness Additional NP-complete problems

2 Measuring complexity How much time needed to decide a language A= {0 1 0} Use Machine M1 Measuring complexity Worst-case versus average-case The time complexity of a TM M is the function f : Ν Ν, where f( n) is the max number of steps that M uses on any input of length n. We typically use big-o and small-o notation (cf. ACS I); asymptotic analysis

3 Big-O + Let f, g: Ν R be two functions. We say that f( n) is O( g( n)) if positive integers c and n exist so that for every integer Ogn ( ( )) is an upper bound for f( n) 3 2 ( ) f n = n + n + n+ 3 = ( ) choose = 6 and 0 = 10 f O n c n 4 2 is also ( ) but not ( ) f O n O n f( n) = 3 n.log n+ 5 n.log log n+ 2 f( n) = O( n.log n) n n holds that f( n) cg. ( n) 0 0 p log b x = p if and only if b = x (definition) log x*y = log x + log y b b b log x/y = log x - log y b b b log x = p log x which implies that (x ) = x b p p q (pq) b log x = log x * log a b a b

4 Small-o + Let f, g: Ν R be two functions. We say that f( n) is o( g( n)) if f( n) lim = 0, i.e. for any real number c > 0, n gn ( ) there is a number n where f( n) < cg. ( n) for all n n o 0 Time Complexity Let t : Ν Νbe a function. Define the time complexity class TIME( t( n)) ={ L L is a language decided by an O( t( n)) time Turing machine} 2 So, {0 1 0} TIME( n ) Can we find a machine that decides A assymptotically more quicly? 2 i.e. is A TIME( t( n)) for t( n) = o( n )?

5 Analyzing algorithms Analyzing algorithms 2-tape machine

6 Complexity relationship among models Theorem 7.8 Let tn ( ) be a function where tn ( ) n. Then every tn ( ) time multitape Turing machine has an equivalent Otn 2 ( ( ) ) time single-tape Turing machine. Proof elements Let M be a tape TM that runs in t( n) 2 Construct (single-tape TM) that runs in ( ( ) ) time S 2 ( ( ) ) steps 2 2 ( ) (( )) (( )) when ( ) O t n Analyze simulation To simulate each step of M, S maes two scans over active input tape each of the active parts of the tapes are of length at most t( n) i.e. t. ( n) = O( t( n)) and possibly shifts (for inserting a new cell) shifts tae Otn ( ( )) time First phase : copying inputs from M to S taes On ( ) steps Second phase : simulate the tn ( ) steps of M taes tnotn ( ). ( ( )) = Otn On + Otn = Otn tn n

7 Definition Let N be an NTM that is a decider. The running time of Nis the function f : Ν Ν, where f( n) is the maximum number of steps that N uses on any branch of its computation of any input of length n as shown below Definition 7.9 including figure 7.1 Theorem 7.10 Let tn ( ) be a function where tn ( ) n. Then every tn ( ) time nondeterministic Turing machine has an equivalent 2 Otn (( )) time deterministic single-tape Turing machine.

8 Proof elements Every branch of N has length at most t( n)for input of length n Every node in tree has at most b children (determined by δ ) Exploration tree is breadth first : b+ b + + b 2 t( n) nodes and le Ob t( n) = ( ) t( n) O( t( n)) aves Time for processing one node (or leaf) starting from root Otn ( ( )) Running time of deterministic 3-tape TM Otn ( ( )). Ob ( ) = 2 Use transformation to deterministic single tape gives (2 ) = 2 Otn (( )) 2 Otn (( )) Differences in models Polynomial difference between singleand multiple tape deterministic TM Exponential difference between deterministic and non-deterministic TM Compare exponential versus polynomial behaviour!

9 Towards the class P Distinction between : Exponential algorithms Brute force search Impractical Polynomial Practical Robust for reasonable models of computation (I.e. polynomially equivalent) Here, we ignore differences between, 5 e.g. On ( )and On ( ), level of abstraction Definition The class P P is the class of languages that are decidable in polynomial time on a deterministic single -tape TM, i.e. P= TIME( n ) U P is invariant for all models of computation that are polynomially equivalent to a single tape deterministic TM P roughly corresponds to the class of problems that are realistically solvable on a computer

10 Conventions High-level descriptions of algorithms Abstraction of specific model used (as long as polynomially equivalent) Algorithms proceed in numbered stages Analysis in two steps : A polynomial bound on the number of stages Each stage can be implemented in polynomial time on deterministic single tape TM (or poly. Equivalent model) Conventions Encoding method for problems We use <.> notation as before Reasonable methods require encoding and decoding of objects into other (internal?) representation to require polynomial time. E.g (unreasonable) instead of 9 (or other base -notation)

11 Option 1 : Encoding Graphs List of nodes List of edges Option 2 : Adjacency matrix 1 iff there is an edge from node i to node j Entry ( i, j) = 0 otherwise PATH = { < G, s, t > G is a directed graph that has a directed path from s to t} Figure 7.2

12 Theorem PATH P Insert algorithm RELPRIME = { < x, y > x and y are relative prime} i.e. the greatest common divisor of x and y is 1 Theorem RELPRIME P Insert algorithm

13 Theorem Every context-free language is a member of P Assume language in Chomsy Normal Form Then each derivation of a string whas 2n 1steps (Generating all derivations is not polynomial!) Dynamic programming : accumulate information about smaller problems eep trac of table of size n n table( i, j) = the set of variables generating w... w with i j i j

14 Towards the class NP HAMPATH = { < G, s, t > G is a directed graph with a hamiltonian path from s to g} i.e. it is directed and contains all nodes in G HAMPATH Easy to obtain exponential time algorithm : Generate all paths of length m (no. of nodes) and chec whether they are hamiltonian Polynomial verifiability : Checing whether a given path is hamiltonian can be done in polynomial time Verifying existence of hamiltonian path is easier than determining the existence Consider also HAMPATH Similar for COMPOSITES COMPOSITES = { x x = p. q for integers p, q > 1}

15 The class NP Definition A verifier for a language A is an algorithm Vs.t. A= { w V accepts < w, c> for some string c} c : certificate, proof Time of verifiers is measured in terms of size of w (implies that c is of size polynomial in size w) Polynomial time verifiers Definition NPis the class of languages that have polynomial time verifiers NP comes from Nondeterministic polynomial time NTM for HAMPATH Algorithm

16 Theorem A language is in NP iff it is decided by some nondeterministic polynomial time algorithm idea Proof Definition NTIME( tn ( )) = { L Lis a language decided by an Otn ( ( )) time NTM} Corollary U NP= NTIME( n ) CLIQUE = { < G, > G is an undirected graph with a -clique}

17 Theorem CLIQUE is in NP SUBSET SUM = { < S, t > S = { x,..., x } and for some { y,..., y } { x,..., x }we have y = t} The y and x are multi-sets. 1 1 l 1 i i Theorem SUBSET SUM is in NP

18 P versus NP P versus NP P= the class of languages where membership can be decided quicly NP=the class of languages where membership can be verified quicly n NP EXPTIME= UTIME(2 ) NP-complete problems : if a polynomial time (determ.) algorithm exists for one NP-complete problem, then P=NP

19 Boolean Logic true is denoted by 1 false is denoted by 0 and or not SAT = { < φ > φ is a satisfiable boolean formula} Theorem (Coo-Levin) SAT P iff P=NP 0 0= 0 0 0= 0 0= 0= 1 0 1= 0 0 1= 1 1= 1= 0 1 0= 0 1 0= 1 1 1= 1 1 1= 1 A Boolean formula is an expression involving Boolean variables and operations E.g. φ=( x y) ( x z) Assignment of values (0 and 1) to variables For each assignment one can evalue a formula If the formula evaluates to 1, we say that the assignment satisfies the formula. The satisfiabilit y problem is to test whether a formula is satisfiable (i.e. whether there exists an assignment that satisfies the formula) A function f : Σ Σ Polynomial time reducibility Fig 7.6 * * is a polynomial time computable function if some polynomial time TM M exists that halts with just f( w) on its tape, when started on any input w Language Ais polynomial time (mapping) reducible to language B written A B, if a polynomial time computable function f : Σ Σ P exists where for every w: w Aiff f( w) B f is a polynomial time reduction * *

20 Theorem If A B and B P, then A P P Proof: N wors in polynomial time 3 SAT{ < φ > φis a satisfiable 3cnf formula} cnf : conjunctive normal form, i.e. a conjunction of disjunctions (disjunctions are clauses, elements of disjunctions are literals) 3cnf : each clause/disjunction has exactly three literals e.g. ( x y z) ( y x w) Theorem 3SATis polynomial time reducible to CLIQUE Proof Let φ be of the form ( a b c )... ( a b c ) f( φ) =< G, > where all nodes in G are organised in triples t1,..., t each triple corresponds to a cluase and has 3 nodes The edges of G connect all nodes except for : 1. edges between the nodes of the same triple 2. nodes with contradictory label, i.e. x and x Now prove that f is a polynomial time reduction

21 Fig 7.7 NP-completeness Definition A language B is NP-complete if 1. B is in NP, and 2. every A in NP is polynomial time reducible to Theorem If B is NP-complete and B P, then P=NP Theorem If Bis NP-complete and B C for C in NP, then C is NP-complete P B

22 Theorem Coo-Levin SAT is NP-complete Proof 1. SAT is in NP (easy) 2. any language A in NP is polynomial time reducible to SAT Let N be a NTM that decides Ain time n Define an (accepting) tableau for N on w define reduction f from Ato SAT on input w, define f( w) let us first define the boolean variables. Let C = Q Γ {#} for 1 i, j n, for s C: define x = cell( i, j) contains symbol s i, j, s Now define φ = φ φ φ cell start move φ accept ( x,,) ( ( x,, x,,)) s C s, t C, s t φ = "each cell contains exactly one symbol" cell i j s i j s i j t 1 i, j n φ = x x x... x x... x x start q w n+ w n+ n n 1,1,# 1,2, 0 1,3, 1 1, 2, 1, 3, n 1, 1, 1,,# "the first row corresponds to the starting situation"

23 φ accept = 1 i, j n x i, j, q accept "at least one of the configurations is an accept state" concept of legal windows e.g. δ( q, a) = {( q, b, R)} and δ( q, b) = {( q, c, L),( q, a, R)} Claim if the top row of the table is the start configuration and if every window (2x3) is legal then each row legally follows the preceding one. φ move 1 6 = 1 < i n,1< j< n for each window : a,..., a is a legal window "the i, j window is legal" ( x x x x x x ) i, j 1, a i, j, a i, j+ 1, a i+ 1, j 1, a i+ 1, j, a i+ 1, j+ 1, a Analyze the complexity tableau has n n = n 2 cells each cell has lvariables (for symbols in C) thus number of variables is On ( ) Now investigate different components of φ = φ φ φ φ On On O n) 2 (All ( ) or ( ); indices can be encoded in (log cell start move accept f( w)is of polynomial size and can be computed in polynomial time

24 Theorem 3SATis NP-complete Proof modify construction in SATproof first rewrite φ in cnf form. second rewrite clauses such that they have 3 literals 1. if less than 3 literals e.g. a brewrite to e.g. a b b 2. if more than 3 literals ( a... a ) rewrite into ( a a z ) ( z a z ) ( z a a ) l 3 l 1 l 1 l Corollary CLIQUE is NP-complete VERTEX COVER = { < G, > Gis an undirected graph that has a -node vertex cover} Let Gbe an undirected graph ( V, E) and a number A vertex cover is a set of nodes V ' Vsuch that for all edges ( uv, ) Eholds that u V 'or v V ' Theorem VERTEX COVER is NP-complete Proof produce clause gadget and variable gadget set = m+ 2l where φ has m variables and l clauses then prove satisfiability if and only if graph has a vertex cover with nodes

25 Theorem HAMPATH is NP-complete Proof we show that 3SAT P HAMPATH Let φ be of the form ( a b c )... ( a b c ) 1 1 1

26 UHAMPATH = { < G > Gis an undirected graph that contains a hamiltonian path} Theorem UHAMPATH is NP-complete Proof HAMPATH f( G) = G' P UHAMPATH in mid out for all nodes u in G, introduce nodes u, u, u in G' out for s and t in G, introduce only s, t introduce edges between u and u and between u and u out in connect also u to v if there is an edge from u to v each path su,,..., u, tin Ghas a counterpart 1 in mi d out in 1 1 u t out s, u, u, 1,..., and vice versa in in mid mid out

27 SUBSET SUM = { < S, t > S = { x,..., x } and for some { y,..., y } { x,..., x }we have y = t} The y and x are multi-sets. 1 1 l 1 i i Theorem SUBSET SUM is NP-complete. Proof 1. SUBSET SUM NP 2.3SAT SUBSET SUM P Let φ be in 3cnf and have vars x,..., x and clauses c,..., c 1 l 1 Set Covering SetCover={< T,..., T, M, n> T M (finite set) and there exist n 1 sets T,..., T such that T = M} i1 in i j U j i Example <{a,b,c},{b,c,d},{c.d},{a,b,c,d},2> SetCover <{a,b,c},{b,c,d},{c.d},{a,b,c,d},1> SetCover

28 BinPacing BinPacing={< a,..., a,, b> a b (all in ) and the objects a 1 n i i be divided over the bins of size b such that no bin overflows} Example <4,5,7,2,2,9> BinPacing because 4+5 and 7+2 are both smaller than 9 2 bins of size 9 <4,5,7,2,2,8> BinPacing Traveling Salesman Problem TSP={< M, > M contains distance between city iand j ij and there exists a path through all cities of size less than } path 1,2,3 is a solution for 9 instead of 10, no solution

29 Coloring Coloring={< G, > Gis a graph and there exists a coloring with colors so that no two adjacent have the same color} Key Reductions SAT 3SAT SetCover Clique SubsetSum Hampath Coloring VertexCover Partition UHampath BinPacing TSP

30 Conclusions The class P The class NP NP-completeness(e.g. 3SAT) Polynomial time reductions Two further classes : conp: conp={ A A NP} NP-hard: A language B is NP-hard if every A in NP is polynomial time reducible to B Standard Reference Garey and Johson, Computers and Intractability A guide to the theory of NPcompleteness, W.H.Freeman, 1979.