Polycopié d'informatique Fondamentale IMA4-SC Version 2013 Polytech'Lille, Villeneuve d'ascq Laure Gonnord
A man provided with paper, pencil, and rubber, and subject to strict discipline, is in eect a universal machine. Alan Turing, Intelligent Machinery : A Report by A. M. Turing (Summer 1948) An d'améliorer ce poly n'hésitez pas à me soumettre toute critique, suggestion, remarque ou correction, dans mon casier ou, électroniquement, à l'adresse Laure.Gonnord@polytech-lille.fr
Laure Gonnord Polycopié d'informatique Fondamentale IMA4SC S8 2013 Table des matières 1 Introduction, automates nis et langages réguliers 4 2 Automates à pile et grammaires hors contexte 15 3 Automates à compteurs et machines de turing 22 4 Graphes, notions de NP-complétude 31 5 Construction d'un compilateur en Java 48 3/58
Laure Gonnord Polycopié d'informatique Fondamentale IMA4SC S8 2013 Chapitre 1 Introduction, automates nis et langages réguliers Dans ce cours nous abordons le modèle de calcul le plus simple, les automates nis, ainsi que la notion de langage régulier. 4/58
Course introduction and schedule Informatique Fondamentale IMA S8 Cours 1 - Intro + schedule + finite state machines Until now Laure Gonnord http://laure.gonnord.org/pro/teaching/ Laure.Gonnord@polytech-lille.fr Université Lille 1 - Polytech Lille 2013 During S5,S6,S7, IMA students have learnt : (programming stuff) C programing and compiling, (conception stuff) algorithm design and encoding, (microelec) circuits and embedded systems hardware, (auto) design flows of industrial processes Solved (computation) problems by ad-hoc solutions. Until now 2/2 Course introduction and schedule Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 2 / 38 Schedule Course introduction and schedule But : No general scheme to design algorithms. (manual) evaluation of the cost of our programs. worse no assurance of correctness. even worse it there always a solution? All these problems will be addressed in this course Finite state machines (regular automata), regular languages. Notion of non determinism. Link with circuits. I/O automata, stack automata and grammars. Link to simple languages. Counter automata, Turing machines and undecidable problems. Link to classical programs. Graphs and classical problems/algorithms on graphs. Compiler Construction. (+ lab) Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 3 / 38 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 4 / 38
Course introduction and schedule Finite state machines 1 Finite state machines 2 Regular Languages I - Regular languages and automata 3 The notion of non determinism 4 Classical Algorithms 5 Expressivity of regular languages 6 Link to other models Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 5 / 38 What for? Finite state machines Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 6 / 38 Example Finite state machines Opening/Closing door - Source Wikipedia. We want to model : the behaviour of systems with behavioural modes. the behaviour of (Boolean) circuits. sets of words. A finite representation. state transition close_door transition condition 1 Opened E: open door open_door 2 Closed entry action E: close door Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 7 / 38 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 8 / 38
General definition Finite state machines Finite state machine (FSM) or regular automata States are labeled, and there are finitely many (s Q) Initial state(s) (i I) and accepting (terminating/finishing) states (t F ) Transitions are finitely many. Transitions are labeled with letters (a A). The transition function is δ : Q A Q. b Finite state machines Accepted language 1/2 Accepted word A word w on the alphabet A is accepted by the automaton iff there exists a finite path from an initial state to an accepting state which is labeled by w. b s a 0 s a 1 s 2 s a 0 s a 1 s 2 w = aa is accepted/recognised. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 9 / 38 Finite state machines Accepted language 2/2 Accepted language The accepted language of a given automaton A is the set of all accepted words and is denoted by L(A). Remark : it can be infinite. L(A) = {ab k a, k N}. b s a 0 s a 1 s 2 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 10 / 38 Finite state machines Data Structure for implementation Problem : how to encode an automata? b s a 0 s a 1 s 2 The transition function can be (for instance) encoded as a transition table : s 0 s 1 s 2 s 0 a s 1 b a s 2 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 11 / 38 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 12 / 38
Regular Languages Regular Languages Goal 1 Finite state machines 2 Regular Languages 3 The notion of non determinism 4 Classical Algorithms Problem : how to describe languages easily in a textual linear way? use regular expressions. 5 Expressivity of regular languages 6 Link to other models Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 13 / 38 Regular expressions Regular Languages Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 14 / 38 Regular Languages Regular expression vs word Regular expression (recursive def) A regular expression e on the alphabet A is defined by induction. It can be of any of the following kinds : the empty word ε a letter a A a choice between an expression e 1 and another expression e 2 : e 1 + e 2 two successive expressions : e 1 e 2 0,1 or more successive occurrences of e 1 : e 1. A regular expression encodes the form of a word. For instance : i m a (3 + 4 + 5) (on the alphabet {i, m, a, 3, 4, 5}) describes all words beginning by the prefix ima and finishing by one of the numbers 3, 4 or 5. A regular expression describes a language Example with A = {a, b, c, d} : e = a (b + c d) Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 15 / 38 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 16 / 38
Regular Languages Regular Languages Regular language Linux world Regular language Given a regular expression e, L(e) denotes the set of words (the language) that are described by the regular expression e. Example with A = {0,..., 9} : e = (1 + 2 +... + 9) (0 + 1 + 2 +... + 9). What is L(e)? Some commands use (extended) regular expressions (regexp) : ls *.pdf lists all pdfs of the current directory grep ta*.* *.c find all lines in.c files that contains words that begin with t + some a s. sed ''s ta*.*/toto/g'' file replace all occurrences... Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 17 / 38 Regular Languages Relationship between automata and languages - 1/3 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 18 / 38 Regular Languages Relationship between automata and languages - 2/3 First, some experiments. Given the following automata A, are you able to give a regular expression e such that L(e) = L(A)? b s a 0 s a 1 s 2 And the converse : Given the following regular expression e = b (c + a), are you able to give an automaton A such that L(A) = L(e)? e =? Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 19 / 38 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 20 / 38
Regular Languages The notion of non determinism Relationship between automata and languages - 3/3 1 Finite state machines 2 Regular Languages General result - Kleene Theorem The regular languages are exactly the languages that are described by finite automata. What we ve done before is always possible. 3 The notion of non determinism 4 Classical Algorithms 5 Expressivity of regular languages 6 Link to other models Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 21 / 38 Goal The notion of non determinism Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 22 / 38 Definition The notion of non determinism Sometimes some info lacks to make a choice between two transitions : c q a 0 q c 1 q 2 From state q 1, there is a non deterministic choice while reading c : either go to state q 2 or stay in q 1. Non deterministic FSM A deterministic automaton is A =< A, Q, I, F, δ > with δ : A Q Q A non deterministic automata is the same with : δ : A Q P (Q) A non deterministic automata with ε-transitions is the same with δ : (A {ε}) Q P (Q) Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 23 / 38 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 24 / 38
The notion of non determinism Classical Algorithms Example 1 Finite state machines A non deterministic automata with ε-transitions : Then L(A) = a (c d ). c p a q d r ε 2 Regular Languages 3 The notion of non determinism 4 Classical Algorithms 5 Expressivity of regular languages 6 Link to other models Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 25 / 38 Classical Algorithms Find the associated language Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 26 / 38 Classical Algorithms Construction of automaton from a regular expression Goal : Given A an automaton, find the associated language. Exercises! Goal : Given a regular expression, construct a regular automaton that recognises it. Exercises! Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 27 / 38 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 28 / 38
Classical Algorithms Classical Algorithms Determinisation Other Algorithms Goal : transform a non deterministic automaton into a deterministic one. Exercises! In the literature you will easily find : algorithms to eliminate ε transitions without determinising (ε closure) ; algorithms to minimise automata (the number of states) ; algorithms to use automata to find words in a text ; algorithms to test language inclusion (if they are regular)... Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 29 / 38 Expressivity of regular languages Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 30 / 38 Expressivity of regular languages Non regular languages 1 Finite state machines 2 Regular Languages 3 The notion of non determinism 4 Classical Algorithms 5 Expressivity of regular languages 6 Link to other models Important result There exists some non-regular languages. Examples of non-regular languages : {a n b n, n N}. palindromes on a non-singleton alphabet. {a p, p prime} There exists a quite systematic way to prove that a given language is not regular (Pumping Lemma). Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 31 / 38 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 32 / 38
1 Finite state machines Link to other models Link to other models Moore and Mealy Machines - 1 Moore : I/O machine whose output values are determined solely by the current state : 2 Regular Languages 3 The notion of non determinism 4 Classical Algorithms 5 Expressivity of regular languages 6 Link to other models source Wikipedia Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 33 / 38 Link to other models Moore and Mealy Machines - 2 Mealy : output values are determined both by the current state and the value of the input. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 34 / 38 Link to other models UML Implementation of FSMs UML variants of FSMs are hierarchical, react to messages and call functions. source Wikipedia source Wikipedia Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 35 / 38 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 36 / 38
Link to other models Conclusion Hardware Implementation of FSMs Summary It requires : a register to store state variables a block of combinational logic for the state transition (optional) a block of combinatorial logic for the output Regular Automata or Finite State Machines are : Acceptors for regular languages. But some languages are not regular. Algorithmically efficient. Useful to describe (simple) behaviours of systems. Closely linked to circuits. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 37 / 38 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 38 / 38
Laure Gonnord Polycopié d'informatique Fondamentale IMA4SC S8 2013 Chapitre 2 Automates à pile et grammaires hors contexte Les automates à pile sont un modèle de calcul un peu plus puissant que les automates nis. Nous caractériserons leur pouvoir d'expressivité en terme de grammaire. 15/58
Informatique Fondamentale IMA S8 Cours 2 - Stack automata + Grammars Laure Gonnord http://laure.gonnord.org/pro/teaching/ Laure.Gonnord@polytech-lille.fr II - Stack automata and Grammars Université Lille 1 - Polytech Lille 2013 Stack automata Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 2 / 22 Stack automata What for? 1 Stack automata 2 Grammars Express more than regular languages! Give a way for automata to count, to have a memory. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 3 / 22 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 4 / 22
Stack automata Stack automata Example General definition (source : Wikipedia) Stack/P automata States (Q), initial state (q 0 ), finite states (F). Two alphabets (one for read : Σ one for stack : Γ) γ 0 Γ the end of stack character. Transitions are finitely many. A stack to write into. Transitions use the stack. The transition function is : Q (Σ ε) Γ P(Q Γ ) Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 5 / 22 How it works? 1/3 Stack automata Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 6 / 22 How it works? 2/3 Stack automata Configuration A configuration is a triple (q, w, α) where : q is the current state w Σ the part of the input tape which is not yet read α Γ the current stack word Two different conditions for accepting words : accepting state : the read word leads to a configuration of the form (q, ε, α) where q F (with any α) empty stack :... (q, ε, γ 0 ) (with any q, but γ 0 is the empty stack character. Given a configuration (q, w, α), the next one is computed by the help of the transition function Q (Σ ε) Γ P(Q Σ ) : enabled transitions : those of the form (q, a, A) (q, A ) where a is the next letter to read on the input tape (or ε), and A the letter on top of the stack : w = aw and α = Aα. Pick one enabled transition, and compute the next configuration (q, w, A α ). Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 7 / 22 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 8 / 22
Stack automata Stack automata How it works? 3/3 Important theoretical results on PDA (source : wikipedia) Important Stack automata are non deterministic! The two acceptance criteria define the same class of languages Try to derive 0011 and 00111! The PDA recognises {0 n 1 n n N} by accepting state. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 9 / 22 Grammars Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 10 / 22 Grammars Goal 1 Stack automata 2 Grammars Problem : Express languages with the same expressivity as stack automata. use grammars Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 11 / 22 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 12 / 22
Grammars Grammars General grammars Grammars Grammar rule A grammar rule (production rule) is of the form w w where w and w are words. A grammar is a set of rules. Grammar A grammar is composed of : A finite set N of non terminal symbols A finite set Σ of terminal symbols (disjoint from N) A finite set of production rules, each rule of the form w w where w is a word on Σ N with at least one letter of N. w is a word on Σ N. A start symbol S N. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 13 / 22 Grammars Grammars Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 14 / 22 Associated Language Grammars Example : S asb S ε Derivation G a grammar defines the relation : x G y iff u, v, p, qx = upv and y = uqv and (p q) P is a grammar with N =... and... A grammar describes a language (the set of words on Σ that can be derived from the start symbol). Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 15 / 22 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 16 / 22
Grammars Grammars Examples Context-free grammars S asb S ε The grammar defines the language {a n b n, n N} S absc S abc Context-free grammar A CF-grammar is a grammar where all production rules are of the form N (Σ N). Ba ab Bb bb The grammar defines the language {a n b n c n, n N} Exercises Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 17 / 22 Grammars Example of CF-grammar Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 18 / 22 Grammars Relationship between stack automata and grammars S S + S S S a The grammar defines a language of arithmetical expressions. Notion of derivation tree. Draw a derivation tree of a*a+a, of S+S! General result The context-free/algebraic languages are exactly the languages that are described by stack automata. The proof is not difficult. Exercises Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 19 / 22 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 20 / 22
Grammars Conclusion Some other results/definitions Summary Regular languages are algebraic languages, but not the converse. There exists normal forms for algebraic grammars A grammar can be ambiguous. Stack Automata or PushDown Automata are : Acceptors for context-free languages. But some languages are not context free. Non deterministic. http://en.wikipedia.org/wiki/pushdown_automaton for useful pointers Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 21 / 22 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 - S8SC 22 / 22
Laure Gonnord Polycopié d'informatique Fondamentale IMA4SC S8 2013 Chapitre 3 Automates à compteurs et machines de turing Dans ce chapitre nous étudions des modèles plus proches de nos programmes habituels, les automates à compteurs, et introduisons le modèle couramment utilisé pour formaliser la notion de calcul, les machines de Turing. 22/58
Informatique Fondamentale IMA S8 Cours 3: Counter automata, Turing Machines and decidable problems Laure Gonnord http://laure.gonnord.org/pro/teaching/ Laure.Gonnord@polytech-lille.fr III - Counter automata, Turing Machines and decidable problems Université Lille 1 - Polytech Lille 2013 Counter automata Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 2 / 30 Counter automata What for? 1 Counter automata 2 Programs 3 Turing Machines Express more than context-free languages! Give a way for automata to count more : include variables. 4 Decidability - Complexity Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 3 / 30 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 4 / 30
Counter automata Counter automata Example General definition cos(y) x x := x+1 k init k 1 x y y := y x true y := y +2 Counter Automata A finite number of counters (variables) A finite number of control points Transitions between them that operate on counters. The transition function is of the form g a (g is the guard, a is the action) Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 5 / 30 How it works? 1/3 Counter automata Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 6 / 30 How it works? 2/3 Counter automata State A state is (q, σ) q is the current control point σ : V ar V al is a function that assigns a value (a real one or ) to all counters. Non deterministic/no notion of acceptance/notion of reachability. Given a state (q, σ), the next one is computed by the help of the transition function : enabled transitions are transition of the current control points where the current valuations of variables satisfy the guard. Pick one enabled transition, and compute the next state : (q, σ ). The new values for the variables are computed w.r.t. the action. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 7 / 30 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 8 / 30
Counter automata Counter automata How it works? 3/3 Example of an affine automata : y +2x 2 x := 2x+5 true x := 0;y := 0 x 1 x := x+1;y := 2y 4 k init k 1 k 2 What kind of systems? These automata are used to encode, for example : Simple systems (coffee machine,...) specifications Energy consumption of sensors : 400 µs 145.8 mw Transmit 145.8 mw Compute successive states. Exercises. Sleep 35.1 mw 332 µs 140.4 mw Idle 140.4 mw 144 µs 140.4 mw 144 µs 140.4 mw 100 µs 100 µs 140.4 mw 140.4 mw Receive 140.4 mw Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 9 / 30 Counter automata Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 10 / 30 Programs What for? 1 Counter automata Some classical problems : Automatically finding invariants Reachability analysis Formula proving. (deterministic) Code generation. 2 Programs 3 Turing Machines 4 Decidability - Complexity Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 11 / 30 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 12 / 30
Programs Expressing programs as counter automata An example : x:=0;y:=0 while (x<=100) do read(b); if b then x:=x+2 else begin x:=x+1; y:=y+1; end; endif endwhile x 100 x := x+1 y := y +1 p in p p out (x,y) := (0,0) x 100 Some approximations are made (arrays, Boolean,... ) x 100 x := x+2 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 13 / 30 Programs Expressing properties of programs - 2 Automatically deriving bad states : Programs Expressing properties of programs Some (safety) properties of programs can be expressed inside the counter automaton : x 100 x := x+1 y := y +1 p in p p out (x,y) := (0,0) x 100 x 100 x := x+2 Encode the fact that state = p out y > 100 is bad Some modifications of the automaton can be automatically done. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 14 / 30 Programs Expressing programs as counter automata int j; char user[usersz]; for(j = 0; line[j]!= EOS; ++j) if (!strchr("-", line[j])) break; if(j == J && line[j] == ' ') { /* long list */ /* BUG! No bounds check. */ assert(usersz>=n-j,"badstate"); Pros : finding and proving program properties Cons : how do we define the operations? the fact that integers are encoded on bits? There is a need for a more accurate (not specialised) model! } r_strcpy (user, line + j); badstate is reachable. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 15 / 30 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 16 / 30
Turing Machines Turing Machines Turing Machine, def - 1 1 Counter automata 2 Programs 3 Turing Machines 4 Decidability - Complexity Turing Machine, Turing, 1935 A tape (infinite in both sides) : the memory. (it has a working alphabet which contains a blank symbol). A head : read/write symbols on the tape. A finite transition table (or function) A PushDown automaton which is more flexible. image credits : Wikipedia.org Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 17 / 30 Turing Machines Turing Machine, def - 2 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 18 / 30 An Example Turing Machines Turing Machine elements States (Q), initial state (q 0 ), final (accepting) states (F). One alphabet Γ for the tape. b Γ the blank letter. Transitions are finitely many : read the letter under the head, and then : Erase a symbol or write one under the head Move the head (read, write, or stays) The state can be modified. The transition function is : TM that decides if x is odd : (final State : q 4 ) State/char 0 1 B q 1 (q 1,0,R) (q 1,1,R) (q 2,B,L) q 2 (q 3,0,L) (q 4,1,L) Play on BBBBBBB11BBBBB and BBBBBBB10BBBBB A configuration is a tuple (word on the tape, position of the head, state). Adapted from : http://www.madchat.fr/coding/algo/algo_epfl.pdf slide 4 Q Γ Q Γ {L, R, S} Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 19 / 30 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 20 / 30
Turing Machines Turing Machines Another Example Demo of a TM recognising a language : a n b n c n Program found here : http://www.cs.columbia.edu/~zeph/software/bjdweck/ General results 1/2 There exists Turing machines for the following languages : palindromes a n b n c n (non algebraic language) a i with i prime (non algebraic) a n2, with n 0 Turing machines are more powerful than all other models (we have seen yet) Decidable Languages A language that is recognised by a Turing Machine is said to be decidable. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 21 / 30 Turing Machines Accepting or computing Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 22 / 30 Another Example Turing Machines A TM can also compute functions TM that writes 1 if x is odd, 0 else (q 6 is final state) : State/char 0 1 B q 1 (q 1,0,R) (q 1,1,R) (q 2,B,L) q 2 (q 3,B,L) (q 4,B,L) q 3 (q 3,B,L) (q 3,B,L) (q 5,1,R) q 4 (q 4,B,L) (q 4,B,L) (q 5,0,R) q 5 (q 6,B,R) Demo of a TM computing the subtraction. Play on BBBBBBB11BBBBB and BBBBBBB10BBBBB Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 23 / 30 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 24 / 30
Turing Machines Decidability - Complexity General results 2/2 There exists Turing machines for the computation of : x x + 1 (x, y) x + y x x 2 all the functions you are able to write on computers Computable functions A function (defined for all its input) that is computable by a Turing Machine is said to be TM computable 1 Counter automata 2 Programs 3 Turing Machines 4 Decidability - Complexity A Model of what can be computed with machines. (Church Thesis) Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 25 / 30 Decidability - Complexity Decidable - Semi-decidable languages Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 26 / 30 Complexity Decidability - Complexity If L is a language, L is decidable if there exists a Turing Machine (or an algorithm) that outputs for all w : 1 if w L else 0. Semi-decidable : 1 if w L else does not terminate equivalent definition for problems. Link with computational complexity : The number of steps in the execution of a TM gives the complexity in time, The number of seen squares in the execution of a TM gives the complexity in space. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 27 / 30 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 28 / 30
Decidability - Complexity Examples of undecidable problems The halting problem for TM or counter automata (for more that 2 counters) Given a program, does it loops? Is a given algebraic expression (with log, *, exp, sin, abs) equal to 0? (Richardson, 1968) The 10 th Hilbert Problem (Diophantine equations) Summary Turing Machines : Conclusion A model closed to programming languages. Non deterministic. Acceptors for decidable languages. But some languages are not decidable! (equivalently) Computes solutions to problems. But... Important fact Some common problems are undecidable! Counter automata : are a more simpler model have the same power of expression as Turing Machines. Reachability is undecidable too. But we can do approximations. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 29 / 30 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8SC 30 / 30
Laure Gonnord Polycopié d'informatique Fondamentale IMA4SC S8 2013 Chapitre 4 Graphes, notions de NP-complétude Les graphes orent une modélisation intéressante de beaucoup de problèmes d'informatique. Nous abordons leur représentation machine et présentons rapidement les algorithmes classiques. Nous protons du modèle pour parler de complexité théorique, de problèmes diciles, et de NP-complétude Figure 4.1 Pillow talk, http://xkcd.com/69/, sous License Creative Commons Figure 4.2 Travelling salesman Problem, http://xkcd.com/399/, sous License Creative Commons 31/58
Informatique Fondamentale IMA S8 Cours 4 : graphs, problems and algorithms on graphs, (notions of) NP completeness Laure Gonnord http://laure.gonnord.org/pro/teaching/ Laure.Gonnord@polytech-lille.fr IV - Graphs, problems and algorithms on graphs, NP-completeness Université Lille 1 - Polytech Lille 2013 Some generalities on graphs 1 Some generalities on graphs Generalities and first definitions Internal representation 2 Paths finding - search, and applications 3 Algorithms specific to oriented graphs 4 More difficult problems on graphs 5 NP - completeness Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 2 / 64 Goal Model : Some generalities on graphs Generalities and first definitions relationships between objects ( I know him, This register is in conflict with with one, there is a road between these two towns ) classical problems of paths ( Do I have a friend that knows a friend... who knows the Dalai-Lama?, It is possible to go from this town to this another in less than 600 km?, how to get out of the labyrinth? ) 6 Docs Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 3 / 64 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 4 / 64
Some generalities on graphs Generalities and first definitions Some generalities on graphs Generalities and first definitions Who? Other classical problems Euler (1740) formalised the theory and the Konisberg problem : Problem : compute an eulerian path. Source (fr) : http://fr.wikipedia.org/wiki/th%c3%a9orie_des_graphes The travelling salesman problem : TSP Task scheduling (on parallel machines or not!), the PERT method. Coloring maps Covering (telecom) trees Minimal paths... Graphs are everywhere. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 5 / 64 Some generalities on graphs Generalities and first definitions General definition Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 6 / 64 Some generalities on graphs Generalities and first definitions Other definitions Graph A finite set of vertices (a vertex) : V A finite set of edges. E V V v 1 e 1 e 3 v3 Non Oriented Graph iff (v1, v2), (v1, v2) E (v2, v1) E. Tree (non oriented acyclic connex) - Forest. Directed Acyclic Graph : DAG. Oriented Graph with no cycle. trees are dags! v1 v3 v2 v2 v4 A simplier model than the previous ones! Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 7 / 64 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 8 / 64
Some generalities on graphs Internal representation Some generalities on graphs Internal representation Sources Adjacency matrix Picture and Java codes from http://pauillac.inria.fr/~levy/courses/x/if/a1/a1.html 3 4 0 2 0 1 0 0 0 1 0 1 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 1 0 1 If the graph is oriented, then the matrix is NOT symmetric Memory complexity : O(V 2 ) Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 9 / 64 Some generalities on graphs Internal representation Adjacency matrix - Java Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 10 / 64 Some generalities on graphs Internal representation Array of lists boolean [ ] [ ] m; Graph ( i n t n ) { m = new boolean [ n ] [ n ] ; for ( i n t i = 0; i < n ; ++ i ) for ( i n t j = 0; j < n ; ++ j ) m[ x ] [ y ] = f a l s e ; } 3 4 0 1 2 s t a t i c void addedge ( Graph g, i n t x, i n t y ) { g.m[ x ] [ y ] = t r u e ; g.m[ y ] [ x ] = t r u e / / only i f non o r i e n t e d } Memory complexity : O(V + E) Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 11 / 64 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 12 / 64
Array of lists - Java Some generalities on graphs Internal representation Paths finding - search, and applications 1 Some generalities on graphs } L i s t e [ ] succ ; Graph ( i n t n ) { succ = new L i s t e [ n ] ; } s t a t i c void addedge ( Graph g, i n t x, i n t y ) { g. succ [ x ] = new L i s t e ( y, g. succ [ x ] ) ; / / i f non o r i e n t e d also modify g. succ [ y ]! } 2 Paths finding - search, and applications Looking for all nodes Connexity Topological Sort 3 Algorithms specific to oriented graphs 4 More difficult problems on graphs 5 NP - completeness 6 Docs Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 13 / 64 Paths finding - search, and applications Looking for all nodes Search Algorithms Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 14 / 64 Paths finding - search, and applications Looking for all nodes Deep First Search on Graphs! Translation problem! (en) Search Algorithms - (fr) Parcours de Graphes Reminder : search/do algorithm on a binary tree : On graphs, there can be cycles! 1 If the tree is not empty, do f(root). 2 recursively apply the function on the left sub-tree 3 recursively apply the function on the right sub-tree Deep-First Search Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 15 / 64 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 16 / 64
Paths finding - search, and applications Looking for all nodes Paths finding - search, and applications Looking for all nodes Deep First Search on Graphs - Java Breadth First Search on Graphs s t a t i c i n t [ ] num; / / numbering the edges according to the v i s i t i n g order s t a t i c void v i s i t e r ( Graphe g ) { i n t n = g. succ. length ; num = new i n t [ n ] ; numordre = 1; for ( i n t x = 0; x < n ; ++x ) num[ x ] = 1; / / 1 = not seen for ( i n t x = 0; x < n ; ++x ) i f (num[ x ] == 1) dfs ( g, x ) ; } s t a t i c void dfs ( Graphe g, i n t x ) { num [ x ] = ++numordre ; for ( L i s t e l s = g. succ [ x ] ; l s!= n u l l ; l s = l s. s u i v a n t ) { i n t y = l s. v a l ; i f (num[ y ] == 1) dfs ( g, y ) ; } } Complexity = O(V + E). Tarjan [1972] Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 17 / 64 Conventions Paths finding - search, and applications Looking for all nodes BLANC = not (yet) processed NOIR = has been processed GRIS = being processed f i n a l s t a t i c i n t BLANC = 0, GRIS = 1, NOIR = 2; s t a t i c i n t [ ] couleur ; Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 18 / 64 Paths finding - search, and applications Looking for all nodes Breadth First Search on Graphs - Java i n t n = g. succ. length ; couleur = new i n t [ n ] ; FIFO f = new FIFO ( n ) ; for ( i n t x = 0; x < n ; ++x ) couleur [ x ] = BLANC; for ( i n t x = 0; x < n ; ++x ) i f ( couleur [ x ] == BLANC) { FIFO. a j o u t e r ( f, x ) ; couleur [ x ] = GRIS ; bfs ( g, f ) ; } s t a t i c void bfs ( Graphe g, FIFO f ) { while (! f. vide ( ) ) { i n t x = FIFO. supprimer ( f ) ; for ( L i s t e l s = g. succ [ x ] ; l s!= n u l l ; l s = l s. s u i v a n t ) { i n t y = l s. v a l ; i f ( couleur [ y ] == BLANC) { FIFO. a j o u t e r ( f, y ) ; couleur [ y ] = GRIS ; } couleur [ x ] = NOIR ; } } } Complexity? Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 19 / 64 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 20 / 64
Paths finding - search, and applications Looking for all nodes Paths finding - search, and applications Connexity Application : Labyrinth Finds exit, ie a path from d to s? } couleur [ d ] = GRIS ; i f ( d == s ) return new L i s t e ( d, n u l l ) ; for ( L i s t e l s = g. succ [ d ] ; l s!= n u l l ; l s = l s. s u i v a n t ) { i n t x = l s. v a l ; i f (num[ x ] == BLANC) { L i s t e r = chemin ( g, x, s ) ; i f ( r!= n u l l ) return new L i s t e ( d, r ) ; } } return n u l l ; Definition - Connex Component Connex component A connex component is a maximal set of connected vertices. Exercise : Algorithm for computing CCs. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 21 / 64 Paths finding - search, and applications Connexity Definition - Connex Graph Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 22 / 64 Paths finding - search, and applications Topological Sort Other Algorithms There exists variants of DFS algo to : Compute Cover Forets (in bold in the picture) : Connex graph A graph is connex if it has only one connex component. Exercise : Algorithm that tests the connexity of a graph. Detect Cycles. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 23 / 64 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 24 / 64
Algorithms specific to oriented graphs Algorithms specific to oriented graphs Topological sorting of a DAG 1 Some generalities on graphs 2 Paths finding - search, and applications 3 Algorithms specific to oriented graphs 4 More difficult problems on graphs 5 NP - completeness 6 Docs Give a list of vertices in an order compatible with the DAG order : v i < v j v j v i (total order from a partial header). Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 25 / 64 Algorithms specific to oriented graphs Topological sorting of a DAG - Example Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 26 / 64 Algorithms specific to oriented graphs Topological sorting of a DAG - implementation 1 4 6 2 5 7 3 If we know an entrant node, a dfs gives the result. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 27 / 64 f i n a l s t a t i c i n t BLANC = 0, GRIS = 1, NOIR = 2; s t a t i c L i s t e afaire ( Graphe g, i n t x ) { i n t n = g. succ. length ; i n t [ ] couleur = new i n t [ n ] ; for ( i n t x = 0; x < n ; ++x ) couleur [ x ] = BLANC; return dfs ( g, x, n u l l, couleur ) ; } s t a t i c L i s t e dfs ( Graphe g, i n t x, L i s t e r, i n t [ ] couleur ) { couleur [ x ] = GRIS ; for ( L i s t e l s = g. succ [ x ] ; l s!= n u l l ; l s = l s. s u i v a n t ) { i n t y = l s. v a l ; i f ( couleur [ y ] == BLANC) r = dfs ( g, y, r, couleur ) ; } return new L i s t e ( x, r ) ; } dfs here returns a list. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 28 / 64
Algorithms specific to oriented graphs Topological sorting of a DAG - Applications Transitive closure Algorithms specific to oriented graphs instruction scheduling cell evaluation in spreadsheets order of compilation tasks (Makefiles) symbol dependencies in linkers sheduling in project managements (PERT, 1960) http://en.wikipedia.org/wiki/topological_sorting The graph G + = (V, E + ) of the transitive closure of G = (V, E) is such that E + is the smallest set verifying : E E + (x, y) E + and (y, z) E + (x, z) E + Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 29 / 64 Algorithms specific to oriented graphs Algorithm for Transitive closure Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 30 / 64 Algorithms specific to oriented graphs Definition of oriented valuated graph Valuated graph It s a graph with a function d : V V N called distance. If E is the adjacency matrix : E + = E + E 2 +... E n 1. An algorithm in O(n 4 ). Is it possible to have a better complexity? Yes (Warshall, in O(n 3 ), see later). G is represented by an adjacency matrix with integer values, or + if there is no edge. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 31 / 64 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 32 / 64
Algorithms specific to oriented graphs The shortest path problem Algorithms specific to oriented graphs Multiple Source Shortest Path with dynamic programming -1 Given G, find a path whose sum of distances is minimal! Between all the vertices! Floyd-Warshall algorithm Principle : Let d k x,y be the minimal distance between x and y passing through vertices whose number is < k. Then d 0 x,y = d x,y and d k x,y = min{d k 1 x,y, d k 1 x,k + dk 1 k,y } more variants on http://en.wikipedia.org/wiki/shortest_path_problem Store all these temporary distances. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 33 / 64 Algorithms specific to oriented graphs Multiple Source Shortest Path with dynamic programming -2 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 34 / 64 Algorithms specific to oriented graphs Simple Source Shortest Path s t a t i c Graphe pluscourtschemins ( Graphe g ) { i n t n = g. d. length ; Graphe gplus = copiegraphe ( g ) ; for ( i n t k = 0; k < n ; ++k ) for ( i n t x = 0; x < n ; ++x ) for ( i n t y = 0; y < n ; ++y ) gplus. d [ x ] [ y ] = Math. min ( gplus. d [ x ] [ y ], gplus. d [ x ] [ k ] + gplus. d [ k ] [ y ] ) ; return gplus ; } All d are > 0 : Dijkstra, 1959 Negative distances : Bellman Ford 1960 Routing algorithms (Bellman uses only local information). Complexity O(n 3 ), space O(n 2 ). Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 35 / 64 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 36 / 64
More difficult problems on graphs More difficult problems on graphs Eulerian and Hamiltonian paths 1 Some generalities on graphs 2 Paths finding - search, and applications 3 Algorithms specific to oriented graphs 4 More difficult problems on graphs Eulerian/Hamiltonian Path An Eulerian path : each edge is seen only once. An Hamiltonien path : each node... 5 NP - completeness 6 Docs Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 37 / 64 More difficult problems on graphs Hamiltonian Tour, one solution Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 38 / 64 More difficult problems on graphs Application 1 : Knight s Tour Extend a path until not possible Backtrack one time, and try with another neighbour And so on. Worst case complexity time : exponential in n Is it possible to make a Knight s tour? «The knight is placed on the empty board and, moving according to the rules of chess, must visit each square exactly once.» A variant of the Hamiltonian tour that can be solved in polynomial time. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 39 / 64 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 40 / 64
More difficult problems on graphs The Travelling Salesman Problem More difficult problems on graphs Travelling Salesman Problem - 2 A (non-polynomial) algorithm for this problem : Transform into a linear programming problem with integers Solve it with the simplex! This algorithm is exponential in the size of the graph with valuated (non oriented) graph. Find an hamiltonian cycle of minimum weight Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 41 / 64 More difficult problems on graphs Graph Coloring Problem Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 42 / 64 More difficult problems on graphs Graph Coloring Problem - 2 Are you able to design a polynomial algorithm? Application to the register allocation in compilers. The sudoku problem (9-coloring of a 81-vertices graph) Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 43 / 64 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 44 / 64
More difficult problems on graphs NP - completeness Graph Coloring Problem - 3 We do not know any polynomial algorithm for this problem. 1 Some generalities on graphs 2 Paths finding - search, and applications 3 Algorithms specific to oriented graphs 4 More difficult problems on graphs 5 NP - completeness 6 Docs Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 45 / 64 Complexity NP - completeness Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 46 / 64 P Problems NP - completeness The complexity of an implementation / an algorithm is linked to Turing machines : The number of steps in the execution of a TM gives the complexity in time, The number of seen squares in the execution of a TM gives the complexity in space. We can replace the TM by a C implementation. P A problem (or a language) belongs to P if there exists a deterministic Turing Machine that gives the result in polynomial time. there is a polynom p such that for all entry x, the Turing machine stops (with the good result) in less than p( x ) steps. Example : L = a n b n c n, the TM of the previous course checks any a i b i c i in O(n) steps Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 47 / 64 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 48 / 64
NP - completeness NP - completeness P Problems NP Problems Integer multiplication Eulerian tour (each edge once) Linear programming (recall that polynomial diophantine equations are undecidible.) Primality test http://www.cs.uu.nl/groups/ad/compl-diaz.pdf NP A problem (or a language) belongs to NP if there exists a deterministic Turing Machine that checks the result in polynomial time. Example : there exists a TM that is able to check if a given path is an hamiltonien tour, in polynomial time. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 49 / 64 NP - completeness NP Problems - Examples Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 50 / 64 NP vs P - some facts NP - completeness Factoring 3-SAT (formulae in CNF with 3 litterals on each term) Hamiltonian tour 3-colorability http://www.cs.uu.nl/groups/ad/compl-diaz.pdf If a problem belongs to NP, then there exists an exponential brute-force deterministic algorithm to solve it (easy to prove) P NP (trivial) P = NP? P NP? Open question! Problem of the millenium! Is finding more difficult that verifying? Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 51 / 64 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 52 / 64
NP - completeness NP-complete problems - 1 NP - completeness NP-complete problems - 2 Given a problem : if we find a polynomial algorithm P if we find a polynomial time checking algorithm NP, but we want to know more : it it really hard to solve it? is it worth looking for a better algorithm? The key notion is the notion of polynomial reduction Polynomial reduction of problems Given two problems P 1 and P 2, P 1 reduces polynomially to P 2 if there exists an f such that : f is polynomial x 1 solution of P 1 iff f(x 1 ) solution of P 2. Example : Hamiltonian circuit is polynomially reductible to TSP. Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 53 / 64 NP - completeness NP-complete problems - 3 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 54 / 64 NP - completeness NP-complete problems - 4 The most difficult problems in NP. All problems in NPC are computationally equivalent in the sense that, if one problem is easy, all the problems in the class are easy. Therefore, if one problem in NPC turns out to be in P, then P=NP Reminder : we want to characterise the most difficult problems in NP. NP-complete A problem is said to be NP-complete if : It belongs to NP All other problems in NP reduce polynomially to it. In other words, if we solve any NP-complete in polynomial time, we have proved P = NP (and we become rich) Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 55 / 64 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 56 / 64
NP - completeness NP-complete problems - 4 NP - completeness NP-complete problems - The historical example Problem : how to prove that a problem is NP-complete? The very first one is hard to prove Then, we use the following result : If two problems / languages L 1 and L 2 belong to NP and L 1 is NP-complete, and L 1 reduces polynomially to L 2, then L 2 is NP-complete. Pick one NP complete problem and try to reduce it to your problem. SATISFIABILITY is NP-complete (Cook, 1971) SAT reduces polynomially to 3SAT 3SAT reduces polynomially to Vertex Cover Vertex Cover reduces polynomially to Hamiltonian circuit So Hamiltonian circuit, then TSP are also NP-complete problems Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 57 / 64 NP - completeness Some NP-complete (common) problems Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 58 / 64 Conclusion - 1 NP - completeness Some classical examples : Hamiltonian cycle and TSP Graph Coloring Problem Vertex cover Knapsack (integer) Flow shop problem Sudoku The book : Computers and Intractability : A Guide to the Theory of NP-Completeness. A short list on the french webpage : http://fr.wikipedia.org/wiki/liste_de_probl%c3%a8mes_ NP-complets Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 59 / 64 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 60 / 64
NP - completeness Docs Conclusion - 2 1 Some generalities on graphs 2 Paths finding - search, and applications Knowing that a given problem is NP complete is just the beginning : handle less general classes of inputs compute less precise solutions 3 Algorithms specific to oriented graphs 4 More difficult problems on graphs 5 NP - completeness 6 Docs Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 61 / 64 More docs on graphs Docs Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 62 / 64 More docs on complexity Docs In french : http://laure.gonnord.org/site-ens/mim/ graphes/cours/cours_graphes.pdf or type cours de Théorie des graphes in Google In english : an interactive tutorial here : http://primes. utm.edu/cgi-bin/caldwell/tutor/graph/intro. (en) The theorical book Graph Theory, R. Diestel (electronic version : http://diestel-graph-theory.com/basic.html) The Book : computers and intractability, a guide to the theory of NP completeness, by Garey/Johnson (en) e-book for algorithms : http: //code.google.com/p/graph-theory-algorithms-book/ Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 63 / 64 Laure Gonnord (Lille1/Polytech) Informatique Fondamentale IMA S8 2013 S8-SC 64 / 64