Complexity 1: Motivation. Outline. Dusko Pavlovic. Introductions. Algorithms. Introductions. Complexity 1: Motivation. Contact.

Transcription

1 Outline Complexity Theory Part 1: Defining complexity Complexity of algorithms RHUL Spring 2012 Outline Contact Defining complexity phone: ( ) Complexity of algorithms office: 227 office hours: W 15-17, Θ 16-17, appointment Books Course Sanjeev Arora and Boaz Barak, Computational Complexity. Cambridge University Press (2009) What do we expect from the course? Christos Papadimitriou, Computational Complexity. Addison-Wesley (1994) Steven Rudich and Avi Vidgerson (eds.), Computational Complexity Theory. American Mathematical Society (2000)

2 Course Course What do we expect from the course? What do we expect from the course? Why are we interested in complexity? Why are we interested in complexity? Outline Why are there complex phenomena? Defining complexity Complexity of algorithms If the initial conditions are simple... Why are there complex phenomena? Why are there complex phenomena?...wheredoesthecomplexitycomefrom? Why does complexity increase and spread?

3 Why are there complex phenomena? Complexity is easily recognized Why are we so complex? Astraightlineissimple. Complexity is easily recognized Complexity is easily recognized Acurvedlineislesssimple. Alineofthehorizoniscomplex. Which strings are complex? How do you know which strings are complex? digits digits digits digits digits digits

4 (01) (01) 50 do i=1..50 write 01 od (01) (01) 50 do i=1..50 write 01 od do i=1..50 write 01 od i 1 i i=1; do until length= write 0 i 1 i ;i=i+1od i 1 i i=1; do until length= write 0 i 1 i ;i=i+1od print Idea Fix a programming language and define CX(x) = length of the shortest program that outputs x CX( ) < CX(010...) < CX( ) where x any bitstring.

5 Conclusion Conclusion Complexity is relative to our capability Complexity is relative to our capability to model, predict, compute... to model, predict, compute... Complexity is studied as computational complexity Outline Example 1: Defining complexity Complexity of algorithms Task Given a directed graph Γ and the nodes a, b in it, determine whether there is a path a b. Example 1: Example 1: Formalize directed graphs Adirectedgraphisapairoffinitesets and a pair of functions between them: E δ ϱ V

6 Example 1: Algorithm Example 1: Algorithm Set R i = S i = v V j i. a 1 2 j v v V a 1 2 i v no shorter path Set R i = S i = v V j i. a 1 2 j v v V a 1 2 i v no shorter path Compute S 1 = ϱδ 1 a R 1 = S 1 S i+1 = ϱδ 1 S i \ R i R i+1 = R i S i+1 Example 1: Algorithm Set R i = v V j i. a 1 2 j v S i = v V a 1 2 i v no shorter path Example 1: Analysis The algorithm spans the maximal subtree rooted in a. Compute S 1 = ϱδ 1 a R 1 = S 1 S i+1 = ϱδ 1 S i \ R i R i+1 = R i S i+1 At each step check if b S i. Example 1: Example 1: Analysis Analysis The algorithm spans the maximal subtree rooted in a. The algorithm spans the maximal subtree rooted in a. Atreewithn vertices has n 1edges. Atreewithn vertices has n 1edges. If the graph has n vertices, then at each step at most n 1edgesmaybetested; the search must halt after at most n 1steps.

7 Example 1: Digression: O-notation Definition 1 Complexity For f, g : N N write f (n) O (g(n)) whenever c n 0 n. n 0 n = f (n) c g(n) #operations = #tests #steps (n 1) 2 = O(n 2 ) Digression: O-notation Definition 1 Example 1: For f, g : N N write f (n) O (g(n)) whenever c n 0 n. n 0 n = f (n) c g(n) Complexity Examples CX() O(n 2 ) (n 1) 2 = O(n 2 ) n 2 = O ( (n 1) 2) 5 n = n 2 2(n 1) 2 Example 2: Example 2: Task Formalize road maps AroadmapisadirectedN-labelled graph Given a road map, find the shortest tour through all cities. N λ E δ ϱ V where the labels λ(e) represent distances.

8 Example 2: Example 2: Analysis Algorithm List all possible tours. If there is a road between every two cities, then there are n! tours between n cities. If all distances are different, then all n! tours must be computed and compared. Example 2: Example 2: Remark. Complexity We don t know a better way to do this with a computer. CX() O(n!) Ants do it in O(n) operations, using pheromones and parallel search. Example 2: Example 3: Remark. Fact Some programs always halt We don t know a better way to do this with a computer. Ants do it in O(n) operations, using pheromones and parallel search. Computational complexity is relative to computational resources.

9 Example 3: Example 3: Fact Fact Some programs always halt: Some programs always halt: (i) m(x, y) =x y (ii) s(x) =m(x, x) (iii) t(x) =s(x)+1 (i) m(x, y) =x y (ii) s(x) =m(x, x) (iii) t(x) =s(x)+1 Some programs do not always halt: (1) u(x, y) =x(y) (2) v(x) =u(x, x) (3) w(x) =v(x)+1 Example 3: Example 3: Fact Some programs always halt: (i) m(x, y) =x y (ii) s(x) =m(x, x) (iii) t(x) =s(x)+1 Some programs do not always halt: Task Determine which programs halt for which inputs. (1) u(x, y) =x(y) (2) v(x) =u(x, x) (3) w(x) =v(x)+1 because v(w) (2) = u(w, w) (1) = w(w) (3) = v(w)+1 Example 3: Example 3: Proof. Suppose that has a solution, and set Claim (1) h(x, y) z. x(y) =z (2) k(x) ="[B] if h(x,x) then go to B else 0" There is no algorithm solving.

10 Example 3: Proof. Example 3: Proof. Suppose that has a solution, and set Suppose that has a solution, and set (1) h(x, y) z. x(y) =z (1) h(x, y) z. x(y) =z (2) k(x) ="[B] if h(x,x) then go to B else 0" (2) k(x) ="[B] if h(x,x) then go to B else 0" Then for any x holds h(k, x) (1) z.k(x) =z (2) h(x, x) Then for any x holds h(k, x) (1) z.k(x) =z (2) h(x, x) Hence for x = k holds h(k, k) h(k, k) Example 3: There are problems of different complexities Complexity CX() O(n 2 ) CX( ) = CX() O(n!) CX( ) = There are problems of different complexities CX() O(n 2 ) CX() O(n!) CX( ) = Can we tell more?