Algorithms, CSE, OSU. Introduction, complexity of algorithms, asymptotic growth of functions. Instructor: Anastasios Sidiropoulos

Transcription

1 Algorithms, CSE, OSU Introduction, complexity of algorithms, asymptotic growth of functions Instructor: Anastasios Sidiropoulos

2 Why algorithms? Algorithms are at the core of Computer Science

3 Why algorithms? Algorithms are at the core of Computer Science Data bases: Data structures.

4 Why algorithms? Algorithms are at the core of Computer Science Data bases: Data structures. Networks: Routing / communication algorithms.

5 Why algorithms? Algorithms are at the core of Computer Science Data bases: Data structures. Networks: Routing / communication algorithms. Operating systems: Scheduling, paging, etc.

6 Why algorithms? Algorithms are at the core of Computer Science Data bases: Data structures. Networks: Routing / communication algorithms. Operating systems: Scheduling, paging, etc. AI: Learning algorithms, etc.

7 Why algorithms? Algorithms are at the core of Computer Science Data bases: Data structures. Networks: Routing / communication algorithms. Operating systems: Scheduling, paging, etc. AI: Learning algorithms, etc. Graphics: Rendering algorithms.

8 Why algorithms? Algorithms are at the core of Computer Science Data bases: Data structures. Networks: Routing / communication algorithms. Operating systems: Scheduling, paging, etc. AI: Learning algorithms, etc. Graphics: Rendering algorithms. Robotics: Motion planning / control algorithms, etc.

9 Why algorithms? Algorithms are at the core of Computer Science Data bases: Data structures. Networks: Routing / communication algorithms. Operating systems: Scheduling, paging, etc. AI: Learning algorithms, etc. Graphics: Rendering algorithms. Robotics: Motion planning / control algorithms, etc. Game development

10 Algorithms beyond Computer Science Algorithms are a transformative force in Science & Engineering.

11 Algorithms beyond Computer Science Algorithms are a transformative force in Science & Engineering. Algorithms for processing complicated data.

12 Algorithms beyond Computer Science Algorithms are a transformative force in Science & Engineering. Algorithms for processing complicated data. Computational Biology.

13 Algorithms beyond Computer Science Algorithms are a transformative force in Science & Engineering. Algorithms for processing complicated data. Computational Biology. Medicine: drug design / delivery.

14 Algorithms beyond Computer Science Algorithms are a transformative force in Science & Engineering. Algorithms for processing complicated data. Computational Biology. Medicine: drug design / delivery. Sociology / Economics: Algorithmic game theory.

15 Algorithms beyond Computer Science Algorithms are a transformative force in Science & Engineering. Algorithms for processing complicated data. Computational Biology. Medicine: drug design / delivery. Sociology / Economics: Algorithmic game theory....

16 What makes a good algorithm? How can we quantify the performance of an algorithm?

17 What makes a good algorithm? How can we quantify the performance of an algorithm? Computational resources Time complexity: How much time does an algorithm need to terminate?

18 What makes a good algorithm? How can we quantify the performance of an algorithm? Computational resources Time complexity: How much time does an algorithm need to terminate? Space complexity: How much memory does an algorithm require?

19 What makes a good algorithm? How can we quantify the performance of an algorithm? Computational resources Time complexity: How much time does an algorithm need to terminate? Space complexity: How much memory does an algorithm require? In other contexts, we might also be interested in different parameters.

20 What makes a good algorithm? How can we quantify the performance of an algorithm? Computational resources Time complexity: How much time does an algorithm need to terminate? Space complexity: How much memory does an algorithm require? In other contexts, we might also be interested in different parameters. Communication complexity (i.e. the total amount of bits exchanged in a system).

21 What makes a good algorithm? How can we quantify the performance of an algorithm? Computational resources Time complexity: How much time does an algorithm need to terminate? Space complexity: How much memory does an algorithm require? In other contexts, we might also be interested in different parameters. Communication complexity (i.e. the total amount of bits exchanged in a system). Waiting / service time (e.g. in queuing systems).

22 Worst-case complexity The worst-case time complexity, or worst-case running time of an algorithm is a function f : N N, where f (n) = maximum # of steps required on any input of size n

23 Worst-case complexity The worst-case time complexity, or worst-case running time of an algorithm is a function f : N N, where f (n) = maximum # of steps required on any input of size n More precisely: For any input x {0, 1} n, let T (x) = # of steps required on input x

24 Worst-case complexity The worst-case time complexity, or worst-case running time of an algorithm is a function f : N N, where f (n) = maximum # of steps required on any input of size n More precisely: For any input x {0, 1} n, let T (x) = # of steps required on input x Then, f (n) = max T (x) x {0,1} n

25 Example of worst-case complexity Finding an element in an array.

26 Example of worst-case complexity Finding an element in an array. Input: integer array A[1... n], and integer x.

27 Example of worst-case complexity Finding an element in an array. Input: integer array A[1... n], and integer x. Find some i, if one exists, such that A[i] = x.

28 Example of worst-case complexity Finding an element in an array. Input: integer array A[1... n], and integer x. Find some i, if one exists, such that A[i] = x. Algorithm for i = 1 to n if A[i] = x output i, and terminate end output not found

29 Example of worst-case complexity Finding an element in an array. Input: integer array A[1... n], and integer x. Find some i, if one exists, such that A[i] = x. Algorithm for i = 1 to n if A[i] = x output i, and terminate end output not found What is the worst-case time complexity of this algorithm?

30 Example of worst-case complexity Finding an element in an array. Input: integer array A[1... n], and integer x. Find some i, if one exists, such that A[i] = x. Algorithm for i = 1 to n if A[i] = x output i, and terminate end output not found What is the worst-case time complexity of this algorithm? What is the best possible time complexity?

31 How do we compare different functions? We will mostly deal with non-decreasing functions.

32 How do we compare different functions? We will mostly deal with non-decreasing functions. In general, we cannot compare functions the same way we compare numbers.

33 How do we compare different functions? We will mostly deal with non-decreasing functions. In general, we cannot compare functions the same way we compare numbers. E.g., n 2 vs n. Which one is smaller?

34 O-notation O(g(n)) = {f (n) : there exists positive constants c and n 0 such that 0 f (n) c g(n) for all n n 0 }

35 O-notation O(g(n)) = {f (n) : there exists positive constants c and n 0 such that 0 f (n) c g(n) for all n n 0 } E.g. 10n 2 + 5n 100 O(n 2 )

36 O-notation O(g(n)) = {f (n) : there exists positive constants c and n 0 such that 0 f (n) c g(n) for all n n 0 } E.g. We write 10n 2 + 5n 100 O(n 2 ) 10n 2 + 5n 100 = O(n 2 )

37 O-notation O(g(n)) = {f (n) : there exists positive constants c and n 0 such that 0 f (n) c g(n) for all n n 0 } E.g. We write 10n 2 + 5n 100 O(n 2 ) 10n 2 + 5n 100 = O(n 2 ) Examples: n 2 vs n?

38 O-notation O(g(n)) = {f (n) : there exists positive constants c and n 0 such that 0 f (n) c g(n) for all n n 0 } E.g. We write 10n 2 + 5n 100 O(n 2 ) 10n 2 + 5n 100 = O(n 2 ) Examples: n 2 vs n? n 100 vs 2 n?

39 O-notation O(g(n)) = {f (n) : there exists positive constants c and n 0 such that 0 f (n) c g(n) for all n n 0 } E.g. We write 10n 2 + 5n 100 O(n 2 ) 10n 2 + 5n 100 = O(n 2 ) Examples: n 2 vs n? n 100 vs 2 n? n log n vs 2 n?

40 O-notation O(g(n)) = {f (n) : there exists positive constants c and n 0 such that 0 f (n) c g(n) for all n n 0 } E.g. We write 10n 2 + 5n 100 O(n 2 ) 10n 2 + 5n 100 = O(n 2 ) Examples: n 2 vs n? n 100 vs 2 n? n log n vs 2 n? 2 2n vs 2 n?

41 Ω-notation Ω(g(n)) = {f (n) : there exists positive constants c and n 0 such that 0 c g(n) f (n) for all n n 0 } Theorem f (n) = O(g(n)) if and only if g(n) = Ω(f (n)).

42 Ω-notation Ω(g(n)) = {f (n) : there exists positive constants c and n 0 such that 0 c g(n) f (n) for all n n 0 } Theorem f (n) = O(g(n)) if and only if g(n) = Ω(f (n)). Question: Suppose that f (n) = Ω(n). Does this imply that f (n) is increasing?

43 Θ-notation f (n) = Θ(g(n)) if and only if both of the following hold: f (n) = O(g(n)) f (n) = Ω(g(n))

44 Θ-notation f (n) = Θ(g(n)) if and only if both of the following hold: f (n) = O(g(n)) f (n) = Ω(g(n)) Examples: n 2 + n + 5 vs 100n 2 + 5n + 3?

45 Θ-notation f (n) = Θ(g(n)) if and only if both of the following hold: f (n) = O(g(n)) f (n) = Ω(g(n)) Examples: n 2 + n + 5 vs 100n 2 + 5n + 3? n log n vs n ?

46 o-notation o(g(n)) = {f (n) : for any positive constant c > 0, there exists a constant n 0 such that 0 f (n) < c g(n) for all n n 0 }

47 o-notation o(g(n)) = {f (n) : for any positive constant c > 0, there exists a constant n 0 such that 0 f (n) < c g(n) for all n n 0 } If f (n) = o(g(n)), then f (n) lim n g(n) = 0

48 o-notation o(g(n)) = {f (n) : for any positive constant c > 0, there exists a constant n 0 such that 0 f (n) < c g(n) for all n n 0 } If f (n) = o(g(n)), then f (n) lim n g(n) = 0 Examples: 100n = o(n 2 )

49 o-notation o(g(n)) = {f (n) : for any positive constant c > 0, there exists a constant n 0 such that 0 f (n) < c g(n) for all n n 0 } If f (n) = o(g(n)), then f (n) lim n g(n) = 0 Examples: 100n = o(n 2 ) n 2 o(n 2 )

50 ω-notation ω(g(n)) = {f (n) : for any positive constant c > 0, there exists a constant n 0 such that 0 c g(n) < f (n) for all n n 0 }

51 ω-notation ω(g(n)) = {f (n) : for any positive constant c > 0, there exists a constant n 0 such that 0 c g(n) < f (n) for all n n 0 } f (n) = o(g(n)) if and only if g(n) = ω(f (n)).

52 ω-notation ω(g(n)) = {f (n) : for any positive constant c > 0, there exists a constant n 0 such that 0 c g(n) < f (n) for all n n 0 } f (n) = o(g(n)) if and only if g(n) = ω(f (n)). Examples: 2 n vs n 10?

53 ω-notation ω(g(n)) = {f (n) : for any positive constant c > 0, there exists a constant n 0 such that 0 c g(n) < f (n) for all n n 0 } f (n) = o(g(n)) if and only if g(n) = ω(f (n)). Examples: 2 n vs n 10? n vs n log n?

54 ω-notation ω(g(n)) = {f (n) : for any positive constant c > 0, there exists a constant n 0 such that 0 c g(n) < f (n) for all n n 0 } f (n) = o(g(n)) if and only if g(n) = ω(f (n)). Examples: 2 n vs n 10? n vs n log n? log(n) vs log(log(n))?