Analysis of Algorithms Fall Basics of Algorithm Analysis Computational Tractability

Transcription

1 Analysis of Algorithms Fall 2017 Basics of Algorithm Analysis Computational Tractability Mohammad Ashiqur Rahman Department of Computer Science College of Engineering Tennessee Tech University

2 A Strikingly Modern Thought As soon as an Analytic Engine exists, it will necessarily guide the future course of the science. Whenever any result is sought by its aid, the question will arise By what course of calculation can these results be arrived at by the machine in the shortest time? Charles Babbage (1864) How many times do you have to turn the crank? Analytic Engine 2

3 Brute-Force Algorithm For many nontrivial problems, there is a natural brute-force search algorithm that checks every possible solution. Typically takes 2 N time or worse for inputs of size N. Unacceptable in practice. M. Ashiq Rahman, Tennessee Tech University 3

4 4 Polynomial Running Time Desirable scaling property. When the input size doubles, the algorithm should only slow down by some constant factor C. Def. An algorithm is poly-time if the above scaling property holds. There exists constants c > 0 and d > 0 such that on every input of size n, its running time is bounded by cn d primitive computationalsteps O(N d ). If the input size increases, the running time will slow down. If the size becomes 2N, what will be the impact? Lower degree polynomials has better scalability than higher degree polynomials.

5 Polynomial Running Time (2) It is said that an algorithm is efficient if it has a polynomial running time. Justification. It really works in practice! In practice, the poly-time algorithms often have low constants and low exponents. Breaking through the exponential barrier of brute force typically exposes some crucial structure of the problem. However, some poly-time algorithms do have high constants and/or exponents, and/or are useless in practice Question. Which would you prefer: 20 n 100 vs. n ln n? M. Ashiq Rahman, Tennessee Tech University 5

6 6 Worst-Case Analysis Worst case running time. Obtain bound on largest possible running time of algorithm on input of a given size N. Generally captures efficiency in practice. Strict view, but hard to find effective alternative. Exceptions. Some exponential-time algorithms are used widely in practice because the worst-case instances seem to be rare.

7 Average-Case Running Time Average case running time. Obtain bound on running time of algorithm on random input as a function of input size N. Hard (or impossible) to accurately model real instances by random distributions. Algorithm tuned for a certain distribution may perform poorly on other inputs. M. Ashiq Rahman, Tennessee Tech University 7

8 Other Running Times Probabilistic. Expected running time of a randomized (probabilistic) algorithm. Amortized. Worst-case running time for any sequence of n operations. Ex. Starting from an empty stack, any sequence of n push and pop operations takes O(n) operations using a resizing array. M. Ashiq Rahman, Tennessee Tech University 8

9 9 Why Running Time Matters? Intractable!

10 THANKS Source: - Chapter 2, Basics of Algorithm Analysis, Kleinberg and Tardos - Thanks to Dr. Kevin Wayne (Princeton University) and Dr. Martha Kosa (Tennessee Tech) M. Ashiq Rahman, Tennessee Tech University 10