Enumerate all possible assignments and take the An algorithm is a well-defined computational

Transcription

1 EMIS 8374 [Algorithm Design and Analysis] 1 EMIS 8374 [Algorithm Design and Analysis] 2 Designing and Evaluating Algorithms A (Bad) Algorithm for the Assignment Problem Enumerate all possible assignments and take the An algorithm is a well-defined computational cheapest one. Slide 1 procedure that takes some values as input and produces some values as output. An MCNFP algorithm takes a network G=(N,A) with Slide 3 Let n = N 1 = N 2. There could be n! possible assignments. Suppose n = 70: vectors b, c, l, and u and returns an optimal solution or else indicates that there is no solution. 1. Correctness 2. Time Complexity 70! = Space Complexity Slide 2 Reasons for classifying special classes of MCNFP 1. The more specialized the problem, the more efficient are the algorithms for solving it. It is important to identify the problem category when selecting a method for solving the problem of interest. 2. We can often exploit the special structure of some problems to design efficient special-purpose algorithms. Slide 4 An Algorithm For Finding Duplicates algorithm AK1 for i = 1 to n 1 do for j = i + 1 to n do if a[i] = a[j] then output i and j; Example taken from The Touring Omnibus: 61 Excursions in Computer Science by A. K. Dewdney.

2 EMIS 8374 [Algorithm Design and Analysis] 3 EMIS 8374 [Algorithm Design and Analysis] 4 Execution of AK1 when a = [7, 8, 4, 5] Instruction Given a list of n integers, what is the maximum Slide 5 for i = 1 to n 1 do for j = i + 1 to n do if a[i] = a[j] then output i and j; Total Executed 15 Slide 7 number of instructions, T (n), AK1 will require to determine if there are any duplicates in the list? Worst Case: the last two integers (and only the last two) are the same. An iteration of the inner For loop executes two instructions if the test fails and four if it succeeds. Slide 6 Execution of AK1 when a = [7, 8, 4, 4] Instruction for i = 1 to n 1 do for j = i + 1 to n do if a[i] = a[j] then output i and j; Total Executed 17 Slide 8 i = Outer Loop Inner Loop 1 1 2(n 1) 2 1 2(n 2) i 1 2(n i) n 2 1 2(n (n 2)) = 2(2) n = 2 + 2(1) Total n n 1 k=1 k

3 EMIS 8374 [Algorithm Design and Analysis] 5 EMIS 8374 [Algorithm Design and Analysis] 6 Slide 9 T (n) = n n 1 = n n 1 = n = n k=1 k k=1 k (n)(n 1) 2 Slide 11 T 1 (n) = n n 1 = n n 1 k=1 k=1 = n (1.25)(n)(n 1) k = 1.25n 2.25n + 2 T 1 (n) is a quadratic function like T (n). k Suppose testing a[i] = a[j] takes 1.5 times as long as incrementing the counter in the For statement and we want T 1 (n) to measure time. Big O Notation Slide 10 i Outer Loop Inner Loop (n 1) (n 2) Slide 12 A function f(n) is said to be big O of a function g(n) if there is a constant c and an integer N such that i 1 2.5(n i) n (2) n = (1) f(n) cg(n) for all n N We write f(n) = O(g(n)). Total n n 1 k=1 k

4 EMIS 8374 [Algorithm Design and Analysis] 7 EMIS 8374 [Algorithm Design and Analysis] 8 Slide 13 To see that T (n) = O(n 2 ), let c = 2 and N = 1: T (n) = n n 2 for all n 1. Slide 15 Basic Operations: simple arithmetic operations such as {+,,, /, <} assigning a value to a variable/parameter (e.g., x 3) To see that T 1 (n) = O(n 2 ), let c = 5 and N = 1: looking up the cost of an arc c ij T 1 (n) = 1.25n 2.25n + 2 5n 2 for all n 1. have unit cost or are O(1) The instruction d j d i + c ij is assumed to be O(1). Worst-Case Complexity Another Algorithm For Finding Duplicates Slide 14 Gives an upper bound (maximum) on the number of basic operations required by an algorithm as a function of the input size. Since T (n) = O(n 2 ), we say the following: The worst-case time complexity AK1 is O(n 2 ). Or, AK1 runs in (or requires) O(n 2 ) time. Asymptotic analysis: measure the rate, ignoring constants, at which the running time grows as a function of input size. Slide 16 Assume b[i] = 0 for all i algorithm AK2 for i = 1 to n do if b[a[i]] 0 then output a[i]; else b[a[i]] 1;

5 EMIS 8374 [Algorithm Design and Analysis] 9 EMIS 8374 [Algorithm Design and Analysis] 10 Execution of AK2 when a = [7, 8, 4, 4] n 2 log n n 1 2 n n! Slide 17 Instruction for i = 1 to n do if b[a[i]] 0 then output a[i]; Slide else b[a[i]] 1; Total Executed T (n) = 3(n 1) + 4 = O(n) Slide 18 Comparing Worst-Complexity of Algorithms Since n n 2 for all integers, AK2 is considered more efficient (in terms of time) than AK1. O(n) O(n log n) O(n 2 ) O(2 n ). An algorithm is said to be polynomial if its running time is bounded by a polynomial. An algorithm is said to be exponential if its running time cannot be bounded by a polynomial. Slide 20 Our complexity will usually be looser than the preceding example. for i = 1 to n 1 do for j = i + 1 to n do if a[i] = a[j] then output i and j; Polynomial-time algorithms are preferred to exponential-time algorithms. The block of pseudo code in the inner loop is O(1). It gets executed at most (n 1) (n 1) times. Therefore, this algorithm is O(n 2 ).

6 EMIS 8374 [Algorithm Design and Analysis] 11 EMIS 8374 [Algorithm Design and Analysis] 12 Caveats Concerning Complexity 1. Each operation is given unit cost, regardless of the size Slide 21 of the numbers involved. But multiplying two huge numbers, may require more effort than multiplying two small numbers. 2. An algorithm that is polynomial is considered to be good. So an algorithm with O(n 100 ) complexity is considered good even though it may be completely impractical. 3. Asymptotic behavior: it may be that an algorithm Slide 23 Space Complexity AK1 vs. AK2 AK1: Needs to store n + 2 numbers: a, i, and j. AK2: Needs to store a, b, and i. The size of b depends on the value of the largest integer our computer store in a single memory location. with better complexity only s to perform better for instances of inordinately large size. How to measure input size for network flow algorithms. 4. Can be overly pessimistic (worst-case behavior): Linear Programming: simplex method vs. the ellipsoid Slide 22 algorithm. QuickSort: O(n 2 ) worst-case complexity, but Slide 24 O(n log n) average-case complexity. Constants are smaller than other O(n log n) algorithms

7 EMIS 8374 [Algorithm Design and Analysis] 13 EMIS 8374 [Algorithm Design and Analysis] 14 Node-Arc Incidence Matrix An n m matrix M where n = N and m = A. If (i, j) A then M ij = +1 and M ji = 1. Vectors Specifying Parameter Values for the Arcs If (i, j) / A then M ij = M ji = 0. c = { c 1,3, c 2,3, c 3,4, c 2,1, c 3,2, c 4,3 } Slide 25 Node 1 Node 2 Node 3 Node 4 x 1,3 x 2,4 x 3,4 x 2,1 x 3,2 x 4, Slide 27 l = { l 1,3, l 2,3, l 3,4, l 2,1, l 3,2, l 4,3 } u = { u 1,3, u 2,3, u 3,4, u 2,1, u 3,2, u 4,3 } b T = { 10, 6, 2, 4 } The complexity of most algorithms we will study does not depend on these values. Node-Node Incidence Matrix Adjacency Lists An n n matrix M. 1 3 c[1,3] l[1,3] u[1,3] 0 If (i, j) A then M ij = 1. If (i, j) / A then M ij = c[2,1] l[2,1] u[2,1] 3 Slide 26 Node Slide c[2,4] l[2,4] u[2,4] 2 c[3,2] l[3,2] u[3,2] 4 c[3,4] l[3,4] 3 c[4,3] l[4,3] u[4,3] 0 u[3,4] 0 0

8 EMIS 8374 [Algorithm Design and Analysis] 15 EMIS 8374 [Algorithm Design and Analysis] 16 Measuring the Size of an MCNFP Instance Slide 29 We say that a network G = (N, A) has n = N nodes and m = A arcs. The complexity of a network algorithm may be measured as a function of n, m, or both. Note that m n 2 Sparse networks: m = O(n) Dense networks: m close to n 2. Slide 31 Show that ln n = O(n ɛ ) for any constant ɛ > 0. ln n lim = lim n n ɛ n 1 n = lim ɛnɛ 1 n 1 ɛn = 0 ɛ Therefore ln n cn ɛ for all n N for some constant c. Since n 1 m n 2, we say n = O(m). Sometimes the size of the parameters c ij, b(i), etc plays a role. The next property is often used in conjunction with L hôpital s Rule. Slide 30 Suppose that lim n f(n) g(n) exists: f(n) lim n g(n) = c. Then, 0 c < implies that f(n) = O(g(n)).