Discrete Mathematics: Logic Discrete Mathematics: Lecture 17. Recurrence
greedy algorithm greedy algorithm makes the best choice at each step (locally optimal solution) according to a specified criterion with the hope of finding a global optimum make 67 cents change with quaters(25 cents), dimes(10 cents), nickels(5 cents), and pennies(1 cent), and using the least total number of coins 2 quaters => 50 cents 1 dime => 10 cents 1 nickel => 5 cents 2 cents => 2 cents
greedy algorithm procedure change (a1, a2,..., ar: different types of coins, a1 > a2 >... > ak n: positive integer) for i := 1 to k di := 0 { di counts the coins ai used } while n ai di := di + 1 { add a coin ai} n := n - ai {di is the number of the coin ai for i = 1, 2,... k}
greedy algorithm make 40 cents change with quaters(25 cents), dimes(10 cents), and pennies(1 cent), and using the least total number of coins 1 quaters => 25 cents 1 dime => 10 cents 1 cents => 5 cents
greedy algorithm make 40 cents change with quaters(25 cents), dimes(10 cents), and pennies(1 cent), and using the least total number of coins 1 quaters => 25 cents 1 dime => 10 cents 1 cents => 5 cents 4 dime => 40 cents
modeling with recurrence relations Minimum number of changes Input: n denominations of coins V1 (=1) < V2 < V3 <... < Vk problem: use the smallest number of coins to make the amount of money C M(j): minimum number of coins required to make the amount of money j M(j) = min {M(j - vi)} + 1 vi <= j
modeling with recurrence relations Input: n denominations of coins V1 (=1) < V2 < V3 <... < Vk change(p) if (p = 0) then return 0 else return 1 + min{ change(p-v1), change(p-v2),..., change(p-vk)}
modeling with recurrence relations Input: n denominations of coins V1 (=1) < V2 < V3 <... < Vk change_dp(n) C[0] = 0 for p = 2 to n min = inf for i = 1 to k if (p >= vi && C[p - vi] + 1 < min) min = C[p - vi] + 1 C[p] = min
modeling with recurrence relations Scheduling talks with the maximum attendees when we have n talks, we want to maximize the total number of student attendees. T(j): the maximum number of total attendees for an optimal schedule from the first j talks P(j): talk ending latest among talks compatible with talk j that end before talk j the largest integer i, i< j for which ei <= sj wj: the number of students for talk j T(j) = max(wj + T(p(j)), T(j-1))
modeling with recurrence relations procedure Maximum Attendees(s1, s2,... sn: start times of talks; e1, e2,...en: end times of talks; w1, w2,..., wn: number of attendees to talks) sort talks by end time and relabel so that e1 e2... en for j := 1 to n if no job i with i < j is compatible with job j p(j) = 0 else p(j) := max{ i i < j and job i is compatible with job j} T(0) := 0 for j := 1 to n T(j) := max(wj + T(p(j)), T(j-1)) return T(n) {T(n) is the maximum number of attendees}
linear homogeneous recurrence relation( ) linear homogeneous recurrence relation of degree k with constant coefficients an = c1an-1 + c2an-2 +... +ckan-k, c1, c2,... ck are real numbers and ck 0 Pn = (1.11)Pn-1 : fn = fn-1 + fn-2 : an = an-5 : an = an-1 + an-2 2 : Hn = 2Hn-1 + 1: Bn = nbn-1: linear homogeneous recurrence relation of degree one linear homogeneous recurrence relation of degree two linear homogeneous recurrence relation of degree five not linear not homogeneous does not have constant coefficients
linear homogeneous recurrence relation Let an = c1an-1 + c2an-2 +... +ckan-k be a linear homogeneous recurrence relation and bn = c1bn-1 + c2bn-2 +... +ckbn-k be a linear homogeneous recurrence relation pn = an + bn = (c1an-1 + c2an-2 +... +ckan-k ) + (c1bn-1 + c2bn-2 +... +ckbn-k) = c1(an-1 + bn-1) + c1(an-2 + bn-2) +.. c1(an-k + bn-k) = c1pn-1 + c2pn-2 +... +ckpn-k thus, pn is a solution of the recurrence qn = α an = c1(α an-1) + c2 (α an-2) +... +ck (α an-k) = c1qn-1 + c2qn-2 +... +ckqn-k thus, qn is a solution of the recurrence if we find some solutions to a linear homogeneous recurrence, then any linear combination of them will be a solution to the linear homogeneous recurrence.
linear homogeneous recurrence relation solving linear homogeneous recurrence relation is to find solutions of the form an = r n for the recurrence relation an = c1an-1 + c2an-2 +... +ckan-k r n = c1r n-1 + c2r n-2 +... +ckr n-k ( = r k - c1r k-1 + c2r k-2 +... +ck-1r - ck = 0 : characteristic equation) the solutions of this equation are called the characteristic roots of the recurrence relation
linear homogeneous recurrence relation Theorem consider an = c1an-1 + c2an-2. if characteristic equation r 2 - c1r - c2 = 0 has two distinct roots r1 and r2, the solution to the recurrence relation is an = α1r1 n + α2r2 n if r1 and r2 are roots of r 2 - c1r - c2 = 0, r1 2 = c1r1 + c2, r2 2 = c1r2 + c2 c1an-1 + c2an-2 = c1(α1r1 n-1 + α2r2 n-1 ) + c2(α1r1 n-2 + α2r2 n-2 ) = α1r1 n-2 (c1r1 + c2) + α2r2 n-2 (c1r2 + c2) = α1r1 n-2 r1 2 + α2r2 n-2 r2 2 = α1r1 n + α2r2 n = an
linear homogeneous recurrence relation an = an-1 + 2an-2 with a0 = 2 and a1 = 7? r 2 -r -2 = 0 (r - 2)(r + 1) = 0 r = 2, r = -1 an = α12 n + α2(-1) n an = α1r1 n + α2r2 n a0 = α1 + α2 = 2 a1 = α1 2 + α2 (-1) = 7 α1 = 3, α2 = -1, an = 3 2 n - (-1) n
linear homogeneous recurrence relation an = an-1 + an-2 with a0 = 0 and a1 = 1 (Fibonacci numbers)? r 2 -r -1 = 0 (1 ± (1+4))/2 an = α1 ((1 + 5)/2) n + α2((1-5)/2) n an = α1r1 n + α2r2 n a0 = α1 + α2 = 0 a1 = α1 ((1 + 5)/2) + α2((1-5)/2) = 1 α1 = 1/ 5, α2 = -1/ 5, an = 1/ 5((1 + 5)/2) n - 1/ 5((1-5)/2) n
linear homogeneous recurrence relation Theorem consider an = c1an-1 + c2an-2 if characteristic equation r 2 - c1r - c2 = 0 has only one root r0, the solution to the recurrence relation is an = α1r0 n + α2nr0 n an = 6an-1-9an-2 with initial conditions a0 = 1 and a1 = 6? the only root of r 2-6r + 9 = 0 is r=3 an = α13 n + α2n3 n a0 = α1 = 1 a1 = 1 3 + α2 3 = 6 α2 = 1 an = 3 n + n3 n
linear homogeneous recurrence relation consider an = c1an-1 + c2an-2 +... + ckan-k if characteristic equation r k - c1r k-1 -... - ck = 0 has k distinct roots r1, r2,..., rk, the solution to the recurrence relation is an = α1r1 n + α2r2 n +... + αkrk n an = 6an-1-11an-2 + 6an-3 with initial conditions a0 = 2, a1 = 5, and a2 = 15? the roots of r 3-6r 2 + 11r - 6 = (r - 1)(r - 2)(r - 3) are r=1, r=2, and r=3 an = α11 n + α22 n + α33 n a0 = α1 + α2 + α3 = 2 a1 = α1 + α2 2 + α3 3 = 5 a2 = α1 + α2 4 + α3 9 = 15 α1 = 1, α2 = -1, α3 = 2, an = 1 1 n -1 2 n + 2 3 n
divide-and-conquer recurrence relations when a recursive algorithm divides a problem of size n into a subproblems, f(n) = a f(n/b) + g(n) n/b: the size of each subproblem g(n): the number of operation in the conquer step
recursive binary search procedure binary search(i, j, x: integers, 1 i j n) m := (i+j)/2 if x = am then return m else if (x < am and i < m) then return binary search(i, m-1, x) else if (x > am and j > m) then return binary search(m+1, j, x) else return 0 {i is the start output is the location of x in a1, a2,... an if it appears; otherwise it is 0} f(n) = f(n/2) + 1
recursive binary search f(n) = f(n/2) + 1 = (f(n/4) + 1) + 1 = f(n/2 2 ) + 2 = (f(n/8) + 1) + 1 = f(n/2 3 ) + 3 = : = f(n/2 k ) + k = 1 + log n when n = 2 k, k = log n, f(n) = log n
merge sort 27 13 26 1 15 2 24 38 1 11 2 27 13 26 1 15 2 24 38 6 divide 27 13 26 1 15 2 24 38 3 4 7 8 27 13 26 1 15 2 24 38 5 9 13 27 1 26 2 15 24 38 10 conquer 1 13 26 27 2 15 24 38 21 1 2 13 15 24 26 27 38
merge sort procedure mergesort(l=a1,... an) if n > 1 then m := n/2 L1 := a1, a2,..., am L2 := am+1, am+2,...,an L := merge(mergesort(l1), mergesort(l2)) {L is now sorted into elements in nondecreasing order} procedure merge(l1, L2) L := empty list while L1 and L2 are nonempty remove smaller of first elements of L1 and L2 from its list put it at the right end of L if this removal makes one list empty then remove all elements from the other list and append them to L return L
merge sort f(1) = 1 f(n) = 2f(N/2) + N f(n)/n = f(n/2) / (N/2) + 1 f(n/2) / (N/2) = f(n/4) / (N/4) + 1 f(n/4) / (N/4) = f(n/8) / (N/8) + 1 : f(2) / (2) = f(1) / (1) + 1 f(n)/n = f(1)/1 + log N f(n) = N log N + N = O(N log N)