feb abhi shelat FFT,Median

Transcription

1 L8 feb abhi shelat FFT,Median

2 merge-sort (A, p, r) if p<r q (p + r)/2 merge-sort (A, p, q) merge-sort (A, q +1,r) merge(a, p, q, r) MERGE(A[1.. n],m): i 1; j m +1 for k 1 to n if j>n B[k] A[i]; i i +1 else if i>m B[k] A[j]; j j +1 else if A[i] <A[j] B[k] A[i]; i i +1 else B[k] A[j]; j j +1 for k 1 to n A[k] B[k] jeff erickson

3 Karatsuba(ab, cd) Base case: return b*d if inputs are 1-digit ac = Karatsuba(a,c) bd = Karatsuba(b,d) 3T (n/2) + 2n t = Karatsuba((a+b),(c+d)) mid = t - ac - bd RETURN ac* mid*100 + bd 4n 3n

4 Closest(P,SX,SY) Let q be the middle-element of SX Divide P into Left, Right according to q delta,r,j = MIN(Closest(Left, LX, LY) Closest(Right, RX, RY) ) Mohawk = { Scan SY, add pts that are delta from q.x } For each point x in Mohawk (in order): Compute distance to its next 15 neighbors Update delta,r,j if any pair (x,y) is < delta Return (delta,r,j) Can be reduced to 7!

5 second attempt arbit+(a[1...n]) base case if A <=2, (lg,minl,maxl) = arbit(left(a)) (rg,minr,maxr) = arbit(right(a)) return max{maxr-minl,lg,rg}, min{minl, minr}, max{maxl, maxr}

6 [strassen]

7 FFT(f=a[1,...,n]) Base case if n<=2 E[...] <- FFT(Ae) O[...] <- FFT(Ao) // eval Ae on n/2 roots of unity // eval Ao on n/2 roots of unity combine results using equation: Return n resulting values.

8 Fast Fourier Transform 2

9 FFT input: a 0,a 1,a 2,...,a n 1 output: evaluate polynomial A at (any) n different points. n points on a curve

10 Brute force method to evaluate A at n points:

11

12 FFT(f=a[1,...,n]) Evaluates degree n poly on the n th roots of unity

13 Last remaining issue: Which points to use? Roots of unity should have n solutions what are they?

14 the n solutions are: n1,e 2 i/n,e 2 i2/n,e 2 i3/n,...,e 2 i(n 1)/no because e 2 i =1

15 the n solutions are: consider for j=0,1,2,3,...,n-1 = is an n th root of unity

16 What is this number? = is an n th root of unity

17 Taylor series expansion of a function f around point a f (y) = f (a)+ f 0 (a) (y a)+ f 00 (a) 1! 2! (y a) 2 + f 000 (a) 3! (y a) 2 + e x = around 0

18 What is this number? = is an n th root of unity e 2pij/n = cos(2pj/n)+i sin(2pj/n)

19 = is an n th root of unity Lets compute! 1,8

20 Compute all 8 roots of unity Then graph them

21 roots of unity should have n solutions e 2pij/n = cos(2pj/n)+i sin(2pj/n)

22 squaring the n th roots of unity

23 Thm: Squaring an n th root produces an n/2 th root. 1! 1,8 = p2 + p i example: 2! 2 1,8 = 1 p2 + p i 2 = 2 1 p p2 i p 2 + i p 2 2 =1/2+i 1/2 = i

24 squaring the n th roots of unity

25 Thm: Squaring an n th root produces an n/2 th root. n1,e 2 i(1/n),e 2 i(2/n),e 2 i(3/n),...,e 2 i(n/2)/n,e 2 i(n/2+1)/n 2 i(n,...,e 1)/no

26 Thm: Squaring an n th root produces an n/2 th root. n1,e 2 i(1/n),e 2 i(2/n),e 2 i(3/n),...,e 2 i(n/2)/n,e 2 i(n/2+1)/n 2 i(n,...,e 1)/no 1 e 2 i(1/(n/2)) e 2 i(2/(n/2)) e 2 i(3/(n/2)) 1 e 2 i((n/2)+1/(n/2)) = e 2 i(1+1/(n/2)) =1 e 2 i(1/(n/2))

27 If n=16

28 evaluate at a root of unity A(! i,n )=A e (! 2 i,n)+! i,n A o (! 2 i,n) n th root of unity n/2 th root of unity n/2 th root of unity

29 FFT(f=a[1,...,n]) Evaluates degree n poly on the n th roots of unity

30 FFT(f=a[1,...,n]) Base case if n<=2 E[...] <- FFT(Ae) O[...] <- FFT(Ao) // eval Ae on n/2 roots of unity // eval Ao on n/2 roots of unity combine results using equation: Return n resulting values.

31

32 a3 a2 a1 a0 b3 b2 b1 b0 a3b0 a2b0 a1b0 a0b0 a3b1 a2b1 a1b1 a0b1 a3b2 a2b2 a1b2 a0b2 a3b3 a2b3 a1b3 a0b3

33 A(x) =a 3 x 3 + a 2 x 2 + a 1 x + a 0 B(x) =b 3 x 3 + b 2 x 2 + b 1 x + b 0 C(x) = a 3 b 3 x 6 + (a 3 b 2 + a 2 b 3 )x 5 + (a 3 b 1 + a 2 b 2 + a 1 b 3 )x 4 + (a 3 b 0 + a 2 b 1 + a 1 b 2 + a 0 b 3 )x 3 + (a 2 b 0 + a 1 b 1 + a 0 b 2 )x 2 + (a 1 b 0 + a 0 b 1 )x+ a 0 b 0

34 y=2x+1 y=x+1

35 y=2x+1 y=x+1

36 a3 a2 a1 a0 b3 b2 b1 b0 A(x) =a 0 + a 1 x + a 2 x 2 + a 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 B(x) =b 0 + b 1 x + b 2 x 2 + b 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7

37 a3 a2 a1 a0 b3 b2 b1 b0 A(x) =a 0 + a 1 x + a 2 x 2 + a 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 B(x) =b 0 + b 1 x + b 2 x 2 + b 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 A(! 0 ) A(! 1 ) A(! 2 ) A(! 7 )...

38 a3 a2 a1 a0 b3 b2 b1 b0 A(x) =a 0 + a 1 x + a 2 x 2 + a 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 B(x) =b 0 + b 1 x + b 2 x 2 + b 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 A(! 0 ) A(! 1 ) A(! 2 ) A(! 7 )... B(! 0 ) B(! 1 ) B(! 2 )... B(! 7 ) FFT FFT

39 a3 a2 a1 a0 b3 b2 b1 b0 A(x) =a 0 + a 1 x + a 2 x 2 + a 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 B(x) =b 0 + b 1 x + b 2 x 2 + b 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 A(! 0 ) A(! 1 ) A(! 2 ) A(! 7 )... B(! 0 ) B(! 1 ) B(! 2 )... B(! 7 ) FFT FFT C(! 0 ) C(! 1 ) C(! 2 )... C(! 7 )

40 a3 a2 a1 a0 b3 b2 b1 b0 A(x) =a 0 + a 1 x + a 2 x 2 + a 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 B(x) =b 0 + b 1 x + b 2 x 2 + b 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 A(! 0 ) A(! 1 ) A(! 2 ) A(! 7 )... B(! 0 ) B(! 1 ) B(! 2 )... B(! 7 ) FFT FFT C(! 0 ) C(! 1 ) C(! 2 )... C(! 7 ) C(x) =c 0 + c 1 x + c 2 x 2 + c 7 x 7 IFFT

41 application to mult karatsuba a b c d

42 application to mult karatsuba a b c d

43 Multiplying n-bit numbers Schönhage Strassen 71 Fürer 07 O(n log n log log n) O(n log(n)2 log (n) )

44 AGMP-BASEDIMPLEMENTATIONOFSCHÖNHAGE-STRASSEN S LARGE INTEGER MULTIPLICATION ALGORITHM PIERRICK GAUDRY, ALEXANDER KRUPPA, AND PAUL ZIMMERMANN Abstract. Schönhage-Strassen s algorithm is one of the best known algorithms for multiplying large integers. Implementing it efficiently is of utmost importance, since many other algorithms rely on it as a subroutine. We present here an improved implementation, based on the one distributed within the GMP library. The following ideas and techniques were used or tried: faster arithmetic modulo 2 n +1, improved cache locality, Mersenne transforms, Chinese Remainder Reconstruction, the 2trick,Harley sandgranlund stricks, improved tuning. We also discuss some ideas we plan to try in the future. Introduction Since Schönhage and Strassen have shown in 1971 how to multiply two N-bit integers in O(N log N log log N) time[21],severalauthorsshowedhowtoreduceotheroperations inverse, division, square root, gcd, base conversion, elementary functions to multiplication, possibly with log N multiplicative factors [5, 8, 17, 18, 20, 23]. It has now become common practice to express complexities in terms of the cost M(N) to multiply two N-bit numbers, and many researchers tried hard to get the best possible constants in front of M(N) forthe above-mentioned operations (see for example [6, 16]). Strangely, much less effort was made for decreasing the implicit constant in M(N) itself, although any gain on that constant will give a similar gain on all multiplication-based operations. Some authors reported on implementations of large integer arithmetic for specific hardware or as part of a number-theoretic project [2, 10]. In this article we concentrate on the question of an optimized implementation of Schönhage-Strassen s algorithm on a classical workstation.

45 Applications of FFT

46 Applications of FFT

47

48 String matching with * ACAAGATGCCATTGTCCCCCGGCCTCCTGCTGCTGCTGCTCTCCGGGGCCACGGCCACCGCTGCCCTGCC CCTGGAGGGTGGCCCCACCGGCCGAGACAGCGAGCATATGCAGGAAGCGGCAGGAATAAGGAAAAGCAGC CTCCTGACTTTCCTCGCTTGGTGGTTTGAGTGGACCTCCCAGGCCAGTGCCGGGCCCCTCATAGGAGAGG AAGCTCGGGAGGTGGCCAGGCGGCAGGAAGGCGCACCCCCCCAGCAATCCGCGCGCCGGGACAGAATGCC CTGCAGGAACTTCTTCTGGAAGACCTTCTCCTCCTGCAAATAAAACCTCACCCATGAATGCTCACGCAAG TTTAATTACAGACCTGAA Looking for all occurrences of GGC*GAG*C*GC where I don't care what the * symbol is.

49 Median

50 problem: given a list of n elements, find the element of rank n/2. (half are larger, half are smaller)

51 problem: given a list of n elements, find the element of rank n/2. (half are larger, half are smaller) can generalize to i first solution: sort and pluck.

52 problem: given a list of n elements, find the element of rank i. key insight: we do not have to fully sort. semi sort can suffice.

53 pick first element partition list about this one see where we stand

54 review: how to partition a list

55 review: how to partition a list GOAL: start with THIS LIST and END with THAT LIST less than greater than

56 review: how to partition a list

57 review: how to partition a list

58 review: how to partition a list

59 review: how to partition a list

60 review: how to partition a list

61 review: how to partition a list partitioning a list about an element takes linear time.

62 select

63 select handle base case. partition list about first element if pivot p is position i, return pivot else if pivot p is in position > i select else select

64 Assume our partition always select splits list into two eql parts handle base case. partition list about first element if pivot is position i, return pivot else if pivot is in position > i select else select

65 Assume our partition always select splits list into two eql parts handle base case. partition list about first element if pivot is position i, return pivot else if pivot is in position > i select else select

66 problem: what if we always pick bad partitions?

67 problem: what if we always pick bad partitions?

68 problem: what if we always pick bad partitions?

69 problem: what if we always pick bad partitions?

70 select handle base case. partition list about first element if pivot is position i, return pivot else if pivot is in position > i select else select

71 select handle base case. partition list about first element if pivot is position i, return pivot else if pivot is in position > i select else select T (n) =T (n 1) + O(n) (n 2 )

72 Needed: a good partition element partition

73 Needed: a good partition element partition produce an element where 30% smaller, 30% larger

74 solution: bootstrap image: gucci image: mark nason image: d&g

75 partition

76 partition

77 partition

78 partition median of each group form a smaller list select use the median of this smaller list as the partition element

79 partition

80 partition divide list into groups of 5 elements find median of each small list gather all medians call select(...) on this sublist to find median return the result

81 partition divide list into groups of 5 elements find median of each small list gather all medians call select(...) on this sublist to find median return the result

82 a nice property of our partition

83 a nice property of our partition

84 a nice property of our partition

85 SWITCH TO A BIGGER EXAMPLE

86 a nice property of our partition < < < < < < < < < < <

87 a nice property of our partition < < < < < < < < < < < this implies there are at most numbers larger than /smaller

88 a nice property of our partition

89

90 select

91 select handle base case for small list else pivot = FindPartitionValue(A,n) partition list about pivot if pivot is position i, return pivot else if pivot is in position > i select else select

92

93 FindPartition divide list into groups of 5 elements find median of each small list gather all medians call select(...) on this sublist to find median return the result

94 select handle base case for small list else pivot = FindPartitionValue(A,n) partition list about pivot if pivot is position i, return pivot else if pivot is in position > i select else select

95 select handle base case for small list else pivot = FindPartitionValue(A,n) partition list about pivot if pivot is position i, return pivot else if pivot is in position > i select else select

96

97

98 Stairs(n) if n<=1 return 1 return Stairs(n-1) + Stairs(n-2)

99 Stairs(n) if n<=1 return 1 ret Stairs(n-1) + Stairs(n-2) Stairs(5) Stairs(4) Stairs(3) Stairs(3) Stairs(2) Stairs(2) Stairs(1) Stairs(2) Stairs(1) Stairs(1) Stairs(0) Stairs(1) Stairs(0)

100 initialize memory M Stairs(n) if n<=1 then return 1 if n is in M, return M[n] answer = Stairs(i-1)+ Stairs(i-2) M[n] = answer return answer

101 Stairs(n) if n<=1 then return 1 if n is in M, return M[n] answer = Stairs(i-1)+ Stairs(i-2) M[n] = answer return answer Stairs(5)

102 Stairs(n) stair[0]=1 stair[1]=1 for i=2 to n stair[i] = stair[i-1]+stair[i-2] return stair[i]

103 initialize memory M Stairs(n)

104 Stairs(n) if n<=1 then return 1 if n is in M, return M[n] answer = Stairs(i-1)+ Stairs(i-2) M[n] = answer return answer Stairs(5)

105 Stairs(n) stair[0]=1 stair[1]=1 for i=2 to n stair[i] = stair[i-1]+stair[i-2] return stair[i]

106 Dynamic Programming

107 two big ideas

108 two big ideas recursive structure memoizing

109 wood cutting

110

111 Spot price for lumber

112 Spot price for lumber

113 Log cutter dilemna input to the problem: n, (p 1,...,p n ) goal:

114 Observation

115 Solution equation

116 Approach 0... i

117 BestLogs( n, (p 1,...,p n ) ) if n<=0 return 0

118 BestLogs( n, (p 1,...,p n ) ) if n<=0 return 0 for i=1 to n Best[i] = max k=1...i {p k +Best[i k]}

119 The actual cuts?

120 BestLogs( n, (p 1,...,p n ) ) if n<=0 return 0 for i=1 to n Best[i] = max k=1...i {p k +Best[i k]}