shelat 16f-4800 sep Matrix Mult, Median, FFT

Size: px

Start display at page:

Download "shelat 16f-4800 sep Matrix Mult, Median, FFT"

Hester Kennedy
5 years ago
Views:

1 L5 shelat 16f-4800 sep Matrix Mult, Median, FFT

2 merge-sort (A, p, r) if p<r q (p + r)/2 merge-sort (A, p, q) merge-sort (A, q +1,r) merge(a, p, q, r) MERGE(A[1.. n],m): i 1; j m +1 for k 1 to n if j>n B[k] A[i]; i i +1 else if i>m B[k] A[j]; j j +1 else if A[i] <A[j] B[k] A[i]; i i +1 else B[k] A[j]; j j +1 for k 1 to n A[k] B[k] jeff erickson

3 Karatsuba(ab, cd) Base case: return b*d if inputs are 1-digit ac = Karatsuba(a,c) bd = Karatsuba(b,d) 3T (n/2) + 2n t = Karatsuba((a+b),(c+d)) mid = t - ac - bd RETURN ac* mid*100 + bd 4n 3n

4 Closest(P,SX,SY) Let q be the middle-element of SX Divide P into Left, Right according to q delta,r,j = MIN(Closest(Left, LX, LY) Closest(Right, RX, RY) ) Mohawk = { Scan SY, add pts that are delta from q.x } For each point x in Mohawk (in order): Compute distance to its next 15 neighbors Update delta,r,j if any pair (x,y) is < delta Return (delta,r,j) Can be reduced to 7!

5 second attempt arbit+(a[1...n]) base case if A <=2, (lg,minl,maxl) = arbit(left(a)) (rg,minr,maxr) = arbit(right(a)) return max{maxr-minl,lg,rg}, min{minl, minr}, max{maxl, maxr}

7 multiplication

8 =

9 =

11 nx c i,j = a i,k b k,j k=1

16 = AE + BG AF + BH CE + DG CF + DH [Strassen] P 1 = A(F H) P 2 =(A + B)H P 3 =(C + D)E P 4 = D(G E) P 5 =(A + D)(E + H) P 6 =(B D)(G + H) P 7 =(A C)(E + F )

17 [strassen]

18 [strassen]

19 [strassen]

20 taking this idea further 3x3 matricies [Laderman 75]

21 1978 victor pan method 70x70 matrix using mults what is the recurrence:

24 Median

25 problem: given a list of n elements, find the element of rank n/2. (half are larger, half are smaller)

26 problem: given a list of n elements, find the element of rank n/2. (half are larger, half are smaller) can generalize to i first solution: sort and pluck.

27 problem: given a list of n elements, find the element of rank i. key insight: we do not have to fully sort. semi sort can suffice.

28 pick first element partition list about this one see where we stand

29 review: how to partition a list

30 review: how to partition a list GOAL: start with THIS LIST and END with THAT LIST less than greater than

31 review: how to partition a list

32 review: how to partition a list

33 review: how to partition a list

34 review: how to partition a list

35 review: how to partition a list

36 review: how to partition a list partitioning a list about an element takes linear time.

37 select

38 select handle base case. partition list about first element if pivot p is position i, return pivot else if pivot p is in position > i select else select

39 Assume our partition always select splits list into two eql parts handle base case. partition list about first element if pivot is position i, return pivot else if pivot is in position > i select else select

40 Assume our partition always select splits list into two eql parts handle base case. partition list about first element if pivot is position i, return pivot else if pivot is in position > i select else select

41 problem: what if we always pick bad partitions?

42 problem: what if we always pick bad partitions?

43 problem: what if we always pick bad partitions?

44 problem: what if we always pick bad partitions?

45 select handle base case. partition list about first element if pivot is position i, return pivot else if pivot is in position > i select else select

46 select handle base case. partition list about first element if pivot is position i, return pivot else if pivot is in position > i select else select T (n) =T (n 1) + O(n) (n 2 )

47 Needed: a good partition element partition

48 Needed: a good partition element partition produce an element where 30% smaller, 30% larger

49 solution: bootstrap image: gucci image: mark nason image: d&g

50 partition

51 partition

52 partition

53 partition median of each group form a smaller list select use the median of this smaller list as the partition element

54 partition

55 partition divide list into groups of 5 elements find median of each small list gather all medians call select(...) on this sublist to find median return the result

56 partition divide list into groups of 5 elements find median of each small list gather all medians call select(...) on this sublist to find median return the result

57 a nice property of our partition

58 a nice property of our partition

59 a nice property of our partition

60 SWITCH TO A BIGGER EXAMPLE

61 a nice property of our partition < < < < < < < < < < <

62 a nice property of our partition < < < < < < < < < < < this implies there are at most numbers larger than /smaller

63 a nice property of our partition

65 select

66 select handle base case for small list else pivot = FindPartitionValue(A,n) partition list about pivot if pivot is position i, return pivot else if pivot is in position > i select else select

68 FindPartition divide list into groups of 5 elements find median of each small list gather all medians call select(...) on this sublist to find median return the result

69 select handle base case for small list else pivot = FindPartitionValue(A,n) partition list about pivot if pivot is position i, return pivot else if pivot is in position > i select else select

70 select handle base case for small list else pivot = FindPartitionValue(A,n) partition list about pivot if pivot is position i, return pivot else if pivot is in position > i select else select

71 Fast Fourier Transform

75 big ideas:

76 f(x) =5+2x + x

77 f(x) =5+2x + x

81 degree polynomial n points on a curve

82 FFT input: a 0,a 1,a 2,...,a n 1 output:

83 FFT input: a 0,a 1,a 2,...,a n 1 output: evaluate polynomial A at (any) n different points. n points on a curve

84 Later, we shall see that the same ideas for FFT can be used to implement Inverse-FFT. Inverse FFT: Given n-points,

85 Later, we shall see that the same ideas for FFT can be used to implement Inverse-FFT. Inverse FFT: Given n-points, y 0,y 1,...,y n 1 find a degree n polynomial A such that y i = A(! i )

86 Brute force method to evaluate A at n points:

87 solve the large problem by solving smaller problems and combining solutions T(n)=

90 suppose we had already had eval of Ae,Ao on {4,9,16,25} A e (4) A e (9) A e (16) A e (25) A 0 (4) A 0 (9) A 0 (16) A 0 (25)

91 suppose we had already had eval of Ae,Ao on {4,9,16,25} A e (4) A e (9) A e (16) A e (25) A 0 (4) A 0 (9) A 0 (16) A 0 (25) Then we could compute 8 terms: A(2) = A e (4) + 2A o (4) A( 2) = A e (4) + ( 2)A o (4) A(3) = A e (9) + 3A o (9) A( 3) = A e (9) + ( 3)A o (9) A(4), A(-4), A(5), A(-5)

92 FFT(f=a[1,...,n]) Evaluates degree n poly on the n th roots of unity

93 Last remaining issue:

94 Roots of unity should have n solutions what are they?

95 Remember this? e 2 i =1

96 consider the n solutions are: n1,e 2 i/n,e 2 i2/n,e 2 i3/n,...,e 2 i(n 1)/no

97 the n solutions are: consider for j=0,1,2,3,...,n-1 = is an n th root of unity

98 What is this number? = is an n th root of unity

99 Taylor series expansion of a function f around point a f (y) = f (a)+ f 0 (a) (y a)+ f 00 (a) 1! 2! (y a) 2 + f 000 (a) 3! (y a) 2 + e x = around 0

100 What is this number? = is an n th root of unity e 2pij/n = cos(2pj/n)+i sin(2pj/n)

101 = is an n th root of unity Lets compute! 1,8

102 Compute all 8 roots of unity Then graph them

103 roots of unity should have n solutions e 2pij/n = cos(2pj/n)+i sin(2pj/n)

104 squaring the n th roots of unity

105 squaring the n th roots of unity

106 Thm: Squaring an n th root produces an n/2 th root. 1! 1,8 = p2 + p i example: 2! 2 1,8 = 1 p2 + p i 2 = 2 1 p p2 i p 2 + i p 2 2 =1/2+i 1/2 = i

107 Thm: Squaring an n th root produces an n/2 th root. n1,e 2 i(1/n),e 2 i(2/n),e 2 i(3/n),...,e 2 i(n/2)/n,e 2 i(n/2+1)/n 2 i(n,...,e 1)/no

108 Thm: Squaring an n th root produces an n/2 th root. n1,e 2 i(1/n),e 2 i(2/n),e 2 i(3/n),...,e 2 i(n/2)/n,e 2 i(n/2+1)/n 2 i(n,...,e 1)/no 1 e 2 i(1/(n/2)) e 2 i(2/(n/2)) e 2 i(3/(n/2)) 1 e 2 i((n/2)+1/(n/2)) = e 2 i(1+1/(n/2)) =1 e 2 i(1/(n/2))

109 If n=16

110 evaluate at a root of unity

111 evaluate at a root of unity A(! i,n )=A e (! 2 i,n)+! i,n A o (! 2 i,n) n th root of unity n/2 th root of unity n/2 th root of unity

112 FFT(f=a[1,...,n]) Evaluates degree n poly on the n th roots of unity

113 FFT(f=a[1,...,n]) Base case if n<=2 E[...] <- FFT(Ae) O[...] <- FFT(Ao) // eval Ae on n/2 roots of unity // eval Ao on n/2 roots of unity combine results using equation: Return n points.

114

115 a3 a2 a1 a0 b3 b2 b1 b0 a3b0 a2b0 a1b0 a0b0 a3b1 a2b1 a1b1 a0b1 a3b2 a2b2 a1b2 a0b2 a3b3 a2b3 a1b3 a0b3

116 A(x) =a 3 x 3 + a 2 x 2 + a 1 x + a 0 B(x) =b 3 x 3 + b 2 x 2 + b 1 x + b 0 C(x) = a 3 b 3 x 6 + (a 3 b 2 + a 2 b 3 )x 5 + (a 3 b 1 + a 2 b 2 + a 1 b 3 )x 4 + (a 3 b 0 + a 2 b 1 + a 1 b 2 + a 0 b 3 )x 3 + (a 2 b 0 + a 1 b 1 + a 0 b 2 )x 2 + (a 1 b 0 + a 0 b 1 )x+ a 0 b 0

117 y=2x+1 y=x+1

118 y=2x+1 y=x+1

119 a3 a2 a1 a0 b3 b2 b1 b0 A(x) =a 0 + a 1 x + a 2 x 2 + a 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 B(x) =b 0 + b 1 x + b 2 x 2 + b 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7

120 a3 a2 a1 a0 b3 b2 b1 b0 A(x) =a 0 + a 1 x + a 2 x 2 + a 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 B(x) =b 0 + b 1 x + b 2 x 2 + b 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 A(! 0 ) A(! 1 ) A(! 2 ) A(! 7 )...

121 a3 a2 a1 a0 b3 b2 b1 b0 A(x) =a 0 + a 1 x + a 2 x 2 + a 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 B(x) =b 0 + b 1 x + b 2 x 2 + b 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 A(! 0 ) A(! 1 ) A(! 2 ) A(! 7 )... B(! 0 ) B(! 1 ) B(! 2 )... B(! 7 ) FFT FFT

122 a3 a2 a1 a0 b3 b2 b1 b0 A(x) =a 0 + a 1 x + a 2 x 2 + a 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 B(x) =b 0 + b 1 x + b 2 x 2 + b 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 A(! 0 ) A(! 1 ) A(! 2 ) A(! 7 )... B(! 0 ) B(! 1 ) B(! 2 )... B(! 7 ) FFT FFT C(! 0 ) C(! 1 ) C(! 2 )... C(! 7 )

123 a3 a2 a1 a0 b3 b2 b1 b0 A(x) =a 0 + a 1 x + a 2 x 2 + a 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 B(x) =b 0 + b 1 x + b 2 x 2 + b 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 A(! 0 ) A(! 1 ) A(! 2 ) A(! 7 )... B(! 0 ) B(! 1 ) B(! 2 )... B(! 7 ) FFT FFT C(! 0 ) C(! 1 ) C(! 2 )... C(! 7 ) C(x) =c 0 + c 1 x + c 2 x 2 + c 7 x 7 IFFT

124 application to mult karatsuba a b c d

125 application to mult karatsuba a b c d

126 Multiplying n-bit numbers Schönhage Strassen 71 Fürer 07 O(n log n log log n) O(n log(n)2 log (n) )

127 AGMP-BASEDIMPLEMENTATIONOFSCHÖNHAGE-STRASSEN S LARGE INTEGER MULTIPLICATION ALGORITHM PIERRICK GAUDRY, ALEXANDER KRUPPA, AND PAUL ZIMMERMANN Abstract. Schönhage-Strassen s algorithm is one of the best known algorithms for multiplying large integers. Implementing it efficiently is of utmost importance, since many other algorithms rely on it as a subroutine. We present here an improved implementation, based on the one distributed within the GMP library. The following ideas and techniques were used or tried: faster arithmetic modulo 2 n +1, improved cache locality, Mersenne transforms, Chinese Remainder Reconstruction, the 2trick,Harley sandgranlund stricks, improved tuning. We also discuss some ideas we plan to try in the future. Introduction Since Schönhage and Strassen have shown in 1971 how to multiply two N-bit integers in O(N log N log log N) time[21],severalauthorsshowedhowtoreduceotheroperations inverse, division, square root, gcd, base conversion, elementary functions to multiplication, possibly with log N multiplicative factors [5, 8, 17, 18, 20, 23]. It has now become common practice to express complexities in terms of the cost M(N) to multiply two N-bit numbers, and many researchers tried hard to get the best possible constants in front of M(N) forthe above-mentioned operations (see for example [6, 16]). Strangely, much less effort was made for decreasing the implicit constant in M(N) itself, although any gain on that constant will give a similar gain on all multiplication-based operations. Some authors reported on implementations of large integer arithmetic for specific hardware or as part of a number-theoretic project [2, 10]. In this article we concentrate on the question of an optimized implementation of Schönhage-Strassen s algorithm on a classical workstation.

128 Applications of FFT

129 Applications of FFT

130

131 String matching with * ACAAGATGCCATTGTCCCCCGGCCTCCTGCTGCTGCTGCTCTCCGGGGCCACGGCCACCGCTGCCCTGCC CCTGGAGGGTGGCCCCACCGGCCGAGACAGCGAGCATATGCAGGAAGCGGCAGGAATAAGGAAAAGCAGC CTCCTGACTTTCCTCGCTTGGTGGTTTGAGTGGACCTCCCAGGCCAGTGCCGGGCCCCTCATAGGAGAGG AAGCTCGGGAGGTGGCCAGGCGGCAGGAAGGCGCACCCCCCCAGCAATCCGCGCGCCGGGACAGAATGCC CTGCAGGAACTTCTTCTGGAAGACCTTCTCCTCCTGCAAATAAAACCTCACCCATGAATGCTCACGCAAG TTTAATTACAGACCTGAA Looking for all occurrences of GGC*GAG*C*GC where I don't care what the * symbol is.

feb abhi shelat FFT,Median

feb abhi shelat FFT,Median L8 feb 16 2016 abhi shelat FFT,Median merge-sort (A, p, r) if pn B[k] A[i];