shelat 16f-4800 sep Matrix Mult, Median, FFT

L5 shelat 16f-4800 sep 23 2016 Matrix Mult, Median, FFT

merge-sort (A, p, r) if p<r q (p + r)/2 merge-sort (A, p, q) merge-sort (A, q +1,r) merge(a, p, q, r) MERGE(A[1.. n],m): i 1; j m +1 for k 1 to n if j>n B[k] A[i]; i i +1 else if i>m B[k] A[j]; j j +1 else if A[i] <A[j] B[k] A[i]; i i +1 else B[k] A[j]; j j +1 for k 1 to n A[k] B[k] jeff erickson

Karatsuba(ab, cd) Base case: return b*d if inputs are 1-digit ac = Karatsuba(a,c) bd = Karatsuba(b,d) 3T (n/2) + 2n t = Karatsuba((a+b),(c+d)) mid = t - ac - bd RETURN ac*100 2 + mid*100 + bd 4n 3n

Closest(P,SX,SY) Let q be the middle-element of SX Divide P into Left, Right according to q delta,r,j = MIN(Closest(Left, LX, LY) Closest(Right, RX, RY) ) Mohawk = { Scan SY, add pts that are delta from q.x } For each point x in Mohawk (in order): Compute distance to its next 15 neighbors Update delta,r,j if any pair (x,y) is < delta Return (delta,r,j) Can be reduced to 7!

second attempt arbit+(a[1...n]) base case if A <=2, (lg,minl,maxl) = arbit(left(a)) (rg,minr,maxr) = arbit(right(a)) return max{maxr-minl,lg,rg}, min{minl, minr}, max{maxl, maxr}

multiplication

nx c i,j = a i,k b k,j k=1

= AE + BG AF + BH CE + DG CF + DH [Strassen] P 1 = A(F H) P 2 =(A + B)H P 3 =(C + D)E P 4 = D(G E) P 5 =(A + D)(E + H) P 6 =(B D)(G + H) P 7 =(A C)(E + F )

[strassen]

taking this idea further 3x3 matricies [Laderman 75]

1978 victor pan method 70x70 matrix using 143640 mults what is the recurrence:

Median

problem: given a list of n elements, find the element of rank n/2. (half are larger, half are smaller)

problem: given a list of n elements, find the element of rank n/2. (half are larger, half are smaller) can generalize to i first solution: sort and pluck.

problem: given a list of n elements, find the element of rank i. key insight: we do not have to fully sort. semi sort can suffice.

pick first element partition list about this one see where we stand

review: how to partition a list

review: how to partition a list GOAL: start with THIS LIST and END with THAT LIST less than greater than

review: how to partition a list

review: how to partition a list partitioning a list about an element takes linear time.

select

select handle base case. partition list about first element if pivot p is position i, return pivot else if pivot p is in position > i select else select

Assume our partition always select splits list into two eql parts handle base case. partition list about first element if pivot is position i, return pivot else if pivot is in position > i select else select

problem: what if we always pick bad partitions?

select handle base case. partition list about first element if pivot is position i, return pivot else if pivot is in position > i select else select

select handle base case. partition list about first element if pivot is position i, return pivot else if pivot is in position > i select else select T (n) =T (n 1) + O(n) (n 2 )

Needed: a good partition element partition

Needed: a good partition element partition produce an element where 30% smaller, 30% larger

solution: bootstrap image: gucci image: mark nason image: d&g

partition

partition median of each group form a smaller list select use the median of this smaller list as the partition element

partition 1. 2. 3. 4. 5.

partition divide list into groups of 5 elements find median of each small list gather all medians call select(...) on this sublist to find median return the result

a nice property of our partition

SWITCH TO A BIGGER EXAMPLE

a nice property of our partition < < < < < < < < < < <

a nice property of our partition < < < < < < < < < < < this implies there are at most numbers larger than /smaller

a nice property of our partition

select

select handle base case for small list else pivot = FindPartitionValue(A,n) partition list about pivot if pivot is position i, return pivot else if pivot is in position > i select else select

FindPartition divide list into groups of 5 elements find median of each small list gather all medians call select(...) on this sublist to find median return the result

select handle base case for small list else pivot = FindPartitionValue(A,n) partition list about pivot if pivot is position i, return pivot else if pivot is in position > i select else select

Fast Fourier Transform

big ideas:

f(x) =5+2x + x 2 75 50 25-12.5-10 -7.5-5 -2.5 0 2.5 5 7.5 10

degree polynomial n points on a curve

FFT input: a 0,a 1,a 2,...,a n 1 output:

FFT input: a 0,a 1,a 2,...,a n 1 output: evaluate polynomial A at (any) n different points. n points on a curve

Later, we shall see that the same ideas for FFT can be used to implement Inverse-FFT. Inverse FFT: Given n-points,

Later, we shall see that the same ideas for FFT can be used to implement Inverse-FFT. Inverse FFT: Given n-points, y 0,y 1,...,y n 1 find a degree n polynomial A such that y i = A(! i )

Brute force method to evaluate A at n points:

solve the large problem by solving smaller problems and combining solutions T(n)=

suppose we had already had eval of Ae,Ao on {4,9,16,25} A e (4) A e (9) A e (16) A e (25) A 0 (4) A 0 (9) A 0 (16) A 0 (25)

suppose we had already had eval of Ae,Ao on {4,9,16,25} A e (4) A e (9) A e (16) A e (25) A 0 (4) A 0 (9) A 0 (16) A 0 (25) Then we could compute 8 terms: A(2) = A e (4) + 2A o (4) A( 2) = A e (4) + ( 2)A o (4) A(3) = A e (9) + 3A o (9) A( 3) = A e (9) + ( 3)A o (9) A(4), A(-4), A(5), A(-5)

FFT(f=a[1,...,n]) Evaluates degree n poly on the n th roots of unity

Last remaining issue:

Roots of unity should have n solutions what are they?

Remember this? e 2 i =1

consider the n solutions are: n1,e 2 i/n,e 2 i2/n,e 2 i3/n,...,e 2 i(n 1)/no

the n solutions are: consider for j=0,1,2,3,...,n-1 = is an n th root of unity

What is this number? = is an n th root of unity

Taylor series expansion of a function f around point a f (y) = f (a)+ f 0 (a) (y a)+ f 00 (a) 1! 2! (y a) 2 + f 000 (a) 3! (y a) 2 + e x = around 0

What is this number? = is an n th root of unity e 2pij/n = cos(2pj/n)+i sin(2pj/n)

= is an n th root of unity Lets compute! 1,8

Compute all 8 roots of unity Then graph them

roots of unity should have n solutions e 2pij/n = cos(2pj/n)+i sin(2pj/n)

squaring the n th roots of unity

Thm: Squaring an n th root produces an n/2 th root. 1! 1,8 = p2 + p i example: 2! 2 1,8 = 1 p2 + p i 2 = 2 1 p2 2 +2 1 p2 i p 2 + i p 2 2 =1/2+i 1/2 = i

Thm: Squaring an n th root produces an n/2 th root. n1,e 2 i(1/n),e 2 i(2/n),e 2 i(3/n),...,e 2 i(n/2)/n,e 2 i(n/2+1)/n 2 i(n,...,e 1)/no

Thm: Squaring an n th root produces an n/2 th root. n1,e 2 i(1/n),e 2 i(2/n),e 2 i(3/n),...,e 2 i(n/2)/n,e 2 i(n/2+1)/n 2 i(n,...,e 1)/no 1 e 2 i(1/(n/2)) e 2 i(2/(n/2)) e 2 i(3/(n/2)) 1 e 2 i((n/2)+1/(n/2)) = e 2 i(1+1/(n/2)) =1 e 2 i(1/(n/2))

If n=16

evaluate at a root of unity

evaluate at a root of unity A(! i,n )=A e (! 2 i,n)+! i,n A o (! 2 i,n) n th root of unity n/2 th root of unity n/2 th root of unity

FFT(f=a[1,...,n]) Evaluates degree n poly on the n th roots of unity

FFT(f=a[1,...,n]) Base case if n<=2 E[...] <- FFT(Ae) O[...] <- FFT(Ao) // eval Ae on n/2 roots of unity // eval Ao on n/2 roots of unity combine results using equation: Return n points.

1 1 7 8 9 4 3 2 3 5 7 8 5 3 6 7 7 1 5 6 1 7 8 9

a3 a2 a1 a0 b3 b2 b1 b0 a3b0 a2b0 a1b0 a0b0 a3b1 a2b1 a1b1 a0b1 a3b2 a2b2 a1b2 a0b2 a3b3 a2b3 a1b3 a0b3

A(x) =a 3 x 3 + a 2 x 2 + a 1 x + a 0 B(x) =b 3 x 3 + b 2 x 2 + b 1 x + b 0 C(x) = a 3 b 3 x 6 + (a 3 b 2 + a 2 b 3 )x 5 + (a 3 b 1 + a 2 b 2 + a 1 b 3 )x 4 + (a 3 b 0 + a 2 b 1 + a 1 b 2 + a 0 b 3 )x 3 + (a 2 b 0 + a 1 b 1 + a 0 b 2 )x 2 + (a 1 b 0 + a 0 b 1 )x+ a 0 b 0

y=2x+1 y=x+1

a3 a2 a1 a0 b3 b2 b1 b0 A(x) =a 0 + a 1 x + a 2 x 2 + a 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 B(x) =b 0 + b 1 x + b 2 x 2 + b 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7

a3 a2 a1 a0 b3 b2 b1 b0 A(x) =a 0 + a 1 x + a 2 x 2 + a 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 B(x) =b 0 + b 1 x + b 2 x 2 + b 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 A(! 0 ) A(! 1 ) A(! 2 ) A(! 7 )...

a3 a2 a1 a0 b3 b2 b1 b0 A(x) =a 0 + a 1 x + a 2 x 2 + a 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 B(x) =b 0 + b 1 x + b 2 x 2 + b 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 A(! 0 ) A(! 1 ) A(! 2 ) A(! 7 )... B(! 0 ) B(! 1 ) B(! 2 )... B(! 7 ) FFT FFT

a3 a2 a1 a0 b3 b2 b1 b0 A(x) =a 0 + a 1 x + a 2 x 2 + a 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 B(x) =b 0 + b 1 x + b 2 x 2 + b 3 x 3 +0x 4 +0x 5 +0x 6 +0x 7 A(! 0 ) A(! 1 ) A(! 2 ) A(! 7 )... B(! 0 ) B(! 1 ) B(! 2 )... B(! 7 ) FFT FFT C(! 0 ) C(! 1 ) C(! 2 )... C(! 7 )

application to mult karatsuba 1 7 8 9 1 4 3 2 a b c d

Multiplying n-bit numbers https://en.wikipedia.org/wiki/file:integer_multiplication_by_fft.svg Schönhage Strassen 71 Fürer 07 O(n log n log log n) O(n log(n)2 log (n) )

AGMP-BASEDIMPLEMENTATIONOFSCHÖNHAGE-STRASSEN S LARGE INTEGER MULTIPLICATION ALGORITHM PIERRICK GAUDRY, ALEXANDER KRUPPA, AND PAUL ZIMMERMANN Abstract. Schönhage-Strassen s algorithm is one of the best known algorithms for multiplying large integers. Implementing it efficiently is of utmost importance, since many other algorithms rely on it as a subroutine. We present here an improved implementation, based on the one distributed within the GMP library. The following ideas and techniques were used or tried: faster arithmetic modulo 2 n +1, improved cache locality, Mersenne transforms, Chinese Remainder Reconstruction, the 2trick,Harley sandgranlund stricks, improved tuning. We also discuss some ideas we plan to try in the future. Introduction Since Schönhage and Strassen have shown in 1971 how to multiply two N-bit integers in O(N log N log log N) time[21],severalauthorsshowedhowtoreduceotheroperations inverse, division, square root, gcd, base conversion, elementary functions to multiplication, possibly with log N multiplicative factors [5, 8, 17, 18, 20, 23]. It has now become common practice to express complexities in terms of the cost M(N) to multiply two N-bit numbers, and many researchers tried hard to get the best possible constants in front of M(N) forthe above-mentioned operations (see for example [6, 16]). Strangely, much less effort was made for decreasing the implicit constant in M(N) itself, although any gain on that constant will give a similar gain on all multiplication-based operations. Some authors reported on implementations of large integer arithmetic for specific hardware or as part of a number-theoretic project [2, 10]. In this article we concentrate on the question of an optimized implementation of Schönhage-Strassen s algorithm on a classical workstation.

Applications of FFT

418.222 417.929 418.127 398.417 397.617 401.902 405.7328 414.795 408.15 400.868 411.8386

String matching with * ACAAGATGCCATTGTCCCCCGGCCTCCTGCTGCTGCTGCTCTCCGGGGCCACGGCCACCGCTGCCCTGCC CCTGGAGGGTGGCCCCACCGGCCGAGACAGCGAGCATATGCAGGAAGCGGCAGGAATAAGGAAAAGCAGC CTCCTGACTTTCCTCGCTTGGTGGTTTGAGTGGACCTCCCAGGCCAGTGCCGGGCCCCTCATAGGAGAGG AAGCTCGGGAGGTGGCCAGGCGGCAGGAAGGCGCACCCCCCCAGCAATCCGCGCGCCGGGACAGAATGCC CTGCAGGAACTTCTTCTGGAAGACCTTCTCCTCCTGCAAATAAAACCTCACCCATGAATGCTCACGCAAG TTTAATTACAGACCTGAA Looking for all occurrences of GGC*GAG*C*GC where I don't care what the * symbol is.