CSE 421. Dynamic Programming Shortest Paths with Negative Weights Yin Tat Lee

Transcription

1 CSE 421 Dynamic Programming Shortest Paths with Negative Weights Yin Tat Lee 1

2 Shortest Paths with Neg Edge Weights Given a weighted directed graph G = V, E and a source vertex s, where the weight of edge (u,v) is c u,v (that can be negative) Goal: Find the shortest path from s to all vertices of G. Recall that Dikjstra s Algorithm fails when weights are negative source s 2 3 s

3 Impossibility on Graphs with Neg Cycles Observation: No solution exists if G has a negative cycle. This is because we can minimize the length by going over the cycle again and again. So, suppose G does not have a negative cycle s

4 DP for Shortest Path (First Attempt) Def: Let OPT(v) be the length of the shortest s - v path OPT v = 0 if v = s min OPT u + c u,v u: u,v an edge The formula is correct. But it is not clear how to compute it. 4

5 DP for Shortest Path Def: Let OPT(v, i) be the length of the shortest s - v path with at most i edges. Let us characterize OPT(v, i). Case 1: OPT(v, i) path has less than i edges. Then, OPT v, i = OPT v, i 1. Case 2: OPT(v, i) path has exactly i edges. Let s, v 1, v 2,, v i 1, v be the OPT(v, i) path with i edges. Then, s, v 1,, v i 1 must be the shortest s - v i 1 path with at most i 1 edges. So, OPT v, i = OPT v i 1, i 1 + c vi 1,v 5

6 DP for Shortest Path Def: Let OPT(v, i) be the length of the shortest s - v path with at most i edges. 0 if v = s OPT v, i = if v s, i = 0 min(opt v, i 1, min OPT u, i 1 + c u,v) u: u,v an edge So, for every v, OPT v,? is the shortest path from s to v. But how long do we have to run? Since G has no negative cycle, it has at most n 1 edges. So, OPT(v, n 1) is the answer. 6

7 Bellman Ford Algorithm for v=1 to n if v s then M[v,0]= M[s,0]=0. for i=1 to n-1 for v=1 to n M[v,i]=M[v,i-1] for every edge (u,v) M[v,i]=min(M[v,i], M[u,i-1]+c u,v ) Running Time: O nm Can we test if G has negative cycles? Yes, run for i=1 3n and see if the M[v,n-1] is different from M[v,3n] 7

8 DP Techniques Summary Recipe: Follow the natural induction proof. Find out additional assumptions/variables/subproblems that you need to do the induction Strengthen the hypothesis and define w.r.t. new subproblems Dynamic programming techniques. Ordering is important: longest path in DAG Adding a new variable: knapsack. Dynamic programming over intervals: RNA secondary structure. Top-down vs. bottom-up: Different people have different intuitions Bottom-up is useful to optimize the memory 8

9 CSE 421 Divide and Conquer Ο(n logn) time polynomial multiplication 9

10 Multiplication Polynomials Naïve: (n 2 ) Karatsuba: (n 1.59 ) Best known: (n log n) "Fast Fourier Transform FFT widely used for signal processing Integers Similar, but some ugly details re: carries, etc. due to Schonhage- Strassen in 1971 gives (n log n loglog n) Improvement in 2007 due to Furer gives (n log n 2 log* n ) Used in practice in symbolic manipulation systems like Maple 10

11 Interpolation Given set of values at 5 points 11

12 Interpolation Given set of values at 5 points Can find unique degree 4 polynomial going through these points 12

13 Multiplying Polynomials by Evaluation & Interpolation Any degree n-1 polynomial R(y) is determined by R(y 0 ),... R(y n-1 ) for any n distinct y 0,...,y n-1 To compute PQ (assume degree at most n/2-1) Evaluate P(y 0 ),..., P(y n-1 ) Evaluate Q(y 0 ),...,Q(y n-1 ) Multiply values P(y i )Q(y i ) for i=0,...,n-1 Interpolate to recover PQ 13

14 Interpolation Given values of degree n-1 polynomial R at n distinct points y 0,,y n-1 R(y 0 ),,R(y n-1 ) Compute coefficients c 0,,c n-1 such that R(x)=c 0 +c 1 x+c 2 x 2 + +c n-1 x n-1 System of linear equations in c 0,,c n-1 c 0 +c 1 y 0 +c 2 y c n-1 y n-1 0 =R(y 0 ) known c 0 +c 1 y 1 +c 2 y c n-1 y n-1 1 =R(y 1 ) c 0 +c 1 y n-1 +c 2 y n c n-1 y n-1 n-1 =R(y n-1 ) unknown 14

15 Interpolation: n equations in n unknowns Matrix form of the linear system 1 y 0 y 02 y n-1 0 c 0 R(y 0 ) 1 y 1 y 12 y n-1 1 c 1 R(y 1 ) c 2 =... 1 y n-1 y n-12 y n-1 n-1 c n-1 R(y n-1 ) Fact: Determinant of the matrix is P i j (y i -y j ) which is not 0 since points are distinct System has a unique solution c 0,,c n-1 15

16 Evaluation & Interpolation P: a 0,a 1,...,a n/2-1 Q: b 0,b 1,...,b n/2-1 evaluation at y 0,...,y n-1 O(?) ordinary polynomial multiplication (n 2 ) c k i j k a b i j R:c 0,c 1,...,c n-1 interpolation from y 0,...,y n-1 O(?) P(y 0 ),Q(y 0 ) P(y 1 ),Q(y 1 )... P(y n-1 ),Q(y n-1 ) point-wise multiplication of numbers O(n) R(y 0 ) P(y 0 ) Q(y 0 ) R(y 1 ) P(y 1 ) Q(y 1 )... R(y n-1 ) P(y n-1 ) Q(y n-1 ) 16

17 Karatsuba s algorithm and evaluation and interpolation Strassen gave a way of doing 2x2 matrix multiplies with fewer multiplications Karatsuba s algorithm can be thought of as a way of multiplying degree 1 polynomials (which have 2 coefficients) using fewer multiplications PQ=(P 0 +P 1 z)(q 0 +Q 1 z) = P 0 Q 0 + (P 1 Q 0 +P 0 Q 1 )z + P 1 Q 1 z 2 Evaluate at 0,1,-1 (Could also use other points) A = P(0)Q(0)= P 0 Q 0 C = P(1)Q(1)=(P 0 +P 1 )(Q 0 +Q 1 ) D = P(-1)Q(-1)=(P 0 -P 1 )(Q 0 -Q 1 ) Interpolating, Karatsuba s Mid=(C-D)/2 and B=(C+D)/2-A 17

18 Evaluation at Special Points Evaluation of polynomial at 1 point takes O(n) time So 2n points (naively) takes O(n 2 ) no savings But the algorithm works no matter what the points are So choose points that are related to each other so that evaluation problems can share subproblems 18

19 The key idea: Evaluate at related points P(x) = a 0 +a 1 x+a 2 x 2 +a 3 x 3 +a 4 x a n-1 x n-1 = a 0 +a 2 x 2 +a 4 x a n-2 x n-2 + a 1 x+a 3 x 3 +a 5 x a n-1 x n-1 = P even (x 2 ) + x P odd (x 2 ) P(-x)=a 0 -a 1 x+a 2 x 2 -a 3 x 3 +a 4 x a n-1 x n-1 = a 0 +a 2 x 2 +a 4 x a n-2 x n-2 - (a 1 x+a 3 x 3 +a 5 x a n-1 x n-1 ) = P even (x 2 ) - x P odd (x 2 ) where P even (z) = a 0 +a 2 z +a 4 z a n-2 z n/2-1 and P odd (z) = a 1 +a 3 z +a 5 z a n-1 z n/2-1 19

20 The key idea: Evaluate at related points So if we have half the points as negatives of the other half i.e., y n/2 = -y 0, y n/2+1 = -y 1,,y n-1 = -y n/2-1 then we can reduce the size n problem of evaluating degree n-1 polynomial P at n points to evaluating 2 degree n/2-1 polynomials P even and P odd at n/2 points y 02, y n/2-12 and recombine answers with O(1) extra work per point But to use this idea recursively we need half of y 02, y n/2-1 2 to be negatives of the other half If y n/4 2 = -y 02, say, then (y n/4 /y 0 ) 2 = -1 Motivates use of complex numbers as evaluation points 20

21 Complex Numbers i 2 = -1 e+fi -1 c+di q j i j q a+bi 1 To multiply complex numbers: 1. add angles 2. multiply lengths (all length 1 here) e+fi = (a+bi)(c+di) e 2pi = 1 e pi = -1 -i a+bi =cos q +i sin q = e iq c+di =cos j +i sin j = e ij e+fi =cos (q j) +i sin (q j) = e i(q j) 21

22 Primitive n th root of 1 2p/n w=w n = e i w 3 w 2 =i w Let w = w n = ei 2p /n = cos (2p/n) +i sin (2p/n) w 4 =-1 w 0 =1=w 8 w 5 w 6 = -i w 7 i 2 = -1 e 2p i = 1 22

23 Facts about w=e 2pi /n for even n w = e 2pi /n for i = 1 w n = 1 w n/2 = -1 w n/2+k = - w k for all values of k w 2 = e 2pi / m where m=n/2 w k = cos(2kp/n)+i sin(2kp/n) so can compute with powers of w w k is a root of x n -1= (x-1)(x n-1 +x n ) =0 for k 0, w k 1 so w k(n-1) +w k(n-2) + +1=0 but 23

24 The key idea for n even P(w) = a 0 +a 1 w+a 2 w 2 +a 3 w 3 +a 4 w a n-1 w n-1 = a 0 +a 2 w 2 +a 4 w a n-2 w n-2 + a 1 w+a 3 w 3 +a 5 w a n-1 w n-1 = P even (w 2 ) + w P odd (w 2 ) P(-w)=a 0 -a 1 w+a 2 w 2 -a 3 w 3 +a 4 w a n-1 w n-1 = a 0 +a 2 w 2 +a 4 w a n-2 w n-2 - (a 1 w+a 3 w 3 +a 5 w a n-1 w n-1 ) = P even (w 2 ) - w P odd (w 2 ) where P even (x) = a 0 +a 2 x +a 4 x a n-2 x n/2-1 and P odd (x) = a 1 +a 3 x +a 5 x a n-1 x n/2-1 24

25 The recursive idea for n a power of 2 Goal: Evaluate P at 1,w,w 2,w 3,...,w n-1 Now P even and P odd have degree n/2-1 where P(w k )=P even (w 2k )+w k P odd (w 2k ) P(-w k )=P even (w 2k )-w k P odd (w 2k ) Recursive Algorithm Evaluate P even at 1,w 2,w 4,...,w n-2 Evaluate P odd at 1,w 2,w 4,...,w n-2 Combine to compute P at 1,w,w 2,...,w n/2-1 Combine to compute P at -1,-w,-w 2,...,-w n/2-1 (i.e. at w n/2, w n/2+1, w n/2+2,..., w n-1 ) w 2 is e2pi /m where m=n/2 so problems are of same type but smaller size 25

26 Analysis and more Run-time T(n)=2 T(n/2)+cn so T(n)=O(n log n) So much for evaluation... what about interpolation? Given r 0 =R(1), r 1 =R(w), r 2 =R(w 2 ),..., r n-1 =R(w n-1 ) Compute c 0, c 1,...,c n-1 s.t. R(x)=c 0 +c 1 x+...+c n-1 x n-1 26

27 Interpolation Evaluation: strange but true Non-obvious fact: If we define a new polynomial S(x) = r 0 + r 1 x + r 2 x r n-1 x n-1 where r 0, r 1,..., r n-1 are the evaluations of R at 1, w,..., w n-1 Then c k =S(w -k )/n for k=0,...,n-1 Relies on the fact the interpolation (inverse) matrix has ij entry w -(i+j) /n instead of w i+j So... evaluate S at 1,w -1,w -2,...,w -(n-1) then divide each answer by n to get the c 0,...,c n-1 w -1 behaves just like w did so the same O(n log n) evaluation algorithm applies! 27