CSE Introduction to Parallel Processing. Chapter 3. Parallel Algorithm Complexity

Transcription

1 Dr. Izadi CSE Itroductio to Parallel Processig Chapter 3 Parallel Algorithm Complexity Review algorithm complexity ad various complexity classes Itroduce the otios of time ad time-cost optimality Derive tools for aalyzig, comparig, ad fietuig parallel algorithms

2 Itroductio to Parallel Processig: Algorithms ad Architectures Istructor s Maual, Vol. 2 (4/00), Page Asymptotic Complexity f() = O(g()) if c, 0 such that > 0, f() < c g() f() = Ω(g()) if c, 0 such that > 0, f() > c g() f() = Θ(g()) if c, c', 0 such that > 0, cg() < f() < c'g() c g() c' g() f() g() g() f() c g() f() c g() f() = O(g()) f() = Ω(g()) f() = Θ(g()) Fig Graphical represetatio of the otios of asymptotic complexity. f() = o(g()) < Growth rate strictly less tha f() = O(g()) Growth rate o greater tha f() = Θ(g()) = Growth rate the same as f() = Ω(g()) Growth rate o less tha f() = ω(g()) > Growth rate strictly greater tha

3 Itroductio to Parallel Processig: Algorithms ad Architectures Istructor s Maual, Vol. 2 (4/00), Page 37 Table 3.1. Comparig the Growth Rates of Subliear ad Superliear Fuctios (K = 1000, M = ) Subliear Liear Superliear log 2 log 2 3/ K 1K K 81K 31K K 1.7M 1M K 26M 32M 361 1K 1M 361M 1000M Table 3.2. Effect of Costats o the Growth Rates of Selected Fuctios Ivolvig Costat Factors (K = 1000, M = ) 4 log2 log / K 1K 1K 1K 20K 81K 3.1K 31K 10K 423K 1.7M 10K 1M 100K 6M 26M 32K 32M 1M 90M 361M 100K 1000M Table 3.3. Effect of Costats o the Growth Rates of Selected Fuctios Usig Larger Time Uits ad Roud Figures 4 log2 log / s 2 mi 5 mi 30 s mi 1 hr 15 mi 15 mi 1K 6 hr 1 day 1 hr 9 hr 10K 5 days 20 days 3 hr 10 days 100K 2 mo 1 yr 1 yr 1 yr 1M 3 yr 11 yr 3 yr 32 yr

4 Itroductio to Parallel Processig: Algorithms ad Architectures Istructor s Maual, Vol. 2 (4/00), Page Algorithm Optimality ad Efficiecy f() Ruig time of fastest (possibly ukow) algorithm for solvig a problem g() Ruig time of some algorithm A f() = O(g()) h() Mi time for solvig the problem f() = Ω(h()) g() = h() Algorithm A is time-optimal Redudacy = Utilizatio = 1 A is cost-time optimal Redudacy = Utilizatio = Θ(1) A is cost-time efficiet Lower Bouds Ω (log ) (log 2 Ω ) log Fig Optimal Algorithm? 1996 Daa's Algor. Shiftig Upper Bouds Chi's Bert's Algor. Algor. l Å b 1982 Ae's Algor. e log 2 /log log log log 2 Typical Complexity Classes Upper & lower bouds may tighte over time. Machie or Algorithm A Solutio Machie or Algorithm B

5 Itroductio to Parallel Processig: Algorithms ad Architectures Istructor s Maual, Vol. 2 (4/00), Page 39 Fig Five times fewer steps does ot ecessarily mea five times faster. 3.3 Complexity Classes NP-hard (Itractable?) NP-complete (e.g. the subset sum problem) NP Nodetermiistic Polyomial P Polyomial (Tractable)? P = NP Coceptual view of complexity classes P, NP, NP-complete, ad NP-hard. Example NP(-complete) problem: the subset sum problem Give a set of itegers ad a target sum s, determie if a subset of the itegers i the set add up to s. This problem looks deceptively simple, yet o oe kows how to solve it other tha by tryig practically all of the 2 subsets of the give set.

6 Itroductio to Parallel Processig: Algorithms ad Architectures Istructor s Maual, Vol. 2 (4/00), Page 40 Eve if each of these trials takes oly oe picosecod, the problem is virtually usolvable for = 100.

7 Itroductio to Parallel Processig: Algorithms ad Architectures Istructor s Maual, Vol. 2 (4/00), Page Parallelizable Tasks ad the NC Class NP-hard (Itractable?) NP-complete (e.g. the subset sum problem) P-complete NP Nodetermiistic Polyomial P Polyomial (Tractable)? P = NP NC Nick's Class "efficietly" parallelizable? NC = P Fig A coceptual view of complexity classes ad their relatioships. NC (Nick s class, Niclaus Pippeger) Problems solvable i polylogrithmic time (T = O(log k )) usig a polyomially bouded umber of processors Example P-complete problem: the circuit-value problem Give a logic circuit with kow iputs, determie its output. The circuit-value problem is obvioudly i P, but o geeral algorithm exists for efficiet parallel evaluatio of a circuit s output.

8 Itroductio to Parallel Processig: Algorithms ad Architectures Istructor s Maual, Vol. 2 (4/00), Page Parallel Programmig Paradigms Divide ad coquer Decompose problem of size ito smaller problems Solve the subproblems idepedetly Combie subproblem results ito fial aswer T() = T d () + T s + T c () Decompose Solve i parallel Combie Radomizatio Ofte it is impossible or difficult to decompose a large problem ito subproblems with equal solutio times. I these cases, oe might use radom decisios that lead to good results with very high probability. Example: sortig with radom samplig Other forms of radomizatio: Radom search Cotrol radomizatio Symmetry breakig Approximatio Iterative umerical methods ofte use approximatio to arrive at the solutio(s). Example: Solvig liear systems usig Jacobi relaxatio. Uder proper coditios, the iteratios coverge to the correct solutios; more iteratios more accurate results

9 Itroductio to Parallel Processig: Algorithms ad Architectures Istructor s Maual, Vol. 2 (4/00), Page Solvig Recurreces Solutio via urollig 1. f() = f( 1) + {Rewrite f( 1) as f(( 1) 1) + 1} = f( 2) = f( 3) = f(1) = ( + 1)/2 1 = Θ( 2 ) 2. f() = f(/2) + 1 {Rewrite f(/2) as f((/2)/2 + 1} = f(/4) = f(/8) = f(/) log 2 times = log 2 = Θ(log ) 3. f() = 2f(/2) + 1 = 4f(/4) = 8f(/8) = f(/) + / = 1 = Θ()

10 Itroductio to Parallel Processig: Algorithms ad Architectures Istructor s Maual, Vol. 2 (4/00), Page f() = f(/2) + = f(/4) + /2 + = f(/8) + /4 + / = f(/) /4 + /2 + = 2 2 = Θ() 5. f() = 2f(/2) + = 4f(/4) + + = 8f(/8) = f(/) log 2 times = log 2 = Θ( log ) Alterate solutio for the recurrece f() = 2f(/2) + : Rewrite the recurrece as f() = f(/2) /2 + 1 ad deote f()/ by h() to covert the problem to Example 2 6. f() = f(/2) + log 2 = f(/4) + log 2 (/2) + log 2 = f(/8) + log 2 (/4) + log 2 (/2) + log 2... = f(/) + log log log 2 (/2) + log 2 = log 2 = log 2 (log 2 + 1)/2 = Θ(log 2 )

11 Itroductio to Parallel Processig: Algorithms ad Architectures Istructor s Maual, Vol. 2 (4/00), Page 45 Solutio via guessig Guess the solutio ad verify it by substitutio Substitutio also useful to fid the costat multiplicative factors ad lower-order terms Example: f() = f( 1) + ; guess f( ) = Θ( 2 ) Write f() = a 2 + g(), where g() = o( 2 ) Substitutig i the recurrece equatio, we get: a 2 + g() = a( 1) 2 + g( 1) + This equatio simplifies to: g() = g( 1) + (1 2a) + a Choose a = 1/2 to make g() = o( 2 ) possible g() = g( 1) + 1/2 = /2 1 {g(1) = 0} The solutio to the origial recurrece the becomes f() = 2 /2 + /2 1 Solutio via a basic theorem Theorem 3.1 (basic theorem for recurreces): Give f() = a f(/b) + h(); a, b costat, h a arbitrary fuctio the asymptotic solutio to the recurrece is f() = Θ( logba ) if h() = O( log ba ε ) for some ε > 0 f() = Θ( log ba log ) if h() = Θ( logba ) f() = Θ(h()) if h() = Ω( log ba + ε ) for some ε > 0