Algorithms 演算法. Multi-threaded Algorithms

Transcription

1 演算法 Multi-threaded Professor Chie-Mo James Li 李建模 Graduate Istitute of Electroics Egieerig Natioal aiwa Uiversity Outlie Multithreaded, CH7 7. Basics 7. Matrix Multiplicatio 7.3 Merge sort

2 Leoardo Fiboacci (70-50, Itally) Cosidered by some "the most taleted wester mathematicia of the Middle Ages Fiboacci is best kow for the spreadig of the Hidu-Arabic umeral system i Europe, primarily through the publicatio i the early 3th cetury of his Book of Calculatio, the Liber Abaci. 3 Fiboacci Numbers Fiboacci umbers are defied by recurrece 0,,,, 3, 5, 8, 3,, 34, 55, 89, 44, 33, 377, 60, 987 wo cosecutive "Fiboacci umbers" divided by each other approaches the golde ratio (approximately.68 : ). 4

3 Recursive ree Fig 7. 5 ime complexity Use substitutio method Ruig time is expoetial ( ) = Θ(φ ) + 5 φ =, golde ratio 6

4 Logical parallelism Parallel FIB Spaw: Paret may execute i parallel with its spawed child Syc: procedure must wait for all its spawed childre to complete before proceedig to the ext statemet Dyamic multi-threadig At ru time, a scheduler determies which procedure actually ru o which available processor 7 Ideal parallel computer Igore scheduler cost Assumptios All processors are the same Sequetially cosistet shared memory Memory produces correct results eve if may processors stores ad read at the same time 8

5 Computatio DAG Strad is a group of istructios without parallel cotrol No spaw, syc, retur A vertex represets a strad A edge (u,v) meas istructio u must execute before istructio v Fig 7. If a directed path from u to v, two strads are logically i series Otherwise, they are logically i parallel Procedure istace strad Black: up to lie 3 Shaded: lie 4 White: lie Computatio DAG () Cotiuatio edge: u calls u i the same procedure, horizotal Spaw edge: u spaws v, dowward I parallel Call edge: u calls v (ormal procedure call), also dowward I series Retur edge: retur, upward 0

6 Performace Measures P = ruig time o P processors Work: total time to execute the etire computatio o oe processor Work = Spa: logest time to execute the strads alog ay path Critical path o the dag Spa = = ruig time o ulimited umber of processors Example: assume each strad takes uit time work = = 7 spa= = 8 Work Law: ideal parallel computer with P processors ca do at most P uits of work P P Spa Law: P-processor ideal parallel computer caot ru faster tha a machie wuth ulimited umber of processors P Speedup: how may times faster the computatio is o P processors tha processor speedup = P Θ( P ), liear speedup = P, perfect liear speedup

7 Parallelism: Parallelism ad Slackess Average work performed i parallel alog the critical path Maximum possible speedup Limit o perfect liear speedup parallelsim = Slackess: parallelism divided by P processors slackess = P <, o hope for perfect liear speedup >, ca add more processors to speed up Example: P-FIB, = 7; = 8 Parallelism = 7/8=.5 Whe P 3, slackess <, o more liear speed up 3 ime Complexity of P-FIB ( ) = ( ) = Θ( φ ) ( ) = max( = ( ), ( ) + Θ( ) ( )) + Θ( ) Parallelism is very large, slackess > for practical P-processor ca achieve perfect liear speed up φ = Θ( ) 4

8 Multithreaded, CH7 7. Basics 7. Matrix Multiplicatio 7.3 Merge sort Outlie 5 Matrix-Vector Multiplicatio y=ax A is x matrix; y is -vector; x is -vector Parallel for: iteratios of the respective loops may ru cocurretly i mid i 6

9 Parallelism ( ) = Θ( ( ) = Θ(lg ) + max[ iter ( i )] ) i = Θ(lg ) + Θ( ) = Θ( ) 7 Matrix Multiplicatio C=AB All x matrices ( ) = Θ( ( ) = Θ( ) = Θ( 3 ) ) Ca we do better? 8

10 Divide ad Coquer (7.6) Review ch 4. 9 D&C Matrix Multiplicatio M ( ) = 8M ( / ) + Θ( = Θ( 3 ) ) M ( ) = M ( / ) + Θ( lg 3 = Θ( ) lg see EX ) Better parallelism 0

11 Multithreaded, CH7 7. Basics 7. Matrix Multiplicatio 7.3 Merge sort Outlie Parallel Merge Sort Parallelism = lg Merge is bottleeck Ca we do better? MS' ( ) = MS' ( / ) + Θ( ) MS' MS' MS' = Θ( lg ) ( ) = MS' = Θ( ) ( ) = Θ( lg ( ) ( / ) + Θ( ) )

12 Multithread Merge Fig 7.6 merge [p r ] ad [p r ] x = [q ] = media of [p r ] q = the place where x falls betwee [q -] ad [q ] subarrays [p q -] ad [p q -] are < x subarrays [q - r ] ad [q - r ] are x Copy x ito A[q 3 ] Recursively merge [p q -] with [p q -] ito A[p 3 q 3 -] [q + r ] with [q r ] ito A[q 3 + r 3 ] 3 Parallel Merge PM ( ) = Θ( ) PM ( ) = Θ( lg ) (details see textbook) 4

13 Ruig time is Θ(lg ) Biary Search 5 Parallel Merge Sort better parallelism PMS ( ) = PMS ( / ) + PM ( ) PMS = Θ( lg ) ( ) = PMS = Θ( lg ) PMS( ) = Θ( / lg PMS ( ) 3 ( / ) + PM ) ( ) 6

14 Readig CH 7 7 Schedulig Schedule a strad to a available processor Operate o-lie: o advace kowledge of whe strads will be spawed or completed Greedy scheduler: assig as may strads to processors as possible i each time step Corollary 7.: ruig time of a multithread computatio scheduled by a greedy scheduler is withi a factor of of optimal 8