The Quest for Efficiency

Transcription

1 CST8130: Data Structures Algorithm Efficiecy ad the Big-O Created by Rex Woollard Use PageUp ad PageDow to move from scree to scree. Olie use arrow buttos. Click o speaker to play soud. The Quest for Efficiecy Why does it matter? Click o each speaker. How do you compare differet algorithms? How do you measure? 1

2 Liear Loops: Case 1 Coutig by 1 for (it i = 0; i < 1000; i++) Liear Loops: Case 1: Doublig Coutig by 1 for (it i = 0; i < 2000; i++) Absolute Measure of Efficiecy: f() = Value of Program Effort (Iteratios) Effect of Doublig Twice Effort Twice Effort 2

3 Liear Loops: Case 2 Coutig by 2 for (it i = 0; i < 1000; i = i + 2) Liear Loops: Case 2: Doublig Coutig by 2 for (it i = 0; i < 2000; i = i + 2) Absolute Measure of Efficiecy: f() = Value of Program Effort (Iteratios) Effect of Doublig Twice Effort Twice Effort 3

4 Big-O Priciple Big-O measures the effect as you scale to larger values of. We do t care that a loop required 1000 iteratios. We do care that the loop required twice as may iteratios whe we doubled. Thus, the previous two cases are deemed to have the same Big-O. Logarithmic Loops: Multiply Double o Each Iteratio for (it i = 1; i <= 1000; i = i * 2) Iteratio exit Value of i

5 Logarithmic Loops: Multiply: Doublig Double o Each Iteratio for (it i = 1; i <= 2000; i = i * 2) Absolute Measure of Efficiecy: f() = log 2 Value of Program Effort (Iteratios) Effect of Doublig Logarithmic Loops: Divide Divide i Half o Each Iteratio for (it i = 1000; i >= 1; i = i / 2) Iteratio exit Value of i

6 Logarithmic Loops: Divide: Doublig Divide i Half o Each Iteratio for (it i = 2000; i >= 1; i = i / 2) Absolute Measure of Efficiecy: f() = log 2 Value of Program Effort (Iteratios) Effect of Doublig So What? Loop goes from 1000 to 1. Obviously, loop that subtracts 1 takes loger tha a loop that divides by 2. Loops are doig differet thigs! Right! Wrog! Same goal for both. Move from 1000 to 1. Algorithm is differet for each loop, but goal is the same. 6

7 So What? Cosider a commo problem. Coil a rope. Coilig Rope: Two Solutios Two differet approaches (algorithms): 1: Loop the rope aroud your arm, oe coil at a time. 2: Divide the rope i half agai ad agai. 7

8 Coilig Rope: Two Solutios: Effort Assume 100 metre rope. Each loop i coil about 1 metre. 1: Loop rope aroud arm: 100 iteratios. 2: Divide rope i half: 6 or 7 iteratios. Radically Differet Efficiecies Same goal. Differet algorithms. Differet efficiecies. 8

9 Nested Loops: Quadratic: Doublig for (it i = 0; i < 1000; i++) for (it j = 0; j < 1000; j++) Absolute Measure of Efficiecy: f() = 2 Value of Program Effort (Iteratios) 1,000,000 4,000,000 16,000,000 64,000, ,000,000 1,024,000,000 Effect of Doublig 4 times effort 4 times effort 4 times effort 4 times effort 4 times effort Nested Loops: Quadratic: Triplig for (it i = 0; i < 1000; i++) for (it j = 0; j < 1000; j++) Absolute Measure of Efficiecy: f() = 2 Value of Program Effort (Iteratios) 1,000,000 9,000,000 81,000, ,000,000 6,561,000,000 Effect of Triplig 9 times effort 9 times effort 9 times effort 9 times effort 9

10 Nested Loops: Quadratic: Quadruplig for (it i = 0; i < 1000; i++) for (it j = 0; j < 1000; j++) Absolute Measure of Efficiecy: f() = 2 Value of Program Effort (Iteratios) 1,000,000 16,000, ,000,000 4,096,000,000 Effect of Quadruplig 16 times effort 16 times effort 16 times effort Calculatig Efficiecy with Nestig Each sigle iteratio through outer loop requires complete set of iteratios of ier loop. Outer loop had efficiecy of. Ier loop had efficiecy of. Combied efficiecy: multiply each ested loop, thus 2. 10

11 Nested Loops: Liear Logarithmic for (it i = 0; i < 1000; i++) for (it j = 1000; j >= 1; j = j / 2) Outer loop has efficiecy of, sice it is a simple coutig loop. Ier loop has efficiecy of log 2, sice it is a divide by 2 loop. Multiply the two efficiecies because of the estig. Absolute Measure of Efficiecy: f() = log 2 Dealig with Icreasig Complexity Simple examples so far. Cosider somethig more complex. Depedet Quadratic for (it i = 1; i <= 10; i++) for (it j = 1; j <= i; j++) Cotrol of ier loop is depedet o value of outer loop variable. 11

12 Nested Loops: Depedet Quadratic for (it i = 1; i <= 10; i++) for (it j = i; j <= i; j++) = 55 Absolute Measure of Efficiecy: f() = ( ( + 1 ) / 2 ) Need to simplify i order to compare algorithms The Big-O What really matters i measurig efficiecy? Order of Magitude Efficiecy: f() = 2 Writte i Big-O: O( 2 ) 12

13 The Big-O: Simplifyig: 1 Commoly Used Big-O Values to Express Algorithm Performace Raked from Best (Fastest) to Worst (Slowest) O(1) O(log 2 ) O() O( log 2 ) O( 2 ) O( 3 )... O( k ) O(2 ) O(!) I each term, set coefficiet of term to 1. Keep largest term ad discard others. How would you simplify Depedet Quadratic? Absolute Measure of Efficiecy: f() = ( ( + 1 ) / 2 ) The Big-O: Simplifyig: 2 How would you simplify Depedet Quadratic? Expad to express all terms without paretheses. Absolute Measure of Efficiecy: f() = ( ( + 1 ) / 2 ) ( ( + 1 ) / 2 ) 13

14 The Big-O: Simplifyig: 3 How would you simplify Depedet Quadratic? Expad to express all terms without paretheses. Absolute Measure of Efficiecy: f() = ( ( + 1 ) / 2 ) ( ( + 1 ) / 2 ) ( / / 2 ) The Big-O: Simplifyig: 4 How would you simplify Depedet Quadratic? Expad to express all terms without paretheses. Absolute Measure of Efficiecy: f() = ( ( + 1 ) / 2 ) ( ( + 1 ) / 2 ) ( / / 2 ) / / 2 14

15 The Big-O: Simplifyig: 5 How would you simplify Depedet Quadratic? Expad to express all terms without paretheses. Absolute Measure of Efficiecy: f() = ( ( + 1 ) / 2 ) ( ( + 1 ) / 2 ) ( / / 2 ) / / 2 2 / 2 + / 2 The Big-O: Simplifyig: 6 I each term, set coefficiet of term to 1. Keep largest term ad discard others. Absolute Measure of Efficiecy: f() = ( ( + 1 ) / 2 ) 2 / 2 + / 2 15

16 The Big-O: Simplifyig: 7 I each term, set coefficiet of term to 1. Keep largest term ad discard others. Absolute Measure of Efficiecy: f() = ( ( + 1 ) / 2 ) 2 / 2 + / The Big-O: Simplifyig: 8 I each term, set coefficiet of term to 1. Keep largest term ad discard others. Absolute Measure of Efficiecy: f() = ( ( + 1 ) / 2 ) 2 / 2 + / Big-O: O( 2 ) 16

17 The Big-O: Practice Simplifyig: 1 Try simplifyig the followig measure of efficiecy to its correspodig Big-O. Absolute Measure of Efficiecy: f() = The Big-O: Practice Simplifyig: 2 Try simplifyig the followig measure of efficiecy to its correspodig Big-O. Absolute Measure of Efficiecy: f() = 6 log

18 Fidig the Big-O: 1 Fid the Big-O for the followig pseudocode algorithm. i = 1 while (i <= ) { j = 1 while (j <= ) { prit (i, j) j = j + 1 i = i + 1 Icremet Loop: Icremet Loop: k = while (k > 0) { prit (k) k = k / 2 Divide by 2 Loop: log 2 Step 1: Fid efficiecy of each loop. Fidig the Big-O: 2 Fid the Big-O for the followig pseudocode algorithm. i = 1 while (i <= ) { j = 1 while (j <= ) { prit (i, j) j = j + 1 i = i + 1 Icremet Loop: Icremet Loop: k = while (k > 0) { prit (k) k = k / 2 Divide by 2 Loop: log 2 Step 1: Fid efficiecy of each loop. Step 2: Combie: Multiply ested, add sequetial. + log 2 18

19 Fidig the Big-O: 3 Fid the Big-O for the followig pseudocode algorithm. i = 1 while (i <= ) { j = 1 while (j <= ) { prit (i, j) j = j + 1 i = i + 1 Icremet Loop: Icremet Loop: k = while (k > 0) { prit (k) k = k / 2 Divide by 2 Loop: log 2 Step 1: Fid efficiecy of each loop. Step 2: Combie: Multiply ested, add sequetial. Step 3: Simplify. + log log 2 Fidig the Big-O: 4 Fid the Big-O for the followig pseudocode algorithm. i = 1 while (i <= ) { j = 1 while (j <= ) { prit (i, j) j = j + 1 i = i + 1 Icremet Loop: Icremet Loop: k = while (k > 0) { prit (k) k = k / 2 Divide by 2 Loop: log 2 Step 1: Fid efficiecy of each loop. Step 2: Combie: Multiply ested, add sequetial. Step 3: Simplify. + log log 2 2 O( 2 ) 19

20 Big-O Graph Curves Processig Steps Needed to Complete Task (Liear Scale) O(2 ) O( 2 ) O( log 2 ) O() O(log 2 ) O(1) Number of Items i Data Set (Liear Scale) Algorithm Efficiecy for of 10,000 Efficiecy Big-O Iteratios Estimated Time Logarithmic O(log 2 ) 14 microsecods Liear O() 10, secods Liear Logarithmic O( log 2 ) 140,000 2 secods Quadratic O( 2 ) 10, miutes Polyomial O( k ) 10,000 k hours Expoetial O(2 ) 2 10,000 itractable Factorial O(!) 10,000! itractable 20

21 Algorithm Efficiecy for of 10,000 Efficiecy Big-O Iteratios Estimated Time Logarithmic O(log 2 ) 14 microsecods Liear O() 10, secods Liear Logarithmic O( log 2 ) 140,000 2 secods Quadratic O( 2 ) 10, miutes Polyomial O( k ) 10,000 k hours Expoetial O(2 ) 2 10,000 itractable Factorial O(!) 10,000! itractable Expoetial case with equal to oly 1,000 (ot 10,000) would take 3.39e+288 years. The age of the uiverse is estimated to be 1.5e+10 years. 21