Data Structures Dr Ahmed Rafat Abas Computer Sciece Dept, Faculty of Computer ad Iformatio, Zagazig Uiversity arabas@zu.edu.eg http://www.arsaliem.faculty.zu.edu.eg/
Algorithm Aalysis Chapter 3
3. Itroductio Algorithm aalysis is a method that measures the efficiecy of: a algorithm, or its implemetatio as a program, whe the iput size becomes large. This method aalyzes: the time required for a algorithm, or its implemetatio as a program, ad the storage space required for a data structure.
Estimatig a algorithm's performace ca be carried out through estimatig the umber of basic operatios required by the algorithm to process a iput of a certai size. The term size is the umber of iputs processed. example: whe comparig sortig algorithms, the size of the problem is measured by the umber of records to be sorted. A basic operatio requires a time to be completed, which does ot deped o the values of the operads of this basic operatio. example: addig two itegers is a basic operatio.
c Example: compute the ruig time of the followig code: Sum=0; For (i=; i<=; i++) For (j=; j<=; j++) Sum++; Assumig the time required for icremetig the sum variable is c, the ruig time is computed as: T( ) c c This equatio describes the growth rate of the ruig time of the previous code.
The cocept of growth rate is importat as it allows us to compare the ruig time of two algorithms without actually writig two programs ad ruig them o the same computer.
A graph illustratig the growth rates for five equatios. The horizotal axis represets iput size. The vertical axis ca represet time, space, or ay other measure of cost.
Slow T() Expoetial growth rate Quadratic growth rate 5log Logarithmic growth rate 0 Liear growth rate 0 Liear growth rate Fast Orderig of the ruig time correspodig to five growth rate equatios from slow to fast.
3. Best, Worst ad Average Cases The best case for a algorithm is the case correspodig to miimum ruig time for this algorithm. The worst case for a algorithm is the case correspodig to maximum ruig time for this algorithm. The average case for a algorithm is the case correspodig to ruig time equal to the average of the best ad the worst cases ruig time. I practice, it is preferred to deal with a worst case aalysis of a algorithm.
3.3 Asymptotic Aalysis 3.3. Upper Bouds The upper boud for the ruig time of a algorithm is the upper boud or the highest growth rate the algorithm ca have. The upper bouds are measured o the best case, average-case or worst case iputs. example: - if the growth rate i the worst case for a algorithm is T( ) c, - the this algorithm has a upper boud to its growth rate of i the worst case. -This phrase ca be rewritte usig a special otatio called the big-oh otatio as follows: this algorithm is i O(), i the worst case.
3.3. Lower Bouds The lower boud for the ruig time of a algorithm is the least amout of time required by this algorithm for some class of iput. This is a measure of the growth rate of this algorithm. The iput ca be the worst, average, or best-case of size. example: - if the lower boud of the ruig time of a algorithm i the worst case is T( ) c, - the this algorithm has a lower boud to its growth rate of i the worst case. - Alteratively, we ca use the big otatio as follows: this algorithm is i, i the worst case. ()
3.3.3 -Notatio Whe the upper ad lower bouds are the same withi a costat factor, this case ca be idicated by the use of (big-theta) otatio. example: a algorithm is said to be i ( h( )) ad it is i ( h( )) ( h( )) if it is Note that the word "i" is dropped for otatio, sice there is a strict equality for two equatios with the same.
3.3.4 Simplifyig Rules I determiig,, from the ruig time equatio, the followig rules are cosidered:. if f () is i ( g( )) ad g () is i ( h( )), the f () is i ( h( )). This rule holds true for ad otatios.
. if f () is i ( Kg( )) for ay costat K 0, the f () is i ( g( )). This rule holds true for ad otatios.
3. if f ( ) is i ( g ( )) ad f ( ) is i ( g ( )), the f ) f ( ) is i g ( ), g ( ))) ( (max(. This rule holds true for ad otatios. 4. if f ( ) is i ( g ( )) ad f ( ) is i ( g ( )), the f ) f ( ) is i g ( ) g ( )) (. ( This rule holds true for ad otatios.
Example: If 4 3, the () T( ) ( 4 ). T is i
3.3.5 Summatios ad Closed Form Solutios Summatio = closed form solutio i f ( i) f () f ()... f ( ) 3 ( ) 3 i, i i i 6 log, i a i i a log,, for 0 a 0
i i, 0 a a a i i, for a i i i, 0 i i log log 0 i i Harmoic series i i, e e log log
3.4 Calculatig The Ruig Time for a Program Example: a=b; the T () is ().
Example: sum=0; for (i=; i<=; i++) sum += ; the T () is ( c c), which is simply ().
Example: Sum = 0; for (j=; j<=; j++) for (i=; i<=j; i++) sum ++; for (k=0; k<; k++) A[k] = k; T () The is ( c j cj c) ( ), which ca be simplified to ( c c c) that is ( ).
Example: Compare the asymptotic aalysis for the followig two code segmets: sum=0; sum=0; for (i=;i<=;i++) for (i=;i<=;i++) for (j=;j<=;j++) for (j=;j<=i;j++) sum++; sum++; the T ( ) is ( c c ), which is simply ( ). ( ) T is ( c ic), which is c ) i is simply ( ). ( ( ) c that
Therefore, both code segmets cost ( ), but the secod requires half the time of the first whe both of them are ru o the same computer.
Example: Compare the ruig time of the followig two code segmets: sum=0; sum=0; for (k=; k<=; k*=) for (k=; k<=; k*=) for (j=; j<=; j++) for (j=; j<=k; j++) sum++; sum++; the, T ( ) is ( c c ), which is ( c c ( log )) log i0 that is ( log ). T i ( ) is ( c c ), which is ( c c ( )) that is (). log i0
3.5 Space Bouds Storage space is also a importat factor for the program efficiecy. The aalysis techiques used to measure space requiremets are similar to those used to measure time requiremets.
Example: The space requiremets for a array of size itegers is ().
There are two importat priciples of algorithm desig: The space/time tradeoff this priciple says that oe ca ofte achieve a reductio i time if oe is willig to sacrifice space or vice versa. The disk-based space/time tradeoff this priciple states that the smaller you ca make your disk storage requiremets, the faster your program will ru.