Search Algorithms. Analysis of Algorithms. John Reif, Ph.D. Prepared by

Transcription

1 Search Algorithms Analysis of Algorithms Prepared by John Reif, Ph.D.

2 Search Algorithms a) Binary Search: average case b) Interpolation Search c) Unbounded Search (Advanced material)

3 Readings Reading Selection: CLR, Chapter 12

4 Binary Search Trees (in sorted Table of) keys k 0,, k n-1 Binary Search Tree property: at each node x key (x) > key (y) # y nodes on left subtree of x key (x) < key (z) # z nodes on right subtree of x

5 Binary Search Trees (cont d)

6 Binary Search Trees (cont d) Assume 1) Keys inserted into tree in random order 2) Search with all keys equally likely length = # of edges n + 1 = number of leaves Internal path length I = sum of lengths of all internal paths of length > 1 (from root to nonleaves)

7 Binary Search Trees (cont d) External path length E = sum of lengths of all external paths (from root to leaves) = I + 2n

8 Successful Search Expected # comparisons C = 1 + (I/n) n! n-1 " # i + $ % i= 0 & = (C' 1) / n

9 Unsuccessful Search Expected # comparisons C' = E/(n+1) = (I+2n)/(n+1) n = (n C n +n)/(n+1) n-1! " # i + $ % i=0 & = (C' 2) / (n+1) n 2 = 2 ln(n) (i+1) i=0 = log n

10 Model of Random Real Inputs over an Interval Input Set S of n keys each independently randomly chosen over real interval L,U for 0 < L < U Operations Comparison operations +, -, *, % operations: r =largest integer below or equal to r r = smallest integer above of equal to r

11 Sorting and Selection with Random Real Inputs Results 1) Sorting in 0(n) expected time 2) Selection in 0(loglogn) expected time

12 Bucket Sorting with Random Inputs Input set of n keys, S randomly chosen over [L,U] Algorithm BUCKEY-SORT(S): begin end for i = 1 to n do B[i] empty list for to do for $ n(x -L) % ' &! (U-L)! " '" i i = 1 n add x i to B & + 1 i output = 1 to n do sort (B[i]) B[1] B[2] B[n]

13 Bucket Sorting with Random Inputs (cont d) Theorem The expected time T of BUCKET-SORT is 0(n) Proof B[i] is upper bounded by a Binomial 1 variable with parameters n, p = n Hence c>1 i,j Prob { B[i] > j} < c n -j So T n c (jlogj) = O(n) j=0 Generalizes to case keys have nonuniform distribution -j

14 Random Search Table Table X = (x 0 < x 1 < < x n < x n+1 ) where x 1,, x n random reals chosen independently from interval (x 0, x n+1 ) Selection Problem Input key Y Problem find index k* with X k* = Y Note k* has Binomial distribution with parameters n,p = (Y-X 0 )/(X n+1 -X 0 )

15 Algorithm PSEUDO INTERPOLATION-SEARCH (X,Y) Random Table X = (X 0, X 1,, X n, X n+1 ) Algorithm pseudo interpolation search (X,Y) [0] k pn where p = (Y-X 0 )/(X n+1 -X 0 ) [1] if Y = X k then return k

16 Algorithm PSEUDO INTERPOLATION- SEARCH (X,Y) (cont d) [2] If Y > X k then for if k' = k, k+ n, k+2 n,... Y < X then exit with " k'+ n! output pseudo interpolation search (X,Y) where X' = (X k',...,x ) k'+ n

17 Algorithm PSEUDO INTERPOLATION- SEARCH (X,Y) (cont d) [3] Else if Y < X k then for if k' = k, k- n, k-2 n,... Y > X then exit with output pseudo interpolation search (X,Y) where " k'- n! X" = (X,..., X ) k'- n k'

18 Probability Distribution of Pseudo Interpolation Search

19 Probabilistic Analysis of Pseudo Interpolation Search k* is Binomial with mean pn variance σ 2 = p(1-p)n So * k Hence " # pn σ approximates normal as n grows * " # $ % k pn Prob * Z Ψ(Z)/Z σ +, - where Ψ(Z) = 2 -Z 2 e 2π

20 Probabilistic Analysis of Pseudo Interpolation Search (cont d) So Prob ( < i probes used in given call) *! " < Prob( k pn > (i - 2) n ) Ψ(Z )/Z where i i (i - 2) n (i - 2) Z i = = 2(i - 2) σ p(i-p) since p(1-p) 1 4

21 Probabilistic Analysis of Pseudo Interpolation Search (cont d) Lemma C 2.03 where C = expected number of probes in given call proof C = i>1 i>1 i Prob(i probes used) = Prob( i probes used) 2 + Ψ(Z )/Z 2.03 i 3 i i

22 Probabilistic Analysis of Pseudo Interpolation Search (cont d) Theorem Pseudo Interpolation Search has expected time T(n) C loglogn proof T(n) C + T( n ) C loglogn

23 Algorithm INTERPOLATION-SEARCH (X,Y) 1) Initialize k = np comment k = E(k*) 1) If X k = Y then return k 2) If X k < Y then output INTERPOLATION-SEARCH (X,Y) where X = (x k,, x n+1 ) 3) Else X k > Y and output INTERPOLATION-SEARCH (X,Y) where X = (x 0,, x k )

24 Probability Distribution of INTERPOLATION-SEARCH (X,Y)

25 Advanced Material: Probabilistic Analysis of Interpolation Search Lemma Prob( k * " pn # 0( nlogn)) 1 n a where a is a constant Proof Since k* is Binomial with parameters p,n 2 -Z Prob( k * pn Za) 2 e 2 Z 2π 1 n a for σ 2 = p(1-p)n and Z = 0( logn )

26 Probabilistic Analysis of Interpolation Search (cont d) Theorem The expected number of comparisons of Interpolation Search is T(n) loglogn + c (logloglogn) 1 2

27 Probabilistic Analysis of Interpolation Search (cont d) Proof! T(n) 1+ (1-1 n ) T (O( n log n )) + n #! # " a n a $ 1+ loglog n log n + c 1 logloglog n log n ) 2 + o(1) & 1+ log( 1 ' 2 logn ) + + c 1 (logloglogn) 2 * loglogn + c 1 (logloglogn) 2 since log2=1

28 Advance Material: Unbounded Search Input table X[1], X[2], where for j = 1, 2, X[j] =! 0 j < n " $ 1 j n Unbounded Search Problem Find n such that X[n-1] = 0 and X[n]=1 Cost for algorithm A: C A (n)=m if algorithm A uses m evaluations to determine that n is the solution to the unbounded search problem

29 Applications of Unbounded Search 1) Table Look-up in an ordered, infinite table 2) Binary encoding of integers if S n represents integer n, then S n is not a prefix of any S j for n > j {S 1, S 2, } called a prefix set Idea: use S n (b 1, b 2,, b CA (n) ) where b m = 1 if the m th evaluation of X is 1 in algorithm A for unbounded search

30 Unary Unbounded Search Algorithm Algorithm B 0 Try X[1], X[2],, until X[n] = 1 Cost C B0 (n) = n

31 Binary Unbounded Search Algorithm Algorithm B 1 1st stage i try X[2-1] for i=1,2,...,m m until X[2-1]=1 m-1 m ( cost m = logn + 1 where 2 n 2 1)! " m-1 2 nd stage binary search over 2 elements cost log(2 m-1 ) = m-1= logn! " Total Cost C (n) = 2 logn + 1! " B 1

32 Binary Unbounded Search Algorithm (cont d)

33 Double Unbounded Search Search Algorithm B 2 1st stage 1 m1 (2 1) (2 1) try X[2-1],..., X[2 ] = 1 where m = logn (cost is C (m ) = 2 logm + 1) B 1 " 1# 1 " # 2 nd stage same as 2nd stage of B after m was found. Total Cost Cost is C B 0 (n) = m-1= logn C (n) = C (m ) + C (n) B B 1 B = 2( log ( logn+1) + 1 ) + logn 1 " # " " # # " #

34 Double Unbounded Search Algorithm (cont d)

35 Ultimate Unbounded Search Algorithm

36 Search Algorithms Analysis of Algorithms Prepared by John Reif, Ph.D.