EFFICIENT REMEZ ALGORITHMS FOR THE DESIGN OF NONRECURSIVE FILTERS

Transcription

1 EFFICIENT REMEZ ALGORITHMS FOR THE DESIGN OF NONRECURSIVE FILTERS Copyright Andreas Antoniou Victoria, BC, Canada July 24, 2007 Frame # 1 Slide # 1 A. Antoniou EFFICIENT REMEZ ALGORITHMS

2 Introduction Two methods have evolved for the design of nonrecursive (FIR) filters over the past 30 years or so: The window method Frame # 2 Slide # 2 A. Antoniou EFFICIENT REMEZ ALGORITHMS

3 Introduction Two methods have evolved for the design of nonrecursive (FIR) filters over the past 30 years or so: The window method The weighted-chebyshev method Frame # 2 Slide # 3 A. Antoniou EFFICIENT REMEZ ALGORITHMS

4 Window Method The window method uses the Fourier series in conjunction with a class of functions known as window functions. Advantages: Closed-form method. Frame # 3 Slide # 4 A. Antoniou EFFICIENT REMEZ ALGORITHMS

5 Window Method The window method uses the Fourier series in conjunction with a class of functions known as window functions. Advantages: Closed-form method. It is easy to apply. Frame # 3 Slide # 5 A. Antoniou EFFICIENT REMEZ ALGORITHMS

6 Window Method The window method uses the Fourier series in conjunction with a class of functions known as window functions. Advantages: Closed-form method. It is easy to apply. The design entails a relatively insignificant amount of computation. Frame # 3 Slide # 6 A. Antoniou EFFICIENT REMEZ ALGORITHMS

7 Window Method Cont d Disadvantages: Designs are suboptimal. Frame # 4 Slide # 7 A. Antoniou EFFICIENT REMEZ ALGORITHMS

8 Window Method Cont d Disadvantages: Designs are suboptimal. A higher-order filter is needed to satisfy a given set of required specifications. Frame # 4 Slide # 8 A. Antoniou EFFICIENT REMEZ ALGORITHMS

9 Window Method Cont d Disadvantages: Designs are suboptimal. A higher-order filter is needed to satisfy a given set of required specifications. A higher-order filter means more computations per sample, which implies that these filters are slower and less efficient in real-time applications. Frame # 4 Slide # 9 A. Antoniou EFFICIENT REMEZ ALGORITHMS

10 Weighted-Chebyshev Method This is an iterative multi-variable optimization method based on the Remez Exchange Algorithm. Advantages: Designs are optimal. Frame # 5 Slide # 10 A. Antoniou EFFICIENT REMEZ ALGORITHMS

11 Weighted-Chebyshev Method This is an iterative multi-variable optimization method based on the Remez Exchange Algorithm. Advantages: Designs are optimal. Method is very flexible - can be used to design filters, differentiators, Hilbert transformers, etc. Frame # 5 Slide # 11 A. Antoniou EFFICIENT REMEZ ALGORITHMS

12 Weighted-Chebyshev Method This is an iterative multi-variable optimization method based on the Remez Exchange Algorithm. Advantages: Designs are optimal. Method is very flexible - can be used to design filters, differentiators, Hilbert transformers, etc. It yields equiripple solutions. Frame # 5 Slide # 12 A. Antoniou EFFICIENT REMEZ ALGORITHMS

13 Weighted-Chebyshev Method This is an iterative multi-variable optimization method based on the Remez Exchange Algorithm. Advantages: Designs are optimal. Method is very flexible - can be used to design filters, differentiators, Hilbert transformers, etc. It yields equiripple solutions. Minimum filter-order is required to satisfy a given set of required specifications. Frame # 5 Slide # 13 A. Antoniou EFFICIENT REMEZ ALGORITHMS

14 Weighted-Chebyshev Method This is an iterative multi-variable optimization method based on the Remez Exchange Algorithm. Advantages: Designs are optimal. Method is very flexible - can be used to design filters, differentiators, Hilbert transformers, etc. It yields equiripple solutions. Minimum filter-order is required to satisfy a given set of required specifications. Minimum filter order implies a more efficient and faster filter for real-time applications. Frame # 5 Slide # 14 A. Antoniou EFFICIENT REMEZ ALGORITHMS

15 Weighted-Chebyshev Method Cont d Disadvantages: Their design requires a very large amount of computation. Frame # 6 Slide # 15 A. Antoniou EFFICIENT REMEZ ALGORITHMS

16 Weighted-Chebyshev Method Cont d Disadvantages: Their design requires a very large amount of computation. Not suitable for applications where the design has to be carried out in real- or quasi-real time, for example, in programable or adaptable filters. Frame # 6 Slide # 16 A. Antoniou EFFICIENT REMEZ ALGORITHMS

17 Objectives The purpose of the lecture is the describe the basics of the weighted-chebyshev method. Frame # 7 Slide # 17 A. Antoniou EFFICIENT REMEZ ALGORITHMS

18 Objectives The purpose of the lecture is the describe the basics of the weighted-chebyshev method. Examine ways by which the efficiency of the design process can be improved and the amount of computation reduced. Frame # 7 Slide # 18 A. Antoniou EFFICIENT REMEZ ALGORITHMS

19 Objectives The purpose of the lecture is the describe the basics of the weighted-chebyshev method. Examine ways by which the efficiency of the design process can be improved and the amount of computation reduced. Suggest possible leads to further research on the subject. Frame # 7 Slide # 19 A. Antoniou EFFICIENT REMEZ ALGORITHMS

20 Historical Evolution Herrmann published a short paper in Electronics Letters in May Frame # 8 Slide # 20 A. Antoniou EFFICIENT REMEZ ALGORITHMS

21 Historical Evolution Herrmann published a short paper in Electronics Letters in May These contributions were followed by a series of papers, during the seventies, by Parks, McClellan, Rabiner, and Herrmann. Frame # 8 Slide # 21 A. Antoniou EFFICIENT REMEZ ALGORITHMS

22 Historical Evolution Herrmann published a short paper in Electronics Letters in May These contributions were followed by a series of papers, during the seventies, by Parks, McClellan, Rabiner, and Herrmann. These developments led, in turn, to the well-known McClellan-Parks-Rabiner computer program for the design of nonrecursive filters which has found widespread applications. Frame # 8 Slide # 22 A. Antoniou EFFICIENT REMEZ ALGORITHMS

23 Historical Evolution Herrmann published a short paper in Electronics Letters in May These contributions were followed by a series of papers, during the seventies, by Parks, McClellan, Rabiner, and Herrmann. These developments led, in turn, to the well-known McClellan-Parks-Rabiner computer program for the design of nonrecursive filters which has found widespread applications. The approach to weighted-chebyshev filters to be presented is based on some of the papers published by McClellan, Parks, and Rabiner and includes several enhancements proposed by the speaker. Frame # 8 Slide # 23 A. Antoniou EFFICIENT REMEZ ALGORITHMS

24 Problem Formulation Consider a nonrecursive filter characterized by the transfer function and assume that N is odd, H(z) = N 1 n=0 h(nt )z n the impulse response is symmetrical, and the sampling frequency is ω s = 2π. Frame # 9 Slide # 24 A. Antoniou EFFICIENT REMEZ ALGORITHMS

25 Problem Formulation Cont d Since T = 2π/ω s = 1 s, the frequency response of the filter can be expressed as H(e jω ) = e jcω P c (ω) where P c (ω) = is the gain function and a 0 = h(c) a k = 2h(c k ) c = (N 1)/2 c a k cos k ω k =0 for k = 1, 2,..., c (A) Note that P c (ω) is the frequency response of a noncausal version of the required filter. Frame # 10 Slide # 25 A. Antoniou EFFICIENT REMEZ ALGORITHMS

26 Error Function If e jcω D(ω) is the idealized frequency response of the desired filter and W (ω) is a weighting function, an error function E (ω) can be constructed as where E (ω) = W (ω)[d(ω) P c (ω)] P c (ω) = c a k cos k ω k =0 Frame # 11 Slide # 26 A. Antoniou EFFICIENT REMEZ ALGORITHMS

27 Error Function If e jcω D(ω) is the idealized frequency response of the desired filter and W (ω) is a weighting function, an error function E (ω) can be constructed as where E (ω) = W (ω)[d(ω) P c (ω)] P c (ω) = c a k cos k ω k =0 If E (ω) is minimized such that E (ω) = W (ω)[d(ω) P c (ω)] δ p for ω (B) with respect to some compact (dense) subset of the frequency interval [0, π], say, a filter can be obtained in which E 0 (ω) = D(ω) P c (ω) δ p W (ω) for ω Frame # 11 Slide # 27 A. Antoniou EFFICIENT REMEZ ALGORITHMS

28 Lowpass Filters In the case of a lowpass filter, the minimization of E (ω) will force the inequality E 0 (ω) = D(ω) P c (ω) δ p W (ω) for ω (C) where D(ω) = { 1 for 0 ω ω p 0 for ω a ω π In effect, a minimization algorithm will force the actual gain function P c (ω) to approach the ideal gain function D(ω). Frame # 12 Slide # 28 A. Antoniou EFFICIENT REMEZ ALGORITHMS

29 Lowpass Filters Cont d 1.0 D(!) P c (!) Gain E 0 (!)! p! a! Frame # 13 Slide # 29 A. Antoniou EFFICIENT REMEZ ALGORITHMS

30 Lowpass Filters Cont d If W (ω) = then from Eq. (C), i.e., { 1 for 0 ω ωp δ p δ a for ω a ω π E 0 (ω) = D(ω) P c (ω) δ p W (ω) for ω we get E 0 (ω) { δ p δ a for 0 ω ω p for ω a ω π Frame # 14 Slide # 30 A. Antoniou EFFICIENT REMEZ ALGORITHMS

31 Lowpass Filters Cont d If W (ω) = then from Eq. (C), i.e., { 1 for 0 ω ωp δ p δ a for ω a ω π E 0 (ω) = D(ω) P c (ω) δ p W (ω) for ω we get E 0 (ω) { δ p δ a for 0 ω ω p for ω a ω π Weighted-Chebyshev filters are so called because they have an equiripple amplitude response just like Chebyshev filters, as shown in the graph. Note: There is no other relation between weighted- Chebyshev and Chebyshev filters! Frame # 14 Slide # 31 A. Antoniou EFFICIENT REMEZ ALGORITHMS

32 Lowpass Filters Cont d 1+δ p δ p D(ω) P c (ω) Gain δ a ω p ω a ω Frame # 15 Slide # 32 A. Antoniou EFFICIENT REMEZ ALGORITHMS

33 Minimax Problem The most appropriate approach for the solution of the optimization problem just described is to solve the minimax problem minimize x {max E (ω) } ω where x =[a 0 a 1 a c ] T The solution of this problem exists by virtue of the so-called alternation theorem. Frame # 16 Slide # 33 A. Antoniou EFFICIENT REMEZ ALGORITHMS

34 Alternation Theorem If P c (ω) is a linear combination of r = c + 1 cosine functions of the form c P c (ω) = a k cos k ω k =0 then a necessary and sufficient condition that P c (ω) be the unique, best, weighted-chebyshev approximation to a continuous function D(ω) on, where is a compact (dense) subset of the frequency interval [0, π], is that the weighted error function E (ω) exhibit at least r + 1 extremal frequencies in, i.e., there must exist at least r + 1 points ˆω i in such that and ˆω 0 < ˆω 1 < < ˆω r E ( ˆω i ) = E ( ˆω i+1 ) for i = 0, 1,..., r 1 E ( ˆω i ) =max E (ω) ω for i = 0, 1,..., r Frame # 17 Slide # 34 A. Antoniou EFFICIENT REMEZ ALGORITHMS

35 Alternation Theorem Cont d From the alternation theorem and Eq. (B), i.e., E (ω) = W (ω)[d(ω) P c (ω)] we can write E ( ˆω i ) = W ( ˆω i )[D( ˆω i ) P c ( ˆω i )]=( 1) i δ for i = 0, 1,..., r, where δ is a constant. Frame # 18 Slide # 35 A. Antoniou EFFICIENT REMEZ ALGORITHMS

36 Alternation Theorem Cont d From the alternation theorem and Eq. (B), i.e., we can write E (ω) = W (ω)[d(ω) P c (ω)] E ( ˆω i ) = W ( ˆω i )[D( ˆω i ) P c ( ˆω i )]=( 1) i δ for i = 0, 1,..., r, where δ is a constant. The above system of equations can be put in matrix form as 1 a 1 cos ˆω 0 cos ˆω 0 cos ˆω 0 D( ˆω 0 ) 0 W ( ˆω 0 ) 1 1 cos ˆω 1 cos ˆω 1 cos ˆω a 1 D( ˆω 1 ) 1 W ( ˆω 1 ). =. ( 1) 1 cos ˆω r cos ˆω r cos ˆω r a c D( ˆω r 1 ) r W ( ˆω r ) δ D( ˆω r ) Frame # 18 Slide # 36 A. Antoniou EFFICIENT REMEZ ALGORITHMS

37 Alternation Theorem Cont d If the extremal frequencies (or extremals for short) were known, coefficients a k and, in turn, the frequency response of the filter could be computed using Eq. (A), i.e., P c (ω) = c a k cos k ω k =0 Frame # 19 Slide # 37 A. Antoniou EFFICIENT REMEZ ALGORITHMS

38 Alternation Theorem Cont d If the extremal frequencies (or extremals for short) were known, coefficients a k and, in turn, the frequency response of the filter could be computed using Eq. (A), i.e., P c (ω) = c a k cos k ω k =0 The solution of this system exists since the above (r + 1) (r + 1) matrix can be shown to be nonsingular. Frame # 19 Slide # 38 A. Antoniou EFFICIENT REMEZ ALGORITHMS

39 Remez Exchange Algorithm The Remez exchange algorithm is an iterative multivariable algorithm which is naturally suited for the solution of the minimax problem just described. It is based on the second optimization method of Remez. See bibliography for details. Frame # 20 Slide # 39 A. Antoniou EFFICIENT REMEZ ALGORITHMS

40 Basic Remez Exchange Algorithm 1. Initialize extremal frequencies ˆω 0, ˆω 1,..., ˆω r and ensure that an extremal is assigned at each band edge. Frame # 21 Slide # 40 A. Antoniou EFFICIENT REMEZ ALGORITHMS

41 Basic Remez Exchange Algorithm 1. Initialize extremal frequencies ˆω 0, ˆω 1,..., ˆω r and ensure that an extremal is assigned at each band edge. 2. Solve the system of equations to get δ and the coefficients a 0, a 1,..., a c. Frame # 21 Slide # 41 A. Antoniou EFFICIENT REMEZ ALGORITHMS

42 Basic Remez Exchange Algorithm 1. Initialize extremal frequencies ˆω 0, ˆω 1,..., ˆω r and ensure that an extremal is assigned at each band edge. 2. Solve the system of equations to get δ and the coefficients a 0, a 1,..., a c. 3. Using the coefficients a 0, a 1,..., a c,calculatep c (ω) and the magnitude of the error E (ω) = W (ω)[d(ω) P c (ω)] Frame # 21 Slide # 42 A. Antoniou EFFICIENT REMEZ ALGORITHMS

43 Basic Remez Exchange Algorithm 1. Initialize extremal frequencies ˆω 0, ˆω 1,..., ˆω r and ensure that an extremal is assigned at each band edge. 2. Solve the system of equations to get δ and the coefficients a 0, a 1,..., a c. 3. Using the coefficients a 0, a 1,..., a c,calculatep c (ω) and the magnitude of the error E (ω) = W (ω)[d(ω) P c (ω)] 4. Locate the frequencies ω 0, ω 1,..., ω ρ at which E (ω) is maximum and E ( ω i ) δ. (These frequencies are potential extremals for the next iteration.) Frame # 21 Slide # 43 A. Antoniou EFFICIENT REMEZ ALGORITHMS

44 Basic Remez Exchange Algorithm Cont d 5. Compute the convergence parameter Q = max E ( ω i ) min E ( ω i ) max E ( ω i ) where i = 0, 1,..., ρ. Frame # 22 Slide # 44 A. Antoniou EFFICIENT REMEZ ALGORITHMS

45 Basic Remez Exchange Algorithm Cont d 5. Compute the convergence parameter Q = max E ( ω i ) min E ( ω i ) max E ( ω i ) where i = 0, 1,..., ρ. 6. Reject ρ r superfluous potential extremals ω i according to an appropriate rejection criterion and renumber the remaining ω i by setting ˆω i = ω i for i = 0, 1,..., r. Frame # 22 Slide # 45 A. Antoniou EFFICIENT REMEZ ALGORITHMS

46 Basic Remez Exchange Algorithm Cont d 5. Compute the convergence parameter Q = max E ( ω i ) min E ( ω i ) max E ( ω i ) where i = 0, 1,..., ρ. 6. Reject ρ r superfluous potential extremals ω i according to an appropriate rejection criterion and renumber the remaining ω i by setting ˆω i = ω i for i = 0, 1,..., r. 7. If Q >ε, where ε is a convergence tolerance (say ε = 0.01), repeat from step 2; otherwise continue to step 8. Frame # 22 Slide # 46 A. Antoniou EFFICIENT REMEZ ALGORITHMS

47 Basic Remez Exchange Algorithm Cont d 5. Compute the convergence parameter Q = max E ( ω i ) min E ( ω i ) max E ( ω i ) where i = 0, 1,..., ρ. 6. Reject ρ r superfluous potential extremals ω i according to an appropriate rejection criterion and renumber the remaining ω i by setting ˆω i = ω i for i = 0, 1,..., r. 7. If Q >ε, where ε is a convergence tolerance (say ε = 0.01), repeat from step 2; otherwise continue to step Compute P c (ω) using the last set of extremal frequencies; then deduce h(n), the impulse response of the required filter, and stop. Frame # 22 Slide # 47 A. Antoniou EFFICIENT REMEZ ALGORITHMS

48 Initialization of Extremal Frequencies Bands: 1 Extremals: r+1 (12) Intervals: r (11) B Bands: 2 Extremals: r+1 (13) Intervals: r-1 (11) W 0 =B 1 /r Bands: 3 Extremals: r+1 (14) Intervals: r-2 (11) W 1 =B 1 /m 1 W 2 =B 2 /m W 2 =B 2 /m 2 W 3 =B 3 /m 3 W 1 =B 1 /m 1 Frame # 23 Slide # 48 A. Antoniou EFFICIENT REMEZ ALGORITHMS

49 Initialization of Extremal Frequencies Cont d For a filter with J bands with bandwidths B 1, B 2,..., B J,the number of extremals and interval between extremals for each band can be calculated by using the following formulas: 1 J W 0 = r + 1 J j=1 ( ) Bj m j = W 0 and B j for j = 1, 2,..., J 1 J 1 m J = r (m j + 1) j=1 W j = B j m j for j = 1, 2,..., J where r = (N + 1)/2 and N is the filter length. Frame # 24 Slide # 49 A. Antoniou EFFICIENT REMEZ ALGORITHMS

50 Updating of Extremals In each iteration the extremals need to be updated this is done by finding the maxima of the error function E (ω) = W (ω)[d(ω) P c (ω)] Frame # 25 Slide # 50 A. Antoniou EFFICIENT REMEZ ALGORITHMS

51 Updating of Extremals In each iteration the extremals need to be updated this is done by finding the maxima of the error function E (ω) = W (ω)[d(ω) P c (ω)] This could be done by solving the system 1 a 1 cos ˆω 0 cos ˆω 0 cos ˆω 0 0 W ( ˆω 0 ) 1 1 cos ˆω 1 cos ˆω 1 cos ˆω a 1 1 W ( ˆω 1 ). = ( 1) 1 cos ˆω r cos ˆω r cos ˆω r a c r W ( ˆω r ) δ for the coefficients a k and then calculating c P c (ω) = a k cos k ω and in turn E (ω). k =0 Frame # 25 Slide # 51 A. Antoniou EFFICIENT REMEZ ALGORITHMS D( ˆω 0 ) D( ˆω 1 ). D( ˆω r 1 ) D( ˆω r )

52 Updating of Extremals Cont d This approach is inefficient and may be subject to numerical ill-conditioning, in particular if δ is small and N is large. Note: A50 50 matrix is quite typical. Frame # 26 Slide # 52 A. Antoniou EFFICIENT REMEZ ALGORITHMS

53 Updating of Extremals Cont d This approach is inefficient and may be subject to numerical ill-conditioning, in particular if δ is small and N is large. Note: A50 50 matrix is quite typical. An alternative and more efficient approach is to deduce δ analytically (by using Cramer s rule) and then interpolate P c (ω) on the r frequency points using the barycentric form of the Lagrange interpolation formula, as follows: Frame # 26 Slide # 53 A. Antoniou EFFICIENT REMEZ ALGORITHMS

54 Updating of Extremals Cont d This approach is inefficient and may be subject to numerical ill-conditioning, in particular if δ is small and N is large. Note: A50 50 matrix is quite typical. An alternative and more efficient approach is to deduce δ analytically (by using Cramer s rule) and then interpolate P c (ω) on the r frequency points using the barycentric form of the Lagrange interpolation formula, as follows: Calculate parameter δ as δ = r k =0 α k D( ˆω k ) r k =0 ( 1)k α k W ( ˆω) Frame # 26 Slide # 54 A. Antoniou EFFICIENT REMEZ ALGORITHMS

55 Updating of Extremals Cont d An alternative and more efficient approach is to deduce δ analytically (by using Cramer s rule) and then interpolate P c (ω) on the r frequency points using the barycentric form of the Lagrange interpolation formula, as follows: Frame # 27 Slide # 55 A. Antoniou EFFICIENT REMEZ ALGORITHMS

56 Updating of Extremals Cont d An alternative and more efficient approach is to deduce δ analytically (by using Cramer s rule) and then interpolate P c (ω) on the r frequency points using the barycentric form of the Lagrange interpolation formula, as follows: Calculate parameter δ as δ = r k =0 α k D( ˆω k ) r k =0 ( 1)k α k W ( ˆω) Frame # 27 Slide # 56 A. Antoniou EFFICIENT REMEZ ALGORITHMS

57 Updating of Extremals Cont d With δ known, P c (ω) can be obtained as where C k r 1 β k C k P c (ω) = x x k k =0 r 1 β k x x k k =0 α k = r i=0, i =k 1 x k x i, and C k = D( ˆω k ) ( 1) k δ W ( ˆω k ) for ω =ˆω 0, ˆω 1,..., ˆω r 1 otherwise β k = r 1 i=0, i =k 1 x k x i with x = cos ω and x i = cos ˆω i for i = 0, 1,..., r Frame # 28 Slide # 57 A. Antoniou EFFICIENT REMEZ ALGORITHMS

58 Rejection of Superfluous Potential Extremals It follows from the alternation theorem that the minimized error function E (ω) has precisely r + 1 extremals where r = (N 1)/2. Note: The problem formulation is such that there must be exactly r + 1 extremals in each iteration. Frame # 29 Slide # 58 A. Antoniou EFFICIENT REMEZ ALGORITHMS

59 Rejection of Superfluous Potential Extremals It follows from the alternation theorem that the minimized error function E (ω) has precisely r + 1 extremals where r = (N 1)/2. Note: The problem formulation is such that there must be exactly r + 1 extremals in each iteration. Analysis will show that E (ω) can have as many as r + 2J 1 maxima where J is the number of bands. If in any iteration the number of maxima exceeds r + 1, then the iteration is said to have generated superfluous potential extremals. Frame # 29 Slide # 59 A. Antoniou EFFICIENT REMEZ ALGORITHMS

60 Rejection of Superfluous Potential Extremals Cont d In the standard McClellan, Rabiner, and Parks algorithm, this difficulty is circumvented by rejecting the ρ r potential extremals ω i that yield the lowest error E (ω). Frame # 30 Slide # 60 A. Antoniou EFFICIENT REMEZ ALGORITHMS

61 Computation of Impulse Response The impulse response in Step 8 of the algorithm can be determined by recalling that function P c (ω) is the frequency response of a noncausal version of the required filter. Frame # 31 Slide # 61 A. Antoniou EFFICIENT REMEZ ALGORITHMS

62 Computation of Impulse Response The impulse response in Step 8 of the algorithm can be determined by recalling that function P c (ω) is the frequency response of a noncausal version of the required filter. The impulse response of the noncausal filter, denoted as h 0 (n) for c n c, can be determined by computing P c (k ) for k = 0, 1,..., c where = 2π/N, and then using the inverse discrete Fourier transform. Frame # 31 Slide # 62 A. Antoniou EFFICIENT REMEZ ALGORITHMS

63 Computation of Impulse Response Cont d It can be shown that { h 0 (n) = h 0 ( n) = 1 P c (0) + N for n = 0, 1,..., c. c ( ) } 2πkn 2P c (k ) cos N k =1 Frame # 32 Slide # 63 A. Antoniou EFFICIENT REMEZ ALGORITHMS

64 Computation of Impulse Response Cont d It can be shown that { h 0 (n) = h 0 ( n) = 1 P c (0) + N c ( ) } 2πkn 2P c (k ) cos N k =1 for n = 0, 1,..., c. The impulse response of the required causal filter is given by h(n) = h 0 (n c) for n = 0, 1,..., c. Frame # 32 Slide # 64 A. Antoniou EFFICIENT REMEZ ALGORITHMS

65 Example Band D(ω) W (ω) Left band edge Right band edge π Sampling frequency: 2π Frame # 33 Slide # 65 A. Antoniou EFFICIENT REMEZ ALGORITHMS

66 Example Cont d E(ω) Frame # 34 Slide # 66 A. Antoniou EFFICIENT REMEZ ALGORITHMS

67 Example Cont d E(ω) Frame # 35 Slide # 67 A. Antoniou EFFICIENT REMEZ ALGORITHMS

68 Example Cont d E(ω) Frame # 36 Slide # 68 A. Antoniou EFFICIENT REMEZ ALGORITHMS

69 Example Cont d E(ω) Frame # 37 Slide # 69 A. Antoniou EFFICIENT REMEZ ALGORITHMS

70 Example Cont d E(ω) Frame # 38 Slide # 70 A. Antoniou EFFICIENT REMEZ ALGORITHMS

71 Example Cont d E(ω) Frame # 39 Slide # 71 A. Antoniou EFFICIENT REMEZ ALGORITHMS

72 Selective Step-by-Step Search When the system of equations 1 a 1 cos ˆω 0 cos ˆω 0 cos ˆω 0 0 W ( ˆω 0 ) 1 1 cos ˆω 1 cos ˆω 1 cos ˆω a 1 1 W ( ˆω 1 ). = ( 1) 1 cos ˆω r cos ˆω r cos ˆω r a c r W ( ˆω r ) δ is solved, the error function E (ω) is forced to satisfy the relation E ( ˆω i ) = W ( ˆω i )[D( ˆω i ) P c ( ˆω i )] = δ D( ˆω 0 ) D( ˆω 1 ). D( ˆω r 1 ) D( ˆω r ) Frame # 40 Slide # 72 A. Antoniou EFFICIENT REMEZ ALGORITHMS

73 Selective Step-by-Step Search When the system of equations 1 a 1 cos ˆω 0 cos ˆω 0 cos ˆω 0 0 W ( ˆω 0 ) 1 1 cos ˆω 1 cos ˆω 1 cos ˆω a 1 1 W ( ˆω 1 ). = ( 1) 1 cos ˆω r cos ˆω r cos ˆω r a c r W ( ˆω r ) δ is solved, the error function E (ω) is forced to satisfy the relation E ( ˆω i ) = W ( ˆω i )[D( ˆω i ) P c ( ˆω i )] = δ This relation can be satisfied in a number of ways but the most likely possibility for the jth band is illustrated in the next slide where ω Lj and ω Rj are the left-hand and right-hand edges, respectively. D( ˆω 0 ) D( ˆω 1 ). D( ˆω r 1 ) D( ˆω r ) Frame # 40 Slide # 73 A. Antoniou EFFICIENT REMEZ ALGORITHMS

74 Selective Step-by-Step Search Cont d E(ω) δ ω Lj ω Rj ωˆ 1j ωˆ 2j ωˆ 3j ω^ 4j ωˆ 5j ωˆ 6j ˆ ω 7j Frame # 41 Slide # 74 A. Antoniou EFFICIENT REMEZ ALGORITHMS

75 Selective Step-by-Step Search Cont d Because of the special nature of the error function (a) the maxima of E (ω) can be easily found by searching in the vicinity of the extremals; (b) gradient information can be used to expedite the search for the maxima of E (ω) ; and (c) the closer we get to the solution, the closer are the maxima of the error function to the extremals. By using a selective step-by-step search, a large amount of computation can be eliminated. Frame # 42 Slide # 75 A. Antoniou EFFICIENT REMEZ ALGORITHMS

76 Selective Step-by-Step Search Cont d Extra ripples can arise in the first and last bands.: E(ω) δ 0 π ωˆ 1j ωˆ 2j ˆ ω (μj 1)J ωˆ μjj (b) (c) δ 0 ωˆ 1j ωˆ 2j (d) ω ˆ ω (μj 1)J (e) ωˆ μjj ω π Frame # 43 Slide # 76 A. Antoniou EFFICIENT REMEZ ALGORITHMS

77 Selective Step-by-Step Search Cont d Also in interior bands: E(ω) δ ω Lj ω Rj ωˆ 1j ωˆ 2j ωˆ 3j (f) (g) ˆω (μj 1)j ωˆ μjj δ ω Lj ω Rj ωˆ 1j ωˆ 2j (h) ˆω (μj 1)j (i) ωˆ μjj Frame # 44 Slide # 77 A. Antoniou EFFICIENT REMEZ ALGORITHMS

78 Cubic Interpolation Search Increased computational efficiency can be achieved by using a search based on cubic interpolation. Frame # 45 Slide # 78 A. Antoniou EFFICIENT REMEZ ALGORITHMS

79 Cubic Interpolation Search Increased computational efficiency can be achieved by using a search based on cubic interpolation. Assuming that the error function shown in the figure can be represented by the third-order polynomial E (ω) =M = a + bω + cω 2 + d ω 3 where a, b, c, and d are constants then dm = G = b + 2cω + 3d ω2 d ω Hence, the frequencies at which M has stationary points are given by ω = 1 [ c ± ] (c 2 3bd ) 3d Frame # 45 Slide # 79 A. Antoniou EFFICIENT REMEZ ALGORITHMS

80 Cubic Interpolation Search Increased computational efficiency can be achieved by using a search based on cubic interpolation. Assuming that the error function shown in the figure can be represented by the third-order polynomial E (ω) =M = a + bω + cω 2 + d ω 3 where a, b, c, and d are constants then dm = G = b + 2cω + 3d ω2 d ω Hence, the frequencies at which M has stationary points are given by ω = 1 [ c ± ] (c 2 3bd ) 3d Therefore, E (ω) has a maximum if d 2 M d ω 2 = 2c + 6d ω<0 or ω< c 3d Frame # 45 Slide # 80 A. Antoniou EFFICIENT REMEZ ALGORITHMS

81 Cubic Interpolation Search Cont d E(ω) ~ ~ ~ ω 1 ω 2 ω 3 ω ω Frame # 46 Slide # 81 A. Antoniou EFFICIENT REMEZ ALGORITHMS

82 Cubic Interpolation Search Cont d The cubic interpolation method requires four function evaluations per potential extremal consistently. Frame # 47 Slide # 82 A. Antoniou EFFICIENT REMEZ ALGORITHMS

83 Cubic Interpolation Search Cont d The cubic interpolation method requires four function evaluations per potential extremal consistently. The selective step-by-step search may require as many as 8 function evaluations per potential extremal in the first two or three iterations but as the solution is approached only two or three function evaluations are required. Frame # 47 Slide # 83 A. Antoniou EFFICIENT REMEZ ALGORITHMS

84 Cubic Interpolation Search Cont d The cubic interpolation method requires four function evaluations per potential extremal consistently. The selective step-by-step search may require as many as 8 function evaluations per potential extremal in the first two or three iterations but as the solution is approached only two or three function evaluations are required. By using the cubic interpolation to start with and then switching over to the step-by-step search, an very efficient algorithm can be constructed. Frame # 47 Slide # 84 A. Antoniou EFFICIENT REMEZ ALGORITHMS

85 Cubic Interpolation Search Cont d The cubic interpolation method requires four function evaluations per potential extremal consistently. The selective step-by-step search may require as many as 8 function evaluations per potential extremal in the first two or three iterations but as the solution is approached only two or three function evaluations are required. By using the cubic interpolation to start with and then switching over to the step-by-step search, an very efficient algorithm can be constructed. The decision to switch from cubic to selective can be based on the value of the convergence parameter Q (see Step 5). Switching from the cubic to the selective when Q is reduced below 0.65 works well. Frame # 47 Slide # 85 A. Antoniou EFFICIENT REMEZ ALGORITHMS

86 Improved Rejection Scheme If an extremal does not move from one iteration to the next, then the minimum value of E ( ω i ) is simply δ, as can be easily shown, and this happens quite often even in the first or second iteration of the Remez algorithm. Frame # 48 Slide # 86 A. Antoniou EFFICIENT REMEZ ALGORITHMS

87 Improved Rejection Scheme If an extremal does not move from one iteration to the next, then the minimum value of E ( ω i ) is simply δ, as can be easily shown, and this happens quite often even in the first or second iteration of the Remez algorithm. As a consequence, rejecting potential extremals on the basis of the individual values of E ( ω i ) tends to become random and this can slow the Remez algorithm quite significantly particularly for multiband filters. Frame # 48 Slide # 87 A. Antoniou EFFICIENT REMEZ ALGORITHMS

88 Improved Rejection Scheme If an extremal does not move from one iteration to the next, then the minimum value of E ( ω i ) is simply δ, as can be easily shown, and this happens quite often even in the first or second iteration of the Remez algorithm. As a consequence, rejecting potential extremals on the basis of the individual values of E ( ω i ) tends to become random and this can slow the Remez algorithm quite significantly particularly for multiband filters. An improved scheme for the rejection of superfluous extremals based the rejection on the lowest average band error as well as the individual values of E ( ω i ) is described in the next transparency. Frame # 48 Slide # 88 A. Antoniou EFFICIENT REMEZ ALGORITHMS

89 Improved Rejection Scheme Cont d Compute the average band errors E j = 1 ν j E ( ω i ) ω i j for j = 1, 2,..., J where j is the set of extremals in band j given by j ={ ω i : ω Lj ω i ω Rj } ν j is the number of potential extremals in band j, and J is the number of bands. Frame # 49 Slide # 89 A. Antoniou EFFICIENT REMEZ ALGORITHMS

90 Improved Rejection Scheme Cont d Compute the average band errors E j = 1 ν j E ( ω i ) ω i j for j = 1, 2,..., J where j is the set of extremals in band j given by j ={ ω i : ω Lj ω i ω Rj } ν j is the number of potential extremals in band j, and J is the number of bands. Rank the J bands in the order of lowest average error and let l 1, l 2,..., l J be the ranked list obtained, i.e., l 1 and l J are the bands with the lowest and highest average error, respectively. Frame # 49 Slide # 90 A. Antoniou EFFICIENT REMEZ ALGORITHMS

91 Improved Rejection Scheme Cont d Reject one ω i in each of bands l 1, l 2,..., l J 1, l 1, l 2,... until ρ r superfluous ω i are rejected. In each case, reject the ω i, other than a band edge, that yields the lowest E ( ω i ) in the band. Example: If J = 3, ρ r = 3, and the average errors for bands 1, 2, and 3 are 0.05, 0.08, and 0.02, then ω i are rejected in bands 3, 1, and 3. Note: The potential extremals are not rejected in band 2 which is the band of highest average error. Frame # 50 Slide # 91 A. Antoniou EFFICIENT REMEZ ALGORITHMS

92 Example Band D(ω) W (ω) Left band edge Right band edge π Sampling frequency: 2π Frame # 51 Slide # 92 A. Antoniou EFFICIENT REMEZ ALGORITHMS

93 Example Cont d E(ω) Frame # 52 Slide # 93 A. Antoniou EFFICIENT REMEZ ALGORITHMS

94 Example Cont d E(ω) Frame # 53 Slide # 94 A. Antoniou EFFICIENT REMEZ ALGORITHMS

95 Example Cont d E(ω) Frame # 54 Slide # 95 A. Antoniou EFFICIENT REMEZ ALGORITHMS

96 Example Cont d E(ω) Frame # 55 Slide # 96 A. Antoniou EFFICIENT REMEZ ALGORITHMS

97 Example Cont d E(ω) Frame # 56 Slide # 97 A. Antoniou EFFICIENT REMEZ ALGORITHMS

98 Example Cont d E(ω) Frame # 57 Slide # 98 A. Antoniou EFFICIENT REMEZ ALGORITHMS

99 Example Cont d E(ω) Frame # 58 Slide # 99 A. Antoniou EFFICIENT REMEZ ALGORITHMS

100 Comparisons Amount of Computation Type of No. of Range Ave. Funct. Evals. Saving, % Filter Examples of N A B C CvB CvA LP HP BP BS A: Exhaustive search B: Selective search C: Selective plus cubic search Frame # 59 Slide # 100 A. Antoniou EFFICIENT REMEZ ALGORITHMS

101 Comparisons Robustness Type of No. of No. Failures Filter Examples A B C LP HP BP BS A: Exhaustive search B: Selective search C: Selective plus cubic search Frame # 60 Slide # 101 A. Antoniou EFFICIENT REMEZ ALGORITHMS

102 Prescribed Specifications Given a filter length N, a set of passband and stopband edges, and a ratio δ p /δ a, a nonrecursive filter with approximately piecewise-constant amplitude-response specifications can be readily designed. Frame # 61 Slide # 102 A. Antoniou EFFICIENT REMEZ ALGORITHMS

103 Prescribed Specifications Given a filter length N, a set of passband and stopband edges, and a ratio δ p /δ a, a nonrecursive filter with approximately piecewise-constant amplitude-response specifications can be readily designed. While the filter obtained will have passband and stopband edges at the correct locations and the ratio δ p /δ a will be as exactly as required, the amplitudes of the passband and stopband ripples are highly unlikely to have the specified values. Frame # 61 Slide # 103 A. Antoniou EFFICIENT REMEZ ALGORITHMS

104 Prescribed Specifications Given a filter length N, a set of passband and stopband edges, and a ratio δ p /δ a, a nonrecursive filter with approximately piecewise-constant amplitude-response specifications can be readily designed. While the filter obtained will have passband and stopband edges at the correct locations and the ratio δ p /δ a will be as exactly as required, the amplitudes of the passband and stopband ripples are highly unlikely to have the specified values. An acceptable design can be obtained by predicting the value of N on the basis of the required specifications and then designing filters for increasing or decreasing values of N until the lowest value of N that satisfies the specifications is found. Frame # 61 Slide # 104 A. Antoniou EFFICIENT REMEZ ALGORITHMS

105 Filter Length Prediction A reasonably accurate empirical formula for the prediction of the required filter length, N, for the case of lowpass and highpass filters, due to Herrmann, Rabiner, and Chan, is where B = ω a ω p /2π [ (D FB 2 ) N = int B ] D =[ (log 10 δ p ) log 10 δ p ] log 10 δ a [ (log 10 δ p ) log 10 δ p ] F = (log 10 δ p log 10 δ a ) Frame # 62 Slide # 105 A. Antoniou EFFICIENT REMEZ ALGORITHMS

106 Filter Length Prediction The formula of Herrmann et al. can also be used to predict the filter length in the design of bandpass, bandstop, and multiband filters in general. Frame # 63 Slide # 106 A. Antoniou EFFICIENT REMEZ ALGORITHMS

107 Filter Length Prediction The formula of Herrmann et al. can also be used to predict the filter length in the design of bandpass, bandstop, and multiband filters in general. In these filters, a value of N is computed for each transition band between a passband and stopband or a stopband and passband and the largest value of N so obtained is taken to be the predicted filter length. Frame # 63 Slide # 107 A. Antoniou EFFICIENT REMEZ ALGORITHMS

108 Algorithm 1. Compute N using the prediction formula of Herrmann et al.; if N is even, set N = N Design a filter of length N using the Remez algorithm and determine the minimum value of δ, say δ. (A) If δ>δ p, then do: (a) Set N = N + 2, design a filter of length N using the Remez algorithm, and find δ ; (b) If δ δ p, then go to step 3; else, go to step 2(A)(a). (B) If δ<δ p, then do: (a) Set N = N 2, design a filter of length N using the Remez algorithm, and find δ ; (b) If δ>δ p, then go to step 4; else, go to step 2(B)(a). Frame # 64 Slide # 108 A. Antoniou EFFICIENT REMEZ ALGORITHMS

109 Algorithm Cont d 3. If part A of the algorithm was executed, use the last set of extremals and the corresponding value of N to obtain the impulse response of the required filter and stop. 4. If part B of the algorithm was executed, use the last but one set of extremals and the corresponding value of N to obtain the impulse response of the required filter and stop. Frame # 65 Slide # 109 A. Antoniou EFFICIENT REMEZ ALGORITHMS

110 Example In an application, a nonrecursive equiripple bandstop filter is required which should satisfy the following specifications: Odd filter length Maximum passband ripple A p : 0.5 db Minimum stopband attenuation A a : 50.0 db Lower passband edge ω p1 : 0.8 rad/s Upper passband edge ω p2 : 2.2 rad/s Lower stopband edge ω a1 : 1.2 rad/s Upper stopband edge ω a2 : 1.8 rad/s Sampling frequency ω s : 2π rad/s Design the lowest-order filter that will satisfy the specifications. Frame # 66 Slide # 110 A. Antoniou EFFICIENT REMEZ ALGORITHMS

111 Example Cont d The design algorithm gave a filter with the following specifications: Passband ripple: db Minimum stopband attenuation: db Progress of Algorithm N Iters. FE s A p, db A a, db Frame # 67 Slide # 111 A. Antoniou EFFICIENT REMEZ ALGORITHMS

112 Example Cont d Gain, db ω, rad/s Note: Passband errors multiplied by a factor of 40. Frame # 68 Slide # 112 A. Antoniou EFFICIENT REMEZ ALGORITHMS

113 D-Filter A DSP software package that incorporates the design techniques described in this presentation is D-Filter. Please see for more information. Frame # 69 Slide # 113 A. Antoniou EFFICIENT REMEZ ALGORITHMS

114 Conclusions Three techniques that bring about substantial improvements in the efficiency of the Remez algorithm have been described: A step-by-step exhaustive search A cubic interpolation search An improved scheme for the rejection of superfluous potential extremals Frame # 70 Slide # 114 A. Antoniou EFFICIENT REMEZ ALGORITHMS

115 Conclusions Three techniques that bring about substantial improvements in the efficiency of the Remez algorithm have been described: A step-by-step exhaustive search A cubic interpolation search An improved scheme for the rejection of superfluous potential extremals Extensive experimentation has shown that the selective and cubic interpolation searches reduce the amount of computation required by the Remez algorithm by almost 90% without degrading its robustness. Frame # 70 Slide # 115 A. Antoniou EFFICIENT REMEZ ALGORITHMS

116 Conclusions Cont d The rejection scheme described increases the efficiency and robustness of the Remez algorithm further but the scheme has not been compared with the original method of McClellan, Rabiner, and Parks. Frame # 71 Slide # 116 A. Antoniou EFFICIENT REMEZ ALGORITHMS

117 Conclusions Cont d The rejection scheme described increases the efficiency and robustness of the Remez algorithm further but the scheme has not been compared with the original method of McClellan, Rabiner, and Parks. By using a prediction technique for the required filter length proposed by Herrmann, Rabiner, and Chan, filters that satisfy prescribed specifications can be designed. Frame # 71 Slide # 117 A. Antoniou EFFICIENT REMEZ ALGORITHMS

118 Conclusions Cont d The rejection scheme described increases the efficiency and robustness of the Remez algorithm further but the scheme has not been compared with the original method of McClellan, Rabiner, and Parks. By using a prediction technique for the required filter length proposed by Herrmann, Rabiner, and Chan, filters that satisfy prescribed specifications can be designed. For off-line applications, the Remez algorithm continues to be the method of choice for the design of linear-phase filters, multiband filters, differentiators, Hilbert transformers. Frame # 71 Slide # 118 A. Antoniou EFFICIENT REMEZ ALGORITHMS

119 Conclusions Cont d Despite the improvements described, the Remez algorithm continues to require a large amount of computation. For applications that need the filter to be designed on-line in real or quasi-real time, the window method is preferred although the filters obtained are suboptimal. Frame # 72 Slide # 119 A. Antoniou EFFICIENT REMEZ ALGORITHMS

120 References A. Antoniou, Accelerated procedure for the design of equiripple nonrecursive digital Filters, IEE Proc., vol. 129, pt. G, pp. 1 10, February 1982 (see IEE Proc., vol. 129, pt. G, p. 107, June 1982 for errata). A. Antoniou, New improved method for the design of weighted-chebyshev, nonrecursive, digital filters, IEEE Trans. Circuits Syst., vol. CAS-30, pp , October O. Herrmann, Design of nonrecursive digital filters with linear phase, Electron.Lett., vol. 6, pp , May E. Hofstetter, A. Oppenheim, and J. Siegel, A new technique for the design of non-recursive digital Filters, 5th Annual Princeton Conf. Information Sciences and Systems, pp , March Frame # 73 Slide # 120 A. Antoniou EFFICIENT REMEZ ALGORITHMS

121 References Cont d J. H. McClellan and T. W. Parks, A unified approach to the design of optimum FIR linear-phase digital filters, IEEE Trans. Circuit Theory, vol. CT-20, pp , November J. H. McClellan, T. W. Parks, and L. R. Rabiner, A computer program for designing optimum FIR linear phase digital filters, IEEE Trans. Audio Electroacoust., vol. AU-21, pp , December J. H. McClellan, T. W. Parks, and L. R. Rabiner, FIR linear phase filter design program, Programs for digital signal processing, IEEE Press, New York, pp , T. W. Parks and J. H. McClellan, Chebyshev approximation for nonrecursive digital filters with linear phase, IEEE Trans. Circuit Theory, vol. CT-19, pp , March Frame # 74 Slide # 121 A. Antoniou EFFICIENT REMEZ ALGORITHMS

122 References Cont d T. W. Parks and J. H. McClellan, A program for the design of linear phase finite impulse response digital filters, IEEE Trans. Audio Electroacoust., vol. AU-20, pp , August L. R. Rabiner and O. Herrmann, On the design of optimum FIR low-pass filters with even impulse response duration, IEEE Trans. Audio Electroacoust., vol. AU-21, pp , August L. R. Rabiner, J. H. McClellan, and T. W. Parks, FIR digital filter design techniques using weighted chebyshev approximation, Proc. IEEE, vol. 63, pp , April E. Ya. Remes, General Computational Methods for Tchebycheff Approximation, Kiev, 1957 (Atomic Energy Commission Translation 4491, pp. 1 85). Frame # 75 Slide # 122 A. Antoniou EFFICIENT REMEZ ALGORITHMS

123 References Cont d O. Herrmann, L. R. Rabiner, and D. S. K. Chan, Practical design rules for optimum finite impulse response low-pass digital filters, Bell Syst. Tech. J., vol. 52, pp , Jul.-Aug D. J. Shpak and A. Antoniou, A generalized Reméz method for the design of FIR digital filters, IEEE Trans. Circuits Syst., vol. CAS-37, pp , February Frame # 76 Slide # 123 A. Antoniou EFFICIENT REMEZ ALGORITHMS

124 This slide concludes the presentation. Thank you for your attention. Frame # 77 Slide # 124 A. Antoniou EFFICIENT REMEZ ALGORITHMS