Iterative Methods for Designing Orthogonal and Biorthogonal Two-channel FIR Filter Banks with Regularities

Transcription

1 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, Abstract Iterative Methods for Designing Orthogonal and Biorthogonal Two-channel FIR Filter Bans with Regularities Robert Bregović and Taio Saramäi Signal Processing Laboratory Tamere University of Technology P. O. Box 553, FIN-33 Tamere, Finland and Efficient iterative methods are described for designing orthogonal and biorthogonal two-channel erfect-reconstruction FIR filter bans in such a way that for the analysis and the synthesis lowass filter the number of zeros at z = are fixed and the energies in the given filter stoband regions are minimized. The regularity of the analysis and the synthesis filter bans resulting when using only half of the tree structure (only the low-ass branch is slit into two branches) are roughly roortional to the number of fixed zeros at z = (vanishing moments) in the analysis and the synthesis filter, resectively. The frequency selectivity of these bans, in turn, is recirocally related to the energies in the filter stoband regions. These two arameters are contradictory. By increasing the number of fixed zeros, the frequency selectivity of the overall filter ban is decreasing and vice versa. Since the selection of these two contradictory arameters deend on the alication for which the filter ban is designed, it is necessary to find comromise solutions between them for every articular case. Using the roosed methods with different design requirements enables us to generate, for both orthogonal and biorthogonal filter bans, all ossible combination between the maximally flat filter bans (maximum number of vanishing moments in the analysis and the synthesis filter) and standard frequency selective filter bans (no regularity requirements). Comaring the roosed method with some existing methods, for a given number of fixed zeros, filter bans with increased regularity and decreased stoband energies are obtained. The efficiency and flexibility of the roosed synthesis techniques are illustrated by means of several examles. Keywords: FIR, Two-channel filter bans, Wavelets, Octave filter bans, Least-squared error, Perfect reconstruction, Vanishing moments, Regularity, De-noising, Orthogonal, Biorthogonal Introduction During the last two decades, two-channel erfect-reconstruction (PR) FIR filter bans have been studied intensively due to their numerous alications [] [3]. They are widely used as building blocs for generating filter bans of the following three basic tyes. First, it is straightforward to generate multi-channel filter bans by building u tree structures using such two-channel filter bans as basic blocs. For these filter bans, the number of filters in both the analysis and synthesis ban is a ower of two and the filter bandwidths are equal. In this case, the equal bandwidths are achieved by using building-bloc two-channel filter bans of different lengths. If only half of the tree structure is used (only the low-ass branch is slit into lowass and higass comonents), octave filter bans are obtained. Usually, at all stages,

2 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, the same two-channel filter ban is used. These octave filter bans are structurally identical with discrete-time wavelet bans being the third tye of filter bans. Desite of the same structure, the design of the building-bloc two-channel filter ban is very different for conventional octave filter bans and discrete-time wavelet bans. For octave filter bans, it is desired that the frequency selectivity of the filters in the overall ban as well as in the building-bloc two-channel filter bans are high. For wavelet bans, due to their different alications, instead of the frequency selectivity, such roerties as the regularity and the number of vanishing moments are of great imortance [4], [5]. The number of vanishing moments is directly the number of zeros being located at z = for the lowass analysis (synthesis) filter. For wavelet bans, the regularity is connected with the number of vanishing moments. It gives the number of continuous derivatives of the corresonding continuous-time mother wavelet. This function can be generated using the rocedure described by Rioul in [6]. In ractice, the regularity is difficult to determine exactly. A good estimate is the lower bound and the uer bound of the Hölder regularity as reorted in [6]. For PR orthogonal wavelets, the regularities of the analysis and the synthesis art are equal whereas for biorthogonal wavelets, the regularity of the analysis art deends on the lowass analysis filter and the regularity of the synthesis art deends on the lowass synthesis filter. Tyically, the regularity of discrete-time wavelets is maximized by designing the analysis (synthesis) lowass filter to have the maximum ossible number of zeros at z =, resulting in a very oor selectivity for the filters in the ban. On the other hand, for selective octave bans the analysis (synthesis) lowass filter is not forced to have any zeros at z = and the regularity becomes unaccetable. The high frequency selectivity and the high regularity (large number of vanishing moments) are thus conflicting requirements. Many authors have show that in order to achieve a good overall erformance a roer comromise between these conflicting measures of goodness is needed. Rioul has studied in [7] the effects of the regularity, the frequency selectivity, and the hase linearity of the subfilters on the erformance of the overall filter ban in image comression. According to his observations, the regularity is the most imortant roerty among the above-mentioned three characteristics, that is, the greater the regularity is, the better comression is achieved. For the same alication, Villasenor, Belzer, and Liao [8] have erformed a survey over 43 different biorthogonal FIR filter bans with less than 36 tas in the analysis/synthesis air. Their results show that filter bans with the highest regularity do not achieve the best erformances. Therefore, the regularity alone is not the best or at least not the sufficient measure of goodness for designing filter bans roviding good image comression roerties. Furthermore, it is nown that the regularity is roughly directly connected with the number of vanishing moments, but, as has been shown by Lang and Heller in [9], a system with the maximum number of vanishing moments does not always result in the largest regularity value. To achieve the maximum ossible regularity for given filter lengths, the use of otimization is necessary. Since the number of vanishing moments is easier to include in to the design rocedure, Balasingham and Ramstad have investigated in [] how the effectiveness of image comression deends on the number of vanishing moment. They have shown that the best results are again not obtained using filter bans with maximum number of vanishing moments (maximal flat filters), but with some intermediate solutions between the maximal flat and frequency-selective filter bans. Similar results can be also obtained in other filter ban alications. Generally, it is not clear what requirements must be taen into account and in what manner to otimize a filter (wavelet) ban to achieve best erformances for some articular alication. In most cases, it is a question of maing a roer comromise between the regularity and the frequency selectivity.

3 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, The urose of this aer is to introduce fast methods for synthesizing building-bloc two-channel PR orthogonal and biorthogonal FIR filter bans in such a way that the resulting octave filter bans rovide tradeoffs between the regularity and the frequency selectivity. Before considering these synthesis techniques, we start with a brief review of existing techniques for both orthogonal and biorthogonal octave filter bans in order to osition the roosed technique among them. First, orthogonal filter bans are considered and, then, biorthonal filter bans are studied. Building blocs for conventional frequency-selective orthogonal FIR filter bans have been introduced by Mintzer [] and Smith and Barnwell []. Using their design techniques, filter bans with subfilters exhibiting a minimax behavior in their stobands are obtained. As shown by authors of this aer in [3], by alying an iterative technique, the subfilters can also be designed to exhibit a least-mean-square behavior in their stobands. Rioul and Duhamel have suggested in [4] a synthesis method for designing orthogonal filter bans with a minimax aroximation in the filter stobands and various regularities. In their synthesis scheme, the number of zeros of the lowass analysis filter being located at z = can be varied and the remaining arameters are otimized to mae the filter amlitude resonse equirile in the given stoband. In [5], Lu has roosed a arameterization method, whereas iterative methods have been described by Lang, Selesnic, Odegard, and Burrus in [6] and Blu in [7]. The synthesis scheme roosed by Blu deals with the design of rational (nonuniform) filter bans. The classical wavelet tye filter bans can be considered as a secial case of rational filter bans. This aer introduce an efficient technique for synthesizing building-bloc orthogonal two-channel FIR filter bans in such a manner that the stoband energy of the analysis lowass filter is minimized for a given number of fixed zeros at z =. Comared with the above-mentioned synthesis schemes, the roosed iterative method is more straightforward to imlement, it is faster, and the convergence to the otimum solution is indeendent of the starting-oint filter ban. Since the stoband energy of the analysis lowass filter is minimized, for the same number of vanishing moments, more regular wavelet bans are obtained. Secial cases of the roosed two-channel filter bans are bans used for generating the Daubechies wavelets [8] and the classical two-channel orthogonal filter bans without any regularity constraints [3]. Techniques for designing PR linear-hase biorthogonal FIR filter bans are based on otimization methods, lie the one resented by Nguyen in [], iterative method resented by Horng and Willson in [], and an imroved combination of the above two techniques described by the authors of this aer in []. All these design schemes result in frequencyselective filters of low regularity and a least-mean-square behavior in their stobands. Coolev, Nishihara, and Sablatash in [3] as well as Zhao and Swamy in [4] have incororated into the biorthogonal filter ban design rocedure regularity constraints to obtain more regular filter bans. In this aer, lie for the orthogonal case, an aroach for designing biorthogonal filter bans is roosed. The stoband energies of the analysis and the synthesis lowass filter are minimized after the numbers of vanishing moments for both filters are selected. The resulting overall filter ban (wavelet) has equal energies in all filter stobands but indeendent regularities for the analysis and the synthesis arts. As for the orthogonal filter bans, two secial cases of the roosed two-channel filter bans are bans used for generating the standard biorthogonal wavelets [5] and the classical two-channel biorthogonal filter bans without any regularity constraints []. An intermediate solution between orthogonal and biorthogonal filter bans, linear-hase nearly orthogonal wavelet bans, has been roosed by Saramäi and Egiazarian in [6]. They use nonlinear otimization to design such filter bans with additionally imosed regularity constraints. They show that such filter bans (wavelets) have a better erformance

4 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, in many alication than classical orthogonal filter bans even if these filter bans suffer from small alias and reconstruction errors. The differences between the orthogonal and the biorthogonal filter bans is that for orthogonal filter bans, only one nonlinear-hase filter has to be designed and the regularity of the analysis and the synthesis filter ban is the same. In the biorthogonal case, different linear-hase filters can be used in the analysis and the synthesis art and the number of zeros at z = may be different for the building-bloc lowass analysis and synthesis filters. This can be utilized to achieve better filter ban roerties. Many authors have noticed that it is beneficial to have different filter lengths for the analysis and the synthesis lowass filters. Usually, filter bans with shorter synthesis lowass filters result in a better overall filter ban erformance. The organization of this aer is as follows: Section introduces the building-bloc twochannel filter ban. PR orthogonal FIR filter bans are considered in Section 3, whereas Section 5 considers PR biorthogonal FIR filter bans. The roosed iterative methods for orthogonal and biorthogonal filter bans are described in Sections 4 and 6, resectively. In Section 7, several design examles and comarisons with filter bans obtained using existing synthesis schemes are included. Finally, conclusions are given in Section 8. Two-Channel Filter Bans This section reviews some basic relations of alias-free two-channel FIR filter bans and introduces a transform that simlifies the filter ban synthesis to be described in the following sections.. Alias-Free Two-Channel Filter Bans The bloc diagram for a two-channel filter ban roviding the basic building-bloc for both the half tree-structured octave filter bans and discrete-time wavelet bans is shown in Figure. This system consists of an analysis ban containing a lowass-highass filter air with transfer functions H (z) and H (z) and down-samling their outut signals by a factor of two and an synthesis ban containing u-samling by a factor of two the two inut signal followed by lowass-highass filter air with transfer functions F (z) and F (z). The oututs of these two filters are added to form the overall outut signal. In ractice, there is a rocessing unit between down- and u-samling oerations for comressing and coding the two signals for transmission and storage uroses. x[n] H (z) H (z) Processing unit F (z) F (z) + y[n] Figure. Two-channel filter ban. By omitting the rocessing unit, the inut and the outut of the overall system are related in the z-domain as [] [3] where the first term Y = T X + A( z) X, () T = [ H F + H( z) F ] (a) is the distortion transfer function and the second term

5 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, A = [ H ( z) F + H( z) F ] (b) is the aliasing transfer function. The last term becomes zero by selecting the synthesis filter transfer functions as giving the following inut-outut relation: where F (z) = H ( z) and F (z) = H ( z) (3) Y = T X, (4) T z) = H H ( z) H H ( ). (5) ( z. Transforms Simlifying the Filter Ban Synthesis In order to simlify the overall roblem definition and the overall filter ban synthesis (esecially for biorthogonal filter ban to be considered in Sections 5 and 6) we use, instead of H (z) and H (z), the following auxiliary transfer functions [3], []: N N n n G = g [ n] z H = h[ n] z (6a) n= n= N N n n n G ( z) = g[ n] z H( z) = ( ) h[ n] z. (6b) n= n= H (e jω ) H (e jω ) π/ π ω +δ δ G (e jω ) H (e j(π ω) ) G (e jω ) H (e jω ) π () ω () ω π/ () ω s () ω s ω Figure. Secifications for G (z) and G (z) and the relations between H (z) and G (z) and H (z) and G (z).

6 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, G (z) and H (z) are thus identical, whereas G (z) = H ( z). Therefore, G (e jω ) = H (e j(π ω ) so that the amlitude resonse of G (z) is obtained from that of H (z) using the substitution π ω ω and vice versa. If ω and ω are the assband and stoband edges of H (z), () () s then the corresonding edges for G (z) are ω ( ) () π ω = and ω ( ) () s π ω s =, resectively. Figure exemlifies the above relations in additional to showing the constraints for G (z) and G (z) to be stated in the design roblems. Hence, the conversion of the design of H (z) to that of G (z) straightforward. Once G (z) has been determined, the corresonding imulseresonse values of H (z) can be determined according to Equation (6b). The advantage of using G (z) and G (z) as rimary transfer functions lies in the fact that both of them are transfer functions of lowass filters, enabling us to treat them in the same way. In terms of G (z) and G (z), the transfer function T(z) taes the following form: T z) = G G G ( z) G ( ). (7) ( z This transfer function can be further simlified after defining the roerties and relations of the transfer functions G (z) and G (z) under consideration. 3 Statement of the Problem for Two-Channel PR Orthogonal FIR Filter Bans This section introduces the PR two-channel FIR filter ban under consideration and states the roblem for otimizing this filter ban. An efficient algorithm for solving this roblem will be described in Section General PR Two-Channel Orthogonal FIR Filter Bans A system deicted in Figure eresents a PR orthogonal filter ban if in addition to the conditions of Equation (3), G (z) and G (z), as given by Equations (6a) and (6b), meet following three conditions: ) N, the order of G (z), is an odd integer. N ) G = z G ( z ). G G G ( z) G N 3) T = ( z) = z. In the above, Condition is necessary due to the fact that a PR orthogonal system can be generated using only odd order filters [], whereas Condition defines the relation between the analysis filters. In terms of imulse-resonse coefficients, the coefficients of G (z) are related to those of G (z) via g [ n] g [ N n] for n =,, L N = (8), maing them time-reversed version of each other. Therefore, only N + coefficients of filter G (z) are the unnowns. Condition 3 means that for the system of Figure the outut and inut signal are related as y[n] = x[n N ], thereby guaranteeing that the PR condition is satisfied. Equivalently, Condition 3 can be exressed, after some maniulations, in terms of the imulse-resonse coefficients of G (z) as for l =, l g[ N + + r l] g[ r] = r / for l = = +, L, ( N ) ( N ). This form will be exloited later on when solving the otimization roblem to be stated for the roosed orthogonal PR filter ban in the following subsections. (9)

7 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, Proosed Orthogonal Filter Ban For the roosed filter ban with the given odd filter order N, it is assumed that G (z) ossesses K fixed zeros at z =. In this case G (z) is exressible as K L ( z ) b[ n] + n G = z () n= where L = N K is the number of remaining adjustable zeros. 3.3 Statement of the Problem The otimization roblem for the roosed two-channel orthogonal filter ban is stated as follows: Given N, K, and ω s, find the L + (L = N K ) adjustable coefficients of G (z), as given by Equation (), to minimize ω ( e ) π G j ε = dω () ω s subject to the condition that the coefficients of G (z) exressed in the direct form of Equation (6a) satisfy the conditions of Equation (9). Solving this roblem results in a PR orthogonal filter ban where G (z) has at least K zeros at z = and its energy is minimized in the given stoband region. 4 Efficient Iterative Algorithm for Solving the Stated Problem for Orthogonal Filter Bans This subsection describes an efficient algorithm for solving the otimization roblem stated in Subsection Reformulation of the Stated Problem According to Equation (), there are only L + unnowns b [n] for G (z) in the stated roblem. Using the vector and matrix notations, the otimization roblem can be restated in the following modified form: Find the unnowns included in the vector to minimize subject to [ [ ] b[] L b[ L ]] T b (a) = b ε (b) = b T Sb Cb = m, (c) where S is an (L +) (L +) Toelitz matrix with the elements given by with and s rl K π K [( ) ] = = cos r l + K ω dω K ω s K = ( K)! ( K )!! (d) (e)

8 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, π ω s cos ( ηω ) m is an (N +)/ length vector given by ( ηω ) sin s for η d ω = η (f) π ω s for η =. [ L ] T m = (g) and C is an (N +)/ (L +) matrix with the elements given by K K c rl = b[ L + r + l] K (h) = for r =,,, (N +)/ and l =,,, (L +). In the above equation, b [n] = for n < and n > L. Since the matrix C deends on the unnowns included in b, the restated otimization roblem cannot be solved directly by alying the Lagrange multilier method [3]. However, this roblem can be avoided by determining C searately. This can be accomlished by starting with roerly determined initial values for the filter coefficients and then using them for calculating the matrix C. After fixing C, the overall roblem becomes solvable with the aid of the Lagrange multilier method. In the sequel, b is used for determining C, whereas b is used as the solution of Lagrange multilier method. Introducing the Lagrange multilier vector λ = [λ λ... λ (N+)/] T, the Lagrangian function for the linearized otimization roblem can be exressed as T T Λ b, λ ) = b Sb λ ( Cb ). (3) ( m Imosing the following necessary and sufficient conditions for the solution: b Λ b, λ) (4a) and ( = λλ( b, λ) =, (4b) results in the following system of linear equations: T S C b = (5) C λ m. Solving this linear system of equations for b, a vector containing the new filter coefficient values is obtained. The next ste involves udating b according to b : = τ b + τ b (6) ( ) where τ ( < τ < ) is the convergence factor. Then, the resulting b is used for regenerating the fixed matrix C. b is determined from Equation (5), and b is udated according to Equation (6). This udating scheme is reeated until b and b become equal according to the redetermined tolerance.

9 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, Proosed Algorithm for Orthogonal Filter Bans The overall algorithm is carried out in the following stes: Ste ) Select b = [ ( L + ) ( L + ) L ( L ) ] T. + Ste ) Calculate the (L +) (L +) matrix S using Equations (d), (e), and (f) and form the (N +)/ length vector m according to Equation (g). Ste 3) Use the old filter coefficient values b to calculate the matrix C with the aid of Equations (h) and (e). Ste 4) Calculate new values for the filter coefficients by solving the system of Equation (5) for b and udate the filter coefficients b using Equation (6) with τ =.7. Ste 5) If b b δ, where δ is a rescribed tolerance, then the filter coefficients included in the vector b are the desired solution. Otherwise, go to Ste 3. It has turned out that very small number of iterations is required by the above algorithm to arrive at the otimum solution. The above algorithm results in a filter ban where the amlitude resonse of H (z) G (z) achieves the value at unity at the zero frequency. When this ban is used as a building bloc for discrete-time wavelet bans, the corresonding value is desired to be. This is achieved by using the substitution [ n] : b[ n] for n,, L and relacing the relations of Equation (3) by b = =..., (7) F (z) = H ( z) and F (z) = H ( z). (8) 5 Statement of the Problem for Two-channel PR linear-hase Biorthogonal FIR filter Bans This section introduces the PR linear-hase biorthogonal two-channel FIR filter ban under consideration and states the roblem for otimizing this filter ban. An efficient algorithm for solving this roblem will be described in Section General PR Two-Channel Biorthogonal Linear-Phase FIR Filter Bans A PR biorthogonal filter ban with linear-hase subfilters G (z) and G (z), as given by Equations (6a) and (6b), satisfies the following conditions [3]: ) The imulse resonses of G (z) and G (z) ossess an even symmetry, that is, g [N n] = g [n] for n =,,, N and g [N n] = g [n] for n =,,, N. ) The sum of the filter orders N and N is two times an odd integer, that is, N + N = K with K being an odd integer. 3) E(z) = G (z)g (z) is a linear-hase half-band FIR filter of order N + N. Here, Condition 3 imlies that E(z) is exressible as where and N + N n E = G G = e[ n] z, (9a) n= [( N + N ) ] e (9b) = [( N + N ) + r] = for r = ±, ±,,( N + N ). e L (9b)

10 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, Only the following two filter classes can meet these conditions []: Tye A: N and N are odd integers and their sum is two times an odd integer K. Tye B: N and N are even integers and their sum is two times an odd integer K. For both Tye A and B, the overall filter ban delay is an odd integer given by K = (N + N )/. The frequency resonses of G (z) for =, are exressible as G ( e where the zero-hase frequency resonse G ˆ ( ω ) are given by jω jωn / ) = e Gˆ ( ω ) (a) ( N )/ g[ ( N ) / n][ cos(( n + / ) ω )] for Tye A ˆ n= G ( ω ) = (b) N / g[ N / ] + [ ][ ] g N / n cos( nω ) for Tye B. n= As shown in [], [3], Condition 3, guaranteeing the PR roerty, can be restated in terms of the imulse-resonse coefficients of G (z) and G (z) as or where l r = l r = g [ l r] g[ r] = δ ( l ( K + ) / ) for l =,, L, Nc (a) g [ l r] g[ r] = δ ( l ( K + ) / ) for l =,, L, N c, (b) N c = N + N ) / 4 (c) ( + and δ (r) = for r = and δ (r) = for r. 5. Proosed Biorthogonal Filter Ban For the roosed filter ban, it is assumed that G (z) ( G (z) ) has K ( K ) fixed zeros at z =. These transfer functions can be written as K L ( + z ) n G = b[ n] z for =, () n= where L = N K, is the number of remaining adjustable zeros for G (z). Because G (z) and G (z) are linear-hase filters, the remaining coefficients satisfy b [L n] = b [n] for n =,,, L and =,. The synthesis of the roosed filter bans can be simlified by utilizing the fact that linear-hase FIR filter transfer functions having an imulse resonse with an even symmetry is characterized by the following fact. Filters of even order have an even number or no zeros at z =, whereas filters of odd order have an odd number of zeros at z = [7]. Therefore, without lost of generality, the following conditions can be stated for Tye A and Tye B designs considered in the revious subsection: Tye A Designs: K and K are odd integers and L = N K and L = N K are even integers. Tye B Designs: K as well as K is either zero or an even integer and L = N K and L = N K are even integers.

11 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, These restrictions enable us to write for both design tyes the zero-hase frequency resonses of the transfer functions G (z) for =,, given by Equation (a), as Gˆ L / ( ) cos( / ) [ / ] [ ] cos[ ( / ) ] = + K ω ω b L b n L nω. (3) n= Hence, after fixing the filter tye as well as the integers N, N, K, and K, there are L / = ( N K ) / + (4) + remaining unnowns for determining the transfer functions G (z) for =,. 5.3 Statement of the Problem The otimization roblem for the roosed two-channel linear-hase biorthogonal filter ban ( ) ( ) is stated as follows: Given the filter tye as well as N, K, ρ, ρ, α, and α s for =,, find the L / + adjustable coefficients of G (z) and the L / + adjustable coefficients of G (z) to minimize the following quantity: where E = α s () ω= ω s + α π () ω ω= ( ) ( ) π [ Gˆ ( ω )] dω + α [ Gˆ ( ω )] () ω [ Gˆ ( ω ) ] dω + α [ Gˆ ( ω ) ] dω, s () ω= ω s ω= s dω ( ) ( ) ( ρ ) π / and ω = ( + ρ ) π / for =, (5a) ω = (5b) s s subject to the constraint that the overall two-channel filter bans satisfies the PR roerty. This means that the coefficients of G (z) and G (z) exressed in the direct form has to satisfy the conditions of Equations (). Solving this roblem results in a PR biorthogonal filter ban with filters with transfer functions G (z) having minimized stoband energies and K fixed zeros at z =. 6 Efficient Iterative Algorithm for Solving the Stated Problem for Biorthogonal Filter Bans This section describes an efficient algorithm for solving the otimization roblem stated in Subsection 5.3. First, the otimization roblem is reformulated in Subsection 6. and, then, an efficient iterative rocedure described in Subsection 6. for solving the stated roblem. 6. Reformulation of the Stated Problem Similar to the orthogonal case (Subsection 4.), the unnown coefficients are included in the following two vectors: [ b [ ] b [] L b [ L / ] ] T b =, for =,. (6a) After some maniulations, Equation (5a) can be re-exressed using the matrix notations as where for =, E = E + E (6b)

12 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, E T ( ) ( ) T ( α ss + α P ) b+ α d + ω = b b. (6c) Here, the d s for =, are vectors of the form with and d [ d [ ] d [] d [ L ] d [ L ] / ] T d = L (6d) [] l = cos[ ( L / l + K / ) ] 4 K K ω K = ω dω for l =,, L / ω (6e) sin( ηω ) for η cos( ηω ) d ω = η (6f) ω for =. η Matrices S () and P () are (L /+) (L /+) Toelitz matrices with the elements given by for r =,, L / and l =,, L / as where s rl srl / for r l, r = L / : = srl / for r l, l = L / and srl / 4 for r = l = L / rl rl / for r l, r = L / : = rl / for r l, l = L / (6g) rl / 4 for r = l = L /, rl K ω ω K [( ) ] [( ) ] = = cos r l + K + + ω dω cos L K r l Kω dω, (6h) K π π K [( ) ] [( ) ] = s = rl cos r l Kω dω cos L K r l Kω dω, (6i) ω s ω s and K (K )! =. (6j) (K )!! In the above, the integral terms can be calculated according to Equations (f) and (6f). The PR roerty imlies that the conditions of Equation () are satisfied. Lie in the orthogonal case, they can be rewritten as and where m is an N c = (N +N +) / 4 length vector of the form C b= m (7a) C b= m (7b) m = [ ] T, and C and C are matrices defined for =, by (7c)

13 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, where the vector with ( ) ( ) ( ) ( ) ( ) ( ) ( ) [ v + v v + v v v v ] C + L L L L + L = L (7d) ( ) v is the lth column of the following matrix: l V ( ) ( ) ( ) [ v v L v ] = (7e) K ( ) K vrl = b[ L + r + l] K (7f) = for r =,,, (N c ) and l=,,..., L. In the above equation, b [n] = for n > L and n < and the integer L is given by L L for = L = (8) L for =. In order to treat both transfer functions G (z) and G (z) in the same manner, Equations (7a) and (7b) are combined to give C b+ Cb= m. (9) In tree-structured filter bans, it is desired to have the filter gain equal to zero at the zero frequency [6] ( for wavelets). For orthogonal filter bans this was always satisfied due to the time-reverse relation between coefficients of the lowass analysis and the lowass synthesis filter. For biorthogonal filter bans, because there is no direct relation between the analysis and the synthesis filter, an extra constraint has to be imosed. The unity gain requirement at the zero frequency imlies G ( ) = for =,. (3) The above equations can be rewritten using the unnown filter coefficients as N l= and, after alying Equation (), as L l= [] l= for =, g (3a) [] l= for =, b. (3b) Taing into account the linear-hase roerty, Equation (3b) is exressible in terms of the vectors b for =, as L T = for =,, (3a) b z where L z is the following vector of length L / + : z T [ L ]. L = (3b) Due to the PR roerty of filter ban it is enough to imose only one out of the two constraints of Equation (3a), that is, the constraint for = or = is used. The second one is automatically satisfied due to the PR constraint. It has been observed that if both constraints are simultaneously alied, then the iterative rocedure to be described in the next

14 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, subsection fails in some cases to give the otimized solution due to the too tight constraints. Exerimental results have shown that it is better to aly the constraint is for the filter with the larger number of unnowns L. Using the above-derived equations, the roblem for the two-channel biorthogonal FIR linear-hase filter ban can be restated in the following modified form: Minimize: subject to and where E T T = Qb+ d b b +ω + ω (33a) () () Cb m = (33b) t T b =, (33c) T T [ ] [ L ] for L L z C C, and t = T T [ L ] for L > L. Q d b Q =, d =, b =, C = (33d) Q d b z with Q = α S + α P and the s for =, being length L / + zero vectors s ( ) ( ) = [ L ]. Since the matrix C deends on the unnowns included in b, the overall roblem cannot be solved directly by alying the Lagrange multilier method. The above roblem has similarities with the one described in Section 4 and can be solved using a similar iterative rocedure. In the sequel, b is used for determining C, whereas b is used as the solution of the Lagrange multilier method. Introducing the Lagrange multilier vectors λ = [λ λ... λ ] T and µ = [µ ], the Lagrangian function for the linearized otimization roblem taes the following form: b T T Λ (, λ, µ ) = b Qb + d b T ( Cb λ m) T ( t T () () µ b ) + ω +. (34) ω Imosing the following necessary and sufficient conditions for the solution: g Λ( b, λ, µ ) =, (35a) and results in the following system of linear equations: T T Q C t b d = C λ m t µ. λλ( b, λ, µ ) =, (35b) µ Λ( b, λ, µ ) =, (35c) Solving this linear system of equations for b, a vector containing the new filter coefficient values is obtained. Using these filter coefficient values, the initial filter coefficient values can be udated by using the following linear udate formula: b : = ( τ ) b + τ b. (37) (36)

15 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, These udated filter coefficients are then used as the new initial values for determining C. Reeating this rocedure several times until the difference between the initial and resulting filter coefficients are within a small tolerance, a very close solution to the original roblem is found. In the above relation, the arameter τ is a convergence factor and it has been observed that for most cases τ =.5 is a good selection. 6. Proosed Algorithm for Biorthogonal Filter Bans The overall algorithm can be carried out in the following stes: Ste ) Use a conventional method to design two linear-hase lowass filters of orders N and N with the given band edges and rescribed numbers of zeros at z =. Ste ) Calculate the matrix Q and the vector d using Equations (6d-j) and (33d) and form the (N +N +)/4 length vector m according to Equation ((7c). Ste 3) Use the old filter coefficient values b to calculate the matrix C with the aid of Equations (7d-f) and (33d). Ste 4) Calculate new values for the filter coefficients by solving the system of Equation (36) for b and udate the filter coefficients using Equation (37). Ste 5) If b b < δ, where δ is a rescribed tolerance, then the filter coefficients included in the vector b are the desired solution. Otherwise, go to Ste 3. Very few iterations are required by the above algorithm to arrive at the otimum solution. Due to the higher non-linearity of the design roblem comared with the orthogonal case, it is necessary to reeat the above design rocedure a several times for different weighting factors and, then, to choose the most aroriate solution. Another aroach would be to utilize a roer non-linear otimization technique to directly arrive at the desired solution. Lie in the orthogonal case, the above algorithm results in a filter ban with normalized filters, that is, G () = for =,. If these filters are used as building bloc for discrete-time wavelets, then the corresonding values are desired to be. This is achieved by first using the substitution [ n] b [ n], for n,,..., L / b = = (38) : for =, and, then, by relacing, for the synthesis filters, the relations of Equation (3) by the relations of Equation (8). 7 Numerical examles This section illustrates, by means of examles, the roerties of the roosed orthogonal and biorthogonal two-channel filter bans and their efficiency in roviding tradeoffs between the regularity and selectivity of the multistage filter bans. In addition, a de-noising examle is included and the roosed orthogonal filter bans are comared with the corresonding minimax solutions of Rioul and Duhamel [4]. Examle. Orthogonal filter bans with N = have been designed for ω s =.64π and for different values of K, the number of vanishing moments. Figure 3 shows the amlitude characteristics of G (z) for the resulting filter bans. For K =, a classical orthogonal irregular filter ban is obtained. As K increases, the stoband attenuation decreases, but simultaneously the regularity of the corresonding wavelet ban increases due to the larger number of vanishing moments. For K = (N +)/ = 6, the lowass analysis filter gives rise to the Daubechies wavelet [8]. In all the cases δ = 5 has been used as the convergence criterion in the algorithm described in Subsection 4.

16 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, Amlitude in db Normalized frequency (w/()) Figure 3. Amlitude resonses for G (z) of order N = for different numbers of vanishing moments: K = (dot-dashed line), K = (solid line), K = 4 (dashed line), and K = 6 (dotted line). Examle. In order to illustrate the tradeoffs between the frequency selectivity of the filter bans and the regularity of corresonding wavelet bans, several filter bans have been designed in the ω s =.64π case for various values of N, the order of the lowass analysis filter and K, the number of vanishing moments. The regularity of the corresonding wavelet ban has been estimated by the lower bound of the Hölder regularity. This lower bound has been determined using the method described by Rioul in [6]. The real regularity is more or less larger than this lower bound. Figure 4 shows the attenuation of G (z) at the stoband edge and the regularity of the resulting wavelet ban for various roosed two-channel filter bans, whereas in Figure 5 as a measure of selectivity the stoband energy of G (z) has been used. These figures rovide information on how to choose the minimum filter order and the number of vanishing moments in order to simultaneously achieve the required regularity and filter selectivity. The aroriate numerical data for some values of N and K are given in Table. The quantities included in the table are: A sb, the attenuation of the first ea and A e, the attenuation at the stoband edge, both given in decibels, A b, the assband rile, E sb, the stoband energy, r, the regularity, and t, the design time in seconds (Pentium II, 333MHz, WIN98). In order to illustrate the efficiency of the filter bans resulting when using the roosed design scheme, these filter bans have been comared in the above case with the corresonding bans of Rioul and Duhamel [4]. The main difference of the Rioul-Duhamel designs comared with the resent solutions is that, instead of the least-mean-square error criterion, the amlitude resonse G (z) is minimized in the given stoband region for the given values of N and K. Table shows for the Rioul-Duhamel designs the same results as resented in Table for the roosed filter bans. The quantities in the table are again, A b, the assband rile, E sb, the stoband energy, r, the regularity, and t, the design time in seconds as well as A m, the minimum stoband attenuation for the lowass analysis filter given in decibels. When comaring Tables and, it is observed that the roosed bans rovide better regularities for the corresonding wavelet bans, and, as can be exected, the stoband energy is lower for the roosed bans.

17 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, N=7 N=9 N=5 K= K=3 K=5 K=7 K=9 Filter attenuation at w=w s 4 3 N=3 N=9 N=5 N= N=7 N=3 K= N=3 N=9 N=5 N= N=7 K= K= Hölder lower bound Figure 4. Comarison between the regularity of the wavelet ban and the attenuation of G (z) at the stoband edge for the roosed two-channel filter bans for various values of N and K. K=6 K=8 K= K= K=4 K= K=5 K=3 Stoband energy N=3 N=7 N= N=5 N=9 N=3 N=7 N=5 N=9 N=3 N=7 N= N=5 K=4 K=6 N=9 K= K= K= K=3 K=8 K=5 K= K=7 K= K=9 K=4 K= K=5 K= Hölder lower bound Figure 5. Comarison between the regularity of the wavelet ban and the stoband energy of G (z) for the roosed two-channel filter bans for various values of N and K.

18 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, Table. Proerties of filter bans designed using the roosed least-mean-square method. Poosed least-mean-square method N K A sb [db] A e [db] A b E sb r t [s] Table. Proerties of filter bans designed using the minimax method [4]. Minimax method [4] N K A m [db] A b E sb r t [s] Examle 3. A tyical alication of wavelet bans in digital signal rocessing is de-noising. The method used in this examle is the one roosed by Donoho and Johnstone in [8]. It consists of three stes: First, the noisy data is transformed into wavelet coefficients by alying a wavelet transform. Second, a soft or hard thresholding is alied to these coefficients resulting in a suression of the coefficients of lower energies. Third, the results are transformed bac in to the original domain using the inverse wavelet transform. To avoid some artifacts near singularities (seudo-gibbs henomenon) an undecimated (stationary, shift invariant) wavelet transform has been used as suggested by Coifman and Donoho in [9]. All the necessary calculations have been erformed using the MATLAB Wavelet toolbox [3] and the Standford WaveLab toolbox for MATLAB [3]. In this examle, 5 level wavelet decomosition has been used followed by hard thresholding. The threshold level was selected as roosed in [8]. A set of four test signals was generated, 48 samles each, as shown in Figure 6. To these signals Gaussian white noise (noise iid N(,) [9]) has been added. Figure 7 shows the resulting noisy signals.

19 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, (a) Doler 3 (b) Blocs 4 (c) Bums (d) Heavisin Figure 6. Test signals for de-noising. (a) Noisy doler 3 (b) Noisy blocs 4 (c) Noisy bums (d) Noisy heavisin Figure 7. Test signals with added Gaussian white noise. For the noisy signals, de-noising was erformed using different well-nown wavelets as well as some filter bans obtained by using the roosed methods. The following wavelets have been considered: sym9 (symlet wavelet, N = 7), coif3 (Coiflet wavelet, N = 7), db9 (Daubechies wavelet, N = 7) and Haar wavelet (N = ) as reresentatives of orthogonal wavelets, and some standard biorthogonal, namely, bior6.8, bior.8, and rbio.8 [3]. These wavelets have been comared in the above de-noising alication with two filter bans (wavelets) designed with the roosed methods. For the first roosed wavelet, denoted by 78, the building bloc two-channel filter ban is the orthogonal one designed for N =, K = 8, L = 9, and ρ = using the algorithm of Subsection 4.. For the second wavelet, denoted by 6, a biorthogonal two-channel filter ban is used. It was generated using N =, K = 6, L = 6, ρ =., N = 6, K =, L = 4, ρ =., α s = 5, α s =, α =, and α = in the algorithm of Subsection 6.. Figures 8 and 9 show the

20 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, amlitude resonses of the resulting building-bloc two-channel filter bans 78 and 6, resectively, whereas Figures and show the corresonding zero locations. Amlitude in db Normalized frequency (ω/(π)) Figure 8. Amlitude resonses of the analysis filters for the orthogonal filter ban 78. Amlitude in db Normalized frequency (ω/(π)) Figure 9. Amlitude resonses of the analysis filters for the biorthogonal filter ban 6. Imaginary Part 8 3 Real Part Figure. Zero locations for the lowass analysis filter of the orthogonal filter ban 78.

21 R. Bregović and T. Saramäi, Iterative methods for designing orthogonal and biorthogonal two-channel FIR filter bans with regularities, Proc. Of Int. Worsho on Sectral Transforms and Logic Design for Future Digital Systems SPECLOG, Tamere, Finland, May, vol. TICSP #, (a) (b) Imaginary Part 6 Imaginary Part 3 Real Part 4 4 Real Part Figure. Zero locations for the analysis filters of the biorthogonal filter ban 6. (a) Lowass analysis filter H (z). (b) Highass analysis filter H (z). In Table 3 quantitative results are given that were obtained by alying the denoising method on the test signals using the above mentioned wavelets (filter bans). For every numerical entry in the table, 4 exeriments were made and then an averaging over the obtained results was erformed. Table 3. Quantitative results for denoising by using different wavelets (filter bans). Filter tye Signal RMSE / MAE Regularity N K L N K L Doler Blocs Bums HeaviSin r a r s Noisy signal 45.3/ / / / sym9.4/ /.4 7.9/.6 9.6/ coif3.66/ /.3 6.6/.4 8.6/ db9.3/.6.63/.9 9.9/.8.78/ Haar 8./ /. 8.3/. 9.8/ /.4.87/.9 9.9/.8.84/ bior6.8.58/.5 6./ / / bior.8.9/.7 4.3/ /. 8./ rbio.8.74/.6 7.4/.5 9.7/ / /.7.63/.7 3.7/ / Orthogonal Biorthog. The numerical erformances of the wavelets have been estimated by evaluating the root mean square error (RMSE) and the maximum absolute error (MAE). These values are given for every signal and every wavelet (filter ban) under consideration. The quantities, denoted by r a and r s in the Table 3, show the regularities of the analysis and the synthesis art, resectively. For orthogonal filter bans, the regularities of the synthesis and the analysis arts are the same. The regularities were calculated using the method described by Rioul in [6] using a 9-level decomosition. It is seen from Table 3 that in order to obtain good results with orthogonal filter bans, the lowass analysis filter must be a mixed-hase filter. It is observed that the db9 filter results in a very oor erformance for Blocs and Bums tyes of signals, whereas sym9, that is a mixed hase design with the most symmetrical version out of the db9 filters, gives a significantly better result. Releasing some constraints on regularity, even better results can be obtained, lie in the case of the filter 78 (a mixed hase filter, see Figure ) for the doler signal. Releasing constraints too much, for achieving a better filter symmetry lie the case of the coif3 wavelet, has an oosite effect, namely, the results become worse. When comaring in Table 3 the biorthogonal wavelets bior.8 and rbio.8 that are built using the same two filters, but in different arrangement (the lowass analysis filter and the lowass synthesis filters are swaed), it is seen that better results are obtained by