Wavelets and Filter Banks Course Notes

Transcription

1 Página Web 1 de 2 Next: Contents Contents Wavelets and Filter Banks Course Notes Copyright Dr. W. J. Phillips January 9, 2003 Contents 1. Analysis and Synthesis of Signals 2. Time-Frequency Analysis 2.1 The Short Time Fourier Transform 2.2 The spectrogram 2.3 An Orthgonal Basis of Functions 3. Time-Scale Analysis 3.1 The Continuous Wavelet Transform 3.2 Comparision with STFT 3.3 The Scalogram 3.4 Examples of Wavelets 3.5 Analysis and Synthesis with Wavelets 3.6 The Haar Wavelet 4. Multiresolution Analysis 4.1 The Scaling Function 4.2 The Discrete Wavelet Transform 5. Filter Banks and the Discrete Wavelet Transform 5.1 Analysis: From Fine Scale to Coarser Scale Filtering and Downsampling The One-Stage Analysis Filter Bank The Analysis Filter Bank 5.2 Synthesis: From Course Scale to Fine Scale Upsampling and Filtering The One-Stage Synthesis Filter Bank Perfect Reconstruction Filter Bank The Synthesis Filter Bank Approximations and Details 5.3 Numerical Complexity of the Discrete Wavelet Transform 5.4 Matlab Examples One-Stage Perfect Reconstruction Approximations and Details A Useful Function 5.5 Initialization of the Discrete Wavelet Transform 6. Properties of the Filters, and the Scale and Wavelet Functions 6.1 Double Shift Orthogonality of the Filters 6.2 Frequency Domain Formulas 6.3 Support of the Scale Function 6.4 The Cascade Algorithm 7. Designing Wavelets 7.1 Short Filters Length 2 Filter

2 Página Web 2 de Length 4 Filter Length 6 Filter 7.2 K-Regular Scaling Filters The db2 Wavelet The db3 Wavelet 7.3 Characterizing K-Regular Filters 7.4 The Daubechies Maximally Flat Polynomial Factoring the Daubechie Maximally Flat Polynomial 7.5 Coiflets Coif Coif2 About this document... Dr. W. J. Phillips

3 Página Web 1 de 1 Next: 1. Analysis and Synthesis Up: Wavelets and Filter Banks Previous: Wavelets and Filter Banks Contents Contents 1. Analysis and Synthesis of Signals 2. Time-Frequency Analysis 2.1 The Short Time Fourier Transform 2.2 The spectrogram 2.3 An Orthgonal Basis of Functions 3. Time-Scale Analysis 3.1 The Continuous Wavelet Transform 3.2 Comparision with STFT 3.3 The Scalogram 3.4 Examples of Wavelets 3.5 Analysis and Synthesis with Wavelets 3.6 The Haar Wavelet 4. Multiresolution Analysis 4.1 The Scaling Function 4.2 The Discrete Wavelet Transform 5. Filter Banks and the Discrete Wavelet Transform 5.1 Analysis: From Fine Scale to Coarser Scale 5.2 Synthesis: From Course Scale to Fine Scale 5.3 Numerical Complexity of the Discrete Wavelet Transform 5.4 Matlab Examples 5.5 Initialization of the Discrete Wavelet Transform 6. Properties of the Filters, and the Scale and Wavelet Functions 6.1 Double Shift Orthogonality of the Filters 6.2 Frequency Domain Formulas 6.3 Support of the Scale Function 6.4 The Cascade Algorithm 7. Designing Wavelets 7.1 Short Filters 7.2 K-Regular Scaling Filters 7.3 Characterizing K-Regular Filters 7.4 The Daubechies Maximally Flat Polynomial 7.5 Coiflets Dr. W. J. Phillips

4 Página Web 1 de 7 Next: 2. Time-Frequency Analysis Up: Wavelets and Filter Banks Previous: Contents Contents 1. Analysis and Synthesis of Signals Signal Spaces Signals defined on the time interval can be added, subtracted and multiplied by constants. This should remind us of vectors. In fact if we take samples of the signal at times for, then a signal,, gives rise to a vector. Given two signals, and, the dot product of the corresponding vectors, and is: In this sense we can regard the integral of times as a sort of dot product of the two signals. Note that the analogy is not exact except in the limit where two signals as:. We define the inner product of If and are complex signals then we use the complex conjugate of in computing the inner product: Note that in the wedge bracket notation for the inner product the variable is not necessary as the integration is over the variable. We use this notation so that we can write things like. In analogy with the length of a vector we define the norm of a the signal to be: Note that the square of the norm of a signal is the energy in the siganl (think of the signal as a voltage driving a 1 ohm resistor).

5 Página Web 2 de 7 Given the analogy with vectors, we define two signals and to be orthogonal if Orthogonal Vectors in the Plane There is a simple formula for resolving a vector, vectors and., in the plane in terms of a a pair of orthogonal Recall that we can project onto each of the vectors and obtaining multiples and from which we can synthesize. This geometric construction can be obtained through dot products. By taking the dot product of first with we have: So, we can solve for Similarily, by taking the inner product of with we have:

6 Página Web 3 de 7 Recall that we say that the vectors and are an orthogonal basis for the set of vectors in the plane. When we compute the coefficients and we say that we are analyzing the vector in terms of the basis and. When we express the vector as the basis. we say that we are synthesising the vector from That is, we have two processes going on: Orthogonal Basis of Signals In the case of signals we say that a set of signals (where ranges over some infinite set of values like from to ) is an orthgonal basis provided: 1. The signals are mutually orthgonal. That is, if. 2. The signals are complete in the sense that the only signal,, which is orthgonal to all is the zero signal. That is, if for all then. Generalized Analysis and Synthesis Given an orthogonal basis of signals we can then analyze any signal in terms of the basis and synthesize the signal back again:

7 Página Web 4 de 7 The Analysis coefficients equation is called the Generalized Fourier Series. are called the Generalized Fourier Coefficients and the Synthesis To see that the Generalized Fourier Series does converge to the original signal let's call the signal which it does converge to. That is, Now consider the difference orthogonality of the basis shows that:. A straight forward calculation using the The completeness condition implies that must be zero or that. Parseval's Theorem The energy in the signal can be obtained from the Generalized Fourier Coefficents just as in the case of Fourier Series. That is, where

8 Página Web 5 de 7 Example: Fourier Series If we have a signal defined on then we know that we can compute its complex Fourier Series ( ): The functions are orthogonal and the formula for is the usual analysis formula: Note that the functions are solutions to the boundary value problem,. Example: Cosine Basis If we have defined on then we can extend to be an even function on and then its Fourier Series, with period orthgonal basis of functions on given by: has only cosine terms. This gives an The Analysis-Synthesis equations are:

9 Página Web 6 de 7 Note that completes 1/2 of a cycle on the interval. In general completes cycles on the interval. Note that the functions are solutions to the boundary value problem,. Example: 1/4 Cycle Cosine Basis Given defined on we can extend evenly to. We can then extend oddly across and to obtain an even function on. The Fourier Series ( has only cosine terms but because of the odd extensions at and the terms of even frequency have zero amplitude. This gives an orthogonal basis of functions on given by: Note that completes 1/4 of a cycle on the interval. To simplify the notation let's define: With this definition completes cycles on. The Analysis-Synthesis equations are: Note that the functions are solutions to the boundary value problem,.

10 Página Web 7 de 7 Example: Sinc Basis The Shannon Sampling Theorem gives an orthogonal basis of sinc functions. Recall that if is band limited to Hz and we take samples of with a sampling interval then To see that this is the Analysis/Synthesis formula in terms of inner products, recall that we have the Fourier Transform pair: So, Using the fact that the Fourier Transform preserves inner products we can show that the sinc functions are orthogonal. Also, the Analysis formula becomes: Next: 2. Time-Frequency Analysis Up: Wavelets and Filter Banks Previous: Contents Contents Dr. W. J. Phillips

11 Página Web 1 de 8 Next: 3. Time-Scale Analysis Up: Wavelets and Filter Banks Previous: 1. Analysis and Synthesis Contents Subsections 2.1 The Short Time Fourier Transform 2.2 The spectrogram 2.3 An Orthgonal Basis of Functions 2. Time-Frequency Analysis In many applications such as speech processing, we are interested in the frequency content of a signal locally in time. That is, the signal parameters (frequency content etc.) evolve over time. Such signals are called non-stationary. For a non-stationary signal,, the standard Fourier Transform is not useful for analyzing the signal. Information which is localized in time such as spikes and high frequency bursts cannot be easily detected from the Fourier Transform. Time-localization can be achieved by first windowing the signal so as to cut off only a well-localized slice of and then taking its Fourier Transform. This gives rise to the Short Time Fourier Transform, (STFT) or Windowed Fourier Transform. The magnitude of the STFT is called the spectrogram. By restricting to a discrete range of frequencies and times we can obtain an orthogonal basis of functions. 2.1 The Short Time Fourier Transform The Short Time Fourier Transform of a signal using a window function is defined as follows. Think of the window as sliding along the signal and for each shift we compute the usual Fourier Transform of the product function. For example, if is the box of width 1/2 then we have (see the Matlab m-file fig1.m):

12 Página Web 2 de 8 In the frequency domain we can use the convolution theorem to recognize as the convolution of with the Fourier transform of (which is ). Recall that we have the Fourier Transform pair: In the case where is a box of width, that is, then. That is, the nulls of are at multiple of. See the figure below where the box has width. In the case where the signal is a pure sinusoid of frequency the windowed transform will be the sinc function shifted by. In the figure below the box has width and the first sinusoid has frequency Hz. In the case where the signal consists of two sinusoids of frequencies and the windowed transform will be the superposition of two shifted sinc functions. The individual frequencies cannot

13 Página Web 3 de 8 be resolved unless. In fact, for adequate separation we should have. That is, the ``frequency resolution'' of this analysis is. In the following figure a signal is the sum of two sinusoids with frequencies Hz and Hz. The window size is. We get two distinct peaks in the frequency response (see fig2.m). In the case where the signal consists of two spikes close together in time we can resolve the spikes if the window size is smaller that the time difference between the spikes. This analysis shows the ``trade-off'' between time resolution and frequency resolution: if we use a window of length then we have a ``time-resolution'' of but our frequency resolution is. 2.2 The spectrogram The magnitude of the Short Time Fourier Transform is called the spectrogram. We can make 2 dimensional plots of the spectrogram with time on the horizontal axis, frequency on the vertical axis and amplitude given by a gray-scale colour. Alternately we can make 3 dimensional plots where we plot amplitude on the third axis. The Matlab command specgram can be used to generate these

14 Página Web 4 de 8 plots. In the following example, (see fig3.m) a signal is the sum of two sinusoids of frequencies and and two impulses at times ms and ms. We use a window width of ms ( Hz).

15 Página Web 5 de 8 The resolution in frequency is Hz. The time resolution is ms. As the plots show, we can can resolve both the sinusoids and the impulses. Now suppose that we move the two frequencies closer together. Let's use a signal which is the sum of two sinusoids of frequencies and and two impulses at times ms and ms with a window width of ms (see fig4.m). As the spectrograms now show we cannot resolve the frequencies but we can still resolve the spikes.

16 Página Web 6 de 8 Now suppose that we change the window size to ms. As the spectrograms below show, we can resolve the frequencies but not the spikes (see fig4cd.m).

17 Página Web 7 de An Orthgonal Basis of Functions We can obtain an orthogonal basis of functions related to the Short Time Fourier Transform when using the window function = the box of width as follows. Instead of computing for all frequencies and all time shifts we restrict the calculation to and. To see that this corresponds to orthonormal functions define:

18 Página Web 8 de 8 Then we have: Since is non-zero only for it is clear that these are orthogonal functions. Because we have analysis and synthesis on each interval to it follows that we have analysis and synthesis in general. That is: In summary, if we restrict the STFT calculation to a discrete set of frequencies and times we can regard the STFT values as the coordinates of our signal with respect to an orthogonal basis. Hence we can recover our signal from these STFT values. Next: 3. Time-Scale Analysis Up: Wavelets and Filter Banks Previous: 1. Analysis and Synthesis Contents Dr. W. J. Phillips

19 Página Web 1 de 13 Next: 4. Multiresolution Analysis Up: Wavelets and Filter Banks Previous: 2. Time-Frequency Analysis Contents Subsections 3.1 The Continuous Wavelet Transform 3.2 Comparision with STFT 3.3 The Scalogram 3.4 Examples of Wavelets 3.5 Analysis and Synthesis with Wavelets 3.6 The Haar Wavelet 3. Time-Scale Analysis The Continous Wavelet Transform (CWT) provides a time-scale description similar to the STFT with a few important differences: Frequency is related to scale which may have a better relationship to the problem at hand. The CWT is able to resolve both time and scale (frequency) events better than the STFT. By restricting to a discrete set of paramaters we get the Discrete Wavelet Transform (DWT) which corresponds to an orthogonal basis of functions all derived from a single function called the mother wavelet. The basis functions in the DWT are not solutions of differential equations as in the Fourier case. The basis functions are ``near optimal'' for a wide class of problems. This means that the analysis coefficients drop off rapidly. There is a connection and equivalence to filter bank theory from DSP which leads to a computationally efficient algorithm. The computational complexity of the FFT is while that of the DWT is. 3.1 The Continuous Wavelet Transform The formula for the continuous wavelet transform (CWT) is: Note that the roles of and are reversed from the usage in the book WWT. The function is called the (mother) wavelet. It is taken to be a ``small wave''. For example, the

20 Página Web 2 de 13 Haar wavelet is a single cycle of the square wave of period 1. The morlet wavelet has formula: It is also a small wave since the gaussian exponential,., is effectively zero outside the interval The graph of is obtained stretching the graph of by a the factor, called the scale, and shifting in time by. The time-shifted and time-scaled wavelet is sometimes called a baby wavelet. The figure below shows a signal shifts (see fig5.m). The subsequent figure shows a signal (see fig6.m). along with the Haar wavelet with two different scales and along with the morelet wavelet at three scales and shifts

21 Página Web 3 de 13 We can think of the CWT in different ways: 1. The CWT is the inner product or cross correlation of the signal with the scaled and time shifted wavelet. This cross correlation is a measure of the similarity between signal and the scaled and shifted wavelet. It is this point of view that is illustrated in the figures above. 2. For a fixed scale,, the CWT is the convolution of the signal with the time reversed wavelet. That is, the CWT is the output when we feed our signal to the filter with with impulse response. It is this filter point of view which will show the connection to STFT 3.2 Comparision with STFT We can write the STFT as:

22 Página Web 4 de 13 We use the variable,, for frequency so that later when we take the Fourier Transform we avoid confusing this frequency variable with the usual one in the transform. Aside from the initial phase factor,, this last equation is the convolution of the signal,, with the frequency shifted and time reversed window function,. That is, To understand the significance of the filter interpretations of CWT and STFT we can consider the case of the Morlet wavelet,, and the STFT with gaussian window function,. The Fourier Transform of the gaussian window function is:. Note that this is a window function in the frequency domain. It is a low pass filter which blocks all frquencies above

23 Página Web 5 de 13 The frequency response of the filter in the STFT is this transform shifted by frequency. That is,. This is a band pass filter centered at frquency and of approximate width 1 Hz. That is, computing the spectrogram of a signal using a Gaussian window function is the same as passing the signal through a series of bandpass filters of constant bandwidth 1 Hz. In the case of the CWT the frequency repsonse of the filter when the scale,, is: This is a band pass filter centered at frquency Hz with bandwidth 1 Hz. At scale the frequency response is. This is band pass filter centered at frequency with a bandwidth of Hz.

24 Página Web 6 de 13 The is a constant Q filter That is, computing the CWT of a signal using the Morlet wavelet is the same as passing the signal through a series of bandpass filters centered at with constant Q of. This shows the essential difference between the STFT and the CWT. In the STFT the frquency bands have a fixed width (1 Hz for Gaussian). In the CWT the frequency bands grow and shrink with the frequency (scale) being used. This allows good frequency resolution at low frequencies and good time resolution at high frequencies. 3.3 The Scalogram The magnitude of the Continuous Wavelet Transform is called the scalogram. We can make 2 dimensional plots of the scalogram with time on the horizontal axis, scale on the vertical axis, and amplitude given by a gray-scale colour. Alternately, we can make 3 dimensional plots. The matlab command cwt can be used to generate these plots. In the following example (see fig7.m) we use the same signal as in fig4.m. That is, is the sum of two sinusoids of frequencies and and two impulses at times ms and ms. Using the Morlet wavelet we obtain the following scalogram.

25 Página Web 7 de 13 We can convert between scale, to frequency, using the formula. We can make a new scalogram using frequency instead of scale. To see clearly that the frequencies are resolved by the scalograms we can make a 3 dimensional plot.

26 Página Web 8 de Examples of Wavelets There are a number of wavelets with explicit formulas. These wavelets are generally not useful in practice but are good examples for working out the theory. It turns out the the useful wavelets do not have explicit formulas but they do have a ``fast wavelet transform'' using filter banks. The Morlet Wavelet The morlet wavelet and its Fourier Transform are:

27 Página Web 9 de 13 The Haar Wavelet The Haar wavelet and its Fourier Transform are: The Mexican Hat Wavelet The Mexican Hat wavelet and its Fourier Transform are:

28 Página Web 10 de 13 The Shannon Wavelet The Shannon wavelet and its Fourier Transform are: 3.5 Analysis and Synthesis with Wavelets

29 Página Web 11 de 13 Recall that with the STFT we obtained an orthogonal basis of functions by choosing equally spaced frequency and time samples. This does not apply in general. To ensure the orthogonality we chose the window function to be the box of width. In some cases we can get an orthogonal basis of functions in the CWT case by choosing the scales to be powers of 2 and the times to be an integer multiple of the scales. That is, for integers and we consider: To simplify the notation we define a doubly indexed set of baby wavelets as follows: It then follows that the values are the analysis coefficients for these functions. That is, There is a large class of wavelet functions for which the the set of baby wavelets is an orthogonal basis. These are the orthogonal wavelets. the simplest of these is the Haar wavelet. In the case of an orthogonal wavelet the analysis formula is called the Discrete Wavelet Transform. The recovery of the signal through the synthesis formula is called the Inverse Discrete Wavelet Transform. Note that the Time-Scale Diagram for the Discrete Wavelet Transform is a a set of samples of the Time-Scale Diagram for the Continuous Wavelet Transform. The samples are quite ``sparse'' for large scale and more ``dense'' for small scale.

30 Página Web 12 de The Haar Wavelet Most results about wavelets are simple to see in the case of the Haar wavelet. It is best to keep this case in mind to guide your thinking about wavelets in general. With this in mind we should thoroughly understand the Haar case. The first point to understand is that the Haar baby wavelets are orthogonal to each other. The wavelet function is a single cycle of a square wave of period 1. Then, The factor of is to make the energy of the signal 1. The function is a single cycle of a square wave extending from time to. From this description it is easy to see that baby Haar wavelets, and of the same scale,, but different positions and, are orthogonal because their graphs don't overlap.

31 Página Web 13 de 13 It is also true that Haar baby wavelets of differnt scales are orthogonal. To see this it is best to first consider the case of and. Since it follows (see the figure below) that completes its cycle from positive to negative while is constantly 1 so that the integral of the product is 0. In the general case of and if the graphs overlap then one of the functions completes its cycle from 1 to -1 while the other is constant. This shows that these two are orthogonal. The other property of an orthogonal basis is that of completeness. We will see later that the Haar baby wavelets are complete. Next: 4. Multiresolution Analysis Up: Wavelets and Filter Banks Previous: 2. Time-Frequency Analysis Contents Dr. W. J. Phillips

32 Página Web 1 de 13 Next: 5. Filter Banks and Up: Wavelets and Filter Banks Previous: 3. Time-Scale Analysis Contents Subsections 4.1 The Scaling Function 4.2 The Discrete Wavelet Transform 4. Multiresolution Analysis We would like to find wavelets,, with the same properties as the Haar case. That is, The baby wavelets,, for all and, form an orthogonal basis. This implies that we have the usual Analysis-Synthesis for all signals: Such wavelets give rise to a Multiresolution Analysis derived as follows. Define to be set of all signals,, which can be synthesized from the baby wavelets,. These spaces are orthogonal to each other and we can synthesize any (energy) signal, as (Note that in the following formula is in the space ): There is another way to express this idea. Define to be the set of all signals,, which can be synthesized from the baby wavelets where and. That is

33 Página Web 2 de 13 The spaces are nested inside each other. That is, As goes to infinity enlarges to become all energy signals ( ). As goes to negative infinity shrinks down to only the zero signal. It is clear from the definitions that every signal in is a sum of a signal in and because: That is, we can write: This shows that the spaces and. are the differences (in the subspace sense) between adjacent spaces We can visualize the spaces and as follows: The term Multiresolution Analysis refers to analyzing signals in relation to this nested sequence of subspaces. To get a better idea of multiresolution analysis, let's decompose a signal,, in a few times. We can use the breakdown:

34 Página Web 3 de 13 This leads to various decompositions: Where, in, is called the detail at level and, in, is called the approximation at level. Example: This decompostion can be carried out in Matlab using the wavemenu interface. There are a number of sample signals which can be used for a demo analysis. The signal sumsin is a sum of two sine waves. In the following figures we decompose the signal 4 times as above.

35 Página Web 4 de 13

36 Página Web 5 de 13 Note that different aspects of the signal appear in the details and the approximations. The spaces have a very important property related to time compression by factors of 2. The Two Scale Property of Multiresolution: A signal is in the space if and only if is in the next space. This follows from the formula: Investigation of the multiresolution analysis leads to a scaling function, a pair of discrete time filters, and a perfect reconstruction filter bank which can be used to calculate the DWT quickly. 4.1 The Scaling Function The useful wavelets,, have a scaling function which can produce the Multiresolution subspaces as follows. Define the ``baby scaling functions'': where and. Just as for the wavelet, the ``scale'' of is and the ``position'' is. We want to find so that the signals in the space can be synthesized from the baby scale functions,. Since the spaces are obtained from by time compression or dilation by powers of 2 we only need to check the space. That is, we want to find a function so that the signals in can be synthesized from the integer translates of the scale function. Example: Haar Case In the Haar case the scaling function is the unit box delayed by :

37 Página Web 6 de 13 Then is the box of length extending from to. To see that the integer translates of form a basis for note that: By a similar formula we can synthesize from and its translates for any negative. The Two Scale Equation and the Filters There is an important formula connecting the scale function to itself at two different time scales. This fundamental formula is called the Two Scale Equation and it gives rise to one of the filters. There are discrete time filter coefficients such that: This follows trivially from the assumption that equation involving the scale function. but is probably the most important Since is also a subset of there is another two scale equation for the wavelet which gives rise to another filter, such that: Example: Haar Case In the Haar case, the scale function is the box of width 1 extending from time 0 to time 1. It follows that is the box of width 1/2 extending from time 0 to time 1/2.

38 Página Web 7 de 13 Similarily, is the box of width 1/2 extending from time 1/2 to time 1. When we add these two smaller box functions we obtain. That is, The filter for the scale function is Similarily, the Haar wavelet can be expressed as: The filter for the wavelet is

39 Página Web 8 de The Discrete Wavelet Transform We have previously defined the Discrete Wavelet Transform of a signal, analysis coefficients:, to be the set of From these we can recover the signal as: Assuming the existence of a scaling function, we can now modify this defintion as follows. Since the spaces are getting larger and larger as goes to we can approximate any signal,, closely by choosing a large enough value of and projecting the signal into using the basis, (all values of ). From these we can approximately recover the signal as:

40 Página Web 9 de 13 In effect, we replace the signal,, by the approximate signal given by the projection coefficients,. After this approximation our signal is now in and we can decompose it using the subspaces and with their bases and. Note that the scale is getting larger and larger as the index gets more negative. If we take we get: Using the basis in and in we have: As before, we call the signals and the approximation and detail at level 1. We call the coefficients and the approximation coefficients and the detail coeffients at level 1. We can further decompose to get: We call the signals and the approximation and detail at level 2. We call the coefficients and the approximation coefficients and the detail coeffients at level 2.

41 Página Web 10 de 13 We can continue in this way to decompose our signal further and further. In terms of coefficients we have (using [...] to indicate vectors as in Matlab): The calculations of the coefficients can be carried out in Matlab using the dwt (Discrete Wavelet Transform) command. The signal can be recovered from the coefficinets using the idwt (Inverse Discrete Wavelet Transform) command. The Matlab implementation uses the filters and previously mentioned. We shall see how this is done in the next section. The continuous time meaning of the decomposition can be visualized as follows: Matlab Example: The Matlab routines used in this example will be discussed in the next section. In this example we will decompose a speech signal using the Haar wavelet and a 4 level decomposition. This signal is the word ``two'' sampled at 8000 samples per second.

42 Página Web 11 de 13

43 Página Web 12 de 13

44 Página Web 13 de 13 Next: 5. Filter Banks and Up: Wavelets and Filter Banks Previous: 3. Time-Scale Analysis Contents Dr. W. J. Phillips

45 Página Web 1 de 21 Next: 6. Properties of the Up: Wavelets and Filter Banks Previous: 4. Multiresolution Analysis Contents Subsections 5.1 Analysis: From Fine Scale to Coarser Scale Filtering and Downsampling The One-Stage Analysis Filter Bank The Analysis Filter Bank 5.2 Synthesis: From Course Scale to Fine Scale Upsampling and Filtering The One-Stage Synthesis Filter Bank Perfect Reconstruction Filter Bank The Synthesis Filter Bank Approximations and Details 5.3 Numerical Complexity of the Discrete Wavelet Transform 5.4 Matlab Examples One-Stage Perfect Reconstruction Approximations and Details A Useful Function 5.5 Initialization of the Discrete Wavelet Transform 5. Filter Banks and the Discrete Wavelet Transform In the previous section we found that the a Multiresolution Analysis allows us to decompose a signal into approximations and details. On the theoretical level this is an Analysis-Synthesis situation. That is, we have bases and and we use these bases to decompose our signal. On the practical level, we assume that our signal is represented by its approximation coefficients at some scale and we decompose it in terms of its coefficients at larger scale. Both points of view are necessary for a real understanding of the subject. In this section we will show that the approximation and detail coefficients can be computed using the filters previously mentioned. As we must compute these coefficients at many different scales we will need a filter bank. 5.1 Analysis: From Fine Scale to Coarser Scale In the Discrete Wavelet Transform (DWT) we have. That is, each signal in can be expressed in two ways using the basis functions in each of the spaces.

46 Página Web 2 de 21 We start with the coefficients at scale index and produce the two sets of coefficients and at scale index (Analysis). Alternately, we can start with the two sets of coefficients and at scale index and produce the coefficients at scale index (Synthesis). We can show that the two operations of Analysis and Synthesis are produced by certain filter banks. As the wavelets and the scales at each index level are orthogonal we can compute the coefficients and by the usual inner product formula: To complete this calculation we have to compute the inner product:

47 Página Web 3 de 21 We can now complete the calculation previously started: The detail coefficients can be computed similarily. To complete this calculation we have to compute the inner product:

48 Página Web 4 de 21 Upon substitution of this formula into the previous calculation we get: Filtering and Downsampling The two formulas for the approximation and detail coefficients look similar to convolution but there is a downsampling involved. Downsampling a discrete time signal consists of omitting every other value. We can think of a system whose input is and whose output is. To understand the approximation and detail formulas it will help to define the time reversed filters and. We temporarily use to see the convolution. If we follow this filter by the downsampler we get the approximation coefficients at the next level.

49 Página Web 5 de 21 The same calculation holds for the detail coefficients. That is, convolution with the time reversed filter followed by downsampling produces the detail coefficients at the next level The One-Stage Analysis Filter Bank We really should think of the two filtering operations followed by downsampling as a filter bank. We are analyzing a function in into a detail in and an approximation,, in, using a filter bank to calculate the coefficients and. The dwt command in the Wavelet toolbox of Matlab carries out the calculations in this one stage filter bank. The syntax is: [ca, cd] = dwt(x, Lo, Hi); = dwt(x, 'wname'); Note that the number of data values produced by the filter bank is about the same as the number of data values entering the system. To see this let, be the length of the input vector and assume that the filters both have length. The length of the convolution is so that the lengths of and are. The overall size of the data emerging from the filterbank is increased by the length of the filter minus The Analysis Filter Bank

50 Página Web 6 de 21 We can further decompose to get: We can then decompose to get: The coefficients, and form can be calculated by iterataing or cascading the single stage filter bank to obtain a multiple stage filter bank. The wavedec command in the Wavelet toolbox of Matlab carries out the calculations in the multistage filter bank. The syntax is: [C, L] = wavedec(x, N, Lo, Hi); = wavedec(x, N, 'wname'); The vector is the set of details coefficients at each level together with the approximation coefficients at the final level.

51 Página Web 7 de 21 The vector is the vector of lengths of each of the entries in and the length of (see the wavedec reference page). Matlab provides functions, appcoef and detcoef compute any of the approximation or detail coefficients from the output of the wavedec command. 5.2 Synthesis: From Course Scale to Fine Scale The decomposition of a signal into an approximation and a detail can be reversed. That is, we start with the two sets of coefficients and at scale index and produce the coefficients at scale index (Synthesis). We have: Using the fact that is an orthogonal basis for we have: This synthesis formula can be understood in terms of upsampling and filtering Upsampling and Filtering The expressions:

52 Página Web 8 de 21 look like convolutions but upsampling is involved. Upsampling of a discrete time signal consists of inserting zeros between the values. We can think about a system with input and output for even values of and for odd values of. The expressions for and consist of upsampling followed by filtering The One-Stage Synthesis Filter Bank It follows that the synthesis formula consists of adding the outputs of the upsampled and filtered approximation and detail coefficients. The idwt command in the Wavelet toolbox of Matlab carries out the calculations in this one stage filter bank. The syntax is: x = idwt(ca, cd, Lo, Hi); = idwt(ca, cd, 'wname'); Perfect Reconstruction Filter Bank If we feed the output of the one-stage analysis filterbank to the input of the one-stage synthesis fitler

53 Página Web 9 de 21 bank then we get the original coefficients back. We say the we have a perfect reconstruction filter bank. Note that the non-causal nature of the Analysis Bank filters is does not cause a problem in practice as we are dealing with FIR filters. This means that we can apply a fixed delay, to each filter to make it causal before applying the input signal. This is the same as delaying the input signal by before applying it to the filter bank. To make things simple, assume that the filters and both have length. If we delay the time reversed filter by then we obtain the ``causal flip'' of the filter. Denote the reversed filters by,. Filtering a signal by is the same a delaying the signal by then filtering by. Hence, we can have a filter bank consisting of causal filters which gives perfect reconstruction with an overall delay of. The analysis filters are and while the synthesis filters are and The Synthesis Filter Bank The outputs of the multiple stage analysis filter bank can be fed into a multiple stage synthesis filter bank to reproduce the original coefficients. For example, a 3 level analysis bank produces outputs,,, and. These are fed into the 3 level synthesis filter bank as

54 Página Web 10 de 21 shown: Approximations and Details We have seen that we can reconstruct the signal in from the approximation and detail coefficients, and at level 1. As and are in we can resolve them as: Using the same reasoning as before:

55 Página Web 11 de 21 That is, we obtain the approximation coefficients at level 0 by upsampling the approximation coefficients, at level 1 and then filtering with the low pass filter. Similarily, we obtain the detail coefficients at level 0 by upsampling the detail coefficients, at level 1 and then filtering with the high pass filter. 5.3 Numerical Complexity of the Discrete Wavelet Transform We have previously analyzed the one-stage analysis filterbank to count the number of coefficients produced. We can also count the number of floating point operations involved. Assume that we are feeding in coefficients and the filter have length. Each convolution takes approximately operations. So, the system requires total floating point operations. Now if we use two-stage filter bank then the input to the next bank has length (approximately) so that the second stage adds only operations to the first stage operations. Continuing in this way, a multi-level bank requires: That is, the computational complexity of the algorithm is linear in the size of the data.

56 Página Web 12 de 21 Compare this to the FFT algorithm which has complexity. We really should not make too much of this apparent computational advantage as we know the CWT reflects a logarithmic division of frequency while the FFT uses equally spaced frequency divisions. The point is that for some applcations the logarithmic frquency divisions are sufficent for analyzing the situation so that the DWT has a computational advantage is these cases. 5.4 Matlab Examples One-Stage Perfect Reconstruction The m-file checkfb.m produced the following diary file. echo on % m file to generate and test the analysis and synthesis banks % % % % y1 v1 u1 w1 % rh dwn up h % % %x z % % % % y0 v0 u0 w0 % rh dwn up h % % % % Analysis Bank Synthesis Bank % % % N = 2; % which db filters to use len = 2*N; % filter length p = 10; % size of input vectors % construct the string for dbn wave = ['db',int2str(n)]; % get some filters to work with [rh0 rh1 h0 h1] = wfilters(wave); % Have a look at the filters [rh0' rh1' h0' h1'] ans = x=randn(p,1); % white noise % first handle the analysis bank y0 = conv(rh0',x); y1 = conv(rh1',x);

57 Página Web 13 de 21 v0 = dyaddown(y0); v1 = dyaddown(y1); % print them out [y0 y1] ans = [v0 v1] ans = % now do the synthesis bank u0 = dyadup(v0); u1 = dyadup(v1); w0 = conv(h0,u0); w1 = conv(h1,u1); % print them out [u0 u1] ans = [w0 w1]

58 Página Web 14 de 21 ans = % now add up the streams z = w0 + w1; % to compare x to z we pad the end with zeros cx = [x ; zeros(2*(len-1),1)]; [cx z] ans = >> diary off Approximations and Details The m-file wavdemo.m produced the following plots. % File: wavdemo.m % demonstrates various commands in the wavelet toolbox load sumsin x=sumsin; % Compute the wavelet decomposition at level 3 [C,L] = wavedec(x,3,'db2');

59 Página Web 15 de 21 figure(1) subplot(2,1,1); plot(x); title('original signal');axis([ ]); subplot(2,1,2); plot(c); title('wavelet decomposition');axis([ ]); orient landscape; print wavdemofig1.eps % Now extract the coefficients ca3 = appcoef(c, L, 'db2', 3); cd3 = detcoef(c, L, 3); cd2 = detcoef(c, L, 2); cd1 = detcoef(c, L, 1); % plot these figure(2) subplot(4,2,1); plot(ca3); title('ca3');axis([ ]); subplot(4,2,3); plot(cd3); title('cd3');axis([ ]); subplot(4,2,5); plot(cd2); title('cd2');axis([ ]); subplot(4,2,7); plot(cd1); title('cd1');axis([ ]); % Now compute the reconstructed coefficients A3 = wrcoef('a', C, L, 'db2', 3); D3 = wrcoef('d', C, L, 'db2', 3); D2 = wrcoef('d', C, L, 'db2', 2); D1 = wrcoef('d', C, L, 'db2', 1); % plot these subplot(4,2,2); plot(a3); title('a3');axis([ ]); subplot(4,2,4); plot(d3); title('d3');axis([ ]); subplot(4,2,6); plot(d2); title('d2');axis([ ]); subplot(4,2,8); plot(d1); title('d1');axis([ ]); orient landscape; print wavdemofig2.eps

60 Página Web 16 de 21

61 Página Web 17 de A Useful Function The function examine_recon.m can be useful in examining the reconstruction or synthesis of a signal. It was used to produce the plots of approximations and details for the speech signal in the multiresolution section. function [ca,a,cd,d] = examine_recon(x, N, wname) %[ca,a,cd,d] = examine_recon(x,n,wname) calculate and plot approx and details %up to level N for a signal x using wavelet type given by wname [m,n]=size(x); if m == 1 x=x'; end % perform wavelet decompostion at level N [C,L] = wavedec(x,n,wname); A = zeros(length(x),n); D = zeros(length(x),n); ca = zeros(length(x),n); cd = zeros(length(x),n); lena = zeros(1,n); lend = zeros(1,n); % compute the approximations at various levels for i=1:n A(:,i) = wrcoef('a',c,l,wname,i);

62 Página Web 18 de 21 end % compute the approximation coeff at various levels for i=1:n temp = appcoef(c,l,wname,i); lena(i)=length(temp); ca(1:lena(i),i) = temp; end % compute the details at various levels for i=1:n D(:,i) = wrcoef('d',c,l,wname,i); end % compute the detail coeff at various levels for i=1:n temp = detcoef(c,l,i); lend(i)=length(temp); cd(1:lend(i),i) = temp; end % make plots figure; subplot(n+1,1,1); plot(x); title(['signal and approximations 1 to ', num2str(n)]); for i=1:n subplot(n+1,1,i+1); plot(a(:,i)); end figure; subplot(n+1,1,1); plot(x); title(['signal and approximation coef 1 to ', num2str(n)]); for i=1:n subplot(n+1,1,i+1); plot(ca(1:lena(i),i)); end figure; subplot(n+1,1,1); plot(x); title(['signal and details 1 to ', num2str(n)]); for i=1:n subplot(n+1,1,i+1); plot(d(:,i)); end figure; subplot(n+1,1,1); plot(x); title(['signal and detail coef 1 to ', num2str(n)]); for i=1:n subplot(n+1,1,i+1); plot(cd(1:lend(i),i)); end 5.5 Initialization of the Discrete Wavelet Transform Given a signal, to apply the Discrete Wavelet Transform our starting point must be a signal in one of the spaces. Since the spaces,, are getting larger and larger as goes to infinity we can approximate our signal,, closely by choosing a large enough value of and projecting the signal into using the orthogonal basis, (all values of ). That is:

63 Página Web 19 de 21 The projection coefficients, continuous time filter. That is,, in this summation can be obtained by sampling the output of a This shows that we obtain the coordinates of the projection of onto the space by passing through the filter with impulse response and then sampling at times. We will show that if is bandlimited then by choosing an appropriate sampling rate and a scale index,, we can recognize the projection coefficents in terms of a discrete time filtering process. Assume that we have sampled a band limited signal at 20% above the Nyquist rate. After normalization we can assume that the sampling rate is 1Hz so that our signal contains only frequencies below 0.5/1.2 = That is, we can represent our signal as Where the Discrete Fourier Transform,, of is zero for. We can now pass this signal into the above filter to find the projection onto the summation and itegration we find:. After rearranging

64 Página Web 20 de 21 Where evaluated at. Hence the coordinates of the projection of our signal onto are just the samples,, filtered by. Fact: For almost all orthogonal wavelets the filter is approximately magnitude 1 over frequencies and the phase response is linear. That is, the filter acts as a delay on our signal and so aside from that delay, we may use the samples of the signal as the scaling coefficients. Example: The m-file initdwt.m can be used to investigate the filter for various wavelets. % This m file investigates the filter which carries out the projection of % a band limited signal into V0 % % This filter is alpha(n) = sinc(r) * phi(-r), at r = n % % alpha(n) = integral phi(-s) sinc(n-s) ds % = integral phi(t) sinc(t+n) dt % % Since the support of phi(t) is 0 <= t <= N-1 the values of this % integral will become quite small outside the interval -2N <= n <= 2N-1. % So, we approximate alpha(n) by an FIR filter of length 4N. iter = 10; dt = 1/2^iter; wave = 'db7'; [f0 f1 h0 h1] = wfilters(wave); [phi, psi, t] = wavefun(wave,iter); N = length(h0); alpha = zeros(1,4*n); for n=(-2*n:(2*n-1)) alpha(n+2*n+1) = sinc(t + n) * phi' * dt; end subplot(3,1,1) ;stem(alpha); title(['projection Filter for ' wave]); [Alpha,W]=freqz(alpha); subplot(3,1,2); plot(w/(2*pi), abs(alpha)); axis([ ]); title('magnitude Response'); subplot(3,1,3); plot(w/(2*pi), unwrap(angle(alpha)));