Wavelets and Wavelet Transforms. Collection Editor: C. Sidney Burrus

Transcription

1 Wavelets and Wavelet Transforms Collection Editor: C. Sidney Burrus

2

3 Wavelets and Wavelet Transforms Collection Editor: C. Sidney Burrus Authors: C. Sidney Burrus Ramesh Gopinath Haitao Guo Online: < > C O N N E X I O N S Rice University, Houston, Texas

4 This selection and arrangement of content as a collection is copyrighted by C. Sidney Burrus. It is licensed under the Creative Commons Attribution 3.0 license ( Collection structure revised: February 11, 2013 PDF generated: February 11, 2013 For copyright and attribution information for the modules contained in this collection, see p. 281.

5 Table of Contents 1 Preface Introduction to Wavelets A multiresolution formulation of Wavelet Systems Filter Banks and the Discrete Wavelet Transform Bases, Orthogonal Bases, Biorthogonal Bases, Frames, Tight Frames, and unconditional Bases The Scaling Function and Scaling Coecients, Wavelet and Wavelet Coecients Regularity, Moments, and Wavelet System Design Generalizations of the Basic Multiresolution Wavelet System Filter Banks and Transmultiplexers Calculation of the Discrete Wavelet Transform Wavelet-Based Signal Processing and Applications Summary Overview Appendix A Appendix B Appendix C Bibliography Index Attributions

6 iv

7 Chapter 1 Preface 1 This book develops the ideas behind and properties of wavelets and shows how they can be used as analytical tools for signal processing, numerical analysis, and mathematical modeling. We try to present this in a way that is accessible to the engineer, scientist, and applied mathematician both as a theoretical approach and as a potentially practical method to solve problems. Although the roots of this subject go back some time, the modern interest and development have a history of only a few years. The early work was in the 1980's by Morlet, Grossmann, Meyer, Mallat, and others, but it was the paper by Ingrid Daubechies [82] in 1988 that caught the attention of the larger applied mathematics communities in signal processing, statistics, and numerical analysis. Much of the early work took place in France [3], [6] and the USA [82], [7], [93], [357]. As in many new disciplines, the rst work was closely tied to a particular application or traditional theoretical framework. Now we are seeing the theory abstracted from application and developed on its own and seeing it related to other parallel ideas. Our own background and interests in signal processing certainly inuence the presentation of this book. The goal of most modern wavelet research is to create a set of basis functions (or general expansion functions) and transforms that will give an informative, ecient, and useful description of a function or signal. If the signal is represented as a function of time, wavelets provide ecient localization in both time and frequency or scale. Another central idea is that of multiresolution where the decomposition of a signal is in terms of the resolution of detail. For the Fourier series, sinusoids are chosen as basis functions, then the properties of the resulting expansion are examined. For wavelet analysis, one poses the desired properties and then derives the resulting basis functions. An important property of the wavelet basis is providing a multiresolution analysis. For several reasons, it is often desired to have the basis functions orthonormal. Given these goals, you will see aspects of correlation techniques, Fourier transforms, short-time Fourier transforms, discrete Fourier transforms, Wigner distributions, lter banks, subband coding, and other signal expansion and processing methods in the results. Wavelet-based analysis is an exciting new problem-solving tool for the mathematician, scientist, and engineer. It ts naturally with the digital computer with its basis functions dened by summations not integrals or derivatives. Unlike most traditional expansion systems, the basis functions of the wavelet analysis are not solutions of dierential equations. In some areas, it is the rst truly new tool we have had in many years. Indeed, use of wavelets and wavelet transforms requires a new point of view and a new method of interpreting representations that we are still learning how to exploit. More recently, work by Donoho, Johnstone, Coifman, and others have added theoretical reasons for why wavelet analysis is so versatile and powerful, and have given generalizations that are still being worked on. They have shown that wavelet systems have some inherent generic advantages and are near optimal for a wide class of problems [116]. They also show that adaptive means can create special wavelet systems for particular signals and classes of signals. 1 This content is available online at < 1

8 2 CHAPTER 1. PREFACE The multiresolution decomposition seems to separate components of a signal in a way that is superior to most other methods for analysis, processing, or compression. Because of the ability of the discrete wavelet transform to decompose a signal at dierent independent scales and to do it in a very exible way, Burke calls wavelets The Mathematical Microscope" [44], [221]. Because of this powerful and exible decomposition, linear and nonlinear processing of signals in the wavelet transform domain oers new methods for signal detection, ltering, and compression [116], [123], [114], [366], [453], [189]. It also can be used as the basis for robust numerical algorithms. You will also see an interesting connection and equivalence to lter bank theory from digital signal processing [422], [15]. Indeed, some of the results obtained with lter banks are the same as with discretetime wavelets, and this has been developed in the signal processing community by Vetterli, Vaidyanathan, Smith and Barnwell, and others. Filter banks, as well as most algorithms for calculating wavelet transforms, are part of a still more general area of multirate and time-varying systems. The presentation here will be as a tutorial or primer for people who know little or nothing about wavelets but do have a technical background. It assumes a knowledge of Fourier series and transforms and of linear algebra and matrix theory. It also assumes a background equivalent to a B.S. degree in engineering, science, or applied mathematics. Some knowledge of signal processing is helpful but not essential. We develop the ideas in terms of one-dimensional signals [357] modeled as real or perhaps complex functions of time, but the ideas and methods have also proven eective in image representation and processing [379], [268] dealing with two, three, or even four dimensions. Vector spaces have proved to be a natural setting for developing both the theory and applications of wavelets. Some background in that area is helpful but can be picked up as needed. The study and understanding of wavelets is greatly assisted by using some sort of wavelet software system to work out examples and run experiments. Matlab T M programs are included at the end of this book and on our web site (noted at the end of the preface). Several other systems are mentioned in Chapter: Wavelet-Based Signal Processing and Applications (Chapter 11). There are several dierent approaches that one could take in presenting wavelet theory. We have chosen to start with the representation of a signal or function of continuous time in a series expansion, much as a Fourier series is used in a Fourier analysis. From this series representation, we can move to the expansion of a function of a discrete variable (e.g., samples of a signal) and the theory of lter banks to eciently calculate and interpret the expansion coecients. This would be analogous to the discrete Fourier transform (DFT) and its ecient implementation, the fast Fourier transform (FFT). We can also go from the series expansion to an integral transform called the continuous wavelet transform, which is analogous to the Fourier transform or Fourier integral. We feel starting with the series expansion gives the greatest insight and provides ease in seeing both the similarities and dierences with Fourier analysis. This book is organized into sections and chapters, each somewhat self-contained. The earlier chapters give a fairly complete development of the discrete wavelet transform (DWT) as a series expansion of signals in terms of wavelets and scaling functions. The later chapters are short descriptions of generalizations of the DWT and of applications. They give references to other works, and serve as a sort of annotated bibliography. Because we intend this book as an introduction to wavelets which already have an extensive literature, we have included a rather long bibliography. However, it will soon be incomplete because of the large number of papers that are currently being published. Nevertheless, a guide to the other literature is essential to our goal of an introduction. A good sketch of the philosophy of wavelet analysis and the history of its development can be found in a recent book published by the National Academy of Science in the chapter by Barbara Burke [44]. She has written an excellent expanded version in [221], which should be read by anyone interested in wavelets. Daubechies gives a brief history of the early research in [103]. Many of the results and relationships presented in this book are in the form of theorems and proofs or derivations. A real eort has been made to ensure the correctness of the statements of theorems but the proofs are often only outlines of derivations intended to give insight into the result rather than to be a formal proof. Indeed, many of the derivations are put in the Appendix in order not to clutter the presentation. We hope this style will help the reader gain insight into this very interesting but sometimes obscure new mathematical signal processing tool.

9 We use a notation that is a mixture of that used in the signal processing literature and that in the mathematical literature. We hope this will make the ideas and results more accessible, but some uniformity and cleanness is lost. The authors acknowledge AFOSR, ARPA, NSF, Nortel, Inc., Texas Instruments, Inc. and Aware, Inc. for their support of this work. We specically thank H. L. Resniko, who rst introduced us to wavelets and who proved remarkably accurate in predicting their power and success. We also thank W. M. Lawton, R. O. Wells, Jr., R. G. Baraniuk, J. E. Odegard, I. W. Selesnick, M. Lang, J. Tian, and members of the Rice Computational Mathematics Laboratory for many of the ideas and results presented in this book. The rst named author would like to thank the Maxeld and Oshman families for their generous support. The students in EE-531 and EE-696 at Rice University provided valuable feedback as did Bruce Francis, Strela Vasily, Hans Schüssler, Peter Steen, Gary Sitton, Jim Lewis, Yves Angel, Curt Michel, J. H. Husoy, Kjersti Engan, Ken Castleman, Je Trinkle, Katherine Jones, and other colleagues at Rice and elsewhere. We also particularly want to thank Tom Robbins and his colleagues at Prentice Hall for their support and help. Their reviewers added signicantly to the book. We would appreciate learning of any errors or misleading statements that any readers discover. Indeed, any suggestions for improvement of the book would be most welcome. Send suggestions or comments via to csb@rice.edu. Software, articles, errata for this book, and other information on the wavelet research at Rice can be found on the world-wide-web URL: with links to other sites where wavelet research is being done. C. Sidney Burrus, Ramesh A. Gopinath, and Haitao Guo Houston, Texas, Yorktown Heights, New York, and Sunnyvale, California 1.1 Instructions to the Reader Although this book in arranged in a somewhat progressive order, starting with basic ideas and denitions, moving to a rather complete discussion of the basic wavelet system, and then on to generalizations, one should skip around when reading or studying from it. Depending on the background of the reader, he or she should skim over most of the book rst, then go back and study parts in detail. The Introduction at the beginning and the Summary at the end should be continually consulted to gain or keep a perspective; similarly for the Table of Contents and Index. The Matlab programs in the Appendix or the Wavelet Toolbox from Mathworks or other wavelet software should be used for continual experimentation. The list of references should be used to nd proofs or detail not included here or to pursue research topics or applications. The theory and application of wavelets are still developing and in a state of rapid growth. We hope this book will help open the door to this fascinating new subject. 3

10 4 CHAPTER 1. PREFACE

11 Chapter 2 Introduction to Wavelets1 This chapter will provide an overview of the topics to be developed in the book. Its purpose is to present the ideas, goals, and outline of properties for an understanding of and ability to use wavelets and wavelet transforms. The details and more careful denitions are given later in the book. A wave is usually dened as an oscillating function of time or space, such as a sinusoid. Fourier analysis is wave analysis. It expands signals or functions in terms of sinusoids (or, equivalently, complex exponentials) which has proven to be extremely valuable in mathematics, science, and engineering, especially for periodic, time-invariant, or stationary phenomena. A wavelet is a small wave", which has its energy concentrated in time to give a tool for the analysis of transient, nonstationary, or time-varying phenomena. It still has the oscillating wave-like characteristic but also has the ability to allow simultaneous time and frequency analysis with a exible mathematical foundation. This is illustrated in Figure 2.1 with the wave (sinusoid) oscillating with equal amplitude over t and, therefore, having innite energy and with the wavelet having its nite energy concentrated around a point. Figure 2.1: A Wave and a Wavelet: A Sine Wave 1 This content is available online at < 5

12 6 CHAPTER 2. INTRODUCTION TO WAVELETS Figure 2.2: A Wave and a Wavelet: Daubechies' Wavelet ψ D20 We will take wavelets and use them in a series expansion of signals or functions much the same way a Fourier series uses the wave or sinusoid to represent a signal or function. The signals are functions of a continuous variable, which often represents time or distance. From this series expansion, we will develop a discrete-time version similar to the discrete Fourier transform where the signal is represented by a string of numbers where the numbers may be samples of a signal, samples of another string of numbers, or inner products of a signal with some expansion set. Finally, we will briey describe the continuous wavelet transform where both the signal and the transform are functions of continuous variables. This is analogous to the Fourier transform. 2.1 Wavelets and Wavelet Expansion Systems Before delving into the details of wavelets and their properties, we need to get some idea of their general characteristics and what we are going to do with them [405] What is a Wavelet Expansion or a Wavelet Transform? A signal or function f (t) can often be better analyzed, described, or processed if expressed as a linear decomposition by f (t) = l a l ψ l (t) (2.1) where l is an integer index for the nite or innite sum, a l are the real-valued expansion coecients, and ψ l (t) are a set of real-valued functions of t called the expansion set. If the expansion (2.1) is unique, the set is called a basis for the class of functions that can be so expressed. If the basis is orthogonal, meaning < ψ k (t), ψ l (t) > = ψ k (t) ψ l (t) dt = 0 k l, (2.2) then the coecients can be calculated by the inner product a k = < f (t), ψ k (t) > = f (t) ψ k (t) dt. (2.3) One can see that substituting (2.1) into (2.3) and using (2.2) gives the single a k coecient. If the basis set is not orthogonal, then a dual basis set ψ k (t) exists such that using (2.3) with the dual basis gives the desired coecients. This will be developed in Chapter: A multiresolution formulation of Wavelet Systems (Chapter 3).

13 For a Fourier series, the orthogonal basis functions ψ k (t) are sin (kω 0 t) and cos (kω 0 t) with frequencies of kω 0. For a Taylor's series, the nonorthogonal basis functions are simple monomials t k, and for many other expansions they are various polynomials. There are expansions that use splines and even fractals. For the wavelet expansion, a two-parameter system is constructed such that (2.1) becomes f (t) = a j,k ψ j,k (t) (2.4) k j where both j and k are integer indices and the ψ j,k (t) are the wavelet expansion functions that usually form an orthogonal basis. The set of expansion coecients a j,k are called the discrete wavelet transform (DWT) of f (t) and (2.4) is the inverse transform What is a Wavelet System? The wavelet expansion set is not unique. There are many dierent wavelets systems that can be used eectively, but all seem to have the following three general characteristics [405]. 1. A wavelet system is a set of building blocks to construct or represent a signal or function. It is a two-dimensional expansion set (usually a basis) for some class of one- (or higher) dimensional signals. In other words, if the wavelet set is given by ψ j,k (t) for indices of j, k = 1, 2,, a linear expansion would be f (t) = k j a j,k ψ j,k (t) for some set of coecients a j,k. 2. The wavelet expansion gives a time-frequency localization of the signal. This means most of the energy of the signal is well represented by a few expansion coecients, a j,k. 3. The calculation of the coecients from the signal can be done eciently. It turns out that many wavelet transforms (the set of expansion coecients) can calculated with O (N) operations. This means the number of oating-point multiplications and additions increase linearly with the length of the signal. More general wavelet transforms require O (Nlog (N)) operations, the same as for the fast Fourier transform (FFT) [47]. Virtually all wavelet systems have these very general characteristics. Where the Fourier series maps a onedimensional function of a continuous variable into a one-dimensional sequence of coecients, the wavelet expansion maps it into a two-dimensional array of coecients. We will see that it is this two-dimensional representation that allows localizing the signal in both time and frequency. A Fourier series expansion localizes in frequency in that if a Fourier series expansion of a signal has only one large coecient, then the signal is essentially a single sinusoid at the frequency determined by the index of the coecient. The simple time-domain representation of the signal itself gives the localization in time. If the signal is a simple pulse, the location of that pulse is the localization in time. A wavelet representation will give the location in both time and frequency simultaneously. Indeed, a wavelet representation is much like a musical score where the location of the notes tells when the tones occur and what their frequencies are More Specic Characteristics of Wavelet Systems There are three additional characteristics [405], [94] that are more specic to wavelet expansions. 1. All so-called rst-generation wavelet systems are generated from a single scaling function or wavelet by simple scaling and translation. The two-dimensional parameterization is achieved from the function (sometimes called the generating wavelet or mother wavelet) ψ (t) by ψ j,k (t) = 2 j/2 ψ ( 2 j t k ) j, k Z (2.5) where Z is the set of all integers and the factor 2 j/2 maintains a constant norm independent of scale j. This parameterization of the time or space location by k and the frequency or scale (actually the logarithm of scale) by j turns out to be extraordinarily eective. 7

14 8 CHAPTER 2. INTRODUCTION TO WAVELETS 2. Almost all useful wavelet systems also satisfy the multiresolution conditions. This means that if a set of signals can be represented by a weighted sum of ϕ (t k), then a larger set (including the original) can be represented by a weighted sum of ϕ (2t k). In other words, if the basic expansion signals are made half as wide and translated in steps half as wide, they will represent a larger class of signals exactly or give a better approximation of any signal. 3. The lower resolution coecients can be calculated from the higher resolution coecients by a treestructured algorithm called a lter bank. This allows a very ecient calculation of the expansion coecients (also known as the discrete wavelet transform) and relates wavelet transforms to an older area in digital signal processing. The operations of translation and scaling seem to be basic to many practical signals and signal-generating processes, and their use is one of the reasons that wavelets are ecient expansion functions. Figure 2.3 is a pictorial representation of the translation and scaling of a single mother wavelet described in (2.5). As the index k changes, the location of the wavelet moves along the horizontal axis. This allows the expansion to explicitly represent the location of events in time or space. As the index j changes, the shape of the wavelet changes in scale. This allows a representation of detail or resolution. Note that as the scale becomes ner ( j larger), the steps in time become smaller. It is both the narrower wavelet and the smaller steps that allow representation of greater detail or higher resolution. For clarity, only every fourth term in the translation (k = 1, 5, 9, 13, ) is shown, otherwise, the gure is a clutter. What is not illustrated here but is important is that the shape of the basic mother wavelet can also be changed. That is done during the design of the wavelet system and allows one set to well-represent a class of signals. For the Fourier series and transform and for most signal expansion systems, the expansion functions (bases) are chosen, then the properties of the resulting transform are derived and Figure 2.3: Translation (every fourth k) and Scaling of a Wavelet ψ D4 analyzed. For the wavelet system, these properties or characteristics are mathematically required, then the resulting basis functions are derived. Because these constraints do not use all the degrees of freedom,

15 other properties can be required to customize the wavelet system for a particular application. Once you decide on a Fourier series, the sinusoidal basis functions are completely set. That is not true for the wavelet. There are an innity of very dierent wavelets that all satisfy the above properties. Indeed, the understanding and design of the wavelets is an important topic of this book. Wavelet analysis is well-suited to transient signals. Fourier analysis is appropriate for periodic signals or for signals whose statistical characteristics do not change with time. It is the localizing property of wavelets that allow a wavelet expansion of a transient event to be modeled with a small number of coecients. This turns out to be very useful in applications Haar Scaling Functions and Wavelets The multiresolution formulation needs two closely related basic functions. In addition to the wavelet ψ (t) that has been discussed (but not actually dened yet), we will need another basic function called the scaling functionϕ (t). The reasons for needing this function and the details of the relations will be developed in the next chapter, but here we will simply use it in the wavelet expansion. The simplest possible orthogonal wavelet system is generated from the Haar scaling function and wavelet. These are shown in Figure 2.4. Using a combination of these scaling functions and wavelets allows a large class of signals to be represented by f (t) = k= c k φ (t k) + k= j=0 d j,k ψ ( 2 j t k ). (2.6) Haar [198] showed this result in 1910, and we now know that wavelets are a generalization of his work. An example of a Haar system and expansion is given at the end of Chapter: A multiresolution formulation of Wavelet Systems (Chapter 3) What do Wavelets Look Like? All Fourier basis functions look alike. A high-frequency sine wave looks like a compressed low-frequency sine wave. A cosine wave is a sine wave translated by 90 o or π/2 radians. It takes a 9 Figure 2.4: Haar Scaling Function and Wavelet large number of Fourier components to represent a discontinuity or a sharp corner. In contrast, there are many dierent wavelets and some have sharp corners themselves. To appreciate the special character of wavelets you should recognize that it was not until the late 1980's that some of the most useful basic wavelets were ever seen. Figure 2.5 illustrates four dierent scaling

16 10 CHAPTER 2. INTRODUCTION TO WAVELETS functions, each being zero outside of 0 < t < 6 and each generating an orthogonal wavelet basis for all square integrable functions. This gure is also shown on the cover to this book. Several more scaling functions and their associated wavelets are illustrated in later chapters, and the Haar wavelet is shown in Figure 2.4 and in detail at the end of Chapter: A multiresolution formulation of Wavelet Systems (Chapter 3). Figure 2.5: Example Scaling Functions (See Section: Further Properties of the Scaling Function and Wavelet (Section 6.8: Further Properties of the Scaling Function and Wavelet) for the meaning of α and β) Why is Wavelet Analysis Eective? Wavelet expansions and wavelet transforms have proven to be very ecient and eective in analyzing a very wide class of signals and phenomena. Why is this true? What are the properties that give this eectiveness? 1. The size of the wavelet expansion coecients a j,k in (2.4) or d j,k in (2.6) drop o rapidly with j and k for a large class of signals. This property is called being an unconditional basis and it is why wavelets are so eective in signal and image compression, denoising, and detection. Donoho [117], [131] showed that wavelets are near optimal for a wide class of signals for compression, denoising, and detection. 2. The wavelet expansion allows a more accurate local description and separation of signal characteristics. A Fourier coecient represents a component that lasts for all time and, therefore, temporary events must be described by a phase characteristic that allows cancellation or reinforcement over large time periods. A wavelet expansion coecient represents a component that is itself local and is easier to interpret. The wavelet expansion may allow a separation of components of a signal that overlap in both time and frequency. 3. Wavelets are adjustable and adaptable. Because there is not just one wavelet, they can be designed to t individual applications. They are ideal for adaptive systems that adjust themselves to suit the

17 signal. 4. The generation of wavelets and the calculation of the discrete wavelet transform is well matched to the digital computer. We will later see that the dening equation for a wavelet uses no calculus. There are no derivatives or integrals, just multiplications and additionsoperations that are basic to a digital computer. While some of these details may not be clear at this point, they should point to the issues that are important to both theory and application and give reasons for the detailed development that follows in this and other books. 2.2 The Discrete Wavelet Transform This two-variable set of basis functions is used in a way similar to the short-time Fourier transform, the Gabor transform, or the Wigner distribution for time-frequency analysis [65], [68], [217]. Our goal is to generate a set of expansion functions such that any signal in L 2 (R) (the space of square integrable functions) can be represented by the series 11 f (t) = j,k a j,k 2 j/2 ψ ( 2 j t k ) (2.7) or, using (2.5), as f (t) = j,k a j,k ψ j,k (t) (2.8) where the two-dimensional set of coecients a j,k is called the discrete wavelet transform (DWT) of f (t). A more specic form indicating how the a j,k 's are calculated can be written using inner products as f (t) = j,k < ψ j,k (t), f (t) > ψ j,k (t) (2.9) if the ψ j,k (t) form an orthonormal basis 2 for the space of signals of interest [94]. The inner product is usually dened as < x (t), y (t) > = x (t) y (t) dt. (2.10) The goal of most expansions of a function or signal is to have the coecients of the expansion a j,k give more useful information about the signal than is directly obvious from the signal itself. A second goal is to have most of the coecients be zero or very small. This is what is called a sparse representation and is extremely important in applications for statistical estimation and detection, data compression, nonlinear noise reduction, and fast algorithms. Although this expansion is called the discrete wavelet transform (DWT), it probably should be called a wavelet series since it is a series expansion which maps a function of a continuous variable into a sequence of coecients much the same way the Fourier series does. However, that is not the convention. This wavelet series expansion is in terms of two indices, the time translation k and the scaling index j. For the Fourier series, there are only two possible values of k, zero and π/2, which give the sine terms and the cosine terms. The values j give the frequency harmonics. In other words, the Fourier series is also a two-dimensional expansion, but that is not seen in the exponential form and generally not noticed in the trigonometric form. 2 Bases and tight frames are dened in Chapter: Bases, Orthogonal Bases, Biorthogonal Bases, Frames, Right Frames, and unconditional Bases. (Chapter 5)

18 Thank You for previewing this ebook You can read the full version of this ebook in different formats: HTML (Free /Available to everyone) PDF / TXT (Available to V.I.P. members. Free Standard members can access up to 5 PDF/TXT ebooks per month each month) Epub & Mobipocket (Exclusive to V.I.P. members) To download this full book, simply select the format you desire below