S A 1 D 1 A 2 D 2 A 3 D 3 Wavelet and Application in Signal and Image Processing Dr. M.H.Morad 2
3
Objectives This course is aimed to deal with some of the basic concepts, methodologies and tools of signal and Image processing using wavelets. will explore the ideas behind, and the properties of, wavelets. to establish the theory necessary to understand wavelets and related constructions. 4
It is expected that the course work will enable the students to develop the necessary background and proficiency to pursue some of the current research works in wavelets and assist to work with current wavelet software packages in their own area of wavelet applications. The material will be presented in a manner accessible to engineers by emphasizing wavelets as practical methods of Processing. Mathematics of moderate complexity will be introduced only as means necessary to. prior background Fourier Series and Transforms, linear algebra, and matrix theory. MATLAB 5
6 Course Outline
Ch.1: Introduction ; Motivation and Historical Backgrounds Introduction The Origins of Wavelets-Are They Fundamentally New? Wavelets and Other Reality Transforms Managing Heisenberg's Uncertainty Ghost History of Wavelet from Morlet to Daubechies Via Mallat Different Communities of Wavelets Different Families of Wavelets within Wavelet Communities Interesting Recent Developments Wavelets in the Future 7
Ch.2: Bases, Orthogonal Bases, Biorthogonal Bases, Frames, Tight Frames, and Unconditional Bases Introduction Vector Space Definition and properties Vector Subspace Linear Combination,Span Basis Vectors,Vector Norms,Inner Product Completeness Orthogonality, Orthonormal Bases, Orthogonal Projections Biorthogonal bases Frames Overcomplete Expansions: General Frames Overcomplete Expansions: Tight Frames Conditional and Unconditional Bases 8
Ch.3: Continuous Wavelet Introduction and Short Time Fourier Transform Wavelet Transform-A First Level Introduction Mathematical Preliminaries-Fourier Transform Continuous Time-Frequency Representation of Signals The Windowed Fourier Transform (Short Time Fourier Transform) The Uncertainty Principle and Time Frequency Tiling Properties of Wavelets Used in Continuous Wavelet Transform Continuous Versus Discrete Wavelet Transform 9
Ch.4: Discrete Wavelet Transform INTRODUCTION Haar Scaling Functions and Function Spaces Translation and Scaling of () t Orthogonality of Translates of () t Function Space Vo MRA and orthogonal wavelet basis Orthogonality of () t and () t Refinement Relation with Respect to Normalized Bases Haar Wavelet Function Scaled Haar Wavelet Functions Normalization of Haar Bases at Different Scales Solution of the Refinement Equation Iterate the box function - brute force approach Daubechies Wavelets Triangle Scaling Function Cubic Spline Function 10
Ch.5: DESIGNING ORTHOGONAL WAVELET SYSTEMS Introduction A Direct Approach Refinement Relation for Orthogonal Wavelet Systems Restrictions on Filter Coefficients Wavelet Selection Criteria Compactly Supported Wavelets Example Scaling Functions and Wavelets Haar Wavelets Designing Daubechies Orthogonal Wavelet System Coefficients Design of Coiflet wavelets 11
Ch.6 : Discrete WaveIet Transform and Relation to Filter Banks Introduction DECIMATION & INTERPOLATION Down-Sampler & Up-Sampler Aliasing & Imaging Properties of Downsampling & Upsampling Useful Identities Polyphase Representation Cascade of sampling rate converters Computational Requirements FUNDAMENTALS OF FILTER BANKS Perfect Reconstruction quadrature mirror filter (QMF) Uniform Digital Filter Banks Polyphase Implementation Nyquist Filters -Lth-Band Filters 12
Analysis: From Fine Scale to Coarser Scale Mallat s algorithm Number of Levels Input Coefficients Synthesis: From Coarser Scale to Fine Scale PERFECT MATCHING FILTERS DWT and FWT: Significance FWT Algorithm: Outline & An Overview Mallat Filter Bank Transversal Filters Short-length Filters Binomial QMF Classical Lattice CORDIC Lattice Lifting Ladder 13
Ch.7: Computing and Plotting Introduction Scaling and Wavelet Functions Daubechies Lagarias Algorithm Discrete Dilation Equation Statement of Daubechies Lagarias Algorithm Generating binary Equivalent of a Decimal Number Computing Wavelet Function Subdivision scheme Successive Approximation 14
Ch.8: Biorthogonal Wavelets Introduction Biorthogonality in Vector Space Biorthogonal Wavelet Systems Signal Representation Using Biorthogonal Wavelet System Biorthogonal Analysis Biorthogonal Synthesis-From Coarse Scale to Fine Scale Construction of Biorthogonal Wavelet Systems B-splines B-spline Biorthogonal Wavelet System or Cohen- Daubechies- Feauveau Wavelets (CDF) 15
Ch.9: Designing Wavelets Introduction Frequency Domain Approach Basic Properties of Filter Coefficients Filter Properties in Terms of H( ) Filter Properties in Terms of Hz ( z) Frequency Domain Characterization of Filter Coefficients Choice of Wavelet Function Coefficients {g(k)} Vanishing Moment Conditions in Fourier Domain Derivation of Daubechies Wavelets Step Involved in Derivation of Daubechies Wavelets Daubechies Wavelets with One Vanishing Moment Daubechies Wavelets with Two Vanishing Moment 16
Ch.10: Wavelet Packet Analysis Introduction Construction of Wavelet Packets Haar Wavelet Packet Choosing the Best Tree Application Best Basis Selection for Signal or Image Compression Other Cost Functions 17
Ch.11: M-Band Wavelets Introduction Motivation Multi-resolution Formulation of M-Band Wavelet Systems Derivation of the Properties of M-Band Filter Coefficients Supports of Scaling Functions and Wavelets Design of 4-band Symmetric Orthogonal Wavelet Filter Banks Based on { h ( k) 0 } 18
Ch.12: Lifting Scheme Introduction Wavelet Transform Using Polyphase Matrix Factorization Geometrical Foundations of Lifting Scheme Lifting Scheme in the 2-Domain Mathematical Preliminaries for Polyphase Factorization Dealing with Signal Boundary 19
Ch.13: Wavelet System Generalizations Introduction Multiwavelets Two-Dimensional Wavelets Limitations of Wavelet Transforms Complex Wavelet Transforms (CWT) 20
21 Ch.14: Beyond Wavelets Introduction The failure of Wavelets in 2-D The Ridgelet Transform Links with the Radon and wavelet transforms Ridgelets on the sphere Ridgelet Sampling Motivation: Local Ridgelet Transform Orthonormal Ridgelets What is the advantage of ridgelets? Curvelet Transform Point and Curve Discontinuities Sub-band decomposition Smooth partitioning Renormalization Ridgelet analysis
22 The Frequency-Domain Definition of Curvelets Image Reconstruction Second Generation of Curvelets Problem With Curvelet Contourlet Pyramidal Directional Filter Bank Directional Decomposition Sampling in Multiple Dimensions Directional Filter Bank 2 Level Directional Filter Bank 3 Level Directional Filter Bank Pyramid Directional Filter Banks Basis Functions Simulation Results Conclusion
23 Ch.15: Selected Applications Denoising Using Wavelets A Simple Explanation and a 1-D Example Denoising Using Wavelet Shrinkage-Statistical Modelling and Estimation Noise Estimation Shrinkage Functions Shrinkage Rules Denoising Images with Matlab Matlab Programs for Denoising Simulation for Finding Effectiveness of Thresholding Method Compression using wavelets 4W What are the principles behind compression? Compression Framework
24 Image compression steps Compression Performance Compression ratio mean observers score (MOS) Signal to Noise Ratio PRD Cross Correlation Coding delay Coding complexity Reconstruction Aspect Lossless Lossy Functional Aspect Basic ideas of linear transformation Why Wavelet-based Compression? Compression Framework
Embedded Zero-Tree Wavelet Coding (EZW) What does EZW stand for? Type of scanning Significance Pass Refinement Pass EZW Algorithm and Examples Set Partitioning in Hierarchical Trees (SPIHT) Specifications Transformation BitPlane SPIHT algorithm ECG Data Compression A Quality-on-Demand Scheme 25
Texbooks: INSIGHTS INTO WAVELETS: FROM THEORY TO PRACTICE Soman K. P., Ramachandran K. I., Prentice-Hall of India Third Edition 2010. The rapid growth of the theory of wavelets and their application in diverse areas, ranging from oil exploration to bioinformatics and astrophysics-has made it imperative that engineers and scientists working in these areas have a working knowledge of wavelets. In the past few years, the study of wavelets and the exploration of the principles governing their behaviour have brought about sweeping changes in the disciplines of pure and applied mathematics and sciences. Intended primarily as a textbook for the postgraduate students of computer science, electrical/electronics and communication engineering, this book would also be useful for the practising engineers and researchers. 26
27
RECOMMENDED TEXBOOKS : A Wavelet Tour of Signal Processing Stéphane Mallat, Elsevier,Third Edition 2009. Wavelets have opened the door to a flow of new ideas and algorithms that have invaded most traditional fortresses of signal processing. This book travels along the bridges between applications, algorithms and theorems, with an emphasis on intuitive explanations. It is organized as a textbook for electrical engineering and applied mathematical classes. All algorithms and figures are implemented in WaveLab, which is a free Matlab/Octave toolbox, to let readers and students make their own numerical experiments. 28
Introduction to Wavelets and Wavelet Transforms : A Primer C. S. Burrus, Ramesh A. Gopinath, and Haitao Guo, Prentice Hall, 1997. This book is an introduction to wavelets and the discrete wavelet transform intended for people with a background that includes Fourier analysis, matrix algebra, and the equivalent to a BS in engineering, science, or mathematics. It assumes no knowledge of wavelets. The book could be used as a text in an upper level undergraduate or first year graduate course or it could be used for self study by a graduate student, faculty member, or practicing engineer or scientist. The main approach of the book is from the point of view of a time-domain expansion of a signal in terms of a basis or frame. However, it also has a rather complete chapter on filter banks. A small number of basic Matlab programs for designing and displaying wavlets and for taking the discrete wavelet transform are included in the appendix. 29
The World of Fourier and Wavelets M.Vetterli, J. Kovačević and V. K Goyal, (Version 1.1 available, March 2006). 30
Wavelets and Subband Coding Martin Vetterli and Jelena Kovacevic,Prentice Hall,1995. In this book, a comprehensive and unified presentation of discrete and continuous wavelets, filter banks and subband coding, as well as multiresolution signal processing, is given. It is intended for practitioners and researchers in the fields of signal processing and telecommunications, as well as applied mathematics and computer vision. Wavelets and Filter Banks Gilbert Strang and Truong Nguyen Wellesley-Cambridge Press, 1996. The book is already chosen for classes on wavelets.the book is closely linked with the new Wavelet Toolbox for MATLAB, which is available from MathWorks. 31
Ten Lectures on Wavelets, I. Daubechies, SIAM, 1992. This book contains ten lectures I delivered as the principal speaker at the CBMS conference on wavelets organized in June 1990 by the Mathematics Department at the University of Lowell, Massachusetts. Wavelets in Medicine and Biology Akram Aldroubi and Michael Unser,CRC Press, 1996. This work provides guidelines for all those interested in wavelets and their applications to biomedical problems. It includes ready-to-use algorithms for immediate application, and examples illustrating how wavelet transforms are put into practice. 32
Advances in Signal Transforms Theory and Applications, by Astola J., Hindawi Publishing Corporation, 2007. This volume collects some most recent developments in the theory and practice of the design and usage of transforms in digital signal and image processing. The book consists of two parts. The first part contains four chapters devoted to topical issues in the theory of signal transforms. In the second part, advanced practical transform-based signal and image processing algorithms are considered. Wavelet Analysis and Application, Tao Q., Springer,2007. 33
Grading: Final Exams: 50%, Homework / Class Participation: 20%, Final Project: 30%. The grade will be based on homework assignments, including some assignments using MATLAB, and on a project of the student's own choosing to be presented at the end of the semester. A final project report and an oral presentation are required from each project. 34
Projects: Project requirements: Projects should be done individually. Each project must involve using the wavelet transform as a tool. A signal is analyzed/classified, etc by computing its wavelet transform and then the required task (e.g. denoising/classification) is performed in the transform domain. The final projects provide you an opportunity to independently research a topic of your choice in the wavelets and related areas, apply the learnt concepts and techniques to a problem at hand, and explore your original ideas. The projects will be judged based on the level of understanding, originality, and the amount of work during the semester. The selected projects could come from your own interests, either from your current thesis work or an area that you want to explore. 35
Course Project Evaluation : Following guidelines are used in project evaluations and subsequent grading. Complexity of the problem and availability/accessibility to prior research work relevant to the problem, Extent of literature search and study of related research results, Extent of the project effort ( exhibited by the amount of the work done in the project, extent of analysis work, program development, simulation runs, results obtained # of hrs/day, week spent ). Possible new approaches developed in the study (or extension of other research works). Accessibility to existing software codes, development of codes or addition and modification of the existing codes Results and findings 36 Reporting
37 Sample Project Topics : Wavelets in biomedical signal analysis for feature extraction (EEG, ECG,EMG, PPG, radiographic X-rays, MRI and CT images), EMG Decomposition to Motor Unit Action Potentials With Wavelet Unsupervised spike sorting with wavelets Wavelets in Medical Applications for Image Inhancement, Contrast amplification, Edge detection, Wavelet Estimation of Weak Biosignals Wavelets in Watermarking, EEG Segmentation and Wavelet Analysis Wavelets for fingerprint detection Wavelets and Data Fusion. Optimized Wavelets for Blind Separation of Nonstationary Surface Myoelectric Signals
Modeling Fuzziness Measures for Best Wavelet Selection Coding Implementation of a Pyramidal Image Coder Compression of finite-length discrete-time signals using flexible adaptive wavelet packets Wavelet Descriptors for Planar Curves Sinusoidal Modeling of Audio Signals Using Frame- Based Perceptually Weighted Matching Pursuits Automatic Music Genre Classification of Audio Signals Music/Speech Classifier using Wavelets Low Complexity Motion Estimation Algorithm for Long-term Memory Motion Compensation Using Hierarchical Motion Estimation 38
Classification/Recognition Shift Invariant Texture Classification by Using Wavelet Frame Texture Feature Extraction with Non-Separable Wavelet Transforms Comparison of Two Wavelet-Based Image Watermarking Techniques Application of Wavelet Transform in Analysis of Fractal Signals Human-Face Detection and Location in Color Images Using Wavelet Decomposition Global/Local Motion Compensation for 3D Video Coding Based on Lifting Techniques 39
Wavelet Decomposition for the Analysis of Heart Rate Variability Wavelet-based fmri dynamic activation detection Wavelet analysis of evoked potentials Detection of Microcalcifications in Mammograms using Wavelet Transforms Comparison of Denoising via Block Weiner Filtering in Wavelet Domain with Existing Ad-hoc Linear and Nonlinear Denoising Techniques Wavelet-domain filtering of data with Poisson noise Wavelet Packet Best Basis Selection for Signal Classification With Application to Brain Computer Interfaces 40
Watermarking/Halftoning Introduction of IWT to wavelet-based watermarking and its effect on performance Inverse Halftoning using Wavelets Contrast Enhancement and De-noising using Wavelets Wavelet Denoising Applied to Time Delay Estimation Comparison of image denoising using Wavelet Shrinkage vs. MMSE using an exponential decay autocorrelation model Threshold Denoising Effects on Covariance Matrices Information Driven Denosing of MEG data in the Wavelets Domain 41
Continuous wavelet and derivative transforms for the simultaneous quantitative analysis and dissolution test of levodopa benserazide tablets Determination of bismuth and copper using adsorptive stripping voltammetry couple with continuous wavelet transform Entropy-Based Optimization of Wavelet Spatial Filters 42
Useful Links Wavelet Digest A free monthly newsletter which contains all kinds of information concerning wavelets Amara's Wavelet Page An extensive collection of wavelet resources on WWW 43 Wavelet Tutorial An excellent wavelet tutorial for engineers Wavelet Books Online Online resources on wavelet books used in class Wavelet Applets:
44