VIETNAM NATIONAL UNIVERSITY UNIVERSITY OF SCIENCE FACULTY OF MATHEMATICS, MECHANICS AND INFORMATICS. Hoang Dinh Linh WAVELETS ON Z N

Transcription

1 1 VIETAM ATIOAL UIVERSITY UIVERSITY OF SCIECE FACULTY OF MATHEMATICS, MECHAICS AD IFORMATICS Hoang Dinh Linh WAVELETS O Z Undergraduate Thesis Advanced Undergraduate Program in Mathematics Hanoi

2 VIETAM ATIOAL UIVERSITY UIVERSITY OF SCIECE FACULTY OF MATHEMATICS, MECHAICS AD IFORMATICS Hoang Dinh Linh WAVELETS O Z Undergraduate Thesis Advanced Undergraduate Program in Mathematics Thesis advisor: PhD. Dang Anh Tuan Hanoi

3 Acknowledgments It could be said that this undergraduate thesis would not have been able to write without the help, and support, of the kind people around me, to only some of whom it is possible to give particular mention here. Firstly, I am deeply grateful to my advisor PhD. Dang Anh Tuan. His patient guidance and kind support made this work possible. I am thankful for the chance to study with him and appreciate all the help he has given to me for all these years. Secondly, I wish to thank my teachers at Faculty of Mathematics, Mechanics and Informatics, University of Sciences, Vietam ational University who equip me with important mathematics knowledge during first four years at the university. ext, I would like to express my deepest gratitude to my family. Without their unconditional love and support, I would not be able to do what I have accomplished. Last, but not least, I would like to thank to my classmates for their friendship and suggestion. I will never forget their care and kindness. Thank you for all the help and making the class like a family. Hanoi, October 30 th, 2013 Student. Hoang Dinh Linh. i

4 Contents Acknowledgments Contents Introduction i ii iii 1 The Discrete Fourier Transform Definitions and Basic Properties Translation-Invariant Linear Transformations The Fast Fourier Transform Wavelets on Z Construction of Wavelets on Z : The First Stage Construction of Wavelets on Z : The Iteration Step Conclusion 63 References 64 ii

5 Introduction Wavelets are mathematical functions that cut up data into different frequency components, and then study each component with a resolution matched to its scale. They have advantages over traditional Fourier methods in analyzing physical situations where the signal contains discontinuities and sharp spikes. Wavelets were developed independently in the fields of mathematics, quantum physics, electrical engineering, and seismic geology. Interchanges between these fields during the last ten years have led to many new wavelet applications such as image compression, turbulence, human vision, radar, and earthquake prediction. In this thesis, we study how to construct Wavelets on Z and its properties. The thesis is divided into the two chapters, namely, Chapter 1: The Discrete Fourier Transform. Chapter 2: Wavelets on Z. Chapter 1 introduces the basic concepts related to the Discrete Fourier Transform. This chapter concerns with two advantages of the Fourier basis, one is diagonalizes translation- invariant linear transformation. And another is using the Fast Fourier Transform to compute the coordinates in the Fourier basis. Chapter 2 provides the basic properties of Wavelets on Z, from that we do step by step to construct a general first-stage wavelets basis for l 2 (Z ). Then, inductively, we find the way to build p th -stage wavelets basis in the case is divisible by 2 p. The main materials of the thesis were taken from the book: An Introduction to Wavelets through Linear Algebra, written by Prof. Michael W.Frazier. Beside most of contents of the thesis come from the book, I have calculated some proposition in some particular cases, in Chapter 2. Moreover, I have used some result of Prof. Ole Chritensen in [2]. iii

6 CHAPTER 1 The Discrete Fourier Transform 1.1 Definitions and Basic Properties We work with the vector space C,that is, sequences of complex numbers in this thesis. When working with vectors and bases in C, one has to be particularly careful with the meaning of the notation. For example: f k, g k be vectors in C ; c k be scalars. In order to avoid confusion, we will change the notation slightly in several ways. First, for reasons that will be more clear later, we index these numbers over j {0, 1,..., 1} instead of {1, 2,..., }. Second, instead of writing the components of z as z j, we write them as z(j). This indicates a new point of view: we regard z as a function defined on the finite set Z {0, 1,..., 1} To save space, we usually write such a z horizontally instead of vertically: z (z(0), z(1),..., z( 1)). However, when convenient, we still identify z with the column vector z(0) z(1) z.. (1.1.1) z( 1) This allows us to write the product of an matrix A by z as Az. Finally, we write l 2 (Z ) in place of C. So, formally, l 2 (Z ) {z (z(0), z(1),..., z( 1)) : z(i) C, 0 i 1}. With the usual component wise addition and scalar multiplication, l 2 (Z ) is an -dimensional vector space over C. One basis for l 2 (Z ) is the standard, or 1

7 1.1. Definitions and Basic Properties Euclidean, basis E {e 0, e 1,..., e }, where e j (n) 1 if n j and e j (n) 0 if n j. In this notation, the complex inner product on l 2 (Z ) is with the associated norm z, w z(n)w(n), z ( z(n) 2 ) 1 2. (called the l 2 -norm). We maintain the notion of orthogonality: z w if and only if z, w 0. We make one more convention. Originally, for z l 2 (Z ), z(j) is defined for j 0, 1,..., 1. ow we extend z to be defined at all integers by requiring z to be periodic with period : z(j + ) z(j), for all j Z. Hence, to find z(j) for j 0, 1,..., 1, add some positive or negative integer multiple m of to j until j + m 0, 1,..., 1; then define z(j) z(j + m). For example, if 13, then z( 9) z( 22) z(4) z(17). ote that the value of z(j) depends only on the residue of j modulo ; one can regard z as defined on the equivalence classes of Z mod. In particular we can regard z as defined on any other set of consecutive integers instead of {0, 1,..., 1}. Definition 1.1 Defined E 0, E 1,..., E in l 2 (Z ) by E k (n) 1 e 2πik n, for n, k 0, 1,..., 1. (1.1.2) That is E k 1 1 e 2πi k e 4πi k., for k 0, 1,..., 1. e 2πi() k We use the capital E notation to suggest the exponential function instead of using the lower-case e, which we reserve for the standard basis vectors noted earlier. 2

8 1.1. Definitions and Basic Properties Lemma 1.2 The set {E 0, E 1,..., E } constitute an orthonormal basis for l 2 (Z ). PROOF Suppose k, l {0, 1,..., 1}. Then Using the formula with x e k l 2πi E k, E l 1 E k (n)e l (n) 1 e 2πik n e 2πil n 1 k l 2πin e. (1 x)(1 + x x n 1 ) 1 x n. Then, if k l, E k, E l And E k, E l 0 for all k l. Thus {E 0, E 1,..., E } is an orthonormal set, and hence is linearly independent. Therefore {E 0, E 1,..., E } is a basis for l 2 (Z ). Example 1.3 Let 3, then E 0 (E 0 (0), E 0 (1), E 0 (2)) 1 (1, 1, 1), 3 E 1 (E 1 (0), E 1 (1), E 1 (2)) 1 (1, i 2, i 2 ), E 2 (E 2 (0), E 2 (1), E 2 (2)) 1 (1, i 2, i 2 ). It is clear that {E 0, E 1, E 2 } is an orthonormal basis for l 2 (Z 3 ). For 4, then E 0 1 (1, 1, 1, 1), 2 E 1 1 (1, i, 1, i), 2 E 2 1 (1, 1, 1, 1), 2 E 3 1 (1, i, 1, i). 2 One can check directly that {E 0, E 1, E 2, E 3 } is an orthonormal basis for l 2 (Z 4 ). 3

9 1.1. Definitions and Basic Properties For all z, w l 2 (Z ), since {E 0, E 1,..., E } is an orthonormal basis for l 2 (Z ) then and By definition, z, E m z z, E m E m, (1.1.3) m0 z, w z, E m w, E m, (1.1.4) z 2 m0 m0 z(n) 1 e 2πim n z, E m 2. (1.1.5) z(n) 1 e 2πim n. (1.1.6) Definition 1.4 Suppose z (z(0),..., z( 1)) l 2 (Z ). For m 0, 1,..., 1, define Let ^z(m) z(n)e 2πim n. (1.1.7) ^z (^z(0), ^z(1),..., ^z( 1)). (1.1.8) Then ^z l 2 (Z ). The map^: l 2 (Z ) l 2 (Z ), which takes z to ^z, is called the discrete Fourier transform, usually abbreviated DFT. otice that if we use formula (1.1.7) to define ^z(m) for all m Z, the result is periodic with period : ^z(m + ) z(n)e 2πi(m+) n z(n)e 2πim n e 2nπi ^z(m). Thus using formula (1.1.7) for all m is consistent with regarding ^z as an element of l 2 (Z ), thought of as defined on Z and having period. There are a couple of advantages of formula (1.1.7) compared with formula (1.1.6). First, for numerical calculations, it is better to avoid computing. Second, we 4

10 1.1. Definitions and Basic Properties will see later that certain formulas (e.g., the formula for the DFT of a convolution, in Lemma below) are simpler with the normalization in formula (1.1.7). Comparing formulae (1.1.7) and (1.1.6), note that ^z z, E m. (1.1.9) Lemma 1.5 Let z (z(0), z(1),..., z()), w (w(0), w(1),..., w()) l 2 (Z ). Then i. (Fourier inversion formula) ii. (Parseval s relation) iii. (Plancherel s formula) PROOF We have z(n) 1 z, w 1 z(n) z, E m E m (n) Similarly, m0 m0 z 2 1 m0 m0 ^z(m)e 2πim n. (1.1.10) ^z(m) ^w(m) 1 ^z, ^w. (1.1.11) m0 z, w z, E m w, E m m0 1 m0 ^z(m) 2 ^z 2. (1.1.12) ^z(m) 2 e 2πim n 1 ^z(m)e 2πim n. m0 m0 ^z(m) ^w(m 1 ^z, ^w. 1 2 ^z(m) 1 2 ^w(m) Then equation (1.1.12) follows either by a similar argument or by letting w z in equation (1.1.11). To interpret the Fourier inversion formula (1.1.10), we make the following definition. 5

11 1.1. Definitions and Basic Properties Definition 1.6 For m 0, 1,..., 1, define F m l 2 (Z ) by Let We call F the Fourier basis for l 2 (Z ). F m (n) 1 e2πim n, for n 0, 1,..., 1. (1.1.13) F (F 0, F 1,..., F ). (1.1.14) By equation (1.1.2), F m 1 2 E m. Hence Lemma 1.2 shows that F is an orthogonal basis for l 2 (Z ). With this notation, equation (1.1.10) becomes z m0 ^z(m)f m. (1.1.15) In other words, if we expand z in terms of the Fourier basis F, the coefficient of F m is ^z(m). Therefore, the vector representing z with respect to the Fourier basis is ^z; denoted by, ^z [z] F. Thus the Fourier inversion formula (1.1.10) is the change-of-basis formula for the Fourier basis. The DFT components ^z(m) are the components of z in the Fourier basis. Since the map taking z to ^z is a linear transformation then the DFT can be represented by a matrix. To simplify notation, define In this notation ^z(m) ω e 2π i. z(n)ω mn. Definition 1.7 Let W be the matrix [w mn ] 0 m,n such that w mn ω mn. Written out, this is W ω ω 2 ω 3 ω 1 ω 2 ω 4 ω 6 ω 2().. 1 ω. ω 2(). (1.1.16).. ω 3() ω ()() 6

12 1.1. Definitions and Basic Properties Regarding z, ^z l 2 (Z ) as column vectors, as in equation (1.1.1), the m th component (0 m 1) of W z is w mn z(n) z(n)ω mn, which is ^z(m). In other words, ^z W z. (1.1.17) We will see that there is a fast algorithm for computing ^z latter. For now, we only compute a simple example in order to demonstrate the definitions. Example 1.8 Let z (0, 1, 2, 3) l 2 (Z 4 ). Find ^z. We have: i 1 i 1 ^z W 4 z i 2. 1 i 1 i 3 2 2i The Fourier inversion formula (1.1.10) shows that the linear transformation ^: l 2 (Z l 2 (Z )) is a 1 1 map: if ^z ^w, then z w. Therefore^is invertible, denoted byˇ Definition 1.9 For w (w(0),..., w( 1)) l 2 (Z ), define Let ˇw(m) 1 w(n)e 2πim n, for n 0, 1,..., 1. (1.1.18) ˇw ( ˇw(0), ˇw(1),..., ˇw( 1)). The mapˇ: l 2 (Z ) l 2 (Z ) is the inverse discrete Fourier transform, or IDFT. ote that we also have the same relation as Parseval s relation z, w And so is Plancherel s formula m0 z 2 ˇz(m) ˇw(m) ˇz, ˇw. m0 ˇz(m) 2 ˇz 2. 7

13 1.1. Definitions and Basic Properties In this notation, equation (1.1.10) states that for z l 2 (Z ), or just (^z)ˇ(n) z(n) for 0, 1,..., 1, (1.1.19) (^z)ˇ z. (1.1.20) For an arbitrary w l 2 (Z ), there exists z l 2 (Z ) such that ^z w (since ^ : l 2 (Z ) l 2 (Z ) is onto). Taking the DFT of both sides of equation (1.1.20) and substituting w for ^z gives ( ˇw)^ w. (1.1.21) Since the DFT is an invertible linear transformation, the matrix W is invertible, and we must have z W 1 z. Substituting ^z w and (equivalently) z ˇw in equation (1.1.17) gives We have ˇw(n) m0 ˇw W 1 w. (1.1.22) w(m) 1 ω mn 1 ωmn m0 w(m). This shows that the (n, m) entry of W 1 is ωmn, which is 1 times the complex conjugate of the (n, m) entry of W. If we denote by W the matrix whose entries are the complex conjugates of the entries of W, we have W 1 1 W. (1.1.23) Example 1.10 Let w (2, 4 + 4i, 6, 4 4i) l 2 (Z 4 ). Find ˇw. We have: ˇw W 1 4 w i 1 i 2 + 2i i 1 i 2 2i 3 8

14 1.1. Definitions and Basic Properties By the same reasoning as for the DFT, if we regard ˇw as defined on Z by formula (1.1.18), then ˇw has period : ˇw(m + ) ˇw(m) for all n. With this understanding, comparing formulas (1.1.7) and (1.1.18), we see that ˇw(n) 1 ^w( n) 1 ^w( n). (1.1.24) We summarize the basic facts about the DFT. The map ^ : l 2 (Z ) l 2 (Z ) defined by equations (1.1.7) and (1.1.8) is an invertible linear transformation with inverse ˇz defined by formula (1.1.18). We interpret the Fourier inversion formula (1.1.10) in the form of equation (1.1.15): z m0 where F m is the m th element of the Fourier basis F : ^z(m)f m, (1.1.25) F m (n) 1 e2πim n. Thus ^z(m) is the weight of the vector F m used in making up z. We consider a simple example. Let 27 and By Euler s formula, Then, by equation (1.1.15) z(n) 12 cos(2π 13n 11n ) + 5 cos(2π ). z(n) n (e2π 27 + e 2π 13n 1 11n 27 ) (e2π 27 + e 2π 11n 27 ) 1 13n (162e2π e 2π 14n e 2π 11n e 2π 16n 27 ). 27 ^z(13) ^z(14) 162; ^z(11) ^z(16) 135. and ^z(m) 0 for the other values of m in {0, 1, 2,..., 26}. ext we consider the DFT behavior under a few important operations. The first of these is translation. Definition 1.11 Suppose z l 2 (Z ) and k Z. Define (R k z)(n) z(n k) for n Z. (1.1.26) We call R k z the translate of z by k. We call R k the translation by k operator. We note that the translation operator is invertible, and R 1 k R k. Example 1.12 Let z (1, i, 0, 3 + 2i, 1 i) l 2 (Z 5 ). Find R 2 z? 9

15 1.1. Definitions and Basic Properties By definition, R 2 z(0) z(0 2) z( 2) z(3) 3 + 2i, R 2 z(1) z( 1) z(4) 1 i, R 2 z(2) z(0) 1, R 2 z(3) z(1) i, R 2 z(4) z(2) 0. Therefore, R 2 z (3 + 2i, 1 i, 1, i, 0). Thus the effect of R 2 on z is moving the components two positions to the right, except for the last two, which were rotated around into the first two positions, in the same order as originally. This can be visualized as a rotation by two if the positions 0, 1, 2, 3, 4, 5 are marked off on a circle. For this reason, this operation is sometimes known as circular translation or rotation, which explains the notation R k. There are some properties about the translation operator Proposition 1.13 Suppose z l 2 (Z ) and k Z. Let R k be the translation by k operator. Then R k is the unitary operator, that mean where R * k is the conjugate of R k. PROOF We have R k z, w R * k R 1 k R k for all k Z. (1.1.27) R k z(n)w(n) z(n k)w(n) z(l)w(l + k) z, R k w. l0 A question arises is : How is the DFT affected by translation?. Then we get the following lemma: Lemma 1.14 Suppose z l 2 (Z ) and k Z. Then for any m Z, (R k z)^(m) e 2πim k ^z(m). (1.1.28) 10

16 1.1. Definitions and Basic Properties PROOF By definition, (R k z)^(m) Letting l n k, then R k z(n)e 2πim n z(n k)e 2πim n. (R k z)^(m) k 1 l k l+k 2πim z(l)e e 2πim k e 2πim k k 1 z(l)e 2πim l l k z(l)e 2πim l l0 e 2πim k ^z(m). Therefore, the translation should not affect the magnitudes of the different frequencies making up the signal, but it might change their phase. The next operation we consider is complex conjugation. Definition 1.15 For z (z(0), z(1),..., z( 1)) l 2 (Z ). Let z be the vector that is z(n) z(n). Lemma 1.16 For z l 2 (Z ), for all m Z. PROOF By the definition, z (z(0), z(1),..., z( 1)), (z)^(m) ^z( m) ^z( m), (1.1.29) (z)^(m) z(n)e 2πim n z(n)e 2πim n ^z( m). Corollary 1.17 Suppose z l 2 (Z ). Then z is real (i.e., every component of z is a real number) if and only if ^z(m) ^z( m) for all m. Similarly, z is pure imaginary (i.e., every component of z is pure imaginary) if and only if ^z(m) ^z( m) for all m. PROOF ote that z is real if and only if z z. By the invertibility of the DFT, this holds if and only if ^z ^z. By Lemma 1.16, this is equivalent to ^z(m) ^z( m) for all m. Similar, z is pure imaginary if and only if z z. Therefore, ^z(m) ^z( m) for all m. 11

17 1.1. Definitions and Basic Properties ext,we define the convolution of two vectors Definition 1.18 For z, w l 2 (Z ), the convolution z * w l 2 (Z ) is the vector with components for all m. z * w(m) z(m n)w(n) If we fix one vector in the convolution,we can regard convolution with this fixed vector as a linear transformation Definition 1.19 Suppose b l 2 (Z ). Define T b : l 2 (Z ) l 2 (Z ) by T b (z) b * z, for all z l 2 (Z ). Any transformation T of the form T T b, for some b l 2 (Z ), is called a convolution operator. We now consider how the DFT interacts with convolution. Lemma 1.20 Suppose z, w l 2 (Z ). Then for each m, (z * w)^(m) ^z(m) ^w(m), (1.1.30) and (^z * ^w)(m) (zw)^(m). (1.1.31) PROOF By the definition, (z * w)^(m) (z * w)(n)e 2πim n z(n k)w(k)e w(k)e 2πim k By setting l n k in the last sum, we obtain (z * w)^(m) w(k)e 2πim k w(k)e 2πim k 12 n k 2πim e 2πim k n k 2πim z(n k)e. k 1 l k l0 z(l)e 2πim l z(l)e 2πim l z(n k)w(k)e 2πim n ^w(m)^z(m).

18 1.1. Definitions and Basic Properties Beside, (^z * ^w)(m) ^z(m n) ^w(n) z(k)e 2πi(m n) k z(k)e 2πim k l0 l0 w(l) w(l)e 2πin l k l 2πin e. If k l then the last sum become. Otherwise, it should be 0. Indeed, we have k l 2πin e 1 e 2πi(k l) k l 2πi 1 e k l 2πi Since k l 0 then 1 e 0 for all k, l 0, 1,..., 1. Therefore, (^z * ^w)(m) z(k)w(k)e 2πim k (zw)^(m). ote that we also have the same properties for IDFT (z * w)ˇ(n) ˇz(n) ˇw(n), (1.1.32) and (ˇz * ˇw)(n) (zw)ˇ(n). (1.1.33) ow, we consider the connection between convolution and inner products Definition 1.21 For any w l 2 (Z ), define w l 2 (Z ) by w(n) w( n) w( n) for all n. (1.1.34) We call w the conjugate reflection of w. ow, we consider some properties of z relative with convolution and inner product. Lemma 1.22 Suppose z, w l 2 (Z ). For any n, k Z, ( w)^(n) ^w(n), (1.1.35) and (z * w)(k) z, R k w, (1.1.36) 13

19 1.1. Definitions and Basic Properties Beside, (z * w)(k) z, R k w. (1.1.37) (z * w) (n) ( z * w)(n). (1.1.38) PROOF We have By the definition, ( w)^(n) z, R k w +1 l0 ^w(n). w(k)e 2πin w(l)e 2πin l k z(n)r k w(n) l0 w( k)e 2πin k w(l)e 2πin l z(n)w(n k) z(n) w(k n) ( w * z)(k) (z * w)(k). Moreover, we note that w w then By the Definition 1.21, (z * w)(k) (z * w)(k) z, R k w. (z * w) (n) (z * w)( n) l0 m0 z( n + l)w(l) ( z * w)(n). z( n m)w(m) l0 z(n l) w(l) Suppose w l 2 (Z ) is such that B {R k w} l 2 (Z ). Then is an orthonormal basis for [z] B z * w. Therefore, the coefficients of the expansion of a vector z in terms of B are just the components of z * w. ow, we define another definition 14

20 1.2. Translation-Invariant Linear Transformations Definition 1.23 For any z l 2 (Z ), define z * l 2 (Z ) such that z * (n) ( 1) n z(n), n Z. (1.1.39) We will compute the DFT of z * Lemma 1.24 Suppose 2M. For any z, w l 2 (Z ) then (z * )^(n) ^z(n + M) (1.1.40) for all n. And (w * w + (w * w) * )(n) { 2w * w(n) if n even, 0 if n odd. (1.1.41) PROOF By the definition, (z * )^(n) z * (k)e 2πin k ( 1) k z(k)e 2πin k e πik z(k)e 2πin k ^z(n + M). z(k)e 2πi(n+M) k ow, we observe that for any z l 2 (Z ), with 2M { (z + z * 2z(n) if n even, )(n) 0 if n odd. (1.1.42) Then, (w * w + (w * w) * )(n) { 2w * w(n) if n even, 0 if n odd. 1.2 Translation-Invariant Linear Transformations ow a day, we sometime study about Signals and Systems in the electrical engineer. In mathematics we consider a signal is just a function which may be on an interval of real numbers (a continuous, or analog, signal); on a finite set of points or on an infinite discrete set such as Z (a discrete, or digital, signal). A system is something that transforms an input signal into an output signal. 15

21 1.2. Translation-Invariant Linear Transformations Mathematically, a system is a transformation. A transformation T should be linear and time-invariant(or shift-invariant). Time-invariant means that if we delay our input signal by a certain amount, the only effect on the output is to delay it by the same amount. In other words, the system does not behave differently at different times of the day.this transformation is called translation invariant. Definition 1.25 Let T : l 2 (Z ) l 2 (Z ) be a linearly transformation. T is translation invariant if By the definition, we have implies that T commute with R k. T (R k z) R k T (z), z l 2 (Z ), k Z. (1.2.1) T (z) R 1 k T (R kz). In this section, we will prove the main idea that is all translation invariant linear transformations T : l 2 (Z ) l 2 (Z ) are diagonalized by Fourier basis. Theorem 1.26 Let T : l 2 (Z ) l 2 (Z ) be a translation invariant linear transformation. Then each element of the Fourier basis F is an eigenvector of T. In particular, T is diagonalizable. PROOF Fix m {0, 1,.., 1}. Let F m be the m th element of the Fourier basis. There exist a 0, a 1,..., a C such that We have Therefore, T (F m )(n) (R 1 F m )(n) F m (n 1) e 2πi m Fm (n). T (R 1 F m )(n) e 2πi m T (Fm )(n) a k F k (n), n. (1.2.2) a k e 2πi m Fk (n), (R 1 T (F m ))(n) (T (F m ))(n 1) a k F k (n 1) Beside, T (R 1 F m )(n) (R 1 T (F m ))(n), n, then a k e 2πi m ak e 2πi k for each k 0, 1,..., 1. a k e 2πi k Fk (n). 16

22 1.2. Translation-Invariant Linear Transformations If k m then e 2πi m e 2πi k for each k 0, 1,..., 1. Thus, a k 0 for all k 0, 1,..., 1. This implies that T (F m )(n) a m F m (n), n. Therefore, F m is an eigenvector of T with eigenvalue a m. Since m is arbitrary then F is eigenvectors of T. In particular, T is diagonalized by F. Along the way we will encounter a variety of key concepts such as circulant matrices, convolutions, and Fourier multipliers. Definition 1.27 A matrix Λ [a mn ] 0 m,n, is periodized with period, i.e., a m+,n a mn and a m,n+ a mn for m, n Z, is circulant if a m+k,n+k a m,n (1.2.3) for all m, n, k Z. Example 1.28 The matrix i 1 4i 4i i 1 1 4i i 2 + i 1 4i 3 is circulant. ow, we are going to state and prove the following theorem to see the relation between translation invariant linear transformations, convolution operators and multiplier operators. Theorem 1.29 Let T : l 2 (Z ) l 2 (Z ) be a linear transformation. Then the following statements are equivalent: i, T is translation invariant. ii, The matrix Λ T,E representing T in the standard basis E is circulant. iii, T is a convolution operator. iv, T is a Fourier multiplier operator. v, The matrix Λ T,F is diagonal. 17

23 1.2. Translation-Invariant Linear Transformations PROOF The strategy of the proof of Theorem 1.29 is to prove i ii iii i, and then to prove iii iv and finally iv v: i ii, Suppose that T is translation invariant. Since z [z] E then T (z) Λ T,E z. For each m, n Z, we have So, Λ T,E is circulant. ii iii, Let b(n) a n,0 (i.e.,b Λe 0 ). a m+1,n+1 (Λ T,E e n+1 )(m + 1) (T (e n+1 ))(m + 1) (T (R 1 e n ))(m + 1) (R 1 T (e n ))(m + 1) T (e n )(m) (Λ T,E e n )(m) a m,n. Since Λ T,E is circulant then a mn a m n,0 b(m n). Therefore, T z(m) (Λ T,E z)(m) a mn z(n) b(m n)z(n) b * z(m). iii i, Let z l 2 (Z ), k Z. For all m, we have By setting l n k, T b (R k z)(m) b * (R k z)(m) T b (R k z)(m) k 1 l k b(m n)(r k z)(n) b(m n)z(n k). b(m l k)z(l) l0 (b * z)(m k) R k (b * z)(m) R k T b (z)(m). b(m l k)z(l) We have proved that statements i, ii, and iii in Theorem 1.29 are equivalent. ow we consider linear transformations obtained by taking the DFT, multiplying the resulting components by some numbers, and taking the IDFT of the result. 18

24 1.2. Translation-Invariant Linear Transformations Definition 1.30 Let m l 2 (Z ). Define T (m) : l 2 (Z ) l 2 (Z ) by T (m) (z) (m^z)ˇ, (1.2.4) where m^z is the vector obtained from multiplying m and ^z componentwise; that is, (m^z)(n) m(n)^z(n) for each n. Any transformation of this form is called a Fourier multiplier operator. ote that (T (m) (z))^(k) m(k)^z(k) and z T (m) (z) (T (m) (z))^(k)f k m(k)^z(k)f k. ^z(k)f k. Therefore, Thus the effect of T (m) on z is to multiply the k th DFT coefficient ^z(k) by m(k). iii iv, Applying Lemma 1.20, we have So, one can choose m ^b. T b (z) b * z ((b * z)^)ˇ (^b^z)ˇ T (^b) (z). iii v, Suppose T is a convolution operator. Since ^z [z] F then m(0)^z(0) m(1)^z(1) [T (m) (z)] F. m( 1)^z( 1) d ^z(0) 0 d 11 0 ^z(1) d, ^z( 1) Λ T,F ^z Λ T,F [z] F. Conversely, assume that Λ T,F diag(m(0), m(1),..., m( 1)) represents T with respect to the Fourier basis F. Then [T (z)] F Λ T,F [z] F [T (m) (z)] F where m (m(0), m(1),..., m( 1)). Therefore, T T (m). 19

25 1.2. Translation-Invariant Linear Transformations Example 1.31 Let T : l 2 (Z 4 ) l 2 (Z 4 ) by (T (z))(n) 3z(n 1) + z(n). Find the eigenvalues and eigenvectors of T, and diagonalize the matrix A representing T in the standard basis, if possible. We can check that T is translation-invariant T (R k z)(n) 3R k z(n 1) + R k z(n) 3z(n 1 k) + z(n k) R k (3z(n 1) + z(n)) R k (T (z))(n). We have (T (z))(0) 3z( 1) + z(0) z(0) + 3z(3), (T (z))(1) 3z(0) + z(1), (T (z))(2) 3z(1) + z(2), (T (z))(3) 3z(2) + z(3). Therefore, [T (z)] E [z] E We can see that Λ T,E is circulant. ow, consider the first column vector of Λ T,E. We have b Λ T,E e 0 (1, 3, 0, 0). Moreover, (b * z)(n) 3 n 3 b(n k)z(k) b(l)z(n l) ln 3 b(l)z(n l) z(n) + 3z(n 1) (T (z))(n). l0 Therefore, T T b is a convolution operator with b (1, 3, 0, 0). Applying Theorem 1.29, we obtain m ^b (4, 1 3i, 2, 1 + 3i). 20

26 1.3. The Fast Fourier Transform Then T T (m) is a multiplier operator. And it implies that Λ T,F are diagonal Λ T,F W 4 Λ T,E W i i Hence, all eigenvalues of Λ T,E are 4, 1 3i, 2, 1 + 3i. 1.3 The Fast Fourier Transform In this section we discuss another key feature of the Fourier basis that we can compute the DFT by a fast algorithm, abbreviated by FFT. ow, we try to calculate the amount of computations required for general computing the DFT. We have ^z [z] F W z, where W is the matrix in equation (1.1.16). So direct computation of ^z takes 2 complex multiplications. In detail, there are many additional computations but multiplication is much slower on a computer than addition. Then we just considering the number of complex multiplications required. In signal and image processing, the vectors under consideration can be very large. So a fast algorithm is needed. If the length is assumed to be even then Lemma 1.32 Suppose M, 2M. Let z l 2 (Z ). Define u, v l 2 (Z M ) by u(k) z(2k) for k 0, 1,..., M 1, v(k) z(2k + 1) for k 0, 1,..., M 1. In other words, u (z(0), z(2),..., z(2m 2)), v (z(1), z(3),..., z(2m 1)). Let ^z denote the DFT of z defined on points, that is, ^z W z. Let ^u and ^v denote the DFTs of u and v respectively, defined on M points, that is, ^u W 2 Mu and ^v W M v. Then for m 0, 1,..., M 1, ^z(m) ^u(m) + e 2πi m ^v(m). (1.3.1) Also, for m M, M +1, M +2,..., 1, let l m M. ote that the corresponding values of l are l 0, 1,..., M 1. Then ^z(l) ^u(l) e 2πi l ^v(l). (1.3.2) 21

27 1.3. The Fast Fourier Transform PROOF By the definition, for m 0, 1,..., M 1 ^z(m) M 1 M 1 z(n)e 2πim n 2k 2πim z(2k)e M 1 + M 1 2k 2πim z(2k)e + e 2πi m M 1 u(k)e 2πim k M ^u(m) + e 2πi m ^v(m). + e 2πi m 2k+1 2πim z(2k + 1)e 2k 2πim z(2k + 1)e M 1 v(k)e 2πim k M Similarly, for m M, M + 1,..., 1 let l m M. Then ^z(m) M 1 u(k)e 2πim k M M 1 u(k)e 2πi(l+M) k M ^u(l) e 2πi l ^v(l). + e 2πi m M 1 + e 2πi l+m v(k)e 2πim k M M 1 v(k)e 2πi(l+M) k M Example 1.33 Let z (1, 1, 1, 1 + i 3, 1, 1 i 3). Find ^z? Applying Lemma 1.32, we get u (1, 1, 1) and v (1, i 2, i 2 ). We can compute ^z as the following FIGURE 1 22

28 1.3. The Fast Fourier Transform We have Hence, by equation (1.3.1) ^u (3, 0, 0) and ^v (0, 3, 0). By equation (1.3.2) ^z(0) ^u(0) + ^v(0) 3, ^z(1) ^u(1) + e 2πi 1 6 ^v(1) 3( 1 2 i ^z(2) ^u(2) + e 2πi 2 6 ^v(2) 3( 1 2 i 3 2 ), 3 2 ). ^z(3) ^u(0) ^v(0) 3, ^z(4) ^u(1) e 2πi 1 6 ^v(1) 3( 1 2 i ^z(5) ^u(2) e 2πi 2 6 ^v(2) 3( 1 2 i 3 2 ), 3 2 ). The main idea of this procedure obtains ^z(m) and ^z(m + M) from the values of ^u(m) and ^v(m). For any positive integer, denote to be the least number of complex multiplications required to compute the DFT of a vector of length. Since each vector ^u(m) and ^v(m) is of the length M, each can be computed 2 with M. We then compute the products e 2πi m ^v(m) for m 0, 1,..., M 1. This requires an additional M multiplications. So, 2 M + M. (1.3.3) What if is not even? If is prime then the method of the FFT is not available. If is composite, say pq, then we can apply the FFT Lemma 1.34 Suppose p, q, pq. Let z l 2 (Z ). Define ω 0, ω 1,..., ω p 1 l 2 (Z q ) and v 0, v 1,..., v q 1 l 2 (Z p ) ω l (k) z(kp + l) for k 0, 1,..., q 1, (1.3.4) v m (l) e 2πm l ^ωl (m) for l 0, 1,..., p 1. (1.3.5) Then for n 0, 1,..., p 1 and m 0, 1,..., q 1 ^z(nq + m) ^v m (n). (1.3.6) 23

29 1.3. The Fast Fourier Transform PROOF ote that for r 0, 1,..., 1 then r is of the form kp + l, for l 0, 1,..., p 1 and k 0, 1,..., q 1, uniquely. We have ^z(nq + m) r0 p 1 q 1 l0 p 1 l0 p 1 l0 p 1 l0 p 1 z(r)e 2πi(nq+m) r kp+l 2πi(nq+m) z(kp + l)e pq e 2πin l p e 2πim l pq e 2πin l p e 2πim l pq e 2πin l p e 2πim l pq e 2πin l p vm (l) l0 ^v m (n). q 1 z(kp + l)e 2πim k q q 1 ω l (k) 2πim k q ^ωl (m) Consider the number of multiplications required for the algorithm in Lemma We first compute the vectors ^ω l, for l 0, 1,..., p 1. Each of these is a vector of length q, so computing each ^ω l requires q complex multiplications. So this step requires a total of p q complex multiplications. The next step is to multiply each ^ω l (b) by e 2πil b to obtain the vectors v b (l). This requires a total of pq complex multiplications, one for each of the q values of b and p values of l. Finally we compute the vectors ^v b for b 0, 1,..., q 1. Each v b is a vector of length p, so each of the q vectors ^v b requires p complex multiplications, for a total of q p multiplications. Adding up, we have an estimate for the number of multiplications required to compute a DFT of size pq, namely pq p q + q p + pq. (1.3.7) In case p n, p is a prime number then we can obtain the following result Lemma 1.35 Suppose p is a prime number and p n for some positive integer n. Then PROOF We will prove by induction, for n 1 we have p n np n+2 p 2 log p. (1.3.8) p p 2. 24

30 1.3. The Fast Fourier Transform then it holds for n 1. Assume that (1.3.8) holds for n k 1. Then p k kp k+2. (1.3.9) For n k + 1, using the equation (1.3.7) with q p n, and by (1.3.9) we have p k+1 p p k + p k p + p k+1 pkp k+2 + p k+2 + p k+1 kp k+3 + p k+3 (k + 1)p k+3. Therefore, (1.3.8) holds for n k + 1. Thus, for all n. p n np n+2 p 2 log p. ow, we consider the case 2 n. Any m {0, 1,..., 1} can be expanded in bases Z m m 0 + 2m m n 1 m n 1 (1.3.10) where m 0, m 1,..., m n 1 {0, 1}. For z l 2 (Z ), z(m) z(m n 1, m n 2,..., m 0 ). Then ^z(k) m0 1 z(m)e 2πim k 1 m 0 0 m m n 1 0 z(m n 1, m n 2,..., m 0 ) exp( 2πi(k 0 + 2k n 1 k n 1 )(m 0 + 2m n 1 m n 1 ) ) 2 n z(m n 1, m n 2,..., m 0 ) m 0 0 m 1 0 m n 1 0 exp( 2πi(k 0 + 2k n 1 k n 1 )2 n 1 m n 1 2 n ) exp( 2πi(k 0 + 2k n 1 k n 1 )2 n 2 m n 2 2 n )... exp( 2πi(k 0 + 2k n 1 k n 1 )m 0 2 n ) 25

31 1.3. The Fast Fourier Transform 1 1 m 0 0 m m n 1 0 exp( 2πi(k 02 n 1 m n 1 2 n )) z(m n 1, m n 2,..., m 0 ) exp( 2πi(k 0 + 2k 1 )2 n 2 m n 2 2 n )... exp( 2πi(k 0 + 2k n 1 k n 1 )m 0 2 n ). We see that the inside sum depends on the outside summation variables m 0, m 1,..., m n 2 and on k 0 but not on k 1,..., k n 1. So define w 1 (k 0, m n 2,..., m 0 ) At the next step, we define w 2 (k 0, k 1, m n 3,..., m 0 ) 1 m n 1 0 z(m n 1, m n 2,..., m 0 ) exp( 2πi(k 02 n 1 m n 1 ) 2 n ) z(0, m n 2,..., m 0 ) + z(1, m n 2,..., m 0 ) exp( 2πi(k 02 n 1 ) 2 n ). 1 m n 2 0 w 1 (k 0, m n 2,..., m 0 ) exp( 2πi(k 0 + 2k 1 )2 n 2 m n 2 2 n ). We continue in this way, each time replacing the highest remaining m index by the next k index. Thus the scheme is to make the sequence of transformations z(m n 1, m n 2,..., m 0 ) w 1 (k 0, m n 2,..., m 0 ) w 1 (k 0, m n 2,..., m 0 ) w 2 (k 0, k 1,..., m 0 )... w n 1 (k 0, k 1,..., k n 2, m 0 ) w n (k 0, k 1,..., k n 1 ). The final vector w n (k 0, k 1,..., k n 1 ) is exactly ^z(k). For l 1, 2,..., n, computing each w l (k 0,..., k l 1, m nl 1,..., m 0 ) requires only one complex multiplication for each of the 2 n choices of k 0,..., k l 1, m n l 1,..., m 0 {0, 1}, hence 2 n total to compute all possible values of w l. So each step requires at most 2 n complex multiplications, and there are a total of n steps, for at most n2 n log 2 complex multiplications. More exactly, the total of multiplications is just 2 log 2. In- 26

32 1.3. The Fast Fourier Transform deed, we have: w j+1 (k 0, k 1,..., k j,m n j 2,..., m 0 ) 1 m n j 1 0 w j (k 0,..., k j 1, m n j 1,..., m 0 ) exp( 2πi(k j 1 k j j k j )2 n j 1 m n j 1 ) 2 n 1 e 2πi2 j kj 2 n j 1 m n j 1 2 n w j (k 0,..., k j 1, m n j 1,..., m 0 ) m n j 1 0 exp( 2πi(k j 1 k j 1 )2 n j 1 m n j 1 2 n ), where e 2πi2 j kj 2 n j 1 m n j 1 2 n is always either +1 or 1. Thus the two choices of k j are done via the same multiplication. Therefore, there are 2 n 1 multiplications for each step. Consequently, after n steps, this requires n2 n 1 2 log 2 complex multiplications. Example 1.36 Let z (1, 1, 1, i, 1, 1, 1, i). By the FFT, find ^z? Since then we can write {0, 1, 2,..., 7} in the binary system with 3 bits: 0 (0, 0, 0), 1 (0, 0, 1), 2 (0, 1, 0), 3 (0, 1, 1), 4 (1, 0, 0), 5 (1, 0, 1), 6 (1, 1, 0), 7 (1, 1, 1). Then we can compute ^z as the following. ote that ^z(k) w n (k 0, k 1,..., k n 1 ) w n (2 n 1 k n 2 k k n 2 + k n 1 ). For m 0, m 1, m 2 {0, 1}, to compute ^z(0) w 3 (0, 0, 0), ^z(4) w 3 (0, 0, 1), ^z(2) w 3 (0, 1, 0), ^z(6) w 3 (0, 1, 1), at the first step: w 1 (0, 0, 0) z(0, 0, 0) + z(1, 0, 0) z(0) + z(4) 2, w 1 (0, 0, 1) z(0, 0, 1) + z(1, 0, 1) z(1) + z(5) 0, w 1 (0, 1, 0) z(0, 1, 0) + z(1, 1, 0) z(2) + z(6) 2, w 1 (0, 1, 1) z(0, 1, 1) + z(1, 1, 1) z(3) + z(7) 0. 27

33 1.3. The Fast Fourier Transform At the second step: w 2 (0, 0, 0) w 1 (0, 0, 0) + w 1 (0, 1, 0) 4, w 2 (0, 0, 1) w 1 (0, 0, 1) + w 1 (0, 1, 1) 0, w 2 (0, 1, 0) w 1 (0, 0, 0) + w 1 (0, 1, 0) exp( πi) 0, w 2 (0, 1, 1) w 1 (0, 0, 1) + w 1 (0, 1, 1) exp( πi) 0. At the last step: w 3 (0, 0, 0) w 2 (0, 0, 0) + w 2 (0, 0, 1) 4, w 3 (0, 0, 1) w 2 (0, 0, 0) + w 2 (0, 0, 1) exp( πi) 4, w 3 (0, 1, 0) w 2 (0, 1, 0) + w 2 (0, 1, 1) exp( πi 2 ) 0, w 3 (0, 1, 1) w 2 (0, 1, 0) + w 2 (0, 1, 1) exp( 3πi 2 ) 0. Therefore, we get In order to compute ^z(0) w 3 (0, 0, 0) 4, ^z(2) w 3 (0, 1, 0) 0, ^z(4) w 3 (0, 0, 1) 4, ^z(6) w 3 (0, 1, 1) 0. ^z(1) w 3 (1, 0, 0), ^z(3) w 3 (1, 1, 0), ^z(5) w 3 (1, 0, 1), ^z(7) w 3 (1, 1, 1), at the first step: w 1 (1, 0, 0) z(0, 0, 0) + z(1, 0, 0) exp( πi) z(0) z(4) 0, w 1 (1, 0, 1) z(0, 0, 1) + z(1, 0, 1) exp( πi) z(1) z(5) 2, w 1 (1, 1, 0) z(0, 1, 0) + z(1, 1, 0) exp( πi) z(2) z(6) 0, w 1 (1, 1, 1) z(0, 1, 1) + z(1, 1, 1) exp(πi) z(3) z(7) 2i. At the second step: w 2 (1, 0, 0) w 1 (1, 0, 0) + w 1 (1, 1, 0) exp( πi 2 ) 0, w 2 (1, 0, 1) w 1 (1, 0, 1) + w 1 (1, 1, 1) exp( πi 2 ) 4, w 2 (1, 1, 0) w 1 (1, 0, 0) + w 1 (1, 1, 0) exp( 3πi 2 ) 0, w 2 (1, 1, 1) w 1 (1, 0, 1) + w 1 (1, 1, 1) exp( 3πi 2 ) 0. 28

34 1.3. The Fast Fourier Transform At the last step: w 3 (1, 0, 0) w 2 (1, 0, 0) + w 2 (1, 0, 1) exp( πi 4 ) 2 2 2i 2, w 3 (1, 0, 1) w 2 (1, 0, 0) + w 2 (1, 0, 1) exp( 5πi 4 ) i 2, w 3 (1, 1, 0) w 2 (1, 1, 0) + w 2 (1, 1, 1) exp( 3πi 4 ) 0, w 3 (1, 1, 1) w 2 (1, 1, 0) + w 2 (1, 1, 1) exp( 7πi 4 ) 0. Therefore, we get ^z(1) w 3 (1, 0, 0) 2 2 2i 2, ^z(3) w 3 (1, 1, 0) 0, ^z(5) w 3 (1, 0, 1) i 2, ^z(7) w 3 (1, 1, 1) 0. In order to see the process, we can image as the following picture PICTURE 1 29

35 CHAPTER 2 Wavelets on Z 2.1 Construction of Wavelets on Z : The First Stage We have studied that translation-invariant linear transformations are diagonalized by the Fourier basis, and the coordinates in the Fourier basis can be computed quickly using the FFT. However, for many purposes in signal analysis and other fields, the Fourier basis has serious limitations. One of these come from the fact that the Fourier basis elements are not localized in space. Definition 2.1 A vector z l 2 (Z ) is localized in space near n 0 if most of component z(n) of z are 0 or at least relative small except for a few values of n close to n 0. A Fourier basis element F m is not localized in space because its components F m (n) 1 e2πim n have the same magnitude for every n Z. Suppose B {v 0, v 1,..., v } is a basis for l 2 (Z ) such that all the basis elements of B are localized in space. For z l 2 (Z ), z a n v n. for a 0, a 1,..., a C. Suppose that we wish to focus on the portion of z near some particular point n 0. Terms involving basis vectors that are 0 or negligibly small near n 0 can be deleted without changing the behavior near n 0 significantly. Thus we may be able to replace a full sum over terms by a much smaller sum when considering only the portion of z near n 0.That mean we can focus on this location and analyze it in more detail. Euclidean is one example of a localized basis. However, we would also like to obtain the advantages of the Fourier basis discussed in the first chapter that 30

36 2.1. Construction of Wavelets on Z : The First Stage is the fast computation of translation-invariant linear transformations. For this we would like our basis to be frequency localized. This mean that we would like the DFTs of our basis vectors to be negligibly small except near one particular region. This means that the basis vectors should consist of a very small group of frequencies. ote that a standard basis vector e m is not frequency localized. It may even be that some high-frequencies come from noise added to the signal, so that the signal becomes more clear when these terms are removed. With a frequency localized expansion, we know which terms in our expansion to delete to remove the high frequency components of the signal. If the result is that the signal is satisfactorily represented by a reduced number of data bits, then we have obtained compression. Thus, our ultimate goal is to obtain a basis whose elements are both spatially and frequency localized. Then a vectors expansion coefficients in this basis will provide both spatial and frequency information. Hence,we would obtain a simultaneous space/frequency analysis of this vector. Wavelets will provide such a basis. ow, we consider two vectors u, v such that their translates by even integers form an orthonormal basis. Definition 2.2 Suppose is an even integer, say 2M for some M. An orthonormal basis for l 2 (Z ) of the form {R 2k u} M 1 {R2k v} M 1 (2.1.1) for some u, v l 2 (Z ), is called a first-stage wavelets basis for l 2 (Z ). We call u and v the generators of the first-stage wavelet basis. We sometimes also call u the father wavelet and v the mother wavelet. We will determine when a pair u, v generates a first-stage wavelet basis. Lemma 2.3 Let w l 2 (Z ). Then {R k w} is an orthonormal basis for l2 (Z ) if and only if ^w(n) 1 for all n Z. PROOF By Proposition 1.13, we claim that {R k w} and only if w, R k w { 1 if k 0, 0 if k 1, 2,..., 1. is an orthonormal basis if By Lemma 1.22 we get (w * w)(k) { 1 if k 0, 0 if k 1, 2,..., 1. 31

37 2.1. Construction of Wavelets on Z : The First Stage Therefore, using Lemma 1.20 then However, {R k w} 1 (w * w)^(n) ^w(n)( w)^(n) ^w(n) ^w(n) ^w(n) 2. is not frequency localized because (R k w)^(n) ^w(n) 1, n Z. Lemma 2.4 Suppose M, 2M, and w l 2 (Z ). Then {R 2k w} M 1 orthonormal set with M elements if and only if is an ^w(n) 2 + ^w(n + M) 2 2 for n 0, 1,..., M 1. (2.1.2) PROOF We have {R 2k w} M 1 is an orthonormal basis if and only if { 1 if k 0, w * w(2k) w, R 2k w 0 if k 1, 2,..., M 1. Therefore, by (1.1.41) (w * w + (w * w) * )(n) { 2 if n 0, 0 if n 1, 2,..., 1. (2.1.3) Consequently, applying Lemma 1.24 we take its DFT 2 (w * w)^(n) + ((w * w) * )^(n) (w * w)^(n) + (w * w)^(n + M) ^w(n) 2 + ^w(n + M) 2. Definition 2.5 Suppose M, 2M, and u, v l 2 (Z ). For n Z, define Λ(n), the system matrix of u and v, by ( Λ(n) 1 ^u(n) 2 ^u(n + M) ^v(n) ^v(n + M) ). (2.1.4) ow we can characterize orthonormal bases generated by the even integer translates of two vectors. Theorem 2.6 Suppose M and 2M. Let u, v l 2 (Z ). Then B {R 2k v} M 1 {R2k u} M 1 {v, R 2 v, R 4 v,..., R 2 v, u, R 2 u, R 4 u,..., R 2 u} 32

38 2.1. Construction of Wavelets on Z : The First Stage is an orthonormal basis for l 2 (Z ) if and only if the system matrix Λ(n) of u and v is unitary for each n 0, 1,..., M 1. Equivalently, B is a first-stage wavelet basis for l 2 (Z ) if and only if ^u(n) 2 + ^u(n + M) 2 2, (2.1.5) ^v(n) 2 + ^v(n + M) 2 2, (2.1.6) and ^u(n)^v(n) + ^u(n + M)^v(n + M) 0 (2.1.7) for all n 0, 1,..., M 1. Moreover, ^u(n) ^v(n + M) and ^u(n + M) ^v(n). (2.1.8) PROOF By the definition, the matrix A is unitary if A is invertible and A 1 A *, where A * is the conjugate transpose of A. Therefore, Λ(n) is unitary if and only if ( ) Λ(n) Λ * (n) Λ * 1 0 (n) Λ(n). 0 1 where Λ * (n) ( ^u(n) ^v(n) ) ^u(n + M). ^v(n + M) We have ( Λ(n) Λ * (n) 1 ^u(n) 2 ^u(n + M) ( ) 1 0, 0 1 ^v(n) ^v(n + M) ) ( ^u(n) ^v(n) ) ^u(n + M) ^v(n + M) and ( Λ * (n) Λ(n) 1 ^u(n) 2 ^v(n) ( ) ) ( ^u(n + M) ^u(n) ^v(n + M) ^u(n + M) ^v(n) ^v(n + M) ) Therefore, ^u(n) 2 + ^u(n + M) 2 ^u(n) 2 + ^v(n) 2 2, ^v(n) 2 + ^v(n + M) 2 ^u(n + M) 2 + ^v(n + M) 2 2, ^u(n)^v(n) + ^u(n + M)^v(n + M) 0. 33

39 2.1. Construction of Wavelets on Z : The First Stage Equivalently, ^u(n) ^v(n + M), ^u(n + M) ^v(m), ^u(n) 2 + ^u(n + M) 2 2, ^u(n)^v(n) + ^u(n + M)^v(n + M) 0. In other words, Λ(n) is a unitary matrix if and only if ( ) ( ) 1 ^u(n) 1 ^v(n), 2 ^u(n + M) 2 ^v(n + M) forms an orthonormal basis for l 2 (Z 2 ). Then applying Lemma 2.4, we obtain: 1. {R 2k u} M 1 is orthonormal if and only if ^u(n) 2 + ^u(n + M) 2 2 for n 0, 1,..., M 1. (2.1.9) 2. {R 2k v} M 1 is orthonormal if and only if ^v(n) 2 + ^v(n + M) 2 2 for n 0, 1,..., M 1. (2.1.10) For j, k 0, 1,..., M 1, we consider R 2k u, R 2j v R 2k u(n)r 2j v(n) u(n 2k)v(n 2j) u(m)v(m + 2(j k)) u, R 2(k j)v. m0 Thus, for j, k 0, 1,..., M 1.,using equation (1.1.36) R 2k u are orthogonal to R 2j v if and only if for k 0, 1,..., M 1. It s equivalent to u * v(2k) u, R 2k v 0, u * v + (u * v) * 0 because all odd indexes are automatically zero. Take the DFT of it, we obtain the result. This Theorem can be generalized for lq. 34

40 2.1. Construction of Wavelets on Z : The First Stage Theorem 2.7 Suppose l and l (i.e., there exists q Z such that ql).let u 0, u 1,..., u l 1 l 2 (Z ). Prove that {R lk u 0 } l 1 {Rlk u 1 } l 1... {Rlk u l 1 } l 1 is an orthonormal basis for l 2 (Z ) if and only if the matrix ^u 0 (n) ^u 1 (n) ^u l 1 (n) ^u 0 (n + 1 ) ^u l 1(n + ) ^u l l 1(n + ) l l ^u 0 (n + 2 ) ^u l 1(n + 2 ) ^u l l 1(n + 2 ) l ^u 0 (n + (l 1) ) ^u l 1 (n + (l 1) ) ^u l l 1 (n + (l 1) ) l is unitary for all n 0, 1,..., l. PROOF We see that {R lk u 0 } l 1 {Rlk u 1 } l 1... {Rlk u l 1 } l 1 is an orthonormal basis for l 2 (Z ) if and only if the two following hold: 1. {R lk z} l 1 is an orthonormal set with l elements. 2. R lk u m are orthogonal to R lj u n for all m, n 0, 1,..., l 1, and j, k 0, 1,..., l 1. Suppose z, w, v l 2 (Z ), then {R lk z} l 1 is an orthonormal set with l elements if and only if and if and only if l 1 ^z(n + k l ) 2 l for all n. (2.1.11) R lk v, R lj w 0 for all j, k 0, 1,..., l 1. l 1 ^v(n + k l ) ^w(n + k ) 0 for all n. (2.1.12) l Indeed, using Parseval s relation we have v, R lk w 1 ^v, (R lkw)^ 1 1 l 1 j0 l 1 1 l 1 (/l) m0 ^v(m)(r lk )^(m) 1 m0 e 2πil(m+j l ) k ^v(n + j l ) ^w(n + j l ) e 2πik n 1 l 1 (/l) ^v(n + j l l ) ^w(n + j l ). j0 ^v(m)e 2πilk m ^w(m) 35

41 2.1. Construction of Wavelets on Z : The First Stage Let ^ω(n) 1 l 1 ^v(n + j l l ) ^w(n + j l ), j0 then v, R lk w ω(k). Therefore, {R lk z} l 1 is an orthonormal set with elements if and only if l { 1 for k 0, ω(k) z, R lk z 0 for all k 1, 2,..., 1. l It implies that Similarly, 1 ^ω(n) 1 l l 1 j0 ^z(n + j l )^z(n + j l ) 1 l 1 ^z(n + j l l ) 2. j0 R lk v, R lj w 0 for all j, k if and only if ω(k) 0 for all k 0, 1,..., l 1. It s equivalent to 0 ^ω(n) 1 l 1 ^v(n + j l l ) ^w(n + j l ). j0 Since 2 2 unitary matrices are easy to characterize then Theorem 2.6 can be used to describe all first-stage wavelet bases explicitly. Corollary 2.8 Suppose M and 2M. Let u, v l 2 (Z ) such that {R 2k v} M 1 {R2k u} M 1 is a first-stage wavelet basis for l 2 (Z ). Then ^u, ^v always are of the form stated as the following: and ^u(n) r(n)e iθ(n), ^u(n + M) 2 r 2 (n)e iφ(n) (2.1.13) ^v(n) 2 r 2 (n)e iσ(n), ^v(n + M) r(n)e iρ(n), (2.1.14) satisfying θ(n) + ρ(n) φ(n) σ(n) (2k(n) + 1)π, k(n) Z where {θ(n)} M 1, {ρ(n)} M 1, {φ(n)} M 1, {σ(n)} M 1, {r(n)} M 1, be real numbers such that 0 < r(n) < 2 for all n 0, 1,..., M 1. If r(n) 0 or r(n) 2, then θ(n), φ(n), ρ(n) and σ(n) are unconstrained. 36

42 2.1. Construction of Wavelets on Z : The First Stage PROOF Applying Theorem 2.6, then {R 2k v} M 1 {R2k u} M 1 basis for l 2 (Z ) if and only if ow, we parameterize then ^u(n) ^v(n + M), ^u(n + M) ^v(m), And the orthogonality implies that ^u(n) 2 + ^u(n + M) 2 2, ^u(n)^v(n) + ^u(n + M)^v(n + M) 0. ^u(n) r(n)e iθ(n), ^v(n) 2 r 2 (n)e iσ(n), ^u(n + M) 2 r 2 (n)e iφ(n), ^v(n + M) r(n)e iρ(n). is a first-wavelets 0 ^u(n)^v(n) + ^u(n + M)^v(n + M) r(n) 2 r 2 (n)(e i(θ(n) σ(n)) + e i(φ(n) ρ(n)) ). It is equivalent to θ(n) + ρ(n) φ(n) σ(n) (2k(n) + 1)π for all n 0, 1,..., M 1. ow we will state the following result very useful. It says that every potential father wavelet u has a companion mother wavelet v such that u and v generate a first-stage wavelet basis. Lemma 2.9 Suppose M, 2M, and u l 2 (Z ) is such that {R 2k u} M 1 an orthonormal set with M elements. Define v l 2 (Z ) by is v(k) ( 1) k 1 u(1 k) (2.1.15) for all k. Then {R 2k v} M 1 {R2k u} M 1 is a first-stage wavelets basis for l2 (Z ). PROOF We have ^v(n) v(k)e 2πin k ( 1) l u(l)e l0 e 2πi n l0 ( 1) k 1 u(1 k)e 2πin k 1 l 2πin u(l)e 2πi(n+M) l e 2πi n (e πi ) l u(l)e 2πin l 37 l0 e 2πi n ^u(n + M).

43 2.1. Construction of Wavelets on Z : The First Stage Therefore, Hence, n+m 2πi ^v(n + M) e ^u(n + 2M) e 2πi n ^u(n). ^v(n) 2 + ^v(n + M) 2 ^u(n + M) 2 + ^v(n + M) 2 2 for n 0, 1,..., M 1. Moreover, n+m 2πi ^u(n)^v(n) + ^u(n + M)^v(n + M) ^u(n)^u(n + M)e 0. ^u(n)^u(n + M)e 2πi n+m By Theorem 2.6, the pair u, v generates a first-stage wavelet basis for l 2 (Z ). Proposition 2.10 Suppose M, 2M, and u l 2 (Z ) is such that {R 2k u} M 1 is an orthonormal set with M elements. Define v l (k) ( 1) k l u(l k). (2.1.16) for all l odd numbers in [0, 1]. Then {R 2k v l } M 1 {R2k u} M 1 is a first-stage wavelets basis for l 2 (Z ). More precisely, {v 2l+1 } M 1 l0 forms an orthonormal basis for the space of dimension M, which include all vectors v such that {R 2k v} M 1 {R2k u} M 1 form a first-stage wavelets basis for l 2 (Z ). So that, {^v 2l+1 } M 1 l0 is also form an orthogonal set with M elements. PROOF We have R 2k v 1 (n) v 1 (n 2k) ( 1) n 2k 1 u(1 + 2k n) v 2k+1 (n). We can compute the DFT of v 2l+1 similar as of v 1. We obtain ^v 2l+1 (m) e 2πi(2l+1) m ^u(m + M), ^v 2l+1 (m) e 2πi(2l+1) m ^u(m). Since {v 2l+1 } M 1 l0 is orthonormal then {^v 2l+1 } M 1 l0 is also form an orthogonal set by Parseval s relation. Corollary 2.8 say that any first-stage wavelets basis for l 2 (Z ), ^u, ^v always are of the form stated as (2.1.13), (2.1.14). Therefore, if we take ^v as (2.1.14) then ^v can be written as a linearly independent combination of ^v 1, ^v 3,..., ^v,i.e., ^v M 1 α k^v 2k+1 where α k ^v, ^v 2k+1 ^v 2k

44 2.1. Construction of Wavelets on Z : The First Stage Then α k 1 ^v(n)^v 2k+1 (n) 1 M 1 [e i(σ(n)+φ(n))^u(n + M)e 2πi(2k+1) n ^u(n + M) e i(ρ(n)+θ(n))^u(n)e 2πi(2k+1) n ^u(n)] 1 M 1 e i(σ(n)+φ(n)) 2πi(2k+1) n ( ^u(n + M) 2 + ^u(n) 2 ) 2 M 1 e i(σ(n)+φ(n)) 2πi(2k+1) n. We have M 1 ^v 2 α k^v 2k+1 2 We have known that {R 2k v} M 1 M 1 α k 2 ^v 2k+1 2. is orthonormal if and only if ^v(n) 2 + ^v(n + M) 2 2, n. It is equivalent to ^v 2 2M. Therefore, M 1 α k 2 1. Hence, for 2M, from a potential father wavelet u we can generate v such that u, v generate a first-stage wavelets basis for l 2 (Z ). ow we try to do the same with 3M, that is how can generate u 1, u 2 from a potential wavelet u 0 such that {R 3k u 0 } M 1 {R3k u 1 } M 1 {R3k u 2 } M 1 is a firststage wavelets basis for l 2 (Z ). Proposition 2.11 Suppose 3M. Let u 0 l 2 (Z ) such that ^u 0 (n) r 1, ^u 0 (n + M) r 2 e πi 3, ^u 0 (n + 2M) r 3 e πi 3, where r 1, r 2, r 3 are positive numbers satisfying { r r r 2 3 3, 1 r 1 1 r r 3 39

45 2.1. Construction of Wavelets on Z : The First Stage Define and Then {R 3k u 0 } M 1 k 1 2πi u 1 (k) e 3 u0 (1 k), (2.1.17) k 1 2πi u 2 (k) e 3 u0 (1 k). (2.1.18) {R3k u 1 } M 1 {R3k u 2 } M 1 is an orthonormal basis. is an or- PROOF Applying Theorem 2.7, {R 3k u 0 } M 1 thonormal basis for l 2 (Z ) if and only if {R3k u 1 } M 1 {R3k u 2 } M 1 Λ 3 (n) 1 ^u 0 (n) ^u 1 (n) ^u 2 (n) ^u 0 (n + ) ^u 3 3 1(n + ) ^u 3 2(n + ) 3 ^u 0 (n + 2 ) ^u 3 1(n + 2 ) ^u 3 2(n + 2 ) 3 is unitary. That means the column vectors of Λ 3 (n) form an orthonormal basis for l 2 (Z 3 ). We compute the DFT of u 1 and u 2 : Therefore, Similarly, we obtain ^u 1 (n) m0 m 1 2πi e 3 u 0 (1 m)e 2πim n e 2πi l 3 u0 (l)e 2πi(1 l) n l0 e 2πi n e 2πi n l0 l0 u 0 (l)e 2πi(M n) l u 0 (l)e 2πi(n M) l e 2πi n ^u0 (n M) e 2πi n ^u0 (n + 2M). ^u 1 (n) e 2πi n ^u0 (n + 2M), ^u 1 (n + M) e 2πi n e 2πi 3 ^u 0 (n), ^u 1 (n + 2M) e 2πi n e 4πi 3 ^u0 (n + M). ^u 2 (n) e 2πi n ^u0 (n + M), ^u 2 (n + M) e 2πi n e 2πi 3 ^u 0 (n + 2M), ^u 2 (n + 2M) e 2πi n e 4πi 3 ^u0 (n). 40

46 2.1. Construction of Wavelets on Z : The First Stage Parameterizing by writing ^u 0 in the polar coordinates. Set ^u 0 (n) r 1 e iα, ^u 0 (n + M) r 2 e iβ, ^u 0 (n + 2M) r 3 e iγ. Therefore, the matrix Λ 3 (n) is unitary if and only if r1 2 + r2 2 + r3 2 3, r 1 r 3 e i(α+γ) + r 1 r 2 e i(α+β+ 2π 3 ) + r 2 r 3 e i(β+γ 2π 3 ) 0, r 2 r 3 e i(β γ) + r 1 r 3 e i(γ α) + r 1 r 2 e i(α β) 0. (2.1.19) We can chose α, β, γ such that α + β + 2π 3 β + γ 2π 3 α + γ γ α, α β, β γ. Then, Therefore, r 1, r 2, r 3 satisfying { α 0, β π 3, γ π 3. r r r 2 3 3, 1 r 1 1 r r 3. This system equations has infinitely many solutions (r 1, r 2, r 3 ), for example: { r 1 1 3, r 2 r or r , r , r Consequently, we have considered some cases in which we can construct a wavelets first-stage for l 2 (Z ) from a potential father wavelets. ow, we consider the case 2M. Suppose B {R 2k v} M 1 {R2k u} M 1 is a 41