Deconvolution: A Wavelet Frame Approach

Transcription

1 Numeische Mathematik manuscipt No. (will be inseted by the edito) Anwei Chai Zuowei Shen Deconvolution: A Wavelet Fame Appoach Received: date / Revised vesion: date Abstact This pape devotes to analyzing deconvolution algoithms based on wavelet fame appoaches, which has aleady appeaed in [6,8,9] as wavelet fame based high esolution image econstuction methods. We fist give a complete fomulation of deconvolution in tems of multiesolution analysis and its appoximation, which completes the fomulation given in [6, 8, 9]. This fomulation convets deconvolution to a poblem of filling the missing coefficients of wavelet fames which satisfy cetain minimization popeties. These missing coefficients ae ecoveed iteatively togethe with a built-in denoising scheme that emoves noise in the data set such that noise in the data will not blow up while iteating. This appoach has aleady been poven to be efficient in solving vaious poblems in high esolution image econstuctions as shown by the simulation esults given in [6,8,9]. Howeve, an analysis of convegence as well as the stability of algoithms and the minimization popeties of solutions wee absent in those papes. This pape is to establish the theoetical foundation of this wavelet fame appoach. In paticula, a poof of convegence, an analysis of the stability of algoithms and a study of the minimization popety of solutions ae given. Keywods deconvolution denoising famelets quasi-affine system unitay extension pinciple Mathematics Subject Classification (2000) 42C40 65T60 68U99 1 Intoduction 1.1 Geneal This pape is to constuct a solution of the convolution equation h 0 v = b+ε := c (1.1) whee h 0 is a low pass filte (i.e. h 0 [k] = 1), b, c and ε ae in l 2 (Z). The sequence ε is the eo tem satisfying ε l2 (Z) ε. Thee ae many eal life poblems which can be modeled by a deconvolution pocess. Fo example, measuement devices and signal communication can intoduce distotions and add noise to oiginal signals. Inveting the degadation is often modeled by a deconvolution pocess, i.e. a pocess of finding a solution in (1.1). In fact, the deconvolution poblem is a citical facto in many applications, especially visual-communication elated applications including emote sensing, militay imaging, suveillance, medical imaging, and high esolution image econstuctions. Solving equation (1.1) is an inveting pocess, which is often numeically unstable and thus amplifies the noise consideably. Hence, an efficient pocess of noise emoval must be built in numeical algoithms. The Anwei Chai Zuowei Shen Depatment of Mathematics, National Univesity of Singapoe, 2 Science Dive 2, Singapoe , Singapoe anwei.chai@alumni.nus.edu.sg matzuows@nus.edu.sg Pesent addess of F. Autho: Institute fo Computational and Mathematical Engineeing, Stanfod Univesity, CA 94305, USA

2 2 Anwei Chai, Zuowei Shen ealie fomulation of the poblem was poposed in [34] using linea algoithm and in [25] and [33] applying the egulaization idea to solve a system of linea equations the coefficient matix of which is ill-conditioned. Since then, thee ae many papes devoted to this method in the liteatue. Because this appoach is not the focus of this pape, instead of a detailed count, we simply efe eades to [24] and [28] and the efeences thee fo a complete efeence. The focus of this pape is to use wavelet (moe geneally, wavelet fame) to solve (1.1). Recently, thee ae seveal papes on solving invese poblems by using wavelet methods, and in paticula, deconvolution poblems. One of the main ideas is to constuct a wavelet o wavelet inspied basis that can almost diagonalize the given opeato and the undelying solution has a spase expansion with espect to the chosen basis. The Wavelet-Vaguelette decomposition poposed in [20], [22] and [23] and the deconvolution in mio wavelet bases in [28] and [29] can be both viewed as examples of succuss of this stategy. Anothe appoach is to apply Galekin-type methods to invese poblems using an appopiate, but fixed wavelet basis (see e.g. [1] and [16]). The undelying intuition is that if the given opeato has a close to spase epesentation in wavelets and the solution has a spase expansion with espect to the wavelet basis, then the invesion is educed appoximately to the invesion of a tuncated opeato. The method is adaptive, so that the fine-scale wavelets ae used whee lowe-scales indicate the pesence of singulaities. A few new iteative thesholding algoithms diffeent fom the above appoaches and developed simultaneously and independently ae poposed in [6, 8, 9,18,20]. It only equies that the undelying solution has a spase expansion with espect to a given system without any attempt to almost diagonalize o spasely epesenting the convolution opeatos. The main idea of [18,20] is to expand each iteation with espect to the chosen othonomal basis fo a given algoithm such as the Landwebe method, then a thesholding algoithm is applied to the coefficients of this expansion. The esult is then used to deive the next iteation. The algoithm is shown to convege to the minimize of cetain cost functional. In the studies of high esolution image econstuctions, the wavelet based (in fact the fame-based) econstuction algoithms ae developed in [5 7], and late [8, 9] though the pefect econstuction fomula of a bi-fame o tight fame system which has h 0 as its pimay low pass filte. The algoithms appoximate iteatively the coefficients of wavelet fame folded by the given low pass filte. By this appoach, many available techniques developed in the wavelet liteatues, such as wavelet-based denoising schemes, can be built in the iteation. When thee ae no displacement eos, the high esolution image econstuction is exactly a deconvolution poblem. Hee, we extend the algoithms in the papes mentioned above to solve equation (1.1). Algoithm 4.3 is used in the papes mentioned above, in paticula in [6, 8]. This method has been extended to algoithms fo high esolution image econstuctions with displacement eos in [8] and [9]. Algoithm 4.1 is given in [9] as one of the options which is motivated by the appoaches taken by [18, 20]. Algoithms given in [10,11] ae based on Algoithm 4.2 whee high esolution images ae constucted fom a seies of video clips. The main ideas of all thee algoithms ae the same. i.e. an iteative pocess combined with a denoising scheme applied to each iteation. The diffeences lay in the diffeent denoising schemes applied to diffeent algoithms which in tun minimize diffeent cost functionals. Finally, we emak that conveting a debluing poblem of ecoveing v fom c to a poblem of inpainting of lost wavelet coefficients can be also found in [12] with a diffeent appoach. The inteested eade should consult [12] fo details. The convegence analysis of Algoithm 2.1 (the iteation without built-in denoising scheme) has aleady been established in [6] and [9]. Howeve, the convegence of Algoithm 2.2, 2.3, 4.1, 4.2, 4.3 has not been discussed so fa. The cuent pape aims to build up a complete theoy fo these algoithms. We will fist give a solid and complete fomulation of econstuctions of a solution to equation (1.1) in tems of multiesolution analysis and its associated fame system. Then the convegence of all algoithms will be given. A complete analysis of minimization popeties, i.e. in which sense the solution deived fom the algoithms attains its optimal popety, will be given. Finally, the stability of the algoithms is also discussed, which shows that numeical solution appoaches to the exact solution when the noise level deceases to zeo. As it has aleady been shown in the papes [6, 8 11], algoithms ae numeically efficient, easy to implement and adaptive to diffeent applications such as high esolution image econstuctions with displacement eos (see e.g. [8] and [9]). In this pape, a theoetical foundation of the undelying algoithms used in those papes is fully laid out. The pape is oganized as follows: Section 2 is devoted to giving a fomulation of the deconvolution poblem in tems of multiesolution analysis and its associated tight wavelet fame. Algoithms ae also deived fom this fomulation. Section 3 gives a complete analysis of the algoithms, including the convegence and minimization popeties of the algoithms. Section 4 focuses on the finite dimensional data set, i.e. the data set has only finitely many enties. Algoithms 2.2 and 2.3 fo infinite dimensional data set can be conveted by imposing pope bounday conditions. Since any numeical solution of deconvolution ultimately deals with

3 Deconvolution: A Wavelet Fame Appoach 3 finite dimensional data sets, such convesion is necessay. As we will see, in many cases, the discussion will be simple and we ae able to obtain bette esults. Since the numeical implementations and simulations, togethe with compaison of algoithms povided hee with othe algoithms, e.g. egulaization method, ae discussed in details in [6,8,9], and since ou focus hee is to lay the foundation of the algoithms, we omit the detailed discussions of numeical implementations hee. Instead, we give a numeical compaison between Algoithms 4.1, 4.2 and 4.3 in Section 4. Finally, Seveal poofs ae left ove to the appendices. In the emaining pat of this section, we give the notation and collect basic esults of tight fame system that will be used in this pape. Reades whose main inteests ae the majo pat of this pape may only need to biefly go though this pat to get familia with the notations. 1.2 Tight Wavelet Fames We give hee a bief intoduction of the tight wavelet fame and its quasi-affine countepat. The decompositions and econstuctions fo the affine tight fame system ae known (e.g. [19]); howeve, the analysis of decomposition and econstuction of quasi-affine systems is not systematically given. Since these esults ae cucial fo ou analysis, we intoduce them hee in details and leave the poofs to the appendix. At the same time, we set the notations used in this pape. The space L p (R) is the set of all the functions f(x) satisfying { ( f Lp (R) := R f(x) p dx) 1 p <, 1 p < ; esssup x R f(x) <, p = ; and l p (Z) is the set of all sequences defined on Z which satisfy that { h lp (Z) := ( h[k] p ) 1 p <, 1 p < ; sup h[k] <, p =. The Fouie tansfom of a function f L 1 (R) is defined as usual by: f(ω) := f(x)e iωx dx, ω R, R and its invese is f(x) = 1 f(ω)e iωx dω, x R. 2π R They can be extended to moe geneal functions, i.e. the functions in L 2 (R). Similaly, we can define the Fouie seies fo a sequence h l 2 (Z) by ĥ(ω) := h[k]e ikω, ω R. Fo any function f L 2 (R), the dyadic dilation opeato D is defined by D f(x) := 2 f(2x) and the tanslation opeato T is defined by T a f(x) := f(x a) fo a R. Given j Z, we have T a D j = D j T 2 j a. Futhe, a space V is said to be intege-shift invaiant if given any function f V, T j f V fo j Z. A system X L 2 (R) is called a tight fame of L 2 (R) if f 2 L 2 (R) = f,g 2, g X holds fo all f L 2 (R), whee, is the inne poduct in L 2 (R) and L2 (R) =,. This is equivalent to f = f,g g, f L 2 (R). g X It is clea that an othonomal basis is a tight fame. Fo given Ψ := {ψ 1,...,ψ } L 2 (R), define the affine system X(Ψ) := {ψ l, j,k : 1 l ; j,k Z},

4 4 Anwei Chai, Zuowei Shen whee ψ l, j,k = D j T k ψ l = 2 j/2 ψ l (2 j k). When X(Ψ) foms an othonomal basis of L 2 (R), then ψ l, l = 1,...,, ae called the othonomal wavelets. When X(Ψ) foms a tight fame of L 2 (R), then it is called a tight wavelet fame and ψ l, l = 1,...,, ae called the tight famelets. The tight famelets can be constucted by the unitay extension pinciple (UEP) given in [31], which uses the multiesolution analysis (MRA). The MRA stats fom a efinable function φ. A compactly suppoted function φ is efinable if it satisfies a efinement equation φ(x) = 2 h 0 [k]φ(2x k), (1.2) fo some sequence h 0 l 2 (Z). By the Fouie tansfom, the efinable equation (1.2) can be given as φ(ω) = ĥ 0 (ω/2) φ(ω/2), a.e. ω R. We call the sequence h 0 the efinement mask of φ and ĥ 0 (ω) the efinement symbol of φ. Fo given finitely suppoted h 0 with ĥ 0 (0) = 1, the efinement equation (1.2) always has distibution solution which can be witten in the Fouie domain as φ(ω) = j=1 ĥ 0 (2 j ω), a.e. ω R. In this pape, we equie h 0 being finitely suppoted. Then the coesponding efinable function φ satisfies that esssup ω R φ(ω + 2kπ) 2 <, (1.3) wheneve φ L 2 (R) (see [27]). Fo a compactly suppoted efinable function φ L 2 (R), let V 0 be the closed shift invaiant space geneated by {φ( k) : k Z} and V j := { f(2 j ) : f V 0 }, j Z. It is known that when φ is compactly suppoted, then {V j } j Z foms a multiesolution analysis. Recall that a multiesolution analysis is a family of closed subspaces {V j } j Z of L 2 (R) that satisfies: (i) V j V j+1, (ii) j V j is dense in L 2 (R), and (iii) j V j = {0} (see [2] and [26]). Fo given MRA of nested spaces V j, j Z with the undelying efinable function φ and the efinement mask h 0, it is well known that (e.g. see [2]) fo any ψ V 1, thee exists a 2π peiodic function ϑ, such that ψ(2 ) = ϑ φ. Let Ψ := {ψ 1,...,ψ } V 1, then ψ l (2 ) = ĥ l φ, l = 1,...,, (1.4) whee ĥ 1,...,ĥ ae 2π peiodic functions and ae called famelet symbols. In the time domain, (1.4) can be witten as ψ l (x) = 2 h l [k]φ(2x k). (1.5) We call h 1,...,h famelet masks. We also call the efinement mask h 0 the low pass filte and h 1,...,h the high pass filtes of the system. The UEP says when Ψ becomes a set of tight famelets with X(Ψ) being a tight fame of L 2 (R). Theoem 1.1 (Unitay Extension Pinciple, [31]) Let φ L 2 (R) be the efinable function with efinement mask h 0 satisfying ĥ 0 (0) = 1 that geneates an MRA {V j } j Z. Let (h 1,...,h ) be a set of sequences with (ĥ1,...,ĥ ) being a set of 2π-peiodic measuable functions in L [0,2π]. If the equalities l=0 ĥ l (ω) 2 = 1 and l=0 ĥ l (ω)ĥ l (ω + π) = 0 (1.6) hold fo almost all ω [ π,π], then the system X(Ψ) whee Ψ = {ψ 1,...,ψ } defined in (1.5) by (h 1,...,h ) and φ foms a tight fame in L 2 (R).

5 Deconvolution: A Wavelet Fame Appoach 5 We will use (1.6) in tems of sequences h 0,...,h. The fist condition ĥ l (ω) 2 = 1 in tems of coesponding sequences is l=0 l=0 h l [k]h l [k p] = δ 0,p, p Z, (1.7) whee δ 0,p = 1 when p = 0 and 0 othewise. The second condition ĥ l (ω)ĥ l (ω + π) = 0 can be witten as l=0 l=0 () k p h l [k]h l [k p] = 0, p Z. (1.8) With the UEP, the constuction of tight famelets become painless. Fo example, one can constuct tight famelets fom spline easily. Next, we give two examples of spline tight famelets. Example 1.1 Let h 0 = [ 1 4, 1 2, 1 4 ] be the efinement mask of the piecewise linea function φ(x)=max(1 x,0). Define h 1 = [ 1 4, 1 2, 4 ] and h 2 = [ 2 4,0, 2 4 ]. Then ĥ 0, ĥ 1 and ĥ 2 satisfy (1.6). Hence, the system X(Ψ) whee Ψ = {ψ 1,ψ 2 } defined in (1.5) by using h 1, h 2 and φ is a tight fame of L 2 (R). This is the fist example constucted via the UEP in [31]. Example 1.2 Let h 0 = [ 16 1, 1 4, 3 8, 1 4, 16 1 ] be the efinement mask of φ. Then φ is the piecewise cubic B-spline. Define h 1, h 2, h 3, h 4 as follows: h 1 = [ 1 16, 4, 3 8, 4, 1 16 ], h 2 = [ 1 8, 1 4,0, 4, 1 8 ], h 3 = [ 6 16,0, 6 8,0, 6 16 ], h 4 = [ 1 8, 4,0, 1 4, 1 8 ]. Then ĥ 0,ĥ 1,ĥ 2,ĥ 3,ĥ 4 satisfy (1.6) and hence the system X(Ψ) whee Ψ = {ψ 1,ψ 2,ψ 3,ψ 4 } defined in (1.5) by h 1,h 2,h 3,h 4 and φ is a tight fame of L 2 (R). This is also fist constucted in [31]. The deconvolution pocessing has to be fomulated by quasi-affine systems that wee fist intoduced in [31]. A quasi-affine system fom level J is defined as Definition 1.1 Let Ψ = {ψ 1,...,ψ } be a set of functions. A quasi-affine system fom level J is defined as whee ψ q l, j,k is defined by X q J (Ψ) = {ψq l, j,k : 1 l ; j,k Z}, ψ q l, j,k := { 2 j J D j T k ψ l, j J; 2 T 2 J k D j ψ l, j < J. The quasi-affine system is obtained by ove sampling the affine system. Moe pecisely, we ove sample the affine system stating fom level J and downwad to a 2 J -shift invaiant system. Hence, the whole quasiaffine system is a 2 J -shift invaiant system. The quasi-affine system fom level 0 was fist intoduced in [31] to convet a non-shift invaiant affine system to a shift invaiant system. Futhe, it was shown in [31, Theoem 5.5] that the affine system X(Ψ) is a tight fame of L 2 (R) if and only if the quasi-affine countepat X q J (Ψ) is a tight fame of L 2 (R). In ou analysis, we use the quasi-intepolatoy opeato. Let {V j }, j Z be a given MRA with undelying efinable function φ and Ψ = {ψ 1,...,ψ } be the set of coesponding tight famelets deived fom the UEP. The quasi-intepolatoy opeato in the affine system X(Ψ) geneated by Ψ is defined, fo f L 2 (R), P j : f f,φ j,k φ j,k. It is clea that P j f V j. As shown in [19, Lemma 2.4], this quasi-intepolatoy opeato is the same as tuncated epesentation Q j : f j < j, f,ψ l, j,k ψ l, j,k.

6 6 Anwei Chai, Zuowei Shen Futhemoe, a standad famelet decomposition given in [19] says that P j+1 f = P j f + f,ψ l, j,k ψ l, j,k and P j f = Q j f. (1.9) When we conside the MRA based quasi-affine system X q J (Ψ) geneated by Ψ, the spaces V j, j < J in the MRA fo the affine system ae eplaced by V q,j j, j < J, fo the quasi-affine system. Note that the space V j is spanned by functions φ j,k, while the space V q,j j is spanned by functions φ q j,k, whee φ q j,k is defined by φ q j,k := { 2 j J D j T k φ, j J; 2 T 2 J k D j φ, j < J. The spaces V q,j j, j < J ae 2 J -shift invaiant. We define the quasi-intepolatoy opeato P q,j j and the tuncated opeato Q q,j j, j Z, fo the quasi-affine system by P q,j j : f f,φ q j,k φ q j,k (1.10) and Q q,j j : f j < j, f,ψ q l, j,k ψq l, j,k. (1.11) P q,j j The quasi-intepolatoy opeato P q,j j = P j when j J and these two opeatos ae diffeent only when j < J. Moeove, since fo an abitay f L 2 (R) and j < J, P q,j j maps f L 2 (R) to V q,j j. Fom the definition of φ q j,k, we can see that f = f,φ q j,k φ q j,k = DJ D J f,2 j J 0 2 T k D j J φ 2 j J 0 2 T k D j J φ = D J P q,0 j J D J f, one only needs to undestand the case J = 0. In this case we simplify ou notation by setting P q j := P q,0 j, Q q j := Qq,0 j (1.12) fo the quasi-intepolatoy opeatos and V q j := V q,0 j, fo j Z. Fom now on, we only give the popeties fo P q j and coesponding spaces V q j and the associated quasi-affine system X q (Ψ) := X q 0 (Ψ). The coesponding esults fo the ove sampling ate of 2 J Z can be obtained similaly. Fo opeato P q j, j Z, we also have the decomposition and econstuction fomula simila to (1.9). Lemma 1.1 Let X(Ψ), whee the famelets Ψ = {ψ 1,...,ψ }, be the affine tight fame system obtained fom h 0 and φ via the UEP and X q (Ψ) be the quasi-affine fame deived fom X(Ψ). Then we have P q j+1 f = Pq j f + f,ψ q l, j,k ψq l, j,k, f L 2(R). (1.13) Moe geneal, it was poven in [19, Lemma 2.4] that the identity P j f = Q j f holds fo all f L 2 (R). Next esult shows that a simila esult also holds fo the quasi-affine systems. Poposition 1.1 Let X(Ψ) with Ψ = {ψ 1,...,ψ } be the affine tight fame system obtained fom h 0 and φ via the UEP and X q (Ψ) be the coesponding quasi-affine fame. Then we have P q j f = Qq j f fo all f L 2(R). We postpone the poof to Appendix A.

7 Deconvolution: A Wavelet Fame Appoach Discete Fom The identity (1.13) essentially gives the decomposition and econstuction of a function in quasi-affine tight fame systems. In the implementation, one needs a completely discete fom of the decomposition and econstuction and we give such fom below. We intoduce the Toeplitz matix to descibe the discete fom of the decomposition and econstuction pocedue. Given a sequence h 0 = {h 0 [k]}, the Toeplitz matix geneated by h 0 is a matix satisfying H 0 = (H 0 [l,k]) = (h 0 [l k]), whee the (l,k)th enty in H 0 is fully detemined by the (l k)th enty in h 0. The Toeplitz matix is also called the convolution matix since it can be viewed as the matix epesentation of linea time invaiant filte which can be witten as convolution. Hence the convolution of two sequences can be expessed in tems of matix vecto multiplication, i.e. h 0 v = H 0 v. (1.14) In the following, we will denote the Toeplitz matix geneated fom h 0 by H 0 = Toeplitz(h 0 ). Let H l denote the infinite dimensional Toeplitz matix Toeplitz(h l ) fo l = 1,...,. Using the matix notation, the UEP condition (1.7) can be witten as H 0 H 0 +H 1 H 1 + +H H = I, (1.15) whee I is the identity opeato. To wite the decomposition and econstuction in convolution fom, the filtes used in decomposition below the 0th level need to be dilated. In level j < 0, the dilated filte is denoted by h l, j, which is defined by (also see (A.1)) { h h l, j [k] = l [2 j+1 k], k 2 j Z; 0, k Z\2 j (1.16) Z. The coesponding Toeplitz matix is H l, j = Toeplitz(h l, j ). (1.17) By the definition of h l, j, we have ĥ l, j = ĥ l (2 j ) and hence ĥ l, j 1 a.e. ω R. Moeove, as a bypoduct in the poof of Lemma 1.1, we have a condition simila to (1.15) fo dilated filtes h 0, j,...,h, j, j < 0: H0, j H 0, j +H1, j H 1, j + +H, j H, j = I. (1.18) We can see that when j =, (1.15) and (1.18) ae the same. The discete foms of decomposition and econstuction fom level j 1 to level j 2, whee j 1, j 2 0, ae the same as those in the affine system, which ae given in [19]. We only conside the discete fom of decomposition and econstuction fom level j 1 to level j 2, whee j 1, j 2 < 0. Fo a function f L 2 (R), we decompose f in X q (Ψ) and collect the coefficients in each level j < 0 to fom an infinite column vecto v l, j := [..., f,ψ q l, j,k,...]t, whee ψ q 0 := φ q and [ ] t is the tanspose of a ow vecto. Set the Toeplitz block matix H j := [ ] t H 0, j,h 1, j,...,h, j. With this, condition (1.18) implies H j H j = I. The decomposition pocess (1.13) can be witten in the matix fom as: v l, j = H l, j v 0, j+1, l = 0,...,, o [ ] t v0, j,...,v, j = H j v 0, j+1. (1.19) Because of (1.18), the econstuction pocess of Lemma 1.1 can be intepeted in the discete fom as v 0, j+1 = H j H j v 0, j+1 = H0, jh 0, j v 0, j+1 +H1, jh 1, j v 0, j+1 + +H, jh, j v 0, j+1 = H0, jv 0, j +H1, jv 1, j + +H, jv, j. (1.20)

8 8 Anwei Chai, Zuowei Shen The identities (1.19) and (1.20) togethe give the equivalent discete epesentation of (1.13). The above discussion essentially is one level decomposition and econstuction. Next, we intoduce the notation of seveal to infinite levels decomposition and econstuction. Fo any sequence v, it is decomposed by H v fist, then the low fequency component H 0 v is futhe decomposed by the same pocedue. The same pocess goes inductively. To descibe this discete pocess, we define the decomposition opeato A J, J < 0 and A. They ae composed of matix block like H l, j j = j H 0, j whee H 0, j is the composition of j j = j Toeplitz matices H 0, j, j j, given by (A.1) and (1.17), and acts on any sequence v l 2 (Z) in the following ode: H 0, j v = H 0, j H 0, j+1 H 0, v. j = j The decomposition opeato A J is a (ectangula) block matix defined as: and A is defined as [ ( ) ( H 0, j ; H1,J j=j ) ( H 0, j ;...; H,J j=j+1 [...; ( ) ( H 1,J H 0, j ;...; H,J j=j j=j ) H 0, j ;...; H1, ;...;H, ] t (1.21) j=j+1 H 0, j ) ; ( H1,J j=j ) ( ) H 0, j ;...; H,J H 0, j ; j=j ( ) ( ) H1,J+1 H 0, j ;...; H,J+1 H 0, j ;...; H1, ;...;H, ] t. j=j+1 j=j+1 In (1.21) and (1.22), H l, = H l, l = 0,1,..., and thus A = H. As we will see that both A J and A ae the opeatos defined on l 2 (Z) into the tenso poduct space, J l=0, j=1 l l, j 2 (Z) and, l=0, j=1 espectively, with l l, j 2 (Z) = l 2(Z). The econstuction opeatos and l l, j 2 (Z) (1.22) AJ = [( J H ) ( J+1 0, j ; H ) ( J+1 0, j H 1,J ;...; H0, j,j) H ;...; H 1, ;...;H, ] (1.23) j= j= j= A = [...; ( J H ) ( J 0, j H 1,J ;...; H ( J 0, j,j) H ; H ( J 0, j 1,J) H ;...; H0, j,j) H ; j= j= j= j= ( J+1 H ) ( J+1 (1.24) 0, j H 1,J+1 ;...; H ) 0, j H,J+1 ;...;H 1, ;...;H, ] j= j= ae the adjoint opeatos of A J and A espectively. The opeatos A J and A ae closely elated to P 0 and Q q 0. By Lemma 1.1 we have the identity P 0 f = P q J f + j=j f,ψ q l, j,k, J < 0. The coesponding coefficients in the ight hand side is A J v 0,0 with v 0,0 = { f,φ 0,k }. Similaly, the coefficients in the ight hand side of the identity used in analysis P 0 f = Q q 0 f can be obtained by A v 0,0. Futhemoe, the next poposition shows that the decomposition and econstuction pocess is pefect, i.e. A J A J = I and A A = I, which will be poven in Appendix A. Poposition 1.2 The decomposition opeatos A J and A, as defined in (1.21) and (1.22) espectively, satisfy A J A J = I and A A = I whee I is the identity opeato.

9 Deconvolution: A Wavelet Fame Appoach 9 2 Fomulation and Algoithms This section is to fomulate the deconvolution poblem via the multiesolution analysis and the famelet analysis. It convets the deconvolution poblem to the poblem of filling the missing famelet coefficients. Conside the convolution equation h 0 v = b+ε = c, (2.1) whee h 0 is a finitely suppoted low pass filte and b, c ae the sequences in l 2 (Z). The eo tem ε l 2 (Z) satisfies ε l2 (Z) ε. To simplify ou notation, we use := l2 (Z). Ou appoach stats with the efinable function geneated by the low pass filte h 0. Thee ae many sufficient conditions on the low pass filte h 0 with ĥ 0 (0) = 1 unde which φ is in L 2 (R). Hee we assume that h 0 satisfies the following condition ĥ 0 (ω) 2 + ĥ 0 (ω + π) 2 1, a.e. ω R. (2.2) As we will show in Appendix A, the coesponding efinable function φ is in L 2 (R) unde assumption (2.2). We futhe emak that this is not a stong assumption. Fo example, all efinement masks of B-splines, the efinable functions whose shifts fom an othonomal system deived in [17], the base functions of intepolatoy functions, and moe geneal, pseudo-splines intoduced by [19] and [21] satisfy this assumption. In fact, many low pass filtes used in pactical poblems satisfy (2.2). Fo example, the low pass filtes used in high esolution image econstuctions satisfy (2.2). Futhemoe, with this assumption, we can constuct a tight fame system via unitay extension pinciple of [31] which is used in ou algoithm. To make ou ideas wok hee, the cucial step is to constuct a tight fame o a bi-fame system via a multiesolution analysis with undelying efinement mask being the given low pass filte. The assumption (2.2) is a necessay and sufficient condition to constuct a tight fame system associated with the given low pass filte. Howeve, assumption (2.2) is not cucial fo ou idea to wok. Fo example, when the undelying efinable function φ is in L 2 (R), whose efinement mask is the given low pass filte in (2.1), togethe with some additional mino conditions, we can always obtain a bi-fame system via the mixed unitay extension pinciple of [32] and moe geneally the mixed oblique extension pinciple of [15] and [19]. Fo example, let h 0 (z) := h 0 [k]z k. Then (2.2) can be eplaced by the condition that h 0 (z) and h 0 ( z) have no common zeos in complex domain. With this, one can constuct a bi-fame system by using the mixed unitay extension pinciple. This is essentially the appoach taken by [6]. Ou analysis can be caied out fo this case with some effots. To simplify ou discussion hee, we only use the tight fame system, hence assume (2.2). Finally, since ou appoach is based on denoising schemes with theshold of famelet coefficients, we implicitly assume that the undelying function of the data set has a spase epesentation in the tight fame system and the eos ae elatively small and spead out unifomly in the fame tansfom domain. 2.1 Fomulation in MRA This section is to fomulate the poblem of solving h 0 v = b+ε = c (2.3) via the multiesolution analysis famewok. As we will see, the appoach hee educes solving equation (2.3) to the poblem of filling the missing famelet coefficients. This appoach was fist taken by [6], howeve, we give a complete analysis and fomulation hee. As we mentioned befoe, by using P J f = D J P 0 D J f and P q,j J f = DJ P q D J f, we may assume that data set is given on Z (i.e. J = 0) without loss of geneality. In fact, when the data set is given on 2 J Z, we conside function f(2 J ) instead of f. The appoximation powe of a function f in space V J is the same as that of the function f(2 J ) in space V 0.

10 10 Anwei Chai, Zuowei Shen Let φ L 2 (R) be the efinable function with efinement mask h 0 and h 1,...,h be high pass filtes obtained via the UEP which ae the famelet masks of ψ 1,...,ψ. Fist we suppose that the given data set contains no eo, i.e. ε = 0. The convolution equation h 0 v = b implies that b is obtained by passing the oiginal sequence v though a low pass filte h 0. Assume that b = { S,φ q,k }, whee S L 2(R) is the undelying function fom which the data set b is obtained. Then we ae given Let v S = { S,φ 0,k }, then P q S = S,φ q,k φ q,k = b[k]φ q,k. (2.4) P 0 S = S,φ 0,k φ 0,k = v S [k]φ 0,k. (2.5) Applying the famelet decomposition algoithm (1.13), one obtains that h 0 v S = b. This implies that solving equation (2.3) is equivalent to econstucting the quasi-intepolation P 0 S V 0 fom the quasi-intepolation P q S V q. Since P 0 S = P q S+ S,ψ q l,,k ψq l,,k, to ecove v S = { S,φ 0,k } fom given b, we need the famelet coefficients { S,ψ q l,,k }. This leads to an algoithm that estoes v S fom data b iteatively by updating the famelet coefficients { S,ψ q l,,k } in each iteation. All these have been given in [6] and consequent papes [8, 9] in thei econstuctions of high esolution images. In fact, it motivates the algoithms developed in [6,8,9]. By this appoach, we not only give a solution of (2.3), but also give an intepetation in tems of the undelying function S whee we view the data b = { S,φ q,k } as the given sample of S. Unde this setting, we ae given P q S V q, and the solution of (2.3) leads to P 0S V 0, which is a highe esolution subspace in the multiesolution analysis. Although thee ae moe than one function whose quasi-intepolations ae P q S and P 0S given as (2.4) and (2.5), we neve get the undelying function S. One can only expect to obtain a bette appoximation P 0 S of S fom P q S. The appoximation powe of P 0S and P q S and thei diffeence can be established fo smooth functions by applying the coesponding esults in [19] which depend on the popeties of the undelying efinable function; moe geneal fo piecewise smooth functions, it can be studied by applying esults and ideas fom [3] and [4] which depend on the popeties of the famelets. We omit the detailed discussion hee. Roughly speaking, the idea of solving equation (2.3) hee can be undestood as fo a given coase level appoximation P q S to find a fine level appoximation P 0S is educed to finding the coefficients v S = { S,φ 0,k }. The deivation of v S is an iteative pocess which ecoves P 0 S fom P q S as discussed befoe and detailed in the algoithms given in the next section. Then h 0 v S = b by the decomposition algoithm (1.13) and we conclude that v S is a solution of (2.3). Howeve, the data given may contain eos, i.e. instead of b, the data is given in the fom of c = b+ε. Futhemoe, the given data set b may not be necessay of the fom of { S,φ q,k }, fo some S L 2(R). In both cases, the exact l 2 (Z) solution of h 0 v = c may not exist o it may not be desiable o possible to get the exact solution. Nevetheless, thee is a need to have s = j<0, s l, j,k ψ q l, j,k V 0 to appoximate the undelying function whee the sample data set c comes fom. Let s = { s l, j,k } and s = A s, (2.6) whee A is the econstuction opeato given in (1.24). Fo the vecto s being a candidate of the solution of (2.3), it equies h 0 s within the ε ball of c and the function s has some smoothness. The smoothness of the function is eflected by the decay of the famelet coefficients which is measued by the l p nom of s. Given any sequence ṽ detemined by thee indices (l, j,k) with l = 1,...,, j < 0 and k Z, we say ṽ is in space l p, fo a given p, if j<0, ṽ l, j,k p <.

11 Deconvolution: A Wavelet Fame Appoach 11 Assuming that thee exists a function S such that s l, j,k = S,ψ q l, j,k, then function s = Qq 0S. Fo a given p, 1 p 2, we say that the pai (s, s) defined in (2.6) is the solution of (2.3) (and s is an appoximation of the undelying function of the data set) if fo an abitay g L 2 (R), the pai (g, g) whee g = { g,ψ q l, j,k } and g = A g, satisfies the following inequality h 0 g c 2 + j<0, λ j g,ψ q l, j,k p h 0 s c 2 + j<0, λ j S,ψ q l, j,k p. (2.7) Hee γ λ j γ, j Z, whee 0 < γ γ, ae paametes which will be detemined by the eo level. The function s is consideed as an appoximation of the undelying function whose sample is given by c. The fist tem measues the esidue of the solution s and the given data set c. The second tem is a penalization tem using a weighted (with weights λ j ) l p -nom of the coefficients of famelets. Since the famelet coefficients ae closely elated to the smoothness of the undelying function (see [3, 4]), minimization poblem (2.7) balances the fitness of the solution and the smoothness of the solution function s. The minimization condition (2.7) can be stated as following: fo a fixed p, 1 p 2, the pai (s, s) defined in (2.6) is a solution of (2.3) (the function s is an appoximation of the undelying function of the data) if fo an abitay η L 2 (R) satisfying { η,ψ q l, j,k } = { η l, j,k} = η l p, the pai (η, η), whee η = A η, satisfies the following inequality h 0 (s+η) c 2 + j<0, λ j s l, j,k + η l, j,k p h 0 s c 2 + j<0, λ j s l, j,k p. (2.8) Howeve, as we will see that the sequence s is uniquely detemined by algoithms, it may not be of the fom { S,ψ q l, j,k } fo any S L 2(R), since {ψ q l, j,k } j<0 is edundant which implies that the epesentation of s is not unique. Nevetheless, the pai (s, s) still can be consideed as a solution of equation (2.3) if (2.8) holds fo an abitay pai (η, η), whee η = { η l, j,k } = { η,ψ q l, j,k } l p, and η = A η with η L 2 (R). Hee, we note that since η = { η,ψ q l, j,k }, η = A η implies that η = A η by the decomposition algoithm. The function s entes into the discussion that gives an analysis in the function fom of the undelying solution. The undelying function s plays a ole in analysis, but does not ente the algoithm. Next, we link the fomulation to a discete fom of minimization poblem (2.8). The minimization poblem (2.8) can be stated as follows: fo a given p, 1 p 2, a pai of sequences (s, s), satisfying s l p and s = A s, is the solution of (2.3) if fo an abitay pai (η, η) satisfying η = A η l p, the following inequality holds: h 0 (s+η) c 2 + j<0, λ j s l, j,k + η l, j,k p h 0 s c 2 + j<0, λ j s l, j,k p. (2.9) We note that (2.8) and (2.9) look simila, but they ae deived in a diffeent setting. Fo example, sequences in (2.8) ae deived fom the analysis sequences of functions unde the given wavelet fame system, while sequences (2.9) ae obtained in a puely discete sense via filtes of the given wavelet fame system. We should also emak hee the condition s = A s on the pai (s, s) is diffeent fom the condition η = A η on the pai (η, η). The condition η = A η implies η = A η, since A η = A A η = η by A A = I. Howeve, the condition s = A s, in geneal, does not implies s = A s, unless A A = I o s happens to be A s. Note that the identity A A = I does not hold fo any edundant system. The easons fo imposing the diffeent conditions ae due to that (s, s) is obtained by the algoithm which only satisfies s = A s, while fo given η, thee is moe than one η such that A η = η. We choose the canonical pai (η, η) with η = A η. 2.2 Algoithms We give algoithms to solve (2.3) with the fomulation in MRA. In ou appoach, the algoithms iteatively impove the famelet coefficients using the esult in pevious iteation. Let h 1,...,h be the sequences deived fom h 0 via the UEP and H 0,H 1,...,H be the coesponding Toeplitz matices. Ou algoithm based on the UEP condition H 0 H 0 + H l H l = I. (2.10)

12 12 Anwei Chai, Zuowei Shen Let v n be the solution fo the nth iteation, then H 0 H 0v n + H l H lv n = v n. (2.11) Fist, we conside the case that b = { S,φ q,k }, whee S is the undelying function and b is the given data as a set of the samples of S, and ε = 0. Then by h 0 v S = b with v S = { S,φ 0,k }, we have v S is a solution to equation (2.3). In each iteation, we can eplace H 0 v n by the known data b to impove the appoximation. This can also be viewed as that we use the famelet coefficients of the n th iteation to appoximate the famelet coefficients of the undelying function S. We summaize the algoithm as follows: Algoithm 2.1 (i) Choose an initial appoximation v 0 (e.g. v 0 = b); (ii) Iteate on n until convegence: v n+1 = H 0 b+ H l H lv n. (2.12) As we will see in the next section, Algoithm 2.1 conveges, but it conveges slowly. We need to adjust the iteation in Algoithm 2.1 to quicken the convegence, which motivates us to intoduce the acceleation facto 0 < β < 1 into the above algoithm. The new iteation with β is given below: v n+1 = β(h 0 c+ H l H lv n ) = H 0 βc+ H l H lβv n. (2.13) This scheme can be viewed as the taditional egulaization method used in noise emoval, the solution of which satisfies the matix equation ( H0 ) H 0 +(1 β) Hl H l v = H0 βc. Hee β is a egulaization paamete. The solution of the oiginal convolution equation (2.3) is v = v β /β with v β the solution to the above matix equation. The solution v minimizes the following functional: H 0 v c β β v 2. This is the standad egulaization fom with a special egulaization opeato, which was moe o less the [6, Algoithm 2] given to us. The paamete β has to be caefully chosen to balance the eo and smoothness of the solution. It plays a ole in both convegence acceleation and eo emoval. Howeve, when a diffeent penalty functional instead of l 2 nom of the solution (e.g. the one given in the fomulation), which is desiable in many applications, is used, we need a diffeent appoach. In ou new algoithms, the acceleation facto β is mainly used to acceleate the convegence and leave the egulaization pat to a theshold pocess. Finally, we emak that, as will see in 4, in the numeical implementation, when pope bounday conditions (e.g. peiodic bounday condition with some modifications) ae used, the matix H 0 becomes a nonsingula finite ode matix. The iteation in Algoithm 2.1 conveges with ate 1 λ, whee λ is the minimum eigenvalue of H 0 H 0. Hence, we do not need to intoduce the acceleation facto β. Next, we intoduce the following denoising opeatos to the iteation (2.13). Denoising Opeato When data ae contaminated with eos, we need to emove the eos fom each iteation befoe putting it into the next iteation. The denoising scheme is needed to pevent the limit of iteation (2.13) fom following the noise esiding in c. Fo any vecto v and given p, 1 p 2, let theshold opeato be D p λ (v) := [ t p λ (v[0]),t p λ (v[1]),...] t, (2.14) whee t p λ (x) is the theshold function. When p = 1, t λ(x) := t 1 λ (x) = sgn(x)max( x λ/2,0) is the softtheshold function; when 1 < p 2, the theshold function is defined by the invese of function F p λ (x) := x+ pλ 2 sgn(x) x p. (2.15)

13 Deconvolution: A Wavelet Fame Appoach 13 Function F p λ (x) is a one-to-one diffeentiable function with unique invese. Fo 1 < p 2, the explicit fomula of the invese of function F p λ is not always available. Numeical method may be needed to calculate the value of t p λ (x) := (Fp λ ) (x). As we will see, the diffeence of the theshold opeatos D p λ accoding to diffeent p is that the limit of the algoithm has diffeent minimization popeties. When a signal v is given, the nomal pocedue is fist tansfoming v to the famelet domain via the decomposition opeato A to decoelate the signal, and then applying the theshold opeato D p λ j with the theshold paamete λ j depending on the decomposition level j. Fo a given sequence v l 2 (Z), the denoising opeato T p which applies the theshold opeato D p λ j on A v with the theshold paametes {λ j } is defined as: T p A (v) = [D p λ (H j l, j H 0, j v)] t j l, j, (2.16) = j+1 whee 1 p 2, l = 1,2,...,, j < 0. This noise emoval scheme will then be applied at each iteation befoe applying the next iteation in Algoithm 2.1. Algoithm 2.2 is given in [9] which was motivated by [18]. At the nth step, the theshold opeato is applied to the famelet decomposition of H 0 βc+ H l H lβv n. The paametes λ j ae fixed duing the iteation. Algoithm 2.2 (i) Choose an initial appoximation v 0 (e.g. v 0 = c); (ii) Iteate on n until convegence: v n+1 = A T p A (H 0 βc+ (iii) Suppose the limit of step (ii) is v β. Then the final solution is s β = v β /β. H l H lβv n ); (2.17) We will pove that the pai (s β, s β ) whee s β = 1 β T p A (H 0 βc+ H l H lβv β ) obtained fom step (iii) of Algoithm 2.2 satisfies inequality (2.9) (up to an abitay small ε). Next algoithm has a diffeent denoising scheme fom Algoithm 2.2. Instead of applying the denoising opeato to each iteation befoe it is put in the next iteation, the denoising opeato only acts on the appoximation of the missing famelet coefficients. This is the pocess suggested by [6,8,9]. Algoithm 2.3 (i) Choose an initial appoximation v 0 (e.g. v 0 = c); (ii) Iteate on n until convegence: v n+1 = H 0 βc+ H l (A T p A )(βh l v n ); (2.18) (iii) Let v β be the final iteative solution fom (ii). Then the solution to the algoithm is s β = v β /β. Fo bette denoising effect, we may apply the denoising scheme to the final esult s β, i.e. we take an additional step (iv) υ = A T p A (s β ) to futhe emove the eo effect aose by c, which is used in [6,8,9]. 3 Analysis of Algoithms This section focuses on the analysis of the algoithms given in 2.2. We fist show that all algoithms convege. Secondly, we pove that the solutions of Algoithm 2.2 and 2.3 satisfy some minimization popety.

14 14 Anwei Chai, Zuowei Shen 3.1 Convegence In this section, we will show the convegence of Algoithm 2.1, 2.2 and 2.3. The poof of the convegence of Algoithm 2.1 was given in [6] and [9]. We include the poof hee fo the sake of the self completeness of the pape. Howeve, the convegence of Algoithm 2.2 and 2.3 is new. This is impotant, since both algoithms ae the ones used in pactice. Poposition 3.1 Let h 1,...,h be the high pass filtes of a tight fame system deived by the UEP with finitely suppoted h 0 being the given low pass filte which satisfies (2.2). Suppose thee exists a function S such that c = { S,φ q,k }. Then fo abitay v 0 l 2 (Z), the sequence v n defined by (2.12) conveges to v = { S,φ q 0,k }. Especially, h 0 v = c. Poof The poof was given in [6]. Witing (2.12) in fequency domain, one obtains v n+1 = ĥ 0 ĉ+ ĥ l ĥ l v n. Let v = { S,φ 0,k }. Since c = { S,φ q,k }, v is the solution to (2.3). Using the UEP condition, we have v = ĥ 0 ĉ+ ĥ l ĥ l v. Fo abitay v 0 l 2 (Z), applying the iteation n times, we have ( ) n v n v = ĥ l ĥ l ( v 0 v). Fom (2.2), we have 0 ĥ 0 (ω) 1 a.e. ω R and ĥ 0 (ω) = 0 only holds on a zeo measue set since ĥ 0 (ω) is a polynomial the zeo points of which ae finite. Because h 1,...,h satisfy (1.6), it follows that ĥ l (ω) 2 1, a.e. ω R and the equality only holds on a zeo measue set. Thus we have v n v v 0 v and v n v 0 a.e. ω R as n. Then by Dominated Convegence Theoem, v n v l2 (Z) = 1 2π v n v L2 [ π,π] 0, i.e. v n conveges to v as n. Since ĥlĥl = 1 at π, the convegence of the algoithm is slow. That is the eason why we intoduce the acceleation facto β into iteation. The convegence of iteation (2.13) can be poved similaly. Next we show the convegence of the iteations in Algoithm 2.2 and Algoithm 2.3. The following lemma is needed, the poof of which is given in [18, Lemma 2.2]. Poposition 3.2 The denoising opeato D p λ, 1 p 2 is non-expansive, i.e. fo any two sequences v 1 and v 2 in l 2 (Z), D p λ (v 1) D p λ (v 2) v 1 v 2. Futhemoe, let T p be the denoising opeato defined by (2.16), it also satisfies that T p A (v 1 ) T p A (v 2 ) v 1 v 2. In paticula, T p A is continuous and T p A v 1 v 1. Now we ae eady to show the convegence of Algoithm 2.2. Theoem 3.1 Let h 1,...,h be the high pass filtes of a tight fame system deived by the UEP with h 0 being the given low pass filte which satisfies (2.2). Then the sequence v n defined by (2.17) in Algoithm 2.2 conveges fo abitay initial seed v 0 l 2 (Z) to v β which satisfies v β = A T p A (H 0 βc+ H l H lβv β ). (3.1)

15 Deconvolution: A Wavelet Fame Appoach 15 Poof The idea of the poof is to show that the sequence {v n } is a Cauchy sequence. We fist note that A 1. Let and fo m > 0 v n = A T p A (H 0 βc+ v n+m = A T p A (H 0 βc+ H l H lβv n ) H l H lβv n+m ). Fo convenience, denote u = H 0 βc + H l H lβv n and u = H 0 βc + H l H lβv n+m. Then using Poposition 3.2 we have: v n+m v n = A (T p A u T p A u) T p A u T p A u u u β v n+m v n. Inductively, we finally obtain that v n+m v n β n v m v 0. (3.2) Then sequence {v n } is a Cauchy sequence if {v n } is bounded. Since 0 < β < 1, indeed due to Poposition 3.2 we have v n = A T p A u T p A u u β c +β v n β 1 β c + v 0. (3.3) Hence the limit of the iteation (2.17) exists. The limit v β satisfying v β = A T p A (H0 βc+ H l H lβv β ) follows the continuity of T p A. Hee we note that the limit v β of iteation (2.17) satisfies (3.1). Let ṽ β = T p A (H 0 βc+ H l H lβv β ), then the pai (v β,ṽ β ) satisfies v β = A ṽ β. As a consequence, the pai (s β, s β ) with s β = 1 β vβ and s β = 1 β T p A (H 0 βc+ H l H lβv β ) also satisfies s β = A s β. We will pove in the next subsection that the pai (s β, s β ) satisfies the inequality (2.9) up to a small ε > 0 when β is close to 1. A simila poof shows the convegence of iteation (2.18) in Algoithm 2.3 as stated below. Theoem 3.2 Let h 1,...,h be the high pass filtes of a tight fame system deived by the UEP with h 0 being the given low pass filte which satisfies (2.2). Then the sequence v n defined by (2.18) in Algoithm 2.3 conveges fo abitay initial seed v 0 l 2 (Z) to v β which satisfies v β = H 0 βc+ H l A T p A (H l βv β ). (3.4) 3.2 Minimization Popety of Algoithm 2.2 In this section, we discuss to what extend that the solution s β obtained fom Algoithm 2.2 satisfies (2.9). Without futhe claification, we fixed p [1,2] in the following discussion. By Algoithm 2.2, whee ṽ β = T p A (H 0 βc+ s β = 1 β vβ and s β = 1 β ṽβ, H l H lβv β ) and v β = A ṽ β (3.5) ae obtained fom the limit of iteation (2.17). Fist, if s β l p, then fo any pai (η, η) with η = A η l p, the values of both sides in (2.9) ae infinite and the inequality holds. Fo the case s β l p, what we will pove is a slightly weake esult than (2.9) fo the pai (s β, s β ) as stated below.

16 16 Anwei Chai, Zuowei Shen Fo the given constant C > 0 and abitay ε > 0, the pai (s β, s β ) satisfies the following inequality h 0 (s β + η) c 2 + j<0, λ j s β l, j,k + η l, j,k p h 0 s β c 2 + j<0, λ j s β l, j,k p ε, (3.6) fo any pai (η, η) satisfying η = A η l p with η C, as long as the acceleation facto β is close enough to 1. As we will see in the next section, when cetain bounday conditions ae imposed in numeical implementations, the solution will satisfy (2.9). We fist pove the following statement: fo given ε > 0 and C > 0, the pai (v β,ṽ β ) given in (3.5) satisfies the following inequality H 0 (v β +η) βc 2 + j<0, H 0 v β βc 2 + β 2 p λ j ṽ β l, j,k + η l, j,k p +(1 β) 2 j<0, β 2 p λ j ṽ β l, j,k p +(1 β) 2 H l (v β + η) 2 H l v β 2 ε, (3.7) wheneve the pai (η, η) satisfying η = A η l p with η C and the acceleation facto β is close enough to 1. Note that the theshold paametes β 2 p λ j ae smalle than those in (3.6). It is easonable because the use of acceleation facto β helps to damp out the noise esiding in c. To show (3.7), we intoduce the following functionals. Fo a given pai of sequences (v, ṽ) satisfying v = A ṽ and a sequence a, define Φ(v) := H 0 v βc 2 + j<0, β 2 p λ j ṽ l, j,k p +(1 β) 2 H l v 2 (3.8) and Φ(v;a) := H 0 v βc 2 + j<0, β 2 p λ j ṽ l, j,k p + H l (v βa) 2. (3.9) It is clea that when a = v, we have Φ(v;v) = Φ(v). Futhemoe, the following esult on Φ(v;a) holds. Poposition 3.3 Suppose c = A c is in l p. Let h 1,...,h be the high pass filtes obtained fom h 0 by the UEP and H 0,H 1,...,H be the coesponding matix countepats of these filtes as defined in (1.14). Given a pai (a,ã) satisfying a = A ã and ã l p, let ṽ β = T p A (H 0βc+ H l H lβa) = T p A (βa (H 0 β(c H 0 a))) (3.10) and v β = A ṽ β. Then the pai (v β,ṽ β ) satisfies that fo any pai (η, η) with η = A η l p, Φ(v β + η;a) Φ(v β ;a)+ η 2. (3.11) A simila poposition is poved in [18], whee the undelying system used in denoising is othonomal basis. The poof depends on the fact A A = A A = I. Howeve, fo the tight fame system, one only has A A = I, while A A I. This adds difficulties to the poof and it also leads to the conditions on the pais (v β,ṽ β ) and (η, η). We povide a poof of Poposition 3.3 in Appendix B. To give the minimization popety of (v β,ṽ β ), we need that v β is unifomly bounded egadless of β. This equies the assumption that the theshold paametes λ j ae independent of β and inf j λ j γ > 0, j < 0. This condition is natual in applications. Indeed, this assumption equies to discad the famelet coefficients when j is sufficiently lage, because fo a given signal, when j is lage enough, the coefficients of the low fequency subband ae vey small and can be discaded anyway. We fist pove the following lemma:

17 Deconvolution: A Wavelet Fame Appoach 17 Lemma 3.1 Let h 1,...,h be the high pass filtes of a tight fame system deived by the UEP with h 0 being the given low pass filte. Suppose the theshold paametes λ > 0, then thee exists a constant 0 < ρ < 1 such that fo any sequence v l 2 (Z) D p λ (v) ρ v, whee D p λ is the theshold opeato defined in (2.14). Futhe, let T p be the denoising opeato. Assuming that inf j λ j γ > 0, we have T p A (v) ρ v, 0 < ρ < 1. Poof By (2.14), we have D p λ (v) 2 = t p λ (v[k]) 2. When p = 1, it is the soft-theshold function: t λ (x) = sgn(x)max( x λ/2,0). If λ 2sup v[k], then D p λ (v) = 0 and hence the inequality t λ(v[k]) ρ v[k] holds fo any 0 < ρ < 1. If λ < 2sup v[k], then fo a given k Z, we have t λ (v[k]) v[k] 1 λ 2 v[k] 1 λ 2 v. t Since v l 2 (Z), we have ρ = sup λ (v[k]) v[k] 1 λ 2 v < 1. Next, when 1 < p 2, by (2.15), we have t p p λ (x) = (Fλ ) (x) whee F p λ (x) = x+ pλ 2 sgn(x) x p. Given v[k] fo a fixed k Z, if (F p λ ) (v[k]) 0, then let y = (F p λ ) (v[k]) and we have (F p λ ) (v[k]) v[k] = y y+ pλ 2 sgn(y) y p = 1 1+ pλ 1 2 y p 2 1+ pλ < 1. 2 v p 2 When (F p λ ) (v[k]) = 0, it is clea that (F p λ ) (v[k]) 1 1+ pλ v[k]. 2 v p 2 Thus when 1 < p 2, we take (F p λ j ) (v[k]) ρ = sup v[k] 1 1+ pλ < 1. 2 v p 2 Thus theshold opeato D p λ satisfies Fo the denoising opeato, by (2.16), then fo each sequence H l, j T p A (v) 2 = D p λ (v) ρ v, 1 p 2. j= D p λ j (H l, j H 0, j v, thee exists ρ l, j such that j = j+1 D p λ j (H l, j H 0, j v) 2, j = j+1 H 0, j v) ρ l, j H l, j H 0, j v. j = j+1 j = j+1 Since H l, j j = j+1 H 0, j v v and inf j λ j γ > 0, we can take ρ = supρ l, j l, j { 1 γ 2 v 1 1+ pγ 2 < 1, when p = 1; v p 2 < 1, when 1 < p 2.