Applied atheatics 96-6 doi:.436/a.. Published Online February (http://www.scirp.org/ournal/a) Wavelet Bases ade of Piecewise Polynoial Functions: Theory and Applications * Abstract Lorella Fatone aria Cristina Recchioni Francesco Zirilli 3 Dipartiento di ateatica e Inforatica Università di Caerino Caerino Italy Dipartiento di Scienze Sociali D. Serrani Università Politecnica delle arche Ancona Italy 3 Dipartiento di ateatica G. Castelnuovo Università di Roa La Sapienza Roa Italy E-ail: lorella.fatone@unica.it.c.recchioni@univp.it f.zirilli@caspur.it Received October 8 ; revised oveber 6 ; accepted Deceber We present wavelet bases ade of piecewise (low degree) polynoial functions with an (arbitrary) assigned nuber of vanishing oents. We study soe of the properties of these wavelet bases; in particular we consider their use in the approxiation of functions and in nuerical quadrature. We focus on two applications: integral kernel sparsification and digital iage copression and reconstruction. In these application areas the use of these wavelet bases gives very satisfactory results. Keywords: Approxiation Theory Wavelet Bases Kernel Sparsification Iage Copression. Introduction In the last few decades wavelets and wavelets techniques have generated uch interest both in atheatical analysis as well as in signal processing and in any other application fields. In atheatical analysis wavelet bases whose eleents have good localization properties both in the spatial and in the frequency doains are very useful since for exaple they consent to approxiate functions using translates and dilates of one or of several given functions. In signal processing wavelets were initially used in the context of subband coding and of quadrature irror filters but later they have been used in a variety of applications including coputer vision iage processing and iage copression. The link between the atheatical analysis and signal processing approaches to the study of wavelets was given by Coif-an allat and eyer (see [-6]) with the introduction of ultiresolution analysis and of the fast wavelet transfor and by Daubechies (see [7]) with the introduction of orthonoral bases of copactly supported wavelets. Let A be an open subset of a real Euclidean space and let L A be the Hilbert space of the square integrable (with respect to Lebesgue easure) real functions defined on A. In this paper when A is a suffici- * The nuerical experience reported in this paper has been obtained using the coputing resources of CASPUR (Roa Italy) under grant: Algoriti di alte prestazioni per problei di scattering acustico. The support and sponsorship of CASPUR are gratefully acknowledged. ently siple set (i.e. a parallelepiped in the exaples considered) starting fro the notion of ultiresolution analysis we construct wavelet bases of L A with an (arbitrary) assigned nuber of vanishing oents. The ain feature of these wavelet bases is that they are ade of piecewise polynoial functions of copact support; oreover the polynoials used are of low degree and generate bases with an arbitrarily high assigned nuber of vanishing oents. This fact akes possible to perfor very efficiently soe of the ost coon coputations such as for exaple differentiation and integration. However the lack of regularity of the piecewise polynoial functions used can create undesirable effects in the points where the discontinuities occur when for exaple continuous functions are approxiated with discontinuous functions. ote that the wavelet bases studied here in general ake use of ore than one wavelet other function. Thanks to these properties these wavelet bases in several applications can outperfor in actual coputations the classical wavelet bases and for exaple in this paper we show that they have very good approxiation and copression properties. The nuerical results of Section 4 corroborate these stateents both fro the qualitative and the quantitative point of view. The wavelet bases introduced generalize the classical Haar s basis that has only one vanishing oent and is ade of piecewise constant functions (see for exaple [8]) and are a siple variation of the ulti-wavelets Copyright SciRes. A
L. FATOE ET AL. 97 bases introduced by Alpert in [9]. The results reported here extend those reported in [] and ai to show not only the theoretical relevance of these wavelet bases (also shown for exaple in [934]) but also their effective applicability in real probles. In particular in this paper we study soe properties of the wavelet bases considered and the advantages of using soe of the in siple circustances. In fact the orthogonality of the wavelets to the polynoials up to a given degree (i.e. the vanishing oents property) plays a crucial role in producing sparsity in the representation using these wavelet bases of functions integral kernels iages and so on. These wavelet bases as the wavelet bases used previously have good localization properties both in the spa- tial and in the frequency doains and can be used fruit- fully in several classical probles of functional analysis. In particular we focus on the representation of a function in the wavelet bases and we present soe ad hoc quadrature forulae that can be used to copute efficiently the coefficients of the wavelet expansion of a function. We consider also the use of these wavelet bases in soe applications initially we focus on integral kernel sparsification. This is a relevant task see for exaple [5] since it akes possible aong other things the approxiation of soe integral operators with sparse atrices allowing the approxiation and the solution of the corresponding integral equations in very high diensional subspaces at an affordable coputational cost. In [6] for exaple we exploit this property to solve soe tie dependent acoustic obstacle scattering probles involving realistic obects hit by waves of sall wavelength when copared to the diension of the obects. Let us note that these scattering probles are translated atheatically in probles for the wave equation and that they are nuerically challenging oreover thanks to the use of the wavelet bases when boundary integral ethods or soe of their variations are used they can be approxiated by sparse systes of linear equations in very high diensional spaces (i.e. linear systes with illions of unknowns and equations). Later on we focus on another iportant application of wavelets: digital iage copression and reconstruction. In this fraework the basic idea is to distinguish between relevant and less relevant parts of the iage details disregarding if necessary the second ones. In particular we proceed as follows: a digital iage is represented as a sequence of wavelet coefficients (wavelet transfor of the original iage) a siple truncation procedure puts to zero soe of the calculated wavelet coefficients (i.e. those that are saller than a given threshold in absolute value) and keeps the reaining ones unaltered (co- pression). The truncation procedure is perfored in such a way that the reconstructed iage (i.e. the iage obtained acting with the inverse wavelet transfor on the truncated sequence of wavelet coefficients) is of quality coparable with the quality of the original iage but the aount of data needed to store the copressed iage is uch saller than the aount of data needed to store the original iage. We present soe interesting nuerical results in wavelet-based iage copression and reconstruction. oreover we define a copression coefficient to evaluate the perforance of the copression procedure and we study the behaviour of the copression coefficients on soe test probles in particular we show that the copression coefficients increase when the nuber of vanishing oents of the wavelet basis used increases. This property can be exploited in several practical applications. The paper is organized as follows. In Section using a ultiresolution approach we present the wavelet bases. In Section 3 soe atheatical properties of the wavelet bases introduced are discussed and soe applications of these bases to function approxiation are shown. Furtherore soe quadrature forulae that exploit the bases properties are presented. In Section 4 applications of the wavelet bases introduced to kernel sparsification and iage copression are shown. In particular in Subsection 4. we study soe interesting properties of the bases considered and we present soe results about integral kernel sparsification. In Subsection 4. we focus on applications of the wavelet bases to iage coding and copression showing soe interesting nuerical results. Finally in the Appendix we give the wavelet other functions necessary to construct the wavelet bases eployed in the nuerical experience presented in Section 4. To build these other functions we have used the Sybolic ath Toolbox of ATLAB. The website http://www. econ.univp.it/recchioni/scattering/w7 contains auxiliary aterial and aniations that help the understanding of the results presented in this paper and akes available to the interested users the software progras used to generate the wavelet other functions of the wavelet bases used to produce the nuerical results presented. A ore general reference to the work of the authors and of their coworkers in acoustic and electroagnetic scattering where the wavelet bases introduced have been widely used is the website http://www.econ.univp.it/recchioni /scattering.. ultiresolution Analysis and Wavelets Let us use the ultiresolution analysis introduced by Copyright SciRes. A
98 L. FATOE ET AL. allat [] eyer [3-5] and Coifan and eyer [6] to construct wavelet bases. Let us begin introducing soe notation. Let R be the set of real nubers given a posis tive integer s let R be the s-diensional real Eucli- T s dean space and let x = x x xs R be a generic vector where the superscript T eans transposed. Let () and denote the Euclidean scalar product and the corresponding Euclidean vector nor respectively. For siplicity we restrict our analysis to the interval (). ore precisely we choose A = R. Let L be the Hilbert space of square integrable (with respect to Lebesgue easure) real functions defined on the interval (). We want to construct ortho- noral wavelet bases of L via a ultiresolution ethod siilar to the ethods used in [9-6]. ote that using the ideas suggested in [34] it is possible to generalize the work presented here to rather general doains A in diension greater or equal to one. Given an integer we consider the following L : decoposition of L P V = () where denotes the direct su of two orthogonal closed subspaces P and V of L. In other words the vector space generated by the union of P and V is L and we have: dx f x g x = f P g V. We take P to be the space of the polynoials of degree saller than defined on () and we consider a basis of P ade of polynoials orthogonal in the interval () with respect to the weight function wx= x having L -nor equal to one. For exaple we can choose as basis of P the first Legendre orthonoral polynoials defined on () and we refer to the as Lq ( x ) x () q =. To construct a basis of V we use the ideas behind the ultiresolution analysis. Let us begin defining the so called wavelet other functions. Let be T = R be a an integer and let vector whose eleents i i = are such that ηi < ηi i =. Given the integers J such that J we define the following piecewise polynoial functions on (): () p ( x) x ( ) i i i p x x ( x) = p ( x) x[ ) i = ( ) [ ) = J (3) where p x l q x P = i l = l i i = = J are polynoials of degree saller than to be deterined. The functions = J defined in (3) will be used as wavelet other functions. In fact through the we generate the eleents of a wavelet faily via the ultiresolution analysis. For siplicity let us choose i = i/ i =. We note that results analogous to the ones obtained here with this choice can be derived for the general choice of i i = at the price of having ore involved forulae. Let us define the functions: x x = x x = = = J (4) whose supports are the intervals = = define the set of functions J. oreover we W as follows: J q W = L x x q = x x () = J = = where Lq x q = and when we choose = 3 ( x) (5) x = J are the functions defined above. We want to choose J and the coefficients of the polynoials that constitute the functions = J defined in (3) that is the coefficients q li l = of p i T = J i = in order to guarantee that the set W defined in (5) is an orthonoral J basis of L. This can be done when J Copyright SciRes. A
L. FATOE ET AL. 99 satisfy soe constraints specified later iposing the following conditions to the wavelet other functions (3): i) the functions = J have the first oents vanishing that is: l dx x x = l = = J ii) the functions noral functions that is: (6) = J are ortho- dx = x ' x = = J. ote that depending on the choice of the integers J it ay not be possible to satisfy the relations (6) (7) with functions of the for (3). We note that in general the nuber of the unknown coefficients q l = of p i li (7) = J i = is bigger than the nuber of distinct equations contained in (6) and (7). ore precisely only when J = = and = the nuber of equations is equal to the nuber of unknowns and the unknown coefficients are deterined up to a sign perutation. That is we can change sign to the resulting wavelet other functions. In this case the set of functions defined in (5) (6) (7) when = is the well-known Haar s basis (see [8]). In all the reaining cases when the relations (3) (6) (7) are copatible the functions that satisfy (6) (7) generate through the ulti- resolution analysis an orthonoral set of L. When the integers J satisfy soe relations this orthonoral set is coplete that is is an orthonoral basis of L and can be regarded as a generalization of the Haar s basis. We ust choose soe criterion to deterine the coefficients of the polynoials contained in (3) that reain undeterined after iposing (6) (7). It will be seen later that the criterion used to choose the undeterined coefficients influences greatly the sparsifying properties of the resulting wavelet basis that is influences greatly the practical value of the resulting wavelet basis. On grounds of experience we restrict our analysis to two possible criteria: ) ipose soe regularity properties to the wavelet other functions (3) ) require that the wavelet other functions (3) have extra vanishing oents after those required in (6). ote that the analysis that follows considers only soe siple exaples of use of these criteria and can be easily extended to ore general situations to take care of several other eaningful criteria besides ) and ) such as for exaple a cobination of these two criteria or ad hoc criteria dictated by special features of the concrete probles considered. We begin adopting criterion ). We choose J = and in order to deterine the coefficients left undeterined by (6) (7) we ipose the following regularity conditions to the piecewise polynoial functions : d p = i i p i i d dx dx for i and where the sets of indices (8) are chosen such that (6) (7) (8) (and (3)) are copatible. ote that when all the undeterined paraeters have been chosen conditions (6) (7) (8) (and (3)) deterine the functions = up to a sign perutation. We denote with = a choice of the functions = given in (3) satisfying (6) (7) v and (8). Siilarly we denote with = = = the functions defined in (4) when we replace with and with W ponding to J the set corres- W when we choose J = and we replace with We will see later that W noral basis of L. Let us adopt criterion ). We choose = = = =. is an ortho- J and in order to deterine the coefficients q li l = of p i = i = left undeterined by (6) (7) we add to (6) (7) (and (3)) the following conditions: l dx x x = l = (9) = Copyright SciRes. A
L. FATOE ET AL. where the integers are chosen such that (6) (7) (9) (and (3)) are copatible. That is we ipose the vanishing of soe extra oents beside those contained in (6). Let us observe that in (9) when we have = for soe the corresponding index l ranges in a set of decreasing indices i.e. l = in this case with abuse of notation we assue that no extra conditions on are added to (6) (7). In other words when for soe we have = condition (9) for is epty. We note that only soe wavelet other functions (not all of the) can have extra vanishing oents (i.e. only for soe we can have ) in fact if we choose = incopatible. In fact when = coeffints of conditions (6) (7) (9) (and (3) are J the unknown = defined in (3) constitute a vector space of diension. Iposing one extra vanishing oent to the functions = that is requiring (6) (7) and: l dx x x = l = = () is equivalent to choose linearly independent vectors in a vector space of diension and this is ipossible. However for exaple it is possible to ipose one extra vanishing oent to wavelet other functions or two extra vanishing oents to wavelet other functions and so on down to extra vanishing oents to only one wavelet other function. Let us note that copatible sets of conditions (6) (7) (9) (and (3)) correspond in general to situations where the nuber of conditions is saller than the nuber of coefficients of the piecewise polynoial functions that is even after adding (9) to (6) (7) (and (3)) soe of the coefficients of the wavelet other functions ay reain undeterined. In this case the undeterined coefficients can be chosen arbitrarily or for exaple they can be chosen using criterion ) or soe other criterion. We denote with = a choice of the functions = satisfying conditions (6) (7) (9) (and (3)) with a non trivial choice of = (i.e. a choice such that > for soe ) with = v = defined in (4) when we replace and with W J = the functions W when we choose = we replace with v with the set corresponding to J We will see later that W noral basis of L. ote that if J < and = = =. is an ortho- the relations (3) (6) (7) are copatible and the corresponding set (5) is orthonoral but is not coplete oreover if J > the relations (3) (6) (7) are incopaible. ote that the fact that the wavelet other functions (3) are piecewise polynoials fro one hand akes easy and efficient their use in the ost coon coputations (i.e. for exaple differentiation integration...) but on the other hand ay create undesired effects in the discontinuity points of the wavelets when regular functions are approxiated with discontinuous functions. The choice of the criteria ) and ) is otivated by the following reasons. When criterion ) is adopted we try to approxiate regular functions using a basis ade of as uch as possible regular wavelets. This is done in order to iniize the undesired effects coing fro the fact that regular functions are approxiated with non regular wavelets choosing large and sall. The goal that we pursue when we adopt criterion ) is the construction of wavelet bases ade of piecewise polynoial wavelets ade of polynoials of low degree with as uch as possible vanishing oents so that as will be seen better in Section 4 the sparsifying properties of the resulting wavelet bases are iproved in coparison with those of the wavelet bases that do not satisfy criterion ). This is done choosing sall and large so that a great nuber of extra vanishing oents can be iposed to any wavelet other functions ade of polynoials of low degree. As a atter of fact choosing and appropriately it is possible to construct wavelet bases satisfying to soe extent the two criteria ) and ) siultaneously. However this is beyond the scope of this paper. We note that increasing the nuber of the wavelet other functions and the nuber of their discontinuity points increase. In the Appendix we give the wavelet other functions necessary to construct the wavelet bases used in the nuerical experience presented in Section 4. Copyright SciRes. A
3. Soe atheatical Properties of the Wavelet Bases Let us prove that the set of functions considered previously are orthonoral bases of L. Lea 3.. Let be a positive integer and T = be the following closed subspaces of L : T f L f x p x = = p R = =. () Then we have: T T T T3 () (3) = T = P = T = L. (4) Proof. Properties () (3) can be easily derived fro (). The proof of (4) follows fro the density of the piecewise constant functions in L (see [7] Theore 3.3 p. 84). This concludes the proof. ote that for = f x T the fact that iplies that for = as a function of x for x f x= f x T. Theore 3.. Let J be three integers such that the conditions (6) (7) can be satisfied with functions of the for (3) and let x L. FATOE ET AL. x = J = = be the functions defined in (4) we have: and dx x x = p = = = = J dx ( x) ( x) or o r = = a nd = a nd = = = = = J. p (5) (6) Proof. Property (5) follows fro definition (4) and Equation (6). The proof of (6) when = = and follows fro the fact that the supports of the functions and are either disoint sets or sets contained one into the other. When = and the supports are disoint and when let us suppose for exaple > the supports are either disoint sets or sets contained one into the other depending on the values of the indices and. In fact when the supports are disoint condition (6) is obvious when the supports are contained one into the other condition (6) follows fro (5). Finally when = = = condition (6) follows fro Equation (7). This concludes the proof. ote that if we choose = v = = = then (5) can be iproved that is we can add to it the following condition: dx x x p = = = = = p (7) where are the non negative integers (not all zero) that have been used in conditions (6) (7) (9) to deterine the wavelet other functions. Theore 3.3. If J = the conditions (6) (7) can be satisfied with functions of the for (3) and the set W defined in (5) is an orthonoral ( ) basis of L. Proof. Let = J it is easy to see that conditions (3) (6) (7) are copatible and the set W is an orthonoral set of functions such that for = the subspace T defined in () is contained in the subspace generated by W. So that fro Lea 3. it follows that W is a basis of L. This concludes the proof. The following corollary is a particular case of Theore 3.3: Corollary 3.4. The sets W ( ) () and W L. are orthonoral bases of To unify the notation let us renae the functions be- W L longing to the basis defined in (5) as follows: ( ) of Copyright SciRes. A
L. FATOE ET AL. x = x x = = = x = L x x = = = so that the basis W ( ) of ( ) L defined in (5) can be rewritten as: W () = ( x) x() = ( ) = w hen < = when = ( ) (8) (9) where ( ) denotes the axiu between and. For later convenience in the study of integral operators and iages let us reark that wavelet bases in L can be easily constructed as tensor product of wavelet bases of L that is for exaple the following set is a wavelet basis of L : W () () = x y = x y x y ( ) = = w hen < = when when when = = () where are the quantities specified previously and we have chosen J =. The previous construction based on the tensor product can be easily extended to the case when A is a parallelepiped in diension s 3. ote that with straightforward generalizations of the aterial presented here it is easy to construct wavelet bases for L A when A is a sufficiently siple subset of a real Euclidean space (see [6] to find several choices of A useful in soe scattering probles). The analysis that follows of the wavelet bases when A = can be extended with no substantial changes to the other choices of A considered here. The L -approxiation of a function with a wavelet expansion is based on the calculation of the inner pro- ducts of the function to be approxiated with the wavelets to find the coefficients that represent the function in the chosen wavelet basis. That is the function is approxiated by a weighted su of the wavelet basis functions. Let us note that in contrast with the Fourier basis that is localized only in frequency the wavelet basis is localized both in frequency and in space and in the ost coon circustances only a few coefficients of the wavelet expansion ust be calculated to obtain a satisfactory approxiation. In order to calculate the wavelet coefficients of a generic function we proceed as follows. Given for = let I be the following set of indices: T ˆ I = = ˆ = ; ˆ = ; = () < The wavelet expansion of a function f L on the basis (9) can be written as follows: = () f x c x x = I where for = the coefficients are given by: I c c = dx f x x I (3) and the series () converges in L. ote that when we have I I = so that for = it is not necessary to write explicitly the dependence on of the coefficients c I defined in (3). The integrals (3) are the L -inner product of the function f either with a wavelet function or with a Legendre polynoial depending on the values of the indices. Copyright SciRes. A
L. FATOE ET AL. 3 In order to calculate efficiently the integrals (3) we construct soe siple ad hoc interpolatory quadrature forulae that take into account the definition of the basis functions. In particular let and I = I = (4) I = I < =. (5) For = and I using (8) and (4) we can rewrite (3) as follows: = c dxf x x ( ) = dxf x x I. (6) ote that the integrals in (6) depend on the function f and on the wavelet other functions = given in (3). The change of variable t = x x in (6) gives: c = dt f t t (7) I where f t = f t t (8) = =. Let tk k = n n couples tk f tk be n distinct nodes given k = n we consider the interpolating Lagrange polynoial of degree n f of the data t f t k n n that is given by: n f t f tkk t k = k k = n = (9) where k k = n are the characteristic Lagrange polynoials defined as: n t t k t = k = n (3) t t = k k and characterized by the property t k = k k = n where k is the Kronecker sybol. We have: f = f R f (3) n n where Rn f is the interpolation error. Substituting f with f in (7) we obtain the following n approxiation: n c dt f t t = n f. tk dt k t t I k = (3) ote that in (3) the sybol eans approxiates. Defining d = dt t t k k (33) = k = n Equation (3) can be rewritten as: n k k k = c f t d I. (34) This technique leads to interpolatory integration rules with weights defined in ters of the wavelet other functions. We note that in general the weights of these quadrature rules can have alternating signs this ipacts negatively on the stability of the coputations. evertheless choosing n sall it is possible to iniize the nuber of the quadrature weights large in absolute value and not all of the sae sign and it is possible in everyday coputing experience to avoid the Runge phenoenon. Having in ind definition (3) and choosing a sall nuber of quadrature nodes the integrals (33) that define d = k = n k are very easy to calculate since the integrands are low degree piecewise polynoial functions. It is easy to see that (34) is valid also for choosing = = fining d k as: f t f t k k k = k d = dt t L t I and de- = k = n. (35) In conclusion once the wavelet basis and the nodes t k k = n (with n sall) have been chosen the integrals that define d k = ( ) k = n that is the integrals (33) and (35) can be calculated explicitly and tabulated so that the coefficients c with I = defined in (3) can be approxiated with the quantities defined in (34) and therefore the wavelet expansion () can be approxiated very efficiently. Let us define the quadrature error ade approxiating with (34) the wavelet coefficients given in (6) that is: Copyright SciRes. A
4 L. FATOE ET AL. n n k k k = Ec = dtf t t f t d I. (36) For the quadrature error we prove the following lea: Lea 3.5. Let tk k = n be n distinct nodes and let t be a point belonging to the doain of the function f defined in (8). Assue where n t that is t = n t t n n that f C where C n n n is the space of the real continuous functions defined on () n -ties continuously differentiable. Then there exists a continuous function t t such that the quadrature error is given by: Ec = dtf ( ( t )) ( t ) ( t! ) I n = (37) is the nodal polynoial of degree n. n i= i n n Proof. It is sufficient to note that fro (9) and (3) we have: Ec = dtrf t t I =. (38) The thesis follows fro standard nuerical analysis results see for exaple [8] p. 49. This concludes the proof. ote that a result siilar to (37) is valid also for. In fact it is sufficient to choose in (37) I = = f t f t k = k and to replace () t with L =. It is worthwhile to note that usually the convergence properties of general quadrature rules such as the one presented here depend on the soothness of the integrand (i.e. discontinuities of the integrand or of any of its derivatives ay disturb the convergence properties of the quadrature rules) when non sooth functions such as for exaple piecewise sooth functions are involved in integrals like (3) it is convenient to split the integration interval () into subintervals corresponding to the sooth parts of the integrands and to apply a low order quadrature rule on each subinterval. Ad hoc quadrature rules for the coputation of wavelet coefficients have been developed by several authors see for exaple [9-3]. It could be interesting to extend the work of these authors to the wavelet bases presented here. The explicit coputation of the integrals d k = k = n has been done easily with the help of the Sybolic ath Toolbox of ATLAB. ore in detail in the nuerical experients of the next section we use coefficients d = k = k n calculated with coposite quadrature forulae choosing in each interval n = and the node t given by the iddle point of each subinterval. This choice corresponds to a one-point quadrature forula in each subinterval for the evaluation of the wavelet coefficients is otivated by the fact that in Subsection 4. we anipulate digital iages and due to the usual pixel structure of the digital iages a digital iage can be regarded as a piecewise constant real function defined on a rectangular region of the two- diensional Euclidean space R. So that if we consider bidiensional coposite quadrature forulae with as any intervals as the pixels of the iage in each diension with the intervals coinciding with the pixel edges and in each interval we calculate (33) and (35) with the choice n = at the price of very siple calculations exact wavelet coefficients can be obtained. oreover these quadrature forulae turn out to be useful for the rapid evaluation of the wavelet coefficients in the approxiation of integral kernels. It is worthwhile to note that coposite quadrature forulae with n > can be obtained with no substantial odifications of the procedure described above. 4. Applications: Kernel Sparsification and Iage Copression 4.. Wavelet Bases Decay Estiates and Kernel Sparsification We present soe interesting properties of the wavelet bases introduced in Section. In particular we show how the representation in these wavelet bases of certain classes of linear operators acting on L ay be viewed as a first step for their copression that is as a step to approxiate the with sparse atrices. After being copressed these operators can be applied to arbitrary functions belonging to L in a fast anner and linear equations involving these operators can be approxiated in high diensional spaces with Copyright SciRes. A
L. FATOE ET AL. 5 sparse systes of linear equations and solved at an affordable coputational cost. In particular we show how the orthogonality of the wavelet functions to the polynoials up to a given degree (the vanishing oents property) plays a crucial role in producing these sparse approxiations. Let be the closure of and be a C non-negative integer we denote with the space of the real continuous functions defined on -ties continuously differentiable on. We have: Theore 4.. Let be two integers J = W be the set of func- tions given in (5) and let K xy xy be a real function such that: K C. (39) oreover let = = = = = be the following quantities: = dx dy x yk x y = = = = = ( ). then there exists a positive constant have: D ax (4) D such that we = = = = = ( ) (4) Proof. The proof is analogous to the proof of Proposition 4. in [9]. In fact fro (39) it follows that there exists a positive constant C such that: Kx y Kx y C x y. x y (4) That is let be as above * * and x y be the center of ass of the set using the Taylor polynoial of degree of y = * K xy y and base point y = y when < or the Taylor polynoial of degree of * ( x)= K( x y) x with base point x = x when > Equation (5) the inequality (4) and using the fact that the functions and have support in the sets and respectively fro the reainder forula of the Taylor polynoial it follows that the estiate (4) for holds. ote that the constant D depends on C. This concludes the proof. For the wavelet basis function having extra vanishing oents the previous theore can be iproved. That is let v x and v y = = = = be as above we have: Theore 4.. Let be two integers * J = and let = ax = ( ) where the constants = are those appearing in (9) and are such that the Equations (6) (7) (9) (and (3)) are copatible. Let K xy x y be a real function such that: * KC oreover let W I. (43) be the set of functions defined in Section and. = = = = = be the following quantities: = dx x dy yk x y = = = = = ( ) (44) where v and v have respectively and vanishing oents. Then there exists a positive constant F such that we have: F v v ax = = = (45) = = ( ) Proof. The proof follows the lines of the proof of Theore 4.. Going into details condition (43) iplies that there exists a positive constant E such that: x * Kx y Kx y E x y y (46) = ax =. Let where * Copyright SciRes. A
6 be as above and be the center of ass of the set L. FATOE ET AL. x y * *. Initially let us consider the case <.When (and therefore ) we can use the Taylor polynoial of y= Kx y y of degree and base * point y = y. Because of Equation (7) and assuption (46) iicking the proof of Theore 4. we can obtain the estiate (45) for with raised to the power ( ) in the denoinator. On the other hand if > when ( ) < ( ) it is convenient to proceed as done previously when ( ) ( ) it is better to use the Taylor polynoial of degree of x= K x y * x with base point x = x obtaining with a sii- lar procedure the estiate (45) for with in the denoinator. The raised to the power other estiates contained in (45) for when > or = can be obtained in a siilar way. ote that the constant F depends on η * E. This concludes the proof. The estiates (4) and (45) are the basic properties that together with a siple truncation procedure ake the wavelet bases introduced previously useful to approxiate with sparse atrices linear operators. Let be L of the for: an integral operator acting on f x dykx y f y x f L = (47) where the kernel K is a real function of two variables. The copression algoriths we are interested in are based on the following observation. Generalizing what done in Section for a function f L when is represented on a wavelet basis i.e. when the kernel K is expanded as a function of two variables on the wavelet basis () the calculation of the entries of the (infinite) atrix that represents the operator in the wavelet basis involves the evaluation of the integrals (4) or (44). If the wavelet basis considered has several vanishing oents and we look at truncated wavelet expansions that is to an expansion on a truncated wavelet basis when the kernel K is sufficiently regular thanks to the estiates (4) or (45) the fraction of the entries whose absolute value is saller than a chosen threshold approaches one when the truncated expansion approaches the true expansion. The entries whose absolute value is saller than a suitable threshold can be set to zero coitting a sall error. This property akes possible the approxiation of the integral operator (47) with sparse atrices and akes possible to solve nuerically integral equations very efficiently. ote that when two different wavelet bases with the sae nuber of vanishing oents are used to represent the operator the estiates (4) and (45) show that the wavelets with the saller between the constants D or F will have expansion coefficients that decay faster to zero. Siilar arguents can be ade in the discrete case where we consider a piecewise constant function defined on a bounded rectangular doain of the for: xy = i xy Ai (48) i = r x y where r is a positive integer and h > is such that h r = and where A i is the pixel of indices i x y i.e. Ai = xy int = iint = h h i = r. ote that int() denotes the integer part of. Soe applications of these estiates to specific exaples are shown below. We use the Wavelet Bases 3 4 of the Appendix to test the perforances of the integral operators copression algorith described previously. We apply the algorith to a nuber of operators. The direct and inverse wavelet transfors used in the nuerical experience associated to these wavelet bases have been ipleented in FORTRA 9. Briefly we reind that the Wavelet Bases and 3 of the Appendix are done with piecewise polynoial functions ade of polynoials of degree zero while the Wavelet Bases and 4 of the Appendix are done with piecewise polynoial functions ade of polynoials of degree one. oreover the Wavelet Bases and are obtained using the regularity criterion (i.e. criterion )) while the Wavelet Bases 3 and 4 are obtained using the extra vanishing oents criterion (i.e. criterion )). The choice of reporting here and in Subsection 4. the nuerical results obtained with the Wavelet Bases 3 4 of the Appendix is otivated by the fact that these are the siplest bases aong the bases introduced in Section that it is possible to construct and anipulate. oreover the Bases and 3 ade of piecewise constant functions are particularly well suited to deal with piecewise constant functions. This type of functions is very iportant since it can be identified with digital signals Copyright SciRes. A
L. FATOE ET AL. 7 and iages. In particular we represent a linear operator in a wavelet basis copress it by setting entries of its representation on the wavelet basis whose absolute value is below a chosen threshold to zero and reconstruct the copressed operator using the inverse wavelet transfor. The coparison between the original and the reconstructed copressed operator is ade evaluating the re- lative L -difference between the original and the reconstructed copressed kernels and is very satisfactory. oreover coparing our results with those obtained by Beylkin Coifan and Rokhlin in [9] and by Keinert in [4] we observe that the wavelet bases studied here show siilar or better copression properties of the wavelet bases used in [94] even when we use a saller nuber of vanishing oents than that used in [9] and in [4]. The results of two experients are presented in this section and are suarized in Tables -3 and illustrated in Figures -4. In these tables di indicates the diension of the vector space generated by the truncated wavelet basis and C cop is the copre- ssion coefficient obtained with our algorith when the truncation threshold > is used. The copression coefficient C cop is defined as the ratio between di and the nuber of atrix eleents whose absolute value exceeds a threshold >. Exaple. In this exaple we consider the kernel: Kx y= x y x y (49) and we expand it in the Wavelet Bases 3 4 of the Appendix. We set to zero the entries of the resulting atrix whose absolute value is saller than a threshold and we obtain the results shown in Tables and. In particular in Table we show the copression coefficients C cop for three different values of the threshold and in Table we report the relative error in L -nor ade substituting the original operator with the reconstructed copressed operator after the trun- 6 cation procedure when =. ote that in Tables and in the tables that follow the sybol indicates that using all the eleents of the specified basis up to a certain resolution level it is ipossible to reach the diension di specified in the corresponding second colun of the tables. The structures of the non-zero entries after the truncation procedure in the atrices obtained using the Wave- 6 let Bases and 4 of the Appendix when = are shown in black in Figures and respectively. In particular in Figures and the atrices shown have di =5 that is they are atrices of 5 rows and coluns. Table. Exaple : The copression coefficient versus ε and di. C cop C cop C cop C cop di Basis Basis Basis 3 Basis 4 64..83.6-7 8.3 5.5 4.59 56.76.67 4.8 5 4.39 3.8 8.73 64.5 3.76.48-6 8.9 7.35 6.84 56 4.8 6.47 8.9 5.96 47. 44.5 64.4 4.9 3.3-5 8 5.3..7 56.96 43.69 3.37 5 38.95 74.76 59.65 C ε cop Table. Exaple : Relative difference in L -nor between the original and the reconstructed operator when -6 ε =. di Basis Basis Basis 3 Basis 4 64. -3.48-4. -3 8. -3 6.38-5 8.68-5 56 5.34-4 5.9-5 5.35-4 5.77-4 5.64-5 8. -5 Figure. Exaple : Sparsity structure of the atrix obtained with the Wavelet Basis of the Appendix for di = -6 5 The entries above the threshold ε = in absolute value are shown in black. Copyright SciRes. A
8 L. FATOE ET AL. cients than those obtained with the Wavelet Bases and 3 of the Appendix. oreover the L -relative errors obtained coparing the original and the reconstructed copressed operators are always at least one order of agnitude saller when the Wavelet Bases 4 of the Appendix are used instead than the Wavelet Bases 3 of the Appendix. Table 3. Exaple : The copression coefficient versus ε and di. C cop C cop C cop C cop C cop di Basis Basis Basis 3 Basis 4 64..48. -8 8.7.. 56.35.85.33 5 5.4 5.8 3.43 Figure. Exaple : Sparsity structure of the atrix obtained with the Wavelet Basis 4 of the Appendix for -6 di = 5. The entries above the threshold ε = in absolute value are shown in black. Exaple. In this exaple we copress the following piecewise constant function: i Hx y= Hi = i (5) i = =. xya i dixy i where A i is the pixel of index i defined previously. Reind that h di =. We use the Wavelet Bases 3 4 of the Appendix and three different values of the threshold. Table 3 and Figures 3 and 4 describe the results of these experi- 7 ents. The entries above the threshold = in absolute value of the atrices relative to the atrix (5) obtained using the Wavelet Bases and 3 of the Appendix when di = 56 are shown in black in Figures 3 and 4 respectively. Fro Tables -3 and Figures -4 the following observations can be ade. As far as the copression property is concerned we have a really ipressive iproveent going fro the Wavelet Basis to the Wavelet Basis 3 of the Appendix that is there is a real advantage in using the extra vanishing oents criterion. This fact is ore evident in Exaple where a continuous kernel is considered (in this case the copression coefficient is approxiately ultiplied by two). There is not the sae iproveent going fro the Wavelet Basis to the Wavelet Basis 4 of the Appendix however using these two bases we obtain uch higher copression coeffi- 64..78.48-7 8.4.75.56 56. 4.9 3.43 5 8.5 8.39 5.38 64.49.36.99-6 8.5 3.87 3.9 56 3.77 6.6 5.44 5 5.8 3. 8.4 Figure 3. Exaple : Sparsity structure of the atrix obtained with the Wavelet Basis of the Appendix for -7 di = 56. The entries above the threshold ε = in absolute value are shown in black. Copyright SciRes. A
L. FATOE ET AL. 9 Figure 4. Exaple : Sparsity structure of the atrix obtained with the Wavelet Basis 3 of the Appendix for -7 di = 56. The entries above the threshold ε = in absolute value are shown in black. 4.. Iage Copression and Wavelets Let us present soe results about digital iage copression and reconstruction. With the growth of technology in the last decades and the entrance into the so-called Digital-Age a vast aount of digital inforation has becoe available for storage and/or exchange and the efficient treatent of this enorous aount of data often present difficulties. Wavelet analysis is one way to deal with this proble. It produces several iportant benefits particularly in iage copression. Copression is a way of encoding data ore concisely or efficiently. It relies on two ain strategies: getting rid of redundant inforation ( redundancy reduction ) and getting rid of irrelevant inforation ( irrelevancy reduction ). Redundancy reduction concentrates on ore efficient ways of encoding the iage. It looks for patterns and repetitions that can be expressed ore efficiently. Irrelevancy reduction ais to reove or alter inforation without coproising the quality of the iage. These two strategies conduct to two kinds of copression schees: lossless and lossy ones. Lossless copression is generally based on redundancy reduction and the key point is that no inforation is irreversibly lost in the process. Once decopressed a lossless copressed iage will always appear exactly the sae as the original uncopressed iage. Lossy copression is based on both irrelevancy and redundancy reductions. It transfors and siplifies the iage in a way that a uch greater copression than the co- pression obtained with lossless approaches can be achieved but this process is by definition irreversible that is it peranently loses inforation. The lossy copressions are suitable for situations where size is ore crucial than quality such as downloading via Internet. The JPEG copression is the best known lossy copression ethodology and the JPEG copression is based on the use of wavelets [5]. Fro the theoretical point of view wavelet copression is capable of both lossless and lossy copression. In fact the wavelet transfor furnishes siplified versions of the iage together with all the inforation necessary to reconstruct the coplete original iage. All the inforation can be kept and encoded as a lossless copressed iage. Alternatively only the ost significant details can be added back into the iage producing a siplified version of the iage. As you ight expect this second application has a uch larger area of interest. We consider here soe applications of the wavelet bases constructed in the previous sections in iage copression. In particular we show soe reconstructions of iages and we focus on a lossy copression procedure. We liit our attention to grayscales iages. Despite the appearance copressing grayscale iages is ore difficult than copressing colour iages. In fact huan visual perception can distinguish ore easily brightness (luinance) than colour (chroinance). Going into details the key steps of our iage copression and reconstruction schee are the following: ) digitize the original iage ) apply the wavelet transfor to represent the iage through a set of wavelet coefficients 3) only for lossy copression: anipulate (i.e. set to zero) soe of the wavelet coefficients 4) reconstruct the iage with the inverse wavelet transfor. The first step that is the digitization of the iage consists in transforing the iage in a atrix of nubers. Since we consider black and white iages the digitized iage can be characterized by its intensity levels or scales of gray which range fro (black) to 55 (white) and its resolution that is the nuber of pixels per square inch. The second step is to decopose the digitized iage into a sequence of wavelet coefficients. The lossy copression step is based on the fact that any of the wavelet coefficients are sall in absolute value so that they can be set to zero with little daage to the iage. This procedure is called thresholding. In practice the ost siple thresholding procedure consists in choosing a fixed tolerance and in doing the following truncation procedure: the wavelet coefficients whose absolute value falls below the tolerance are put to zero. The goal is to introduce as any zeros as possible without losing a great aount of details. The crucial Copyright SciRes. A
L. FATOE ET AL. issue consists in the choice of the threshold. There is not a straightforward way to choose it although the larger the threshold is chosen the larger is the error introduced into the process. In the lossy copression schee only the wavelet coefficients that are non zero after the thresholding procedure are used to build the so called copressed iage that ay be stored and transferred electronically using uch less storage space than the space needed to store the original iage. Obviously this type of copression is a lossy copression since it introduces error into the process. If all the wavelet coefficients are used in the inverse wavelet transfor (or equivalently if we take the threshold equal to zero) and an exact arithetic is used the original iage can be reconstructed exactly. iiking what done in Subsection 4. we copress each test iage by taking its wavelet atrix representation and setting to zero the entries of the atrix wavelet representation whose absolute value is saller than a fixed threshold. After this truncation procedure we perfor an inverse wavelet transfor on the resulting copressed atrix representation and we copare the resulting iage with the original iage both fro the qualitative and the quantitative point of view. As done in Subsection 4. we copute the resulting copression coefficient Ccop as a function of the truncation threshold and we show how the vanishing oents property plays a crucial role in producing copressed iages. Let us note that soeties in the scientific literature the copression coefficient is not defined as done in Subsection 4. but it is defined dividing the original nuber of bytes needed to represent the iage by the nuber of bytes needed to store the copressed iage. However for exaple Wikipedia defines the copression ratio as the size of the copressed iage copared to that of the uncopressed iage leaving undeterined how to easure the size of an iage. In [6] DeVore Jawerth and Lucier have shown that between these two definitions of copression coefficient (i.e. nuber of non zero coefficients and nuber of bytes needed to represent these coefficients) there is a very high correlation (i.e..998). The copression coefficient (ratio) is one of the coon perforance indices of iage copression level. Exaple 3. In this exaple we consider the iage of Figure 5 which is the faous Lena iage. This iage has 56 56 pixels. In Table 4 we report the co pression coefficients Ccop obtained using the Wavelet Bases 3 4 of the Appendix and two different values of the threshold. The relative L -errors ade substituting the original iage with the copressed iage range between 3 and. Figures 6-8 show the copressed Lena iages obtained with the Wavelet Basis 3 of the Appendix for Copyright SciRes. di = 64 = (see Figure 6) and di = 56 = (see Figure 7) and with the Wavelet Basis 4 of the Appendix for di = 5 = (see Figure 8). Table 4. Exaple 3: The copression coefficient Ccop versus ε and di. Ccop Ccop Ccop Ccop Basis 4 di Basis Basis Basis 3 64... - 8.34.36.3 56.83..89 5 7.3 5.4 4.8 64.9.88.97 8 3.63 3.9 3.6 56 9.36.73.9 5 37.46 43.36 4.94 - Figure 5. Lena: Original iage. Figure 6. Lena: Copressed iage with the Wavelet Basis 3 of the Appendix when di = 64 and ε = -. A
L. FATOE ET AL. different behaviour of the Wavelet Bases 3 4 of the Appendix between operator copression and iage ε Table 5. Exaple 4: The copression coefficient C cop versus ε and di. Figure 7. Lena: Copressed iage with the Wavelet Basis 3 of the Appendix when di = 56 and ε = -. - 5 - Ccop Ccop Ccop Ccop di Basis Basis Basis 3 Basis 4 64.4.3.48 8 4.55 4.75 4.3 56.73 3.9.67 5 38.53 46.56 4.5 64 4.77 5.3 5.9 8 4.4 5.8 4.38 56 5.68 56.69 53.63 5 8.5 6.38 7.73 Figure 8. Lena: Copressed iage with the Wavelet Basis 4 of the Appendix when di = 5 and ε = -. Figure 9. Exaple 4: Original iage. Exaple 4. We consider the iage of Figure 9. In this iage are evident three different types of obects: a shape (the black star) a nuber ( 5 ) and an inscription ( black ). This iage has 78 68 pixels. Table 5 shows the copression coefficients Ccop obtained using the Wavelet Bases 3 4 of the Appendix and two different values of the threshold. The relative L -errors ade substituting the original iage with the copressed iage range between 3 and. Figures - show the copressed iages of Figure 9 obtained for di = 8 = 5 and the Wavelet Bases 4 of the Appendix respectively. Figures 3-5 show the copressed iages of Figure 9 obtained for di = 56 = and the Wavelet Bases 3 of the Appendix respectively. The coparisons between Tables 4-5 and Figures 5-5 suggest the following observations and coents. With respect to the copression property there is a Copyright SciRes. Figure. Exaple 4: Copressed iage with the Wavelet Basis of the Appendix when di = 8 and ε = 5 -. A
L. FATOE ET AL. Figure. Exaple 4: Copressed iage with the Wavelet - Basis of the Appendix when di = 8 and ε =5. Figure 4. Exaple 4: Copressed iage with the Wavelet - Basis of the Appendix when di = 56 and ε =. Figure. Exaple 4: Copressed iage with the Wavelet - Basis 4 of the Appendix when di = 8 and ε =5. Figure 3. Exaple 4: Copressed iage with the Wavelet - Basis of the Appendix when di = 56 and ε =. Figure 5. Exaple 4: Copressed iage with the Wavelet - Basis 3 of the Appendix when di = 56 and ε =. copression. In fact in this last case the use of the Wavelet Bases and 3 (ade of piecewise polynoial functions ade of polynoials of degree zero and obtained with the two different criteria proposed in Section ) and the use of Wavelet Bases and 4 (ade of piecewise polynoial functions ade of polynoials of degree one and obtained with the two different criteria proposed in Section ) furnish siilar copression coefficients. This is not really surprising since the iages have a natural discontinuity structure given by the pixels. evertheless the copression coefficients obtained with the Wavelet Basis 3 are always higher than those obtained using the Wavelet Basis and the Wavelet Bases and 4 give copression coefficients higher than those obtained with the Wavelet Bases and 3. Furtherore the use of the Wavelet Bases and 4 sees to have a regularizing effect on the iages that is the iages reconstructed with these two bases appear to have a saller nuber of Copyright SciRes. A