IEEE Transactions on Signal Processing, accepted October 2005, in press

Transcription

1 IEEE Personal use of this material is permitted However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE IEEE Transactions on Signal Processing, accepted October 5, in press

2 The aussian Transform of distributions: definition, computation and application Teodor Iulian Alecu*, Sviatoslav Voloshynovskiy and Thierry Pun Abstract This paper introduces the general purpose aussian Transform of distributions, which aims at representing a generic symmetric distribution as an infinite mixture of aussian distributions We start by the mathematical formulation of the problem and continue with the investigation of the conditions of existence of such a transform Our analysis leads to the derivation of analytical and numerical tools for the computation of the aussian Transform, mainly based on the Laplace and Fourier transforms, as well as of the afferent properties set (eg the transform of sums of independent variables) The aussian Transform of distributions is then analytically derived for the aussian and Laplacian distributions, and obtained numerically for the eneralized aussian and the eneralized Cauchy distribution families In order to illustrate the usage of the proposed transform we further show how an infinite mixture of aussians model can be used to estimate/denoise non-aussian data with linear estimators based on the Wiener filter The decomposition of the data into aussian components is straightforwardly computed with the aussian Transform, previously derived The estimation is then based on a two-step procedure, the first step consisting in variance estimation, and the second step in data estimation through Wiener filtering To this purpose we propose new generic variance estimators based on the Infinite Mixture of aussians prior It is shown that the proposed estimators compare favorably in terms of distortion with the shrinkage denoising technique, and that the distortion lower bound under this framework is lower than the classical MMSE bound Index Terms aussian mixture, aussian Transform of distributions, eneralized aussian, denoising, shrinkage I INTRODUCTION aussian distributions are extensively used in the (broad sense) signal processing community, mainly for computational benefits For instance, in an estimation problem aussian priors yield quadratic functionals and linear solutions In ratedistortion and coding theories, closed form results are mostly available for aussian source and channel descriptions [] However, real data is most often non aussian distributed, This work is supported in part by the Swiss NCCR IM (Interactive Multimodal Information Management, the EU Network of Excellence Similar ( and the SNF grant N PP-68653/ T I Alecu, S Voloshynovskiy, T Pun are with the Computer Vision and Multimedia Laboratory, University of eneva, 4 Rue énéral-dufour, 4 eneva, Switzerland (corresponding author: TeodorAlecu@cuiunigech, phone: +4()37984; fax: +4()379778) and is best described by other types of distribution (eg in image processing, the most commonly used model for the wavelet coefficients distribution is the eneralized aussian distribution [6]) The goal of the work presented in this paper is to describe non-aussian distributions as an infinite mixture of aussian distributions, through direct computation of the mixing functions from the non-aussian distribution In a related work [4], it was proven that any distribution can be approximated through a mixture of aussian up to an arbitrary level of precision However, no hint was given by the author on how to obtain the desired mixture in the general case In the radar community the problem of modeling non- aussian radar clutters led to the theory of Spherically Invariant Random Processes (SIRP) [3], which aimed at generating non-aussian multivariate distributions using univariate aussian distributions However, the proposed solutions bypassed the mixing function ([] and references therein), performing direct computation of the multivariate non-aussian distribution from the marginal probability distribution function and the desired covariance matrix In [5], an analytical formula is given for an infinite mixture of aussians equivalent to the Laplacian distribution, and used in a source coding application Unfortunately, no generalization was attempted by the authors The work presented here has the roots in their proof and extends the concept through the introduced aussian Transform of distributions, which permits straightforward derivation of exact mixing functions for a wide range of symmetric distributions, including the eneralized Cauchy and the eneralized aussian distributions In order to exemplify the possible usage of the transform we consider then the problem of estimation/denoising, motivated by works such as [9] and [], where, in order to preserve the computational advantages provided by aussian modelling, non-aussian distributions are approximated with finite aussian mixtures obtained through iterative numerical optimization techniques Relying on the aussian Transform, we propose new generic algorithms for denoising non- aussian data, based on the description of non-aussian distributions as an Infinite Mixture of aussians (IM) The rest of the paper is divided in three main sections In Section II we define the aussian Transform, analyze its existence, investigate its properties and derive the mathematical tools for analytical and/or numerical computation In Section III we exemplify both the transform for some typical distributions such as eneralized aussian and eneralized Cauchy, and some of the properties deduced in Section II The last section tackles the problem of

3 estimation/denoising and proposes new aussian Transform based denoising schemes, exemplifying them for the eneralized aussian Distribution (D) family The obtained results are compared in terms of distortion with the state-of-the-art denoising method of Moulin and Liu [7] II THE AUSSIAN TRANSFORM OF DISTRIBUTIONS A Definition and existence We consider a generic symmetric continuous distribution p(x) As we are aiming at representing it through an infinite mixture of aussians, we can safely disregard the mean, and assume for simplicity reasons that p(x) is zero-mean We are looking for an integral representation in the form: ( ) N ( x ) d = p( x), () where N ( x ) is the zero-mean aussian distribution: x N ( x ) = e, π and ( ) is the mixing function that should reproduce the original p(x) We can now introduce the aussian Transform Definition : aussian Transform The direct aussian Transform is defined as the operator which transforms p( x ) into ( ), and the Inverse aussian Transform - is defined as the operator which maps ( ) back to p( x ) : : p( x) ( ); : ( ) p( x) Obviously, - is simply given by (): ( ( )) = ( ) ( x ) d = p( x) N () The aussian Transform of a distribution exists if, given p( x ), a mixing distribution ( ) can be found such as to comply with () This can be summarized in three conditions Condition For a given p(x), a function ( ) defined according to () exists Condition This function is non-negative Condition 3 Its integral ( ) d is equal to The last condition is a consequence of Condition Indeed, if ( ) exists, then integrating both sides of () with respect to x and inverting the integration order one obtains: ( ) N ( x ) d dx = p( x) dx, x dx d = N Finally, since ( x ) N is a distribution: ( ) d = In order to investigate Condition, the existence of ( ), we perform the following variable substitutions: s = x and t = Since p(x) is symmetric, it can be rewritten as: p( x) = p( x ) = p( s) The left hand side of () transforms to: ( ) ( ) st N x d = e dt tt π t According to the definition of the Laplace Transform L [], equation () finally takes the form: L = p( s) (3) tt π t Thus, ( ) is linked to the original probability distribution p(x) through the Laplace Transform and can be computed using the Inverse Laplace Transform L - The direct aussian Transform is therefore given by: = π ( p( x) ) = ( p( s) )() t L, (4) t and the Inverse aussian Transform can be computed as: ( ( )) = L ( s) (5) tt π t s= x Consequently, the existence of the aussian Transform is conditioned by the existence of the Inverse Laplace Transform of p( s ) Using the general properties of the Laplace Transform, it is sufficient [] to prove that the limit at infinity of s p( s) is bounded, or equivalently : lim x x p x < (6) The above condition is satisfied by all the distributions from the exponential family, as well as by all the distributions with finite variance The equation (4) allows for straightforward identification of aussian Transforms for distributions whose Laplace Transforms are known, by simply using handbook tables Unfortunately, the condition (6) does not guarantee compliance with the Condition : non-negativity As it is rather difficult to verify a priori this constraint, the test should be performed a posteriori, either analytically or numerically However, by imposing the non-negativity constraint in (), one can see that it is necessary to have a strictly decreasing distribution function p(x) on positive x, as p(x) is described by a sum of strictly decreasing zero-mean aussians functions From the same reason, but not related to non-negativity, the distribution function p(x) should also be of class C B Properties of the aussian Transform Property : Uniqueness If two functions and

4 ( ) exist such that ( ( )) = p( x) ( ( )) = p( x) ( ) ( ) and then they are identical = Conversely, if two distribution p x have the same aussian probabilities p x and Transform ( ) then they are identical Proof The second proposition of the uniqueness property is proved straightforwardly using () In what concerns the first proposition, it is a direct consequence of the uniqueness of the Laplace Transform [], which states that if two functions have the same Laplace transform, then they are identical It is then sufficient to observe that is computed as a Laplace Transform (5) Property : Mean Value (variance conservation) The mean value of the aussian Transform is equal to the variance of the original distribution x p( x) dx = ( ) d (7) x p x dx = x x d dx = Proof N ( ) N = x x dx d = d qed Property 3: Value at infinity If lim x x p( x) < then the aussian Transform tends to when tends to infinity: lim ( ) = (8) Proof We use the initial value theorem for the Laplace Transform [] and the direct formula (3): limt = lims s p s = lim x x p x, tt π t then lim < lim = 3 ( )( ) ( ) We study now the influence of basic operations on the aussian Transform, such as scalar multiplication and addition of independent variables Property 4: Scaling If and are random variables such that = α and ( p ( x) ) =, then the aussian Transform corresponding to the scaled variable is a scaled version of : = (9) α α Proof We begin by observing that the distribution probability of the random variable can be expressed as: x p ( x) = p α α Then we compute the Inverse aussian Transform of the expression proposed in (9) using (): ( ) = N x d ; α α x α * α ( ( )) e d α α α = π α x ( ( )) = p p = x α α Finally using the uniqueness property we conclude that the aussian Transform ( p ( x) ) is unique and given by (9) Property 5: Convolution If and are two independent p x = and random variables with ( p ( x) ) = ( ) ( ), then the aussian Transform of their sum is the convolution of their respective aussian Transforms (the result can be generalized for the sum of multiple variables): p + ( x) = () Proof Consider the random variable = + Since exists, is a random aussian variable with variance distributed according to the distribution probability Similarly, is a random aussian variable with variance distributed according to Then is also a random aussian variable with variance = + But and are independent variables drawn from and It follows that is a random variable described by the probability distribution =, qed Corollary If and are independent random variables and is aussian distributed p ( x) = N ( x ), then the aussian transform of their sum is a shifted version of :, < + = () ( ), C Numerical computation The computation of the aussian Transform for distributions not available in handbooks is still possible through the complex inversion method for Laplace Transforms known as the Bromwich integral [3]: ( ) ε + i ut L f s = f ( u) e du π i, () ε i where ε is a real positive constant satisfying ε > sup( Re( poles ( f ))) and u is an auxiliary variable The equation () can be rearranged as a Fourier Transform, allowing the use of the numerous numerical and/or symbolical packages available ( ω is the variable in the Fourier space): εt L ( f ( s) ) = e F ( f ( ε + iω )) (3) Very often p(x) has no poles, being a continuous and bounded

5 function, and in this case it might be very practical to evaluate (3) in the limit case ε Using (4) and (3): π ( ) = F ( p( iω) )() t (4) t = When the original distribution is only numerically known, approximation of ( ) is still possible either through analytical approximations of p(x), followed by (4) or (4), or through solving the inverse problem yielded by (), or through real inversion of the Laplace transform using (4) The accuracy of the obtained transforms can then be assessed using an appropriate metric or distance such as Kullback- Leibler However, the abovementioned approximation methods and the accuracy study constitute stand-alone problems out of the scope of the present paper C b p x b = p s b = π + π + b C ( ) ; ( ) C ( ) b x b s b = (7) p x b e π The results (5), (6) and (7) are plotted in Fig To exemplify the Corollary to the Convolution Property, consider now Cauchy data contaminated with independent additive white aussian (AW) noise Then the aussian Transform of the measured data (Fig ) is a shifted version of the original data transform () III EAMPLES OF AUSSIAN TRANSFORMS A Analytic aussian transforms The most obvious and natural example is the aussian Transform of a aussian distribution N ( x ) One would expect to have (with δ the Dirac function): ( N ( x )) = δ ( ) (5) Proof It is trivial that (5) verifies the equation () Then, by using the uniqueness of the aussian Transform one can conclude that the aussian Transform of a aussian distribution N ( x ) is a Dirac function centered in The convolution property () can now be used to prove a well known result in statistics: the sum of two independent aussian variables, with respective probability laws x N x, is another aussian variable with N ( ) and ( ) probability distribution ( x + ) N (the extension to non-zero mean distributions is trivial) Proof: ( p ( x) ) = δ ( ); ( ) p x = δ p x = p x * p x = δ + ( + ) ( ) Then, inverting the aussian Transform: p ( ( )) + = δ + = N x + Similarly to (5), it is possible to compute the aussian Transforms of other usual symmetric distributions using the Laplace transform tables [] We exemplify with the Laplacian and Cauchy distributions Laplacian distribution: L λ λx L λ λ s p ( x λ) = e ; p ( s λ) = e (6) λ L λ ( p ( s )) λ = e As mentioned, the result (6) was already proven in [5] Cauchy distribution: Figure aussian Transforms of the Laplacian, aussian and Cauchy distributions Figure Shift of the aussian Transform following data contamination with AW noise The previous analytical results will be generalized in subsection B through numerical computations B Numerical computation of aussian Transforms This part illustrates the numerical computation of the aussian Transform through (4) for the eneralized aussian and eneralized Cauchy distributions families ) eneralized aussian Distribution (D) The D family is described by an exponential probability density function with parameters and : η ( ) x p ( x, ) = e η, (8) Γ ( ) where η( ) 3 ( ) ( ) = Γ Γ For = the D particularizes to the Laplacian distribution, while for = one obtains the aussian distribution The aussian Transform of the eneralized aussian distribution can not be obtained in analytical form using (4) However, it does exist (6) and can be calculated numerically through (4): ( ) π ( ) = F ( ω), p i t, t = η ( ) η ( ) ( ω) ( ) ( ω) p i = e i, Γ

6 The aussian Transforms for ranging from 5 to with fixed = are plotted in Fig 3 The transforms evolve from a Dirac-like distribution centered on for small to exponential for =, then Rayleigh-like for =, bellshaped for = 5 and again Dirac-like centered at for = As expected, the aussian Transform of the Laplacian distribution ( = ) is exponential (6) Figure 3 aussian Transforms of D Unfortunately, the real part of the complex probability function diverges for periodical values of, which impedes the computation of the transform through this method for > However, real data (in transform domains such as discrete cosine and wavelet) is mostly confined to << [6] ) eneralized Cauchy Distribution (CD) The CD probability density function is given by Γ ( ν + ) ν C b px x ν, b =, ν + 5 πγ ( ν) ( b + x ) and it particularizes to the Cauchy distribution for ν = 5 Its aussian Transform can be computed through: ν b Γ ( ν + 5) ν 5 ν, b( ) = ( F (( b + iω) ) ) Γ( ν) t = Corresponding plots are given in Fig 4 Figure 4 aussian Transforms of CD As a remark, the variance of the Cauchy distribution being infinite, its aussian Transform has infinite mean value (7) The CD family possesses finite variance only for ν > IV DENOISIN We are interested in the generic denoising problem of estimating the original scalar data x from the measured data y, degraded by additive noise z: y = x+ z (9) We assume that the noise random variable is zero-mean aussian distributed with variance : z p ( z) = e () π We further assume that the probability density function p (x), which describes the source random variable, is a zeromean symmetric distribution, and that its aussian Transform ( ) = ( p ( x) ) exists For exemplification of the proposed techniques we consider p (x) to be the eneralized aussian distribution (8) We base our estimation on the Maximum a Posteriori (MAP) principle [4]: xˆ = arg max x p Y ( x y), which yields for iid zero-mean aussian data and noise (with variances and ) the classical Wiener filter: xˆ = y () + We also note that the estimate () is also the Minimum Mean Square Error (MMSE) estimate for iid zero-mean aussian data and noise: xˆmmse = arg min ( ˆ x xx) p Y ( x y) dx x () xˆ = xp x y dx MMSE Y x According to the IM model, each data sample is interpreted to be drawn from a aussian distribution with variance distributed accordingly to the aussian Consequently, using the prior Transform knowledge of ( ), we can estimate the variance from y and use Wiener filtering () Thus, our estimator is given by: ( y) xˆ = y (3) y + The initial problem is now reduced to a variance estimation problem For this purpose, we begin by introducing the cumulative estimation technique A Cumulative Estimation We are looking for an estimator that provides variance estimates consistent with the original data distribution, in the sense that the description of the data as a aussian mixture should be identical to the non-aussian original description We set the above assertion as a consistency principle

7 Principle : Consistency The probability distribution function of the variance estimate should be identical to the aussian Transform of the prior distribution p (x) Moreover, since the distributions p (x), p (z) are symmetric and zero-mean, we assume an estimator of the form: ˆ ( y) = ξ M ( M ( y) ), where ξ M denotes the consistent variance estimator depending on the symmetric real non-negative function: + M: ; M( y) = M( y), + where and denote real, respectively real nonnegative numbers As an example, a possible M function is the absolute value of y We further assume that this estimator is monotonically increasing with the function M: y < y ξ M y M M M M < ξ M y (4) ( ) ( ) If one chooses the M function as the absolute value y, this simply means that if the absolute value of y is larger, then the estimated variance ˆ ( y) is also larger If the variables are vectors (eg local estimation paradigm) one can similarly use as the M function the variance of the measurement vector, and (4) would mean that if the variance of the measurement is larger, then the estimated variance is also larger The assumption in (4) implies that the cumulative distribution function of the variance estimates is equal to the cumulative distribution function of M: p( ˆ ˆ ( y) ) = p( M M ( y) ) (5) Considering continuous probability density functions and incorporating the consistency condition (the distribution of the estimates should be identical to ), (5) can be rewritten as: M PM ( M( y) ) = P ( ξ ( M( y) )), M( y) x (6) P M y = p u du; P = u du, M ( ) M where p M is the probability density function describing M ( y), and P M and P are cumulative probability functions, as defined in (6) Then the estimator we are looking for is simply given by: M ( M( y) ) = P PM ( M ( y) ), (7) ξ where P is the mathematical inverse of the function P : ( if P ) then = u P u = We denote (7) as the cumulative estimator, simply because it uses cumulative probability functions for the estimation We further infer that the cumulative estimator is robust with respect to monotonically increasing transformations of M Theorem If M and M are two symmetric real nonnegative functions M, M + :, and if M( y) < M( y) M( y) < M ( y), then the associated cumulative estimators are identical: M ( ( y) ) = ξ ( ( y) ) M ξ M M Proof Since the inequalities are preserved (theorem hypothesis), then the cumulative distribution functions P M and P M are equal: p( M M( y) ) = p( M M( y) ) then M ( y) M ( y) P M y = p u du = p u du = P M y ( ) M M M M ( ) The proof is ended by corroborating the above equality with the cumulative estimation formula (7) B IM cumulative estimator The cumulative estimator (7) can be directly used in (3) by setting the M function as M ( y) = y, and the IM cumulative estimator (IM CE) is simply: ˆ _ CE ( y) xˆ IM _ CE = y with ˆ _ CE ( y) + (8) y ˆ _ CE ( y) = P ( PM( y) ), PM( y) = py ( u) du As and are assumed to be independent, p Y (y) can be computed as the convolution of p (x) and p (z): py = p p In the general case the operations in (8) have to be performed numerically However, if the distribution p (x) is Laplacian (D with = ), its aussian Transform is (6), (8): ( ), = e, (9) and the cumulative estimator from (8) reduces to: _ CE, = ( y) = ln ( P M ( y) ) (3) As tends to, the aussian Transform of the D tends asymptotically to a Dirac function centered at (5), and the estimator (8) to the classical Wiener filter () In order to asses the performances of the proposed estimators we measure the accuracy of the estimate according to the L norm: e( x, xˆ) = ( x xˆ) The global distortion induced by an estimator is therefore: D = e( x, xˆ ) p ( x) p ( z) dxdz (3) We compare our estimator in terms of distortion with the estimators known as shrinkage functions from [7] The idea of shrinking originates from [8], but [7] provides both improved results and statistical interpretation of the functions In fact, the shrinkage functions from [7] are the result of direct MAP estimation, which in the case of D data and aussian noise yields ˆx as the solution of the equation in x: y = x η( ) x (3) Equation (3) has analytical solutions for = 5 (cubic), = or (linear) and = 5 (quadratic) The resulting shrinkage functions have thresholding characteristics for

8 and yield the Wiener filter for = The equivalent IM CE functions do not present thresholding characteristics for any, behaving in continuous manner raphic illustration of the shrinkage functions and of the equivalent IM CE functions obtained via (8) are presented in Fig 5 of separable IM components In fact, for a given y, the MMSE estimator is unique (injective estimation), which is not the case for the ideal estimator that would lead to the proposed IM lower bound, and that would yield, for the same value of y, different estimates for each isolated aussian component Figure 5 Shrinkage functions and IM CE estimation curves One can also compute the lower bound on the distortion induced by an estimation procedure based on signal decomposition into an IM Assuming that the aussian components of the IM are separable, and that the variance estimate is constant over each separate component, the equation () is the MMSE estimate for each component The bound is attained when the estimator "knows" the aussian component of the mixture that has generated the estimated sample The ensuing distortion lower bound is: Dlower bound = ( ) d, (33) + where the ratio + is the theoretical distortion of the MMSE filter () We also compare our results with the MMSE distortion bound, given by the MMSE estimator (), which can be rewritten as follows: xˆ MMSE = xp( x y) dx = x p( x y, ) p( y) d dx x x Using Bayes rule and switching integrals, one obtains: p( y ) p( ) xˆ MMSE = xp ( x y, ) dx d x p( y) Considering aussian noise the first parenthesis yields the MMSE estimate (), and by integrating the aussian p y one finds: Transform and developing xˆ p y and N( + ) ( y + ) ( ) d y d + MMSE = N y (34) The MMSE distortion bound is then computed by plugging the result (34) into (3) The IM lower bound (33) and the MMSE bound (34) are superposed in Fig 6, which displays the distortion of the estimation (3) as a function of the signal to noise ratio SNR = log (the variance of the D family is given by, see the moments method from [6] and equation (8)) One may notice that the IM lower bound is lower than the MMSE bound, a result due to the assumption Figure 6 Distortion for D data as a function of SNR (db) A considerable difference in performance in the favor of IM CE with respect to the shrinkage functions is observed for = 5 The difference tends to decrease for larger, and inverts for = 5, albeit with a very small relative module However, the IM CE estimator being unique for a given y, the IM CE distortion curve necessarily remains above the MMSE bound, as for any other injective estimator All curves tend to the distortion of the Wiener filter when approaches, plotted dashed-dotted in Fig on the graph corresponding to = 5 C IM ML and MAP variance estimation The simplest way of obtaining the variances in a statistical framework is through Maximum Likelihood (ML) estimation: ˆ _ ML = arg max p( y ), (35) which for zero-mean independent aussian source and noise data yields the solution: ˆ _ ML = max ( y,) (36) If the variance distribution were known, one could improve (36) through MAP estimation Using the aussian Transform the MAP variance estimate is: ˆ _ MAP = arg max ( ) p( y ) (37)

9 enerally, the expression (37) requires numerical maximization, but an analytical form exists for = and aussian noise (using (9) and differentiation of (37)): y ˆ _ MAP, = = max, (38) 8y + + One would expect the IM MAP estimator to behave similarly to the original shrinkage functions of Moulin and Liu, as it applies the MAP principle two times (37), (3) to obtain an indirect estimate, whereas the shrinkage functions are the result of direct MAP estimation (3) This is confirmed in Fig 7, which compares the equivalent shrinkage functions of the IM ML and IM MAP estimators with the IM CE curve and with the Moulin and Liu curve for = Figure 7 IM ML and MAP equivalent shrinkage functions Unlike the IM CE estimator, the two proposed IM ML and MAP estimators do present thresholding characteristics, and, as expected, the equivalent IM MAP curve behaves very similarly to the original shrinkage curve We tested empirically the MAP (38) estimator by generating D eneralized aussian data of length N= 6 with = 5 and =, adding independent white aussian noise, and performing denoising according to () The source and noise probability densities were assumed to be known, and the corresponding distortion was computed as the mean square error (Fig 8) We also tested the cumulative estimator (8) and the shrinkage function (3) Figure 8 Empirical distortion curves As expected, empirical results reproduce the theoretical results obtained for the scalar estimators (Fig 6) The IM MAP results are close, but above the direct MAP estimation of the shrinkage functions for = (this result was predictable from the curves in Fig7), however they notably improve on the Moulin and Liu results for = 5 We expect the IM MAP curves to remain above the Moulin and Liu curves for, both falling slowly down to the aussian-aussian curve in Fig6 as the IM MAP estimator tends to the Wiener filter for = While the IM MAP estimator does not outperform the cumulative estimator, nor the shrinkage estimator for, and while the IM CE estimator does not outperform the classical MMSE estimate, the linear nature of the estimator (3), coupled with the simple analytical forms (38) and (3) for =, allowed the successful use of the IM MAP and IM CE estimators in the highly underdetermined EE (electroencephalogram) inverse problem [5] with Laplacian prior, where a direct application of the MAP principle would lead to an underdetermined quadratic programming problem, and where a direct application of the MMSE principle would lead to highly expensive computations V CONCLUSION AND FUTURE WORK We introduced in this paper the aussian Transform, which allows for the representation of symmetric distributions as an infinite aussian mixture The scope of applicability of the aussian Transform is potentially very broad, from denoising and regularization to filtering, coding, compression, watermarking etc However, an extension of the concept to non-symmetric distribution would be required for some specific applications Further investigation of the existence conditions, especially non-negativity, is also necessary Finally, one would need adapted numerical packages (most likely based on existing Laplace and Fourier transform computational packages) for the computation of aussian Transforms of both analytically and numerically defined distributions We also presented in this paper various denoising algorithms based on the Infinite Mixture of aussians model Their application to the denoising of eneralized aussian data showed significant improvements in terms of distortion, both theoretically and empirically, with respect to the shrinkage functions of Moulin&Liu Also, the lower bound under the proposed IM estimation framework is lower than the classical MMSE bound However the potential of IM based denoising is not yet fully explored, as the lower bound presented in this paper is still below current results The estimation methods proposed may reach this lower bound in the case where the data is generated locally from the same aussian component, which would allow for exact variance estimation ACKNOWLEDMENT The authors thank Dr Oleksiy Koval for careful reading

10 and fruitful discussions The authors also thank anonymous reviewers for valuable comments and suggestions REFERENCES A D Poularikas, S Seely, Laplace Transforms The Transforms and Applications Handbook: Second Edition Ed Alexander D Poularikas Boca Raton: CRC Press LLC, Arfken, Mathematical Methods for Physicists, 3rd ed Orlando, FL: Academic Press, pp , Arfken, "Inverse Laplace Transformation" 5 in Mathematical Methods for Physicists, 3rd ed Orlando, FL: Academic Press, pp , R Wilson, MMM: Multiresolution aussian Mixture Models for Computer Vision, in IEEE International Conference on Pattern Recognition, Barcelona, Spain, September 5 A Hjørungnes, J M Lervik, and T A Ramstad, Entropy Coding of Composite Sources Modeled by Infinite Mixture aussian Distributions, in Proc 996 IEEE Digital Signal Processing Workshop, Loen, Norway, September S Mallat, A Theory for Multiresolution Signal Decomposition: The Wavelet Representation, in IEEE Transactions on Pattern Analysis and Machine Intelligence Volume, Issue 7 (July 989) p P Moulin and J Liu, "Analysis of Multiresolution Image Denoising Schemes Using eneralized - aussian and Complexity Priors", IEEE Trans Info Theory, Special Issue on Multiscale Analysis, Vol 45, No 3, pp 99-99, Apr D L Donoho and I M Johnstone, "Ideal Spatial Adaptation Via Wavelet Shrinkage", Biometrika, Vol 8, pp , A Bijaoui, "Wavelets, aussian mixtures and Wiener filtering'', Signal Processing, Vol8, No4, April, pp 79-7 Benaroya, L; Bimbot, F; "Wiener-based source separation with HMM/MM using a single sensor'', Proc 4th Int Symp on Independent Component Anal and Blind Signal Separation, April 3, Nara, Japan T M Cover, J A Thomas, "Elements of Information Theory", John Wiley & Sons Inc, 99 M Rangaswamy, D Weiner, A Ozturk, "Non-aussian Random Vector Identification Using Spherically Invariant Random Processes", in IEEE Transactions on Aerospace and Electronic Systems, vol 9, No, January E Conte and M Longo, "Characterization of radar clutter as a spherically invariant random process", in IEEE Proceedings, Pt F (Communication, Radar and Signal Processing), 34, (987), MBertero, PBoccacci, "Introduction to Inverse Problems in Imaging", IOP Publishing, Bristol, T I Alecu, P Missionnier, S Voloshynovskiy, P iannakopoulos and T Pun, "Soft/hard focalization in the EE inverse problem", in IEEE Workshop on Statistical Signal Processing (SSP'5), Bordeaux, France, July, 7-5