Satellite image deconvolution using complex wavelet packets André Jalobeanu, Laure Blanc-Féraud, Josiane Zerubia ARIANA research group INRIA Sophia Antipolis, France CNRS / INRIA / UNSA www.inria.fr/ariana
Problem statement Efficient representations and basis choice Complex wavelet transform Complex wavelet packet transform Transform thresholding Different methods Parameter estimation Two algorithms : COWPATH 1 and 2 Results Conclusion and future work Satellite image deconvolution / CWP 2
Observation equation Observed images are corrupted : Observed image Y = h * X 0 + N Convolution kernel (known PSF) Original image Noise : White and Gaussian (known variance) Satellite image deconvolution / CWP 3
Problem statement Ill-posed inverse problem [Hadamard 23] existence, unicity, stability of the solution? Inversion noise amplification Small errors of Y high errors of X Satellite image deconvolution / CWP 4
Introduction Monoscale methods [Geman & McClure 85, Charbonnier 97, ] Regularization + edge preservation Find X by minimizing U(X) : U(X) = Y-h*X 2 / 2s 2 + F(X) Data term Non-quadratic regularization term Multiscale methods [Mallat 89, Bijaoui 94, ] Multiresolution analysis wavelets Regularization of classical iterative methods (statistics) (shift invariant wavelet transform thresholding) Multiresolution variational models Satellite image deconvolution / CWP 5
Introduction Filtering after inversion [Donoho, Mallat, Kalifa 99] Non-regularized inversion (Fourier domain) Transform (change the basis) Coefficient thresholding Inverse transform (return to image space) Satellite image deconvolution / CWP 6
Representations for efficient filtering Efficient separation of the signal and the deconvolved noise : - compact reprensentation of the signal - efficient compression of the noise in high frequencies signal noise Image deconvolved without regularization Transform Satellite image deconvolution / CWP 7
Filtering the deconvolved noise Cancel the coefficients corresponding only to the noise Thresholding the coefficients corrupted by noise! The deconvolved noise is colored! In the new basis, the coefficients of the noise transform must be independent enable separate thresholding noise covariance «nearly diagonalized» [Kalifa 99] Satellite image deconvolution / CWP 8
Choice of the basis Compact representation «nearly diagonal» noise covariance The thresholding estimator is optimal [Donoho, Johnstone 94] q T (x) Thresholding function -T T x Coefficient in the new basis Satellite image deconvolution / CWP 9
Algorithm design Rough deconvolution Direct transform thresholding Inverse transform Choice of the basis : compacity diagonalization reconstruction invariance properties Choice of the thresholding function Optimal threshold value? Satellite image deconvolution / CWP 10
Complex wavelets Properties : Shift invariance Directional selectivity Perfect reconstruction Fast algorithm O(N) quad-tree (4 parallel wavelet trees) [Kingsbury 98] filters shifted by ½ and ¼ pixel between trees combination of trees complex coefficients biorthogonal wavelets filter bank implementation Satellite image deconvolution / CWP 11
Quad-tree : 1 st level a 1 d 1 1 a 1A d 1 1A a 1B d 1 1B a 0 (image) d 2 1A da 3 1C 1A d 1 d 2 1B d 3 1C a 1D 1B d 1 1D d 2 1 d 3 1 d 2 1C d 3 1C d 2 1D d 3 1D Non-decimated transform Parallel trees ABCD Perfect reconstruction : mean (A+B+C+D)/4 A B C D A B C D A B C D A B C D A A B A A A A A A A A A A B C B B B A B A A A B B B C B B B D C C C B C B B B C C C D C C C D D D D C C C C D D D D D D D D D D Satellite image deconvolution / CWP 12
Quad-tree : level j different length filters : h o, g o, h e, g e shift < pixel a j,a a j,b a j,c a j,a h e h e h o h o g e g e g o g o 2 e 2 e 2 o 2 o 2 e 2 e 2 o 2 o h e h o h e g e h o g o g e h e g o h o h e g e h o g o g e g o 2 e 2 o 2 e 2 e 2 o 2 o 2 e 2 e 2 o 2 o 2 e 2 e 2 o 2 o 2 e 2 o a j+1,a a j+1,b a j+1,c d 1 a j+1,a j+1,d d 1 j+1,b d 1 d 2 j+1,c d 1 j+1,a d 2 j+1,d j+1,b d 2 d 3 j+1,c d 2 j+1,a d 3 j+1,d j+1,b d 3 j+1,c d 3 j+1,d Satellite image deconvolution / CWP 13
Complex coefficients 4 real subbands d k j,a d k j,b d k j,c d k j,d M Z + = (A - D) + i (B + C) Z - = (A + D) + i (B - C) Z k j+ Z k j- 2 symmetric Complex subbands! The wavelet function is not a complex function. Not exactly complex wavelets! Satellite image deconvolution / CWP 14
Necessity of the packets Complex wavelets : Compact representation Poor representation of the deconvolved noise Image deconvolved without regularization Transform High frequencies not recoverable Satellite image deconvolution / CWP 15
Wavelet packets j=0 j=1 decompose the detail spaces [Coifman et al. 92] j=2 Satellite image deconvolution / CWP 16
Choice of the tree no unicity of the decomposition tree application dependent deconvolution : must adapt to the deconvolved noise energy wavelets packets Limit the growth of the noise variance 0 1/2 Normalized frequency Deconvolved noise power spectrum Satellite image deconvolution / CWP 17
Complex wavelet packets (CWP) decompose the detail spaces of the complex wavelet transform for each tree A,B,C,D f 2 f 2 f 2 f 2 image Original image Transform Satellite image deconvolution / CWP 18
Complex wavelet packets (CWP) Complex Wavelet Packets : Compact representation Nice representation of the deconvolved noise Image deconvolved without regularization Transform High frequencies recoverable Satellite image deconvolution / CWP 19
Frequency plane partition Satellite image deconvolution / CWP 20
Directional selectivity impulse responses real part Complex wavelets Complex wavelet packets Satellite image deconvolution / CWP 21
Directional selectivity impulse responses imaginary part Complex wavelets Complex wavelet packets Satellite image deconvolution / CWP 22
Comparison with real wavelet packets No shift invariance artefacts (mean over translations) Impulse responses No rotation invariance Privileged directions : horizontal / vertical poor texture representation variously oriented (diagonals) Satellite image deconvolution / CWP 23
Example Test image, 512x512 Satellite image deconvolution / CWP 24
Example Complex wavelet packet transform, level 6 Satellite image deconvolution / CWP 25
Transform thresholding Filter only the magnitude enable shift invariance Observed coefficient (CWP transform of the deconvolved image) xˆ ( x) xa ( x) = θt = recall : observed images are corrupted : Y = h * X 0 + N each coefficient of the subband k of the CWP transform is corrupted : x = x + n T Original coefficient unknown (original image CWP transform) Deconvolved noise standard dev. σ k Satellite image deconvolution / CWP 26
Thresholding functions Data : image deconvolved without regularization Fix a thresholding function q T Optimal threshold computation : minimize the risk Minimax risk [Donoho 94] subband modeling Bayesian methods : Coefficient estimation by MAP function q T Models for the subbands : Homogeneous generalized Gaussian Inhomogeneous Gaussian Satellite image deconvolution / CWP 27
Deconvolved noise variance max σ k Estimation of σ k : simulation (CWP transform of a white Gaussian noise) direct computation, with known h and σ σ 2 k =σ 2 i,j [ k ] FFT R [ ] FFT h ij ij min Satellite image deconvolution / CWP 28 2 Impulse response for subband k
Optimal risk Impose a thresholding function q T Minimize the risk of the thresholding estimator (Xˆ,X ) = E Xˆ = E θ m 2 2 r 0 X0 T( x) ξ Theoretical results [Donoho, Johnstone 94] not useful in practice (too large threshold) [Kalifa 99] Subband modeling (Generalized Gaussian [Mallat 89]) model parameters estimation Satellite image deconvolution / CWP 29
Subband modeling Generalized Gaussian : P( ξ) = Z 1 α,p Experimental study : e Histograms : original image CWP transform subbands ξ/ α p α,p model parameters Im Re Im Re Satellite image deconvolution / CWP 30
Subband modeling parameter estimation Bayesian methods No arbitrary choice of thresholding function estimate x by Maximum A Posteriori (MAP) Max P P P ( x ξ) xˆ = θ ( ξ x) = Max P( x ξ) P( ξ) ( ξ) e ( x) e x ξ ξ/ α 2 / 2 σ 2 p Classical thresholding functions for particular values of p k xˆ = Minξ x ξ 2 /2σ Estmation of the model parameters α,p : Maximum Likelihood, 2 + ξ/ α p k =1 p k =0,2 p Satellite image deconvolution / CWP 31
COWPATH 1.0 «COmplex Wavelet Packets Automatic Thresholding» Satellite image deconvolution / CWP 32
Inhomogeneous Gaussian Model Insufficiency of homogeneous models (constant areas / edges / textures) Parameter s ij : depends on the location of the coefficient ξ ij P ( ξ ) ij = 2π 1 s 2 ij e ξ ij 2 /2s 2 ij! Estimation problems for the model parameters s ij (not enough data / number of unknown parameters!) Hybrid method : parameter estimation from a good approximation of the original image Complete Data Maximum Likelihood Satellite image deconvolution / CWP 33
COWPATH 2.0 Noise variances computation Rough deconvolution CWP thresholding (MAP) CWP -1 Xˆ Y Deconvolution (Non-quadratic j variational model) CWP Parameter estimation (ML) Computation time 3,5s on PII 400 (quadratic ϕ) Satellite image deconvolution / CWP 34
Nîmes, original Satellite image image 512 x deconvolution 512 French / CWP Space Agency (CNES) 35
Nîmes, Satellite blurred image and deconvolution noisy image / CWP(σ~1.4) 36
Satellite Nîmes, image COWPATH deconvolution 1 result / CWP 37
Satellite Nîmes, image COWPATH deconvolution 2 result / CWP 38
sharp and regular diagonal edges sharp textures smooth homogeneous areas Nîmes, Satellite COWPATH image deconvolution 2 result /-CWP enlarged 39
Nîmes, deconvolution Satellite using image real deconvolution wavelet packets / CWP [Kalifa, Mallat 99] 40
Nîmes, deconvolution using Satellite RHEA image (non-quadratic deconvolution ϕ/ function CWP regularization [Jalobeanu 98]) 41
Nîmes, deconvolution Satellite using image a deconvolution quadratic regularization / CWP (~Wiener) 42
X 0 Y ϕ func COWPATH 1 COWPATH 2 Wiener Real packets Satellite Result image deconvolution comparison/ CWP 43
Results / Astronomy (Hubble PSF) X 0 Deconvolution using quadratic regularization (~Wiener) Y Deconvolution using CWP Satellite image deconvolution / CWP 44
X 0 COWPATH 2 Wiener Y Satellite image deconvolution / CWP 45
Conclusion and future work Providing better results by Adapting the structure of the tree to the problem taking into account images and PSFs Better subband modeling inhomogeneous Generalized Gaussian model? More accurate data term noise transform coefficients not fully independent Taking into account the interactions between scales Hidden Markov Trees [Nowak et al. 98] Hybrid method : DEPA [Jalobeanu et al. 00] Result of COWPATH estimation of the parameters of an adaptive regularizing model Satellite image deconvolution / CWP 46