Deterministic sampling masks and compressed sensing: Compensating for partial image loss at the pixel level

Transcription

1 Deterministic sampling masks and compressed sensing: Compensating for partial image loss at the pixel level Alfredo Nava-Tudela Institute for Physical Science and Technology and Norbert Wiener Center, University of Maryland, College Park Joint work with John J. Benedetto Department of Mathematics and Norbert Wiener Center, University of Maryland, College Park

2 Overview Problem statement Image representation concepts Image compression basics Sparsity is the key, l 0 -minimization, OMP Image compression revisited Imagery metrics Solving our problem: deterministic sampling masks and compressed sensing Solving our problem: results 1

3 Problem statement Image: Satellite Imaging Corporation Antigua 2

4 Problem statement JPEG, JPEG 2000 Sampling Compression Representation 3

5 Image representation concepts[1] 4

6 Image representation concepts[1] An image is Pixel = I[n 1,n 2 ] = intensity, brightness, at [n 1,n 2 ] n 2 n 1 0 n 1 < N 1, 0 n 2 < N 2 5

7 Image representation concepts[1] I[n 1,n 2 ] I[n 1,n 2 ] {0,, 2 B -1}, or {-2 B-1,, 2 B-1-1}, where I[n 1,n 2 ] = round(2 B J[n 1,n 2 ]) and J[n 1,n 2 ] [0,1) or [-½, ½) B is the depth of the image 6

8 Image representation concepts [1] 7

9 Image compression[1] 512 x 512 x 8 x 3 = 6,291,456 bits 8

10 Image compression[1] JPEG, JPEG

11 Image compression[1] 1) Partitioning of the image I in sub-images [1] D. S. Taubman and M. W. Mercellin, JPEG 2000: Image Compression Fundamentals,! Standards and Practice, Kluwer Academic Publishers,

12 Image compression[1] 1) Partitioning of the image I in sub-images 2) Transform sub-images to exploit correlations within them [1] D. S. Taubman and M. W. Mercellin, JPEG 2000: Image Compression Fundamentals,! Standards and Practice, Kluwer Academic Publishers,

13 Image compression[1] 1) Partitioning of the image I in sub-images 2) Transform sub-images to exploit correlations within them 3) Quantize and encode [1] D. S. Taubman and M. W. Mercellin, JPEG 2000: Image Compression Fundamentals,! Standards and Practice, Kluwer Academic Publishers,

14 Image compression 13

15 Sparsity is the key Cn u rd ths? vs Can you read this? 14

16 Sparsity is the key 15

17 Sparsity The l 0 norm : x 0 = # {k : x k 0} 16

18 l 0 -minimization ~ sparse solution (P 0 ): min x x 0 subject to Ax - b 2 = 0 17

19 l 0 -minimization ~ sparse solution (P 0ε ): min x x 0 subject to Ax - b 2 < ε 18

20 l 0 -minimization ~ sparse solution (P 0ε ): min x x 0 subject to Ax - b 2 < ε Solving (P 0 ε ) is NP-hard![2] Is there any hope? [2] B. K. Natarajan, Sparse approximate solutions to linear systems," SIAM Journal on Computing, 24 (1995), pp

21 Finding sparse solutions:omp Orthogonal Matching Pursuit algorithm: [3] [3] A. M. Bruckstein, D. L. Donoho, and M. Elad, From sparse solutions of systems of! 20 equations to sparse modeling of signals and images, SIAM Review, 51 (2009), pp "

22 Finding sparse solutions:omp Orthogonal Matching Pursuit algorithm: 21

23 Finding sparse solutions:omp Orthogonal Matching Pursuit algorithm: 22

24 Finding sparse solutions:omp Orthogonal Matching Pursuit algorithm: 23

25 Image compression T = T ε = OMP(A, -,ε), T = A 24

26 We need a matrix A DCT DCT2 25

27 We need a matrix A 26

28 We need a matrix A A = (c 3 ( ) c 3 ( ) c 3 ( ) c 3 ( )) 27

29 Compressing a test image Image c 3 ( ) = b x 0 = T b = OMP(A, b,ε) = c 3-1 (b ) b = T x 0 = A x 0 28

30 Compressing a test image ~? b - b 2 < ε But what does that mean visually? How many bits were used? 29

31 Imagery metrics Peak Signal-to-Noise Ratio (PSNR), measured in db: PSNR(X,Y) = 20 log 10 (MAX B / MSE), with MAX B = 2 B -1, and MSE = i,j [X(i,j) - Y(i,j)] 2 /nm. In our case, n = m = 512, and B = 8, i.e. MAX B =

32 Imagery metrics Structural Similarity (SSIM), and Mean Structural Similarity(MSSIM) indices: [4] [4] Z. Wang, A.C. Bovik, H.R. Sheikh and E.P. Simoncelli, Image quality assessment: from error! visibility to structural similarity, IEEE Transactions on Image Processing," vol.13, no.4 pp , April

33 Imagery metrics The normalized sparse bit-rate is nsbr(i,a,ε) = x j 0 /N 1 N 2, where image I is of size N 1 by N 2. 32

34 Compression results Original Compressed SSIM ε = 32 d = 4, average error per pixel for 8 x 8 blocks PSNR = db, MSSIM = , nsbr = bpp 33

35 Back to our original problem k = 40 (62.5%) 34

36 Compressed sensing and sampling min x x 0 subject to PA x c 2 < ε P in R k x n, A in R n x m, and c in R k 35

37 Deterministic sampling masks If d = 4, then use ε = d k 36

38 Deterministic sampling masks A x c 2 < ε, with x = OMP(A,c,ε), and x in R m 37

39 Deterministic sampling masks A x c 2 < ε, with x = OMP(A,c,ε), and x in R m = c 3-1 (A x ) 38

40 Results Original Masked Luminance: PSNR = db MSSIM = k = 40 (62.5%) d=4 Luminance SSIM Reconstruction CB: PSNR = db MSSIM = C R: PSNR = db MSSIM =

41 Results PSNR = PSNR = k = 40 (62.5%) d=4 PSNR =

42 Results Original detail Masked detail Reconstruction detail k = 40 (62.5%) d = 4 Deterministic sampling masks ~ In-painting? 41

43 Thank you! 42