GradientShop: A Gradient-Domain Optimization Framework for Image and Video Filtering

Transcription

1 GradientShop: A Gradient-Domain Optimization Framework for Image and Video Filtering Pravin Bhat C. Lawrence Zitnick Michael Cohen Brian Curless Presentation : Michael Tao Discussion : Sean Arietta

2 Input Image

3 Input Image + few user inputs

4 Output Image: Relighting

5 Output Image: Recoloring

6 Output Image: Non-Photorealistic Effects

7 Agenda - Motivation - Goal and Algorithm - Applications and Results - Discussion

8 Motivation: Why Gradient Domain? 1. Great for human perception

9 Motivation: Why Gradient Domain? 1. Great for human perception

10 Motivation: Why Gradient Domain? 1. Great for human perception

11 Motivation: Why Gradient Domain? 1. Great for human perception What are the gray-scale values?

12 Motivation: Why Gradient Domain? 1. Great for human perception

13 Motivation: Why Gradient Domain? 2. High-level control over images pixel gradient

14 Motivation: Why Gradient Domain? 2. High-level control over images pixel gradient

15 Motivation: Why Gradient Domain? 1. Great for human perception 2. High-level control over images High-level Applications - enhance texture - change shading/lighting - reduce artifacts - change colors, preserve edges

16 Agenda - Motivation - Goal and Algorithm - Applications and Results - Discussion

17 Goal and Algorithm 1. Great for human perception 2. High-level control over images High-level Applications - enhance texture - change shading/lighting - reduce artifacts - change colors, preserve edges

18 Goal and Algorithm Gradient Domain GradientShop High-level Applications - enhance texture - change shading/lighting - reduce artifacts - change colors, preserve edges

19 Goal and Algorithm Start with Gradient Domain: How Matrices Work? Input Image (4x4): Each square = 1 pixel

20 Goal and Algorithm Start with Gradient Domain: How Matrices Work? Input Image (4x4): What we want: Matrix form of Gradient Domain Δx = I(x,y) I(x-1,y) Δy = I(x,y) I(x, y-1)

21 Goal and Algorithm Start with Gradient Domain: How Matrices Work? Image Matrix (I) Input Image (4x4): (16 x 1)

22 Goal and Algorithm Start with Gradient Domain: How Matrices Work? Fancy Smancy Laplacian Matrix (L) Image Matrix (I) Δx = I(x,y) I(x-1,y) X Δy = I(x,y) I(x, y-1) (32 x 16) (16 x 1)

23 Goal and Algorithm Start with Gradient Domain: How Matrices Work? Fancy Smancy Laplacian Matrix (L) Image Matrix (I) Δx Δy (32 x 16) X (16 x 1)

24 Goal and Algorithm Start with Gradient Domain: How Matrices Work? Fancy Smancy Laplacian Matrix (L) Image Matrix (I) Gradient (G) Δx Δx Δy (32 x 16) X (16 x 1) = Δy (1 x 32)

25 Goal and Algorithm How to GradientShop This?

26 Goal and Algorithm 1. Give Gradient Constraints Fancy Smancy Laplacian Matrix (L) Image Matrix (I) Gradient (G) Δx Δx Δy (32 x 16) X (16 x 1) = Δy (32 x 1)

27 Goal and Algorithm 1. Give Gradient Constraints 1. User Modifies: Fancy Smancy Laplacian Matrix (L) Image Matrix (I) Gradient (G) Δx Δy (32 x 16) X? (16 x 1) = (32 x 1) Δx Δy

28 Goal and Algorithm 1. Give Gradient Constraints Fancy Smancy Laplacian Matrix (L) 2. Solve Image: Image Matrix (I) Gradient (G) Δx Δy (32 x 16) X? (16 x 1) = (32 x 1) Δx Δy

29 Goal and Algorithm Gradient (G) Curl Not Equating to Zero Δx +4-2 Δy (32 x 1)

30 Goal and Algorithm We want: L x I = G L x I G = 0 L x I G 0 Poisson ( backslash ) To solve for unknown I Error = L x I G

31 Goal and Algorithm (L) (I) (G) +1 Every Constraint Treated Equally ? X = Δx Δy (32 x 16) (16 x 1) (32 x 1)

32 Goal and Algorithm 2. Add weights User Modifies: Not everyone is Equal (I) (L) (G) +1 W (32 x 32) ? X = W (32 x 32) (32 x 16) (16 x 1) (32 x 1)

33 Goal and Algorithm 3. How about the image values themselves? (L) (I) (G) +1 W (34 x 34) ? X = W (34 x 34) (34x 16) (16 x 1) 32 (34 x 1)

34 Goal and Algorithm Previous Works: Error = (Gradient Constraint) 1. Give Gradient Constraints 2. Add weights 3. How about the image values themselves? Gradient Shop: Error = Weights Gradient * (Gradient Constraint) + Weights Image * (Image Constraint)

35 Agenda - Motivation - Goal and Algorithm - Applications and Results - Discussion

36 Applications and Results Gradient Shop: Error = Weights Gradient * (Gradient Constraint) + Weights Image * (Image Constraint) High-level Applications - enhance texture - change shading/lighting - reduce artifacts - change colors, preserve edges

37 Applications and Results Input Image Softened Edges

38 Applications and Results Input Image Input Image Relit Image Softened Edges

39 Applications and Results Input Image Non-photorealistic Stuff

40 Discussion Questions?