C2 Differential Equations : Computational Modeling and Simulation Instructor: Linwei Wang

Transcription

1 C2 Differential Equations : Computational Modeling and Simulation Instructor: Linwei Wang

2 Part II Variational Principle

3 Calculus Revisited Partial Derivatives Function of one variable df dx = lim h!0 f (x) f (x + h)" f (x) h Function of multiple variables f (x, y,z), x, y, z are independent! f!z (x, y, z) = lim h!0 f (x, y, z + h)" f (x, y, z) h Computing partial derivatives: simply treats other variables as fixed constants.

4 Exercises Suppose "f (x, y,z) "x "f (x, y,z) "y "f (x, y,z) "z "f (x, y,z) "z"y f (x, y,z) = x 2 y 2 z 2 + y cos(z) =? 2xy 2 z 2 =? 2yx 2 z 2 + cos(z) =? 2zx 2 y 2 " y sin(z) =? 4yzx 2 " sin(z)

5 Calculus Revisited Total derivative f (x, y,z) = f (x,y(x),z(x)) Chain rule df (x, y, z) dx =! f!x +! f!y dy dx +! f!z dz dx

6 Variational Principle How to set up a differential equation or its approximation? A scientific principle that develop general methods for finding functions that minimize or maximize the quantities that are functions of the unknown functions States a problem in terms of an unknown function & its derivative that makes an integral take on an extremum To find the system behavior of interest (differential equations), by looking at the property of the system, such as the minimum energy of the system (locating the extremumm of a functional)

7 Calculus of Variation Finding the extremum of an integral involving a function and its derivatives Unknown function: y(x) :! " R Integral on an extremum I = " F[y,!y, x]dx!y = #! x y :! $ R n I : R % R n %! $ R Problem of interest is cast into an optimization problem

8 Variation Principle Theorem (Variation Principle) If ŷ is an extremum of a functional I :V! R, then!i(ŷ;!y) = lim!y!0 I(y +!y)" I(y)!y = 0,#!y $ V

9 Euler-Lagrange Equation A PDE to be satisfied by the solution!i(ŷ,!y) = 0 ŷ in order for Let Then ŷ I = " F[y,!y, x]dx! satisfies the Euler-Lagrange Equation: (!F!y! div x!f!y ) = 0

10 !I = lim!y!0 I(y +!y)" I(y)!y 1 = lim!y!0!y ( # x 2 F(y +!y, d x 1 dx 1 x2 # (F(y+!y, d!y!0!y x1 dx = lim = # x 2 ( "F "F!y + x1 "y "!y = # x 2 ( "F x1 "y " d dx Euler-Lagrange Equation Deriving Euler-Lagrange equation (y+!y),x)"f(y, dy dx,x))dx "F "!y d!y dx )dx dy (y +!y), x)dx " # x 2 F(y, dx, x)dx ) x 1 Chain rule Integral by parts )!ydx + "F "!y!y x=x 2 " "F "!y!y x=x 1 = 0

11 Euler-Lagrange Equation On one function y over one variable x: Boundary condition: (!F!y! d dx!f!y x=x 2 = 0!F!y x=x 1 = 0!F!y ) = 0 "f # d "y i dt ("f ) = 0 " y i

12 Beltrami Identity If function F does not directly depend on u(x) I =! F[!y, x]dx Euler equation is equivalent to x 2 x 1!F!y = 0! d dx!f!y = 0!!F!y = C

13 Beltrami Identity If function F does not directly depend on x I =! F[y,!y ]dx x 1 If!F, Euler equation is equivalent to!x = 0 x 2 F!!y!F!y = C d(f!!y!f!y ) dx =!F!x +!F!y =!y(!f!y! d!f dx!y ) = 0!y +!F!y!!y!!!y!F!y!!y d!f dx!y

14 Eular-Lagrange Equation In general: (!F!y! div x!f!y ) = 0 One function y over one variable x: Several function y i over one variable (!F!y i! d dx One function over several variables x i (!F!y! d dx!f!y i ) = 0 for i =1, 2, N (!F N!y! " d i=1 dx i!f!y xi ) = 0!F!y ) = 0

15 Classic Example in Physics 1: Functional of one function of one variable

16 Problem 1: Newton s Equation Extremize the following functional: I = t ( m 2 " 2!x2 (t)!v(x(t),t)) dt t 0 Solve with Euler s equation:!f!y! d dx (!F!y ) = 0 Newton s equa-on: m!!x 2 (t) =! "V(x(t),t)) "x(t) Now if V(x,t) = V(x) is independent of t F!!y!F!y = C Energy conserva-on: m 2!!x2 (t)+v(x(t)) = C

17 Problem 2: Minimum-Distance The function y(x) that minimizes the distance between two points in a plane x 2 x I = " x1 ds= " 2 x1 dx 2 + dy 2 = Solve with Euler s equation: x " 2 1+ ( dy x 1 dx )2 dx!f!y! d dx (!F!y ) = 0 - F = 1+!y 2 - d dx (!y -!F!y = 0!F!y =!y 1+!y 2 - y 1+!y 2 ) = 0 1+ y 2 = C " y 2 = C 2 1# C 2

18 Classic Example in Physics 2: Functional of several functions of one variable

19 Problem 3 Find the shortest path on a given surface embedded in three dimensions which connects two given points A and B on this surface Surface: z = z(x, y) Path connecting A and B: parameterized by a vector valued function x(t) = (x(t), y(t)), 0! t!1 x(0) = (x 0, y 0 ) = A, x(1) = (x 1, y 1 ) = B Length of the path B I(x) = " = " dx 2 +dy 2 +dz 2 " B ds A A 1 = x'(t ) 2 +y'(t ) 2 +[z x (x(t ),y(t ))x'(t )+z y (x(t ),y(t ))y'(t )] 2 dt 0

20 Problem 3 Functionals depending on several function of one variable I(x) = Euler Lagrange equations:! 1 0 x'(t) 2 +y'(t) 2 +[z x (x(t),y(t))x'(t)+z y (x(t),y(t))y'(t)] 2 dt " (!F!x! d $ dt # $ (!F!y! d % $ dt!f!x t ) = 0!F!y t ) = 0 x 2 I =! F[y,!y, x]dx, y = (y x 1, y 2, y N ) " (!F # d 1!y i dx!f!y i ) = 0, i =1, 2, N

21 Classic Example in Physics 3: Functional of one function of several variables

22 Problem 4 A homogeneous membrane is fixed in a given frame. Find the shape of the membrane Membrane shape: u(x) = u(x, y) (x, y) " # Boundary condition: fixed frame u(x, y) = a(x, y) (x,y) " #$ - - The physical principle determines the shape of the membrane is the potential energy is a minimum The potential energy of a homogeneous membrane is proportional to the surface area A(u) = # ds = 1+ u 2 " x + u 2 y dxdy " # $ [ (u x 2 + u y 2 )]dxdy I(u) = # " 1 2 (u x 2 + u y 2 )dxdy # "

23 Memebrane shape I(u) = Problem 4 1 # 2 (u 2 x + u 2 " y )dxdy Functionals depending on functions of several variables I = "! F[u(x, y),!u(x, y), x, y]dx dy Euler Lagrange equations: (!F!u! d dx!f!u x! d dy!f!u y ) = 0 u!" = u 0 u xx + u yy = 0 u!! = u 0

24 Classic Example in Physics 4: Functional with Constraints

25 Problem 5 Find the shape of a homogeneous chain, length L 0, fixed in the two end points in the earth gravitational field chain shape: y(x) x 0 " x " x 1 Minimum potential energy gy(x)dm Chain: a collection of small parts that have length ds = y(x) 1+ y'(x) 2 xdx 0 " x " x 1 mass m = "ds potential energy Functional: under constraints I = "g x 1 x # yds = "g # 1 y 1+ y' 2 dx x 0 x 0 x 1 x " ds = " 1 1+ y' 2 dx = L 0 x 0 gy(x)dm = g"y(x)ds x 0

26 Lagrange Multiplier One kind of optimization method to find the maxima/ minima of a function s.t. constraints subject to argmin f (x), x g(x) = c A Lagrange multiplier λ can be introduced to study the following function (Lagrange function) instead h(x,!) = f (x)+!!(g(x)" c)

27 Lagrange Multiplier If f(x*) is a minimum of the original constrained problem, then there exists such λ so that (x*, λ ) is the stationary point for the Lagrange function h(x,!) = f (x)+!!(g(x)" c) Stationary point:!h(x,!)!x x* = 0,!h(x,!)!!! = 0 Necessary condition. Not all stationary points are the local optima of the original problem

28 Lagrange Multiplier How to understand? h(x,!) = f (x)+!!(g(x)" c)

29 Lagrange Multiplier How to understand? h(x,!) = f (x)+!!(g(x)" c) If g(x)=c intersects with the contour line of f, it means the value of f will change if we keep moving along g(x) = c Only when g(x)=c meets f tangentially, we do not increase of decrease the value of f!f = "!!g,!f = [ #f #x 1, #f #x 2,... #f #x n ],!g = [ #g #x 1, #g #x 2,... #g #x n ], This is the same as introducing the Lagrange function, and solve! x,! h(x,!) = 0 Implies g(x)=c

30 Lagrange Multiplier- A Toy Example Find the minimum value of f (x, y) = 5x! 3y subject to x 2 + y 2 =136 Solution h(x,!) = 5x! 3y +! "(x 2 + y 2!136) #!h(x, y,!) %!x %!h(x, y,!) $ %!y %!h(x, y,!) % &!! = 5" 2!x = 0 = "3" 2!y = 0 = x 2 + y 2 "136 = 0 The constraints

31 Lagrange Multiplier for Functionals Variational problem with constraints given by functionals Extremizing x 1 I =! F[y(x),!y (x), x]dx x 0 under constraints x 1 J =! G[y(x),!y (x), x]dx = C x 0 Equivalent to extremizing the functional x 1 x 1! = " F[y(x),!y (x), x]dx +!(" G[y(x),!y (x), x]dx # C) x 0 x 1 x 1 = " (F[y(x),!y (x), x]+! " G[y(x),!y (x), x])dx # C x 0 x 0 x 0 I(y) " #J(y)!(F + "G) (! d!y dx!(f + "G) ) = 0!y Solution y depends on, which then should be fixed so that J(y( )) = C

32 Problem 5 To minimize potential energy subject to fixed length Equivalent to extremizing the functional x 1 x 1 I =!g! yds =!g! y 1+ y' 2 dx x 1 x 1! ds =! 1+ y' 2 dx = L 0 x 0 x 0 x 0 x 0 " x 1! = (!gy 1+ y' 2 + " 1+ y' 2 )dx x 0!(F + "G) (! d!y dx!(f + "G) ) = 0!y I(y) " #J(y)

33 Generalization Variational problem with constraints given by functionals Extremizing under a set of L constraints I =! F[y(x),!y (x), x]dx Equivalent to extremizing the functional x 1 x 1 x 0 J i =! G i [y(x),!y (x), x]dx = C i x 0 L! = I + "! i J i i=1 L!(F +! " i G i ) i=1 ( " d!y dx L!(F +! " i G i ) i=1 ) = 0!y

34 Hamilton Principle Mechanical system Generalized coordinate (trajectory) & velocity: x(t) & dx(t) Kinetic energy K(q,dq/dt) Potential energy: U(x, t) Action 3 x = (x, y, z), K =! 1 2 mvi2 =! 1 2 m!x i i=1 3 i=1 '2 I = t 1! L(x, dx / dt,t)dt =! (K(x, dx / dt,t)"u(x,t))dt t 0 t 1 t 0! t 1 i=n # i=1 = ( ( m i x 2! i (t) 2 "U(x 1 (t), x 2 (t), x N (t))))dt t 0

35 Hamilton Variational Principle Hamilton variational principle: the motion of a system is the stationary point of its action L(x, dx / dt,t) = K(x, dx / dt,t)!u(x,t) on the set of all paths beginning at point q 0 and instant t 0, and ending at point q 1 and instant t 1 Newton s second law of mo3on!l!x i! d dt!l!x i = 0 m i!!x i (t) =!!U(x(t),t)!x i (t) i =1, 2, N

36 Problem 6: Membrane Oscillation Oscillation of a membrane described by u(x,t) = u(x, y,t) Hamilton variational principle Potential energy U = # " S 2 (u 2 x + u 2 y )dxdy Kinetic energy K = $ # " 2 u 2 t dxdy Action functional I = % ( " 2 u 2 t # S 2 (u 2 x + u 2 $ y ))dxdy Motion for the membrane Hyperbolic PDE: Wave equation "u tt # S(u xx + u yy ) = 0

37 Homework Suppose P and Q are two points in the plane. Imagine there is a thin flexible wire connecting the two points. Suppose P is above Q, and we let a frictionless bead travel down the wire impelled by gravity alone. Determine the shape of the curve that will result in the minimum travel time of the bead.

38 Variational Methods in General Domains

39 Variational Approach in Computer Vision A unifying concept to deal with a variety of problems in computer vision, such as image segmentation, denoising, 3D reconstruction, etc Problem solution is the minimizer of an energy functional E, argmin u!v E(u) Images as functions: A greyscale image is a real-valued function A color image is a vector-valued function u :! " R,! # R 2 u :! " R 3,! # R 2

40 Example 1: Image De-noising Given image: f (x) = u(x)+ n(x) Denoised solution: u(x) Typical least-square restoration methods: Objective: argmin " (u # f ) 2 dx u(x)! under desirable constraints: e.g., smoothness, edge-preserving, etc.

41 Total Variation Image De-noising f (x) = u(x)+ n(x) Optimization problem: Objective: s.t. argmin u(x) #!u dx dy = # (u 2 " x + u 2 y ) dx dy " "! udx dy = "! 1 2 (u " f )2 dx dy f dx dy #! =! 2 White noise is zero- mean Prior informa-on on the standard devia-on of noise

42 TV-L 1 Model #!u dx dy = # (u 2 " x + u 2 y ) dx dy " Total variation of the image Consider L 1 norm instead of L 2 norm of the gradients The simplicity of L 2 models is now lost and complexity emerges

43 Euler-Lagrange Equations Objec-ve func-onal Constraint 1 F = (u x 2 + u y 2 ) G 1 = u Constraint 2 Equivalent to: G 2 = 1 2 (u! f )2 (!(F + " 1G 1 + " 2 G 2 )!u! d dx!(f + " 1 G 1 + " 2 G 2 )! d!(f + " 1 G 1 + " 2 G 2 ) ) = 0!u x dy!u y

44 Euler-Lagrange Equations (!(F + " 1G 1 + " 2 G 2 )!u! d dx!(f + " 1 G 1 + " 2 G 2 )! d!(f + " 1 G 1 + " 2 G 2 ) ) = 0!u x dy!u y F +! 1 G 1 +! 2 G 2 = (u 2 x + u 2 y ) +! 1 u + 1 2! 2 (u! f )2!!x ( u x (u x 2 + u y 2 ) )+!!y ( u y (u x 2 + u y 2 ) )"! 1 "! 2 (u " f ) = 0 # $!u!! n = 0, "!# What is left is how to solve the above differential equations, which will be covered later

45 Euler-Lagrange Equations!u 2 =! x 2 u +! y 2 u is not always differentiable! u x (u x 2 + u y 2 ) u y (u x 2 + u y 2 ) has no meaning when! x 2 u +! y 2 u = 0 Consider: L! =! x 2 u +! y 2 u +! + " 1 u " 2(u " f ) 2!L!!u " div x!l!!!u =! 1u +! 2 (u " f )" div x ( #u #u " ) = 0

46 Variations of TV-Denoising For example Instead of min u(x) # (!u(x) +! # 1 (u " f )2)dx dy 2 min u(x) # (!u(x) +! # u(x)" f (x) ) dx dy Total Varia3on: preserve sharp edge beger than # "u(x) 2 Fidelity term: preserve contrast & image beger than # (u(x) " f (x)) 2 dx

47 Example 2: Computer Graphics Variational Surface Design Smooth surface design Generating a smooth surface from a set of 3D data points Design of cars, air planes, ship bodies; modeling robots, etc

48 Variational Surface Design Given 3D point sets c i, h i Surface representation by a typical family of functions Variational Optimization problem: x) Minimal energy: min E =! "!u 2 ds + "! 2 u 2 ds Constraints: u(c i ) = h i First order form Stretch energy Second order form Bend energy

49 Variational Surface Design: I Given 3D point sets Surface representation Optimization problem: c i, h i x) f (x) =!t + " h i ( x) n i=1 h i (x) = e x!c i 2 /! 2 s.t. Objective: min E = " ( f 2 xx + 2 f 2! xy + f 2 yy )dxdy Curvature f (c i ) = h i

50 Variational Surface Design: II Given 3D point sets c i, h i Surface representation (biquintic b-spline surface) f (x, y) = 4n+2 4m+2 Optimization problem:!! d ij N 5 i (x) N 5 j (y) i=1 j=1 Objective: n p " min [ f (c i )! h i ] 2 f i=1 With minimum curvature

51 Variational Surface Design Variational Approach: minimize energy term n p (1!!){"" pi [ f (c i )! h i ] 2 } i=1 n m y # +!{ "" 3 j+1 x i+1 f (x, y) " 3u # # " y x 3uij #x 3 j i i=1 j=1 2 dx dy n m y # + "" 3 j+1 x i+1 f (x, y) " 3v # # " y x 3vij #y 3 j i i=1 j=1 2 dx dy Constraints: Minimum curvature variation

52 Solution Least square fitting Variational Surface Design Calculus of variation n p "! pi [ f (c i )! h i ] 2 # min i=1 n m y "!! 3 j+1 x i+1 f (x, y) " (! 3u! y x 3uij i=1 j=1 "x 3 j i 2 " 3 f (x, y) " +! 3v! 3vij )dx dy # min "y 3 2 Weighted least square (1!!)A +!B

53 Example 3 Super-Resolution Texture Given all images Determine the surface color min T n Estimation I i :! i " R 3 T : S! R 3 2 % (b!(t!! i )" I i ) dx + " $ & S T ds i=1 $ #i S Blur & downsample Back- projec-on Goldlucke, Crembers, ICCV 09, DAGM 09 Cremers, Goldlucke, Pock, ICCV 11 tutorial

54 Example 3 Super-Resolution Texture Estimation n 2 min (b!(t!! i )" I i ) T % $ dx + " & #i $ S T ds S i=1 Euler-Lagrange equation gives a PDE on the surface n de dt =!div " S T S( )+#! i " S T S " ((J id i )!! i ) = 0 D i = b $(b$(t!! i )! I) J i = %! i %x & %! i %y!1 i=1 Goldlucke, Crembers, ICCV 09, DAGM 09 Cremers, Goldlucke, Pock, ICCV 11 tutorial

55 Example 3 Super-Resolution Texture Estimation Simultaneous estimation of geometry & texture T : S! R 3 D : S! R min D,T n (b!(t!! D 2 % i )" I i ) dx + " $ & S T ds i=1 $ #i S Geometry- dependent Back- projec-on Goldlucke, Crembers, ICCV 09, DAGM 09 Cremers, Goldlucke, Pock, ICCV 11 tutorial

56 Example 3 Super-Resolution Texture Estimation Goldlucke, Crembers, ICCV 09, DAGM 09 Cremers, Goldlucke, Pock, ICCV 11 tutorial

57 Example 3 Super-Resolution Texture Estimation Goldlucke, Crembers, ICCV 09, DAGM 09 Cremers, Goldlucke, Pock, ICCV 11 tutorial

58 Summary Key Steps Define a natural objective with constraints Formulate into variational optimization problem that involves the unknown function(s) of interest and its/their derivatives Obtain PDE from Euler-Lagrange equations Solving the PDE Benefit The use of an action to describe the system shifts the emphasis away from system behavior as a basis for modeling Instead, the emphasis is on factors that affect the system, which consequently determine its behavior