Lecture 13 Copyright y Hongyun Wang, UCSC Recap: Fitting to exact data *) Data: ( x j, y j ), j = 1,,, N y j = f x j *) Polynomial fitting Gis phenomenon *) Cuic spline Convergence of cuic spline *) Application #1 of cuic spline: Find the period of a function represented y N data points We look at another application of cuic spline. Application # of cuic spline: Find the distance from a point to a trajectory Consider a D trajectory {x(t), y(t), 1 < t < } that is only represented y N data points. ( t i, x i, y i ), i = 1,,, N x i = xt ( i ) y i = yt ( i ) Data format: td = [ t 1, t,, t N ]: t-values of the data xd = [ x 1, x,, x N ]: x-values of the data yd = [ y 1, y,, y N ]: y-values of the data Goal: Find the distance from a given point (u, v) to the trajectory {x(t), y(t), 1 < t < }. Consider the distance etween (u, v) and a point (x(t), y(t)) on the trajectory: dd ()= t u xt ( ()) + v yt () - 1 -
he distance from (u, v) to the trajectory is defined as min dd t t () We look at how to calculate dd(t) for any given value of t. Matla code dd.m: function [dist] = dd(t, td, xd, yd, u, v) xt=spline(td, xd, t); yt=spline(td, yd, t); dist=norm([u,v]-[xt,yt]); Once we have the distance function, we use the golden search method to minimize dd(t). t = arg min t dd () t he distance from (u, v) to the trajectory is dd ( t ) Fitting to data with noise For fitting to data with noise, first we are going to look at the least squares fitting. hen we are going to discuss the Fourier transform as a separate suject ecause the Fourier transform has many roader applications, not limited to fitting to data with noise. Least squares fitting Given a set of N data points: ( x j, y j ), j = 1,,, N y j = f x j + j Measured value Exact value Noise Consider a fitting function of the form gx = 1 g 1 ( x) + g ( x) + + p g p ( x) where ase functions g 1 ( x), g ( x),, g p ( x) are given and coefficients 1,,, p are to e determined (undetermined coefficients). - -
he fitting function gx is a linear comination of the given ase functions. N: numer of data points p: numer of undetermined coefficients In the least squares fitting, usually N is much larger than p: N >> p (for example, N =, p = 3) Example: gx = 1 + x + 3 x In this example, the ase functions are ( x)= 1, g ( x)= x, g ( x)= x g 1 Example: gx = 1 + cos( x) + 3 sin( x) In this example, the ase functions are ( x)= 1, g ( x)= cos( x), g ( x)= sin x g 1 Let us put the undetermined coefficients into a vector. = 1,,, p Consider a measure of distance etween the fitting function gx and the date. N = ( gx ( j ) y j ) q j =1 which is the sum of squares of differences. he distance q is a function of the undetermined coefficients Goal: Solve the minimization prolem arg min q = ( 1,,, p ). Note that in the least squares fitting, the undetermined coefficients appear linearly. As a result, in the least squares fitting, we only need to solve a linear system instead of a non-linear system. We like to write everything in vector notation. - 3 -
Vector notation: = 1,,, p a vector of size p x = ( x 1, x,, x N ) a vector of size N y = y 1, y,, y N g k = g k ( x 1 ), g k ( x ),, g k x N Or g k = g k ( x) g = gx ( 1 ), gx ( ),, gx ( N ) a vector of size N ( ) a vector of size N a vector of size N Or g = g( x ) We write vector g as g = 1 g 1 + g + + p g p In terms of matrix-vector multiplication, it is g = G where G = ( g 1, g,, g p ) an N p matrix With the matrix-vector notation, we write q = q ( g y ) ( g y ) = G as ( y ) ( G y ) o solve the minimization prolem, we need the gradient of q he gradient of q( ) is q( )= ( G y ) G (See Appendix for the derivation). Note that the gradient q Suppose q is a row vector. attains the minimum at ==> ( q( )) = ==> (( G y ) G) =. - 4 -
==> G ( G y )= ==> G G G y = ==> ( G G) = G y his is called the normal equation of the least squares prolem. It has the form of a linear system A = c where A = G G c = G y a p p matrix a vector of size p In Matla, to solve A= c, we simply use = A \ c ; Steps in implementing the least squares 1) Get data x = ( x 1, x,, x N ) y = y 1, y,, y N ) Calculate g k = g k ( x) G = g 1, g,, 3) Calculate ( g p ) A = G G c = G y 4) Solve A = c Fourier transform Review of the complex exponential function he special imaginary numer: i = 1 Euler formula: exp i = cos + isin( ) - 5 -
Special cases: exp i = 1 exp i = i exp( i ) = 1 exp( i )= 1 =1 for n = integer exp in he complex exponential function has the same properties as the real exponential function: exp( i 1 + i ) = exp( i 1 )exp( i ) p exp i = exp ip d exp( i) = i exp( i) d Integrals of the complex exponential function: exp( i)d = 1 exp i i a a for For k = integer, we have exp i k 1 d = i k For k =, we have exp i k 1 = exp i k d = 1d = Summarizing the two cases, we have exp i k, k d =, k = for k = integer Fourier transform theorem: Suppose f ( t) is periodic with period. hen f ( t) can e expressed as - 6 -
= c k exp i kt f t k = { c k, < k < } are called Fourier coefficients. Question: How to find c k? o find c k, we multiply oth sides y exp i jt f()exp t i jt = c k exp i ( k j)t k = hen we integrate from to f()exp t i jt dt = c k exp i ( k j)t dt k = In the aove, we derived exp i ( k j)t dt, k j =, k = j ==> f()exp t i jt dt = c j he Fourier coefficients are given y c k = 1 Fourier transform is a mapping f(), t t < f()exp t i kt dt { } { c k, < k < } Applications of Fourier transform: Solving PDE Noise removal Data compression Example: Using Fourier transform to solve an initial value prolem Consider the initial value prolem of 1-D heat equation - 7 -
u t = u xx u( x,)= u x u(x, t) is the temperature at position x at time t. he 1-D heat equation descries the heat conduction along an insulated thin rod. Suppose u (x) is periodic with period = L. Function u (x) has the Fourier transform u ( x)= c k exp i kx L k = where the Fourier coefficients have the expression L c k = 1 L u ( x)exp i kx L dx heorem: he solution of the initial value prolem is given y u ( x,t )= c exp k k L k = kx t exp i L Verify the theorem: We need to verify that u(x, t) given aove satisfies i) the initial value and ii) the PDE. u ( x, )= c exp k k L k = u t = u xx = k = = c k exp i kx L = u x k = k = ==> u t = u xx c k exp k L c k exp k L kx exp i L t k L t i k L kx exp i L exp i kx L Example: Using Fourier transform to solve an initial-oundary value prolem Consider the initial-oundary value prolem of 1-D heat equation - 8 -
u t = u xx u(, t)=, u( L, t)= u( x,)= u ( x), x L We first extend u (x) to an odd function. w ( x)= u ( x), x L u ( x), L x hen we extend w (x) periodically with period = L. Function w (x) has the Fourier transform w ( x)= d k exp i kx ( L) k = where the Fourier coefficients have the expression d k = 1 L = 1 L L w L L u ( x)exp i kx L dx ( x) exp i kx kx ( L) exp i ( L) dx heorem: he solution of the initial-oundary value prolem is given y u( x,t)= d k exp k L k = kx t exp i L - 9 -
Appendix: Gradient of q( ) First, it is straightforward to verify that z ( z )= z z Let z = G y. We have q( )= z z z z ( z )= z Using the chain rule, we have q( )= z z ( z ) z ==> q( )= ( G y ) G - 1 -