MATH 235/W08: Orthogonality; Least-Squares; & Best Approximation SOLUTIONS to Assignment 6

Transcription

1 MATH 235/W08: Orthogonality; Least-Squares; & Best Approximation SOLUTIONS to Assignment 6 Solutions to questions 1,2,6,8. Contents 1 Least Squares and the Normal Equations*** Solution Best Polynomial Fit*** Solution Least Squares Solutions and Errors 10 4 Best Quadratic Polynomial Fit 10 5 Best Approximation in Continuous Function Space 10 6 Orthogonality*** Solution MATLAB; Best Line Fit 12 8 MATLAB; Best Quadratic Fit*** Solution BONUS: Eigenvalues and Best Approximation 14 1

2 1 Least Squares and the Normal Equations*** Find the least-squares approximation and the error to a solution of Ax = b, by constructing the normal equations for ˆx and then solving for ˆx: (a) A = 2 6, b = i i (b) A = 1 3i, b = 4 + 2i 1 + 2i 1 + 3i 2 + i Note that the normal equations over C for Ax = b are A Ax = A b where the operation denotes complex conjugation plus transposition, that is, A = A T. (c) A = , b = Hint: The file tempoutput.txt at hwolkowi/henry/teaching/w08/235.w08/miscfiles/tempoutput.txt and MATLAB may help. 1.1 Solution BEGIN SOLUTION: (a) Following is the solution first using \ in MATLAB and then using GE. The MATLAB output follows. format compact A=[ ]; b=[3;3;3]; disp( construct matrix and RHS for normal equations ) construct matrix and RHS for normal equations AtA=A *A;Atb=A *b; disp( solve for xhat using \ ) 2

3 solve for xhat using \ xhat=ata\atb xhat = disp( the error A xhat -b is ) the error A xhat -b is norm(a*xhat-b) ans = disp( solve for xhat using GE with a check col. ) solve for xhat using GE with a check col. disp( First form the augmented matrix ) First form the augmented matrix augm=[ata Atb ] augm = disp( Now add a check column ) Now add a check column augm=[augm augm*ones(3,1)] augm = tempa=sym(augm) [ 38, 82, -12, 108] [ 82, 182, -36, 228] disp( Now start to pivot ) Now start to pivot tempa(1,:)=tempa(1,:)/tempa(1,1) [ 1, 41/19, -6/19, 54/19] [ 82, 182, -36, 228] tempa(2,:)=tempa(2,:) - tempa(2,1)*tempa(1,:) [ 1, 41/19, -6/19, 54/19] [ 0, 96/19, -192/19, -96/19] tempa(2,:)=tempa(2,:)/tempa(2,2) [ 1, 41/19, -6/19, 54/19] [ 0, 1, -2, -1] tempa(1,:)=tempa(1,:) - tempa(1,2)*tempa(2,:) [ 1, 0, 4, 5] [ 0, 1, -2, -1] disp( the solution is from column 3 ) 3

4 the solution is from column 3 xhat=tempa(:,3) xhat = 4-2 disp( the error A xhat -b is ) the error A xhat -b is norm(a*double(xhat)-b) ans = echo off (b) Following is the solution first using \ in MATLAB and then using GE. The MATLAB output follows. format compact A=[ -2*i 1; -1 3*i;-1+2*i -1+3*i ]; b=[-3+i;-4+2*i;2+i]; disp( construct matrix and RHS for normal equations ) construct matrix and RHS for normal equations AtA=A *A AtA = i i Atb=A *b Atb = i i disp( solve for xhat using \ ) solve for xhat using \ xhat=ata\atb xhat = i i disp( the error A xhat -b is ) the error A xhat -b is norm(a*xhat-b) ans = disp( solve for xhat using GE with a check col. ) solve for xhat using GE with a check col. disp( First form the augmented matrix ) First form the augmented matrix augm=[ata Atb ] augm = i i 4

5 i i disp( Now add a check column ) Now add a check column augm=[augm augm*ones(3,1)] augm = i i i i i i tempa=sym(augm) [ 10, (7)-(2)*sqrt(-1), (2)-(13)*sqrt(-1), (19)-(15)*sqrt(-1)] [ (7)+(2)*sqrt(-1), 20, (4)+(6)*sqrt(-1), (31)+(8)*sqrt(-1)] disp( Now start to pivot ) Now start to pivot tempa(1,:)=tempa(1,:)/tempa(1,1) [ 1, 7/10-1/5*sqrt(-1), 1/5-13/10*sqrt(-1), 19/10-3/2*sqrt(-1)] [ (7)+(2)*sqrt(-1), 20, (4)+(6)*sqrt(-1), (31)+(8)*sqrt(-1)] tempa(2,:)=tempa(2,:) - tempa(2,1)*tempa(1,:) [ 1, 7/10-1/5*sqrt(-1), 1/5-13/10*sqrt(-1), 19/10-3/2*sqrt(-1)] [ 0, 147/10, 147/10*sqrt(-1), 147/10+147/10*sqrt(-1)] tempa(2,:)=tempa(2,:)/tempa(2,2) [ 1, 7/10-1/5*sqrt(-1), 1/5-13/10*sqrt(-1), 19/10-3/2*sqrt(-1)] [ 0, 1, sqrt(-1), 1+sqrt(-1)] tempa(1,:)=tempa(1,:) - tempa(1,2)*tempa(2,:) [ 1, 0, -2*sqrt(-1), 1-2*sqrt(-1)] [ 0, 1, sqrt(-1), 1+sqrt(-1)] disp( the solution is from column 3 ) the solution is from column 3 xhat=tempa(:,3) xhat = -2*sqrt(-1) sqrt(-1) disp( the error A xhat -b is ) the error A xhat -b is norm(a*double(xhat)-b) ans = echo off (c) Following is the solution using \ in MATLAB. Due to the size of the problem we do not show the individual row operations for GE. format compact A =[

6 ]; b =[ ]; disp( construct matrix and RHS for normal equations ) construct matrix and RHS for normal equations AtA=A *A AtA = Atb=A *b Atb = disp( solve for xhat using \ ) solve for xhat using \ xhat=ata\atb xhat =

7 disp( the error A xhat -b is ) the error A xhat -b is norm(a*xhat-b) ans = echo off END SOLUTION. 2 Best Polynomial Fit***. You are given the four data points: y = 4 at t = 2; y = 3 at t = 1; y = 1 at t = 0; y = 0 at t = 2. (a) Find the best straight-line fit (least squares) to this data having the form y = β 0 + β 1 t. (b) Find the best quadratic polynomial fit (least squares) to this data having the form y = β 0 + β 1 t + β 2 t Solution BEGIN SOLUTION: Let the points for y and t be denoted by y i and t i, respectively. (a) First form the equations, in matrix form, using y i = β 0 + β 1 t i : ( ) y = 3 1 = 1 1 β β Then form and solve the normal equations. The MATLAB outputs appear below with the plots of the best fits in Figure 1. (b) First form the equations, in matrix form, using y i = β 0 + β 1 t i + β 2 t 2 i : y = 3 1 = β β 1. β Then form and solve the normal equations. The MATLAB outputs appear below with the plots of the best fits in Figure 1. 7

8 format compact disp( first find the parameters for the best line fit ) first find the parameters for the best line fit A=[ ]; b= [4;3;1;0]; AtA=A *A AtA = Atb=A *b Atb = 8-11 disp( y-inters. and slope for best line fit are: ) y-inters. and slope for best line fit are: beta=sym(ata)\sym(atb) beta = 61/35-36/35 betad=double(beta); plot(a(:,2),b, xr ) legend( best line fit ) hold on ts=linspace(-4,4); ys=betad(1)+betad(2).*ts; plot(ts,ys) disp( second find the parameters for the best quadratic fit ) second find the parameters for the best quadratic fit A=[ ]; b= [4;3;1;0]; AtA=A *A AtA = Atb=A *b Atb = 8 8

9 disp( beta_0, beta_1, beta_2 are: ) beta_0, beta_1, beta_2 are: beta=sym(ata)\sym(atb) beta = 73/55-58/55 2/11 betad=double(beta); hold on ys=betad(1)+betad(2).*ts+betad(3).*ts.^2; plot(ts,ys, g: ) legend( original data, best line fit, best quadratic fit ) title( Best straight-line and quadratic polynomial fits for problem 2 ) xlabel( t-axis ) ylabel( y-axis ) print bestfitsprob2.ps echo off 10 8 Best straight line and quadratic polynomial fits for problem 2 original data best line fit best quadratic fit 6 y axis t axis Figure 1: Plots for the best polynomial fits for Problem 2. END SOLUTION. 9

10 3 Least Squares Solutions and Errors For the inconsistent linear system Ax = b where A = , b = find a least-squares solution and the corresponding least-squares error for the solution. 4 Best Quadratic Polynomial Fit For the data set {(x,y) R 2 ( 1, 4),(0, 2),(2, 2),(3, 5)}, find the least squares quadratic polynomial of the form y(x) = a + bx + cx 2 that best approximates this data. 5 Best Approximation in Continuous Function Space 1. Consider the vector space C[ 1, 1] of continuous functions on the interval [-1,1], together with the inner product given by < f,g >= 1 1 f(t)g(t)dt, for f,g C[ 1,1]. Let P 2[ 1,1] be the { subspace of C[ 1,1] having } an orthonormal basis given by 45 B = 8 (t2 1 3 ), 3 2 t, 2 1. Find the best quadratic approximation to e t on [ 1,1]. 2. Consider the vector space C[0,1] of continuous functions on the interval [0,1], together with the inner product given by < f,g >= 1 0 f(t)g(t)dt, for f,g C[0,1]. Let P 2[0,1] be the subspace of C[0,1] having a basis given by B = {1,x,x 2 }. Find the best quadratic approximation to x on [0,1]. 6 Orthogonality*** 1. Suppose that A is a m n real matrix. And suppose that Ax = 0 and A T y = 2y. Show that x is orthogonal to y. 2. State and justify whether the following three statements are True or False (give an example in either case): , (a) Q 1 is an orthogonal matrix when Q is an orthogonal matrix. (b) If Q (a m n matrix with m > n) has orthonormal columns, then Qx always equals x. (c) If Q (a m n matrix with m > n) has orthonormal columns, then Q T y always equals y. 6.1 Solution BEGIN SOLUTION: 10

11 1. Since 2y = A T y (y is an eigenvector of A T with eigenvalue 2), we see that y is in the range of A T. Since Ax = 0, we see that x is in the nullspace of A. Therefore, by a theorem proved in class and by a theorem in the text, x y. A direct proof follows: 2x T y = x T (2y) = x T (A T y) = x T A T y = (Ax) T y = 0 T y = (a) TRUE. Justification: The definition of Q orthogonal is QQ T = I. (Equivalently Q T Q = I.) But, (Q T ) 1 = (Q 1 ) T so I = I 1 = (QQ T ) 1 = (Q T ) 1 Q 1 = (Q 1 ) T Q 1, i.e. since the inverse is unique, Q 1 is( an orthogonal ) matrix if Q is. A trivial example 0 1 is Q = I. Another trivial example is. Both these are examples of symmetric 1 0 ( ) cases and so not very interesting. A nonsymmetric example is (b) TRUE. Justification: Note Qx ( 2 = ) x T Q T Qx = x T x = x 2, since Q T Q = I n. Again, ( ) In x a trivial example is with Q = and x R 0 n any vector. We see that Qx =. m n,n 0 Therefore Qx 2 = x = x 2. 1 (c) FALSE. Justification: A simple counter-example to the claim is Q = 0 and y = We get Q T y = 0 y = 2. 1 END SOLUTION. 11

12 7 MATLAB; Best Line Fit Over the past four Winter Olympics Games, the Canadian team won the following numbers of medals: Year Location Number of Medals 1994 Lillehammer, Norway Nagano, Japan Salt Lake City, USA Turin, Italy 24 Scaling the years, we can translate this information to the data points (0,13), (1,15), (2,17), (3,24). (a) Determine the least squares line y = β 0 + β 1 x which best fits these data points. (b) Using your line from (a), predict the total number of medals that Canada will win at the Winter Olympics in Vancouver in MATLAB; Best Quadratic Fit*** Newton s Laws of Motion imply that an object thrown vertically with a velocity of v meters per second will be at a height of h(t) = vt 1 2 gt2 meters after t seconds, where g is the acceleration due to gravity. The values in the table were observed. By fitting a quadratic h(t) = c 0 + c 1 t + c 2 t 2 to these data, estimate the values of v and g in this question. 8.1 Solution t h BEGIN SOLUTION: The MATLAB program below (and the output) show that the estimates are: v = 503/10 and g = 2( 5) = 10. A plot appears in Figure 2. format compact disp( first find the parameters for the best line fit ) first find the parameters for the best line fit A=[ 1.5.5^ ^ ^ ^2]; b= [23.7;45.1;64;80.4]; AtA=A *A AtA = 12

13 Atb=A *b Atb = disp( beta_0, beta_1, beta_2 are: ) beta_0, beta_1, beta_2 are: beta=sym(ata)\sym(atb) beta = -1/5 503/10-5 betad=double(beta); plot(a(:,2),b, rx ) hold on ts=linspace(0,4); ys=betad(1)+betad(2).*ts+betad(3).*ts.^2; plot(ts,ys, g: ) legend( original data, best quadratic fit ) title( best quadratic fit for Newton s Law of Motion ) xlabel( t-axis ) ylabel( y-axis ) print bestnewton.ps echo off END SOLUTION. 13

14 best quadratic fit for Newton s Law of Motion original data best quadratic fit y axis t axis Figure 2: Plots for the best polynomial fits for Problem 8. 9 BONUS: Eigenvalues and Best Approximation 1. Consider the 3 3 matrix Determine the entries a, b, c, d, e, f so that: a b c A = 1 d e. 0 1 f The top left 1 1 block is a matrix with eigenvalue 2; The top left 2 2 block is a matrix with eigenvalues 3 and 3; The matrix A has eigenvalues 0,1 and Find the best line y = c + dt by calculus (not MATLAB). Choose c and d to minimize the least squares problem E(c,d) := 1 0 (c + dt t 2 ) 2 dt. 3. (MATLAB) First note that: The command a=ones(n,1) produces an n 1 matrix of 1 s. The command r=(1:n) produces the vector ( 1,2,...,n ), transposed to a column vector. The command s = r. 2 produces the vector ( 1 2,2 2,...,n 2), because the dot means a component at a time. This problem looks for the line y = c + dt closest to the parabola y = t 2. With n = 10, choose C and D to give the line y = C + Dt that is closest to t 2 at the points t = 1 10, 2 10,...,1 (in the vector r/10 with r). The inconsistent equations Ax = b are 14

15 ( ) ( ) C a r/n = s/n D 2. 15