Experiment # 3 The Gram-Schmidt Procedure and Orthogonal Projections. Student Number: Lab Section: PRA. Student Number: Grade: /20

Transcription

1 ECE 417 CommLab Experiment # 3 The Gram-Schmidt Procedure and Orthogonal Projections Name: Date: Student Number: Lab Section: PRA Name: Student Number: Grade: /20 1 Purpose The purpose of this lab is to investigate the Gram-Schmidt Procedure and use the resulting orthonormal bases to compute orthogonal projections. You may work on this lab during your scheduled lab period using the computers available in the lab (TAs will be available to assist you), or, if you wish, you may work on this lab outside of your scheduled lab period. If you decide to work on this outside of your lab period, you will need Matlab TM or (open source, freely available, largely Matlabcompatible) GNU Octave 1. You may do the lab individually or in groups of 2. The lab must be handed in to a Lab TA during any of the scheduled Lab 3 sessions (refer to the course web site for the schedule). 2 The Gram-Schmidt Procedure Given any vector v in an inner-product space V define the projection operator proj v : V V via { u,v proj v (u) = v if v 0, v,v 0 if v = 0; where, denotes the inner-product operation in V, and 0 denotes the zero vector. 1 Available online at 1

2 Show that proj v (u) is the orthogonal projection of the vector u on the linear subspace spanned by the vector v. (Hint: show that the error u proj v (u) is orthogonal to v, and hence is orthogonal to every vector in the linear subspace spanned by v.) In Matlab or Octave, the inner product between vectors u and v can be computed using dot(u, v). Write a function (saving it in the file PROJ.m ) function w = PROJ(u,v) that implements the projection operator proj v. (Caution: in Matlab, dot works correctly for real-valued and for complex-valued vectors, but in Octave, dot is incorrectly implemented for complex-valued vectors. Throughout this lab, we will only use real-valued vectors in R n.) Test that your function PROJ works correctly by computing the following projections: PROJ([1 2 3],[1 1 1]) = PROJ([1 2 3],[0 0 0]) = PROJ([1 2 3],[1 1-1]) = The Gram-Schmidt procedure takes in a finite set of vectors {v 1,..., v n } (not all 0) in an inner-product space V and produces an orthonormal set {w 1,..., w m }, m n, such that L({w 1,..., w m }) = L({v 1,..., v n }), where L( ) denotes the linear span, i.e., the subspace obtained from all possible linear combinations of the elements of the corresponding set. In a naive implementation, the Gram-Schmidt procedure would compute as follows 2

3 (keeping only the nonzero w i s that are produced): u 1 = v 1 w 1 = u 1 / u 1 (or 0 if u 1 = 0) u 2 = v 2 proj w1 (v 2 ) w 2 = u 2 / u 2 (or 0 if u 2 = 0) u 3 = v 3 proj w1 (v 3 ) proj w2 (v 3 ) w 3 = u 3 / u 3 (or 0 if u 3 = 0). u i = v i i 1 k=1 proj w k (v i ) w i = u i / u i. (or 0 if u i = 0) Write a function (saving it in the file NAIVEGS.m ) function w = NAIVEGS(v) that implements the naive Gram-Schmidt procedure. The argument to the function is an n l matrix v whose rows are vectors in R l. The output is an m l matrix w whose rows are the desired orthonormal basis. Programming notes (in Matlab or Octave): 1. The number of rows of a matrix v can be obtained as size(v,1). 2. The ith row of a matrix v can be addressed as v(i, :). 3. The function zeros(size(x)) returns an all-zero array with the same shape as that of x. 4. A row can be appended to a matrix simply by assigning to it, e.g., w(i,:) = u(j,:) assigns the jth row of u to the ith row of w. Test your function by computing the following. A = NAIVEGS([1 1; 1 0]) B = NAIVEGS([ ; ]) C = NAIVEGS([ ; ; ; ]) 3

4 To verify that the resulting basis is in fact orthonormal, compute the inner product of each row of the basis with every other row of the basis. This is conveniently done in Matlab or Octave by computing, say, C C where C is a basis matrix. (Here C denotes the transpose of the matrix C.) If C is an m l matrix whose rows form an orthonormal basis, what matrix should result from the computation C C? Unfortunately, the naive Gram-Schmidt procedure is numerically unstable, which means that it is highly sensitive to numerical errors. For example, try running NAIVEGS on the 7th order Hilbert matrix: H7 = [ 1/1 1/2 1/3 1/4 1/5 1/6 1/7; 1/2 1/3 1/4 1/5 1/6 1/7 1/8; 1/3 1/4 1/5 1/6 1/7 1/8 1/9; 1/4 1/5 1/6 1/7 1/8 1/9 1/10; 1/5 1/6 1/7 1/8 1/9 1/10 1/11; 1/6 1/7 1/8 1/9 1/10 1/11 1/12; 1/7 1/8 1/9 1/10 1/11 1/12 1/13 ]; C = NAIVEGS(H7); Z = abs(c * C - eye(7)) In this computation, eye(7) denotes the 7 7 identity matrix, and abs computes the absolute value of each matrix component. Ideally, what should Z be? What is the actual largest entry in Z? Fortunately, the Gram-Schmidt procedure can be made numerically stable via a simple modification. Instead of computing i 1 u i = v i proj wk (v i ) as in the naive Gram-Schmidt algorithm, compute u i in a series of steps: k=1 u (1) i = v i proj w1 (v i ) 4

5 u (2) i = u (1) i proj w2 (u (1) i ) u (3) i = u (2) i proj w3 (u (2) i ).. u i = u (i 2) i proj wi 1 (u (i 2) i ) Copy your file NAIVEGS.m to the file GS.m and modify GS.m to implement the modified Gram-Schmidt procedure as the function function w = GS(v) Using your new modified GS procedure, try running: D = GS(H7); Z = abs(d * D - eye(7)) Now what is the largest entry in Z? 3 Orthogonal Projections Let P be a finite-dimensional subspace of an inner-product space V. Once an orthonormal basis {w 1, w 2,..., w m } for P is obtained, the orthogonal projection of an arbitrary vector v V on P is easily computed as ˆv = m v, w i w i. i=1 The vector ˆv is the orthogonal projection of v on P and the components of the m-tuple ( v, w 1, v, w 2,..., v, w m ) are the coordinates of ˆv with respect to the given orthonormal basis. Let B be an m l matrix whose rows form an orthonormal basis for subspace P. (For example, B might be produced by your GS function.) Let v be an arbitrary 1 l row vector, and let ˆv be the orthogonal projection of v on P. 5

6 Give a Matlab or Octave expression (in terms of B and v) for the coordinates of ˆv with respect to basis given in the rows of B. Give a Matlab or Octave expression (in terms of B and v) for ˆv. For example, let P be the subspace of R 3 consisting of all vectors (x, y, z) with the property that x + y + z = 0. Find an orthonormal basis for P and compute the orthogonal projection of the following vectors on P. v = [1,2,3], ˆv = v = [1,1,1], ˆv = v = [0,1,0], ˆv = In the above computations, let e denote the error vector v ˆv. Compute v, v, ˆv, ˆv and e, e for the three cases above. Write a general relationship among the three quantities v, v, ˆv, ˆv and e, e, where e = v ˆv. 6

7 4 The QR Decomposition Let V be a square matrix (assumed invertible) and let Q = GS(V) be the orthogonal matrix returned by your Gram-Schmidt procedure. Each row of V has a set of coordinates with respect to the rows of Q. This implies that for some matrix R. V = RQ, After trying a few examples, explain what special property the matrix R has. random n n matrix can be obtained in Matlab or Octave via rand(n, n). A In linear algebra, this decomposition is usually written in transposed form, expressing an invertible matrix V as V = QR where Q is an orthogonal matrix and R has a similar special property as above. Such a decomposition is referred to as the QR decomposition. Thus we see that the Gram-Schmidt procedure results in a QR decomposition. By typing help qr, investigate whether your version of Matlab or Octave implements the QR decomposition. 5 Approximate Curve Fitting Consider the approximation of a function f(x) over the interval [0, 1] as a quadratic polynomial. Here we show how this can be done approximately using orthogonal projections. In Matlab or Octave, run the following: N = 9; a = linspace(0,1,n); v(1,:) = ones(size(a)); v(2,:) = a; v(3,:) = a.* a; b = GS(v); 7

8 The linspace(a,b,n) function returns a vector of length N with samples uniformly spaced from a to b, inclusive. The.* operator takes the componentwise product of the corresponding vectors. The matrix v now contains three rows: the first row contains samples from the constant function; the second row contains samples from a linearly increasing function; and the third row contains samples from a quadratically increasing function. The matrix b contains a corresponding orthonormal basis. To see the corresponding samples, try the following command: plot(b(1,:)); hold; plot(b(2,:)); plot(b(3,:)); Let us populate a vector f with samples from some function, e.g., f = exp(a); Let g denote the orthogonal projection of f on the space spanned by the rows of v or b. Both f and g can be plotted as follows: hold off; plot(f); hold; plot(g); Express g as a linear combination of the rows of v. (N.B.: we are not asking for the coordinates of g with respect to the orthonormal basis, but rather with respect to the original basis.) A more accurate representation can be obtained by increasing the number of samples. Repeat this exercise for N = 100. When N = 100, the coordinates of g with respect to the rows of v are: 8