LECTURE 8: ORTHOGONALITY (CHAPTER 5 IN THE BOOK)

Transcription

1 LECTURE 8: ORTHOGONALITY (CHAPTER 5 IN THE BOOK) Everythig marked by is ot required by the course syllabus I this lecture, all vector spaces is over the real umber R. All vectors i R is viewed as a colum vector. 1. Ier proudcut Defiitio 1.1. A ier product o a real vector space V is the followig: For a pair of vectors u, v V we associate a real umber u, v ad it satisfies (1) u, u 0 with equality if ad oly if u = 0. (Positive) (2) u, v = v, u (Symmetric) (3) αu + βv, w = α u, w + β v, z for all scaler α ad β. (Liear) A vector space with a ier product is called a ier product space. Defiitio 1.2. We defie the legth or orm of a vector v by (the o-egative umber) v = v, v. Example 1.3 (Stadard ier product for R ). Defie the stadard ier product o R by u, v = u T v. I R 2,, is the usual ier product of vectors ad x = x x2 2 is just the usual defiio of the legth of a vector x = (x 1, x 2 ) R 2. Example 1.4. Recall that the space Mat m (R) of m matrices is a vector space. Defie ier product by m A, B = a ij b ij for A = (a ij ), B = (b ij ) Mat m. j=1 Example 1.5 ( space of fuctios). Defie ier product o the space C([a, b]) of cotious fuctios by f, g = a b f(x)g(x)dx Exercise ( see the book.). Check that f, f = 0 if ad oly if f(x) = 0 for all x [a, b]. Example 1.6. Let x 1, x 2,, x be distict real umbers. For each pair of polyomials i P, defie product, p, q = p(x i )q(x i ). Defiitio 1.7. Two vectors u ad v are said to be orthogoal if u, v = 0. If u ad v are orthogoal to eachother, we write u v. Example 1.8. (a) 0 is orthogoal to every vector i V. (b) 0 is the oly vector i V that is orthogoal to itself. 1

2 2 LECTURE 8-9: ORTHOGONALITY (1) Note that We get u + v 2 = u + v, u + v = u, u + v, u + u, v + v, v = u u, v + v 2. Theorem 1.9 (The Pythagorea Law). If u ad v are orthogoal vectors i a ier product space V, the u + v 2 = u 2 + v 2. I R 2, this is the familiar Pythagorea theorem, see followig picture. u + v u v 2. Projectio Defiitio 2.1. If u ad v are vectors i a ier product space V ad v 0, the the scaler projectio of u oto v is give by α = u, v v ad the (vector) projectio of u oto v is give by p = α ( 1 u, v v) = v v, v v. Notatio. We deote the projectio p of u to v 0 by proj v (u). The defiig property of the projectio. If v 0, the the (vector) projectio p of u oto v is the uique vector satisfies followig two coditios: (i) p is a scaler multiple of v. (ii) u p is orthogoal to v (hece also orthogoal to p). Suppose p is a vector satisfies the above coditio. The by coditio (i), p = βv for some β R. O the other had, (ii) meas Sice v 0, v, v 0. Hece we get 0 = u p, v = u, v βv, v = u, v β v, v. β = u, v v, v, ad u, v p = v, v v Remark: From the above discussio, we see that u = proj v (u) if ad oly if u is a scaler multiple of v. Theorem 2.2 (The Cauchy-Schwarz Iequality). If u ad v are ay two vectors i a ier product space V, the u, v u v Equality holds if ad oly if u ad v are liearly depedet. More simply stated, equality will hold if ad oly if u ad v are liearly depedet.

3 Proof. If v = 0, the the iequality is trivial, sice LECTURE 8-9: ORTHOGONALITY 3 u, v = 0 = u v. If v 0, the let p be the vector projectio of u otov. Sice p is orthogoal to u p, it follows from the Pythagorea law that 2 u, v ( ) ( v ) = p 2 = u 2 u p 2 u 2 Therefore, u, v u v. Moreover, the equality hold i eq. ( ) if ad oly u p = 0, i.e. u = p. Therefore u is a scalar multiple of v. As a cosequece of Cauchy-Schwarz iequality is that if u ad v are ozero vectors, the u, v 1 u v 1. Hece there is a uique agle θ i [0, π] such that cos θ = u, v u v, ad the agle two ozero vectors u ad v is defied to be this θ. Example 2.3. Cauchy-Schwarz iequality gives lots of useful iequality. Here is a example. Cosider the stadard ier product o R. Let u = (1, 1,, 1) ad x = (x 1,, x ). The Cauchy-Schwarz iequality gives x i = u, x x 2 i. O other words, x x x x2 ad the equality holds if ad oly if x i are all the same. Defiitio 2.4. A vector space V is said to be a ormed liear space if, to each vector v V, there is associated a real umber v, called the orm of v, satisfyig (i) v 0 with equality if ad oly if v = 0. (ii) αv = α v for ay α R. (iii) u + v u + v for all v, u V. (triagle iequality) Theorem 2.5. If V is a ier product space, the defies a orm o V. v = v, v. Proof. Part (i) ad (ii) i Defiitio 2.4 is easy. We check part (iii): Thus u + v 2 = u u, v + v 2 u u v + v 2 =( u + v ) 2 u + v u + v (Cauchy Schwarz)

4 4 LECTURE 8-9: ORTHOGONALITY Remark: There are other orms ot comig from ay ier products. For example, for every x = (x 1,, x ) R, defie x 1 = x i ad x = max 1 i x i. The 1 ad are also orms but they are ot realted to ay ier product. 3. Orthogoal Subspaces Let V be a vector space with ier product,. Defiitio 3.1. Two subspaces X ad Y of V are said to be orthogoal if x, y = 0 for every x X ad every y Y. If X ad Y are orthogoal, we write X Y Example 3.2. I R 3 with stadard ier product. Let X = Spae 1 ad Y = Spae 2. The X Y. This is because, for ay α, β R, Lemma 3.3. If X Y, the X Y = 0. αe 1, βe 2 = (αe T 1 )(βe 2 ) = 0. Proof. If x X Y, the x, x = 0 ad hece x = 0. Notatio (The sum of two subspaces). Let X ad Y be two subspaces of V, defie a subspace of V X + Y = { x + y x X ad y Y }. Exercise. Check that X + Y is a subspace of V. Lemma 3.4. If X Y = 0, the for each vector w X + Y there is a uique way two write w = x + y such that x X ad y Y. Proof. Sice w X + Y, there is a x X ad y Y such that w = x + y by defiitio. Suppose w = x + y is aother expressio such that x X ad y Y. We get x x = y y. The left hadside of the above equatio is i X ad the right hadside is i Y. Hece both sides are i X Y = 0. Hece x x = 0 ad y y = 0. This fiished the proof. Defiitio 3.5 (The direct sum of two subspaces). Let X ad Y be two subspace of V. If X Y = 0 is the zero spaces, the X + Y is called a direct sum, ad writte as X Y. Defiitio 3.6. Let X be a subspace of V. The set of all vectors i V that are orthogoal to every vector i X will be deoted X. Thus, X = { v V v, x = 0 for every x X }. The set X is called the orthogoal complemet of X. Lemma 3.7. X is a subspace of V. Proof. Let y, z X ad α R. We have αy, x =α y, x = 0 ad y + z, x = y, x + z, x = for all x X, i.e. αy X ad y + z X. Therefore X is a subspace of R. Example 3.8. It is clear that (c.f. Example 1.8). (1) 0 = V

5 (2) V = 0. LECTURE 8-9: ORTHOGONALITY 5 The followig lemma is true for ay fiite dimesioal ier product space, however we will first prove the versio i R with the stadard ier product. Theorem 3.9. Let X be a subspace of V, the X + X = V. Combie this with X X = 0, we get V = X X. Corollary If X is a subspace of V, the (X ) = X. Proof. O the oe had, if x X, the x is orthogoal to each y X. Therefore, x (X ) ad hece X (X ). O the other had, suppose that z is a arbitrary elemet i (X ). By Theorem 3.9, we ca write z as a sum u + v, where u X ad v X. Sice v X, we get v u ad v z. Therefore, 0 = z, v = u + v, v = u, v + v, v = 0 + v, v. ad, cosequtly, v = 0. Hece z = u X ad so X = (X ). 4. The stadard ier product o R I this sectio, we cosider the vector space R with stadard ier product give by x, y = x T y for all x, y R. Let A be a m matrix. It is also viewed as a liear trasformatio from R to R m. Notatio. The colum space of A is the image of the liear map A, deoted by R(A) = { b R m b = Ax for some x R }. Similarly, the row space is the colum space of A T i R, i.e. the space R(A T ) = { y R y = A T x for some x R m }. Theorem 4.1 (Fudametal Space Theorem). If A is a m matrix, the N(A) = R(A T ) ad N(A T ) = R(A). Proof. For y = A T c R(A) with c R m, ad x N(A), we have (2) y, x = (A T c) T x = c T Ax. Sice Ax = 0, we get y, x = c T 0 = 0, i.e. x y. Hece N(A) R(A). O the other had, if x R(A), the eq. (2) implies c Ax for all c R m. Hece, Ax = 0, i.e. x N(A). Therefore, N(A) = R(A T ). The reuslts for A T is immediat if oe repace the role of A by A T. Theorem 4.2. If X is a subspace of R. The dim X + dim X = Furthermore, if { x 1,, x r } is a basis for X ad { x r+1,, x } is a basis for X, the { x 1,, x r, x r+1,, x } is a basis for R. Proof. Suppose X = 0 is the zero space, the X = R ad dim X + dim X = 0 + =. Suppose X 0, the let { x 1,, x r } be a basis of X. Defie x T 1 A = x T r

6 6 LECTURE 8-9: ORTHOGONALITY to be the r matrix whose i-th row is x T i. By costructio R(A T ) = X ad N(A) = X. So dim X = rak(a) ad dim X = ull(a). Now by the Nullity-Rak theorem, we get dim X + dim X = rak(a) + (A) =. To show that { x 1,, x r, x r+1,, x } is a basis it is suffice to show that the vectors are liearly idepeedet. Suppose that, for some c i R, c i x i = 0. Let y = r c i x i ad y = i=r+1 c i x 2. The y X ad z X. Sice X X, we see that each vector i X + X has a uique decompusitio, i particular, = 0 it the uique decompositio of 0. Hece, y = 0 ad z = 0. Now by the liearly idepedecey of { x 1, x r } ad { x r+1, x } we coclude c i = 0 for all i. Hece we showed that the vectors are liearly idepeedet. This fiished the proof. Theorem 4.3. If X is a subspace of R, the R = X + X ad hece R = X X. Proof. This is clear by Theorem 4.2, sice the uio of a pair of baises of X ad X will form a basis of R. Corollary 4.4. We have (X ) = X. (c.f. Corollary 3.10) Corollary 4.5. If A is a m matrix ad b R m, the either there is a vector x R such that Ax = b (i.e. b R(A)) or there is a vector y R m ( y R(A) = N(A T ))such that A T y = 0 ad y T b 0 (b / R(A) = (N(A T )) ) Defiitios. 5. Orthoormal basis Defiitio 5.1. Let v 1, v 2,, v be ozero vectors i a ier product space V. If v i, v j = 0 wheever i j, the { v 1, v 2,, v } is said to be a orthogoal set of vectors. Lemma 5.2. If { v 1, v 2,, v } is a orthogoal set of ozero vectors i a ier product space V, the v 1, v 2,, v are liearly idepedet. Proof. Suppose that c 1 v 1 + c 2 v c v = 0. Take the ier product of the equatio with v j for a 1 j. we get Hece c j = 0. 0 = c 1 j, v 1 + c 2 j, v c j, v = c j v j 2. Defiitio 5.3. A vector v V is called a uit vector if v = 1. A orthoormal set of vectors is a orthogoal set of uit vectors. Let { u 1, u } be a orthoormal set, the u i, u j = δ ij = { 1 i = j 0 i j. Defiitio 5.4. A basis B of a ier product space V is called a orthoormal basis if B is also a orthoormal set.

7 LECTURE 8-9: ORTHOGONALITY 7 Example 5.5. Let S = { u 1, u 2,, u k } be a orthoormal set i V. The S is a orthoormal basis of Spau 1, u 2,, u k (Note that a orthoormal set is liearly idepedet) Example 5.6. The vectors u 1 = ( 1 2, 1 2 ) ad u 2 = ( 1 2, 1 2 ) form a orthoormal basis for R 2. Lemma 5.7. Let { u 1, u } be a orthoormal basis for a ier product space V. (i) If v = c i u i the c i = v, u i. (ii) If v = c i u i ad w = d i u i, the v, w = (iii) (Parseval s Formula) If v = c i u i the Proof. (i) v, u i = c j u j, u i = j=1 (ii) v, w = v, d i u i = (iii) v 2 = v, v = c 2 i. j=1 v, u i d i = v 2 = c i d i. c 2 i. c j u j, u i = c i d i j=1 c j δ ji = c i. Remark: Recall that, give a basis B, we will get a liear isomorphism R V. (So V ca be idetified with R ) Now Lemma 5.7 says that, whe B is a orthoormal basis, the ier product structure o V is same as the stadard ier product structure o R via this isomorphism. (So V ca ba idetified with R equipped with the stadard ier product) More precisely, let, V ad, R deote the ier product o V ad R respectively, the v, w V = [v] B, [w] B R. I the Sectio 6, we will see that orthoormal basis always exists Orthogoal Matrices. Defiitio 5.8. A matrix Q is said to be a orthogoal matrix if Q T Q = I or equivaletly, the colum vectors of Q form a orthoormal set i R. Let q i be the i-th colum of Q. The equivalece is clear, sice { q 1,, q } is a orthoormal set if ad oly if q i, q j = q T i q j = δ ij ad q T i q j is the (i, j)-th etry of Q T Q. Immediately, from the defiitio, we get. Lemma 5.9. Let Q be a matrix. The Q is a orthoormal matrix if ad oly if Q is ivertible ad Q 1 = Q T. Example Fix θ R, the matrix Q = ( cos θ si θ si θ cos θ )

8 8 LECTURE 8-9: ORTHOGONALITY is a orthogoal matrix ad Q 1 = Q T = ( cos θ si θ si θ cos θ ) I fact, all 2 2 orthoormal matrices are i this form. Note that Q represet the trasitio matrix from the basis { q 1, q } to the stadard basis { e 1,, e 2 }, the Lemma 5.7 implies that the stadard ier product is preserved by left multiplicatio of Q. This is also ca be see directly from the followig calculatio: for x, y R, We collect Qx, Qy = (Qx T )Qy = x T Q T Qy = x T Iy = x T y = x, y. Lemma 5.11 (Properties of Orthogoal Matrices). Let Q be a matrix. The the followig are equivalet, (a) the colum vectors of Q form a orthoormal basis for R ; (b) Q T Q = I; (c) Q is ivertiable ad Q T = Q 1 ; (d) Qx, Qy = x, y ; (e) Qx = x. Whe Q satifies oe of the above coditio, it is called a orthogoal matrix. Proof. The equivalece of (a), (b) ad (c) are clear. We already see that (b) implies (d). Now suppose (d) hold, substitue e i ad e j i the formula, we get q i, q j = Qe i, Qe j = e i, e j = δ ij, i.e. the colums of Q form a orthoormal set i R. Therefore (a) holds. Now cosdier the equivalece betwee (d) ad (e). Clearly, (d) implies (e). O the other had, (c.f. eq. (1)) Hece, (e) impies (d), sice x, y = ( x + y 2 x 2 y 2 )/2. Qx, Qy = ( Q(x + y) 2 Qx 2 Qy 2 )/2 = ( x + y 2 x 2 y 2 )/2 = x, y. Remark: Cosider the the set, called the orthogoal group of dimesio, O() = { Q Mat (R) Q is a orthogoal matrix }. The above lemmas says that O() is exactly the set of matrices such that multiply with the matrix preserves the ier product (or preserve the orm). Exercise. Recall the defiitio of Group (i Lecture 5 Remark after the Defiitio 1.1). Show that O() is a group. I fact, O(1) = { ±1 } ad O(2) are commutative groups. But O() is o-commutative whe 3. (c.f. Tutorial 3, Questio 2)

9 LECTURE 8-9: ORTHOGONALITY Permutatio Matrix. Let σ be a permutatio of { 1, 2,, }. (c.f. i Defiitio 2.1 Lecture 4) I short, σ permutes the tuple of ordered umber (1, 2,, ) to the tuple (σ(1), σ(2),, σ()) by sed umber i to umber σ(i). Defie a matrix P σ correspodig to σ by let P σ = (e σ(1) e σ(2) e σ(3) e σ() ). The matrix P σ is clearly a orthogoal matrix. Moreover, P σ have the followig properites: the right multiplicatio by P σ permutes the colums accordig to σ, i.e. ad Let Q σ = Pσ T have (a 1 a 2 a ) P σ = (a σ(1) a σ(2) a σ() ) P 1 σ = P T σ = e T σ(1) e T σ(2) e T σ() be the matrix obtaied by permutig the rows accordig to σ. The we (3) Q σ b T 1 b T 2 b T = b T σ(1) b T σ(2) b T σ() Let τ be aother permutatio of { 1, 2,, } the we have σ τ is also a permutatio. Moreover, it is easy to check by eq. (3) that Q σ Q τ = Q σ τ. Remark: Recall that the set S of all permutatios of { 1, 2,, } form a group. The above equatio meas that the map S O() defied by σ Q σ is a group homomorphism Projectio to a subspace. Defiitio Let S be a subspace of a ier product space V ad let x V. Let { u 1,, u } be a orthoormal basis for S. The projectio of x oto S is defied to be proj S (x) = p =. c i u i with c i = x, u i. Theorem Uder the assumptio of Defiitio 5.12, x p S. Proof. First (x p) u i for each i: If y S, the y = α i u i. Hece, Therefore x p S. x p, u i = x, u i p, u i = x, u i c j u j, u i =c i j=i x p, y = x p, α i u i = j=1 c j u j, u i = 0 α i x p, u i = 0,

10 10 LECTURE 8-9: ORTHOGONALITY Remark: The projectio proj S (x) of x is characterised by the property i Theorem 5.13: it is the uique vector p i S such that x p S. I particular, proj S (x) is idepedet of the choice of orthoormal basis for S. (ote that we will show i Sectio 6 that there always exists a orthoormal basis.) O the other had, proj S (x) is also characterised by the fact that it is the elemet i S closest to x: Theorem Uder the assumptio of Defiitio 5.12, p = proj S (x) is the elemet i S that is closest to x, that is Proof. If y S ad y p, the x y 2 = (x p) + (p y) 2 = x p 2 + p y 2 x y > x p for ay p y S. > x p 2. (sice p y 0) Now let s calculate the projectio oto a subspace S i R (sice p y ad so (x p) (p y)) Corollary Let S be a ozero subspace of R ad let x R. If { u 1, u 2,, u k } is a orthoormal basis for S ad U = (u 1 u 2 u k ), the the projectio p of x oto S is give by proj S (x) = UU T x. Proof. By defiitio with Hece proj S (x) = UU T x. proj S (x) = Uc = x, u i. c 1 u T 1 x c c = 2 u = T 2 x = U T x. u T x c 5.5. Least Squares Problems. Let A be a m matrix. Cosider a system of liear equatio Ax = b. For each x R, form a residual: r(x) = b Ax. If the equatio Ax = b has a solutio ˆx, the r(ˆx) = 0. I geeral, the equatio may ot have a solutio. I this case, we ask a vector ˆx such that the legth of the residual r(x) is miimized. Such vector ˆx is called a least squares solutio of the system Ax = b. Cosider the space S spaed by the colum vector of A, the clearly, S = { Ax x R }. By the remark after Theorem 5.13, r(x) = b Ax take miimal value exactly whe Ax is the projectio of b oto S, i.e. b Ax is orthogoal to all vectors i S. Therefore, for each y R, we get a equatio: 0 = Ay, b Ax = y T (A T b A T Ax). Sice y is arbitray, we get a equatio called the ormal equatio A T Ax = A T b.

11 LECTURE 8-9: ORTHOGONALITY 11 Theorem If A is a m matrix of rak, the ormal equatio A T Ax = A T b has a uique solutio ˆx = (A T A) 1 A T b. Proof. It suffice to prove that A T A is ivertible. We claim that: for ay matrix A, N(A) = N(A T A). First, it is clear that N(A) N(A T A). O the other had, if x N(A T A), the Ax 2 = x T (A T Ax) = 0. But this implies Ax = 0, i.e. x N(A). This proves the claim. Now back to the proof of the theorem. We have rak(a) =. Therefore, N(A) = 0, sice ull(a) = rak(a) = 0 by the ullity-rak theorem. By the above claim, we get N(A T A) = 0. Hece A T A is ivertible sice it is a square matrix. Example Give a table of data Fid a quadratic fuctio x x 1 x 2 x m y y 1 y 2 y m y = c 0 + c 1 x + c 2 x 2 that best fits the data i the least square sece. This gives a system of equatios with ukows c 0, c 1 ad c 2 : 1 x 1 x x 2 x 2 1 c 0 y 1 y c 1 = 2 1 x m x 2 c 2 m y m Now oe ca use Theorem 5.16 to get the least square fit. 6. Gram-Schmidt Orthogoalizatio Process I this sectio we give a algorithm for costructig a orthoormal basis form a (ordiary) basis { x 1, x 2,, x }. Theorem 6.1. Let { x 1, x 2,, x } be a basis for a ier product space V. Let ad defie u 2,, u recursively by where u k+1 = u 1 = 1 x 1 x 1 1 x k+1 p k (x k+1 p k ) fork = 1,, 1 p k = proj Xk (x k+1 ) = x k+1, u 1 u 1 + x k+1, u 2 u x k+1, u k u k is the projectio of x k+1 oto X k = Spa { u 1, u 2,, u k }. The the set is a orthoormal basis for V. Moreover, { u 1, u 2,, u } X k = Spa { u 1, u 2,, u k } = Spa { x 1, x 2,, x k } fork = 1,,.

12 12 LECTURE 8-9: ORTHOGONALITY Proof. We prove iductively. Clearly, X 1 = Spa { u 1 } = Spa { x 1 } ad u 1 is a uit vector, i.e. { u 1 } is a orthoormal basis of X 1. Now suppose that u 1 1, u 2,, u k have bee costructed so that { u 1, u 2,, u k } is a orthoormal set ad X k = Spa { u 1, u 2,, u k } = Spa { x 1, x 2,, x k }. Sice p k Spa { u 1, u 2,, u k } = Spa { x 1, x 2,, x k }, it follows that x k+1 p k Spa { x 1, x 2,, x k+1 }. Sice x 1,, x k+1 are liearly idepedet, it follows that x k+1 p k is ozero. O the other had, by Theorem 5.13, x k+1 p k X k, i.e. it is orthogoal to each u i, 1 i k. Thus, { u 1, u 2,, u k+1 } is a orthoormal set of vectors i Spa { x 1, x 2,, x k+1 }. Sice u 1,, u k+1 are liearly idepedet, they form a basis for Spa { x 1, x 2,, x k+1 } ad, cosequetly, X k+1 = Spa { u 1, u 2,, u k+1 } = Spa { x 1, x 2,, x k+1 }. It follows by methematical iductio that { u 1, u 2,, u } is a orthoormal basis of V = Spa { x 1, x 2,, x }. Theorem 6.2 (Gram-Schmidt QR Factorizatio). If A is a m matrix of rak, the A ca be factored ito a product QR, where Q is a m matrix with orthoormal colum vectors ad R is a upper triagular matrix whose diagoal etries are all positive. Proof. Let a 1, a 2,, a be the colums of A. Sice rak(a) =, { a 1, a 2,, a } from a basis of the colums space R(A) of A. Apply the Gram-Schmidt process o the basis { a 1, a 2,, a }. Let p 1,, p 1 be the projectio vectors ad { q 1, q 2,, q } be the orthoormal basis derived for the process. Defie r 11 = a 1. r kk = a k a k 1 for k = 2,, r ik = a k, q i for i = 1,, k 1 ad k = 2,,. By the Gram-Schmidt process, we have This is equivalet to r 11 q 1 =a 1 rr kk q k =a k r 1k q 1 r 2k q 2 r k 1,k q k for k = 2,, Let a 1 =r 11 q 1 a =r 12 q 1 + r 22 q 2 ad let R to be the upper triagular matrix a =r 1 q 1 + r 2 q r 1, q Q = (q 1 q 2 q ) r 11 r 12 r 1 0 r R = 22 r r

13 The we get LECTURE 8-9: ORTHOGONALITY 13 QR = A Remark: I the fractrizatio, R must be osigular, otherwise rak(a) = rak(qr) rak(r) < which cotradicte to rak(a) =. Moreover, it is ot hard to see that the QR dcompositio is uique. Here is a applicatio of QR Factorizatio. Example 6.3. Let A be a m matrix of rak. The the least square solutio of Ax = b is give by ˆx = R 1 Q T b, where Q ad R are matrices obtaied from the factrizatio give i Theorem 6.2. This is clear, by substitue A = QR ad the fact that Q T Q = I : ˆx =(A T A) 1 A T b = ((QR) T (QR)) 1 (QR) T b =(R T Q T QR) 1 R T Q T b = (R T I R) 1 R T Q T b =R 1 (R T ) 1 R T Q T b =R 1 Q T b.