(VII.A) Review of Orthogonality

Transcription

1 VII.A Review of Orthogoality At the begiig of our study of liear trasformatios i we briefly discussed projectios, rotatios ad projectios. I III.A, projectios were treated i the abstract ad without regard as to whether they were orthogoal ; while i III.B, the examples of rotatios ad projectios i R ad R 3 made use of a orthogoal basis. The idea was that if you wat say to rotate about ê, or project perpedicularly to spa{ê, ê 3 } i R 3, you ca write immediately [R]ê = cos θ si θ ad P x = x ê }{{} ê + x ê }{{} 3 ê 3 si θ cos θ =x =x 3. The same formulas hold for ay basis B = { v, v, v 3 } of R 3 like ê i the sese that v i v j = δ ij i.e. the vectors are of uit legth ad satisfy v v, v v 3, v v 3. But for a arbitrary basis ot like ê, these formulas produce elliptical rotatios ad skew projectios. So what to do whe you eed to rotate about i R 3, or project perpedicularly to the spa of,, 8 i 6 R 4? You eed to costruct the right basis, oe like ê. We shall ow stadardize these ideas rather tha cotiuig i the ad hoc vei of III.A-B. I this sectio, we will stick with the dot product o R ad its geeralizatio to C, while subsequet oes will cosider more geeral biliear forms.

2 VII.A REVIEW OF ORTHOGONALITY Orthoormal bases ad Projectios. DEFINITION. Give v, w R, we write v = v v = orm or legth of v ; ad v w v w = v ad w are orthogoal. DEFINITION. A basis B = v,..., v for R is called orthogoal if v i v j = for i = j. If i additio v i v i = i =,..., the the basis is called orthoormal. For such a basis the rotatio about spa{ v 3,..., v }, for example, is the give by cos θ si θ si θ cos θ [R]ê = S B [R] B SB where [R] B =.... More importatly, we claim that the orthogoal projectio to W = spa{ v r,..., v } is give by P r x = v r x v r v x v = i=r v i x v i. Clearly P r x W, but we must also check that x P x W : x x Px Px W

3 VII.A REVIEW OF ORTHOGONALITY 3 It suffices, of course, to show x P x v j for each j = r,..., : x P x v j = x v i x v i v j = x v j i=r v i x v i v j i=r }{{} =δ ij = x v j v j x =. Settig r = so that W = R ad P x = x we have x = v x v + + v x v wheever { v,..., v } is a orthoormal basis. This is a sort of fiite-dimesioal Fourier expasio, ad the { v i x} are sometimes called Fourier coefficiets. A immediate cosequece of is the Pythagorea theorem which implies x = x x = v i x v j x v i v j = i,j= }{{} δ ij i= v i x. x = v x v x v r x v x = P r x ; that is, orthogoal projectio caot icrease the orm of x R. I particular, give ay y R with spa L, x P L x = y y x y y = x that is, the Cauchy-Schwarz iequality x y x y y y y y x y = y y ; holds for the dot product. Thus we may exted the otio of agle betwee x ad y from R to R : takig x y θ x, y = arccos x y makes sese, sice the argumet is betwee ±.

4 4 VII.A REVIEW OF ORTHOGONALITY Gram-Schmidt Orthogoalizatio. Suppose we are give a arbitrary basis { w,..., w k } for a subspace W R. Here is how to tur it ito a orthoormal oe: Begi by ormalizig w : set the hat idicates a uit vector. ˆv := w w w w v Referrig to the above picture, we would like to make w ˆv by w gettig rid of its horizotal compoet w ˆv ˆv : w := w w ˆv ˆv ormalize ˆv := w w. To make w 3 to both ˆv ad ˆv, we take w 3 := w 3 w 3 ˆv ˆv w 3 ˆv ˆv ˆv 3 := w 3 w 3, ad so o. Specialize to the case k = W = R ad rewrite the equatios relatig the ˆv s ad ŵ s as follows: w = w ˆv w = w ˆv ˆv + w = w ˆv ˆv + w ˆv w 3 = w 3 ˆv ˆv + w 3 ˆv ˆv + w 3 ˆv 3, etc. This looks very ice i matrix terms: W = V M or w w = ˆv ˆv w w ˆv w 3 ˆv w r v w w 3 ˆv w r ˆv w wr ˆv r. w

5 VII.A REVIEW OF ORTHOGONALITY Here we have take ay ivertible matrix = the left-had side ad writte it as a product of a matrix with orthoormal colums ad a upper-triagular matrix. This is called the QR-decompositio where Q is V ad R is M, or i a broader cotext the Iwasawa decompositio for GL R [= ivertible matrices]. We will establish some properties of V i a bit; i the meatime, provided you believe that det V = ± aother way i which a orthoormal basis is like ê, this decompositio of W give a very ice secod proof that det W = vol{parallelepiped with edges w,..., w }. What eeds to be show is that the product det M = w w w 3 w gives the parallelepiped s volume. Writig W r = spa{ w,..., w r } = spa{ ˆv,..., ˆv r } for the orthogoal projectio to W r, this product becomes w w P w w 3 P w 3 w P w. But this is just the geeralizatio of Volume = Base Height = w w P w w 3 P w 3 as show i this picture w 3 project w 3 project w w is projected to w =w P w w is projected to w =w P w w

6 6 VII.A REVIEW OF ORTHOGONALITY to higher dimesios. Orthogoal matrices. There is a ice algebraic coditio o a matrix equivalet to the statemet that its colums form a orthoormal basis of R. Oe otices that if ˆv,..., ˆv are orthoormal the ˆv. ˆv ˆv =... ; ˆv that is, t V V = I. Clearly the coverse also holds. DEFINITION 3. Q M R is called orthogoal if For such a matrix, t QQ = I = Q t Q. = det I = det t Q det Q = det Q = det Q = ±. REMARK 4. This is, of course, the kid of matrix appearig i the QR-decompositio above. Notice that if oe applies Gram-Schmidt to the colums of a matrix A to fid the colums of Q, we ca obtai R immediately from A = QR = R = t QQR = t QA. Orthogoal trasformatios. Oe ca show that the correspodig liear trasformatios are compositios of rotatios ad reflectios. But here is the stadard formal DEFINITION. A liear trasformatio T : R R is orthogoal if it preserves legth, T x = x for all x R. Now for ay T, let A = [T]ê, so that the colums of A are the Tê i. If these are a orthoormal basis for R, the T x = Tx ê x ê = x Tê x Tê the formal proof is by iductio, where the volume formula for dimesios the iductive hypothesis takes care of the base, ad the added dimesio is characterized as height= w P w.

7 VII.A REVIEW OF ORTHOGONALITY = x Tê x Tê = x x = x. So if [T]ê is orthogoal the T is. Coversely if T is orthogoal the Tê i = ê i =, while = ê i + ê j = ê i + ê j ê i + ê j = ê i + ê j = = Tê i + ê j = Tê i + Tê j = Tê i + Tê j Tê i + Tê j = Tê i + Tê i Tê j + Tê j = + Tê i Tê j = Tê i Tê j =. CONCLUSION. T a orthogoal trasformatio A = [T]ê a orthogoal matrix. We ow address the computatioal problems rotatio ad projectio poited out at the begiig of the sectio. EXAMPLE 6. How to fid the matrix with respect to ê of rotatio by 3 about ker i R 3. First of all, { = spa, Perform Gram-Schmidt o these last two vectors: w = ˆv = }. w = w = [ ] ˆv = 3 4 =. 4

8 8 VII.A REVIEW OF ORTHOGONALITY Now simply ormalize the rotatio axis to get ˆv 3 = 3 put B = { ˆv, ˆv, ˆv 3 } so that [R]ê = S B [R] B SB, where S B = ad [R] B = 3 3., ad EXAMPLE. How to fid the matrix with respect to ê of the projectio to W = spa,, 8 6 R4. Apply Gram-Schmidt to the spaig vectors to get a orthoormal basis for W : w = w = w = w 3 = 8 6 ˆv = ˆv = =.

9 w 3 = 8 6 VII.A REVIEW OF ORTHOGONALITY From the projectio formula ˆv 3 = P W x = x ˆv ˆv + x ˆv ˆv + x ˆv 3 ˆv 3 =. we ca ow evaluate P W o ê, ê, ê 3, ê 4 to get the colums of the matrix [P W ]ê. But there is a shortcut: writig V for the 4 3 matrix with colmus ˆv, ˆv, ˆv 3, [P W x]ê = V ˆv x ˆv x ˆv 3 x = V t V x = [P W ]ê = V t V = = Notice that from which is valid quite geerally it is trasparet that orthogoal projectios are always give i ê, or at least some o.. basis by symmetric matrices. As we shall see i VII.D, this is closely related to the fact that they are diagoalizable. Uitary trasformatios. All of the above geeralizes to C. Recall that i R we had the followig equivalet ways of writig the dot product: x y = x i y i = t x y, i=

10 VII.A REVIEW OF ORTHOGONALITY where the last is matrix multiplicatio. If x, y C we have the followig complex dot product x y = x i y i = t x y = x y i= where for ay matrix or vector idicates the cojugate traspose. The resultig orm x = x i x i = x i coicides with the absolute value of a complex umber i case = : a + bi = a bia + bi = a + b. Note also that x y = y x ad α x = α x. DEFINITION 8. A trasformatio T : C C is called uitary if T v = v for all v C ; this is the complex versio of orthogoal. I claim for such a T that the colums Tê i of [T]ê satisfy Tê i Tê j = δ ij. First of all sice T is uitary 3 Tê i Tê i = Tê i = ê i =, while takig i = j for ay α C 4 ê i + αê j = Tê i + αê j. Let s cosider the left- ad right-had sides of 4: l.h.s. = t ê i + αê j ê i + αê j = ê i + α ê j + ᾱê j ê i + αê i ê j }{{} if i =j r.h.s. = t Tê i + αtê j Tê i + αtê j = Tê i + α Tê j + ᾱtê j Tê i + αtê i Tê j.

11 EXERCISES Now usig 3 to cacel ê i + α ê j with Tê i + α Tê j, we are left with ᾱtê j Tê i + αtê i Tê j = for ay α C. Plug i α =, i to get the two equatios Tê j Tê i = Tê i Tê j, Tê j Tê i = Tê i Tê j which of course imply Tê i Tê j = i = j. What we have show is that the matrix of T satisfies t [T]ê [T]ê = I, which motivates the followig geeralizatio of orthogoal matrices to C : DEFINITION 9. A matrix U M C is uitary if U U = I. Notice that = det U det U = det U det U = det U = det U = which says det U lies o the uit circle i the complex plae. A uitary basis is oe like the colums of U = [T]ê : it satisfies t v i v j = v i v j = δ ij. Exercises Show that a orthogoal trasformatio of R preserves all agles. [Hit: use the fact that [T]ê = A is a orthogoal matrix.] Apply Gram-Schmidt to the colums of 6 M = to write M = QR where Q is orthogoal ad R is upper triagular. Use Remark 4!

12 VII.A REVIEW OF ORTHOGONALITY 3 Apply Gram-Schmidt to the set,, 3 i R 4. Writig W for their spa, fid the matrix of the orthogoal projectio to W i the stadard basis ê. 4 What value of b if ay will make the matrix uitary? A = +i b i i