5.1. The Rayleigh s quotient. Definition 49. Let A = A be a self-adjoint matrix. quotient is the function. R(x) = x,ax, for x = 0.

Transcription

1 40 RODICA D. COSTIN 5. The Rayleigh s priciple ad the i priciple for the eigevalues of a self-adjoit matrix Eigevalues of self-adjoit matrices are easy to calculate. This sectio shows how this is doe usig a imizatio, or imizatio procedure The Rayleigh s quotiet. Defiitio 49. Let A = A be a self-adjoit matrix. quotiet is the fuctio Note that R(x) = x,ax, for x = 0 x2 R(x) = x x,a x x = u,au where u = x x The Rayleigh s so i fact, it suffices to defie the Rayleigh s quotiet o uit vectors. The set of uit vectors i R (or i C ), is called the 1 dimesioal sphere i R (or i C ): S 1 F = {u F u =1} For example, the sphere i R 2 is the uit circle (it is a curve, it has dimesio 1), the sphere i R 3 is the uit sphere (it is a surface, it has dimesio 2); for higher dimesios we eed to use our imagiatio Extrema of the Rayleigh s quotiet Closed sets, bouded sets, compact sets. You probably kow very well the extreme value theorem for cotiuous fuctio o the real lie: Theorem 50. The extreme value theorem i dimesio oe. A fuctios f(x) which is cotiuous o a closed ad bouded iterval [a, b] has a imum value (ad a imum value) o [a, b]. To formulate a aalogue of this theorem i higher dimesios we eed to specify what we mea by a closed set ad by a bouded set. Defiitio 51. A set S is called closed if it cotais all its limit poits: if a sequece of poits i S, {x k } k S coverges, lim k = x, the the limit x is also i S. For example, the itervals [2, 6] ad [2, + ) are closed i R, but[2, 6) is ot closed. The closed uit disk {x R 2 x 1} is closed i R 2,butthe puctured disk {x R 2 0 < x 1} or the ope disk {x R 2 x < 1} are ot closed sets. Defiitio 52. A set S is called bouded if there is a umber larger tha all the legths of the vectors i S: there is M>0 so that x M for all x S.

2 SPECTRAL PROPERTIES OF SELF-AJOINT MATRICES 41 For example, the itervals [2, 6] ad [2, 6) are bouded i R, but[2, + ) is ot. The square {x R 2 x 1 < 1, ad x 2 < 1} is bouded i R 2,but the vertical strip {x R 2 x 1 < 1} is ot. Theorem 53. The extreme value theorem i fiite dimesios. A fuctios f(x) which is cotiuous o a closed ad bouded set S i R or C has a imum value (ad a imum value) o S. I ifiite dimesios Theorem 53 is ot true i this form. A more striget coditio o the set S is eeded to esure existece of extreme values of cotiuous fuctios o S (the set must be compact). It is ituitively clear (ad rigorously proved i mathematical aalysis) that ay sphere i F is a closed ad bouded set Miimum ad imum of the Rayleigh s quotiet. The Rayleigh s quotiet calculated o uit vectors is a quadratic polyomial, ad therefore it is a cotiuous fuctio o the uit sphere, ad therefore Propositio 54. The Rayleigh s quotiet has a absolute imum ad a imum. What happes if A is ot self-adjoit? Recall that the quadratic form x,ax has the same value if we replace A by its self-adjoit part, 1 2 (A+A ), therefore the Rayleigh s quotiet of A is the same as he Rayleigh s quotiet of the self-adjoit part of A (hece iformatio about A is lost). The extreme values of the Rayleigh s quotiet ca be easily liked to the eigevalues of the self-adjoit matrix A. To see this, diagoalize the quadratic form x,ax: cosider a uitary matrix U which diagoalizes A: U AU = Λ = diag(λ 1,...,λ ) I the ew coordiates y = U x we have x,ax = x,uλu x = U x, ΛU x = y, Λy = λ j y j 2 which together with x = Uy = y gives for the Rayleigh s quotiet (37) R(x) =R(U y) = λ j y j 2 y j 2 y 2 = λ j y 2 R U(y) Sice A is self-adjoit, its eigevalues λ j are real; assume them ordered i a icreasig sequece: λ 1 λ 2... λ

3 42 RODICA D. COSTIN ad The clearly therefore λ j y j 2 λ λ j y j 2 λ 1 y j 2 = λ y 2 y j 2 = λ 1 y 2 λ 1 R(x) λ for all x = 0 Equalities are attaied sice R U (e 1 ) = λ 1 ad R U (e ) = λ. Goig to coordiates x imum is attaied for x = Ue 1 = v 1 = eigevector correspodig to λ 1 sice R(v 1 ) = R U (e 1 ) = λ 1, ad for x = Uv = v = eigevector correspodig to λ, imum is attaied sice R(v )= R U (e )=λ.thisproves: Theorem 55. If A is a self-adjoit matrix the ad x,ax x 2 = λ the eigevalue of A, attaied for x = v x,ax x 2 = λ 1 the eigevalue of A, attaied for x = v 1 As a importat cosequece i umerical calculatios: the imum eigevalue of A ca be foud by solvig a imizatio problem, ad the imum eigevalue - by a imizatio problem The i priciple. Reducig the dimesio of A we ca fid all the eigevalues, oe by oe. This reductio of the dimesio is doe usig: Spittig of the space uder the actio of a self-adjoit matrix. Ay matrix leaves ivariat its eigespaces. Sice eigevectors of a self-adjoit matrix A form a orthogoal set, the self-adjoit matrices leaves ivariat their orthogoal complemet as well: Remark. Let A be ay self-adjoit matrix. a) The vector space F splits F = λ σ(a) V λ where each eigespace V λ is ivariat uder A, ad so is its orthogoal complemet, which equals the direct sum of all the other eigespaces: V λ = λ σ(a),λ =λ V λ b) If v is a eigevector the A leaves ivariat Sp(v) ad Sp(v). Proof: Oly part b) eeds a proof.

4 SPECTRAL PROPERTIES OF SELF-AJOINT MATRICES 43 A(Sp(v)) Sp(v) because if x Sp(v) thex = xv therefore Ax = cav = cλv Sp(v). A(Sp(v) ) Sp(v) because if y Sp(v) the this meas that y, v = 0. The Ay, v = y,a v = y,av = y, λv = λy, v = Mii ad i. We saw that R(x) =λ = R(v ). The the matrix A, as a liear trasformatio of the 1 dimesioal vector space Sp(v ) to itself has its largest eigevalue λ 1 (we reduced the dimesio!). The R(x) =λ 1 is attaied for x = v 1 x Sp(v ) Note that the statemet x Sp(v ) ca be formulated as the costrait x, v = 0: x,v =0 R(x) =λ 1 We ca do eve better: we ca obtai λ 1 without kowig v or λ. To achieve this, subject x to ay costrait: x, z = 0 for some z = 0. It is easier to see what happes i coordiates y = U x i which A is diagoal. The costrait x, z = 0 is equivalet to y, w =0where w = Uz is some ozero vector. O oe had, obviously y: y,w=0 R U(y) λ (38) y: y,w=0 R U(y) λ 1 which implies that (39) w=0 y: y,w=0 R U(y) λ 1 We ow argue that equality is ataied i (39) for special w ad y. Ideed, cosider a ozero vector ỹ =(0,...,0,y 1,y ) T with ỹ, w = 0. (ỹ is easy to fid: if w = 0 take y 1 = 1 ad y = w 1 /w, ad if w =0 take y 1 =0, y = 1). Usig formula (37) R U (ỹ) = λ 1 y λ y 2 y y 2 λ 1 y λ 1 y 2 y y 2 = λ 1 Sice for w = e we have equality: the i (39) there is equality y: y,e =0 R U(y) =λ 1 w=0 y: y,w=0 R U(y) =λ 1

5 44 RODICA D. COSTIN I a similar way it is show that λ 2 is obtaied by a imum-imum process, but with two costraits: (40) w 1,w 2 =0 y, w 1 =0 y, w 2 =0 R U (y) =λ 2 Ideed, cosider a ozero vector ỹ =(0,...,0,y 2,y 1,y ) T satisfyig ỹ, w 1 = 0 ad ỹ, w 2 = 0. The i formula (37) R U (ỹ) = λ 2 y λ 1 y λ y 2 y y y 2 λ 2 y λ 2 y λ 2 y 2 y y y 2 = λ 2 which shows that (41) y, w 1 =0 y, w 2 =0 R U (y) λ 2 Sice for w 1 = e ad w 2 = e 1 we have equality i (41), this implies (40). Step by step, addig oe extra costrait, the i procedure yields the ext largest eigevalue. Goig back to the variable x it is foud that: Theorem 56. The i priciple Let A be a self-adjoit matrix, with its eigevalues umbered i a icreasig sequece: λ 1 λ 2... λ correspodig to the eigevectors v 1,...,v. The its Rayleigh s quotiet satisfies z=0 R(x) = x,ax x 2 x=0 R(x) =λ x,z=0 R(x) =λ 1 ad i geeral z 1,z 2 =0 x, z 1 =0 x, z 2 =0 R(x) =λ 2 z 1,...z k =0 x, z 1 =0. x, z k =0 R(x) =λ k, k =1, 2,..., 1

6 SPECTRAL PROPERTIES OF SELF-AJOINT MATRICES 45 Remark. Sometimes the i priciple is formulated as V j R(x) =λ j, j =1,..., x V j where V j deotes a arbitrary subspace of dimesio j. The two formulatios are equivalet sice the set V k = {x x, z 1 =0,...,x, z k =0} is a vector space of dimesio k if z 1,...,z k are liearly idepedet. A similar costructio startig with the lowest eigevalue produces: Theorem 57. The i priciple Uder the assumptios of Theorem 56 x=0 R(x) =λ 1 ad i geeral z=0 x,z=0 R(x) =λ 2 z 1,...z k =0 x, z 1 =0. x, z k =0 R(x) =λ k+1, k =1, 2,..., The i priciple for the geeralized eigevalue problem. Suppose λ 1 λ 1... λ are eigevalues for the problem (42) Av = λbv, A symmetric, B positive defiite It was see i 4.8 that if S =[v 1,...,v ] is the matrix whose colums are the geeralized eigevectors of the problem (42), the both matrices A ad B are diagoalized usig a cogruece trasformatio: S T AS = Λ ad S T BS = I. Defiig R(x) = x,ax x,bx it is foud that i coordiates x = Sy: R(x) =R(Sy) = ad therefore Sy, ASy Sy,BSy = y,st ASy y,s T BSy = λ 1y λ y 2 y y2 R(x) =λ, R(x) =λ 1