Eigevalue-Eigevector Istructor: Nam Su Wag eigemcd Ay vector i real Euclidea space of dimesio ca be uiquely epressed as a liear combiatio of liearly idepedet vectors (ie, basis) g j, j,,, α g α g α g α j g j j hus, give a liear trasformatio A, we ca fid the actio of A o ay of the vectors that reside i the same real Euclidea space if we kow the actio of A o each of the liearly idepedet basis vectors g i A α A g α A g α A g A ivariat subspace for liear trasformatio A is a subset of the LS that is shared by both the origial vector v ad the vector resultig from applyig the liear trasformatio A to v I other words, liear trasformatio A does ot take v out of the origial subspace where v origiates If that subspace has a dimesio of (ie, there is oly oe liearly idepedet vector i that subspace), it is called a -dimesioal ivariat subspace Eigevector is aother commo ame for that loe liearly idepedet vector i a -dimesioal ivariat subspace (here are may vectors i a -dimesioal ivariat subspace, but there is oly oe liearly idepedet oe -- that comes directly from the defiitio of dimesio Basically, every vector i a -dimesioal subspace differs from all the others oly i legth) Eigevalue-Eigevector equatio A v v v α scalar A vector v that satisfies the above equatio is a eigevector A eigevector for liear trasformatio A is a special vector such that, whe A is applied to it, the resultig vector is simply the origial vector multiplied by a scalar he scalar is called the eigevalue I real Euclidea space, we ca defie two metrics of a vector based o the scalar product hese two metrics are: magitude (legth) ad agle (directio) hus, a liear trasformatio A applied to a eigevector merely chages its magitude by the associated eigevalue, but ot its agle I other words, we have a case of pure epasio or cotractio mappig his characteristics makes eigevectors a ideal cadidate as the basis vectors to describe all other vectors i the LS he geeral idea is to represet ay give vector i the LS as a liear combiatio of eigevectors v (if we ca fid liearly idepedet oes) α g α g α g α j g j Give the basis g j, a set of umbers (α, j α,, α ) defies the vector Apply chage-of-basis formula to rewrite as a liear combiatio of eigevectors v j β v β v β v j β j v j Give the basis v j, a set of umbers (β, β,, β ) defies the vector If the basis v j are orthoormal, projectio gives the values of β j β j, v j
eigemcd Without much work, we fid the effect of applyig the liear trasformatio A o ay vector A β A v β A v β A v j A β v β v β v j A β v β v β v j β j A v j β j j v j β j j We wish to seek out these eat vectors A equivalet eigevalue-eigevector equatio is, Eigevalue-Eigevector equatio: ( A I) v or ( I A) v v v j Sice v, we look for solutio to A I characteristic equatio he algebraic equatio det A- I is called the characteristic equatio for the liear trasformatio A Note that we ca determie a eigevector oly up to a scalar multiplier If v is a eigevector, ay vector yα v, where α is a scalar, is also e eigevector A y A ( α ) α ( A ) ( α ) y I geeral, we would like to ormalize a eigevector so that v Eigevectors associated with distict eigevalues are liearly idepedet (Why must this be true?) Uses of eigevalue-eigevector: aywhere liear trasformatio applies he procedure geerally follows the above descriptio Give vectors represeted with the origial set of basis g, we fid a equivalet represetatio with the eigevectors v as the basis Apply trasformatio to the eigevectors Epress the effect of liear trasformatio i terms of the origial set of basis Eample of usage: Similarity rasform For symmetric A A Λ A Λ A Λ A Λ A Λ
Eample of usage: Liear Regressio eigemcd We wish to epress depedet variable as a liear combiatio of the idepedet variables X y X a compare to liear trasformatio A or A y A or y A (just a differet fot) Do ot get cofuse the above regressio epressio "yx a" as a liear trasformatio he regressio equatio is merely a matri-vector represetatio; X is ot a liear trasformatio Liear trasformatio acts o a vector i LS ad yield aother vector i the same LS he colum of umbers "a" ad the depedet vector "y" i the ormal equatios are two very differet objects! he ormal equatio i liear regressio is X X a X y Let ad v to be the eigevalues ad eigevectors for X X X X v v X X Λ Because X X is symmetrical ad positive defiite, ( X ) ( X ) X X X X Λ hus, the idepedet variable X epressed i terms of eigevectors is X, which is called the scores he ormal equatio for the ew idepedet variable X is ( X ) ( X ) b ( X ) y score score b score y So, basically we epress the give data X i terms of eigevectors ; X score, where the score vectors are mutually orthogoal (because the eigevectors are mutually orthogoal), ad the matri of the score's scalar products is diagoal, with eigevalues of X X (which is the variace i the eigevector directio) beig the diagoal elemets Fid the regressio coefficiets b (which is reduced to scalar iverse) Λ b ( X ) y After we fid the regressio coefficiet b, the regressio equatio is y ( X ) b Eample of usage: A set of first-order ODEs A d dt ( t ) A( t) ( t) A is a liear trasformatio that acts o, a vector of fuctios, to yield aother vector of fuctios A(t) is a matri that spells out i mathematical terms how the liear trasformatio A chages each of the Let ad v to be the eigevalues ad eigevectors for A A v v Similarity trasform A J where J is either a diagoal or early diagoal Jorda matri Epress z i the eigevector directios ( t ) z( t) z( t ) ( t) d he differetial equatio for z is dt z( t ) d dt ( t ) A( t) ( t ) A( t) z( t ) J( t) z( t) d If A(t) is a costat, dt z( t ) J z( t ) z( t ) ep( J t) z( ) Fid ep(j*t), which is either diagoal or i a early diagoal Jorda form Epress the solutio i the origial vector ( t ) z( t ) ep( J t) z( ) ep( J t) ( ) ( t ) ep( A t) ( )
4 eigemcd Eample: Rotatio of arrows i -dimesio aroud a ais of rotatio he arrows i a plae to the ais of rotatio resides i a -dimesioal ivariat subspace he arrows // to the ais of rotatio resides i a -dimesioal ivariat subspace Eample: -dimesioal colums of real umbers -- eigevectors Defie a liear trasformatio A A A v v A v v v v v v v v v v ( ) v v det ( ) v v Eigevector calculatio: v v ( ) v 4 v v he two equatios are liearly depedet v v v v hus, the eigevector associated with is v 4 v v he two equatios are liearly depedet v v v v hus, the eigevector associated with 4 is v Eample: -dimesioal colums of real umbers eigevector Defie a liear trasformatio A A A v v A v v v v v v v v v ( ) v v det ( ) v v ( ) v Eigevector calculatio: v he secod equatio gives o ew iformatio hus, the eigevector associated with is v
5 eigemcd Eample: -dimesioal colums of real umbers -- eigevector Defie a liear trasformatio A A A v v A v v v v v v v v v det v v v v???? here is o -dimesioal ivariat subspace i this eample, although there is a -dimesioal ivariat subspace If we stick strictly with real scalars, there is o eigevalue If we allow comple umbers, the i, -i, where i is a imagiary umber Sice there are o imagiary umbers i a real physical world, we will have to fid a way back to the real world later if we dive ito a imagiary world Eample: -dimesioal colums of real umbers Defie a liear trasformatio A A det( A I) det Eigevector calculatio: I compact matri otatio A ( )( ( ) ) ( ) ( )( )( ) v A I v v v v v v v Oly two of the above three equatios are liearly depedet v Normalize v <> v v v v
eigemcd v v A I v v v v v v v v Oly two of the above three equatios are liearly depedet v v v v v Normalize v <> v v v A I v v v v v v v v Oly two of the above three equatios are liearly depedet v v v v v Normalize v <> v Combie all the eigevalues together i a diagoal matri Combie all the eigevectors together i a matri Λ v v < > v
7 eigemcd Check For orthoormal eigevectors I all permutatios of scalar products idetity I orthoormal A Λ diagoal elemets of eigevalues A Λ what we started Eample: -dimesioal colums of real umbers Repeated eigevalues A ( ) det( ) A I det
8 eigemcd Eigevector calculatio: v v A I v v, v are ay umber v eigevector v ormalize v Fid v <> such that it is orthogoal to the first eigevector v v v v v v eigevector v ormalize v Fid v <> such that it is orthogoal to the first two eigevectors v v v v v v v v v v v v v v v v his is ot a eigevector It is, however, orthogoal to the first two Aother way to fid a third vector orthogoal to the first two is to rely o the cross product betwee two vectors Combie all the vectors together i a matri his forms a good set of orthogoal basis v v < > v
Check For orthoormal vectors 9 eigemcd I scalar products Sice the last vector is ot a eigevector, the similarity trasform does ot leads to a diagoal matri A Λ 77 77 Eample: Cosider the followig liear trasform that acts o real cotiuous fuctios f(t), for t[, ] (Sice the dimesio for this LS is ifiite, we caot pick a fiite umber of basis vectors f i ad describe the actio of the liear trasform A o each of these basis vectors with a matri A) g( t ) A Defie trasform A g( t ) A f( τ )( t caot write as: A f( t ) A Check: erify that A is a liear trasform by checkig the followig two properties? (If A were ot a liear trasform, we caot eve talk about eigevalues, ad we could waste time chasig after somethig that does ot eist) Distributive A ( f g ) A f A g A ( f g) ( f( g( τ ) )( t f( τ )( t g( τ )( t A f A g Associative A ( α f ) α ( A f) A ( α f) ( α f( τ ) )( t τ ) α A f f f( τ )( t τ ) f( t ) f( τ )( t τ ) α ( A f) Fredholm itegral equatio
eigemcd Case, ie, we are lookig for the ull space of the liear trasformatio A f( τ ) t f( τ For the above relatioship to hold for all t i [, ], we must have f( ad f( τ hus, ay fuctio whose th momet (average) ad whose st momet both vaish i t[, ] is a eigefuctio here are may fuctios that satisfies these coditios Case f( t ) f( τ ) t f( τ α β t f( τ ) f( τ t Note that the itegrals evaluate to scalar umbers, ad the eigefuctio is a st degree polyomial i t α β α f( τ α β f( τ α β τ τ α β α β β α β Set det (Note that we cotiue to have a characteristic equatio det that govers the eigevalues eve whe we caot describe the actio of A as "ga f" -- remember, the umber of basis vectors is ifiite) Eigevector calculatio for characteristic equatio for the liear trasformatio A ( ) α β α ( ) β Oly oe of the above two equatios are liearly idepedet f( t ) α ( t) is a eigefuctio Sice ay scalar multiple of this fuctio is also a eigefuctio, we ca take out the costat factor, or we ca ormalize the f ( t) t eigefuctio α β
Eigevector calculatio for eigemcd ( ) α β α ( ) β Oly oe of the above two equatios are liearly idepedet α β f( t ) α ( t) is a eigefuctio We ca take out the costat factor, or we ca ormalize the eigefuctio f ( t) t Orthogoality betwee various eigefuctios f (t) for, f (t) for /, ad f (t)-/ Defie a scalar product ( f, g) f, f f, f f, f f ( f ( f ( f ( f ( f ( f( g( f ( τ )( f ( f ( τ f ( τ )( f ( τ ) f ( τ ( τ )( I summary, we have f, f f, f f, f hus, all three eigefuctios are mutually orthogoal Eample: Successive approimatio o a vector φ( ) If φ is a liear trasformatio, the fidig solutio is idetical to fidig a eigevector that correspods to φ( ) his is a bit silly, but it is sometimes easier to see thigs, after we arrage them i the "stadard" form A