Linerity, liner opertors, nd self djoint eigenvlue problems 1 Elements of liner lgebr The study of liner prtil differentil equtions utilizes, unsurprisingly, mny concepts from liner lgebr nd liner ordinry differentil equtions. Here brief overview of the required concepts is provided. 1.1 Vector spces nd liner combintions A vector spce S is set of elements - numbers, vectors, functions - together with notions of ddition, sclr multipliction, nd their ssocited properties. A liner combintion of elements v 1, v 2,..., v k S is ny expression of the form c 1 v 1 + c 2 v 2 +... + c k v k where the coefficients c j re rel or complex sclrs. A significnt property of vector spces is tht ny liner combintion of elements in S is lso in S. This is esily verified in most cses - for exmple, R n the set of n-dimensionl vectors nd C R the set of continuous functions on the rel line re vector spces. 1.2 Liner trnsformtions nd opertors Suppose A is n n mtrix, nd v is n-dimensionl vector. The mtrix-vector product y Av cn be regrded s mpping tht tkes v s input nd produces the n-dimensionl vector y s n output. More precisely this mpping is liner trnsformtion or liner opertor, tht tkes vector v nd trnsforms it into y. Conversely, every liner mpping from R n R n is represented by mtrix vector product. The most bsic fct bout liner trnsformtions nd opertors is the property of linerity. In words, this sys tht trnsformtion of liner combintion is the liner combintion of the liner trnsformtions. This clerly pplies to the cse where the trnsformtion is given by mtrix A, since Ac 1 v 1 + c 2 v 2 c 1 Av 1 + c 2 Av 2. 1 More generlly, if f 1, f 2 re elements of vector spce S, then liner opertor L is mpping from S to some other vector spce frequently lso S so tht 1.3 Eigenvlues nd eigenvectors Recll tht v, λ is n eigenvector-eigenvlue pir of A if Lc 1 f 1 + c 2 f 2 c 1 Lf 1 + c 2 Lf 1. 2 Av λv. Except in rre cses for so clled non-norml mtrices, there re exctly n linerly independent eigenvectors v 1, v 2,..., v n corresponding to eigenvlues λ 1, λ 2,..., λ n the eigenvlues cn sometimes be the sme for different eigenvectors, however. Why is this useful fct? Note tht if x is ny n-dimensionl vector, it cn be written in terms of the bsis given by the eigenvectors, i.e. x c 1 v 1 + c 2 v 2 +... + c n v n. 3 1
The c i solve liner system Vc x where V hs columns of eigenvectors nd c is the vector c 1 c 2... T. When A is self-djoint see below, there is much esier wy to find the coefficients. Note tht ction of liner trnsformtion A on the vector x cn be written simply s Ax Ac 1 v 1 + c 2 v 2 +... + c n v n c 1 Av 1 + c 2 Av 2 +... + c n Av n c 1 λ 1 v 1 + c 2 λ 2 v 2 +... + c n λv n. In other words, eigenvectors decompose liner opertor into liner combintion, which is fct we often exploit. 1.4 Inner products nd the djoint opertor It is frequently helpful to ttch geometric ides to vector spces. One wy of doing this is to specify n inner product, which is mp S S R or S S C. The inner product is bsiclly wy of specifying how to mesure ngles nd lengths. For v 1, v 2 S, we will write n inner product s v 1, v 2. There re mny different wys to define inner products, but they must ll hve the following properties: 1. Symmetry: v, u u, v for every u, v S. If the inner product is complex vlued, this needs to be v, u u, v, where z denotes the complex conjugte. 2. Linerity in the first vrible: for ny vector v nd vectors v 1, v 2,..., v n we hve c 1 v 1 + c 2 v 2 +... + c n v n, v c 1 v 1, v + c 2 v 2, v +... + c n v n, v. 4 3. Positivity: v, v > unless v. Note for complex vector spces, the first two properties imply conjugte linerity in the second vrible: v, c 1 v 1 + c 2 v 2 +... + c n v n c 1 v, v 1 + c 2 v, v 2 +... + c n v, v n. 5 The inner product defines length in the sense tht v, v is thought of s the squre of the mgnitude or norm of v. The inner product lso mesures how prllel two elements of vector spce re. In prticulr, we define v 1, v 2 to be orthogonl if v 1, v 2. Once n inner product is defined, then for ny liner trnsformtion or opertor L, there is nother opertor clled the djoint of L, written L. Wht defines the djoint is tht, for ny two vectors v 1, v 2, Lv 1, v 2 v 1, L v 2. 6 This definition is bit confusing becuse L is not explicitly constructed. You should think of this s if I find n opertor L tht stisfies property 6, it must be the djoint. The dot product for finite dimensionl vectors is the best known exmple of n inner product there re in fct mny wys of defining inner products, even for vectors. As n exmple, consider the liner opertor on n-dimensionl rel vectors given by multiplying by mtrix, i.e. Lv Av. By the usul rules of mtrix multipliction Av 1 v 2 v 1 A T v 2, which mens tht the trnspose of A is sme s the djoint of A, with respect to the inner product defined by the dot product. Note tht if one used different inner product, the djoint might be different s well. 2
1.5 Self-djointness nd expnsion in eigenvectors Sometimes n opertor is its own djoint, in which cse its clled self-djoint. Self-djoint opertors hve some very nice properties which we will exploit. The most importnt re 1. The eigenvlues re rel. 2. The eigenvectors corresponding to different eigenvlues re orthogonl. Suppose mtrix A is symmetric nd therefore self-djoint, nd we know its eigenvectors. As in 3, we cn try to write ny vector x s liner combintion of the eigenvectors. Tking the inner product of 3 with ny prticulr eigenvector v k nd using 4, we hve x, v k c 1 v 1 + c 2 v 2 +... + c n v n, v n c 1 v 1, v k + c 2 v 2, v k +... + c 1 v n, v k c k v k, v k 7 since v k is orthogonl to ll eigenvectors except itself. Therefore we hve simple formul for ny coefficient c k : c k x, v k v k, v k. 8 In some cses the eigenvectors re rescled or normlized so tht v k, v k 1, which mens tht 8 simplifies to c k x, v k. 2 Differentil liner opertors We cn think of derivtives s liner opertors which ct on vector spce of functions. Although these spces re infinite dimensionl recll, for instnce, tht tht 1, x, x 2,... re linerly independent, notions such s linerity, eigenvlues, nd djoints still pply. As n exmple, consider the second derivtive of function d 2 f/dx 2. This cn be thought of s mpping fx to the output f x. The bsic linerity property 1 is esily verified since d 2 d 2 f 1 dx 2 c 1 f 1 x + c 2 f 1 x c 1 dx 2 + c d 2 f 2 2 dx 2. There is technicl point to be mde here. We sometimes hve to worry bout wht set of functions ctully constitutes our vector spce. For the exmple bove, nturl vector spce for the domin of the liner opertor is C 2 R, the set of ll twice differentible functions on the rel line. We could hve insted chosen the set of infinitely differentible functions C R. Chnging the spce of functions on which differentil opertor cts my ffect things like eigenvlues nd djointness properties. 2.1 Liner differentil equtions All liner equtions involve liner opertor L. There re two types of liner equtions, homogeneous nd inhomogeneous, which hve the forms Lf, homogeneous, Lf g, inhomogeneous. 3
Here f is the solution the function to be found, L is some differentil liner opertor, nd g is nother given function. As rule of thumb, identifying liner eqution is just mtter of mking sure tht ech term in the eqution is liner opertor cting on the unknown function, or term which does not involve the unknown. 2.2 The superposition principle The big dvntge of linerity is the superposition principle, which permits divide-nd-conquer strtegy for solving equtions. If we cn find enough linerly independent solutions, we cn get ll solutions simply by forming liner combintions out of these building blocks. More precisely, Superposition principle for homogeneous equtions: If f 1, f 2,... re solutions of Lf, then so is ny liner combintion: Lc 1 f 1 + c 2 f 2 +... c 1 Lf 1 + c 2 Lf 2 +.... This bsic ide cn be mended for inhomogeneous equtions. In this cse, one needs to find ny single prticulr solution f p which solves Lf p g, so tht the difference h f f p solves homogeneous eqution Lh. The result is Superposition principle for inhomogeneous equtions: If h 1, h 2,... re solutions of Lh, then ny liner combintion of these plus the prticulr solution solves the inhomogeneous eqution Lf g: Lf p + c 1 h 1 + c 2 h 2 +... Lf p + c 1 Lh 1 + c 2 Lh 2 +... g. 2.3 Superposition principle nd boundry conditions One difficulty tht rises from using the superposition principle is tht, while the eqution itself my be stisfied, not every liner combintion will stisfy the boundry or initil conditions unless they re of specil form. We cll ny side condition liner homogeneous if it cn be written in the form Bf, where B is liner opertor. The most bsic types of boundry conditions re of this form: for the Dirichlet condition, B is the restriction opertor which simply mps function to its boundry vlues. For the Neumnn condition, the opertor first tkes norml derivtive before restriction to the boundry. We my extend the superposition principle to hndle boundry conditions s follows. If f 1, f 2,... stisfy liner, homogeneous boundry conditions, then so does n superposition of these c 1 f 1 +c 2 f 2 +.... Therefore in most PDEs, the superposition principle is only useful if t lest some of the side conditions re liner nd homogeneous. 2.4 Sturm-Liouville opertors As n illustrtion, we will consider clss of differentil opertors clled Sturm-Liouville opertors tht re firly esy to work with, nd frequently rise in the study of ordinry nd prtil differentil equtions. For specified coefficient functions qx nd px >, these opertors hve the form L d px d + qx. 9 dx dx the nottion mens tht to pply L to function fx, we put f on the right hnd side nd distribute terms so Lf pxf + qxfx The functions p nd q re prescribed, nd in the 4
most bsic cses re simply constnt. The vector spce tht L cts on will be C [, b], which mens tht f is infinitely differentible nd f fb i.e. homogeneous Dirichlet boundry conditions. 2.5 Inner products nd self djointness As pointed out erlier, there re mny wys of defining n inner product. Given two functions f, g in C [, b], we define the so clled L2 inner product s f, g fxgxdx. 1 We cn now show tht the Sturm-Liouville opertor 9 cting on C [, b] is self-djoint with respect to this inner product. How does one compute the djoint of differentil opertor? The nswer lies in using integrtion by prts or in higher dimensions, Green s formul. For ny two functions f, g in C [, b] we hve Lf, g f, Lg. d px df gx + qxfxgxdx dx dx px df dg df + qxfxgxdx + pxgx b dx dx dx d px dg fx + qxfxgxdx pxfx dg dx dx dx Integrtion by prts ws used twice to move derivtives off of f nd onto g. Therefore compring to 6 L is its own djoint. Note tht the boundry conditions were essentil to mke the boundry terms in the integrtion by prts vnish. As nother exmple, consider the weighted inner product on C [, R] defined by f, g rfrgrdr. 11 Then for the liner opertor Lf r 1 rf, integrtion by prts twice gives Lf, g f, Lg. [ r r 1 d r df dr dr r df dg dr dr dr, d r dg dr dr [ r r 1 d dr fr dr, r dg dr ] grdr int. by prts ] fr dr int. by prts Note in the finl step, weight fctor of r ws needed inside the integrl to obtin the inner product 11. Thus L is self djoint with respect to the weighted inner product. It turns out not to be self djoint, however, with respect to the unweighted one 1. b 5
Wht bout non-self-djoint opertors? The simplest exmple is L d/dx cting on C [, b]. We gin compute Lf, g df b dx gx dg fxdx f, Lg. dx It follows tht L L. Opertors which re their negtive djoints re clled skew symmetric. 2.6 Eigenvlue problems for differentil opertors Anlogous to the mtrix cse, we cn consider n eigenvlue problem for liner opertor L, which sks to find non-trivil function-number pirs vx, λ which solve Lvx λvx. 12 It should be noted tht sometimes the eigenvlue problem is written insted like Lvx+λvx which reverses the sign of λ, but the theory ll goes through just the sme. The functions vx which stisfy this re clled eigenfunctions nd ech corresponds to n eigenvlue λ. As with eigenvectors, we cn rescle eigenfunctions: if vx, λ is n eigenvector-vlue pir, so is cvx, λ for ny sclr c. Wht cn we expect to hppen in these problems? Since we re working in infinite dimensions, there re often n infinite number of distinct eigenvlues nd eigenfunctions. The set of eigenvlues is lso clled the discrete spectrum there is nother prt clled the essentil or continuous spectrum which we will not get into. Often the eigenfunctions form bsis for the underlying vector spce, nd the set is is clled complete. This property is extremely vluble in PDEs, since it llows for solutions to be expressed s liner combintions of eigenfunctions. The most fmous exmple of this is the Fourier series, which we discuss lter. Self-djoint opertors hve some properties equivlent to self-djoint mtrices. In prticulr, their eigenvlues re rel, nd their eigenfunctions re orthogonl. Orthogonlity is tken with respect to the sme inner product which gives self-djointness. 2.7 Sturm-Liouville eigenvlue problems Sturm-Liouville opertors 9 on C [, b] hve some other nice properties side from those of ny self-djoint opertor. For the eigenvlue problem the following properties hold: d dx px dv dx + qxvx + λvx, 1. The rel eigenvlues cn be ordered λ 1 < λ 2 < λ 3... so tht there is smllest but not lrgest eigenvlue. 2. The eigenfunctions v n x corresponding to ech eigenvlue λ n form complete set, i.e. for ny f C [, b], we cn write f s infinite liner combintion f c n v n x. 13 n1 6
The infinite series in 13 should cuse some concern. It should be noted tht for functions, there re different wys to define convergence. One wy is just to fix ech point x nd consider the limit of function vlues, which is clled pointwise convergence. We ctully hve something stronger here: the rte t which the sequences t ll x converge is the sme. This is clled uniform convergence. Here is the simplest exmple of Sturm-Liouville eigenvlue problem. Consider the opertor L d 2 /dx 2 on the vector spce C [, π]. The eigenvlue problem reds d 2 v + λv, v, vπ. 14 dx2 This is just second order differentil eqution, nd writing down the generl solution is esy. Recll tht we guess solutions of the form vx exprx. Provided λ >, we get r ±i λ, which mens tht the generl solution hs the form vx A cos λx + B sin λx. Not ll of these solutions re vlid; we require tht v vπ. Therefore A cos+b sin, so tht A. The other boundry condition implies B sinπ λ which is only true if π λ is multiple of π. Therefore λ n n 2, n 1, 2, 3,.... Corresponding to ech of these eigenvlues is the eigenfunction v n x sinnx, n 1, 2, 3,.... Recll tht we don t cre bout the prefctor B since eigenfunctions cn lwys be rescled. Properly speking, we lso need to consider the cses λ, λ <. For the first cse, vx Ax + B, nd it s obvious one needs A B to stisfy the boundry conditions. If λ <, vx A exp λ x + B exp λ x. The boundry conditions imply A + B nd A exp λ π + B exp λ π, which written in mtrix form is 1 1 exp λ π exp λ π A B Such system only hs nonzero solutions if the mtrix is singulr, which it is not since its determinnt is exp λ π exp λ π. Be creful: there re Sturm-Liouville eigenvlue problems which do hve non-positive eigenvlues. On the other hnd, there is no need to worry bout complex eigenvlues becuse the liner opertor is self-djoint. 2.8 Fourier series There is big pyoff from fct tht the eigenfunctions of the previous exmple re complete: we cn write ny function in C [, π] s liner combintion of the eigenfunctions of d2 /dx 2. In other words we cn write ny smooth function with zero boundry vlues s fx. B n sinnx. 15 This is the fmous Fourier sine series - just one of severl types of Fourier series. n1 7
Series type Spce of functions Orthogonl expnsion for fx Coefficients Fourier Sine Cosine Complex fx : [ L, L] R f L fl f L f L fx : [, L] R f fl fx : [, L] R f f L fx : [ L, L] C f L fl f L f L A 2 + n1 A n cos nπx + n1 B n sin nπx n1 B n sin nπx L L L A 2 + n1 A n cos nπx L A n 1 L B n 1 L B n 2 L A n 2 L L L L L n c n exp inπx L c n 1 L 2L L Tble 1: Vrious Fourier series nπx fx cos L dx nπx fx sin L dx L nπx fx sin L dx L nπx fx cos L dx inπx fx exp L dx The question tht rises is, how do we ctully compute the coefficients B n? We hve lredy nswered this question in more generl setting. Becuse the eigenfunctions sinnx re orthogonl, we cn use 7 nd 8. For the present cse, this is equivlent to tking n inner product of 15 with ech eigenfunction sinnx. This gives the equivlent of 7, nmely B n fx, sinnx sinnx, sinnx π fx sinnxdx π sin2 nxdx. It should be emphsized tht Fourier coefficient formuls like this one don t need to be memorized. They rise quite simply from the bsic ide of finding coefficients of liner combintion of orthogonl vectors. There re other types of Fourier series involving cosines or complex exponentils. These functions re ll eigenfunctions of the second derivtive opertor L d 2 /dx 2, but with different vector spces of functions on which it cts remember the technicl point erlier: properties of differentil opertors crucilly depend on the vector spce in question. Tble 2.8 summrizes the stndrd types of Fourier series. The coefficient formuls ll derive from tking inner products with ech eigenfunction note tht for the complex series, the inner product is u, v L L uvdx. We shll see lter tht other eigenvlue problems give rise to new orthogonl sets of functions: the Bessel functions, nd the sphericl hrmonics. 2.9 Integrl opertors Another importnt type of liner opertor involves integrls, such s the Hilbert-Schmidt clss of integrl opertors Lux kx, yuxdx. 16 Notice tht the output of this opertion is function of y, not x. Here is the domin of u nd kx, y is clled the kernel, which is function from to R. Linerity rises simply becuse of linerity of the integrl: Lc 1 u 1 + c 2 u 2 c 1 kx, yu 1 xdx + c 2 kx, yu 2 xdx c 1 Lu 1 + c 2 Lu 2. 8
We will not encounter mny equtions which hve integrl opertors, but some of our solutions will involve integrl opertors. This is becuse integrl opertors re often inverses of differentil opertors. This mens tht the inhomogeneous eqution Lu g hs solution u L 1 g where L 1 is n integrl opertor. Integrl opertors hve djoints. For exmple, using the L 2 inner product, the opertor given in 16 stisfies Lu, v vy ux where L is nother integrl opertor L vx kx, yux dxdy kx, yvy dydx u, L v ky, xvxdx. This is not the sme s L; the inputs to the kernel ky, x hve been reversed. 9