1 Introducton and Fundamental Concepts The numercal expresson of a scentfc statement has tradtonally been the manner by whch scentsts have verfed a theoretcal descrpton of the physcal world. Durng ths century there has been a revoluton n both the nature and extent to whch ths numercal comparson can be made. Indeed, t seems lkely that when the hstory of ths century s defntvely wrtten, t wll be the development of the computer, whch wll be regarded as ts greatest technologcal achevement - not nuclear power. Whle t s true that the orgns of the dgtal computer can be traced through the work of Isaac Babbtt, Hermann Hollerth, and others n the nneteenth century, the real advance came after the Second World War when machnes were developed that were able to carry out an extended sequence of nstructons at a rate that was very much greater than a human could manage. We call such machnes programmable. The electronc dgtal computer of the sort developed by John von Neumann and others n the 1950s really ushered n the present computer revoluton. Whle t s stll to soon to delneate the form and consequences of ths revoluton, t s already clear that t has forever changed the way n whch scence and engneerng wll be done. The entre approach to numercal analyss has changed n the past two decades and that change wll most certanly contnue rapdly nto the future. Pror to the advent of the electronc dgtal computer, the emphass n computng was on short cuts and methods of verfcaton whch nsured that computatonal errors could be caught before they propagated through the soluton. Lttle attenton was pad to "round off error" snce the "human computer" could easly control such problems when they were encountered. Now the relablty of electronc machnes has nearly elmnated concerns of random error, but round off error can be a persstent problem.
Numercal Methods and Data Analyss The extreme speed of contemporary machnes has tremendously expanded the scope of numercal problems that may be consdered as well as the manner n whch such computatonal problems may even be approached. However, ths expanson of the degree and type of problem that may be numercally solved has removed the scentst from the detals of the computaton. For ths, most would shout "Hooray"! But ths removal of the nvestgator from the detals of computaton may permt the propagaton of errors of varous types to ntrude and reman undetected. Modern computers wll almost always produce numbers, but whether they represent the soluton to the problem or the result of error propagaton may not be obvous. Ths stuaton s made worse by the presence of programs desgned for the soluton of broad classes of problems. Almost every class of problems has ts pathologcal example for whch the standard technques wll fal. Generally lttle attenton s pad to the recognton of these pathologcal cases whch have an uncomfortable habt of turnng up when they are least expected. Thus the contemporary scentst or engneer should be skeptcal of the answers presented by the modern computer unless he or she s completely famlar wth the numercal methods employed n obtanng that soluton. In addton, the soluton should always be subected to varous tests for "reasonableness". There s often a tendency to regard the computer and the programs whch they run as "black boxes" from whch come nfallble answers. Such an atttude can lead to catastrophc results and beles the atttude of "healthy skeptcsm" that should pervade all scence. It s necessary to understand, at least at some level, what the "Black Boxes" do. That understandng s one of the prmary ams of ths book. It s not my ntenton to teach the technques of programmng a computer. There are many excellent texts on the multtudnous languages that exst for communcatng wth a computer. I wll assume that the reader has suffcent capablty n ths area to at least conceptualze the manner by whch certan processes could be communcated to the computer or at least recognze a computer program that does so. However, the programmng of a computer does represent a concept that s not found n most scentfc or mathematcal presentatons. We wll call that concept an algorthm. An algorthm s smply a sequence of mathematcal operatons whch, when preformed n sequence, lead to the numercal answer to some specfed problem. Much tme and effort s devoted to ascertanng the condtons under whch a partcular algorthm wll work. In general, we wll omt the proof and gve only the results when they are known. The use of algorthms and the ablty of computers to carry out vastly more operatons n a short nterval of tme than the human programmer could do n several lfetmes leads to some unsettlng dfferences between numercal analyss and other branches of mathematcs and scence. Much as the scentst may be unwllng to admt t, some aspects of art creep nto numercal analyss. Knowng when a partcular algorthm wll produce correct answers to a gven problem often nvolves a nontrval amount of experence as well as a broad based knowledge of machnes and computatonal procedures. The student wll acheve some feelng for ths aspect of numercal analyss by consderng problems for whch a gven algorthm should work, but doesn't. In addton, we shall gve some "rules of thumb" whch ndcate when a partcular numercal method s falng. Such "rules of thumb" are not guarantees of ether success or falure of a specfc procedure, but represent nstances when a greater heght of skeptcsm on the part of the nvestgator may be warranted. As already ndcated, a broad base of experence s useful when tryng to ascertan the valdty of the results of any computer program. In addton, when tryng to understand the utlty of any algorthm for 2
1 - Fundamental Concepts calculaton, t s useful to have as broad a range of mathematcal knowledge as possble. Mathematcs s ndeed the language of scence and the more profcent one s n the language the better. So a student should realze as soon as possble that there s essentally one subect called mathematcs, whch for reasons of convenence we break down nto specfc areas such as arthmetc, algebra, calculus, tensors, group theory, etc. The more areas that the scentst s famlar wth, the more he/she may see the relatons between them. The more the relatons are apparent, the more useful mathematcs wll be. Indeed, t s all too common for the modern scentst to flee to a computer for an answer. I cannot emphasze too strongly the need to analyze a problem thoroughly before any numercal soluton s attempted. Very often a better numercal approach wll suggest tself durng the analyses and occasonally one may fnd that the answer has a closed form analytc soluton and a numercal soluton s unnecessary. However, t s too easy to say "I don't have the background for ths subect" and thereby never attempt to learn t. The complete study of mathematcs s too vast for anyone to acqure n hs or her lfetme. Scentsts smply develop a base and then contnue to add to t for the rest of ther professonal lves. To be a successful scentst one cannot know too much mathematcs. In that sprt, we shall "revew" some mathematcal concepts that are useful to understandng numercal methods and analyss. The word revew should be taken to mean a superfcal summary of the area manly done to ndcate the relaton to other areas. Vrtually every area mentoned has tself been a subect for many books and has occuped the study of some nvestgators for a lfetme. Ths short treatment should not be construed n any sense as beng complete. Some of ths materal wll ndeed be vewed as elementary and f thoroughly understood may be skmmed. However many wll fnd some of these concepts as beng far from elementary. Nevertheless they wll sooner or later be useful n understandng numercal methods and provdng a bass for the knowledge that mathematcs s "all of a pece". 1.1 Basc Propertes of Sets and Groups Most students are ntroduced to the noton of a set very early n ther educatonal experence. However, the concept s often presented n a vacuum wthout showng ts relaton to any other area of mathematcs and thus t s promptly forgotten. Bascally a set s a collecton of elements. The noton of an element s left delberately vague so that t may represent anythng from cows to the real numbers. The number of elements n the set s also left unspecfed and may or may not be fnte. Just over a century ago Georg Cantor bascally founded set theory and n dong so clarfed our noton of nfnty by showng that there are dfferent types of nfnte sets. He dd ths by generalzng what we mean when we say that two sets have the same number of elements. Certanly f we can dentfy each element n one set wth a unque element n the second set and there are none left over when the dentfcaton s completed, then we would be enttled n sayng that the two sets had the same number of elements. Cantor dd ths formally wth the nfnte set composed of the postve ntegers and the nfnte set of the real numbers. He showed that t s not possble to dentfy each real number wth a nteger so that there are more real numbers than ntegers and thus dfferent degrees of nfnty whch he called cardnalty. He used the frst letter of the Hebrew alphabet to denote the cardnalty of an nfnte set so that the ntegers had cardnalty ℵ0 and the set of real numbers had cardnalty of ℵ1. Some of the brghtest mnds of the twenteth century have been concerned wth the propertes of nfnte sets. 3
Numercal Methods and Data Analyss Our man nterest wll center on those sets whch have constrants placed on ther elements for t wll be possble to make some very general statements about these restrcted sets. For example, consder a set wheren the elements are related by some "law". Let us denote the "law" by the symbol. If two elements are combned under the "law" so as to yeld another element n the set, the set s sad to be closed wth respect to that law. Thus f a, b, and c are elements of the set and a b = c, (1.1.1) then the set s sad to be closed wth respect to. We generally consder to be some operaton lke + or, but we shouldn't feel that the concept s lmted to such arthmetc operatons alone. Indeed, one mght consder operatons such as b 'follows' a to be an example of a law operatng on a and b. If we place some addtonal condtons of the elements of the set, we can create a somewhat more restrcted collecton of elements called a group. Let us suppose that one of the elements of the set s what we call a unt element. Such an element s one whch, when combned wth any other element of the set under the law, produces that same element. Thus a = a. (1.1.2) Ths suggests another useful constrant, namely that there are elements n the set that can be desgnated "nverses". An nverse of an element s one that when combned wth ts element under the law produces the unt element or a -1 a =. (1.1.3) Now wth one further restrcton on the law tself, we wll have all the condtons requred to produce a group. The restrcton s known as assocatvty. A law s sad to be assocatve f the order n whch t s appled to three elements does not determne the outcome of the applcaton. Thus (a b) c = a (b c). (1.1.4) If a set possess a unt element and nverse elements and s closed under an assocatve law, that set s called a group under the law. Therefore the normal ntegers form a group under addton. The unt s zero and the nverse operaton s clearly subtracton and certanly the addton of any two ntegers produces another nteger. The law of addton s also assocatve. However, t s worth notng that the ntegers do not form a group under multplcaton as the nverse operaton (recprocal) does not produce a member of the group (an nteger). One mght thnk that these very smple constrants would not be suffcent to tell us much that s new about the set, but the noton of a group s so powerful that an entre area of mathematcs known as group theory has developed. It s sad that Eugene Wgner once descrbed all of the essental aspects of the thermodynamcs of heat transfer on one sheet of paper usng the results of group theory. Whle the restrctons that enable the elements of a set to form a group are useful, they are not the only restrctons that frequently apply. The noton of commutvty s certanly present for the laws of addton and scalar multplcaton and, f present, may enable us to say even more about the propertes of our set. A law s sad to be communtatve f a b = b a. (1.1.5) A further restrcton that may be appled nvolves two laws say and. These laws are sad to be dstrbutve wth respect to one another f a (b c) = (a b) (a c). (1.1.6) 4
1 - Fundamental Concepts Although the laws of addton and scalar multplcaton satsfy all three restrctons, we wll encounter common laws n the next secton that do not. Subsets that form a group under addton and scalar multplcaton are called felds. The noton of a feld s very useful n scence as most theoretcal descrptons of the physcal world are made n terms of felds. One talks of gravtatonal, electrc, and magnetc felds n physcs. Here one s descrbng scalars and vectors whose elements are real numbers and for whch there are laws of addton and multplcaton whch cause these quanttes to form not ust groups, but felds. Thus all the abstract mathematcal knowledge of groups and felds s avalable to the scentst to ad n understandng physcal felds. 1.2 Scalars, Vectors, and Matrces In the last secton we mentoned specfc sets of elements called scalars and vectors wthout beng too specfc about what they are. In ths secton we wll defne the elements of these sets and the varous laws that operate on them. In the scences t s common to descrbe phenomena n terms of specfc quanttes whch may take on numercal values from tme to tme. For example, we may descrbe the atmosphere of the planet at any pont n terms of the temperature, pressure, humdty, ozone content or perhaps a polluton ndex. Each of these tems has a sngle value at any nstant and locaton and we would call them scalars. The common laws of arthmetc that operate on scalars are addton and multplcaton. As long as one s a lttle careful not to allow dvson by zero (often known as the cancellaton law) such scalars form not only groups, but also felds. Although one can generally descrbe the condton of the atmosphere locally n terms of scalar felds, the locaton tself requres more than a sngle scalar for ts specfcaton. Now we need two (three f we nclude alttude) numbers, say the lattude and longtude, whch locate that part of the atmosphere for further descrpton by scalar felds. A quantty that requres more than one number for ts specfcaton may be called a vector. Indeed, some have defned a vector as an "ordered n-tuple of numbers". Whle many may not fnd ths too helpful, t s essentally a correct statement, whch emphaszes the mult-component sde of the noton of a vector. The number of components that are requred for the vector's specfcaton s usually called the dmensonalty of the vector. We most commonly thnk of vectors n terms of spatal vectors, that s, vectors that locate thngs n some coordnate system. However, as suggested n the prevous secton, vectors may represent such thngs as an electrc or magnetc feld where the quantty not only has a magntude or scalar length assocated wth t at every pont n space, but also has a drecton. As long as such quanttes obey laws of addton and some sort of multplcaton, they may ndeed be sad to form vector felds. Indeed, there are varous types of products that are assocated wth vectors. The most common of these and the one used to establsh the feld nature of most physcal vector felds s called the "scalar product" or nner product, or sometmes smply the dot product from the manner n whch t s usually wrtten. Here the result s a scalar and we can operatonally defne what we mean by such a product by A r B r = c = A B. (1.2.1) One mght say that as the result of the operaton s a scalar not a vector, but that would be to put to restrctve an nterpretaton on what we mean by a vector. Specfcally, any scalar can be vewed as vector havng only one component (.e. a 1-dmensonal vector). Thus scalars become a subgroup of vectors and snce the vector scalar product degenerates to the ordnary scalar product for 1-dmensonal vectors, they are actually a subfeld of the more general noton of a vector feld. 5
Numercal Methods and Data Analyss It s possble to place addtonal constrants (laws) on a feld wthout destroyng the feld nature of the elements. We most certanly do ths wth vectors. Thus we can defne an addtonal type of product known as the "vector product" or smply cross product agan from the way t s commonly wrtten. Thus n Cartesan coordnates the cross product can be wrtten as î ĵ kˆ A r B r = A A A = î(a B A B ) ĵ(a B A B ) + kˆ(a B A B ). (1.2.2) B B B k k k k The result of ths operaton s a vector, but we shall see later that t wll be useful to sharpen our defnton of vectors so that ths result s a specal knd of vector. Fnally, there s the "tensor product" r or r vector outer product that s defned as AB = C. (1.2.3) C = A B Here the result of applyng the "law" s an ordered array of (n m) numbers where n and m are the dmensons of the vectors A r and B r respectvely. Agan, here the result of applyng the law s not a vector n any sense of the normal defnton, but s a member of a larger class of obects we wll call tensors. But before dscussng tensors n general, let us consder a specal class of them known as matrces. The result of equaton (1.2.3) whle needng more than one component for ts specfcaton s clearly not smply a vector wth dmenson (n m). The values of n and m are separately specfed and to specfy only the product would be to throw away nformaton that was ntally specfed. Thus, n order to keep ths nformaton, we can represent the result as an array of numbers havng n columns and m rows. Such an array can be called a matrx. For matrces, the products already defned have no smple nterpretaton. However, there s an addtonal product known as a matrx product, whch wll allow us to at least defne a matrx group. Consder the product defned by AB = C C = A kb. (1.2.4) k k Wth ths defnton of a product, the unt matrx denoted by 1 wll have elements δ specfed for n = m = 2 by 1 0 δ =. (1.2.5) 0 1 The quantty δ s called the Kronecker delta and may be generalzed to n-dmensons. Thus the nverse elements of the group wll have to satsfy the relaton k k AA -1 = 1, (1.2.6) 6
1 - Fundamental Concepts and we shall spend some tme n the next chapter dscussng how these members of the group may be calculated. Snce matrx addton can smply be defned as the scalar addton of the elements of the matrx, and the 'unt' matrx under addton s smply a matrx wth zero elements, t s temptng to thnk that the group of matrces also form a feld. However, the matrx product as defned by equaton (1.2.4), whle beng dstrbutve wth respect to addton, s not communtatve. Thus we shall have to be content wth matrces formng a group under both addton and matrx multplcaton but not a feld. There s much more that can be sad about matrces as was the case wth other subects of ths chapter, but we wll lmt ourselves to a few propertes of matrces whch wll be partcularly useful later. For example, the transpose of a matrx wth elements A s defned as T A = A. (1.2.7) We shall see that there s an mportant class of matrces (.e. the orthonormal matrces) whose nverse s ther transpose. Ths makes the calculaton of the nverse trval. Another mportant scalar quantty s the trace of a matrx defned as TrA = A. (1.2.8) A matrx s sad to be symmetrc f A = A. If, n addton, the elements are themselves complex numbers, then should the elements of the transpose be the complex conugates of the orgnal matrx, the matrx s sad to be Hermtan or self-adont. The conugate transpose of a matrx A s usually denoted by A. If the Hermtan conugate of A s also A -1, then the matrx s sad to be untary. Should the matrx A commute wth t Hermtan conugate so that AA = A A, (1.2.9) then the matrx s sad to be normal. For matrces wth only real elements, Hermtan s the same as symmetrc, untary means the same as orthonormal and both classes would be consdered to be normal. Fnally, a most mportant characterstc of a matrx s ts determnant. It may be calculated by expanson of the matrx by "mnors" so that a a a 11 det A = a 21 a 22 a 23 = a11(a 22a 33 a 23a 32 ) a12 (a 21a 33 a 23a 31) + a13 (a 21a 32 a 22a13 ). (1.2.10) a a a 13 12 23 13 33 Fortunately there are more straghtforward ways of calculatng the determnant whch we wll consder n the next chapter. There are several theorems concernng determnants that are useful for the manpulaton of determnants and whch we wll gve wthout proof. 1. If each element n a row or column of a matrx s zero, the determnant of the matrx s zero. 2. If each element n a row or column of a matrx s multpled by a scalar q, the determnant s multpled by q. 3. If each element of a row or column s a sum of two terms, the determnant equals the sum of the two correspondng determnants. 7
Numercal Methods and Data Analyss 4. If two rows or two columns are proportonal, the determnant s zero. Ths clearly follows from theorems 1, 2 and 3. 5. If two rows or two columns are nterchanged, the determnant changes sgn. 6. If rows and columns of a matrx are nterchanged, the determnant of the matrx s unchanged. 7. The value of a determnant of a matrx s unchanged f a multple of one row or column s added to another. 8. The determnant of the product of two matrces s the product of the determnants of the two matrces. One of the mportant aspects of the determnant s that t s a sngle parameter that can be used to characterze the matrx. Any such sngle parameter (.e. the sum of the absolute value of the elements) can be so used and s often called a matrx norm. We shall see that varous matrx norms are useful n determnng whch numercal procedures wll be useful n operatng on the matrx. Let us now consder a broader class of obects that nclude scalars, vectors, and to some extent matrces. 1.3 Coordnate Systems and Coordnate Transformatons There s an area of mathematcs known as topology, whch deals wth the descrpton of spaces. To most students the noton of a space s ntutvely obvous and s restrcted to the three dmensonal Eucldan space of every day experence. A lttle reflecton mght persuade that student to nclude the flat plane as an allowed space. However, a lttle further generalzaton would suggest that any tme one has several ndependent varables that they could be used to form a space for the descrpton of some phenomena. In the area of topology the noton of a space s far more general than that and many of the more exotc spaces have no known counterpart n the physcal world. We shall restrct ourselves to spaces of ndependent varables, whch generally have some physcal nterpretaton. These varables can be sad to consttute a coordnate frame, whch descrbes the space and are farly hgh up n the herarchy of spaces catalogued by topology. To understand what s meant by a coordnate frame, magne a set of rgd rods or vectors all connected at a pont. We shall call such a collecton of rods a reference frame. If every pont n space can be proected onto the rods so that a unque set of rod-ponts represent the space pont, the vectors are sad to span the space. If the vectors that defne the space are locally perpendcular, they are sad to form an orthogonal coordnate frame. If the vectors defnng the reference frame are also unt vectors say ê then the condton for orthogonalty can be wrtten as ê = δ, (1.3.1) ê 8
1 - Fundamental Concepts where δ s the Kronecker delta. Such a set of vectors wll span a space of dmensonalty equal to the number of vectors. Such a space need not be Eucldan, but f t s then the coordnate frame s sad to be ê a Cartesan coordnate frame. The conventonal xyz-coordnate frame s Cartesan, but one could magne such a coordnate system drawn on a rubber sheet, and then dstorted so that locally the orthogonalty condtons are stll met, but the space would no longer be Eucldan or Cartesan. Of the orthogonal coordnate systems, there are several that are partcularly useful for the descrpton of the physcal world. Certanly the most common s the rectangular or Cartesan coordnate frame where coordnates are often denoted by x, y, z or x 1, x 2, x 3. Other common three dmensonal frames nclude sphercal polar coordnates (r,θ, ϕ) and cylndrcal coordnates (ρ,ϑ,z). Often the most mportant part of solvng a numercal problem s choosng the proper coordnate system to descrbe the problem. For example, there are a total of thrteen orthogonal coordnate frames n whch Laplace's equaton s separable (see Morse and Feshbach 1 ). In order for coordnate frames to be really useful t s necessary to know how to get from one to another. That s, f we have a problem descrbed n one coordnate frame, how do we express that same problem n another coordnate frame? For quanttes that descrbe the physcal world, we wsh ther meanng to be ndependent of the coordnate frame that we happen to choose. Therefore we should expect the process to have lttle to do wth the problem, but rather nvolve relatonshps between the coordnate frames themselves. These relatonshps are called coordnate transformatons. Whle there are many such transformatons n mathematcs, for the purposes of ths summary we shall concern ourselves wth lnear transformatons. Such coordnate transformatons relate the coordnates n one frame to those n a second frame by means of a system of lnear algebrac equatons. Thus f a vector x r n one coordnate system has components x, n a prmed-coordnate system a vector x r ' to the same pont wll have components x' In vector notaton we could wrte ths as x = A x + B. (1.3.2) r r r x' = A x + B. (1.3.3) Ths defnes the general class of lnear transformaton where A s some matrx and B r s a vector. Ths general lnear form may be dvded nto two consttuents, the matrx A and the vector. It s clear that the vector B r may be nterpreted as a shft n the orgn of the coordnate system, whle the elements A are the cosnes of the angles between the axes X and X', and are called the drectons cosnes (see Fgure 1.1). r Indeed, the vector B s nothng more than a vector from the orgn of the un-prmed coordnate frame to the orgn of the prmed coordnate frame. Now f we consder two ponts that are fxed n space and a vector connectng them, then the length and orentaton of that vector wll be ndependent of the orgn of the coordnate frame n whch the measurements are made. That places an addtonal constrant on the types of lnear transformatons that we may consder. For nstance, transformatons that scaled each coordnate by a constant amount, whle lnear, would change the length of the vector as measured n the two coordnate systems. Snce we are only usng the coordnate system as a convenent way to descrbe the vector, the coordnate system can play no role n controllng the length of the vector. Thus we shall restrct our nvestgatons of lnear transformatons to those that transform orthogonal coordnate systems whle preservng the length of the vector. 9
Numercal Methods and Data Analyss Thus the matrx A must satsfy the rfollowng r condton r r r r x' x' = ( A x) ( Ax) = x x, (1.3.4) whch n component form becomes 2 A x A k x k = A A k x x k = x. k (1.3.5) Ths must be true for all vectors n the coordnate system so that 1 A A = δ = A A. (1.3.6) k k Now remember that the Kronecker delta δ s the unt matrx and any element of a group that multples another and produces that group's unt element s defned as the nverse of that element. Therefore A = [A ] -1. (1.3.7) Interchangng the rows wth the columns of a matrx produces a new matrx whch we have called the transpose of the matrx. Thus orthogonal transformatons that preserve the length of vectors have nverses that are smply the transpose of the orgnal matrx so that A -1 = A T. (1.3.8) Ths means that gven the transformaton A n the lnear system of equatons (1.3.3), we may nvert the transformaton, or solve the lnear equatons, by multplyng those equatons by the transpose of the orgnal matrx or r T r r T x = A x' A B. (1.3.9) Such transformatons are called orthogonal untary transformatons, or orthonormal transformatons, and the result gven n equaton (1.3.9) greatly smplfes the process of carryng out a transformaton from one coordnate system to another and back agan. We can further dvde orthonormal transformatons nto two categores. These are most easly descrbed by vsualzng the relatve orentaton between the two coordnate systems. Consder a transformaton that carres one coordnate nto the negatve of ts counterpart n the new coordnate system whle leavng the others unchanged. If the changed coordnate s, say, the x-coordnate, the transformaton matrx would be 1 0 0 A = 0 1 0, (1.3.10) 0 0 1 whch s equvalent to vewng the frst coordnate system n a mrror. Such transformatons are known as reflecton transformatons and wll take a rght handed coordnate system nto a left handed coordnate system. The length of any vectors wll reman unchanged. The x-component of these vectors wll smply be replaced by ts negatve n the new coordnate system. However, ths wll not be true of "vectors" that result from the vector cross product. The values of the components of such a vector wll reman unchanged mplyng that a reflecton transformaton of such a vector wll result n the orentaton of that vector beng changed. If you wll, ths s the orgn of the "rght hand rule" for vector cross products. A left hand rule k 10
1 - Fundamental Concepts results n a vector pontng n the opposte drecton. Thus such vectors are not nvarant to reflecton transformatons because ther orentaton changes and ths s the reason for puttng them n a separate class, namely the axal (pseudo) vectors. It s worth notng that an orthonormal reflecton transformaton wll have a determnant of -1. The untary magntude of the determnant s a result of the magntude of the vector beng unchanged by the transformaton, whle the sgn shows that some combnaton of coordnates has undergone a reflecton. Fgure 1.1 shows two coordnate frames related by the transformaton angles ϕ. Four coordnates are necessary f the frames are not orthogonal As one mght expect, the elements of the second class of orthonormal transformatons have determnants of +1. These represent transformatons that can be vewed as a rotaton of the coordnate system about some axs. Consder a transformaton between the two coordnate systems dsplayed n Fgure 1.1. The components of any vector C r n the prmed coordnate system wll be gven by C x' cosϕ11 cosϕ12 0 C x C y' = cosϕ21 cosϕ22 0 C y. (1.3.11) Cz' 0 0 1 C z If we requre the transformaton to be orthonormal, then the drecton cosnes of the transformaton wll not be lnearly ndependent snce the angles between the axes must be π/2 n both coordnate systems. Thus the angles must be related by ϕ11 = ϕ22 = ϕ ϕ12 = ϕ11 + π / 2 = ϕ + π / 2. (1.3.12) (2π ϕ ) = π / 2 ϕ = π / 2 ϕ 21 11 Usng the addton denttes for trgonometrc functons, equaton (1.3.11) can be gven n terms of the sngle 11
Numercal Methods and Data Analyss angle φ by C x' cosϕ sn ϕ 0 C x C y' = sn ϕ cosϕ 0 C y. (1.3.13) Cz' 0 0 1 C z Ths transformaton can be vewed as a smple rotaton of the coordnate system about the Z-axs through an angle ϕ. Thus, cosϕ sn ϕ 0 Det sn ϕ cosϕ 0 0 1 0 = cos 2 ϕ + sn 2 ϕ = + 1. (1.3.14) In general, the rotaton of any Cartesan coordnate system about one of ts prncpal axes can be wrtten n terms of a matrx whose elements can be expressed n terms of the rotaton angle. Snce these transformatons are about one of the coordnate axes, the components along that axs reman unchanged. The rotaton matrces for each of the three axes are 1 0 0 Px ( φ) = 0 cosφ sn φ 0 sn φ cosφ cosφ 0 sn φ Py ( φ) = 0 1 0. (1.3.15) sn φ 0 cosφ cosφ sn φ 0 P ( φ) = sn φ cosφ 0 z 0 0 1 It s relatvely easy to remember the form of these matrces for the row and column of the matrx correspondng to the rotaton axs always contans the elements of the unt matrx snce that component s not affected by the transformaton. The dagonal elements always contan the cosne of the rotaton angle whle the remanng off dagonal elements always contan the sne of the angle modulo a sgn. For rotatons about the x- or z-axes, the sgn of the upper rght off dagonal element s postve and the other negatve. The stuaton s ust reversed for rotatons about the y-axs. So mportant are these rotaton matrces that t s worth rememberng ther form so that they need not be re-derved every tme they are needed. One can show that t s possble to get from any gven orthogonal coordnate system to another through a seres of three successve coordnate rotatons. Thus a general orthonormal transformaton can always be wrtten as the product of three coordnate rotatons about the orthogonal axes of the coordnate systems. It s mportant to remember that the matrx product s not commutatve so that the order of the 12
1 - Fundamental Concepts rotatons s mportant. 1.4 Tensors and Transformatons Many students fnd the noton of tensors to be ntmdatng and therefore avod them as much as possble. After all Ensten was once quoted as sayng that there were not more than ten people n the world that would understand what he had done when he publshed General Theory of Relatvty. Snce tensors are the foundaton of general relatvty that must mean that they are so esoterc that only ten people could manage them. Wrong! Ths s a beautful example of msnterpretaton of a quote taken out of context. What Ensten meant was that the notaton he used to express the General Theory of Relatvty was suffcently obscure that there were unlkely to be more than ten people who were famlar wth t and could therefore understand what he had done. So unfortunately, tensors have generally been represented as beng far more complex than they really are. Thus, whle readers of ths book may not have encountered them before, t s hgh tme they dd. Perhaps they wll be somewhat less ntmdated the next tme, for f they have any ambton of really understandng scence, they wll have to come to an understandng of them sooner or later. In general a tensor has N n components or elements. N s known as the dmensonalty of the tensor by analogy wth vectors, whle n s called the rank of the tensor. Thus scalars are tensors of rank zero and vectors of any dmenson are rank one. So scalars and vectors are subsets of tensors. We can defne the law of addton n the usual way by the addton of the tensor elements. Thus the null tensor (.e. one whose elements are all zero) forms the unt under addton and arthmetc subtracton s the nverse operaton. Clearly tensors form a communtatve group under addton. Furthermore, the scalar or dot product can be generalzed for tensors so that the result s a tensor of rank m n. In a smlar manner the outer product can be defned so that the result s a tensor of rank m + n. It s clear that all of these operatons are closed; that s, the results reman tensors. However, whle these products are n general dstrbutve, they are not communtatve and thus tensors wll not form a feld unless some addtonal restrctons are made. One obvous way of representng tensors of rank 2 s as N N square matrces Thus, the scalar product of a tensor of rank 2 wth a vector would be wrtten as A B r = C r C = A B, (1.4.1) whle the tensor outer product of the same tensor and vector could be wrtten as t AB = C. (1.4.2) C = k A Bk 13
Numercal Methods and Data Analyss It s clear from the defnton and specfcally from equaton (1.4.2) that tensors may frequently have a rank of more than two. However, t becomes more dffcult to dsplay all the elements n a smple geometrcal fashon so they are generally ust lsted or descrbed. A partcularly mportant tensor of rank three s known as the Lev-Cvta Tensor (or correctly the Lev-Cvta Tensor Densty). It plays a role that s somewhat complmentary to that of the Kronecker delta n that when any two ndces are equal the tensor element s zero. When the ndces are all dfferent the tensor element s +1 or -1 dependng on whether the ndex sequence can be obtaned as an even or odd permutaton from the sequence 1, 2, 3 respectvely. If we try to represent the tensor ε k as a successon of 3 3 matrces we would get 0 0 0 ε1k = 0 0 + 1 0 1 0 0 0 1 ε 2 k = 0 0 0. (1.4.3) + 1 0 0 0 1 0 ε = + 1 0 0 3 k 0 0 0 Ths somewhat awkward lookng thrd rank tensor allows us to wrte the equally awkward vector cross product n summaton notaton as r r t r r A B = ε :(AB) = ε A B C. (1.4.4) k k = Here the symbol : denotes the double dot product whch s explctly specfed by the double sum of the rght hand term. The quantty εk s sometmes called the permutaton symbol as t changes sgn wth every permutaton of ts ndces. Ths, and the dentty εk εpq = δ pδ kq δ qδ kp, (1.4.5) makes the evaluaton of some complcated vector denttes much smpler (see exercse 13). In secton 1.3 we added a condton to what we meant by a vector, namely we requred that the length of a vector be nvarant to a coordnate transformaton. Here we see the way n whch addtonal constrants of what we mean by vectors can be specfed by the way n whch they transform. We further lmted what we meant by a vector by notng that some vectors behave strangely under a reflecton transformaton and callng these pseudo-vectors. Snce the Lev-Cvta tensor generates the vector cross product from the elements of ordnary (polar) vectors, t must share ths strange transformaton property. Tensors that share ths transformaton property are, n general, known as tensor denstes or pseudo-tensors. Therefore we should call εk defned n equaton (1.4.3) the Lev-Cvta tensor densty. Indeed, t s the nvarance of tensors, vectors, and scalars to orthonormal transformatons that s most correctly used to defne the elements of the group called tensors. k 14
1 - Fundamental Concepts Fgure 1.2 shows two neghborng ponts P and Q n two adacent coordnate systems X and X'. The dfferental dstance between the two s dx r. The vectoral dstance to the two ponts s X r (P) or X r ' (P) and X r (Q) or?x r '(Q) respectvely. Snce vectors are ust a specal case of the broader class of obects called tensors, we should expect these transformaton constrants to extend to the general class. Indeed the only fully approprate way to defne tensors s to defne the way n whch they transform from one coordnate system to another. To further refne the noton of tensor transformaton, we wll look more closely at the way vectors transform. We have wrtten a general lnear transformaton for vectors n equaton (1.3.2). However, except for rotatonal and reflecton transformatons, we have sad lttle about the nature of the transformaton matrx A. So let us consder how we would express a coordnate transformaton from some pont P n a space to a nearby neghborng pont Q. Each pont can be represented n any coordnate system we choose. Therefore, let us consder two coordnate systems havng a common orgn where the coordnates are denoted by x and x' respectvely. Snce P and Q are near each other, we can represent the coordnates of Q to those of P n ether coordnate system by x (Q) = x (P) + dx. (1.4.6) x' (Q) = x' (P) + dx' = Now the coordnates of the vector from P to Q wll be dx and dx, n the un-prmed and prmed coordnate systems respectvely. By the chan rule the two coordnates wll be related by 15
Numercal Methods and Data Analyss x' dx ' = dx. (1.4.7) x Note that equaton (1.4.7) does not nvolve the specfc locaton of pont Q but rather s a general expresson of the local relatonshp between the two coordnate frames. Snce equaton (1.4.7) bascally descrbes how the coordnates of P or Q wll change from one coordnate system to another, we can dentfy the elements A from equaton (1.3.2) wth the partal dervatves n equaton (1.4.6). Thus we could expect any vector x? to transform accordng to x' x ' = x. (1.4.8) x Vectors that transform n ths fashon are called contravarant vectors. In order to dstngush them from covarant vectors, whch we shall shortly dscuss, we shall denote the components of the vector wth superscrpts nstead of subscrpts. Thus the correct form for the transformaton of a contravarant vector s x' x ' = x. (1.4.9) x We can generalze ths transformaton law to contravarant tensors of rank two by kl x' x' T ' = T l k. (1.4.10) x x k l Hgher rank contravarant tensors transform as one would expect wth addtonal coordnate changes. One mght thnk that the use of superscrpts to represent contravarant ndces would be confused wth exponents, but such s generally not the case and the dstncton between ths sort of vector transformaton and covarance s suffcently mportant n physcal scence to make the accommodaton. The sorts of obects that transform n a contravarant manner are those assocated wth, but not lmted to, geometrcal obects. For example, the nfntesmal dsplacements of coordnates that makes up the tangent vector to a curve show that t s a contravarant vector. Whle we have used vectors to develop the noton of contravarance, t s clear that the concept can be extended to tensors of any rank ncludng rank zero. The transformaton rule for such a tensor would smply be T ' = T. (1.4.11) In other words scalars wll be nvarant to contravarant coordnate transformatons. Now nstead of consderng vector representatons of geometrcal obects mbedded n the space and ther transformatons, let us consder a scalar functon of the coordnates themselves. Let such a functon be Φ(x ). Now consder components of the gradent of Φ n the x' -coordnate frame. Agan by the chan rule Φ x' = x x Φ x'. (1.4.12) If we call Φ / x' a vector wth components V, then the transformaton law gven by equaton (1.4.12) appears very lke equaton (1.4.8), but wth the partal dervatves nverted. Thus we would dentfy the elements A of the lnear vector transformaton represented by equaton (1.3.2) as A = x / x', (1.4.13) and the vector transformaton would have the form 16
1 - Fundamental Concepts V = A V. (1.4.14) Vectors that transform n ths manner are called covarant vectors. In order to dstngush them from contravarant vectors, the component ndces are wrtten as subscrpts. Agan, t s not dffcult to see how the concept of covarance would be extended to tensors of hgher rank and specfcally for a second rank covarant tensor we would have l k x x T ' = Tlk. (1.4.15) x' x' k l The use of the scalar nvarant Φ to defne what s meant by a covarant vector s a clue as to the types of vectors that behave as covarant vectors. Specfcally the gradent of physcal scalar quanttes such as temperature and pressure would behave as a covarant vector whle coordnate vectors themselves are contravarant. Bascally equatons (1.4.15) and (1.4.10) defne what s meant by a covarant or contravarant tensor of second rank. It s possble to have a mxed tensor where one ndex represents covarant transformaton whle the other s contravarant so that l k x x T ' = Tl. (1.4.16) x' x k l k Indeed the Kronecker delta can be regarded as a tensor as t s a two ndex symbol and n partcular t s a mxed tensor of rank two and when covarance and contravarance are mportant should be wrtten as. Remember that both contravarant and covarant transformatons are locally lnear transformatons of the form gven by equaton (1.3.2). That s, they both preserve the length of vectors and leave scalars unchanged. The ntroducton of the terms contravarance and covarance smply generate two subgroups of what we earler called tensors and defned the members of those groups by means of ther detaled transformaton propertes. One can generally tell the dfference between the two types of transformatons by notng how the components depend on the coordnates. If the components denote 'dstances' or depend drectly on the coordnates, then they wll transform as contravarant tensor components. However, should the components represent quanttes that change wth the coordnates such as gradents, dvergences, and curls, then dmensonally the components wll depend nversely on the coordnates and the wll transform covarantly. The use of subscrpts and superscrpts to keep these transformaton propertes straght s partcularly useful n the development of tensor calculus as t allows for the development of rules for the manpulaton of tensors n accord wth ther specfc transformaton characterstcs. Whle coordnate systems have been used to defne the tensor characterstcs, those characterstcs are propertes of the tensors themselves and do not depend on any specfc coordnate frame. Ths s of consderable mportance when developng theores of the physcal world as anythng that s fundamental about the unverse should be ndependent of man made coordnate frames. Ths s not to say that the choce of coordnate frames s unmportant when actually solvng a problem. Qute the reverse s true. Indeed, as the propertes of the physcal world represented by tensors are ndependent of coordnates and ther explct representaton and transformaton propertes from one coordnate system to another are well defned, they may be qute useful n reformulatng numercal problems n dfferent coordnate systems. δ 17
Numercal Methods and Data Analyss 1.5 Operators The noton of a mathematcal operator s extremely mportant n mathematcal physcs and there are entre books wrtten on the subect. Most students frst encounter operators n calculus when the notaton [d/dx] s ntroduced to denote the operatons nvolved n fndng the dervatve of a functon. In ths nstance the operator stands for takng the lmt of the dfference between adacent values of some functon of x dvded by the dfference between the adacent values of x as that dfference tends toward zero. Ths s a farly complcated set of nstructons represented by a relatvely smple set of symbols. The desgnaton of some symbol to represent a collecton of operatons s sad to represent the defnton of an operator. Dependng on the detals of the defnton, the operators can often be treated as f they were quanttes and subect to algebrac manpulatons. The extent to whch ths s possble s determned by how well the operators satsfy the condtons for the group on whch the algebra or mathematcal system n queston s defned. The operator [d/dx] s a scalar operator. That s, t provdes a sngle result after operatng on some functon defned n an approprate coordnate space. It and the operator represent the fundamental operators of the nfntesmal calculus. Snce [d/dx] and carry out nverse operatons on functons, one can defne an dentty operator by [d/dx] so that contnuous dfferentonable functons wll form a group under the acton of these operators. In numercal analyss there are analogous operators and Σ that perform smlar functons but wthout takng the lmt to vanshngly small values of the ndependent varable. Thus we could defne the forward fnte dfference operator by ts operaton on some functon f(x) so that f(x) = f(x+h) - f(x),.(1.5.1) where the problem s usually scaled so that h = 1. In a smlar manner Σ can be defned as n = 0 f (x ) = f (x) + f (x + h) + f (x + 2h) + f (x + h) + f (x + nh). (1.5.2) Such operators are most useful n expressng formulae n numercal analyss. Indeed, t s possble to buld up an entre calculus of fnte dfferences. Here the base for such a calculus s 2 nstead of e=2.7182818... as n the nfntesmal calculus. Other operators that are useful n the fnte dfference calculus are the shft operator E[f(x)] and the Identty operator I[f(x)] whch are defned as E[f(x)] f(x + h). (1.5.3) I[f(x)] f(x) These operators are not lnearly ndependent as we can wrte the forward dfference operator as = E - I. (1.5.4) The fnte dfference and summaton calculus are extremely powerful when summng seres or evaluatng convergence tests for seres. Before attemptng to evaluate an nfnte seres, t s useful to know f the seres converges. If possble, the student should spend some tme studyng the calculus of fnte dfferences. In addton to scalar operators, t s possble to defne vector and tensor operators. One of the most common vector operators s the "del" operator or "nabla". It s usually denoted by the symbol and s defned n Cartesan coordnates as 18
1 - Fundamental Concepts = î + ĵ + kˆ. (1.5.5) x y z Ths sngle operator, when combned wth the some of the products defned above, consttutes the foundaton of vector calculus. Thus the dvergence, gradent, and curl are defned as r A = b r a = B, (1.5.6) r r A = C respectvely. If we consder A r to be a contnuous vector functon of the ndependent varables that make up the space n whch t s defned, then we may gve a physcal nterpretaton for both the dvergence and curl. The dvergence of a vector feld s a measure of the amount that the feld spreads or contracts at some gven pont n the space (see Fgure 1.3)..Fgure 1.3 schematcally shows the dvergence of a vector feld. In the regon where the arrows of the vector feld converge, the dvergence s postve, mplyng an ncrease n the source of the vector feld. The opposte s true for the regon where the feld vectors dverge. Fgure 1.4 schematcally shows the curl of a vector feld. The drecton of the curl s determned by the "rght hand rule" whle the magntude depends on the rate of change of the x- and y- components of the vector feld wth respect to y and x.. 19
Numercal Methods and Data Analyss The curl s somewhat harder to vsualze. In some sense t represents the amount that the feld rotates about a gven pont. Some have called t a measure of the "swrlness" of the feld. If n the vcnty of some pont n the feld, the vectors tend to veer to the left rather than to the rght, then the curl wll be a vector pontng up normal to the net rotaton wth a magntude that measures the degree of rotaton (see Fgure 1.4). Fnally, the gradent of a scalar feld s smply a measure of the drecton and magntude of the maxmum rate of change of that scalar feld (see Fgure 1.5). Fgure 1.5 schematcally shows the gradent of the scalar dot-densty n the form of a number of vectors at randomly chosen ponts n the scalar feld. The drecton of the gradent ponts n the drecton of maxmum ncrease of the dot-densty whle the magntude of the vector ndcates the rate of change of that densty. Wth these smple pctures n mnd and what we developed n secton 1.4 t s possble to generalze the noton of the Del-operator to other quanttes. Consder the gradent of a vector feld. Ths represents the outer product of the Del-operator wth a vector. Whle one doesn't see such a thng often n freshman physcs, t does occur n more advanced descrptons of flud mechancs (and many other places). We now know enough to understand that the result of ths operaton wll be a tensor of rank two whch we can represent as a matrx. What do the components mean? Generalze from the scalar case. The nne elements of the vector gradent can be vewed as three vectors denotng the drecton of the maxmum rate of change of each of the components of the orgnal vector. The nne elements represent a perfectly well defned quantty and t has a useful purpose n descrbng many physcal stuatons. One can also consder the dvergence of a second rank tensor, whch s clearly a vector. In hydrodynamcs, the dvergence of the pressure tensor may reduce to the gradent of the scalar gas pressure f the macroscopc flow of the materal s small compared to the nternal speed of the partcles that make up the materal. Wth some care n the defnton of a collecton of operators, ther acton on the elements of a feld or group wll preserve the feld or group nature of the orgnal elements. These are the operators that are of the greatest use n mathematcal physcs. 20