Physics 202 Laboratory 5 Linear Algebra Laboratory 5 Physics 202 Laboratory We close our whirlwind tour of numerical methods by advertising some elements of (numerical) linear algebra. There are three major problems one needs to solve numerically the eigenvalue problem, the matrix inverse problem, and least squares. We ll focus on the physical targets for the first two, and outline the solution methods relevant there. Least squares, since we will ignore it, deserves additional mention. The least squares problem is defined as follows given a matrix A IR n m, with n rows and m columns 2, and a vector b IR n, find a vector x IR m such that: A x b (5.) is minimized. This problem is similar in spirit to matrix inversion (to which it reduces when m = n), and can be used to solve for best fit coefficients in data analysis, for example. 5. Physical Motivation We ll think about problems that reduce to matrix inversion first, then consider physics that leads to the eigenvalue problem. You have seen multiple examples of each, and my job here is to remind you of them, and set up the matrix form of their solution. The matrix inversion problem can be expressed as: Given a matrix A IR n n, and a vector b IR n, find x IR n such that: A x = b. (5.2) Some of this material appears in Computational Methods for Physics by Joel Franklin, Cambridge University Press, 203 and is used with the author s permission. 2 It is customary to let n > m, so we think of long matrices.
5.. PHYSICAL MOTIVATION Laboratory 5 When the solution exists, it is generally written: x = A b. The eigenvalue problem is defined as follows: Given a matrix A IR n n, find the vectors {v i } n i= and numbers {λ i} n i= such that A v i = λ i v i for all i = n. 5.. ODEs and PDEs by Inversion A linear, first order differential equation with initial condition, like: f (x) + p(x) f(x) = q(x) f(0) = f 0 (5.3) for given p(x) and q(x) can be solved on a grid with uniform spacing let x j j x for constant x, and write the projections of p, q and (the approximation to) f(x) on the grid as p j p(x j ), q j q(x j ) and f j f(x j ) respectively. Then the ODE is approximated by the difference equation: f j+ f j + p j f j = q j for all j =... N. (5.4) Suppose we think of the elements of the projection f j as entries in a vector of length N: f f 2 f =., (5.5) f N then we can construct a matrix that represents the left-hand side of the approximation (5.4), and a vector approximating the right-hand side: p 0 0 0 0... f q + f 0 p 2 0 0 0... 0 f 2 q 2 p 3 0 0... f 3 q 3 0 0 p 4 0... f 4 = q 4 0 0 0 p 5... f 5 q 5. 0 0 0 0.......... 0 0 0 0 0 p N f N q N (5.6) Now the discretization is relatively straightforward the first row is a little different than the others because, for j =, our ODE approximation reads: f N+. f 2 f 0 + p f = q, (5.7) 2
5.. PHYSICAL MOTIVATION Laboratory 5 and since f 0 is given, it goes over to the right-hand side of the equation and acts as part of the known vector. The same thing happens at j = N: f N+ f N + p N f N = q N, (5.8) but we are not justified in counting f N+ as a known value. Our matrix method appears to require two boundary conditions, and artificially so usually, something in the problem will help us approximate a value at N + (beyond the grid, and hence outside of our interest in the solution). For example, take f (x) + x f(x) = 0, so that p(x) = x and q(x) = 0. We know that for x very large, we have x f(x) 0, so that f(x) must be small, and we might use that observation to define a large value for x N+ and set f(x N+ ) = 0, at least numerically. However we manage to set the final value, it is clear that we have achieved matrix inverse form, since (5.6) clearly defines a known matrix A IR N N, and right-hand-side vector b IR N, our solution to this, f = A b is precisely our numerical approximation to the function f(x) evaluated on the grid. Many times, we are interested in second order ODEs, as in Newton s second law, for example, and in these cases, a boundary condition like f N+ = f f, provided, is reasonable, so our numerical approximation matches the physical boundaries. Think of approximating the most general second order linear ODE: f (x) + s(x) f (x) + p(x) f(x) = q(x) f(0) = f 0 f(x) = f X (5.9) (for final position X, where the value of f is given: f X )this equation would have second derivative approximated using: f (x j ) f j+ 2 f j + f j x 2 (5.0) and would also rely on known values for f 0 and f N+, but these are provided in the problem specification. We might let x N = X x = N x, giving us the definition of x in terms of the endpoint and N: x = X N +. (5.) Then x N+ = X, and we are given the value f N+ = f X. 3
5.. PHYSICAL MOTIVATION Laboratory 5 Example Suppose we take a damped harmonic oscillator that will give us a good check of the numerical method. Start with the usual form: where ω k m ẍ(t) = ω 2 x(t) 2 γ ω ẋ(t) (5.2) is the usual frequency of oscillation, and the damping term is governed by the parameter γ (dimensionless, in this case). We ll start by nondimensionalizing the equation, let t t 0 s and x x 0 q, then the above becomes d 2 q(s) ds 2 = q(s) ω 2 t 2 0 2 γ dq(s) ds ω t 0, (5.3) and it is clear that we should take t 0 /ω, defining a natural timescale. The only parameter left is γ, and our choice of scaling for x 0 (typically the initial extension): d 2 q(s) ds 2 = q(s) 2 γ dq(s) ds. (5.4) We ll start, and end, with the ansatz: through (5.4) gives q(s) = e α s, running this α 2 + 2 γ α + α = 2 γ ± 4 γ 2 4 2 = γ ± γ 2 (5.5) from which we extract the two independent solutions (for γ ): ( ) ( ) γ+ γ q(s) = A e 2 s γ γ + B e 2 s. (5.6) Let s put all the constants back in, just to see how that goes: x(t) = x 0 e γ ω t [ A e γ 2 ω t + B e γ 2 ω t ]. (5.7) Now the particular behavior of the system is governed by the factor γ 2 if this is less than zero, there is oscillation (in addition to the damping), while if it is greater than zero, the system has only the decaying (and growing) exponentials as solutions. 4
5.. PHYSICAL MOTIVATION Laboratory 5 Partial differential equations can also be handled using a discretization, and reduction to matrix inverse. In E&M, the electrostatic potential V is sourced by charge density (charge per unit volume) ρ via: 2 V = ρ ɛ 0, where ɛ 0 sets the units for V given charge in Coulombs. This is an example of Poisson s equation, and it is the PDE version of the ODE f (x) = q(x). The analytic solution is more complicated, but numerically, all we do is take a grid (in two dimensions, just to show the pattern): x j j x, y k k y, and then approximate the Laplacian on the grid via: ( 2 2 ) V V (x j, y k ) x 2 + 2 V y 2 x=x j,y=y k V (x j+, y k ) 2 V (x j, y k ) + V (x j, y k ) x 2 + V (x j, y k+ ) 2 V (x j, y k ) + V (x j, y k ) y 2. (5.8) The unknown values V (x j, y k ) must be put in a column vector somehow, and the matrix approximating the derivative values must be adapted to that embedding. But, when the dust has settled, we recover a matrix inverse problem. 5..2 Eigenvalue Problem The eigenvalue problem is already interesting in its pure matrix instantiation here we ll look at how discretization can render continuous versions of this problem into a finite form. Schrödinger s equation governing the allowed energies, E, of a particle of mass m moving under the influence of a potential V (x) reads: 2 2 m ψ (x) + V (x) ψ(x) = E ψ(x). (5.9) There are two targets to a solution of this equation, the first is ψ(x) itself, giving us a tool to calculate experimental outcomes (remember that ψ(x) ψ(x) dx represents the probability of finding the particle in the dx vicinity of x). The second is E, interpreted as the energy of the particle that is not an input, and must come from the solution. So if you squint at (5.9), you see that [ 2 2 m d 2 ] dx 2 + V (x) ψ(x) = E ψ(x) (5.20) 5
5.. PHYSICAL MOTIVATION Laboratory 5 defines a sort of continuous eigenvalue problem, with ψ(x) playing the role of the eigenvectors, and E playing the role of the eigenvalues. Remember that, as a second order ODE, the solutions will have two undetermined constants that can be set by provided boundary conditions. For simplicity, we ll try to find solutions that have well-defined zeroes for the wavefunction, so let ψ(0) = ψ(a) = 0. These are reasonable, and could come from the form of the potential, or our demand that at some local infinity, the wavefunction must vanish. We nondimensionalize the above in the usual way, letting x = a q and then defining Ẽ 2 m a2 2 m a2 E and Ṽ (q) V (a q), so that: 2 2 d2 ψ(q) dq 2 + Ṽ (q) ψ(q) = Ẽ ψ(q). (5.2) Suppose we put this problem onto a uniform grid with q j = j q for fixed q. We have q 0 = 0, and we ll let q N+ = q f, giving q = q f N+. Now the discrete approximation to (5.2), using the usual notation ψ j ψ(q j ), is: ψ j+ 2 ψ j + ψ j q 2 + Ṽj ψ j = Ẽ ψ j j =,..., N. (5.22) Once again, we can put the elements ψ j into a vector, ψ = ψ ψ 2. ψ N, (5.23) and define the matrix Q as: Ṽ + 2 0 0 0 0... q 2 q 2 Ṽ q 2 2 + 2 0 0 0... q 2 q 2 0 Ṽ q 2 3 + 2 0 0... q 2 q 2 0 0 Ṽ q 2 4 + 2 0... q 2 q 2 0 0 0 Ṽ q 2 5 + 2... q 2 q 2. 0 0 0 0........ 0 0 0 0 0 Ṽ q 2 N + 2 q 2, (5.24) 6
5.. PHYSICAL MOTIVATION Laboratory 5 and then the discretized problem (5.22) reads: Q ψ = Ẽ ψ. (5.25) There will be N vectors ψ and N values of Ẽ, so this equation defines an eigenvalue problem. Note that the matrix Q is symmetric and real, so it is guaranteed to have orthogonal eigenvectors with real eigenvalues (which is good if we are to understand Ẽ as an energy). Notice that our boundary condition is enforced: ψ 0 = ψ N+ = 0, since the j = version of (5.22) reads: ψ 2 2 ψ q 2 + Ṽ ψ = Ẽ ψ (5.26) which is correctly encoded in the first row of the matrix Q similar considerations apply to the j = N case. Example the hydrogen spectrum Hydrogen, from a physics point of view, consists of an electron (carrying charge e) moving in the central potential set up by a proton (with charge e). The potential is just the usual Coulomb one: e2 V (x) = 4 π ɛ 0 x, (5.27) and we can put this in our non-dimensionalized form: Ṽ (q) = 2 m a2 e 2 [ ] m e 2 a 4 π ɛ 0 2 a q = 2 4 π ɛ 0 2 q. (5.28) The factor in brackets on the right defines a length, called the Bohr radius, and usually denoted a 4 π ɛ 0 2 (lucky coincidence) let this m e 2 be our fundamental scaling length a, and then the final form of the potential is: Ṽ (q) = 2 q. (5.29) The nondimensionalized Schrödinger equation for this specific potential reads d2 ψ(q) dq 2 2 ψ(q) = Ẽ ψ(q). (5.30) q 7
5.2. METHODS Laboratory 5 If we solve this equation by discretizing it, we can get accurate approximations to the bound state energies of hydrogen. There are two fundamentally different behaviors we can study in both classical and quantum mechanics bound states are, here, states of negative energy, and are something like the classical orbits associated with central potentials. There are also scattering states. In the case of hydrogen, these have positive energy, and are a quantum mechanical analogue of unbound classical motion (shoot an electron passed a central charge). 5.2 Methods Here, I will just sketch the ideas, since they are fundamentally good ideas, and form the basis for much of the actual algorithms that are implemented to solve the matrix inversion and eigenvalue problems. But, in the end, this is just to give you a taste it is interesting to note that what a numerical linear algebra package does to solve these problems is very different than what you have learned to do. 5.2. Inverse Problem Most methods for solving for x in A x = b proceed from a particular factorization of the matrix A. We ll consider the special case of the so-called QR-factorization. For an invertible matrix A IR n n, it is possible to decompose A into an orthogonal matrix Q (orthogonal matrices, remember, have the property that Q = Q T ), and an upper-triangular matrix R (upper triangular matrices have non-zero entries only on and above the diagonal) such that: A = Q R. (5.3) The advantage of this observation is that we have reduced the inversion of A to the inversion of two matrices, each with simple inverses: and then we just need to solve x in R x = c. Q R x = b R x = Q T b c (5.32) 8
5.2. METHODS Laboratory 5 Let s think about that process the equation R x = c is really n equalities. The last of these reads: R nn x n = c n (5.33) (since R is upper triangular, its last row only has one entry, the last) from which we can easily solve for x n : x n = c n /R nn. One row up, we have, again using the upper triangularity of R: R (n )(n ) x n + R (n )n x n = c n (5.34) but since we know x n, we can again solve directly for x n : x n = c n R (n )n x n R (n )(n ). (5.35) We can continue to work upwards until we reach x, which will depend on the now known x 2... x n. This inversion scheme is easy to implement, and that was the whole point of the factorization the Q matrix is trivial to invert, and R isn t much worse. How do we know that this factorization, A = Q R, exists? The proof is typical of linear algebra factorization proofs we can sketch it by constructing the desired decomposition. Recall the Gram-Schmidt procedure: Given a collection of vectors a, a 2,..., a n all in IR n, we can form a spanning set (orthogonal and normalized in this case) by working sequentially. Start with q = a, and let R = q q, then take Q = q /R, a vector that points in the same direction as a but is normalized. Next let q 2 = a 2 (Q a 2 ) }{{} Q, (5.36) R 2 i.e. a vector that points in the a 2 direction but with the component in the Q direction projected out. Again define R 22 = q 2 q 2 and let Q 2 = q 2 /R 22, a normalized vector. Moving along, we start with a vector that points in the a 3 direction, with components in the Q and Q 2 directions projected out: q 3 = a 3 (Q a 3 ) Q }{{} (Q 2 a 3 ) Q }{{} 2, (5.37) R 3 R 23 and as before, set R 33 = q 3 q 3, then Q 3 = q 3 /R 33. So far, we have a collection of three orthonormal vectors {Q i } 3 i=, and a = R Q a 2 = R 2 Q + R 22 Q 2 a 3 = R 3 Q + R 23 Q 2 + R 33 Q 3, (5.38) 9
5.2. METHODS Laboratory 5 a situation we can summarize as follows: [ ] [ ] a a 2 a 3 = Q Q 2 Q 3 R R 2 R 3 0 R 22 R 23 0 0 R 33, (5.39) where the square brackets on the left and right indicate matrices whose columns are the {a i } 3 i= (on the left) and {Q i} 3 i= (on the right). If we continue the process, we will eventually get the matrix A on the left (with columns that are precisely the provided {a i } n i= ), the first matrix on the left is an orthogonal matrix, call it Q, and the second matrix on the right is upper triangular, the R in A = Q R. The Power Method Finally, we close with a clever idea for computing the eigenvalues and eigenvectors of a matrix. For our purposes, let A IR n n be a symmetric matrix, so that its eigenvalues are real and its eigenvectors are orthogonal and complete (they span IR n, and can be taken as a basis set) take the eigenvectors to be normalized, so that vj T v j = for all j =... n. We ll further assume that the eigenvalues are non-degenerate, and can be ordered as follows: λ > λ 2 > λ 3 >... > λ n >. Take a generic vector x IR n. We know that it can be decomposed into the eigenvectors, even though we don t know what the eigenvectors are: n x = a i v i. (5.40) i= Now multiply both sides of this equation by A by assumption, we have A v i = λ i v i, so: n n A x = a i A v i = a i λ i v i. (5.4) i= Because the set {v i } n i= are eigenvectors, subsequent multiplication will just introduce additional factors of λ i : n A p x = a i λ p i v i. (5.42) i= Since λ > {λ i } n i=2 >, we know that the factor λp will grow faster than all the others, so for p large: i= A p x a λ p v, (5.43) 0
5.2. METHODS Laboratory 5 and if we set y A p x, it is clear that v = y y T y. (5.44) We have found the first eigenvector, and can even find the associated eigenvalue. But what about the rest? Suppose we now want to find v 2? Well, with v in hand, we can iterate the following to find v 2 this is another example of the Gram-Schmidt procedure: w 0 = x for j = p w j = A w j w j = w j ( w T j v ) v end v 2 = w p w T p w p. (5.45) Here, at each stage of the multiplication, we project out any v component, leaving us with the second-fastest-growing eigenvalue, the one associated with v 2. The process continues we now find the third eigenvector by multiplying by A and projecting out the v and v 2 directions. Finally, with all the eigenvectors in place, we can collect the eigenvalues: A v i = λ i (5.46) i.e. we just take the norm of the product A v i.
5.2. METHODS Laboratory 5 Lab Problem 5. Take the matrix: and the vector A =5 w = 2 3 4 2 0 20 30 3 20 5 6 4 30 6 8 2 2 (5.47). (5.48) Use the power method to find the first eigenvector and eigenvalue of A, record the normalized eigenvector, the eigenvalue, and the number of multiplications by A you used (how did you settle on this number?) below: 2
5.2. METHODS Laboratory 5 Problem 5.2 Find the negative energy eigenstates of hydrogen using the built-in function Eigenvalues (start by writing a function that generates the appropriate matrix). Use q f = 5, 20, 40, and 80 and N = 000. a. Write the negative energies you get for each value of q f below (you can use the built-in function Sort to order the eigenvalues): b. Take the q f = 80 case using the first four (negative) eigenvalues, find a function f(n) for n =, 2, 3, 4 that (roughly) gives you the numerical result. Write your function below: Finally, calculate the pre-factor to get E(n) for hydrogen (what you have in your numerical routine is Ẽ(n)) use the electron-volt as your unit of energy. 3