Lecture 31: Some Applications of Eigenvectors: Markov Chains and Chemical Reaction Systems Winfried Just Department of Mathematics, Ohio University April 9 11, 2018
Review: Eigenvectors and left eigenvectors A nonzero column vector x is an eigenvector aka right eigenvector of a square matrix A with eigenvalue λ if A x = λ x. A nonzero row vector y is a left eigenvector of a square matrix A with eigenvalue λ if ya = λ y. Note that y left-multiplies A here. By Homework 91, x is a (right) eigenvector of A with eigenvalue λ if, and only if, y = x T is a left eigenvector of A T with the same eigenvalue λ.
Review: Markov chains A Markov chain is a stochastic process. Time proceeds in discrete steps t = 0, 1, 2,... At each time t the process can only be in one of several states that are numbered 1,..., n. The probability of being in a given state at time t + 1 depends only on the state at time t. The matrix P = [p ij ] n n gives the transition probabilities p ij from state i at time t to state j at time t + 1. When x(t) = [x 1 (t),..., x n (t)] is the probability distribution for the states at time t, then the probability distribution x(t + 1) at time t + 1 is given by x(t + 1) = x(t)p = [x 1 (t),..., x n (t)]p.
Review: Markov chains for weather.com light Time proceeds in steps of days. State 1: sunny day, State 2: rainy day. Each day is somehow unambiguously classified in this way. The meaning of the transition probabilities: p 11 is the probability that a sunny day is followed by another sunny day. p 12 is the probability that a sunny day is followed by a rainy day. p 21 is the probability that a rainy day is followed by a sunny day. p 22 is the probability that a rainy day is followed by another rainy day. [ ] p11 p P = 12 p 21 p 22 x(t) = [x 1 (t), x 2 (t)], where x 1 (t) is the probability that day t will be a sunny day. x 2 (t) is the probability that day t will be a rainy day.
An example of P for weather.com light [ ] p11 p Let P = 12 = p 21 p 22 [ 0.6 ] 0.4 0.3 0.7 A sunny day is followed by another sunny day with probability 0.6. A sunny day is followed by a rainy day with probability 0.4. A rainy day is followed by a sunny day with probability 0.3. A rainy day is followed by another rainy day with probability 0.7. P is a stochastic matrix, which means that each row adds up to 1. This will be true for every transition probability matrix of a Markov chain, as each state i must be followed by some state in the next time step.
One-state transitions for our example of P [ ] p11 p Let P = 12 = p 21 p 22 [ 0.6 ] 0.4 0.3 0.7 Consider the following probability distributions for day t: x(t) = [1, 0] means that day t is sunny for sure. y(t) = [0.5, 0.5] means equal likelihood of a sunny or a rainy day. Note that the probabilities of all states always add up to 1. The corresponding probabilities for the next day are: x(t + 1) = [1, 0] P = [1, 0] y(t + 1) = [0.5, 0.5] P = [0.5, 0.5] [ ] 0.6 0.4 = [0.6, 0.4] 0.3 0.7 [ ] 0.6 0.4 = [0.45, 0.55] 0.3 0.7
The eigenvalues and an eigenvector for P of our example Let P = [ 0.6 ] 0.4 0.3 0.7 P λi = [ 0.6 λ 0.4 ] 0.3 0.7 λ det(p λi) = λ 2 1.3λ + 0.3 = (1 λ)(0.3 λ). The eigenvalues are λ 1 = 1 and λ 2 = 0.3. Let s find an eigenvector with eigenvalue 1: Form P 1I = [ ] 0.6 1 0.4 = 0.3 0.7 1 [ ] [ ] 0.4 0.4 x1 Solve = 0.3 0.3 x 2 [ ] 0 0 [ 0.4 ] 0.4 0.3 0.3 0.4x 1 + 0.4x 2 = 0 0.3x 1 0.3x 2 = 0 By setting x 1 = 1, we see that x = [1, 1] T is an eigenvector with eigenvalue 1 of P.
The meaning of the eigenvector [1, 1] T Eigenvectors with eigenvalues λ 1 are less important for transition matrices of Markov chains, so we will skip finding a eigenvector with eigenvalue λ 2 = 0.3 in our example. But we will take a closer look at the eigenvector [1, 1] T with eigenvalue λ 1 = 1. Let A = [a ij ] n n be any square matrix. Then [1, 1,..., 1] T is an eigenvector of A with eigenvalue λ if, and only if, a 11 a 12... a 1n 1 a 11 + a 12 + + a 1n λ a 21 a 22... a 2n 1.... = a 21 + a 22 + + a 2n. = λ. a n1 a n2... a nn 1 a n1 + a n2 + + a nn λ Thus [1, 1,..., 1] T is an eigenvector of A with eigenvalue λ if, and only if, each row of A adds up to λ. In particular, [1, 1,..., 1] T is an eigenvector of A with eigenvalue 1 if, and only if, A is a stochastic matrix.
We have proved a theorem... The observation on the previous slide proves parts (a) and (b) of the following result: Theorem Let P = [p ij ] n n be the matrix of transition probabilities for a Markov chain. Then (a) λ = 1 is an eigenvalue of P. (b) [1, 1,..., 1] T is an eigenvector of P with eigenvalue 1. (c) Every eigenvalue λ of P satisfies λ λ = 1. Part (c) is a consequence of a more general theorem called the Perron-Frobenius Theorem that goes beyond the scope of this course. This part says that λ = 1 is a so-called leading eigenvalue of P.
How about the eigenvectors of P T? Since every square matrix has the same eigenvalues as its transpose, λ = 1 must also be an eigenvalue of P T. Let s find a corresponding eigenvector for our example of P: Form P T 1I = [ 0.6 0.3 0.4 0.7 [ ] [ ] 0.4 0.3 x1 Solve = 0.4 0.3 x 2 ] [ ] 0 0 [ 1 0 0 1 ] = [ 0.4 0.3 0.4 0.3 0.4x 1 + 0.3x 2 = 0 0.4x 1 0.3x 2 = 0 We find that every vector of the form x = [ x 1, 4 3 x 1] T is an eigenvector with eigenvalue 1 of P T. These are the only eigenvectors with eigenvalue 1 of P T. Here it will be useful to find the eigenvector x = [x 1, x 2 ] T with x 1 + x 2 = 1 = x 1 + 4 3 x 1 = 7 3 x 1. It is x = [ 3 7, 4 7] T [0.4286, 0.5714] T. ]
The meaning of the eigenvector [0.4286, 0.5714] T of P T By the result of Homework 91, the vector ([0.4286, 0.5714] T ) T = [0.4286, 0.5714] is a left eigenvector of P. Moreover, since the coordinates add up to 1 and are nonnegative, [0.4286, 0.5714] is a probability distribution. It follows that if the probability distribution of the weather in our example on day t is x(t) = [0.4286, 0.5714], then the probability distribution of the weather on day t + 1 is x(t + 1) = [0.4286, 0.5714]P = [0.4286, 0.5714]. x = [0.4286, 0.5714] is a stationary (probability) distribution, which means that it remains the same on the next and all future days. In fact, x = [0.4286, 0.5714] is the only stationary distribution in this example.
These observations generalize Theorem Let P be the transition probability matrix of a Markov chain with n states and let x = [x 1, x 2,... x n] be a probability distribution. (a) x is a stationary distribution for this Markov chain if, and only if, x is a left eigenvector with eigenvalue 1 of P. (b) There exists at least one stationary distribution x of the Markov chain. (c) If x is the only stationary distribution of the Markov chain, then for any given initial distribution x(0), the distributions x(t) always approach x as t. Point (b) follows from point (a) and the previous theorem. Note also that in point (c) it is necessary that x is unique, because when we start in one stationary distribution y then we cannot approach another stationary distribution x.
Some alternative versions of weather.com light Let us consider some other transition probability matrices P for weather.com light Markov chains. Homework 93: (a) Let P 1 = I 2 (the weather always stays the same). Show that in this case every probability distribution x = [x 1, x 2 ] is a stationary distribution. [ ] 0 1 (b) Let P 2 = 1 0 Show that in this case x = [0.5, 0.5] is the unique stationary distribution. (c) Find a third transition probability matrix P 3 with stationary distribution x = [0.5, 0.5]. (d) Formulate a condition on P that appears to guarantee that x = [0.5, 0.5] is a stationary distribution and prove that it does.
Remember Waldo? Waldo is a highly gregarious and motivated and spends all of his evenings working with six students on his MATH 3200 homework. At 7p.m. he visits a randomly chosen student i among those six, and then operates as follows: He starts working with i. After 10 minutes, he flips a fair coin. If the coin comes up heads, he continues working with i for another 10 minutes before flipping the coin again. If the coin comes up tails, he moves to the room of a randomly chosen friend of i and repeats the procedure. He never tires of these efforts until 1a.m. Where should we go looking for Waldo at midnight?
The stationary distribution for Waldo Waldo s itinerary can be modeled as a Markov chain with states i = 1, 2,..., 6, where one time step lasts 10 minutes. State i simply means that Waldo is in i s room. The transition probability matrix for this Markov chain is 1/2 0 0 1/4 0 1/4 0 1/2 0 1/4 0 1/4 P = 0 0 1/2 1/4 1/4 0 1/8 1/8 1/8 1/2 0 1/8 = [p ij ] 6 6 0 0 1/2 0 1/2 0 1/6 1/6 0 1/6 0 1/2 Homework 94: Show that this Markov chain has a unique stationary probability distribution and find it.
Eigenvectors with eigenvalue 0 and the nullspace of A Let A be a square matrix. Let N(A) denote the set of all eigenvectors of A with eigenvalue 0 together with the zero vector 0. It is the nullspace of A. (A nullspace can be defined for any matrix A, but only for square matrices in terms of eigenvectors.) Proposition (a) N(A) has a nonzero element x 0 if, and only if, A is singular. (b) N(A) is the set of all nonzero solutions x of the homogeneous system A x = 0. (c) If a 1, a 2,..., a n denote the column vectors of A, then N(A) is the set of all vectors [x 1, x 2,..., x n ] T of coefficients such that x 1 a 1 + x 2 a 2 + + x n a n = 0. (d) N(A) is the set of all vectors x such that T A ( x) = 0. N(A) is also called the kernel of T A.
Review: Chemical reaction networks; net change of concentrations Consider a chemical reaction network like: A + 2B 2C A + B D A + 2C 2D B + D 2C If initial concentrations are denoted by [A] 0, [B] 0, [C] 0, [D] 0 and concentrations are measured again after some time and denoted by [A] 1, [B] 1, [C] 1, [D] 1, then the vector w = [[A] 1 [A] 0, [B] 1 [B] 0, [C] 1 [C] 0, [D] 1 [D] 0 ] represents the net change in concentrations. If some coordinate [X ] 1 [X ] 0 is positive, then a net production of compound X was observed, if some coordinate [X ] 1 [X ] 0 is negative, then a net consumption of compound X was observed.
Review: Chemical reaction networks; reaction vectors and stoichiometric matrix The reaction vectors of the chemical reaction network 1 A + 2B 2C 2 A + 2C 2D 3 A + B D 4 B + D 2C 1 1 1 0 v 1 = 2 2 v 2 = 0 2 3 = 1 0 v 4 = 1 2 0 2 1 1 represent the net changes in concentrations if only one reaction occurs and consumes one mole of its first reactant. They can be written as the columns of the stoichiometric matrix S.
Review: The linear transformation T S If we let k = [k 1, k 2, k 3, k 4 ] T be the column vector of average net rates at which the reactions occur over a given time interval, then the matrix product S k = w gives us the net change in concentrations. Positive values k i > 0 signify that the forward reaction dominates; negative values k i < 0 signify that the backward reaction dominates. When k = 0 over arbitrarily short time intervals, then each reaction is at equilibrium. When S k = 0 over arbitrarily short time intervals, then no observable change occurs and the system is at equilibrium. The nullspace N(S) is the set of all rate vectors k where the system is at equilibrium.
The rank of the stoichiometric matrix S Recall the result of Group Work 6 : Proposition Suppose S represents a stoichiometric matrix of order m n for n reactions between m chemical species in a closed reaction system (without net inflow, net outflow, or contributions from or to other reactions). Then r(s) < m. It follows that if m = n, then S is singular, so that it has at least one eigenvector with eigenvalue 0. Each such eigenvector represents a vector of reaction rates where the system is at equilibrium, but at least one reaction is not at equilibrium.
Homework problems Homework 95: Let S be a stoichiometric matrix of order n n, and let k be an eigenvector with eigenvalue 0 for S. Show that k must have at least 2 nonzero coordinates. Homework 96: Let S be a stoichiometric matrix for the chemical reaction network A + 2B 2C A + B D A + 2C 2D B + D 2C Find the set of all eigenvectors of S with eigenvalue 0.