Math 166: Topics in Contemporary Mathematics II

Math 166: Topics in Contemporary Mathematics II Xin Ma Texas A&M University November 26, 2017 Xin Ma (TAMU) Math 166 November 26, 2017 1 / 10

Announcements 1. Homework 27 (M.1) due on this Wednesday and Homework 28&29(M.2 and M.3) due on next Wednesday. 2. There will be a (the last) quiz (on M.1 and M.2) on this Wednesday. 3. Markov Process is the last chapter in this semester. We will start to review this Friday. 4. The COMPREHENSIVE final exam is 10:30 a.m.-12:30 p.m. on Dec 11, Monday in this Classroom. Xin Ma (TAMU) Math 166 November 26, 2017 2 / 10

Steady-State Distribution Example: Recall the example of cars. We have the transition matrix T and initial matrix X 0 0.5 0.4 0.2 0.6 T = 0.2 0.5 0.1 X 0 = 0.3 0.3 0.1 0.7 0.1 Then we can find that 0.3501302203 X 10 = 0.2251302024 0.3500001272 X 20 = 0.2250001272 0.3500000001 X 30 = 0.2250000001 0.4247395774 0.4249997457 0.4249999998 As the number of stages increases, you can see that the distributions are approaching 0.35 X L = 0.225 0.425 Xin Ma (TAMU) Math 166 November 26, 2017 3 / 10

Steady-State Distribution We call X L the steady-state distribution or sometimes the limiting distribution. What this means is that in the long run, we would expect 35% will own a Ford, 22.5% will own a Honda, and 42.5% will own a Chevy. Note: If a Markov process has a steady-state distribution, it will ALWAYS approach this column matrix X L, no matter what the initial distribution is. This is because T n heading for the limiting matrix L and each column of L is the same as X L. If a Markov process has a steady-state distribution, we call this Markov process is regular. Xin Ma (TAMU) Math 166 November 26, 2017 4 / 10

Regular Stochastic Matrix Definition: A stochastic (or transition) matrix T is regular if some power of T has all positive entries, in other words, all the entries are strictly greater than 0. A Markov process is regular if, and only if, the associated stochastic matrix is regular. Example: Determine whether the following matrices are regular. [ ] 0.2 0.4 a. A = (Yes) 0.8 0.6 b. B = c. C = (No) [ 0.2 ] 1 0.8 0 [ ] 0 1 1 0 [ ] [ ] 0.2 1 0.2 1 B 2 = = 0.8 0 0.8 0 C 2 = [ ] 0.84 0.2. (Yes) 0.16 0.8 [ ] 1 0 = I 0 1 2. Then C 2n = I 2 and C 2n+1 = C. Xin Ma (TAMU) Math 166 November 26, 2017 5 / 10

If T is a regular stochastic matrix, one can find the steady-state distribution X L by solving the equations (system) TX L = X L together with the fact that the sum of the entries in X L must be one. Example: Find the steady-state distribution for the regular Markov chain whose transition matrix is [ 0.2 ] 1 0.8 0 [ ] [ ] [ ] [ ] x 0.2 1 x x Suppose X L = with x + y = 1. Then =. Then we y 0.8 0 y y [ ] [ ] 0.2x + y = x (a) 0.2x + y x have =. It turns out that we have 0.8x = y (b). 0.8x y x + y = 1 (c) Xin Ma (TAMU) Math 166 November 26, 2017 6 / 10

5 0.8 1 0 1 0 9 Then we have: 0.8 1 0 4 0 1 9. 1 1 1 0 0 0 ] Then solve the system above, we have X L = [ 5 9 4 9 The process above is the typical one to find steady-state distributions. NEED TO REMEMBER! Xin Ma (TAMU) Math 166 November 26, 2017 7 / 10

Example: Suppose that in a study of Ford, Honda, and Chevrolet, the following data was found: Every year, it is found that of the customers who own a Ford, 50% will stay with a Ford, 20% would switch to a Honda, and 30% would switch to a Chevy. Of the customers who own a Honda, 50% would stick with Honda, 40% would buy a Ford, and 10% would buy a Chevy. Of the customers who own a Chevy, 70% would stick with Chevy, 20% would switch to Ford, and 10% would buy a Honda. Transition matrix: F H C F 0.5 0.4 0.2 x T = H 0.2 0.5 0.1 ; Suppose X L = y. C 0.3 0.1 0.7 z Xin Ma (TAMU) Math 166 November 26, 2017 8 / 10

Example: Suppose that in a study of Ford, Honda, and Chevrolet, the following data was found: Every year, it is found that of the customers who own a Ford, 50% will stay with a Ford, 20% would switch to a Honda, and 30% would switch to a Chevy. Of the customers who own a Honda, 50% would stick with Honda, 40% would buy a Ford, and 10% would buy a Chevy. Of the customers who own a Chevy, 70% would stick with Chevy, 20% would switch to Ford, and 10% would buy a Honda. Transition matrix: F H C F 0.5 0.4 0.2 x T = H 0.2 0.5 0.1 ; Suppose X L = y. Then what we need to C 0.3 0.1 0.7 z solve is TX L = X L. Then same process above will give X L. An equivalent way to do that is TX L = X L = I 3 X L. Then it is equivalent to the linear system (T I 3 )X L = 0. Xin Ma (TAMU) Math 166 November 26, 2017 8 / 10

0.5 0.4 0.2 1 0 0 0.5 0.4 0.2 T I 3 = 0.2 0.5 0.1 0 1 0 = 0.2 0.5 0.1. 0.3 0.1 0.7 0 0 1 0.3 0.1 0.3 0.5 0.4 0.2 x 0 Then the linear system is 0.2 0.5 0.1 y = 0. Plus the 0.3 0.1 0.3 z 0 equation x + y + z = 1, the original augment matrix is 0.5 0.4 0.2 0 1 0 0 0.35 0.2 0.5 0.1 0 0.3 0.1 0.3 0 0 1 0 0.225 0 0 1 0.425. 1 1 1 1 0 0 0 0 0.35 Thus X L = 0.225. 0.425 Xin Ma (TAMU) Math 166 November 26, 2017 9 / 10

Example: It is known that a certain businessman always wears a white shirt (W) or a blue shirt (B) to work and 60% of the time when wearing a white shirt he changes color the next day, and 90% of the time when wearing a blue shirt he changes color the next day. What is the steady state distribution? ( W B ) [ ] W 0.4 0.9 x The transition matrix: T = ; Suppose X B 0.6 0.1 L =. y Then consider TX L = X L (or (T I 2 )X L = 0) and x + y = 1, we have: 0.4x + 0.9y = x 0.6x + 0.9y = 0 0.6x + 0.1y = y) or ( 0.6x 0.9y = 0) x + y = 1 x + y = 1 [ ] 0.6 Solve this, we have X L =. 0.4 Xin Ma (TAMU) Math 166 November 26, 2017 10 / 10