Math 166: Topics in Contemporary Mathematics II

Similar documents
Math 166: Topics in Contemporary Mathematics II

QUEUING MODELS AND MARKOV PROCESSES

Math "Matrix Approach to Solving Systems" Bibiana Lopez. November Crafton Hills College. (CHC) 6.3 November / 25

MATH 310, REVIEW SHEET 2

Special Mathematics. Tutorial 13. Markov chains

ECE 541 Project Report: Modeling the Game of RISK Using Markov Chains

Chapter 16 focused on decision making in the face of uncertainty about one future

Chapter 10 Markov Chains and Transition Matrices

Matrix Basic Concepts

Markov Chains and Transition Probabilities

Chapter 29 out of 37 from Discrete Mathematics for Neophytes: Number Theory, Probability, Algorithms, and Other Stuff by J. M.

IEOR 6711: Professor Whitt. Introduction to Markov Chains

Designing Information Devices and Systems I Spring 2019 Lecture Notes Note 6

The Boundary Problem: Markov Chain Solution

TMA 4265 Stochastic Processes Semester project, fall 2014 Student number and

Equilibrium. Why? Model 1 A Reversible Reaction. At what point is a reversible reaction completed?

MATH Mathematics for Agriculture II

Math 2J Lecture 16-11/02/12

Chapter 35 out of 37 from Discrete Mathematics for Neophytes: Number Theory, Probability, Algorithms, and Other Stuff by J. M. Cargal.

(b) What is the variance of the time until the second customer arrives, starting empty, assuming that we measure time in minutes?

each nonabsorbing state to each absorbing state.

Math x + 3y 5z = 14 3x 2y + 3z = 17 4x + 3y 2z = 1

Markov Processes Hamid R. Rabiee

8-15. Stop by or call (630)

Solving Linear Systems

n α 1 α 2... α m 1 α m σ , A =

30.5. Iterative Methods for Systems of Equations. Introduction. Prerequisites. Learning Outcomes

Next topics: Solving systems of linear equations

Chill Out: How Hot Objects Cool

The probability of going from one state to another state on the next trial depends only on the present experiment and not on past history.

Markov Chains and Pandemics

Math 138: Introduction to solving systems of equations with matrices. The Concept of Balance for Systems of Equations

Lecture 20 : Markov Chains

Review of matrices. Let m, n IN. A rectangle of numbers written like A =

Markov Chains, Stochastic Processes, and Matrix Decompositions

Designing Information Devices and Systems I Fall 2018 Lecture Notes Note 6

30.3. LU Decomposition. Introduction. Prerequisites. Learning Outcomes

18.06 Problem Set 1 Solutions Due Thursday, 11 February 2010 at 4 pm in Total: 100 points

Eigenvalues and eigenvectors

Pre-Calculus I. For example, the system. x y 2 z. may be represented by the augmented matrix

10.2 ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS. The Jacobi Method

A Sudoku Submatrix Study

Math 22AL Lab #4. 1 Objectives. 2 Header. 0.1 Notes

GAUSSIAN ELIMINATION AND LU DECOMPOSITION (SUPPLEMENT FOR MA511)

Linear Algebra Section 2.6 : LU Decomposition Section 2.7 : Permutations and transposes Wednesday, February 13th Math 301 Week #4

spring, math 204 (mitchell) list of theorems 1 Linear Systems Linear Transformations Matrix Algebra

Lectures on Probability and Statistical Models

Name: Exam 2 Solutions. March 13, 2017

LINEAR SYSTEMS, MATRICES, AND VECTORS

Solving Linear Systems

MATH 310, REVIEW SHEET

Algebra & Trig. I. For example, the system. x y 2 z. may be represented by the augmented matrix

Linear Algebra Tutorial for Math3315/CSE3365 Daniel R. Reynolds

22A-2 SUMMER 2014 LECTURE Agenda

The Leslie Matrix. The Leslie Matrix (/2)

MATH 56A: STOCHASTIC PROCESSES CHAPTER 1

1. (3pts) State three of the properties of matrix multiplication.

3.2 Iterative Solution Methods for Solving Linear

4.1 Markov Processes and Markov Chains

Invariants II. LA Math Circle (Advanced) November 15, 2015

RISKy Business: An In-Depth Look at the Game RISK

1. Algebraic and geometric treatments Consider an LP problem in the standard form. x 0. Solutions to the system of linear equations

( 3, 4) means x = 3 and y = 4

Elementary Linear Algebra Review for Exam 3 Exam is Friday, December 11th from 1:15-3:15

Executive Assessment. Executive Assessment Math Review. Section 1.0, Arithmetic, includes the following topics:

Any live cell with less than 2 live neighbours dies. Any live cell with 2 or 3 live neighbours lives on to the next step.

Lecture 10: Markov Chains

CHAPTER 2 Matrices. Section 2.1 Operations with Matrices Section 2.2 Properties of Matrix Operations... 36

Elementary Linear Algebra Review for Exam 2 Exam is Monday, November 16th.

eigenvalues, markov matrices, and the power method

[Disclaimer: This is not a complete list of everything you need to know, just some of the topics that gave people difficulty.]

Math 2331 Linear Algebra

Evaluating Determinants by Row Reduction

Lecture 4: Applications of Orthogonality: QR Decompositions

Math 344 Lecture # Linear Systems

7.6 The Inverse of a Square Matrix

LU Factorization. Marco Chiarandini. DM559 Linear and Integer Programming. Department of Mathematics & Computer Science University of Southern Denmark

. Get closed expressions for the following subsequences and decide if they converge. (1) a n+1 = (2) a 2n = (3) a 2n+1 = (4) a n 2 = (5) b n+1 =

MTH 2032 Semester II

TMA Calculus 3. Lecture 21, April 3. Toke Meier Carlsen Norwegian University of Science and Technology Spring 2013

Section 1.1: Systems of Linear Equations

Math 301 Test III. Dr. Holmes. November 4, 2004

MATH 56A: STOCHASTIC PROCESSES CHAPTER 7

MATH 19B FINAL EXAM PROBABILITY REVIEW PROBLEMS SPRING, 2010

Math 671: Tensor Train decomposition methods II

System of Linear Equation: with more than Two Equations and more than Two Unknowns

Concepts. Materials. Objective

5.1 Introduction to Matrices

Leslie matrices and Markov chains.

Linear Equations in Linear Algebra

Finite-Horizon Statistics for Markov chains

Chapter 1. Vectors, Matrices, and Linear Spaces

Week 2. Section Texas A& M University. Department of Mathematics Texas A& M University, College Station 22 January-24 January 2019

Linear Equations in Linear Algebra

Solving Ax = b w/ different b s: LU-Factorization

Outlines. Discrete Time Markov Chain (DTMC) Continuous Time Markov Chain (CTMC)

Direct Methods for Solving Linear Systems. Matrix Factorization

Markov Chains Absorption (cont d) Hamid R. Rabiee

Roberto s Notes on Linear Algebra Chapter 9: Orthogonality Section 2. Orthogonal matrices

Introduction to Measurements. Introduction to Measurements. Introduction to Measurements. Introduction to Measurements. Introduction to Measurements

Transcription:

Math 166: Topics in Contemporary Mathematics II Xin Ma Texas A&M University November 26, 2017 Xin Ma (TAMU) Math 166 November 26, 2017 1 / 14

A Review A Markov process is a finite sequence of experiments in which 1. Each experiment has the same possible outcomes. 2. The probabilities of the outcomes depend only on the preceding experiment. We can apply matrix multiplication to find the probability distribution X n at the n-th stage T n X 0 = X n where T is the transition matrix, X 0 is the initial distribution. We say that a matrix T is a transition (or stochastic) matrix if 1. the matrix T is square 2. all elements in the matrix T are between 0 and 1 3. the sum of all the elements in any column is 1. Xin Ma (TAMU) Math 166 November 26, 2017 2 / 14

A Review Sometimes, X n X L as n becomes large. We call X L the steady-state distribution or the limiting distribution. If a Markov process has a steady-state distribution, we call this Markov process is regular. A Markov process is regular if, and only if, the associated transition (stochastic) matrix is regular. Note: If a Markov process has a steady-state distribution, it will ALWAYS approach this column matrix X L, no matter what the initial distribution is. A transition (or stochastic) matrix T is regular if some power of T has all positive entries, in other words, all the entries are strictly greater than 0. If there is a matrix L such that T n L as n becomes large and unbounded, we call L the limiting matrix. If T is regular, the limits L exists. If T is absorbing (will be talked soon), the limits L also exists. Xin Ma (TAMU) Math 166 November 26, 2017 3 / 14

Definition: In a Markov Processes, a state is called absorbing if it has the property that when entered it is impossible to leave. For example, consider a 3-state Markov process with the transition matrix T. If the second state is absorbing, then for any natural number n we always have 0 0 T n 1 = 1 0 0 Definition: A Markov process is called absorbing if 1. There is at least one absorbing state. 2. It is possible to move from any nonabsorbing state to one of the absorbing states in a finite number states. The transition matrix for an absorbing Markov process is said to be an absorbing stochastic matrix. Xin Ma (TAMU) Math 166 November 26, 2017 4 / 14

Example: A mouse house has 3 rooms: Room A, B, and C. Room A has cheese in it. Every minute, the mouse makes a choice. If the mouse is in Room B, the probability is 0.6 that it will move to Room A and 0.4 that it will move to Room C. If the mouse is in Room C, the probability that it will stay there is 0.1, the probability that it will move to Room A is 0.7, and the probability that it will move to Room B is 0.2. Once the mouse enters Room A, it never leaves. a. Write the transition matrix T for this Markov process. Is it an absorbing Markov process? Xin Ma (TAMU) Math 166 November 26, 2017 5 / 14

Example: A mouse house has 3 rooms: Room A, B, and C. Room A has cheese in it. Every minute, the mouse makes a choice. If the mouse is in Room B, the probability is 0.6 that it will move to Room A and 0.4 that it will move to Room C. If the mouse is in Room C, the probability that it will stay there is 0.1, the probability that it will move to Room A is 0.7, and the probability that it will move to Room B is 0.2. Once the mouse enters Room A, it never leaves. a. Write the transition matrix T for this Markov process. Is it an absorbing Markov process? A B C A 1 0.6 0.7 T = B 0 0 0.2 ; Yes, Because A is absorbing and C 0 0.4 0.1 P(B A) = 0.6 > 0 and P(C A) = 0.7 > 0 Xin Ma (TAMU) Math 166 November 26, 2017 5 / 14

Remark: If a Markov process is absorbing, then there is at least a ONE on the main diagonal of the associated transition matrix. This is a necessary condition but not sufficient. You also need to verify condition 2 in the definition. b. If there are a bunch of mice in the house, with 10% in Room A, 60% in Room B, and 30% in Room C, what is the room distribution after 5 minutes? Xin Ma (TAMU) Math 166 November 26, 2017 6 / 14

Remark: If a Markov process is absorbing, then there is at least a ONE on the main diagonal of the associated transition matrix. This is a necessary condition but not sufficient. You also need to verify condition 2 in the definition. b. If there are a bunch of mice in the house, with 10% in Room A, 60% in Room B, and 30% in Room C, what is the room distribution after 5 minutes? 0.1 0.995839 X 0 = 0.6. Then X 5 = T 5 X 0 = 0.00135 0.3 0.002811 Xin Ma (TAMU) Math 166 November 26, 2017 6 / 14

c. If the mouse is initially in Room C, what are the probabilities of being in each room after 5 minutes? Xin Ma (TAMU) Math 166 November 26, 2017 7 / 14

c. If the mouse is initially in Room C, what are the probabilities of being in each room after 5 minutes? 0 0.99597 Z 0 = 0. Then Z 5 = T 5 Z 0 = 0.00178 1 0.00225 d. Find T 5 and guess what the limiting matrix L should be. Xin Ma (TAMU) Math 166 November 26, 2017 7 / 14

c. If the mouse is initially in Room C, what are the probabilities of being in each room after 5 minutes? 0 0.99597 Z 0 = 0. Then Z 5 = T 5 Z 0 = 0.00178 1 0.00225 d. Find T 5 and guess what the limiting matrix L should be. 1 0.99508 0.99597 1 1 1 T 5 = 0 0.00136 0.00178 We may expect that L = 0 0 0. 0 0.00356 0.00225 0 0 0 e. Find the long-term behavior if the mouse starts at B and C. Xin Ma (TAMU) Math 166 November 26, 2017 7 / 14

c. If the mouse is initially in Room C, what are the probabilities of being in each room after 5 minutes? 0 0.99597 Z 0 = 0. Then Z 5 = T 5 Z 0 = 0.00178 1 0.00225 d. Find T 5 and guess what the limiting matrix L should be. 1 0.99508 0.99597 1 1 1 T 5 = 0 0.00136 0.00178 We may expect that L = 0 0 0. 0 0.00356 0.00225 0 0 0 e. Find the long-term behavior if the mouse starts at B and C. 0 0 1 Y 0 = 1 Z 0 = 0. The long-term behavior: LY 0 = 0 and 0 1 0 1 LZ 0 = 0 as well. 0 Xin Ma (TAMU) Math 166 November 26, 2017 7 / 14

Remark: If a absorbing Markov process has only one absorbing state, say the ith state, then the long term probability of ending in the ith state is one no matter what the initial distribution is. This is why we call it absorbing. And all the entries of ith row of the limiting matrix are 1 and the other entries are 0 as the example above shows. Now let us consider an example of a Markov process with more than one absorbing states. Example: A mouse house has 3 rooms: Room A, B, and C. Room A has cheese in it and Room B has a trap in it. Every minute, the mouse makes a choice. If the mouse is in Room C, the probability is 0.2 that it will stay there, 0.3 that it will move to Room A, and 0.5 that it will move to Room B. Once the mouse enters either Room A or Room B, it never leaves. Xin Ma (TAMU) Math 166 November 26, 2017 8 / 14

a. Write the transition matrix T for this Markov process. Is it an absorbing Markov process? Xin Ma (TAMU) Math 166 November 26, 2017 9 / 14

a. Write the transition matrix T for this Markov process. Is it an absorbing Markov process? A B C A 1 0 0.3 T = B 0 1 0.5 ; C 0 0 0.2 Yes, Because states A and B are absorbing and P(C B) = 0.5 > 0 and P(C A) = 0.3 > 0. b. If there are a bunch of mice in the house, with 10% in Room A, 60% in Room B, and 30% in Room C, what is the room distribution after 5 minutes? Xin Ma (TAMU) Math 166 November 26, 2017 9 / 14

a. Write the transition matrix T for this Markov process. Is it an absorbing Markov process? A B C A 1 0 0.3 T = B 0 1 0.5 ; C 0 0 0.2 Yes, Because states A and B are absorbing and P(C B) = 0.5 > 0 and P(C A) = 0.3 > 0. b. If there are a bunch of mice in the house, with 10% in Room A, 60% in Room B, and 30% in Room C, what is the room distribution after 5 minutes? 0.1 X 0 = 0.6. Then X 5 = T 5 X 0 = 0.3 0.212464 0.78744 9.6 10 5 Xin Ma (TAMU) Math 166 November 26, 2017 9 / 14

c. If the mouse is initially in Room C, what are the probabilities of being in each room after 5 minutes? Xin Ma (TAMU) Math 166 November 26, 2017 10 / 14

c. If the mouse is initially in Room C, what are the probabilities of being in each room after 5 minutes? 0 0.37488 X 0 = 0. Then X 5 = T 5 X 0 = 0.6248 1 3.2 10 4 d. Find T 5 and T 50 and guess what the limiting matrix L should be. Xin Ma (TAMU) Math 166 November 26, 2017 10 / 14

c. If the mouse is initially in Room C, what are the probabilities of being in each room after 5 minutes? 0 0.37488 X 0 = 0. Then X 5 = T 5 X 0 = 0.6248 1 3.2 10 4 d. Find T 5 and T 50 and guess what the limiting matrix L should be. By calculator: 1 0 0.37488 1 0 0.375 T 5 = 0 1 0.6248 and T 50 = 0 1 0.625 0 0 3.2 10 4 0 0 1.1259 10 35 1 0 0.375 Then we may expect L = 0 1 0.625. 0 0 0 Xin Ma (TAMU) Math 166 November 26, 2017 10 / 14

e. Find the long-term behavior if the mouse starts at C. Xin Ma (TAMU) Math 166 November 26, 2017 11 / 14

e. Find the long-term behavior if the mouse starts at C. 0 Z 0 = 0. The long-term behavior: 1 0.375 LZ 0 = 0.625. 0 Xin Ma (TAMU) Math 166 November 26, 2017 11 / 14

Remark: 1. If a Markov process has 2 or more absorbing states, then the long term probabilities of ending in the nonabsorbing states will be 0 and the long term probabilities of ending in the absorbing states will depend on the initial distribution. 2. We can deduce the limiting matrix L by computing T n for large n (usually n=50 is sufficiently large and then round your answers to four decimal places). Then find the ending distribution according to the initial distribution X 0, say LX 0. Note: No matter how many absorbing states a Markov process has, eventually everything will be absorbed by the absorbing states. Xin Ma (TAMU) Math 166 November 26, 2017 12 / 14

Example: Determine if the following stochastic transition matrices are absorbing. If absorbing, find the limiting matrix. 0.3 0 0.2 a. A = 0.4 1 0.7 0.3 0 0.1 Xin Ma (TAMU) Math 166 November 26, 2017 13 / 14

Example: Determine if the following stochastic transition matrices are absorbing. If absorbing, find the limiting matrix. 0.3 0 0.2.0 0 0 a. A = 0.4 1 0.7 (Yes). L = 1 1 1 since there is only one 0.3 0 0.1 0 0 0 absorbing state (also try A 50 to verify this). 0 1 0.3 b. B = 1 0 0.3 0 0 0.4 Xin Ma (TAMU) Math 166 November 26, 2017 13 / 14

Example: Determine if the following stochastic transition matrices are absorbing. If absorbing, find the limiting matrix. 0.3 0 0.2.0 0 0 a. A = 0.4 1 0.7 (Yes). L = 1 1 1 since there is only one 0.3 0 0.1 0 0 0 absorbing state (also try A 50 to verify this). 0 1 0.3 b. B = 1 0 0.3 (no) since there is no absorbing state (no entries in 0 0 0.4 diagonal is one). Xin Ma (TAMU) Math 166 November 26, 2017 13 / 14

c. 0.3 0 0.4 0.5 1 0 0.2 0 0.6 Xin Ma (TAMU) Math 166 November 26, 2017 14 / 14

c. d. 0.3 0 0.4 0.5 1 0 (Yes) second state is absorbing. and 1st 2nd and 0.2 0 0.6 3rd 1st 2nd. 1 0 0 0 0.2 0.6 0 0.8 0.4 Xin Ma (TAMU) Math 166 November 26, 2017 14 / 14

0.3 0 0.4 c. 0.5 1 0 (Yes) second state is absorbing. and 1st 2nd and 0.2 0 0.6 3rd 1st 2nd. 1 0 0 d. 0 0.2 0.6 (No). Though 1st state is absorbing. but the other 0 0.8 0.4 two states cannot go to 1st state. e. 1 0.5 0 0.4 0 0.2 0 0.5 0 0 1 0 0 0.3 0 0.1 Xin Ma (TAMU) Math 166 November 26, 2017 14 / 14

c. d. 0.3 0 0.4 0.5 1 0 (Yes) second state is absorbing. and 1st 2nd and 0.2 0 0.6 3rd 1st 2nd. 1 0 0 0 0.2 0.6 (No). Though 1st state is absorbing. but the other 0 0.8 0.4 two states cannot go to 1st state. 1 0.5 0 0.4 e. 0 0.2 0 0.5 0 0 1 0 (Yes.) The 1st and 3rd state are absorbing. No 0 0.3 0 0.1 other two states can go to the 3rd state. But both of them can go to the 1st state. Xin Ma (TAMU) Math 166 November 26, 2017 14 / 14

2024 © sciencedocbox.com Privacy Policy | Terms of Service | Feedback