Two State Markov Chains

Markov chains use matrices to calculate future probabilities and can model complex systems.  In maths methods we will only ever study two state markov chains.

Markov Chains

Markov chains are used to observe a system at a discrete moment in time, n, denoted X_n. If the state of the system in the future always depends on the state in the present then it is called a Markov chain.

For a two state markov chain it is assumed:

• Each step will result in one of two outcomes, for example it can either rain or not rain
• The probability of each outcome depends only on the previous step, how your footy team plays this week will decide how they play the next week
• The conditional probability is the same at each point in time, basically the probability one outcome will occur based on the previous outcome is constant

Transition Matrix

For an example lets pretend that you can either be passing or failing. We will call failing state 0 and passing state 1.

Let the probability that if you are now passing, by the end of the week you will be failing be  zero point zero five.

Therefore our other two possibilities will be the probability that you are failing in the present and the future and the probability that you are passing in the present and the future.

These can all be combined to form our transition matrix. The transition matrix is array of conditional possibilities whose size corresponds to its number of states. For our two states it is a two by two matrix.

State vector

The other matrix which is important when studying Markov chains is the initial state vector, S₀. This contain information about the markov chain at the initial point in time. The initial state vector that we deal with is a two by one matrix.

Now that we have the initial state and conditional possibilities we can multiply the two together to get future probabilities.

Practice Exam Question

Part 2 of Question 3b in Exam 2 of 2013

The most straight forward way to answer this question is to use markov chains. We know we can use two state markov chains as each question only has two options and they are conditional on the previous option.

Let’s set our state zero to be a correct answer and state one to be incorrect.

Then we need to define our transition matrix:

Now in order to find out the probability she answers the 25^th question in a row wrong we need to multiply the transition matrix by the state matrix 24 times.

However this is very time consuming by hand so you will have to rely on your matrix operator on your calculator to help you get the answer.

As we want the state zero correct answer, our final answer it 0.5714.

It is really important to define both the state one and state zero so you and the marker know what you are doing. Also if the question doesn’t ask for a series of markov chains to be done by hand, use your calculator to help you out.