Steady state Markov chains is the idea that as the time period heads towards infinity then a two state Markov chain’ state vector will stabilise.
If we keep multiplying the initial state vector like we showed in the last video, the initial state input becomes less important and the conditional probability input more dominant in the final answer.
Transition matrix decides the likelihood over a large time period what the probability of the system is.
Luckily for us there is an easier way to find the steady state than putting the conditional probability to the power of infinity. Once we reach the steady state we will find that the next time period will be equal to the previous one that is:
Our standard Markov chain probability calculation is:
To get the elements of our transition matrix we will set the probabilities to:
Therefore if we substitute these values into the transition matrix, and the current state in for the the future state we get, a new matrix that has two equations and two unknowns therefore we can expand the matrix, and solve for the steady state probabilities:
Now we also know that since it is a two state system:
$
Now we can sub this into equation one:
Substitute this into equation 3:
A Multiple Choice Question from Exam 2 in 2013
As soon as we are asked about the long term probability then we know that we need to find the steady state of a markov chain.
Step 1: Set each option to a state
Step 2: Find the probabilities of a and b
Step 3: Use the equation for steady state.
So the multiple choice answer is C
It is critical to state which is your state one and state zero at the start of the question to limit any confusion in the question.
Want to suggest an edit? Have some questions? General comments? Let us know how we can make this resource more useful to you.