Stationary distribution, CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET PDF Download

Henry Maltby, Samir Khan, and Jimin Khim contributed

A stationary distribution of a Markov chain is a probability distribution that remains unchanged in the Markov chain as time progresses. Typically, it is represented as a row vector π  whose entries are probabilities summing to 1, and given transition matrix P, it satisfies

= π + πP.

In other words,  π is invariant by the matrix P

Ergodic Markov chains have a unique stationary distribution, and absorbing Markov chains have stationary distributions with nonzero elements only in absorbing states. The stationary distribution gives information about the stability of a random process and, in certain cases, describes the limiting behavior of the Markov chain.

A reasonable way to approach this problem is to suppose there are x million Michigan fans and y  million Michigan State fans. The state's population is 10 million, so x + y = 10. These numbers do not change each year. It follows that

Rearranging either equation,    So there are 6.25 million Michigan fans and 3.75 million Michigan state fans. In other words, the stationary distribution is (0.625, 0.375). 

Note that the limiting distribution does not depend on the number of fans in the state!

Finding Stationary Distributions

Students of linear algebra may note that the equation πP = π looks very similar to the column vector equation  Mν = λν for eigenvalues and eigenvectors, with λ = 1. In fact, by transposing the matrices,

In other words, the transposed transition matrix P^T has eigenvectors with eigenvalue 1 that are stationary distributions expressed as column vectors. Therefore, if the eigenvectors of P^T are known, then so are the stationary distributions of the Markov chain with transition matrix P. In short, the stationary distribution is a left eigenvector (as opposed to the usual right eigenvectors) of the transition matrix.

When there are multiple eigenvectors associated to an eigenvalue of 1, each such eigenvector gives rise to an associated stationary distribution. However, this can only occur when the Markov chain is reducible, i.e. has multiple communicating classes.

 The transition matrix is

The transpose of this matrix has eigenvalues satisfying the equation

It follows that  So the eigenvalues are  λ = 0, λ = 0.5, and λ = 1. The eigenvalue λ = 0 gives rise to the eigenvector (1, -2, 1) , the eigenvalue λ = 0.5 gives rise to the eigenvector (-1, 0, 1), and the eigenvalue λ = 1  gives rise to the eigenvector (1, 2, 1) . The only possible candidate for a stationary distribution is the final eigenvector, as all others include negative values.

Then, the stationary distribution must be 

Relation to Limiting Distribution

The limiting distribution of a Markov chain seeks to describe how the process behaves a long time after . For it to exist, the following limit must exist for any states i and j:

Furthermore, for any state i, the following sum must be 

This ensures that the numbers obtained do, in fact, constitute a probability distribution. Provided these two conditions are met, then the limiting distribution of a Markov chain with X_o = i is the probability distribution given by 

For any time-homogeneous Markov chain that is aperiodic and irreducible,

converges to a matrix with all rows identical and equal to π. Not all stationary distributions arise this way, however. Some stationary distributions (for instance, certain periodic ones) only satisfy the weaker condition that the average number of times the process is in state i in the first n steps approaches the corresponding value of the stationary distribution. That is, if (x₁, x₂, ......,x_m) is the stationary distribution, then