# Markov chain

Main Article
Talk
Related Articles  [?]
Bibliography  [?]
Citable Version  [?]

This editable Main Article is under development and not meant to be cited; by editing it you can help to improve it towards a future approved, citable version. These unapproved articles are subject to a disclaimer.

A Markov chain is a Markov process with a discrete time parameter [1]. The Markov chain is a useful way to model systems with no long-term memory of previous states. That is, the state of the system at time  is solely a function of the state , and not of any previous states [2].

## A Formal Model

The influence of the values of  on the distribution of  can be formally modelled as:

  Eq. 1

In this model,  is any desired subset of the series . These  indexes commonly represent the time component, and the range of  is the Markov chain's state space [1].

## Probability Density

The Markov chain can also be specified using a series of probabilities. If the initial probability of the state  is , then the transition probability for state  occurring at time  can be expressed as:

  Eq. 2

In words, this states that the probability of the system entering state  at time  is a function of the summed products of the initial probability density and the probability of state  given state  [2].

## Invariant Distributions

In many cases, the density will approach a limit  that is uniquely determined by  (and not ). This limiting distribution is referred to as the invariant (or stationary) distribution over the states of the Markov chain. When such a distribution is reached, it persists forever[2].

## References

1. Neal, R.M. (1993) Probabilistic Inference using Markov Chain Monte Carlo Methods. Technical Report TR-931. Department of Computer Science, University of Toronto http://www.cs.toronto.edu/~radford/review.abstract.html
2. Peter M. Lee (2004) Bayesian Statistics: An Introduction. New York: Hodder Arnold. 368 p.