Sampling

Monte Carlo：

通过极限情况下的分布关系$pi (x’) =sumlimits_{x}{ pi (x)P(x->x’)} $

有p(x’)$approxsumlimits_{x}{p(x)T(x—>x’)}$

若T满足regular markov chain的条件，则Monte Carlo方法保证在极限条件下收敛到目标分布。

Regular Markov Chain

转移矩阵经过若干次相乘后，所有项都不为0的马尔科夫链就是规则马尔科夫链。

   充分条件：任意两个状态都相连，每个状态自转移概率不为0.

An square matrix is called regular if for some integer all entries of are positive.

Example

The matrix

is not a regular matrix, because for all positive integer ,

The matrix

is a regular matrix, because has all positive entries.

It can also be shown that all other eigenvalues of A are less than 1, and algebraic multiplicity of 1 is one.

It can be shown that if is a regular matrix then approaches to a matrix whose columns are all equal to a probability vector which is called the steady-state vector of the regular Markov chain.

where .

It can be shown that for any probability vector when gets large, approaches to the steady-state vector

.

That is

where .

It can also be shown that the steady-state vector q is the only vector such that

Note that this shows q is an eigenvector of A and is eigenvalue of A.

Mixed：收敛的

验证方法，通常不能验证已经mixed，但是能验证还不是mixed：

1、使用windows，截取一个时间段的数据看是否相近。但是可能在收敛过程中有小部分数据先聚集到一起，这不能说明是收敛的。

2、使用两个不同的初始状态的马尔科夫链。在同一个时间观察，如果数据不相近，则不是mixed。

实际中可以使用一个随机初始的，和一个高概率初始的来比较。

MCMC方法取得的样本不是IID的，所以有时需要间隔一段再取。

The faster the Markov Chain converges, the less correlated are the samples.

Gibbs Sampling

对多维数据有效。

不能mix的gibbs sampling chain

metropolis-hastings