朴素贝叶斯
在经典的分类器模型中,Naive Bayes Classifier应该是比较简单的一种了,比之前的决策树要简单得多,但是它虽然简单,但是一点都不简约,在很多情况下它往往能得到比较好的分类效果。
通常的分类问题中,每一个实例都可以用一个特征向量来表示的,其相应的类别用
来表示,且
,
。其中
表示样本集合,
表示类标记集合。
朴素贝叶斯模型的基本思想就是,通过直接从样本中学习得到条件概率分布,对此,我们可以做如下的推导:
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/007C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0058.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0058.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0058.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0058.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/007C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/22C5.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0058.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0031.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
当我们需要判断一个未知实例的类别时,利用如下的优化公式:
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0079.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0061.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0072.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0067.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/006D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0061.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0078.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/168/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/2208.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0058.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0078.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/007C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/22C5.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0058.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0078.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0061.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0072.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0067.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/006D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0061.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0078.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/168/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/2208.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0058.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0078.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/007C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/22C5.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0032.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
上面的第二个等号之所以成立,是因为对于
来说是一个常数,因此在这里可以直接省略。
在上面的最后一个等式中,称为后验概率,而
称为先验概率,贝叶斯模型的学习过程就是从数据集上获得这两个概率分布的过程。
对于后验概率来说,我们可以将其展开:
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0058.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0078.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/007C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0078.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/0031.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0066.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/0031.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/2026.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0078.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006E.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0066.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006E.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/007C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0033.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
上式中的表示,实例
的第
个属性取值为
。
上式中的等号是严格成立的,但是不幸的是,如果根据上述的公式直接计算后验概率,那么空间复杂度是呈指数增长的,真实计算时是完全不可行的。假设可能的取值有
个,
可以取的值有
个,那么需要估计的参数总数将会是
.
为了解决计算复杂度的问题,朴素贝叶斯模型对条件概率分布作了条件独立性的假设,因为这是一个很强的假设,朴素贝叶斯也是因此而得名的。条件独立性假设是指:
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0058.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0078.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/007C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0078.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/0031.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0066.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/0031.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/2026.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0078.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006E.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0066.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006E.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/007C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Size2/Regular/336/220F.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006A.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/0031.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006E.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0078.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006A.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0066.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006A.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/007C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0034.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
上述公式中的第二个公式是朴素贝叶斯中最重要的核心部分,它是指每个属性在给定分类结果的条件下是相互独立的
经过条件独立性的假设的化简,原来的优化目标就可以写成:
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0079.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0061.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0072.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0067.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/006D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0061.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0078.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/168/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/2208.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Size2/Regular/336/220F.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006A.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0078.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006A.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0066.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006A.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/007C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0035.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
经过这样的化简之后,我们需要学习的参数个数就变成了,这和原来的
相比,已经大大化简了。
朴素贝叶斯法实际上学习到生成数据的机制,所以属于生成模型。条件独立假设等于是说用于分类的特征在雷确定的条件下都是条件独立的,这一假设使得朴素贝叶斯变得简单,但有时会牺牲一定的分类准确率.
参数估计
经过上面的分析,我们现在需要从数据中学习的分布有以下两个:
先验概率分布
后验概率分布
极大似然估计
利用极大似然估计是比较容易从数据中学习上述两个概率分布的。
先验概率的极大似然估计是:
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Size1/Regular/336/2211.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/0069.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/0031.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/004E.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0049.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0079.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/0069.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/004E.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0031.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0032.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/2026.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/004B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0036.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
其中表示训练实例的个数,
表示在训练集中标记为
的实例的个数.
设可能取值的集合为
,条件概率
的极大似然估计是:
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0078.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006A.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0066.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006A.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/007C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Size1/Regular/336/2211.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/0069.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/0031.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/004E.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0049.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0078.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006A.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/0069.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0066.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006A.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0079.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Size1/Regular/336/2211.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/0069.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/0031.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/004E.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0049.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0079.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/0069.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/006A.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0031.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0032.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/2026.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/006E.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/006C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0031.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0032.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/2026.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0053.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006A.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0031.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0032.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/2026.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/004B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0037.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
其中,是第
个样本的第
个特征,
是第
特征可能取的第
个值。
贝叶斯估计
但是,从上面的估计方法中,我们可以发现,其实极大似然估计还是有一些问题的,如果的情况在训练数据中完全没有出现的话,那么整个估计就将变为0,这显然是不合理的,因此我们要对极大似然估计做出一点修改,修改之后的估计方法称为贝叶斯估计,后验概率的贝叶斯估计是:
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/03BB.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0078.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006A.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0066.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006A.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/007C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Size1/Regular/336/2211.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/0069.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/0031.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/004E.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0049.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0078.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006A.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/0069.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0066.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006A.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002C.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0079.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/03BB.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Size1/Regular/336/2211.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/0069.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/0031.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/004E.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0049.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0079.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/0069.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0053.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006A.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/03BB.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0038.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
上式中的,当
时,就退化为极大似然估计;通常情况下
,这时称为拉普拉斯平滑
同样,先验概率的贝叶斯估计是:
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0050.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/03BB.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0059.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Size1/Regular/336/2211.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/0069.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/238/0031.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/004E.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0049.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0079.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/0069.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/003D.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/0063.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/238/006B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/03BB.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/004E.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/002B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/004B.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Math/Italic/336/03BB.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0028.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0039.png?rev=2.5.3)
![](http://cdn.mathjax.org/mathjax/latest/fonts/HTML-CSS/TeX/png/Main/Regular/336/0029.png?rev=2.5.3)
具体实现
朴素贝叶斯算法的Python简单实现如下:
#coding:utf-8
"""
Program: Naive Bayes Algorithm
Description:
Author: Flyaway - flyaway1217@gmail.com
Date: 2014-01-13 20:30:29
Last modified: 2014-01-13 21:58:27
Python release: 3.2.3
"""
from collections import Counter
class NaiveBayes:
def __init__(self,dataset,labels,lam = 1):
self.dataset = dataset
self.labels = labels
self.instance_num = len(dataset)
self.lam = lam #lambda
self.count = {}
self.prior = {}
def getPrior(self,cla):
'''
get the prior probability
'''
member = self.prior[cla] + self.lam
denominator = self.instance_num + len(self.prior) * self.lam
return float(member/denominator)
def train(self):
self.prior={}
m = Counter(self.labels).most_common()
for item in m:
self.prior[item[0]] = item[1]
for i,vector in enumerate(self.dataset):
cla = self.labels[i]
if cla not in self.count:
self.count[cla] = [{}] * len(vector)
for j,feat in enumerate(vector):
self.count[cla][j][feat] = self.count[cla][j].get(feat,0) + 1
def getPost(self,cla,index,feat):
'''
get the post probability
'''
member = self.count[cla