• SVM用于线性回归


    SVM用于线性回归

    方法分析

    在样本数据集()中,不是简单的离散值,而是连续值。如在线性回归中,预测房价。与线性回归类型,目标函数是正则平方误差函数:

    在SVM回归算法中,目的是训练出超平面,采用作为预测值。为了获得稀疏解,即计算超平面参数w,b不依靠所有样本数据,而是部分数据(如在SVM分类算法中,支持向量的定义),采用误差函数

    误差函数定义为,如果预测值与真实值的差值小于阈值将不对此样本做惩罚,若超出阈值,惩罚量为。

    下图为误差函数与平方误差函数的图形

    目标函数

    观察上述的误差函数的形式,可以看到,实际形成了一个类似管道的样子,在管道中样本点,不做惩罚,所以被称为,如下图阴影红色部分

    用替代平方误差项,因此可以定义最小化误差函数作为优化目标:

    由于上述目标函数含有绝对值项不可微。我们可以转化成一个约束优化问题,常用的方法是为每一个样本数据定义两个松弛变量,表示度量与的距离。

    如上图所示:

    当样本点真实值位于管道上方时,写成表达式:时,

    当样本点真实值位于管道下方时,写成表达式:时,

    因此使得每个样本点位于管道内部的条件为:

    当位于管道上方时,,有

    当位于管道下方时,,有

    误差函数可以写成为一个凸二次优化问题:

    约束条件为:

    写成拉格朗日函数:

    对偶问题

    上述问题为极小极大问题

    与SVM分类分析方法一样,改写成对偶问题

    首先求偏导数

    超平面计算

    Support Vector Machine - Regression (SVR)

    Support Vector Machine can also be used as a regression method, maintaining all the main features that characterize the algorithm (maximal margin). The Support Vector Regression (SVR) uses the same principles as the SVM for classification, with only a few minor differences. First of all, because output is a real number it becomes very difficult to predict the information at hand, which has infinite possibilities. In the case of regression, a margin of tolerance (epsilon) is set in approximation to the SVM which would have already requested from the problem. But besides this fact, there is also a more complicated reason, the algorithm is more complicated therefore to be taken in consideration. However, the main idea is always the same: to minimize error, individualizing the hyperplane which maximizes the margin, keeping in mind that part of the error is tolerated. 

    Linear SVR

    Non-linear SVR

    The kernel functions transform the data into a higher dimensional feature space to make it possible to perfom the linear separation

    Kernel functions

  • 相关阅读:
    ui优化
    《行尸走肉:行军作战》移动端优化经验
    Git master branch has no upstream branch的解决
    在线压图
    Git 分支管理-git stash 和git stash pop
    ScriptableObject
    Git解决pull无法操作成功
    Unity自带IAP插件使用(googleplay)
    Unity苹果(iOS)内购接入(Unity内置IAP)
    VMWare 装mac os x 一个必备优化神器 beamoff
  • 原文地址:https://www.cnblogs.com/kexinxin/p/9858496.html
Copyright © 2020-2023  润新知