• 损失函数


    Loss Function

    Loss function is used to measure the degree of fit. So for machine learning a few elements are:

    1. Hypothesis space: e.g. parametric form of the function such as linear regression, logistic regression, svm, etc.
    2. Measure of fit: loss function, likelihood
    3. Tradeoff between bias vs. variance: regularization. Or bayesian estimator (MAP)
    4. Find a good h in hypothesis space: optimization. convex - global. non-convex - multiple starts
    5. Verification of h: predict on test data. cross validation.

    Among all linear methods y=f(θTx)y=f(θTx), we need to first determine the form of ff, and then finding θθ by formulating it to maximizing likelihood or minimizing loss. This is straightforward.

    For classification, it's easy to see that if we classify correctly we have yf=yθTx>0y⋅f=y⋅θTx>0, and yf=yθTx<0y⋅f=y⋅θTx<0 if incorrectly. Then we formulate following loss functions:

    1. 0/1 loss: minθiL0/1(θTx)minθ∑iL0/1(θTx). We define L0/1(θTx)=1L0/1(θTx)=1 if yf<0y⋅f<0, and =0=0 o.w. Non convex and very hard to optimize.
    2. Hinge loss: approximate 0/1 loss by minθiH(θTx)minθ∑iH(θTx). We define H(θTx)=max(0,1yf)H(θTx)=max(0,1−y⋅f). Apparently HH is small if we classify correctly.
    3. Logistic loss: minθilog(1+exp(yθTx))minθ∑ilog(1+exp⁡(−y⋅θTx)). Refer to my logistic regression notes for details.

    For regression:

    1. Square loss: minθi||y(i)θTx(i)||2minθ∑i||y(i)−θTx(i)||2

    Fortunately, hinge loss, logistic loss and square loss are all convex functions. Convexity ensures global minimum and it's computationally appleaing.

        https://www.kaggle.com/wiki/LogarithmicLoss
  • 相关阅读:
    Windows 7 远程协助
    Windows 7 帮助和支持资源—第三方网站
    Windows 7 帮助和支持资源—第三方软件
    数据结构-队列
    数据结构-栈
    pycharm每日技巧-2
    数据结构-链表
    时间处理-1
    二维数组的排序
    《疯狂的简洁》书摘
  • 原文地址:https://www.cnblogs.com/zhizhan/p/5901613.html
Copyright © 2020-2023  润新知