2.9 Model Selection and the Bias–Variance Tradeoff

结论

• 模型复杂度↑Bias↓Variance↓

• 

例子

• $y_i=f(x_i)+epsilon_i,E(epsilon_i)=0,Var(epsilon_i)=sigma^2$
  使用knn做预测，在点$x_0$处的Excepted prediction error：
  $EPE(x_0)=Eleft[left(y_0-hat{f}(x_0) ight)^2|x_0 ight]\ =Eleft[left(y_0-E(y_0) ight)^2|x_0 ight]+left[E(hat{f}(x_0))-E(y_0)|x_0 ight]^2+Eleft[hat{f}(x_0)-E(hat{f}(x_0)) ight]^2\ =sigma^2+{Bias}^2(hat{f}(x_0))+Var(hat{f}(x_0))\ =sigma^2+left[f(x_0)-frac{1}{k}sum_l^k y_l ight]^2+frac{sigma^2}{k}$
  k↑在训练集上的表现(考虑k=1，k=2)↓模型复杂度(模型越复杂，在训练集上的表现越好)↓