自适应信号处理----读书笔记

　　记得去上第一节课的时候，老师就问，你们知道什么是自适应吗？还举了个例子说，北方人来到了南方，

会觉得天气特别热，饮食也不太一样，这时他就开始去慢慢适应南方的环境，自适应就开始了。

　　The adaptive linear combiner, which is the simplest and most widely applicable adaptive processor.

　　Several ways to classify adaptive schemes have been proposed in the literature. It is most convenient

here to begin by thinking in terms of open-loop and closed-loop adaptation. The open-loop adaptive process

involves making measurements of input or environmental characteristics, applying this information to a formula

or to a computational algorithm, and using the results to set the adjustments of the adaptive system. Closed-loop

adaptation, on the other hand, involves automatic experimentation with these adjustments and knowledge of their

outcome in order to optimize a measures system performance. The latter process may be called adaptation by

"performance feedback".

 　　In the adaptation process with performance feedback. the weight vector of the linear combiner is adjusted to cause

the output, y_k, to agree as closely as possible with desired response signal. 后来看机器学习的书，发现这与linear regression

是一样的。通过比较系统的输出与目标值(desired response)之间的误差（ε_k = d_k - y_k），调整或者优化权重向量，使误差最小。在绝

大部分的实际应用中，自适应过程都是朝着最小化均方误差的方向进行，MSE(mean-square error)=E[ε_k²]=E[(d_k-X_k^TW_k)²].

　　均方误差展开即为：E[ε_k²]=E[d_k²]+W^TE[X_kX_k^T]W-2E[d_kX_k^T]W.

　　Many useful adaptive process that cause the weight vector to seek the minimum of the performance surface

do so by gradient methods.

　　权重向量的获取一般通过梯度方法得到，比如最速下降法：w_k+1 = w_k + μ*(-Δ_k)和牛顿法：w_k+1 = w_k - f(w_k)/f'(w_k)。The convergence

of Newton's method obviously depends on the initial guess, w₀, and on the nature of f(w), but it is known to converge rapidly for a

large class of functions.

　　The least-mean-square algorithm, or LMS algorithm, uses a special estimate of the gradient that is valid for the adaptive linear

combiner. On the other hand, the LMS algorithm is important because of its simplicity and ease of computation, and because it 

does not require off-line gradient estimations or repetitions of data. W_k+1 = W_k + 2με_kX_k. If the adaptive system is an adaptive

linear combiner, and if the input vector X_k and the desired response d_k are available at each iteration, the LMS algorithm is generally

the best choice for many different applications of adaptive signal processing.这个算法就是机器学习线性回归中常用的LMS算法。 

　　记得老师上课时候讲，不要等到基础都打好了，再去学习新知识或新的领域，基础是永远打不完的，等用到时候，发现缺哪些知识，再去补。