week3编程作业: Logistic Regression中一些难点的解读

%% ============ Part 2: Compute Cost and Gradient ============
%  In this part of the exercise, you will implement the cost and gradient
%  for logistic regression. You neeed to complete the code in 
%  costFunction.m

%  Setup the data matrix appropriately, and add ones for the intercept term
[m, n] = size(X);

% Add intercept term to x and X_test
X = [ones(m, 1) X];

% Initialize fitting parameters
initial_theta = zeros(n + 1, 1);

% Compute and display initial cost and gradient
[cost, grad] = costFunction(initial_theta, X, y);

难点1：X和theta的维度变化，怎么变得，为什么？

           

           X加了一列1，θ加了一行θ₀,因为最后边界是θ₀+θ₁X₁+θ₂X₂,要符合矩阵运算

难点2：costFunction中grad是什么函数，有什么作用？

                

w.r.t 什么意思？

with regard to
with reference to

关于的意思

难点3：linear regression的代价函数和logistic regression的代价函数，为什么不一样？

            和幂次有关，如果还用原来的代价函数，那么会有很多局部最值，没有办法梯度下降到最小值。

           

        另一个解读角度：从最大似然函数出发考虑。  

       下面的文章写得很好，参考链接：

　　　 http://blog.csdn.net/lu597203933/article/details/38468303  

            http://blog.csdn.net/it_vitamin/article/details/45625143

%% ============= Part 3: Optimizing using fminunc  =============
%  In this exercise, you will use a built-in function (fminunc) to find the
%  optimal parameters theta.

%  Set options for fminunc
options = optimset('GradObj', 'on', 'MaxIter', 400);

%  Run fminunc to obtain the optimal theta
%  This function will return theta and the cost 
[theta, cost] = ...
    fminunc(@(t)(costFunction(t, X, y)), initial_theta, options);

难点4：optimset是什么作用？

  options = optimset('GradObj', 'on', 'MaxIter', 400); //

通过optimset函数设置或改变这些参数。其中有的参数适用于所有的优化算法，有的则只适用于大型优化问题，另外一些则只适用于中型问题。

  LargeScale – 当设为'on'时使用大型算法，若设为'off'则使用中型问题的算法。
      适用于大型和中型算法的参数：
Diagnostics – 打印最小化函数的诊断信息。
Display – 显示水平。选择'off'，不显示输出；选择'iter'，显示每一步迭代过程的输出；选择'final'，显示最终结果。打印最小化函数的诊断信息。
GradObj – 用户定义的目标函数的梯度。对于大型问题此参数是必选的，对于中型问题则是可选项。
MaxFunEvals – 函数评价的最大次数。
MaxIter – 最大允许迭代次数。
TolFun – 函数值的终止容限。
TolX – x处的终止容限。

 

options = optimset('param1',value1,'param2',value2,...) %设置所有参数及其值，未设置的为默认值

options = optimset('GradObj', 'on', 'MaxIter', 400);

在使用options的函数中，参数是由用户自行定义的，最大递归次数为400

 难点5：[theta, cost] = fminunc(@(t)(costFunction(t, X, y)), initial_theta, options)是怎么实现功能？

关于句柄@，参考偏文章：http://blog.csdn.net/gzp444280620/article/details/49252491

关于fiminuc,参考这篇文章：http://blog.csdn.net/gzp444280620/article/details/49272977

fminunc(@(t)(costFunction(t, X, y)), initial_theta, options);    %是为了当代码里面有这样的函数的时候，fminunc(1,θ，option)第一个参数，

会传递给costFunction(t, X, y)的第一个参数里面进行计算。

function plotDecisionBoundary(theta, X, y)
%PLOTDECISIONBOUNDARY Plots the data points X and y into a new figure with
%the decision boundary defined by theta
%   PLOTDECISIONBOUNDARY(theta, X,y) plots the data points with + for the 
%   positive examples and o for the negative examples. X is assumed to be 
%   a either 
%   1) Mx3 matrix, where the first column is an all-ones column for the 
%      intercept.
%   2) MxN, N>3 matrix, where the first column is all-ones

% Plot Data
plotData(X(:,2:3), y);
hold on

if size(X, 2) <= 3
    % Only need 2 points to define a line, so choose two endpoints
    plot_x = [min(X(:,2))-2,  max(X(:,2))+2];

    % Calculate the decision boundary line//令theta装置乘以X等于0，即可
   
   plot_y = (-1./theta(3)).*(theta(2).*plot_x + theta(1));

    % Plot, and adjust axes for better viewing
    plot(plot_x, plot_y)
    
    % Legend, specific for the exercise
    legend('Admitted', 'Not admitted', 'Decision Boundary')
    axis([30, 100, 30, 100])
else
    % Here is the grid range
    u = linspace(-1, 1.5, 50);
    v = linspace(-1, 1.5, 50);

    z = zeros(length(u), length(v));
    % Evaluate z = theta*x over the grid
    for i = 1:length(u)
        for j = 1:length(v)
            z(i,j) = mapFeature(u(i), v(j))*theta;
        end
    end
    z = z'; % important to transpose z before calling contour

    % Plot z = 0
    % Notice you need to specify the range [0, 0]
    contour(u, v, z, [0, 0], 'LineWidth', 2)
end
hold off

end

难点6：这个plotDecisionBoundary函数是怎么画出边界的？

 plotDecisionBoundary中的下面的两行看不懂：

 plot_x = [min(X(:,2))-2,  max(X(:,2))+2];  //直线的参数其实已经得到，选划线的范围，把直线画出来即可。为了美观，扩大了两个单位

  % Calculate the decision boundary line//令theta装置乘以X等于0，即可 plot_y = (-1./theta(3)).*(theta(2).*plot_x + theta(1));

 plot_y = (-1./theta(3)).*(theta(2).*plot_x + theta(1));

function plotData(X, y)    //会按照真类和假类分类 画出输入的点
%PLOTDATA Plots the data points X and y into a new figure 
%   PLOTDATA(x,y) plots the data points with + for the positive examples
%   and o for the negative examples. X is assumed to be a Mx2 matrix.

% Create New Figure
figure; hold on;

% ====================== YOUR CODE HERE ======================
% Instructions: Plot the positive and negative examples on a
%               2D plot, using the option 'k+' for the positive
%               examples and 'ko' for the negative examples.
%
% Find Indices of Positive and Negative Examples              //返回 y=1和y=0的位置，也就是行数
  pos = find(y==1);        neg = find(y == 0);                //利用find的查找功能，把正类和负类分开，并把横纵坐标保存在pos里面
% Plot Examples
plot(X(pos, 1), X(pos, 2), 'k+','LineWidth', 2, ...
'MarkerSize', 7);              
plot(X(neg, 1), X(neg, 2), 'ko', 'MarkerFaceColor', 'y', ...
'MarkerSize', 7);

% =========================================================================



hold off;

end

%% =========== Part 1: Regularized Logistic Regression ============
%  In this part, you are given a dataset with data points that are not
%  linearly separable. However, you would still like to use logistic
%  regression to classify the data points.
%
%  To do so, you introduce more features to use -- in particular, you add
%  polynomial features to our data matrix (similar to polynomial
%  regression).
%

% Add Polynomial Features

% Note that mapFeature also adds a column of ones for us, so the intercept
% term is handled
X = mapFeature(X(:,1), X(:,2));

% Initialize fitting parameters
initial_theta = zeros(size(X, 2), 1);

% Set regularization parameter lambda to 1
lambda = 1;

% Compute and display initial cost and gradient for regularized logistic
% regression
[cost, grad] = costFunctionReg(initial_theta, X, y, lambda);

难点7：mapFeature是怎么工作的，原理？

难点8：[cost, grad] = costFunctionReg(initial_theta, X, y, lambda);怎么工作？

%% ============= Part 2: Regularization and Accuracies =============
%  Optional Exercise:
%  In this part, you will get to try different values of lambda and
%  see how regularization affects the decision coundart
%
%  Try the following values of lambda (0, 1, 10, 100).
%
%  How does the decision boundary change when you vary lambda? How does
%  the training set accuracy vary?
%

% Initialize fitting parameters
initial_theta = zeros(size(X, 2), 1);

% Set regularization parameter lambda to 1 (you should vary this)
lambda = 1;

% Set Options
options = optimset('GradObj', 'on', 'MaxIter', 400);

% Optimize
[theta, J, exit_flag] = ...
    fminunc(@(t)(costFunctionReg(t, X, y, lambda)), initial_theta, options);

难点9：为什么fminunc可以返回三个变量？

ps:先把问题记录一下，稍后会一个个解决。