深度学习中numpy的一些基本操作

sigmoid 求导:

import numpy as np

def sigmoid_derivative(x):
    s = 1 / (1+np.exp(-x))
    ds = s*(1-s)
    return ds

改变arrays的形状:

两个常用的函数： np.shape() 、 np.reshape()

def image2vector(image):
    v = image.reshape(image.shape[0]*image.shape[1]*image.shape[2], 1)
    return v

normalizing rows:

It often leads to a better performance because gradient descent coverges faster after normalizations.
Here, by normalizing means changing x to (frac{x}{Vert x Vert})(deviding each of row vector of x by its norm.

def normalizeRows(x):
    x_norm = np.linalg.norm(x, axis=1, keepdims=True)
    x = x/x_norm
    return x

x = np.array([[0,3,4],[1,6,4]])
print("normalizeRows(x) = " + str(normalizeRows(x)))

normalizeRows(x) = [[0.         0.6        0.8       ]
 [0.13736056 0.82416338 0.54944226]]

broadcasting and softmax function:

Broadcasting is very useful for performing mathematical operations between arrays of different shapes.

softmax function: It can be thought as normalizing function used when your algorithm needs to classify two or more classes.

for (xinmathbb{R}^{1 imes n}),

$softmax(x) = softmax([x_1 x_2 ... x_n])$ $ = lbrackfrac{e^x_1}{sum _{j}e^x_j} frac{e^x_2}{sum _{j}e^x_j} ... frac{e^x_n}{sum _{j}e^x_j} brack $

for a matrix (xinmathbb{R}^{m imes n})

$softmax(x)=egin{pmatrix} softmax(first row of x)\softmax(second of row of x) \ ... \softmax(last row of x) end{pmatrix}$

def softmax(x):
    x_exp = np.exp(x)
    x_sum = np.sum(x_exp, axis=1, keepdims = True)
    s = x_exp / x_sum
    
    return s

L1 and L2 loss function:

L1 loss is defined as:

$L_1(hat{y} ,y) = sum_{i=0}^{m}|y^{(i)} - hat{y}^{(i)}|$

def L1(yhat, y):
    
    loss = sum(abs(y-yhat))
    
    return loss

L2 loss is defined as:

$L_2(hat{y}, y)=sum_{i=0}^{m}(y^{(i)} - hat{y}^{(i)})^2$

def L2(yhat, y):
    
    loss = np.dot(y-yhat, y-yhat)
    
    return loss

yhat = np.array([0.9, 0.2 ,0.1, 0.4, .9])
y = np.array([1, 0, 0, 1, 1])
print("L2 = " + str(L2(yhat, y)))

L2 = 0.43