• 卷积神经网络的简单可视化


    卷积神经网络的简单可视化


    本次将进行卷积神经网络权重的简单可视化。

    在本篇教程的前半部分,我们会首先定义一个及其简单的 CNN 模型,并手工指定一些过滤器权重参数,作为卷积核参数。

    后半部分,我们会使用 FashionMNIST 数据集,并且定义一个 2 层的 CNN 模型,将模型训练至准确率在 85% 以上,再进行模型卷积核的可视化。

    1. 简单卷积网络模型的可视化

    1.1 指定过滤器卷积层的可视化

    在下面的练习中,我们将手动定义几个类似索比尔算子的过滤器,并将它们指定给一个极其简单地卷积神经网络模型。然后可视化卷积层 4 个过滤器的输出(即 feature maps)。

    加载目标图像

    import cv2
    import matplotlib.pyplot as plt
    %matplotlib inline
    
    img_path = 'images/udacity_sdc.png'
    bgr_img = cv2.imread(img_path)
    
    gray_img = cv2.cvtColor(bgr_img, cv2.COLOR_BGR2GRAY)
    gray_img = gray_img.astype("float32")/255
    
    plt.imshow(gray_img, cmap='gray')
    plt.show()
    

    手动定义过滤器

    import numpy as np
    
    filter_vals = np.array([[-1, -1, 1, 1], [-1, -1, 1, 1], [-1, -1, 1, 1], [-1, -1, 1, 1]])
    
    # 变化产生更丰富的过滤器
    filter_1 = filter_vals
    filter_2 = -filter_1
    filter_3 = filter_1.T
    filter_4 = -filter_3
    filters = np.array([filter_1, filter_2, filter_3, filter_4])
    
    fig = plt.figure(figsize=(10, 5))
    for i in range(4):
        ax = fig.add_subplot(1, 4, i+1, xticks=[], yticks=[])
        ax.imshow(filters[i], cmap='gray')
        ax.set_title('Filter %s' % str(i+1))
        width, height = filters[i].shape
        for x in range(width):
            for y in range(height):
                ax.annotate(str(filters[i][x][y]), xy=(y,x),
                           horizontalalignment='center',
                           verticalalignment='center', 
                           color='white' if filters[i][x][y] < 0 else 'black')
    

    定义简单卷积神经网络

    import torch
    import torch.nn as nn
    import torch.nn.functional as F
    
    class Net(nn.Module):
        def __init__(self, weight):
            super(Net, self).__init__()
            k_height, k_width = weight.shape[2:]
            self.conv = nn.Conv2d(1, 4, kernel_size=(k_height, k_width), bias=False)
            self.conv.weight = torch.nn.Parameter(weight)
            self.pool = nn.MaxPool2d(4,4)
            
        def forward(self, x):
            conv_x = self.conv(x)
            activated_x = F.relu(conv_x)
            pooled_x = self.pool(activated_x)
            
            return conv_x, activated_x, pooled_x
        
    # filters 的大小为 4 4 4
    # weight 的大小被增加为 4 1 4 4,1 的维度是针对输入的一个通道
    weight = torch.from_numpy(filters).unsqueeze(1).type(torch.FloatTensor)
    model = Net(weight)
    
    print('Filters shape: ', filters.shape)
    print('weights shape: ', weight.shape)
    print(model)
    
    Filters shape:  (4, 4, 4)
    weights shape:  torch.Size([4, 1, 4, 4])
    Net(
      (conv): Conv2d(1, 4, kernel_size=(4, 4), stride=(1, 1), bias=False)
      (pool): MaxPool2d(kernel_size=4, stride=4, padding=0, dilation=1, ceil_mode=False)
    )
    

    可视化卷积输出

    定义一个函数 viz_layer,在这个方法可以可视化某一层卷积的输出。

    def viz_layer(layer, n_filters=4):
        fig = plt.figure(figsize=(20, 20))
        
        for i in range(n_filters):
            ax = fig.add_subplot(1, n_filters, i+1, xticks=[], yticks=[])
            ax.imshow(np.squeeze(layer[0,i].data.numpy()), cmap='gray')
            ax.set_title('Output %s' % str(i+1))
    
    # 输出原图
    plt.imshow(gray_img, cmap='gray')
    # 格式化输出过滤器(卷积核)
    fig = plt.figure(figsize=(12, 6))
    fig.subplots_adjust(left=0, right=1.5, bottom=0.8, top=1, hspace=0.05, wspace=0.05)
    for i in range(4):
        ax = fig.add_subplot(1, 4, i+1, xticks=[], yticks=[])
        ax.imshow(filters[i], cmap='gray')
        ax.set_title('Filter %s' % str(i+1))
        
    # 为 gray img 添加 1 个 batch 维度,以及 1 个 channel 维度,并转化为 tensor
    gray_img_tensor = torch.from_numpy(gray_img).unsqueeze(0).unsqueeze(1)
    print(gray_img.shape)
    print(gray_img_tensor.shape)
    
    # 将输入图传入模型,获得输出
    conv_layer, activated_layer, pooled_layer = model(gray_img_tensor)
    
    # 可视化卷积输出
    viz_layer(conv_layer)
    
    (213, 320)
    torch.Size([1, 1, 213, 320])
    

    # 可视化卷积后激活函数后的输出
    viz_layer(activated_layer)
    

    1.2 指定过滤器池化层的可视化

    下面可视化池化层后的输出。

    # 可视化池化层后的输出
    viz_layer(pooled_layer)
    

    2. 多层卷积网络模型的可视化

    在下面的练习中,我们将定义一个相对复杂点的神经网络,并使用 FashionMNIST 数据集训练至 85% 以上的准确率,其后再对神经网络进行可视化分析。

    2.1 加载 FashionMNIST 数据集

    FashionMNIST 相当于一种对 MNIST 数据集的升级。MNIST 数据集的数字识别在目前来说,模式比较简单,可能作为深度神经网络模型的目标数据集稍显简单。FashionMNIST 将图像内容变为“时尚衣物”,图像格式不变,使用起来几乎与 MNIST 无异,且比 MNIST 更能考验模型对数据模式的学习能力。

    FashionMNIST 的类别列表:

    0:T-shirt/top(T恤) 
    1:Trouser(裤子) 
    2:Pullover(套衫) 
    3:Dress(裙子) 
    4:Coat(外套) 
    5:Sandal(凉鞋) 
    6:Shirt(汗衫) 
    7:Sneaker(运动鞋) 
    8:Bag(包) 
    

    加载 FashionMNIST 数据集

    import torch
    import torchvision
    
    from torchvision.datasets import FashionMNIST
    from torch.utils.data import DataLoader
    from torchvision import transforms
    
    data_transform = transforms.ToTensor()
    
    train_data = FashionMNIST(root='./data', train=True,
                             download=False, transform=data_transform)
    test_data = FashionMNIST(root='./data', train=False,
                             download=False, transform=data_transform)
    
    # Print out some stats about the training and test data
    print('Train data, number of images: ', len(train_data))
    print('Test data, number of images: ', len(test_data))
    
    Train data, number of images:  60000
    Test data, number of images:  10000
    

    创建数据加载器

    batch_size = 20
    
    train_loader = DataLoader(train_data, batch_size=batch_size, shuffle=True)
    test_loader = DataLoader(test_data, batch_size=batch_size, shuffle=True)
    
    # specify the image classes
    classes = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 
               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
    

    可视化目标数据集的部分数据

    import numpy as np
    import matplotlib.pyplot as plt
    
    %matplotlib inline
    
    dataiter = iter(train_loader)
    images, labels = dataiter.next()
    images = images.numpy()
    
    # plot the images in the batch, along with the corresponding labels
    fig = plt.figure(figsize=(25, 4))
    for idx in np.arange(batch_size):
        ax = fig.add_subplot(2, batch_size/2, idx+1, xticks=[], yticks=[])
        ax.imshow(np.squeeze(images[idx]), cmap='gray')
        ax.set_title(classes[labels[idx]])#### 加载 FashionMNIST 数据集
    

    2.2 训练多层卷积模型

    定义模型

    下面定义一个具有两层卷积的模型,加入的 dropout 在一定程度上起到防止过拟合的作用。

    import torch.nn as nn
    import torch.nn.functional as F
    
    class Net(nn.Module):
        def __init__(self):
            super(Net, self).__init__()
            
            self.conv1 = nn.Conv2d(1, 16, 3, padding=1)
            self.pool1 = nn.MaxPool2d(2, 2)
            self.conv2 = nn.Conv2d(16, 32, 3, padding=1)
            self.pool2 = nn.MaxPool2d(2, 2)
            self.activation_l = nn.ReLU()
            
            self.fc = nn.Linear(32 * 7 * 7, 24)
            self.out = nn.Linear(24, 10)
            self.dropout = nn.Dropout(p=0.5)
            self.activation_out = nn.Softmax(dim=1)
            
        def forward(self, x):
            x = self.activation_l(self.conv1(x))
            x = self.pool1(x)
            x = self.activation_l(self.conv2(x))
            x = self.pool2(x)
            
            x = x.view(x.size(0), -1)
            x = self.activation_l(self.fc(x))
            x = self.dropout(x)
            x = self.activation_out(self.out(x))
            
            return x
    

    训练模型

    import torch.optim as optim
    
    criterion = nn.CrossEntropyLoss()
    
    optimizer = torch.optim.Adam(net.parameters())
    
    def train(n_epochs):
        for epoch in range(n_epochs):
            running_loss = 0.0
            for batch_i, data in enumerate(train_loader):
                inputs, labels = data
                optimizer.zero_grad()
                outputs = net(inputs)
                loss = criterion(outputs, labels)
                loss.backward()
                optimizer.step()
                
                running_loss += loss.item()
                
                if batch_i % 1000 == 999:
                    print('Epoch: {}, Batch: {}, Avg. Loss: {}'.format(epoch + 1, batch_i+1, running_loss/1000))
                    running_loss = 0.0
                    
        print('Finished Training')
        
    n_epochs = 10
    
    train(n_epochs)
    
    model_dir = 'saved_models/'
    model_name = 'model_best.pt'
    
    torch.save(net.state_dict(), model_dir+model_name)
    

    加载训练的模型

    net = Net()
    
    net.load_state_dict(torch.load('saved_models/model_best.pt'))
    
    print(net)
    
    Net(
      (conv1): Conv2d(1, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (pool1): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
      (conv2): Conv2d(16, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (pool2): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
      (activation_l): ReLU()
      (fc): Linear(in_features=1568, out_features=24, bias=True)
      (out): Linear(in_features=24, out_features=10, bias=True)
      (dropout): Dropout(p=0.5)
      (activation_out): Softmax()
    )
    

    在测试数据集上测试模型

    test_loss = torch.zeros(1)
    class_correct = list(0. for i in range(10))
    class_total = list(0. for i in range(10))
    
    print(class_correct)
    print(test_loss)
    
    [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
    tensor([ 0.])
    
    net.eval()
    
    criterion = torch.nn.CrossEntropyLoss()
    
    for batch_i, data in enumerate(test_loader):
        inputs, labels = data
        output = net(inputs)
        loss = criterion(outputs, labels)
        
        # update average test loss 
        test_loss = test_loss + ( (torch.ones(1) / (batch_i+1)) * (loss.data - test_loss) )
        
        _, predicted = torch.max(output.data, 1)
        
        correct = np.squeeze(predicted.eq(labels.data.view_as(predicted)))
        
        for i in range(batch_size):
            label = labels.data[i]
            class_correct[label] += correct[i].item()
            class_total[label] += 1
            
    print('Test Loss: {:.6f}
    '.format(test_loss.numpy()[0]))
    
    for i in range(10):
        if class_total[i] > 0:
            print('Test Accuracy of %5s: %2d%% (%2d/%2d)' % (
                classes[i], 100 * class_correct[i] / class_total[i],
                np.sum(class_correct[i]), np.sum(class_total[i])))
        else:
            print('Test Accuracy of %5s: N/A (no training examples)' % (classes[i]))
    
            
    print('
    Test Accuracy (Overall): %2d%% (%2d/%2d)' % (
        100. * np.sum(class_correct) / np.sum(class_total),
        np.sum(class_correct), np.sum(class_total)))
    
    Test Loss: 2.362950
    
    Test Accuracy of T-shirt/top: 85% (850/1000)
    Test Accuracy of Trouser: 96% (963/1000)
    Test Accuracy of Pullover: 84% (842/1000)
    Test Accuracy of Dress: 91% (911/1000)
    Test Accuracy of  Coat: 85% (856/1000)
    Test Accuracy of Sandal: 98% (989/1000)
    Test Accuracy of Shirt: 49% (495/1000)
    Test Accuracy of Sneaker: 94% (948/1000)
    Test Accuracy of   Bag: 97% (978/1000)
    Test Accuracy of Ankle boot: 93% (930/1000)
    
    Test Accuracy (Overall): 87% (8762/10000)
    

    2.3 特征可视化

    模型得到训练并且在测试数据上可以达到 87% 的准确率,下面让我们进行可视化。

    可视化策略是从模型中将各卷积层的参数提取出来,作为独立的过滤器,使用 OpenCV 的 filter2D 函数,施加在一张从测试集抽样出的图像中。观察过滤器对图像起到的作用,并尝试去解释当前过滤器对原图起到了怎样的滤波作用。

    从数据集中抽取单张图片

    dataiter = iter(test_loader)
    images, labels = dataiter.next()
    images = images.numpy()
    
    idx = 15
    img = np.squeeze(images[idx])
    
    import cv2
    plt.imshow(img, cmap='gray')
    
    <matplotlib.image.AxesImage at 0x124832a90>
    

    进行第一层卷积核的可视化

    weights = net.conv1.weight.data
    w = weights.numpy()
    print(w.shape)
    
    fig = plt.figure(figsize=(30, 10))
    columns = 4 * 2
    row = 4
    for i in range(0, columns * row):
        fig.add_subplot(row, columns, i+1)
        if ((i%2)==0):
            plt.imshow(w[int(i/2)][0], cmap='gray')
        else:
            c = cv2.filter2D(img, -1, w[int((i-1)/2)][0])
            plt.imshow(c, cmap='gray')
    plt.show()
    
    (16, 1, 3, 3)
    

    进行第一层卷积核的可视化

    weights = net.conv2.weight.data
    w = weights.numpy()
    print(w.shape)
    
    fig = plt.figure(figsize=(30, 20))
    columns = 4 * 2
    row = 8
    for i in range(0, columns * row):
        fig.add_subplot(row, columns, i+1)
        if ((i%2)==0):
            plt.imshow(w[int(i/2)][0], cmap='gray')
        else:
            c = cv2.filter2D(img, -1, w[int((i-1)/2)][0])
            plt.imshow(c, cmap='gray')
    plt.show()
    
    (32, 16, 3, 3)
    

    可以看到一些卷积核起到了边缘检测的功能,不同的卷积核对不同方向,不同的纹理,或者说不同的图像内容敏感。

    感觉这种人以主观想法可视化卷积的方法还不够丰满,可能这就算是简单的神经网络的可视化方法。除了卷积核的可视化,还可以进行全连接层的可视化。

    关于全连接层的可视化,有教程表示是通过可视化类似类别间不同数据单例的“嵌入向量”距离进行可视化的,可能还需要对全连接层产生的“嵌入向量”进行 T-SNE 将为后再进行可视化。如果后续遇到了相关内容,会在本文中再补上。

    后记

    本文内容参考自 Udacity 计算机视觉纳米学位练习,官方源码连接:

    https://github.com/udacity/CVND_Exercises/tree/master/1_5_CNN_Layers

  • 相关阅读:
    HTTPS和HTTP的区别
    .NET反射、委托技术与设计模式
    中文化和国际化问题权威解析之三:Java中文问题分析
    Windows下Critical Section、Event、Mutex、Semaphores区别
    使用Forms Authentication 身份验证 之 Basic Knowledge
    介绍几个java虚拟机性能监测工具
    理解Semaphore和Mutex
    中文化和国际化问题权威解析之四:Java中文化和国际化攻略
    中文化和国际化问题权威解析之一:字符编码发展历程
    Happy new year!
  • 原文地址:https://www.cnblogs.com/alexme/p/11366792.html
Copyright © 2020-2023  润新知