• DataWhale 动手学深度学习PyTorch版-task3+4+5:循环神经网络;机器翻译;卷积网络


    课程引用自伯禹平台:https://www.boyuai.com/elites/course/cZu18YmweLv10OeV

    《动手学深度学习》官方网址:http://zh.gluon.ai/ ——面向中文读者的能运行、可讨论的深度学习教科书。

    第二次打卡:

    Task03: 过拟合、欠拟合及其解决方案;梯度消失、梯度爆炸;循环神经网络进阶

    Task04:机器翻译及相关技术;注意力机制与Seq2seq模型;Transformer

    Task05:卷积神经网络基础;leNet;卷积神经网络进阶

    有部分内容学过了,所以着重学习了RNN的代码、CNN的简洁实现,记录内容如下:

    VGG11的实现:

    def vgg_block(num_convs, in_channels, out_channels): #卷积层个数,输入通道数,输出通道数
        blk = []
        for i in range(num_convs):
            if i == 0:
                blk.append(nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1))
            else:
                blk.append(nn.Conv2d(out_channels, out_channels, kernel_size=3, padding=1))
            blk.append(nn.ReLU())
        blk.append(nn.MaxPool2d(kernel_size=2, stride=2)) # 这里会使宽高减半
        return nn.Sequential(*blk)
    
     
    conv_arch = ((1, 1, 64), (1, 64, 128), (2, 128, 256), (2, 256, 512), (2, 512, 512))
    # 经过5个vgg_block, 宽高会减半5次, 变成 224/32 = 7
    fc_features = 512 * 7 * 7 # c * w * h
    fc_hidden_units = 4096 # 任意
    
     
    def vgg(conv_arch, fc_features, fc_hidden_units=4096):
        net = nn.Sequential()
        # 卷积层部分
        for i, (num_convs, in_channels, out_channels) in enumerate(conv_arch):
            # 每经过一个vgg_block都会使宽高减半
            net.add_module("vgg_block_" + str(i+1), vgg_block(num_convs, in_channels, out_channels))
        # 全连接层部分
        net.add_module("fc", nn.Sequential(d2l.FlattenLayer(),
                                     nn.Linear(fc_features, fc_hidden_units),
                                     nn.ReLU(),
                                     nn.Dropout(0.5),
                                     nn.Linear(fc_hidden_units, fc_hidden_units),
                                     nn.ReLU(),
                                     nn.Dropout(0.5),
                                     nn.Linear(fc_hidden_units, 10)
                                    ))
        return net
    
     
    net = vgg(conv_arch, fc_features, fc_hidden_units)
    X = torch.rand(1, 1, 224, 224)
    
    # named_children获取一级子模块及其名字(named_modules会返回所有子模块,包括子模块的子模块)
    for name, blk in net.named_children(): 
        X = blk(X)
        print(name, 'output shape: ', X.shape)
    
     
    vgg_block_1 output shape:  torch.Size([1, 64, 112, 112])
    vgg_block_2 output shape:  torch.Size([1, 128, 56, 56])
    vgg_block_3 output shape:  torch.Size([1, 256, 28, 28])
    vgg_block_4 output shape:  torch.Size([1, 512, 14, 14])
    vgg_block_5 output shape:  torch.Size([1, 512, 7, 7])
    fc output shape:  torch.Size([1, 10])
    
     
    ratio = 8
    small_conv_arch = [(1, 1, 64//ratio), (1, 64//ratio, 128//ratio), (2, 128//ratio, 256//ratio), 
                       (2, 256//ratio, 512//ratio), (2, 512//ratio, 512//ratio)]
    net = vgg(small_conv_arch, fc_features // ratio, fc_hidden_units // ratio)
    print(net)
    batchsize=16
    #batch_size = 64
    # 如出现“out of memory”的报错信息,可减小batch_size或resize
    # train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size, resize=224)
    
    lr, num_epochs = 0.001, 5
    optimizer = torch.optim.Adam(net.parameters(), lr=lr)
    d2l.train_ch5(net, train_iter, test_iter, batch_size, optimizer, device, num_epochs)
     
  • 相关阅读:
    virtualenv wrapper安装配置
    Docker 制作镜像-redis
    nginx+redis多进程镜像制作
    Docker 制作镜像
    docker操作常用命令
    docker设置镜像加速
    Centos7安装docker CE社区版
    定时器线程Timer
    linux系统history记录不全的原因
    zabbix监控windows系统的磁盘IO情况
  • 原文地址:https://www.cnblogs.com/haiyanli/p/12332918.html
Copyright © 2020-2023  润新知