• YOLOV3实战(darknet)


    一. 准备工作

    1)实验环境:

      darknet 是由 C 和 CUDA 开发的,不需要配置其他深度学习的框架(如,tensorflow、caffe 等),支持 CPU 和 GPU 运算,而且安装过程非常简单。本文使用的 CUDA 的版本如下所示:

      CUDA:9.0
      CUDNN:7.0
    2)下载 github 源码:

    git clone https://github.com/pjreddie/darknet.git

     3)配置darknet编译环境:

    Ⅰ. 编译一直在darknet文件夹下;

    Ⅱ. 每次修改 Makefile 文件后需重新 make 一下才能生效;

    Ⅲ. 默认的 Makefile 是使用 CPU;

      修改Makefile编译环境配置文件:

    GPU=1    # 是否打开GPU
    CUDNN=1   # 是否打开cudnn
    OPENCV=0  # 是否打开opencv
    OPENMP=0
    DEBUG=1   # 是否进行debug

    ARCH= -gencode arch=compute_61,code=compute_61  # 根据GPU计算能力选择对应数值(GTX 1080Ti:61、Tesla K80:37)官网查看:https://developer.nvidia.com/cuda-gpus
    ...
    NVCC=/usr/local/cuda-9.0/bin/nvcc   #修改nvcc路径
    ...
    ifeq ($(GPU), 1)       #修改cuda路径--黄色部分,若不需要更改删除黄色部分即可
    COMMON+= -DGPU -I/usr/local/cuda-9.0/include
    CFLAGS+= -DGPU
    LDFLAGS+= -L/usr/local/cuda-9.0/lib64 -lcuda -lcudart -lcublas -lcurand
    endif
    ifeq ($(CUDNN), 1)      #修改cudnn路径--黄色部分,若不需要更改删除黄色部分即可
    COMMON+= -DCUDNN -I/usr/local/cuda-9.0/include
    CFLAGS+= -DCUDNN
    LDFLAGS+= -L/usr/local/cuda-9.0/lib64 -lcudnn
    endif

      重新编译

    make clean
    make

       执行./darknet就可以执行编译

     4) 实验环境测试

      下载预训练模型权重yolov3.weights

      下载地址:https://pjreddie.com/media/files/yolov3.weights

      基于yolov3.weights模型权重的测试

    测试单张图片(下面两个指令相同)
    ./darknet detector test cfg/coco.data cfg/yolov3.cfg yolov3.weights data/dog.jpg
    ./darknet detect cfg/yolov3.cfg yolov3.weights data/dog.jpg
    测试多张图片
    ./darknet detect cfg/yolov3.cfg yolov3.weights
    ~根据提示输入图片路径

      输出:保存./darknet目录下的predict.png 

    5) ./darknet编译格式

    ./darknet detector test <data_cfg> <models_cfg> <weights> <test_file> [-thresh] [-out]
    ./darknet detector train <data_cfg> <models_cfg> <weights> [-thresh] [-gpu] [-gpus] [-clear]
    ./darknet detector valid <data_cfg> <models_cfg> <weights> [-out] [-thresh]
    ./darknet detector recall <data_cfg> <models_cfg> <weights> [-thresh]

    '<>'必选项,’[ ]‘可选项

      data_cfg:数据配置文件,eg:cfg/voc.data

      models_cfg:模型配置文件,eg:cfg/yolov3-voc.cfg

      weights:权重配置文件,eg:weights/yolov3.weights

      test_file:测试文件,eg:*/*/*/test.txt

      -thresh:显示被检测物体中confidence大于等于 [-thresh] 的bounding-box,默认0.005

      -out:输出文件名称,默认路径为results文件夹下,eg:-out "" //输出class_num个文件,文件名为class_name.txt;若不选择此选项,则默认输出文件名为comp4_det_test_"class_name".txt

      -i/-gpu:指定单个gpu,默认为0,eg:-gpu 2

      -gpus:指定多个gpu,默认为0,eg:-gpus 0,1,2

    二. 实验步骤

    基于darknet在VOC格式的数据集上训练yolov3

    1)下载 Pascal VOC 2007 和 2012 的数据集

    wget https://pjreddie.com/media/files/VOCtrainval_11-May-2012.tar
    wget https://pjreddie.com/media/files/VOCtrainval_06-Nov-2007.tar
    wget https://pjreddie.com/media/files/VOCtest_06-Nov-2007.tar
    tar xf VOCtrainval_11-May-2012.tar
    tar xf VOCtrainval_06-Nov-2007.tar
    tar xf VOCtest_06-Nov-2007.tar
    

       注:若非voc格式的数据集需先生成voc格式的数据集,数据集的目录严格按照voc数据集的目录结构

    2)生成 VOC 数据集的标签

      darknet 需要一个“.txt”格式的标签文件,每行表示一张图像的信息,包括(x, y, w, h)格式为:<object-class> <x> <y> <width> <height>

      darknet 官网提供了一个针对 VOC 数据集,处理标签的脚本,darknet/scripts文件夹下的voc_label.py文件,若无执行如下命令下载文件:

    wget https://pjreddie.com/media/files/voc_label.py # 获取脚本

       修改 voc_label.py(4处)

    ①sets=[('2007', 'train'), ('2007', 'val'), ('2007', 'test')] #替换为自己的数据集
    ②classes = ["head", "eye", "nose"] #修改为自己的类别
    ③in_file = open('VOCdevkit/VOC%s/Annotations/%s.xml'%(year, image_id)) #将数据集放于当前目录下
    ④os.system("cat 2007_train.txt 2007_val.txt > train.txt") #修改为自己的数据集用作训练

      修改完成后执行:

    python voc_label.py    # 执行脚本,获得所需的“.txt”文件

      生成的xml文件在 VOCdevkit/VOC%s/labels 目录下

      标签处理完成后,在voc_label.py所在目录下生成如下几个文件:

      2007_text.txt、2007_train.txt、2007_val.txt、2012_text.txt、2012_train.txt、train.txt、train_all.txt

    3)修改配置文件

      Ⅰ. 数据配置文件——cfg/voc.data
    classes= 20                                           # 类别总数
    train = /home/xieqi/project/train.txt                        # 训练数据所在的位置
    valid = /home/xieqi/project/2007_test.txt                     # 测试数据所在的位置
    names = data/voc.names                                 # 修改见voc.names
    backup = backup                                     # 输出的权重信息保存的文件夹
    eval =
    results =                                       
      Ⅱ. 模型配置文件——cfg/yolov3-voc.cfg
    batch=64         # 一批训练样本的样本数量,每batch个样本更新一次参数
    subdivisions=32     # 它会让你的每一个batch不是一下子都丢到网络里。而是分成subdivision对应数字的份数,一份一份的跑完后,在一起打包算作完成一次iteration
    width=416         # 只可以设置成32的倍数
    height=416        # 只可以设置成32的倍数
    
    channels=3       # 若为灰度图,则chennels=1,另外还需修改/scr/data.c文件中的load_data_detection函数;若为RGB则 channels=3 ,无需修改/scr/data.c文件
    
    momentum=0.9        # 最优化方法的动量参数,这个值影响着梯度下降到最优值得速度 
    decay=0.0005       # 权重衰减正则项,防止过拟合
    angle=0          # 通过旋转角度来生成更多训练样本
    saturation = 1.5    # 通过调整饱和度来生成更多训练样本
    exposure = 1.5      # 通过调整曝光量来生成更多训练样本
    hue=.1           # 通过调整色调来生成更多训练样本
    
    
    learning_rate=0.001        # 学习率, 刚开始训练时, 以 0.01 ~ 0.001 为宜, 一定轮数过后,逐渐减缓。
    burn_in=1000            # 在迭代次数小于burn_in时,其学习率的更新有一种方式,大于burn_in时,才采用policy的更新方式
    max_batches = 50200       # 训练步数
    policy=steps           # 学习率调整的策略
    steps=40000,45000          # 开始衰减的步数
    scales=.1,.1           # 在第40000和第45000次迭代时,学习率衰减10倍
    ...
    [convolutional]——YOLO层前一层卷积层
    ...
    filters=24            # 每一个[yolo]层前的最后一个卷积层中的 filters=num(yolo层个数)*(classes+5)
    ...
    
    [yolo]
    mask = 6,7,8
    anchors = 10,13,  16,30,  33,23,  30,61,  62,45,  59,119,  116,90,  156,198,  373,326  #如果想修改默认anchors数值,使用k-means即可;
    classes=3        # 修改为自己的类别数
    num=9           # 每个grid cell预测几个box,和anchors的数量一致。调大num后训练时Obj趋近0的话可以尝试调大object_scale
    jitter=.3       # 利用数据抖动产生更多数据, jitter是crop的参数, jitter=.3,就是在0~0.3中进行crop
    ignore_thresh = .5   # 决定是否需要计算IOU误差的参数,大于thresh,IOU误差不会夹在cost function中
    truth_thresh = 1
    random=1         # 如果为1,每次迭代图片大小随机从320到608,步长为32,如果为0,每次训练大小与输入大小一致
    ...
      Ⅲ. 标签配置文件——data/voc.names
    head #自己需要探测的类别,一行一个
    eye
    nose
       Ⅳ.  修改数据格式——scr/data.c
    data load_data_detection(int n, char **paths, int m, int w, int h, int boxes, int classes, float jitter, float hue, float saturation, float exposure)
    {
        char **random_paths = get_random_paths(paths, n, m);
        int i;
        data d = {0};
        d.shallow = 0;
    
        d.X.rows = n;
        d.X.vals = calloc(d.X.rows, sizeof(float*));
        d.X.cols = h*w;                   //灰阶图
        //d.X.cols = h*w*3;               //RGB图
        
      ...    

    4)下载预训练的参数(卷积权重)

      “darknet53.conv.74”是使用 Imagenet 数据集进行预训练:

    wget https://pjreddie.com/media/files/darknet53.conv.74 # 下载预训练的网络模型参数

      获取其他模型类似 darknet53.conv.74 的预训练权重

        Ⅰ. 下载官方预训练模型的权重

          https://pjreddie.com/darknet/yolo/    //COCO数据集训练的

          https://pjreddie.com/darknet/imagenet/  //Imagenet数据集训练的

         eg: wget https://pjreddie.com/media/files/yolov3-tiny.weights 

        Ⅱ. 转换为类似darknet.conv.74的预训练权重

          https://github.com/AlexeyAB/darknet/blob/57e878b4f9512cf9995ff6b5cd6e0d7dc1da9eaf/build/darknet/x64/partial.cmd#L24

         eg:  ./darknet partial cfg/yolov3-tiny yolov3-tiny.weights yolov-tiny.conv.15 15 

    5)训练模型:

      Ⅰ. 单GPU训练: ./darknet detector train <data_cfg> <train_cfg> <weights> -gpu <gpu_id> 

    ./darknet detector train cfg/voc.data cfg/yolov3-voc.cfg darknet53.conv.74 -i 1

      Ⅱ. 多GPU训练:  ./darknet detector train <data_cfg> <model_cfg> <weights> -gpus <gpu_list> 

    ./darknet detector train cfg/voc.data cfg/yolov3-voc.cfg darknet53.conv.74 -gpus 0,1,2,3

      Ⅲ. 从checkpoint继续训练

    ./darknet detector train cfg/voc.data cfg/yolov3-voc.cfg backup/yolov3-voc.backup -gpus 0,1,2,3

      Ⅳ. CPU训练: 

    ./darknet detector train <data_cfg> <model_cfg> <weights> -nogpu 

      Ⅴ. 生成loss-iter曲线

      在执行训练命令的时候加一下管道,tee一下log:

    ./darknet detector train cfg/voc.data cfg/yolov3-voc.cfg | tee result/log/training.log

      将下面的python代码保存为drawcurve.py。并执行

    python  drawcurve.py training.log 0

      ~drawcurve.py

    import argparse
    import sys
    import matplotlib.pyplot as plt
    def main(argv):
        parser = argparse.ArgumentParser()
        parser.add_argument("log_file",  help = "path to log file"  )
        parser.add_argument( "option", help = "0 -> loss vs iter"  )
        args = parser.parse_args()
        f = open(args.log_file)
        lines  = [line.rstrip("\n") for line in f.readlines()]
        # skip the first 3 lines
        lines = lines[3:]
        numbers = {'1','2','3','4','5','6','7','8','9','0'}
        iters = []
        loss = []
        for line in lines:
            if line[0] in numbers:
                args = line.split(" ")
                if len(args) >3:
                    iters.append(int(args[0][:-1]))
                    loss.append(float(args[2]))
        plt.plot(iters,loss)
        plt.xlabel('iters')
        plt.ylabel('loss')
        plt.grid()
        plt.show()
    if __name__ == "__main__":
        main(sys.argv)

    6)训练过程:

      训练的log格式如下:

    Loaded: 4.533954 seconds
    Region Avg IOU: 0.262313, Class: 1.000000, Obj: 0.542580, No Obj: 0.514735, Avg Recall: 0.162162,  count: 37
    Region Avg IOU: 0.175988, Class: 1.000000, Obj: 0.499655, No Obj: 0.517558, Avg Recall: 0.070423,  count: 71
    Region Avg IOU: 0.200012, Class: 1.000000, Obj: 0.483404, No Obj: 0.514622, Avg Recall: 0.075758,  count: 66
    Region Avg IOU: 0.279284, Class: 1.000000, Obj: 0.447059, No Obj: 0.515849, Avg Recall: 0.134615,  count: 52
    1: 629.763611, 629.763611 avg, 0.001000 rate, 6.098687 seconds, 64 images
    Loaded: 2.957771 seconds
    Region Avg IOU: 0.145857, Class: 1.000000, Obj: 0.051285, No Obj: 0.031538, Avg Recall: 0.069767,  count: 43
    Region Avg IOU: 0.257284, Class: 1.000000, Obj: 0.048616, No Obj: 0.027511, Avg Recall: 0.078947,  count: 38
    Region Avg IOU: 0.174994, Class: 1.000000, Obj: 0.030197, No Obj: 0.029943, Avg Recall: 0.088889,  count: 45
    Region Avg IOU: 0.196278, Class: 1.000000, Obj: 0.076030, No Obj: 0.030472, Avg Recall: 0.087719,  count: 57
    2: 84.804230, 575.267700 avg, 0.001000 rate, 5.959159 seconds, 128 images
    iter 总损失 平均损失 学习率 花费时间 参与训练的图片总数
    Region cfg文件中yolo-layer的索引
    Avg IOU 当前迭代中,预测的box与标注的box的平均交并比,越大越好,期望数值为1
    Class 标注物体的分类准确率,越大越好,期望数值为1
    obj 越大越好,期望数值为1
    No obj 越小越好,但不为零
    .5R 以IOU=0.5为阈值时候的recall; recall = 检出的正样本/实际的正样本
    .75R 以IOU=0.75为阈值时候的recall
    count 正样本数目
    Region 82 Avg IOU: 0.798032, Class: 0.559781, Obj: 0.515851, No Obj: 0.006533, .5R: 1.000000, .75R: 1.000000, count: 2
    Region 94 Avg IOU: 0.725307, Class: 0.830518, Obj: 0.506567, No Obj: 0.000680, .5R: 1.000000, .75R: 0.750000, count: 4
    Region 106 Avg IOU: 0.579333, Class: 0.322556, Obj: 0.020537, No Obj: 0.000070, .5R: 1.000000, .75R: 0.000000, count: 2

      以上输出显示了所有训练图片的一个批次(batch),批次大小的划分根据我们在 .cfg 文件中设置的subdivisions参数。

      在我使用的 .cfg 文件中 batch = 64 ,subdivision = 16,所以在训练输出中,训练迭代包含了16组,每组又包含了4张图片,跟设定的batch和subdivision的值一致。 

      但是此处有16*3条信息,每组包含三条信息,分别是:Region 82、Region 94、Region 106。

      三个尺度上预测不同大小的框:

        82卷积层 为最大的预测尺度,使用较大的mask,但是可以预测出较小的物体;

        94卷积层 为中间的预测尺度,使用中等的mask;

        106卷积层为最小的预测尺度,使用较小的mask,可以预测出较大的物体

      每个batch都会有这样一个输出:

    2706: 1.350835, 1.386559 avg, 0.001000 rate, 3.323842 seconds, 173184 images
    batch 总损失 平均损失 学习率       花费时间  参与训练的图片总数 = 2706 * 64

    三. 模型评价

    1)更改配置文件:

      Ⅰ. 模型参数(cfg/yoloc3.cfg):

        batch=1

        subdivisions=1

    注:测评模型时batch、subdivisions必须为1

      Ⅱ.  更改为批处理(example/detector.c):

        ①. 在detector.c中增加头文件:

    #include <unistd.h>  /* Many POSIX functions (but not all, by a large margin) */
    #include <fcntl.h>   /* open(), creat() - and fcntl() */
    

        ②. 在前面添加GetFilename(char *fullname)函数

    #include <unistd.h>
    #include <fcntl.h>
    
    #include "darknet.h"
    #include <sys/stat.h>
    #include <stdio.h>
    #include <time.h>
    #include <sys/types.h>
    
    static int coco_ids[] = {1,2,3,4,5,6,7,8,9,10,11,13,14,15,16,17,18,19,20,21,22,23,24,25,27,28,31,32,33,34,35,
                   36,37,38,39,40,41,42,43,44,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,
                   67,70,72,73,74,75,76,77,78,79,80,81,82,84,85,86,87,88,89,90};
    //产生的文件名与原文件名相同(去除路径和格式)
    char *GetFilename(char *fullname) { int from,to,i; char *newstr,*temp; if(fullname!=NULL){ //if not find dot if((temp=strchr(fullname,'.'))==NULL){ newstr = fullname; } else { from = strlen(fullname) - strlen(temp); to = (temp-fullname); //the first dot's index for (i=from; i<=to; i--){ if (fullname[i]=='.') break;//find the last dot } newstr = (char*)malloc(i+1); strncpy(newstr,fullname,i); *(newstr+i)=0; } } static char name[50] = {""}; char *q = strrchr(newstr,'/') + 1; strncpy(name,q,40); return name; }

        ③. 用下面代码替换detector.c文件的void test_detector函数(注意有3处要改成自己的路径)

    void test_detector(char *datacfg, char *cfgfile, char *weightfile, char *filename, float thresh, float hier_thresh, char *outfile, int fullscreen)
    {
        list *options = read_data_cfg(datacfg);
        char *name_list = option_find_str(options, "names", "data/names.list");
        char **names = get_labels(name_list);
     
        image **alphabet = load_alphabet();
        network *net = load_network(cfgfile, weightfile, 0);
        set_batch_network(net, 1);
        srand(2222222);
        double time;
        char buff[256];
        char *input = buff;
        float nms=.45;
        int i=0;
        while(1){
            if(filename){
                strncpy(input, filename, 256);
                image im = load_image_color(input,0,0);
                image sized = letterbox_image(im, net->w, net->h);
              //image sized = resize_image(im, net->w, net->h);
              //image sized2 = resize_max(im, net->w);
              //image sized = crop_image(sized2, -((net->w - sized2.w)/2), -((net->h - sized2.h)/2), net->w, net->h);
              //resize_network(net, sized.w, sized.h);
                layer l = net->layers[net->n-1];
     
     
                float *X = sized.data;
                time=what_time_is_it_now();
                network_predict(net, X);
                printf("%s: Predicted in %f seconds.\n", input, what_time_is_it_now()-time);
                int nboxes = 0;
                detection *dets = get_network_boxes(net, im.w, im.h, thresh, hier_thresh, 0, 1, &nboxes);
                //printf("%d\n", nboxes);
                //if (nms) do_nms_obj(boxes, probs, l.w*l.h*l.n, l.classes, nms);
                if (nms) do_nms_sort(dets, nboxes, l.classes, nms);
                    draw_detections(im, dets, nboxes, thresh, names, alphabet, l.classes);
                    free_detections(dets, nboxes);
                if(outfile)
                 {
                    save_image(im, outfile);
                 }
                else{
                    save_image(im, "predictions");
    #ifdef OPENCV
                    cvNamedWindow("predictions", CV_WINDOW_NORMAL); 
                    if(fullscreen){
                    cvSetWindowProperty("predictions", CV_WND_PROP_FULLSCREEN, CV_WINDOW_FULLSCREEN);
                    }
                    show_image(im, "predictions");
                    cvWaitKey(0);
                    cvDestroyAllWindows();
    #endif
                }
                free_image(im);
                free_image(sized);
                if (filename) break;
             } 
            else {
                printf("Enter Image Path: ");
                fflush(stdout);
                input = fgets(input, 256, stdin);
                if(!input) return;
                strtok(input, "\n");
       
                list *plist = get_paths(input);
                char **paths = (char **)list_to_array(plist);
                 printf("Start Testing!\n");
                int m = plist->size;
                if(access("/home/xieqi/darknet/data/out_img",0)==-1)  //修改成自己的路径
                {
                  if (mkdir("/home/xieqi/darknet/data/out_img",0777))  //修改成自己的路径
                   {
                     printf("creat file bag failed!!!");
                   }
                }
                for(i = 0; i < m; ++i){
                 char *path = paths[i];
                 image im = load_image_color(path,0,0);
                 image sized = letterbox_image(im, net->w, net->h);
            //image sized = resize_image(im, net->w, net->h);
            //image sized2 = resize_max(im, net->w);
            //image sized = crop_image(sized2, -((net->w - sized2.w)/2), -((net->h - sized2.h)/2), net->w, net->h);
            //resize_network(net, sized.w, sized.h);
            layer l = net->layers[net->n-1];
     
     
            float *X = sized.data;
            time=what_time_is_it_now();
            network_predict(net, X);
            printf("Try Very Hard:");
            printf("%s: Predicted in %f seconds.\n", path, what_time_is_it_now()-time);
            int nboxes = 0;
            detection *dets = get_network_boxes(net, im.w, im.h, thresh, hier_thresh, 0, 1, &nboxes);
            //printf("%d\n", nboxes);
            //if (nms) do_nms_obj(boxes, probs, l.w*l.h*l.n, l.classes, nms);
            if (nms) do_nms_sort(dets, nboxes, l.classes, nms);
            draw_detections(im, dets, nboxes, thresh, names, alphabet, l.classes);
            free_detections(dets, nboxes);
            if(outfile){
                save_image(im, outfile);
            }
            else{
                char b[2048];
                sprintf(b,"/home/xieqi/darknet/data/out_img/%s",GetFilename(path));  //修改成自己的路径
                
                save_image(im, b);
                printf("save %s successfully!\n",GetFilename(path));
    #ifdef OPENCV
                cvNamedWindow("predictions", CV_WINDOW_NORMAL); 
                if(fullscreen){
                    cvSetWindowProperty("predictions", CV_WND_PROP_FULLSCREEN, CV_WINDOW_FULLSCREEN);
                }
                show_image(im, "predictions");
                cvWaitKey(0);
                cvDestroyAllWindows();
    #endif
            }
     
            free_image(im);
            free_image(sized);
            if (filename) break;
            }
          }
        }
    }
    

        ④. validate_detector_recall函数定义和调用改为:

    void validate_detector_recall(char *datacfg, char *cfgfile, char *weightfile)
    validate_detector_recall(datacfg, cfg, weights);

        ⑤.validate_detector_recall内的plistpaths的如下初始化代码:

    list *plist = get_paths("data/voc.2007.test");
    char **paths = (char **)list_to_array(plist);
    

        修改为:

    list *options = read_data_cfg(datacfg);
    char *valid_images = option_find_str(options, "valid", "data/train.list");
    list *plist = get_paths(valid_images);
    char **paths = (char **)list_to_array(plist);

     Ⅲ. 在darknet下重新make

    make clean
    make
    

    2). 测试单张图片

      Ⅰ. ./darknet detector test <data_cfg> <models_cfg> <weights> <image_path>   # 本次测试无opencv支持

      Ⅱ. <models_cfg>文件中batch和subdivisions两项必须为1;

      Ⅲ. 测试时还可以用-thresh-hier选项指定对应参数;

      Ⅳ. 结果都保存在./data/out_img 文件夹下

    ./darknet detector test cfg/voc.data cfg/yolov3-voc.cfg backup/yolov3-voc_final.weights Eminem.jpg        # 测试单张图片

    3). 生成预测结果

      Ⅰ. 批量测试——输出文本检测结果

    ./darknet detector valid cfg/voc.data cfg/yolov3-voc.cfg backup/yolov3-voc_final.weights -i 1 -out ""

        ① ./darknet detector valid <data_cfg> <models_cfg> <weights>;

        ② 结果生成在<data_cfg>的指定的目录下以<out_file>开头的若干文件中,若<data_cfg>没有指定results,那么默认为<darknet_root>/results;
        ③ <models_cfg>文件中batch和subdivisions两项必须为1;

        ④ 若-out 未指定字符串,则在results文件夹下生成comp4_det_test_[类名].txt文件并保存测试结果;

        ⑤ 本次实验在results文件夹下生成 [类名].txt 文件;

      Ⅱ. 批量测试——输出图片检测结果

    ./darknet detector test cfg/voc.data cfg/yolov3-voc.cfg backup/yolov3-voc_final.weights -i 2 #enter

    Enter Image Path: data/voc/2007_test.txt

        ① ./darknet detector test <data_cfg> <models_cfg> <weights>;

        ② <models_cfg>文件中batch和subdivisions两项必须为1;
        ③ 本次实验结果,在<darknet_root>/data/out_img文件夹下(detector.c/test_detector函数更改的路径),文件名以原图片名前6位命名(detector.c/GetFilename函数定义的位数);

    4). 计算recall(执行这个命令需要修改detector.c文件,修改信息请参考“detector.c修改”)

      Ⅰ. ./darknet detector recall <data_cfg> <test_cfg> <weights>;
      Ⅱ. <test_cfg>文件中batch和subdivisions两项必须为1;
      Ⅲ. 输出在stderr里,重定向时请注意;
      Ⅳ. RPs/Img、IOU、Recall都是到当前测试图片的均值;
      Ⅴ. detector.c中对目录处理有错误,可以参照validate_detector对validate_detector_recall最开始几行的处理进行修改;

    ./darknet detector valid cfg/voc.data cfg/yolov3-voc.cfg backup/yolov3-voc_20000.weights

       最后得到的log如下:

    306   746   783       RPs/Img: 21.45  IOU: 75.59%     Recall:95.27%
    307   748   785       RPs/Img: 21.43  IOU: 75.62%     Recall:95.29%
    308   750   787       RPs/Img: 21.42  IOU: 75.59%     Recall:95.30%
    309   752   789       RPs/Img: 21.43  IOU: 75.62%     Recall:95.31%
    310   754   791       RPs/Img: 21.44  IOU: 75.63%     Recall:95.32%
    No. Correct total

      输出的具体格式为:

    Number 处理的第几张图
    Correct

    正确的识别出了多少bbox。这个值算出来的步骤是这样的,丢进网络一张图片,网络会预测出很多bbox,每个bbox都有其置信概率,概率大于threshold的bbox与实际的bbox,也就是labels中txt的内容计算IOU,

    找出IOU最大的bbox,如果这个最大值大于预设的IOU的threshold,那么correct加一

    Total 实际有多少个bbox
    Rps/img 平均每个图片会预测出来多少个bbox
    IOU 预测出的bbox和实际标注的bbox的交集 除以 他们的并集。显然,这个数值越大,说明预测的结果越好
    Recall 召回率, 检测出物体的个数 除以 标注的所有物体个数。通过代码我们也能看出来就是Correct除以Total的值

      error:

      本次实验出现 IOU:inf% —— 交并比 数值爆炸

      打印bbox详细信息(修改detector.c/validate_detector_recall)

            for (j = 0; j < num_labels; ++j) {
                ++total;
                box t = {truth[j].x, truth[j].y, truth[j].w, truth[j].h};
                printf("truth x=%f, y=%f, w=%f,h=%f\n",truth[j].x,truth[j].y,truth[j].w,truth[j].h);//添加代码
                float best_iou = 0;
                for(k = 0; k < l.w*l.h*l.n; ++k){
                    float iou = box_iou(dets[k].bbox, t);
                    if(dets[k].objectness > thresh && iou > best_iou){
                        printf("predict=%f x=%f, y=%f, w=%f,h=%f\n",dets[k].objectness,dets[k].bbox.x,dets[k].bbox.y,dets[k].bbox.w,dets[k].bbox.h);//添加代码
                        best_iou = iou;
                    }
                }

      

      由显示结果可以看出,预测框的中心坐标x和y出现-nan

      解决方法:修改函数

    void validate_detector_recall(char *datacfg, char *cfgfile, char *weightfile)
    {
        ...
        //layer l = net->layers[net->n-1]; //注释掉
        ...
                for(k = 0; k < l.w*l.h*l.n; ++k){ //改为for(k = 0; k < nboxes; ++k){
        ...

    5). 计算AP、mAP

    先使用 3)-Ⅰ计算出验证集结果,再使用py-faster-rcnn下的voc_eval.py计算AP、mAP

      Ⅰ. 下载voc_eval.py到 darknet 的根目录

        voc_eval.py下载地址:https://github.com/rbgirshick/py-faster-rcnn/blob/master/lib/datasets/voc_eval.py

        根据标签修改voc_eval.py的内容

    .xml文件结构中
    
          ①若无pose
    22    #obj_struct['pose'] = obj.find('pose').text                          #    注释掉
    
          ②若无truncated
    23    #obj_struct['truncated'] = int(obj.find('truncated').text)               #    注释掉
    
          ③若无difficult
    24    #obj_struct['difficult'] = int(obj.find('difficult').text)                #    注释掉
    137    #if use_diff:                                         #    注释掉
    138        #difficult = np.array([False for x in R]).astype(np.bool)             #  注释掉
    139    #else:                                             #    注释掉           
    140        #difficult = np.array([x['difficult'] for x in R]).astype(np.bool)       #   注释掉
    142    npos = npos +len(R)                                        #   更改npos = npos + sum(~difficult)
    145    #'difficult': difficult,                                    #    注释掉
    197    #if not R['difficult'][jmax]:                                 #  注释掉

      Ⅱ. 计算单类AP

        ①. 在darknet根目录下新建computer_mAP.py

    #!/home/xieqi/anaconda2/envs/py2.7/bin python2.7
    # -*- coding:utf-8 -*-
    
    from voc_eval import voc_eval
    
    print(voc_eval('/home/xieqi/darknet/results/{}.txt', '/home/xieqi/project/traffic_object_detection/voc_data/Annotations/{}.xml', 
    '/home/xieqi/project/traffic_object_detection/voc_data/ImageSets/Main/test.txt', 'vehicle', '/home/xieqi/project/traffic_object_detection/result/')) # "第一个参数为detector valid 按类别分类后的txt路径" # "第二个参数为验证集对应的xml标签路径" # "第三个为验证集txt文本路径,内容必须是无路径无后缀的图片名" # "第四个为待验证的类别名" # "第五个为pkl文件保存的路径"

        ②. 用python2执行computer_mAP.py

          ① 重复执行,检测其他类别需要删除生成的annots.pkl文件或改变computer_mAP.py中pkl文件保存的路径

          ② 输出两个array(),分别为rec和prec,最后一个数字为单类AP。

      Ⅲ. 计算总mAP

        在darknet根目录下新建computer_all_mAP.py

    #!/home/xieqi/anaconda2/envs/py2.7/bin python2.7
    # -*- coding:utf-8 -*-
    
    from voc_eval import voc_eval
    
    import os
    
    current_path = os.getcwd()
    results_path = current_path+"/results"
    sub_files = os.listdir(results_path)
    
    mAP = []
    for i in range(len(sub_files)):
        class_name = sub_files[i].split(".txt")[0]
        rec, prec, ap = voc_eval('/home/xieqi/darknet/results/{}.txt',
                     '/home/xieqi/project/traffic_object_detection/voc_data/Annotations/{}.xml',
                     '/home/xieqi/project/traffic_object_detection/voc_data/ImageSets/Main/test.txt',
                     class_name, '/home/xieqi/project/traffic_object_detection/result/mAP') print("{} :\t {} ".format(class_name, ap)) mAP.append(ap) mAP = tuple(mAP) print("***************************") print("mAP :\t {}".format( float( sum(mAP)/len(mAP)) )) # results文件夹只能有'类名.txt'文件 # test.txt文件包含所有results中txt文件的图片名(保证是共同的验证集) # 保证最后的输出路径下无pkl文件。 # 上述代码中的ap就是针对输入单一类别计算出的AP。

    四、文件解析

    1). ~detector.c

    #include <unistd.h>
    #include <fcntl.h>
    
    #include "darknet.h"
    #include <sys/stat.h>
    #include <stdio.h>
    #include <time.h>
    #include <sys/types.h>
    
    static int coco_ids[] = {1,2,3,4,5,6,7,8,9,10,11,13,14,15,16,17,18,19,20,21,22,23,24,25,27,28,31,32,33,34,35,36,37,38,39,40,
    41,42,43,44,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,67,70,72,73,74,75,76,77,78,79,80,81,82,84,85,86,87,88,89,90}; char *GetFilename(char *fullname) { int from,to,i; char *newstr,*temp; if(fullname!=NULL){ //if not find dot if((temp=strchr(fullname,'.'))==NULL){ newstr = fullname; } else { from = strlen(fullname) - strlen(temp); to = (temp-fullname); //the first dot's index for (i=from; i<=to; i--){ if (fullname[i]=='.') break;//find the last dot } newstr = (char*)malloc(i+1); strncpy(newstr,fullname,i); *(newstr+i)=0; } } static char name[50] = {""}; char *q = strrchr(newstr,'/') + 1; strncpy(name,q,40); return name; } void train_detector(char *datacfg, char *cfgfile, char *weightfile, int *gpus, int ngpus, int clear) { list *options = read_data_cfg(datacfg); //解析data文件,用自定义链表options存储训练集基本信息,函数位于option_list.c char *train_images = option_find_str(options, "train", "data/train.list"); //从options中找训练集 char *backup_directory = option_find_str(options, "backup", "/backup/"); //从options中找backup路径 srand(time(0)); //初始化随机种子数 char *base = basecfg(cfgfile); //此函数位于utils.c,返回cfg文件不带后缀的名字 printf("%s\n", base); float avg_loss = -1; network **nets = calloc(ngpus, sizeof(network)); srand(time(0)); int seed = rand(); int i; for(i = 0; i < ngpus; ++i){ srand(seed); #ifdef GPU cuda_set_device(gpus[i]); #endif nets[i] = load_network(cfgfile, weightfile, clear); nets[i]->learning_rate *= ngpus; } srand(time(0)); network *net = nets[0]; int imgs = net->batch * net->subdivisions * ngpus; printf("Learning Rate: %g, Momentum: %g, Decay: %g\n", net->learning_rate, net->momentum, net->decay); data train, buffer; layer l = net->layers[net->n - 1]; int classes = l.classes; float jitter = l.jitter; list *plist = get_paths(train_images); //int N = plist->size; char **paths = (char **)list_to_array(plist); load_args args = get_base_args(net); args.coords = l.coords; args.paths = paths; args.n = imgs; args.m = plist->size; args.classes = classes; args.jitter = jitter; args.num_boxes = l.max_boxes; args.d = &buffer; args.type = DETECTION_DATA; //args.type = INSTANCE_DATA; args.threads = 64; pthread_t load_thread = load_data(args); double time; int count = 0; //while(i*imgs < N*120){ while(get_current_batch(net) < net->max_batches){ if(l.random && count++%10 == 0){ printf("Resizing\n"); int dim = (rand() % 10 + 10) * 32; if (get_current_batch(net)+200 > net->max_batches) dim = 608; //int dim = (rand() % 4 + 16) * 32; printf("%d\n", dim); args.w = dim; args.h = dim; pthread_join(load_thread, 0); train = buffer; free_data(train); load_thread = load_data(args); #pragma omp parallel for for(i = 0; i < ngpus; ++i){ resize_network(nets[i], dim, dim); } net = nets[0]; } time=what_time_is_it_now(); pthread_join(load_thread, 0); train = buffer; load_thread = load_data(args); /* int k; for(k = 0; k < l.max_boxes; ++k){ box b = float_to_box(train.y.vals[10] + 1 + k*5); if(!b.x) break; printf("loaded: %f %f %f %f\n", b.x, b.y, b.w, b.h); } */ /* int zz; for(zz = 0; zz < train.X.cols; ++zz){ image im = float_to_image(net->w, net->h, 3, train.X.vals[zz]); int k; for(k = 0; k < l.max_boxes; ++k){ box b = float_to_box(train.y.vals[zz] + k*5, 1); printf("%f %f %f %f\n", b.x, b.y, b.w, b.h); draw_bbox(im, b, 1, 1,0,0); } show_image(im, "truth11"); cvWaitKey(0); save_image(im, "truth11"); } */ printf("Loaded: %lf seconds\n", what_time_is_it_now()-time); time=what_time_is_it_now(); float loss = 0; #ifdef GPU if(ngpus == 1){ loss = train_network(net, train); } else { loss = train_networks(nets, ngpus, train, 4); } #else loss = train_network(net, train); #endif if (avg_loss < 0) avg_loss = loss; avg_loss = avg_loss*.9 + loss*.1; i = get_current_batch(net); printf("%ld: %f, %f avg, %f rate, %lf seconds, %d images\n", get_current_batch(net), loss, avg_loss, get_current_rate(net), what_time_is_it_now()-time, i*imgs); if(i%100==0){ #ifdef GPU if(ngpus != 1) sync_nets(nets, ngpus, 0); #endif char buff[256]; sprintf(buff, "%s/%s.backup", backup_directory, base); save_weights(net, buff); } if(i%10000==0 || (i < 1000 && i%100 == 0)){ #ifdef GPU if(ngpus != 1) sync_nets(nets, ngpus, 0); #endif char buff[256]; sprintf(buff, "%s/%s_%d.weights", backup_directory, base, i); save_weights(net, buff); } free_data(train); } #ifdef GPU if(ngpus != 1) sync_nets(nets, ngpus, 0); #endif char buff[256]; sprintf(buff, "%s/%s_final.weights", backup_directory, base); save_weights(net, buff); } static int get_coco_image_id(char *filename) { char *p = strrchr(filename, '/'); char *c = strrchr(filename, '_'); if(c) p = c; return atoi(p+1); } static void print_cocos(FILE *fp, char *image_path, detection *dets, int num_boxes, int classes, int w, int h) { int i, j; int image_id = get_coco_image_id(image_path); for(i = 0; i < num_boxes; ++i){ float xmin = dets[i].bbox.x - dets[i].bbox.w/2.; float xmax = dets[i].bbox.x + dets[i].bbox.w/2.; float ymin = dets[i].bbox.y - dets[i].bbox.h/2.; float ymax = dets[i].bbox.y + dets[i].bbox.h/2.; if (xmin < 0) xmin = 0; if (ymin < 0) ymin = 0; if (xmax > w) xmax = w; if (ymax > h) ymax = h; float bx = xmin; float by = ymin; float bw = xmax - xmin; float bh = ymax - ymin; for(j = 0; j < classes; ++j){ if (dets[i].prob[j]) fprintf(fp, "{\"image_id\":%d, \"category_id\":%d, \"bbox\":[%f, %f, %f, %f], \"score\":%f},\n", image_id, coco_ids[j], bx, by, bw, bh, dets[i].prob[j]); } } } void print_detector_detections(FILE **fps, char *id, detection *dets, int total, int classes, int w, int h) { int i, j; for(i = 0; i < total; ++i){ float xmin = dets[i].bbox.x - dets[i].bbox.w/2. + 1; float xmax = dets[i].bbox.x + dets[i].bbox.w/2. + 1; float ymin = dets[i].bbox.y - dets[i].bbox.h/2. + 1; float ymax = dets[i].bbox.y + dets[i].bbox.h/2. + 1; if (xmin < 1) xmin = 1; if (ymin < 1) ymin = 1; if (xmax > w) xmax = w; if (ymax > h) ymax = h; for(j = 0; j < classes; ++j){ if (dets[i].prob[j]) fprintf(fps[j], "%s %f %f %f %f %f\n", id, dets[i].prob[j], xmin, ymin, xmax, ymax); } } } void print_imagenet_detections(FILE *fp, int id, detection *dets, int total, int classes, int w, int h) { int i, j; for(i = 0; i < total; ++i){ float xmin = dets[i].bbox.x - dets[i].bbox.w/2.; float xmax = dets[i].bbox.x + dets[i].bbox.w/2.; float ymin = dets[i].bbox.y - dets[i].bbox.h/2.; float ymax = dets[i].bbox.y + dets[i].bbox.h/2.; if (xmin < 0) xmin = 0; if (ymin < 0) ymin = 0; if (xmax > w) xmax = w; if (ymax > h) ymax = h; for(j = 0; j < classes; ++j){ int class = j; if (dets[i].prob[class]) fprintf(fp, "%d %d %f %f %f %f %f\n", id, j+1, dets[i].prob[class], xmin, ymin, xmax, ymax); } } } void validate_detector_flip(char *datacfg, char *cfgfile, char *weightfile, char *outfile) { int j; list *options = read_data_cfg(datacfg); char *valid_images = option_find_str(options, "valid", "data/train.list"); char *name_list = option_find_str(options, "names", "data/names.list"); char *prefix = option_find_str(options, "results", "results"); char **names = get_labels(name_list); char *mapf = option_find_str(options, "map", 0); int *map = 0; if (mapf) map = read_map(mapf); network *net = load_network(cfgfile, weightfile, 0); set_batch_network(net, 2); fprintf(stderr, "Learning Rate: %g, Momentum: %g, Decay: %g\n", net->learning_rate, net->momentum, net->decay); srand(time(0)); list *plist = get_paths(valid_images); char **paths = (char **)list_to_array(plist); layer l = net->layers[net->n-1]; int classes = l.classes; char buff[1024]; char *type = option_find_str(options, "eval", "voc"); FILE *fp = 0; FILE **fps = 0; int coco = 0; int imagenet = 0; if(0==strcmp(type, "coco")){ if(!outfile) outfile = "coco_results"; snprintf(buff, 1024, "%s/%s.json", prefix, outfile); fp = fopen(buff, "w"); fprintf(fp, "[\n"); coco = 1; } else if(0==strcmp(type, "imagenet")){ if(!outfile) outfile = "imagenet-detection"; snprintf(buff, 1024, "%s/%s.txt", prefix, outfile); fp = fopen(buff, "w"); imagenet = 1; classes = 200; } else { if(!outfile) outfile = "comp4_det_test_"; fps = calloc(classes, sizeof(FILE *)); for(j = 0; j < classes; ++j){ snprintf(buff, 1024, "%s/%s%s.txt", prefix, outfile, names[j]); fps[j] = fopen(buff, "w"); } } int m = plist->size; int i=0; int t; float thresh = .005; float nms = .45; int nthreads = 4; image *val = calloc(nthreads, sizeof(image)); image *val_resized = calloc(nthreads, sizeof(image)); image *buf = calloc(nthreads, sizeof(image)); image *buf_resized = calloc(nthreads, sizeof(image)); pthread_t *thr = calloc(nthreads, sizeof(pthread_t)); image input = make_image(net->w, net->h, net->c*2); load_args args = {0}; args.w = net->w; args.h = net->h; //args.type = IMAGE_DATA; args.type = LETTERBOX_DATA; for(t = 0; t < nthreads; ++t){ args.path = paths[i+t]; args.im = &buf[t]; args.resized = &buf_resized[t]; thr[t] = load_data_in_thread(args); } double start = what_time_is_it_now(); for(i = nthreads; i < m+nthreads; i += nthreads){ fprintf(stderr, "%d\n", i); for(t = 0; t < nthreads && i+t-nthreads < m; ++t){ pthread_join(thr[t], 0); val[t] = buf[t]; val_resized[t] = buf_resized[t]; } for(t = 0; t < nthreads && i+t < m; ++t){ args.path = paths[i+t]; args.im = &buf[t]; args.resized = &buf_resized[t]; thr[t] = load_data_in_thread(args); } for(t = 0; t < nthreads && i+t-nthreads < m; ++t){ char *path = paths[i+t-nthreads]; char *id = basecfg(path); copy_cpu(net->w*net->h*net->c, val_resized[t].data, 1, input.data, 1); flip_image(val_resized[t]); copy_cpu(net->w*net->h*net->c, val_resized[t].data, 1, input.data + net->w*net->h*net->c, 1); network_predict(net, input.data); int w = val[t].w; int h = val[t].h; int num = 0; detection *dets = get_network_boxes(net, w, h, thresh, .5, map, 0, &num); if (nms) do_nms_sort(dets, num, classes, nms); if (coco){ print_cocos(fp, path, dets, num, classes, w, h); } else if (imagenet){ print_imagenet_detections(fp, i+t-nthreads+1, dets, num, classes, w, h); } else { print_detector_detections(fps, id, dets, num, classes, w, h); } free_detections(dets, num); free(id); free_image(val[t]); free_image(val_resized[t]); } } for(j = 0; j < classes; ++j){ if(fps) fclose(fps[j]); } if(coco){ fseek(fp, -2, SEEK_CUR); fprintf(fp, "\n]\n"); fclose(fp); } fprintf(stderr, "Total Detection Time: %f Seconds\n", what_time_is_it_now() - start); } void validate_detector(char *datacfg, char *cfgfile, char *weightfile, char *outfile) { int j; list *options = read_data_cfg(datacfg); char *valid_images = option_find_str(options, "valid", "/home/xieqi/project/traffic_object_detection/voc_data/test.txt"); char *name_list = option_find_str(options, "names", "data/53w.names"); char *prefix = option_find_str(options, "results", "results"); char **names = get_labels(name_list); char *mapf = option_find_str(options, "map", 0); int *map = 0; if (mapf) map = read_map(mapf); network *net = load_network(cfgfile, weightfile, 0); set_batch_network(net, 1); fprintf(stderr, "Learning Rate: %g, Momentum: %g, Decay: %g\n", net->learning_rate, net->momentum, net->decay); srand(time(0)); list *plist = get_paths(valid_images); char **paths = (char **)list_to_array(plist); layer l = net->layers[net->n-1]; int classes = l.classes; char buff[1024]; char *type = option_find_str(options, "eval", "voc"); FILE *fp = 0; FILE **fps = 0; int coco = 0; int imagenet = 0; if(0==strcmp(type, "coco")){ if(!outfile) outfile = "coco_results"; snprintf(buff, 1024, "%s/%s.json", prefix, outfile); fp = fopen(buff, "w"); fprintf(fp, "[\n"); coco = 1; } else if(0==strcmp(type, "imagenet")){ if(!outfile) outfile = "imagenet-detection"; snprintf(buff, 1024, "%s/%s.txt", prefix, outfile); fp = fopen(buff, "w"); imagenet = 1; classes = 200; } else { if(!outfile) outfile = "comp4_det_test_"; fps = calloc(classes, sizeof(FILE *)); for(j = 0; j < classes; ++j){ snprintf(buff, 1024, "%s/%s%s.txt", prefix, outfile, names[j]); fps[j] = fopen(buff, "w"); } } int m = plist->size; int i=0; int t; float thresh = .005; float nms = .45; int nthreads = 4; image *val = calloc(nthreads, sizeof(image)); image *val_resized = calloc(nthreads, sizeof(image)); image *buf = calloc(nthreads, sizeof(image)); image *buf_resized = calloc(nthreads, sizeof(image)); pthread_t *thr = calloc(nthreads, sizeof(pthread_t)); load_args args = {0}; args.w = net->w; args.h = net->h; //args.type = IMAGE_DATA; args.type = LETTERBOX_DATA; for(t = 0; t < nthreads; ++t){ args.path = paths[i+t]; args.im = &buf[t]; args.resized = &buf_resized[t]; thr[t] = load_data_in_thread(args); } double start = what_time_is_it_now(); for(i = nthreads; i < m+nthreads; i += nthreads){ fprintf(stderr, "%d\n", i); for(t = 0; t < nthreads && i+t-nthreads < m; ++t){ pthread_join(thr[t], 0); val[t] = buf[t]; val_resized[t] = buf_resized[t]; } for(t = 0; t < nthreads && i+t < m; ++t){ args.path = paths[i+t]; args.im = &buf[t]; args.resized = &buf_resized[t]; thr[t] = load_data_in_thread(args); } for(t = 0; t < nthreads && i+t-nthreads < m; ++t){ char *path = paths[i+t-nthreads]; char *id = basecfg(path); float *X = val_resized[t].data; network_predict(net, X); int w = val[t].w; int h = val[t].h; int nboxes = 0; detection *dets = get_network_boxes(net, w, h, thresh, .5, map, 0, &nboxes); if (nms) do_nms_sort(dets, nboxes, classes, nms); if (coco){ print_cocos(fp, path, dets, nboxes, classes, w, h); } else if (imagenet){ print_imagenet_detections(fp, i+t-nthreads+1, dets, nboxes, classes, w, h); } else { print_detector_detections(fps, id, dets, nboxes, classes, w, h); } free_detections(dets, nboxes); free(id); free_image(val[t]); free_image(val_resized[t]); } } for(j = 0; j < classes; ++j){ if(fps) fclose(fps[j]); } if(coco){ fseek(fp, -2, SEEK_CUR); fprintf(fp, "\n]\n"); fclose(fp); } fprintf(stderr, "Total Detection Time: %f Seconds\n", what_time_is_it_now() - start); } void validate_detector_recall(char *datacfg, char *cfgfile, char *weightfile) { network *net = load_network(cfgfile, weightfile, 0); set_batch_network(net, 1); fprintf(stderr, "Learning Rate: %g, Momentum: %g, Decay: %g\n", net->learning_rate, net->momentum, net->decay); srand(time(0)); list *options = read_data_cfg(datacfg); char *valid_images = option_find_str(options, "valid", "/home/xieqi/project/traffic_object_detection/voc_data/test.txt"); list *plist = get_paths(valid_images); char **paths = (char **)list_to_array(plist); //layer l = net->layers[net->n-1]; int j, k; int m = plist->size; //测试的图片总数 int i=0; float thresh = .001; float iou_thresh = .5; float nms = .4; int total = 0; //实际有多少个bbox int correct = 0; //正确识别出了多少个bbox int proposals = 0; //测试集预测的bbox总数 float avg_iou = 0; //printf("l.w*l.h*l.n = %d\n",l.w*l.h*l.n); for(i = 0; i < m; ++i){ char *path = paths[i]; image orig = load_image_color(path, 0, 0); image sized = resize_image(orig, net->w, net->h); char *id = basecfg(path); network_predict(net, sized.data); int nboxes = 0; detection *dets = get_network_boxes(net, sized.w, sized.h, thresh, .5, 0, 1, &nboxes); if (nms) do_nms_obj(dets, nboxes, 1, nms); char labelpath[4096]; find_replace(path, "images", "labels", labelpath); find_replace(labelpath, "JPEGImages", "labels", labelpath); find_replace(labelpath, ".jpg", ".txt", labelpath); find_replace(labelpath, ".JPEG", ".txt", labelpath); int num_labels = 0; //测试集实际的标注框数量 box_label *truth = read_boxes(labelpath, &num_labels); for(k = 0; k < nboxes; ++k){ if(dets[k].objectness > thresh){ ++proposals; } } for (j = 0; j < num_labels; ++j) { ++total; box t = {truth[j].x, truth[j].y, truth[j].w, truth[j].h}; //printf("truth x=%f, y=%f, w=%f,h=%f\n",truth[j].x,truth[j].y,truth[j].w,truth[j].h); float best_iou = 0; //对每一个标注的框 for(k = 0; k < nboxes; ++k){ //for(k = 0; k < l.w*l.h*l.n; ++k){ float iou = box_iou(dets[k].bbox, t); //printf("predict=%f iou=%f x=%f, y=%f, w=%f,h=%f\n",dets[k].objectness,iou,dets[k].bbox.x,dets[k].bbox.y,dets[k].bbox.w,dets[k].bbox.h); if(dets[k].objectness > thresh && iou > best_iou){ best_iou = iou; } } //printf("best_iou=%f\n",best_iou); avg_iou += best_iou; if(best_iou > iou_thresh){ ++correct; } } fprintf(stderr, "%5d %5d %5d\tRPs/Img: %.2f\tIOU: %.2f%%\tRecall:%.2f%%\n", i, correct, total, (float)proposals/(i+1), avg_iou*100/total, 100.*correct/total); free(id); free_image(orig); free_image(sized); } } void test_detector(char *datacfg, char *cfgfile, char *weightfile, char *filename, float thresh, float hier_thresh, char *outfile, int fullscreen) { list *options = read_data_cfg(datacfg); //options存储分类的标签等基本训练信息 char *name_list = option_find_str(options, "names", "data/names.list"); //抽取标签名称 char **names = get_labels(name_list); image **alphabet = load_alphabet(); //加载位于data/labels下的字符图片,用于显示矩形框名称 network *net = load_network(cfgfile, weightfile, 0); //用netweork.h中自定义的network结构体存储模型文件,函数位于parser.c set_batch_network(net, 1); srand(2222222); double start_time; double end_time; double img_time; double sum_time=0.0; char buff[256]; char *input = buff; float nms=.45; int i=0; while(1){ //读取结构对应的权重文件 if(filename){ strncpy(input, filename, 256); image im = load_image_color(input,0,0); image sized = letterbox_image(im, net->w, net->h); //输入图片大小经过resize至输入大小 //image sized = resize_image(im, net->w, net->h); //image sized2 = resize_max(im, net->w); //image sized = crop_image(sized2, -((net->w - sized2.w)/2), -((net->h - sized2.h)/2), net->w, net->h); //resize_network(net, sized.w, sized.h); layer l = net->layers[net->n-1]; float *X = sized.data; //X指向图片的data元素,即图片像素 start_time=what_time_is_it_now(); network_predict(net, X); //network_predict函数负责预测当前图片的数据X end_time=what_time_is_it_now(); img_time= end_time - start_time; printf("%s: Predicted in %f seconds.\n", input, img_time); int nboxes = 0; detection *dets = get_network_boxes(net, im.w, im.h, thresh, hier_thresh, 0, 1, &nboxes); //printf("%d\n", nboxes); //if (nms) do_nms_obj(boxes, probs, l.w*l.h*l.n, l.classes, nms); if (nms) do_nms_sort(dets, nboxes, l.classes, nms); draw_detections(im, dets, nboxes, thresh, names, alphabet, l.classes); free_detections(dets, nboxes); if(outfile) { save_image(im, outfile); } else{ //save_image(im, "predictions"); char image[2048]; sprintf(image,"./data/predict/%s",GetFilename(filename)); save_image(im,image); printf("predict %s successfully!\n",GetFilename(filename)); #ifdef OPENCV cvNamedWindow("predictions", CV_WINDOW_NORMAL); if(fullscreen){ cvSetWindowProperty("predictions", CV_WND_PROP_FULLSCREEN, CV_WINDOW_FULLSCREEN); } show_image(im, "predictions"); cvWaitKey(0); cvDestroyAllWindows(); #endif } free_image(im); free_image(sized); if (filename) break; } else { printf("Enter Image Path: "); fflush(stdout); input = fgets(input, 256, stdin); if(!input) return; strtok(input, "\n"); list *plist = get_paths(input); char **paths = (char **)list_to_array(plist); printf("Start Testing!\n"); int m = plist->size; if(access("/home/xieqi/darknet/data/out_img",0)==-1)//修改成自己的路径 { if (mkdir("/home/xieqi/darknet/data/out_img",0777))//修改成自己的路径 { printf("creat file bag failed!!!"); } } for(i = 0; i < m; ++i){ char *path = paths[i]; image im = load_image_color(path,0,0); image sized = letterbox_image(im, net->w, net->h); //输入图片大小经过resize至输入大小 //image sized = resize_image(im, net->w, net->h); //image sized2 = resize_max(im, net->w); //image sized = crop_image(sized2, -((net->w - sized2.w)/2), -((net->h - sized2.h)/2), net->w, net->h); //resize_network(net, sized.w, sized.h); layer l = net->layers[net->n-1]; float *X = sized.data; //X指向图片的data元素,即图片像素 start_time = what_time_is_it_now(); network_predict(net, X); //network_predict函数负责预测当前图片的数据X end_time = what_time_is_it_now(); img_time = end_time - start_time; sum_time = sum_time+img_time; printf("Try Very Hard:"); printf("%s: Predicted in %f seconds.\n", path, img_time); int nboxes = 0; detection *dets = get_network_boxes(net, im.w, im.h, thresh, hier_thresh, 0, 1, &nboxes); //printf("%d\n", nboxes); //if (nms) do_nms_obj(boxes, probs, l.w*l.h*l.n, l.classes, nms); if (nms) do_nms_sort(dets, nboxes, l.classes, nms); draw_detections(im, dets, nboxes, thresh, names, alphabet, l.classes); free_detections(dets, nboxes); if(outfile){ save_image(im, outfile); } else{ char b[2048]; sprintf(b,"/home/xieqi/darknet/data/out_img/%s",GetFilename(path));//修改成自己的路径 save_image(im, b); printf("save %s successfully!\n",GetFilename(path)); #ifdef OPENCV cvNamedWindow("predictions", CV_WINDOW_NORMAL); if(fullscreen){ cvSetWindowProperty("predictions", CV_WND_PROP_FULLSCREEN, CV_WINDOW_FULLSCREEN); } show_image(im, "predictions"); cvWaitKey(0); cvDestroyAllWindows(); #endif } free_image(im); free_image(sized); if (filename) break; } printf("fps: %.2f totall image %d\n",(float)m/sum_time,m); } } } /* void test_detector(char *datacfg, char *cfgfile, char *weightfile, char *filename, float thresh, float hier_thresh, char *outfile, int fullscreen) { list *options = read_data_cfg(datacfg); char *name_list = option_find_str(options, "names", "data/names.list"); char **names = get_labels(name_list); image **alphabet = load_alphabet(); network *net = load_network(cfgfile, weightfile, 0); set_batch_network(net, 1); srand(2222222); double time; char buff[256]; char *input = buff; float nms=.45; while(1){ if(filename){ strncpy(input, filename, 256); } else { printf("Enter Image Path: "); fflush(stdout); input = fgets(input, 256, stdin); if(!input) return; strtok(input, "\n"); } image im = load_image_color(input,0,0); image sized = letterbox_image(im, net->w, net->h); //image sized = resize_image(im, net->w, net->h); //image sized2 = resize_max(im, net->w); //image sized = crop_image(sized2, -((net->w - sized2.w)/2), -((net->h - sized2.h)/2), net->w, net->h); //resize_network(net, sized.w, sized.h); layer l = net->layers[net->n-1]; float *X = sized.data; time=what_time_is_it_now(); network_predict(net, X); printf("%s: Predicted in %f seconds.\n", input, what_time_is_it_now()-time); int nboxes = 0; detection *dets = get_network_boxes(net, im.w, im.h, thresh, hier_thresh, 0, 1, &nboxes); //printf("%d\n", nboxes); //if (nms) do_nms_obj(boxes, probs, l.w*l.h*l.n, l.classes, nms); if (nms) do_nms_sort(dets, nboxes, l.classes, nms); draw_detections(im, dets, nboxes, thresh, names, alphabet, l.classes); free_detections(dets, nboxes); if(outfile){ save_image(im, outfile); } else{ save_image(im, "predictions"); #ifdef OPENCV make_window("predictions", 512, 512, 0); show_image(im, "predictions", 0); #endif } free_image(im); free_image(sized); if (filename) break; } } void censor_detector(char *datacfg, char *cfgfile, char *weightfile, int cam_index, const char *filename, int class, float thresh, int skip) { #ifdef OPENCV char *base = basecfg(cfgfile); network *net = load_network(cfgfile, weightfile, 0); set_batch_network(net, 1); srand(2222222); CvCapture * cap; int w = 1280; int h = 720; if(filename){ cap = cvCaptureFromFile(filename); }else{ cap = cvCaptureFromCAM(cam_index); } if(w){ cvSetCaptureProperty(cap, CV_CAP_PROP_FRAME_WIDTH, w); } if(h){ cvSetCaptureProperty(cap, CV_CAP_PROP_FRAME_HEIGHT, h); } if(!cap) error("Couldn't connect to webcam.\n"); cvNamedWindow(base, CV_WINDOW_NORMAL); cvResizeWindow(base, 512, 512); float fps = 0; int i; float nms = .45; while(1){ image in = get_image_from_stream(cap); //image in_s = resize_image(in, net->w, net->h); image in_s = letterbox_image(in, net->w, net->h); layer l = net->layers[net->n-1]; float *X = in_s.data; network_predict(net, X); int nboxes = 0; detection *dets = get_network_boxes(net, in.w, in.h, thresh, 0, 0, 0, &nboxes); //if (nms) do_nms_obj(boxes, probs, l.w*l.h*l.n, l.classes, nms); if (nms) do_nms_sort(dets, nboxes, l.classes, nms); for(i = 0; i < nboxes; ++i){ if(dets[i].prob[class] > thresh){ box b = dets[i].bbox; int left = b.x-b.w/2.; int top = b.y-b.h/2.; censor_image(in, left, top, b.w, b.h); } } show_image(in, base); cvWaitKey(10); free_detections(dets, nboxes); free_image(in_s); free_image(in); float curr = 0; fps = .9*fps + .1*curr; for(i = 0; i < skip; ++i){ image in = get_image_from_stream(cap); free_image(in); } } #endif } void extract_detector(char *datacfg, char *cfgfile, char *weightfile, int cam_index, const char *filename, int class, float thresh, int skip) { #ifdef OPENCV char *base = basecfg(cfgfile); network *net = load_network(cfgfile, weightfile, 0); set_batch_network(net, 1); srand(2222222); CvCapture * cap; int w = 1280; int h = 720; if(filename){ cap = cvCaptureFromFile(filename); }else{ cap = cvCaptureFromCAM(cam_index); } if(w){ cvSetCaptureProperty(cap, CV_CAP_PROP_FRAME_WIDTH, w); } if(h){ cvSetCaptureProperty(cap, CV_CAP_PROP_FRAME_HEIGHT, h); } if(!cap) error("Couldn't connect to webcam.\n"); cvNamedWindow(base, CV_WINDOW_NORMAL); cvResizeWindow(base, 512, 512); float fps = 0; int i; int count = 0; float nms = .45; while(1){ image in = get_image_from_stream(cap); //image in_s = resize_image(in, net->w, net->h); image in_s = letterbox_image(in, net->w, net->h); layer l = net->layers[net->n-1]; show_image(in, base); int nboxes = 0; float *X = in_s.data; network_predict(net, X); detection *dets = get_network_boxes(net, in.w, in.h, thresh, 0, 0, 1, &nboxes); //if (nms) do_nms_obj(boxes, probs, l.w*l.h*l.n, l.classes, nms); if (nms) do_nms_sort(dets, nboxes, l.classes, nms); for(i = 0; i < nboxes; ++i){ if(dets[i].prob[class] > thresh){ box b = dets[i].bbox; int size = b.w*in.w > b.h*in.h ? b.w*in.w : b.h*in.h; int dx = b.x*in.w-size/2.; int dy = b.y*in.h-size/2.; image bim = crop_image(in, dx, dy, size, size); char buff[2048]; sprintf(buff, "results/extract/%07d", count); ++count; save_image(bim, buff); free_image(bim); } } free_detections(dets, nboxes); free_image(in_s); free_image(in); float curr = 0; fps = .9*fps + .1*curr; for(i = 0; i < skip; ++i){ image in = get_image_from_stream(cap); free_image(in); } } #endif } */ /* void network_detect(network *net, image im, float thresh, float hier_thresh, float nms, detection *dets) { network_predict_image(net, im); layer l = net->layers[net->n-1]; int nboxes = num_boxes(net); fill_network_boxes(net, im.w, im.h, thresh, hier_thresh, 0, 0, dets); if (nms) do_nms_sort(dets, nboxes, l.classes, nms); } */ // ./darknet [xxx]中如果命令如果第二个xxx参数是detector,则调用这个 void run_detector(int argc, char **argv) { char *prefix = find_char_arg(argc, argv, "-prefix", 0); //寻找是否有参数prefix, 默认参数0,argv为二维数组,存储了参数字符串 float thresh = find_float_arg(argc, argv, "-thresh", .5); //寻找是否有参数thresh,thresh为输出的阈值,默认参数0.24 float hier_thresh = find_float_arg(argc, argv, "-hier", .5); //寻找是否有参数hier_thresh,默认0.5 int cam_index = find_int_arg(argc, argv, "-c", 0); //寻找是否有参数c,默认0 int frame_skip = find_int_arg(argc, argv, "-s", 0); //寻找是否有参数s,默认0 int avg = find_int_arg(argc, argv, "-avg", 3); //如果输入参数小于4个,输出正确语法如何使用 //printf 等价于 fprintf(stdout, ...),这里stderr和stdout默认输出设备都是屏幕,但是stderr一般指标准出错输入设备 if(argc < 4){ fprintf(stderr, "usage: %s %s [train/test/valid] [cfg] [weights (optional)]\n", argv[0], argv[1]); return; } char *gpu_list = find_char_arg(argc, argv, "-gpus", 0); //寻找是否有参数gpus,默认0 char *outfile = find_char_arg(argc, argv, "-out", 0); //检查是否指定GPU运算,默认0 int *gpus = 0; int gpu = 0; int ngpus = 0; if(gpu_list){ printf("%s\n", gpu_list); int len = strlen(gpu_list); ngpus = 1; int i; for(i = 0; i < len; ++i){ if (gpu_list[i] == ',') ++ngpus; } gpus = calloc(ngpus, sizeof(int)); for(i = 0; i < ngpus; ++i){ gpus[i] = atoi(gpu_list); gpu_list = strchr(gpu_list, ',')+1; } } else { gpu = gpu_index; gpus = &gpu; ngpus = 1; } int clear = find_arg(argc, argv, "-clear"); //检查clear参数 int fullscreen = find_arg(argc, argv, "-fullscreen"); int width = find_int_arg(argc, argv, "-w", 0); int height = find_int_arg(argc, argv, "-h", 0); int fps = find_int_arg(argc, argv, "-fps", 0); //int class = find_int_arg(argc, argv, "-class", 0); char *datacfg = argv[3]; //存data文件路径 char *cfg = argv[4]; //存cfg文件路径 char *weights = (argc > 5) ? argv[5] : 0; //存weight文件路径 char *filename = (argc > 6) ? argv[6]: 0; //存待检测文件路径 //根据第三个参数的内容,调用不同的函数,并传入之前解析的参数 if(0==strcmp(argv[2], "test")) test_detector(datacfg, cfg, weights, filename, thresh, hier_thresh, outfile, fullscreen); else if(0==strcmp(argv[2], "train")) train_detector(datacfg, cfg, weights, gpus, ngpus, clear); else if(0==strcmp(argv[2], "valid")) validate_detector(datacfg, cfg, weights, outfile); else if(0==strcmp(argv[2], "valid2")) validate_detector_flip(datacfg, cfg, weights, outfile); else if(0==strcmp(argv[2], "recall")) validate_detector_recall(datacfg, cfg, weights); else if(0==strcmp(argv[2], "demo")) { list *options = read_data_cfg(datacfg); int classes = option_find_int(options, "classes", 20); char *name_list = option_find_str(options, "names", "data/names.list"); char **names = get_labels(name_list); demo(cfg, weights, thresh, cam_index, filename, names, classes, frame_skip, prefix, avg, hier_thresh, width, height, fps, fullscreen); } //else if(0==strcmp(argv[2], "extract")) extract_detector(datacfg, cfg, weights, cam_index, filename, class, thresh, frame_skip); //else if(0==strcmp(argv[2], "censor")) censor_detector(datacfg, cfg, weights, cam_index, filename, class, thresh, frame_skip); }

    """
    Note:If during training you see nan values for avg (loss) field - then training goes wrong, 
    but if nan is in some other lines - then training goes well.

    When should I stop training:
    When you see that average loss 0.xxxxxx avg no longer decreases at many iterations then you should stop training.
    Once training is stopped, you should take some of last .weights-files from darknet\build\darknet\x64\backup and choose the best of them.

    Overfitting - is case when you can detect objects on images from training-dataset, 
    but can't detect objects on any others images. You should get weights from Early Stopping Point.

    IoU (intersect of union) - average instersect of union of objects and detections for a certain threshold = 0.24

    How to improve object detection:
    Before training:
    set flag random=1 in your .cfg-file - it will increase precision by training Yolo for different resolutions.
    increase network resolution in your .cfg-file (height=608, width=608 or any value multiple of 32) - it will increase precision.
    recalculate anchors for your dataset for width and height from cfg-file: 
    darknet.exe detector calc_anchors data/obj.data -num_of_clusters 9 -width 416 -height 416 then set the same 9 anchors in each of 3 [yolo]-layers in your cfg-file
    设置锚点

    desirable that your training dataset include images with objects at diffrent: 
    scales, rotations, lightings, from different sides, on different backgrounds
    样本特点尽量多样化,亮度,旋转,背景,目标位置,尺寸

    desirable that your training dataset include images with non-labeled objects that you do not want to detect - negative samples without bounded box (empty .txt files)
    可以添加没有标注框的图片和其空的txt文件,作为negative数据

    for training with a large number of objects in each image, add the parameter max=200 or higher value in the last layer [region] in your cfg-file

    to speedup training (with decreasing detection accuracy) do Fine-Tuning instead of Transfer-Learning, 
    set param stopbackward=1 in one of the penultimate convolutional layers before the 1-st [yolo]-layer, 
    for example here: https://github.com/AlexeyAB/darknet/blob/0039fd26786ab5f71d5af725fc18b3f521e7acfd/cfg/yolov3.cfg#L598
    可以在第一个[yolo]层之前的倒数第二个[convolutional]层末尾添加 stopbackward=1,以此提升训练速度

    After training - for detection:
    Increase network-resolution by set in your .cfg-file (height=608 and width=608) or (height=832 and width=832) 
    or (any value multiple of 32) - this increases the precision and makes it possible to detect small objects,
    you do not need to train the network again, just use .weights-file already trained for 416x416 resolution.
    即使在用416*416训练完之后,也可以在cfg文件中设置较大的width和height,增加网络对图像的分辨率,从而更可能检测出图像中的小目标,而不需要重新训练

    if error Out of memory occurs then in .cfg-file you should increase subdivisions=16, 32 or 64

    """

  • 相关阅读:
    程序猿财务自由之路·规划篇
    如何教女友学编程?
    粤港澳大湾区9城最新购房政策一览
    一个漂亮妹子的美团面试经历,4轮2小时,成功拿到Offer
    在北京的互联网公司工作多少年可以买房?
    这六个 MySQL 死锁案例,能让你理解死锁的原因!
    阿里双11:「线上全链路压测」完整经验分享
    千亿级公司低代码平台的测试体系介绍
    Weblogic
    30分钟?不需要,轻松读懂IL
  • 原文地址:https://www.cnblogs.com/xieqi/p/9818056.html
Copyright © 2020-2023  润新知