Jetson Nano Developer Kit

The Jetson Nano Developer Kit is an AI computer for learning and for making.
​
一个推理框架,用于部署模型到嵌入式设备.
​
Four Steps to Deep Learning

• System Setup
• Image Recognition
• Object Detection
• Segmentation
  ​
  CUDA
  一种并行计算技术
  https://zh.wikipedia.org/wiki/CUDA

https://github.com/dusty-nv/jetson-inference#system-setup

Hello AI World

环境准备

• https://github.com/dusty-nv/jetson-inference/blob/master/docs/jetpack-setup-2.md
• https://github.com/dusty-nv/jetson-inference/blob/master/docs/building-repo-2.md
  
  预训练好的模型位于data目录下.

图像分类

核心类imageNet https://github.com/dusty-nv/jetson-inference/blob/master/imageNet.h
imageNet接收image作为input，输出每一种类别的概率.
​
在编译出来的build/aarch64/bin目录下有2个示例程序

• imagenet-console
• iamgenet-camera
  ​
  编写自己的图像识别程序.
  https://github.com/dusty-nv/jetson-inference/tree/master/examples/my-recognition
  ​

/*
 * Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
 *
 * Permission is hereby granted, free of charge, to any person obtaining a
 * copy of this software and associated documentation files (the "Software"),
 * to deal in the Software without restriction, including without limitation
 * the rights to use, copy, modify, merge, publish, distribute, sublicense,
 * and/or sell copies of the Software, and to permit persons to whom the
 * Software is furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
 * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
 * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
 * DEALINGS IN THE SOFTWARE.
 */
​
// include imageNet header for image recognition
#include <jetson-inference/imageNet.h>
​
// include loadImage header for loading images
#include <jetson-utils/loadImage.h>
​
​
// main entry point
int main( int argc, char** argv )
{
    // a command line argument containing the image filename is expected,
    // so make sure we have at least 2 args (the first arg is the program)
    if( argc < 2 )
    {
        printf("my-recognition:  expected image filename as argument
");
        printf("example usage:   ./my-recognition my_image.jpg
");
        return 0;
    }
​
    // retrieve the image filename from the array of command line args
    const char* imgFilename = argv[1];
​
    // these variables will be used to store the image data and dimensions
    // the image data will be stored in shared CPU/GPU memory, so there are
    // pointers for the CPU and GPU (both reference the same physical memory)
    float* imgCPU    = NULL;        // CPU pointer to floating-point RGBA image data
    float* imgCUDA   = NULL;        // GPU pointer to floating-point RGBA image data
    int    imgWidth  = 0;       // width of the image (in pixels)
    int    imgHeight = 0;       // height of the image (in pixels)
        
    // load the image from disk as float4 RGBA (32 bits per channel, 128 bits per pixel)
    if( !loadImageRGBA(imgFilename, (float4**)&imgCPU, (float4**)&imgCUDA, &imgWidth, &imgHeight) )
    {
        printf("failed to load image '%s'
", imgFilename);
        return 0;
    }
​
    // load the GoogleNet image recognition network with TensorRT
    // you can use imageNet::ALEXNET to load AlexNet model instead
    imageNet* net = imageNet::Create(imageNet::GOOGLENET);
​
    // check to make sure that the network model loaded properly
    if( !net )
    {
        printf("failed to load image recognition network
");
        return 0;
    }
​
    // this variable will store the confidence of the classification (between 0 and 1)
    float confidence = 0.0;
​
    // classify the image with TensorRT on the GPU (hence we use the CUDA pointer)
    // this will return the index of the object class that the image was recognized as (or -1 on error)
    const int classIndex = net->Classify(imgCUDA, imgWidth, imgHeight, &confidence);
​
    // make sure a valid classification result was returned 
    if( classIndex >= 0 )
    {
        // retrieve the name/description of the object class index
        const char* classDescription = net->GetClassDesc(classIndex);
​
        // print out the classification results
        printf("image is recognized as '%s' (class #%i) with %f%% confidence
", 
              classDescription, classIndex, confidence * 100.0f);
    }
    else
    {
        // if Classify() returned < 0, an error occurred
        printf("failed to classify image
");
    }
    
    // free the network's resources before shutting down
    delete net;
​
    // this is the end of the example!
    return 0;
}

• 载入图像 loadImageRGBA
  加载的图像存储于共享内存，映射到cpu和gpu.实际的内存里的image只有1份,cpu/gpu pointer指向的都是同一份物理内存。

The loaded image will be stored in shared memory that's mapped to both the CPU and GPU. There are two pointers available for access in the CPU and GPU address spaces, but there is really only one copy of the image in memory. Both the CPU and GPU pointers resolve to the same physical memory, without needing to perform memory copies (i.e. cudaMemcpy()).
​

• 载入神经网络模型
  imageNet::Create()
  GOOGLENET是一个预先训练好的模型,使用的数据集是ImageNet(注意不是imageNet对象).类别有1000个,包括了动植物,常见生活用品等.
  ​

    // load the GoogleNet image recognition network with TensorRT
    // you can use imageNet::ALEXNET to load AlexNet model instead
    imageNet* net = imageNet::Create(imageNet::GOOGLENET);
​
    // check to make sure that the network model loaded properly
    if( !net )
    {
        printf("failed to load image recognition network
");
        return 0;
    }

​

• 对图片进行分类
  Classify返回的是类别对应的index
  ​

    //this variable will store the confidence of the classification (between 0 and 1)
    float confidence = 0.0;
​
    // classify the image with TensorRT on the GPU (hence we use the CUDA pointer)
    // this will return the index of the object class that the image was recognized as (or -1 on error)
    const int classIndex = net->Classify(imgCUDA, imgWidth, imgHeight, &confidence);
​

​

• 解释结果
  ​

   // make sure a valid classification result was returned  
    if( classIndex >= 0 )
    {
        // retrieve the name/description of the object class index
        const char* classDescription = net->GetClassDesc(classIndex);
​
        // print out the classification results
        printf("image is recognized as '%s' (class #%i) with %f%% confidence
", 
              classDescription, classIndex, confidence * 100.0f);
    }
    else
    {
        // if Classify() returned < 0, an error occurred
        printf("failed to classify image
");
    }

​

These descriptions of the 1000 classes are parsed from ilsvrc12_synset_words.txt when the network gets loaded (this file was previously downloaded when the jetson-inference repo was built).
​
​

• 退出
  程序退出前要释放掉资源

    // free the network's resources before shutting down
    delete net;
​
    // this is the end of the example!
    return 0;
}

​

cmake文件

# require CMake 2.8 or greater
cmake_minimum_required(VERSION 2.8)
​
# declare my-recognition project
project(my-recognition)
​
# import jetson-inference and jetson-utils packages.
# note that if you didn't do "sudo make install"
# while building jetson-inference, this will error.
find_package(jetson-utils)
find_package(jetson-inference)
​
# CUDA and Qt4 are required
find_package(CUDA)
find_package(Qt4)
​
# setup Qt4 for build
include(${QT_USE_FILE})
add_definitions(${QT_DEFINITIONS})
​
# compile the my-recognition program
cuda_add_executable(my-recognition my-recognition.cpp)
​
# link my-recognition to jetson-inference library
target_link_libraries(my-recognition jetson-inference)

没什么要特别说的,主要的依赖如下:

• find_package(jetson-utils)
• find_package(jetson-inference)
• target_link_libraries(my-recognition jetson-inference)
  ​

实时图片识别

上面的代码展示的是本地图片的识别,这一节给出实时的摄像头拍摄图片识别的demo.

• iamgenet-camera

​