• Camera Calibration 相机标定:Opencv应用方法



    本系列文章由 @YhL_Leo 出品,转载请注明出处。
    文章链接: http://blog.csdn.net/yhl_leo/article/details/49427383


    Opencv中Camera Calibration and 3D Reconstruction中使用的是Z. Zhang(PAMI, 2000). A Flexible New Technique for Camera Calibration的方法。原理见原理简介(五)本文将对其进行介绍。

    1 标定步骤

    简单来说,Opencv中基于二维标定平面的标定方法主要步骤有:

    • 1 读取相关设置信息,包括采用的pattern 信息(类型,尺寸),输入标定数据的信息(图像列表文件,视频采样方法),输出文件设置等,这些信息可以存为XML或YAML文件的形式或者在代码里直接显示设置。这里给出Opencv中提供的configuration file:
    <?xml version="1.0"?>
    <opencv_storage>
    <Settings>
    <!--
     Number of inner corners per a item row and column. (square, circle) 
    -->
    <BoardSize_Width>9</BoardSize_Width>
    <BoardSize_Height>6</BoardSize_Height>
    <!--
     The size of a square in some user defined metric system (pixel, millimeter)
    -->
    <Square_Size>50</Square_Size>
    <!--
     The type of input used for camera calibration. One of: CHESSBOARD CIRCLES_GRID ASYMMETRIC_CIRCLES_GRID 
    -->
    <Calibrate_Pattern>"CHESSBOARD"</Calibrate_Pattern>
    <!--
     The input to use for calibration. 
            To use an input camera -> give the ID of the camera, like "1"
            To use an input video  -> give the path of the input video, like "/tmp/x.avi"
            To use an image list   -> give the path to the XML or YAML file containing the list of the images, like "/tmp/circles_list.xml"
    
    -->
    <Input>"images/CameraCalibraation/VID5/VID5.xml"</Input>
    <!--
      If true (non-zero) we flip the input images around the horizontal axis.
    -->
    <Input_FlipAroundHorizontalAxis>0</Input_FlipAroundHorizontalAxis>
    <!--  Time delay between frames in case of camera.  -->
    <Input_Delay>100</Input_Delay>
    <!--  How many frames to use, for calibration.  -->
    <Calibrate_NrOfFrameToUse>25</Calibrate_NrOfFrameToUse>
    <!--
     Consider only fy as a free parameter, the ratio fx/fy stays the same as in the input cameraMatrix. 
           Use or not setting. 0 - False Non-Zero - True
    -->
    <Calibrate_FixAspectRatio>1</Calibrate_FixAspectRatio>
    <!--
     If true (non-zero) tangential distortion coefficients  are set to zeros and stay zero.
    -->
    <Calibrate_AssumeZeroTangentialDistortion>1</Calibrate_AssumeZeroTangentialDistortion>
    <!--
     If true (non-zero) the principal point is not changed during the global optimization.
    -->
    <Calibrate_FixPrincipalPointAtTheCenter>1</Calibrate_FixPrincipalPointAtTheCenter>
    <!--  The name of the output log file.  -->
    <Write_outputFileName>"out_camera_data.xml"</Write_outputFileName>
    <!--
     If true (non-zero) we write to the output file the feature points.
    -->
    <Write_DetectedFeaturePoints>1</Write_DetectedFeaturePoints>
    <!--
     If true (non-zero) we write to the output file the extrinsic camera parameters.
    -->
    <Write_extrinsicParameters>1</Write_extrinsicParameters>
    <!--
     If true (non-zero) we show after calibration the undistorted images.
    -->
    <Show_UndistortedImage>1</Show_UndistortedImage>
    </Settings>
    </opencv_storage>

    其中,图像文件列表images/CameraCalibraation/VID5/VID5.xmlOpencv中采用列举法:

    <?xml version="1.0"?>
    <opencv_storage>
    <images>
    images/CameraCalibraation/VID5/xx1.jpg 
    images/CameraCalibraation/VID5/xx2.jpg 
    images/CameraCalibraation/VID5/xx3.jpg 
    images/CameraCalibraation/VID5/xx4.jpg 
    images/CameraCalibraation/VID5/xx5.jpg 
    images/CameraCalibraation/VID5/xx6.jpg 
    images/CameraCalibraation/VID5/xx7.jpg 
    images/CameraCalibraation/VID5/xx8.jpg
    </images>
    </opencv_storage>

    文件中参数的含义比较清晰明了,此处就不累述。

    • 2 依次从图像中检测pattern信息,如果检测成功,角点信息将会存储记录,用于标定解算。
    cv::Mat viewGray;
    if ( view.channels() == 3 )
        cv::cvtColor( view, viewGray, CV_BGR2GRAY );
    else
        view.copyTo( viewGray );
    
    std::vector<cv::Point2f> imagePoints;   
    bool success = cv::findChessboardCorners( viewGray , boardSize, imagePoints);
    • 3 优化角点检测精度,将上述检测成功的角点,通过精确角点定位方法,提高精度,下图为Opencv提供的检测结果。
    cv::cornerSubPix( viewGray, 
                  imagePoints, 
                  cv::Size(11,11),
                  cv::TermCriteria( CV_TERMCRIT_EPS+CV_TERMCRIT_ITER, 30, 0.1 ));

    Opencv

    • 4 标定解算,每幅图像都进行上述的角点检测后,一般给像点对应的物方角点虚拟坐标的方式赋予对应的坐标,即可进行相机标定解算,包括相机内参,相机畸变系数,以及相机在虚拟坐标所在坐标系中相对于每幅图像的相对位置姿态(旋转向量和平移向量)。
    double reprojectionError= cv::calibrateCamera(
        objectPoints,   // calibration pattern points in the calibration pattern coordinate space
        imagePoints,    // projections of calibration pattern points
        imageSize,      // Size of the image used only to initialize the intrinsic camera matrix
        cameraMatrix,   // camera matrix A
        distCoeffs,     // distortion coefficients (k1,k2,p1,p2[,k3[,k4,k5,k6]])
        rvecs,          // rotation vectors
        tvecs,          // translation vectors
        flag,           // different calibration model
        criteria);      // Termination criteria for iterative optimization algorithm
    • 5 标定精度评估,为了评价标定后的结果,可以按照标定得到的相机成像模型,由像点反算出物方空间坐标,进而得到一系列点云,通过对比解算点云与虚拟点云之间的差异性,就可以知道获得模型的好坏(严格来讲,如果误差较小,两者基本应该是一致的)。

    • 6 图像畸变校正,在opencv示例中,作为标定的最后一个步骤,但是个人认为,这个应该可以作为一个相机标定后的副产品,对于处理的图像产品精度要求较高时,可以先进行畸变校正,再投入生产。下图为Opencv提供的畸变校正结果。

    Opencv2

    2 代码及结果

    下面是个人的代码程序,有些部分并没完全按照Opencv的做法:

    /*
       Calibrate camera with chess board pattern.
    
       - Editor: Menghan Xia, Yahui Liu.
       - Data:   2015-07-28
       - Email:  yahui.cvrs@gmail.com
       - Address: Computer Vision and Remote Sensing(CVRS) Lab, Wuhan University.
    **/
    
    #include<iostream>
    #include <vector>
    #include <string>
    
    #include "cv.h"
    #include "highgui.h"
    
    #include "toolFunction.h"
    #define DEBUG_OUTPUT_INFO
    
    using namespace std;
    using namespace cv;
    
    void main()
    {   
        char* folderPath = "E:/Images/New";           // image folder
        std::vector<std::string> graphPaths;
        std::vector<std::string> graphSuccess;
    
        CalibrationAssist calAssist;
    
        graphPaths = calAssist.get_filelist(folderPath); // collect image list
    
    #ifdef DEBUG_OUTPUT_INFO
        std::cout << "loaded " << graphPaths.size() << " images"<< std::endl;
    #endif
    
        if ( !graphPaths.empty() )
        {
    #ifdef DEBUG_OUTPUT_INFO
            std::cout << "Start corner detection ..." << std::endl;
    #endif
    
            cv::Mat curGraph;  // current image
            cv::Mat gray;      // gray image of current image
    
            int imageCount = graphPaths.size();
            int imageCountSuccess = 0;
            cv::Size image_size; 
            cv::Size boardSize  = cv::Size(19, 19);     // chess board pattern size
            cv::Size squareSize = cv::Size(15, 15);     // grid physical size, as a scale factor
    
            std::vector<cv::Point2f> corners;                  // one image corner list
            std::vector<std::vector<cv::Point2f> > seqCorners; // n images corner list
    
            if ( graphPaths.size() < 3 )
            {
    #ifdef DEBUG_OUTPUT_INFO
                std::cout << "Calibrate failed, with less than three images!" << std::endl;
    #endif
                return ;
            }
    
            for ( int i=0; i<graphPaths.size(); i++ )
            {   
                string graphpath = folderPath;
                graphpath += "/" + graphPaths[i];
                curGraph = cv::imread(graphpath);
    
                if ( curGraph.channels() == 3 )
                    cv::cvtColor( curGraph, gray, CV_BGR2GRAY );
                else
                    curGraph.copyTo( gray );
    
                // for every image, empty the corner list
                std::vector<cv::Point2f>().swap( corners );  
    
                // corners detection
                bool success = cv::findChessboardCorners( curGraph, boardSize, corners ); 
    
                if ( success ) // succeed
                {
    #ifdef DEBUG_OUTPUT_INFO
                    std::cout << i << " " << graphPaths[i] << " succeed"<< std::endl;
    #endif
                    int row = curGraph.rows;
                    int col = curGraph.cols;
    
                    graphSuccess.push_back( graphpath );
                    imageCountSuccess ++;
    
                    image_size = cv::Size( col, row );
    
                    // find sub-pixels
                    cv::cornerSubPix( 
                        gray, 
                        corners, 
                        cv::Size( 11, 11 ), 
                        cv::Size( -1, -1 ),
                        cv::TermCriteria( CV_TERMCRIT_EPS + CV_TERMCRIT_ITER, 30, 0.1 ) );
                    seqCorners.push_back( corners );
    
    #if 1
                    // draw corners and show them in current image
                    cv::Mat imageDrawCorners;
                    if ( curGraph.channels() == 3 )
                        curGraph.copyTo( imageDrawCorners );
                    else
                        cv::cvtColor( curGraph, imageDrawCorners, CV_GRAY2RGB );
    
                    for ( int j = 0; j < corners.size(); j ++)
                    {
                        cv::Point2f dotPoint = corners[j];
                        cv::circle( imageDrawCorners, dotPoint, 3.0, cv::Scalar( 0, 255, 0 ), -1 );
                        cv::Point2f pt_m = dotPoint + cv::Point2f(4,4);
                        char text[100];
                        sprintf( text, "%d", j+1 );  // corner indexes which start from 1
                        cv::putText( imageDrawCorners, text, pt_m, 1, 0.5, cv::Scalar( 255, 0, 255 ) );
                    }
    
                    std::string pathTemp;
                    pathTemp = folderPath;
                    pathTemp += "/#" + graphPaths[i];
    
                    // save image drawn with corners and labeled with indexes
                    cv::imwrite( pathTemp, imageDrawCorners ); 
    #endif
                }
    #ifdef DEBUG_OUTPUT_INFO
                else // failed
                {
                    std::cout << graphPaths[i] << " corner detect failed!" << std::endl;
                }
    #endif
    
            }
    #ifdef DEBUG_OUTPUT_INFO
            std::cout << "Corner detect done!" << std::endl 
                << imageCountSuccess << " succeed! " << std::endl;
    #endif
    
    
            if ( imageCountSuccess < 3 )
            {
    #ifdef DEBUG_OUTPUT_INFO
                std::cout << "Calibrated success " << imageCountSuccess 
                    << " images, less than 3 images." << std::endl;
    #endif
                return ;
            }
            else
            {
    #ifdef DEBUG_OUTPUT_INFO
                std::cout << "Start calibration ..." << std::endl;
    #endif
                cv::Point3f point3D;
                std::vector<cv::Point3f> objectPoints;
                std::vector<double> distCoeffs;
                std::vector<double> rotation;
                std::vector<double> translation;
    
                std::vector<std::vector<cv::Point3f>> seqObjectPoints;
                std::vector<std::vector<double>> seqRotation;
                std::vector<std::vector<double>> seqTranslation;
                cv::Mat_<double> cameraMatrix;
    
                // calibration pattern points in the calibration pattern coordinate space
                for ( int t=0; t<imageCountSuccess; t++ )
                {
                    objectPoints.clear();
                    for ( int i=0; i<boardSize.height; i++ )
                    {
                        for ( int j=0; j<boardSize.width; j++ )
                        {
                            point3D.x = i * squareSize.width;
                            point3D.y = j * squareSize.height;
                            point3D.z = 0;
                            objectPoints.push_back(point3D);
                        }
                    }
                    seqObjectPoints.push_back(objectPoints);
                }
    
                double reprojectionError = calibrateCamera(
                    seqObjectPoints, 
                    seqCorners, 
                    image_size, 
                    cameraMatrix, 
                    distCoeffs, 
                    seqRotation, 
                    seqTranslation,
                    CV_CALIB_FIX_ASPECT_RATIO|CV_CALIB_FIX_PRINCIPAL_POINT );
    
    #ifdef DEBUG_OUTPUT_INFO
                std::cout << "Calibration done!" << std::endl;
    #endif
                // calculate the calibration pattern points with the camera model
                std::vector<cv::Mat_<double>> projectMats;
    
                for ( int i=0; i<imageCountSuccess; i++ )
                {
                    cv::Mat_<double> R, T;
                    // translate rotation vector to rotation matrix via Rodrigues transformation
                    cv::Rodrigues( seqRotation[i], R ); 
                    T = cv::Mat( cv::Matx31d( 
                        seqTranslation[i][0], 
                        seqTranslation[i][1],
                        seqTranslation[i][2]) );
    
                    cv::Mat_<double> P = cameraMatrix * cv::Mat( cv::Matx34d( 
                        R(0,0), R(0,1), R(0,2), T(0),  
                        R(1,0), R(1,1), R(1,2), T(1),  
                        R(2,0), R(2,1), R(2,2), T(2) ) ); 
    
                    projectMats.push_back(P);
                }
    
                std::vector<cv::Point2d> PointSet;
                int pointNum = boardSize.width*boardSize.height;
                std::vector<cv::Point3d> objectClouds;
                for ( int i=0; i<pointNum; i++ )
                {
                    PointSet.clear();
                    for ( int j=0; j<imageCountSuccess; j++ )
                    {
                        cv::Point2d tempPoint = seqCorners[j][i];
                        PointSet.push_back(tempPoint);
                    }
                    // calculate calibration pattern points
                    cv::Point3d objectPoint = calAssist.triangulate(projectMats,PointSet);
                    objectClouds.push_back(objectPoint);
                }
                std::string pathTemp_point;
                pathTemp_point = folderPath;
                pathTemp_point += "/point.txt";
                calAssist.save3dPoint(pathTemp_point,objectClouds);
    
                std::string pathTemp_calib;
                pathTemp_calib = folderPath;
                pathTemp_calib += "/calibration.txt";
    
                FILE* fp = fopen( pathTemp_calib.c_str(), "w" );
                fprintf( fp, "The average of re-projection error : %lf
    ", reprojectionError );
                for ( int i=0; i<imageCountSuccess; i++ )
                {
                    std::vector<cv::Point2f> errorList;
                    cv::projectPoints( 
                        seqObjectPoints[i], 
                        seqRotation[i], 
                        seqTranslation[i], 
                        cameraMatrix, 
                        distCoeffs, 
                        errorList );
    
                    corners.clear();
                    corners = seqCorners[i];
    
                    double meanError(0.0);
                    for ( int j=0; j<corners.size(); j++ )
                    {   
                        meanError += std::sqrt((errorList[j].x - corners[j].x)*(errorList[j].x - corners[j].x) + 
                            (errorList[j].y - corners[j].y)*(errorList[j].y - corners[j].y));
                    }
                    rotation.clear();
                    translation.clear();
    
                    rotation = seqRotation[i];
                    translation = seqTranslation[i];
                    fprintf( fp, "Re-projection of image %d:%lf
    ", i+1, meanError/corners.size() );
                    fprintf( fp, "Rotation vector :
    " );
                    fprintf( fp, "%lf %lf %lf
    ", rotation[0], rotation[1], rotation[2] );
                    fprintf( fp, "Translation vector :
    " );
                    fprintf( fp, "%lf %lf %lf
    
    ", translation[0], translation[1], translation[2] );
                }
                fprintf( fp, "Camera internal matrix :
    " );
                fprintf( fp, "%lf %lf %lf
    %lf %lf %lf
    %lf %lf %lf
    ", 
                    cameraMatrix(0,0), cameraMatrix(0,1), cameraMatrix(0,2),
                    cameraMatrix(1,0), cameraMatrix(1,1), cameraMatrix(1,2),
                    cameraMatrix(2,0), cameraMatrix(2,1), cameraMatrix(2,2));
                fprintf( fp,"Distortion coefficient :
    " );
                for ( int k=0; k<distCoeffs.size(); k++)
                    fprintf( fp, "%lf ", distCoeffs[k] );
    #ifdef DEBUG_OUTPUT_INFO
                std::cout << "Results are saved!" << std::endl;
    #endif  
            }
        }
    }
    // toolFunction.h
    #ifndef TOOL_FUNCTION_H
    #pragma once
    #define TOOL_FUNCTION_H
    
    #include<iostream>
    #include <Windows.h>
    #include <math.h>
    #include <fstream>
    #include <vector>
    #include <string>
    
    #include "cv.h"
    #include "highgui.h"
    
    using namespace cv;
    using namespace std;
    
    class CalibrationAssist
    {
    public:
        CalibrationAssist() {}
        ~CalibrationAssist() {}
    
    public:
        std::vector<std::string>get_filelist( std::string foldname );
    
        cv::Point3d triangulate( std::vector<cv::Mat_<double>> &ProjectMats, 
            std::vector<cv::Point2d> &imagePoints );
    
        void save3dPoint( std::string path_, std::vector<cv::Point3d> &Point3dLists );
    };
    #endif // TOOL_FUNCTION_H
    // toolFunction.cpp
    #include "toolFunction.h"
    
    std::vector<std::string> CalibrationAssist::get_filelist( std::string foldname )
    {
        foldname += "/*.*";
        const char * mystr=foldname.c_str();
        std::vector<std::string> flist;
        std::string lineStr;
        std::vector<std::string> extendName;
        extendName.push_back("jpg");
        extendName.push_back("JPG");
        extendName.push_back("bmp");
        extendName.push_back("png");
        extendName.push_back("gif");
    
        HANDLE file;
        WIN32_FIND_DATA fileData;
        char line[1024];
        wchar_t fn[1000];
        mbstowcs( fn, mystr, 999 );
        file = FindFirstFile( fn, &fileData );
        FindNextFile( file, &fileData );
        while(FindNextFile( file, &fileData ))
        {
            wcstombs( line, (const wchar_t*)fileData.cFileName, 259);
            lineStr = line;
            // remove the files which are not images
            for (int i = 0; i < 4; i ++)
            {
                if (lineStr.find(extendName[i]) < 999)
                {
                    flist.push_back(lineStr);
                    break;
                }
            }   
        }
        return flist;
    }
    
    
    cv::Point3d CalibrationAssist::triangulate(
        std::vector<cv::Mat_<double>> &ProjectMats, 
        std::vector<cv::Point2d> &imagePoints)
    {
        int i,j;
        std::vector<cv::Point2d> pointSet;
        int frameSum = ProjectMats.size();
        cv::Mat A(2*frameSum,3,CV_32FC1);
        cv::Mat B(2*frameSum,1,CV_32FC1);
        cv::Point2d u,u1;
        cv::Mat_<double> P;
        cv::Mat_<double> rowA1,rowA2,rowB1,rowB2;
        int k = 0;
        for ( i = 0; i < frameSum; i++ )     //get the coefficient matrix A and B
        {
            u = imagePoints[i];
            P = ProjectMats[i];
            cv::Mat( cv::Matx13d( 
                u.x*P(2,0)-P(0,0),
                u.x*P(2,1)-P(0,1),
                u.x*P(2,2)-P(0,2) ) ).copyTo( A.row(k) );
    
            cv::Mat( cv::Matx13d( 
                u.y*P(2,0)-P(1,0),
                u.y*P(2,1)-P(1,1),
                u.y*P(2,2)-P(1,2) ) ).copyTo( A.row(k+1) );
    
            cv::Mat rowB1( 1, 1, CV_32FC1, cv::Scalar( -(u.x*P(2,3)-P(0,3)) ) );
            cv::Mat rowB2( 1, 1, CV_32FC1, cv::Scalar(-(u.y*P(2,3)-P(1,3)) ) );
            rowB1.copyTo( B.row(k) );
            rowB2.copyTo( B.row(k+1) );
            k += 2;
        }
        cv::Mat X;  
        cv::solve( A, B, X, DECOMP_SVD );  
        return Point3d(X); 
    }
    
    void CalibrationAssist::save3dPoint( std::string path_, std::vector<cv::Point3d> &Point3dLists)
    {
        const char * path = path_.c_str();
        FILE* fp = fopen( path, "w" );
        for ( int i = 0; i < Point3dLists.size(); i ++)
        {
            //      fprintf(fp,"%d ",i);
            fprintf( fp, "%lf %lf %lf
    ", 
                Point3dLists[i].x, Point3dLists[i].y, Point3dLists[i].z);
        }
        fclose(fp);
    #if 1
        std::cout << "clouds of points are saved!" << std::endl;
    #endif
    }


    使用数据为91200×800的图像:

    Opencv4

    程序运行结果:

    • 1 运行控制台输出结果

      Opencv3

    • 2 角点检测图

    Opencv5

    • 3 反投影点云(CloudCompare显示)

    Opencv6

    对于上述结果的生成文件,此处用了C语言写成txt的方式,读者完全可以考虑使用XML或YAML格式文件保存,至于畸变纠正的问题,也很简单,直接利用标定得到的相机内参和畸变系数,查询remap函数的使用方法即可。此外,处理较大图像时,Opencv提供的方法速度可能会较慢,遇到这种情况,可以考虑把图像缩小或重写角点检测算法。

  • 相关阅读:
    提升键盘可访问性和AT可访问性
    2个小技巧
    设计模式(6): 数据抽象与业务封装
    目录
    医疗经济学
    医疗场景下的行为经济学(三)
    抗体、免疫
    医疗场景下的行为经济学(二)
    医疗场景下的行为经济学(一)
    单身社会如何生活-日本纪录片ガイアの夜明け系列
  • 原文地址:https://www.cnblogs.com/hehehaha/p/6332227.html
Copyright © 2020-2023  润新知