ORB 主要特性实验
我们现在将探讨 ORB 算法的几个主要属性：
- 尺度不变性
- 旋转不变性
- 光照不变性
- 噪声不变性
同样，为了更清楚地了解 ORB 算法的特性，在下面的示例中训练图像和查询图像将使用相同内容的图片。
1. 尺度不变性
ORB 算法具有尺度不变性。这意味着它能够检测图像中的对象，而不管其大小。为了验证这一点，我们现在将使用 Brute-Force 匹配器来匹配训练图像和查询图像之间的关键点。注意这里的查询图像是原始训练图像大小的 1/4。
import cv2
import matplotlib.pyplot as plt
# Set the default figure size
plt.rcParams['figure.figsize'] = [14.0, 7.0]
# Load the training image
image1 = cv2.imread('./images/face.jpeg')
# Load the query image
image2 = cv2.imread('./images/faceQS.png')
# Convert the training image to RGB
training_image = cv2.cvtColor(image1, cv2.COLOR_BGR2RGB)
# Convert the query image to RGB
query_image = cv2.cvtColor(image2, cv2.COLOR_BGR2RGB)
# Display the images
plt.subplot(121)
plt.title('Training Image')
plt.xlim([training_image.shape[1], 0])
plt.ylim([training_image.shape[0], 0])
plt.imshow(training_image)
plt.subplot(122)
plt.xlim([training_image.shape[1], 0])
plt.ylim([training_image.shape[0],0])
plt.title('Query Image')
plt.imshow(query_image)
plt.show()
# Convert the training image to gray scale
training_gray = cv2.cvtColor(training_image, cv2.COLOR_BGR2GRAY)
# Convert the query image to gray scale
query_gray = cv2.cvtColor(query_image, cv2.COLOR_BGR2GRAY)
# Set the parameters of the ORB algorithm by specifying the maximum number of keypoints to locate and
# the pyramid decimation ratio
orb = cv2.ORB_create(1000, 2.0)
# Find the keypoints in the gray scale training and query images and compute their ORB descriptor.
# The None parameter is needed to indicate that we are not using a mask in either case.
keypoints_train, descriptors_train = orb.detectAndCompute(training_gray, None)
keypoints_query, descriptors_query = orb.detectAndCompute(query_gray, None)
# Create a Brute Force Matcher object. Set crossCheck to True so that the BFMatcher will only return consistent
# pairs. Such technique usually produces best results with minimal number of outliers when there are enough matches.
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck = True)
# Perform the matching between the ORB descriptors of the training image and the query image
matches = bf.match(descriptors_train, descriptors_query)
# The matches with shorter distance are the ones we want. So, we sort the matches according to distance
matches = sorted(matches, key = lambda x : x.distance)
# Connect the keypoints in the training image with their best matching keypoints in the query image.
# The best matches correspond to the first elements in the sorted matches list, since they are the ones
# with the shorter distance. We draw the first 30 mathces and use flags = 2 to plot the matching keypoints
# without size or orientation.
result = cv2.drawMatches(training_gray, keypoints_train, query_gray, keypoints_query, matches[:30], query_gray, flags = 2)
# Display the best matching points
plt.title('Best Matching Points')
plt.imshow(result)
plt.show()
# Print the shape of the training image
print('
The Training Image has shape:', training_gray.shape)
#Print the shape of the query image
print('The Query Image has shape:', query_gray.shape)
# Print the number of keypoints detected in the training image
print("
Number of Keypoints Detected In The Training Image: ", len(keypoints_train))
# Print the number of keypoints detected in the query image
print("Number of Keypoints Detected In The Query Image: ", len(keypoints_query))
# Print total number of matching points between the training and query images
print("
Number of Matching Keypoints Between The Training and Query Images: ", len(matches))
在上面的示例中，训练图像为 553 x 471 像素，而查询图像为 138 x 117 像素，为原始训练图像的 1/4 大小。另外请注意，在查询图像中检测到的关键点数量只有 65 个，远小于在训练图像中发现的 831 个关键点。然而，还是可以看到 Brute-Force 匹配器可以将查询图像中的大多数关键点与其训练图像中的相应关键点匹配。
2.旋转不变性
ORB 算法具有旋转不变性。这意味着它能够检测图像中的对象而不管它们的方向如何。为了验证这一点，现在将使用 Brute-Force 匹配器来匹配训练图像和旋转 90 度的查询图像之间的关键点。
# Set the parameters of the ORB algorithm by specifying the maximum number of keypoints to locate and
# the pyramid decimation ratio
orb = cv2.ORB_create(1000, 2.0)
# Find the keypoints in the gray scale training and query images and compute their ORB descriptor.
# The None parameter is needed to indicate that we are not using a mask in either case.
keypoints_train, descriptors_train = orb.detectAndCompute(training_gray, None)
keypoints_query, descriptors_query = orb.detectAndCompute(query_gray, None)
# Create a Brute Force Matcher object. Set crossCheck to True so that the BFMatcher will only return consistent
# pairs. Such technique usually produces best results with minimal number of outliers when there are enough matches.
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck = True)
# Perform the matching between the ORB descriptors of the training image and the query image
matches = bf.match(descriptors_train, descriptors_query)
# The matches with shorter distance are the ones we want. So, we sort the matches according to distance
matches = sorted(matches, key = lambda x : x.distance)
# Connect the keypoints in the training image with their best matching keypoints in the query image.
# The best matches correspond to the first elements in the sorted matches list, since they are the ones
# with the shorter distance. We draw the first 100 mathces and use flags = 2 to plot the matching keypoints
# without size or orientation.
result = cv2.drawMatches(training_gray, keypoints_train, query_gray, keypoints_query, matches[:100], query_gray, flags = 2)
# Display the best matching points
plt.title('Best Matching Points')
plt.imshow(result)
plt.show()
在上面的示例中，我们看到在两个图像中检测到的关键点数量非常相似，即使查询图像被旋转，Brute-Force 匹配器仍然可以匹配大约 78% 的关键点。另外值得一提的是，大多数匹配的关键点都接近于特定的面部特征，如眼睛、鼻子和嘴巴。
3.光照不变性
ORB 算法具有光照不变性。这意味着它能够检测图像中的对象，而不管其亮度如何。为了验证这一点，使用 Brute-Force 匹配器来匹配训练图像和更亮的查询图像之间的关键点。
# Set the parameters of the ORB algorithm by specifying the maximum number of keypoints to locate and
# the pyramid decimation ratio
orb = cv2.ORB_create(1000, 2.0)
# Find the keypoints in the gray scale training and query images and compute their ORB descriptor.
# The None parameter is needed to indicate that we are not using a mask in either case.
keypoints_train, descriptors_train = orb.detectAndCompute(training_gray, None)
keypoints_query, descriptors_query = orb.detectAndCompute(query_gray, None)
# Create a Brute Force Matcher object. Set crossCheck to True so that the BFMatcher will only return consistent
# pairs. Such technique usually produces best results with minimal number of outliers when there are enough matches.
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck = True)
# Perform the matching between the ORB descriptors of the training image and the query image
matches = bf.match(descriptors_train, descriptors_query)
# The matches with shorter distance are the ones we want. So, we sort the matches according to distance
matches = sorted(matches, key = lambda x : x.distance)
# Connect the keypoints in the training image with their best matching keypoints in the query image.
# The best matches correspond to the first elements in the sorted matches list, since they are the ones
# with the shorter distance. We draw the first 100 mathces and use flags = 2 to plot the matching keypoints
# without size or orientation.
result = cv2.drawMatches(training_gray, keypoints_train, query_gray, keypoints_query, matches[:100], query_gray, flags = 2)
# Display the best matching points
plt.title('Best Matching Points')
plt.imshow(result)
plt.show()
在上面的示例中，可以看到在两个图像中检测到的关键点数量非常相似，即使查询图像更亮，Brute-Force 匹配器仍然可以匹配找到的大约 63% 的关键点。
4.噪声不变性
ORB 算法具备噪声不变性。这意味着它能够检测图像中的对象，即使图像有一定程度的噪声。为了验证这一点，现在使用Brute-Force 匹配器来匹配训练图像和具有大量噪声的查询图像之间的点。
# Set the parameters of the ORB algorithm by specifying the maximum number of keypoints to locate and
# the pyramid decimation ratio
orb = cv2.ORB_create(1000, 1.3)
# Find the keypoints in the gray scale training and query images and compute their ORB descriptor.
# The None parameter is needed to indicate that we are not using a mask in either case.
keypoints_train, descriptors_train = orb.detectAndCompute(training_gray, None)
keypoints_query, descriptors_query = orb.detectAndCompute(query_gray, None)
# Create a Brute Force Matcher object. We set crossCheck to True so that the BFMatcher will only return consistent
# pairs. Such technique usually produces best results with minimal number of outliers when there are enough matches.
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck = True)
# Perform the matching between the ORB descriptors of the training image and the query image
matches = bf.match(descriptors_train, descriptors_query)
# The matches with shorter distance are the ones we want. So, we sort the matches according to distance
matches = sorted(matches, key = lambda x : x.distance)
# Connect the keypoints in the training image with their best matching keypoints in the query image.
# The best matches correspond to the first elements in the sorted matches list, since they are the ones
# with the shorter distance. We draw the first 100 mathces and use flags = 2 to plot the matching keypoints
# without size or orientation.
result = cv2.drawMatches(training_gray, keypoints_train, query_gray, keypoints_query, matches[:100], query_gray, flags = 2)
# we display the image
plt.title('Best Matching Points')
plt.imshow(result)
plt.show()
在上面的示例中，可以再次看到在两个图像中检测到的关键点数量非常相似，即使查询图像有很多噪声，Brute-Force 匹配器仍然可以匹配找到的大约 63% 的关键点。另外，注意大多数匹配的关键点都接近于特定的面部特征，如眼睛、鼻子和嘴巴。此外，可以看到一些没有不相同的特征点也被匹配了，这可能是由具有相同源的噪声导致的。需要注意的是，在本例中使用的是 1.3 的金字塔缩放比率，而不是前面示例中使用的 2.0。
简单目标识别
下面将实现使用 ORB 算法来基于训练图像检测另一个图像中的人脸。
import cv2
import matplotlib.pyplot as plt
# Set the default figure size
plt.rcParams['figure.figsize'] = [14.0, 7.0]
# Load the training image
image1 = cv2.imread('./images/face.jpeg')
# Load the query image
image2 = cv2.imread('./images/Team.jpeg')
# Convert the training image to RGB
training_image = cv2.cvtColor(image1, cv2.COLOR_BGR2RGB)
# Convert the query image to RGB
query_image = cv2.cvtColor(image2, cv2.COLOR_BGR2RGB)
# Display the images
plt.subplot(121)
plt.imshow(training_image)
plt.title('Training Image')
plt.subplot(122)
plt.imshow(query_image)
plt.title('Query Image')
plt.show()
# Set the default figure size
plt.rcParams['figure.figsize'] = [34.0, 34.0]
# Convert the training image to gray scale
training_gray = cv2.cvtColor(training_image, cv2.COLOR_BGR2GRAY)
# Convert the query image to gray scale
query_gray = cv2.cvtColor(query_image, cv2.COLOR_BGR2GRAY)
# Set the parameters of the ORB algorithm by specifying the maximum number of keypoints to locate and
# the pyramid decimation ratio
orb = cv2.ORB_create(5000, 2.0)
# Find the keypoints in the gray scale training and query images and compute their ORB descriptor.
# The None parameter is needed to indicate that we are not using a mask in either case.
keypoints_train, descriptors_train = orb.detectAndCompute(training_gray, None)
keypoints_query, descriptors_query = orb.detectAndCompute(query_gray, None)
# Create copies of the query images to draw our keypoints on
query_img_keyp = copy.copy(query_image)
# Draw the keypoints with size and orientation on the copy of the query image
cv2.drawKeypoints(query_image, keypoints_query, query_img_keyp, flags = cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
# Display the query image with the keypoints with size and orientation
plt.title('Keypoints With Size and Orientation', fontsize = 30)
plt.imshow(query_img_keyp)
plt.show()
# Print the number of keypoints detected
print("
Number of keypoints Detected: ", len(keypoints_query))
我们可以看到查询图像在图像的许多部分都有关键点。既然我们已经掌握了训练和查询图像的关键点和 ORB 描述符，那么就可以使用 Burte-Force 匹配器来尝试在查询图像中定位女人的脸。
# Set the default figure size
plt.rcParams['figure.figsize'] = [34.0, 34.0]
# Create a Brute Force Matcher object. We set crossCheck to True so that the BFMatcher will only return consistent
# pairs. Such technique usually produces best results with minimal number of outliers when there are enough matches.
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck = True)
# Perform the matching between the ORB descriptors of the training image and the query image
matches = bf.match(descriptors_train, descriptors_query)
# The matches with shorter distance are the ones we want. So, we sort the matches according to distance
matches = sorted(matches, key = lambda x : x.distance)
# Connect the keypoints in the training image with their best matching keypoints in the query image.
# The best matches correspond to the first elements in the sorted matches list, since they are the ones
# with the shorter distance. We draw the first 85 mathces and use flags = 2 to plot the matching keypoints
# without size or orientation.
result = cv2.drawMatches(training_gray, keypoints_train, query_gray, keypoints_query, matches[:85], query_gray, flags = 2)
# we display the image
plt.title('Best Matching Points', fontsize = 30)
plt.imshow(result)
plt.show()
# Print the number of keypoints detected in the training image
print("Number of Keypoints Detected In The Training Image: ", len(keypoints_train))
# Print the number of keypoints detected in the query image
print("Number of Keypoints Detected In The Query Image: ", len(keypoints_query))
# Print total number of matching Keypoints between the training and query images
print("
Number of Matching Keypoints Between The Training and Query Images: ", len(matches))
可以清楚地看到，即使在查询图像中有许多人脸，Burte-Force 匹配器也能够在查询图像中正确地定位女人的脸。
后记
关于 ORB 的讨论就到这里就告一段落了，下一篇将讨论 HOG 算法。
本文翻译整理自 Udacity 计算机视觉纳米学位练习，官方源码连接：
https://github.com/udacity/CVND_Exercises/blob/master/1_4_Feature_Vectors/2. ORB.ipynb