图像角点特征之Harris、SIFT、SURF、ORB

角点检测(Corner Detection)是计算机视觉系统中用来获得图像特征的一种方法,广泛应用于运动检测、图像匹配、视频跟踪、三维建模和目标识别等领域中,也称为特征点检测。 角点通常被定义为两条边的交点,更严格的说,角点的局部邻域应该具有两个不同区域的不同方向的边界。而实际应用中,大多数所谓的角点检测方法检测的是拥有特定特征的图像点,而不仅仅是“角点”。这些特征点在图像中有具体的坐标,并具有某些数学特征,如局部最大或最小灰度、某些梯度特征等。

这些角点通常在图像中是稳定存在的。角点的微小偏移就能反映出图像帧的相对运动。

一、Harris角点

Harris角点检测算法就是对角点响应函数R进行阈值处理:R > threshold,即提取R的局部极大值。
特点:具有角度不变性

1、OpenCV版本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# coding = utf-8
import numpy as np
import cv2 as cv
import matplotlib.pyplot as plt

# 导入资源
image = cv.imread('123.jpg')
image_copy = np.copy(image)
image_copy = cv.cvtColor(image_copy, cv.COLOR_BGR2RGB)
gray = cv.cvtColor(image_copy, cv.COLOR_RGB2GRAY)
gray = np.float32(gray)

# 检测角点
dst = cv.cornerHarris(gray, 2, 3, 0.04)
dst = cv.dilate(dst, None)

# 提取并显示强角点
thresh = 0.1*dst.max()
corner_image = np.copy(image_copy)

for j in range(0, dst.shape[0]):
for i in range(0, dst.shape[1]):
if(dst[j,i] > thresh):
# image, center pt, radius, color, thickness
cv.circle(corner_image, (i, j), 1, (0,255,0), 1)

plt.imshow(corner_image)
plt.show()

2、源码版本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
# coding = utf-8
import numpy as np
import cv2 as cv
import matplotlib.pyplot as plt
from PIL import Image
from skimage import filters
from datetime import datetime

WITH_NMS = False # 是否非极大值抑制,True/False
k = 0.04 # 响应函数参数k
threshold = 0.01 # 界定阈值

# 导入资源
image = cv.imread('123.jpg')
image_copy = np.copy(image)
image_copy = cv.cvtColor(image_copy, cv.COLOR_BGR2RGB)
gray = cv.cvtColor(image_copy, cv.COLOR_RGB2GRAY)
gray = np.float32(gray)
img_num = np.pad(np.asarray(gray, dtype=np.float32), ((1, 1), (1, 1)), 'constant')
h, w = img_num.shape # padding之后的图像尺寸

# 计算Ix,Iy
grad = np.empty([h, w, 2], dtype=np.float)
grad[:, 1:-1, 0] = img_num[:, 2:] - img_num[:, :-2] # Ix
grad[1:-1, :, 1] = img_num[2:, :] - img_num[:-2, :] # Iy

# 计算Ixx,Iyy,Ixy
m = np.empty([h, w, 3], dtype=np.float)
m[:, :, 0] = filters.gaussian(grad[:, :, 0]**2, sigma=2) # Ixx
m[:, :, 1] = filters.gaussian(grad[:, :, 1]**2, sigma=2) # Iyy
m[:, :, 2] = filters.gaussian(grad[:, :, 0]*grad[:, :, 1], sigma=2) # Ixy
m = [np.array([[m[i, j, 0], m[i, j, 2]],
[m[i, j, 2], m[i, j, 1]]]) for i in range(h) for j in range(w)]

# 记录一下R计算时耗
start = datetime.now()
D, T = list(map(np.linalg.det, m)), list(map(np.trace, m))
R = np.array([d-k*t**2 for d, t in zip(D, T)])
end = datetime.now()
print(end-start)

R_max = np.max(R)
R = R.reshape(h, w)

# 标注角点
record = np.zeros_like(R, dtype=np.int)
img_row = np.pad(np.asarray(image_copy, dtype=np.float32), ((1, 1), (1, 1), (0, 0)), 'constant')
for i in range(1, h-2):
for j in range(1, w-2):
if WITH_NMS:
if R[i, j] > R_max*threshold and R[i, j] == np.max(R[i-1:i+2, j-1:j+2]):
record[i, j] = 255
img_row[i, j] = [255, 255, 255]
else:
if R[i, j] > R_max*0.01:
record[i, j] = 255
img_row[i, j] = [255, 255, 255]

# 图像展示与保存
res = Image.fromarray(np.uint8(record[1:-1, 1:-1]))
img_row = Image.fromarray(np.uint8(img_row[1:-1, 1:-1]))

plt.imshow(img_row)
plt.show()

二、SIFT角点

SIFT克服了Harris的不足,缩放也没影响,具有尺度不变性。
特点:角度不变性,尺度不变性

1、OpenCV版本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# coding = utf-8
import numpy as np
import cv2 as cv
import matplotlib.pyplot as plt

# 导入资源
image = cv.imread('123.jpg')
image_copy = np.copy(image)
image_copy = cv.cvtColor(image_copy, cv.COLOR_BGR2RGB)
gray = cv.cvtColor(image_copy, cv.COLOR_RGB2GRAY)
gray = np.float32(gray)

# SIFT角点检测
sift = cv.xfeatures2d.SIFT_create()
# 找到关键点和描述符
keypoints, des = sift.detectAndCompute(gray, None)

# 把特征点标记到图片上
corner_image = np.copy(image_copy)
for point in keypoints:
cv.circle(corner_image, (int(point.pt[0]), int(point.pt[1])), 1, (0,255,0), 1)
plt.imshow(img)
plt.show()

三、SURF角点

SURF是SIFT的加速版,它善于处理具有模糊和旋转的图像,但是不善于处理视角变化和光照变化。在SIFT中使用DoG对LoG进行近似,而在SURF中使用盒子滤波器对LoG进行近似,这样就可以使用积分图像了(计算图像中某个窗口内所有像素和时,计算量的大小与窗口大小无关)。总之,SURF最大的特点在于采用了Haar特征以及积分图像的概念,大大加快了程序的运行效率。
特点:角度不变性,尺度不变性

1、OpenCV版本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# coding = utf-8
import numpy as np
import cv2 as cv
import matplotlib.pyplot as plt

# 导入资源
image = cv.imread('123.jpg')
image_copy = np.copy(image)
image_copy = cv.cvtColor(image_copy, cv.COLOR_BGR2RGB)
gray = cv.cvtColor(image_copy, cv.COLOR_RGB2GRAY)
gray = np.float32(gray)

# 参数为hessian矩阵的阈值
surf = cv.xfeatures2d.SURF_create(400)
# 设置是否要检测方向
surf.setUpright(True)
# 找到关键点和描述符
keypoints, des = surf.detectAndCompute(gray, None)

# 把特征点标记到图片上
corner_image = np.copy(image_copy)
for point in keypoints:
cv.circle(corner_image, (int(point.pt[0]), int(point.pt[1])), 1, (0,255,0), 1)
plt.imshow(corner_image)
plt.show()

更多

四、ORB角点

ORB(Oriented FASTand Rotated BRIEF)算法是目前最快速稳定的特征点检测和提取算法,许多图像拼接和目标追踪技术利用ORB特征进行实现。
ORB采用FAST(features from accelerated segment test)算法来检测特征点,采用BRIEF算法来计算一个特征点的描述子。
特点:角度不变性,尺度不变性,计算速度快(ORB是sift的100倍,是surf的10倍)

1、图像金字塔

ORB算法采用了图像金字塔技术。所谓图像金字塔,是在保持观测窗口不变的情况下,获得输入图像在不同尺寸(分辨率)下的表达,在不同尺寸上提取到的特征在整体上做到了尺寸(分辨率)无关。实际使用中,一般采用2倍下采样,即金字塔中图像长宽逐层折半。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
import numpy as np
import matplotlib.pyplot as plt
import cv2 as cv

image = cv.imread('images/rainbow_flag.jpg')
image = cv.cvtColor(image, cv.COLOR_BGR2RGB)

level_1 = cv.pyrDown(image)
level_2 = cv.pyrDown(level_1)
level_3 = cv.pyrDown(level_2)

f, (ax1,ax2,ax3,ax4) = plt.subplots(1, 4, figsize=(20,10))

ax1.set_title('original')
ax1.imshow(image)

ax2.imshow(level_1)
ax2.set_xlim([0, image.shape[1]])
ax2.set_ylim([image.shape[0], 0])

ax3.imshow(level_2)
ax3.set_xlim([0, image.shape[1]])
ax3.set_ylim([image.shape[0], 0])

ax4.imshow(level_3)
ax4.set_xlim([0, image.shape[1]])
ax4.set_ylim([image.shape[0], 0])

了解更多

2、OpenCV版本的ORB角点检测

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# coding = utf-8
import numpy as np
import cv2 as cv
import matplotlib.pyplot as plt
from mahotas.features import surf

# 导入资源
image = cv.imread('123.jpg')
image_copy = np.copy(image)
image_copy = cv.cvtColor(image_copy, cv.COLOR_BGR2RGB)
gray = cv.cvtColor(image_copy, cv.COLOR_RGB2GRAY)

# 检测角点
orb = cv.ORB_create(200, 2.0)
keypoints, desc = orb.detectAndCompute(gray, None)

# 绘制角点
corner_image = np.copy(image_copy)

def draw_keypoints(img, keypoints, color = (0, 255, 0)):
for kp in keypoints:
x, y = kp.pt
cv.circle(img, (int(x), int(y)), 2, color)

draw_keypoints(corner_image, keypoints)

plt.imshow(corner_image)
plt.show()

0%