目标检测中的框合并策略nms及其改进

检测算法可能对同一目标产生多次检测的结果,这就需要使用某些算法对检测框去重。常用的两种算法是NMS和Soft-NMS。

NMS

NMS(non maximum suppression,非极大值抑制)算法思想很简单,按照分类概率排序,概率最高的框作为候选框,其它所有与它的IOU高于一个阈值(这是人工指定的超参)的框其概率被置为0。然后在剩余的框里寻找概率第二大的框,其它所有与它的IOU高于一个阈值(这是人工指定的超参)的框其概率被置为0。依次类推。
最终所有的框相互之间的IOU都是小于超参阈值的,或者概率被置为0了。剩下的所有概率非0的框就是最终的检测框。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
import numpy as np

def nms(dets, thresh):
x1 = dets[:, 0] # pred bbox top_x
y1 = dets[:, 1] # pred bbox top_y
x2 = dets[:, 2] # pred bbox bottom_x
y2 = dets[:, 3] # pred bbox bottom_y
scores = dets[:, 4] # pred bbox cls score

areas = (x2 - x1 + 1) * (y2 - y1 + 1) # pred bbox areas
order = scores.argsort()[::-1] # 对pred bbox按置信度做降序排序

keep = []
while order.size > 0:
i = order[0] # top-1 score pred bbox
keep.append(i)
xx1 = np.maximum(x1[i], x1[order[1:]])
yy1 = np.maximum(y1[i], y1[order[1:]])
xx2 = np.minimum(x2[i], x2[order[1:]])
yy2 = np.minimum(y2[i], y2[order[1:]])

w = np.maximum(0.0, xx2 - xx1 + 1)
h = np.maximum(0.0, yy2 - yy1 + 1)
inter = w * h
IoU = inter / (areas[i] + areas[order[1:]] - inter) # IoU计算

inds = np.where(IoU <= thresh)[0] # 保留非冗余bbox
order = order[inds + 1]

return keep # 最终NMS结果返回

if __name__ == '__main__':

dets = np.array([[100,120,170,200,0.98],
[20,40,80,90,0.94],
[20,38,82,88,0.99],
[200,380,282,488,0.9],
[19,38,75,91, 0.98]])

print(nms(dets, 0.5))

Soft-NMS

NMS算法是略显粗暴,因为NMS直接将删除所有IoU大于阈值的框。soft-NMS吸取了NMS的教训,在算法执行过程中不是简单的对IoU大于阈值的检测框删除,而是降低得分。算法流程同NMS相同,但是对原置信度得分使用函数运算,目标是降低置信度得分。
可以说NMS是Soft-NMS特殊形式,当得分重置函数采用二值化函数时,Soft-NMS和NMS是相同的。Soft-NMS算法是一种更加通用的非最大抑制算法。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
import numpy as np

def soft_nms(dets, method, thresh=0.001, Nt=0.1, sigma=0.5):
x1 = dets[:, 0]
y1 = dets[:, 1]
x2 = dets[:, 2]
y2 = dets[:, 3]
scores = dets[:, 4]

areas = (y2 - y1 + 1.) * (x2 - x1 + 1.)
orders = scores.argsort()[::-1]
keep = []

while orders.size > 0:
i = orders[0]
keep.append(i)
for j in orders[1:]:
xx1 = np.maximum(x1[i], x1[j])
yy1 = np.maximum(y1[i], y1[j])
xx2 = np.minimum(x2[i], x2[j])
yy2 = np.minimum(y2[i], y2[j])

w = np.maximum( xx2 - xx1 + 1., 0. )
h = np.maximum( yy2 - yy1 + 1., 0. )
inter = w * h
IoU = inter / (areas[i] + areas[j] - inter)

if method == 1: # linear
if IoU > Nt:
weight = 1 - IoU
else:
weight = 1
elif method == 2: # gaussian
weight = np.exp(-(IoU * IoU)/sigma)
else: # original NMS
if IoU > Nt:
weight = 0
else:
weight = 1
scores[j] = weight * scores[j]

if scores[j] < thresh:
orders = np.delete(orders, np.where(orders == j))

orders = np.delete(orders, 0)

return keep

if __name__ == '__main__':

dets = np.array([[100,120,170,200,0.98],
[20,40,80,90,0.94],
[20,38,82,88,0.99],
[200,380,282,488,0.9],
[19,38,75,91, 0.98]])

print(soft_nms(dets, 2))
0%