faster-RCNN 加入新的Ground Truth

Posted 2020-12-29 beatets

出于project的需要，不仅要detect和classify对象，还希望建立不同class的对象之间的关系，进行所谓的pair-matching。

需要完成以下几步：

1. dataset中label的增加：

在自定义的dataset类中，方法_load_XXX_annotation(self, index) 增加 gt_pair 这个annotation：

./lib/datasets/bar_noGroup.py:172: gt_pair = np.zeros((num_objs), dtype=np.int32) # for pair-matching algorithm
./lib/datasets/bar_noGroup.py:198: gt_pair[ix] = int(obj.find(‘ind_class‘))
./lib/datasets/bar_noGroup.py:207: ‘gt_pair‘: gt_pair,

2. 

lib/layer_utils/proposal_target_layer.py:21:def proposal_target_layer(rpn_rois, rpn_scores, gt_boxes, gt_texts, gt_pair, _num_classes):
lib/layer_utils/proposal_target_layer.py:49: labels, texts, pair, rois, roi_scores, bbox_targets, bbox_inside_weights = _sample_rois(
lib/layer_utils/proposal_target_layer.py:50: all_rois, all_scores, gt_boxes, gt_texts, gt_pair, fg_rois_per_image,
lib/layer_utils/proposal_target_layer.py:60: return rois, roi_scores, labels,texts, pair, Variable(bbox_targets), Variable(bbox_inside_weights), Variable(bbox_outside_weights)
lib/layer_utils/proposal_target_layer.py:107:def _sample_rois(all_rois, all_scores, gt_boxes, gt_texts, gt_pair, fg_rois_per_image, rois_per_image, num_classes):
lib/layer_utils/proposal_target_layer.py:119: pair = [gt_pair[i] for i in gt_assignment]
lib/layer_utils/proposal_target_layer.py:120: print(‘pair:‘)
lib/layer_utils/proposal_target_layer.py:121: print(pair)
lib/layer_utils/proposal_target_layer.py:153: pair = [pair[i] for i in keep_inds]
lib/layer_utils/proposal_target_layer.py:167: return labels, texts, pair, rois, roi_scores, bbox_targets, bbox_inside_weights

3. 

lib/nets/network.py:215: rois, roi_scores, labels,texts,pair, bbox_targets, bbox_inside_weights, bbox_outside_weights =
lib/nets/network.py:217: rois, roi_scores, self._gt_boxes, self._gt_texts, self._gt_pair, self._num_classes)
lib/nets/network.py:222: self._proposal_targets[‘pair‘] = pair
lib/nets/network.py:228: if (k != ‘texts‘) and (k != ‘pair‘):
lib/nets/network.py:293: pair_targets = self._proposal_targets[‘pair‘]
lib/nets/network.py:294: #print(pair_targets)
lib/nets/network.py:522: def forward(self, image, im_info, gt_boxes=None, gt_texts=None, gt_pair=None, mode=‘TRAIN‘):
lib/nets/network.py:534: self._gt_pair = gt_pair
lib/nets/network.py:601: blobs[‘gt_texts‘], blobs[‘gt_pair‘])
lib/nets/network.py:610: blobs[‘gt_texts‘], blobs[‘gt_pair‘])
lib/nets/network.py:629: blobs[‘gt_texts‘], blobs[‘gt_pair‘])

4.

lib/roi_data_layer/minibatch.py:50: blobs[‘gt_pair‘] = [roidb[0][‘gt_pair‘][i] for i in gt_inds]