使 DeepLabV3 损失函数加权

Posted 2023-03-27

【中文标题】使 DeepLabV3 损失函数加权

【英文标题】：Make the DeepLabV3 loss function weighted

【发布时间】：2020-03-11 21:32:43

【问题描述】：

我正在使用DeepLabV3+ repository，我注意到loss_weight 设置为1.0，这意味着我们以相同的方式对类进行加权。然而，我有一个非常不平衡的数据集，比如 80% 的负数和 20% 的正数。

def add_softmax_cross_entropy_loss_for_each_scale(scales_to_logits,
                                                  labels,
                                                  num_classes,
                                                  ignore_label,
                                                  loss_weight=1.0,
                                                  upsample_logits=True,
                                                  hard_example_mining_step=0,
                                                  top_k_percent_pixels=1.0,
                                                  scope=None):
  """Adds softmax cross entropy loss for logits of each scale.

  Args:
    scales_to_logits: A map from logits names for different scales to logits.
      The logits have shape [batch, logits_height, logits_width, num_classes].
    labels: Groundtruth labels with shape [batch, image_height, image_width, 1].
    num_classes: Integer, number of target classes.
    ignore_label: Integer, label to ignore.
    loss_weight: Float, loss weight.
    upsample_logits: Boolean, upsample logits or not.
    hard_example_mining_step: An integer, the training step in which the hard
      exampling mining kicks off. Note that we gradually reduce the mining
      percent to the top_k_percent_pixels. For example, if
      hard_example_mining_step = 100K and top_k_percent_pixels = 0.25, then
      mining percent will gradually reduce from 100% to 25% until 100K steps
      after which we only mine top 25% pixels.
    top_k_percent_pixels: A float, the value lies in [0.0, 1.0]. When its value
      < 1.0, only compute the loss for the top k percent pixels (e.g., the top
      20% pixels). This is useful for hard pixel mining.
    scope: String, the scope for the loss.

  Raises:
    ValueError: Label or logits is None.
  """
  if labels is None:
    raise ValueError('No label for softmax cross entropy loss.')

  for scale, logits in six.iteritems(scales_to_logits):
    loss_scope = None
    if scope:
      loss_scope = '%s_%s' % (scope, scale)

    if upsample_logits:
      # Label is not downsampled, and instead we upsample logits.
      logits = tf.image.resize_bilinear(
          logits,
          preprocess_utils.resolve_shape(labels, 4)[1:3],
          align_corners=True)
      scaled_labels = labels
    else:
      # Label is downsampled to the same size as logits.
      scaled_labels = tf.image.resize_nearest_neighbor(
          labels,
          preprocess_utils.resolve_shape(logits, 4)[1:3],
          align_corners=True)

    scaled_labels = tf.reshape(scaled_labels, shape=[-1])
    not_ignore_mask = tf.to_float(tf.not_equal(scaled_labels,
                                               ignore_label)) * loss_weight
    one_hot_labels = tf.one_hot(
        scaled_labels, num_classes, on_value=1.0, off_value=0.0)

    if top_k_percent_pixels == 1.0:
      # Compute the loss for all pixels.
      tf.compat.v1.losses.softmax_cross_entropy(
          one_hot_labels,
          tf.reshape(logits, shape=[-1, num_classes]),
          weights=not_ignore_mask,
          scope=loss_scope)
    else:
      logits = tf.reshape(logits, shape=[-1, num_classes])
      weights = not_ignore_mask
      with tf.name_scope(loss_scope, 'softmax_hard_example_mining',
                         [logits, one_hot_labels, weights]):
        one_hot_labels = tf.stop_gradient(
            one_hot_labels, name='labels_stop_gradient')
        pixel_losses = tf.nn.softmax_cross_entropy_with_logits_v2(
            labels=one_hot_labels,
            logits=logits,
            name='pixel_losses')
        weighted_pixel_losses = tf.multiply(pixel_losses, weights)
        num_pixels = tf.to_float(tf.shape(logits)[0])
        # Compute the top_k_percent pixels based on current training step.
        if hard_example_mining_step == 0:
          # Directly focus on the top_k pixels.
          top_k_pixels = tf.to_int32(top_k_percent_pixels * num_pixels)
        else:
          # Gradually reduce the mining percent to top_k_percent_pixels.
          global_step = tf.to_float(tf.train.get_or_create_global_step())
          ratio = tf.minimum(1.0, global_step / hard_example_mining_step)
          top_k_pixels = tf.to_int32(
              (ratio * top_k_percent_pixels + (1.0 - ratio)) * num_pixels)
        top_k_losses, _ = tf.nn.top_k(weighted_pixel_losses,
                                      k=top_k_pixels,
                                      sorted=True,
                                      name='top_k_percent_pixels')
        total_loss = tf.reduce_sum(top_k_losses)
        num_present = tf.reduce_sum(
            tf.to_float(tf.not_equal(top_k_losses, 0.0)))
        loss = _div_maybe_zero(total_loss, num_present)
        tf.losses.add_loss(loss)

这是他们使用的损失函数，你可以看到loss_weight 是 1.0。

not_ignore_mask = tf.to_float(tf.not_equal(scaled_labels,
                                               ignore_label)) * loss_weight

我想给负类赋予 0.2 的权重，给正类（预测）赋予 0.8 的权重。 有谁知道如何做到这一点或任何以前做过的回购/示例？

问候

【参考方案1】：

您可以在“utils”文件夹中的“train_utils.py”中更改权重。

在“def add_softmax_cross_entropy_loss_for_each_scale(...)”中

类似这样的：

for scale, logits in six.iteritems(scales_to_logits):
  loss_scope = None
  irgore_weight = 0
  label0_weight = 0.2 #I don't know your labels order...
  label1_weight = 0.8 #I don't know your labels order...

同时更改not_ignore_mask，像这样：

not_ignore_mask = tf.to_float(tf.equal(scaled_labels, 0)) * label0_weight + tf.to_float(tf.equal(scaled_labels, 1)) * label1_weight + tf.to_float(tf.equal(scaled_labels, ignore_label)) * irgore_weight

希望对你有帮助。

【讨论】：

我有这个问题，但是 utils/train_utils.py 脚本中没有 not_ignore_mask

嗨 Naser，我查看了 deeplab github，它确实不再是 not_ignore_mask on train_utils.py。但是，根据此评论 (github.com/tensorflow/models/issues/3739#issuecomment-600460636)，您似乎可以在 train.sh 脚本中添加权重作为选项。我现在无法测试，所以如果你尝试这个，如果它有效，我会很感激反馈，好吗？最好的问候！

您好 thiago，感谢您的回复。是的，我终于想出了如何解决这个问题。在 utils/train_utils.py 中有一个名为 loss_weight 的标志。在默认情况下，它的值为 1.0。但是您可以将其值更改为所需标签权重的列表。例如：在我的情况下，我有 2 个标签。所以 loss_weight 应该是这样的：loss_weight = [0.1,0,9] for lables_0 和 labels_1。

酷！感谢分享！

对不起，我犯了一个错误。 loss_weight 应该是这样的：loss_weight = [0.1,0.9] for lable_0 和 lable_1。