NTU 课程笔记：self-supervised learning

Posted 2021-12-12 UQI-LIUWJ

1 为什么需要 self-supervision？

• 为每个新的任务都创建一个新的数据集，开销是很大的
• 有些领域标注是很稀缺的（比如医疗领域）
• 有些领域 数据集太大，来不及标注
• self-supervision 很像婴儿学习的方式

2 什么是self-supervison

• A form of unsupervised learning where the data provides the supervision（用数据本身创造label）
• In general, withhold some part of the data, and task the network with predicting it
• The task defines a proxy loss, and the network is forced to learn what we really
  care about, e.g. a semantic representation, in order to solve it（proxy loss~~text loss）