Paper简读 - ChatGPT相关的GPT-1GPT-2GPT-3

Posted 2023-03-03 SpikeKing

欢迎关注我的CSDN：https://spike.blog.csdn.net/
本文地址：https://blog.csdn.net/caroline_wendy/article/details/128909400

GPT、GPT-2、GPT-3：Generative Pre-trained Transformer，生成式预训练Transformer

• Wiki: https://en.wikipedia.org/wiki/GPT-3

• GPT-3 Demo: https://gpt3demo.com/

时间线：

1. Transformer, 2017.6, Attention is all you need
2. GPT, 2018.6, Improving Language Understanding by Generative Pre-Training: 使用Transformer的解码器，在没有标签的文本上，预训练模型
3. BERT, 2018.10, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding: Bidirectional Encoder Representations from Transformers，Transformer的编码器
4. GPT-2, 2019.2,