PyTorch笔记 - Position Embedding (Transformer/ViT/Swin/MAE)
Posted SpikeKing
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了PyTorch笔记 - Position Embedding (Transformer/ViT/Swin/MAE)相关的知识，希望对你有一定的参考价值。
欢迎关注我的CSDN：https://blog.csdn.net/caroline_wendy
本文地址：https://blog.csdn.net/caroline_wendy/article/details/128447794
Position Embedding(位置编码)
- Transformer
- 1d absolute
- sin/cos constant
- Vision Transformer
- 1d absolute
- trainable
- Swin Transformer
- 2d relative bias
- trainable
- Masked AutoEncoder
- 2d absolute
- sin/cos constant
Paper:
- Transformer - Attention Is All You Need
- ViT - An Image is Worth 16x16 Words Transformers for Image Recognition at Scale
- SwinTransformer - Hierarchical Vision Transformer using Shifte
以上是关于PyTorch笔记 - Position Embedding (Transformer/ViT/Swin/MAE)的主要内容，如果未能解决你的问题，请参考以下文章
PyTorch笔记 - Position Embedding (Transformer/ViT/Swin/MAE)