---

toc: true title: Self Supervised Vision Transformers

tags: [‘temp’]

---

Self Supervised Vision Transformers

• An Empirical Study of Training Self-Supervised Vision Transformers
  • recipes for Vision Transformer are yet to be built
  • Self Supervised
  • instability is a major issue that degrades accuracy, and it can be hidden by apparently good results
  • improved when training is made more stable
  • MoCO v3, a framework which offers an incremental improvement of MoCO
  •