---

toc: true title: Distillation Token

tags: [‘temp’]

---

Distillation Token

• A learned vector that flows through the network along with the transformed Image Data
• cues the model for its distillation output, which can differ from its class output
• Specific to Transformers