Isotropic Architectures

• (of an object or substance) having a physical property which has the same value when measured in different directions.
• And precisely that is what an isotropic architecture is. Isotropic architectures do not produce pyramid shaped data transformations, but rather fixed ones where data does not change in shape and size
• In other (simpler) words, when you take a look at the value going through an isotropic network, it doesn’t change in size.
• Like Transformer , MLP-Mixer