Sparse Evolutionary Training

• (Mocanu et al., 2018; Liu et al., 2021b)
• which randomly initializes the sparse connectivity between layers randomly and dynamically adjusts the sparse connectivity via a parameter prune-and-grow scheme during the course of training
• The parameter prune-and-grow scheme allows the model’s sparse structure to gradually evolve, achieving better performance than naively training a static sparse network