Teacher Student Architecture

• The gap is further reduced by residual learning, i.e., the assistant structure is used to learn the residual error (Gao et al., 2021).