AbstractWe investigate the design aspects of feature distillation methods achieving network compression and propose a no