Computer vision
Theoretical Foundations of Knowledge Distillation: A Systematic Overview
Researchers at Purdue University have shed new light on the theoretical justification of knowledge distillation (KD), a widely used technique in machine learning that enables the transfer of predictive behavior from high-capacity teacher models to compact student models. With financial support from the National Science Foundation (NSF), the team conducted