WAVER

A large scale study of Knowledge Distillation.

Awesome Knowledge Distillation

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

Collection of AWESOME vision-language models for vision tasks

"Effective Whole-body Pose Estimation with Two-stages Distillation" (ICCV 2023, CV4Metaverse Workshop)

A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility

A curated list for Efficient Large Language Models

OpenMMLab Model Compression Toolbox and Benchmark.

Improving Convolutional Networks via Attention Transfer (ICLR 2017)

NLP DNN Toolkit - Building Your NLP DNN Models Like Playing Lego