Popular GitHub Repositories for Model Distillation

Awesome Knowledge Distillation

? PyTorch Implementation of "Progressive Distillation for Fast Sampling of Diffusion Models(v-diffusion)"

Our open source implementation of MiniLMv2 (https://aclanthology.org/2021.findings-acl.188)

The Codebase for Causal Distillation for Language Models (NAACL '22)

AI Community Tutorial, including: LoRA/Qlora LLM fine-tuning, Training GPT-2 from scratch, Generative Model Architecture, Content safety and control implementation, Model distillation techniques, Dreambooth techniques, Transfer learning, etc for practice with real project!

Repository for the publication "AutoGraph: Predicting Lane Graphs from Traffic"

Awesome Deep Model Compression

[Master Thesis] Research project at the Data Analytics Lab in collaboration with Daedalean AI. The thesis was submitted to both ETH Zürich and Imperial College London.