quantized-lm-confidence

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

[ICLR 2020] Once for All: Train One Network and Specialize it for Efficient Deployment

A curated list for Efficient Large Language Models

PaddleSlim is an open-source library for deep model compression and architecture search.

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

A Github Action that automatically compresses JPEGs, PNGs and WebPs in Pull Requests.

[ICLR 2019] ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware

A PyTorch library and evaluation platform for end-to-end compression research

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM