auto-round

Production ready LLM model compression/quantization toolkit with hw accelerated inference support for both cpu/gpu via HF, vLLM, and SGLang.

Large Language Models for All, ? Cult and More, Stay in touch !

? X—LLM: Cutting Edge & Easy LLM Finetuning

An innovative library for efficient LLM inference via low-bit quantization

Run any Large Language Model behind a unified API

? Lightweight OpenAI drop-in replacement for Kubernetes

A guide about how to use GPTQ models with langchain

zero零训练llm调参

An OpenAI Compatible API which integrates LLM, Embedding and Reranker. 一个集成 LLM、Embedding 和 Reranker 的 OpenAI 兼容 API

ChatSakura：Open-source multilingual conversational model.（开源多语言对话大模型）