LLM-Quantization-Techniques-Absmax-Zeropoint-GPTQ-GGUF

Stable Diffusion and Flux in pure C/C++

llama and other large language models on iOS and MacOS offline using GGML library.

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization

Go manage your Ollama models

Suno AI's Bark model in C/C++ for fast text-to-speech generation

Self-evaluating interview for AI coders

Run inference on MPT-30B using CPU

Port of MiniGPT4 in C++ (4bit, 5bit, 6bit, 8bit, 16bit CPU inference with GGML)