Popular GitHub Repositories for Local Inference

High-speed Large Language Model Serving for Local Deployment

[ICLR'25] Fast Inference of MoE Models with CPU-GPU Orchestration

Tool for test diferents large language models without code.

Local LLM Inference Library

LLM chatbot example using OpenVINO with RAG (Retrieval Augmented Generation).

Script which takes a .wav audio file, performs speech-to-text using OpenAI/Whisper, and then, using Llama3, summarization and action point from the transcript generated