Popular GitHub Repositories for Model Inference

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

Resources of our survey paper "Optimizing Edge AI: A Comprehensive Survey on Data, Model, and System Strategies"

CLIP as a service - Embed image and sentences, object recognition, visual reasoning, image classification and reverse image search

Large-scale Auto-Distributed Training/Inference Unified Framework | Memory-Compute-Control Decoupled Architecture | Multi-language SDK & Heterogeneous Hardware Support

EmbeddedLLM: API server for Embedded Device Deployment. Currently support CUDA/OpenVINO/IpexLLM/DirectML/CPU

Build self-contained local LLM powered SaaS and APIs with no external AI API subscriptions required.

Генерация описаний к изображениям с помощью различных архитектур нейронных сетей

Accelerating AI Training and Inference from Storage Perspective (Must-read Papers on Storage for AI)