Popular GitHub Repositories for Gateway

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]

Intelligent (edge and LLM) proxy for agents. Move faster by letting Arch handle the pesky heavy lifting in building agentic apps. Fast ?? task clarification, agent routing, seamless integration of prompts with tools for common tasks, and unified access and observability of LLMs.

?云原生、超高性能 AI&API网关，LLM API 管理、分发系统、开放平台，支持所有AI API，不限于OpenAI、Azure、Anthropic Claude、Google Gemini、DeepSeek、字节豆包、ChatGLM、文心一言、讯飞星火、通义千问、360 智脑、腾讯混元等主流模型，统一 API 请求和返回，API申请与审批，调用统计、负载均衡、多模型灾备。一键部署，开箱即用。Cloud native, ultra-high performance AI&API gateway, LLM API management, distribution system, open platform, supporting all AI APIs.

Gateway for secure & reliable communications with OpenAI and other LLM providers

API 200 is an open source API gateway to simplify 3rd-party integrations. Import endpoints, set up caching, retries, and mocks. Access all services via one URL. Monitor logs, track errors, and get alerts on API incidents.

? A open blazing-fast simple model gateway for rapid development of production GenAI apps

Modular, open source LLMOps stack that separates concerns: LiteLLM unifies LLM APIs, manages routing and cost controls, and ensures high-availability, while Langfuse focuses on detailed observability, prompt versioning, and performance evaluations.

A powerful, AI Gateway designed from scratch for AI

Gateway and load balancer to your LLM inference endpoints

MultiversX AI Agent Kit

An open-source, high-performance gateway unifying multiple LLM providers, from local solutions like Ollama to major cloud providers such as OpenAI, Groq, Cohere, Anthropic and Cloudflare.

Practical use of Azure API Management's AI Gateway to manage Azure OpenAI operations.

Asterisk gateway interface (AGI) library

Built for demanding AI workflows, this gateway offers low-latency, provider-agnostic access, ensuring your AI applications run smoothly and quickly.

Gateway to control LLM API/SDK calls. Supports access to OpenAI, Azure, Anthropic and Bedrock calls

This is a simple AI gateway implemented in Python, which has functions such as load balancing, error alarm, disaster recovery and backup.

This project provides a central gateway to manage multiple MCP (Model Context Protocol) servers, preventing the need to configure and run duplicate server processes for each LLM client (like Cline, Cursor, etc.). Connect your LLM client to the single gateway-client endpoint provided by this project to access tools from all your managed MCP servers

轻量级 AI 推理网关——本地模型注册中心和参数转换器（Python SDK），可选 Envoy 代理处理器和 FastAPI 注册服务器部署方案。