Popular GitHub Repositories for Gateway

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]

Intelligent (edge and LLM) proxy for agents. Move faster by letting Arch handle the pesky heavy lifting in building agentic apps. Fast ?? task clarification, agent routing, seamless integration of prompts with tools for common tasks, and unified access and observability of LLMs.

?云原生、超高性能 AI&API网关，LLM API 管理、分发系统、开放平台，支持所有AI API，不限于OpenAI、Azure、Anthropic Claude、Google Gemini、DeepSeek、字节豆包、ChatGLM、文心一言、讯飞星火、通义千问、360 智脑、腾讯混元等主流模型，统一 API 请求和返回，API申请与审批，调用统计、负载均衡、多模型灾备。一键部署，开箱即用。Cloud native, ultra-high performance AI&API gateway, LLM API management, distribution system, open platform, supporting all AI APIs.

Gateway for secure & reliable communications with OpenAI and other LLM providers

? A open blazing-fast simple model gateway for rapid development of production GenAI apps

Modular, open source LLMOps stack that separates concerns: LiteLLM unifies LLM APIs, manages routing and cost controls, and ensures high-availability, while Langfuse focuses on detailed observability, prompt versioning, and performance evaluations.

Gateway and load balancer to your LLM inference endpoints

MultiversX AI Agent Kit

A powerful, AI Gateway designed from scratch for AI

An open-source, high-performance gateway unifying multiple LLM providers, from local solutions like Ollama to major cloud providers such as OpenAI, Groq, Cohere, Anthropic and Cloudflare.

Practical use of Azure API Management's AI Gateway to manage Azure OpenAI operations.

Asterisk gateway interface (AGI) library

Built for demanding AI workflows, this gateway offers low-latency, provider-agnostic access, ensuring your AI applications run smoothly and quickly.

Gateway to control LLM API/SDK calls. Supports access to OpenAI, Azure, Anthropic and Bedrock calls

This is a simple AI gateway implemented in Python, which has functions such as load balancing, error alarm, disaster recovery and backup.