Popular GitHub Repositories for Observability

Stunning status pages, batteries included!

?️ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more.

Easy token price estimates for 400+ LLMs. TokenOps.

A collection of production-ready Generative AI Agent templates built for Google Cloud. It accelerates development by providing a holistic, production-ready solution, addressing common challenges (Deployment & Operations, Evaluation, Customization, Observability) in building and deploying GenAI agents.

一款多语言的API，满足您所有可观测性需求。轻量、快速，可替代LGTM :rocket: 独立的OpenTelemetry、Loki、Prometheus、Tempo、Pyroscope兼容 :star:

The ultimate LLM Ops platform - Monitoring, Analytics, Evaluations, Datasets and Prompt Optimization ?

The production toolkit for LLMs. Observability, prompt management and evaluations.

Distribute and run AI workloads magically in Python, like PyTorch for ML infra.

? LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). ? Extracts signals from prompts & responses, ensuring safety & security. ?? Features include text quality, relevance metrics, & sentiment analysis. ? A comprehensive tool for LLM observability. ?

Langtrace ? is an open-source, Open Telemetry based end-to-end observability tool for LLM applications, providing real-time tracing, evaluations and metrics for popular LLMs, LLM frameworks, vectorDBs and more.. Integrate using Typescript, Python. ???

LangSmith Client SDK Implementations

Sister project to OpenLLMetry, but in Typescript. Open-source observability for your LLM application, based on OpenTelemetry

Markdown for the AI era

Foremast adds application resiliency to Kubernetes by leveraging machine learnt patterns of application health to keep applications healthy and stable

? Langfuse documentation -- Langfuse is the open source LLM Engineering Platform. Observability, evals, prompt management, playground and metrics to debug and improve LLM apps

A powerful AI observability framework that provides comprehensive insights into agent interactions across platforms, enabling developers to monitor, analyze, and optimize AI-driven applications with minimal integration effort.

Create custom DevOps AI agents that understand and manage your infrastructure.

Create, List, Update, Delete Amazon EKS clusters. Deploy and manage software on EKS. Run distributed model training and inference examples.

? Langfuse JS/TS SDKs - Instrument your LLM app and get detailed tracing/observability. Works with any LLM or framework

Jupyter notebooks for course Building and Evaluating Advanced RAG Applications, taught by Jerry Liu (Co-founder and CEO of LlamaIndex) and Anupam Datta (Co-founder and chief scientist of TruEra).