Popular GitHub Repositories for Evaluations

Langtrace ? is an open-source, Open Telemetry based end-to-end observability tool for LLM applications, providing real-time tracing, evaluations and metrics for popular LLMs, LLM frameworks, vectorDBs and more.. Integrate using Typescript, Python. ???

Python client library for improving your LLM app accuracy

A framework to build scenario simulation projects where human and LLM based agents can participant in, with a user-friendly web UI to visualize simulation, support automatically evaluation on agent action level.

The Python SDK for Asteroid, the platform for make your AI agent safe and reliable

NLP tool for wide-range model reliability evaluations

LLM evaluation framework