athina-evals

Supercharge Your LLM Application Evaluations ?

A framework for few-shot evaluation of language models.

The LLM Evaluation Framework

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including OpenAI Agents SDK, CrewAI, Langchain, Autogen, AG2, and CamelAI

AutoRAG: An Open-Source Framework for Retrieval-Augmented Generation (RAG) Evaluation & Optimization with AutoML-Style Automation

The easiest tool for fine-tuning LLM models, synthetic data generation, and collaborating on datasets.

SuperCLUE: 中文通用大模型综合性基准 | A Benchmark for Foundation Models in Chinese

《大模型白盒子构建指南》：一个全手搓的Tiny-Universe

A unified evaluation framework for large language models