Timo

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM Observability all in one place.

Evaluate your LLM's response with Prometheus and GPT4 ?

? Agent-as-a-Judge and DevAI dataset

xVerify: Efficient Answer Verifier for Large Language Model Evaluations

CodeUltraFeedback: aligning large language models to coding preferences

This is the repo for the survey of Bias and Fairness in IR with LLMs.

Code and data for Koo et al's ACL 2024 paper "Benchmarking Cognitive Biases in Large Language Models as Evaluators"

A set of examples demonstrating how to evaluate Generative AI augmented systems using traditional information retrieval and LLM-As-A-Judge validation techniques

LLM-as-judge evals as Semantic Kernel Plugins

根信号评估平台的 MCP（主控处理器）