Promptfoo

An LLM prompt testing library.

CommonProductProductivityLLMprompt testing

Promptfoo is a library for evaluating LLM prompt quality and conducting testing. It helps you create test cases, set evaluation metrics, and integrate with existing testing and CI workflows. Promptfoo also offers a Web Viewer to easily compare different prompts and model outputs. It is used in LLM applications serving over 10 million users.

Create a list of test cases
Use built-in evaluation metrics or custom metrics
Compare different prompts and model outputs

Used for evaluating LLM prompt quality and conducting testing.

Evaluate if the text generated by an LLM is accurate
Test the impact of different prompts on model output
Compare the performance of different models on the same prompt

Visit

Promptfoo Visit Over Time

Monthly Visits

71550

Bounce Rate

49.30%

Page per Visit

1.9

Visit Duration

00:00:55

Promptfoo Visit Trend

Promptfoo Visit Geography

Promptfoo Traffic Sources

Promptfoo Alternatives

GraphRAG Visualizer — A web-based tool for visualizing and exploring Microsoft's GraphRAG framework.

Programming•gpt•graph-visualization

108

Flow by Laminar — A lightweight task engine that builds stateful AI agents and supports multiple task processing.

Programming•workflow•ai

ElevenLabs Conversational AI — Rapid deployment of a conversational AI agent

InternationalSelection•AI Agent•Speech-to-Text

210

AI-Data-Analysis-MultiAgent — AI-Driven Multi-Agent Data Analysis System

Productivity•python•agent

144

llms.txt Generator — Generate an llms.txt file to facilitate the use of your website by LLMs during inference.

Programming•LLM•Chatbot

162

aisuite — Unified interface for accessing multiple generative AI services.

Programming•Generative AI•LLM

108

Model Context Protocol — MCP is an open protocol for seamless integration of large language model applications with external data sources and tools.

InternationalSelection•LLM•Integration

ollama-ebook-summary — A tool for summarizing long texts, automatically converting books and documents into key point notes.

Writing•Long Text Summarization•Automation

Extractous — A fast and efficient tool for unstructured data extraction

Programming•nlp•rust

126

HumanLayer — API and SDK for human-in-the-loop feedback, inputs, and approval for AI agents.

Programming•API•SDK

Sidecar — Sidecar is the AI brain of the Aide editor, working in tandem with the editor.

Programming•rust•typescript

Mistral Moderation API — AI content moderation service to ensure downstream deployment safety.

Others•Content Moderation•Multilingual Support

108

askrepo — Utilizes LLM to read source code and provide answers to questions.

Programming•Code Analysis•LLM

pdf-extract-api — An API for high-precision conversion of images or PDFs to Markdown text or structured JSON documents.

Productivity•api•pdf

156

Chat Nio — Leading all-in-one enterprise solution provider for LLM in the domestic market.

ChineseSelection•LLM•Enterprise Solutions

738

Laminar.ai — An open-source full-stack platform that provides support for building top-tier LLM products.

Programming•AI Engineering•LLM

168

Dabarqus — A tool for integrating private data with AI large language models.

Programming•RAG•LLM

GitHub to LLM Converter — Convert GitHub links into a format suitable for LLM.

Programming•GitHub•LLM

Meta Lingua — Efficient repository for large language model (LLM) research.

Programming•LLM•PyTorch

Prompt Engineering — A comprehensive resource library for Prompt Engineering techniques.

Productivity•Prompt Engineering•Machine Learning

Llama-3.1-Nemotron-70B-Instruct — A large language model customized by NVIDIA to enhance the supportiveness of query answering.

Productivity•NVIDIA•large language model

198

rag-chatbot — A chatbot that can interact with multiple PDF files locally.

Programming•chatbot•question-answering

150

LlamaIndex.TS — A framework for building LLM applications

Programming•LLM•Natural Language Processing

ComfyUI LLM Party — A collection of LLM workflow nodes developed based on the ComfyUI frontend.

Programming•LLM•Workflow

240

gptme — A personal AI assistant in the terminal with local tools.

Programming•cli•chatbot

126

promptic — A simple yet powerful Python library for using large language models (LLMs).

Programming•Python•LLM

108

Best AI Websites & Tools