Promptfoo

An LLM prompt testing library.

CommonProductProductivityLLMprompt testing

Promptfoo is a library for evaluating LLM prompt quality and conducting testing. It helps you create test cases, set evaluation metrics, and integrate with existing testing and CI workflows. Promptfoo also offers a Web Viewer to easily compare different prompts and model outputs. It is used in LLM applications serving over 10 million users.

Create a list of test cases
Use built-in evaluation metrics or custom metrics
Compare different prompts and model outputs

Used for evaluating LLM prompt quality and conducting testing.

Evaluate if the text generated by an LLM is accurate
Test the impact of different prompts on model output
Compare the performance of different models on the same prompt

Visit

Promptfoo Visit Over Time

Monthly Visits

67467

Bounce Rate

44.02%

Page per Visit

3.1

Visit Duration

00:02:14

Promptfoo Visit Trend

Promptfoo Visit Geography

Promptfoo Traffic Sources

Promptfoo Alternatives

Inductor — Evaluate, ensure, and improve the quality of LLM applications

Programming•Large Language Model•LLM

Extractous — A fast and efficient tool for unstructured data extraction

Programming•nlp•rust

HumanLayer — API and SDK for human-in-the-loop feedback, inputs, and approval for AI agents.

Programming•API•SDK

Sidecar — Sidecar is the AI brain of the Aide editor, working in tandem with the editor.

Programming•rust•typescript

Mistral Moderation API — AI content moderation service to ensure downstream deployment safety.

Others•Content Moderation•Multilingual Support

askrepo — Utilizes LLM to read source code and provide answers to questions.

Programming•Code Analysis•LLM

pdf-extract-api — An API for high-precision conversion of images or PDFs to Markdown text or structured JSON documents.

Productivity•api•pdf

Chat Nio — Leading all-in-one enterprise solution provider for LLM in the domestic market.

ChineseSelection•LLM•Enterprise Solutions

174

Laminar.ai — An open-source full-stack platform that provides support for building top-tier LLM products.

Programming•AI Engineering•LLM

144

Prompt Engineering — A comprehensive resource library for Prompt Engineering techniques.

Productivity•Prompt Engineering•Machine Learning

Llama-3.1-Nemotron-70B-Instruct — A large language model customized by NVIDIA to enhance the supportiveness of query answering.

Productivity•NVIDIA•large language model

174

rag-chatbot — A chatbot that can interact with multiple PDF files locally.

Programming•chatbot•question-answering

120

LlamaIndex.TS — A framework for building LLM applications

Programming•LLM•Natural Language Processing

ComfyUI LLM Party — A collection of LLM workflow nodes developed based on the ComfyUI frontend.

Programming•LLM•Workflow

150

gptme — A personal AI assistant in the terminal with local tools.

Programming•cli•chatbot

108

promptic — A simple yet powerful Python library for using large language models (LLMs).

Programming•Python•LLM

Epsilla — Build production-ready LLM applications without coding.

InternationalSelection•RAG•LLM

138

MemoryScope — A powerful and flexible long-term memory system for LLM chatbots.

Programming•LLM•Long Term Memory

102

Weavel — AI Prompt Engineer, optimizing large language model applications

Productivity•Dataset Management•Evaluation Tools

CyberScraper 2077 — A futuristic AI-driven web scraping tool.

Programming•Web scraping•OpenAI

156

RAG_Techniques — A revolutionary collection of Retrieval-Augmented Generation system technologies.

Programming•RAG•NLP

150

Data-Juicer — A one-stop data processing system that provides high-quality data for large language models.

Productivity•Machine Learning•Data Science

246

AI Artifacts — Open-source interface for Anthropic's Claude Artifacts

Programming•react•javascript

114

GitHub Models — Next-generation AI engineers building on GitHub

Programming•Generative AI•GitHub Copilot

396

IncarnaMind — Engage in conversations with your documents using large language models like GPT.

Productivity•Machine Learning•PDF

156

Composio — A toolkit for AI agents enabling complex task handling.

Programming•javascript•python

Best AI Websites & Tools