Promptfoo

An LLM prompt testing library.

CommonProductProductivityLLMprompt testing

Promptfoo is a library for evaluating LLM prompt quality and conducting testing. It helps you create test cases, set evaluation metrics, and integrate with existing testing and CI workflows. Promptfoo also offers a Web Viewer to easily compare different prompts and model outputs. It is used in LLM applications serving over 10 million users.

Create a list of test cases
Use built-in evaluation metrics or custom metrics
Compare different prompts and model outputs

Used for evaluating LLM prompt quality and conducting testing.

Evaluate if the text generated by an LLM is accurate
Test the impact of different prompts on model output
Compare the performance of different models on the same prompt

Visit

Promptfoo Visit Over Time

Monthly Visits

57941

Bounce Rate

46.97%

Page per Visit

2.9

Visit Duration

00:02:34

Promptfoo Visit Trend

Promptfoo Visit Geography

Promptfoo Traffic Sources

Promptfoo Alternatives

Inductor — Evaluate, ensure, and improve the quality of LLM applications

Programming•Large Language Model•LLM

Mistral Moderation API — AI content moderation service to ensure downstream deployment safety.

Others•Content Moderation•Multilingual Support

askrepo — Utilizes LLM to read source code and provide answers to questions.

Programming•Code Analysis•LLM

pdf-extract-api — An API for high-precision conversion of images or PDFs to Markdown text or structured JSON documents.

Productivity•api•pdf

Chat Nio — Leading all-in-one enterprise solution provider for LLM in the domestic market.

ChineseSelection•LLM•Enterprise Solutions

162

Laminar.ai — An open-source full-stack platform that provides support for building top-tier LLM products.

Programming•AI Engineering•LLM

108

Prompt Engineering — A comprehensive resource library for Prompt Engineering techniques.

Productivity•Prompt Engineering•Machine Learning

Llama-3.1-Nemotron-70B-Instruct — A large language model customized by NVIDIA to enhance the supportiveness of query answering.

Productivity•NVIDIA•large language model

144

rag-chatbot — A chatbot that can interact with multiple PDF files locally.

Programming•chatbot•question-answering

LlamaIndex.TS — A framework for building LLM applications

Programming•LLM•Natural Language Processing

ComfyUI LLM Party — A collection of LLM workflow nodes developed based on the ComfyUI frontend.

Programming•LLM•Workflow

126

gptme — A personal AI assistant in the terminal with local tools.

Programming•cli•chatbot

102

promptic — A simple yet powerful Python library for using large language models (LLMs).

Programming•Python•LLM

Epsilla — Build production-ready LLM applications without coding.

InternationalSelection•RAG•LLM

132

MemoryScope — A powerful and flexible long-term memory system for LLM chatbots.

Programming•LLM•Long Term Memory

Weavel — AI Prompt Engineer, optimizing large language model applications

Productivity•Dataset Management•Evaluation Tools

CyberScraper 2077 — A futuristic AI-driven web scraping tool.

Programming•Web scraping•OpenAI

144

RAG_Techniques — A revolutionary collection of Retrieval-Augmented Generation system technologies.

Programming•RAG•NLP

144

Data-Juicer — A one-stop data processing system that provides high-quality data for large language models.

Productivity•Machine Learning•Data Science

234

AI Artifacts — Open-source interface for Anthropic's Claude Artifacts

Programming•react•javascript

108

GitHub Models — Next-generation AI engineers building on GitHub

Programming•Generative AI•GitHub Copilot

372

IncarnaMind — Engage in conversations with your documents using large language models like GPT.

Productivity•Machine Learning•PDF

144

Composio — A toolkit for AI agents enabling complex task handling.

Programming•javascript•python

RouteLLM — A framework that saves costs on LLM without sacrificing quality.

Programming•LLM•Cost Optimization

Awan LLM — An unlimited token, unrestricted, cost-effective LLM inference API platform.

InternationalSelection•LLM•API

270

vLLM — Fast and Easy-to-Use LLM Inference and Serving Platform

InternationalSelection•LLM•Inference

288

Best AI Websites & Tools