All Repos
PromptsTypeScriptMIT

promptfoo

promptfoo/promptfoo · Updated

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

promptfoo/promptfoo is a TypeScript repository in the Prompts category with 26,583 stars and 2,607 forks on GitHub, released under the MIT license, last updated today.

#ci#ci-cd#cicd#evaluation#evaluation-framework#llm#llm-eval#llm-evaluation
View on GitHub 27kGitHub stars

How to install promptfoo in your AI client

Setup for Claude Code, Claude Desktop, Cursor, Codex, Windsurf, Cline and VS Code — copy-paste ready.

How to install promptfoo in Claude CodeAnthropic's agentic CLI

  1. Clone & open

    bash
    git clone https://github.com/promptfoo/promptfoo.git
cd promptfoo
claude

  2. Let Claude read it

    Ask “/init” to generate a CLAUDE.md, then have Claude explain, run, or extend the codebase. It reads files on demand.

Documentation

Full README

Promptfoo: LLM evals & red teaming

<code>promptfoo</code> is a CLI and library for evaluating and red-teaming LLM-based apps. Stop using trial-and-error... start shipping secure, reliable agents

Website · Getting Started · Red Teaming · Documentation · Discord

Promptfoo is now part of OpenAI. Promptfoo remains open source and MIT licensed. Read the company update.

Quick Start

sh
npm install -g promptfoo
promptfoo init --example getting-started

Also available via brew install promptfoo and pip install promptfoo. Or use npx promptfoo@latest to run any command without installing.

Most LLM providers require an API key. Set yours as an environment variable:

sh
export OPENAI_API_KEY=sk-abc123

Run an eval and view results:

sh
cd getting-started
promptfoo eval
promptfoo view

See [Getting Started](https://www.promptfoo.dev/docs/getting-started/) (evals) or [Red Teaming](https://www.promptfoo.dev/docs/red-team/) (vulnerability scanning) for more!

What can you do with Promptfoo?

  • Test your prompts and models with automated evaluations
  • Secure your LLM apps with red teaming and vulnerability scanning
  • Compare models side-by-side (OpenAI, Anthropic, Azure, Bedrock, Ollama, and more)
  • Automate checks in CI/CD
  • Review pull requests for LLM-related security and compliance issues with code scanning
  • Share results with your team

More Prompts repositories

Catalog data refreshed . Something wrong? Report it.