Introduction
Promptfoo is a developer-focused local tool for testing and red-teaming LLM applications. It helps teams automate prompt and model evaluations, run security-focused red team tests, and integrate checks into CI.
Key Features
- Automated prompt & model evaluation matrices
- Red teaming and vulnerability scanning
- Support for multiple model providers and CI integration
- Visual reports and historical result storage
Use Cases
- Automated model/prompt regression tests in CI/CD
- Pre-deployment security audits and red team checks
- Team-shared evaluation configs and reports
Technical Highlights
- Open-source, CLI-first, runs locally to protect sensitive prompts
- Supports multi-provider comparisons (OpenAI, Anthropic, Ollama, etc.)
- Integrates with common CI systems (GitHub Actions, etc.)