Promptfoo

What is Promptfoo?

Promptfoo is a testing tool for teams building applications on large language models. The open-source command-line version runs evaluations on your prompts and models, with test cases defined in YAML and results viewed locally, and it also does red teaming: it generates attacks tailored to your app, such as direct and indirect prompt injection, jailbreaks, PII leaks, business rule violations and insecure tool use by agents. It works on RAG systems, agents and any live model endpoint, and it plugs into CI/CD pipelines like GitHub, GitLab and Jenkins so findings can appear in pull requests with remediation steps. You can start with npx promptfoo@latest redteam setup. The free Community edition is described as free forever, covers all evaluation features, all model providers and 10k red-team probes per month, and can run locally or be self-hosted. Enterprise, a hosted service, and Enterprise On-Premise add team sharing, monitoring, SSO, an API and a dedicated runner; both are custom-priced.

Use cases

• Red team an LLM app before release
• Test prompts against many models
• Scan agents for insecure tool use
• Catch prompt injection in CI/CD
• Check RAG apps for data leaks
• Track fixes for security findings

Who it's for

• AI engineers
• Security teams
• Developers
• Compliance teams

Why use Promptfoo

• Attacks are generated for your specific application instead of a generic fuzzing list.
• Tests live in config files and run in CI, so a regression shows up before it ships.
• The Community edition is free forever and can run entirely on your own machine.
• Results can be shared by URL, and existing open-source results work with the paid products.
• Findings map to frameworks such as OWASP and NIST for compliance reviews.

Promptfoo features

• LLM evaluations configured in YAML
• Automated red teaming for agents and RAG apps
• Direct and indirect prompt injection tests
• Jailbreak, PII leak and toxicity probes
• Support for all model providers
• CI/CD integration with findings in pull requests
• Vulnerability scanning with remediation guidance
• Run locally or self-host
• Team sharing, SSO and API on Enterprise
• Dedicated runner with Enterprise On-Premise

Promptfoo integrations

• GitHub
• GitLab
• Jenkins
• AWS
• Azure
• GCP

How to get started with Promptfoo

1. Run npx promptfoo@latest redteam setup and describe your target app or model.
2. Promptfoo generates context-aware attacks and sends probes to the target.
3. Review the results and the vulnerabilities found in the local web view.
4. Add the scan to your CI/CD pipeline so each pull request is tested.
5. Fix the issues using the remediation guidance and rerun to confirm.

Promptfoo limitations

• Community edition caps red teaming at 10k probes per month
• Continuous monitoring, SSO and API access are Enterprise features
• Enterprise and On-Premise prices are not published
• Some red-team plugins need inference for test generation and grading

Promptfoo pricing

Promptfoo is open source, so you can use and self-host it for free.

Promptfoo plans

• Community

  Free

  Free Forever

  • All LLM evaluation features
  • All model providers and integrations
  • Red teaming up to 10k probes per month
  • Run locally or self-host
  • Community support

• Enterprise

  Custom

  • Custom red teaming limits
  • Team sharing and collaboration
  • Continuous monitoring
  • SSO and permission profiles
  • API access
  • Managed cloud deployment
  • Priority support and SLA

• On-Premise

  Custom

  • All Enterprise features
  • Deployment on your own infrastructure
  • Complete data isolation
  • Dedicated runner
  • Assigned deployment engineer

Plans as listed on the official pricing page on 24 Oct 2026. Prices can change and may vary by country or exclude tax. Check the official pricing page.

Promptfoo alternatives

The main Promptfoo alternatives listed on Debuly are Braintrust, Langfuse, Openlayer, OpenLIT, Agent Governance Toolkit and Fiddler AI. The table compares pricing and platforms; order is A–Z.

1. 

   Agent Governance Toolkit

   Microsoft's open-source toolkit for enforcing policy, identity, sandboxing and audit records on autonomous AI agents before they reach production.

   Developer platformsAutomation and agents
   Open sourceVisit Agent Governance Toolkit (official site)
2. 

   Braintrust

   AI observability and evaluation platform for tracing LLM apps and agents, scoring outputs and catching regressions before release.

   Developer platformsCoding
   FreemiumVisit Braintrust (official site)
3. 

   Fiddler AI

   Control plane for enterprise AI agents: observability, evaluations, inline guardrails and governance for agentic and predictive models.

   Developer platformsAutomation and agents
   FreemiumVisit Fiddler AI (official site)
4. 

   Langfuse

   Open-source LLM observability and evaluation platform for tracing agents, managing prompts and scoring outputs.

   Developer platformsAutomation and agentsCoding
   FreemiumVisit Langfuse (official site)
5. 

   Openlayer

   Platform for testing, tracing and governing AI systems, with CI/CD evals, production monitoring and guardrails.

   Developer platformsAutomation and agents
   FreemiumVisit Openlayer (official site)
6. 

   OpenLIT

   Open-source, OpenTelemetry-native platform for tracing, evaluating and guarding AI agents, with cost and GPU monitoring

   Developer platformsCodingAutomation and agents
   Open sourceVisit OpenLIT (official site)

Promptfoo FAQ

Sources & verification

Facts on this page come from the official sources below, each with the date we read it. Written by Debuly; the vendor did not review this page.

Image credits

• Logo: from promptfoo.dev · fetched 24 Oct 2026 · Vendor trademark, used only to identify the product; removed on request
• Cover: none collected yet, so a generated tile is shown.

Spot an error? Report an issue with this listing. We also remove a logo or image on the owner's request.