Why promptfoo security hardening matters before the first shared run
Hardening a promptfoo deployment means controlling where secrets, prompts, model responses and attack payloads are stored, who can reach them, and how they are backed up and removed. The question usually arrives once a team that has run promptfoo locally for evals and red teaming decides to share results, and at that point the data it holds has grown to include provider API keys, eval datasets, prompt variants and the adversarial outputs produced by red team runs. Each of these needs a defined home, an owner and a deletion path before the first shared run, not after. Promptfoo runs on the machine that starts it and talks directly to the model provider. The documentation states that evals run locally and call the LLM directly. That means the hardening burden sits with the team's own laptops, CI runners and any self-hosted host, rather than with a vendor-hosted service. Provider keys sit in environment variables or configuration on those machines, and red team outputs can contain harmful test content that should not land in general-purpose logs, chat channels or artifact stores with broad read access. Ask each team which machines and pipelines currently hold provider keys and where result files are written.
The difficulty is that the self-hosted app does not supply most of the controls a shared instance would need. The Express server stores data in a local SQLite database that cannot be shared across replicas, and the self-hosted edition has no RBAC, no horizontal scaling and no built-in authentication or SSO. The project does not recommend it for production use. The practical consequence is that access control, encryption at rest, backup, retention and network exposure have to come from the surrounding infrastructure: the host operating system, the container platform, the reverse proxy, the storage layer and the secrets manager. A team that assumes the application will handle these will discover the gap after data has already been written. Two related topics are covered elsewhere on this page's sibling set. SSO and access control, including what the Enterprise edition adds for teams-based access and custom roles, are addressed on the promptfoo SSO and access control page, and the sequencing of a rollout is covered in the implementation plan. This section sets the boundary: until the team has decided where the SQLite file lives, who can read it, how it is backed up and how records are removed, a shared run would expose data the team has not yet classified. Those decisions are the first items to put on the hardening checklist.
Which promptfoo components hold secrets and sensitive data
Five components in promptfoo hold keys, prompts, model responses or attack data that can expose credentials or how an application behaves. The Community edition is free and includes all evaluation features, providers and red teaming with 10k probes per month, and these hardening steps apply to any deployment model.
Provider API key configuration
Provider API keys usually reach promptfoo through environment variables or through config files that reference them. Keys grant billable access to model APIs, so a leaked key can cost money and expose the account behind it. Check whether any key is written literally into a configuration file committed to version control, which keys each developer or pipeline holds, and who can read the environment or shell history of the host running evals. Confirm the exact variable names and config file locations for the version you install.
Local SQLite eval database
Self-hosted promptfoo stores eval results in a local SQLite database. Rows can contain full prompts, model responses and any test data placed in test cases, so the file is sensitive even though it looks like an ordinary data file. Engineers who run evals and operators who back up or copy the host usually touch it. Confirm the database path for your version, restrict file permissions to the service account, and plan backups with the knowledge that the database cannot be shared across replicas.
Express server for UI and API
The self-hosted app runs as an Express server that serves both the web UI and the API. Anyone who can reach the port can view stored evaluations unless a reverse proxy or network control sits in front of it. The self-hosting documentation lists no built-in authentication, RBAC or SSO for the self-hosted app, so access control is the operator's responsibility. Check which interface the server binds to, whether a load balancer or ingress exposes it, and which users can reach it from their browsers.
Red team output files
Red team runs write output that records attack probes, the responses models gave and grading results. Probes often include jailbreak attempts, prompt injection payloads and simulated personal data, which makes these files more sensitive than typical test output. Security engineers and the people who review findings usually handle them, and they are often copied into tickets or shared drives. Confirm where your version writes these files, set a retention period, and limit who can export or share them.
CI/CD secret stores
Pipelines that run promptfoo usually inject provider keys from a CI/CD secret store. Depending on the platform's settings, those secrets may be available to every job in a repository, including jobs triggered by pull requests from contributors. Check which jobs receive model API keys, whether logs echo configuration or environment values, and whether artifacts uploaded from eval runs contain stored responses. Pin the promptfoo version used in pipelines so that upgrades go through review.
Decisions to make before hardening a promptfoo deployment
This table compares the main deployment and data-handling choices for promptfoo, with when each option fits and what it costs to run. Read each row as a decision: pick the option that matches your users and data, then accept the operational cost listed beside it.
| Decision | Option and when it fits | Operational cost |
|---|---|---|
| Where promptfoo runs | Local-only on engineer workstations fits small teams and early evals where each engineer runs their own suites. A shared self-hosted instance fits teams that need one place to review results. Enterprise or On-Premise fits teams that need shared access control and vendor support. | Local runs leave no central audit trail. A shared self-hosted instance needs a host, backups and access control you provide. Enterprise and On-Premise add licensing at custom pricing. |
| Where provider keys are injected | Developer shell variables fit local-only use with a single trusted user. A CI secret store fits scheduled and pull request runs. An external secrets manager fits shared or regulated environments that need central rotation. | Shell variables are hard to rotate and easy to leak into history or logs. CI secret stores need per-pipeline setup. External managers add integration work and a dependency on that service. |
| Where the SQLite file lives | Encrypted local disk fits single-user workstations where the laptop already has full-disk encryption. A managed volume with encryption and snapshots fits a shared self-hosted instance. | Local files are lost with the device unless backed up. Managed volumes need backup and restore testing, and a single-node file cannot be shared across replicas. |
| Reverse proxy with authentication in front of the app | Put a proxy with authentication in front of any self-hosted instance reachable beyond one machine. The self-hosted app has no built-in authentication or RBAC, so this is the only access gate. | You maintain the proxy, its certificates and its identity integration. Access is all-or-nothing per user unless the proxy adds finer rules. |
| Retention of eval and red team outputs | Short retention fits exploratory runs that contain only synthetic prompts. Longer retention fits results needed as evidence for release sign-off or audit requests. | Longer retention grows the SQLite file and backups, and stored outputs may contain sensitive model responses. Set a written retention period and a deletion routine. |
| Redacting or excluding sensitive datasets | Redact or exclude personal, customer or regulated data before runs when the eval goes to a third-party model. Use full datasets only where the provider agreement and internal policy allow it. | Redaction scripts need ownership and review. Excluding data can lower test coverage, so confirm with the risk owner what coverage remains. |
| Multi-user design | A single-writer approach, with one designated instance that writes results and others that read exports, fits teams without Enterprise. A plan to move to Enterprise or On-Premise fits teams that need concurrent multi-user work. | The self-hosted app cannot be shared across replicas, so adding replicas will not scale it. Enterprise adds SSO and teams-based access control, covered on the SSO and access control page. |
How to harden promptfoo step by step
This sequence takes a self-hosted promptfoo instance from default settings to one where provider keys, the SQLite database, eval outputs and CI logs are controlled by the team that runs it. Each step closes with a check that confirms the control works in your environment.
Inventory where keys and data appear
Before changing anything, the team that operates the instance lists every place provider API keys, eval data and red team outputs can appear: config files, environment files, shell history, CI variables, the SQLite database, exported reports and container images. Record an owner for each location and whether it is committed to version control. Run a secret scanner over the repository and the image to find keys already stored. Check: the inventory lists each location with an owner, and the scan results are saved as the baseline for the checks in later steps. Validate every scanner and command against your installed version before relying on its output.
Move provider keys into a secret store
Remove provider keys from promptfoo configuration files and load them at runtime from environment variables supplied by a secret manager or your orchestrator's secret mechanism. Config files should reference variable names only. Rotate any key that was ever committed, because version control history keeps it. Check: search committed configs and eval output files for key prefixes and key values, and confirm there are no matches. Also confirm that your installed version reads the variables you intend it to read.
Restrict permissions on the SQLite database
Set file ownership so only the service account that runs the promptfoo process can read and write the SQLite database file, along with any journal files created beside it during writes. Remove group and other read access. Apply the same restriction to the directory that holds these files so other local users cannot copy or replace them. Check: list the files as a different local user and confirm access is denied for the database and any journal files present. Repeat this check after each container rebuild, because volume ownership can reset.
Place the database on an encrypted volume
Move the database off the container's writable layer onto a dedicated volume. Enable encryption at rest through your storage platform or disk layer, and confirm the key management meets your policy. Map the volume in your Docker Compose file or Helm chart so the path the application writes to is backed by it. The self-hosting documentation states that SQLite cannot be shared across replicas, so run one database per instance. Check: write a test eval, restart the container, and confirm the record is still present. Also confirm the volume shows as encrypted in your platform.
Front the app with an owned boundary
Self-hosted promptfoo has no built-in authentication, SSO or RBAC, so the team places a reverse proxy, VPN or identity-aware gateway in front of it and owns that layer. Bind the application to an internal interface, limit ingress to named networks, and require authentication before any request reaches the web UI or API. Check: from a host outside the approved network, including a machine on the public internet, confirm the application does not respond. Then confirm an unauthenticated request to the UI and API is refused.
Set retention and redaction rules for outputs
Decide how long eval results, red team outputs and logs are kept, who can export them, and which fields are redacted before storage or sharing. Red team outputs often contain adversarial prompts and model responses that can include sensitive text, so treat them as sensitive data. Implement the rules with the tooling your installed version supports, such as a scheduled deletion job on the volume, and document what the job does. Check: seed a record with illustrative test data, including a fake key-like string and a sample personal identifier. Run the retention and redaction job and confirm the record is removed or masked in the database and in any export.
Restrict CI logs and artifacts
Configure the CI pipelines that run promptfoo so provider responses, request headers and environment values are not printed. Mask secret variables, limit verbose or debug output, and do not publish raw eval JSON as a public artifact. Limit who can read job logs and artifact storage to the team that owns the evals. Check: run a pipeline job against a test provider, open the log and the stored artifacts, and search them for the key value and for response text. Confirm there are no matches before the pipeline is trusted with production provider keys.
Pin the Node.js runtime and promptfoo version
Pin the promptfoo version in package manifests and CI, and pin the Node.js runtime in the base image and the CI runner. The npm release 0.124.1 requires Node.js 22.22.0 or newer, so the runtime pin must meet that minimum. Keep the lockfile and image digest with the hardening evidence so a rebuild reproduces the same versions. Check: rebuild the image from a clean checkout, print the promptfoo and Node.js versions from inside the container, and confirm they match the pinned values. Validate install and version commands against your installed version before adding them to runbooks.
Controls for provider keys, SQLite data and red team outputs
Five controls cover the data promptfoo handles on a self-hosted instance: provider keys, the SQLite file, the datasets feeding evals, red team results and network access. They share one principle: decide what must be stored, who can reach it and how long it stays. Each card names the decision to make and the check to run before go-live.
Scope provider keys per project
Give each promptfoo project its own provider key with only the permissions its evals need, so one leaked key exposes one project. Where the provider offers spending limits on keys, set them with the provider before the first run. Document a rotation procedure that names the owner, lists where the key lives in CI secrets and local environments, and states how the old key is revoked. Confirm with each provider which key scopes and limits it supports.
Protect the SQLite database file
Self-hosted promptfoo keeps results in a local SQLite database, so the file holds prompts, model outputs and eval history. Restrict file permissions to the service account that runs the app, and keep the file out of any path served by a web server or synced to shared drives. Backups should be encrypted at rest, stored with the same access controls as the live file and never left world-readable. Do not expose the file over a network share. Test backup and restore steps in your own environment before relying on them.
Minimize sensitive data in test suites
Eval datasets should use synthetic or masked records wherever the test can still measure the behavior. Real customer or personal data should not enter prompt files, assertions or sample outputs without documented approval. Write a short data handling note that lists the categories that must never be uploaded to a shared instance or sent to a provider, and review new datasets against it. Model responses can echo inputs, so masking inputs alone may not be enough. Check each provider's data retention terms for your account.
Treat red team outputs as sensitive
Attack probes and model responses can reveal weaknesses in your application and sometimes sensitive content. Limit who can view, export or copy these results, share summaries widely and send raw transcripts only to people who need them. Set a retention period for result files and schedule purges that also cover backups and CI artifacts. Red teaming documentation references the OWASP LLM Top 10, the NIST AI Risk Management Framework and the EU AI Act as frameworks that support systematic testing before deployment. Use them to decide which test evidence to keep, but they produce no compliance result on their own.
Limit network exposure of the server
The self-hosted app is an Express server with a web UI and API, and it has no built-in authentication, RBAC or SSO. Bind it to an internal interface only, and place it behind your team's own authentication layer, such as a reverse proxy with SSO. Restrict inbound routes with firewall or network policy and keep the instance off public ingress. Enable access logging at the proxy and review who reached the instance. Because the SQLite database cannot be shared across replicas, plan for a single instance and confirm how your deployment handles restarts.
How hardening changes by deployment model
On a single engineer workstation, promptfoo runs locally. Evals execute on your machine and talk directly with the model provider, so the operating system account and the local disk are the main boundary. Hardening focuses on where provider keys are held and what files accumulate during testing.
- Provider keys: store them in the OS keychain or a secrets manager CLI, and load them into the shell session only when a run needs them.
- Configuration files: keep literal keys out of promptfoo configurations and test suites so those files can be shared or committed safely.
- Local SQLite data: eval results, including prompts and model outputs, persist on disk. Restrict the data file to the owning user and rely on full-disk encryption.
- Red team outputs: write generated reports to a directory covered by the same disk controls, and remove them when the work is finished.
Confirm that the workstation's disk encryption and screen lock policies match the classification of the data you test, and agree with your security team where red team reports may be kept.
A single self-hosted node runs the Express server that serves the web UI and API, with data kept in a local SQLite database. The self-hosting documentation states that this setup is not recommended for production use, and the app has no built-in authentication, RBAC or SSO. Hardening therefore sits around the application: a proxy with an authentication layer, controlled volume permissions and tested backups.
- Volume permissions: run the container as a dedicated non-root user and limit write access on the data volume to that account.
- Authentication layer: place a reverse proxy or identity-aware gateway in front of the app, bind the service to an internal interface, and do not publish its port directly.
- TLS: terminate TLS at the proxy so prompts, outputs and any credentials are not sent in clear text.
- Database backups: copy the SQLite file while the app is idle or use the SQLite backup method, encrypt the copy, and test a restore on a separate host.
- Provider keys: inject them from your secret store at container start, and keep them out of the Compose file and image layers.
Confirm the current self-hosting guidance with the vendor before you rely on this setup for any shared or production use.
On Kubernetes with Helm, the cluster supplies several controls, but the SQLite constraint still defines the design. The self-hosting documentation states that the local database cannot be shared across replicas, so the deployment must stay at one replica. That limit moves the design toward recovery planning, since a single writer also means a single point of failure.
- Replica count: keep replicas at one and set the rollout strategy so that two pods never write to the same database at once.
- Secrets: source provider keys from the cluster secret store through your external secrets integration, and limit read access by namespace and service account.
- Persistent volume: place the data on storage that provides encryption at rest, and choose a reclaim policy that prevents accidental deletion of the data.
- Network policies: allow ingress only from the ingress controller or proxy, and allow egress only to the provider endpoints you have approved.
- Backups: back up the volume on a schedule outside the cluster, and test restores into a separate namespace.
Confirm the chart version and values with the vendor, and check the replica setting after every upgrade, since a changed default would break the single-writer assumption.
In air-gapped or restricted networks, the installation process becomes the first control. Dependencies arrive through an internal mirror, provider endpoints are reachable only through approved egress, and evidence stays on local storage. Some provider endpoints may be unreachable from these networks, so confirm which evaluation targets your tests can use before the design is fixed.
- Offline dependencies: mirror the npm packages and container images you need into an internal registry, and pin versions in the mirror.
- Approved egress: allow outbound traffic only to named provider endpoints through a logged proxy.
- Local evidence storage: keep eval results and red team reports on approved storage with access logs and a retention rule.
- Keys and secrets: use the internal secret store, and keep provider keys inside the enclosure.
- Enterprise On-Premise: promptfoo On-Premise runs Enterprise functionality on your own infrastructure with complete data isolation and a dedicated deployment engineer. Confirm the setup details with the vendor.
Confirm with your network team which egress paths are approved, and check the Node.js requirement for each release before it enters the mirror.
How to verify that promptfoo fails safely
This section explains how to prove that the hardening controls work, and how to record that proof for your own security review. Start with provider API keys, since they are the most sensitive secrets in the setup. Search the full repository history, not only the current tree, for key patterns from every provider you configure. Then check configuration files, eval exports, application logs, CI job logs and any screenshots shared in tickets or slide decks. Run an invalid key and a revoked key through a test eval, and confirm the error output reports the failure without echoing the secret. Use a secret scanning tool of your choice, and record its version and rule set so the result can be repeated.
Next, verify the storage layer. The self-hosted app keeps its data in a local SQLite file, so the first check is filesystem permissions. Sign in as a second local account on the host and confirm that it cannot read or copy the database file, its journal files or the directory that holds them. Then test the backup path. Restore an encrypted backup into a clean environment, rerun a known eval set and compare the results with the originals by content. Confirm that the decryption key is stored separately from the backup store, and that the people named in your runbook can perform a restore without the engineer who built the setup.
Then confirm the network boundary. From a host outside the approved network, attempt to reach the web UI and API on every address the deployment exposes, including any ingress or load balancer created through Docker Compose, Docker or Helm. The connection should fail at the network layer, and the attempt should appear in your firewall or ingress logs. Repeat the test from inside the approved network to show that the intended access path still works. For deletion, remove a red team output through the interface, note the time, and track when it disappears from each backup generation. Compare the observed timing with the retention schedule in your policy, and log any gap as a finding before go-live.
Test the redaction rule with a deliberately sensitive dataset made of synthetic records. Include examples of the identifiers that your regulators or internal policy care about, such as account numbers, patient identifiers or personal names, and label the file as illustrative test data. Run it through an eval and an export, then check that the blocked content is absent from stored results, the exported file, the logs and any generated report. Also test record formats you did not expect, because a pattern written for one format often misses another.
Finally, rehearse the failure modes and write down what happens. For a corrupted database, stop the app, restore from the last verified backup, confirm that the restored file opens, and check that no partial red team run is presented as complete. For a full volume, fill a test volume and observe whether the app stops writing and avoids leaving partial files behind. If a key leaks, time how long your team takes to rotate it and confirm that the old key is rejected. For two engineers writing to the same file, reproduce concurrent eval runs against one SQLite file, since the database cannot be shared across replicas. Record each result with the date, the promptfoo version, the configuration used and the person who ran the test.
What drives the effort of promptfoo hardening
This table lists the factors that change how much work promptfoo hardening takes, with the lever that keeps each one manageable. Read the first column as the question to answer in a scoping call, and the last column as the design choice to make before build starts. The main takeaway is that most effort comes from decisions made late, such as adding environments, copying real data into test suites, or leaving retention undefined.
| Driver | Why it changes effort | How to keep it down |
|---|---|---|
| Environments and deployment models | Each environment needs its own keys, storage, network path and backup job. Self-hosted Docker, Docker Compose and Kubernetes with Helm each bring different secret injection and persistence work. | Standardize on one deployment model and run one shared instance rather than per-team copies. Confirm which model the team already operates. |
| Provider accounts and API keys | Every model provider adds a key to issue, store, scope, rotate and revoke. Keys used in local runs, CI and the shared server multiply the surface. | Reuse the enterprise secret manager as the single source. Issue scoped keys per environment and record an owner for each. |
| CI/CD systems needing secrets | Each pipeline that runs evals or red team scans needs a secure way to receive keys, and each needs its own audit trail and failure handling. | Use one secret integration pattern across pipelines. Avoid storing keys in pipeline variables that are readable by every project. |
| Data sensitivity in test suites | If real personal or regulated data enters prompts, datasets or eval outputs, classification, access limits and handling rules apply to every run. | Use synthetic or de-identified datasets from the start. Keep regulated data out of test suites unless the data owner approves it in writing. |
| Size and retention of red team output | Stored results accumulate prompts, responses and findings. Volume and the period you keep them set how much purging, access review and evidence work follows. | Set a retention period before the first production run. Confirm with the data owner which fields are stored and for how long. |
| Existing network and identity controls | Reusing existing SSO, a reverse proxy, a secrets manager and a backup platform reduces custom build. Missing controls push that work into the project. | Inventory what already exists before design. Ask whether the self-hosted app can sit behind your existing proxy and identity layer, and confirm the answer in writing. |
| Depth of evidence for reviewers | Internal audit or external assessors may ask for key inventories, access records, backup restore tests and retention proof. Each request adds collection and documentation work. | Agree the evidence list in the written scope. DeployOpen prepares evidence for assessments but does not issue audit reports or certifications. |
What to monitor, back up, rotate and document after hardening
Hardening only holds if it is maintained, so the first routine is key hygiene. Review provider key usage on a cadence you set and write down, and rotate each provider key on that schedule as well as after any suspected exposure. Use the provider's usage dashboards to confirm that every key still maps to a known eval job, pipeline or team member, and retire keys tied to workloads that no longer exist. Keep the key inventory current, recording each key's provider, purpose, owner and next rotation date, but never the secret value itself. Before rotating, confirm how each provider handles overlapping keys, and update the secret store that CI pipelines read from first, since a job that caches the old key will fail on its next run.
The host running promptfoo needs monitoring for two exposures: access to the SQLite database file and access to the web UI. The self-hosted app has no built-in authentication, so anyone who can reach its port can read eval data. Log requests to the UI and API at the reverse proxy or network layer and alert on unexpected sources. On the host, audit read and write events on the database file and its directory, and alert when file permissions change or when a copy appears outside the service account. Confirm with your security operations team which of these signals your SIEM or log platform can ingest, who receives the alerts and how they are triaged. Treat unexplained access as a possible data exposure until the logs show otherwise.
Back up the SQLite database on the schedule your recovery needs require, and store the copies with the same access controls as the live file, because eval results can contain prompts, model outputs and red team findings. Restoring is the step most teams never test, so run a restore into an isolated host at a fixed interval you document. Confirm that the eval results open, that the web UI starts and that the configuration matches your baseline, then record the date, the backup used and the outcome as verification evidence. The database cannot be shared across replicas, so a restore recovers a single instance's data. Ask your platform team to confirm that the backup method produces a consistent copy of a database that may be in use during the backup window.
Red team outputs are the most sensitive data the instance holds, because they record the attack prompts and the model responses that exposed a weakness. Set a retention period for eval results and red team outputs, write it into the retention policy and purge on that schedule with a job you can show an auditor. Purge from the database and from every export, report and CI artifact, since deleting a row does not remove copies made elsewhere. Route data deletion requests through the same runbook so that any request touching personal data in prompts is traced to specific records. Confirm with your legal or privacy team whether different categories of eval data need different retention periods.
Re-run the verification checks after every promptfoo or Node.js upgrade. The npm release 0.124.1, published on 8 October 2026, requires Node.js 22.22.0 or newer, so a host on an older runtime will fail to install or run it. Upgrades can also move file paths, change defaults or relocate the database, which can quietly reopen a permission gap you closed earlier. Keep a written runbook covering key rotation, database restore, data deletion requests and incident response for leaked keys. Hand over an architecture diagram of data flows, a key inventory without secret values, the configuration baseline, verification evidence, the retention policy, and a named owner for each control.
Questions teams ask before hardening promptfoo
Can we harden promptfoo ourselves without a partner?
Yes, if your team already has the skills to run the host, manage secrets and control network access. The work is mostly ordinary platform engineering: placing the self-hosted app behind your existing identity and reverse proxy controls, moving provider API keys into your secrets manager, restricting file permissions and backups for the SQLite database, and setting egress rules so evaluations reach only the providers you approved. The harder parts are deciding where red team outputs may be stored and who may read them, and documenting those decisions so an auditor or security reviewer can follow them. If one of those skills is missing, the gap is usually the first thing to close, whether internally or with a partner.
What does promptfoo hardening cost, and what is separate?
The cost comes from four components. Internal staff time is usually the largest, covering engineers who build and review the controls and the security staff who approve them. A partner, if used, is paid under a fixed-scope written agreement, so the fee follows the agreed scope and deliverables rather than an hourly estimate. Infrastructure you already own, such as hosts, a secrets manager, logging and identity services, is a cost you carry but may not pay extra for. Model API spend is separate from hardening: it is what you pay providers when evaluations and red team probes call their models, and it rises with test volume. Ask any quote to list each component, which items are one-time and which recur, and what the scope excludes.
How long does promptfoo hardening take?
The duration depends on the number of environments, how many provider keys you hold, how sensitive the evaluation data is, and how much of the surrounding control set already exists. A team with a single environment, one key to rotate and existing secrets management will finish far sooner than one running separate development, staging and production instances with many keys and regulated data. Because these factors vary so much, do not rely on a generic figure. Ask for a written timeline that lists the environments, the key inventory, the data classification decisions and the approvals each phase depends on, and confirm which of those inputs you can supply on the first day.
Is the promptfoo Community edition enough for a secured deployment?
The Community edition is free and includes all LLM evaluation features, all model providers and integrations, and red teaming with 10k probes per month, with community support. It does not add team sharing, teams-based access control or SSO. If your organization needs a shared instance where users are separated by team and authenticate through your identity provider, you will need to add those controls yourself at the proxy or platform layer, or look at the Enterprise tier. promptfoo Enterprise adds those features, along with custom roles, continuous monitoring and priority support with SLAs, at custom pricing. Ask the vendor for a current written quote before deciding.
Can SQLite be used in production for self-hosted promptfoo?
The promptfoo self-hosting documentation says the bundled SQLite database is not recommended for production use. It stores data locally, cannot be shared across replicas, and the self-hosted app has no RBAC, horizontal scaling or built-in authentication or SSO. For a production path, first confirm with promptfoo which database and deployment options are supported for your version, then plan the migration and backup approach with your own platform team. Until that is settled, treat the SQLite file as sensitive data: restrict its permissions, encrypt its storage location, back it up under a documented procedure and keep it on a single instance. Verify each of these against the release you deploy.
What does our team need to provide before hardening starts?
Four inputs make the work predictable. First, infrastructure access: accounts or clusters where the self-hosted app runs, plus the network and DNS details needed for reverse proxy and egress rules. Second, a key inventory listing every provider API key, who owns it, where it is stored and which environments use it. Third, data classification: a statement of which evaluation inputs, outputs and red team results contain personal, confidential or regulated data, and how long they must be kept. Fourth, named owners for each control, such as identity, secrets, logging, backups and approvals, with the authority to make decisions. Missing any of these usually adds time because the work stops until someone answers.
Which compliance frameworks does promptfoo red teaming reference?
The promptfoo red teaming documentation references the OWASP LLM Top 10, the NIST AI Risk Management Framework and the EU AI Act as frameworks that support systematic testing before deployment. Those references help you map test coverage to the risks your reviewers ask about. They do not make the deployment compliant. DeployOpen prepares systems, controls and evidence for compliance assessments, but independent auditors and certification bodies issue any reports or certifications. Confirm with your assessor which framework applies to your system and which evidence they will accept.
How DeployOpen scopes, hardens and hands over promptfoo
DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside the client's own infrastructure. For promptfoo, that means working from the written scope to protect provider API keys, the SQLite data of a self-hosted instance and the eval and red team outputs it produces.
How we scope
- Inventory where promptfoo runs today: laptops, CI runners, or a self-hosted Docker, Compose or Helm deployment.
- Map every model provider key, its owner, and the environments and pipelines that read it.
- Agree in writing which red team outputs and eval datasets count as sensitive and where they may be stored.
How we build
- Move provider API keys into the client's secrets manager and inject them at runtime, not into config files.
- Place the SQLite database on persistent storage with restricted file permissions, backups and a documented restore path.
- Put the self-hosted app behind network controls and an authentication layer the client chooses, since promptfoo self-hosted has no built-in authentication or SSO.
How we hand over
- Deliver written configuration, key-handling and data-retention runbooks matched to the agreed scope.
- Document how red team results map to the frameworks the client selects, such as the OWASP LLM Top 10 or the NIST AI Risk Management Framework.
- Walk the client's team through verification steps so they can confirm the hardening themselves after handover.
How we support
- Ongoing support, upgrades and continuous testing are available as retainers, scoped separately from the initial work.
- Review hardening settings when promptfoo releases change Node.js requirements or the self-hosting setup.
- Agree the support boundary in writing, including what the client's own team keeps responsibility for.
