LLM Guard security risks after archival

Because the LLM Guard README states that its associated models on Hugging Face are no longer under active development or maintained, the model files your deployment loads are now a fixed risk surface. No upstream team will publish fixes, re-scan artifacts or announce a compromised revision, so the integrity checks must happen on your side. Start by writing down, for every scanner that downloads a model, the exact model identifier and the commit revision your deployment resolves to. A floating reference such as a branch name or an unpinned default can change without any notice, and an inventory built from identifiers alone will not show that change. Record the revisions in the same configuration repository that holds your LLM Guard version pins, so a reviewer can see both in one place. Once the revisions are recorded, download the files at those exact commits and compute a cryptographic checksum for each file, including the configuration, tokenizer and weight files. Store the hash manifest in a location your change control process protects, separate from the model files themselves, so that an attacker who can replace a model cannot also replace its record. Then mirror the files into an internal artifact store that your organization controls. Confirm with your own Hugging Face access method and internal artifact store whether revision pinning and offline loading work in your setup. Test this in a non-production environment, because some client libraries still attempt network calls at load time, and a mirror that appears to work in a quick test can fail when the network path changes. On each deployment, fetch the mirrored files, recompute the checksums and compare them with the stored manifest before the scanner loads anything. A mismatch should stop the rollout, not produce a warning in a log. Detecting silent changes at load time also means setting the offline mode so the process cannot pull a different revision from the public hub at startup, and logging the resolved revision and file hashes on every start. Treat any unexpected difference as an incident until you have explained it, and keep the failed manifest as evidence. Decide in advance what happens when a model cannot be fetched or verified. Options include keeping the last verified copy running, failing closed so the guarded endpoint refuses traffic, or routing requests to a documented fallback control. Each option has a different risk to the business, so the choice should be approved by the owner of the application, not left to the on-call engineer. Write the decision into the runbook with named roles, and rehearse it once so the team knows where the verified copies live and who can authorize a restore. Finally, plan for the models to outlive the scanner code. Keep the verified copies and the manifest under retention rules that match your audit requirements, and review them whenever you change the LLM Guard version or the Python runtime. The same manifests will help your team when it moves to a maintained guardrail, because they show exactly which model artifacts were in production and when each one was last verified.

The test plan should show that the controls around LLM Guard hold even when the scanners themselves miss something, because an archived project cannot be assumed to still catch what it was built to catch. Run four checks against the deployed configuration, not a developer workstation: a fail-closed check, a regression set of known attack samples, an access check on the API server, and an artifact hash check against the internal mirror. Record who ran each check, against which build and configuration, and when. Define the expected result for every check before the first run so that a result cannot be reinterpreted afterward.

Fail-closed check. Confirm that a scanner error or timeout produces the outcome your written policy requires, either blocking the request or flagging it for review. Stop the scanner process or the service it depends on in a test environment that mirrors production, or send an input that forces a timeout, then submit a request the policy would normally stop. Test both the input and output paths, because output scanners such as Sensitive and Malicious URLs can fail independently of the input scanners. A request that reaches the model or the user unscanned is a failed result. Read the timeout value from your own configuration before the test begins.

Regression set and access checks. Build a regression set of known prompt injection samples, secret-leak samples that use clearly fake key-shaped strings, and invisible-text samples that exercise the Prompt Injection, Secrets and Invisible Text scanners. Store the set in a repository your security team controls and version it, so the team can show what was tested. Run it against the deployed scanner settings and compare each outcome with the threshold in policy. Then call the API server without credentials and from an address outside the approved network ranges. Both calls should be rejected, and the rejections should appear in logs.

Artifact hash check and failure handling. After deployment, compare the hashes of the installed packages and the Hugging Face model files with the hashes recorded in the internal mirror manifest. Any mismatch is a failed result until it is explained. When a check fails, open a finding with the test input, the log output and the configuration version. The service owner, with security sign-off, decides whether to roll back to the last passing build. The security lead decides whether to tighten the scanner set, such as enabling additional scanners or lowering thresholds. Rerun the full set after any model, package, configuration or scanner change, and keep the inputs outside the LLM Guard deployment's own write path.

The interim period needs a monitoring set that shows whether the archived LLM Guard deployment still behaves as it did on the day it was accepted. Track scanner errors and latency for each input and output scanner the application calls, so a failing or slowing scanner is visible before users notice it. Track verdict rates per scanner as well. A sudden drop in flagged prompts from a scanner such as Prompt Injection or Invisible Text can mean the scanner has stopped working, and a sudden rise can mean the traffic has changed. Record model load failures at startup and after every restart, because the Hugging Face models will not receive fixes. Watch for unexpected egress from the scanner host too. A model or dependency that reaches an unknown endpoint is a security event, so it belongs in the incident queue alongside other security alerts.

Back up three things so the deployment can be rebuilt from a known state. First, the configuration: scanner selection, thresholds, the pinned LLM Guard version (the last PyPI release is 0.3.16, uploaded on 19 May 2025), the Python interpreter version, which must be 3.10 or newer and below 3.13, and the API server settings if the service runs in that mode. Second, a mirror of every model the enabled scanners load, stored in the client's own artifact store, so that a model changed or removed upstream cannot alter the running system without anyone noticing. Third, cryptographic hashes for the package files, model files and configuration bundle, recorded at acceptance and checked on every restore. Confirm with your engineers which models your chosen scanner set downloads at runtime, because that list depends on what you enabled.

Document the interim state in six artifacts that an assessor can read without a walkthrough. The system description states what the service does, which data it receives and where it runs. The inventory lists every host, container image, Python package, model and scanner in use, with versions. The risk acceptance records each known exposure from the archive, who accepted it, the compensating controls in place and the date of the next review. The test results show that the scanners were exercised against your own prompt and response samples after the most recent change. The change log records every configuration, version or model change with its approver. The replacement plan sets out the milestones toward a maintained guardrail, with dates the client sets and owns.

Assessors usually ask for the same evidence in a predictable order: who owns the system, what is in it, which risks were accepted and by whom, whether the controls operated over the period, and how the exposure will be removed. Have ready the monitoring exports for the period, the restore test records from the backup, the hash verification logs and the signed risk acceptance. DeployOpen prepares this evidence and the controls that generate it, and the support guide covers the routine checks that produce it day to day. The replacement schedule is set by the client, so the milestones should come from the client's governance and budget cycle. Independent auditors or certification bodies decide whether the evidence meets a given standard and issue any opinion or certificate. DeployOpen does not issue audit reports, attestations or certifications.