Which open-source model file scanner fits your intake pipeline
A team that pulls pre-trained models from public hubs into its own environment should scan each file before anything loads it. For multi-format CI gating, ModelScan is the best fit of the three. It is Apache-2.0 licensed, reads model files byte by byte for unsafe code signatures instead of loading them, and covers PyTorch, TensorFlow SavedModel, Keras H5 and V3, and classic formats saved with Pickle, Cloudpickle, Dill and Joblib. Its CLI returns nonzero exit codes, and it ranks findings as CRITICAL, HIGH, MEDIUM or LOW, so a pipeline can fail a build at a chosen threshold. The 0.8.8 release from February 18, 2026 supports Python 3.10 up to but not including 3.13, so confirm your runner's interpreter before you standardize on it.
Fickling and picklescan fill narrower roles. Fickling, from Trail of Bits, decompiles, statically analyzes and rewrites Python pickle files, and it symbolically executes them so analysis does not run the payload. Its enforcement feature hooks Python's pickle module and checks every load against an allowlist of ML library imports, raising UnsafeFileError when a check fails. That suits a deep review of a suspicious file or a load-time control inside a service. Check its repository for the current license before procurement. picklescan is an MIT-licensed lightweight check that reports dangerous globals, such as a global import of eval. It reads local files, directories, URLs and zip archives such as PyTorch models, and its --huggingface option scans a Hugging Face model by name, which makes it a quick first pass for a single pickle-based download.
None of these replaces the check Hugging Face runs on its own Hub. Hugging Face scans every pushed file with ClamAV and a pickle import scan that lists imports without executing them, and its Pickle Scanning documentation says the scan is not foolproof. The same is true of every scanner here, so treat a clean result as one control among several and never as proof of safety. Before you commit to a tool, run it against a known-bad sample in your own environment, confirm which formats your intake actually loads, and scan again before deployment, as the ModelScan README recommends alongside scanning before loading and after training. Ask each maintainer for its current release cadence and vulnerability disclosure process, since this comparison does not cover them.
Which scanner suits which intake scenario
This table maps common model intake scenarios to the scanner whose supplied capabilities fit them best. Read each row against your own pipeline, since the best fit depends on where the scan runs, who reads the output and which formats you accept. Treat the columns as a starting shortlist to test on your own model files, not as a ranking.
| Scenario | Best fit | Why |
|---|---|---|
| CI gating for a platform team | ModelScan | Its CLI returns nonzero exit codes and it has a Python API, so a pipeline can fail a build on a finding without custom parsing. |
| Incident triage of one suspicious file | Fickling | It decompiles and symbolically executes pickles without running them, which helps an analyst see what the file would do. |
| Research team pulling Hugging Face models by name | picklescan | Its --huggingface flag scans a model by name, which fits quick checks on downloads during experiments. |
| Security team needing analysis, not a yes or no | Fickling | It supports decompiling and rewriting pickles, so reviewers can inspect the opcodes and imports behind a verdict. |
| Mixed formats such as Keras, TensorFlow and PyTorch | ModelScan | It covers PyTorch Pickle, TensorFlow SavedModel, Keras H5 and V3, and several classic ML serialization formats. |
| Scanning PyTorch zip archives and NumPy files | picklescan | It loads zip archives such as PyTorch models and scans NumPy .npy files when numpy is installed. |
| Enforcing a strict import allowlist at load time | Fickling | It can hook Python's pickle module to check imports against an allowlist and raise UnsafeFileError on failure. Confirm the allowlist fits your libraries. |
| Enterprise-wide Hugging Face enforcement | Confirm with Protect AI | Protect AI describes its commercial Guardian product as covering Hugging Face enforcement. Verify scope and terms with the vendor before relying on it. |
How pickle and model files can carry code that runs on load
Pickle is Python's object serialization format, and it works by instructions that rebuild objects when the file is read. Those instructions can name any importable callable, so a model file can ask the loader to call a function such as eval or os.system during deserialization. The consequence is that loading a pickle-based model file is equivalent to running code from its author, before your application has inspected a single weight. PyTorch checkpoints, many scikit-learn and joblib artifacts, and other classic formats built on pickle inherit this property, which is why intake controls belong before the first load, not after.
The three open-source scanners in this comparison take different routes to the same question. picklescan is MIT licensed and checks the globals a pickle imports against a list of suspicious ones, without executing the file. Fickling, from Trail of Bits, symbolically executes the pickle so analysis does not run it, and it can also hook Python's pickle module to check each load against an allowlist of imports from ML libraries it considers safe. ModelScan reads model files byte by byte looking for unsafe code signatures instead of loading them. Ask each tool which of these three methods it uses, because a denylist, a symbolic analysis and a loader hook fail in different ways when an attacker uses an unlisted callable.
The distinction that matters most in practice is between scanning before load and enforcing an allowlist at load time. A scan is a gate in your intake pipeline: it reports findings and returns a nonzero exit code that can stop promotion. An allowlist enforced at load stops execution even when the scan was skipped, but it can also break legitimate models that import a library outside the list. The ModelScan README recommends scanning before loading pre-trained models, after training and before deployment, which covers both the model you download and the artifact your own pipeline produces. Confirm with your platform team whether a load-time control will run in every environment where models are deserialized, including notebooks and batch jobs.
Capability across ModelScan, Fickling and picklescan
This table compares the three open-source scanners on license, maintenance, formats, detection method, output, integration points and Python support. Where a cell says to confirm, the repository is the place to check before you add a scanner to an intake pipeline. The right choice depends on which model formats and loading paths your team actually uses.
| Capability | ModelScan | Fickling | picklescan |
|---|---|---|---|
| License | Apache-2.0, from Protect AI | Confirm the license in the Trail of Bits repository | MIT |
| Maintenance signal | Latest release 0.8.8, published February 18, 2026 | Trail of Bits project; confirm recent release activity in its repository | Confirm current release activity in its repository before adopting |
| Formats covered | PyTorch pickle, TensorFlow SavedModel, Keras H5 and V3, and Pickle, Cloudpickle, Dill and Joblib files | Python pickle files | Pickle files, PyTorch zip archives and NumPy .npy files when numpy is installed |
| Detection method | Reads file bytes for unsafe code signatures without loading the model | Symbolically executes pickles to analyze them without running them | Flags suspicious globals, such as an import of eval, and lists the dangerous ones found |
| Severity ranking | Ranks findings as CRITICAL, HIGH, MEDIUM or LOW | Confirm in the repository whether findings carry severity levels | Confirm in the repository whether findings carry severity levels |
| CLI and Python API | CLI with nonzero exit codes and a Python API for CI/CD pipelines | Usable as a library or command-line tool | Command-line scanner; confirm the Python API and exit code behavior |
| Hugging Face lookup | Confirm whether it scans Hub models by name | Confirm whether it scans Hub models by name | Scans a Hugging Face model by name using the --huggingface option |
| Load-time hook | Runs as a separate scan step before loading; confirm whether it offers any hook into loading | Hooks Python's pickle module to check each load against an import allowlist and raises UnsafeFileError on failure | Documented as a scanner; confirm whether a load-time hook exists |
| Python version notes | Version 0.8.8 supports Python 3.10 up to but not including 3.13 | Confirm supported Python versions in its repository | Confirm supported Python versions in its repository |
The three open-source scanners in brief
Pickle-based model files can run code the moment they load, so teams screen them before they reach a training job, notebook or serving stack. The three open-source options below take different approaches to that screen, and the Hugging Face Hub scan sits upstream of all of them as a platform-level check.
ModelScan
ModelScan is an Apache-2.0 project from Protect AI that reads model files byte by byte for unsafe code signatures instead of loading them, so embedded code never runs. It covers PyTorch Pickle, TensorFlow SavedModel, Keras H5 and V3, and classic formats saved with Pickle, Cloudpickle, Dill and Joblib. Its CLI returns nonzero exit codes, which suits a CI gate. Check that your Python version falls within the supported range for release 0.8.8, which was released on February 18, 2026. Use it as the intake gate before any pre-trained model is loaded.
Fickling
Fickling is a decompiler, static analyzer and bytecode rewriter for Python pickle files from Trail of Bits, usable as a library or a CLI. It symbolically executes pickles, so analysis does not run them. Its allowlist mode can hook Python's pickle module and raise UnsafeFileError when a load imports something outside the approved set. Choose it when reviewers need to see what a pickle actually does, not only a yes or no verdict. Confirm the license terms in the repository before adopting it in a commercial pipeline.
picklescan
picklescan is an MIT-licensed scanner that flags pickle files performing suspicious actions, such as a global import of eval, and lists the dangerous globals it finds. It accepts local files, directories, URLs and zip archives such as PyTorch models, and it can scan a Hugging Face model by name with a flag. Its README credits it as the inspiration for ModelScan. It works well as a lightweight pre-load check, but its detection depends on known dangerous globals, so pair it with an allowlist review for high-risk models.
Hugging Face Hub scanning
Hugging Face scans every file pushed to the Hub with ClamAV and a pickle import scan that lists imports without executing them. Hugging Face states that this scan is not foolproof, so it is a platform signal rather than a guarantee. It is not a self-hosted option you can configure or audit inside your own network. Treat it as context for what the public Hub already checks, and run your own scanner on every model you download, regardless of where it came from.
What Hugging Face's Hub scanning covers and what it does not
Platform teams own the path a model file takes from download to serving, so the first question is which controls already apply before a file reaches your environment. Hugging Face scans every file pushed to the Hub with ClamAV, which checks for known malware signatures, and runs a pickle import scan that lists the imports inside pickled files without executing them. That is useful context, but it is a control on Hugging Face's side of the transfer, not on yours. Files you pull into your own registry, object store or cluster are your responsibility from the moment they arrive.
- Scope of the Hub scan: It applies to files pushed to the Hub, so models mirrored, converted or copied from other sources are not covered by it.
- Import listing only: The pickle import scan reports what a file imports, which is a different question from whether a given import is dangerous in your context.
- Version drift: Hub-side checks can change over time, so record which scan results you relied on for each artifact you admit.
- Intake placement: A self-hosted scanner such as ModelScan can run as a CI step or admission gate in your own pipeline, independent of the Hub.
- Mirrors and caches: Confirm whether internal mirrors or proxies preserve, re-run or ignore upstream scan status.
Confirm with your team which artifacts enter through the Hub and which through other channels, then place a check at each entry point you operate.
Security engineers should read the Hub scan as a filter that reduces obvious risk, not as evidence that a file is safe. Hugging Face states that its scan is not foolproof, which means a file can pass the ClamAV check and the import listing and still contain unsafe behavior. The pickle import scan lists imports without executing them, so it depends on recognizing what a file imports, and a reviewer still has to decide whether those imports matter. Your residual risk therefore depends on what you do after the Hub check, including how you validate, isolate and load the artifact.
- Known signatures: ClamAV detects malware matching known signatures, so novel payloads may not be flagged.
- Static import view: The import list shows what a pickle references, not every path through which it could act once loaded.
- Independent check: A self-hosted scanner adds a second, separately configured judgment that you control and can log.
- Load isolation: Load untrusted model files in a sandboxed environment with no access to secrets or production networks.
- Evidence: Keep scan outputs, hashes and reviewer decisions so you can show what was checked for each model version.
Ask your vendor or internal review team to document which failure modes the Hub scan does not address and which of them your own intake check covers.
Researchers and evaluators who download models to test them need to know that a clean Hub result is a starting point, not a conclusion. Hugging Face's scan can tell you something about the file's contents at push time, but it does not tell you whether the model behaves as its card describes or whether its weights are trustworthy. If you plan to run a model on a workstation or inside a lab network, the safest habit is to scan the file locally before any load, then load it in a disposable environment that cannot reach your data or credentials.
- Local scan first: Run a file scanner on the downloaded artifact before importing it in any notebook or script.
- Disposable environments: Use throwaway containers or virtual machines for first loads of unfamiliar models.
- Format awareness: Pickle-based formats carry the most serialization risk, so treat them with more caution than formats that do not execute code on load.
- Provenance notes: Record the repository, revision and file hash you tested, since the Hub page can change.
- Shared caches: Do not let a shared model cache become a route that bypasses the checks your team applies.
Before you move a model from a research sandbox into a shared environment, confirm that it passed the intake check your platform team has defined.
How to choose and pilot a model file scanner
This sequence takes a team from an inventory of the model formats it loads to a scanner wired into CI with a documented block and review path. The output is a pilot record that shows which files were caught, which were missed, and what the team will enforce.
Inventory the model formats you load
The ML platform or security engineering lead lists every model file type your intake accepts, including PyTorch pickle archives, Keras H5 or V3 files, TensorFlow SavedModel, joblib or dill artifacts, and NumPy .npy files, along with the sources each one comes from, such as internal training jobs, Hugging Face, or vendors. Record which formats can be pickled, since those carry the code execution risk the scanners target. Done when the team has a written list of formats and sources that the scanner must cover.
Match each format to a candidate scanner
The security engineer compares the inventory against each candidate's documented coverage: ModelScan's README lists PyTorch, TensorFlow SavedModel, Keras H5 and V3, and classic formats saved with Pickle, Cloudpickle, Dill and Joblib, while picklescan focuses on pickle files, zip archives such as PyTorch models, and NumPy .npy files. Fickling is a decompiler and analyzer for pickle, so it suits deeper investigation of a flagged file. Done when each format on the list has a named scanner and any gaps are flagged for a second tool or manual review.
Confirm Python version support against your runtime
The platform engineer checks the Python version of the intake runner against each candidate's supported versions before installing anything. ModelScan 0.8.8 supports Python 3.10 up to but not including 3.13, per its GitHub releases page and PyPI, so a runner on another version needs a different plan or a version check against the current release. Done when the runner version is recorded next to the chosen scanner version in the pilot notes.
Run the scanner on approved and quarantined samples
The security engineer builds a sample set with models already known to be clean and a set of deliberately crafted malicious pickle files from an isolated test environment, then runs the candidate scanner over both sets. Record each detection, each miss, and any false positives on the clean set. Because ModelScan reads bytes without loading the model, the pilot can run on quarantined files without executing embedded code. Done when the team has a table of detections, misses and false positives per scanner.
Wire the CLI exit code into CI
The DevOps or platform engineer adds the scanner to the model intake pipeline so that a failed scan stops promotion of the artifact. ModelScan offers a CLI with nonzero exit codes and a Python API for CI/CD pipelines, which lets the pipeline treat a finding as a failed job. Run a known-bad sample through the pipeline to confirm it fails the build. Done when a deliberately flagged file blocks the job and the pipeline log shows the scanner output.
Set the severity threshold that blocks a load
The security lead and the ML platform owner agree which findings stop a load. ModelScan ranks findings as CRITICAL, HIGH, MEDIUM or LOW, so the team decides whether only CRITICAL blocks, or whether HIGH and above also block and MEDIUM goes to review. Write the threshold into the pipeline configuration and test it against the pilot samples. Done when the written threshold matches the configured behavior on every sample in the pilot set.
Document the review path for flagged files
The security lead writes the steps that follow a blocked or flagged file: who receives the alert, who can release a model after manual inspection, which tools are used for deeper analysis such as Fickling for pickle, and where the decision is recorded. Scanners are one control, and Hugging Face states that its own pickle import scan is not foolproof, so the review path must not assume a clean scan means a safe file. Done when the procedure is approved and linked from the intake runbook.
Questions buyers ask before adding a model scanner
What does it cost to run ModelScan, Fickling or picklescan?
The scanners are open source, so the license fee is zero, and the real cost is the engineering time to install them, wire them into intake, tune the findings and keep them current. ModelScan is Apache-2.0, picklescan is MIT, and Fickling is also open source under its published license, so none of them adds a license line item. Plan for recurring work: reviewing findings, handling false positives, updating the tool when new releases arrive, and maintaining the CI step. Ask any vendor or consultant for a quote built from a written scope that lists the formats, pipeline stages and review workflow you need, because those inputs drive the effort far more than the tool choice does.
How long does a pilot of a model scanner take?
No fixed duration is published for these tools, so the length of a pilot depends on your inputs. The main drivers are how many model sources you pull from, how many file formats appear in your intake, whether you already have a Python environment that the scanner can run in, and how many findings your team must triage before it trusts the output. A narrow pilot might scan one model source against one formats list in CI, while a broader one covers every pipeline that loads third-party weights. Agree on the exit criteria in writing before the pilot starts, including which findings block a model and who signs off on exceptions.
Can our own engineers run a model scanner without outside help?
Often yes, for a team with Python experience and a clear intake process. The CLI tools are straightforward to run locally or in a pipeline, and ModelScan offers a CLI with nonzero exit codes plus a Python API for CI/CD pipelines. The harder parts are policy, not installation: deciding which CRITICAL or HIGH findings stop a model, documenting exceptions, and deciding what happens when a scanner misses something. Hugging Face itself says its pickle import scan is not foolproof, so a single scanner should not be your only control. If your team lacks capacity for that ongoing review, a fixed-scope engagement can cover the setup and handover, and retainers can cover upgrades and continued support.
What does a model scanner need from our environment and team?
The requirements are modest but specific:
- A Python environment that matches the scanner's supported versions. ModelScan 0.8.8 supports Python 3.10 up to but not including 3.13, so check that range before you pin a version.
- CI access to add a scan step before any model is loaded, plus a place to store reports and exit codes.
- A named review owner, usually from the security or ML platform team, who triages findings and approves exceptions.
- Optional NumPy, if you want picklescan to scan .npy files, and access to Hugging Face if you use its scanning by model name.
Also confirm whether scans run on isolated runners, since you are handling untrusted files.
Do Guardian or other commercial scanners change the decision?
They can, depending on what you need beyond open-source scanning. Protect AI describes Guardian as its enterprise model scanning product with newer scanners, broader model support and enforcement for Hugging Face models. If you need vendor support, centralized enforcement across many teams, or coverage the open-source tools do not list, a commercial product may fit better. If your need is a scan step in CI with findings you can inspect and version, the open-source tools may be enough. Ask the vendor which formats, scanners and enforcement points are included, what support terms apply, and how findings are exported, and compare those answers against your own intake list rather than a general feature sheet.
Which model file formats should we confirm before relying on a scanner?
Start with the formats your intake actually receives, then check each against the tool's documented coverage. ModelScan lists PyTorch (Pickle), TensorFlow SavedModel and Protocol Buffer, Keras H5 and Keras V3, and classic formats saved with Pickle, Cloudpickle, Dill and Joblib. picklescan handles local files, directories, URLs and zip archives such as PyTorch models, and NumPy .npy files when numpy is installed. Fickling focuses on Python pickle files. For any format outside these lists, confirm with the maintainer or by testing a known-bad sample in your own environment. Do not assume a scanner covers a format because it reads a related one.
Does scanning a model file replace the need to review it?
No. A scanner reduces risk at intake, but it is one control among several. ModelScan reads files byte by byte for unsafe code signatures instead of loading them, which means embedded code does not run during the scan. That is a useful property, but signature-based detection can miss patterns it has not been built to recognize. Pair scanning with source checks on the model, a restricted load environment, and a rule that scanned models are only loaded after review. Hugging Face's documentation makes the same point about its own scanning, so treat any single result as evidence rather than proof.
What no scanner catches and how to keep the gate current
No file scanner is a guarantee. ModelScan, picklescan and Fickling look for known unsafe patterns, such as dangerous imports inside a pickle, so a payload that uses an unfamiliar technique or a format the tool parses poorly can pass. Hugging Face describes its own pickle import scan the same way: it lists imports without executing them, and its documentation says the scan is not foolproof. Treat a clean result as one signal in the intake record, not as approval to load. Pair the gate with provenance checks: record who published the model, the source repository, and the exact revision you reviewed. Pin that revision by commit hash in your loader configuration so an upstream update cannot bypass review, and rescan any new revision before it reaches a notebook or serving node.
Load-time controls cover what a scanner cannot. Where your framework supports it, prefer weight formats that do not execute code on load, and confirm with the framework's documentation which formats it reads safely. For pickle files that must still load, Fickling can hook Python's pickle module to check each load against an allowlist of imports from ML libraries, raising UnsafeFileError when a check fails. Run loads in an isolated environment with no access to credentials, production data or unrestricted outbound network paths, so a missed payload has little to reach. Agree the allowlist with your ML engineers, because a list that is too broad recreates the risk and one that is too narrow blocks legitimate models.
Keep scanner versions current on a schedule your intake team owns. For ModelScan, watch the ModelScan releases page for new versions and confirm your Python runtime falls within the supported range. Version 0.8.8, released February 18, 2026, supports Python 3.10 up to but not including 3.13, so a runtime upgrade can silently break the gate. Check each tool's repository for recent commits, open security issues and archive notices before you rely on it, and confirm the current release of Fickling and picklescan with their maintainers. Pin scanner versions in your CI image, record the version in every scan report, and rerun the scanner over your existing model inventory when a detection update ships. Other AI security tools may matter to your program, but they fall outside this comparison and need their own assessment.
How DeployOpen scopes model intake controls
DeployOpen plans, implements and documents model intake controls inside your own infrastructure, placing scanners such as ModelScan, Fickling or picklescan where they fit your pipeline. Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts.
How we scope
- Inventory which model formats and sources your teams load, including Hugging Face downloads and internal artifacts
- Agree in writing which scanners run, where they sit in intake, and what happens on a CRITICAL or HIGH finding
- Define who approves a quarantined model and how exceptions are recorded and reviewed
- Confirm Python version constraints and whether scanning runs in CI, on a build host or in an air-gapped network
How we build
- Integrate scanner CLIs or Python APIs into your CI/CD pipeline with nonzero exit codes that block promotion
- Configure model storage so flagged files are isolated before any framework loads them
- Test detection against known-malicious sample pickles and benign models to confirm the gate behaves as scoped
- Pin scanner versions and document how updates are reviewed before they reach production
How we hand over
- Deliver written intake procedures that state scan timing relative to download, training and deployment
- Provide runbooks for triaging findings, including who owns each decision and how evidence is retained
- Document the scanner inventory, licenses and maintenance status so your team can reassess choices over time
- Walk your engineers through the pipeline and record the handover in the agreed deliverables
How we support
- Offer ongoing support, upgrades and continuous testing as a retainer after handover
- Review scanner releases and maintenance changes, such as archived projects, and propose scoped updates
- Prepare intake controls and evidence for your compliance assessments, while auditors issue any reports or certifications
- Quote retainer scope from the written engagement, with no published price list
