What Article 5(1)(g) bans and why a biometric dataset label is not enough
Article 5(1)(g) bans biometric categorisation systems that sort individuals using biometric data to deduce or infer race, political opinions, trade union membership, religious or philosophical beliefs, sex life or sexual orientation. If your system produces any of those inferences, the ban applies to it, whatever the product is called and whatever its commercial purpose. The prohibition covers placing on the market, putting into service and use, so a provider that builds the system and a deployer that runs it on people are both within reach. It has applied since 2 February 2025, and breaches carry fines of up to EUR 35 million or 7% of total worldwide annual turnover for the preceding financial year, whichever is higher (Article 99(3)). The hard part is that the ban targets the inference, not the data. Two carve-outs leave room for legitimate work. The first covers labeling or filtering of lawfully acquired biometric datasets, such as sorting a licensed image collection by a tag. Engineers should not assume that a label on the dataset makes the downstream model safe, because the question is what the system deduces about a person once it runs. The second covers categorising biometric data in the area of law enforcement, which is a separate legal regime with its own conditions. Confirm with counsel which of these, if either, your use case relies on, and document that reasoning before any model reaches production. Plan for the line to blur. A demographic attribute classifier, a fairness audit that estimates group membership, or an accessibility feature may look similar to a prohibited system in code and output. Map each feature to the attribute it infers, record whether the output is used to sort or act on individuals, and check whether the same capability falls under the high-risk category for other sensitive or protected attributes in Annex III point 1(b), which carries transparency duties under Article 50(3). Where the answer is unclear, treat it as a scoping question for legal review, not an engineering choice.
Which systems, data and teams fall inside the ban
Article 5(1)(g) turns on a handful of scoping questions, and the answer to each one decides whether a system is prohibited, high-risk under Annex III point 1(b), or outside both. Work through these decisions before design starts, because the same model can land in different categories depending on what it outputs and who uses it.
Does the system categorize people
The ban applies to biometric categorisation systems that assign individuals to categories based on their biometric data. A system that only detects faces, counts people or verifies identity is a different category of tool. Check whether the output places a person into a group, score band or label derived from face, voice, gait or similar features. If it does, the prohibition analysis starts. If the output only describes the image, the ban is unlikely to be the controlling question.
Is a protected attribute being inferred
The prohibition covers categorisation that deduces or infers race, political opinions, trade union membership, religious or philosophical beliefs, sex life or sexual orientation. Map every label your system can emit against that list, including labels that look neutral, such as proxy categories. The test is what the output implies, not what the developer intended. Write down the inferred attribute for each output class.
Could a general-purpose model be repurposed
A model built for a neutral task, such as face embedding or attribute estimation, can be pointed at a sensitive inference by a downstream team. Who sets the use case determines who carries the obligation. Document the intended purpose, restrict the model's permitted outputs in contracts and configuration, and monitor for new label sets added after release. A model that is safe in one deployment can become prohibited in another.
Is a vendor API in the chain
If your product calls a third-party biometric or emotion API and then maps its output to a sensitive attribute, you still place the system into service or use it. Vendor documentation that says a feature is for demographic analytics does not settle the question for your use. Ask the vendor which outputs the API returns, whether those outputs can be used for sensitive inference, and what contractual limits apply. Keep the answers in your inventory.
Is the work law enforcement
The Act carves out categorisation of biometric data in the area of law enforcement from the ban, so the same technical output can fall on different sides of the line depending on the user. Confirm whether your customer or your own team is a law enforcement authority acting under its powers. Even where the carve-out applies, check other rules in the Act and national law before deployment, because the exclusion does not remove every obligation. Record the legal basis in writing.
Was the dataset lawfully acquired and only labeled
Labeling or filtering a lawfully acquired biometric dataset, such as a set of images, falls outside the ban. The exception covers the dataset work itself, not a system that then infers sensitive attributes from people in live use. Keep provenance records showing how the data was collected and what consent or legal basis applied. Separate dataset preparation from the production system so that the carve-out is not used to justify a deployment it does not cover.
How an assessor tests that a biometric categorisation system stays outside the ban
This table compares the questions an assessor asks about a biometric categorisation system, the approach used to test each one, the evidence to prepare, and the finding that most often causes trouble. Read it as a pre-assessment checklist: each row is a point where a system can drift from a permitted categorisation into the Article 5(1)(g) prohibition. Confirm the final scope and conclusions with your legal counsel and an independent assessor.
| Test question | Approach | Evidence requested | Common finding |
|---|---|---|---|
| Does the stated intended purpose avoid inferring protected attributes? | Read the documented purpose against the prohibited list in Article 5(1)(g), then check whether the marketing, product specification and configuration match that purpose. | Signed intended-purpose statement, product specification, configuration records and sales materials showing the same purpose. | Purpose statement describes a benign use (such as access control or age estimation), but configuration or a customer use case infers ethnicity or political views. |
| Does the output category list include any protected attribute? | Compare every output label, score and class against the prohibited attributes: race, political opinions, trade union membership, beliefs, sex life and sexual orientation. | Versioned output schema or label list, model cards showing each output head, and a change log for added categories. | A retrained model or a vendor update adds a label that maps to a protected attribute, and nobody reviews the new output list. |
| Do outputs actually infer protected attributes when tested? | Run the system on representative test data, look for outputs that correlate with protected attributes, and document proxy risks such as skin tone or name-based inference. | Test protocol, dataset descriptions, bias and correlation test results, and records of remediation or removal of affected outputs. | Testing covers only accuracy, not whether outputs separate people by a protected attribute, so proxy inference goes undetected. |
| Was the biometric dataset lawfully acquired, and was labeling limited to filtering or organizing it? | Trace dataset provenance and the labeling purpose. The carve-out covers labeling or filtering lawfully acquired biometric datasets, such as images, but not inferring protected attributes from individuals. | Data provenance records, lawful basis documentation, labeling instructions for annotators and a statement that labels were used only for filtering or organizing. | Annotation guidelines ask labelers to tag ethnicity or religion from faces, which falls outside the dataset carve-out. |
| Does the system operate in law enforcement, and does that change the analysis? | Determine whether the deployer is a law enforcement authority and whether the categorisation supports law enforcement. The carve-out for law enforcement categorisation is narrow and needs a case-by-case legal determination. | Written legal determination, deployer identity and mandate, use-case description and any national law relied on. | Organization assumes the law enforcement carve-out applies because the buyer is public, without a documented determination. |
| Where categorisation is permitted but sensitive, are high-risk duties and disclosure in place? | Classify the system under Annex III point 1(b) if it categorises by other sensitive or protected attributes, then check the Article 50(3) notice to exposed persons. | High-risk classification memo, Article 50(3) notice text and placement, deployer procedure for informing people, and records that notices were shown. | Deployer assumes the system is outside scope and never provides the notice to people exposed to the categorisation. |
How to show a biometric system is not prohibited, step by step
This sequence lets an organization document why a biometric categorisation feature falls outside the Article 5(1)(g) ban, which features fall under the Annex III point 1(b) high-risk category instead, and what evidence keeps that position current as the system changes.
Define the intended purpose and inference boundary
The system owner writes a short statement of what the biometric feature is for, which individuals it covers, and which inferences it must never make. The boundary should name the protected attributes listed in Article 5(1)(g): race, political opinions, trade union membership, religious or philosophical beliefs, sex life and sexual orientation. Legal or compliance review signs the statement. The step is complete when the boundary is approved, versioned and linked to the system record, so later testers know what the system must not infer.
Inventory every biometric categorisation feature
Engineering lists each component that takes biometric input (face, voice, gait, fingerprint or other traits) and assigns it a category, sorting out any output that assigns people to groups. Include models bought from vendors, open-source components and downstream features that reuse their outputs. The inventory records the input data, the output labels, the model version and the owner. It is complete when every feature that produces a grouping has a row and a named owner, and the list is reviewed against the system architecture.
Test outputs against protected attributes
Testers run the categorisation on representative labeled or synthetic test sets and check whether any output label, score or cluster corresponds to a protected attribute, either directly or through a proxy the model can learn. They record the test design, datasets, metrics and every finding, including findings that turn out to be false alarms. Where a model can be queried with arbitrary inputs, testers also attempt to elicit these inferences. The output is a dated test report that states which features passed, failed or need further analysis.
Classify each feature against the Act
Using the inventory and test report, the compliance lead assigns each feature to one of three results: prohibited under Article 5(1)(g) because it infers a protected attribute from biometric data, high-risk under Annex III point 1(b) because it categorizes people by other sensitive or protected attributes, or outside both categories. The reasoning for each classification is written down with reference to the Article text. Legal review confirms borderline cases. The step closes with a classification register that shows one decision per feature.
Record the dataset labeling or filtering purpose
Where the team labels or filters lawfully acquired biometric datasets, such as image collections, it documents that purpose separately from any deployed categorisation. The record explains the source of the data, the lawful basis for holding it, the labeling or filtering task, and confirmation that the output is not used to categorize individuals by protected attributes in production. This record supports the carve-out the Article provides for those activities. Privacy and legal teams approve it, and it is stored with the dataset documentation.
Add Article 50(3) notices where required
For any feature classified as high-risk under Annex III point 1(b), the deployer prepares the notice that informs exposed persons that the biometric categorisation system is operating, as Article 50(3) requires. The team confirms where the notice appears, how it is presented to people before or during exposure, and how the wording matches the actual function. The completed notice text, placement screenshots or design files, and a sign-off from the deployer form the evidence for this step.
Set a review trigger for model changes
The owner defines the events that reopen the classification: a new model version, new training data, a new output label, a new deployment context or a change to the intended purpose. Each trigger links back to the inventory and the test process, so a change cannot ship without a repeat of steps two to four. The review record lists the trigger, the date of the review and the outcome. The step is complete when the change management process includes these triggers and the most recent review is signed off.
Where the dataset and law enforcement carve-outs stop applying
The two carve-outs in Article 5(1)(g) are narrower than they first appear. Labeling or filtering a lawfully acquired biometric dataset, such as a set of images, falls outside the ban. That exclusion covers the dataset work itself, not what happens next. If the system that labels or filters the data then infers race, political opinions, union membership, religious or philosophical beliefs, sex life or sexual orientation from the individuals it processes, the carve-out no longer protects it, and the ban applies to that inference. Engineering teams should record which step produces the protected-attribute output, because the boundary falls at that step, not at the dataset.
Categorisation in the area of law enforcement is treated as a separate question and should not be assumed to fall under either side of the line. Organizations that sell, buy or run biometric categorisation for police, border or other public authorities should obtain a written legal review of the specific use case, the data sources, the legal basis and any national rules that apply on top of the AI Act. Ask counsel to confirm whether the use is prohibited, high-risk under Annex III point 1(b), or outside both categories, and to document the reasoning. Do not treat the carve-out as a general exemption for public-sector use.
The ban reaches placing on the market, putting into service and use. A vendor that builds or supplies a system capable of inferring protected attributes can be caught when it places that system on the market, and a deployer can be caught when it puts the system into service or uses it. Both parties should therefore check the system's documented intended purpose, the outputs it can produce and the contractual terms for configuration. Where an edge case remains open, such as a model that infers an attribute only as a side effect, confirm the position with qualified EU counsel before launch and keep the written opinion with the technical file. The Commission's AI Act overview is a useful starting point, but it does not settle individual cases.
Evidence that shows a categorisation feature is outside Article 5(1)(g)
A feature that categorizes people by biometric data is only outside the Article 5(1)(g) ban if its purpose, data, outputs and testing all point the same way. The six artifacts below form one file that an assessor or market surveillance authority can read end to end. Each one needs an owner and a refresh trigger, or the file stops describing the system that is actually running.
Intended purpose and label register
This document states what the system is for and lists every output label it can produce, including any label that could be read as race, political opinion, union membership, beliefs, sex life or sexual orientation. Write the register from the product's actual output schema, not from marketing copy. Each label should carry a stated purpose and a note on whether it is permitted for the use case. Review it whenever a new label, model head or customer use case is added, and record the review date and approver.
Dataset provenance and labeling records
The ban does not reach labeling or filtering of lawfully acquired biometric datasets, such as images, so the file must show how each dataset was obtained, under what legal basis and for what labeling purpose. Keep source, licence, collection date, annotation guidelines and annotator instructions with each dataset version. Check that no annotation instruction asks labelers to assign sensitive attributes to the people shown. Refresh the records whenever a dataset is added, merged or relabeled.
Test reports on sensitive inference
Test reports show, with recorded inputs and outputs, that the system does not deduce the listed attributes from biometric data. Include test sets built to probe these attributes, pass and fail criteria set before the run, and results for every model version in production. Keep the raw logs so a reviewer can rerun a sample. Repeat the suite after each retraining, threshold change or change of vendor model, since a new model can reintroduce an inference the earlier version did not make.
Signed legal classification memo
A short memo, signed by the accountable owner and reviewed by counsel, records why the feature falls outside Article 5(1)(g) and whether it is instead high-risk under Annex III point 1(b). It should name the intended purpose, the data used, the outputs considered and the conclusion, with the facts the conclusion depends on. Set a trigger for re-review: any new attribute, new use case, new jurisdiction or a material change in test results. The memo does not replace legal advice, so confirm its scope with qualified counsel.
Article 50(3) disclosure text
Where a categorisation system falls under the high-risk category, deployers must inform the people exposed to it. Keep the exact disclosure wording, the places where it appears (signage, interface notices, contract terms) and the logs showing it was shown. Confirm with counsel whether a disclosure is needed for each deployment context, and version the text alongside the product so the wording a person saw can be produced later.
Change log linking updates to classification
The change log ties each model update, data change and configuration change to a decision on whether the classification still holds. Each entry should name the change, the test run that covered it, the person who approved it and whether the legal memo was re-checked. Keep it in the same version control system as the code, so the history can be audited without relying on memory. Review the log at each release and before any customer rollout.
What meeting Article 5(1)(g) costs in money and effort
This table compares the cost components of showing that a system avoids Article 5(1)(g) categorisation by sensitive attributes, who is paid for each, and what moves it up or down. No fee or duration figures were supplied for this topic, so treat every cost as something to scope and quote in writing, then use the drivers column to challenge those quotes.
| Cost component | One-time or recurring | What drives it | How to keep it down |
|---|---|---|---|
| Output label inventory and intended purpose work | One-time, then updated when labels change | Number of categorisation outputs the system can produce and how many of them map to protected attributes | Remove unused output labels early and document the intended purpose in one place before engineering starts |
| Engineering time for attribute testing | Recurring with each release | Number of attribute families tested, size of the evaluation set and how many model versions are in service | Run attribute-proxy tests in the existing CI pipeline using self-hosted open-source test tooling where it fits your stack, and confirm each tool's capabilities first |
| Legal review | One-time per use case, plus review when the purpose changes | Whether the system sits inside the prohibition, the dataset carve-out or the law enforcement carve-out, and how many jurisdictions apply | Ask counsel for a short written classification memo covering one use case per document, so re-reviews stay narrow |
| Dataset provenance records | One-time setup, recurring for new data sources | Number of data sources, whether data was lawfully acquired and whether labelling or filtering touches biometric data | Record source, lawful basis and labelling steps at ingestion so the evidence is built as data arrives, not reconstructed later |
| Re-testing on model change | Recurring | Frequency of retraining, vendor model updates and changes to inputs or prompts | Automate regression runs and limit the trigger to changes that affect outputs or the input population |
| External testing or review | Recurring, usually per engagement | Scope of the system under test, whether a fixed-scope test is needed and how often it is repeated | Agree a written scope with defined test cases and deliverables before work starts, and reuse prior test cases in later engagements |
| Independent assessor time where a conformity route applies | Recurring per assessment cycle | Whether the system is high-risk under Annex III point 1(b) and which conformity route applies, since most Annex III systems use internal control and some biometric systems involve a notified body | Confirm the route early with the assessor, then prepare evidence to their written request list so their hours go to testing rather than gathering |
Who owns the ban, who prepares the evidence and who tests it
Your organization owns the decision. The provider or deployer must determine whether a biometric categorization system infers race, political opinions, trade union membership, religious or philosophical beliefs, sex life or sexual orientation, and if it does, whether the dataset labeling or law enforcement carve-out applies. The prohibition is set out in Article 5 of the AI Act, and the position you take under it belongs to named people inside the organization, usually legal counsel working with the product owner and the data protection officer.
- Classification record: A written determination for each biometric categorization feature that states which attributes the system outputs and how it derives them.
- Carve-out analysis: Documented grounds for any claim that the system only labels or filters lawfully acquired biometric datasets, or that it operates in law enforcement, including the limits you rely on.
- Fallback high-risk path: Where the system categorizes people by other sensitive or protected attributes, a plan to meet the Annex III point 1(b) high-risk duties and the Article 50(3) duty to inform exposed persons.
- Accountability: Named owners, a review trigger whenever the model, data or intended purpose changes, and board-level awareness of the fine ceiling in Article 99(3).
Before any partner starts work, confirm in writing who signs the classification decision and who can stop a release.
A partner prepares the technical ground that lets your team and an assessor check the position. The work is engineering and documentation, scoped in writing before it starts. The partner does not decide whether your system is lawful and does not sign any statement about it, so its deliverables should be described as inputs to your decision rather than as proof of compliance in themselves.
- Test harnesses: Automated tests that send labeled and synthetic inputs through the categorization pipeline and record whether any output attempts to infer an excluded attribute. Illustratively, a harness might flag any output field that carries a protected attribute name or a proxy value.
- Self-hosted tracing: Observability inside your own environment that records inputs, outputs, timing and metadata for each step, for example with Langfuse, so that each categorization decision can be reconstructed later.
- Adversarial testing: Probing of any language model component with garak to check for leakage or jailbreak behavior that could reveal sensitive inferences, with its JSONL report and hit log retained as evidence.
- Evidence packs: Indexed records of test runs, configuration baselines, dataset provenance and access logs, organized by the classification decision each one supports. Confirm with the vendor whether CISO Assistant can track these assessments for your framework set.
- Remediation support: Engineering changes that remove or disable sensitive-attribute outputs, followed by retesting that shows the change worked.
Ask for a deliverables list, the tests in scope and who keeps the evidence after handover.
Testing and reporting belong to independent parties. For most Annex III systems the provider follows internal control under Annex VI, but notified bodies are involved for some biometric systems under Article 43, so confirm which route applies to your system before planning. National market surveillance authorities enforce the Act for AI systems, and the AI Office enforces the general-purpose AI model rules. Neither a partner nor your own team can issue the report or certificate that these bodies issue.
- Notified body route: Where Article 43 requires a notified body for a biometric system, that body assesses the system and issues any certificate. Ask which documents and tests it will sample.
- Market surveillance: National authorities can request documentation, examine systems and act on complaints. Evidence should be retrievable quickly, in the form the authority would expect to see it.
- Management system certification: An ISO/IEC 42001 certification body, if you pursue one, certifies a management system. Treat that result as separate from your AI Act position.
- Independence limits: The firm that helped classify or build the system should not also be the party that assesses it. Check for conflicts before engaging any assessor.
Before the engagement, confirm who issues the output, what they sample and what your team must have ready.
Where organizations fall short on this prohibition
Most organizations that get this prohibition wrong do not intend to categorize people by sensitive attributes. They miss it because the dataset is lawful, the model came from a vendor, or the system was reclassified in their head but not on paper. The cards below cover the gaps assessors and market surveillance authorities tend to probe first, each with a closing action.
Assuming dataset labeling is exempt
The Article 5(1)(g) carve-out covers labeling or filtering lawfully acquired biometric datasets, such as images. It does not cover a system that assigns those labels to people in production. Teams often treat a labeling pipeline and a live categorisation service as one thing. Check whether any output from the system is applied to an individual, and document the boundary between dataset preparation and deployment.
No test of what outputs infer
A categorisation model may be built for a harmless purpose, such as demographic analytics or audience sorting, yet still predict a protected attribute from face, voice or gait features. Accuracy testing alone does not show what a model infers. Run inference tests on labelled samples that include protected attributes, record which attributes the outputs correlate with, and keep the results with the system's technical file.
Vendor models repurposed without review
Procurement often buys a biometric analytics product for one use, then a different team points it at a new population or adds a new output field. The vendor's documentation may describe only the original purpose. Before any new use, ask the vendor whether the model was designed or tested to infer sensitive attributes, and review the contract for permitted uses. Treat a changed use as a new deployment decision.
No record of intended purpose
Without a written intended purpose, nobody can show that a categorisation system avoids the prohibited categories. The record should name the attributes the system categorises by, the population it covers, and the decisions its outputs feed. It should also list the outputs the team has explicitly excluded. Keep this record under version control and link it to each release.
Missing Article 50(3) notices
Biometric categorisation that uses sensitive or protected attributes, but falls outside the ban, is high-risk under Annex III point 1(b). Deployers must inform the people exposed to it under Article 50(3). Many teams classify the system correctly as high-risk yet never build the notice into the product or the site where it operates. Confirm who gives the notice, when, and how it is recorded.
No reclassification after model updates
A system that was non-prohibited at launch can drift into the prohibition after retraining, a new feature, or a change in the data it receives. Teams rarely repeat the classification review after a model update. Add a checkpoint to the change process that asks whether any new output could infer a sensitive attribute, and record the outcome before the update goes live.
Questions buyers ask about Article 5(1)(g)
Does the ban cover a labeling tool that tags biometric datasets?
No, not when the tool only labels or filters a lawfully acquired biometric dataset, such as a set of images. Article 5(1)(g) targets systems that categorize individuals from their biometric data to deduce or infer race, political opinions, trade union membership, religious or philosophical beliefs, sex life or sexual orientation. The line falls at the purpose and the output. A tool that sorts a dataset for annotation or quality control sits outside the ban, but a tool that places live people into sensitive groups does not. The same ban also excludes categorization in the area of law enforcement. Document the dataset's lawful basis, the labeling purpose and the fact that no individual is categorized by a sensitive attribute, then have counsel confirm the boundary for your specific use.
What is the maximum fine for breaching this prohibition?
Breaching the prohibition can cost up to EUR 35 million or 7% of total worldwide annual turnover for the preceding financial year, whichever is higher, under Article 99(3). For an undertaking, the percentage is usually the larger figure for any group with substantial revenue, so turnover drives the exposure more than headcount or system count. Fines are a ceiling, and the actual amount depends on the nature, gravity and duration of the breach and on what the organization did to mitigate it. The practical cost of a finding also includes a suspended or withdrawn system, remediation work, and the reputational cost of a regulator's public enforcement. Ask counsel to model exposure against your group turnover, not only the headline figure.
Does using a vendor's model shift the prohibition onto the vendor?
No. The prohibition covers placing on the market, putting into service and use, so it binds both providers and deployers. A vendor that builds a categorization system may be a provider, while your organization becomes a deployer the moment it puts that system to work on people in the EU. Contract terms can allocate responsibilities, but they cannot remove a prohibition from either party. Before you deploy any third-party biometric or face-analysis feature, confirm in writing what the vendor's system outputs, whether it can infer any of the listed attributes (including as a side effect of other outputs), and how the vendor restricts configuration. Keep the vendor's documentation and your own use-case review in the same file.
How long does a classification review of a biometric feature take?
The duration depends on how many systems you have, how complex their outputs are and how many teams must sign off, and no supplied figure gives a standard timeline. The main drivers are the inventory (whether you know every system that processes biometric data), the number of outputs each system produces, and whether the review must cover datasets, training pipelines and downstream uses. A narrow review of one system with clear outputs is far shorter than a review of a platform that exposes several classifiers to multiple business units. Ask any provider for a written schedule that lists the inputs they need from you, the deliverables and the decision points, and agree that schedule before work starts.
What does the work need from our internal staff?
The work needs decisions and access from your staff, which an outside partner cannot make for you. Product owners must describe each intended use and every output the system produces. Engineers must provide model cards, data sources, labeling procedures and logs of how outputs reach users. Legal or privacy staff must confirm the lawful basis for any dataset, and a named owner must accept the classification decision and the residual risk. Where a system could fall into the high-risk category under Annex III point 1(b), someone must also plan the deployer duty to inform exposed persons under Article 50(3). Assign these roles before the review starts, because most delays come from missing answers rather than from technical testing.
Can an outside partner issue a compliance statement for us?
An implementation partner can prepare systems, documentation and evidence, but it cannot issue a compliance statement, attestation or certificate. That role belongs to the organization, which owns its controls, and to independent assessors or notified bodies where the AI Act requires them. DeployOpen plans, implements, documents and tests self-hosted open-source platforms in your own infrastructure, and it can help you gather the evidence that a reviewer will inspect, such as inventories, logs and test records. Use the outside party's written scope to define what it delivers and what remains your responsibility. Any statement that a system meets Article 5 should come from your accountable executive, supported by counsel and, where relevant, an independent assessor.
How self-hosted tools produce evidence for this prohibition
No self-hosted tool decides whether a biometric system falls under Article 5(1)(g). That judgment belongs to the organization, its legal team and, where a regulator asks, the market surveillance authority. What the tools can do is produce records that show how a system behaves and what the organization checked. The evidence is only as useful as the scope it covers, so confirm in your own environment that each tool can see the biometric pipeline, including any categorisation step that runs on images, voice or other biometric inputs.
garak probes language models for weaknesses such as data leakage, toxicity and prompt injection. It writes a JSONL report of every attempt and evaluation, plus a hit log of attempts that found a vulnerability. For this prohibition, that matters where a language model sits inside a pipeline that interprets biometric-derived outputs or where prompts could push the system to infer sensitive attributes. Treat garak as a probe of model behavior, not as a test that a system does or does not categorise people by protected traits. Confirm with your security team which probes and prompts reflect your actual use case, and keep the JSONL files with the test plan that produced them.
Langfuse records traces of each request, including LLM calls, retrieval steps, tool executions and custom logic, with timing, inputs, outputs, token usage and metadata. A trace can show that a categorisation step was invoked, what attributes were returned and which tool or model produced them. Set retention and access controls on traces that contain biometric-derived data, because the traces themselves may be personal data. CISO Assistant (AGPL-3.0-only) can record assessments, link them to controls and track their status, so the legal analysis, the scoping decision and the review dates sit next to the technical evidence. Confirm that the framework content you load covers the obligations you need to track, and that a qualified reviewer signs off each conclusion.
Pre-launch checks before a biometric feature ships
This sequence confirms that a biometric feature falls outside the Article 5(1)(g) ban before release, that it does not infer sensitive attributes in practice, and that the decision to ship is recorded with named owners. Each check ends with a document the organization can show an assessor or market surveillance authority.
Confirm the intended purpose in writing
The product owner and legal or compliance lead write down what the feature categorizes, for which users, and why. The statement must make clear that the system does not deduce or infer race, political opinions, trade union membership, religious or philosophical beliefs, sex life or sexual orientation. Output: a signed purpose statement that names every categorization label the system can emit. If any label touches a protected attribute, the team records the case for a different legal route before moving on.
Run attribute inference tests on a held-out set
The evaluation team runs the categorizer on a held-out set of biometric samples that were not used in training, and checks whether outputs correlate with any sensitive attribute, including attributes inferred through proxies such as age or appearance clusters. Testers document the test population, the metrics used and any subgroup where outputs diverge. Output: a test report with pass or fail results per attribute, and a rule that a failed attribute blocks release until the feature is changed or removed.
Confirm dataset provenance and labeling scope
The data team records where every biometric dataset came from, under what lawful basis it was collected, and whether any labels for sensitive attributes were created or stored. The Article 5(1)(g) ban does not cover labeling or filtering of lawfully acquired biometric datasets, so the team must show that sensitive labels are not used to power a categorization function. Output: a provenance register and a data flow record showing which fields feed the model and which are excluded.
Confirm disclosure and high-risk classification
Legal and product teams decide whether the feature falls under the high-risk category for other biometric categorization by sensitive or protected attributes under Annex III point 1(b), or whether it is excluded. Where the system is used, they confirm the notice that deployers must give exposed persons under Article 50(3) and check the wording with counsel. Output: a classification memo and the approved notice text, with the place where users will see it.
Record sign-off from accountable owners
The accountable executive, the security lead and the compliance lead review the purpose statement, test report, provenance register and notice, then sign a release record that states the decision and any conditions. The record names each approver and the version of the model it covers. Output: a dated release record stored in the governance system, which an independent auditor or regulator can request. Your organization keeps this decision, and an implementation partner can prepare the packet but does not approve it.
Schedule re-testing on every model change
The team sets a trigger for repeating the inference tests whenever the model, training data, label set, or deployment context changes, and records who owns that trigger. Re-testing uses the same held-out approach so results can be compared across versions. Output: a change log linking each model version to a test report and an updated sign-off. Ask your vendor and implementation partner to confirm how their tooling records these version links before you rely on them.
How DeployOpen works on this
DeployOpen treats Article 5(1)(g) as a scoping question first: which biometric categorisation functions exist, whom they touch, and whether any carve-out is claimed. Its work is limited to self-hosted open-source platforms, security testing and compliance readiness inside the client's own infrastructure, and the organization keeps ownership of its legal position and controls.
How we scope
- Inventory every system that derives sensitive attributes from face, voice or other biometric data.
- Record which carve-out, if any, the organization relies on, with the legal basis confirmed by its own counsel.
- Agree the scope, deliverables and timeline in writing before any work starts, as fixed-scope engagements.
How we build
- Configure self-hosted platforms so that sensitive-attribute outputs are blocked or never generated in production.
- Log model inputs, outputs and access decisions in the client's own environment for later review.
- Run security tests against the categorisation pipeline to check that prohibited outputs cannot be reached.
How we hand over
- Document system configurations, data flows, access controls and test results for the client's compliance file.
- Provide a written record of which functions were tested, what was found and how each finding was resolved.
- Hand over runbooks so the client's own staff can operate and monitor the controls after the engagement ends.
How we support
- Offer ongoing support, upgrades and continuous testing as a retainer when the client wants continued coverage.
- Retest affected components after any model, platform or configuration change that touches categorisation.
- Work with the client's independent assessor on requests for evidence, without issuing any audit report or certificate ourselves.

