MEASURE 3.3 End user problem reporting and appeals

MEASURE 3.3 end user problem reporting and appeals: what it asks for

If a customer, regulator or internal audit asks how people affected by your AI system can report problems and appeal its decisions, MEASURE 3.3 is the subcategory they have in mind. It asks that feedback processes exist for end users and impacted communities to report problems and to appeal system outcomes, and that those processes feed into the metrics you use to evaluate the AI system. In plain terms, you need a working channel for complaints and challenges, and you need to show that what arrives through that channel changes how the system is measured. MEASURE 3.3 sits in MEASURE 3, which covers tracking AI risks after deployment. It is part of the voluntary NIST AI RMF, which has no certification, so no one issues a badge for meeting it. An assessor, customer or regulator will test it by asking for the process, the records and the evidence that the process was used.

The risk this addresses is simple to describe and hard to manage. A deployed model can make errors that only the people it affects will notice, such as a wrongly flagged transaction, an unfair screening outcome or a generated answer that misleads. Feedback about those errors usually arrives in scattered places: support tickets, emails to a sales contact, social posts, notes from a branch or clinic. Appeals often have no named owner, so a decision may be reversed informally or not at all. Nobody measures whether the process works, so the organization cannot say how long a report takes to reach a reviewer, how often appeals succeed, or whether the same problem keeps coming back. Confirm early who owns each channel, where reports are logged, and which team reviews appeals, because the rest of the evidence depends on those answers.

Before you buy tooling or hire outside help, write down the decisions this subcategory forces. Decide which user groups and affected communities can report problems, which outcomes can be appealed, who reviews each appeal and what authority that reviewer has to change a result. Decide which measures will be reviewed in the AI system's evaluation, such as the rate and type of reports, appeal outcomes and participation in feedback activities. The NIST AI RMF Playbook lists suggested actions under MEASURE 3.3, and the NIST AI 100-1 framework document sets out the outcome itself, so use those as a checklist of questions rather than as a script. An implementation partner can help set up intake, ticket routing and metric reporting, but the organization must own the policy, the reviewers and the decisions made on appeal.

Which AI systems, users and third parties fall under MEASURE 3.3

MEASURE 3.3 only works where the feedback channel reaches the people a system affects. The scoping choices below decide which systems need a reporting and appeal path, which outcomes must be contestable, and who owns each step when a vendor hosts the model.

Which deployed systems have affected people

Start with the deployed inventory, not the development backlog. A system used only by internal analysts with no effect on customers, applicants or staff may need little more than an internal issue queue. A system that scores, recommends or filters outcomes for external people needs a documented route for those people to report errors. Check each system against its actual deployment context, since the same model can move between low and high impact uses. Record the decision so an assessor can see why a system was excluded.

Which outcomes can be appealed

Appeal scope depends on whether a system output drives a decision with consequences for a person, such as a denial, a flag, a ranking or a withheld service. Outputs that only inform a human reviewer still need a way to report errors, but the appeal path may run through that reviewer. List each decision type, the person affected and the human who can overturn it. Appeals that have no owner or no recorded outcome cannot produce the categorized appeal data MEASURE 3.3 asks organizations to analyze.

Which data subjects are in scope

Identify every group whose data is processed or whose outcomes the system changes, including people who never interact with the interface. Impacted communities can be wider than users, for example a neighborhood affected by a scoring tool or patients affected by triage rules. Map each group to a reporting channel it can actually reach, with language, accessibility and contact needs considered. Feedback from groups left out of this map will not show up in the metrics MEASURE 3.3 expects you to build.

Which feedback channels exist today

Check whether existing ticketing, complaint or service desk tools already receive user reports about the AI system. Reuse lowers effort, but only if AI-related tickets can be tagged, routed to the model owner and counted separately from other issues. Confirm whether the tool can record category, outcome and appeal status fields. Without those fields, the required analysis of appeal rates and results has to be rebuilt by hand each review cycle.

How vendor-hosted models shift responsibility

When a third party hosts the model, the vendor may collect telemetry and provide a support queue, but the deploying organization still owns the decision and the affected person's complaint. Write down which party receives each report, which party investigates model behavior and which party answers the affected person. Contract terms should cover access to logs, timelines for vendor responses and notice of model changes. If those terms are missing, the deployer cannot show the feedback loop MEASURE 3.3 describes.

Whether self-hosted traces hold the evidence

For self-hosted models, the deploying organization controls the request and response data needed to investigate a reported problem. Decide which traces, inputs, outputs and model versions must be retained to reproduce a contested outcome, and how long they stay accessible for appeals. Privacy obligations may limit what can be kept, so the retention decision and the appeal evidence need a joint review. Confirm retention periods with the privacy and legal owners rather than assuming a default.

How an assessor tests end user reporting and appeals

This table compares what a reviewer tests, the evidence requested and the findings that usually come up for MEASURE 3.3. Read it as a sequence: sample the feedback, trace each item to a disposition, then check whether results reached evaluation metrics. Since no certification body issues a MEASURE 3.3 result, the reviewer may be internal audit or a customer's reviewer.

Test step	What the reviewer does	Evidence requested	Common findings
Confirm channels exist	Checks that users and affected communities can report problems and appeal outcomes through documented, reachable channels	Channel inventory, published instructions, intake form or ticket queue configuration	Channels exist only in engineering notes or are not linked from the product
Sample reports and appeals	Selects a sample across the reporting period and across system outcome types, including appeals of automated decisions	Intake log with timestamps, report category, source and linked system output	Sample drawn only from closed, successful cases, so open or rejected items are missing
Trace each item to a disposition	Follows each sampled item to a recorded decision, owner, action and closure date	Ticket history, triage notes, decision record, reviewer identity and final outcome	Dispositions recorded in email or chat, or items closed with no stated reason
Measure error reporting effectiveness	Asks whether the organization measures how well the reporting process works, such as time to acknowledge and time to resolve	Defined reporting metrics, dashboard or report history, and the threshold owners use to act	Metrics collected but never reviewed, or no target against which performance is judged
Categorize appeal types, rates and results	Checks that appeals are grouped by type and that rates and outcomes (upheld, reversed, modified) are tracked over time	Appeal taxonomy, periodic summaries, and examples of changes made after reversed appeals	Appeals counted in total only, with no breakdown by type or outcome
Check that results reach evaluation metrics	Confirms that feedback and appeal findings feed back into the metrics used to evaluate the system and into retraining or rule changes	Metric definitions that cite feedback sources, change requests linked to feedback, and before-and-after metric records	Feedback reviewed in a meeting but absent from the evaluation metrics or model change history
Measure participation and awareness	Asks how the organization knows people know the channels exist and whether they use them	Survey or usage data, in-product prompts, and records of outreach to affected communities	No measure of awareness, so low reporting is read as low problem rates
Review analysis with domain experts	Checks that feedback on sensitive or high-impact issues is reviewed with subject matter experts, not only the engineering team	Meeting notes or review records listing domain reviewers and the findings they accepted or rejected	Experts consulted once at launch and never again, or review records that name no reviewer

How to build a reporting and appeals process that auditors can follow

This sequence turns MEASURE 3.3 into a working process: named reporting and appeal channels for each deployed system, consistent triage, accountable reviewers, outcomes that feed evaluation metrics, and periodic checks that the channels still work. Each step ends with a record an assessor can trace from a complaint to a measured result.

Register reporting and appeal channels for each system
Start with an inventory of every deployed AI system, its intended users and the communities it affects. For each system, the owning team records every route by which end users, operators and affected people can report a problem or appeal an outcome, such as a web form, a support queue, an email address or a contact point for a public service. Record who staffs each channel and where its tickets are stored. The output is a channel register, reviewed by the system owner, that shows no deployed system lacks a documented way to raise a concern.
Set intake rules and classify each report
Define the minimum information every report must capture, such as the system involved, the date, the outcome disputed, the user's role and whether the problem affects a protected or vulnerable group. Add a classification scheme covering error, harmful output, bias concern, privacy issue, security incident and appeal, so that reports can be counted and compared later. Configure the helpdesk or ticketing tool to apply these fields at intake. The output is a written intake standard and a tested ticket template, approved by the risk owner.
Write the triage and escalation procedure
Describe how each new report is first read, which reports go to the AI product team, which go to security or privacy teams, and which go to a governance committee. Specify who may close a report without review, what evidence is required to close it and when a report must be linked to an existing incident under the organization's incident process. Test the procedure on a set of sample tickets, including at least one that involves a suspected security problem. The output is a triage procedure with an escalation matrix that the support staff have acknowledged in writing.
Assign appeal owners and domain reviewers
Name an accountable appeal owner for each system who can change or override an outcome, and separate that person from the reviewer who first made or tuned the decision where the setting allows it. Identify domain experts, such as clinicians for a diagnostic tool or credit specialists for a lending model, who can assess disputed outcomes and whose review is logged. Confirm that reviewers have the authority, training and access to the underlying records they need. The output is a roles table with the appeal owner, reviewers and their qualifications, approved by the governance function.
Log appeals and record their results
Every appeal is recorded in an appeal log with the original outcome, the grounds raised, the reviewer's findings, the decision, the time it took and whether the outcome changed. Keep the reasoning in the record rather than only the final answer, because assessors test whether decisions can be reconstructed. Periodically categorize appeals by type and by affected group so that patterns are visible. The output is an appeal log that a third party could read end to end for a sample of cases.
Connect report and appeal outcomes to evaluation metrics
Define the metrics that the reporting data feeds, such as the rate and type of reports per system, the share of appeals that change an outcome, the proportion of reports tied to a known error, and the themes that domain experts identify. Document how each metric is calculated, the data source, the owner and the decision it informs, including whether it triggers changes to monitoring, retraining or deactivation under the organization's MANAGE functions. Ask domain experts to validate the metric definitions. The output is a metrics definition sheet that links each measure to the system's evaluation plan.
Schedule reviews of channel effectiveness
Set a recurring review, at an interval the governance body chooses and documents, that examines whether people can find and use each channel, whether reports are triaged as the procedure requires and whether affected communities are aware of the appeal route. Include a check of the metrics from the previous step and of any open escalations. Record findings and actions, including changes to channels, forms or staffing. The output is a dated review record signed by the governance owner, with the actions and their owners listed, which shows the process is maintained rather than set up once.

Evidence that shows reporting and appeals work between audits

MEASURE 3.3 is not satisfied by having a feedback form. An assessor will look for records showing that reports and appeals were received, triaged, decided and fed back into how the AI system is measured. The artifacts below connect those steps, and each one needs a named owner and a refresh rhythm agreed before the next review.

Register of feedback channels

A current list of every route by which end users, operators and affected communities can report a problem or appeal an output, kept per AI system. For each channel, record who can use it, where submissions land, who owns the queue and whether the channel is advertised to users. Check that the register matches what users can actually see in the product, because a channel that exists only in a design document cannot show participation or awareness. Owned by the system owner and reviewed whenever the interface or intake route changes, and checked at each planned audit cycle.

Intake and triage records

Timestamped records of each report from first receipt through categorization, severity, assignment and closure. Ticketing tools such as GLPI or Zammad can hold this trail, provided fields for AI system, failure type and affected group are mandatory. Confirm that triage decisions carry a reason and that reports are not silently closed as duplicates. Operations staff keep these current daily, and the program owner samples them monthly to confirm the queue is not stalling.

Appeal log with decisions and reasons

A log of each appeal of an AI-influenced outcome, with the decision, the reviewer, the reasons given to the person and whether the original outcome was upheld, changed or sent for human review. Categorize appeals by type and track results over time, since MEASURE 3.3 expects the type, rate and outcome of appeals to be analyzed. Confirm that appeal reviewers are independent of the team that produced the original decision. The compliance lead reviews the log at least quarterly and before any audit.

Metric definitions that use feedback data

Written definitions for the evaluation metrics that draw on reports and appeals, such as error-report rate per system, appeal overturn rate and time to resolution. Each definition should state the data source, the calculation, the threshold that triggers review and the owner. Assessors look for evidence that feedback changed the metric set, not only that metrics exist. Measurement owners update definitions when the system changes and review them at each evaluation cycle.

Participation and awareness measures

Measures of how many users and affected community members know about and use feedback channels, such as submission counts relative to active users, survey results on channel awareness and the share of users who reach an appeal path. These show whether the process works in practice rather than existing on paper. Product and communications teams own the data, and results are reviewed in each quarterly governance meeting. Confirm the survey method and sample are documented so the figures can be reproduced.

Domain expert review notes

Records of sessions in which domain experts and, where relevant, representatives of affected groups review clustered feedback and agree on root causes and corrective actions. The notes should name attendees, the feedback analyzed, the conclusions and any change to the system or its metrics. The NIST AI RMF Playbook suggests analyzing feedback with domain experts, so these notes are the clearest proof of that step. The program owner schedules reviews on a fixed cycle and keeps the notes with the related change records.

How the Generative AI Profile adds MEASURE 3.3 actions for AI-generated content

NIST AI 600-1, the Generative AI Profile published in July 2024, adds five suggested actions under MEASURE 3.3, numbered MS-3.3-001 to MS-3.3-005. They are tagged to three generative AI risks: harmful bias and homogenization, human-AI configuration, and information integrity. Two of them matter most for an end user reporting and appeals process. The first, MS-3.3-001, asks teams to conduct impact assessments of how AI-generated content may affect different social, economic and cultural groups. The second, MS-3.3-005, asks teams to record structured feedback about content provenance from operators, users and communities. Read the actions against the base MEASURE 3.3 subcategory, which covers the feedback channel itself. The Profile does not replace that channel; it tells you what the channel should be able to surface for generated content.

For generative systems, a complaint about an output is often a complaint about where it came from. A user may dispute whether a summary cites a real source, whether an image is synthetic, or whether a generated answer reflects a skewed view of a community. Build the reporting form and ticket fields to capture provenance: the model version, the prompt or workflow that produced the content, whether retrieval sources were used, and whether the user believes the content was machine-generated. Storing these fields in the same ticket as the complaint is what makes MS-3.3-005 testable. Confirm with your own engineering team which of these values your logging already records, since the tooling you choose will determine how much of this is captured automatically.

When a team adopts these two actions, its evidence changes in specific ways. It gains a documented impact assessment by group, with the reasoning for which social, economic and cultural groups were considered and who from those groups or their representatives reviewed the findings. It gains a provenance-tagged feedback log that can be broken down by content type and by the reporter's role. It also gains evidence that the MEASURE 3.3 metrics, such as appeal rates and outcomes, are cut by these dimensions. The Profile is a voluntary companion to the AI RMF, not a separate certification or a requirement. Its actions are suggestions that a team can adopt, adapt or decline, and an independent assessor will judge whether the evidence supports the outcomes your own program claims.

Which open-source tools can produce MEASURE 3.3 evidence

Ticketing tools are the most direct source of evidence for the reporting and appeal part of MEASURE 3.3. They record who raised a problem, what the outcome was, how long it took to resolve and whether the same issue recurred. Two open-source options fit this job, and both can be self-hosted inside the organization's own infrastructure.

GLPI Assistance module: Lets users create, follow and process tickets, problems and changes following ITIL practices. Its plannings, statistics, recurrent ticket tracking and Kanban view produce counts and trends that an assessor can compare against the appeal and error metrics the organization chose. GLPI is licensed under GPL-3.0-or-later, so review the license terms with counsel before modifying or redistributing it.
Zammad: A web-based helpdesk licensed under AGPL-3.0-only that brings email, chat, telephone and social media messages into one ticket system. This helps when reports arrive through several channels and need a single record. The AGPL network terms can apply if you modify Zammad and make it available to users over a network, so confirm your obligations before customizing it.
Category and outcome fields: Configure categories for error reports and appeals so the data supports the type and rate analysis the Playbook suggests.
Affected-party routing: Set up queues that send appeals to a reviewer independent of the original decision.

Confirm that the ticket fields, retention settings and export options in your chosen tool can produce the reports your metrics require.

What meeting MEASURE 3.3 costs in money and effort

This table lists the cost components of running end user problem reporting and appeals for an AI system, who is paid for each, whether it is one-time or recurring, and what moves it up or down. Use it to structure a request for quote, since no price or duration is published for this work and a quote follows from a written scope.

Cost component	Who is paid or who spends time	One-time or recurring	What drives it up or down
Ticketing and intake tooling	Your organization, or a partner for setup; self-hosted open-source tools carry no license fee but still need hosting and administration	One-time setup, recurring hosting and upkeep	Number of deployed systems sharing one queue; reusing a single helpdesk such as GLPI or Zammad lowers duplicate spend
Engineering time to integrate reporting channels	Internal platform or product engineers, or a partner under a fixed-scope agreement	One-time per channel, plus recurring changes	Number of channels (in-app, email, web form, API) and whether AI interaction logs can be linked to each report
Triage and appeal review staff time	Your operations, support or review staff	Recurring	Appeal volume and how many outcomes are regulated decisions; clear triage rules and severity categories reduce handling time
Domain expert review time	Subject-matter experts who analyze feedback, paid internally or through contractors	Recurring, with peaks after incidents or model changes	Number of languages, communities and use cases; narrower scoping keeps expert hours predictable
Metrics and reporting build-out	Data, analytics or engineering staff who build dashboards for appeal rates, outcomes and reporting participation	One-time build, recurring review	Number of metrics tracked and whether measures feed existing evaluation dashboards; LLM traces from tools such as Langfuse can supply some data
Evidence upkeep	Compliance, governance or risk staff who keep records of reports, appeal decisions and metric reviews	Recurring	Retention periods, audit cycles and how many frameworks draw on the same records
External testing or review	Independent testers or reviewers, paid per written scope	Periodic	Whether affected communities or independent assessors review the process, and how often
Assessor or customer sampling	Independent auditors or customer assessors who sample records; their fees are set by them, not by DeployOpen	Periodic, per assessment	Sample sizes, number of systems in scope and how well records are indexed for retrieval

Who owns appeals, who prepares evidence and what the auditor tests

Your organization owns MEASURE 3.3 in full. Feedback channels, appeal outcomes and the metrics used to judge them are business and governance decisions, and no partner or assessor can make them for you. The people who decide appeals need enough domain knowledge to judge whether an AI output was wrong, harmful or unfair in the affected context, so the decision owner is usually a product or operations lead working with subject matter experts rather than only the engineers who built the model.

Appeal decision authority: Name who can uphold, reverse or escalate an appeal, and document the criteria they apply.
Domain expertise: Confirm which reviewers understand the affected population, such as clinicians, underwriters or case workers, and when they must be consulted.
Accepted metrics: Decide which measures the business will act on, such as error report resolution time, appeal reversal rate by category or participation in feedback activities.
Thresholds and escalation: Set the conditions that trigger a review of the system itself, not only the individual case.
Records: Keep a dated record of who decided what and why.

Before relying on any metric, confirm with your governance owner that it has a documented owner, a review cadence and a link to a decision someone is accountable for.

Where feedback programs fall short of MEASURE 3.3

Most organizations have some way for users to complain, but MEASURE 3.3 asks for more: a feedback process that people can find, that has an owner, and whose results feed the system's evaluation metrics. The gaps below are the ones assessors and internal reviewers most often find when they trace a report from the user to a changed metric.

Channels exist but are unpublished

A reporting form buried in an internal wiki does not give end users or affected communities a practical way to report a problem. Check that every deployed AI system names its reporting channel in the user interface, in the help text and in the customer documentation, and that the channel is tested from the outside. Keep a dated record of where each channel is published and who approved the wording. Ask whether impacted communities that do not use the product can reach the channel at all.

Appeals with no owner or deadline

An appeal that nobody is assigned to will sit until the user gives up. For each system, assign a named owner for appeals, a documented review path and a maximum response time that the organization sets itself. Record each appeal's outcome and whether the original decision was reversed. Without these fields, the organization cannot show that appeals work as a control.

Feedback never reaches metrics

MEASURE 3.3 requires feedback to be built into evaluation metrics, not stored beside them. Map each feedback category to a specific metric in the evaluation plan, such as error rate by user group or appeal reversal rate, and set a review cadence. Confirm that the TEVV team can export feedback data and that a change in feedback trends triggers a documented reassessment. A feedback log that no metric reads is a finding waiting to happen.

No measure of awareness or participation

The Playbook suggests measuring participation in feedback activities and awareness of them, which shows whether the channel is used or simply exists. Track how many users see the channel, how many submit reports, and which groups submit none. Low numbers from a specific community are a signal to revisit access and outreach, not only the design of the form.

Reports handled outside the system of record

When reports arrive by email or chat and are tracked in personal spreadsheets, the organization cannot prove completeness, response times or trends. Route every report and appeal into one ticketing system with categories, timestamps, status history and links to the affected system version. Export the records on a schedule so the evidence survives staff changes. Confirm which system is the authoritative record before the first audit request arrives.

No domain expert review

Reports about medical, financial or hiring outcomes are often misread by engineers who lack subject knowledge. Establish a review step in which domain experts classify reports, judge whether a pattern indicates harm, and recommend changes to evaluation criteria. Record the reviewer, the date and the decision for each adjudicated item. This review is the link between user feedback and the expert validation that MEASURE 4 expects.

What buyers ask about MEASURE 3.3 feedback programs

How much does a MEASURE 3.3 feedback program cost?

No public price applies to MEASURE 3.3, because the NIST AI RMF is voluntary and nobody sells a certificate for it. Cost comes from the work: designing the reporting and appeal channels, connecting them to ticket handling, setting up metrics for error reporting and appeal outcomes, and running review sessions with domain experts. Costs split into one-time setup (scope, process design, tooling configuration, and documentation) and recurring effort (triage, analysis, and periodic metric reviews). The main drivers are the number of AI systems in scope, how many user groups and affected communities must be reached, whether an existing helpdesk can be reused, and how many appeal types exist. Ask any partner for a fixed-scope quote that lists each deliverable and who performs it. DeployOpen does not publish prices and issues quotes from written scope.

How long does it take to set up feedback and appeal processes?

Timing depends mostly on how many systems are covered and how much of the channel already exists. A team that already runs a helpdesk can often extend it with new categories and routing rules, while a team starting from email threads needs intake design, ownership rules and reporting first. Other drivers include legal review of appeal rights, the number of impacted groups that need accessible channels, and how often domain experts can meet to review feedback. Ask for a timeline tied to written milestones, with named dependencies on your staff and approvals. No duration is given here because none has been researched for this topic.

What does the work require from our internal staff?

Your staff must own decisions that no outside party can make for you. These include which outcomes are appealable, who may reverse a system output, what counts as a reportable error, and which risks the feedback should feed back into. Operations staff need to triage tickets consistently, TEVV staff need to turn feedback into metric changes, and a named accountable owner must review results on a set cadence, consistent with your GOVERN 2 accountability structure. Expect the core team to spend recurring time on review, not only on setup. Confirm named owners and backup coverage before the engagement starts.

Can our internal team implement MEASURE 3.3 without a partner?

Yes, an internal team can implement MEASURE 3.3 without an outside partner, since the framework is voluntary and sets outcomes rather than prescribing tools or steps. The NIST AI RMF Playbook offers suggested actions, documentation prompts and references for each subcategory, though NIST says it is neither a checklist nor a set of steps. A partner is worth considering when the team lacks experience connecting helpdesk tools to AI monitoring, needs help with self-hosted deployment, or wants an independent review of metrics. Where a partner is used, the organization still owns the controls and decisions, and the partner prepares systems and evidence. Only an independent assessor can issue an assessment report.

Is there a certification for MEASURE 3.3?

No. There is no certification, accreditation or conformity assessment for MEASURE 3.3 or for the NIST AI RMF overall. NIST does not validate or endorse any organization's use of the framework, and NIST's AI RMF use cases page says so directly. A customer or regulator may still ask you to show how you meet MEASURE 3.3, so the practical question is what evidence you can produce. Any claim of certification for this subcategory should be treated as a red flag. Ask a vendor or consultant to show their written scope and the evidence they will prepare, not a certificate.

What will a customer assessor sample for MEASURE 3.3?

A customer assessor, auditor or procurement reviewer is likely to ask to see how reports reach the system owner, how each appeal is recorded and resolved, and whether results change the system. Typical samples include a set of recent user reports with their triage and closure records, appeal logs showing type, rate and outcome, the metrics used to judge error-reporting performance, and records of expert analysis of feedback. Evidence should show the process operating over time, not only a design document. Assessors also check awareness of the channels among end users and affected communities. Ask the customer which evidence they expect before you set up logs, since a sampling plan is their choice, not yours.

Which parts of MEASURE 3.3 are worth outsourcing?

Outsourcing is most useful for work that needs specialist skill or an independent view. Strong candidates include designing the intake and ticket workflow, configuring a self-hosted helpdesk, connecting it to trace and evaluation data, and building metric dashboards. A partner can also run an independent review of how well error reporting works. Keep the decisions on appeal rights, risk acceptance and accountable ownership in-house. Make sure the contract states who keeps the ticket data, the configuration and the documentation when the engagement ends. DeployOpen plans, implements, integrates and documents self-hosted platforms within your own infrastructure, and prepares evidence for assessments, but does not issue audit reports or certifications.

How DeployOpen works on this

DeployOpen helps organizations set up the systems, workflows and evidence that let end users and affected communities report problems and appeal AI system outcomes, and that feed those signals into evaluation metrics. The organization owns its AI risk decisions and its MEASURE 3.3 control design, while DeployOpen prepares the platforms and records behind it.

How we scope

The written scope names the AI systems, user groups and channels covered by reporting and appeals.
We confirm which MEASURE 3.3 suggested actions the organization is adopting and which owner signs off on each.
Engagements are fixed-scope, with deliverables and timelines agreed in writing before work begins.

How we build

We configure the ticketing and helpdesk platforms that capture reports, appeals and their outcomes in your infrastructure.
We connect model traces and evaluation logs so that complaint categories can be compared with measured system behavior.
We test intake, routing, escalation and appeal closure with sample cases before the workflow is handed over.

How we hand over

We document the reporting channels, triage rules, appeal steps and metrics such as response and reversal rates.
We deliver a reporting schedule and template for analysis with domain experts and the people who own the AI system.
We provide evidence records that show how feedback changed evaluation metrics, so an independent assessor can review them.

How we support

Ongoing support, upgrades and continuous testing of the reporting platforms are available as a retainer.
Retainers can cover periodic review of appeal trends, participation and awareness measures as the AI system changes.
Independent auditors and certification bodies issue any assurance reports, since DeployOpen prepares systems and evidence only.