How Apache NiFi supports GDPR records of processing and security of processing
A data team running Apache NiFi flows that move personal data, asked by its compliance lead for Article 30 records and Article 32 evidence, should expect the platform to make flows documentable and to produce provenance, but not to write the records or certify compliance. NiFi can show where personal data enters, how it moves between processors and where it leaves, which gives the controller and processor records a factual basis. The records themselves, the lawful basis, the retention decisions and the sign-off remain the organization's work. Apache NiFi GDPR records of processing are therefore a joint exercise between the flow team, the data protection officer and the business owners of each purpose.
On the platform side, the controls that help most with Article 30 and Article 32 are the ones you configure: TLS on every listening port and remote connection, authentication through one of the supported providers such as LDAP, OpenID Connect or SAML 2.0, and policies that restrict who can view, modify or replay data in each process group. Provenance events record the flowfile lineage and can be retained and queried, which supports Article 30(2) entries on categories of processing and gives auditors something to test. Pseudonymisation in flight, such as replacing direct identifiers with keyed tokens before data reaches analysts, is a design choice you make in the flow. Confirm with your counsel whether the tokens meet the separation requirement in Article 4(5) and where the mapping key is held. Confirm the security settings against the NiFi System Administrator's Guide for your version.
NiFi does not decide whether processing has a lawful basis, whether a transfer to a third country is permitted, how long each category of data is kept, or whether a data protection impact assessment is needed, and it cannot answer a supervisory authority on your behalf. Those decisions must be documented outside the flows and then linked to them. DeployOpen plans and implements the flow design, access model, logging and evidence collection, and prepares documentation for your assessors. DeployOpen does not issue audit reports, attestations or certifications, and certification under Article 42 is issued by accredited bodies, not by an implementation partner. Ask any partner to state in writing which records they will produce and which remain yours.
Mapping GDPR Articles 30 and 32 to NiFi capabilities and configuration
This table maps each GDPR requirement in scope to the NiFi capability that can support it and the configuration you need to set. Read the last column as the work your team must do, and treat NiFi features as documentation aids that feed your records and controls, not as compliance proof on their own.
| Requirement | NiFi capability | Configuration needed |
|---|---|---|
| Article 30(1): controller records | Process group descriptions and canvas annotations can hold purposes, data subject categories, recipients and erasure periods alongside the flow | Maintain a register outside NiFi that links each process group to controller contact details, purposes, data categories, recipients and third-country transfers |
| Article 30(2): processor records | Flow documentation records the processing categories run for each controller, and provenance shows where data was received, routed and sent | Tag process groups by controller, record security measures, and confirm which transfers the flow makes |
| Article 30(3) and (4): written form and availability | Flows versioned through the GitHub or GitLab Flow Registry Clients keep an electronic history of changes to the processing | Store exported records and flow versions where the supervisory authority request can be answered, and confirm your retention for those records |
| Article 32(1): appropriate technical and organisational measures | TLS and authentication providers (single-user, LDAP, OpenID Connect, SAML 2.0, X.509) control access, and the UI binds to 127.0.0.1 without security configuration | Enable TLS on every hop, pick an authentication provider, set authorization policies, and run NiFi on a supported Java 21 runtime |
| Article 32(2): risks from destruction, loss, alteration or unauthorised disclosure | Provenance records events for each FlowFile, and bulletins surface failed or misrouted data | Define which provenance and bulletin events are retained, restrict who can query provenance, and document the risks each flow addresses |
| Article 32(1)(a): pseudonymisation and encryption | Flows can apply pseudonymising or encrypting processors before data leaves a trusted boundary, and provenance records when this happened | Choose the processors and keys in your design, store keys separately from flow data, and test that clear values do not reach the wrong destination |
| Article 4(5): pseudonymisation definition | NiFi does not decide whether data is pseudonymised; the flow only applies the transformation you design | Keep the additional information needed for re-identification in a separate, access-controlled system, and document that separation in the flow record |
What evidence a NiFi platform can give a GDPR assessor
A supervisory authority asking for records of processing will want to see how the flows were built, what they did with personal data and who could change them. The cards below describe the evidence NiFi can produce for each of those questions, how to keep it and who should be able to read it. None of this evidence replaces the Article 30 record itself, which the controller must maintain as its own written document.
Versioned flow definitions in Git
Flows deployed through a Git-backed Flow Registry Client give you a commit history of every processor, connection and parameter change, tied to the author and the review that approved it. Check that production only runs flow versions that exist on the protected main branch, and that the repository keeps the full history rather than squashing it. Retain the repository under the same policy as the processing record, and restrict write access to the team that owns the flows. Confirm the exact client and repository setup with your NiFi version before you rely on it.
Provenance event records
Provenance records show where each FlowFile came from, which processors touched it, where it was routed and where it was sent. For Article 30 this answers the question of which recipients actually received personal data, not only which ones the design intended. Decide how long provenance is kept, since that period sets how far back you can answer a request, and make sure the provenance repository is stored on a protected volume. Check that provenance content views are limited to staff with a documented need, because the event data itself can contain personal data.
Authentication and authorization configuration exports
Exports of the login provider settings, user and group policies, and component-level access rules show who could view, modify or operate each flow at a given time. Take them on a schedule and after each access change, and store them with the flow versions they relate to. Confirm that the provider in use, such as LDAP, OpenID Connect or SAML 2.0, is the one your records state, and that no local single-user accounts remain on production nodes. An assessor will compare these exports with the staff list you give them.
TLS configuration on each hop
The security configuration and the endpoint settings of each processor show whether a transfer used TLS and which certificate authority it trusted. Collect these settings per flow and map each external destination to the transfer basis recorded in the processing inventory. Review the certificate expiry dates and the protocol settings during each review cycle, and keep the results with the flow version they describe. Where a flow sends data to a third country, the transport encryption is only one part of the assessment, so link these records to the transfer analysis.
Processing inventory kept beside the flows
A list that maps each flow to its purposes, data subject categories, personal data categories, recipients, storage locations and erasure periods gives the Article 30 record its structure and keeps it in step with the flows. Store the inventory in the same repository as the flow definitions and require a change to the inventory in the same review as any change that alters a recipient, a data category or a retention setting. Name an owner for each flow who signs off the entry. Reconcile the inventory with the flows on each review, because a record that drifts from production is weaker evidence than no record at all.
Pseudonymizing personal data in flight without losing the key
Tokenization replaces a direct identifier, such as an email address or national identifier, with a random token. The original value is written to a mapping store that sits apart from the flow data, so only an authorized service can reverse the token. This is the option that most clearly fits the Article 4(5) definition of pseudonymisation, because the additional information that links the token to the person is kept separately and protected by technical and organisational measures. Under that definition, the mapping store is not a detail of the flow. It is its own asset that needs an owner, an access list, a retention rule and an entry in the records of processing.
- Separate location: Host the mapping store in a different network segment, account or database instance from the flow nodes, with credentials that the flow identity cannot read in plain form.
- Token generation: Generate tokens randomly rather than deriving them from the input, so that the token cannot be recomputed by anyone who guesses the original value.
- Access logging: Log every detokenization request with the requester, purpose and time, and feed those events into your monitoring.
- Lifecycle: Define how long mappings are kept and how erasure requests remove both the mapping and any tokens that remain in downstream systems.
- Processor choice: Confirm the lookup, cache and database processors you plan to use against the component documentation for your NiFi version before you design around them.
Check who can query the mapping store, where backups of it are kept, and whether the store is listed as a separate processing activity with its own security measures.
Keyed hashing computes a message authentication code, such as an HMAC, over an identifier using a secret key. The same input always yields the same output, which keeps records joinable across datasets without exposing the identifier. The security of this option depends almost entirely on the key. If the key leaks, anyone who holds a list of candidate identifiers can recompute the outputs and link them back to people. Because the key is the additional information in this design, it must be kept separately from the data and from the flow definition, and it must be documented as an asset in its own right.
- Key storage: Keep the key in a dedicated secrets manager or hardware-backed store, and reference it from NiFi through a sensitive parameter rather than writing it into the flow definition.
- Key scope: Use separate keys for separate purposes and zones, so that a compromise in one dataset does not link records across all of them.
- Input hygiene: Normalize identifiers, such as case and whitespace, before hashing, because small differences produce different outputs.
- Rotation: Decide whether rotation is needed and what happens to joins across the old and new keys before the first key is issued.
- Verification: Test that a sample of outputs cannot be matched to known identifiers without the key, and record the test result as evidence.
Confirm with your security team how the key is generated, who can read it, and how a key compromise would be detected and handled.
Field-level removal drops or generalizes identifiers before data crosses a zone boundary, for example from an EU-hosted flow into a system outside the EEA or into a shared analytics store. Removing a field is not pseudonymisation when the remaining data can still be linked to a person by other means, so the decision needs an explicit assessment of the fields that stay. Free-text fields, timestamps, postal codes and rare combinations of attributes often carry identifying information even after the obvious fields are gone. This option is usually the simplest to build in NiFi, but it is also the easiest to get wrong because the flow may pass the original record through a branch that was never updated.
- Field inventory: List every field in each record type and mark it as removed, generalized or kept, with the reason for each decision.
- Free text: Treat free-text and attachment fields as high risk, since they can contain identifiers that no schema lists.
- Branch control: Confirm that every path to the destination, including error and retry branches, applies the same removal, because a single unmodified branch defeats the control.
- Provenance checks: Use provenance to confirm that the output content has no removed fields before it leaves the zone.
- Residual risk: Record the fields that remain and why they are considered acceptable for the stated purpose.
Ask whether the removal rules are tested against sample data that contains the edge cases your records actually hold, and whether the resulting data is still personal data for the purposes of the records of processing.
Where NiFi flows create third-country transfers
A NiFi flow can create a third-country transfer without anyone choosing to send data abroad. EDPB Guidelines 05/2021 state that remote access from a third country counts as a transfer, even when the person only views data on a screen for support or administration, and that storing data in a cloud outside the EEA is also a transfer. For NiFi, that means the cluster's physical location is not the only question. A node in an EU data center administered by staff in another country, a remote site whose nodes are reachable from abroad, or a vendor support session that opens provenance events or flow content can each be a transfer. Map where the cluster is hosted, where administrators and support staff log in from, and where provenance repositories and content repositories are stored, including any backups.
Every flow that sends personal data to a SaaS endpoint, a cloud bucket or a database outside the EEA needs its own entry in the Article 30 record, naming the destination country and the Chapter V basis relied on. Article 44 applies to onward transfers as well, so a processor step that forwards data to a further recipient abroad must be captured too. The EU-US Data Privacy Framework adequacy decision, Commission Implementing Decision (EU) 2023/1795, covers only US organizations on the Data Privacy Framework List, and an appeal in Case C-703/25 P is pending. Record which recipients are certified, check the list at the time of each review, and treat the appeal as an open risk. Article 83(5)(c) allows fines of up to EUR 20 million or 4% of worldwide annual turnover for transfer infringements, which is the reason to trace transfers flow by flow rather than in a single summary. Confirm the transfer tools you rely on, such as standard contractual clauses or an adequacy decision, with your counsel and data protection officer.
Configuring NiFi for an auditable GDPR data flow inventory
This sequence turns NiFi into a documented, attributable and version-controlled source for the flow-level facts a records of processing needs. It ends with a tested change record that a reviewer can follow from a flow edit back to the purpose it serves.
Inventory personal data by flow and purpose
The privacy lead and the flow owners list every planned flow that touches personal data, then record the purpose, the categories of data subjects and personal data, the recipients and any third-country destination for each one. Article 30(1) and 30(2) drive the fields, so the inventory is the source for the rest of the sequence. The step is done when every flow on the NiFi canvas has a matching entry, and any flow without one is flagged for review before it is promoted.
Set up the cluster with TLS and authentication
The platform team installs NiFi and sets a deliberate security configuration before any flow carries personal data. The administration guide binds NiFi to 127.0.0.1 when no security configuration is provided, so the team must set the bind address, TLS for the UI and cluster traffic, and an authentication provider such as LDAP, OpenID Connect, SAML 2.0 or X.509 client certificates. Single-user access should not be the production model because it does not attribute actions to people. The step is done when each user logs in with an individual identity and a test login is recorded in the logs.
Store flow definitions in Git
The team connects a Flow Registry Client, such as the GitHub or GitLab client, to a repository that the change owners control, then versions each process group from the canvas. Commits and their authors become the history of what each flow did at any point, which an auditor can compare with the inventory. Confirm with your own administrators how the client authenticates and which branch protections apply, because those settings are environment-specific. The step is done when a process group is committed, and the commit can be traced to a named user.
Document each process group
Each process group gets a description and a set of parameters or annotations that carry the purpose, data categories, recipients, transfer destination and retention period from the inventory. Keeping these values inside the flow definition means the Git history records changes to them too. Where a retention period is not yet decided, the field should say so instead of being left blank, so the gap stays visible. The step is done when a reviewer can read any process group and match it to a single inventory entry.
Set provenance retention to the record-keeping decision
The administrator configures the provenance repository settings, including how long provenance events are kept, to match the retention and evidence decisions the privacy and security owners have agreed. Provenance records what happened to each FlowFile, so its retention is a record-keeping choice, not a default to accept. Check the retention settings in the administration guide for the NiFi version in use, and record the chosen values in the inventory. The step is done when the configured values match the signed-off decision and a test flow shows provenance events for the expected period.
Restrict change rights and control evidence export
The administrator assigns policies so that only named flow owners can modify process groups, and only approved users can view or export provenance data and bulletins. Read access to flows and write access to flows should be granted separately, and the list of users with each right should be reviewed and recorded. Exports of provenance and logs are treated as personal data copies, so they need the same access discipline. The step is done when a non-authorized account is blocked from editing a flow and the access list is saved as evidence.
Test that a flow change produces a reviewable record
The team makes a test change to a non-production process group, such as adding a processor or altering a parameter, and commits it through the registry. It then checks that the change shows the author, the timestamp and the diff in the repository, that the inventory entry is updated through a review step, and that the provenance and log entries for the run are retrievable. The step is done when a reviewer can follow the test change from the commit to the inventory record, and the result is written into the change log for the platform.
Retention and erasure limits NiFi cannot decide for you
Article 30(1)(f) asks the controller to record the envisaged time limits for erasure of the different categories of data, where possible. NiFi cannot set that limit for you. It can only hold data for as long as its own repository settings allow, and those settings were not chosen with your retention schedule in mind. The first task is to write the erasure limit for each personal data flow in the record, then trace every place that flow leaves a copy: the content repository, the FlowFile repository, the provenance repository, any queue that holds data during a backlog, and the destination systems. Each of those copies needs a retention rule that matches the stated limit, or a documented reason why it does not.
The provenance repository is the usual gap. Provenance events record attributes and content references for each step, and they can outlive the content they describe. The content repository keeps claim files until no FlowFile references them, so a long-retention setting can keep payloads for much longer than the flow itself needs. Confirm in the administration guide for your exact NiFi version how the content, provenance and FlowFile repositories age out data, which properties in nifi.properties control retention and archiving, and whether archived content remains readable. Test the behavior with a flow that carries known personal data, then measure when that data actually disappears from disk.
Erasure requests under Articles 17 and 19 may therefore need to reach beyond the system of record. Decide in advance whether a request can be satisfied by letting repository retention expire, by purging specific FlowFiles or provenance data, or only by correcting the downstream system. Record that procedure in the same document as the Article 30 record, and make sure the people who can run it know which repositories and destinations hold each data category. Where a retention decision depends on the destination, such as a SIEM or data lake that keeps its own copy, the NiFi configuration alone will not satisfy the limit, and the owner of that system needs to confirm the policy.
What NiFi and its configuration do not cover
A correctly built NiFi flow can document and protect personal data in motion, but it cannot make the legal decisions behind it. The cards below cover obligations that stay with named people in your organization, and the points you should confirm before relying on flow documentation as evidence.
Lawful basis and purpose decisions
NiFi records which processors move personal data, but it does not decide whether the processing is lawful or why it happens. The legal basis and stated purpose for each flow belong to the controller, usually the data protection officer or legal counsel working with the business owner. Confirm that every flow in your Article 30 record traces to a documented purpose, and that flow names, processor descriptions and Git commit messages use the same purpose wording.
The Article 30 record and its ownership
The record of processing is a document that must exist in writing and be produced to the supervisory authority on request. NiFi can supply flow diagrams, processor inventories and provenance, but the record itself needs a named owner, usually the privacy team, who maintains controller and processor entries. Confirm who approves changes to the record and whether flow changes trigger a review of the record before deployment.
Transfer assessments and supplementary measures
NiFi processors can send data to a third country, and remote administration from outside the EEA counts as a transfer under EDPB guidance. Choosing the transfer tool, running a transfer impact assessment and deciding on supplementary measures belong to the controller and legal team. Confirm where every NiFi node, Registry, repository and backup is located, including support access, and whether a supplementary measure is effective for each destination.
Data subject request handling
Access, rectification and erasure requests require a process that finds personal data across flows, queues, content repositories and downstream systems. NiFi provenance can show where data was routed, but it does not decide whether a request is valid or what must be deleted. Assign this to the data protection team with IT operations, and confirm that provenance retention, content repository purging and copies held in destination systems can each be tied to a request.
Breach notification under Articles 33 and 34
Articles 33 and 34 require the controller to assess a personal data breach, notify the supervisory authority and, where needed, inform affected individuals. NiFi bulletins, logs and provenance can help establish what was exposed and when, but the assessment and the notification decision belong to the incident response lead and the data protection officer. Confirm that alerting on flow failures, unexpected routing and access to provenance data feeds into your breach procedure with defined owners and timelines.
Third-country authority access and Article 48
A processor subject to foreign law may be required to disclose data to authorities outside the EU, and the CLOUD Act can reach providers regardless of data location. Article 48 limits recognition of such orders to those based on an international agreement, but whether that protects a given deployment is a legal question. Involve counsel before choosing a hosting provider, support vendor or managed service that can access NiFi data, and document the outcome in the transfer assessment.
Questions buyers ask about GDPR and NiFi
Does NiFi flow documentation alone satisfy GDPR Article 30?
No. Article 30 requires a written record, including electronic form, that the controller or processor can make available to the supervisory authority on request. NiFi can supply much of the raw material: flow definitions under version control, processor configuration, and provenance that shows where data went. But the record also needs contact details, purposes, data subject and data categories, recipients, third-country transfers, and where possible erasure time limits and a general description of security measures. Those business facts come from your legal and data owners and must be written down alongside the flows.
How much effort does a flow inventory and record take, and what drives it?
The effort depends mainly on how many flows touch personal data, how many systems each flow crosses, and how well existing flows are documented. Cost components are internal time from legal, security and data owners, the engineering time to trace each flow, and any partner fees for a fixed-scope inventory. The work is largely one-time, but the record must be kept current, so each new flow or change adds recurring effort. Ask any provider to scope the work by flow count, personal data categories and systems in the transfer path, and to state what they will deliver in writing before work starts.
Can our own team build the record, or do we need a partner?
Your team can do it if it has NiFi engineers who know the flows, a data protection lead who can make the legal determinations, and time to work through both. A partner adds value when the flow estate is large, when flows were built by several people without documentation, or when you need an independent check of the mapping before a supervisory request. DeployOpen plans and documents self-hosted open-source platforms in your infrastructure and prepares systems and evidence for compliance assessments, but it does not issue audit reports or certifications, and legal determinations stay with your organization.
What does the work need from legal, security and data owners?
Legal and the data protection officer must define purposes, lawful bases, recipients, retention periods and whether any transfer is a third-country transfer under Chapter V. Security owners must describe the Article 32 measures in place, such as TLS on each hop, access control to flows, and key handling. Data owners must confirm which fields are personal data, which records are processed, and which processors or subprocessors receive them. Each of these inputs should be signed off and dated, because the flow documentation can only describe what those owners have decided.
Does certification change what we need to do for Article 32?
Certification does not change your responsibility. Under Article 42, certification is voluntary, issued for at most three years by accredited certification bodies or supervisory authorities, and Article 32(3) allows it only as one element in showing compliance. The GDPR itself requires no audit, attestation or certificate. Certification therefore never replaces the record of processing or the security measures in your NiFi flows. Confirm with your supervisory authority or legal counsel how any certification you hold would be presented as evidence.
Do we need to upgrade from NiFi 1.x before building GDPR records?
Not before you start mapping, but the upgrade affects how long the record stays accurate. NiFi 1.28 is the last minor release of the version 1 series, and the downloads page lists end of support for 1.28.1 as December 8, 2024. The project explains that 1.x cannot be maintained further because of its Jetty, Spring and AngularJS dependencies. The migration guidance requires moving to NiFi 1.27.0 before 2.0.0 and changing flow.xml.gz to flow.json.gz. Plan the inventory against the target 2.x configuration so the record does not need rewriting after migration.
What should we ask for in a quote for this work?
Ask for a written scope that lists the flows in scope, the systems each flow touches, the personal data categories you expect, and the records to be produced. Ask which deliverables you receive, such as the flow inventory, the processing record template, the transfer map and the retention table, and in what format. Ask who signs off each input, what assumptions the price rests on, what happens if flows are larger than expected, and whether ongoing updates are included or offered as a separate retainer. Engagements are typically fixed-scope, so the agreed scope should define when work is complete.
Primary sources for GDPR records and security of processing
Start with the regulation itself. The consolidated text of Regulation (EU) 2016/679 is available on EUR-Lex, the EU law database, and two articles govern this page. Article 30 sets what a controller's record of processing must contain and what a processor's record must contain, including the purposes, data subject and data categories, recipients, third-country transfers and, where possible, erasure time limits. Article 32 sets the security duty and names pseudonymisation and encryption as examples of appropriate measures. Read the recitals alongside these articles, since they explain how the risk-based standard is expected to be applied. The GDPR does not require an audit or certificate, so the record and the security analysis are documents you must be able to produce yourself for a supervisory authority.
Next, read the EDPB's small business data protection guide, titled Be compliant, which is written for organizations with limited legal resources. It is a practical starting point for the record of processing and for deciding which personal data your NiFi flows actually touch. Use it to check that your flow inventory answers the same questions the regulation asks, such as who the data subjects are, which recipients receive the data and where it goes. Treat it as orientation rather than a full legal analysis, and confirm any interpretation that affects your lawful basis or transfer position with your data protection officer or counsel.
For the pseudonymisation design, read the EDPB's Guidelines 01/2025 on Pseudonymisation, which were published in a version for public consultation. Check the current status of that document before you cite it, because consultation versions can change. The key idea for NiFi is that Article 4(5) only counts data as pseudonymised when the additional information needed for re-identification is kept separately and protected. In a flow, that means the lookup table, key or tokenization service must sit outside the processors that handle the pseudonymised data, with its own access controls and logging. Confirm with your team which document version your auditors accept as the reference.
How DeployOpen scopes NiFi compliance configuration and evidence
DeployOpen plans, implements, documents and supports self-hosted open-source platforms inside your own infrastructure, including Apache NiFi flows that carry personal data. For GDPR records of processing, the work starts with a written scope that maps each personal data flow to the record fields Article 30 asks for, then builds and documents the controls behind it.
How we scope
- Inventory the NiFi flows, controllers and processors that handle personal data, with owners named for each flow.
- Map each flow to the Article 30 record fields, noting where a field is required and where it is required only where possible.
- Agree in writing which transfer paths, pseudonymization points and retention rules are in scope before any build begins.
How we build
- Configure TLS, authentication and authorization on the NiFi cluster so access to personal data flows is limited and attributable.
- Design pseudonymization steps in flows, with the separately held key or mapping stored and protected outside the data path.
- Set up Git-versioned flow definitions so changes to personal data handling go through review and leave a traceable history.
How we hand over
- Deliver a written record template per flow, populated from the agreed scope and the configured flow design.
- Document provenance and log retention settings, and show how a reviewer can trace a record to the processing that produced it.
- Provide a runbook for keeping records current when flows change, and a list of items your legal and security teams must confirm.
How we support
- Engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts.
- Ongoing support, upgrades and continuous testing are available as a retainer after handover.
- Records of processing and Article 32 measures remain your responsibility. DeployOpen prepares the systems and evidence, and independent assessors judge them.
