Why replace NiFi XML templates before moving to NiFi 2
XML flow templates were deprecated in NiFi 1.14.0, with JSON flow definitions as the replacement, so a team that reuses flows through XML templates should convert them before it starts the move to NiFi 2, not during it. Templates that stay in XML form add a second migration step at the point where the upgrade is already the heaviest part of the work. Treat the conversion as a prerequisite that gets its own change record, its own test run and its own rollback point.
The supported path is staged. The NiFi migration guidance requires upgrading to NiFi 1.27.0 before moving to 2.0.0, and it changes the flow configuration file from flow.xml.gz to flow.json.gz. Your template conversion belongs on the 1.x side of that sequence, where you can compare exported JSON against the running flow before the configuration file format changes underneath it. The 1.x line itself has a limit: the NiFi project explains that it cannot be maintained further because it depends on Jetty 9.4, Spring Framework 5.3 and AngularJS 1.8, none of which can be upgraded within that line. The downloads page identifies NiFi 1.28 as the last minor release of the version 1 series and lists December 8, 2024 as the end-of-support date for 1.28.1. Confirm the current support status of your exact 1.x build against the downloads page before you schedule the work, because a team that waits for a convenient window may find it has no supported release to fall back on.
What changes for templates, flows and registries
This table compares four changes that affect reusable flows before an upgrade to NiFi 2, with the impact on each and the action to take. Read it as a checklist: each row names something you can find in your own instance and decide on before cutover. The main takeaway is that the template, configuration file and registry changes are independent, so each needs its own inventory and sign-off.
| Change | Impact | Action needed |
|---|---|---|
| XML flow templates deprecated in NiFi 1.14.0 | Templates exported as XML are deprecated, with JSON flow definitions as the replacement. Reusable flows stored only as XML need conversion before they can be relied on. | Inventory every XML template, re-export the flow as a JSON flow definition, and test the import in a non-production instance. |
| Flow configuration file changes from flow.xml.gz to flow.json.gz | The migration guidance requires moving to NiFi 1.27.0 before 2.0.0 and changing the flow file format. Backups and scripts that read or copy flow.xml.gz will not work unchanged. | Update backup, restore and deployment scripts to expect flow.json.gz, and confirm the 1.27.0 step is complete on each node. |
| NiFi Registry deprecated, planned for removal in NiFi 3.0 | Versioned flows stored in NiFi Registry have a finite future. Buckets and versions must move before the registry is retired. | Plan the move to a Git-based flow registry client, and record which process groups are currently version controlled. |
| Git-based flow registry clients for GitHub and GitLab in NiFi 2 | Flow versions can be stored in a Git repository, giving review history and branch control. Repository layout and access rules need design before the first commit. | Choose the repository, branch strategy and access model, then connect the process groups and confirm each one can commit and pull. |
| Variable registry removed in NiFi 2.0 | Exported flows that reference variables from the variable registry will not resolve after upgrade. Expression language scope changes also affect how flows read values. | Find variable references in exported flows and move them to parameter contexts, using the ENVIRONMENT scope where the migration guidance applies. |
What to inventory and back up before exporting templates
Exporting templates is the easy part. The work that decides whether the migration is safe happens first, when you know which templates exist, which flows depend on them, and where you can roll back. The six items below connect the inventory, the backup, the test environment and the storage decision into one preparation step.
Inventory every XML template
List each XML template in your 1.x instances, with its name, owning team, the process groups it was built from and every instance where it was imported. Templates are often imported into several environments and copied by hand, so the inventory must cover all of them, not only the production canvas. Record the NiFi version each template was exported from. Use this list as the scope for the conversion and as the checklist you close out after cutover.
Map flows that depend on templates
Identify which running flows were instantiated from a template and whether anyone still edits them through that link. A flow created from a template is a copy, so changing the template later does not change the flow. Map those dependencies so you know which process groups need to be re-exported, which are standalone, and which owners must approve changes. Missing this map is a common reason for drift between what teams believe is deployed and what is actually running.
Back up the 1.x configuration
Before any export, take a copy of the flow configuration file, the flow definition, the authorizer and user configuration, and the properties files. Include the sensitive properties key and confirm you can restore it, because encrypted values in flows cannot be read without it. Back up the flowfile, content, provenance and database repositories only if your recovery plan requires them. Store the copy outside the NiFi host and test one restore on a separate machine.
Stand up a test instance on the target path
Prepare a non-production NiFi that follows the same upgrade path you plan for production, rather than jumping straight to the newest release. The migration guidance requires upgrading to NiFi 1.27.0 before moving to 2.0.0, so the test instance must be able to reach that point. Import the JSON definitions there first, confirm that processors, controller services and parameters resolve, and keep the results as evidence for change approval.
Decide where shared flows live
Choose whether reusable flows will be stored in a Git repository through a flow registry client or kept in a different store. NiFi 2 includes Flow Registry Clients for GitHub and GitLab, and NiFi Registry is deprecated for removal in NiFi 3.0. Decide the repository host, branch strategy, access rights and who approves merges before exporting anything. Changing this decision after conversion means moving every process group twice.
Assign owners and a change window
Name an owner for each template and each dependent flow, plus a person who approves changes to shared process groups. Agree a change window with the teams that run the flows, because exported definitions will be re-imported and may restart components. Confirm how credentials and sensitive parameter values will be supplied at import time, so no secret ends up in a JSON file committed to the repository. Clear ownership makes the rest of the migration traceable for auditors.
How to export, store and re-import flows as JSON definitions
This sequence moves reusable flows from XML templates to JSON flow definitions kept in version control, then confirms they load on a NiFi 2 path before production uses them. Each step ends with a check and a rollback point so the 1.x instance stays usable until the new definitions are verified.
Inventory every template and process group
Open the Templates list on the 1.x instance and record each template name, the process group it came from, its creation date and who owns it. Next, search the canvas for flows that reference variables, parameters, controller services and any component on the NiFi 2 removed or deprecated lists. Save the inventory in a spreadsheet or issue tracker. Done when every template has one row, an owner and a note on its dependencies. Keep the original XML template files untouched as your rollback copy.
Export each flow as a JSON definition
For each process group in the inventory, export it from the 1.x canvas as a flow definition, which is the JSON replacement for XML templates deprecated in NiFi 1.14.0. Confirm the exported file contains the processors, connections and controller services you expect by opening it in a text editor. Note whether sensitive property values are present, because you will need to supply them separately. Done when each group has one JSON file that you have opened and checked. Rollback: the XML template and a copy of the 1.x flow configuration remain the source of truth until step four passes.
Commit definitions to a version-controlled repository
Create a Git repository with one folder per process group and a short README beside each file that records the source template, the 1.x version it came from and the controller services it needs. Use a consistent naming pattern for files, for example one folder per domain, and keep environment-specific values out of the flow files. Commit each group in its own commit so you can revert a single flow later. Done when a fresh clone of the repository contains every definition and a reviewer can read the README without asking the author. Rollback: revert the commit that added the file.
Re-import definitions into a test instance
Stand up a test instance that follows the migration guidance: upgrade to NiFi 1.27.0 first, then move to 2.0.0, with the flow configuration changed from flow.xml.gz to flow.json.gz. Import each definition from the repository, using the Flow Registry Client for GitHub or GitLab if you version flows through NiFi, otherwise through the canvas upload. Review the import log and the processor list for missing or invalid components. Done when every imported group loads without missing components. Rollback: delete the imported group and restore the test instance from its backup.
Verify parameters and controller services
Map every variable reference found in the inventory to a parameter context, using the ENVIRONMENT scope for values that the variable registry used to supply, since NiFi 2 removed that scope. Confirm that distributed map cache components are updated to their renamed NiFi 2 versions. Supply sensitive values through the parameter context, then enable each controller service and confirm it reaches a valid state. Done when all controller services are enabled, no processor reports invalid properties and a test run passes on sample data. Rollback: keep production on the 1.x instance until this check passes.
Promote verified definitions to production
Promote one process group at a time from the repository into production, starting with the lowest-risk flow, and record the commit hash of each version you deploy. Compare provenance events and output counts against the 1.x flow for a defined observation period that your operations team sets. Plan the move away from NiFi Registry now, because the project has deprecated it for removal in NiFi 3.0. Done when each production flow matches the committed version and its outputs match the 1.x baseline. Rollback: redeploy the previous commit or restore the 1.x flow for that group.
How Git-versioned process groups replace template sharing
Git-versioned process groups replace XML templates by moving reusable flow logic into a repository that the team reviews, branches and tags like any other code. In NiFi 2, the GitHub and GitLab flow registry clients let a team version-control a process group directly, so each saved version becomes a commit with an author, a message and a diff. An XML template, by contrast, is a file exported from one canvas and imported into another, with no record of who changed it, why, or which copies are still in use. Once a process group is under version control, teams can see what changed between releases, roll back to a known version and require a merge request before a shared component changes.
The practical shift is in how reuse is governed. Each consuming instance keeps a reference to a specific version of the process group, so an update in the shared repository does not silently alter production until someone chooses to upgrade the reference. Parameter contexts and controller services still have to be wired up per environment, so the reviewable unit is the flow definition, not the environment values. Confirm in the Flow Registry Client documentation for your chosen provider which authentication method, repository permissions and bucket or directory layout it expects before you design the flow.
Two decisions need agreement before any template is migrated. The first is the repository layout, such as one repository per domain, one folder per process group, or a monorepo with shared components, because it determines who can approve changes. The second is the branch strategy, including which branch consuming instances track and how releases are tagged. The move from NiFi Registry to these clients is covered on its own page, so this section assumes the target repository already exists. Agree on both points early, since changing them after several process groups are committed is far more work than setting them correctly at the start.
How the approach differs for single instances and clusters
A single standalone NiFi instance has one flow configuration and one set of local repositories, so the export and re-import work is contained on one host. Your main risks are a broken import that leaves a process group half-applied and a flow that depends on local state that a fresh import does not recreate. Work from a copy of the running flow, import into a non-production instance first, and only then change the live canvas.
- One source of truth: The flow you export is the flow the instance runs, so confirm no one has edited the canvas since the export.
- Local state: Check for processors with stored state, such as listing or tracking processors, and record what they will restart from.
- Controller services: Confirm each referenced controller service is enabled after import, since disabled services stop dependent processors.
- Rollback path: Keep the previous flow definition file and the repository backup together so you can restore the prior canvas quickly.
- Restart discipline: A standalone restart is still a stop point for every flow, so schedule imports when queued data can drain.
Before changing the live instance, check that the JSON definition re-imports cleanly into a test instance and that every controller service and parameter reference resolves.
In a secured cluster, every node must run the same flow definition, and the cluster upgrade sequence governs when a new definition can be applied. A flow that imports on one node but not another causes divergence that the cluster cannot reconcile cleanly. Plan the import as part of the upgrade order, not as a separate change, and confirm that the target version and the flow definition format are agreed before the first node restarts.
- Identical definitions: Store the JSON flow definition in version control and deploy the same revision to every node.
- Upgrade sequencing: Imports that change the flow format belong in the same window as the version change, following the cluster upgrade guidance.
- Sensitive values: Sensitive property values are protected with the sensitive properties key, so confirm the key is present and identical on every node before importing.
- Parameter contexts: Parameter context references must resolve on every node, and any parameter values held outside the flow must be available to all of them.
- Coordination: Confirm that the coordination layer, whether ZooKeeper or Kubernetes state, is healthy before you start the import.
Before the first node restarts, confirm that all nodes report the same flow revision and that sensitive keys and parameter contexts match across them.
Flows that reference sensitive values or parameter contexts behave differently from flows with inline settings, because the definition file alone does not carry the values. The JSON definition records references, while the values live in the target environment. This makes the import portable across environments, but it also means a successful import can still fail at runtime if a value is missing. Treat parameter contexts and sensitive values as a separate deliverable from the flow file.
- References versus values: Export the flow definition without assuming the values come with it, and list every parameter and sensitive property the flow uses.
- Per-environment values: Keep values out of the definition and supply them through the target environment, so the same file works in test and production.
- Sensitive properties key: Changing or losing the key makes stored sensitive values unreadable, so back it up before any import.
- Inheritance: Check which parameter contexts a process group inherits, because a missing parent context changes resolution.
- Access checks: Confirm the importing user can read the referenced parameter contexts and has permission to apply them.
Before import, check that each referenced parameter and sensitive value exists in the target environment and resolves to the expected value.
Cluster size does not change the principle that every node must agree, but it changes how you verify agreement and how long a rolling change takes. Larger clusters make manual checks on each node impractical, so verification should rely on the cluster reporting one flow revision rather than on operators inspecting nodes. The exact number of nodes that affects your timing should be confirmed against your own environment and the upgrade guidance, not assumed from this page.
- Automated comparison: Compare the flow revision reported across nodes through the API or your deployment tooling rather than by hand.
- Staged rollout: Apply the definition in a planned order, and stop if any node reports a different revision.
- Queue draining: Drain queues before import so in-flight data does not reference components that are about to change.
- Monitoring: Watch bulletins and node status during and after the import, since divergence often appears as warnings first.
- Capacity: Confirm that the cluster has headroom to absorb a node restart during the window.
Before starting, ask your team to confirm the expected node count, the verification method and the rollback window for your own cluster.
Air-gapped or regulated environments often cannot reach external registries, so the JSON definitions and any supporting files must travel through an approved transfer path. The approach stays the same, but the evidence trail and transfer controls become part of the work. Plan how definitions are approved, moved and recorded so that an auditor can trace which revision ran on which environment and when.
- Approved transfer: Move definition files through your sanctioned media or transfer process, with checksums recorded at each hop.
- Change records: Tie each imported revision to a change ticket and the reviewer who approved it.
- Separation of duties: Consider whether the person who exports a definition should be different from the person who imports it into production.
- Offline references: Confirm that any component or version the flow needs is available inside the network.
- Evidence retention: Keep the exported files and import logs for the period your policies require.
Before a regulated import, confirm with your security and change management teams which approvals and records they require for each environment.
What makes template migration take longer
This table lists the factors that expand the work of moving XML templates to JSON flow definitions and how to keep each one contained. Read each row as a question to answer during scoping, because the same template can be quick to move in one estate and slow in another. The main takeaway is that inventory and dependency checks before any export shorten the migration more than any single conversion step.
| Driver | Why it adds effort | How to reduce it |
|---|---|---|
| Number of templates and nested process groups | Each template may embed nested process groups, so one exported file can hide many components that must be reviewed and tested separately. | Inventory every template and its nested groups first, then retire unused templates before conversion. |
| Flows that use the variable registry | Variable references must be found and replaced with parameter contexts or the ENVIRONMENT scope before the flow runs on NiFi 2. | Search flows for variable references and map each one to a parameter context before export. |
| Flows that use removed components | Components removed from NiFi 2 need a replacement or a decision to keep a 1.x island, which can change the flow shape. | Flag removed components during inventory and decide replacements before converting templates. |
| Custom processors referenced by the flow | Each custom NAR must be rebuilt for NiFi 2 and matched to the flow definition, so templates cannot move until their code is ready. | List every custom component each template references and sequence rebuilds ahead of the flow move. |
| Number of environments the flows must reach | Each environment needs its own parameter values, registry links and test runs, which multiplies verification work. | Standardize parameter names across development, test and production so one definition deploys everywhere. |
| Sensitive configuration that must be re-entered | Passwords, tokens and keys are not carried in exported definitions, so they must be supplied again and checked in each target. | Keep a controlled list of sensitive values per flow and confirm the secret store plan before cutover. |
| Undocumented flow ownership | Without a named owner, questions about intended behavior stall the conversion and testing. | Assign an owner for each process group and record expected inputs and outputs. |
Questions teams ask before replacing NiFi templates
What does replacing NiFi templates cost, and what drives the quote?
The cost depends on how many templates exist, how many flows instantiate them, and how much of each template depends on environment-specific values. Most engagements are fixed-scope, so the written scope sets the price, and no list price applies. The main cost drivers are the number of templates in use, the number of process groups built from each one, the number of controller services and parameters that must be recreated, and the amount of testing needed before cutover. One-time work covers the inventory, conversion, flow export into version control and re-import testing. Recurring costs appear only if you add ongoing support or a retainer. Ask for a quote that lists each template, the target process group layout, the test scenarios and the handover format.
How long does it take to move templates to JSON flow definitions?
Duration scales with the size of the template estate and how many teams depend on it. A small set of standalone templates with no external dependencies moves quickly. A large library shared across many flows, with environment-specific parameters and custom components, takes longer because each dependency needs checking and each flow needs re-import testing. Timing also depends on how quickly your flow owners can confirm expected behavior and on the availability of a test environment. Ask any partner, including DeployOpen, to state the duration in the written scope after an inventory has been completed, since the inventory is what reveals the real volume of work.
What must our team provide before the work starts?
Three inputs make the largest difference. First, named flow owners who can explain what each template is for, which data it handles and what a correct output looks like. Second, repository access, including read access to the running NiFi instances, the flow configuration, any Git or NiFi Registry repositories, and permission to create a version-control repository for the exported flows. Third, a test environment that matches production closely enough to exercise the flows, with representative input data and non-production credentials. Teams often also need to provide a list of parameter values, certificates and downstream endpoints so that re-imported flows can be validated without touching live systems.
Can our internal team replace the templates without a partner?
Yes, many teams can do this themselves, especially when the template count is small and the NiFi team already works with process groups and version control. The work is mostly methodical: inventory each template, rebuild or export it as a JSON flow definition, store it in version control, re-import it into a test instance and verify the results. Internal teams usually struggle with time pressure, unclear flow ownership and testing coverage rather than with the technical steps. A partner is most useful when the estate is large, when flows depend on components that are changing in NiFi 2, or when the same team must also run the upgrade on a deadline.
What should we do with templates that cannot be exported cleanly?
Some templates fail to convert because they reference controller services, processors or properties that do not map directly to a JSON flow definition, or because they embed values that should be parameters. Treat each failure as a finding, not a blocker. Record the template name, the component that fails, the error and every flow that uses it. Then decide per template whether to rebuild the process group by hand, replace a component that NiFi 2 no longer ships, or retire the template if nothing uses it. Check the NiFi 2 deprecation and removed component guidance before deciding, because some failures are caused by components that are changing in the upgrade rather than by the template itself.
Must templates be replaced before or during the NiFi 2 upgrade?
Replace templates before the upgrade wherever possible. The NiFi migration guidance requires upgrading to NiFi 1.27.0 before moving to 2.0.0 and changing the flow configuration file from flow.xml.gz to flow.json.gz. Doing the template conversion on the 1.x estate first lets you test JSON flow definitions against a known-good baseline, so any later failure can be traced to the upgrade rather than to the conversion. Replacing templates during the upgrade window adds risk because the flow configuration, runtime and component changes all move at once. The exception is a template that depends on a component removed in NiFi 2, which may need to be retired or replaced as part of the same plan.
What happens to flows held in NiFi Registry?
NiFi Registry is deprecated and planned for removal in NiFi 3.0, so flows versioned there should be planned for migration rather than left in place. Export the buckets and flow versions you still rely on, then store them in a Git-based flow registry client repository. The NiFi downloads page and the component documentation describe the GitHub and GitLab flow registry clients, which version flows in a Git repository. Plan the repository layout and branching model before moving flows, and confirm during the inventory which flow versions are the authoritative ones. Note that a dedicated service page covers moving from NiFi Registry to Git-based registry clients in detail.
How to test imported flows and roll back safely
Testing a re-imported flow means proving that the JSON flow definition behaves like the template it replaced, not just that it loads. Start with processor validation: every processor in the imported process group should show a valid state with no missing relationships or unresolved properties. Then check parameter and controller service resolution. Parameter references that were once variable references need to resolve in the target parameter context, and every controller service a processor depends on must be present, enabled and pointing at the right endpoint. Record each unresolved reference as a defect before any data moves, because a flow can validate in the canvas and still fail when a referenced service is disabled.
Data comparison is the test that matters most. Run the same sample input through the original template flow on the 1.x instance and through the re-imported definition on the target, then compare the outputs field by field, including attribute values, record counts, routing relationships and error paths. Use provenance to confirm that each event followed the expected route through the process group, and that failures landed in the failure relationship rather than being dropped. Sample inputs should include malformed records and duplicates, since those reveal differences in error handling that a clean happy-path run hides. Ask the team that owns each flow to sign off on the comparison results, not only the platform team.
Rollback depends on keeping the original artifacts until the cutover is formally accepted. Keep the 1.x configuration, the original XML templates and the exported JSON definitions under version control, and do not delete the 1.x flow until the comparison results are approved and the first production runs have completed without defects. If a regression appears, you can stop the new process group, restore the previous instance, and re-run the sample comparison against the fixed definition. Confirm with your own change management process who has authority to trigger a rollback and how long the 1.x environment must remain available.
How DeployOpen works on this
DeployOpen plans, builds, documents and supports self-hosted open-source platforms inside your own infrastructure, so template replacement is handled as a defined piece of work. Each engagement starts from a written scope that lists which templates and flows move, and the work is delivered and handed over in your environment.
How we scope
- Inventory every XML template in your NiFi 1.x instance and record which flows, groups and teams depend on each one.
- Agree in writing which templates convert to JSON flow definitions, which are retired and which are rebuilt as version-controlled process groups.
- Confirm the target NiFi 2 version, Git repository and registry client choice before any conversion work begins.
How we build
- Export each template as a JSON flow definition and check that processor properties, controller services and parameter references survive the conversion.
- Store converted process groups in a Git repository with a documented folder layout, branch strategy and review step.
- Test each converted flow against representative input in a non-production NiFi instance and record the results.
How we hand over
- Deliver the written inventory, conversion log, repository layout and per-flow test records as project documentation.
- Walk your platform, security and data teams through how to export, commit and re-import a process group.
- Provide a checklist showing which flows are ready for the NiFi 2 upgrade and which still need work.
How we support
- Engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts.
- Ongoing support, upgrades and continuous testing for your flows are available as a retainer.
- Ask any quote to itemize conversion, testing, documentation and handover so you can see what recurs and what is one-time.
