Apache NiFi 1.x to 2.x migration services

What a NiFi 1.x to 2.x migration involves

NiFi 1.28 is the last minor release of the 1.x series, and the project lists an end-of-support date of December 8, 2024 for 1.28.1. The project also states that 1.x cannot be maintained further, because it depends on Jetty 9.4, Spring Framework 5.3 and AngularJS 1.8, and those components cannot be upgraded within that line. A 1.x estate therefore has no supported path forward, and moving to NiFi 2 is the only way to regain upstream fixes.

The migration runs as a controlled sequence through NiFi 1.27.0 before the move to NiFi 2.0.0. Three changes need work on every flow estate. The flow configuration file changes from flow.xml.gz to flow.json.gz. The variable registry expression language scope was removed and replaced by an ENVIRONMENT scope, so every expression that uses the old scope must be found and rewritten. The distributed map cache components were renamed, so each processor and controller service that references them needs checking before cutover.

Most projects start from one of three triggers: exposure from the end-of-support date, a security review that flags the unsupported dependencies, or a need for Java 21 or NiFi 2 features. The administration guide lists Java 21 as the minimum runtime and lists Python 3.10, 3.11 or 3.12 for the optional Python-based processors, which it describes as a beta feature. Confirm which Python processors your flows use, and confirm the runtime against the release you are targeting before you set a date.

Teams that already run NiFi and have a small flow estate with few custom components often do the migration in-house, using a staging copy to test each flow before cutover. Large flow counts, custom processors that need rebuilding, or a strict cutover window usually push teams toward a partner. Whichever route you choose, start with a complete inventory of flows, controller services and custom components, because that inventory determines the effort and the quote.

What the work includes

The migration runs as seven work streams, from inventorying what 1.x currently does through to a tested cutover and handover. Each stream produces named outputs that the next one depends on.

Inventory of running 1.x flows

We catalog every running flow, its processors, controller services, parameter contexts and variable references, and any custom NARs it depends on. The output is an inventory and gap register that lists each item that will not carry over unchanged to NiFi 2. Confirm early whether any flows rely on variable registry scope, since NiFi 2 removes it.

Upgrade to 1.27.0 and verification

The migration guidance requires moving to NiFi 1.27.0 before 2.0.0, so the 1.x environment is first upgraded to that stepping stone. Each flow is checked for behavior after the upgrade and before any change to the flow format. Ask the team to record the baseline results so later comparisons have a reference point.

Flow configuration conversion to flow.json.gz

The flow configuration file changes from flow.xml.gz to flow.json.gz as part of the move to NiFi 2. We run the conversion on a copy, review the converted file against the inventory, and keep the original for rollback. Confirm that backups of both files are stored separately before the conversion starts.

Component and expression language remediation

We replace or rename components that NiFi 2 changes, including the distributed map cache components, and rewrite expression language that depends on the removed variable registry scope to use the ENVIRONMENT scope. Each change is logged in the gap register with its owner and test status. Custom processors that call removed APIs are flagged here and handled under the custom processor work.

Java 21 and Python runtime setup

NiFi 2 requires Java 21 as its minimum runtime, so hosts, JVM options and any custom NAR build pipelines are updated first. If you need Python-based processors, the administration guide lists Python 3.10, 3.11 or 3.12 and describes this feature as beta. Decide whether to enable it only after confirming that your required processors work under beta support.

Parallel-run testing and cutover

Migrated flows run alongside the 1.x originals against the same or replayed inputs, and outputs are compared at each destination. Cutover proceeds only when the comparison results are accepted, and a rollback plan keeps the 1.x flows and configuration files available until the sign-off period ends. Agree the comparison criteria and the rollback trigger with the flow owners before the parallel run begins.

Deliverables handed over at close

The engagement ends with an inventory and gap register, a migrated and tested flow set, updated runbooks for operations and rollback, and a handover of known residual risks. Residual risks include any component behavior that could not be verified in the test environment. Ask for the residual risk list to name an owner and a review date for each item.

What drives cost and duration

This table lists the drivers that change NiFi 1.x to 2.x migration effort and how to keep each one down, and the written scope sets the hours, fees and timelines.

Driver	Why it moves effort	How to keep it down
Number of flows and process groups	Each active flow needs inventory, mapping to 2.x components and test cases, so effort grows with the count.	Retire unused or duplicate flows before migration and move process groups in planned batches.
Variables and parameter contexts	The variable registry expression language scope was removed in 2.0, so each variable reference must move to a parameter context or the ENVIRONMENT scope.	Standardize parameter contexts across environments and inventory every variable reference before cutover.
Custom processors and NARs	Each NAR must build against NiFi 2 and run on Java 21, and its own dependencies may need updating.	Count and age NARs early, retire those without an owner, and plan rebuilds for the rest.
Distributed map cache and renamed components	Renamed components break flows and scripts that reference the old component types, and the breaks surface only in testing.	List every renamed component in use, map each to its 2.x replacement, and test those flows first.
Authentication changes	Moving users, groups and policies to a new provider adds configuration work and access testing for each role. Kerberos is deprecated.	Choose the target provider early and confirm how groups map to policies before the cluster is cut over.
Cluster topology	Node count, leader election method and load-balanced connections determine how cutover and rollback are sequenced.	Settle the target topology before migration and rehearse cutover and rollback on a non-production cluster.
Number of environments	Each environment repeats flow testing, parameter value checks and configuration review.	Promote one tested flow definition and vary only the environment-specific parameter contexts.
Data volume in repositories	Large flowfile, content and provenance repositories extend migration runs and the storage needed for backups and rollback.	Measure repository sizes, review provenance retention and remove data that no longer needs to be kept before cutover.
Compliance evidence needs	Assessors may ask for change records, test results and access reviews for the migrated flows, which adds collection work.	Agree the evidence list in the written scope. Independent auditors make the assessment, and DeployOpen prepares the material.

Migration phases and what each produces

The sequence moves a NiFi 1.x estate to NiFi 2 through a complete inventory, a validated 1.27.0 baseline, converted configuration, tested flows and a controlled cutover, with each phase closing on a written output.

Inventory flows, components and cluster layout
Your team or DeployOpen's engineers export every flow, process group, controller service, custom NAR, authentication setting and cluster topology from the running 1.x estate. The output is an inventory register listing each item, its owner and its dependencies. The step is done when flow owners review and sign off the register.
Analyze gaps against NiFi 2 migration guidance
Each inventory entry is checked against the NiFi migration guidance, covering the removal of the variable registry expression language scope, the renamed distributed map cache components, the move from flow.xml.gz to flow.json.gz, and any custom NAR that depends on changed APIs. The output is a gap list with an owner and a remediation route per item, and the step closes when every item is marked unchanged, changed or blocked.
Upgrade to 1.27.0 and validate flows
The 1.x estate is staged to NiFi 1.27.0, the version the migration guidance requires before moving to 2.0.0. Teams run representative flows and compare throughput, error behavior and provenance against the pre-upgrade baseline, recording every difference. The output is a validated 1.27.0 baseline, which is the only accepted starting point for conversion, and flow owners confirm it in writing.
Convert configuration to flow.json.gz
The flow definition is converted from flow.xml.gz to flow.json.gz and checked against the gap list. Variable references are moved to the ENVIRONMENT scope, renamed cache components are updated, and authentication and cluster settings are carried over and reviewed. The output is converted configuration held in version control, and the step is done when a test node loads it without manual edits.
Prepare Java 21 and remediate test flows
The non-production cluster is built on Java 21, the minimum runtime in the NiFi administration guide. Optional Python processors are tested against Python 3.10, 3.11 or 3.12, and the team notes that the guide describes Python-based processors as a beta feature. Remediated flows and custom NARs run against test data and are compared with the baseline. The output is test evidence: run logs, result comparisons and a list of accepted differences.
Run a parallel cutover from a runbook
Production flows run on 1.x and NiFi 2 in parallel for an agreed period, and outputs are compared at each destination system before any source is switched. The cutover runbook sets out the order of changes, the go and no-go checks, the people on call and the conditions that trigger rollback. The output is the cutover runbook and a rollback record signed at the decision point.
Stabilize NiFi 2 after cutover
After cutover, the operations team watches error rates, back pressure, provenance and repository growth, closes open items from the gap list, and retires the 1.x nodes once the rollback window in the runbook has closed. The step is done when the operations team accepts the stabilized state in writing and the rollback record is filed with the migration evidence.

What teams keep in-house and what they outsource

The table compares, for each migration part, what teams usually run themselves and what a partner typically supports, along with the skills each part requires.

Migration part	Usually in-house	Usually partner-supported	Skills required
Inventory of flows and components	Listing every process group, controller service and custom NAR, with owners named for each	Scripted extraction of flow.xml.gz contents and a first-pass list of components that need attention	Flow owners who know business purpose; NiFi administrators who can read the flow configuration
Gap analysis against NiFi 2	Deciding which flows matter, which can be retired and which need a business decision on behavior change	Mapping removed variable registry scope, renamed distributed map cache components and deprecated items to affected flows	NiFi administrators; flow owners for sign-off on behavior
Staged upgrade to 1.27.0 then 2.x	Scheduling change windows, approving rollback criteria and running the production cluster	Preparing staging environments, performing the staged steps and documenting each version transition	NiFi administrators; operations for change control and rollback
Flow remediation and flow.json.gz conversion	Reviewing every changed flow and confirming it still matches the intended data path	Converting configuration, updating renamed components and producing a change log per process group	Flow owners for review; NiFi administrators for conversion checks
Custom processor porting and rebuilds	Owning the source code, approving API changes and deciding which processors are retired	Porting processors to the NiFi 2 API, rebuilding NARs on Java 21 and fixing compile or runtime issues (detailed in the custom processor topic)	Java developers familiar with the NiFi extension API; build engineers for NAR packaging
Test design and cutover testing	Defining acceptance criteria, sample data sets and sign-off for each flow	Building test harnesses, replaying representative data and comparing outputs between 1.x and 2.x	QA engineers; flow owners for expected results; NiFi administrators for environment setup
Cutover execution	Holding the go or no-go decision, communicating with data consumers and owning the production change	Preparing runbooks, rehearsing the cutover and supporting the team during the change	Operations leads; NiFi administrators; release managers
Post-cutover monitoring and handover	Owning alerts, on-call rotation and continued flow changes after the engagement ends	Setting up dashboards, reviewing early errors and training staff on the migrated flows	Operations engineers; flow owners who keep working knowledge of each flow

What usually goes wrong

Most NiFi 1.x to 2.x migration problems trace back to a skipped step in the upgrade path or an assumption about what carried over. Check each item below against your own flows before you schedule cutover.

Skipping the 1.27.0 stepping stone

The migration guidance requires upgrading to NiFi 1.27.0 before moving to 2.0.0, and changing the flow configuration file from flow.xml.gz to flow.json.gz. Teams that jump directly to 2.x, or copy the old file forward, risk flows that load incorrectly. Confirm that every environment, including staging, passed through 1.27.0 and compare the converted flow.json.gz with the original.

Flows that reference removed or renamed items

NiFi 2.0 removed the variable registry expression language scope, replacing it with an ENVIRONMENT scope, and renamed the distributed map cache components. Search exported flows, parameter contexts and templates for the old scope and cache names. Review templates against the migration guidance instead of assuming they behave the same in 2.x.

Custom NARs built for Java 8 or 11

The administration guide lists Java 21 as the minimum runtime for NiFi 2. Custom NARs compiled for Java 8 or 11, or depending on libraries that do not run on Java 21, will fail at deployment or on first use. Rebuild each NAR on Java 21 against the NiFi 2 APIs, then confirm in a non-production cluster that its processors start and run.

Python processors treated as production-ready

The administration guide describes the optional Python-based processors as a beta feature and lists Python 3.10, 3.11 or 3.12 as supported versions. List every Python processor in use and pin a supported interpreter version. For each one, decide whether a Java component can replace it or whether the beta risk is accepted and documented.

Deprecated Kerberos authentication carried forward

The administration guide marks the Kerberos provider as deprecated and lists single-user, LDAP, OpenID Connect, SAML 2.0 and X.509 client certificate authentication as supported options. Identify which 1.x clusters depend on Kerberos login, select a supported provider with your identity team, and test group mappings for each user role before cutover.

Cutover without a tested rollback path

Cutover needs a restore point for the 1.x flow configuration and a rollback path that has been rehearsed and written down. Back up the flow configuration, parameter contexts and repositories, then restore them on a separate host to confirm the backups work. Name the person who makes the rollback decision and the window in which rollback remains possible.

How the work changes by starting point

A standalone instance has the smallest set of moving parts, so the work centers on the flow definitions and a single upgrade path through 1.27.0.

Confirm the exact 1.x version, and whether it is at 1.28.x, the last minor release of the 1 series.
Plan the stop at 1.27.0 before 2.0.0, then convert the flow configuration from flow.xml.gz to flow.json.gz.
Inventory every use of the variable registry, since that expression language scope was removed in 2.0 and replaced by ENVIRONMENT scope.
Rehearse the migration on a copy of the flows in a non-production NiFi 2 instance.
Confirm which authentication provider is configured and who holds the administrator credentials.

What you need to provide

The fastest way to scope a NiFi 1.x to 2.x migration is to give us read access to the running 1.x instances and their flow configuration files, along with a current copy of the flow definitions. We also need to know where parameter contexts, variable registry references and expression language are used across the flows, because the variable registry scope was removed in NiFi 2.0 and replaced by an ENVIRONMENT scope. A written inventory of those dependencies lets us size the rewrite before any work starts.

Provide an inventory of every custom NAR in the estate, with its source location, the owning team and whether the source is still maintained. Each custom processor or controller service must be rebuilt against NiFi 2 and tested, so unknown or unavailable source code is one of the largest scope risks. If authentication changes during the move, also send the identity provider details, such as the OpenID Connect discovery endpoint, SAML metadata or LDAP directory settings, and confirm who can approve access changes.

Make a non-production environment available with representative data volumes and the same external systems the flows call. Confirm that Java 21 can be installed on the target hosts, and confirm whether optional Python-based processors are in use, since those require Python 3.10, 3.11 or 3.12 and are described as a beta feature. Name an owner for each flow who can confirm expected outputs during testing, and get business sign-off on cutover windows in advance.

Decide how long the 1.x estate will run in parallel with NiFi 2 and who owns that decision. This choice drives how much effort goes into keeping both environments in step and how long old credentials and endpoints stay active. DeployOpen does not need production credentials beyond what the agreed scope requires, so access for production work is limited to the named tasks in the written scope and removed when the work ends.

Questions buyers ask before a migration

Why must we stop at NiFi 1.27.0 before moving to 2.x?

The NiFi migration guidance requires upgrading to NiFi 1.27.0 before moving to 2.0.0, and it also requires changing the flow configuration file from flow.xml.gz to flow.json.gz. The downloads page states that NiFi 1.28 is the last minor release of the version 1 series and lists an end-of-support date of December 8, 2024 for 1.28.1. The project explains that 1.x cannot be maintained further because it depends on Jetty 9.4, Spring Framework 5.3 and AngularJS 1.8, which cannot be upgraded in that line. Plan the 1.27.0 step as the first phase of the project, and check the current 2.x minor release on the downloads page before you set a cutover date.

Can our own team migrate NiFi 1.x to 2.x without a partner?

Yes, if you have NiFi operators who already run your flows and a small estate with few custom components. Use the official migration guidance as the baseline and work through each change it lists, including the 1.27.0 step, the flow file format change and the removed or renamed components. Teams with a large estate, many custom processors, or no one who has upgraded a NiFi cluster before usually find the testing and cutover work heavier than expected. Ask a partner for a written review of your flows and components if you are unsure which path fits.

What drives migration cost and how do we get a fixed-scope quote?

Cost follows the written scope, so the main drivers are the number of flows, the number and complexity of custom processors and controller services, the templates and variables that need rework, the size of your clusters, and the testing you require before cutover. Most engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts. To request a quote, provide a flow inventory, a list of custom NARs, your cluster and security setup, your identity provider, and the environments where flows run. Ask the provider to separate one-time migration work from any recurring support.

Do custom processors and NARs need changes for Java 21 and NiFi 2?

Expect to review them, and plan to rebuild and retest each one against NiFi 2. The NiFi administration guide lists Java 21 as the minimum runtime, so any component that depends on an older Java runtime or on APIs changed in NiFi 2 needs attention. Python-based processors are an optional feature that the administration guide describes as beta, and the guide lists Python 3.10, 3.11 or 3.12 as supported versions. Build an inventory of every NAR, confirm each against the migration guidance, and test each component in a staging cluster before cutover.

Do templates and variables need rework, and should we replace NiFi Registry now?

Yes, plan time for both. NiFi 2.0 removed the variable registry expression language scope and replaced it with an ENVIRONMENT scope, and it renamed the distributed map cache components, so flows that rely on those features need updates. Confirm each template against the migration guidance and test them in staging. On NiFi Registry, the downloads page notes that it is deprecated and planned for removal in NiFi 3.0. Many teams move flow versioning to a Git-based Flow Registry Client, which NiFi 2 includes for GitHub and GitLab, while they complete the migration.

What support do we get after cutover to NiFi 2?

Support after cutover depends on the agreement you sign, and the migration itself does not include ongoing care unless the written scope says so. Ongoing support, upgrades and continuous testing are available as retainers from DeployOpen. A typical post-cutover agreement covers following minor releases, checking deprecated components, rolling cluster upgrades and rebuilding custom processors when needed. Ask the provider to list in writing what the agreement covers, who responds to incidents, and what you must provide, such as access to staging and change windows.

How DeployOpen runs NiFi migrations

DeployOpen plans, builds, tests and documents Apache NiFi 1.x to 2.x migrations inside your own infrastructure, with the written scope agreed before work starts. The work follows the project's migration guidance, from the NiFi 1.27.0 stepping stone through to cutover on NiFi 2.

How we scope

The written scope lists every flow, processor and controller service in the 1.x estate to be migrated.
We inventory removed variable registry references and renamed distributed map cache components before estimating any work.
Engagements are fixed-scope, with deliverables and timelines agreed in writing before any migration work starts.

How we build

We move flows through NiFi 1.27.0 first, then convert flow.xml.gz to flow.json.gz for the 2.x target.
Runtime is moved to Java 21, and any Python processors are tested against 3.10, 3.11 or 3.12.
Security settings are reviewed as part of the build, including identity options and the deprecated Kerberos provider.

How we hand over

Flows are committed to a Git repository through Flow Registry Clients, with migration history recorded.
Test results, cutover steps and rollback notes are documented for your operations team.
Your team receives the runbook and a walkthrough of the 2.x configuration before cutover.

How we support

Ongoing support, upgrades and continuous testing are available as a retainer after cutover.
Post-cutover checks cover following NiFi 2 minor releases and reviewing deprecated components as they change.
Your team can plan for NiFi Registry deprecation, which is slated for removal in NiFi 3.0.

Apache NiFi 1.x to 2.x migration