Why NiFi scripts in Jython, ECMAScript, Lua and Ruby must be replaced before upgrading
If a NiFi 1.x flow runs an ExecuteScript or InvokeScriptedProcessor component written in ECMAScript, Lua, Ruby or Jython, that code has to be rewritten before the flow can move to NiFi 2. The deprecation of these engines started in 1.x. ExecuteScript deprecated the ECMAScript (Nashorn) engine and the lua and ruby engines in NiFi 1.22.0, and the Jython engine followed in 1.24.0. The deprecation notes point to Groovy or native Python processors as the replacements in NiFi 2.0. Your flows may still run on these engines today, but they are on a path that ends with the upgrade, so the work belongs in the upgrade plan rather than after it.
The timing is tied to the 1.x line itself. The NiFi project states that 1.x cannot be maintained further because it depends on Jetty 9.4, Spring Framework 5.3 and AngularJS 1.8, which cannot be upgraded within that line. The downloads page names NiFi 1.28 as the last minor release of the version 1 series and lists an end-of-support date of December 8, 2024 for 1.28.1. The supported route is to move to NiFi 1.27.0 first, as the migration guidance requires before 2.0.0, and then change the flow configuration file from flow.xml.gz to flow.json.gz. Script rewrites can be done in 1.x where the old engine still runs, but they should be tested against the replacement language before the cutover, not discovered during it.
Scripts are an inventory task, not a single switch. Each ExecuteScript or InvokeScriptedProcessor instance carries its own code, properties and dependencies, and the same script is often copied into several process groups. Start by listing every instance across all process groups and parameter contexts, recording the engine and the module dependencies each script imports, and noting whether the logic is simple attribute handling or a long-running transformation. Confirm the exact replacement for each engine against the current NiFi documentation and your own environment, because the choice between Groovy in ExecuteGroovyScript and a native Python processor affects libraries, runtime and testing. Assign an owner to each flow before work starts.
Which script engines change, and what each change does to your flows
This table compares each scripting language used in ExecuteScript or InvokeScriptedProcessor with its deprecation point, its NiFi 2 replacement path, and the work it creates for existing flows. Use it to find the rows that match your flows, then check every script against the action column before you upgrade. It is not exhaustive, so inventory any other engines or scripted components you run.
| Language or engine | Deprecation and NiFi 2 path | Impact on existing flows | Action required |
|---|---|---|---|
| Jython (Python 2) | Deprecated in NiFi 1.24.0. Move to Groovy or the native Python processors in NiFi 2. | Jython scripts in ExecuteScript will not carry over unchanged. Python 2 syntax and Jython-only libraries need rewriting. | Confirm each script is Python 2 or Jython, then port it to Groovy or a native Python processor and test it with sample data. |
| ECMAScript (Nashorn) | Deprecated in NiFi 1.22.0. Move to Groovy or the native Python processors in NiFi 2. | JavaScript based on the Nashorn engine is a dependency on a deprecated engine. Flows built on it need a full rewrite, not a configuration change. | Inventory every JavaScript script, rewrite the logic in the chosen replacement language, and compare outputs on representative FlowFiles. |
| Lua | Deprecated in NiFi 1.22.0. Move to Groovy or the native Python processors in NiFi 2. | Lua scripts stop being a supported path, and any Lua-only helper modules must be reimplemented. | Locate Lua scripts in ExecuteScript and InvokeScriptedProcessor, document their inputs and outputs, and rebuild them. |
| Ruby | Deprecated in NiFi 1.22.0. Move to Groovy or the native Python processors in NiFi 2. | Ruby scripts and any gems they call must be replaced. Behaviour on edge cases may differ after porting. | Identify gem dependencies, port the logic, and add tests for the record and attribute cases the script handles. |
| Groovy (ExecuteGroovyScript) | The NiFi 2 replacement path named for the deprecated engines. ExecuteGroovyScript runs custom Groovy scripts. | Scripts validate with a 5 minute script validation timeout, so long or slow validation logic can fail at configuration time. | Keep scripts short, move heavy logic into a proper processor, and confirm validation completes within the timeout. |
| Native Python processors | Optional Python-based processors, described as beta in the NiFi administration guide. Require Python 3.10, 3.11 or 3.12. | Python code moves to packaged processors rather than inline scripts. Hosts need a supported Python version installed. | Confirm the Python version on every node, check the beta status against your risk policy, and test each processor before production use. |
| ExecuteScript in NiFi 2.12.0 | Labelled experimental in NiFi 2.12.0. Treat it as a transitional component, not a long-term home for logic. | Flows that depend on ExecuteScript may behave differently from a stable processor and may change in later releases. | Record the NiFi version you target, pin it, and plan to move stable logic into a custom processor or a supported processor. |
What to inventory and back up before you rewrite a script
Before any script changes, you need a complete list of where scripts live, what they depend on, and a restore path if the rewrite goes wrong. The cards below cover the inventory, the dependencies that hide inside scripts, the backups that make rollback possible, and the test environment that proves the replacement behaves the same way.
Find every script instance
Search every process group, including nested groups, disabled processors, and versions stored in your flow registry, for ExecuteScript and InvokeScriptedProcessor. Count instances per engine, since the Jython, ECMAScript, Lua and Ruby engines each need a different replacement path. Include processors stopped for months, because they often hide in archived branches and get re-imported later. Record the process group path, the processor name and its state for each hit.
Record script bodies and engine settings
Export each script body with its engine, script file or inline body, and the processor's scheduling and relationship settings. A rewrite that starts from memory of what the script does will miss edge cases, such as a branch that only fires on a malformed record. Store the exact text in version control with the flow name and the NiFi version it ran on. Mark which scripts read from or write to the flow file content, which set attributes, and which call external systems.
Map module and library dependencies
Scripts often import Python packages, Groovy jars, Ruby gems, or Lua modules that live on the NiFi host, in a module path, or in a shared directory. List every import and every path referenced in the Module Directory property, and confirm which ones the replacement will still need. Check whether any dependency is loaded from a host path that is not in your configuration management, because that file will not travel with a flow export. Note any library that a native processor cannot provide.
Trace property values and parameter references
Each script processor property can hold literal values, NiFi Expression Language, or parameter references. Export the property values and list every #{...} parameter and every variable reference that the script reads. Variable registry references need migration to parameter contexts before NiFi 2, so flag them now. Mark properties with sensitive values, because they must be re-entered or rebound in the target environment rather than copied as plain text.
Back up flow.xml.gz and flow definitions
Take a copy of flow.xml.gz and the conf directory from every node before any change, and export each affected process group as a versioned flow definition. Keep the sensitive properties key and nifi.properties alongside the backup, because a flow restored without its key cannot decrypt sensitive script properties. Store backups outside the NiFi hosts, with access restricted, and test one restore on a scratch instance. Confirm that your backup process covers the registry, since the flow definition alone does not capture the registry state.
Build a test environment with real traffic shape
Create a non-production NiFi instance on the target version and load representative sample data, including malformed records, large files, and the edge cases the scripts already handle. Run the original script and the replacement on identical input, then compare output content, attributes, and routing relationships. Use NiFi provenance to confirm that each flow file takes the same path through both versions, and record any intentional differences. Ask your team to confirm that the test data reflects production volumes and formats before sign-off.
Choosing Groovy, native Python or a custom processor for each script
Groovy is the path the NiFi project points to for scripts that were written in Jython, ECMAScript, Lua or Ruby, and it fits best when the existing logic is short, stays close to the flow and does not need its own release cycle. The script still runs inside a processor, so the conversion is mostly a language and API change: you rewrite the session calls, attribute handling and error routing in Groovy and keep the flow layout the same.
- Short, flow-bound logic: Scripts that set attributes, reshape a small payload or make a routing decision are good candidates for a direct port.
- Few external dependencies: Check which libraries the script imports. Jars that must be added to the processor's module path add packaging work that should be counted before you commit to this route.
- Inline maintenance is acceptable: The logic is owned by the flow team and changes with the flow, not with a separate software release.
- Validation timeout: ExecuteGroovyScript applies a 5 minute script validation timeout, so confirm that any startup-heavy initialization in the script does not hit it.
- Experimental status: ExecuteScript is labelled experimental in NiFi 2.12.0, so confirm the current status of each processor in the version you run before standardizing on it.
Before choosing this route, check the component documentation for ExecuteGroovyScript and test the ported script against representative FlowFiles, including malformed ones.
NiFi 2 includes native Python processors, which let teams extend flows in Python without embedding a Jython interpreter. This route suits scripts whose logic is already Python, whose owners want Python's library ecosystem, and whose behavior should be visible to flow operators as a named processor with properties. The Python-based processors are a beta feature, and the administration guide lists Python 3.10, 3.11 or 3.12 as the supported interpreters.
- Beta status: Treat the Python-based processors as a beta feature and confirm that your risk owners accept that status for production data paths.
- Interpreter version: Install Python 3.10, 3.11 or 3.12 on every node, including the hosts or images used for each cluster member, and verify the same version across them.
- Dependency isolation: Decide how third-party Python packages reach each node, since packages installed on one node but not another create inconsistent behavior.
- Testing: Build unit tests around the processor logic before cutover, because the beta status means behavior may change between releases.
- Ownership: Confirm who maintains the Python code and its packages after the migration, since this becomes an ongoing platform dependency.
Ask your DeployOpen contact or your NiFi vendor documentation to confirm the exact Python processor set and interpreter behavior in the release you plan to run.
A compiled custom processor is the right choice when a script has outgrown inline code. Signs include logic reused across several flows, a need for automated tests in a build pipeline, complex configuration exposed to operators, or performance-sensitive work that benefits from a packaged component. The effort is higher because you build, version and deploy a NAR, but the logic then has a release history, review process and test suite that inline scripts rarely get.
- Reuse across flows: When the same transformation appears in several process groups, one tested component reduces drift between copies.
- Testability: Processors can be covered by unit and integration tests that run in your build pipeline before deployment.
- Configuration model: Properties, validators and relationships become explicit, which helps operators and auditors see what the component does.
- Build and packaging work: Plan for a build toolchain, NAR packaging and a deployment path to every cluster node.
- Upgrade exposure: Custom NARs must be rebuilt and retested for NiFi 2, which adds work to the upgrade plan rather than removing it.
Before choosing this route, ask who will own the component's code, release process and compatibility testing across future NiFi 2.x releases.
Most NiFi estates end up with a mix of all three options, because scripts differ in size, reuse and ownership. A practical approach is to inventory every ExecuteScript and InvokeScriptedProcessor instance, record the language, the flow it belongs to, how many flows share the logic and whether it has tests, then assign each script to one route. The sorting question is less about language preference and more about how the logic will be maintained after the upgrade.
- Inventory first: List each script with its language, owning process group, input and output attributes, and any external calls it makes.
- Keep small logic inline: Scripts that are short, single-flow and owned by the flow team are usually cheapest as Groovy.
- Move shared logic to components: Logic reused by several flows or needing tests is usually better as a compiled processor.
- Use Python where the team already writes Python: The native route fits teams with existing Python skills and packages, subject to the beta status and version constraints.
- Test each route the same way: Run the same representative FlowFiles through the old script and the new implementation, and compare attributes, content and routing.
When the mapping is complete, record the decision for each script so the cutover plan and the sign-off record show which route was taken and why.
How to port a script and verify it before the NiFi 2 cutover
This sequence turns an inventory of legacy script engines into ported, tested and promotable processors, with a recorded rollback point for each script before the NiFi 2 cutover.
Inventory every scripted processor in the flows
Export the flows from the 1.x production instance and search for ExecuteScript and InvokeScriptedProcessor instances, noting the script engine each one selects (Jython, ECMAScript, Lua or Ruby), the script file or inline body, and the attributes and properties it reads and writes. Include scripts held in process groups that are rarely changed, and scripts referenced from shared templates or reusable flow definitions. The step is done when you have a list that maps every script to its flow, its engine and its owner, and when a second person has checked the list against the running canvas.
Classify each script by its target replacement
For each script, decide whether it moves to Groovy in ExecuteGroovyScript, to a native Python processor, to a built-in processor that already covers the logic, or to a custom processor because the logic is reused, long-running, or needs unit tests. Record the reason next to each entry. Scripts that depend on Lua or Ruby, or on engines deprecated in the 1.x line, usually need a rewrite rather than a direct translation. The step is done when every script has one target and one owner, and the list of scripts that need a custom processor is short enough for your team to build and test.
Port each script in a non-production test instance
Build the ported processor in a non-production instance, using the 1.x version for baseline comparisons and the 2.x version for the target. Replace engine-specific calls, adjust attribute and FlowFile handling, and confirm that ExecuteScript is treated as experimental in your target release before relying on it for ongoing work. Keep the original script unchanged so the two versions can be run side by side. The step is done when each ported script loads, validates and runs without errors on the test instance, and when the port is saved in version control with a commit that references the original script.
Compare outputs and provenance on sample data
Run the original and ported flows on the same representative sample set, including malformed records, empty content, large content and attributes with unusual characters. Compare output FlowFile content byte for byte where the format requires it, compare attributes and routing relationships, and compare the provenance events for the same input. Document every expected difference, such as changed timestamps or processor names in provenance. The step is done when the sample results match, or each difference has a written reason approved by the flow owner.
Check Java 21 and Python runtime prerequisites
Confirm that the target hosts or container images run Java 21, the minimum runtime listed in the NiFi administration guide, and that any native Python processor runs on Python 3.10, 3.11 or 3.12, the versions that guide lists for the optional Python-based processors. Note that the guide describes these Python processors as a beta feature, so record that status in your risk register. Verify the interpreter path and installed packages on every node, not only the one you tested on. The step is done when a health check on each node confirms the runtime versions and each Python processor starts on every node.
Promote scripts through the flow registry
Move the ported processors and their flow versions through your development, test and production environments using a flow registry client, such as the GitHub or GitLab clients in NiFi 2, rather than copying process groups by hand. Use parameter contexts for paths, endpoints and credentials so the same version runs in each environment. Confirm that the production instance is not the first place a ported script runs. The step is done when the production instance runs the same committed version that passed the sample comparison, and the version identifier is recorded against the change ticket.
Define the rollback point for each script
For each script, record the flow version and the 1.x instance or snapshot that still runs the original code, and state how long that fallback remains available before the 1.x instance is retired. Name the owner who can switch a flow back, and the signal that triggers the switch, such as a rise in failed FlowFiles in the processor's failure relationship or a mismatch in provenance counts. Keep the original script in version control so it can be restored without reconstructing it. The step is done when each script has a documented rollback point that the cutover team has reviewed and accepted.
What makes script replacement take longer
Script rewrites rarely fail on the code itself. The effort grows with how many scripts exist, how much they depend on language-specific libraries and removed NiFi features, and how many environments must receive the rewritten flows. Each card below is a factor to measure in your own flows before estimating the work.
Number of script instances
Count every ExecuteScript and InvokeScriptedProcessor instance across all process groups, including copies that were cloned into other flows and scripts stored outside the flow in files. Each instance is a unit of rewrite, review and testing, so the total drives most of the schedule. Ask for an inventory that lists the engine, the script file and the processor location for each one. Scripts that share a single file can be replaced once, while duplicated inline copies must each be found and changed.
Language-specific libraries
Jython scripts often import Python standard library modules or third-party packages, Lua and Ruby scripts depend on their own gems and rocks, and ECMAScript scripts may rely on Nashorn-specific behavior. Each library dependency must be matched to a Groovy class, a native Python processor capability or a custom processor. Check whether any script calls native code or a package that will not run in the target runtime. This is often the point where a simple port turns into a redesign.
Variable registry and removed scopes
Scripts that read variables through the variable registry, or that depend on the scope removed in NiFi 2.0, must be changed to use parameter contexts and the ENVIRONMENT scope. Find these references in each script body and in the flow properties around it. Parameter values that hold secrets need a sensitive parameter design before the script can be promoted. Treat every variable lookup as a separate change item.
Stateful logic
Scripts that keep state between FlowFiles, such as counters, caches, deduplication sets or running totals, behave differently once they are rewritten as processors or once the cluster runs on several nodes. Identify where state is stored, whether it lives in script variables, the distributed map cache or an external store, and how it survives restarts. Decide whether the state needs to be shared across nodes before choosing the replacement design.
Binary and external dependencies
Some scripts call operating system commands, load native libraries, or use JDBC drivers and certificate stores placed on the host. These dependencies must be packaged for the new runtime, which requires Java 21 and, for the optional Python processors, a supported Python version. Check each external file, its location on every node and its license. Missing dependencies usually surface only at runtime, so list them before the rewrite starts.
Test data and promoted environments
Each rewritten script needs representative input, expected output and a comparison method, and the volume of this test data sets how long validation takes. Count the environments that must receive the change, such as development, test, staging and production, and the number of nodes in each cluster. Ask whether those environments can be promoted from a version-controlled flow and whether parameter values differ between them. More environments and more distinct data formats extend both testing and change approval.
Common questions about replacing NiFi scripts before an upgrade
What does it cost to replace Jython, JavaScript, Lua and Ruby scripts in NiFi 2?
Cost depends on how many scripts exist, how much logic each one holds, and how many flows call them. The main components are discovery (finding every script and the flows that run it), rewriting, testing against real data, and handover. Discovery and testing usually take more effort than the rewrite itself when scripts have no tests or depend on undocumented behavior. One-time work covers the rewrite and test harness. Recurring cost comes from maintaining the new code, updating dependencies, and running upgrades later. Ask any provider for a quote that lists each script, its target (Groovy, a native Python processor, or a custom processor), and the test evidence it will produce.
How long does a script replacement take, and what makes it longer?
Duration is driven by the count and size of scripts, the number of flows that depend on them, and the quality of existing documentation. Scripts that only reformat attributes are quick to port. Scripts that call external services, manage state, or parse complex records take longer because each behavior must be reproduced and verified. Scripts shared across many process groups add coordination work, since a change affects every caller. Ask for a schedule that separates inventory, rewrite, test and cutover so you can see where time goes.
What must our team provide before a script replacement starts?
The team should provide flow exports (JSON flow definitions or templates) containing the script processors, the script files themselves, and the list of attributes, records and external systems each script reads or writes. Representative test data is essential, including malformed and edge-case inputs, because the replacement must match current output. Confirm the Python runtime (3.10, 3.11 or 3.12 for the native Python processors, which the administration guide describes as a beta feature) and the Java 21 runtime that NiFi 2 requires. Also confirm access to a non-production NiFi 2 instance where tests can run.
Can our internal team replace the scripts without an outside partner?
Yes, if the team has NiFi operators who know the flows, developers who can read the existing script language, and capacity to write tests. The work is manageable in-house when scripts are few, small and well documented. A partner adds value when many scripts are undocumented, when the upgrade deadline is fixed, or when the team lacks Groovy or Java experience. Whichever route you choose, agree on who owns the test evidence, because that evidence is what your change board will ask for.
Does ExecuteScript still work in NiFi 2.12?
ExecuteScript remains available in NiFi 2.12, but the NiFi components documentation labels it experimental. The project deprecated the ECMAScript (Nashorn), Lua and Ruby engines in NiFi 1.22.0 and the Jython engine in 1.24.0, and pointed users to Groovy or native Python processors. Treat ExecuteScript as a short-term bridge for simple logic, not a long-term home for production code. Plan to move scripts that must keep running to ExecuteGroovyScript or a native processor, and test any remaining ExecuteScript code on each upgrade.
Which Groovy limits should we check when moving scripts to ExecuteGroovyScript?
ExecuteGroovyScript runs custom Groovy scripts and applies a 5 minute script validation timeout, according to the NiFi components documentation. Check that every ported script validates within that window, especially scripts that load large libraries or perform network calls during validation. Review how each script reads flow file content, attributes and session state, since these APIs differ from the old engines. Confirm the exact behavior of your NiFi 2 version against the component documentation before cutover.
When should a script become a proper custom processor instead?
Convert a script into a custom processor when it is reused across many flows, has grown beyond a few dozen lines, needs its own configuration properties and validation, or must be versioned and tested like other code. Scripts that hold state, call external APIs with retries, or parse custom formats are strong candidates. A custom processor is packaged as a NAR and must follow NiFi 2 rules such as Java 21 and updated package names. Keep a script in place when it is a one-off transformation that a standard processor or a small Groovy step handles clearly.
How DeployOpen scopes and delivers a NiFi script migration
DeployOpen approaches the replacement of Jython, ECMAScript, Lua and Ruby scripts in NiFi 2 as a scoped engineering task on your own infrastructure. We start by inventorying every script that runs in your flows, then agree in writing which scripts become Groovy, native Python or proper processors before any code changes.
How we scope
- Inventory every ExecuteScript and InvokeScriptedProcessor instance and record its engine, language and upstream and downstream flow dependencies.
- Classify each script as a Groovy rewrite, a native Python processor candidate, or a case for a custom processor, with the reasons stated.
- Agree the scope, deliverables and timeline in writing before work starts, since most engagements are fixed-scope.
How we build
- Rewrite scripts in the agreed language and keep input and output attribute names and record formats unchanged so connected processors still work.
- Test each replacement against recorded sample FlowFiles from your environment, including error paths and large or malformed inputs.
- Validate scripts within NiFi 2.12.0 limits, noting that ExecuteScript is labelled experimental and ExecuteGroovyScript has a 5 minute validation timeout.
How we hand over
- Deliver the rewritten flow definitions, any custom processor source and build outputs, and a list of every changed component.
- Document the replacement mapping from each original script to its new implementation, so your team can trace decisions later.
- Walk your engineers through the test evidence and the operational notes needed to run and change the new code.
How we support
- Offer ongoing support, upgrades and continuous testing for the rewritten scripts as a retainer after handover.
- Re-test scripts when you move to later NiFi 2.x releases, so behavior changes are caught before they reach production.
- Confirm with your team which scripts depend on external libraries or credentials, since these need separate review before cutover.
