Accepting webhooks in Apache NiFi with ListenHTTP and HandleHttpRequest
When a source system pushes data over HTTP, NiFi has to accept each request, keep the payload intact and hand it to downstream flow logic without losing it. NiFi supports two approaches for this. ListenHTTP is an ingest processor: it starts an HTTP server and turns each accepted request into a FlowFile. HandleHttpRequest paired with HandleHttpResponse builds a request and response web service, where the flow receives the request, processes it and returns a reply to the caller. In both cases, each request becomes one FlowFile, and when a multipart request arrives, each part becomes its own FlowFile. Choosing between them starts with a simple question: does the sender need a response body that depends on your flow, or does it only need a successful acknowledgement?
ListenHTTP is the simpler choice for webhooks that fire and forget. By default it listens at a contentListener path on the configured hostname and port, and it accepts only HEAD and POST requests, so the sender's configuration must match that method. Other methods receive a 405 response. The processor also serves a GET on a /healthcheck path, which gives load balancers and monitoring a target without sending data through the flow. Confirm the exact path, port and hostname behavior against the ListenHTTP component documentation for the NiFi version you run before you publish the endpoint URL to a partner.
HandleHttpRequest and HandleHttpResponse suit cases where the caller needs a status code or body that reflects processing, such as a validation result. Each request is paired with a response that must be routed back through HandleHttpResponse, so the flow design has to guarantee that every path, including error paths, reaches a response processor. Plan for this during design, because a request that never receives a response will leave the caller waiting. Review the HandleHttpRequest component documentation and the matching response documentation before you commit to this pattern, and test both the success and failure paths with a real sender.
Choosing ListenHTTP or HandleHttpRequest for the endpoint
ListenHTTP suits sources that send data and do not need a meaningful response body. A webhook provider, a device, or an upstream system posts a payload, NiFi accepts it as a FlowFile, and the sender only needs a success status. The processor starts an HTTP server at ` by default, so the sender's URL path is fixed unless you change the Base Path setting. Use this pattern when the endpoint's only job is to receive events and hand them to downstream processors for routing, parsing, or delivery.
- Fixed receiving path: Senders post to
/contentListenerunless the Base Path is changed, so record the exact URL in the provider's configuration. - Narrow method set: The processor accepts only HEAD and POST. Any sender that relies on PUT, DELETE, or OPTIONS will receive a 405 response.
- Built-in health path: A GET request to
/healthcheckis served by the processor, which gives load balancers and monitors a target without a separate flow. - No custom response logic: You cannot shape the response body per request, so this pattern fits fire-and-forget ingestion.
- Simple flow shape: One processor feeds the rest of the flow, which keeps the number of components to review small.
Before go-live, send a test POST from the actual sender, confirm the path it uses, and verify that the sender's retry behavior matches the success status NiFi returns.
Use HandleHttpRequest with HandleHttpResponse when the caller needs a reply that depends on what the flow does with the request. Examples include an API that returns a computed result, a lookup service that answers queries, or an endpoint that must return a custom status and body after validation. The two processors are used as a pair: the request processor creates one FlowFile per request or per part of a multipart request, the flow processes it, and the response processor sends the answer back to the caller using the connection that the request carried. Because the response is part of the flow, the pair needs careful design for timeouts and error paths.
- Caller-visible responses: The response processor sets the status code and body the caller receives, which ListenHTTP does not allow.
- Multipart handling: Each part of a multipart request becomes its own FlowFile, so downstream logic must handle several items per request.
- Matched pairs: Requests and responses must be wired together, so an unmatched or failed branch can leave callers waiting.
- Broader method support: The request processor is designed for web service patterns, so confirm the methods your clients send against its current documentation.
- Greater design surface: Validation, error responses, and timeouts become part of the flow you must test and operate.
Before choosing this pattern, confirm which status codes and response bodies the caller expects, and test error paths as carefully as success paths.
The method and path behavior of ListenHTTP is the most common reason integrations fail after deployment. The processor accepts HEAD and POST for data, returns 405 for GET, PUT, DELETE, OPTIONS, and TRACE on the data path, and serves GET on /healthcheck. Senders that use a different verb, or that probe the root path with GET, will see errors that look like a broken endpoint even when the flow is healthy. Document these rules in the integration runbook so that application owners know what the endpoint accepts before they write code against it.
- HEAD and POST only: Confirm that the sender uses POST for payloads, since other verbs will not be accepted on the data path.
- 405 responses: Expect a 405 for unsupported methods, and treat it as a configuration mismatch rather than a NiFi fault.
- Health check path: Point load balancer probes at
/healthcheckwith GET, not at the data path. - Path matching: Confirm the full path, including any Base Path change, in every sender configuration.
- Test matrix: Send one request for each method and path the sender may use, and record the status codes.
Before sign-off, run the test matrix against the deployed hostname and port, including the path a load balancer uses.
Most teams decide faster by listing what each sender needs from the endpoint. Ask whether the sender reads a response body, whether it expects a specific status code for a validation failure, whether it sends multipart content, and whether it uses methods beyond POST. If the answers are all no, ListenHTTP is usually the simpler choice. If the sender needs a computed reply or a custom error that drives its retry behavior, the paired request and response processors are the better fit. Mixed cases, such as a webhook that needs a fast acknowledgement plus a later callback, may be better handled by one ListenHTTP endpoint feeding a separate outbound flow.
- Response requirements: Ask whether the sender reads anything beyond the status code.
- Validation feedback: Determine whether a rejected payload must return a specific code that the sender uses to retry or stop.
- Payload shape: Note whether multipart uploads or large bodies are common.
- Method inventory: List every verb the sender uses, not only the one in its documentation.
- Callback patterns: Separate the acknowledgement from later processing where the sender supports it.
Before choosing, write the sender's requirements in one table and confirm each row with the sender's owner, since provider documentation may differ from actual behavior.
What you need before the first webhook arrives
Most delays on an HTTP ingestion flow come from missing decisions, not from NiFi itself. The cards below cover the runtime, the network path, the security configuration and the certificates that have to be settled before the listener is opened to a sender.
A supported NiFi 2 release
Start from a NiFi 2 release rather than 1.x, because the 1.x line cannot be maintained further and the migration path runs through 1.27.0. Pin the exact version in your deployment records so the ListenHTTP and HandleHttpRequest behavior you test is the behavior you run. Confirm the release notes for any processor changes that affect HTTP listeners before you upgrade an existing cluster.
Java 21 runtime
NiFi's administration guide lists Java 21 as the minimum runtime, so every node in the cluster needs it installed and set as the runtime before NiFi starts. Mixed runtimes across nodes are a common source of confusing listener failures. If you plan to use the optional Python-based processors, check the supported Python versions separately, since that feature is described as beta.
Reachable hostname and port
ListenHTTP listens on a hostname and port that the sender must resolve and reach, so decide both before the flow is built. Check firewall rules between the sender's network and every node that will listen, and plan how a load balancer or DNS name will point at the cluster. In a cluster, confirm whether the listener runs on every node or only on the primary node, because that determines which addresses the sender must target.
Security configuration decision
Without a security configuration, NiFi binds to 127.0.0.1, so a sender on another host cannot reach the UI or the listener. Choose the authentication provider for administrators (single-user, LDAP, OpenID Connect, SAML 2.0 or X.509 client certificates) and the network binding before the first test. Avoid opening a listener on a public interface before the endpoint has TLS and an access path that matches your policy.
Certificate chain for TLS
Gather the server certificate, its private key and the full intermediate chain from your certificate authority before configuring the listener, since a missing intermediate is a frequent cause of failed sender connections. Confirm which certificate authority your senders trust, and whether they need to present client certificates. Record the expiry date and assign an owner for renewal, because an expired certificate stops every webhook without changing the flow.
Sender retry and payload behavior
Before the first webhook arrives, confirm with the sending vendor how it retries failed deliveries, how long it waits between attempts, and how many attempts it makes. Also confirm the maximum payload size, the content types it sends, whether it signs requests and whether it expects a specific response code. These answers determine your listener limits, your deduplication approach and whether NiFi can keep up during a burst.
Configuring the listener, health check and response path
This sequence takes a NiFi flow from an empty canvas to a working HTTP entry point, confirming the listener, the health check and the request and response pairing before any production sender is connected. Each step ends with a check you can repeat after every change.
Set the listener port and base path
Add a ListenHTTP processor and set the Listening Port and Base Path properties, which default to a contentListener path on the port you choose. Confirm the hostname and port are reachable from the network segment where senders sit, and that the port is not already used by another NiFi component or host service. The step is done when the processor starts without a binding error in the bulletin area and the NiFi logs show the HTTP server listening on the expected address. Your network team or platform owner usually confirms firewall rules, and the flow engineer owns the processor settings.
Confirm the health check with a GET request
From a host that can reach the listener, send a GET request to the health check path served by ListenHTTP and record the status code. ListenHTTP accepts GET only on its health check path, so this confirms the server is up without generating a FlowFile. Run the same request against the content path and confirm it returns 405, which shows the method restriction is active. Use this check as the basis for any load balancer or monitoring probe you add later.
Agree the accepted methods with the sender
Before the sending system is configured, write down the methods it will use. ListenHTTP accepts only HEAD and POST for data delivery, and returns 405 for GET, PUT, DELETE, OPTIONS and TRACE on the content path. Ask the sender's owner to confirm they can send POST with the payload in the body, and whether they need HEAD for validation probes. If a sender requires PUT or a different path, the design needs a different entry point, which is a decision to make now rather than after go-live.
Route the FlowFiles to downstream processors
Connect the success relationship of ListenHTTP to the first processor that validates or parses the body, such as a record reader or a routing processor that checks headers. Set a sensible back-pressure threshold on the connection so a slow downstream step does not fill the repository without warning. Send a test POST with a small JSON body, then inspect the queued FlowFile in the connection. The check is complete when the content matches what the test client sent and the attributes include the HTTP request details you expect for routing.
Pair HandleHttpRequest with HandleHttpResponse
For a request and response service, add HandleHttpRequest and set its Listening Port and Allowed Paths. Then add HandleHttpResponse and set its HTTP Status Code and HTTP Context Map property so it uses the same identifier that HandleHttpRequest writes to each FlowFile. Each request produces one FlowFile, or one FlowFile per part for multipart requests, and the response processor must receive that FlowFile to close the client connection. Send a POST from a test client and confirm the client receives the status and body you configured, and that the request FlowFile carries the matching context identifier.
Test failure paths and the response timing
Send requests that should fail, such as a wrong path, an unsupported method and an oversized or malformed body, and record what the client receives. For request and response services, confirm that a branch that does not reach HandleHttpResponse leaves the client waiting, so every failure relationship needs a response path. Check the bulletins and the failure queue after each test. The step is done when every expected failure returns a deliberate status to the client and nothing sits in a queue without an owner.
Record the configuration for the change process
Save the processor settings, the port, the base path, the allowed methods and the test results in your change record or flow documentation. If the flow is versioned in a Flow Registry Client, commit it with a message that names the listener and the response mapping. Ask the platform owner to confirm the expected listener address and whether a gateway will sit in front of it, since that decision affects TLS and rate limiting in the next sections. The step is done when another engineer can reproduce the listener and the test results from the record alone.
How HTTP headers, paths and bodies map to FlowFile attributes and content
This table shows where each part of an incoming HTTP request lands in NiFi, from the body and path to headers and multipart parts, and what the response processor sends back. Read it as a checklist before you write routing logic, because attribute names vary by processor and version. Confirm each name in the processor documentation for your NiFi 2 release before you depend on it.
| Request element | Where it lands in NiFi | What to confirm before building on it |
|---|---|---|
| Request body | FlowFile content, unchanged by the receiving processor | Confirm how ListenHTTP and HandleHttpRequest handle bodies of different content types in your version |
| Request path | Matched against the processor path setting, with the remainder exposed as an attribute | Confirm the exact attribute name and whether the base path is included |
| HTTP headers | FlowFile attributes, one per header, usually with the header name carried through | Confirm which header names are normalized to lowercase or otherwise altered |
| Query string parameters | Available as request attributes, not merged into content | Confirm the attribute naming pattern for parameters with repeated names |
| Multipart form parts | One FlowFile per part on HandleHttpRequest, with part headers as attributes | Confirm how file names and form field names are exposed for each part |
| HTTP method | Request attribute on HandleHttpRequest, while ListenHTTP accepts only HEAD and POST | Confirm which methods reach your flow before adding method-based routing |
| Response status and body | Set by HandleHttpResponse from FlowFile attributes and content in the response path | Confirm the attribute that carries the request identifier so responses return to the right client |
| Health check response | Served on the /healthcheck path by ListenHTTP, outside the data FlowFile path | Confirm the response body and status before using it in a load balancer probe |
Why rate limiting changed in NiFi 2.4 and what to put in its place
NiFi 2.4.0 removed the Max Data to Receive per Second property from ListenHTTP, so a processor upgraded to 2.4 or later accepts requests without a built-in rate limit. Teams that relied on that property to protect downstream systems now have an open listener unless they add a control elsewhere. Before the upgrade, check each ListenHTTP instance for a value set in the old property, record what it was limiting, and decide where that protection will live afterward. The listener itself no longer throttles, so any flow that depended on it needs a replacement before the upgrade reaches production.
Three options are worth evaluating. Flow-level throttling uses NiFi's own scheduling, such as connection back-pressure thresholds and run schedules on downstream processors, to slow processing once queues fill; it protects NiFi and its sinks but still accepts the incoming traffic. A gateway in front of the listener, such as an API gateway or reverse proxy that your organization already operates, can enforce request limits per client, return clear status codes and absorb abusive traffic before it reaches NiFi. Sender-side limits apply when the webhook source can retry with backoff or batch its events. The right choice depends on how the source behaves, including whether it retries on rejection, how many senders exist, and whether your team's gateway standard already covers this traffic.
Confirm three things with the team that owns the gateway and the source before choosing. Ask whether the source honors HTTP 429 or 503 responses and how it retries. Ask whether the gateway already fronts this endpoint and can apply limits per source address or credential. Then test the chosen design under a burst that matches your worst observed sender load, and verify that rejected requests are logged so that data loss is visible rather than silent. Document the decision in the flow's change record, because a future upgrade could again change what the processor enforces.
Securing the HTTP endpoint with TLS, authentication and a gateway
A ListenHTTP or HandleHttpRequest endpoint is only as safe as the controls around it. The cards below cover transport encryption, network reach, sender authentication and a gateway in front of NiFi, and they show where each control sits so the team does not assume one layer covers another.
TLS on the listener
Configure the processor's SSL Context Service so the endpoint accepts HTTPS rather than plain HTTP, and use a certificate issued by the CA your senders already trust. Confirm the processor's TLS and client-authentication properties in the component documentation for your NiFi 2 version. Plan certificate renewal before expiry, because a lapsed certificate stops every sender at once. Check that plain HTTP is disabled on any port reachable from outside the network.
Restricting network reach to the port
Limit which source addresses can reach the listening port using host firewalls, security groups or network policies on Kubernetes. Only the systems that send webhooks should have a route to the port, and the health check path should be reachable only by your load balancer or monitoring. Confirm that the NiFi host does not expose the same port on additional interfaces, since the listener binds to the address you configure.
Sender authentication
Decide how each sender proves its identity before data enters the flow. Options include a shared secret sent in a header, validated by a flow step, or client certificates validated at the TLS layer when the processor supports mutual TLS. Record the chosen method per sender, define how secrets are rotated, and store them as sensitive parameters. Confirm the exact options in your processor version rather than assuming a built-in check exists.
Gateway in front of NiFi
Place an API gateway in front of the listener to handle authentication, request throttling and centralized access logs. The gateway terminates or forwards TLS, rejects unauthenticated calls before they reach NiFi, and absorbs bursts so the flow is not overwhelmed. Because ListenHTTP no longer carries a per-second rate limit after NiFi 2.4.0, throttling at the gateway is the place to set it.
NiFi UI login is not webhook auth
NiFi's user authentication, such as LDAP, OpenID Connect or client certificates, protects the NiFi UI and API. It does not authenticate incoming webhook calls to a listener processor. Treat every listener as an unauthenticated entry point unless a gateway or processor-level check enforces identity. Confirm this behavior in your environment with a test call from an unapproved client before go-live.
Common webhook failures and how to fix them
This table pairs each common failure with its likely cause and the fix to try first, covering both ListenHTTP and HandleHttpRequest setups. Read it by matching the symptom your sender or operators report to a row, then confirming the cause in your own flow and network before changing settings.
| Symptom | Likely cause | Fix |
|---|---|---|
| Sender gets 405 on GET or PUT | ListenHTTP accepts only HEAD and POST on its listener path, so other methods return 405 by design. | Configure the sender to POST to the contentListener path. Confirm the exact method the sender uses before changing the flow. |
| Health check fails | The health check is served as GET on the /healthcheck path, so a probe using another method or path gets no healthy response. | Point the load balancer or monitor at a GET request to /healthcheck on the listener port, and confirm the path with a test request. |
| Sender cannot reach the instance | With no security configuration, NiFi binds to 127.0.0.1, so only local clients can reach the UI and listeners. | Set the bind address and listener host for the network interface senders use, then open the port in host and network firewalls and retest from outside. |
| Duplicate deliveries after sender retries | Senders retry when they do not receive a timely success response, and NiFi may already have written the FlowFile downstream. | Respond only after the payload is persisted, include an idempotency key or event ID, and deduplicate in a downstream step keyed on that ID. |
| Unbounded inbound volume after upgrading to 2.4 or later | NiFi 2.4.0 removed the ListenHTTP Max Data to Receive per Second property, so upgraded processors have no rate limit by default. | Place a gateway or reverse proxy in front for rate limiting, and size back-pressure thresholds and connection limits in the flow to protect downstream systems. |
| TLS handshake errors | The sender trusts a different certificate chain than the one presented, the protocol or cipher does not match, or the client certificate is missing when required. | Check the certificate chain and hostname against the sender's trust store, align TLS versions, and confirm the expected client authentication setup with the sender's documentation. |
Questions teams ask before building webhook ingestion in NiFi
How much effort does a single webhook flow take to build?
The effort depends less on the NiFi work itself than on what the sender needs from the endpoint. A basic flow that accepts POST requests with ListenHTTP, checks the payload, and writes FlowFiles to a queue or store is a small build. The effort grows with the number of things that must be settled first:
- Authentication: how the sender proves identity, and where the shared secret or certificate is kept.
- Payload handling: which content types arrive, how malformed bodies are rejected, and whether records need a reader and schema.
- Response behavior: what status code the sender should see when the flow accepts, rejects or cannot keep up.
- Testing: replaying real payloads from the sender's side, including failure cases.
Ask any partner for a written scope that lists these items, because each one changes the build and the test plan.
Can an internal team build webhook ingestion in NiFi alone?
Usually yes, if the team already runs NiFi and has someone who can administer its TLS, authentication and cluster settings. The processors involved are documented in the NiFi component guides, so the flow itself is within reach of a NiFi-literate engineer. Teams often need outside help in three places: agreeing the security design with the security function, testing against the sender's real delivery behavior, and running the endpoint in production with monitoring and a rate-limiting approach. Confirm internally who owns the NiFi cluster, the certificate lifecycle and the on-call path before deciding whether to build alone or with a partner.
What does the work need from the source system owner?
The sender's owner decides most of the design, so get their answers early. The questions that matter most are:
- Which HTTP methods the sender uses, and whether it expects a specific success status code.
- Whether it signs payloads, sends a bearer token or relies on mutual TLS, and who rotates those credentials.
- How long it waits for a response, and whether a slow response causes it to retry.
- Whether it guarantees delivery order, and whether it sends a unique event identifier.
- Which IP ranges it calls from, so the network team can allow them.
Without these answers, the endpoint design is guesswork.
How should retries and duplicate deliveries be handled?
Assume the sender will retry, and assume the same event may arrive more than once. Most webhook senders retry when they do not get a success response in time, so a flow that accepts a request and then fails downstream can create duplicates. The safer pattern is to record a unique event identifier from the payload or headers and check it before processing. The identifier store needs to survive restarts and work across cluster nodes. Confirm with the sender what identifier it provides, how long it retries, and what happens after the final retry. Those answers set the retention window for the identifier store.
Is a gateway required in front of a NiFi webhook endpoint?
A gateway is not required by NiFi, but it is often the better place for controls that do not belong in the data flow. An API gateway can terminate TLS, validate tokens, limit request rates and publish a stable public hostname, while NiFi handles parsing and routing. NiFi 2.4.0 removed the ListenHTTP Max Data to Receive per Second property, so an upgraded flow runs without that rate limit unless you configure another approach. Decide early whether rate limiting lives in the gateway, in NiFi, or both, and test what a burst does to the cluster. DeployOpen can plan that split as part of a written scope, and the Apache APISIX pillar covers gateway design separately.
What drives the security effort for a webhook endpoint?
The main drivers are the exposure of the endpoint, the sensitivity of the data and the number of parties involved. A listener reachable from the internet needs TLS, sender authentication and input validation before any data reaches downstream systems. Certificate issuance and renewal, secret storage and logging of rejected requests add recurring work. Ask for a written list of which controls sit in NiFi, which sit in the gateway and which sit in the network, and who operates each one. Compliance evidence follows from that list, but a partner prepares the controls and evidence for an independent assessor. It does not issue the assessment.
Does the work end after the flow is deployed?
No. Deployment is the start of the operating cost. Recurring work includes certificate renewal, monitoring of accepted and rejected request counts, review of the health check path, NiFi and processor upgrades, and checks after sender changes. Ask any partner which of these are included in the fixed scope and which would move to a retainer. Support and continuous testing are available as retainers, so the decision is about how much of this the internal team wants to carry.
How DeployOpen scopes and delivers NiFi webhook ingestion
DeployOpen plans, builds, documents and supports HTTP ingestion in Apache NiFi inside your own infrastructure, from the first listener design to the operating runbook. Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts.
How we scope
- Inventory every sender, endpoint path, HTTP method and payload format the written scope must cover.
- Agree in writing how TLS, authentication, health checks and any rate limiting will be handled for each listener.
- Confirm the NiFi version in use, since rate limiting behavior changed in 2.4.0 and affects the design.
How we build
- Configure ListenHTTP or HandleHttpRequest and HandleHttpResponse flows with the agreed paths, methods and response codes.
- Test each sender against the listener, including rejected methods, large bodies, multipart requests and failure responses.
- Place the listeners behind the gateway design agreed in scope and verify routing, TLS termination and client identification.
How we hand over
- Deliver the flow definitions, parameter and controller service configuration, and the list of endpoints with their expected behavior.
- Provide a runbook covering health check monitoring, certificate renewal, scaling of the listener and rollback steps.
- Document known limits, such as the absence of built-in rate limiting after upgrading processors, and the controls that compensate.
How we support
- Ongoing support, upgrades and continuous testing are available as retainers after handover.
- Retainers can cover review of the listener configuration when NiFi versions change, including the 2.x release path.
- Ask in any quote which support hours, response expectations and testing cadence are included before work starts.
