Why the real client IP in Apache APISIX depends on which headers you trust
The client IP APISIX records is only as trustworthy as the proxy chain in front of it, and that chain has to be declared, not assumed. When APISIX sits behind a cloud load balancer, a CDN or another ingress, the TCP peer it sees is that intermediate hop, not the client. Rate limits, ip-restriction rules, access logs and any WAF decision that keys on the client address all read that peer unless the gateway is told how to recover the original address. The real-ip plugin is the APISIX mechanism for doing this. It reads a header such as X-Forwarded-For or X-Real-IP and substitutes the address it contains as the client IP, but only for connections that come from addresses you have listed as trusted. Teams usually need it as soon as a layer of infrastructure they run terminates connections in front of the gateway.
The risk is the mirror image of that benefit. A request reaching APISIX directly from the internet can carry any X-Forwarded-For value it likes. If the gateway trusts that header from every source, a client can claim to be any address, bypass an allow list, reset its rate-limit bucket on each request or pollute logs that an investigation later relies on. The fix is to restrict the trusted list to the addresses of your load balancers and proxies, and to make sure the public network path cannot reach APISIX without passing through them. Confirm in your own topology which hop is the last one you control, because that determines which address the trusted list should contain.
Before you rely on any of this, run a test from outside your network that sends a forged X-Forwarded-For header and check which address your rate limit, your logs and your ip-restriction rule actually use. Also confirm that the exact X-Forwarded-* behavior you depend on matches your APISIX version, since sanitization of these headers changed in 3.18 and the release notes in the apache/apisix CHANGELOG describe the change.
Configuration options and when each fits
This table compares the settings that decide which client address and scheme APISIX sees behind a load balancer, and the chaitin-waf option that passes that address on. Read each row as a decision: choose the trust boundary first, then the header that carries the client address, then the scheme and WAF settings that depend on them. Confirm exact field names in the plugin reference before writing any configuration.
| Option | What it controls | When it fits |
|---|---|---|
| real-ip plugin with trusted address ranges | Restores the client address from a forwarded header only when the directly connected peer falls inside the ranges you list | Any deployment behind a load balancer or proxy whose address range you can name and keep stable |
| Trusting all upstream addresses | Accepts forwarded headers from any peer that connects, so a client can set its own address | Rarely appropriate; avoid when the gateway is reachable from untrusted networks |
| X-Forwarded-For as the client address header | Carries a chain of addresses added by each hop, so the leftmost entry is only as reliable as the proxies that wrote it | Common default when your load balancer appends to this header; confirm which entry your proxies write |
| Other real IP headers set by your load balancer | Single-value headers some cloud and CDN load balancers write, which are harder to spoof when the proxy overwrites them | When your provider documents a dedicated header; confirm the header name and whether the proxy overwrites client-supplied copies |
| X-Forwarded-Proto for scheme handling | Tells the gateway whether the client used HTTP or HTTPS, which affects redirects, cookies and scheme-dependent plugins | When TLS terminates at a load balancer in front of APISIX; from 3.18, the NGINX configuration sanitizes this header |
| Header sanitization in APISIX 3.18 and later | Removes client-supplied X-Forwarded-* values in the gateway configuration and makes some variables read-only from Lua | Upgrade to 3.18 or later where this behavior matters, then test routes that relied on passing these headers through |
| chaitin-waf real_client_ip set to true (default) | Sends the client address APISIX resolved, not the peer that connected, to the WAF for inspection and logging | When the WAF should see end users, which is usually the case; confirm the setting in the chaitin-waf reference |
| chaitin-waf real_client_ip set to false | Sends the directly connected peer to the WAF, typically the load balancer address | Only when the resolved address is unreliable and you accept that WAF logs and rules see the proxy instead of the user |
How to configure real client IP handling step by step
This sequence takes you from an inventory of every proxy in front of APISIX to a verified configuration in which the gateway resolves the true client address and scheme, and ignores forged forwarding headers. Each step closes with a check you can run and record.
Map every hop in front of APISIX
Your network or platform team lists each component that handles a request before it reaches APISIX: cloud load balancers, CDN edges, web application firewalls, and any Kubernetes ingress or service mesh layer. For each hop, record whether it appends to X-Forwarded-For, sets X-Forwarded-Proto or X-Forwarded-Host, and which address it uses to reach APISIX. The output is a diagram or table showing the order of hops and the header behavior at each one. Confirm the header behavior with the owner of each component, because defaults differ by product and version.
Decide the trusted address ranges
Using the hop inventory, the security and platform teams agree the exact address ranges that APISIX may trust as proxies, such as the subnets of your load balancers. Anything outside those ranges is treated as a client, and its forwarding headers are not believed. Document the ranges, the owner of each one, and the change process for when a load balancer subnet changes. The check is a written list that a second engineer can compare against the live load balancer addresses without asking the author.
Enable real-ip on a non-production route
The engineer adds the real-ip plugin to a single test route in a non-production APISIX instance, setting the trusted ranges and the header that carries the client address. Review the plugin fields against the APISIX plugin documentation for your release, since option names and behavior are version dependent. Confirm that the route still returns the expected responses. The check is a request sent through the trusted load balancer that shows the client address in your test upstream or in the APISIX access log, matching the address of the test client.
Confirm header sanitization for your APISIX version
Your platform engineers check the APISIX changelog for the release you run, since handling of X-Forwarded-* headers changed in 3.18.0. Where the release sanitizes these headers, a client-supplied value should not reach the upstream as trusted. Where it does not, plan a route or edge rule that strips the headers from untrusted sources. The output is a short note stating your release, the header behavior you observed, and any compensating control. Confirm the result by sending a request with forged headers and inspecting what the upstream receives.
Verify rate limits and IP restrictions with a forged header
A tester sends requests from a single test client that carries a different X-Forwarded-For value on each request. If the rate limit or ip-restriction policy changes its decision when the header changes, the gateway is trusting the client's own header and the configuration is wrong. Repeat the test from an address on the denied list while spoofing an allowed address. The check passes when both policies act on the true connecting address or the resolved address from trusted hops only, and the results are recorded with timestamps and request identifiers for the change file.
Confirm scheme detection for HTTPS redirects
The tester calls an HTTP endpoint through the load balancer and checks the redirect location, then sends a request with a forged X-Forwarded-Proto value of http over an HTTPS connection. The application should receive the scheme that the trusted proxy reports, and the forged value should not change redirects, secure cookie flags or absolute URLs generated by the upstream. The check is a set of recorded responses from the upstream showing the scheme it saw for each test, along with the list of routes where scheme matters for your applications.
Check access logs and roll out to production
Operations staff confirm that the APISIX access log records the resolved client address in the field your log platform uses, and that the original connecting address is still available for investigation. Compare log entries for the test requests against the values sent. Then the change owner applies the same configuration to production routes in a planned window, repeats the forged-header checks there, and keeps the test evidence with the change record. The check is a log query in your log platform that returns the expected client address for recent production traffic.
How spoofed headers break rate limits and IP restrictions
Client address and scheme decisions feed every control that keys on IP: restrictions, rate limits, logs, redirects and cookie flags. The cards below cover the failure modes that show up when headers are trusted too broadly or too narrowly, and the check to run for each.
Forged X-Forwarded-For bypasses restrictions
If APISIX takes the first address in X-Forwarded-For at face value, any client can send a header claiming an address on your allow list and pass an ip-restriction rule. The gateway must trust forwarded headers only from the proxies you operate, never from the open internet. Test this by sending a request with a spoofed allow-listed address directly to the gateway and confirming the decision is still based on the socket peer.
Rotating headers evade rate limits
A client that changes the X-Forwarded-For value on every request can make each request look like a new caller to a rate limiter keyed on the client address. The fix is to resolve the client address only through trusted hops and to key limits on the value APISIX resolved, not on a raw header. Confirm with a test that repeated requests with changing forged headers still hit the same limit counter.
Shared NAT addresses cause false blocks
When many users reach the gateway through one corporate or carrier NAT address, a per-IP limit or deny rule can block all of them after one client misbehaves. This is an allocation problem rather than a header problem, so the decision belongs to the limit design: pick thresholds and keys that account for shared egress addresses, and confirm with your network team how many users share each egress range.
Proxy chains that change address ranges
Trusted proxy lists are often written once and then go stale when a cloud load balancer, ingress tier or CDN changes its address range or adds a hop. A stale list either stops trusting a real proxy, so every client appears to be the proxy, or trusts an address that no longer belongs to you. Review the trusted range list whenever the network path changes and alert when the resolved client address is an internal proxy address.
Wrong scheme breaks redirects and cookies
If the gateway sees plain HTTP because the TLS-terminating load balancer did not pass the forwarded protocol, applications may build http:// redirect URLs, loop between HTTP and HTTPS, or omit the Secure flag on cookies. APISIX 3.18.0 sanitizes X-Forwarded-* headers in the NGINX configuration and makes $var_x_forwarded_host and $var_x_forwarded_port read-only from Lua, so confirm your upstream scheme handling after upgrading. Check the Location header and Set-Cookie flags for a sample login flow.
Logs record attacker-chosen addresses
Access logs and the chaitin-waf real_client_ip option both record whichever client address APISIX resolved, so a spoofed header can write a false address into your audit trail and into the inspection context. Log the directly connected peer alongside the resolved client address, and keep the trusted proxy configuration in version control so an investigator can explain which value was trusted for any given request.
Questions teams ask before fixing real client IP handling
How much effort does a typical client IP fix take?
Effort depends mainly on how many proxy hops sit between the client and APISIX, how many routes and plugins read the client address, and how many rate limit and IP restriction rules depend on it. A gateway behind one load balancer with a handful of routes is a much smaller job than a chain of CDN, WAF and load balancer layers feeding dozens of services. No fixed duration applies to every environment, so ask for an estimate built from your own hop count, route inventory and test coverage. Most of the work is verification, not configuration.
What drives the cost of fixing forwarded headers?
The main cost drivers are:
- Number of proxy hops and the load balancer settings at each one
- Number of routes, consumers and plugins that use the client address
- Existing rate limit and ip-restriction rules that must be retested
- The size of the test matrix, including IPv4, IPv6 and spoofed header cases
- Whether a non-production environment matches production topology
A quote should itemize discovery, configuration, testing and handover, and state which items are one-time and which need ongoing review after changes to the network.
Can our in-house team fix real client IP handling without a partner?
Yes, if the team can read the network path, change load balancer settings and test from outside the trusted network. The work needs engineers who understand APISIX configuration, your load balancer and any CDN or WAF in front of it, and who can run controlled tests. Teams often underestimate the cross-team coordination: network, platform and security owners must agree on which addresses are trusted. A partner is useful when hop counts are high, rate limits are business critical, or the team lacks time to build a repeatable test set.
What do you need from our team to start?
Plan to provide:
- A network diagram showing every hop from the client to APISIX, including CDN, WAF and load balancers
- Load balancer and proxy settings for each hop, including which headers they add or pass on
- A list of routes, plugins and rules that depend on the client address
- A non-production test environment that matches the production proxy chain as closely as possible
- Named contacts for network, platform and security decisions
Missing diagrams usually add the most time, because trusted proxy ranges cannot be set safely without them.
Do we need to upgrade to APISIX 3.18 or later to fix this?
Not necessarily, but 3.18.0 is the release where APISIX sanitizes X-Forwarded-* headers in the NGINX configuration, removes $var_x_forwarded_proto and makes $var_x_forwarded_host and $var_x_forwarded_port read-only from Lua, according to the APISIX changelog. Teams on earlier versions can still restore the real client address with the real-ip plugin and correct trusted proxy settings, but they must check their own header handling carefully. Confirm your current version, read the changelog entries between your version and 3.18.0, and test the effect on your routes before choosing an upgrade path.
How do we validate that the client IP is correct?
Validate from outside the trusted network, not from inside the proxy chain. Send requests with a known source address and with spoofed X-Forwarded-For, X-Forwarded-Proto and X-Forwarded-Host values, then confirm the gateway logs, rate limits and ip-restriction decisions use the expected address. Repeat the checks after each load balancer or CDN change. If you run the chaitin-waf plugin, confirm its real_client_ip setting (default true) sends the resolved address rather than the peer address. Keep the test cases in version control so they can run again after every upgrade.
What breaks if we trust forwarded headers from everyone?
Any client can set X-Forwarded-For to an address you allow, which bypasses ip-restriction rules and lets attackers rotate identities to evade rate limits. Logs and audit trails also record false source addresses, which weakens incident investigation. Trust should be limited to the specific addresses of your own proxies. Confirm the exact trusted ranges with your network team, because a range that is too broad recreates the same risk.
What changes for existing routes when you upgrade past 3.18
Upgrading past 3.18 can change how existing routes behave if they trusted client-supplied X-Forwarded-* values. APISIX 3.18.0 sanitizes these headers in the NGINX configuration, removes $var_x_forwarded_proto, and makes $var_x_forwarded_host and $var_x_forwarded_port read-only from Lua. Any route, plugin or upstream that read a header a client could set, and then used it for routing, redirects, signed URLs, or logging decisions, may see different values after the upgrade. The exact scope of each change is recorded in the apache/apisix CHANGELOG, which should be read against the version you are moving to before any change window is booked.
Start the review with an inventory rather than a test run. List every route and global rule that uses the real-ip plugin or depends on scheme and host, and search your custom Lua plugins for reads and writes of $var_x_forwarded_proto, $var_x_forwarded_host and $var_x_forwarded_port. Ask the team that owns each upstream whether it builds absolute URLs, enforces HTTPS, or validates the Host header, because those services are the most likely to break silently. Confirm that your load balancer in front of APISIX sets the forwarded headers itself, since the gateway now treats what arrives from outside as untrusted.
After the upgrade, verify in a staging environment that mirrors production topology. Send requests through the same load balancer chain and check the scheme, host and client address that reach each upstream and each log line. Run rate limit and IP restriction tests from a known address to confirm the counted client is correct. Keep the previous configuration and image available so a rollback is possible, and treat any upstream that generated redirects or cookies from request headers as a priority for sign-off.
How DeployOpen scopes and delivers client IP and header trust work
DeployOpen treats the client address and scheme as a security decision, not a configuration default. We scope which proxies sit in front of APISIX, build and test the trust boundary against real traffic, and hand over the settings and runbooks your team needs to keep it correct.
How we scope
- Map every load balancer, CDN and proxy hop between clients and the APISIX data plane.
- Record which hop headers each layer sets, appends or strips today.
- Agree in writing which rate limits, IP restrictions and WAF rules depend on the resolved client IP.
How we build
- Configure the real-ip plugin with trusted proxy ranges limited to your own load balancer addresses.
- Verify on APISIX 3.18 or later that X-Forwarded headers are sanitized as the release notes describe, and confirm the behavior on your exact version.
- Test spoofed X-Forwarded-For and X-Forwarded-Proto values from untrusted sources and confirm they do not change limits or restrictions.
How we hand over
- Deliver the final trusted proxy list, plugin settings and the reasoning behind each trust decision.
- Document the chaitin-waf real_client_ip behavior if a WAF is in the request path, with test evidence.
- Provide a runbook for changing load balancers or CDNs so the trusted proxy list is updated before traffic moves.
How we support
- Engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts.
- Ongoing support, upgrades and re-testing after infrastructure changes are available as a retainer.
- Ask any partner to confirm which headers their upstream and CDN overwrite before you rely on them.
