Apache APISIX high availability guide

Apache APISIX high availability starts with where configuration lives

An API gateway in front of production traffic cannot be a single point of failure, but making APISIX highly available is not just a matter of running a second node. High availability means choosing a deployment mode, making the configuration store resilient, running several gateway nodes and checking the upstreams those nodes forward to. The first decision, and the one that shapes the rest, is where configuration lives, because that determines what keeps working when a component fails.

The harder part is that the data plane and the control path fail differently. Gateway nodes keep serving traffic from the configuration they have already loaded, so a short outage of the control path does not by itself stop requests. What stops is change. New routes, consumer updates, certificate rotations and plugin settings all reach the gateway through the Admin API and etcd in the traditional and decoupled modes, so an etcd problem or an unreachable Admin API can leave production running on stale settings. A team that tests only whether traffic flows during a failure will miss the more likely problem: nobody can safely change the gateway until the control path is back.

APISIX documents three deployment modes. Traditional mode runs the data plane and control plane together on each node and depends on etcd for configuration. Decoupled mode separates the data plane from the control plane, and both share one etcd cluster. Standalone mode removes etcd entirely and loads configuration from a YAML or JSON file or through an API. Each mode trades resilience of change management against operational load. Traditional mode is the simplest to start with but ties every node to the etcd cluster. Decoupled mode lets you isolate the control plane but adds components to run and secure. Standalone mode removes the etcd dependency but moves change control onto your file distribution and review process.

Before you pick a mode, confirm several things with your platform team and with the current APISIX deployment modes documentation: how each mode behaves when etcd is unavailable, which Admin API ports must be reachable from which networks, how configuration changes are rolled out and reverted, and whether your change process can tolerate a period where changes are blocked. Ask the same questions of any partner's proposed design. A design that answers them in writing is easier to verify than one that describes only the steady state.

The mode decision also sets the work that follows: how many etcd members you run and where they sit, how gateway nodes are placed behind a load balancer, and how health checks on upstreams are configured so that a failing backend is removed from rotation. Record the chosen mode, the reasons for it and the failure behavior you accept as part of the design, because the rest of the high availability work depends on that record.

Which APISIX components carry availability

An APISIX gateway is only as available as the parts that route, store and expose its configuration. The cards below list each component that needs redundancy or a documented failure plan, and they show which ones change with the deployment mode you choose.

Gateway data plane nodes

In traditional mode, the gateway nodes accept client traffic on port 9080 and run the routing and plugin logic for every request. Each node is a single point of failure only if it is the only one, so plan at least two nodes across separate hosts or zones. Check that each node can keep serving from its last loaded configuration when a connection to the control plane is lost, and confirm how your team will add or remove nodes without dropping traffic.

Admin API on port 9180

The Admin API on port 9180 is how operators and automation create and change routes, upstreams and plugins. It is not client-facing, and it should not be reachable from the public internet or from general application networks. Availability here affects change control more than traffic, so decide which systems may call it, who holds the admin keys, and what happens to changes if the Admin API is briefly unavailable.

etcd cluster for configuration

In traditional and decoupled modes, etcd stores all route, upstream and plugin configuration that the gateway reads. Losing etcd quorum stops configuration changes, and a damaged etcd store can leave nodes without a valid source of truth. Run an odd number of etcd members across failure domains, monitor leader elections and disk latency, and rehearse restoring from snapshots. Your team should own this cluster or agree the support terms in writing.

File-driven configuration in standalone mode

In standalone mode, APISIX has no etcd and reads conf/apisix.yaml or conf/apisix.json, checking for changes every second and hot-reloading without a restart. The availability risk moves to how that file is written, distributed and versioned across nodes. Confirm that every node receives the same file, that a malformed edit is rejected before it reaches production, and that you can roll back to the previous file quickly.

External load balancer in front of gateways

The load balancer distributes client connections across gateway nodes and removes unhealthy nodes from rotation. Without it, clients depend on one node address or on DNS caching that is slow to reflect failures. Check its health probe settings against the gateway's behavior, its own redundancy, and how it handles TLS termination and connection draining during node replacement.

APISIX Ingress Controller on Kubernetes

If Kubernetes is the target, the APISIX Ingress Controller is a separate project that maps Kubernetes resources to APISIX routes. Its availability affects how quickly new or changed resources reach the gateway, not whether existing traffic flows. Confirm the controller version you will run, how it is deployed with replicas, and how it runs alongside any existing ingress during the transition. Verify the current release details on the APISIX downloads page before you plan the upgrade path.

Choosing traditional, decoupled or standalone mode for resilience

This table compares the three APISIX deployment modes and the sub-decisions that come with each, including where etcd runs, how configuration changes are made, and how many gateway nodes sit behind the load balancer. Read each row against your own operating model, since the choice shapes who carries the failure domain and who performs routine changes.

Option	When it fits	What it costs you operationally
Traditional mode (data and control plane together)	Single-team estates that want the simplest topology and can run etcd alongside gateway nodes with clear ownership	Gateway nodes and the Admin API share one deployment unit, so Admin API exposure and node sizing must be planned together; etcd health directly affects every node
Decoupled mode (separate control plane, shared etcd)	Organizations that want the Admin API and configuration write path isolated from client traffic nodes while keeping dynamic configuration	Requires a shared, highly available etcd cluster and its backup, upgrades and access control; etcd becomes a critical dependency for the control plane and must be monitored closely
Standalone mode (no etcd, file or API configuration)	Environments that prefer not to run etcd, or that already manage configuration as code through version control and automated distribution	Change control moves to file distribution and per-node reloads, so you need tooling to keep every node on the same file version and to verify each reload
etcd placement and ownership	Use a dedicated etcd cluster with an operator who owns backup and upgrades when you need configuration that survives gateway node loss	Who patches, backs up and restores etcd must be named in the runbook; an unmanaged etcd cluster is the most common source of outage risk in dynamic modes
Admin API versus file-driven changes	Admin API suits teams that automate route changes through pipelines; file-driven changes suit teams that review every route in version control	Admin API changes need authentication, network restriction and audit logging; file-driven changes need a sync mechanism and a check that every node has loaded the same revision
Number of gateway nodes behind the load balancer	Two or more nodes in separate failure domains are the usual starting point for high availability, with capacity added horizontally as traffic grows	Each added node adds configuration to keep in step and health checks to tune; node count does not replace the need for a resilient etcd or file distribution path
Separate control plane or not	Choose a separate control plane when change frequency or access rules differ sharply from client traffic handling	A second deployment to build, secure and upgrade; confirm the required configuration for your release in the deployment modes documentation before committing

How to build an Apache APISIX high availability setup step by step

This sequence takes a team from a pinned APISIX release to a gateway tier that keeps serving traffic when a node, an etcd member or a backend fails. Each step ends with a check you can run and record, so the finished setup is shown to work before anyone relies on it.

Pin the APISIX release line
The platform team pins one APISIX release and records it in change management before any node is built. Confirm the current release on the APISIX downloads page on the day of the build. At the time of writing, that page lists APISIX 3.19.0, released September 28, 2026, and marks no release as long-term support, so your team must decide how long each release stays in service and how upgrades are scheduled. Done when the pinned version appears in the change record and every node package or image refers to that same version.
Select the deployment mode
The architecture lead chooses traditional, decoupled or standalone mode by comparing the requirements in the decisions table earlier on this page with your network layout, change process and failure tolerance. Traditional and decoupled modes depend on etcd for shared configuration, while standalone mode reads configuration from a YAML or JSON file or the API and runs without etcd. Record who owns configuration changes in the chosen mode. Done when a signed decision record names the mode, the ports in use (9080 for client traffic and 9180 for the Admin API in traditional mode) and the owner of each configuration path.
Build the etcd cluster or file path
In etcd-backed modes, the infrastructure team builds a cluster with an odd number of members, so that a majority remains after one member fails. Confirm member count, disk type, network latency between members and placement across failure domains against the etcd documentation, since that is the authoritative source for its requirements. In standalone mode, instead, place the identical configuration file on every gateway node and confirm the file path each node reads. Done when every etcd member reports healthy, one leader is elected, and stopping a single member leaves writes working. In standalone mode, done when an edit to the file on one node is picked up without a restart.
Install gateway nodes and secure the Admin API
The platform engineer installs at least two gateway nodes from the pinned release, using identical configuration files and the same set of enabled plugins. Before any load balancer points at the nodes, restrict the Admin API on port 9180 to the management network, replace every default key and confirm that the Admin API requires the new credentials. Done when the Admin API is unreachable from the client network, reachable from the management subnet with the new key, and every node returns the same route list when queried.
Front the nodes with a load balancer
The network team places a load balancer in front of the gateway nodes so that clients reach the nodes only on the client traffic port, 9080 in traditional mode. The load balancer runs its own health checks against the gateway nodes, and those checks must confirm that the gateway itself responds, not that a backend responds. Clients should not address nodes directly. Done when stopping one node removes it from the pool, requests continue to succeed through the remaining nodes, and the node rejoins the pool after it restarts.
Configure upstream health checks and retries
The API team configures each upstream with active health checks that probe backends on a schedule and passive health checks that count failed live requests. Passive checks catch failures that the scheduled probes miss. Set retries only for idempotent requests, such as GET and HEAD, and confirm that non-idempotent methods like POST are not retried. Done when stopping a backend causes it to fail probes and leave rotation, restoring it returns it to rotation, and a test POST that fails at the backend is not sent a second time.
Change-test configuration propagation
The platform team makes a visible, reversible change, such as a route or a plugin setting, and confirms that every gateway node serves it. In traditional mode, apply the change through the Admin API. In standalone mode, edit the configuration file and confirm that the node hot-reloads it, since the deployment modes documentation describes a one-second check for changes without a restart. Done when requests sent to each node return the changed behavior, no node needed a restart, and any node that did not pick up the change is investigated before go-live.
Rehearse node, etcd and backend failures
The operations team rehearses three failures in a test environment or an agreed maintenance window: stopping one gateway node, stopping one etcd member and stopping one backend. Before each test, write the expected result, including which requests should fail and which should succeed. After each test, record the observed result, client error counts and recovery steps. Done when every rehearsal has an expected and an actual result on file, the gaps found are fixed and written into the runbook, and the rehearsal is repeated after each upgrade.

How the high availability design changes by environment

On a single node, APISIX runs as one gateway instance, so the gateway layer has no redundancy. If etcd is part of your chosen mode, it also runs as one member unless you deploy more. A single node is usually a reasonable choice for development, test and staging, where short outages are tolerable and the configuration can be rebuilt quickly.

One point of failure: a host failure, process crash or failed upgrade stops all client traffic through the gateway on port 9080.
Admin API exposure: the Admin API on port 9180 runs on the same host, so its access controls matter even when traffic is light.
Single etcd member: losing its data volume means losing configuration history unless you hold recent backups.
File-driven standalone mode: this removes the etcd dependency, but configuration lives in a file that must be version-controlled and restored by hand.
Maintenance windows: restarts and upgrades interrupt traffic, so schedule them with the application owners.

Before accepting a single node for any environment, confirm the downtime tolerance in writing, the procedure for rebuilding the node from source control, and whether the test routes and plugins match production closely enough to validate changes.

What drives the effort of an APISIX high availability build

This table lists the factors that change how much design, build and recurring operations work an APISIX high availability setup needs. Read each row as a question to answer about your own estate before asking for a quote. The main takeaway is that route volume, environment count and etcd experience shape the work more than the gateway software itself.

Driver	How it changes the work	How to keep it down
Number of routes and upstreams	Larger configuration sets make changes slower to review, test and roll out, and make failover tests longer to script and verify.	Group routes by domain or team, template repeated routes, and keep configuration in version control so changes are reviewed in batches.
Number of environments and failure domains	Each environment needs its own etcd, gateway nodes and tests, and each failure domain (zone, rack or site) adds placement rules and failover scenarios.	Standardize one reference topology and reuse it across environments, and confirm early which failure domains the business actually requires.
Identity and backend integrations needing health and failover checks	Each identity provider, token service or backend pool needs its own health check, timeout and failover behavior, and its own test cases.	Inventory integrations first, group them by criticality, and agree which ones must fail over automatically and which fail closed.
Team experience with etcd and NGINX-based operations	Teams new to etcd quorum, compaction and backup, or to NGINX worker tuning, need more design review, runbooks and training before go-live.	Confirm which skills exist in-house, plan knowledge transfer into the build, and ask for runbooks covering etcd recovery and gateway restarts.
Compliance evidence for availability controls	Assessors typically expect documented failover design, tested recovery and retained logs, so evidence collection and tests add scope.	Map availability requirements to controls before design starts, and define which test results and logs will be retained as evidence.
Kubernetes involvement	If the gateway runs on Kubernetes or uses the APISIX Ingress Controller, node scheduling, pod disruption, and etcd placement add design decisions and operational coverage.	Decide early whether the gateway runs inside or outside the cluster, and document how the Ingress Controller and gateway pods are scheduled and upgraded.
One-time design and build work	Architecture, etcd and node build, health check configuration, failover testing and documentation are paid once per build, and scale with the drivers above.	Ask for a fixed-scope quote listing environments, gateway nodes, etcd members, deployment modes and tests included.
Recurring operations	Patching, etcd maintenance, certificate renewal and periodic failover rehearsal continue after go-live and are usually billed or staffed as ongoing work.	Agree the rehearsal frequency and patch cadence in writing, and confirm whether a support agreement or internal staff will own them.

Protecting the high availability control path

In a high availability APISIX design, whoever can change configuration can change every route, plugin and upstream at once. The controls below protect the path that carries configuration from the Admin API, etcd or configuration files to each gateway node, and they should be in place before any node takes live traffic.

Restrict the Admin API

In traditional mode, APISIX serves the Admin API on port 9180, separate from client traffic on port 9080. Limit access to that port to named management networks, and replace the default Admin API keys before any gateway node is reachable from those networks. Confirm the key and access settings for your release in the official deployment documentation, and test that the API refuses requests from outside the approved range.

Encrypt etcd traffic

The etcd cluster holds the routes, consumers and plugin settings that every gateway node reads. Encrypt client-to-etcd and peer-to-peer traffic using the TLS and authentication mechanisms the chosen APISIX and etcd releases document, and confirm that each gateway node connects with the expected certificates. Verify the setup by checking that an unauthenticated connection to etcd fails.

Control standalone configuration files

In standalone mode, APISIX reads conf/apisix.yaml or conf/apisix.json and checks for changes every second, so anyone who can write that file can change live routing on every node that receives it. Restrict write access to a small deployment group, route all edits through version control with review, and confirm that the distribution method cannot be used to push unreviewed files.

Protect load balancer health endpoints

The load balancer in front of the gateway nodes relies on health checks to decide which nodes receive traffic. Restrict which networks can reach those health endpoints and the Admin API, so that outsiders cannot read node status or manipulate routing decisions. Also review how TLS certificates for the listeners are stored, rotated and limited to the principals who need them.

Back up and encrypt etcd data

An etcd backup is the recovery point for the whole control plane, so it needs the same protection as the live cluster. Set a backup schedule, store copies encrypted, and limit restore access to named operators. Test restores in a separate environment, because an untested backup does not show that the configuration can be rebuilt in the time your recovery targets allow.

Log configuration changes

Every change to routes, upstreams, consumers and plugins should leave a record that shows who made it, when, and through which path. Forward Admin API and deployment logs to a central store that the gateway operators cannot quietly edit, and confirm that alerts fire for unexpected changes. Auditors will ask for this evidence, so align retention periods with your compliance obligations.

How to prove the Apache APISIX high availability design fails safely

The only reliable proof that an APISIX high availability design fails safely is a test plan run against your own topology, with pass criteria written down before the first failure is injected. Start by reading the expected failure behavior for your exact APISIX release and deployment mode in the official documentation, because the behavior you are testing for depends on that release and on whether you run traditional, decoupled or standalone mode. Agree the pass criteria with the platform, security and application owners, then record them in the test plan so nobody can redefine success after the results arrive.

The core tests exercise the failure points the design depends on. Stop one gateway node while a sustained load test runs, and confirm that the load balancer in front of the gateway fleet stops sending it traffic and that client error rates stay within the threshold you set. Stop one etcd member in any mode that uses etcd, then confirm that reads, configuration changes through the Admin API and recovery when the member returns all behave as the design expects. Take one backend offline and confirm that the active health checks on the upstream remove it from rotation, then confirm it returns to rotation after it recovers. Each test should state the injected fault, the load profile, the observed signal and the exact condition that counts as a pass or a fail.

Two tests examine the separation between control and data paths. Change a route through the control plane and confirm that the change reaches every gateway node within the propagation window you define, checking the result on each node rather than on one. Then cut the control path, for example by blocking gateway access to etcd or the control plane, and confirm that the data plane keeps serving traffic using the last configuration it loaded. In file-driven standalone mode, also test the file-based reload by altering the configuration file and confirming the change is applied, since that path has no etcd dependency. Log every timestamp so the team can calculate the time from fault to detection, detection to recovery and recovery to full traffic.

Restoring etcd from a backup into a separate test cluster proves that your backup is usable, not just present. Restore the most recent snapshot into an isolated environment, point a non-production gateway at it, and verify that routes, upstreams, consumers and plugin configuration match what production held at the snapshot time. Measure how long the restore takes from start to a working gateway, because that number belongs in your recovery runbook. Keep the test cluster isolated from production credentials and network paths, and destroy it once the evidence is captured.

Keep a results record with one row per test, containing the release tested, the topology, the date, the measured timings, the pass or fail result, and a link to the raw logs. Record timings as the team measured them in its own environment, and do not copy figures from vendor material. When a test fails, stop the rollout decision, classify whether the fault is in the design, the configuration, the runbook or the test itself, fix it, and rerun the full test that failed along with any test it could affect. Keep the failed run in the record, because a documented failure and its fix is stronger evidence for auditors and risk owners than a clean report with no history.

Questions teams ask before building APISIX high availability

What drives the cost and effort of an APISIX high availability build?

Scope, the number of environments and the number of gateway nodes drive most of the effort. The cost has four parts: planning and architecture, build and configuration, testing of failure scenarios, and handover. Who is paid for what varies. A fixed-scope quote from DeployOpen covers the work listed in the written scope, while cloud or data center charges for gateway nodes, load balancers and etcd hosts are paid to your own infrastructure provider. Effort rises with multiple regions, strict change windows, many upstreams that need health checks, and incomplete identity or backend inventories. The design, build and first failover tests are one-time work. Infrastructure, etcd and APISIX upkeep, release testing and any support retainer recur. Ask any quote to itemize these parts and to state what is excluded.

How long does an APISIX high availability project take?

The duration depends on scope, the number of environments and the amount of testing in the agreed plan. DeployOpen sets the timeline in the written scope before work starts, so ask for the milestones, the dependencies on your teams and the test windows in that document. Schedule risk usually comes from waiting on access, network approvals, identity provider configuration and change windows. Ask the plan to show which tasks run in parallel and which block others. If you plan a failover rehearsal in staging, reserve time for it and repeat it after significant configuration or release changes.

What do we need to provide before an APISIX high availability build starts?

The build depends on access and information that only your organization can supply. Plan to provide:

Access: accounts or credentials for the cloud, data center or air-gapped environment where the gateway runs
Network diagrams: current topology, firewall zones, DNS and load balancer layout
Inventories: identity providers, upstream services and their health check endpoints
Change windows: dates and approval steps for staging and production changes
Named owners: one person each for networking, identity, backend services and go-live approval

Missing owners and late access are the most common causes of schedule slips. Confirm each item in the written scope before work begins.

Can our internal team build APISIX high availability without outside help?

Yes, an internal team can build it alone if it has operators with etcd and NGINX-based operations experience and time to rehearse failures. APISIX runs on NGINX and stores its configuration in etcd, so the people who run the cluster need to understand quorum, backup and restore, and how gateway nodes reload configuration. The hardest part is usually not the first deployment but proving that the cluster survives node loss, etcd member failure and network partitions. Budget time for those tests and for writing runbooks that someone other than the builder can follow. A partner can help, but the decision depends on your team's capacity and skills.

Does standalone mode remove the need for etcd in APISIX?

Yes, standalone mode runs APISIX without etcd and loads configuration from a YAML or JSON file or through the Admin API. In file-driven standalone mode, APISIX checks the file for changes every second and hot-reloads it without restarting. The trade-off moves into change control. There is no shared configuration store, so each gateway node must receive the same file version, and you must verify that delivery. Rollout order, drift detection, rollback and audit trails become processes your team owns. Teams that already run Git-based pipelines often find this manageable, but confirm the design with Apache APISIX documentation before you choose the mode.

What does an implementation partner add beyond the APISIX build itself?

A partner adds a second set of experienced hands on design, testing and handover. DeployOpen plans the topology, builds the deployment in your environment, tests failure scenarios and writes runbooks for your operators. It can also harden the gateway, run security testing against the APIs and prepare systems and evidence for compliance assessments. DeployOpen does not issue audit reports, attestations or certifications. Those come from independent auditors and certification bodies. The main value is a documented, tested result that your team can operate after handover, along with a clear record of what was decided and why.

Which parts of APISIX high availability recur after go-live?

Several parts recur after go-live. Plan for regular etcd upkeep, including backups, restore tests, member health checks and certificate rotation. Plan for upgrades to APISIX and the APISIX Ingress Controller, which means tracking the release line, testing plugin and configuration changes, and rehearsing rollback. The APISIX downloads page does not mark any release as long-term support, so confirm the support window for your chosen line directly with Apache APISIX and, if you use a commercial product, with API7.ai. Infrastructure costs, monitoring and any support retainer also recur. Ask any quote to separate one-time build work from these ongoing items.

How DeployOpen scopes, builds and hands over APISIX high availability

DeployOpen plans, builds, documents and supports Apache APISIX high availability inside your own infrastructure, whether that is your cloud accounts, data centers or air-gapped networks. Each engagement starts from a written, fixed scope that names the deployment mode, the etcd topology, the gateway node layout and the health checks before any work begins.

How we scope

Confirm which deployment mode fits the availability target: traditional, decoupled or file-driven standalone
Inventory the gateway nodes, etcd members, network paths and upstream services that must stay reachable during a failure
Agree in writing which failure scenarios are tested, including node loss, etcd member loss and upstream outages

How we build

Deploy gateway nodes behind your load balancer with the data plane and control plane layout agreed in scope
Configure a highly available etcd cluster, or standalone configuration files distributed through your existing change process
Set active health checks on upstreams and record the thresholds, ports and Admin API restrictions in the build notes

How we hand over

Run the agreed failover tests and record results, including what was tested, how, and what remained open
Deliver runbooks for adding gateway nodes, replacing an etcd member and restoring configuration from backup
Walk your platform and operations teams through the architecture, monitoring signals and escalation paths before sign-off

How we support

Offer ongoing support, upgrades and continuous testing as a retainer, separate from the fixed-scope build
Review APISIX and etcd release notes against your deployment and plan upgrades in test environments first
Repeat failover testing after material configuration or topology changes so the documented behavior stays accurate

Apache APISIX high availability