Apache APISIX access logs to Elasticsearch

How to send Apache APISIX access logs to Elasticsearch

The supported way to get Apache APISIX request logs into Elasticsearch is the elasticsearch-logger plugin, which writes each log entry to an index named by the required field.index setting. That setting accepts date patterns such as apisix-{%Y.%m.%d}, so the plugin can create daily indices without a separate rotation job. The gateway stays in your infrastructure throughout: APISIX sends the entries directly to your Elasticsearch endpoint, and no log data passes through a hosted service. The plugin stores the Elasticsearch connection and any auth credentials, which are AES-encrypted in etcd. It verifies TLS by default, so the CA chain for your cluster must be available to the gateway before you turn on the plugin. Entries are batched every 5 seconds or every 1,000 entries, whichever comes first, and up to 8,192 entries can wait in the pending queue. Confirm those defaults against the plugin documentation for your APISIX version, and decide what happens to entries that cannot be delivered once the queue fills.

This integration does not size or operate the Elasticsearch cluster. Node roles, shard counts, hot and warm tiers, retention policy, index lifecycle and the number of master-eligible nodes all belong to the Elasticsearch design, which should be settled before the gateway starts writing. Choosing the distribution and its license is also a separate decision, covered in the sibling sections on Elasticsearch and OpenSearch options. Before you enable the plugin in production, ask the platform team to confirm three things: the index naming pattern matches your retention and access model, the write user has only the privileges needed to create and write to those indices, and the TLS configuration is verified in a non-production environment. Plan the rollout so that a failed or slow log destination cannot degrade client traffic, and test what the queue does during an outage.

What you need before connecting APISIX to Elasticsearch

Five items need confirmation before any plugin configuration is written: the APISIX build, the Elasticsearch endpoint and write path, the network route between nodes and the cluster, the CA chain for TLS, and the licensing and version choice. Each one changes what the logger plugin needs, so settle them first and record the answers with the configuration.

Running APISIX instance and version

Confirm which APISIX release each gateway node runs and whether the cluster is in traditional, decoupled or standalone mode. The current release in the supplied facts is APISIX 3.19.0, released September 28, 2026, and the downloads page does not mark any release as long-term support. Mixed versions across nodes make plugin behavior harder to verify, so align them before rollout. Ask whether the elasticsearch-logger configuration is stored in etcd or in a standalone file.

Reachable Elasticsearch endpoint

Confirm the HTTPS endpoint the gateway will write to, including host, port and whether a load balancer or coordinating-only node fronts the cluster. Check that the target index pattern, such as a daily apisix-{%Y.%m.%d} pattern, is allowed for the writing account. Confirm whether the cluster enforces index templates or lifecycle policies that could reject new daily indices.

Write-only service account

Create a dedicated account whose role grants write access to the APISIX index pattern and nothing else. The plugin stores these credentials encrypted in etcd, so a narrow role limits the damage if the gateway configuration is read. Record who owns rotation and how a password change is applied across all gateway nodes. Avoid reusing an administrator account from the Elasticsearch console.

Network path from gateway nodes

Map the route from every APISIX node to the Elasticsearch endpoint, including firewall rules, proxies, egress restrictions and any service mesh policy. Test from each node, not from a jump host, because nodes in different zones or subnets often have different rules. Confirm that the path is allowed during a failover of the Elasticsearch endpoint. A blocked path shows up as a growing backlog, not as an immediate error.

CA chain for TLS verification

The plugin verifies TLS certificates by default, so the gateway must trust the full chain from the Elasticsearch server certificate up to its root or intermediate CA. Obtain the chain from the team that issues the cluster certificates and confirm which CA bundle path the gateway nodes will read. Plan how the bundle is updated when certificates renew. Disabling verification to get past a chain error is a risk to avoid.

Distribution and version choice

Confirm which Elasticsearch distribution the team runs. The default Elastic distribution is under the Elastic License 2.0, and AGPLv3 is offered as an OSI-approved option, while OpenSearch is released under Apache License 2.0. Confirm that the plugin behaves the same against the chosen distribution by testing it. Elastic 8.x end of maintenance is January 15, 2027, which affects the version you pick for a new deployment.

Configuring the elasticsearch-logger plugin in sequence

This sequence connects selected APISIX routes to daily Elasticsearch indices using a dedicated write user, encrypted credentials and verified TLS. Each step ends with a check, so the team confirms the gateway writes the right data before production traffic depends on it.

Create a least-privilege Elasticsearch write user
The Elasticsearch security administrator creates a dedicated user for APISIX log writes only. Grant that user the privileges needed to create indices and write documents to the index pattern you plan to use, such as apisix-*, and no more. Do not reuse an administrator account or a user that reads logs for dashboards. Check: sign in as the new user and confirm it can index a test document into a matching index, but cannot read or delete other indices.
Store the password through the Admin API
The APISIX administrator adds the log user's password to the elasticsearch-logger configuration through the Admin API. In traditional deployment mode the Admin API listens on port 9180. APISIX stores the auth credentials AES-encrypted in etcd, so the password is not kept as plain text in the data store. Keep the value out of version-controlled files and shell history. Check: read the plugin configuration back from the Admin API, confirm the password is not returned in plain text, and confirm the value does not appear in your Git repository or deployment scripts.
Set the endpoint and date-based index name
The administrator sets the Elasticsearch address in the endpoint configuration and sets field.index, which is required. Use a date pattern such as apisix-{%Y.%m.%d} so APISIX writes each day to its own index, which makes retention and deletion easier to run later. Confirm that every gateway node can reach the Elasticsearch endpoint on the network path you intend to use. Check: after the first request, the index appears in Elasticsearch with the expected name for the current date.
Enable TLS verification against your CA
Point the plugin at the certificate authority that signed the Elasticsearch HTTP certificate, and keep certificate verification enabled. The plugin verifies TLS by default, so the work is to make the trust chain resolve correctly on the gateway hosts. If the CA is internal, install it in the trust store the gateway uses and confirm the certificate hostname matches the endpoint address. Check: a test write from the gateway succeeds with verification on, and a deliberately wrong CA causes the write to fail with a certificate error in the APISIX error log.
Attach the plugin to target routes
Enable elasticsearch-logger on the routes whose access logs you need, or on global rules if every route should be logged. Start with a narrow scope so the write volume and index size are predictable, then widen it once the output is verified. Plan for the batching behavior of the plugin: entries are sent every 5 seconds or at 1,000 entries, and up to 8,192 entries can wait as pending before the plugin stops accepting more. Check: the plugin appears in the route configuration returned by the Admin API, and no other routes gained it unintentionally.
Send test traffic and confirm documents
Send a known set of requests through the logged routes, including successful and failing responses, and tag them with recognizable paths or request IDs. Wait for the batch window to pass, because entries flush on a timer or when the entry count is reached. Query the daily index in Elasticsearch to confirm the document count rises after the traffic and that the test paths appear in the documents. Check: the test entries are present, and neither the password nor any Authorization header value appears in any document or configuration output.

Mapping gateway fields, routes and indices

This table maps the request data APISIX can place in a log entry to the Elasticsearch fields and index design you will need, and says who reads each field and whether it is safe to store. Confirm every field name against the elasticsearch-logger plugin documentation and your APISIX version before you write a mapping, because names change between releases. Treat anything that can carry a credential or personal data as a review item, not a default.

Field group	Source in APISIX	Target in Elasticsearch	Who reads it	Safe to store
Route and service identifiers	Route ID and service ID from the matched route	Keyword field, one value per route	Platform team, API owners	Yes
Consumer identifier	Consumer name from the authentication plugin	Keyword field, pseudonymous name only	Security team, API owners	Not applicable
Upstream status and timing	Upstream status code and request timing variables	Integer and float fields for latency and status	SRE, on-call engineers	Yes
Client address	Client IP and forwarded headers, subject to header trust settings	IP field, retention set by policy	Security team, privacy office	Not applicable
Request path and method	Request URI and method, including query strings	Keyword field, review query strings before storing	API owners, security analysts	Not applicable
Authorization and cookie headers	Request headers such as Authorization and Cookie	Do not index. Drop or redact before shipping	Nobody should need these in search	No
Daily index name	Index pattern in the plugin field.index setting, such as a date pattern	Index per day, aligned with ILM or retention rules	Index owners, retention administrators	Yes

How batching and backlog limits decide what you can lose

The elasticsearch-logger plugin sends a batch every 5 seconds or every 1,000 entries, whichever comes first, and holds up to 8,192 pending entries when the cluster cannot take them. Those three numbers decide your exposure. While Elasticsearch accepts writes, the backlog stays near empty and log delivery lags by seconds. When the cluster is slow or unreachable, entries queue in the gateway's memory. Once the queue reaches its limit, new entries are discarded, so the outage length that matters is the time it takes for incoming request volume to fill 8,192 entries at your peak traffic. Measure that rate on your own gateway rather than relying on an average.

Size the gateway around that failure, not the happy path. Confirm with your vendor or in your own test how the plugin behaves at the backlog limit and whether it logs the drop. Then decide where the gateway's own access log, or a second log destination, will keep entries during an Elasticsearch outage. Retention elsewhere is the control that matters for audit: a local file or a message queue that outlives the cluster gives you a replay path, while the plugin's queue alone does not. Plan disk and memory headroom for that secondary path as part of the design.

Test the failure before go-live. Stop the Elasticsearch cluster (or block the route from the gateway) under representative request load, let the backlog fill, restore the cluster, and compare the request count at the gateway with the document count that arrived in the index. The gap is your measured loss. Then decide in advance, with your compliance owner, whether that loss is acceptable for the log categories you collect. If it is not, a second log stream is required for audit, and it should be designed and tested with the same drill.

Choosing Elasticsearch, Elastic Cloud on Kubernetes or OpenSearch as the target

Self-managed Elasticsearch on virtual machines gives your team direct control over node placement, storage and upgrades. Elasticsearch separates work across node roles, so the cluster design starts with deciding which nodes are master-eligible, which hold data (including hot, warm, cold and frozen tiers), and which act as ingest or coordinating nodes. A highly available cluster needs at least three master-eligible nodes, so plan for that minimum before sizing anything else. The free default distribution includes TLS, the file and native user realms and role-based access control, which covers the basic controls for an APISIX logging credential. Single sign-on, LDAP or Active Directory authentication and field- or document-level security sit in paid subscription tiers.

Node roles: Assign master-eligible, data and ingest roles deliberately rather than running every node with all roles.
Three master-eligible nodes: Required for a highly available control plane.
Free-tier security: TLS, native users and role-based access control are available without a subscription.
Paid security: Single sign-on, LDAP or Active Directory and field-level security need a subscription tier.
Operations: Your team owns patching, snapshots, capacity and upgrades.

Confirm current tier contents on the Elastic subscriptions page before you commit, because feature placement changes between releases.

Protecting credentials, transport and log content

The elasticsearch-logger plugin sits between live API traffic and a search cluster that often holds sensitive request data, so the risk spans three places: the credentials stored in APISIX, the connection to Elasticsearch, and the content of each log entry. The cards below cover those controls in the order a reviewer would check them.

Encrypted credentials in etcd

The elasticsearch-logger plugin stores the authentication credentials for the Elasticsearch endpoint AES-encrypted in etcd, so the cluster's stored configuration does not hold them in plain text. Encryption does not protect against anyone who can read the encryption key or call the Admin API, which can change the plugin configuration and point logs at another destination. Confirm who holds etcd access, who can call the Admin API on port 9180, and where the encryption key is kept and rotated.

TLS verification kept on

The plugin verifies the Elasticsearch server certificate by default. When a log shipment fails with a certificate error, the fix is to add the internal certificate authority to the trust chain or correct the hostname, not to turn verification off. A disabled check lets anyone on the network path read or alter the log stream, including the credentials sent with each batch. Document the CA bundle used by the gateway and review any change to the verification setting.

Dedicated write-only Elasticsearch role

Create a separate Elasticsearch user for APISIX whose role can only create and write to the daily index pattern, such as apisix-*. It should not have read access to other indices, cluster administration rights or the ability to delete indices. If the gateway credentials are exposed, this limits the damage to log injection rather than exposure of other data. Role-based access control is included in the free Basic tier, so this control does not depend on a paid subscription.

Log format without secrets

Build the custom log format to include only the fields the investigation needs, such as route, status, latency and client address. Exclude the Authorization header, bearer tokens, API keys and any query string parameter that carries a secret, such as a signed URL token. Test the format with a request carrying a dummy token and search the resulting index for that value before go-live. Logs are often copied into backups and dashboards, so a leaked token there stays exposed.

Index access, retention and subscription tier

Limit who can search the apisix indices with roles, and decide how long daily indices are kept before deletion. Field-level and document-level security, along with SSO through SAML or OpenID Connect, appear in Elastic's paid tiers according to the supplied subscription information, so confirm the tier before designing the access model. OpenSearch offers similar controls under its Apache-licensed Security plugin, which you should evaluate against your own requirements.

Why entries are missing or rejected

This table lists the failures most often seen when APISIX request logs do not reach Elasticsearch, with the likely cause and the check that confirms each one. Read each row in order of symptom, then run the check before changing the plugin configuration. Most of these failures trace back to the CA chain, the role's index privileges, the index name pattern, the pending queue or the etcd restore procedure.

Symptom	Likely cause	Check that confirms it
TLS handshake errors in the APISIX error log	The Elasticsearch certificate chain is not trusted by the gateway, often because the intermediate CA is missing from the configured trust bundle	Run openssl s_client against the Elasticsearch endpoint from the gateway host and confirm the full chain is presented and validates against the CA file referenced by the plugin
HTTP 401 responses from Elasticsearch	The username or password in the plugin auth block is wrong, or the credential was rotated in Elasticsearch but not in APISIX	Authenticate with the same credential using curl from the gateway host against the cluster root endpoint and confirm the response is authenticated
HTTP 403 responses for bulk writes	The user's role lacks create or write privileges on the index pattern the plugin writes to, such as apisix-* or the daily pattern	Review the role's index privileges in Elasticsearch security settings and test a bulk write to a sample index name with the same user
No daily index appears, or entries land in an unexpected index name	The date pattern in field.index does not match the expected format, or the index is written in UTC while dashboards and index patterns assume local time	List indices with the cat indices API and compare names against the pattern; compare the gateway host time zone and the timestamps in the newest documents
Gaps in the log stream during an Elasticsearch outage	The pending queue reached the 8,192 entry limit, so newer entries were dropped while the cluster was unreachable	Search the APISIX error log for queue or drop messages during the outage window and compare request counts at the gateway with document counts in Elasticsearch for the same period
Duplicate documents for the same requests	A batch was retried after Elasticsearch accepted it but the response was lost or timed out, so the same entries were sent again	Group documents by request ID or trace field and count repeats; confirm the duplicates fall inside a window where the cluster reported timeouts or restarts
Logging stops after an etcd restore, with decryption or authentication errors	Stored credentials are AES-encrypted in etcd, and the restored data or the encryption key used by APISIX does not match the original, so the stored credential cannot be decrypted	Confirm the encryption key configuration on the restored gateway matches the source environment, then re-submit the plugin configuration through the Admin API and verify a test entry is indexed

Questions teams ask before connecting APISIX logs to Elasticsearch

How much effort does connecting APISIX logs to Elasticsearch take?

The plugin configuration itself is small, but the effort depends on the surrounding decisions. The main drivers are the number of routes that need logging, the expected log volume, how much of an Elasticsearch estate already exists, which licensing tier and security features the cluster uses, and how TLS is set up between APISIX and the cluster. A team with a working cluster and a clear index naming plan will need far less work than a team that must also design node roles, retention and access. Ask any partner to split the quote into design, configuration, testing and handover so you can see where the effort sits.

How long does a typical configuration and test cycle take?

The duration depends on how many routes and log formats are in scope, whether the target cluster is ready, and how long your team needs to review sample log output. The configuration can be applied quickly, but testing needs representative traffic, checks that daily indices appear as expected, and a review of what fields reach the index. Plan a test window long enough to watch a full batching cycle and to confirm that no sensitive header values appear in stored documents. Any timeline in a written scope should name the dependencies, such as cluster access and certificate issuance, that could delay the start.

What does our team need to provide before the work starts?

Most of the delay in logging projects comes from missing inputs, so prepare these before the engagement begins:

Network access from the APISIX data plane to the Elasticsearch endpoint, and a service account with permission to write to the target index pattern.
The CA certificate or chain that APISIX will use to verify the Elasticsearch TLS certificate, plus the hostnames it presents.
A written log policy that lists which request fields may be stored, which headers and query parameters must be excluded, and how long indices are kept.
A test window on a non-production route, with a named person who can review the sample documents.
Named owners for the Elasticsearch cluster and the APISIX gateway, so that configuration changes have an approver.

Can one internal engineer set this up alone?

A single engineer who knows APISIX and has administrative access to the Elasticsearch cluster can apply the plugin configuration alone. The harder part is the review: deciding which fields are safe to store, confirming that tokens and credentials do not appear in documents, and checking that TLS verification is working as intended. Those decisions usually need a security reviewer and an owner for the Elasticsearch cluster, even if one person does the typing. Consider how the work will be handed over, since the configuration will need to be maintained when routes change, plugin versions move forward or the index strategy is revised.

Can DeployOpen run the APISIX to Elasticsearch configuration as a fixed-scope engagement?

Yes. DeployOpen plans, implements, documents and hardens self-hosted open-source platforms inside your own infrastructure, and most engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts. For this integration, the written scope would normally cover the routes in scope, the index naming and retention approach, the log format and field exclusions, the TLS configuration, the test plan and the handover documentation. Ongoing support and upgrades can be added as a retainer. DeployOpen does not publish a price list, so a quote follows from that written scope.

Which Elasticsearch licensing and security features affect this integration?

Elasticsearch and Kibana are offered under the Elastic License 2.0, SSPL and AGPLv3, and the default distribution has been under ELv2 since 2021. ELv2 and SSPL are not OSI-approved open source licenses, while AGPLv3 is. Before you choose a distribution, confirm which license your legal team accepts and which tier provides the features you need. Under Elastic's current subscription page, single sign-on and field- or document-level security sit in paid tiers, and TLS and role-based access control became free in the Basic tier from version 6.8.0 and 7.1.0. If your organization prefers a fully open source option, OpenSearch under Apache License 2.0 is an alternative, but confirm compatibility with the APISIX plugin behavior you need before choosing it.

Which settings should the engineer check first after go-live?

After go-live, check that daily indices are created with the expected name pattern, that batching behaves as configured, and that the backlog limit does not fill during an Elasticsearch outage. Confirm that TLS verification remains enabled and that the stored credentials are not visible in configuration exports. Then sample several stored documents and confirm that no authorization header, token, cookie or sensitive query parameter is present. Repeat this check after any route change or plugin upgrade, since a new field in the log format can reintroduce data your policy excludes.

How DeployOpen scopes and delivers the APISIX to Elasticsearch integration

DeployOpen plans, configures, tests and documents the elasticsearch-logger setup inside infrastructure you control, so that APISIX request logs reach your Elasticsearch cluster with the index pattern, credentials handling and backlog limits agreed in advance. Engagements are fixed-scope, with the scope agreed in writing before any work starts.

How we scope

We confirm which APISIX routes and log fields go to Elasticsearch and which fields must be excluded.
We agree the daily index naming pattern, retention owner and the Elasticsearch deployment target in writing.
We record the Elasticsearch version, license edition and TLS trust chain you will run before build starts.

How we build

We configure the elasticsearch-logger plugin on the agreed routes, with batch and pending-entry settings documented.
We test TLS verification against your internal certificate authority and confirm credentials are stored encrypted in etcd.
We run load and outage tests to observe backlog behavior, dropped entries and recovery after Elasticsearch is unavailable.

How we hand over

We deliver the plugin configuration as files, with a log format reviewed for tokens, cookies and personal data.
We document index templates, lifecycle settings, the runbook for failed batches and the owners of each setting.
We walk your operations and security teams through verification steps and the evidence each control produces.

How we support

We can review index growth, batch errors and plugin changes after each APISIX upgrade under a retainer.
We recommend re-testing the log pipeline whenever the Elasticsearch version or APISIX release changes.
Ongoing support, upgrades and continuous testing are available as a retainer, scoped separately from the initial build.