How the Graylog IPFIX input turns router flow exports into searchable logs
A team that wants traffic visibility from its routers and firewalls needs flow records to land in its own Graylog, searchable like any other log source. The Graylog IPFIX input receives IP Flow Information Export records from exporters on the network, and the design of the integration depends on how the exporter's templates reach that input. A template describes the layout of the fields in each flow record, so the input must learn the template before it can decode the data that follows it. Plan the design around that dependency before you configure anything.
The path runs from the exporter, which sends flow records over the network to the IPFIX input on one or more Graylog server nodes. The input turns each record into a message, the message passes through any processing pipelines and into a stream, and the stream writes it to an index set in the search backend, which is Graylog Data Node or a self-managed OpenSearch cluster in your environment. Each step can be checked: confirm that messages appear on the input's diagnostics, that the right stream receives them, and that the index set holds them. Use the documented IPFIX input as the reference for the input's settings, and confirm them against the Graylog version you run.
Because the input can only decode records once it holds the matching template, a sender that is not yet delivering templates produces messages Graylog cannot parse. Confirm with your exporter documentation how often templates are re-sent and which template IDs each exporter uses, and record that in the design. This is the decision that shapes the rest of the build, including how traffic reaches the input, which the next sections address.
Prerequisites before you open an IPFIX port
Four parallel checks decide whether IPFIX flow records will parse on the first try. Each one touches a different part of the path: the Graylog version and backend, the node that receives packets, the exporter that sends templates, and the storage that absorbs flow volume. Confirm all four before you open a UDP port to a router or firewall.
Supported Graylog and backend versions
Start from the current compatibility matrix, not from a version you remember. Graylog 7.0 to 7.2 pair with MongoDB 7.x up to 8.2.x, and the search backend must be Graylog Data Node or a supported OpenSearch release. Using OpenSearch 1.x is deprecated since Graylog 7.0 and is scheduled for removal in Graylog 8.0, so a flow pipeline built on it will need rework. Confirm the exact Graylog release you will deploy and check the IPFIX input page for that version before you build.
Receiving node and its reachability
Decide which Graylog node hosts the IPFIX input before exporters are configured. Because templates and data must reach the same input, plan one fixed listener address and port rather than a pool behind a load balancer. Confirm that the firewall between each exporter and that node allows the UDP port you choose, and that the node's host firewall accepts the same traffic. Record the address in your change ticket so exporter configuration and network rules stay aligned.
Exporter and template export settings
Each router or firewall must export IPFIX with templates it sends regularly enough for Graylog to hold them before data arrives. Check the template refresh interval, the observation domain ID and the transport on each device, and note which values differ between devices. Records that arrive before their template are not parsed, so a device restart can produce a gap in traffic data. Confirm these settings with the vendor documentation for each platform and test them on one device first.
Storage and journal capacity for flow volume
Flow records can add more daily volume than syslog from the same device, so size storage from measured flow rates rather than from device counts. The Graylog architecture guidance sizes log storage as daily ingest times retention days times 120 percent, and suggests a message journal of 3 to 5 times daily ingest on SSDs. Ask the device owner for a sampling rate and export interval, then estimate daily ingest from a sample. Set the index set retention before the input starts so old flow data does not fill the disk.
Why IPFIX templates break when flows are load balanced
IPFIX exporters send template records before the data they describe. A template defines how later data records are laid out: which fields appear, in what order, and how long each one is. The Graylog IPFIX input waits for the template from a source before it can process that source's messages. Data records that arrive without a matching template cannot be decoded into fields, so they are not parsed. This dependency is the reason the input behaves differently from a plain syslog listener, and it is the first thing to check when flow records show up as undecoded or missing fields.
A load balancer in front of several inputs can split one exporter's stream. Suppose, as an illustrative case, a router sends its IPFIX traffic to a virtual address that spreads packets across two Graylog inputs. One input may receive the template while the other receives the data records. The input that holds the data never sees the template it needs, so those records are not parsed, and the input that holds the template may see little or no data. The result looks like partial traffic data with gaps that change from one hour to the next, which makes the fault hard to trace back to the balancing layer. Load balancing that is based on source address or port can also move one exporter's packets between inputs over time.
Send flows from each exporter to a single Graylog IPFIX input. Point each router or firewall at one input and avoid balancing its UDP or TCP sessions across several listeners. Where capacity needs more than one input, distribute by exporter instead, so that each exporter's templates and data always reach the same input. Before you rely on this design, confirm the current behavior and any template timeout or restart handling in the Graylog IPFIX input documentation, and test a template refresh on each exporter model in your environment.
Configure the IPFIX input and confirm templates arrive
This sequence creates a dedicated IPFIX input, routes its flow data to its own index set and stream, and confirms that templates arrive before exporters send traffic. The result is a verified path from a router or firewall into searchable flow fields.
Choose the Graylog node and listener
Decide which Graylog node will host the IPFIX input and which port and protocol the exporters will use. Take the exact port, protocol and bind settings from the IPFIX input documentation, and confirm that the chosen node is reachable from the flow exporters through any firewall in between. Done when the planned address and port are written down and the firewall rule is confirmed by the network team.
Create the IPFIX input on that node
In the Graylog web interface, open the system inputs page and launch an IPFIX input on the chosen node. Enter the settings from the IPFIX input documentation and keep the input running after saving. Done when the input appears in the list of running inputs with a running state and no startup errors in the node logs.
Assign a dedicated index set
Create or select an index set reserved for flow records, separate from application and security logs. Flow volume can be high, so keeping it apart lets the team set its own retention and rotation without affecting other data. Done when the input's messages are shown as targeting that index set and the index set's rotation and retention settings have been reviewed against the storage plan.
Route flow data into a dedicated stream
Create a stream that matches the flow records, for example by the source input or an agreed field, and send it to the flow index set. Add any stream rules the team needs and confirm that messages from the IPFIX input are not also captured by unrelated streams. Done when the stream's preview or message count shows flow records arriving from the input.
Point one exporter at the input and watch for templates
Configure a single exporter on one router or firewall to send to the IPFIX input, and leave other exporters pointed at other inputs for now. Check the input's received message counts and the node logs for template records before expecting data messages. Graylog's IPFIX input waits for templates from the source before it can process data, so data messages are not expected until templates have arrived. Done when template records are shown as received and the input logs show no template-related errors.
Send a test flow and check parsed fields
Generate known test traffic through the exporter and search the flow index set for the time window. Confirm that flow fields such as source and destination addresses and ports, protocol and byte counts appear as separate, typed fields rather than one raw message. Done when a search returns the test flow with the expected field values populated.
Confirm the input behaves across exporters and restarts
Add the remaining exporters only after the single-exporter test passes, and check that each one's templates and data appear in the same stream. Restart the Graylog node in a maintenance window and confirm that the input comes back and that templates are received again. Done when every exporter's flows are searchable and the restart check is recorded in the change notes.
Which flow fields land where in Graylog and how to use them
This table maps the common IPFIX flow elements to the fields you will see in Graylog, the question each one answers in traffic analysis, and the check to run before you trust it. Field names depend on the template your exporter sends, so confirm each one in the running system before building dashboards on it.
| Exporter field or template element | Graylog field (confirm in your input) | Purpose in traffic analysis | Check to run |
|---|---|---|---|
| Source IPv4 or IPv6 address | Source address field from the template | Identifies which host or subnet initiated a flow | Compare against a known host and confirm the address family matches your network |
| Destination address | Destination address field from the template | Shows where traffic is going and supports top talker and outbound reviews | Check a flow to a known service returns the expected address |
| Source and destination ports | Source and destination port fields | Reveals services in use and supports top port views | Confirm ports are populated for TCP and UDP and blank or zero where the protocol has none |
| Protocol identifier | Protocol number or name field | Separates TCP, UDP and ICMP traffic for filtering and summaries | Verify the number maps to the expected protocol in your exporter documentation |
| Octet (byte) count | Byte count field from the template | Measures volume per flow for top talker and bandwidth charts | Compare totals for one flow against the exporter's own counters for the same interval |
| Packet count | Packet count field from the template | Shows flow size and helps spot small-packet floods or scans | Confirm the field is not swapped with the byte count in your template |
| Flow start and end timestamps | Timestamp fields, possibly converted to message time | Anchors flows to time windows and sets retention and search ranges | Check time zone and clock source on the exporter and Graylog nodes match |
| Ingress and egress interface identifiers | Interface index fields from the template | Supports traffic by interface and identifies which link carries a flow | Map interface indexes to names on each exporter before labeling dashboards |
Storage growth from IPFIX flow volume and the layout options
One IPFIX input on a single node, feeding a dedicated index set, keeps template handling simple because every exporter's templates and data arrive at the same place. All flow volume from every router and firewall is concentrated in one journal and one index set, so the cost lands on one node's disks and one retention policy.
- Exporter count: each router or firewall adds records, and the total is the sum across every exporter that points at the input.
- Flow sampling: sampled exporters send fewer records, so confirm the sampling rate on each device before estimating volume.
- Active conversations: busy links with many short-lived connections produce more records than a few long transfers of the same byte count.
- Operations cost: there is one node to patch and monitor, but it is also a single point of failure for flow ingestion unless you plan a failover path.
- Journal pressure: all flow traffic passes through one node's SSD-backed journal, so disk pressure there affects every exporter at once.
Check the node's journal and disk headroom against measured peak rates before any production device is pointed at the input.
Running one IPFIX input per site or exporter group, each writing to its own index set, lets you set retention and access per group and isolates a noisy exporter from the rest. The trade-off is more inputs, index sets and dashboards to maintain, and a stricter requirement that each exporter always reaches the input that holds its templates.
- Exporter grouping: assign each device to exactly one input so templates and data reach the same place, and document the mapping.
- Retention per group: separate index sets can keep different retention periods, which changes the storage total for each group.
- Operations cost: each new site adds an input, an index set and configuration to review, so ongoing effort grows with the number of groups.
- Node placement: inputs can be spread across nodes, but each input's load and journal still need headroom on the node that runs it. Confirm how your Graylog version places inputs in its documentation.
- Access boundaries: separate index sets allow different teams to search different traffic, if your role model needs that separation.
Confirm the exporter-to-input mapping in writing before the first production device sends flows, and test that a failed device does not leave an input waiting for templates it will never receive.
The Graylog system architecture guidance gives two planning rules for log storage: daily ingest multiplied by retention days multiplied by 120 percent, and a message journal sized at 3 to 5 times daily ingest on SSDs. Apply both to the flow volume of each index set, because retention and journal load differ by layout.
- Storage formula: daily ingest times retention days times 120 percent. For example (illustrative only), 50 GB per day kept for 30 days gives 50 × 30 × 1.2 = 1,800 GB.
- Journal size: 3 to 5 times daily ingest. In the same illustrative case, that is 150 to 250 GB of SSD journal capacity, sized per node that actually receives the flow data.
- Per-layout totals: a single index set carries one formula result, while per-site layouts produce one result per group, plus their own journal allowance.
- Retention drivers: longer retention for security or capacity planning multiplies every other term, so set retention per group only after the business case is agreed.
- Disk type: the guidance calls for SSDs for the journal, so budget for them on each node that handles flow data.
Replace the illustrative figures with your measured ingest before ordering disks or setting retention, and confirm the formula against the current architecture page for your version.
The architecture figures are planning rules, and they are only as accurate as the daily ingest number you feed them. Measure flow rates on your own exporters over a period that includes peak business hours, month-end activity and any scheduled batch jobs, then size from the peak window rather than from an average.
- Records per exporter: count flow records and bytes per second per device during the busiest windows, and record which exporters contribute most.
- Stored size after parsing: the indexed size depends on the fields you keep, so measure it on a test index set rather than assuming the raw size.
- Seasonal and incident peaks: month-end processing or a security incident can push flow counts well above normal levels.
- Configuration changes: record sampling changes, new exporters and template updates during the test, because each one changes the baseline.
- Repeat the measurement: new sites, applications or routing changes alter volume, so schedule a review each time the exporter inventory changes.
Ask for the measurement method in any quote or proposal, including the test period and the peak window used, before accepting sizing figures for disks or retention.
Securing flow exports across the network path
Flow records describe who talks to whom, on which ports, and how much data moves, so they are sensitive even though they carry no payload. The four controls below cover who may send flows in, who may read them after they are stored, how the Graylog nodes and backends holding them are protected, and how the export path itself is reviewed. Confirm each control against your own network design and the IPFIX input documentation before you rely on it.
Restrict which exporters can send
Only the routers and firewalls you have approved should reach the IPFIX input port. Anything else that can reach that port can inject flow records that look genuine on dashboards. Map each exporter's source address to the input, then block all other sources at the host firewall and at the network segment that carries the exports. Record the exporter list as part of the change process, so a new device is added deliberately. Confirm in the IPFIX input documentation whether the input can filter by source address or whether this control must sit entirely in the network.
Limit who reads flow data
Flow data reveals internal addressing, server placement and communication patterns that many users have no need to see. Use Graylog roles and streams so that the team responsible for network operations sees the flow stream, while application teams and general users do not. Review which roles hold search and dashboard permissions on that stream, and keep dashboards that expose broad traffic views limited to the same group. Document the access model before the first exporter is connected.
Protect Graylog nodes and backends
Flows are stored on Graylog server nodes, in MongoDB for configuration and in the search backend for the records themselves. Anyone who can reach the search backend directly can read every flow without passing Graylog's role checks. Keep the Data Node or OpenSearch cluster on a private network, allow access only from Graylog nodes, and restrict administrative access to MongoDB. Include these components in your hardening baseline and in your backup scope, since flow indices can hold a large share of stored data.
Review the export path across segments
Flow exports often cross VLANs, firewalls or sites before they reach Graylog, and each crossing is a point where records can be read, altered or dropped. Map every hop between each exporter and the IPFIX input, noting which segments, firewalls and WAN links carry the traffic. Decide whether exports from remote sites should travel over a private link or a protected tunnel, and confirm with the IPFIX input documentation which transport protections it supports. Where the input cannot provide them, the protection must come from the network path.
IPFIX troubleshooting: why flows arrive but fields stay unparsed
This table pairs common IPFIX symptoms in Graylog with their likely causes and the first check to run. Read it from the top down, since the template handling check resolves most parsing problems. The main takeaway is that the same input must receive both the template and the flow data.
| Symptom | Likely cause | Fix and check |
|---|---|---|
| Messages arrive but no data fields are parsed | The input has not received the exporter's template, so it cannot decode the flow records that follow | Confirm the template was received by the same IPFIX input that receives data. Check the input's message log for template handling, then confirm the exporter re-sends templates on its configured interval |
| The input receives nothing from one exporter | The exporter sends to a different address or port, a firewall blocks UDP to the input, or the exporter's template export settings are not enabled | Verify the exporter's destination, protocol and port against the input's bind address. Check the firewall path, then review the exporter's template export settings and confirm the input is running |
| Flows parse on some messages and fail on others when a load balancer or second input is in the path | Templates and data reach different inputs, so the input holding the data never gets the matching template | Send all flows from each exporter to a single IPFIX input, without load balancing across inputs. Confirm by checking that one input logs both template and data receipt for the same exporter |
| Storage fills faster than planned | Flow volume is far higher than the daily ingest estimate, often because every flow record becomes a message | Measure actual daily ingest per exporter in the index set, then compare it with the sizing formula in the system architecture guidance. Review the index rotation and retention settings so the oldest indices are removed on schedule |
| Message journal fills or backs up during flow spikes | Ingest bursts exceed what the journal and processing can absorb, leaving messages waiting before indexing | Check journal utilization and size on the Graylog nodes against the journal guidance for daily ingest. Confirm the journal sits on fast storage and that the search backend has capacity during peaks |
| Dashboards show gaps after a restart or exporter change | The input lost its template cache, or a new exporter uses a template ID the input has not yet received | Expect a short parsing gap until the exporter re-sends its templates. Confirm the template refresh interval on the exporter, then check that dashboard time ranges and index sets cover the gap period |
| Fields appear under generic names instead of flow field names | The template uses enterprise-specific or unrecognized information elements that the input does not map | Compare the template's field list with the input's supported fields in the Graylog IPFIX input documentation. Record the unmapped element IDs and decide whether to adjust the exporter template or extract the fields in a pipeline |
| Dashboards show traffic from unexpected sources | Several exporters share one input without source identification, so flows cannot be attributed to a device | Confirm the exporter's source address is retained in each message. Add a stream or pipeline rule per exporter so dashboards can filter by device before analyzing traffic |
IPFIX input questions buyers ask before hiring help
How much effort does an IPFIX setup in Graylog take?
The effort depends on how many exporters you have, how many sites send flows, how many template sets those devices use, and how you plan to store the data. A single router sending to one input is a small job. Many firewalls across several sites, with different exporter software versions and a need for long retention, is a larger one. Before estimating, count your exporters, list the flow sources by site, and confirm which templates each device sends. The storage design is often the largest unknown, because flow volume grows quickly and sizing has to account for daily ingest, retention days and a buffer. Graylog's architecture guidance suggests sizing log storage as daily ingest times retention days times 120 percent, so ask for that calculation in any quote.
Can our team set up IPFIX collection without outside help?
Yes, if you have people who already run Graylog and understand how your routers and firewalls export flow records. The IPFIX input itself is not difficult to configure. The hard parts are template handling, input placement behind any load balancer, and the storage plan. Graylog's IPFIX input documentation says the input waits for templates from the source before it can process messages, and that behind a load balancer one input may receive the template while another receives the data, so Graylog suggests sending to a single input. Teams without this experience often find that flows look fine in one test and then fail after a network change. A short review by an outside engineer can catch these issues early, and your team keeps ownership of the result.
What does our team need to provide for an IPFIX project?
The work needs three groups of people and access. First, someone with admin rights on the flow exporters, such as routers and firewalls, who can confirm export settings, template refresh intervals and the collector address. Second, a network owner who can approve changes to routing, firewall rules that allow UDP or other transport to the Graylog input, and any load balancer placement. Third, Graylog administrators who can create inputs, index sets and dashboards, and a storage owner who can confirm disk, the OpenSearch or Data Node capacity, and retention targets. Agree on a test window with the network team, because exporter changes can affect production traffic.
How long does an IPFIX project take?
Duration depends on scope: the number of exporters, whether you add new sites or firewall rules, how much storage work is needed, and how many dashboards you want. A pilot with one exporter and one input is much shorter than a multi-site rollout with separate retention tiers. Ask any provider for a timeline tied to a written scope, with named milestones for exporter testing, template verification, storage validation and handover. DeployOpen does not publish fixed durations; timelines are agreed in writing for each fixed-scope engagement.
Does the IPFIX input require a Graylog Enterprise license?
Confirm this before you plan the build. Check the IPFIX input documentation for the edition it is listed under, and compare it with the licensing details on Graylog's pricing page. Graylog Open is licensed under SSPL-1.0, which is not an OSI-approved open source license, and some inputs are restricted to paid editions. Ask your Graylog account contact or the vendor to confirm the edition in writing, because a license requirement changes the cost model and the design.
What does a fixed-scope IPFIX engagement include?
A fixed-scope engagement is agreed in writing before work starts and normally lists the exporters in scope, the Graylog inputs to build, the template and load-balancer approach, the storage and retention design, the dashboards for traffic analysis, and the handover materials. It should also state what is out of scope, such as changes to device firmware, network redesign or licensing purchases. Ask for the deliverables list, the acceptance tests for each exporter, the responsibilities split between your team and the provider, and what support is available after handover. Ongoing support, upgrades and continuous testing can be added as a retainer.
What should a quote for IPFIX work ask about?
Ask the provider to break the quote into the work items in the written scope, showing who performs each one and what you will receive. Good questions include how exporter testing is done, how template issues are diagnosed, how storage sizing is calculated, whether dashboard work is included, and what happens if a device needs a firmware change. Ask which costs recur, such as storage growth or retainer support, and which are one-time, such as the initial build. DeployOpen does not publish a price list, and a quote follows from the written scope.
How DeployOpen scopes and delivers a Graylog IPFIX integration
DeployOpen treats IPFIX collection as a design problem before it is a configuration task. We start from which exporters send flows, how they are load-balanced, and what traffic questions the team needs answered. The written scope then sets the input layout, storage plan, dashboards and handover before any build work begins.
How we scope
- Inventory every router and firewall that will export IPFIX, with its exporter address and template behavior noted
- Confirm whether any load balancer or relay sits between exporters and Graylog, since it affects template delivery
- Agree the flow volume estimate, retention period and storage sizing method before the build is priced in writing
How we build
- Configure a single IPFIX input endpoint per exporter path so templates and data reach the same input
- Set index sets and rotation for flow data separately from application logs to control storage growth
- Build traffic dashboards and saved searches for top talkers, protocols, ports and blocked sessions
How we hand over
- Test template arrival and message parsing from each exporter, recording the results against the agreed scope
- Document the input layout, exporter settings, index retention and the reason each design choice was made
- Walk your operations team through dashboard use, storage checks and what to do if a template is missing
How we support
- Engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts
- Ongoing support, upgrades and periodic checks of flow volume and storage are available as a retainer
- Confirm with DeployOpen how upgrades to newer Graylog releases will be tested against your IPFIX inputs
