Graylog aggregation alerts per user or host

When a Graylog aggregation alert per user or host is the right tool

Use a Filter and Aggregation event definition when the signal is a count or calculation over many messages, not a single message. A security or operations team that wants to know when one account fails logins repeatedly, or when one host sends an unusual volume of events, cannot get that from a plain filter. A plain filter fires on each matching message, which produces noise for a brute-force pattern and misses the pattern entirely if each attempt on its own looks ordinary. An aggregation definition runs a search query over a lookback window, groups the results by a field such as a username or source host, and raises an event only when the group crosses a condition you set. Event definitions in Graylog cover both real-time and historical monitoring, so the same logic can be tested against past data before it is trusted on live traffic.

Confirm the edition before you design around this feature. The supplied product information places event definitions in Graylog alerting, but the Graylog pricing page places the correlation engine and some detection content in Enterprise and Security, so check with Graylog which alert types and notification channels your license includes. Then decide whether the question really needs an aggregation. If the rule is a known bad value, a single field match, or a standard detection already published as a Sigma rule, a filter or a Sigma-based rule is simpler to test and explain. If the rule depends on how an entity behaves across several sources, such as a user active in both identity logs and cloud audit logs, a correlation approach or a dedicated SIEM workflow may fit better, and the aggregation logic is then one part of a larger design.

Before you build the first definition, write down the decision it supports. Name the field that identifies the entity (for example, the username or host field that your parsing actually populates), the threshold that would justify someone's attention, and the time window in which that threshold should be reached. Ask the team who will receive the event and whether they can act on it at the volume you expect. Illustrative example: a team might ask to be told when one account generates more than a set number of failed logins within a short window, and then check the lookback and grouping choices against a week of historical data before enabling notifications. Those choices determine whether the alert is useful, so they belong in the design review, not after go-live.

Choosing count, math or grouping for the condition

Count asks how many messages matching the search and stream arrived within the lookback window. It needs no numeric field, which makes it the most forgiving choice when log formats vary. It fits questions such as how many failed logins hit the directory in five minutes, or how many firewall denies a source generated. The main false-positive risk is that a count reflects volume, not intent: a batch job, a retry loop or a noisy health check can cross the threshold without any attacker involved.

Use case: overall spikes or bursts of one event type across the environment.
Field requirements: only the fields used in the search query must be present, typically a message field or an event identifier.
False-positive trade-off: low setup effort, but benign bursts from automation are common and need exclusions in the query or stream.
Threshold tuning: run the search over historical data before setting the number, so you see normal peaks.
Scope limit: a count across all hosts can hide a single host failing badly beneath a healthy average.

Before you enable the alert, check that your search returns the same count you expect when you run it manually over the same time range.

Grouping splits the aggregation so the threshold applies to each value of a field, such as each user, source IP or host, instead of to the whole stream. This is what makes an alert about one account or machine rather than about the environment as a whole. Grouping is only reliable when the grouping field exists and is spelled the same way in every message being aggregated. If a firewall writes the source address as src_ip and an application writes client_ip, the grouped alert will count each system separately and miss the pattern across both.

Use case: repeated failures per account, unusual activity per host, or one source touching many targets.
Field requirements: the group-by field must be present in every matching message, with the same name and format, such as a normalized username in lowercase.
False-positive trade-off: grouping by a field that is often empty creates a missing-value group, and shared accounts or NAT addresses can make one group look like many users.
Missing values: decide whether empty values should be excluded in the query so they do not form misleading groups.
Lost context: the event carries the grouping value, but many raw fields from individual messages are not included in the event.

When you add a group-by field, run the search and confirm that each source contributes to the expected groups before you trust the alert.

Event definition settings and when each fits

This table lists the settings you choose when building a Filter and Aggregation event definition, what each one controls, when to change it and the mistake teams make most often. Read it top to bottom while drafting a definition, then check each row against the search results you expect. The main takeaway is that the grouping fields and the lookback window decide who gets alerted and how often, so set them before tuning the threshold.

Setting	What it controls	When to change it	Common mistake
Search query	Which messages the aggregation counts, using Graylog search syntax on the chosen stream	Narrow it to one event type, such as failed logins, before adding grouping	Using a broad query, so unrelated messages inflate counts per user or host
Stream	Limits the event definition to messages routed to a specific stream	Use when the same message type arrives from several sources and only some should alert	Leaving it on all messages, which makes the query heavier and counts traffic you did not intend
Lookback time	How far back each evaluation searches for matching messages	Lengthen for slow patterns such as repeated access attempts over a longer window, shorten for bursts	Setting a lookback shorter than the interval, which can miss events between evaluations
Interval	How often Graylog runs the search and evaluates the condition	Shorten when response time matters, lengthen to reduce search load on the cluster	Choosing an interval that re-alerts on the same burst because the lookback overlaps repeatedly
Aggregation function	Whether Graylog counts messages or applies a count, math or field-based calculation to a numeric field	Switch from count to a field calculation when the value of a field, such as bytes or failures, matters more than volume	Choosing a function on a field that is missing or stored as text in some messages, which silently skips them
Grouping fields	Which fields split results into separate aggregates, such as user name or source host	Add a grouping field to evaluate each user or host on its own rather than all activity together	Grouping on a high-cardinality field or a field that is often empty, which produces many small or unattributed results
Threshold type and value	The comparison that decides when an aggregate becomes an event, such as greater than a set number	Adjust after reviewing historical results so routine activity for a user or host stays below the line	Setting the threshold from one busy day and never revisiting it as the user or host population changes
Event priority and carried fields	The triage priority and which message or aggregate fields move into the event	Carry the user, host and source fields forward when responders need them in notifications and the triage queue	Carrying no grouping fields forward, so the event shows a count but not which user or host caused it

Building a user or host threshold event definition step by step

This sequence takes a threshold question, such as too many failed logins per user or too many connections per host, from a validated search to a Filter and Aggregation event definition that notifies the right people and has been tested with a controlled event.

Confirm fields are extracted and consistent
Before writing any alert, check that the user and host fields you plan to group by exist on the messages in the target stream. Open a sample of recent messages in Graylog Search, expand a few results, and confirm the field names and value formats match across sources. If one source writes the user as an email address and another as a short account name, normalize them in a processing pipeline or extractor first. This step is done when the same field name appears on every message type you intend to count and the values look the same across sources.
Write and test the search query on historical data
Build the search query that selects the messages you want to count, such as failed authentication events, and scope it to the stream that carries them. Run it over a past time range where you already know the outcome, for example a period that contains a known batch of failed logins. Use the field list and the result count to confirm the query returns the expected messages and nothing from unrelated sources. This step is done when the query returns the count you expect for the known period and you can explain each match.
Create the Filter and Aggregation event definition
In the Graylog event definitions area, create a new event definition and choose the Filter and Aggregation type. Enter the same search query and select the stream you tested, then give the event a clear title and description that name the condition being detected. Save the definition in a disabled state if your process requires review before it runs. This step is done when the definition is saved and its query preview matches the results you saw in Search.
Set grouping by user or host
In the aggregation configuration, add the user or host field as the group-by key so that Graylog evaluates the count separately for each value rather than across all messages combined. Choose one key per definition where possible, because a combined key multiplies the number of groups and makes the alert harder to explain. Preview the aggregation and check that each group corresponds to a single user or host with a sensible count. This step is done when the preview shows one row per user or host with its own count.
Set lookback, interval and threshold
Set the lookback window, which is how far back each evaluation looks, and the interval, which is how often the definition runs. Then choose the aggregation function, such as a message count, and the threshold that triggers an event for a group. Use the historical search from step two to estimate whether the threshold fires on the known test period and stays quiet on normal traffic. Record the values and the reason for each in the definition description so the next reviewer understands the choice. This step is done when the preview shows the threshold crossed only for the groups you expected.
Configure the notification and the event fields
Attach the notification that matches the responder workflow, such as email for a first pilot, and confirm which notification types your license includes before you rely on them. Check that the event carries the grouped user or host value, because aggregation alerts lose the individual messages that produced the count. If you need the source address or the message that triggered the alert, add it through the event fields or plan a follow-up search link. This step is done when a test notification arrives and it names the user or host that crossed the threshold.
Validate with a controlled test event
Enable the definition and generate a controlled test event by sending a known number of test messages for one test user or host, using a non-production account. Confirm that the event fires once the threshold is crossed, that it does not fire below the threshold, and that the notification and event fields match what you expected. Then remove the test traffic and check that the event resolves or stops firing as designed. This step is done when the test run produces exactly one event at the expected count and you have documented the results.

The fields you lose when you aggregate

An aggregation event definition does not forward every message that matched its query. It produces one event per group, carrying the grouping keys you chose (such as a user name or host) and the aggregate result (such as a count or a computed value over a field). The full message text, the source IP of each individual event, and per-message details generally do not reach the notification. Plan on the assumption that a notification for a threshold breach on user_name will tell the responder who and how many, but not which specific requests produced the count. Confirm the exact field carry-over in your Graylog version by running a test definition against sample data and inspecting the resulting event fields before you wire up any notification.

Decide early which fields the event must carry for triage, and which can stay in the search. Grouping keys and the aggregate result are the minimum. If responders need a source address or a request path, either group by that field too, which splits the alert into more events, or accept that they will look it up. Each extra grouping key multiplies the number of events, so this trade-off affects alert volume as well as content. Keep the event title and description written so they name the group and the threshold, because those are what reach the responder first.

For triage, give responders a repeatable path back to the underlying messages. Save a search for the same stream, query and time window as the event definition, and include the grouping field as a filter so the responder lands on the matching messages within a few clicks. Record that search link or its query text in the event description. Check the time range carefully, since the lookback and interval of the definition determine which messages the count covered. Keep this runbook consistent with your retention settings, because messages that have already rotated out cannot be reviewed.

Where aggregation thresholds go wrong

Most aggregation alerts that fail do so before they reach a responder: the threshold has no baseline, the grouping field is not reliable, or the window misses the behavior that matters. The cards below cover the failure modes we see most often and the check to run for each one before the event definition goes live.

Thresholds set without a baseline

A count such as 50 failed logins in 5 minutes is only meaningful against what normal activity looks like for that user, host or service. Before setting a threshold, run the search over a period of ordinary activity and look at the distribution of counts per group. A threshold that sits below the normal peak for a busy batch host will fire every night. Record the baseline window and the reason for each number so the threshold can be reviewed later.

Grouping on missing or inconsistent fields

Grouping by user or host only works when every source writes that field the same way. One source may send username, another user, and a third a domain-qualified name such as DOMAIN\\alice. Messages without the field are dropped from the grouping, which can hide an attack entirely. Check field coverage across each stream in the search before grouping, and normalize names in a pipeline rule if the sources disagree.

Lookback too short for slow activity

A lookback of a few minutes catches bursts, but it misses activity spread across hours or days, such as password spraying that stays under a per-window limit. Match the lookback to the behavior you are trying to detect, and consider a second definition with a longer window for slow patterns. Confirm that the index retention in your deployment covers the longest window you plan to use, because older data may no longer be searchable.

Alert floods from shared hosts and service accounts

A service account that runs a nightly job, a load balancer health check or a shared jump host can exceed any per-user threshold on every run. Each group then creates its own event, and the responder queue fills with known, expected behavior. Identify shared identities before the rollout, exclude them with a documented filter or a separate definition, and assign an owner who reviews exclusions on a schedule.

Late-arriving logs and duplicate events

Logs from network devices, cloud APIs or buffered agents can arrive minutes after the event they describe. A window that has already been evaluated may then show too few messages, and a later evaluation may fire again for the same group. Compare the timestamp used by the message with the time it was indexed, add a delay that covers your slowest source, and confirm how repeated events are suppressed in your version before relying on the alert for escalation.

Historical and real-time evaluation differences

A definition that behaves correctly on live traffic may produce different results when run against historical data, because the search window, stream membership and index state at the time of evaluation can differ. Test each definition against a known replay period before enabling it, and record which results came from historical runs. Confirm the evaluation mode and its limits in the event definition documentation for the version you run.

Security and operational considerations for alert definitions

An aggregation alert is only as trustworthy as the people who can change it, the channels it reaches, and the messages behind it. The four areas below connect who controls thresholds, what leaves Graylog, how long evidence survives, and how you notice when an alert stops firing.

Control who edits thresholds

Event definitions hold the query, the stream, the lookback window and the threshold, so a change to any of them changes what the SOC sees. Restrict edit rights in Graylog's role model to a small group, and keep the reviewers who approve threshold changes separate from the people who make them. Before go-live, export the definition configuration and record who changed what and when, so a quiet raise from 10 failed logins to 500 can be traced later. Confirm how your Graylog edition records definition changes in its audit trail.

Protect the notification channel

A threshold alert is sent to a chat channel, a paging service or an inbox, and each of those destinations becomes a place where user names and host names are stored. Use dedicated integration credentials with the narrowest permissions the destination allows, and rotate them on a schedule your team can actually follow. Confirm that the webhook URLs and tokens are stored as secrets and not pasted into shared notes. Notifications for paging and chat need an Enterprise license, so check the edition before you design around them.

Limit fields that leave Graylog

An aggregation event carries the grouped field, such as user or source, plus the count or math result, and a notification template can add message text from the underlying logs. Decide which fields a responder needs and remove the rest from the notification body, because a Slack or email message is copied into more systems than the log platform itself. Review the template with the data owner for personal data and any log content that should not leave your boundary. Keep the detailed evidence in Graylog and link to it rather than copying it out.

Keep source messages long enough

When an aggregation alert fires, the investigation depends on the individual messages that produced the count, and those messages are only available while their index is retained. Set the index set retention so that the window you alert on and the window your investigators need both fit inside it, with margin for delayed reviews. Remember that retention deletes the oldest indices, so a short retention can remove the evidence before anyone opens the event. Compare your retention with any log retention duty your organization or sector has, and document the decision.

Monitor the definitions themselves

An event definition that references a renamed field, a stream that was moved, or a query that now returns errors can stop alerting without any visible failure. Schedule a recurring check that confirms each definition still runs, that its stream still receives messages, and that its last evaluation produced results as expected. Run a known test event through each critical definition at a set interval, and alert the platform team when the test does not appear. Ask your Graylog support or partner to confirm which health signals your edition exposes for event definitions.

Questions teams ask before building aggregation alerts

How much effort does a set of threshold alerts take to build?

Effort depends mostly on how many distinct conditions you need and how clean the fields behind them are. Each aggregation event definition needs a search query, a stream, a lookback window, an interval, a counting or math condition, and a grouping field. Writing one definition is quick once the fields are known, but the time goes into confirming which events matter, testing each threshold against real traffic, and wiring notifications. Ask any partner for a quote built from a written list of alert conditions, the number of log sources involved, and whether notifications are in scope. Build cost is largely one-time, while review of thresholds and definitions recurs as your environment changes.

Can our internal team build aggregation alerts without a partner?

Yes, an internal team with Graylog experience can build aggregation alerts alone, provided it has someone who understands the log sources and the security or operations question each alert answers. The skills needed are query writing, understanding of how Graylog groups messages by fields such as user or host, and disciplined testing. Teams without that experience usually lose time on field naming and on thresholds that fire constantly. A partner is most useful when you want a documented set of conditions reviewed against your environment, or when you need the work done alongside your staff so they can maintain it afterward.

What does the work need from our side?

The work needs three things from your side: log field quality, sample data and named notification owners. Field quality matters because an aggregation grouped by user or host is only as reliable as those fields are populated and consistently named across sources. Sample data from normal and abnormal periods lets you set thresholds from observed behavior instead of guesses. Notification owners are the people who receive each alert and act on it, and they should be agreed before build starts, so the alert routes to someone who can respond. Ask the vendor to list exactly which of these inputs they expect from you.

How long does tuning take?

Tuning is the stage where thresholds move from first guess to usable alert, and its length depends on how much your traffic varies by time of day, by host and by user. Expect several rounds: run the definition over historical data, review what it would have fired on, adjust the lookback window, interval and count, and check again. Teams with stable, well-understood sources tune faster. Teams with bursty batch jobs or shared service accounts need longer, because those patterns produce repeated false positives. Ask for the tuning approach and the review checkpoints in writing rather than a fixed duration, since no supplied fact gives a reliable figure.

Do we need a Graylog license for aggregation alerts?

Filter and Aggregation event definitions are part of the alerting feature set, and the license you need depends on the option you pick, not on the definition type alone. Confirm the exact edition for each option against the current Graylog pricing and product pages before you commit, because edition boundaries change between releases. Notification channels are a separate question: several notification types are documented as Enterprise features, so an alert that works on Graylog Open may still need a different delivery route. Ask your licensing contact to map each planned alert and each notification to an edition in writing.

How should we handle false positives?

Start by deciding what a false positive means for each alert, then fix the cause rather than raising the threshold until the noise disappears. Common causes include scheduled jobs, service accounts, health checks and shared hosts that produce large counts for legitimate reasons. Exclude those sources in the search query or the stream, group by a more specific field where that helps, and extend the lookback window if bursts are short and frequent. Record every change with the reason so reviewers can see why a threshold moved. Plan a recurring review of alert volume and outcomes, because a threshold that was correct at build time can become noisy as traffic grows.

What fields do we lose when we aggregate?

When Graylog aggregates messages, the result is a count or calculation per group, so individual message fields such as the full raw message, unrelated fields and per-event detail are not carried into the alert as they would be in a raw search. Only the grouping field and the computed value reliably appear in the event. Design the event so responders still have enough context, such as the user or host, the count and the time window, and check in a test that the fields you need survive. If a responder needs the underlying messages, plan a link or a saved search that reproduces the same query for the same window.

How DeployOpen scopes and delivers Graylog alert definitions

DeployOpen approaches Graylog aggregation alerts as a scoped piece of detection engineering inside your own Graylog deployment. We agree the search, the stream, the thresholds and the notification path in writing, build and test each event definition against your data, and hand over the documentation your team needs to tune and operate it.

How we scope

Inventory the log sources, streams and fields each aggregation rule depends on, such as user, host and source address
Agree in writing which event definitions, lookback windows, grouping keys and thresholds are in scope
Confirm which notification types your Graylog edition supports before any alert path is designed
Engagements are fixed-scope, with deliverables and timelines agreed before work starts

How we build

Write and test Filter and Aggregation event definitions against historical messages in your environment
Set lookback and interval values from your event volume, then check that counts and math on fields behave as expected
Document which fields are lost when a rule aggregates, and map the event fields that remain for triage
Review alert volume with your responders so thresholds are not left at untested defaults

How we hand over

Deliver a register of each event definition with its query, stream, grouping, threshold and owner
Provide test records showing which historical activity triggers each rule and which does not
Walk your operations and security staff through tuning steps, change control and rollback
Record open decisions, such as thresholds that need your risk owner's approval, in the handover pack

How we support

Ongoing support, upgrades and continuous testing are available as a retainer after handover
Re-test aggregation rules when Graylog upgrades change search, event or notification behavior
Review alert noise and threshold drift with your team on the agreed schedule
Confirm with your Graylog vendor which features and edition your target environment requires before planning changes

Graylog aggregation alerts with thresholds per user or host