Open source SIEM and log management

Connect the identity provider by forwarding its authentication and administrative events to the SIEM, so every console login and privilege change carries a user name, source address and result. The work is mostly log export configuration and parsing: the identity provider's audit events are mapped to normalized fields so detection rules can tell a successful login from a failed one. The data involved includes sign-in events, MFA challenges, failed attempts and admin actions. Confirm with your identity vendor or the open-source project which event types it emits, whether they are delivered by syslog, webhook or API polling, and whether user identifiers match those in your directory. If you run Keycloak for workforce single sign-on, confirm its event listener and log configuration for your version before you write parsing rules.

Endpoint and server collection is the largest share of the agent and parser work, because every operating system and application writes logs differently. The work covers deploying a collector or agent to each host, defining which Windows event channels, Linux audit and authentication logs, and application logs are shipped, and building parsers so fields are consistent. The data volume depends on the log levels you choose, so this is where ingest cost usually grows fastest. Confirm with your SIEM project which agents it supports, how they are managed centrally, how agent upgrades are handled, and how the platform behaves when a host goes silent. Ask for a list of hosts by class and the expected log sources per class before any sizing discussion.

Cloud audit logs are usually pulled from the provider's native logging service into a storage bucket or message queue, then read by the SIEM through a connector or forwarder. The work includes granting read-only access for the collector, setting retention on the provider side, and mapping control-plane events such as role assignments, key creation and storage policy changes. Cloud logs often arrive in batches and can be delayed, so detection timing must be tested rather than assumed. Confirm with your cloud provider which audit streams are available, whether they can be exported without extra cost, and what the export format is. Confirm with the SIEM project which cloud connectors it ships and which ones your team would need to build or maintain.

Network sensors such as Suricata for intrusion detection and Zeek for network analysis produce structured alerts and connection logs that the SIEM ingests alongside host data. The work involves placing sensors at chosen network points, forwarding their output in a consistent format, and deciding which fields to keep, since full packet capture and detailed connection logs are heavy on storage. Sensor placement and tuning are covered in the network security monitoring topic. Confirm with your platform team which traffic points are mirrored or tapped, whether the sensors can be run inside your network segments, and what capacity your links support. If you use Security Onion, confirm whether its bundled sensors match your collection design before you add separate forwarders.

Ticketing and on-call integrations send alerts from the SIEM into your service desk and paging tools, so that each validated alert creates a tracked item with an owner and a severity. The work involves creating webhook or API connections, mapping alert fields to ticket fields, deduplicating repeated alerts, and defining which severities page a person at night. Poor mapping is a common cause of ticket floods, so the routing rules are worth designing before go-live. Confirm with your ticketing vendor the rate limits and the format its API expects, and confirm with your on-call provider how escalation and acknowledgment are passed back. Ask who owns the routing rules after launch, because they change as detection content changes.

Threat intelligence feeds reach the SIEM as indicator lists such as IP addresses, domains and file hashes, which the SIEM matches against logs and turns into alerts. The work is to export indicators from MISP or OpenCTI on a schedule, convert them to the format your SIEM expects, set expiry and confidence rules, and handle false positives from shared sources. Indicator volume grows quickly, so filtering matters for both performance and analyst time. Confirm with your threat intelligence team which feeds are licensed for your use, and check the license of your chosen platform edition before you connect it. The ongoing curation effort is covered in the threat intelligence topic.

A SOAR platform such as Shuffle connects to the SIEM through its API to pull alerts, run enrichment steps, and push results or case updates back. The work includes creating a service account with read and limited write permissions, building the first playbooks, and testing that each action can be audited. Start with a small number of low-risk playbooks so the connection is proven before automated response is enabled. Confirm with the SOAR project which connectors it provides for your SIEM and ticketing tools, how credentials are stored, and what happens to playbooks when a connector version changes. The automation scope and maintenance model are covered in the SOAR topic.

Effort depends less on the software license, which is free for the core platforms, and more on the volume of events you ingest, how many log sources you onboard, and how much detection content you maintain. Ongoing work falls into four areas: keeping the platform and agents patched, tuning rules to reduce false positives, managing storage and retention, and responding to alerts. Each new source adds parsing and testing work, and each new detection needs review after go-live. Ask any partner to split its estimate into one-time build work and recurring operating work, and to state which sources and retention periods the estimate assumes.

A small team can run a self-hosted SIEM if it has someone who understands Linux administration, log parsing and alert triage, and if the scope stays narrow at first. The risk is less about the platform and more about coverage gaps: nobody watches alerts overnight, or nobody updates parsers when a log format changes. Teams that succeed usually name a platform owner, a triage rotation and a change process before onboarding sources. Before you decide, confirm who will handle patching, backups and capacity reviews, and whether that work competes with existing duties.

The client typically provides the hosting environment, network paths from log sources, service accounts or agent deployment rights, and named owners for each log source and for alert decisions. You also need a written list of the systems in scope, their data classification and the retention periods that compliance or legal teams require. Access to change windows, identity systems for administrator login, and a decision on where alert data is stored all affect the schedule. A partner should return a written list of these prerequisites before work begins, so gaps show up in scoping rather than mid-build.

Duration is set in the written scope, not by a general rule. The main drivers are the number and type of log sources, whether network sensors are included, how many detections must be built and tested, the state of the existing hosting and identity environment, and how quickly the client can approve changes and assign owners. Retention design and integration with ticketing or a SOAR platform add time when they are in scope. Ask for a phased plan that separates collection, correlation and alerting, and retention, with a clear list of what each phase delivers.

Choose Wazuh when the main need is host-based detection, file integrity monitoring and log collection from endpoints and servers, with its core repository under GPL version 2. Choose Security Onion when network visibility and threat hunting matter most, since it bundles Suricata, Zeek, osquery and the Elastic stack and is licensed under the Elastic License 2.0. Many organizations use both for different layers, but that adds operating surface. Confirm the license terms against your distribution and redistribution plans, and test each platform against a sample of your real log sources before deciding.

Open source removes the license fee for the core platform, but it moves cost into infrastructure, engineering time, support and content maintenance. Commercial SIEM pricing often scales with ingest or event volume and includes vendor support, managed content and some built-in detections, which you must build or source yourself. The comparison should use the same log sources, retention periods and availability targets for both options. Ask each vendor and partner for pricing tied to a written volume and retention assumption, and include staff time for tuning and on-call coverage in your model.