Shuffle Detection Manager for Sigma rules

Log forwarding into Tenzir pipelines decides what a Sigma rule can see, so map the full data path before writing any rule. Events leave each log source, pass through a Tenzir pipeline that receives, parses and reshapes them, and reach the point where the Detection Manager evaluates rules and Shuffle runs the matching workflow. On one page, record the sending agent or collector, the transport between it and the pipeline, each pipeline output, and the Shuffle workflow that consumes it. Confirm in the beta how events enter Shuffle, whether by webhook or by a workflow that pulls from a store, because the answer determines which components must be reachable and where failures appear. Shuffle's backend listens on port 5001 and Orborus starts a worker for each workflow execution, so an execution can fail on the runtime side even when the pipeline delivered the event.

Four questions should be answered before the first pipeline is written. First, which log sources are in scope, since each source adds field names, volume and an owner who can change them. Second, what normalization happens before matching: Sigma rules refer to fields by name, so the pipeline must map source fields to the names the rules expect. Decide whether that mapping lives in the pipeline, in the rule or in a shared field map. Third, where events are stored after forwarding: in Tenzir outputs, in the OpenSearch document store that Shuffle uses, in the SIEM, or in several places, each with its own retention and access controls. Fourth, who owns each pipeline, meaning who approves changes, reviews failed deliveries and signs off on schema updates. Write the answers down, because each one changes the test plan.

Three failure modes matter most. Dropped events happen when a pipeline, network path or workflow trigger rejects or times out on a payload and nothing records the loss. Schema drift happens when a source renames or adds a field, so a rule that matched last week stops matching while every component still reports healthy. Duplicate forwarding happens when a retry or a second route sends one event twice, producing repeat alerts. Shuffle's pricing article meters app runs, which are actions that actually execute, so duplicates that start actions also raise cost on paid tiers. In a test environment, confirm four things: how Tenzir behaves when the receiving endpoint is unavailable (retry, buffer or discard), whether each event carries an identifier that Shuffle can use to deduplicate, how a missing field appears to a rule, and how Shuffle handles a malformed payload. Record what you observe before relying on any of it.

Forwarding into Tenzir does not replace the SIEM. Keep the SIEM as the system of record for alerts that analysts already work, and treat Tenzir copies as the input to Sigma matching and Shuffle workflows unless your team decides otherwise. That split creates two retention policies, two access models and two places where a missing event must be explained. Decide which store answers an auditor's question about a given day of logs, and test that answer end to end. On the Shuffle side, confirm that the OpenSearch cluster holding workflow data is sized, backed up and monitored on the same schedule as your other security data, since the configuration docs recommend an OpenSearch cluster for high availability. Finally, agree on a change process so that a source change, a field map update and a rule update are reviewed together.

Once Sigma rules are live in the Shuffle Detection Manager, the recurring work falls into five tasks: tracking Shuffle releases, testing rule changes, reviewing false positives, backing up the encryption secret and monitoring runtime health. Each task needs a named owner before go-live, because none of them is finished by a one-time configuration. Write the owners into a runbook that also records who approves a rule change, who may schedule a Shuffle upgrade and who responds when a workflow stops running outside business hours. The split described below is a starting point your team can adjust.

Release tracking comes first because it decides when the platform changes at all. The Shuffle docs recommend the latest Docker tag for stable releases and reserve the nightly tag for experimental changes, so production should run pinned stable tags, and nightly builds belong only in a non-production instance. The most recent release cited for this topic is v2.3.0, published on GitHub on September 28, 2026, but confirm the current tag on the Shuffle releases page when you read this. Assign tracking to the platform owner, who reviews release notes on a schedule your change process sets. Security engineering should approve each upgrade, and the platform owner should note that Shuffle stopped supporting versions older than v2.1.1 after January 31, 2026.

Every rule change should be tested in a non-production Shuffle instance before it reaches the detection workflow. The test should replay sample events that the rule is expected to match and events it should ignore, and detection engineers should attach that replay evidence to the change record. False positive review is a separate, continuing task. Assign it to the SOC analysts who close alerts, with a defined path for sending a noisy rule back to engineering along with example events, the field values that triggered it and the benign source behind them. Ask your implementer whether the Detection Manager beta exposes per-rule alert counts and records suppression changes, because that determines how much of this review can happen inside the interface.

The SHUFFLE_ENCRYPTION_MODIFIER secret needs the most careful ownership. Shuffle uses it to encrypt saved app credentials and API keys in OpenSearch with AES-256, and the docs warn that losing or changing it breaks all saved app authentications. Store a copy in your approved secrets store, restrict access to the platform owner and one named backup holder, and test restoration on the non-production instance to confirm the copy works. Monitoring covers Orborus, which polls the backend for jobs and starts a worker for each workflow execution. Watch whether Orborus nodes stay connected, whether jobs queue without starting, and whether a second Orborus node is available if your design follows the high-availability guidance in the docs. Confirm with your Tenzir pipeline vendor or implementer which pipeline health signals they expose.

Decide how coverage works before the first incident. An internal rota suits teams that already run on-call for their SIEM and can absorb a few hours of pipeline checks each week, provided the runbook names the people and the escalation path. A retainer suits teams that want upgrade testing, Orborus and backend monitoring, secret restoration tests and rule change support handled by a partner under a written scope. Either way, ask any provider to state in writing which tasks they perform, which they only advise on, how alerts about stalled workflows reach them and how an in-flight rule change is handled during an upgrade. DeployOpen offers ongoing support and upgrades as retainers, and the terms follow from the written scope agreed before work starts.