Shuffle playbook and custom app development

What this service covers for Shuffle

This service builds the workflows, which Shuffle calls playbooks, and the apps those workflows call. Apps can be generated from OpenAPI specifications in the App Creator or written in Python with the Shuffle app SDK, and each app runs as a versioned Docker container. The work also covers testing of each workflow and app, version control of the content, and documentation so your analysts and engineers can change it after handover.

Most engagements start from a trigger. Common ones are a new SIEM or EDR integration that needs response actions, a manual analyst process such as enrichment, ticket creation or containment steps that is repeated often enough to automate, and a homegrown internal tool that security operations must query or act on. Before any build starts, list each trigger, the systems it touches and the people who approve its actions. That list sets the scope and the order of work.

In-house or partner is mainly a question of skills and capacity. Teams with Python and API experience can build simple playbooks themselves, especially if they already know the Shuffle editor and their tools' APIs. A partner usually adds the most value on apps for internal systems that need authentication design, on test harnesses that replay sample events before production, and on handover material that lets your own staff maintain the content. Ask for a written list of which items you keep and which the partner delivers.

What the work includes and what you receive

The engagement breaks into work packages that run from process mapping through to documentation. Each card below names one package and what to check before accepting it.

Process mapping per playbook

Each in-scope playbook is mapped step by step before any Shuffle workflow is built. The map records triggers, decision points, data each step needs, tools called and the people who approve actions. Check that the agreed scope lists every playbook by name and that each has a written owner.

Workflow build in Shuffle

Each mapped playbook is built as a Shuffle workflow with its branches, error paths and conditional actions. Workflows are built against the agreed app versions and reviewed against the process map. Ask how manual approval steps and failure notifications are handled before build starts.

Custom apps from OpenAPI or Python

Where no suitable Shuffle app exists, a custom app is created from an OpenAPI specification in the App Creator or written in Python with the Shuffle app SDK. Each app is packaged as a versioned Docker container. Confirm which tools need custom apps and whether the vendor API specifications are available.

Credential and authentication setup

Credentials for each app are stored in Shuffle's encrypted credential store, not hard-coded in workflow logic or exported files. Each integration uses a dedicated, least-privilege API key or account. Confirm who creates those accounts and who can rotate them, because changing the encryption secret breaks saved authentications.

Test runs against non-production systems

Workflows and apps are exercised against non-production systems or sandbox tenants before any production trigger is enabled. Test records capture inputs, expected and actual outputs, and any defects fixed. Ask which test systems are available to the project and who approves moving a workflow to production.

Versioning of apps and workflows

Workflow exports and app source are kept under version control, and each deployed app is tied to a container version. Changes go through a documented review before they reach production. Confirm which repository and branching model your team uses so the handover fits existing practice.

Runbook documentation

Runbooks describe each playbook's purpose, triggers, dependencies, failure modes and the steps an operator takes when a run fails. They are written for the team that will maintain the workflows after handover. Ask who reviews the runbooks and how often they are updated.

Handover deliverables

At handover you receive exported workflows, app source with container versions, test records, a documented list of apps with their owners, and a short maintenance guide. The scope is fixed and agreed in writing before work starts, so confirm the list of deliverables matches what your team needs to operate the workflows.

What drives cost and duration

This table lists the main drivers of Shuffle build effort and metered usage, why each one matters, and how to keep it manageable.

Driver	Why it matters	How to keep it down
Playbooks and branches	Each playbook and branch needs its own logic, error handling and tests, so the count scales build and test effort.	Reuse shared subflows and keep branches to the decisions that change outcomes.
Target systems and interfaces	Systems with OpenAPI descriptions are faster to wrap than systems needing custom authentication or SDK work.	Inventory interfaces early and prioritize systems with OpenAPI descriptions and standard authentication.
Number of custom apps	Each custom app is built, versioned and tested as its own container, which adds design and maintenance work.	Check whether an existing app covers the system first, then limit new apps to actual gaps.
Approval and human-in-the-loop steps	Approval gates add decision logic, timeouts and handoffs between analysts and the workflow that must be tested.	Limit gates to actions with material impact and name one owner and escalation path for each.
Test environments available	Without a non-production instance, testing runs against live systems, which slows sign-off and adds risk.	Provision a staging Shuffle instance and sandboxed target systems before the build starts.
Change control and documentation depth	Formal change approval and written runbooks add review cycles and documentation effort for every release.	Agree a documentation standard early and keep workflows and apps under version control.
App-run volume per workflow	Pricing is based on app runs for executed actions. Triggers and skipped actions do not count toward volume.	Filter events before they reach action steps and model expected event volume from your own data.
Test executions	Test runs that execute actions may consume app runs, so test volume belongs in the plan.	Confirm metering with Shuffle before planning test volume, and limit test runs to the paths that need them.

Project phases and outputs

The sequence moves one security process at a time from inventory to a documented, tested and monitored Shuffle workflow that your team can maintain.

Inventory processes and current automations
The team maps each security process to be automated, the systems and credentials it touches, its owner, and the Shuffle version your environment runs. The build targets that version. Output: a signed-off inventory listing each workflow, trigger, connected tool and owner.
Design apps from specifications or SDK plans
Each custom app is designed from an OpenAPI specification for the App Creator or from a plan for a Python app built with the Shuffle app SDK. The design fixes actions, inputs, outputs, authentication method and error handling. Output: an approved design record per app that the build can be checked against.
Build workflows in a non-production instance
Workflows and apps are built in a non-production Shuffle instance that runs the same version as your production environment, so behavior reflects the target release. Apps are packaged as versioned Docker containers and kept in source control. The nightly tag is reserved for experimental changes and stays out of client builds. Check: the instance version and the repository showing each app version and workflow export.
Test workflows and record results
Each workflow runs against defined test inputs, including failure paths such as unreachable APIs, expired credentials and empty results. Outcomes are recorded per test case with inputs, outputs and run identifiers. Output: a test record your team can rerun after any later change to a workflow or app.
Roll out in stages with run monitoring
Workflows move to production in stages, beginning with a limited set of triggers and expanding once the first stage is stable. App runs and failures are monitored, and each failure is classified as a workflow, app or connectivity fault. Output: a rollout log showing each stage, its monitoring results and the approval to proceed.
Hand over code, runbooks and documentation
Your team receives exported workflows, app source, version tags, notes on how encryption secrets and app credentials are handled, runbooks for common failures and the test record. Sessions cover how to update an app after a Shuffle upgrade. Output: a handover checklist signed by your owner, with repository and documentation locations confirmed.

What stays in-house and what to outsource

This table separates the Shuffle work your security team should keep from the build work a partner can take, with the skills each part needs.

Work area	Usually kept in-house	Often outsourced	Skills needed
Playbook ownership	Deciding which playbooks exist, their priority, their owner and their retirement date	Drafting the first inventory of existing scripts and cron jobs as a proposed playbook catalog	Security operations knowledge, incident process ownership, change approval practice
Business rules and approvals	Defining escalation paths, approval thresholds, containment limits and who may authorize blocking actions	Encoding approved rules as workflow branches with approval steps and audit output	Security operations knowledge, Shuffle workflow editing, clear written decision records
Analyst feedback and tuning	Reviewing false positives, changing rule logic and deciding when a workflow is trusted to act alone	Adding logging and metrics to workflows so tuning data is easy to pull	Security operations knowledge, reading execution logs, basic Python for small script changes
On-call response	Being the first responder when a workflow fails, stalls or takes a destructive action	Writing runbooks and health checks for Orborus, workers and app containers	Docker, incident response practice, reading container logs, on-call rota ownership
OpenAPI app creation	Reviewing the generated app against the vendor API and approving its permissions	Building the app in the App Creator from the vendor OpenAPI specification and mapping authentication	API design reading, OpenAPI specification structure, Shuffle App Creator use
SDK-based apps for internal systems	Confirming the internal API owner, data sensitivity and who may call the system	Writing Python apps with the Shuffle app SDK for internal ticketing, asset or directory systems	Python, Docker image builds, internal API authentication, secrets handling
Secrets and app credentials	Owning the encryption secret, rotation schedule and least-privilege scope of each integration key	Configuring saved app credentials so secrets never sit in workflow text or logs	Secrets handling, understanding of the encryption modifier, access review practice
Test harness design	Supplying sample alerts, expected outcomes and sign-off criteria for each playbook	Building repeatable test workflows and mock endpoints that replay sample events	Python, API design reading, test data handling, Docker for local mock services
Versioning and documentation	Approving release notes, change windows and the version each runtime uses	Setting up app version tags, image naming and a workflow documentation template	Git practice, Docker image tagging, technical writing for operators

What usually goes wrong and how to prevent it

Each card names a failure mode in Shuffle playbook and app work and the control that prevents it.

Apps without a named owner

Shuffle apps wrap vendor APIs, so a change to an endpoint, field or authentication method can stop a playbook without any error in Shuffle itself. The failure often surfaces later, when an analyst finds a stalled case. Assign each app a named owner, record the vendor API version it targets, and transfer ownership in writing with the OpenAPI specification and test cases when staff change roles.

Hardcoded credentials copied between environments

Playbooks exported from a test instance often carry API keys typed directly into action fields. Importing them into production exposes those keys and can point production workflows at test systems. Store each credential as a Shuffle-managed authentication, reference it by name, and check every imported workflow for literal secrets and environment-specific hostnames before enabling it.

Lost or changed encryption modifier

Shuffle encrypts saved app authentications with AES-256 using the SHUFFLE_ENCRYPTION_MODIFIER secret. If that value is lost, changed carelessly or differs from the original on a restored backend, every saved app login stops working at once. Keep the value in your secrets vault, document it in the recovery runbook, and test restoring a backup into a separate environment before go-live.

Untested branches on rare alerts

Playbook branches for uncommon alert types, such as a malware verdict on a server or a missing field in a SIEM payload, often never run during normal testing. They fail on the first real incident that reaches them. Build test inputs for each branch, replay recorded alerts into a staging instance, and review conditions for missing or null fields.

Unbounded app runs from loops and triggers

A loop over a large list, or a webhook trigger fed by a noisy detection rule, can start a very large number of app executions in a short time. Shuffle licensing counts app runs that actually execute, and workers and OpenSearch carry the load. Cap loop iterations, add deduplication or throttling at the trigger, and alert on run volume per workflow.

Apps built on unsupported versions

Shuffle stopped supporting versions older than v2.1.1 after January 31, 2026. Apps built against an older release can depend on behavior that has since changed, and older servers miss later fixes and edition limit enforcement. Confirm your running version, keep frontend, backend and Orborus images aligned, and test each app on the target version before upgrading production.

How the work differs by starting point

Teams starting without any automation spend the most effort before any workflow code exists. The work centers on choosing the first processes, mapping each step to a tool and deciding which integrations need custom apps.

Use case selection: agree on a short list of processes, such as alert enrichment or phishing triage (illustrative), before building workflows.
App scope: most integrations are built from OpenAPI specifications or the Python SDK, so each tool's API access must be confirmed early.
Credential ownership: every saved app credential needs a named owner and a rotation plan, since Shuffle stores them encrypted in OpenSearch.
Expected outputs: with no existing runbooks, expected results must be written down so each workflow has something to test against.

Ask whether the quote covers process workshops or only the workflow and app build.

What you need to provide before work starts

Most delays in a Shuffle playbook and custom app project come from inputs the client has not yet confirmed, not from the build itself. Before work starts, name one owner for each playbook and one owner for each target system. The owner answers questions about business logic, approves test outcomes and decides when a playbook can run against live tools. Without named owners, decisions stall in review and the team builds against assumptions that surface late in testing.

For each system a custom app will wrap, provide API documentation or an OpenAPI specification, plus sample alerts or events that show the real payload shape. An OpenAPI file lets the team generate an app in the App Creator and check endpoints before writing logic. Sample events drive the test cases for branching, field mapping and error handling. If the specification is missing or outdated, the team reverse-engineers endpoints from traffic or vendor pages, which adds rework and makes the app harder to maintain later.

Provide a test account and a non-production target for each integration, and confirm that its permissions match the least privilege the playbook needs. Shuffle workflows that execute actions can change records in a SIEM, EDR or ticketing system, so testing against production is a risk to plan for, not a shortcut. Provide a running Shuffle instance or an agreed deployment target, and confirm which Orborus runtime location will execute each workflow, since the apps must reach that network. Confirm the secrets store or approved method for supplying API keys, because Shuffle stores app credentials encrypted using the SHUFFLE_ENCRYPTION_MODIFIER secret, and losing that secret breaks every saved authentication.

Agree change approval rules before the first build: who approves a workflow change, who signs off a release to production, and how emergency fixes are handled. Also decide who maintains apps after handover. Choose between your own engineers, a support retainer, or a mix, and name the people. Apps are versioned Docker containers, so upkeep includes rebuilding after dependency changes and updating them when a target API changes. Ask the partner to confirm in writing which of these decisions must be in place before each build milestone starts.

Frequently asked questions

Can our own team build Shuffle playbooks and custom apps?

Yes, if your engineers know the target systems' APIs, Docker and Shuffle's workflow model well enough to own the result after handover. Teams often underestimate the same few areas: designing trigger and decision logic, handling credentials with the encryption secret in mind, testing against a non-production system, and documenting each app version. Check whether at least two people can make builds and changes, because a single maintainer is a common single point of failure.

How are cost, effort and duration determined for a Shuffle build?

Scope drives all three. The number of playbooks, apps, target systems, test depth and handover material sets the effort. Engagements are fixed-scope, so the written quote states deliverables, timelines and what your staff must provide before work starts. One-time costs cover design, build, testing and handover. Recurring costs cover app upkeep and any Shuffle licensing, which is priced on app runs, with users, apps and workflows unlimited. Shuffle says all features are in the open-source edition, so confirm which tier you need. DeployOpen does not publish a price list.

Do OpenAPI and Python Shuffle apps differ in maintenance?

Both kinds run as versioned Docker containers, so packaging and upgrade steps are shared. The difference is what changes over time. Apps built from OpenAPI specifications in the App Creator depend on the accuracy of the specification and on the upstream API's versioning. Python apps carry your own code, libraries and tests, so someone must patch dependencies and review changes. Choose the approach based on who will maintain the code and how stable the target API is.

Do test runs count toward Shuffle app runs?

Yes, actions that execute during testing are app runs under the metric. Shuffle defines app runs as actions that actually execute in a workflow, and triggers and skipped actions do not count. Test plans should estimate how many executions a test cycle produces, especially for playbooks that poll APIs or process many alerts. Confirm with Shuffle how test or non-production instances are licensed for your tier, and record expected test volumes in the written scope.

How do Shuffle upgrades affect custom apps and playbooks?

Apps run as versioned containers, so a change to the frontend, backend or worker can change app behavior in your environment. Shuffle stopped supporting versions older than v2.1.1 after January 31, 2026, and published v2.3.0 on September 28, 2026. Run every custom app and playbook on a staging instance before each upgrade, keep frontend, backend and worker images aligned, and confirm the SHUFFLE_ENCRYPTION_MODIFIER value is unchanged, since changing it breaks saved app authentications.

What documentation should we receive at handover?

Handover should give your team what it needs to run the work without the original builders. Typical items include OpenAPI specifications or Python source in version control, a version log for each app, test cases and results, and a playbook inventory listing triggers, dependencies and owners. Also include credential names and rotation steps without secret values, a runbook for common failures, and a record of who holds the encryption secret. Agree the list in the written scope so acceptance can be checked.

How DeployOpen delivers this service

DeployOpen scopes, builds, tests and documents Shuffle playbooks and custom apps inside infrastructure you control, so your team can maintain what we deliver. Each engagement starts from a written scope and ends with handover material your own engineers can use.

How we scope

Each engagement starts with a written scope listing the playbooks, triggers and apps to be built for your security processes.
We agree which tools each workflow calls, which API credentials it needs and where the Shuffle runtime will execute.
Custom apps are scoped from an OpenAPI specification or a Python SDK plan, with the endpoints and actions named up front.
Engagements are fixed-scope, so deliverables and timelines are agreed in writing before build work begins.

How we build

Workflows are built in Shuffle and custom apps are packaged as versioned Docker containers that your environment can pull.
Each playbook is tested against defined inputs, including failure paths, skipped actions and rerun behavior, before handover.
Saved app credentials are stored using Shuffle's encryption secret, and we document how that secret is generated and kept.
Changes are tracked with versioning so every playbook and app revision can be traced and rolled back if needed.

How we hand over

You receive playbook and app documentation covering purpose, inputs, outputs, dependencies and the owner of each workflow.
We walk your engineers through the build, the test cases and the steps to update an app after an API or Shuffle upgrade.
Handover includes the version records and container build steps your team needs to maintain apps without DeployOpen.
Agreed deliverables are listed in the written scope, so you can confirm completion against that list before acceptance.

How we support

Ongoing support, upgrades and testing of playbooks and apps are available as a retainer after handover.
A retainer can cover updating apps when upstream APIs change and retesting workflows after Shuffle version updates.
Confirm with us which support level, response expectations and review cadence apply before a retainer starts.
Your team keeps ownership of the environment, so support scope and access are defined in writing from the start.