Shuffle high availability guide

Why Shuffle high availability is harder than adding a second server

When one Shuffle server goes down, a playbook that isolates a host or opens a ticket can stop halfway, and the analysts waiting on it may not notice until the case stalls. Shuffle high availability means no single node, database or runtime holds state or work that cannot be picked up elsewhere. Adding a second server covers only the most visible part of that goal, because the platform has several layers, and each one keeps state in a different place. On the server side, Shuffle runs a frontend, a Go backend API listening on port 5001, and an OpenSearch document database. The Shuffle configuration docs recommend an OpenSearch cluster, the backend and frontend scaled across all servers, optional Memcached for a shared cache, and NFS for shared files when several backends run at once. On the runtime side, Orborus polls the backend for jobs, starts a worker for each workflow execution, and that worker starts the app containers the workflow calls. The docs recommend more than one Orborus node so workflows keep running if one fails. Each layer needs its own answer to the question of what happens when it disappears. Four areas make this harder than duplicating a server. The first is shared state: OpenSearch must form a healthy cluster, and every backend must read the same files and cache entries. The second is consistent secrets. The SHUFFLE_ENCRYPTION_MODIFIER value encrypts saved app credentials and API keys, and the docs warn that losing or changing it breaks all saved app authentications, so every node must hold the identical value from the start. The third is in-flight executions: a worker that dies mid-run leaves a partial record, and actions with side effects, such as blocking an address, must not run twice. Confirm with the vendor how interrupted executions are marked and resumed. The fourth is proof: failover that has never been tested is a hope, not a control. High availability is worth the added complexity when playbooks gate incident response, such as containment steps that analysts must wait on, or when scheduled enrichment cannot miss runs because a missed run leaves cases without data. It is usually overkill when Shuffle automates notifications or low-stakes enrichment, when a short restart inside an agreed maintenance window is acceptable, or when the team has no capacity to operate a multi-node OpenSearch cluster. Before deciding, ask which workflows would stop, how long your team could work without them, and who will run the failover test each quarter.

Which Shuffle components need redundancy and why

Shuffle's frontend, backend, data store, runtime and shared services each fail in different ways. The cards below cover each component's role, what breaks when it is lost, and the high availability recommendation from the configuration docs, so you can decide which parts need more than one instance and what to test before go-live.

Frontend web interface

The frontend is the web interface where analysts build, edit and run workflows. If it runs on only one server, losing that server removes access to the interface even though workflows may keep executing. The configuration docs recommend scaling the frontend across all servers, with a load balancer or DNS name in front. Check that users reach a surviving node without changing their address, and that any webhook or callback URLs you expose still resolve after a node fails.

Backend API on port 5001

The Go backend serves API calls on port 5001, stores workflow changes and provides the job queue that Orborus polls. If the only backend stops, the UI stops working and no new executions are scheduled. The configuration docs recommend running the backend on every server. Place a health-checked load balancer in front of port 5001, and confirm every backend reads the same shared storage before you add a second node.

OpenSearch cluster

OpenSearch is the document database that holds workflows, execution results, app definitions and saved app credentials, which are encrypted with the SHUFFLE_ENCRYPTION_MODIFIER secret. A single OpenSearch node is a single point of failure for all Shuffle state, and restarting Shuffle containers does not recover lost data. The configuration docs recommend an OpenSearch cluster. Ask for the node count, replica settings, snapshot schedule and a restore test, and confirm the encryption secret is backed up separately from the data.

Orborus nodes and per-run containers

Orborus polls the backend for jobs and starts a worker for each workflow execution, and the worker starts the app containers that run each action. Workers and app containers are created per execution, so redundancy comes from running more than one Orborus node, not from copying those containers. If the only Orborus node fails, queued workflows wait. Confirm each node can reach the backend and its container runtime, and check how node placement affects apps that reach internal systems.

Shared file storage over NFS

When several backends run, the configuration docs recommend NFS for shared files so every backend reads and writes the same content. If the share becomes unavailable, backends may disagree about files or fail to start, and a single NFS server becomes its own single point of failure. Confirm which paths Shuffle writes to, the mount options each backend uses, and whether the NFS service is clustered or backed up. Restart one backend against the shared mount during testing before go-live.

Optional Memcached shared cache

Memcached is an optional shared cache that the configuration docs list as part of a high availability design. It holds cached data for the backends, so the decision to run it depends on your load and on how many backends share it. If you run it, place it so every backend can reach the same instance or cluster. Because it is a cache, confirm with the current docs for your version what happens to requests when it is unavailable before you rely on it.

Which high availability layout fits your risk and operating model

This table compares the four deployment layouts Shuffle documents, plus the OpenSearch and Memcached choices that come with them. Read each row for when the layout fits, who carries patching, OpenSearch and log operations, and what remains exposed. The main takeaway is that a single Compose server suits only non-critical workflows, and any layout that matters needs several Orborus nodes and a supported OpenSearch cluster.

Layout	When it fits	Who operates what	What it leaves uncovered
Single Docker Compose server	Non-critical workflows where a host outage can be tolerated for a while	Your team patches the host, runs OpenSearch on it and keeps logs on that host	Host failure stops frontend, backend, Orborus and OpenSearch together
Distributed Docker Swarm	Several servers and a team comfortable with Swarm, without Kubernetes	Your team patches each node, maintains manager quorum, places OpenSearch nodes and ships logs centrally	Shared file storage, OpenSearch quorum and node placement must be designed and tested
Shuffle Cloud with local Orborus	Apps must reach private SIEM, EDR or directory services without opening inbound ports	Shuffle Cloud hosts the platform, your team runs Orborus and network access to internal tools	Platform availability depends on Shuffle Cloud; run more than one Orborus node for runtime failover
Kubernetes with Helm charts	You already run Kubernetes and have a platform team to operate it	Your team runs the cluster, nodes, storage classes, registries and OpenSearch placement	Control plane and storage design are your responsibility; scaling details sit in the scaling topic
OpenSearch cluster hosted by your team	You want full control of cluster sizing, snapshots and placement for Shuffle data	Your team runs node patching, snapshots, index lifecycle and cluster monitoring	Snapshots only protect data if restores are tested; quorum loss halts stored workflows
Managed OpenSearch you already run	Your organization already operates a managed OpenSearch service with backup support	The provider handles node operations; you confirm version support, credentials and network access	Confirm snapshot, index and version compatibility with Shuffle's documentation before adoption
Memcached shared cache	Several backends where a shared cache is needed across instances	Your team adds one more service to patch, monitor and secure	Optional per Shuffle's configuration docs; a single-server layout gains little from it

How to build Shuffle high availability in the right order

This sequence builds Shuffle from the data layer outward so each component is confirmed before the next depends on it. The result is a Shuffle deployment where the database, backends, frontends and workflow runtimes can each fail without stopping the platform. Each step ends with a check you can record and repeat after changes.

Provision the OpenSearch cluster and confirm health
The cluster stores Shuffle's documents, so it comes first. Your infrastructure team provisions the OpenSearch nodes inside your own network with the security settings your standards require, then checks cluster health from each node. The sequence calls for at least three nodes. Do not continue until health reports the cluster as available and every node has joined. Save the health output so later steps have a baseline to compare against.
Set one encryption modifier on every backend
Every backend must use the same SHUFFLE_ENCRYPTION_MODIFIER value, because Shuffle encrypts saved app credentials and API keys in OpenSearch with that secret. Generate the value once, store it in your secrets manager, and set the identical value on each backend before its first start. Losing or changing it later breaks every saved app authentication, so back it up like any other key. Confirm each backend starts cleanly, then check that an app authentication saved through one backend works through another.
Mount shared NFS storage on every backend
When several backends run, they need access to the same shared files, so mount one NFS export on every backend server. Your storage administrator creates the export, limits access to the backend addresses, and confirms the mount is writable from each host. Use the same mount path in every backend configuration so files written by one backend are readable by the rest. Check by writing a test file on one server, reading it from every other server, then deleting the file.
Run backends and frontends on every server
Place a backend and a frontend on every server in the cluster, then put a load balancer in front of them. Confirm the backend API answers on port 5001 on each host before adding that host to the load balancer pool. Set the load balancer health check to a path your configuration docs define. Check that every host shows as healthy in the pool, then stop one server and confirm the frontend still loads through the load balancer.
Start at least two Orborus nodes
Orborus polls the backend for jobs and starts a worker for each workflow execution, so two or more nodes keep executions running if one fails. Point each node at the backend through the load balancer address, using the runtime location settings in your configuration docs. Confirm that both nodes poll for jobs by reviewing their logs or the runtime location status in the Shuffle interface. Stop one node briefly and confirm the other continues picking up work.
Add Memcached and confirm cache hits
Memcached is optional, but the configuration docs recommend it as a shared cache when you run several backends. Deploy a Memcached service that every backend can reach, and set the same cache address in each backend configuration. Repeat a read-heavy request through different backends, then check the Memcached statistics for rising hit counts. If you do not need a shared cache, skip this step and record that decision in your design.
Restrict traffic between cluster nodes
Limit the ports each component uses to reach the others. Allow OpenSearch node-to-node traffic only between OpenSearch hosts, allow backend traffic only from the load balancer and other backends, and allow Orborus traffic only to the backend endpoint. Keep frontends reachable only through the load balancer. Take the port numbers from the configuration docs and your OpenSearch settings. Confirm the rules with a port scan from a host outside the approved list and a review of the firewall policy.
Run a test workflow through each node
Finish by running one test workflow through each node so you know every path works. Use a harmless workflow that calls a test app action and writes a visible result, and trigger it on each backend and each Orborus node. Confirm each run completes, appears in the execution history, and shows the expected output. Keep the test workflow in version control so you can reuse it in recovery tests and repeat the runs after any upgrade or cluster change.

Why the encryption secret and shared files must match on every node

The encryption secret and the shared file storage must be identical on every Shuffle backend, because each backend has to read the same saved state. Shuffle encrypts saved app credentials and API keys in OpenSearch with AES-256, using the value of SHUFFLE_ENCRYPTION_MODIFIER. Any backend that handles a request for a workflow or app must decrypt those records before it can use them. If one node carries a different value, it cannot read credentials that another node saved, and the failure appears only on the requests that land on that node.

Changing or losing this value is a serious incident, not a routine configuration edit. The Shuffle configuration documentation warns that losing or changing SHUFFLE_ENCRYPTION_MODIFIER breaks all saved app authentications. Every integration then needs its credentials re-entered, and workflows that depend on them stop until that work is done. Store the value in a secrets manager that your platform team controls, restrict who can read it, and keep a documented restore procedure. Test that procedure on a non-production cluster, and confirm with your own change records that the value has not been rotated without a planned re-entry of credentials.

Shared files are the second item that must match. When you run several backends, the Shuffle docs recommend NFS for shared files so that every backend sees the same uploaded content and generated files. Confirm that each backend mounts the same export with the same path and permissions, and that the mount is present before the backend process starts. An empty or stale mount on one node will serve different files than its neighbors. Optional Memcached, used as a shared cache, follows the same rule: every backend should point to the same cache endpoint, or none should use it.

Inconsistent configuration across backends is hard to diagnose because it produces intermittent failures. A workflow may succeed on one request and fail on the next, an app authentication may work for one user and not another, and logs from one backend may not match the others. Teams often treat these symptoms as bugs in a workflow or an app and spend days rewriting code that is not at fault. Before investigating an app, compare the effective environment of each backend container, including the encryption value, OpenSearch endpoint, shared file mount and cache setting, against a reference you keep under version control or in your secrets tooling.

To confirm that all nodes read the same state, run a simple check across the backends. Create a test credential through one backend, then trigger a workflow that uses it through each backend in turn and confirm that every run decrypts it successfully. Check that a file uploaded through one backend is visible from the others. Record the results and repeat the check after every upgrade, configuration change or node replacement. Hardening decisions for the secret, such as access controls and rotation, are covered in the security hardening section.

How the HA design differs between Compose, Swarm and hybrid layouts

On a single server deployed with Docker Compose, the frontend, backend, OpenSearch and Orborus all share one host. Redundancy therefore has to be added around the stack, because no component has a second instance on the same machine that can take over. The gaps are predictable, and they are the first things to close.

Backend and frontend: each runs as one instance. The backend API on port 5001 has no peer to fail over to if the host or container fails.
OpenSearch: one node holds all workflow and credential documents. Disk loss or corruption takes that data with it unless snapshots are stored off the host.
Orborus: one runtime polls for jobs and starts workers. If it stops, new executions wait until it returns.
What to add: a second server with its own Orborus, an OpenSearch cluster spanning both nodes, shared file storage for the backend if several instances run, and off-host backups of the encryption secret and OpenSearch data.

After adding a second node, list every component again and mark which ones still have one instance. Confirm the options available in your version with the Shuffle configuration documentation before you design the second node.

What drives high availability effort and cost, and how to keep it down

This table lists the factors that change the build time, test effort, infrastructure and ongoing operations for a highly available Shuffle deployment, with the cost component each one moves and how to reduce it. Read it before requesting a quote, because most of the variation in scope comes from the first few rows. Use it to decide which drivers your organization can simplify early.

Driver	Cost components affected	What increases it	How to keep it down
Number of servers and regions	Infrastructure, build time, testing	Each additional server runs its own frontend and backend, and a second region adds replication, latency and failover paths to test	Start with the number of servers needed for your load, and confirm with your team whether a second region is a hard requirement or a later phase
OpenSearch data volume and retention	Infrastructure, ongoing operations	Long retention of workflow executions and results increases storage, backup size, restore time and index management work	Agree a retention period with your SOC and compliance owners, then set index lifecycle rules before go-live
Number of workflows and concurrent executions	Infrastructure, testing, ongoing operations	More workflows and higher parallel execution need more Orborus capacity, more worker and app container placement decisions, and more load tests	Prioritize the workflows that must survive a node failure and size concurrency from measured peaks rather than estimates
Number of integrations whose credentials must be replicated	Build time, testing, ongoing operations	Every saved app authentication depends on the encryption secret and the shared OpenSearch data, so each integration adds recovery and validation steps	Reduce the number of integrations in the first phase and document which credentials must survive a restore
Number of environments (test, production, air-gapped)	Infrastructure, build time, testing	Each environment needs its own cluster, images, secrets and recovery test, and air-gapped environments need offline image and app delivery	Keep test topology close to production, and confirm whether air-gapped sites need a separate delivery process before scoping
Compliance evidence needed	Testing, ongoing operations	Assessors may ask for failover test records, backup restore logs, configuration baselines and change history for each component	Define the evidence list in the written scope, and have your team store test results and configuration records in one place from the start
In-house skills in OpenSearch and Docker	Build time, ongoing operations	Limited experience with OpenSearch cluster health, shard handling and Docker networking extends build time and raises the chance of recovery mistakes	Assess skills during scoping, plan knowledge transfer and runbooks into the scope, and ask for documented handover sessions
Fixed-scope quote from a written scope	Build time, testing, ongoing operations	Vague requirements, undefined recovery targets and unnamed environments lead to change requests and extra testing	Request a fixed-scope quote from a written scope that lists servers, regions, environments, integrations, retention and evidence items

How to test Shuffle failover before you rely on it

A Shuffle high availability design is only proven when a failure is injected and the outcome is measured. Before each test, write down the expected result: which user actions should still work, what cluster state should appear, what should happen to a running workflow, and how a restored environment should behave. Recording expectations first keeps the team from explaining away a surprising result afterward. Run the suite in a staging copy that mirrors production, with the same number of backends, OpenSearch nodes and Orborus nodes, and the same shared file storage and Memcached arrangement. Staging results only carry over to production if the topology matches, so confirm that before anyone stops a node.

Start with the web and API tier. Stop one backend instance, which listens on port 5001, and confirm that the frontend still loads, users can sign in, and existing workflows can be opened and edited. Check the load balancer or proxy in front of the backends, because a stopped backend that remains in the pool will produce intermittent errors that look like a design failure. Then stop one OpenSearch node. Confirm that cluster health matches the state you expected for your replica settings, that workflow and execution records remain readable, and that new writes such as saving a workflow or recording an execution succeed. If health turns red or writes fail with one node down, the cluster is not configured for the failure you intended to survive.

The runtime tier needs its own test. Start a workflow containing an action that runs long enough to interrupt, then stop one Orborus node while the execution is in progress. Orborus polls the backend for jobs and starts a worker for each execution, so the outcome depends on where that worker was running and how your configuration handles a lost worker. Write down whether the execution should complete on another node, be retried, or be marked failed for an operator to restart, and whether any action could run twice. Then confirm that app containers can start on the surviving nodes. Every Orborus host must reach the image registry and pull the app images it will run, so trigger an app on each remaining node and record which node handled each execution.

Two tests protect the stored secrets and the data. First, bring up a node with an incorrect SHUFFLE_ENCRYPTION_MODIFIER value, then one with the variable missing. Shuffle encrypts saved app credentials and API keys in OpenSearch with AES-256 using that secret, and the configuration docs warn that losing or changing it breaks all saved app authentications. The expected result is that the node reports it cannot decrypt stored credentials, and that integrations do not appear to work. Confirm that your monitoring catches this case, because a node that starts but cannot decrypt is the failure most likely to go unnoticed. Finally, restore OpenSearch data and shared files into a clean environment, supply the secret separately, and check that workflows, apps and saved credentials all work end to end.

Define pass criteria for your own environment before the test window. The recovery time and data loss tolerance should come from your business owners and recovery objectives, and this page does not set those targets. Record actual results beside the expected ones, log every gap as an issue, and repeat the full suite after any upgrade, topology change or secret rotation, since each can change the behavior you measured. A passing run shows that the design holds for the failures you tested. It does not prove that every failure mode is covered, so treat the suite as a baseline you extend over time.

Access and secrets controls that must survive a failover

A failover moves work to a node that may not share the same configuration, so access and secrets must match and stay tightly scoped on every server. The cards below cover the controls most likely to break or widen exposure when a backend, frontend or Orborus node takes over. The full control set is covered in the security hardening topic.

Encryption secret on every node

Shuffle encrypts saved app credentials and API keys in OpenSearch with AES-256, keyed by the SHUFFLE_ENCRYPTION_MODIFIER secret. Every backend that reads those records must hold the same value, or a surviving node cannot decrypt them and saved app authentications fail. Confirm the value is injected identically on each server from one secret store, and that it is included in backup and restore procedures, since changing or losing it breaks every saved app authentication.

Least-privilege keys per integration

Each app authentication should use its own API key or service account, scoped only to the actions its workflows need. When a node fails over, it inherits whatever rights the stored credentials carry, so a shared administrator key turns a runtime failure into wider exposure. Review each integration's permissions, confirm read and write access matches the design, and test that a workflow still runs on a second Orborus node using the same key.

TLS termination at the load balancer

Frontend and webhook traffic should terminate TLS at the load balancer in front of all frontend and backend servers, so certificates are managed in one place and every node is reached over the same path. Confirm the load balancer forwards correctly to the backend API on port 5001, that webhook URLs given to external tools use the public HTTPS address, and that certificate renewal does not depend on restarting a single node.

OpenSearch limited to Shuffle nodes

The OpenSearch cluster holds workflows, execution history and encrypted credentials, so it should accept connections only from Shuffle backend and worker hosts. Enable OpenSearch's own security features, use separate users for the backend and for administrators, and restrict network access so cluster nodes reach only their peers and the Shuffle hosts. Verify that a backend moved to another server still authenticates without widening the allowed sources.

Docker socket on runtime hosts

Orborus starts a worker for each workflow execution, and the worker starts app containers, which requires access to the Docker engine. Mounting the Docker socket gives a process control equivalent to root on the host, so limit which hosts mount it, keep Orborus on dedicated runtime hosts, and confirm that a replacement Orborus node receives the same restricted setup. Check container placement and socket access during each failover test.

SSO configuration identical across backends

Single sign-on for Shuffle, whether OpenID Connect or SAML, must use identical provider settings, redirect URIs and per-tenant enforcement on every backend. Otherwise users may be routed differently depending on which node answers. After any change, compare the configuration on each backend and test login through the load balancer while one backend is stopped.

What to monitor, back up, upgrade and document after go-live

Redundancy only protects you while someone can see when a component fails. After go-live, monitor the OpenSearch cluster health status, the number of nodes that have joined, and shard allocation, since an unassigned shard means a replica is no longer protecting your data. Check backend and frontend availability through the load balancer's health checks, so a failed server is removed from rotation automatically and your team learns about it. For Orborus, track node heartbeats and the count of queued jobs. A node that stops heartbeating while jobs accumulate means workflows are stalling without any visible error in the frontend. Confirm which metrics and health endpoints each component exposes in your chosen deployment architecture before you write alert rules.

NFS and Memcached are the components teams most often forget to watch. NFS holds the shared files that several backends read, so a stale or unreachable mount can cause backend errors that look like application bugs. Alert on mount availability from every backend host. Memcached is a shared cache, so its loss should slow the platform rather than stop it, but test that assumption in a non-production environment and measure the effect before relying on it. Certificate expiry belongs on the same dashboard. Track expiry dates for the frontend and load balancer certificates and any TLS endpoints that Orborus nodes use, with alerts well ahead of renewal deadlines. Record who holds the renewal process and where the private keys are stored, because an expired certificate on a single endpoint can take down every workflow that depends on it.

OpenSearch snapshots and the encryption secret must be backed up separately, and both are needed for a working restore. Snapshots capture the indexed data in OpenSearch. The SHUFFLE_ENCRYPTION_MODIFIER secret is what decrypts the app credentials and API keys stored in that data. Shuffle's configuration documentation warns that losing or changing this secret breaks all saved app authentications, so a snapshot without the matching secret gives you data that cannot run integrations. Keep the secret in a controlled store with restricted access and logging, apart from the snapshot repository. Schedule restore tests into an isolated environment, then check that stored credentials decrypt, that a representative workflow executes, and that the test result is written down. Confirm snapshot frequency, retention and repository location with your own team, because those depend on your recovery objectives.

Upgrade nodes in a rolling order: remove one node from the load balancer or Orborus pool, upgrade it, verify its health and jobs, return it to service, then move to the next. Keep the frontend, backend and worker images on the same release tag throughout, since mismatched images are a common cause of failed workflows after an upgrade. The Shuffle documentation recommends the latest Docker tag for stable releases and the nightly tag for experimental changes, so production environments should pin a specific tested release instead of following nightly builds. Shuffle also stopped supporting versions older than v2.1.1 after January 31, 2026, so check your current version against that date when planning. The upgrade sequence, app updates and rollback steps are covered in the upgrades topic, so align your maintenance calendar with that plan.

Auditors and new staff will ask for the same four artifacts, so keep them current and version them. The topology document should list which servers run which containers, the OpenSearch nodes, Orborus nodes, NFS and Memcached endpoints, and the load balancer configuration. The secret custody process should name who can access the encryption secret, how access is granted and logged, and the documented steps for any change or re-encryption, which must be planned because a changed secret breaks saved authentications. The failover runbook should give step-by-step actions for losing a backend, a frontend, an Orborus node, an OpenSearch node, or the NFS share, including who decides and how service is verified. The test results should record each restore and failover test with its date, scope, outcome and remediation items. DeployOpen can prepare these documents and the evidence behind them, but independent auditors and certification bodies issue any assessment outcome.

Shuffle high availability questions teams ask before hiring help

How much extra effort does high availability add over a single Shuffle server?

Highly available Shuffle adds several moving parts that a single Docker Compose host does not have: an OpenSearch cluster instead of one node, backends and frontends on more than one server, redundant Orborus runtime nodes, shared file storage for multiple backends, and an optional Memcached cache. The effort is mostly in design and verification rather than in installing containers. The main drivers are the number of servers and availability zones, how your network and load balancing are set up, how much data OpenSearch holds, and how many workflows and runtime locations must survive a node loss. Each added component also needs monitoring, backups and an upgrade path that keeps versions aligned.

How long does a Shuffle high availability project take to complete?

The duration depends on the drivers, not on a fixed number. Shorter projects usually have an existing OpenSearch skill set, standard infrastructure, a small number of runtime locations and clear change windows. Longer projects involve approvals for new servers, network changes, storage decisions for shared files, integration with your identity and secrets tooling, and failover testing that must be repeated after fixes. Ask any partner for a written plan that lists the phases, the environments covered and the test scenarios, and confirm which of your teams must be available at each step. DeployOpen agrees scope, deliverables and timelines in writing before work starts.

What does our team need to provide for a Shuffle high availability build?

Plan to provide the infrastructure and the people. That means at least the servers or virtual machines for the cluster, a network design that allows the nodes to reach each other and your integration targets, storage for shared files, and a load balancer or DNS approach for the frontend. You also need someone who can operate OpenSearch, including snapshots and cluster health, and someone who can run Docker in production. Decide who holds the SHUFFLE_ENCRYPTION_MODIFIER secret and where it is stored, because losing or changing it breaks saved app authentications. Finally, agree on change windows for upgrades and failover tests.

Can our in-house team build and run Shuffle high availability without a partner?

Yes, if the team can operate OpenSearch and Docker and is willing to run failover tests on purpose. The architecture is documented by Shuffle, so capable engineers can build it from the official configuration guidance. The harder part is proving it works: stopping a backend, an Orborus node or an OpenSearch member during a workflow run, confirming that jobs resume, and recording the results. Teams without that habit often discover gaps during a real outage. A partner can help with design review, test planning and runbooks, while your team keeps the knowledge in-house.

Does the open-source edition of Shuffle support these high availability recommendations?

Shuffle states that all features are available in the open-source edition, and its paid tiers differ in feature limits, scaling and support. Since version 2.1.1, the limits between the open-source and enterprise editions are actively enforced, so you should not assume that a scaled design is unrestricted. Confirm the current scaling limits against the Shuffle configuration documentation and the release notes for the version you plan to run before you finalize the architecture. Also confirm which Docker tag you will use, since the docs recommend the latest tag for stable releases and the nightly tag for experimental changes.

What does ongoing support cover after a Shuffle high availability build?

Ongoing support can cover keeping frontend, backend, Orborus and app images on supported versions, aligning them after upgrades, reviewing OpenSearch health and capacity, updating apps when vendor APIs change, and repeating failover tests after changes. DeployOpen offers ongoing support, upgrades and continuous testing as retainers, with the scope written in advance. Ask for the specific response expectations, the systems in scope and whether backups and restore tests are included. Clarify who handles host-level patching and certificate renewal, since those often sit with your infrastructure team.

What is the main cost of running Shuffle high availability, and what recurs?

Costs fall into one-time and recurring groups. One-time costs include design, implementation, migration of configuration, and the first round of failover testing. Recurring costs include the servers and storage for OpenSearch, workers and backups, staff time for monitoring and upgrades, and any support retainer. Shuffle's licensing is based on app runs in paid editions, and the open-source edition has no such metric, so confirm which edition and runtime model applies to your plan. When requesting a quote, ask for each component, who pays for it, what is included in testing, and what happens after the first upgrade cycle.

How DeployOpen scopes, builds and hands over Shuffle high availability

DeployOpen plans and builds Shuffle high availability inside your own infrastructure, so the cluster, shared storage and runtime nodes stay under your control. Each engagement starts with a written, fixed scope, followed by testing, documentation and handover to your team.

How we scope

Map every Shuffle component to a server or tier: frontend, backend, OpenSearch, Orborus, Memcached and shared storage.
Confirm which deployment architecture fits your estate: Docker Swarm, Kubernetes with Helm, or a hybrid runtime.
Agree the failure scenarios the design must survive, such as losing one server, one Orborus node or one storage path.
Record the written scope, deliverables and timeline before work begins, since engagements are fixed-scope.

How we build

Deploy an OpenSearch cluster and scale the backend and frontend across all servers as the Shuffle docs recommend.
Configure shared file storage and an optional Memcached cache for the backends, then verify both are reachable from every node.
Run more than one Orborus node so workflow execution continues if one runtime fails, and check that jobs move between them.
Store the SHUFFLE_ENCRYPTION_MODIFIER secret in your secrets manager and confirm it is backed up before first use.

How we hand over

Test failover for each component and record the observed behavior, including recovery steps for each failure case.
Document the architecture, configuration values, backup and restore procedures, and the upgrade path for your team.
Walk your operators through routine checks, alert points and the recovery runbook, then confirm they can run them unaided.
Deliver the encryption secret handling notes, since losing or changing that secret breaks all saved app authentications.

How we support

Offer ongoing support, upgrades and recovery testing as a retainer, separate from the initial fixed-scope build.
Retest the cluster after Shuffle version changes, keeping frontend, backend and worker images aligned.
Review OpenSearch capacity, storage and Orborus concurrency as workflow volume grows, and report findings in writing.
Confirm with your team which support scope, response expectations and testing cadence the retainer will cover.

Shuffle high availability